I think it depends on the child and their interest, but for my particular two girls, 5-6 was the
perfect age. On our first trip, my older daughter was just about to start kindergarten (5.75) and deep in her princess/character obsessive phase, and the trip was absolutely magical for her (and for us seeing everything through her eyes!). She still counts meeting Elsa for the first time and having dinner at Cinderella's castle dressed up in her own princess dress as some of her best memories. On that trip my younger daughter was just under 3 and it was overwhelming for her - she was afraid of lots of the characters and nervous to meet others, plus she tired easily and still needed a good nap each day.
On our most recent trip, my younger daughter was 5.5 and older daughter 8.5. This was the trip of magic for my younger - starry eyed for all of the characters, SO excited to go on all of the rides (I think we rode It's a Small World five times...), lots more energy for longer days in the park. My older daughter still had fun and she was a bit braver about rides (she even went on FoP!), but she is just on the cusp of being "too cool" for some of the characters and littler kid rides. And at this age she obviously was totally aware that all of the characters are people dressed up. But she did still light up when she saw Tinker Bell "fly" during HEA, so the magic wasn't completely lost on her yet.
It did make me really happy that we squeezed in a trip when she was younger and the magic was SO alive for her.
For us, Disney is about the magic and the total experience. There are plenty of rides we didn't have a chance to ride and that didn't diminish the experience for us. When it comes to rides, we feel we can easily spend a day at a local amusement park as our kids get older. If rides are the most important thing to you though, you would definitely want to wait until the kids are above all height restrictions so you can take full advantage while there.
I will say, if we'd had to choose only ONE time total to go with both kids, taking both ages into account, I would prefer when the younger was 5 and older was 8 because it was the best of both worlds (still plenty of magic, more energy, more bravery for rides, etc), plus the girls were good ages in their interactions together as well (sat together on some rides with parents behind them, swam together at the pool, etc).