We took our son at 2.5 and he loved it. The last trip he was 3.5 and he enjoyed it even more.
Although we stayed at the French Quarter, after staying at Riverside, I would recommend that you stay in the AB if you can get in either bldg. 14, 15, 16 or 18. Bldg. 18 with a waterview is right close to the path. All four are close to the bus stop (the first one), the food court, a quiet pool, and it isn't too far from the main pool.
The bus stop for you will be a big one. There are 4 at Riverside. The bldgs. I mentioned are by the first one. With us, in the morning it was always nice to get on before it got too crowded, with the stroller and our stuff. In the evenings, it was always nice to get off first because mom was always cranky and ready to be in the room. The short walk was always welcome especially if I could go through the food court and grab a drink on the way to the room (a little longer way but well worth it for me).
The room is the same size as the mansions but it is more rustic in decor and the vegetation is thicker and not quite as "manicured" as it is in the mansion area (just like the south). Be sure to request a first floor room in the AB as there are no elevators and with a stroller that will make a difference. You might also want to request a room "away" from the parking lot. A couple of them in bldg. 15 are pretty close to the parking lot and with a toddler that concerned me. We were in one on the other side and it worked out great.
Any questions, ask. Glad to answer them. You will have a blast with your little one. I can't wait to go back in Nov. as ours will be 4.5 and I'm sure will enjoy it even more.