If you go with the question in your mind whether it will be too stuffy for you, it will. I never thought the GF was too stuffy until I got it in my head that maybe it was and I just didn't realize it. When I next visited it there for lunch, I started thinking that maybe I didn't fit in. NONSENSE! It is a beautiful place for everyone who has a taste for the victorian.
GF is definitely not too stuffy. Many people save and save all year in order to stay there, with it's gorgeous victorian theming, proximity to MK, view of the castle, good restaurants, etc. I love this resort, and have stayed at many. I have the most fond memories of staying there. I have had at least six trips since my last stay at GF, and look forward to having the opportunity to return there.
I had a toddler when I stayed there last. It was perfect for her. Not overstimulating, but lots of room to walk around, watch the monorail, sit in the lobby and listen to music, take the boat to MK.
This is a great place for little ones. I would return there in a heartbeat, as soon as I get a really, really good rate. You will not be disappointed. Especially that you have young ones, for whom the days can be very long and tiring. The GF offers a nice, quiet retreat for families who like to just take it easy and return to the resort for a rest.