I am a teacher, so we can only go on school vacations. It really isn't a big deal if you are willing to do a little research and make smart touring decisions. We've been Christmas/ New Year's, Easter week, President's week, summer and we never wait in long lines.
I would just pick whenever fits your schedule the best and then plan accordingly. Each time of year has its pros and cons. One nice thing about busy times is you tend to get better park hours, less refurbishments, and more efficient transportation.
To me, the only downsides to summer are the heat (which center really bother some people), no festival at Epcot, and you have to wait until later for the fireworks due to darkness being later.