Well we have gone in the fall and I would say if you want to go and get the most bang for your buck go the 2nd week of December. The overlay of the haunted mansion is still in effect, the Its a Small World Holiday is in effect and you can see the X-mas decs and the "snow" on main street.
If you go on a Friday you can also catch fantasmic and if you have never been I recommend doing the balcony or preferred seating. It is a little pricey but WORTH it!!
The crowds aren't too bad then either. If you want a REALLY slow time with not as much stuff going on you can go in Jan or Feb. Just make sure it is not around any holiday weekends. There is presidents day in Feb and Martin Luther King in Jan.
Otherwise I heard mid week in feb (non holiday week) is the slowest.
Good luck and feel free to ask any more questions.