We went the first week of June last year and we're going back again. The crowds aren't as bad as the later weeks of June. Weather is still hot, but it isn't any worse than Memphis in July so we were used to it.
Just avoid the Magic Kingdom on Saturday if you can... the crowds there will be huge. Other than that, the first week of June is alright. I think people aren't ready to jump right to Florida when school lets out.