The fact is, only you can answer that question, and likely only after you've done it once.
I don't think the crowds (this was early August, FYI) were much less than those in late June....but the heat was MUCH worse, IMHO, and everyone was moving MUCH slower, which made it feel even MORE crowded to me. We still had a great time, FYI, it was just a LOT more "work" than it is during the cooler and less crowded times (late September/early October, January, etc).
But I know others who SWEAR by going in late July and early August. The heat doesn't get to them and they do pretty much what you're describing as wanting to do.