Crowd-wise, early May noticeably lighter and more casual. Just about every school kid in North America is still in school in early May. This often means shorter park hours, less staffing, more closures, fewer CS dining options, etc.
Crowds in late August are significantly lighter relative to June through mid-August. Not just because maybe 75% of kids are back in school, or because of the 100+ degree weather with 100% humidity, but mid-August marks the start of hurricane season. Hurricanes can mean a week of heavy rain (and, of course, the potential for far worse). This is traditionally why WDW attendance was so low in autumn and why Disney used FD and F&W to entice Fall visitors. But, Florida has been spared a bad hurricane for years, and people have short memories, so it's less of a deterrent right now.
We go the week before Labor Day because we are still out of school (you'll find the vast majority of visitors that week are from Va., Md., NJ or the UK), we are used to the heat (my native D.C. see the same heat and humidity), and the crowds are very manageable. Longer hours are more common in late August (compared to early May), but they are not long days like June thru mid-August.
Note: This year FD starts before Labor Day, which does increase crowds some, but has a very noticeable impact on ADR's, wait times, food service, food quality, etc.
