I think GF is a better resort for two adults with a very young baby/toddler, in my opinion. Both pools have zero entry areas and a great splash pad that's pretty shallow, and it's usually not hard to find chairs/shade. GF is an 'easy' resort with a little one.
Both of Poly's pools have zero entry areas, but they are generally more crowded/hectic and seating can be a challenge at times at the main pool (Lava Pool). The splash area is better suited for kids that are a bit older. Poly is just a more active/busy resort overall that we've enjoyed as our kids were more like 3/4+, less so when they were really young.
Both are great resorts though!
BTW,
to the DIS!