Yes, you can - you can't swim in the pool at another resort, but you can definately visit any resort you like. I highly recommend taking Disney transportation if you're planning a night of 'pool bar hopping'. Take a bus to the TTC then take the monorail (or even walk) over to the Poly.
Thanks,
Pete