I'll give you my $0.02...I'd stay on the ship, or at most take them to the "Welcome to Nassau" sign for a photo op just so you can say you were there and then head back to the ship. It has nothing to do with safety (because I think most people's fear of Nassau is overblown, you'd be safe on any excursion) and everything to do with the kids' enjoyment. My experience has been that kids at that age have no interest in Nassau, and they're probably too young to do any of the dolphin/stingray activities that are offered. They will have a much better time enjoying activities on the ship, especially since your time will be limited on a 3 night cruise. They'll get plenty of beach time on Castaway Cay.
Whatever you decide to do...enjoy your cruise!!!