Your adventure sounds like so much fun! The S.D. downtown station is safe, I've ridden through there on the trolley every year for Comic Con. I'm jealous of the fun you will have. I love taking the train for something like that. And that's more than enough time to have the Disney experience you're looking for. Have a great trip!