Make sure you exchange at the airport in Japan. The rates in US airports tend to be awful, but Japanese airports tend to have fair and decent rates. Overall, exchanging money in Japan is easier than in Europe imho.
In the past, I've rented from Softbank:
https://www.softbank-rental.jp/en/ They're one of the big carriers in Japan and have lots of rental locations. I used to rent at the airport and return at the airport because it was easiest. Since Softbank is so large I could rent from Narita and return at KIX even. There are lots of other companies that do smartphone rental as well. I think Japan Experience (I got my rail pass from them last trip) and JTB might offer rentals as well. I like Softbank because I like picking it up and dropping it off at the airport. It's one less thing I need to remember to pack. I use to rent a physical phone because my US phone didn't work. I guess if I were to do it now, I would think about just renting the SIM card, I don't know. I've thought about renting personal WiFi but I'm not sure how well that will work in rural locations.
Depending on your phone brand and carrier in the US, your phone might work in Japan with just allowing roaming. I am on Project Fi, and my phone works in Japan as is. I love it, it means cheaper phone calls home. Apple products also tend to work in Japan. Everything else is sort of mixed bag.
Hope this helps!