Honestly all - please don't stress about the ATMs! It is absolutely true that you cannot withdraw money at just any ATM, but the convenience stores are absolutely everywhere. (And they are a legitimate place to get some food if you want something fast! They'll heat it up for you and you can eat it there.) I live 2 miles from work and pass at least 8! There are ATMs at the airport that you can use if you worried about needing yen right away. Some non-convenience store ATMs are usable, usually in popular tourist spots - they advertise "International ATM."
Many places do take card now but a lot of smaller eateries still only take yen, as well as some shops. This is definitely not a place you need to be nervous about carrying cash on you.
Something you may want to consider when deciding how much to pull out - most ATM will charge fee of 108yen for 10,000, 216yen for 20k, and maxes out at 216 if you go higher. So if you know you'll need it, you save a bit by minimizing withdrawals. Oh and some will give you a "partial change" option!! This will give you 10,000 of your withdrawal in 1000 bills, instead of getting all 10,000 bills! (This is now something my husband and I yell at each other when going to ATM - "don't forget the partial change!")