First and foremost, always bring your passport and identification documents, as well as enough cash, with you.

Keep yourself updated on the latest situation via news or radio channels.
You can also download warning apps in your phone to alert you in the event of a natural disaster. I recommend Yahoo！防災速報 (available for iPhones and Android) which is free to install. The app also provides information on nearby evacuation zones. Unfortunately, its only available in Japanese.

Last but most importantly, stock up on emergency rations like water and food. Its best to have an emergency pack ready with rations, first-aid supplies, warm clothing and a torchlight.