Though most foreigners think that Japan only has pagodas and deep-red architecture, the building landscape is different than what you see in anime (manga is a little more realistic but only by a slight margin). Actually,pagodas are influences from China.
Many buildings are really old and damaged-looking. They are built to last for a long time. Some apartments are built with the ground floor as the parking lot while the rest of the building is supported only by pillars or columns. I suspect that these kind of buildings are remnants of Japan’s earlier days when the nomads began to settle and more permanent settlements began. The buildings were similar, except the purpose: to protect food stores. But it’s funny to see these things still around after several hundreds of years.
I live in Okinawa, so you are more likely to find some non-Japanese architecture, like the old house in the picture.