Speaking as a linguist, you're completely correct.
This ends up being kind of a neat semantic debate that linguists go through a lot. Many Austronesean languages (such as
Ambae) have spatial referential systems that are adapted to their historical location as an island culture - specifically, they don't talk about things as being 'North" or 'west' but rather as being "inland' and 'towards the sea.' Because some of these islands are volcanic, the further you go inland, the further you go up, so inland means up as well. Likewise, toward the sea means down. What happens when you put these people in a giant crater, and how do they talk about where they are or where they need to go?
This gets really interesting when you talk about less round area. Theres a language I can't remember off the top of my head that is spoken entirely by people living on a mountainside that faces south. When these people talk about going south, what they actually say is "downhill." Because the rain in the area comes principally from the south, it is often described as falling "uphill," as the rain itself is moving north. What happens when you put these people in the middle of the great plains, where everything is flat?
Sorry about the off-topic.