Well, people with Aphantasia [1] don't have to have poor navigation sense.

You could try to study a map of an area you want to be in and think about where you were on it one day and where you wanted to go to.

[1]: https://en.wikipedia.org/wiki/Aphantasia