I am also like this. I had a fight once with my SO where they were upset with me for not trying to level up my navigation skills. I have a growth mindset about many other things, but I've just absolutely given up on navigation without a smartphone.
I also have a hard time recognizing faces of celebrities, but not people I know in real life.
From reading about prosopagnosia and other agnosias, I suspect there's something funny with my fusiform gyrus.
I am an excellent navigator but can't recognise people I know on the street, even if I know them reasonably well. If I know I will be meeting them, and I know them well I mostly recognise them but even then I am sometimes surprised when they show up and say hello to me.
Celebrities is even worse...
(Good) navigation is not really about recognising landmarks (although that helps) but about having a mental version of the map of the place you are in, with you in the centre. Think of it like the GTA mini-map that live updates as you walk around.
I think that comes naturally to some people and just does not exist for many others.
I have a hard time visualizing things when I'm fully awake. Ask me to picture my wife or my kids' faces, and I can't do it. I feel I can subconsiously visualize them with no problem, but as soon as a try to consciously do it, the picture disolves. Same with any fully awake visualization - can't quite see the image in my head - it's like it's in my peripheral vision but scoots away as soon as I look. But I can lucid dream, and do things like complex 3D mechanism design while lucid dreaming (that I can later turn concrete using CAD).
Anyway, I have no problem with navigation - I can look at a map, and navigate from the (non visualizable!) memory for ages (an hour if hiking, maybe several hours if driving), and pretty much always tell you where north is. But I still cannot bring the image of the map back into my conscious mind. The brain is a very weird thing indeed.
I've always linked my inability to navigate to my inability to visualize things in my head. Even in the area where I've been living for about 15 years now, I still struggle to determine routes if I don't travel them very regularly.
This afternoon, we drove to a place in the city nearby. By now, after so many years, I can guess which exit to take, but I don't know whether to turn left or right at the end of the exit until I'm at the end and recognise it from previous times I've been there. But I can't picture it in my head beforehand.
I also have aphantasia but can navigate just fine. I read a comment on here once from someone with aphantasia that couldn't navigate in the real world, but they worked in network admin and knew that layout just fine (knew where everything routed and what it connected to etc). I pondered that if they could remember cable routes, why don't they apply the same mental map to roads? Roads are fixed and have a beginning and end too! But they reckon they could not. So it's a strange problem.
I think the challenge is relating your current locatiom in a 3d environment to a top down 2d map?
The 3d environment is full of noise and info which is a lot to take in while something like routing is very simple, and you don't have to convert your own placement in 3d map to 2d.
Also routing likely has good intentional reasons why something follows the other. While roads have evolved more naturally throughout history.
I am an excellent navigator but can't recognise people I know on the street, even if I know them reasonably well. If I know I will be meeting them, and I know them well I mostly recognise them but even then I am sometimes surprised when they show up and say hello to me.
I also have a hard time recognizing faces of celebrities, but not people I know in real life.
From reading about prosopagnosia and other agnosias, I suspect there's something funny with my fusiform gyrus.