Reading your post, I felt like you were describing what I've been feeling only you've stated it far more succinctly than I could have ever done.
I'd just add to your list that, personally, when I'm speaking with someone and they're giving me directions, I end up feeling a blitz of confusion in my mind coupled with embarrassment because I'm feeling like they must be able to see it on my face. I'm rarely able to retain any of what people tell me. When I'm reading instructions, I swear to god that the words all rearrange themselves into mumbo jumbo right before my eyes. Many times I'll have to read one line, several times before I get the flow of the sentence. Then, I have to re-read it to get the meaning of the sentence. It's horribly frustrating.
I also suffer from fatigue and the associated anxiety of being fatigued - like when you need to be somewhere at a certain time and are afraid that you'll need a nap right at that moment
OR being so stressed about being fatigued the next day, that you can't get a good sleep the night prior. A double edged sword, so to speak.
I've had a poor memory all my life. My Mother used to tell me it was selective but I know for a fact it's not and it's growing worse by the day, to the point which I, like you, wonder if my brain is deteriorating. It sure does feel like it. Additionally, in the last 15 years, my ability to recall faces has diminished incredibly. I'd say that overall, my observation skills have gone seriously downhill. I can't tell you how embarrassing it is when someone approaches you in a very friendly manner and starts talking about something as if you should A) know them, and B) know what they're talking about, but you don't have a clue who they are or what they're referring to. You just know you're wrong and try to fake your way out of it, best you can without insulting them.
I began telling people (therapists included) that I feel as though I will have Alzheimer's at an early age (I'm only 54 now). No one listened. They all think I'm joking. I do joke though on occasion and say the I have "Sometimers" but really, it's no joke. Its in fact, quite startling and scary.
Anyway, these, combined with a few other ailments, have now rendered me unemployed. I now fear interviews because I can't retain or comprehend "in the moment" discussions and more than likely, I won't remember their face if I were to see them for a 2nd interview.
Oh, and what you'd mentioned about losing track of time? Happens to me all the time, especially when I'm driving. I can be so deep in thought that I'll go anywhere up to a few city blocks and have no recollection of how I got there. I mean, obviously I got there in the car but I have no visual memory of the path there.
This Nuvigil sounds interesting.
Best of luck!