Mattie, you have antibodies, so there is definitely something going on with your thyroid. Do you have the lab ranges for the FT4 test? If you're in the lower part of the range then you'd be feeling bad. Your TSH is rising, as well, so that a sign your thyroid has to work harder.
BTW, having your TSH tested in the afternoon will give you a lower results - this particular result can vary by as much as 3 points during the day, being highest in the morning.