What Do You Think The Problem Is?

This is an Improved Meyer's Lemon with Dutch Microclover planted at the bottom as a living mulch. It's the tree's first year in zone 7b. Removed all the baby lemons a month ago, but left one. Its not really cold enough for yellowing yet. I've heard you don't fertilize in the fall. So what do you think is causing the yellowing? Should I remove the lemon?

FYI, I know it's supposed to not survive in our climate, but there are people doing it, and we have other citrus surviving, but just not producing. We decided to give Meyer's Lemon a try as it was the most cold hardy. This is an experiment, but we don't want it to die before we have a chance to test it against the cold. Our calamondin doesn't lose its leaves until late January or early February, but comes back in late March or early April.

I think it needs fertilizer, or over watered, but I dunno. It is in a pot, but it is a huge pot with holes in the bottom for drainage. The pot is somewhat isolated on the outside, we will cover with white xmas lights and white shade cloth to protect from frost. This is how it survives the winter and have done it with this method for years.