Dating someone with social anxiety

I’ve been seeing this person for almost a year now and throughout the year he’s changed .. in both a good and a bad way. He used to be so warm and loving and now he feels distant and cold. He wouldn’t let me get any close to him in public, claiming that I make his social anxiety get worse. At first, I was confused because of his sudden change in behavior. So, naturally, I was very much upset. But after a while he told me the truth and I’ve been trying to be understanding and help him out with this issue. But .. the bad side is that he’s using words he said he’d never use on me ever. He would say things like who am I’m to pity you when u cry? I cry too, saying he doesn’t care which hurt me tremendously. After which he does apologize but still .. the damage has been made. A few months ago we had a fight concerning about me having a wet dream about my rebound and it seem to me that he’s changed ever since. It makes me blame myself.. I love him and I want to help. I know he’s going through a rough patch right now and I just want to say it’s alright if he wants to focus on himself and that he doesn’t have to respond to any of the occasional tantrums I might throw because of his .. condition. Would that be fair of me? Or would it be just dumb? I’m lost on who’s right and who’s wrong at this point. Any advice would be very much welcomed . Thanks!!