Boyfriends depressive mood spreads to me

My boyfriend gets depressed when he’s stressed about work or family and he becomes very quiet and distant. His feelings spread to me and then I will also feel stressed and anxious too. I always feel like he loses feelings when he’s like this and there’s no affection, if I try to show him affection he doesn’t return it and any communication with him just feels empty.

I told him how I feel about this and I don’t want to get upset or angry with him because these are issues he needs to figure out in his own time, and I will support him with it. When he was feeling better, I did tell him how it made me feel bad to see him so down and he said that I need to take care of myself first and don’t worry about his problems. When he is happy, we have a great time and we can talk about the future. When he is down, he can be a completely different person and it feels difficult to have a conversation with him.

I am worried about losing him since he supports me so much and cares. I don’t want to tell him that I’m feeling down because he’s the one doing it because I don’t want to make him feel worse. I would just like him to be happy again so we can joke and talk all day like before