Relationship with autism dad

My partners 17 year old has autism.

We are (were) dating for 10 months. He never really explained to me the severity of his sons condition and basically glossed over things when speaking or barely speaking about him.

He was constantly pushing for our kids to meet as it would be a way for us to have more time with each other. The first time I met his son he was swaying back and forth in the car seat. I felt like I was thrown off the deep end of a pier and no clue what or how to react. He at no point checked in with me to see if I was ok, how I am coping. He reacted as if it was the most natural thing in the world and I could not possibly have an issue with it.

Time went by, I saw his son in total about less than 10 times. During the summer we were thinking of spending two weeks on holiday with the kids but my daughter just started school and her dad was a bit concerned as to how she would respond and asked if we could keep our same routine with her. I agreed and so not really able to travel far with my daughter and so stayed locally, while my partner went abroad with his son.

Upon returning his son to his mom, he had a meltdown, was very aggressive, his mom could not restrain him and my partner had to go to the house to restrain him. He left from there with lots of scratches. Things were really tense for his son and his mom that week to the point where his mom thought it best that he be hospitalized for observation. The mom also texted frantically once because he took a nail file away from her and would not give it back and she did not know what to do.

My pater then gets back his sone about 3 weeks later. No discussion with me about how his son is doing. No checking in with me as to how I feel about the whole thing, if I have concerns, etc. The first time I see his son after the summer incident I am a bit on edge, especially as I had a bad experience with an autistic young man (he came up to my bed while I was sleeping and was leaning over me. I had a sensation of someone else being in the room with me Only to see this young man leaning over me while I was sleeping. I freaked out and started screaming).

So after hearing about his sons meltdown, my pervious experience and not sure if he can have a meltdown again (no one seems to know what triggers it) i told my partner that I did not feel comfortable sleeping over at his apartment when his son there. He refuses to understand why I would be afraid as he is there, refuses to accept that I could be afraid. We then had a chat about it. I told him that I am trying to get to know his son but our contact is very infrequent, and I am a bit at a loss as to who he is as my partner is not forthcoming and when he does he comes off defensive. We decide maybe the best course of action is to have a chat with my therapist would could maybe give us some tools as to how to move forward. He agree to think about it and get back to me.That was about 2 weeks ago.

Last Monday, he calls saying his feelings about me have changed. That I hurt his feelings when I told him I was not comfortable around his son, that he cannot see how my daughter and his son would get along as other kids need to adhere to certain requirements around his son, that I am too negative (stemming from bad experiences I have with my new landladies), that I allow my ex to change the days we have with our daughter which then affects our time together (which was my fault and I have being profusely apologizing for this for months now), that sometimes I let my daughter get desert even when she is not finished her dinner and the list of all the ‘bad things about me are just spewed out.

Now he claims he needs time to think about what to do about us, and I am left feeling gob-smacked and at a loss for words. What do I do?