Dating a guy with children HELP

A little over a year ago I met a guy through work. We started talking every day and feelings started to develop between us, he was married with two young girls in the process of separating from his wife. They filed for a separation agreement and he moved out. He was to continue to pay for everything until the end of their lease and his ex wife would move out of state with the children roughly 6 hours away. Per the agreement, they are supposed to meet halfway to allow him to have the children every two weeks. This has not been the case. He drives roughly 6 hours after working a 10hr day to her home and stays the 2 or 3 days at her home to be with his kids. The divorce is not final until November and he has been doing this for a couple of months now. Now I understand he wants to be with his children but I am not comfortable with him staying at her home with them. When he visits them he will barely communicate with me and this upsets me greatly. I unfortunately have terrible anxiety and he knows this. I have expressed how I feel multiple times but it's like my feelings are not taken into consideration. I find it extremely unfair for him to be doing all of the work to see his kids and her not helping at all. She moved away to be closer to family to have help with the children so she could work after being a stay at home mom. I am so lost and do not know what to do. I love him but it hurts so much what he is doing. Please help!