Postpartum anxiety for new dad

Hello everyone I know this may sound different than normal postpartum issues but I am a new dad to a beautiful 4 week old baby girl. I have been struggling with severe anxiety and panick attacks with the baby specifically when the thought of being alone with the baby when my wife needs to leave the house. I have yet to be in a comfortable and confident enough position to allow my wife time to get out of the house whether it’s a quick run to the store or to go get her nails done, the thought of being alone and not having her there as a security if I need her has been crippling. This puts me in a tough position because as the father/husband I feel it is my duty to allow my wife the time and relaxation she needs to do her own recovering and post partum relaxation. It’s hard to do that when I’m not okay with the baby myself. She has been incredibly loving and supporting of my overthinking/anxiety and has even allowed me to get out with the boys and go golfing to see if that would help me cope with my anxiety. Although it did help it also brings an enormous amount of guilt that I carry that I am unable to provide her with the same support, as I get immense anxiety at the thought of her leaving the house and leaving me alone with the baby. This has been a troubling issue in our lives the last 4 weeks since the baby has been born. I’m hoping it’s just an adjustment period for me and it will all get better down the line, however it has been very difficult to deal with day in and day out that I am not providing the relief or relaxation that my wife truly needs. Just curious if any other first time dads have experienced this as well and have any advice on how to cope or make change.