Husband's Fetish is Causing Stress

Hi, all. I am new to the site and am looking for some objectivity with an ongoing sexual issue in my marriage. I have been married for 6 years to a kind, loving, creative man, but we have always had issues in the bedroom. When we got together, he had considerably more sexual experience than me, and I think that is part of the root of our problems.

For several years, he has had a fantasy that I should and should want to sleep with other men. The fantasy is so pervasive that he cannot orgasm without thinking or talking about me with another man. I honestly would not mind engaging this type of dity talk on occassion as a fantasy, but it is something he truly wants and cannot have sex without at least thinking about it. I have no interest in ever sleeping with another man, as conventional as that may be. It makes me feel like he is my pimp and can pass me around to any man he chooses. I have expressed my feelings on the matter many times but we both end up feeling guilty and defeated. I have tried to take sex in other directions (toys, fantasies of us doing things together), but he always brings the talk back to me being with another man. Currently, I feel so disconnected from sex because I feel like my true sexual expression is not enough for him.

I have made genuine efforts in the past to engage in his fantasy by watching cuckhold porn together and even chatting/sending pics to men online while he watches. It never seems to be enough. I have discovered him on several occassions posting to Craigslist and on swingers sites looking to engage in this lifestyle. I have even caught him talking to people we know on Facebook. Those are the most painful to discover. I don't have concrete evidence that he has physically cheated on me, but I would not be surprised if that were the case.

I guess I am here to find out which one of us is being unreasonable in this situation? Should I just let go of inhibition (against my better judgement) and try harder to emrbace his fantasy? Or should he be more respectful of my boundaries?