Hello dear @Invisible61114,
I have been myself in a similar situation with one of my ex boyfriends. I was already having trust issues and the fact that he wouldn't let me play on his phone or really see anything that he was doing there was driving me crazy. Now people have different degrees of tolerance for having their privacy "stolen" and some just don't like it to be controlled because it triggers their own schema and negative believes and emotions. I think the most important thing is to be able to discuss this issue openly with your boyfriend.. if he isn't really in for discussing it at all it might mean that he either really has something to hide or he has dysfunctional communication and interaction patterns. Both things are not easy to deal with and I wouldn't personally invest in a relationship with someone I feel I can't trust.. but no one can tell you exactly what to do because after all you know the situation better. :)