he has a “friend”

My husband has a friend (female) he met through a previous job known her about the sam asme 20 yrs. His son from a previous marriage went to school with her kids. Anyway, she is married about 10 yrs older then him. Through phone records he talks to her every day several times a day. Now, weve been together a long time so i know his habbits i know when he is lying and hiding something from me. He talks to her sometimes for hours a day. last time she called we were watching tv and he walked into the bedroom went into the bathroom to talk to her. I confronted he said the tv was too loud he didnt think about turning it down. he never has an issue with this when talking to his friends. And this is not the first time he has waked away or go outside or go into the bedroom. he says they are just friends but in the 20 plus yrs ive only seen her once. should i be worried?