I think that you should and probably should have on the first date. That being said, there are several reasons for him to wear a wedding band. First, the obvious, he is married - although I doubt that is the reason or he would have taken it off. Second - the ring is of emotional value to him as he is divorced or a widower.
Either way on the former, he is carrying emotional baggage that you should be aware of. I am separated but still wear my wedding band out of respect for our marriage. I don't know when I will take it off since I do not know when I will be ready to date again.
Be careful and look out for yourself. If he's carrying a torch for an ex or his heart still belongs to someone that passed, you should know. A wedding band means a great deal.