Husbands Lack of Sexual Desire

My husband and I were together for 16 years, since we were the ages of 16 & 19. We had 4 children together. Nearly two years ago he had an affair and we divorced. Present day, we get along very well. We are extremely close friends and we love each other. We talk about getting back together but we have never been able to address the issue that lured him to the affair to begin with. My husband says making love to me feels as though he is making love to his sister. It wasn’t always this way, just in the later years of our marriage. He is afraid to go to counseling. He was sexually abused as a child and mental illness runs in his family. He refused to go to counseling because he doesn’t want to dig up the past. Perhaps his past doesn’t have anything to do with his lack of sexual desire towards me. Are we better off staying divorced or can we overcome this issue and if so, can you point me down that path?
Thanks,
Paula
( I posted this earlier under another user name by mistake.)