Years of Hurt - How do we Repair?

[summerd]

After more than a year of separation, my husband and I have reconciled our marriage eight months ago. We've been working on things as best we can alone. One of the biggest areas of dysfunction for our marriage was sexuality and still is. He simply, very rarely, wants to have sex with me. There are so many issues under the surface that I'm afraid we may be unable to solve. Many years of marriage where he felt inadequate in just about every way and I tried to earn his love and affection by just doing stuff. His depression, his affair, my control, my devastation over so much rejection - I could go on and on. At the root of it all is a desparate pursuit of love we both lacked in our upbringing (guess it's that way for all us wounded folks) Things have been so much better since he moved back home and we do applaud our successes. But I feel us slipping back into some of our same patterns and it is so very frightening.
We recently took a wonderful family trip together and it was almost bliss. We had one sexual encounter during the week and the intimacy, affection, touches, everything felt so special. But in the airport on the way home, it was as if I developed the plague and it's been rough off and on for many weeks now. When I try to encourage the good things, he feels patronized. When I try to tell him how I feel, he often becomes defensive or angry. He views my fears about our difficulties as a message to him that everything is his fault despite my realization and acknowledgement of all I contributed to the problems we have faced and still face. He feels so bad about himself that I have spent years afraid to tell him how I feel because he views most everything as another reason he's an inadequate person. It is very difficult to ALWAYS hold back the tears and emotion when trying to tell the person you love that their lack of desire for you makes you feel ugly and less than a woman. It is also very tough to watch the man you love feel so bad about himself. I want to suggest a counselor again but it seems like many things I suggest lead him to feel worse about himself - the last thing I want to do. I loved this man enough to love him back home. But I don't know how the love him out of his own feelings of inadequacy. I realize that's probably impossible.
But I know that I made him comfortable enough with me to come home. He quit his job of 20 years to get away for the OW. What an amazing step he took and I know how hard it was. However, much of that matter still feels unresolved for me. He has shown me in many ways that he's glad to be home but I really would someday like him to look me in the eyes and tell me he's sorry. There is still so much work for us to do. I don't think I'm advanced enough in my healing process to turn every conversation into a fruitful one. I don't get angry anymore but every now and then I say something that he takes the wrong way. And it seems every conversation that isn't fruitful sets us back months. Many days I feel like giving up - but then I think about how much worse I felt when he was gone. I don't want to give up but I don't want to live the rest of my life feeling this way nor do I want that for him.