You say you only want to be with your husband because of security, but you have put that security in jeopardy with your affair. Sooner or later he will find out and the longer you let this affair drag on the worse the outcome is going to be. If you think everyone is going to think badly about you because of a divorce, imagine what they’ll think when they find out about your affair. And they will.
I think you need to do some soul searching and decide what is most important to you for your life and your future. If you don’t think your marriage is going to work you need to face that issue and get divorced sooner rather than later. Whatever you do, do not have children. That would be a disaster.