We man are born to be polygamous but it doesn’t mean that we should always keep it in our mind. All you have to do is to avoid the affection that you had with your co-worker. You can do that. You made that marriage without anticipation and I hope you can find a way to avoid problem with your wife. If you do not love your wife anymore, then you both decide for divorce. Make sure she will agree with you so it would be fair for the both of you.