Look The Fathers know best :
"In regard to chastity, [Jesus] has-this to say: 'If anyone look with lust at a woman, he has already before God committed adultery in his heart.' And, 'Whoever marries a woman who has been divorced from another husband, commits adultery.' According to our Teacher, just as they are sinners who contract a second marriage, even though it be in accord with human law, so also are they sinners who look with lustful desire at a woman. He repudiates not only one who actually commits adultery, but even one who wishes to do so; for not only our actions are manifest to God, but even our thoughts"
First Apology 15
"No one is permitted to know a woman other than his wife. The marital right is given you for this reason: lest you fall into the snare and sin with a strange woman. 'If you are bound to a wife do not seek a divorce'; for you are not permitted, while your wife lives, to marry another."
Abraham 1:7:59 [387 CE].
"Do not tell me about the violence of the ravisher, about the persuasiveness of a mother, about the authority of a father, about the influence of relatives, about the intrigues and insolence of servants, or about household [financial] losses. So long as a husband lives, be he adulterer, be he sodomite, be he addicted to every kind of vice, if she left him on account of his crimes he is still her husband still and she may not take another"
Letters 55:3 [396 CE].
"Wherever there is fornication and a suspicion of fornication a wife is freely dismissed. Because it is always possible that someone may calumniate the innocent and, for the sake of a second joining in marriage, act in criminal fashion against the first, it is commanded that when the first wife is dismissed a second may not be taken while the first lives"
Commentaries on Matthew 3:19:9 [398 CE].
"Neither can it rightly be held that a husband who dismisses his wife because of fornication and marries another does not commit adultery. For there is also adultery on the part of those who, after the repudiation of their former wives because of fornication, marry others. This adultery, nevertheless, is certainly less serious than that of men who dismiss their wives for reasons other than fornication and take other wives. Therefore, when we say: 'Whoever marries a woman dismissed by her husband for reason other than fornication commits adultery,' undoubtedly we speak the truth. But we do not thereby acquit of this crime the man who marries a woman who was dismissed because of fornication. We do not doubt in the least that both are adulterers. We do indeed pronounce him an adulterer who dismissed his wife for cause other than fornication and marries another, nor do we thereby defend from the taint of this sin the man who dismissed his wife because of fornication and marries another. We recognize that both are adulterers, though the sin of one is more grave than that of the other. No one is so unreasonable to say that a man who marries a woman whose husband has dismissed her because of fornication is not an adulterer, while maintaining that a man who marries a woman dismissed without the ground of fornication is an adulterer. Both of these men are guilty of adultery"
Adulterous Marriages 1:9:9 [419 CE].
"A woman begins to be the wife of no later husband unless she has ceased to be the wife of a former one. She will cease to be the wife of a former one, however, if that husband should die, not if he commit fornication. A spouse, therefore, is lawfully dismissed for cause of fornication; but the bond of chastity remains. That is why a man is guilty of adultery if he marries a woman who has been dismissed even for this very reason of fornication"