Useless gossip. Women will throw themselves at men, and you've got to decide what you want to deal with. He's cool with being a baby-daddy; she's cool with whatever child support he can provide, which is plenty.
The real problem is the publicity. Dumb peasants will do the same -- the resulting unsupported fatherless children are the source of a criminal plague. This is why Christian social structure is important.