In such a case, neither party has consented, but neither has committed a crime.
My gf was horrified and quickly ushered her sister out the door and locked it behind her. I can find a man physically attractive but in no way want to have intimate times with him because his personality isn't compatible.
After a while (when they open their mouths enough) the physical attraction disappears completely and once you get to know someone and love them, even their flaws are just beautiful. It breaks my little heart that so many people have had (or perceive to have had) different experiences.