Somebody’s face doesn’t fit
Definition 1 of 1
Pro Tip 1/3Social Acceptance
This phrase often relates to social dynamics and indicates someone doesn't fit in the group's social criteria. 
Despite his skills, his face didn’t fit in the exclusive club's standards.
Pro Tip 2/3Not Just Looks
The phrase isn't about physical appearance only, but also behavior, personality, or social background. 
She found it hard in the new city; her face just didn't fit there.
Pro Tip 3/3Subjective Feeling
It's often used to express a subjective opinion and not an objective fact about the person. 
I felt like my face didn’t fit at the party, even though no one said anything.