On the distaff side
Definition 1 of 1
Pro Tip 1/3Gender Specific
The phrase exclusively refers to female relatives, not male ones. 
He inherited his love for music from the distaff side.
Pro Tip 2/3Archaic Term
'On the distaff side' is old-fashioned and rarely used in modern conversation. 
Our family's crest comes from the distaff side, from many generations ago.
Pro Tip 3/3Not Literal
This phrase is metaphorical and doesn't involve a real distaff; it signifies the female line. 
The secret recipe was passed down on the distaff side.