To the detriment of somebody/something
Definition 1 of 1
Pro Tip 1/3Negative Outcome
Use when an action results in something unfavorable or harmful, not just any negative feeling. 
Skipping meals may lead to weight loss, but to the detriment of one's energy levels.
Pro Tip 2/3Priority Conflict
Apply when prioritizing one thing results in neglecting another, which suffers as a consequence. 
She prioritized her career to the detriment of her personal relationships.
Pro Tip 3/3Emotional Nuance
Highlights a negative emotional impact, especially when sacrifice or harm wasn't anticipated. 
His obsession with perfection led to success but to the detriment of his happiness.