Go to rack and ruin
Definition 1 of 1
Pro Tip 1/3Neglect and Damage
The phrase emphasizes that something has been neglected and fallen into disrepair, often over time. 
Without maintenance, the garden will go to rack and ruin.
Pro Tip 2/3Not Literal
Can be used metaphorically to describe things that are not physical, like relationships or skills. 
Their friendship went to rack and ruin after the argument.
Pro Tip 3/3Usually Irreversible
Suggests a state that is difficult to reverse or recover from. 
The company's reputation has gone to rack and ruin.