Drift apart
Definition 1 of 2
Pro Tip 1/3Emotional Impact
Implies a sense of loss or sadness at the reduced closeness in a relationship. 
It was tough to acknowledge that we had drifted apart after college.
Pro Tip 2/3Natural Process
Suggests the separation is gradual and often unintentional. 
Despite our efforts, we just drifted apart over the years.
Pro Tip 3/3Not Immediate
Indicates that drifting apart from someone doesn't happen suddenly. 
Over the decade, they drifted apart as they pursued different paths.