Kick somebody upstairs
Definition 1 of 1
Pro Tip 1/3Hidden Criticism
The phrase can imply a lack of competence despite the promotion. 
The board had to kick him upstairs to avoid any more PR disasters.
Pro Tip 2/3Not Literal
It's a metaphorical expression, not referring to actual physical movement. 
After the merger mess, they kicked her upstairs to a corner office.
Pro Tip 3/3Organizational Politics
Common in workplace politics where an actual demotion isn't possible. 
With his father on the board, they kicked him upstairs instead of firing him.