Atualmente isso é impossível, de acordo com o suporte do Slack (eu perguntei sobre isso hoje):
Unfortunately this is a problem we currently have with the Windows app, and I'm sorry it is getting in your way.
We do have it on our list to address, and hopefully that will roll out as you update in the near future. I can't provide an ETA, but if you'd like I can follow up as we resolve this one.