Eu encontrei uma maneira de melhorar o desempenho massivamente: Eu fiz uma configuração personalizada para o driver radeon. Este é o meu /etc/X11/xorg.conf.d/20-radeon.conf
Section "Device"
Identifier "Radeon"
Driver "radeon"
Option "SWcursor" "off" #software cursor might be necessary on some rare occasions, hence set off by default
Option "EnablePageFlip" "on" #supported on all R/RV/RS4xx and older hardware and set off by default
Option "AccelMethod" "EXA" #valid options are XAA and EXA. EXA is the newest acceleration method and its the default.
Option "RenderAccel" "on" #enabled by default on all radeon hardware
Option "ColorTiling" "off" #enabled by default on RV300 and later radeon cards.
Option "EXAVSync" "off" #default is off, otherwise on
Option "EXAPixmaps" "on" #when on icreases 2D performance, but may also cause artifacts on some old cards
Option "AccelDFS" "on" #default is off, read the radeon manpage for more information
EndSection
A opção chave que faz a diferença é o EXAPixmaps, que aumenta muito o desempenho.