O problema é a ordem das regras :)
Quando mudei a ordem da seguinte forma, tudo funcionou:
RewriteRule ^test/([0-9]+)$ test.php?id=$1 [L]
# To remove .php extension
RewriteCond %{THE_REQUEST} ^[A-Z]{3,}\ (.*)\.php [NC]
RewriteRule ^ %1 [R=301,L]
# To check whether the file exists then set it back internally
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME}.php -f
RewriteRule ^.*$ $0.php [L]=