Tudo bem, depois de várias horas de tentativa e erro. E um monte de olhos diferentes que deram uma olhada finalmente cheguei a uma configuração que se parecia com isso:
ProxyRequests off
ProxyPass "/map" "http://192.168.2.31:8123/"
ProxyPass "/js" "http://192.168.2.31:8123/js/"
ProxyPass "/js/minecraft" "http://192.168.2.31:8123/js/minecraft/"
ProxyPass "/js/map" "http://192.168.2.31:8123/js/map/"
ProxyPass "/css" "http://192.168.2.31:8123/css/"
ProxyPass "/standalone" "http://192.168.2.31:8123/standalone"
ProxyPass "/up/configuration" "http://192.168.2.31:8123/up/configuration"
ProxyPass "/version.js" "http://192.168.2.31:8123/version.js"
ProxyPass "/images" "http://192.168.2.31:8123/images/"
ProxyPass "/tiles/world/t" "http://192.168.2.31:8123/tiles/world/t"
ProxyPass "/tiles/world/t_day" "http://192.168.2.31:8123/tiles/world/t_day"
ProxyPass "/tiles/world_nether/nt" "http://192.168.2.31:8123/tiles/world_nether/nt"
ProxyPass "/tiles/world_nether/flat" "http://192.168.2.31:8123/tiles/world_nether/flat"
ProxyPass "/tiles/world_the_end/st" "http://192.168.2.31:8123/tiles/world_the_end/st"
ProxyPass "/up/world/world" "http://192.168.2.31:8123/up/world/world"
ProxyPass "/up/world/world_nether" "http://192.168.2.31:8123/up/world/world_nether"
ProxyPass "/up/world/world_the_end" "http://192.168.2.31:8123/up/world/world_the_end"
ProxyPass "/mapimages" "http://192.168.2.31:8123/mapimages/"
ProxyPass "/tiles/_markers_" "http://192.168.2.31:8123/tiles/_markers_"
ProxyPass "/tiles/faces/16x16" "http://192.168.2.31:8123/tiles/faces/16x16"
ProxyPass "/tiles/faces/32x32" "http://192.168.2.31:8123/tiles/faces/32x32"
ProxyPassReverse "/map" "http://192.168.2.31:8123/"
ProxyPassReverse "/js" "http://192.168.2.31:8123/js/"
ProxyPassReverse "/js/minecraft""http://192.168.2.31:8123/js/minecraft/"
ProxyPassReverse "/js/map" "http://192.168.2.31:8123/js/map/"
ProxyPassReverse "/css" "http://192.168.2.31:8123/css/"
ProxyPassReverse "/standalone" "http://192.168.2.31:8123/standalone"
ProxyPassReverse "/up/configuration" "http://192.168.2.31:8123/up/configuration"
ProxyPassReverse "/version.js" "http://192.168.2.31:8123/version.js"
ProxyPassReverse "/images" "http://192.168.2.31:8123/images/"
ProxyPassReverse "/tiles/world/t" "http://192.168.2.31:8123/tiles/world/t"
ProxyPassReverse "/tiles/world/t_day" "http://192.168.2.31:8123/tiles/world/t_day"
ProxyPassReverse "/tiles/world_nether/nt" "http://192.168.2.31:8123/tiles/world_nether/nt"
ProxyPassReverse "/tiles/world_the_end/st" "http://192.168.2.31:8123/tiles/world_the_end/st"
ProxyPassReverse "/up/world/world" "http://192.168.2.31:8123/up/world/world"
ProxyPassReverse "/up/world/world_nether" "http://192.168.2.31:8123/up/world/world_nether"
ProxyPassReverse "/up/world/world_the_end" "http://192.168.2.31:8123/up/world/world_the_end"
ProxyPassReverse "/mapimages" "http://192.168.2.31:8123/mapimages/"
ProxyPassReverse "/tiles/_markers_" "http://192.168.2.31:8123/tiles/_markers_"
ProxyPassReverse "/tiles/faces/16x16" "http://192.168.2.31:8123/tiles/faces/16x16"
ProxyPassReverse "/tiles/faces/32x32" "http://192.168.2.31:8123/tiles/faces/32x32"
Isso funciona (no meu caso), mas sei que tenho que redefinir todas as regras se houver uma alteração na estrutura do catálogo. Tudo funciona como deveria com esta configuração.
Apenas altere os nomes do mundo se você for diferente.
Também recomendo que qualquer leitor tente a sugestão "AnrDaemons"