Você pode tentar adicionar o controle de cache:
# CACHE-CONTROL HEADERS
<FilesMatch "\.(ico|jpe?g|png|gif|swf|gz|ttf)$">
Header set Cache-Control "max-age=2797200, public"
</FilesMatch>
<FilesMatch "\.(css)$">
Header set Cache-Control "max-age=2797200, public"
</FilesMatch>
<FilesMatch "\.(js)$">
Header set Cache-Control "max-age=2797200, private"
</FilesMatch>
<filesMatch "\.(html|htm)$">
Header set Cache-Control "max-age=86400, public"
</filesMatch>
# Disable caching for scripts and other dynamic files
<FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
FileETag None
Header unset ETag
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires "Sat, 2 Aug 1980 15:15:00 GMT"
</FilesMatch>
e defina seu nível de compactação
# MOD_DEFLATE COMPRESSION
SetOutputFilter DEFLATE
AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css application/x-javascript application/x-httpd-php
DeflateCompressionLevel 9