Apache 2.4 no CentOS:
Etapa 1:
cd /etc/yum.repos.d/
wget http://repos.fedorapeople.org/repos/jkaluza/httpd24/epel-httpd24.repo
Etapa 2:
yum install httpd24.x86_64
Etapa 3:
$ /opt/rh/httpd24/root/usr/sbin/httpd -version
Server version: Apache/2.4.6 (Red Hat)
Server built: Sep 25 2013 05:25:46
NOTA: os arquivos de configuração estão em: /opt/rh/httpd24/root/etc/httpd
$ ls
conf conf.d conf.modules.d logs modules run
EDIT: caso você queira desativar o Apache 2.2
$ chkconfig httpd off
$ chkconfig --list | grep httpd
httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
httpd24-httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
EDIT 2: link
yum install php-fpm
/etc/init.d/php-fpm start