Obviamente, o 64 bit se você quiser fazer uso total da capacidade de hardware do seu sistema, mas aqui estão alguns pontos a serem considerados:
What is in the 32-bit version of Office but is not included in the 64-bit version of Office?
ActiveX controls library, ComCtl This library contains ActiveX controls that are used to build solutions. It is most commonly used in the following Microsoft Office programs: Access, Excel, and Word. SharePoint List control The list view in SharePoint Technology is not available to people using the 64-bit version of Office.
Esse cara defende o Office de 32 bits, já que a grande maioria dos usuários não ganha nada com 64 bits:
O mesmo vale para o Office 2013 também.