A primeira resposta (aceita) aqui funcionou para mim:
The Git Extension files are stored in your 'My Documents' folder in a "Visual Studio 2012" subfolder. Removing these files will make the toolbar disappear from within MSVC.
More specifically, they're under "Addins". Two .dlls with gitplugins in the name. Kill those and the annoying context menu stuff goes away next time you restart vs.net.