Parece que cada um dos apt-get e pip instalou sua própria versão da biblioteca.
Tente remover o sistema mysql com sudo apt-get remove python-mysqldb
e
você deveria ter algo assim:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
python-mysqldb
0 upgraded, 0 newly installed, 1 to remove and 3 not upgraded.
After this operation, 196 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 109333 files and directories currently installed.)
Removing python-mysqldb (1.2.3-2ubuntu1) ...