A menos que você tenha qualquer plugin aplicando sua mágica, isso provavelmente é causado pelo valor % em 'viminfo' :
% When included, save and restore the buffer list. If Vim is started with a file name argument, the buffer list is not restored. If Vim is started without a file name argument, the buffer list is restored from the viminfo file. Buffers without a file name and buffers for help files are not written to the viminfo file.