É melhor colocá-lo em uma variável, como recomendado pelo manual de referência do Bash 3.2 .4.2 Construções condicionais :
Storing the regular expression in a shell variable is often a useful way to avoid problems with quoting characters that are special to the shell. It is sometimes difficult to specify a regular expression literally without using quotes, or to keep track of the quoting used by regular expressions while paying attention to the shell’s quote removal. Using a shell variable to store the pattern decreases these problems.
No entanto, para escrevê-lo diretamente dentro do bash extended test, você precisa remover as aspas e escapar dos espaços:
$ [[ ' 123 ' =~ ^\ [0-9]+\ $ ]]; echo $?
0