Você precisa citar o token de EOF:
cat <<"EOF" > Dockerfile
RUN apt-get update -y \
&& apt-get install -y \
bsdtar \
git \
locales
EOF
Se você quiser expandir as variáveis também, é necessário escapar das barras invertidas e não usar aspas.
Aqui está a seção man bash
correspondente.
[n]<<[-]word
here-document
delimiter
No parameter and variable expansion, command substitution, arithmetic
expansion, or pathname expansion is performed on word. If any part of
word is quoted, the delimiter is the result of quote removal on word,
and the lines in the here-document are not expanded. If word is
unquoted, all lines of the here-document are subjected to parameter
expansion, command substitution, and arithmetic expansion, the charac‐
ter sequence \<newline> is ignored, and \ must be used to quote the
characters \, $, and '.