A solução foi simplesmente limpar as variáveis de ambiente, assim:
lxc-attach --clear-env -n mycontainer -- apt-get install -y wget
De lxc-attach --help
:
--clear-env Clear all environment variables before attaching.
The attached shell/program will start with only
container=lxc set.
--keep-env Keep all current environment variables. This
is the current default behaviour, but is likely to
change in the future.