Citação da documentação do arquivo de configuração do Supervisord:
redirect_stderr
If true, cause the process’ stderr output to be sent back to supervisord on its stdout file descriptor (in UNIX shell terms, this is the equivalent of executing/the/program 2>&1).
stderr_logfile
Put process stderr output in this file unless redirect_stderr is true.
Então, sim, stderr e stdout aparecerão em stdout.log e stderr.log permanecerão vazios.