Ah, você usa supervisorctl start groupname:*
Descobri isso digitando apenas supervisorctl start
e sendo informado:
Error: start requires a process name
start <name> Start a process
start <gname>:* Start all processes in a group
start <name> <name> Start multiple processes or groups
start all Start all processes
... da mesma forma para parar e reiniciar.