Uma string entre aspas simples manterá a string como literal. Uma string de aspas duplas reterá a string com interpolação e expansão variáveis. Isso é explicado na página man bash - veja a seção intitulada QUOTING
There are three quoting mechanisms: the escape character, single quotes, and double quotes.
A non-quoted backslash (\) is the escape character. It preserves the literal value of the next character that follows, with the exception of {newline}. If a \{newline} pair appears, and the backslash is not itself quoted, the \{newline} is treated as a line continuation (that is, it is removed from the input stream and effectively ignored).
Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of $, ', \, and, when history expansion is enabled, !. The characters $ and ' retain their special meaning within double quotes. The backslash retains its special meaning only when followed by one of the following characters: $, ', ", \, or {newline}. A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ! appearing in double quotes is escaped using a backslash. The backslash preceding the ! is not removed.