Unexpected bash variable expansion

 · 3 min · torgeir

Bash variables are expanded before your command runs. Beware if you set them in front of your commands.

Terminal Bash

I tuck variable assignments in front of commands in the terminal all the time. Mostly to prevent them from sticking around the shell after running commmands. They appear like environment variables to the process that follows them. I also setopt HIST_IGNORE_SPACE in ~/.zshrc so that commands I choose to prefix with a space will not be saved to the history. Useful if you need to set a token or something.

So, today I did this.

 TOKEN=asdf curl http://localhost:3000 \
     -H "Authorization: bearer $TOKEN"

In an attempt to prevent needing to export it in my shell (so it sticks around), and to prevent it from appearing in the zsh history.

Test it by starting a local server, e.g. with netcat

nc -l 3000

Then run the curl command.

GET / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.85.0
Accept: */*
Authorization: bearer

To my surprise the token was not present. At first I thought it might be the \, used to wrap long lines for readability. But as an even simpler example, this also does not work

 TOKEN=asdf echo $TOKEN

Why?

This is because the shell first expands $TOKEN, that has yet to be set. Hence, it expands to the empty string "" . This happens before the command is run, so what actually ends up running is this

 TOKEN=asdf echo

No wonder there’s no token.

There are lots of ways to solve this, here’s a few

Solution 1: A program reads the variable

I guess I’m usually in luck that this is the case for me, which is probably why I expected the initial curl command to work.

When some program reads the environment variable, e.g. if this was the contents of test.sh

#!/bin/bash
echo $TOKEN

Make your user able to run it

chmod u+x test.sh

The following works as you’d expect

 TOKEN=123 ./test.sh

123

Solution 2: Pass the literal string to bash -c

Use single quotes, i.e. ' , to pass the literal string, including the (not expanded) $TOKEN to bash, using the -c option to make it read the command from a string.

 TOKEN=asdf bash -c '
   curl http://localhost:3000 \
       -H "Authorization: bearer $TOKEN"'

GET / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.85.0
Accept: */*
Authorization: bearer asdf

Note that using double quotes, i.e. ", will not work, as $TOKEN again is expanded before the command runs.

Solution 3: Export the variable in a subshell

By exporting the token, but inside a subshell, by wrapping the command in parens, the token is visible to all programs that are forked inside the subshell. Once the subshell (the command in parens) is finished, the TOKEN is again gone.

(export TOKEN=asdf;
 curl http://localhost:3000 \
     -H "Authorization: bearer $TOKEN")

GET / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.85.0
Accept: */*
Authorization: bearer asdf

Solution 4: But I use zsh

Yeah, me too. In that case, the following also works.

(TOKEN=asdf;
 curl http://localhost:3000 \
     -H "Authorization: bearer $TOKEN")

GET / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.85.0
Accept: */*
Authorization: bearer asdf