Karma Computing

Bash deep dive: Inspecting Jenkins Autopilot Docker setup.sh Script

Christopher Simpson -

tldr; You should. You learn some incredibly esoteric things when reading shell scripts line-by-line written by people who know what they're doing.

This is a line-by-line analysis of start.sh which is used to check your environment before deploying Jenkins using the autopilot pattern, an idea developed by staf at Joyent.

Inspired by Ashley Williams talk- A brief history and mishistory of modularity

1 #!/bin/bash

This is a bash script using the Bourne again shell.

2 set -e -o pipefail

Using the set command to set options "change shell and/or script behavior".

  • -e Abort script at first error, when a command exits with non-zero status (except in until or while loops, if-tests, list constructs)

The is especially useful in scrips using piped commands "|" commands, such as:

wget example.com | wc > wordcount.txt

The above would not exit if wget failed, it would simply carry on. Here's a simple example showing what set -e means.

  • -o pipefail Causes a pipeline to return the exit status of the last command in the pipe that returned a non-zero return value.

So -o pipefail could be used to reason about the specific exit code returned from a command upon failure.

  4 help() {
  5     echo 'Usage ./setup.sh [-f docker-compose.yml] [-p project]'
  6     echo
  7     echo 'Checks that your Triton and Docker environment is sane and configures'
  8     echo 'an environment file to use.'
  9     echo
 10     echo 'Optional flags:'
 11     echo '  -f    use this file as the docker-compose config file'
 12     echo '  -p     use this name as the project prefix for docker-compose'
 13 }

Displays the help menu.

 16 # default values which can be overriden by -f or -p flags
 17 export COMPOSE_PROJECT_NAME=jenkins
  • COMPOSE_PROJECT_NAME Sets Docker compose [project's name environment variable](This value is prepended along with the service name to the container on start up.)

This is useful for when inspecting your docker instances (docker ps) to be able to quickly distinguish between them.

COMPOSE_PROJECT_NAME "This value is prepended along with the service name to the container on start up." Docker Docs

Line 18:

 18 export COMPOSE_FILE=
  • COMPOSE_FILE Specfies the Docker compose file name & path, but read on!

This is Docker's COMPOSE_FILE environment variable but note, if it's not provided, Docker "Compose looks for a file named docker-compose.yml in the current directory and then each parent directory in succession until a file by that name is found." which might surprise or please if you've got multiple docker compose builds. We made this mistake on our Jenkins build pipline causing it to fail because a Git repository was wrongly presumed to require building by Docker, except it didn't, causing Docker Compose to directory traverse looking for a compose file which it never found.

20 # give the docker remote api more time before timeout
21 export COMPOSE_HTTP_TIMEOUT=300

Configures the time (in seconds) a request to the Docker daemon is allowed to hang before Compose considers it failed. Defaults to 60 seconds. (docs)

 23 # populated by `check` function whenever we're using Triton
 24 TRITON_USER=
 25 TRITON_DC=
 26 TRITON_ACCOUNT=
  • TRITON_USER Triton username
  • TRITON_DC Which of Joyent's 4 Data centers to use
  • TRITON_ACCOUNT Your Triton account id
    These values are automatically populated by the script, but you need to install the Triton CLI tool and Cloud API first using npm.
 32 # Check for correct configuration
 33 check()

The call to check() commands attempt to verify your environment is configured correctly before proceeding...

 35     command -v docker >/dev/null 2>&1 || {
 36         echo
 37         tput rev  # reverse
 38         tput bold # bold
 39         echo 'Docker is required, but does not appear to be installed.'
 40         tput sgr0 # clear
 41         echo 'See https://docs.joyent.com/public-cloud/api-access/docker'
 42         exit 1
 43     }

The bash command called command is used to "Execute a simple command or display information about commands." (Use help command within your shell to see the options.

It checks that the docker command exists. The standard output from command is thrown away to /dev/null. The 2>&1 means temporarily sending any errors (such as docker not being found) to the shell's stdout and displaying an error message accordingly.

So command -v docker >/dev/null 2>&1 || runs command as normal, is it finds Docker then commands output is redirected to /dev/null, otherwise we have an error and a relevant error message is spat out.

The bash command command also "Can be used to invoke commands on disk when a function with the same name exists" thus avoiding some unlikely but name collision bugs producing false positives.

  • tput rev inverts the display colour (known as "reverse video mode") so black is displayed as white and vice-versa:

  • tput bold # bold makes the error message bold
  • tput sgr0 turns off these settings.
  • All the various options to tpu are known as 'capnames' (terminal capability names)
  • Exits exit 1 with exit code one (error condition) if so
 44     command -v json >/dev/null 2>&1 || {
 45         echo
 46         tput rev  # reverse
 47         tput bold # bold
 48         echo 'Error! JSON CLI tool is required, but does not appear to be installed.'
 49         tput sgr0 # clear
 50         echo 'See https://apidocs.joyent.com/cloudapi/#getting-started'
 51         exit 1
 52     }