Build and Deploy Elixir Phoenix releases with CircleCI

I'd really prefer to avoid using Docker or Vagrant if it's not really necessary, and I am getting tired of paying for and running a build server for edeliver to build releases of my applications.

I had thought about using a CI service to build releases in the past but never really gave it a shot, until recently I heard someone talking about using their CI services to build releases and deploy. This motivated me to try it for myself.

I decided to use CircleCi because I like the way their config works, their docs aren't bad, and they have a decent free tier (also their pricing isn't bad if I need to grow).

1: Set up your project

First, you will need to have distillery installed and have already run mix release.init. Follow distillery's install instructions.

2: Set up your server

You will need to have a user with the proper permissions for doing all the things(TM). I generally would use something like my_name and have my app under /home/my_name/apps/my_app. Create the directories now. If you don't use this structure, you will have to tweak the following config and scripts to match your own structure.

3: Set up the CI service

Connect your repository to the CI service and set up your environment variables. I am using these in my config, although you could just hard-code these into your config.yml, doing it this way makes the script more copypasta-friendly.

In project settings under BUILD SETTINGS > Environment Variables:

APP_HOST=123.4.567.89
APP_NAME=my_app
APP_USER=my_name

Then you will also need to have an SSH key pair you will use for deploying your release to your production server. If you don't have a pair you'd like to use, you can generally just make it like:

ssh-keygen -t rsa -C "my_name@my_app.com"

Put the private key in your project settings PERMISSIONS > SSH Permissions and add the public key to your production server user's ~/.ssh/authorized_keys file.

4: Start with your config file

Let's start off with the basics. You will want to create your config file in .circleci/config.yml in your project root.

version: 2
jobs:
  build:
    docker:
      - image: circleci/elixir:1.5
      - image: circleci/postgres:9.6
        environment:
          POSTGRES_USER: postgres
          
    environment:
      - APP_VERSION: "0.0.1"

    working_directory: ~/repo

    steps:
      - checkout

      # Setup
      - run: cp config/dev.secret.example.exs config/dev.secret.exs
      - run: mix local.hex --force
      - run: mix local.rebar

This just sets up the basic elixir stuff, plus postgres for phoenix (if you need it for tests, or something).

I'm using a pre-built circle-ci image as my primary image here, where the base OS is Ubuntu, which is what I also use in production. If you are not using Ubuntu on your production servers, you'll probably want to use some different docker images. Check the different sections in the circle-ci docs about containers to help you figure out how to choose different images if you are unsure.

You'll notice APP_VERSION should match your actual app version for release builds.

I have a dev.secret.example.exs file in my own project, but if you don't have something like that you can remove that line. If you don't need postgres you can remove the three lines for that as well.

Next to add more steps for testing, add this to your config:

- run: mix deps.get
- run: mix test

And now we get to the fun part. For the release and deploy steps, we'll go a bit at a time here.

We will need to make sure ssh and rsync are available.

- run:
    name: Install some System dependencies
    command: sudo apt-get update -qq && sudo apt-get install -y ssh rsync

And for myself, since I use yarn, I need this. You can at the very least skip the yarn step if you don't need it. Maybe node, too (I'm not sure if there is a default version installed that will work).

- run:
    name: Install nodejs
    command: |
      curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -;
      sudo apt-get update && sudo apt-get install -y nodejs;
- run:
    name: Install yarn
    command: |
      curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -;
      echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list;
      sudo apt-get update && sudo apt-get install yarn;

Next, make sure you have your dependencies, and then compile your app.

- run:
    name: Get Mix dependencies
    command: MIX_ENV=prod mix deps.get
- run:
    name: Compile App
    command: MIX_ENV=prod mix compile

And then build the frontend. You might want to replace yarn with npm install

- run:
    name: Install, compile, and digest Frontend
    command: |
      cd assets;
      yarn;
      npm run deploy;
      cd ..;
      MIX_ENV=prod mix phx.digest;

We're getting close now. Build the release...

- run:
    name: Build App release
    command: MIX_ENV=prod mix release --env=prod;

And finally, deploying is two steps here, copying over the release, and then restarting the app.

- deploy:
    name: Deploy to Production
    command: |
      ssh-keyscan -H $APP_HOST >> ~/.ssh/known_hosts;
      rsync -ravhz --stats --progress ./rel/$APP_NAME $APP_USER@$APP_HOST:apps;
- deploy:
    name: Post-deploy and App restart on Production
    command: ./.circleci/post-deploy.sh

The post deploy script is another file I added at .circleci/post-deply.sh:

#!/bin/bash

ssh $APP_USER@$APP_HOST <<ENDSSH
  echo "changing dir to ./apps/$APP_NAME"
  cd ./apps/$APP_NAME;
  date >> deploy-log.txt;

  echo "Symlinking static"
  ln -sfn lib/$APP_NAME-$APP_VERSION/priv/static static;

  echo "symlinking .well-known for LetsEncrypt"
  cd static;
  ln -sfn /var/www/html/.well-known .well-known;
  cd ..;

  echo "stopping app";
  ./bin/$APP_NAME stop >> deploy-log.txt || true;
ENDSSH

ssh $APP_USER@$APP_HOST <<ENDSSH
  cd ./apps/$APP_NAME;
  date >> deploy-log.txt;

  # You may want to use this if you are using Cowboy without proxy
  # echo "giving beam permission to start the webserver"
  # sudo setcap 'cap_net_bind_service=+ep' /home/$APP_USER/$APP_NAME/erts-9.1.2/bin/beam.smp;

  echo "starting app";
  ./bin/$APP_NAME start >> deploy-log.txt;

  echo "Finished";
ENDSSH

Don't forget to chmod +x the post-deploy script.

IMPORTANT The line with sudo setcap, I have here in case I am not proxying with Nginx and am just using Cowboy. I would need to set this in order to give beam permission to use the ports I want (like port 80). I also have made this sudo command run without a sudo password by editing my sudoers file. If you are using Nginx or something, and a non-protected port like 4000 for cowboy you can get rid of this line.

IMPORTANT 2 The line symlinking /var/www/html/.well-known is because that's where I've set that directory for LetsEncrypt. I also have it set in my MyAppWeb.Endpoint config for static files:

plug Plug.Static,
    at: "/", from: :my_app, gzip: true,
    only: ~w(css font fonts images js vendor favicon.ico robots.txt .well-known)

Conclusion

Now, you can push your code, run your tests, build your release, deploy and restart, all in one command and without needing a build server.

There could be more to do, like automatic versioning (I might explore that in a future update), or building upgrade releases, or exploring how to possibly improve build times with circleci caching and artifacts. However, this is a very good start for me now that I can get rid of my build server.

It may have been a dry topic, but I hope you enjoyed this as much as I did.

Here is the final config file:

version: 2
jobs:
  build:
    docker:
      - image: circleci/elixir:1.5
      - image: circleci/postgres:9.6
        environment:
          POSTGRES_USER: postgres

    environment:
      - APP_VERSION: "0.0.1"

    working_directory: ~/repo

    steps:
      - checkout

      # Setup
      - run: cp config/dev.secret.example.exs config/dev.secret.exs
      - run: mix local.hex --force
      - run: mix local.rebar

      # Testing
      - run: mix deps.get
      - run: mix test

      # Release & Deploy
      - run:
          name: Install some System dependencies
          command: sudo apt-get update -qq && sudo apt-get install -y ssh rsync
      - run:
          name: Install nodejs
          command: |
            curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -;
            sudo apt-get update && sudo apt-get install -y nodejs;
      - run:
          name: Install yarn
          command: |
            curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -;
            echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list;
            sudo apt-get update && sudo apt-get install yarn;
      - run:
          name: Get Mix dependencies
          command: MIX_ENV=prod mix deps.get
      - run:
          name: Compile App
          command: MIX_ENV=prod mix compile
      - run:
          name: Install, compile, and digest Frontend
          command: |
            cd assets;
            yarn;
            npm run deploy;
            cd ..;
            MIX_ENV=prod mix phx.digest;
      - run:
          name: Build App release
          command: MIX_ENV=prod mix release --env=prod;
      - deploy:
          name: Deploy to Production
          command: |
            ssh-keyscan -H $APP_HOST >> ~/.ssh/known_hosts;
            rsync -ravhz --stats --progress ./_build/prod/rel/$APP_NAME $APP_USER@$APP_HOST:apps;
      - deploy:
          name: Post-deploy and App restart on Production
          command: ./.circleci/post-deploy.sh

Please comment with any encouragement, feedback, or suggestions.