Build and Deploy Elixir Phoenix releases with CircleCI
I'd really prefer to avoid using Docker or Vagrant if it's not really necessary, and I am getting tired of paying for and running a build server for edeliver to build releases of my applications.
I had thought about using a CI service to build releases in the past but never really gave it a shot, until recently I heard someone talking about using their CI services to build releases and deploy. This motivated me to try it for myself.
I decided to use CircleCi because I like the way their config works, their docs aren't bad, and they have a decent free tier (also their pricing isn't bad if I need to grow).
1: Set up your project
First, you will need to have distillery installed and have already run mix release.init
. Follow distillery's install instructions.
2: Set up your server
You will need to have a user with the proper permissions for doing all the things(TM). I generally would use something like my_name
and have my app under /home/my_name/apps/my_app
.
Create the directories now. If you don't use this structure, you will have to tweak the following config and scripts to match your own structure.
3: Set up the CI service
Connect your repository to the CI service and set up your environment variables.
I am using these in my config, although you could just hard-code these into your config.yml
, doing it this way makes the script more copypasta-friendly.
In project settings under BUILD SETTINGS > Environment Variables:
APP_HOST=123.4.567.89
APP_NAME=my_app
APP_USER=my_name
Then you will also need to have an SSH key pair you will use for deploying your release to your production server. If you don't have a pair you'd like to use, you can generally just make it like:
ssh-keygen -t rsa -C "my_name@my_app.com"
Put the private key in your project settings PERMISSIONS > SSH Permissions and add the public key
to your production server user's ~/.ssh/authorized_keys
file.
4: Start with your config file
Let's start off with the basics. You will want to create your config file in .circleci/config.yml
in your project root.
version: 2
jobs:
build:
docker:
- image: circleci/elixir:1.5
- image: circleci/postgres:9.6
environment:
POSTGRES_USER: postgres
environment:
- APP_VERSION: "0.0.1"
working_directory: ~/repo
steps:
- checkout
# Setup
- run: cp config/dev.secret.example.exs config/dev.secret.exs
- run: mix local.hex --force
- run: mix local.rebar
This just sets up the basic elixir stuff, plus postgres for phoenix (if you need it for tests, or something).
I'm using a pre-built circle-ci image as my primary image here, where the base OS is Ubuntu, which is what I also use in production. If you are not using Ubuntu on your production servers, you'll probably want to use some different docker images. Check the different sections in the circle-ci docs about containers to help you figure out how to choose different images if you are unsure.
You'll notice APP_VERSION
should match your actual app version for release builds.
I have a dev.secret.example.exs
file in my own project, but if you don't have something like that you can remove
that line. If you don't need postgres you can remove the three lines for that as well.
Next to add more steps for testing, add this to your config:
- run: mix deps.get
- run: mix test
And now we get to the fun part. For the release and deploy steps, we'll go a bit at a time here.
We will need to make sure ssh and rsync are available.
- run:
name: Install some System dependencies
command: sudo apt-get update -qq && sudo apt-get install -y ssh rsync
And for myself, since I use yarn, I need this. You can at the very least skip the yarn step if you don't need it. Maybe node, too (I'm not sure if there is a default version installed that will work).
- run:
name: Install nodejs
command: |
curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -;
sudo apt-get update && sudo apt-get install -y nodejs;
- run:
name: Install yarn
command: |
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -;
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list;
sudo apt-get update && sudo apt-get install yarn;
Next, make sure you have your dependencies, and then compile your app.
- run:
name: Get Mix dependencies
command: MIX_ENV=prod mix deps.get
- run:
name: Compile App
command: MIX_ENV=prod mix compile
And then build the frontend. You might want to replace yarn
with npm install
- run:
name: Install, compile, and digest Frontend
command: |
cd assets;
yarn;
npm run deploy;
cd ..;
MIX_ENV=prod mix phx.digest;
We're getting close now. Build the release...
- run:
name: Build App release
command: MIX_ENV=prod mix release --env=prod;
And finally, deploying is two steps here, copying over the release, and then restarting the app.
- deploy:
name: Deploy to Production
command: |
ssh-keyscan -H $APP_HOST >> ~/.ssh/known_hosts;
rsync -ravhz --stats --progress ./rel/$APP_NAME $APP_USER@$APP_HOST:apps;
- deploy:
name: Post-deploy and App restart on Production
command: ./.circleci/post-deploy.sh
The post deploy script is another file I added at .circleci/post-deply.sh
:
#!/bin/bash
ssh $APP_USER@$APP_HOST <<ENDSSH
echo "changing dir to ./apps/$APP_NAME"
cd ./apps/$APP_NAME;
date >> deploy-log.txt;
echo "Symlinking static"
ln -sfn lib/$APP_NAME-$APP_VERSION/priv/static static;
echo "symlinking .well-known for LetsEncrypt"
cd static;
ln -sfn /var/www/html/.well-known .well-known;
cd ..;
echo "stopping app";
./bin/$APP_NAME stop >> deploy-log.txt || true;
ENDSSH
ssh $APP_USER@$APP_HOST <<ENDSSH
cd ./apps/$APP_NAME;
date >> deploy-log.txt;
# You may want to use this if you are using Cowboy without proxy
# echo "giving beam permission to start the webserver"
# sudo setcap 'cap_net_bind_service=+ep' /home/$APP_USER/$APP_NAME/erts-9.1.2/bin/beam.smp;
echo "starting app";
./bin/$APP_NAME start >> deploy-log.txt;
echo "Finished";
ENDSSH
Don't forget to chmod +x
the post-deploy script.
IMPORTANT The line with sudo setcap
, I have here in case I am not proxying with Nginx and am just
using Cowboy. I would need to set this in order to give beam permission to use the ports I want (like port 80).
I also have made this sudo command run without a sudo password by editing my sudoers file.
If you are using Nginx or something, and a non-protected port like 4000 for cowboy you can get rid of this line.
IMPORTANT 2 The line symlinking /var/www/html/.well-known
is because that's where I've set that directory
for LetsEncrypt. I also have it set in my MyAppWeb.Endpoint
config for static files:
plug Plug.Static,
at: "/", from: :my_app, gzip: true,
only: ~w(css font fonts images js vendor favicon.ico robots.txt .well-known)
Conclusion
Now, you can push your code, run your tests, build your release, deploy and restart, all in one command and without needing a build server.
There could be more to do, like automatic versioning (I might explore that in a future update), or building upgrade releases, or exploring how to possibly improve build times with circleci caching and artifacts. However, this is a very good start for me now that I can get rid of my build server.
It may have been a dry topic, but I hope you enjoyed this as much as I did.
Here is the final config file:
version: 2
jobs:
build:
docker:
- image: circleci/elixir:1.5
- image: circleci/postgres:9.6
environment:
POSTGRES_USER: postgres
environment:
- APP_VERSION: "0.0.1"
working_directory: ~/repo
steps:
- checkout
# Setup
- run: cp config/dev.secret.example.exs config/dev.secret.exs
- run: mix local.hex --force
- run: mix local.rebar
# Testing
- run: mix deps.get
- run: mix test
# Release & Deploy
- run:
name: Install some System dependencies
command: sudo apt-get update -qq && sudo apt-get install -y ssh rsync
- run:
name: Install nodejs
command: |
curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -;
sudo apt-get update && sudo apt-get install -y nodejs;
- run:
name: Install yarn
command: |
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -;
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list;
sudo apt-get update && sudo apt-get install yarn;
- run:
name: Get Mix dependencies
command: MIX_ENV=prod mix deps.get
- run:
name: Compile App
command: MIX_ENV=prod mix compile
- run:
name: Install, compile, and digest Frontend
command: |
cd assets;
yarn;
npm run deploy;
cd ..;
MIX_ENV=prod mix phx.digest;
- run:
name: Build App release
command: MIX_ENV=prod mix release --env=prod;
- deploy:
name: Deploy to Production
command: |
ssh-keyscan -H $APP_HOST >> ~/.ssh/known_hosts;
rsync -ravhz --stats --progress ./_build/prod/rel/$APP_NAME $APP_USER@$APP_HOST:apps;
- deploy:
name: Post-deploy and App restart on Production
command: ./.circleci/post-deploy.sh
Please comment with any encouragement, feedback, or suggestions.