Skip to content

CI and GitOps

Before setting up CI/CD and GitOps for your project, make sure you complete all steps in the "Deployment from GitHub, Bitbucket, GitLab" chapter and verify that your deployed project works.

You can find many third-party services that help set up CI/CD. However, in most cases, it's easier and faster to set up CI/CD without external services. This avoids vendor lock-in, reduces deployment time significantly, and provides more predictable CI/CD costs. This chapter will guide you in setting up CI/CD on your own cloud server.

How CI/CD Works

  • You push code to a remote Git repository (e.g., GitHub).
  • GitHub (or another Git service) sends a notification (or webhook) to a server that the repository has been updated.
  • A service on the server clones the repository, starts the app from the cloned repository, and runs tests if necessary.

It sounds simple, right? Now, let’s implement this process for the project we deployed in "Deployment from GitHub, Bitbucket, GitLab." The goal is to run a service that handles notifications from a remote Git repository, clones the repository, and starts a container (if you have a Dockerfile in the project repository) or runs a start.sh script if you're using systemd to manage apps/services on the server.

How to Set Up CI/CD and GitOps

Prerequisites

  • A domain or subdomain with an A record pointing to your server's IPv4 address.

Steps

There are a few options for handling webhooks (or notifications) from services like GitHub and automating tasks like cloning a repository and starting an app/service:

  • Develop a custom web server.
  • Use a proxy server.

We'll use the second option—handling webhooks with Caddy (a proxy server we use)—because it eliminates the need for writing code, managing additional tools, or updating them.

1. SSH into the Server

ssh root@server_ip

2. Install Caddy and the Required Module for Running Commands

# Install Go
sudo wget https://go.dev/dl/go1.22.6.linux-amd64.tar.gz
rm -rf /usr/local/go && tar -C /usr/local -xzf go1.22.6.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin

# Install Caddy
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-xcaddy-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-xcaddy.list
sudo apt update
sudo apt install xcaddy

# Build Caddy with the exec module
xcaddy build --with github.com/abiosoft/caddy-exec
sudo mv caddy /usr/bin/

# Set up the Caddy user and group
sudo groupadd --system caddy
sudo useradd --system --gid caddy --create-home --home-dir /var/lib/caddy --shell /usr/sbin/nologin --comment "Caddy web server" caddy
wget https://raw.githubusercontent.com/caddyserver/dist/master/init/caddy.service -O /etc/systemd/system/caddy.service

3. Create a Deployment Script

This script will be triggered by Caddy when a webhook event is received. It will clone the repository, start or restart the app/service, and update the Caddy configuration if necessary. Create the script named deploy.sh and place it in the home directory:

cat > ~/deploy.sh <<EOL
#!/bin/sh

git clone repository_url demo_app
cd demo_app
./start.sh
EOL

4. Add a Webhook Route to Caddy

Configure Caddy to run the deploy.sh script when a GitHub/Bitbucket/GitLab webhook event is received. To provide an additional layer of security, add a random string to the webhook URL (e.g., "aw523fsk129q"). The final URL will be something like https://deploy.turbocloud.dev/aw523fsk129q.

Update the Caddy configuration as follows:

sudo echo -e "deploy.turbocloud.dev {\n reverse_proxy /aw523fsk129q {\n    exec ./deploy.sh --destination=~ \n}\n}" >> /etc/caddy/Caddyfile
caddy reload -c /etc/caddy/Caddyfile

5. Set Up Webhooks in the Git Repository

Now, add the webhook URL in the settings of your GitHub, Bitbucket, or GitLab repository. Go to the repository settings, navigate to the "Webhooks" tab, and add the URL from the previous step (e.g., https://deploy.turbocloud.dev/aw523fsk129q).

Great! Your CI pipeline is ready. Test it by pushing changes to the remote Git repository, then check if the new version of your app/service is deployed successfully by opening it in the browser.