Configuring /etc/hosts and Caddy Reverse Proxy for Local Development in Ansible

Files for this project are located https://github.com/branhamt/proxy-local-dev.

It’s pretty handy to setup a domain on your local machine that can point to a local server/port combo for local development. Instead of typing localhost:8080 in your address bar, you can type somesite.lcl, or some other domain name you’ve chosen.

This is really handy when you don’t want to have to remember port numbers for all the things running on your machine. Since it’s a multi-part process, it can be a bit of a chore to set it up. That makes it perfect for automation!

In this article, I’ll show how you can:

  • edit /etc/hosts
  • install and run Caddy webserver on Debian-flavored Linux
  • configure Caddy as a reverse proxy
  • run sites with or without SSL
  • and do it all in an Ansible playbook

This setup assumes that there is another server (like a dev server from your stack) serving up a website on a port on your machine. Of course, you can run as many servers and serve as many sites locally as you want.

There is a little streamlining that can be done to the Ansible role file, but I won’t update it until it starts feeling more clunky and ungrokkable.

Set Up /etc/hosts

/etc/hosts lets you map domain names to IP addresses on your machine. This is useful for blocking websites, or redirecting them to other sites. We’re going to map our sites in /etc/hosts like this:

127.0.0.1     somesite1.lcl
127.0.0.1     somesite2.lcl
127.0.0.1     somesite3.lcl

Since /etc/hosts doesn’t handle ports, we don’t include them here. All of these domains will be “forwarded” to our local webserver, and that webserver will decide where to direct a request for each domain, based on the port.

Set Up Caddy Webserver

You could use nginx, Apache, or any other webserver, but I’m going to use Caddy for it’s easy setup.

Caddy is really just proxying to a development server, for my use cases, such as:

  • the Django webserver
  • Node server
  • Hugo webserver

I’m using a Caddyfile, which has very simple setup for a reverse proxy. For a reverse proxy to a site on your machine, specify the domain and port, like this:

somesite1.lcl {
  reverse_proxy :8080
  tls internal

Once your /etc/hosts is set up, this will cause your browser to load https://somesite1.lcl, served from your machine on port 8080, once you type it in your address bar.

Caddy uses it’s own SSL set up on your machine, so we need to specify this with tls internal. You’ll probably need to allow your browser to connect since it will inherently not trust Caddy’s local SSL setup.

Caddy with no SSL

Note that Caddy will configure SSL by default, meaning that your local setup will be served from an address like https://somesite.lcl. This could cause problems depending on what you’re serving, but there is an alternative: no SSL.

In Caddy, you can specify no ssl with this config:

http://somesite2.lcl {
  reverse_proxy :8081

This site will be served on port 8081 from your machine at the address http://somesite2.lcl.

Automating with Ansible

I wanted to be able to automate the above process with one file edit and one command. Let’s start with variables:

Variables

caddy_service_running: true
manage_dns_reverse_proxies: true
dns_and_reverse_proxies:
- { domain: "somesite1.lcl", present: true, caddy_served: true, port: 1313, ssl: true }
- { domain: "somesite2.lcl", present: true, caddy_served: false, port: 8081, ssl: false }
- { domain: "somesite3.lcl", present: true, caddy_served: true, port: 8082, ssl: false }

caddy_service_running controls whether the Caddy webserver runs. This makes it easy-ish to turn off, and lives alongside some other settings that turn off other services.

manage_dns_reverse_proxies controls whether entries appear in /etc/hosts and the Caddyfile. If this is false, all entries are removed.

dns_and_reverse_proxies is self-explanatory.

Caddy Installation

We start with installing Caddy. The Caddy installation instructions look like this:

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy

I translated these instructions into Ansible YAML here:

- name: Install Caddy Server
  block:
    - name: Update apt cache
      ansible.builtin.apt:
        update_cache: true

    - name: Install utilities
      ansible.builtin.apt:
        pkg:
          - ca-certificates
          - gnupg

    - name: Install Caddy signing key
      ansible.builtin.apt_key:
        url: https://dl.cloudsmith.io/public/caddy/stable/gpg.key
        keyring: /usr/share/keyrings/caddy-stable-archive-keyring.gpg
        state: present

    - name: Add Caddy "deb" to sources
      ansible.builtin.blockinfile:
        path: /etc/apt/sources.list.d/caddy-stable.list
        create: true
        block: |
          # Source: Caddy
          # Site: https://github.com/caddyserver/caddy
          # Repository: Caddy / stable
          # Description: Fast, multi-platform web server with automatic HTTPS
          deb [signed-by=/usr/share/keyrings/caddy-stable-archive-keyring.gpg] https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main
          deb-src [signed-by=/usr/share/keyrings/caddy-stable-archive-keyring.gpg] https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main          

    - name: Update apt cache
      ansible.builtin.apt:
        update_cache: true

    - name: Install Caddy
      ansible.builtin.apt:
        pkg:
          - caddy

Control the Caddy Systemd Service

To control whether the service runs, I use this Ansible code:

- name: Turn on/off Caddy services
  vars:
    services:
      - caddy
  block:
    - name: Enable caddy services
      ansible.builtin.systemd:
        name: "{{ item }}"
        enabled: true
      when: caddy_service_running
      loop: "{{ services }}"

    - name: Turn on caddy services
      ansible.builtin.systemd:
        name: "{{ item }}"
        state: started
      when: caddy_service_running
      loop: "{{ services }}"

    - name: Disable caddy services
      ansible.builtin.systemd:
        name: "{{ item }}"
        enabled: false
      when: not caddy_service_running
      loop: "{{ services }}"

    - name: Turn off caddy services
      ansible.builtin.systemd:
        name: "{{ item }}"
        state: stopped
      when: not caddy_service_running
      loop: "{{ services }}"
  tags: update-local-dns-proxy

I’m using the tag update-local-dns-proxy so that I can run a small subset of the role when I make an update to the variables.

Update /etc/hosts and the Caddyfile

This section is a bit more complicated because of the number of variables. We add or remove sites based on the present variable, add SSL if required, and reload Caddy at the end.

- name: Configure reverse proxies
  block:
    - name: Add host to /etc/hosts
      ansible.builtin.lineinfile:
        path: /etc/hosts
        line: "127.0.0.1      {{ item.domain }}"
        state: present
      loop: "{{ dns_and_reverse_proxies }}"

    - name: Add proxy to Caddyfile
      ansible.builtin.blockinfile:
        path: /etc/caddy/Caddyfile
        state: present
        block: |
          {% if not item.ssl %}http://{% endif %}{{ item.domain }} {
            reverse_proxy :{{ item.port }}
            {% if item.ssl %}
            tls internal
            {% endif %}
          }          
        marker: "# {mark} {{ item.domain }}:{{ item.port }} ANSIBLE MANAGED BLOCK"
      when: item.caddy_served
      loop: "{{ dns_and_reverse_proxies }}"
  when: manage_dns_reverse_proxies and item.present
  tags: update-local-dns-proxy

- name: Remove reverse proxies
  block:
    - name: Remove host from /etc/hosts
      ansible.builtin.lineinfile:
        path: /etc/hosts
        line: "127.0.0.1      {{ item.domain }}"
        state: absent
      loop: "{{ dns_and_reverse_proxies }}"

    - name: Remove proxy from Caddyfile
      ansible.builtin.blockinfile:
        path: /etc/caddy/Caddyfile
        state: absent
        block: |
          {% if not item.ssl %}http://{% endif %}{{ item.domain }} {
            reverse_proxy :{{ item.port }}
            {% if item.ssl %}
            tls internal
            {% endif %}
          }          
        marker: "# {mark} {{ item.domain }}:{{ item.port }} ANSIBLE MANAGED BLOCK"
      loop: "{{ dns_and_reverse_proxies }}"
  become: true
  when: not manage_dns_reverse_proxies or not item.present
  tags: update-local-dns-proxy

- name: Reload Caddy
  ansible.builtin.systemd:
    name: caddy
    state: reloaded
  # become: true
  tags: update-local-dns-proxy

Commands

To update what sites are being reverse proxied (or only added to /etc/hosts) I run:

ansible-playbook x270.yml --ask-become-pass --tags=update-local-dns-proxy

This is the playbook for my local machine, but the tag will cause only the tagged tasks above to run.

This setup does require root to run the service and modify files.

Bash Alias

Finally, to make things a little easier, I added a function in my bash aliases:

function proxy-dns () {
    venv ansible; # alias to activate the ansible virtualenv
    cd ~/workstation/ansible || exit;
    ansible-playbook x270.yml --ask-become-pass --tags=update-local-dns-proxy
    cd - || exit;
    deactivate
}

This is pretty self-explanatory. I’m using Ansible in a virtualenv, so that is activated (by an alias) and deactivated at the end. I’m sure I’ll forget this command!

.dev TLD Issues

I thought I might, maybe, have some problems with the .dev TLD, based on articles like this.

Using Firefox, some sites with a .dev domain don’t appear to be loading correctly. I may switch to only using .lcl.