Staging server for Discourse hosting

Documentation
1

I’m working with a client who is currently hosted on Discourse’s Business plan and I’d like to show what the Subscription feature is like. Having already broken their production site once I decided to set up a staging server instead. This guide is heavily informed by:

Set up a new Droplet

I won’t go into too much detail because I’m planning on writing detailed documentation for how to set this up. But the gist is that I create a new project on DigitalOcean for the customer and create a Droplet. For Discourse, the minimum size[1] is:

  • 1 vCPU
  • 2GB / 50GB Disk

Right now that costs $12 a month. It’s helpful to be able to destroy the site when you are done with it and automate the setup for later. Not only does that save money, it also means you can get a clean staging environment when needed. I’ve been burnt by not rebuilding staging and having a forgotten change mess up my testing.

Ansible playbook for installing Discourse

I’m automating my process using Ansible and I found a little trick to specify hosts on the command line:

- name: Install Discourse on hosts given by the option --limit
  hosts: '{{ ansible_limit | default(omit) }}'

This way I can add a new test server to my inventory and test it out immediately with:

$ ansible-playbook test_discourse.yml -l test.example.com -i inventory.yml

Clone the Discourse Docker image

Since I always want to have both Git and Docker, I include those in the Ansible playbook I use for setting up new servers. The playbook for installing Discourse in particular starts with cloning the Discourse image.[2]

  tasks:
    - name: Clone the Discourse repo
      ansible.builtin.git:
        repo: 'https://github.com/discourse/discourse_docker.git'
        dest: /var/discourse

    - name: Container permission
      ansible.builtin.file:
        path: /var/discourse/containers
        state: directory
        mode: '0700'

This requires root access. If I’m not using the root account, I can add --become to the command line. Another approach is to add become: true to the task. Indentation is the key in YAML files. This key-value pair must nested be under - name: not under the ansible.builtin command.

Kernel configuration

I’ve taken this step not from the standard installation instructions, but from MKJ’s Opinionated Discourse Deployment Configuration. Instead of issuing shell commands, I’d planned on using the ansible.posix.sysctl module, which takes care of everything including the reload step. This works well enough for the vm.overcommit_memory setting:

    - name: Memory setting for Discourse
      ansible.posix.sysctl:
        sysctl_file: /etc/sysctl.d/90-vm_overcommit_memory.conf
        name: vm.overcommit_memory
        value: 1

Ubuntu handles disabling transparent huge pages a bit differently, however. I’m using ansible.builtin.apt to install sysfsutils and ansible.builtin.lineinfile to add the configuration to /etc/sysfs.conf. Finally I’m rebooting with ansible.builtin.reboot. Is this how it’s supposed to work? Honestly I’m not sure.

    # https://askubuntu.com/questions/597372/how-do-i-modify-sys-kernel-mm-transparent-hugepage-enabled/610707#610707
    - name: Install sysfsutils
      ansible.builtin.apt:
        name: sysfsutils
        state: present

    - name: Kernel setting for Redis
      ansible.builtin.lineinfile:
        path: /etc/sysfs.conf
        line: kernel/mm/transparent_hugepage/enabled = never

Discourse setup script

Next I call discourse-setup, but I’m using --skip-rebuild so that it just generates the container file and doesn’t actually build it yet. I’m using ansible.builtin.expect to answer the interactive questions.[3]

    - name: Call discourse-setup
      ansible.builtin.expect:
        command: sudo /var/discourse/discourse-setup --skip-rebuild
        responses:
          Hostname for your Discourse?: "{{ ansible_host }}"
          Email address for admin account(s)?: jon@buildcivitas.com
          SMTP server address?: "{{ lookup('env', 'MAILJET_SMTP') }}"
          SMTP port?: 587
          SMTP user name?: "{{ lookup('env', 'MAILJET_USER_NAME') }}"
          SMTP password?: "{{ lookup('env', 'MAILJET_PASSWORD') }}"
          Let's Encrypt account email?: ''
          Optional Maxmind License key: ''
          notification email address?: noreply@buildcivitas.com
          Optional email address for Let's Encrypt warnings?: ''
          ENTER to continue: ''
        timeout: 7200 # Oracle Free Tier instances are slow
        creates: /var/discourse/containers/app.yml

I’m saving all the secrets in my environment and looking them up using "{{ lookup('env', 'SECRET_ENV_VAR') }}". In the future I might release these scripts in a public repository, so I don’t want other people trying to use my mail server.

I have another playbook for installing the Civitas architecture that uses ansible.builtin.template to insert variable values into my container files. It’s a cleaner solution, but I also want to be able to test the standard install that’s likely closer to what my clients are using.

Add staging configuration

Next I update app.yml with configurations suitable for a staging server using ansible.builtin.blockinfile:

    - name: Insert staging config
      ansible.builtin.blockinfile:
        path: /var/discourse/containers/app.yml
        append_newline: true
        prepend_newline: true
        marker: '# {mark} Staging configuration -->'
        insertafter: 1234567890123456
        block: "{{ lookup('ansible.builtin.file', '../discourse/staging.yml') }}"

The actual block comes from a file I’m calling staging.yml that includes the actual settings:

  ## Staging server specific settings
  DISCOURSE_AUTOMATIC_BACKUPS_ENABLED: false
  DISCOURSE_LOGIN_REQUIRED: true
  DISCOURSE_DISABLE_EMAILS: 'non-staff'
  DISCOURSE_S3_DISABLE_CLEANUP: true
  DISCOURSE_ALLOW_RESTORE: true

These settings are designed to keep the server private (especially from search engines) and not overwrite backups if I’m sharing an S3 bucket.

Insert plugins

Since my client is on Discourse’s Business plan, I also need to make sure I have the same set of plugins. I started by getting a complete list of official plugins and editing it down to just the Business plan set. I put that block in a file called business.yml. Again, ansible.builtin.blockinfile puts this into the container file:

    - name: Insert "Business plan" plugins
      ansible.builtin.blockinfile:
        path: /var/discourse/containers/app.yml
        marker: '# {mark} Business Plugins -->'
        insertafter:  '          - git clone https://github.com/discourse/docker_manager.git'
        block: "{{ lookup('ansible.builtin.file', '../discourse/business.yml') }}"

Start the server

After all that, we should be ready to bootstrap and start the server using the command module:

    - name: Bootstrap the Discourse container
      ansible.builtin.command: sudo /var/discourse/launcher bootstrap app

    - name: Start the Discourse container
      ansible.builtin.command: sudo /var/discourse/launcher start app

  1. Officially the limit is 1GB, but the site is noticeably slow. For really simple tests, that might be ok, but not for showing to a client. ↩︎

  2. RIght at the moment, I’m cloning my fork of the image which has this fix for smaller servers. ↩︎

  3. The other option would be to skip the setup script and use a template to build the container file. That’s what I do with another script I use to install a test server using the Civitas architecture. ↩︎