GitHub Workflow Tips

GitHub Workflow Tips

Some tips on GitHub workflows that I’ve picked up.

Make Room for Things

To make room from the default “runner” so that I can install other things, add something like this step:

    steps:
    - name: Remove unnecessary files
      run: |
        df -h /
        # Remove software and language runtimes we're not using
        sudo rm -rf \
          "$AGENT_TOOLSDIRECTORY" \
          /opt/google/chrome \
          /opt/microsoft/msedge \
          /opt/microsoft/powershell \
          /opt/pipx \
          /usr/lib/mono \
          /usr/local/julia* \
          /usr/local/lib/android \
          /usr/local/lib/node_modules \
          /usr/local/share/chromium \
          /usr/local/share/powershell \
          /usr/share/dotnet \
          /usr/share/swift
        df -h /
      if: runner.os == 'Linux'

This looks like just a shell script to remove a bunch of things under /opt/ and /usr/local/. Of course, add or remove items as needed. However, the trick is to know what exactly comes with the runner’s image (and where) so I can add it to the list.

Start here: https://docs.github.com/en/actions/concepts/runners/github-hosted-runners?ref=techblog#preinstalled-software-for-github-owned-images. It says to look at the logs of a previously run workflow under:

  • Set up job
    • Runner Image
      • Included Software: <some URL>

As of this time, the URL link in my workflow is: https://github.com/actions/runner-images/blob/ubuntu24/20260323.65/images/ubuntu/Ubuntu2404-Readme.md. It lists a bunch of stuff well enough. However, where exactly are they installed to so that I can rm -rf them?

The best place I can find that hints at this is the “.../scripts/build/” folder in the same repo: https://github.com/actions/runner-images/tree/ubuntu24/20260323.65/images/ubuntu/scripts/build. Each script that installs something has hints in there of where the software may be placed.

Add a DB (PostgreSQL) Locally

Most of my Django repos worked fine with the default SQLite DB setup. However, I recently worked on some repos that are specific to PostgreSQL. Specifically, I am working with the vector extension of PostgreSQL that allows me to store vectors into the DB. This requires my environments to be set up with a PostgreSQL DB.

Initially this broke my GitHub workflow that runs unit tests because the runner doesn’t have installed or access to an instance of PostgreSQL. One way to get around this is to have an instance somewhere that the runner can connect to. However, that just opens up a can of worms w/r to security, costs, maintenance, etc.

Another way is to actually install a copy of PostgreSQL into the runner during workflow before running the unit tests. Similar to how docker compose’s services work, I can add a PostgreSQL service:

    services:
      postgres:
        image: pgvector/pgvector:pg18
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
        options: >-
          --health-cmd="pg_isready -U postgres"
          --health-interval=10s
          --health-timeout=5s
          --health-retries=5
    steps:
        ...

In fact, I think that image: pgvector/pgvector:pg18 is exactly referring to the Docker image for pgvector, the extension for PostgreSQL that enables use of vectors. The various settings should be self-explanatory for PostgreSQL usage.

The last thing to note about this is that the DB will be running on localhost (not a magical postgres as done in docker compose).

For example, the step that runs pytest for unit tests using the DB created above would be:

    - name: Run Tests
      env:
        DATABASE_URL: postgres://postgres:postgres@localhost:5432/test_db
      run: |
        uv run pytest -v -s

By the way, the command above works because the environment variable DATABASE_URL is used in the settings to pass to dj_database_url.config(). If that is not used, then a group of environment variables for the user, password, host (localhost in this case), DB name, port, etc. should be used.

Tags: , ,