Fast Docker Builds in GitHub Actions with Compose files

The Situation

I've recently been creating some new GitHub Actions workflows for running integration tests. This involved spinning up several Docker containers all configured to talk to each other.

Handily each of the services I needed to spin up had a docker-compose.yml file already defined that defined how to start the relevant database and application containers.

First Attempt (Slow)

I started with what was working well locally: just running `docker build .` in each of the service directories, and then bring everything up with `docker compose ...`. This was always fast, particularly if the build was mostly cached.

cd service-one
docker build .
docker compose -f docker-compose.yml up

I believe this worked well locally because after I built locally on my MacBook Docker would automatically load the built image into the Docker service and cache the Docker image layers, and then when `docker compose` came to rebuild the image it would note that everything was cached already and just start bringing containers online.

However when I tried this in GitHub Action steps, just using Bash commands, I found that `docker build .` would have to do a full (slow) rebuild from scratch (not unexpected), but then `docker compose ...` would then do another full (slow) rebuild!

I tried several variations with trying to force the initial build to load the image into the Docker service, but there wasn't much documentation on getting this working in GitHub Actions - so I pivoted to what seemed to be the most common path.

Second Attempt (Still Slow)

GitHub's own documentation all points towards using the `docker/build-push` action. This has handy flags to automatically `load` the image into the Docker service so they should be immediately available to use after a build.

I also started using `actions/cache` to store the generated image and image layers between GitHub Actions runs. This means that most of my Dockerfile's steps wouldn't be repeated between builds.

Unfortunately this caching only ever really hit a 50% layer cache rate, even on a build that should have been 100% cached because nothing changed between runs.

Also `docker compose` still wasn't effectively re-using the image that was just explicitly built - it would often rebuild it again!

Making It Fast

I then discovered the `docker/bake` action, which allows you to just tell Docker to build the images required for a given compose file, and then load them.

I thought this might help reduce any discrepancies between the context of `docker build` and `docker compose`, and indeed it did!

Caching was now working as expected! It would still take a couple of minutes to load in larger images (around 1GB) but it was much better than waiting 15+ minutes for full rebuilds.

Here's how the GitHub caching is set up:

jobs:
  run-integ-tests:
    runs-on: ubuntu-latest
    steps:
      
      - name: "Set up Docker cache (integ-tests)"
        id: integ-tests-cache
        uses: actions/cache@v4
        with:
          path: /tmp/.cache-integ-tests
          key: integ-tests-${{ hashFiles('**/Dockerfile*', '**/*.lock', '**/package-lock.json', '**/requirements.txt') }}
          restore-keys: |
            integ-tests-

And how we tell Docker to build the images it's going to need:

      - name: "Set up Docker Buildx"
        if: ${{ !env.ACT }}
        id: buildx
        uses: docker/setup-buildx-action@v3
        with:
          driver: docker-container
          
      - name: "Build Integ Test Image (using cache)"
        uses: docker/bake-action@v5
        with:
          workdir: .
          source: .
          files: docker-compose.yml
          load: true
          builder: ${{ steps.buildx.outputs.name }}
          # Don't spend time writing cache if GH won't write it because it was a cache-hit
          set: |
            *.cache-from=type=local,src=/tmp/.cache-integ-tests
            ${{ (steps.integ-tests-cache.outputs.cache-hit != 'true' && '*.cache-to=type=local,dest=/tmp/.cache-integ-tests-new,mode=max') || '' }}

And then starting services (or running integration tests) with `docker compose`:

      - name: "Run End-To-End Test"
        run: |
          # Won't rebuild, because any/all Docker images are already loaded into Docker!
          docker compose up --exit-code-from integ-tests --abort-on-container-exit

There's a final bit of logic to keep cache sizes small:

      - name: "Move Docker Caches"
        # This step keeps cache sizes small, as Docker won't remove old layers
        # if we just always append new data to the cache directory.
        run: |
          # Delete old caches
          rm -r /tmp/.cache-integ-tests || true
          # Move any new cache data into location where they will be uploaded
          mv /tmp/.cache-integ-tests-new /tmp/.cache-integ-tests || true

      # Note: There is an implicit post step here to upload the contents of the /tmp/.cache-integ-tests dir if it was not a perfect cache hit.

Hopefully this is useful to someone in the future who's wondering how to get Docker image caching working with `docker compose`!