Eric J Ma's Website

Nobody likes waiting around for continuous integration (CI) pipelines to finish. I'm sure many of you can relate to the frustration of slow build times. In my recent post, How to choose a (conda) distribution of Python, I touched upon Python distributions. However, a comment on LinkedIn by Wade Rosko, referencing Hugo Shi's post about speeding up a Saturn Cloud CI job with micromamba, got me thinking. They achieved a 40X speedup! This was something I had to try.

The Experiment

I decided to use LlamaBot's CI system as a test case. The original setup used Miniconda with the following YAML configuration:

- name: Setup miniconda
  if: matrix.environment-type == 'miniconda'
  uses: conda-incubator/setup-miniconda@v2
  with:
    auto-update-conda: true
    miniforge-variant: Mambaforge
    channels: conda-forge
    activate-environment: llamabot
    environment-file: environment.yml
    use-mamba: true
    python-version: ${{ matrix.python-version }}

In my new approach, I switched to using micromamba with the following YAML:

- uses: mamba-org/setup-micromamba@v1
  if: matrix.environment-type == 'miniconda'
  with:
    micromamba-version: '1.4.5-0'
    environment-file: environment.yml
    init-shell: bash
    cache-environment: true
    cache-environment-key: environment-${{ steps.date.outputs.date }}
    cache-downloads-key: downloads-${{ steps.date.outputs.date }}
    post-cleanup: 'all'

The rest of the YAML file remained unchanged.

Benchmarking Results

I meticulously recorded the timings, and you can find the full record here.

Analysis and Conclusion

The timings on my latest PRs with the new setup feel more consistent. Not only is there a noticeable reduction in build and test times, averaging around 1 minute, but there's also no difference in the final environment, and all tests pass.

The primary win here comes from the built-in, turnkey caching of the entire environment, a stark contrast to the more complicated caching methods suggested by mambaforge. Opting for caching based on the date seems to be a practical compromise, especially since I often test against bleeding-edge packages.

Wrapping Up

Switching to micromamba for the LlamaBot's CI system was a rewarding experience. It's a straightforward and effective way to reduce CI times significantly. If you're dealing with similar CI delays, consider giving micromamba a try. It could just be the solution you're looking for.