As an MLOps Engineer, I’m always on the lookout for tools and technologies that can streamline workflows, boost performance, and enhance the maintainability of MLOps projects. Recently, I made a significant shift in the MLOps Python Package — a repository dedicated to sharing best practices for MLOps in Python. I transitioned from Poetry, a popular dependency management and packaging tool, to Uv, a newer, blazing-fast alternative.

_{Photo by Chris Briggs on Unsplash}

This wasn’t a decision I took lightly. Poetry has served my teammates and me well on many MLOps projects, but Uv’s promise of enhanced speed and stricter adherence to Python Enhancement Proposals (PEPs) piqued our interest. After thoroughly testing it out, we were convinced. The transition is a resounding success, and we’re excited to share our experience and why we believe Uv might be the right choice for your AI/ML projects too.

The Need for Speed (and PEPs)

The “MLOps Python Package” is a side project, but it reflects the best practices defined in the organizations I’m working with. As our MLOps projects grew, we started noticing that dependency resolution and installation with Poetry were becoming increasingly sluggish. This was particularly noticeable in our CI/CD pipelines with runs often exceeding 25 minutes. Moreover, while Poetry is a powerful tool, it sometimes requires its own configurations, deviating from standard PEP practices.

Enter Uv. This new tool, developed by Astral, promised to be a drop-in replacement for pip, pip-tools, pipx, poetry, pyenv, twine, and virtualenv, boasting significant speed improvements thanks to its Rust foundation. It also claimed strict compliance with PEPs, which meant a more standardized and potentially less complex project structure.

The Transition: Smooth Sailing with Uv

Migrating from Poetry to Uv was surprisingly straightforward. The most interesting part was adapting our pyproject.toml, CI/CD workflows (GitHub Actions in our case), development tasks and Dockerfile.

A Cleaner, More Compliant pyproject.toml

The move to Uv brought welcome changes to our pyproject.toml file. We were able to remove the Poetry-specific sections, like [tool.poetry] and its associated configurations:

Now we could rely solely on standards-compliant sections like [project] and [project.optional-dependencies]:

GitHub Actions: Setup

Previously, our setup action involved installing pipx, invoke, and poetry:

With Uv we simply need to adopt a new GitHub Action:

The astral-sh/setup-uv action made installing Uv incredibly simple, and we could rely on the action to specify the python version in the .python-version file.

GitHub Actions: Checks and Publishing

Our check workflow also saw few changes. Instead of running a single checker step with Poetry:

We now have separate checkers with Uv to improve speed and debugging:

Similarly, in our publishing workflow, poetry install --with docs and poetry run invoke docs became uv sync --group=docs and uv run invoke docs, respectively. These changes highlight Uv's ability to seamlessly integrate into existing workflows while being faster than Poetry.

Development Tasks

Our pyinvoke tasks, defined in the tasks/ directory, also required only minor changes. For instance, our installation and format task went from:

With Uv it becomes:

Slimmer, Faster Docker Builds with Uv

The benefits of Uv extends into our Docker builds as well. Our initial Dockerfile relied on installing the package wheel using pip:

This was functional but not optimal. With Uv, we were able to switch to a more streamlined approach, leveraging Uv’s speed within the Docker build process. We also switched to the official uv image maintained by Astral:

The key change here is using uv pip install --system *.whl instead of pip install *.whl. By using the --system flag, we ensure that the package and its dependencies are installed into the system's site packages directory, making for a cleaner environment. This, combined with Uv's installation speed, resulted in significantly faster Docker builds.

The Results: Speed, Simplicity, and Standards

The impact of switching to Uv was immediately noticeable. Our CI/CD pipelines became significantly faster, thanks to Uv’s rapid dependency resolution and installation. Locally, development felt snappier too.

But it wasn’t just about speed. Uv’s strict adherence to PEP standards meant we could simplify our project configuration. We no longer needed a separate poetry.toml file. Everything was managed within the standard pyproject.toml, making our project more aligned with the broader Python ecosystem.

Why You Might Want to Consider Uv

Our experience with Uv has been overwhelmingly positive. If you’re working on an MLOps project (or any Python project, for that matter) and value speed, simplicity, and adherence to standards, we highly recommend giving Uv a try. Here’s a quick recap of the benefits we observed:

1. Blazing Fast: Uv’s Rust-based architecture delivers significant speed improvements over Poetry.
2. PEP Compliance: Strict adherence to PEPs means a more standardized and potentially less complex project structure.
3. Seamless Integration: Uv integrates smoothly into existing workflows, often simplifying them in the process.
4. Drop-in Replacement: Uv can replace pip, pip-tools, pipx, poetry, pyenv, twine, virtualenv, and more, making it a versatile tool for various project needs.

Conclusion: Embracing the Future of Python Dependency Management

The transition from Poetry to Uv in our “MLOps Python Package” project has been a rewarding experience. It’s a testament to the evolving landscape of Python tooling and the benefits of embracing new technologies that align with our goals of efficiency, maintainability, and community standards.

We encourage you to explore Uv and see if it’s the right fit for your projects. The future of Python dependency management is here, and it’s faster and more standardized than ever before!

_{Photo by Harley-Davidson on Unsplash}

Originally posted at:

https://fmind.medium.com/poetry-was-good-uv-is-better-an-mlops-migration-story-f52bf0c6c703