Another tool won’t fix your MLOps problems

There are 100+ MLOps tools
Thanks to the AI Infrastructure alliance for capturing the sprawl
  • 39 tools that help with monitoring or observability
  • 32 tools to help deploy models
  • 31 tools for experiment tracking
  • … (it doesn’t get better)
Percentages entirely based on vibes

MLOps has Tool Overload

In the last few years, I’ve worked with hundreds of companies trying to improve their ML infrastructure. Almost all of those teams are building towards a target architecture, something that looks like this:

Overview of the ML infrastructure components
Courtesy of a16z (Matt Bornstein, Jennifer Li, Martin Casado)
  • Lack of Buy-In: Many teams don’t have sufficient executive or practitioner buy-in, meaning it can be almost impossible to get the help you need to actually deliver products (cloud engineers, product engineers, etc.).
  • Confusion: Tools are used as a crutch for documentation and process. If every practitioner uses the same tools in different ways, confusion will ensue.
  • Alignment: Thanks to the first two problems, most practitioners still choose to build in silos. Data scientists in particular end up with isolated tools that make collaboration painful.

Learning from DevOps

The easiest way to shift culture is universal buy-in, in particular among decision-makers. The biggest barrier for MLOps is that most people don’t care yet!

  • For Practitioners: You understand ML and its potential — that makes it your job to be loud about the opportunities that ML presents to disrupt industries. You need to paint a clear picture about both opportunities in ML ( $$$ 📈) and the price of not acting (you lose to competition).
  • For early adopters: If you have already had measurable success, and your ability to rapidly develop ML systems has made your company measurably more successful, shout it from the rooftops! Write blogs, attend conferences, whatever! Your voice will raise the whole industry, and bring about the tide change we need to reorganize the culture of work in ML.
  • For Tool-Makers: If you’re making an MLOps tool, you cannot be (extremely) successful unless the culture comes along for the ride. If you simply build another MLOps tool, you’ll find yourself lost in the landscape. If you’re a loud voice that helps ML practitioners convince decision-makers that their culture needs to change, you will stick out from the crowd.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
David Hershey

David Hershey


Investor at Unusual Ventures| Machine Learning Infrastructure Enthusiast