Each interview we have done with guests has been unique and insightful. that being said since we are putting out so much content and there are so many new people joining the community every day I wanted to put together some of my favorite interviews as a ‘best of’ over the last year.

This was really hard to boil down to a list that didn’t go into the double digits and if there is enough interest I can make a part 2. But for now, I have compiled in no particular order the following list.

D. Sculley
Elizebeth Chabot
Todd Underwood
Lak
Lina Weichbrodt
Feature Store Panel — Mike, Willem and David
Laszlo Sragner
Luigi Patruno

Bonus sessions:

4 part google paper commentary
Neal Lathia

Give us a follow on Spotify or Youtube to stay up to date with weekly episodes.

The Godfather Of MLOps

If the name D. Sculley does not ring a bell that is because you, like me, have read his papers but never even realized it. Check out a list of all the papers he has contributed to here if you are still wondering who he is.

What’s Changed in MLOps?

Right before we hit record I let D. know how grateful we were for him to come on the pod and share his wisdom with us especially considering he hasn’t been doing many appearances as of late.

“I just got tired of talking about this stuff”

That’s right, the Godfather of MLOps has been talking about the need for attention on the Ops side of things since 2015. Now we are starting to understand him.

Like an oracle, he revealed to us a glimpse of the future then stepped out of the limelight to move on to the next hard problem. Meanwhile, we all continue to learn from his papers written years ago. We asked him, “what’s changed since he wrote these papers?” His answer, “well to start, now we have an MLOps community.”

This is easily the most influential person in MLOps that we have had the pleasure of interviewing. Aside from being an A+ human, he was a wealth of information. I cannot stress enough, go check out this interview.

All That Hype

I First spoke with Elizabeth Chabot at the beginning of 2020 while she was head of data science looking for tooling to complement their rapidly maturing data science infrastructure. In this conversation, Elizebeth drove home some incredible points. The ML reality of current-day small to medium size startups is not as it seems.

ML and AI may sound sexy to investors, but if you work in the field you’ve probably spent late nights reviewing outputs manually, poured over logs and ran root cause analyses until your eyes hurt. If you’ve created data products at a company where analytics and data science held no meaning before your arrival, you’ve probably spent many-a-late-night explaining the basics of data collection, why ETL cannot be half-baked and that when you create a supervised model it needs to be supervised.

Companies hoping to create a data product can have a data scientist show them how ML/AI can further their product, help them scale, or create better recommendations than their competitors. What companies are not always aware of is once the algorithm is created the data scientist is usually handicapped until more data-hires are made to build the necessary pipelines and frontend to put the algorithm in production. With the number of unique data-titles growing each year, how should the first data-evangelist-wrangler-wizard navigate title assignment?

Video here

A PM for MLOps

I am not sure how it happened or how I found him or how I even convinced him but we had none other than the head of SRE for ML infrastructure at Google come on to the coffee session and talk all things tech, managerial, and ethics around ML these days. (I couldn’t help asking him about the infamous email). Oh yeah, and I know there are a few out there that will be happy to know we got Vishnu to join us for this one. (Yes, he already has a fanclub)

Favorite quote from Todd: “Bad things happen to good data”

Couldn’t have been more right about that one. Two main points he made that kept me thinking and then stimulated more questions were: the idea of ML “trustworthiness”. Next, those of us interested in MLOps right now are either going to be the ones who write the books on it or are the first ones to read em!

This still is probably my most referenced guest. Todd’s ability to get his point across is unequivocal. I have misquoted him around just about every point he makes in this coffee session. Half of the quotes warrant their own dedicated podcast.

ML Design Patterns

It wouldn’t be a proper round-up if I didn’t include the two sessions from the authors of ML design patterns. Sara told us about the logistics of writing a technical book. Lak got down into the nitty-gritty of the design patterns.

Lak was one of the most entertaining people I have gotten to interview! Every sentence that came out of his mouth rang so true. It is obvious he is knee-deep in MLOps, all day, every day. Hats off to the man, he is trying to move the industry forward.

It’s one thing to work at Google, on Google size problems. Writing a book about those problems might not be useful for the rest of us. Lak broke down how each contributing writer brought something useful to the table. Sara talked about the extensive research they did before actually publishing the book.

Check out Lak’s conversation here and Sara’s here

When It Comes To Monitoring

Lina has become my go-to expert when it comes to anything monitoring. The depth of her knowledge has been more than helpful for me on various occasions. She earned her stripes at one of Europe’s biggest online eCommerce platforms Zalando. There she focused heavily on ML monitoring.

In this meetup, Lina shares hard learnings she had from trying to get models into prod. She reveals unexpected areas that bugs pop up in ML. A key takeaway here is that you may be monitoring and everything looks fine yet if you don’t know what to monitor for there may be a disaster happening behind the scenes you are utterly unaware of.

Full video here (Recommended due to slides)

Panel

I read recently 2021 would be the year of MLOps. For those of us in the community, it feels like 2021 has been the year of the feature store. We can hardly go a week without hearing news about them.

In this panel, I caught up with some heavy hitter in the space. Willem Pinaar is the creator of FEAST, Mike Del Balso is the CEO of Tecton, and David Aronchick is one of the creators of Kubeflow.

Greatest takeaway from the conversation was that we need some standards in the industry. Something that all the different tooling companies in this space can adhere to. As David puts it contracts with a small c, just enough standardization to play nice together.

The Greatest Slacker

Laszlo is legendary in the community slack. As it turns out, for good reasons too.

There are so many nuances when it comes to managing an ML project. ML dev cycles are a world apart from normal software dev cycles… or are they? I like that Laszlo says “get an end-to-end MVP as soon as possible”. Get something working and out then iterate on that.

To break with the Waterfall process you need an organizational structure that can attempt each part of the model lifecycle at the same time. This requires a rigorous version control of data and code what we call “Chain-of-Evidence” and ensemble modeling. You need a well-organized MLOps infra so you can focus on the above rather than struggle with low-level issues.

It’s not so easy to find PMs who have had experience in ML before, which begs the question, is it a prerequisite? Laszlo summed it up nicely, and another answer came from Korri in the chat “I think there is a place for product management for those that have been engineers/data scientists. However, there is the capability of managing an ML product, but then there is the case of having a ‘VISION’, which is an existing gap across”

Luigi In Production

Another with legendary status in the MLOps world. Luigi used to send out an email newsletter seen by thousands until recently he stepped away to spend more time with his family.

We bounced all over the place. Luigi has so much experience we wanted to try and get the most out of it.

We sourced the questions from the community. Sadly we didnt get through all of them. So what questions did we ask?

1. Who does he learn from? Favorite resources?
2. Any companies that stand out in terms of MLOps excellence?
3. Dos and don’t of MLOps
4. What is his process of identifying use cases that are suitable for machine learning as a solution? How do they proceed methodically?
5. What part of the ML in Production process do people underestimate the most? What are the low hanging fruits that many people don’t take advantage of?
6. How has he seen ML in production evolve over the last few years and where does he think it’s headed next?

My Favorite quote from him was around automation and it’s something that David and I have been discussing at length in the series around the google paper continuous delivery for ML. Luigi said:

“Do it manually first til you feel confident that you can automate it, then automate so you avoid unwanted human errors”

Bonus

Well if you made it this far you are dedicated. So here is a 4 part series and commentary we did around the google paper for continuous training.

An who could forget the amazing session we had with Neal Lathia from Monzo Bank. I tend to recycle many of his quotes from this session. A must listen for anyone in fintech or finance!