You spoke, I listened

Over the last month, I sent out over (no joke) 1k DMs to people in the MLOps community asking for feedback on how we can be better. After receiving 300+ responses I thought I would try and address some of the main themes that are repeatedly coming up. If you haven’t shared your opinion with me yet, don’t worry it’s not too late! You can do so here.

We have lots going on in this community. My main learning from the feedback is that most people don’t know about everything we do.

Solution: I created and continuously update the MLOps Community Orientation doc. This gives a clear list of everything we do on one page. I realize now this page needs to be more predominant. I am working on making something like this our landing page so that it is more clear for everyone.

Insights, but first…

Ok so let’s get started with the nitty-gritty insights from all the data. Before we do, I gotta say something. Next time I take on a task like this, I am going to get an NLP model to go through the data. Talk about needing thick skin! I mean I guess I asked for it but… hot damn I had such an emotional rollercoaster while analyzing all this data.

I was touched deeply by so many kind words from everyone. I also realize now that gracefully receiving feedback is an art form. The kind that comes with deep self-mastery. Some of the responses were really hard for me to hear. The truth hurts! Some responses were hard not to instantly disregard as ridiculous. “They don’t know what they are talking about!” I would shout.

One thing is for sure, for the last month this has consumed me. I would log onto my computer, scroll endlessly through the responses new and old trying to squeeze out any patterns. I would close my laptop after seeing responses that triggered me. My constant self-talk was to say they don’t know what they are talking about. Yet, I couldn’t get away. Soon enough I would be back looking at those responses, trying to parse out if there was some truth in those words. Eventually, I came to each piece of feedback with an open mind about what can be done better.

I couldn’t stop thinking about all this feedback, morning til night. What action items could I take away from it? All this new information I had was great but what was I going to do with it?

Welp, here we go.

Slack

It’s busy

First thing’s first. The community has grown and it is no longer a place where a few people geek out about operationalizing ML. With close to 9k ppl there are lots of messages being exchanged each day.

This is great if you are a die-hard fan and love drinking from the firehose. I personally enjoy it.

It’s not so good if you are trying to get the most useful info at a glance in your 5 min coffee break between meetings. We have incredible engagement in Slack and I don’t want to do anything to throttle that.

What I do want to do is help those that can’t catch up on all the unread messages in the day. Up until now, at the bottom of the weekly newsletter, we have the ‘Best of Slack’ section. It has become apparent that some kind of digest summarizing some of the main threads is needed for those part-time Slack enthusiasts.

As of next week, we will be doing a weekly email dedicated to this; synthesizing the top threads in Slack. This is for those who are easily overwhelmed. Hopefully, the Slack summary will be a way you can stay up to date on the weekly ‘best of’ without the anxiety. You can sign up to receive the email newsletter here. The best of slack will be our curated list of the most thought-provoking threads over the week.

Slack MLOps FAQ

Each day around 15 new people show up in slack. That’s about 100 new people per week. After 3 months, we have 1k new people who may answer the question differently. New discussions in threads with new members can create new insights on old questions. I know you think the question of putting jupyter notebooks in prod has been settled, but maybe not. keep an open mind and you might learn something.

Read: make a repository of the most commonly asked questions. The goal? Point people to that if they ask a question that has already been asked before.

This one tormented me. It sounds great in theory. It’s not the first time we have thought about it either. However, I have come to the decision that we will not do this for a few reasons.

First and foremost, this space moves fast. If the same question is asked 3 months apart the answers can be completely different. The way that each of us approaches a problem is unique. The conversations that are stimulated in the threads are what challenge us to grow.

Next up, each day around 15 new people show up in slack. That’s about 100 new people per week. After 3 months, we have 1k new people who may answer the question differently. New discussions in threads with new members can create new insights on old questions. I know you think the question of putting jupyter notebooks in prod has been settled, but maybe not. keep an open mind and you might learn something.

Plus, let’s be honest for a minute. If you want this common questions repo so bad, you can go create it and keep it up to date. Everybody wants an FAQ but nobody wants to do the work. If I created the FAQ, it would quickly become the only thing that I did.

Crossposts

I am enforcing a new rule that cross-posting is not allowed except in very unique circumstances. When I see it, I will delete it. I won’t even reach out to the person cross-posting. I have updated the community rules to reflect that. Please, @admins if you see it before I do, delete it.

So what are the edge cases where it’s ok?

For example, I post something in #general about various in-person meetups around the globe. Someone in Scottland shares that to their local channels saying something like “Hey we should meet IRL like these other cities”. That’s cool.

Another example, you ask a question in #mlops-questions-answered and also share the original questions to the healthcare channel with some context like “posting this here because its a specific healthcare-related questions we have been debating on my team”.

The key here is that if you cross-post in different channels with some context as to why you are also putting it in that channel, it’s cool. I hope you can see the pattern. Let me know if it’s not clear.

Vendors pushing their product

A year ago I had to create the rule that it is not OK for a vendor to talk about their product. You can’t imagine how much pushback I got from that. Even still, in the feedback form, I had people saying there were too many vendors pushing their products.

This is the bane of my existence. Still. I shut off #general for non-admins. If you think it is bad now, you should have seen what was happening a year ago. Seriously, vendors and open source project leads alike hate me. I come down hard on blatant vendor content.

I had to create the rule that it is not OK for a vendor to talk about their product. You can’t imagine how much pushback I got from that. Even still, in the feedback form, I had people saying there were too many vendors pushing their products.

Guess what else I saw in the feedback? All the people associated with products complain about how they can’t talk about their tools. ¯\_(ツ)_/¯

I am going to have to ask for the communities help on this one. If you see something, say something. We need “slack police”. I can’t be in every thread all the time.

As mentioned above, we are about to hit 9k members. If you feel someone is pushing their product, call them out. You can even put an #ads emoji reaction and it will automatically notify the admin team. Don’t settle for spam, it truly is toxic for the integrity of the community. Rest assured that you are not the only one irritated. See something, say something.

Lazy posts

If you just put a link to your LinkedIn or Twitter post, I think that is lazy. Add some context, or better yet bring the post into slack and make it native. You won’t get rewarded with “likes” or a viral tweet but you may create a great discussion.

I understand I am creating more work for myself by making this rule. It’s necessary. I am going to start deleting posts that are just a link to personal Twitter or LinkedIn posts. Take some time to craft what you are writing.

Common Themes

IRL meetups. Doesn’t have to be fancy. In fact, the majority of the first ones we have done around the globe were just grabbing some drinks together. Barcelona just met last week, we are meeting in London next week, and SF the week after. All of the action is happening in the local channels on slack and I will also try to keep the public calendar updated. If you want to be a local champion and help organize one of these in your city reach out!

More diversity on the meetups and podcasts

Yes. I completely agree. I am working with Brian John and Stephen Oladele to get more people from Africa involved. We’ve got 5 female podcast guests lined up in the next 4 months. That’s nothing to the 20 male guests we have lined up. We don’t have any LGBTQ+ guests, but I would like to.

I am asking all of you for help. If you know anyone from underrepresented groups that would be good for the pod or meetup please reach out.

More networking opportunities

I understand there are some amazing people in this community! Lets figure out a way to connect with one another more. The good news is that we have two new initiatives that are specifically for this.

IRL meetups. Doesn’t have to be fancy. In fact the majority of the first ones we have done around the globe were just grabbing some drinks together. Barcelona just met last week, we are meeting in London next week, and SF the week after. All of the action is happening in the local channels on slack and I will also try to keep the public calendar updated. If you want to be a local champion and help organize one of these in your city reach out!

As far as virtual ways to meet, we also got that covered. The new platform we are using allows for random matching video calls to happen so if 20 of us come to the networking event, we will be randomly matched with another participant for a 10min video call. It’s pretty cool and I have also started adding it as part of the meetup sessions.

And who could forget, we have the Slack channel called #virtual-coffee that takes advantage of the donut app. Donut pairs you up with another person in that channel at random. This can be a quick fix for you to meet others in the MLOps space if you live in rural German farmland like me. (I am so desperate I started talking to cows about what they think the best strategy is for auto retraining models.)

Mentorship-type stuff

I am putting serious thought into this and how we can make it happen. I think it is a really cool idea and considering we have so many OGs in the community, it could be very valuable. For those that are open to being a mentor, please reach out to me!

Meet People Where They Are

From now on I am working with various volunteers to take key insights from podcasts and put them into a blog post.

Podcast key takeaways.

Lots of people don’t like listening to hour-long podcasts. I get it. Personally, I am a fan of longer-form podcasts. But Vishnu told me a secret. He never listens to podcasts. Turns out, many of you are the same. From now on, I am working with various volunteers to take key insights from podcasts and put them into a blog post.

An interesting point here was many said they were fine with receiving more communications or DMs from me about content we are putting out. Some even said they are fine with getting a summary of things even if it is opinionated. Be careful what you wish for! You’d be surprised how many opinions cows have on MLOps topics.

More hands-on workshops, more tooling sessions, more hands-on tutorials

I am gonna chalk this up to practice.

I ask that you check out the community docs page. We are doing this already. It’s my fault for not being vocal enough about it. A few, in particular, I want to reference are:

Engineering labs. This initiative gives participants hands-on experience with different tools over a 6 week period. You can participate on a team or by yourself. We just started a new cohort. if you ask the right person you may be able to jump in still.

Monthly coding workshops. In these meetups, you get to take different tools for a test run. We also record all of them. If you missed the first couple, you can them check out on our Youtube page. Side note, there were lots of requests for someone to give an Argo workflows workshop. I can’t find anyone that wants to lead the workshop. If you know somebody, please tell me!

We also plan to hold a tooling summit in June. There will be plenty of opportunities to get your hands dirty.

Show and tell – real-world examples – real stacks – actual architecture – company’s design decisions – deep dives

Let me introduce you to a little initiative that Skylar Paine is leading called Pancake-stacks. Yes, I know the naming convention is amazing. Thank you.

What it is – a monthly round table discussion where someone presents their current ML stack. Everyone else joining the call digs in to ask deeper questions. Questions can be about trade-offs the presenter made, what they think of certain design decisions, what they would do differently, how they plan to scale, etc.

The first two we have had were absolute 🔥. We do not record these. People can feel free to ask and share whatever they want. We also do not have a set schedule so stay up to date on all the latest info in the Slack channel or by subscribing to our public calendar.

The Long Tail

Then there were the long tail responses of everything else. These were really hard to interpret. So. Many. Contradicting. Statements.

I kid you not, one response said shorter podcasts and literally the next response said longer podcasts.

One response said please stop singing. Now let me be honest about something real fast. Everyone could have said they don’t want any more songs. I would keep doing it. I’ll die on that mountain. Wanna be part of the community? You gotta put up with my singing out of key and off-rhythm every once in a while.

Enough with the jokes.

Thank you all for taking the time to fill out the feedback forms. It means more than you could ever imagine. Especially because some things were hard to hear. I never wanna get complacent with this community. Hearing your opinions helps me to keep what we are doing relevant and useful. Feel free to reach out if you think we can do something better. I am always open to critical criticism/feedback, not only once a year when I ask you to fill out a questionnaire.