💡 This is a project for the MLOps Community to fully understand what different people touching ML do at their jobs. We want to find out what their day-to-day looks like.

From the most granular to the most mundane, they tell us everything! This is our chance to bring clarity to the different parts of MLOps ranging from big companies to small startups.

Machine Learning Engineers At Work

Name: Matheus Frata

Official Title: Machine Learning Engineer – Tech Lead

Company: Neoway

Years in the game (years in the labor market): 6

Years specifically working on ML: 5

Direct reports: Head of Analytics, VP Platform

What was your path into Machine Learning Engineering?

I got into ML totally by accident.

My background is in Electronics. I was into hardware and embedded systems. During my undergrad, I saw a data science boot camp ad on the job board of our campus. I decided to give it a try. I passed the interview process and got in.

I learned Python, SQL, statistics, and a little bit of front-end as well back-end. We also had to create a complete application with a machine learning aspect to it. I liked every minute of it.

The best teams received internships and I got one as a Data Scientist.

Six months into the internship I noticed that we were struggling more with the engineering aspects of the models instead of with the statistical part. I had some background in software so I decided to shift my work to this.

This is how I got into ML Engineering and MLOps (these terms did not exist back then).

What interests you about your current position?

I enjoy the hybrid aspect of working low level in the code, tweaking with algorithms, and at the same time, discussing higher level topics like architectures and business problems.

What are some things that drive you crazy about your position?

People reinventing the wheel. From time to time, we discover some people from other parts of the company trying to create something that we already solved.

Sometimes, they don’t know about our tools. Sometimes, it doesn’t cover all their requirements. In both cases, we do consulting to help them with their problem.

What does your company do?

It provides companies with intelligence for decision-making. How exactly? I always like to tell this story: imagine that you have a Ketchup factory.

You want to expand your business to the south of Brazil. You can enter Neoway’s platform and use a lot of filters to find possible clients in the south. You may search for markets or restaurants. Filter based on size, revenue, and other filters.

Or use one product that searches for you based on your current clients!

What is your team responsible for?

An excerpt of our mission on our team’s page:

The machine learning platform team has the main mission of enabling data scientists to execute the ML life cycle tasks quickly and efficiently but always keeping in mind aspects of governance and security.

We also talk about our team in this post about the cultural side of MLOps, and in this post talking about how the Machine Learning Engineering team built a feature store.

What are some use cases you have with ML?

We have models for Sales n’Marketing, Risk n’Compliance, and Legal Analysis.

Models can vary from the companies’ estimated revenues, the probability of a company’s activity, or their sentiment analysis from the news.

In some cases, the model is the product. On others, it is part of it.

What projects are you working on in the next 6 months?

We have an ML Platform for internal users. We are expanding to reach external ones. Our goal is to offer Neoway’s partners our data and infrastructure to create their models.

What tech do you touch on a daily bases? For what?

Python, PySpark, Airflow, Kubernetes, and Docker most of the time.

These cover all the important services that we offer, like the ML pipeline and Feature Store.

Data Scientists have control of what they want to use to build their models. The most common libs are pandas, PySpark, Catboost, and Shap.

What are your main responsibilities?

Ensure our development is aligned with the company’s vision while at the same time keeping the current users happy. Meaning, I have to choose the tradeoffs between new features and bug fixes (broadly speaking).

Also, since the team has new members I must make sure that they are getting the whole context of the team, area, and company.

I talk more about what my responsibilities are and how I think about Machine Learning engineering in general in the podcast that we did with the MLOps community a few months back.

What do your days consist of?

The team works with biweekly sprints. With review, planning, and daily meetings.

Also, I have some grooming meetings with the Product Owner. Other meetings are syncs with people: 1:1s with my direct reports, a chapter meeting with the whole analytics team and others with the engineering, a chat with people from other teams (asking questions about the services), and so on.

So I’d say that is 30 to 40% meetings.

Then another great amount of time that I spend is reading code. I review code from the ml-platform team and read (and run locally) code from other teams.

The latter is just to check out what they are doing, what technologies they use, and how we would integrate with their services. I learn a lot during these explorations.

That’s 10-20% of my time.

The rest is working on the tickets that we select in our sprint. Could be support, bug fixing, or adding new features.

What was the last bug you smashed?

We have a diff algorithm that runs with scala. We discovered a bug on a model that used this diff algo. A “super edge” case! Four people hopped on a sudden call because it was a crucial model. Eventually, we figured it out! But damn!! That one took a while! 😆

What are you most proud of in your current position?

After we shifted our mindset on what to prioritize, users first, we were able to develop and bring value to the users of the platform. And we can see that with the NPS and the feedback that we get from the users.

What did you expect the job to be like vs reality?

I did not have any expectations on the job since my transition was based on what I saw that the company needed me to act on.

What kind of metrics do you follow closely?

NPS and mean daily users.

Also non “hard” ones such as feedback from the users, and the time that they put a feature/model into production (time of the merge request).

What do you wish you knew before becoming a Machine Learning Engineer?

Stop focusing on the model. Become one with the data! Understand the problem and check if the data available is enough to start an ML solution. Once you get this then you are ready to go to the modeling!

Speaking of modeling: Start simple. Build a baseline and build up from that.

This means: Do not rush to neural nets just because they are cool! Build a simple model (sometimes not even a model is needed), and slowly improve it.

Any random stories from the job?

One day someone from OpenAI came to visit our office to check out our analytics stack.

They were dancing Brazilian funk in the happy hour 🤣

Outliers stories aside, pre-pandemic we used to have happy hours almost every Friday! This was the time we got to know people from different departments, teams, and also other companies!

Who do you admire?

One that I can cite is Cal Newport with his book “So good that they can’t ignore you“.

That book changed my mindset about work and life in general! The title says it all: Be so good that they can’t ignore you and you will have the best traits in your job and consequently life!

Where do you want to take your career next and why?

I see myself in the future going towards the strategic layer. I’m a gregarious person. I enjoy talking to people and helping them. Maybe management.

At the same time, I like the engineering aspect of the job but I am always connected with the business side, solving problems to bring value to a stakeholder. So I also see myself going to positions like VP or CTO.

We will see what happens in the next 5 years!

What advice do you have for someone starting now?

For companies: start with software engineering best practices! Use git, create tests, and so on. It is harder to add these things afterward. They seem like purely technical aspects, but instead, IMO they are cultural ones.

If the team culture is consistent and strong, all newcomers will enjoy it from the very beginning.

The result: sustainable models, happy engineers/data scientists.

For people: try things out. Not sure where to start on ML? Try all the paths.

Analyze some data, try to extract insights from it, and create visualizations to share your insights.

Build a model, do some feature engineering, and hyperparameter optimization, test some algorithms and see what their pros and cons are.

Put it in Production, put your model out there, create an API, use a framework, or even put it on a mobile device.

Too technical for you, but still want to work with ML? Try to connect with the business, and see what ML products are out there. Check the user journey, and find out how a product fits with the market!

The result: you make an informed decision about what should be your career path.

If your job touches the machine learning life cycle and you would like to share what your day to day is like please get in touch with Demetrios to kick off the process.

This post was written by Matheus Frata. Edited by Stefano Bosisio and Sarah Khalid