MLOps Community
+00:00 GMT
Sign in or Join the community to continue

Create Multi-Agent AI Systems in JavaScript // Dariel Vila // Agents in Production

Posted Nov 26, 2024 | Views 931
# javascript
# multi-agent
# AI Systems
Share
speaker
avatar
Dariel Vila
Lead Developer @ KaibanJS

Passionate about JavaScript and entrepreneurship, with over 15 years of experience. Proud to have helped create many cool projects: AniJS, an open-source library with 3500+ GitHub stars, led the development of the travel industry's first micro-frontend platform at Everymundo, co-founded Front10 to build the first mobile real-time COVID-19 map, & guided the creation of WOXO's AI video generator for hyper-personalized content.

Now focused on bringing the AI Revolution to the JavaScript community.

+ Read More
SUMMARY

In a world dominated by Python-centric AI frameworks, JavaScript developers have often found themselves at a disadvantage when it comes to building intelligent, multi-agent systems. KaibanJS aims to change that. In this talk, we’ll explore how this first-of-its-kind, JavaScript-native framework enables developers to create and integrate AI agents effortlessly into their projects. We’ll break down real-world use cases, dig into the technical features, and demonstrate how KaibanJS is helping to bridge the gap for JavaScript developers in the AI space.

+ Read More
TRANSCRIPT

Adam Becker [00:00:01]: One of the things that James spoke about is sort of like the need to upskill many of these engineers that are interested in starting to play with LLMs and with agents and with AI. One thing that I had seen in a bunch of different startups is that they then figure out, okay, well the first thing I got to do is learn Python because there aren't sufficient frameworks to be working in JavaScript or TypeScript or wherever else they happen to be. Next up we have Dariel. Dariel, can you hear us?

Dariel Vila [00:00:30]: Yes, yes. Okay. Hi everyone. Hi Adam.

Adam Becker [00:00:34]: I have a feeling that you're trying to address this problem that I had just spoken about.

Dariel Vila [00:00:39]: Yeah, I'm trying hard.

Adam Becker [00:00:42]: You've been trying hard. We're very excited to hear what your attempts have resulted in thus far. I will come back in 20 minutes with some questions from the audience. Until then, take it away.

Dariel Vila [00:00:57]: Awesome. Hi guys. Hi. Hi Adam. I'm very pleased to be here. The conference have been great so far, amazing speakers, amazing talk so far in this one we will talk about how to create a multi agent systems in JavaScript. Yes, Adam was right. So there is a huge need for AI tooling for JavaScript developers and we were trying to solve that.

Dariel Vila [00:01:25]: So to start like a bit about myself, I've been a front end Engineer since the jQuery days and I also been lucky to work in some like cool projects like an open source library that got some love from the community back in 2014. Then I worked and lead the creation of the first micro frontend platform for the travel industry which is used for big airlines like United American. All those guys and also like two years ago helped to architect an AI agent that we call it that way like for a video generator. So I'm passionate about JavaScript since the early days of AI we were struggling to try to find like the tools that work in the JavaScript language with AI and from struggling we tried to create and solve Gitkyba. So in today's presentation like we want to talk about a bit about why AI agents for JavaScript we want to see like a few like frameworks on the current landscape. We will talk a bit about Kaiban and we will do all that through a real world example. So first like why AI agent for JavaScript? First of all of all like JavaScript have a huge developer base just surpassed by Python this year on GitHub. But still like JavaScript and TypeScript are the second and third most active community.

Dariel Vila [00:02:50]: GitHub JavaScript has also like a lot of enterprise adoption like it's trusted by many companies like Netflix, PI, PayPal, Google, all those guys. Another cool stuff about JavaScript too is that it's very cost effective. So if you play smart and make some of the processing in a browser right, you are shifting the cost from doing all those processing in the server and now with the smaller models working on the, on the browser level, that Gemma model on Google and I think that I, I have seen how this trend is accelerating with the time and if you are able to like compartmentalize your application in small pieces and make some of them run in the front end. So you're going to save a huge cost if you have a large user base. Another cool property of JavaScript is ubiquity. So you have JavaScript everywhere in browser service, llt another device. So that four properties makes JavaScript like a very good language to explore to try to create agents using it. Here are some of the frameworks that you can use today to create multi agents in JavaScript.

Dariel Vila [00:04:08]: In one side you have Lanchain, Vercel and Llama Index. Those were like those started with the first wave of the AI frameworks and they try to cover a lot of use cases so they have been like adding agentic capabilities as soon as they are released. And then in other part you have other frameworks like landgraph that let you create fine tune agents. But Landgraph it's so it's very low code. Yeah, so that's why we try to solve all those problems like creating Kaiban. So we tried to create a multi agent framework with just a focus on multi agent work. And we tried the API to be really simple to use because one pattern found in the JavaScript comp they are not super aware about LLMs about AI. So we try to make it super simple to understand.

Dariel Vila [00:05:06]: So we also try to create visual and interactive tooling so people can debug the agent. Not in black terminals but they have like a proper UI to see the agents and that's why we call the Kaiban board. The Kaiban board is like a Kanban view when you can see your agents like moving cars like this is a really cool UI to see the agents working. And another thing we tried to push since the very beginning was like trying to make Kaiban work in the browser as well as in the server. So right now instead of me going through every Kyban feature, kaibanjs feature, we will let's do this real example. So meet this guy John. John is another tech leak just like any of us that he's trying to Automate whatever boring part of his job it is. And let's say that John work in this cool dashboard like every day, like triage tickets.

Dariel Vila [00:06:04]: So people open tickets and Johnstein have to go through every ticket running like a triage process. He has to go and do an initial assessment, then he does an issue analysis, then he find out in GitHub the related issues and try to correlate all that information. So after attending to this beautiful conference, like John has an idea. Hey, let's create a bunch of agents, one for each part of the journey. So everything was nice. He was almost doing a party. But then he realized that sometimes it's not just about implementation. When you work in a big company on a big corporate, you have to navigate the corporate rabbit hole.

Dariel Vila [00:06:47]: You know, like the systems are distributed sometimes across many teams and he has to get like buyout from key stakeholders and all of that. So John is thinking about how can he effectively navigate that rabbit hole and have the results he wants. Luckily for him he was in this talk and he knew about Kaiban. Kaiban will allow John to just with two files, these two files that I'm going to show you in more in detail have this component here in the front end and create the agents. He wants to quickly have a prototype, a cho in the next meeting instead of he running through everyone asking for like things he needs, he can provide them with a real prototype and he can build it really quick in the front end. So here like let's go through the first file is when you define the agents you I'm pretty sure you are very familiar with this type of like API. Like the agents API like crew AI swarm for from OpenAI tool. When you define the agents, the tasks and the team you put, you create all them, you put them together in a team.

Dariel Vila [00:08:00]: So this is the main primitive that the Kaiban JS framework have. Like you have agents, the agents have tools and the agents does perform tasks and the tasks and agents and tools are orchestrated through a team. So let's take a look at the component, like the front end component that he will use. As you see this is like another react component and here are the main parts. So he will use a store because KaivanJS provide all the state of the agents in a global store that you can plug in in your application and have access to everything that the agent is doing in real time. The tool he's using, the amount of tokens that he's using, or if he's getting some errors, how many agentic cycle he completes and all that. So and to represent the work you see here on this component when he have the list of tasks and he's just like representing the state, like really easy and quick. Yes, John is happy again because he have a prototype working really quick.

Dariel Vila [00:09:09]: We have no like driving the maze, the maze and he can show it to his co workers and then from there move on with the working theme. So what does this mean to you and what's the like? I mean the take from this like example is that you can quickly create prototypes, MVPs, even working projects just like in JavaScript with the JavaScript stack you have. So Kaiban is not just about this use case. We have many others like in the website, you know the classic ones like the three planning team you have also the ones for sport news, the trip planning, it's half an agent and that is capable of doing all the agentic work like using tools, memory on top of Lanchain JS. So if there is something that KaivanJS doesn't cover, there's always the possibility that you can use all the like the big ecosystem that launch supports. So with that said, a pretty cool I would say like conclusion that you can take from here is that most of the times you guys have dozen of interfaces built already and there are a bunch of JavaScript developers already working in the front end, working in the backend and you can use those resources, you just have to give them the tools they need to also participate in the revolution. So you kind of scale all that workforce just by giving them the access to the right tools. So I think I went super fast trying to watch this clock.

Dariel Vila [00:10:59]: So I'm pretty like I'm going to be here if you have any questions. I tried to win like super fast.

Adam Becker [00:11:07]: You did go fast.

Dariel Vila [00:11:08]: Yeah.

Adam Becker [00:11:09]: Thank you very much, Dariel. Let's see if we have any questions. I'm going to put up the ticker. So far none yet. I have a question for you, can you go back to the use cases then?

Dariel Vila [00:11:22]: Sure.

Adam Becker [00:11:24]: Can you tell us a little bit more about those? So let's say the AI agents for trip planning, what gave them, what motivated them to Write this in JavaScript in the first place? Is it about the lack of availability of other engineers or the fact that they already have those engineers on hand and then they wanted to like.

Dariel Vila [00:11:43]: Yeah, I would say this is a combination of two things. Right. So when you see the amount of developers out there and use it in big companies, you want to see like JavaScript developers and you want to see Python developers, but there Are a lot of JavaScript developers out there already working in companies but because they don't have the right tools to work in, so companies have to hire people from outside. And also like when you have in an enterprise and a stack, a given stack, adding a new server, adding new infrastructure, sometimes it's painful, right? So there is a combination of this factor, like the human factor and also the combination that with JavaScript you can touch many other surfaces. Like for example, you can have your agents working in the browser if they are not doing like a critical and a power consuming thing. Or you can even split it up your team of agents. You have one agent in the browser just getting the user data and then send it out to another agent in the back end. That is like handling the levy, lifting the heavy, lifting them, send it back to the browser and if you split it out, all that processing at the end of the day going to cost less for the company if you have a huge volume.

Dariel Vila [00:13:12]: You know, can I ask is your.

Adam Becker [00:13:15]: Background and you might have addressed this already primarily in front end or back end. Like how did you pick up? I remember you saying like jquery, like this is. Did you start in front end and then gradually start doing Node JS and stuff?

Dariel Vila [00:13:28]: Like yeah, yeah. I originally started computer science like 15 years ago and we were like doing C and all that. But when I learned that you have the ability to like share your work through an URL like this one, like it was like a magical moment for me. So for now, for that moment on I switch up and I learned PHP JavaScript and I've been in love with that stack since those days that I've been just working in the, in the JavaScript stack for the last, I say like 12, 13 years or so.

Adam Becker [00:14:07]: Yeah, do you have, and I know that some people have been speaking about this though I haven't quite like dug in very deeply. Do you have a sense of what would happen to front end frameworks and to front end engineering now that you have a whole generation of developers that let's say will adopt like more agentic frameworks and LLMs.

Dariel Vila [00:14:30]: Yeah, that's a good question. And the other day like actually for this presentation I did the mockups from the UI that John was using, like using B0 ball all new and in just about 30 minutes. And yes, I will say that it will change. Not that the job will disappear, but there are going to be less people working because like before doing an interface like that it took like a team of three people working in for a month and you don't even get the result that you wanted. So I would say that that will help to get the ideas, what the people really want into production. And eventually these roles are going to be, are going to, you know, like shift. So instead of writing the code just like you, redirecting the agents to help you out, you know, so that's why it's important. Like I say, as companies, if you have like front end developers or people that their jobs are going to have to shift in some way if you put them to work with AI tool, you are reusing the same people that are working in one place, you are using them to produce in other places that there are more demands right now.

Adam Becker [00:15:49]: Do you have some, I mean, I suspect that there might also be some forms of resistance at the organizational level about, you know, they're already thinking about their, let's say front end engineers or the front end engineer, the ones that are building any AI systems and there's going to be some kind of like paradigmatic shift. It feels where managers and product managers will start to think that maybe it's okay to let one of these engineers, and this is primarily doing front end or back end to begin to build out AI features. Do you have any suggestions or recommendations for JavaScript software developers to show up to the product managers or to the rest of their organization and tell them that it's okay to allow them to do this sort of thing because it feels like sometimes they might need some form of like explicit or implicit or cultural kind of permission?

Dariel Vila [00:16:53]: No. Yeah, I think I've seen that kind of resistance. And what I think has happened is like people keep thinking that AI is still the same AI that it was before. When you have to fine tune the model yourself, you have to train the model and you have to know neural networks and all that. But after ChatGPT and all this cool model from OpenAI, all the state of the art things have cheated. So instead of you having a deep knowledge of how the models work, so what you need to do is learn about the APIs and learn what other people are doing. For example, two years ago we made like, and I mention it because like internally we have like big discussions about like, should we fine tune this model? It was a model for video creation.

Adam Becker [00:17:51]: Yeah.

Dariel Vila [00:17:52]: So we took an input from a user and created JSON that describes a video. So we had these discussions if we have to fine tune the models or we just have prompt engineer a way for the LLMs, like the big ones, like giving us the results that we want. And at the end of the day we ended up using the LLM, let's say we have any fine tuning and we were just like four or five JavaScript developers. You know, we know AI, previous knowledge and we will be able to figure it out how to create the thing that we wanted and most importantly the thing that our users wanted. Because one thing is what is in the head of an engineer and one thing is what is in the head of your users. And sometimes people get crazy and create this all like big architectures with redundancies and backups and all that. And at the end of the day, the final word, the user is the one that have it. So that's why we also create Kaiban to let all the developers in JavaScript just go out and experiment with AI.

Dariel Vila [00:19:02]: It is funny because people started using it in a very simple way and they do very simple stuff, but they use it as a starting lesson to then start going deep, deep, deep, deep into the rabbit hole. At the end of the day you will ended up using a very complex architecture. But in this stage a lot of Companies are doing MVPs, POCs and all that. What you need is, what you need is most of the time tools that allow you to quickly create stuff and show it there and get feedback refined and then with data you are able to create a more robust solution.

Adam Becker [00:19:44]: There's something similar that I thought about at one point, which was the tension between AI and machine learning. And I feel like a lot of people still kind of maintain that conception that like you're to do AI, you're doing machine learning, but that's no longer really the case. You could be playing with AI and know nothing about machine learning. And I would probably argue that most people who do play with AI right now know nothing about machine learning and that's okay sufficiently. So you just have to still, you still need some paradigm of testing and evaluation and training and data and all these different things. So some concepts do carry over, but nevertheless it feels like a slightly different discipline at this point.

Dariel Vila [00:20:23]: No, yeah, exactly that one word is the AI research or if you are creating a framework, all that. And another word is integrating it in the real life. And for that you don't need what I would say you need is like the will to do it and just go for it. Go to every Reddit, go to every Slack, every discord and you will learn a lot. There is a lot of conversations happening in next on Reddit, in Slack. You put it out there, you test it, you have to be responsible, of course you are not going to put like any thing that have security issues or anything like that out there. But as soon as you are responsible and you own the actions that you are doing, just like try an experiment, try an experiment and learn. That will give you amazing results.

Adam Becker [00:21:13]: We have one last question here from the chat Tamer is asking. Do you think that it is possible to create a complete back end of a product just by using AI agents? So not like developing the code, but more like that agents are going to put things into the database, create responses that will be rendered in the backend, et cetera. Can I add a spin to this question which is would frontend engineers be able to interface with agents instead of interfacing with like a backend and perhaps the agents themselves then do the kind of work of building up the back end. But really it is driven by front end concerns.

Dariel Vila [00:21:54]: I would say, like the way things are moving. It seems to be that that will be the case because like, you know, we as humans have now the ability to like creating instead of reusing stuff, just creating things in the spot, like generating things. So. And that will happen in the front end too. So why not? It's going to happen in the, in the backend. You know, when you will just generate on demand that let's say you're asking a query and that's actually a very good, like, I would say like a very good product. I haven't seen it yet. That when you ask a query, just like a GraphQL query and then the back end build itself, the API build itself.

Dariel Vila [00:22:44]: I think if you do the right wire rails, it could be something really interesting because you don't need to create these huge API contracts anymore when you have to learn all the API documentation. You just teach an agent how to work with your schema and then you let it go. And the APIs are going to be like, I know a lot of guys are going to be super help, super grateful for a product like that. Yeah.

Adam Becker [00:23:14]: Dariel, thank you very much for coming and sharing with us. Best of luck to you and to Kaivan. Next.

Dariel Vila [00:23:21]: Thank you Adam. And thank you everyone.

Adam Becker [00:23:22]: Thank you.

+ Read More
Sign in or Join the community

Create an account

Change email
e.g. https://www.linkedin.com/in/xxx or https://xx.linkedin.com/in/xxx
I agree to MLOps Community’s Code of Conduct and Privacy Policy.

Watch More

1:04:41
Managing Small Knowledge Graphs for Multi-agent Systems
Posted May 28, 2024 | Views 1.6K
# Knowledge Graphs
# Generative AI
# RAG
# Whyhow.ai
Lessons From Building Replit Agent // James Austin // Agents in Production
Posted Nov 26, 2024 | Views 999
# Replit Agent
# Repls
# Replit
Generative AI Agents in Production: Best Practices and Lessons Learned // Patrick Marlow // Agents in Production
Posted Nov 15, 2024 | Views 1.8K
# Generative AI Agents
# Vertex Applied AI
# Agents in Production