Implement LLM pipeline at the AI runner side [60 LPT]

[rickstaa]

Overview

Important

This can be viewed as a retroactive bounty since both this one and the subsequent go-livepeer bounty were already completed by Nico after a discussion in the community chats. Unfortunately, I didn't have time to post the bounty earlier.

To enhance the capabilities of our AI subnet, we aim to implement LLM (Large Language Model) capabilities. With the recent release of the fully open-source LLama 3.1 model, this is the perfect opportunity to introduce advanced language processing functionality. Implementing LLama 3.1 will allow applications to leverage sophisticated language processing on a fully permissionless and open decentralized network, benefiting various applications within the ecosystem.

As the core AI team is currently focused on video-centric pipelines and core network improvements, we are calling on the community to help implement this crucial pipeline on the AI-worker side of the AI subnet. This implementation will not only provide a new pipeline but also enhance existing functionalities by incorporating language understanding into our pipelines. We are excited to see this capability on the Livepeer network 🚀.

Required Skillset

• Familiarity with FastAPI.
• Strong Python programming skills.

Bounty Requirements

1. Implementation: Develop a working /llm route and pipeline in the AI worker repository. This pipeline should be accessible on port 6007.
2. Functionality: The pipeline must accept text input and utilize the LLama 3.1 model to generate and return appropriate responses.

Scope Exclusions

• This bounty does NOT cover the complete end-to-end implementation of this pipeline on the go-livepeer side, including payment logic and job routing. These aspects will be addressed by the AI SPE team or in a future bounty equal to another 30-40 LPT.

Implementation Tips

To understand how to create a new AI worker pipeline, you can refer to recent pull requests where new pipelines were added:

• Pull Request #96
• Pull Request #103

Additionally, make sure to:

• Utilize Earlier Work: Review the frame interpolation implementation done by @yondon in an earlier version of the AI subnet. This can provide valuable insights and a foundation for your work.
• Utilize Developer Documentation: Check out our developer documentation for the worker and runner. These resources provide valuable tips for speeding up your development process by mocking pipelines and enabling direct debugging.
• Generate OpenAPI Spec: Run the runner/gen_openapi.py file to generate the updated OpenAPI spec.
• Generate Go-Livepeer Bindings: In the main repository folder, run the make command to generate the necessary go-livepeer bindings, ensuring your implementation works seamlessly with the go-livepeer repository.

How to Apply

1. Express Your Interest: Comment on this issue to indicate your interest and explain why you're the ideal candidate for the task.
2. Wait for Review: Our team will review expressions of interest and select the best candidate.
3. Get Assigned: If selected, we'll assign the GitHub issue to you.
4. Start Working: Dive into your task! If you need assistance or guidance, comment on the issue or join the discussions in the #🛋│developer-lounge channel on our Discord server.
5. Submit Your Work: Create a pull request in the relevant repository and request a review.
6. Notify Us: Comment on this GitHub issue when your pull request is ready for review.
7. Receive Your Bounty: We'll arrange the bounty payment once your pull request is approved.
8. Gain Recognition: Your valuable contributions will be showcased in our project's changelog.

Thank you for your interest in contributing to our project 💛!

Warning

Please wait for the issue to be assigned to you before starting work. To prevent duplication of effort, submissions for unassigned issues will not be accepted.

[benya7]

Hi, I'm interested in taking on this task. I've reviewed the spec thoroughly, and it seems achievable for me. I have a strong background in Python and am very familiar with FastAPI.

Thanks.

[rickstaa]

Hi, I'm interested in taking on this task. I've reviewed the spec thoroughly, and it seems achievable for me. I have a strong background in Python and am very familiar with FastAPI.

Thanks.

Hey @benya7,

Thanks a lot for your interest in our bounties! 🚀

This particular bounty has already been taken on by one of our core contributors as part of a larger initiative. You can check out the details here: AI Worker PR #137 and Go Livepeer.

However, we will be posting more bounties soon, so stay tuned! 👍🏻

In the meantime, don’t forget to join our Discord. It’s an excellent place to stay updated and connect with other developers in our open-source ecosystem.

Also, we have a Livepeer Grants program that features some larger tasks (i.e., >$5k) which might be of interest to an experienced developer like yourself.

Thanks again, and we look forward to your contributions 🙏🏻!

[rickstaa]

This feature was implemented by @kyriediculous in livepeer/ai-worker#137, who spearheaded the creation of a Service Provider Entity (SPE) to drive Large Language Model (LLM) development on the Livepeer network. This SPE has just been successfully funded, as seen on the Livepeer Explorer here.