Skip to content

Latest commit

 

History

History
70 lines (44 loc) · 3.21 KB

README.md

File metadata and controls

70 lines (44 loc) · 3.21 KB
Raw

Karpathy-GPT

Context

This app is a template for using LangChain to build a LLM Q+A assistant from any set of YouTube videos.

We use Karpathy's course on LLMs as an example.

image

We use LangChain to:

(1) convert YouTube urls to text

(2) feed the text into LangChain auto-evaluator to test different chain parameters

(3) with our chosen parameters, build a vectorstore retriever back-end with FastAPI (deployed to Railway)

(4) stream the generated results (answer and retrieved docs) to a front-end (deployed to Vercel)

Step 1: URLs to text

See the notebook in /index folder:

  • Uses LangChain's OpenAIWhisperParser to convert urls to text in < 10 lines of code

Step 2: Testing

See the text files in /eval folder:

  • Feed the text from step 1 and, optionally, an eval set to the auto-evaluator app
  • We can use this to test different parameters (see full README in the repo here)
  • Use the UI to run experiments
  • Select your best retriever, chain settings (e.g., k, split size, split overlap, etc), LLM, embeddings

image

Step 3: text to VectorDB

See the notebook in /index folder:

  • Split the text from step 1 using parameters you found in step 2
  • Upsert the vectors to a VectorDB (e.g., in this example, Pinecone) with metadata
  • See this PR / notebook if you want to use locally with a different VectorDB

Step 4: Back-end

See the karpathy_app.py file in /api folder:

  • We use LangChain's load_qa_chain with a user specified LLM and prompt (see default_prompt_template)
  • Given a question, this will stream answer the text back to front-end and pass the retrieved documents back
  • We deploy this FastAPI API to Railway
  • See README.md in /api for local testing instructions

Step 5: Front-end

See /nextjs directory for nextJS app:

  • This will call the back-end with the query and fetch the documents / answer
  • Test the app locally by launching the back-end:
uvicorn karpathy_app:app
  • To run front-end locally with you locally running back-end, simply change the source in fetchEventSource here and here to http://localhost:8000/karpathy-docs and http://localhost:8000/karpathy-stream
  • To run the front-end locally, run:
npm run dev