Skip to content

Commit

Permalink
creating note admonition
Browse files Browse the repository at this point in the history
  • Loading branch information
Het Trivedi committed Sep 7, 2023
1 parent d6133c1 commit 6700e27
Showing 1 changed file with 9 additions and 7 deletions.
16 changes: 9 additions & 7 deletions vector-stores.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,15 @@ Vector Embeddings

Many closed-source models like [text-embedding-ada-002](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings) from OpenAI and the [embeddings model](https://docs.cohere.com/docs/embeddings) from Cohere allow developers to convert raw text into vector embeddings. It's important to not that the models used to generate vector embeddings are NOT the same models used for text generation.


```{note}
For NLP, embeddings are trained on a language modeling objective.
This means they are trained to predict surrounding words/context, not to generate text.
Embeddings models are encoder-only models without decoders.
They output an embedding, not generated text.
Generation models like GPT-2/3 have a decoder component trained explicitly for text generation.
```{admonition} Embeddings vs Text Generation
:name: embeddings-vs-generation
:class: note
- For NLP, embeddings are trained on a language modeling objective.
- This means they are trained to predict surrounding words/context, not to generate text.
- Embeddings models are encoder-only models without decoders.
- They output an embedding, not generated text.
- Generation models like GPT-2/3 have a decoder component trained explicitly for text generation.
```

## Vector Databases
Expand Down

0 comments on commit 6700e27

Please sign in to comment.