Possible embedding bug.

[scottiesan]

Describe the bug
The reason I think this is a bug is because the cost of embeding is on the upward of a dollar or more when this happen and I had to kill the process.

seemingly stuck on this:

2024-10-04 13:52:55,759 - INFO - 🔎 Getting relevant written content based on query: The Importance of Interdisciplinary Knowledge in Future Careers...
2024-10-04 13:52:55,918 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:55,927 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:55,940 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:55,969 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:55,987 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:55,989 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:55,991 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,028 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,034 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,037 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,040 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,107 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,118 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,155 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,179 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,192 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,222 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,261 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,295 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,348 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,353 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,381 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,381 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,382 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,418 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,427 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,430 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,450 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,480 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,527 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,555 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,655 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,667 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,780 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,793 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,829 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,832 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,849 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,849 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:56,930 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:57,519 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-10-04 13:52:57,769 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"

To Reproduce
Run details report. does not happen all the time, but 1 out of 10 at least

Expected behavior
Embed quickly and out the article
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: mac vs code
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

[assafelovic]

Hey @scottiesan thanks for reporting this. Can you share what version of GPTR you're using? Is this using the PIP or the cloned repo? Any additional info can help us investigate

[scottiesan]

I am running pip install
Name: gpt-researcher
Version: 0.9.6
Summary: GPT Researcher is an autonomous agent designed for comprehensive online research on a variety of tasks.
Home-page: https://github.com/assafelovic/gpt-researcher
Author: Assaf Elovic
Author-email: [email protected]
License: MIT
Location: /Users/scott/.pyenv/versions/3.11.0/lib/python3.11/site-packages
Requires: aiofiles, arxiv, beautifulsoup4, colorama, htmldocx, json-repair, json5, langchain, langchain-community, langchain-openai, loguru, lxml-html-clean, markdown, md2pdf, mistune, pydantic, PyMuPDF, python-docx, python-dotenv, python-multipart, pyyaml, requests, tiktoken, unstructured, websockets

I did run it in a for loop generating one article after another. Most recent case happen when I was generating the DetailedReport.

Thanks for the quick reply. I hope that helps :).