Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Self-Hosted Ingest #309

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open

Self-Hosted Ingest #309

wants to merge 22 commits into from

Commits on Oct 7, 2024

  1. created a demo pipeline

    @star-nox
    star-nox committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    34c1c66 View commit details
    Browse the repository at this point in the history

  2. added ingest class to RQ

    @star-nox
    star-nox committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    5cae03f View commit details
    Browse the repository at this point in the history

  3. aded openaiembeddings.py file to RQ

    @star-nox
    star-nox committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    416f701 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. modified ingest.py to resolve errors with RQ

    @star-nox
    star-nox committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a08687b View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. minor changes

    @star-nox
    star-nox committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    826903a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into self-hosted-ingest

    @star-nox
    star-nox committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    dc2453e View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. minor changes

    @star-nox
    star-nox committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    85d0a96 View commit details
    Browse the repository at this point in the history

  2. added ingest endpoint to main.py

    @star-nox
    star-nox committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    fe84c39 View commit details
    Browse the repository at this point in the history

  3. Fix import paths, add ./run-dev.sh script

    @KastanDay
    KastanDay committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    a078f69 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. Make Redis Queue work with a single ./run-dev.sh command. All automatic

    @KastanDay
    KastanDay committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    537a3fb View commit details
    Browse the repository at this point in the history

  2. modified run-dev.sh

    @star-nox
    star-nox committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    1c234b6 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. minor debugging

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    a442096 View commit details
    Browse the repository at this point in the history

  2. corrected content input in ingest.py

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    2ddd2c2 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into self-hosted-ingest

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    a23b61e View commit details
    Browse the repository at this point in the history

  4. added callback functions to log errors

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    ee0a5d0 View commit details
    Browse the repository at this point in the history

  5. removed dump.rdb and run-dev-local.sh files

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    d0440a0 View commit details
    Browse the repository at this point in the history

  6. removed langchain_community import

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    ba419c1 View commit details
    Browse the repository at this point in the history

  7. added redis and rq to requirements.txt

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    fb93cd5 View commit details
    Browse the repository at this point in the history

  8. updated run.sh to start workers.py for RQ

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    ef4fda6 View commit details
    Browse the repository at this point in the history

  9. uncommented some libs from requirements.txt

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    2dfa72b View commit details
    Browse the repository at this point in the history

  10. added pdfplumber to requirements.txt

    @star-nox
    star-nox committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    06e27eb View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. removed warning supression

    @star-nox
    star-nox committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    6c19121 View commit details
    Browse the repository at this point in the history