Pre/beta #682

VinciGit00 · 2024-09-19T08:06:35Z

No description provided.

…-ai-with-screenshot-scraping into pre/beta

Co-Authored-By: Marco Vinciguerra <[email protected]>

feat: add togetherai

## [1.16.0-beta.1](v1.15.1...v1.16.0-beta.1) (2024-08-28) ### Features * add togheterai ([8f615ad](8f615ad)) ### chore * **examples:** create Together AI examples ([34942de](34942de))

Screenshot scraping

…ionerror-with-smartscrapergraph fix: update generate answernode

## [1.16.0-beta.2](v1.16.0-beta.1...v1.16.0-beta.2) (2024-08-31) ### Bug Fixes * update generate answernode ([c348f67](c348f67))

fixed tool and dependencies

## [1.16.0-beta.3](v1.16.0-beta.2...v1.16.0-beta.3) (2024-09-01) ### Bug Fixes * pyproject.toml ([360ce1c](360ce1c)) ### CI * **release:** 1.15.2 [skip ci] ([d88730c](d88730c))

…ccess to get_openai_callback

…ed-graphs Added CustomOpenaiCallback to ensure exclusive access to nested data.

## [1.19.0-beta.12](v1.19.0-beta.11...v1.19.0-beta.12) (2024-09-14) ### Bug Fixes * Refactor code to use CustomOpenAiCallbackManager for exclusive access to get_openai_callback ([e657113](e657113)) ### Docs * added telemetry info ([62912c2](62912c2))

allignment

## [1.20.0-beta.1](v1.19.0...v1.20.0-beta.1) (2024-09-14) ### Features * **AbstractGraph:** add adjustable rate limit ([2859fb7](2859fb7)) * add copy for smart_scraper_multi_concat ([9e3171b](9e3171b)) * add scrape_do_integration ([94e69a0](94e69a0)) * add togheterai ([8f615ad](8f615ad)) * ConcatNode.py added for heavy merge operations ([bd4b26d](bd4b26d)) * fetch_node improved ([167f970](167f970)) * refactoring of the tokenization function ([ec6b164](ec6b164)) * removed semchunk and used tikton ([1a7f21f](1a7f21f)) * return urls in searchgraph ([afb6eb7](afb6eb7)) ### Bug Fixes * Added support for nested structure ([66ea166](66ea166)) * **AbstractGraph:** Bedrock init issues ([63a5d18](63a5d18)), closes [#633](#633) * correctly parsing output when using structured_output ([8e74ac5](8e74ac5)) * Fixed pydantic error on SearchGraphs ([039ba2e](039ba2e)) * **ScreenshotScraper:** impose dynamic imports ([b8ef937](b8ef937)) * **Ollama:** instance model from correct package ([398b2c5](398b2c5)) * OmniScraerGraph working. ([c3d1b7c](c3d1b7c)) * parse node ([947ebd2](947ebd2)) * Parse Node scraping link and img urls allowing OmniScraper to work ([66a3b6d](66a3b6d)) * **SmartScraper:** pass llm_model to ParseNode ([5242166](5242166)) * **DeepSeek:** proper model initialization ([74dfc69](74dfc69)) * pyproject.toml dependencies ([b805aea](b805aea)) * Refactor code to use CustomOpenAiCallbackManager for exclusive access to get_openai_callback ([e657113](e657113)) * Removed link_urls and img_ulrs from FetchNode output ([57337a0](57337a0)) * screenshot scraper ([388630c](388630c)) * screenshot_scraper ([ef7a589](ef7a589)) * **ScreenShotScraper:** static import of optional dependencies ([52fe441](52fe441)) * temporary fix for parse_node ([f2bb22d](f2bb22d)) * update all nodes that were using MergeNode or IteratorNode ([a92dddb](a92dddb)) * update generate answernode ([c348f67](c348f67)) ### chore * **examples:** create Together AI examples ([34942de](34942de)) ### Docs * Updated the graph_config in the documentation. ([57a58e1](57a58e1)) ### CI * **release:** 1.16.0-beta.1 [skip ci] ([d7f6036](d7f6036)) * **release:** 1.16.0-beta.2 [skip ci] ([1c37d5d](1c37d5d)) * **release:** 1.16.0-beta.3 [skip ci] ([886c987](886c987)) * **release:** 1.16.0-beta.4 [skip ci] ([ba5c7ad](ba5c7ad)) * **release:** 1.17.0-beta.1 [skip ci] ([13efd4e](13efd4e)) * **release:** 1.17.0-beta.10 [skip ci] ([af28885](af28885)) * **release:** 1.17.0-beta.11 [skip ci] ([a73fec5](a73fec5)) * **release:** 1.17.0-beta.2 [skip ci] ([08afc92](08afc92)) * **release:** 1.17.0-beta.3 [skip ci] ([fc55418](fc55418)) * **release:** 1.17.0-beta.4 [skip ci] ([5e99071](5e99071)) * **release:** 1.17.0-beta.5 [skip ci] ([16ab1bf](16ab1bf)) * **release:** 1.17.0-beta.6 [skip ci] ([50c9c6b](50c9c6b)) * **release:** 1.17.0-beta.7 [skip ci] ([4347afb](4347afb)), closes [#633](#633) * **release:** 1.17.0-beta.8 [skip ci] ([85c374e](85c374e)) * **release:** 1.17.0-beta.9 [skip ci] ([77d0fd3](77d0fd3)) * **release:** 1.19.0-beta.1 [skip ci] ([eddcb79](eddcb79)), closes [#633](#633) [#633](#633) * **release:** 1.19.0-beta.10 [skip ci] ([92f5df2](92f5df2)) * **release:** 1.19.0-beta.11 [skip ci] ([edfb185](edfb185)) * **release:** 1.19.0-beta.12 [skip ci] ([bd2afef](bd2afef)) * **release:** 1.19.0-beta.2 [skip ci] ([23a260c](23a260c)) * **release:** 1.19.0-beta.3 [skip ci] ([38cba96](38cba96)) * **release:** 1.19.0-beta.4 [skip ci] ([24c38f9](24c38f9)) * **release:** 1.19.0-beta.5 [skip ci] ([7621a7c](7621a7c)) * **release:** 1.19.0-beta.6 [skip ci] ([ed8e173](ed8e173)) * **release:** 1.19.0-beta.7 [skip ci] ([4ab26a2](4ab26a2)) * **release:** 1.19.0-beta.8 [skip ci] ([88b2c46](88b2c46)) * **release:** 1.19.0-beta.9 [skip ci] ([7ad6f21](7ad6f21))

fix: Add mistral-common dependency

fix: Error in pyproject dependencies

## [1.20.0-beta.2](v1.20.0-beta.1...v1.20.0-beta.2) (2024-09-17) ### Bug Fixes * Add mistral-common dependency ([7681a45](7681a45)) * Error in pyproject dependencies ([5b5cb5b](5b5cb5b)) * fetch_node condition ([3f45c17](3f45c17))

feat: output parser and pydantic update

## [1.20.0-beta.3](v1.20.0-beta.2...v1.20.0-beta.3) (2024-09-18) ### Features * updated pydantic to v2 ([eb89549](eb89549)) ### Refactor * Output parser code ([28b85a3](28b85a3))

…ph-ai into pre/beta

## [1.20.0-beta.4](v1.20.0-beta.3...v1.20.0-beta.4) (2024-09-18) ### Bug Fixes * update pyproject.toml ([932412e](932412e))

feat: added Bedrock and Mistral to exec info

## [1.20.0-beta.5](v1.20.0-beta.4...v1.20.0-beta.5) (2024-09-18) ### Features * added Bedrock and Mistral to exec info ([8a37c6b](8a37c6b))

github-actions · 2024-09-19T08:07:55Z

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

Package

Version

Score

Details

pip/langchain

>= 0.3.0

🟢 5.3

Details

Check	Score	Reason
Code-Review	🟢 7	Found 22/30 approved changesets -- score normalized to 7
Maintained	🟢 10	30 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices	⚠️ 0	no effort to earn an OpenSSF best practices badge detected
License	🟢 10	license file detected
Signed-Releases	⚠️ -1	no releases found
Branch-Protection	🟢 3	branch protection is not maximal on development and all release branches
Security-Policy	🟢 10	security policy file detected
Dangerous-Workflow	🟢 10	no dangerous workflow patterns detected
Token-Permissions	⚠️ 0	detected GitHub workflow tokens with excessive permissions
SAST	⚠️ 0	SAST tool is not run on all commits -- score normalized to 0
Binary-Artifacts	🟢 10	no binaries found in the repo
Fuzzing	⚠️ 0	project is not fuzzed
Packaging	🟢 10	packaging workflow detected
Pinned-Dependencies	⚠️ 0	dependency not pinned by hash detected -- score normalized to 0
Vulnerabilities	⚠️ 0	35 existing vulnerabilities detected

pip/langchain-ollama

>= 0.1.3

🟢 5.3

Details

Check	Score	Reason
Code-Review	🟢 7	Found 22/30 approved changesets -- score normalized to 7
Maintained	🟢 10	30 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices	⚠️ 0	no effort to earn an OpenSSF best practices badge detected
License	🟢 10	license file detected
Signed-Releases	⚠️ -1	no releases found
Branch-Protection	🟢 3	branch protection is not maximal on development and all release branches
Security-Policy	🟢 10	security policy file detected
Dangerous-Workflow	🟢 10	no dangerous workflow patterns detected
Token-Permissions	⚠️ 0	detected GitHub workflow tokens with excessive permissions
SAST	⚠️ 0	SAST tool is not run on all commits -- score normalized to 0
Binary-Artifacts	🟢 10	no binaries found in the repo
Fuzzing	⚠️ 0	project is not fuzzed
Packaging	🟢 10	packaging workflow detected
Pinned-Dependencies	⚠️ 0	dependency not pinned by hash detected -- score normalized to 0
Vulnerabilities	⚠️ 0	35 existing vulnerabilities detected

pip/mistral-common

>= 1.4.0

Unknown

pip/semchunk

2.2.0

Unknown

pip/transformers

4.44.2

🟢 4.4

Details

Check	Score	Reason
Code-Review	🟢 9	Found 28/30 approved changesets -- score normalized to 9
Maintained	🟢 10	30 commit(s) and 19 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices	⚠️ 0	no effort to earn an OpenSSF best practices badge detected
License	🟢 10	license file detected
Branch-Protection	⚠️ -1	internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
Signed-Releases	⚠️ -1	no releases found
Security-Policy	🟢 10	security policy file detected
Dangerous-Workflow	⚠️ 0	dangerous workflow patterns detected
Token-Permissions	⚠️ 0	detected GitHub workflow tokens with excessive permissions
Binary-Artifacts	🟢 10	no binaries found in the repo
SAST	⚠️ 0	SAST tool is not run on all commits -- score normalized to 0
Fuzzing	⚠️ 0	project is not fuzzed
Packaging	🟢 10	packaging workflow detected
Pinned-Dependencies	⚠️ 0	dependency not pinned by hash detected -- score normalized to 0
Vulnerabilities	⚠️ 0	463 existing vulnerabilities detected

pip/langchain-ollama

>= 0.1.3

🟢 5.3

Details

Check	Score	Reason
Code-Review	🟢 7	Found 22/30 approved changesets -- score normalized to 7
Maintained	🟢 10	30 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices	⚠️ 0	no effort to earn an OpenSSF best practices badge detected
License	🟢 10	license file detected
Signed-Releases	⚠️ -1	no releases found
Branch-Protection	🟢 3	branch protection is not maximal on development and all release branches
Security-Policy	🟢 10	security policy file detected
Dangerous-Workflow	🟢 10	no dangerous workflow patterns detected
Token-Permissions	⚠️ 0	detected GitHub workflow tokens with excessive permissions
SAST	⚠️ 0	SAST tool is not run on all commits -- score normalized to 0
Binary-Artifacts	🟢 10	no binaries found in the repo
Fuzzing	⚠️ 0	project is not fuzzed
Packaging	🟢 10	packaging workflow detected
Pinned-Dependencies	⚠️ 0	dependency not pinned by hash detected -- score normalized to 0
Vulnerabilities	⚠️ 0	35 existing vulnerabilities detected

Scanned Manifest Files

pyproject.toml

langchain@>= 0.3.0
langchain-ollama@>= 0.1.3
mistral-common@>= 1.4.0
[email protected]
[email protected]
langchain@>= 0.2.14
semchunk@>= 1.0.1

requirements.txt

langchain-ollama@>= 0.1.3

github-actions · 2024-09-19T08:12:59Z

🎉 This PR is included in version 1.21.0 🎉

The release is available on:

v1.21.0
GitHub release

Your semantic-release bot 📦🚀

Santabot123 and others added 30 commits August 23, 2024 19:00

Added screenshot preparation script for screenshot scraping

49ae56c

Added text_detection.py and updated screenshot_preparation.py

e11f0cd

add __init__.py and docstrings

b4f8ea4

correct the typo and updated select_area_with_ipywidget()

0cf7c44

correct the typo

7e23c3d

remove some comments and image

c0a0e69

Updated requirements.txt

92bec28

remove some comments and image

aa9e85f

updated requirements.txt

90d7549

Merge branch 'pre/beta' of https://github.com/Santabot123/Scrapegraph…

6e9911c

…-ai-with-screenshot-scraping into pre/beta

Update requirements.txt

55a7727

feat: add togheterai

8f615ad

chore(examples): create Together AI examples

34942de

Co-Authored-By: Marco Vinciguerra <[email protected]>

Merge pull request #605 from ScrapeGraphAI/togheter_ai_integration

5f604d1

feat: add togetherai

ci(release): 1.16.0-beta.1 [skip ci]

d7f6036

## [1.16.0-beta.1](v1.15.1...v1.16.0-beta.1) (2024-08-28) ### Features * add togheterai ([8f615ad](8f615ad)) ### chore * **examples:** create Together AI examples ([34942de](34942de))

fix: update generate answernode

c348f67

Merge branch 'screenshot-scraper-fix' into pre/beta

735120d

Merge pull request #606 from Santabot123/pre/beta

405f28e

Screenshot scraping

refactoring of folders

a0d2113

fix: screenshot scraper

388630c

update version

a73573d

Update pyproject.toml

9fd6509

Merge pull request #612 from ScrapeGraphAI/598-1140+-pydantic-validat…

9c2aefa

…ionerror-with-smartscrapergraph fix: update generate answernode

ci(release): 1.16.0-beta.2 [skip ci]

1c37d5d

## [1.16.0-beta.2](v1.16.0-beta.1...v1.16.0-beta.2) (2024-08-31) ### Bug Fixes * update generate answernode ([c348f67](c348f67))

Merge branch 'pre/beta' into temp

0e0b280

Merge pull request #615 from ScrapeGraphAI/temp

86f9442

fixed tool and dependencies

ci(release): 1.16.0-beta.3 [skip ci]

886c987

## [1.16.0-beta.3](v1.16.0-beta.2...v1.16.0-beta.3) (2024-09-01) ### Bug Fixes * pyproject.toml ([360ce1c](360ce1c)) ### CI * **release:** 1.15.2 [skip ci] ([d88730c](d88730c))

add example

f51b155

feat:expose the search engine params to user

8422463

updated token calculation on parsenode

a8b0e4a

LorenzoPaleari and others added 25 commits September 14, 2024 02:06

fix: Refactor code to use CustomOpenAiCallbackManager for exclusive a…

e657113

…ccess to get_openai_callback

Merge pull request #670 from LorenzoPaleari/576-exec-info-misses-nest…

d7afdb1

…ed-graphs Added CustomOpenaiCallback to ensure exclusive access to nested data.

Merge branch 'pre/beta' into temp

5d1fe68

Merge pull request #672 from ScrapeGraphAI/temp

0eec93e

allignment

fix: Add mistral-common dependency

7681a45

Merge pull request #673 from LorenzoPaleari/fix-no-mistral_common-error

cb505ce

fix: Add mistral-common dependency

Merge branch 'main' into pre/beta

c717bb6

fix: fetch_node condition

3f45c17

fix: Error in pyproject dependencies

5b5cb5b

Merge pull request #677 from LorenzoPaleari/fix-pyproject-error

1b85f54

fix: Error in pyproject dependencies

ci(release): 1.20.0-beta.2 [skip ci]

4f8b55d

## [1.20.0-beta.2](v1.20.0-beta.1...v1.20.0-beta.2) (2024-09-17) ### Bug Fixes * Add mistral-common dependency ([7681a45](7681a45)) * Error in pyproject dependencies ([5b5cb5b](5b5cb5b)) * fetch_node condition ([3f45c17](3f45c17))

refactor: Output parser code

28b85a3

feat: updated pydantic to v2

eb89549

feat: added Bedrock and Mistral to exec info

8a37c6b

Merge pull request #679 from LorenzoPaleari/output-parser-update

e6e2ce6

feat: output parser and pydantic update

ci(release): 1.20.0-beta.3 [skip ci]

cca783c

## [1.20.0-beta.3](v1.20.0-beta.2...v1.20.0-beta.3) (2024-09-18) ### Features * updated pydantic to v2 ([eb89549](eb89549)) ### Refactor * Output parser code ([28b85a3](28b85a3))

fix: update pyproject.toml

932412e

Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegra…

1079553

…ph-ai into pre/beta

ci(release): 1.20.0-beta.4 [skip ci]

c81f970

## [1.20.0-beta.4](v1.20.0-beta.3...v1.20.0-beta.4) (2024-09-18) ### Bug Fixes * update pyproject.toml ([932412e](932412e))

Merge pull request #680 from LorenzoPaleari/exec-info-enhanced

95a5ee2

feat: added Bedrock and Mistral to exec info

Update CONTRIBUTING.md

6ae2028

ci(release): 1.20.0-beta.5 [skip ci]

b0fef3f

## [1.20.0-beta.5](v1.20.0-beta.4...v1.20.0-beta.5) (2024-09-18) ### Features * added Bedrock and Mistral to exec info ([8a37c6b](8a37c6b))

Merge branch 'main' into temp

0cdd47e

VinciGit00 merged commit 8b51d50 into main Sep 19, 2024
3 of 4 checks passed

github-actions bot added the released on @stable label Sep 19, 2024

VinciGit00 deleted the temp branch September 22, 2024 14:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pre/beta #682

Pre/beta #682

VinciGit00 commented Sep 19, 2024

github-actions bot commented Sep 19, 2024

github-actions bot commented Sep 19, 2024

Pre/beta #682

Pre/beta #682

Conversation

VinciGit00 commented Sep 19, 2024

github-actions bot commented Sep 19, 2024

Dependency Review

OpenSSF Scorecard

Scanned Manifest Files

github-actions bot commented Sep 19, 2024