Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RT-Migration #45

Merged
merged 36 commits into from
Oct 1, 2024
Merged

RT-Migration #45

merged 36 commits into from
Oct 1, 2024

Conversation

andyli11
Copy link
Contributor

@andyli11 andyli11 commented Sep 25, 2024 •
edited
Loading

Feature:

  • migrate RT sdk logic over
  • create updated jupyter notebooks including:
    • SYNC: pdf_to_markdown, image_to_markdown, docx_to_markdown
    • ASYNC: async_pdf_to_markdown, did not make async tests for other file formats due to overcrowding
  • update README
  • add unittests for both async and sync extraction using Levenshtein Distance algorithm, which calculates the minimum number of single-character edits (insertions, deletions, or substitutions) required to transform one string into another.
  • updated Github actions to run each python version one by one instead of all at the same time to prevent Error: 429 Too Many Requests
  • fixed the pre-commit logic for unittests

Tested:

  • async word/pptx/pdf/image: 98.54% compatibility
  • sync word/pptx/pdf/image: 98.78% compatibility

Todo:

  • pypi release

@andyli11 andyli11 requested a review from Sdddell as a code owner September 25, 2024 05:09
@CambioML
Copy link
Collaborator

please fix the build failure.

@andyli11 andyli11 changed the title [WIP] RT-Migration RT-Migration Sep 25, 2024
CambioML
CambioML reviewed Sep 25, 2024
View reviewed changes
any_parser/__init__.py Show resolved Hide resolved
any_parser/any_parser.py Show resolved Hide resolved
examples/test_sync_example.py Outdated Show resolved Hide resolved
@CambioML
Copy link
Collaborator

Great job to get the test passing now, but I would encourage you to think whether this is a good full coverage tests.

CambioML
CambioML reviewed Sep 26, 2024
View reviewed changes
examples/test_example.py Outdated Show resolved Hide resolved
tests/test.py Outdated Show resolved Hide resolved
tests/test.py Outdated Show resolved Hide resolved
CambioML
CambioML reviewed Sep 27, 2024
View reviewed changes
examples/image_to_markdown.ipynb Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
any_parser/any_parser.py Show resolved Hide resolved
tests/test.py Show resolved Hide resolved
CambioML
CambioML approved these changes Oct 1, 2024
View reviewed changes
Copy link
Collaborator

@CambioML CambioML left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

examples/async_example.ipynb
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a title and short description of what this notebook is for. Can take care of this in future PR

examples/sync_example.ipynb
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a title and short description of what this notebook is for. Can take care of this in future PR

@CambioML CambioML merged commit bd4b8c1 into main Oct 1, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CambioML CambioML CambioML approved these changes

@Sdddell Sdddell Awaiting requested review from Sdddell Sdddell is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@andyli11 @CambioML