Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(c/driver/framework): Add documentation for building a driver using the driver framework #2186

Merged
merged 33 commits into from
Oct 29, 2024

Conversation

paleolimbot
Copy link
Member

@paleolimbot paleolimbot commented Sep 24, 2024

This PR adds a tutorial with how to build a basic driver using the C++ framework. You can read the recipe by downloading the docs.tgz asset from the CI job:

https://github.com/apache/arrow-adbc/actions/runs/11431710883/artifacts/2080179525

Example driver in action:

import os
import pyarrow
from pathlib import Path
from adbc_driver_manager import dbapi


def connect(uri: str):
    build_dir = Path(__file__).parent / "build"
    for lib in [
        "libdriver_example.dylib",
        "libdriver_example.so",
        "driver_example.dll",
    ]:
        driver_lib = build_dir / lib
        if driver_lib.exists():
            return dbapi.connect(
                driver=str(driver_lib.resolve()), db_kwargs={"uri": uri}
            )

    raise RuntimeError("Can't find driver shared object")


with connect(uri=Path(__file__).parent.as_uri()) as con:
    data = pyarrow.table({"col": [1, 2, 3]})
    with con.cursor() as cur:
        cur.adbc_ingest("example.arrows", data, mode="create")

    with con.cursor() as cur:
        cur.execute("SELECT * FROM example.arrows")
        print(cur.fetchall())

    os.unlink(Path(__file__).parent / "example.arrows")
#> [(1,), (2,), (3,)]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm not mistaken we already have cpp_recipe.sh for testing? Can we just fold all the test scripts into one?

(We could even have a single parent CMake that adds the quickstart/driver example as child CMakes, though I can understand if you want the driver example to be fully self-contained there)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a self-contained CMakeLists.txt is more realistic although I'm open to any suggestions here! I mostly just tried to emulate the existing art here which is great!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I'm mostly just saying, we can just have a single shell script to invoke all the CMake builds for recipes. Not too big deal to me though

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I did this (but happy to restructure as required!)

../../../../c/include)
target_link_libraries(adbc_driver_framework PRIVATE nanoarrow::nanoarrow)

# TODO: Do we want any symbol visiblity presets here to ensure that the only one exposed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be nice but we don't have this worked out in the main project yet either do we? Maybe worth punting until then

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed!

target_link_libraries(driver_example PRIVATE adbc_driver_framework
nanoarrow::nanoarrow_ipc)

# TODO: Do we want to have this as part of the example? We could make the validation library
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am +1 to including this in the example - I think the test is helpful to read

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok! I leaned into this comment and made it a part of the recipe!

@paleolimbot paleolimbot marked this pull request as ready for review October 21, 2024 15:24
@paleolimbot paleolimbot requested a review from kou as a code owner October 21, 2024 15:25
@github-actions github-actions bot added this to the ADBC Libraries 15 milestone Oct 21, 2024
Comment on lines 29 to 30
# TODO: This currently depends on about-to-be-released nanoarrow, which allows
# access to both nanoarrow and the IPC reader using a single fetchcontent.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we update this now?

/// Next, we'll bring a few essential framework types into the namespace
/// to reduce the verbosity of the implementation:
///
/// * ``Option``: Options can be set on an ADBC database, connection, and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think something like

:cpp:class:`adbc::driver::Option`

should work now (and eventually result in a proper link to the API docs from the prose docs), though no big deal either way (it can be finicky to set up and test locally)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll give it a try!

@lidavidm
Copy link
Member

Hmm, there's something not right with the scrape...I'll see if I can take a look too

  File "/adbc/docs/source/ext/doxygen_inventory.py", line 209, in <module>
    main()
  File "/adbc/docs/source/ext/doxygen_inventory.py", line 193, in main
    domains = make_fake_domains(args.html_path, args.xml_path, args.url)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/adbc/docs/source/ext/doxygen_inventory.py", line 145, in make_fake_domains
    for domain, name, typ, anchor, url in scrape_links(item_id_to_url, root):
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/adbc/docs/source/ext/doxygen_inventory.py", line 66, in scrape_links
    url = item_id_to_url[anchor]
          ~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'structadbc_1_1driver_1_1Status_1_1Impl'

@paleolimbot
Copy link
Member Author

I ran out of time today to debug but I can take a look tomorrow too!

@lidavidm
Copy link
Member

I took a look and interestingly, Status::Impl is documented in the XML, but not in the HTML.

@lidavidm
Copy link
Member

Ok I think the main issue is that I just haven't implemented enough to also scrape C++ headers well...

@lidavidm
Copy link
Member

Ok, I think that should fix the main things...

@lidavidm
Copy link
Member

Sorry, I pushed a bunch of crap here...let's hope the docs work now 😅

@paleolimbot
Copy link
Member Author

paleolimbot commented Oct 29, 2024

No problem! Thank you! They look great!

Screenshot 2024-10-29 at 10 10 07 AM

@paleolimbot paleolimbot merged commit e93d70f into apache:main Oct 29, 2024
65 checks passed
@paleolimbot paleolimbot deleted the docs-recipe-driver branch October 29, 2024 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants