Extension vs Library on Linux

[arcivanov]

Could please someone make sense of a setuptools.extension.Library to me, especially the way it's supposed to work on Linux?
It generates an ar library named lib<x>.cpython-<pyver>-<arch>-linux-gnu.a.

What is/was its purpose and why it generates a Python-suffix-styled object archive that is guaranteed to not be usable at runtime?

[abravalheri]

If I am not wrong, they are used to compile static libraries that will be linked against an extension module. So it is like an intermediate build artifact.

[arcivanov]

That's setup(libraries=), you can't even get ext_modules=[Library] into the dependency path because it's not an lib<x>.a which automatically adds as -la but lib<x>.cpython-<pyver>-<arch>-linux-gnu.a varies by pyver or arch so you can't really add it statically to the library dependencies of the extension.

[abravalheri]

Yes. They are meant to be used to populate setup(libraries=[Library(...),...]), right?

[arcivanov]

No, Library is a form of Extension and is used with ext_modules only. I have supplied an example below.

[arcivanov]

Here's an example (disregard globs):

setup(name="test",
      version="1.0.0",
      description="Python test module",
      author="Karellen, Inc.",
      author_email="[email protected]",
      ext_modules=[Library("shlib",
                           ["src/shlib/*.c", "src/shlib/**/*.c"]),
                   Extension("test",
                             ["src/module/*.c", "src/module/**/*.c"],
                             ),
                   ],
      libraries=[("alib", {"sources": ["src/alib/*.c", "src/alib/**/*.c"]})
                 ],
      script_args=["-vvv", "build_clib", "-f", "build_ext", "-f", "-d", "bdist_wheel"]
      )

[arcivanov]

building 'alib' library
llvm-ar rcs build/temp.linux-x86_64-cpython-312/libalib.a build/temp.linux-x86_64-cpython-312/src/alib/alib.o
running build_ext
building 'shlib' extension
clang -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC --save-temps=obj -fno-discard-value-names -I/home/arcivanov/.pyenv/versions/clang-build-ext-3.12/include -I/home/arcivanov/.pyenv/versions/3.12.0/include/python3.12 -c src/shlib/shlib.c -o build/temp.linux-x86_64-cpython-312/src/shlib/shlib.o
llvm-ar rcs build/lib.linux-x86_64-cpython-312/libshlib.cpython-312-x86_64-linux-gnu.a build/temp.linux-x86_64-cpython-312/src/shlib/shlib.o
building 'test' extension
clang -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC --save-temps=obj -fno-discard-value-names -I/home/arcivanov/.pyenv/versions/clang-build-ext-3.12/include -I/home/arcivanov/.pyenv/versions/3.12.0/include/python3.12 -c src/module/module.c -o build/temp.linux-x86_64-cpython-312/src/module/module.o
clang -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC --save-temps=obj -fno-discard-value-names -I/home/arcivanov/.pyenv/versions/clang-build-ext-3.12/include -I/home/arcivanov/.pyenv/versions/3.12.0/include/python3.12 -c src/module/subdir/module1.c -o build/temp.linux-x86_64-cpython-312/src/module/subdir/module1.o
clang -shared -L/home/arcivanov/.pyenv/versions/3.12.0/lib -Wl,-rpath,/home/arcivanov/.pyenv/versions/3.12.0/lib -L/home/arcivanov/.pyenv/versions/3.12.0/lib -Wl,-rpath,/home/arcivanov/.pyenv/versions/3.12.0/lib build/temp.linux-x86_64-cpython-312/src/module/module.o build/temp.linux-x86_64-cpython-312/src/module/subdir/module1.o -L/home/arcivanov/.pyenv/versions/3.12.0/lib -Lbuild/temp.linux-x86_64-cpython-312 -lalib -o build/lib.linux-x86_64-cpython-312/test.cpython-312-x86_64-linux-gnu.so

[abravalheri]

No, Library is a form of Extension and is used with ext_modules only. I have supplied an example below.

Ok, I see... sorry for the confusion

---

Looking at the file history, we can tell that initially (long time ago) the functionality was planned to build shared library, but an early commit reverted to static libraries outside of Windows: 5d73220 (probably because it was not working properly, or initial tests were failing).

It seems that there was an intention to investigate this more in future changes, but I guess no other committer got the experience or managed to advance the functionality.

[arcivanov]

So I would be correct in assuming this is a half-baked feature that doesn't really have a practical use on *nix systems and is either incomplete or broken?

[abravalheri]

In closer inspection of the file, we can see it states that Library produces static libraries (.a) on Linux and DLLs on Windows. If that is not the use case you have in your hands, so definitely the feature is not for you.

Since the piece of code is kind of old, the only thing that we can assume for now is that the original author decided that it was good enough as it is for his use case, and reduce the scope of the implementation.

If any user is interested in implementing/improving the feature or documenting behaviour, the best way of doing that would be via PRs (setuptools is a community project based on volunteering, any help is always welcome).

[arcivanov]

The library used is not usable since the name of the library produced includes the entire galore of CPython version, platform and arch attributes, e.g. libshlib.cpython-312-x86_64-linux-gnu.a.
I'm not trying to use this feature but augment setuptools and cover all scenarios: libraries, Extension and Library.
And given what I'm observing I'm not sure how anybody uses it (if anybody does), since the name of the library changes depending on python version, arch and os, i.e. the library name dependency has to be computed at runtime.

[arcivanov]

Do you know of any use-cases for this feature? It's definitely not even documented in setuptools documentation.