Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Build] Generate OSGi metadata for JSVG #107

Merged
merged 1 commit into from
Jan 16, 2025

Conversation

HannesWell
Copy link
Contributor

@HannesWell HannesWell commented Dec 7, 2024

This change has the goal to extend the jar's META-INF/MANIFEST.MF generated during the build to contiain OSGi metadata so that JSVG can be used as bundle in OSGi runtimes out of the box.
This is needed for eclipse-platform/eclipse.platform.swt#1638

To generate OSGi metadata the bnd-tools library is used respectivly it's gradle plugin.
While I know how to configure bnd-tools, for JSVG it should actually be quite simple since there are no dependencies, I have almost zero knowledge about Gradle and how to integrate the bnd-gradle plugin into JSVG's build and failed in my attempts to make it work.
I tried to follow the guide provided at:
https://github.com/bndtools/bnd/tree/master/gradle-plugins#gradle-plugin-for-non-bnd-workspace-builds ff

@weisJ could you please advice me what I need to do in order to make the bnd-gradle plugin work?
Then i can work out a suitable bnd-configuration to generated proper OSGi metadata. Therefore this is currently a work-in-progress draft to get the work started.
Thanks in advance!

@HannesWell
Copy link
Contributor Author

HannesWell commented Dec 7, 2024

@Michael5601 FYI.

@weisJ if it would helpful, I can offer to have video-call, maybe together with @Michael5601, to make this work.
I had a private mail coversiation with Michael about this and he said, you are in contact. If you are interested, maybe Michael could start a private mail conversion for an appointment?

@weisJ
Copy link
Owner

weisJ commented Dec 8, 2024

I have just pushed a commit to your branch which makes the plugin work. You should now be able to configure the bundle properties.

@Michael5601
Copy link

@Michael5601 FYI.

@weisJ if it would helpful, I can offer to have video-call, maybe together with @Michael5601, to make this work. I had a private mail coversiation with Michael about this and he said, you are in contact. If you are interested, maybe Michael could start a private mail conversion for an appointment?

Thank you for the draft and the work you put into this already @HannesWell. Should I still create the appointment or is the problem fixed by the commit of @weisJ?

@HannesWell
Copy link
Contributor Author

I have just pushed a commit to your branch which makes the plugin work. You should now be able to configure the bundle properties.

Thanks a lot. I was now to configure the bundle task, but only through the manifest-attributes as described in the last section of
https://github.com/bndtools/bnd/tree/master/gradle-plugins#instructing-bnd-on-how-to-build-your-bundle

I also tried to use, what I think is the regular way, through a bnd element in the bundle task, but failed miserably. Sorry for that, but I have really zero knowledge about Gradle. I tried hard to find something through internet search, etc. but failed. I you have better advice, I would be very glad.

build.gradle.kts Outdated
Comment on lines 268 to 269
bundle {
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I added the following, the configuration was just ignored and the generated MANIFEST.MF in weisJ-jsvg\jsvg\build\libs\jsvg-1.6.2-SNAPSHOT.jar did had the content that should result from the specified instructions.

Suggested change
bundle {
}
bundle {
bnd("""
Bundle-SymbolicName: com.github.weisj.jsvg
-exportcontents: !*.impl.*,*
-removeheaders: Private-Package,Tool
""")
}

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why it is ignored. The documentation on this isn't very clear in how you are supposed to set things up. I have moved bundle setup into the gradle file of the jsvg itself (not the root) and now it works. See latest commit for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now it works.

That's really awesome. Thank you.

I don't know why it is ignored.

I'm sorry, but I cannot answer that since by Gradle knowledge is almost zero and the more I tried to make this change work the more I felt it's not only close to but zero.

build.gradle.kts Outdated
// bnd instructions to generate OSGi metadata
attributes["Bundle-SymbolicName"] = "com.github.weisj.jsvg"
attributes["-exportcontents"] = "!*.impl.*,*"
attributes["-removeheaders"] = "Private-Package,Tool"
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the bnd instruction to a separate bundle task (?) here

Suggested change
}
}
bundle {
bnd("""
Bundle-SymbolicName: com.github.weisj.jsvg
-exportcontents: !*.impl.*,*
-removeheaders: Private-Package,Tool
""")
}

lead to a build failure with:

Could not determine the dependencies of task ':annotations:assemble'.
> Could not create task ':annotations:sourcesJar'.
   > Extension with name 'bundle' does not exist. Currently registered extension names: [ext]

@HannesWell
Copy link
Contributor Author

The current state is probably not beautiful, but seems to work, although I think some parts can be cleaned up?
If possible, it might still be cleaner to make the bnd element within a bundle task work?

@Michael5601 FYI.
@weisJ if it would helpful, I can offer to have video-call, maybe together with @Michael5601, to make this work. I had a private mail coversiation with Michael about this and he said, you are in contact. If you are interested, maybe Michael could start a private mail conversion for an appointment?

Thank you for the draft and the work you put into this already @HannesWell. Should I still create the appointment or is the problem fixed by the commit of @weisJ?

From my point of view this is not necessary, I hope we can sort out the remaining issues here. But if any one of you is still interested I'm open for it. :)

bundle {
bnd(
"Bundle-SymbolicName: com.github.weisj.jsvg2",
"-exportcontents: !*.impl.*,*",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only added the 2 for testing purposes. I guess -exportcontents relates to which classes are exposed as API? In this case not everything outside impl subpackages should be considered as being public. I have defined some exports in the module-info.java file. Though it is not up to date. I will get back to you with which packages at the moment are considered to be public.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess -exportcontents relates to which classes are exposed as API?

Yes exactly.

In this case not everything outside impl subpackages should be considered as being public. I have defined some exports in the module-info.java file. Though it is not up to date. I will get back to you with which packages at the moment are considered to be public.

It's no problem to update that list. Just let me know what should be in there.

Btw. you can also use bnd-tools to generate the JPMS module-info.class which ensures that OSGi and JPMS metadata are in sync:

If you are interested I can try to make that work too, hoping that my Gradle (non) skills don't hinder me from doing it.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's no problem to update that list. Just let me know what should be in there.

I'll get back to you once I have updated it. If I don't get to it soon enough I'll just adjust it myself.

Btw. you can also use bnd-tools to generate the JPMS module-info.class which ensures that OSGi and JPMS metadata are in sync:

If you are interested I can try to make that work too, hoping that my Gradle (non) skills don't hinder me from doing it.

Does this also ensure that code conforms to the generated module-info.java? It does sound intriguing not having to maintain two places, where these things are defined, but with the current setup the build will fail if the module-info.java is not respected.

@HannesWell
Copy link
Contributor Author

The generated module-info now looks like this:

//  (version 9 : 53.0, no super bit)
 module com.github.weisj.jsvg  {
  // Version: 1.6.2.SNAPSHOT

  requires static com.google.errorprone.annotations;
  requires java.base;
  requires transitive java.desktop;
  requires java.logging;
  requires transitive java.xml;

  exports com.github.weisj.jsvg;
  exports com.github.weisj.jsvg.attributes;
  exports com.github.weisj.jsvg.attributes.paint;
  exports com.github.weisj.jsvg.geometry.size;
  exports com.github.weisj.jsvg.nodes;
  exports com.github.weisj.jsvg.parser;
  exports com.github.weisj.jsvg.parser.css;
  
  Module packages:
    com.github.weisj.jsvg
    com.github.weisj.jsvg.animation
    com.github.weisj.jsvg.animation.interpolation
    com.github.weisj.jsvg.animation.time
    com.github.weisj.jsvg.animation.value
    com.github.weisj.jsvg.attributes
    com.github.weisj.jsvg.attributes.filter
    com.github.weisj.jsvg.attributes.font
    com.github.weisj.jsvg.attributes.paint
    com.github.weisj.jsvg.attributes.stroke
    com.github.weisj.jsvg.attributes.text
    com.github.weisj.jsvg.attributes.value
    com.github.weisj.jsvg.geometry
    com.github.weisj.jsvg.geometry.mesh
    com.github.weisj.jsvg.geometry.noise
    com.github.weisj.jsvg.geometry.path
    com.github.weisj.jsvg.geometry.size
    com.github.weisj.jsvg.geometry.util
    com.github.weisj.jsvg.nodes
    com.github.weisj.jsvg.nodes.animation
    com.github.weisj.jsvg.nodes.container
    com.github.weisj.jsvg.nodes.filter
    com.github.weisj.jsvg.nodes.mesh
    com.github.weisj.jsvg.nodes.prototype
    com.github.weisj.jsvg.nodes.prototype.impl
    com.github.weisj.jsvg.nodes.prototype.spec
    com.github.weisj.jsvg.nodes.text
    com.github.weisj.jsvg.parser
    com.github.weisj.jsvg.parser.css
    com.github.weisj.jsvg.parser.css.impl
    com.github.weisj.jsvg.parser.resources
    com.github.weisj.jsvg.renderer
    com.github.weisj.jsvg.renderer.awt
    com.github.weisj.jsvg.renderer.jdk
    com.github.weisj.jsvg.ui
    com.github.weisj.jsvg.util

}

@HannesWell
Copy link
Contributor Author

HannesWell commented Dec 31, 2024

I have now incooperated your charges and the build works perfectly fine. Thank you very much for your help.

I have also adjusted the exports to match the old, manually maintained module-info.java.
@Michael5601 I'll check in the next days if this already sufficient for our use-case.

For the generated module-info I have relied on the generated requirements as far as possible. I hope the values written above are fine from the assumed uses (I.e.. java.logging doesn't have to be required transitively, does it?)?

@weisJ
Copy link
Owner

weisJ commented Jan 8, 2025

For the generated module-info I have relied on the generated requirements as far as possible. I hope the values written above are fine from the assumed uses (I.e.. java.logging doesn't have to be required transitively, does it?)?

Yes logging isn't required transitively. The generated module-info looks good, except for all "subpackages" of the declared packages also are exported. Especially for the impl packages I don't want to guarantee any API stability.

@HannesWell
Copy link
Contributor Author

For the generated module-info I have relied on the generated requirements as far as possible. I hope the values written above are fine from the assumed uses (I.e.. java.logging doesn't have to be required transitively, does it?)?

Yes logging isn't required transitively. The generated module-info looks good, except for all "subpackages" of the declared packages also are exported. Especially for the impl packages I don't want to guarantee any API stability.

That's relatable, but they are not exported, they are just listed, I assume for informational purpose only. After a quick search I haven't found a way to disable that, but I hope it also doesn't harm.
Only the packages prefixed with export are actually exported:

  exports com.github.weisj.jsvg;
  exports com.github.weisj.jsvg.attributes;
  exports com.github.weisj.jsvg.attributes.paint;
  exports com.github.weisj.jsvg.geometry.size;
  exports com.github.weisj.jsvg.nodes;
  exports com.github.weisj.jsvg.parser;
  exports com.github.weisj.jsvg.parser.css;

I have now made an update to make the bundle resolve in a usual environment and our use case for Eclipse seems to compile for the current state of exports, but from looking at the types used in the signatures of public methods in the SVGDocument I assume that at least the following packages should be exported too:

com.github.weisj.jsvg.animation
com.github.weisj.jsvg.attributes.
com.github.weisj.jsvg.attributes.font
com.github.weisj.jsvg.geometry.size.
com.github.weisj.jsvg.nodes
com.github.weisj.jsvg.renderer
com.github.weisj.jsvg.renderer.awt

Does this make sense or can you tell immediately if more are necessary?

@weisJ
Copy link
Owner

weisJ commented Jan 13, 2025

That's relatable, but they are not exported, they are just listed, I assume for informational purpose only. After a quick search I haven't found a way to disable that, but I hope it also doesn't harm. Only the packages prefixed with export are actually exported: [...]

I actually misread the Module packages: part. Simply listing them doesn't bother me :)

I have now made an update to make the bundle resolve in a usual environment and our use case for Eclipse seems to compile for the current state of exports, but from looking at the types used in the signatures of public methods in the SVGDocument I assume that at least the following packages should be exported too:

com.github.weisj.jsvg.animation
com.github.weisj.jsvg.attributes.
com.github.weisj.jsvg.attributes.font
com.github.weisj.jsvg.geometry.size.
com.github.weisj.jsvg.nodes
com.github.weisj.jsvg.renderer
com.github.weisj.jsvg.renderer.awt

Does this make sense or can you tell immediately if more are necessary?

Looks good to me.

@HannesWell
Copy link
Contributor Author

assume that at least the following packages should be exported too:

com.github.weisj.jsvg.animation
com.github.weisj.jsvg.attributes.
com.github.weisj.jsvg.attributes.font
com.github.weisj.jsvg.geometry.size.
com.github.weisj.jsvg.nodes
com.github.weisj.jsvg.renderer
com.github.weisj.jsvg.renderer.awt

Does this make sense or can you tell immediately if more are necessary?

Looks good to me.

Great. I have added them and also incorporated your update of the module-info from today.

After another round of testing this is ready from my point of view.

@HannesWell HannesWell marked this pull request as ready for review January 14, 2025 18:04
@HannesWell
Copy link
Contributor Author

Additionally I have done a greater analysis which packages are used in the public (API) signatures of which JSVG packages and only based on that metric only the following packages could be hidden:

  • com.github.weisj.jsvg.geometry.noise
  • com.github.weisj.jsvg.geometry.util
  • com.github.weisj.jsvg.parser.css.impl
  • com.github.weisj.jsvg.renderer.jdk
  • com.github.weisj.jsvg.ui

And I assume prototype is also not intended for public usage?

  • com.github.weisj.jsvg.nodes.prototype
  • com.github.weisj.jsvg.nodes.prototype.impl
  • com.github.weisj.jsvg.nodes.prototype.spec

All packages not mentioned are referenced in public API of other JSVG packages.
And if one adds JSVG to the module-path or uses it (with this change) in an OSGi runtime a user wont be able to use the types defined in package that are not exported which means class-loading might fail.
That being said, at least for our use-case in Eclipse I have tested it that the current set of exported packages is sufficient and more could still be added later on demand.

@weisJ
Copy link
Owner

weisJ commented Jan 14, 2025

Additionally I have done a greater analysis which packages are used in the public (API) signatures of which JSVG packages and only based on that metric only the following packages could be hidden:

  • com.github.weisj.jsvg.geometry.noise
  • com.github.weisj.jsvg.geometry.util
  • com.github.weisj.jsvg.parser.css.impl
  • com.github.weisj.jsvg.renderer.jdk
  • com.github.weisj.jsvg.ui

I see these export statements as a way to document which part of the API I see as being stable and dependable on by other code (which doesn't control the specific version of the library). Just because a type is "exposed" (in the sense that it is a parentclass, interface, parameter type etc.) through a class in the public API shouldn't be understood at the type itself being part of the public API. This is some unfortunate side effect of how the project started.

Note that com.github.weisj.jsvg.ui is a new package which should be exposed as it contains a helper class for implementing animations for consumers of the library.

And I assume prototype is also not intended for public usage?

  • com.github.weisj.jsvg.nodes.prototype
  • com.github.weisj.jsvg.nodes.prototype.impl
  • com.github.weisj.jsvg.nodes.prototype.spec

Yes.

All packages not mentioned are referenced in public API of other JSVG packages. And if one adds JSVG to the module-path or uses it (with this change) in an OSGi runtime a user wont be able to use the types defined in package that are not exported which means class-loading might fail. That being said, at least for our use-case in Eclipse I have tested it that the current set of exported packages is sufficient and more could still be added later on demand.

Most people won't need anything more than what is currently exposed. If there is need for making further classes available this can always happen in the future.

Copy link
Owner

@weisJ weisJ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you run spotlessApply this should be ready to merge 👍🏼

@weisJ
Copy link
Owner

weisJ commented Jan 15, 2025

One additional question. Where can I find the generated source for the module-info? The compiled class is included in the jar but the decompiled file doesn't include com.github.weisj.jsvg.annotations or org.jetbrains.annotations. Also com.google.errorprone.annotations isn't indicated as being static. Though maybe this is a decompilation error.

Generated OSGi metadata into JSVG's META-INF/MANIFEST.MF using the
bnd-lib gradle plugin.

Additionally use the bnd-gradle-plugin to also generated the JPMS
module-info.class for JSVG. This avoids manual maintenance and ensures
that OSGi and JPMS metadata are always in sync.

Co-authored-by: Jannis Weis <[email protected]>
@HannesWell
Copy link
Contributor Author

If you run spotlessApply this should be ready to merge 👍🏼

Awesome! Done.

@HannesWell
Copy link
Contributor Author

One additional question. Where can I find the generated source for the module-info? The compiled class is included in the jar but the decompiled file doesn't include com.github.weisj.jsvg.annotations or org.jetbrains.annotations. Also com.google.errorprone.annotations isn't indicated as being static. Though maybe this is a decompilation error.

Unfortunately I also didn't find a way to output the source of the generated module-info and only relayed on the de-compiled form Eclipse showed me.

I see error-prone annotations marked as static:
requires static com.google.errorprone.annotations

But yes the other annotation-packages aren't mentioned at all, probably because they have retention policy source and bnd-lib therefore ignores it (at least the jsvg-annotation do have that policy). But since they have retention policy source I don't see a problem for not listing them. Since the module-info doesn't drive the build here I'm not aware of any use case for static requirements. Or do you need them for another reason?

I see these export statements as a way to document which part of the API I see as being stable and dependable on by other code (which doesn't control the specific version of the library). Just because a type is "exposed" (in the sense that it is a parentclass, interface, parameter type etc.) through a class in the public API shouldn't be understood at the type itself being part of the public API. This is some unfortunate side effect of how the project started.

The problem with this approach is, when being used with strict module-systems like JPMS or OSGi, then a class who's package is not exported cannot be loaded. So in case the parent of a public API class resides in a not-exported package I'm very sure this will lead to class-loading errors in OSGi or JPMS. But as said, as far as I've tested it this doesn't happen in our use-case, so I'm also fine how it is now.
At when targeting eclipse one can add the ;x-internal:=true directive to an import to generate a warning at code that references an element from that package but not fail the class-loading.

@weisJ
Copy link
Owner

weisJ commented Jan 15, 2025

Unfortunately I also didn't find a way to output the source of the generated module-info and only relayed on the de-compiled form Eclipse showed me.

I see error-prone annotations marked as static: requires static com.google.errorprone.annotations

Then this seems to be a decompilation error on my side.

But yes the other annotation-packages aren't mentioned at all, probably because they have retention policy source and bnd-lib therefore ignores it (at least the jsvg-annotation do have that policy).

Yes the jsvg annotations do, however the jetbrains annotations have class retention. Not sure if this might introduce issues during class loading. Maybe annotation classes only get loaded if they are accessed reflectively, which is not the case here, as the packages aren't opened?

But since they have retention policy source I don't see a problem for not listing them. Since the module-info doesn't drive the build here I'm not aware of any use case for static requirements. Or do you need them for another reason?

I do not need them. Just want to be safe it doesn't cause issues when using the library with JPMS.

[...]

The problem with this approach is, when being used with strict module-systems like JPMS or OSGi, then a class who's package is not exported cannot be loaded. So in case the parent of a public API class resides in a not-exported package I'm very sure this will lead to class-loading errors in OSGi or JPMS.

My understanding (at least on the JPMS side, I don't have experience with OSGi) is that having those classes not exported will simply prohibit consumers to from calling the involved methods e.g. because they cannot instantiate the argument type or use the return type. It won't lead to any class loading error as long as the class compiles. This is my experience from java.desktop, which sometimes does this same thing as well.

Though as I said the current state is rather unfortunate and I meant to clean up the public interface for some time now. Just didn't find the time to do so.

@HannesWell
Copy link
Contributor Author

But yes the other annotation-packages aren't mentioned at all, probably because they have retention policy source and bnd-lib therefore ignores it (at least the jsvg-annotation do have that policy).

Yes the jsvg annotations do, however the jetbrains annotations have class retention. Not sure if this might introduce issues during class loading. Maybe annotation classes only get loaded if they are accessed reflectively, which is not the case here, as the packages aren't opened?

I'm not sure when exactly they got loaded, but at least no exception is thrown if one tries to access them reflectively and they fail to load. In this regard annotations are special.

The problem with this approach is, when being used with strict module-systems like JPMS or OSGi, then a class who's package is not exported cannot be loaded. So in case the parent of a public API class resides in a not-exported package I'm very sure this will lead to class-loading errors in OSGi or JPMS.

My understanding (at least on the JPMS side, I don't have experience with OSGi) is that having those classes not exported will simply prohibit consumers to from calling the involved methods e.g. because they cannot instantiate the argument type or use the return type. It won't lead to any class loading error as long as the class compiles. This is my experience from java.desktop, which sometimes does this same thing as well.

I think that depends on when exactly classes are loaded and this probably differs on the exact use-case of a class (e.g. parent-class or type of a method argument or field, etc.). But one has to pay attention for the small details than.
But as you said, such errors can be fixed later if they occur. I'm satisfied with the current state and would be glad if this would be submitted and released. :)

@weisJ weisJ merged commit 9ee5481 into weisJ:master Jan 16, 2025
7 checks passed
@HannesWell HannesWell deleted the osgi-metadata branch January 16, 2025 16:33
@HannesWell
Copy link
Contributor Author

Thank you @weisJ for having this and all your help to make this work!

Can you already tell when you will create the next release of JSVG so that we can consume this?

@Michael5601
Copy link

Thank you for your work guys, I don't understand most of what you did but I am glad that it is working now.

@weisJ
Copy link
Owner

weisJ commented Jan 18, 2025

Can you already tell when you will create the next release of JSVG so that we can consume this?

Currently in the process of writing release notes and documentation on some of the new features. New version should be out by tuesday.

@HannesWell
Copy link
Contributor Author

New version should be out by tuesday.

Awesome. Thanks in advance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants