Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build GTK 4 binaries #1422

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Conversation

laeubi
Copy link
Contributor

@laeubi laeubi commented Aug 27, 2024

Currently the GTK4 binaries are not build as part of the build, this has several drawbacks:

  1. If anything is adjusted for GTK4 part it might break without notice 2) We have no GTK4 binaries by default

This enables compilation of the gtk4 parts to see at least everything can be compiled.

The following has to be checked:

  • do we have everything to compile for Jenkins Pod
  • do we have everything to compile for Github
  • ...

Copy link
Contributor

github-actions bot commented Aug 27, 2024

Test Results

   383 files   -   102     383 suites   - 102   6m 12s ⏱️ - 2m 2s
 4 140 tests +   16   4 133 ✅ +   15   7 💤 +2  0 ❌  - 1 
12 282 runs   - 4 073  12 199 ✅  - 4 066  83 💤  - 6  0 ❌  - 1 

Results for commit 7d668f6. ± Comparison against base commit b0413a8.

This pull request removes 19 and adds 35 tests. Note that renamed tests count towards both.
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicASCII_dollarSign
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicASCII_emptyString
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicASCII_letterA
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicASCII_letters
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicUTF16LE_null
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicUTF16_AsciiLetters
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicUTF16_Asciiletter
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicUTF16_LotsOfLetters
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicUTF16_letter
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_HeuristicUTF16_letters
…
org.eclipse.swt.graphics.ImageWin32Tests ‑ testImageShouldHaveDimesionAsPerZoomLevel
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_dnd_DND ‑ testByteArrayTransfer
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_dnd_DND ‑ testFileTransfer
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_dnd_DND ‑ testHtmlTransfer
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_dnd_DND ‑ testImageTransfer_fromCopiedImage
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_dnd_DND ‑ testImageTransfer_fromImage
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_dnd_DND ‑ testImageTransfer_fromImageData
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_dnd_DND ‑ testImageTransfer_fromImageDataFromImage
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_dnd_DND ‑ testRtfTransfer
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_dnd_DND ‑ testTextTransfer
…
This pull request removes 1 skipped test and adds 3 skipped tests. Note that renamed tests count towards both.
org.eclipse.swt.tests.gtk.Test_GtkConverter ‑ test_Heuristic_specialSingleCases
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_events_KeyEvent ‑ testEnglishUs_multipleLetters
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_events_KeyEvent ‑ testEnglishUs_unorderedCtrlC
org.eclipse.swt.tests.win32.Test_org_eclipse_swt_events_KeyEvent ‑ testEnglishUs_unpairedKeyUp

♻️ This comment has been updated with latest results.

@laeubi

This comment was marked as outdated.

@@ -10,6 +10,7 @@
*
* Contributors:
* Syntevo - initial API and implementation
* DUMMY CHANGE
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to be reverted before merge

@laeubi
Copy link
Contributor Author

laeubi commented Aug 27, 2024

Now Github succeed and Jenkins fails as expected:

11:31:19  gcc -shared -fPIC  -s -o libswt-cairo-gtk-4966r6.so swt.o cairo.o cairo_structs.o cairo_stats.o `pkg-config --libs-only-L cairo` -lcairo

11:31:19  gcc -O -Wall -fPIC -DSWT_VERSION=4966r6   -DLINUX -DGTK -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include/linux -DJNI64 -Werror   -c -o swt_awt.o swt_awt.c

11:31:19  gcc -shared   -s -o libswt-awt-gtk-4966r6.so swt_awt.o -L/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/lib -ljawt

11:31:19  cp libswt-gtk-4966r6.so libswt-awt-gtk-4966r6.so libswt-pi3-gtk-4966r6.so libswt-cairo-gtk-4966r6.so libswt-atk-gtk-4966r6.so libswt-glx-gtk-4966r6.so libswt-webkit-gtk-4966r6.so /home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/libs

11:31:19  
GTK3 Build succeeded

11:31:19  
Note: When building multiple GTK versions, a cleanup is required (and automatically performed) between them.

11:31:19  
Cleaning up...

11:31:19  rm -f *.o *.so

11:31:19  
Building GTK4 bindings:

11:31:19  gcc -O -Wall -fPIC -DSWT_VERSION=4966r6   -DLINUX -DGTK -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include/linux -DJNI64 -Werror -c swt.c

11:31:19  gcc -O -Wall -fPIC -DSWT_VERSION=4966r6   -DLINUX -DGTK -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include/linux -DJNI64 -Werror   -c -o c.o c.c

11:31:19  gcc -O -Wall -fPIC -DSWT_VERSION=4966r6   -DLINUX -DGTK -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include/linux -DJNI64 -Werror   -c -o c_stats.o c_stats.c

11:31:19  gcc -O -Wall -fPIC -DSWT_VERSION=4966r6   -DLINUX -DGTK -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include/linux -DJNI64 -Werror `pkg-config --cflags gtk4 gtk4-x11 gtk4-unix-print` -DUSE_ASSEMBLER -c callback.c

11:31:19  Package gtk4 was not found in the pkg-config search path.

11:31:19  Perhaps you should add the directory containing `gtk4.pc'

11:31:19  to the PKG_CONFIG_PATH environment variable

11:31:19  Package 'gtk4', required by 'virtual:world', not found

11:31:19  Package 'gtk4-x11', required by 'virtual:world', not found

11:31:19  Package 'gtk4-unix-print', required by 'virtual:world', not found

11:31:23  gcc -shared -fPIC  -s -o libswt-gtk-4966r6.so swt.o c.o c_stats.o callback.o

11:31:23  gcc -O -Wall -fPIC -DSWT_VERSION=4966r6   -DLINUX -DGTK -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include -I/home/jenkins/agent/workspace/eclipse.platform.swt_PR-1422/jdk.resources/include/linux -DJNI64 -Werror `pkg-config --cflags gtk4 gtk4-x11 gtk4-unix-print` -c os.c

11:31:23  Package gtk4 was not found in the pkg-config search path.

11:31:23  Perhaps you should add the directory containing `gtk4.pc'

11:31:23  to the PKG_CONFIG_PATH environment variable

11:31:23  Package 'gtk4', required by 'virtual:world', not found

11:31:23  Package 'gtk4-x11', required by 'virtual:world', not found

11:31:23  Package 'gtk4-unix-print', required by 'virtual:world', not found

11:31:23  In file included from os_structs.h:19,

11:31:23                   from os.c:20:

11:31:23  os.h:30:10: fatal error: gtk/gtk.h: No such file or directory

11:31:23   #include <gtk/gtk.h>

11:31:23            ^~~~~~~~~~~

11:31:23  compilation terminated.

11:31:23  make: *** [make_linux.mak:144: os.o] Error 1

11:31:23  
*** GTK4 Build failed, aborting further actions..

@laeubi laeubi force-pushed the build_gtk4 branch 3 times, most recently from e447b1c to 397da2e Compare August 27, 2024 11:33
@ptziegler
Copy link
Contributor

Given that I wanted to have a look at some of those deprecation errors in the near future, what version of GTK4 do you plan on compiling against?

With #1421, SWT would require at least require 4.10 but then the build would have to continue despite the deprecations. With older versions, the build would succeed but then the PR no longer builds as the new methods don't exist yet.

@akurtakov
Copy link
Member

As GTK4 is WIP it should be fine to require as new version as needed . By the time this is ready for prime time that version would have made it to mainstream distros.

@laeubi
Copy link
Contributor Author

laeubi commented Aug 28, 2024

Main goal is to build it with "what is available right now", if we have this working one can go step further to support a specific version. But still then one need to take care as using "the latest" might simply make it impossible for some linux distributions (e.g LTE) that not always have latest and greatest of libraries, but "latest stable" version.

@laeubi
Copy link
Contributor Author

laeubi commented Aug 28, 2024

Even though GTK4 was added here:

and container build seem to have completed

https://ci.eclipse.org/releng/view/Miscellaneous/job/Build-Docker-images/732/

the build still complains:

Package gtk4 was not found in the pkg-config search path.

@fredg02 anything else that needs to be done so the new container is visible? Can we do anything to verify the change is actually used? I didn't find anything in the log that indicates the actual used container has/version/date/....

@iloveeclipse
Copy link
Member

Main goal is to build it with "what is available right now", if we have this working one can go step further to support a specific version.

We should have target Linux versions specified in the project plan, similar to Java versions.

  • RHEL 9
  • SUSE 15 SP4
  • Ubuntu 24.04
    The minimal GTK4 version could be derived from there.

@akurtakov
Copy link
Member

Main goal is to build it with "what is available right now", if we have this working one can go step further to support a specific version.

We should have target Linux versions specified in the project plan, similar to Java versions.

* RHEL 9

* SUSE 15 SP4

* Ubuntu 24.04
  The minimal GTK4 version could be derived from there.

I would agree with such version targeting when we discuss making GTK 4 default one. Unfortunately we are very far from it so by the time this discussion comes we will most likely have RHEL 10(most likely 11) and Ubuntu 26.04.
Speaking from history - when port to GTK 3 was done we had to give feedback to gtk developers, report bugs, test patches and several times target even development versions in order to manage to complete SWT on GTK3 port. Setting targets based on what current LTS distros ships will most likely make the port far more harder than it should be (and some SWT features even impossible).

@akurtakov
Copy link
Member

One such impossible feature on RHEL 9 would be Browser support as even though there is gtk4 there is no webkitgtk4.

@laeubi
Copy link
Contributor Author

laeubi commented Aug 28, 2024

That's why I said lets first get something compiled, then its easier to iterate, I tried to add some commands to find out actual installed versions for the github runners it is this versions:

libgtk-3-0/jammy-updates,jammy-security,now 3.24.33-1ubuntu2.2 amd64 [installed]
libgtk-3-common/jammy-updates,jammy-security,now 3.24.33-1ubuntu2.2 all [installed,automatic]
libgtk-3-dev/jammy-updates,jammy-security,now 3.24.33-1ubuntu2.2 amd64 [installed]
libgtk-4-1/jammy-updates,jammy-security,now 4.6.9+ds-0ubuntu0.22.04.2 amd64 [installed,automatic]
libgtk-4-bin/jammy-updates,jammy-security,now 4.6.9+ds-0ubuntu0.22.04.2 amd64 [installed,automatic]
libgtk-4-common/jammy-updates,jammy-security,now 4.6.9+ds-0ubuntu0.22.04.2 all [installed,automatic]
libgtk-4-dev/jammy-updates,jammy-security,now 4.6.9+ds-0ubuntu0.22.04.2 amd64 [installed]

@laeubi laeubi force-pushed the build_gtk4 branch 3 times, most recently from ec1b98b to 60dc09c Compare August 28, 2024 08:48
@laeubi
Copy link
Contributor Author

laeubi commented Aug 28, 2024

Jenkins CentOS currently using

gtk3-devel-3.22.30-11.el8.aarch64
webkit2gtk3-jsc-2.38.5-1.el8.aarch64
webkit2gtk3-2.38.5-1.el8.aarch64
gtk-update-icon-cache-3.22.30-11.el8.aarch64
webkit2gtk3-jsc-devel-2.38.5-1.el8.aarch64
gtk2-2.24.32-5.el8.aarch64
gtk2-devel-2.24.32-5.el8.aarch64
gtk3-3.22.30-11.el8.aarch64
libcanberra-gtk3-0.30-18.el8.aarch64
webkit2gtk3-devel-2.38.5-1.el8.aarch64

so it seems docker file change has had no effect...

@fredg02
Copy link
Contributor

fredg02 commented Aug 28, 2024

so it seems docker file change has had no effect...

https://ci.eclipse.org/releng/view/Miscellaneous/job/Build-Docker-images builds docker images from https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/tree/master/cje-production/dockerfiles. I don't know if there is a Jenkins job that actually builds https://github.com/eclipse-platform/eclipse.platform.swt/tree/master/container.

I'd recommend to talk to project members that know more about the platform/swt docker builds. This should be more effective than you or me trying to decipher the inner workings of platform releng builds.

I would also recommend to not use a matrix build while trying to get this running. It makes debugging harder than it should be.

@fredg02
Copy link
Contributor

fredg02 commented Aug 28, 2024

When you have found the right job to build the right docker image, I'd verify that it includes everything that is needed by running it locally. Once that is done, you can specify the sha256 of the image in your Jenkinsfile to make sure that the exact image is pulled.

@fredg02
Copy link
Contributor

fredg02 commented Aug 28, 2024

As noted above, https://ci.eclipse.org/releng/job/Build-Docker-images/ is not building the Dockerfile that @laeubi modified.

@laeubi
Copy link
Contributor Author

laeubi commented Aug 28, 2024

@fredg02

I changed the file here:

And Jenkins config says it runs build-centos_9_swt_build.sh what includes:

set -e

pushd centos-gtk4-mutter/9-swtBuild
echo "Building Centos 9 swt build image"
docker build --pull -t eclipse/platformreleng-centos-swt-build:9 .
popd

do I miss something obvious?

@laeubi
Copy link
Contributor Author

laeubi commented Aug 28, 2024

@sravanlakkimsetti can you maybe shade some light here, the Jenkins file config and collection of shellscripts seems not easy to understand :-\

I even wonder if it would not be easier to have one Jeankinsfile (in git) that includes the shell calls directly instead of many different shell scripts with only a few line in it.

@fredg02
Copy link
Contributor

fredg02 commented Aug 28, 2024

@fredg02
I changed the file here:
eclipse-platform/eclipse.platform.releng.aggregator#2259

Ok. That was not obvious to me, based on the comments here.

@laeubi
Copy link
Contributor Author

laeubi commented Aug 28, 2024

Finally the docker images was updated and now we have GTK4 at the build node, this is the used GTK version and it shows a lot of deprecation warnings:

19:52:21  Name        : gtk4-devel
19:52:21  Version     : 4.12.3
19:52:21  Release     : 2.el9
19:52:21  Architecture: x86_64
19:52:21  Install Date: Wed 28 Aug 2024 05:27:02 PM UTC
19:52:21  Group       : Unspecified
19:52:21  Size        : 13102429
19:52:21  License     : LGPL-2.0-or-later
19:52:21  Signature   : RSA/SHA256, Mon 18 Dec 2023 09:49:21 PM UTC, Key ID 05b555b38483c65d
19:52:21  Source RPM  : gtk4-4.12.3-2.el9.src.rpm
19:52:21  Build Date  : Fri 08 Dec 2023 12:37:08 PM UTC
19:52:21  Build Host  : x86-04.stream.rdu2.redhat.com
19:52:21  Packager    : [email protected]
19:52:21  Vendor      : CentOS
19:52:21  URL         : https://www.gtk.org
19:52:21  Summary     : Development files for GTK

See https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/PR-1422/14/pipeline-console/?selected-node=195
https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/PR-1422/14/pipeline-console/log?nodeId=501

@HannesWell
Copy link
Member

@sravanlakkimsetti can you maybe shade some light here, the Jenkins file config and collection of shellscripts seems not easy to understand :-\

I even wonder if it would not be easier to have one Jeankinsfile (in git) that includes the shell calls directly instead of many different shell scripts with only a few line in it.

That were exaclty my toughts as well when I came along the docker images in the releng-aggregator repo the last time, but I never had the time to do that. But I think there is no reason that speaks against doing the latter if the Job is maintained in the releng.aggregator repo. In order to not lose it again:

Lets have further discussion about improvements of that process there.

@laeubi
Copy link
Contributor Author

laeubi commented Oct 9, 2024

GTK3+4 Builds with the new docker image succeed! But the arm and alike fail due to missing gtk4 :-\

@laeubi
Copy link
Contributor Author

laeubi commented Oct 9, 2024

@fredg02 can I somehow run my docker image on the native (e.g. aarch64 machine) e.g. something like

podTemplate(inheritFrom: 'centos-latest' /* inhert general configuration */, containers: [
	containerTemplate(name: 'swtbuild', image: 'eclipse/platformreleng-debian-swtnativebuild:12',
		resourceRequestCpu:'1000m', resourceRequestMemory:'512Mi',
		resourceLimitCpu:'2000m', resourceLimitMemory:'4096Mi',
		alwaysPullImage: true, command: 'cat', ttyEnabled: true)
]) {
	node('native.builder-'gtk.linux.aarch64) { stage(nativeBuildStageName) { container('swtbuild') { body() } } }
}

@HannesWell
Copy link
Member

If the native build agent's have docker installed, we could maybe launch another docker node from that agent?

And we could maybe even build a docker-image for SWT on the fly using:
https://www.jenkins.io/doc/book/pipeline/docker/#building-containers

If that works as desired/hoped, we could even maintain the docker-image in this repo without the need to push it to docker-hub.

@laeubi
Copy link
Contributor Author

laeubi commented Oct 10, 2024

If that works as desired/hoped, we could even maintain the docker-image in this repo without the need to push it to docker-hub.

One drawback is that building a docker image can take some time and requires download of a lot of stuff, so I think using a prebuild image from a registry can still be good. On the other hand on a "real" machine one can probably benefit from caching (in both cases).

Anyways as ther is no easy "out of the box" way I first will head on to make the lib available on the machines:

Currently the GTK4 binaries are not build as part of the build, this has
several drawbacks:

1) If anything is adjusted for GTK4 part it might break without notice
2) We have no GTK4 binaries by default

This enables *compilation* of the gtk4 parts to see at laest everything
can be compiled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants