ResourcesPlugin: Multithreaded lazy start #1219
Conversation
Test Results 198 files - 438 198 suites - 438 39m 12s ⏱️ -42s Results for commit 35c4169. ± Comparison against base commit adb8427. This pull request removes 1035 tests.♻️ This comment has been updated with latest results. |
|
.../bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java
Outdated
Show resolved
Hide resolved
.../bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java
Show resolved
Hide resolved
| @@ -475,7 +483,12 @@ private static String getSystemEncoding() { | |||
| * @return the single instance of this plug-in runtime class | |||
| */ | |||
| public static ResourcesPlugin getPlugin() { | |||
| return plugin; | |||
| ResourcesPlugin p = plugin; | |||
There was a problem hiding this comment.
This is wrong as it will prevent the workspace from become available unless someone calls this static method
There was a problem hiding this comment.
i added new Thread(ResourcesPlugin::getPlugin, "Async ResourcesPlugin start").start(); to start() - ok?
There was a problem hiding this comment.
Not really why do you do the init in this method and not in the activate that makes no sense to me at all...if you want to defer the tracking to happen in a different thread (for whatever reason) why not simply wrap the open call in a new thread right there?
There was a problem hiding this comment.
Other plugins cal the static as soon as the Resourceplugin is started and a synchronize needs to wait for completion to finish.
There was a problem hiding this comment.
This is most likely not correct, it is the other way round. A plugin is starting, calling the static method and this triggers the activator to run and this then is automatically synchronized by the classloader lock already, so why one wants to break this behavior and instead add another synchronization.
These static access is actually bad and has lead to numerous problem in the past, so now requiring this for the startup of the plugin is wrong in many ways and you probably want to better sync on the project loading if projects are accessed.
There was a problem hiding this comment.
show me that THIS implementation makes an error, not yours. It does not even be a junit. just put breakpoints such that it fails if executed in a predefined order of your choice. But it boils down to that synchronize() just works: only one thread will do the init without waiting and all others wait.
The problem is simple that Eclipse on Windows needs >10 sec to start and >2 of them are caused by loading the projects.
loading them in parallel speeds that up.
There was a problem hiding this comment.
show me that THIS implementation makes an error, not yours.
If you show me that MY implementation makes an error and not YOURS I might be convinced... longer loading times are not an error, you can for sure make the loading IO of MarkerManager parallel, but not by changing the Activator.
There was a problem hiding this comment.
Optimizing loading time of IDE is a valid and desired topic. Giving only spurious arguments does not help and contradict our code of conduct.
There was a problem hiding this comment.
Claiming other wasting your time and their implementation is failed is also "contradict our code of conduct" .. I gave you several insight and ways to archive your goal differently so please don't claim any contradiction anytime someone does not agree with you.
There was a problem hiding this comment.
@jukzi : startup performance and functional integrity of code are two different things.
Independently how fast this PR will make startup for some use case, if it will be broken for other use case, it is not acceptable.
And as you propose the change, it is your duty to proof it is safe and works, assuming current state is not broken (which is the case).
jukzi
force-pushed
the
ResourcesPlugin
branch
3 times, most recently
from
be22e6f
to
c584596
Compare
| @@ -542,7 +562,7 @@ public void start(BundleContext context) throws Exception { | |||
| workspaceInitCustomizer); | |||
| plugin = this; // must before open the tracker, as this can cause the registration of the | |||
| // workspace and this might trigger code that calls the static method then. | |||
| instanceLocationTracker.open(); | |||
| new Thread(this::initialized, "Async ResourcesPlugin start").start(); //$NON-NLS-1$ | |||
There was a problem hiding this comment.
This is still wrong, the contract of the Resources plugin is that it tracks the workspace in the activator, as the activation usually (but that's not guaranteed!) happens by a lazy-activation and in this case must be guarded by the classloader lock.
There was a problem hiding this comment.
I agree with Cristoph. The code expected to have workspace initialized after activator is done, not asynchronously after that.
Please put breakpoint at org.eclipse.core.internal.resources.Workspace.open(IProgressMonitor) and check that this is executed in context of org.eclipse.core.resources.ResourcesPlugin.start(BundleContext) call.
Initialization of ResourcesPlugin should be changed very carefully, there are different actors playing here and not everything can be validated via automated tests.
| "Error with project " + project.getName(), e); //$NON-NLS-1$ | ||
| } | ||
| return null; | ||
| }).filter(Objects::nonNull).toArray(IStatus[]::new)).get(); |
There was a problem hiding this comment.
You must not call get() here as it blocks, and you don't want to be blocked. Instead you should either return a Stream<..> with an onClose( ... ), a Future<..> or similar or even better a CompletableFuture<..> then you can use that in an event driven non blocking way.
| @@ -542,7 +562,7 @@ public void start(BundleContext context) throws Exception { | |||
| workspaceInitCustomizer); | |||
| plugin = this; // must before open the tracker, as this can cause the registration of the | |||
| // workspace and this might trigger code that calls the static method then. | |||
| instanceLocationTracker.open(); | |||
| new Thread(this::initialized, "Async ResourcesPlugin start").start(); //$NON-NLS-1$ | |||
There was a problem hiding this comment.
I agree with Cristoph. The code expected to have workspace initialized after activator is done, not asynchronously after that.
Please put breakpoint at org.eclipse.core.internal.resources.Workspace.open(IProgressMonitor) and check that this is executed in context of org.eclipse.core.resources.ResourcesPlugin.start(BundleContext) call.
Initialization of ResourcesPlugin should be changed very carefully, there are different actors playing here and not everything can be validated via automated tests.
Yes because of that any code that calls into resource plugin while the tracker.open is runs is not an indication of something is blocking, it means that the synchronization is actually effective, synchronize after that anything can happen (including it works, deadlock, get an exception) and you maybe just notice that randomly. If one want to improve that there are some ways (and that was already performed by many components but still work in progress):
|
It is not clear what you try say here. ||: Please give a concrete example how it can go wrong :|| |
Reads all projects in parallel. As this was done during ResourcePlugin.start() it had to be deferred as multithreaded classloading during BundleActivator#start(BundleContext) is not supported. All that happens while splash screen still shown.
|
To continue here, you could ensure that the parallel reading finishes before the Activator is left @jukzi. This would still read in parallel but keep the contract. |
I would love todo, and i had no plan to not so, but it turns out it is not possible, because using the resourceplugin classes in another thread before the activator finished can (and will very often) cause a deadlock. |
This PR does not fix that it simply breaks out the classloader synchronization, another thread is not allowed to load additional classes (because that's what lazy-activation is ensuring) that does not mean that the current thread is not allowed to offload some work to other threads. And that's what I have already written here you can:
both things will let you escape out of your classlaoderlock,improves the code and do not require to break the lazy-activation. |
|
BTW the timeout (after 5 or 30sec) looks like Can be found for other plugins on stackoverflow aswell and the solution there was for example: i.e starting a asynchronus thread in the BundleActivator.start() and now guess who create that commit? |
One should not compare apples with oranges. Here we are talking about code contract that is as old as Eclipse is, in a bundle that one of the basic building blocks of the SDK with lot of consumers that relied on the existing behavior, implicitly or explicitly. Trying to gain some (not even quantified) startup improvement by ignoring possible compatibility break is not an option. |
|
@jukzi As mentioned before you can fork a job for any loading in any component (if it is guaranteed that before the component access the results is joining on the job) and for example CheckMissingNaturesListener does exactly that, but here you do fork the whole workspace init into an arbitrary thread and because of that sync on arbitrary other threads. Ans especially the case where lazy-activation thru calls to these methods happens because on classlaoding is a special case of activation, that is even problematic because it could result in "activation rush", but this is not to be fixed on the Activator side, it must be fixed on the Caller site! So again, if any code calls |
The instanceLocationTracker was introduced just 2 year ago. I Ionly added the the async initialisation thread to make that still work even though barely nobody uses it but only the static method are used. For example Eclipse IDE and all its tests work without it, because the Resourceplugin is started in the UI thread as one of the first plugins anyway.
So what exactly do you think is breaking here? |
More exactly it is implicitly started in IDEApplication.start(IApplicationContext) as the IDEWorkbenchAdvisor uses IContainer from org.eclipse.core.resources |
Please look how the code was before it simply FAILED, but it never did anything async... thats the point and as said you have two options:
that's not true, this is one possible entry point it could be started earlier, e.g. if there is a Instance Area saved/set already set before and a bundle wants a workspace it will be started before the IDE shows up. |
Reads all projects in parallel. As this was done during ResourcePlugin.start() it had to be deferred as multithreaded classloading during BundleActivator#start(BundleContext) is not supported.
All that happens while splash screen still shown.