Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Runtime Shutdown Support - Runtime Thread Cleanup #276

Draft
wants to merge 1 commit into
base: unity-main
Choose a base branch
from

Conversation

scott-ferguson-unity
Copy link

Add support for shutting down worker threads used by the runtime. For IL2CPP on the Mono classlibs we support shutting down all managed code and unloading. The dotnet/runtime classlibs were not built to support this.

For IL2CPP we have intercepted thread waits to make the interruptable so we can shutdown any responsive threads. (It is still up to the end user to ensure that they cleanly shutdown their code). What we largely need to be sure we handle is any threads that the runtime creates/manages.

These include:
1. Threadpool Threads
2. Threadpool Gate Thread
3. Timer Thread
4. Finalizer Thread
5. .NET SigHandler Thread (Non Windows Only)
6. .NET Sockets Thread (Non Windows Only)

For 1-3 our interruptable waits will work - these therads block on managed locks that we handle in the IL2CPP runtime so we can inject exceptions. The finalizer thread is part in the IL2CPP runtime so is under our control.

The .NET SigHandler thread currently appears to only be used to handle Console signals on desktop platforms. We do not support runtime unload on OSX or Linux - so this has not been handled.

The .NET Sockets thread however blocks in libSystemNative on on non- interruptable OS wait (kevent or epoll_wait). We are currently handling this in the IL2CPP runtime by, at shutdown, looking for a thread named ".NET Sockets" and doing a native thread abort. Native thread aborts are dangerous (could lead to crashes/memory leaks/deadlocks) and not all platforms support them.

To handle this this PR adds a new call to libSystemNative, SystemNative_WakeUpSocketEventThread, that will register an OS event (eventfd or pipe) into the events that the .NET Sockets thread waits on so we can wake it up. The .NET Sockets managed code on SocketAsyncEngine.Unix.cs does then need to be changed to check for a flag to determine if it should exit and cleanup its resources.

To try to make special shutdown handling reasonable general a new internal API was added to System.Thread -
RegisterShutdownHandler(Action). This registers a managed callback with the runtime that will be invoked when the runtime is shutting down. This is only implemented for our IL2CPP specific System.Private.CoreLib with an icall that we've added.

The workflow this new API is when starting a new thread that requires cooperative cleanup, call Thread.RegisterShutdownHandler(Action) will the shutdown steps. Then when the runtime shutdowns this action will be invoked and should invoke the shutdown and wait for the thread to exit.

This PR also adds shutdown support to the Timer thread and the Threadpool Gate Thread. As stated above we can still shutdown these threads but it is cleaner to do that cooperatively rather than injecting an Exception at a wait point. And for the Threadpool Gate thread we do want it to shutdown before we start shutting down any worker threads so it doesn't spin and try to create more worker threads to replace the ones that exited. (Note: We could also co-operatively exit the worker threads as well, but that hasn't be explored). The real motivation for adding the shutdown support here was the come up with an API/pattern that was more general and not just tied to the .NET Sockets thread.

TODOS:

  1. Do we have the correct defines for eventfd/pipe support?
  2. Should we remove eventfd and just use pipes?
  3. Should we co-operatlively shutdown TP threads?
  4. Do we need to do anything with the .NET SigHandler Thread

Add support for shutting down worker threads used by the runtime.  For
IL2CPP on the Mono classlibs we support shutting down all managed code
and unloading.  The dotnet/runtime classlibs were not built to support
this.

For IL2CPP we have intercepted thread waits to make the interruptable so
we can shutdown any responsive threads.  (It is still up to the end user
to ensure that they cleanly shutdown their code).  What we largely need
to be sure we handle is any threads that the runtime creates/manages.

These include:
    1. Threadpool Threads
    2. Threadpool Gate Thread
    3. Timer Thread
    4. Finalizer Thread
    5. .NET SigHandler Thread (Non Windows Only)
    6. .NET Sockets Thread (Non Windows Only)

For 1-3 our interruptable waits will work - these therads block on
managed locks that we handle in the IL2CPP runtime so we can inject
exceptions.  The finalizer thread is part in the IL2CPP runtime so
is under our control.

The .NET SigHandler thread currently appears to only be used to handle
Console signals on desktop platforms.  We do not support runtime unload
on OSX or Linux - so this has not been handled.

The .NET Sockets thread however blocks in libSystemNative on on non-
interruptable OS wait (kevent or epoll_wait).  We are currently handling
this in the IL2CPP runtime by, at shutdown, looking for a thread named
".NET Sockets" and doing a native thread abort.  Native thread aborts
are dangerous (could lead to crashes/memory leaks/deadlocks) and not all
platforms support them.

To handle this this PR adds a new call to libSystemNative,
SystemNative_WakeUpSocketEventThread, that will register an OS event
(eventfd or pipe) into the events that the .NET Sockets thread waits
on so we can wake it up.  The .NET Sockets managed code on
SocketAsyncEngine.Unix.cs does then need to be changed to check for a
flag to determine if it should exit and cleanup its resources.

To try to make special shutdown handling reasonable general a new
internal API was added to System.Thread -
RegisterShutdownHandler(Action).  This registers a managed callback
with the runtime that will be invoked when the runtime is shutting down.
This is only implemented for our IL2CPP specific System.Private.CoreLib
with an icall that we've added.

The workflow this new API is when starting a new thread that requires
cooperative cleanup, call Thread.RegisterShutdownHandler(Action) will
the shutdown steps.  Then when the runtime shutdowns this action will be
invoked and should invoke the shutdown and wait for the thread to exit.

This PR also adds shutdown support to the Timer thread and the
Threadpool Gate Thread.  As stated above we can still shutdown these
threads but it is cleaner to do that cooperatively rather than injecting
an Exception at a wait point.  And for the Threadpool Gate thread we do
want it to shutdown before we start shutting down any worker threads so
it doesn't spin and try to create more worker threads to replace the
ones that exited.  (Note: We could also co-operatively exit the worker
threads as well, but that hasn't be explored).  The real motivation
for adding the shutdown support here was the come up with an API/pattern
that was more general and not just tied to the .NET Sockets thread.

TODOS:
1. Do we have the correct defines for eventfd/pipe support?
2. Should we remove eventfd and just use pipes?
3. Should we co-operatlively shutdown TP threads?
4. Do we need to do anything with the .NET SigHandler Thread
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant