Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compnerd/1324.41.2 updates #584

Open
wants to merge 30 commits into
base: rokhinip-darwin-libdispatch-1324.41.2-merge-main
Choose a base branch
from

Conversation

compnerd
Copy link
Member

No description provided.

@compnerd
Copy link
Member Author

CC: @rokhinip @das @MadCoder @3405691582

@compnerd
Copy link
Member Author

@swift-ci please test

@compnerd compnerd changed the title Compnerd/1324.41.2 updates compnerd/1324.41.2 updates Nov 12, 2021
@3405691582
Copy link
Contributor

FWIW, still getting (on Linux)

.../tests/dispatch/mach_private.h:39:10: fatal error: 'mach/mach.h' file not found
.../tests/dispatch/queue_private.h:540:2: error: unknown type name 'qos_class_t'
.../os/voucher_private.h:131:1: error: unknown type name '__header_always_inline'

amongst other errors.

@compnerd
Copy link
Member Author

@3405691582 yes, I expected as much, I wanted to get an enumeration of where Linux stands compared to Windows. I still have ~3-4 more changes.

@3405691582
Copy link
Contributor

Sure, I thought I'd mention because the CI output doesn't look very helpful :)

@compnerd
Copy link
Member Author

I agree, the CI results are "helpful" (not surprising, this is trying to merge into a branch). I think that I might be reaching a fixed point where I might be unsure on how to proceed. There are some items which I don't really understand and cannot fix as a result. The biggest offenders are the dispatch_channel_async and dispatch_channel_async_f aliases in src/queue.c.

This adjusts the dispatch debug logging to use to `%llx` for the
conversion of some values.  This avoids a format conversion warning on
Windows.
clang objects to the return from function marked as noreturn.  Add in a
marker to indicate that the return path is dead.  This should be
equivalent to the existing code but silences the compiler warning.
Use `long long` rather than `long` for the value being asserted.  This
is important as there are many places where we use the assertions on
pointer values, which would be truncated on a LLP64 environment.  This
helps clean up the warning spew on Windows.
…ooks_s`

This type is referenced before the declaration.  This hoists the
declaration earlier to have the type available when used.
Use the internal `DISPATCH_SIZEOF_POINTER` rather than `__LP64__` which
does not account for LLP64 environments.
There are three different `base.h` headers, one meant for Apple
platforms (`base.h`), one for Windows (`generic_win_base.h`), and one
for Unix systems (`generic_unix_base.h`).  Conditionally include the
correct one for the build target as we do throughout the rest of
dispatch for portability.
Follow the advice from @MadCoder and limit workgroup headers to Apple
targets only.  This is required to restore the ability to build dispatch
on other platforms.
This adds additional macros from base.h to the windows and unix variants
to allow building dispatch on non-Apple targets.   This ensures that we
have:

- `SPI_AVAILABLE`
- `SPI_DEPRECATED`
- `OS_REFINED_FOR_SWIFT`
- `OS_SWIFT_NAME`

consistently across all the targets.
This function was not previously used in all codepaths and was
unavailable on Windows.  This replicates the shim from the Unix path to
Windows.
This removes some `mach_port_t` usage in the eventlink support.
Additionally guard the inclusion of a mach header with `HAVE_MACH` to
allow building on non-Apple platforms.
The default backing for `enum` types on Windows are signed values, and
the enumerators are handled via integer promotion to the type.
Explicitly type the enums to ensure that the type is correct.
`typeof` is a GNU extension, use the reserved spelling for the keyword.
This allows clang to properly deal with this symbol on Windows builds.
The `qos_class_t` is typealiased as `disaptch_qos_class_t` in
`dispatch/object.h` to allow portability for platforms which do not vend
the type.  Use this to allow building the new interfaces on Windows.
Because `_dispatch_preemption_yield` is used an expression as:

~~~
((void) (q), _dispatch_preemption_yield(n))
~~~

we need to convert the expansion to an expression.  We _could_ also just
wrap the invocation into a parenthesis as an alternative approach, but
this should be safe enough to let us get away with it.
Since we build on non-XNU environments, we cannot rely on osfmk to
provide the definition for `__header_always_inline`.  Instead adopt
`DISPATCH_ALWAYS_INLINE` to achieve the desired effect.
This is adopted from osfmk to provide the `os_atomic_init` macro which
still relies on the C11 atomics.
@compnerd
Copy link
Member Author

S:\b\2>ninja test
[0/1] Running tests...
Test project S:/b/2
      Start  1: dispatch_apply
 1/22 Test  #1: dispatch_apply ...................   Passed    0.18 sec
      Start  2: dispatch_api
 2/22 Test  #2: dispatch_api .....................   Passed    0.14 sec
      Start  3: dispatch_debug
 3/22 Test  #3: dispatch_debug ...................   Passed    0.17 sec
      Start  4: dispatch_queue_finalizer
 4/22 Test  #4: dispatch_queue_finalizer .........   Passed    1.19 sec
      Start  5: dispatch_overcommit
 5/22 Test  #5: dispatch_overcommit ..............   Passed    1.29 sec
      Start  6: dispatch_context_for_key
 6/22 Test  #6: dispatch_context_for_key .........   Passed    0.16 sec
      Start  7: dispatch_after
 7/22 Test  #7: dispatch_after ...................   Passed    9.17 sec
      Start  8: dispatch_timer
 8/22 Test  #8: dispatch_timer ...................   Passed    2.27 sec
      Start  9: dispatch_timer_short
 9/22 Test  #9: dispatch_timer_short .............   Passed    1.25 sec
      Start 10: dispatch_timer_timeout
10/22 Test #10: dispatch_timer_timeout ...........   Passed    6.18 sec
      Start 11: dispatch_sema
11/22 Test #11: dispatch_sema ....................   Passed    0.30 sec
      Start 12: dispatch_timer_bit31
12/22 Test #12: dispatch_timer_bit31 .............   Passed    2.34 sec
      Start 13: dispatch_timer_bit63
13/22 Test #13: dispatch_timer_bit63 .............   Passed    1.29 sec
      Start 14: dispatch_timer_set_time
14/22 Test #14: dispatch_timer_set_time ..........   Passed    2.60 sec
      Start 15: dispatch_data
15/22 Test #15: dispatch_data ....................   Passed    0.20 sec
      Start 16: dispatch_io_muxed
16/22 Test #16: dispatch_io_muxed ................   Passed    1.23 sec
      Start 17: dispatch_io_net
17/22 Test #17: dispatch_io_net ..................   Passed    0.47 sec
      Start 18: dispatch_io_pipe
18/22 Test #18: dispatch_io_pipe .................   Passed   10.48 sec
      Start 19: dispatch_io_pipe_close
19/22 Test #19: dispatch_io_pipe_close ...........   Passed    0.16 sec
      Start 20: dispatch_select
20/22 Test #20: dispatch_select ..................   Passed    0.22 sec
      Start 21: dispatch_c99
21/22 Test #21: dispatch_c99 .....................   Passed    0.16 sec
      Start 22: dispatch_plusplus
22/22 Test #22: dispatch_plusplus ................   Passed    0.18 sec

100% tests passed, 0 tests failed out of 22

Total Test time (real) =  41.74 sec

@3405691582
Copy link
Contributor

Linux test results:

$ ninja test
[0/1] Running tests...
Test project .../dispatch-build
      Start  1: dispatch_apply
 1/22 Test  #1: dispatch_apply ...................   Passed    0.13 sec
      Start  2: dispatch_api
 2/22 Test  #2: dispatch_api .....................   Passed    0.11 sec
      Start  3: dispatch_debug
 3/22 Test  #3: dispatch_debug ...................   Passed    0.11 sec
      Start  4: dispatch_queue_finalizer
 4/22 Test  #4: dispatch_queue_finalizer .........   Passed    1.11 sec
      Start  5: dispatch_overcommit
 5/22 Test  #5: dispatch_overcommit ..............   Passed    0.11 sec
      Start  6: dispatch_context_for_key
 6/22 Test  #6: dispatch_context_for_key .........   Passed    0.11 sec
      Start  7: dispatch_after
 7/22 Test  #7: dispatch_after ...................   Passed    9.11 sec
      Start  8: dispatch_timer
 8/22 Test  #8: dispatch_timer ...................   Passed    2.11 sec
      Start  9: dispatch_timer_short
 9/22 Test  #9: dispatch_timer_short .............   Passed    1.20 sec
      Start 10: dispatch_timer_timeout
10/22 Test #10: dispatch_timer_timeout ...........   Passed    6.11 sec
      Start 11: dispatch_sema
11/22 Test #11: dispatch_sema ....................   Passed    0.12 sec
      Start 12: dispatch_timer_bit31
12/22 Test #12: dispatch_timer_bit31 .............   Passed    2.25 sec
      Start 13: dispatch_timer_bit63
13/22 Test #13: dispatch_timer_bit63 .............   Passed    1.11 sec
      Start 14: dispatch_timer_set_time
14/22 Test #14: dispatch_timer_set_time ..........   Passed    2.41 sec
      Start 15: dispatch_data
15/22 Test #15: dispatch_data ....................   Passed    0.11 sec
      Start 16: dispatch_io_muxed
16/22 Test #16: dispatch_io_muxed ................   Passed    0.61 sec
      Start 17: dispatch_io_net
17/22 Test #17: dispatch_io_net ..................   Passed    0.23 sec
      Start 18: dispatch_io_pipe
18/22 Test #18: dispatch_io_pipe .................   Passed    2.18 sec
      Start 19: dispatch_io_pipe_close
19/22 Test #19: dispatch_io_pipe_close ...........   Passed    0.11 sec
      Start 20: dispatch_select
20/22 Test #20: dispatch_select ..................   Passed    0.13 sec
      Start 21: dispatch_c99
21/22 Test #21: dispatch_c99 .....................   Passed    0.11 sec
      Start 22: dispatch_plusplus
22/22 Test #22: dispatch_plusplus ................   Passed    0.11 sec

100% tests passed, 0 tests failed out of 22

Total Test time (real) =  29.67 sec

src/queue.c Show resolved Hide resolved
src/queue.c Outdated Show resolved Hide resolved
src/queue.c Outdated Show resolved Hide resolved
src/queue.c Outdated Show resolved Hide resolved
compnerd and others added 14 commits November 19, 2021 13:15
This adds the Windows specific path for setting the thread name.  This
may only work partially as the thread name is meant to be UCS-2, but for
now, matches the behaviour in Foundation.

Similarly, on Linux, `pthread_setname_np` takes a thread identifier to
set the name.  Add a platform specific path here as well.
This adds the missing shim for atomics from osfmk to allow usage of
`_os_atomic_auto_dependency.
Handle the signal thread construction on Windows by using
`_beginthreadex` as POSIX threads are not used on Windows.  This reapirs
the build for Windows.
This preprocesses away a function definition to avoid a warning on
Windows.  This cleans all but one warning on Windows.
This removes the reference to mach.h which is pulled in when building
tests.  This allows building tests on Windows.
The dispatch object context for the cooperative root queue was not
initialized which resulted in the pthread root queue being invalid
when initializing the root queues. This corrects the default value
for the root queue.
We need to add in the `rawValue:` label when constructing the types as
identified by the compiler.
This adjusts the construction of the `dispatch_autorelease_frequency_t`
to explicitly add the `rawValue:` label and force unwrap the optional as
identified by the compiler.
* The `VOUCHER_USE_MACH_VOUCHER` branch defines `_voucher_atfork_parent`
  but the `#else` branch does not. Assume it should look like the
  implementation for `_voucher_atfork_child` nearby.

* Compiler complains about `voucher_process_can_use_arbitrary_personas`
  not having a previous prototype. This actually isn't used anywhere
  outside the `VOUCHER_USE_MACH_VOUCHER` branch, so remove it.

* Ensure `voucher_needs_adopt` is `static` inline, otherwise the linker
  gives "multiple definition" errors despite it being marked as inline.
The new revision adds a dispatch/workloop.h which is required to be
installed.  This adds that header to the install set.  Without this
header Swift is unable to build the Dispatch C module.
Without this change, `Block.swift` fails to compile complaining that
"'numericCast' requires that 'dispatch_block_flags_t' conform to
'BinaryInteger'".

A number of changes may have tickled this, possibly: improved recent
type handling in the Swift compiler, more stringent handling of C enum
types, or changes to the `DISPATCH_OPTIONS` macro. Regardless, this
change solves the problem.
Add an explicit cast from `void *` to `uintptr_t` before truncating the
context to the `dispatch_flags_t`.  Add an assertion to ensure that we
are not dropping something.
The functions are meant to be aliases that are exported.  They should be
defined to be the typeof the target function.  This corrects the
aliasing.
This introduces a new `HAVE_WAIT_ON_ADDRESS` which is the equivalent to
`HAVE_FUTEX` on Linux.  Extend the other sites to use this rather than
`defined(_WIN32)`.  This identified a couple of additional sites that
required updating which is included in this change now.
@@ -7955,7 +7973,7 @@ _dispatch_context_cleanup(void *ctxt)
#pragma mark -
#pragma mark dispatch_init

#if !DISPATCH_USE_COOPERATIVE_WORKQUEUE
#if !DISPATCH_USE_INTERNAL_WORKQUEUE && !DISPATCH_USE_COOPERATIVE_WORKQUEUE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of doing this, can we fold !INTERNAL_WORKQUEUE into the definition of DISPATCH_USE_COOPERATIVE_WORKQUEUE?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two negatives there which are disjoint, so I'm not sure how that would help here.

3405691582 added a commit to 3405691582/swift-corelibs-libdispatch that referenced this pull request Dec 10, 2021
Different platforms may define dispatch_unote_ident_t differently
and subsequently this type may have different bit widths. Therefore,
when printing du_ident, cast explicitly to deal with format mismatch
errors.

The alternative is to specify explicit format string macros, but
this is a little complex and casting is already being used in swiftlang#584.

One special case for consideration is `_dispatch_timer_unote_disarm`;
this gets the `du_ident` and converts that into a `uint32_t` timer
index. When `dispatch_unote_ident` is a `uint32_t` this is of course
fine, but needs consideration for when it isn't. Here, we just cast
down. This is somewhat reasonable since this is initialized from
`_dispatch_timer_unote_idx` which returns an `unsigned int`. (Of course,
we are not actually bounds checking that index, but that's outside the
scope of this commit).
3405691582 added a commit to 3405691582/swift-corelibs-libdispatch that referenced this pull request Jul 18, 2022
Different platforms may define dispatch_unote_ident_t differently
and subsequently this type may have different bit widths. Therefore,
when printing du_ident, cast explicitly to deal with format mismatch
errors.

The alternative is to specify explicit format string macros, but
this is a little complex and casting is already being used in swiftlang#584.

One special case for consideration is `_dispatch_timer_unote_disarm`;
this gets the `du_ident` and converts that into a `uint32_t` timer
index. When `dispatch_unote_ident` is a `uint32_t` this is of course
fine, but needs consideration for when it isn't. Here, we just cast
down. This is somewhat reasonable since this is initialized from
`_dispatch_timer_unote_idx` which returns an `unsigned int`. (Of course,
we are not actually bounds checking that index, but that's outside the
scope of this commit).
3405691582 added a commit to 3405691582/swift-corelibs-libdispatch that referenced this pull request Aug 12, 2022
Different platforms may define dispatch_unote_ident_t differently
and subsequently this type may have different bit widths. Therefore,
when printing du_ident, cast explicitly to deal with format mismatch
errors.

The alternative is to specify explicit format string macros, but
this is a little complex and casting is already being used in swiftlang#584.

One special case for consideration is `_dispatch_timer_unote_disarm`;
this gets the `du_ident` and converts that into a `uint32_t` timer
index. When `dispatch_unote_ident` is a `uint32_t` this is of course
fine, but needs consideration for when it isn't. Here, we just cast
down. This is somewhat reasonable since this is initialized from
`_dispatch_timer_unote_idx` which returns an `unsigned int`. (Of course,
we are not actually bounds checking that index, but that's outside the
scope of this commit).
3405691582 added a commit to 3405691582/swift-corelibs-libdispatch that referenced this pull request Aug 12, 2022
Different platforms may define dispatch_unote_ident_t differently
and subsequently this type may have different bit widths. Therefore,
when printing du_ident, cast explicitly to deal with format mismatch
errors.

The alternative is to specify explicit format string macros, but
this is a little complex and casting is already being used in swiftlang#584.

One special case for consideration is `_dispatch_timer_unote_disarm`;
this gets the `du_ident` and converts that into a `uint32_t` timer
index. When `dispatch_unote_ident` is a `uint32_t` this is of course
fine, but needs consideration for when it isn't. Here, we just cast
down. This is somewhat reasonable since this is initialized from
`_dispatch_timer_unote_idx` which returns an `unsigned int`. (Of course,
we are not actually bounds checking that index, but that's outside the
scope of this commit).
3405691582 added a commit to 3405691582/swift-corelibs-libdispatch that referenced this pull request Aug 13, 2022
Different platforms may define dispatch_unote_ident_t differently
and subsequently this type may have different bit widths. Therefore,
when printing du_ident, cast explicitly to deal with format mismatch
errors.

The alternative is to specify explicit format string macros, but
this is a little complex and casting is already being used in swiftlang#584.

One special case for consideration is `_dispatch_timer_unote_disarm`;
this gets the `du_ident` and converts that into a `uint32_t` timer
index. When `dispatch_unote_ident` is a `uint32_t` this is of course
fine, but needs consideration for when it isn't. Here, we just cast
down. This is somewhat reasonable since this is initialized from
`_dispatch_timer_unote_idx` which returns an `unsigned int`. (Of course,
we are not actually bounds checking that index, but that's outside the
scope of this commit).
3405691582 added a commit to 3405691582/swift-corelibs-libdispatch that referenced this pull request Aug 22, 2022
Different platforms may define dispatch_unote_ident_t differently
and subsequently this type may have different bit widths. Therefore,
when printing du_ident, cast explicitly to deal with format mismatch
errors.

The alternative is to specify explicit format string macros, but
this is a little complex and casting is already being used in swiftlang#584.

One special case for consideration is `_dispatch_timer_unote_disarm`;
this gets the `du_ident` and converts that into a `uint32_t` timer
index. When `dispatch_unote_ident` is a `uint32_t` this is of course
fine, but needs consideration for when it isn't. Here, we just cast
down. This is somewhat reasonable since this is initialized from
`_dispatch_timer_unote_idx` which returns an `unsigned int`. (Of course,
we are not actually bounds checking that index, but that's outside the
scope of this commit).
@AreaZR
Copy link
Contributor

AreaZR commented Oct 2, 2022

Any updates on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants