py313-mypy: fix compilation on OS X <= 10.9

[erikbs]

Description

Build failed on 10.9 because the code uses C11 features, but AppleClang 600 (provided by Xcode 6.2, the latest available version on 10.9) defaults to an earlier dialect. It issues a warning, which turns into an error because of compiler flags. Clang 3.6, upon which AppleClang 602 (from Xcode 6.3, which requires OS X 10.10) is based, switched to C11 as its default dialect. Setting compiler.c_standard 2011 did not seem to set the correct flags to fix the problem, so the easiest solution was to blacklist Clang < 602. This causes MacPorts to pick a newer compiler, MacPorts Clang 17 in my case, and the build succeeds.

Type(s)

• bugfix
• enhancement
• security fix

Tested on

macOS 10.9.5 13F1911 x86_64
Xcode 6.2 6C131e

Verification

Have you

• followed our Commit Message Guidelines?
• squashed and minimized your commits?
• checked that there aren't other open pull requests for the same change?
• checked your Portfile with port lint --nitpick?
• tried existing tests with sudo port test?
• tried a full install with sudo port -vst install?
• tested basic functionality of all binary files?

Tested by running the binaries without options.

[macportsbot]

Notifying maintainers:
@tobypeterson for port py-mypy.

[barracuda156]

@erikbs If the code uses C11 now, it should declare compiler.c_standard 2011. which automatically gets compiler choice right, and then blacklist can be dropped.

[erikbs]

@erikbs If the code uses C11 now, it should declare compiler.c_standard 2011. which automatically gets compiler choice right, and then blacklist can be dropped.

I tried, but that did not work. Do I need to add a special port group?

[barracuda156]

@erikbs It could be a special case for that clang (no idea on that), but if C11 is required, it is still better to declare that, even if blacklist is needed for clangs regardless.

[tobypeterson]

I believe that compiler.c_standard is used to identify acceptable Xcode versions, not clang versions. However, I still think it'd be good to add, assuming that this builds with BOTH set (c_standard and blacklist).

[barracuda156]

Once #26830 is merged, we will not need per-port requirement for C11 with Python 3.13.

[tobypeterson]

@barracuda156 Is this working now? (Without changes to the mypy port.)

[barracuda156]

@barracuda156 Is this working now? (Without changes to the mypy port.)

@tobypeterson py-mypy built fine for me because it already blacklists Xcode gcc (and that is still needed, because all subports need a newer compiler). I cannot check with the old Clang now (it exists neither on powerpc nor on arm64), but compiler choice should be whatever MacPorts sets for C11 for a given system.
I.e. from my end no changes to your port are needed, just keep existing blacklist for Xcode gcc (PG only blacklists it for py313).

[erikbs]

I still cannot build py313-mypy from master on 10.9 without these changes. MacPorts picks /usr/bin/clang, which is not C11-compatible. compiler.c_standard 2011 usually prevents that version from being used, but that does not happen with this port. I have no idea why.

[barracuda156]

I still cannot build py313-mypy from master on 10.9 without these changes. MacPorts picks /usr/bin/clang, which is not C11-compatible. compiler.c_standard 2011 usually prevents that version from being used, but that does not happen with this port. I have no idea why.

But does it work with other Python ports for you? In other cases do you get the correct compiler picked?

[erikbs]

I picked a random Python 3.13 port and installed it from source: install -s py313-zstd. The log shows that it too used /usr/bin/clang, but it built successfully.

[barracuda156]

@erikbs Ok, it does not work now indeed. There is no problem on your end, compiler.c_standard 2011 is just not being set. It is easy to verify by picking a random Python port and adding in the same manually – and compare port deps.
No idea what has broken it.

[erikbs]

My first comment today was not correct. A few weeks have passed, so I forgot the details. The correct description is that the chosen compiler is C11 compatible, but it does not enable C11 by default.

As an experiment, I added compiler.c_standard 2011 to a random port (aha) and built it from source. MacPorts picked /usr/bin/clang (i.e. Xcode), as expected (again, it is C11 compatible). I then tried with c_standard 2017 and it picked MacPorts Clang instead, again as expected (since AppleClang 600 does not know what C17 is).

Then I tried c_standard 2017 in mypy, but then MacPorts picked Xcode, which was unexpected. So it seems that for Python packages (I tried with both mypy and zstd), compiler.c_standard is ignored. I do see indications of blacklisting in the zstd log, but the blacklisted compiler ends up being used anyway:

:debug:clean compiler clang 600.0.57 blacklisted because it matches {clang < 1001}
:debug:clean compiler clang 600.0.57 blacklisted because it matches {clang < 1204}
…
:debug:clean compiler clang 600.0.57 not blacklisted because it doesn't match {clang < 500}
…
:debug:clean compiler clang 600.0.57 blacklisted because it matches {clang < 1001}
:debug:clean compiler clang 600.0.57 blacklisted because it matches {clang < 1204}
…
:debug:clean compiler clang 600.0.57 blacklisted because it matches {clang < 602}
…
:debug:clean compiler clang 600.0.57 blacklisted because it matches {clang < 701}
…
:debug:clean compiler clang 600.0.57 blacklisted because it matches {clang < 801}

The compiler blacklist PortGroup is sourced multiple times. Note that once the incompatible compiler passes because the blacklist pattern is only clang < 500. I have no idea where these blacklist entries come from – in portconfigure.tcl the cutoff for C17 is Clang 1000.11.45.2. However, this does not really matter for this particular port, since we only need C11.

The goal must be either to make MacPorts use a Clang version that defaults to C>=11 or somehow make MacPorts pass -std=c11 to Clang. What surprises me is that c_standard does not make MacPorts pass -std=c11 to the compiler. As long as that is the case, the only solutions I see are to blacklist Clang < 602 (even though 500.2.75 os the real minimum) or pass -std=c11 manually from this Portfile (but as far as I remember the flags I put in the Portfile were ignored).

[barracuda156]

Someone has to look into why it does not work as expected, but anyway, it should be fixed in the portgroup, not in a given port (as long as requirements match). Commit history does not have anything immediately obvious. Maybe the change of python313 to the default one affected how options are processed?

[erikbs]

Someone has to look into why it does not work as expected, but anyway, it should be fixed in the portgroup, not in a given port (as long as requirements match).

I agree. And I think that there are two things that need to be fixed:

• c_standard currently seems to be ignored for Python ports
• Even if c_standard 2011 is honoured, it only blacklists C11-incompatible compilers. It does not actually ensure that C11 is enabled for those compilers that support C11, but default to an earlier version. The solution would be to either blacklist those versions as well or pass -std=c11 (without overriding individual ports that specify a different standard or dialect)

[barracuda156]

c_standard currently seems to be ignored for Python ports

It is not ignored if added into portfile (or already present there). What seems to have broken is adding it via PG automatically for py313 subports.

[erikbs]

c_standard currently seems to be ignored for Python ports

It is not ignored if added into portfile (or already present there).

It was ignored when I added compiler.c_standard 2017 directly to the py-zstd Portfile. That should have blacklisted my Xcode Clang (600) from being used, but it was used anyway. The log lines above are taken from that experiment. When I added it to a pure C port (aha), it was honoured, so a newer MacPorts Clang was picked instead.

[erikbs]

An interesting observation: I can make MacPorts honour compiler.c_standard in an unexpected way.

Setup:

• Using py-black, as it is small and builds very fast
• In its Portfile, I added compiler.c_standard 2017 right above if {${subport} ne ${name}} {

Now I install using install -d py312-black. Without any modifications, MacPorts will pick Xcode Clang (/usr/bin/clang), which on 10.9 is AppleClang 600 and way too old for C17 support.

I only have experience with Tcl scripting through MacPorts and do not really know the language, so I do not know the difference between set variable_name value and variable_name value. Anyway, I made the following change to /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/python-1.0.tcl as an experiment:

-                compiler.c_standard 2011
+                set compiler.c_standard 2011

This makes MacPorts honour compiler.c_standard 2017 (without set) in the Portfile (picking MacPorts Clang 16), but I have no idea how that works. If I replace 2017 with 2023 or 2011, it picks a different compiler.

I also discovered that if I set compiler.cxx_standard (without set), whether I do in the Portfile or PortGroup, it is always honoured. I have grep-ed and looked through the MacPorts Base scripts for clues as to why it honours cxx_standard and ignores c_standard, but so far I have found none.

@barracuda156 Any ideas?

[barracuda156]

@jmroot @ryandesign Could someone comment on this, please? You know MacPorts base much better.

[erikbs]

Attaching the complete build log, in case that helps: main.log

[jmroot]

#26830 (comment)

[erikbs]

My understanding after reading that thread is that it is intentional that compiler.c_standard does not cause -std= to be passed to the compiler. I also just stumbled upon python.add_cflags, so I tried reverting the compiler blacklisting and instead adding the following three lines:

compiler.c_standard 2011
configure.cflags    -std=c11
python.add_cflags   yes

With this I can build py-mypy successfully. If this is an acceptable solution I can push these changes.