Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch CI to 1.11. #2525

Merged
merged 2 commits into from
Oct 22, 2024
Merged

Switch CI to 1.11. #2525

merged 2 commits into from
Oct 22, 2024

Conversation

maleadt
Copy link
Member

@maleadt maleadt commented Oct 17, 2024

No description provided.

@maleadt
Copy link
Member Author

maleadt commented Oct 17, 2024

Looks like 1.11 is significantly slower; I bumped the timeouts.

@maleadt
Copy link
Member Author

maleadt commented Oct 18, 2024

The hangs are legitimate. Subpackage tests only take up to 10 minutes on 1.10 (so the timeouts should be adjusted accordingly).

@maleadt
Copy link
Member Author

maleadt commented Oct 18, 2024

Depends on new TensorOperations.jl release: Jutho/TensorOperations.jl#193 (comment)

@maleadt
Copy link
Member Author

maleadt commented Oct 21, 2024

Looks like on 1.11 the cuTENSOR tests really do hang -- lots of compile time being spent on signatures like:

Base.Broadcast.materialize
Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{4}, Nothing, typeof(Base.:(+)), Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{4}, Nothing, typeof(Base.:(*)), Tuple{Float16, Array{Float32, 4}}}, Array{Float16, 4}}}(style=Base.Broadcast.DefaultArrayStyle{4}(), f=typeof(Base.:(+))(), args=(Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{4}, Nothing, typeof(Base.:(*)), Tuple{Float16, Array{Float32, 4}}}(style=Base.Broadcast.DefaultArrayStyle{4}(), f=typeof(Base.:(*))(), args=(Float16(0x33f4), Array{Float32, 4}(dims=(3, 10, 12, 14), mem=Memory{Float32}(5040, 0x6425044cc8c0)[0.386422f, ..., 0.933269f, 0.919605f, 0.140272f, 0.979564f, 0.964575f, 0.393861f, 0.0486779f, 0.441696f, 0.375366f, 0.497105f])), axes=nothing), Array{Float16, 4}(dims=(3, 10, 12, 14), mem=Memory{Float16}(5040, 0x6424fe479940)[Float16(0x3bcd), ..., Float16(0x3967)])), axes=nothing)

Another one I observed was:

materialize(Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2}, Nothing, typeof(Base.:(+)), Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2}, Nothing, typeof(Base.:(*)), Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2}, Nothing, typeof(Base.:(*)), Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2}, Nothing, typeof(Base.:(*)), Tuple{Float16, Array{Float32, 2}}}, Float16}}, Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2}, Nothing, typeof(Base.conj), Tuple{Array{Float32, 2}}}}}, Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2}, Nothing, typeof(Base.:(*)), Tuple{Float16, Array{Float16, 2}}}}}) from materialize(Base.Broadcast.Broadcasted{Style, Axes, F, Args} where Args<:Tuple where F where Axes where Style<:Union{Nothing, Base.Broadcast.BroadcastStyle})

This reproduces on the 1.11.2 backports branch too.

@maleadt
Copy link
Member Author

maleadt commented Oct 22, 2024

Not a hang, 1.11 is just a lot slower:

1.11.1 --check-bounds=yes

elementwise binary |  352    352  3m58.3s
elementwise trinary |  252    252  6m34.7s
permutations  |   64     64  19.7s
reductions    |  200    200  1m40.8s

1.10 --check-bounds=yes

elementwise binary |  352    352  1m08.6s
elementwise trinary |  252    252  2m02.9s
permutations  |   64     64  6.8s
reductions    |  200    200  20.5s

@maleadt maleadt enabled auto-merge (squash) October 22, 2024 14:28
@maleadt maleadt disabled auto-merge October 22, 2024 17:04
@maleadt maleadt merged commit a95ccfb into master Oct 22, 2024
1 check failed
@maleadt maleadt deleted the tb/1.11 branch October 22, 2024 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant