functional-types-with-big-arity-on-jvm.md

Functional types with big arity on JVM

Discussion of this proposal is held in this issue.

This proposal describes a way to support functional types and lambdas that take 23 or more parameters in Kotlin/JVM. It's largely based on a previous (outdated) document with description of a pre-1.0 reform of functional types: spec-docs/function-types.md.

Motivation

In Kotlin, functional types are represented as generic classes taking different number of parameters: Function0<R>, Function1<P0, R>, Function2<P0, P1, R>, ... This approach has a problem in that this list is finite, and it currently ends with Function22. Ideally, we'd like to support functions of arbitrary arity in Kotlin. In practice, a JVM method cannot have more that 255 parameters, so this is the limit for us as well.

Related issue: KT-13764 Support lambdas and function references for arities bigger than 22

Proposal

The proposal is to use one class internally for all functional types with arity > 22:

package kotlin.jvm.functions

@SinceKotlin("1.3")
interface FunctionN<out R> : Function<R> {
    operator fun invoke(vararg args: Any?): R
    
    override val arity: Int
}

It's declared in package kotlin.jvm.functions where all other JVM function classes are, which is the package that is not supposed to be used in Kotlin sources but could be used in Java (see Java interop below).

Any Kotlin's functional type with big arity is erased to this type on the JVM, and the compiler wraps all arguments into array at each call site of invoke:

// In generated bytecode, this function is "call(Lkotlin/jvm/functions/FunctionN;)V"
fun call(block: (Any, Any, ... /* 42 more */, Any) -> Any) {
    // Here, we load all arguments on the stack, then wrap them into array and then
    // invoke Lkotlin/jvm/functions/FunctionN.invoke([Ljava/lang/Object;)Ljava/lang/Object;
    block(Any(), Any(), ..., Any())
}

A lambda with big arity is compiled to a subclass of Lambda and FunctionN. At the beginning of its invoke, the generated bytecode checks if the length of the passed vararg array is equal to the function arity:

val lambda = { p1: Any, p2: Any, ..., p42: Any ->
    // In generated bytecode of this lambda class, there are two methods:
    // 1) invoke(Ljava/lang/Object;Ljava/lang/Object;...Ljava/lang/Object;)V,
    //    which contains this lambda body, generated as a normal lambda.
    // 2) invoke([L/java/lang/Object;)Ljava/lang/Object;,
    //    which is a bridge implementing FunctionN.invoke, where we do
    //    ALOAD 0 + ARRAYLENGTH + throw IAE if it's not 42,
    //    and then unpack arguments from the array and call the first invoke
    ...
}

Similar algorithm is used when generating anonymous function reference classes for functions that take 23 or more parameters.

Type checks and casts

Since Kotlin 1.0, is/as on functional types on JVM work via an internal runtime function TypeIntrinsics.isFunctionOfArity, which checks that the object is an instance of Function and its arity value is equal to the arity on the right-hand side of is/as. This code needs no modification with regard to this proposal, it'll work automatically with lambdas and references of big arities.

Java interop

To pass a lambda with big arity from Java to Kotlin, one would have to implement both of the FunctionN's methods:

class JavaClass {
    void test() {
        CallKt.call(new FunctionN<String>() {
            @Override
            public int getArity() { return 42; }
            
            @Override
            public String invoke(Object... args) {
                // Checking the vararg size is a part of the contract of FunctionN.invoke
                if (args.length != 42) throw new IllegalArgumentException();
                
                ...
            }
        });
    }
}

Note that the Java code is required to check the array length of args at the beginning of invoke manually. This requirement will be documented in the contract of FunctionN.invoke.

If the FunctionN type is used in a signature, it will be seen as a "foreign" type and not as a true Kotlin functional type.

Future improvements

For Java interop, we could introduce an annotation Arity with one integer parameter. This annotation would be used in Java sources on the FunctionN type usage or on the corresponding declaration, to make the Kotlin compiler treat that type usage as a functional type with that arity. For example, @Arity(42) FunctionN<String> in a Java signature would be treated as (Any?, Any?, ..., Any?) -> String! in Kotlin. This is a topic of a future proposal.