ReScript support

[cannorin]

Related: #166

Adds ReScript as a target.

TODOs:

• Research how ReScript bindings works
  • shared data types
  • interop cheatsheet
  • string enums & literal types do not need to be escaped (link)
  • support for raw js (link)
  • no support for implicit type conversion (similar to to_js and of_js in gen_js_api)
  • import is platform-agnostic, no need to convert import/require on our side (link)
  • not very sure how to add documentation comments
• Check if we need to wait or make issues/PRs
  • ReScript is lacking some features: ReScript support #32 (comment)
  • bugs in rescript-vscode may impact the user experience with the generated bindings:
    • Completion for a type which is an alias to some record type? rescript-lang/rescript-vscode#311
    • Possible bugs on resolving type alias to record type rescript-lang/rescript-vscode#349
• Implement it
• Set up tests
• Set up automated package publishing (not needed since the stdlib is minimal)
• Write docs

Notes:

Subtyping

The "tag" subtyping should look like the following:

@unboxed type intf<-'tags, 'a> = { value: 'a }

module Node = {
  type content = { kind: string }
  type t = intf<[#Node], content>

  let from = (x: intf<[> #Node], _>) : t => Obj.magic(x)
}

module FooNode = {
  type content = {
    kind: [#"foo"],
    mutable foo: string
  }
  type t = intf<[#FooNode|#Node], content>

  let from = (x: intf<[> #FooNode], _>) : t => Obj.magic(x)
}

module BarNode = {
  type content = {
    kind: [#"bar"],
    mutable bar: float
  }
  type t = intf<[#BarNode|#Node], content>

  let from = (x: intf<[> #BarNode], _>) : t => Obj.magic(x)
}

In this way, it will allow one-step casting as well as an easy access to its fields:

let fooNode : FooNode.t = { value: { kind: #"foo", foo: "hello" } }
fooNode.value.foo = "goodbye"
let node = Node.from(fooNode)
Js.log(node.value.kind)

• the content type should only have value fields (no methods = no overloading)
• methods should have the signature of (this: t, ...) => ... rather than (this: content, ...) => ...

Discriminated Unions

There is an open feature request about this: rescript-lang/rescript#5207

A discriminated union should look like this:

module AnonymousUnion1 = {
  type t = { kind: string }
  type cases = [ #"foo"(FooNode.t) | #"bar"(BarNode.t) ]

  let foo : FooNode.t => t = Obj.magic
  let bar : BarNode.t => t = Obj.magic
  let classify = (t: t) : cases => %raw(`{ NAME: t.kind, VAL: t }`)
}

This makes it easy to handle union types with almost zero overhead:

let foo = AnonymousUnion1.foo({ value: { kind: #"foo", foo: "hello" } })
Js.log(foo)
let foo2 = AnonymousUnion1.classify(foo)
switch foo2 {
  | #"foo"(x) => Js.log(x.value.foo)
  | #"bar"(x) => Js.log(x.value.bar)
}

Callable & Newable Objects

A callable/newable object should look like this:

module FooConstructor {
  type t
  let invoke = (it: t, arg) : Foo.t => %raw(`it(arg)`)
  let create = (it: t, arg1, arg2) : Foo.t => %raw(`new it(arg1, arg2)`)
}

Maybe we should generate .resi files too

ReScript bindings are written in .res files, in which recursive modules are hard to write. But we can just generate a flat .res file and then generate a corresponding .resi file to hide internal types.

// Example.res

module Types = {
    module rec A : { type t = ... } = A
    and B : { type t = ... } = B
}

open Types

module A = {
    include Types.A
    ...
}

module B = {
    include Types.B
    ...
}

// Example.resi

module rec A : {
    type t = ...
    ...
}

and B : {
    type t = ...
    ...
}

[cannorin]

I'll hold this PR a bit (about 1 month) because I will focus on separate the target-agnostic part of the current implementation as a library:

• AST
• Parser
• Typer
• JS utilities

This refactoring will make it easier to work on ReScript support.

[cannorin]

ReScript difficulties

Hard/unable to workaround

• Can't bind to float enums: enum Version = { 1_1 = 1.1, 1_2 = 1.2 } and float literal types: type version = 1.1 | 1.2

  • @deriving(jsConverter) could be modified to allow float values?
  • → ReScript support #32 (comment)

• Can't bind to constructors with variadic argument

  • @variadic @new is rejected
  • → @new @variadic is not supported? rescript-lang/rescript#5363
  • → Support @new @variadic rescript-lang/rescript#5364

Can be workaround, but cumbersome to use

• Can't bind to functions with variadic callback function: bind<T, AX, R>(this: (this: T, ...args: AX[]) => R, thisArg: T, ...args: AX[]): (...args: AX[]) => R;

  • can be workaround:

    ts2ocaml/dist_rescript/src/Ts__min.res

    Lines 176 to 216 in 8e2fe01

    	module Variadic = {
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	type t0<'variadic, 't>
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	type t1<'arg1, 'variadic, 't>
    	@ocaml.doc(`\`'args\` must be a tuple type. \`'variadic\` is expected to be array or some other iterable type.`)
    	type tn<'args, 'variadic, 't>
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	let make0 : ('variadic => 't) => t0<'variadic, 't> = f => %raw(`(function(...args) { return f(args); })`)
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	let make1 : (('arg1, 'variadic) => 't) => t1<'arg1, 'variadic, 't> = f => %raw(`(function(arg1, ...args) { return f(arg1, args); })`)
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	let make2 : (('arg1, 'arg2, 'variadic) => 't) => tn<('arg1, 'arg2), 'variadic, 't> = f => %raw(`(function(arg1, arg2, ...args) { return f(arg1, arg2, args); })`)
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	let make3 : (('arg1, 'arg2, 'arg3, 'variadic) => 't) => tn<('arg1, 'arg2, 'arg3), 'variadic, 't> = f => %raw(`(function(arg1, arg2, arg3, ...args) { return f(arg1, arg2, arg3, args); })`)
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	let make4 : (('arg1, 'arg2, 'arg3, 'arg4, 'variadic) => 't) => tn<('arg1, 'arg2, 'arg3, 'arg4), 'variadic, 't> = f => %raw(`(function(arg1, arg2, arg3, arg4, ...args) { return f(arg1, arg2, arg3, arg4, args); })`)
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	let make5 : (('arg1, 'arg2, 'arg3, 'arg4, 'arg5, 'variadic) => 't) => tn<('arg1, 'arg2, 'arg3, 'arg4, 'arg5), 'variadic, 't> = f => %raw(`(function(arg1, arg2, arg3, arg4, arg5, ...args) { return f(arg1, arg2, arg3, arg4, arg5, args); })`)
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	let make6 : (('arg1, 'arg2, 'arg3, 'arg4, 'arg5, 'arg6, 'variadic) => 't) => tn<('arg1, 'arg2, 'arg3, 'arg4, 'arg5, 'arg6), 'variadic, 't> = f => %raw(`(function(arg1, arg2, arg3, arg4, arg5, arg6, ...args) { return f(arg1, arg2, arg3, arg4, arg5, arg6, args); })`)
    	@ocaml.doc(`\`'variadic\` is expected to be array or some other iterable type.`)
    	let make7 : (('arg1, 'arg2, 'arg3, 'arg4, 'arg5, 'arg6, 'arg7, 'variadic) => 't) => tn<('arg1, 'arg2, 'arg3, 'arg4, 'arg5, 'arg6, 'arg7), 'variadic, 't> = f => %raw(`(function(arg1, arg2, arg3, arg4, arg5, arg6, arg7, ...args) { return f(arg1, arg2, arg3, arg4, arg5, arg6, arg7, args); })`)
    	@ocaml.doc(`\`'args\` must be a tuple type. \`'variadic\` is expected to be array or some other iterable type.`)
    	let makeN : (('args, 'variadic) => 't, int) => tn<'args, 'variadic, 't> = (f, n) => %raw(`(function(...args) { return f(args.slice(0, n), args.slice(n)); })`)
    	let apply0 = (f0: t0<'variadic, 't>, variadic: 'variadic) : 't => %raw(`f0(...variadic)`)
    	let apply1 = (f1: t1<'arg1, 'variadic, 't>, arg1: 'arg1, variadic: 'variadic) : 't => %raw(`f1(arg1, ...variadic)`)
    	let applyN = (fn: tn<'args, 'variadic, 't>, args: 'args, variadic: 'variadic) : 't => %raw(`fn(...args, ...variadic)`)
    	}

• Can't bind to "newable" functions, which must be called with new: new (arg: string): Foo

  • can be workaround:

    ts2ocaml/dist_rescript/src/Ts__min.res

    Lines 164 to 174 in 8e2fe01

    	module Newable = {
    	type t0<'t>
    	type t1<'arg1, 't>
    	@ocaml.doc(`\`'args\` must be a tuple type.`)
    	type tn<'args, 't>
    	let apply0 = (f0: t0<'t>) : 't => %raw(`new f0()`)
    	let apply1 = (f1: t1<'arg1, 't>, arg1: 'arg1) : 't => %raw(`new f1(arg1)`)
    	let applyN = (fn: tn<'args, 't>, args: 'args) : 't => %raw(`new fn(...args)`)
    	}

• Can't bind to global mutable values / static setters (e.g. location.href)

  • only reading is supported: https://rescript-lang.org/docs/manual/latest/bind-to-global-js-values
  • can be workaround with @set, treating it as if it were a field of globalThis/imported object rather than a global value: location->setHref("...")

• Can't bind to global values with exotic name (e.g. $&), while members with exotic name are allowed

  • @send external _and: regexp => string = "$&" is allowed
  • @scope("RegExp") @val external _and: string = "$&" is rejected
  • real world example: https://github.com/microsoft/TypeScript/blob/ddad61e1b00275efa0719a640f77934562e00ec1/src/lib/es5.d.ts#L990
    • it's deprecated but it's irrevant here:
      • if you have interface FooConstructor { someProperty: int } and declare var Foo: FooConstructor, you would want to bind directly to Foo.someProperty with @scope("Foo") @val.
      • but if the said property has an exotic name, then you can't use @scope("Foo") @val, forcing you to create an instance of FooConstructor to access its property with @get.
      • and exotic names are abundant in JS world (e.g. https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/yargs-parser/index.d.ts#L12)
  • can be workaround with @get/@set/@send, treating it as if it were a field or a method of globalThis/imported object rather than a global value/function.

• Can't bind to enums with negative value: enum Foo = { A = -1, B = 0, ... }

  • real world example: https://github.com/microsoft/vscode/blob/33aeada8904e540d1fa39ffe676704a46e1f0e0c/src/vscode-dts/vscode.d.ts#L5896-L5945
  • can be workaround with @deriving(jsConverter), but tToJs and tFromJs functions are required: https://rescript-lang.org/try?code=AIEwpgTglgblB2BzAFAbwFYGcDCB7eMkALpAFwAE8YA7gCoCeADmAL4CUAUEU2OQGYQArlCLkAvB3LkAPuWABDTMgAMbKQEFGjADZhJMuYuQBaAIxqA8hHlI9U2QqUAmNeQDSUalH0OjxgMxqAOryJBAAtmDa+BwcuqK44uRWNohgxgB8AsJEtLgAUphAA
  • → Numeric poly var support rescript-lang/syntax#340 (comment)
  • → ReScript support #32 (comment)

Easy to workaround

• Tagged union is not supported.
  • Can be workaround with %raw (see notes above).
  • There is an open feature request: [Feature request] zero-cost binding to tagged JS objects rescript-lang/rescript#5207
• Can't bind to callable objects, which is an object but also a function.
  • Can be workaround with %raw (see notes above).
• BigInt is not supported.
  • Can provide on our own.
  • BigInt.t is now available as a placeholder
  • There is an open PR: Add BigInt bindings rescript-lang/rescript#5350

[cannorin]

Many things need to be polished (e.g. use @unwrap instead of Union.tn on args) and other packages are still failing to compile, but we have a first compilable ReScript binding to the typescript package, generated with ts2ocaml.

https://gist.github.com/cannorin/4cfd43318e4efe811d7d595a8f2c710b

[zth]

So cool!! 😄 👍

[cannorin]

On hold until rescript-lang/rescript#5364 and rescript-lang/rescript#5368 are merged/addressed.

[smorimoto]

I believe that new and variadic have finally landed with the latest ReScript version!
https://www.npmjs.com/package/rescript/v/10.0.0

[cannorin]

... it's compiling with rescript v10!

[d4h0]

@cannorin: Thanks, for your work on this! I'm pretty excited about the possibility of not having to write every single binding by hand anymore! :)

I'm wondering: Is this already in a more or less usable state?

Currently, I'm trying to use Playwright by creating bindings manually, but there are like a million types (even if I only create bindings for what I use), so something like ts2ocaml would be fabulous... :)

Do you think, ts2ocaml could handle something as complex/huge as Playwright?

[cannorin]

@d4h0 It should be possible to parse the type definitions of Playwright and generate a binding that compiles, but I don't think the generated binding would be user-friendly; it's still lacking some important optimizations that affect QoL (e.g. emit [#1 | #2 | #3] instead of Union.t3<[#1],[#2],[#3]>).

I'll fix the broken CI tomorrow or next just in case you want to try it out.

[zth]

@cannorin beta.4 is out with variant spreads and variant coercion, in case that helps.
https://github.com/rescript-lang/rescript-compiler/releases/tag/v11.0.0-beta.4

[cannorin]

@zth Thanks! Variant-to-variant coercion will definitely help.

[mxj4]

Thanks for creating this, the generated binding looks great! What is the strategy for TS definitions with lots of 3rd party TS definition dependencies? To build ReScript typing for a React component from TS definition, I had to build my own script on top of npm package managers to fetch all dependencies and manipulate the output file names.

Then I found, since the TS definition depends on @types/react, now I have to generate ReScript binding from @types/react, but ReScript community already has its own binding for React, and you already have a react application using the hand-crafted react binding @rescript/react, now the generated binding is incompatible with the hand-crafted binding.

[cannorin]

@mxj4 Thank you for trying it out! In general, users may need to modify the generated bindings to make it work with the existing hand-crafted bindings. However, I do think React component packages deserve some special treatment.

[cannorin]

@zth In ts2ocaml, classes and interfaces are represented as abstract types to allow subtyping, field shadowing, and method overloading at the same time (which is impossible if they are defined as record types).

Would it be possible to "inline" the arguments of zero-cost-binding variant types to accomplish something like below?:

type intf<-'tags>

// interface ShapeLike { kind: string }
module ShapeLike = {
  type t = intf<[#Shape]>
  type tags = [#Shape]
  type this<'tags> = intf<'tags> constraint 'tags = [> #Shape]
  
  @get external kind: this<'tags> => string = "kind"
}

// interface Circle extends ShapeLike { kind: "circle", radius: number }
module Circle = {
  type t = intf<[#Circle | ShapeLike.tags]>
  type tags = [#Circle | ShapeLike.tags]
  type this<'tags> = intf<'tags> constraint 'tags = [> #Circle]

  @get external kind: this<'tags> => [#circle] = "kind"
  @get external radius: this<'tags> => float = "radius"
  @obj external make: (~kind: [#circle], ~radius: float) => t = ""
}

// interface Square extends ShapeLike { kind: "square", sideLength: number }
module Square = {
  type t = intf<[#Square | ShapeLike.tags]>
  type tags = [#Square | ShapeLike.tags]
  type this<'tags> = intf<'tags> constraint 'tags = [> #Square]

  @get external kind: this<'tags> => [#square] = "kind"
  @get external sideLength: this<'tags> => float = "sideLength"
  @obj external make: (~kind: [#square], ~sideLength: float) => t = ""
}

let circle: Circle.t = Circle.make(~kind=#circle, ~radius=4.2)
let circleKind: string = circle->ShapeLike.kind // this works because of subtyping via `intf<-'tags>`

// type Shape = Circle | Square
module Shape = {
  @tag("kind")
  type t =
    | @as("circle") @unsafe_inline Circle(Circle.t)
    | @as("square") @unsafe_inline Square(Square.t)
}

let circleShape = Shape.Circle(circle)
// I want this to be compiled to:
//
//   var circleShape = circle

let getAreaOfShape = (shape: Shape.t) =>
  switch (shape) {
    | Shape.Circle(c) => c->Circle.radius *. c->Circle.radius *. Js.Math._PI
    | Shape.Square(s) => s->Square.sideLength *. s->Square.sideLength
  }
// I want this to be compiled to:
//
// function getAreaOfShape(shape) {
//   if (shape.kind === "circle") {
//     return shape.radius * shape.radius * Math.PI;
//   }
//   return shape.sideLength * shape.sideLength;
// }

[listepo]

Hi @cannorin in your opinion, how long do we have to wait before we can use it?

[cannorin]

@listepo You could use it today if you can build it yourself. As for the official release, I guess it will be around the v11 release of ReScript, as ts2ocaml depends on the new features.

[baku-yaki]

Rescript 11 is released. Is this package ready now?

[cannorin]

@baku-yaki Thank you for letting me know, I'll make some changes for releasing (since it is currently using a quite old RC version of v11). I think it will be live within a month!

[cannorin]

TODO List:

• Remove the last unit argument from functions with optional arguments (with the arrival of uncurried mode)
• Implement doc comment generators (we put it off since ReScript didn't have it when this PR was created).
  • Doc comment syntax (/** */) is not working when attached to recursive modules rescript-lang/rescript#6598 affects the generated bindings. We will omit the doc comment for recursive modules until it gets fixed.
  • It will be fixed in ReScript 11.1.0.
• Revisit tagged union. We can't utilize ReScript v11's tagged union, so we must emit a custom classifier function for each tagged union types (described in ReScript support #32 (comment))
• Setup automated publishing of ts2ocaml-rescript-stdlib? (might not be needed since the stdlib is not that big and users can just do ts2ocaml res --create-stdlib?)
• Write a tutorial!

Future Works:

• Add a direct support for React components (requested in ReScript support #32 (comment))
• Record mode (unsafe): ts2ocaml currently uses abstract types (intf<'tags>) to represent TS classes and interfaces, but an idiomatic ReScript binding would use record types. In general, record types are insufficient for binding to TS classes (since records cannot have overloaded methods). But library authors would want to have them generated as record types anyway to avoid boilerplate works.
• Enable emitting the doc comment for recursive modules once ReScript v11.1 is released.

[cannorin]

I have just released the ReScript support as 2.0.0-beta.0!

Although I have tested that the generated binding compiles for a variety of NPM packages (namely, react, yargs, and typescript's SDK), but there may still be some edge cases that break the generated code. There may also be some cases that the generated binding compiles but does not work (e.g. broken import/require). Some may be hard to fix, but I want to fix the easier and/or critical ones before the release of v2.0.0.

Please try it out by npm install -g @ocsigen/ts2ocaml@beta!