Support conversion to Unicode line-column positions

[dennwc]

Support conversions from byte offset to Unicode/UTF-16 byte line-column pairs. Also, refactor the code for consistency and add more test cases.

This allows using positional info generated by Babelfish to calculate Unicode positions used in code editors.

Required by bblfsh/libuast#102.

---

This change is 

[creachadair]

Reviewed 2 of 2 files at r1.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @bzz and @dennwc)

---

uast/transformer/positioner/positions.go, line 261 at r1 (raw file):

}

// LineCol returns a one-based line and col given a zero-based byte offset.

Suggestion: "…zero-based byte offset within the line."

---

uast/transformer/positioner/positions.go, line 291 at r1 (raw file):

		return -1, err
	}
	const minCol = 1

Is this right? I thought column offsets were 0-based.

---

uast/transformer/positioner/positions.go, line 410 at r1 (raw file):

}

// toUnicodeLineCol returns a one-based Unicode character line-col or a UTF-16 code unit line-col given a zero-based byte offset.

Here again—are the columns 0-indexed too?

[dennwc]

Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @bzz, @creachadair, and @dennwc)

---

uast/transformer/positioner/positions.go, line 291 at r1 (raw file):

Previously, creachadair (M. J. Fromberger) wrote…

Is this right? I thought column offsets were 0-based.

They are 1-based. It looks like the function comment should be clearer.

I hoped the "and" in "one-based line and column" would imply this. "one-based line and one-based column" may be a bit too verbose.

---

uast/transformer/positioner/positions.go, line 410 at r1 (raw file):

Previously, creachadair (M. J. Fromberger) wrote…

Here again—are the columns 0-indexed too?

Would clarify the docs here as well.

[creachadair]

Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @bzz, @creachadair, and @dennwc)

---

uast/transformer/positioner/positions.go, line 291 at r1 (raw file):

Previously, dennwc (Denys Smirnov) wrote…

They are 1-based. It looks like the function comment should be clearer.

I hoped the "and" in "one-based line and column" would imply this. "one-based line and one-based column" may be a bit too verbose.

Ah, I see—I misunderstood this above as well. I would have expected 0-indexed columns, but do not mind that much either way as long as it's consistent. 😄

[dennwc]

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @bzz and @creachadair)

---

uast/transformer/positioner/positions.go, line 261 at r1 (raw file):

Previously, creachadair (M. J. Fromberger) wrote…

Suggestion: "…zero-based byte offset within the line."

Looking at this more carefully, the byte offset here is an absolute one (from the beginning of the file).

---

uast/transformer/positioner/positions.go, line 410 at r1 (raw file):

Previously, dennwc (Denys Smirnov) wrote…

Would clarify the docs here as well.

Seems to be resolved as well :)

[creachadair]

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @bzz, @creachadair, and @dennwc)

---

uast/transformer/positioner/positions.go, line 261 at r1 (raw file):

Previously, dennwc (Denys Smirnov) wrote…

Looking at this more carefully, the byte offset here is an absolute one (from the beginning of the file).

Yes, I misunderstood. I think your suggestion below was a good one, though:

// LineCol returns a one-based line number and one-based column offset given
// a zero-based byte offset within the file.

There might be other people as easily confused as I am 😁

[bzz]

Reviewed 2 of 2 files at r1.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @creachadair and @dennwc)

[bzz]

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @creachadair and @dennwc)

[dennwc]

Reviewable status: 1 of 2 files reviewed, 1 unresolved discussion (waiting on @bzz and @creachadair)

---

uast/transformer/positioner/positions.go, line 261 at r1 (raw file):

Previously, creachadair (M. J. Fromberger) wrote…

Yes, I misunderstood. I think your suggestion below was a good one, though:

// LineCol returns a one-based line number and one-based column offset given
// a zero-based byte offset within the file.

There might be other people as easily confused as I am 😁

Done.

---

uast/transformer/positioner/positions.go, line 410 at r1 (raw file):

Previously, dennwc (Denys Smirnov) wrote…

Seems to be resolved as well :)

Done.

[creachadair]

Reviewed 1 of 1 files at r2.
Reviewable status: complete! all files reviewed, all discussions resolved

---

uast/transformer/positioner/positions.go, line 410 at r1 (raw file):

Previously, dennwc (Denys Smirnov) wrote…

Done.

Thank you! 👍