Skip to content

Commit

Permalink
Add Go SQL/JSON Path post
Browse files Browse the repository at this point in the history
  • Loading branch information
theory committed Jul 6, 2024
1 parent ac74fc0 commit 94df6b6
Show file tree
Hide file tree
Showing 4 changed files with 128 additions and 7 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
- name: Check out Source
uses: actions/checkout@v3
- name: Build the Site
uses: docker://hugomods/hugo:base-0.128.0
uses: docker://hugomods/hugo:base-0.128.2
with: { args: hugo }
- name: Generate Publish Script
run: ./bin/publish public justatheory.com E1X44SJ45FTNGI 1 > publish.sh
Expand Down
117 changes: 117 additions & 0 deletions content/post/postgres/go-sqljson-path.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
title: Introduccing Go SQL/JSON Path and Playground
slug: go-sqljson-path
date: 2024-07-08T13:59:10Z
lastMod: 2024-07-08T13:59:10Z
description: |
Introducing the SQL/JSON Path package, a PostgresSQL-compatible jsonpath
parser and executor in Go. Also: a Wasm-based playground!
tags: [Postgres, SQL/JSON, JSON Path, Go, Playground]
type: post
---

For a personal project, I needed to parse and execute PostgreSQL-compatible
[jsonpath] expressions.[^why] So I've spent quite a few evenings and weekends
the last several months porting Postgres jsonpath to Go, and it's finally
ready to ship.

Introducing [Go SQL/JSON], featuring the [path package]. This project provides
full support for all of the PostgresSQL 17 jsonpath features[^nearly] in the
Go programming language. An [example]:

``` go
package main

import (
"context"
"encoding/json"
"fmt"
"log"

"github.com/theory/sqljson/path"
"github.com/theory/sqljson/path/exec"
)

func main() {
var value any
err := json.Unmarshal([]byte(`{"a":[1,2,3,4,5]}`), &value)
if err != nil {
log.Fatal(err)
}

p := path.MustParse("$.a[*] ? (@ >= $min && @ <= $max)")
res, err := p.Query(
context.Background(),
value,
exec.WithVars(exec.Vars{"min": float64(2), "max": float64(4)}),
)
if err != nil {
log.Fatal(err)
}

fmt.Printf("%v\n", res)
// Output: [2 3 4]
}
```

I think the API is decent, but may implement better patterns as I discover
them. Overall I'm quite satisfied with how it turned out, and just how well
its implementation and performance compare to the Postgres original.

🛝 Playground
-------------

But why stop there? One of the nice things about this project is that Go
supports compiling applications into [WebAssembly] (a.k.a. Wasm) via [Go
WebAssembly]. Borrowing from the [Goldmark] project, I created and published
the [sqljson/path playground] and populated [the docs] with links for all of
its examples.

Now anyone can experiment with SQL/JSON path expressions, and share links that
demonstrate patterns and techniques the like, like [this one] (JSON [borrowed
from MDM]). The Playground is a *stateless* JavaScript/Wasm web application:
data persists only in permalink URLs.

That's it, that's the Post
--------------------------

I've enjoyed learning how to implement a lexer, a [goyacc] parser, an [AST],
and an execution engine. The Playground was a bonus bit of fun!

I'm excited to build cool stuff on this package, but I don't know whether
anyone else will find it useful. If you do --- or even just enjoy messing
about on the Playground, let me know!

[^why]: "Whatever for," you ask? Well, aside from wanting to see if I could
do it, [this post] describes a POC. Now I'm working to create the real
thing --- done right and entirely from scratch.
[^nearly]: Well, nearly full. The only feature missing is the
`datetime(template)` method. See also the comprehensive [compatibility
notes].

[jsonpath]: https://www.postgresql.org/docs/16/datatype-json.html#DATATYPE-JSONPATH
"PostgreSQL Docs: jsonpath Type"
[this post]: {{% ref "/post/postgres/cipherdoc" %}}
"CipherDoc: A Searchable, Encrypted JSON Document Service on Postgres"
[Go SQL/JSON]: https://github.com/theory/sqljson
"theory/gosqljson: PostgreSQL-compatible SQL-standard SQL/JSON in Go"
[path package]: https://pkg.go.dev/github.com/theory/sqljson/path
"go.pkg.dev: github.com/theory/sqljson/path"
[compatibility notes]: https://github.com/theory/sqljson/blob/main/path/README.md#compatibility
[example]: https://pkg.go.dev/github.com/theory/sqljson/path#example-package-WithVars
[WebAssembly]: https://webassembly.org
[Go WebAssembly]: https://go.dev/wiki/WebAssembly "Go Wiki: WebAssembly"
[Goldmark]: https://github.com/yuin/goldmark/
"yuin/goldmark: 🏆 A markdown parser written in Go"
[path package]: https://github.com/theory/sqljson/tree/main/path
"Go SQL/JSON Path"
[sqljson/path playground]: https://theory.github.io/sqljson/playground
[the docs]: https://github.com/theory/sqljson/blob/main/path/README.md
"Go SQL/JSON Path: The SQL/JSON Path Language"
[this one]: https://theory.github.io/sqljson/playground/?p=%2524.members%255B*%255D%2520%253F%28%2540.age%2520%253C%252030%2520%257C%257C%2520exists%28%2540.powers%255B*%255D%2520%253F%28%2520%2540%2520%253D%253D%2520%2522Inferno%2522%29%29%29.name&j=%257B%250A%2520%2520%2522squadName%2522%253A%2520%2522Super%2520hero%2520squad%2522%252C%250A%2520%2520%2522homeTown%2522%253A%2520%2522Metro%2520City%2522%252C%250A%2520%2520%2522formed%2522%253A%25202016%252C%250A%2520%2520%2522secretBase%2522%253A%2520%2522Super%2520tower%2522%252C%250A%2520%2520%2522active%2522%253A%2520true%252C%250A%2520%2520%2522members%2522%253A%2520%255B%250A%2520%2520%2520%2520%257B%250A%2520%2520%2520%2520%2520%2520%2522name%2522%253A%2520%2522Molecule%2520Man%2522%252C%250A%2520%2520%2520%2520%2520%2520%2522age%2522%253A%252029%252C%250A%2520%2520%2520%2520%2520%2520%2522secretIdentity%2522%253A%2520%2522Dan%2520Jukes%2522%252C%250A%2520%2520%2520%2520%2520%2520%2522powers%2522%253A%2520%255B%2522Radiation%2520resistance%2522%252C%2520%2522Turning%2520tiny%2522%252C%2520%2522Radiation%2520blast%2522%255D%250A%2520%2520%2520%2520%257D%252C%250A%2520%2520%2520%2520%257B%250A%2520%2520%2520%2520%2520%2520%2522name%2522%253A%2520%2522Madame%2520Uppercut%2522%252C%250A%2520%2520%2520%2520%2520%2520%2522age%2522%253A%252039%252C%250A%2520%2520%2520%2520%2520%2520%2522secretIdentity%2522%253A%2520%2522Jane%2520Wilson%2522%252C%250A%2520%2520%2520%2520%2520%2520%2522powers%2522%253A%2520%255B%250A%2520%2520%2520%2520%2520%2520%2520%2520%2522Million%2520tonne%2520punch%2522%252C%250A%2520%2520%2520%2520%2520%2520%2520%2520%2522Damage%2520resistance%2522%252C%250A%2520%2520%2520%2520%2520%2520%2520%2520%2522Superhuman%2520reflexes%2522%250A%2520%2520%2520%2520%2520%2520%255D%250A%2520%2520%2520%2520%257D%252C%250A%2520%2520%2520%2520%257B%250A%2520%2520%2520%2520%2520%2520%2522name%2522%253A%2520%2522Eternal%2520Flame%2522%252C%250A%2520%2520%2520%2520%2520%2520%2522age%2522%253A%25201000000%252C%250A%2520%2520%2520%2520%2520%2520%2522secretIdentity%2522%253A%2520%2522Unknown%2522%252C%250A%2520%2520%2520%2520%2520%2520%2522powers%2522%253A%2520%255B%250A%2520%2520%2520%2520%2520%2520%2520%2520%2522Immortality%2522%252C%250A%2520%2520%2520%2520%2520%2520%2520%2520%2522Heat%2520Immunity%2522%252C%250A%2520%2520%2520%2520%2520%2520%2520%2520%2522Inferno%2522%252C%250A%2520%2520%2520%2520%2520%2520%2520%2520%2522Teleportation%2522%252C%250A%2520%2520%2520%2520%2520%2520%2520%2520%2522Interdimensional%2520travel%2522%250A%2520%2520%2520%2520%2520%2520%255D%250A%2520%2520%2520%2520%257D%250A%2520%2520%255D%250A%257D%250A&a=&o=1
[borrowed from MDM]: https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Objects/JSON
"MDN Web Docs: Working with JSON"
[goyacc]: https://pkg.go.dev/golang.org/x/tools/cmd/goyacc
"pkg.go.dev: golang.org/x/tools/cmd/goyacc"
[AST]: https://en.wikipedia.org/wiki/Abstract_syntax_tree
"Wikipedia: Abstract syntax tree"
15 changes: 9 additions & 6 deletions content/post/postgres/rfc-pgxn-metadata-sketch.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: "RFC: PGXN Metadata Sketch"
slug: rfc-pgxn-metadata-sketch
date: 2024-03-21T18:50:58Z
lastMod: 2024-03-25T22:11:37Z
lastMod: 2024-07-06T21:16:23Z
description: |
Request for comments on a sketch of a new metadata standard for Postgres
extension packaging, distribution, and delivery, building on the
Expand All @@ -20,6 +20,8 @@ and thank you!
**Update 2024-03-25:** Clarified the definition of "Release" and made
"Maintainers" plural. Thanks to Matthias van de Meent for the suggestions!

**Update 2024-07-06** Fixed some typos.

---

This post proposes a new metadata standard for extension packaging,
Expand All @@ -29,7 +31,8 @@ shortcomings and emerging use cases 12 years on. The goals include:
* Don't break the existing standard, but add to it
* Add explicit support for different types of Postgres extensions,
such as [background workers] and [loadable modules]
* Add additional metadata for curation and automated binary compilation
* Add additional metadata for curation and automated compilation and binary
packaging
* Add hooks for registry and third-party-contributed metadata, including
binary distribution options and stats & reports.

Expand All @@ -41,10 +44,10 @@ files:
2. [Registry Metadata](#registry-metadata) aggregated by the root registry
from various sources, including data derived from the extension source
code or package metadata, but also trusted third parties such as
packagers, smoke testers, security scanners, and more.
packagers, smoke testers, security scanners, and more

Following community discussion of this proposal, the [Package
Metadata](#package-metadata) will lead to a draft for PGXN Meta Spec version
Metadata](#package-metadata) will lead to a draft for PGXN Meta Spec version
2.0.0, while the [Registry Metadata](#registry-metadata) will guide the design
and implementation of the Root Registry APIs required to provide it.

Expand All @@ -53,7 +56,7 @@ and implementation of the Root Registry APIs required to provide it.
Definition of terms as used in this document, because every language and
distribution system uses terms differently. These are ours.

* **Extension:** a software component that extends the capabilities of a
* **Extension:** A software component that extends the capabilities of a
PostgreSQL database or cluster. Extensions may be `CREATE EXTENSION`
[extensions], [background workers], command-line apps, [loadable modules],
shared libraries, and more.
Expand Down Expand Up @@ -438,7 +441,7 @@ Example:
"windows": [ "amd64" ],
"freebsd": [ "amd64" ]
},
"dependencies": {
"prereqs": {
"configure": {
"requires": {
"external": { "cargo-pgrx": "" }
Expand Down
1 change: 1 addition & 0 deletions themes/justatheory/static/css/screen.css
Original file line number Diff line number Diff line change
Expand Up @@ -933,6 +933,7 @@ pre {
overflow: scroll;
overflow-y: hidden;
overflow-x: auto;
tab-size: 4;
}

p code, li code, p tt, li tt {
Expand Down

0 comments on commit 94df6b6

Please sign in to comment.