From 77e3fcef57e335a211f317ebe53f380e9a063479 Mon Sep 17 00:00:00 2001 From: Max Penet Date: Fri, 15 Sep 2023 11:15:18 +0200 Subject: [PATCH] docs --- API.md | 389 ++++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 2 +- deps.edn | 21 ++- quickdoc.sh | 1 + 4 files changed, 405 insertions(+), 8 deletions(-) create mode 100644 API.md create mode 100644 quickdoc.sh diff --git a/API.md b/API.md new file mode 100644 index 0000000..997fbe0 --- /dev/null +++ b/API.md @@ -0,0 +1,389 @@ +# qbits.spandex + + + + + +## `->request` +``` clojure + +(->request + {:as request-map, + :keys [url method query-string headers body response-consumer-factory request-options], + :or {method :get, headers default-headers}}) +``` + +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L259-L289) +## `BodyEncoder` +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L148-L149) +## `Closable` +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L133-L134) +## `ExceptionDecoder` +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L228-L229) +## `bulk-chan` + +Bulk-chan takes a client, a partial request/option map, returns a + map of `:input-ch`, `:output-ch` and `:flush-ch`. + + `:input-ch` is a channel that will accept bulk fragments to be + sent (either single or collection). It then will + wait (:flush-interval request-map) or (:flush-threshold request-map) + and then trigger an async request with the bulk payload accumulated. + + Parallelism of the async requests is controllable + via (:max-concurrent-requests request-map). + + If the number of triggered requests exceeds the capacity of the job + buffer, puts! in `:input-ch` will block (if done with async/put! you + can check the return value before overflowing the put! pending + queue). + + Jobs results returned from the processing are a pair of job and + responses map, or exception. The :output-ch will allow you to + inspect [job responses] the server returned and handle potential + errors/failures accordingly (retrying etc). + + If you close! the `:input-ch` it will close the underlying resources + and exit cleanly (consuming all jobs that remain in queues). + + By default requests are run against _bulk, but the option map is + passed as is to request-chan, you can overwrite options here and + provide your own url, headers and so on. + + `:flush-ch` is provided to afford an immediate flush of the contents + of the job buffer by putting anything onto this channel. +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L417-L522) +## `chunks->body` +``` clojure + +(chunks->body chunks) +``` + + +Utility function to create _bulk/_msearch bodies. It takes a + sequence of clj fragments and returns a newline delimited string of + JSON fragments +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L406-L415) +## `client` +``` clojure + +(client) +(client options) +``` + + +Returns a client instance to be used to perform requests. + + Options: + + * `:hosts` : Collection of URIs of nodes - Defaults to ["http://localhost:9200"] + + * `:default-headers` : Sets the default request headers, which will be + sent along with each request. Request-time headers will always + overwrite any default headers. + + * `:failure-listener` : Sets the RestClient.FailureListener to be + notified for each request failure + + * `:request` : request scoped config - extendable via the `qbits.client-options/set-request-option!` multimethod) + * `:authentication?` + * `:circular-redirect-allowed?` + * `:connect-timeout` + * `:connection-request-timeout` + * `:content-compression?` + * `:expect-continue?` + * `:local-address` + * `:max-redirects` + * `:proxy` + * `:redirects?` + * `:relative-redirects-allowed?` + * `:socket-timeout` + * `:target-preferred-auth-schemes` + + * `:http-client` : http-client scoped config - extendable via the `qbits.client-options/set-http-client-option!` multimethod) + * `:max-conn-per-route` + * `:max-conn-total` + * `:proxy` + * `:ssl-context` + * `:ssl-noop-hostname-verifier?` + * `:user-agent` + * `:auth-caching?` + * `:cookie-management?` + * `:basic-auth` (map of `:user` `:password`) + + If you need extra/custom building you can hook into the builder by + extending the multimethod + `qbits.spandex.client-options/set-option!` +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L40-L86) +## `close!` +``` clojure + +(close! this) +``` + +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L133-L134) +## `decode-exception` +``` clojure + +(decode-exception x) +``` + + +Controls how to translate a client exception +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L228-L229) +## `default-exception-handler` +``` clojure + +(default-exception-handler ex) +``` + + +Exception handler that will try to decode Exceptions via + ExceptionDecoder/decode-exception. If after decoding we still have a + throwable it will rethrow, otherwise it will pass on the value + returned. You can then extend ExceptionDecoder/decode-exception to + do whatever you'd like. +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L237-L247) +## `default-headers` +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L168-L168) +## `encode-body` +``` clojure + +(encode-body x) +``` + +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L148-L149) +## `node-selector` +``` clojure + +(node-selector f) +``` + + +Creates a NodeSelector instance that will call `f` as select(): + see: https://github.com/elastic/elasticsearch/blob/master/client/rest/src/main/java/org/elasticsearch/client/NodeSelector.java#L29 +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L125-L131) +## `raw` +``` clojure + +(raw body) +``` + + +Marks body as raw, which allows to skip JSON encoding +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L143-L146) +## `request` +``` clojure + +(request + client + {:keys [method url headers query-string body keywordize? response-consumer-factory exception-handler], + :or + {method :get, + keywordize? true, + exception-handler default-exception-handler, + response-consumer-factory HttpAsyncResponseConsumerFactory/DEFAULT}, + :as request-params}) +``` + +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L291-L306) +## `request-async` +``` clojure + +(request-async + client + {:keys [method url headers query-string body success error keywordize? response-consumer-factory exception-handler], + :or + {method :get, + keywordize? true, + exception-handler decode-exception, + response-consumer-factory HttpAsyncResponseConsumerFactory/DEFAULT}, + :as request-params}) +``` + + +Similar to `qbits.spandex/request` but returns immediately and works + asynchronously and triggers option `:success` once a results is + received, or `:error` if it was a failure +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L308-L330) +## `request-chan` +``` clojure + +(request-chan client {:as options, :keys [ch]}) +``` + + +Similar to `qbits.spandex/request` but runs asynchronously and + returns a `core.async/promise-chan` that will have the result (or + error) delivered upon reception +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L332-L348) +## `response-ex->ex-info` +``` clojure + +(response-ex->ex-info re) +``` + + +Utility function to transform an ResponseException into an ex-info +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L221-L226) +## `response-ex->response` +``` clojure + +(response-ex->response re) +``` + + +Return the response-map wrapped in a ResponseException +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L216-L219) +## `scroll-chan` +``` clojure + +(scroll-chan client {:as request-map, :keys [ttl output-ch], :or {ttl "1m"}}) +``` + + +Returns a core async channel. Takes the same args as + `qbits.spandex/request`. Perform async scrolling requests for a + query, request will be done as the user takes from the + channel. Every take! will request/return a page from the + scroll. You can specify scroll :ttl in the request map otherwise + it'll default to 1m. The chan will be closed once scroll is + complete. If you must stop scrolling before that, you must + async/close! manually, this will release all used resources. You can + also supply a :output-ch key to the request map, a core.async/chan that + will receive the results. This allow you to have custom buffers, or + have multiple scroll-chan calls feed the same channel instance +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L350-L404) +## `set-sniff-on-failure!` +``` clojure + +(set-sniff-on-failure! sniffer) +``` + + +Register a SniffOnFailureListener that allows to perform sniffing + on failure. +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L118-L123) +## `sniffer` +``` clojure + +(sniffer client) +(sniffer + client + {:as options, + :keys [scheme timeout], + :or {scheme :http, timeout ElasticsearchNodesSniffer/DEFAULT_SNIFF_REQUEST_TIMEOUT}}) +``` + + +Takes a Client instance (and possible sniffer options) and returns + a sniffer instance that will initially be bound to passed client. + Options: + + * `:sniff-interval` : Sets the interval between consecutive ordinary + sniff executions in milliseconds. Will be honoured when + sniffOnFailure is disabled or when there are no failures between + consecutive sniff executions. + + * `:sniff-after-failure-delay` : Sets the delay of a sniff execution + scheduled after a failure (in milliseconds) + + + If you need extra/custom building you can hook into the builder by + extending the multimethod + `qbits.spandex.sniffer-options/set-option!` +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex.clj#L90-L116) +# qbits.spandex.client-options + + + + + +## `builder` +``` clojure + +(builder {:as options, :keys [hosts]}) +``` + +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/client_options.clj#L209-L215) +## `set-http-client-option!` +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/client_options.clj#L100-L100) +## `set-request-option!` +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/client_options.clj#L41-L41) +## `ssl-context-trust-all` +``` clojure + +(ssl-context-trust-all) +``` + + +Return a SSLContext that trusts all certificate +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/client_options.clj#L30-L38) +# qbits.spandex.sniffer-options + + + + + +## `builder` +``` clojure + +(builder client sniffer options) +``` + +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/sniffer_options.clj#L28-L32) +# qbits.spandex.url + + + + + +## `URL` +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/url.clj#L8-L9) +## `URLFragment` +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/url.clj#L11-L12) +## `encode` +``` clojure + +(encode x) +``` + +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/url.clj#L8-L9) +## `encode-fragment` +``` clojure + +(encode-fragment value) +``` + +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/url.clj#L11-L12) +# qbits.spandex.utils + + + + + +## `chan->seq` +``` clojure + +(chan->seq ch) +``` + + +Convert a channel to a lazy sequence. + + Will block on after the last element if the channel is not closed. +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/utils.clj#L9-L15) +## `escape-query-string` +``` clojure + +(escape-query-string query) +``` + + +Escape or remove special characters in query string coming from users. + + See: + https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_reserved_characters +
[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/utils.clj#L17-L29) +## `url` +[source](https://github.com/mpenet/spandex/blob/main/src/clj/qbits/spandex/utils.clj#L7-L7) diff --git a/README.md b/README.md index e52e7b8..6e8a1c5 100644 --- a/README.md +++ b/README.md @@ -177,7 +177,7 @@ it's quite cheap. ## API Docs -[![cljdoc badge](https://cljdoc.xyz/badge/cc.qbits/spandex)](https://cljdoc.xyz/d/cc.qbits/spandex/CURRENT) +[QuickDoc](API.md) Or the [clj.specs](https://github.com/mpenet/spandex/blob/master/src/clj/qbits/spandex/spec.clj) if that's your thing: diff --git a/deps.edn b/deps.edn index d56c75e..b4a4a09 100644 --- a/deps.edn +++ b/deps.edn @@ -16,10 +16,17 @@ :paths ["src/clj"] - :aliases {:test {:extra-paths ["test"] - :extra-deps {eftest/eftest {:mvn/version "0.6.0"} - orchestra/orchestra {:mvn/version "2021.01.01-1"}}} - :project - {:extra-deps {io.github.exoscale/tools.project {:git/sha "5f24196ebea4dc6e601d201d97b463ea26923c7e"}} - :ns-default exoscale.tools.project - :jvm-opts ["-Dclojure.main.report=stderr"]}}} + :aliases + {:test {:extra-paths ["test"] + :extra-deps {eftest/eftest {:mvn/version "0.6.0"} + orchestra/orchestra {:mvn/version "2021.01.01-1"}}} + :project + {:extra-deps {io.github.exoscale/tools.project {:git/sha "5f24196ebea4dc6e601d201d97b463ea26923c7e"}} + :ns-default exoscale.tools.project + :jvm-opts ["-Dclojure.main.report=stderr"]} + :quickdoc + {:deps {org.babashka/cli {:mvn/version "0.4.36"} + io.github.borkdude/quickdoc + {:deps/root "jvm" + :git/sha "c5320cbe311b651a60b47f4d00d7e8ab63291b6e"}} + :main-opts ["-m" "babashka.cli.exec" "quickdoc.api" "quickdoc"]}}} diff --git a/quickdoc.sh b/quickdoc.sh new file mode 100644 index 0000000..1e37f04 --- /dev/null +++ b/quickdoc.sh @@ -0,0 +1 @@ +clj -M:quickdoc :github/repo https://github.com/mpenet/spandex :git/branch main