The native document API now supports the same :version option for optimistic locking that the REST api does.
Contributed by Richie Vos (Groupon).
GH issues: #54.
Elastisch now uses ElasticSearch exceptions instead of generic ones in the native client.
Contributed by Richie Vos (Groupon).
GH issues: #54, #57.
Elastisch now depends on ElasticSearch native client version 0.90.8.
clojurewerkz.elastisch.native.document/update-with-script invoked without
script parameters no longer raises an exception.
Cheshire dependency has been updated to 5.3.1.
:sort option can now be a com.elasticsearch.search.sort.SortBuilder instance
and not just a string.
Contributed by Mark Wong-VanHaren.
Elastisch now depends on ElasticSearch native client version 0.90.7.
Elastisch no longer erroneously inserts _index and _type fields
into documents inserted via bulk API.
Contributed by Max Barnash.
clojurewerkz.elastisch.native.document/scroll-seq and
clojurewerkz.elastisch.rest.document/scroll-seq
are new functions that accept a search query response
and return a lazy sequence of paginated search results.
This makes working with result sets that require pagination much more natural:
(require '[clojurewerkz.elastisch.native.document :as doc])
(let [index-name "articles"
mapping-type "article"
res-seq (doc/scroll-seq
(doc/search index-name mapping-type
:query (q/term :title "Emptiness")
:search_type "query_then_fetch"
:scroll "1m"
:size 2))]
res-seq))Contributed by Max Barnash.
Native client now supports upserts of documents:
(require '[clojurewerkz.elastisch.native.document :as doc])
(doc/upsert "people" "person" "elastisch" {:name "Elastisch" :language "Clojure"})clj-http dependency has been upgraded to version 0.7.7.
Date histogram in the native client now includes :total field.
Contributed by Jim Dunn.
Native client now returns the same value in :fields and :_fields
keys in search hits. This makes it both backwards compatible with
earlier versions and the format ElasticSearch HTTP API uses.
Elastisch now depends on ElasticSearch native client version 0.90.5.
Bulk index and delete operations support _parent and _routing
keys.
Contributed by Baptiste Fontaine.
Elastisch now requires Clojure 1.4.
Cheshire dependency has been upgraded to version 5.2.0.
clj-http dependency has been upgraded to version 0.7.6.
Elastisch now depends on ElasticSearch native client version 0.90.3.
Elastisch now will not perform a bulk operation if its list of operations is empty.
Contributed by Baptiste Fontaine.
Elastisch now depends on ElasticSearch native client version 0.90.2.
clojurewerkz.elastisch.rest.document/search,
clojurewerkz.elastisch.rest.document/search-all-types,
clojurewerkz.elastisch.rest.document/count,
clojurewerkz.elastisch.rest.document/delete-by-query, and
clojurewerkz.elastisch.rest.document/delete-by-query-across-all-types
now accepts the :ignore_indices option:
(doc/search [index-name, missing-index-name,...] mapping-type :query (q/match-all)
:ignore_indices "missing")See also elasticsearch/guide/reference/api
Contributed by Joachim De Beule
Search queries that only retrieve a subset of fields using
the :fields option are now correctly converted to Clojure maps.
Contributed by Soren Macbeth.
Elastisch now depends on ElasticSearch native client version 0.90.1.
clojurewerkz.elastisch.native.document/search now accepts maps as
:search option values:
(doc/search index-name mapping-type :query (q/match-all)
:sort (array-map "title" "asc")This is identical to how the option works with the REST client.
clojurewerkz.elastisch.rest.document/update-with-script is a new function
that updates a document with a provided script:
(require '[clojurewerkz.elastisch.rest.document :as doc])
;; initializes a counter at 1
(doc/update-with-script index-name mapping-type "1"
"ctx._source.counter = 1")
;; increments the counter by 4
(doc/update-with-script index-name mapping-type "1"
"ctx._source.counter += inc"
{"inc" 4})clojurewerkz.elastisch.native.document/update-with-script is the native
client counterpart that takes the same arguments.
Native client is now over 50% faster on most commonly used operations thanks to much lower conversion overhead from ElasticSearch native client data structures to Clojure maps.
Contributed by Jon Pither.
Elastisch 1.1.0 includes a major new feature: native ElasticSearch client.
The client uses ElasticSearch's Java API, and can be used with
both transport and node clients.
Native client is more bandwidth efficient. It also can use SMILE (binary JSON format) to be more efficient on the wire.
Native client API in Elastisch is nearly identical to that of the REST API client
and resides in clojurewerkz.elastisch.native and clojurewerkz.elastisch.native.*
namespaces (similarly to how clojurewerkz.elastisch.rest clojurewerkz.elastisch.rest.*
namespaces are organized).
Transport client (used for TCP/remote connections) connections are set up using
clojurewerkz.elastisch.native/connect!. Note that you need to provide node
configuration that at least has cluster name in it:
(require '[clojurewerkz.elastisch.native :as es])
;; note that transport client uses port 9300 by default.
;; it also can connect to multiple cluster nodes
(es/connect! [["127.0.0.1" 9300]]
{"cluster.name" "elasticsearch_antares" })Cluster name and transport node addresses can be retrieved via HTTP API, for example:
curl http://localhost:9200/_cluster/nodes
{"ok":true,"cluster_name":"elasticsearch_antares","nodes":...}}
The Native client tries to be as close as possible to the existing REST client API.
For example, document operation functions in clojurewerkz.elastisch.native.document,
such as clojurewerkz.elastisch.native.document/create,
follow clojurewerkz.elastisch.rest.document function signatures as closely as
possible:
;; in the REPL
(require '[clojurewerkz.elastisch.native :as es])
(require '[clojurewerkz.elastisch.native.document :as doc])
(es/connect! [["127.0.0.1" 9300]]
{"cluster.name" "elasticsearch_antares" })
(doc/put index-name index-type id document)
(doc/get index-name index-type id)The same with returned results. Note, however, that ES transport client does have (very) minor differences with the REST API and it is not always possible for Elastisch to completely cover such differences.
Native client offers a choice of synchronous (blocking calling thread until a response is received) and asynchronous (returns a future) versions of multiple API operations:
;; in the REPL
(require '[clojurewerkz.elastisch.native :as es])
(require '[clojurewerkz.elastisch.native.document :as doc])
(es/connect! [["127.0.0.1" 9300]]
{"cluster.name" "elasticsearch_antares" })
(doc/put index-name index-type id document)
;; returns a response
(doc/get index-name index-type id)
;; returns a future that will eventually
;; contain a response
(doc/async-get index-name index-type id)One notable exception to this is administrative operations (such as opening or closing an index). The rationale for this is that they are rarely executed on the hot code path (e.g. in tight loops), so convenience and better error visibility is more important for them.
GH issues: #17, #18, #20.
Note that native ElasticSearch client currently relies on ElasticSearch 0.90.0.Beta1 client libraries and some operations will only work with that version.
Elastisch now depends on org.clojure/clojure version 1.5.0. It is still compatible with Clojure 1.3+ and if your project.clj depends
on 1.3 or 1.4, it will be used, but 1.5 is the default now.
We encourage all users to upgrade to 1.5, it is a drop-in replacement for the majority of projects out there.
Bulk requests are now supported. All the relevant code is in the clojurewerkz.elastisch.rest.bulk
namespace. Here is a small example of bulk document indexing using this new API:
(require '[clojurewerkz.elastisch.rest.bulk :as eb])
(eb/bulk (eb/bulk-index doc1 doc2 doc3) :refresh true)Contributed by Davie Moston.
Scroll queries are now easier to perform thanks to the new clojurewerkz.elastisch.rest.document/scroll
function that takes a scroll id and amount of time retrieved documents and related information
will be kept in memory for future retrieval. They are analogous to database cursors.
A short code example:
(require '[clojurewerkz.elastisch.rest.document :as doc])
(require '[clojurewerkz.elastisch.query :as q])
(require '[clojurewerkz.elastisch.rest.response :refer [hits-from]])
(let [index-name "articles"
mapping-type "article"
response (doc/search index-name mapping-type
:query (q/query-string :query "*")
:search_type "scan"
:scroll "1m"
:size 1)
scroll-id (:_scroll_id response)
scan-response (doc/scroll scroll-id :scroll "1m")
scan-hits (hits-from scan-response)]
(println scan-hits))Contributed by Davie Moston.
Cheshire dependency has been upgraded to version 5.1.1.
clj-http dependency has been upgraded to version 0.7.2.
clojurewerkz.elastisch.rest.document/count no longer ignores mapping types.
GH issue: #6.
clojurewerkz.elastisch.rest.document/count now correctly uses GET for requests without
the query part andPOST for request that have it.
GH issue: #5.
Elastisch will now URL-encode document ids.
clj-http dependency has been upgraded to version 0.5.5.
Cheshire dependency has been upgraded to version 4.0.3.
clojurewerkz.elastisch.rest.document/validate-query is a new function that implements support for the Validation API:
(require '[clojurewerkz.elastisch.rest.document :as doc])
(require '[clojurewerkz.elastisch.query :as q])
(require '[clojurewerkz.elastisch.rest.response :as r])
(let [response (doc/validate-query "myproduct_development" (q/field "latest-edit.author" "Thorwald") :explain true)]
(println response)
(println (r/valid? response)))Query validation does not execute the query.
clojurewerkz.elastisch.rest.percolation is a new namespace with functions that implement the Percolation API.
(require '[clojurewerkz.elastisch.rest.percolation :as pcl])
(require '[clojurewerkz.elastisch.rest.response :as r])
;; register a percolator query for the given index
(pcl/register-query "myapp" "sample_percolator" :query {:term {:title "search"}})
;; match a document against the percolator
(let [response (pcl/percolate "myapp" "sample_percolator" :doc {:title "You know, for search"})]
(println (r/ok? response))
;; print matches
(println (r/matches-from response)))
;; unregister the percolator
(pcl/unregister-query "myapp" "sample_percolator")clj-http dependency has been upgraded to version 0.5.2.
Elastisch now uses (and depends on) Cheshire for JSON serialization. clojure.data.json is no longer a dependency.
clojurewerkz.elastisch.rest.response/facets-from is a new convenience function that returns the facets section of a response.
The exact response format will vary between facet types and queries but it is always returned as an immutable map and has the same
structure as in the respective ElasticSearch response JSON document.
Elastisch now depends on org.clojure/clojure version 1.4.0. It is still compatible with Clojure 1.3 and if your project.clj depends
on 1.3, it will be used, but 1.4 is the default now.
We encourage all users to upgrade to 1.4, it is a drop-in replacement for the majority of projects out there.
ElasticSearch 0.19.9 renames Text Query to Match Query. Elastisch adapts by introducing clojurewerkz.elastisch.query/match that
is effectively an alias for clojurewerkz.elastisch.query/text (ElasticSearch still supports :text in the query DSL for backwards
compatibility).
Documentation guides were greatly improved.
clj-http dependency has been upgraded to version 0.5.1.
clojurewerkz.elastisch.query/text now takes two arguments, the first being the name of the field.
It was mistakenly hardcoded previously.¯
clojurewerkz.elastisch.rest.index/create-template, clojurewerkz.elastisch.rest.index/delete-template
and clojurewerkz.elastisch.rest.index/get-template are new functions that implement support for index templates:
(clojurewerkz.elastisch.rest.index/create-template "accounts" :template "account*" :settings {:index {:refresh_interval "60s"}})
(clojurewerkz.elastisch.rest.index/get-template "accounts")
(clojurewerkz.elastisch.rest.index/delete-template "accounts")clojurewerkz.elastisch.rest.index/update-aliases and clojurewerkz.elastisch.rest.index/get-aliases
are new functions that implement support for index aliases:
(clojurewerkz.elastisch.rest.index/update-aliases [{:add {:index "client0000001" :alias "alias1"}}
{:add {:index "client0000002" :alias "alias2"}}])
(clojurewerkz.elastisch.rest.index/get-aliases "client0000001")clojurewerkz.elastisch.rest.index/optimize is a function that optimizes an index:
(clojurewerkz.elastisch.rest.index/optimize "my-index" :refresh true :max_num_segments 48)clojurewerkz.elastisch.rest.index/flush is a function that flushes an index:
(clojurewerkz.elastisch.rest.index/flush "my-index" :refresh true)clojurewerkz.elastisch.rest.index/snapshot is a function that takes a snapshot of an index or multiple indexes:
(clojurewerkz.elastisch.rest.index/snapshot "my-index")clojurewerkz.elastisch.rest.index/clear-cache is a function that can be used to clear index caches:
(clojurewerkz.elastisch.rest.index/clear-cache "my-index" :filter true :field_data true)It takes the same options as documented in the ElasticSearch guide on the Clear Cache Index operation
clojurewerkz.elastisch.rest.index/status is a function that returns status an index or multple indexes:
(clojurewerkz.elastisch.rest.index/status "my-index" :recovery true :snapshot true)clojurewerkz.elastisch.rest.index/segments is a function that returns segments information for an index or multiple indexes:
(clojurewerkz.elastisch.rest.index/segments "my-index")clojurewerkz.elastisch.rest.index/stats is a function that returns statistics for an index or multiple indexes:
(clojurewerkz.elastisch.rest.index/stats "my-index" :docs true :store true :indexing true)It takes the same options as documented in the ElasticSearch guide on the Stats Index operation
clj-http dependency has been upgraded to version 0.5.0.
clojurewerkz.elastisch.rest.document/delete-by-query-across-all-types is a new function that searches across
one or more indexes and all mapping types:
(doc/delete-by-query-across-all-types index-name (q/term :username "esjoe"))clojurewerkz.elastisch.rest.document/delete-by-query-across-all-indexes-and-types is another new function that searches across all indexes and all mapping types:
(doc/delete-by-query-across-all-indexes-and-types (q/term :username "esjoe"))clojurewerkz.elastisch.rest.document/search-all-types is a new function that searches across
one or more indexes and all mapping types:
(doc/search-all-types ["customer1_index" "customer2_index"] :query (q/query-string :query "Austin" :default_field "title"))clojurewerkz.elastisch.rest.document/search-all-indexes-and-types is another new function that searches across all indexes and all mapping types:
(doc/search-all-indexes-and-types :query (q/query-string :query "Austin" :default_field "title"))It is now possible to create indexes without specifying mapping types: clojurewerkz.elastisch.rest.index/create
no longer requires :mapping to be passed.
Elastisch now uses clj-http 0.4.x.
HTTP/REST API namespaces are now grouped under clojurewerkz.elastisch.rest, for example, what used to be clojurewerkz.elastisch.document is now
clojurewerkz.elastisch.rest.document. This is done to leave room for Memcached transport support in the future.
Elastisch now supports Custom Filters Score queries.
Elastisch now supports Nested queries.
Elastisch now supports Indices queries.
Elastisch now supports Top Children queries.
Elastisch now supports Has Child queries.
Elastisch now supports Wildcard queries.
Elastisch now supports span queries:
Span queries are used for proximity search.
Elastisch now supports Query String queries.
Elastisch now supports More Like This Field queries.
clj-time dependency has been upgraded to version 0.4.1.
Elastisch now supports More Like This queries.
Elastisch now supports match all queries.
Elastisch now supports fuzzy (edit distance) queries.
Elastisch now supports fuzzy like this and fuzzy like this field queries.
Elastisch now supports prefix queries, see clojurewerkz.elastisch.query/prefix,
clojurewerkz.elastisch.query/field, clojurewerkz.elastisch.query/filtered, and clojurewerkz.elastisch.document/search to learn more.
clojurewerkz.elastisch.document/more-like-this provides access to the Elastic Search More Like This API for
documents and returns documents similar to a given one:
(doc/more-like-this "people" "person" "1" :min_term_freq 1 :min_doc_freq 1)Please note that :min_doc_freq and :min_term_freq parameters may be very important for small data sets.
If you observe responses with no results, try lowering them.
clojurewerkz.elastisch.document/count provides access to the Elastic Search count API
and is almost always used with a query, for example:
(doc/count "people" "person" (q/term :username "clojurewerkz"))clojurewerkz.elastisch.document/delete-by-query provides access to the Delete by query API of Elastic Search, for example:
(doc/delete-by-query "people" "person" (q/term :tag "mongodb"))clojurewerkz.elastisch.document/replace deletes a document by id and immediately adds a new one
with the same id.
clojurewerkz.elastisch.response was extracted from clojurewerkz.elastisch.utils
Elastisch now uses Leiningen 2.