From c3e098734eb5b751918cc6ecb624fbf3536d12c7 Mon Sep 17 00:00:00 2001 From: Arne Fischereit <79841228+arnefischereit@users.noreply.github.com> Date: Tue, 28 Nov 2023 16:00:06 +0100 Subject: [PATCH] IS :: STRING NOT NULL as an index compatible predicate (#776) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Jens Pryce-Åklundh <112686610+JPryce-Aklundh@users.noreply.github.com> --- ...ions-additions-removals-compatibility.adoc | 11 ++++ .../query-tuning/indexes.adoc | 53 +++++++++++++------ 2 files changed, 48 insertions(+), 16 deletions(-) diff --git a/modules/ROOT/pages/deprecations-additions-removals-compatibility.adoc b/modules/ROOT/pages/deprecations-additions-removals-compatibility.adoc index 6505aed34..8c82b53bb 100644 --- a/modules/ROOT/pages/deprecations-additions-removals-compatibility.adoc +++ b/modules/ROOT/pages/deprecations-additions-removals-compatibility.adoc @@ -83,6 +83,17 @@ SHOW VECTOR INDEXES | Extended xref:indexes-for-search-performance.adoc#indexes-list-indexes[`SHOW INDEXES`] with easy filtering for vector indexes. This is equivalent to `SHOW INDEXES WHERE type = 'VECTOR'`. +a| +label:functionality[] +label:updated[] + +[source, cypher, role="noheader"] +---- +MATCH (n:Label) WHERE $param IS :: STRING NOT NULL AND n.prop = $param +---- + +| `IS :: STRING NOT NULL` is now an xref:planning-and-tuning/query-tuning/indexes.adoc#text-index-type-predicate-expressions[index-compatible predicate]. + |=== === New features diff --git a/modules/ROOT/pages/planning-and-tuning/query-tuning/indexes.adoc b/modules/ROOT/pages/planning-and-tuning/query-tuning/indexes.adoc index 7de1cbf7d..34ee4b2af 100644 --- a/modules/ROOT/pages/planning-and-tuning/query-tuning/indexes.adoc +++ b/modules/ROOT/pages/planning-and-tuning/query-tuning/indexes.adoc @@ -172,7 +172,6 @@ For more information, see xref::planning-and-tuning/query-tuning/indexes.adoc#ex == TEXT indexes In combination with node label and relationship type predicates, `TEXT` indexes only solve predicates operating on strings. -That means that `TEXT` indexes are only used when it is known that the predicate evaluates to `null` for all non-string values. Predicates that only operate on strings are always solvable by a `TEXT` index: @@ -187,7 +186,7 @@ However, other predicates are only used when it is known that the property is co This means that a `TEXT` index is not able to solve e.g. `a.prop = b.prop`, unless a type constraint also exists on the property. -In summary, `TEXT` indexes support the following predicates: +`TEXT` indexes support the following predicates: [cols="2", options="header"] |=== @@ -223,34 +222,56 @@ ENDS WITH | Substring search. a| -[source, syntax, role="noheader"] +[source,syntax,role="noheader"] ---- CONTAINS ---- +| +xref:values-and-types/type-predicate.adoc[`STRING` type predicates with existence check]. +For more information, see xref:planning-and-tuning/query-tuning/indexes.adoc#text-index-type-predicate-expressions[TEXT indexes and type predicate expressions]. +a| +[source,syntax,role="noheader"] +---- +IS :: STRING NOT NULL +---- + |=== -[NOTE] -==== -As of Neo4j 5.11, the above set of predicates can be extended with the use of type constraints, see xref::planning-and-tuning/query-tuning/indexes.adoc#extending-index-compatibility-with-type-constraints[Extending index compatibility with type constraints]. -==== -In some cases, the system cannot determine whether an expression is of type string. +[[text-index-type-predicate-expressions]] +=== TEXT indexes and type predicate expressions + +`TEXT` indexes only solve predicates if it is known that the predicate evaluates to `false` or `null` for all non-`STRING` values. +However, in Cypher, `null IS {two-colons} STRING` is `true`, and indexes do not store `null` values. +This means that `TEXT` indexes cannot be used if any of the queried properties are `null` instead of a `STRING` value. + +To use `TEXT` indexes in situations where any of the queried properties may be `null` rather than a `STRING` value, add the type predicate expression `IS {two-colons} STRING NOT NULL` (or its alias, introduced in Neo4j 5.14, `IS {two-colons} STRING!`) to the query. +This will enforce both the existence of a property and its `STRING` type, discarding any rows where the property is missing or not of type `STRING`, and thereby enable the use of `TEXT` indexes + +For example, the following `MATCH` clause may not be solvable with a `TEXT` index because some of the queried properties may be `null`: -For example when the compared value is a parameter: [source, cypher, role="noheader", role=test-skip] ---- MATCH (n:Label) WHERE n.prop = $param ---- -Such queries can be modified to provide this information. -Depending on how values that are not of type string should be treated, there are two options: +The same clause, with `IS {two-colons} STRING NOT NULL` added, would, however, be solvable with an existing `TEXT` index: + +[source, cypher, role="noheader", role=test-skip] +---- +MATCH (n:Label) WHERE $param IS :: STRING NOT NULL AND n.prop = $param +---- + +[NOTE] +While type predicate expressions were introduced in Neo4j 5.9, the `IS {two-colons} STRING NOT NULL` syntax only became an index-compatible predicate in Neo4j 5.15. + +For more information, see the page about xref:values-and-types/type-predicate.adoc[type predicate expressions]. + +There are two further methods which can be used to ensure that an expression is of type `STRING`: -* If rows in which the expression is not of type string should be discarded, then adding `WHERE STARTS WITH ''` is the right option: -`MATCH (n:Label) WHERE $param STARTS WITH '' AND n.prop = $param` -* If expressions which are not of type string should be converted to string, then wrapping these in `toString()` is the right choice: -`MATCH (n:Label) WHERE n.prop = toString($param)` -* If it is known that the property is always of type `STRING`, then it's also possible to xref::planning-and-tuning/query-tuning/indexes.adoc#extending-index-compatibility-with-type-constraints[add a type constraint to help the planner]. +* It is possible to use the xref:functions/string.adoc#functions-tostring[`toString()`] function to convert an expression to `STRING` values. +* If it is known that the property is always of type `STRING`, then it is also possible to xref::planning-and-tuning/query-tuning/indexes.adoc#extending-index-compatibility-with-type-constraints[add a type constraint to help the planner]. [[extending-index-compatibility-with-type-constraints]] == Extending index compatibility with type constraints