Fall Release 3.4.0.4

We're really excited about this APOC release. It contains a lot of solved issues and new features.

Thanks a lot to all contributors that helped crafting the release, especially the folks from our partner Larus, Italy (@conker84 , @AngeloBusato, @omarlarus, @albertodelazzari ), @szarnyasg, my colleagues @sarmbruster, @michael-simons, @kvegter

Let's have a look at the changes:

• apoc.export.json - exports your database/query/subgraph as stream of json objects
• apoc.text.format - additional locale for formatting
• apoc.coll.frequenciesAsMap - the item is the key an the count the value
• added fast procedures for finding distinct neighbours of a node many hops apart
• cypher initializer for running statements at startup of the database
• Added apoc.trigger.removeAll

Issues:

• support older JDBC drivers with missing isClosed method
• Support Temporal types in apoc.load.jdbc
• remove support for XML entities (CVE)
• fix one-line XML import
• handle relative import paths correctly if configured
• apoc.schema.assert does not drop existence constraints
• JSON numbers are loaded as long values, like Cypher supports
• apoc.export.cypher.all works with temporal types
• deal gracefully with URLStreamHandlerFactory init issues
• better argument handling for date format functions
• Fix Graph Refactoring for merge nodes

As usual, please try them out and let us know if everything works for you.
If you run into any issues please raise a GitHub issue here, or ask on the Community Site.

Beta Release for Neo4j 3.5.0-beta01

This release is mostly fixing API issues with Neo4j 3.5 and to test out APOC with Neo4j 3.5

Notable changes:

• we will remove several the graph algorithms as the internal Neo4j APIs have been removed
• startup changes slightly internally from having one startup cycle to one for the dbms and one per db.

No real changes since 3.4.0.3, some minor documentation updates.

GraphConnect Release 3.4.0.3

Just in time for GraphConnect 2081 in NYC, we are happy to release APOC 3.4.0.3.

Thanks so much to all the contributors that have been really busy.

This release has some neat new features.

We added support for defining custom procedures and functions implemented in plain Cypher and then call them like regular ones.

CALL apoc.custom.asProcedure('neighbours',
  'MATCH (n:Person {name:$name})-->(nb) RETURN nb as neighbour','read',
  [['neighbour','NODE']],[['name','STRING']]);

CALL custom.neighbours('Keanu Reeves') YIELD neighbour;

You can now load data from inside of zip archives by appending the !path/to/file.csv to your file url.

To scrape data from websites you can use apoc.load.html, which is really useful.

The warmup.run() procedure was rewritten to also allow warm-up of index files and others.

You can now stream the results of apoc.export.csv.* directly to the client within your Cypher result instead writing a file on the server.

Explicit index procedures can now return the number of hits instead of the data.

Date formatting can now use named formats, something we got inspiration for from ElasticSearch.

Couchbase (4.x and 5.x) new authentication methods are now supported.

With apoc.text.index(es)Of you can get one or more occurrences of substrings.

There were a lot of docs updates, for JDBC, text, map, XML, schema and much more.

Please try out the new features and report back how well they work for you.

You can also ask questions around APOC in the new Neo4j Community Forum

Summer Release 3.4.0.2

This summer release of APOC has seen a lot of contributions from many folks.

• Alberto De Lazzari
• Angelo Busato
• Gabor Szarnyas
• Alex Iudice
• Cayetano
• ArGeBre
• Benjamin
• Tobias Lindaaker
• Adam Cowley
• Karol Brejna
• Andrew Bowman

Thanks so much to everyone who contributed new functionality, documentation updates and fixes.

Here are the highlights:

• Version upgrade Neo4j 3.4.5 APOC 3.4.0.2

• Support reverse geocoding in apoc.spatial (#876)

• Support of Base64url encoding and decoding (#870)

• apoc.text.clean strips utf8 characters (incl. cyrillic/chinese/japanese symbols) (#869)

• Add support for loading CSVs formatted according the import tool's specification (#581)

• add a configuration option to mergeNodes that allows to combine relationships of the same type and direction (#850)

• implement functions from pgsql - degree, label exists (#860)

• Adds text similarity/distance methods and double metaphone text encoder (#865)

• Added support to function apoc.date.fromISO8601 (#837)

• Added unit "weekday" to apoc.date.field() (#838)

• Added optional 4th parameter to procedure 'apoc.periodic.repeat'. (#853)

• Added optional parameter to procedure 'apoc.trigger.add' that allows configuration to be passed in. (#859)

• Add concurrency parameter (default to 50) to periodic.iterate

• add index selectivity to apoc.meta.data apoc.meta.schema and the apoc.index procedures (#842)

• Add apoc.diff user functions (#760)

• Read headers for load csv from config object (#834)

• Allow passing parameters to apoc.cypher.runFile (#813)

• Allow direction-only (typeless) relationships in the path expander relationshipFilter and sequences (#821)

• Added whitelistNodes and blacklistNodes, to finish up node-specific filtering for path expanders. (#796)

Bugfixes

• Also index collection values as arrays so that they are properly added to Lucene. Fixes #814
• fix runFirstColumn for neo4j-graphql-js, fixes #819 fixes #776
• Added thrown runtime exception if an 'apoc.trigger.*' procedure is called but 'apoc.trigger.enabled=true' has not been set in the neo4j.conf file. (#861)

Documentation

• Update Grouping Docs
• Corrected Elasticsearch methods to match their documentation. (#863)

Summer Release 3.3.0.4

This summer release of APOC has seen a lot of contributions from many folks.

• Alberto De Lazzari
• Angelo Busato
• Gabor Szarnyas
• Alex Iudice
• Cayetano
• ArGeBre
• Benjamin
• Tobias Lindaaker
• Adam Cowley
• Karol Brejna
• Andrew Bowman

Thanks so much to everyone who contributed new functionality, documentation updates and fixes.

Here are the highlights:

• Version upgrade Neo4j 3.3.6 APOC 3.3.0.4

• Support reverse geocoding in apoc.spatial (#876)

• Support of Base64url encoding and decoding (#870)

• apoc.text.clean strips utf8 characters (incl. cyrillic/chinese/japanese symbols) (#869)

• Add support for loading CSVs formatted according the import tool's specification (#581)

• add a configuration option to mergeNodes that allows to combine relationships of the same type and direction (#850)

• implement functions from pgsql - degree, label exists (#860)

• Adds text similarity/distance methods and double metaphone text encoder (#865)

• Added support to function apoc.date.fromISO8601 (#837)

• Added unit "weekday" to apoc.date.field() (#838)

• Added optional 4th parameter to procedure 'apoc.periodic.repeat'. (#853)

• Added optional parameter to procedure 'apoc.trigger.add' that allows configuration to be passed in. (#859)

• Add concurrency parameter (default to 50) to periodic.iterate

• add index selectivity to apoc.meta.data apoc.meta.schema and the apoc.index procedures (#842)

• Add apoc.diff user functions (#760)

• Read headers for load csv from config object (#834)

• Allow passing parameters to apoc.cypher.runFile (#813)

• Allow direction-only (typeless) relationships in the path expander relationshipFilter and sequences (#821)

• Added whitelistNodes and blacklistNodes, to finish up node-specific filtering for path expanders. (#796)

Bugfixes

• Also index collection values as arrays so that they are properly added to Lucene. Fixes #814
• fix runFirstColumn for neo4j-graphql-js, fixes #819 fixes #776
• Added thrown runtime exception if an 'apoc.trigger.*' procedure is called but 'apoc.trigger.enabled=true' has not been set in the neo4j.conf file. (#861)

Documentation

• Update Grouping Docs
• Corrected Elasticsearch methods to match their documentation. (#863)

Spring Release 3.4.0.1

The first release for Neo4j 3.4 comes with many changes under the hood but not that many new features.

As Neo4j 3.4 completely changed the SPI (Kernel-API) we had to rewrite large parts in APOC to adapt to these changes. Thanks so much to Stefan Armbruster for handling this large chunk of work.

Features/Improvements

• apoc.load.xls for loading data from Excel files, supports both xls and xlsx
• Improvements for apoc.group.nodes, e.g. filtering of rel-types or of outputs by counts
• Accessor functions for (virtual) entities (e.g. to postfilter them by property or label)
• dijkstra algorithm supporting multiple results
• date.format(null) returns null, also add ISO8601 convenience format

Bugfixes

• fix for apoc.periodic.iterate with statements that already started with WITH
• fix for deleted nodes in explicit index
• apoc.cypher.runTimeboxed uses separate thread
• Missing Iterator Utils in apoc .jar file
• Add missing apoc.coll.combinations()
• check for availability before running sync index update thread

Documentation

• docs for apoc.load.csv and apoc.load.xls
• docs for apoc.group.nodes
• docs for apoc.coll.contains

Spring Release 3.3.0.3

This release comes with not that many new features as we were busy making APOC ready for Neo4j 3.4.

Thanks so much to Stefan Armbruster for handling that large chunk of work.

Features/Improvements

• apoc.load.xls for loading data from Excel files, supports both xls and xlsx
• Improvements for apoc.group.nodes, e.g. filtering of rel-types or of outputs by counts
• Accessor functions for (virtual) entities (e.g. to postfilter them by property or label)
• dijkstra algorithm supporting multiple results
• date.format(null) returns null, also add ISO8601 convenience format

Bugfixes

• fix for apoc.periodic.iterate with statements that already started with WITH
• fix for deleted nodes in explicit index
• apoc.cypher.runTimeboxed uses separate thread
• Missing Iterator Utils in apoc .jar file
• Add missing apoc.coll.combinations()
• check for availability before running sync index update thread

Documentation

• docs for apoc.load.csv and apoc.load.xls
• docs for apoc.group.nodes
• docs for apoc.coll.contains

Winter Release 3.3.0.2

Happy to announce the 3.3.0.2 release of APOC

APOC wouldn't be there where it is today with countless people contributing, reporting ideas and issues and everyone liking it telling their friends. Please do the same.

I also added a code of conduct and contribution guidelines to APOC, so every contributor feels welcome and safe and also quickly knows how to join our efforts.

For this release again, our friends at Larus Italy, did a lot of the work, besides many bugfixes Angelo Busato also added S3 URL support, which is really cool.

Andrea Santurbano worked on the HDFS support (read / write)

Andrew Bowman worked on a number of improvements around Path expanders

• added support for sequences of labels and rel-types to express more complex paths
• support for known end nodes
• support for compound labels

I also found some time to code and added a bunch of things :)

Aggregation functions (something I wanted to add for a long time)

• more efficient variants of collect(x)[a..b]
• apoc.agg.nth, apoc.agg.first, apoc.agg.last, apoc.agg.slice
• apoc.agg.median(x)
• apoc.agg.percentiles(x,[0.5,0.9])
• apoc.agg.product(x)
• apoc.agg.statistics() provides a full numeric statistic

Indexing

Implemented an idea of my colleague Ryan Boyd to allow indexing of full "documents", i.e. map-structures per node or relationship that can also contain information from the neighborhood or computed data. Later those can be searched as keys and values of the indexed data.

• apoc.index.addNodeMap(node, {map})
• apoc.index.addRelationshipMap(node, {map})

As part of that work, I also wanted to add support for deconstructing complex values or structs.
apoc.map.values to select the values of a subset of keys into a mixed type list
apoc.coll.elements is used to deconstruct a sublist into typed variables (this can also be done with WITH, but requires an extra declaration of the list to be concise)

Path Functions (from a request in neo4j.com/slack)

• apoc.path.create(startNode, [rels])
• apoc.path.slice(path, offset, length)
• apoc.path.combine(path1, path2)

Text functions

• apoc.text.code(codepoint), apoc.text.hexCharAt(), apoc.text.charAt() (thanks Andrew Bowman)
• apoc.text.toCypher(value, {}) for generating valid cypher representations of nodes, relationships, paths, values
• Sørensen–Dice similarity (thanks Florent Biville)
• Roman <-> arabic conversions (thanks Marcin Cylke)
• new email and domain extraction functions (thanks David)
• apoc.text.bytes/apoc.text.byteCount

Data Integration

• generic XML import with apoc.import.xml() (thanks Stefan Armbruster)
• pass Cypher parameters to apoc.export.csv.query
• mongodb integration Added paging parameter in the get and find procedure (Thanks Gleb Belokrys)
• stream apoc.export.cypher script export back to client when no file-name is given
• apoc.load.csv
• handling of converted null values/null columns
• explicit "nullValues" option to define values that will be replaced by null (global and per field)
• explicit "results" option to determine which output columns are provided

Collection Functions

• apoc.coll.combinations(), apoc.coll.frequencies() (Thanks Andrew)
• update/remove/insert value at collection index (Thanks Brad Nussbaum)

Graph Refactoring

• per property configurable merge strategy for mergeNodes
• means to skip properties for cloneNodes

Other Additions

• added apoc.date.field UDF

Bugfixes around:

• apoc.load.jdbc (type conversion, connection handling, logging)
• apoc.refactor.mergeNodes
• apoc.cypher.run*
• apoc.schema.properties.distinctCount
• composite indexes in Cypher export
• ElasticSearch integration for ES 6
• Made larger parts of apoc not needing the unrestricted configuration
• apoc.json.toTree (also config for relationship-name casing)
• Warmup Improvements (dynamic properties, rel-group)
• Compound index using apoc.schema.assert (thanks Chris Skardon)
• Explicit Index Reads don't require read-write-user
• Enable parsing of lists in GraphML Import (thanks Alex Wilson)
• Change CYPHER_SHELL format from upper case to lower case. (:begin,:commit)
• Allowed apoc.node.degree() to use untyped directions (thanks Andrew)

Winter Release 3.2.3.6

Happy to announce the 3.2.3.6 release of APOC

APOC wouldn't be there where it is today with countless people contributing, reporting ideas and issues and everyone liking it telling their friends. Please do the same.

I also added a code of conduct and contribution guidelines to APOC, so every contributor feels welcome and safe and also quickly knows how to join our efforts.

For this release again, our friends at Larus Italy, did a lot of the work, besides many bugfixes Angelo Busato also added S3 URL support, which is really cool.

Andrea Santurbano worked on the HDFS support (read / write)

Andrew Bowman worked on a number of improvements around Path expanders

• added support for sequences of labels and rel-types to express more complex paths
• support for known end nodes
• support for compound labels

I also found some time to code and added a bunch of things :)

Aggregation functions (something I wanted to add for a long time)

• more efficient variants of collect(x)[a..b]
• apoc.agg.nth, apoc.agg.first, apoc.agg.last, apoc.agg.slice
• apoc.agg.median(x)
• apoc.agg.percentiles(x,[0.5,0.9])
• apoc.agg.product(x)
• apoc.agg.statistics() provides a full numeric statistic

Indexing

Implemented an idea of my colleague Ryan Boyd to allow indexing of full "documents", i.e. map-structures per node or relationship that can also contain information from the neighborhood or computed data. Later those can be searched as keys and values of the indexed data.

• apoc.index.addNodeMap(node, {map})
• apoc.index.addRelationshipMap(node, {map})

As part of that work, I also wanted to add support for deconstructing complex values or structs.
apoc.map.values to select the values of a subset of keys into a mixed type list
apoc.coll.elements is used to deconstruct a sublist into typed variables (this can also be done with WITH, but requires an extra declaration of the list to be concise)

Path Functions (from a request in neo4j.com/slack)

• apoc.path.create(startNode, [rels])
• apoc.path.slice(path, offset, length)
• apoc.path.combine(path1, path2)

Text functions

• apoc.text.code(codepoint), apoc.text.hexCharAt(), apoc.text.charAt() (thanks Andrew Bowman)
• apoc.text.toCypher(value, {}) for generating valid cypher representations of nodes, relationships, paths, values
• Sørensen–Dice similarity (thanks Florent Biville)
• Roman <-> arabic conversions (thanks Marcin Cylke)
• new email and domain extraction functions (thanks David)
• apoc.text.bytes/apoc.text.byteCount

Data Integration

• generic XML import with apoc.import.xml() (thanks Stefan Armbruster)
• pass Cypher parameters to apoc.export.csv.query
• mongodb integration Added paging parameter in the get and find procedure (Thanks Gleb Belokrys)
• stream apoc.export.cypher script export back to client when no file-name is given
• apoc.load.csv
• handling of converted null values/null columns
• explicit "nullValues" option to define values that will be replaced by null (global and per field)
• explicit "results" option to determine which output columns are provided

Collection Functions

• apoc.coll.combinations(), apoc.coll.frequencies() (Thanks Andrew)
• update/remove/insert value at collection index (Thanks Brad Nussbaum)

Graph Refactoring

• per property configurable merge strategy for mergeNodes
• means to skip properties for cloneNodes

Other Additions

• added apoc.date.field UDF

Bugfixes around:

• apoc.load.jdbc (type conversion, connection handling, logging)
• apoc.refactor.mergeNodes
• apoc.cypher.run*
• apoc.schema.properties.distinctCount
• composite indexes in Cypher export
• ElasticSearch integration for ES 6
• Made larger parts of apoc not needing the unrestricted configuration
• apoc.json.toTree (also config for relationship-name casing)
• Warmup Improvements (dynamic properties, rel-group)
• Compound index using apoc.schema.assert (thanks Chris Skardon)
• Explicit Index Reads don't require read-write-user
• Enable parsing of lists in GraphML Import (thanks Alex Wilson)
• Change CYPHER_SHELL format from upper case to lower case. (:begin,:commit)
• Allowed apoc.node.degree() to use untyped directions (thanks Andrew)

Winter Release 3.1.3.9

This is a maintenance release for Neo4j 3.1.x

It contains the following changes since the last release.

• Make apoc.meta.* to use db-statistics for total node and rel counts
• Fixes/Improvements for apoc.refactor.mergeNodes
• Add more error handling + logging for load jdbc
• Cypher fails to execute because of empty statement
• Close all Kernel statements using try-with-resources
• Added apoc.text.base64Encode, apoc.text.base64Decode functions
• Add apoc.cypher.runFirstColumn, apoc.coll.sortMulti, apoc.coll.flatten functions
• add an failOnError:false option to load.json and friends
• implementing read locks
• apoc.import.file.use_neo4j_config=true does not work as expected
• Allow accessing other databases via bolt
• Added apoc.text.capitalize, apoc.text.capitalizeAll, apoc.text.decapitalize, apoc.text.swapCase, apoc.text.camelCase, apoc.text.snakeCase, apoc.text.toUpperCase functions -
• Added apoc.text.random(length, [valid_chars]) - Fixes
• Fix memory issue with periodic.iterate with too large tx-size
• Added support for multi value JSON sources in apoc.load.json
• Make default thread pool size configurable
• Parallelize Warmup
• Added aggregation to grouping nodes, made the implementation parallel
• Move versions.json generation to master branch
• added apoc.load.ldap
• Added function apoc.convert.toListOf(value, type)
• Added parallel degree distribution apoc.stats.degrees
• Adding apoc.log.* with procedures for logging out messages
• Adding apoc.date.currentTimestamp
• MongoDB compatible values flag for non-packable values
• add apoc.text.split function that splits a string using a regexp (fixes )
• fixes - Add a apoc.version function