forked from arangodb/arangodb
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathCHANGELOG
14239 lines (9365 loc) · 553 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
devel
-----
* Updated arangosync to 0.7.2.
* Renamed document / index / vertex "handle" to "identifier" / "id" for
consistency in documentation and error messages.
* Agency relational operators TTL fix.
* Improve latency in scheduler.
* Fixed internal issue #4748: Editing a single edgeDefinition using the graph
API failed if it was not shared between all available graphs.
* Changed HTTP return code for an error case in /_api/cluster/endpoints REST API.
Now, if the API is called on a single server, it will return HTTP 501 instead
of HTTP 403.
* Changed HTTP return code for an error case in /_api/cluster/agency-dump REST API.
Now, if the API is called on a server type other than coordinator, it will return
HTTP 501 instead of HTTP 403.
* Fixed undefined behavior on node delete.
* Fixed agency invalid operation
* Bugfix: An AQL ternary expression with the condition being true at query
compile time would not execute its false branch.
* Fixed uptime in _admin/statistics.
* Fixed a bug in smart graph bfs traversals that might violate path uniqueness
requirements in rare cases.
* Fix execution ability in CentOS 6 regarding newer boost.
* Fixed permissions for dump/restore.
* The _users collection is now properly restored when using arangorestore.
* Updated arangosync to 0.7.1.
* rename `minReplicationFactor` into `writeConcern` to make it consistent with
`--cluster.write-concern` and avoid confusion with
`--cluster.min-replication-factor`
* ArangoDB web UI switched build system from grunt to webpack.
It supports React now, but the previous framework is still in use.
* Enable the `parallelize-gather` AQL optimizer rule for certain write queries.
The optimization is turned on by default and can be disabled by setting the
startup option `--query.parallelize-gather-writes` to `false`.
* Shard synchronization readlock aware of rebootId.
* Bugfix: In an AQL cluster query, when gathering unsorted data in combination
with a LIMIT with non-zero offset, if this offset exactly matches the number
of documents in the first shards consumed, the rest of the documents was not
returned.
* Fixed GET _api/gharial to also include the name property in every returned graph.
This is a consistency fix within the API as all other APIs include the name.
As a work around the returned _key can be used, which is identical to the name.
* REMOTE and GATHER no longer make subqueries unsuitable for the
`splice-subqueries` optimization.
* New internal counter and histogram support.
* Add a Prometheus endpoint for metrics, expose new metrics, old statistics
and RocksDB metrics.
* Fixed known issue #509: ArangoSearch index consolidation does not work during creation of a link
on existing collection which may lead to massive file descriptors consumption.
* Added support of array comparison operators to ArangoSearch.
* Added support of arrays to TOKENS function.
* Added support of arrays to PHRASE function.
* Added a new optimization called `late-document-materialization`, `late-document-materialization`
for indexes and views correspondingly.
This optimization reduces amount of documents to read from storage engine to the limit explicitly
stated in LIMIT node.
* Added support of "Edge N-grams" to `text` analyzer.
* Added support of UTF8 to `ngram` analyzer.
* Added ability to mark beginning/end of the sequence to `ngram` analyzer.
* Added a new subquery optimization called `splice-subqueries`.
This optimization splices the execution of subqueries inside the
execution of their surrounding query. For subqueries with a fairly
small number of returned documents this saves a significant amount of
overhead initializing cursors.
This optimization is performed by the optimizer rule
`splice-subqueries`, and currently only works on suitable
subqueries; A subquery becomes unsuitable if it contains a LIMIT,
REMOTE, GATHER or a COLLECT node where the operation is WITH COUNT INTO.
A subquery also becomes unsuitable if it is contained in an
(sub)query which contains unsuitable operations after the subquery.
The implementation of this feature required reworking the dataflow
query execution.
* Fixed issue #10470: The WebUI now shows potential errors and details which occurred using _api/import (e.g.
unique constraint violated).
* Added startup option `--query.optimizer-rules` to selectively enable or disable
optimizer rules by default. The option can be specified multiple times, and takes
the same input as the query option of the same name.
For example, to turn off the rule _use-indexes-for-sort_, use
--query.optimizer-rules "-use-indexes-for-sort"
The purpose of this option is to be able to enable potential future experimental
optimizer rules, which may be shipped in a disabled-by-default state.
* Allow the AQL query optimizer to remove the DistributeNodes for several data-modification
queries. So far, only REMOVE queries benefitted. Now the optimization can also be
applied for REPLACE and UPDATE queries in case the query does not use LIMIT and
there is no further cluster-internal communication after the REMOVE, REPLACE or UPDATE
node.
* Include ArangoSearch data in HotBackups.
* Allow to restore 3.5 HotBackups in 3.6.
* Fixed ArangoSearch index removes being discarded on committing consolidation results with
pending removes after some segments under consolidation were already committed
* Fixed an issue where removeServer left behind current coordinators
* Allow usage of AQL function `RANDOM_TOKEN` with an argument value of `0`. This
now produces an empty string, whereas in older versions this threw an invalid
value exception.
* Add startup option `--rocksdb.exclusive-writes` to avoid write-write conflicts.
This options allows for an easier transition from MMFiles to the RocksDB storage
engine, but comes with a big performance penalty as all collections will be locked
exclusively for writes.
* Added new maxRuntime option for queries. If a query does not finish execution within
the given time (in seconds) it will be killed.
* Fixed undefined behavior with creation of ArangoSearch links with custom
analyzers in cluster environment.
* Fixed internal issue #651: analyzer duplication in _analyzers collection.
* Fixed internal issue #4597: rebalanceShards API cannot work on any database
other than the _system database.
* Fixed UPSERT matching.
Empty objects in the `UPSERT { ... }` expression will not be omitted anymore:
db._collection("col").insert({ "find" : "me" });
db._query(` UPSERT { "find" : "me", "foo" : {} }
UPDATE { "foo" : "not gonna happen" }
INSERT { "find" : "me", "foo" : {} }
INTO col
`)
This will now correctly insert a document instead of updating the existing,
that only partially matches the upsert-expression.
* Fixed issue #10371: K_SHORTEST_PATHS LIMIT 1 can not return the shortest path.
Now the shortest path is returned as the first one in such queries.
* Added limit for AQL range materialization to prevent out-of-memory errors.
When materializing ranges created by either the AQL `RANGE` function or by using
the built-in `..` operator (e.g. `1 .. 1000000`), a check is now performed if
the range is too big to be materialized. The threshold value is set to 10 million
members. Ranges with at most that many members can be materialized, ranges with
more members will fail to materialize and abort the query with the exception
`number out of range` (error code 1504).
It is still possible to create ranges with more than 10 million members as long
as they are not materialized. For example, the following is still valid:
FOR i IN 1 .. 1000000000 INSERT {_key: CONCAT('test', i)} INTO collection
* Separately account for superuser and user request traffic. This is
needed for Oasis.
* No longer put system services into `_apps` on single server. On cluster, this
has never worked. This was unnecessary.
* Fixed available flag for HotBackup.
* Fixed list with id for partially available HotBackups.
* Added AQL optimizer rule "move-filters-into-enumerate", to allow for
early pruning of non-matching documents while full-scanning or index-
scanning documents. This optimization can help to avoid a lot of
temporary document copies.
* Added "SmartJoins for Views" to the ArangoDB Enterprise Edition that allows running
cluster joins between two certain sharded collections or views with performance close
to that of a local join operation.
* Allow collection names to be at most 256 characters long, instead of 64 characters
in previous versions.
* Upgraded bundled Boost library version to 1.71.
* The General Graph document API is now consistent with the document API in its error messages.
When attempting to create / modify edges pointing to non existing vertex collections
HTTP 400 is returned instead of 404.
* Use `-std=c++17` for ArangoDB compilation.
* Made the mechanism in the Web UI of replacing and upgrading a foxx app more clear.
* Fixed search not working in document view while in code mode.
* Show shards of all collections (including system collections) in the web UI's shard
distribution view.
This is necessary to access the prototype collections of a collection sharded via
`distributeShardsLike` in case the prototype is a system collection, and the prototype
should be moved to another server.
* Rclone URL normalization.
* Fixed unintended multiple unlock commands from coordinator to
transaction locked db servers.
* Disallow using `_id` or `_rev` as shard keys in clustered collections.
Using these attributes for sharding was not supported before, but didn't trigger
any errors. Instead, collections were created and silently using `_key` as
the shard key, without making the caller aware of that an unsupported shard
key was used.
* DB server locking / unlocking for hot backup revisited and enhanced.
* Use execvp instead of execv in HotBackup restore.
* Re-enabled the AQL sort-limit optimization rule in conjunction with fullCount
in the cluster. It now also may speed up fullCount with sorted indexes and a
limit.
* Fix config directory handling, so we don't trap into UNC path lookups on Windows.
* Prevent spurious log message "Scheduler queue is filled more than 50% in last x s"
from occurring when this is not the case. Due to a data race, the message could
previously also occur if the queue was empty.
* Make the scheduler enforce the configured queue lengths. The values of the options
`--server.scheduler-queue-size`, `--server.prio1-size` and `--server.maximal-queue-size`
will now be honored and not exceeded.
The default queue sizes in the scheduler for requests buffering have
also been changed as follows:
request type before now
-----------------------------------
high priority 128 4096
medium priority 1048576 4096
low priority 4096 4096
The queue sizes can still be adjusted at server start using the above-
mentioned startup options.
* Fix compilation issue with clang 10.
* Fixed issue #10062: AQL: Could not extract custom attribute.
* Add replicationFactor, minReplicationFactor and sharding strategy to database creation
dialog in web UI. Preselect database default values for collection creation in web UI.
* Add new JavaScript function `db._properties()` that provides information about
the current database's properties.
* Add new options `sharding` and `replicationFactor` for database creation methods. The
specified values will provide the defaults for all collections created in a database.
Valid values for `sharding` are `""`, "flexible", "single". The first 2 values are
treated equally. Values for `replicationFactor` are natural numbers or the string
`satellite`.
* Add new server option `--cluster.default-replication-factor` that allows to set the
default replication factor for non-system collections (default: 1).
* Made the mechanism in the Web UI of replacing and upgrading a foxx app more clear.
* Fix a problem with AQL constrained sort in the cluster, which might abort
queries.
* Fix strange shutdown hanger which came from the fact that currently
libgcc/libmusl wrongly detect multi-threadedness in statically linked
executables.
* Enabled IPO with cmake as an option, default is on for release builds without
google tests.
* Bugfix: The AQL sort-limit optimization was applied in some cases it shouldn't,
resulting in undefined behavior.
* Remove operations for documents in the cluster will now use an optimization,
if all sharding keys are specified. Should the sharding keys not match the values in
the actual document, a not found error will be returned.
* Fixed AQL constrained-heap sort in conjunction with fullCount.
* Fixed "ArangoDB is not running in cluster mode" errors in active failover setups.
This affected at least /_admin/cluster/health.
* Fixed the removal (including a collection drop) of an orphanCollection from a
graph definition when using the ArangoShell. The boolean
flag whether to drop the collection or not was not transferred properly.
* Retry hot backup list in cluster for 2 minutes before reporting error.
* Improved database creation within the cluster. In the case of coordinator outages
during the creation of the database there was a chance that not all relevant
system collections had been created. Although this database was accessible now
some features did not work as expected (e.g. creation of graphs). We modified
creation of a new database as an all or nothing operation and only allow access
to the database after all system collections are properly prepared to avoid the
above inconsistencies. Also creation of databases are now secured against
coordinator outages, they will either be fully created or not visible and
eventually dropped. This does not require any change on the client code.
* Added UI support to create documents in a collection using smartGraphAttribute
and/or smartJoinAttribute.
* Allowing inconsistent rather than forcing hot backups.
* Fixed adding an orphan collections as the first collection in a SmartGraph.
* Fixed issue #9862: ServerException: RestHandler/RestCursorHandler.cpp:279.
This fixes an issue with the RocksDB primary index IN iterator not resetting its
internal iterator after being rearmed with new lookup values (which only happens
if the IN iterator is called from an inner FOR loop).
* Geo functions will now have better error reporting on invalid input.
* Fixed issue #9795. Fixed NOT IN clause in ArangoSearch.
* The graph viewer of the web interface now tries to find a vertex document of
all available vertex collections before it aborts.
* Upgraded bundled jemalloc library to version 5.2.1.
* Fixed internal issue #4407: remove storage engine warning.
* Added support for TLS 1.3 for the arangod server and the client tools.
The arangod server can be started with option `--ssl.protocol 6` to make it require
TLS 1.3 for incoming client connections. The server can be started with option
`--ssl.protocol 5` to make it require TLS 1.2, as in previous versions of arangod.
The default TLS protocol for the arangod server is now generic TLS, which will allow
the negotiation of the TLS version between the client and the server.
All client tools also support TLS 1.3, by using the `--ssl.protocol 6` option when
invoking them. The client tools will use TLS 1.2 by default, in order to be
compatible with older versions of ArangoDB that may be contacted by these tools.
To configure the TLS version for arangod instances started by the ArangoDB starter,
one can use the `--all.ssl.protocol=VALUE` startup option for the ArangoDB starter,
where VALUE is one of the following:
- 4 = TLSv1
- 5 = TLSv1.2
- 6 = TLSv1.3
- 9 = generic TLS
* Added TransactionStatistics to ServerStatistics (transactions started /
aborted / committed and number of intermediate commits).
* Agents to remove callback entries when responded to with code 404.
* Added AQL function DATE_ROUND to bin a date/time into a set of equal-distance
buckets.
* Enforced the valid date range for working with date/time in AQL. The valid date
ranges for any AQL date/time function are:
- for string date/time values: `"0000-01-01T00:00:00.000Z"` (including) up to
`"9999-12-31T23:59:59.999Z"` (including)
- for numeric date/time values: -62167219200000 (including) up to 253402300799999
(including). These values are the numeric equivalents of
`"0000-01-01T00:00:00.000Z"` and `"9999-12-31T23:59:59.999Z"`.
Any date/time values outside the given range that are passed into an AQL date
function will make the function return `null` and trigger a warning in the query,
which can optionally be escalated to an error and stop the query.
Any date/time operations that produce date/time outside the valid ranges stated
above will make the function return `null` and trigger a warning too. An example
for this is
DATE_SUBTRACT("2018-08-22T10:49:00+02:00", 100000, "years")
* Fixed bug in MoveShard::abort which causes a duplicate entry in the follower list. (Internal Bug #4378)
* Fixed cut'n'pasting code from the documentation into arangosh.
* Added initial support for wgs84 reference ellipsoid in GEO_DISTANCE through third
optional parameter to AQL function.
* Added support for area calculations with GEO_AREA AQL function.
* Added resign leadership job to supervision.
* Keep followers in sync if the old leader resigned and stopped writes.
* Abort a FailedLeader job when its _to_ server fails.
* Removed content from Documentation/Books, but keeping the subfolders.
The documentation is in a separate repository (except DocuBlocks and Scripts):
https://github.com/arangodb/docs.git
* Updated TOKENS function to deal with primitive types and arrays.
* Fixed agency nodes to not create bogus keys on delete / observe / unobserve.
* Fixed an agency bug found in Windows tests.
v3.5.0-rc.7 (2019-08-01)
------------------------
* Upgraded arangodb starter version to 0.14.12.
v3.5.0-rc.6 (2019-07-29)
------------------------
* Fixed issue #9459: Optimization rule remove-collect-variables does not KEEP all necessary data.
* Added gzip and encryption options to arangoimport and arangoexport.
* Added missing REST API route GET /_api/transaction for retrieving the list of
currently ongoing transactions.
* Fixed issue #9558: RTRIM not working as expected.
* Added startup error for bad temporary directory setting.
If the temporary directory (--temp.path) setting is identical to the database
directory (`--database.directory`) this can eventually lead to data loss, as
temporary files may be created inside the temporary directory, causing overwrites of
existing database files/directories with the same names.
Additionally the temporary directory may be cleaned at some point, and this would lead
to an unintended cleanup of the database files/directories as well.
Now, if the database directory and temporary directory are set to the same path, there
will be a startup warning about potential data loss (though in ArangoDB 3.4 allowing to
continue the startup - in 3.5 and higher we will abort the startup).
* Make TTL indexes behave like other indexes on creation.
If a TTL index is already present on a collection, the previous behavior
was to make subsequent calls to `ensureIndex` fail unconditionally with
the error "there can only be one ttl index per collection".
Now we are comparing the attributes of the to-be-created index with the
attributes of the existing TTL index and make it only fail when the
attributes differ. If the attributes are identical, the `ensureIndex`
call succeeds and returns the existing index.
v3.5.0-rc.5 (2019-07-22)
------------------------
* MinReplicationFactor:
Collections can now be created with a minimal replication factor (minReplicationFactor), which defaults to 1.
If minReplicationFactor > 1 a collection will go into "read-only" mode as soon as it has less then minReplicationFactor
many insync followers. With this mechanism users can avoid to have collections diverge too much in case of failure scenarios.
minReplicationFactor can have the values: 1 <= minReplicationFactor <= replicationFactor.
Having minReplicationFactor == 1 ArangoDB behaves the same way as in any previous version.
* Fixed a query abort error with smart joins if both collections were restricted to a
single shard using the "restrict-to-single-shard" optimizer rule.
* Fixed a performance regression of COLLECT WITH COUNT INTO.
* Fixed some races in cluster collection creation, which allowed collections with the
same name to be created in parallel under some rare conditions.
* arangoimport would not stop, much less report, communications errors. Add CSV reporting
of line numbers that are impacted during such errors
* Prevent rare cases of duplicate DDL actions being executed by Maintenance.
* Coordinator code was reporting rocksdb error codes, but not the associated detail message.
Corrected.
* The keep alive timeout specified via --http.keep-alive-timeout is now being honored
* Replication requests on Document API are now on higher priority then client-triggered requests.
This should help to keep in sync replication up and running even if the server is overloaded.
* Bugfix: Save distinct WAL ticks for multiple replication clients from the same
server. Also, when a follower is added for synchronous replication, the WAL
tick held by the client is freed immediately, rather than waiting for a
timeout.
The corresponding APIs get a new parameter `syncerId`, which, if given,
supersedes `serverId`. This affects the routes /_api/wal/tail,
/_api/replication/batch, /_api/replication/logger-follow and the internal
route /_api/replication/addFollower. The new field `syncerId` is also added to
the response of /_api/replication/logger-state.
* Disallow indexing the `_id` attribute even as a sub-attribute.
Previous versions of ArangoDB did not allow creating indexes on the `_id` attribute, but didn't
check if an index was created on the `_id` attribute of a sub-attribute, e.g. `referredTo._id`
or `data[*]._id`.
Such indexes could be created with previous versions of ArangoDB, but they were non-functional.
From now on, such indexes cannot be created anymore, and any attempts to create them will fail.
* Added option `--replication.max-parallel-tailing-invocations` to limit the maximum number
of concurrent WAL tailing invocations.
The option can be used to limit the usage of the WAL tailing APIs in order to control
server load.
* Fixed agency bug with election lock step.
* Fixed some error reporting and logging in Maintenance.
* Fixed an error condition in which an ex-leader for a short still believed
to be the leader and wrongly reported to Current.
v3.5.0-rc.4 (2019-06-15)
------------------------
* Speed up collection creation process in cluster, if not all agency callbacks are
delivered successfully.
* Increased performance of document inserts, by reducing the number of checks in unique / primary indexes.
* Fixed a callback function in the web UI where the variable `this` was out of scope.
* Fixed editing a user within the web UI if the user added a gravatar profile picture.
* Allow pregel to select the shard key via `shardKeyAttribute` in pregel start parameters.
* Added --server.jwt-secret-keyfile to arangosh.
* Fixed internal issue #4040: gharial api is now checking existence of `_from` and `_to` vertices
during edge replacements and edge updates.
* Fixed `Buffer.alloc` method.
* `Buffer` is now iterable and accepts `ArrayBuffer` values as input.
* Fix timeout-response in case of simultaneous index create/drop in cluster.
* Enabled dropping an index by its name.
* Fixed lookup of index from collection by fully qualified name, e.g.
`db.testCollection.index('testCollection/primary')`.
* Fixed agency bug with TTL object writes discovered in 3.4.6.
v3.5.0-rc.3 (2019-05-31)
------------------------
* Fixed issue #9106: Sparse Skiplist Index on multiple fields not used for FILTER + SORT query.
Allow AQL query optimizer to use sparse indexes in more cases, specifically when
indexes could not be used for filtering and there finally was an `EnumerateCollectionNode`
in the query execution plan followed by a `SortNode`. In this case, sparse indexes were
not considered for enumeration in sorted order, because it was unclear to the optimizer
if the result set would contain null values or not.
* Upgraded RocksDB to version 6.2.
* Updated ArangoDB Starter to 0.14.4.
* The system collection '_jobs' will from now on be created with non-unique, non-sparse indexes.
* Bugfix for smart graph traversals with uniqueVertices: path, which could
sometimes lead to erroneous traversal results.
* Pregel algorithms can be run with the option "useMemoryMaps: true" to be
able to run algorithms on data that is bigger than the available RAM.
* Fixed a race in TTL thread deactivation/shutdown.
* Fixed internal issue #3919: The web UI is now using precompiled ejs templates.
* Fixed agency issue in abort of cleanOutServer job.
v3.5.0-rc.2 (2019-05-23)
------------------------
* Fixed "collection not found" exception during setup of 3-way smart join queries in the
cluster.
* Fixed an edge case of handling `null` values in the AQL function `MIN` for input
sequences that started with a `null` value. In this case, `null` was always returned as the
minimum value even though other non-null values may have followed, and `MIN` was supposed
to return `null` only if there are no input values or all input values are `null`.
* Fixed a crash when posting an async request to the server using the "x-arango-async"
request header and the server's request queue was full.
* Added error code 1240 "incomplete read" for RocksDB-based reads which cannot retrieve
documents due to the RocksDB block cache being size-restricted (with size limit enforced)
and uncompressed data blocks not fitting into the block cache.
The error can only occur for collection or index scans with the RocksDB storage engine
when the RocksDB block cache is used and set to a very small size, plus its maximum size is
enforced by setting the `--rocksdb.enforce-block-cache-size-limit` option to `true`.
Previously these incomplete reads could have been ignored silently, making collection or
index scans return less documents than there were actually present.
* Fixed internal issue #3918: added optional second parameter "withId" to AQL
function PREGEL_RESULT.
This parameter defaults to `false`. When set to `true` the results of the Pregel
computation run will also contain the `_id` attribute for each vertex and not
just `_key`. This allows distinguishing vertices from different vertex collections.
* Made Swagger UI work with HTTPS-enabled ArangoDBs too.
* Improved scheduler performance for single-connection cases.
* Internally switched unit tests framework from catch to gtest.
* Disabled selection of index types "hash" and "skiplist" in the web interface when
using the RocksDB engine. The index types "hash", "skiplist" and "persistent" are
just aliases of each other with the RocksDB engine, so there is no need to offer all
of them. After initially only offering "hash" indexes, we decided to only offer
indexes of type "persistent", as it is technically the most
appropriate description.
* Fixed client id lookup table in state.
v3.5.0-rc.1 (2019-05-14)
------------------------
* Removed bug during start up with a single agent, that leads to dbserver crash.
* Fixed issue #7011: description when replacing a foxx application was misleading.
* Fixed issue #8841: Graph Viewer dropped ability to edit an edge after
rerendering.
* Upgraded arangodb starter version to 0.14.3.
* ArangoQueryStreamCursor.prototype.id needs to be a string, v8 32 bit integers
can't hold the full data.
* Upgraded Swagger UI to 3.22.1.
* Added --compress-output flag to arangodump. Activates gzip compression for
collection data. Metadata files, such as .structure.json and .view.json,
do not get compressed. No option is needed for arangorestore to restore
.data.json.gz files.
* Added options to make server more secure:
- `--server.harden`: denies access to certain REST APIs that return server internals
- `--foxx.api`: set to false disables Foxx management API
- `--foxx.store`: set to false disables Foxx UI
- `--javascript.allow-port-testing`: enables internal.testPort()
- `--javascript.allow-external-process-control`: enables external process control
- `--javascript.harden`: disables getPid() and logLevel()
- `--javascript.startup-options-whitelist`: control startup options visible in JavaScript
- `--javascript.environment-variables-whitelist`: control environment variables visible in JavaScript
- `--javascript.endpoints-whitelist`: control accessible endpoints in JavaScript
- `--javascript.files-whitelist`: control file access in JavaScript
Note: There is a [detailed description of all options](https://www.arangodb.com/docs/devel/security-security-options.html).
* Prevented arangod from making a call to www.arangodb.com at startup.
This call was done to check for available updates, but it could have contributed
to small startup delays in case outgoing connections were blocked.
* Removed support for undocumented HTTP header `x-arango-v8-context`, which
allowed controlling in which particular V8 context number a JavaScript-based
action was executed. This header was only used internally for testing.
* `db._query` now handles additional arguments correctly when passing an AQL
query object instead of a query string and separate bindVars.
* Added req.auth property to Foxx.
* Added collection.documentId method to derive document id from key.
* Indexes created with the 'inBackground', will not hold an
exclusive collection lock for the entire index creation period (rocksdb only).
* Fixed internal issue #536: ArangoSearch may crash server during term lookup.
* Fixed internal issue #2946: Create graph autocomplete was not working under
certain circumstances.
* Added `filter` option to Foxx HTTP API for running tests.
* Added function `db.<collection>.getResponsibleShard()` to find out which is the
responsible shard for a given document. Note that this function is only available
in a cluster coordinator.
* Updated bundled version of jemalloc memory allocator to 5.2.0.
* Don't create per-database system collection `_frontend` automatically.
This collection is only needed by the web UI, and it can be created lazily
when needed.
* Added logging option `--log.time-format` to configure the time format used
in log output. The possible values for this option are:
- timestamp: unix timestamps, in seconds
- timestamp-millis: unix timestamps, in seconds, with millisecond precision
- timestamp-micros: unix timestamps, in seconds, with microsecond precision
- uptime: seconds since server start
- uptime-millis: seconds since server start, with millisecond precision
- uptime-micros: seconds since server start, with microsecond precision
- utc-datestring: UTC-based date and time in format YYYY-MM-DDTHH:MM:SSZ
- utc-datestring-millis: UTC-based date and time in format YYYY-MM-DDTHH:MM:SS.FFFZ
- local-datestring: local date and time in format YYYY-MM-DDTHH:MM:SS
This change deprecates the existing options `--log.use-microtime` and
`--log.use-localtime`, because the functionality provided by these options
is covered by `--log.time-format` too.
* Added "smart joins" to the ArangoDB Enterprise Edition that allows running cluster
joins between two certain sharded collections with performance close to that of a
local join operation.
* Fixed internal issue #3815: fixed the removal of connected edges when
removing a vertex graph node in a smart graph environment.
* Show startup warning in case kernel setting `vm.overcommit_memory` is set
to a value of 2 and the jemalloc memory allocator is in use. This combination
does not play well together.
* Added AQL function DECODE_REV for decomposing `_rev` values into their
individual parts.
* Added AQL functions CRC32 and FNV64 for hashing data.
* Renamed attribute key `openssl-version` in server/client tool version
details output to `openssl-version-compile-time`.
This change affects the output produced when starting one of the ArangoDB
executables with the `--version` command. It also changes the attribute
name in the detailed response of the `/_api/version` REST API.
* Fixed the sorting of the databases in the database
selection dropdown in the web ui. The sort order differed based on whether
authentication was enabled or disabled. (Internal issue #2276)
* Improved the shards view in the web ui if there
is only one shard to display. (Internal issue #3546)
* Restricted the allowed query names for user
defined custom queries within the web ui. (Internal issue #3789)
* Upgraded bundled RocksDB version to 6.0.
* Added "--log.ids" option to arangod.
The default value for this option is `true`. Setting the option to `false`
will prevent embedding unique log ids into all log messages generated by
ArangoDB C++ code. The unique ids allow for easy access to the location in
the source code from which a message originates. This should help customers
to configure custom monitoring/alerting based on specific log id occurrences
and will also be helpful for support and development in identifying problems.
* Fixed wrong equals behavior on arrays with ArangoSearch. (Internal issue #8294)
* Fixed ArangoSearch range query sometimes not working
correctly with numeric values. (Internal issue #528)
* Changed type of the startup option `--rocksdb.recycle-log-file-num` from
numeric to boolean, as this is also the type the options has in the RocksDB
library.
* Renamed hidden startup option `--rocksdb.delayed_write_rate` to the more
consistent `--rocksdb.delayed-write-rate`. When the old option name is
used, the arangod startup will be aborted with a descriptive error message.
* If not explicitly configured, make agency nodes start removing their unused
WAL files a few seconds after the completed server startup already. This is
because on agency nodes, unused WAL files do not need to be retained for
potential replication clients to read them.
* Added option `--all-databases` to arangodump and arangorestore.
When set to true, this makes arangodump dump all available databases
the current user has access to. The option `--all-databases` cannot be
used in combination with the option `--server.database`.
When `--all-databases` is used, arangodump will create a subdirectory
with the data of each dumped database. Databases will be dumped one
after the other. However, inside each database, the collections of the
database can be dumped in parallel using multiple threads.
For arangorestore, this makes it restore all databases from inside the
subdirectories of the specified dump directory. Using the option for
arangorestore only makes sense for dumps created with arangodump and
the `--all-databases` option. As for arangodump, arangorestore cannot
be invoked with the options `--all-databases` and `--server.database`
at the same time. Additionally, the option `--force-same-database` cannot
be used together with `--all-databases`.
If the to-be-restored databases do not exist on the target server, then
restoring data into them will fail unless the option `--create-database`
is also specified. Please note that in this case a database user must
be used that has access to the `_system` database, in order to create
the databases on restore.
* Added index hints feature to AQL.
* Added "name" property for indices.
If a name is not specified on index creation, one will be auto-generated.
* Under normal circumstances there should be no need to connect to a
database server in a cluster with one of the client tools, and it is
likely that any user operations carried out there with one of the client
tools may cause trouble.
The client tools arangosh, arangodump and arangorestore will now emit
a warning when connecting with them to a database server node in a cluster.
* Fixed compaction behavior of followers.
* Added "random" masking to mask any data type, added wildcard masking.
* Added option `--console.history` to arangosh for controlling whether
the command-line history should be loaded from and persisted in a file.
The default value for this option is `true`. Setting it to `false`
will make arangosh not load any command-line history from the history
file, and not store the current session's history when the shell is
exited. The command-line history will then only be available in the
current shell session.
* Display the server role when connecting arangosh against a server (e.g.
SINGLE, COORDINATOR).
* Fixed overflow in Windows NowNanos in RocksDB.
* Allowed MoveShard from leader to a follower, thus swapping the two.
* Supervision fix: Satellite collections, various fixes.
* Added coordinator route for agency dump.
* Supervision fix: abort MoveShard job does not leave a lock behind.
* Supervision fix: abort MoveShard (leader) job moves forwards when point
of no return has been reached.
* Supervision fix: abort CleanOutServer job does not leave server in
ToBeCleanedServers.
* Supervision fix: move shard with data stopped to early due to wrong usage
of compare function.
* Supervision fix: AddFollower only counts good followers, fixing a
situation after a FailedLeader job could not find a new working
follower.
* Supervision fix: FailedLeader now also considers temporarily BAD
servers as replacement followers and does not block servers which
currently receive a new shard.
* Supervision fix: Servers in ToBeCleanedServers are no longer considered
as replacement servers.
* Maintenance fix: added precondition of unchanged Plan in phase2.
* Added "PRUNE <condition>" to AQL Traversals. This allows to early abort searching of
unnecessary branches within a traversal.
PRUNE is only allowed in the Traversal statement and only between the graph
definition and the options of the traversal.
e.g.:
`FOR v, e, p IN 1..3 OUTBOUND @source GRAPH "myGraph"
PRUNE v.value == "bar"
OPTIONS {} /* These options remain optional */
RETURN v`
for more details refer to the documentation chapter.
* Fixed a display issue when editing a graph within the web UI.
* Fixed some escaping issues within the web UI.
* Follow up to fix JWT authentication in arangosh (Internal issue #7530):
also fixed reconnect.
* Now also syncing _jobs and _queues collections in active failover mode.
* Upgraded lodash to 4.17.11 because of CVE-2018-16487.
* `--query.registry-ttl` is now honored in single-server mode, and cursor TTLs
are now honored on DBServers in cluster mode.
* Added "TTL" index type, for optional auto-expiration of documents.
* Disabled selection of index types "persistent" and "skiplist" in the web
interface when using the RocksDB engine. The index types "hash", "skiplist"
and "persistent" are just aliases of each other with the RocksDB engine,
so there is no need to offer all of them.
* Fixed JS AQL query objects with empty query strings not being recognized
as AQL queries.
* Updated JavaScript dependencies, including semver major updates to joi, mocha
and eslint. The eslint config modules were incompatible with the existing
coding style, so the old rules were inlined and the config dependencies removed.
Note that joi removed Joi.date().format() in v10.0.0. You can bundle your own
version of joi if you need to rely on version-dependent features.
- accepts: 1.3.4 -> 1.3.5
- ansi_up: 2.0.2 -> 4.0.3
- content-disposition: 0.5.2 -> 0.5.3
- dedent: 0.6.0 -> 0.7.0
- error-stack-parser: 1.3.6 -> 2.0.2
- eslint: 2.13.1 -> 5.16.0
- eslint-config-semistandard: 6.0.2 -> removed
- eslint-config-standard: 5.3.1 -> removed
- eslint-plugin-promise: 1.3.2 -> removed
- eslint-plugin-standard: 1.3.2 -> removed
- highlight.js: 9.12.0 -> 9.15.6
- http-errors: 1.6.2 -> 1.7.2
- iconv-lite: 0.4.19 -> 0.4.24
- joi: 9.2.0 -> 14.3.1
- joi-to-json-schema: 2.3.0 -> 4.0.1
- js-yaml: 3.10.0 -> 3.13.1
- marked: 0.3.9 -> 0.6.2
- mime-types: 2.1.12 -> 2.1.22
- mocha: 2.5.3 -> 6.1.3
- qs: 6.5.1 -> 6.7.0
- semver: 5.4.1 -> 6.0.0
- statuses: 1.4.0 -> 1.5.0
- timezone: 1.0.13 -> 1.0.22
- type-is: 1.6.15 -> 1.6.16
- underscore: 1.8.3 -> 1.9.1
* Updated V8 to 7.1.302.28.
New V8 behavior introduced herein:
- ES2016 changed the default timezone of date strings to be conditional on
whether a time part is included. The semantics were a compromise approach
based on web compatibility feedback from V8, but until now, we have been
shipping ES5.1 default timezone semantics. This patch implements the
new semantics, following ChakraCore and SpiderMonkey (though JSC
implements V8's previous semantics).
* Fixed JS AQL query objects with empty query strings not being recognized as AQL queries.
* Report run-time openssl version (for dynamically linked executables).
* Added greeting warning about maintainer mode.
* Improved insertion time into non-unique secondary indexes with the RocksDB
engine.
* Fixed possible segfault when using COLLECT with a LIMIT and an offset.
* Fixed COLLECT forgetting top-level variables after 1000 rows.
* Added sort-limit optimization in AQL; Improved memory usage and execution
time for some queries.
* Upgraded to OpenSSL 1.1.0j.
* Added configurable masking of dumped data via `arangodump` tool to obfuscate
exported sensible data.
* Fixed arangoimp script for MacOSX CLI Bundle.
* Added "peakMemoryUsage" in query results figures, showing the peak memory
usage of the executed query. In a cluster, the value the peak memory usage
of all shards, but it is not summed up across shards.
* Fixed an issue where a crashed coordinator can lead to some Foxx queue jobs
erroneously either left hanging or being restarted.
* Fixed bind values of `null` are not replaced by
empty string anymore, when toggling between json and table
view in the web-ui. (Interal issue #7900)