ecs-sync 3.5.3

• Added support to migrate S3 Object Lock Retention to ECS S3 buckets.
  • ECS Classic Retention will be converted to S3 Object Lock Retention with configurable default Retention Mode.
  • Object Lock Retention is not supported to convert back to ECS Classic Retention.
• ECS Infinite Retention is now supported to migrate between ECS S3 buckets.
• Minor dependency version updates.

File	Checksum (SHA-512)
ecs-sync-3.5.3.zip	e28b3fa721e5e6d4762e44c9ec92cf3a66e7b5215cc31e1985834a6fa636271c3918a5b8aa0e67067562aac1dc3ab6d4111a054d07641379d009e16abf07d6c7
ecs-sync-ui-3.5.3.jar	916f8e72faeaf07bc6eeb216dfcfbe2b510eeb1af8f1b66eb0fb952960cdf250646c7430958ed85a465d059fb5b3a13c4201fab34478b09427b2bea6fb6a4e38

ecs-sync 3.5.2

• Note: the lifecycle of a storage plugin instance is within a single job
  • After that job completes, the storage plugin is close()d and can no longer be used
  • This was not previously enforced, which allowed calling code to abuse this contract by reusing plugins
  • Now it is strictly enforced - if you try to reuse plugins, they will fail
• Fixed throttle enforcement with S3 MPU
• Fixed bug where DB connections were not closed properly after a job completes
  • Could lead to DB connection leak / exhaustion
• Fixed forceSync option when used with database
• Minor bug fixes
• Minor dependency version updates

File	Checksum (SHA-512)
ecs-sync-3.5.2.zip	fc96bb8ec3696c2639f508cefc8ba7bd9676689805e1f3bbb09aa85317f0dc0495885123d633947866b98d018f3ff83ef03b0902ea080865ed3f8f2da111de20
ecs-sync-ui-3.5.2.jar	fc630128b83247ac3763d436f9eca27ed2da77a8143f91ab30b0ca82cb51bfffdaa4be31060064da56ba9ac8c63caf02b5c229e6fceaaf5c17183173c7fbd7f8

ecs-sync 3.5.1

• Many dependency version updates, including:
  • Gradle 7.4.2
  • Grails 5.2.1
  • Spring 5.3.22
• Build: Dependency versions for all modules are now specified in platform/build.gradle
• Packaging: New option to exclude plugins from a manually built package - only include the plugins you need:
  • -Pfilter.excludes="cas-extractors,cifs-ecs-ingester"
  • -Pstorage.excludes="cas,nfs"
  • works for both the distribution zip and the UI jar (more info in readme.txt)
• Options: New job-level throttles for objects-per-second and bytes-per-second
  • SyncOptions.throughputLimit and SyncOptions.bandwidthLimit
• S3: New option for endpoint TLS cert
  • you can provide the base64-encoded server (or CA) cert directly in the job config without modifying the JVM keystore
• S3: MPU transfers now use a shared thread pool, sized the same as the main sync thread pool, making them more efficient
• S3: New option to "pause" MPUs (leave existing parts) when a sync job is terminated early by a command (mpuResumeEnabled)
  • MPUs will resume if the same batch of objects is copied again
• S3: New option to enable SSE-S3 for all writes to the target bucket
• New PathMappingFilter - maps paths between source and target based on CSV, metadata value, or regex
• New OperationListener for EcsSync instances (when used as a library)
  • register a listener and receive events for all operations executed against a storage system
  • currently only supported in S3 plugins

File	Checksum (SHA-512)
ecs-sync-3.5.1.zip	c731ef2087637810ea1cc776e43516708f91905df1e0d7136aba3a97af73b8684b69defb14c9cee3704ca12ddadb1fcebab6db73a15fca938affe31551810eb2
ecs-sync-ui-3.5.1.jar	54a8a8a8e33ef10b2d97d6463a3a7c6ea80db0e1b0db1a7ee4161fe39c4267e8321fc611e5d2a0d949169a8de2f8c6739afea66586aecb2599e85a2ef8728862

ecs-sync 3.5.0

• Major refactor of code repository
  • The model, core logic, CLI, and All plugins are now separate publishable components
• Updated versions of many (most) dependencies, including:
  • Gradle 6.9.1
  • Grails 4.0.6
  • Log4j 2.17.1
    • Note: previous versions use Log4j 1.2, which is not impacted by CVE-2021-44228
  • Spring 5.3.10
• New useMetadataChecksumForVerification option uses storage system checksum for verification instead of reading back data from the target
  • Currently, only S3 plugins support this (via ETag) - other plugins will fall back to reading back the data
• Added sourceList option to directly insert object lists into job configs (intended for micro-batching)
  • Supports CSV lines so it can be used with multiple plugins
  • Note this is a CDATA element in XML
• Sensitive info (like passwords, secret keys) are now scrubbed in the logs

Checksum values (SHA-512):

• ecs-sync-3.5.0.zip (541e8e4bd50ce0e20ffccd1bc4ac01448b6f1e4c9282091e9dfe6b27d1e4eef4770d603653af95f315d533e232d5a7f7d68599c698c2acf91e1de1d2d2ae8ed8)
• ecs-sync-ui-3.5.0.jar (fbf274b755704d681729d42c3564cb62f7e33bb70a0d56d8e72c3c07b0ceeb79fe95cdf6068ef98cc71bb3ab006f44162c4aaaf6bf3126180a08ee4c41c39775)

ecs-sync 3.4.2

• Added an option for the following additional "extended" fields in the status DB (these are not enabled by default):
  • Source MD5
    • This was removed from the default status tracking table
  • Source Retention End-time
  • Target Modification Time
  • Target MD5
  • Target Retention End-time
  • First Error Seen
    • Only the very first error seen for an object will be recorded and never touched after that
• Corrected syncRetention behavior for ECS S3 to apply remaining retention periods to copied objects
  • Prior versions would reset object-level retention periods to their original duration
  • NOTE: migrating infinite retention is not supported
• Client timeouts are now disabled by default for ECS S3
• Fixed sourceListFileRawValues option so it does not parse as CSV (allowing for commas in raw identifiers)
  • Also fixed a bug where a hash (#) in a quoted value would truncate the entire CSV line

ecs-sync 3.4.1 (DEPRECATED - use 3.4.2 or above)

• Enhancements to support in-line virus scanning (via shell script)

DEPRECATED

• This version has an issue when syncing retention using the ECS S3 plugin - please use v3.4.2 or above

ecs-sync 3.4.0 (DEPRECATED - use 3.4.2 or above)

• New Azure blob storage plugin (read-only)
• STS and alternative credential support in AWS S3 plugin (many thanks to @theronl)
• Raw values option for source list file (allows for leading/trailing spaces in object names)
• Fixed some signature errors in ECS S3 plugin with plus (+) and double-slash (//) object names

DEPRECATED

• This version has an issue when syncing retention using the ECS S3 plugin - please use v3.4.2 or above

ecs-sync 3.3.1 (DEPRECATED - use 3.4.2 or above)

• Fixed errors in UI when job names contain spaces
• Fixed large object (MPU) write failures in AWS S3 plugin
• Added support for syncing object-level retention in ECS S3
• Upgraded gradle to 4.10.3

DEPRECATED

• This version has an issue when syncing retention using the ECS S3 plugin - please use v3.4.2 or above

ecs-sync 3.3.0

• UI:
  • New Storage and Migration features in UI:
    • Atmos to ECS (S3 only) migration w/ S3 proxy orchestration support
    • Please see wiki for details - COMING SOON
  • Config now persists between restarts
  • Changing log level now changes both ecs-sync service and UI log levels
• AWS S3:
  • Added exclude-keys option (source-only) to allow regex-based exclusion patterns during bucket enumeration
  • Added option to disable URL-encoding of keys on list operations in source bucket (some S3 systems do not support it)
  • Fixed bug where some keys in bucket listings were getting double-decoded
  • Fixed bug in verification when migrating versions
• ECS-S3: added handling of invalid content-type values from source object (configurable, enabled by default) - resets to default content type (application/octet-stream)
• FilesystemStorage: added relativeLinkTargets (enabled by default) - translates any absolute links targets under the source directory to be relative (portable)
• File attribute perserve/restore filters: updated to use ClarityNow!'s data mover format for attributes (can also read legacy ecs-sync attributes)
• MetadataFilter: added support for changing UMD names
• Added support for naming sync jobs
  • ecs-sync-ctl submit will set xml filename as job name if none is specified
• First pass at Windows service installation (not quite ready to use yet)

ecs-sync 3.2.9

• Fixed bug in status database that caused all CAS objects to fail
• CLI execution now defaults to --rest-only mode if no arguments provided (--help will still print help text)
  • This makes it easy to run on Windows (in most cases, just double-click the .jar file)