Improve precision for Service Level

[arolfes]

We are happy to announce our new major TASKANA release. This release contains crucial changes regarding Classification.serviceLevel, the configuration of TASKANA and its job execution. Since these are significant changes we deviate from the regular release notes in order to provide you with more details and a guideline on how to migrate to the new version.
If anything is unclear or if you need assistance during the upgrade, please reach out to us (e.g. via GitHub discussions).

Overview

• The modules taskana-common-data and taskana-common-test are no longer released. They are internal test modules which should never have been released.
• Overhaul of TaskanaEngineConfiguration
  • its new name is TaskanaConfiguration.
  • it is immutable. Therefore, a builder class will be provided to configure it.
  • the provided builder class now fully supports the java API and configuration via property file (or mixed). Because of that the usage of a properties file is now optional.
  • initialization of the TaskanaEngine has been moved to TaskanaEngine#buildTaskanaEngine(...).
  • Instead of writing error messages TASKANA will now interrupt the startup (via RuntimeException) if any error was detected (e.g. value for specific property does not match).
• Minor rework of Exceptions. Some have been removed, some have been renamed, some have been modified.
• Increased precision of service level to milliseconds
  • The WorkingTimeCalculator class is replaced by an interface. An instance can be retrieved from the TaskanaEngine.
  • To acess the WorkingTimeCalculator in the PriorityServiceProvider we have created a new default method PriorityServiceProvider#initialize(TaskanaEngine)
  • The DaysToWorkingDaysConverter is removed. It's method are now part of the WorkingTimeCalculator.
  • The old algorithm for the WorkingTimeCalculator is replaced by a better, more compatible algorithm. This new algorithm works with completely new global configuration properties regarding the working time. This allows a user-friendly usage of the newly created precision.
    In order to fully embrace this feature, we are currently thinking about expanding the configuration properties to e.g. allow a working time configuration per Workbasket. As of right now this is just an idea, but the WorkingTimeCalculator is prepared to receive its configuration from somewhere else.
• Scheduling of TASKANA jobs is now part of taskana-core. Previously we have provided a JobScheduler-Example which shows how to schedule TASKANA jobs. That class is not necessary anymore. Instead the scheduling of jobs is now part of the TASKANA configuration (with new properties)
• TaskService#selectAndClaim(TaskQuery) now return Optional<Task> instead of Task. Previously an Exception was thrown when the given TaskQuery returned 0 Tasks. That Exception is replaced with the new return type.

Details / Migration Guide

Let's have a look at the details of this release and work through them step-by-step.

Removal of taskana-common-data and taskana-common-test from the release artifacts

These modules will no longer be released. If for some reason you have used them in the past reach out to us; we will find a solution. If not, you have nothing to do here.

Overhaul of TaskanaEngineConfiguration

The TaskanaEngineConfiguration is the entry point to the TASKANA library. Therefore, we should start migrating here.

1. Replace all usages of TaskanaEngineConfiguration with TaskanaConfiguration (its new name).

2. Replace the initalization with the new builder class TaskanaConfiguration.Builder. That builder has 4 required configuration parameters which have to be passed as constructor parameters: DataSource dataSource, boolean useManagedTransactions, String schemaName, boolean securityEnabled. All other parameters are optional (and have a default value). They can be configured through their corresponding builder methods. Please have a look at TaskanaConfiguration.Builder for their default values. Furthermore this builder has some copy constructors if you want to modify some values of an existing TaskanaConfiguration.

3. Here is a table of all changes to the TASKANA properties. If you want to keep using a properties file, please adjust your properties file according to this table. To include the TASKANA properties file you have to call TaskanaConfiguration.Builder#initTaskanaProperties(String propertiesFile, String separator). This will parse all properties within that file and set them accordingly.
   WARNING: Since initTaskanaProperties is a builder function you can override previously set values through this method. Builder methods after this will also override the values of the TASKANA properties. We think this is intuitive, just wanted to give you a heads up.
   INFO: This table already contains new properties and describes their data types. They will be referenced later in the migration guide.
   INFO: Unfortunately this table is wide and you have to scroll vertically. Alternatively you can install wide-github so that you don't have to scroll vertically anymore.

   Change	Old Name	New Name	Comment
   UNCHANGED	taskana.domains	taskana.domains
   RENAMED	taskana.validation.allowTimestampServiceLevelMismatch	taskana.servicelevel.validation.enforce	The new property is inversed.
   UNCHANGED	taskana.roles.user	taskana.roles.user
   RENAMED	taskana.roles.businessadmin	taskana.roles.business_admin
   UNCHANGED	taskana.roles.admin	taskana.roles.admin
   UNCHANGED	taskana.roles.monitor	taskana.roles.monitor
   RENAMED	taskana.roles.taskadmin	taskana.roles.task_admin
   RENAMED	taskana.roles.taskrouter	taskana.roles.task_router
   UNCHANGED	taskana.classification.types	taskana.classification.types
   UNCHANGED	taskana.classification.categories.<TYPE>	taskana.classification.categories.<TYPE>	<TYPE> = value from taskana.classification.types
   NEW		taskana.workingTime.schedule.<DayOfWeek>	<DayOfWeek> = one of: [ MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY, SUNDAY]
   NEW		taskana.workingTime.timezone	Value is parsed via java.time.ZoneId#of(String zoneId)
   RENAMED	taskana.custom.holidays	taskana.workingTime.holidays.custom
   RENAMED	taskana.german.holidays.enabled	taskana.workingTime.holidays.german.enabled
   RENAMED	taskana.german.holidays.corpus-christi.enabled	taskana.workingTime.holidays.german.corpus-christi.enabled
   RENAMED	taskana.history.deletion.on.task.deletion.enabled	taskana.history.simple.deleteOnTaskDeletion.enabled
   RENAMED	taskana.historylogger.name	taskana.history.logger.name
   NEW		taskana.jobs.scheduler.enabled	boolean
   NEW		taskana.jobs.scheduler.initialStartDelay	long
   NEW		taskana.jobs.scheduler.period	long
   NEW		taskana.jobs.scheduler.periodTimeUnit	TimeUnit. one of: [ NANOSECONDS, MICROSECONDS, MILLISECONDS, SECONDS, MINUTES, HOURS, DAYS ]
   UNCHANGED	taskana.jobs.maxRetries	taskana.jobs.maxRetries
   UNCHANGED	taskana.jobs.batchSize	taskana.jobs.batchSize
   RENAMED	taskana.jobs.cleanup.firstRunAt	taskana.jobs.firstRunAt
   RENAMED	taskana.jobs.cleanup.runEvery	taskana.jobs.runEvery
   NEW		taskana.jobs.cleanup.task.enable	boolean
   RENAMED	taskana.jobs.cleanup.minimumAge	taskana.jobs.cleanup.task.minimumAge
   RENAMED/NEW	taskana.jobs.cleanup.allCompletedSameParentBusiness	taskana.jobs.cleanup.task.allCompletedSameParentBusiness	Property is now split up between history and task cleanup
   NEW		taskana.jobs.cleanup.workbasket.enable	boolean
   NEW		taskana.jobs.cleanup.history.simple.enable	boolean
   RENAMED	taskana.jobs.history.batchSize	taskana.jobs.cleanup.history.simple.batchSize
   RENAMED	taskana.jobs.history.cleanup.minimumAge	taskana.jobs.cleanup.history.simple.minimumAge
   RENAMED/NEW	taskana.jobs.cleanup.allCompletedSameParentBusiness	taskana.jobs.cleanup.history.simple.allCompletedSameParentBusiness	Property is now split up between history and task cleanup
   NEW		taskana.jobs.priority.task.enable	boolean
   RENAMED	taskana.jobs.priority.batchSize	taskana.jobs.priority.task.batchSize
   RENAMED	taskana.jobs.priority.firstRunAt	taskana.jobs.priority.task.firstRunAt
   RENAMED	taskana.jobs.priority.runEvery	taskana.jobs.priority.task.runEvery
   DELETED	taskana.jobs.priority.active		The TaskUpdatePriorityJob should not check if it should run. This is the job of the JobScheduler.
   NEW		taskana.jobs.refresh.user.enable	boolean
   UNCHANGED	taskana.jobs.user.refresh.firstRunAt	taskana.jobs.refresh.user.firstRunAt
   UNCHANGED	taskana.jobs.user.refresh.runEvery	taskana.jobs.refresh.user.runEvery
   NEW		taskana.jobs.customJobs	fully-qualified class names
   RENAMED	taskana.addAdditionalUserInfo	taskana.user.addAdditionalUserInfo
   UNCHANGED	taskana.user.minimalPermissionsToAssignDomains	taskana.user.minimalPermissionsToAssignDomains

4. Previously some TASKANA properties ignored the given separator and used , to separate multiple values. Now all TASKANA properties respect the given separator. Please skim through your properties and replace the old hardcoded , separator with your defined separator (default: |).

5. Finally, we can initialize a TaskanaEngine. The old TaskanaEngineConfiguration#buildTaskanaEngine methods were moved to TaskanaEngine#buildTaskanaEngine in order to remove a cyclic dependency between TaskanaConfiguration <-> TaskanaEngine

6. We have introduced a valiation of the TaskanaConfiguration during the TaskanaConfiguration.Builder#build method and during TaskanaEngine#buildTaskanaEngine. If any error occurs during startup TASKANA will throw a RuntimeException telling you what is wrong. Adjust accordingly. Reach out to us if any error message is ambigious.

Overhaul of Exceptions.

In v4.8.0 we have introduced ErrorCodes for each Exception. During that process we already reworked all our Exceptions. Due to compatibility reasons some "legacy" Exception were still part of TASKANA. In this release we have removed them.

1. Please adjust your usages of our Exceptions according to this table (unchanged Exceptions are not listed).

   Change	Old Exception	New Exception	Comment
   REMOVED	InvalidStateException		Split into InvalidTaskStateException and InvalidCallbackStateException
   REMOVED	NotFoundException		Split into [Domain|Classification|TaskanaHistoryEvent|TaskComment|Task|User|Workbasket]NotFoundException
   REMOVED	NotAuthorizedException		Split into NotAuthorizedException, NotAuthorizedOnWorkbasketException and NotAuthorizedOnTaskCommentException
   RENAMED	MismatchedWorkbasketPermissionException	NotAuthorizedOnWorkbasketException
   RENAMED	MismatchedTaskCommentCreatorException	NotAuthorizedOnTaskCommentException
   RENAMED	MismatchedRoleException	NotAuthorizedException

2. The InvalidArgumentException is now a RuntimeException and thus does not need any error handling anymore.

Increased precision of service level to milliseconds

The increased precision of the service level has some impacts on TASKANA's behaviour. We gave our best to stay compatible to the old precision (which was days only). Unfortunately it is not possible to stay 100% compatible. We've tried to adjust everything with the compatibility in mind. Please test this new behaviour intensively. It is possible that we've missed an edge case. We are prepared to release a hotfix as soon as possible.

Changes

• It is now possible to define the Classification.servicelevel using the ISO-8601 duration format without any limitations. Previous limitations (format: PnD) to the service level have been removed.
• Previously the due timestamp was interpreted as inclusive. Example: Due 2023-03-21T07:00:00.000Z would be interpreted as "Task ist due on 21.03.2023 EOB". Because of this Task.due is now inclusive aswell and represents the exact last possible moment, where a Task could be finished in time. One millisecond after that timestamp means that the Task is overdue. In the given example the exact due date would be 2023-03-21T06:59:59.999Z, since that is exactly the last moment a task could be finished without beeing overdue. In order to stay compatible with existing Tasks we allow a delta of 1 millisecond between the computated due and the given due date (e.g. during an update operation). Furthermore, this allowed delta enables a more user-friendly usage with TASKANA. The adjustment from exclusive -> inclusive is done during the processing of a Task and thus is not required by the user.
• In order to keep compatible with the old precision (working with days) the default configuration for TASKANA regarding the working times is:

  taskana.workingTime.schedule.MONDAY=00:00-00:00
  taskana.workingTime.schedule.TUESDAY=00:00-00:00
  taskana.workingTime.schedule.WEDNESDAY=00:00-00:00
  taskana.workingTime.schedule.THURSDAY=00:00-00:00
  taskana.workingTime.schedule.FRIDAY=00:00-00:00
  taskana.workingTime.timezone=Europe/Berlin
  

  This is interpreted as a 24 hour workday from monday to friday in the Europe/Berlin timezone. This default configuration is compatible with the previous precision (except for the exclusive/inclusive due timestamp). Previously only the date of the planned/due timestamp was relevant. Our goal: If you extract the date from those timestamps the result should match to the previous TASKANA versions.
• The normalization of due/planned have changed aswell. Previously, when a Task due/planned date was on a weekend or on a holiday we subtracted (for due) or added (for planned) 24 hours to that date until the given timestamp was a regular workday. This behaviour changed: Now we search for next workslot (for planned) or for the previous workslot (for due). Together with the default configuration this yields different timestamps than previous TASKANA Versions. Example: Due is on a Saturday (outside of working time) 2023-03-25T07:00:00.000Z. Previously TASKANA would have normalized that due date to 2023-03-27T07:00:00.000Z (Monday) by just adding 24 hours until we reach a workday. The new normalization will search for the next working slot. This is Monday at midnight (German timezone): 2023-03-26T22:00:00Z. After applying the timezone (defined in taskana.workingTime.timezone) to both timestamps and stripping the date we get 27.03.2023 as a due date.
• The WorkigTimeCalculator is now an interface. You don't need to manually initialize it anymore. Please use TaskanaEngine#getWorkingTimeCalculator to get an instance of a WorkingTimeCalculator. To stay compatible with our PriorityServiceProvider-Interface we have added PriorityServiceProvider#initialize(TaskanaEngine) so that you have access to an instance of a WorkingTimeCalculator during your Service Provider implementation.

Migration

1. Decide if you want to use the new precision of service level. If you want to keep the old precision, you have do to nothing since the default configuration is backwards compatible. If you want to use the new precision you have to do the following:
   1. configure working times
   • taskana.workingTime.schedule.<DayOfWeek>= [<FROM>-<TO>]. <FROM> and <TO> have to be compatible to LocalTime.parse(String)
   • taskana.workingTime.timezone=<The timezone of the taskana.workingTime.schedule values>
   2. Adjust your service levels to your new working times (and let the ClassificationChangedJob update all your Tasks according to the new service levels).
2. If you have used methods of the WorkingTimeCalculator please migrate accordingly
   1. Don't initialize WorkingTimeCalculator manually anymore. Please use TaskanaEngine#getWorkingtimeCalculator
   2. WorkingTimeCalculator#workingTimeBetweenTwoTimestamps(Instant from, Instant to) -> WorkingTimeCalculator#workingTimeBetween(Instant first, Instant second)
3. The WorkingDaysToDaysConverter has been replaced by the new WorkingTimeCalculator. Please adjust your method calls if you have used the WorkingDaysToDaysConverter
   1. WorkingDaysToDaysConverter#addWorkingDaysToInstant(Instant instant, Duration workingDays) -> WorkingTimeCalculator#addWorkingTime(Instant workStart, Duration workingTime)
   2. WorkingDaysToDaysConverter#subtractWorkingDaysFromInstant(Instant instant, Duration workingDays) -> WorkingTimeCalculator#subtractWorkingTime(Instant workStart, Duration workingTime)
   3. WorkingDaysToDaysConverter#hasWorkingDaysInBetween(Instant left, Instant right) -> WorkingTimeCalculator#isWorkingTimeBetween(Instant first, Instant second)
   4. WorkingDaysToDaysConverter#isWorkingDay(Instant referenceDate) -> WorkingTimeCalculator#isWorkingDay(Instant instant)
   5. WorkingDaysToDaysConverter#isWeekend(LocalDate dateToCheck) -> WorkingTimeCalculator#isWeekend(Instant instant)
   6. WorkingDaysToDaysConverter#isHoliday(LocalDate date) -> WorkingTimeCalculator#isHoliday(Instant instant)
   7. WorkingDaysToDaysConverter#isGermanHoliday(LocalDate date) -> WorkingTimeCalculator#isGermanHoliday(Instant instant)
4. If you have used WorkingTimeCalculator in the PriorityServiceProvider-Interface: Use the new method PriorityServiceProvider#initialize(TaskanaEngine) and TaskanaEngine#getWorkingTimeCalculator to access an instance of a WorkingTimeCalculator.

Scheduling of TASKANA jobs

We have integrated the execution of TASKANA jobs into the taskana-core module. Since some background jobs are crucial for TASKANA's functionality we want to include the responsibility of executing them within TASKANA. This integration creates a nested TaskanaEngine which the jobs use. That allows the job execution to be independent of TASKANA transactions. Therefore, the nested TaskanaEngine uses ConnectionManagementMode.EXPLICIT and the new JobScheduler manages the job transactions. Please don't be irritated whenever you debug through your code. Furthermore, the initalization of the nested TaskanaEngine will cause duplicate log messages during the initialization of TASKANA; don't be irritated about that.

1. If you don't want to be responsible of executing TASKANA jobs you can define taskana.jobs.scheduler.enabled=false. This will disable the job execution within taskana-core. [NOT RECOMMENDED]

2. Enable the JobScheduler by setting this property taskana.jobs.scheduler.enabled=true (or using the corresponding Java API configuration method).

3. Configure the execution cycle of the new JobScheduler. Previously our JobScheduler-Example has used a cron expression for this. This is the equivalent to that.

   • taskana.jobs.scheduler.initialStartDelay=<Initial delay of taskana.jobs.scheduler.periodTimeUnit during startup of TASKANA> [Default: 100]
   • taskana.jobs.scheduler.period=<Period of taskana.jobs.scheduler.periodTimeUnit for repeated execution> [Default: 5]
   • taskana.jobs.scheduler.periodTimeUnit=<TimeUnit for taskana.jobs.scheduler.initialStartDelay and taskana.jobs.scheduler.period> [Default: MINUTES]. (Must be one of [NANOSECONDS, MICROSECONDS, MILLISECONDS, SECONDS, MINUTES, HOURS, DAYS])

4. Configure which jobs you want to have enabled. Since we have reworked all configuration properties we also recommend double checking the job configuration at this point.
   INFO: You can only configure async jobs. On-demand-jobs like the ClassificationChangedJob cannot be disabled because they are crucial for TASKANAs functionality.

   Job	Property	Default value
   TaskCleanupJob	taskana.jobs.cleanup.task.enable	true
   WorkbasketCleanupJob	taskana.jobs.cleanup.workbasket.enable	true
   HistoryCleanupJob	taskana.jobs.cleanup.history.simple.enable	false
   TaskUpdatePriorityJob	taskana.jobs.priority.task.enable	false
   UserInfoRefreshJob	taskana.jobs.refresh.user.enable	false

5. With our JobScheduler-Example you had the option to implement custom jobs and schedule them yourself. We support this usecase. Use taskana.jobs.customJobs=< list of fully-qualified class names, separated by the property separator>. The jobs you configure here will be initialized by our new JobScheduler during startup. If any of the following requirements are not met TASKANA will not start:

   • A class of the given fully-qualified class name exists in the classpath
   • Your custom job is a public class
   • Your custom job is a subclass of AbstractTaskanaJob
   • Your custom job has a constructor with these parameters: TaskanaEngine.class, TaskanaTransactionProvider.class, ScheduledJob.class
   • The needed constructor is public
   • Your job is an async job (= passes the async property to the AbstractTaskanaJob constructor).

6. Finally you can remove your own JobScheduler implementation.

That's it! You should be done with your migration. It is possible that we have missed something and therefore your TASKANA instance is not starting up. If this is the case you can either work through the error messages or reach out to us.

Special thanks to @arolfes and @quandor! They have worked intensively on this release and put a lot of time and effort in making this work. Thank you very much! ❤️

Complete list of features and fixes

• TSK-1972: Configure Working Hours for TASKANA
• TSK-1985: Rework modules which are released
• TSK-1986: Remove cycle between TaskanaEngineConfiguration and TaskanaEngine
• TSK-1987: immutable TaskanaEngineConfiguration
• TSK-1988: Refactor TaskanaEngineConfiguration
• TSK-1989: Integrate execution of or Jobs in taskana-core
• TSK-1991: Rename taskana properties, so that they are not ambiguous anymore
• TSK-1992: Remove unnecessary Exceptions
• TSK-1993: Make InvalidArgumentException a TaskanaRuntimeException
• TSK-1994: Refactor TimeInterval & IntInterval
• TSK-1996: taskana.properties file is optional
• TSK-1997: Refactor methods of DB enum
• TSK-1998: Use DB Enum to determine SQL Scripts
• TSK-2000: Add filter for parentClassificationKeyIn
• TSK-2001: Add priorityWithin and priorityNotWithin as filter criteria
• TSK-2002: Rename TaskanaEngineConfiguration to TaskanaConfiguration
• TSK-2003: Remove reset of connection schema name in SqlConnectionRunner
• TSK-2004: document TaskanaConfiguration.Builder#initTaskanaProperties
• TSK-2005: validate TaskanaConfiguration
• TSK-2007: Failure in PostgreSQL update script
• TSK-2012: Rename Exceptions to make them more understandable
• TSK-2013: selectAndClaimTask returns Optional
• TSK-2016: Replace MapCreator methods with Map#of calls

Full Changelog: v5.10.0...v6.0.0