Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Procedural Scheduling activity deletion #1610

Open
wants to merge 13 commits into
base: develop
Choose a base branch
from

Conversation

JoelCourtney
Copy link
Contributor

@JoelCourtney JoelCourtney commented Nov 22, 2024

Description

This PR does two main things:

  • allows procedural goals to delete activities
    • If the deleted activity has others anchored to it, they can choose whether to cascade delete them, reanchor them, or throw an error. Default is error
    • Unlike the UI, the ReAnchor option doesn't let you specify "reanchor to plan" vs "reanchor to grandparent". This is because allowing forced reanchoring to the plan is doesn't produce a predictable start time (see below). This unpredictability is more acceptable in the UI because the user will immediately see the results of their action and correct any mistakes. But in a scheduling goal, many activities might be updated unsoundly making it hard to fix the damage.
  • Separates responsibility for satisfying the behavior guarantees of the editable plan interface into a reusable driver called EasyEditablePlanDriver. This way all the adapter has to do is translate the plan representations.

Unpredictability of re-anchoring

The timeline/scheduling procedural libraries try to keep track of an estimated start time for anchored activities, but it is not guaranteed to be right. In practice, I expect the estimates will usually be wrong due to the interop between eDSL and procedural; and even if an estimate is right, it can become wrong in the future. This can definitely be improved, but its a separate task and it will probably never be perfect.

This is usually fine because its an "unofficial" estimate. But if we have the activity chain A <- B <- C, and we delete B and try to reanchor C to the plan, we need to turn the start estimate into an official start time. This way too fallible and unpredictable to do automatically on a large batch of activities.

On the other hand, we could reanchor A <- C in a predictable and reliable way. Its still not necessarily accurate, because if the original state was A <- B (end)<- C then C will be shifted forward by the duration of B, because B is assumed to be instantaneous. So its not perfect, but its at least reproducible and predictable.

Verification

I've added unit tests for the changes to InMemoryEditablePlan, and e2e tests to make sure the changes are reflected in the database.

Documentation

I've added doc comments, and I'll open a PR for the docs website soon.

Future work

It turns out that implementing activity modification is so easy that I went ahead and did it in this PR, hidden in the backend. You'll notice in GraphQLMerlinDatabaseService that it not has routines for uploading creations, deletions, and modifications. The modifications part is used to update anchored activities during a deletion. I think extending support for this to the user will be almost trivial, but does deserve a separate PR.

@JoelCourtney JoelCourtney added the DON'T MERGE Do Not Merge This Branch label Nov 22, 2024
@JoelCourtney JoelCourtney self-assigned this Nov 22, 2024
@JoelCourtney JoelCourtney force-pushed the feat/scheduler-activity-deletion branch from 8265777 to 908f4b9 Compare November 22, 2024 18:22
@JoelCourtney JoelCourtney force-pushed the feat/scheduler-activity-deletion branch from 908f4b9 to 5a03889 Compare November 22, 2024 19:53
@JoelCourtney JoelCourtney force-pushed the feat/scheduler-activity-deletion branch from 5a03889 to 62049ea Compare November 22, 2024 21:07
@JoelCourtney JoelCourtney force-pushed the feat/scheduler-activity-deletion branch from c9abefa to c2e20c2 Compare November 23, 2024 00:05
@JoelCourtney JoelCourtney marked this pull request as ready for review November 23, 2024 00:10
@JoelCourtney JoelCourtney requested a review from a team as a code owner November 23, 2024 00:10
@JoelCourtney JoelCourtney requested review from mattdailis and Mythicaeda and removed request for dandelany and srschaffJPL December 3, 2024 21:44
@dandelany dandelany removed the DON'T MERGE Do Not Merge This Branch label Dec 5, 2024
@JoelCourtney JoelCourtney force-pushed the feat/scheduler-activity-deletion branch from c2e20c2 to 327fdec Compare December 11, 2024 00:35
@JoelCourtney
Copy link
Contributor Author

JoelCourtney commented Jan 15, 2025

Notes from walkthrough:

  • require that modified activities stay the same type
    • for now, require that only startOffset/anchorId can change
  • rename ReAnchor (now PreserveTree)
  • rename EasyEditablePlanDriver and use composition
  • add test for rolling back a deleted anchor

Copy link
Contributor

@Mythicaeda Mythicaeda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly minor comments, as we already discussed in the walkthrough the things that need to be tweaked with this PR (most blocking for me is restricting modifying activity directives)

@JoelCourtney
Copy link
Contributor Author

I was accidentally pushing to the wrong branch for the last two hours 🤦 @Mythicaeda all changes should be done now.

Copy link
Contributor

@Mythicaeda Mythicaeda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Files left unreviewed as of this review: DefaultEditablePlanDriver.kt, Procedure.java, SchedulerPlanEditAdapter.kt, and EditablePlanTest.java

Comment on lines +22 to +23
*

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nonblocking: Extra lines

Comment on lines +679 to +683
final var activityObject = Json
.createObjectBuilder()
.add("start_offset", act.startOffset().toString())
.add("anchored_to_start", act.anchoredToStart())
.add("name", act.name());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ought to be:

      final var activityObject = Json
          .createObjectBuilder()
          .add("start_offset", act.startOffset().toString())
          .add("anchored_to_start", act.anchoredToStart())
          .add("anchor_id", act.anchorId())
          .add("name", act.name());

Otherwise you can't actually save what the activity got reanchored to.

activityObject.add("arguments", insertionObjectArguments.build());
arguments.add("activity_%d".formatted(id), activityObject);
}
postRequest(request.toString(), arguments.build()).orElseThrow(() -> new NoSuchPlanException(planId));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: Missing a request.append("}") before postRequest. See L710 for reference.

Comment on lines +705 to +709
for (final var id : ids) {
request.append("""
delete_%d: delete_activity_directive_by_pk(id: %d, plan_id: %d) {affected_rows}
""".formatted(id.id(), id.id(), planId.id()));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Alternate GQL Proposal:

mutation deletePlanActivityDirectives($planId: Int! = %d, $directiveIds: [Int!]!=%s) {
  delete_activity_directive(where: {_and: {plan_id: {_eq: $planId}, id: {_in: $directiveIds}}}) {
    affected_rows
  }
}

planId has the format 1, directiveIds has the format [1, 2, 3].
This lets you avoid needing to build up the statement per-directive.

A test that will show that only the relevant activities will be deleted is to create a plan with some activites, branch it, then perform a scheudling run that deletes some of the activities. The branch will be unaffected, as will the remaining activities in the plan.

throw new MerlinServiceException("The scheduler should not be updating activity instances");
//updateActivityDirective(planId, schedulerActIntoMerlinAct, activityDirectiveId, activityToGoalId.get(activity));
final var newState = activityDirectiveFromSchedulingDirective.serializedActivity();
final var oldState = actFromInitialPlan.get().serializedActivity();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idle question seeing this and L444: why is actFromInitialPlan an Optional instead of calling .get during assignment?

That is, why isn't L435 final var actFromInitialPlan = initialPlan.getActivityById(activity.id()).get();

@@ -440,8 +442,21 @@ public Map<ActivityDirectiveId, ActivityDirectiveId> updatePlanActivityDirective
activity.anchoredToStart()
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't select it in a comment, but this is a question re: L429-434: Why are we changing the activity's arguments on these lines? Asking as I'm concerned about the following case:

An activity type has a controllable duration, and the parameter that controls it has a default value. There's an instance in the plan that is using this default duration (so its arguments would not contain the parameter). This plan is then scheduled. Wouldn't this activity cause an exception to be raised on L455?

}
}

//Create
ids.putAll(createActivityDirectives(planId, toAdd, activityToGoalId, schedulerModel));
modifyActivityDirectives(planId, toModify);
deleteActivityDirectives(planId, toDelete);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updatePlanActivityDirectiveAnchors ought to be called between modify and delete. Not only would this let the method become private (and have storeFinalPlan actually store the final plan), but if it's not done in this order, then a preserved anchor tree will cause the scheduling run to fail during deleteActivityDirectives, since you can't delete an activity that is an active anchor.

Copy link
Contributor

@Mythicaeda Mythicaeda Jan 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I just checked updatePlanActivityDirectiveAnchors and it only runs for new activities. This means that either

  1. that method needs to be changed to run for modified activities too, including supporting reanchoring existing activities to the plan, (in which case, this comment can be disregarded), or
  2. the comment about including anchorId in the fields being updated in modifyActivityDirectives needs to have its scope increased to support reanchoring to activities that are newly-created in scheduling.

If 2) is chosen, I still think updatePlanActivityDirectiveAnchors ought to be moved into this function

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: None of these tests appear to excersize deleting and reanchoring activities that were already in the plan, only ones that were created as part of scheduling. Please add tests where activities in the plan prior to scheduling are 1) deleted and 2) reachored so that the new GQL code can be tested. I would especially prefer if these tests involved multiple existing activities being reanchored/have their start times adjusted to unique offsets to fully test the modifyActivityDirecitves behavior, since it's chaining sub-mutations.

Comment on lines -83 to -84
} else {
throw new IllegalStateException("Unexpected value: " + edit);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed instead of changing the else to an else if (!(edit instanceof Edit.Delete))? Additionally, is there any value in having a procedure track updated and deleted activities as well as added ones?

return id
}

override fun delete(directive: Directive<AnyDirective>, strategy: DeletedAnchorStrategy) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nonblocking: could you add a couple of inline comments to this function? The outer else branch isn't doing much, but that's hard to see without dissecting the Kotlin-isms.

Also L188 is an extra newline.

Comment on lines +199 to +200
for (d in directives) {
when (val childStart = d.start) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nonblocking: Does Kotlin have the same kind of streamable support Java does? Ie val childDirectives = directives.filter{ d -> d.start is DirectiveStart.Anchor && d.start.parentId == directive.id }. (or even skipping childDirectives and chaining into a forEach)

I don't know if that would be more performant, but it may help collapse L199-L202 from a forloop -> switch -> if statement into what that works out into, which is, "get the activities directly anchored to this one, then loop over them and resolve according to anchor strat"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Delete activities via Procedural Scheduling API
3 participants