Explicit cleanup for instances #1090
Conversation
| @@ -28,6 +28,13 @@ def process(self, context): | |||
| self._remove_full_paths(cleanup_full_paths) | |||
| self._remove_empty_dirs(cleanup_empty_dirs) | |||
|
|
|||
| for instance in context: | |||
| instance_cleanup_full_paths = instance.data.get("cleanupFullPaths") | |||
There was a problem hiding this comment.
Wouldn't this directly clean it up - even BEFORE it hits the farm - in the local publish session? (likely less of a problem because it didn't create data yet) but I wonder if that's a potential unintentional side effect of this?
Because now this is in the instance.data now and also in the instance data when the publish job runs on the farm. My gut feeling says that you're trying to say: "when the farm job finished, then remove these files please" - but now it also deletes them now (if they happened to already be there)?
There was a problem hiding this comment.
You are absolutely right. That is indeed the side-effect and it could bite you hard .
And you are correct in my desired effect. I re-read cleanup_farm.py and it does exactly what I need. I thought I had read through it the other day and it did something wildly different then 😁 .
So this is wholly unnecessary. I'll probably make another merge request to add 'gaffer' to the allowed hosts in that plugin. This PR can be deleted.
When we render with Gaffer, to get an exr with the aov's as channels instead of their own separate exr files, we render the aovs separately (since that's what Gaffer does) and then combine them, this requires deletion of these temporary aov folders. The existing
explicit_cleanup.pyplugin operates on contexts only and that's not good enough since then we'd collect e.g. 4 aov folder paths to delete for the various render layers, and then the first publish job would just delete them all regardless of if they are finished enough. So I needed to have some explicit cleanup for instances.So I added this to the
explicit_cleanup.pyplugin. Maybe it would be better to have it's dedicated plugin. Or perhaps the clean-up-farm plugin which I am unsure about how is used.This then propagates the 'cleanupFullPaths' key on the instances on to the farm metadata json files.