Thursday, 2024/01/18
10:00 am ET
Terrell Russell, Kory Draughn, Laura Capps, Mike Conway (NIEHS), Josh Moore (GerBI / OME), Guillaume Gay (France Bioimaging), Pascal de Boer (Groningen), Rodrigo Rosas (Maastricht University), Alice Stuart-Lee (SURF), Judith Lacoste (MIA Cellavie)
- Serialization of OMERO format to iRODS via PRC
- No NFSRODS
- AA) 'Save' is working, two commits in PR
- Python script, uploaded to OMERO, executed with Image payload
- #30
- Saves a new data object into iRODS, currently always named 'transfer.tar'
- DEMO
- Have to decide where in the iRODS namespace to drop these data objects - just pass a 'full path'? But that requires an OMERO user to 'know' where they want it to go in iRODS (iRODS knowledge). Could be derived/generated from the 'open' files in OMERO.
- Have to determine naming structure for data object (timestamp? Username? User gets to pass a string for the named file? Concatenation of all of these?)
- Mapping of OMERO user to iRODS users and permissions and passwords
- No place to store secrets, currently OMERO user info is just hashed passwords in OMERO db or in LDAP
- When in a browser, can just OAuth/OIDC everywhere else needed
- COULD just be that OMERO is always just a service account (a single irods user), and attach a payload/instructions for some policy on the iRODS side to open/parse/takeaction
- Simpler, no additional code on the OMERO side
- Complex, more code on the iRODS side
- Scripts not currently interactive with the user - so cannot prompt the human for credentials
- Want to make sure no overwriting existing data in iRODS
- No delay/async mechanism in OMERO proper
- Past users have done async operations 'out of band'
- Would just do the same here, pushing to subprocess
- BB) Load - Read snapshot from iRODS into active OMERO object
- Would create a new instance, could overwrite something in OMERO, TBD
- CC) List - shows the existing snapshots in the iRODS side
- Transfer Tool TODO
- On OMERO side, tar gets created, then gets pushed…
- Should be possible, to NOT copy… possibly using libarchive(.c)?
- Plan for long-term support for archives.
- Do we need to support more than one iRODS Zone?
- I think this is 'solved' by the user abstraction layer on the OMERO side
- Will need a secure place in the OMERO server OR a mapping/pointer into an already secure area
- Next time - research
- How does Bisque interconnect work? Users? Secrets? Memory footprint? Async?
- France Bioimaging
- Still working on the sync using NFSRODS
- https://gitlab.in2p3.fr/fbi-data/omero-quay/-/tree/dev?ref_type=heads
- Passing around schemas via zeroMQ with the truth, and then each side is responsible for making that truth, actually true. Latest timestamp wins in either direction.
- Proof of concepts getting deployed next week
- Next Meeting
- Feb 2024