2d streamer not working

[azazellochg]

To reproduce:
0) 2+ million particles extracting in steaming with relion protocol

1. create and launch relion cl2d on the queue, stop the protocol
2. created and launched 2d streamer protocol, batch size 10000

It starts, doing nothing for 20 min and then crashes with db lock

00012:   STARTED: monitorStep, step 1
00013:     2020-06-24 10:25:40.973385
00014:   Checking new input...
00015:   Traceback (most recent call last):
00016:     File "/lmb/home/gsharov/soft/scipion3/scipion-pyworkflow/pyworkflow/mapper/sqlite.py", line 747, in __init__
00017:       self.doCreateTables = self.db.missingTables()
00018:     File "/lmb/home/gsharov/soft/scipion3/scipion-pyworkflow/pyworkflow/mapper/sqlite.py", line 1082, in missingTables
00019:       self.executeCommand(self.CHECK_TABLES)
00020:   sqlite3.OperationalError: database is locked
00021:   
00022:   During handling of the above exception, another exception occurred:
00023:   
00024:   Traceback (most recent call last):
00025:     File "/lmb/home/gsharov/soft/scipion3/scipion-pyworkflow/pyworkflow/protocol/protocol.py", line 188, in run
00026:       self._run()
00027:     File "/lmb/home/gsharov/soft/scipion3/scipion-pyworkflow/pyworkflow/protocol/protocol.py", line 239, in _run
00028:       resultFiles = self._runFunc()
00029:     File "/lmb/home/gsharov/soft/scipion3/scipion-pyworkflow/pyworkflow/protocol/protocol.py", line 235, in _runFunc
00030:       return self._func(*self._args)
00031:     File "/lmb/home/gsharov/soft/scipion-em-plugins/scipion-em-facilities/emfacilities/protocols/protocol_monitor_2d_streamer.py", line 107, in monitorStep
00032:       self._checkNewInput()
00033:     File "/lmb/home/gsharov/soft/scipion-em-plugins/scipion-em-facilities/emfacilities/protocols/protocol_monitor_2d_streamer.py", line 147, in _checkNewInput
00034:       for particle in self._iterParticles():
00035:     File "/lmb/home/gsharov/soft/scipion-em-plugins/scipion-em-facilities/emfacilities/protocols/protocol_monitor_2d_streamer.py", line 172, in _iterParticles
00036:       inputParts.load()
00037:     File "/lmb/home/gsharov/soft/scipion3/scipion-pyworkflow/pyworkflow/object.py", line 1196, in load
00038:       self._mapper = self._MapperClass(fn, self._loadClassesDict(), prefix, self._indexes)
00039:     File "/lmb/home/gsharov/soft/scipion3/scipion-pyworkflow/pyworkflow/mapper/sqlite.py", line 752, in __init__
00040:       raise SqliteFlatMapperException('Error creating SqliteFlatMapper, '
00041:   pyworkflow.mapper.sqlite.SqliteFlatMapperException: Error creating SqliteFlatMapper, dbName: Runs/000477_ProtRelionExtractParticles/particles.sqlite, tablePrefix:  
00042:    error: database is locked
00043:   Protocol failed: Error creating SqliteFlatMapper, dbName: Runs/000477_ProtRelionExtractParticles/particles.sqlite, tablePrefix:  
00044:    error: database is locked

[dmaluenda]

Thanks @azazellochg,

It seems some task have leave open a database or a concurrency error occurred... I will have a look.

When you say "1. create and launch relion cl2d on the queue, stop the protocol", you mean you are launching relion thru a queue system and after a while you stop the protocol before it's finished?

[azazellochg]

@dmaluenda yes, I launched cl2d on the queue, wait till it start the 1st step and kill it. I have no idea how this undocumented streamer works but I dont see another way to save a queue params without launching the template cl2d protocol. Runs/000477_ProtRelionExtractParticles/particles.sqlite will obviously be open since it is streaming particles.

[dmaluenda]

I see. The 2dStreamer usually works with a saved protocol template instead of a just launched and aborted protocol, but the queue approach was not taken into account during that design, I guess... Your way might be good but honestly I've never thought on that before.

Regarding the sqlite, it should not be opened. The set must be opened (it has a properties field indicating this: _streamState). However, sqlite files must be closed to be able to get access. If not, a concurrence error appear. TBH, I'm not sure that the problem is this. Could you check if exists the Runs/000477_ProtRelionExtractParticles/particles.sqlite-journal? This would indicate it remains open.

@delarosatrevin developed this protocol, he might help us here.