dtbook-to-pef : script is crashing the pipeline

[NPavie]

On windows 11, with the pipeline app embedded engine (from the UI and from the cli with the engine)

When using the 1.14.19-bugfix branch of the assembly (that references the 1.14.27-SNAPSHOT version of the module repo), the dtbook to braille script is crashing very often on the "Transforming from OBFL to PEF".

Note that when it's not crashing, relaunching the job does provoke the following error in the job log

2024-07-16 15:40:24,127 [DEBUG] o.d.p.b.d.c.impl.OBFLToPEFStep - Created CompoundBrailleTranslator{translators={uncontracted=CompoundBrailleTranslator{translators={auto=LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl{translator=Translator{table=file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/spaces.cti,http://www.liblouis.org/tables/en-us-g1.ctb}, displayTable=file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/unicode.dis,file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/spaces.dis, hyphenator=LibhyphenJnaImpl$LibhyphenHyphenatorImpl{language=en, table=hyph_en_US.dic, id=transform9}, id=transform10}, none=UnityBrailleTranslator{id=transform7}}, id=transform14}, auto=CompoundBrailleTranslator{translators={auto=LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl{translator=Translator{table=file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/spaces.cti,http://www.liblouis.org/tables/en_CA.tbl}, displayTable=file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/unicode.dis,file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/spaces.dis, hyphenator=LibhyphenJnaImpl$LibhyphenHyphenatorImpl{language=en, table=hyph_en_US.dic, id=transform9}, id=transform15}, none=UnityBrailleTranslator{id=transform2}}, id=transform17}}, id=transform20}
2024-07-16 15:40:24,127 [DEBUG] o.d.p.b.d.c.impl.OBFLToPEFStep - Created CounterHandlingBrailleTranslator{id=transform21}
2024-07-16 15:40:26,934 [INFO ] org.daisy.pipeline.job.Job - Changing job status to: ERROR
2024-07-16 15:40:26,935 [ERROR] org.daisy.pipeline.job.Job - job finished with error state
Unexpected error in pxi:obfl-to-pef
	at {http://www.daisy.org/ns/pipeline/xproc/internal}obfl-to-pef(obfl-to-pef.xpl:69)
	at {http://www.daisy.org/ns/pipeline/xproc}obfl-to-pef(dotify-transform.xpl:90)
	at {http://www.w3.org/ns/xproc}when(dotify-transform.xpl:89)
	at {http://www.w3.org/ns/xproc}choose(dotify-transform.xpl:88)
	at {http://www.daisy.org/ns/pipeline/xproc}transform(dtbook-to-pef.convert.xpl:265)
	at {http://www.w3.org/ns/xproc}otherwise(dtbook-to-pef.convert.xpl:254)
	at {http://www.w3.org/ns/xproc}choose name="transform"(dtbook-to-pef.convert.xpl:171)
	at {http://www.daisy.org/ns/pipeline/xproc}dtbook-to-pef name="convert"(__processed__dtbook-to-pef.xpl:307)
Caused by: Invalid memory access
	at com.sun.jna.Native.invokeInt(Native Method)
	at com.sun.jna.Function.invoke(Function.java:429)
	at com.sun.jna.Function.invoke(Function.java:364)
	at com.sun.jna.Library$Handler.invoke(Library.java:270)
	at jdk.proxy2/jdk.proxy2.$Proxy13.hnj_hyphen_hyphenate2(Unknown Source)
	at ch.sbs.jhyphen.Hyphenator.hyphenate(Hyphenator.java:134)
	at org.daisy.pipeline.braille.libhyphen.impl.LibhyphenJnaImpl$LibhyphenHyphenatorImpl$1.getHyphenationOpportunities(LibhyphenJnaImpl.java:267)
	at org.daisy.pipeline.braille.common.AbstractHyphenator$util$DefaultFullHyphenator.transform(AbstractHyphenator.java:331)
	at org.daisy.pipeline.braille.common.AbstractHyphenator$util$DefaultFullHyphenator.transform(AbstractHyphenator.java:239)
	at org.daisy.pipeline.braille.common.AbstractHyphenator$util$LanguageBasedDispatchingFullHyphenator.transform(AbstractHyphenator.java:83)
	at org.daisy.pipeline.braille.common.AbstractHyphenator$util$DefaultFullHyphenator.transform(AbstractHyphenator.java:138)
	at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$HyphenatorAsFullHyphenator.transform(LiblouisTranslatorJnaImplProvider.java:1554)
	at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl.transform(LiblouisTranslatorJnaImplProvider.java:1052)
	at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl.access$600(LiblouisTranslatorJnaImplProvider.java:212)
	at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl$3.transform(LiblouisTranslatorJnaImplProvider.java:366)
	at org.daisy.pipeline.braille.common.BrailleTranslator$FromStyledTextToBraille.transform(BrailleTranslator.java:28)
	at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl$LineBreaker.translateAndHyphenate(LiblouisTranslatorJnaImplProvider.java:401)
	at org.daisy.pipeline.braille.common.AbstractBrailleTranslator$util$DefaultLineBreaker.transform(AbstractBrailleTranslator.java:316)
	at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$2$1.transform(CompoundBrailleTranslator.java:256)
	at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$TransformImpl.transform(CompoundBrailleTranslator.java:157)
	at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$2.transform(CompoundBrailleTranslator.java:266)
	at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$2$1.transform(CompoundBrailleTranslator.java:256)
	at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$TransformImpl.transform(CompoundBrailleTranslator.java:157)
	at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$2.transform(CompoundBrailleTranslator.java:266)
	at org.daisy.pipeline.braille.dotify.impl.CounterHandlingBrailleTranslator.transform(CounterHandlingBrailleTranslator.java:64)
	at org.daisy.pipeline.braille.dotify.impl.BrailleTranslatorFactoryServiceImpl$BrailleTranslatorFromBrailleTranslator.translate(BrailleTranslatorFactoryServiceImpl.java:172)
	at org.daisy.pipeline.braille.dotify.impl.BrailleTranslatorFactoryServiceImpl$BrailleTranslatorFromBrailleTranslator.translate(BrailleTranslatorFactoryServiceImpl.java:168)
	at org.daisy.dotify.formatter.impl.row.SegmentProcessor.toResult(SegmentProcessor.java:845)
	at org.daisy.dotify.formatter.impl.row.SegmentProcessor.layoutTextSegment(SegmentProcessor.java:603)
	at org.daisy.dotify.formatter.impl.row.SegmentProcessor.loadNextSegment(SegmentProcessor.java:533)
	at org.daisy.dotify.formatter.impl.row.SegmentProcessor.prepareNext(SegmentProcessor.java:459)
	at org.daisy.dotify.formatter.impl.row.SegmentProcessor.getNext(SegmentProcessor.java:503)
	at org.daisy.dotify.formatter.impl.row.SegmentProcessor.getNext(SegmentProcessor.java:477)
	at org.daisy.dotify.formatter.impl.row.BlockContentManager.hasNext(BlockContentManager.java:123)
	at org.daisy.dotify.formatter.impl.page.RowGroupProvider.hasNext(RowGroupProvider.java:96)
	at org.daisy.dotify.formatter.impl.page.BlockProcessor.hasNextInBlock(BlockProcessor.java:82)
	at org.daisy.dotify.formatter.impl.page.ScenarioData.processBlock(ScenarioData.java:81)
	at org.daisy.dotify.formatter.impl.page.ScenarioProcessor.processBlock(ScenarioProcessor.java:123)
	at org.daisy.dotify.formatter.impl.page.ScenarioProcessor.process(ScenarioProcessor.java:60)
	at org.daisy.dotify.formatter.impl.page.BlockSequence.selectScenario(BlockSequence.java:65)
	at org.daisy.dotify.formatter.impl.page.PageSequenceBuilder2.<init>(PageSequenceBuilder2.java:156)
	at org.daisy.dotify.formatter.impl.sheet.SheetDataSource.ensureBuffer(SheetDataSource.java:258)
	at org.daisy.dotify.formatter.impl.sheet.SheetDataSource.getRemaining(SheetDataSource.java:160)
	at org.daisy.dotify.formatter.impl.VolumeProvider.prepare(VolumeProvider.java:122)
	at org.daisy.dotify.formatter.impl.FormatterImpl.getVolumes(FormatterImpl.java:194)
	at org.daisy.dotify.formatter.impl.FormatterImpl.write(FormatterImpl.java:157)
	at org.daisy.dotify.formatter.impl.engine.LayoutEngineImpl.convert(LayoutEngineImpl.java:121)
	at org.daisy.pipeline.braille.dotify.calabash.impl.OBFLToPEFStep.run(OBFLToPEFStep.java:309)
	at {http://www.daisy.org/ns/pipeline/xproc/internal}obfl-to-pef(obfl-to-pef.xpl:69)
	... 7 more

When debugging other attempts where the engine and jvm crashes without reporting errors, I followed the same traces (i got up from the run function of the step up to org.daisy.dotify.formatter.impl.sheet.SheetDataSource.ensureBuffer(SheetDataSource.java:258) but I stopped here)

[bertfrees]

FIxed by daisy/jhyphen#8.