WIP: Implement Object stack to support marshaling context-free data

[jrte]

Currently all named values defined in a ribose model are globally addressable, behaving analogously to general purpose registers on a CPU, which is fine for models with limited stack depth but problematic for more complex data. A value stack is required to support deeply nested structures.

This is going to take a long time, on the poc branch. Dev and master may receive minor updates in the interim.

[jrte]

Most likely solution will be to make named values local to each transducer on the transduction stack. Communication between adjacent stack frames can be effected by pushing data onto the input stack.

[jrte]

If transducer fields are cleared when transducer is pushed onto transducer stack but not cleared when popped, called transducer's fields will still be available on the popped value stack until another transducer is pushed. In that interval, caller can perhaps reference called transducers residual fields, eg @called~field. This would be more efficient than pushing return data onto the input stack for the caller to transduce.

Or called transducer can package a subset of its field values in a named data set where the caller could access them on return, eg $dataset~field. In that case data sets and their names would be global to the model.

[jrte]

Starting to debug this after significant refactoring. Will take some time. Likely interfaces changes:

• IField removed
• IEffector.getField() removed
• IOutput subsumes field access methods and data transfer methods from IField and IEffector
• IToken adds getTransducerOrdinal() method, which returns the ordinal of the defining transducer

Fields are now local to the defining transducer, so in parameter compilation contexts effectors combine transducer and field names as key to obtain field ordinals. These field ordinals are local, relative to a transducer stack frame. The IOutput javadoc will provide more details on field access and data transfer methods.

[jrte]

In the example below HeaderEffector.invoke() is set up to assemble fields defined by fields extracted by the Automaton transducer into a Header record and send it to the transduction target, ModelCompiler.

record Header (int version, int tapes, int transitions, int states, int symbols) {}

final class HeaderEffector extends BaseEffector<ModelCompiler> {
  private final static String transducer = "Automaton";
  private final String[] fields = new String[] {
    "version", "tapes", "transitions", "states", "symbols"
  };
  private final int[] fieldOrdinals;

  HeaderEffector(ModelCompiler compiler) {
    super(compiler, "header");
    this.fieldOrdinals = new int[fields.length];
  }

  @Override // called once for all effectors, before parameter compilation
  public void setOutput(IOutput output) throws EffectorException {
    super.setOutput(output);
    for (int i = 0; i < this.fields.length; i++) {
      this.fieldOrdinals[i] = super.output.getFieldOrdinal(transducer,  fields[i]);
    }
  }

  @Override // invoked from running transducer when header fields have been extracted
  public int invoke() throws EffectorException {
    ModelCompiler.this.putHeader(new Header(
      (int)super.output.asInteger(fieldOrdinals[0]),
      (int)super.output.asInteger(fieldOrdinals[1]),
      (int)super.output.asInteger(fieldOrdinals[2]),
      (int)super.output.asInteger(fieldOrdinals[3]),
      (int)super.output.asInteger(fieldOrdinals[4])));
    return IEffector.RTX_NONE;
  }
}

[jrte]

This is working but new functionality (stacked transducer local field values) is not tested with stack depth > 1 in present ci tests.

To complete this ribose needs a way to enable complex object models to be constructed on the transducer stack. This is just one transduction use case but it is probably the most familiar one for many developers. So I'll hold off closing this until I have a solution that builds an (almost) immutable object model from context free input (xml). Using records to hold primitive field values extracted by transducer in current frame and references to records extracted by called transducers. Any transducer that assembles a record as a part of the object model will invoke a specialized effector to direct the assimilation of records produced by called transducers into the transducer's stack frame. When the transducer stops it signals the caller which responds by invoking its specialized effector to pull the callee's final assembly into it own record. And so on down the stack until the completed object model is available in the bottom frame of the empty transducer stack.

Maybe that'll work.