Demo pool is returned as a stream and goes through demo verbalization.

[yoavkatz]

Example:

dataset = load_dataset(
card="cards.wnli",
template="templates.classification.multi_class.relation.default",
num_demos=2,
demos_pool_size=10,
loader_limit=20,
)
print(dataset)
print(dataset["demos_pool"][0]['source'])

DatasetDict({
validation: Dataset({
features: [‘metrics’, ‘data_classification_policy’, ‘target’, ‘references’, ‘postprocessors’, ‘source’, ‘task_data’, ‘group’],
num_rows: 3
})
test: Dataset({
features: [‘metrics’, ‘data_classification_policy’, ‘target’, ‘references’, ‘postprocessors’, ‘source’, ‘task_data’, ‘group’],
num_rows: 20
})
demos_pool: Dataset({
features: [‘metrics’, ‘data_classification_policy’, ‘target’, ‘references’, ‘postprocessors’, ‘source’, ‘task_data’, ‘group’],
num_rows: 10. <--- The demo pools is returned
})
train: Dataset({
features: [‘metrics’, ‘data_classification_policy’, ‘target’, ‘references’, ‘postprocessors’, ‘source’, ‘task_data’, ‘group’],
num_rows: 7
})
})

Each demo, in itself is verbalized, with its own demos. I’m not sure it’s needed:

---

Given a premise and hypothesis classify the entailment of the hypothesis to one of entailment, not entailment.
premise: Steve follows Fred’s example in everything. He influences him hugely.
hypothesis: Steve influences him hugely.
The entailment class is entailment
premise: George got free tickets to the play, but he gave them to Eric, because he was particularly eager to see it.
hypothesis: George was particularly eager to see it.
The entailment class is entailment
premise: I stuck a pin through a carrot. When I pulled the pin out, it had a hole.
hypothesis: The carrot had a hole.
The entailment class is