Code

assistant prefill #2615

Code

Merged

baberabb merged 21 commits intomainEleutherAI/lm-evaluation-harness:mainfrom prefixEleutherAI/lm-evaluation-harness:prefixCopy head branch name to clipboardJan 15, 2025

+744-57

Commits

Commits on Jan 6, 2025

• add assistant prefix

  baberabb

  committed
  c2d6780

  Copy full SHA for c2d6780

Commits on Jan 7, 2025

• add arc_challenge from llama

  baberabb

  committed
  d2ab5e1

  Copy full SHA for d2ab5e1

• nit

  baberabb

  committed
  9699111

  Copy full SHA for 9699111

• nit

  baberabb

  committed
  268e01a

  Copy full SHA for 268e01a

• nit

  baberabb

  committed
  18d0edb

  Copy full SHA for 18d0edb

Commits on Jan 8, 2025

• add assistant prefix

  baberabb

  committed
  71c9788

  Copy full SHA for 71c9788

• add mmlu_llama

  baberabb

  committed
  001abf6

  Copy full SHA for 001abf6

• nit

  baberabb

  committed
  c84649a

  Copy full SHA for c84649a

• nit

  baberabb

  committed
  6a97f83

  Copy full SHA for 6a97f83

• Revert "nit"

  Show description for 7225977
  baberabb

  committed
  7225977

  Copy full SHA for 7225977

• fix regex bug

  baberabb

  committed
  6af0772

  Copy full SHA for 6af0772

• add assistant_prefix to vllm

  baberabb

  committed
  572daa5

  Copy full SHA for 572daa5

• add Question:

  baberabb

  committed
  fb3a02f

  Copy full SHA for fb3a02f

Commits on Jan 9, 2025

• add mmlu_pro

  baberabb

  committed
  d9bd8cc

  Copy full SHA for d9bd8cc

Commits on Jan 14, 2025

• add fewshot assistant_prefix

  baberabb

  committed
  1399339

  Copy full SHA for 1399339

Commits on Jan 15, 2025

• use assistant_prefill

  baberabb

  committed
  278c0a8

  Copy full SHA for 278c0a8

• typehints

  baberabb

  committed
  6dec331

  Copy full SHA for 6dec331

• nits

  baberabb

  committed
  3ee673d

  Copy full SHA for 3ee673d

• nits

  baberabb

  committed
  1ed35fe

  Copy full SHA for 1ed35fe

• add to docs

  baberabb

  committed
  d2a8641

  Copy full SHA for d2a8641

• add readme

  baberabb

  committed
  6d62432

  Copy full SHA for 6d62432