EleutherAI/lm-evaluation-harness: v0.4.11

v0.4.11 Release Notes Minor release. Stay tuned for bigger changes next release. New Platform Support Windows ML Backend — Native Windows ML inference support by @chapsiru and @chemwolf6922 in #3470, #3564, #3565 New Benchmarks & Tasks BEAR knowledge probe by @plonerma in #3496 Task Version Changes The following tasks have updated versions. Results from a previous task versions may not be directly comparable. See the linked PRs or individual task READMEs for changelogs. afrobench_belebele (all variants): 2 → 3 in #3551 evalita_llm: 0.0 → 0.1 in #3551 include (all 90 language variants): 0.0 → 0.1 in #3551 mgsm_direct (all 11 language variants): 3.0 → 4.0 by @LakshyaChaudhry in #3574 Fixes & Improvements Fixed SQuAD v2 evaluation by @HydrogenSulfate in #3535 Fixed MasakhaNEWS tasks — replaced non-existent headline_text field with headline by @Mr-Neutr0n in #3567 Fixed incorrect task configs by @baberabb in #3552 Replaced eval() with ast.literal_eval in task configs for safer parsing by @baberabb in #3577 Fixed SGLang duplicate registration error by @enpimashin in #3543 Restored hf_transfer import check by @baberabb in #3563 Fixed modify_gen_kwargs call in vLLM VLMs by @hmellor in #3573 Refactored vLLM gen_kwargs normalization inline to modify_gen_kwargs; fixed cached gen_kwargs mutation by @baberabb in #3582 Fixed README for task-listing CLI command by @UltimateJupiter in #3545 Updated dependencies by @baberabb in #3546 New Contributors @HydrogenSulfate made their first contribution in https://github.com/EleutherAI/lm-evaluation-harness/pull/3535 @UltimateJupiter made their first contribution in https://github.com/EleutherAI/lm-evaluation-harness/pull/3545 @enpimashin made their first contribution in https://github.com/EleutherAI/lm-evaluation-harness/pull/3543 @chapsiru made their first contribution in https://github.com/EleutherAI/lm-evaluation-harness/pull/3470 @chemwolf6922 made their first contribution in https://github.com/EleutherAI/lm-evaluation-harness/pull/3565 @plonerma made their first contribution in https://github.com/EleutherAI/lm-evaluation-harness/pull/3496 @hmellor made their first contribution in https://github.com/EleutherAI/lm-evaluation-harness/pull/3573 @Mr-Neutr0n made their first contribution in https://github.com/EleutherAI/lm-evaluation-harness/pull/3567 @LakshyaChaudhry made their first contribution in https://github.com/EleutherAI/lm-evaluation-harness/pull/3574 Full Changelog: https://github.com/EleutherAI/lm-evaluation-harness/compare/v0.4.10...v0.4.11