feat: Add context size support for benchmarks and update example usage

This commit adds support for specifying context size when running benchmarks, which is passed to the Ollama client as the `num_ctx` option. The changes include:

- Updated the `run` method in the base benchmark class to accept an optional `context_size` parameter
- Modified the Ollama client call to include context size in the options when provided
- Updated the `run_benchmarks` function to accept and pass through the context size
- Added example usage to the help output showing how to use the new context size parameter
- Fixed prompt formatting in the summarization benchmark to use `text` instead of `task`

The changes enable running benchmarks with custom context sizes, which is useful for testing models with different context window limitations.
This commit is contained in:
2026-01-26 15:21:55 +03:00
parent 2048e4e40d
commit f60dbf49f1
10 changed files with 44 additions and 16 deletions

3
run.sh
View File

@@ -73,7 +73,8 @@ else
echo ""
echo "Примеры использования:"
echo " * ./run.sh run -m second_constantine/t-lite-it-1.0:7b -b translation summarization"
echo " * ./run.sh run -m second_constantine/t-lite-it-1.0:7b --num-ctx 16000"
echo " * ./run.sh run -m second_constantine/t-lite-it-1.0:7b -u http://10.0.0.4:11434 -c 2048 -b translation summarization"
echo " * ./run.sh run -m translategemma:4b -u http://10.0.0.4:11434 -c 128000 -b summarization"
echo " * ./run.sh gen"
echo " * ./run.sh gen-mongo 507f1f77bcf86cd799439011"
echo " * ./run.sh gen-mongo --id-file ids.txt"