Skip to main content

Documentation Index

Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt

Use this file to discover all available pages before exploring further.

Applies to:
  • Plan -
  • Deployment -

​
Summary

Code-based scorers fail in online scoring rules with the error: 
The following keys are not permitted {'_bt_internal_override_pagination_key'}
Even though the scorer tests successfully. This occurs when a code based scorer that was uploaded using an older Python SDK version that includes internal keys incompatible with the current runtime environment.

​
Resolution Steps

​
Step 1: Update the Python SDK

Upgrade to Python SDK version 0.5.2 or later. 
pip install --upgrade braintrust==0.5.2

​
Step 2: Re-upload the scorer

Upload the scorer again using the updated SDK version. The scorer code itself doesn’t need to change. 
import braintrust

project = braintrust.projects.create(name="my-project")

# Re-upload your existing scorer with the updated SDK
project.scorers.create(
    name="my-scorer",
    function=my_scorer_function
)

​
Step 3: Test the online scoring rule

Verify the scorer now works correctly in the online scoring rule without the pagination key error.