GeezerVison

Draft. Do not merge or release yet. The deploy part only works once the Data Cloud platform accepts the AgentforceScorer invoke option and its feature gate is open. Until then, a deploy would be rejected. The types, template and local run work today. This PR is open now so the request/response contract can be reviewed early.

Summary

Adds AgentforceScorer, a new function feature for scoring a single Agentforce session. The scorer receives the session tracing detail view and returns open-ended scorer outputs.

datacustomcode init my_scorer --code-type function --use-in-feature AgentforceScorer
datacustomcode run my_scorer/payload/entrypoint.py --test-with my_scorer/payload/tests/test.json

Changes

• Types: new function/feature_types/agentforce_scorer.py with AgentforceScorerV1Request / AgentforceScorerV1Response, following the chunking.py conventions (V1 names, Field(description=...), extra="ignore").
  • Request: python_scorer_input: AgentforceScorerV1SessionTracingDetailView, made up of sessionState, actors, metrics and runs. Each run has messages and an agentLoop.steps.
  • Response: outputs: List[AgentforceScorerV1Output], where each output has label, value and is_passed.
  • Field names stay in the wire format (camelCase). The free-text fields (message, step input/output/attributes) are capped at 100,000 characters.
• Template: new templates/function/agentforce_scorer/ with a typed entrypoint.py, config.json and requirements.txt. test.json is generated from the request model.
• constants.py: registers the template folder and maps both type names to AgentforceScorer for deploy-time inference. Also adds FUNCTION_INVOKE_OPTIONS_FEATURES and FUNCTION_USE_IN_FEATURE_DEFAULT.
• init: unknown function features are now rejected. Before, a typo silently produced the chunking template.
• deploy: AgentforceScorer is sent as invokeOptions: ["AgentforceScorer"], which routes to the v67.0 endpoint. The legacy functionInvokeOptions field only accepts UnstructuredChunking. SearchIndexChunking behaves as before.

Testing

• New tests/test_agentforce_scorer.py (8 tests) covers:
  • parsing, defaults and serialization of the types
  • init scaffolding, test.json generation and feature inference
  • rejection of unknown features
  • running the template entrypoint
  • both deploy routes: invokeOptions for the scorer, functionInvokeOptions for chunking
• Full suite on Python 3.11: 707 passed.
• black and isort are clean on the changed files.
• Manual check: init → run --test-with returns {"outputs": [{"label": "user_turns", "value": "1", "is_passed": true}]}.
• Not tested: a real deploy against an org, which is blocked as described in the note above.

W-24740772

🤖 Generated with Claude Code