Load a Braintrust workflow guide before using its related tools.
Available skills:
- braintrust/automations-workflow: Set up, validate, and manage Braintrust alerts and scheduled Loop automations, including Patterns, threshold-triggered work, Slack or webhook delivery, and existing automation refinement. Use when a user asks to react to matching log or environment events, monitor a threshold, schedule recurring agent analysis, configure Patterns, or inspect an alert or Loop automation.
- braintrust/bro: Restate the last message in plain human language, with no jargon.
- braintrust/evaluator-workflow: Create, test, refine, deploy, rewind, and visualize Braintrust evaluators (scorers and classifiers) interactively. Use when a user asks to build or improve an evaluator, apply it to production logs with an online-scoring rule, backfill results, or review results.
- braintrust/topics-workflow: Topics allow you to discover clusters from traces using a pipeline that involves preprocessing traces (converting raw trace data into a textual representation with JavaScript code), computing facets (using an LLM to convert the preprocessed data into 1-2 sentences), and clustering (using algorithms like HDBSCAN). These clusters are useful for discovering use cases, monitoring failures, and tracking these trends over time. Use when configuring, evaluating, or improving Topics, facets, preprocessors, or Topics automations.
- braintrust/pattern-analysis: Investigate Braintrust project traces for high-confidence recurring behaviors, then create, validate, or update Pattern records with supporting evidence. Use when a user asks to find patterns, investigate a suspected recurring behavior, or create or update an existing Pattern based on trace analysis. Do not use for routine queries, summaries, chart creation, or configuring a Patterns automation.
Follow the returned Markdown for the rest of the conversation.
Protocol
Pass100/10020 / 20 pts
Connection, authentication, and protocol compatibility.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description86/100
Load a Braintrust workflow guide before using its related tools.
Available skills:
- braintrust/automations-workflow: Set up, validate, and manage Braintrust alerts and scheduled Loop automations, including Patterns, threshold-triggered work, Slack or webhook delivery, and existing automation refinement. Use when a user asks to react to matching log or environment events, monitor a threshold, schedule recurring agent analysis, configure Patterns, or inspect an alert or Loop automation.
- braintrust/bro: Restate the last message in plain human language, with no jargon.
- braintrust/evaluator-workflow: Create, test, refine, deploy, rewind, and visualize Braintrust evaluators (scorers and classifiers) interactively. Use when a user asks to build or improve an evaluator, apply it to production logs with an online-scoring rule, backfill results, or review results.
- braintrust/topics-workflow: Topics allow you to discover clusters from traces using a pipeline that involves preprocessing traces (converting raw trace data into a textual representation with JavaScript code), computing facets (using an LLM to convert the preprocessed data into 1-2 sentences), and clustering (using algorithms like HDBSCAN). These clusters are useful for discovering use cases, monitoring failures, and tracking these trends over time. Use when configuring, evaluating, or improving Topics, facets, preprocessors, or Topics automations.
- braintrust/pattern-analysis: Investigate Braintrust project traces for high-confidence recurring behaviors, then create, validate, or update Pattern records with supporting evidence. Use when a user asks to find patterns, investigate a suspected recurring behavior, or create or update an existing Pattern based on trace analysis. Do not use for routine queries, summaries, chart creation, or configuring a Patterns automation.
Follow the returned Markdown for the rest of the conversation.
The description explains the capability, when to use it, and what to expect.
Inputs75/100
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
skill_name
The exact Braintrust skill name to load.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "load_braintrust_skill",
"description": "Load a Braintrust workflow guide before using its related tools.\n\nAvailable skills:\n- braintrust/automations-workflow: Set up, validate, and manage Braintrust alerts and scheduled Loop automations, including Patterns, threshold-triggered work, Slack or webhook delivery, and existing automation refinement. Use when a user asks to react to matching log or environment events, monitor a threshold, schedule recurring agent analysis, configure Patterns, or inspect an alert or Loop automation.\n- braintrust/bro: Restate the last message in plain human language, with no jargon.\n- braintrust/evaluator-workflow: Create, test, refine, deploy, rewind, and visualize Braintrust evaluators (scorers and classifiers) interactively. Use when a user asks to build or improve an evaluator, apply it to production logs with an online-scoring rule, backfill results, or review results.\n- braintrust/topics-workflow: Topics allow you to discover clusters from traces using a pipeline that involves preprocessing traces (converting raw trace data into a textual representation with JavaScript code), computing facets (using an LLM to convert the preprocessed data into 1-2 sentences), and clustering (using algorithms like HDBSCAN). These clusters are useful for discovering use cases, monitoring failures, and tracking these trends over time. Use when configuring, evaluating, or improving Topics, facets, preprocessors, or Topics automations.\n- braintrust/pattern-analysis: Investigate Braintrust project traces for high-confidence recurring behaviors, then create, validate, or update Pattern records with supporting evidence. Use when a user asks to find patterns, investigate a suspected recurring behavior, or create or update an existing Pattern based on trace analysis. Do not use for routine queries, summaries, chart creation, or configuring a Patterns automation.\n\nFollow the returned Markdown for the rest of the conversation.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"skill_name"
],
"properties": {
"skill_name": {
"enum": [
"braintrust/automations-workflow",
"braintrust/bro",
"braintrust/evaluator-workflow",
"braintrust/topics-workflow",
"braintrust/pattern-analysis",
"braintrust/patterns-slack-digest"
],
"type": "string",
"description": "The exact Braintrust skill name to load."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Load Braintrust skill",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description91/100
Only call this tool while analyzing the work steps of a trace from the trace Work view. It returns a deterministic, chronological list of meaningful LLM and tool spans for work-section and per-span analysis.
Do not use it for general trace queries, summaries, or debugging.
The description explains the capability, when to use it, and what to expect.
Inputs58/100
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
object_type
Object type of the trace being analyzed.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
object_id
Object ID of the trace being analyzed.
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
root_span_id
Root span ID of the trajectory being partitioned.
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
Align the title with the capability actually described by this tool.
Description82/100
Only call this tool while analyzing the work steps of a single trace from the trace Work view. Its sole purpose is to save chronological work sections and concise annotations for the meaningful LLM and tool spans in that trace.
Call this tool exactly once after the entire trajectory has been partitioned. Define each contiguous section with span IDs from parallel_inference, covering every item exactly once in chronological order. Include one annotation per work item in the same order with its matching span ID. Do not calculate time or token totals because the UI derives them from the trace. Do not call this tool for general trace debugging, summaries, or unrelated analysis.
The description explains the capability, when to use it, and what to expect.
Inputs38/100
5 of 10 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
object_type
Object type of the trace being analyzed.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
object_id
Object ID of the trace being analyzed.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
root_span_id
Root span ID of the trajectory being partitioned.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
row_id
Row ID of the trace root span.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
span_id
Span ID of the trace root span.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
sections › items › title
Not provided
Missing description. Explain what this value means and how to supply it.
sections › items › description
Not provided
Missing description. Explain what this value means and how to supply it.
sections › items › span_ids
Not provided
Missing description. Explain what this value means and how to supply it.
span_annotations › items › span_id
Not provided
Missing description. Explain what this value means and how to supply it.
span_annotations › items › summary
Not provided
Missing description. Explain what this value means and how to supply it.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description83/100
Save or replace the diagnosis report for one trace. Use this independently from update_trace_work_report after identifying concrete failure modes and evidence, or when a user asks to edit the existing diagnosis.
Pass the complete replacement report, not a partial patch. Every evidence item should identify its supporting span and exact field or JSON path.
The description explains the capability, when to use it, and what to expect.
Inputs3/100
1 of 29 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
object_type
Not provided
Missing description. Explain what this value means and how to supply it.
object_id
Not provided
Missing description. Explain what this value means and how to supply it.
row_id
Not provided
Missing description. Explain what this value means and how to supply it.
span_id
Not provided
Missing description. Explain what this value means and how to supply it.
root_span_id
Not provided
Missing description. Explain what this value means and how to supply it.
debugger_span
Opaque Braintrust span reference to update.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
report › summary
Not provided
Missing description. Explain what this value means and how to supply it.
report › findings › items › title
Not provided
Missing description. Explain what this value means and how to supply it.
report › findings › items › severity
Not provided
Missing description. Explain what this value means and how to supply it.
report › findings › items › kind
Not provided
Missing description. Explain what this value means and how to supply it.
report › findings › items › hypothesis
Not provided
Missing description. Explain what this value means and how to supply it.
report › findings › items › root_cause
Not provided
Missing description. Explain what this value means and how to supply it.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description98/100
Preview a Braintrust Monitoring chart for project logs.
CHART DESIGN:
- Use a timeseries for trends over time: bars for countable discrete things (sums and counts) and lines for gauges (averages and percentiles). Use a toplist for groups ranked by an aggregate, and a single value for one simple large number.
- For counts, use a raw BTQL value with a separate count aggregator, such as { btql: "1", aggregator: { type: "count" } }. If you write the aggregate inline, use count(1); count() is invalid.
- Use a meaningful unit for the measure if applicable.
- Group when it makes the chart easier to interpret or reveals interesting trends. Avoid high-cardinality groupings that can be slow to query or difficult to read.
Use this after inspecting the available fields and before saving a chart. It returns a timeless chartConfig plus the optional page-level timeFrame used for the preview, without modifying any Monitoring view. Relative timeFrame presets remain live; omit timeFrame to use the Monitoring default. If the user asks to save it, pass chartConfig unchanged as an entry in create_monitoring_view.charts or as update_monitoring_view.chart. When creating a view, pass timeFrame separately as create_monitoring_view.timeFrame.
The description explains the capability, when to use it, and what to expect.
Inputs7/100
3 of 34 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
title
The title for the chart
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
definition › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
Missing description. Explain what this value means and how to supply it.
timeFrame › anyOf › 1 › from
Not provided
Missing description. Explain what this value means and how to supply it.
timeFrame › anyOf › 1 › to
Not provided
Missing description. Explain what this value means and how to supply it.
dataSource › entity
The entity type (project_logs)
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
dataSource › id
The UUID of the entity. UUIDs are 32 hexadecimal digits, divided into five groups separated by hyphens (8 digits, 4 digits, 4 digits, 4 digits, 12 digits). An example id is '20db338f-26cb-4056-acd7-fe7cf7de0e05'. THIS MUST BE A VALID UUID.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description90/100
Inspect a saved Monitoring view, including its options and ordered chart definitions. Copy the returned chart_id unchanged when updating or deleting a chart.
The description explains the capability, when to use it, and what to expect.
Inputs87/100
Input documentation gives an agent enough context to populate the arguments.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description86/100
Create a project-scoped Monitoring view, optionally containing charts that were already previewed with generate_monitor_chart.
Only call this when the user explicitly asks to create or save a view. Pass each returned chartConfig unchanged in the charts array, using a one-element array for a single chart. Pass the preview timeFrame separately as the optional view-level timeFrame; relative presets such as "3d" remain live. Omit timeFrame to use the Monitoring page default.
The description explains the capability, when to use it, and what to expect.
Inputs7/100
4 of 38 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
The project for the new view
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
name
The name of the new Monitoring view
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
timeFrame › anyOf › 1 › from
Not provided
Missing description. Explain what this value means and how to supply it.
timeFrame › anyOf › 1 › to
Not provided
Missing description. Explain what this value means and how to supply it.
charts › items › title
Not provided
Missing description. Explain what this value means and how to supply it.
charts › items › definition › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
Missing description. Explain what this value means and how to supply it.
charts › items › dataSource › entity
The entity type (project_logs)
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
charts › items › dataSource › id
The UUID of the entity. UUIDs are 32 hexadecimal digits, divided into five groups separated by hyphens (8 digits, 4 digits, 4 digits, 4 digits, 12 digits). An example id is '20db338f-26cb-4056-acd7-fe7cf7de0e05'. THIS MUST BE A VALID UUID.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
charts › items › timeFrame › from
Not provided
Missing description. Explain what this value means and how to supply it.
charts › items › timeFrame › to
Not provided
Missing description. Explain what this value means and how to supply it.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description85/100
Apply one or more chart or section edits to an existing project Monitoring view. Pass project_id and view_id. For a single chart edit, set the top-level operation to exactly insert, update, or remove. For section edits or bulk edits, pass an operations array instead; all operations are applied in order with one save. insert_group creates a section containing the provided existing chart_ids. remove_group deletes the section and keeps its charts at the section's position.
Use get_monitoring_view first. For update or remove, pass its chart_id unchanged as chart_id. For insert or update, pass chartConfig returned unchanged by generate_monitor_chart as chart. Use bulk operations when several charts need the same coordinated change. Only call this when the user explicitly asks to save the change.
The description explains the capability, when to use it, and what to expect.
Inputs19/100
11 of 45 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
The project containing the views
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
view_id
The Monitoring view to inspect
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
operation
A single chart edit. Omit when passing the operations array
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
chart_id
Required for update or remove. Copy chart_id unchanged from get_monitoring_view
Clear enough to supply this argument.
chart › title
Not provided
Missing description. Explain what this value means and how to supply it.
chart › definition › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
Missing description. Explain what this value means and how to supply it.
chart › dataSource › entity
The entity type (project_logs)
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
chart › dataSource › id
The UUID of the entity. UUIDs are 32 hexadecimal digits, divided into five groups separated by hyphens (8 digits, 4 digits, 4 digits, 4 digits, 12 digits). An example id is '20db338f-26cb-4056-acd7-fe7cf7de0e05'. THIS MUST BE A VALID UUID.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
chart › timeFrame › from
Not provided
Missing description. Explain what this value means and how to supply it.
chart › timeFrame › to
Not provided
Missing description. Explain what this value means and how to supply it.
operations › items › anyOf › 0 › operation
A single chart edit. Omit when passing the operations array
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
operations › items › anyOf › 0 › chart_id
Required for update or remove. Copy chart_id unchanged from get_monitoring_view
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description81/100
Create a versioned Braintrust prompt using the same function representation as project.prompts.create() and bt functions push.
Provide either a completion-style prompt or chat messages. By default, the tool fails if the slug already exists; set if_exists to replace to create a new version, or ignore to leave the existing prompt unchanged.
The description explains the capability, when to use it, and what to expect.
Inputs12/100
9 of 62 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
name
The display name.
Explain the meaning of this value, not just its name.
slug
The unique slug within the project.
Clear enough to supply this argument.
description
Not provided
Missing description. Explain what this value means and how to supply it.
tags
Not provided
Missing description. Explain what this value means and how to supply it.
metadata
Not provided
Missing description. Explain what this value means and how to supply it.
if_exists
What to do when the slug already exists: error, replace it with a new version, or ignore this write.
Clear enough to supply this argument.
prompt
A completion-style prompt. Provide either prompt or messages, but not both.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description80/100
Create a versioned Braintrust LLM or inline code evaluator. Set output_type to score for numeric scores or classification for categorical labels.
For an LLM score evaluator, provide choice_scores. For an LLM classification evaluator, provide choices without numeric mappings. By default, the tool fails if the slug already exists; set if_exists to replace to create a new version, or ignore to leave the existing evaluator unchanged.
The description explains the capability, when to use it, and what to expect.
Inputs16/100
14 of 76 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
name
The display name.
Explain the meaning of this value, not just its name.
slug
The unique slug within the project.
Clear enough to supply this argument.
description
Not provided
Missing description. Explain what this value means and how to supply it.
tags
Not provided
Missing description. Explain what this value means and how to supply it.
metadata
Not provided
Missing description. Explain what this value means and how to supply it.
if_exists
What to do when the slug already exists: error, replace it with a new version, or ignore this write.
Clear enough to supply this argument.
evaluator › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
Missing description. Explain what this value means and how to supply it.
evaluator › anyOf › 0 › model
Not provided
Missing description. Explain what this value means and how to supply it.
evaluator › anyOf › 0 › params
Not provided
Missing description. Explain what this value means and how to supply it.
evaluator › anyOf › 0 › tools
Not provided
Missing description. Explain what this value means and how to supply it.
evaluator › anyOf › 0 › use_cot
Not provided
Missing description. Explain what this value means and how to supply it.
evaluator › anyOf › 0 › choice_scores
Object mapping each choice to its score (0-1), e.g. {"a": 1, "b": 0}. Must have at least one entry; choices are case-sensitive. Scores must be numbers 0-1 — never null. To add a not-applicable/skip option, set allow_skip: true instead of using null.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
evaluator › anyOf › 0 › allow_skip
If true, adds a 'Skip' option; when the judge selects it the scorer returns null. Use this for not-applicable cases instead of putting null in choice_scores.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description79/100
Create a native Human Review score definition for a project: categorical choices, a numeric slider, or a free-form text field such as Review notes. Call once per score. Fails if the name already exists.
Explain mutations, irreversible effects, or prerequisites that matter before calling this tool.
Inputs53/100
5 of 9 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Project ID. In hosted Agent Runtime, call braintrust_context to obtain the current page project.
Clear enough to supply this argument.
name
The name shown to reviewers
Clear enough to supply this argument.
description
Guidance for reviewers
Clear enough to supply this argument.
score_type
The native human review input type
Clear enough to supply this argument.
categories › items › name
Name of the category
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
categories › items › value
Not provided
Missing description. Explain what this value means and how to supply it.
config › object_types
Not provided
Missing description. Explain what this value means and how to supply it.
config › visibility › anyOf › 0 › users
Not provided
Missing description. Explain what this value means and how to supply it.
config › visibility › anyOf › 0 › groups
Not provided
Missing description. Explain what this value means and how to supply it.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description97/100
List a project's automations, including online scoring rules, alerts, exports, retention policies, Topics automations, and other automation kinds. Returns complete configurations so an existing automation can be inspected before using a specialized update tool.
The description explains the capability, when to use it, and what to expect.
Inputs69/100
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
project_id
The project containing the automations
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
automation_id
Optional exact automation ID to retrieve
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
name
Optional exact automation name to filter by
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
kind
Optional automation kind to filter by
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description81/100
Pause or activate an alert, scheduled job, or online scoring rule. This controls configured online scoring rules, not reusable scorer functions. Repeating the current status is a successful no-op. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.
Explain mutations, irreversible effects, or prerequisites that matter before calling this tool.
Inputs67/100
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
project_id
The project containing the automations
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
automation_id
The automation to update
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
status
The desired lifecycle status. Missing stored status is treated as active.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description81/100
Run one saved, global, or inline evaluator on representative inputs without writing results. Inputs may be inline evaluator rows containing fields such as input, output, expected, and metadata, or canonical span, trace, and group references. Inline rows can include only the fields the evaluator needs. The function field accepts a saved function_id, a global_function, or inline_evaluator using the same definition as create_evaluator. Inline evaluators do not need to be saved first. Inline code must define a top-level handler function. Use representative passes, failures, and ambiguous cases before creating an online-scoring rule.
Describe the returned information so the agent knows whether it will answer the request.
Inputs12/100
12 of 76 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
function › anyOf › 0 › function_id
Not provided
Missing description. Explain what this value means and how to supply it.
function › anyOf › 0 › version
Not provided
Missing description. Explain what this value means and how to supply it.
function › anyOf › 1 › global_function
Not provided
Missing description. Explain what this value means and how to supply it.
function › anyOf › 1 › function_type
Not provided
Missing description. Explain what this value means and how to supply it.
function › anyOf › 2 › inline_evaluator › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
Object mapping each choice to its score (0-1), e.g. {"a": 1, "b": 0}. Must have at least one entry; choices are case-sensitive. Scores must be numbers 0-1 — never null. To add a not-applicable/skip option, set allow_skip: true instead of using null.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
If true, adds a 'Skip' option; when the judge selects it the scorer returns null. Use this for not-applicable cases instead of putting null in choice_scores.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description86/100
Save or rewind an online-scoring rule that runs saved evaluator functions. This tool is the approval surface for both mutations: state the proposed parameters, then call it without asking the user to type a separate confirmation. For save, provide function_ids, the rule scope, filters, and sampling rate. New rules default to paused; pass status active only when the user has authorized processing new traffic. Existing rules preserve their status when omitted. A group scope requires group_by and placement and accepts optional interval_seconds and max_traces. For rewind, require an existing active automation_id and choose start_time from the user's requested window. Activate a paused rule with set_automation_status before rewinding it. Do not repeatedly resubmit it to poll for progress.
The description explains the capability, when to use it, and what to expect.
Inputs73/100
19 of 22 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
operation
Save a rule definition or rewind an existing rule.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
automation_id
Required for rewind. For save, set this to update an existing rule or omit it to create one.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
name
Name of the project score
Explain the meaning of this value, not just its name.
description
Textual description of the project score
Explain the meaning of this value, not just its name.
function_ids
Required for save. The saved evaluator function IDs to run.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
sampling_rate
The sampling rate for online scoring
Explain what happens when this optional value is omitted.
status
Lifecycle status for save. New rules default to paused; existing rules preserve their current status when omitted.
Clear enough to supply this argument.
scope › anyOf › 0 › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
scope › anyOf › 0 › anyOf › 1 › type
Not provided
Missing description. Explain what this value means and how to supply it.
scope › anyOf › 0 › anyOf › 1 › idle_seconds
Consider trace complete after this many seconds of inactivity (default: 30)
Clear enough to supply this argument.
scope › anyOf › 0 › anyOf › 2 › type
Not provided
Missing description. Explain what this value means and how to supply it.
scope › anyOf › 0 › anyOf › 2 › group_by
Field path to group by, e.g. metadata.session_id
Explain what happens when this optional value is omitted.
scope › anyOf › 0 › anyOf › 2 › interval_seconds
Maximum time range to include when constructing a group
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
scope › anyOf › 0 › anyOf › 2 › max_traces
Maximum number of traces to include when constructing a group (default/max: 64)
Clear enough to supply this argument.
scope › anyOf › 0 › anyOf › 2 › placement
Which trace or traces to write grouped scorer results to
Clear enough to supply this argument.
scope › anyOf › 0 › anyOf › 2 › idle_seconds
Optional: trigger after this many seconds of inactivity
Clear enough to supply this argument.
btql_filter
Filter logs using BTQL
Clear enough to supply this argument.
apply_to_root_span
Whether to trigger online scoring on the root span of each trace. Only applies when scope is 'span' or unset.
Clear enough to supply this argument.
apply_to_span_names
Trigger online scoring on any spans with a name in this list. Only applies when scope is 'span' or unset.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
skip_logging
Whether to skip adding scorer spans when computing scores
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
start_time
Required for rewind. Inclusive ISO-8601 timestamp to reprocess from.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "update_online_scoring_rule",
"description": "Save or rewind an online-scoring rule that runs saved evaluator functions. This tool is the approval surface for both mutations: state the proposed parameters, then call it without asking the user to type a separate confirmation. For save, provide function_ids, the rule scope, filters, and sampling rate. New rules default to paused; pass status active only when the user has authorized processing new traffic. Existing rules preserve their status when omitted. A group scope requires group_by and placement and accepts optional interval_seconds and max_traces. For rewind, require an existing active automation_id and choose start_time from the user's requested window. Activate a paused rule with set_automation_status before rewinding it. Do not repeatedly resubmit it to poll for progress.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"operation"
],
"properties": {
"name": {
"type": "string",
"minLength": 1,
"description": "Name of the project score"
},
"scope": {
"anyOf": [
{
"anyOf": [
{
"type": "object",
"required": [
"type"
],
"properties": {
"type": {
"type": "string",
"const": "span"
}
},
"description": "Process individual spans",
"additionalProperties": false
},
{
"type": "object",
"required": [
"type"
],
"properties": {
"type": {
"type": "string",
"const": "trace"
},
"idle_seconds": {
"type": "number",
"description": "Consider trace complete after this many seconds of inactivity (default: 30)"
}
},
"description": "Process entire traces (all spans sharing the same root_span_id)",
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"group_by",
"placement"
],
"properties": {
"type": {
"type": "string",
"const": "group"
},
"group_by": {
"type": "string",
"description": "Field path to group by, e.g. metadata.session_id"
},
"placement": {
"enum": [
"first",
"each"
],
"type": "string",
"description": "Which trace or traces to write grouped scorer results to"
},
"max_traces": {
"type": "integer",
"maximum": 64,
"minimum": 1,
"description": "Maximum number of traces to include when constructing a group (default/max: 64)"
},
"idle_seconds": {
"type": "number",
"description": "Optional: trigger after this many seconds of inactivity"
},
"interval_seconds": {
"type": "number",
"minimum": 1,
"description": "Maximum time range to include when constructing a group"
}
},
"description": "Process spans/traces grouped by a field (e.g., session_id)",
"additionalProperties": false
}
]
},
{
"type": "null"
}
],
"description": "Required for save. The span, trace, or group unit to evaluate."
},
"status": {
"enum": [
"active",
"paused"
],
"type": "string",
"description": "Lifecycle status for save. New rules default to paused; existing rules preserve their current status when omitted."
},
"operation": {
"enum": [
"save",
"rewind"
],
"type": "string",
"description": "Save a rule definition or rewind an existing rule."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"start_time": {
"type": "string",
"format": "date-time",
"description": "Required for rewind. Inclusive ISO-8601 timestamp to reprocess from."
},
"btql_filter": {
"type": "string",
"minLength": 1,
"description": "Filter logs using BTQL"
},
"description": {
"type": "string",
"description": "Textual description of the project score"
},
"function_ids": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
},
"minItems": 1,
"description": "Required for save. The saved evaluator function IDs to run."
},
"skip_logging": {
"type": "boolean",
"description": "Whether to skip adding scorer spans when computing scores"
},
"automation_id": {
"type": "string",
"minLength": 1,
"description": "Required for rewind. For save, set this to update an existing rule or omit it to create one."
},
"sampling_rate": {
"type": "number",
"maximum": 1,
"minimum": 0,
"description": "The sampling rate for online scoring"
},
"apply_to_root_span": {
"type": "boolean",
"description": "Whether to trigger online scoring on the root span of each trace. Only applies when scope is 'span' or unset."
},
"apply_to_span_names": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
},
"description": "Trigger online scoring on any spans with a name in this list. Only applies when scope is 'span' or unset."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Update online scoring rule",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description93/100
Enable Topics for a Braintrust project.
Before using this tool, load and follow "braintrust/topics-workflow" with load_braintrust_skill, then inspect existing automations with list_automations using kind "topic". Provide a distinct name to create an additional Topics automation; reusing a name returns the existing automation without changing it. This creates the configured facet topic maps, supports span, trace, or group scope, and defaults new automations to trace scope and the Task facet when omitted. It seeds processing for new traffic but does not rewind historical data.
The description explains the capability, when to use it, and what to expect.
Inputs35/100
14 of 29 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
name
Automation name. Defaults to Topics. Use a distinct name to create an additional Topics automation.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
description
Not provided
Missing description. Explain what this value means and how to supply it.
status
Whether the automation is active or paused.
Explain the meaning of this value, not just its name.
sampling_rate
The sampling rate for topic automation
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
facet_model
Optional facet model override for topic automation
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
scope › anyOf › 0 › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
scope › anyOf › 0 › anyOf › 1 › type
Not provided
Missing description. Explain what this value means and how to supply it.
scope › anyOf › 0 › anyOf › 1 › idle_seconds
Consider trace complete after this many seconds of inactivity (default: 30)
Clear enough to supply this argument.
scope › anyOf › 0 › anyOf › 2 › type
Not provided
Missing description. Explain what this value means and how to supply it.
scope › anyOf › 0 › anyOf › 2 › group_by
Field path to group by, e.g. metadata.session_id
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
scope › anyOf › 0 › anyOf › 2 › interval_seconds
Maximum time range to include when constructing a group
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
scope › anyOf › 0 › anyOf › 2 › max_traces
Maximum number of traces to include when constructing a group (default/max: 64)
Explain what happens when this optional value is omitted.
scope › anyOf › 0 › anyOf › 2 › placement
Which trace or traces to write grouped scorer results to
Explain the meaning of this value, not just its name.
scope › anyOf › 0 › anyOf › 2 › idle_seconds
Optional: trigger after this many seconds of inactivity
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
btql_filter
Optional BTQL filter applied before topic automation.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
rerun_seconds
How often to recompute topic maps
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description91/100
Update an existing Topics automation's configuration.
Before using this tool, load and follow "braintrust/topics-workflow" with load_braintrust_skill and inspect the current automation with list_automations using kind "topic". This changes facets, topic maps, scope, filters, sampling, or timing without rewinding historical data.
The description explains the capability, when to use it, and what to expect.
Inputs45/100
15 of 27 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
name
Not provided
Missing description. Explain what this value means and how to supply it.
description
Not provided
Missing description. Explain what this value means and how to supply it.
status
Whether the automation is active or paused.
Clear enough to supply this argument.
sampling_rate
The sampling rate for topic automation
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
facet_model
Optional facet model override for topic automation
Clear enough to supply this argument.
scope › anyOf › 0 › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
scope › anyOf › 0 › anyOf › 1 › type
Not provided
Missing description. Explain what this value means and how to supply it.
scope › anyOf › 0 › anyOf › 1 › idle_seconds
Consider trace complete after this many seconds of inactivity (default: 30)
Clear enough to supply this argument.
scope › anyOf › 0 › anyOf › 2 › type
Not provided
Missing description. Explain what this value means and how to supply it.
scope › anyOf › 0 › anyOf › 2 › group_by
Field path to group by, e.g. metadata.session_id
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
scope › anyOf › 0 › anyOf › 2 › interval_seconds
Maximum time range to include when constructing a group
Explain what happens when this optional value is omitted.
scope › anyOf › 0 › anyOf › 2 › max_traces
Maximum number of traces to include when constructing a group (default/max: 64)
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
scope › anyOf › 0 › anyOf › 2 › placement
Which trace or traces to write grouped scorer results to
Explain what happens when this optional value is omitted.
scope › anyOf › 0 › anyOf › 2 › idle_seconds
Optional: trigger after this many seconds of inactivity
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
btql_filter
Optional BTQL filter applied before topic automation.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description75/100
Rewind an existing Topics automation to process historical data from an explicit start time or recent window.
Before using this tool, load and follow "braintrust/topics-workflow" with load_braintrust_skill. Estimate the eligible population and obtain authorization for the historical scope before calling it.
Explain mutations, irreversible effects, or prerequisites that matter before calling this tool.
Inputs90/100
Input documentation gives an agent enough context to populate the arguments.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
automation_id
The target Topics automation. May be omitted when the project has exactly one Topics automation.
Clear enough to supply this argument.
start_time
For rewind, the inclusive ISO-8601 timestamp to reprocess from.
Clear enough to supply this argument.
rewind_window_seconds
For rewind, reprocess this many seconds of recent history.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "rewind_topics_automation",
"description": "Rewind an existing Topics automation to process historical data from an explicit start time or recent window.\n\nBefore using this tool, load and follow \"braintrust/topics-workflow\" with load_braintrust_skill. Estimate the eligible population and obtain authorization for the historical scope before calling it.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"start_time": {
"type": "string",
"format": "date-time",
"description": "For rewind, the inclusive ISO-8601 timestamp to reprocess from."
},
"automation_id": {
"type": "string",
"minLength": 1,
"description": "The target Topics automation. May be omitted when the project has exactly one Topics automation."
},
"rewind_window_seconds": {
"type": "integer",
"description": "For rewind, reprocess this many seconds of recent history.",
"exclusiveMinimum": 0
}
},
"additionalProperties": false
},
"annotations": {
"title": "Rewind Topics automation",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description86/100
Create a versioned Braintrust facet that extracts structured information from spans or traces.
Before using this tool, load and follow "braintrust/topics-workflow" with load_braintrust_skill. Topics and topic-classification requests use that workflow.
This is a deployment/promotion tool, not a prompt-testing tool. Develop with inline test_facet_on_trace calls and run_eval experiments first; do not create a facet merely to test a candidate.
Facet extraction always uses Braintrust's built-in facet model; callers do not select a model.
The project default preprocessor is used when none is provided. By default, the tool fails if the slug already exists; set if_exists to replace to create a new version, or ignore to leave the existing facet unchanged.
The description explains the capability, when to use it, and what to expect.
Inputs45/100
8 of 15 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
name
The display name.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
slug
The unique slug within the project.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
description
Not provided
Missing description. Explain what this value means and how to supply it.
tags
Not provided
Missing description. Explain what this value means and how to supply it.
metadata
Not provided
Missing description. Explain what this value means and how to supply it.
if_exists
What to do when the slug already exists: error, replace it with a new version, or ignore this write.
Clear enough to supply this argument.
prompt
The prompt to use for LLM extraction. The preprocessed text will be provided as context.
Clear enough to supply this argument.
preprocessor › anyOf › 0 › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
preprocessor › anyOf › 0 › anyOf › 0 › id
Not provided
Missing description. Explain what this value means and how to supply it.
preprocessor › anyOf › 0 › anyOf › 0 › version
The version of the function
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
preprocessor › anyOf › 0 › anyOf › 1 › type
Not provided
Missing description. Explain what this value means and how to supply it.
preprocessor › anyOf › 0 › anyOf › 1 › name
Not provided
Missing description. Explain what this value means and how to supply it.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "create_facet",
"description": "Create a versioned Braintrust facet that extracts structured information from spans or traces.\n\nBefore using this tool, load and follow \"braintrust/topics-workflow\" with load_braintrust_skill. Topics and topic-classification requests use that workflow.\n\nThis is a deployment/promotion tool, not a prompt-testing tool. Develop with inline test_facet_on_trace calls and run_eval experiments first; do not create a facet merely to test a candidate.\n\nFacet extraction always uses Braintrust's built-in facet model; callers do not select a model.\n\nThe project default preprocessor is used when none is provided. By default, the tool fails if the slug already exists; set if_exists to replace to create a new version, or ignore to leave the existing facet unchanged.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"name",
"slug",
"prompt"
],
"properties": {
"name": {
"type": "string",
"minLength": 1,
"description": "The display name."
},
"slug": {
"type": "string",
"minLength": 1,
"description": "The unique slug within the project."
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
},
"prompt": {
"type": "string",
"description": "The prompt to use for LLM extraction. The preprocessed text will be provided as context."
},
"metadata": {
"type": "object",
"additionalProperties": {}
},
"if_exists": {
"enum": [
"error",
"ignore",
"replace"
],
"type": "string",
"default": "error",
"description": "What to do when the slug already exists: error, replace it with a new version, or ignore this write."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"description": {
"type": "string"
},
"preprocessor": {
"anyOf": [
{
"anyOf": [
{
"type": "object",
"required": [
"type",
"id"
],
"properties": {
"id": {
"type": "string"
},
"type": {
"type": "string",
"const": "function"
},
"version": {
"type": "string",
"description": "The version of the function"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"name"
],
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string",
"const": "global"
},
"function_type": {
"enum": [
"llm",
"scorer",
"task",
"tool",
"custom_view",
"preprocessor",
"facet",
"classifier",
"tag",
"parameters",
"sandbox"
],
"type": "string",
"default": "scorer",
"description": "The type of global function. Defaults to 'scorer'."
}
},
"additionalProperties": false
}
]
},
{
"type": "null"
}
],
"description": "The saved or global preprocessor to attach to the facet."
},
"no_match_pattern": {
"type": "string",
"description": "Regex pattern to identify outputs that do not match the facet. If the output matches, the facet will be saved as 'no_match'"
}
},
"additionalProperties": false
},
"annotations": {
"title": "Create facet",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description84/100
Create a versioned Braintrust preprocessor from inline JavaScript using the same QuickJS runtime as the Braintrust UI.
When the preprocessor is for Topics or a facet, first load and follow "braintrust/topics-workflow" with load_braintrust_skill.
By default, the tool fails if the slug already exists; set if_exists to replace to create a new version, or ignore to leave the existing preprocessor unchanged.
The description explains the capability, when to use it, and what to expect.
Inputs56/100
5 of 8 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
name
The display name.
Explain the meaning of this value, not just its name.
slug
The unique slug within the project.
Clear enough to supply this argument.
description
Not provided
Missing description. Explain what this value means and how to supply it.
tags
Not provided
Missing description. Explain what this value means and how to supply it.
metadata
Not provided
Missing description. Explain what this value means and how to supply it.
if_exists
What to do when the slug already exists: error, replace it with a new version, or ignore this write.
Clear enough to supply this argument.
code
The complete JavaScript preprocessor implementation, including its handler.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "create_preprocessor",
"description": "Create a versioned Braintrust preprocessor from inline JavaScript using the same QuickJS runtime as the Braintrust UI.\n\nWhen the preprocessor is for Topics or a facet, first load and follow \"braintrust/topics-workflow\" with load_braintrust_skill.\n\nBy default, the tool fails if the slug already exists; set if_exists to replace to create a new version, or ignore to leave the existing preprocessor unchanged.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"name",
"slug",
"code"
],
"properties": {
"code": {
"type": "string",
"minLength": 1,
"description": "The complete JavaScript preprocessor implementation, including its handler."
},
"name": {
"type": "string",
"minLength": 1,
"description": "The display name."
},
"slug": {
"type": "string",
"minLength": 1,
"description": "The unique slug within the project."
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
},
"metadata": {
"type": "object",
"additionalProperties": {}
},
"if_exists": {
"enum": [
"error",
"ignore",
"replace"
],
"type": "string",
"default": "error",
"description": "What to do when the slug already exists: error, replace it with a new version, or ignore this write."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"description": {
"type": "string"
}
},
"additionalProperties": false
},
"annotations": {
"title": "Create preprocessor",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description88/100
Get the typed settings for a Braintrust project. The response includes the stored settings and the effective default preprocessor, resolving an unset default to the built-in "thread" preprocessor.
The description explains the capability, when to use it, and what to expect.
Inputs99/100
Input documentation gives an agent enough context to populate the arguments.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description88/100
Set or clear the default preprocessor for a Braintrust project.
When configuring Topics, first load and follow "braintrust/topics-workflow" with load_braintrust_skill. Pass a saved preprocessor reference, a global preprocessor such as "thread", or null to restore the built-in default. This changes the default used by facets and other project functions that do not explicitly select a preprocessor.
The description explains the capability, when to use it, and what to expect.
Inputs29/100
3 of 7 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
preprocessor › anyOf › 0 › anyOf › 0 › type
Not provided
Missing description. Explain what this value means and how to supply it.
preprocessor › anyOf › 0 › anyOf › 0 › id
Not provided
Missing description. Explain what this value means and how to supply it.
preprocessor › anyOf › 0 › anyOf › 0 › version
The version of the function
Explain the meaning of this value, not just its name.
preprocessor › anyOf › 0 › anyOf › 1 › type
Not provided
Missing description. Explain what this value means and how to supply it.
preprocessor › anyOf › 0 › anyOf › 1 › name
Not provided
Missing description. Explain what this value means and how to supply it.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "set_project_default_preprocessor",
"description": "Set or clear the default preprocessor for a Braintrust project.\n\nWhen configuring Topics, first load and follow \"braintrust/topics-workflow\" with load_braintrust_skill. Pass a saved preprocessor reference, a global preprocessor such as \"thread\", or null to restore the built-in default. This changes the default used by facets and other project functions that do not explicitly select a preprocessor.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"preprocessor"
],
"properties": {
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"preprocessor": {
"anyOf": [
{
"anyOf": [
{
"type": "object",
"required": [
"type",
"id"
],
"properties": {
"id": {
"type": "string"
},
"type": {
"type": "string",
"const": "function"
},
"version": {
"type": "string",
"description": "The version of the function"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"name"
],
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string",
"const": "global"
},
"function_type": {
"type": "string",
"const": "preprocessor",
"default": "preprocessor",
"description": "The type of global function. Defaults to 'preprocessor'."
}
},
"additionalProperties": false
}
]
},
{
"type": "null"
}
],
"description": "The saved or global preprocessor to use by default, or null to restore the built-in default."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Set project default preprocessor",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description91/100
Run a saved, global, or inline preprocessor on one to 50 canonical span, trace, or group references concurrently using the same preview path as the Braintrust UI. For trace and group references, a custom preprocessor runs independently on each source span. Top-level array items are merged in trace order and exact-deduplicated; distinct joined transcript strings remain separate items, so return atomic items such as {role, content} for conversations. Each result is capped at 128 KiB. Use run_eval for larger sets.
When testing for Topics or a facet, first load and follow "braintrust/topics-workflow" with load_braintrust_skill.
This executes code and may create invocation logs, but it does not write a score to the source trace.
Describe the returned information so the agent knows whether it will answer the request.
Inputs15/100
4 of 22 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description88/100
Run an inline facet definition with a saved, global, inline, or project-default preprocessor on one to ten canonical span, trace, or group references concurrently using the same preview path as the Braintrust UI. Use run_eval for larger sets.
Before using this tool, load and follow "braintrust/topics-workflow" with load_braintrust_skill. Do not use one or a few previews as a substitute for its representative-trace discovery stage.
Inline facet previews use Braintrust's built-in facet model; callers do not select a model.
This may call an AI provider and create invocation logs, but it does not write the facet result to the source trace.
Describe the returned information so the agent knows whether it will answer the request.
Inputs22/100
7 of 25 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description96/100
Run a Braintrust experiment through the authenticated Eval API with a hosted dataset, inline rows, or a prior experiment, any saved or inline functional task, and zero or more saved or inline scorers.
When a prior experiment is used as data, its outputs become expected values unless an expected value was already recorded. This does not require sandbox credentials. It creates an experiment and may execute user-supplied code or invoke AI providers, so it can incur compute and model usage. The MCP service supplies the caller's Braintrust token, forces streaming for long runs, and returns the experiment start metadata and final summary.
The description explains the capability, when to use it, and what to expect.
Inputs68/100
18 of 25 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID where the experiment will be created. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
data › anyOf › 0 › dataset_id
Not provided
Missing description. Explain what this value means and how to supply it.
data › anyOf › 0 › dataset_version
The version of the dataset to evaluate
Explain the meaning of this value, not just its name.
data › anyOf › 0 › dataset_environment
The environment tag that resolves to the dataset version to evaluate
Explain the meaning of this value, not just its name.
data › anyOf › 0 › _internal_btql
Not provided
Missing description. Explain what this value means and how to supply it.
data › anyOf › 1 › project_name
Not provided
Missing description. Explain what this value means and how to supply it.
data › anyOf › 1 › dataset_name
Not provided
Missing description. Explain what this value means and how to supply it.
data › anyOf › 1 › dataset_version
The version of the dataset to evaluate
Clear enough to supply this argument.
data › anyOf › 1 › dataset_environment
The environment tag that resolves to the dataset version to evaluate
Clear enough to supply this argument.
data › anyOf › 1 › _internal_btql
Not provided
Missing description. Explain what this value means and how to supply it.
data › anyOf › 2 › data
Not provided
Missing description. Explain what this value means and how to supply it.
data › anyOf › 3 › experiment_name
Not provided
Missing description. Explain what this value means and how to supply it.
parameters
Values for any parameters used in the eval
Clear enough to supply this argument.
task
The Braintrust functional task. Use {function_id, version?} for a saved function, {project_name, slug, version?} for a saved project function, or an inline function descriptor supplied by the active workflow skill.
Clear enough to supply this argument.
scores
Zero or more Braintrust scorer function descriptors. Use saved function references or controlled inline scorer descriptors supplied by the active workflow skill.
Clear enough to supply this argument.
experiment_name
An optional name for the experiment created by this eval. If it conflicts with an existing experiment, it will be suffixed with a unique identifier.
Clear enough to supply this argument.
metadata
Optional experiment-level metadata to store about the evaluation. You can later use this to slice & dice across experiments.
Clear enough to supply this argument.
trial_count
The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results.
Clear enough to supply this argument.
timeout
The maximum duration, in milliseconds, to run the evaluation. Defaults to undefined, in which case there is no timeout.
Clear enough to supply this argument.
max_concurrency
The maximum number of tasks/scorers that will be run concurrently. Defaults to 10. If null is provided, no max concurrency will be used.
Clear enough to supply this argument.
base_experiment_name
An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.
Clear enough to supply this argument.
base_experiment_id
An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment.
Clear enough to supply this argument.
strict
If true, throw an error if one of the variables in the prompt is not present in the input
Clear enough to supply this argument.
extra_messages
A template path of extra messages to append to the conversion. These messages will be appended to the end of the conversation, after the last message.
Clear enough to supply this argument.
tags
Optional tags that will be added to the experiment.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "run_eval",
"description": "Run a Braintrust experiment through the authenticated Eval API with a hosted dataset, inline rows, or a prior experiment, any saved or inline functional task, and zero or more saved or inline scorers.\n\nWhen a prior experiment is used as data, its outputs become expected values unless an expected value was already recorded. This does not require sandbox credentials. It creates an experiment and may execute user-supplied code or invoke AI providers, so it can incur compute and model usage. The MCP service supplies the caller's Braintrust token, forces streaming for long runs, and returns the experiment start metadata and final summary.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"data",
"task"
],
"properties": {
"data": {
"anyOf": [
{
"type": "object",
"required": [
"dataset_id"
],
"properties": {
"dataset_id": {
"type": "string"
},
"_internal_btql": {
"anyOf": [
{
"type": "object",
"additionalProperties": {}
},
{
"type": "null"
}
]
},
"dataset_version": {
"type": [
"string",
"null"
],
"description": "The version of the dataset to evaluate"
},
"dataset_environment": {
"type": [
"string",
"null"
],
"description": "The environment tag that resolves to the dataset version to evaluate"
}
},
"description": "Dataset id",
"additionalProperties": false
},
{
"type": "object",
"required": [
"project_name",
"dataset_name"
],
"properties": {
"dataset_name": {
"type": "string"
},
"project_name": {
"type": "string"
},
"_internal_btql": {
"anyOf": [
{
"type": "object",
"additionalProperties": {}
},
{
"type": "null"
}
]
},
"dataset_version": {
"type": [
"string",
"null"
],
"description": "The version of the dataset to evaluate"
},
"dataset_environment": {
"type": [
"string",
"null"
],
"description": "The environment tag that resolves to the dataset version to evaluate"
}
},
"description": "Project and dataset name",
"additionalProperties": false
},
{
"type": "object",
"required": [
"data"
],
"properties": {
"data": {
"type": "array",
"items": {}
}
},
"description": "Dataset rows",
"additionalProperties": false
},
{
"type": "object",
"required": [
"experiment_name"
],
"properties": {
"experiment_name": {
"type": "string"
}
},
"description": "Experiment whose inputs and outputs should be used as dataset inputs and expected values",
"additionalProperties": false
}
],
"description": "The dataset to use"
},
"tags": {
"type": "array",
"items": {
"type": "string"
},
"description": "Optional tags that will be added to the experiment."
},
"task": {
"type": "object",
"description": "The Braintrust functional task. Use {function_id, version?} for a saved function, {project_name, slug, version?} for a saved project function, or an inline function descriptor supplied by the active workflow skill.",
"additionalProperties": {}
},
"scores": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": {
"$ref": "#/properties/task/additionalProperties"
}
},
"default": [],
"description": "Zero or more Braintrust scorer function descriptors. Use saved function references or controlled inline scorer descriptors supplied by the active workflow skill."
},
"strict": {
"type": [
"boolean",
"null"
],
"description": "If true, throw an error if one of the variables in the prompt is not present in the input"
},
"timeout": {
"type": [
"number",
"null"
],
"description": "The maximum duration, in milliseconds, to run the evaluation. Defaults to undefined, in which case there is no timeout."
},
"metadata": {
"type": "object",
"description": "Optional experiment-level metadata to store about the evaluation. You can later use this to slice & dice across experiments.",
"additionalProperties": {}
},
"parameters": {
"type": "object",
"description": "Values for any parameters used in the eval",
"additionalProperties": {}
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID where the experiment will be created. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"trial_count": {
"type": [
"number",
"null"
],
"description": "The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results."
},
"extra_messages": {
"type": "string",
"description": "A template path of extra messages to append to the conversion. These messages will be appended to the end of the conversation, after the last message."
},
"experiment_name": {
"type": "string",
"description": "An optional name for the experiment created by this eval. If it conflicts with an existing experiment, it will be suffixed with a unique identifier."
},
"max_concurrency": {
"type": [
"number",
"null"
],
"default": 10,
"description": "The maximum number of tasks/scorers that will be run concurrently. Defaults to 10. If null is provided, no max concurrency will be used."
},
"base_experiment_id": {
"type": [
"string",
"null"
],
"description": "An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment."
},
"base_experiment_name": {
"type": [
"string",
"null"
],
"description": "An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Run eval",
"readOnlyHint": false,
"openWorldHint": true,
"idempotentHint": false,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description75/100
Insert, update, or delete up to 1000 Braintrust dataset rows using explicit operations.
Target an existing dataset by ID or name. Set create_if_missing only when a new named dataset should be created. In hosted Agent Runtime, call braintrust_context to obtain the current project and page dataset IDs.
The description explains the capability, when to use it, and what to expect.
Inputs64/100
17 of 22 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Project ID. Required when targeting by dataset name; optional when dataset_id is provided because the project is resolved from the dataset.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
dataset_id
The dataset ID. In hosted Agent Runtime, call braintrust_context to obtain the current page dataset. Provide either dataset_id or dataset_name.
Clear enough to supply this argument.
dataset_name
The dataset name within the project. Provide either dataset_name or dataset_id.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
create_if_missing
When targeting by name, create the dataset if it does not exist. Defaults to false.
Clear enough to supply this argument.
reason
A short explanation of why these dataset changes are useful.
Clear enough to supply this argument.
changes › items › anyOf › 0 › operation
Not provided
Missing description. Explain what this value means and how to supply it.
changes › items › anyOf › 0 › input
The dataset row input. Omit on update to leave it unchanged.
Explain the meaning of this value, not just its name.
changes › items › anyOf › 0 › expected
The dataset row expected value. Omit on update to leave it unchanged.
Explain the meaning of this value, not just its name.
changes › items › anyOf › 0 › metadata
The dataset row metadata object. Omit on update to leave it unchanged.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "edit_dataset_rows",
"description": "Insert, update, or delete up to 1000 Braintrust dataset rows using explicit operations.\n\nTarget an existing dataset by ID or name. Set create_if_missing only when a new named dataset should be created. In hosted Agent Runtime, call braintrust_context to obtain the current project and page dataset IDs.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"changes"
],
"properties": {
"reason": {
"type": "string",
"description": "A short explanation of why these dataset changes are useful."
},
"changes": {
"type": "array",
"items": {
"anyOf": [
{
"type": "object",
"required": [
"operation"
],
"properties": {
"tags": {
"type": "array",
"items": {
"type": "string"
},
"description": "Tags for the new row."
},
"input": {
"description": "The dataset row input. Omit on update to leave it unchanged."
},
"origin": {
"type": "object",
"required": [
"object_type",
"object_id",
"id"
],
"properties": {
"id": {
"type": "string",
"description": "ID of the original event."
},
"created": {
"type": [
"string",
"null"
],
"description": "Created timestamp of the original event. Used to help sort in the UI"
},
"_xact_id": {
"type": [
"string",
"null"
],
"description": "Transaction ID of the original event."
},
"object_id": {
"type": "string",
"format": "uuid",
"description": "ID of the object the event is originating from."
},
"object_type": {
"enum": [
"project_logs",
"experiment",
"dataset",
"prompt",
"function",
"prompt_session"
],
"type": "string",
"description": "Type of the object the event is originating from."
}
},
"description": "Reference to the source row this new row was copied from.",
"additionalProperties": false
},
"expected": {
"description": "The dataset row expected value. Omit on update to leave it unchanged."
},
"metadata": {
"type": "object",
"description": "The dataset row metadata object. Omit on update to leave it unchanged.",
"additionalProperties": {}
},
"operation": {
"type": "string",
"const": "insert"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"operation",
"id"
],
"properties": {
"id": {
"type": "string",
"minLength": 1
},
"input": {
"$ref": "#/properties/changes/items/anyOf/0/properties/input"
},
"expected": {
"$ref": "#/properties/changes/items/anyOf/0/properties/expected"
},
"metadata": {
"$ref": "#/properties/changes/items/anyOf/0/properties/metadata"
},
"operation": {
"type": "string",
"const": "update"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"operation",
"id"
],
"properties": {
"id": {
"type": "string",
"minLength": 1
},
"operation": {
"type": "string",
"const": "delete"
}
},
"additionalProperties": false
}
]
},
"maxItems": 1000,
"minItems": 1,
"description": "Insert, update, or delete operations to perform against the dataset. Inserts and updates require at least one of input, expected, or metadata."
},
"dataset_id": {
"type": "string",
"minLength": 1,
"description": "The dataset ID. In hosted Agent Runtime, call braintrust_context to obtain the current page dataset. Provide either dataset_id or dataset_name."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Project ID. Required when targeting by dataset name; optional when dataset_id is provided because the project is resolved from the dataset."
},
"dataset_name": {
"type": "string",
"minLength": 1,
"description": "The dataset name within the project. Provide either dataset_name or dataset_id."
},
"create_if_missing": {
"type": "boolean",
"default": false,
"description": "When targeting by name, create the dataset if it does not exist. Defaults to false."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Edit dataset rows",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description87/100
Add up to 1000 project log rows to a Braintrust dataset by row ID, like the UI's "Add to dataset", without reading and re-sending their contents.
Find candidate rows with sql_query against project_logs, then pass their `id` values. Each copied row keeps the log's input, metadata, and tags, takes its expected value from the log's expected (or output), and links back to the source log via origin. Target an existing dataset by ID or name. Set create_if_missing only when a new named dataset should be created. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.
Describe the returned information so the agent knows whether it will answer the request.
Inputs98/100
Input documentation gives an agent enough context to populate the arguments.
project_id
Project ID whose logs to copy. The target dataset must be in this project.
Clear enough to supply this argument.
log_row_ids
The `id` values of the project log rows (spans) to copy, as returned by sql_query against project_logs. Up to 1000 per call; split larger copies across calls.
Clear enough to supply this argument.
dataset_id
The dataset ID. In hosted Agent Runtime, call braintrust_context to obtain the current page dataset. Provide either dataset_id or dataset_name.
Clear enough to supply this argument.
dataset_name
The dataset name within the project. Provide either dataset_name or dataset_id.
Clear enough to supply this argument.
create_if_missing
When targeting by name, create the dataset if it does not exist. Defaults to false.
Clear enough to supply this argument.
reason
A short explanation of why these dataset changes are useful.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "add_to_dataset",
"description": "Add up to 1000 project log rows to a Braintrust dataset by row ID, like the UI's \"Add to dataset\", without reading and re-sending their contents.\n\nFind candidate rows with sql_query against project_logs, then pass their `id` values. Each copied row keeps the log's input, metadata, and tags, takes its expected value from the log's expected (or output), and links back to the source log via origin. Target an existing dataset by ID or name. Set create_if_missing only when a new named dataset should be created. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"project_id",
"log_row_ids"
],
"properties": {
"reason": {
"type": "string",
"description": "A short explanation of why these dataset changes are useful."
},
"dataset_id": {
"type": "string",
"minLength": 1,
"description": "The dataset ID. In hosted Agent Runtime, call braintrust_context to obtain the current page dataset. Provide either dataset_id or dataset_name."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Project ID whose logs to copy. The target dataset must be in this project."
},
"log_row_ids": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
},
"maxItems": 1000,
"minItems": 1,
"description": "The `id` values of the project log rows (spans) to copy, as returned by sql_query against project_logs. Up to 1000 per call; split larger copies across calls."
},
"dataset_name": {
"type": "string",
"minLength": 1,
"description": "The dataset name within the project. Provide either dataset_name or dataset_id."
},
"create_if_missing": {
"type": "boolean",
"default": false,
"description": "When targeting by name, create the dataset if it does not exist. Defaults to false."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Add to dataset",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description94/100
List the Slack workspaces connected to a project's org and the public channels in a workspace, so a Slack alert action targets the channel the user named instead of a guessed ID.
Call this before setting any Slack action's workspace_id and channel, passing the user's requested channel name as query. Match the requested channel to a returned channel and use its exact id; never invent or reuse a channel ID from context. If the org has multiple connected workspaces, workspace_id is null and channels is null until you call again with a specific workspace_id. If the channel isn't listed, ask the user for the channel ID rather than guessing. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.
Describe the returned information so the agent knows whether it will answer the request.
Inputs92/100
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
workspace_id
Optional Slack workspace ID to list channels for. Omit when the org has a single connected workspace to use it automatically.
Clear enough to supply this argument.
query
Optional case-insensitive substring to filter channels by name.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "list_slack_channels",
"description": "List the Slack workspaces connected to a project's org and the public channels in a workspace, so a Slack alert action targets the channel the user named instead of a guessed ID.\n\nCall this before setting any Slack action's workspace_id and channel, passing the user's requested channel name as query. Match the requested channel to a returned channel and use its exact id; never invent or reuse a channel ID from context. If the org has multiple connected workspaces, workspace_id is null and channels is null until you call again with a specific workspace_id. If the channel isn't listed, ask the user for the channel ID rather than guessing. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"query": {
"type": "string",
"minLength": 1,
"description": "Optional case-insensitive substring to filter channels by name."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"workspace_id": {
"type": "string",
"minLength": 1,
"description": "Optional Slack workspace ID to list channels for. Omit when the org has a single connected workspace to use it automatically."
}
},
"additionalProperties": false
},
"annotations": {
"title": "List Slack channels",
"readOnlyHint": true,
"openWorldHint": true,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description73/100
Create an alert for individual matching project logs. config.btql_filter is a boolean expression, not a SELECT query, and config.interval_seconds throttles repeated notifications without aggregating logs. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.
Explain mutations, irreversible effects, or prerequisites that matter before calling this tool.
Inputs68/100
18 of 20 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
name
The automation name.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
description
Not provided
Missing description. Explain what this value means and how to supply it.
if_exists
What to do when an automation with the same project and name exists: error, replace it, or ignore this write.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › event_type
The type of automation.
Explain what happens when this optional value is omitted.
config › status
Whether the automation is active or paused.
Explain the meaning of this value, not just its name.
config › btql_filter
BTQL filter to identify rows for the automation rule
Clear enough to supply this argument.
config › interval_seconds
Perform the triggered action at most once in this interval of seconds
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › action › anyOf › 0 › type
The type of action to take
Clear enough to supply this argument.
config › action › anyOf › 0 › url
The webhook URL to send the request to
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › action › anyOf › 0 › formatting_prompt
Instructions for Loop to format content sent to this destination
Clear enough to supply this argument.
config › action › anyOf › 1 › type
The type of action to take
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
config › action › anyOf › 1 › workspace_id
The Slack workspace ID to post to
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
config › action › anyOf › 1 › channel
The Slack channel ID to post to
Explain the meaning of this value, not just its name.
config › action › anyOf › 1 › message_template
Custom message template for the alert
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › action › anyOf › 1 › formatting_prompt
Publish a Slack mrkdwn digest.
Pattern selection requirement: include every selected Pattern whose final status is active at the end of this automation run, plus every selected Pattern that changed from active to inactive during this run (a newly inactive outcome). Exclude Patterns that were already inactive before this run and remained inactive. Do not mention excluded Patterns anywhere in the digest, including Highlights. If no selected Patterns meet these criteria, say "No current pattern outcomes." This requirement overrides conflicting formatting guidance.
Include a complete "*Pattern outcomes*" section with one row for every included Pattern.
Use this row format exactly:
• <pattern_url|Pattern title> — `outcome`
If a Pattern has no URL, use the plain title instead. Do not use GitHub Markdown tables or code-block tables, because links must remain clickable. Do not omit any active selected Pattern.
After the outcome list, include a "*Highlights*" section with one very short paragraph, 2-3 sentences maximum. Summarize what changed or what broadly stands out among the included active Patterns. Do not introduce new claims beyond the run report.
Clear enough to supply this argument.
config › action › anyOf › 2 › type
The type of action to take
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "create_log_alert",
"description": "Create an alert for individual matching project logs. config.btql_filter is a boolean expression, not a SELECT query, and config.interval_seconds throttles repeated notifications without aggregating logs. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"name",
"config"
],
"properties": {
"name": {
"type": "string",
"minLength": 1,
"description": "The automation name."
},
"config": {
"type": "object",
"required": [
"event_type",
"btql_filter",
"interval_seconds",
"action"
],
"properties": {
"action": {
"anyOf": [
{
"type": "object",
"required": [
"type",
"url"
],
"properties": {
"url": {
"type": "string",
"description": "The webhook URL to send the request to"
},
"type": {
"type": "string",
"const": "webhook",
"description": "The type of action to take"
},
"formatting_prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Instructions for Loop to format content sent to this destination"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"workspace_id",
"channel"
],
"properties": {
"type": {
"type": "string",
"const": "slack",
"description": "The type of action to take"
},
"channel": {
"type": "string",
"description": "The Slack channel ID to post to"
},
"workspace_id": {
"type": "string",
"description": "The Slack workspace ID to post to"
},
"message_template": {
"type": "string",
"description": "Custom message template for the alert"
},
"formatting_prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Publish a Slack mrkdwn digest.\n\nPattern selection requirement: include every selected Pattern whose final status is active at the end of this automation run, plus every selected Pattern that changed from active to inactive during this run (a newly inactive outcome). Exclude Patterns that were already inactive before this run and remained inactive. Do not mention excluded Patterns anywhere in the digest, including Highlights. If no selected Patterns meet these criteria, say \"No current pattern outcomes.\" This requirement overrides conflicting formatting guidance.\n\nInclude a complete \"*Pattern outcomes*\" section with one row for every included Pattern.\n\nUse this row format exactly:\n• <pattern_url|Pattern title> — `outcome`\n\nIf a Pattern has no URL, use the plain title instead. Do not use GitHub Markdown tables or code-block tables, because links must remain clickable. Do not omit any active selected Pattern.\n\nAfter the outcome list, include a \"*Highlights*\" section with one very short paragraph, 2-3 sentences maximum. Summarize what changed or what broadly stands out among the included active Patterns. Do not introduce new claims beyond the run report."
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"routing_key_secret_name",
"severity"
],
"properties": {
"type": {
"type": "string",
"const": "pagerduty",
"description": "The type of action to take"
},
"severity": {
"enum": [
"critical",
"error",
"warning",
"info"
],
"type": "string"
},
"formatting_prompt": {
"$ref": "#/properties/config/properties/action/anyOf/0/properties/formatting_prompt"
},
"routing_key_secret_name": {
"type": "string",
"format": "uuid",
"description": "The data-plane secret containing the PagerDuty routing key"
}
},
"additionalProperties": false
}
],
"description": "The action to take when the automation rule is triggered"
},
"status": {
"enum": [
"active",
"paused"
],
"type": "string",
"description": "Whether the automation is active or paused."
},
"event_type": {
"type": "string",
"const": "logs",
"description": "The type of automation."
},
"btql_filter": {
"type": "string",
"description": "BTQL filter to identify rows for the automation rule"
},
"interval_seconds": {
"type": "number",
"maximum": 2592000,
"minimum": 1,
"description": "Perform the triggered action at most once in this interval of seconds"
}
},
"description": "The complete log alert configuration.",
"additionalProperties": false
},
"if_exists": {
"enum": [
"error",
"ignore",
"replace"
],
"type": "string",
"default": "error",
"description": "What to do when an automation with the same project and name exists: error, replace it, or ignore this write."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"description": {
"type": "string"
}
},
"additionalProperties": false
},
"annotations": {
"title": "Create log alert",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description68/100
Create an alert for environment updates. Use config.environment_filter to limit notifications to specific environment slugs, or omit it to match all environments. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.
Explain mutations, irreversible effects, or prerequisites that matter before calling this tool.
Inputs68/100
17 of 19 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
name
The automation name.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
description
Not provided
Missing description. Explain what this value means and how to supply it.
if_exists
What to do when an automation with the same project and name exists: error, replace it, or ignore this write.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › event_type
The type of automation.
Explain what happens when this optional value is omitted.
config › status
Whether the automation is active or paused.
Clear enough to supply this argument.
config › environment_filter
Optional list of environment slugs to filter by
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › action › anyOf › 0 › type
The type of action to take
Clear enough to supply this argument.
config › action › anyOf › 0 › url
The webhook URL to send the request to
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › action › anyOf › 0 › formatting_prompt
Instructions for Loop to format content sent to this destination
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › action › anyOf › 1 › type
The type of action to take
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
config › action › anyOf › 1 › workspace_id
The Slack workspace ID to post to
Clear enough to supply this argument.
config › action › anyOf › 1 › channel
The Slack channel ID to post to
Explain the meaning of this value, not just its name.
config › action › anyOf › 1 › message_template
Custom message template for the alert
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › action › anyOf › 1 › formatting_prompt
Publish a Slack mrkdwn digest.
Pattern selection requirement: include every selected Pattern whose final status is active at the end of this automation run, plus every selected Pattern that changed from active to inactive during this run (a newly inactive outcome). Exclude Patterns that were already inactive before this run and remained inactive. Do not mention excluded Patterns anywhere in the digest, including Highlights. If no selected Patterns meet these criteria, say "No current pattern outcomes." This requirement overrides conflicting formatting guidance.
Include a complete "*Pattern outcomes*" section with one row for every included Pattern.
Use this row format exactly:
• <pattern_url|Pattern title> — `outcome`
If a Pattern has no URL, use the plain title instead. Do not use GitHub Markdown tables or code-block tables, because links must remain clickable. Do not omit any active selected Pattern.
After the outcome list, include a "*Highlights*" section with one very short paragraph, 2-3 sentences maximum. Summarize what changed or what broadly stands out among the included active Patterns. Do not introduce new claims beyond the run report.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › action › anyOf › 2 › type
The type of action to take
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "create_environment_update_alert",
"description": "Create an alert for environment updates. Use config.environment_filter to limit notifications to specific environment slugs, or omit it to match all environments. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"name",
"config"
],
"properties": {
"name": {
"type": "string",
"minLength": 1,
"description": "The automation name."
},
"config": {
"type": "object",
"required": [
"event_type",
"action"
],
"properties": {
"action": {
"anyOf": [
{
"type": "object",
"required": [
"type",
"url"
],
"properties": {
"url": {
"type": "string",
"description": "The webhook URL to send the request to"
},
"type": {
"type": "string",
"const": "webhook",
"description": "The type of action to take"
},
"formatting_prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Instructions for Loop to format content sent to this destination"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"workspace_id",
"channel"
],
"properties": {
"type": {
"type": "string",
"const": "slack",
"description": "The type of action to take"
},
"channel": {
"type": "string",
"description": "The Slack channel ID to post to"
},
"workspace_id": {
"type": "string",
"description": "The Slack workspace ID to post to"
},
"message_template": {
"type": "string",
"description": "Custom message template for the alert"
},
"formatting_prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Publish a Slack mrkdwn digest.\n\nPattern selection requirement: include every selected Pattern whose final status is active at the end of this automation run, plus every selected Pattern that changed from active to inactive during this run (a newly inactive outcome). Exclude Patterns that were already inactive before this run and remained inactive. Do not mention excluded Patterns anywhere in the digest, including Highlights. If no selected Patterns meet these criteria, say \"No current pattern outcomes.\" This requirement overrides conflicting formatting guidance.\n\nInclude a complete \"*Pattern outcomes*\" section with one row for every included Pattern.\n\nUse this row format exactly:\n• <pattern_url|Pattern title> — `outcome`\n\nIf a Pattern has no URL, use the plain title instead. Do not use GitHub Markdown tables or code-block tables, because links must remain clickable. Do not omit any active selected Pattern.\n\nAfter the outcome list, include a \"*Highlights*\" section with one very short paragraph, 2-3 sentences maximum. Summarize what changed or what broadly stands out among the included active Patterns. Do not introduce new claims beyond the run report."
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"routing_key_secret_name",
"severity"
],
"properties": {
"type": {
"type": "string",
"const": "pagerduty",
"description": "The type of action to take"
},
"severity": {
"enum": [
"critical",
"error",
"warning",
"info"
],
"type": "string"
},
"formatting_prompt": {
"$ref": "#/properties/config/properties/action/anyOf/0/properties/formatting_prompt"
},
"routing_key_secret_name": {
"type": "string",
"format": "uuid",
"description": "The data-plane secret containing the PagerDuty routing key"
}
},
"additionalProperties": false
}
],
"description": "The action to take when the automation rule is triggered"
},
"status": {
"enum": [
"active",
"paused"
],
"type": "string",
"description": "Whether the automation is active or paused."
},
"event_type": {
"type": "string",
"const": "environment_update",
"description": "The type of automation."
},
"environment_filter": {
"type": "array",
"items": {
"type": "string"
},
"description": "Optional list of environment slugs to filter by"
}
},
"description": "The complete environment-update alert configuration.",
"additionalProperties": false
},
"if_exists": {
"enum": [
"error",
"ignore",
"replace"
],
"type": "string",
"default": "error",
"description": "What to do when an automation with the same project and name exists: error, replace it, or ignore this write."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"description": {
"type": "string"
}
},
"additionalProperties": false
},
"annotations": {
"title": "Create environment update alert",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description76/100
Create a threshold alert for an aggregate over a recent project-data window. Use this for averages, counts, rates, percentages, percentiles, distributions, or other conditions evaluated on a schedule.
config.threshold.calculation.btql_query must return at most one row with the numeric or null column named by config.threshold.calculation.output.value_column. Put any minimum sample size in the query, and do not add absolute time bounds because the runtime supplies the evaluation window. Use config.window.evaluation_delay_seconds when ingestion or asynchronous scoring arrives late. config.threshold.policy.pending_seconds controls how long a breach must persist, while renotify_interval_seconds controls reminders.
Explain mutations, irreversible effects, or prerequisites that matter before calling this tool.
Inputs68/100
34 of 45 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
name
The automation name.
Explain the meaning of this value, not just its name.
description
Not provided
Missing description. Explain what this value means and how to supply it.
if_exists
What to do when an automation with the same project and name exists: error, replace it, or ignore this write.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › event_type
The type of automation.
Explain the meaning of this value, not just its name.
config › product_origin
The product surface that created and manages the automation
Explain what happens when this optional value is omitted.
Publish a Slack mrkdwn digest.
Pattern selection requirement: include every selected Pattern whose final status is active at the end of this automation run, plus every selected Pattern that changed from active to inactive during this run (a newly inactive outcome). Exclude Patterns that were already inactive before this run and remained inactive. Do not mention excluded Patterns anywhere in the digest, including Highlights. If no selected Patterns meet these criteria, say "No current pattern outcomes." This requirement overrides conflicting formatting guidance.
Include a complete "*Pattern outcomes*" section with one row for every included Pattern.
Use this row format exactly:
• <pattern_url|Pattern title> — `outcome`
If a Pattern has no URL, use the plain title instead. Do not use GitHub Markdown tables or code-block tables, because links must remain clickable. Do not omit any active selected Pattern.
After the outcome list, include a "*Highlights*" section with one very short paragraph, 2-3 sentences maximum. Summarize what changed or what broadly stands out among the included active Patterns. Do not introduce new claims beyond the run report.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "create_threshold_alert",
"description": "Create a threshold alert for an aggregate over a recent project-data window. Use this for averages, counts, rates, percentages, percentiles, distributions, or other conditions evaluated on a schedule.\n\nconfig.threshold.calculation.btql_query must return at most one row with the numeric or null column named by config.threshold.calculation.output.value_column. Put any minimum sample size in the query, and do not add absolute time bounds because the runtime supplies the evaluation window. Use config.window.evaluation_delay_seconds when ingestion or asynchronous scoring arrives late. config.threshold.policy.pending_seconds controls how long a breach must persist, while renotify_interval_seconds controls reminders.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"name",
"config"
],
"properties": {
"name": {
"type": "string",
"minLength": 1,
"description": "The automation name."
},
"config": {
"type": "object",
"required": [
"event_type",
"window"
],
"properties": {
"loop": {
"type": "object",
"required": [
"prompt",
"agent_slug"
],
"properties": {
"model": {
"type": "string",
"minLength": 1,
"description": "Required when saving a Loop automation."
},
"prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Instructions for the Loop agent"
},
"harness": {
"enum": [
"native",
"codex",
"claude-code",
"pi"
],
"type": "string",
"description": "Required when saving a Loop automation. Currently only Codex is supported."
},
"agent_slug": {
"type": "string",
"minLength": 1,
"description": "The Loop agent to run"
},
"endpoint_name": {
"type": "string",
"minLength": 1
},
"reasoning_effort": {
"enum": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh",
"max"
],
"type": "string"
},
"auto_approve_tools": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
},
"default": [],
"description": "Write tools that may run without interactive approval"
},
"include_trigger_input": {
"type": "boolean",
"default": false,
"description": "Whether to include the automation trigger payload as input"
}
},
"description": "Optional Loop agent to run for each triggered window",
"additionalProperties": false
},
"status": {
"enum": [
"active",
"paused"
],
"type": "string",
"description": "Whether the automation is active or paused."
},
"window": {
"type": "object",
"required": [
"window_seconds",
"schedule",
"evaluation_delay_seconds"
],
"properties": {
"schedule": {
"anyOf": [
{
"type": "object",
"required": [
"type",
"evaluation_interval_seconds"
],
"properties": {
"type": {
"type": "string",
"const": "interval"
},
"evaluation_interval_seconds": {
"type": "integer",
"maximum": 2592000,
"minimum": 1,
"description": "How often the automation runs"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"cron_expression"
],
"properties": {
"type": {
"type": "string",
"const": "cron"
},
"timezone": {
"anyOf": [
{
"type": "string",
"minLength": 1
},
{
"type": "null"
}
],
"description": "IANA timezone used to interpret the cron expression (defaults to UTC)"
},
"cron_expression": {
"type": "string",
"minLength": 1,
"description": "A standard five-field cron expression (minute hour day-of-month month day-of-week) controlling when the automation runs"
}
},
"additionalProperties": false
}
],
"description": "How often the windowed automation runs: at a fixed interval or on a cron schedule"
},
"window_seconds": {
"type": "integer",
"maximum": 2592000,
"minimum": 1,
"description": "How much recent data each scheduled run covers"
},
"evaluation_delay_seconds": {
"type": "integer",
"maximum": 2592000,
"minimum": 0,
"description": "How far behind the present each evaluation window ends"
}
},
"additionalProperties": false
},
"actions": {
"type": "array",
"items": {
"anyOf": [
{
"type": "object",
"required": [
"type",
"url"
],
"properties": {
"url": {
"type": "string",
"description": "The webhook URL to send the request to"
},
"type": {
"type": "string",
"const": "webhook",
"description": "The type of action to take"
},
"formatting_prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Instructions for Loop to format content sent to this destination"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"workspace_id",
"channel"
],
"properties": {
"type": {
"type": "string",
"const": "slack",
"description": "The type of action to take"
},
"channel": {
"type": "string",
"description": "The Slack channel ID to post to"
},
"workspace_id": {
"type": "string",
"description": "The Slack workspace ID to post to"
},
"message_template": {
"type": "string",
"description": "Custom message template for the alert"
},
"formatting_prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Publish a Slack mrkdwn digest.\n\nPattern selection requirement: include every selected Pattern whose final status is active at the end of this automation run, plus every selected Pattern that changed from active to inactive during this run (a newly inactive outcome). Exclude Patterns that were already inactive before this run and remained inactive. Do not mention excluded Patterns anywhere in the digest, including Highlights. If no selected Patterns meet these criteria, say \"No current pattern outcomes.\" This requirement overrides conflicting formatting guidance.\n\nInclude a complete \"*Pattern outcomes*\" section with one row for every included Pattern.\n\nUse this row format exactly:\n• <pattern_url|Pattern title> — `outcome`\n\nIf a Pattern has no URL, use the plain title instead. Do not use GitHub Markdown tables or code-block tables, because links must remain clickable. Do not omit any active selected Pattern.\n\nAfter the outcome list, include a \"*Highlights*\" section with one very short paragraph, 2-3 sentences maximum. Summarize what changed or what broadly stands out among the included active Patterns. Do not introduce new claims beyond the run report."
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"routing_key_secret_name",
"severity"
],
"properties": {
"type": {
"type": "string",
"const": "pagerduty",
"description": "The type of action to take"
},
"severity": {
"enum": [
"critical",
"error",
"warning",
"info"
],
"type": "string"
},
"formatting_prompt": {
"$ref": "#/properties/config/properties/actions/items/anyOf/0/properties/formatting_prompt"
},
"routing_key_secret_name": {
"type": "string",
"format": "uuid",
"description": "The data-plane secret containing the PagerDuty routing key"
}
},
"additionalProperties": false
}
],
"description": "A delivery action available to an automation"
},
"default": [],
"maxItems": 20,
"description": "Delivery actions exposed to Loop as tools, or run directly when Loop is not configured"
},
"threshold": {
"type": "object",
"required": [
"calculation",
"policy"
],
"properties": {
"policy": {
"type": "object",
"required": [
"condition",
"pending_seconds",
"no_data_behavior"
],
"properties": {
"condition": {
"type": "object",
"required": [
"type",
"operator",
"threshold"
],
"properties": {
"type": {
"type": "string",
"const": "threshold"
},
"operator": {
"enum": [
"lt",
"lte",
"gt",
"gte",
"eq",
"neq"
],
"type": "string"
},
"threshold": {
"type": "number"
}
},
"additionalProperties": false
},
"pending_seconds": {
"type": "integer",
"maximum": 2592000,
"minimum": 0,
"description": "How long the condition must remain breached before firing"
},
"no_data_behavior": {
"enum": [
"keep_last",
"resolve",
"alert"
],
"type": "string",
"description": "How the lifecycle changes when the calculation returns no data"
},
"notify_on_recovery": {
"type": "boolean",
"default": true,
"description": "Whether to deliver actions when a firing automation recovers"
},
"renotify_interval_seconds": {
"anyOf": [
{
"type": "integer",
"maximum": 2592000,
"minimum": 1
},
{
"type": "null"
}
],
"description": "Optional reminder interval while the automation is firing"
}
},
"description": "The lifecycle policy applied to each calculation result",
"additionalProperties": false
},
"calculation": {
"anyOf": [
{
"type": "object",
"required": [
"type",
"btql_query",
"output"
],
"properties": {
"type": {
"type": "string",
"const": "btql"
},
"output": {
"type": "object",
"required": [
"type",
"value_column"
],
"properties": {
"type": {
"type": "string",
"const": "scalar"
},
"value_column": {
"type": "string",
"minLength": 1,
"description": "The numeric result column produced by the query"
}
},
"additionalProperties": false
},
"btql_query": {
"type": "string",
"minLength": 1,
"description": "A project-scoped BTQL or SQL query without runtime-owned evaluation time bounds"
}
},
"additionalProperties": false
}
],
"description": "The calculation evaluated for each window"
}
},
"description": "Optional calculation and lifecycle policy that gate scheduled delivery",
"additionalProperties": false
},
"event_type": {
"type": "string",
"const": "windowed",
"description": "The type of automation."
},
"product_origin": {
"anyOf": [
{
"enum": [
"patterns"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "The product surface that created and manages the automation"
}
},
"description": "The complete threshold alert configuration. It must include a threshold.",
"additionalProperties": false
},
"if_exists": {
"enum": [
"error",
"ignore",
"replace"
],
"type": "string",
"default": "error",
"description": "What to do when an automation with the same project and name exists: error, replace it, or ignore this write."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"description": {
"type": "string"
}
},
"additionalProperties": false
},
"annotations": {
"title": "Create threshold alert",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description73/100
Create a custom Loop job that runs on an interval or cron schedule over a recent project-data window. The config must include Loop instructions and cannot include a threshold, so the job runs for every scheduled window. Delivery actions are exposed to Loop as tools. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.
Resolve contradictions between the description, input schema, and annotations.
Inputs58/100
34 of 45 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
name
The automation name.
Explain the meaning of this value, not just its name.
description
Not provided
Missing description. Explain what this value means and how to supply it.
if_exists
What to do when an automation with the same project and name exists: error, replace it, or ignore this write.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
config › event_type
The type of automation.
Explain the meaning of this value, not just its name.
config › product_origin
The product surface that created and manages the automation
Explain what happens when this optional value is omitted.
config › status
Whether the automation is active or paused.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
Publish a Slack mrkdwn digest.
Pattern selection requirement: include every selected Pattern whose final status is active at the end of this automation run, plus every selected Pattern that changed from active to inactive during this run (a newly inactive outcome). Exclude Patterns that were already inactive before this run and remained inactive. Do not mention excluded Patterns anywhere in the digest, including Highlights. If no selected Patterns meet these criteria, say "No current pattern outcomes." This requirement overrides conflicting formatting guidance.
Include a complete "*Pattern outcomes*" section with one row for every included Pattern.
Use this row format exactly:
• <pattern_url|Pattern title> — `outcome`
If a Pattern has no URL, use the plain title instead. Do not use GitHub Markdown tables or code-block tables, because links must remain clickable. Do not omit any active selected Pattern.
After the outcome list, include a "*Highlights*" section with one very short paragraph, 2-3 sentences maximum. Summarize what changed or what broadly stands out among the included active Patterns. Do not introduce new claims beyond the run report.
Clear enough to supply this argument.
config › actions › items › anyOf › 2 › type
The type of action to take
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "create_scheduled_loop_job",
"description": "Create a custom Loop job that runs on an interval or cron schedule over a recent project-data window. The config must include Loop instructions and cannot include a threshold, so the job runs for every scheduled window. Delivery actions are exposed to Loop as tools. In hosted Agent Runtime, call braintrust_context to obtain the current project ID.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"name",
"config"
],
"properties": {
"name": {
"type": "string",
"minLength": 1,
"description": "The automation name."
},
"config": {
"type": "object",
"required": [
"event_type",
"window"
],
"properties": {
"loop": {
"type": "object",
"required": [
"prompt",
"agent_slug"
],
"properties": {
"model": {
"type": "string",
"minLength": 1,
"description": "Required when saving a Loop automation."
},
"prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Instructions for the Loop agent"
},
"harness": {
"enum": [
"native",
"codex",
"claude-code",
"pi"
],
"type": "string",
"description": "Required when saving a Loop automation. Currently only Codex is supported."
},
"agent_slug": {
"type": "string",
"minLength": 1,
"description": "The Loop agent to run"
},
"endpoint_name": {
"type": "string",
"minLength": 1
},
"reasoning_effort": {
"enum": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh",
"max"
],
"type": "string"
},
"auto_approve_tools": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
},
"default": [],
"description": "Write tools that may run without interactive approval"
},
"include_trigger_input": {
"type": "boolean",
"default": false,
"description": "Whether to include the automation trigger payload as input"
}
},
"description": "Optional Loop agent to run for each triggered window",
"additionalProperties": false
},
"status": {
"enum": [
"active",
"paused"
],
"type": "string",
"description": "Whether the automation is active or paused."
},
"window": {
"type": "object",
"required": [
"window_seconds",
"schedule",
"evaluation_delay_seconds"
],
"properties": {
"schedule": {
"anyOf": [
{
"type": "object",
"required": [
"type",
"evaluation_interval_seconds"
],
"properties": {
"type": {
"type": "string",
"const": "interval"
},
"evaluation_interval_seconds": {
"type": "integer",
"maximum": 2592000,
"minimum": 1,
"description": "How often the automation runs"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"cron_expression"
],
"properties": {
"type": {
"type": "string",
"const": "cron"
},
"timezone": {
"anyOf": [
{
"type": "string",
"minLength": 1
},
{
"type": "null"
}
],
"description": "IANA timezone used to interpret the cron expression (defaults to UTC)"
},
"cron_expression": {
"type": "string",
"minLength": 1,
"description": "A standard five-field cron expression (minute hour day-of-month month day-of-week) controlling when the automation runs"
}
},
"additionalProperties": false
}
],
"description": "How often the windowed automation runs: at a fixed interval or on a cron schedule"
},
"window_seconds": {
"type": "integer",
"maximum": 2592000,
"minimum": 1,
"description": "How much recent data each scheduled run covers"
},
"evaluation_delay_seconds": {
"type": "integer",
"maximum": 2592000,
"minimum": 0,
"description": "How far behind the present each evaluation window ends"
}
},
"additionalProperties": false
},
"actions": {
"type": "array",
"items": {
"anyOf": [
{
"type": "object",
"required": [
"type",
"url"
],
"properties": {
"url": {
"type": "string",
"description": "The webhook URL to send the request to"
},
"type": {
"type": "string",
"const": "webhook",
"description": "The type of action to take"
},
"formatting_prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Instructions for Loop to format content sent to this destination"
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"workspace_id",
"channel"
],
"properties": {
"type": {
"type": "string",
"const": "slack",
"description": "The type of action to take"
},
"channel": {
"type": "string",
"description": "The Slack channel ID to post to"
},
"workspace_id": {
"type": "string",
"description": "The Slack workspace ID to post to"
},
"message_template": {
"type": "string",
"description": "Custom message template for the alert"
},
"formatting_prompt": {
"type": "string",
"maxLength": 10000,
"minLength": 1,
"description": "Publish a Slack mrkdwn digest.\n\nPattern selection requirement: include every selected Pattern whose final status is active at the end of this automation run, plus every selected Pattern that changed from active to inactive during this run (a newly inactive outcome). Exclude Patterns that were already inactive before this run and remained inactive. Do not mention excluded Patterns anywhere in the digest, including Highlights. If no selected Patterns meet these criteria, say \"No current pattern outcomes.\" This requirement overrides conflicting formatting guidance.\n\nInclude a complete \"*Pattern outcomes*\" section with one row for every included Pattern.\n\nUse this row format exactly:\n• <pattern_url|Pattern title> — `outcome`\n\nIf a Pattern has no URL, use the plain title instead. Do not use GitHub Markdown tables or code-block tables, because links must remain clickable. Do not omit any active selected Pattern.\n\nAfter the outcome list, include a \"*Highlights*\" section with one very short paragraph, 2-3 sentences maximum. Summarize what changed or what broadly stands out among the included active Patterns. Do not introduce new claims beyond the run report."
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"type",
"routing_key_secret_name",
"severity"
],
"properties": {
"type": {
"type": "string",
"const": "pagerduty",
"description": "The type of action to take"
},
"severity": {
"enum": [
"critical",
"error",
"warning",
"info"
],
"type": "string"
},
"formatting_prompt": {
"$ref": "#/properties/config/properties/actions/items/anyOf/0/properties/formatting_prompt"
},
"routing_key_secret_name": {
"type": "string",
"format": "uuid",
"description": "The data-plane secret containing the PagerDuty routing key"
}
},
"additionalProperties": false
}
],
"description": "A delivery action available to an automation"
},
"default": [],
"maxItems": 20,
"description": "Delivery actions exposed to Loop as tools, or run directly when Loop is not configured"
},
"threshold": {
"type": "object",
"required": [
"calculation",
"policy"
],
"properties": {
"policy": {
"type": "object",
"required": [
"condition",
"pending_seconds",
"no_data_behavior"
],
"properties": {
"condition": {
"type": "object",
"required": [
"type",
"operator",
"threshold"
],
"properties": {
"type": {
"type": "string",
"const": "threshold"
},
"operator": {
"enum": [
"lt",
"lte",
"gt",
"gte",
"eq",
"neq"
],
"type": "string"
},
"threshold": {
"type": "number"
}
},
"additionalProperties": false
},
"pending_seconds": {
"type": "integer",
"maximum": 2592000,
"minimum": 0,
"description": "How long the condition must remain breached before firing"
},
"no_data_behavior": {
"enum": [
"keep_last",
"resolve",
"alert"
],
"type": "string",
"description": "How the lifecycle changes when the calculation returns no data"
},
"notify_on_recovery": {
"type": "boolean",
"default": true,
"description": "Whether to deliver actions when a firing automation recovers"
},
"renotify_interval_seconds": {
"anyOf": [
{
"type": "integer",
"maximum": 2592000,
"minimum": 1
},
{
"type": "null"
}
],
"description": "Optional reminder interval while the automation is firing"
}
},
"description": "The lifecycle policy applied to each calculation result",
"additionalProperties": false
},
"calculation": {
"anyOf": [
{
"type": "object",
"required": [
"type",
"btql_query",
"output"
],
"properties": {
"type": {
"type": "string",
"const": "btql"
},
"output": {
"type": "object",
"required": [
"type",
"value_column"
],
"properties": {
"type": {
"type": "string",
"const": "scalar"
},
"value_column": {
"type": "string",
"minLength": 1,
"description": "The numeric result column produced by the query"
}
},
"additionalProperties": false
},
"btql_query": {
"type": "string",
"minLength": 1,
"description": "A project-scoped BTQL or SQL query without runtime-owned evaluation time bounds"
}
},
"additionalProperties": false
}
],
"description": "The calculation evaluated for each window"
}
},
"description": "Optional calculation and lifecycle policy that gate scheduled delivery",
"additionalProperties": false
},
"event_type": {
"type": "string",
"const": "windowed",
"description": "The type of automation."
},
"product_origin": {
"anyOf": [
{
"enum": [
"patterns"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "The product surface that created and manages the automation"
}
},
"description": "The complete scheduled Loop job configuration. It must include Loop work and must not include a threshold.",
"additionalProperties": false
},
"if_exists": {
"enum": [
"error",
"ignore",
"replace"
],
"type": "string",
"default": "error",
"description": "What to do when an automation with the same project and name exists: error, replace it, or ignore this write."
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "Required project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"description": {
"type": "string"
}
},
"additionalProperties": false
},
"annotations": {
"title": "Create scheduled Loop job",
"readOnlyHint": false,
"openWorldHint": false,
"idempotentHint": false,
"destructiveHint": true
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description99/100
Search Braintrust documentation to find relevant information and guidance.
IMPORTANT: For SDK installation/setup tasks, do not use this tool. Instead, read the mcp resource docs://sdk-install
This tool searches through the official Braintrust documentation using semantic search to find the most relevant content for your query.
Use this tool to:
- Learn more about what experiments, datasets, prompts, playgrounds, and logs are, and how to use them in the API or SDK.
- Understand how to write evals and create scorers
- Find sample code and guidance on how to add tracing for custom code as well as popular frameworks like the Vercel AI SDK.
- Find code examples that are relevant to your use cases.
What it returns:
- Multiple documentation snippets ranked by relevance
- Each snippet includes the content, headers hierarchy, and path
- Content is Markdown formatted
Example queries:
- "How do I create an experiment?"
- "Setting up scorers"
- "Using metadata fields"
- "Prompt playground features"
The description explains the capability, when to use it, and what to expect.
Inputs88/100
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
query
A search string to find relevant documentation. The string will be embedded and then used to search for relevant snippets in a vector database.
Clear enough to supply this argument.
top_k
The number of top relevant document snippets to return. Higher values return more results but may include less relevant snippets.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "search_docs",
"description": "Search Braintrust documentation to find relevant information and guidance.\n\n IMPORTANT: For SDK installation/setup tasks, do not use this tool. Instead, read the mcp resource docs://sdk-install\n\nThis tool searches through the official Braintrust documentation using semantic search to find the most relevant content for your query.\n\nUse this tool to:\n- Learn more about what experiments, datasets, prompts, playgrounds, and logs are, and how to use them in the API or SDK.\n- Understand how to write evals and create scorers\n- Find sample code and guidance on how to add tracing for custom code as well as popular frameworks like the Vercel AI SDK.\n- Find code examples that are relevant to your use cases.\n\nWhat it returns:\n- Multiple documentation snippets ranked by relevance\n- Each snippet includes the content, headers hierarchy, and path\n- Content is Markdown formatted\n\nExample queries:\n- \"How do I create an experiment?\"\n- \"Setting up scorers\"\n- \"Using metadata fields\"\n- \"Prompt playground features\"",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"query"
],
"properties": {
"query": {
"type": "string",
"description": "A search string to find relevant documentation. The string will be embedded and then used to search for relevant snippets in a vector database."
},
"top_k": {
"type": "integer",
"description": "The number of top relevant document snippets to return. Higher values return more results but may include less relevant snippets.",
"exclusiveMinimum": 0
}
},
"additionalProperties": false
},
"annotations": {
"title": "Search documentation",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description82/100
Look up human Braintrust organization members by exact user IDs, partial name or email, or an explicit request to list all users. Provide exactly one of user_ids, search, or list_all: true.
Use this to resolve span_attributes.created_by_user_id and Braintrust reviewer or assignee IDs. Do not interpret arbitrary metadata.user_id values as Braintrust organization users. When resolving query results, deduplicate and pass all user IDs in one call rather than making one call per ID. If a name search returns multiple possible users, ask the user to disambiguate before continuing.
The description explains the capability, when to use it, and what to expect.
Inputs88/100
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
project_id
Project ID used to authorize the lookup and determine the organization. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
user_ids
Braintrust organization user IDs to resolve. Deduplicate and pass every needed ID in one call instead of calling once per ID.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
search
Case-insensitive partial match across given name, family name, full name, and email. Multiple matches are returned for disambiguation.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
list_all
Explicitly return every human user in the organization.
{
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"project_id",
"organization_id",
"users_by_id",
"unresolved_user_ids",
"returned_count"
],
"properties": {
"project_id": {
"type": "string",
"format": "uuid",
"description": "Project ID used to authorize the lookup and determine the organization. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"users_by_id": {
"type": "object",
"additionalProperties": {
"type": "object",
"required": [
"display_name"
],
"properties": {
"email": {
"type": [
"string",
"null"
],
"description": "The user's email"
},
"given_name": {
"type": [
"string",
"null"
],
"description": "Given name of the user"
},
"family_name": {
"type": [
"string",
"null"
],
"description": "Family name of the user"
},
"display_name": {
"type": "string"
}
},
"additionalProperties": false
}
},
"returned_count": {
"type": "integer",
"minimum": 0
},
"organization_id": {
"type": "string",
"format": "uuid"
},
"unresolved_user_ids": {
"type": "array",
"items": {
"type": "string",
"format": "uuid",
"description": "Unique identifier for the user"
}
}
},
"additionalProperties": false
}
Original tool metadata
{
"name": "lookup_users",
"description": "Look up human Braintrust organization members by exact user IDs, partial name or email, or an explicit request to list all users. Provide exactly one of user_ids, search, or list_all: true.\n\nUse this to resolve span_attributes.created_by_user_id and Braintrust reviewer or assignee IDs. Do not interpret arbitrary metadata.user_id values as Braintrust organization users. When resolving query results, deduplicate and pass all user IDs in one call rather than making one call per ID. If a name search returns multiple possible users, ask the user to disambiguate before continuing.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"project_id"
],
"properties": {
"search": {
"type": "string",
"minLength": 1,
"description": "Case-insensitive partial match across given name, family name, full name, and email. Multiple matches are returned for disambiguation."
},
"list_all": {
"type": "boolean",
"const": true,
"description": "Explicitly return every human user in the organization."
},
"user_ids": {
"type": "array",
"items": {
"type": "string",
"format": "uuid",
"description": "Unique identifier for the user"
},
"maxItems": 1000,
"minItems": 1,
"description": "Braintrust organization user IDs to resolve. Deduplicate and pass every needed ID in one call instead of calling once per ID."
},
"project_id": {
"type": "string",
"format": "uuid",
"description": "Project ID used to authorize the lookup and determine the organization. In hosted Agent Runtime, call braintrust_context to obtain it."
}
},
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"project_id",
"organization_id",
"users_by_id",
"unresolved_user_ids",
"returned_count"
],
"properties": {
"project_id": {
"type": "string",
"format": "uuid",
"description": "Project ID used to authorize the lookup and determine the organization. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"users_by_id": {
"type": "object",
"additionalProperties": {
"type": "object",
"required": [
"display_name"
],
"properties": {
"email": {
"type": [
"string",
"null"
],
"description": "The user's email"
},
"given_name": {
"type": [
"string",
"null"
],
"description": "Given name of the user"
},
"family_name": {
"type": [
"string",
"null"
],
"description": "Family name of the user"
},
"display_name": {
"type": "string"
}
},
"additionalProperties": false
}
},
"returned_count": {
"type": "integer",
"minimum": 0
},
"organization_id": {
"type": "string",
"format": "uuid"
},
"unresolved_user_ids": {
"type": "array",
"items": {
"type": "string",
"format": "uuid",
"description": "Unique identifier for the user"
}
}
},
"additionalProperties": false
},
"annotations": {
"title": "Look up organization users",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description95/100
Look up Braintrust API keys and service tokens by exact IDs, exact name, or an explicit request to list all visible keys. Provide exactly one of api_key_ids, name, or list_all: true. Results follow the existing API-key and service-token visibility rules; inaccessible IDs remain unresolved.
Use this to resolve span_attributes.created_by_api_key_id, which may identify either a personal API key or a service token. When resolving query results, deduplicate and pass all IDs in one call rather than making one call per ID. If an exact name lookup returns multiple possible keys, ask the user to disambiguate before continuing.
The description explains the capability, when to use it, and what to expect.
Inputs94/100
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
project_id
Project ID used to authorize the lookup and determine the organization. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
api_key_ids
Braintrust API key or service token IDs to resolve. Deduplicate and pass every needed ID in one call instead of calling once per ID.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
name
Exact API key or service token name. Multiple matches are returned for disambiguation.
Clear enough to supply this argument.
list_all
Explicitly return every API key and service token visible in the organization.
Clear enough to supply this argument.
Annotations100/100
All four behavior hints are declared.
{
"title": "Look up API keys",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
Output schema73/100
Describe the returned fields so an agent can interpret the result.
{
"name": "lookup_api_keys",
"description": "Look up Braintrust API keys and service tokens by exact IDs, exact name, or an explicit request to list all visible keys. Provide exactly one of api_key_ids, name, or list_all: true. Results follow the existing API-key and service-token visibility rules; inaccessible IDs remain unresolved.\n\nUse this to resolve span_attributes.created_by_api_key_id, which may identify either a personal API key or a service token. When resolving query results, deduplicate and pass all IDs in one call rather than making one call per ID. If an exact name lookup returns multiple possible keys, ask the user to disambiguate before continuing.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"project_id"
],
"properties": {
"name": {
"type": "string",
"minLength": 1,
"description": "Exact API key or service token name. Multiple matches are returned for disambiguation."
},
"list_all": {
"type": "boolean",
"const": true,
"description": "Explicitly return every API key and service token visible in the organization."
},
"project_id": {
"type": "string",
"format": "uuid",
"description": "Project ID used to authorize the lookup and determine the organization. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"api_key_ids": {
"type": "array",
"items": {
"type": "string",
"format": "uuid",
"description": "Unique identifier for the api key"
},
"maxItems": 1000,
"minItems": 1,
"description": "Braintrust API key or service token IDs to resolve. Deduplicate and pass every needed ID in one call instead of calling once per ID."
}
},
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"project_id",
"organization_id",
"api_keys_by_id",
"unresolved_api_key_ids",
"returned_count"
],
"properties": {
"project_id": {
"type": "string",
"format": "uuid",
"description": "Project ID used to authorize the lookup and determine the organization. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"api_keys_by_id": {
"type": "object",
"additionalProperties": {
"type": "object",
"required": [
"name",
"type"
],
"properties": {
"name": {
"type": "string"
},
"type": {
"enum": [
"api_key",
"service_token"
],
"type": "string"
}
},
"additionalProperties": false
}
},
"returned_count": {
"type": "integer",
"minimum": 0
},
"organization_id": {
"type": "string",
"format": "uuid"
},
"unresolved_api_key_ids": {
"type": "array",
"items": {
"type": "string",
"format": "uuid",
"description": "Unique identifier for the api key"
}
}
},
"additionalProperties": false
},
"annotations": {
"title": "Look up API keys",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description78/100
Convert names to IDs (or vice versa) and parse Braintrust URLs. App URLs retain their organization scope. Pattern, prompt, and scorer links return the containing project as object_id and the resource as row_id; query that project collection with an id filter. Read docs://url-formats for supported URL patterns.
Describe the returned information so the agent knows whether it will answer the request.
Inputs83/100
Resolve the mismatch between this description and its schema constraints.
url
A Braintrust URL to parse. Supports both app links (/app/{org}/p/{project}/{type}/{name}) and legacy object URLs (/app/object?object_type=...&object_id=...). The tool will extract object_type, object_id, project_name, object_name, and root_span_id (from the 'r=' parameter) as applicable.
Clear enough to supply this argument.
object_type
The type of the object. If you specify project_logs, then you do not need to specify an object_name, as the tool will return the logs for the specified project. If you specify a project_name with object_type='logs', the object_id will be the project's id.
Resolve the mismatch between this description and its schema constraints.
project_name
The name of the project. If specified together with an object_name, the resolver will look for an object with that name within the specified project.
Clear enough to supply this argument.
org_name
Organization name used to disambiguate project and object names. Extracted automatically from app URLs.
Clear enough to supply this argument.
object_name
The name of the object. Specify this together with a project_name to look for an object within a specific project and return its id.
Clear enough to supply this argument.
object_id
The id of the object. If you specify this, you do not need to specify a project_name or object_name.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "resolve_object",
"description": "Convert names to IDs (or vice versa) and parse Braintrust URLs. App URLs retain their organization scope. Pattern, prompt, and scorer links return the containing project as object_id and the resource as row_id; query that project collection with an id filter. Read docs://url-formats for supported URL patterns.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"url": {
"type": "string",
"description": "A Braintrust URL to parse. Supports both app links (/app/{org}/p/{project}/{type}/{name}) and legacy object URLs (/app/object?object_type=...&object_id=...). The tool will extract object_type, object_id, project_name, object_name, and root_span_id (from the 'r=' parameter) as applicable."
},
"org_name": {
"type": "string",
"description": "Organization name used to disambiguate project and object names. Extracted automatically from app URLs."
},
"object_id": {
"type": "string",
"description": "The id of the object. If you specify this, you do not need to specify a project_name or object_name."
},
"object_name": {
"type": "string",
"description": "The name of the object. Specify this together with a project_name to look for an object within a specific project and return its id."
},
"object_type": {
"enum": [
"experiment",
"dataset",
"prompt_session",
"playground_logs",
"project_logs",
"project_prompts",
"project_functions",
"project_patterns",
"audit_logs",
"org_prompts",
"org_functions",
"org_project_metadata",
"project"
],
"type": "string",
"description": "The type of the object. If you specify project_logs, then you do not need to specify an object_name, as the tool will return the logs for the specified project. If you specify a project_name with object_type='logs', the object_id will be the project's id."
},
"project_name": {
"type": "string",
"description": "The name of the project. If specified together with an object_name, the resolver will look for an object with that name within the specified project."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Resolve object",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description77/100
List recently created projects, experiments, datasets, prompts, or functions you have access to. You can use this to browse for objects
and as a crude way of searching for objects (you will need to filter the results yourself).
If an object you expect does not show up, then use the `startin_after` parameter to paginate through more results.
If a user asks for something like "list my recent experiments", then you do not need to list through the set of projects. You can just provide
the project name, and then list the experiments within that project. If you get an error, then perhaps the project name is incorrect, and you
should fall back to listing projects first to find the one the user is referring to.
Describe the returned information so the agent knows whether it will answer the request.
Inputs84/100
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
object_type
The type of the object.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
project_name
Experiments, datasets, prompts, functions, prompt sessions live in projects, so you must specify a project_name to list those object types.
Clear enough to supply this argument.
limit
The maximum number of objects to return. Defaults to 10 if not specified.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
starting_after
For pagination, provide the last id from the previous result set to get the next page.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "list_recent_objects",
"description": "List recently created projects, experiments, datasets, prompts, or functions you have access to. You can use this to browse for objects\nand as a crude way of searching for objects (you will need to filter the results yourself).\n\nIf an object you expect does not show up, then use the `startin_after` parameter to paginate through more results.\n\nIf a user asks for something like \"list my recent experiments\", then you do not need to list through the set of projects. You can just provide\nthe project name, and then list the experiments within that project. If you get an error, then perhaps the project name is incorrect, and you\nshould fall back to listing projects first to find the one the user is referring to.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"object_type"
],
"properties": {
"limit": {
"type": "integer",
"default": 10,
"maximum": 100,
"description": "The maximum number of objects to return. Defaults to 10 if not specified.",
"exclusiveMinimum": 0
},
"object_type": {
"enum": [
"organization",
"project",
"experiment",
"dataset",
"prompt",
"prompt_session",
"group",
"role",
"project_group",
"project_group_projects",
"ai_secret",
"org_account"
],
"type": "string",
"description": "The type of the object."
},
"project_name": {
"type": "string",
"description": "Experiments, datasets, prompts, functions, prompt sessions live in projects, so you must specify a project_name to list those object types."
},
"starting_after": {
"type": "string",
"description": "For pagination, provide the last id from the previous result set to get the next page."
}
},
"additionalProperties": false
},
"annotations": {
"title": "List recent objects",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description88/100
Analyze the structure and content of a data source by inferring the schema and showing the most common values for each field.
This tool discovers:
- All available fields in the data source (input, output, metadata.*, scores.*, etc.)
- The data type of each field (string, number, array, object)
- The most frequently occurring values for each field with their counts
- The complete field paths for use in SQL queries
For each field, you'll see:
- The full path as an array (e.g., ['metadata', 'user_id'] which translates to metadata.user_id in SQL)
- The data type (string, number, boolean, etc.)
- Top 10 most common values sorted by frequency with exact counts
Large results are truncated to fit the context window: sample values may be shortened and capped at 3 per field, and if there are still too many fields, data contains a subset and totalFields reports the full field count. When totalFields is present, use sql_query to explore fields not shown here.
This is essential for:
- Discovering what fields exist before writing queries
- Understanding the actual values in your data (e.g., what user IDs exist, what models are used)
- Finding the exact field paths for SQL queries
- Identifying patterns in your data (e.g., most common inputs, frequent error types)
- Understanding cardinality (how many unique values exist)
Example insights from the tool:
- See all metadata fields like metadata.user_id, metadata.model, metadata.session_id
- Discover custom fields you may not know about
- Find the most active users or most used models
- Understand what values to filter on in your queries
Time range filter guidance:
- All queries MUST have a time range filter.
- If you do not provide a time range filter the request will fail.
- If the user does not specify a time range, always use the default time range filter provided in the prompt.
- If the user specifies a time range in the prompt, always use the time range they specified.
- If default time range is missing and user does not specify a time range, always use the last 3 days.
- Unless the user specifies a specific from and to hour, always provide 06:00:00.000Z as the from hour and 05:59:59.999Z as the to hour to go along with the year, month, and day.
- When generating a custom time range for a relative date request, derive the year, month, and day from the <current_date> tag or user's specified date rather than from examples.
The description explains the capability, when to use it, and what to expect.
Inputs60/100
4 of 5 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
object_type
Not provided
Missing description. Explain what this value means and how to supply it.
object_ids
One or more object ids to analyze the schema for. For example, if you need to understand the schema of a specific experiment, just provide its object id.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
include_sample_values
Whether to include sample values for each field in the schema. This can help illustrate the type of data stored in each field.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
limit
The maximum number of fields to return (defaults to 1000; cannot exceed 1000).
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
fields
If specified, only infer schema for these top-level fields. Otherwise, infer schema for all fields.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "infer_schema",
"description": "Analyze the structure and content of a data source by inferring the schema and showing the most common values for each field.\n\nThis tool discovers:\n- All available fields in the data source (input, output, metadata.*, scores.*, etc.)\n- The data type of each field (string, number, array, object)\n- The most frequently occurring values for each field with their counts\n- The complete field paths for use in SQL queries\n\nFor each field, you'll see:\n- The full path as an array (e.g., ['metadata', 'user_id'] which translates to metadata.user_id in SQL)\n- The data type (string, number, boolean, etc.)\n- Top 10 most common values sorted by frequency with exact counts\n\nLarge results are truncated to fit the context window: sample values may be shortened and capped at 3 per field, and if there are still too many fields, data contains a subset and totalFields reports the full field count. When totalFields is present, use sql_query to explore fields not shown here.\n\nThis is essential for:\n- Discovering what fields exist before writing queries\n- Understanding the actual values in your data (e.g., what user IDs exist, what models are used)\n- Finding the exact field paths for SQL queries\n- Identifying patterns in your data (e.g., most common inputs, frequent error types)\n- Understanding cardinality (how many unique values exist)\n\nExample insights from the tool:\n- See all metadata fields like metadata.user_id, metadata.model, metadata.session_id\n- Discover custom fields you may not know about\n- Find the most active users or most used models\n- Understand what values to filter on in your queries\n\n\nTime range filter guidance:\n- All queries MUST have a time range filter.\n- If you do not provide a time range filter the request will fail.\n- If the user does not specify a time range, always use the default time range filter provided in the prompt.\n- If the user specifies a time range in the prompt, always use the time range they specified.\n- If default time range is missing and user does not specify a time range, always use the last 3 days.\n- Unless the user specifies a specific from and to hour, always provide 06:00:00.000Z as the from hour and 05:59:59.999Z as the to hour to go along with the year, month, and day.\n- When generating a custom time range for a relative date request, derive the year, month, and day from the <current_date> tag or user's specified date rather than from examples.\n\n",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"object_type",
"object_ids"
],
"properties": {
"limit": {
"type": "integer",
"default": 1000,
"maximum": 1000,
"description": "The maximum number of fields to return (defaults to 1000; cannot exceed 1000).",
"exclusiveMinimum": 0
},
"fields": {
"type": "array",
"items": {
"enum": [
"input",
"output",
"expected",
"metadata",
"scores",
"metrics"
],
"type": "string"
},
"description": "If specified, only infer schema for these top-level fields. Otherwise, infer schema for all fields."
},
"object_ids": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more object ids to analyze the schema for. For example, if you need to understand the schema of a specific experiment, just provide its object id."
},
"object_type": {
"enum": [
"project_logs",
"experiment",
"dataset",
"prompt_session",
"playground_logs",
"project_prompts",
"project_functions",
"project_patterns",
"org_prompts",
"org_functions"
],
"type": "string"
},
"include_sample_values": {
"type": "boolean",
"default": false,
"description": "Whether to include sample values for each field in the schema. This can help illustrate the type of data stored in each field."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Infer schema",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description77/100
Report a new project pattern. First load braintrust/pattern-analysis, which uses search_patterns to check for potential duplicates and defines evidence and reporting limits. Always include Frequency (evidence.frequency) when existing data can identify affected traces. Other monitor charts are optional additions, not a substitute. If Frequency cannot be measured, attach representative trace evidence and explain the missing signal. Include an audit reason.
The description explains the capability, when to use it, and what to expect.
Inputs27/100
14 of 49 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
The current project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
name
A concise pattern name.
Explain the meaning of this value, not just its name.
description
Short Markdown description of the behavior and impact; omit investigation, monitoring, and next steps.
Clear enough to supply this argument.
recommended_next_step
Concrete next fix or investigation in Markdown. Use backticks for tools, field paths, and errors.
Clear enough to supply this argument.
severity
User-impact severity; the pattern analysis skill defines each level.
Clear enough to supply this argument.
evidence › monitors › items › anyOf › 0 › title
Not provided
Missing description. Explain what this value means and how to supply it.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description85/100
Search existing patterns in a project by text, semantic similarity, ID, status, or quality. limit must be 1–100. Omit query for a listing. Ordinary listings and text searches use stable insertion order, newest first; while has_more is true, pass the returned cursor with the same project and filters to continue. Cursors do not support similar_to. Load braintrust/pattern-analysis before a pattern investigation. Use similar_to with one or more existing pattern IDs or unreported pattern drafts to catch duplicates with different wording.
The description explains the capability, when to use it, and what to expect.
Inputs54/100
8 of 14 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
The current project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
pattern_id
Not provided
Missing description. Explain what this value means and how to supply it.
exclude_pattern_id
A pattern ID to omit from the results.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
query
Full-text search across pattern names, descriptions, recommended next steps, and feedback. Omit for a listing; blank text is treated as omitted. Use similar_to instead when wording may differ.
Clear enough to supply this argument.
similar_to › anyOf › 0 › anyOf › 0 › pattern_id
The ID of an existing pattern to compare.
Clear enough to supply this argument.
similar_to › anyOf › 0 › anyOf › 1 › name
Not provided
Missing description. Explain what this value means and how to supply it.
similar_to › anyOf › 0 › anyOf › 1 › description
Not provided
Missing description. Explain what this value means and how to supply it.
Missing description. Explain what this value means and how to supply it.
status
Not provided
Missing description. Explain what this value means and how to supply it.
cursor
The opaque BTQL cursor returned by the previous page. Keep the same project and filters. A full final page may require an empty follow-up page. Not supported with similar_to.
Clear enough to supply this argument.
quality
Not provided
Missing description. Explain what this value means and how to supply it.
include_evidence
Include monitor and trace evidence in results. Defaults to true for a pattern_id lookup and false otherwise.
Clear enough to supply this argument.
return_url
Store the complete search response and return a temporary download URL instead of inline JSON.
Clear enough to supply this argument.
limit
Maximum patterns to return (1–100), per target for semantic searches. For ordinary listings or text searches, pass the returned cursor to retrieve the next page when has_more is true.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "search_patterns",
"description": "Search existing patterns in a project by text, semantic similarity, ID, status, or quality. limit must be 1–100. Omit query for a listing. Ordinary listings and text searches use stable insertion order, newest first; while has_more is true, pass the returned cursor with the same project and filters to continue. Cursors do not support similar_to. Load braintrust/pattern-analysis before a pattern investigation. Use similar_to with one or more existing pattern IDs or unreported pattern drafts to catch duplicates with different wording.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"project_id"
],
"properties": {
"limit": {
"type": "integer",
"default": 20,
"maximum": 100,
"minimum": 1,
"description": "Maximum patterns to return (1–100), per target for semantic searches. For ordinary listings or text searches, pass the returned cursor to retrieve the next page when has_more is true."
},
"query": {
"type": "string",
"description": "Full-text search across pattern names, descriptions, recommended next steps, and feedback. Omit for a listing; blank text is treated as omitted. Use similar_to instead when wording may differ."
},
"cursor": {
"type": "string",
"description": "The opaque BTQL cursor returned by the previous page. Keep the same project and filters. A full final page may require an empty follow-up page. Not supported with similar_to."
},
"status": {
"enum": [
"active",
"inactive",
"duplicate"
],
"type": "string"
},
"quality": {
"enum": [
"useful",
"noise"
],
"type": "string"
},
"pattern_id": {
"type": "string",
"format": "uuid"
},
"project_id": {
"type": "string",
"minLength": 1,
"description": "The current project ID. In hosted Agent Runtime, call braintrust_context to obtain it."
},
"return_url": {
"type": "boolean",
"default": false,
"description": "Store the complete search response and return a temporary download URL instead of inline JSON."
},
"similar_to": {
"anyOf": [
{
"anyOf": [
{
"type": "object",
"required": [
"pattern_id"
],
"properties": {
"pattern_id": {
"type": "string",
"format": "uuid",
"description": "The ID of an existing pattern to compare."
}
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"name",
"description"
],
"properties": {
"name": {
"type": "string",
"minLength": 1
},
"description": {
"type": "string",
"minLength": 1
},
"recommended_next_step": {
"type": "string",
"minLength": 1
}
},
"additionalProperties": false
}
]
},
{
"type": "array",
"items": {
"$ref": "#/properties/similar_to/anyOf/0"
},
"maxItems": 200,
"minItems": 1
}
],
"description": "One pattern or an array of patterns to compare semantically against existing patterns. Use pattern_id for a reported pattern, or name and description for an unreported pattern. Array results are grouped by target."
},
"include_evidence": {
"type": "boolean",
"description": "Include monitor and trace evidence in results. Defaults to true for a pattern_id lookup and false otherwise."
},
"exclude_pattern_id": {
"type": "string",
"format": "uuid",
"description": "A pattern ID to omit from the results."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Search patterns",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description84/100
Update an existing project pattern's content, status, quality, or feedback, and optionally attach quoted evidence to matching traces. Closing an active pattern saves the reason as feedback. Before using this tool, load braintrust/pattern-analysis and use search_patterns to inspect the current pattern. When revising evidence, include Frequency (evidence.frequency) whenever measurable; other monitors are optional additions.
The description explains the capability, when to use it, and what to expect.
Inputs27/100
15 of 53 inputs have descriptions. Add descriptions to the missing inputs, including where to obtain IDs and how to supply values.
project_id
The current project ID. In hosted Agent Runtime, call braintrust_context to obtain it.
Clear enough to supply this argument.
pattern_id
The pattern ID to update.
Clear enough to supply this argument.
name
Not provided
Missing description. Explain what this value means and how to supply it.
description
Not provided
Missing description. Explain what this value means and how to supply it.
recommended_next_step
Not provided
Missing description. Explain what this value means and how to supply it.
severity
User-impact severity; the pattern analysis skill defines each level.
Clear enough to supply this argument.
evidence › monitors › items › anyOf › 0 › title
Not provided
Missing description. Explain what this value means and how to supply it.
How the evaluator would detect or measure the pattern
Clear enough to supply this argument.
evidence › frequency › affected_filter
Not provided
Missing description. Explain what this value means and how to supply it.
evidence › frequency › population_filter
Not provided
Missing description. Explain what this value means and how to supply it.
evidence › frequency › population_description
Not provided
Missing description. Explain what this value means and how to supply it.
evidence › frequency › snapshot › affected_trace_count
Not provided
Missing description. Explain what this value means and how to supply it.
evidence › frequency › snapshot › population_affected_trace_count
Not provided
Missing description. Explain what this value means and how to supply it.
evidence › frequency › snapshot › population_trace_count
Not provided
Missing description. Explain what this value means and how to supply it.
evidence › frequency › snapshot › total_trace_count
Not provided
Missing description. Explain what this value means and how to supply it.
evidence › frequency › snapshot › time_window › from
Not provided
Missing description. Explain what this value means and how to supply it.
evidence › frequency › snapshot › time_window › to
Not provided
Missing description. Explain what this value means and how to supply it.
status
Not provided
Missing description. Explain what this value means and how to supply it.
duplicate_of
Canonical pattern ID in this project. Required when closing as duplicate. To undo duplicate closure, explicitly clear it with null and set active or inactive.
Clear enough to supply this argument.
quality
Whether the pattern was useful or noise. Omit to preserve the existing quality or pass null to clear it.
Clear enough to supply this argument.
feedback
A comment to append to the pattern's feedback. Omit or pass null to preserve existing feedback.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
trace_evidence › items › root_span_id
The root span ID of a trace demonstrating the pattern.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
trace_evidence › items › summary
Why this trace demonstrates the pattern.
Clear enough to supply this argument.
trace_evidence › items › evidence › items › title
Not provided
Missing description. Explain what this value means and how to supply it.
The title clearly communicates the tool’s action and subject. Keep it concise.
Description89/100
# Braintrust MCP `sql_query` tool
This tool is NOT a SQL string. You fill separate parameters and the server assembles:
```
SELECT {select} FROM {object_type}({object_ids}[, shape => '{shape}']) [WHERE {where}] [GROUP BY {group_by}] [ORDER BY {order_by}] [LIMIT {limit}]
```
Every clause parameter holds ONLY the fragment after its keyword — never include `WHERE`, `GROUP BY`, `ORDER BY`, `SELECT`, or `FROM` inside `where`/`group_by`/`order_by`/`select`, and never write a `FROM` yourself (`object_type`/`object_ids` generate it).
## Parameters
| Param | Put | Never put |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `select` | columns/exprs, comma-separated: `"id, root_span_id, created, error"` | `SELECT`, a `FROM` |
| `object_type` | one of `project_logs` \| `experiment` \| `dataset` (also valid but rarer: `prompt_session`, `playground_logs`, `project_prompts`, `project_functions`, `org_prompts`, `org_functions`) | anything else |
| `object_ids` | array of UUID strings, same type, e.g. `["<project_id>"]` | mixed object types in one call |
| `where` | condition text only: `"error IS NOT NULL AND created > now() - interval 1 day"`. **`project_logs` must be scoped** — either a `created` range **or** an `id`/`root_span_id` filter (see Filters) | leading `WHERE`; a `created` range added on top of an `id`/`root_span_id` lookup (silently returns nothing if the target is older); a `created` range on `experiment`/`dataset`/`view` |
| `group_by` | column list only: `"metadata.model"`; keys must be columnar — see Grouping | leading `GROUP BY`; a dynamic expr like `lower(metadata.model)` (blocked, see table); `input`/`output` or any path under them (blocked, see Grouping) |
| `order_by` | `"created DESC"` — **only `created` sorts via the index** | leading `ORDER BY`; `metrics.*`, a computed expr (`metrics.end - metrics.start`), `root_span_id`, or multi-key sorts (all trip the sort-pushdown lint — drop `order_by` and sort client-side, or `force_ignore_linter` if already id-scoped) |
| `limit` | integer, default 10, max 1000 | omit when projecting large fields (see lint table) |
| `shape` | `spans` (default) \| `traces` \| `summary` | any `group_by`, aggregate, or `order_by` with `summary` — always lint-blocked (do those on `spans`; see Shape selection) |
| `preview_length` | leave default (1024) unless you need full untruncated values (`-1`) | — |
| `force_ignore_linter` | only after rewriting per the lint message once; use when the query is already selective/bounded and the residual warning is unavoidable | as a first response to a lint block |
| `return_url` | `true` only for large/unbounded downloads | — |
## Field names that actually exist (bind errors are silent-wrong, not always loud)
- `type`, `parent_id` as bare fields **fail to bind**: `Field 'type' not found in schema`. Use `span_attributes.type`, `span_parents`.
- **Span hierarchy lives in `span_parents`, an array of parent `span_id`s — there is no `parent_id` column.** To attribute spans to their parents: select `span_parents` on the child rows, then run a second query filtering `span_id IN (<those ids>)`. Do not try to join in one query.
- Token/latency fields are **shape-dependent** (picking the wrong one is silent — returns `null`, no error):
- On `spans`/`traces`: per-span `metrics.tokens` (also `metrics.prompt_tokens`, `metrics.completion_tokens`); latency is `metrics.end - metrics.start`. `metrics.duration`/`metrics.total_tokens` are null here.
- On `summary`: the per-trace rollups `metrics.total_tokens` and `metrics.duration` (`metrics.tokens` is null) — but reachable only via a plain summary read (no GROUP BY/aggregate/ORDER BY; see Shape selection). **To aggregate tokens/latency across traces, use spans**: `SUM(metrics.tokens)`, `metrics.end - metrics.start`.
- **`metrics.estimated_cost` aggregates but never filters.** It is a computed value, not a stored column — the binder rewrites it to `estimated_cost()`, and a function on the field side cannot be pushed into the index. `SUM(metrics.estimated_cost)` / `AVG(...)` in `select` are fine; **any** use in `where` is blocked (`metrics.estimated_cost IS NOT NULL`, `metrics.estimated_cost > 0`, and the explicit `estimated_cost() ...` all fail with `Cannot execute filter over the index`). To restrict to rows that have cost, filter a real column instead — `metrics.prompt_tokens IS NOT NULL` or `span_attributes.type = 'llm'` — or drop the filter and let the aggregate skip nulls.
- Nested fields always need the parent: `metadata.user_id`, `scores.Factuality`, `span_attributes.name`.
- Braintrust organization members are recorded in `span_attributes.created_by_user_id`. Resolve those IDs in bulk with `lookup_users`. API keys and service tokens are recorded in `span_attributes.created_by_api_key_id`; resolve them in bulk with `lookup_api_keys`. These are system span attributes, not metadata. `metadata.user_id` and similarly named metadata fields are application-defined and must not be assumed to identify Braintrust users or keys.
- `is_root = true` restricts to root spans (needed for root-only metadata/latency).
- Aggregate functions: `count`, `sum`, `avg`, `min`, `max`, `percentile(expr, p)` (not `PERCENTILE_CONT`/`percentile_cont`).
- **Aggregation only constrains `select` once you use an aggregate.** A plain projection needs no `group_by` — `select: "id, root_span_id, created, error"` returns a table of rows and is the normal shape for browsing. As soon as `select` contains an aggregate, every _other_ column in `select` must also appear in `group_by`:
- aggregate + a bare column, no `group_by` → `Cannot mix aggregate and non-aggregate expressions without GROUP BY`
- `group_by` set, but a selected column is missing from it → `Each measure must contain an aggregate function` (the "measure" is that ungrouped column)
- Fix either by adding the column to `group_by`, or removing it from `select`.
- Bucketing a timestamp: `date_trunc('<unit>', <field>)`, e.g. `date_trunc('hour', created)`. Units include `minute`, `hour`, `day`, `week`, and arbitrary intervals such as `'300 seconds'`. Works on any timestamp field.
- **This is not ClickHouse, Postgres, or BigQuery SQL.** Builtins from those dialects do not exist and fail with `Unknown function: X` — there is no `date_bin`, `toStartOfHour`/`toStartOfDay`, `toDate`, `countIf`, `has`, or `position*`. Use only the functions listed here; if you need something else, compute it from the returned rows instead of inventing a builtin.
- Time: `now()`, `interval N day` — number **unquoted**, unit **singular** (`interval 7 day`, `interval 24 hour`; plural `7 days`/`24 hours` fails with `interval unit Days`), no quotes (`interval '1 day'` also fails).
## Shape selection
Default to **spans**, and do all grouping / aggregation / sorting there. The tool's strict lint blocks every GROUP BY, aggregate, and ORDER BY on `summary` (`Group by over summary queries is inefficient; consider using a 'traces' or 'spans' shaped query`).
- **Counts, breakdowns, rollups across traces → spans.** Trace count = `COUNT(DISTINCT root_span_id)`; per-trace token/cost totals = `SUM(metrics.tokens)` / `SUM(metrics.estimated_cost)` (add `GROUP BY root_span_id` for per-trace); use `WHERE is_root = true` for one-row-per-trace metadata.
- **summary** is usable **only** as a plain scoped read — a `created` range or an `id`/`root_span_id`, small `LIMIT`, and **no GROUP BY / aggregate / ORDER BY** — to fetch pre-rolled per-trace fields (`metrics.total_tokens`, `metrics.duration`, `span_type_info.*`, averaged scores, root-span input/output).
- **traces** returns all spans of matching traces; keep `select` narrow (large fields are blocked on traces).
## Filters
**A `project_logs` query must be scoped**, or it is rejected with `No filters available for segment elimination`. Either form satisfies this:
- a range filter on `created` (or `_xact_id` / `_pagination_key`) — the usual choice when browsing or aggregating, or
- a filter pinning a specific `id` or `root_span_id` — the right choice when the user names a particular trace or span.
**Do not add a time range to an ID-scoped lookup.** Scoping by `id`/`root_span_id` is sufficient on its own, and bolting on a range silently returns zero rows whenever the target is older than that range — a wrong answer with no error. Only when the query is _not_ ID-scoped and the user gave no time window should you default to `created > now() - interval 1 day`, and say so in `explanation`.
`experiment`, `dataset`, and `view` do **not** require any scope, and adding a `created` range to them can silently exclude rows.
Filters compare a **field to a literal**. Two shapes cannot be pushed into the index and are blocked with `Cannot execute filter over the index`:
- **Comparing two fields to each other.** `span_id = root_span_id` fails — use `is_root = true`. `metrics.end > metrics.start` fails — compare against a literal instead: `(metrics.end - metrics.start) > 1`.
- **A function on the field side.** `lower(span_attributes.name) = 'x'` fails — filter the raw field. This is also why `metrics.estimated_cost` can never be filtered (see above).
Nearly everything else pushes down: `=`, `!=`, `IN`, `IS NULL`/`IS NOT NULL`, numeric comparisons on `metrics.*`, `AND`/`OR`/`NOT`, and `LIKE`/`ILIKE` (leading wildcards included).
For arrays of objects, a dot path matches any element: `classifications."Classifier name".id = 'inconsistent'`. A positional lookup such as `classifications."Classifier name"[0].id` selects only the first element and cannot be pushed into the index. Omit the numeric index when you intend to match any element; keep it only when the position matters.
Full-text search uses the **infix** operator `<field> MATCH '<term>'` (negate with `<field> NOT MATCH '<term>'`) — it is an operator, not a function, so `MATCH(field, 'term')` is a parse error. It matches whole terms lexically, not substrings or meanings: `output MATCH 'time'` does not match `timeout`, and there is no semantic/embedding search. For substring matching use `LIKE`/`ILIKE`.
`input`/`output` sub-paths are a special case — **filterable, but not with ranges**:
- Works: equality (`output.<field> = '<value>'`), null checks (`input.<field> IS NOT NULL`, `output IS NOT NULL`), and negated equality (`NOT (output.<field> = '<value>')`)
- Blocked: numeric range comparisons on these paths — `output.<numeric_field> >= 0.8`, `< 1`, `>`, `BETWEEN` and friends all fail even though the same comparison works fine on `metrics.*`. Compare for equality instead, or project the field and compare the returned rows yourself.
- Grouping by these paths is never allowed — see Grouping.
## Grouping: `group_by` keys must be columnar
This is the single most common lint failure. A `group_by` key has to be a columnar ("fast") field; otherwise the engine cannot push the grouping into the index and blocks with `Cannot push down group by to the index which will result in an inefficient query plan. Reasons: group by or unpivot fields are not Tantivy fast fields: X`.
- **Group-able:** `span_attributes.*` (`name`, `type`, `purpose`, …), `metadata.*` at any depth (`metadata.model`, `metadata.user_id`), `scores.*`, `tags`, `metrics.*`, `error`, `root_span_id`, `span_id`, `is_root`.
- **NOT group-able: `input`, `output`, and any path under them** — `input.<field>`, `output.<field>`, and deeper paths such as `output.<a>.<b>.<c>` all fail, however narrow the sub-path looks. These live inside large document fields rather than columns. Array indexing (`tags[0]`) fails for the same reason.
- **To break down by a non-columnar field, do one of these two things** (both work; pick by whether you already know the values):
1. **Values known/enumerable** — one aggregate query per value, using an equality filter, which does push down: `select: "COUNT(1) as n"`, `where: "<range> AND <field> = '<value>'"`. Repeat per value and assemble the breakdown yourself.
2. **Values unknown** — project the field and bucket the rows client-side: `select: "root_span_id, <field>"` with a bounded `limit`, then count the values in the returned rows. Sub-paths like `output.foo` are ordinary projections and are **not** subject to the ≤100 large-field limit (that applies to bare `input`/`output`/`expected`/`metadata`), so a few hundred rows is fine.
Do not attempt the `group_by` anyway — it is rejected, not merely slow.
The same error also fires for reasons unrelated to the key — they are listed after `Reasons:`, so read them before rewriting:
| Reason text | Fix |
| --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `group by or unpivot fields are not Tantivy fast fields: X` | X is not columnar — pick a different key per the lists above |
| `summary-shaped queries do not support group by pushdown` | drop `shape: "summary"` and group on `spans` (see Shape selection) |
| `a filter could not be fully pushed down to Tantivy` / `trace filter could not be fully pushed down to Tantivy` | the group keys are fine — the `where` clause is the problem. Check for `metrics.estimated_cost` first, then field-to-field compares and functions on the field side |
| `subquery filters could not be fully pushed down to Tantivy` | simplify or remove the sub-filter |
| `sampling with group by pushdown is not supported` | drop the sample |
## Recipes (concrete param fills)
**Recent rows**
`select: "id, root_span_id, created, error"`, `order_by: "created DESC"`, `limit: 20`
**Errors in the last day**
`select: "id, root_span_id, created, error"`, `where: "error IS NOT NULL AND created > now() - interval 1 day"`, `limit: 20`
**Count/aggregate per trace (breakdown by model)**
`select: "metadata.model, COUNT(1) as calls, COUNT(DISTINCT root_span_id) as traces"`, `where: "created > now() - interval 1 day"`, `group_by: "metadata.model"`, `order_by: "calls DESC"`, `limit: 20` (shape stays default `spans`)
**Per-trace token / cost totals (on spans, not summary)**
`select: "root_span_id, SUM(metrics.tokens) as tokens, SUM(metrics.estimated_cost) as cost"`, `where: "created > now() - interval 1 day"`, `group_by: "root_span_id"`, `order_by: "tokens DESC"`, `limit: 20`
**Breakdown by tool call**
`select: "span_attributes.name, COUNT(1) as calls"`, `where: "created > now() - interval 1 day AND span_attributes.type = 'tool'"`, `group_by: "span_attributes.name"`, `order_by: "calls DESC"`, `limit: 20`
**Latency percentile (root spans only)**
`select: "PERCENTILE(metrics.end - metrics.start, 0.95) as p95"`, `where: "created > now() - interval 1 day AND is_root = true"`, `limit: 1`
**Full trace context for errors**
`select: "id, root_span_id, created, error"`, `where: "error IS NOT NULL AND created > now() - interval 1 day"`, `shape: "traces"`, `limit: 20` — keep the `select` narrow; large fields (`input`/`output`) on `traces` shape are separately blocked (see table).
## Error substring → cause → fix
| Error contains | Cause | Fix |
| ------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Field 'X' not found in schema` | bare/wrong field name in `select`/`where`/`group_by`/`order_by` | qualify it: `span_attributes.type`, `span_parents`, `metadata.X`, `scores.X` |
| `Unsupported expression: invalid interval value` | quoted interval, e.g. `interval '1 day'` | unquote: `interval 1 day` |
| `Unknown function: X` | wrong builtin name (`PERCENTILE_CONT`, `MATCH_AGAINST`, etc.) | use the real name, e.g. `percentile(expr, p)` |
| `Group by over summary queries is inefficient` | any GROUP BY / aggregate / ORDER BY on `shape: "summary"` | move to `shape: "spans"` (default): `COUNT(DISTINCT root_span_id)`, `SUM(metrics.tokens)`, etc. — summary is read-only for pre-rolled fields |
| `No filters available for segment elimination` | no selective filter on `created`/`_xact_id`/`_pagination_key`, or no `id`/`root_span_id` scope | add a time range to `where`, e.g. `created > now() - interval 1 day` |
| `Cannot push down group by to the index` | a `group_by` key is not columnar (`input`/`output` or a path under them, array index), or the reason names a filter/shape/sample issue | see Grouping — pick a columnar key, or fix the reason listed after `Reasons:` |
| `group by on a dynamic expression ... not columnar-compatible` | `group_by` uses a function/expression (`lower(metadata.model)`) | group by the raw field; post-process casing/derivation after the query returns |
| `standalone Project above Index search ... sort pushdown may have failed` | `order_by` on a non-indexed column (`metrics.*`, computed expr, `root_span_id`, or multi-key) | sort by `created` only, or drop `order_by` and sort client-side; if already scoped by `id`/`root_span_id`, `force_ignore_linter: true` |
| `projects large fields (...) without LIMIT <= 100` | `select` includes `input`/`output`/similar with `limit` > 100 on spans/summary | set `limit` ≤ 100, or scope `where` to a specific `id`/`root_span_id` |
| `projects large fields (...) on traces shape` | `select` includes large fields with `shape: "traces"` | switch to `spans`/`summary`, or scope `where` to specific ids |
| `Cannot mix aggregate and non-aggregate expressions without GROUP BY` | `select` has an aggregate plus a bare column, and `group_by` is empty | add the bare column to `group_by`, or drop it from `select` |
| `Each measure must contain an aggregate function` | `group_by` is set but a column in `select` is not in it | add that column to `group_by`, or drop it from `select` |
| `Cannot execute filter over the index` | `where` filter isn't pushdown-friendly. Most common cause by far: `metrics.estimated_cost` in `where` (becomes `ESTIMATED_COST()`). Also: comparing two fields (`metrics.end > metrics.start`), or a function on the field side (`lower(name) = 'x'`) | drop the cost filter (aggregate it in `select` instead) or filter `metrics.prompt_tokens`; compare a field to a literal, not to another field; filter the raw field, not a function of it |
| `Limit must be <= 1000` | `limit` over 1000 | cap `limit` at 1000; page with `where` scoping instead of raising it further |
## Notes
- `object_type` and `object_ids` are **required** — always set both; omitting them fails schema validation before the query runs.
- The result the tool returns already reflects lint outcomes — `force_ignore_linter: true` runs it anyway and reports the same messages as warnings instead of errors, so read them even when bypassing.
The description explains the capability, when to use it, and what to expect.
Inputs81/100
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
explanation
Optional user-facing explanation of what the query is checking.
Clear enough to supply this argument.
select
Columns to select. Prefer explicit lightweight fields for discovery, then fetch large payload fields only after narrowing the candidate set.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
object_type
The object type to query. In hosted Agent Runtime, call braintrust_context or get_data_source to determine the current page target.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
object_ids
One or more object ids to query. For example, if you need to get results from a specific experiment, just provide its object id. If you want to look at results across multiple experiments, provide multiple object ids.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
where
Filter condition without the WHERE keyword. Prefer selective, indexable filters such as a time range or exact identifier.
Clear enough to supply this argument.
group_by
Grouping clause without the GROUP BY keyword.
Explain the meaning of this value, not just its name.
order_by
Sort order without the ORDER BY keyword. Prefer raw indexed fields and combine sorting with a small limit.
Clear enough to supply this argument.
limit
Maximum number of rows to return. Defaults to 10. Keep discovery queries small and bound payload queries explicitly.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
shape
The shape of data to return:
- spans (default): Returns individual spans and is the safest default for discovery and bounded inspection
- traces: Expands matching traces to all of their spans, which can return substantially more data than the limit suggests
- summary: Returns one rolled-up row per trace for trace-level aggregate analysis.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
preview_length
Maximum characters per field before truncation. Defaults to 1024. Set to -1 to disable truncation only when an already selective and bounded query requires complete values.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
force_ignore_linter
Bypass SQL lint failures. First rewrite the query according to the lint feedback. Use true only when the query is already selective and bounded and the remaining warning is unavoidable for the user's request.
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
return_url
Return a signed URL instead of inline rows. This changes delivery only and does not make an unbounded or expensive query safe. For an untruncated download, also set preview_length to -1.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "sql_query",
"description": "# Braintrust MCP `sql_query` tool\n\nThis tool is NOT a SQL string. You fill separate parameters and the server assembles:\n\n```\nSELECT {select} FROM {object_type}({object_ids}[, shape => '{shape}']) [WHERE {where}] [GROUP BY {group_by}] [ORDER BY {order_by}] [LIMIT {limit}]\n```\n\nEvery clause parameter holds ONLY the fragment after its keyword — never include `WHERE`, `GROUP BY`, `ORDER BY`, `SELECT`, or `FROM` inside `where`/`group_by`/`order_by`/`select`, and never write a `FROM` yourself (`object_type`/`object_ids` generate it).\n\n## Parameters\n\n| Param | Put | Never put |\n| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `select` | columns/exprs, comma-separated: `\"id, root_span_id, created, error\"` | `SELECT`, a `FROM` |\n| `object_type` | one of `project_logs` \\| `experiment` \\| `dataset` (also valid but rarer: `prompt_session`, `playground_logs`, `project_prompts`, `project_functions`, `org_prompts`, `org_functions`) | anything else |\n| `object_ids` | array of UUID strings, same type, e.g. `[\"<project_id>\"]` | mixed object types in one call |\n| `where` | condition text only: `\"error IS NOT NULL AND created > now() - interval 1 day\"`. **`project_logs` must be scoped** — either a `created` range **or** an `id`/`root_span_id` filter (see Filters) | leading `WHERE`; a `created` range added on top of an `id`/`root_span_id` lookup (silently returns nothing if the target is older); a `created` range on `experiment`/`dataset`/`view` |\n| `group_by` | column list only: `\"metadata.model\"`; keys must be columnar — see Grouping | leading `GROUP BY`; a dynamic expr like `lower(metadata.model)` (blocked, see table); `input`/`output` or any path under them (blocked, see Grouping) |\n| `order_by` | `\"created DESC\"` — **only `created` sorts via the index** | leading `ORDER BY`; `metrics.*`, a computed expr (`metrics.end - metrics.start`), `root_span_id`, or multi-key sorts (all trip the sort-pushdown lint — drop `order_by` and sort client-side, or `force_ignore_linter` if already id-scoped) |\n| `limit` | integer, default 10, max 1000 | omit when projecting large fields (see lint table) |\n| `shape` | `spans` (default) \\| `traces` \\| `summary` | any `group_by`, aggregate, or `order_by` with `summary` — always lint-blocked (do those on `spans`; see Shape selection) |\n| `preview_length` | leave default (1024) unless you need full untruncated values (`-1`) | — |\n| `force_ignore_linter` | only after rewriting per the lint message once; use when the query is already selective/bounded and the residual warning is unavoidable | as a first response to a lint block |\n| `return_url` | `true` only for large/unbounded downloads | — |\n\n## Field names that actually exist (bind errors are silent-wrong, not always loud)\n\n- `type`, `parent_id` as bare fields **fail to bind**: `Field 'type' not found in schema`. Use `span_attributes.type`, `span_parents`.\n- **Span hierarchy lives in `span_parents`, an array of parent `span_id`s — there is no `parent_id` column.** To attribute spans to their parents: select `span_parents` on the child rows, then run a second query filtering `span_id IN (<those ids>)`. Do not try to join in one query.\n- Token/latency fields are **shape-dependent** (picking the wrong one is silent — returns `null`, no error):\n - On `spans`/`traces`: per-span `metrics.tokens` (also `metrics.prompt_tokens`, `metrics.completion_tokens`); latency is `metrics.end - metrics.start`. `metrics.duration`/`metrics.total_tokens` are null here.\n - On `summary`: the per-trace rollups `metrics.total_tokens` and `metrics.duration` (`metrics.tokens` is null) — but reachable only via a plain summary read (no GROUP BY/aggregate/ORDER BY; see Shape selection). **To aggregate tokens/latency across traces, use spans**: `SUM(metrics.tokens)`, `metrics.end - metrics.start`.\n- **`metrics.estimated_cost` aggregates but never filters.** It is a computed value, not a stored column — the binder rewrites it to `estimated_cost()`, and a function on the field side cannot be pushed into the index. `SUM(metrics.estimated_cost)` / `AVG(...)` in `select` are fine; **any** use in `where` is blocked (`metrics.estimated_cost IS NOT NULL`, `metrics.estimated_cost > 0`, and the explicit `estimated_cost() ...` all fail with `Cannot execute filter over the index`). To restrict to rows that have cost, filter a real column instead — `metrics.prompt_tokens IS NOT NULL` or `span_attributes.type = 'llm'` — or drop the filter and let the aggregate skip nulls.\n- Nested fields always need the parent: `metadata.user_id`, `scores.Factuality`, `span_attributes.name`.\n- Braintrust organization members are recorded in `span_attributes.created_by_user_id`. Resolve those IDs in bulk with `lookup_users`. API keys and service tokens are recorded in `span_attributes.created_by_api_key_id`; resolve them in bulk with `lookup_api_keys`. These are system span attributes, not metadata. `metadata.user_id` and similarly named metadata fields are application-defined and must not be assumed to identify Braintrust users or keys.\n- `is_root = true` restricts to root spans (needed for root-only metadata/latency).\n- Aggregate functions: `count`, `sum`, `avg`, `min`, `max`, `percentile(expr, p)` (not `PERCENTILE_CONT`/`percentile_cont`).\n- **Aggregation only constrains `select` once you use an aggregate.** A plain projection needs no `group_by` — `select: \"id, root_span_id, created, error\"` returns a table of rows and is the normal shape for browsing. As soon as `select` contains an aggregate, every _other_ column in `select` must also appear in `group_by`:\n - aggregate + a bare column, no `group_by` → `Cannot mix aggregate and non-aggregate expressions without GROUP BY`\n - `group_by` set, but a selected column is missing from it → `Each measure must contain an aggregate function` (the \"measure\" is that ungrouped column)\n - Fix either by adding the column to `group_by`, or removing it from `select`.\n- Bucketing a timestamp: `date_trunc('<unit>', <field>)`, e.g. `date_trunc('hour', created)`. Units include `minute`, `hour`, `day`, `week`, and arbitrary intervals such as `'300 seconds'`. Works on any timestamp field.\n- **This is not ClickHouse, Postgres, or BigQuery SQL.** Builtins from those dialects do not exist and fail with `Unknown function: X` — there is no `date_bin`, `toStartOfHour`/`toStartOfDay`, `toDate`, `countIf`, `has`, or `position*`. Use only the functions listed here; if you need something else, compute it from the returned rows instead of inventing a builtin.\n- Time: `now()`, `interval N day` — number **unquoted**, unit **singular** (`interval 7 day`, `interval 24 hour`; plural `7 days`/`24 hours` fails with `interval unit Days`), no quotes (`interval '1 day'` also fails).\n\n## Shape selection\n\nDefault to **spans**, and do all grouping / aggregation / sorting there. The tool's strict lint blocks every GROUP BY, aggregate, and ORDER BY on `summary` (`Group by over summary queries is inefficient; consider using a 'traces' or 'spans' shaped query`).\n\n- **Counts, breakdowns, rollups across traces → spans.** Trace count = `COUNT(DISTINCT root_span_id)`; per-trace token/cost totals = `SUM(metrics.tokens)` / `SUM(metrics.estimated_cost)` (add `GROUP BY root_span_id` for per-trace); use `WHERE is_root = true` for one-row-per-trace metadata.\n- **summary** is usable **only** as a plain scoped read — a `created` range or an `id`/`root_span_id`, small `LIMIT`, and **no GROUP BY / aggregate / ORDER BY** — to fetch pre-rolled per-trace fields (`metrics.total_tokens`, `metrics.duration`, `span_type_info.*`, averaged scores, root-span input/output).\n- **traces** returns all spans of matching traces; keep `select` narrow (large fields are blocked on traces).\n\n## Filters\n\n**A `project_logs` query must be scoped**, or it is rejected with `No filters available for segment elimination`. Either form satisfies this:\n\n- a range filter on `created` (or `_xact_id` / `_pagination_key`) — the usual choice when browsing or aggregating, or\n- a filter pinning a specific `id` or `root_span_id` — the right choice when the user names a particular trace or span.\n\n**Do not add a time range to an ID-scoped lookup.** Scoping by `id`/`root_span_id` is sufficient on its own, and bolting on a range silently returns zero rows whenever the target is older than that range — a wrong answer with no error. Only when the query is _not_ ID-scoped and the user gave no time window should you default to `created > now() - interval 1 day`, and say so in `explanation`.\n\n`experiment`, `dataset`, and `view` do **not** require any scope, and adding a `created` range to them can silently exclude rows.\n\nFilters compare a **field to a literal**. Two shapes cannot be pushed into the index and are blocked with `Cannot execute filter over the index`:\n\n- **Comparing two fields to each other.** `span_id = root_span_id` fails — use `is_root = true`. `metrics.end > metrics.start` fails — compare against a literal instead: `(metrics.end - metrics.start) > 1`.\n- **A function on the field side.** `lower(span_attributes.name) = 'x'` fails — filter the raw field. This is also why `metrics.estimated_cost` can never be filtered (see above).\n\nNearly everything else pushes down: `=`, `!=`, `IN`, `IS NULL`/`IS NOT NULL`, numeric comparisons on `metrics.*`, `AND`/`OR`/`NOT`, and `LIKE`/`ILIKE` (leading wildcards included).\n\nFor arrays of objects, a dot path matches any element: `classifications.\"Classifier name\".id = 'inconsistent'`. A positional lookup such as `classifications.\"Classifier name\"[0].id` selects only the first element and cannot be pushed into the index. Omit the numeric index when you intend to match any element; keep it only when the position matters.\n\nFull-text search uses the **infix** operator `<field> MATCH '<term>'` (negate with `<field> NOT MATCH '<term>'`) — it is an operator, not a function, so `MATCH(field, 'term')` is a parse error. It matches whole terms lexically, not substrings or meanings: `output MATCH 'time'` does not match `timeout`, and there is no semantic/embedding search. For substring matching use `LIKE`/`ILIKE`.\n\n`input`/`output` sub-paths are a special case — **filterable, but not with ranges**:\n\n- Works: equality (`output.<field> = '<value>'`), null checks (`input.<field> IS NOT NULL`, `output IS NOT NULL`), and negated equality (`NOT (output.<field> = '<value>')`)\n- Blocked: numeric range comparisons on these paths — `output.<numeric_field> >= 0.8`, `< 1`, `>`, `BETWEEN` and friends all fail even though the same comparison works fine on `metrics.*`. Compare for equality instead, or project the field and compare the returned rows yourself.\n- Grouping by these paths is never allowed — see Grouping.\n\n## Grouping: `group_by` keys must be columnar\n\nThis is the single most common lint failure. A `group_by` key has to be a columnar (\"fast\") field; otherwise the engine cannot push the grouping into the index and blocks with `Cannot push down group by to the index which will result in an inefficient query plan. Reasons: group by or unpivot fields are not Tantivy fast fields: X`.\n\n- **Group-able:** `span_attributes.*` (`name`, `type`, `purpose`, …), `metadata.*` at any depth (`metadata.model`, `metadata.user_id`), `scores.*`, `tags`, `metrics.*`, `error`, `root_span_id`, `span_id`, `is_root`.\n- **NOT group-able: `input`, `output`, and any path under them** — `input.<field>`, `output.<field>`, and deeper paths such as `output.<a>.<b>.<c>` all fail, however narrow the sub-path looks. These live inside large document fields rather than columns. Array indexing (`tags[0]`) fails for the same reason.\n- **To break down by a non-columnar field, do one of these two things** (both work; pick by whether you already know the values):\n 1. **Values known/enumerable** — one aggregate query per value, using an equality filter, which does push down: `select: \"COUNT(1) as n\"`, `where: \"<range> AND <field> = '<value>'\"`. Repeat per value and assemble the breakdown yourself.\n 2. **Values unknown** — project the field and bucket the rows client-side: `select: \"root_span_id, <field>\"` with a bounded `limit`, then count the values in the returned rows. Sub-paths like `output.foo` are ordinary projections and are **not** subject to the ≤100 large-field limit (that applies to bare `input`/`output`/`expected`/`metadata`), so a few hundred rows is fine.\n Do not attempt the `group_by` anyway — it is rejected, not merely slow.\n\nThe same error also fires for reasons unrelated to the key — they are listed after `Reasons:`, so read them before rewriting:\n\n| Reason text | Fix |\n| --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `group by or unpivot fields are not Tantivy fast fields: X` | X is not columnar — pick a different key per the lists above |\n| `summary-shaped queries do not support group by pushdown` | drop `shape: \"summary\"` and group on `spans` (see Shape selection) |\n| `a filter could not be fully pushed down to Tantivy` / `trace filter could not be fully pushed down to Tantivy` | the group keys are fine — the `where` clause is the problem. Check for `metrics.estimated_cost` first, then field-to-field compares and functions on the field side |\n| `subquery filters could not be fully pushed down to Tantivy` | simplify or remove the sub-filter |\n| `sampling with group by pushdown is not supported` | drop the sample |\n\n## Recipes (concrete param fills)\n\n**Recent rows**\n`select: \"id, root_span_id, created, error\"`, `order_by: \"created DESC\"`, `limit: 20`\n\n**Errors in the last day**\n`select: \"id, root_span_id, created, error\"`, `where: \"error IS NOT NULL AND created > now() - interval 1 day\"`, `limit: 20`\n\n**Count/aggregate per trace (breakdown by model)**\n`select: \"metadata.model, COUNT(1) as calls, COUNT(DISTINCT root_span_id) as traces\"`, `where: \"created > now() - interval 1 day\"`, `group_by: \"metadata.model\"`, `order_by: \"calls DESC\"`, `limit: 20` (shape stays default `spans`)\n\n**Per-trace token / cost totals (on spans, not summary)**\n`select: \"root_span_id, SUM(metrics.tokens) as tokens, SUM(metrics.estimated_cost) as cost\"`, `where: \"created > now() - interval 1 day\"`, `group_by: \"root_span_id\"`, `order_by: \"tokens DESC\"`, `limit: 20`\n\n**Breakdown by tool call**\n`select: \"span_attributes.name, COUNT(1) as calls\"`, `where: \"created > now() - interval 1 day AND span_attributes.type = 'tool'\"`, `group_by: \"span_attributes.name\"`, `order_by: \"calls DESC\"`, `limit: 20`\n\n**Latency percentile (root spans only)**\n`select: \"PERCENTILE(metrics.end - metrics.start, 0.95) as p95\"`, `where: \"created > now() - interval 1 day AND is_root = true\"`, `limit: 1`\n\n**Full trace context for errors**\n`select: \"id, root_span_id, created, error\"`, `where: \"error IS NOT NULL AND created > now() - interval 1 day\"`, `shape: \"traces\"`, `limit: 20` — keep the `select` narrow; large fields (`input`/`output`) on `traces` shape are separately blocked (see table).\n\n## Error substring → cause → fix\n\n| Error contains | Cause | Fix |\n| ------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `Field 'X' not found in schema` | bare/wrong field name in `select`/`where`/`group_by`/`order_by` | qualify it: `span_attributes.type`, `span_parents`, `metadata.X`, `scores.X` |\n| `Unsupported expression: invalid interval value` | quoted interval, e.g. `interval '1 day'` | unquote: `interval 1 day` |\n| `Unknown function: X` | wrong builtin name (`PERCENTILE_CONT`, `MATCH_AGAINST`, etc.) | use the real name, e.g. `percentile(expr, p)` |\n| `Group by over summary queries is inefficient` | any GROUP BY / aggregate / ORDER BY on `shape: \"summary\"` | move to `shape: \"spans\"` (default): `COUNT(DISTINCT root_span_id)`, `SUM(metrics.tokens)`, etc. — summary is read-only for pre-rolled fields |\n| `No filters available for segment elimination` | no selective filter on `created`/`_xact_id`/`_pagination_key`, or no `id`/`root_span_id` scope | add a time range to `where`, e.g. `created > now() - interval 1 day` |\n| `Cannot push down group by to the index` | a `group_by` key is not columnar (`input`/`output` or a path under them, array index), or the reason names a filter/shape/sample issue | see Grouping — pick a columnar key, or fix the reason listed after `Reasons:` |\n| `group by on a dynamic expression ... not columnar-compatible` | `group_by` uses a function/expression (`lower(metadata.model)`) | group by the raw field; post-process casing/derivation after the query returns |\n| `standalone Project above Index search ... sort pushdown may have failed` | `order_by` on a non-indexed column (`metrics.*`, computed expr, `root_span_id`, or multi-key) | sort by `created` only, or drop `order_by` and sort client-side; if already scoped by `id`/`root_span_id`, `force_ignore_linter: true` |\n| `projects large fields (...) without LIMIT <= 100` | `select` includes `input`/`output`/similar with `limit` > 100 on spans/summary | set `limit` ≤ 100, or scope `where` to a specific `id`/`root_span_id` |\n| `projects large fields (...) on traces shape` | `select` includes large fields with `shape: \"traces\"` | switch to `spans`/`summary`, or scope `where` to specific ids |\n| `Cannot mix aggregate and non-aggregate expressions without GROUP BY` | `select` has an aggregate plus a bare column, and `group_by` is empty | add the bare column to `group_by`, or drop it from `select` |\n| `Each measure must contain an aggregate function` | `group_by` is set but a column in `select` is not in it | add that column to `group_by`, or drop it from `select` |\n| `Cannot execute filter over the index` | `where` filter isn't pushdown-friendly. Most common cause by far: `metrics.estimated_cost` in `where` (becomes `ESTIMATED_COST()`). Also: comparing two fields (`metrics.end > metrics.start`), or a function on the field side (`lower(name) = 'x'`) | drop the cost filter (aggregate it in `select` instead) or filter `metrics.prompt_tokens`; compare a field to a literal, not to another field; filter the raw field, not a function of it |\n| `Limit must be <= 1000` | `limit` over 1000 | cap `limit` at 1000; page with `where` scoping instead of raising it further |\n\n## Notes\n\n- `object_type` and `object_ids` are **required** — always set both; omitting them fails schema validation before the query runs.\n- The result the tool returns already reflects lint outcomes — `force_ignore_linter: true` runs it anyway and reports the same messages as warnings instead of errors, so read them even when bypassing.\n",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"select",
"object_type",
"object_ids"
],
"properties": {
"limit": {
"type": "integer",
"default": 10,
"description": "Maximum number of rows to return. Defaults to 10. Keep discovery queries small and bound payload queries explicitly.",
"exclusiveMinimum": 0
},
"shape": {
"enum": [
"spans",
"traces",
"summary"
],
"type": "string",
"description": "The shape of data to return:\n- spans (default): Returns individual spans and is the safest default for discovery and bounded inspection\n- traces: Expands matching traces to all of their spans, which can return substantially more data than the limit suggests\n- summary: Returns one rolled-up row per trace for trace-level aggregate analysis."
},
"where": {
"type": "string",
"description": "Filter condition without the WHERE keyword. Prefer selective, indexable filters such as a time range or exact identifier."
},
"select": {
"type": "string",
"minLength": 1,
"description": "Columns to select. Prefer explicit lightweight fields for discovery, then fetch large payload fields only after narrowing the candidate set."
},
"group_by": {
"type": "string",
"description": "Grouping clause without the GROUP BY keyword."
},
"order_by": {
"type": "string",
"description": "Sort order without the ORDER BY keyword. Prefer raw indexed fields and combine sorting with a small limit."
},
"object_ids": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more object ids to query. For example, if you need to get results from a specific experiment, just provide its object id. If you want to look at results across multiple experiments, provide multiple object ids."
},
"return_url": {
"type": "boolean",
"default": false,
"description": "Return a signed URL instead of inline rows. This changes delivery only and does not make an unbounded or expensive query safe. For an untruncated download, also set preview_length to -1."
},
"explanation": {
"type": "string",
"description": "Optional user-facing explanation of what the query is checking."
},
"object_type": {
"enum": [
"project_logs",
"experiment",
"dataset",
"prompt_session",
"playground_logs",
"project_prompts",
"project_functions",
"project_patterns",
"org_prompts",
"org_functions"
],
"type": "string",
"description": "The object type to query. In hosted Agent Runtime, call braintrust_context or get_data_source to determine the current page target."
},
"preview_length": {
"type": "integer",
"default": 1024,
"description": "Maximum characters per field before truncation. Defaults to 1024. Set to -1 to disable truncation only when an already selective and bounded query requires complete values."
},
"force_ignore_linter": {
"type": "boolean",
"default": false,
"description": "Bypass SQL lint failures. First rewrite the query according to the lint feedback. Use true only when the query is already selective and bounded and the remaining warning is unavoidable for the user's request."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Run SQL query",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description84/100
Get aggregated performance metrics for an experiment, optionally compared to a baseline. Read docs://experiments for background. For row-level data, use sql_query.
Describe the returned information so the agent knows whether it will answer the request.
Inputs75/100
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
experiment_id
The id of the experiment to summarize.
Clarify units, where to obtain this value, or conventions not already expressed in the schema.
comparison_experiment_id
Optional: the id of an experiment to compare against. If provided, the summary will include comparisons to this baseline.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "summarize_experiment",
"description": "Get aggregated performance metrics for an experiment, optionally compared to a baseline. Read docs://experiments for background. For row-level data, use sql_query.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"experiment_id"
],
"properties": {
"experiment_id": {
"type": "string",
"description": "The id of the experiment to summarize."
},
"comparison_experiment_id": {
"type": "string",
"description": "Optional: the id of an experiment to compare against. If provided, the summary will include comparisons to this baseline."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Summarize experiment",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
The title clearly communicates the tool’s action and subject. Keep it concise.
Description82/100
Generate a direct web link (permalink) to a Braintrust object for sharing or bookmarking.
This tool creates URLs that link directly to projects, experiments, datasets, prompts, functions, or specific log entries in the Braintrust web application. Use this when users want to:
- Share a link to an experiment or dataset with teammates
- Create bookmarks to specific objects
- Generate URLs for external documentation or reports
- Navigate directly to specific log entries or traces
The tool accepts either object_id directly, or project_name + object_name to resolve the object first. Supply org_name to disambiguate names across organizations. For project_logs, project_patterns, project_prompts, and project_functions, project_name alone is sufficient; object_id must be the containing project UUID. For project_logs, put the root span ID from the query's root_span_id field in row_id, not a child span's id. For patterns, prompts, and scorers, put the resource ID in row_id.
Describe the returned information so the agent knows whether it will answer the request.
Inputs96/100
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
object_type
The type of object to create a permalink for
Remove validation rules already expressed in this field’s schema; describe its meaning and usage instead.
project_name
The project name. Required with object_name for most types. For project_logs, project_patterns, project_prompts, and project_functions, this alone is sufficient.
Clear enough to supply this argument.
org_name
Organization name used to disambiguate project and object names.
Clear enough to supply this argument.
object_name
The object name within the project (alternative to object_id). Not needed for project_logs, project_patterns, project_prompts, or project_functions.
Clear enough to supply this argument.
object_id
The UUID of the object (alternative to project_name + object_name). For project_logs, project_patterns, project_prompts, and project_functions, this must be the project UUID, not a project name or row ID.
Clear enough to supply this argument.
row_id
Optional: the root_span_id for a log entry or trace, or the resource ID for a pattern, prompt, or scorer. Keep it separate from object_id. For logs, do not pass a child span's id.
No output schema is declared. Add one for structured results so clients can interpret returned fields.
Not declared
Original tool metadata
{
"name": "generate_permalink",
"description": "Generate a direct web link (permalink) to a Braintrust object for sharing or bookmarking.\n\nThis tool creates URLs that link directly to projects, experiments, datasets, prompts, functions, or specific log entries in the Braintrust web application. Use this when users want to:\n- Share a link to an experiment or dataset with teammates\n- Create bookmarks to specific objects\n- Generate URLs for external documentation or reports\n- Navigate directly to specific log entries or traces\n\nThe tool accepts either object_id directly, or project_name + object_name to resolve the object first. Supply org_name to disambiguate names across organizations. For project_logs, project_patterns, project_prompts, and project_functions, project_name alone is sufficient; object_id must be the containing project UUID. For project_logs, put the root span ID from the query's root_span_id field in row_id, not a child span's id. For patterns, prompts, and scorers, put the resource ID in row_id.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"object_type"
],
"properties": {
"row_id": {
"type": "string",
"description": "Optional: the root_span_id for a log entry or trace, or the resource ID for a pattern, prompt, or scorer. Keep it separate from object_id. For logs, do not pass a child span's id."
},
"org_name": {
"type": "string",
"description": "Organization name used to disambiguate project and object names."
},
"object_id": {
"type": "string",
"description": "The UUID of the object (alternative to project_name + object_name). For project_logs, project_patterns, project_prompts, and project_functions, this must be the project UUID, not a project name or row ID."
},
"object_name": {
"type": "string",
"description": "The object name within the project (alternative to object_id). Not needed for project_logs, project_patterns, project_prompts, or project_functions."
},
"object_type": {
"enum": [
"experiment",
"dataset",
"prompt_session",
"playground_logs",
"project_logs",
"project_prompts",
"project_functions",
"project_patterns",
"audit_logs"
],
"type": "string",
"description": "The type of object to create a permalink for"
},
"project_name": {
"type": "string",
"description": "The project name. Required with object_name for most types. For project_logs, project_patterns, project_prompts, and project_functions, this alone is sufficient."
}
},
"additionalProperties": false
},
"annotations": {
"title": "Generate permalink",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}
Loading score analytics…
Put your MCP to the test
A few tasks worth trying with this server, chosen from its reviewed tools. These have not been run.
01Search documentationSearch Braintrust documentation to find relevant information and guidance.
02List recent objectsList recently created projects, experiments, datasets, prompts, or functions you have access to.
03Summarize experimentGet aggregated performance metrics for an experiment, optionally compared to a baseline.
04Run SQL query# Braintrust MCP `sql_query` tool
05Resolve objectConvert names to IDs (or vice versa) and parse Braintrust URLs.
Want to run these tasks against your MCP server with us?
api.braintrust.dev setup, tools, quality score, publishing readiness, and alternatives.
How do I add api.braintrust.dev to Claude, ChatGPT, or Cursor?
Add api.braintrust.dev to Claude, ChatGPT, or Cursor by connecting its remote MCP URL in each client's MCP settings. In Claude, add a custom connector. In ChatGPT, add an MCP app. In Cursor, add an MCP server under Settings. The MCP URL is https://api.braintrust.dev/mcp.
Is api.braintrust.dev MCP ready for ChatGPT and Claude?
As of October 14, 2026, api.braintrust.dev scores 96/100 on Manufact's MCP publishing checklist. The scan meets ChatGPT publishing checks and meets Claude publishing checks. Passing these checks does not guarantee marketplace approval.
How clear is api.braintrust.dev's MCP tool documentation?
As of October 14, 2026, api.braintrust.dev scores 74/100 in Manufact's review of tool titles, descriptions, input and output schemas, and behavior annotations. The review covers 45 of 45 discovered tools. The quality score measures documentation clarity; runtime behavior and marketplace readiness require separate checks.
What are the best alternatives to api.braintrust.dev?
As of October 14, 2026, alternatives to api.braintrust.dev in developer-tools on the Manufact MCP catalog include killfear-held-at-the-gate.netlify.app, AgentMail, Manufact Cloud, Authdog, and Apify MCP Server.
How can api.braintrust.dev improve its MCP documentation?
create_monitoring_view is the lowest-scoring reviewed tool at 61/100. No output schema is declared. Add one for structured results so clients can interpret returned fields. Open its review for feedback and a copyable fix prompt, then rerun the documentation review after updating your tool metadata.
Not your MCP?
Review any public or OAuth MCP server for protocol compatibility, tool documentation, and ChatGPT and Claude publishing readiness. No account required.