How It Works
When a tool needs an LLM response, it can use theContext to send a sampling request to the client. The client’s configured LLM processes the request and returns the result to the tool.
Basic Usage
Use thesample() method on the context object to request LLM completions:
Use Case Example
A content moderation server that uses the client’s LLM to analyze user-generated content:Important Notes
- The client must provide a
sampling_callbackto support sampling requests - If no callback is configured, sampling requests will fail with an error
- Sampling requests are processed by the client’s configured LLM, not the server
Next Steps
- See Context API for more context methods
- Learn about Elicitation for requesting user input