Stream and configure responses
Use the model and messages from your first model call. To display text as it arrives, replace the blocking Invoke call with Stream.
chunks, err := model.Stream(ctx, messages)if err != nil { return err}for chunk, err := range chunks { if err != nil { return err } fmt.Print(message.Content(chunk.Content).Text())}Set generation options
The fluent API returns a configured model. Choose options supported by the model you selected.
model = model.WithTemperature(0.7). WithMaxTokens(1000). WithStop("User:", "Assistant:")Structured Output (JSON Schema)
If a provider supports it, you can enforce a specific JSON schema for the model’s response.
model = model.WithStructuredOutput(myJsonSchema)Thinking (Reasoning) Mode
For models that support “thinking” or extended reasoning (like Anthropic Claude or Google Gemini), you can configure the reasoning budget.
model = model.WithThinking(4000) // 4000 tokens for thinkingExtensions & Hints
Providers often have specialized features (like caching) that aren’t part of the common LLM parameters. Loom uses a type-safe Extensions system to pass these hints.
Extensions come in two flavors:
- Call Extensions (
llm.Extension): Apply to the entire request (e.g., enable system prompt caching). - Message Extensions (
message.Extension): Apply to a specific turn in the conversation (e.g., mark a checkpoint).
// Example: Setting a call-level extension for Anthropicmodel = model.WithExtension(loomanthropic.PromptCaching{CacheHeader: true})
// Example: Passing a call-level extension for a single callmodel.Invoke(ctx, msgs, llm.WithExtensionOption(loomopenai.PromptCache{ Key: "user-session-123",}))