Implement a custom provider
Loom is designed to be provider-agnostic. While it comes with built-in support for OpenAI, Anthropic, Gemini, and Ollama, you can easily add support for any other LLM backend by implementing the llm.Provider interface.
1. The Provider Interface
To create a custom provider, you must implement the following interface:
type Provider interface { profile.Provider
Name() string Stream(context.Context, *Request) (StreamResponse, error)}Optional: Live Model Discovery (llm.ModelLister)
If your backend exposes endpoints to list or inspect models dynamically, implement the llm.ModelLister interface:
type ModelLister interface { ListModels(ctx context.Context) ([]profile.Profile, error) GetModel(ctx context.Context, id string) (profile.Profile, error)}If your provider also maintains a static catalog (staticProfiles), use profile.Merge(staticBase, liveOverlay) so live discovery preserves curated pricing and capability metadata.
Optional: Cache Management
If your provider supports explicit resource management for context caching, you can optionally implement the CacheManager interface:
type CacheManager interface { CreateCache(context.Context, *Request) (string, error) DeleteCache(context.Context, string) error}Optional: Deferred Tools Support
If your provider or backend model natively supports deferred tool definitions (server-side deferred tools or native tool search), implement the llm.DeferredToolsSupport interface:
type DeferredToolsSupport interface { SupportsDeferredTools(model string) bool}When implemented, llm.DeferredToolsMiddleware checks this method to avoid applying unnecessary client-side tool transformations on models that natively support deferred tools.
Optional: Inline Tools Support
If your provider or backend model natively supports mid-conversation inline tool definitions (*message.ToolDefinitionBlock), implement the llm.InlineToolsSupport interface:
type InlineToolsSupport interface { SupportsInlineTools(model string) bool}When implemented, llm.InlineToolsMiddleware checks this method to avoid applying unnecessary client-side tool transformations on models that natively support inline tool injection.
Optional: Idle Notifications & Sync-on-Write (llm.IdleMessageNotifier, llm.OutOfSyncError)
If your provider wraps a stateful session or subprocess that can change state while idle (e.g., background compaction or session termination), implement llm.IdleMessageNotifier to push message.Message notifications and return *llm.OutOfSyncError from Stream() when sync-on-write detects drift:
type IdleMessageNotifier interface { OnIdleMessage(fn IdleMessageHandler) (unsubscribe func())}
type OutOfSyncError struct { Provider string Reason string Messages message.MessageList Err error}Optional: Extensions
If your provider has specialized features, you should define your own extension structs.
- Define the struct: It must implement either
llm.Extensionormessage.Extension. - Implement
ExtensionID(): Return a unique string (e.g.myprovider.feature). - Register (Message level only): If it’s a message-level extension, call
message.RegisterExtensionin yourinit()function so it can be restored from JSON.
func init() { message.RegisterExtension(func() message.Extension { return &MyMsgExt{} })}2. Implementing the Stream Method
The core of a provider is the Stream method. It takes a generic llm.Request and returns an iterator over message.AssistantChunk values.
func (p *MyCustomProvider) Stream(ctx context.Context, req *llm.Request) (llm.StreamResponse, error) { // 1. Translate llm.Request to your provider's specific API format apiReq := translate(req)
// 2. Call your backend's streaming API resp, _ := p.client.CallStreaming(ctx, apiReq)
// 3. Return an iterator return func(yield func(message.AssistantChunk, error) bool) { for { chunk, err := resp.Next() if err != nil { yield(message.AssistantChunk{}, err) return }
// 4. Translate back to Loom's AssistantChunk loomChunk := translateBack(chunk) if !yield(loomChunk, nil) { return } } }, nil}3. Mapping Messages & Tools
Most of the work in creating a provider involves mapping between Loom’s block-based message system and the provider’s specific message format (e.g., Anthropic’s ContentBlock vs OpenAI’s ToolCall).
Key Mapping Areas:
- Multimodal Content & Strict Validation: Map supported media blocks (
ImageBlock,AudioBlock,VideoBlock,DocumentBlock) to your provider’s base64 or URL wire format, and never silently drop unsupported media blocks, roles, or empty attachments. Validate against known model profiles usingprof.ValidateMessagesModalities/prof.ValidateToolCallCapabilityand returnprofile.NewUnsupportedModalityError(...)(profile.ErrUnsupportedModality) for any media block your wire protocol cannot represent. Keep historical*message.ThinkingBlockblocks lenient if your backend does not accept prior reasoning blocks on replay. - Tool Definitions: Translating
tool.Definitioninto the provider’s JSON schema format. - Tool Calls: Capturing tool call IDs and arguments from the stream.
4. Registering Your Provider
Once implemented, you can register your provider with the llm.Registry to make it available to the rest of your application.
registry := llm.NewRegistry()registry.Register("my-provider", func() (llm.Provider, error) { return &MyCustomProvider{}, nil})Summary
- Implementing a custom provider allows Loom to talk to any internal or specialized LLM API.
- Focus on accurate mapping of
llm.Requestandmessage.AssistantChunk. - Use the
providers/openaiorproviders/anthropicpackages as reference implementations.