Skip to content

Implement a custom provider

Loom is designed to be provider-agnostic. While it comes with built-in support for OpenAI, Anthropic, Gemini, and Ollama, you can easily add support for any other LLM backend by implementing the llm.Provider interface.

1. The Provider Interface

To create a custom provider, you must implement the following interface:

type Provider interface {
profile.Provider
Name() string
Stream(context.Context, *Request) (StreamResponse, error)
}

Optional: Live Model Discovery (llm.ModelLister)

If your backend exposes endpoints to list or inspect models dynamically, implement the llm.ModelLister interface:

type ModelLister interface {
ListModels(ctx context.Context) ([]profile.Profile, error)
GetModel(ctx context.Context, id string) (profile.Profile, error)
}

If your provider also maintains a static catalog (staticProfiles), use profile.Merge(staticBase, liveOverlay) so live discovery preserves curated pricing and capability metadata.

Optional: Cache Management

If your provider supports explicit resource management for context caching, you can optionally implement the CacheManager interface:

type CacheManager interface {
CreateCache(context.Context, *Request) (string, error)
DeleteCache(context.Context, string) error
}

Optional: Deferred Tools Support

If your provider or backend model natively supports deferred tool definitions (server-side deferred tools or native tool search), implement the llm.DeferredToolsSupport interface:

type DeferredToolsSupport interface {
SupportsDeferredTools(model string) bool
}

When implemented, llm.DeferredToolsMiddleware checks this method to avoid applying unnecessary client-side tool transformations on models that natively support deferred tools.

Optional: Inline Tools Support

If your provider or backend model natively supports mid-conversation inline tool definitions (*message.ToolDefinitionBlock), implement the llm.InlineToolsSupport interface:

type InlineToolsSupport interface {
SupportsInlineTools(model string) bool
}

When implemented, llm.InlineToolsMiddleware checks this method to avoid applying unnecessary client-side tool transformations on models that natively support inline tool injection.

Optional: Idle Notifications & Sync-on-Write (llm.IdleMessageNotifier, llm.OutOfSyncError)

If your provider wraps a stateful session or subprocess that can change state while idle (e.g., background compaction or session termination), implement llm.IdleMessageNotifier to push message.Message notifications and return *llm.OutOfSyncError from Stream() when sync-on-write detects drift:

type IdleMessageNotifier interface {
OnIdleMessage(fn IdleMessageHandler) (unsubscribe func())
}
type OutOfSyncError struct {
Provider string
Reason string
Messages message.MessageList
Err error
}

Optional: Extensions

If your provider has specialized features, you should define your own extension structs.

  1. Define the struct: It must implement either llm.Extension or message.Extension.
  2. Implement ExtensionID(): Return a unique string (e.g. myprovider.feature).
  3. Register (Message level only): If it’s a message-level extension, call message.RegisterExtension in your init() function so it can be restored from JSON.
func init() {
message.RegisterExtension(func() message.Extension { return &MyMsgExt{} })
}

2. Implementing the Stream Method

The core of a provider is the Stream method. It takes a generic llm.Request and returns an iterator over message.AssistantChunk values.

func (p *MyCustomProvider) Stream(ctx context.Context, req *llm.Request) (llm.StreamResponse, error) {
// 1. Translate llm.Request to your provider's specific API format
apiReq := translate(req)
// 2. Call your backend's streaming API
resp, _ := p.client.CallStreaming(ctx, apiReq)
// 3. Return an iterator
return func(yield func(message.AssistantChunk, error) bool) {
for {
chunk, err := resp.Next()
if err != nil {
yield(message.AssistantChunk{}, err)
return
}
// 4. Translate back to Loom's AssistantChunk
loomChunk := translateBack(chunk)
if !yield(loomChunk, nil) {
return
}
}
}, nil
}

3. Mapping Messages & Tools

Most of the work in creating a provider involves mapping between Loom’s block-based message system and the provider’s specific message format (e.g., Anthropic’s ContentBlock vs OpenAI’s ToolCall).

Key Mapping Areas:

  • Multimodal Content & Strict Validation: Map supported media blocks (ImageBlock, AudioBlock, VideoBlock, DocumentBlock) to your provider’s base64 or URL wire format, and never silently drop unsupported media blocks, roles, or empty attachments. Validate against known model profiles using prof.ValidateMessagesModalities / prof.ValidateToolCallCapability and return profile.NewUnsupportedModalityError(...) (profile.ErrUnsupportedModality) for any media block your wire protocol cannot represent. Keep historical *message.ThinkingBlock blocks lenient if your backend does not accept prior reasoning blocks on replay.
  • Tool Definitions: Translating tool.Definition into the provider’s JSON schema format.
  • Tool Calls: Capturing tool call IDs and arguments from the stream.

4. Registering Your Provider

Once implemented, you can register your provider with the llm.Registry to make it available to the rest of your application.

registry := llm.NewRegistry()
registry.Register("my-provider", func() (llm.Provider, error) {
return &MyCustomProvider{}, nil
})

Summary

  • Implementing a custom provider allows Loom to talk to any internal or specialized LLM API.
  • Focus on accurate mapping of llm.Request and message.AssistantChunk.
  • Use the providers/openai or providers/anthropic packages as reference implementations.