Manage caches and stateful sessions
Some providers can reuse a cached prompt or keep a conversation open between turns. Choose the operation your provider supports.
Reuse a prompt cache
For providers that support explicit resource management (like Google Gemini’s Context Caching), Loom provides a CacheManager API.
// 1. Create a long-lived cache from a list of messagescacheID, err := model. WithExtension(loomgenai.CacheCreation{DisplayName: "Project Docs"}). CreateCache(ctx, documentMessages)
// 2. Use the cache ID in subsequent callsresp, _ := model.Invoke(ctx, prompt, llm.WithExtensionOption(loomgenai.ContextCache{ID: cacheID}))
// 3. Delete when no longer neededmodel.DeleteCache(ctx, cacheID)Reconcile conversation history
Stateful providers (such as providers/claudecode and providers/copilot) can push unsolicited message.Message events while idle between turns via llm.IdleMessageNotifier and reject out-of-sync writes with llm.OutOfSyncError:
if notifier, ok := provider.(llm.IdleMessageNotifier); ok { unsub := notifier.OnIdleMessage(func(msg message.Message) { // Promptly enqueue or react to idle updates (e.g. background compaction) }) defer unsub()}
resp, err := model.Invoke(ctx, messages)var syncErr *llm.OutOfSyncErrorif errors.As(err, &syncErr) && len(syncErr.Messages) > 0 { // Reconcile local conversation history using syncErr.Messages and resubmit}