Class CloudflareKVCache

Represents a specific implementation of a caching mechanism using Cloudflare KV as the underlying storage system. It extends the BaseCache class and overrides its methods to provide the Cloudflare KV-specific logic.

Example

// Example of using OpenAI with Cloudflare KV as cache in a Cloudflare Worker
const cache = new CloudflareKVCache(env.KV_NAMESPACE);
const model = new ChatAnthropic({
  cache,
});
const response = await model.invoke("How are you today?");
return new Response(JSON.stringify(response), {
  headers: { "content-type": "application/json" },
});

Hierarchy

  • BaseCache
    • CloudflareKVCache

Constructors

constructor

Methods

lookup update

Constructors

constructor

Methods

lookup

  • lookup(prompt, llmKey): Promise<null | Generation[]>

  • Retrieves data from the cache. It constructs a cache key from the given prompt and llmKey, and retrieves the corresponding value from the Cloudflare KV namespace.

    Parameters

    • prompt: string

      The prompt used to construct the cache key.

    • llmKey: string

      The LLM key used to construct the cache key.

    Returns Promise<null | Generation[]>

    An array of Generations if found, null otherwise.

update

  • update(prompt, llmKey, value): Promise<void>

  • Updates the cache with new data. It constructs a cache key from the given prompt and llmKey, and stores the value in the Cloudflare KV namespace.

    Parameters

    • prompt: string

      The prompt used to construct the cache key.

    • llmKey: string

      The LLM key used to construct the cache key.

    • value: Generation[]

      The value to be stored in the cache.

    Returns Promise<void>

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc