Back to Blog
AICloudflareEngineering

Deploy AI to the Edge: LLMs on Cloudflare Workers

9 min read

Most AI applications are deployed in a single region. Your users in Tokyo, São Paulo, and Cairo are all hitting the same server in Virginia. That's a problem.

Edge computing changes this. By running AI inference close to your users, you get faster responses, lower costs, and better reliability. Cloudflare Workers makes this possible.

Why Edge AI Matters

Traditional AI deployment:

  • User in Dubai sends request → travels to US-East → processes → travels back
  • Round-trip latency: 200-400ms just for the network
  • Single point of failure
  • Scaling means bigger servers in one location

    • Edge AI deployment:

  • User in Dubai sends request → nearest Cloudflare PoP processes it
  • Round-trip latency: 10-50ms for the network
  • 300+ locations worldwide
  • Automatic scaling at every edge location

    • What You Can Build

    1. AI Gateway

    Use Workers as an intelligent proxy between your frontend and AI providers:

  • Route requests to the cheapest/fastest provider
  • Cache common responses at the edge
  • Rate limit by IP with zero latency overhead
  • Log and monitor all AI usage

    • 2. RAG at the Edge

    Combine Workers with Vectorize (Cloudflare's vector database) and Workers AI:

  • Store embeddings in Vectorize
  • Run similarity search at the edge
  • Generate responses using Workers AI or proxy to Claude/GPT
  • Cache frequently asked questions in KV

    • 3. AI Agents

    Build agents that run entirely on the edge:

  • Durable Objects for maintaining agent state
  • Workers AI for local inference
  • D1 for conversation history
  • KV for tool configurations

    • The Architecture

    Here's the stack I use for production AI applications on Cloudflare:

  • Workers — API layer, routing, orchestration
  • Workers AI — Local inference for supported models
  • Vectorize — Vector storage and similarity search
  • D1 — Structured data (users, conversations, metadata)
  • KV — Configuration, caching, rate limiting
  • R2 — Document storage for RAG pipelines
  • Durable Objects — Stateful agents and real-time features

    • Performance Results

    In a recent project, moving from a centralized deployment to Cloudflare Workers:

  • P50 latency dropped from 340ms to 89ms
  • P99 latency dropped from 1.2s to 280ms
  • Availability went from 99.9% to 99.99%
  • Cost decreased by 40% (fewer compute-heavy servers)

    • Limitations to Know

  • CPU time limits — Workers have a 30s CPU limit on paid plans. Long-running AI tasks need Durable Objects or queues.
  • Bundle size — 10 MiB for paid plans. Large ML models won't fit — use Workers AI or external APIs.
  • Cold starts — Workers have near-zero cold starts, but Durable Objects can take 50-100ms on first request.
  • Workers AI model selection — Limited compared to hosted solutions. Great for embeddings and small models, use external APIs for frontier models.

    • When to Use Edge AI

    Edge AI makes sense when:

  • Your users are globally distributed
  • Latency directly impacts user experience
  • You need to process data close to where it's generated
  • You want to reduce costs on AI API calls through edge caching

    • It doesn't make sense when:

  • You need large model fine-tuning
  • Your workload requires GPU-heavy inference
  • Your users are all in one region

    • Getting Started

    The fastest way to start:

  • Create a Cloudflare account (free tier is generous)
  • Set up a Workers project with Wrangler
  • Use Workers AI for embeddings and small models
  • Proxy to Claude or GPT for complex reasoning
  • Add Vectorize for semantic search
  • Deploy and iterate

    • The edge is where AI applications are heading. The companies building there now will have a massive advantage in latency, cost, and reliability.