CDN Architecture & Workflow — A Deep Dive

Aug 2024 | 12 min read | CDN Networking Distributed Systems

Content Delivery Networks (CDNs) are one of the most consequential yet least visible pieces of internet infrastructure. When you stream a video, load a webpage in under a second from halfway around the world, or make a software update download in moments — a CDN is doing the heavy lifting. At Oracle Cloud Infrastructure, I was directly involved in designing and developing APIs and services for the CDN control plane. In this post I'll break down the architecture, the request lifecycle, caching strategies, and the design decisions that make a CDN work at global scale.

What Problem Does a CDN Solve?

The internet is physically constrained by the speed of light and network hop counts. A server in New York serving content to a user in Mumbai adds 150–200 ms of round-trip latency — before a single byte of content is transferred. Multiply this by dozens of resources on a modern page, and the user experience degrades severely. CDNs solve this by placing content closer to users through a network of geographically distributed Points of Presence (PoPs).

Key Insight: A CDN doesn't make your origin server faster — it reduces how often users need to reach your origin server at all. Cache hit rate is the primary metric.

High-Level Architecture

A CDN consists of three logical tiers that work in concert:

  ┌─────────────────────────────────────────────────────────────────────────┐
  │                        GLOBAL CDN ARCHITECTURE                          │
  └─────────────────────────────────────────────────────────────────────────┘

  ┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐
  │  User    │   │  User    │   │  User    │   │  User    │
  │ (Asia)   │   │ (Europe) │   │  (US)    │   │(LatAm)   │
  └────┬─────┘   └────┬─────┘   └────┬─────┘   └────┬─────┘
       │              │              │              │
       ▼ DNS Anycast  ▼              ▼              ▼
  ┌─────────────────────────────────────────────────────────┐
  │                   DNS RESOLUTION LAYER                   │
  │        (GeoDNS routes user to nearest PoP)              │
  └──────────────────────────┬──────────────────────────────┘
                             │
       ┌──────────┬──────────┼──────────┬──────────┐
       ▼          ▼          ▼          ▼          ▼
  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
  │PoP      │ │PoP      │ │PoP      │ │PoP      │ │PoP      │
  │Singapore│ │Frankfurt│ │Ashburn  │ │Mumbai   │ │São Paulo│
  │ EDGE    │ │ EDGE    │ │ EDGE    │ │ EDGE    │ │ EDGE    │
  │ CACHE   │ │ CACHE   │ │ CACHE   │ │ CACHE   │ │ CACHE   │
  └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘
       │           │           │           │           │
       └───────────┴───────────┼───────────┴───────────┘
                               │  Cache MISS → forward to mid-tier
                               ▼
  ┌────────────────────────────────────────────────────────────┐
  │                   MID-TIER / SHIELD LAYER                  │
  │  (Regional aggregation cache — shields origin from spikes) │
  │                                                            │
  │   ┌──────────────┐         ┌──────────────┐               │
  │   │ Shield Node  │         │ Shield Node  │               │
  │   │   (US-East)  │         │  (EU-West)   │               │
  │   └──────┬───────┘         └──────┬───────┘               │
  └──────────┼──────────────────────────┼─────────────────────┘
             │  Cache MISS              │
             └──────────────┬───────────┘
                            │
                            ▼
  ┌────────────────────────────────────────────────────────────┐
  │                      ORIGIN LAYER                          │
  │                                                            │
  │   ┌──────────────┐  ┌──────────────┐  ┌──────────────┐   │
  │   │  App Server  │  │  App Server  │  │  App Server  │   │
  │   │  (Primary)   │  │  (Primary)   │  │  (Standby)   │   │
  │   └──────────────┘  └──────────────┘  └──────────────┘   │
  │         │                  │                              │
  │   ┌─────▼──────────────────▼─────┐                       │
  │   │    Object / Block Storage     │                       │
  │   │   (S3, OCI Object Storage)    │                       │
  │   └──────────────────────────────┘                       │
  └────────────────────────────────────────────────────────────┘

Figure 1: Three-tier CDN architecture — Edge PoPs, Mid-tier Shield, and Origin

The Request Lifecycle — Step by Step

Let's trace a user in Singapore requesting https://example.com/images/hero.png:

  USER (Singapore)
     │
     │ 1. DNS lookup: example.com
     ▼
  ┌─────────────────────────────────────────────────────────┐
  │  AUTHORITATIVE DNS (GeoDNS / Anycast)                   │
  │  → Detects user's IP is in APAC                         │
  │  → Returns IP of Singapore PoP edge node                │
  └─────────────────────────────────────────────────────────┘
     │ 2. TCP + TLS handshake to edge node (~5ms, same city)
     ▼
  ┌──────────────────────────────────────────────────────────────────┐
  │  EDGE NODE — Singapore PoP                                        │
  │                                                                  │
  │  3. Check local RAM cache (L1) ──────────────── HIT? ──► serve  │
  │     └── MISS                                                     │
  │  4. Check local SSD cache (L2) ──────────────── HIT? ──► serve  │
  │     └── MISS → forward upstream                                  │
  │                                                                  │
  │  Request Collapse: if 500 concurrent requests for same URL,      │
  │  only ONE is sent upstream. Others wait for the first response.  │
  └──────────────────────────────────────────────────────────────────┘
     │ 5. Cache MISS — forward to mid-tier shield (same region)
     ▼
  ┌──────────────────────────────────────────────────────────────────┐
  │  SHIELD NODE — APAC Regional                                      │
  │                                                                  │
  │  6. Check shield cache ─────────────────── HIT? ──► serve back  │
  │     └── MISS                                                     │
  │  7. One request sent to origin (shields origin from stampede)    │
  └──────────────────────────────────────────────────────────────────┘
     │ 8. Cache MISS — request reaches origin
     ▼
  ┌─────────────────────────────────────────────────────────┐
  │  ORIGIN SERVER                                          │
  │  → Fetches hero.png from object storage                 │
  │  → Returns response with Cache-Control headers:         │
  │    Cache-Control: public, max-age=86400, s-maxage=3600  │
  └─────────────────────────────────────────────────────────┘
     │ 9. Response travels back through the chain
     ▼
  Shield stores copy → Edge stores copy → User receives content

  ┌──────────────────────────────────────────────────────────┐
  │ Next request for same asset from any APAC user:          │
  │ → Edge cache HIT → served in <2ms with zero origin load  │
  └──────────────────────────────────────────────────────────┘

Figure 2: Full request lifecycle — DNS resolution through cache hierarchy to origin

Cache Key Design

The cache key determines what constitutes a "unique" cached object. Poor cache key design is one of the most common CDN misconfiguration problems.

Default Cache Key Components

Scheme: https://
Host: example.com
Path: /images/hero.png
Query string: optionally included (often stripped for static assets)

Advanced CDNs allow Vary header normalization — for example, accepting both Accept-Encoding: gzip and Accept-Encoding: br and caching two variants. Without normalization, every unique Accept-Encoding string produces a different cache entry, destroying hit rates.

Caching Strategies

Strategy	How It Works	Best For
TTL-Based (max-age)	Object is served from cache until TTL expires, then re-fetched from origin	Static assets — images, JS, CSS with versioned filenames
Stale-While-Revalidate	Serve stale content immediately; revalidate in background	APIs where slightly stale data is acceptable
Surrogate Keys / Cache Tags	Tag objects with logical keys; purge entire tag groups instantly	CMS pages, product pages that share components
Conditional GET (ETag / Last-Modified)	CDN revalidates with origin using 304 Not Modified responses	Frequently changing but bandwidth-heavy objects

Edge Logic — Moving Compute to the PoP

Modern CDNs are no longer just caches. Edge computing capabilities let you run code at PoPs, enabling:

  ┌──────────────────────────────────────────────────────────────────┐
  │                      EDGE PoP — Request Pipeline                  │
  │                                                                  │
  │  Incoming Request                                                │
  │       │                                                          │
  │       ▼                                                          │
  │  ┌─────────────────┐                                            │
  │  │  WAF / DDoS     │ ← Block malicious traffic at edge          │
  │  │  Filtering      │   (saves origin bandwidth)                 │
  │  └────────┬────────┘                                            │
  │           ▼                                                      │
  │  ┌─────────────────┐                                            │
  │  │  Edge Function  │ ← A/B testing, auth token validation,      │
  │  │  (JavaScript /  │   geo-routing, header rewriting,           │
  │  │   Lua / WASM)   │   feature flags                            │
  │  └────────┬────────┘                                            │
  │           ▼                                                      │
  │  ┌─────────────────┐                                            │
  │  │  Cache Lookup   │ ── HIT ──────────────────────► Response    │
  │  └────────┬────────┘                                            │
  │           │ MISS                                                 │
  │           ▼                                                      │
  │  ┌─────────────────┐                                            │
  │  │  Origin Fetch   │ ← Load-balanced across origin pool         │
  │  └────────┬────────┘                                            │
  │           ▼                                                      │
  │  ┌─────────────────┐                                            │
  │  │  Response       │ ← Image optimization, compression,         │
  │  │  Transformation │   protocol upgrade (HTTP/1.1 → H/2),       │
  │  └────────┬────────┘   minification                             │
  │           ▼                                                      │
  │       Response to User                                           │
  └──────────────────────────────────────────────────────────────────┘

Figure 3: Edge PoP request pipeline showing WAF, edge functions, cache, and response transforms

Health Checking & Failover

One of the most critical control-plane responsibilities is detecting origin health and rerouting traffic. At OCI's CDN and Load Balancer teams, we implemented:

Active health checks: Periodic synthetic requests to each origin backend; failures trigger removal from the pool
Passive health checks: Monitor real traffic error rates; backends with sustained 5xx rates are temporarily ejected
Automatic failover: Secondary origin or backup CDN edge pool activated within seconds of primary failure
Graceful degradation: Serve stale cache content when origin is completely unreachable (configured via stale-if-error)

Performance Optimizations Built Into CDNs

Connection Optimizations

Connection pooling: Edge nodes maintain persistent keep-alive connections to origin — eliminates TCP handshake overhead per request
HTTP/2 & HTTP/3: Multiplex many requests over a single connection; particularly valuable for browser-to-edge traffic
TLS session resumption: Reuse TLS sessions to skip the full handshake for repeat visitors
TCP fast open: Send data in the SYN packet, reducing round trips for first request

Key Metrics to Monitor

Metric	Target	What It Tells You
Cache Hit Ratio	> 90%	Efficiency of cache — low ratio means origin is overloaded
TTFB (Time to First Byte)	< 50ms (cache hit)	End-user perceived latency
Origin Error Rate	< 0.1%	Reliability of backend services
Bandwidth Offload	> 85%	% of traffic served from edge vs. origin
Purge Latency	< 5 seconds globally	How fast invalidations propagate

Lessons from Building CDN Control Plane at OCI

Building the control plane APIs — the management layer that allows customers to configure CDN behavior — taught me several things that aren't obvious from the outside:

Configuration propagation is hard. Pushing a configuration change to thousands of PoPs consistently and atomically requires careful versioning and rollback strategies.
Cache invalidation is not instant. "Eventual consistency" in purge propagation is a real problem. Design your cache key strategy to rely on versioned URLs (cache busting) rather than frequent purges.
Observability must be edge-aware. Standard APM tools miss edge-layer metrics. You need distributed tracing that captures PoP → shield → origin spans.
Security and CDN are inseparable. CDN is the first line of defense — WAF, rate limiting, and bot detection all belong at the edge layer, not deep in the backend.

CDNs are a fascinating layer of internet infrastructure where distributed systems theory, networking, and real-world engineering trade-offs all converge. Understanding them deeply makes you a better architect, regardless of what layer you normally work in.

Written by Sudhir Kumar Tiwari — Senior Engineer, formerly at Oracle Cloud Infrastructure (CDN & Load Balancer team)

Back to Portfolio