Digital Asset Ingestion, R2 Storage Boundaries, and Edge Image Pipeline

Digital Asset Ingestion, R2 Storage Boundaries, and Edge Image Pipeline

Document Status: 🟢 Approved & Active True Specification
Target Audience: Core Platform Architects, Edge Infrastructure Engineers, CMS Developers, and Autonomous AI Agent Contributors
Governing Epic: Epic #200 (epic(media): digital asset ingestion, r2 storage boundaries, and edge responsive image pipeline)
Implementation Tracking: Issue #195 (Spec), Issue #197 (Infra), Issue #198 (Components), Issue #278 (CMS Pipeline & Linter), Issue #277 (Future Video/Stream)
Companion Specifications: HIGH_LEVEL_DESIGN.md, docs/CLIENT_CMS.md, docs/DATA_ISOLATION_AND_STORAGE.md, docs/specs/CLIENT_APPLICATION_ANATOMY.md, docs/specs/CONTENT_INVALIDATION_AND_CACHE_FRESHNESS.md

---

1. Executive Summary & The Media Invariant

Small business websites are intensely visual. High-resolution photography of fresh artisan pastries, auto repair facilities, team portraits, and seasonal menus drive consumer trust and commercial conversion. However, high-resolution media is historically the #1 cause of degraded web performance, mobile data bloat, runaway bandwidth bills, and jarring layout shifts.

In standard agency web operations, a non-technical client uploads an uncompressed 8 MB JPEG straight from an iPhone 16 Pro camera (48 megapixels). Without automated edge infrastructure, serving that raw file directly to mobile visitors results in:

1. Severe Bandwidth & Egress Penalties: Traditional cloud providers (AWS S3) charge up to $0.09/GB for data egress. A viral local blog post or food review can cost hundreds of dollars in unexpected bandwidth egress fees alone.
2. Degraded Mobile Experience: Cellular visitors on 4G LTE face 4 to 8-second page load times, poor Largest Contentful Paint (LCP > 4.5s), and severe battery drain.
3. Cumulative Layout Shift (CLS): Unconstrained <img> elements without explicit intrinsic aspect ratios cause the browser to jump and re-render the page layout as image dimensions resolve, destroying user experience and Google SEO rankings.

flowchart TD
    subgraph ClientCMS["1. Client Intake & CMS Studio (iPhone / Desktop)"]
        Upload["Client uploads uncompressed 8 MB photo\n(iPhone 48MP JPEG/HEIC)"]
        IngestEngine["EmDash CMS / @siteswarm/media Ingestion"]
        Upload --> IngestEngine
    end

    subgraph StorageLayer["2. Cloudflare R2 Storage Tier"]
        R2["Cloudflare R2 Object Storage\n(Canonical Raw Master Vault)\n• Content-Addressed UUID Key\n• Cache-Control: public, max-age=31536000, immutable\n• $0 Egress Bandwidth Fees"]
        IngestEngine -->|Store Master Asset| R2
    end

    subgraph EdgePipeline["3. Cloudflare Images Edge Pipeline (/cdn-cgi/image/)"]
        Transform["Edge Transform & Optimization Engine\n• Format Conversion: AVIF (preferred) & WebP\n• Responsive Widths: 320, 640, 960, 1280, 1920\n• Visual Quality Tuning: q=80-85\n• Automatic Edge Caching in Cloudflare Anycast CDN"]
        R2 -->|Stream Origin Bytes| Transform
    end

    subgraph VisitorBrowser["4. Visitor Browser Delivery"]
        PictureTag["Zero-Layout-Shift <Picture /> Component\n• Strict Aspect-Ratio CSS Container\n• CSS Shimmer / Native Progressive Rendering\n• loading='lazy' & decoding='async' by Default\n• Max 1 Hero Priority Image per Page"]
        Transform --> PictureTag
    end

The Three Foundational Media Invariants:

1. The Zero-Egress Storage Invariant: Master raw media files are stored exclusively in Cloudflare R2 with zero egress fees. The platform never charges clients or absorbs bandwidth penalties when public traffic surges.
2. The Zero-Layout-Shift (0.00 CLS) Invariant: All images rendered across client applications must be wrapped in strict aspect-ratio containers matching the image’s intrinsic dimensions. No image load may ever displace neighboring textual content or navigation landmarks.
3. The Mandatory Lazy-Loading Invariant: All off-screen media assets must load lazily (loading="lazy" and decoding="async"). At most one above-the-fold hero image per page may be marked priority (loading="eager"). Unmanaged raw <img> tags are strictly prohibited by the architectural governance linter.

---

2. Storage Boundaries & R2 Ingestion Architecture

2.1 R2 Bucket Partitioning & Lifecycle Strategy

Client applications within the SiteSwarm fleet leverage dedicated R2 bucket bindings or structured multi-tenant paths depending on client commercial tier:

• Bucket Naming Convention: siteswarm-media-<env>-<client-slug> (e.g., siteswarm-media-prod-bakery or centralized siteswarm-media-production).
• Object Key Structure: Content-addressed, collision-free object paths:

  uploads/<collection>/<uuid-or-content-hash>/<sanitized-filename>.<ext>

  Example: uploads/pastries/a9e72f10-c92f-48d5-b0b3-9c5957b98d3c/sourdough-boule.jpg
• Key Determinism: Because every uploaded or updated image receives a cryptographically unique UUID or hash prefix, filenames are immutable. Overwrites never corrupt active CDN cache entries.

2.2 Presigned Upload & CMS Ingestion

In accordance with docs/CLIENT_CMS.md and ADR-0003:

• Worker B Privileged Isolation: The R2 storage binding (storage: r2({ binding: "MEDIA" })) is attached strictly to Worker B (CMS editing engine). Worker A (public static frontend) has zero direct write credentials.
• Client Direct Streaming: Uploads stream directly from the client browser into R2 via standard multipart form or presigned PUT operations, completely bypassing public SSR worker memory buffers and CPU limits.

2.3 Immutable HTTP Caching Policy

Because object keys in R2 are content-addressed and immutable, all media assets are served with permanent client-side and CDN caching headers:

Cache-Control: public, max-age=31536000, immutable

• Browser Behavior: The browser stores the asset in its local disk/memory cache for up to 365 days. Subsequent page navigations render the image with 0ms latency and 0 network requests.
• Edge Behavior: Cloudflare Anycast edge caches retain transformed image renditions permanently, avoiding repetitive compute transformations on the origin.

---

3. Cloudflare Images Edge Transformation Pipeline

SiteSwarm standardizes on Cloudflare Images Zone Transformations (/cdn-cgi/image/) integrated with Cloudflare R2 custom domain delivery.

3.1 Transformation URL Breakdown

All image rendering requests resolve through the Cloudflare Images URL transformation grammar:

https://<domain>/cdn-cgi/image/<options>/<source-path>

Where <options> is a comma-delimited string specifying exact transformation parameters:

• format=auto: Evaluates the incoming browser’s Accept HTTP header. Serves next-generation AVIF to modern Chromium and Safari browsers, falls back to WebP, and gracefully serves original JPEG/PNG to legacy agents.
• width=<N>: Resizes the raster asset to target pixel dimensions with bicubic downscaling.
• quality=80 or quality=85: Balances pristine visual fidelity with minimal byte size. Downscaling 48MP photos to a 640px WebP rendition at q=80 typically achieves a 96% to 98% file size reduction (from 8 MB down to ~65 KB).
• fit=scale-down: Ensures images never upscale past their intrinsic dimensions.
• metadata=none or metadata=copyright: Strips unneeded EXIF, GPS location tags, and camera metadata while preserving creator copyright.

3.2 Dynamic Responsive Source Sets (srcset)

The headless media engine generates standardized responsive breakpoints optimized for mobile, tablet, and high-DPI desktop displays:

Breakpoint Tier	Target Viewport Width	Typical Rendition Size (AVIF/WebP)
Mobile Portrait	360px – 480px	18 KB – 35 KB
Mobile Landscape / Tablet	640px – 768px	45 KB – 80 KB
Desktop / Laptop	1024px – 1280px	95 KB – 160 KB
Retina / 2x Displays	1920px	180 KB – 290 KB

<picture class="relative block overflow-hidden rounded-xl bg-slate-100 dark:bg-slate-800">
  <!-- AVIF next-gen format -->
  <source
    type="image/avif"
    srcset="
      /cdn-cgi/image/width=360,format=avif,quality=80/uploads/pastry.jpg 360w,
      /cdn-cgi/image/width=640,format=avif,quality=80/uploads/pastry.jpg 640w,
      /cdn-cgi/image/width=1024,format=avif,quality=80/uploads/pastry.jpg 1024w,
      /cdn-cgi/image/width=1920,format=avif,quality=80/uploads/pastry.jpg 1920w
    "
    sizes="(max-width: 640px) 100vw, (max-width: 1024px) 50vw, 33vw"
  />
  <!-- WebP fallback -->
  <source
    type="image/webp"
    srcset="
      /cdn-cgi/image/width=360,format=webp,quality=80/uploads/pastry.jpg 360w,
      /cdn-cgi/image/width=640,format=webp,quality=80/uploads/pastry.jpg 640w,
      /cdn-cgi/image/width=1024,format=webp,quality=80/uploads/pastry.jpg 1024w,
      /cdn-cgi/image/width=1920,format=webp,quality=80/uploads/pastry.jpg 1920w
    "
    sizes="(max-width: 640px) 100vw, (max-width: 1024px) 50vw, 33vw"
  />
  <!-- Default Fallback Img -->
  <img
    src="/cdn-cgi/image/width=640,format=auto,quality=80/uploads/pastry.jpg"
    alt="Fresh sourdough bread"
    width="640"
    height="480"
    loading="lazy"
    decoding="async"
    class="h-auto w-full object-cover transition-opacity duration-300"
  />
</picture>

---

4. Zero-Layout-Shift <Picture /> Component & Progressive Fidelity

4.1 Native Browser Progressive Rendering

As decided in design alignment, SiteSwarm rejects fragile client-side JavaScript hydration state machines for image loading. Instead, we leverage native browser progressive rendering:

1. CSS Aspect-Ratio Reservation: The wrapper container calculates aspect-ratio: width / height or applies inline style="aspect-ratio: ${width} / ${height}". The layout space is locked in before the first byte arrives over the wire.
2. CSS Background Shimmer Placeholder: A lightweight CSS pulse/shimmer animation provides immediate visual feedback to the visitor that content is loading, eliminating layout jumps.
3. Progressive AVIF/WebP Decoding: Browsers natively render progressive scans as packets stream in from Cloudflare edge caches, increasing fidelity seamlessly without runtime JS listeners.

4.2 Hero Image Priority Handling

For above-the-fold hero banners (LCP candidates):

• Passing priority={true} or loading="eager" switches the rendered attributes:
  • loading="eager"
  • decoding="sync" (or "auto")
  • fetchpriority="high"
• The governance linter strictly verifies that at most 1 image per page has priority={true}.

---

5. Architectural Governance Linter (arch/require-lazy-loading-media)

To prevent performance regressions across the client fleet, SiteSwarm enforces automated AST and template linting via packages/governance:

5.1 Linter Rules & Verification Gates

1. Ban on Raw Unmanaged <img> for Content Images:
   • Direct <img src="...png"> or dynamic CMS media expressions in templates trigger an immediate lint error:

     [bakery] ❌ FAILED (arch/require-lazy-loading-media) in src/components/PastryGrid.astro:
       • Line 42: Raw <img> tag detected for content image ('{pastry.image}').
         → Remediation: Replace with <Picture /> from '@siteswarm/components' or '@siteswarm/media' to guarantee responsive srcsets and 0.00 CLS.

   • Exception: Inline SVG brand marks and vector icons are permitted as <img> provided they declare width, height, and loading="lazy" (or loading="eager" in the header).
2. Mandatory Lazy Loading & Async Decoding:
   • Any <Picture /> or permitted image tag omitting loading="lazy" and decoding="async" triggers a build failure unless explicitly marked with priority={true} or loading="eager".
3. Hero Priority Budget Gate (Max 1 Priority Image):
   • If a page or component declares more than 1 image with priority or loading="eager", the linter fails:

     [bakery] ❌ FAILED (arch/require-lazy-loading-media) in src/pages/index.astro:
       • Multiple eager/priority images detected (found 3).
         → Remediation: At most 1 above-the-fold hero image may declare priority={true}; all subsequent images must be lazily loaded.

---

6. Comprehensive Fleet Financial Economics & Pricing Models

A core axiom of SiteSwarm is maintaining near-$0 hosting costs per client while maintaining software engineer peace of mind. Here is the rigorous, accurate financial breakdown for Cloudflare R2 and Cloudflare Images across our fleet.

6.1 Cloudflare Unit Pricing Architecture (Ground Truth)

Service Component	Cloudflare Official Unit Rate	Free Allocation per Month	SiteSwarm Architecture Notes
Cloudflare Workers Paid	$5.00 / month flat	Shared fleet account	Includes 10 million requests/mo, up to 10 MB compressed script bundles, and unlocks R2 bindings and Image Transformations.
Cloudflare R2 Storage	$0.015 / GB-month	10 GB / month free	Storing 100 GB of master media costs just $1.35/month.
Cloudflare R2 Class A Ops (Write, List, Put)	$4.50 / million ops	1,000,000 ops / mo free	Client uploads and CMS media scans.
Cloudflare R2 Class B Ops (Read, Get)	$0.36 / million ops	10,000,000 ops / mo free	Read by Cloudflare edge transform workers on origin cache misses.
Cloudflare R2 Egress Bandwidth	$0.00 / GB (ZERO egress)	Unlimited $0 egress	$0.00 worldwide, completely eliminating AWS S3 egress bill shock.
Cloudflare Images Transformations	$0.50 / 1,000 unique transforms	Included in zone packs / initial allocation	Charged only once per unique image transformation (width, height, format). Subsequent hits serve from edge cache at $0.00.

6.2 The Cloudflare Edge Caching Multiplier

Unlike on-demand serverless functions that invoke CPU cycles on every single hit, Cloudflare Images Transformations cache transformed renditions at the Anycast edge:

• Transformation Cache Hit Rate: > 98.5% across typical small business traffic.
• Example: A bakery with 50 menu photos receiving 50,000 monthly page views (500,000 total image requests):
  • Total unique transforms (50 images × 4 responsive widths = 200 transforms).
  • Billable transform events: 200 transforms = $0.10.
  • Remaining 499,800 image views serve directly from edge cache: Cost = $0.00.

6.3 Fleet TCO Comparison (SiteSwarm vs. Traditional Cloud Stacks)

Let us compare total monthly infrastructure hosting costs across fleets of 10, 50, and 100 small business clients:

Assumptions per Client:

• 150 high-resolution photos in media library (average 2 MB raw = 300 MB storage per client).
• 25,000 monthly page views per client (~150,000 image impressions per client/month).
• 30 new/updated photos uploaded per month.

Comprehensive Fleet TCO Model:

+---------------------------------------------------------------------------------------+
|  FLEET SIZE: 10 CLIENTS (Total Monthly Views: 250,000 | Total Media: 3 GB)            |
+---------------------------------------------------------------------------------------+
|  Component                      | SiteSwarm (Cloudflare)     | Traditional AWS + Imgix|
|  Workers / Compute              | $5.00 (Workers Paid)       | $40.00 (ECS/Lambda)    |
|  Storage (3 GB)                 | $0.00 (within 10GB free)   | $0.07 (S3 Standard)    |
|  Egress Bandwidth (150 GB)      | $0.00 ($0 R2 egress)       | $13.50 (CloudFront)    |
|  Image Optimization             | $0.50 (1,000 unique transforms)| $75.00 (Imgix/Cloudinary) |
|  TOTAL MONTHLY COST             | $5.50 / month              | $128.57 / month        |
|  Amortized Cost Per Client      | $0.55 / client / month     | $12.86 / client / month|
+---------------------------------------------------------------------------------------+
|  FLEET SIZE: 50 CLIENTS (Total Monthly Views: 1,250,000 | Total Media: 15 GB)         |
+---------------------------------------------------------------------------------------+
|  Workers / Compute              | $5.00 (Workers Paid)       | $120.00 (ECS/Lambda)   |
|  Storage (15 GB)                | $0.08 (5 GB billable)      | $0.35 (S3 Standard)    |
|  Egress Bandwidth (750 GB)      | $0.00 ($0 R2 egress)       | $63.75 (CloudFront)    |
|  Image Optimization             | $2.50 (5,000 unique transforms)| $175.00 (Imgix/Cloudinary)|
|  TOTAL MONTHLY COST             | $7.58 / month              | $359.10 / month        |
|  Amortized Cost Per Client      | $0.15 / client / month     | $7.18 / client / month |
+---------------------------------------------------------------------------------------+
|  FLEET SIZE: 100 CLIENTS (Total Monthly Views: 2,500,000 | Total Media: 30 GB)        |
+---------------------------------------------------------------------------------------+
|  Workers / Compute              | $5.00 (Workers Paid)       | $220.00 (ECS/Lambda)   |
|  Storage (30 GB)                | $0.30 (20 GB billable)     | $0.69 (S3 Standard)    |
|  Egress Bandwidth (1.5 TB)      | $0.00 ($0 R2 egress)       | $127.50 (CloudFront)   |
|  Image Optimization             | $5.00 (10,000 transforms)  | $300.00 (Imgix/Cloudinary)|
|  TOTAL MONTHLY COST             | $10.30 / month             | $648.19 / month        |
|  Amortized Cost Per Client      | $0.10 / client / month     | $6.48 / client / month |
+---------------------------------------------------------------------------------------+

6.4 Strategic Takeaway

SiteSwarm’s R2 + Cloudflare Images architecture scales to 100 local businesses for just $10.30/month across the entire agency (~$0.10 per client per month). In contrast, traditional AWS S3 + CloudFront + Imgix setups cost over $648/month (a 60x cost inflation), driven primarily by aggressive egress bandwidth fees and monthly SaaS image tier lock-ins.

---

7. Implementation Roadmap & Sub-Ticket Decomposition

flowchart TD
    T1["Sub-Ticket 1 (Issue #195): Author True Spec & Pricing\ndocs/specs/CLIENT_INTAKE_AND_ASSETS.md"] --> T2["Sub-Ticket 2 (Issue #278): Headless Media Pipeline\npackages/capabilities/media & URL Builders"]
    T2 --> T3["Sub-Ticket 3 (Issue #198): Responsive <Picture /> Component\nZero-layout-shift aspect ratio & progressive rendering"]
    T3 --> T4["Sub-Ticket 4 (Issue #278): Governance Linter\narch/require-lazy-loading-media AST rule"]
    T4 --> T5["Sub-Ticket 5 (Issue #277): Future Video & Stream Pipeline\nCloudflare Stream & animated GIF conversion"]

7.1 Deliverables Checklist:

• Sub-Ticket 1 (#195): Author Living True Specification docs/specs/CLIENT_INTAKE_AND_ASSETS.md with verified Cloudflare pricing models.
• Sub-Ticket 2 (#278): Implement headless media capability @siteswarm/media with Cloudflare Images URL transformation grammar, responsive srcset generation, and content-addressed key helpers.
• Sub-Ticket 3 (#198): Author reusable <Picture /> component with intrinsic aspect ratio calculation, CSS shimmer placeholder, and progressive AVIF/WebP rendering.
• Sub-Ticket 4 (#278): Implement arch/require-lazy-loading-media governance linter in packages/governance/src/cli/lint.ts with comprehensive unit test coverage.
• Sub-Ticket 5 (#277): Register future tracking issue for video streaming and GIF optimization via Cloudflare Stream.

---

8. Conclusion

By unifying digital asset storage on Cloudflare R2 and delivery on Cloudflare Images zone transformations, SiteSwarm guarantees:

1. Flawless Lighthouse Scores: 0.00 Cumulative Layout Shift, next-generation AVIF/WebP compression, and sub-100ms visual paints.
2. Defensive Governance: Strict automated linting that prevents non-lazy or unoptimized raw images from entering production.
3. Rock-Bottom Operating Economics: Near-$0 hosting costs amortized to under $0.15/month per client, unlocking high-margin retainer economics for the agency.