Ultron Workers

Storage

The marketing worker writes to three stores. D1 holds the canonical row per run plus the QA notes. R2 holds the component index and the slide PNGs. Vectorize holds the brief embedding so retrieval finds similar past runs. Each store has a specific job and a specific access pattern; together they make the pipeline self-improving over time.

Updated today

Overview

At a glance

D1 database: carousel-engine
R2 buckets: carousel-engine-index, carousel-engine-exemplars
Vectorize index: carousel-briefs (768 dim, cosine)
Schema source: cf-workers/marketing-swarm/carousel-engine/schema.sql
Write paths: Promote stage (carousels + briefs + exemplars), every run (render_runs), QA stage (slide_qa)
Read paths: Retrieve stage (briefs + exemplars), star endpoint (carousels), analytics (all three)

D1 database

Cloudflare D1 is a serverless SQLite that lives at the edge. The marketing worker reads and writes it directly through the DB binding.

cf-workers/marketing-swarm/wrangler.tomltoml

1[[d1_databases]]

2binding = "DB"

3database_name = "carousel-engine"

D1 fits this pipeline well because reads are infrequent (only retrieval queries) and writes are small (one row per run, plus N rows for slide QA). No background process needs to crawl the database, which means SQLite's write lock never becomes a contention point.

Schema

Three tables. One row per run, one per slide, one per render attempt.

cf-workers/marketing-swarm/carousel-engine/schema.sqlsql

1-- The canonical record per promoted run

2create table carousels (

3 id text primary key,

4 brief text not null,

5 spec text not null, -- JSON CarouselSpec

6 score real not null,

7 status text not null default 'active', -- 'active' | 'archived'

8 created_at integer not null

9);

11-- Per-slide QA notes from the vision pass

12create table slide_qa (

13 id integer primary key autoincrement,

14 carousel_id text not null,

15 slide_index integer not null,

16 score real not null,

17 critique text,

18 created_at integer not null

19);

21-- Every run, promoted or not, with timing breakdown

22create table render_runs (

23 id text primary key,

24 brief text not null,

25 spec text,

26 outcome text not null, -- 'promoted' | 'kept' | 'failed'

27 failure text, -- short reason when outcome=failed

28 ms_retrieve integer,

29 ms_plan integer,

30 ms_validate integer,

31 ms_render integer,

32 ms_qa integer,

33 ms_promote integer,

34 created_at integer not null

35);

Tip

render_runs exists for analytics. It captures every attempt, including failures, so you can plot the failure rate per stage and the latency breakdown across releases. carousels is the public catalogue of promoted runs that the retrieval stage reads.

R2 buckets

Two buckets. One for the component index, one for the slide assets.

Bucket	Holds	Read path	Write path
`carousel-engine-index`	component-index.json with metadata for every registered component	Read by the planner at the start of every run	Written by the index build script after a library change
`carousel-engine-exemplars`	Slide PNGs plus spec JSON for promoted runs at <InlineCode>carousels/<id>/<file></InlineCode>	Read by retrieval to hydrate exemplars; signed URL returned in the response	Written by promote

cf-workers/marketing-swarm/src/carousel/promote.tsts

1for (const slide of pngs) {

2 await env.EXEMPLARS.put(

3 `carousels/${runId}/slide-${slide.index}.png`,

4 slide.bytes,

5 { httpMetadata: { contentType: "image/png" } }

6 )

7}

8await env.EXEMPLARS.put(

9 `carousels/${runId}/spec.json`,

10 JSON.stringify(spec),

11 { httpMetadata: { contentType: "application/json" } }

12)

Vectorize index

A 768-dimensional cosine index keyed by run id, with status as a filter dimension.

cf-workers/marketing-swarm/src/carousel/promote.tsts

1await env.BRIEFS.upsert([

2 {

3 id: runId,

4 values: briefEmbedding, // 768 dim from bge-base-en-v1.5

5 metadata: {

6 status: "active", // filter dimension for retrieval

7 score: averageScore,

8 type: "carousel", // separates carousel briefs from design-system briefs

9 },

10 },

11])

The Vectorize index is also used for a smaller secondary corpus: design-system documents the planner pulls in when the brief asks for an on-brand carousel. Those entries are differentiated by the type metadata field so the retrieval step can scope its query.

Storage lifecycle

A run touches the stores in a fixed order.

Stage	Reads	Writes
Retrieve	Vectorize (briefs), R2 (exemplars)	-
Plan	R2 (component index)	-
Validate	-	-
Render	R2 (fonts)	-
QA	-	-
Promote (if score >= 0.85)	-	D1 (carousels), Vectorize (briefs), R2 (exemplars), D1 (slide_qa)
Promote (every run)	-	D1 (render_runs)

File map

cf-workers/marketing-swarm

wrangler.tomlD1, R2, Vectorize bindings

carousel-engine

schema.sqlcarousels, slide_qa, render_runs

src/carousel

retrieve.tsVectorize + R2 reads

promote.tsall three stores