Scrapers

Instagram carousels

Filters a public Instagram profile to carousel posts only (multi-slide image posts) and emits one dataset row per carousel with every slide's media URL. Used as a reference corpus for the marketing swarm's carousel renderer.

Updated today

Overview

At a glance

Worker: ig-carousel-scraper
Targets: Public IG profiles, carousel posts only
Distinct field: slides[] array per row with all media URLs
Sibling: Shares cookie + proxy contract with reels and trends scrapers
Webhook target: scraper-webhook.51ultron.com/apify

The marketing swarm's carousel renderer learns from exemplars. This scraper is how that exemplar corpus is populated: pull carousels from accounts whose visual system we want to emulate (one slide per image plus the caption per post), normalise into the dataset shape, then write to the carousel-engine-exemplars R2 bucket downstream.

Input schema

.actor/INPUT_SCHEMA.jsonjson

1{

2 "usernames": ["nike", "spotify", "linear"],

3 "maxItemsPerTarget": 50,

4 "proxyConfiguration": {

5 "useApifyProxy": true,

6 "apifyProxyGroups": ["RESIDENTIAL"],

7 "apifyProxyCountry": "US"

8 },

9 "requestDelayMs": 2000,

10 "igSessionId": "<secret, optional>",

11 "igDsUserId": "<secret>",

12 "igCsrfToken": "<secret>",

13 "igMid": "<secret>",

14 "igDatr": "<secret>",

15 "igDid": "<secret>",

16 "igRur": "<secret>"

17}

Crawl flow

src/__main__.pypy

1# Per username:

2# 1. Resolve userId via web_profile_info.

3# 2. If no cookies: take embedded carousels from web_profile_info (~12).

4# 3. If cookies: paginate /api/v1/feed/user/<userId>/.

5# Filter: media_type == 8 (carousel album).

6# 4. Per carousel:

7# a. carousel_media[] gives each slide's image_versions2 + video_versions.

8# b. Build slides[] with index, mediaUrl, isVideo, durationSec when video.

9# c. Push as kind="carousel" with the slides[] array.

10# 5. Sleep requestDelayMs + 50% jitter between requests.

Output shape

dataset-row.jsonjson

1{

2 "kind": "carousel",

3 "source": "instagram",

4 "capturedFrom": "user-feed:nike",

5 "shortcode": "C7xxxxx",

6 "authorUsername": "nike",

7 "caption": "Three frames that taught us to ...",

8 "likeCount": 89321,

9 "commentCount": 2342,

10 "publishedAt": "2026-05-18T14:00:00Z",

11 "slides": [

12 { "index": 0, "mediaUrl": "https://scontent.../s1.jpg", "isVideo": false },

13 { "index": 1, "mediaUrl": "https://scontent.../s2.jpg", "isVideo": false },

14 { "index": 2, "mediaUrl": "https://scontent.../s3.jpg", "isVideo": false }

15 ],

16 "scrapedAt": "2026-05-19T12:00:00Z"

17}

Use cases

Carousel exemplar corpus for the marketing swarm. The brief indexer compares incoming briefs against the exemplar embeddings via the Vectorize index so the planner can pull similar prior carousels as reference material.

Competitor visual audit. Pull a competitor's last 50 carousels, run the slides through OCR + canvas-block classification, surface a brand-visual summary.

Trend mining. Combine with the Instagram trends scraper to find which carousel patterns are gaining traction in a niche this week.

Webhook contract

actor.jsonjson

1{

2 "eventTypes": ["ACTOR.RUN.SUCCEEDED", "ACTOR.RUN.FAILED", "ACTOR.RUN.TIMED_OUT", "ACTOR.RUN.ABORTED"],

3 "requestUrl": "https://scraper-webhook.51ultron.com/apify",

4 "payload": { "actorName": "ig-carousel-scraper", "source": "instagram", "datasetId": "<id>" }

5}

File map

actors/ig-carousel-scraper

.actor

actor.json

INPUT_SCHEMA.json

src

__main__.py

Dockerfile

requirements.txt

README.md