Scrapers

Instagram trends

Hashtag-page scraper. Walks the legacy query_hash GraphQL surface with an HTML fallback. As of 2026-05 the sessionid cookie is mandatory; the unauth path has been turned down by Meta. v1 covers the hashtag surface only; explore, reels-by-tag, and top_audio are scheduled for v1.1.

Updated today

Overview

At a glance

Worker: ultron-instagram-trends
Runtime: Python + httpx, no browser
Modes: hashtag (v1). explore, reels-by-tag, top_audio in v1.1.
Auth: 7 session cookies required (sessionid, ds_user_id, csrftoken, mid, datr, ig_did, rur)
Cap: 500 posts per hashtag, default 50
Proxy: Residential US
Default delay: 2000 ms per GraphQL call, +50% jitter

Input schema

.actor/INPUT_SCHEMA.jsonjson

1{

2 "mode": "hashtag", // only value in v1

3 "hashtags": ["food", "gaming"], // hashtag names without #

4 "maxItemsPerTarget": 50, // 1 to 500

5 "proxyConfiguration": {

6 "useApifyProxy": true,

7 "apifyProxyGroups": ["RESIDENTIAL"],

8 "apifyProxyCountry": "US"

9 },

10 "requestDelayMs": 2000,

11 "igSessionId": "<secret>", // required as of 2026-05

12 "igDsUserId": "<secret>",

13 "igCsrfToken": "<secret>",

14 "igMid": "<secret>",

15 "igDatr": "<secret>",

16 "igDid": "<secret>",

17 "igRur": "<secret>"

18}

Warning

As of 2026-05, the unauthenticated hashtag surface no longer returns posts. A full cookie set from a real, aged Instagram account is now mandatory.

Session cookies

What each cookie does and where to harvest it.

Cookie	Purpose
sessionid	The login token. Most important. 30+ day lifetime when account is healthy.
ds_user_id	Numeric user id. Validated against sessionid server-side.
csrftoken	Sent as the X-CSRFToken header. Some endpoints 403 without it.
mid	Long-term browser fingerprint. Instagram cross-validates with sessionid.
datr	Long-term device fingerprint, Meta-wide (Facebook + Instagram + Threads share it).
ig_did	Per-device identifier. Helps the session look like a returning browser.
rur	Region/routing cookie. Instagram uses it for data-center pinning. Wrong rur triggers re-auth.

Tip

Harvest from a logged-in browser via DevTools, Application, Cookies. Store in a workspace secret entry, never in plain config. Rotate when the session expires; aged accounts typically last 30 to 90 days between rotations.

Crawl flow

src/__main__.pypy

1# 1. Build a single httpx.Client with all 7 cookies + the residential

2# proxy. Set User-Agent to the same one the harvested session used.

3# 2. For each hashtag:

4# a. Try GraphQL: POST /api/graphql with query_hash for hashtag-page.

5# b. If empty/error, fall back to the HTML hashtag page and parse the

6# sharedData blob.

7# c. Paginate via the end_cursor returned by GraphQL. Each page

8# returns up to ~30 posts.

9# d. Stop when maxItemsPerTarget or no more posts.

10# 3. Per post: extract shortcode, author username, caption, likeCount,

11# commentCount, mediaUrl, publishedAt. Push as kind="post".

12# 4. Sleep requestDelayMs + 50% jitter between calls.

13# 5. On 429 or "Sorry, this page isn't available", abort the run and

14# flag the cookie set as stale.

Output shape

dataset-row.jsonjson

1{

2 "kind": "post",

3 "source": "instagram",

4 "capturedFrom": "hashtag:food",

5 "shortcode": "C7xxxxxxxxx",

6 "authorUsername": "creator-handle",

7 "caption": "trying the new...",

8 "likeCount": 8421,

9 "commentCount": 342,

10 "viewCount": null, // only set for video / reel posts

11 "mediaUrl": "https://scontent...",

12 "publishedAt": "2026-05-18T14:00:00Z",

13 "scrapedAt": "2026-05-19T12:00:00Z"

14}

Account hygiene

Rules to keep harvested cookies alive.

One account, one IP, forever

Never share a cookie set across IPs. If an Ultron run uses residential IP A for one batch and IP B for the next, Instagram will flag the session within hours. Workspace policy pins each IG cookie set to one residential country code and one IP-rotate group; that mapping must be stable for the cookie set's life.

Rule	Why
Aged account only (3+ months, normal feed activity)	Brand-new accounts get challenged before they can be useful.
No automated likes / follows from the same session	Mixing scraping with engagement signals trips the bot model.
Rotate cookies when the session refreshes server-side	Server-side refreshes invalidate the prior sessionid.
One workspace, one cookie set	Sharing across workspaces means sharing IPs - guaranteed flag.

Webhook contract

actor.jsonjson

1{

2 "eventTypes": ["ACTOR.RUN.SUCCEEDED", "ACTOR.RUN.FAILED", "ACTOR.RUN.TIMED_OUT", "ACTOR.RUN.ABORTED"],

3 "requestUrl": "https://scraper-webhook.51ultron.com/apify",

4 "payload": { "actorName": "instagram-trends", "source": "instagram", "datasetId": "<id>" }

5}

File map

actors/instagram-trends

.actor

actor.json

INPUT_SCHEMA.json

src

__main__.pyGraphQL + HTML fallback

Dockerfile

requirements.txt

README.md