Layer one: the source list
What the Little Brain will collect, and what is left to decide before any collector starts.
Bottom line
Layer one is fully specified on paper: 21 sources — 9 Ours, 9 Market, 3 Culture — each with its origin system, access status, refresh cadence, collector design, and dedupe rule. This is a plan, not a running system. Your 2026-08-10 rulings: A5 (comments on our organic posts) is a keep — Instagram first, TikTok kept separate and hard; A7 (UGC & creator content) dropped as created content, not raw customer voice; A3 (post-purchase surveys) to be integrated, pending a Shopify access check; A10 kept but re-sourced as a creative registry pulled straight from Meta & Google Ads. One proposed addition (A8 Klaviyo email stats) still awaits your keep/drop, and three access items need you. Several B- and Culture-class sources are keeps that still need a method defined — see the notes. After your go, each source gets its own mini-roadmap: collect, verify, then build the system that keeps it fresh.
Your next moves
- Rule on the last proposed addition — keep or drop Klaviyo email stats inside performance summaries (A8). A5, A10 and B8 are now keeps (2026-08-10).
- Get the ~45-competitor list (B2). Either Meg shares the creative-intelligence Notion doc with the Caesar connection (integrations only see pages shared with them), or give the go and I'll sign into Notion in your browser to find it. Their doc names the competitors the Market collectors need.
- Re-auth the main Lifely store's API token — it is dead, and Little Lifely also sells on that store.
- Give the go to check A3 survey access. You asked me to sign into Shopify in your browser and confirm we can read Checkout Plus post-purchase survey responses on the Little Lifely store. I'll run it when your PC is on and you say go.
- Scope the big open topics. How to capture only winning competitor ads and organic (B6/B7) and their landing pages (B8), how to build review taxonomies (B9) and trends (Culture), and whether to build a Little Brain manual-input Chrome extension. Each is its own mini-project.
Then: the go on collectors. None of these blocks the Ours-first start.
The rules the list runs on
- Ours is proof. Only first-party evidence may substantiate a claim or USP.
- Market is the unconverted voice. Top-of-funnel language may only come from people we have not converted.
- Culture is style, never substance. Every Culture item carries a decay state and an expiry date and drops out when stale.
- Order: Ours → Market → Culture. No source blocks the rest.
- Cost: first bounded pull ≈ $10–30 (estimate from bounded volume). Ongoing scraping cost is contained by starting with the top-10 competitors, not all ~45.
Ours — 9 sources proof · decays slowly
First-party evidence: the only class allowed to back a claim.
| Source | Origin & access | Status | Cadence | Collector | Dedupe |
|---|---|---|---|---|---|
| A1 Product reviews | Judge.me (Junip where used) | ready | daily | Incremental pull since last-seen per store/product; keeps rating, text, product, date, verified-buyer flag | review ID |
| A2 Support tickets | Gorgias | ready | daily | Tickets + messages; PII stripped before storage; speaker tagged (customer vs agent) | ticket + message ID |
| A3 Post-purchase surveys integrate | Checkout Plus on the Little Lifely store (identified 2026-08-08); Shopify access to survey responses to be verified in your browser; main-store side unverified — token dead | buildable | monthly | Responses: traffic source + free-text "what were you solving", tagged product + market | response ID |
| A4 Comments on our ads | Meta Graph API | ready | weekly | Recent ads per account → comments on the underlying posts | comment ID |
| A5 Comments on our organic posts kept | Instagram (primary — high-value) & Facebook via Meta Graph API; TikTok kept separate and hard — no reliable route yet, prior attempts failed | buildable | weekly | Same pattern as A4, on organic post IDs. Instagram first; TikTok attempted separately (may need a manual/PC route) | comment ID |
| A6 Interviews & transcripts | No gathering system exists — a slot, not a collector | on-arrival | on arrival | Any transcript that appears is processed into snippets with speaker + date | file hash |
| A8 Performance summaries | Meta + TikTok insights; Klaviyo email stats proposed | ready | weekly | Per-creative-ID rollups, append-only; raw daily spend stays in Chris's system | creative ID + period |
| A9 Approved brand facts | Shopify catalog (ground truth) + brand guidelines, briefs | ready | on change | Versioned documents flagged approved — not snippets | doc + version |
| A10 Creative registry kept | Meta + Google Ads creatives we actually spend on (live from the ad platforms, not the internal workspace DB) | buildable | weekly | Per running creative: image, ad copy, headline, post ID — grouped by post ID. The entity backbone that A8 performance and A4/A5 comments attach to, and that later links to personas, USPs, and raw material to surface correlations | post ID (grouped) / creative ID |
Market — 9 sources unconverted voice · decays slowly
How the category talks — including everyone we have not converted.
| Source | Origin & access | Status | Cadence | Collector | Dedupe |
|---|---|---|---|---|---|
| B1 Search demand | DataForSEO — the "are we done?" ruler | ready | monthly | Seeded keyword expansion per category, AU/US/UK; zero volume = stop researching that spot | keyword + locale + month |
| B2 Competitor reviews kept | ~45 competitors — the named list awaits the Notion doc (I'll sign into Notion in your browser to locate it); Apify: Amazon, Trustpilot, brand sites | needs you | monthly / market | Per-competitor newest-first sweep, stop at last-seen | review ID or content hash |
| B3 Reddit kept | Reddit API / Apify; merge with the existing Reddit → Discord workflow | buildable | weekly | Shift focus from brand mentions to category problem language — people describing the problems our product solves (baby sleep, eczema, parenting), thread context kept | post/comment ID |
| B4 Forums kept (forums only) | Parenting forums scrapeable; FB groups dropped — closed and manual-only, won't work | buildable | weekly | Weekly keyword sweep of scrapeable forums for new discussion; no FB groups, no manual paste-ins | URL + content hash |
| B5 Comments on competitor ads kept | Apify — the Ad Library exposes no comments; collector resolves ad → underlying post | buildable | weekly / market | Competitor ads → post IDs → comments. Scope to top/good competitors and strong ads only, to avoid spend with little value | comment ID |
| B6 Competitor ads winners only — open | Meta Ad Library API + Apify; depends on the competitor list | buildable | weekly / market | Whole topic: capture only winning ads, not all — needs a defined way to identify winners (unsolved via API so far) and a hard volume cap, or we drown in valueless ads. Then: messaging, hooks, formats, benefits, first/last seen | ad archive ID |
| B7 Competitor organic content winners only — open | Apify — largest recurring cost; top-10 competitors first | buildable | weekly / market | Same problem as B6 — winners only, needs a definition and cap. Then: posting patterns, formats, creator partnerships, engagement | post ID |
| B8 Competitor websites & offers kept | Landing pages behind the winning competitor ads (once B6/B7 winners are known); page fetch + snapshot; newsletters later | buildable | monthly | The pages winning ads send people to — per-competitor snapshot, diffed against last | URL + content hash |
| B9 Review-site taxonomies kept — important | Amazon/Trustpilot category trees; Amazon reviews are a rich angle here | buildable | quarterly | What sub-topics exist at all — the atlas bounds. Still need to define how we build it | taxonomy path |
Culture — 3 sources style only · perishable
Every item carries a decay state (early signal → seasonal) and an expiry date; the weekly run re-scores and drops what is stale.
| Source | Origin & access | Status | Cadence | Collector | Dedupe |
|---|---|---|---|---|---|
| C1 TikTok trends route: TikTok Shop | Creative Center unproven; use TikTok Shop instead — works well (tested with Caesar). Apify / TikTok Shop | buildable | weekly | Trending hooks, sounds, formats per market, stamped with decay + expiry | trend ID + week |
| C2 Google Trends kept | pytrends / DataForSEO — should be a free API; complexity to confirm | ready | weekly | Category + seed terms per market | term + locale + week |
| C4 Swipe file / winning posts kept | Curated — ours and other brands, in and out of category; added via the manual-input tool (below) | curated | weekly scan | Each example tagged format, hook style, decay, expiry. Trends are a complete topic to tackle on their own | URL / asset hash |
Where the access facts come from (2026-08-08 check)
- Checkout Plus is the only survey-capable app among the 23 apps installed on the Little Lifely store (full app listing via the store API), and it matches the survey CSV Chris exported in April.
- Main Lifely store token returned "invalid API key or access token" on the same check.
- The ~45-competitor list is named in Meg & Emily's creative-intelligence proposal doc (July); that doc is not yet shared with the Caesar Notion connection, which is why it is unreachable.
- Judge.me and Klaviyo confirmed live on both storefronts; Gorgias chat on the main store; Upfluence and Tolstoy found installed (UGC trail).
- No transcript system: confirmed by Stephane, 2026-08-08 — hence A6 is an on-arrival slot.
- Your rulings (2026-08-10): A5 kept — Instagram is the priority source; TikTok stays separate and is hard (prior attempts failed, may need a manual/PC route). A7 dropped — created content, not raw customer voice. A3 to be integrated, pending a Shopify access check in your browser for Little Lifely survey responses. A10 kept but re-sourced: track the creatives we spend on directly from Meta & Google Ads (image, copy, headline, post ID, grouped by post ID) as the backbone that performance and comments link to — treated as the entity registry, not a customer-voice source. Market rulings: B2 competitor reviews kept (find the ~45 list in Notion via your browser); B3 Reddit kept and mandatory — merge with the existing Reddit→Discord workflow and shift it from brand mentions to category problem language; B4 kept for forums only — FB groups and manual paste-ins dropped; B5 competitor-ad comments kept but scoped to good competitors and strong ads only.
- Your rulings (2026-08-10, cont.): B6/B7 competitor ads & organic kept, but a whole topic — capture only winners, not all, with a volume cap; getting winners via API is unsolved. B8 kept as the landing pages of those winning ads. B9 review taxonomies kept and important (Amazon reviews too). Culture: Pinterest dropped (no manual); TikTok via TikTok Shop not Creative Center; Google Trends likely a free API; swipe file kept. New ideas: a Little Brain manual-input Chrome extension (pick category, add item), potential influencers later, and trends as a complete topic (Stephane's skills + Scale AI workflows).