Mass Social Wisdom Agent

At a glance

Who it serves

People turning saved links and screenshots into knowledge they can review and reuse.

What I owned

Source ingestion, extraction, categorisation and document-export workflow.

Key decision

Preserve source attribution and expose model self-scores as review aids, not independent quality measurements.

Current state

Public repository and a quota-limited live portfolio demo.

Evidence and gap

Runnable workflow and document output. Extraction fidelity and self-score calibration still need labelled evaluation.

Problem

I kept running into the same moment: scrolling past an Instagram Reel from a tech creator breaking down something genuinely new, a YouTube video from a builder walking through a technique nobody else was talking about yet, or a thread from someone on X.com sharing a hard-won lesson from actually shipping something. The advice was real, niche, current - the kind that has not made it into a blog post yet. And the same pattern kept repeating across every product I was building. For Dreamsea, my partner would send me reels of other creators who were posting genuinely good dream interpretation methods I wanted feeding into the Dream Wiki. For nsso's Deity chatbot, LinkedIn was full of business strategy and positioning advice Deity's knowledge base could use. For Qadam, real trading advice from people who actually trade well kept surfacing in the same feeds I was already scrolling.

The problem was what happened after I found it. Screenshots, forwarded links, and open tabs became a second backlog to manage. I wanted to paste the raw material once, extract the source-linked content, and leave with an organised document I could review and import into Notion or a later ingestion workflow. The public extraction tool and public RAG package are separate components; their existence does not establish a live integration into each product.

Architecture

Model: Google Gemini 2.5 Flash handles five stages: OCR, composition, self-scoring, categorisation, and similarity sorting. This is a staged single-model pipeline rather than a team of independent agents. The self-score guides retries and has not been calibrated against human quality judgments.

Context: the context comes from two places. The first is the actual social media content the user drops in - the URLs, screenshots, and slides themselves, whether that is a video transcript, a caption, or text sitting on top of an image. The second is a system-level prompt: built-in instructions that tell the agent exactly how to clean up a messy transcription or OCR scan into something readable, and how to sort that cleaned-up insight into the right topic. Separate prompt profiles govern the standard composition pass versus the lenient retry pass. The quality-assessment prompt is structured to return a single digit from 1 to 5, so there is no ambiguity in reading the result, and the prompt that sorts content into topics lists all eight categories by name and tells the model to reply with just the category name, nothing else.

Orchestration: orchestration is the plumbing that keeps everything moving in the background while the user waits. Each batch of links gets processed as its own background job, so the app does not freeze up while it works. A shared tracker keeps tabs on every job in progress, and the page checks in automatically every two seconds to pull the latest results, so the user watches items appear one by one in real time instead of staring at a loading spinner with no feedback. A stop button is also wired in, so a run can be cancelled partway through if needed.

Workflow controls include URL extraction and normalisation, keyword category overrides, bounded retries, failed-source records, cancellation, and request delays. They help manage processing and failure recovery. They do not establish summary accuracy or category quality; Gemini's self-assessment still needs human-labelled evaluation.

The live version hosted on this portfolio site adds a further layer of cost and usage guardrails, capping how much processing a visitor can trigger so that a spike in web traffic cannot run up token consumption on the underlying AI model.

Human: the user pastes in raw, unorganised material - a WhatsApp conversation, a Notion page, or just a wall of URLs surrounded by other text - and the agent finds every valid link inside it, removes duplicates, and works through all of them without needing anything cleaned up first. The Scan folder is a second way in: the user can drop screenshots, presentation slides, or chart images there before running the agent, and it will read and summarise those images too, sorting them into the same categories and the same final document as everything pulled from a URL.

Why this approach

The most obvious alternative was a RAG pipeline: embed every piece of content into a vector database and let the user query it semantically later. That approach was rejected because the use case is not retrieval - it is curation. The user wants a structured knowledge document after each session; something more useful than a searchable corpus to interrogate. A flat, category-sorted .docx that imports cleanly into Notion or Google Docs is more useful than a query interface, because the user can either read it back at their own leisure or at least be confident that the wisdom was extracted successfully.

One Gemini model handles OCR, composition, self-scoring, categorisation, and sorting, keeping provider integration simple. Specialised models may improve individual stages, but that accuracy and latency comparison has not been benchmarked.

Tradeoffs

The first tradeoff I made was statelessness over persistence. The job tracker is an in-memory Python dictionary, so every server restart clears all session state. This was a deliberate v1 choice: a database layer such as SQLite or Supabase was deferred because the target workflow is single-session extraction runs, not a longitudinal knowledge graph. The cost is that a crash mid-run loses the job log and any already-processed items that had not been exported yet.

The second tradeoff is managing the Gemini Free Tier rate limits with a sleep timer rather than a proper queue. The 4-second inter-URL delay and 3-second inter-slide delay are hardcoded constants, so at 10 URLs with multi-slide carousels, a session can take 5 to 8 minutes. A token-bucket or exponential-backoff queue would handle this more elegantly and scale to paid-tier rate limits without code changes, but the current approach trades that sophistication for zero configuration overhead.

Third, there is no eval coverage for the self-assessment loop. The quality-scoring function returns a 1 to 5 integer from a free-text Gemini response, and the retry threshold is fixed at 3. There is no evaluation suite validating that the quality scores are internally consistent across content types, that the lenient retry reliably produces higher-scoring output, or that the "keep higher result" comparison logic behaves as intended in edge cases, making this the most significant governance gap in v1.

Finally, categorisation accuracy is bounded by eight fixed categories. The category list reflects the specific content diet of the initial user, so content that spans categories - like an AI-powered trading tool breakdown - defaults to whichever keyword override fires first. There is no multi-label support and no confidence threshold, so every item receives exactly one category regardless of ambiguity.

Demo / proof

The portfolio demo sends live API requests subject to availability and visitor quotas. A submitted run can process source URLs and export the returned items; this write-up does not treat every run as a successful extraction or quality test.

The repository is a separate inspection route. Its mock_demo_server.py replays prepared logs and items without API keys, while the README animation illustrates the interaction and document download. A mock replay demonstrates the workflow; it does not verify fresh source retrieval. Docker and Cloud Run-compatible deployment files are scaffolding, not evidence that a particular deployment is currently healthy.

What I would improve

The self-assessment loop is the most intellectually interesting part of the system and also its least validated component. The quality scoring relies on Gemini evaluating its own output, a known reliability concern, and the eval suite currently has no formal test cases. Before expanding the agent to a broader user base, the immediate priority would be building a labelled dataset of 40 to 60 extraction outputs with manually assigned quality scores from 1 to 5 and ground-truth categories, then measuring how often the agent's self-assigned score agrees with the human score within one point.

The second improvement is a request queue with exponential backoff and bounded concurrency. It could reduce large-session time, but the gain needs a benchmark across source types and provider limits before assigning a percentage.

The third gap is the .docx output format itself. While functional and Notion-compatible, it is a one-directional export with no metadata attached to each entry. Adding source type, processing timestamp, quality score, and word count per entry as document properties would make the output auditable and allow downstream filtering without re-running the agent.