--- title: "How it works" description: "The ingestion pipeline, per-platform freshness, and the mention lifecycle." canonical: https://docs.mentio.dev/how-it-works markdown: https://docs.mentio.dev/how-it-works.mdx --- # How it works The ingestion pipeline, per-platform freshness, and the mention lifecycle. ## The pipeline Mentio runs a staged pipeline: platforms are fetched on a schedule, normalized into raw items, matched against every workspace's keywords, classified by an LLM, and delivered. Ingestion happens once, globally. When two workspaces track the same term, the platform fetches and stores those posts once, then matches them to both. Your mention feed is the workspace-scoped view over that shared ingest. ## Per-platform freshness Each platform has its own polling cadence, chosen around its API characteristics: | Platform | Freshness | Notes | | -------------- | ------------ | -------------------------------------------------------------------------------------------------------------------- | | Bluesky | Real time | Firehose subscription, not polling | | GitHub | \~5 minutes | Per tracked term | | News | \~15 minutes | One global GDELT feed, published as one file every 15 minutes | | Reddit | \~30 minutes | Per tracked term | | Hacker News | \~1 hour | One global poll covers every term | | DEV | \~1 hour | One global poll covers every term | | Stack Overflow | \~1 hour | Quota bound | | X | \~1 hour | Kept coarse to respect the read budget | | LinkedIn | \~1 hour | A term that keeps coming back empty is polled less often, down to once a day, and is back to hourly on its first hit | | YouTube | \~12 hours | Search quota bound | A mention can only be as fresh as the platform's own API surface; the pipeline adds sub-minute overhead on top of the cadences above. A keyword's `polling` array reports, per polled platform, the last poll and how many consecutive polls found nothing new. ## Matching rules A term matches as a phrase, case-insensitively, on every platform the keyword tracks. Two layers of rules narrow that before anything is stored, which is what makes them different from an alert's filter: a post a rule rejects never becomes a mention, is never classified and never bills. | Layer | Where | Rules | | --------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Keyword | `matching` on [`POST`](/api/keywords/create-keyword) and [`PATCH /v1/keywords/{id}`](/api/keywords/update-keyword) | `requiredTerms` the post must also contain (`requiredMode` `any` or `all`); `excludedTerms` that drop it, a `*` at an end being a wildcard (`beta.*`); `excludedAuthors` in the same forms an alert's mute list takes; `caseSensitive` for an acronym (`RAG`, never `rag`) | | Workspace | [`GET`](/api/filters/get-filters) and [`PATCH /v1/filters`](/api/filters/update-filters) | `excludedTerms` and `excludedAuthors` for every keyword; `excludedRepos` (GitHub `owner/name`); `subreddits.only` (an allowlist) and `subreddits.excluded` | A keyword also takes a `context` sentence the classifier reads for that term only ("Arc is our browser; ignore the geometry word"). Unlike the rules, it changes the score, not what is stored. Rules apply to new posts from the next poll; stored mentions keep their place. ## Mention lifecycle The pipeline's internal stages never appear in the API. Four fields tell you where a mention stands: | Field | Values | Meaning | | ---------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | `status` | `open`, `ignored`, `done` | Your triage. Every mention starts `open`; you set the other two with [`PATCH /v1/mentions/{id}`](/api/mentions/update-mention) | | `relevant` | `true`, `false` | Scored at or above the relevance threshold (currently 40). `false` until the classifier has run | | `delivered` | `true`, `false` | At least one alert channel has received it | | `classification` | object or `null` | `null` until the classifier has run. `classification.failed` is `true` when the classifier errored out on this mention | Every matched mention stays queryable whatever it scored: noise (`relevant: false`) is never delivered, but `GET /v1/mentions?relevant=false` still lists it. Ignored and done mentions are excluded from delivery. ## Classification The classifier reads your [company profile](/api/company/get-company) and scores each mention. The result is the `classification` object: | Field | Values | | ----------- | ------------------------------------------------------------------------------------------- | | `relevance` | 0 to 100 | | `sentiment` | `positive`, `neutral`, `negative` | | `intents` | `buy_intent`, `question`, `complaint`, `praise`, `comparison` | | `automated` | `true` when the post reads as machine-made: a bot, a templated post, a generated digest | | `language` | The language the post is written in, as an ISO 639-1 code (`en`, `es`); `null` when unknown | | `note` | One-line explanation, at most 200 characters | | `failed` | `true` when the classifier errored; a failed classification is never billed | | `feedback` | Your verdict, when you gave one: `relevant`, `sentiment`, when, and the values it replaced | ### Your verdict The classifier can be overruled. [`PATCH /v1/mentions/{id}`](/api/mentions/update-mention) with `relevant: false` marks a mention as noise: its relevance becomes 0, it leaves the relevant feed, the digests, the counts and the analytics. `relevant: true` does the opposite for one the classifier missed (relevance 100), and `sentiment` corrects the label. `null` withdraws a verdict and restores the classifier's value, which `classification.feedback.original` kept. Verdicts never bill or unbill a mention, and every keyword reports how many it received under `stats.feedback`. A mention still queued for classification answers `409 classification_pending`. Relevance scoring is only as good as the context you provide. Describe what your company does, its use cases, its competitors and your own relevance guidelines with `PATCH /v1/company`; the classifier reads the `context` composed from them. A keyword's own `context` refines it for one term.