backend-services · reference
Three subsystems tag three different subjects against one shared vocabulary. They run the same lifecycle, but they read different things, prompt the model with different things, and wake up for different reasons.
Not the Notify-Me ontology. Notify-Me Part 1 describes a separate tagging scheme — ten notify-categories written to groups.custom_filters_keys in MySQL, still at design stage. Nothing named custom_filters_keys or notify_categories exists in backend-services today. This page covers the city ontology: built, running, MongoDB-backed, LLM-classified.
The city ontology is the per-organization tag vocabulary. None of the three subsystems generate it — an admin authors it, and it is versioned.
apps/organizations/src/city-ontology/, in the city-ontology-config collection, keyed uniquely by organizationId + env + version.upsertOntology never mutates a row. It writes a new one at version = latest + 1; readers always take the highest version.poolVersion.A doc whose poolVersion is behind the pool is stale. Nothing sweeps stale docs. A subject is re-classified only when the subject itself changes — publishing a new ontology version does not trigger anything.
An LLM call can't block a save, so the write path only ever enqueues. A cron drains the queue out of band.
markDirty upserts one row per subject and bumps dirtyRevision. Ten edits in a minute produce one classification, not ten.nextRecomputeAt = now + debounce. It never pulls an existing future time earlier.{lockToken, dirtyRevision} at lock time and clears state only if both still match. An edit that lands mid-classification leaves the row dirty, so it runs again.classifyOne compares a contentHash of the assembled input and the poolVersion. Both unchanged → skipped, no call. This is what makes an over-eager markDirty cheap — but only for subjects that already have a doc.aiTags holds what the model returned; tags is AI tags + manual additions − manualRemovedKeys. A re-classification never resurrects a tag an admin removed.subsystem
apps/groups/src/event-ontology/ · event-ontology · event-ontology-sync-state
Subject: one event (a groups row).
Input: assembleEventText over the event's own columns — name, titles, descriptions, location, type, citizen segment, custom filters, instant tags, and custom-field columns.
What the model sees: the event text, and nothing else. One call per event. This is the only subsystem where the model reads the subject's raw content directly.
| Trigger | Where |
|---|---|
| Event created, any path | groups.service.ts createGroup — always marks dirty |
| A relevant column actually changes | updateOneById / updateManyByIds / updateGroup → resolveClassificationDirtyIds |
| Events-creator submit or save-draft | fires CLASSIFY_EVENT_NOW immediately, bypassing the queue |
| Admin clicks classify | POST events/:resourceId/ontology/classify |
| Org-wide backfill | BACKFILL_EVENT_ONTOLOGY RPC — not exposed on a gateway |
Relevance is decided by value change, not by which columns appear in the payload. Integration syncs resubmit the whole resource on every run; keying off presence would re-enqueue the entire corpus on every sync.
Batch edits through updateManyEventsByIds (community-events-batch) do not currently mark dirty at all.
subsystem
apps/communities/src/community-ontology/ · community-ontology · community-ontology-sync-state
Subject: one community. Blends three signals rather than reading one body of text.
| Signal | Source |
|---|---|
| Community text | SQL communities row — name, title, description, long_description |
| Curated tags | SQL community_tags for that community |
| Event aggregate | RPC GET_EVENT_ONTOLOGY_AGGREGATE_BY_COMMUNITY |
What the model sees: community text and curated tags only. The event aggregate never enters the prompt.
The aggregate RPC reads every communities_groups link for the community, then runs a Mongo $unwind + $group over those events' ontology docs and returns a tag histogram — {categoryCode, groupKey, key, label, count}. Afterwards, mergeCommunityOntology adds the top 3 tags per category to whatever the model produced, marked sources: ['events']. Tags found by both carry both sources.
So: one LLM call per community regardless of event count, but a DB read across all of the community's events. Only already-classified events contribute — while event classification is behind, community docs are built from a partial histogram.
| Trigger | Where |
|---|---|
| Community row updated | communities.service.ts — updateCommunityById, updateCommunities |
| Community tags saved | community-tags.service.ts, two call sites |
| An event's tags change | event cycle pushCommunityDirty → MARK_COMMUNITY_ONTOLOGY_DIRTY for every community the event is shared into |
| Admin classify / manual tags | communities/:communityId/ontology/classify, …/tags |
| Org-wide backfill | BACKFILL_COMMUNITY_ONTOLOGY RPC |
The contentHash covers the community text, the curated tags and the aggregate counts — so a shifting event histogram alone is enough to force a re-classification.
subsystem
apps/user/src/user-ontology/ · user-ontology · user-ontology-sync-state
Subject: one user. Mirrors the community shape.
| Signal | Source |
|---|---|
| User text | SQL users row — occupation, institution, neighborhood, preferred categories, family status, single/multi/big choice fields |
| Event aggregate | RPC GET_EVENT_ONTOLOGY_AGGREGATE_BY_USER — the user's registrations (users_groups), scoped by organizationId |
What the model sees: user text only. Same as community — the aggregate is merged in afterwards by the same mergeCommunityOntology (top 3 per category), never prompted.
| Trigger | Where |
|---|---|
| User registers for an event | user-groups.service.ts — createMany, the subscribe path, and bulk insert → MARK_USER_ONTOLOGY_DIRTY |
| Admin classify | POST users/:userId/ontology/classify |
| Org-wide backfill | BACKFILL_USER_ONTOLOGY RPC |
Gap: profile edits don't trigger anything
Nothing in apps/user marks the ontology dirty on a profile write, even though those columns are exactly what assembleUserText reads. A user's doc only refreshes when their registrations change, or on an explicit classify or backfill.
There is also no manual tag editing for users — no /tags endpoint, unlike events and communities.
| Event | Community | User | |
|---|---|---|---|
| Owning service | groups | communities | user |
| Subject key | resourceId | communityId | userId |
| Prompted input | event text | community text + curated tags | user text |
| Non-prompted signal | — | event tag histogram (its events) | event tag histogram (registrations) |
| LLM calls per run | 1 | 1 | 1 |
| Manual tag editing | yes | yes | no |
| Auto-trigger on subject edit | yes, on real change | yes | no |
| Fans out to | community ontology | — | — |
| Failure backoff + parking | yes | no | no |
| Event | Community | User | |
|---|---|---|---|
| Debounce | 2 min / 1 | 5 min / 1 | 5 min / 1 |
| Lock lease | 5 min / 2 | 5 min / 2 | 5 min / 2 |
| Cooldown after success | 2 min / 1 | 5 min / 1 | 5 min / 1 |
| Subjects per cycle | 25 / 10 | 25 / 10 | 25 / 10 |
Cron cadence lives in the job_schedules collection, seeded per environment. The user-ontology drain is also in DEFAULT_JOB_SCHEDULES at */5 * * * *; the event and community drains are seeded directly.
Throughput ceiling is cycle limit ÷ cron interval. A backlog larger than that drains over hours, not minutes.
Event ontology counts attempts. markFailed increments attempts, reschedules with jittered exponential backoff off the cooldown base (capped at 60 min), and after 5 attempts parks the row: isDirty: false, parkedAt stamped, one warn logged. A successful pass, a markDirty, or a backfill all reset the counter and revive a parked row.
Community and user ontology do not
Their markFailed only pushes nextRecomputeAt out by the cooldown and drops the lock — isDirty stays true with no attempt cap. A subject that fails keeps returning every cooldown, forever.
Because getDueStateDocuments sorts by nextRecomputeAt ascending, permanently failing subjects sit at the head of the queue and starve everything behind them. Porting the event-ontology backoff is an open task.
db.getCollection("<subsystem>-ontology-sync-state").countDocuments({isDirty: true})onlyMissing: true on any backfill skips subjects that already have a doc. It's the right flag for filling gaps rather than re-paying for the whole corpus.poolVersion on each doc makes the staleness detectable; a sweep is a follow-up.AiUsageFeatureEnum.ONTOLOGY with fundedBy: COMPANY.notify-categories scheme.