Design Doc: Internal Resident ID Verification

DraftAuthor: @relbnsCU-86c9qwda5

TL;DR

Some municipalities expose an eligibility API; some do not. For those that do not, BeWith will hold the eligible resident ID numbers itself: an operator uploads the authority's list, and resident verification checks that repository instead of making an outbound call.

Almost none of this is greenfield. The ID input on the event page, the whole verify round-trip, the per-organization provider config, and deterministic encryption of ID numbers all exist today. The design therefore adds a fourth verification provider behind the existing contract rather than building a parallel flow: a new value on the config's discriminator, a new encrypted repository collection, and a support-tool screen that loads and inspects it.

Read §0 first — the parent ticket is now split, and this document covers the infrastructure subtask only. Then §3.1: the resident discount is wired through four independent configuration points across three admin surfaces, and the event names which segment confers it — citizen_segment_id is a segment id, not a flag. Assuming otherwise invalidates the design.

The build is unblocked. Every question that gated it has been decided (§15), and tasks #1 and #2 depend on none of them.

0. Scope — this doc covers the infrastructure subtask

The parent ticket is split into three subtasks. This design covers the second one.

Subtask	What it delivers	Status
CU-86cb72605 — DR	This document	done
CU-86cb79t1h — DB provider infra + support tool	The system can answer "is this ID eligible" from our own data, and there is a screen to load and inspect that data	in progress
CU-86cb79tmx — Connecting consumers	resident_status promotion and not re-asking a verified user	spec — may not be built

The seam, stated plainly: the infra subtask makes the system able to answer the eligibility question from a file we hold. The consumer subtask makes the user not get asked twice.

Why that seam is clean, and why the infra subtask is independently valuable. The event page, the verify round-trip, and segment attachment all exist and work in production today for Jerusalem. Swapping which provider answers the question — our repository instead of the municipality's API — leaves that entire chain untouched. So a resident typing their ID and receiving the discount works at the end of the infra subtask. What the consumer subtask adds is only the last two clauses of the original ticket: writing resident_status and skipping the prompt on future events. That is a convenience improvement, not a precondition for the value.

Definition of Done for this subtask is deliberately narrow: verification returns a correct answer, demonstrated by a saved Postman request that the reviewer runs himself. See §16.

Everything in this document marked [consumer subtask] is documented for completeness and is not in scope here.

1. Approach & Rationale

Resident verification today runs through CitizenIdService.isCityResident(), which validates the Israeli ID check digit and then dispatches on providerInfo.clientName to one of three hardcoded per-city HTTP integrations. The per-organization configuration for it already lives in the Mongo organizations-providers-config document with type: citizen_id.

The approach is to treat "internal database" as one more verification provider on that same dispatch, not as a new feature:

• The organization setting the ticket asks for ("Resident ID verification method") becomes a verificationMethod field on that same config document — api / internal / none. It composes with the existing clientName rather than replacing it: the method says how to verify, clientName says which municipal integration to use when the answer is "by API". A row with no verificationMethod reads as api, so nothing existing moves and no migration is needed.
• The return contract { isCitizen, isIdValid, isAutoCompleteOTP, clientName } is unchanged, so all three existing callers — the internal gateway, the seats.io external-gateway route, and the legacy PHP consumer — keep working untouched.
• Because the dispatch is currently a hardcoded switch, this change introduces the provider interface that the switch should have been, and implements it for the internal-database provider only. The three existing cities move onto the interface in a separate tech-debt task; they sit on a live, payment-adjacent path and do not need to be destabilised to ship this.

Alternatives rejected. Overloading clientName itself with an internal_db value was rejected once the support-tool surface was settled: it would make an operator's method choice and a developer's integration choice the same field, so switching an organization to internal and back would lose its municipality (§6.1). A separate verification endpoint parallel to /auth/citizen was rejected because it would fork the consumer, the seats.io route, and the legacy PHP into two code paths for one concept.

1.1 The governing invariant — nothing existing changes

The feature is meant to be plug-compatible with the API path: where we used to ask a municipality's API, we ask our own. That is what the provider interface buys — the caller does not know or care whether the answer came from an HTTP call or a local query.

The corollary is a hard constraint on this subtask: no existing behaviour may change or stop working as a result of adding it. Four things follow from that, and each is a design rule rather than an aspiration:

1. The existing switch is not modified. The resolver sits in front of it and intercepts only when verificationMethod is internal. Every other value, including absent, falls through to today's code untouched. The three city integrations are not edited in this subtask — that is the separate tech-debt task.
2. verificationMethod absent reads as api. No migration, and the four production organizations behave exactly as they do now.
3. The return contract is byte-identical. { isCitizen, isIdValid, isAutoCompleteOTP, clientName }, so no consumer needs a change.
4. Every consumer of the config document was audited, not assumed. One row feeds four features:

Consumer	Reads	Affected by this change?
auth-id.controller.ts — resident verification	the whole providerInfo	Intended. This is the feature
facility-seatsio.controller.ts — verification on the seat-map path	passes providerInfo straight through, inspects neither clientName nor requestParams	No — and it gains the feature for free. An organization on internal gets repository-backed verification on the seat-map purchase path too, with no code written for it
auth.controller.ts — login-method listing	only whether a row exists	Yes — the one real exposure. See §6.1.1
benefits.service.ts — Tel Aviv benefit registration	routes on providerName, then uses requestParams.registration	No. It never looks at the method, so requestParams is untouched and registration keeps working

The login listing is therefore the single place where this addition would introduce something that does not work. That is what §6.1.1 is for, and it is why the invariant above argues for decoupling rather than warning.

2. Affected Repos

Repo	What changes
backend-services	New ResidentIdentification + batch collections in apps/auth; ResidentVerificationProvider interface + InternalDbProvider; new enum value; import / lookup / batch endpoints on internal-gateway; async-task handler
support-tool-frontend	New page: upload the eligibility list (append / revoke), set the verification method, browse batches, look up a single ID
superco-consumer	[consumer subtask] suppress the ID box for a user whose resident_status is already approved

3. Architecture

The repository lives in apps/auth, co-located with citizen-id/, because verification is a request-path lookup on the event page; putting the collection anywhere else would add an RPC hop to every ID check. apps/auth already registers Mongoose, so this costs no new infrastructure.

The import is driven from internal-gateway (support-tool is an internal surface) but the parsing and writing happen in apps/auth, so the collection has a single owner.

3.1 Configuration surfaces — who sets what

Resident discounts are not driven by one flag. Four independent configuration points must line up, spread across three different admin surfaces, and understanding this is a prerequisite for the change:

#	What	Where it lives	Who sets it
1	Which segment is "the citizen segment" for this event	groups.citizen_segment_id — varchar(255) holding a Mongo segment _id	Event owner / community admin, via a dropdown on the event edit form ("payments" section)
2	Whether that dropdown is even visible	options row Plus details: citizen_segment (plus_select), resolved group → community → default	Support-tool operator (or a migration). Only ever enabled for JLM in code
3	The discount itself	A segment attached to the product/ticket, carrying discountMethod + discountValue	Event owner, on the ticket
4	Which segment gets attached on successful verification	organizations-providers-config.segmentation.label — by label, not by _id	Inserted into Mongo by hand today

The runtime chain is therefore: verify the ID → attach the member to the segment named by (4) → the member now belongs to that segment → the ticket's discount dropdown, which lists only segments the member belongs to, shows it → client-side JS auto-selects it by data-segid.

Two consequences worth stating plainly:

• citizen_segment_id is not a boolean. It names which segment confers the resident discount on this event. Any assumption that turning resident verification "on" is a flag is wrong.
• (1) and (4) are resolved differently and can point at different segments. The provider config names a segment by label and the lookup filters by communityId, so it correctly finds that community's segment — production has one segment labelled ירושלמי per community (376 segment documents in total, all the ירושלמי ones under org 229). The event, by contrast, pins one hard-coded _id. The two agree as long as the event's pinned _id belongs to the event's own community; a copied event, a moved event, or a community merge breaks that. See the failure mode in §9.

3.2 What production actually looks like

Measured read-only against IL production on 2026-08-18. The gap between "configured" and "used" is wide enough to change how this should be rolled out.

Fact	Count
citizen_id provider configs, all environments-wide	4 — herzliya (org 291), rishon_lezion (org 268), jlm (org 229), tel_aviv (org 5)
Communities with Plus details: citizen_segment enabled	111, across 4 orgs: jlm (229), JLM_East (482), TLV (5), jlm_youth (518)
Events with citizen_segment_id actually set	7, using 4 distinct segments

Three things fall out of this, and each is a finding in its own right:

1. The feature is configured for 111 communities and used on 7 events. Whatever the internal-DB path costs to build, it is not being retrofitted onto a large installed base. This lowers the regression risk of §13 task #8 considerably, and it means the rollout in §11 is effectively a greenfield launch for the first authority.
2. rishon_lezion is a clientName the dispatch does not handle. CitiesNamesEnum has only herzliya / jlm / tel_aviv, so org 268 falls to the default: branch and can only ever return "not a resident" — while auth.controller.ts still advertises a citizen_id login method for it because a config row exists. This is live today and predates the change.
3. Only jlm has segmentation.label set. attacheMemberSegment reads providerInfo.segmentation.label unguarded, so for herzliya, rishon_lezion, and tel_aviv a successful verification with a communityId and memberId would dereference undefined. It has evidently not been hit — consistent with 7 events using the feature — but the new path must guard it.

JLM_East (482) and jlm_youth (518) have the option enabled but no provider config at all, which is precisely the §9 "verification configured nowhere, box still shown" case, confirmed in production rather than hypothesised.

4. Data Model

Two new Mongo collections, both in apps/auth.

Collection	Fields	Indexes / constraints
resident-identification	organizationId, identification (ciphertext), batchId, uploadedBy, revokedAt?, revokedByBatchId?	unique(organizationId, identification); (organizationId, batchId)
resident-identification-batch	organizationId, kind (append | revoke), status, fileName, uploadedBy, totalRows, insertedCount, duplicateCount, invalidCount, revokedCount, rejectedRows[{line,reason}], totalBytes, processedBytes, lastProgressAt, failureReason	(organizationId, createdAt); partial (organizationId, status) for the running-import guard

One document per ID, not one document per organization holding a list. This was considered explicitly and rejected:

• The 16MB BSON document ceiling is a live constraint at this data size. A 24-character ciphertext plus document overhead is roughly 100 bytes per row, so a 200k-resident city exceeds the limit — and a city-sized list is exactly the expected input.
• Append-without-overwrite falls out of the unique index via insertMany({ ordered: false }). An array would need $addToSet against one hot document, serialising concurrent writes and rewriting the whole document each time.
• Verification becomes an indexed equality lookup, identical in shape to the existing search on users.identification.

Schema conventions follow the repo rules: singular class name implementing its interface, no extends Document, interfaces under libs/interfaces/auth/, and all indexes configured on the Schema instance rather than in @Prop.

The field is named identification — not citizenId — deliberately: libs/logger/src/sensitive-keys.ts already redacts that key, so log protection is inherited rather than re-implemented.

Migrations / backfills required: No. Both collections are new, and the enum change reuses existing string values.

4.1 How an import runs

Nothing large ever crosses a request or an RPC boundary. The browser uploads straight to S3; the request that follows carries only a pointer; a worker reads the object back in ranges and never holds more than one chunk.

support-tool ──PUT──▶ S3
support-tool ──POST {key,bucket,kind}──▶ internal-gateway
                                          │ creates the batch record (running)
                                          │ enqueues a pointer, returns immediately
                                          ▼
                                     auth worker
                                          │ HEAD → totalBytes
                                          │ loop: GET range → parse → bulkWrite → $inc progress
                                          │ delete the object only once every byte is read
                                          ▼
                                     batch record (done)

The queue message is a pointer, never the file — { key, bucket, organizationId, kind, uploadedBy, batchId }. Locally this costs no infrastructure at all: SqsDevRpcClientProxy falls back to the plain RPC client when no queue URL is configured, so the same code path runs against the local stack.

Reads are ranged, not whole-object. getStreamData materialises an entire object into a string, and an RPC boundary cannot carry a stream anyway. The worker instead asks storage-manager for byte ranges (S3 supports Range natively), keeps the remainder of a partial last line between chunks, and processes as it goes. Peak memory is therefore set by the chunk size, not the file size — which is what makes the row ceiling a promise rather than a hope. It also keeps S3 ownership inside storage-manager rather than giving apps/auth its own S3 credentials.

Read size and progress granularity are deliberately decoupled. Tying them together produces a useless progress bar: at roughly 10 bytes per row a 5 MB read holds ~500k rows, so a whole city would be two chunks and the bar would jump 0 → 50 → 100 with minutes of silence between. Reads are therefore 1 MB (about 100k rows, a few MB of parsed strings at peak), while progress is written every 5,000 rows regardless of where chunk boundaries fall. For a million-row file that is ten range reads and about two hundred progress writes — negligible load, and smooth movement at a 1.5s poll. On a small file the 5,000 is a threshold rather than a cadence, so a 3,000-row import simply reports once at the end.

Limit	Value
Row cap	1,500,000
File cap	25 MB
Read chunk	1 MB

Sizing check: Jerusalem is roughly a million residents, Tel Aviv 470k, Rishon LeZion 250k, and an eligibility list is normally a subset. At about 10 bytes per row a million rows is a ~10 MB file, so neither cap binds on a legitimate upload — the file cap exists to reject a wrong file, not to constrain a real one. Storage is a non-issue at roughly 150-200 bytes per row including the index.

These are sized by the largest real case, not the average, and deliberately so: Israel has roughly 257 local authorities across three tiers, with 80 municipalities holding about 78% of the population. That puts the average municipality near 100k residents and the average authority near 40k, while Jerusalem alone is about a million. A cap set near the average would fail exactly the authorities that matter most.

4.2 Validation: two tiers, and only one of them can fail the job

Tier 1 — the file, checked before anything is queued. Synchronous, so the operator gets an immediate, specific reason instead of a job that dies later:

Rejected when	Message
Not a .csv upload	wrong file type
Over the file cap	file too large
Empty object	file has no rows
The first chunk contains no usable identification	this does not look like an identification list

The last one reads only the first range, not the whole file, so it stays cheap. It is what catches "you uploaded the wrong export".

Tier 2 — individual rows, during the job, never fatal. A bad row is data to report, not a reason to throw away a good import. Rows are counted by reason and skipped; the batch still completes as done.

Reason	Meaning
not_numeric	the first column is not a number
wrong_length	more than 9 digits
check_digit	well-formed but fails the Israeli check digit
duplicate_in_file	the same person appears twice in the upload

The summary reports line, reason, and a masked value. rejectedRows holds the first 100 as { line, reason, identificationMasked } alongside full per-reason totals. A line number alone is enough to locate the row, but an operator scanning a report needs visual confirmation that they are looking at the right one, so the last four digits are shown — the same mask the row list uses, kept identical on purpose so one screen never means two things by a mask. The full number is never stored on the batch or sent to the browser.

A template is offered for download next to the upload control — a header row plus a few fictitious but checksum-valid identifications, served as a static asset from the support tool. It removes the most common cause of a rejected file, which is a column layout nobody documented.

4.3 What is true while an import is running

• The data is partially visible. Rows already written verify immediately. For an append that is monotonic — nobody loses eligibility mid-import, people gain it progressively. For a revoke the reverse holds. Both converge correctly and neither is harmful, but it is a real intermediate state and worth knowing.
• The object stays in S3 until every byte has been read. A failure therefore leaves the file in place and the batch failed, and re-running is safe because the writes are idempotent.
• Progress is written continuously, not once at the end — see §6.1.

5. API / Contracts

Written in the shape the team's Confluence design reviews use — literal paths with gateway placeholders and actual response bodies, not prose descriptions.

Presign an upload

GET {{internal}}/api/v2/organizations/:organizationId/resident-identifications/presign?fileName=&contentType=

{ url: String, key: String, bucket: String }   // key is always resident-identifications/{organizationId}/{uuid}

Start an import

POST {{internal}}/api/v2/organizations/:organizationId/resident-identifications/import

// request
{ key: String, bucket: String, kind: "append" | "revoke" }
// response — the existing AsyncTaskPollingInterface, unchanged
{ taskId: String, status: String, progress: Number, progressMessage: String, errors: [], pollingUrl: String }

Batch history

GET {{internal}}/api/v2/organizations/:organizationId/resident-identifications/batches?page=&pageSize=

{
  pagination: { currentPage, pageSize, totalCount, totalPages },
  data: [{
    _id: ObjectId,
    kind: "append" | "revoke",
    fileName: String,
    uploadedBy: Number,
    totalRows: Number,
    insertedCount: Number,
    duplicateCount: Number,
    invalidCount: Number,
    status: "running" | "done" | "failed" | "rolled_back",
    createdAt: Date
  }]
}

Look up one ID — "is this person in the list?"

GET {{internal}}/api/v2/organizations/:organizationId/resident-identifications/lookup?identification=

{ found: Boolean, revokedAt: Date | null, batchId: String | null, addedAt: Date | null }

The operator's real question when a resident complains they are not recognised. The query is encrypted and matched against the index - nothing is decrypted and no list is returned. This is the primary way an operator inspects the repository.

List rows for an organization — masked

GET {{internal}}/api/v2/organizations/:organizationId/resident-identifications?page=&pageSize=&batchId=

{
  pagination: { currentPage, pageSize, totalCount, totalPages },
  data: [{
    identificationMasked: String,   // "•••••6789" - last 4 digits only
    batchId: String,
    addedAt: Date,
    addedBy: Number,
    revokedAt: Date | null
  }]
}

Masking happens server-side: the row is decrypted, truncated to its last four digits, and only the masked form crosses the wire. A full ID number is never sent to the browser by this endpoint. Every call is logged as a PII read with the operator's id.

Roll back a whole upload

POST {{internal}}/api/v2/organizations/:organizationId/resident-identifications/batches/:batchId/rollback

Deletes only rows carrying this batchId; revokedAt set by a later batch is left intact. Returns AsyncTaskPollingInterface.

Unchanged — for reference, because the whole design hinges on not touching it

GET {{internal}}/api/v2/auth/citizen?citizenId=&organizationId=&communityId=&memberId=

{ isCitizen: Boolean, isIdValid: Boolean, isAutoCompleteOTP: Boolean, clientName: String, segmentAttached?: Object }

Import progress is polled through the existing GET {{internal}}/api/v2/tasks/status/:taskId; no new polling surface is introduced.

New RPC message patterns on apps/auth: IMPORT_RESIDENT_IDENTIFICATIONS, ROLLBACK_RESIDENT_IDENTIFICATION_BATCH, GET_RESIDENT_IDENTIFICATION_BATCHES_AND_COUNT. Endpoint enums live in @libs/enums/endpoints, per repo convention; internal RPC payloads are interfaces, gateway bodies are class DTOs.

Unchanged: GET /api/v2/auth/citizen keeps its request and response shape exactly. That is the point of the design.

Auth: every new endpoint is internal-gateway only and requires a support-tool operator. No organization-admin surface is exposed — the repository is a municipal resident register and stays behind internal staff.

6. Frontend & Components

The support-tool page reuses the established upload composition rather than inventing one: the presign → PUT → POST {key,bucket} flow already used for external-coupon CSV import, and the existing usePersistedAsyncTaskPolling progress/resume behaviour.

Component	Reuse / new	Source	States handled
File dropzone	reuse pattern	file-parse-dropzone (management-webapp)	default / validating / too-large / parse-error
Import progress	reuse	async-task polling client + progress component	queued / running / done / failed / cancelled
Batch history table	reuse	existing support-tool table	default / loading / empty / error
Verification-method selector	new (small)	plain select bound to the provider config	external API / internal database
Single-ID lookup	new (small)	text input + result state	idle / searching / found / not-found / revoked
Masked row list	reuse	existing support-tool table	default / loading / empty / error

6.1 The support-tool screen — what an operator sees

Four jobs, in the order an operator needs them:

1. Set how an organization verifies. A three-way choice per organization: API / internal / none. Picking API never asks which municipality — see below. None switches residency verification off.
2. Load a list. Pick the organization, choose append or revoke, download the template if needed, drop the file, watch progress.
3. Answer "why isn't this resident recognised?" Type the ID, get found / not-found / revoked plus when it was added and by which upload. This is the day-to-day use and needs no listing at all.
4. See what was loaded. Batch history with counts and rejected-row summaries, and — when a row-level view is genuinely needed — a masked list showing only the last four digits.

Progress is polled, not pushed. The screen polls the batch record roughly every 1.5s, matching usePersistedAsyncTaskPolling in organization-dashboard, which already survives a page reload. No socket is introduced for this.

That requires the worker to write during the import rather than once at the end. It takes totalBytes from a HEAD before the first range, then $incs processedBytes and the counters after every chunk. The screen therefore shows a real percentage plus running totals of inserted, duplicate and invalid, and finishes on a summary that includes the rejected-row breakdown.

Job 1 is not optional. Verification starts by loading the organization's config row and returns "provider not found" before touching the repository if there isn't one — so without this control the acceptance test in §16 cannot pass without a developer inserting a document into Mongo by hand. Today there is no create path at all: the only update method accepts three unrelated fields, has no upsert, and is scoped to type: events. All four production rows were inserted manually.

The operator picks the method; the code picks the integration. Two fields with different owners:

Field	Values	Who sets it	When it changes
verificationMethod	api | internal | none	Operator, in the support tool	Whenever the organization's arrangement changes
clientName	the city integration, as today	Developer, when an integration is written	Only when new code ships

Choosing api dispatches on the clientName already attached to that organization, exactly as it does now. Support cannot pick a wrong municipality because support never picks one.

When api is offered. Only when the organization's clientName resolves to an integration the code actually implements; otherwise it is greyed out with a plain reason. That rule keys off what the resolver implements rather than whether the field merely holds a value, so Rishon LeZion's api option is correctly disabled (§3.2) without anyone fixing that defect as such.

This is a change from an earlier draft of this document, which argued for a single field. That argument holds when two fields encode the same axis and can therefore contradict. These do not: one says how we verify, the other says which implementation to use when the answer is "by API". clientName never expresses "internal", so the pair cannot disagree, and the two-field shape survives a round trip — switching to internal and back keeps the city, which a single overloaded field would lose.

No migration: a row carrying a clientName and no verificationMethod is read as api, which is exactly today's behaviour.

none is worth having as an explicit state so an organization can be switched off without deleting its config and losing its segmentation and citizenTestId. A missing row displays as none, and choosing a method on such an organization creates one.

none means "does not verify residency" — not "off". The same document feeds four features (§1.1), and two of them ignore the method entirely. Most consequentially, Tel Aviv's external benefit registration routes on providerName and reads requestParams.registration, so it keeps running regardless. Two requirements follow: the label reads "does not verify residency", never "disabled"; and when an organization has other things hanging off the same document, the screen says so.

Why masked rather than full numbers. The repository is a municipal resident register. A screen that pages through full ID numbers turns every support-tool operator into a holder of that register, which is a materially larger exposure than anything the tool grants today. The masked list plus the point lookup delivers the whole operational need without creating a browsable register. Masking is applied server-side so the full value never reaches the browser, and both the lookup and the list log a PII read against the operator's id.

6.1.1 The login coupling — what the decision actually is

First, what "log in with an ID number" is. It is a separate, pre-existing login method, unrelated to this feature. On the login screen a person types their ID instead of an email or phone. The system asks the municipality's API who this is, gets back that resident's email and phone from the municipal register, shows "where should we send the code?", and sends an OTP there. The benefit is that the person does not have to remember which address they signed up with. AuthController::actionValidateId stores exactly that:

Yii::$app->session->set('citizen_id_data', [
    'id' => $id,
    'auto' => $response->isAutoCompleteOTP,
    'email' => $response->citizenEmail,
    'phone' => $response->citizenPhone,
]);

Now the coupling. auth.controller.ts:97-121 builds the list of login methods a community offers. It fetches the organization's citizen_id config and, if any row exists at all, appends that login option:

if (citizenIdLogin) {
  result.push({ type: 'citizen_id', name: 'citizen_id', order: 99, isPrivate: true });
}

It inspects nothing else — not clientName, not the method. And creating that row is precisely what the new screen exists to do.

Why that breaks. Our repository holds ID numbers and nothing else — no email, no phone. It is a list of numbers from a file. So for an organization set to internal:

1. The operator sets the method; a config row is created.
2. "Log in with ID number" appears on that organization's login screen.
3. A real person picks it and types their ID.
4. Verification succeeds — the ID genuinely is in the list.
5. But there is nothing to return. citizenEmail and citizenPhone are empty.
6. They reach the "where should we send the code?" step with no contact details, and fall back to typing their phone by hand.

They typed an ID, passed through a pointless screen, and ended up doing what they would have done anyway. This is the same family of defect as the Rishon LeZion case in §3.2 — a login option that appears and leads nowhere — and it would be introduced by us rather than inherited.

Two things are conflated: how an organization verifies residency (this feature) and whether people can log in with an ID number (an authentication choice that depends on an external register we do not have). One row currently decides both.

Option	What it takes	Consequence
A. Warn in the UI	One sentence next to the selector: setting a method also enables ID login for this organization	No backend change, no risk to a live auth path. The dead-end login option still reaches real users of that organization — the operator just knows it will
B. Decouple	A new explicit field on the config, defaulted to true so today's four organizations behave identically; the login listing reads that field instead of row-existence. An organization on internal gets it false	The real fix: no dead-end option ever appears, and it also disarms the Rishon oddity. Touches a production authentication surface, which this subtask otherwise does not
C. Nothing	—	An operator enabling a discount feature silently changes how people log in. Not recommended

Recommendation: B.

An earlier draft of this document recommended A, on the grounds that the acceptance test does not touch login and B edits a live authentication path. The governing invariant in §1.1 settles it the other way: nothing may change or stop working as a result of this addition. A knowingly ships a login flow that cannot complete — introduced by us, not inherited — which is exactly what the invariant forbids. A warning tells the operator about the breakage instead of preventing it.

B also costs very little and carries no risk to existing organizations: one field, one condition in the listing, and a default of true so the four production organizations behave byte-identically to today. An organization on internal gets it false, so the dead-end option never appears at all.

The remaining tech-debt — that one document quietly feeds four features (§1.1) — is not solved by B. B separates one of the four. It is the right first step, and the rest belongs in its own task.

What we need from you: just A or B. Everything else about the screen is settled.

Why masked rather than full numbers. The repository is a municipal resident register. A screen that pages through full ID numbers turns every support-tool operator into a holder of that register, which is a materially larger exposure than anything the tool grants today. The masked list plus the point lookup delivers the whole operational need — a specific resident's status, and what a given upload contained — without creating a browsable register. Masking is applied server-side so the full value never reaches the browser, and both the lookup and the list log a PII read against the operator's id.

6.2 Client-side behaviour on the event page — [consumer subtask]

Not in scope for the infra subtask. Documented so the seam is explicit: today the box's render condition checks the Redis session key citizen_id_data, not the user's status, which is why an already-approved user is asked again. There is no client state on this page — the verify button re-POSTs the whole form and the server rebuilds from the session.

Condition	Today	After this change
Event has no citizen_segment_id	No ID box, no citizen discount	Unchanged
citizen_segment_id set, session has no valid citizen data	ID box shown	Shown only if the user's resident_status is not already approved
citizen_segment_id set, user already approved	ID box shown again — the condition checks the session, not the user	Not shown; the member already belongs to the segment, so the discount is simply available
Verify succeeds	Form re-renders; JS selects the matching option[data-segid] in the ticket's discount dropdown	Unchanged, plus resident_status is promoted to AUTO_APPROVED
Verify fails	Inline error in the box's help-block	Unchanged

Two gates on this page are client-side only and stay that way — they are not part of this change but bound its guarantees: resident-only ticket enable/disable via the data-residents tri-state, and the Continue-button block. Server-side enforcement of ticket eligibility remains segment-based.

All operator- and user-facing copy for the box comes from the options table (subscriber citizen id label, subscriber citizen id button), so wording changes are a per-community DB edit, not a code change — the migration only seeds defaults.

7. Observability Instrumentation

Event	Where it fires	Payload
Import started / finished	ResidentIdentificationImportService	{ organizationId, batchId, kind, totalRows, insertedCount, duplicateCount, invalidCount }
Verification outcome	InternalDbProvider.verify()	{ organizationId, isCitizen, isIdValid } — never the ID itself
Repository lookup failure	InternalDbProvider.verify()	logger.error with { organizationId }, then rethrow
PII read from the support tool	lookup and masked-list handlers	{ organizationId, operatorUserId, action: 'lookup' | 'list', rowCount } — never an ID, masked or otherwise
Verification method changed	provider-config update handler	{ organizationId, from, to, operatorUserId }

Audit destination: the structured log, not a collection. There is no shared audit facility in the repo — what exists is a per-domain pattern (agent-tool-audits, platform-jobs/src/audit, report-schedule-audit.service). A dedicated collection was considered and rejected for this subtask: the view is masked, access is internal-staff-only, and the log already flows to Loki. If a retention or regulatory requirement appears later, a collection can be added without changing anything built here.

Every log message must be unique and carry context in the object, not interpolated into the string, per the repo's error-logging rules. No log line may contain a raw or encrypted ID number.

8. Integration Points & Dependencies

Dependency	Why	Failure mode handling
S3 (via storage-manager)	Carries the file past the 3MB internal-gateway body limit	Presign failure → 502 to the operator; import failure → batch marked failed, object retained for retry
SQS	Carries the pointer to the worker so no request holds the import	Redelivery is safe (idempotent writes + counter reset). Locally SqsDevRpcClientProxy substitutes a plain RPC call, so no queue is needed to run this
IdentificationService	Encryption, shared with users.identification	Missing key → fail fast at boot, never silently store plaintext

Cross-repo coordination. backend-services ships first (additive: new collections, new enum value, new endpoints — no existing behaviour changes until an organization's config is switched to internal_db). support-tool-frontend and superco-consumer follow independently.

9. Failure Modes & Error Handling

Scenario	Technical handling
ID fails the Israeli check digit	Existing AuthHelper.isValidateCitizenILId gate returns { isCitizen: false, isIdValid: false } before any lookup — unchanged
ID not in the repository	{ isCitizen: false, isIdValid: true } — a clean "not eligible", distinct from a malformed ID
ID present but revoked	Treated as not found; the verification query filters revokedAt: null
Mongo unavailable during verification	Log and throw. Do not return { isCitizen: false } — see the note below
Uploaded file has invalid rows	Rows counted into invalidCount and skipped; the batch still succeeds and reports the count. A file that is entirely invalid fails the batch
Duplicate IDs within the file, or already present	Deduped in-file, then absorbed by the unique index with ordered: false; counted into duplicateCount, never an error
Import interrupted mid-way	Batch stays running; rows already inserted are valid and idempotent, so a re-run of the same file is safe
Rollback of a batch that was partly revoked by a later file	Rollback removes only rows carrying that batchId; revokedAt set by a later batch is left intact
Verification succeeds but no discount appears	The member is attached to their own community's segment (resolved by label + communityId), but the event pins a segment _id belonging to a different community — a copied or moved event, or a community merge. The user gets AUTO_APPROVED, the box disappears, and the ticket still shows full price. Guard: on saving an event, validate that citizen_segment_id resolves to a segment whose communityId is the event's own. Detect: log the resolved segment id on attach, so the two can be compared after the fact
A second import starts while one is running	Rejected. The upserts are idempotent so it is technically safe, but it confuses the operator and distorts the counters. Guarded by a partial index on (organizationId, status)
SQS redelivers a message after a worker crash	The writes are idempotent, but a blind $inc would double-count. On picking up a message the worker $sets the counters and processedBytes back to zero before incrementing, so a redelivery restarts cleanly
The worker dies without redelivery	The batch stays running forever. lastProgressAt is written on every chunk and the screen shows "running since X", so a stuck import is visible rather than silent. An automatic staleness sweep is a follow-up, not v1
Invalid rows in an otherwise good file	Counted by reason, skipped, and reported. They never fail the job — see §4.2
The whole file is unusable	Rejected up front by tier-1 validation on the first chunk, before anything is queued, with a specific reason
Provider config has no segmentation	attacheMemberSegment dereferences providerInfo.segmentation.label unguarded. Three of the four production configs have no segmentation at all (§3.2), so this is a latent TypeError, not a hypothetical. The new path must null-check and log rather than throw
Event's citizen_segment_id dropdown not visible to the admin	The Plus details: citizen_segment option is not enabled for that community (§3.1 row 2). Verification will work but no event can be wired to use it — so onboarding an authority means enabling this option too, not only creating the provider config

A note on an adjacent defect, not fixed here. CitizenIdService currently wraps its whole dispatch in a catch that returns { isCitizen: false, isIdValid: false }, so a municipality API outage is indistinguishable from "this person is not a resident" — residents silently lose their discount with no signal. The new provider must not copy that behaviour; it logs and throws. Repairing the existing three cities is left to the tech-debt task in §13, and is called out here so the inconsistency is a known, recorded decision rather than an oversight.

10. Breaking Changes

None. All collections and endpoints are new; the enum addition reuses existing string values so stored configs are untouched; the verification contract is unchanged. An organization's behaviour changes only when an operator explicitly sets its verification method.

11. Rollout

• Feature flag: not required. The config discriminator is the switch — an organization is on the new path only if its citizen_id config says internal_db. Rollback is editing one field back.
• Migration / backfill: none.
• Phased rollout: per organization, naturally. First target is a single authority with no API, agreed with Aviad.
• Rollback plan: set the config value back to the previous provider (or remove the config). No deploy needed.
• QA bypass: citizenTestId already exists on the provider config and is honoured before the check-digit gate — reuse it rather than adding a test hatch.

12. Testing Strategy

• Unit: the method resolver (api dispatches on clientName, internal hits the repository, none and a missing verificationMethod behave correctly); InternalDbProvider.verify() for found / not-found / revoked / DB-error; the CSV parser for valid, malformed, duplicate, empty, header and leading-zero rows; the masking helper.
• Integration: the import against mongodb-memory-server, following the repo's existing functional-spec pattern — seed a file, run the real import, assert inserted/duplicate/invalid counts and that a re-run is idempotent. Same for revoke and rollback.
• Acceptance: the Postman cases in §16, run by the reviewer against a seeded organization. This is the subtask's gate.
• Manual / QA: RTL rendering of the support-tool screen; operator-facing error copy for an oversized or wrong-format file; the lookup screen's not-found and revoked states.
• [consumer subtask] E2E: on the event page, an ID in the list makes the discounted ticket selectable and flips users.resident_status_id; reloading as the same user no longer shows the box.
• Data fidelity: validate the parser against a real authority file (or a faithfully-shaped sample) before shipping, not only a hand-written CSV. Real municipal exports carry BOMs, leading zeros stripped by Excel, headers in Hebrew, and trailing blank rows — every one of which silently corrupts an ID. Leading-zero loss is the dangerous one: it turns a valid ID into a different valid-looking string. Confirm the real column layout with the data owner.

13. Task Breakdown

In scope — CU-86cb79t1h (infra + support tool)

#	Task	Repo	Effort	Risk	Depends on
1	ResidentIdentification + batch schemas, interfaces, indexes	backend-services	S	Low	—
2	ResidentVerificationProvider interface + resolver + InternalDbProvider; new enum value	backend-services	S	Med	#1
3	Import: presign + import endpoints, tier-1 validation, ranged CSV reader, chunked insert, progress writes, delete the object on completion	backend-services	M	Med	#1
3b	AUTH_SERVICE_SQS token, connection module, config, SQS transport strategy in auth main.ts, worker handler	backend-services	S	Med	#1
3c	GET_FILE_RANGE on storage-manager	backend-services	XS	Low	—
4	Revoke file + batch rollback	backend-services	S	Low	#3
5	Lookup endpoint + masked list endpoint + provider-config create/update, all audit-logged	backend-services	S	Low	#1
5b	Decouple ID-login from row-existence: explicit field defaulted true, listing reads it (§6.1.1 option B)	backend-services	XS	Low	—
6	Support-tool screen: method selector, upload with template download and polled progress, batch history with rejected-row summary, single-ID lookup, masked list	support-tool-frontend	M	Low	#3, #4, #5
7	Save the verification endpoint in the R&D Postman workspace with sample requests — this is the reviewer's acceptance test	—	XS	Low	#2

Tasks #1 and #2 depend on none of the open questions and can start immediately.

Deferred — CU-86cb79tmx (connecting consumers)

#	Task	Repo	Effort
8	resident_status → AUTO_APPROVED promotion, guarded to promote only	backend-services	XS
9	Hide the ID box for already-approved users	superco-consumer	XS

These two ship together or not at all — #8 alone has no visible effect, because the box's render condition checks the session rather than the user's status.

Separate — tech debt

#	Task	Repo	Effort
10	Move herzliya / jlm / tel_aviv onto the provider interface	backend-services	M

Effort scale: XS = <1 day · S = 1-3 days · M = up to 1 week · L = 1-2 weeks.

Note that the parent ClickUp estimate is 3 story points, set before the scope was mapped. The infra subtask alone does not fit 3 points; the estimate should be revisited at planning.

14. Risks & Alternatives Considered

• Risk: this is a wholesale municipal resident register in our database. The encryption reuses IdentificationService (AES-256-CBC with a fixed IV), which must be described honestly: the fixed IV makes the ciphertext deterministic — that is precisely what makes lookup possible — and deterministic encryption therefore leaks equality. The key is shared with users.identification. This protects against a database dump without the key; it does not protect against anyone holding it. → Mitigation: internal-staff-only access, no organization-admin surface, no ID in any log, S3 object removed after import, and a privacy-compliance review before this doc is approved.
• Risk: leading-zero corruption — an ID that passed through Excel on the authority's side arrives with its leading zero stripped, silently becoming another valid-looking number. → Mitigation: parse IDs as strings, left-pad to 9 digits, and reject rows failing the check digit into invalidCount where they are visible rather than silently accepted.
• Risk: a support-tool screen over a municipal resident register. → Mitigation: masked to the last four digits server-side, a point lookup instead of a browsable list for the common case, and an audit line per PII read (§6.1).
• Risk: the tech-debt refactor touches three live payment-adjacent integrations. → Mitigation: §3.2 measured the real exposure — 7 events in total production use this path — so the blast radius of a regression is small and knowable. That is an argument for doing the refactor, not for deferring it indefinitely.
• Risk: apps/auth has never been a queue consumer. It has no SQS token, connection module, or transport strategy in main.ts — all five services that do were wired for it. → Mitigation: the connection module is a well-trodden ~40-line pattern with a built-in local fallback, so the only genuinely external work is provisioning a queue and one env var per environment. Verified end to end locally before it ships.
• Alternative considered: hashing (HMAC + pepper) instead of encryption → rejected because the system already has a vetted, deployed mechanism for storing ID numbers and a second scheme would fragment key management for no added protection given the same lookup requirement.
• Alternative considered: organization-dashboard upload → rejected; the register stays with internal staff.

15. Open Questions for the Build Stage

Resolved

Question	Decision
Definition of Done for the infra subtask	Verification returns a correct answer, demonstrated by a saved Postman request the reviewer runs himself. Not "a resident gets a discount" — that needs the consumer subtask and the four configuration points of §3.1.
Method values	api / internal / none. internal rather than local, because "local authority" is the domain term for a municipality and would read ambiguously.
File format	CSV only. Municipal systems export CSV, the reader streams so the ceiling is high, and dropping XLSX removes the CVE that would otherwise be an accepted risk. See §4.1.
S3 object retention	Delete the object once the rows are written to the collection. The batch record keeps counts and metadata without the PII.
Support-tool scope	Upload, verification-method selector, batch history, single-ID lookup, masked row list. It does not create the community option or the segment — those belong to the consumer subtask. It must be able to set the verification method on the provider config, or the reviewer's Postman call returns "provider not found".
Displaying ID numbers	Masked to the last four digits, server-side, plus a point lookup for "is this ID in the list". Not a browsable register. See §6.1.
How many communities have the option enabled?	111 across 4 orgs; only 7 events actually use it. See §3.2.
Audit destination	The structured log, not a dedicated collection. See §7.
Where the config row comes from	The support-tool screen owns it — a three-way method selector (api / internal / none) in verificationMethod, with clientName left to the code. Support never picks a municipality. See §6.1.
Volume limits	1.5M rows / 25 MB, inline below 5,000 rows — sized by the largest municipality, not the average. See §4.1.
The two live defects in §3.2	Not addressed. Noted for the record; neither is caused by this change, and neither is reachable under this subtask's acceptance test, which passes no communityId / memberId and so never enters segment attachment.

Still open

#	Question	Owner	Resolution path
1	The login coupling — A or B Resolved: B (§6.1.1). The §1.1 invariant decides it — A would ship a login flow that cannot complete, introduced by this change. B is one field, one condition, and a default that leaves the four existing organizations byte-identical.	—	✅ resolved
2	Confirm the staging target and the reviewer's internal-gateway access. Recommendation in §16: a purpose-made organization, or staging org 53 as the fallback.	Ariel / Aviad	Before the review is scheduled
3	[consumer subtask] Staleness. The list is a point-in-time snapshot, so a permanent AUTO_APPROVED means someone who left the city keeps the discount indefinitely. Keep it permanent, re-verify per event, or add a validity window? Note the revoke file (D5) is the data-side half of this and is in the infra subtask.	Aviad / Gali	Before the consumer subtask
4	[consumer subtask] Which statuses count as "resident". CITIZEN_RESIDENT_STATUS_IDS in the seats.io path is [AUTO_APPROVED, ADMIN_APPROVED, DECLARED_RESIDENT]; the legacy widget uses ApprovalStatuses::isApprovedStatus(). These disagree today.	eng	Before the consumer subtask

Known issue, deliberately out of scope

The ID box on the event page renders based only on the event's citizen_segment_id, with no check that the organization has any verification configured. An organization with no citizen_id config still shows the box; the user types an ID and the verification fails silently (the gateway returns an empty payload with "provider not found"). This predates the change and is recorded here rather than fixed, to keep this task's scope contained. It matters less after this feature ships, since any organization using resident discounts will then have a config — but it should be filed.

16. Definition of Done

The acceptance test

The reviewer's check is a saved Postman request against a seeded test organization. It exercises verification only — passing no communityId / memberId means segment attachment is skipped, so the call has no side effects:

GET {{internal}}/api/v2/auth/citizen?citizenId=<in the list>&organizationId=<test org>
→ { isCitizen: true,  isIdValid: true }

GET {{internal}}/api/v2/auth/citizen?citizenId=<not in the list>&organizationId=<test org>
→ { isCitizen: false, isIdValid: true }

GET {{internal}}/api/v2/auth/citizen?citizenId=<fails the check digit>&organizationId=<test org>
→ { isCitizen: false, isIdValid: false }

This is an internal-gateway route, so the reviewer needs the internal base URL — worth confirming access before the review is scheduled.

Which organization to test against

Staging currently holds five citizen_id configs: org 76 (herzliya, has a citizenTestId), org 53 (rishon_lezion), org 229 (jlm, with segmentation.label), and orgs 5 and 234 (tel_aviv).

Recommendation: a purpose-made staging organization. The repository is append-only by design, so loading fictitious ID numbers into a real municipality's organization leaves data whose only removal path is the batch rollback we are building in the same subtask — testing the loader with the loader. A dedicated organization also lets the test own a known set of in-list and out-of-list IDs, which a shared organization cannot.

If creating one is a hassle, use staging org 53 (rishon_lezion). Its clientName matches no implemented case, so its ID verification is already dead — nothing that works today can regress. It doubles as the natural test of the greyed-out api state, and switching it to internal is a strict improvement over its current behaviour.

Do not use org 229 (jlm). It is the one staging organization whose citizen verification actually functions end to end, including segmentation.label; it is the reference for comparing old and new behaviour and should stay untouched.

Test IDs should be fictitious but checksum-valid, so the check-digit gate is exercised rather than bypassed. citizenTestId exists as a bypass but is not needed here.

Checklist — infra subtask

• The three Postman cases above return the stated results against a seeded organization.
• A revoked ID returns isCitizen: false with isIdValid: true.
• Import is idempotent — re-running the same file inserts nothing new and reports duplicates.
• CSV parses correctly, including leading zeros, a BOM, a header row and trailing blank lines.
• A CSV of ~1M rows imports within the async path; a non-CSV upload is rejected with a clear message.
• The method selector creates a config row for an organization that had none, and the reviewer's Postman call then succeeds.
• Parser validated against a real authority file, not only a hand-written fixture.
• Batch rollback removes only that batch's rows and leaves later revocations intact.
• The S3 object is gone after a successful import.
• No ID number, encrypted or plain, appears in any log.
• The masked list never sends a full ID to the browser; lookup and list both write an audit line.
• Verification failure due to an unavailable database logs and throws — it does not return "not a resident".
• Endpoints saved in the R&D Postman workspace with sample requests and saved responses.
• privacy-compliance review completed.
• Support-tool screen renders correctly in RTL.
• Tech-debt task (#10) filed.
• Code reviewed and merged; tests per §12 passing.

Deferred to the consumer subtask

• resident_status is promoted, never downgraded.
• A returning approved user is not asked to verify again and keeps the discount.
• End-to-end on a real event: the segment attached on verification is the one the event's citizen_segment_id names, and the ticket price actually drops (§3.1 / §9).