Annotation question extensibility: architecture and delivery map¶

Decision record for how SyRF's annotation-question model grows. Individual features ship in focused PRs, each aligned to this document. The underlying ideas are in response-modes-and-metadata-context.md and the original plan 2026-08-05-configurable-response-modes-and-metadata.md; this document records what was decided on 2026-08-11 and the approved amendments recorded on 2026-08-12.

Decisions¶

Vehicle: additive on live schema-v0. New capabilities are optional, additive fields on today's AnnotationQuestion (and response documents), the way ADR-011 added ConditionalParentAnswers. Nothing waits on the dormant QM v2 stack; when QM v2 revives, these fields become AQVersion content properties (edits create a new AQVersion) without re-design.
Rendering surface: AF2 first. AF2 is in active delivery (Phase 4, PRs #2731–#2737 et seq.). New reviewer-facing UI — the outline panel, response-mode chooser, metadata display — is built in AF2. The legacy v1 form receives no new interaction surfaces beyond the repeatable-group chrome it already has, driven by the answer label (with heuristic fallback). The cross-renderer DisplayLabel ?? Value correction in Decision 10 changes copy in existing option controls but adds no control or workflow. We do not otherwise grow the form AF2 is replacing.
Branch overview: persistent outline/navigation panel (not a per-question summary list). It lists branch questions and their branches with status, and navigates on click. Because it spans the whole form it is an AF2 feature, integrated with AnnotationFormV2Store's open-unit/paging state — a jump must open the target unit/page, honouring AF2's bounded rendering rather than assuming the target is mounted.
Umbrella doc lives here (this file), merged via the same PR that promoted the design context, so implementation PRs cite a tracked, merged decision.
requiresReason is carried by a platform-reserved _reason metadata key, injected into the definition as a required, mode-scoped field whenever requiresReason is set — not by a first-class reason property on AnnotationResponse. One validation path instead of two, an export column from the existing q__<key> scheme, and no extra persisted field needing its own writer-compatibility floor. It follows that the explanation is exactly as strong as mode-scoped metadata validation; if that turns out too weak to enforce a required reason, this decision is what should be revisited, not patched around.
Required metadata binds an ordinary value only. A question-level required field is not enforced when the reviewer selects a response mode; requirements that should apply to a mode are declared on mode-scoped definitions. Asking for the unit of a value the reviewer has just recorded as not reported is incoherent, and that is the case response modes exist to serve. Chosen as a blanket rule over per-field applicability because blanket → per-field (appliesTo) is additive later, while per-field → blanket would invalidate configuration already written.
Every response records the definitionVersion it was written against. A monotonic integer on the response, stamped at write time. Schema-v0 has no immutable AQVersion, so without it a later definition edit leaves export and audit unable to tell which semantics a historical answer was recorded under. One integer per response is the cheap half of that problem, and it keeps the expensive half open: whether definitions are frozen once used or snapshotted per response is not decided (see Open questions).
Production bulk question import is a .NET + existing SyRF web feature. CSV is the core interchange format; the guided workbook is a second view of the same versioned tabular contract. The production runtime parses, validates, previews, and commits in .NET, and the current question-design surface hosts the Angular flow. QM v2 later mounts the same UI/API contract. The Python importer is an admin-only bridge and reference implementation during delivery, not a second permanent production runtime.
Bulk-import success receipts and operational attempts have separate lifecycles. A successful import gets an immutable audit receipt in the same Mongo transaction as the project mutation. Authorized attempts also get a privacy-bounded, short-retention operational job visible in recent history; denied/pre-authorization activity belongs in security telemetry. A failed job can never masquerade as a transactionally committed import.
Option values and display labels are distinct. The stable option Value remains the persistence, condition, annotation, and export key. An optional DisplayLabel is reviewer-facing copy and falls back to Value when absent. This is a committed capability, not merely importer syntax: schema-v0 and schema-v1 must both round-trip it before any import writer can create a non-null label.
Repeatable split branches are locally collapsible in v1. This is a focused extension of the existing v1 repeatable-group chrome from Decision 2, not an AF2 feature substitute. When a repeatable question has two or more separate annotation instances (Multiple && !AnswerArray), each branch exposes an independent accessible disclosure with its top-level answer and status. Disclosure state is session/UI-only, keyed by annotation identity, and never persisted. The supplied screenshot established the scan/collapse/expand outcome but is inspiration rather than a pixel-perfect contract; existing SyRF/Material patterns, accessibility, and responsive evidence govern the implementation.

The spine and its optional extensions¶

The question definition remains SyRF's nested tree. Every addition is an optional field; a definition using none of them is byte-for-byte today's behaviour.

AnnotationQuestionDefinition (schema-v0 today; AQVersion content under QM v2)
├─ existing: text, description, controlType, options, Optional,
│            Multiple/AnswerArray, Target (+ ADR-011 conditional parents)
├─ [1] answerLabel?    — singular noun for one repeatable answer instance
│     answerLabelGenerated? — true generated; false authored; null legacy/unknown
├─ [2] responseModes?  — project-defined alternatives to answering
│        platform supplies mechanics only:
│        suppressDescendants?, requiresReason?, mode-scoped metadataFields?
└─ [3] metadataFields? — typed response qualifiers (unit, evidence, confidence)
         key, label, type, options?, required?, display: inline | details

AnnotationResponse
├─ value XOR responseModeId   — mutually exclusive, never both
├─ metadata?                  — validated against the definition's fields
└─ definitionVersion?         — monotonic integer stamped at write (Decision 7)

Derived runtime state (computed, never stored)
└─ available | suppressedByAncestor (+ which ancestor)

Hard constraints¶

These are load-bearing; every implementation PR must respect them.

Writer-compatibility floor (ADR-011 precedent). Project writes replace the full Mongo document, so writers that do not know a field silently drop it. For every additive field: persistence round-trip (API + PM readers and writers) deploys before anything writes the field. Rollback below that release loses data written since.
Boundary discipline. Metadata fields never have children or conditional logic. Only response modes may affect descendants. This is the anti-"second form engine" rule; it is what keeps each PR small.
Derived state is never stored. suppressedByAncestor is computed from the hierarchy, conditions, and selected modes. It is not a reviewer answer and must never overwrite one.
Suppression is scoped to the answer instance, not the question. A repeatable parent stores each instance as its own Annotation, with descendants linked by annotation ParentId/Children. Suppression state keyed by questionId alone cannot distinguish one branch from its siblings, so selecting a suppressing mode in branch 2 would suppress descendants in branch 1 as well. Key it by the ancestor annotation instance.
Preserved means the backend must not tree-shake it. Today ExtractionInfo.AddAnnotations removes every existing annotation whose question is in the submitted stage before adding the submission. If AF2 omits suppressed descendants, that path deletes the answers this document promises to preserve. The domain change lands before the reviewer UI (see the delivery map), with a suppress-after-answer regression test.
Preserving an answer creates an export hazard — resolve it there too. The constraint above keeps a descendant answer when an ancestor later suppresses it. WideDataFormatRowWriter.AddAnswerQuadFromGroup exports every stored matching annotation, so that preserved answer would be emitted as though it were live and silently corrupt a downstream analysis — a worse outcome than deleting it, because it is indistinguishable from real data. Exports must resolve suppression: either omit inactive values or expose an unambiguous activity/status column. A suppress-after-answer export test belongs alongside the submission one.
Authoring is gated on actual AF2 eligibility, not just the flag. The schema flag is environment-wide, but AF2 fails closed to v1 for extraction stages, Experiment questions, unsupported question graphs and other eligibility failures, and eligibility is evaluated per host (stage-review, preview, reconcile) rather than once per environment, so "what the renderer supports" is not a single answer. Without a project/stage capability gate, an administrator can configure response modes and required metadata for a question whose reviewers permanently receive v1, which has no controls for supplying them — the reviewer then cannot complete the form and cannot see why. The designer surface must check the same eligibility the renderer does.
Metadata keys must not collide with generated export stems. Keys are project-defined and exported as q__<key>, so a key of value or response_mode would collide with the generated core columns. Reserve those stems, and validate at definition-save time rather than at export time — failing an export is far too late to tell an administrator their key is invalid. Project-defined keys must also not start with _: that prefix is reserved for platform-injected fields such as Decision 5's _reason, and without the rule a later platform key would silently collide with project data already written under the same name. Because the export scheme is q__<key>, the leading underscore is stripped when generating the column, so _reason exports as q__reason rather than q___reason — and that stripped form is itself reserved, or a project key of reason would collide with it.
Project-defined labels, stable machine keys. SyRF ships no global "Not reported" enum; projects define labels. IDs/keys are stable for persistence, export, and audit even if labels are edited or localised. (Covidence's own extraction guidance — never leave blanks; record "Not applicable/Not reported/Unclear" — confirms the modes are standard practice and the labels review-specific.)
Option display copy never becomes identity. Conditions, annotations, parent filters, exports, and audit continue to bind the stable option Value; v1 and AF2 render DisplayLabel ?? Value. Renaming display copy must not change an answer or reference. Because DisplayLabel is additive inside a whole-document aggregate, its persistence release must deploy, become healthy, and drain every older writer before the first non-null label is written.
Feature-flag discipline. answerLabel: no flag — additive, render falls back to the existing heuristic, no rollback floor beyond the writer rule. Response modes + metadata fields: flagged (schema-driven flag via env-mapping.yaml), enabled per environment only after compatible writers are deployed; reviewer-side rendering is AF2-only, so project-level enablement should track AF2 serving the annotation form.
Bulk import is a first-class writer. Preview is read-only, but commit creates a complete in-memory plan and persists it atomically rather than replaying the public per-question PUT endpoint. It inherits every additive field's writer-compatibility floor: an import capability may parse a field before activation, but it must fail closed rather than write that field until the relevant persistence build is deployed, healthy, and old writers have drained.

The extensions¶

[1] Answer label (`answerLabel`)¶

A short singular noun phrase ("experimental protocol", "treatment cohort") that names one repeatable answer instance. It drives all repeatable-group chrome — group headings, Add/Delete buttons, delete-confirmation copy, status text — so wording can never disagree across surfaces.

Stored on the definition. The stored value is always used when present.
answerLabelGenerated persists authoring provenance: true means the label still follows the generated heuristic, false means authored, and null means legacy/unknown. Only exact true may be treated as generated; false and null fail closed as authored so a later question-text edit cannot overwrite wording whose origin is unknown. A null/blank normalised label always clears provenance, preventing an orphan boolean.
Authoring: field visible only for Multiple && !AnswerArray. Prefilled by the existing derivation heuristic (repeatableAnswerConcept) and it keeps tracking the question text until the author edits the label manually; thereafter their value wins.
Rendering: answerLabel?.trim() || repeatableAnswerConcept(question) — the heuristic remains as legacy fallback, so existing questions render unchanged and no migration is needed.
Consumed by v1 chrome today and by AF2 (including the outline panel).

Option display labels (`DisplayLabel`)¶

An option has one stable machine Value and optional reviewer-facing DisplayLabel. A missing label renders as the value, so existing documents and clients retain today's behaviour without migration. A present label may be edited without changing stored annotations, conditional-answer references, parent filters, or exports.

The compatibility slice adds nullable DisplayLabel to both schema-v0 OptionInfo and schema-v1 QuestionOption persistence/domain shapes, their DTOs, the OpenAPI contract, and regenerated NSwag client. Mongo omits null labels, but Mongo deserialization cannot recover whether an absent nested property was omitted or sent as null. The API command boundary therefore retains a three-state intent for every option before mapping to the domain: property absent means preserve, property present with null means clear, and a non-null string means set. The OpenAPI property is both optional and nullable. The server write DTO uses a presence-aware wrapper or JSON converter that records { isSpecified, value } while exposing no extra wire property: absent is {false, null}, explicit null is {true, null}, and a string is {true, value}. A bare nullable CLR property is insufficient. Generated NSwag clients plus any local write-wire type expose the equivalent optional-nullable shape and omit the JSON key only for the preserve state; their serializers must emit an explicit JSON null for clear rather than mapping absent and null to one CLR/TypeScript value.

For an update, the server loads the current persisted aggregate and, before reconstructing and replacing the whole document, merges every preserve intent from the current option matched by stable Value. A preserve intent for a new stable value resolves to null. Clear writes null and set writes the validated label. The aggregate version read for this merge is the version used by the optimistic-concurrency replacement, so a concurrent option edit fails rather than being overwritten; retry reloads and recomputes the merge. Blank or whitespace-only labels are invalid rather than a second spelling of null. Question copy, option update/reordering, and seed/fixture paths preserve the label.

V1 and AF2 render DisplayLabel ?? Value. Conditions, annotations, exports, and all equality/reference logic continue to use Value. Focused tests cover schema-v0 and schema-v1 BSON round trips and null omission, DTO/OpenAPI/NSwag round trips, generated-client and server-command preservation of absent versus explicit null, merge-on-omission for loaded options, new-option omission, concurrent-version rejection, blank rejection, copy/update/reorder/seed survival, both renderers' fallback, and stable-value condition/annotation/export behaviour. This upgraded server protects requests from older clients; it does not make an older server preserve an unknown nested field. The exact persistence build must therefore be observed healthy in staging and every older whole-document writer must drain before Python, .NET, or any client writes the first non-null DisplayLabel.

A persistent panel listing each repeatable (branching) question and its branches: answerLabel + branch ordinal, primary answer excerpt, and the non-colour status (complete / incomplete / N errors). Clicking navigates.

AF2-only. Navigation goes through AnnotationFormV2Store: open the target unit/page first, then focus — AF2's virtualisation means the target may not be mounted, and AF2 already requires explicit continuation into unmounted ranges.
Also the natural home for "why is this question missing?" explanations once suppression exists (derived state, ancestor named).
The v1 form keeps its in-place grouping (headings, chip, containment work); it does not get the panel. Decision 11's local disclosure controls and the panel share one reveal/focus seam, so outline navigation can open the target branch without duplicating local controls.

[2a] Repeatable branch disclosures (v1)¶

For two or more separate answer instances, the v1 branch header is an accessible disclosure. The single-instance form remains unchanged. Every branch starts expanded and can be collapsed independently. At three or more branches, symmetric Collapse all and Expand all actions appear on the right of the set footer while Add another … retains the left-hand action position. Three is the threshold because that is the first point where the bulk action saves more disclosure operations than it adds controls; two branches retain the simpler comparison surface.

The collapsed header shows the answerLabel plus ordinal, the top-level answer using resolved display copy (never an unresolved lookup ID), and direct repeatable-descendant counts only where the current form can prove them. Long summaries ellipsize visually while their full text remains in the disclosure's accessible name. Complete, Incomplete, and Needs attention each use a distinct Material icon/shape plus visible text and an accessible name. Error state includes a plain-language count/reason; colour is additional rather than the only signal.

Collapse applies native hidden/inert semantics to the existing branch body; it never conditionally removes the Angular subtree. Form-control instances, values, dirty/touched/validation state, nested disclosure state, branch identity, and order therefore survive. Adding a branch expands only the new identity. Deleting one forgets only that identity, preserves surviving state, and restores focus logically; the one remaining branch returns to the original always-expanded presentation.

Validation navigation uses a generic ancestor reveal callback. Preparing a nested target expands every containing branch before the existing after-render focus handoff. The save path's existing iteration across invalid answers consequently expands every error-bearing branch, and the first invalid control receives focus after the error dialog. Native buttons provide keyboard semantics and truthful aria-expanded/aria-controls; delete remains a separate button. Focus visibility, reduced motion, forced-colour boundaries, mobile wrapping, and deep nesting are acceptance requirements.

Branch layout has an explicit indentation budget. At desktop widths the legacy question-number gutter is capped at 4rem and the decorative branch header inset at 1rem; nested rings, numbering, labels, and disclosure icons carry the remaining hierarchy. Below 600px, cumulative header/comment spacers are removed and each question follows one full-width flow: number, label/control/help, then comments. Header state/actions and footer bulk actions reflow within the same branch boundary. This breakpoint keeps 200%-zoom and phone layouts usable without clipping controls, focus indicators, validation, or overlays.

This reader-only UI is deliberately not feature-flagged: it adds no wire or persisted shape, defaults to the already-expanded presentation, leaves the single-branch form unchanged, and is fully reversible without a data rollback. The isolated preview and explicit visual confirmation remain its rollout gate.

[3] Response modes (`responseModes`)¶

Project-defined alternatives to the ordinary answer ("Not reported", "Not applicable", "Unable to determine" — labels the project's own). Platform mechanics only: optionally suppress descendants, optionally require a reason, optionally attach mode-scoped metadata fields.

Reviewer UI (AF2): ordinary control stays central; a quiet "I cannot provide this answer" disclosure reveals the mode choice only where configured. Selecting a mode disables/replaces the ordinary control. Suppressed descendants leave the active form and never raise required-field errors; audit views explain the suppression.
Response: responseModeId mutually exclusive with value.
If a suppressing mode is chosen after descendants hold answers, those answers are preserved inactive (not cleared) pending explicit reconciliation — destructive clearing needs its own decision.

[4] Metadata fields (`metadataFields`)¶

Typed, admin-defined qualifiers stored alongside a response: unit, evidence location, reviewer confidence, note. display: inline for inseparable qualifiers (units, beside the control); display: details for documentary context (under a disclosure).

Test for what belongs here: if the information independently affects the review's substantive logic it is an ordinary question; if it qualifies or documents an existing response it may be metadata.

Designer (authoring) surface¶

The live question editor (annotation-question-designer) gains, with progressive disclosure:

the answer-label field (repeatable questions only, prefill-tracking);
later, a collapsed Response handling section: define modes (label, stable ID, suppression, reason), define metadata fields (key, label, type, options, required, display), duplicate-key validation, and a reviewer-facing preview.

QM v2, when revived, absorbs the same configuration into its properties panel as AQVersion content; changing a stable key or deleting a used mode is a breaking version change there.

Bulk question import contract¶

Bulk import complements individual editing; it does not replace either the live question designer or future QM v2. Users download the supported workbook, fill its Questions table, and upload that workbook or a UTF-8 CSV containing the same columns. SyRF validates and previews the normalized plan before an explicit confirmation can populate the selected project's question set.

Planning-only authority boundary. Merging PR #2779 approves only this future delivery contract. It does not authorize implementation; merging PR #2781; parser, uploader, API, or UI code; feature-flag changes; enabling Python or API commands; activation; deployment; staging use; or production use. Every later delivery slice retains its own review, merge, deployment, writer-drain, activation, and production-approval gates.

Shared CSV/workbook schema¶

The contract is versioned and canonical. The workbook contains Instructions, Reference, and Questions sheets; only the Questions table is imported, and exporting that table as CSV produces the accepted CSV shape. Column names, reference syntax, supported values, examples, and the limits approved in step 2c appear both here and in the downloadable template. This record deliberately does not select those numeric limits.

Required columns:

format_version, row_id, order, question, category;
question_type, control_type, optional, multiple, answer_array.

Optional columns:

description, parent_ref, condition_json, options_json;
answer_label, default_checkbox_status;
metadata_fields_json, response_modes_json.

The CSV dialect is fixed so the Python reference, .NET runtime, spreadsheet exports, and workbook reader cannot infer different tables:

bytes are strict UTF-8; one leading UTF-8 BOM is accepted and discarded;
the delimiter is comma (U+002C), the quote character is ", and a literal quote inside a quoted field is escaped as ""; an opening quote is valid only as the first character of a field and closes only under that rule;
fields containing a comma, quote, CR, or LF must be quoted; other fields may be quoted. Spaces outside quotes are data, not ignorable padding;
CRLF and LF terminate records. CRLF, LF, and CR are accepted inside quoted fields and normalize to LF in the typed plan; a bare CR outside quotes, unclosed quote, characters after a closing quote before delimiter/end of record, NUL byte, or blank logical record fails with a stable INVALID_CSV_DIALECT error;
embedded newlines are therefore supported in quoted question/description and structured cells. Every logical record has exactly the header's field count. One final record terminator is optional, and no locale-specific delimiter, encoding, or quote convention is auto-detected.

The header uses the exact contract column names, each at most once. Shared fixtures include an optional BOM, escaped quotes, embedded commas, all accepted record terminators, embedded cell newlines, and each fail-closed dialect case.

format_version must be 1, the sole supported contract version, on every row. Mixed versions fail with stable MIXED_FORMAT_VERSION errors and an unknown version fails with UNSUPPORTED_FORMAT_VERSION; neither is guessed or upgraded implicitly. CSV scalar cells use exact ASCII contract lexemes: optional, multiple, and answer_array are lowercase true or false only; order is 0 or a non-zero ASCII digit followed by zero or more ASCII digits, with no sign, leading zero, decimal point, exponent, whitespace, or locale separator; and format_version is the single character 1. In a workbook, those boolean columns accept either a native Boolean cell or text containing the same exact lowercase lexeme. order accepts either a native numeric cell that is a finite, exact, non-negative integer within the approved bound or text matching the CSV grammar; format_version accepts native numeric integer 1 or text 1. Dates, errors, blanks, formula cells, numeric booleans, floating-point order values, and all other implicit spreadsheet coercions fail closed. Both parsers convert these accepted representations to the same JSON Boolean/integer typed plan values, and shared fixtures pair every accepted CSV lexeme with its native and text workbook equivalents plus every rejected coercion. row_id is an opaque file-local machine key matching [A-Za-z][A-Za-z0-9_-]{0,63}; the contract and template prohibit participant or clinical data in it. order is a required, bounded integer unique among imported siblings under the same resolved parent; physical row order has no meaning. Imported siblings sort by order and append after that parent's existing children, whose relative order never changes. The canonical plan records the resulting final append order. A blank parent_ref means the documented root/category anchor. @row:<row_id> references another imported custom question and may point forward. @question:<question UUID> references an existing custom parent by its stable ID; the UUID is canonical lowercase ASCII, resolves only within the selected project, and must identify exactly one placement-valid custom parent. Unknown, foreign-project, ambiguous, and placement-invalid references fail closed before any write, without exposing foreign-project details. @system:<alias> references an allowlisted built-in anchor. Format 1 publishes exactly experiment_label, cohort_label, outcome_assessment_label, disease_model_induction_control, and treatment_control. Disease Model Induction and Treatment label questions are non-anchor system questions and therefore have no parent_ref aliases. The template's Reference sheet publishes these aliases and their current meaning; raw GUIDs outside the namespaced @question: form and display-label matching are not an import contract. Unknown aliases fail with stable UNKNOWN_SYSTEM_ALIAS and identify the unknown alias in the ephemeral validation response. Adding a new alias for an existing, unambiguous built-in is a backwards-compatible registry extension and does not by itself force a format_version bump: older servers fail explicitly rather than guessing. Removing an alias, changing its meaning, or changing reference grammar is breaking and requires a new format version. The allowlist, template Reference sheet, Python reference, .NET resolver, and shared fixtures change together in one reviewed contract PR.

options_json is an array of stable Value/nullable DisplayLabel pairs. condition_json expresses either boolean equality or option-key anyOf and references stable option values, never display labels. A nonblank answer_label is authored, so commit persists AnswerLabelGenerated=false; blank omits both label and provenance. Metadata and response-mode JSON use the definition shapes in this record, including reserved-key and export-stem validation. Before step 10b, nonempty metadata_fields_json or response_modes_json fails with a stable CAPABILITY_NOT_AVAILABLE error: parsing future columns is not permission to write them.

Workbook files containing macros, formulas in imported cells, external links, or unsupported sheets fail closed. CSV and workbook representations of the same table must yield byte-equivalent canonical plans.

Both parsers produce the same typed plan before hashing; neither hashes a parser-specific dictionary or raw worksheet representation. All text is normalized to Unicode NFC and all embedded CRLF/CR cell line breaks normalize to LF. Booleans are JSON booleans, order is a JSON integer inside the contract's approved bounded range. Numeric option stable values are JSON strings, never JSON number tokens. For an integer question the accepted grammar is ^-?[0-9]+$; for a decimal question it is ^-?[0-9]+(?:\.[0-9]+)?$. A leading plus, omitted integer part, trailing decimal point, whitespace, locale separator, NaN/infinity, and exponent notation such as 1e0 or 1E+3 fail closed with INVALID_NUMERIC_OPTION_VALUE; no runtime is allowed to accept a wider numeric grammar.

Both implementations normalize the accepted digit string with the same exact base-10 algorithm, without binary floating point or runtime decimal formatting: remove the sign; split once at the dot; remove leading zeroes from the integer part, retaining 0 if it becomes empty; remove trailing zeroes from the fractional part; omit the dot when the fractional part becomes empty; and restore - only when at least one remaining digit is nonzero. Thus decimal 1, 1.0, and 1.00 all become "1"; -0 and -0.000 become "0"; and 001.2300 becomes "1.23". The approved step-2c limits must bound the raw numeric token's digits and the normalized value's precision and scale; both parsers count those quantities identically and reject an excess with NUMERIC_LIMIT_EXCEEDED before constructing a runtime numeric value. The same grammar and normalization apply to numeric option-key operands in condition_json. These conversions happen before duplicate-value and reference validation, so semantically equal numeric keys cannot survive under different spellings.

Canonical arrays use their defined semantic order rather than source-container iteration order. Questions use Kahn topological ordering over imported-parent edges, so a referenced imported parent is always emitted before its child. At each ready-queue step, the total tie-break is the tuple (category ordinal, canonical parent reference, order, row_id): category ordinals in format 1 are Study=0, Disease Model Induction=1, Treatment=2, Outcome Assessment=3, Cohort=4, and Experiment=5; the reference and row_id components compare their ASCII bytes; and order compares as the validated integer. Blank parents and every @row:, @question:, and @system: reference first normalize to their canonical contract spelling. This defines one order across independent trees as well as within siblings; golden fixtures reverse physical input rows and include multiple simultaneously ready roots to pin it. Options retain their declared display order, and set-valued arrays are deduplicated and sorted by their contract key. The typed plan is constrained to the I-JSON data model accepted by RFC 8785: no duplicate object property names, non-finite numbers, lone Unicode surrogates, or implementation-specific numeric types enter the canonicalizer. The resulting plan is serialized by the full RFC 8785 JSON Canonicalization Scheme, including its recursive UTF-16-code-unit property sorting, JSON escaping, and ECMAScript number serialization rules, as UTF-8 bytes with no BOM or insignificant whitespace. No separate .NET ordinal, Python insertion-order, or serializer-default rule supplements JCS. Its canonicalPlanHash is SHA-256 encoded as lowercase hexadecimal. Shared golden fixtures pin the exact bytes and hash, not only structurally equivalent JSON. Preview and commit call the same canonicalizer; Python, .NET, CSV, and workbook parity tests must all match those golden bytes and hashes.

Stable option values are unique within a question. Validation rejects blanks and duplicates after conversion to the declared question_type. A string value is first stored in Unicode NFC; its duplicate-comparison key then applies Unicode 17.0.0 Default Case Folding using the C and F mappings from that version's CaseFolding.txt (never locale-sensitive or Turkic T mappings), normalizes the result to NFC again, and compares Unicode scalar values ordinally. Integer and decimal values use the numeric equivalence defined above. Shared cross-runtime fixtures pin non-ASCII edge cases as well as ASCII case pairs, so .NET and Python cannot inherit different runtime or locale behaviour. A duplicate display label is allowed; references bind the stable value, not the label. This validation is fail-closed in preview, before reference resolution or any project write.

The import contract accepts only combinations the current designer and reviewer runtime can represent. The fields obey this matrix:

`control_type`	Allowed `question_type`	`optional`	`multiple`	`answer_array`	`default_checkbox_status`	Options
`textbox`	`string`, `integer`, `decimal`	either boolean	`false`, or `true`	`false` when not multiple; either value when multiple	blank	absent
`dropdown`	`string`, `integer`, `decimal`	either boolean	`false`, or `true`	`false` when not multiple; either value when multiple	blank	one or more unique stable values
`autocomplete`	`string`	either boolean	`false`, or `true`	`false` when not multiple; either value when multiple	blank	one or more unique stable values
`radio`	`string`, `integer`, `decimal`	either boolean	`false`	`false`	blank	one or more unique stable values
`checklist`	`string`, `integer`, `decimal`	either boolean	`true`	`true`	blank	one or more unique stable values
`checkbox`	`boolean`	`false`	`false`	`false`	`unchecked`, `checked`, or `indeterminate`; blank canonicalizes to the designer default, `unchecked`	absent

answer_array=true means one annotation contains the repeated values; multiple=true, answer_array=false means separate annotation instances, each able to own its own child branch. Any combination outside the matrix fails preview with a stable compatibility error; the importer does not silently coerce it (apart from the documented blank checkbox default).

Preview, commit, and audit boundary¶

The .NET API owns authoritative parsing and business validation:

The multipart request envelope carries the file, a caller-generated UUID operationId, and an explicit assignmentMode of unassigned or selectedStages. selectedStages requires distinct, nonempty selectedStageIds; unassigned requires none. There is no implicit assignment default; invalid combinations fail with stable INVALID_ASSIGNMENT_SCOPE or DUPLICATE_STAGE_ID errors. Assignment scope participates in the canonical-plan and assignment-concurrency hashes and in audit.
preview returns a normalized parent-before-child plan, resolved references, final append order, operation counts, the exact assignment delta, stable row/column error locations, a file hash, a canonical-plan hash, and project/assignment concurrency hashes. It writes no project, assignment, or immutable success-audit data; it may update the short-retention operational job described below.
commit receives the same file and preview hashes. After authorization and request-envelope validation, it hashes the supplied file and looks up the immutable receipt by operationId before current-project or assignment drift validation. An existing receipt returns its recorded result when every retry-identity field and supplied hash matches, and otherwise returns 409; it is not rejected merely because project state has advanced since the original successful import. Only an absent receipt proceeds to reparse, revalidate, and compare current state, returning 409 if project state, stage assignments, file bytes, normalized plan, or assignment scope drifted. Tests cover the matching sequential retry after unrelated later project changes, conflicting operation-ID reuse, existing-sibling order changes, and concurrent assignment changes. A successful preview is never a reusable write token for a new operation after state changes.
Commit applies the entire plan to one in-memory Project aggregate and persists once with optimistic concurrency. It must not loop over today's per-question HTTP endpoint. Assignment is additive and union-only: each selected stage's final assignment is the union of its existing assignments, the imported questions selected for that stage, and the complete missing ancestor closure of every imported question through its resolved custom and system parents. Existing unrelated selected and unselected relationships and sibling order remain unchanged. If an ancestor cannot be assigned under the domain/eligibility rules for any selected stage, the whole preview fails with stable ANCESTOR_ASSIGNMENT_NOT_ALLOWED, identifying the selected stage and safe canonical ancestor reference; it never creates an orphaned stage assignment. The plan reports every imported and ancestor edge in the exact added delta, and removed is always empty. Tests include an imported child below an existing unselected custom parent and below each supported system anchor, the stable failure when either ancestor cannot be included, and proof that concurrent relationship changes fail closed rather than overwrite either state.

Both endpoints require ProjectDesignPolicy; selectedStages additionally requires an explicit StageDesignPolicy authorization check for every selectedStageId. Authorization is ordered before upload processing: the route project is authorized before the upload body is parsed or consumed, and the assignment envelope is authorized for every distinct selected stage before file bytes are parsed, hashed, copied to storage, or recorded as an authorized project attempt wherever the platform transport permits that short circuit. The endpoint contract may use a metadata preflight or separately readable envelope to preserve this order. Any denial returns 403, consumes no file content beyond unavoidable HTTP-server transport buffering, creates no project history entry, and emits only structured security telemetry. Validation then reuses the domain's placement, hierarchy, conditional-parent, and option rules. Preview fails before any write on mixed or unknown schema versions, unknown columns, duplicate or ambiguous IDs, unresolved system/custom references, cycles, invalid category placement, duplicate sibling order, invalid conditional-answer references, malformed structured cells, an ancestor that cannot be included in every selected-stage assignment (ANCESTOR_ASSIGNMENT_NOT_ALLOWED), unsupported capabilities, and breaches of the limits approved in step 2c. Errors carry a stable code plus file/sheet, row, column, and row_id where available.

From step 10b, any plan that adds a response-mode/metadata definition, or assigns an existing extended-definition question, invokes one authoritative server-side actual AF2 eligibility decision for every affected existing or selected stage and every host required by that stage's workflow. A flag value or a client-side approximation is not eligibility. Any unsupported, unresolved, or ineligible stage/host returns a stable fail-closed error and the entire preview/commit writes nothing. The common individual/bulk assignment path rechecks the same decision at mutation time, including assignments of already-extended questions. Shared eligibility vectors are consumed by the .NET tests, importer fixtures, and Angular/UI tests so the preview explanation and server decision cannot drift.

Each successful commit inserts a separate append-only QuestionImportAudit aggregate in its own collection. It stores a unique operation ID, actor, timestamp, project, explicit assignment scope, contract version, file and canonical-plan hashes, the exact preview projectConcurrencyHash and assignmentConcurrencyHash, counts, before/after concurrency versions, exact createdQuestions mappings of a generated canonical rowOrdinal to questionId, and an assignmentDelta whose added edges are exact and whose removed set is empty. The ordinal is assigned after validation in canonical parent-before-child/final append order and contains no user-authored text; durable receipts never persist raw row_id values. The file hash, canonical-plan hash, project hash, assignment hash, assignment scope, and project ID all participate in the identical-retry comparison. The project replace and audit insert receive the same IClientSessionHandle and execute in that one Mongo transaction; there is no nontransactional fallback. A topology without transaction support fails closed before either write. An identical retry of an existing operation ID returns the recorded result without another write, while reuse with different file, plan, scope, project, or assignment hashes returns 409.

After validation, the unique operation-ID receipt insert occurs inside that same transaction before the project replacement and is the concurrent-first-use arbiter: exactly one transaction wins. A loser that gets the duplicate-key result aborts its transaction before reading, reloads the winner's immutable receipt, and returns the recorded result only when all retry-identity fields and hashes listed above match; otherwise it returns 409. Neither path retries the project mutation: the winner applies it once and the loser never applies it. Step 2d1's real replica-set integration tests prove shared-session commit and rollback, unsupported-topology failure, insert-only uniqueness, sequential idempotent retry and conflicting reuse, plus concurrent identical and conflicting first-use races. Each race asserts one winner, one project mutation and one receipt; the identical loser returns that receipt, while the conflicting loser returns 409.

The audit does not retain workbook/CSV contents, cell values, raw row_id, or participant or clinical data. Its generated ordinals and server-generated question/assignment IDs are the durable change-set identifiers. The first production slice is additive by default; updating, replacing, deactivating, or deleting an existing question remains out of scope until a separate data-impact decision defines how existing annotations survive.

Authorized, user-initiated imports also create a separate short-retention operational attempt/job record, visible through a bounded per-project recent history. It records status, actor, project, timestamps, a sanitized basename and file hash only where the privacy policy permits them, structured error codes and counts, safe row ordinals/column names where useful, and retry/report linkage. It never stores spreadsheet contents, cell values, or raw row_id. The default retention is a configurable 30-day TTL; its lifecycle and reconciliation are independent of the immutable success receipt. Authorization-denied and other pre-authorization requests are excluded from project history and emit structured security telemetry instead.

A failed attempt is recorded only in that operational history and must never appear as a transactionally committed import. On success, the project mutation and immutable QuestionImportAudit receipt commit atomically; the operational job may then link to that receipt. Reconciliation handles interruption between an operational status update and an aborted or committed import transaction by consulting the immutable receipt and project version, never by inferring success from the job status alone.

Angular flow, fixture/operational parity, and Python retirement¶

The current question-design page mounts reusable Download template and Import questions actions plus a bounded recent-import history; QM v2 later mounts the same component/service against the same API. The flow provides a native labelled file input, keyboard-complete preview and confirmation, focus management, non-colour status, aria-live progress, and an error summary whose links focus row-level errors. Commit is unavailable until preview succeeds and the user explicitly confirms it.

The Python foundation remains an admin-only ad hoc bridge during the rollout. Fixture parity is the offline contract bar: shared golden fixtures cover valid trees plus unknown, ambiguous, cyclic, conditional, and invalid-reference cases, and Python and .NET must emit the same JCS canonical-plan bytes, hashes, and stable validation results. Fixture parity alone does not establish operational parity. That deployed bar requires the complete Angular download/upload/preview/confirm/recent-history flow, authoritative .NET validation, atomic apply/audit behavior, CSV/workbook equivalence, permissions, and error reporting to be exercised successfully in staging and then activated and verified in production under their separate approvals. Production services never invoke Python.

Step 2c specifies the atomic rollback contract but does not deliver an endpoint. Step 2c1 implements, deploys, and proves that endpoint. The rollback must be one atomic server operation that conditions an exact, non-recursive, non-cascading delete on the expected relationship, expected project version, and expected absence of children, or enforces the equivalent predicates in one project-aggregate compare-and-swap. A failed predicate deletes nothing, including the parent. Today's recursive delete is not a safe rollback primitive: a child can attach after the bridge's safety read and before deletion, causing the rollback to erase an unjournalled descendant. Repeating the client-side read narrows but cannot close that race, so the existing child-first and outsider checks remain defence in depth rather than an activation control. Contract tests cover stale project version, wrong relationship, and a concurrently attached child, and prove that nothing is deleted in every case.

The temporary bridge may activate before the .NET preview/apply and Angular UI steps 2d–2f, but only after the importer foundation, 2c, 2c1, 2c2, and 2c3 are complete; step 2b has landed; personal evidence shows the exact 2c1 endpoint build live and healthy in staging; the exact step-2 answer-label persistence build and exact 2c2 option-label persistence build are healthy there with every older whole-document writer drained; and an explicit activation approval is recorded. The additive 2c1 endpoint itself needs no old-writer drain, but that does not relax either whole-document field drain. Until every condition holds, all Python API commands remain hard-disabled.

Step 2f rolls out the .NET/Angular path; it does not remove the bridge in the same PR. Only after 2f is merged and deployed, staging operational-parity evidence is accepted, the production path receives explicit activation approval, and production activation/health evidence is recorded does a focused post-activation step 2g retire the Python live-import path by removing or permanently disabling its network-write commands and live instructions. The parser, shared fixtures, and golden behavioural tests may remain in-repo as reference evidence, avoiding two permanent production implementations without creating a replacement gap.

Delivery map¶

Focused PRs, each citing this document. Order matters only where stated.

#	Delivers	Vehicle / notes
1	This PR: design context + architecture	docs only
2	`answerLabel` + `answerLabelGenerated` persistence round-trip: domain + DTO + NSwag + render fallback	schema-v0; additive fields follow ADR-011's whole-document writer floor; deploy and verify before step 2b
2b	`answerLabel` editor prefill + import template + seed fixtures — the first thing that writes the field	Must not ship until every project writer from step 2 is deployed and old instances have drained. Project writes replace the whole document, so one un-upgraded writer erases a saved label; ADR-011 drains writers before enabling writes for exactly this reason. Rolling or independently-ordered API/PM/web deploys make "same PR" insufficient.
2c0	Code-disabled admin/reference importer foundation in `tools/question-template-importer/` (PR #2781)	Foundation only: every API command remains hard-disabled. Its fixture schema is reference behaviour, not yet the production shared-contract version.
2c	Versioned CSV/workbook schema, template definition, golden fixtures, and architecture/ADR detail	Requires 2c0. Reconciles and versions the foundation's differing fixture schema before Python/.NET fixture-parity claims. Docs/fixtures only, no production writer. Defines aliases, columns, the exact JCS canonicalization/golden-byte contract, error codes, the existing-question data-impact boundary, and the atomic server rollback contract that 2c1 must implement, deploy, and prove before the Python bridge's API commands may be enabled. It specifies rollback; it does not implement or deploy an endpoint or activate the bridge. Numeric resource limits and bulk-import feature-flag posture remain explicit user decisions before this slice is implementation-ready.
2c1	Guarded rollback server contract implementation and deployment proof	Requires 2c. Implements one atomic exact, non-recursive, non-cascading delete conditioned on the expected relationship, expected project version, and expected absence of children, or an equivalent project-aggregate compare-and-swap. Tests for stale version, wrong relationship, and a concurrently attached child must prove nothing is deleted, including the parent. Personal evidence must show the exact build live and healthy in staging before bridge activation. This additive endpoint has no old-writer drain of its own. No bridge activation in this step.
2c2	Distinct option `DisplayLabel` persistence round-trip	Requires 2c1. Adds nullable `DisplayLabel` to schema-v0/schema-v1 option domain, persistence, DTO, OpenAPI, and NSwag shapes. BSON omits null, while the API command retains absent/preserve, explicit-null/clear, and string/set intent. Before whole-document replacement the server loads the current aggregate and merges omitted labels by stable `Value` under the same optimistic-concurrency version; generated clients and mappings must not collapse the tri-state. Copy/update/reorder/seed paths survive, v1/AF2 render `DisplayLabel ?? Value`, and conditions/annotations/exports remain keyed by `Value`. Tests cover both schemas, wire states, merge/concurrency behavior, and every boundary. The exact build must deploy healthy and all old whole-document writers drain before Python, .NET, or any client writes a non-null label.
2c3	Python bridge new-field compatibility and field-survival activation gate	Requires 2c0, 2c1, and 2c2. Round-trip/rollback tests cover `answerLabel`, nullable provenance, option display labels, and preservation of fields unknown to the bridge. Activation also requires step 2b landed; personal evidence that the exact 2c1 build is live and healthy; both the exact step-2 and 2c2 persistence builds healthy in staging with all old whole-document writers drained; and explicit activation approval. Otherwise every API command remains disabled.
2d	.NET parser, template-download endpoint, authoritative validation, and read-only preview API	Requires 2c. CSV/workbook fixture-parity tests consume the reconciled, versioned shared fixtures produced there, not 2c0's original differing schema. May deploy read-only; no UI activation or project write.
2d1	Transactional `QuestionImportAudit` persistence proof	Requires 2d. Real Mongo replica-set integration tests prove the project replace and insert-only audit receive the same `IClientSessionHandle`, commit or roll back together, fail closed on a non-transaction topology, and implement sequential plus concurrent identical/conflicting operation-ID semantics without a fallback. A unique insert elects one concurrent winner; a loser aborts, reloads, compares every retry-identity hash, and never retries the project mutation. No production writer activation.
2d2	Short-retention operational import-attempt history	Requires 2d. Stores authorized attempts separately from immutable success receipts, with configurable 30-day TTL and a bounded per-project list; denied/pre-auth requests go only to security telemetry. Tests cover privacy exclusions, TTL expiry, list bounds, retry/report linkage, and interruption between an attempt update and an aborted or committed import transaction. No production writer activation.
2e	Authorized atomic apply API/service plus durable audit	Requires 2d1 and 2d2, step 2b landed, and personal staging evidence that the exact step-2 and 2c2 persistence builds are healthy after every older whole-document writer drains. Applies once to the aggregate with concurrency protection; never sequences public per-question PUTs. A server-enforced activation gate, whose exact flag/topology is decided in 2c, remains closed in every environment when this endpoint first deploys; direct HTTP clients cannot bypass it. Staging opens only for the approved 2f operational-parity walkthrough, and production opens only after the separate 2f production-activation approval. The separately gated Python bridge may already be active after 2c3; 2e does not weaken or replace that gate.
2f	Accessible Angular download/upload/preview/confirm/recent-history flow in the live Design surface; operational-parity rollout	Requires 2e deployed and healthy. Remains inactive until CSV and workbook E2E proof, normal review, and explicit merge/activation approval. Records accepted staging operational-parity evidence before a separately approved production activation and verifies production health after activation. Does not retire the bridge in this rollout PR. QM v2 later mounts this same UI/API contract.
2g	Post-activation Python live-writer retirement	Requires 2f merged/deployed, accepted staging operational-parity evidence, explicit production activation approval, and recorded production activation/health evidence. In a separate focused cleanup PR, removes or permanently disables Python network-write commands and live instructions while retaining useful parser/shared-fixture reference tests.
3	Branch chip in the number gutter (v1)	already queued
4	Branch `role="group"` subtree containment (v1)	already queued; template refactor
4a	Independent repeatable-branch disclosures, summaries/status, bulk collapse/expand, and validation reveal (v1; PR #2790)	Requires 3–4. Reader-only and UI-state-only; no writer/deployment-drain dependency. Keeps the subtree mounted, keys state by annotation identity, and exposes a generic ancestor reveal/focus seam for step 5. The isolated preview must demonstrate multiple and nested branches, add/delete, all three non-colour statuses, error expansion/focus, responsive behavior, and the available light/dark theme matrix before guarded merge.
5	AF2 outline/navigation panel	AF2; consumes `answerLabel`; store-integrated navigation
6	ADR + domain/persistence for response modes + metadata (writers first)	schema-v0 additive; flagged; ADR records the rollback floor
6a	Stop tree-shaking suppressed descendants in `ExtractionInfo.AddAnnotations`, with a suppress-after-answer regression test	blocks 7 — without it the reviewer UI silently deletes the answers step 7 claims to preserve
7	AF2 reviewer UI for response modes (chooser, suppression, validation)	flag-gated; requires 6a
8	AF2 metadata rendering (inline + details)	flag-gated
9	Designer "Response handling" section	after 6; gated on actual AF2 eligibility for the stage, not on the environment flag alone
10	Export/API columns for modes + metadata (`q__value`, `q__response_mode`, `q__<key>`) with definition manifest	after 6; must resolve suppression so preserved-inactive answers are not emitted as live values, with a suppress-after-answer export test
10b	Allow bulk import to write `responseModes` and `metadataFields` definitions	Requires steps 6–10 deployed and every writer, reconciliation, assignment, and export gate satisfied. When a plan adds extended definitions or assigns an existing extended question, preview and commit invoke one authoritative server-side actual-AF2-eligibility decision for every affected existing/selected stage and required host; stable errors fail the whole plan closed with no write. Definition import alone does not lock a stage: preview presents the affected stages/hosts and a non-dismissible acknowledgement. The first extended response locks that stage one-way, and the common individual/bulk assignment path rechecks the same decision. Shared .NET/importer/UI vectors prevent drift. Until then those nonempty import columns fail closed.

Activation order and validation rules¶

The delivery map says ordering matters only where stated. Review established that several orderings are load-bearing, because the intermediate states lose or corrupt data rather than merely degrading. These bind the implementing PRs.

Nothing may write a mode or metadata response until the export path understands them (step 10). WideDataFormatRowWriter.AddAnswerQuadFromGroup calls GetAnswer() and notes only, so in the window between step 7 and step 10 a mode response exports as an ordinary blank and its metadata vanishes entirely — an export that is silently wrong, which is worse for a systematic review than one that fails. The feature flag may not be enabled in an environment until step 10 is deployed there.

The mode chooser may not ship before the metadata renderer. Selecting a mode with requiresReason immediately requires the injected _reason field (Decision 5). Step 7 without step 8 offers reviewers a choice they cannot complete, or submits an invalid response.

Reconciliation must be able to express what annotation can express. The reconciler's own editable form is still v1, which by Decision 2 gets no response-mode controls, and a reconciler must create a new authoritative annotation rather than pick a candidate. Enabling modes on a stage that requires reconciliation before the AF2 reconcile host is editable leaves the reconciler unable to submit at all. Gate on the reconcile host, not just the stage-review host.

Once a mode or metadata response exists, the stage may not fall back to v1. A metadata-only response — no mode selected, but qualifiers recorded — carries exactly the same exposure, because v1 cannot rehydrate metadata either. Disabling the flag, or any eligibility change that makes AF2 fail closed, routes the stage to a renderer that cannot rehydrate responseModeId or metadata — and the next v1 submission reaches the replacement-based ExtractionInfo.AddAnnotations and drops them. Step 6a protects suppressed descendants; this is the same hazard for the mode-bearing annotation itself. The flag is therefore one-way per stage once responses exist, and eligibility loss must fail the stage closed rather than silently downgrade it.

Authoring gates on every stage the question is assigned to. A question can be assigned to more than one stage — ExtractionInfoChildrenMergeTests covers exactly that — and the definition is project-level. Checking only the stage the administrator happens to be editing from lets modes be configured from an AF2-eligible stage and silently break reviewers on a v1 or extraction stage that shares the question. The check is not only at authoring time: assigning an already-mode-configured question to a new stage must revalidate, or the same breakage arrives through the assignment path instead of the editor.

Metadata keys are unique across scopes. AnnotationResponse has one flat metadata dictionary and exports one q__<key> column per key, so a question-level and a mode-scoped field sharing a key cannot be validated, persisted, or manifested unambiguously. Reject the collision at definition-save.

answerLabel records whether it was generated or authored. The prefill tracks the derived heuristic until someone edits it, after which a question-text change must not overwrite their wording. The stored string alone cannot distinguish the two, so provenance has to be persisted alongside it — and being itself an additive field, it falls under the writer-compatibility floor, so it ships in step 2's round-trip rather than being added later once labels exist and their provenance is already unrecoverable.

_reason is mode-scoped, not question-scoped. More than one mode on a question may set requiresReason, and each needs its own required instance. One reserved key with a per-mode requirement, not one shared field that a second mode silently inherits or overwrites. Because the response carries a single flat metadata dictionary, changing the selected mode clears _reason: otherwise switching from mode A to mode B leaves A's explanation in place and B's validation sees its required key already populated, accepting a reason written about a different answer.

A submitted responseModeId must exist in the definition the response is validated against. The contract as stated requires only that it be mutually exclusive with value, so a stale client or a direct API call could submit an unknown ID — which has no label for the export manifest, no reason or metadata requirements, and no suppression rules, producing a response nothing can interpret and which bypasses every configured check. Reject unknown IDs at write time rather than discovering them at export.

Future direction (deliberately unspecified): reference responses and customisable relationships¶

Recorded 2026-08-11 so nearer-term decisions do not foreclose it. No user requirements exist yet; this section is a thinking aid for when a specification arrives, not a commitment.

Today's mechanism is hard-coded. Cross-annotation relationships — how a Cohort is associated with a Disease Model, Treatment, and Outcome Assessment — are implemented as lookup system questions: an annotationLookup flag on the question entity, a fixed GUID enum (systemAnnotationQuestionGuids.treatmentControl / modelControl / cohortDiseaseModels …, annotation-question.entity.ts:293), and special-cased logic keyed on those GUIDs (annotation-form.service.ts, annotation-question.component.ts:370). AF2 carries the same annotationLookup flag through its models. The relationship specification is code, not configuration.

The idea. Generalise this into a configurable reference response: an answer whose value is a reference to answer instances of another question or category — a custom lookup. A project could then define its own relationship edges (which entity-like repeatable questions exist, and which questions reference them, with what cardinality), so the currently hard-coded Experiment → Cohort → {Disease Model, Treatment, Outcome Assessment} graph becomes one instance of a project-definable specification rather than the only possible shape.

Placement note (the one analytical claim worth recording now). Although this arose from response-mode thinking, a reference is not a response mode: modes are alternatives to answering ("cannot answer"), whereas a reference is an ordinary answer whose value happens to be a reference. The two should stay orthogonal — a reference question can still offer "Not applicable". The natural extension point is the answer-type system (a reference answer kind with definition-side configuration: target question/category, scope, cardinality), i.e. a fifth optional extension on the same spine, not a widening of responseModes. Widening modes into "any non-plain response" would repeat the second-form-engine mistake this document forbids for metadata.

What an eventual specification must answer (so requirements-gathering asks the right questions):

Reference scope: same study, same session, same stage?
Cardinality and ordering; are dangling references possible when the target repeatable instance is deleted, and what does deletion of a referenced branch mean (block, cascade, or mark-dangling)?
Versioning: how references survive AQVersion/SessionVersion transitions.
Interaction with suppression (referenced target suppressed by an ancestor).
Export and meta-analysis semantics — OutcomeData and PRISMA derivations are consumers of today's hard-coded graph, so any generalisation must keep their inputs derivable.
Reconciliation: comparing reference answers across annotators.
AF2's mutable-unit machinery (unit categories, one label question per category) is where relationship structure lives at render time; a configurable graph must map onto or replace it deliberately.

Preserved now, at zero cost: AnnotationResponse.value remains open to additive value shapes, and nothing in the delivery map assumes the system graph is fixed forever.

Open questions¶

Blocking step 2c — user decisions required before implementation¶

Numeric resource limits. The contract requires bounded input and a bounded order integer, but this record does not choose maximum file bytes, rows/questions, hierarchy depth, options, structured-cell bytes, raw numeric token digits, normalized numeric precision/scale, or the integer range. Step 2c must record the approved values and expose the same limits in the template, preview errors, and tests.
Bulk-import feature-flag posture. This record preserves every deployment, writer-drain, authorization, review, and explicit activation gate, but does not decide whether the read-only preview, apply API, recent-history UI, and Python bridge also sit behind generated environment flags, nor whether those flags are separate. Step 2c must record the user-approved posture before implementation; no row in the delivery map silently answers it.

Blocking step 6 — must be decided before domain/persistence¶

Definition-edit policy: freeze on first use, or snapshot per response?

Decision 7 stamps every response with the definitionVersion it was written against, which records that semantics may have changed. It does not decide what the platform then does about it, and that half is open:

Freeze on first use — once a key or mode has been answered, its key, type, options and suppression behaviour become immutable; only cosmetic properties (label, help text, display, order) stay editable. Cheap, and aligned with the existing "stable machine keys, project-defined labels" constraint, which already assumes label edits are safe because keys carry meaning. But it is a real restriction on administrators — "you cannot change this field's type now that anyone has answered it" — and that is a product call, not an engineering one.
Snapshot per response — store the definition alongside each response. No restriction on administrators, but it duplicates a definition onto every annotation, which is heavy at realistic form sizes and is precisely the job QM v2's immutable AQVersion exists to do properly.

Held pending a user specification. The definitionVersion stamp is what keeps both routes open — and a later QM v2 revival can reconstruct from it — so nothing in step 6 is blocked on choosing today, provided the stamp ships with the response shape.

Carried (not blocking the map)¶

Initial metadata type set (date? specialised unit fields?).
Schema reuse across category/stage vs per-question only.
Rename/removal semantics for keys and modes already used in responses (narrower than Q3, which is about already-recorded responses).
Localisation of labels vs stable keys in exports.
Which parts of suppression explanation belong in reconciliation views.

References¶

Design context (recovered)
Original QM v2 plan
ADR-011 — schema-v0 additive precedent
GOV.UK-family repeatable-entry patterns (add another / add to a list): MOJ add another, MOJ add to a list, DWP add another thing
Covidence data-extraction practical guide (no blank fields; record Not applicable / Not reported / Unclear)