Skip to content

Advanced Screening & Filtering

Primary audience: Alex (PO), Chris (Senior Dev), Nuri (Junior Dev). Secondary: Gill (Scrum/Helpdesk).

This specification reflects the current state of SyRF screening/selection, latest Clarifying Questions decisions, and intent from prior architectural drafts. It separates the Stage Study Pool (what is eligible) from Selection (who sees what next), introduces Screening Profiles, and sets a foundation for Phase-2 features (PRISMA, annotation-based filtering, tie-breaker groups).

Executive Summary

SyRF's legacy screening model couples screening to a single, project-wide state, limiting multi-stage pipelines (e.g., TA → FT) and complicating pass-forward. Teams use multi-project workarounds, losing auditability and adding friction.

This feature introduces Screening Profiles (formerly "definitions") and stage-scoped Filter Sets to enable flexible, multi-stage screening workflows within a single project.

Problem Statement

Current limitations:

  • Screening is coupled to a single project-wide state
  • Multi-stage pipelines (Title/Abstract → Full-Text) require workarounds
  • Pass-forward between stages is cumbersome
  • Teams lose auditability when using multi-project workarounds

Proposed Solution

Introduce Screening Profiles and stage-scoped Filter Sets to:

  • Configure TA, FT, or other bespoke stages within a single project
  • Define the Stage Study Pool via Filter Sets that can reference outcomes from one or more profiles
  • Keep Selection reviewer-aware and policy-driven (per-reviewer suppression, HideExcludedStudiesFromReviewers, MaxInProgress), independent of pool definition
  • Preserve auditability and pave the way for annotation-based filtering and PRISMA 2020 in Phase-2

User Stories

Project Admin Stories

  • As a Project Admin, I can create a Screening Profile with criteria text so reviewers know what to screen for
  • As a Project Admin, I can clone an existing profile to create a new version without affecting existing outcomes
  • As a Project Admin, I can define a Filter Set for a stage to control which studies are eligible
  • As a Project Admin, I can configure stage settings (mode, policies) to control reviewer experience
  • As a Project Admin, I can run a Migration Wizard to convert legacy projects to the new model

Reviewer Stories

  • As a Reviewer, I can see the active screening criteria while reviewing
  • As a Reviewer, I can get the next study based on my role and stage settings
  • As a Reviewer, I can resume saved sessions when returning to review

Reconciler Stories

  • As a Reconciler, I can access reconciliation mode to resolve annotation conflicts
  • As a Reconciler, I can see studies that have sufficient candidate sessions for reconciliation

Success Criteria

  • 80% fewer new projects requiring multi-project workaround for TA→FT
  • ≤ 5 minutes to configure a two-stage pipeline (profile + filter set + stage settings)
  • Reviewer throughput no worse than legacy; select_next p95 < 400 ms
  • Clean audit for screening and reconciliation actions; no data loss

MVP Goals

  1. Screening Profiles

  2. CRUD under Project. Immutable once used; Clone to revise. Holds criteria text and screening policy.

  3. Stage Settings

  4. Require StudySelectionMode at stage creation (no global default)

  5. Stage policies: HideExcludedStudiesFromReviewers, MaxInProgress, SessionCountTarget, SelfReconciliation

  6. Filter Sets (Stage-scoped)

  7. Storage schema supports nested groups and multi-profile conditions from day one

  8. MVP UI exposes a simple pass-forward rule (e.g., Outcome(Profile TAv1) ∈ {Included, Conflict, Maybe})

  9. Reviewer Experience

  10. "Get next" works per mode (screening | annotation | screeningAndAnnotation | reconciliation)

  11. Stats reflect per-caller availability

  12. Compatibility & Migration

  13. Legacy Mode continues to work
  14. Optional Migration Wizard creates first Screening Profile and backfills outcomes
  15. Snapshot/rollback supported

Out of Scope (MVP)

  • Annotation-based filtering (only reconciled annotation values will be filterable in Phase-2)
  • PRISMA 2020 diagram generation (Phase-2; counts can be exported later)
  • Formal tie-breaker groups/roles (Phase-2)

Ubiquitous Language (DDD)

  • Screening Profile — The criteria and policy used to screen. Immutable once used; clone-to-modify.
  • Screening Outcome — Structured result of screening a study under a specific profile. Status values: Included | Excluded | Conflict | Pending (legacy InsufficientlyScreened ⇒ Pending; legacy Disagree ⇒ Conflict).
  • Stage Study Pool — Reviewer-agnostic set defined only by the stage's Filter Set.
  • Selection Subset — Reviewer/reconciler-aware subset derived at selection time (applies stage policies like per-reviewer suppression, HideExcludedStudiesFromReviewers, MaxInProgress).
  • Candidate Annotation Session — An annotation session that counts toward SessionCountTarget (Reconciliation = false).
  • Reconciliation Annotation Session — Special session to consolidate candidate responses into Reconciled Annotations (Reconciliation = true); not counted toward SessionCountTarget.

Users & Roles

  • Project Admin — Manages Screening Profiles, stage settings, and Filter Sets; runs migration; can export.
  • Reviewer — Screens and/or annotates per stage; cannot manage profiles; can view criteria text on review UI.
  • Reconciler — Runs Reconciliation Mode to produce Reconciled Annotations.

Permission summary:

  • Manage Profiles/Filter Sets: Admins only
  • View criteria text on review UI: All reviewers (read-only context)

End-to-End Workflow (TA → FT example)

Systematic Search → Project
  ├─ Stage: Title/Abstract (Profile: TAv1)
  │   ├─ Filter Set: (none or simple)
  │   └─ Selection(mode=screening) → Reviewer decisions → ScreeningOutcome[TAv1]
  └─ Stage: Full-Text (Profile: FTv1)
      ├─ Filter Set: Outcome(TAv1) ∈ {Included, Conflict, Maybe}
      └─ Selection(mode=screeningAndAnnotation or annotation)
            → Candidate annotation sessions (count to SessionCountTarget)
            → Reconciliation sessions (produce Reconciled Annotations)

Dependencies

  • MongoDB indexes for outcomes, tallies, and per-reviewer session lookups
  • MassTransit consumers for materialisation (if using async updates; not used for tallies, per technical plan)
  • Feature flag system for gradual rollout

Risks & Mitigations

Risk Impact Mitigation
Document growth High Cap decision history per profile; archive or materialise summaries in Phase-2
Query complexity Medium Schema supports nested groups now; compile to efficient $elemMatch trees; add compile-time simplification
UX complexity Medium Progressive disclosure in Filter Builder; sensible templates in Phase-2
Migration correctness High Freeze + snapshot; batch migration with error handling; rollback plan

Open Questions

  1. Should HideExcludedStudiesFromReviewers apply in Reconciliation Mode?
  2. Where to surface TA/FT semantics (profile vs stage vs template)?
  3. PRISMA mapping defaults; how to capture duplicate counts and pre-screen removals UX
  4. Screening Profile immutability/clone policy: do we need an admin flow to transfer existing decisions to a new profile after a mistake?