E2E Testing Infrastructure

Problem Statement

SyRF has no end-to-end tests. Unit and integration tests cover individual components (saga state transitions, service methods, concurrency handling), but nothing validates the full user journey: clicking a button in the Angular frontend through to data persisted in MongoDB.

Core system flows — search upload, data export, bulk study update, screening, annotation — are only verified manually. Regressions in cross-service wiring, auth integration, or message routing go undetected until someone manually tests or a user reports a bug.

Goals

1. Validate core user journeys end-to-end: browser → Angular → .NET API → MassTransit → MongoDB
2. Test the real authentication flow (OIDC) without depending on Auth0
3. Test the real S3 upload → Lambda → RabbitMQ → saga flow without depending on AWS
4. Run in CI with no external service dependencies — fully self-contained
5. Two tiers: fast smoke suite for on-demand PR validation, comprehensive suite on merge to main
6. Replace the incomplete MockAuthProvider with a proper mock OIDC server

Non-Goals

• Testing Auth0-specific features (social account linking, Management API)
• Testing Elasticsearch integration (search indexing)
• Testing SES email delivery
• Testing Quartz scheduled jobs (deferred to future iteration)
• Performance or load testing
• Visual regression testing

Architecture

┌──────────────────────────────────────────────────────────────────────┐
│                     Playwright Test Runner                           │
│                                                                      │
│  ┌────────────┐  ┌──────────────────┐  ┌──────────────────────────┐ │
│  │ Tier 1     │  │ Tier 2           │  │ Fixtures                 │ │
│  │ @smoke     │  │ @full            │  │ - auth (storageState)    │ │
│  │ ~3 min     │  │ ~15 min          │  │ - db (MongoDB client)    │ │
│  │ label-gated│  │ merge to main    │  │ - api (HTTP client)      │ │
│  └────────────┘  └──────────────────┘  └──────────────────────────┘ │
└───────────────────────────┬──────────────────────────────────────────┘
                            │ HTTP (Chromium)
                            ▼
                 ┌─────────────────────┐
                 │  Angular Dev Server  │  (ng serve, port 4200)
                 │  OidcAuthProvider (generic OIDC)
                 │  issuer → mock OIDC server
                 └──────────┬──────────┘
                            │
          ┌─────────────────┼─────────────────┐
          ▼                 ▼                  ▼
  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐
  │ .NET API     │  │ .NET PM      │  │ Mock OIDC Server  │
  │ (dotnet run) │  │ (dotnet run) │  │ (Docker)          │
  │ port 5080    │  │ port 5081    │  │ port 9000         │
  │              │  │              │  │ navikt/mock-       │
  │ JWT Authority│  │ JWT Authority│  │ oauth2-server      │
  │ → :9000      │  │ → :9000      │  └──────────────────┘
  └──────┬───────┘  └──────┬───────┘
         │                 │
         ▼                 ▼
  ┌──────────────────────────────┐   ┌────────────────────────────┐
  │  MongoDB (Docker)             │   │  RabbitMQ (Docker)          │
  │  port 27017                   │   │  port 5672                  │
  │  DB: syrf_e2e                 │   │  MassTransit transport      │
  └──────────────────────────────┘   └─────────────┬──────────────┘
                                                    │
                                    ┌───────────────┴───────────────┐
                                    │  LocalStack (Docker)           │
                                    │  S3 (port 4566)                │
                                    │  Lambda: s3-notifier           │
                                    │  S3 event → Lambda → RabbitMQ  │
                                    └────────────────────────────────┘

Key Principle

The Angular app and .NET services run production code. The only configuration differences from a real deployment are:

• OIDC authority URL → mock OIDC server instead of Auth0
• MongoDB connection → local Docker instead of Atlas
• S3/Lambda → LocalStack instead of AWS
• RabbitMQ → local Docker instead of in-cluster broker
• External services (SES, Elasticsearch, Elastic APM) → disabled

Infrastructure Components

1. Mock OIDC Server — navikt/mock-oauth2-server

Replaces Auth0 for E2E tests. Acts as a real OIDC provider:

• Serves /.well-known/openid-configuration and JWKS endpoints
• Built-in login page where Playwright selects test users and configures claims
• Issues real signed JWTs that .NET services validate via standard JWT Bearer middleware
• Pre-configured test users carry SyRF-specific claims (https://claims.syrf.org.uk/user_id, syrf_groups)

How it integrates:

• Angular: The @auth0/auth0-angular SDK is not a generic OIDC client — it hardcodes Auth0-specific URL paths (/oauth/token, /authorize) rather than using OIDC discovery. Pointing its domain at the mock server will not work. Instead, a new OidcAuthProvider implementation of the existing IAuthProvider interface will use a standard OIDC client library (e.g., angular-auth-oidc-client) and be registered via the AUTH_PROVIDER injection token when running in E2E mode. The @auth0/auth0-angular package remains for production. This is exactly what the IAuthProvider abstraction was designed for.
• .NET: Standard .AddJwtBearer() with Authority pointed at the mock server. Same JWT validation pipeline, same claims extraction, same [Authorize] policies. No backend code changes required.
• Auth0-specific features (social account linking, Management API, silent logout) are guarded by feature-availability checks and will be no-ops when those features are unavailable from the mock OIDC server.

Angular environment injection: The app loads config from appConfig.default.json at runtime (not compile-time environments). For E2E, the config is overridden via environment variable substitution in env.template.generated.json, setting auth0CustomDomain and auth0ClientId to the mock OIDC server coordinates. An authProvider config key selects between Auth0AuthProvider (production) and OidcAuthProvider (E2E). The ng serve proxy config routes API calls to the local .NET services.

Test users (seeded in both mock OIDC config and MongoDB pmInvestigator):

User	Role	Purpose
e2e-admin@test.com	Administrator, SyrfAdmin	Full access for setup/admin tests
e2e-reviewer@test.com	Member	Screening and annotation tests
e2e-user@test.com	Member	Standard user journey tests

2. MockAuthProvider Removal

The existing MockAuthProvider (frontend-only, hardcoded user, X-Dev-Api-Key header) is incomplete — the backend has no handler for X-Dev-Api-Key (noted as a TODO in the interceptor code). It will be removed and replaced by the mock OIDC server approach.

Files to remove: - src/services/web/src/app/core/auth/mock-auth.provider.ts - src/services/web/src/app/core/auth/mock-auth.provider.spec.ts

Configuration to remove (complete scope): - mockAuth flag from appConfig.default.json, appConfig.default.generated.json, env.template.generated.json - mockAuth branching in auth-interceptor.service.ts — remove _handleMockAuth method, X-Dev-Api-Key logic, DEV_API_KEY import - mockAuth references in app-settings.interface.ts, mock-config.ts - mockAuthMode property in nav.component.ts and template usage - mockAuth guard in admin.effects.ts - Auto-generated files referencing mockAuth: - generated/feature-flags.constants.ts (FEATURE_FLAG_DEFAULTS, ENV_VAR_NAMES, JSON_PATHS) - generated/feature-flags.types.ts (AppConfig, ExternalConfig interfaces) - generated/feature-flags.parser.ts (mockAuth parsing) - generated/feature-flags.selectors.ts (selectMockAuth selector) - app-config.generated.ts, external-config.generated.ts, load-config.generated.ts - Note: generated files are produced by pnpm run generate:flags — the generator config must be updated, then files regenerated - app-config.service.ts — remove parseMockAuth special-casing - .chart/values.yaml for web — remove mockAuth feature flag

mockAuth guard replacement strategy for auth.effects.ts:

Effect	Current guard	Replacement
connectAccount$	!mockAuth	!!config.auth0ManagementApiAudience
disconnectAccount$	!mockAuth	!!config.auth0ManagementApiAudience
managementAccessToken$	!mockAuth	!!config.auth0ManagementApiAudience
signInActions$	mockAuth → EMPTY	Remove guard — OidcAuthProvider.loginWithRedirect() handles this
refreshTokenSilentlyRefreshed$	!mockAuth	Remove guard — OidcAuthProvider.getAccessTokenSilently() handles this
userRegistered$	!mockAuth	Remove guard — OidcAuthProvider.loginWithRedirect() handles this
silentSignOut$	!mockAuth	!!config.auth0CustomDomain (hardcoded Auth0 logout URL)
signInRecorded$	!mockAuth	Remove guard — both providers emit user$
mockUserSignInRecorded$	mockAuth	Remove entirely — signInRecorded$ covers both

Hardcoded Auth0 URL in silentSignOut$: The effect contains a hardcoded https://signin.syrf.org.uk/v2/logout?client_id=... URL. This should be made configurable (read from auth0CustomDomain config) and guarded by !!config.auth0CustomDomain.

What replaces MockAuthProvider: - For E2E tests: mock OIDC server (full OIDC flow, real JWT validation) - For local dev without Auth0: developers run the mock OIDC server locally via docker compose -f e2e/docker-compose.e2e.yml up mock-oidc

3. MongoDB — mongo:7

• Database: syrf_e2e (isolated from dev/staging)
• Seeded per test run with baseline data (test users in pmInvestigator)
• Test data uses per-test unique identifiers to avoid collisions during parallel execution
• Read-only reference data (user accounts) shared across tests
• Mutable data (projects, studies, annotations) created per test

4. RabbitMQ — rabbitmq:3-management

• Standard MassTransit transport configuration
• Management UI available on port 15672 for debugging
• Used by both .NET services and the Lambda (via LocalStack)

5. LocalStack — S3 + Lambda

Emulates the AWS S3 → Lambda flow with the real Lambda code.

Why LocalStack instead of MinIO + substitute:

The S3 notifier Lambda contains significant business logic that must be tested: - Two distinct workflows based on uploadkind metadata (search upload vs bulk study update) - Complex JSON deserialization of ScreeningImportSettings (includes Dictionary fields) and BulkStudyUpdateJobInfo (nested record type) - LibraryFileType enum parsing with validation - 12 required metadata fields across both workflows with specific error messages - Different MassTransit patterns: Publish<T>() for events vs SubmitJob<T>() for commands

A substitute that bypasses this logic would leave real bugs untested.

Setup flow: 1. E2E setup script builds Lambda package: dotnet lambda package 2. Deploys to LocalStack via AWS CLI: aws --endpoint-url=http://localhost:4566 lambda create-function ... 3. Creates S3 bucket (syrfappuploads) with event notification → Lambda trigger 4. Lambda environment variables configured with RabbitMQ connection pointing to local Docker

Trade-off: LocalStack image is ~1.9 GB and takes 30-60s to start. This is acceptable because: - One-time cost per CI run, not per test - The Lambda's metadata extraction, routing, and deserialization are critical business logic - Maintaining a substitute creates divergence risk

6. .NET Services — Native dotnet run

• API on port 5080, PM on port 5081 (offset from default dev ports to avoid conflicts)
• Environment: E2ETest with appsettings.E2ETest.json:
• MongoDB → mongodb://localhost:27017/syrf_e2e
• JWT Authority → http://localhost:9000
• RabbitMQ → amqp://guest:guest@localhost:5672
• S3 endpoint → http://localhost:4566 (LocalStack)
• Elasticsearch, SES, Elastic APM, Sentry → disabled

7. Angular Dev Server — ng serve

• Port 4200 with proxy config routing /api/* to .NET services
• E2E runtime config selects OidcAuthProvider (generic OIDC) instead of Auth0AuthProvider
• OidcAuthProvider implements the existing IAuthProvider interface using angular-auth-oidc-client, configured with the mock OIDC server's issuer URL
• All downstream code (interceptors, effects, guards, ngrx state) runs identically — they depend on IAuthProvider, not on Auth0 directly

Test Tiers and CI Integration

Tier 1: Smoke Suite (@smoke)

Trigger: PR label run:e2e-smoke, or auto-trigger on paths touching saga/auth/export code

Duration target: ~3 minutes

Tests: - Login → land on dashboard - Create a project - Upload a systematic search → verify studies appear in project - Bulk study update (PDF path, Custom ID) → verify changes reflected - Export data → verify CSV downloads with correct content

Tier 2: Comprehensive Suite (@full)

Trigger: PR label run:e2e-full, merge to main, or workflow_dispatch

Duration target: ~15 minutes

Tests: - Everything in Tier 1 - Screening workflow (include/exclude studies across reviewers) - Annotation workflow (add annotations, verify persistence) - Multi-user scenarios (two users screening the same project) - Data integrity checks (export matches what was entered via screening/annotation/bulk update) - Error paths (invalid reference files, upload timeouts) - Bulk study update with all supported field types

CI Workflow

Label-triggered, same pattern as preview environments:

on:
  pull_request:
    types: [labeled, synchronize]
  push:
    branches: [main]
  workflow_dispatch:
    inputs:
      suite:
        type: choice
        options: [smoke, full]

jobs:
  e2e-smoke:
    if: >
      contains(github.event.pull_request.labels.*.name, 'run:e2e-smoke')
      || (github.event_name == 'workflow_dispatch' && inputs.suite == 'smoke')

  e2e-full:
    if: >
      github.event_name == 'push'
      || contains(github.event.pull_request.labels.*.name, 'run:e2e-full')
      || (github.event_name == 'workflow_dispatch' && inputs.suite == 'full')

Infrastructure: Docker containers declared as GitHub Actions services: (health-checked automatically). .NET services started via dotnet run in background with health check polling. Angular started via Playwright's webServer config.

Label lifecycle: Label automatically removed after run completes. Re-add to re-trigger.

Artifacts on failure: - Playwright HTML report (14-day retention) - Traces on first retry - Screenshots on failure - Video retained on failure

Project Structure

e2e/                                    # Top-level, spans all services
├── package.json                        # Playwright, MongoDB driver, helpers
├── tsconfig.json
├── playwright.config.ts                # Projects, webServer, tiers
├── docker-compose.e2e.yml              # MongoDB, RabbitMQ, mock OIDC, LocalStack
├── scripts/
│   ├── setup-localstack.sh             # Build + deploy Lambda to LocalStack
│   ├── seed-db.ts                      # Baseline MongoDB seeding
│   └── wait-for-services.ts            # Health check polling
├── config/
│   ├── mock-oidc/                      # navikt test users + client config
│   ├── angular-e2e-env.json            # Angular environment overrides
│   └── appsettings.E2ETest.json        # .NET service config
├── global-setup.ts                     # Start infra, seed DB, start services
├── global-teardown.ts                  # Cleanup
├── fixtures/
│   ├── auth.fixture.ts                 # Authenticated page contexts
│   ├── auth.setup.ts                   # Login via mock OIDC, save storageState
│   ├── db.fixture.ts                   # MongoDB client for seeding/assertions
│   └── api.fixture.ts                  # Direct API client for test setup
├── pages/                              # Page Object Models
│   ├── dashboard.page.ts
│   ├── project.page.ts
│   ├── screening.page.ts
│   ├── annotation.page.ts
│   ├── data-export.page.ts
│   ├── search-upload.page.ts
│   └── bulk-study-update.page.ts
├── tests/
│   ├── smoke/                          # @smoke tagged — Tier 1
│   │   ├── login.spec.ts
│   │   ├── create-project.spec.ts
│   │   ├── search-upload.spec.ts
│   │   ├── bulk-study-update.spec.ts
│   │   └── data-export.spec.ts
│   └── full/                           # @full tagged — Tier 2
│       ├── screening-workflow.spec.ts
│       ├── annotation-workflow.spec.ts
│       ├── multi-user-screening.spec.ts
│       ├── data-integrity.spec.ts
│       └── error-paths.spec.ts
├── helpers/
│   ├── mongo-client.ts                 # CSUUID-aware MongoDB queries
│   ├── test-data-factory.ts            # Generate unique test data per test
│   └── wait-for-saga.ts               # Poll API for saga/job completion
└── playwright/
    └── .auth/                          # gitignored — storageState files

Test Patterns

Per-Test Data Isolation

Every test creates its own project/data with unique identifiers to avoid collisions during parallel execution:

test('can upload a systematic search @smoke', async ({ authenticatedPage, db }) => {
  const projectName = `e2e-search-upload-${Date.now()}`;
  // Create project via UI or API fixture
  // Upload search file
  // Assert studies appear
  // Assert MongoDB state
});

Waiting for Async Operations

MassTransit saga processing is asynchronous. Tests poll for expected outcomes:

// Preferred: wait for UI to reflect the change
await expect(page.getByText('5 studies imported')).toBeVisible({ timeout: 30_000 });

// Alternative: poll API when UI doesn't have a visible indicator
await expect.poll(async () => {
  const res = await apiClient.get(`/api/projects/${projectId}/searches/${searchId}`);
  return (await res.json()).status;
}, { timeout: 30_000, intervals: [1000, 2000, 5000] }).toBe('completed');

Database Assertions

Verify backend state directly when UI assertions aren't sufficient:

// Custom fixture provides CSUUID-aware MongoDB client
const study = await db.collection('pmStudy').findOne({
  SystematicSearchId: searchId,
});
expect(study.ScreeningInfo.ProjectAgreementThresholds).toBeDefined();

Multiple User Roles

Playwright setup project authenticates each test user and caches storageState:

// Multi-user test
test('two reviewers can screen the same study @full', async ({ browser }) => {
  const reviewer1 = await browser.newContext({
    storageState: 'playwright/.auth/reviewer1.json',
  });
  const reviewer2 = await browser.newContext({
    storageState: 'playwright/.auth/reviewer2.json',
  });
  // Both navigate to same project, screen same study
  // Verify agreement/disagreement logic
});

Angular Considerations

• Disable Angular animations in E2E mode to avoid timing issues with Material components
• Use data-testid attributes for stable Playwright selectors where role/text locators are insufficient
• Playwright's auto-waiting handles Angular zone stability automatically

Risks and Mitigations

Risk	Mitigation
LocalStack S3→Lambda notifications unreliable	Pin LocalStack version; verify notification chain in global setup before tests run
Flaky tests from shared mutable data	Per-test data isolation with unique identifiers
Slow CI from 4 Docker containers + 2 .NET services	Tier 1 is fast (~3 min); Tier 2 is label/merge-gated
@auth0/auth0-angular incompatible with mock OIDC	New OidcAuthProvider using generic OIDC library behind existing IAuthProvider interface
LocalStack .NET 10 runtime issues	Fallback: build Lambda as container image, deploy to LocalStack as container Lambda
Port conflicts with local dev	E2E uses offset ports (5080/5081) distinct from dev ports (8080/8081)
LocalStack image size (~1.9 GB)	Cached in GitHub Actions; one-time pull per runner
LocalStack requires Docker socket mount	GitHub Actions ubuntu-latest supports this; document in setup guide
Mock OIDC port 9000 protocol ambiguity	Use HTTP on port 9000 (non-HTTPS port); avoids self-signed cert handling
.NET service health check not specified	Services expose /health endpoint; wait-for-services.ts polls with 2s interval, 60s timeout, fail-fast on process exit

Future Iterations

• Quartz integration: Add SQL Server container, test background job scheduling flows
• Visual regression: Add screenshot comparison for key UI states
• Performance baseline: Measure response times for critical flows
• Accessibility testing: Integrate axe-core via Playwright
• Preview environment E2E: Run smoke suite against deployed PR preview environments