PR Preview Workflow: Parallel Builds Plan¶

Note: This is a temporary planning document. Once the parallel build implementation is verified and stable, this document can be deleted.

Overview¶

Convert the PR preview workflow from sequential Docker builds (single job) to parallel builds using a matrix strategy, matching the pattern used in the main CI/CD workflow.

Current State¶

File: .github/workflows/pr-preview.yml

The build-and-push-images job (lines 352-848) builds all services sequentially:

API → PM → Quartz → Web → Docs → User Guide

Each build waits for the previous to complete, resulting in ~15-20 minute total build time when multiple services change.

Target State¶

Use a matrix strategy to build services in parallel on separate runners, reducing wall-clock time to ~5-8 minutes (limited by the slowest individual build).

Implementation Plan¶

Step 1: Extract Reusable Docker Build Workflow¶

The main CI/CD already has _docker-build.yml. Verify it supports all PR preview requirements:

Action: Review _docker-build.yml and add PR preview support if needed.

Step 2: Create Service Matrix in detect-changes Job¶

Add output for PR preview service matrix similar to docker_services_matrix in ci-cd.yml:

outputs:
  preview_services_matrix: ${{ steps.matrix.outputs.services }}

The matrix should include:

Service name
Image name
Dockerfile path
Context path
Whether it needs web artifacts
Whether it needs multi-repo checkout

Step 3: Replace Sequential Build Steps with Matrix Job¶

Before (current):

build-and-push-images:
  name: Build and push preview images
  needs: [check-label, detect-changes, version-*, build-web-artifacts]
  steps:
    - name: Build and push API image
      if: needs.detect-changes.outputs.api_changed == 'true'
      # ...
    - name: Build and push project-management image
      if: needs.detect-changes.outputs.project_management_changed == 'true'
      # ...
    # ... repeat for each service

After (parallel):

build-and-push-images:
  name: "Build: ${{ matrix.service.name }}"
  needs: [check-label, detect-changes, version-*, build-web-artifacts]
  if: needs.detect-changes.outputs.preview_services_matrix != '[]'
  strategy:
    matrix:
      service: ${{ fromJson(needs.detect-changes.outputs.preview_services_matrix) }}
    fail-fast: false

  uses: ./.github/workflows/_docker-build.yml
  with:
    service_name: ${{ matrix.service.name }}
    image_name: ${{ matrix.service.image }}
    dockerfile: ${{ matrix.service.dockerfile }}
    # ...

Step 4: Handle Version Resolution¶

Current workflow uses direct references to version job outputs:

tags: ghcr.io/.../syrf-api:${{ needs.version-api.outputs.semver }}

Matrix approach needs dynamic version lookup. Options:

Option A: Pass all versions as inputs, select in reusable workflow

with:
  api_version: ${{ needs.version-api.outputs.semver }}
  pm_version: ${{ needs.version-project-management.outputs.semver }}
  # ... etc

Option B: Use a single version job with matrix outputs (like ci-cd.yml)

version:
  outputs:
    api_version: ${{ steps.api.outputs.semver }}
    # ... consolidated outputs

Recommendation: Option B - matches ci-cd.yml pattern.

Step 5: Handle Unchanged Service Tagging¶

Current workflow tags unchanged services with sha-{shortsha} at the end:

- name: Tag latest images for unchanged services
  run: |
    if [ "$API_CHANGED" != "true" ]; then
      crane copy "ghcr.io/.../syrf-api:latest" \
                 "ghcr.io/.../syrf-api:sha-${SHORT_SHA}"
    fi

Options:

Keep as separate job after matrix builds complete
Add retag logic to reusable workflow with action: retag parameter

Recommendation: Separate retag-unchanged job (simpler, matches ci-cd pattern).

Step 6: Move PR Description Update to Separate Job¶

The Update PR description with K8s status step needs all build results. Move to a dedicated job that runs after all builds complete:

update-pr-status:
  name: Update PR Status
  needs: [build-and-push-images, retag-unchanged]
  if: always()
  # ... update PR description with consolidated status

Step 7: Update Downstream Jobs¶

Jobs that depend on build-and-push-images need adjustment:

write-versions - needs all builds complete
create-tags - needs all builds complete

Both should work with needs: [build-and-push-images] since matrix jobs complete as a group.

File Changes Summary¶

File	Change
`.github/workflows/pr-preview.yml`	Major refactor - matrix strategy
`.github/workflows/_docker-build.yml`	Add PR preview tagging support (if needed)
`.github/workflows/_detect-changes.yml`	Add `preview_services_matrix` output (optional)

Considerations¶

Pros¶

Faster builds: ~5-8 min vs ~15-20 min wall clock
Consistency: Matches main ci-cd.yml pattern
Isolation: Service build failures don't block others

Cons¶

Higher cost: 6 parallel runners vs 1 sequential
Complexity: Matrix + reusable workflow vs simple steps
Cache efficiency: Separate Docker caches per runner (mitigated by GHA cache)

Risk Mitigation¶

Test on a feature branch before merging
Keep sequential version as fallback (can revert if issues)
Monitor runner costs after deployment

Testing Plan¶

Create feature branch with changes
Open PR with preview label
Verify all 6 services build in parallel
Verify unchanged services get correct sha- tags
Verify PR description updates correctly
Verify cluster-gitops version files written correctly
Compare total build time vs sequential

Rollout¶

Implement changes on feature branch
Test with real PR preview
Merge to main
Monitor first few PR previews for issues