Skip to content

ArgoCD Deployment - GitOps Migration

Status: 🔄 In Progress (50% - 7/14 child work items complete) Priority: High Assignee: @chrissena Story Points: 58 (32 original + 26 cluster remediation) Parent Epic: #2128 (SyRF GitOps Migration)

Problem Statement

SyRF needs to migrate from Jenkins X to a modern GitOps-based deployment workflow using ArgoCD. The current Jenkins X platform is deprecated and requires a complete infrastructure overhaul.

The legacy deployment system has several limitations:

  • Jenkins X is no longer maintained
  • Complex pipeline configuration
  • Difficult to debug deployment issues
  • No clear separation between infrastructure and application concerns
  • Manual intervention often required for deployments

Proposed Solution

Install and configure ArgoCD on a new GKE Kubernetes cluster with a complete GitOps workflow:

  1. Cluster Provisioning: New GKE cluster (camaradesuk) in europe-west2-a
  2. ArgoCD Installation: High-availability mode with Web UI
  3. Infrastructure Components: cert-manager, ingress-nginx, external-dns, rabbitmq
  4. App-of-Apps Pattern: Bootstrap all applications via ArgoCD
  5. Service Deployment: Deploy all 6 SyRF services to staging
  6. Validation: End-to-end GitOps flow testing

Acceptance Criteria

Phase 1: Infrastructure (Completed ✅)

  • GKE cluster provisioned: camaradesuk (europe-west2-a)
  • Kubernetes v1.33.5-gke.1201000, 3-6 nodes (e2-standard-2)
  • ArgoCD installed in HA mode
  • ArgoCD Ingress configured (argocd.camarades.net)
  • Infrastructure components deployed:
  • cert-manager (v1.15.0)
  • ingress-nginx (4.11.1) - LoadBalancer IP: 34.13.36.98
  • external-dns (1.14.5) - DNS automation for syrf.org.uk
  • rabbitmq (14.6.6)

Phase 2: Application Bootstrap (Completed ✅)

  • App-of-Apps pattern implemented and tested
  • All 6 staging services deployed:
  • API 9.2.3
  • Project Management 11.3.2
  • Quartz 0.5.1
  • Web 5.4.2
  • Docs 1.6.5
  • User Guide 1.1.0
  • Helm charts standardized (removed Jenkins X legacy patterns)
  • ExternalSecrets configuration fixed
  • Project-management renamed from pm to project-management
  • IdentityServer4 removed (replaced by Auth0)
  • All staging image tags fixed

Phase 3: Configuration & Validation (In Progress 🔄)

Completed

  • #2149: Kubernetes cluster provisioning (13 pts)
  • #2150: ArgoCD installation (5 pts)
  • #2151: Platform add-ons installation (8 pts)
  • #2152: App-of-Apps bootstrap (3 pts)
  • #2172: Helm Chart Standardization (3 pts)
  • #2195: Fix SecretStore Configuration (8 pts)
  • #2196: Fix Staging Image Tags (5 pts)

In Progress

  • #2153: First service deployment (5 pts) - 75% complete, waiting for secrets

Ready for Work

  • #2197: Fix Extra-Secrets Directory Structure (2 pts) - P1
  • #2201: Configure ArgoCD Sync Policies (5 pts) - P1
  • #2198: Fix User Guide TLS Certificate (3 pts) - P2
  • #2199: Sync Out-of-Sync Applications (2 pts) - P2
  • #2200: Clean Up Orphaned Resources (1 pt) - P3
  • #2154: End-to-end GitOps flow validation (8 pts)

Technical Notes

Architecture

GitOps Workflow:

Git Repository (cluster-gitops)
  ↓
ArgoCD (App-of-Apps Pattern)
  ↓
Kubernetes Cluster (GKE)
  ↓
SyRF Services (Staging)

Key Components:

  • ArgoCD: GitOps continuous delivery tool
  • App-of-Apps: Single ApplicationSet generates all service Applications
  • Helm Charts: Monorepo-based charts in syrf/src/services/{service}/.chart/
  • Environment Values: Separated in cluster-gitops/environments/{env}/services/
  • ExternalSecrets: Operator for managing secrets from external sources

Repository Structure

cluster-gitops/
├── applicationsets/
│   └── syrf.yaml              # Generates all Applications
├── environments/
│   ├── staging/
│   │   └── services/          # Service configurations
│   └── production/
│       └── services/
└── syrf/
    ├── api/
    ├── project-management/
    ├── quartz/
    ├── web/
    ├── docs/
    └── user-guide/

Dependencies

Blocked by (Resolved):

  • ✅ GKE cluster provisioning (#2149)
  • ✅ Image tag issues (#2196)

Blocks:

  • 2155: Production cutover

  • 2156: Legacy infrastructure decommissioning

Configuration Files

File Purpose
cluster-gitops/applicationsets/syrf.yaml ApplicationSet definition
cluster-gitops/environments/staging/ Staging environment configuration
cluster-gitops/environments/production/ Production environment configuration
src/services/{service}/.chart/ Helm charts (monorepo)

Secrets Management

ExternalSecrets Operator:

  • Configured for Google Cloud Secret Manager
  • Workload Identity enabled for secure access
  • SecretStore resources in each namespace
  • No secrets committed to git (GitOps principle)

Blockers and Risks

Current Blockers

  • ⏳ Secrets Configuration: First service deployment waiting for secrets setup (#2153)

Risks Mitigated

  • ✅ Cluster Provisioning: Completed 2025-11-12
  • ✅ Image Tag Issues: Resolved 2025-11-18
  • ✅ Helm Chart Legacy Patterns: Removed 2025-11-18

Remaining Risks

  • 🔶 Production Cutover: Requires careful planning and rollback strategy
  • 🔶 DNS Migration: Need to update DNS records to point to new cluster
  • 🔶 Data Migration: MongoDB data needs to be migrated or pointed to shared instance

Testing Strategy

Unit Tests

  • Helm template rendering tests
  • YAML linting and validation

Integration Tests

  • ArgoCD Application sync tests
  • Service health checks
  • Ingress connectivity tests

End-to-End Tests

  • Full GitOps flow: git commit → ArgoCD sync → service updated
  • Rollback testing
  • Multi-environment promotion (staging → production)

Documentation

External Documentation:

  • cluster-gitops/docs/cluster-bootstrap.md
  • cluster-gitops/docs/deploying-services.md
  • camarades-infrastructure/docs/terraform-guide.md

Success Metrics

  • ✅ All 6 services deployed to staging (100%)
  • 🔄 Configuration issues resolved (7/12 complete - 58%)
  • ⏳ End-to-end GitOps flow validated (pending)
  • ⏳ Production cutover completed (pending)

Timeline

Started: 2025-11-07 Latest Update: 2025-11-18 Target Completion: TBD (depends on secrets configuration and production cutover planning)

  • Parent: #2128 (Epic: SyRF GitOps Migration)
  • Child Issues: 14 work items (7 completed, 1 in progress, 6 ready)
  • Reference: docs/planning/migration/backlog.md Work Item 4

Source: GitHub Issue #2148 Last Synced: 2025-11-24 16:00

This feature brief was auto-generated from the GitHub issue. The GitOps migration is well underway with significant progress on infrastructure and application deployment. Remaining work focuses on configuration cleanup and validation.

Next Actions:

  1. Complete secrets configuration for first service deployment (#2153)
  2. Fix extra-secrets directory structure (#2197)
  3. Configure ArgoCD sync policies (#2201)
  4. Validate end-to-end GitOps flow (#2154)
  5. Plan production cutover (#2155)