Cluster GitOps Structural Changes Summary¶

Overview¶

This document summarizes the structural changes made to the cluster-gitops repository to address issues identified in the comprehensive critique.

Date: 2025-11-16 Changes By: User + Claude Related Documents: - Cluster GitOps Critique - README.md

Changes Made¶

1. Removed Duplicate ApplicationSet Directory ✅¶

Issue: Two ApplicationSet directories existed with different implementations.

Action: - ❌ Deleted: argo/applicationsets/ directory (obsolete) - ✅ Kept: argocd/applicationsets/ (canonical implementation)

Files Affected: - Deleted: argo/applicationsets/syrf.yaml

Rationale: The argo/ directory contained an experimental merge generator implementation that was superseded by the final version in argocd/.

2. Removed custom-charts ApplicationSet ✅¶

Issue: The custom-charts.yaml ApplicationSet was not being used.

Action: - ❌ Deleted: argocd/applicationsets/custom-charts.yaml

Rationale: Custom charts (like extra-secrets) are better managed through the existing structure rather than a separate ApplicationSet.

3. Updated ApplicationSet Implementation ✅¶

File: argocd/applicationsets/syrf.yaml

Changes: - Switched from matrix generator to merge generator pattern - Updated variable names from .service.name / .environment.name to .serviceName / .envName - Added selector to filter disabled services and missing chartTags - Maintained multi-source Helm pattern

Benefits: - More flexible configuration merging - Cleaner variable naming - Better filtering of incomplete configurations

4. Fixed Plugins ApplicationSet ✅¶

File: argocd/applicationsets/plugins.yaml

Changes: - Updated path from plugins/git/*/config.yaml to plugins/local/*/config.yaml - Updated valueFiles path to match plugins/local/{{.plugin.name}}/values.yaml

Rationale: Matches actual directory structure (plugins/local/ exists, not plugins/git/).

5. Updated Documentation ✅¶

File: README.md

Changes: - Fixed repository structure diagram to match reality - Updated all paths from old structure to new structure - Corrected deployment workflow paths - Updated AppProjects table (infrastructure → plugins) - Fixed "Check Current Versions" commands - Updated "Add New Service" instructions

Key Path Updates:

OLD: environments/staging/syrf.yaml
NEW: syrf/environments/staging/{service}/config.yaml

OLD: applicationsets/
NEW: argocd/applicationsets/

OLD: infrastructure.yaml
NEW: plugins.yaml

OLD: plugins/git/
NEW: plugins/local/

6. Updated Monorepo CI/CD Workflow ✅¶

File: /home/chris/workspace/syrf/.github/workflows/ci-cd.yml

Changes:

Staging Promotion (promote-to-staging job):¶

# OLD
yq e ".service.chartTag = \"$TAG\"" -i "syrf/environments/staging/$FILE/config.yaml"

# NEW
yq e ".service.chartTag = \"$TAG\"" -i "syrf/environments/staging/$SERVICE/config.yaml"

Validation:¶

# OLD
for file in environments/staging/services/*.yaml; do

# NEW
for file in syrf/environments/staging/*/config.yaml; do

Production Promotion (promote-to-production job):¶

# OLD
for file in environments/staging/services/*.yaml; do
  filename=$(basename "$file")
  cp "$file" "environments/production/services/$filename"
done

# NEW
for service_dir in syrf/environments/staging/*/; do
  service=$(basename "$service_dir")
  yq eval '.envName = "production"' \
    "syrf/environments/staging/$service/config.yaml" > \
    "syrf/environments/production/$service/config.yaml"
done

Benefits: - Correctly updates service config files - Properly sets envName field when promoting to production - Uses correct directory structure

Repository Structure (After Changes)¶

cluster-gitops/
├── argocd/                       # ArgoCD bootstrap and configuration
│   ├── applicationsets/         # ApplicationSet definitions
│   │   ├── syrf.yaml            # ✅ CANONICAL - Merge generator pattern
│   │   └── plugins.yaml         # ✅ Infrastructure plugins
│   ├── projects/                # AppProject definitions (RBAC)
│   ├── bootstrap/               # Bootstrap Applications (root.yaml)
│   └── resources/               # ArgoCD resources (ingress, etc.)
│
├── syrf/                         # SyRF application configurations
│   ├── services/                # Base service configurations
│   │   ├── api/
│   │   │   ├── config.yaml      # Base metadata (chartPath, chartRepo)
│   │   │   └── values.yaml      # Shared Helm values
│   │   └── ...
│   │
│   └── environments/            # Environment-specific configurations
│       ├── staging/
│       │   ├── namespace.yaml   # Environment metadata
│       │   ├── shared-values.yaml
│       │   ├── api/
│       │   │   ├── config.yaml  # ✅ UPDATED BY CI/CD (chartTag)
│       │   │   └── values.yaml
│       │   └── ...
│       │
│       └── production/
│           ├── namespace.yaml
│           ├── shared-values.yaml
│           ├── api/
│           │   ├── config.yaml  # ✅ PROMOTED FROM STAGING
│           │   └── values.yaml
│           └── ...
│
├── plugins/                      # Infrastructure components
│   ├── helm/                    # Helm chart repository plugins
│   │   ├── cert-manager/
│   │   ├── ingress-nginx/
│   │   └── ...
│   │
│   └── local/                   # ✅ Local/git-based chart plugins
│       └── rabbitmq-secrets/
│
├── charts/                       # Custom Helm charts
│   └── extra-secrets/
│
├── global/
│   └── values.yaml
│
├── scripts/
│   └── promote-to-production.sh
│
└── docs/
    ├── architecture/
    │   ├── cluster-gitops-critique.md
    │   └── structural-changes-summary.md (this file)
    ├── decisions/
    ├── how-to/
    ├── troubleshooting/
    └── ...

Deployment Workflow (Updated)¶

Staging Deployment¶

1. Developer pushes to main
2. CI/CD builds image & creates tag (e.g., api-v9.1.1)
3. CI/CD updates: syrf/environments/staging/api/config.yaml
   └─ Sets: service.chartTag = "api-v9.1.1"
4. GitHub webhook → ArgoCD
5. ApplicationSet regenerates api-staging Application
6. ArgoCD syncs to cluster

Production Promotion¶

1. After successful staging deployment
2. CI/CD copies staging configs to production
3. Updates: syrf/environments/production/api/config.yaml
   └─ Copies chartTag from staging
   └─ Sets envName = "production"
4. Creates PR with "requires-review" label
5. Team reviews and manually merges
6. ArgoCD syncs to production

Testing Required¶

Before considering these changes complete, test the following:

ApplicationSet Generation¶

# Verify ApplicationSet generates correct Applications
kubectl -n argocd get applicationset syrf -o yaml
kubectl -n argocd get applications -l platform=syrf

Service Config Files¶

# Verify all services have correct structure
for env in staging production; do
  for service in api web project-management quartz docs user-guide; do
    echo "Checking $env/$service..."
    yq eval . "syrf/environments/$env/$service/config.yaml"
  done
done

CI/CD Workflow¶

Trigger a build and verify staging promotion works
Check that production PR is created correctly
Verify YAML validation passes

Remaining Concerns¶

1. Grep Output Corruption (LOW PRIORITY)¶

When running grep -r commands, output appears corrupted with repeated text. This may be: - Terminal rendering issue - WSL2 filesystem issue - Grep version issue

Impact: Low - doesn't affect actual files, only search results

Workaround: Use fd + individual file reads instead of grep -r

2. Configuration Complexity (DOCUMENTED)¶

The merge generator pattern adds complexity but provides flexibility. This is documented in the critique as a trade-off.

3. No Automated Validation (MEDIUM PRIORITY)¶

Recommendation from critique: Add validation script to check: - YAML syntax - Required fields present - Path references are valid - ApplicationSet templates render correctly

Success Criteria¶

✅ All changes implemented ✅ Documentation updated to match reality ✅ CI/CD workflow updated ✅ No orphaned directories ✅ ApplicationSets using canonical paths

⏳ Testing in actual cluster (pending) ⏳ Validation script creation (future enhancement)

Conclusion¶

The cluster-gitops repository structure has been cleaned up and aligned with best practices:

Removed confusion - Single ApplicationSet implementation, no duplicates
Fixed documentation - README matches actual structure
Updated automation - CI/CD workflows use correct paths
Maintained functionality - All features intact, just better organized

Next Steps: 1. Test ApplicationSet generation in cluster 2. Verify CI/CD promotion workflow 3. Consider adding validation tooling (optional) 4. Update team documentation as needed

Document Status: Approved - Changes implemented and documented Last Updated: 2025-11-16