Skip to content

AWS-to-GCP Consolidation

Executive Summary

SyRF currently runs across two cloud providers (AWS + GCP) for historical reasons, not architectural ones. AWS hosts email sending (SES), file storage (S3), a notification Lambda, and mailboxes (WorkMail). GCP hosts the GKE cluster, databases, DNS, and all application services.

This dual-cloud setup costs approximately \(587/year in AWS** plus **~\)162/year in dead GCP resources, creates operational overhead (two IAM systems, two billing accounts, cross-cloud credential management, ACK controllers), and delivers no redundancy benefit since both clouds are required for the system to function.

Proposed outcome: Eliminate AWS entirely by migrating S3 to Cloud Storage, replacing the Lambda with Pub/Sub, moving email templates into the codebase, and migrating WorkMail to Zoho Mail. Projected annual savings: ~$850-1,000/year with significant operational simplification.

Current State: AWS Audit

AWS Costs (Feb 2025 - Jan 2026, from Cost Explorer API)

Service Monthly Avg Annual Notes
WorkMail $16.00 $192 4 users at $4/user/month
VPC (Elastic IPs) $10.93 $131 3 idle EIPs, 100% waste
EC2 - Other $5.92 $71 EBS volumes (\(5.50) + snapshots (\)0.42)
KMS $2.00 $24 2 customer-managed keys (likely from old EKS)
Route 53 $1.50 $18 DNS hosting
Secrets Manager $0.60 $7 Grew from $0.40 to $0.80 in Sep 2025
S3 $1.27 $15 File storage (slowly growing)
ECR $0.01 $0.13 Container Registry
SES $0.02 $0.29 Email sending
Lambda $0.00 $0.00 Free tier
Tax (20% VAT) $8.12 $97
Domain registrations -- $31 Two domains renewed (Jun + Aug)
Total ~$47 ~$587

SES Monthly Detail

Month Cost
Feb 2025 $0.02
Mar 2025 $0.02
Apr 2025 $0.01
May 2025 $0.02
Jun 2025 $0.01
Jul 2025 $0.02
Aug 2025 $0.01
Sep 2025 $0.10
Oct 2025 $0.03
Nov 2025 $0.02
Dec 2025 $0.02
Jan 2026 $0.02

S3 Monthly Detail

Month Cost
Feb 2025 $1.14
Mar 2025 $1.19
Apr 2025 $1.20
May 2025 $1.23
Jun 2025 $1.25
Jul 2025 $1.28
Aug 2025 $1.30
Sep 2025 $1.31
Oct 2025 $1.33
Nov 2025 $1.34
Dec 2025 $1.35
Jan 2026 $1.31

Dead AWS Resources

Resource Details Monthly Waste
3 Elastic IPs All unassociated/idle (see below) $11.16
3 EBS volumes (50 GB) Attached to stopped instances $5.50
2 EBS snapshots (28 GB) From 2020 $0.42
2 customer KMS keys Likely from old EKS cluster (Apr 2020) $2.00
4 stopped EC2 instances No compute cost, but EBS/EIP costs $0.00
Total \(19.08/month (\)229/year)

Elastic IPs (3, ALL idle)

Allocation ID IP Address Original Purpose Status
eipalloc-012a7ebb94c516e36 108.128.196.80 NAT IP for syrfcluster-eksctl (old EKS) Unassociated since cluster deleted
eipalloc-052c538c58377b604 108.128.9.113 NAT IP for eksworkshop-eksctl (tutorial) Unassociated since cluster deleted
eipalloc-beb19f84 52.209.78.31 Attached to stopped "syrf" Windows instance Idle since Dec 2022

Stopped EC2 Instances (4, ALL stopped)

Instance ID Name Type Platform Stopped Since Launched
i-0039c6c66834f6c31 SyRF-Events t2.micro Linux Apr 2020 May 2017
i-097b9c54f05f4edab syrf t2.medium Windows Dec 2022 Sep 2017
i-0485ae1d3109b9225 eksworkshop t3.small Linux (Cloud9) ~2020 Apr 2020
i-0ddd1b18ca4decfd3 SyRF-Manager t3.small Linux (Cloud9) ~2020 Apr 2020

The "syrf" Windows instance was the original SyRF IIS server from 2017.

EBS Volumes (3, 50 GB total)

Volume ID Size Description Cost/mo
vol-0ecc30bf5becb851e 30 GB gp2 SyRF Web Server Volume (Sep 2017) $3.30
vol-04acc32e54b0b9eb7 10 GB gp2 Cloud9 SyRF-Manager (Apr 2020) $1.10
vol-000c7e3baa5b7cb41 10 GB gp2 Cloud9 eksworkshop (Apr 2020) $1.10

WorkMail

Total cost: \(16/month (\)192/year) for 4 users across 2 organizations.

Organization 1: syrf (syrf.org.uk)

User Email Usage
Chris Sena chris@syrf.org.uk Has alias no-reply@syrf.org.uk (catches SES bounce-backs)
SyRF Helpdesk helpdesk@syrf.org.uk Actively used by Gill via Outlook (IMAP/Exchange ActiveSync)

Organization 2: camarades-net (camarades-net.awsapps.com)

User Email Usage
Chris chris@camarades.net Personal
Can can@camarades.net Personal

Current State: GCP Audit

GCP Resource Inventory

Resource Details Est. Monthly Cost
GKE cluster camaradesuk, 6 x e2-standard-2, autoscaling 3-6 ~$216 (with SUD)
Boot disks 6 x 100 GB pd-standard (600 GB) ~$24
Active PVCs ~7 persistent volumes ~$3-4
Load balancer 1 regional forwarding rule (ingress-nginx) ~$18
Cloud DNS 3 zones (syrf.org.uk, camarades.net, syrf-test.co.uk) ~$1
Secret Manager 58 secrets (project: 760841397090) ~$3.48
Cloud Storage 6 buckets (see below) ~$8
Logging/Monitoring GKE integrated ~$10-40
Estimated total ~$230-260/month

Cloud Storage Buckets

Bucket Location Size Status
artifacts.camarades-net.appspot.com US 368 GB Dead -- old Container Registry from JX era
camarades-dns-backups US-CENTRAL1 0 bytes Dead -- empty
camarades-terraform-state EUROPE-WEST2 ~37 KB Active -- Terraform state
logs-camarades-a49932280ac8 US Unknown Dead -- JX bucket
reports-camarades-a49932280ac8 US Unknown Dead -- JX bucket
repository-camarades-a49932280ac8 US Unknown Dead -- JX bucket

Dead GCP Resources

Resource Details Monthly Waste
15 orphaned PVCs ~150 GB (13x8GB + 16GB + 19GB + 1GB + 10GB) ~$5
368 GB old Docker images artifacts.camarades-net.appspot.com ~$7.50
18 legacy secrets Jenkins, JX, Tekton, NewRelic, Lighthouse era ~$1
4 empty/JX buckets dns-backups, logs, reports, repository ~$0
Total ~\(13.50/month (~\)162/year)

Legacy Secrets (18 confirmed deletable)

From Jenkins/JX/Tekton pipeline era (2021-2023):

  • camarades-jenkins-docker-cfg, camarades-jenkins-release-gpg
  • camarades-jx-admin-user, camarades-jx-basic-auth-htpasswd, camarades-jx-basic-auth-user, camarades-jx-maven-settings, camarades-jx-pipeline-user
  • camarades-kubecost-grafana
  • camarades-lighthouse-hmac, camarades-lighthouse-oauth
  • camarades-loki, camarades-promtail
  • camarades-newrelic-bundle-newrelic-infrastructure-license, camarades-newrelic-bundle-newrelic-logging-config, camarades-newrelic-bundle-newrelic-prometheus-agent-license, camarades-newrelic-bundle-nri-kube-events-license
  • camarades-nexus-credentials
  • camarades-tekton-container-registry-auth

Additionally suspect: camarades-identity-server, camarades-google-test, camarades-dev-postgres-credentials, rowlingregistries-db-password-staging, snapshot-producer-mongodb

Migration Plan

Phase 0: Dead Resource Cleanup (Immediate)

No code changes. Pure infrastructure cleanup.

AWS Cleanup ($229/year savings)

  • Release 3 idle Elastic IPs
  • Terminate 4 stopped EC2 instances
  • Delete 3 EBS volumes (50 GB)
  • Delete 2 EBS snapshots (28 GB)
  • Schedule deletion of 2 customer-managed KMS keys

GCP Cleanup (~$162/year savings)

  • Delete 15 orphaned PVCs
  • Delete artifacts.camarades-net.appspot.com bucket (368 GB old Docker images)
  • Delete 18 legacy secrets
  • Delete empty/JX buckets (dns-backups, logs, reports, repository)

Total Phase 0 savings: ~$391/year

Phase 1: Email Refactor

Goal: Separate email rendering from transport, move templates into the codebase, enable MailPit for preview environments.

Current Architecture

  • SDK: SES v2 (Amazon.SimpleEmailV2) -- API calls, not SMTP
  • From address: SyRF <no-reply@syrf.org.uk>
  • Templates: 17 types stored in SES, referenced by enum names
  • Substitution: Simple {{variable}} mustache-style replacement
  • Configuration Set: syrf-emails tagged on sends but nothing consumes the events
  • Email tags: Tagged but nothing reads them
  • Bounce/complaint handling: Not implemented
  • Suppression list: Not managed
  • Dev safeguards: SESSettings.DevEmail + SESSettings.RestrictEmailToDev redirect all emails in dev

Proposed Architecture

Current:  Send*(params) -> build JSON data -> SES does rendering + delivery
Proposed: Send*(params) -> IEmailRenderer renders locally -> IEmailSender delivers

New interfaces:

  • IEmailRenderer -- renders template name + data model into subject + HTML body
  • IEmailSender -- delivers pre-rendered email. Implementations:
  • SesEmailSender -- production/staging (uses SES SendSimpleEmail with pre-rendered HTML)
  • SmtpEmailSender -- preview environments (sends to MailPit via SMTP)

Template engine: Scriban (syntax compatible with existing SES {{variable}} format, near-verbatim template migration).

MailPit:

  • Deploy in preview-infrastructure Helm chart as shared service
  • Expose UI via ingress at mail-pr-{n}.syrf.org.uk
  • Toggle via config: EmailProvider: Smtp vs Ses

Key Files

File What Changes
src/services/api/SyRF.API.Endpoint/Models/AwsEmailService.cs Refactor: extract rendering from 17 Send* methods
src/services/api/SyRF.API.Endpoint/Models/SESSettings.cs Replace with provider-agnostic config
src/services/api/SyRF.API.Endpoint/Interfaces/IAccountEmailService.cs Unchanged (consumers don't change)
src/libs/project-management/SyRF.ProjectManagement.Core/Interfaces/IProjectManagementEmailService.cs Unchanged
AdminEmailController + IEmailTemplateService Delete (templates move to git)

NuGet: Remove AWSSDK.SimpleEmailV2. Add Scriban.

Benefits

  • Templates become version-controlled, reviewable, testable
  • Preview environments get MailPit web UI for browsing sent emails
  • Dev email redirect logic (DevEmail/RestrictEmailToDev) replaced by MailPit
  • Unused SES features (Configuration Set, tags, suppression) dropped cleanly

Phase 2: S3 to Cloud Storage

Goal: Replace AWSSDK.S3 with Google.Cloud.Storage.V1, eliminate cross-cloud file storage.

Migration Map

Current (AWS) Target (GCP)
AWSSDK.S3 NuGet Google.Cloud.Storage.V1 NuGet
S3PostSigner (custom AWS4 signature) GCS V4 signed URLs (StorageClient.CreatePostPolicy)
S3FileService (upload/download/delete) GcsFileService using StorageClient
Object metadata (x-amz-meta-*) GCS custom metadata
Bucket: syrfappuploads (AWS eu-west-1) New bucket in europe-west2 (same region as GKE)
AWS credentials (aws-s3 K8s secret) Workload Identity (no credentials)

Frontend upload pattern unchanged: Presigned URL from backend, PUT from browser.

Data migration: gsutil rsync or Transfer Service for existing files.

Key benefit: Workload Identity eliminates all credential management. The aws-s3 secret and External Secrets plumbing disappears.

Phase 3: Lambda Elimination (Pub/Sub)

Goal: Replace the S3 notification Lambda with native Cloud Storage Pub/Sub notifications.

Current Chain

Frontend -> S3 (direct upload) -> S3 Event -> Lambda -> RabbitMQ -> PM Service

Proposed Chain

Frontend -> Cloud Storage (direct upload) -> Pub/Sub notification -> PM Service (pull subscription)

No Cloud Function needed. The Lambda logic (~30 lines: read object metadata, publish a message) moves into the subscribing service as a thin Pub/Sub handler.

What Gets Eliminated

  • The entire src/services/s3-notifier/ service/project
  • The Lambda build/deploy pipeline in ci-cd.yml
  • The ACK CRDs for Lambda management
  • The Lambda IAM role and permissions
  • The S3 bucket notification configuration
  • The dotnet lambda package build step
  • Preview Lambda creation/cleanup in pr-preview.yml

Phase 4: WorkMail to Zoho Mail

Goal: Migrate the one actively-used mailbox (helpdesk@syrf.org.uk) to Zoho Mail.

Options Evaluated

Option Cost/month Outlook Support Notes
Keep WorkMail (trim to 1 user) $4 Yes (Exchange ActiveSync) Still requires AWS account
Zoho Mail Lite $1 Yes (IMAP/POP) Recommended
Google Workspace Business Starter $7.20 Yes Overkill
Microsoft 365 Business Basic $6 Yes Overkill
Cloudflare Email Routing $0 No (forwarding only) Won't work for Outlook

Migration Steps

  1. Create Zoho account, add syrf.org.uk domain, verify via Cloud DNS
  2. Create helpdesk@syrf.org.uk mailbox
  3. Export/migrate emails from WorkMail (IMAP sync between old and new)
  4. Update Outlook settings (new IMAP server/credentials)
  5. Update MX records in Cloud DNS to point to Zoho
  6. Delete WorkMail organizations

Savings: $192/year (WorkMail) -> \(12/year (Zoho) = **\)180/year**

Phase 5: AWS Cleanup and Account Closure

Goal: After all migrations complete, close the AWS account entirely.

Pre-Closure Checklist

  • S3 bucket syrfappuploads fully migrated and verified
  • SES sending migrated (no-reply@syrf.org.uk still works via SES until Phase 1 complete)
  • Lambda function deleted
  • WorkMail organizations deleted
  • Route 53 hosted zones migrated to Cloud DNS (or domains transferred)
  • Domain registrations transferred (or allowed to renew elsewhere)
  • All Secrets Manager secrets deleted
  • ECR images deleted
  • Final billing cycle verified at $0
  • AWS account closed

Cost Summary

Before / After

Line Item Current Annual After Migration Savings
AWS WorkMail $192 $0 (-> Zoho $12) $180
AWS VPC (idle EIPs) $131 $0 $131
AWS EC2/EBS (dead resources) $71 $0 $71
AWS KMS $24 $0 $24
AWS Route 53 $18 $0 (Cloud DNS exists) $18
AWS S3 $15 ~$16 (Cloud Storage) -$1
AWS other (SES, ECR, Secrets Mgr) $7 $0 $7
AWS Tax (20% VAT) $97 $0 $97
AWS domain registrations $31 $31 (transfer or renew elsewhere) $0
GCP dead resources $162 $0 $162
Zoho Mail $0 $12 -$12
Totals ~$749 ~$59 ~$690

Additional Operational Savings (not costed)

Before After
Two cloud accounts + billing Single cloud (GCP)
3 AWS credential sets to rotate 0 credentials (Workload Identity)
ACK controllers running on GKE Eliminated
S3 notifier Lambda + CI/CD pipeline Eliminated
5 AWS NuGet packages 1 GCP package
Cross-region latency (GKE europe-west2, S3 eu-west-1) Same-region
Two IAM systems One IAM system
Email templates outside version control Templates in git, testable, reviewable
No email preview in dev/preview MailPit web UI

Technical Details per Phase

Phase 0: Dead Resource Cleanup

AWS: Use AWS CLI to release EIPs, terminate instances, delete volumes/snapshots, schedule KMS key deletion (30-day minimum pending period).

GCP: Use gcloud or kubectl to delete orphaned PVCs, gsutil rm -r for buckets, gcloud secrets delete for legacy secrets.

No application code changes. No deployments. Can be done in one session.

Phase 1: Email Refactor - Key Implementation Details

17 email template types (extracted from AwsEmailService.cs):

Each Send* method builds a JSON data model and calls SES SendTemplatedEmail. The data models are already fully defined in code -- every variable name and shape is known. Templates use simple {{variable}} substitution.

Migration strategy:

  1. Export all 17 SES templates via aws ses get-template
  2. Save as .scriban files in src/services/api/Templates/
  3. Create IEmailRenderer with Scriban implementation
  4. Create IEmailSender with SesEmailSender (production) and SmtpEmailSender (preview)
  5. Refactor each Send* method to use render + send pattern
  6. Delete AdminEmailController + IEmailTemplateService (template CRUD API)
  7. Remove AWSSDK.SimpleEmailV2 NuGet package

MailPit Helm values (preview-infrastructure chart):

mailpit:
  enabled: true
  image: axllent/mailpit:latest
  port: 1025      # SMTP
  uiPort: 8025    # Web UI
  ingress:
    host: mail-pr-{{ .Values.prNumber }}.syrf.org.uk

Phase 2: S3 to Cloud Storage - Key Implementation Details

Files that change:

  • S3FileService -> GcsFileService (same interface, different backend)
  • S3PostSigner -> deleted (GCS signed URLs are simpler, built into SDK)
  • Helm values: remove aws-s3 secret references, add GCS bucket config
  • External Secrets: remove camarades-aws-s3 sync

Workload Identity: GKE pods authenticate to Cloud Storage automatically via the node service account. No secrets, no rotation, no External Secrets plumbing.

Phase 3: Lambda Elimination - Key Implementation Details

Current Lambda code (syrfAppUploadS3Notifier): ~30 lines that read S3 event metadata and publish a RabbitMQ message.

Replacement: Add a Pub/Sub pull subscription handler to the PM service. Cloud Storage natively publishes to Pub/Sub topics on object creation -- no intermediary function needed.

CI/CD cleanup: Remove deploy-lambda, create-lambda-release, and version-s3-notifier jobs from ci-cd.yml. Remove pr-preview-lambda.yml workflow.

Phase 4: WorkMail to Zoho - Key Implementation Details

DNS changes (Cloud DNS, syrf-org-uk-zone):

# Remove AWS WorkMail MX records
# Add Zoho MX records:
syrf.org.uk. MX 10 mx.zoho.eu.
syrf.org.uk. MX 20 mx2.zoho.eu.
syrf.org.uk. MX 50 mx3.zoho.eu.

SES sending is unaffected -- SES uses API calls (not MX records) to send email. MX records only affect receiving. SES continues to work as the sending transport until Phase 1 replaces it.

Risks and Dependencies

Phase Dependencies

Phase 0 ──────────────> (independent, do immediately)

Phase 1 (email) ──────> Phase 5 (AWS closure)
Phase 2 (S3) ────┬────> Phase 3 (Lambda elimination)
                  └────> Phase 5 (AWS closure)
Phase 4 (WorkMail) ───> Phase 5 (AWS closure)

Phases 1, 2, and 4 are independently deployable -- they can be done in any order or in parallel.

Phase 3 depends on Phase 2 (S3 must be on Cloud Storage before Lambda can be replaced with Pub/Sub).

Phase 5 depends on all others being complete.

Risks

Risk Impact Mitigation
SES email deliverability is mature; new transport may have issues Emails go to spam Keep SES as transport (Phase 1 only changes rendering, not transport). SES can remain until proven unnecessary
S3 presigned URL format differs from GCS signed URL format Frontend upload breaks Test thoroughly in preview environments. Frontend only needs URL + method; format details are abstracted
Pub/Sub message format differs from RabbitMQ PM service handler needs adaptation The Lambda already translates S3 events to RabbitMQ messages -- same translation moves into PM service
WorkMail -> Zoho transition has downtime for helpdesk Gill can't receive email briefly Schedule during low-traffic period. MX propagation typically takes < 1 hour
AWS account closure is irreversible Can't go back Verify all data migrated, keep final S3 backup, wait 90 days before closure
Domain registrations may be locked to AWS Domains unreachable if account closes Transfer domains to another registrar before closure

Out of Scope

  • MongoDB isolation (staging/preview databases) -- tracked separately in MongoDB Testing Strategy
  • GKE cluster right-sizing
  • Switching away from SES for sending (Phase 1 keeps SES as transport; only rendering moves locally)

Data Sources

All cost figures and resource inventories in this document were gathered on 2026-02-14 from:

  • AWS Cost Explorer API (12-month lookback: Feb 2025 - Jan 2026)
  • AWS resource APIs (EC2 describe-instances, describe-addresses, describe-volumes, describe-snapshots, describe-key, WorkMail list-organizations/list-users)
  • GCP gcloud CLI (compute disks, secrets, storage buckets)
  • Source code analysis (AwsEmailService.cs, S3FileService, service interfaces)