Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
version: 2
updates:
- package-ecosystem: "gomod"
directory: "/fleet-argocd-plugin"
schedule:
interval: "weekly"
labels:
- "dependencies"

- package-ecosystem: "gomod"
directory: "/argocd-clusterprofile-syncer"
schedule:
interval: "weekly"
labels:
- "dependencies"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
labels:
- "dependencies"
20 changes: 20 additions & 0 deletions .github/workflows/action-scanning.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: "Actions Workflow Security Scan"

on:
pull_request:
branches:
- main
paths:
- ".github/workflows/**"

permissions:
contents: read

jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5

- name: Ensure actions are pinned to SHAs
uses: zgosalvez/github-actions-ensure-sha-pinned-actions@40ba2d51b6b6d8695f2b6bd74e785172d4f8d00f # v3.0.17
12 changes: 8 additions & 4 deletions .github/workflows/golangci-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,23 +27,27 @@ jobs:
- argocd-clusterprofile-syncer
- fleet-argocd-plugin
steps:
- uses: actions/checkout@v5
- uses: dorny/paths-filter@v3
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
- uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3
id: changes
with:
filters: |
src:
- ${{ matrix.folder }}/**
- ".github/workflows/golangci-lint.yml"
- if: steps.changes.outputs.src == 'true'
uses: actions/setup-go@v5
uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5
with:
go-version-file: ${{ matrix.folder }}/go.mod
cache-dependency-path: ${{ matrix.folder }}/go.sum
- if: steps.changes.outputs.src == 'true'
name: golangci-lint
uses: golangci/golangci-lint-action@v8
uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8
with:
version: latest
working-directory: ${{ matrix.folder }}
args: --timeout=5m
- if: steps.changes.outputs.src == 'true'
name: go-test
working-directory: ${{ matrix.folder }}
run: go test -v -race -count=1 ./...
8 changes: 4 additions & 4 deletions .github/workflows/helm-charts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,22 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
with:
fetch-depth: 0

- name: Set up Helm
uses: azure/setup-helm@v4.3.1
uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4.3.1
with:
version: v3.17.0

- uses: actions/setup-python@v5.6.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: '3.x'
check-latest: true

- name: Set up chart-testing
uses: helm/chart-testing-action@v2.7.0
uses: helm/chart-testing-action@cf48dbf901ed202ae2c5aee26422dd6dfdf41e47 # v2.7.0

- name: Run chart-linting
run: ct lint --target-branch ${{ github.event.repository.default_branch }} --chart-dirs fleet-charts --validate-maintainers=false --helm-lint-extra-args '--set hf_api_token=NONE'
4 changes: 2 additions & 2 deletions .github/workflows/helm-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
with:
fetch-depth: 0

Expand All @@ -22,7 +22,7 @@ jobs:
git config user.email "$GITHUB_ACTOR@users.noreply.github.com"

- name: Run chart-releaser
uses: helm/chart-releaser-action@v1.7.0
uses: helm/chart-releaser-action@a0d2dc62c5e491af8ef6ba64a2e02bcf3fb33aa1 # v1.7.0
with:
charts_dir: fleet-charts
env:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/terraform.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v5
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
name: Checkout source code

- uses: terraform-linters/setup-tflint@v5
- uses: terraform-linters/setup-tflint@6770de8d186019a148cc2b144e8bfa627f7d4aa8 # v5
name: Setup TFLint
with:
tflint_version: latest
Expand Down
61 changes: 61 additions & 0 deletions README-PROTECTION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Fleet API Protection

Adds protection against transient Fleet API issues that cause application deletions.

## Problem

During Redis restarts or maintenance, Fleet API occasionally returns incomplete responses.
The plugin didn't validate or retry, causing ArgoCD to delete applications.

**Incident 2026-01-30:**
- API returned 6 bindings instead of 12
- 3 applications deleted
- 2-12 minutes downtime each

## Solution

Three-layer protection:

1. **Detection** - Identifies transient issues by pattern:
- Oscillation: count changes 2+ times in 10 minutes (12→6→12)
- Sudden drop: >30% decrease

2. **Retry** - 3 attempts with exponential backoff (2s, 4s, 8s)

3. **Cache** - Falls back to last known good response (60 min TTL)

## Configuration

Environment variables with defaults:

```bash
MAX_API_RETRIES=3
RETRY_BASE_DELAY_SECONDS=2
CACHE_MAX_AGE_MINUTES=60
DETECTION_WINDOW_MINUTES=10
OSCILLATION_THRESHOLD=2
DROP_THRESHOLD_PERCENT=30
```

## Testing

```bash
cd fleet-argocd-plugin
go test ./protection/...
go test ./fleetclient/...
```

All tests pass. Incident replay confirms 2026-01-30 pattern is detected.

## Changes

**Modified files (2):**
- `fleetclient/fleetclient.go` - Added protection wrapper
- `main.go` - Added config loading

**New files (4):**
- `protection/detector.go` - Transient issue detection
- `protection/cache.go` - Response caching
- `protection/*_test.go` - Test coverage

Zero breaking changes. Protection is transparent in normal operation.
1 change: 1 addition & 0 deletions fleet-argocd-plugin/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
fleet-sync
9 changes: 4 additions & 5 deletions fleet-argocd-plugin/applicationset-demo.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@ spec:
scopeID: "$TEAM_ID"
requeueAfterSeconds: 10
syncPolicy:
applicationsSync: sync
preserveResourcesOnDeletion: false
# IMPORTANT: Use create-update to prevent app deletion when Fleet API returns partial data
applicationsSync: create-update
# Preserve K8s resources even if Application is deleted
preserveResourcesOnDeletion: true
template:
metadata:
name: '{{name}}-webserver'
Expand All @@ -28,6 +30,3 @@ spec:
path: guestbook
repoURL: https://github.com/argoproj/argocd-example-apps.git
targetRevision: HEAD
syncPolicy:
# The controller will delete Applications when the ApplicationSet is deleted.
preserveResourcesOnDeletion: false
Loading
Loading