Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 83 additions & 7 deletions internal/mirror/README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,9 @@ d8 mirror pull <images-bundle-path> [flags]

| Flag | Description |
|------|-------------|
| `--since-version` | Minimal Deckhouse release to pull. Ignored if above current Rock Solid release. Conflicts with `--deckhouse-tag` |
| `--deckhouse-tag` | Specific Deckhouse build tag to pull. Conflicts with `--since-version`. If the registry contains a release channel image for the specified tag, all release channels in the bundle will point to it |
| `--since-version` | Minimal Deckhouse release to pull (lower bound, inclusive). Ignored if above current Rock Solid release. Conflicts with `--deckhouse-tag` and `--include-platform` |
| `--include-platform` | Select platform releases by semver constraint (e.g. `">=1.64 <=1.68"`). Uses the same constraint dialect as `--include-module`. Conflicts with `--deckhouse-tag` and `--since-version`. See [Platform Version Filtering](#platform-version-filtering) |
| `--deckhouse-tag` | Specific Deckhouse build tag to pull. Conflicts with `--since-version` and `--include-platform`. If the registry contains a release channel image for the specified tag, all release channels in the bundle will point to it |

#### Module Filtering

Expand Down Expand Up @@ -91,6 +92,53 @@ d8 mirror pull <images-bundle-path> [flags]
| `--insecure` | Interact with registries over HTTP |
| `--tmp-dir` | Path to temporary directory for processing. Ensure sufficient disk space for the entire bundle |

### Platform Version Filtering

The `--include-platform` flag accepts a semver constraint expression to pull only a specific window of platform releases. It uses exactly the same constraint dialect as the version part of `--include-module`.

#### When to use

| Goal | Recommended flag |
|------|-----------------|
| Pull all releases from a given minimum upward | `--since-version 1.64.0` |
| Pull a bounded range (e.g. for an incremental update) | `--include-platform ">=1.64 <=1.68"` |
| Pull every release of a single minor | `--include-platform "~1.65.0"` |
| Pin a single exact release (same as `--deckhouse-tag`) | `--include-platform "=v1.65.3"` |

#### Constraint syntax

The same rules apply as for `--include-module` version constraints (see [Module Filtering](#module-filtering)):

- **Latest-patch-per-minor collapsing** — for semver ranges, only the highest patch in each `(major, minor)` bucket is pulled. Inclusive boundary operators (`>=`, `<=`) always preserve the named version as an anchor even when a newer patch exists.
- **Channel filtering** — release channels whose current snapshot points outside the constraint window are dropped from the bundle so no entry references an image that was not downloaded.
- **Exact-tag form (`=`)** — operationally identical to `--deckhouse-tag`: a single tag is pulled and all default release channels in the bundle are pointed at it.

#### Examples

```bash
# Pull releases v1.64.x through v1.68.x (incremental update scenario)
d8 mirror pull /tmp/d8-bundle \
--license $LICENSE_TOKEN \
--include-platform ">=1.64 <=1.68"

# Pull only the latest patch of v1.65.x
d8 mirror pull /tmp/d8-bundle \
--license $LICENSE_TOKEN \
--include-platform "~1.65.0"

# Pull every minor from v1.65 onward that the registry exposes
d8 mirror pull /tmp/d8-bundle \
--license $LICENSE_TOKEN \
--include-platform "^1.65.0"

# Pin an exact release (all channels point to v1.65.3)
d8 mirror pull /tmp/d8-bundle \
--license $LICENSE_TOKEN \
--include-platform "=v1.65.3"
```

---

### Module Filtering

The `--include-module` and `--exclude-module` flags support version constraints for fine-grained control over which module versions to include.
Expand Down Expand Up @@ -166,6 +214,11 @@ module-name[@version-constraint]
# Download all modules
d8 mirror pull /tmp/d8-bundle --license $LICENSE_TOKEN

# Download a bounded range of platform releases (e.g. v1.64.x – v1.68.x)
d8 mirror pull /tmp/d8-bundle \
--license $LICENSE_TOKEN \
--include-platform ">=1.64 <=1.68"

# Download only specific modules
d8 mirror pull /tmp/d8-bundle \
--license $LICENSE_TOKEN \
Expand All @@ -177,6 +230,12 @@ d8 mirror pull /tmp/d8-bundle \
--license $LICENSE_TOKEN \
--include-module neuvector@^1.2.0

# Combine platform range with a module filter
d8 mirror pull /tmp/d8-bundle \
--license $LICENSE_TOKEN \
--include-platform ">=1.64 <=1.68" \
--include-module neuvector

# Exclude specific modules
d8 mirror pull /tmp/d8-bundle \
--license $LICENSE_TOKEN \
Expand Down Expand Up @@ -301,12 +360,20 @@ The same environment variables used by `d8 mirror pull` are also supported:

1. **On a machine with internet access**, download the bundle:
```bash
# Full pull from a minimum version
d8 mirror pull /tmp/deckhouse-bundle \
--license $DECKHOUSE_LICENSE_TOKEN \
--since-version 1.59.0 \
--include-module prometheus \
--include-module ingress-nginx \
--images-bundle-chunk-size 10

# Or pull only a specific version window for an incremental update
d8 mirror pull /tmp/deckhouse-bundle \
--license $DECKHOUSE_LICENSE_TOKEN \
--include-platform ">=1.64 <=1.68" \
--include-module prometheus \
--include-module ingress-nginx
```

2. **Transfer the bundle** to the air-gapped environment (USB drive, secure file transfer, etc.)
Expand Down Expand Up @@ -394,10 +461,11 @@ d8 mirror pull /tmp/bundle --license $LICENSE_TOKEN

### Version Management

1. **Use `--since-version`:** To pull only newer releases, reducing bundle size
2. **Specific Versions:** Use `--deckhouse-tag` for controlled, predictable deployments
3. **Module Versioning:** Leverage semver constraints for flexible module version management
4. **Release Channels:** Understand that pulling includes all configured release channels (alpha, beta, early-access, stable, rock-solid)
1. **Use `--since-version`:** To pull all releases from a given minimum upward, reducing bundle size relative to a full pull
2. **Use `--include-platform`:** To pull a bounded window of releases (e.g. `">=1.64 <=1.68"`) — useful for staged incremental updates where you want to move up a few minors at a time without pulling the entire channel history
3. **Specific Versions:** Use `--deckhouse-tag` (or `--include-platform "=vX.Y.Z"`) for fully controlled, single-release deployments
4. **Module Versioning:** Leverage semver constraints in `--include-module` for flexible module version management; `--include-platform` speaks the same dialect
5. **Release Channels:** Pulling respects all configured release channels (alpha, beta, early-access, stable, rock-solid); channels whose snapshot falls outside an `--include-platform` constraint are automatically excluded from the bundle

### Bundle Management

Expand All @@ -418,11 +486,19 @@ The mirror system handles three primary component types:
2. **Modules:** Optional Deckhouse modules with versioned releases
3. **Security Databases:** Vulnerability scanning databases and related data

### Platform Release Discovery

By default, the platform service discovers releases between the current `rock-solid` channel version (lower bound) and the current `alpha` channel version (upper bound), keeping the highest patch per `(major, minor)`.

`--since-version` raises the lower bound above `rock-solid` when the user wants to skip older minors.

`--include-platform` replaces this window with a user-supplied semver constraint, applying the same latest-patch-per-minor and inclusive-anchor rules as module version filtering. Channel snapshots outside the constraint are pruned so the bundle stays internally consistent.

### Module Filtering Logic

- **Whitelist Mode:** Only specified modules are included
- **Blacklist Mode:** All modules except specified ones are included (default)
- **Version Constraints:** Semver-based filtering for granular control
- **Version Constraints:** Semver-based filtering for granular control (same dialect as `--include-platform`)
- **Release Channels:** Each module can have multiple release channel versions

### Storage Layout
Expand Down
32 changes: 31 additions & 1 deletion internal/mirror/cmd/pull/flags/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import (

"github.com/Masterminds/semver/v3"
"github.com/spf13/pflag"

"github.com/deckhouse/deckhouse-cli/internal/mirror/modules"
)

const (
Expand All @@ -46,6 +48,9 @@ var (
SinceVersionString string
SinceVersion *semver.Version

PlatformConstraintString string
PlatformConstraint modules.VersionConstraint

DeckhouseTag string
InstallerTag string

Expand Down Expand Up @@ -106,11 +111,36 @@ func AddFlags(flagSet *pflag.FlagSet) {
"",
"Minimal Deckhouse release to pull. Ignored if above current Rock Solid release. Conflicts with --deckhouse-tag.",
)
flagSet.StringVar(
&PlatformConstraintString,
"include-platform",
"",
`Select platform releases to download by a semver constraint expression, using the same dialect as --include-module's version part.
Conflicts with --since-version and --deckhouse-tag.

Semver constraints (caret, tilde, range) keep only the highest patch in each (major, minor) series, mirroring the release-discovery rules used for full pulls.
Versions explicitly named with an inclusive boundary operator (>= or <=) are always preserved — that boundary is part of the user's request and must round-trip even when a newer patch exists in the same minor.
Use the exact-tag form (=) when you need to pin a specific tag, optionally propagating it to a release channel via the +channel suffix.

Examples (available platform versions: v1.63.x, v1.64.x, v1.65.x, v1.66.x, v1.67.x, v1.68.x, v1.69.x, v1.70.x, v1.71.x):

--include-platform ">=1.64 <=1.68" → bounded range: latest patch per minor in v1.64..v1.68, anchors v1.64.0 and v1.68.0 always preserved if present in the registry.

--include-platform "~1.65.0" → semver ~ constraint (>=1.65.0 <1.66.0): latest v1.65.x patch only.

--include-platform "^1.65.0" → semver ^ constraint (>=1.65.0 <2.0.0): latest patch per minor starting at v1.65.x.

--include-platform "1.65.0" → implicit caret (^1.65.0): same as above; shorthand kept for parity with --include-module.

--include-platform "=v1.65.3" → exact-tag pin: only v1.65.3 is pulled and propagated to all default release channels, just like --deckhouse-tag.

--include-platform "=v1.65.3+stable" → exact-tag pin with channel suffix: only v1.65.3 is pulled (channel propagation matches --deckhouse-tag).`,
)
flagSet.StringVar(
&DeckhouseTag,
"deckhouse-tag",
"",
"Specific Deckhouse build tag to pull. Conflicts with --since-version. If registry contains release channel image for specified tag, all release channels in the bundle will be pointed to it.",
"Specific Deckhouse build tag to pull. Conflicts with --since-version and --include-platform. If registry contains release channel image for specified tag, all release channels in the bundle will be pointed to it.",
)
flagSet.StringVar(
&InstallerTag,
Expand Down
29 changes: 16 additions & 13 deletions internal/mirror/cmd/pull/pull.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ func NewCommand() *cobra.Command {

pullflags.AddFlags(pullCmd.Flags())
pullCmd.MarkFlagsMutuallyExclusive("include-module", "exclude-module")
pullCmd.MarkFlagsMutuallyExclusive("include-platform", "deckhouse-tag")
pullCmd.MarkFlagsMutuallyExclusive("include-platform", "since-version")
pullflags.ParseEnvironmentVariables()

return pullCmd
Expand Down Expand Up @@ -281,19 +283,20 @@ func (p *Puller) Execute(ctx context.Context) error {
pullflags.TempDir,
pullflags.DeckhouseTag,
&mirror.PullServiceOptions{
SkipPlatform: pullflags.NoPlatform,
SkipSecurity: pullflags.NoSecurityDB,
SkipModules: pullflags.NoModules,
SkipVexImages: pullflags.SkipVexImages,
SkipInstaller: pullflags.NoInstaller,
InstallerTag: pullflags.InstallerTag,
OnlyExtraImages: pullflags.OnlyExtraImages,
IgnoreSuspend: pullflags.IgnoreSuspend,
ModuleFilter: filter,
BundleDir: pullflags.ImagesBundlePath,
BundleChunkSize: pullflags.ImagesBundleChunkSizeGB * 1000 * 1000 * 1000,
Timeout: pullflags.MirrorTimeout,
DryRun: pullflags.DryRun,
SkipPlatform: pullflags.NoPlatform,
SkipSecurity: pullflags.NoSecurityDB,
SkipModules: pullflags.NoModules,
SkipVexImages: pullflags.SkipVexImages,
SkipInstaller: pullflags.NoInstaller,
InstallerTag: pullflags.InstallerTag,
OnlyExtraImages: pullflags.OnlyExtraImages,
IgnoreSuspend: pullflags.IgnoreSuspend,
PlatformConstraint: pullflags.PlatformConstraint,
ModuleFilter: filter,
BundleDir: pullflags.ImagesBundlePath,
BundleChunkSize: pullflags.ImagesBundleChunkSizeGB * 1000 * 1000 * 1000,
Timeout: pullflags.MirrorTimeout,
DryRun: pullflags.DryRun,
},
logger.Named("pull"),
p.logger,
Expand Down
14 changes: 14 additions & 0 deletions internal/mirror/cmd/pull/validation.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import (
"github.com/spf13/cobra"

pullflags "github.com/deckhouse/deckhouse-cli/internal/mirror/cmd/pull/flags"
"github.com/deckhouse/deckhouse-cli/internal/mirror/modules"
)

func parseAndValidateParameters(_ *cobra.Command, args []string) error {
Expand Down Expand Up @@ -117,6 +118,12 @@ func parseAndValidateVersionFlags() error {
if pullflags.SinceVersionString != "" && pullflags.DeckhouseTag != "" {
return errors.New("Using both --deckhouse-tag and --since-version at the same time is ambiguous")
}
if pullflags.PlatformConstraintString != "" && pullflags.DeckhouseTag != "" {
return errors.New("Using both --deckhouse-tag and --include-platform at the same time is ambiguous")
}
if pullflags.PlatformConstraintString != "" && pullflags.SinceVersionString != "" {
return errors.New("Using both --since-version and --include-platform at the same time is ambiguous: --include-platform already expresses a lower bound (e.g. \">=1.64\")")
}

var err error
if pullflags.SinceVersionString != "" {
Expand All @@ -126,6 +133,13 @@ func parseAndValidateVersionFlags() error {
}
}

if pullflags.PlatformConstraintString != "" {
pullflags.PlatformConstraint, err = modules.ParseVersionConstraint(pullflags.PlatformConstraintString)
if err != nil {
return fmt.Errorf("Parse --include-platform constraint: %w", err)
}
}

return nil
}

Expand Down
60 changes: 55 additions & 5 deletions internal/mirror/cmd/pull/validation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -331,11 +331,12 @@ func TestValidationValidateImagesBundlePathArg(t *testing.T) {

func TestValidationParseAndValidateVersionFlags(t *testing.T) {
tests := []struct {
name string
sinceVersionString string
deckhouseTag string
expectError bool
errorMsg string
name string
sinceVersionString string
platformConstraintString string
deckhouseTag string
expectError bool
errorMsg string
}{
{
name: "no version flags",
Expand Down Expand Up @@ -392,23 +393,69 @@ func TestValidationParseAndValidateVersionFlags(t *testing.T) {
deckhouseTag: "",
expectError: true,
},
{
name: "valid include-platform range",
platformConstraintString: ">=1.64 <=1.68",
expectError: false,
},
{
name: "valid include-platform caret shorthand",
platformConstraintString: "1.65.0",
expectError: false,
},
{
name: "valid include-platform exact tag",
platformConstraintString: "=v1.65.3",
expectError: false,
},
{
name: "valid include-platform exact tag with channel suffix",
platformConstraintString: "=v1.65.3+stable",
expectError: false,
},
{
name: "include-platform conflicts with deckhouse-tag",
platformConstraintString: ">=1.64 <=1.68",
deckhouseTag: "v1.65.0",
expectError: true,
errorMsg: "ambiguous",
},
{
name: "include-platform conflicts with since-version",
platformConstraintString: ">=1.64 <=1.68",
sinceVersionString: "1.64.0",
expectError: true,
errorMsg: "ambiguous",
},
{
name: "invalid include-platform constraint",
platformConstraintString: "not-a-constraint",
expectError: true,
errorMsg: "Parse --include-platform constraint",
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
originalSinceVersionString := pullflags.SinceVersionString
originalDeckhouseTag := pullflags.DeckhouseTag
originalSinceVersion := pullflags.SinceVersion
originalPlatformConstraintString := pullflags.PlatformConstraintString
originalPlatformConstraint := pullflags.PlatformConstraint

defer func() {
pullflags.SinceVersionString = originalSinceVersionString
pullflags.DeckhouseTag = originalDeckhouseTag
pullflags.SinceVersion = originalSinceVersion
pullflags.PlatformConstraintString = originalPlatformConstraintString
pullflags.PlatformConstraint = originalPlatformConstraint
}()

pullflags.SinceVersionString = tt.sinceVersionString
pullflags.DeckhouseTag = tt.deckhouseTag
pullflags.SinceVersion = nil
pullflags.PlatformConstraintString = tt.platformConstraintString
pullflags.PlatformConstraint = nil

err := parseAndValidateVersionFlags()

Expand All @@ -423,6 +470,9 @@ func TestValidationParseAndValidateVersionFlags(t *testing.T) {
assert.NotNil(t, pullflags.SinceVersion)
assert.Equal(t, tt.sinceVersionString, pullflags.SinceVersion.String())
}
if tt.platformConstraintString != "" {
assert.NotNil(t, pullflags.PlatformConstraint, "constraint must be parsed")
}
}
})
}
Expand Down
Loading
Loading