Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions internal/mirror/README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -114,21 +114,30 @@ module-name[@version-constraint]
--include-module mymodule@^1.3.0 # equivalent
```
- Example: For available versions `v1.0.0, v1.1.0, v1.2.0, v1.3.0, v1.3.3, v1.4.1`
- Constraint `^1.3.0` includes: `v1.3.0, v1.3.3, v1.4.1`
- Constraint `^1.3.0` includes: `v1.3.3, v1.4.1`
- Only the highest patch in each `(major, minor)` series is kept, matching the platform release discovery rule. Use the exact-tag form (`=`) to pin a specific older patch.
- Versions currently pinned by release channels (alpha, beta, early-access, stable, rock-solid, lts) are pulled in addition, regardless of the patch filter.
- Also pulls current versions from all release channels

3. **Semver tilde constraint (~)** - Patch-level changes only
```bash
--include-module mymodule@~1.3.0
```
- Example: For available versions `v1.0.0, v1.1.0, v1.2.0, v1.3.0, v1.3.3, v1.4.1`
- Constraint `~1.3.0` (equivalent to `>=1.3.0 <1.4.0`) includes: `v1.3.0, v1.3.3`
- Constraint `~1.3.0` (equivalent to `>=1.3.0 <1.4.0`) includes: `v1.3.3` (latest patch of 1.3.x)
- Also pulls current versions from all release channels

4. **Semver range constraint** - Explicit version range
```bash
--include-module mymodule@">=1.1.0 <1.3.0"
```
- Example: For available versions `v1.0.0, v1.1.0, v1.1.2, v1.2.0, v1.2.4, v1.3.0`
- Constraint `>=1.1.0 <1.3.0` includes: `v1.1.0, v1.1.2, v1.2.4`
- The latest patch in each `(major, minor)` is kept; **inclusive boundary operators (`>=`, `<=`) always preserve the named version**, even when a newer patch exists in the same minor. `>=1.1.0` literally means "v1.1.0 OR newer" — the equality is part of the operator.
- Strict bounds (`>`, `<`) do NOT preserve the named version; they exclude it by definition.
- Caret (`^`) and tilde (`~`) are syntactic shorthand for a range: their lower bounds are NOT anchors and the latest-patch-per-minor filter applies in full.
- Anchors that are not present in the registry are silently skipped — this command never invents a tag.
- Also pulls current versions from all release channels

5. **Exact tag match (=)** - Single specific version
```bash
Expand Down
14 changes: 11 additions & 3 deletions internal/mirror/cmd/pull/flags/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -125,12 +125,20 @@ func AddFlags(flagSet *pflag.FlagSet) {
nil,
`Whitelist specific modules for downloading. Use one flag per each module. Disables blacklisting by --exclude-module."

Semver constraints (caret, tilde, range) keep only the highest patch in each (major, minor) series, mirroring how platform releases are discovered.
Versions explicitly named with an inclusive boundary operator (>= or <=) are always preserved — that boundary is part of the user's request and must round-trip even when a newer patch exists in the same minor.
Use the exact-tag form (=) when you need a specific older patch unconditionally.

Example:
Available versions for <module-name>: v1.0.0, v1.1.0, v1.2.0, v1.3.0, v1.3.3, v1.4.1
Available versions for <module-name>: v1.0.0, v1.1.0, v1.2.0, v1.3.0, v1.3.3, v1.4.0, v1.4.1

[email protected] → semver ^ constraint (^1.3.0): keep latest patch per minor — includes v1.3.3 (1.3.x) and v1.4.1 (1.4.x). Versions currently pinned by release channels are pulled in addition.

module-name@~1.3.0 → semver ~ constraint (>=1.3.0 <1.4.0): keep latest patch per minor in range — includes v1.3.3. Versions currently pinned by release channels are pulled in addition.

[email protected]semver ^ constraint (^1.3.0): include v1.3.0, v1.3.3, v1.4.1. In addition pulls current versions from release channels
module-name@>=1.3.0 → range constraint with explicit >= anchor: keep latest patch per minor AND the named anchor v1.3.0 — includes v1.3.0 (anchor), v1.3.3 (1.3.x latest), v1.4.1 (1.4.x latest).

module-name@~1.3.0 → semver ~ constraint (>=1.3.0 <1.4.0): include only v1.3.0, v1.3.3. In addition pulls current versions from release channels
module-name@>=1.3.0 <=1.4.0 → both anchors honoured: includes v1.3.0, v1.3.3, v1.4.0; v1.4.1 is excluded by the upper bound.

module-name@=v1.3.0 → exact tag match: include only v1.3.0 and publish it to all release channels (alpha, beta, early-access, stable, rock-solid).

Expand Down
76 changes: 75 additions & 1 deletion internal/mirror/modules/constraints.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package modules

import (
"fmt"
"regexp"

"github.com/Masterminds/semver/v3"
)
Expand All @@ -30,14 +31,87 @@ type VersionConstraint interface {

type SemanticVersionConstraint struct {
constraint *semver.Constraints
// anchors are versions explicitly named with an inclusive boundary
// operator (>=, <=) in the user's constraint string. These versions
// must round-trip through the latest-patch-per-minor filter — the user
// has named them by hand, so dropping them in favor of a newer patch
// in the same (major, minor) bucket would silently override an
// explicit user choice.
//
// Only `>=`, `<=` (and their reversed forms `=>`, `=<`) contribute
// anchors. Caret (`^`), tilde (`~`), and implicit constraints do not:
// those operators are shorthand for a range and the patch filter is
// free to collapse same-minor patches inside them.
anchors []*semver.Version
}

// anchorOpRegex captures version literals that follow an inclusive boundary
// operator. Exclusive bounds (>, <) and shorthand operators (^, ~) are
// intentionally excluded — see the anchors field doc above for the rationale.
//
// The version capture is deliberately permissive (any non-space, non-comma
// run); the result is re-validated through semver.NewVersion before being
// stored.
var anchorOpRegex = regexp.MustCompile(`(?:>=|<=|=>|=<)\s*([^\s,]+)`)

func NewSemanticVersionConstraint(c string) (*SemanticVersionConstraint, error) {
constraint, err := semver.NewConstraint(c)
if err != nil {
return nil, fmt.Errorf("invalid semantic version constraint %q: %w", c, err)
}
return &SemanticVersionConstraint{constraint: constraint}, nil

anchors, err := extractInclusiveAnchors(c)
if err != nil {
return nil, fmt.Errorf("invalid semantic version constraint %q: %w", c, err)
}

return &SemanticVersionConstraint{
constraint: constraint,
anchors: anchors,
}, nil
}

// Anchors returns the versions explicitly named with an inclusive boundary
// operator (>=, <=). Callers must re-check membership against the constraint
// itself (Match) before consuming an anchor: an anchor that fails Match means
// the user wrote a contradictory constraint (e.g. `>=2.0.0 <1.0.0`) and we
// will not silently widen the range.
func (s *SemanticVersionConstraint) Anchors() []*semver.Version {
return s.anchors
}

// extractInclusiveAnchors finds every >=X / <=X literal in the constraint
// string and parses X with semver.NewVersion. Duplicates are removed.
// The returned slice is nil when no inclusive boundary literals are present.
func extractInclusiveAnchors(constraintStr string) ([]*semver.Version, error) {
matches := anchorOpRegex.FindAllStringSubmatch(constraintStr, -1)
if len(matches) == 0 {
return nil, nil
}

seen := make(map[string]struct{}, len(matches))
out := make([]*semver.Version, 0, len(matches))
for _, m := range matches {
if len(m) < 2 {
continue
}
raw := m[1]
if _, dup := seen[raw]; dup {
continue
}
seen[raw] = struct{}{}

v, err := semver.NewVersion(raw)
if err != nil {
// The constraint already passed semver.NewConstraint, so this
// only fires on programming errors in the regex (we'd extract
// something that the constraint parser had accepted but the
// version parser hadn't). Surface it as a real error.
return nil, fmt.Errorf("anchor %q not a valid semver: %w", raw, err)
}
out = append(out, v)
}
return out, nil
}

func (s *SemanticVersionConstraint) HasChannelAlias() bool {
Expand Down
112 changes: 108 additions & 4 deletions internal/mirror/modules/filter.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ import (
"fmt"
"strings"

"github.com/Masterminds/semver/v3"

"github.com/deckhouse/deckhouse-cli/internal"
"github.com/deckhouse/deckhouse-cli/pkg/libmirror/operations/params"
)
Expand Down Expand Up @@ -155,6 +157,27 @@ func (f *Filter) ShouldMirrorReleaseChannels(moduleName string) bool {

// VersionsToMirror resolves module constraints from --include-module into concrete tags to pull.
// Returns nil when no explicit version tags should be added for this module.
//
// For semver constraints (caret, tilde, ranges) only the highest patch in each
// (major, minor) bucket that satisfies the constraint is returned. This mirrors
// the platform-level discovery rule (filterOnlyLatestPatches in
// internal/mirror/platform/platform.go) and avoids pulling N redundant patches
// per minor when the user wires a single module pin like `[email protected]`.
//
// Anchor exception: versions named with an inclusive boundary operator (`>=`
// or `<=`) are always restored to the result if they exist in the registry.
// `>=1.40.0` literally encodes "v1.40.0 OR newer" — the user named v1.40.0
// by hand and we MUST honour that even when a newer patch (v1.40.1) exists
// in the same minor. Caret (`^`) and tilde (`~`) are syntactic shorthand
// for a range; their lower bounds are NOT anchors.
//
// Exact-tag constraints (`module@=vX.Y.Z`) bypass this filter — when the user
// asks for a specific tag they get exactly that tag.
//
// Channel snapshot versions (alpha/beta/early-access/stable/rock-solid) are
// merged into the pull list outside this method, so an older patch that a
// channel still points at remains reachable through the channel snapshot even
// when filterOnlyLatestPatches drops it from the constraint set.
func (f *Filter) VersionsToMirror(mod *Module) []string {
constraint, hasConstraint := f.modules[mod.Name]
if !hasConstraint {
Expand All @@ -169,17 +192,98 @@ func (f *Filter) VersionsToMirror(mod *Module) []string {
return []string{exact.Tag()}
}

semver, isSemver := constraint.(*SemanticVersionConstraint)
semverConstraint, isSemver := constraint.(*SemanticVersionConstraint)
if !isSemver {
return nil
}

var tags []string
matched := make([]*semver.Version, 0)
for _, v := range mod.Versions() {
if semver.Match(v) {
tags = append(tags, "v"+v.String())
if semverConstraint.Match(v) {
matched = append(matched, v)
}
}

selected := filterOnlyLatestPatches(matched)
selected = restoreInclusiveAnchors(selected, matched, semverConstraint.Anchors())

tags := make([]string, 0, len(selected))
for _, v := range selected {
tags = append(tags, "v"+v.String())
}
return tags
}

// restoreInclusiveAnchors re-introduces any anchor versions (named via >=/<=
// in the user's constraint string) that were dropped by filterOnlyLatestPatches.
//
// An anchor is only restored when it is present in `available` (i.e. the
// constraint actually matched it against the registry's tag list). This guards
// against two failure modes:
// - silently widening the constraint by appending versions the user
// contradicted in another sub-constraint;
// - emitting tags the registry doesn't have.
//
// The function preserves the order of `selected` and appends restored anchors
// at the end. Duplicates are de-duplicated by version equality.
func restoreInclusiveAnchors(selected, available []*semver.Version, anchors []*semver.Version) []*semver.Version {
if len(anchors) == 0 {
return selected
}

availableByKey := make(map[string]*semver.Version, len(available))
for _, v := range available {
if v == nil {
continue
}
availableByKey[v.String()] = v
}

selectedKeys := make(map[string]struct{}, len(selected))
for _, v := range selected {
selectedKeys[v.String()] = struct{}{}
}

for _, anchor := range anchors {
key := anchor.String()
if _, already := selectedKeys[key]; already {
continue
}
registryVersion, isAvailable := availableByKey[key]
if !isAvailable {
continue
}
selected = append(selected, registryVersion)
selectedKeys[key] = struct{}{}
}
return selected
}

// filterOnlyLatestPatches keeps a single highest-patch version for every
// (major, minor) bucket. It is the modules-package counterpart of the same
// helper used by platform release discovery, kept private to avoid an
// import cycle and to make it easy for both packages to evolve independently.
func filterOnlyLatestPatches(versions []*semver.Version) []*semver.Version {
type majorMinor struct {
major uint64
minor uint64
}

latest := make(map[majorMinor]*semver.Version, len(versions))
for _, v := range versions {
if v == nil {
continue
}
key := majorMinor{major: v.Major(), minor: v.Minor()}
current, ok := latest[key]
if !ok || v.GreaterThan(current) {
latest[key] = v
}
}

result := make([]*semver.Version, 0, len(latest))
for _, v := range latest {
result = append(result, v)
}
return result
}
Loading
Loading