Skip to content

Add salt.cache.etcd3_cache: native etcd v3 minion data cache backend (fixes #61037)#69242

Open
ggiesen wants to merge 1 commit into
saltstack:masterfrom
ggiesen:add-etcd3-cache
Open

Add salt.cache.etcd3_cache: native etcd v3 minion data cache backend (fixes #61037)#69242
ggiesen wants to merge 1 commit into
saltstack:masterfrom
ggiesen:add-etcd3-cache

Conversation

@ggiesen
Copy link
Copy Markdown
Contributor

@ggiesen ggiesen commented May 27, 2026

What does this PR do?

Adds salt.cache.etcd3_cache, a minion data cache backend for the etcd v3 API, enabled with cache: etcd3 (requires etcd3-py).

It is a from-scratch v3-native design rather than a port of the v2 etcd_cache:

  • Each cache entry is a single etcd key holding salt.payload.dumps({"d": data, "t": <epoch>}) -- an atomic single PUT, with no sibling timestamp key to coordinate.
  • store(..., expires=N) is honoured natively via etcd v3 leases, so salt.auth tokens are reaped by etcd rather than persisting until a manual flush.
  • list/contains follow salt.cache.localfs semantics (e.g. cache.list('minions') returns the minion IDs), and listing uses keys-only range scans so listing a bank does not transfer the stored values.
  • Refuses to initialize with an empty/root etcd.path_prefix (a flush safety guard), and the docstring documents the in-etcd key layout and a prefix-scoped etcd RBAC role for shared-cluster isolation.

Connection setup (profile, auth, TLS) reuses salt.utils.etcd_util.EtcdClientV3; all data-plane operations then go through the etcd3-py client directly.

What issues does this PR fix or reference?

Fixes #61037 -- provides the etcd v3 cache anticipated by that tech-debt item (and its closed parent #60325). It delivers v3 support as a separate etcd3 backend rather than retrofitting the v2 etcd_cache module (v2 and v3 differ enough that one code path is more noise than signal); the v2 driver and its existing functional tests are unchanged. If the intent of #61037 was specifically to migrate etcd_cache.py itself in place, please re-scope.

Related / follow-ups

  • EtcdClientV3.__init__ has a latent bug: self.encode_values = encode_values or self.conf.get("etcd.encode_values", True) -- the or short-circuit means a caller passing encode_values=False is silently overridden by the default. This cache avoids the wrapper's codec entirely so it is not affected, but the bug is worth a separate fix (a bug fix, so 3006.x). Flagging rather than folding it in here.
  • Auth-enabled etcd depends on Fix etcdv3 auth #69202, which adds the required client.auth() call; I have suggested there that the call be guarded so it does not break no-auth clusters.

Previous Behavior

No native etcd v3 cache backend; only the v2 etcd_cache (python-etcd / v2 API).

New Behavior

cache: etcd3 provides a v3-native minion data cache.

Tests

  • Unit tests mocking the etcd3-py client.
  • The cross-backend functional contract suite (run_common_cache_tests) plus etcd3-specific integration tests (single-key layout, trailing-slash bank boundary, native lease expiry, the in-tree bank/key patterns), both using the shared tests.support.pytest.etcd container fixture so they run in CI where docker is available.

Validated locally against etcd v3.5: the unit suite, the full cache contract, and an end-to-end run through salt.cache.factory (loader-resolved cache: etcd3) including native lease expiry.

Merge requirements satisfied?

  • Docs (module docstring + doc/ref/cache/all)
  • Changelog (changelog/61037.added.md)
  • Tests written

Commits signed with GPG?

No

A from-scratch cache backend for the etcd v3 API. Each cache entry is a
single etcd key holding salt.payload.dumps({"d": data, "t": <epoch>}).
expires= is honoured natively via etcd v3 leases (so salt.auth tokens are
reaped by etcd rather than lingering until a manual flush). list/contains
follow salt.cache.localfs semantics, and listing uses keys-only range
scans so listing a bank does not transfer the stored values.

Connection setup (profile, auth, TLS) reuses
salt.utils.etcd_util.EtcdClientV3 and then operates on the etcd3-py client
directly. Enable with cache: etcd3 (requires the etcd3-py library).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[TECH DEBT][etcd v2-v3] cache/etcd_cache.py - ensure functional tests & v3 compatibility

1 participant