IFC-2275 Add Jinja2 filters for artifact content composition (#889)

Introduce artifact_content, file_object_content, from_json, and from_yaml
Jinja2 filters to enable modular configuration pipelines where templates
can reference and inline rendered content from other artifacts.

Replace FilterDefinition.trusted boolean with a flag-based ExecutionContext
model (CORE, WORKER, LOCAL) for context-aware template validation.
Jinja2Template gains an optional client parameter and a set_client() method
for deferred client injection.

Author: gmazoyer

changelog/+artifact-composition.added.md

@@ -0,0 +1 @@
+Add `artifact_content`, `file_object_content`, `from_json`, and `from_yaml` Jinja2 filters for artifact content composition in templates.

changelog/+artifact-composition.changed.md

@@ -0,0 +1 @@
+Replace `FilterDefinition.trusted: bool` with flag-based `ExecutionContext` model (`CORE`, `WORKER`, `LOCAL`) for context-aware template validation. `validate()` now accepts an optional `context` parameter. Backward compatible.

dev/specs/infp-504-artifact-composition/contracts/filter-interfaces.md

@@ -19,7 +19,7 @@ async def artifact_content(storage_id: str) -> str
 | Permission denied (401/403) | — | `JinjaFilterError("artifact_content", "permission denied for storage_id: {id}")` |
 | No client provided | — | `JinjaFilterError("artifact_content", "requires InfrahubClient", hint="pass client via Jinja2Template(client=...)")` |
 
-**Validation**: Blocked in `CORE` context. Allowed in `WORKER` context.
+**Validation**: Blocked in `CORE` context. Allowed in `WORKER` and `LOCAL` contexts.
 
 ### file_object_content
 
@@ -37,7 +37,7 @@ async def file_object_content(storage_id: str) -> str
 | Permission denied (401/403) | — | `JinjaFilterError("file_object_content", "permission denied for storage_id: {id}")` |
 | No client provided | — | `JinjaFilterError("file_object_content", "requires InfrahubClient", hint="pass client via Jinja2Template(client=...)")` |
 
-**Validation**: Blocked in `CORE` context. Allowed in `WORKER` context.
+**Validation**: Blocked in `CORE` context. Allowed in `WORKER` and `LOCAL` contexts.
 
 ### from_json
 

dev/specs/infp-504-artifact-composition/data-model.md

@@ -104,7 +104,19 @@ def __init__(
 - New optional `client` parameter
 - When `client` provided: instantiate `InfrahubFilters`, register `artifact_content` and `file_object_content`
 - Always register `from_json` and `from_yaml` (no client needed)
-- File-based environment must add `enable_async=True` for async filter support
+- File-based environment already has `enable_async=True` (no change needed)
+
+### Jinja2Template.set_client() (new method)
+
+```python
+def set_client(self, client: InfrahubClient) -> None:
+```
+
+**Purpose**: Deferred client injection — allows creating a `Jinja2Template` first and adding the client later. Also supports replacing a previously set client.
+
+- Calls `_register_client_filters(client)` to bind real filter methods
+- If the Jinja2 environment was already created, patches it in place
+- Without calling `set_client()` (and without passing `client` to `__init__`), client-dependent filters raise `JinjaFilterError` with a descriptive message at render time
 
 ### Jinja2Template.validate() (modified signature)
 
@@ -140,9 +152,9 @@ async def get_file_by_storage_id(self, storage_id: str, tracker: str | None = No
 ```python
 # In AVAILABLE_FILTERS:
 
-# Infrahub client-dependent filters (worker context only)
-FilterDefinition("artifact_content", allowed_contexts=ExecutionContext.WORKER, source="infrahub"),
-FilterDefinition("file_object_content", allowed_contexts=ExecutionContext.WORKER, source="infrahub"),
+# Infrahub client-dependent filters (worker and local contexts)
+FilterDefinition("artifact_content", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),
+FilterDefinition("file_object_content", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),
 
 # Parsing filters (trusted, all contexts)
 FilterDefinition("from_json", allowed_contexts=ExecutionContext.ALL, source="infrahub"),

dev/specs/infp-504-artifact-composition/spec.md

@@ -93,13 +93,13 @@ The existing single `restricted: bool` parameter on `validate()` is insufficient
 
 ### Functional requirements
 
-- **FR-001**: `Jinja2Template.__init__` MUST accept an optional `client` parameter of type `InfrahubClient | None` (default `None`). `InfrahubClientSync` is not supported.
-- **FR-002**: A dedicated class (for example, `InfrahubFilters`) MUST be introduced to hold the client reference and expose the Infrahub-specific filter callable methods. `Jinja2Template` instantiates this class when a client is provided and registers its filters into the Jinja2 environment.
+- **FR-001**: `Jinja2Template.__init__` MUST accept an optional `client` parameter of type `InfrahubClient | None` (default `None`). Additionally, `Jinja2Template` MUST expose a `set_client(client)` method for deferred client injection, allowing the template to be created first and the client added later. `InfrahubClientSync` is not supported.
+- **FR-002**: A dedicated class (for example, `InfrahubFilters`) MUST be introduced to hold the client reference and expose the Infrahub-specific filter callable methods. `Jinja2Template` instantiates this class when a client is provided (via `__init__` or `set_client()`) and registers its filters into the Jinja2 environment.
 - **FR-003**: The system MUST provide an `artifact_content` Jinja2 filter that accepts a `storage_id` string and returns the raw string content of the referenced artifact, using the artifact-specific API path.
 - **FR-004**: The system MUST provide a `file_object_content` Jinja2 filter that accepts a `storage_id` string and returns the raw string content of the referenced file object, using the file-object-specific API path or metadata handling — this implementation is distinct from `artifact_content`.
 - **FR-005**: Both `artifact_content` and `file_object_content` MUST raise `JinjaFilterError` when the input `storage_id` is null or empty, or when the object store cannot retrieve the content for any reason (not found, network failure, auth failure). Additionally, `file_object_content` MUST raise `JinjaFilterError` when the retrieved content has a non-text content type (i.e., not `text/*`, `application/json`, or `application/yaml`).
 - **FR-006**: Both `artifact_content` and `file_object_content` MUST raise `JinjaFilterError` when invoked and no `InfrahubClient` was supplied to `Jinja2Template` at construction time. The error message MUST name the filter and explain that an `InfrahubClient` is required.
-- **FR-007**: Both `artifact_content` and `file_object_content` MUST be registered with `allowed_contexts=ExecutionContext.WORKER` in the `FilterDefinition` registry. The `validate()` method accepts an `ExecutionContext` flag; these filters are blocked in the `CORE` context (API server computed attributes) and permitted in the `WORKER` context (Prefect workers, where an `InfrahubClient` is available). Within Infrahub, any Jinja2-based computed attributes that use these new filters should cause a schema violation when loading the schema.
+- **FR-007**: Both `artifact_content` and `file_object_content` MUST be registered with `allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL` in the `FilterDefinition` registry. The `validate()` method accepts an `ExecutionContext` flag; these filters are blocked in the `CORE` context (API server computed attributes) and permitted in the `WORKER` context (Prefect workers) and `LOCAL` context (CLI/unrestricted rendering). Within Infrahub, any Jinja2-based computed attributes that use these new filters should cause a schema violation when loading the schema.
 - **FR-008**: The system MUST provide `from_json` and `from_yaml` Jinja2 filters (adding them only if not already present in the environment) that parse a string into a Python dict/list. Applying them to an empty string MUST return an empty dict without raising. Applying them to malformed content MUST raise `JinjaFilterError`.
 - **FR-009**: `from_json` and `from_yaml` MUST be registered as trusted filters (`trusted=True`) since they perform no external I/O.
 - **FR-010**: All new filters MUST work correctly with `InfrahubClient` (async). `InfrahubClientSync` is not a supported client type for `Jinja2Template`. Both the sandboxed environment (string-based templates) and the file-based environment MUST have `enable_async=True` to support async filter callables via Jinja2's `auto_await`.

dev/specs/infp-504-artifact-composition/tasks.md

@@ -22,8 +22,8 @@
 
 **CRITICAL**: No user story work can begin until this phase is complete
 
-- [ ] T001 [P] Create `JinjaFilterError` exception class in `infrahub_sdk/template/exceptions.py` — subclass of `JinjaTemplateError` with `filter_name`, `message`, and optional `hint` attributes. Include unit tests for instantiation, inheritance chain, and message formatting. (IFC-2367)
-- [ ] T002 [P] Implement `ExecutionContext` flag enum and migrate `FilterDefinition` in `infrahub_sdk/template/filters.py` — add `ExecutionContext(Flag)` with `CORE`, `WORKER`, `LOCAL`, `ALL` values. Replace `FilterDefinition.trusted: bool` with `allowed_contexts: ExecutionContext`. Add backward-compat `trusted` property. Migrate all 138 existing filter entries (`trusted=True` → `ALL`, `trusted=False` → `LOCAL`). Update `validate()` in `infrahub_sdk/template/__init__.py` to accept optional `context: ExecutionContext` parameter (takes precedence over `restricted`; `restricted=True` → `CORE`, `restricted=False` → `LOCAL`). Include unit tests for all 3 contexts with existing filters, backward compat path, and no regressions. (IFC-2368)
+- [x] T001 [P] Create `JinjaFilterError` exception class in `infrahub_sdk/template/exceptions.py` — subclass of `JinjaTemplateError` with `filter_name`, `message`, and optional `hint` attributes. Include unit tests for instantiation, inheritance chain, and message formatting. (IFC-2367)
+- [x] T002 [P] Implement `ExecutionContext` flag enum and migrate `FilterDefinition` in `infrahub_sdk/template/filters.py` — add `ExecutionContext(Flag)` with `CORE`, `WORKER`, `LOCAL`, `ALL` values. Replace `FilterDefinition.trusted: bool` with `allowed_contexts: ExecutionContext`. Add backward-compat `trusted` property. Migrate all 138 existing filter entries (`trusted=True` → `ALL`, `trusted=False` → `LOCAL`). Update `validate()` in `infrahub_sdk/template/__init__.py` to accept optional `context: ExecutionContext` parameter (takes precedence over `restricted`; `restricted=True` → `CORE`, `restricted=False` → `LOCAL`). Include unit tests for all 3 contexts with existing filters, backward compat path, and no regressions. (IFC-2368)
 
 **Checkpoint**: Foundation ready — JinjaFilterError and ExecutionContext available for all stories
 
@@ -37,9 +37,9 @@
 
 ### Implementation for US1 + US4
 
-- [ ] T003 [US1] Create `InfrahubFilters` class in `infrahub_sdk/template/infrahub_filters.py` — new file. Class holds `InfrahubClient` reference, exposes async filter methods. Methods are `async def` (Jinja2 `auto_await` handles them in async render mode per R-001). Raises `JinjaFilterError` when called without a client. Include unit tests for instantiation with/without client. (IFC-2371)
-- [ ] T004 [US1] Implement `artifact_content` async method on `InfrahubFilters` in `infrahub_sdk/template/infrahub_filters.py` — uses `self.client.object_store.get(identifier=storage_id)`. Raises `JinjaFilterError` on: null/empty storage_id, retrieval failure, permission denied (catch `AuthenticationError` per R-006). Artifacts are always text (no binary check needed per R-003). Include unit tests: happy path (mocked ObjectStore), null, empty, not-found, network error, permission denied, no-client error with descriptive message. (IFC-2372)
-- [ ] T005 [US1] [US4] Add `client` parameter to `Jinja2Template.__init__` and wire up filter registration in `infrahub_sdk/template/__init__.py` — add `client: InfrahubClient | None = None` param. When client provided: instantiate `InfrahubFilters`, register `artifact_content` into Jinja2 env filter map. Add `enable_async=True` to `_get_file_based_environment()` (per R-001 caveat). Register `artifact_content` in `FilterDefinition` registry with `allowed_contexts=ExecutionContext.WORKER`. Include unit tests: render with client (mocked), render without client (error), validation in CORE (blocked), WORKER (allowed), LOCAL (allowed). Verify existing untrusted filters like `safe` remain blocked in WORKER context (US4 AC3). (IFC-2375 partial + IFC-2376 partial)
+- [x] T003 [US1] Create `InfrahubFilters` class in `infrahub_sdk/template/infrahub_filters.py` — new file. Class holds `InfrahubClient` reference, exposes async filter methods. Methods are `async def` (Jinja2 `auto_await` handles them in async render mode per R-001). Raises `JinjaFilterError` when called without a client. Include unit tests for instantiation with/without client. (IFC-2371)
+- [x] T004 [US1] Implement `artifact_content` async method on `InfrahubFilters` in `infrahub_sdk/template/infrahub_filters.py` — uses `self.client.object_store.get(identifier=storage_id)`. Raises `JinjaFilterError` on: null/empty storage_id, retrieval failure, permission denied (catch `AuthenticationError` per R-006). Artifacts are always text (no binary check needed per R-003). Include unit tests: happy path (mocked ObjectStore), null, empty, not-found, network error, permission denied, no-client error with descriptive message. (IFC-2372)
+- [x] T005 [US1] [US4] Add `client` parameter to `Jinja2Template.__init__` and wire up filter registration in `infrahub_sdk/template/__init__.py` — add `client: InfrahubClient | None = None` param. When client provided: instantiate `InfrahubFilters`, register `artifact_content` into Jinja2 env filter map. Add `enable_async=True` to `_get_file_based_environment()` (per R-001 caveat). Register `artifact_content` in `FilterDefinition` registry with `allowed_contexts=ExecutionContext.WORKER`. Include unit tests: render with client (mocked), render without client (error), validation in CORE (blocked), WORKER (allowed), LOCAL (allowed). Verify existing untrusted filters like `safe` remain blocked in WORKER context (US4 AC3). Also added `set_client()` setter for deferred client injection per PR #885 feedback. (IFC-2375 partial + IFC-2376 partial)
 
 **Checkpoint**: US1 + US4 fully functional. `artifact_content` renders in WORKER context, blocked in CORE. MVP complete.
 
@@ -53,9 +53,9 @@
 
 ### Implementation for US2
 
-- [ ] T006 [P] [US2] Add `get_file_by_storage_id()` method to `ObjectStore` in `infrahub_sdk/object_store.py` — async method using endpoint `GET /api/files/by-storage-id/{storage_id}`. Check `content-type` response header: allow `text/*`, `application/json`, `application/yaml`, `application/x-yaml`; reject all others with descriptive error. Handle 401/403 as `AuthenticationError`. Include unit tests: text response, binary rejection, 404, auth failure, network error. (IFC-2373)
-- [ ] T007 [US2] Implement `file_object_content` async method on `InfrahubFilters` in `infrahub_sdk/template/infrahub_filters.py` — uses new `self.client.object_store.get_file_by_storage_id(storage_id)`. Same error handling as `artifact_content` plus binary content error (delegated to ObjectStore). Include unit tests: happy path, all error conditions, binary content rejection. (IFC-2374)
-- [ ] T008 [US2] Register `file_object_content` filter in `Jinja2Template` and `FilterDefinition` in `infrahub_sdk/template/__init__.py` and `infrahub_sdk/template/filters.py` — register when client provided. `allowed_contexts=ExecutionContext.WORKER`. Include unit tests: render with client, validation in CORE (blocked), WORKER (allowed). (IFC-2375 partial + IFC-2376 partial)
+- [x] T006 [P] [US2] Add `get_file_by_storage_id()` method to `ObjectStore` in `infrahub_sdk/object_store.py` — async method using endpoint `GET /api/files/by-storage-id/{storage_id}`. Check `content-type` response header: allow `text/*`, `application/json`, `application/yaml`, `application/x-yaml`; reject all others with descriptive error. Handle 401/403 as `AuthenticationError`. Include unit tests: text response, binary rejection, 404, auth failure, network error. (IFC-2373)
+- [x] T007 [US2] Implement `file_object_content` async method on `InfrahubFilters` in `infrahub_sdk/template/infrahub_filters.py` — uses new `self.client.object_store.get_file_by_storage_id(storage_id)`. Same error handling as `artifact_content` plus binary content error (delegated to ObjectStore). Include unit tests: happy path, all error conditions, binary content rejection. (IFC-2374)
+- [x] T008 [US2] Register `file_object_content` filter in `Jinja2Template` and `FilterDefinition` in `infrahub_sdk/template/__init__.py` and `infrahub_sdk/template/filters.py` — register when client provided. `allowed_contexts=ExecutionContext.WORKER`. Include unit tests: render with client, validation in CORE (blocked), WORKER (allowed). (IFC-2375 partial + IFC-2376 partial)
 
 **Checkpoint**: US2 complete. `file_object_content` works alongside `artifact_content`.
 
@@ -69,9 +69,9 @@
 
 ### Implementation for US3
 
-- [ ] T009 [P] [US3] Implement `from_json` filter function in `infrahub_sdk/template/infrahub_filters.py` — pure sync function (no client needed). Empty string → `{}` (explicit special-case since `json.loads("")` raises). Malformed JSON → `JinjaFilterError`. Register in `FilterDefinition` with `allowed_contexts=ExecutionContext.ALL`. Register unconditionally in `Jinja2Template._set_filters()`. Include unit tests: valid JSON dict, valid JSON list, empty string → `{}`, malformed → error, render through template. (IFC-2369)
-- [ ] T010 [P] [US3] Implement `from_yaml` filter function in `infrahub_sdk/template/infrahub_filters.py` — pure sync function. Empty string → `{}` (explicit special-case since `yaml.safe_load("")` returns `None`). Malformed YAML → `JinjaFilterError`. Register in `FilterDefinition` with `allowed_contexts=ExecutionContext.ALL`. Register unconditionally in `Jinja2Template._set_filters()`. Include unit tests: valid YAML mapping, valid YAML list, empty string → `{}`, malformed → error, render through template. (IFC-2370)
-- [ ] T011 [US3] Integration test for filter chaining in `tests/unit/template/test_infrahub_filters.py` — test `artifact_content | from_json` and `artifact_content | from_yaml` end-to-end with mocked ObjectStore returning JSON/YAML content. Verify template can access parsed fields. (IFC-2376 partial, SC-006)
+- [x] T009 [P] [US3] Implement `from_json` filter function in `infrahub_sdk/template/infrahub_filters.py` — pure sync function (no client needed). Empty string → `{}` (explicit special-case since `json.loads("")` raises). Malformed JSON → `JinjaFilterError`. Register in `FilterDefinition` with `allowed_contexts=ExecutionContext.ALL`. Register unconditionally in `Jinja2Template._set_filters()`. Include unit tests: valid JSON dict, valid JSON list, empty string → `{}`, malformed → error, render through template. (IFC-2369)
+- [x] T010 [P] [US3] Implement `from_yaml` filter function in `infrahub_sdk/template/infrahub_filters.py` — pure sync function. Empty string → `{}` (explicit special-case since `yaml.safe_load("")` returns `None`). Malformed YAML → `JinjaFilterError`. Register in `FilterDefinition` with `allowed_contexts=ExecutionContext.ALL`. Register unconditionally in `Jinja2Template._set_filters()`. Include unit tests: valid YAML mapping, valid YAML list, empty string → `{}`, malformed → error, render through template. (IFC-2370)
+- [x] T011 [US3] Integration test for filter chaining in `tests/unit/template/test_infrahub_filters.py` — test `artifact_content | from_json` and `artifact_content | from_yaml` end-to-end with mocked ObjectStore returning JSON/YAML content. Verify template can access parsed fields. (IFC-2376 partial, SC-006)
 
 **Checkpoint**: US3 complete. All 4 filters work, chain correctly, and are validated per context.