Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,40 @@ async def file_object_content(storage_id: str) -> str

**Validation**: Blocked in `CORE` context. Allowed in `WORKER` and `LOCAL` contexts.

### file_object_content_by_id

```python
async def file_object_content_by_id(node_id: str) -> str
```

| Input | Output | Error |
| ----- | ------ | ----- |
| Valid node UUID (text file) | Raw file content (text) | — |
| Valid node UUID (binary file) | — | `JinjaFilterError("file_object_content_by_id", "binary content not supported...")` |
| `None` | — | `JinjaFilterError("file_object_content_by_id", "node_id is null", hint="...")` |
| `""` (empty) | — | `JinjaFilterError("file_object_content_by_id", "node_id is empty", hint="...")` |
| Permission denied (401/403) | — | `JinjaFilterError("file_object_content_by_id", "permission denied for node_id: {id}")` |
| No client provided | — | `JinjaFilterError("file_object_content_by_id", "requires InfrahubClient", hint="...")` |

**Validation**: Blocked in `CORE` context. Allowed in `WORKER` and `LOCAL` contexts.

### file_object_content_by_hfid

```python
async def file_object_content_by_hfid(hfid: str | list[str], kind: str = "") -> str
```

| Input | Output | Error |
| ----- | ------ | ----- |
| Valid HFID + kind (text file) | Raw file content (text) | — |
| Valid HFID + kind (binary file) | — | `JinjaFilterError("file_object_content_by_hfid", "binary content not supported...")` |
| `None` | — | `JinjaFilterError("file_object_content_by_hfid", "hfid is null", hint="...")` |
| Missing `kind` argument | — | `JinjaFilterError("file_object_content_by_hfid", "'kind' argument is required", hint="...")` |
| Permission denied (401/403) | — | `JinjaFilterError("file_object_content_by_hfid", "permission denied for hfid: {hfid}")` |
| No client provided | — | `JinjaFilterError("file_object_content_by_hfid", "requires InfrahubClient", hint="...")` |

**Validation**: Blocked in `CORE` context. Allowed in `WORKER` and `LOCAL` contexts.

### from_json

```python
Expand Down Expand Up @@ -73,9 +107,15 @@ def from_yaml(value: str) -> dict | list

Used by `artifact_content`. Returns plain text content.

### GET /api/files/by-storage-id/{storage_id} (new)
### File object endpoints

All three endpoints return file content with the node's `file_type` as content-type. The SDK validates that the content-type is text-based.

Used by `file_object_content`. Returns file content with appropriate content-type header.
| Endpoint | Used by | Identifier |
| -------- | ------- | ---------- |
| `GET /api/files/by-storage-id/{storage_id}` | `file_object_content` | storage_id |
| `GET /api/files/{node_id}` | `file_object_content_by_id` | node UUID |
| `GET /api/files/by-hfid/{kind}?hfid=...` | `file_object_content_by_hfid` | kind + HFID components |

**Accepted content-types** (text-based):

Expand Down
30 changes: 21 additions & 9 deletions dev/specs/infp-504-artifact-composition/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,23 +65,29 @@ class JinjaFilterError(JinjaTemplateError):

```python
class InfrahubFilters:
def __init__(self, client: InfrahubClient) -> None:
CLIENT_FILTER_NAMES = ("artifact_content", "file_object_content", ...)

def __init__(self, client: InfrahubClient | None = None) -> None:
self.client = client

async def artifact_content(self, storage_id: str) -> str:
"""Retrieve artifact content by storage_id."""
def _require_client(self, filter_name: str) -> InfrahubClient:
"""Raise JinjaFilterError if no client is available."""
...

async def file_object_content(self, storage_id: str) -> str:
"""Retrieve file object content by storage_id."""
...
async def artifact_content(self, storage_id: str) -> str: ...
async def file_object_content(self, storage_id: str) -> str: ...
async def file_object_content_by_id(self, node_id: str) -> str: ...
async def file_object_content_by_hfid(self, hfid: str | list[str], kind: str = "") -> str: ...
```

**Key design decisions**:

- Client is optional — `InfrahubFilters` is always instantiated, each method checks for a client at call time via `_require_client()`
- `CLIENT_FILTER_NAMES` is the single source of truth for all client-dependent filter names, used by `Jinja2Template` for registration
- Methods are `async` — Jinja2's `auto_await` handles them in async rendering mode
- Holds an `InfrahubClient` (async only), not `InfrahubClientSync`
- Each method validates inputs and catches `AuthenticationError` to wrap in `JinjaFilterError`
- File object retrieval is split into 3 filters matching the server's 3 endpoints (`by-storage-id`, `by-id`, `by-hfid`)

## Modified Entities

Expand Down Expand Up @@ -114,9 +120,9 @@ def set_client(self, client: InfrahubClient) -> None:

**Purpose**: Deferred client injection — allows creating a `Jinja2Template` first and adding the client later. Also supports replacing a previously set client.

- Calls `_register_client_filters(client)` to bind real filter methods
- Updates `self._infrahub_filters.client` on the existing `InfrahubFilters` instance (no re-registration needed since the bound methods are already registered)
- If the Jinja2 environment was already created, patches it in place
- Without calling `set_client()` (and without passing `client` to `__init__`), client-dependent filters raise `JinjaFilterError` with a descriptive message at render time
- Without calling `set_client()` (and without passing `client` to `__init__`), client-dependent filters raise `JinjaFilterError` with a descriptive message at render time via `_require_client()`

### Jinja2Template.validate() (modified signature)

Expand All @@ -143,7 +149,11 @@ async def get_file_by_storage_id(self, storage_id: str, tracker: str | None = No
...
```

**API endpoint**: `GET /api/files/by-storage-id/{storage_id}`
**API endpoints**:

- `GET /api/files/by-storage-id/{storage_id}` — used by `file_object_content`
- `GET /api/files/{node_id}` — used by `file_object_content_by_id`
- `GET /api/files/by-hfid/{kind}?hfid=...` — used by `file_object_content_by_hfid`

**Content-type check**: Allow `text/*`, `application/json`, `application/yaml`, `application/x-yaml`. Reject all others.

Expand All @@ -155,6 +165,8 @@ async def get_file_by_storage_id(self, storage_id: str, tracker: str | None = No
# Infrahub client-dependent filters (worker and local contexts)
FilterDefinition("artifact_content", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),
FilterDefinition("file_object_content", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),
FilterDefinition("file_object_content_by_hfid", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),
FilterDefinition("file_object_content_by_id", allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL, source="infrahub"),

# Parsing filters (trusted, all contexts)
FilterDefinition("from_json", allowed_contexts=ExecutionContext.ALL, source="infrahub"),
Expand Down
12 changes: 10 additions & 2 deletions dev/specs/infp-504-artifact-composition/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,17 @@ hostname {{ device.hostname.value }}

### Inline file object content

File objects can be retrieved by storage ID, node UUID, or HFID:

```jinja2
{% set file_content = file_object.storage_id.value | file_object_content %}
{{ file_content }}
{# By storage_id (most common) #}
{{ file_object.storage_id.value | file_object_content }}

{# By node UUID #}
{{ file_object.id | file_object_content_by_id }}

{# By Human-Friendly ID #}
{{ hfid_components | file_object_content_by_hfid(kind="NetworkCircuitContract") }}
```

### Parse structured content
Expand Down
17 changes: 12 additions & 5 deletions dev/specs/infp-504-artifact-composition/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,19 +94,26 @@ The existing single `restricted: bool` parameter on `validate()` is insufficient
### Functional requirements

- **FR-001**: `Jinja2Template.__init__` MUST accept an optional `client` parameter of type `InfrahubClient | None` (default `None`). Additionally, `Jinja2Template` MUST expose a `set_client(client)` method for deferred client injection, allowing the template to be created first and the client added later. `InfrahubClientSync` is not supported.
- **FR-002**: A dedicated class (for example, `InfrahubFilters`) MUST be introduced to hold the client reference and expose the Infrahub-specific filter callable methods. `Jinja2Template` instantiates this class when a client is provided (via `__init__` or `set_client()`) and registers its filters into the Jinja2 environment.
- **FR-002**: A dedicated class (for example, `InfrahubFilters`) MUST be introduced to hold an optional client reference and expose the Infrahub-specific filter callable methods. `InfrahubFilters` is always instantiated by `Jinja2Template` (even without a client); each filter method checks for a client at call time and raises `JinjaFilterError` if none is available. `set_client()` updates the existing `InfrahubFilters` instance rather than creating a new one. `InfrahubFilters.CLIENT_FILTER_NAMES` is the single source of truth for all client-dependent filter names.
- **FR-003**: The system MUST provide an `artifact_content` Jinja2 filter that accepts a `storage_id` string and returns the raw string content of the referenced artifact, using the artifact-specific API path.
- **FR-004**: The system MUST provide a `file_object_content` Jinja2 filter that accepts a `storage_id` string and returns the raw string content of the referenced file object, using the file-object-specific API path or metadata handling — this implementation is distinct from `artifact_content`.
- **FR-005**: Both `artifact_content` and `file_object_content` MUST raise `JinjaFilterError` when the input `storage_id` is null or empty, or when the object store cannot retrieve the content for any reason (not found, network failure, auth failure). Additionally, `file_object_content` MUST raise `JinjaFilterError` when the retrieved content has a non-text content type (i.e., not `text/*`, `application/json`, or `application/yaml`).
- **FR-006**: Both `artifact_content` and `file_object_content` MUST raise `JinjaFilterError` when invoked and no `InfrahubClient` was supplied to `Jinja2Template` at construction time. The error message MUST name the filter and explain that an `InfrahubClient` is required.
- **FR-007**: Both `artifact_content` and `file_object_content` MUST be registered with `allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL` in the `FilterDefinition` registry. The `validate()` method accepts an `ExecutionContext` flag; these filters are blocked in the `CORE` context (API server computed attributes) and permitted in the `WORKER` context (Prefect workers) and `LOCAL` context (CLI/unrestricted rendering). Within Infrahub, any Jinja2-based computed attributes that use these new filters should cause a schema violation when loading the schema.
- **FR-004**: The system MUST provide three file object content filters, each retrieving content via a different identifier:
- `file_object_content` — accepts a `storage_id` string, uses `GET /api/files/by-storage-id/{storage_id}`
- `file_object_content_by_id` — accepts a node UUID string, uses `GET /api/files/{node_id}`
- `file_object_content_by_hfid` — accepts an HFID string or list and a required `kind` argument, uses `GET /api/files/by-hfid/{kind}?hfid=...`
All three share the same binary content-type rejection and error handling behavior.
- **FR-005**: All client-dependent filters (`artifact_content`, `file_object_content`, `file_object_content_by_id`, `file_object_content_by_hfid`) MUST raise `JinjaFilterError` when the input identifier is null or empty, or when the object store cannot retrieve the content for any reason (not found, network failure, auth failure). Additionally, all file object filters MUST raise `JinjaFilterError` when the retrieved content has a non-text content type (i.e., not `text/*`, `application/json`, or `application/yaml`). `file_object_content_by_hfid` MUST also raise `JinjaFilterError` when the `kind` argument is missing.
- **FR-006**: All client-dependent filters MUST raise `JinjaFilterError` when invoked and no `InfrahubClient` was supplied to `Jinja2Template` at construction time. The error message MUST name the filter and explain that an `InfrahubClient` is required.
- **FR-007**: All client-dependent filters (`artifact_content`, `file_object_content`, `file_object_content_by_id`, `file_object_content_by_hfid`) MUST be registered with `allowed_contexts=ExecutionContext.WORKER | ExecutionContext.LOCAL` in the `FilterDefinition` registry. The `validate()` method accepts an `ExecutionContext` flag; these filters are blocked in the `CORE` context (API server computed attributes) and permitted in the `WORKER` context (Prefect workers) and `LOCAL` context (CLI/unrestricted rendering). Within Infrahub, any Jinja2-based computed attributes that use these new filters should cause a schema violation when loading the schema.
- **FR-008**: The system MUST provide `from_json` and `from_yaml` Jinja2 filters (adding them only if not already present in the environment) that parse a string into a Python dict/list. Applying them to an empty string MUST return an empty dict without raising. Applying them to malformed content MUST raise `JinjaFilterError`.
- **FR-009**: `from_json` and `from_yaml` MUST be registered as trusted filters (`trusted=True`) since they perform no external I/O.
- **FR-010**: All new filters MUST work correctly with `InfrahubClient` (async). `InfrahubClientSync` is not a supported client type for `Jinja2Template`. Both the sandboxed environment (string-based templates) and the file-based environment MUST have `enable_async=True` to support async filter callables via Jinja2's `auto_await`.
- **FR-011**: All `JinjaFilterError` instances MUST carry an actionable error message that identifies the filter name, the cause of failure, and any remediation hint (for example: "artifact_content requires an InfrahubClient — pass one via Jinja2Template(client=...)").
- **FR-012**: A new `JinjaFilterError` exception class MUST be added to `infrahub_sdk/template/exceptions.py` as a subclass of `JinjaTemplateError`.
- **FR-013**: Documentation MUST include a Python transform example demonstrating artifact content retrieval via `client.object_store.get(identifier=storage_id)`. No new SDK convenience method will be added.
- **FR-014**: If the current user isn't allowed due to a permission denied error to query for the artifact or object file the filter should catch such permission error and raise a Jinja2 error specifically related to the permission issue.
- **FR-015**: The auto-generated templating reference (`sdk_template_reference.j2` and `_generate_infrahub_sdk_template_documentation()`) MUST be updated to include `INFRAHUB_FILTERS` as a third section ("Infrahub filters"). The table MUST show which execution contexts each filter is allowed in (`CORE`, `WORKER`, `LOCAL`) rather than only a binary "Trusted" column.
- **FR-016**: The templating reference documentation MUST explain the `ExecutionContext` model — what `CORE`, `WORKER`, and `LOCAL` contexts mean, how they map to Infrahub's execution environments (computed attributes, Prefect workers, local CLI), and how `validate(context=...)` is used to enforce filter restrictions.
- **FR-017**: Usage examples for the new Jinja2 filters MUST be included in the SDK documentation, covering at minimum: `artifact_content` with a storage_id, `file_object_content` by storage_id, `file_object_content_by_id` by node UUID, `file_object_content_by_hfid` with `kind` argument, and `from_json`/`from_yaml` chaining.

### Key entities

Expand Down
89 changes: 88 additions & 1 deletion docs/_templates/sdk_template_reference.j2
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,35 @@ title: Python SDK Templating
---
Filters can be used when defining [computed attributes](https://docs.infrahub.app/guides/computed-attributes) or [Jinja2 Transforms](https://docs.infrahub.app/guides/jinja2-transform) within Infrahub.

## Execution contexts

Filters are restricted based on the execution context in which a template is rendered:

- **CORE** — Computed attributes evaluated inside the Infrahub API server. Only fully trusted filters (no I/O, no side effects) are allowed.
- **WORKER** — Jinja2 transforms executed on Prefect background workers. Trusted filters and Infrahub client-dependent filters are allowed.
- **LOCAL** — Local CLI rendering and unrestricted usage. All filters are allowed.

The `validate()` method on `Jinja2Template` accepts an optional `context` parameter to enforce these restrictions:

{% raw %}```python
from infrahub_sdk.template import Jinja2Template
from infrahub_sdk.template.filters import ExecutionContext

template = Jinja2Template(template="{{ sid | artifact_content }}")

# Raises JinjaTemplateOperationViolationError — blocked in CORE
template.validate(context=ExecutionContext.CORE)

# Passes — allowed in WORKER
template.validate(context=ExecutionContext.WORKER)
```
{% endraw %}

For backward compatibility, `validate(restricted=True)` maps to `CORE` and `validate(restricted=False)` maps to `LOCAL`.

## Builtin Jinja2 filters

The following filters are those that are [shipped with Jinja2](https://jinja.palletsprojects.com/en/stable/templates/#list-of-builtin-filters) and enabled within Infrahub. The trusted column indicates if the filter is allowed for use with Infrahub's computed attributes when the server is configured in strict mode.
The following filters are [shipped with Jinja2](https://jinja.palletsprojects.com/en/stable/templates/#list-of-builtin-filters) and enabled within Infrahub. The "Trusted" column indicates if the filter is allowed in the CORE context (computed attributes).

<!-- vale off -->
| Name | Trusted |
Expand All @@ -26,6 +52,67 @@ The following Jinja2 filters from <a href="https://netutils.readthedocs.io">Netu
{% endfor %}
<!-- vale on -->

## Infrahub filters

These filters are provided by the Infrahub SDK for artifact and file object content composition. The table shows which execution contexts each filter is allowed in.

<!-- vale off -->
| Name | CORE | WORKER | LOCAL |
| ---- | ---- | ------ | ----- |
{% for filter in infrahub %}
| `{{ filter.name }}` | {% if filter.core %}✅{% else %}❌{% endif %} | {% if filter.worker %}✅{% else %}❌{% endif %} | {% if filter.local %}✅{% else %}❌{% endif %} |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to report on all filters using the same kind of table? I.e. for the previous table we only have the "trusted" option? Might be easier to understand if the format for the trust model is the same for all filter types?

{% endfor %}
<!-- vale on -->

### Usage examples

**Inline artifact content by `storage_id`:**

```jinja2
{% raw %}{{ artifact.node.storage_id.value | artifact_content }}
{% endraw %}```

**Inline file object content:**

```jinja2
{% raw %}{# By storage_id #}
{{ file_object.storage_id.value | file_object_content }}

{# By node UUID #}
{{ file_object.id | file_object_content_by_id }}

{# By Human-Friendly ID #}
{{ hfid_components | file_object_content_by_hfid(kind="NetworkCircuitContract") }}
{% endraw %}```

**Parse structured content with chaining:**

```jinja2
{% raw %}{# JSON artifact → access parsed fields #}
{% set config = artifact.node.storage_id.value | artifact_content | from_json %}
interface {{ config.interface_name }}
ip address {{ config.ip_address }}

{# YAML artifact → iterate parsed data #}
{% set config = artifact.node.storage_id.value | artifact_content | from_yaml %}
{% for route in config.static_routes %}
ip route {{ route.prefix }} {{ route.next_hop }}
{% endfor %}
{% endraw %}```

Client-dependent filters (`artifact_content`, `file_object_content`, `file_object_content_by_id`, `file_object_content_by_hfid`) require an `InfrahubClient` to be passed to `Jinja2Template`:

```python
from infrahub_sdk.template import Jinja2Template

# At construction time
template = Jinja2Template(template=my_template, client=client)

# Or via deferred injection
template = Jinja2Template(template=my_template)
template.set_client(client)
```

## Known issues

### Unable to combine the map and sort filters (https://github.com/pallets/jinja/issues/2081)
Expand Down
Loading
Loading