fix(core): fall back to podman CLI when no API socket is found

[russellb]

Summary

Auto-detection fails to discover Podman on macOS because the API socket symlink is not always present — it varies by Podman version, machine provider, and platform. This adds a podman info CLI fallback so auto-detection succeeds when Podman is functional but the socket isn't at a well-known path.

Related Issue

Closes #1834

Changes

• Add podman_cli_responds() fallback to is_podman_available() in openshell-core config
• When no socket candidate responds, tries podman info which uses Podman's own internal discovery

Testing

• cargo test -p openshell-core — 164 tests pass
• Verified on macOS (Apple Silicon) with Podman 5.8.2 / machine 5.7.1
• E2E tests added/updated (if applicable)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[drew]

/ok to test e068662

[elezar]

One question I have: This seems to fix detection, but doesn't change how the gateway interacts with the podman driver? Why are no changes needed there?

[russellb]

One question I have: This seems to fix detection, but doesn't change how the gateway interacts with the podman driver? Why are no changes needed there?

It's a fair question. I have podman on mac, but I don't have this problem. I was hoping the reporter would test this and see if it was enough. It seems like a reasonable change on the detection side. We go from "podman not detected at all" to either:

1. It works (seems doubtful, but let's see ...)
2. We detected podman is present, but we aren't finding the socket for some reason

In either case, it's more consistent with docker in this part of the code. Docker does the same thing with falling back to a CLI check at this stage.

A next improvement could be to discover the socket using podman info --format '{{.Host.RemoteSocket.Path}}' if it's not in one of the paths we expected to find it.

cc @r3v5, reporter of the issue

[r3v5]

One question I have: This seems to fix detection, but doesn't change how the gateway interacts with the podman driver? Why are no changes needed there?

It's a fair question. I have podman on mac, but I don't have this problem. I was hoping the reporter would test this and see if it was enough. It seems like a reasonable change on the detection side. We go from "podman not detected at all" to either:

1. It works (seems doubtful, but let's see ...)
2. We detected podman is present, but we aren't finding the socket for some reason

In either case, it's more consistent with docker in this part of the code. Docker does the same thing with falling back to a CLI check at this stage.

A next improvement could be to discover the socket using podman info --format '{{.Host.RemoteSocket.Path}}' if it's not in one of the paths we expected to find it.

cc @r3v5, reporter of the issue

Hey @russellb ! Sure, I will test your fix, no worries.

[russellb]

@r3v5 thanks. The output of that podman info command would be helpful too

[elezar]

I have a couple of concerns / questions here.

The first is the one that I've already mentioned. This change checks that podman info works, but does not use the configured socket for actually constructing the driver. I would thus be surprised if this change actually allows the podman driver to be used. Due to the precedence of container engine detection, this also means that this is a breaking change for systems where Podman was running on a non-standard socket, but Docker was also installed and usable. On these systems, the gateway will now try to use the Podman driver and fail.

Then, although this seems to align Podman functionality with Docker, there are subtle differences between the two paths. podman info is a slower (and stronger) contract than running docker --version which only checks CLI existence. Podman always uses the configured socket, whereas Docker includes logic to resolve the actual Docker connection later through the local client.

Although it is a slightly larger change than originally proposed, I think there is some benefit in trying to better align the detection paths for Podman and Docker. Ideally these would return a usable driver config (including, for example socket information) and not just a boolean. This config could then be used directly when instantiating the driver(s) instead of rediscovering the relevant config (as is done in the Docker case).

[russellb]

Thanks, @elezar. I'm happy to work on the changes you described.

[krishicks]

I made a change to podman auto-detection recently (#1536), to avoid using just the existence of the CLI to determine that podman was available. In that change I made sure to align the auto-detection with the actual client socket choice mechanism.

Podman unfortunately has different behavior on macos depending on how you install it, which I talked about in #1690 (comment)

This PR could supersede #1690 (which is scoped to documentation), but it needs to keep the auto-detection mechanism and what the client uses to make the actual connection be aligned, like @elezar has raised.

[russellb]

Great feedback, thanks @krishicks. I'll iterate on this.