Skip to content

Reduce number of layers for Beam container images #38440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Abacn merged 1 commit into from
May 11, 2026
+22 −19
Merged

Reduce number of layers for Beam container images #38440

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions sdks/go/container/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,20 @@
# limitations under the License.
###############################################################################

FROM gcr.io/distroless/base-nossl-debian12:latest
FROM gcr.io/distroless/base-nossl-debian12:latest AS base_image

# Distroless base image has many layers. Squash into single layer
FROM scratch
COPY --from=base_image / /

LABEL Author "Apache Beam <dev@beam.apache.org>"

ARG TARGETOS
ARG TARGETARCH

ADD target/${TARGETOS}_${TARGETARCH}/boot /opt/apache/beam/
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use COPY instead of ADD for local files when no extraction is required. This is a Docker best practice and maintains consistency with the improvements made in the Java Dockerfile.

COPY target/${TARGETOS}_${TARGETARCH}/boot /opt/apache/beam/


COPY target/LICENSE /opt/apache/beam/
COPY target/NOTICE /opt/apache/beam/
COPY target/LICENSE target/NOTICE /opt/apache/beam/

# Add Go licenses.
COPY target/go-licenses/* /opt/apache/beam/third_party_licenses/golang/
Expand Down
31 changes: 15 additions & 16 deletions sdks/java/container/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,22 +24,21 @@ ARG TARGETARCH

ARG pull_licenses

ADD target/slf4j-api.jar /opt/apache/beam/jars/
ADD target/slf4j-jdk14.jar /opt/apache/beam/jars/
ADD target/jcl-over-slf4j.jar /opt/apache/beam/jars/
ADD target/log4j-over-slf4j.jar /opt/apache/beam/jars/
ADD target/log4j-to-slf4j.jar /opt/apache/beam/jars/
ADD target/beam-sdks-java-harness.jar /opt/apache/beam/jars/

# Required to use jamm as a javaagent to get accurate object size measuring
# COPY fails if file is not found, so use a wildcard for open-module-agent.jar
# since it is only included in Java 9+ containers
COPY target/jamm.jar target/open-module-agent.jar /opt/apache/beam/jars/

COPY target/${TARGETOS}_${TARGETARCH}/boot /opt/apache/beam/

COPY target/LICENSE /opt/apache/beam/
COPY target/NOTICE /opt/apache/beam/
# Dependency jars
COPY target/slf4j-api.jar \
target/slf4j-jdk14.jar \
target/jcl-over-slf4j.jar \
target/log4j-over-slf4j.jar \
target/log4j-to-slf4j.jar \
# Required to use jamm as a javaagent to get accurate object size measuring
target/jamm.jar \
/opt/apache/beam/jars/
Comment thread
Abacn marked this conversation as resolved.

# Built jars
COPY target/open-module-agent.jar target/beam-sdks-java-harness.jar /opt/apache/beam/jars/
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The open-module-agent.jar is only present in Java 9+ builds. Using a specific filename in COPY will cause the build to fail if the file is missing (e.g., in Java 8 environments). Using a wildcard (e.g., target/open-module-agent.jar*) makes this file optional and ensures the build remains robust across different Java versions.

COPY target/open-module-agent.jar* target/beam-sdks-java-harness.jar /opt/apache/beam/jars/


# Built binary with licenses
COPY target/${TARGETOS}_${TARGETARCH}/boot target/LICENSE target/NOTICE /opt/apache/beam/

# copy third party licenses
ADD target/third_party_licenses /opt/apache/beam/third_party_licenses/
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For local files and directories, COPY is preferred over ADD unless you specifically need the auto-extraction feature of ADD (e.g., for tarballs). Using COPY is more explicit and follows Docker best practices, consistent with other changes in this pull request.

COPY target/third_party_licenses /opt/apache/beam/third_party_licenses/

Expand Down
Loading