Skip to content

Commit 4350fb8

Browse files
committed
Fix flaky FileIOTest.testMatchWatchForNewFiles test under CI filesystems
addresses #19480 The `FileIOTest.testMatchWatchForNewFiles` test occasionally flakes in the CI environment because the `updOptions` configuration in `CopyFilesFn` does not preserve file attributes when overwriting existing files. In some CI filesystems, this causes the copied file to register a `lastModifiedMillis` timestamp of `0`. Thus, when `ExtractFilenameAndLastUpdateFn` parses this file, it throws a `RuntimeException` at `FileIO.java:800`, failing the pipeline run. This PR adds `StandardCopyOption.COPY_ATTRIBUTES` to preserve the file's original timestamps, avoiding the exception.
1 parent c8e45e7 commit 4350fb8

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

sdks/java/core/src/test/java/org/apache/beam/sdk/io/FileIOTest.java

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -249,7 +249,9 @@ public void processElement(ProcessContext context, @StateId("count") ValueState<
249249
context.output(Objects.requireNonNull(context.element()).getValue());
250250

251251
CopyOption[] cpOptions = {StandardCopyOption.COPY_ATTRIBUTES};
252-
CopyOption[] updOptions = {StandardCopyOption.REPLACE_EXISTING};
252+
CopyOption[] updOptions = {
253+
StandardCopyOption.REPLACE_EXISTING, StandardCopyOption.COPY_ATTRIBUTES
254+
};
253255
final Path sourcePath = Paths.get(sourcePathStr);
254256
final Path watchPath = Paths.get(watchPathStr);
255257

0 commit comments

Comments
 (0)