Skip to content

Manifest IT fails with pinned seed #7835

@nadove-ucsc

Description

@nadove-ucsc

To reproduce, run azul_it_flags=seed=0 make integration_test twice in succession:

======================================================================
FAIL: test_indexing (integration_test.IndexingIntegrationTest.test_indexing) [manifest] (catalog='dcp3-it', format=None, i=0, fetch=True)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 399, in subTest
    yield
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 643, in _test_manifest
    self.assertEqual(1, len(execution_ids))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0

======================================================================
FAIL: test_indexing (integration_test.IndexingIntegrationTest.test_indexing) [manifest] (catalog='dcp3-it', format=<ManifestFormat.compact: 'compact'>, i=1, fetch=True)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 399, in subTest
    yield
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 646, in _test_manifest
    self.assertEqual(2, len(execution_ids))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 3

======================================================================
FAIL: test_indexing (integration_test.IndexingIntegrationTest.test_indexing) [manifest] (catalog='dcp3-it', format=<ManifestFormat.compact: 'compact'>, i=2, fetch=False)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 399, in subTest
    yield
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 650, in _test_manifest
    self.assertEqual(2, len(execution_ids))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 3

======================================================================
FAIL: test_indexing (integration_test.IndexingIntegrationTest.test_indexing) [manifest] (catalog='dcp3-it', format=<ManifestFormat.compact: 'compact'>, i=0, fetch=True)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 399, in subTest
    yield
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 643, in _test_manifest
    self.assertEqual(1, len(execution_ids))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0

======================================================================
FAIL: test_indexing (integration_test.IndexingIntegrationTest.test_indexing) [manifest] (catalog='dcp3-it', format=<ManifestFormat.compact: 'compact'>, i=1, fetch=True)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 399, in subTest
    yield
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 646, in _test_manifest
    self.assertEqual(2, len(execution_ids))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 3

======================================================================
FAIL: test_indexing (integration_test.IndexingIntegrationTest.test_indexing) [manifest] (catalog='dcp3-it', format=<ManifestFormat.compact: 'compact'>, i=2, fetch=False)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 399, in subTest
    yield
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 650, in _test_manifest
    self.assertEqual(2, len(execution_ids))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 3

======================================================================
FAIL: test_indexing (integration_test.IndexingIntegrationTest.test_indexing) [manifest] (catalog='dcp3-it', format=<ManifestFormat.terra_pfb: 'terra.pfb'>, i=0, fetch=False)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 399, in subTest
    yield
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 643, in _test_manifest
    self.assertEqual(1, len(execution_ids))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0

======================================================================
FAIL: test_indexing (integration_test.IndexingIntegrationTest.test_indexing) [manifest] (catalog='dcp3-it', format=<ManifestFormat.terra_pfb: 'terra.pfb'>, i=1, fetch=False)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 399, in subTest
    yield
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 646, in _test_manifest
    self.assertEqual(2, len(execution_ids))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 3

...
many similar errors
...

======================================================================
FAIL: test_indexing (integration_test.IndexingIntegrationTest.test_indexing) [manifest_tagging_race] (catalog='lm2-it', format=<ManifestFormat.curl: 'curl'>)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 399, in subTest
    yield
  File "/home/nadove/PycharmProjects/azul/test/integration_test.py", line 724, in _test_manifest_tagging_race
    self.assertEqual(1, len(execution_ids))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0

----------------------------------------------------------------------
Ran 16 tests in 961.812s

FAILED (failures=40, skipped=1)
make: *** [Makefile:284: integration_test] Error 1

Metadata

Metadata

Assignees

No one assigned

    Labels

    spike:1[process] Spike estimate of one pointtest[subject] Unit and integration test code

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions