Search before asking

• I had searched in the issues and found no similar issues.

What happened

On running data collection pipeline second time (incremental mode), the first CircleCI task fails with an SQL error:
| /usr/local/go/src/runtime/asm_amd64.s:1598 Wraps: (4) Error 1054 (42S22): Unknown column 'created_date' in 'where clause' (500) Wraps: (5) Error 1054 (42S22): Unknown column 'created_date' in 'where clause' Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *mysql.MySQLError

On running 'Collect Data in Full Refresh Mode' everything goes fine again, and the data collection after that fails with the same error.

What do you expect to happen

The incremental data collection to run successfully without SQL errors. If a column like created_date is needed, I’d expect it to be created automatically or handled in a way that avoids failures.

How to reproduce

1. Set up a new DevLake project.
2. Add CircleCI connection.
3. Run the first data collection pipeline in full refresh mode – it completes successfully.
4. Run the same pipeline again (incremental mode) without changing any config.

Anything else

The same happens in a new project on the first run with the same connection. And works fine in the Full Refresh Mode

Version

v1.0.3-beta1(Docker Image)
v1.0.3-beta2(Docker Image)
UPD: v1.0.2 is also impacted.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct