Skip to content

overview.md

Latest commit

 

History

History
75 lines (61 loc) · 13.6 KB

overview.md

File metadata and controls

75 lines (61 loc) · 13.6 KB
title Overview
weight 1
type docs
aliases
/connectors/pipeline-connectors/overview

Connectors

Flink CDC provides several source and sink connectors to interact with external systems. You can use these connectors out-of-box, by adding released JARs to your Flink CDC environment, and specifying the connector in your YAML pipeline definition.

Supported Connectors

Connector Supported Type External System Download Page
[Apache Doris]({{< ref "docs/connectors/pipeline-connectors/doris" >}}) Sink
  • Apache Doris: 1.2.x, 2.x.x, 3.x.x
    		•  Apache Doris
    [Elasticsearch]({{< ref "docs/connectors/pipeline-connectors/elasticsearch" >}}) Sink
  • Elasticsearch: 6.x, 7.x, 8.x
    		•  Elasticsearch
    [Fluss]({{< ref "docs/connectors/pipeline-connectors/fluss" >}}) Sink
  • Fluss: 0.7, 0.8, 0.9
    		•  Fluss
    [Hudi]({{< ref "docs/connectors/pipeline-connectors/hudi" >}}) Sink
  • Apache Hudi
    		•  Apache Hudi
    [Iceberg]({{< ref "docs/connectors/pipeline-connectors/iceberg" >}}) Sink
  • Apache Iceberg: 1.6, 1.7, 1.8, 1.9, 1.10
    		•  Apache Iceberg
    [Kafka]({{< ref "docs/connectors/pipeline-connectors/kafka" >}}) Sink
  • Kafka
    		•  Kafka
    [MaxCompute]({{< ref "docs/connectors/pipeline-connectors/maxcompute" >}}) Sink
  • MaxCompute
    		•  MaxCompute
    [MySQL]({{< ref "docs/connectors/pipeline-connectors/mysql" >}}) Source
  • MySQL: 5.7, 8.0.x, 8.4+
  • RDS MySQL: 5.6, 5.7, 8.0.x
  • PolarDB MySQL: 5.6, 5.7, 8.0.x
  • Aurora MySQL: 5.6, 5.7, 8.0.x
  • MariaDB: 10.x
  • PolarDB X: 2.0.1
    		•  MySQL
    [OceanBase]({{< ref "docs/connectors/pipeline-connectors/oceanbase" >}}) Sink
  • OceanBase: 3.x, 4.x
    		•  OceanBase
    [Oracle]({{< ref "docs/connectors/pipeline-connectors/oracle" >}}) Source
  • Oracle
    		•  Oracle
    [Paimon]({{< ref "docs/connectors/pipeline-connectors/paimon" >}}) Sink
  • Paimon: 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3
    		•  Paimon
    [Postgres]({{< ref "docs/connectors/pipeline-connectors/postgres" >}}) Source
  • PostgreSQL
    		•  Postgres
    [StarRocks]({{< ref "docs/connectors/pipeline-connectors/starrocks" >}}) Sink
  • StarRocks: 2.x, 3.x
    		•  StarRocks

    Supported Flink Versions

    The following table shows the version mapping between Flink CDC Pipeline Connectors and Flink:

    Flink CDC Version Flink Version Pipeline Source Pipeline Sink Notes
    3.6.x 1.20.*, 2.2.* MySQL, PostgreSQL, Oracle StarRocks, Doris, Paimon, Kafka, Elasticsearch, OceanBase, MaxCompute, Iceberg, Fluss, Hudi
    3.5.x 1.19.*, 1.20.* MySQL, PostgreSQL StarRocks, Doris, Paimon, Kafka, Elasticsearch, OceanBase, MaxCompute, Iceberg, Fluss
    3.4.x 1.19.*, 1.20.* MySQL StarRocks, Doris, Paimon, Kafka, Elasticsearch, OceanBase, MaxCompute, Iceberg
    3.3.x 1.19.*, 1.20.* MySQL StarRocks, Doris, Paimon, Kafka, Elasticsearch, OceanBase, MaxCompute
    3.2.x 1.17.*, 1.18.*, 1.19.* MySQL StarRocks, Doris, Paimon, Kafka, Elasticsearch
    3.1.x 1.17.*, 1.18.*, 1.19.* MySQL StarRocks, Doris, Paimon, Kafka Only flink-cdc 3.1.1 supports Flink 1.19
    3.0.x 1.17.*, 1.18.* MySQL StarRocks, Doris

    Develop Your Own Connector

    If provided connectors cannot fulfill your requirement, you can always develop your own connector to get your external system involved in Flink CDC pipelines. Check out [Flink CDC APIs]({{< ref "docs/developer-guide/understand-flink-cdc-api" >}}) to learn how to develop your own connectors.

    {{< top >}}