Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,16 @@ For detailed streaming query guidance, see **[STREAMING.md](https://github.com/c

For type support and examples, see **[TYPES.md](https://github.com/confluentinc/confluent-sql/blob/main/TYPES.md)**.


## Private Networking Considerations

By default, this driver uses the public Confluent Cloud API networking endpoint for the provided cloud provider and region. However, if the Flink database / Kafka cluster you intend to query requires private networking connectivity, then provide the appropriate Flink private networking base URL as the `endpoint` parameter
to `connect()` or `Connection.__init__()`. Refer to the Confluent Cloud [Flink private networking documentation](https://docs.confluent.io/cloud/current/flink/concepts/flink-private-networking.html) for more information on composing your endpoint URL.

Symptoms of using the public endpoint when private networking is required include:
* HTTP 429-related exceptions raised when submitting statements querying tables whose backing Kafka topics / clusters are configured for private networking only.
* Empty or surprisingly missing results when querying `INFORMATION_SCHEMA` or `SHOW TABLES`, due to silent filtering of private-networking-only tables/topics when querying the system catalog.

## Development

### Setup
Expand Down
16 changes: 12 additions & 4 deletions src/confluent_sql/connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,15 @@ def connect( # noqa: PLR0913
provided; must not be provided if endpoint is specified.
database: The default Flink database (Kafka cluster) to use when resolving
table/view/udf names (optional)
endpoint: The base URL for Confluent Cloud API (optional). If not provided, the endpoint
will be constructed based on the cloud_provider and cloud_region parameters in the
format "https://flink.{cloud_region}.{cloud_provider}.confluent.cloud". A trailing
slash is optional and will be stripped if provided (e.g., both
endpoint: The base URL for Confluent Cloud API (optional). If not provided, the public
networking endpoint will be constructed based on the cloud_provider and cloud_region
parameters in the format "https://flink.{cloud_region}.{cloud_provider}.confluent.cloud".
A trailing slash is optional and will be stripped if provided (e.g., both
"https://custom.example.com" and "https://custom.example.com/" are accepted).

If your Kafka clusters / Flink tables require private networking, supply the base
endpoint URL here (`"https://flink.us-east-2.aws.private.confluent.cloud"` for example,
but cases and private networking technologies vary).
dbname: Deprecated alias for database parameter (optional)
result_page_fetch_pause_millis: Maximum milliseconds to wait between fetching pages of
statement results (per statement). Defaults to 100ms. Prevents tight loops of requests
Expand Down Expand Up @@ -215,6 +219,10 @@ def __init__( # noqa: PLR0913
endpoint will be constructed based on the cloud_provider and cloud_region
parameters in the format "https://flink.{cloud_region}.{cloud_provider}.confluent.cloud".
A trailing slash is optional and will be stripped if provided.

If your Kafka clusters / Flink tables require private networking, supply the base
endpoint URL here (`"https://flink.us-east-2.aws.private.confluent.cloud"` for
example, but cases and private networking technologies vary).
database: The name of the database to use (optional)
result_page_fetch_pause_millis: Milliseconds to possibly wait between fetching pages of
statement results. Defaults to 100ms. If most recent fetch of results for a
Expand Down