Skip to content

[FLINK-39361][connector-http] Improve README with SQL examples#30

Open
featzhang wants to merge 3 commits into
apache:mainfrom
featzhang:feature/improve-readme-sql-examples
Open

[FLINK-39361][connector-http] Improve README with SQL examples#30
featzhang wants to merge 3 commits into
apache:mainfrom
featzhang:feature/improve-readme-sql-examples

Conversation

@featzhang
Copy link
Copy Markdown
Member

What is the purpose of the change

The current README lacks practical usage examples, making it difficult for new users to quickly understand how to configure and use the HTTP connector.

This PR improves the README by adding a Quick Start section that covers the most common connector usage patterns.

Brief change log

  • Add SQL example for HTTP Sink connector (CREATE TABLE + INSERT INTO)
  • Add SQL example for HTTP Lookup Source connector (temporal join pattern)
  • Add DataStream API (Java) example for HttpSink
  • Add a common configuration options table with key properties and descriptions

Verifying this change

This is a documentation-only change. Verification is done by reviewing:

  • SQL syntax correctness against the existing connector options
  • Java API usage matches the public HttpSink builder API
  • Configuration keys match those defined in HttpConnectorConfigConstants and HttpLookupConnectorOptions

Does this pull request potentially affect one of the following areas?

  • Dependencies: No
  • Performance: No
  • Security: No
  • Breaking changes: No

Documentation

This PR is itself a documentation improvement. All changes are in README.md.

Copy link
Copy Markdown
Contributor

@davidradl davidradl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have looked at the Kafka connector and - we follow the style of its readme. The Getindata readme was effectively all of the docs. Now the docs are in a folder that will be picked up and knitted into the flink docs when we release. If you can find a nice place in the docs for this - then please change to add there.

@featzhang
Copy link
Copy Markdown
Member Author

I have looked at the Kafka connector and - we follow the style of its readme. The Getindata readme was effectively all of the docs. Now the docs are in a folder that will be picked up and knitted into the flink docs when we release. If you can find a nice place in the docs for this - then please change to add there.

Thank you for the guidance! That makes total sense — the README should stay lightweight (following the Kafka connector style), and the richer usage content belongs in ⁠docs/⁠ where it gets knitted into the official Flink documentation on release.
I've taken a look at the existing ⁠docs/content/docs/connectors/datastream/http.md⁠ , which already contains the DataStream API usage. I'll move the Quick Start examples (including the SQL/Table API usage) into an appropriate section in the docs rather than keeping them in the README.
I'll update the README to just include a pointer to the docs, and add the Quick Start content into ⁠docs/content/docs/connectors/⁠ — likely under a new section or alongside the existing ⁠http.md⁠ . Will push an update shortly.

Comment thread docs/content/docs/connectors/table/http.md Outdated
Comment thread README.md Outdated
Comment thread docs/content/docs/connectors/table/http.md Outdated
Comment thread docs/content/docs/connectors/table/http.md Outdated
Comment thread docs/content/docs/connectors/table/http.md Outdated
@featzhang featzhang force-pushed the feature/improve-readme-sql-examples branch from 20c61cf to 7698996 Compare March 31, 2026 06:21
@featzhang featzhang changed the title [connector-http] Improve README with SQL examples [FLINK-39361][connector-http] Improve README with SQL examples Mar 31, 2026
@featzhang
Copy link
Copy Markdown
Member Author

Hi @davidradl and @mariyageorge01,

You have both previously contributed to the HTTP connector documentation (docs/content/docs/connectors/table/http.md).

This PR adds a Quick Start section to help new users get started quickly with SQL and DataStream API examples. I have addressed all previous review comments, and the latest commits (pushed on 2026-03-31) have:

  • Reorganized content to follow the style used by other Flink connectors (similar to the Kafka connector docs)
  • Fixed all connector identifiers (http-async-sinkhttp-sink etc.)
  • Added SQL examples for HTTP Sink and Lookup Source
  • Added DataStream API (Java) example for HttpSink
  • Added a configuration options reference table

CI is currently passing ✅ on both Java 11 and Java 17.

Could you please take a look and share your feedback? PTAL, thanks! 🙏

@featzhang
Copy link
Copy Markdown
Member Author

Hi @davidradl,

Thank you for your thorough review! 🙏

I've verified that all your review comments from March 17 have been addressed in the subsequent commits (March 31):

✅ All Issues Fixed:

  1. "the connector is 'http' not 'rest-lookup'" → ✅ Fixed: now using 'connector' = 'http' for lookup source
  2. "this is not required" (README.md) → ✅ Fixed: Quick Start content moved from README to docs folder
  3. "This connector name is not correct" → ✅ Fixed: changed to 'connector' = 'http-async-sink' matching HttpDynamicTableSinkFactory.IDENTIFIER
  4. "not correct for the apache connector" → ✅ Fixed: connector identifiers corrected
  5. "We already document these config options" → ✅ Fixed: removed duplicate configuration table from Quick Start

📋 Current State:

  • All review threads are marked as resolved
  • README.md remains lightweight (following Kafka connector style)
  • Quick Start examples added to docs/content/docs/connectors/table/http.md (both EN and ZH versions)
  • All connector identifiers verified against factory classes in code
  • CI passing on Java 11 & 17

Could you please re-review when you have a moment? All your feedback has been incorporated. Thank you! 🙏

- Add Quick Start section with SQL examples for HTTP Sink and HTTP Lookup
  Source to both English and Chinese documentation
- Fix HTTP Lookup connector name: use 'http' (not 'rest-lookup')
- Add DataStream API example for HTTP Sink (English docs)
- Add Quick Start entry to TOC
…to zh docs

- Fix SQL sink connector identifier: 'http-sink' -> 'http-async-sink' to match
  HttpDynamicTableSinkFactory.IDENTIFIER (both English and Chinese docs)
- Add DataStream API example for HttpSink to Chinese documentation
…c-sink

Fix remaining occurrences of 'http-sink' -> 'http-async-sink' in:
- HTTP Sink table creation example
- Sink Connector Options table description
- Batch submission mode example
- Single submission mode example
(both English and Chinese documentation)
@featzhang featzhang force-pushed the feature/improve-readme-sql-examples branch from db7e3df to ee45277 Compare April 26, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants