-
Notifications
You must be signed in to change notification settings - Fork 4.6k
Add Go example for schema-transform nested field access #37543
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4164,14 +4164,31 @@ as schema fields may have different requirements or restrictions from Go exporte | |
|
|
||
| ### 6.6. Using Schema Transforms {#using-schemas} | ||
|
|
||
| A schema on a `PCollection` enables a rich variety of relational transforms. The fact that each record is composed of | ||
| named fields allows for simple and readable aggregations that reference fields by name, similar to the aggregations in | ||
| a SQL expression. | ||
|
|
||
| {{< paragraph class="language-go">}} | ||
| Beam does not yet support Schema transforms natively in Go. However, it will be implemented with the following behavior. | ||
| In Go, schemas are inferred from struct types. You can use schema-aware | ||
| <code>PCollection</code>s by defining structs and accessing their fields | ||
| directly in transforms. The following example demonstrates extracting | ||
| a nested field from a schema-aware collection. | ||
| {{< /paragraph >}} | ||
|
|
||
| {{< highlight go >}} | ||
| type ShippingAddress struct { | ||
| PostCode string `beam:"postCode"` | ||
| } | ||
| type Purchase struct { | ||
| ShippingAddress ShippingAddress `beam:"shippingAddress"` | ||
| } | ||
| purchases := beam.Create(s, | ||
| Purchase{ | ||
| ShippingAddress: ShippingAddress{PostCode: "12345"}, | ||
| }, | ||
| ) | ||
| postCodes := beam.ParDo(s, func(p Purchase) string { | ||
| return p.ShippingAddress.PostCode | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This just shows how you can access fields of a Go object, it seems unrelated to schemas
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the feedback. I've restored the original schema description . |
||
| }, purchases) | ||
| {{< /highlight >}} | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Instead of dropping this into the top level section, we should follow the same pattern used by the other languages |
||
|
|
||
|
|
||
| #### 6.6.1. Field selection syntax | ||
|
|
||
| The advantage of schemas is that they allow referencing of element fields by name. Beam provides a selection syntax for | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did you remove this?