Commit 75757e1
[SPARK-56169][SQL] Fix
### What changes were proposed in this pull request?
SPARK-53470 added `ExpectsInputTypes` to `GetStructField` so that `checkInputDataTypes()` catches the case where a plan transformation changes the child's type from `StructType` to something else. This can happen when an analyzer rule inserts a projection that changes a column's output type after `GetStructField` was already created referencing that column.
However, when `CheckAnalysis` detects this mismatch, the error formatting path (`toPrettySQL` -> `usePrettyExpression`) accesses `GetStructField.dataType` which calls `childSchema` -> `child.dataType.asInstanceOf[StructType]`, throwing a raw `ClassCastException` before the proper `DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE` error can be reported.
This PR fixes two things:
1. `usePrettyExpression` checks `child.dataType` before accessing `childSchema`, falling back to a safe representation when the child is not a `StructType`
2. `childSchema` uses pattern matching instead of an unsafe cast, throwing a clear `SparkException.internalError` instead of `ClassCastException`
### Why are the changes needed?
Without this fix, users see a raw `ClassCastException: StringType$ cannot be cast to StructType` instead of the proper `DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE` error that `checkInputDataTypes()` was trying to report.
### Does this PR introduce _any_ user-facing change?
Yes - users now get a proper `AnalysisException` with error class `DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE` instead of a raw `ClassCastException`.
### How was this patch tested?
New tests.
### Was this patch authored or co-authored using generative AI tooling?
Yes
Co-authored-by: Claude
Closes #54970 from ilicmarkodb/fix-get-struct-field-classcast.
Authored-by: ilicmarkodb <marko.ilic@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>ClassCastException in error reporting when GetStructField child type is changed by plan transformation1 parent 34ac927 commit 75757e1
3 files changed
Lines changed: 44 additions & 7 deletions
File tree
- sql/catalyst/src
- main/scala/org/apache/spark/sql/catalyst
- expressions
- util
- test/scala/org/apache/spark/sql/catalyst/expressions
Lines changed: 8 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
184 | 184 | | |
185 | 185 | | |
186 | 186 | | |
187 | | - | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
188 | 194 | | |
189 | 195 | | |
190 | 196 | | |
| |||
Lines changed: 15 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
102 | 112 | | |
103 | 113 | | |
104 | 114 | | |
| |||
Lines changed: 21 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
177 | 177 | | |
178 | 178 | | |
179 | 179 | | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
180 | 201 | | |
181 | 202 | | |
182 | 203 | | |
| |||
0 commit comments