Commit 842eb7b
[SPARK-54938][PYTHON][TEST][FOLLOW-UP] Fix inferred time unit for pandas >= 3
### What changes were proposed in this pull request?
Fix inferred time unit for pandas >= 3
### Why are the changes needed?
there is behavior change in pandas 3
### Does this PR introduce _any_ user-facing change?
No, test-only
### How was this patch tested?
manually check
pandas=2.3.3
```
In [7]: pd.__version__
Out[7]: '2.3.3'
In [8]: pd.Series(pd.to_datetime(["2024-01-01", "2024-01-02"])).dtype
Out[8]: dtype('<M8[ns]')
In [9]: pa.array(pd.Series(pd.to_datetime(["2024-01-01", "2024-01-02"]))).type
Out[9]: TimestampType(timestamp[ns])
```
pandas=3.0.1
```
In [6]: pd.__version__
Out[6]: '3.0.1'
In [7]: pd.Series(pd.to_datetime(["2024-01-01", "2024-01-02"])).dtype
Out[7]: dtype('<M8[us]')
In [8]: pa.array(pd.Series(pd.to_datetime(["2024-01-01", "2024-01-02"]))).type
Out[8]: TimestampType(timestamp[us])
```
### Was this patch authored or co-authored using generative AI tooling?
Co-authored-by: Claude code (Opus 4.6)
Closes #55158 from zhengruifeng/fix-pyarrow-ts-inference.
Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>1 parent d9c8eda commit 842eb7b
1 file changed
Lines changed: 12 additions & 10 deletions
Lines changed: 12 additions & 10 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
299 | 299 | | |
300 | 300 | | |
301 | 301 | | |
| 302 | + | |
| 303 | + | |
302 | 304 | | |
303 | 305 | | |
304 | 306 | | |
| |||
324 | 326 | | |
325 | 327 | | |
326 | 328 | | |
327 | | - | |
328 | | - | |
329 | | - | |
330 | | - | |
331 | | - | |
332 | | - | |
333 | | - | |
334 | | - | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
335 | 337 | | |
336 | | - | |
337 | | - | |
| 338 | + | |
| 339 | + | |
338 | 340 | | |
339 | 341 | | |
340 | 342 | | |
| |||
0 commit comments