Skip to content

Add JSON Writer #12474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 48 commits into from
Jan 28, 2023
Merged

Add JSON Writer #12474

merged 48 commits into from
Jan 28, 2023

Conversation

karthikeyann
Copy link
Contributor

@karthikeyann karthikeyann commented Jan 4, 2023 •
edited
Loading

Description

Adds JSON writer with nested support.
It supports numeric, datetime, duration, strings, nested types such as struct and list types.
orient='records' is only supported now, with lines=True/False.
Usage: df.to_json(engine='cudf')

closes #11165

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@karthikeyann karthikeyann added feature request New feature or request 2 - In Progress Currently a work in progress libcudf Affects libcudf (C++/CUDA) code. cuIO cuIO issue non-breaking Non-breaking change labels Jan 4, 2023
@karthikeyann karthikeyann added this to the Nested JSON reader milestone Jan 4, 2023
@karthikeyann karthikeyann self-assigned this Jan 4, 2023
@github-actions github-actions bot added the CMake CMake build issue label Jan 4, 2023
@codecov
Copy link

codecov bot commented Jan 11, 2023 •
edited
Loading

Codecov Report

Base: 86.58% // Head: 85.72% // Decreases project coverage by -0.87% ⚠️

Coverage data is based on head (9b92435) compared to base (b6dccb3).
Patch has no changes to coverable lines.

Additional details and impacted files 
@@               Coverage Diff                @@
##           branch-23.02   #12474      +/-   ##
================================================
- Coverage         86.58%   85.72%   -0.87%     
================================================
  Files               155      155              
  Lines             24368    24931     +563     
================================================
+ Hits              21098    21371     +273     
- Misses             3270     3560     +290
Impacted Files Coverage Δ
python/cudf/cudf/_version.py 1.41% <0.00%> (-98.59%) ⬇️
python/cudf/cudf/core/buffer/spill_manager.py 72.50% <0.00%> (-7.50%) ⬇️
python/cudf/cudf/io/json.py 87.35% <0.00%> (-4.71%) ⬇️
python/cudf/cudf/utils/dtypes.py 77.85% <0.00%> (-1.61%) ⬇️
python/cudf/cudf/options.py 86.11% <0.00%> (-1.59%) ⬇️
python/cudf/cudf/core/single_column_frame.py 94.30% <0.00%> (-1.27%) ⬇️
...ython/custreamz/custreamz/tests/test_dataframes.py 98.38% <0.00%> (-1.01%) ⬇️
python/dask_cudf/dask_cudf/io/csv.py 96.34% <0.00%> (-1.00%) ⬇️
python/dask_cudf/dask_cudf/io/parquet.py 91.81% <0.00%> (-0.59%) ⬇️
python/cudf/cudf/core/column/string.py 88.00% <0.00%> (-0.53%) ⬇️
... and 51 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@vuule vuule self-requested a review January 12, 2023 00:23
@karthikeyann karthikeyann marked this pull request as ready for review January 12, 2023 20:28
@karthikeyann karthikeyann requested review from a team as code owners January 12, 2023 20:28
@karthikeyann karthikeyann requested a review from harrism January 12, 2023 20:28
@karthikeyann karthikeyann requested a review from a team as a code owner January 13, 2023 21:05
@github-actions github-actions bot added the Python Affects Python cuDF API. label Jan 13, 2023
vuule
vuule reviewed Jan 14, 2023
View reviewed changes
Copy link
Contributor

@vuule vuule left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API review

galipremsagar
galipremsagar requested changes Jan 24, 2023
View reviewed changes
galipremsagar
galipremsagar reviewed Jan 24, 2023
View reviewed changes
galipremsagar
galipremsagar requested changes Jan 25, 2023
View reviewed changes
Copy link
Contributor

@galipremsagar galipremsagar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last bit remaining..

davidwendt
davidwendt requested changes Jan 25, 2023
View reviewed changes
davidwendt
davidwendt reviewed Jan 27, 2023
View reviewed changes
isVoid
isVoid approved these changes Jan 27, 2023
View reviewed changes
Copy link
Contributor

@isVoid isVoid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cmake approval

robertmaynard
robertmaynard approved these changes Jan 27, 2023
View reviewed changes
Copy link
Contributor

@robertmaynard robertmaynard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CMake changes LGTM

@karthikeyann karthikeyann added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team 4 - Needs cuIO Reviewer labels Jan 27, 2023
@karthikeyann
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 55ef601 into rapidsai:branch-23.02 Jan 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vuule vuule vuule approved these changes

@robertmaynard robertmaynard robertmaynard approved these changes

@galipremsagar galipremsagar galipremsagar approved these changes

@isVoid isVoid isVoid approved these changes

@davidwendt davidwendt davidwendt approved these changes

@harrism harrism Awaiting requested review from harrism harrism is a code owner automatically assigned from rapidsai/cudf-cpp-codeowners

@vyasr vyasr Awaiting requested review from vyasr vyasr is a code owner automatically assigned from rapidsai/cudf-python-codeowners

@brandon-b-miller brandon-b-miller Awaiting requested review from brandon-b-miller brandon-b-miller is a code owner automatically assigned from rapidsai/cudf-python-codeowners

@shwina shwina Awaiting requested review from shwina

Assignees

@karthikeyann karthikeyann

Labels
5 - Ready to Merge Testing and reviews complete, ready to merge CMake CMake build issue cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Milestone
Nested JSON reader
Development

Successfully merging this pull request may close these issues.

[FEA] Add write_json method to cudf/io/json.hpp
6 participants
@karthikeyann @robertmaynard @galipremsagar @isVoid @vuule @davidwendt