Skip to content

gh-113785: csv: fields starting with escapechar are not quoted #122110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 25, 2024

Conversation

MKuranowski
Copy link
Contributor

@MKuranowski MKuranowski commented Jul 22, 2024

This PR fixes an inconsistency in the _csv.c module, whereby escapechars at the beginning of the field imply that fields are quoted, but escape characters at any other position do not. This only really affects parsing with QUOTE_NONNUMERIC - fields starting with the escape character are incorrectly left as a string, instead of being parsed.

With this change escapechars do not imply a quoted field; which allows fields like "\\.5" with QUOTE_NONNUMERIC to be correctly parsed as 0.5.

Closes #113785.

@ghost
Copy link

ghost commented Jul 22, 2024

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

bedevere-app bot commented Jul 22, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@@ -0,0 +1 @@
:mod:`csv` now correctly parses numeric fields (when used with :const:`csv.QUOTE_NONNUMERIC`) which start with an escape character.
Copy link
Member

@serhiy-storchaka serhiy-storchaka Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or csv.QUOTE_STRINGS.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in cf05e97

@serhiy-storchaka serhiy-storchaka added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes labels Jul 23, 2024
Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@serhiy-storchaka serhiy-storchaka merged commit a3327db into python:main Jul 25, 2024
39 checks passed
@miss-islington-app
Copy link

Thanks @MKuranowski for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 25, 2024
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 25, 2024
@bedevere-app
Copy link

bedevere-app bot commented Jul 25, 2024

GH-122258 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Jul 25, 2024
@bedevere-app
Copy link

bedevere-app bot commented Jul 25, 2024

GH-122259 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 only security fixes label Jul 25, 2024
@serhiy-storchaka
Copy link
Member

Thank you for your contribution @MKuranowski.

serhiy-storchaka pushed a commit that referenced this pull request Jul 25, 2024
serhiy-storchaka pushed a commit that referenced this pull request Aug 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CSV reader inconsistent with combination of QUOTE_NONNUMERIC and escapechar
2 participants