Skip to content

bpo-19610: Warn if distutils is provided something other than a list to some fields #4685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Dec 5, 2017
Merged
6 changes: 3 additions & 3 deletions Doc/distutils/apiref.rst
Original file line number Diff line number Diff line change
Expand Up @@ -286,9 +286,9 @@ the full reference.
Distribution constructor. :func:`setup` creates a Distribution instance.

.. versionchanged:: 3.7
:class:`~distutils.core.Distribution` now raises a :exc:`TypeError` if
``classifiers``, ``keywords`` and ``platforms`` fields are not specified
as a list.
:class:`~distutils.core.Distribution` now warns if ``classifiers``,
``keywords`` and ``platforms`` fields are not specified as a list or
a string.

.. class:: Command

Expand Down
6 changes: 2 additions & 4 deletions Doc/whatsnew/3.7.rst
Original file line number Diff line number Diff line change
Expand Up @@ -298,10 +298,8 @@ README.rst is now included in the list of distutils standard READMEs and
therefore included in source distributions.
(Contributed by Ryan Gonzalez in :issue:`11913`.)

:class:`distutils.core.setup` now raises a :exc:`TypeError` if
``classifiers``, ``keywords`` and ``platforms`` fields are not specified
as a list. However, to minimize backwards incompatibility concerns,
``keywords`` and ``platforms`` fields still accept a comma separated string.
:class:`distutils.core.setup` now warns if the ``classifiers``, ``keywords``
and ``platforms`` fields are not specified as a list or a string.
(Contributed by Berker Peksag in :issue:`19610`.)

http.client
Expand Down
46 changes: 20 additions & 26 deletions Lib/distutils/dist.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,20 @@
command_re = re.compile(r'^[a-zA-Z]([a-zA-Z0-9_]*)$')


def _ensure_list(value, fieldname):
if isinstance(value, str):
# a string containing comma separated values is okay. It will
# be converted to a list by Distribution.finalize_options().
pass
elif not isinstance(value, list):
# passing a tuple or an iterator perhaps, warn and convert
typename = type(value).__name__
msg = f"Warning: '{fieldname}' should be a list, got type '{typename}'"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the prefix Warning redundant? Otherwise message seems good.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another style nit: Can't we use {fieldname!r} instead of '{fieldname}'?

log.log(log.WARN, msg)
value = list(value)
return value


class Distribution:
"""The core of the Distutils. Most of the work hiding behind 'setup'
is really done within a Distribution instance, which farms the work out
Expand Down Expand Up @@ -257,10 +271,7 @@ def __init__(self, attrs=None):
setattr(self, key, val)
else:
msg = "Unknown distribution option: %s" % repr(key)
if warnings is not None:
warnings.warn(msg)
else:
sys.stderr.write(msg + "\n")
warnings.warn(msg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought you wanted to move this in a separate PR?


# no-user-cfg is handled before other command line args
# because other args override the config files, and this
Expand Down Expand Up @@ -1189,36 +1200,19 @@ def get_keywords(self):
return self.keywords or []

def set_keywords(self, value):
# If 'keywords' is a string, it will be converted to a list
# by Distribution.finalize_options(). To maintain backwards
# compatibility, do not raise an exception if 'keywords' is
# a string.
if not isinstance(value, (list, str)):
msg = "'keywords' should be a 'list', not %r"
raise TypeError(msg % type(value).__name__)
self.keywords = value
self.keywords = _ensure_list(value, 'keywords')

def get_platforms(self):
return self.platforms or ["UNKNOWN"]

def set_platforms(self, value):
# If 'platforms' is a string, it will be converted to a list
# by Distribution.finalize_options(). To maintain backwards
# compatibility, do not raise an exception if 'platforms' is
# a string.
if not isinstance(value, (list, str)):
msg = "'platforms' should be a 'list', not %r"
raise TypeError(msg % type(value).__name__)
self.platforms = value
self.platforms = _ensure_list(value, 'platforms')

def get_classifiers(self):
return self.classifiers or []

def set_classifiers(self, value):
if not isinstance(value, list):
msg = "'classifiers' should be a 'list', not %r"
raise TypeError(msg % type(value).__name__)
self.classifiers = value
self.classifiers = _ensure_list(value, 'classifiers')

def get_download_url(self):
return self.download_url or "UNKNOWN"
Expand All @@ -1231,7 +1225,7 @@ def set_requires(self, value):
import distutils.versionpredicate
for v in value:
distutils.versionpredicate.VersionPredicate(v)
self.requires = value
self.requires = list(value)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add tests for the requires and obsolotes fields similar to what you already added for the classifiers field?:

attrs = {...,
         'classifiers': ('Programming Language :: Python :: 3',)}
self.assertIsInstance(d.metadata.classifiers, list)
self.assertEqual(d.metadata.classifiers,
                 list(attrs['classifiers']))

If I don't miss something, the line list(value) is not covered by the current tests:

https://github.com/python/cpython/blob/bce095d9b0595a528a95bae3f764ee195a0c0d63/Lib/distutils/tests/test_dist.py#L301-L315
https://github.com/python/cpython/blob/bce095d9b0595a528a95bae3f764ee195a0c0d63/Lib/distutils/tests/test_dist.py#L323-L337


def get_provides(self):
return self.provides or []
Expand All @@ -1250,7 +1244,7 @@ def set_obsoletes(self, value):
import distutils.versionpredicate
for v in value:
distutils.versionpredicate.VersionPredicate(v)
self.obsoletes = value
self.obsoletes = list(value)

def fix_help_options(options):
"""Convert a 4-tuple 'help_options' list as found in various command
Expand Down
48 changes: 38 additions & 10 deletions Lib/distutils/tests/test_dist.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@
from distutils.dist import Distribution, fix_help_options, DistributionMetadata
from distutils.cmd import Command

from test.support import TESTFN, captured_stdout, run_unittest
from test.support import (
TESTFN, captured_stdout, captured_stderr, run_unittest
)
from distutils.tests import support
from distutils import log

Expand Down Expand Up @@ -319,6 +321,13 @@ def test_requires_illegal(self):
"version": "1.0",
"requires": ["my.pkg (splat)"]})

def test_requires_to_list(self):
attrs = {"name": "package",
"requires": iter(["other"])}
dist = Distribution(attrs)
self.assertIsInstance(dist.metadata.requires, list)


def test_obsoletes(self):
attrs = {"name": "package",
"version": "1.0",
Expand All @@ -341,6 +350,12 @@ def test_obsoletes_illegal(self):
"version": "1.0",
"obsoletes": ["my.pkg (splat)"]})

def test_obsoletes_to_list(self):
attrs = {"name": "package",
"obsoletes": iter(["other"])}
dist = Distribution(attrs)
self.assertIsInstance(dist.metadata.obsoletes, list)

def test_classifier(self):
attrs = {'name': 'Boa', 'version': '3.0',
'classifiers': ['Programming Language :: Python :: 3']}
Expand All @@ -353,9 +368,14 @@ def test_classifier(self):
def test_classifier_invalid_type(self):
attrs = {'name': 'Boa', 'version': '3.0',
'classifiers': ('Programming Language :: Python :: 3',)}
msg = "'classifiers' should be a 'list', not 'tuple'"
with self.assertRaises(TypeError, msg=msg):
Distribution(attrs)
with captured_stderr() as error:
d = Distribution(attrs)
# should have warning about passing a non-list
self.assertIn('should be a list', error.getvalue())
# should be converted to a list
self.assertIsInstance(d.metadata.classifiers, list)
self.assertEqual(d.metadata.classifiers,
list(attrs['classifiers']))

def test_keywords(self):
attrs = {'name': 'Monty', 'version': '1.0',
Expand All @@ -367,9 +387,13 @@ def test_keywords(self):
def test_keywords_invalid_type(self):
attrs = {'name': 'Monty', 'version': '1.0',
'keywords': ('spam', 'eggs', 'life of brian')}
msg = "'keywords' should be a 'list', not 'tuple'"
with self.assertRaises(TypeError, msg=msg):
Distribution(attrs)
with captured_stderr() as error:
d = Distribution(attrs)
# should have warning about passing a non-list
self.assertIn('should be a list', error.getvalue())
# should be converted to a list
self.assertIsInstance(d.metadata.keywords, list)
self.assertEqual(d.metadata.keywords, list(attrs['keywords']))

def test_platforms(self):
attrs = {'name': 'Monty', 'version': '1.0',
Expand All @@ -381,9 +405,13 @@ def test_platforms(self):
def test_platforms_invalid_types(self):
attrs = {'name': 'Monty', 'version': '1.0',
'platforms': ('GNU/Linux', 'Some Evil Platform')}
msg = "'platforms' should be a 'list', not 'tuple'"
with self.assertRaises(TypeError, msg=msg):
Distribution(attrs)
with captured_stderr() as error:
d = Distribution(attrs)
# should have warning about passing a non-list
self.assertIn('should be a list', error.getvalue())
# should be converted to a list
self.assertIsInstance(d.metadata.platforms, list)
self.assertEqual(d.metadata.platforms, list(attrs['platforms']))

def test_download_url(self):
attrs = {'name': 'Boa', 'version': '3.0',
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
``setup()`` now raises :exc:`TypeError` for invalid types.
``setup()`` now warns about invalid types for some fields.

The ``distutils.dist.Distribution`` class now explicitly raises an exception
when ``classifiers``, ``keywords`` and ``platforms`` fields are not
specified as a list.
The ``distutils.dist.Distribution`` class now warns when ``classifiers``,
``keywords`` and ``platforms`` fields are not specified as a list or a string.