Decorator with paren tokens in arguments breaks inspect.getsource

[crusaderky]

BPO	38854
Nosy	@PCManticore, @1st1, @tirkarthi, @crusaderky, @AlexWaygood, @ahopkins
PRs	bpo-38854: Adjust inspect.getsource to properly extract source for decorated functions #17374 bpo-46873: Fix inspect.getsource - Count parentheses open/close in decorators #31605

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-11-19.21:37:42.431>
labels = ['type-bug', 'library', '3.9', '3.10']
title = 'Decorator with paren tokens in arguments breaks inspect.getsource'
updated_at = <Date 2022-02-28.11:08:51.171>
user = 'https://github.com/crusaderky'

bugs.python.org fields:

activity = <Date 2022-02-28.11:08:51.171>
actor = 'AlexWaygood'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2019-11-19.21:37:42.431>
creator = 'crusaderky'
dependencies = []
files = []
hgrepos = []
issue_num = 38854
keywords = ['patch']
message_count = 2.0
messages = ['356993', '357432']
nosy_count = 6.0
nosy_names = ['Claudiu.Popa', 'yselivanov', 'xtreak', 'crusaderky', 'AlexWaygood', 'ahopkins']
pr_nums = ['17374', '31605']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue38854'
versions = ['Python 3.9', 'Python 3.10']

Linked PRs

• gh-83035: handle decorator with nested parens in inspect.getsource #99654
• [3.10] gh-83035: handle decorator with nested parens in inspect.getsource (GH-99654) #100080
• [3.11] gh-83035: handle decorator with nested parens in inspect.getsource (GH-99654) #100081

[crusaderky]

Python 3.7.5 and 3.8.0
A decorator causes inspect.getsource() to return clipped output:

from collections import defaultdict
from functools import wraps
import inspect


def foo(*args):
    def decorator(func):
        @wraps(func)
        def wrapper():
            pass
        return wrapper
    return decorator

@foo(dict(), defaultdict(lambda: 1))
def f():
    pass

print(inspect.getsource(f))

Output:

@foo(dict(), defaultdict(lambda: 1))

Expected output:

@foo(dict(), defaultdict(lambda: 1))
def f():
    pass

These changes to the decorator parameters cause the problem to disappear:

• @foo({}, defaultdict(lambda: 1))
• @foo(dict(), defaultdict(list))

[PCManticore]

Thanks for the report!

Turns out this was a bug in inspect.BlockFinder which is responsible for extracting out the source code from a given block. This class was considering ")" inside a decorator call as closing the decorator itself, even when the decorator call was accepting additional arguments that needed parentheses, such as function calls or tuples.

Just submitted a PR #17374 to address this issue.

[carljm]

Looks like there have been three PRs here already, but all of them seem to be incomplete and/or abandoned. Opened a new one: #99654

[AlexWaygood]

Fixed and backported. Thanks @carljm! 🎉