Skip to content

bpo-30934: Document coverage details for idlelib tests #2711

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 16, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions Lib/idlelib/idle_test/README.txt
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,59 @@ complete, though some tests need improvement. To run all htests, run the
htest file from an editor or from the command line with:

python -m idlelib.idle_test.htest


5. Test Coverage

Install the coverage package into your Python 3.6 site-packages
directory. (Its exact location depends on the OS).
> python3 -m pip install coverage
(On Windows, replace 'python3 with 'py -3.6' or perhaps just 'python'.)

The problem with running coverage with repository python is that
coverage uses absolute imports for its submodules, hence it needs to be
in a directory in sys.path. One solution: copy the package to the
directory containing the cpython repository. Call it 'dev'. Then run
coverage either directly or from a script in that directory so that
'dev' is prepended to sys.path.

Either edit or add dev/.coveragerc so it looks something like this.
---
# .coveragerc sets coverage options.
[run]
branch = True

[report]
# Regexes for lines to exclude from consideration
exclude_lines =
# Don't complain if non-runnable code isn't run:
if 0:
if __name__ == .__main__.:

.*# htest #
if not _utest:
if _htest:
---
The additions for IDLE are 'branch = True', to test coverage both ways,
and the last three exclude lines, to exclude things peculiar to IDLE
that are not executed during tests.

A script like the following cover.bat (for Windows) is very handy.
---
@echo off
rem Usage: cover filename [test_ suffix] # proper case required by coverage
rem filename without .py, 2nd parameter if test is not test_filename
setlocal
set py=f:\dev\3x\pcbuild\win32\python_d.exe
set src=idlelib.%1
if "%2" EQU "" set tst=f:/dev/3x/Lib/idlelib/idle_test/test_%1.py
if "%2" NEQ "" set tst=f:/dev/ex/Lib/idlelib/idle_test/test_%2.py

%py% -m coverage run --pylib --source=%src% %tst%
%py% -m coverage report --show-missing
%py% -m coverage html
start htmlcov\3x_Lib_idlelib_%1_py.html
rem Above opens new report; htmlcov\index.html displays report index
---
The second parameter was added for tests of module x not named test_x.
(There were several before modules were renamed, now only one is left.)
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Document coverage details for idlelib tests.

* Add section to idlelib/idle-test/README.txt.

* Include check that branches are taken both ways.

* Exclude IDLE-specific code that does not run during unit tests.