Skip to content

Commit 075f79a

Browse files
JukkaLsvalentin
andauthored
[mypyc] Updates to dev docs, including debugging segfaults (#18462)
Co-authored-by: Valentin Stanciu <[email protected]>
1 parent ce61d11 commit 075f79a

File tree

1 file changed

+47
-16
lines changed

1 file changed

+47
-16
lines changed

mypyc/doc/dev-intro.md

Lines changed: 47 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -51,11 +51,9 @@ good error message.
5151

5252
Here are some major things that aren't yet supported in compiled code:
5353

54-
* Many dunder methods (only some work, such as `__init__` and `__eq__`)
54+
* Some dunder methods (most work though)
5555
* Monkey patching compiled functions or classes
5656
* General multiple inheritance (a limited form is supported)
57-
* Named tuple defined using the class-based syntax
58-
* Defining protocols
5957

6058
We are generally happy to accept contributions that implement new Python
6159
features.
@@ -73,16 +71,16 @@ compiled code. For example, you may want to do interactive testing or
7371
to run benchmarks. This is also handy if you want to inspect the
7472
generated C code (see Inspecting Generated C).
7573

76-
Run `mypyc` to compile a module to a C extension using your
74+
Run `python -m mypyc` to compile a module to a C extension using your
7775
development version of mypyc:
7876

7977
```
80-
$ mypyc program.py
78+
$ python -m mypyc program.py
8179
```
8280

8381
This will generate a C extension for `program` in the current working
84-
directory. For example, on a Linux system the generated file may be
85-
called `program.cpython-37m-x86_64-linux-gnu.so`.
82+
directory. For example, on a macOS system the generated file may be
83+
called `program.cpython-313-darwin.so`.
8684

8785
Since C extensions can't be run as programs, use `python3 -c` to run
8886
the compiled module as a program:
@@ -95,7 +93,7 @@ Note that `__name__` in `program.py` will now be `program`, not
9593
`__main__`!
9694

9795
You can manually delete the C extension to get back to an interpreted
98-
version (this example works on Linux):
96+
version (this example works on macOS or Linux):
9997

10098
```
10199
$ rm program.*.so
@@ -114,9 +112,9 @@ extensions) in compiled code.
114112

115113
Mypyc will only make compiled code faster. To see a significant
116114
speedup, you must make sure that most of the time is spent in compiled
117-
code -- and not in libraries, for example.
115+
code, and not in libraries or I/O.
118116

119-
Mypyc has these passes:
117+
Mypyc has these main passes:
120118

121119
* Type check the code using mypy and infer types for variables and
122120
expressions. This produces a mypy AST (defined in `mypy.nodes`) and
@@ -193,13 +191,13 @@ information. See the test cases in
193191
`mypyc/test-data/irbuild-basic.test` for examples of what the IR looks
194192
like in a pretty-printed form.
195193

196-
## Testing overview
194+
## Testing Overview
197195

198196
Most mypyc test cases are defined in the same format (`.test`) as used
199197
for test cases for mypy. Look at mypy developer documentation for a
200198
general overview of how things work. Test cases live under
201199
`mypyc/test-data/`, and you can run all mypyc tests via `pytest
202-
-q mypyc`. If you don't make changes to code under `mypy/`, it's not
200+
mypyc`. If you don't make changes to code under `mypy/`, it's not
203201
important to regularly run mypy tests during development.
204202

205203
You can use `python runtests.py mypyc-fast` to run a subset of mypyc
@@ -228,7 +226,7 @@ We also have tests that verify the generate IR
228226

229227
## Type-checking Mypyc
230228

231-
`./runtests.py self` type checks mypy and mypyc. This is pretty slow,
229+
`./runtests.py self` type checks mypy and mypyc. This is a little slow,
232230
however, since it's using an uncompiled mypy.
233231

234232
Installing a released version of mypy using `pip` (which is compiled)
@@ -311,7 +309,7 @@ number of components at once, insensitive to the particular details of
311309
the IR), but there really is no substitute for running code. You can
312310
also write tests that test the generated IR, however.
313311

314-
### Tests that compile and run code
312+
### Tests That Compile and Run Code
315313

316314
Test cases that compile and run code are located in
317315
`mypyc/test-data/run*.test` and the test runner is in
@@ -364,15 +362,48 @@ Test cases can also have a `[out]` section, which specifies the
364362
expected contents of stdout the test case should produce. New test
365363
cases should prefer assert statements to `[out]` sections.
366364

367-
### IR tests
365+
### Debuggging Segfaults
366+
367+
If you experience a segfault, it's recommended to use a debugger that supports
368+
C, such as gdb or lldb, to look into the segfault.
369+
370+
If a test case segfaults, you can run tests using the debugger, so
371+
you can inspect the stack:
372+
373+
```
374+
$ pytest mypyc -n0 -s --mypyc-debug=gdb -k <name-of-test>
375+
```
376+
377+
You must use `-n0 -s` to enable interactive input to the debugger.
378+
Instad of `gdb`, you can also try `lldb`.
379+
380+
To get better C stack tracebacks and more assertions in the Python
381+
runtime, you can build Python in debug mode and use that to run tests
382+
or debug outside the test framework.
383+
384+
Here are some hints that may help (for Ubuntu):
385+
386+
```
387+
$ sudo apt install gdb build-essential libncursesw5-dev libssl-dev libgdbm-dev libc6-dev libsqlite3-dev libbz2-dev libffi-dev libgdbm-compat-dev
388+
$ <download Python tarball and extract it>
389+
$ cd Python-3.XX.Y
390+
$ ./configure --with-pydebug
391+
$ make -s -j16
392+
$ ./python -m venv ~/<venv-location>
393+
$ source ~/<venv-location>/bin/activate
394+
$ cd <mypy-repo-dir>
395+
$ pip install -r test-requirements.txt
396+
```
397+
398+
### IR Tests
368399

369400
If the specifics of the generated IR of a change is important
370401
(because, for example, you want to make sure a particular optimization
371402
is triggering), you should add a `mypyc.irbuild` test as well. Test
372403
cases are located in `mypyc/test-data/irbuild-*.test` and the test
373404
driver is in `mypyc.test.test_irbuild`. IR build tests do a direct
374405
comparison of the IR output, so try to make the test as targeted as
375-
possible so as to capture only the important details. (Many of our
406+
possible so as to capture only the important details. (Some of our
376407
existing IR build tests do not follow this advice, unfortunately!)
377408

378409
If you pass the `--update-data` flag to pytest, it will automatically

0 commit comments

Comments
 (0)