Skip to content

Commit e6e16a2

Browse files
committed
Quick update pass for C Extension documentation
1 parent 60ab03a commit e6e16a2

File tree

1 file changed

+65
-8
lines changed

1 file changed

+65
-8
lines changed

C_EXTENSION.md

Lines changed: 65 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -76,16 +76,73 @@ b'\xe0\x01\x00\xea\xe9\x81\x83\xd6\x87\xb4\x83abc\xd3\x8a!{'
7676

7777
## Development
7878

79-
Architecture of Ion Python C extension:
79+
The Ion Python C extension is built as part of the PEP 517 build process using py-build-cmake, and leaning on
80+
Ion C's existing cmake build. A revision of Ion C is included as a submodule in this repo under `src/ion-c`.
81+
If you would like to update the version of Ion C, simply update the submodule to point to the desired revision.
82+
83+
The file `src/CMakeLists.txt` acts as the build script for the C extension itself, which then includes the Ion C
84+
codebase into the build tree.
85+
86+
With the extension built, it will be exposed to python as `amazon._ioncmodule`. For example:
87+
```python
88+
>>> import amazon._ioncmodule as ionc
89+
>>> ionc.ionc_version()
90+
'v1.1.3 (rev: d61c09a)'
91+
>>>
8092
```
81-
ioncmodule.c
82-
|
83-
|
84-
85-
Ion C -------> Ion C binaries -----> setup.py ------> C extension -------------------> Ion Python simpleion module
86-
compile setup import ionc module
93+
94+
The `amazon.ion.simpleion` module then makes use of this extension when it is available to provide more efficient
95+
Ion parsing. Importing `amazon._ioncmodule` directly can determine if it is available, however simpleion also provides
96+
the field `__IS_C_EXTENSION_SUPPORTED`.
97+
```python
98+
>>> import amazon.ion.simpleion as ion
99+
>>> ion.__IS_C_EXTENSION_SUPPORTED
100+
True
101+
>>>
102+
```
103+
104+
In order to build the extension, along with the package itself, we can use python's build module:
105+
```python
106+
ion-python# python -m build .
107+
* Creating isolated environment: venv+pip...
108+
* Installing packages in isolated environment:
109+
- py-build-cmake~=0.1.8
110+
* Getting build dependencies for sdist...
111+
* Building sdist...
112+
* Building wheel from sdist
113+
* Creating isolated environment: venv+pip...
114+
* Installing packages in isolated environment:
115+
- py-build-cmake~=0.1.8
116+
* Getting build dependencies for wheel...
117+
* Building wheel...
118+
...
119+
Successfully built amazon_ion-0.13.0.tar.gz and amazon_ion-0.13.0-cp310-cp310-linux_x86_64.whl
120+
```
121+
This will build both the source wheel, and the binary wheel for the current system. Installing the module can
122+
be done with pip. Depending on what you're doing with the package you may want to install different dependencies.
123+
Different sets of optional dependencies are provided, such as `test`, and `benchmarking`. More details can be
124+
found in the `pyproject.toml`.
125+
126+
To install the package and dependencies for unit tests you can run:
127+
```python
128+
ion-python# python -m pip install '.[test]'
129+
Processing /ion-python
130+
Installing build dependencies ... done
131+
Getting requirements to build wheel ... done
132+
Preparing metadata (pyproject.toml) ... done
133+
Building wheels for collected packages: amazon_ion
134+
Building wheel for amazon_ion (pyproject.toml) ... done
135+
Created wheel for amazon_ion: filename=amazon_ion-0.13.0-cp310-cp310-linux_x86_64.whl size=573770 sha256=96ef01efea7519a1a38d9fc9e42139ab82125aafa328cfb4a4823458de802fa1
136+
Stored in directory: /root/.cache/pip/wheels/78/55/f9/c6d69051d6a93c725251429f62d0d5b7d16d8c982a3772d666
137+
Successfully built amazon_ion
138+
Installing collected packages: amazon_ion
139+
Attempting uninstall: amazon_ion
140+
Found existing installation: amazon_ion 0.13.0
141+
Uninstalling amazon_ion-0.13.0:
142+
Successfully uninstalled amazon_ion-0.13.0
143+
Successfully installed amazon_ion-0.13.0
87144
```
88-
After setup, C extension will be built and imported to simpleion module. If there are changes in `ioncmodule.c`, build the latest C extension by running `python setup.py build_ext --inplace`.
145+
Installing with `-e` will also allow you to update the python side of the package without having to re-install.
89146

90147

91148
## Technical Details

0 commit comments

Comments
 (0)