Dataclasses broken after upgrade to python3.10

[sterliakov]

More info, demo project with docker etc. here

I tried to upgrade my python dev environment from 3.9.9 to 3.10.1. Everything worked as expected until I started container with apache2. I spent a few days trying to debug this, but can't find any solution yet.

The following code is enough to reproduce the bug:

from dataclasses import dataclass

@dataclass
class Test:
    foo: str

Test('a')

def application(environ, start_response):
    status = '200 OK'
    output = b'Hello World!'
    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)
    return [output]

When I start server with mod_wsgi-express start-server wsgi.py --port 8000, everything works. When I use apache directly, this script throws error:

File "/var/www/html/proof/wsgi.py", line 9, in <module>
  Test('a')
TypeError: Test() takes no arguments

If I define __init__ in dataclass manually, everything works, but it's not an option in my case. If I change python version to 3.9 in docker, everything works too.

I tried playing with apache config file (using embedded or daemon mode) and using venv and system-wide python and mod_wsgi installations with no result.

Please ask for any additional info if you need, also see repo above. I'm ready to elaborate on this issue if you could point me to proper direction (can't find out what can cause this). I know you don't officially support python3.10, so I understand if you won't try to fix this at all.

P.S. I really appreciate your efforts, thanks a lot! This project made my life much easier.

[GrahamDumpleton]

See #729

As I understand it, it is an issue in Python.

[GrahamDumpleton]

BTW, the reason mod_wsgi-express works is because it is using the main interpreter context and not a sub interpreter.

What is the mod_wsgi configuration you are using in Apache? Use of the main interpreter context is recommended over using sub interpreters. So if you can tell me what the mod_wsgi configuration is I can tell you how to change it to use best practice.

[sterliakov]

Wow, thanks, you saved me! With help of link you provided I managed to resolve: adding WSGIApplicationGroup %{GLOBAL} to apache conf made everything work. Are there any drawbacks of such solution? I have now something like this in config (for django app):

WSGIPythonPath /var/www/html/stomatology/:/usr/local/lib/python3.10/site-packages/:/usr/local/lib/python3.10/lib-dynload:/usr/local/lib/python3.10/
WSGIRestrictEmbedded On
WSGIPassAuthorization On

...

<VirtualHost *:443>
    DocumentRoot /var/www/html/stomatology

    Alias /media /var/www/stomatology/media
    <Directory /var/www/stomatology/media>
        Require all granted
    </Directory>
    Alias /static /var/www/html/stomatology/static_files
    <Directory /var/www/html/stomatology/static_files>
        Require all granted
    </Directory>

    SSLEngine On
    SSLProtocol all -SSLv2 -SSLv3
    SSLCipherSuite ...
    SSLHonorCipherOrder on
    SSLOptions +StrictRequire
    SSLCertificateFile ...
    SSLCertificateKeyFile ...
    SSLCertificateChainFile ...

    WSGIDaemonProcess stomatology processes=1 threads=5 display-name=%{GROUP} home=/var/www/html/stomatology
    WSGIProcessGroup stomatology
    WSGIApplicationGroup %{GLOBAL}
    WSGIScriptAlias / /var/www/html/stomatology/stomatology/wsgi.py
</VirtualHost>

[GrahamDumpleton]

No drawbacks. Using daemon mode and forcing main interpreter context is the recommended config and why mod_wsgi-express uses it.

Only comment is that you should not need:

WSGIPythonPath /var/www/html/stomatology/:/usr/local/lib/python3.10/site-packages/:/usr/local/lib/python3.10/lib-dynload:/usr/local/lib/python3.10/

as WSGIPythonPath only relates to embedded mode and you aren't using that.

You might also consider script preloading by using:

    WSGIDaemonProcess stomatology processes=1 threads=5 display-name=%{GROUP} home=/var/www/html/stomatology
    WSGIScriptAlias / /var/www/html/stomatology/stomatology/wsgi.py process-group=stomatology application-group=%{GLOBAL}

instead of:

    WSGIDaemonProcess stomatology processes=1 threads=5 display-name=%{GROUP} home=/var/www/html/stomatology
    WSGIProcessGroup stomatology
    WSGIApplicationGroup %{GLOBAL}
    WSGIScriptAlias / /var/www/html/stomatology/stomatology/wsgi.py

Also watch:

• https://www.youtube.com/watch?v=H6Q3l11fjU0

for more tips.

[sterliakov]

That's brilliant, thank you! It works.
To sum up: using mod_wsgi within sub interpreter is broken in python3.10.1 due to bug in python itself (class decorators are ignored). Using main interpreter context via setting WSGIApplicationGroup to %{GLOBAL} makes everything work properly.

[GrahamDumpleton]

Correct, there is nothing wrong with mod_wsgi. It is Python core implementation which is broken for sub interpreters in Python 3.10, a regression from 3.9 due to changes made in Python core C code. Even if not using mod_wsgi, if you were using new Python APIs of Python to run stuff in sub interpreters, or if you were using another embedded system using sub interpreters, you would see the same problem. As was linked in that other issue, the bug report for Python core can be found at:

• https://bugs.python.org/issue46006

In other words, Python core needs to be fixed. It isn't mod_wsgi that is the problem.

Thought I would just emphasise it further and give link to that Python bug report here for others who find this one and don't follow that link off to other issue.

[GrahamDumpleton]

Underlying problem was fixed in Python 3.10.2.