Processes are spawned while loading

[guptaaka]

My main process gets spawned on multiple cores when I say "import numexpr", even if my code does not use the module anywhere. If I run htop to analyze resource usage for my main task, the memory usage/amount of memory reserved gets split up into multiple tasks, just because the module is imported.
[Edit] Attached below are the snapshots from htop output, the sub-process remain in "Sleep" state all the time, and only the parent process keeps "Running".

[robbmcleod]

Yes, this is true. NumExpr spawns threads on import. What is the issue?

[guptaaka]

The main process gets unnecessarily spawned even if I'm not using numexpr methods and just importing the module at some place.
Say, I run a script which imports and uses module 'x'. x further imports numexpr globally. My current script does not use any of the numexpr function (even indirectly), but still the main process gets spawned.

[robbmcleod]

NumExpr creates a bunch of threads and then they all go to a thread barrier and stop. I'm not clear from your language if you understand the difference between threads and processes?

We made some improvements to the import time last year, you can build from the repo if you want to see if that helps you out:

#291

The time required to create the threads is a small fraction of the import time. If you want to reduce the number of threads, you can edit #define MAX_THREADS 4096 in:

https://github.com/pydata/numexpr/blob/master/numexpr/numexpr_config.hpp

I put the effort in NumExpr3 into having a dynamic thread pool and it was not worth the effort, as it created a bunch of dead-lock bugs that took way too long to iron out.

[robbmcleod]

Dealt with in 694a838 so closing, unless you have further issues.