Skip to content

Add khiops-env to conda packages #343

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

bruno-at-orange
Copy link
Contributor

@bruno-at-orange bruno-at-orange commented Aug 5, 2024

Update of the khiops_env script on Linux/Mac and Windows to be compatible with conda environments (even without CONDA_PREFIX). Conda packages are updated with khiops_env (on all OS). It has an impact on packaging (cmake, NSIS) and github actions.
The important comments are in commit a98e684
Add a new variable KHIOPS_MPI_HOST_FILE (in Linux only) to set up the MPI command line on multi-machines
Remove process affinity on windows

close #225

@bruno-at-orange bruno-at-orange linked an issue Aug 5, 2024 that may be closed by this pull request
@bruno-at-orange bruno-at-orange force-pushed the 258-provide-the-khiops-env-script-for-all-packages branch 5 times, most recently from 875233b to d490460 Compare August 9, 2024 08:31
@bruno-at-orange bruno-at-orange force-pushed the 258-provide-the-khiops-env-script-for-all-packages branch 12 times, most recently from 0acec33 to bed62a6 Compare August 21, 2024 13:30
@bruno-at-orange bruno-at-orange force-pushed the 258-provide-the-khiops-env-script-for-all-packages branch 12 times, most recently from bc5fe99 to 6b2c249 Compare August 28, 2024 16:44
@bruno-at-orange bruno-at-orange force-pushed the 258-provide-the-khiops-env-script-for-all-packages branch 8 times, most recently from 49a22ce to 4f4759d Compare September 25, 2024 14:56
@bruno-at-orange bruno-at-orange force-pushed the 258-provide-the-khiops-env-script-for-all-packages branch 2 times, most recently from 255f7b6 to f001e74 Compare September 27, 2024 15:01
@bruno-at-orange bruno-at-orange force-pushed the 258-provide-the-khiops-env-script-for-all-packages branch 4 times, most recently from 05b8af7 to fc06c05 Compare September 30, 2024 09:03
Add a new tiny binary to show the physical core number. It is only used by khiops_env.
This binary is packaged with khiops, so there is an impact on conda, deb, rpm and NSIS.
@bruno-at-orange bruno-at-orange force-pushed the 258-provide-the-khiops-env-script-for-all-packages branch from fc06c05 to 9447868 Compare September 30, 2024 09:19
Add KHIOPS_MPI_HOST_FILE to khiops_env
- Allows to specify a hostfile to mpiexec
- if this variable is defined, khiops_env will export environment variables required by khiops to run correctly to MPI processes.
These variables relate to GCS/S3 credentials and to khiops itself (KHIOPS_* and Khiops*)
khiops_env is built on cmake configure: the variable export is done only for openmpi and the hostfile keyword is different for openmpi and mpich

Remove process affinity for MS-MPI
- As discussed in #350, the affinity slows down khiops.
- remove the prioryty (below normal) too

Improve UNIXes khiops_env
- java configuration and modules loading are not in khiops_env on conda environment
- To improve the install.cmake readability, 3 files are added to fill variables:
	- export_env_variables.sh
	- java_settings.sh
	- use_environment_module.sh

fix GUI error message in khiops and khiops coclustering
- The message saying to install "khiops" on conda environment is wrong.
- when a non-existing option is passed, e.g. khiops --foo, the GUI message is displayed

fix message in help about KHIOPS_API_MODE: false <=> true

KHIOPS_API_MODE message
bug fix: when running tests without using kht_test, we have to set the variable KHIOPS_API_MODE to true
@bruno-at-orange bruno-at-orange force-pushed the 258-provide-the-khiops-env-script-for-all-packages branch from 9447868 to be2aa9c Compare September 30, 2024 12:20
Copy link
Collaborator

@popescu-v popescu-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bruno-at-orange bruno-at-orange merged commit 3057e60 into dev Sep 30, 2024
66 checks passed
@bruno-at-orange bruno-at-orange deleted the 258-provide-the-khiops-env-script-for-all-packages branch September 30, 2024 13:08
bruno-at-orange added a commit that referenced this pull request Oct 1, 2024
Update of the khiops_env script on Linux/Mac and Windows to be compatible with conda environments (even without CONDA_PREFIX). Conda packages are updated with khiops_env (on all OS). It has an impact on packaging (cmake, NSIS) and github actions.
Add a new variable KHIOPS_MPI_HOST_FILE (in Linux only) to set up the MPI command line on multi-machines
Remove process affinity on windows

backport #343  from dev
@popescu-v popescu-v mentioned this pull request Oct 2, 2024
thekester pushed a commit to thekester/khiops that referenced this pull request Mar 22, 2025
…env-script-for-all-packages

Add khiops-env to conda packages
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide the khiops-env script for all packages Test Conda packages on Iris and Adult
3 participants