[Bug]: ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory #1276

Tubo · 2023-07-15T01:31:55Z

Current Behavior (bug)
When using Python packages via requirements.txt, this time the package is ortools.

This bug was related to #710 and #760.
LD_LIBRARY_PATH is /home/tubo/Sync/Projects/radscheduler/.devbox/nix/profile/default/lib

Content of LD_LIBRARY_PATH is:

libecpg.so       libecpg_compat.so       libpgcommon.a        libpgport_shlib.a     libpython3.so  python3.11
libecpg.so.6     libecpg_compat.so.3     libpgcommon_shlib.a  libpython3.11.so      pgxs           
libecpg.so.6.14  libecpg_compat.so.3.14  libpgport.a          libpython3.11.so.1.0  pkgconfig

Error messages:

============================================================ test session starts =============================================================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
django: settings: radscheduler.settings (from env)
rootdir: /home/tubo/Sync/Projects/radscheduler/radscheduler
configfile: pytest.ini
plugins: django-4.5.2
collected 0 items / 1 error                                                                                                                  

=================================================================== ERRORS ===================================================================
_______________________________________________________ ERROR collecting core/tests.py _______________________________________________________
ImportError while importing test module '/home/tubo/Sync/Projects/radscheduler/radscheduler/core/tests.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/nix/store/xlpbkwgf4nrnr56a3r0k0r32jxlxyf59-python3-3.11.4/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
radscheduler/core/tests.py:2: in <module>
    from .core import model
radscheduler/core/core.py:1: in <module>
    from ortools.sat.python import cp_model
../../../Projects/radscheduler/.devbox/virtenv/python311Packages.pip/.venv/lib/python3.11/site-packages/ortools/sat/python/cp_model.py:56: in <module>
    from ortools.sat.python import swig_helper
E   ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory
========================================================== short test summary info ===========================================================
ERROR radscheduler/core/tests.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================== 1 error in 0.15s ==============================================================

Expected Behavior (fix)
Running pytest should not have this error.

Additional context

OS: KDE Neon (Ubuntu 22.04)
Devbox version: 0.5.5 to 0.5.7
devbox.json:

{
  "packages": [
    "python@latest",
    "python311Packages.pip@latest",
    "openssl@latest",
    "postgresql@latest",
    "python311Packages.numpy@latest"
  ],
  "env": {
    "DJANGO_SETTINGS_MODULE": "radscheduler.settings",
    "PGPORT": "5434"
  },
  "shell": {
    "init_hook": [
      ". $VENV_DIR/bin/activate",
      "pip install -r requirements.txt --use-pep517"
    ],
    "scripts": {
      "create_db": [
        "echo \"Creating DB\"",
        "dropdb --if-exists rs_db",
        "createdb rs_db",
        "psql rs_db -c \"CREATE USER rs_user WITH PASSWORD 'secretpassword';\"",
        "python radscheduler/manage.py makemigrations",
        "python radscheduler/manage.py migrate"
      ],
      "makemigrations": "radscheduler/manage.py makemigrations",
      "migrate": "radscheduler/manage.py migrate",
      "server": "radscheduler/manage.py runserver",
      "test": "pytest radscheduler"
    }
  }
}

requirements.txt:

django==4.2.2
psycopg2==2.9.5
sqlparse==0.4.4
asgiref==3.6.0
pytest==7.4.0
pytest-django==4.5.2
ortools==9.6.2534

The text was updated successfully, but these errors were encountered:

Lagoja · 2023-07-18T18:58:14Z

Hey @Tubo, I'm working on recreating this. Do you have an example of the test you're trying to run with pytest that triggers this issue?

Lagoja · 2023-07-18T20:12:36Z

One thing that might work is running devbox add stdenv.cc.cc.lib, which should add the libstdc++ libraries to your dev environment.

Tubo · 2023-07-19T11:21:02Z

Hey @Tubo, I'm working on recreating this. Do you have an example of the test you're trying to run with pytest that triggers this issue?

It's dead simple:

from django.test import TestCase

def test_pytest():
    assert False

Tubo · 2023-07-19T11:25:54Z

One thing that might work is running devbox add stdenv.cc.cc.lib, which should add the libstdc++ libraries to your dev environment.

Nah didn't work unfortunately:

~/P/radscheduler ❯❯❯ devbox add stdenv.cc.cc.lib

Installing package: stdenv.cc.cc.lib.

[1/1] stdenv.cc.cc.lib
[1/1] stdenv.cc.cc.lib: Success
nix: /home/tubo/Sync/Projects/radscheduler/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /nix/store/nd8dpdgpjkh3g3vvcjx9qdw7mjzz0l90-nix-2.13.3/lib/libnixstore.so)
nix: /home/tubo/Sync/Projects/radscheduler/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /nix/store/nd8dpdgpjkh3g3vvcjx9qdw7mjzz0l90-nix-2.13.3/lib/libnixutil.so)
nix: /home/tubo/Sync/Projects/radscheduler/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /nix/store/wy40dp0iwqs9hch6c6yqxriizzgpgm3z-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-transfer.so)
nix: /home/tubo/Sync/Projects/radscheduler/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /nix/store/wy40dp0iwqs9hch6c6yqxriizzgpgm3z-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-core.so)

Error: There was an error installing nix packages
source: error running "nix profile list": exit status 1

Error: There was an internal error. Run with DEVBOX_DEBUG=1 for a detailed error message, and consider reporting it at https://github.com/jetpack-io/devbox/issues

Debug information:

[DEBUG] 2023/07/19 23:22:28 go.jetpack.io/devbox/internal/boxcli/midcobra/debug.go:72: Command stderr: 
[DEBUG] 2023/07/19 23:22:28 go.jetpack.io/devbox/internal/boxcli/midcobra/debug.go:74: 
ExecutionID:c0c228a182d34f178f2cddfc74900523
error running "nix profile list": exit status 1
go.jetpack.io/devbox/internal/nix.ProfileList
        go.jetpack.io/devbox/internal/nix/profiles.go:47
go.jetpack.io/devbox/internal/nix/nixprofile.ProfileListItems
        go.jetpack.io/devbox/internal/nix/nixprofile/profile.go:28
go.jetpack.io/devbox/internal/impl.(*Devbox).extraPackagesInProfile
        go.jetpack.io/devbox/internal/impl/packages.go:419
go.jetpack.io/devbox/internal/impl.(*Devbox).tidyProfile
        go.jetpack.io/devbox/internal/impl/packages.go:357
go.jetpack.io/devbox/internal/impl.(*Devbox).syncPackagesToProfile
        go.jetpack.io/devbox/internal/impl/packages.go:249
go.jetpack.io/devbox/internal/impl.(*Devbox).ensurePackagesAreInstalled
        go.jetpack.io/devbox/internal/impl/packages.go:207
go.jetpack.io/devbox/internal/impl.(*Devbox).Add
        go.jetpack.io/devbox/internal/impl/packages.go:97
go.jetpack.io/devbox/internal/boxcli.addCmdFunc
        go.jetpack.io/devbox/internal/boxcli/add.go:68
go.jetpack.io/devbox/internal/boxcli.addCmd.func1
        go.jetpack.io/devbox/internal/boxcli/add.go:42
github.com/spf13/cobra.(*Command).execute
        github.com/spf13/[email protected]/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
        github.com/spf13/[email protected]/command.go:1044
github.com/spf13/cobra.(*Command).Execute
        github.com/spf13/[email protected]/command.go:968
go.jetpack.io/devbox/internal/boxcli/midcobra.(*midcobraExecutable).Execute
        go.jetpack.io/devbox/internal/boxcli/midcobra/midcobra.go:58
go.jetpack.io/devbox/internal/boxcli.Execute
        go.jetpack.io/devbox/internal/boxcli/root.go:102
go.jetpack.io/devbox/internal/boxcli.Main
        go.jetpack.io/devbox/internal/boxcli/root.go:123
main.main
        go.jetpack.io/devbox/cmd/devbox/main.go:11
runtime.main
        runtime/proc.go:250
runtime.goexit
        runtime/asm_amd64.s:1598
[DEBUG] 2023/07/19 23:22:28 go.jetpack.io/devbox/internal/impl/dir.go:22: findProjectDir: path is 
[DEBUG] 2023/07/19 23:22:28 go.jetpack.io/devbox/internal/impl/dir.go:66: finding devbox.json in dir: /home/tubo/Projects/radscheduler

Lagoja · 2023-07-19T22:33:41Z

Gotcha that's finding the libstdc file now, but it's not the right version. Let me do a little more debugging from there

aacebedo · 2023-08-25T11:08:05Z

Got the same problem. I cannot import python modules if I do not add the stdlib. But when I do that, other packages such as libsrvg are not working anymore. Unsetting LD_LIBRARY_PATH in the init hook resolve the problem for packages but breaks the import.

savil · 2023-08-25T12:21:47Z

@aacebedo we're digging into this class of errors. Could you please share some files so we can repro it: devbox.json, devbox.lock, requirements.txt, and possibly some command to repro your specific error? While a good fix may take some time, we can try to advise you on how to get unblocked.

Feel free to pull it into a new issue to keep this one scoped to Tubo's scenario?

aacebedo · 2023-08-25T13:00:54Z

Thanks @savil for the answer.

here is my devbox.yaml

{
  "packages": [
    "python310@latest",
    "python310Packages.pip@latest",
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "xorg.xorgserver@latest",
    "[email protected]",
    "[email protected]",
    "stdenv.cc.cc.lib",
    "[email protected]"
  ],
  "shell": {
    "init_hook": [
      ". $VENV_DIR/bin/activate",
      "pip install -q -r requirements.txt"
    ]
  },
  "nixpkgs": {
    "commit": "a16f7eb56e88c8985fcc6eb81dabd6cade4e425a"
  }
}

and the requirements file:

Sphinx==5.3.0
sphinx_rtd_theme==1.1.1
sphinxcontrib-plantuml==0.25
sphinxcontrib-svg2pdfconverter==1.2.1
sphinxcontrib-drawio==0.0.16
sphinxcontrib-bibtex==2.5.0
Pillow==9.3.0
matplotlib==3.7.2

If you try it you will see that python import matplotlib is working but not the srvg-convert binary. If you add
unset LD_LIBRARY_PATH to the init hook, srvg-convert works but not the python import matplotlib with the same error as OP.

savil · 2023-08-25T13:34:13Z

Thanks @aacebedo. I'm not quite able to repro because the drawio nix package appears to be broken, so it won't install (for my intel mac). I can try devbox generate dockerfile and running it in the debian container, assuming the linux package is not broken. What OS are you on?

(I have to pause for a few hours for some personal work, but can look at it a bit later today)

aacebedo · 2023-08-25T13:51:57Z

Thanks ! I am using it on NixOS.

savil · 2023-08-25T18:35:28Z

@aacebedo in a debian-container I can get a devbox shell working.

I can import matplotlib:

(.venv) (devbox) devbox@0616836e2023:/code$ cat main.py
import matplotlib

def main():
    print("hello world")

main()
(.venv) (devbox) devbox@0616836e2023:/code$ python main.py
hello world

but running rsvg-convert seems to complain...

/nix/store/0ccn204v5kds9i08x7plf4x644l6wm87-librsvg-2.52.5/bin/rsvg-convert
/nix/store/0ccn204v5kds9i08x7plf4x644l6wm87-librsvg-2.52.5/bin/rsvg-convert: /nix/store/4s21k8k7p1mfik0b33r2spq5hq7774k1-glibc-2.33-108/lib/libc.so.6: version `GLIBC_2.35' not found (required by /code/.devbox/nix/profile/default/lib/libgcc_s.so.1)
/nix/store/0ccn204v5kds9i08x7plf4x644l6wm87-librsvg-2.52.5/bin/rsvg-convert: /nix/store/4s21k8k7p1mfik0b33r2spq5hq7774k1-glibc-2.33-108/lib/libc.so.6: version `GLIBC_2.34' not found (required by /code/.devbox/nix/profile/default/lib/libgcc_s.so.1)

Is this the error you saw?

savil · 2023-08-25T18:43:03Z

For some reason rsvg is not in PATH for me...

aacebedo · 2023-08-25T18:45:04Z

It is exactly this issue. You can also try convert that exhibit a similar issue.

aacebedo · 2023-08-25T18:45:40Z

you can try adding unset LD_LIBRARY_PATH to the init hook

savil · 2023-08-25T22:48:46Z

@aacebedo I think changing the devbox.lock to use this nixpkgs commit hash for stdenv.cc.cc.lib should unblock you:

     "stdenv.cc.cc.lib": {
      "resolved": "github:NixOS/nixpkgs/c6fd903606866634312e40cceb2caee8c0c9243f#stdenv.cc.cc.lib",
      "source": "nixpkg"
    },

We're investigating more principled fixes.

aacebedo · 2023-09-09T18:14:58Z

Hi,
I tried the modification but without success.
convert and rsvg-convert still does not start with libc.so.6 errors

savil · 2023-09-13T00:54:46Z

@aacebedo I was discussing a similar issue with a user on Discord (is that you as well?).

In that case, something like this helped:

# main.py
import matplotlib

def main():
    print("hello world")

main()

# requirements.txt
matplotlib==3.7.2

# devbox.json
{
  "packages": [
    "python310@latest",
    "python310Packages.pip@latest",
    "drawio@latest",
    "librsvg@latest",
    "github:nixos/nixpkgs/6b70761ea8c896aff8994eb367d9526686501860#stdenv.cc.cc.lib"
  ],
  "shell": {
    "init_hook": [
      "export DEVBOX_DEBUG=1",
      ". $VENV_DIR/bin/activate",
      "pip install -q -r requirements.txt"
    ]
  },
  "nixpkgs": {
    "commit": "73e1976309fc789706b9f306407e9e7622a57d25"
  }
}


# devbox.lock
{
  "lockfile_version": "1",
  "packages": {
    "drawio@latest": {
      "last_modified": "2023-09-10T10:53:27Z",
      "resolved": "github:NixOS/nixpkgs/78058d810644f5ed276804ce7ea9e82d92bee293#drawio",
      "source": "devbox-search",
      "version": "21.6.8"
    },
    "github:nixos/nixpkgs/6b70761ea8c896aff8994eb367d9526686501860#stdenv.cc.cc.lib": {},
    "librsvg@latest": {
      "last_modified": "2023-09-10T10:53:27Z",
      "resolved": "github:NixOS/nixpkgs/78058d810644f5ed276804ce7ea9e82d92bee293#librsvg",
      "source": "devbox-search",
      "version": "2.56.3"
    },
    "python310@latest": {
      "last_modified": "2023-02-24T18:08:35Z",
      "plugin_version": "0.0.1",
      "resolved": "github:NixOS/nixpkgs/806075be2bdde71895359ed18cb530c4d323e6f6#python3",
      "source": "devbox-search",
      "version": "3.10.9"
    },
    "python310Packages.pip@latest": {
      "last_modified": "2023-09-10T10:53:27Z",
      "plugin_version": "0.0.1",
      "resolved": "github:NixOS/nixpkgs/78058d810644f5ed276804ce7ea9e82d92bee293#python310Packages.pip",
      "source": "devbox-search",
      "version": "23.2.1"
    }
  }

The reasoning is:

The error seen was: Original error was: /nix/store/lqz6hmd86viw83f9qll2ip87jhb7p1ah-glibc-2.35-224/lib/libc.so.6: version 'GLIBC_2.36' not found (required by /tmp/test/.devbox/nix/profile/default/lib/libstdc++.so.6)
What's happening is that that the stdenv.cc.cc.lib is expecting a GLIBC 2.36, but the code executed by the python binary (either directly, or indirectly via the matplotlib -> numpy dependencies) is linked against GLIBC 2.35.
I searched https://www.nixhub.io/packages/glibc and version 2.35-224 is from nixpkgs/6b70761ea8c896aff8994eb367d9526686501860#glibc
So, to try fixing it I replaced stdenv.cc.cc.lib with github:nixos/nixpkgs/6b70761ea8c896aff8994eb367d9526686501860#stdenv.cc.cc.lib

aacebedo · 2023-09-14T06:30:51Z

@aacebedo I was discussing a similar issue with a user on Discord (is that you as well?).

In that case, something like this helped:
# main.py
import matplotlib

def main():
    print("hello world")

main()

# requirements.txt
matplotlib==3.7.2

# devbox.json
{
  "packages": [
    "python310@latest",
    "python310Packages.pip@latest",
    "drawio@latest",
    "librsvg@latest",
    "github:nixos/nixpkgs/6b70761ea8c896aff8994eb367d9526686501860#stdenv.cc.cc.lib"
  ],
  "shell": {
    "init_hook": [
      "export DEVBOX_DEBUG=1",
      ". $VENV_DIR/bin/activate",
      "pip install -q -r requirements.txt"
    ]
  },
  "nixpkgs": {
    "commit": "73e1976309fc789706b9f306407e9e7622a57d25"
  }
}


# devbox.lock
{
  "lockfile_version": "1",
  "packages": {
    "drawio@latest": {
      "last_modified": "2023-09-10T10:53:27Z",
      "resolved": "github:NixOS/nixpkgs/78058d810644f5ed276804ce7ea9e82d92bee293#drawio",
      "source": "devbox-search",
      "version": "21.6.8"
    },
    "github:nixos/nixpkgs/6b70761ea8c896aff8994eb367d9526686501860#stdenv.cc.cc.lib": {},
    "librsvg@latest": {
      "last_modified": "2023-09-10T10:53:27Z",
      "resolved": "github:NixOS/nixpkgs/78058d810644f5ed276804ce7ea9e82d92bee293#librsvg",
      "source": "devbox-search",
      "version": "2.56.3"
    },
    "python310@latest": {
      "last_modified": "2023-02-24T18:08:35Z",
      "plugin_version": "0.0.1",
      "resolved": "github:NixOS/nixpkgs/806075be2bdde71895359ed18cb530c4d323e6f6#python3",
      "source": "devbox-search",
      "version": "3.10.9"
    },
    "python310Packages.pip@latest": {
      "last_modified": "2023-09-10T10:53:27Z",
      "plugin_version": "0.0.1",
      "resolved": "github:NixOS/nixpkgs/78058d810644f5ed276804ce7ea9e82d92bee293#python310Packages.pip",
      "source": "devbox-search",
      "version": "23.2.1"
    }
  }
The reasoning is:

The error seen was: Original error was: /nix/store/lqz6hmd86viw83f9qll2ip87jhb7p1ah-glibc-2.35-224/lib/libc.so.6: version 'GLIBC_2.36' not found (required by /tmp/test/.devbox/nix/profile/default/lib/libstdc++.so.6)

What's happening is that that the stdenv.cc.cc.lib is expecting a GLIBC 2.36, but the code executed by the python binary (either directly, or indirectly via the matplotlib -> numpy dependencies) is linked against GLIBC 2.35.

I searched https://www.nixhub.io/packages/glibc and version 2.35-224 is from nixpkgs/6b70761ea8c896aff8994eb367d9526686501860#glibc

So, to try fixing it I replaced stdenv.cc.cc.lib with github:nixos/nixpkgs/6b70761ea8c896aff8994eb367d9526686501860#stdenv.cc.cc.lib

Yes it is me. I thought I could find some help from the community on it. Still I am facing other libc errors on other software installed with devbox so I try to investigate this with the clue about the libc version change you gave me.

savil · 2023-10-06T05:18:39Z

@aacebedo we've been researching a fix for your issue. In this repository https://github.com/savil/devbox-issue-1276, I have a devbox.json and some local flake.nix files that illustrate an approach to solving this problem. The README explains a bit about what its doing.

Could you please try it out in your project, and let us know if this approach works?

If it does, we can build this as a feature in Devbox. Ideally, it should work with minimal user work i.e. no need to manually write your own flake.nix.

aacebedo · 2023-11-01T09:19:45Z

@aacebedo we've been researching a fix for your issue. In this repository https://github.com/savil/devbox-issue-1276, I have a devbox.json and some local flake.nix files that illustrate an approach to solving this problem. The README explains a bit about what its doing.

Could you please try it out in your project, and let us know if this approach works?

If it does, we can build this as a feature in Devbox. Ideally, it should work with minimal user work i.e. no need to manually write your own flake.nix.

Thanks for the work !
Tried it and convert is still exhibiting the issue :(

savil · 2023-11-01T16:31:50Z

@aacebedo That's a bummer. Sorry to hear that. A couple of requests:

Request 1: repro code
I thought we were running the same code as you, so am surprised it is not working. It would help if you could share a github repository with the project code you are trying to run, which exhibits this issue. That would help us debug the exact code you're running.

Request 2: try Devbox 0.8.0-dev's glibc_patch feature
In the upcoming Devbox release (release notes), @gcurtis implemented this approach (with some details modified) as a product feature.

You can use this version of devbox by setting export DEVBOX_USE_VERSION=0.8.0-dev.
You'll need to modify packages in devbox.json to a be json map. And set patch_glibc: true for python and librsvg. Refer to the PR Summary of as an example.
You'll also need to initialize the python virtualenv as shown in the virtenv script in that PR's example.

It would be great if you could try that out, and let us know if it works or not for you. Thanks!

aacebedo · 2023-11-01T16:40:26Z

@aacebedo That's a bummer. Sorry to hear that. A couple of requests:

Request 1: repro code I thought we were running the same code as you, so am surprised it is not working. It would help if you could share a github repository with the project code you are trying to run, which exhibits this issue. That would help us debug the exact code you're running.

Request 2: try Devbox 0.8.0-dev's glibc_patch feature In the upcoming Devbox release (release notes), @gcurtis implemented this approach (with some details modified) as a product feature.

You can use this version of devbox by setting export DEVBOX_USE_VERSION=0.8.0-dev.

You'll need to modify packages in devbox.json to a be json map. And set patch_glibc: true for python and librsvg. Refer to the PR Summary of as an example.

You'll also need to initialize the python virtualenv as shown in the virtenv script in that PR's example.

It would be great if you could try that out, and let us know if it works or not for you. Thanks!

I cloned the devbox-issue-1276 repo and ran devbox shell in it. Then I ran convert and saw the issue :(
I am running devbox 0.7.1 but I'll give a try to the upcoming release as you adviced.

savil · 2023-11-01T16:51:22Z

I cloned the devbox-issue-1276 repo and ran devbox shell in it. Then I ran convert and saw the issue :(

@aacebedo 😱 oh wow. That is odd. Its been a while, but IIRC I'd been running the Dockerfile in that repo to test it out on a debian-slim image (since I have a Mac; this issue affects linux). I'll have to retry my steps in case something changed so that it is no longer working.

savil · 2023-11-06T16:41:28Z

@aacebedo sorry for the delay. After trying out the steps, I think I understand the problem. In my prototype, I've patched python and librsvg using the local flake. Those do work.

You are trying convert that came from the imagemagick nix package. You'll need to patch that via a local flake (as I illustrated with python or librsvg). We don't patch all packages automatically to be conservative, as I explain at the end of the README of https://github.com/savil/devbox-issue-1276, and until we get some validating feedback from real world projects.

Or better, I'd suggest using Devbox 0.8.0-dev and modifying the devbox.json to specify patch_glibc:true for whichever packages are hitting the libc errors. Let us know if that works, and thanks for pursuing this with us!

aacebedo · 2023-11-06T16:44:19Z

@savil thanks a lot to take a look to this !

Ok I'll try the 0.8.0-dev to check this out

aacebedo · 2023-11-06T17:09:33Z

@aacebedo sorry for the delay. After trying out the steps, I think I understand the problem. In my prototype, I've patched python and librsvg using the local flake. Those do work.

You are trying convert that came from the imagemagick nix package. You'll need to patch that via a local flake (as I illustrated with python or librsvg). We don't patch all packages automatically to be conservative, as I explain at the end of the README of https://github.com/savil/devbox-issue-1276, and until we get some validating feedback from real world projects.

Or better, I'd suggest using Devbox 0.8.0-dev and modifying the devbox.json to specify patch_glibc:true for whichever packages are hitting the libc errors. Let us know if that works, and thanks for pursuing this with us!

Ok, so I checked out the v0.8.0-dev, built it (go build ./cmd/devbox), ran a sudo nix-collect-garbage -d.
I exported DEVBOX_USE_VERSION=0.8.0-dev and finally did a devbox shell with the following file:

{
  "packages": {
    "imagemagick": {
      "version": "6.9.11-60",
      "patch_glibc": true
    },
    "librsvg": {
      "version": "2.52.5"
    }
  }
}

and convert still does not work :(

convert: /nix/store/3wa1xwnfv8ada1za1r8m4vmsiz1jifqq-glibc-2.32-35/lib/libc.so.6: version `GLIBC_2.33' not found (required by /tmp/devbox-issue-1276/.devbox/nix/profile/default/lib/librsvg-2.so.2)
convert: /nix/store/3wa1xwnfv8ada1za1r8m4vmsiz1jifqq-glibc-2.32-35/lib/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/nw2gv9l8lzi4scbddbg1vfvivzcpvag3-harfbuzz-3.2.0/lib/libharfbuzz.so.0)

savil · 2023-11-06T19:22:24Z

@aacebedo indeed, I think something is not working correctly. I can repro the error you are seeing. We're looking into it...

savil · 2023-11-06T20:13:08Z

@aacebedo thank you ! @gcurtis and I just looked at this. We added patch_glibc: true to librsvg.

This exposed a couple of issues:

A bug in the current implementation. The patch_glibc was only working for one package. The fix for that is straightforward.
We still do run into an error executing convert. The error has to do with a libpango not finding a g_memdup2 symbol. The root cause of that appears to be that there's a version mismatch with a different library called glib (https://gitlab.gnome.org/GNOME/glib/). We need to investigate a good fix for this, but sadly it'll take a bit of time :(. Will circle back when we have some more news.

aacebedo · 2023-11-06T20:16:15Z

@aacebedo thank you ! @gcurtis and I just looked at this. We added patch_glibc: true to librsvg.

This exposed a couple of issues:

A bug in the current implementation. The patch_glibc was only working for one package. The fix for that is straightforward.

We still do run into an error executing convert. The error has to do with a libpango not finding a g_memdup2 symbol. The root cause of that appears to be that there's a version mismatch with a different library called glib (https://gitlab.gnome.org/GNOME/glib/). We need to investigate a good fix for this, but sadly it'll take a bit of time :(. Will circle back when we have some more news.

Thanks to you both looking into this !
I'll continue to watch the issue and test fixes if it can help to solve this issue.

sandangel · 2024-01-17T05:50:29Z

I have the same issue. I was using poetry command after installing it with devbox add poetry.
Before devbox, when using nix flake, I had to wrap poetry with this:

          poetry = pkgs.symlinkJoin {
            name = "poetry";
            paths = [ pkgs.poetry ];
            buildInputs = [ pkgs.makeWrapper ];
            postBuild = ''
              wrapProgram $out/bin/poetry --set LD_LIBRARY_PATH ${lib.makeLibraryPath [ pksgs.stdenv.cc.cc.lib ]}
            '';
          };
          ciDeps = with pkgs; [
            cmake
          ] ++ [ poetry ];

augustebaum · 2024-05-14T15:43:13Z

Same problem here. Unsetting LD_LIBRARY_PATH does not solve the issue; neither does devbox add stdenv.cc.cc.lib (or any combination of the two).
I also tried managing my Python dependencies by running devbox add [email protected] and following the instructions (installing packages with pip instead of poetry).

EDIT: Solved by setting LD_LIBRARY_PATH to <my-project-path>/.devbox/nix/profile/default/lib/.

EDIT: Not quite solved: My script now works, but:

python3 -c "import numpy" fails with:

(outerbounds-examples-py3.11) ❯ p3 -c 'import numpy'
Traceback (most recent call last):
  File "/home/auguste/Desktop/outerbounds-examples/.venv/lib/python3.11/site-packages/numpy/core/__init__.py", line 24, in <module>
    from . import multiarray
  File "/home/auguste/Desktop/outerbounds-examples/.venv/lib/python3.11/site-packages/numpy/core/multiarray.py", line 10, in <module>
    from . import overrides
  File "/home/auguste/Desktop/outerbounds-examples/.venv/lib/python3.11/site-packages/numpy/core/overrides.py", line 8, in <module>
    from numpy.core._multiarray_umath import (
ImportError: libz.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/auguste/Desktop/outerbounds-examples/.venv/lib/python3.11/site-packages/numpy/__init__.py", line 130, in <module>
    from numpy.__config__ import show as show_config
  File "/home/auguste/Desktop/outerbounds-examples/.venv/lib/python3.11/site-packages/numpy/__config__.py", line 4, in <module>
    from numpy.core._multiarray_umath import (
  File "/home/auguste/Desktop/outerbounds-examples/.venv/lib/python3.11/site-packages/numpy/core/__init__.py", line 50, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.11 from "/home/auguste/Desktop/outerbounds-examples/.venv/bin/python3"
  * The NumPy version is: "1.26.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libz.so.1: cannot open shared object file: No such file or directory


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/auguste/Desktop/outerbounds-examples/.venv/lib/python3.11/site-packages/numpy/__init__.py", line 135, in <module>
    raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

although I think it might've failed even before the fix.

All nix commands now fail with

nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by nix)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by nix)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixexpr.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixexpr.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixmain.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixmain.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixfetchers.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixfetchers.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixstore.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixstore.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixutil.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixutil.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixcmd.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by /nix/store/v7gdmdc9yhx3ngh840xx5kr46840s73b-nix-2.18.2/lib/libnixcmd.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/dgc42gd76af2syp7fk5d9mv2abqrggkg-aws-sdk-cpp-1.11.207/lib/libaws-cpp-sdk-transfer.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/dgc42gd76af2syp7fk5d9mv2abqrggkg-aws-sdk-cpp-1.11.207/lib/libaws-cpp-sdk-s3.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/dgc42gd76af2syp7fk5d9mv2abqrggkg-aws-sdk-cpp-1.11.207/lib/libaws-cpp-sdk-core.so)
nix: /home/auguste/Desktop/outerbounds-examples/.devbox/nix/profile/default/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/a88l22177h3mb4ifiy734mr71c40pfxp-aws-crt-cpp-0.24.7/lib/libaws-crt-cpp.so)

EDIT: Importing numpy does not fail when done in IPython (installed through devbox). Uh, what?

augustebaum · 2024-05-15T12:42:45Z

I ended up using a shell.nix instead, using nixpkgs unstable rather than 23.11, and that seemed to work alright. I'll try devbox again when I have the time.

dwt · 2024-10-02T20:22:28Z

This problem looks very similar to a problem I worked around recently. Maybe it is the same one here?

To See the fix, uncomment the line $DYLD_FALLBACK_LIBRARY_PATH.

Tubo · 2024-10-04T10:39:08Z

Resolved in latest devbox.

Tubo added the bug Something isn't working label Jul 15, 2023

Lagoja added this to Devbox Public Roadmap Sep 14, 2023

github-project-automation bot moved this to Backlog in Devbox Public Roadmap Sep 14, 2023

mvgijssel mentioned this issue Nov 21, 2023

[Bug]: Unable to get Python pandas to work #1641

Open

gcurtis added python Support for Python, pip, Poetry, etc. linker Related to linking (ld, ld.so or dyld) labels Jan 30, 2024

Tubo closed this as completed Oct 4, 2024

github-project-automation bot moved this from Backlog to Merged in Devbox Public Roadmap Oct 4, 2024

ineiti mentioned this issue Oct 30, 2024

[Bug]: ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory #2394

Open

trya2l mentioned this issue Nov 11, 2024

numpy: Error importing numpy: you should not try to import numpy from #2368

Closed

az-wegift mentioned this issue Jan 6, 2025

[Feature request] auto-detect missing libstdc++.so.6 issue, and suggest fix or auto-apply fix #1596

Open

jmarshrossney mentioned this issue Mar 20, 2025

Cannot run Jupyter notebooks with Python from NixPkgs jmarshrossney/portable-jules#5

Open

[Bug]: ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory #1276

[Bug]: ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory #1276

Comments

Tubo commented Jul 15, 2023

Lagoja commented Jul 18, 2023

Uh oh!

Lagoja commented Jul 18, 2023

Uh oh!

Tubo commented Jul 19, 2023

Uh oh!

Tubo commented Jul 19, 2023

Uh oh!

Lagoja commented Jul 19, 2023

Uh oh!

aacebedo commented Aug 25, 2023

Uh oh!

savil commented Aug 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aacebedo commented Aug 25, 2023

Uh oh!

savil commented Aug 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aacebedo commented Aug 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

savil commented Aug 25, 2023

Uh oh!

savil commented Aug 25, 2023

Uh oh!

aacebedo commented Aug 25, 2023

Uh oh!

aacebedo commented Aug 25, 2023

Uh oh!

savil commented Aug 25, 2023

Uh oh!

aacebedo commented Sep 9, 2023

Uh oh!

savil commented Sep 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aacebedo commented Sep 14, 2023

Uh oh!

savil commented Oct 6, 2023

Uh oh!

aacebedo commented Nov 1, 2023

Uh oh!

savil commented Nov 1, 2023

Uh oh!

aacebedo commented Nov 1, 2023

Uh oh!

savil commented Nov 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

savil commented Nov 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aacebedo commented Nov 6, 2023

Uh oh!

aacebedo commented Nov 6, 2023

Uh oh!

savil commented Nov 6, 2023

Uh oh!

savil commented Nov 6, 2023

Uh oh!

aacebedo commented Nov 6, 2023

Uh oh!

sandangel commented Jan 17, 2024

Uh oh!

augustebaum commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

augustebaum commented May 15, 2024

Uh oh!

dwt commented Oct 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Tubo commented Oct 4, 2024

Uh oh!

savil commented Aug 25, 2023 •

edited

Loading

savil commented Aug 25, 2023 •

edited

Loading

aacebedo commented Aug 25, 2023 •

edited

Loading

savil commented Sep 13, 2023 •

edited

Loading

savil commented Nov 1, 2023 •

edited

Loading

savil commented Nov 6, 2023 •

edited

Loading

augustebaum commented May 14, 2024 •

edited

Loading

dwt commented Oct 2, 2024 •

edited

Loading