Skip to content

[Bug]: ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory #710

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
StephenStrickland opened this issue Mar 1, 2023 · 4 comments
Closed

[Bug]: ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory #710

StephenStrickland opened this issue Mar 1, 2023 · 4 comments
Labels
bug Something isn't working glibc Related to linking against glibc linker Related to linking (ld, ld.so or dyld) linux Linux systems python Support for Python, pip, Poetry, etc.

Comments

@StephenStrickland
Copy link

StephenStrickland commented Mar 1, 2023 •
edited
Loading

Current Behavior (bug)
When using python packages like gradio the package fails when calling into pandas.

Steps:

  1. > devbox shell
  2. > gradio

Error Message

> gradio
Traceback (most recent call last):
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/bin/gradio", line 5, in <module>
    from gradio.reload import run_in_reload_mode
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/gradio/__init__.py", line 3, in <module>
    import gradio.components as components
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/gradio/components.py", line 21, in <module>
    import altair as alt
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/altair/__init__.py", line 4, in <module>
    from .vegalite import *
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/altair/vegalite/__init__.py", line 2, in <module>
    from .v4 import *
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/altair/vegalite/v4/__init__.py", line 2, in <module>
    from .schema import *
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/altair/vegalite/v4/schema/__init__.py", line 2, in <module>
    from .core import *
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/altair/vegalite/v4/schema/core.py", line 4, in <module>
    from altair.utils.schemapi import SchemaBase, Undefined, _subclasses
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/altair/utils/__init__.py", line 1, in <module>
    from .core import (
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/altair/utils/core.py", line 14, in <module>
    import pandas as pd
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/pandas/__init__.py", line 48, in <module>
    from pandas.core.api import (
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/pandas/core/api.py", line 47, in <module>
    from pandas.core.groupby import (
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import (
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 76, in <module>
    from pandas.core.frame import DataFrame
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/pandas/core/frame.py", line 172, in <module>
    from pandas.core.generic import NDFrame
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/pandas/core/generic.py", line 169, in <module>
    from pandas.core.window import (
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/pandas/core/window/__init__.py", line 1, in <module>
    from pandas.core.window.ewm import (
  File "/home/stephen/code/poc/.devbox/virtenv/python310Packages.pip/.venv/lib/python3.10/site-packages/pandas/core/window/ewm.py", line 15, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

Expected Behavior (fix)
Running gradio from devbox shell should not return this error.

This exact error message is listed as a quirk in Nix docs: https://nixos.wiki/wiki/Packaging/Quirks_and_Caveats#ImportError:_libstdc.2B.2B.so.6:_cannot_open_shared_object_file:_No_such_file

Additional context
machine: Windows 10 running Ubuntu WSL2
version: 0.4.1
./devbox.json:

{
  "packages": [
    "python310",
    "python310Packages.pip"
  ],
  "shell": {
    "init_hook": [
      "source $VENV_DIR/bin/activate",
      "pip install -r requirements.txt"
    ]
  },
  "nixpkgs": {
    "commit": "f80ac848e3d6f0c12c52758c0f25c10c97ca3b62"
  }
}

./requirements.txt:

fastapi
gradio
requests
ruff

Note:
This bug is reproducible with pip and poetry. Based devbox.json on example repos from docs.

Workaround

  1. Manually add pandas: devbox add python310Packages.pandas
  2. Refresh shell
@StephenStrickland StephenStrickland added the bug Something isn't working label Mar 1, 2023
@Lagoja
Copy link
Contributor

Lagoja commented Mar 1, 2023

Hey @StephenStrickland -- can you try devbox add stdenv.cc.cc.lib to see if that fixes the issue?

@StephenStrickland
Copy link
Author

Didn't work.
Steps:

  1. devbox rm python310Packages.pandas
  2. devbox shell
  3. gradio, fails as expected
  4. exit
  5. devbox add stdenv.cc.cc.lib
  6. devbox shell
  7. gradio
  8. Outputs same error message in ticket.

Relevant devbox.json:

{
  "packages": [
    "python310",
    "poetry",
    "stdenv.cc.cc.lib"
  ],
}

@Lagoja
Copy link
Contributor

Lagoja commented Mar 2, 2023 •
edited
Loading

Hey @StephenStrickland

Did a little more testing, and I got a pytorch + gradio example (https://huggingface.co/spaces/gradio/pictionary/tree/main) to work using the following devbox.json

{
  "packages": [
    "python310",
    "python310Packages.pip",
    "stdenv.cc.cc.lib"
  ],
  "shell": {
    "init_hook": [
      "export LD_LIBRARY_PATH=`pwd`/.devbox/nix/profile/default/lib:/usr/wsl/lib:$LD_LIBRARY_PATH",
      "source $VENV_DIR/bin/activate",
      "pip install -r requirements.txt"
    ]
  },
  "nixpkgs": {
    "commit": "f80ac848e3d6f0c12c52758c0f25c10c97ca3b62"
  }
}

The key here is the line export LD_LIBRARY_PATH=pwd/.devbox/nix/profile/default/lib:/usr/wsl/lib:$LD_LIBRARY_PATH. This adds the stdenv.cc.cc.lib libraries to your LD path, as well as the NVIDIA GPU drivers (which are needed for PyTorch).

I think we could automatically add the lib folder to the LD_LIBRARY_PATH variable, along with a few other flags.

@Lagoja Lagoja mentioned this issue Mar 14, 2023
Closed
@Lagoja
Copy link
Contributor

Lagoja commented Mar 15, 2023

Going to close this issue in favor of #760, which we'll use to track the feature request of automatically setting the LD_LIBRARY_PATH variable. Thanks again for reporting!

@Lagoja Lagoja closed this as completed Mar 15, 2023
@mikeland73 mikeland73 mentioned this issue Mar 21, 2023
Merged
mikeland73 added a commit that referenced this issue Mar 21, 2023
@mikeland73
[lib-path] Add a few common env variables (#792)
5ee847b 
## Summary

Adds `LD_LIBRARY_PATH` and `LIBRARY_PATH` env vars to shellenv (pointing to profile lib directory). 

This is a bit of a blunt approach, so not sure if it's best. This would fix #760 and also lets us simplify the [rust plugin](https://github.com/jetpack-io/devbox/blob/main/plugins/rustup.json#L7-L7)

Two questions/considerations:

* I considered adding to the flake.nix file instead, but wasn't example sure how to point to the right directory (without hard coding). It might look something like `LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath...` 
* I also read the following:

> Not setting LD_LIBRARY_PATH is usually the right choice because usually the native binaries find their libraries using RUNPATH which is specific to the binary. Setting LD_LIBRARY_PATH has an obvious problem of potentially messing up other programs.

@gcurtis any idea how accurate this is?

## How was it tested?

`devbox shell` and confirmed env vars are set.

Also tested with this example: #710 (comment)
@Tubo Tubo mentioned this issue Jul 15, 2023
Closed
@gcurtis gcurtis added python Support for Python, pip, Poetry, etc. linux Linux systems linker Related to linking (ld, ld.so or dyld) glibc Related to linking against glibc labels Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working glibc Related to linking against glibc linker Related to linking (ld, ld.so or dyld) linux Linux systems python Support for Python, pip, Poetry, etc.
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gcurtis @Lagoja @StephenStrickland