Skip to content

[Bug?]: "Couldn't allocate enough memory" #3580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 task
mikob opened this issue Oct 15, 2021 · 11 comments
Closed
1 task

[Bug?]: "Couldn't allocate enough memory" #3580

mikob opened this issue Oct 15, 2021 · 11 comments
Labels
bug Something isn't working stale Issues that didn't get attention

Comments

@mikob
Copy link

mikob commented Oct 15, 2021

Self-service

  • I'd be willing to implement a fix

Describe the bug

when running yarn install after about 1.5 minutes I get this error. I've tried upping --max_old_space_size in the NODE_OPTIONS up to 15000mb, it doesn't seem to have an effect.

To reproduce

I have a monorepo with multiple packages, some are private. This repo seems to be the culprit:

  "devDependencies": {
    "@ava/cooperate": "^0.2.0",
    "@babel/polyfill": "^7.4.4",
    "@rollup/plugin-node-resolve": "^13.0.5",
    "@trust/webcrypto": "^0.9.2",
    "@types/amplitude-js": "^4.4.4",
    "@types/chrome": "~0.0.159",
    "@types/esprima": "^4.0.1",
    "@types/jsdom": "^11.0.4",
    "@types/lodash": "^4.14.165",
    "@types/sinon": "^4.3.0",
    "@types/stripe-checkout": "^1.0.3",
    "@types/tmp": "^0.1.0",
    "@vitejs/plugin-vue": "^1.2.2",
    "@vue/eslint-config-typescript": "7.0.0",
    "ava": "^3.15.0",
    "chrome-remote-interface": "^0.28.0",
    "cli-table": "^0.3.1",
    "commander": "^7.2.0",
    "concurrently": "^5.1.0",
    "copy-webpack-plugin": "^8.1.1",
    "css-loader": "^5.2.4",
    "css-minimizer-webpack-plugin": "^2.0.0",
    "deepdash": "^5.2.1",
    "esbuild": "^0.13.6",
    "esm": "^3.2.25",
    "file-loader": "^6.0.0",
    "fork-ts-checker-webpack-plugin": "^6.2.1",
    "globby": "^11.0.4",
    "html-loader": "^1.1.0",
    "html-minimizer-webpack-plugin": "^2.1.0",
    "ignore-emit-webpack-plugin": "^2.0.2",
    "image-minimizer-webpack-plugin": "^2.2.0",
    "imagemin-gifsicle": "^7.0.0",
    "imagemin-jpegtran": "^7.0.0",
    "imagemin-optipng": "^8.0.0",
    "imagemin-svgo": "^9.0.0",
    "jsdom": "^15.0.0",
    "magic-string": "^0.25.7",
    "mini-css-extract-plugin": "^1.5.0",
    "mock-cloud-firestore": "^0.9.1",
    "moment": "^2.29.1",
    "node-gtts": "^2.0.2",
    "nodemon": "^2.0.12",
    "nyc": "^13.1.0",
    "postcss-import": "^14.0.2",
    "raw-loader": "^4.0.1",
    "remove-files-webpack-plugin": "^1.4.4",
    "rollup": "^2.58.0",
    "rollup-plugin-commonjs": "^10.1.0",
    "rollup-plugin-esbuild": "^4.5.0",
    "rollup-plugin-postcss": "^4.0.1",
    "rollup-plugin-svg": "^2.0.0",
    "rollup-plugin-vue": "^6.0.0",
    "sinon": "^11.1.1",
    "size-plugin": "^1.1.1",
    "style-loader": "^2.0.0",
    "terser": "^5.7.1",
    "terser-webpack-plugin": "^5.1.4",
    "tmp": "^0.1.0",
    "ts-babel-node": "looker/ts-babel-node",
    "ts-loader": "^8.1.0",
    "ts-node": "^10.1.0",
    "tslib": "^1.9.3",
    "typescript": "4.5.0-dev.20211015",
    "vite": "^2.3.3",
    "vue-loader": "^16.2.0",
    "vue-style-loader": "^4.1.3",
    "vue-svg-loader": "^0.17.0-beta.2",
    "vuepress": "^2.0.0-beta.8",
    "webpack": "^5.36.2",
    "webpack-bugsnag-plugins": "^1.6.0",
    "webpack-bundle-analyzer": "^4.4.1",
    "webpack-cli": "4.6.0",
    "webpack-dev-server": "^3.11.2",
    "webpack-merge": "^5.7.3",
    "ws": "^7.3.0"
  },
  "dependencies": {
    "@bugsnag/js": "^6.3.0",
    "amplitude-js": "^5.6.0",
    "awesomplete": "^1.1.4",
    "firebase": "^8.10.0",
    "firebaseui": "^4.8.0",
    "kuromoji": "https://github.com/lipsurf/kuromoji.js#indexeddb-cache",
    "lipsurf-cli": "https://github.com/lipsurf/cli",
    "lipsurf-common": "file:../common",
    "lipsurf-plugins": "https://github.com/lipsurf/plugins",
    "lipsurf-types": "https://github.com/lipsurf/types",
    "lodash": "^4.17.20",
    "premium-plugins": "file:../premium-plugins",
    "primeicons": "^4.1.0",
    "primevue": "^3.4.0",
    "tippy.js": "^6.3.1",
    "vue": "^3.0.11",
    "vue-router": "^4.0.10",
    "vue-slider-component": "^4.0.0-beta.3"
  },
  "packageManager": "[email protected]"

Environment

System:
    OS: Linux 5.11 Linux Mint 20.2 (Uma)
    CPU: (12) x64 Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz
  Binaries:
    Node: 14.16.1 - /tmp/xfs-59a439ba/node
    Yarn: 3.0.2 - /tmp/xfs-59a439ba/yarn
    npm: 6.14.12 - ~/.nvm/versions/node/v14.16.1/bin/npm

Additional context

No response

@mikob mikob added the bug Something isn't working label Oct 15, 2021
@mikob
Copy link
Author

mikob commented Oct 16, 2021

Narrowed it down to this dep being the issue:
"premium-plugins": "file:../premium-plugins",

perhaps because it's also listed in the parent folder's package.json as a workspace?

@yarnbot
Copy link
Collaborator

yarnbot commented Nov 15, 2021

Hi! 👋

This issue looks stale, and doesn't feature the reproducible label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).

Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolutions). It helps us help you! 😃

If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded label. Thanks for helping us triaging our repository! 🌟

@yarnbot yarnbot added the stale Issues that didn't get attention label Nov 15, 2021
@yarnbot yarnbot closed this as completed Nov 20, 2021
@abdolian
Copy link

abdolian commented Sep 8, 2022

I had used .npmrc with a package-lock=false value which was the cause of this error. Remove package-lock=false.

@ltfschoen
Copy link

ltfschoen commented Sep 10, 2023

Please kindly consider re-opening this issue. I believe this issue is still relevant on Linux too, not just to Windows in this existing issue #3972

I've created a sandbox to demonstrate the issue with error Couldn't allocate enough memory here

To replicate the issue follow the below instructions.

I'm using a Macbook Pro Max M2 with 64Gb memory, plenty of it is available.
I've already configured Docker for Mac Settings to use up to 40Gb out of my 64Gb of memory.
The Dockerfile has been configured to use Ubuntu 23.04 to generate a Docker container using aarch64 and Node.js 20.6.1 and Yarn 3.6.3.

  • Run the following to clone the repo and use branch 'eip-x-yarn3'
git clone https://github.com/ltfschoen/lodestar
cd lodestar
git fetch origin eip-x-yarn3:eip-x-yarn3
git checkout eip-x-yarn3
  • Install and run Docker

  • Run the following in terminal

time ./docker/docker.sh

It will output error, without showing the details

...
➤ YN0000: Failed with errors in 45s 358ms

real	0m46.226s
user	0m22.300s
sys	0m7.837s
  • To investigate the errors deeper enter the Docker container by running:
docker exec -w /eip-x/lodestar -it --user=root lodestar-dev /bin/bash
  • Run the following to modify the file .yarn/releases/yarn-3.6.3.cjs so that all instances of text maxOpenFiles:80 are changed to maxOpenFiles:1 instead (trying to see if the kind suggestion by @cxcorp mentioned here would work)
sed -i 's/maxOpenFiles:80/maxOpenFiles:1/g' .yarn/releases/yarn-3.6.3.cjs
  • Manually check that the file has changed as intended

  • Try to install dependencies using Yarn 3 by running the following, where it shouldn't timeout for several days

time yarn install --network-timeout 1000000000
  • View error that is generated very quickly
...
➤ YN0001: │ Error: Couldn't allocate enough memory
    at ZipFS.allocateBuffer ([worker eval]:1:40610)
    at ZipFS.allocateSource ([worker eval]:1:41045)
    at ZipFS.setFileSource ([worker eval]:1:41318)
    at ZipFS.writeFileSync ([worker eval]:1:46409)
    at extractArchiveTo ([worker eval]:1:466938)
    at async MessagePort.<anonymous> ([worker eval]:1:468587)
➤ YN0000: └ Completed in 13s 805ms
➤ YN0000: Failed with errors in 45s 358ms

real	0m46.226s
user	0m22.300s
sys	0m7.837s

@ltfschoen
Copy link

ltfschoen commented Sep 10, 2023

Please kindly consider re-opening this issue. I believe this issue is still relevant on Linux too, not just to Windows in this existing issue #3972

I've created a sandbox to demonstrate the issue with error Couldn't allocate enough memory.

To replicate the issue follow the below instructions.

I'm using a Macbook Pro Max M2 with 64Gb memory. I've already configured Docker for Mac Settings to use up to 40Gb out of my 64Gb of memory. The Dockerfile has been configured to use Ubuntu 23.04 to generate a Docker container using aarch64 and Node.js 20.6.1 and Yarn 3.6.3.

  • Run the following to clone the repo and use branch 'eip-x-yarn3'
git clone https://github.com/ltfschoen/lodestar
cd lodestar
git fetch origin eip-x-yarn3:eip-x-yarn3
git checkout eip-x-yarn3
  • Install and run Docker
  • Run the following in terminal
time ./docker/docker.sh

It will output error, without showing the details

...
➤ YN0000: Failed with errors in 45s 358ms

real	0m46.226s
user	0m22.300s
sys	0m7.837s
  • To investigate the errors deeper enter the Docker container by running:
docker exec -w /eip-x/lodestar -it --user=root lodestar-dev /bin/bash
  • Run the following to modify the file .yarn/releases/yarn-3.6.3.cjs so that all instances of text maxOpenFiles:80 are changed to maxOpenFiles:1 instead (trying to see if the kind suggestion by @cxcorp mentioned here would work)
sed -i 's/maxOpenFiles:80/maxOpenFiles:1/g' .yarn/releases/yarn-3.6.3.cjs
  • Manually check that the file has changed as intended
  • Try to install dependencies using Yarn 3 by running the following, where it shouldn't timeout for several days
time yarn install --network-timeout 1000000000
  • View error that is generated very quickly
...
➤ YN0001: │ Error: Couldn't allocate enough memory
    at ZipFS.allocateBuffer ([worker eval]:1:40610)
    at ZipFS.allocateSource ([worker eval]:1:41045)
    at ZipFS.setFileSource ([worker eval]:1:41318)
    at ZipFS.writeFileSync ([worker eval]:1:46409)
    at extractArchiveTo ([worker eval]:1:466938)
    at async MessagePort.<anonymous> ([worker eval]:1:468587)
➤ YN0000: └ Completed in 13s 805ms
➤ YN0000: Failed with errors in 45s 358ms

real	0m46.226s
user	0m22.300s
sys	0m7.837s

Note: The upstream lodestar repository actually still uses Yarn 1, but i'm trying to switch it to Yarn 3, mainly because installing it with the following approach only installs Yarn 0.32+git (but I want at least Yarn 1)

curl -fsSL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo gpg --dearmor -o /usr/share/keyrings/yarn.gpg && \
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /usr/share/keyrings/yarn.list
apt-get update
apt-get install -y --no-install-recommends yarn

And if I instead installed it with the following, which usually installs Yarn 1.x

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo tee /etc/apt/trusted.gpg.d/yarn.gpg
gpg --no-default-keyring --keyring /etc/apt/trusted.gpg.d/yarn.gpg --export > /etc/apt/trusted.gpg.d/yarn.fixed.gpg
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
apt-get update
apt-get install -y --no-install-recommends yarn

Then I get error:

...
-----END PGP PUBLIC KEY BLOCK-----
gpg: [don't know]: invalid packet (ctb=2d)
gpg: WARNING: nothing exported
gpg: key export failed: Invalid packet

And if I remove the linegpg --no-default-keyring --keyring /etc/apt/trusted.gpg.d/yarn.gpg --export > /etc/apt/trusted.gpg.d/yarn.fixed.gpg that is shown above, which was recommended here https://stackoverflow.com/a/54411131/3208553, then when i run it again I get error:

...
-----END PGP PUBLIC KEY BLOCK-----
deb https://dl.yarnpkg.com/debian/ stable main
Hit:1 https://deb.nodesource.com/node_20.x nodistro InRelease
Get:2 https://dl.yarnpkg.com/debian stable InRelease [17.1 kB]
Err:2 https://dl.yarnpkg.com/debian stable InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY XXX
Hit:3 http://ports.ubuntu.com/ubuntu-ports lunar InRelease
Hit:4 http://ports.ubuntu.com/ubuntu-ports lunar-updates InRelease
Hit:5 http://ports.ubuntu.com/ubuntu-ports lunar-backports InRelease
Hit:6 http://ports.ubuntu.com/ubuntu-ports lunar-security InRelease
Reading package lists...
W: https://dl.yarnpkg.com/debian/dists/stable/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/yarn.gpg are ignored as the file has an unsupported filetype.
W: GPG error: https://dl.yarnpkg.com/debian stable InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY XXX
E: The repository 'https://dl.yarnpkg.com/debian stable InRelease' is not signed.
W: http://ports.ubuntu.com/ubuntu-ports/dists/lunar/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/yarn.gpg are ignored as the file has an unsupported filetype.
W: http://ports.ubuntu.com/ubuntu-ports/dists/lunar-updates/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/yarn.gpg are ignored as the file has an unsupported filetype.
W: http://ports.ubuntu.com/ubuntu-ports/dists/lunar-backports/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/yarn.gpg are ignored as the file has an unsupported filetype.
W: http://ports.ubuntu.com/ubuntu-ports/dists/lunar-security/InRelease: The key(s) in the keyring /etc/apt/trusted.gpg.d/yarn.gpg are ignored as the file has an unsupported filetype.

@ltfschoen
Copy link

ltfschoen commented Sep 10, 2023

i tried running the following based on discussion here, but still same error

export NODE_OPTIONS="--no-huge-max-old-generation-size --max-old-space-size=8192" && yarn install --network-timeout 100000000

and i ran the following in parallel a few times and memory was never shown as depleted

var os = require('os');
os.totalmem();
os.freemem();

@ltfschoen
Copy link

ltfschoen commented Sep 10, 2023

so i get it to work using Yarn 1.22.19 and Node.js v20.6.1 and NPM 9.8.1 using Ubuntu 22.04 in my commit here ltfschoen/lodestar@462050b

so as mentioned earlier, my Dockerfile has been configured to use Ubuntu 23.04, but i just found https://github.com/nodesource/distributions#ubuntu-versions, which shows that Ubuntu Jammy ^22.04 is supported by Node 16x, Node 18x, and Node 20x, which i interpret as meaning that only minor and patch updates of Ubuntu 22.x are supported, but not major version of Ubuntu like 23.x. so i reverted to using Ubuntu 22.04 incase that is the cause of the error, and tried installing Node.js 20x. as it turned out that seemed to be the solution.
i was able to run the following and whilst it installed Yarn 0.32+git, which i didn't want, it didn't generate an error, but at least it also installed NPM, which i don't think it did with Ubuntu 23.x.
also previously when i'd run apt-get install yarn it'd say selecting 'cmdtest' instead of 'yarn' and keep using Yarn 0.32+git instead of Yarn 1.x, even if i ran apt remove cmdtest` first.

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo tee /etc/apt/trusted.gpg.d/yarn.gpg
gpg --no-default-keyring --keyring /etc/apt/trusted.gpg.d/yarn.gpg --export > /etc/apt/trusted.gpg.d/yarn.fixed.gpg
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
apt-get update
apt-get install -y --no-install-recommends yarn

but with Node.js v20.6.1 and NPM 9.8.1 installed, i followed what this comment said yarnpkg/yarn#2821 (comment), which involved simply following https://yarnpkg.com/en/docs/install#linux-tab to install Yarn, by running npm install --global yarn (which worked since NPM was installed when using 22.04 of Ubuntu), and yarn --version returned 1.22.19.

so then i ran yarn cache clean --all, and then ran export NODE_OPTIONS="--no-huge-max-old-generation-size --max-old-space-size=8192" && yarn install --network-timeout 1000000000

but that gave error

error /eip-x/lodestar/node_modules/v8-profiler-next: Command failed.
Exit code: 7
Command: node-gyp rebuild
Arguments:
Directory: /eip-x/lodestar/node_modules/v8-profiler-next
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | arm64
gyp info find Python using Python version 3.10.12 found at "/usr/bin/python3"
gyp http GET https://nodejs.org/download/release/v20.6.1/node-v20.6.1-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v20.6.1/node-v20.6.1-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v20.6.1/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v20.6.1/SHASUMS256.txt
gyp ERR! UNCAUGHT EXCEPTION
gyp ERR! stack TypeError: Cannot assign to read only property 'cflags' of object '#<Object>'
gyp ERR! stack     at createConfigFile (/eip-x/lodestar/node_modules/node-gyp/lib/configure.js:118:21)
gyp ERR! stack     at /eip-x/lodestar/node_modules/node-gyp/lib/configure.js:85:9
gyp ERR! stack     at /eip-x/lodestar/node_modules/mkdirp/index.js:30:20
gyp ERR! stack     at FSReqCallback.oncomplete (node:fs:189:23)
gyp ERR! System Linux 5.15.49-linuxkit-pr
gyp ERR! command "/usr/bin/node" "/eip-x/lodestar/node_modules/.bin/node-gyp" "rebuild"
gyp ERR! cwd /eip-x/lodestar/node_modules/v8-profiler-next
gyp ERR! node -v v20.6.1
gyp ERR! node-gyp -v v5.1.0

warning Error running install script for optional dependency: "/eip-x/lodestar/node_modules/sse4_crc32: Command failed.
Exit code: 7
Command: node-gyp rebuild
Arguments:
Directory: /eip-x/lodestar/node_modules/sse4_crc32
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | arm64
gyp info find Python using Python version 3.10.12 found at \"/usr/bin/python3\"
gyp ERR! UNCAUGHT EXCEPTION
gyp ERR! stack TypeError: Cannot assign to read only property 'cflags' of object '#<Object>'
gyp ERR! stack     at createConfigFile (/eip-x/lodestar/node_modules/node-gyp/lib/configure.js:118:21)
gyp ERR! stack     at /eip-x/lodestar/node_modules/node-gyp/lib/configure.js:85:9
gyp ERR! stack     at /eip-x/lodestar/node_modules/mkdirp/index.js:30:20
gyp ERR! stack     at FSReqCallback.oncomplete (node:fs:189:23)
gyp ERR! System Linux 5.15.49-linuxkit-pr
gyp ERR! command \"/usr/bin/node\" \"/eip-x/lodestar/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /eip-x/lodestar/node_modules/sse4_crc32
gyp ERR! node -v v20.6.1
gyp ERR! node-gyp -v v5.1.0
gyp ERR! This is a bug in `node-gyp`.
gyp ERR! Try to update node-gyp and file an Issue if it does not help:
gyp ERR!     <https://github.com/nodejs/node-gyp/issues>"
info This module is OPTIONAL, you can safely ignore this error
warning Error running install script for optional dependency: "/eip-x/lodestar/node_modules/scrypt: Command failed.
Exit code: 7
Command: node-gyp rebuild
Arguments:
Directory: /eip-x/lodestar/node_modules/scrypt
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | arm64
gyp info find Python using Python version 3.10.12 found at \"/usr/bin/python3\"
gyp ERR! UNCAUGHT EXCEPTION
gyp ERR! stack TypeError: Cannot assign to read only property 'cflags' of object '#<Object>'
gyp ERR! stack     at createConfigFile (/eip-x/lodestar/node_modules/node-gyp/lib/configure.js:118:21)
gyp ERR! stack     at /eip-x/lodestar/node_modules/node-gyp/lib/configure.js:85:9
gyp ERR! stack     at /eip-x/lodestar/node_modules/mkdirp/index.js:30:20
gyp ERR! stack     at FSReqCallback.oncomplete (node:fs:189:23)
gyp ERR! System Linux 5.15.49-linuxkit-pr
gyp ERR! command \"/usr/bin/node\" \"/eip-x/lodestar/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /eip-x/lodestar/node_modules/scrypt
gyp ERR! node -v v20.6.1
gyp ERR! node-gyp -v v5.1.0
gyp ERR! This is a bug in `node-gyp`.
gyp ERR! Try to update node-gyp and file an Issue if it does not help:
gyp ERR!     <https://github.com/nodejs/node-gyp/issues>"
info This module is OPTIONAL, you can safely ignore this error

so i checked node-gyp --version and it was 5.1.0, so i added the following to the package.json file "node-gyp": "^9.0.4", and then ran the following again

export NODE_OPTIONS="--no-huge-max-old-generation-size --max-old-space-size=8192" && \
yarn install --network-timeout 1000000000

but then i got a lot more errors...

error /eip-x/lodestar/node_modules/v8-profiler-next: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments:
Directory: /eip-x/lodestar/node_modules/v8-profiler-next
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | arm64
gyp info find Python using Python version 3.10.12 found at "/usr/bin/python3"
gyp http GET https://nodejs.org/download/release/v20.6.1/node-v20.6.1-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v20.6.1/node-v20.6.1-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v20.6.1/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v20.6.1/SHASUMS256.txt
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/eip-x/lodestar/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/eip-x/lodestar/node_modules/v8-profiler-next/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/eip-x/lodestar/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/root/.cache/node-gyp/20.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/root/.cache/node-gyp/20.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/eip-x/lodestar/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/root/.cache/node-gyp/20.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/eip-x/lodestar/node_modules/v8-profiler-next',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/eip-x/lodestar/node_modules/v8-profiler-next/build'
  CXX(target) Release/obj.target/profiler/src/profiler.o
In file included from ../../nan/nan.h:176,
                 from ../src/profiler.cc:2:
../../nan/nan_callbacks.h:55:23: error: ‘AccessorSignature’ is not a member of ‘v8’
   55 | typedef v8::Local<v8::AccessorSignature> Sig;
      |                       ^~~~~~~~~~~~~~~~~
../../nan/nan_callbacks.h:55:40: error: template argument 1 is invalid
   55 | typedef v8::Local<v8::AccessorSignature> Sig;
      |                                        ^
In file included from ../src/profiler.cc:2:
../../nan/nan.h: In function ‘bool Nan::IdleNotification(int)’:
../../nan/nan.h:682:63: warning: ‘bool v8::Isolate::IdleNotificationDeadline(double)’ is deprecated: Use MemoryPressureNotification() to influence the GC schedule. [-Wdeprecated-declarations]
  682 |     return v8::Isolate::GetCurrent()->IdleNotificationDeadline(
      |            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
  683 |         idle_time_in_ms * 0.001);
      |         ~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /root/.cache/node-gyp/20.6.1/include/node/v8-initialization.h:13,
                 from /root/.cache/node-gyp/20.6.1/include/node/v8.h:34,
                 from /root/.cache/node-gyp/20.6.1/include/node/node.h:73,
                 from ../src/profiler.cc:1:
/root/.cache/node-gyp/20.6.1/include/node/v8-isolate.h:1293:8: note: declared here
 1293 |   bool IdleNotificationDeadline(double deadline_in_seconds);
      |        ^~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../src/profiler.cc:2:
../../nan/nan.h: In function ‘void Nan::AsyncQueueWorker(Nan::AsyncWorker*)’:
../../nan/nan.h:2294:7: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
 2294 |     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../nan/nan.h: In function ‘void Nan::SetAccessor(v8::Local<v8::ObjectTemplate>, v8::Local<v8::String>, Nan::GetterCallback, Nan::SetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, Nan::imp::Sig)’:
../../nan/nan.h:2535:19: error: no matching function for call to ‘v8::ObjectTemplate::SetAccessor(v8::Local<v8::String>&, void (*&)(v8::Local<v8::Name>, const v8::PropertyCallbackInfo<v8::Value>&), void (*&)(v8::Local<v8::Name>, v8::Local<v8::Value>, const v8::PropertyCallbackInfo<void>&), v8::Local<v8::Object>&, v8::AccessControl&, v8::PropertyAttribute&, Nan::imp::Sig&)’
 2535 |   tpl->SetAccessor(
      |   ~~~~~~~~~~~~~~~~^
 2536 |       name
      |       ~~~~
 2537 |     , getter_
      |     ~~~~~~~~~
 2538 |     , setter_
      |     ~~~~~~~~~
 2539 |     , obj
      |     ~~~~~
 2540 |     , settings
      |     ~~~~~~~~~~
 2541 |     , attribute
      |     ~~~~~~~~~~~
 2542 |     , signature);
      |     ~~~~~~~~~~~~
In file included from /root/.cache/node-gyp/20.6.1/include/node/v8-function.h:15,
                 from /root/.cache/node-gyp/20.6.1/include/node/v8.h:33,
                 from /root/.cache/node-gyp/20.6.1/include/node/node.h:73,
                 from ../src/profiler.cc:1:
/root/.cache/node-gyp/20.6.1/include/node/v8-template.h:809:8: note: candidate: ‘void v8::ObjectTemplate::SetAccessor(v8::Local<v8::String>, v8::AccessorGetterCallback, v8::AccessorSetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, v8::SideEffectType, v8::SideEffectType)’
  809 |   void SetAccessor(
      |        ^~~~~~~~~~~
/root/.cache/node-gyp/20.6.1/include/node/v8-template.h:814:22: note:   no known conversion for argument 7 from ‘Nan::imp::Sig’ {aka ‘int’} to ‘v8::SideEffectType’
  814 |       SideEffectType getter_side_effect_type = SideEffectType::kHasSideEffect,
      |       ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/.cache/node-gyp/20.6.1/include/node/v8-template.h:816:8: note: candidate: ‘void v8::ObjectTemplate::SetAccessor(v8::Local<v8::Name>, v8::AccessorNameGetterCallback, v8::AccessorNameSetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, v8::SideEffectType, v8::SideEffectType)’
  816 |   void SetAccessor(
      |        ^~~~~~~~~~~
/root/.cache/node-gyp/20.6.1/include/node/v8-template.h:821:22: note:   no known conversion for argument 7 from ‘Nan::imp::Sig’ {aka ‘int’} to ‘v8::SideEffectType’
  821 |       SideEffectType getter_side_effect_type = SideEffectType::kHasSideEffect,
      |       ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../src/profiler.cc:1:
../src/profiler.cc: At global scope:
/root/.cache/node-gyp/20.6.1/include/node/node.h:1170:7: warning: cast between incompatible function types from ‘void (*)(v8::Local<v8::Object>)’ to ‘node::addon_register_func’ {aka ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, void*)’} [-Wcast-function-type]
 1170 |       (node::addon_register_func) (regfunc),                          \
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/.cache/node-gyp/20.6.1/include/node/node.h:1204:3: note: in expansion of macro ‘NODE_MODULE_X’
 1204 |   NODE_MODULE_X(modname, regfunc, NULL, 0)  // NOLINT (readability/null_usage)
      |   ^~~~~~~~~~~~~
../src/profiler.cc:15:1: note: in expansion of macro ‘NODE_MODULE’
   15 | NODE_MODULE(profiler, InitializeProfiler)
      | ^~~~~~~~~~~
make: *** [profiler.target.mk:120: Release/obj.target/profiler/src/profiler.o] Error 1
make: Leaving directory '/eip-x/lodestar/node_modules/v8-profiler-next/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/eip-x/lodestar/node_modules/node-gyp/lib/build.js:203:23)
gyp ERR! stack     at ChildProcess.emit (node:events:514:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Linux 5.15.49-linuxkit-pr
gyp ERR! command "/usr/bin/node" "/eip-x/lodestar/node_modules/.bin/node-gyp" "rebuild"
gyp ERR! cwd /eip-x/lodestar/node_modules/v8-profiler-next




warning Error running install script for optional dependency: "/eip-x/lodestar/node_modules/scrypt: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments:
Directory: /eip-x/lodestar/node_modules/scrypt
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | arm64
gyp info find Python using Python version 3.10.12 found at \"/usr/bin/python3\"
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/eip-x/lodestar/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/eip-x/lodestar/node_modules/scrypt/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/eip-x/lodestar/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/root/.cache/node-gyp/20.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/root/.cache/node-gyp/20.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/eip-x/lodestar/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/root/.cache/node-gyp/20.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/eip-x/lodestar/node_modules/scrypt',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/eip-x/lodestar/node_modules/scrypt/build'
  SOLINK_MODULE(target) Release/obj.target/copied_files.node
  COPY Release/copied_files.node
  CC(target) Release/obj.target/scrypt_wrapper/src/util/memlimit.o
  CC(target) Release/obj.target/scrypt_wrapper/src/scryptwrapper/keyderivation.o
  CC(target) Release/obj.target/scrypt_wrapper/src/scryptwrapper/pickparams.o
  CC(target) Release/obj.target/scrypt_wrapper/src/scryptwrapper/hash.o
rm -f Release/obj.target/scrypt_wrapper.a Release/obj.target/scrypt_wrapper.a.ar-file-list; mkdir -p `dirname Release/obj.target/scrypt_wrapper.a`
ar crs Release/obj.target/scrypt_wrapper.a @Release/obj.target/scrypt_wrapper.a.ar-file-list
  COPY Release/scrypt_wrapper.a
  CC(target) Release/obj.target/scrypt_lib/scrypt/scrypt-1.2.0/lib/crypto/crypto_scrypt.o
  CC(target) Release/obj.target/scrypt_lib/scrypt/scrypt-1.2.0/lib/crypto/crypto_scrypt_smix.o
  CC(target) Release/obj.target/scrypt_lib/scrypt/scrypt-1.2.0/libcperciva/util/warnp.o
  CC(target) Release/obj.target/scrypt_lib/scrypt/scrypt-1.2.0/libcperciva/alg/sha256.o
  CC(target) Release/obj.target/scrypt_lib/scrypt/scrypt-1.2.0/libcperciva/util/insecure_memzero.o
  CC(target) Release/obj.target/scrypt_lib/scrypt/scrypt-1.2.0/lib/scryptenc/scryptenc_cpuperf.o
rm -f Release/obj.target/scrypt_lib.a Release/obj.target/scrypt_lib.a.ar-file-list; mkdir -p `dirname Release/obj.target/scrypt_lib.a`
ar crs Release/obj.target/scrypt_lib.a @Release/obj.target/scrypt_lib.a.ar-file-list
  COPY Release/scrypt_lib.a
  CXX(target) Release/obj.target/scrypt/src/node-boilerplate/scrypt_common.o
In file included from ../../nan/nan.h:176,
                 from ../src/node-boilerplate/scrypt_common.cc:28:
../../nan/nan_callbacks.h:55:23: error: ‘AccessorSignature’ is not a member of ‘v8’
   55 | typedef v8::Local<v8::AccessorSignature> Sig;
      |                       ^~~~~~~~~~~~~~~~~
../../nan/nan_callbacks.h:55:40: error: template argument 1 is invalid
   55 | typedef v8::Local<v8::AccessorSignature> Sig;
      |                                        ^
In file included from ../src/node-boilerplate/scrypt_common.cc:28:
../../nan/nan.h: In function ‘bool Nan::IdleNotification(int)’:
../../nan/nan.h:682:63: warning: ‘bool v8::Isolate::IdleNotificationDeadline(double)’ is deprecated: Use MemoryPressureNotification() to influence the GC schedule. [-Wdeprecated-declarations]
  682 |     return v8::Isolate::GetCurrent()->IdleNotificationDeadline(
      |            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
  683 |         idle_time_in_ms * 0.001);
      |         ~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /root/.cache/node-gyp/20.6.1/include/node/v8-initialization.h:13,
                 from /root/.cache/node-gyp/20.6.1/include/node/v8.h:34,
                 from /root/.cache/node-gyp/20.6.1/include/node/node.h:73,
                 from ../../nan/nan.h:56,
                 from ../src/node-boilerplate/scrypt_common.cc:28:
/root/.cache/node-gyp/20.6.1/include/node/v8-isolate.h:1293:8: note: declared here
 1293 |   bool IdleNotificationDeadline(double deadline_in_seconds);
      |        ^~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../src/node-boilerplate/scrypt_common.cc:28:
../../nan/nan.h: In function ‘void Nan::AsyncQueueWorker(Nan::AsyncWorker*)’:
../../nan/nan.h:2294:7: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
 2294 |     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../nan/nan.h: In function ‘void Nan::SetAccessor(v8::Local<v8::ObjectTemplate>, v8::Local<v8::String>, Nan::GetterCallback, Nan::SetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, Nan::imp::Sig)’:
../../nan/nan.h:2535:19: error: no matching function for call to ‘v8::ObjectTemplate::SetAccessor(v8::Local<v8::String>&, void (*&)(v8::Local<v8::Name>, const v8::PropertyCallbackInfo<v8::Value>&), void (*&)(v8::Local<v8::Name>, v8::Local<v8::Value>, const v8::PropertyCallbackInfo<void>&), v8::Local<v8::Object>&, v8::AccessControl&, v8::PropertyAttribute&, Nan::imp::Sig&)’
 2535 |   tpl->SetAccessor(
      |   ~~~~~~~~~~~~~~~~^
 2536 |       name
      |       ~~~~
 2537 |     , getter_
      |     ~~~~~~~~~
 2538 |     , setter_
      |     ~~~~~~~~~
 2539 |     , obj
      |     ~~~~~
 2540 |     , settings
      |     ~~~~~~~~~~
 2541 |     , attribute
      |     ~~~~~~~~~~~
 2542 |     , signature);
      |     ~~~~~~~~~~~~
In file included from /root/.cache/node-gyp/20.6.1/include/node/v8-function.h:15,
                 from /root/.cache/node-gyp/20.6.1/include/node/v8.h:33,
                 from /root/.cache/node-gyp/20.6.1/include/node/node.h:73,
                 from ../../nan/nan.h:56,
                 from ../src/node-boilerplate/scrypt_common.cc:28:
/root/.cache/node-gyp/20.6.1/include/node/v8-template.h:809:8: note: candidate: ‘void v8::ObjectTemplate::SetAccessor(v8::Local<v8::String>, v8::AccessorGetterCallback, v8::AccessorSetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, v8::SideEffectType, v8::SideEffectType)’
  809 |   void SetAccessor(
      |        ^~~~~~~~~~~
/root/.cache/node-gyp/20.6.1/include/node/v8-template.h:814:22: note:   no known conversion for argument 7 from ‘Nan::imp::Sig’ {aka ‘int’} to ‘v8::SideEffectType’
  814 |       SideEffectType getter_side_effect_type = SideEffectType::kHasSideEffect,
      |       ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/.cache/node-gyp/20.6.1/include/node/v8-template.h:816:8: note: candidate: ‘void v8::ObjectTemplate::SetAccessor(v8::Local<v8::Name>, v8::AccessorNameGetterCallback, v8::AccessorNameSetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, v8::SideEffectType, v8::SideEffectType)’
  816 |   void SetAccessor(
      |        ^~~~~~~~~~~
/root/.cache/node-gyp/20.6.1/include/node/v8-template.h:821:22: note:   no known conversion for argument 7 from ‘Nan::imp::Sig’ {aka ‘int’} to ‘v8::SideEffectType’
  821 |       SideEffectType getter_side_effect_type = SideEffectType::kHasSideEffect,
      |       ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make: *** [scrypt.target.mk:131: Release/obj.target/scrypt/src/node-boilerplate/scrypt_common.o] Error 1
make: Leaving directory '/eip-x/lodestar/node_modules/scrypt/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/eip-x/lodestar/node_modules/node-gyp/lib/build.js:203:23)
gyp ERR! stack     at ChildProcess.emit (node:events:514:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Linux 5.15.49-linuxkit-pr
gyp ERR! command \"/usr/bin/node\" \"/eip-x/lodestar/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /eip-x/lodestar/node_modules/scrypt
gyp ERR! node -v v20.6.1
gyp ERR! node-gyp -v v9.4.0
gyp ERR! not ok"
info This module is OPTIONAL, you can safely ignore this error

@ltfschoen
Copy link

ltfschoen commented Sep 13, 2023

so in this subsequent commit ltfschoen/lodestar@26b5f51, i tried to get it to work using Yarn 3.6.3, Node.js v20.6.1, NPM 10.1.0, Ubuntu 22.04, but i got the same error as with Ubuntu 23.04

...
➤ YN0013: │ typescript@npm:3.7.5 can't be found in the cache and will be fetched from the remote registry
➤ YN0001: │ Error: Couldn't allocate enough memory
    at ZipFS.allocateBuffer ([worker eval]:1:40610)
    at ZipFS.allocateSource ([worker eval]:1:41045)
    at ZipFS.setFileSource ([worker eval]:1:41318)
    at ZipFS.writeFileSync ([worker eval]:1:46409)
    at extractArchiveTo ([worker eval]:1:466938)
    at async MessagePort.<anonymous> ([worker eval]:1:468587)
➤ YN0000: └ Completed in 2m 20s

i also tried changing the major node version from 20 to 18, and to 16 here but got same error "Couldn't allocate enough memory"

@serybva
Copy link

serybva commented Jan 31, 2024

Had this issue as well on macos with a package installed using the file: protocol, resolved by using the portal: protocol instead

@gndelia
Copy link

gndelia commented Mar 20, 2024

Had this issue as well on macos with a package installed using the file: protocol, resolved by using the portal: protocol instead

I had the same issue with both file and portal, I ended up using regular yarn link commands and seems to have worked

@ergsolo
Copy link

ergsolo commented Jul 25, 2024

I have this problem with file protocol (on linux inside docker, yarn 4.3.1). When adding repo as file dep to some workspace.
With portal there is no error.
But portal produces different file structure, so peer deps is not resolved as they will with file protocol.

We need to fix this issue for file protocol.

danielpza added a commit to LeDDGroup/typescript-transform-paths that referenced this issue Aug 7, 2024
Updated yarn to the latest version 4.4.0 (yarn berry)
Had to resort to using the yarn pack workaround, there seems to be an
issue with yarn file: protocol when the dependency is pointing to the
top directory
yarnpkg/berry#3580 (comment)


There is a very noticeable performance improvement on CI when running
the tests, especially on windows

| master | yarn 4 |
|--------|--------|
|
![image](https://github.com/user-attachments/assets/65ac8d57-8f82-495a-8e4f-1ddec09ee53a)
|
![image](https://github.com/user-attachments/assets/db4b2170-5860-4a01-95a0-a3836bb7fc8c)
|
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working stale Issues that didn't get attention
Projects
None yet
Development

No branches or pull requests

7 participants