-
Notifications
You must be signed in to change notification settings - Fork 186
slow serialization for some cases (happens in Node.js 10) #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
-1. Engine optimizations solved in node 0.12 should be good enough. I suspect that newer version of node+iojs will improve this even more. I think we should stick to what we have. |
With some testing I found that setting the JSON.stringify(obj, replacer, 2); The above has the single-digit ms time in the script as expected/desired, instead of the 100x ms time. |
Using a token for the `space` value, the Node 0.10 CPU issue is avoid while also keeping the file size closer to the no-space size. Using the test script in #3, the results of this method add a bit of CPU time, but its worth the reduction in size: **1 space:** ``` [48885] 803 ms: Scavenge 109.4 (142.2) -> 106.3 (142.2) MB, 0 ms [Runtime::PerformGC]. memory usage: { rss: 140365824, heapTotal: 127084240, heapUsed: 117960976 } memory usage: { rss: 140394496, heapTotal: 127084240, heapUsed: 118006704 } str.replace takes 4 ms. ``` **Tokenized space:** ``` [48879] 880 ms: Mark-sweep 156.3 (190.6) -> 107.8 (148.2) MB, 41 ms [allocation failure] [GC in old space requested]. memory usage: { rss: 151355392, heapTotal: 160808928, heapUsed: 120760240 } memory usage: { rss: 151359488, heapTotal: 160808928, heapUsed: 120818752 } str.replace takes 2 ms. ```
Hi @ericf as we discussed, there might be some case to cause a
GC pause
,Now we have data
~500KB
to be serialized, and found that some case(typically when the data is complicated and big enough) str.replace takes a very long time after JSON.stringify with replacerTo reproduce it, here's the simple script with the data we have,
We will get rid of it with a
gc()
or run withoutreplacer
, it acts like what you mentioned.https://drive.google.com/file/d/0B0ZB5_LvayjrRjV0U2hrMGVjMUE/view?usp=sharing
please run
node --trace-gc --replacer --issue ./slowserialization.js
to reproduce it and see gc solve it withnode --trace-gc --expose-gc --replacer --issue --gc ./slowserialization.js
will see when slow serialization happens, there's a lot of sweep and cause time spend.
The slow serialization only happens with Node.js
10
,12
won't have that issue.I don't think that makes sense to fix this only for Node.js 10, but let's discuss here, like can we try something else for handling the
function/regexp
or make that optional (for our main customer's case, we don't have function and regexp)?@redonkulus @lingyan @mridgway @Vijar
The text was updated successfully, but these errors were encountered: