Skip to content

slow serialization for some cases (happens in Node.js 10) #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kaesonho opened this issue Aug 26, 2015 · 2 comments
Closed

slow serialization for some cases (happens in Node.js 10) #3

kaesonho opened this issue Aug 26, 2015 · 2 comments

Comments

@kaesonho
Copy link

Hi @ericf as we discussed, there might be some case to cause a GC pause,

I'm guessing it's some V8 de-opt we're hitting because the replacer forces V8 to go from native-land to user-land and back. Whereas with JSON.stringify().replace() it can stay in native-land the whole time.
So it's probably causing a serialization/de-serialization of the massive string causing a GC pause.

Now we have data ~500KB to be serialized, and found that some case(typically when the data is complicated and big enough) str.replace takes a very long time after JSON.stringify with replacer

To reproduce it, here's the simple script with the data we have,
We will get rid of it with a gc() or run without replacer, it acts like what you mentioned.
https://drive.google.com/file/d/0B0ZB5_LvayjrRjV0U2hrMGVjMUE/view?usp=sharing
please run node --trace-gc --replacer --issue ./slowserialization.js to reproduce it and see gc solve it with node --trace-gc --expose-gc --replacer --issue --gc ./slowserialization.js

will see when slow serialization happens, there's a lot of sweep and cause time spend.

[41363]     3133 ms: Mark-sweep 331.0 (368.4) -> 327.2 (366.4) MB, 123 ms [last resort gc].
[41363]     3249 ms: Mark-sweep 327.2 (366.4) -> 327.2 (364.4) MB, 116 ms [last resort gc].
[41363]     3361 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 112 ms [last resort gc].
[41363]     3475 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 113 ms [last resort gc].
[41363]     3578 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 103 ms [last resort gc].
[41363]     3676 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 97 ms [last resort gc].
[41363]     3776 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 100 ms [last resort gc].

The slow serialization only happens with Node.js 10, 12 won't have that issue.

I don't think that makes sense to fix this only for Node.js 10, but let's discuss here, like can we try something else for handling the function/regexp or make that optional (for our main customer's case, we don't have function and regexp)?

@redonkulus @lingyan @mridgway @Vijar

@kaesonho kaesonho changed the title slow serialization for some cases (only happens in Node.js 10) slow serialization for some cases (happens in Node.js 10) Aug 26, 2015
@caridy
Copy link
Contributor

caridy commented Aug 26, 2015

-1. Engine optimizations solved in node 0.12 should be good enough. I suspect that newer version of node+iojs will improve this even more. I think we should stick to what we have.

@ericf
Copy link
Collaborator

ericf commented Aug 26, 2015

With some testing I found that setting the space option to something other than a value that results in no space fixes the issue, e.g.:

JSON.stringify(obj, replacer, 2);

The above has the single-digit ms time in the script as expected/desired, instead of the 100x ms time.

ericf added a commit that referenced this issue Aug 26, 2015
@ericf ericf closed this as completed in #4 Aug 26, 2015
ericf added a commit that referenced this issue Aug 28, 2015
Using a token for the `space` value, the Node 0.10 CPU issue is avoid
while also keeping the file size closer to the no-space size.

Using the test script in #3, the results of this method add a bit of
CPU time, but its worth the reduction in size:

**1 space:**
```
[48885]      803 ms: Scavenge 109.4 (142.2) -> 106.3 (142.2) MB, 0 ms [Runtime::PerformGC].
memory usage: { rss: 140365824, heapTotal: 127084240, heapUsed: 117960976 }
memory usage: { rss: 140394496, heapTotal: 127084240, heapUsed: 118006704 }
str.replace takes 4 ms.
```

**Tokenized space:**
```
[48879]      880 ms: Mark-sweep 156.3 (190.6) -> 107.8 (148.2) MB, 41 ms [allocation failure] [GC in old space requested].
memory usage: { rss: 151355392, heapTotal: 160808928, heapUsed: 120760240 }
memory usage: { rss: 151359488, heapTotal: 160808928, heapUsed: 120818752 }
str.replace takes 2 ms.
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants