json gem v2.9 incompatibility with ruby 3.3 runtime

[pjg]

While executing test event in AWS Lambda console for a Ruby 3.3 lambda function, which bundles the json gem version 2.9 (bundled via MailchimpTransactional) I get the following error:

{
    "errorMessage": "undefined method `parse' for class JSON::Ext::Parser",
    "errorType": "Init<NoMethodError>",
    "stackTrace": [
        "/var/task/vendor/bundle/ruby/3.3.0/gems/json-2.9.0/lib/json/common.rb:221:in `parse'",
        "/var/runtime/gems/aws_lambda_ric-3.0.0/lib/aws_lambda_ric/aws_lambda_marshaller.rb:15:in `marshall_request'",
        "/var/runtime/gems/aws_lambda_ric-3.0.0/lib/aws_lambda_ric.rb:76:in `wait_for_invocation'",
        "/var/runtime/gems/aws_lambda_ric-3.0.0/lib/aws_lambda_ric.rb:65:in `start_runtime_loop'",
        "/var/runtime/gems/aws_lambda_ric-3.0.0/lib/aws_lambda_ric.rb:49:in `run'",
        "/var/runtime/gems/aws_lambda_ric-3.0.0/lib/aws_lambda_ric.rb:221:in `bootstrap_handler'",
        "/var/runtime/gems/aws_lambda_ric-3.0.0/lib/aws_lambda_ric.rb:203:in `start'",
        "/var/runtime/index.rb:4:in `<main>'"
    ]
}

Using json gem version 2.5.1 with 3.2 ruby runtime works fine.

[ravi-bebble]

Hey I am also getting the same error on my ruby 3.3 Lambda

{     "errorMessage": "undefined method `parse' for class JSON::Ext::Parser",     "errorType": "Init<NoMethodError>",     "stackTrace": [         "/var/task/vendor/bundle/ruby/3.3.0/gems/json-2.9.1/lib/json/common.rb:221:in `parse'",         "/var/task/vendor/bundle/ruby/3.3.0/gems/jmespath-1.6.2/lib/jmespath/lexer.rb:323:in `requires_wrapping?'",         "/var/task/vendor/bundle/ruby/3.3.0/gems/jmespath-1.6.2/lib/jmespath/lexer.rb:328:in `<class:Lexer>'",         "/var/task/vendor/bundle/ruby/3.3.0/gems/jmespath-1.6.2/lib/jmespath/lexer.rb:7:in `<module:JMESPath>'",         "/var/task/vendor/bundle/ruby/3.3.0/gems/jmespath-1.6.2/lib/jmespath/lexer.rb:5:in `<top (required)>'",         "<internal:/var/lang/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'",         "<internal:/var/lang/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'",         "/var/task/vendor/bundle/ruby/3.3.0/gems/jmespath-1.6.2/lib/jmespath.rb:9:in `<module:JMESPath>'",         "/var/task/vendor/bundle/ruby/3.3.0/gems/jmespath-1.6.2/lib/jmespath.rb:6:in `<top (required)>'",         "<internal:/var/lang/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'",         "<internal:/var/lang/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'",         "/var/task/vendor/bundle/ruby/3.3.0/gems/aws-sdk-core-3.214.1/lib/aws-sdk-core.rb:5:in `<top (required)>'",         "<internal:/var/lang/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'",         "<internal:/var/lang/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'",         "/var/task/vendor/bundle/ruby/3.3.0/gems/aws-sdk-ssm-1.186.0/lib/aws-sdk-ssm.rb:11:in `<top (required)>'",         "<internal:/var/lang/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:141:in `require'",         "<internal:/var/lang/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:141:in `rescue in require'",         "<internal:/var/lang/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:135:in `require'",

But upon checking more logs I am also getting an initialization error as

Ignoring json-2.9.1 because its extensions are not built. Try: gem pristine json --version 2.9.1
--
Init error when loading handler

Wanted to know if you were able to fix it

[coreystinson2]

I started seeing this issue yesterday on Lambda's we deployed readme updates to. e.g. no code changes.

Ruby 3.3.0 runtime with no reference to "json" in the Gemfile, explicitly adding version 2.5.0 resolved the issue.

gem "json", "~> 2.5.0"

Curious as to how this breaks all of a sudden, we have cached build assets so this is 100% a Lambda problem.

[mullermp]

The gem does not declare any dependency on the json gem - it is bundled with your installation in ruby. Both of the stack traces are different - one is actually for the AWS SDK and the other in Lambda. I'm inclined to believe this is related to Ruby 3.3 and JSON 2.9. Can you try using the latest json 2.10?

[halo]

For others who googled their way here because of a similar error:

{
  "errorMessage": "unexpected token at ''",
  "errorType": "Init<JSON::ParserError>",
  "stackTrace": [
    "/var/lang/lib/ruby/3.4.0/json/common.rb:221:in 'JSON::Ext::Parser.parse'",
    "/var/lang/lib/ruby/3.4.0/json/common.rb:221:in 'JSON.parse'",
    "/var/runtime/gems/aws_lambda_ric-3.0.0/lib/aws_lambda_ric/aws_lambda_marshaller.rb:13:in 'AwsLambda::Marshaller.marshall_request'",
  # ...
}

When you run your lambda app in a local docker container, make sure you call it using POST and not GET.

# GET is wrong
curl "http://localhost:9000/2015-03-31/functions/function/invocations"

# POST is correct
curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'

On AWS lambda in the cloud (running that very same container), however, this error does not occur, both GET and POST work fine for some reason.

Also, I tried to reproduce the error originally posted here (undefined method 'parse' for class JSON::Ext::Parser) with ruby 3.2, 3.3, 3.4 and various combinations of the json gem 2.5, 2.9, 2.12, but it always worked fine.