Invalid characters in SPIFFS file

[kakopappa]

Describe the bug

1. serialize the following JSON and save it to SPIFF

{"success":true,"message":"Ok","credentials":{"appkey":"6f7cdcc8-c4b2-49b2-819f-d5e44efd808c","appsecret":"e0daaf3d-59ca-457a-804b-3ee8e241acdf-670a79b8-e323-4335-8375-9a833153ac7b"},"devices":[{"id":"61c9fd21c03c1316ae7dba52"},{"id":"61c9fd21c03c1316ae7dba64"},{"id":"61c9fd21c03c1316ae7dba76"},{"id":"61c9fd21c03c1316ae7dba88"},{"id":"61c9fd21c03c1316ae7dba9a"},{"id":"61c9fd21c03c1316ae7dbaac"},{"id":"61c9fd21c03c1316ae7dbabe"},{"id":"61c9fd21c03c1316ae7dbad0"},{"id":"61c9fd21c03c1316ae7dbae2"},{"id":"61c9fd21c03c1316ae7dbaf4"}]}

2. deserializeJson() failed with InvalidInput

3. If you change "success":false, it works fine. I cannot figure out why..

Troubleshooter report

ArduinoJson Troubleshooter's report

1. The issue happens at run time
2. The issue concerns deserialization
3. deserializeJson() returns InvalidInput
4. Input comes from HTTP
5. Program uses HTTPClient
6. Response comes from HTTPClient::getString()
7. jsonlint says the document is valid
8. Input’s first byte doesn’t suggest a BOM

Environment
Here is the environment that I used:
ESP8266

• Core/runtime: [e.g. ESP8266 core for Arduino v3.0.2]
  3.0.2

Reproduction
Here is a small snippet that reproduces the issue.

#include <ArduinoJson.h>
#include "FS.h"

void setup() {
  Serial.begin(115200); Serial.println();
  delay(1000);

  SPIFFS.begin();

  SPIFFS.remove("/j.json");
 
  DynamicJsonDocument doc(2048);
  String input = "{\"success\":true,\"message\":\"Ok\",\"credentials\":{\"appkey\":\"6f7cdcc8-c4b2-49b2-819f-d5e44efd808c\",\"appsecret\":\"e0daaf3d-59ca-457a-804b-3ee8e241acdf-670a79b8-e323-4335-8375-9a833153ac7b\"},\"devices\":[{\"id\":\"61c9fd21c03c1316ae7dba52\"},{\"id\":\"61c9fd21c03c1316ae7dba64\"},{\"id\":\"61c9fd21c03c1316ae7dba76\"},{\"id\":\"61c9fd21c03c1316ae7dba88\"},{\"id\":\"61c9fd21c03c1316ae7dba9a\"},{\"id\":\"61c9fd21c03c1316ae7dbaac\"},{\"id\":\"61c9fd21c03c1316ae7dbabe\"},{\"id\":\"61c9fd21c03c1316ae7dbad0\"},{\"id\":\"61c9fd21c03c1316ae7dbae2\"},{\"id\":\"61c9fd21c03c1316ae7dbaf4\"}]}";
  deserializeJson(doc, input);

  Serial.println(input[0]); // prints {
 
  // Write to SPIFF
  File configFile = SPIFFS.open("/j.json", "w");
  serializeJsonPretty(doc, configFile);
  configFile.close();


  // Read from SPIFF
  File file2 = SPIFFS.open("/j.json", "r");
  DynamicJsonDocument doc2(2048);
  DeserializationError err = deserializeJson(doc2, file2);

  if (err) {
    Serial.printf("deserializeJson() failed: %s\r\n", err.c_str());
    file2.close();
  }

  File file3 = SPIFFS.open("/j.json", "r");
  while (file3.available()) {  
      Serial.write((char)file3.read());
  }  
}

void loop() {
  // put your main code here, to run repeatedly:

}

Program output

[bblanchon]

Hi @kakopappa,

This looks like a SPIFFS problem.
Unfortunately, I don't have the required hardware with me to run your program, so it's up to you to figure out if the problem comes from either:

1. the ArduinoJson library (hint: it doesn't)
2. the SPIFFS implementation
3. the hardware or its configuration

If the problem is related to the library, you should be able to reproduce it without SPIFFS.
For example, you could print to a String or Serial.

If the problem is related to the SPIFFS implementation, you should reproduce it without ArduinoJson.
For example, you could hard-code the JSON document.

If the problem is related to your hardware, your program should work fine on another board.

Good luck!

Best regards,
Benoit

[kakopappa]

I tested the same code on
ESP32 core : 2.0.1 works fine.

ESP8266 Core: 3.0.2 bug persists
ESP8266 Core: 2.7.4 works fine.

Solution seems to be to downgrade the ESP Core to 2.7.4 on ESP8266

[bblanchon]

@kakopappa, did you open an issue on ESP8266 Core's repository?

[bblanchon]

Probably caused by esp8266/Arduino#8372