direct websocket support between web platform and wireless firmata

[d-a-v]

Hello,

Tthis is not a feature-request, I am trying to implement it myself.

I know it is possible to use the web platform with some sort of bridge hardware to reach a network-enabled arduino device.

I would like to make things more simple, by using a websocket-enabled firmata arduino device, and reach it directly from the web browser (any browser would then work, not only chrom*).

This will allow to make things work out of the box with only

• an arduino device flashed with standard firmata on websockets
• a web browser

I have the websocket-enabled firmata arduino part which is working (and a proof-of-concept-javascript webpage to switch on/off the built-in-led).

I am now looking at snap4arduino. I have so far willingly been using firefox.
Build for chromium starts, Snap is working, and I can see the arduino menu.
I modified the arduino menu to be able to select and start a websocket link which connects to the network device.

However the firmata interpreter is enclosed in crx/bundle.js that is accessed through an intermediary and internal API probably through chromium plugin mechanism.

There are also 6 firmata.js files in the repository, namely

./src/platforms/web/chromeos/root/firmata.js
./src/platforms/mobile/android/root/mobile/firmata.js
./environments/web/chromeos/node_modules/firmata/lib/firmata.js
./environments/desktop/node_modules/firmata/lib/firmata.js
./environments/embedded/cli/node_modules/firmata/lib/firmata.js
./environments/embedded/cli/node_modules/firmata/examples/ssfirmata.js

Before I dive into them, is there any recommendation on which one to use, or if using crx/bundle.js with its pure javascript API is possible or wise.

[jguille2]

Hi David,

I try to share some comments (maybe some of this are known for you). Maybe Bernat wants to add more details or correct me...

• Snap4Arduino can play with firmata devices via network, using a bridge. I use ESP8266 with a "Wifi to serial" sketch.
• When starting Snap4Arduino, an http-server solution was considered, but it was too slow for our purposes. Also, a websocket solution seemed not fast enough. But I know Bernat and John Maloney have done other implementations (for Microblocks project) and maybe Bernat has now a different opinion.
• Anyway I don't see (maybe I'm missing something) a websockets implementation for Snap4Arduino. If you build a web-server on your device, you can implement the client as a Snap! library (you don't need nodejs) and if you build this server outside, as a extra piece of soft, then your solution won't be a only-browser solution.
• We want to track the evolution of Web Serial API. Yes, at the moment only experimental and only for Chromium, but we'll see... But it's again a serial solution.
• To connect to network devices (as web servers), we are using pure Snap! connecting to them. And we like Mozilla WebThings protocol
• And one last commnet. If you are playing with ESP8266 or other 32bits boards to build these connections, I suggest you to look at Microblocks project. Then, no firmata anywhere... just programming your device with a visual and dynamic language, and you can build (with this programming language) your own web server and connect to browsers, mobile devices, Snap!.... with your own "protocol", or using Mozilla WoT or others...

Joan

[jguille2]

Hi David,
I see now your PR to firmata/arduino... and I see you want to build your websocket server in your device (ESP board in that example).
Then, as I said, you can play directly with Snap! See this example of a mqtt project. You will see a initial block loads a js library and then, you "connection block" uses this library to do a webSocket connection. But you can do it from scratch, as your html example.

Ah! And I forgot a comment about our firmata.js files.
See our bundle.js is just a "browserify" version of the node modules required. The, firmata.js code is the same of the desktop version.
This idea is to use the same code for the desktop and web version, and "browserify" convert your "require" calls to "normal browser" calls.

Joan

[d-a-v]

Hello,

Thanks a lot for your prompt answers! There are many exploration paths.

Let me first give my point of view, that initiated my work on Firmata.
There are only two solutions for a standard (out-of-the-box, no extension) web browser to talk with outside, and it is the standard HTTP protocol requests, and Websockets (I still not have the explanation for why is so much complexity needed in the handshake process). As you pointed it out, HTTP protocol can be slow to manage. Websockets are faster.

Snap, microblocks, webthings, also blockly that I just discovered with the otto robot, are meant for kids, teachers, parents. Not technical people, that would be glad to just plug'n'play their techy new things, without cables, adapters, configuration, dependencies.
On our side, we have protocols. Websockets, but also mDNS. We have quite smart wireless devices. Unfortunately our web browsers cannot - without extensions - talk with bluetooth/ble devices out of the box, even with our go'old serial cable !

If I want the teachers of my kids to discover this world, I feel the need to be able to provide simple things, and we can do that, protocols are already there. We have js-compatible Web browsers (including in phones), WiFi, Websockets, mDNS, and the excellent javascript tools you guys are writing or adapting. Being one of the esp8266 arduino maintainer, especially the networking part of it, I try to smooth out every gap on my reach around this ecosystem.

• About your first point, with a wifiSerial sketch, well, yes !
  But I chose the hard way and to not build - not even try - such a setup.

• I agree with part of your second point, HTTP is too slow. But I disagree about websocket. They are fast (meaning: lower latency) once the initial useless handshaking is done.

• About your third and fifth point (you've seen the PR), I need detail about HTTP protocol: Even if it may be too slow, it is a web-browser-only solution. You say that snap4arduino / pure Snap can connect to an http server ? I'll try to get more insights about this. Does it work with Firmata commands, or everything to pilot an arduino board is yet to be done ?

• I've been monitoring the web serial api standardization at w3c for too many years to hope anything from it. And there's a cable 😀 (my avatar is a serial socket btw)

• Microblock seems nice but it's a linux only desktop application. Unfortunately not compatible with the average user.

• Mqtt is also an interesting solution. Thanks, I was not aware mosquitto spoke websockets !
  A mosquitto server (or mqtt integrating webockets) would be needed then.

On the other hand, I am near to the goal. esp8266 arduino (and I think other arduino devices) can speak websockets, Firmata on websockets, can announce itself on the network thanks to mDNS, so once the device is connected to the network (thanks to a wifi manager and a mobile phone to configure it), it is reachable with a friendly name like snap4arduino.local or firmata.local - which could be a default network name (in browser and on device).

I chose Firmata and Snap because you guys already did all the work. Only the websocket link was missing. I have yet to come back to the work I initiated in the OP. I'm waiting for a move from the Firmata repository whose maintainer team seems to be evolving.

Ah! And I forgot a comment about our firmata.js files.
See our bundle.js is just a "browserify" version of the node modules required. The, firmata.js code is the same of the desktop version.
This idea is to use the same code for the desktop and web version, and "browserify" convert your "require" calls to "normal browser" calls.

That will be the direction I'll try to take once I come back to it. Thanks for these indications !