getOps does not include metadata, specifically timestamps.

[kbadk]

getOps does no longer includes metadata as ShareJS (and LiveDB Mongo does).

When looking at the current implementation of _getOps, we can see that the metadata (m field) is deliberately being removed:

ShareDbMongo.prototype._getOps = function(collectionName, id, from, callback) {
  this.getOpCollection(collectionName, function(err, opCollection) {
    if (err) return callback(err);
    var query = getOpsQuery(id, from);
    // Exclude the `d` field, which is only for use internal to livedb-mongo.
    // Also exclude the `m` field, which can be used to store metadata on ops
    // for tracking purposes
    var projection = {d: 0, m: 0};
    var sort = {v: 1};
    opCollection.find(query).project(projection).sort(sort).toArray(callback);
  });
};

This did not happen in LiveDB Mongo:

LiveDbMongo.prototype.getOps = function(cName, docName, start, end, callback) {
  var err; if (err = this._check(cName)) return callback(err);

  var query = end == null ? {$gte:start} : {$gte:start, $lt:end};
  this._opCollection(cName).find({name:docName, v:query}, {sort:{v:1}}).toArray(function(err, data) {
    if (err) return callback(err);

    for (var i = 0; i < data.length; i++) {
      // Strip out _id in the results
      delete data[i]._id;
      delete data[i].name;
    }
    callback(null, data);
  });
};

Could this be reverted, or could there possibly be an option to allow users to fetch the operations along with metadata?

[nateps]

Yes, agree it is useful to have an option to return the metadata along with the ops. Not returning it by default reduces the size of data over the network between mongo and the server and reduces CPU processing incoming data on the server, since this is normally not needed.

Adding an options argument to all snapshot and ops related methods for this and further extensions later.

[kbadk]

Thanks, Nate!

However, this seems to break almost everything for me. The options parameter you've added to the methods of sharedb-mongo are not methods that we call manually; they're called through sharedb's backend.js. When you add an options parameter, the interface between sharedb and sharedb-mongo seems to break.

For instance, when calling fetch on sharedb:

share.fetch(agent, collectionName, documentId, callback);

This internally calls (sharedb/lib/backend.js L270):

backend.db.getSnapshot(collection, id, fields, null, callback);

With this commit, the getSnapshot method signature now looks like:

ShareDbMongo.prototype.getSnapshot = function(collectionName, id, fields, options, callback)

The options are therefore not passed along, and I even get a "callback is not a function" error somewhere down the line.

We can't modify the parameters given to sharedb-mongo directly, as all communication with sharedb-mongo is encapsulated by sharedb itself.

Or am I missing something? What version of sharedb should I be running this with? I tried with sharedb@0.11.36 and sharedb-mongo@1.0.0-beta, but that gives the above error. When trying with sharedb@1.0.0-beta.3, I get

Agent closed due to error 13c9844f370c672bd0160b471785c1a6 { code: 4000, message: 'Received non-object message' }

Whenever I try to fetch a document. I previously used sharedb@0.11.36 and sharedb-mongo@0.8.7.

[nateps]

Yes, this API change was intentional, and you need to use sharedb@1.0.0-beta
versions with sharedb-mongo@1.0.0-beta.

Are you using ShareDB directly or are you using another library as well?
You might see this error if the connection library you are using has not
been updated.

On Sun, Jul 24, 2016 at 5:14 AM, Kristian Antonsen <notifications@github.com

wrote:

Thanks, Nate!

However, this seems to break almost everything. The options parameter
you've added to the methods of sharedb-mongo are not methods that we call
manually; they're called through sharedb's backend.js. When you add an
options parameter, the interface between sharedb and sharedb-mongo breaks.

For instance, when calling fetch on sharedb:

share.fetch(agent, collectionName, documentId, callback);

This internally calls (sharedb/lib/backend.js L270
https://github.com/share/sharedb/blob/master/lib/backend.js#L270-L291):

backend.db.getSnapshot(collection, id, fields, null, callback);

With this commit, the getSnapshot method signature now looks like:

ShareDbMongo.prototype.getSnapshot = function(collectionName, id, fields, options, callback)

Which therefore throws "callback is not a function".

We can't modify the parameters given to sharedb-mongo directly, as all
communication with sharedb-mongo is encapsulated by sharedb itself.

Or am I missing something? What version of sharedb should I be running
this with? I tried with sharedb@0.11.36 and sharedb-mongo@1.0.0-beta, but
that gives the above error. When trying with sharedb@1.0.0-beta.3, I get

Agent closed due to error 13c9844f370c672bd0160b471785c1a6 { code: 4000, message: 'Received non-object message' }

Whenever I try to fetch a document.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#42 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOLNPg9ms_hWtzMCM75ADEBkS3_FSIZks5qY1ctgaJpZM4JFEsj
.

[kbadk]

Hi Nate,

When trying to install sharedb@1.0.0-beta, I'm being told it's not a valid install target.

npm ERR! No compatible version found: sharedb@1.0.0-beta
npm ERR! Valid install targets:
npm ERR! 1.0.0-beta.3, 1.0.0-beta.2, 1.0.0-beta.1, 0.11.37, 0.11.36, 0.11.35, 0.11.34, 0.11.33, 0.11.32, 0.11.31, 0.11.30, 0.11.29, 0.11.28, 0.11.27, 0.11.26, 0.11.25, 0.11.24, 0.11.23, 0.11.22, 0.11.21, 0.11.20, 0.11.19, 0.11.18, 0.11.17, 0.11.16, 0.11.15, 0.11.14, 0.11.13, 0.11.12, 0.11.11, 0.11.10, 0.11.9, 0.11.8, 0.11.7, 0.11.6, 0.11.5, 0.11.4, 0.11.3, 0.11.2, 0.11.1, 0.11.0, 0.10.2, 0.10.1, 0.10.0, 0.9.4, 0.9.3, 0.9.2, 0.9.1, 0.9.0, 0.8.5, 0.8.4, 0.8.3, 0.8.2, 0.8.1, 0.8.0

I've tried with sharedb@1.0.0-beta{1,2,3} together with sharedb-mongo@1.0.0-beta, but they all give me the `Agent closed due to error ...' posted before.

I'm not using any libraries on top of ShareDB.

[nateps]

Yes, I meant the semver 1.0.0-beta, so .3 would be the latest release.

This error is because the connection stream consumer is written to only
accept objects. It is probably getting a string. This was an intentional
API change between share 0.11 and 1.0. Previously, the stream required JSON
string messages.

How did you connect the client connection to the server backend? Did you
implement your own stream interface? If so, you just have to JSON.parse the
messages before pushing to the stream. See the examples in the sharedb repo.

On Jul 28, 2016 5:08 AM, "Kristian Antonsen" notifications@github.com
wrote:

sharedb@1.0.0-beta is not a valid install target.

npm ERR! No compatible version found: sharedb@1.0.0-beta
npm ERR! Valid install targets:
npm ERR! 1.0.0-beta.3, 1.0.0-beta.2, 1.0.0-beta.1, 0.11.37, 0.11.36, 0.11.35, 0.11.34, 0.11.33, 0.11.32, 0.11.31, 0.11.30, 0.11.29, 0.11.28, 0.11.27, 0.11.26, 0.11.25, 0.11.24, 0.11.23, 0.11.22, 0.11.21, 0.11.20, 0.11.19, 0.11.18, 0.11.17, 0.11.16, 0.11.15, 0.11.14, 0.11.13, 0.11.12, 0.11.11, 0.11.10, 0.11.9, 0.11.8, 0.11.7, 0.11.6, 0.11.5, 0.11.4, 0.11.3, 0.11.2, 0.11.1, 0.11.0, 0.10.2, 0.10.1, 0.10.0, 0.9.4, 0.9.3, 0.9.2, 0.9.1, 0.9.0, 0.8.5, 0.8.4, 0.8.3, 0.8.2, 0.8.1, 0.8.0

I've tried with sharedb@1.0.0-beta{1,2,3} together with
sharedb-mongo@1.0.0-beta, but they all give me the `Agent closed due to
error ...' posted before.

I'm not using any libraries on top of ShareDB, no.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#42 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOLNKYDpXbvuKVy_Bgqp5-NgEfgKtJ0ks5qaJvYgaJpZM4JFEsj
.

[nateps]

This feature request has now been implemented and published. The discussion on usage is a question on use of sharedb itself. If this continues to be an issue for you, please open a new issue on the sharedb repo.