How to find only newest document in collection with set of IDs

Question

I have a collection Feeds and a collection FeedElements. FeedElements documents have a reference to the Feeds collection via the feedId field. Furthermore, all FeedElements documents have the date field submitted.

I want to publish only the newest (determined by the field submitted) FeedElements document which corresponds to exactly one Feeds document.

Here is what I have tried:

Meteor.publish('recentFeedElements', function (userId) {
    var feedIds = Feeds.find({'userId': userId}).map(function(feed) {
        return feed._id;
    });
    if (feedsIds.length > 0) return FeedElements.find({feedId: {$in: feedIds}}, {sort: {submitted: -1});
    else this.ready();
});

The problem is, if I use limit in combination with sort inside the FeedElements.find() query, I only get the newest documents of all Feed documents. However, I want to have a strict 1-1 relation. So, one Feed document -> the newest FeedElements document with the appropriate reference.

If I understand your intent, then what you want is the "latest" document for each supplied "feedId" value, if of course it exists, correct? The clear case you should have guessed here is that it is better to submit separate queries for each "feedId", and then combine the results into a single response. Related: [limit and sort each group by in mongodb aggregation](http://stackoverflow.com/questions/33458107/limit-and-sort-each-group-by-in-mongodb-using-aggregation/33458267#33458267) — Blakes Seven, Nov 08 '15 at 10:18
Better example of the "parallel" processing of each query and combination here: [Mongodb aggregate sort and limit within group](http://stackoverflow.com/questions/32885047/mongodb-aggregate-sort-and-limit-within-group/32886815#32886815). Both questions are very similar, and similar to your own request in the "top n" sense. Both say that running seperate queries is the best thing to do, and the latter has a coded example to follow. Build this up in your server side method, and optionally publish. — Blakes Seven, Nov 08 '15 at 10:25
@BlakesSeven Thank you for your help. I read both posts, but I have difficulties with grouping the elements via `$in`. Could you please give me a hint? — user3475602, Nov 08 '15 at 11:40
The "hint" was "don't group". Run separate queries and combine the results. Aggregation is the "wrong" way to do this. Running separate queries is the "right way". — Blakes Seven, Nov 08 '15 at 11:42
How do you want to access recently submitted feed elements on the client? `FeedElements.find().fetch()` ? — imkost, Nov 13 '15 at 17:38
I want to display all Feeds in a list, however I also want to display a timestamp next to it. This timestamp should display the activity of the feed. Consequently, I need the latest FeedElement which corresponds to the Feed. — user3475602, Nov 13 '15 at 20:45

imkost · Answer 1 · 2015-11-16T12:54:57.987

If I understand you right, you want every feed to have lastActivity field which contains timestamp of last submitted feed element of this feed. You want this field to be reactive and you don't want to publish all feed elements.

In this case, aggregation is not a solution, because Meteor does not allow reactive aggregations.

You need to use low-level publish API: http://docs.meteor.com/#/full/meteor_publish (see example section).

Low-level publish API (this.added/this.removed/this.changed/others) gives you full control of what data you send to client via Meteor.publish method.

Here is how you can solve your problem (I use ES2015 syntax):

// client/client.js
Meteor.subscribe('userFeeds', { userId: 1 });

// lib/lib.js
Feeds = new Mongo.Collection('feeds');
FeedElements = new Mongo.Collection('FeedElements');

// server/server.js

// When setting `submitted` field, automatically set `submittedAt` field
// (I use matb33:collection-hooks here)
FeedElements.before.update((userId, elem, fieldNames, modifier) => {
  if (modifier.$set.submitted) {
    modifier.$set.submittedAt = new Date();
  }
});

function getLastActivity(feedId) {
  const lastSubmittedElem = FeedElements.findOne(
    {
      feedId,
      submitted: true,
    },
    {
      sort: { submittedAt: -1 }
    }
  );

  return lastSubmittedElem ? lastSubmittedElem.submittedAt : null;
}

Meteor.publish('userFeeds', function({ userId }) {
  const elemsObservers = {};

  // Observe all user feeds
  var feedObserver = Feeds.find({ userId: userId }).observeChanges({
    added: (feedId, fields) => {
      // Observe feed elements of the feed
      elemsObservers[feedId] = FeedElements.find({ feedId }).observeChanges({
        changed: (elemId, fields) => {
          // Update `lastActivity` field when new element is submitted
          if (fields.submitted) {
            this.changed('feeds', feedId, { lastActivity: fields.submittedAt });
          }
        },
      });

      fields.lastActivity = getLastActivity(feedId);

      this.added('feeds', feedId, fields);
    },

    changed: (feedId, fields) => {
      this.changed('feeds', feedId, fields);
    },

    removed: (feedId) => {
      elemsObservers[feedId].stop();
      delete elemsObservers[feedId];

      this.removed('feeds', feedId);
    },
  });

  this.ready();

  this.onStop(function() {
    feedObserver.stop();

    for (const feedId in elemsObservers) {
      elemsObservers[feedId].stop();
    }
  });
});

Also I prepared github repo https://github.com/imkost/feeds. Just git clone and meteor run.

Thank you for your suggestion! However, I don't want to set another field. I think aggregation is the way to go. I tried it with @Somnath Muluk suggestion, but unfortunately he gave me a wrong pipeline. Reavtive aggregations in Meteor do work, when using the package [meteor-aggregate](https://github.com/meteorhacks/meteor-aggregate) and [ndevr's workaround](https://github.com/meteorhacks/meteor-aggregate/issues/8). Nevertheless, thank you for your effort! — user3475602, Nov 14 '15 at 07:13

score 0 · Answer 2 · answered Nov 11 '15 at 13:00

0

You can get this by aggregation framework by $sort, $group & $first.

db.FeedElements.aggregate(
[
  { $match: { feedId: {$in:[1,2,3]}} },
  { $sort: { submitted: -1} },
  {
    $group:
    {
      _id: "$feedId",
      firstFeedElement: {$first : "$$ROOT"} // Get first whole document or you can take fields you want
    }
     }
   ]
)

answered Nov 11 '15 at 13:00

Somnath Muluk

55,015
38
216
226

Thank you for your help! Unortunately, this does not satisfy my requirement. With your aggregation, I get the following result: `[{feedId: "123", firstFeedElement: Object}]`. However, I want to have this result: `[Object, Object, Object, ...]`. The objects in the array represent the latest document for each supplied `feedId` value. For instance, if I have 10 `feedId`s, I want to have an array with 10 `FeedElements`, which are the latest document corresponding to one `feedId` (if the document exists). – user3475602 Nov 12 '15 at 12:13
@user3475602: Do you want to have 10 feedElementObjects for each of 10 feedIds? (Means 100 feedElement objects, 10 for each) – Somnath Muluk Nov 13 '15 at 03:02
I have two collections: `Feeds` and `FeedElements`. Every `FeedElements` document has its reference via `feedId` to one `Feeds` document. Now, I want to have only one (the newest) `FeedElements` document per `feedId` (corresponds to exactly one `Feeds` document). So, all objects in the result array are `FeedElements` documents, each element references to exactly one `Feeds` document and is the latest one, because there may be multiple `FeedElements` documents which reference via `feedId` to the same `Feeds` document. So, 10 different `feedId`s, 10 different `FeedElements` documents (latest). – user3475602 Nov 13 '15 at 07:47
@user3475602 : I believe you must be getting result like this: [{feedId: "1", firstFeedElement: Object},{feedId: "2", firstFeedElement: Object},{feedId: "3", firstFeedElement: Object}...]. You will get document row per feedId. You need to pass feedId of 10 feeds in $match section. – Somnath Muluk Nov 13 '15 at 17:14

How to find only newest document in collection with set of IDs

2 Answers2