Mongodb full text search with sort by timestamp? - node.js

Question

I've literally researched the entire web and couldn't find a satisfactory answer for this so thought I would ask here.
Basically what I'm trying to do is build a full text search query with pagination, which returns results sorted by time.
The problem is, a naive sort like the following doesn't perform at all:
db.collection
.find({ $text: { $search: "hello" } })
.sort({ created_at: -1 })
.limit(100)
.toArray(function(....
And yes, I've of course indexed it with created_at. And as you can see it's limited to 100 items.
So far what I gather is that the full text index in MongoDB doesn't let you sort by any arbitrary attribute in the collection AT ALL, and the only way to sort it is by adding some $meta attribute to sort it based on some internal scoring mechanism.
But that doesn't work for me, and i really want to sort this by created_at.
Maybe I'm misunderstanding the whole thing, but I refuse to believe that no one has come up with a solution for this very obvious use case. Am I missing something? Does anyone know how to sort a large text search result by a collection attribute? At this point I would appreciate ANY shine of light, even if it's a hack.
[EDIT] For example without the limit and sort, the response would look something like this:
[{
"msg": "hello world",
"created_at": 1000
}, {
"msg": "hello",
"created_at": 899
}, {
"msg": "hello hello",
"created_at": 1003
}, {
...
}]
But I want to limit it to only 100, sorted by created_at, AFTER having searched the database for the occurrrence of "hello". I don't care about relevance and I only want to sort so that it's ordered by time.
[{
"msg": "hello hello",
"created_at": 1003
}, {
"msg": "hello world",
"created_at": 1000
}, {
"msg": "hello",
"created_at": 899
}, {
...
}]
Just to be clear, the query DOES work, but it takes very long time even though I have indexed it with created_at. I don't have this issue when I do a similar find-sort-limit pattern with other queries (not full text search), and I think this is specific to full text search.
I am looking for a way to somehow make this query faster.