mongoDB query find is limited at 100? - node.js

my application is no longer functional !
I found this without success : mongodb db.collection.find({}) does nothing past .limit(101)
My application was stable during few months and I didn't update my scripts.
For few days, some queries don't work anymore !
Here your are : my origin script but it doesn't work anymore :
console.log("GO_1"); // display ok
models.myCollectionOne.find().sort({ created: 'desc' }).then(function (err,data) {
console.log("GO_2"); // here I've got nothing on console
console.log(err); // here I've got nothing on console
console.log(data); // here I've got nothing on console
}
// here nothing happens ?
I tested this :
models.myCollectionOne.find().limit(101).then(function (data) {
console.log("GO_2"); // OK
console.log(data); // ok
}
So, my questions are :
1) Why does my origin script doesn't work anymore ?
2) Do you know if there have been updates impacting my script?
3) Why limit(101) works and not limit(102 or more) ?
Thank for your help

I believe that the problem is on the sort. I think that 'desc' is not a valid value, you could try replacing it with -1: { created: -1 }
(I've just tested using 'desc' here and got "$err" : "Can't canonicalize query: BadValue bad sort specification")
Perhaps you Mongodb got updated and it stopped working?

So, I found my problem, I just need to update all my node module package, mainly mongoose,
So now, all works very well.
By the way, I have seen that I updated my main package.json but not the module npm itself ;-)
#Hampus #andrek thank for you help guys

Related

Cannot set property 'avatar' of undefined at Member.set [as avatar] - Discord Bot

I haven't used my Discord recently but the last time I ran it, it still worked perfectly. However, I keep getting this error Cannot set property 'avatar' of undefined at Member.set [as avatar] these days.
Can someone help me?
I was struggling with this issue and eventually figured it out. It's because the 'this' keyword is getting hijacked somehow in the code within the Object.defineProperty call inside the Member constructor(line 2606 in my version of index.js, but I've made a few other fixes already so yours is probably different). I was able to fix it by caching off the 'this' reference into a private member and referencing that instead. It feels hacky but it works. Like so:
function Member(client, server, data) {
copyKeys(data, this, ['user', 'joined_at',]);
this.id = data.user.id;
this.joined_at = Date.parse(data.joined_at);
this.color = colorFromRole(server, this);
var tempThis = this;
['username', 'discriminator', 'bot', 'avatar', 'game'].forEach(function(k) {
if (k in Member.prototype) return;
Object.defineProperty(Member.prototype, k, {
get: function() { return client.users[tempThis.id][k]; },
set: function(v) { client.users[tempThis.id][k] = v; },
enumerable: true,
});
});
}
I just experienced the same error with my own bot completely out of the blue. After some investigation, I checked the code at DiscordClient.handleWSMessage (on my error it was showing at index.js:1871:31 as opposed to index.js:1891:31, however I'm not sure if it's a matter of having different versions of discord.io installed) - in any case, the error seemed to be stemming from the Event switch statement responding to a GUILD_CREATE event - this may be different for you:
case "GUILD_CREATE":
/*The lib will attempt to create the server using the response from the
REST API, if the user using the lib creates the server. There are missing keys, however.
So we still need this GUILD_CREATE event to fill in the blanks.
If It's not our created server, then there will be no server with that ID in the cache,
So go ahead and create one.*/
client.servers[_data.id] = new Server(client, _data);
return emit(client, message, client.servers[_data.id]);
I don't understand why GUILD_CREATE events are being received, my bot has never been programmed to handle these, however commenting out the executable lines in the switch case above and replacing them with an empty return statement seemed to stop the error occurring, and my bot stayed connected (so far, I've only been testing for a few minutes).
Might not work for everyone, but on my computer this issue was resolved by saving and only running after I saved. Slightly annoying but it worked!

Basic GET Request using Express.js & Mongoose

I'm working on an assignment to list all of the data in a mongoDB database, and am having trouble finding where I'm going wrong. It seems like a fairly simple problem, but whenever I run the provided mocha test, it keeps throwing 404 errors. Here is the relevant portion of the test:
it('should it able to retrieve all listings', function(done) {
agent.get('/api/listings')
.expect(200)
.end(function(err, res) {
should.not.exist(err);
should.exist(res);
res.body.should.have.length(147);
done();
});
});
And here is my code for the GET request. I've tried a few different ways of coding it, but this is seems like the simplest/most direct way to return the desired data as JSON.
exports.list = function(req, res) {
Listing.find(function(err, listing){
if(err){
res.status(404).send(err);
} else {
res.json(listing);
}})
};
Is there anything else I should be doing? I've been looking at tutorials and basic examples of requests and it seems like it should work, but it doesn't. Any help would be greatly appreciated.
Make sure that the middleware function (the code for GET request) is mapped to /api/listings
I'm not sure about exports.list. It should probably be module.exports
I'm assuming, based on ur code, that ur using the mongoose ODM. In which case, I think you need to pass a query to the find method check this out
You might wanna make sure that you're connected to the database at the time of test initialization and that that completes before the test starts
It always helps to log errors
Checkout express-generator to scaffold a boilerplate express app. Might help to compare it with your app, to check if it's wired correctly
Seems like you are not passing the first parameter to the find method. Only the callback ... try this:
Listing.find({}, function(err, listing) {
if (err) {
res.status(404).send(err);
} else {
res.json(listing);
}
})
I am assuming you want all records which is why we pass an empty object {}.

Mongo / Mongoose query works in Mongo shell but not in Node (on EC2 Ubuntu image)

I'm having a hard time determining why this query works in the Mongo shell but not when I try to hit the server endpoint from the browser.
Shell command:
db.playlists.find({keywords: {"$in" : ["featured"]}}).limit(5)
Server code:
app.get('/getFeaturedPlaylists', (req, res) => {
let query = {
keywords: { "$in" : ["featured"]}
}
database.Playlist.find({query}).limit(5).exec(function(err, data) {
err ? console.log(err) : res.send({body: JSON.stringify(data)});
})
})
Expected result is an array of playlist objects:
{"body": "[{\"songList\":[{\"songId\":522,\"bpm\":45},{\"songId\":310,\"bpm\":45},{\"songId\":199,\"bpm\":90},{\"songId\":170,\"bpm\":150},{\"songId\":17,\"bpm\":150},{\"songId\":722,\"bpm\":45},{\"songId\":767,\"bpm\":90},{\"songId\":754,\"bpm\":120},{\"songId\":844,\"bpm\":150},{\"songId\":422,\"bpm\":120},{\"songId\":638,\"bpm\":90},{\"songId\":243,\"bpm\":150},{\"songId\":524,\"bpm\":90},{\"songId\":575,\"bpm\":120},{\"songId\":143,\"bpm\":120}],\"shuffledQueue\":[],\"keywords\":[\"bluegrass\",\"featured\"],\"_id\":\"5a76188fde76d97bda5e342e\",\"playlistId\":2,\"playlistName\":\"Playlist2\"},{\"songList\":[{\"songId\":401,\"bpm\":45},{\"songId\":536,\"bpm\":150},{\"songId\":982,\"bpm\":60},{\"songId\":812,\"bpm\":60},{\"songId\":466,\"bpm\":150}],\"shuffledQueue\":[],\"keywords\":[\"featured\",\"bluegrass\"],\"_id\":\"5a76188fde76d97bda5e3432\",\"playlistId\":6,\"playlistName\":\"Playlist6\"},{\"songList\":[{\"songId\":248,\"bpm\":120},{\"songId\":84,\"bpm\":45},{\"songId\":53,\"bpm\":60},{\"songId\":749,\"bpm\":45},{\"songId\":811,\"bpm\":120}],\"shuffledQueue\":[],\"keywords\":[\"featured\",\"hip-hop\"],\"_id\":\"5a76188fde76d97bda5e3433\",\"playlistId\":7,\"playlistName\":\"Playlist7\"},{\"songList\":[{\"songId\":894,\"bpm\":150},{\"songId\":190,\"bpm\":60},{\"songId\":235,\"bpm\":60},{\"songId\":632,\"bpm\":60},{\"songId\":970,\"bpm\":60},{\"songId\":475,\"bpm\":90},{\"songId\":304,\"bpm\":60},{\"songId\":816,\"bpm\":120},{\"songId\":613,\"bpm\":150},{\"songId\":310,\"bpm\":45},{\"songId\":599,\"bpm\":45},{\"songId\":91,\"bpm\":90},{\"songId\":650,\"bpm\":120},{\"songId\":219,\"bpm\":45},{\"songId\":290,\"bpm\":90}],\"shuffledQueue\":[],\"keywords\":[\"hip-hop\",\"featured\"],\"_id\":\"5a76188fde76d97bda5e3436\",\"playlistId\":10,\"playlistName\":\"Playlist10\"},{\"songList\":[{\"songId\":778,\"bpm\":60},{\"songId\":652,\"bpm\":60},{\"songId\":792,\"bpm\":90},{\"songId\":353,\"bpm\":120},{\"songId\":528,\"bpm\":60},{\"songId\":887,\"bpm\":120},{\"songId\":287,\"bpm\":90},{\"songId\":926,\"bpm\":120},{\"songId\":671,\"bpm\":90}],\"shuffledQueue\":[],\"keywords\":[\"featured\",\"reggae\"],\"_id\":\"5a76188fde76d97bda5e3437\",\"playlistId\":11,\"playlistName\":\"Playlist11\"}]"}
Actual result is an empty array:
{"body":"[]"}
Even more peculiar, this issue only arose when I moved the server and database to EC2 instances. Using the exact same code and data set on my local machine, it responds as I expect it to (via Postman or Chrome -- this is how I got the "expected response" snippet above). Additionally, all of the other server endpoints pointing at this database work, but they're only looking for individual playlists.
Any insights into this would be greatly appreciated, and of course I'm happy to provide any more details as needed.
Thanks!
Here's the issue,
database.Playlist.find({query})
You're not passing the query object but rather an empty object { } with {query}.
Do this instead,
database.Playlist.find(query)
Hope this solves the problem.

After successful db.collection.update always error is thrown

Am using Node.JS to update my collection data based on a logic.
The update happens as expected but in my console after the update it is not returning proper values. It is always returning errors. Below is the code snippet
db.collection('test').update(query,operator,options,function(err,docs){
if(errs) {
console.log('something went wrong somewhere');
}
});
The above snippet always prints something went wrong somewhere. Does that mean that the update is not successful.
I use MongoClient along with Node.js.
P.S: Newbie to MongoDB learning via 10gen courses

Elasticsearch index/cache not clearing/stuck

I'm using https://github.com/firebase/flashlight to index data for searches
However, this morning I deleted the whole firebase index, so it should be empty (this has worked before, but it seems that when the nodejs app.js crashes in some cases, causing the cache to get "stuck"), but I still see old search results from my nodejs app somehow...
I've tried:
http://localhost:9200/_cache/clear
and
http://localhost:9200/_flush
http://localhost:9200/firebase/_flush
They all say successful, but still I get old results, out of, seemingly nowhere.
I can also see in the console that it refreshes every 60 seconds, and, deleting the whole firebase has worked before without problems...
I even saw a message housekeeping: found 60 orphans (removing them now) in the console so it should be refreshed by now...
I tried restarting elasticsearch as well as the whole Linux/Debian server...
In the config.js I have two indexes:
exports.paths = [
{
path: "tags",
index: "firebase",
type: "tag",
filter: function(data) { return data.name !== 'system'; }
},
{
path: "tracks",
index: "firebase",
type: "track",
filter: function(data) { return data.name !== 'system'; }
}
];
And strangely enough, I have no problem whatsoever when using the 'track' store, instead of using the 'tag' one...
What am I missing here?
// Update !
So, I just deleted the firebase tracks index while the nodejs script was running and the script crashed... Same problem, different index. So the crashing script must cause it... so, how do I clear this stuck cache?
So I fixed it by simply doing:
curl -XDELETE localhost:9200/Firebase
Thanks to: https://github.com/elasticsearch/elasticsearch/issues/7541#issuecomment-54724302
I'm guessing Elastic search is not aware (and has not been told) of the relevance of its current index, perhaps the Flashlight script I'm using is not informing it about what the index should have been? But, since this is only necessary when the node script crashes when you suddenly delete your whole firebase index, it should be catchable somehow, but I'm happy that I can at least fix it like this. Rebuilding the index is not a big issue/task right now, but in the future it might be.
A wild guess, maybe you are not posting the queries correctly. You said you have tried following links:
http://localhost:9200/_cache/clear
http://localhost:9200/_flush
http://localhost:9200/firebase/_flush
If you are accessing the urls from browser it won't clear them. You have to POST them. It is ambiguous from your question if you did it, both GET and POST return same results (showing total, successful and failed). Try this from commandline using curl:
curl -XPOST 'http://localhost:9200/_cache/clear'
curl -XPOST 'http://localhost:9200/_flush'
Or create an AJAX request with JQuery or use fiddler.
Try to optimize your indeces by sending the following POST request to your elasticsearch server:
curl -XPOST 'http://localhost:9200/_optimize?max_num_segments=1&wait_for_merge=true'
It makes lucene really delete the deleted documents from disk and merge indeces.

Resources