I have been trying to find out the indexes which are already created via MongoDB manually( I have created 2d sphere indexes for two fields via mongobooster and creating one via schema by defining it). Now if i run this query in mongodbooster
db.collectionname.getIndexes();
It results me the 3 documents with name.key and which indexes i have used. I want to perform this same operation in mongoose i can't find a equivalent query for the same. I tried this
const indexes = OrderSchema.indexes();
console.log('index:', indexes);
But it gives me only one index which i have defined in schema that is _id i need two other fields as well which contains 2d-sphere index how can i get that too. What am trying to achieve here is if 2d sphere indexes are already created don't create an index else create an index that's all am trying to achieve here. Any help is appreciated Thanks
Yeah, you can't do it with a schema. You will need to create the model first, and then you can do something like this:
Order.collection.getIndexes({full: true}).then(indexes => {
console.log("indexes:", indexes);
// ...
}).catch(console.error);
If you dont have access to mongoose model, but the mongoose connection was created and you need to get the indexes from a collection you can access by this way:
const mongoose = require('mongoose');
mongoose.connect('myConnectionString', { useNewUrlParser: true }).then(() => {
getIndexes();
});
const getIndexes = async () => {
const indexes = await mongoose.connection.db.collection('myCollection').
indexes.forEach(function (index) {
console.log(JSON.stringify(index));
});
};
Related
My code below is returning NULL immediately when it should return a document, seems to be specific to this collection as my other findOne's for different collections are working as expected.
function findData(rowObj) {
console.log(rowObj.field);
console.log(Mongoose.connection.readyState);
return DataImport.findOne({
ROW: rowObj.field
});
}
My collection has many fields, ideally I wouldn't use a schema as I am not updating or adding from this collection, just reading from it;
const mongoose = require('mongoose');
const { Schema } = mongoose;
const dataImportSchema = new Schema({});
const DataImport = mongoose.model(
'DataImport',
dataImportSchema,
'DataImport'
);
module.exports = DataImport;
I have however tried this with all of the document fields in the schema but get the same result.
Is it possible an issue with the collection being too large? It is 30GB with around 40 million documents.
I seem to have been using an old way of specifying the collection name and should have had it in the schema definition: https://mongoosejs.com/docs/guide.html#collection
`
TL;DR: Is there a safe way to dynamically define a mongoose discriminator at runtime?
I have an app with a MongoDB collection where users have some control over the underlying schema.
I could add one or two fixed, required fields and just use mongoose.Mixed for the remainder that users can change, but I'd like to make use of Mongoose's validation and discriminators if I can.
So, what I've got is a second collection Grid where the users can define the shape they'd like their data to take, and in my main model Record, I've added a function to dynamically generate a discriminator from the definition in the second collection.
The code for my Record model looks like this:
const mongoose = require("mongoose")
const recordSchema = new mongoose.Schema({
fields: {
type: Array,
required: true
}
}, {
discriminatorKey: "grid"
})
const Record = mongoose.model("Record", recordSchema)
module.exports = grid => {
// Generate a mongoose-compatible schema from the grid's field definitions
const schema = grid.fields.map(field => {
if(field.type === "string") return { [field.name]: String }
if(field.type === "number") return { [field.name]: Number }
if(field.type === "checkbox") return { [field.name]: Boolean }
return { [field.name]: mongoose.Mixed }
})
return Record.discriminator(grid._id, new mongoose.Schema(schema))
}
This is inside an Express app, and I use the model in my middleware handlers something like this:
async (req, res) => {
const grid = await Grid.findById(req.params.id)
const Record = await GenerateRecordModel(grid)
const records = await Record.find({})
res.json({
...grid,
records
})
}
This works great on the first request, but after that I get an error Discriminator with name “ ” already exists.
I guess this is because only one discriminator with its name per model can exist.
I could give every discriminator a unique name whenever the function is called:
return Record.discriminator(uuidv4(), new mongoose.Schema(schema), grid._id)
But I imagine that this isn't a good idea because discriminators seem to persist beyond the lifetime of the request, so am I laying the groundwork for a memory leak?
I can see two ways forward:
COMPLICATED? Define all discriminators when the app boots up, rather than just when a HTTP request comes in, and write piles of extra logic to handle the user creating, updating or deleting the definitions over in the Grid collection.
SIMPLER? Abandon using discriminators, just use mongoose.Mixed so anything goes as far as mongoose is concerned, and write any validation myself.
Any ideas?
I'm trying to query documents near a given point in given radius in Firestore with using GeoFirestore for my node.js project. Firstly, I have a document in the database in this format:
My query code is like this:
// Create the GeoFirestoreQuery object
const geoQuery = geoFirestore.query({
center: new firebase.firestore.GeoPoint(latitute, longitude),
radius: 5,
query: (ref) => ref.where("d.available", "==", true)
});
The thing is that, I cannot figure out how to get the document from the query. I tried the code below, but it simply is not called at all.
// Fetch the nearest couriers
const onKeyEnteredRegistiration = geoQuery.on("key_entered", function(key, document, distance) {
console.log(key);
});
How can I return the document? What I am missing? Is my query and database structure matches? Is there any missing fields or issues? Or should I use another query instead of "key_entered"? Thanks for any help in advance.
I found the issue, it was about the database structure. I changed my structure as shown below and now it works perfectly.
I want to get all distinct keys from a collections in mongoDB.
I refereed the following links:
Get names of all keys in the collection
Querying for a list of all distinct fields in MongoDB collection and etc.
But still i didn't get the right solution...
As i am using mongoose in the first link reference syas runCommand is not a function.
As findOne() will give the first document keys alone but i need all distnct keys
userModel.findOne(condition, projection, callback)
Please share your ideas..
If you are using Mongoose 3.x, then you can try this :
userModel.find().distinct('_id', function(error, ids) {
// ids is an array of all ObjectIds
});
Or you can find all the documents and extract key from that like :
var keys = {};
var docKeys = [];
userModel.find({}, function(err, allDocs){
allDocs.forEach(function(doc){
docKeys = Object.keys(doc);
docKeys.forEach(function(docKey){
if(!keys[docKey]) keys[docKey] = true;
})
})
})
I have just written for getting logic, you can change according to your requirements and efficiency
Try like this you will get all of your keys defined into your mongoose model/Schema.
import Model from 'your_model_path'
for(let property in Model.schema.obj){
console.log("key=====>",property);
}
I am running an iOS app where I display a list of users that are currently online.
I have an API endpoint where I return 10 (or N) users randomly, so that you can keep scrolling and always see new users. Therefore I want to make sure I dont return a user that I already returned before.
I cannot use a cursor or a normal pagination as the users have to be returned randomly.
I tried 2 things, but I am sure there is a better way:
At first what I did was sending in the parameters of the request the IDs of the user that were already seen.
ex:
But if the user keeps scrolling and has gone through 200 profiles then the list is long and it doesnt look clean.
Then, in the database, I tried adding a field to each users "online_profiles_already_sent" where i would store an array of the IDs that were already sent to the user (I am using MongoDB)
I can't figure out how to do it in a better/cleaner way
EDIT:
I found a way to do it with MySQL, using RAND(seed)
but I can't figure out if there is a way to do the same thing with Mongo
PHP MySQL pagination with random ordering
Thank you :)
I think the only way that you will be able to guarentee that users see unique users every time is to store the list of users that have already been seen. Even in the RAND example that you linked to, there is a possibility of intersection with a previous user list because RAND won't necessarily exclude previously returned users.
Random Sampling
If you do want to go with random sampling, consider Random record from MongoDB which suggests using an an Aggregation and the $sample operator. The implementation would look something like this:
const {
MongoClient
} = require("mongodb");
const
DB_NAME = "weather",
COLLECTION_NAME = "readings",
MONGO_DOMAIN = "localhost",
MONGO_PORT = "32768",
MONGO_URL = `mongodb://${MONGO_DOMAIN}:${MONGO_PORT}`;
(async function () {
const client = await MongoClient.connect(MONGO_URL),
db = await client.db(DB_NAME),
collection = await db.collection(COLLECTION_NAME);
const randomDocs = await collection
.aggregate([{
$sample: {
size: 5
}
}])
.map(doc => {
return {
id: doc._id,
temperature: doc.main.temp
}
});
randomDocs.forEach(doc => console.log(`ID: ${doc.id} | Temperature: ${doc.temperature}`));
client.close();
}());
Cache of Previous Users
If you go with maintaining a list of previously viewed users, you could write an implementation using the $nin filter and store the _id of previously viewed users.
Here is an example using a weather database that I have returning entries 5 at a time until all have been printed:
const {
MongoClient
} = require("mongodb");
const
DB_NAME = "weather",
COLLECTION_NAME = "readings",
MONGO_DOMAIN = "localhost",
MONGO_PORT = "32768",
MONGO_URL = `mongodb://${MONGO_DOMAIN}:${MONGO_PORT}`;
(async function () {
const client = await MongoClient.connect(MONGO_URL),
db = await client.db(DB_NAME),
collection = await db.collection(COLLECTION_NAME);
let previousEntries = [], // Track ids of things we have seen
empty = false;
while (!empty) {
const findFilter = {};
if (previousEntries.length) {
findFilter._id = {
$nin: previousEntries
}
}
// Get items 5 at a time
const docs = await collection
.find(findFilter, {
limit: 5,
projection: {
main: 1
}
})
.map(doc => {
return {
id: doc._id,
temperature: doc.main.temp
}
})
.toArray();
// Keep track of already seen items
previousEntries = previousEntries.concat(docs.map(doc => doc.id));
// Are we still getting items?
console.log(docs.length);
empty = !docs.length;
// Print out the docs
docs.forEach(doc => console.log(`ID: ${doc.id} | Temperature: ${doc.temperature}`));
}
client.close();
}());
I have encountered the same issue and can suggest an alternate solution.
TL;DR: Grab all Object ID of the collections on first landing, randomized using NodeJS and used it later on.
Disadvantage: slow first landing if have million of records
Advantage: subsequent execution is probably quicker than the other solution
Let's get to the detail explain :)
For better explain, I will make the following assumption
Assumption:
Assume programming language used NodeJS
Solution works for other programming language as well
Assume you have 4 total objects in yor collections
Assume pagination limit is 2
Steps:
On first execution:
Grab all Object Ids
Note: I do have considered performance, this execution takes spit seconds for 10,000 size collections. If you are solving a million record issue then maybe used some form of partition logic first / used the other solution listed
db.getCollection('my_collection').find({}, {_id:1}).map(function(item){ return item._id; });
OR
db.getCollection('my_collection').find({}, {_id:1}).map(function(item){ return item._id.valueOf(); });
Result:
ObjectId("FirstObjectID"),
ObjectId("SecondObjectID"),
ObjectId("ThirdObjectID"),
ObjectId("ForthObjectID"),
Randomized the array retrive using NodeJS
Result:
ObjectId("ThirdObjectID"),
ObjectId("SecondObjectID"),
ObjectId("ForthObjectID"),
ObjectId("FirstObjectID"),
Stored this randomized array:
If this is a Server side script that randomized pagination for each user, consider storing in Cookie / Session
I suggest Cookie (with timeout expired linked to browser close) for scaling purpose
On each retrieval:
Retrieve the stored array
Grab the pagination item, (e.g. first 2 items)
Find the objects for those item using find $in
.
db.getCollection('my_collection')
.find({"_id" : {"$in" : [ObjectId("ThirdObjectID"), ObjectId("SecondObjectID")]}});
Using NodeJS, sort the retrieved object based on the retrived pagination item
There you go! A randomized MongoDB query for pagination :)