In my application im using node.js with redis database.How can i save many json objects in redis.
db.save({
description:'sdsd',userId:'324324',url:'http://www.abc.com/abc.html',appId:'123456'
}, function (err, res) {
if(err){
console.log(err);
return;
}
else{
console.log(res);
}
});
In couch db we can save the json objects again and again as document by using the above code.How to do this in redis.From their documentation i came to know below code save the json objects
client.hmset("hosts", "mjr", "1", "another", "23", "home", "1234");
Again i want to save the other json object in same "hosts" like below
client.hmset("hosts", "mjr", "2", "another", "33", "home", "1235");
How can i do this.
Redis storage model is different from CouchDB. In Redis, everything gets accessed by its key, so it all depends how you plan to retrieve your data.
So if you'd like to be able to retrieve data by userId, use this as the key.
redis.set('324324', {
description:'sdsd',
url:'http://www.abc.com/abc.html',
appId:'123456'
});
But if you need to retrieve a piece of data using more than one piece of the data, then redis may not be suitable.
In some cases, you may be able to use some tricks, so that to be able to query on both userId and appId, you could use 324324:123456 as the key, and query using
GET 324324:*
to get all apps for one user
or
GET *:123456
to get all users for a given app.
Related
I am using Cloud Function to send a notification to mobile device. I have two collection in Firestore clientDetail and clientPersonalDetail. I have clientID same in both of the collection but the date is stored in clientDetail and name is stored in clientPersonal.
Take a look:
ClientDetail -- startDate
-- clientID
.......
ClientPersonalDetail -- name
-- clientID
.........
Here is My full Code:
exports.sendDailyNotifications = functions.https.onRequest( (request, response) => {
var getApplicants = getApplicantList();
console.log('getApplicants', getApplicants);
cors(request, response, () => {
admin
.firestore()
.collection("clientDetails")
//.where("clientID", "==", "wOqkjYYz3t7qQzHJ1kgu")
.get()
.then(querySnapshot => {
const promises = [];
querySnapshot.forEach(doc => {
let clientObject = {};
clientObject.clientID = doc.data().clientID;
clientObject.monthlyInstallment = doc.data().monthlyInstallment;
promises.push(clientObject);
});
return Promise.all(promises);
}) //below code for notification
.then(results => {
response.send(results);
results.forEach(user => {
//sendNotification(user);
});
return "";
})
.catch(error => {
console.log(error);
response.status(500).send(error);
});
});
}
);
Above function is showing an object like this
{clienId:xxxxxxxxx, startDate:23/1/2019}
But I need ClientID not name to show in notification so I'll have to join to clientPersonal collection in order to get name using clientID.
What should do ?
How can I create another function which solely return name by passing clientID as argument, and waits until it returns the name .
Can Anybody please Help.?
But I need ClientID not name to show in notification so I'll have to join to clientPersonal collection in order to get name using clientID. What should do ?
Unfortunately, there is no JOIN clause in Firestore. Queries in Firestore are shallow. This means that they only get items from the collection that the query is run against. There is no way to get documents from two top-level collection in a single query. Firestore doesn't support queries across different collections in one go. A single query may only use properties of documents in a single collection.
How can I create another function which solely return name by passing clientID as argument, and waits until it returns the name.
So the most simple solution I can think of is to first query the database to get the clientID. Once you have this id, make another database call (inside the callback), so you can get the corresponding name.
Another solution would be to add the name of the user as a new property under ClientDetail so you can query the database only once. This practice is called denormalization and is a common practice when it comes to Firebase. If you are new to NoQSL databases, I recommend you see this video, Denormalization is normal with the Firebase Database for a better understanding. It is for Firebase realtime database but same rules apply to Cloud Firestore.
Also, when you are duplicating data, there is one thing that need to keep in mind. In the same way you are adding data, you need to maintain it. With other words, if you want to update/detele an item, you need to do it in every place that it exists.
The "easier" solution would probably be the duplication of data. This is quite common in NoSQL world.
More precisely you would add in your documents in the ClientDetail collection the value of the client name.
You can use two extra functions in this occasion to have your code clear. One function that will read all the documents form the collection ClientDetail and instead of getting all the fields, will get only the ClientID. Then call the other function, that will be scanning all the documents in collection ClientPersonalDetail and retrieve only the part with the ClientID. Compare if those two match and then do any operations there if they do so.
You can refer to Get started with Cloud Firestore documentation on how to create, add and load documents from Firestore.
Your package,json should look something like this:
{
"name": "sample-http",
"version": "0.0.1",
"dependencies": {
"firebase-admin": "^6.5.1"
}
}
I have did a little bit of coding myself and here is my example code in GitHub. By deploying this Function, will scan all the documents form one Collection and compare the ClientID from the documents in the other collection. When it will find a match it will log a message otherwise it will log a message of not matching IDs. You can use the idea of how this function operates and use it in your code.
I'm trying a very simple CRUD API with the MEAN stack. I entered several documents into a mongolab sandbox db. I could do a GET on all documents and they would all be returned. Then I tested GET by ID:
router.route('/api/v1/resources/:resource_id')
// get the resource with that id (accessed at GET http://localhost:8080/api/resources/:resource_id)
.get(function(req, res) {
Resource.findById(req.params.resource_id, function(err, resources) {
if (err)
res.send(err);
res.json(resources);
});
});
And it simply wouldn't work. I kept getting null. But the document existed. I could see it when I would do a GET on all documents.
Then I got another document to return, finally.
The difference? The record that returned had an id in this format:
{
"_id": { "$oid":"1234567890abc"}
}
But records that did not return had this:
{
"_id": "1234567890abc"
}
Can Anyone explain this to me? I entered the data with Postman and I didn't do anything different between the entries.
What is $oid?
What creates the $oid?
Why does that nesting matter for mongoose's findById()?
Thanks!
$oid is from Strict MongoDB Extended JSON.
All your queries to MongoDB database that contains _id conditions should wrap _id in ObjectId function like the following:
db.resources.findOne({_id: ObjectId("507c35dd8fada716c89d0013")};
MongoLab provides UI for querying to MongoDB via JSON syntax. But you can't use ObjectId in JSON due specification restrictions.
So, MongoLab uses Strict MongoDB Extended JSON for alias ObjectId() -> $oid.
{"_id": {"$oid":"507c35dd8fada716c89d0013"})
Same $oid you see in the output because MongoLab UI uses JSON also.
Mongoose automatically converts a string _id to MongoDB query so you don't need doing it manually. The following queries are equivalent:
Resource.findById("507c35dd8fada716c89d0013");
Resource.findById(new mongoose.Types.ObjectId("507c35dd8fada716c89d0013"));
I have a database which have approx 600000 records.
I 'm using the sails.js but
when I'm fetching the data with waterline approach It takes very long time to fetch 600000 records (approx 17sec) and It has limited query i.e It doesn't have Accessing Join Tables. so I join take the result of two query and then filter the data that's by It take lot's of time.
So I decided to use MongoDB with sails instead of waterline and I'm wondering if I can somehow use the Blueprint API without being linked to a Waterline model.
How to use the MongoDB instead of the waterline?
If you want to use the Sails models api, you can overwrite the blueprint methods in the controller.
So to overwrite, say, User Model, create the following functions in UserController.js:
find, create, update, destroy
find will override 'get api/v1/user'
create will override 'post api/v1/user'
update will override 'put api/v1/user'
destroy will override 'delete api/v1/user'
Once inside the controller, you can run a native query on Mongo like so:
In UserControllelr.js
find: function (req, res) {
var packet = req.params.all();
// packet now has all url and posted parameters
User.native(function (err, UserCollection) {
if(err) {
//handle error
}
else {
// here you can run any native mongo Query using UserCollection
// eg:
UserCollection.aggregate(
{"$match": {"gender": "Male"} },
{"$group": { "_id": "$socialNetwork", "count": {"$sum":1} } },
function (err, results) {
//do stuff with results
})
}
})
}
Hope this helps.
As I understand it, when you enter something into GridFS it gets entered into 2 different collections under the hood. One for the raw chunks of data and one for the meta data files.
Also from what I understand from MongoDB's documentation is that you can only retrieve a document from GridFS with an id or name.
var gs = new mongodb.GridStore(db, "test.png", "w", {
"content_type": "image/png",
"metadata":{
"author": "Daniel"
},
"chunk_size": 1024*4
});
So what if I want to get a subset of documents from GridFS? For example what if I want all GridStores with:
metadata: {author: "Daniel"}
Why can't I use standard mongo queries { field: somevalue } and retrieve documents that way?
Does anybody know how this can be done? I'm using the javascript API on node.js.
You can query the db.files collection just like any other collection:
db.collection('fs.files')
.find({ 'metadata.author' : 'Daniel' })
.toArray(function(err, files) {
if (err) throw err;
files.forEach(function(file) {
var gs = new mongodb.GridStore(db, file._id, 'r');
...
});
});
Although instead of plain forEach you may want to use async.each or any of the other async.* methods.
I have a Problem in with my REST API. It will not correctly communicate with mongodb. Node.js throw following Exception:
Error: Argument passed in must be a single String of 12 bytes or a string of 24 hex characters
My Manager looks like follows:
app.get('/users/:id', function(req, res) {
userProvider.fetchUserById(req.params.id, function(error, user) {
if (user == null) {
res.send(error, 404);
} else {
res.send(user);
}
});
});
My Provider looks like follows:
this.fetchUserById = function(id, cb) {
this.db.collection(usersTable, function(error, users) {
if (error) {
cb(error, null);
} else {
users.findOne({
_id:users.db.bson_serializer.ObjectID.createFromHexString(id)
}, function(error, result) {
cb(error, result);
});
}
});
};
My Mongoobject looks like follows:
{
"title": "title",
"givenname": "Vorname",
"lastname": "Nachname",
"name": "Vorname Nachname",
"username": "Benutzername",
"password": "Kennwort",
"email": "kuerzel#emailadresse.de",
"phone": "0000 - 000000",
"fax": "000000 - 000000",
"lastlogin": "15.11.2013",
"createdate": "15.11.2013",
"changedate": "15.11.2013",
"state": "1",
"_id": ObjectId("5283fbf56e3adb01f1000001")
}
When I now send a GET: /users/1 the error will thrown.
I´am new in Node.js. :-)
If your human readable IDs are unique, you can use them.
Just insert the documents with your nice id:
db.users.insert({ _id: myVeryNiceUniqueId, title: '...' }, ...);
As alternative, you could add an additional 'external id' to the document and create an unique index. But you will get many, many problems when you want to scale (sharding). Who is going to create the unique IDs? You will have something not-so-human-readable, I promise.
Edit
In Mysql I have start the Ids from 5XXXX and used them. How can I realize this with mongodb
You should ask another question or search answers. It is not a trivial task and you have to take concurrency into account. Maybe the concept of findAndModify or "Upsert" could help you. One of the most important things is that you need a way of atomic updates for the ID-generator. In one project in past I used Redis for this, but I'd always use MongoDB IDs since then. You should know that MongoDB ObjectIDs usually are generated by the Client-Driver. For REST services this is ideal: save document to DB and return status 201 Created, url=.../user/98357348957.
btw. should I generate the OrderIDs in Javascript by checking the highest value of an external id in the collection?
As said, you can get into trouble with usage of duplicate IDs when your server is in high contention. Try to find a solution with findAndModify or cache a block of IDs and use them in one process. I probably wouldn't search max and increment but use a special collection for ID-management.
What is here best practice?
Don't know. Using an ID-Generator. Depends on your enviroment (single process, single machine). If it is a simple setup, you'll find simple solutions. The setup for Twitter is not simple and they have developed a solution called twitter / snowflake. If you really want very nice, short IDs, but you have more than one process/machine, Redis might be want you want. And if the ID doesn't have to be contigous (without small gaps), you could draw a bunch of new IDs (e.g 100) and cache them for future consumption per process.
Can I hash the objectID to an 5 digit number?
No, there is no guarantee that this number is unique.