I have my data on MongoDB atlas and want the data from my atlas to sync to my elastic search server so that I can get all my data up for a quick search with an elastic search and NodeJs. How do I go about this?
NB: I know that using the NodeJs package, mongoosastic, you can sync that data coming from a request between the database and the elastic search.
The confusion is that if I quit my server and bring it up again, how will all my data sync to elastic search for a proper search?
Related
I've just built out my first elasticsearch datastore using the documentation and that was great. I've also set myself up a lambda to make PUT and GET calls to the elasticsearch datastore but having read multiple resources, I'm still unclear as to how to to connect my lambda up to the elasticsearch to PUT data onto it.
I am dividing the load on my Database and want to retrieve data from ES and write data to MongoDB. Can I sync them real time? I have checked the Transporter library but I want to do it for realtime.
There are several ways to achieve that :
Using your own application server. Whenever you are inserting a new
document in the mongo, put it in the ES as well at the same time.
That way you will maintain the consistency with minimum latency.
Use logstash. It has near realtime pipelining capabilities.
You can use elasticsearch mongodb river. Its a plugin used for data synchronization between mongo and elasticsearch.
I am having a problem with retrieving values from my rethinkdb database and exposing them in my API.
Everything is running without errors, but I get different results when querying the database with python then the API.
I have created the database, tables and inserted data with REPL queries in python.
My setup is like this:
- AWS ec2 (Ubuntu)
- Rethinkdb as database
- Node API: clone of https://github.com/yoonic/atlas
I have no clue why there is a difference or where to look next for debugging.
Any help to get me going is appreciated!
guys. I have a local database on mongodb. I also have an identical db on mongolab. In the mongolab db, I append document/entries regularly. What I want to do is to sync these changes on my local mongodb collection. I'm currently thinking of doing a periodic query on the mongolab collection to see if the document count increases and if it does, I would know that there are new documents to account for.
Is there a more efficient way to do this? Thanks!
I am new to node js and I am trying to work with elastic search.
Tried a few clients
node-es
https://github.com/phillro/node-elasticsearch-client
sage
elasticsearch
All of them had issues. My questing is:
what's the advantage of a node client against just GET'ing the results from the elastic search server? is it for better performance (keeping the connection open)?
Is it for simplicity? (because i got to tell you, they are not so simple to use..)