I am fairly new to both mongodb and node.js but recently got everything to work well for me until I reached the point where I needed to add a full text search to my website. From my research I figured out that Elasticsearch would be a good fit, but I couldnt figure out exactly how to get it to work with node.js and mongodb. I am currently using Heroku and MongoLab to host my application. Here are my questions.
How do I host Elasticsearch?
How do I make all my mongo data available to elasticsearch do I use a river or do I manually inset and delete all data?
I found something this river but I am not quite sure how to make this happen automatically and where to host it.
How do I query Elasticsearch from node.js? Is there a package that allows for this?
Edit:
Question 2 is really what I am struggling with. I have also included question 1 and 3 to help people that are new to the topic and coming from google.
1) either on your own server/VM/whatever.. or with a hosted service such as https://searchbox.io/
2) you can create a script to index your existing data and then index new data once its created, or use a river to index your current database.
3) ElasticSearch is a simple HTTP API, you can make your own requests using the 'http' module or simplify it with something like https://github.com/mikeal/request
you can also use a 3rd party library like https://github.com/phillro/node-elasticsearch-client
Searchly.com (Aka SearchBox.io)introduced a new feature crawlers includes MongoDB crawler.
It fetches data from a given collection and syncs periodically to ElasticSearch. Check http://www.searchly.com/documentation/crawler-beta/
You can host on your own server or use aws elasticsearch service or use elastic cloud provided by elasticsearch.
Try any of the following three solutions:-
i) Try using mongoosastic. NPM package
ii) Use mongo-connector.
iii) python script for indexing data to elasticsearch
elasticsearch-js. The javascript client library
Related
I am working with NodeJS and Cloudant (alternatively the DashDB Warehouse if that works better). I wonder if it is possible to have a function in NodeJS that gets called each time a document has been added to the database? I have checked out indexed views but can't really understand how to do it. Does anyone have any good tips regarding this or what documentation to look at?
You can listen to DB _changes in Cloudant (https://console.bluemix.net/docs/services/Cloudant/api/database.html) in continous mode.
Every document change in the Cloudant DB (create,update,delete) will be notified through this channel.
There are different nodejs libraries you can use with this purpose. This is
one example: https://www.npmjs.com/package/cloudant-follow
I am trying out React-Starter-Kit for the first time and loving all the cutting edge features baked in (apollo/graphql-client in particular). A crucial part of any app for me is the database, and for that my understanding is the same author provides nodejs-api-starter which sets up a REST interface for accessing Postgres at localhost:5000 and has a graphql webui at localhost:5000/graphl.
That is about as far as I have been able to understand of the setup so far. I have changed the frontend code a little bit so a new Component "Counter" is loaded on the home page. I need to be able to make a new counter, fetch the latest counter, and increment decrement the counter. Write now the component just outputs the 'value' retrieved from the server at 5000.
I do not think I am accessing the 5000 server correctly, do I put the port in this url line somehow?
You can pull the repo down from : https://github.com/Falieson/react-starter-kit-crud-counter-demo
This is my first time setting up a nodejs api server, I am used to using MeteorJS which has pub/sub to MongoDB baked in. I am looking forward to the separation the RSK strategy (which seems more industry standard?) provides.
I've just done setting up the full site with Database from React-Stater-Kit, I'm also a newbie so I understand your frustration.
About this question, you don't need the NodeJS-API-Starter, it has enhanced function ( such as Redis cache ) and it's not suited for newbies. You should look deeper into the RSK, it already has the DB. If you ran the boilerplate and played around, change is you'll see file database.sqlite in your folder, it's the database. Here are the things you should learn:
Use SequelizeJS to connect the NodeJS server with database. Your database can be MySQL/MariaDB, PostgreSQL or SQLite. The connection is easy and there's tool to auto-generate Models from your database
How to create GraphQL's Types and Queries. If your queries need to search through the database, import Sequelize's models and use its functions.
Test your API via GraphQLi
Note: if you want to use MongoDB or other NoSQL, try Mongoose instead of Sequelize.
I have an nodejs server running witch show data on a web interface. The data is fetched from a MongoDB using mongoose. The data is added via an node-red application witch is isolated from the rest.
Currently my nodejs server fetches the data every 5 seconds. Is there a way to know if the data in my MongoDB has changed?
Thanks, I hope my question is clear.
I was also looking for something similar to what you are asking for few months back. Few ways which i know to do it are:
1) You can try to use middlewares while inserting your documents in DB. You can then send that new data either after saving it in DB or at the time of insertion only.
2) Refer to this answer which talks about solving your problem using inbuilt functions provided by mongoDb. You can study in deep about them in mongoDb docs.
3) There is also another way to do this which includes listening to changes in log files. As you know everything done in mongo is recorded and logged in files so whenever there is some change in data you can know it from there also. You will have to do the digging by yourself in this method.
Hope it helps!
Hi i have started working the neo4j and the seraph and seraph-model packages.
A problem poped:
I cannot seem to find an way to create a Node connected to other node in one query example:
Create (n:User)-[r:Has]->(p:UserImage)
I know I can do that using a the native seraph.query but then I lose some of the model features.. (like the timestamps)
Is there some otherway to do that?
How expensive is to do that query in 3 steps? (create user, create image, link then)
seraph-model is extension of seraph. seraph.query is for raw query. model features are accessible only when use modelName.Method
I want to clone a public api into a mongodb database with nodejs so I can run my own queries and extend the data like I want. My problem is that I want to fetch the newest updates as well. The good thing is that I know when the data will change at the api app approximately.
I thought about something like cronjobs or a setInterval.
In the internet I could find many solutions how to transform a database into a rest api but no vice versa.
Please give me suggestions how to solve this problem.