I'm preparing to build a application (CMS) using ElasticSearch on Nodejs backend.
I'm choosing between elasticsearch native from ES and elasticsearch-odm from npm.
I'm test some example with CURL, Kibana RESTful API and see it simple to useful.
My system used Mongooese for other collections so i wonder if elasticsearch-odm reasonable proposal if it support same...
I am total new in this field, near realtime search as ES.
Please give me any infomation . . .
What's your question?
1: Run ES standalone or as part of the application?
It's better to run Elasticsearch standalone. You won't start ES every time when you start your application. Startingup ES can can take some time (starting, syncing; takes longer when you have a lot of data).
2: Is ES a good replacement for MongoDB?
Depends on your requirement. Elasticsearch is for SEARCH. Don't use is as (primary) database. Elasticsearch is not transactional! (there are more reasons why you won't do that)
Related
As a new developer, I am always confused about using NodeJS and MongoDB. So I hope I will get an actual answer. Thanks...
MongoDB and NodeJS are two different technologies. MonogDB is a database system which gives you a chance to efficiently store documents in a database and to perform operations like data updates, or to search documents by some criterias.
NodeJS's responsibilty is especially to execute your application.
Nodejs is a Javascript engine that you can write any application you want with (by programming in the Javascript language). It runs your Javascript code. Most commonly, it is used to build servers that can respond to web requests, though it can be used for lots of other types of code too.
MongoDB is a database engine. Code within some application or server uses MongoDB to save, query or update data in a database. There are many web servers built with nodejs that will then use MongoDB for storing data.
MongoDB offers an API library that runs within a Nodejs application to give you programmatic access to MongoDB so you can create databases and then add, query, update or delete data from the MongoDB database. MongoDB also has API libraries for other programming environments such as Python, Java, etc...
These two technologies are for different parts of a typical web server system. You don't substitute one for the other. Instead, you can use them together.
When should we use Nodejs?
Any project needs a programming environment and a runtime library that offers you basic programming tools/support and can compile and/or interpret your code. Nodejs is such as tool for the Javascript programming language. There are other similar tools for other languages such as Python, Java, PHP, C#, C++, Go, etc...
So, if you want to write some kind of stand-alone program or server in Javascript, then you can use nodejs for it.
When should we use MongoDB?
If your application needs the ability to persistently store data in a way that you can efficiently query or update it later, then you would typically use some form of database. There are dozens of popular databases. MongoDB is one such database. MariaDB, MySql, CouchDB, DynamoDB (on AWS), Postgres are examples of other databases. Different databases have different strengths (things they are best at) and different ways of using them so it's a whole different question to choose the right/best database for what you're doing.
Here is a Blog link that will tell you when you should use NodeJS.
Here is another link to understand when you should use mongoDB.
Well, NodeJS and MongoDB are two different process. Here I tried to explain as simpler as possible:
NodeJS
Shortly we can say, NodeJS is a JavaScript runtime environment. It's actually helps JavaScript to run outside of server. It's used in server side development.
MongoDB
But, MongoDB is NoSQL database which is document oriented. It represents data as of JSON documents. It's used for store data.
Summary
The summary is MongoDB is a database where we can store data and NodeJS helps us to to connect our client site to database by it's server site.
Example:
Suppose you are building a website and you need a database to store the data or information so here you can use MongoDB but to be connected with MongoDB you need a connector, so here you can use NodeJS which will help your website to run outside of server.
Nodejs: Node.js is an interpreter or runtime/ running environment for JavaScript. built on Chrome's V8 JavaScript engine. responsibility is especially to execute your application.
MongoDB: is a No-SQL database for storing data. when you need high availability of data with automatic, fast, and instant data recovery.
I am new to Elasticsearch and also confused how do I actually start implementing it. I have developed an office management software where on a daily basis tasks and other information based to that task belonging to a specific clients are stored. I have written API's in nodejs and the front-end in vuejs and MySQL db is used. So I want to implement a search functionality using Elasticsearch wherein user can search the tasks with any parameters they would like to.
Listed below are some of my questions
Now do Elasticsearch will work as an another db. If so, then how do I keep the record updated in Elasticsearch db as well.
Would it effect the efficiency in any way.
Also what is kibana and logstash in simple terms.
Is implementing Elasticsearch on client side is a good idea? Is Yes, then how can I implement Elasticsearch and kibana using vuejs?
I am confused with all the above things, can anyone kindly share their knowledge on the above listed questions and also tell which articles/docs/videos should I refer for implementing Elasticsearch in the best possible way?
Elasticsearch
It is a data store, all the JSON data will(Single Record/Row) be stored in indexes(Tables)
Update the records in elasticsearch using your backend only, even though we have packages available to connect the frontend to Elasticsearch.
Efficiency, nothing gets affected except the new stack in your application.
Implementing elasticsearch on the client-side is not a recommended option, Same code same API can be used till your MySQL DB connection, add a function to save update any data along with MySQL save call.
Example : MySQLConfig.SaveStudent(student)
ElasticsearchConfig.SaveStudent(student)
Till here there is no code change needed to save/update/delete/getByPrimaryID/GetByParamSearch,
For `getByPrimaryID/GetByParamSearch` search, you have to create a different API either to elasticsearch to MySQL but not both.
Kibana
GUI for your Elasticsearch - Look at it like dbForge Studio, MySQL Workbench, phpMyAdmin
Other than GUI it has a lot of other functionalities like cluster monitoring, all the elastic stack monitoring, analytics, and so on.
Logstash
It ships many files and save it into elasticsearch index, this is not needed until u need it for use cases like
application-prod.log to searchable index
Kafka topic to searchable index
MySQL Table to searchable index
Huge list of use-cases available to ship anything and make it a searchable index
To understand clearly about index, mappings, document in elasticsearch vs database, table, scheme, record in MySQL read from here
My customer has an existing application built on Node and MongoDB. In this app, there is a search functionality which is more complex. I want to convert it to Elasticsearch. How can I achieve it? I have installed Elasticsearch on my system (Windows) and now I am having confusion that, do I need to put all data in Elasticsearch? No, I don't know the concept. How can I implement Elasticsearch to the current app? Is Elasticsearch a separate database like MongoDB?
I am about to add the search engine to my Node application. I can see there is a npm package for ElasticSearch.
But why not simply using ElasticSearch standalone instead of nesting it with the application code?
The use case for me is:
Data is in a MySQL database with a true entity-relationship-diagram.
I intend to use Docker containers to ship my application elements (which can be modified as for now).
The results' positioning is intended to be complex, and I thought of a nightly cronjob that would re-evaluate the documents/results, because they depend on moving data (user's reputation or popular sales, for instance).
Don't you think a standalone ElasticSearch instance / grid is more enviable? I imagine that it would be more scalable, more secure, and easier to deploy in a cloud, for instance...
The npm package you refer to is a Javascript client for Node and in-browser use, that can be used to perform searches, index (create/insert) documents on the Elasticsearch server. What you refer to as Elasticsearch standalone is the server and yes, that can and should be entirely separate from the client application code.
It is definitely possible to not use a client library and access the Elasticsearch REST API directly using whatever HTTP client code, but then it is too easy to reinvent the wheel, badly.
My app has Node JS. I'm trying to connect NodeJS and Neo4j together. Can some one tell me, how to connect both? My queries need to work with labels on Neo4j. Please let me know which module should I use in Node Js to achieve this?I have spent lot of time already with-out luck.
Last I checked there are at least 4 popular and actively developed node.js modules (ordered by number of stars):
https://github.com/thingdom/node-neo4j (npm install neo4j)
https://github.com/bretcope/neo4j-js (npm install neo4j-js)
https://github.com/philippkueng/node-neo4j (npm install node-neo4j)
https://github.com/brikteknologier/seraph (npm install seraph)
They all support the Cypher endpoint, which was a requirement for my inclusion. One key feature that stands out from the list is that philippkueng/node-neo4j is the only one that has transactional API support. Another is the ability to ask for labels of nodes, and that is supported only by seraph and philippkueng/node-neo4j. (usually you can avoid needing to ask for labels of a node if you make your Cypher query ask for labels explicitly, which avoids a request back and forth)
On the other hand, it's really not hard to just implement a few HTTP requests, directly accessing the Cypher or Transactional Cypher endpoints, massaging the results as you see fit for your application.
Another cool new development I've seen recently was https://github.com/brian-gates/cypher-stream, which emits a stream of results from Cypher, enabling streaming JSON parsing, which is another performance-oriented feature lacking from the four listed above.
Edit: 03/2016 There is a new official JS driver for use with the new bolt protocol (binary). For new development this should definitely be considered. Bolt is planned for release in Neo4j 3.0. https://github.com/neo4j/neo4j-javascript-driver
Check out the koa-neo4j framework, it uses the official neo4j-driver under the hood.
One can write native Cypher (as .cyp files) in it on top of the latest stable neo4j (3.0.3 at the time of this writing) which, among other things, allows querying labels.
https://github.com/assister-ai/koa-neo4j
https://github.com/assister-ai/koa-neo4j-starter-kit
In a Neo4j enabled application, conducting queries directly from client side might not be the best choice:
Database is exposed to the client, unless some explicit security mechanism is in place; one can see the innards of the database by View page source
There is no one server to rule them all, queries are strings, scattered around different clients (web, mobile, etc.)
Third-party developers might not be familiar with Cypher
koa-neo4j addresses all of the above issues:
Stands as a middle layer between clients and database
Gives structure to your server's logic in form of a file-based project; finally a home for Cypher! All of the clients can then talk to an instance of this server
Converts Cypher files to REST routes, a cross-platform web standard that developers are familiar with, it does so on top of the widely-adapted koa server, ripe for further customization
Disclosure I was the original author of koa-neo4j
neode - Neo4j OGM for Node JS. here