I am new to node js and I am trying to work with elastic search.
Tried a few clients
node-es
https://github.com/phillro/node-elasticsearch-client
sage
elasticsearch
All of them had issues. My questing is:
what's the advantage of a node client against just GET'ing the results from the elastic search server? is it for better performance (keeping the connection open)?
Is it for simplicity? (because i got to tell you, they are not so simple to use..)
Related
I have my data on MongoDB atlas and want the data from my atlas to sync to my elastic search server so that I can get all my data up for a quick search with an elastic search and NodeJs. How do I go about this?
NB: I know that using the NodeJs package, mongoosastic, you can sync that data coming from a request between the database and the elastic search.
The confusion is that if I quit my server and bring it up again, how will all my data sync to elastic search for a proper search?
Issue
I have an Express (Node.JS) + MongoDB app with a server response load time of 4 - 7 seconds on average (slow).
I understand that the average server response time is under 200ms as per google pagespeed tools.
This app is fetching data from a mongoDB asynchronously but the roundtrip times to the database is extremely slow with each call averaging about 500ms - 1s. These calls are simple findAll calls to retrieve data of less than < 100 records.
Context
Mongoose version: 4.13.14
DB server's MongoDB version is 3.4.16
DB server is hosted on MongoDB Atlas M10 in AWS / Oregon (us-west-1)
Web server is hosted with now.sh in SFO1 (us-west-1)
Have performed recommended indexes as advised by MongoDB Atlas's performance advisor
Data fetching perfectly fine in local environment (local server + local db) as data is queried in a matter of few ms
Mongoose logs for the affected page can be found in this gist
Mongo Server configuration
Mongo Atlas M10
2GB Ram
10 GB Storage
100 IOPS
Encrypted
Auto-expand storage
Attempted solutions:
I have checked my DB metrics, they looked fine. There are also no slow queries. These are simple findAll queries. Performance advisor on mongo atlas reports nothing unusual.
The production application and database are both hosted in the same region.
I have already tried optimising the application layer of the query (mongoose) by running .lean()
Question:
Where else should i look to improve the database latency? How can a simple query take so long? Otherwise, why is my server response time taking up to 4s when the expected is about 200ms?
Hey you can try hosting your server and database in the same region. I think the network is creating a overhead in this case. If the server and the database are in the same region, They are on the same network which will reduce the latency significantly. there is a diagram on aws for this
I add some problem like yours with an app that i developed in my master degree. I add to put a node.js api running online to present it in class room.And i realized that every time i wanted to make a call in the api the response was taking allot of time. I realized that one of the problems was the school network because of the firewalls. Also the place where i put the server heroku.com was giving some delay as well. What i did was use Redis ( https://redis.io/ ) to improve the performance, also heroku was giving me some problems because of the requests being http and not https.
Make a test running the app and data on your localhost and see the performance. if you don´t have any issue try to check if nothing is messing with your request like the place where you host your node server.
Let me know if this helps or if you still have issues so i can try to help you out better.
I had the same issue once with my nodejs code using the same development stack(mongodb,nodejs), I got into trouble of late response from api, and after spending a lot of time I found my server the real culprit I then changed from heroku to amazon aws EC2 instance and things started working fast and amazingly fast, so probably
your web server is culprit
to make sure mongodb is not culprit, write an api endpoint where you can just return some json response without making any query to database.
I'm new to GraphQL, Apollo, AWS S3, and Redux. I've read the tutorials for each and I'm familiar with React Native, Node, Heroku, and Mongo. I'm having trouble understanding the following:
how a "GraphQL Server" is hosted for a mobile device using React Native?
can I create the GraphQL server with Node and host it on AWS S3?
how to grab that data by using Apollo/GraphQL in my React Native code and store that data locally using Apollo/Redux?
do I have to use Graphcool as the endpoint from the start instead? All I'm trying to do is pulling data from my database when the app loads (not looking to stream it, so that I am able to use the data offline).
Where should I look to get a better understanding?
I have a couple comments for you in your exploration of new territory.
GraphQL is simply the query language the talks to your database. So you are free to run any type of api (on a server, serverless, etc.) that will use graphql to take in a graphql query/mutation and interact with your database.
GraphCool is a "production-ready backend" basically back-end as a service. So you wouldn't worry about running a server (as I believe they run most everything on serverless infrastructure) or managing where your DB is housed.
You can run an HTTP server on AWS EC2 or serverless using AWS Lambda. (Or the same flavor with Google or Azure). Whatever you decide to use to accept requests, your endpoint will accept graphql query strings and then do stuff with the db. AWS S3 is more of static storage. You can store files there to be retrieved, or scripts that can be pulled, but S3 probably isn't where you would want any server-like code to run.
Apollo would be a tool to use on your frontend for easily interacting with your graphql server. React-Apollo
Apollo/Redux may help you then manage the state throughout the app. You'll simply be loading the data into the app state on load then interacting with that state without needing to make any more external calls it sounds like.
Hopefully this was helpful.
I am developing a Twitter app that (on the backend) consumes Tweets does some fairly intense processing and then stores the data in a database for use later by the client. All of my servers are running node.js.
I am going to have a server connected to the Twitter Streaming API using nTwitter for node.js. I want to then have this server pass the Tweets along to worker servers and distribute the load based on the Tweet ID (the last digit in the ID would be used).
Right now, I am using Socket.io (and Socket.io-client) which seems to run pretty well. It seems like the Websocket protocol is ideal for this. I am wondering if there are there any reasons not to use Socket.io in this manner?
I just finished a REST server application in nodejs using express, passport and mongodb
I want to create a meteor app for the LIVE Statistics of that mongodb REST api's , like number of connections, user online etc.
But I can't figure out a way to use custom mongodb with meteor. Basically I want to share the same database with nodejs/express application and meteor.
Is it possible? I am sure it should be, but I can't figure out HOW!
Help please!
Okay, I got here what to do.
While deploying meteor, they have an option to choose the database.
So i created an app on meteor and then deployed it for nodejs
here is how,
meteor bundle myapp.tgz
extracted myapp.tgz and used the command below.
PORT=3000 MONGO_URL=mongodb://localhost:27017/myapp node bundle/main.js
Here is complete doc : http://docs.meteor.com/#deploying