I want to practice creating my own RESTful API service to go along with a client-side application that I've created. My plan is to use Node and Express to create a server. On my local machine, I know how to set up a local server, but I would like to be able to host my application (client and server) online as part of my portfolio.
The data that my client application would send to the server would not be significant in size, so there wouldn't be a need for a database. It would be sufficient to just have my server save received data dynamically in an array, and I wouldn't care about having that data persist if the user exits the webpage.
Is it possible to use a service like Netlify in order to host both a client and server for my purposes? I'm picturing something similar to how I can start up a local dev server on my computer so that the front-end can interface with it. Except now I want everything hosted online for others to view. I plan to create the Express server in the same repo as the front-end code.
No, Netlify doesn't allow you to run a server or backend. However, they do allow you to run serverless functions in the cloud. These can run for up to 10 sec. at a time. Furthermore Netlify also have a BETA solution called "background functions" That can run for up to 15 minutes. But honestly for a RESTful API there sure would be better solutions out there?
If you are still looking for the Netlify for Backend you can consider Qovery. They explained here why it is a good fit for their users.
Related
I'm developing a chat-bot using api.ai for NLP and i'm stucked where i need to query some data from a database. And also i need to do some processing of those data. I'm not much clear how can I do this using the webhook. Any thoughts and tips would be really appreciated.
You might want to check out API.AI's getting started guide, the last part of which involves the basics of adding fulfillment to your agent which pulls from this getting started sample
We use Cloud Functions for Firebase in the sample but you can't call non-Google APIs or URLs without setting up billing. You can also use any other hosting providers (Google's AppEngine, Compute Engine, Amazon's Lambda, EC2, Heroku, Azure, etc.) for fulfilling API.AI agents, as long as you have a server that can respond to a HTTPS request, it'll work with API.AI's webhook fulfillment. To get started you can even use your local machine and tools like ngrok to tunnel the connection from a HTTPS URL to your local development machine: https://ngrok.com/. If your just starting out I'd recommend doing whatever your most comfortable with.
Am currently using Node-RED to process the data received through the webhook. The webhook receives the data in JSON and if you’ve setup your intents and entities correctly, you’ll get data that you can act on.
This can be achieved as a stand-alone nodeJS app that you host somewhere, like Heroku or OpenShift. But important point is that you need a server running a program somewhere to process the data that api.ai sends.
Am using firebase as my database and is integrated into my Node-RED setup and now I can do sophisticated queries like “how much did I walk last Wednesday”
This is more like a design question but I have no idea where to start.
Suppose I have a realtime Node.js app that runs on multiple servers. When a user logs in she doesn't know which server she will be assigned to. She will just login, do something and logout and that's it. A user won't be interacting with other users on a different server, nor will her details be stored on another server.
In the backend I assume the Node.js server will put the user's login details to some queue and then when there is space it will assign this user to an available server (A server that has the lowest ping value or is not full). Because there is a limit number of users on one physical server when the users try to login to a "full" server it will direct her to another available server.
I am using ws module of node.js. Is there any service available for this purpose or do I have to build my own? How difficult would that be?
I am not sure how websocket fits into this question. Ignoring it. I guess your actual question is about load balancing... Let me try paraphasing it.
Q: Does NodeJS has any load balancing feature that I can leverage?
Yes and it is called cluster in NodeJS. Instead of the traditional one node process listening on a single port, this module allows you to spawn a group of node processes and have them all binded to the same port.
This means is that all the user know is only the service's endpoint. He sends a request to it and 1 of the available server in the group will serve him whenever possible.
Alternatively using Nginx, the web server, as your load balancer is also a very popular approach to this problem.
References:
Cluster API: https://nodejs.org/api/cluster.html
Nginx as load balancer: http://nginx.org/en/docs/http/load_balancing.html
P.S
I guess the key word for googling solutions to your problem is load balancer.
Out of the 2 solutions I would recommend going the Nginx way as it is a much scalable approach
Example:
Your Node process could possibly be spread across multiple hosts (horizontal scaling). The former solution is more for vertical scaling, taking advantages of multi-cores machine.
I am creating a game by Unity and I want to upload the players' score to MongoDB. Therefore, I have built a node.js server listening to port 3000, and the scores will be sent to the server and store into the database.
My question is that if I want to create a website for viewing/analyzing players' scores, which approach should I use?
create two node.js servers, one for the web, one for the game
one node.js server but listen to port 80 and 3000 (im not sure whether it is possible or not)
any other better suggestions?
Thank you.
I would create one Node server, one to serve both api and web requests.
It sounds like the data served by the API and the web will be the same or subsets of each other. So you'll probably want to share code, lookup the same stuff from the database, etc etc.
From here, you could either create separate routes that the api uses and the web uses (/api/v1/my_scores vs /my_scores) OR realize that you're just asking for different representations of the same data and do something RESTful like checking the accept header and either sending server rendered HTML or sending JSON back to the client.
Alternatively, you could just create a api in Node, then use a purely front end tool like Angular or React to create a web front end for your site.
Using port 3000 is not a good idea because many users access internet through firewalls which block non-standard ports.
I would recommend using 443 port and https to secure the communication for both use cases.
If the site for analyzing scores does not share logic with the api server, then it can be created as a separate site - but in starting it is easier to manage a single application.
If i understand your question easily and according to my limited knowledge i think that you don't require more than one server with a database. The reason is that one web you only want to display the high score nor the end user can insert it anything on website. So the complexity is minimal already so don't bother to create another server. Just make data getting API separate for using in website.
I've gotten some great feedback from Stackoverflow and wanted to check on one more idea.
Currently I've got a webapp that runs nodejs on a PAAS (Heroku and trying out bluemix). The server is being configured to talk to a Couchdb (hosted on cloudant). There are two types of data saved to the db, first, user data (each user will have it's own database), and second, app data itself (metrics, user account info (auth/admin stuff).
After some great feedback from here, the idea is that after the user logs in, they will sync there local (browser) pouchdb instance with Cloudant (probably proxied through my server as was recommended here).
Now the question is, for the app/admin data, maybe I run a couchdb instance on my server so i'm not making repeated network calls for things like user logins, metrics data, etc. The data would not be very big, and is already separated from the user data calls. The point is to have a faster/local instance for authentication mainly, changes/updates get synced outside of user requests.
The backend is in express web framework and it looks like my options are pouchdb.... to sync to the Cloudant instance?
If I want local db access (backed a Couchdb instance), on a node/express server running on a PAAS, is that the recommended setup?
Thanks vm for any feedback,
Paul
Not sure if you found a solution, but this is what I would try.
Because heroku clears any temp data, you wouldn't be able to run a default express-pouch database, you will need to change pouch db from using file system to using LevelDOWN adapter.(Link to Pouchdb adapters: https://pouchdb.com/adapters.html)
Some of these adapters would include:
https://github.com/watson/mongodown
https://github.com/kesla/mysqldown
https://github.com/hmalphettes/redisdown
You can easily get heroku mondo, mysql, or redis addon, and connect that to you express-pouchdb backend.
This way you will be able to keep your data.
I've got an NGinx/Node/Express3/Socket.io/Redis/Backbone/Backbone.Marionette app that proxies requests to a PHP/MySQL REST API. I need to load test the entire stack as a whole.
My app takes advantage of static asset caching with NGinx, clustering with node/express and socket is multi-core enabled using Redis. All that's to say, I've gone through a lot of trouble to try and make sure it can stand up to the load.
I hit it with 50,000 users in 10 seconds using blitz.io and it didn't even blink... Which concerned me because I wanted to see it crash, or at least breath a little heavy; but 50k was the max you could throw at it with that tool, indicating to me that they expect you to not reasonably be able to, or need to, handle more than that... Which is when I realized it wasn't actually incurring the load I was expecting because the load is initiated after the page loads and the Backbone app starts up and kicks off the socket connection and requests the data from the correct REST API endpoint (from different server).
So, here's my question:
How can I load test the entire app as a whole? I need the load test to tax the server in the same way that the clients actually will, which means:
Request the single page Backbone app from my NGinx/Node/Express server
Kick off requests for the static assets from NGinx (simulating what the browser would do)
Kick off requests to the REST API (PHP/MySQL running on a different server)
Create the connection to the Socket.io service (running on NGinx/Node/Express, utilizing Redis to handle multi-core junk)
If the testing tool uses a browser-like environment to load the page up, parsing the JS and running it, everything will be copasetic (NGinx/Node/Express server will get hit and so will the PHP/MySQL server). Otherwise, the testing tool will need to simulate this by firing off at least a dozen different kinds of requests nearly simultaneously. Otherwise it's like stress testing a door by looking at it 10,000 times (that is to say, it's pointless).
I need to ensure my app can handle 1,000 users hitting it in under a minute all loading the same page.
You should learn to use Apache JMeter http://jmeter.apache.org/
You can perform stress tests with it,
see this tutorial https://www.youtube.com/watch?v=8NLeq-QxkSw
As you said, "I need the load test to tax the server in the same way that the clients actually will"
That means that the tests is agnostic to the technology you are using.
I highly recommend Jmeter, is widely used and you can integrate it with Jenkins and do a lot of cool stuff with it.