I have a node.js RESTful API application. There is no web interface (at least as of now) and it is just used as an API endpoint which is called by other services.
I want to host it on Amazon's AWS cloud. I am confused between two options
Use normal EC2 hosting and just provide the hosting url as the API endpoint
OR
Use Amazon's API Gateway and run my code on AWS Lambda
Or can I just run my code on EC2 and use API Gateway?
I am confused on how EC2 and API Gateway are different when it comes to a node.js RESTful api application
Think of API Gateway as an API management service. It doesn't host your application code, it does provide a centralized interface for all your APIs and allows you to configure things like access restrictions, response caching, rate limiting, and version management for your APIs.
When you use API Gateway you still have to host your API's back-end application code somewhere like Lambda or EC2. You should compare Lambda and EC2 to determine which best suits your needs. EC2 provides a virtual Linux or Windows server that you can install anything on, but you pay for every second that the server is running. With EC2 you also have to think about scaling your application across multiple servers and load balancing the requests. AWS Lambda hosts your functions and executes them on demand, scales out the number of function containers automatically, and you only pay for the number of executions (and it includes a large number of free executions every month). Lambda is going to cost much less unless you have a very large number of API requests every month.
Related
I created a Node server that receives events through webhooks, handles them, and posts their data to one API endpoint. Currently I'm deploying it using AWS Elastic Beanstalk, but I don't know if it's the best option.
I don't need load balancers.
I don't need web servers like Apache/Nginx.
My Node server does not have any ports to receive requests, since it's a simple server that only handles webhooks events. So the EBS service will always be without metrics for requests (severe health status - because doesn't handle any of the health requests).
Should I use another type of AWS service? Docker?
Finally, I went for it with the App Runner AWS service for running containers. No load balancers, just elastic sizing. No web servers.
I have deployed nodejs app on ec2 instance. I have used atlas for mongoDB, load balancer, Aws ssl to secure the connection and pm2. As iam already using aws load balancer should i need to use nginx. What is the best practice of deploying nodejs app on ec2. if i need to use ngnix where to configure http to https redirect?
Looks like you need a web application to be developed. I would suggest below architecture. enter image description here. This will eliminate pain of hosting/maintaining servers.
All AWS components can be programmatically integrated using AWS SDK and Amplify is a wrapper on top of this.
Develop your UI in any JavaScript framework and host it in AWS S3. Using CloudFront you can cache the UI layer. Put an API gateway layer which will intercept all the traffic from your UI. Using AWS Amplify can very much simplify your UI development with a lot of built-in wrapper components. It comes with CLI which can be used for deployment as well.
Host your Spring Boot+DB Driver (or NodeJS/C#/PHP/Python/etc.) Application with Rest API in AWS Beanstalk. BeanStalk can be configured with Load Balancing, Auto Scaling Group, etc. If AWS Beanstalk seems complicated for you, consider using AWS Lambda (Serverless, microservices) architecture.
AWS DocumentDB is a MongoDB extension and maybe you can leverage it. Which is again a highly scalable, very flexible NoSQL in this case.
You can leverage AWS Cognito (https://www.slideshare.net/awsugkochi/acdkochi19-enterprise-grade-security-for-web-and-mobile-applications-on-aws)to store User credentials in groups and add permissions and authenticate/authorise the users.
In API Gateway you can configure AWS Cognito Authoriser and protect the APIs from un-authorised calls.
Some of the auxiliary services can be used to integrate email/SMS etc. AWS SNS (Pub/Sub) + SQS (Queue) -> If you want to decouple any process, you can use SNS + SQS. You can send e-mails using AWS SES. AWS Route53 is the DNS and your domain can be hosted here.
If you have to upload any files to cloud and store it for users, leverage AWS S3.
You need to protect internet-facing components like API Gateway and Cloudfront using AWS WAF.
All these systems generate logs and it can be accessed from AWS cloudwatch. Your APIs can be monitored for performance and errors using AWS X-Ray.
I have some doubts about which is the most appropiate way to allow access to my company backend services from public Clouds like AWS or Azure, and viceversa. In our case, we need an AWS app to invoke some HTTP Rest Services exposed in our backend.
I came out with at least two options:
The first one is to setup an AWS Virtual Private Cloud between the app and our backend and route all traffic through it.
The second option is to expose the HTTP service through a reverse proxy and setup IP filtering in the proxy to allow only income connections from AWS. We don´t want the HTTP Service to be public accesible from the Internet and I think this is satisfied whether we choose one option or another. Also we will likely need to integrate more services (TCP/UDP) between AWS and our backend, like FTP transfers, monitoring, etc.
My main goal is to setup a standard way to accomplish this integration, so we don't need to use different configurations depending on the kind of service or application.
I think this is a very common need in hybrid cloud scenarios so I would just like to embrace the best practices.
I would very much appreciate it any kind of advice from you.
Your option #2 seems good. Since you have a AWS VPC, you can get an IP to whitelist by your reverse proxy.
There is another approach. That is, expose your backends as APIs which are secured with Oauth tokens. You need some sort of an API Management solution for this. Then your Node.js app can invoke those APIs with the token.
WSO2 API Cloud allows your to create these APIs in the cloud and run the api gateway in your datacenter. Then the Node.js api calls will hit the on-prem gateway and it will validate the token and let the request go to the backend. You will not need to expose the backend service to the internet. See this blog post.
https://wso2.com/blogs/cloud/going-hybrid-on-premises-api-gateways/
Our system has 3 main components:
A set of microservices running in AWS that together comprise a webapp.
A very large monolithic application that is hosted within our network, and comprises of several other webapps, and exposes a public API that is consumed by the AWS instances.
A locally hosted (and very large) database.
This all works well in production.
We also have a testing version of the monolith that is inaccessible externally.
I would like to able to spin up any number of copies of the AWS environment for testing or demo purposes that can access the demo testing version of the monolith. However, because it's a test system, it needs to remain inaccessbile to the public. I know how to achieve this with AWS easily enough (security groups etc.), but how can I secure the monolith so it can be accessed ONLY by any number of dynamically created instances running in AWS (given that the IP addresses are dynamic and can therefore not be whitelisted)?
The only idea I have right now is to use an access token, but I'm not sure how secure that is.
Edit - My microservices are each running on an EC2 instance.
Assuming you are running your microservices on EC2, if you want API calls from your application servers running in AWS to come from a known IP/IPs then this can be accomplished by using a NAT instance or a proxy. This way even though your application servers are dynamic, the apparent source of the requests is not.
For a NAT you would run your EC2 instances in a private subnet and configure them to send all of their Internet traffic out over the NAT instance which will have a constant IP. Using a proxy server or fleet of proxy servers can be accomplished in much the same way, but would require your microservice applications be configured to use it.
The better approach would be to simply not send the traffic to your microservices over the public Internet.
This can be accomplished by establishing a VPN from your company network to your VPC. Alternatively, you could establish a Direct Connect to bridge the networks.
Side note, if your microservices are actually running in AWS Lambda then this answer does not apply.
I have a Rails app which uses PhantomJS in order to parse pages. I have moved the PhantomJS parser into a NodeJS server. Since PhantomJS is expensive to process, I just have one PhantomJS instance per CPU (using the cluster within a server).
Now, I want to be able to have more machines with NodeJS, and from Rails be able to just say: hey, process this URL. Right now is working with just one machine. But I don't know how would you structure this multi-server NodeJS servers.
So, right now whenever I want to parse a site, I send a request to my NodeJS machine, and once that has been parsed, NodeJS posts backs to Rails. But, how do you scale parallely with multiple servers in NodeJS, in a way that it is clever enough to when a URL is received it is sent to the server that has less jobs processing?
Basically, you'll need to build a proxy server on top of your application servers to handle the routing of traffic.
Not sure which hosting provider you are using, but AWS makes this really easy with Opsworks or Elasticbeanstalk.
Some options include:
AWS OpsWorks
AWS Elastic Beanstalk
Custom HA Proxy Load Balancer
AWS Opsworks Using an AWS elastic load balancer (ELB), you can spin up a new NodeJS stack in the AWS Opsworks service and attach an elastic load balancer to these instances. From there you can designate an autoscaling instance to grow and handle more traffic as needed. AWS ELBs round robin traffic to instances depending on a number of configurable metrics (CPU load, time, network traffic).
AWS Elastic Beanstalk: See this tutorial on how to build a nodejs app using cluster in an autoscaling environment.
HA Proxy: If you're looking to run your own load balancer, you could look into spinning up your own load balancer to handle the traffic.