I want to create a modular node.js application stack containing a set of applications. The idea is that app1, app2, etc can use the controllers and models.
Inside each app folder, I can have app specific package.json, app.js, etc.
I am using express.
I have two issues:
Is it possible to have that structure?
Why I'm not able to deploy such an app set on GCP? When I try It throws 500 internal server error.
enter image description here
To create a similar architecture, even if I didn't find a way to have the same, you should use services. According to the official GAE doc:
Use services in App Engine to factor your large apps into logical
components that can securely share App Engine features and communicate
with one another. Generally, your App Engine services behave like
microservices. Therefore, you can run your whole app in a single
service or you can design and deploy multiple services to run as a set
of microservices.
Does this work for your use case?
Regarding question 2, you didn't provide any information about your current process, so I cannot help you. Please edit the question adding the deployment configuration (app.yaml, etc.) and how is it performed. Please delete any sensitive information before posting it.
Related
I'm new to Node JS development. As an ASP.NET MVC developer, I normally use Repository Design Pattern where I have separate projects for Front-End and Database access in one solution. In addition, when creating a REST api, this can be added to the existing solution. So when publish, it api and front-end is separated by a different route.
I've just created a REST Service in Node JS and it's really simple and I like it. However, when it comes to Front-end I was looking at ReactJS, I've seen a blog (unfortunately, I can't find the link) where it separates the process between the REST service and react front end. I'm just wondering if this is a common design pattern in nodeJS using ReactJS. And if there's a benefit on doing this. Specially nowadays, Full Stack developers are a common thing. I can see the benefit of it from a maintenance stand point but I'm just wondering if there's a benefit in terms of server resources i.e. memory, cpu. Should the OS handle 1 vs 2 nodejs process? Will this differ from using linux vs windows?
I see a huge benefit of separating the frontend from the backend so I would propose you to have your Node backend running in its own project and let's say a React solution running on its own. The React client can then consume that API together with other APIs later. By separating you have the benefit of scaling later.
If you've already built the REST service in node, you can access it via proxy in the React project's package.json by adding
"proxy": "http://127.0.0.1:5001/"
It helps manage CORS issues.
I'm intending to host an online portfolio and I have several node.js projects I'd like to host.
So I'd like to host one project on domain.net/project1 and the other on domain.net/project2
Do I need to use a single nodejs process to do this, or is it possible to have each of my separate nodejs projects running independently?
Is it possible to assign a nodejs process to each subdomain?
A better approach will be to host your NodeJS projects independently and then use the domain(hosted and provided by platform) to Custom Sub-Domain(created by you) pointing to point each of your projects to custom subdomains like project1.mydomain.com. This will provide you the flexibility to deploy your node applications on Heroku, AWS or any other hosting environment.
Thus you get freedom of choosing your own hosting platform and can have as many projects hosted as needed. I have attached an architecture diagram for the same and the link to the original draw.io document. Also please note that this is just a sample, and your design may vary as per your specific requirements
You can refer this Arc diagram for more details.
You can use nginx or apache http for host each project or create a path that contains sub paths for each one.
Here come references:
Apache http
nginx hosting
I currently manage a cluster of VMs on a number of dedicated hosts to provide apache, nginx and node live and development servers. This of course requires constant and time consuming maintenance to ensure security and reliability. I've found more time is spent looking after this platform then coding new and exciting projects. So I've been looking into the Google App Engine to remove the need of managing any VMs but I'm struggling to work out how to get it to function for me!
Currently I find myself developing mostly in Angular (v4-5) for my frontend and nodejs for backend. My development nginx server powers my angular apps and routing to ng-serve and to a separate vm that runs my node apps. I use PM2 to manage the apps on both servers.
This works great! I can code locally push my changes via an rsync script to the servers, the app restarts and changes updated. More importantly, I can affectively code between the front and backend! When ready I can comfortably switch the code to the live servers with little effort - nice!
This is where I am struggling...
I can't seem to work how I would develop and publish versions of both the front and backend code in one App Engine project.
Is this possible? How would I go about deploying/publishing both aspects?
Would I be better having two projects such as example.com & api.example.com? If so, can I get the two projects to talk to one another when developing?
I have and can create a angular/nodejs app in the App Engine but I can't work the basics of front and backend development in this managed service.
I'd like to use the great features of the App Engine such as versioning, easy scaling and importantly deployment of apps and updates. Also, to move all my websites including some older ones in PHP to the App Engine.
Any help surrounding this would be much appreciated. Thanks!
As #Yandrak3 suggested, a microservices architecture is what you need. But keep in mind that that document relates to the App Engine Standard environment which does not support Node.js as a runtime environment. But keep the microservices architecture in mind when deploying to App Engine Flexible.
On backend and frontend
Frontend and backend are no longer used to describe the presentation layer and the data access layer of an App Engine application. The only reference in the documentation is here. The (VM) instances managing a service of your app which are configured with automatic scaling are considered part of the frontend infrastructure, while the ones configured with manual scaling are considered backend infrastructure.
The reason for this is that automatic scaling is one of App Engine's
great features [...] easy scaling,
automatically presenting your app's frontend in a manner scaled with the number of external requests incoming to your app.
Manual scaling is more suited for backend operations, where you might want to run operations dependant on the state of the memory over time, or other scenarios. You can find some more information on scaling types here. Keep in mind that this latter document is under App Engine Standard documentation and it includes basic scaling, a feature not available in the App Engine Flexible environment.
On services and versioning
In your case, your frontend and backend modules of your application will become two separate services in App Engine Flex. For each service you can deploy multiple versions. More, explained here.
Communication between services, in this case between your frontend and backend, can be done through HTTP requests between them.
If the next question is how HTTP requests from users reach the appropriate version of a service (or a service), check this document.
To deploy multiple services, you will use the same commands and you will separate each deployment and service through their afferent configuration file, app.yaml.
Your question requires a response with a pretty wide (and deep) spectrum of concepts. Hopefully, this answer is good to start with.
I am trying to plan out the best way to approach developing a node application, and am not sure what would provide the best performance. A little bit of info on the overall plan: the entire project will involve a web app as well as a 'bot' app. The bot app in question is node-steam, which is quite a substantial application on its own. My question is whether I should run two separate node processes for each app (one for web server and one for node-steam), or code them into one combined node process?
Also please note that I will need for the web app to be able to communicate with node-steam. I am planning on integrating socket.io into node-steam to make calls to it via web app actions. Is this the best approach if I keep the apps as separate node processes?
EDIT: When I refer to letting the web app communicating with node-steam, I meant that there are functions which need to be triggered in node-steam when a user does something in the web app (namely they perform specific actions in the browser), so I am planning on doing this via socket directly to the node-steam app, rather than to the web app and then routing the calls on to node-steam. As far as I can tell this is the simplest way of doing it.
Any guidance is appreciated.
Thank you
If the bot app is quite substantial, you should code it for scalability. You probably will be spawning worker threads or even scale across multiple nodes in no time, for the bot app alone.
I'm making an app that will have:
iOS and Android apps
A web-based "dashboard" to display data gathered from the mobile apps
The app requires that end-users create an account with us (we mostly likely will NOT use Facebook/Twitter logins).
Everything is/will be hosted on AWS using EC2/RDS/S3 (All encapsulated in Elastic Beanstalk)
| Web Browser | <----> | sails.js app | <-------> |actionhero.js API|
⬆︎
⬆︎
| Mobile app(s) | <-------------------------------------/
So far, I've built most of the backing API in actionhero.js, hosted on AWS.
It made sense to me to separate the API and the web app, because there web app is only for a small subset of users -- I'd expect 50x the traffic from our mobile apps over the web app.. We could scale the API to server the mobile users without unnecessarily scaling the sails.js app.
My questions are:
(biuggest unknown) How should I handle authentication? The sails.js app needs to be able to make requests to the API, and so do the mobile applications.
I was looking at the oauth2orize node module for creating our own Auth server, but it is designed for Connect/Express, so I don't think I could leverage it in the actionhero.js-based API.
If the solution is to create an OAuth server, am I supposed to host that on its own EC2 instance?
(AWS-specific question) I don't fully understand the use case for creating what AWS describes as a "worker tier" enviornment. Would there be a reason that the API would fall into that category?
If I want to run a data querying and aggregation task, I would create a separate node process for that, correct? If so, would that background worker have to exist on its own EC2 instance?
Sails.js and Actionhero.js both provide heavy support for socket.io. Should communication between the Sails app and my API happen over a persistent WebSocket connection? Will that scale if I need to create new instances in the future?
This seems like a fairly typical pattern; I'd like to hear if there are any big red flags in this design, before I paint myself into a corner. :-) THANKS!
Bonus question (specific to AWS Elastic Beanstalk)
Will I create separate "Applications" for the sails.js server and the API server? It looks like that's the only way to set it up, anyhow, but I want to make sure.
We have used node and beanstalk for a couple of applications now. For authentication, you can create an account for the user when they first access the app, and store the account id on the device. If you want them to be able to log in from multiple devices, you'll need to provide some kind of way of them identifying themselves, which is either id/password, or using Facebook. It's not that tough to set that up. Use session to allow them to log in and stay logged in. We generally just store the user id in the session.
A worker tier is for something you want to decouple from your app, something that you want to do that you don't need to know whether it succeeded/failed. A notification server is a prime example. You send the info for the notification into an SQS queue, that then gets sent to the worker tier, that does the work. We are just trying to figure this out now.
A big aggregation process, yes, I'd take it elsewhere, so it's not eating up your production server(s). You might want to create some data aggregation ongoing, as transactions are saved, so it accumulates. Big rollups after the fact can be time consuming and fragile.
Sounds like yes, they would be seperate applications.
A good tip. We use grunt to create the zip files for the app. It's a node batch tool. We check the latest info out of SVN, clean it up by doing things like removing .svn directories, apply our configuration into the config files by doing simple string replacement, then zip up resulting output. This then gets loaded into beanstalk. This takes all the guess work and time out of actually doing a new deployment. We can get a new build up in minutes that way.
Beanstalk can be very frustrating. When it fails, it's not very good at telling you why.