The Problem
I am student assigned to a project to create a rudimentary social media app. We are planning to use Flutter to build the app and we are going to use MongoDB and Express for the database and API respectively. The goal is to be able to use continuous integration for our project through Fastlane and GitLab.
Initially, I thought to put the API and Flutter in separate Docker containers and to host the database on my desktop, but I realize that might not be the best solution.
The Question(s)
How should we setup the database and the server that we all have access to the same data in a database? Basically, how should we best set up our project environment to work as team, in terms of:
hosting the database?
setting up Express and Flutter for continuous integration?
If you are using MongoDB just set up a cluster on Atlas it's free as long as it's a relatively small application (up to 500MB). After you sign up, you will create a cluster, and then Atlas will give you instructions on how to connect to that cluster using node.js.
Basically all you do is throw in the link to your cluster with your configured password in your database connection link. This is all in the cloud so you can access it from anywhere after you whitelist the IP's that will be accessing it remotely. (alternatively you can whitelist all IP's which is the easier way of doing things it's just A LOT less secure.) but it's an okay option for a school project.
You can then use Heroku to host your app which allows for a custom server setup like you will have with Express.
You will need to use dotenv for heroku as well as securing your database link and password, so read up on that as well.
Related
I'm currently building a website that I would like to privately host so it can only be used internally. My goal is to store file uploads into a google cloud bucket then document certain things into a cloud sql db for filtering later on.
I've been able to store and pull my files from a bucket and I was also able to set up my database in cloud sql however I'm not totally sure how to send SQL query's to my database from my website. I've tried going through the documentation but I'm a bit lost as to what I need. So if anyone knows what I should be looking at that would be a lot of help. Anything like a guide on the google documentation, because I'm not sure which I need to follow or anything.
Additional information
Currently my website is hosted locally because I haven't decided on a
website hosting service yet any recommendations would be helpful.
I was using a postgres database as practice for creating endpoints and developing my database
structure.
I am using a PERN stack so postgres, express, react and node.js
If there's anything else you would like specified I'm more then happy to provide any additional information.
Update:
I was able to connect my postgres db to cloudsql. I would explain how but this video does a great job of it.
How to connect psql db to cloud sql
However I'm running into an issue when I try to connect using an SSL connection. I have the files my only issue is I don't know where to store these files.
The general recommendation is to use the Cloud SQL Auth Proxy to connect to your Cloud SQL instance.
Basically you'd run the proxy wherever you're running your webapp and then have your webapp connect to the proxy as if it were a local database.
You'll start the proxy like this:
./cloudsql_auth_proxy -instances=<INSTANCE_CONNCECTION_NAME>=tcp:5432
That will start a listener on 5432 on localhost that you can connect your webapp to.
Alternatively, you could use Authorized Networks to allow-list the IP address of the machine where your app runs.
Check out the docs here: https://cloud.google.com/sql/docs/postgres/connect-overview.
I have an application in mind that would be built using node, mongodb + other db, kubernetes , RabbitMQ, docker and react as a front end. The application will be built in a microservice architecture. We all know that for a monolith app all you need is one DB (MongoDB, MySQL etc etc) but for a micro one you can have multiple databases. My question would be, do I need to buy multiple, separate databases and connect each service to them ? or how does it work in a micro-services design.
At the moment a I have a sample microservices app that is running on my local machine using docker and its connected to multiple databases or database/service. I am just to trying to get an idea on how does this work with companies like DigitalOcean or AWS.
Any input on this would be great.
I am just trying to figure out how this going to work when it comes to production later so that I am ware of cost and deployments. I have done some research on Digital ocean, AWS etc etc but I still can figure out how do they work.
thanks in advance.
You don't need having multiple instances of DBMS running. You can easily use one VM with one MongoDB running on it.
When you scale you might want to have separate machines running DB instances for your services, but at start you may just separate it logically to ensure you do not communicate between services using DB.
Chris Richardson on his microservices.io website says:
There are a few different ways to keep a service’s
persistent data private. You do not need to
provision a database server for each service.
For example, if you are using a relational database
then the options are:
- Private-tables-per-service – each service owns a
set of tables that must only be accessed by that
service
- Schema-per-service – each service has a database
schema that’s private to that service
- Database-server-per-service – each service has
it’s own database server.
Source: https://microservices.io/patterns/data/database-per-service.html
I build a web based mobile app (using react), which I'm hosting as an Azure Web app. This website is connected to a node server, which I'm hosting using a azure virtual machine (I don't know if this is the best approach, maybe I should host this as a webapp instead of VM ?).
I'm still new to this strategie of developing a web based mobile app and connecting it to a hosted backend in the cloud so I apologize if this is a trivial question.
I'm using websockets to connect my frontend to the node server and then I consume messages. I don't have a database it is a simple application.
The steps I'm doing to run the app are:
I connect to the Azure VM from my laptop with ssh
run my server using node server.js
When the server is runing, I can refresh my website and everything works fine
Now, the problem is that I want the node server to run all the time. However, I noticed that the server stops running when I deconnect (ssh) my laptop from the virtual machine. So I find myself doing these steps (connecting to the VM with ssh and running the server manually) each time I want to use the app.
Is there a way to do this so that the node server runs all the time without stoping? Also since I'm new to this, is this the right way to deploy frontend and backend? I assume I can't deploy both frontend and backend in the same Azure webapp or am I wrong?
(Since part of your question is around "Is there a better way?", I would answer that instead of fixing the issue in your current VM hosting :)).
To take full advantage of cloud for your applications, PaaS is always preferred over IaaS. In this case, unless you have any specific reason, you should deploy your backend Node app in another App Service (aka Web App). Or you can consider Azure Function also if your node app has a small set of APIs, but it will require code update. Both support multiple platforms including node.js. Since you mentioned you are leveraging Web App for your react mobile app, so I hope you are already bit familiar with it. Also since Azure Function would require code change, so Web App is preferred this case.
Note: I omitted other solutions like AKS, Service Fabric etc. for now, since currently we are talking about the problem of deploying only a single app, for which those will be overkill at this moment.
Also, to your point
I assume I can't deploy both frontend and backend in the same Azure webapp or am I wrong?
Yes technically you can, depending on your scenario by "bundling" into a single app if both are in same platform (like Node in this case). Though whether one should do that would be an opinionated answer. But even if you keep those separate, you can still leverage single App Service Plan for cost saving. So keeping separate like you have now is what I would suggest to maintain "separation of concern".
I recently developed a web app locally, with a React frontend that interacts via proxy with a Node.js backend that interacts with MongoDB Atlas. Everything works locally, and I am ready to actually deploy the web app for public use.
How does hosting work with a full stack web application? Do I host the entire web app in the same place (e.g. S3 bucket), or should the backend and frontend be deployed separately? I have never done this before, so I appreciate any help I can get.
Yes, you can have two different servers for frontend and backend.
You can use theHeroku platform to deploy your backend app, and Mlab to provision a Mongo database. These platforms has free tiers where you can experiment and learn about deployments and clouds.
Once you are comfortable with these then you can move to Elastic Bean Stalk on AWS to provision servers and also database.
Now mlab is not available as it is already been acquired by mongoDB only, so I would recommend you to create the database on Atlas cluster which also offers a free tier.
Rather than using the Heroku, I would suggest you to use MongoDB stitch which is also the backend as a service. If you will use stitch then you can also seek for support from mongoDB people but in case you will use heroku then you will not receive any support from them.
You can refer to the documentation of stitch for more information https://docs.mongodb.com/stitch/. This has complete guidance how you can deploy your app using stich and can use mongodb database.
However if you need more help, please ping me anytime.
I have a Sproutcore application which uses CouchDB hosted by Cloudant. In devenv I'm using Sproutcore proxy to forward requests to Cloudant (due to single origin policy I obviously can't call a arbitrary hosts from my JavaScript).
What would be your suggestion on hosting the app. CouchApps loading script is failing with obscure error (please note Sproutcore app is around 40 MB of JavaScript and resources).
The thing about hosting an app that talks directly to couchdb is that you are restricted by the same origin policy, so the server that serves up your SproutCore app, has to be on the same domain as you access your couchDB database. If your host provides a static file server, you could use that, but your best bet is probably just to save it as a couch app.
The Key with deploying SC as a couch app, is that you have to set up the URL's that sc-build generates to point to the correct location. This can be done with the buildfile setting something like this:
:url_prefix => '<database>/_design/<designDocument>/'
This way the links point to the right location.
You can also setup CouchDB url rewrites to get simpler urls if you wish. Have a look here for more discussion on this:
https://groups.google.com/d/topic/sproutcore/-D-5T5yPg3A/discussion
PS: 40mb seems overly large! do you have many images? I'd be concerned if you had that much JS/CSS.
Cloudant hosts its clusters (mainly) on EC2 in the us-east-1b or us-west-1b availability zones. If you cannot deploy as a CouchApp you should try to colocate your application layer next to your DB cluster. That will minimize the latency between the web server and the databases.
You can do this yourself by getting an EC2 instance and running a web server on it, or you can use a third party host that hosts on AWS (e.g. Heroku).
In both cases, you will need to run a small proxy from your web server to the database. That's why simply hosting on S3 is not an option at the moment.
Once you deploy your app to a web server, don't forget to check that your web server and your database are in the same AWS availability zone. If they're not, just ask Cloudant to move your account.