I am fiddling with heroku to deploy my nodejs based application. The problem is I want to run some pre deployment scripts which contain grunt tasks. These tasks basically minify css and js files and perform some more operations like updating the version of the application in files etc.
The only way that I can think of now is to commit the minified files in the git repo itself and deploy.
This is not a good idea (for me at least) as it unnecessarily maintains revision history of minified files which is a total waste.
Please help.
I've seen people recommending creating a throwaway git branch for deployments. In the second of the above comments: How to deploy node app that uses grunt to heroku.
Personally I prefer to execute a grunt task(s) that packages my app and copies it to a directory outside my source tree/repo. Usually the target directory contains a throwaway git repository that I add to just to be able to do the push to heroku.
My workflow is to do any file processing I need to and then to copy the results to the output directory, skipping unnecessary files, and adding environment specific configurations and artifacts. A series of grunt tasks does this and the final one adds everything to the local reopo, sets the heroku remote (I have several apps on heroku to cover production, staging, and a sandbox usually used to look at old versions of things), and finally doing the push.
Related
I have a DigitalOcean VPS with ubuntu and a few laravel projects, for my projects initial setup I do a git clone to create a folder with my application files from my online repository.
I do all development work in my local machine, where I have two branches (master and develop), what I do is merge develop into my local master, then I push from master into my local repository.
Nw back into my production server, when I want to add all the changes added into production I do a git pull from origin, so far this has resulted into git telling me to stash my changes, why is this?
What would be the best approach to pull changes into production server? take in mind that my production server has no working directory perse, all I do in my VPS is either clone or push upgrades into production.
You can take a look at the CI/CD (continuous integration / continuous delivery) systems. GitLab for example offer free-to-use plan for small teams.
You can create a pipeline with a manual deploy step (you have to press a button after the code is merged to the master branch) and use whatever tool you like to deploy your code (scp, rsync, ftp, sftp etc.).
And the biggest benefit is that you can have multiple intermediate steps (even for the working branches) where you can run unit tests which would prevent you to upload failing builds (whenever you merge non-working code)
For the first problem, do a git status on production to see which files that git sees as changed or added and consider adding them to your .gitignore file (which itself should be a part of your repo). Laravel generally has good defaults for these, but you might have added things or deviated from them in the process of upgrading Laravel.
For the deployment, the best practice is to have something that is consistent, reproducible, loggable, and revertable. For this, I would recommend choosing a deployment utility. These usually do pretty much the same thing:
You define deployment parameters in code, which you can commit as a part of your repo (not passwords, of course, but things like the server name, deploy path, and deploy tasks).
You initiate a deploy directly from your local computer.
The script/utility SSH's into your target server and pulls the latest code from the remote git repo (authorized via SSH key forwarded into the server) into a 'release' folder.
The script does any additional tasks you define (composer install, npm run prod, systemctl restart php-fpm, soft-linking shared files like .env, and etc.)
The script soft-links the document root to your new 'release' folder, which results in an essentially zero-downtime deployment. If any of the previous steps fail, or you find a bug in the latest release, you just soft-link to the previous release folder and your site still works.
Here are some solutions you can check out that all do this sort of thing:
Laravel Envoyer: A 1st-party (paid) service that allows you to deploy via a web-based GUI.
Laravel Envoy: A 1st-party (free) package that allows you to connect to your prod server and script deployment tasks. It's very bare-bones in that you have to write all of the commands yourself, but some may prefer that.
Capistrano: This is (free) a tried-and-tested popular ruby-based deployment utility.
Deployer: The (free) PHP equivalent of Capistrano. Easier to use, has a lot of built-in tasks (including a Laravel one), and doesn't require ruby.
Using these utilities is not necessarily exclusive of doing CI/CD if you want to go that route. You can use these tools to define the CD step in your pipeline while still doing other steps beforehand.
I’m trying to set up GitLab CI/CD for an old client-side project that makes use of Grunt (https://github.com/yeoman/generator-angular).
Up to now the deployment worked like this:
run ’$ grunt build’ locally which built the project and created files in a ‘dist’ folder in the root of the project
commit changes
changes pulled onto production server
After creating the .gitlab-ci.yml and making a commit, the GitLab CI/CD job passes but the files in the ‘dist’ folder in the repository are not updated. If I define an artifact, I will get the changed files in the download. However I would prefer the files in ‘dist’ folder in the to be updated so we can carry on with the same workflow which suits us. Is this achievable?
I don't think commiting into your repo inside a pipeline is a good idea. Version control wouldn't be as clear, some people have automatic pipeline trigger when their repo is pushed, that'd trigger a loop of pipelines.
Instead, you might reorganize your environment to use Docker, there are numerous reasons for using Docker in a professional and development environments. To name just a few: that'd enable you to save the freshly built project into a registry and reuse it whenever needed right with the version you require and with the desired /dist inside. So that you can easily run it in multiple places, scale it, manage it etc.
If you changed to Docker you wouldn't actually have to do a thing in order to have the dist persistent, just push the image to the registry after the build is done.
But to actually answer your question:
There is a feature request hanging for a very long time for the same problem you asked about: here. Currently there is no safe and professional way to do it as GitLab members state. Although you can push back changes as one of the GitLab members suggested (Kamil Trzciński):
git push http://gitlab.com/group/project.git HEAD:my-branch
Just put it in your script section inside gitlab-ci file.
There are more hack'y methods presented there, but be sure to acknowledge risks that come with them (pipelines are more error prone and if configured in a wrong way, they might for example publish some confidential information and trigger an infinite pipelines loop to name a few).
I hope you found this useful.
I have a lengthy discussion with a colleague about how we should deploy our angular 4 app to production server.
Would like practical advise and guide on this issue from the community, if possible.
Premise 1
At production server,
git pull
npm install
{set up production configuration}
ng build --prod --aot
build and compile on production server
production server hardware specs need to support the build process
addition space required on hosting server to house node_modules
git repo master branch does not have compiled codes, therefore is a "clean source repo"
Premise 2
At production server,
git pull
build and compile production codes on local development workstation will be faster
git repo master branch will keep snapshot of the compiled codes for deployment
production server remains as a 128MB RAM with limited space, since it is to serve html, js and css.
faster deployment to another server when required for recovery or scaling, since it is only a git pull
The best way to do it, is to build and compile in the local development Workstation, and deploy only the output of the build.
The git repo master branch needs to contain the source code not the only compiled&built one.
You can deploy in the production server using other method rather then the git pull command but if you insist on using it, you can init a new repo in the /build file and pull this repo to the production server.
If you really can't afford the build farm yet and if it enables the testing or other activites, yeah sure. Test the workflow you describe and see it for yourself, but it's definitely not a good long term practice.
If eventually you're gonna use the CI in your workflow, I would suggest to just start trying to set it up instead of wasting time/money setting up something temporary. Moreover, 5 minutes isn't a big deal trust me.
As a side note: If you would have spent time trying your suggestion instead of writing your SO question and talking with your colleague, you probably would already have figured out the answer yourself.
Context
Web application project has a /build (or /dist) folder with front-end files, generated during build (by Gulp). This folder is not under the source control (see, for example: React.js Starter Kit)
The server-side code doesn't require bundling or compilation step, so the /src folder from your project can be deployed as it is (these source files are used to run Node.js or ASP.NET vNext server)
Web application is deployed via Git (see Git-based deployment options in Heroku or Windows Azure as an example)
Questions
Is it better to build (bundle and minify) front-end files before or after deployment?
If before, you may end-up having a separate repository (or branch), with the /build folder under the source control alongside with the rest of the project files. This repo is used solely for deployment purposes.
If after, the deployment time may increase - time needed to download additional npm modules used in the build process, the server's CPU may spike up to 100% during the build, potentially harming your web application's responsiveness.
Is it better to build front-end files on the remote server before or after running KuduSync command?
If you deploy your web application to Windows Azure with Kudu, should the deployment script copy only the contents of the /build folder (with public, front-end files like .js, .html, .css) to /wwwroot? As opposed to copying all the project files (server-side source code and front-end bundles), which it does by default.
By default Azure's deployment script copies all the project files, from D:\home\site\repository folder to D:\home\site\wwwroot folder, and then Node.js app is started from there. Is it a necessary step? Why not to start the Node.js (or ASP.NET vNext) app from the D:\home\site\repository folder? And if it indeed should be copied to a separate folder, why source files are placed in wwwroot, maybe it's better to copy them to another folder, outside wwwroot?
I am not familiar with both Azure and Heroku so I can't give any ideas about those specific deployment options.
I am using (4 dedicated servers with 2 of them solely for serving static files), the option to build the bundled and minified javascript files (for front-end) and add all those files to the main repository has several advantages
You only need to run it once (either on your dev machine or on staging server, whatever way you want). This is particularly helpful when you have to run multiple static servers since you don't have to run the build command on each server. One might argue that they can use something like Glusterfs to synchronise files from one static server to all other servers and the build process only needs to be run once. However, it is a whole different story when it comes to this kind of setup
It makes your deployment process simple, just pull new code and restart the server(s) if necessary (assuming that you have some mechanism to increase the static file version so that all your clients will receive the latest version)
Avoid unnecessary dependencies on your production servers. This might sound weird for some people but I just don't want to install any extra libraries on my production servers unless they are absolutely necessary. With the build process run locally on my dev machine, my production servers only have what they need to run the production code and nothing else
However this approach also has some disadvantages:
When more than one developer in your team (accidentally) run the build process and commit the code, then you will have a crazy list of conflicts. However, it can be solved by simply running the build process again after you merge all the changes from other guys. This is more about the workflow
Your repository will be bigger. I personally don't think this is a big issue considering few extra MB of my bundled and minified files. If your front-end javascript is big enough for this to be an issue then it is another story
I have a nodejs non open source web application project that I want to deploy to a production server and/or a staging server. Is there a default way for this, or some tool that does it? I want to package all files needed and exclude the files not needed like the .git folder, the tests and other files like Gruntfile, package.json and so on.
I could of course manually package the files in a tar.gz file and send them to the correct server. But I was hoping to find a more complete and configurable tool that can do it for me.
Might be not exactly what you're asking for, but i like git for automated deployment.
You could have branches like staging and production, which are checked out on the remote server.
You can set up a git hook like post-receive to update those remotely, every time you merge changes into those branches.
Here's a tutorial: http://wekeroad.com/2011/09/17/deploying-a-site-with-git-hooks