I currently use docker + travis CI to test/ deploy my app. This works great locally because I have data volumes for things like node_modules etc, and docker's layers provide caching speeding up builds.
However, when I push the code to travis it has to rebuild and install everything from scratch and it takes forever! Travis doesn't support caching docker layers atm. Is there some other way to speed up my builds, or another similar tool that allows docker layer caching?
You might want to investigate how i3wm has solved a similar problem.
The main developer has written on the design behind his Travis CI workflow. Quoting the relevant part:
The basic idea is to build a Docker container based on Debian testing
and then run all build/test commands inside that container. Our
Dockerfile installs compilers, formatters and other development tools
first, then installs all build dependencies for i3 based on the
debian/control file, so that we don’t need to duplicate build
dependencies for Travis and for Debian.
This solves the immediate issue nicely, but comes at a significant
cost: building a Docker container adds quite a bit of wall clock time
to a Travis run, and we want to give our contributors quick feedback.
The solution to long build times is caching: we can simply upload the
Docker container to the Docker Hub and make subsequent builds use the
cached version.
We decided to cache the container for a month, or until inputs to the
build environment (currently the Dockerfile and debian/control)
change. Technically, this is implemented by a little shell script
called ha.sh (get it? hash!) which prints the SHA-256 hash of the
input files. This hash, appended to the current month, is what we use
as tag for the Docker container, e.g. 2016-03-3d453fe1.
See our .travis.yml for how to plug it all together.
Related
Here are some of my current thoughts, but would also like to understand how the community is doing it now
Use caching to reduce build time
Motivation
Currently, every time you change something in the libs, someone else needs to initialize it again when pulling it through git. If you know which package it's in, you can just run the initialize command for the specified package. If you don't know, you need to run the root initialize command, which is actually very slow because it re-runs all npm packages that contain the intialize command, regardless of whether they have changed or not.
reduce initialize time and improve collaborative development experience
support for ci/cd caching of built libs to speed up build time
Current state
lerna run
First initialization of new project 198.54s
Re-initialize for non-new projects 90.17s
Requirements
Execute commands as concurrently as possible according to the dependency graph, and implement caching based on git changes
Execute commands in modules that specify module dependencies
Execute commands in all submodules
Implementation ideas
lerna api is currently severely undocumented: https://github.com/lerna/lerna/issues/2013
scan lerna.json for all modules
run the specified commands in parallel as much as possible according to the dependencies
check git when running the command to see if it needs to be skipped
Schematic: https://photos.app.goo.gl/aMyaewgJxkQMhe1q6
The ultra-runner already exists, so maybe it's better to support this feature based on it
There seems to be no support for running commands other than build at the moment, see: https://github.com/folke/ultra-runner/issues/224
Known solutions
Trying to use Microsoft rush -- hoping for a better solution without replacing the main framework lerna+yarn
I'm working in a private enterprise and want to integrate Cypress E2E tests with our gitlab CI / CD pipeline.
Since cypress needs to install a binary zip file, I can't simply run npm i and hope things will work right.
I'm facing 3 options and not sure which one is the ideal way:
Somehow include the cypress binaries during the CI, cache them, and deploy the app image to both test & dev environments. One will serve for testing and one for the actuall app deployment.
(if possible, it is the fastest way since it doesn't require additional image building, and I would
love to see a gitlab-ci.yml example :) )
Use 2 different images -
nodejs-8-rhel7 image - which will be used for app deployments (and is being used currently)
cypress-included image - which will include cypress as well as the app code (and will be used for testing).
In this case I would have to build the app image twice, one with cypress and one without, looks like a slow process.
Use the cypress-included image for both app & testing deployments.
Since our current application base image is nodejs-8-rhel7, I'm afraid changing the base image will cause some trouble (nodejs version differences etc..)
Which of the above is the best option? do you have a better solution?
Thanks!
When shipping a dockerized node.js to production,
is it correct to ship an image that contains development dependencies?
I am not talking about the development dependencies
Not the devDependencies listed in packages.json, I mean gcc, python, node-gyp, some other *-dev packages, containing a bunch of headers, static libraries.
All of them are needed to compile some node dependencies (like node-sass)
An idea could be a two stage build, one image with all *-dev dependencies, build stuff in there, and export the results to another new image with just the binaries.
Pros: The final "production" image is small
Cons: Not standard way to build images
In general, any compiled sofware I want to distribute in docker images, should not contain the compilers, headers, and tools used to build the binaries.
If you want something to not be included in your final image, you have to do all the related commands in only one layer (one RUN statement).
Something like the following (pseudo code):
RUN install dev-dependencies && build your-project && uninstall dev-dependencies
Only one layer is created for the RUN statement and it won't contain dev dependencies.
The image would not be smaller if you remove the dependencies because the older layers contain them.
Try the new (experimental) --squash option with docker-build, using Docker 1.13.
The answer to OP question depends of how much images OP/his company maintain for production needs.
There few strategies possible:
If the quantity of maintained images is medium to low and system architecture is not very complex and do not uses dozens of images at once there the simplest and easiest solution to maintain is the best one. You can approach it with 1 single build. Or 2 step build if you want to use compiled source as base for containers that could bear different content (In this case 2nd stage could be even done during docker-compose up (system start-up)).
You can remove dev-only dependencies (as other answers suggested) if its necessary to keep the image slim/ there a lot of running containers that use same image / the size of compiled files is huge. This will increase time span of the build process but will result in smaller image.
3rd approach is totally different - if there is compiling process, use CI pipeline that independently compiles the assets within separate container (CI runner process) and provides versioned artifact - which you can use in your production builds (even store it somewhere, on S3/CDN/private, accesible to deployment storage), and then just fetch it from there, or just use files hosted there (in case of CDN).
I am building phoenix application with exrm.
Good practice suggests, that I should make tests against the same binary, I'll be pushing to production.
Exrm gives me the ability to deploy phoenix on machines, that don't have Erlang or Elixir installed, which makes pulling docker images faster.
Is there a way to start mix test against binary built by exrm?
It should be noted that releases aren't a binary file. Sure they are packaged into a tarball, but that is just to ease deployment, what it contains is effectively the binary .beam files generated with MIX_ENV=prod mix compile, plus ERTS (if you are bundling it), Erlang/Elixir .beam files, and the boot scripts/config files for starting the application, etc.
So in short your code will behave identically in a release as it would when running with MIX_ENV=prod (assuming you ran MIX_ENV=prod mix release). The only practical difference is whether or not you've correctly configured your application for being packaged in a release, and testing this boils down to doing a test deployment to /tmp/<app> and booting it to make sure you didn't forget to add dependencies to applications in mix.exs.
The other element you'd need to test is if you are doing hot upgrades/downgrades with your application, in which case you need to do test deploys locally to make sure the upgrade/downgrade is applied as expected, since exrm generates default .appup files for you, which may not always do the correct thing, or everything you need them to do, in which case you need to edit them as appropriate. I do this by deploying to /tmp/<app> starting up the old version, then deploying the upgrade tarball to /tmp/<app>/releases/<new version>/<app>.tar.gz, and running /tmp/<app>/bin/<app> upgrade <version> and testing that the application was upgraded as expected, then running the downgrade command for the previous version to see if it rolls back properly. The nature of the testing varies depending on the code changes you've made, but that's the gist of it.
Hopefully that helps answer your question!
My question is: How do you guys deploy the same code from whatever [D]VCS you use on multiple machines? Do you have an automated deployment system and if so what's that? Is it built in-house? Are there out there any tools that can do this automatically? I am asking because I am pretty bored updating up to 20 machines every time I make some modifications.
P.S.: Probably this belongs on ServerFault, but I am asking here because I am thinking at writing my own custom-made deployment system.
Roll your own rpm/deb/whatever for your package, set up your own repo, and have your machines pull on a regular basis. Its really not that hard to do and its already built-in to your system, is well tested, and loaded with features. You could use something like Func if you needed to push instead.
Depending on your situation deploying straight from the versioning system might not always be the best idea. You can only so much by just updating files, and mixing deployment and development probably will make the development use of the versioning system less free.
I see two alternatives that might be interesting.
Deploy from your continuous integration server. (add a task that runs after every successful build, copies over files and executes some remote commands, I'm using this to deploy to a testserver and would find it to tricky to upgrade production in such a way)
Deploy using an existing package manager. You can set up your own apt (or equivalent) repository and package the updates using apt. Have your continuous build system build apt packages but let an admin decide if the should be pushed to the update server. I think this is the only safe solution for production machines.
We use Capistrano for deployment & Puppet for maintaining the servers and avoiding the inevitable 'configuration drift' when many developers/engineers tinker with the package lists and configuration files.
Both of these programs are written in Ruby, but we use them for our PHP codebase stored in a git repository.
I use a combination of deb packages with puppet to deploy code and configure a bunch of machines.
In most projects i have been involved with the final stage has always been an scripted rsync deployment to live. so the multiple targets are built into this process.