From the documentation site, Pants mentions in the First Concepts that it supports the notion of a distributed cache to cache published artifacts. See https://pantsbuild.github.io/first_concepts.html.
I have been looking around in the documentation for steps to setup a distributed cache, but have been unsuccessful. Can someone point me in the right direction for instructions to follow?
First see if this helps: https://pantsbuild.github.io/setup_repo.html#outside-caches and report back.
First of all, you want to make your pants setup so that you are getting "good" values written to cache. Pants will not upload the results of incremental builds to the cache, so what we usually recommend is that you setup your CI environment to publish to the cache, and then set these pants.ini settings to keep zinc from performing incremental builds in CI.
[compile.zinc]
# We don't want to use incremental compile in CI. This should short-circuit
# some extra work Pants does to support incremental compile.
incremental: False
# Introduce some randomness when choosing the next chunk of work in order
# to share more work between different CI jobs.
size_estimator: random
There are two setups in common use: a REST cache and an NFS cache
It can be convenient to use NFS if all machines are able to access a common NFS server. The configuration for the CI workers is pretty simple:
[cache]
read_from: ["/mnt/nfs/shared-cache/pants-artifact-cache"]
read: True
write_to: ["/mnt/nfs/shared-cache/pants-artifact-cache"]
# For CI builds, turn on writing, but for developer workstations you may
# want to leave this off and mount the NFS dir read-only
write: True
# Turn off auto-purging of the buildcache, leave that to a cron job
max_entries_per_target: 0
Here's the cleanup job installed as a cronjob:
echo /usr/sbin/tmpwatch -c 14d /data2/shared-cache/pants-artifact-cache > /etc/cron.daily/pants-cache-clean
chmod +x /etc/cron.daily/pants-cache-clean
If you want to setup a cache that is shared using REST, you can configure a server to accept PUT and DELETE calls. Then configure the cache settings with URLs as follows:
[cache]
read_from: ["https://pantscache.example.com/pants-artifact-cache/ro"]
read: True
write_to: ["https://pantscache.example.com/pants-artifact-cache/rw"]
# For CI builds, turn on writing, but for developer workstations you may
# want to leave this off.
write: True
See this writeup for setting up a basic NGINX with Pants. Ask on the pants-devel# group or pantsbuild #general slack group if you want details on more sophisticated cache setups using Varnish or multiple NGINX servers with sharding.
Related
BitBucket Pipelines have the ability to add custom caches which are reused between pipeline runs. ParcelJS makes use of its own .parcel-cache file cache to speed up builds between runs.
I have a team running several CI branches at the same time and we're looking to move to ParcelJS.
As far as I can tell BitBucket creates the same cache regardless of the branch you're on - so I was wondering if it is possible and whether it makes sense to create a custom cache pointing to the .parcel-cache?
As far as I understand, .parcel-cache is only good for the development server
If you restart the dev server, Parcel will only rebuild files that have changed since the last time it ran.
https://parceljs.org/features/development/#caching
I reckon you are running build, lint and test steps in Bitbucket Pipelines so, unless I misunderstand parceljs' cache purpose, you should better not save it. Otherwise you would be wasting build time by uploading and downloading a bundle that would not speed-up your scripts in return.
You should test this in your workstation nonetheless. Like, with a clean clone of your repo, does the presence of this cache speed up the instructions that run in the pipeline? If it does, then you can face the issues of cache usefulness across different branches.
We have a Centos machine on which Jenkins is hosted.
This Jenkins have lots of jobs and configurations of interconnected jobs.
The problem is that if anytime we want to do a change in multiple jobs there is a risk of misconfiguration.
So there should be a revert process so that we can revert back to older working version of Jenkins.
Just like git does, if code is buggy I have the option to revert back to the healthy code.
Is there any standard solution available for these type of problems?
You can just copy the files on JENKINS_HOME;
A far better approach would be using thinBackup
Using thinBackup you can restore and make a backup.
Please note: I am not interested in any enterprise/for-pay (Tower?) solutions here, only solutions available via Ansible's OSS offering.
OK so I've got my Ansible project configured and working perfectly, woo hoo! Looks something like this:
myansible01.example.com:/opt/ansible/
site.yml
fizz.yml
buzz.yml
group_vars/
roles/
common/
tasks/
main.yml
handlers
main.yml
foos/
tasks/
main.yml
handlers/
main.yml
There's several things I need to accomplish to get this working in a production environment:
I need to be able to automate the deployment of changes to this project
I need to schedule playbooks to be ran, say, every 30 seconds (to ensure all managed nodes are always in compliance)
So my concerns:
How are changes usually deployed to live Ansible projects? Say the project is located at myansible01.example.com:/opt/ansible (my Ansible server). Is it sufficient to simply delete the Ansible project root (rm -rf /opt/ansible) and then copy the latest (containing changes) Ansible project back to the same location? What happens if Ansible is currently running any plays while I perform this "drop-n-swap"?
It looks like the commercial offering (Ansible Tower) has a scheduling feature built into it, but not the OSS offering. How can I schedule Ansible OSS to run plays at certain times? For instance, I might want certain plays to be ran every 30 seconds, so as to ensure nodes are always within compliance. Is cron sufficient to do this, or is there a more standard approach?
For this kind of task you typically want an orchestration engine such as Jenkins to do all your, well, orchestration.
You can set Jenkins to run playbooks on timers or other events such as a push to an SCM such as git.
Typically a job starts by checking out a tag/branch of our Ansible code base and then applying it to all of our specified servers so you always know what is being run. If you want, this can simply be the head on master (in git terms) so it's always applying the most recent changes. If you were also to have this to hook into your SCM repo then a simple push will force those changes to be applied to all of your servers.
Because of that immediacy you might want to consider only doing this on some test servers that then have some form of testing done against them (such as Serverspec) to verify that your changes are good before rolling them out to a production environment.
Jenkins, by default, will not run a job while the same job is running (or if you are maxed out on executor slots) so you can always be sure that it will only pull the repo (including any changes) after your Ansible run is complete. If you have multiple jobs running you can use blocking to prevent jobs running at the same time (both trying to apply potentially different configurations to the servers) but you don't have to worry about a new job starting and pulling the repo into the already running job as Jenkins separates these into separate work spaces.
We use Jenkins for manual runs of Ansible against our environment but we also have a "self healing" Jenkins job that simply runs a tagged commit of our Ansible code base against our environment, forcing it to an idempotent state to prevent natural drift of configurations. When we need to do something different to the environment or are running a slightly further ahead commit of our code base in to it we can easily disable the self healing job until we're happy with things and then either just re-enable the job to put things back or advance the tag that Jenkins is using to now use the more recent commit.
I'm looking for ways in which to deploy some web services into production in a consistent and timely manner.
I'm currently implementing a deployment pipeline that will end with a manual deployment action of a specific version of the software to a number of virtual machines provisioned by Ansible. The idea is to provision x number of instances using version A whilst already having y number of instances running version B. Then image and flick the traffic over. The same mechanism should allow me to scale new vms in a set using the image I already made.
I have considered the following options but was wondering if theres something I'm overlooking:
TGZ
The CI environment would build a tarball from a project that has passed unit tests and integration tests. Optionally depednencies would be bundled (removing the need to run npm install on the production machine and relying on network connectivity to public or private npm repository).
My main issue here is that any dependencies that depend on system libraries would be build on a different machine (albeit the same image). I don't like this.
NPM
The CI environment would publish to a private NPM repository and the Ansible deployment script would check out a specific version after provisioning. Again this suffers from a reliance on external services being available when you want to deploy. I dont like this.
Git
Any system dependent modules become globally installed as part of provisioning and all other dependencies are checked into the repository. This gives me the flexibility of being able to do differential deployments whereby just the deltas are pushed and the application daemon can be restarted automatically by the process manager almost instantly. Dependencies are then absolutely locked down.
This would mean that theres no need to spinning up new VM unless to scale. Deployments can be pushed straight to all active instances.
First and foremost, regardless of the deployment method, you need to make sure you don't drop requests while deploying new code. One simple approach is removing the node from a load balancer prior to switchover. Before doing so, you may also want to try and evaluate if there are pending requests, open connections, or anything else negatively impacted by premature termination. Or perhaps something like the up module.
Most people would not recommend source controlling your modules. It seems that a .tgz with your node_modules already filled in from an npm install while utilizing a bundledDependencies declaration in your package.json might cover all your concerns. With this approach, an npm install on your nodes will not download and install everything again. Though, it will rebuild node-gyp implementations which may cover your system library concern.
You can also make use of git tags to more easily keep track of versions with specific dependencies and payloads. Manually deploying the code may get tedious, you may want to consider automating the routine while iterating over x amount of known server entries in a database from an interface. docker.io may be of interest.
Our company uses ANT to automate build scripts.
Now somebody raised the question how to secure such build scripts agains (accidental or intended) threats?
Example 1: someone checks in a build script that deletes everything under Windows drive T:\ because that is where the Apache deployment directory is mounted for a particular development machine. Months later, someone else might run the build script and erase everything on T:\ which is a shared drive on this machine.
Example 2: an intruder modifies the default build target in a single project to scan the entire local hard disk. The Continuous Integration machine (e.g. Jenkins) is configured to execute the default build target and will therefore send its entire local directory structure to the intruder, even for projects that the intruder should not have access to.
Any suggestions how to prevent such scenarios (besides "development policies" or "do not mount shared drives")?
My only idea is to use chroot enviroments for builds?!
The issues you describe are the same for any code that you execute on the build machine - you could do the same thing using a unit test.
In this case the best solution may be to place your build scripts under source control and have a code review prior to check in.
At my company, the build scripts (usually a build folder) are an svn:external to another subversion repository that is only controlled by build/release engineers. Developers can control variables such as servers it can deploy to, but not what those functions do. This same code is reused amongst multiple projects in flight, and only a few devops folks can alter it, not the entire development staff.
Addition: When accessing shared resources, we use a system account that has only read access to those resources. Further: jenkins,development projects and build/deploy code are written to handle complete loss of jenkins project workspace and deploy environments. This is basic build automation/deploy automation that leads to infrastructure automation.
Basic rule: Murphy's law is going to happen. You should write scripts that are robust and handle cold start scenarios and not worry about wild intruder theories.