I need a software to manage configurations of linux servers in one central location. It should be able to push changes to servers automaticly. Version control would be an advantage...
I've heard good things about Puppet (as matli suggested) and Cfengine, which are both listed at http://en.wikipedia.org/wiki/Comparison_of_open_source_configuration_management_software
Have a look at Puppet
There's also Chef and bcfg2. If you're a Java guy, Control Tier is nice. There are some new projects in the python space to address the issue as well: Kokki, Overmind, Edison.
They all do essentially the same thing just in different ways. If you're a ruby developer, Chef is going to feel VERY familiar. If you aren't a developer and don't care about the language, puppet, while written in Ruby, abstracts it all out into a DSL.
Checkout Bluepring and Blueprint I/O. Blueprint is an open-source tool for figuring out what's been done to a server. Packages, file modifications and source installs are detected and packaged up in a reusable formtat - a blueprint. Blueprint I/O is a tools for moving blueprints to another server. Together, they make for a drop dead simple configuration management tool. Hope this helps.
https://github.com/devstructure/blueprint (Blueprint # Github)
https://github.com/devstructure/blueprint-io (Blueprint I/O # Github)
Old question, but still might be helpful to you: We are releasing ConfigChief, a hosted central configuration repository with versioning, audit, access control which turns the problem around by pulling configuration from the server rather than pushing it which is the solution provided by Puppet and the rest.
You can signup for a beta at http://woot.configchief.com if you like.
Related
Using a lot of (official and non official) terraform providers, I'm looking for a tool to perform security analysis on terraform providers before executing terraform plan/apply commands (and so executing providers code). I want to prevent malicious code from providers to be executed blindly.
I'm basically executing terraform providers mirror command to save local copies of required providers and I'm wondering if I can security scan that result.
I tested kics, checkov and tfsec but they are all looking for security issues in my terraform static code but not in providers.
Do you have any good advices regarding this topic ?
This is actually quite a good question. There are many other problems that can be reduced to same generic question - how to make sure that the thing you downloaded from the internet does not do anything malicious to you like e.g.:
How to make sure that a minecraft plugin does not hack you?
How to make sure that a spring boot dependency does not hack you?
How to make sure that a library xxx you attach to your project does not do harm to you?
Should you use docker image yyy in your project?
Truth is: everything you use has the potential to explode right in your face (or more correctly: right into the face of the system owner). That's why the system owner (usually a company) defines a set of rules to follow what is allowed and what is not allowed. No set of rules you are aware of? Below a set of rules we came up with ourselves when thinking about on-boarding a new library for some projects to use:
Do not take random stuff from github. Take only products with longer history, small bug backlog, little to none past issues in the CVE list, actively maintained.
Do static code analysis yourself. Sometimes it is possible to have tools that work on binaries level do that for you. Sometimes you can do it on source level only. In case of Java libraries, check what tools like Dependency Track think about the library and version you are about to use.
Run the code and see how it works: what does it write, what does it read, what URLs does it communicate with (do a TCP dump if necessary).
Document everything you have done somewhere.
This gives you no 100% confidence that things will not go terribly wrong. But this is a systematic approach that will reduce the risk of doing something stupid.
I have a bunch of TeamCity agents (Windows, Linux, AWS, OpenShift). To have consistent state it is desirable that all of them have the same software+version installed. Manually checking them is very tedious
Hence I have decided to have an application which shows this information as a dashboard, i.e. a snapshot view of all the agents and the software installed on them. I have decided to use Python(v3.6) for the implementation. I am not a hardcore developer hence this will be learn and do project for me.
I was thinking of some sort of a code-base on all agents that would fetch the necessary details for that agent only.
I will then have a central server that will utilize this data from each agent and then display it in the form of a dashboard.
Please let me know if the above design is the proper way of doing it or please suggest some alternate if any.
If I can have some pointers as how to go about implementing it that would be of great help.
If you have full control over the agents' machines, consider using Ansible to enforce the desired configurations. In general it is much more convenient an safe to control agents' configurations rather than ask them if they have proper ones. And with Ansible or similar configuration management tool you can do this in a scalable way.
Deployment
I currently work for a company that deploys through github. However, we have to log in to all 3 servers to update them manually with a shell script. When talking to the CTO he made it very clear that auto-deployment is like voodoo to him. Which is understandable. We have developers in 4 different countries working remotely. If someone where to accidentally push to the wrong branch we could experience downtime, and with our service we cannot be down for more than 10 minutes. And with all of our developers in different timezones, our CTO wouldn't know till the next morning and we'd have trouble meeting with the developers who had the issue because of vast time differences.
My Problem: Why I want auto-deploy
While working on my personal project I decided that it may be in my best interest to use auto-deployment, but still my project is mission critical and I'd like to mitigate downtime and human error as much as possible. The problem with manual deployment is that I simply cannot manually deploy on up to 20 servers via SSH in a reasonable amount of time. The problem perpetuates when I consider auto-scaling. I'd need to spin up a new server from an image and deploy to it.
My Stack
My service is developed on the Node.js Express framework. These environments are very rich in deployment and bootstraping utilities. My project uses npm's package.json to uglify my scripts on deploy, and also runs my service as a daemon using forever-monitor. I'm also considering grunt.js to further bootstrap my environments for both production and testing environments.
Deployment Methods
I've considered so far:
Auto-deploy with git, using webhooks
Deploying manually with git via shell
Deploying with npm via shell
Docker
I'm not well versed in technologies like Docker, but I'm interested and I'd definitely give points to whoever gave me a good description as to why I should or shouldn't use Docker, because I'm very interested in its use. Other methods are welcome.
My Problem: Why I fear auto-deploy
In a mission critical environment downtime can put your business on hold, and to make matters worse there's a fleet of end users hitting the refresh button. If someone pushes something that's not build passing to the production branch and that's auto-deployed, then I'm looking at a very messy situation.
I love the elegance of auto-deployment, but the risks make me skeptical. I'm very much in favor of making myself as productive as possible. So I'm looking for a way to deploy to many servers with ease, and in very efficient manner.
The Answer I'm Looking For
Explain to me how to mitigate the risks of auto-deployment, or explain to me an alternative which is better suited to my project. Feel free to ask for any missing details in the comments.
No simple answer here. I offer a set of slides published by Mike Brittain from Etsy, a company that practices continuous deployment:
http://www.slideshare.net/mikebrittain/mbrittain-continuous-deploymentalm3public
Selected highlights:
Deploy frequently and in small batches
Use config/feature flags to control system behaviour and "dark release" major features
Code review all changes to the production branch
Invest in monitoring and improve the feedback loop
Manage "services" separately to the "application" and be mindful of run-time version and backwardly compatible changes.
Hope this helps
I have a number of Windows servers at work that are used for staging web sites for clients while they are being created.
I wanted to start using versioning on them so that when we work with outside vendors on a project, if/when they overwrite my work, I'd like to be able to go back and get the version before.
My question is that I think I'm not looking for the correct terms in searching for information, but what kind of resources are there to learn how to install the software for versioning or a site to help me get started.
Any and all suggestions would be appreciated.
Steph
Since your development workflow can be decentralized (as in "there isn't always one central repository), DVCS tools, with their common tasks described here) can be more adapted.
Git-Scm
Mercurial (see HgInit.com for a very good tutorial like the kind you are after)
Plastic SCM (which has a DVCS nature)
A friend of mine and I are developing a web server for system administration in perl, similar to webmin. We have a setup a linux box with the current version of the server working, along with other open source web products like webmail, calendar, inventory management system and more.
Currently, the code is not under revision control and we're just doing periodic snapshots.
We would like to put the code under revision control.
My question is what will be a good way to set this up and software solution to use:
One solution i can think of is to set up the root of the project which is currently on the linux box to be the root of the repository a well. And we will check out the code on our personal machines, work on it, commit and test the result.
Any other ideas, approaches?
Thanks a lot,
Spasski
Version Control with Subversion covers many fundamental version control concepts in addition to being the authority on Subversion itself. If you read the first chapter, you might get a good idea on how to set things up.
In your case, it sounds like you're making the actual development on the live system. This doesn't really matter as far as a version control system is concerned. In your case, you can still use Subversion for:
Committing as a means of backing up your code and updating your repository with working changes. Make a habit of committing after testing, so there are as few broken commits as possible.
Tagging as a means of keeping track of what you do. When you've added a feature, make a tag. This way you can easily revert to "before we implemented X" if necessary.
Branching to developt larger chunks of changes. If a feature takes several days to develop, you might want to commit during development, but not to the trunk, since you are then committing something that is only half finished. In this case, you should commit to a branch.
Where you create a repository doesn't really matter, but you should only place working copies where they are actually usable. In your case, it sounds like the live server is the only such place.
For a more light-weight solution, with less overhead, where any folder anywhere can be a repository, you might want to use Bazaar instead. Bazaar is a more flexible version control system than Subversion, and might suit your needs better. With Bazaar, you could make a repository of your live system instead of setting up a repository somewhere else, but still follow the 3 guidelines above.
How many webapp instances can you run?
You shouldn't commit untested code, or make commits from a machine that can't run your code. Though you can push to backup clones if you like.