I've been assigned a project to write some kind of a script that will perform a sanity check on a Linux server implementation to determine if it has a number of dependencies installed before source code is deployed to it. I need to check for the presence of applications such as PHP, Nginx, PostgreSQL, etc and likely confirm version numbers for these as well. These dependencies are required for the given source code to be able to run properly on the server.
The problem is, I'm not sure how to approach this due to my novelty in working with Linux. I've done some research on this and thought that the solution might be to use a combination of combing through the list of running services with a command such as "chkconfig --list" and pinging individual applications with commands such as "php -v" and then asserting the that results from these equate to what I'm looking for.
Pardon if that makes no sense whatsoever, I really am new to this. I was then thinking I could place these "tests" inside of a shell script or something that could be run whenever a test on the server needed to be executed. I would aggregate the true/false results of my assertions and output whether the sanity check passed based on that. Any guidance would be greatly appreciated.
Thank you.
Revision: In lieu of a shell script, I was also thinking I could write this in Python. Does anybody know of any good Python libraries that allow querying of system services?
If your target systems are managed by reasonable people, the software will be managed by the packaging system. On Redhat, Fedora, CentOS or SUSE systems that will be RPM. On any system derived from Debian it will be APT.
So your script can check for one of those two packaging systems. Although be warned that you can install RPM on a Debian system so the mere presence of RPM doesn't tell you the system type. Packages can also be named differently. For example, SUSE will name things a bit differently from Redhat.
So, use uname and/or /etc/issue to determine system type. Then you can look for a particular package version with rpm -q apache or dpkg-query -s postgresql.
If the systems are managed by lunatics, the software will be hand-built and installed in /opt or /usr/local or /home/nginx and versions will be unknown. In that case good luck.
Related
My customers are novice Linux users, and so am i.
When I gave them my App packaged with ansible, they saw ansible problems, when i gave them manual steps, they also screwed that up, now i have 3 last options, either a perl/bash script or a snappy/deb/rpm package or Linux containers, can anyone share their experience on the safest way to see less problems when installing my app (Written in C)?
This depends on the nature of your application. Debs, rpms etc. are all fine but depend on which distro you're using.
If it's C application, it might make sense to make it a static binary. That way, you'll have to download a single file and just click on it to make it run. It will be big but it should work fine regardless of what else is there. Otherwise, you'll have to worry about dependencies etc.
As it was commented before it depends what you did to deploy the product.
In general, if you have dependencies (previous packages that you assume were already installed) or your installation is complex - use rpm or deb.
However if you target multi-platform bare in mind you will have at least two releases (one rpm and one deb...)
If configuration or installation is easier you can just give them an install script.
If your application requires a specific environment with specific configuration/packages I'd consider containers although I never done that personally before.
Currently I'm manually distributing and updating two applications over 50 computers running CentOS 6.5 and Ubuntu 14.04. Each time the new version is available for either of my applications,i have to copy all files and update it in all the computers by manually.its very time consuming and frustrating.
to avoid this manual process over 50 computers,I like to maintain a central server that contain the latest version of the applications and whenever need to install or update just type a command in client pc like we use in CentOS and Ubuntu to install a software
in Ubuntu
sudo apt-get install vlc
and in Cent OS
sudo yum install vlc
one of the programs written in java and other is written in python
I google it and can't find any good and useful source about how to do this.
some one alrady done this or knows how to achive this please help.
You need to create packages to make this happen.
Ubuntu uses the Debian package format, so you can use Debian's New Maintainer's Guide, which is the canonical tutorial on how to create a Debian package. It makes the assumption that you're going to upload the package to Debian, which in your case isn't true, but that just means you need to skip some sections of the document.
For RPM, there isn't such a document AFAIK, but there is the book 'max rpm' (which unfortunately is somewhat outdated), and fedora has augmented that with some guidelines and best practices which they've put on their wiki. Since RHEL is created by forking fedora and stabilizing that, and since CentOS is based on RHEL, what goes for fedora goes for CentOS, too.
These methods will create packages manually, which is always the best way and will result in the least problems afterwards. However, they take time. If you don't want to spend that time, there are also a few options to generate packages which will automate part or all of the job for you. Personally, however, I'm not a fan of these methods and therefore wouldn't recommend them.
Finally, another option is to not create packages, but to use a config management system like puppet to automate the deployment. It's even available in Ubuntu and EPEL.
edit I notice you may actually be asking about creating a repository instead. That's a different thing. There are several tools to help you do that; at core, all they do is run createrepo for RPM packages, or dpkg-scanpackages for debian packages. You can do that yourself, or investigate time in a tool like reprepro or aptly or some such.
I would like to find out a given disabled or stopped unix/linux service, can we verify if it has a valid binary path?
I am not sure if it necessary to check out this? I am newbie to unix/linux world.
Or is it by default all installed unix/linux services are verified with the existence of its binary and path?
For example, the installed unix/linux service, sshd.
Check its status through the command "service sshd status"
Then, check its binary path using command "which sshd".
Is there a better way to check this out?
How if I would like to checkout all stopped/disabled unix/linux services that are either stopped or disabled?
Modern GNU/Linux-Distributions allow to manage software installations using a software management system. Different such systems are in use, but they all share a common principle: a package carries within itself a description of all of its dependencies, dependencies for installation, configuration and runtime. All of these dependencies are checked prior to an installation. So unlike one forces any actions here one has the guarantee that if a package is installed, then all its requirements are fulfilled.
Looking at your example of the sshd (the openssh daemon) this means:
if you installed this package, then you can rely on the fact that all required components are installed and match in their versions and so on. IN this case the daemon control scripts and the main executable are actually contained inside the same package, this is typical for all daemon packages. So if that package is installed, then all of its content is installed.
You could manually check if all contained files of a package are actually existing. But if you really feel this has to be done, for example if you suspect some haves somehow were deleted by accident (which is actually pretty hard to do for such system files), then you can ask the software management system to verify the package integrities! For example on an openSUSE system you can say zypper verify sshd. That command verifies both: if all files mentioned in that package actually exist, are the ones originally installed (unaltered) and have correct permissions. In addition all package dependencies are checked as well. So if no error is thrown you can rely pretty much on the fact that everything is fine.
This is obviously jsut an example, different distributions use different management systems, as mentioned above. But they all offer more or less the same features. And once you understood the idea behind this approach you probably never want to miss that elegance and security any more...
We have a business application that basically runs on an os-independent stack (tomcat+java+mysql) but we have always run it redhat or centos.
There is a customer that is insisting to run it on opensolaris for his own reasons (an expensive everything-is-included support agreement with Sun).
How painful can such a migration be? We have a lot of configuration file and support scripts such as:
apache
apache/tomcat connector
email interaction with postfix
customized service start/stop
a couple of cron jobs (backup, monitoring)
different users and permissions (java, mysql, email, backup...)
Our build process outputs a .tar.gz file with our business code + some shell scripts that edit all the os-configuration files.
Any previous experience on this.
The biggest issues will be with the non-POSIX (non-standard) options you've used to the GNU tools provided on Linux that are not in the Solaris standard commands. You might decide that porting the relevant tools from the GNU set is simpler than modifying your system. If you've laced the code with absolute pathnames for commands (/usr/bin/ls) but you decide to use the GNU versions instead, you've got to find a way of fixing those. I'd be extremely cautious about replacing the OpenSolaris versions with the GNU versions; you don't know when you would break something that the system relies on. So, you would put the GNU commands in a separate directory - probably not /usr/local because that is for the machine owners to populate, not you as an application-monger - and arrange for that to be used in place of the system commands. (Note: on Solaris, /bin is a symlink to /usr/bin; I assume the same is true of OpenSolaris.) AFAIK, Postfix is not standard on OpenSolaris, so you'd have to ensure you get that installed, too.
All of this is doable - there's nothing insuperable. But a lot depends on your code base.
We run both, though we don't use OpenSolaris as a web servers.
The good:
OpenSolaris comes with the gnu tools, so, get your path right and that's ok.
Most things just build and run just fine.
The not so good:
Make sure that you've installed and are using bash. Otherwise all those bashisms that you are using that you didn't think you were using will bite you.
Make sure that you're not using hard coded paths to /usr/bin or /bin. These tools are not the GNU ones and therefore have different options. Use /usr/gnu as mentioned above.
You don't have the huge number of packages that you can install straight off as you do with yum or apt. Yes, you have a package manager, it's just not quite so well populated.
As a result you probably will be installing packages by hand. They should install, it's just a bit more work for your system admins.
Are you sure that OpenSolaris runs well on your hardware? It's worth a check. You might find that some of the hardware drivers aren't as well tested.
Otherwise we find OpenSolaris to be nice. It has a lot of good ideas.
Have you looked at Nexenta - http://www.nexenta.org/os It's the OpenSolaris kernel with a Ubuntu userland.
OpenSolaris includes all the GNU utilities already, just point your scripts at /usr/gnu/bin
Installing Postfix shouldn't present any problems, and Apache/MySQL are present in a base OpenSolaris install (in truth, the Cool Web Stack stuff makes it about as easy to administer as WAMP/Instant Rails). Beyond which, SMF manifests (SMF is a replacement for rc scripts sort of like OSX's launchd, though you can still use regular init scripts) may make your life easier, since specifying dependencies and run order is somewhat nicer (it'll recursively start/stop all dependent services also).
Tomcat certainly works, though everybody I know on OpenSolaris uses GlassFish. YMMV, but deploying a .war is pretty much the same everywhere.
It may not be a bad first step to deploy into a LX branded zone (think FreeBSD jails or Linux vServer for a comparison), as the LX branded zones can run Linux binaries, and are explicitly CentOS/RHEL based.
Other than that, OpenSolaris is a Xen dom0 since b77 or something, and putting CentOS/RHEL into a domU is dead simple, if that's an option.
You also get all the Solaris goodies along with it (DTrace, ZFS, network virtualization [via CrossBow], etc). Who knows? You may even like it! Java is Java, so that shouldn't pose any issues.
you'll probably have to rewrite a big part of your scripts (user creations, service launch) as it is probably different in CentOS and OpenSolaris.
as previously written, ask your customer to install the GNU tools so you'll have less work to rewrite your scripts.
os configuration files may also not be in the same format, you'll need to check.
your tar.gz file should be extractable without troubles, but again you will have less surprises if you use GNU tools. some unix OS have tar with some limitations
Any previous experience on this.
(maybe a little offtopic)
we package and distribute our java/tomcat/postgresql/unix application with all binaries referenced in our scripts. this implies to have 1 build system for each OS we support, this implie we support our application but also external binaries, but in the end we do not have bad surprises # customers.
we also ask them to do all root operations (user creation, directory creation, sendmail config, system tuning) before we install the application.
we have written shutdown / startup scripts for all supported OS, and their installation is the only thing we do in root on the customer machine.
Beside the fact that you're a troll, somebody just said above that (Open)Solaris has:
- ZFS
- DTrace
We can understand that you are afraid of not losing your RHCE job, but you just proved me once again that my decision as an employer to ignore all the certifications when interviewing people was a good one. It seems that a large percentage of such people (especially in the Microsoft world) are not so... open-minded, to put it nicely.
Regards,
Alex
I'm going to rebuild my Linux box [yet] again. I have to create a few user groups, user accounts and install my standard packages. Until now I've just used the GUI tools. I was wondering if anyone has any recommendations on writing a script to create users, groups and install standard packages after I do a minimal install of my latest Fedora build? Sometimes I run Ubuntu so I'd like the script to be somewhat generic.
For .deb distros, use FAI. For .rpm distros, use Kickstart. For system management after installation, use cfengine.
Fedora and Ubuntu use totally different package managers, so you won't be able to easily do it in any sort of generic way.
In CentOS (which is RedHat Enterprise Edition with the serial numbers filed off, and so therefore pretty close to Fedora), we did this using Kickstart files. These files have a simple syntax that enabled you to specify users, groups and packages to install, and even to script some custom stuff.
While I haven't done this yet, I have a similar problem. I'm considering a virtualization host and multiple client OS (Ubuntu and CentOS being the top 2 candidates) - that way once I get the client configured as I want it, I can save it off for reloading as needed.
Doesn't get around the original setup issue, but does limit the "rebuild my Linux box [yet] again" problem.
You may want to consider it.
It may be overkill but you can check out Puppet.
From their website:
Puppet is a system for automating
system administration tasks.
I'm just starting looking for ways to automate system administration, so I don't have much experience with it yet.
If all you need to do is create users and groups and install packages then I would suggest that you just write two separate scripts.
It might be that you could share the users and groups part but only if all the distributions you use have the same policy for creating them (for example Ubuntu creates a group for each user while I am sure some distributions have a "users" group as well).
You could take a look at the useradd and groupadd commands which should be available everywhere. For Ubuntu there is also the friendlier adduser and addgroup and I would not be surprised if Fedora has a set of similar commands.
After groups are setup you just need to feed the package manager a big list of packages you need to have installed. Trying to install packages which are already installed should be safe, so you could install the packages you need on a "clean" new install and then dump a package list.
So to summarize: If you don't plan to support more than two distributions then I suggest just writing the two scripts separately.
Another option to help with constantly rebuilding a box is Norton Ghost, with ghost you can make an image and then just re-image the drive as needed. You install it and configure it to your liking, then take an image.
It's gonna be difficult to make the script generic, but you could use any sort of scripting tool (bash, or ruby or whatever) and try and check what distro is running and then run the appropriate commands to install software. There are various ways to check what distro is running here
Creating groups should be the same on all distros, and you may even be able to drop in an already configured /etc/passwd and /etc/groups (though I haven't tried that, and it may not work).
The response above, about the different distros using different methods is dead on. It's like trying to use the same part for a Chevy and a Ford (there's the car analogy, for you).
The easiest method I've found is to learn about setting up partitions for the different mount points i.e. / ; /home ; /var ; /opt are the big ones.
This lets you keep your users, groups, and many of your apps during your rebuilds. Changing distros will break a lot of things, but your user accounts should still be there.