We have a set of Mac machines (mostly PPC) that are used for running Java applications for experiments. The applications consist of folders with a bunch of jar files, some documentation, and some shell scripts.
I'd like to be able to push out new version of our experiments to a directory on one Linux server, and then instruct the Macs to update their versions, or retrieve an entire new experiment if they don't yet have it.
../deployment/
../deployment/experiment1/
../deployment/experiment2/
and so on
I'd like to come up with a way to automate the update process. The Macs are not always on, and they have their IP addresses assigned by DHCP, so the server (which has a domain name) can't contact them directly. I imagine that I would need some sort of daemon running full-time on the Macs, pinging the server every minute or so, to find out whether some "experiments have been updated" announcement has been set.
Can anyone think of an efficient way to manage this? Solutions can involve either existing Mac applications, or shell scripts that I can write.
You might have some success with a simple Subversion setup; if you have the dev tools on your farm of Macs, then they'll already have Subversion installed.
Your script is as simple as running svn up on the deployment directory as often as you want and checking your changes in to the Subversion server from your machine. You can do this without any special setup on the server.
If you don't care about history and a version control system seems too "heavy", the traditional Unix tool for this is called rsync, and there's lots of information on its website.
Perhaps you're looking for a solution that doesn't involve any polling; in that case, maybe you could have a process that runs on each Mac and registers a local network Bonjour service; DNS-SD libraries are probably available for your language of choice, and it's a pretty simple matter to get a list of active machines in this case. I wrote this script in Ruby to find local machines running SSH:
#!/usr/bin/env ruby
require 'rubygems'
require 'dnssd'
handle = DNSSD.browse('_ssh._tcp') do |reply|
puts "#{reply.name}.#{reply.domain}"
end
sleep 1
handle.stop
You can use AppleScript remotely if you turn on Remote Events on the client machines. As an example, you can control programs like iTunes remotely.
I'd suggest that you put an update script on your remote machines (AppleScript or otherwise) and then use remote AppleScript to trigger running your update script as needed.
If you update often then Jim Puls idea is a great one. If you'd rather have direct control over when the machines start looking for an update then remote AppleScript is the simplest solution I can think of.
Related
My task is to sync folders between two computers. One which acts as a windows server which is the host and the other one is a linux based server. The file transfer has to be secure and encrypted. Are there are any free softwares which will help me do this task.
Additionally the syncing should automatically happen after every pre decided interval.
I have a recollection that WinSCP can be invoked through command line. There, you have the option to synchronize folders (and the whole hierarchy there in). It may be worth trying.
Total Commander also has FTP/SFTP capabilities, but I'm not sure you can invoke it through command line.
One point to consider: If the process is to run automatically, you need to hard-code the username and password for the connection. There your security becomes compromised.
I'm maintaining the servers of a web game. Whenever we add a new server to our game, I have to configure many environment details and install softwares (for example, testing if some ports of the new machine can be connected from other places, installing mysql-client, pv..., copying the game server files from the other machine, and changing mysql server connection URL) on the new machine.
So my question is "How can I automize the whole process of setting up a new server?" Because most of the works I do are repetitive. I don't want to do this kind of job whenever a new machine comes in.
Is there a tool that allows me to save the state of a linux machine so that next time when we buy a new server, I can copy the state of an old linux machine to the new machine? I think this is one of the ways to automize the process of setting up a new game server.
I've also tried using some *.sh scripts to automize the process. But it's not always possible to get the return value of every command I execute. This is why I come here and ask for help.
Have you looked at Docker, Ansible, Cheff or Puppet?
In Docker you can build a new container by describing required operations in docker file. And you can easily move container between machines.
Ansible, Cheff and Puppet are systems management automation tools.
I doubt you'll find such tool to automatize an entire customization process because it's rather difficult to define/obtain a one-size-fit-all linux machine state, especially if the customisation includes logical/functional sequences.
But with good scripting you can obtain a possibly more reliable customisation from scratch (rather than copying it from another machine). I'd recommend a higher-level scripting language, tho, IMHO regular bash/zsh/csh scripting is not good/convenient enough. I prefer python, which gives easy access to every cmd's return code, stdout, stderr and with the pexpect module it can drive interactive cmds.
There are tools to handle specific types of customisations (sw package installations, config files), but not all I needed, so I didn't bother and went straight for custom scripts (more work, but total control). Personal preference, tho, others will advise against that.
I am trying to pick a version control software for our team but I don't have much experience for it before. After searching and googling, it seems Mercurial is a good try. However, I am a little bit confused about some general information about it. Basically, our team only have 5 people and we all connect to a server machine which will be used to store the repositories. The server is a Redhat Linux system. We probably use a lot of the centralized workflow. Because I like the local commit idea, I still prefer the DVCS kind software. Now I am trying to install mercurial. Here are my questions.
1) Does the server used for repositories always need to be installed the software "mercurial-server "? Or it depends on what kind of workflow it uses ? In other words, is it true if there is no centralized workflow used for works, then the server can be installed by "mercurial client" ?
I am confused about the term "mercurial-server". Or it means the mercurial installed on the server is always called "mercurial server" and it does matter if it is centralized or not. In addition, because we all work on that server, does it mean only one copy of mercurial is required to install there ? We all have our own user directory such as /home/Cassie, /home/John,... and /home/Joe.
2) Is SSH a must ? Or it depends on what kind of connection between users and the server ? So since we all work in the server, the SSH is not required right ?
Thank you very much,
There are two things that can be called a "mercurial server".
One is simply a social convention that "repository X on the shared drive is our common repository". You can safely push and pull to that mounted repository and use it as a common "trunk" for your development.
A second might be particular software that allows mercurial to connect remotely. There are many options for setting this up yourself, as well as options for other remote hosting.
Take a look at the first link for a list of the different connection options. But as a specific answer to #2: No, you don't need to use SSH, but it's often the simplest option if you're in an environment using it anyways.
The term that you probably want to use, rather than "mercurial server", is "remote repository". This term is used to describe the "other repository" (the one you're not executing the command from) for push/pull/clone/incoming/outgoing/others-that-i'm-forgetting commands. The remote repository can be either another repository on the same disk, or something over a network.
Typically you use one shared repository to share the code between different developers. While you don't need it technically, it has the advantage that it is easier to synchronize when there is a single spot for the fresh software.
In the simplest case this can be a repository on a simple file share where file locking is possible (NFS or SMB), where each developer has write access. In this scenario there is no need to have mercurial installed on the server, but there are drawbacks:
Every developer must have a mercurial version installed, which can handle the repo version on the share (as an example, when the repo on the share is created with mercurial 1.9, a developer with 1.3 can't access this repo)
Every developer can issue destructive operations on the shared repo, including the deletion of the whole repo.
You can't reliably run hooks on such a repo, since the hooks are executed on the developer machines, and not on the server
I suggest to use the http or ssh method. You need to have mercurial installed on the server for this (I'm not taking the http-static method into account, since you can't push into a http-static path), and get the following advantages:
the mercurial version on the server does not need to be the same as the clients, since mercurial uses a version-independent wire protocol
you can't perform destructive operations via these protocols (you can only append new revisions to a remote repo, but never remove any of them)
The decision between http and ssh depends on you local network environment. http has the advantage that it bypasses many corporate firewalls, but you need to take care about secure authentication when you want to push stuff over http back into the server (or don't want everybody to see the content). On the other hand ssh has the drawback that you might need to secure the server, so that the clients can't run arbitrary programs there (it depends on how trustworthy your clients are).
I second Rudi's answer that you should use http or ssh access to the main repository (we use http at work).
I want to address your question about "mercurial-server".
The basic Mercurial software does offer three server modes:
Using hg serve; this serves a single repository, and I think it's more used for quick hacks (when the main server is down, and you need to pull some changes from a colleague, for example).
Using hgwebdir.cgi; this is a cgi script that can be used with an HTTP server such as Apache; it can serve multiple repositories.
Using ssh (Secure Shell) access; I don't know much about it, but I believe that it is more difficult to set up than the hgwebdir variant
There is also a separate software package called "mercurial-server". This is provided by a different company; its homepage is http://www.lshift.net/mercurial-server.html. As far as I can tell, this is a management interface for option 3, the mercurial ssh server.
So, no, you don't need to have mercurial-server installed; the mercurial package already provides a server.
I need a tool/script to fetch network card configurations from multiple Linux machines, mostly Red Hat Enterprise 5. I only know some basic bash, and I need something that can be run remotely pulling server names from a CSV. It also needs to be be run quickly and easily by non-technical types from a Windows machine. I've found WBEM/CMI/SBLIM, but I'd rather not write a whole C++ application. Can anyone point me to a tool or script that could accomplish this?
For Red Hat Enterprise Linux servers, you likely just need to take a copy of the files in /etc/sysconfig/networking/devices/ from each server. You can use an sftp client to accomplish that over ssh.
(The files are just easy-to-read text config files containing the network device configuration)
Can you give more details as to what information you need to pull? The various parameters to ifconfig give quite a lot of information about a Linux machine's network card configuration, so if you can do it that way it will be very easy. Simply write a script that converts the CSV into something white-space delimited, and then you can do something like:
#!/bin/bash
for host in $HOSTS ; do
CARDINFO=`ssh $host 'ifconfig'`
# Do whatever processing you need on CARDINFO here
done
That's a very rough sketch of the pseudocode. You'll also need to set up passwordless SSH on the hosts you want to access, but that's easy to do on Red Hat.
If you want to use WBEM/CIM for that (as mentioned in your original question), and you prefer a scripting environment over a programming language such as C/C++/Java, then there are PyWBEM and PowerCIM as two ways to do that in Python. If it needs to be bash etc, then there are command line clients (such as cimcli from the OpenPegasus project or wbemcli from the SBLIM project) and you could parse their output. Personally, I would prefer a Python based approach using PyWBEM. It is very easy to use, connecting to a CIM server is one line and enumerating CIM instances of a class is one more line.
On the side of the Linux system you want to query, the CIM server would need to run (tog-pegasus or sfcb) along with the right CIM provider packages (sblim). This approach has the advantage that your interface will be the same regardless of which Linux distribution you are using. Parsing config files is often dependent on the type of Linux distribution and I have seen them change across versions.
One main purpose of CIM is to provide reliable interfaces that are consistent across different types of environments and that change only compatibly over time.
Last but not least, using CIM allows you to get away without having to install any agent software on the system you want to inspect (as long as you can ensure that the CIM server is running).
Andy
I was recently working with a product from Symantech called Norton EndPoint protection. It consists of a server console application and a deployment application and I would like to incorporate their deployment method into a future version of one of my products.
The deployment application allows you to select computer workstations running Win2K, WinXP, or Win7. The selection of workstations is provided from either AD (Active Directory) or NT Domain (WINs/DNS NetBIOS lookup). From the list, one can click and choose which workstations to deploy the end point software which is Symantech's virus & spyware protection suite.
Then, after selecting which workstations should receive the package, the software copies the setup.exe program to each workstation (presumable over the administrative share \pcname\c$) and then commands the workstation to execute setup.exe resulting in the workstation installing the software.
I really like how their product works but not sure what they are doing to accomplish all the steps. I've not done any deep investigations into this such as sniffing the network, etc... and wanted to check here to see if anyone is familiar with what I'm talking about and if you know how it's accomplished or have ideas how it could be accomplished.
My thinking is that they are using the admin share to copy the software to the selected workstations and then issuing an RPC call to command the workstation to do the install.
What's interesting is that the workstations do this without any of the logged in users knowing what's going on until the very end where a reboot is necessary. At which point, the user gets a pop-up asking to reboot now or later, etc... My hunch is that the setup.exe program is popping this message.
To the point: I'm looking to find out the mechanism by which one Windows based machine can tell another to do some action or run some program.
My programming language is C/C++
Any thoughts/suggestions appreciated.
I was also looking into this, since I too want to remote deploy software. I chose to packet sniff pstools since it has proven itself quite reliable in such remote admin tasks.
I must admit I was definitely over-thinking this challenge. You have probably done your packet sniff by now and discovered the same things I have. I hope by leaving this post behind we can assist other developers.
This is how pstools accomplishes execution of arbitrary code:
It copies a system service executable to \\server\admin$ (you either have to already have local admin on the remote machine, or supply credentials). Once the file is copied, it uses the Service Control Manager API to make the copied file a system service and start it.
Obviously, this system service can now do whatever it wants, including binding to an RPC named pipe. In our case, the system service would install an msi. To get confirmation of successful installation you could either remote poll a registry key, or an rpc function. Either way, you should remove the system service when you are done and delete the file (psexec does not do this, I guess they don't want it to be used surreptitiously, and in that case leaving the service behind would at least give an admin a fighting chance of realizing someone had compromised their box.) This method does not require any preconfiguration of the remote machine, simply that you have admin creds and that file sharing and rpc are open in the firewall.
I've seen demos in C# using WMI, but I don't like those solutions. File sharing and RPC are most likely to be open in firewalls. If they aren't, file sharing and remote MMC management of the remote server wouldn't work. WMI can be blocked and still leave these functional.
I've worked with a lot of software that does remote installations, and a lot of them are not as reliable as pstools. My guess is that this is because those developers are using other methods that are not as likely to be open at the firewall level.
The simple solution is often the most elusive. As always, my hat is off to the SysInternals folks. They are true hackers in the positive, old school meaning of the word!
This sort of functionality is also available with products LANDesk and Altiris. You need a daemonized listener on the client side that will listen for instructions/connections from the server. Once a connection is made any number of things can happen: you can transfer files, kick on installation scripts, etc. usually transparently to any users on that box.
I've used the Twisted Framework (http://twistedmatrix.com) to do this with a small handful of Linux machines. It's Python and Linux, not Windows, but the premise is the same: a listening client accepts instructions from a server and executes them. Very simple.
This functionality can also be accomplished with VB/Powershell scripts in a Windows-based domain.