I use a mercurial repository for global configuration. The system config files are linked to /opt/config which is a hg repo owned by root.
I d like all users to be able to update settings from repo i.e. to call hg pull -u in /opt/config
I tried to create the following script
# -rwsr-x--x 1 root users 343 Mar 15 14:10 /bin/update_config
#! /bin/bash
cd /opt/config
hg pull -u
(Pay attention, the s-bit is set) . In this case, hg does not read the settings from /root/.hgrc which contain the HTTP login parameter (user cannot does not know the parameters)
even if I do export HOME=/root the hgrc file is not read.
How should I change my script to make it possible?
EDIT
It seems to be a general permission problem. I use sles11. The line touch /root/bla does not work in this script, why?
Mercurial being a distributed versioning system, it seems to me that you are not using it correctly. If users are required to modify the repository, every user should handle its own repository and then configure it to push into your desired location (/opt/config). Hence, the mercurial workflow will handle the merge problems. If they are only consumers of the repository, you should either 1) create a cron entry to update it automatically or 2) use a continuous integration system like Jenkins or TeamCity that will automatically update the repository when something is pushed to it.
If you still want to realize what you asked, you should look into the sudo command for this purpose. Make the /opt/config ownership to a new passwordless user, configure sudo to allow the switch to this user without password and make the configuration only in ~theuser/.hgrc . This will make it easier to maintain (only a single .hgrc to handle).
Related
We have a scenario in which single server is running, which is getting data from the network span.
Every developer should work on their machine locally but the data to work on is only available in the server. how can I get the data to be replicated into each developers machine so that once they have completed development on their local machine, developers can push it to a GIT in the server.
PS: The network span data is constantly written to the server (data is in size of 100s of GB's).
What we have tried so far:
So we created a GIT server in the server we were getting the data on. But once a developer log in using his username then he creates a new branch in a directory. This works fine until another developer logs into the server with his username and switches to another branch in the same directory which will cause all the developers branch to the new one. which is not what we were expecting.
Probabily this question should go to https://serverfault.com/, but, anyway...
The git advantage is to have local and remotre repositories, so, in the server, you should have "only" the remote repositories, and they should be cloned in localmachines.
to work with that paradigm, or with the one you are asking for, you need a umask of 007 (depending on your distribution edit /etc/login.defs and change there)
You should have diferent groups for the diferent kind of shared projects, and a user to "own all the repositories", for example, git-adm ).
With all the prerequisites, you create with that user the base folder for all the repositories:
sudo -i
mkdir /srv/git
chown git-adm:gitgrp /srv/git
chmod g+s /srv/git
exit
The last line in the "sticky bite", wich allows to mantain the group (and avoid the problems you previously stated), so, in order to cerate a repository should be something like:
sudo su - git-adm
mkdir /srv/git/<group>/<repoName>.git
cd /srv/git/<group>/<repoName>.git
git init --bare
exit
And thats all: if the folder /srv/git/<group>/ we owned for a diferent group, then it'll keep the group.
The title is general, but I have more specific questions. I am deep in a permissions nightmare trying to set up a "push-to-deploy" system using Git.
From my local machine, I push by SSH to the server (Ubuntu 14.04). I have the server set up as the remote
git remote add development devuser#development.server:/home/dummyuser/bare/repo.git
This bare repository is within the home folder of a dummy user dummyuser that we use to handle deployment tasks. devuser is my own account on the development server.
I have a post-receive hook set up within the remote repository (development.server:/home/dummyuser/bare/repo.git/hooks/post-receive) that's intended to deploy files via git checkout to a web server directory on the same server, call it webfolder/. That folder currently has permissions
drwxr-xr-x dummyuser www-data webfolder/
where www-data is the group associated with the Apache user.
If I have the post-receive hook script use the command
git --work-tree=/var/www/webfolder --git-dir=/home/dummyuser/bare/repo.git checkout -f
I get errors that it can't write to webfolder/, which is predictable since I assume the script is running as me (devuser) since I did the instigating push via SSH, and devuser doesn't have any permissions on webfolder/.
However, if I change the script to act as dummyuser,
sudo -u dummyuser git --work-tree=/var/www/webfolder --git-dir=/home/dummyuser/bare/repo.git checkout -f
just to see what happens, I have the error
warning: unable to access '/home/devuser/.config/git/attributes': Permission denied
There's a couple of things I don't understand about this:
1) Neither /home/devuser/.config/ nor /home/dummyuser/.config/ exist. That's fine, but if Git needs to access a .config/ folder, why wasn't it complaining before when I was setting up bare repos and executing hooks as devuser?
2) Now that I'm trying to act as dummyuser, why is Git looking in ~devuser/ for a .config/ folder? Why isn't it looking in ~dummyuser/?
I've been working on this tiny slice of one single problem in the maddening shitshow that is "using Git" for coming up on four hours now, and my brain is fuzzy, so please use small words.
The problem is something involving sudo -u dummyuser not setting the environment variables that Git expects. If I add HOME=/home/dummyuser to the post-receive hook, the deployment works as expected.
If anyone can provide more details about what's happening or a better solution, write it as an answer and I'll accept it. Couple of notes:
dummyuser doesn't have a login, so using sudo -iu dummyuser in the post-receive script won't work
After setting HOME=/home/dummyuser manually and successfully executing the script, I find that echo $HOME from the terminal returns /home/devuser, so there's no permanent change to $HOME
After successfully executing the hook script, neither ~devuser/ nor ~dummyuser/ nor /root/ have a .config/ folder. So... I still have no idea why Git was hung up on it.
Git expects a .config folder in the user's home directory. If $HOME isn't set correctly, e.g. if it points to a different user's home, Git will try to access $HOME/.config, not knowing that it actually doesn't even exist. However, since the user, and thus Git, doesn't have access to that $HOME, you will receive an error saying Permission denied.
To test that, try to run as dummyuser:
[ -d /home/devuser/.config ] && echo '.config exists!'
You're trying to test if the directory /home/devuser/.config exists. However, since you don't have the needed permissions, you get Permission denied, and you still don't know whether the directory exists or not.
Instead of setting $HOME manually, you could possibly use -H or --set-home:
sudo -Hu dummyuser git --work-tree=/var/www/webfolder --git-dir=/home/dummyuser/bare/repo.git checkout -f
I have a Git repository on Bitbucket named "foo-apps". I have a Linux web server with a local clone of this repository, and I want this server's associated Apache web page to do some Git commands with this repository, such as git pull and git checkout, via a Bash script. The problem is, only the user "foo" has permission to associate with the "foo-apps" repository, and the web page runs as the Apache user, "www-data".
It seems that www-data can do git log and some other commands on the local repository, but not the git pull or the git checkout command. (Just so you know my system: I have an HTML file that contains JavaScript, which contains an AJAX request, which calls a PHP file, which calls my Bash script, which has the Git commands in it.)
What are some ways that I can successfully get those Git commands to work when the process is triggered by the web interface? I am not opposed to any working suggestions, even if they include a complete overhaul of my system... however, I would like to have the simplest effective solution with what I've already got.
Here are some ideas I've thought of and tried out a bit. None of them seem to work, but keep in mind that I've only "half tried" them as I didn't have confidence I was using preferred methods:
Giving www-data permission on my Bitbucket repository
Giving Apache access to foo's ssh keys
Somehow switching to user foo in the script, like with sudo, su, etc.. (I think this type of thinking is more along the lines of what I want... I don't have a lot of control over the settings of the Bitbucket repository. I am fine putting a password in a script, too.)
This web server is on a closed network, and security is not a very high concern for me.
I don't know if it's useful, but here are some of the main Git related errors I've received when trying these methods:
error: cannot open .git/FETCH_HEAD: Permission denied
fatal: BUG: get_tempfile_fd() called for inactive object
/usr/bin/git: /usr/bin/git: cannot execute binary file
I found the answer on this page (thanks odyniec).
I had to add this line to the /etc/sudoers file:
www-data ALL=(foo) NOPASSWD: /var/www/html/my_bash_script.sh
This let Apache have the permission to run that specific script I wanted. And then from my PHP file, instead of running
shell_exec("/var/www/html/my_bash_script.sh");
I had to run
shell_exec("sudo -u foo /var/www/html/my_bash_script.sh");
This answer seems secure and simple.
I have a big problem and I can't understand this topic. I have a server with a website. I created a repository there with git init. Than I made a git add * to add all files from my server to the repository. Than I made a commit to commit all files to the repository.
Than I cloned it with git clone ssh://username#mysite.com/wordpress/.git to my local client.
All worked fine and I got a copy from my project. No I changed something on my local version and made a commit with a push. I looked in FileZilla but the content in the file don't changed. In the other direction when I changed something on the sever and pulled it to the local copy I saw the changes. Do you know why the changes which I made on the local copy are not visible on my sever?
Thank you for your help!
You need to push changes to a central repository that both your local machine and server can pull from (or add them as remotes for each other). A service such as GitHub works nicely for this. Here are instructions for a full workflow that works well for this. Updated instructions can be found in this gist. This workflow uses hooks to do the heavy lifting so that updates to your server are automated.
Using Git to Manage a Live Web Site
Overview
As a freelancer, I build a lot of web sites. That's a lot of code changes to track. Thankfully, a Git-enabled workflow with proper branching makes short work of project tracking. I can easily see development features in branches as well as a snapshot of the sites' production code. A nice addition to that workflow is that ability to use Git to push updates to any of the various sites I work on while committing changes.
You'll need to have Git installed on your development machines as well as on the server or servers where you wish to host your website. This process can even be adapted to work with multiple servers such as mirrors behind a load balancer.
Setting up Passwordless SSH Access
The process for updating a live web server relies on the use of post hooks within the Git environment. Since this is fully automated, there is no opportunity to enter login credentials while establishing the SSH connection to the remote server. To work around this, we are going to set up passwordless SSH access. To begin, you will need to SSH into your server.
ssh user#hostname
Next, you'll need to make sure you have a ~/.ssh in your user's home directory. If not, go ahead and create one now.
mkdir ~/.ssh
On Mac and Linux, you can harness the power of terminal to do both in one go.
if [ ! -d ~/.ssh ]; then mkdir ~/.ssh; fi
Next you'll need to generate a public SSH key if you don't already have one. List the files in your ~/.ssh directory to check.
ls -al ~/.ssh
The file you're looking for is usually named similarly to id_rsa.pub. If you're not sure, you can generate a new one. The command below will create an SSH key using the provided email as a label.
ssh-keygen -t rsa -b 4096 -C "your_email#example.com"
You'll probably want to keep all of the default settings. This will should create a file named id_rsa in the ~/.ssh directory created earlier.
When prompted, be sure to provide a secure SSH passphrase.
If you had to create an SSH key, you'll need to configure the ssh-agent program to use it.
ssh-add ~/.ssh/id_rsa
If you know what you are doing, you can use an existing SSH key in your ~/.ssh directory by providing the private key file to ssh-agent.
If you're still not sure what's going on, you should two files in your ~/.ssh directory that correspond to the private and public key files. Typically, the public key will be a file by the same name with a .pub extension added. An example would be a private key file named id_rsa and a public key file named id_rsa.pub.
Once you have generated an SSH key on your local machine, it's time to put the matching shared key file on the server.
ssh user#hostname 'cat >> ~/.ssh/authorized_keys' < ~/.ssh/id_rsa.pub
This will add your public key to the authorized keys on the remote server. This process can be repeated from each development machine to add as many authorized keys as necessary to the server. You'll know you did it correctly when you close your connection and reconnect without being prompted for a password.
Configuring the Remote Server Repository
The machine you intend to use as a live production server needs to have a Git repository that can write to an appropriate web-accessible directory. The Git metadata (the .git directory) does not need to be in a web-accessible location. Instead, it can be anywhere that is user-writeable by your SSH user.
Setting up a Bare Repository
In order to push files to your web server, you'll need to have a copy of your repository on your web server. You'll want to start by creating a bare repository to house your web site. The repository should be set up somewhere outside of your web root. We'll instruct Git where to put the actual files later. Once you decide on location for your repository, the following commands will create the bare repository.
mkdir mywebsite.git
cd mywebsite.git
git init --bare
A bare repository contains all of the Git metadata without any HEAD. Essentially, this means that your repository has a .git directory, but does not have any working files checked out. The next step is to create a Git hook that will check out those files any time you instruct it to.
If you wish to run git commands from the detached work tree, you'll need to set the environmental variable GIT_DIR to the path of mywebsite.git before running any commands.
Add a Post-Receive Hook
Create a file named post-receive in the hooks directory of your repository with the following contents.
#!/bin/sh
GIT_WORK_TREE=/path/to/webroot/of/mywebsite git checkout -f
Once you create your hook, go ahead and mark it as executable.
chmod +x hooks/post-receive
GIT_WORK_TREE allows you to instruct Git where the working directory should be for a repository. This allows you to keep the repository outside of the web root with a detached work tree in a web accessible location. Make sure the path you specify exists, Git will not create it for you.
Configuring the Local Development Machine
The local development machine will house the web site repository. Relevant files will be copied to the live server whenever you choose to push those changes. This means you should keep a working copy of the repository on your development machine. You could also employ the use of any centralized repository including cloud-based ones such as GitHub or BitBucket. Your workflow is entirely up to you. Since all changes are pushed from the local repository, this process is not affected by how you choose to handle your project.
Setting up the Working Repository
On your development machine, you should have a working Git repository. If not, you can create on in an existing project directory with the following commands.
git init
git add -A
git commit -m "Initial Commit"
Add a Remote Repository Pointing to the Web Server
Once you have a working repository, you'll need to add a remote pointing to the one you set up on your server.
git remote add live ssh://server1.example.com/home/user/mywebsite.git
Make sure the hostname and path you provide point to the server and repository you set up previously. Finally, it's time to push your current website to the live server for the first time.
git push live +master:refs/head/main
This command instructs Git to push the current main branch to the live remote. (There's no need to send any other branches.) In the future, the server will only check out from the main branch so you won't need to specify that explicitly every time.
Build Something Beautiful
Everything is ready to go. It's time to let the creative juices flow! Your workflow doesn't need to change at all. Whenever you are ready, pushing changes to the live web server is as simple as running the following command.
git push live
Setting receive.denycurrentbranch to "ignore" on the server eliminates a warning issued by recent versions of Git when you push an update to a checked-out branch on the server.
Additional Tips
Here are a few more tips and tricks that you may find useful when employing this style of workflow.
Pushing Changes to Multiple Servers
You may find the need to push to multiple servers. Perhaps you have multiple testing servers or your live site is mirrored across multiple servers behind a load balancer. In any case, pushing to multiple servers is as easy as adding more urls to the [remote "live"] section in .git/config.
[remote "live"]
url = ssh://server1.example.com/home/user/mywebsite.git
url = ssh://server2.example.com/home/user/mywebsite.git
Now issuing the command git push live will update all of the urls you've added at one time. Simple!
Ignoring Local Changes to Tracked Files
From time to time you'll find there are files you want to track in your repository but don't wish to have changed every time you update your website. A good example would be configuration files in your web site that have settings specific to the server the site is on. Pushing updates to your site would ordinarily overwrite these files with whatever version of the file lives on your development machine. Preventing this is easy. SSH into the remote server and navigate into the Git repository. Enter the following command, listing each file you wish to ignore.
git update-index --assume-unchanged <file...>
This instructs Git to ignore any changes to the specified files with any future checkouts. You can reverse this effect on one or more files any time you deem necessary.
git update-index --no-assume-unchanged <file...>
If you want to see a list of ignored files, that's easy too.
git ls-files -v | grep ^[a-z]
References
Deploy Your Website Changes Using Git
A simple Git deployment strategy for static sites
Using Git to manage a website
Ignoring Local Changes to Tracked Files in Git
pushing the code merely updates the remote repository's references.
It doesn't change the checked out working copy.
Consider that you could add a colleague's repository as a remote. If you pushed and the behaviour was that it would auto-checkout that new code, that would affect what they're working on.
It sounds like what you really want is a continuous integration tool, be it something full featured or merely an rsync triggered from a git hook.
you should only ever push to a bare repository (unless you know exactly what you are doing; and even then, you should only ever push to a bare repository).
you shouldn't clone a working copy's .git/ directory.
I am wondering if anyone has a better strategy for this scenario.
I am currently hosting my own remote git repo on the same box as the webserver.
All git repos are under the git user.
sudo -uwww-data -gwww-data git --git-dir=/var/www/website/.git --work-tree=/var/www/website pull
I have a cron job running as root every minute that executes this command. The git repo in the web folder is cloned from the same box to git's home dir where it's stored instead of through ssh.
So my question: Since git doesn't own the web files, it can't move the site using a git hook. I would assume I don't want git to have sudo, nor would that work via a git hook, right? Is there something that will deploy the site faster than every minute? I don't want the operation to be very expensive.
Is there some kind of daemon root could run and listen for some kind of notification? Like having it watch a file's last modified time?
Note that this article (in French, translated through Google) reports that sudo works with your approach:
change sudo to allow the gitosis user to use this command as www-data.
To do this, by running "visudo" add the line:
git ALL = (www-data) NOPASSWD: /usr/local/bin/pullhere
Then, in each repository where necessary, add the next hook in a post-receive file:
sudo -u www-data /usr/local/bin/pullhere /html/u/user/here
eg in / home/git/repositories/projet1.git/hooks/post-receive
This might interest you if you're still looking at a way to perform automatic deploys after a git push:
https://github.com/JamesBrooks/git-runner (with the git-runner-deploy gem).