I am attempting to write a generic command-runner in Node.JS - however that's not massively important.
My setup is as follows:
I have a list of string commands that are executed using child_process.exec one after the other.
I want to run git from one of these commands, specifically a pull.
The location I am pulling from requires SSH authentication. HTTPS is not an option.
My private key is passphrased.
I am currently using keychain to manage ssh-agent.
When running git pull from the command line, it succeeds. When running my application as the logged-in user, it succeeds. However, when running my application using forever, it fails.
The error I receive is Permission denied (publickey).. I have tried calling keychain as part of my command, but I cannot get it to recognise the credentials.
How can I fix this?
My mistake was taking the contents of .bash_profile and using that to set up keychain from my exec.
What I needed to do was:
. $HOME/.keychain/$HOSTNAME-sh; git pull
I found this out by looking up examples of how to use keychain with bash scripts.
Related
I want to use gitlabs CI/CD to deploy my app on a external server. i have the IP, username and password, and i understand i need to connect through SSH. How can i runn all the nessesary commands on the server side. Server runs on linux.
Currently i just get the code from reposiroty and to the npm build:prod and npm serve:prod for the API and npm start for the UI. How can i do the same chain of cammands with gitlab CI/CD? Or is this even possible? I basically want it to run similarily like jenkins works. But since the code is already on gitlab, it might be simplerer to let gitlab to handle this process instead of installing and setting up jenkins.
To be able to SSH to your machine from within GitLab CI, you probably should setup ssh key authentication, since you can't just type in the password inside the CI.
When you've got that set up, you have to store the private key in an environment variable so you can use it in the CI job. How to do that can be found here.
The last part is actually executing commands over ssh. That can be done in the following way:
ssh <host> '
command1;
command2;
'
I have a Git repository on Bitbucket named "foo-apps". I have a Linux web server with a local clone of this repository, and I want this server's associated Apache web page to do some Git commands with this repository, such as git pull and git checkout, via a Bash script. The problem is, only the user "foo" has permission to associate with the "foo-apps" repository, and the web page runs as the Apache user, "www-data".
It seems that www-data can do git log and some other commands on the local repository, but not the git pull or the git checkout command. (Just so you know my system: I have an HTML file that contains JavaScript, which contains an AJAX request, which calls a PHP file, which calls my Bash script, which has the Git commands in it.)
What are some ways that I can successfully get those Git commands to work when the process is triggered by the web interface? I am not opposed to any working suggestions, even if they include a complete overhaul of my system... however, I would like to have the simplest effective solution with what I've already got.
Here are some ideas I've thought of and tried out a bit. None of them seem to work, but keep in mind that I've only "half tried" them as I didn't have confidence I was using preferred methods:
Giving www-data permission on my Bitbucket repository
Giving Apache access to foo's ssh keys
Somehow switching to user foo in the script, like with sudo, su, etc.. (I think this type of thinking is more along the lines of what I want... I don't have a lot of control over the settings of the Bitbucket repository. I am fine putting a password in a script, too.)
This web server is on a closed network, and security is not a very high concern for me.
I don't know if it's useful, but here are some of the main Git related errors I've received when trying these methods:
error: cannot open .git/FETCH_HEAD: Permission denied
fatal: BUG: get_tempfile_fd() called for inactive object
/usr/bin/git: /usr/bin/git: cannot execute binary file
I found the answer on this page (thanks odyniec).
I had to add this line to the /etc/sudoers file:
www-data ALL=(foo) NOPASSWD: /var/www/html/my_bash_script.sh
This let Apache have the permission to run that specific script I wanted. And then from my PHP file, instead of running
shell_exec("/var/www/html/my_bash_script.sh");
I had to run
shell_exec("sudo -u foo /var/www/html/my_bash_script.sh");
This answer seems secure and simple.
I inherited a deployment system that is currently broken and I'm at a loss at how to fix it.
The basic setup is adding 3 keys to ssh-agent and pulling a few private repos from Github via the Go deployment software from ThoughtWorks.
I seem to need to have one ssh-agent running that can be accessed by multiple user accounts.
I've started a ssh-agent and added the keys to it and then I was able to clone private repos from the command line without issue, but when the main application( which is using the same user account ) tries to clone it fails with a permission denied error.
My guess is that the ssh-agent that is holding the keys is not accessible to the application for some reason.
Here are the instructions that I have:
export SSH_AUTH_SOCK=/var/go/ssh-agent.sock
ssh-add ~/.ssh/go_deploy_id_rsa
ssh-add ~/.ssh/go_id_rsa
ssh-add ~/.ssh/deploy_id_rsa
When I set the SSH_AUTH_SOCK environment variable it seems to kill any ssh-agent that is/was running and when I issue the ssh-add command I get the classic:
"Could not open a connection to your authentication agent."
So basically how do I start ssh-agent AND have it use the SSH_AUTH_SOCK I defined earlier and stay running so that the Go application uses it when it communicates with Github?
This use to work so I know that the setup is technically valid.
SOLVED: It turns out the ssh-agent socket that I was using was stale. Deleting the socket and re-creating it allowed the keys to be added and communication worked again.
I can successfully execute git pull in linux command line on my VPS, but when I execute a bash file containing "git pull" with execFile in Nodejs, it gave me an error: Command failed: Host key verification failed. How can I solve this problem?
Update:
The whole error message I get is:
{ [Error: Command failed: Host key verification failed. fatal: Could not read
from remote repository. Please make sure you have the correct access rights
and the repository exists. ] killed: false, code: 1, signal: null }
It seems that it's not the same problem with the question dylants provided.
The bash file script is like this, I use it to auto deploy my nodejs app:
git pull && pm2 reload www
I am using ssh protocol instead of https protocol on my vps in order to prevent the password prompt each time I git pull from my bitbucket repository. So ssh keys were generated in my user directory ~/.ssh/. I think the reason why nodejs failed to execute the bash file is this: The user who run the bash file in nodejs app is different from the user who run the bash file in command line. so the user running nodejs can't use the ssh keys located in ~/.ssh for verification.
Is that right? How to fix it?
I think you have correctly identified the problem: the nodejs application does not have access to your ssh credentials. You have a few options available:
If you can make the repository available for anonymous read-only access via http:// or git:// protocols, you can have the nodejs pull changes without requiring any sort of credentials.
You can generate an ssh key for the nodejs user and grant that user read-only access to the repository. You would just need to generate an ssh keypair in the appropriate location for that user.
You could drop your own credentials where your nodejs app can make use of them, but this has a number of security problems -- if your webapp is compromised, the attacker can write changes to your repository that will appear to come from you. So don't use this option.
Background
My question seems simple, but it gets more complex really fast.
Basically, I got really tired of maintaining my servers manually (screams in background) and I decided it was time to find a way to make being a server admin much more liveable. That's when I found Ansible. Great huh? Sure beats making bash scripts (louder scream) for everything I wanted to automate.
What's the problem?
I'm having a lot of trouble figuring out what user my Ansible playbook will run certain things as. I also need the ability to specify what user certain tasks will run as. Here are some specific use cases:
Cloning a repo as another user:
My purpose with this is it run my node.js webapp from another user, who we'll call bill (that can only use sudo to run a script that I made that starts the node server, as opposed to root or my user that can use sudo for all commands). To do this, I need the ability to have Ansible's git module clone my git repo as bill. How would I do that?
Knowing how Ansible will gain root:
As far as I understand, you can set what user Ansible will connect to the server you're maintaining by defining 'user' and the beginning of the playbook file. Here's what I don't understand: if I tell it to connect via my username, joe, and ask it to update a package via the apt module, how will it gain root? Sudo usually prompts me for my password, and I'd prefer keeping it that way (for security).
Final request
I've scoured the Ansible docs, done some (what I thought was thorough) Googling, and generally just tried to figure it out on my own, but this information continues to elude me.
I am very new to Ansible, and while it's mostly straight-forwards, I would benefit greatly if I could understand exactly how Ansible runs, on which users it runs, and how/where I can specify what user to use at different times.
Thank you tons in advance
You may find it useful to read the Hosts and Users section on Ansible's documentation site:
http://docs.ansible.com/playbooks_intro.html#hosts-and-users
In summary, ansible will run all commands in a playbook as the user specified in the remote_user variable (assuming you're using ansible >= 1.4, user before that). You can specify this variable on a per-task basis as well, in case a task needs to run as a certain user.
Use sudo: true in any playbook/task to use sudo to run it. Use the sudo_user variable to specify a user to sudo to if you don't want to use root.
In practice, I've found it easiest to run my playbook as a deploy user that has sudo privileges. I set up my SSH keys so I can SSH into any host as deploy without using a password. This means that I can run my playbook without using a password and even use sudo if I need to.
I use this same user to do things like cloning git repos and starting/stopping services. If a service needs to run as a lower-privileged user, I let the init script take care of that. A quick Google search for a node.js init.d script revealed this one for CentOS:
https://gist.github.com/nariyu/1211413
Doing things this way helps to keep it simple, which I like.
Hope that helps.
My 2 cents:
Ansible uses your local user (eg Mike) to ssh to the remote machine. (That required Mike to be able to ssh to the machine)
From there it can change to a remote user if needed
It can also sudo if needed and if Mike is allowed. If no user is specified then root will be selected via your ~/.ansible.cfg on your local machine.
If you supply a remote_user with the sudo param then like no.3 it will not use root but that user.
You can specify different situations and different users or sudo via the playbooks.
Playbook's define which roles will be run into each machine that belongs to the inventory selected.
I suggest you read Ansible best practices for some explanation on how to setup your infrastructure.
Oh and btw since you are not referring to a specific module that ansible uses and your question is not related to python, then I don't find any use your question having the python tag.
Just a note that Ansible>=1.9 uses privilege escalation commands so you can execute tasks and create resources as that secondary user if need be:
- name: Install software
shell: "curl -s get.dangerous_software.install | sudo bash"
become_user: root
https://ansible-docs.readthedocs.io/zh/stable-2.0/rst/become.html
I notice current answers are a bit old and suffering from link rot.
Ansible will SSH as your current user, by default:
https://docs.ansible.com/ansible/latest/user_guide/intro_getting_started.html#connecting-to-remote-nodes
Ansible communicates with remote machines over the SSH protocol. By default, Ansible uses native OpenSSH and connects to remote machines using your current user name, just as SSH does.
This can be overridden using:
passing the -u parameter at the command line
setting user information in your inventory file
setting user information in your configuration file
setting environment variables
But then you must ensure a route exists to SSH as that user. An approach to maintaining user-level ownership I see more often is become (root) and then to chown -R jdoe:jdoe /the/file/path.
In my 2.12 release of ansible I found the only way I could change the user was by specifying become: yes as an option at the play level. That way I am SSHing as the unprivileged, default, user. This user must have passwordless sudo enabled on the remote and is about the safest I could make my VPS. From this I could then switch to another user, with become_user, from an arbitrary command task.
Like this:
- name: Getting Started
gather_facts: false
hosts: all
become: yes # All tasks that follow will become root.
tasks:
- name: get the username running the deploy
command: echo $USER
become_user: trubuntu # From root we can switch to trubuntu.
If the user permitted SSH access to your remote is, say, victor, and not your current user, then remote_user: victor has a place at the play level, adjacent to become: yes.