When I run pm2 deploy development setup I get the following error. Which is strange because this key has no password set.
Enter passphrase for key '/Users/spongebob/.ssh/dev.pub':
Permission denied (publickey,keyboard-interactive).
setup paths failed
Deploy failed
I have created a key pair with no password, and trasnfered it to my development machine like so:
ssh-copy-id -i ~/.ssh/virtualbox_dev -o "IdentityFile /Users/spongebob/.ssh/dev" -f dev#192.168.0.17
I can log into SSH fine with no password:
ssh -o 'IdentityFile /Users/spongebob/.ssh/dev' 'dev#192.168.0.17'
Last login: Sun May 27 12:39:57 2018 from 192.168.0.11
I have added this key as a deployment key to Github as this is a private repo (and also added the virtualbox_dev key just incase). And tested whether it works ...
Hi MyUsername/blue-mask-api! You've successfully authenticated, but GitHub does not provideshell access.
And here is my deployment configuration (I have this in package.json):
"deploy": {
"development": {
"key": "/Users/spongebob/.ssh/dev.pub",
"user": "dev",
"host": "192.168.0.17",
"ref": "origin/master",
"path": "/home/dev/",
"repo": "git#github.com:MyUsername/blue-mask-api.git",
"post-deploy": "echo 999999999999",
"ssh_options": [
"StrictHostKeyChecking=no",
"PasswordAuthentication=no"
]
}
It's confusing because it's asking for a password and there isn't one set. And when I leave it blank I still get permission denied.
Enter passphrase for key '/Users/spongebob/.ssh/dev.pub':
It appears that you're trying to use the wrong key file. The file with the ".pub" extension is the public key file. The corresponding file without the ".pub" extension is the private key file. When you run an ssh client to connect to a remote server, you have to provide the private key file to the ssh client. The contents of the public key file are stored on the server (github in this case), where it's used by the ssh server software to confirm that this key can be accepted for authentication.
You ought to have a private key file with the same name as the public file, except without the ".pub" extension:
"development": {
"key": "/Users/spongebob/.ssh/dev",
^^^--note no .pub
Related
I'm really stuck here. I inherited a system which stores secrets in a Hashicorp vault, and I'm getting this error, Authentication failed: ldap operation failed: unable to retrieve user bind DN
I am not sure how to resolve this issue, and have been Googling for hours, and trying a lot of things.
I did see the post at ref. [A], but it isn't helpful.
Also the post at ref [B] gives some information about setting the binddn, but in the classic way to frustrate a new user, doesn't say where, how, or give any examples.
Hashicorp Vault v1.6.x
The vault is running on a docker container, on an AWS EC2.
... I have the .pem file, and am able to ssh into the EC2
. I am able to ssh into the docker container with root priv, like so:
... docker exec -it 123abc123abc sh
On the container, some vault commands work; e.g:
... vault version
--> Vault v1.6.0 (123asdf1234adsf1234adsf1234adsf13w4radsf1234asdff)
It is using ldap configuration
When trying to retrieve config and other info, I get this message:
... a. "* missing client token"
How to proceed?
I'm not an expert with this, and would appreciate clear, full, command-line examples.
Thanks for your help.
Sincerely,
Keith
DOCKER COMPOSE FILE
$ cat docker-compose.yml
version: '3'
services:
vault:
image: vault:1.6.0
cap_add:
- IPC_LOCK
environment:
- VAULT_ADDR=http://127.0.0.1:8200
command: vault server -config=/vault/config/config.json
ports:
- 80:8200
volumes:
- vault-data:/vault
- ./config.json:/vault/config/config.json
volumes:
vault-data:ubuntu#ip-192-0-2-1:/home/tarjan-docker
VAULT CONFIG
/vault/config # cat config.json
{
"backend": {
"file": {
"path": "/vault/data"
}
},
"listener": {
"tcp":{
"address": "0.0.0.0:8200",
"tls_disable": 1
}
},
"default_lease_ttl": "30m",
"max_lease_ttl": "30m",
"log_level": "info",
"ui": true
}
A. https://discuss.hashicorp.com/t/ldap-operation-failed-unable-to-retrieve-user-bind-dn/12926
B. https://support.hashicorp.com/hc/en-us/articles/5289574376083-Receiving-ldap-operation-failed-failed-to-bind-as-user-error-when-logging-in-via-LDAP-authentication-method
https://discuss.hashicorp.com/t/authentication-failed-ldap-operation-failed-unable-to-retrieve-user-bind-dn/50123
I'm presently hosting an Azure Scale set running Windows Server 2012 R2 that is setup with the Chef extension (Chef.Bootstrap.WindowsAzure.ChefClient). When the VM is provisioned, the extension reports back that it succeeded via the Azure portal however the registered node on the Chef server is not updated to retain the provided run list and the first run isn't fully completed. This is causing subsequent chef-client runs to be performed with an empty run list. When I observe the reports on chef server, I see a run with a status of aborted with no error.
Upon review of the WindowsAzure Plugins chef-client.log file, I can see that it tries to execute the run list but seems to be interrupted with the following FATAL
FATAL: Errno::EINVAL: Invalid argument # io_writev - <STDOUT>
There is no chef-stacktrace.out file created as well. The ARM extension definition looks like:
{
"type": "extensions",
"name": "ChefClient",
"properties": {
"publisher": "Chef.Bootstrap.WindowsAzure",
"type": "ChefClient",
"typeHandlerVersion": "1210.12",
"autoUpgradeMinorVersion": true,
"settings": {
"client_rb": "ssl_verify_mode :verify_none\nnode_name ENV[\"COMPUTERNAME\"]",
"runlist": "recipe[example]",
"autoUpdateClient": "false",
"deleteChefConfig": "false",
"bootstrap_options": {
"chef_server_url": "https://mychefserver.com/organizations/myorg",
"validation_client_name": "myorg-validator",
"environment": "dev"
}
},
"protectedSettings": {
"validation_key": "-----BEGIN RSA PRIVATE KEY----- ... -----END RSA PRIVATE KEY----"
}
}
}
In order to troubleshoot, I've tried to reduce my example cookbook down to a single DSC script which installs IIS. Even this step, I've executed it multiple ways such as using windows_feature, powershell_script, and dsc_script. All result end up with the same error. Here is the current script
powershell_script 'Install IIS' do
code 'Add-WindowsFeature Web-Server'
guard_interpreter :powershell_script
not_if "(Get-WindowsFeature -Name Web-Server).Installed"
end
If I override the run list and call chef-client manually, everything succeeds. I'm having trouble honing in on whether this is the Azure Chef Extension, the Chef client, or the cookbook.
As far as I can tell, communication with the Chef server looks good as the necessary pem files are exchanged, chef-client is installed, and the cookbook is downloaded and cached from the server. The cache gets removed on the subsequent run however with the empty run list. Here are the contents of first-boot.json:
{"run_list":["recipe[example]"]}
Here are the versions in play:
chef-client version: 14.1.12
Azure Chef Extension version: 1210.12.110.1001
Server version: Windows Server 2012 R2
Any ideas what could be going on?
It turns out my analysis was incorrect about which resource was causing the problem. It appears that the first boot run was failing when using dsc_script as the resource to install the web server. When using the following powershell_script resource, it succeeded and the run list attached for future runs.
powershell_script 'Install IIS' do
code 'Add-WindowsFeature Web-Server'
guard_interpreter :powershell_script
not_if "(Get-WindowsFeature -Name Web-Server).Installed"
end
I'm trying to configure a p-config-server instance to use GitLab.The account has a password. I have tried both the http and ssh urls with username and password for the http url and a private key for the ssh url.
The service reports that it failed to start. But I can't get any more information as to why it failed.
How do I configure this correctly to connect to a GitLab instance?
Where are the logfiles for the service failure so that I can find the issues?
Cheers
The username and password eventually worked.
I used -c with a file containing :-
{
"git": {
"uri": "http://host:port/xyz.git",
"label": "master",
"username": "demo",
"password": "demouser"
}
}
I am switching Macs and don't remember my Docker registry login. I could go in and reset it all, but I would rather copy the hashed login files to the new Mac.
Where are the docker registry credentials stored locally on OSX, Linux and Windows?
I was able to find the information I needed in ~/.docker/config.json
Copying this file from one OSX install to another let me authenticate with the registry without requesting credentials again.
The file looks as such:
{
"auths": {
"registry.domain.com": {
"auth": "HASH_HERE",
"email": "user#domain.com"
}
}
}
If you have stuck with the native docker registry V2 (publicatin) authentication, look for a auth/htpasswd (or just htpasswd) file that was used to docker run the registry.
On Windows or Mac, that file would have been created within the VM, possible under /c/Users/... or /Users/..., as those are the persistent folders mounted by the boot2docker image.
As the OP spechal comments, the login itself is saved in ~/.docker/config.json.
This issue for instance illustrates it:
WARNING: login credentials saved in /var/lib/jenkins/.docker/config.json
Login Succeeded
pushing docker image 0fd658e25fd8 from 06bdc8f-1508171450 to docker-registry.tld:5043/repo/ingest_preview:06bdc8f-1508171450
The push refers to a repository [docker-registry.tld:5043/repo/ingest_preview] (len: 1)
I am trying to deploy my app to Heroku however I rely on using some private git repos as modules. I do this for code reuse between projects, e.g. I have a custom logger I use in multiple apps.
"logger":"git+ssh://git#bitbucket.org..............#master"
The problem is Heroku obviously does not have ssh access to this code. I can't find anything on this problem. Ideally Heroku have a public key I can can just add to the modules.
Basic auth
GitHub has support for basic auth:
"dependencies" : {
"my-module" : "git+https://my_username:my_password#github.com/my_github_account/my_repo.git"
}
As does BitBucket:
"dependencies" : {
"my-module": "git+https://my_username:my_password#bitbucket.org/my_bitbucket_account/my_repo.git"
}
But having plain passwords in your package.json is probably not desired.
Personal access tokens (GitHub)
To make this answer more up-to-date, I would now suggest using a personal access token on GitHub instead of username/password combo.
You should now use:
"dependencies" : {
"my-module" : "git+https://<username>:<token>#github.com/my_github_account/my_repo.git"
}
For Github you can generate a new token here:
https://github.com/settings/tokens
App passwords (Bitbucket)
App passwords are primarily intended as a way to provide compatibility with apps that don't support two-factor authentication, and you can use them for this purpose as well. First, create an app password, then specify your dependency like this:
"dependencies" : {
"my-module": "git+https://<username>:<app-password>#bitbucket.org/my_bitbucket_account/my_repo.git"
}
[Deprecated] API key for teams (Bitbucket)
For BitBucket you can generate an API Key on the Manage Team page and then use this URL:
"dependencies" : {
"my-module" : "git+https://<teamname>:<api-key>#bitbucket.org/team_name/repo_name.git"
}
Update 2016-03-26
The method described no longer works if you are using npm3, since npm3 fetches all modules described in package.json before running the preinstall script. This has been confirmed as a bug.
The official node.js Heroku buildpack now includes heroku-prebuild and heroku-postbuild, which will be run before and after npm install respectively. You should use these scripts instead of preinstall and postinstall in all cases, to support both npm2 and npm3.
In other words, your package.json should resemble:
"scripts": {
"heroku-prebuild": "bash preinstall.sh",
"heroku-postbuild": "bash postinstall.sh"
}
I've come up with an alternative to Michael's answer, retaining the (IMO) favourable requirement of keeping your credentials out of source control, whilst not requiring a custom buildpack. This was borne out of frustration that the buildpack linked by Michael is rather out of date.
The solution is to setup and tear down the SSH environment in npm's preinstall and postinstall scripts, instead of in the buildpack.
Follow these instructions:
Create two scripts in your repo, let's call them preinstall.sh and postinstall.sh.
Make them executable (chmod +x *.sh).
Add the following to preinstall.sh:
#!/bin/bash
# Generates an SSH config file for connections if a config var exists.
if [ "$GIT_SSH_KEY" != "" ]; then
echo "Detected SSH key for git. Adding SSH config" >&1
echo "" >&1
# Ensure we have an ssh folder
if [ ! -d ~/.ssh ]; then
mkdir -p ~/.ssh
chmod 700 ~/.ssh
fi
# Load the private key into a file.
echo $GIT_SSH_KEY | base64 --decode > ~/.ssh/deploy_key
# Change the permissions on the file to
# be read-only for this user.
chmod 400 ~/.ssh/deploy_key
# Setup the ssh config file.
echo -e "Host github.com\n"\
" IdentityFile ~/.ssh/deploy_key\n"\
" IdentitiesOnly yes\n"\
" UserKnownHostsFile=/dev/null\n"\
" StrictHostKeyChecking no"\
> ~/.ssh/config
fi
Add the following to postinstall.sh:
#!/bin/bash
if [ "$GIT_SSH_KEY" != "" ]; then
echo "Cleaning up SSH config" >&1
echo "" >&1
# Now that npm has finished running, we shouldn't need the ssh key/config anymore.
# Remove the files that we created.
rm -f ~/.ssh/config
rm -f ~/.ssh/deploy_key
# Clear that sensitive key data from the environment
export GIT_SSH_KEY=0
fi
Add the following to your package.json:
"scripts": {
"preinstall": "bash preinstall.sh",
"postinstall": "bash postinstall.sh"
}
Generate a private/public key pair using ssh-agent.
Add the public key as a deploy key on Github.
Create a base64 encoded version of your private key, and set it as the Heroku config var GIT_SSH_KEY.
Commit and push your app to Github.
When Heroku builds your app, before npm installs your dependencies, the preinstall.sh script is run. This creates a private key file from the decoded contents of the GIT_SSH_KEY environment variable, and creates an SSH config file to tell SSH to use this file when connecting to github.com. (If you are connecting to Bitbucket instead, then update the Host entry in preinstall.sh to bitbucket.org). npm then installs the modules using this SSH config. After installation, the private key is removed and the config is wiped.
This allows Heroku to pull down your private modules via SSH, while keeping the private key out of the codebase. If your private key becomes compromised, since it is just one half of a deploy key, you can revoke the public key in GitHub and regenerate the keypair.
As an aside, since GitHub deploy keys have read/write permissions, if you are hosting the module in a GitHub organization, you can instead create a read-only team and assign a 'deploy' user to it. The deploy user can then be configured with the public half of the keypair. This adds an extra layer of security to your module.
It's a REALLY bad idea to have plain text passwords in your git repo, using an access token is better, but you will still want to be super careful.
"my_module": "git+https://ACCESS_TOKEN:x-oauth-basic#github.com/me/my_module.git"
I created a custom nodeJS buildpack that will allow you to specify an SSH key that is registered with ssh-agent and used by npm when dynos are first setup. It seamlessly allows you to specify your module as an ssh url in your package.json like shown:
"private_module": "git+ssh://git#github.com:me/my_module.git"
To setup your app to use your private key:
Generate a key: ssh-keygen -t rsa -C "your_email#example.com" (Enter no passphrase. The buildpack does not support keys with passphrases)
Add the public key to github: pbcopy < ~/.ssh/id_rsa.pub (in OS X) and paste the results into the github admin
Add the private key to your heroku app's config: cat id_rsa | base64 | pbcopy, then heroku config:set GIT_SSH_KEY=<paste_here> --app your-app-name
Setup your app to use the buildpack as described in the heroku nodeJS buildpack README included in the project. In summary the simplest way is to set a special config value with heroku config:set to the github url of the repository containing the desired buildpack. I'd recommend forking my version and linking to your own github fork, as I'm not promising to not change my buildpack.
My custom buildpack can be found here: https://github.com/thirdiron/heroku-buildpack-nodejs and it works for my system. Comments and pull requests are more than welcome.
Based on the answer from #fiznool I created a buildpack to solve this problem using a custom ssh key stored as an environment variable. As the buildpack is technology agnostic, it can be used to download dependencies using any tool like composer for php, bundler for ruby, npm for javascript, etc: https://github.com/simon0191/custom-ssh-key-buildpack
Add the buildpack to your app:
$ heroku buildpacks:add --index 1 https://github.com/simon0191/custom-ssh-key-buildpack
Generate a new SSH key without passphrase (lets say you named it deploy_key)
Add the public key to your private repository account. For example:
Github: https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/
Bitbucket: https://confluence.atlassian.com/bitbucket/add-an-ssh-key-to-an-account-302811853.html
Encode the private key as a base64 string and add it as the CUSTOM_SSH_KEY environment variable of the heroku app.
Make a comma separated list of the hosts for which the ssh key should be used and add it as the CUSTOM_SSH_KEY_HOSTS environment variable of the heroku app.
# MacOS
$ heroku config:set CUSTOM_SSH_KEY=$(base64 --input ~/.ssh/deploy_key) CUSTOM_SSH_KEY_HOSTS=bitbucket.org,github.com
# Ubuntu
$ heroku config:set CUSTOM_SSH_KEY=$(base64 ~/.ssh/deploy_key) CUSTOM_SSH_KEY_HOSTS=bitbucket.org,github.com
Deploy your app and enjoy :)
I was able to setup resolving of Github private repositories in Heroku build via Personal access tokens.
Generate Github access token here: https://github.com/settings/tokens
Set access token as Heroku config var: heroku config:set GITHUB_TOKEN=<paste_here> --app your-app-name or via Heroku Dashboard
Add heroku-prebuild.sh script:
#!/bin/bash
if [ "$GITHUB_TOKEN" != "" ]; then
echo "Detected GITHUB_TOKEN. Setting git config to use the security token" >&1
git config --global url."https://${GITHUB_TOKEN}#github.com/".insteadOf git#github.com:
fi
add the prebuild script to package.json:
"scripts": {
"heroku-prebuild": "bash heroku-prebuild.sh"
}
For local environment we can also use git config ... or we can add the access token to ~/.netrc file:
machine github.com
login PASTE_GITHUB_USERNAME_HERE
password PASTE_GITHUB_TOKEN_HERE
and installing private github repos should work.
npm install OWNER/REPO --save will appear in package.json as: "REPO": "github:OWNER/REPO"
and resolving private repos in Heroku build should also work.
optionally you can setup a postbuild script to unset the GITHUB_TOKEN.
This answer is good https://stackoverflow.com/a/29677091/6135922, but I changed a little bit preinstall script. Hope this will help someone.
#!/bin/bash
# Generates an SSH config file for connections if a config var exists.
echo "Preinstall"
if [ "$GIT_SSH_KEY" != "" ]; then
echo "Detected SSH key for git. Adding SSH config" >&1
echo "" >&1
# Ensure we have an ssh folder
if [ ! -d ~/.ssh ]; then
mkdir -p ~/.ssh
chmod 700 ~/.ssh
fi
# Load the private key into a file.
echo $GIT_SSH_KEY | base64 --decode > ~/.ssh/deploy_key
# Change the permissions on the file to
# be read-only for this user.
chmod o-w ~/
chmod 700 ~/.ssh
chmod 600 ~/.ssh/deploy_key
# Setup the ssh config file.
echo -e "Host bitbucket.org\n"\
" IdentityFile ~/.ssh/deploy_key\n"\
" HostName bitbucket.org\n" \
" IdentitiesOnly yes\n"\
" UserKnownHostsFile=/dev/null\n"\
" StrictHostKeyChecking no"\
> ~/.ssh/config
echo "eval `ssh-agent -s`"
eval `ssh-agent -s`
echo "ssh-add -l"
ssh-add -l
echo "ssh-add ~/.ssh/deploy_key"
ssh-add ~/.ssh/deploy_key
# uncomment to check that everything works just fine
# ssh -v git#bitbucket.org
fi
You can use in package.json private repository with authentication example below:
https://usernamegit:passwordgit#github.com/reponame/web/tarball/branchname
In short it is not possible. The best solution to this problem I came up with is to use the new git subtree's. At the time of writing they are not in the official git source and so needs to be installed manual but they will be included in v1.7.11. At the moment it is available on homebrew and apt-get. it is then a case of doing
git subtree add -P /node_modules/someprivatemodue git#github.......someprivatemodule {master|tag|commit}
this bulks out the repo size but an update is easy by doing the command above with gitsubtree pull.
I have done this before with modules from github. Npm currently accepts the name of the package or a link to a tar.gz file which contains the package.
For example if you want to use express.js directly from Github (grab the link via the download section) you could do:
"dependencies" : {
"express" : "https://github.com/visionmedia/express/tarball/2.5.9"
}
So you need to find a way to access you repository as a tar.gz file via http(s).