Which .ssh/known_hosts file is used by npm in Azure? - azure

Question

Running a npm install git+ssh://<git repo url> from the Kudu's console in an Azure App Service instance fails with following error:
npm ERR! Host key verification failed.
npm ERR! fatal: Could not read from remote repository.
People often solve this error by adding the host's key to the .ssh/known_hosts file. The problem is that the correct key is already there. If it wasn't there, git clone <git repo url> would fail with the same error, but it does not. It successfully clones the repository.
In order to debug the issue, I tried to set the SSH's log level to DEBUG3 via the ~/.ssh/config file but the output did not change (while with git clone, it prints the debug info).
So, because of that I suspect that the problem is that the SSH client used by npm in Azure does not take the ~/.ssh directory into account.
My question is, is this documented somewhere or is it bug? Do you have any idea in which component the bug is?
FTR, the full output is:
npm ERR! git clone --template=D:\local\AppData\npm-cache\_git-remotes\_templates --mirror <git repo url> D:\local\AppData\npm-cache\_git-remotes\git-ssh-<git repo url>-dc8c35134031285cb7109c3e32618e85: Cloning into bare repository 'D:\local\AppData\npm-cache\_git-remotes\git-ssh-<git repo url>-dc8c35134031285cb7109c3e32618e85'...
npm ERR! git clone --template=D:\local\AppData\npm-cache\_git-remotes\_templates --mirror <git repo url> D:\local\AppData\npm-cache\_git-remotes\git-ssh-<git repo url>-dc8c35134031285cb7109c3e32618e85:
npm ERR! git clone --template=D:\local\AppData\npm-cache\_git-remotes\_templates --mirror <git repo url> D:\local\AppData\npm-cache\_git-remotes\git-ssh-<git repo url>-dc8c35134031285cb7109c3e32618e85: Host key verification failed.
npm ERR! git clone --template=D:\local\AppData\npm-cache\_git-remotes\_templates --mirror <git repo url> D:\local\AppData\npm-cache\_git-remotes\git-ssh-<git repo url>-dc8c35134031285cb7109c3e32618e85: fatal: Could not read from remote repository.
npm ERR! git clone --template=D:\local\AppData\npm-cache\_git-remotes\_templates --mirror <git repo url> D:\local\AppData\npm-cache\_git-remotes\git-ssh-<git repo url>-dc8c35134031285cb7109c3e32618e85:
npm ERR! git clone --template=D:\local\AppData\npm-cache\_git-remotes\_templates --mirror <git repo url> D:\local\AppData\npm-cache\_git-remotes\git-ssh-<git repo url>-dc8c35134031285cb7109c3e32618e85: Please make sure you have the correct access rights
npm ERR! git clone --template=D:\local\AppData\npm-cache\_git-remotes\_templates --mirror <git repo url> D:\local\AppData\npm-cache\_git-remotes\git-ssh-<git repo url>-dc8c35134031285cb7109c3e32618e85: and the repository exists.
npm ERR! Windows_NT 6.2.9200
npm ERR! argv "D:\\Program Files (x86)\\nodejs\\4.2.3\\node.exe" "D:\\Program Files (x86)\\npm\\3.5.1\\node_modules\\npm\\bin\\npm-cli.js" "install" "git+ssh://<git repo url>"
npm ERR! node v4.2.3
npm ERR! npm v3.5.1
npm ERR! code 128
npm ERR! Command failed: git -c core.longpaths=true clone --template=D:\local\AppData\npm-cache\_git-remotes\_templates --mirror <git repo url> D:\local\AppData\npm-cache\_git-remotes\git-ssh-<git repo url>-dc8c35134031285cb7109c3e32618e85
npm ERR! Host key verification failed.
npm ERR! fatal: Could not read from remote repository.
npm ERR!
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR!
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! <https://github.com/npm/npm/issues>
npm ERR! Please include the following file with any support request:
npm ERR! D:\home\site\foo\npm-debug.log

Answer 1

Here's how SSH settings (config file, known hosts, public / private keys) seem to be handled on Azure App Service:
The default folder for SSH configuration is D:\home\.ssh. This is what SSH and Git in the Kudu Debug Console use.
When you trigger host key generation through https://your-site.scm.azurewebsites.net/sshkey?ensurePublicKey=1, it'll create the D:\home\.ssh folder and place a config file with StrictHostKeyChecking no into it (as well as the newly generated SSH key). This means that SSH run through the Debug Console will now auto-accept hostkeys.
npm on Azure App Service expects the SSH configuration in %USERPROFILE%. You can see that when you run npm in the Debug Console for the first time, it will create the empty folder %USERPROFILE%\.ssh.
For npm compatibility, Kudu's deployment script will copy the D:\home\.ssh folder to %USERPROFILE% (see issue / fix). This should be happening every time you deploy through the web app's local Git repository, through Github or any of the other deployment options that trigger Kudu.
In my experience, that copied .ssh folder in %USERPROFILE% will be removed again when scaling and when restarting the app.
What I think happened in my case was that while debugging a different npm/git/ssh issue, I restarted the web app. I then ended up with the host key verification issue when running npm install manually in the Debug Console.

Answer 2

So for me the problem was that my app service was deployed through Github and hence had a deployment key associated with it's repo on Github.
To be able to access my other private repos I needed to do the following.
Remove the deployment key.
Get the public key from https://[your-web-app].scm.azurewebsites.net/api/sshkey?ensurePublicKey=1
Add the key to my SSH keys in github.
After this npm would happily install my private repos.