bug #1) local packages
Created a test project "azure_test"
In folder "azure_test" ran the command "express --view=hbs app".
/azure_test/app as all the structure of an npm package, but is in a directory.
Essentially a local package.
in Azure_test/package.json is declare a dependency to this local package
"dependencies": {
"app": "./app"
}
a local package as per https://docs.npmjs.com/getting-started/packages
on my windows 10 machine, in folder /azure_test, I run "npm install", "npm start" and all works and expected.
save into github
In Azure, create an "App Service" and configure Azure to get source directly out of github
this fails. Log file at
https://purple01test.scm.azurewebsites.net/DebugConsole/?shell=powershell
contains the line
26 verbose git clone git://github.com/./app fatal: remote error:
If I remove all the dependencies out of /azure_test/app/package.json and copy these dependencies into /azure_test/package.json (ie: not using local package). Then all works on Azure, but I lose modularity of local packages (manually copy dependencies to the top level).
So am I using local packages incorrectly and it just happens to work in windows 10 or does Azure have a bug?
I made this test project public. It can be found at https://github.com/johngrabner/azure_test/
bug #2) private packages hosted in github
"dependencies": {
"purple_shared_enums_pic_pi": "git+https://github.com/johngrabner/purple_shared_enums_PIC_PI.git"
}
This also causes Azure to fail.
If the package hosted in github is public, Azure passed.
So bug #2 appears to be Azure fails to use the github password for dependencies.
I did some research and found that Docker / Kubernetes is probably a better choice for deployment. A number of courses and online video listed "no more, it works on my machine, but not in production".
You package your app in Docker, where you specify the exact OS, middleware, etc. Test on your machine. Then deploy this image to Azure, AWS, or Google. Kubernetes manages the spaning of these images.
Related
My node app is on the big side, so it is split into a number of npm packages. These packages are hosted in private repos on github.
Locally on my pc, I clone the repo, npm init, get prompted for password, npm run and all is good.
Now I'm trying to deploy to google cloud app engine (newbie in google cloud).
Tutorial says to add a file called app.yaml with content
env: flex
runtime: nodejs
Then I enter "gcloud app deploy". The first reported error is
npm ERR! /usr/bin/git ls-remote -h -t https://github.com/johngrabner/moch_time.git
How do I provide my password for gcloud so it can pull private packages?
Or, is "gcloud app deploy" just not mature enough?
Or, is using and storing private npm packages in github a strange way of structuring my project?
The gcloud command does not provide an option to include a password when deploying an app. If you want to specify your credentials to access your private GitHub repositories, you can follow this link.
Before starting with the steps indicated in the previous tutorial, you should generate a new SSH for GitHub authentication. The steps to do so are here.
Alternatively, you can connect your GitHub repo to a GCP repository using Source Repositories (follow this link to learn how to do it). This also allows to store your credentials for further use.
I did some research and found that Docker / Kubernetes is probably a better choice for deployment. A number of courses and online video listed "no more, it works on my machine, but not in production".
You package your app in Docker, where you specify the exact OS, middleware, etc. Test on your machine. Then deploy this image to Azure, AWS, or Google. Kubernetes manages the spaning of these images.
I'm setting up some CI builds of NodeJS projects in VS Team Services. I currently have four projects being cloned from private Github repositories, running npm install, and running unit tests. Because I'm giving the VS Team Services build an access token for the repositories it's cloning, these builds all pass.
The fifth project relies on one of the other four projects as a dependency in package.json. The build for this project fails on npm install when attempting to clone the private repository because the build does not have permission to clone the repository. How can I give this build access to clone our private repositories during npm install?
I have one build definition for each of the five projects, and each project is a separate repository in my-org on Github. In each build definition, the connection to Github is managed by personal access token, and the repository points to my-org/project-name. The package.json file of the project in the fifth, failing, build has a dependencies configuration that looks like this:
"dependencies": {
"project-name": "git+ssh://git#github.com/my-org/project-name.git#master",
"jquery": "^2.2.4",
"react": "^15.0.1",
"react-dom": "^15.0.1"
}
The first image shows the npm install step of my build definition.
The second image shows the npm test step of my build definition.
The problem is that the build agent will not able to authenticate because of lack of SSH keys on the build agent and because the host verification will fail anyway.
Instead you should create a Personal Access Token on GitHub with 'repo' only scope, then you should use it on your packages.json file (notice that ssh is replaced with https protocol):
"project-name": "git+https://<user>:<token>#github.com/my-org/project-name.git#master",
Recently i needed to do this in Azure Devops on a Windows based agent, Instead of changing dependencies to include user and token in the packages.json, it was easier to create a build step to add the authentication information from environment variables to windows credentials store and instructing git to use that.
git config --global credential.helper wincred
cmdkey /generic:LegacyGeneric:target=git:https://our-private-domain.com /user:git-readonly-use#our-private-domain.com /pass:"Personal Access Token With Read Permissions"
After adding these configurations, Npm installs that we were doing using a script in source code went through without any issue
"dependencies": {
"example-1": "git+https://our-private-domain.com/proj/example-1.git#v0.1.9",
"example-2": "git+https://our-private-domain.com/proj/example-2.git#master"
}
I am using WebDeploy to deploy a node website to azure.
I've seen in samples and demos that it should trigger a npm install on deploy.
But it is not. I've also seen almost every demo uses git deployment.
Is automatic npm install not supported for WebDeploy or am I missing something?
when you use WebDeploy, it will just copy over all the file from your machine to cloud, it will not trigger any build process. You will have to responsible to make sure your app is ready to run.
if you want CI function, please setup continues deployment, here is tutorial for setting up local git
https://azure.microsoft.com/en-us/documentation/articles/web-sites-publish-source-control/
and there are other options if you have repository in github/bitbucket/Visual studio Team Service etc ... (go to https://portal.azure.com, select your site --> all settings --> continuous deployment to see all supported optinos)
According to the doc Publish to Microsoft Azure Website using Web Deploy, it said
Deployment will include all the files in your project. Files in the node_modules folder are included automatically, even if they are not part of the project.
So all files under your project folder in VS, including the node_modules folder, will be deployed.
What would be the proper way to deploy a Go app to Azure that have private GitHub repositories as dependencies? Here's the current error from Kudu:
Resolving dependencies
# cd .; git clone https://github.com/my/privaterepo
D:\local\Temp\8d315fa89272e69\gopath\src\github.com\my\privaterepo
Cloning into 'D:\local\Temp\8d315fa89272e69\gopath\src\github.com\my\privaterepo'...
fatal: could not read Username for 'https://github.com': Bad file descriptor
package github.com/my/privaterepo/pkg1: exit status 128
package github.com/my/privaterepo/pkg2: cannot find package $GOROOT)
Building Go app to produce exe file
azureapp\file.go:8:2: cannot find package "github.com/my/privaterepo/pkg1"
in any of:
D:\Program Files\Go\1.5.1\src\github.com\my\privaterepo\pkg1 (from $GOROOT)
I was previously deploying via FTP with web.config's HttpPlatformHandler entry. But using git push is quicker especially for non-Windows team members.
Thanks
As #Not_a_Golfer and #Xiaomin said, vendoring the dependencies worked, here's what I did:
Turned the env variable on locally GO15VENDOREXPERIMENT=1
Installed godep => go get github.com/tools/godep
Making sure your app is passing a go build & go test
Ran godep save this copied all dependencies to ./vendor
On my Azure web app, I also set the environment variable GO15VENDOREXPERIMENT=1
git pushed and voila.
At first I did not set the environment variable on my Azure app, so the dependency resolver was not looking in ./vendor, turning it to 1 fixed everything.
I want to build my Node.JS application in a Azure Website.
There will be an usage of different NPM packages via my packages.json file.
My problem is that I often receive error messages which are related to missing NPM files.
Normally I put my files via FTP or edit them per VS Studio 15 Azure plugin directly on the server. This may be the reason why NPM isn't triggering as Microsoft intended it.
I would prefer a way in which I can just run commands with elevated privileges to have full control over NPM by myself.
Which ways are possible to avaid these problems?
If you're publishing your nodeJS application 'manually' via FTP there are little concerns about that.
First of All, 'manually' means manually.
Git
If you use continuous deployment via Git the final deployment step is to call npm install in your current application folder, this will install all the packages listed in package.json file.
The node_modules folder is excluded by default in .gitignore file, so all packages are downloaded by the server
Web deployment
If you're using web deployment from visual studio or command line, all the files contained by your solution are copied to Hosting environment including node_modules folder , because of this the deployment would take a long time to finish due the huge amount of dependencies and files that the folder contains.
Even worst: this scenario could take you to the same scenario you're facing right now.
FTP deployment
You're copying everything yourself. So the same thing occurs in Web Deployment is happen in FTP deployment method.
--
The thing is that when you copy all those node_modules folder contents you're assuming that those dependencies remains the same in the target enviroment, most of the cases that's true, but not always.
Some dependencies are platform dependent so maybe in you're dev environment a dependency works ok in x86 architectures but what if your target machine or website (or some mix between them) is x64 (real case I already suffer it).
Other related issues could happen. May be your direct dependencies doesn't have the problem but the linked dependencies to them could have it.
So always is strongly recommended to run npm install in your target environment and avoid to copy the dependencies directly from your dev environment.
In that way you need to copy on your target environment the folder structure excluding node_modules folder. And then when files are copied you need to run npm install on the server.
To achieve that you could go to
yoursitename.scm.azurewebsites.net
There you can goto "Debug Console" Tab, then goto this directory D:\home\site\wwwroot> and run
npm install
After that the packages and dependencies are downloaded for the server/website architecture.
Hope this helps.
Azure tweak the Kudu output settings, in local Kudu implementations looks the output is normalized.
A workaround -non perfect- could be this
npm install --dd
Or even more detailed
npm install --ddd
The most related answer from Microsoft itself is this
Using Node.js Modules with Azure applications
Regarding control via a console with elevated privileges there is the way of using the Kudu console. But the error output is quite weird. It's kind of putting blindly commands in the console without much feedback.
Maybe this is a way to go. But I didn't tried this yet.
Regarding deployment it looks like that Azure wants you to prefer Continuous Deployment.
The suggested way is this here.