Azure DevOps - how to build bundles using webpack during deployment - azure

I have been trying to deploy an web app to local IIS by using Azure DevOps Repos and Pipelines.
I would like to build bundles using webpack on Azure DevOps service rather than uploading already bundled javascript, style files to Azure Repos.
FYI,
App uses ASP.NET Core 2.2, webpack 4.30, webpack-cli 3.3.2, and configs with webpack.config.js (contains bunch of settings to create bundles in 'dist' folder)
and javascript, style files are in wwwroot folder.
So far it works fine if I upload to Repos with already bundled files in "dist" folder.
But I would like to upload only source files and let Azure DevOps Repos or Pipelines to build bundles and create artifacts with it.
So far, I have been trying to use azure-pipelines.yml
- task: NodeTool#0
inputs:
versionSpec: '12.x'
- task: CmdLine#2
inputs:
script: |
cd (folder)
cd (other folder-where package.json is located)
npm install -g webpack webpack-cli --save-dev
npm install
npx webpack --config webpack.config.js
or
- task: CmdLine#2
inputs:
script: |
cd (folder)
cd (folder)
npm install webpack webpack-cli --save-dev
webpack
npm run build (which is just 'webpack')
both don't work. It seems npm install works but it doesn't even run the webpack part.
because I use 'webpack-messages' on webpack.config.js file so I am expecting some messages such as "Building something bundle..." but it does not show any message regarding webpack process.
Below messages are from a build log.
Starting: Webpack install and build
==============================================================================
Task : Command line
Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version : 2.163.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
========================== Starting Command Output ===========================
"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "d:\a\_temp\fc9874eb-1e72-4a4f-82af-2d3b62451eb7.cmd""
npm WARN rollback Rolling back readable-stream#2.3.6 failed (this is probably harmless): EPERM: operation not permitted, scandir 'D:\a\1\s\something\something\node_modules\fsevents\node_modules'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents#1.2.11 (node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents#1.2.11: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
+ webpack-cli#3.3.11
+ webpack#4.41.6
added 388 packages from 217 contributors and audited 5566 packages in 31.587s
3 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
I tried also on release pipeline.
I added tasks such as 'npm install', 'npm custom', or 'command line' or 'webpack' extension from Dealogic, but none of them works since I can't specify where package.json is located on the Azure DevOps Cloud machine.
Please let me know if it is possible, and if so, how to do it.

According to Microsoft's documentation, it seems that a "webpack" command in the script section of your YAML pipeline should do the trick:
- script: webpack
The condition is: "To have this work, make sure that webpack is configured as a development dependency in your package.json project file. This will run webpack with the default configuration unless you have a webpack.config.js file in the root folder of your project."
They also mention you need to do this after your compilation and tests. There's a suggestion that you can move this logic out of the pipeline and into your code by using script objects in package.json:
- script: npm run build
Source: Microsoft docs
EDIT: The way I've done it is the with the 2nd recommendation. I have this in my MVC ASP.NET project file:
<Target Name="AfterBuild">
<Exec Condition="$(Configuration) == 'Debug'" Command="npm run build-dev" />
<Exec Condition="$(Configuration) == 'Release'" Command="npm run build-prod"/>
</Target>
and build-prod for example is just a script in the webpack.config.js:
"build-prod": "SET NODE_ENV=production && webpack -p --color"
build-dev has the -d argument instead but that's all...

Related

Getting error while running the Jenkins job for Cypress automation test : The cypress npm package is installed, but the Cypress binary is missing

I am getting the below error while running the jenkins job which i have integrated for cypress test cases.
Running as SYSTEM
Building in workspace C:\Users\Gaurav\CypressAutomation
[CypressAutomation] $ cmd /c call C:\Windows\TEMP\jenkins16040665516745242535.bat
C:\Users\Gaurav\CypressAutomation>npm install && npm run "test"
up to date, audited 651 packages in 5s
65 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
> cypressautomation#1.0.0 test
> node_modules\.bin\cypress run
The cypress npm package is installed, but the Cypress binary is missing.
We expected the binary to be installed here: C:\Windows\system32\config\systemprofile\AppData\Local\Cypress\Cache\11.2.0\Cypress\Cypress.exe
Reasons it may be missing:
- You're caching 'node_modules' but are not caching this path: C:\Windows\system32\config\systemprofile\AppData\Local\Cypress\Cache
- You ran 'npm install' at an earlier build step but did not persist: C:\Windows\system32\config\systemprofile\AppData\Local\Cypress\Cache
Properly caching the binary will fix this error and avoid downloading and unzipping Cypress.
Alternatively, you can run 'cypress install' to download the binary again.
https://on.cypress.io/not-installed-ci-error
----------
Platform: win32-x64 (10.0.19043)
Cypress Version: 11.2.0
Build step 'Execute Windows batch command' marked build as failure
[htmlpublisher] Archiving HTML reports...
[htmlpublisher] Archiving at BUILD level C:\Users\Gaurav\CypressAutomation\cypress\reports\html to C:\Users\Gaurav\.jenkins\jobs\CypressAutomation\builds\18\htmlreports\Cypress_20Report
ERROR: Specified HTML directory 'C:\Users\Gaurav\CypressAutomation\cypress\reports\html' does not exist.
Finished: FAILURE
Please find the configuartion
for CI you can use the npm ci command which designed for this purpose.
Here the doc: https://docs.npmjs.com/cli/v9/commands/npm-ci

Define custom npm install procedure for azure/webapps-deploy#v2

My project currently contains 2 apps. The first is the application Backend (NestJS) and the second is the client (VueJS).
The current folder structure follows:
Root (NestJS)
./client/ (VueJS)
When I am deploying my app to Azure App Service I am using the azure/webapps-deploy#v2 action. It's procedure is to run npm install in the root of the project but I need it to also run in the sub project containing the client packages. How can this be done? Are there any arguments to provide the webapps deploy action to include that addition npm install command?
You could try add the commands directly in your workflow:
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
npm run test --if-present
cd ./client/mern_azure_example # go to your client package
npm install # run npm install under your client package
But it takes very long time.

npm install - jest doesn't exist in Azure Pipeline

Apologies in advance that I can't share source for this one:
I've got a client that has to use Azure DevOps Pipelines to build a github enterprise hosted project.
It is a perfectly regular node.js project with jest specified as a devDependency in package.json.
When the npm install runs on an Azure Pipeline, jest doesn't get installed. I created a local x64 linux agent on Ubuntu 18 on my desktop, and it doesn't get installed their either but when I manually run npm install inside the /s/ directory it's all okay.
What is Azure Devops doing to the script that this is the result?
What is Azure Devops doing to the script that this is the result?
Test to run Npm install in Azure Pipeline and local machine, it seems that they have the same behavior. The Jest will be installed in node_modules folder.
Here are my steps, you could refer to it.
The File structure. I also add the jest to devDependency in package.json.
"devDependencies": {
"enzyme": "^3.3.0",
"enzyme-adapter-react-16": "^1.1.1",
"enzyme-to-json": "^3.3.3",
"jest": "^22.4.3"
}
Use the Npm install task in Azure Pipeline.
- task: Npm#1
displayName: 'npm install'
inputs:
workingDir: package
verbose: false
Run the pipeline and the Jest will be installed in the node_modules folder.
By the way, I test the same steps on the Microsoft-hosted agents: ubuntu-18.04 and it could work fine.
Updates
For running Npm install with the Script:
Here is an example:
steps:
- script: |
cd $(build.sourcesdirectory)/package
npm install
displayName: 'Command Line Script'
The first step is used to navigate to the source folder(contains package.json file).
Then the jest will be installed.

Azure Devops: Cannot Build an Image using NPM private registry even after setting NPM Authenticate

I am always getting an error on npm install after setting NPM Authenticate. I would like to authenticate to my npm private registry during image build and install all the dependencies I need. Maybe I misunderstood how this authentication process works but this is what I am doing:
Build pipeline
I tried establishing a service connection from the project settings page as in Service connections for builds and releases
After that, I also set up my NPM Authentication task following the steps in With a Task Runner (e.g. make gulp work)
But this is not working. These are the errors I am getting:
During 'NPM Authenticate' phase:
[warning]Found and overrode credentials for the
myregistry.pkgs.visualstudio.com registry in the selected .npmrc file.
Remove credentials from the file and store them in an npm service
connection instead (recommended), or remove the npm Authenticate task
from your build to use credentials checked into an .npmrc.
During 'Build an Image' phase:
Step 4/7 : RUN npm install --production ---> Running in 8724f713f1db
[91mnpm ERR! code[0m[91m E404 [0m[91mnpm [0m[91mERR! 404[0m[91m Not
Found: #myregistry/service-logging#latest npm ERR![0m[91m A complete
log of this run can be found in: npm ERR!
/root/.npm/_logs/2018-09-11T04_20_00_513Z-debug.log [0mThe command
'/bin/sh -c npm install --production' returned a non-zero code: 1
[error]The command '/bin/sh -c npm install --production' returned a non-zero code: 1 [error]/usr/local/bin/docker failed with return
code: 1 [section]Finishing: Build an image
This is my .npmrc file:
unsafe-perm=true
package-lock=false
registry=https://myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/registry/
always-auth=true
//myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/registry/:_authToken=${NPM_TOKEN}
//myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/:_authToken=${NPM_TOKEN}
And this is my Dockerfile:
FROM node:8.9-alpine
ARG NPM_TOKEN
WORKDIR /usr/src/srv/
COPY package.json package.json
COPY .npmrc .npmrc
RUN npm install --production
RUN rm -f .npmrc
COPY . .
EXPOSE 8080
CMD npm start
Any help to unblock me from this issue will be highly appreciated! Thanks!
I finally resolved this issue in my pipeline by removing the last two lines in my .npmrc file. the last line was causing an issue. After the NPM Authenticate task, my .npmrc file was being modified to:
unsafe-perm=true
package-lock=false
registry=https://myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/registry/
always-auth=true
//myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/:_authToken=${NPM_TOKEN}
//myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/registry/:username=VssToken
//myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/registry/:_password=***
//myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/registry/:email=VssEmail
//myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/registry/:always-auth=true
Somehow, the following config was being taken into consideration and the config the NPM Authenticate inserted was being ignored, causing the pipeline error:
//myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/:_authToken=${NPM_TOKEN}
Also, no need to include the following line since NPM Authenticate will do the job for you:
//myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/registry/:_authToken=${NPM_TOKEN}
By removing the line above, this warning disappeared:
[warning]Found and overrode credentials for the
myregistry.pkgs.visualstudio.com registry in the selected .npmrc file.
Remove credentials from the file and store them in an npm service
connection instead (recommended), or remove the npm Authenticate task
from your build to use credentials checked into an .npmrc.
So, to conclude, just keep your .npmrc file as simple as this:
unsafe-perm=true
package-lock=false
registry=https://myregistry.pkgs.visualstudio.com/_packaging/myregistry/npm/registry/
always-auth=true
Everything was fine with the Dockerfile.
I encountered this recently, Posting what I found out here in case it helps folks (or myself) in the future.
I was trying to authenticate against a private NPM feed during an Azure DevOps Pipeline build using the NpmAuthenticate task runner.
My initial pipeline looked like this:
- task: npmAuthenticate#0
displayName: Authenticate Npm Feed
inputs:
workingFile: './source/WebApplication/.npmrc'
customEndpoint: ExternalFeedServiceConnection
- task: Npm#1
inputs:
command: 'install'
workingDir: ./source/WebApplication #path to package.json
customRegistry: 'useNpmrc'
displayName: Install NPM Packages
The Npm#1 task would continually fail with a login error. No matter what permutation was attempted. What finally worked was replacing it with a script step instead:
- script: 'npm install'
workingDirectory: ./source/WebApplication
displayName: npm install
The script step appears to be executing the exact same command as the Npm#1 task, but is able to authenticate fine.
The npmAuthenticate azure devops task documentation vaguely suggests this, but it's unclear why it would work with a script step but not the Npm#1 task.
You need to include the token in .npmrc file or update this file before run npm install command.

Google App Engine Standard Node JS how to run build script?

Does GAE standard for Node support a way to have build scripts? I tried using postinstall within package.json but that did not work.
My codebase has subdirectories with package.json within the subdirectories. In my root package.json there is
scripts: {
postinstall: cd vendor && npm install
....
}
However I'm not seeing any vendor packages installed so I'm inclined to believe the postinstall does not get triggered on GAE Node standard.
Is there any way for me to install subdirectory dependencies without having to copy and paste all my vendor/package.json dependencies to the root?
Note: I've also tried putting an "install" within the package.json scripts but that didn't seem to get triggered either.
In GAE standard, installation of dependencies are automatically managed. You should add them in your package.json.
As Google documentation mentioned :
When you deploy your app, the Node.js runtime automatically installs all dependencies declared in your package.json file using the npm install command.
{
"dependencies": {
"lodash": "^4.0.1"
}
}
Installation will be done during app deployment via :
gcloud app deploy
To add a build step, run the following:
gcloud beta app gen-config --custom
This will generate the default dockerfile and config that is run. In your .dockerfile, add your build step:
RUN npm run build --unsafe-perm || \
((if [ -f npm-debug.log ]; then \
cat npm-debug.log; \
fi) && false)
"prestart": "if [ ! -d build ]; then npm run build; fi",
" -d build" here is the build process generated folder, replace it to whatever you actually use.
Not sure if this will work for your case, but seems like GAE standard has added the ability to run a custom build step.
However it does state:
After executing your custom build step, App Engine removes and regenerates the node_modules folder by only installing the production dependencies declared in the dependencies field of your package.json file.
Maybe since the node_modules are in your vendor/ directory, GAE may not detect and remove them, thus accomplishing your goal. This is a pre-install step, unlike postinstall specified in your script. Not sure if it matters.

Resources