express application upload to beanstalk aborts the deployment process - node.js

What I am trying to do is create zip folder of the application and upload it to elastic beanstalk. But somehow it fails and gives me the following error:
[Instance: 'name] Command failed on instance. Return code: 1 Output: (TRUNCATED).../opt/elasticbeanstalk/containerfiles/ebnode.py", line 180, in npm_install raise e subprocess.CalledProcessError: Command '['/opt/elasticbeanstalk/node-install/node-v6.12.2-linux-x64/bin/npm', '--production', 'rebuild']' returned non-zero exit status 1. Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
The package.json for that project is as shown below:
"dependencies": {
"#google/maps": "^0.4.5",
"aws-sdk": "^2.199.0",
"bcrypt": "^1.0.2",
"bluebird": "^3.5.0",
"body-parser": "^1.13.3",
"busboy-body-parser": "^0.3.0",
"describe": "^1.2.0",
"dotenv": "^4.0.0",
"express": "^4.13.3",
"express-validator": "^3.2.1",
"fcm-push": "^1.1.2",
"file-system": "^2.2.2",
"firebase-admin": "^4.2.0",
"jsonwebtoken": "^7.3.0",
"mailgun-js": "^0.13.1",
"mongoose": "^4.8.4",
"morgan": "^1.8.1",
"multer": "^1.3.0",
"node-schedule": "^1.2.5",
"nodemailer": "^3.1.4",
"passport": "^0.4.0",
"passport-http-bearer": "^1.0.1",
"path": "^0.12.7",
"randomstring": "^1.1.5",
"s3-image-uploader": "^1.0.7",
"sqs-consumer": "^3.8.0",
"stripe": "^4.17.0",
"twilio": "^3.7.0",
"useragent": "^2.2.1",
"utf8": "^2.1.2",
"uuid": "^3.0.1",
"winston": "^2.3.1",
"winston-daily-rotate-file": "^1.4.6"
},
"devDependencies": {
"chai": "^4.1.2",
"chai-http": "^3.0.0",
"mocha": "^3.4.2",
"nodemon": "^1.11.0"
}
What am I missing in my deployment process?

['/opt/elasticbeanstalk/node-install/node-v6.12.2-linux-x64/bin/npm', '--production', 'rebuild'] if failing.
Running NPM as part of the deploy process can be problematic especially if you have not shrink wrapped the dependancies.
You may be able to see the specific package install problem on the instance in /var/log/eb-activity.log (eb ssh to the instance)
Can you run npm production locally with an empty node_modules directory?

Related

mocha not recognizing structuredClone is not defined

I'm using visual studio code with node v18.
I use structuredClone in my application and it runs fine but when it comes to running a test I get
"ReferenceError: structuredClone is not defined"
I don't know how mocha works, but is it for some reason using an older version of node or is there some other dependency creating an issue?
"dependencies": {
"#sendgrid/mail": "^7.6.2",
"#types/chai": "^4.3.1",
"#types/express": "^4.17.13",
"#types/mocha": "^9.1.1",
"#types/node": "^18.0.0",
"#types/npm": "^7.19.0",
"bcrypt": "^5.0.1",
"body-parser": "^1.19.2",
"chai": "^4.3.6",
"connect-flash": "^0.1.1",
"cookie-parser": "^1.4.6",
"dotenv": "^16.0.0",
"eventemitter2": "^6.4.5",
"express": "^4.17.3",
"express-validator": "^6.14.0",
"jsdom": "^20.0.0",
"jsonwebtoken": "^8.5.1",
"marked": "^4.0.12",
"memory-cache": "^0.2.0",
"mocha": "^10.0.0",
"mssql": "^8.0.2",
"sanitize-html": "^2.7.0",
"validator": "^13.7.0",
"xsg.spider": "file:./../spider"
},
"devDependencies": {
"#types/bcrypt": "^5.0.0",
"#types/connect-flash": "^0.0.37",
"#types/cookie-parser": "^1.4.2",
"#types/dompurify": "^2.3.3",
"#types/jsdom": "^16.2.14",
"#types/jsonwebtoken": "^8.5.8",
"#types/marked": "^4.0.3",
"#types/memory-cache": "^0.2.2",
"#types/mssql": "^7.1.5",
"#types/passport-local": "^1.0.34",
"#types/sanitize-html": "^2.6.2",
"#types/validator": "^13.7.2",
"nyc": "^15.1.0",
"ts-node": "^10.7.0",
"typescript": "^4.6.4"
}
It's probably a bug, as a turnaround to deep clone your object, you can use JSON.parse & JSON.stringify:
if (global.structuredClone)
clonedObject = structuredClone(object)
else
clonedObject = JSON.parse(JSON.stringify(object))

Azure DevOps build pipeline - npm ci - Unexpected end of JSON input

I have Azure DevOps build pipeline configured to build NPM application, that includes the following tasks:
Install Node.js (Node.js tool installer) - Version Spec 10.x
npm ci (npm task)
And it almost always fails on the npm ci task, with error:
SyntaxError: Unexpected end of JSON input while parsing near ' and then some random string, each time different.
If I re-run the job manually, it will works. Then, at the next build, it fails again (until I manually rerun it again).
I am also using Azure Artifacts feed with configured Upstream sources to https://registry.npmjs.org/ and npm ci task is configured to use packages from this registry.
I have tried to:
remove Artifacts feed and create a new one
remove Install Node.js task
change node version
change npm ci to npm install
and nothing works. How else can I fix that?
EDIT - 03 April 2020
Here is my package.json - I've removed some of the private data.
There are two private packages, #project-name/jodit-vue and #project-name/vuex-orm-plugin-axios hosted on Azure Artifacts - those are forks of jodit-vue and vuex-orm-plugin-axios with some customizations.
Also, less builds failed recently. About one in 8 builds fails now.
{
"name": "name",
"version": "1.0.0",
"private": true,
"scripts": {
},
"dependencies": {
"#project-name/jodit-vue": "^1.2.4-fix",
"#project-name/vuex-orm-plugin-axios": "0.5.0",
"#stomp/stompjs": "^5.4.2",
"#vuex-orm/core": "^0.33.0",
"axios": "^0.19.0",
"chart.js": "^2.9.3",
"chartjs-plugin-colorschemes": "^0.4.0",
"core-js": "^3.4.8",
"crypto-js": "^3.1.9-1",
"drag-drop-touch": "^1.3.0",
"emoji-mart-vue-fast": "^6.1.2",
"file-saver": "^2.0.2",
"jquery": "^3.4.0",
"lodash": "^4.17.14",
"moment": "^2.24.0",
"oidc-client": "^1.9.1",
"shepherd.js": "^5.0.1",
"vue": "^2.6.10",
"vue-chartjs": "^3.5.0",
"vue-class-component": "^7.1.0",
"vue-cookie-law": "^1.8.0",
"vue-cropperjs": "^4.0.0",
"vue-i18n": "^8.15.1",
"vue-matomo": "^3.13.0-0",
"vue-property-decorator": "^8.3.0",
"vue-recaptcha": "^1.3.0",
"vue-router": "^3.1.3",
"vue-stripe-elements-plus": "^0.2.8",
"vuejs-logger": "^1.5.3",
"vuetify": "^1.5.21",
"vuex": "^3.1.2",
"vuex-oidc": "^3.3.1",
"xlsx": "^0.15.2"
},
"devDependencies": {
"#babel/core": "^7.7.5",
"#babel/plugin-transform-runtime": "^7.7.6",
"#babel/preset-env": "^7.7.6",
"#babel/preset-typescript": "^7.7.4",
"#kazupon/vue-i18n-loader": "^0.4.1",
"#storybook/addon-actions": "^5.3.13",
"#storybook/addon-docs": "^5.3.13",
"#storybook/addon-knobs": "^5.3.13",
"#storybook/addon-links": "^5.3.13",
"#storybook/addon-viewport": "^5.3.13",
"#storybook/vue": "^5.3.13",
"#types/chai": "^4.1.6",
"#types/crypto-js": "^3.1.43",
"#types/file-saver": "^1.3.1",
"#types/jquery": "^2.0.47",
"#types/lodash": "^4.14.119",
"#types/mocha": "^2.2.46",
"#types/sinon": "^7.0.13",
"#types/stripe": "^5.0.24",
"#types/stripe-v3": "^3.0.8",
"#types/webpack": "^4.4.14",
"#types/webpack-env": "^1.13.6",
"#vue/cli": "^4.1.1",
"#vue/cli-plugin-babel": "^4.1.1",
"#vue/cli-plugin-eslint": "^4.1.2",
"#vue/cli-plugin-pwa": "^4.1.1",
"#vue/cli-plugin-typescript": "^4.1.1",
"#vue/cli-plugin-unit-jest": "^4.1.1",
"#vue/cli-plugin-unit-mocha": "^4.1.1",
"#vue/cli-service": "^4.1.1",
"#vue/eslint-config-typescript": "^4.0.0",
"#vue/test-utils": "1.0.0-beta.29",
"babel-loader": "^8.0.6",
"babel-plugin-transform-imports": "^2.0.0",
"chai": "^4.2.0",
"eslint": "^5.16.0",
"eslint-plugin-vue": "^5.0.0",
"moment-locales-webpack-plugin": "^1.1.2",
"node-sass": "^4.13.0",
"react": "^16.12.0",
"react-is": "^16.12.0",
"sass-loader": "^8.0.0",
"sinon": "^7.4.1",
"stylus": "^0.54.7",
"stylus-loader": "^3.0.2",
"typescript": "^3.7.4",
"vue-cli-plugin-i18n": "^0.6.0",
"vue-cli-plugin-storybook": "^1.2.0",
"vue-cli-plugin-vuetify": "^2.0.2",
"vue-jest": "^3.0.4",
"vue-loader": "^15.7.1",
"vue-template-compiler": "^2.6.10"
},
"browserslist": [
"> 1%",
"last 2 versions",
"not ie <= 8"
]
}
SOLVED (partially) - 15 April 2020
I have changed npm ci to npm install a few days ago. No more fails, every build finished successfully since this change.
install is slower that ci but we can spend one more minute in each build but be sure that it will not fail.

Azure NodeJS Web Service exists with status code null and signal SIGSEGV

I'm trying to deploy a NodeJS application to an Azure web service (Node version 10.14, Azure B1 plan), however, when I run the application, it always exists with status null and signal SIGSEGV.
There were no errors outputted. I tried deploying the same code to Heroku and there were no issues at all.
I tried using the segfault-handler module in hopes that it will tell what is wrong, but the only thing that it logs is:
PID 147 received SIGSEGV for address: 0x2c26
Any ideas what might cause this? One thing I was able to find that it might be related to native modules, but I still don't know which one might cause this.
EDIT
The Azure web service uses Linux OS and not Windows.
Modules used:
"dependencies": {
"apidoc": "^0.17.7",
"applicationinsights": "^1.3.1",
"bcrypt": "^3.0.5",
"chalk": "^2.4.2",
"concurrently": "^3.6.1",
"dotenv": "^6.2.0",
"excel4node": "^1.7.1",
"express": "^4.17.1",
"express-sslify": "^1.2.0",
"firebase-admin": "^7.2.0",
"get-port": "^4.2.0",
"is-json": "^2.0.1",
"joi": "^14.3.1",
"jsonwebtoken": "^8.5.1",
"lodash": "^4.17.11",
"mailgun-js": "^0.22.0",
"mkdirp": "^0.5.1",
"moment": "^2.24.0",
"morgan": "^1.9.1",
"node-cache": "^4.2.0",
"node-cleanup": "^2.1.2",
"node-schedule": "^1.3.1",
"nodemon": "^1.19.0",
"number-to-words": "^1.2.4",
"pdf-to-base64": "^1.0.2",
"pdfkit": "^0.9.1",
"pg": "^7.9.0",
"pug": "^2.0.3",
"rate-limiter-flexible": "^1.0.2",
"request": "^2.88.0",
"request-promise-native": "^1.0.7",
"segfault-handler": "^1.2.0",
"sequelize": "^5.8.7",
"socketcluster": "^14.3.3",
"socketcluster-client": "^14.2.2",
"socketcluster-server": "^14.4.0",
"systeminformation": "^4.0.15",
"twilio": "^3.30.0",
"umzug": "^2.2.0",
"uuid": "^3.3.2"
},
"devDependencies": {
"#types/chai": "^4.1.7",
"#types/chalk": "^2.2.0",
"#types/cookie-parser": "^1.4.1",
"#types/dotenv": "^6.1.0",
"#types/express": "^4.16.0",
"#types/joi": "^14.3.3",
"#types/lodash": "^4.14.107",
"#types/mocha": "^5.2.6",
"#types/moment": "^2.13.0",
"#types/morgan": "^1.7.35",
"#types/node": "^9.6.5",
"#types/request-promise": "^4.1.42",
"#types/serialize-javascript": "^1.3.2",
"#types/sinon": "^7.0.11",
"#types/uuid": "^3.4.3",
"casual": "^1.6.0",
"chai": "^4.2.0",
"chai-as-promised": "^7.1.1",
"mocha": "^5.2.0",
"sinon": "^7.2.3"
}
As you said, if there are few node modules related to native apis like Win32k.sys, it will be terminated immediately by Azure Web App sandbox once run.
Therefore, the possible reason may be these limits and restrictions described in sandbox secure environment, such as below.
Please inspect your used modules carefully. Hope it helps.

webpack-dev-server Error: EPERM: operation not permitted, stat 'D:\System Volume Information'

After I reinstalled my Windows and cloning project from GitHub, I am getting npm Error: EPERM: operation not permitted D:\System Volume Information
In terminal I run:
webpack-dev-server --open --config webpack.dev.js
And it compiles all good but when I open my browser on http://localhost:8000/ it says:
Internal Server Error
When I check terminal I can see:
i 「wdm」: Compiled successfully.
Error: EPERM: operation not permitted, stat 'D:\System Volume Information'
So my webpack compiles all good, the problem is when I try to open it at localhost:8000
Already tried
removing node_modules
npm cache clean --force
npm install npm#6.4.0 & npm#6.3.0
reinstalling node.js
using port 9000
running terminal as administrator
reinstalling webpack-dev-server
disabling kaspersky antivirus
deleting files from C:\Users\.config\configstore\
{
"name": "site",
"version": "1.0.0",
"main": "webpack.base.js",
"scripts": {
"start": "webpack-dev-server --open --config webpack.dev.js",
"build": "webpack --config webpack.prod.js "
},
"author": "",
"license": "ISC",
"dependencies": {
"#babel/plugin-syntax-optional-catch-binding": "^7.0.0",
"#babel/plugin-transform-async-to-generator": "^7.1.0",
"#babel/preset-react": "^7.0.0",
"#babel/template": "^7.1.2",
"#babel/types": "^7.1.3",
"adjust-sourcemap-loader": "^1.2.0",
"autoprefixer": "^9.3.1",
"babel": "^6.23.0",
"babel-core": "^6.26.3",
"babel-preset-env": "^1.7.0",
"babel-preset-es2015": "^6.24.1",
"bootstrap": "^4.1.3",
"debug": "^4.1.0",
"detect-libc": "^1.0.3",
"glob": "^7.1.3",
"gsap": "^2.0.2",
"html-loader": "^0.5.5",
"html-webpack-plugin": "^3.2.0",
"imagemin-pngquant": "^5.1.0",
"jquery": "^3.3.1",
"locate-path": "^3.0.0",
"lodash": "^4.17.11",
"lodash._baseassign": "^3.2.0",
"lodash._basecopy": "^3.0.1",
"lodash.isarguments": "^3.1.0",
"lodash.keys": "^4.2.0",
"ms": "^2.1.1",
"npm": "^6.5.0",
"only": "0.0.2",
"owl.carousel": "^2.3.4",
"pngquant-bin": "^3.1.1",
"popper.js": "^1.14.4",
"pretty-error": "^2.1.1",
"raw-loader": "^0.5.1",
"svg-url-loader": "^2.3.2",
"toposort": "^2.0.2",
"uglifyjs-webpack-plugin": "^2.0.1",
"update": "^0.7.4",
"util-promisify": "^2.1.0",
"webpack-cli": "^3.1.2"
},
"devDependencies": {
"#babel/cli": "^7.1.2",
"#babel/core": "^7.1.5",
"#babel/parser": "^7.1.3",
"#babel/preset-env": "^7.1.6",
"#webpack-cli/init": "^0.1.2",
"babel-loader": "^8.0.4",
"babel-minify-webpack-plugin": "^0.3.1",
"babel-plugin-transform-async-to-generator": "^6.24.1",
"clean-webpack-plugin": "^0.1.19",
"critters-webpack-plugin": "^2.0.1",
"css-loader": "^1.0.1",
"eslint": "^5.9.0",
"extract-text-webpack-plugin": "^3.0.2",
"file-loader": "^2.0.0",
"html-critical-webpack-plugin": "^2.1.0",
"mini-css-extract-plugin": "^0.4.5",
"node-sass": "^4.9.4",
"optimize-css-assets-webpack-plugin": "^5.0.1",
"postcss-loader": "^3.0.0",
"purify-css": "^1.2.5",
"purifycss-webpack": "^0.7.0",
"resolve-url-loader": "^3.0.0",
"sass-loader": "^7.1.0",
"style-loader": "^0.23.1",
"svg-inline-loader": "^0.8.0",
"url-loader": "^1.1.2",
"webpack": "^4.26.0",
"webpack-manifest-plugin": "^2.0.4",
"webpack-merge": "^4.1.4"
},
"description": ""
}
Versions
npm -v 6.5.0
node -v 11.4.0
Windows 10.0.17134
I solve this issue by opening Visual Studio Code in Admin Mode.
Actually problem was because webpack-dev-server watched wrong files.
Fix
Change your public path of webpack-dev-server and it should work

Deploy Django/React application heroku

I have deployed django app on heroku, and everything is fine, no errors, etc. I'm using react on frontend, and I want to do npm install, gulp build so I can configurate my app properly.
I can not run heroku run npm install, all I have is bash: npm: command not found, and I suppose that is ok because I need to trigger it somehow when he is building the app on the server, so I've modify my package.json like this:
{
"name": "Test app",
"version": "1.0.0",
"description": "Testing",
"main": "bundle.js",
"scripts": {
"postinstall": "npm install"
},
"dependencies": {
"bootstrap": "^3.3.7",
"babel-preset-es2015": "^6.1.2",
"babel-preset-react": "^6.1.2",
"babelify": "^7.2.0",
"browser-sync": "^2.10.0",
"browserify": "^12.0.1",
"browserify-shim": "^3.8.11",
"classnames": "^2.2.0",
"core-util-is": "^1.0.1",
"del": "^2.1.0",
"flux": "^2.1.1",
"gulp-autoprefixer": "^3.1.0",
"gulp-bless": "^3.0.1",
"gulp-connect": "^2.2.0",
"gulp-html-replace": "^1.5.4",
"gulp-if": "^2.0.0",
"gulp-insert": "^0.5.0",
"gulp-livereload": "^3.8.1",
"gulp-load-plugins": "^1.1.0",
"gulp-minify-css": "^1.2.1",
"gulp-rename": "^1.2.2",
"gulp-sass": "^2.1.0",
"gulp-sourcemaps": "^1.6.0",
"gulp-sync": "^0.1.4",
"gulp-uglify": "^1.4.2",
"history": "^1.13.0",
"keymirror": "^0.1.1",
"lodash": "^3.10.1",
"react": "^0.14.2",
"react-addons": "^0.9.0",
"react-bootstrap": "^0.27.3",
"react-dom": "^0.14.2",
"react-mixin": "^3.0.3",
"react-router": "^1.0.0",
"react-style": "^0.5.5",
"reqwest": "^2.0.5",
"rimraf": "^2.4.3",
"run-sequence": "^1.1.4",
"vinyl-buffer": "^1.0.0",
"vinyl-source-stream": "^1.1.0",
"watchify": "^3.6.0",
"when": "^3.7.4"
},
"devDependencies": {
"babel-preset-es2015": "^6.1.2",
"babel-preset-react": "^6.1.2",
"babelify": "^7.2.0",
"browser-sync": "^2.10.0",
"browserify": "^12.0.1",
"browserify-shim": "^3.8.11",
"classnames": "^2.2.0",
"core-util-is": "^1.0.1",
"del": "^2.1.0",
"flux": "^2.1.1",
"gulp-autoprefixer": "^3.1.0",
"gulp-bless": "^3.0.1",
"gulp-connect": "^2.2.0",
"gulp-html-replace": "^1.5.4",
"gulp-if": "^2.0.0",
"gulp-insert": "^0.5.0",
"gulp-livereload": "^3.8.1",
"gulp-load-plugins": "^1.1.0",
"gulp-minify-css": "^1.2.1",
"gulp-rename": "^1.2.2",
"gulp-sass": "^2.1.0",
"gulp-sourcemaps": "^1.6.0",
"gulp-sync": "^0.1.4",
"gulp-uglify": "^1.4.2",
"history": "^1.13.0",
"keymirror": "^0.1.1",
"lodash": "^3.10.1",
"react": "^0.14.2",
"react-addons": "^0.9.0",
"react-bootstrap": "^0.27.3",
"react-dom": "^0.14.2",
"react-mixin": "^3.0.3",
"react-router": "^1.0.0",
"react-style": "^0.5.5",
"reqwest": "^2.0.5",
"rimraf": "^2.4.3",
"run-sequence": "^1.1.4",
"vinyl-buffer": "^1.0.0",
"vinyl-source-stream": "^1.1.0",
"watchify": "^3.6.0",
"when": "^3.7.4"
},
"engines": {
"node": "8.0.0",
"npm": "5.0.4"
},
"author": "",
"license": ""
}
But that still does not trigger npm install process on deployment, so can any one help me understand how can I configurate this properly?
This is my Procfile:
web: python manage.py runserver 0.0.0.0:$PORT --noreload
worker: python worker.py
Heroku supports different server-side languages via buildpacks:
Buildpacks are responsible for transforming deployed code into a slug, which can then be executed on a dyno. Buildpacks are composed of a set of scripts, and depending on the programming language, the scripts will retrieve dependencies, output generated assets or compiled code, and more.
Since the Python part of your application seems to be installing correctly Heroku must be detecting it as a Python application (because it includes a requirements.txt file) or you have configured it as one manually. In many cases a single buildpack is sufficient.
However, since you are also using Node.js you should enable a second buildpack:
There are many scenarios in which a single buildpack is not sufficient when building an application. This includes cases when you need to:
Run a buildpack for each language your app uses. For example, run a JavaScript buildpack for assets and a Ruby buildpack for your application.
The basic flow looks like this (using the Heroku CLI on your development machine):
heroku buildpacks:set heroku/python
heroku buildpacks:add --index 2 heroku/nodejs
The links I provided above contain many more details. I recommend reading both of them in their entirety.

Resources