Serverless Framework not excluding dev dependencies when packaging - node.js

Why is my one computer including dev dependencies when packaging a Serverless Framework project but my other computer is not?
When packaging and deploying my Serverless project targeted for AWS, I found that the zip package contained dev dependencies in the node_modules folder. This only happened on one of my two computers. When performing the same build steps on AWS CodeBuild, the package was also ok with not including dev dependencies.
package.json
{
"name": "project-name",
"version": "0.0.1",
"description": "description",
"main": "index.js",
"dependencies": {
"amazon-cognito-identity-js": "^3.0.10",
"aws-sdk": "^2.488.0",
"axios": "^0.18.0",
"js-sha256": "^0.9.0",
"jsonwebtoken": "^8.5.1",
"jwk-to-pem": "^2.0.1",
"node-fetch": "^2.3.0",
"uuid": "^3.3.2",
"lodash": "^4.17.11"
},
"devDependencies": {
"chai": "^4.2.0",
"eslint": "^5.16.0",
"eslint-config-node": "^4.0.0",
"mocha": "^6.0.2",
"serverless": "^1.69.0",
"sinon": "^7.4.2",
"sinon-test": "^2.4.0"
},
"scripts": {
"test": "mocha ./test --recursive"
},
"repository": {
"type": "git",
"url": "https://git-codecommit.us-east-1.amazonaws.com/v1/repos/reponame"
},
"author": "",
"license": "ISC"
}
serverless.yml file that tried packaging but included dev dependencies.
service: service-name
# pinning serverless version for this project so all contributors are using the same version for consisten results
frameworkVersion: ">=1.60.0"
provider:
name: aws
runtime: nodejs10.x
stage: ${opt:stage, 'dev'} # default stage to use, unless overridden on command lin
region: us-east-1
functions:
create:
handler: create/index.create
events:
- http:
path: /{id}
method: post
cors: true
Both computers were windows 10 with the following Node versions
- npm version
6.4.1
- node version
10.15.3
I tried completely uninstalling Node.js from my Windows computer following Uninstall Node.js and reinstalling Node.js but it did not work.
I tried searching for other node_modules folders on my computer and removing them but the project still included the dev dependencies.
I tried building a simple Serverless project but it also included the dev dependencies.

The only solution I found to solve my problem was to perform a reset of windows (keeping user data, only removing application data). There must have been something in the applications, AppData, etc. that caused this issue.
After resetting windows 10 where user data was kept and only application data was removed, installing Node.js (same version as before) and installing Serverless (same version as before), the dev dependencies were no longer included in the node_modules section of the package that was generated.

Related

Expressjs App on Vercel, not installing peer dependencies

I'm building a small expressjs app that I host on vercel. Testing the app locally works fine, but when deploying and running on vercel, the require stack fails on my serverless function.
This is what vercel tells me :
2023-02-03T14:41:20.325Z undefined ERROR Cannot find module 'ndarray-pixels'
Require stack:
- /var/task/node_modules/#gltf-transform/functions/dist/functions.js
- /var/task/_index.js
My package.json looks like this
{
"name": "test - app",
"version": "1.0.0",
"description": "",
"main": "_index.js",
"scripts": {
"start": "node --experimental-fetch _index.js"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
"#gltf-transform/core": "^2.5.1",
"#gltf-transform/extensions": "^2.5.1",
"#gltf-transform/functions": "^2.5.1",
"cors": "^2.8.5",
"express": "^4.18.2",
"fetch": "^1.1.0",
"gl-matrix": "^3.4.3",
"ktx-parse": "^0.4.5",
"ndarray-pixels": "^1.0.0",
"node-fetch": "^2.6.9",
"three": "^0.148.0"
}
}
.
I've tried adding the build override on vercel, with no luck.
npm install --legacy-peer-deps
Any help or pointers are much appreciated
This problem can happen if you accidentally send the node_modules folder to the git remote repository. If it is the case, remove the node_modules folder, create a .gitignore file if it not exists and just write node_modules inside the gitignore file.
Now you can:
git rm -r --cached .
git add --all .
git commit -a -m "Versioning untracked files"
git push

Yarn link: Importing graphql-js project into another graphql-js project showing another module or realm error

I am using graphql-js instead of SDL for designing my graphql server. For this, I have created a small library which depends on graphql-js.
Thus, I am linking this library into my main project using yarn (yarn add link:../lib) to build graphql objects and schema.
My package.json files are given below
graphql-lib/package.json
{
"name": "graphql-lib",
"private": true,
"version": "0.1.0",
"description": "",
"main": "index.ts",
"dependencies": {
"graphql-iso-date": "^3.6.1"
},
"devDependencies": {
"#types/graphql-iso-date": "^3.4.0",
"#types/jest": "^25.2.3",
"#types/node": "^14.0.5",
"jest": "^26.0.1",
"ts-jest": "^26.1.0",
"typescript": "^3.9.3"
},
"peerDependencies": {
"graphql": "^15.1.0"
}
}
core/package.json
{
"name": "#core/schema",
"private": true,
"version": "0.1.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"dependencies": {
"graphql-lib": "link:../lib",
"graphql": "^15.1.0",
"graphql-iso-date": "^3.6.1"
},
"devDependencies": {
"#types/graphql-iso-date": "^3.4.0",
"#types/jest": "^26.0.0"
}
}
graphql-lib testing using ts-jest is working fine.
However, when I am testing my main project I am getting following error -
Cannot use GraphQLScalarType "Float" from another module or realm.
Ensure that there is only one instance of "graphql" in the node_modules
directory. If different versions of "graphql" are the dependencies of other
relied on modules, use "resolutions" to ensure only one version is installed.
graphql module in the node_modules directory contains only the graphql-js version 15.1.0. I have remove and reinstalled the node_modules in both the packages.
My understanding is that there should single executions instance of graphql. Am I missing something such that graphql instance is created in both the project? Can I link my project using yarn and maintain a single graphql instance?
That error only occurs when there are multiple copies of graphql-js in your dependencies. Most commonly it's because there are multiple versions in your node_modules. You can verify that's the case by running npm ls graphql or yarn ls graphql -- if you see multiple versions listed in your dependencies, that's a problem. Typically, this only happens when you have dependencies that directly depend on graphql-js (instead of making it a peer dependency). If you use yarn, you can use it's selective dependency feature to get around that issue.
When you're doing local development of multiple packages, you can also run into this issue because you have two different copies of graphql-js -- one in each of your two projects. That happens because npm link or yarn add link only creates a symlink from one of your project's node_modules to the other project. As a workaround, you can link graphql-js as well. Go into node_modules/graphql inside project A and run npm link/yarn link. Then go into the root directory for project B and run npm link graphql/yarn link graphql. Now project B will use project A's copy of the library instead of its own.

node_modules disappears after minutes of nmp install on Azure webapp

I am trying to deploy my node.js app on Azure webapp via bitbucket.
when I checked the the wwwroot folder on kudu console, I could not find any node_modules folder and hence the app failed to start
I have tried both npm install and npm install --production in kudu console (inside the wwwroot folder), and I could see the node_modules and files being install via filezilla.... however when I try to start the app again, the node_modules just disappears, can't see it neither in kudu console nor in filezilla.
the package.json file in the project folder:
{
"name": "fo",
"version": "1.0.0",
"description": "xx xx xx",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "xx xx",
"license": "MIT",
"dependencies": {
"angular-map-it": "0.0.20",
"angular-maps": "^6.0.1",
"angular-waypoints": "^2.0.0",
"axios": "^0.19.0",
"bcrypt": "^3.0.6",
"bluebird": "^3.5.5",
"body-parser": "^1.19.0",
"connect-mongodb-session": "^2.2.0",
"convert-json": "^0.5.0",
"csvtojson": "^2.0.10",
"dotenv": "^8.0.0",
"express": "^4.17.1",
"express-session": "^1.16.2",
"fixed-width-string": "^1.0.0",
"guid": "0.0.12",
"json2csv": "^4.5.2",
"jsontoxml": "^1.0.1",
"moment": "^2.24.0",
"moment-business-days": "^1.1.3",
"money": "^0.2.0",
"mongoose": "^5.6.4",
"multer": "^1.4.1",
"ng-storage": "^0.3.2",
"node-crisp-api": "^1.8.3",
"nodemailer": "^6.3.0",
"objects-to-csv": "^1.0.1",
"open-exchange-rates": "^0.3.0",
"sanitize": "^2.1.0",
"svg-assets-cache": "^1.1.3"
}
}
I don't understand, how people make node.js app work on azure? why node_modules is disappearing? and why azure is not automatically installing them based on my package.json??
Azure App Service understands package.json and npm-shrinkwrap.json files and can install modules based on entries in these files.
Azure App Service does not support all native modules and might fail when compiling modules with specific prerequisites. While some popular modules like MongoDB have optional native dependencies and work fine without them, the following workarounds proved successful with almost all native modules available today:
Navigate to Kudu - https://yoursite.scm.azurewebsites.net/
Locate the wwwroot folder and run the install command command.
cd site
cd wwwroot
npm install
Run npm install on a Windows machine that has all the native module's prerequisites installed. Then, deploy the created node_modules folder as part of the application to Azure App Service. Before compiling, check that your local Node.js installation has matching architecture and the version is as close as possible to the one used in Azure (the current values can be checked on runtime from properties process.arch and process.version). So, ensure that
Azure App Service can be configured to execute custom bash or shell scripts during deployment, giving you the opportunity to execute custom commands and precisely configure the way npm install is being run. For a video showing how to configure that environment, see Custom Website Deployment Scripts with Kudu. Kindly ensure that all the configuration is appropriate.
If the issue still persist, kindly let us know what specific error message you receive ( app failed to start) for further investigation and also take a look at this ‘Best practices and troubleshooting guide for node applications on Azure App Service Windows’ for more details on the topic.

npm ERR! Cannot read property 'pause' of undefined --Bluemix

I am trying to deploy a Sails.js app to Bluemix and we are getting the following error during the deploy stage in the toolchain on Bluemix (When the CF PUSH command is called):
npm ERR! Cannot read property 'pause' of undefined
I understand the trace goes to the npm-error.log file, however, I haven't been able to get to it as we cannot ssh in to see whats in the file as the application is in its "off" state after a bad deploy.
This same code was successfully deployed with both Cf push {app name}} and through a simple build and deploy toolchain in Bluemix just 8 days ago.
In the build stage I can run Npm install and npm update just fine. However, it seems to do this again in the deploy stage regardless and it fails. Here is some detail on this failure
NODE_ENV=production
NPM_CONFIG_LOGLEVEL=error
NPM_CONFIG_PRODUCTION=true
Visit http://docs.cloudfoundry.org/buildpacks/node/index.html#vendoring
NODE_HOME=/tmp/app/.cloudfoundry/0/node
NODE_MODULES_CACHE=true
NODE_VERBOSE=false
Restoring cache
Loading 3 from cacheDirectories (default):
- .npm
- .cache/yarn (not cached - skipping)
- bower_components (not cached - skipping)
Installing node modules (package.json) Building dependencies
sails#0.12.13 preinstall /tmp/app/node_modules/sails
node ./lib/preinstall_npmcheck.js
Sails.js Installation: Checking npm-version successful
npm ERR! Cannot read property 'pause' of undefined
npm ERR! /tmp/app/.npm/_logs/2017-09-09T17_02_48_660Z-debug.log
**ERROR** Unable to build dependencies: exit status 1
Failed to compile droplet
Exit status 223
Staging failed: Exited with status 223
Destroying container
npm ERR! A complete log of this run can be found in:
Successfully destroyed container
Package.json file
{
"name": "myApp",
"private": true,
"version": "0.0.1",
"description": "Stuff my app does",
"keywords": [
"Cool Apps"
],
"dependencies": {
"bcryptjs": "^2.4.3",
"cacheman": "^2.2.1",
"ejs": "2.3.4",
"elasticsearch": "^13.0.0-rc2",
"find-remove": "^1.0.1",
"fs": "0.0.1-security",
"grunt": "1.0.1",
"grunt-contrib-clean": "1.0.0",
"grunt-contrib-coffee": "1.0.0",
"grunt-contrib-concat": "1.0.1",
"grunt-contrib-copy": "1.0.0",
"grunt-contrib-cssmin": "1.0.1",
"grunt-contrib-jst": "1.0.0",
"grunt-contrib-less": "1.3.0",
"grunt-contrib-uglify": "1.0.1",
"grunt-contrib-watch": "1.0.0",
"grunt-sails-linker": "~0.10.1",
"grunt-sync": "0.5.2",
"include-all": "^1.0.0",
"jsonwebtoken": "^7.3.0",
"moment": "^2.18.1",
"moment-timezone": "^0.5.13",
"passport": "^0.2.x",
"passport-http-bearer": "^1.0.1",
"passport-idaas-openidconnect": "^1.1.0",
"passport-local": "^1.0.0",
"rc": "1.0.1",
"request": "^2.81.0",
"request-promise": "^4.2.0",
"sails": "~0.12.13",
"sails-disk": "~0.10.9",
"sails-mongo": "^0.12.2",
"stream": "0.0.2",
"uuid-1345": "^0.99.6",
"validator": "^7.0.0",
"natural-sort": "^1.0.0"
},
"scripts": {
"start": "node app.js"
},
"engines": {
"node": "^8.0.x",
"npm": "^5.0.x"
},
"main": "app.js",
"author": "Scott N",
"license": ""
}
Things I've tried so far.
-change node versions
-changed npm versions
-change dependencies versions to include the ^ prefix
-removed all dependencies but Sails in the package.json file
So Sails.js and the Nodebuildpack/Bluemix aren't playing well with each other.
Any help would be appreciated.
Thank you
It happens to me just the same. and I've to roll back the prev version of npm: 5.3.0 and it's working now. (BTW: macOS)
The error only happens when I update my npm version like this:
Update available 5.3.0 → 5.4.1
Run npm i -g npm to update
The best way to handle it is to update your package.json to use engines like so
"description": "a Sails application",
"keywords": [],
"engines": {
"node": "6.10.1",
"npm": "5.3.0"
},
Got this issue while trying to push to heroku which seems to be the npm version specified by #m-t, and also specifying the versions like above makes ibm bluemix pick it up as default node engine instead of using the latest stable version.
And don't forget to always specify your node and npm versions directly, instead of adding tilde or caret, this just to be safe. check https://stackoverflow.com/a/22345808/5836034 for more about tilde and caret
you can still check #sai-vennam answer here for more info
So I rolled back the sails version in the package.json file from 0.12 to 0.11 and it deployed fine. So CF PUSH and the newest sails version are not getting along on Bluemix. This was using Node 8.0 and NPM version ^5.0.x

npm install error for openshift node.js project

When deploying an OpenShift node.js project with a depdency on browserify 4.0.0, I get an error installing browserify's dependencies. Specifically:
...
remote: npm ERR! Error: No compatible version found: stream-browserify#'^1.0.0'
remote: npm ERR! Valid install targets:
remote: npm ERR! ["0.0.0","0.0.1","0.0.2","0.0.3","0.0.4","0.1.0","0.1.1","0.1.2","0.1.3","1.0.0"]
...
Given that stream-browserify's version is ^1.0.0 according to browserify's depdency and that openshift is suggesting 1.0.0 is a valid install target, why is this failing? I have seen this error in other cases, whenever the highest available openshift version fits the careted package.json version.
Am I misunderstanding what the caret means? Is this an OpenShift bug?
My package.json:
{
"name": "SampleApp",
"version": "1.0.0",
"description": "do things online",
"keywords": [
"OpenShift",
"Node.js",
"application",
"openshift"
],
"author": {
"name": "J",
"email": "j#email.com",
"url": ""
},
"homepage": "http://www.openshift.com/",
"repository": {
"type": "git",
"url": "https://github.com/openshift/origin-server"
},
"engines": {
"node": "0.x",
"npm": "1.x"
},
"dependencies": {
"body-parser": "1.x",
"browserify": "4.0.0",
"cookie-parser": "1.x",
"cookie-session": "1.x",
"express": "4.x",
"fast-csv": "0.x",
"multer": "0.0.5",
"pg": "3.x",
"sql": "0.x",
"xlsx-extract": "0.0.4"
},
"devDependencies": {
},
"bundleDependencies": [],
"private": true,
"main": "server.js",
"scripts": {
"build-js": "browserify public/index.js -o public/index-bundle.js & browserify public/intake.js -o public/intake-bundle.js",
"start": "npm run build-js && node server.js"
}
}
This behaviour could be because of different versions on node and npm on your local machine and openshift environment. Start by fixing "engines" attribute in your package.json, something as below:
"engines": {
"node": ">= 0.10",
"npm": ">= 1.4"
}
If still the issue is there (on openshift) it is due to the unavailability of the nodejs/npm required versions on openshift environment. For example, as of today, on my local machine I may be using node version 0.10.28 and npm version 1.4.9, but on openshift nodejs default cartridge I have to be content with nodejs version 0.10.5 and npm version 1.2.17, which is a big gap.
So, in this case, the easiest way to get around is by using "npm shrinkwrap", which freezes the nested dependency versions that ought to be used, hence doing away with varied behaviour of npm versions to figure out the nested dependency to install.
Can read about shrinkwrap here: https://www.npmjs.org/doc/cli/npm-shrinkwrap.html
So, on your local machine:
run npm install and make sure everything works.
fire npm shrinkwrap This will create a file - "npm-shrinkwrap.json", with the required shrinkwrap info. Add, commit and push the file to the openshift git repo.

Resources