Heroku Build Not Copying React Files - node.js

I have a Node application, including an express server, set to respond to various backend requests and then to try to serve static content from a React app's build folder. It works great locally. The two different package.json files in the server and client respectively build the React app, copy its build files into the server's ./dist folder, then builds the server app and does the same, then runs the server. Everything is perfect.
However, on Heroku, the React build folder is never copied into the server's dist folder, and I get 404 errors trying to load the app. I know this worked at some point but I'm not sure what I changed that broke it and nothing is making sense to me to get it working again.
"scripts" section of the client (react) package.json:
"scripts": {
"start": "react-scripts start",
"react-build": "react-scripts build",
"build": "npm run clean && npm run react-build && npm run copy && cd server && npm run client-build",
"clean": "rimraf server/dist/",
"copy": "copyfiles build/**/* server/dist -E",
"test": "react-scripts test",
"eject": "react-scripts eject",
"prod": "cd server && npm run start"
},
and the "scripts" section of the server's package.json:
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"dev": "nodemon -i dist -i creds index.ts",
"clean": "rimraf dist/",
"copy-files": "copyfiles views/**/* assets/**/* creds/**/* dist/",
"copy-web": "copyfiles -u 1 ../build/**/* dist/ -E",
"build": "cd .. && npm run react-build && cd server && npm run clean && tsc && npm run copy-files && npm run copy-web",
"client-build": "npm i && tsc && npm run copy-files",
"start": "node dist/index.js"
},
Heroku, of course, runs npm run build from the root folder to kick off the build process, which is the client's package.json, so it runs:
npm run clean && npm run react-build && npm run copy && cd server && npm run client-build
Specifically the npm run copy doesn't seem to be doing anything. When I hit the ls through the heroku-cli, the folder is not there. I'm not getting any errors in the build.
I have been able to put a cludgey piece of code into the main server so that it checks to see if the build folder exists where it's supposed to exist before setting up the static middleware, and to go up the file tree to where they're supposed to be copied from if it doesn't, so it's working, but I don't think I should have to do this and I can't just point to the initial React build location or I'd be serving stale build's while I'm in dev locally instead of a specific dev page that links to the webpack npm run start instance that updates as I hack at it.
I understand that between the workaround in prod and just not relying on the server's static files in dev is "good enough" for me to move on, but my OCD won't let me until I understand why copyfiles works fine copying the server's extra files into dist but can't manage to copy the React app to the place it's supposed to go.
This seems like a pretty basic thing, and long run my plan here is to serve multiple independent React applications as "widgets" from different paths on the server and if I can't get the root path to behave because I'm an idiot then idk what kinds of problems I'm going to run into down the line.
TIA, lmk if any additional info is needed.
CORRECTION: I had been building an outdated version without the -E flag on the copyfiles call. Once I updated that, I did in fact get an error that no files were copied. To reiterate though, the files ARE in the build folder where they were supposed to be copied from, just not in the dist/build folder where they're supposed to go, as confirmed now by copyfiles deciding it had nothing to copy.
Is this an async issue? I thought the && suggested that the left side (the build) had to finish before the right side (the copy) would execute?

Related

How to make Angular watch multiple libraries for changes and recompile when needed

This question is much the same as Make angular app watch for libraries changes and update itself. But, that question was never successfully answered as applies to the use of multiple libraries. I also reviewed Angular library and live reload and surveyed the answers and links from both questions.
My app is using two libraries: lib-1 and lib-2. When those files are edited, they are ignored and the app does not recompile. To see changes, I have to restart the server which really slows things down.
My expectation is that the app should be recompile when library files are edited, just like when other app-internal files are edited.
This is an Angular project that I have inherited, and the original author is no longer available. I am using Angular v10 and npm 6.14.11
The initial npm scripts are:
"start:staging": "ng serve --configuration-staging --host 0.0.0.0 --port 8080 --disableHostCheck",
"build:lib-1": "ng build lib-1 && cpx projects/lib-1/src/lib/theme.scss dist/lib-1",
"build:lib-2": "ng build lib-2 && cpx projects/lib-2/src/lib/theme.scss dist/lib-2",
"build:libs": "npm run build:lib-1 && npm run build:lib-2",
With those, I first run npm run build:libs, then npm run start:staging. As mentioned, this does not "watch" my libraries for changes.
I reviewed the suggestions and the other SO questions (above), have ensured that the npm-run-all, wait-on and rimraf libraries are now installed.
I have written these new npm scripts:
"clean": "rimraf dist",
"start-app": "wait-on dist/lib-1/fesm2015 dist/lib-2/fesm2015 && start:staging --poll 2000",
"watch:lib-1": "npm run build:lib-1 --watch",
"watch:lib-2": "npm run build:lib-2 --watch",
"watch-libs": "npm-run-all --parallel watch:lib-1 watch:lib-2",
"watch-all": "npm-run-all clean --parallel watch-libs start-app"
And, I am using the pre-existing start:staging script, as written.
I run npm run watch-all.
The script runs and proceeds to the point of building the libraries in parallel (bad idea?), and then throws error: sh: start:staging: command not found.
I removed the --parallel switches and tried again, and got the same error.
The start:staging script is indeed in the scripts object, and I cannot figure out why it's not being found.
I'm hoping to get some sage advice on correcting my syntax so that the app will compile and watch my library files along with the other files that are inside the app's src folder.
After a lot of sleuthing, I came across Nikola Kolev's Angular 6: build — watch multiple dependent libraries in one shell post.
While I don't have it down to one npm script like Nikola was able to do, I am able to do it by running two scripts (there are 7 total scripts involved), and that's good enough for now. I'll work on condensing to one when I get more time.
First, be sure to have wait-on, rimraf and npm-run-all installed. We're also using cpx; but, that's not about getting the libraries to be "watched" -- just including to be overly thorough.
Here are all the scripts:
"clean": "rimraf dist",
"watch-lib:lib-1": "ng build lib-1 --watch",
"watch-lib:lib-2": "ng build lib-2 --watch",
"watch-libs": "npm-run-all clean --parallel watch-lib:*",
"copy-styles:lib-1": "cpx projects/lib-1/src/lib/theme.scss dist/lib-1",
"copy-styles:lib-2": "cpx projects/lib-2/src/lib/theme.scss dist/lib-2",
"start-staging": "ng serve --configuration-staging --host 0.0.0.0 --port 8080 --disableHostCheck",
"watch-staging": "npm-run-all copy-styles:* start:staging"
When I want to work on the libraries and have them be "watched", I run npm run watch-libs in one terminal. When that is finished, I run npm run watch:staging in a second terminal. Then, I'm able to launch the app in a browser, and any edits to any of the code, in libraries or in the app itself are caught, and the app recompiles as desired.

How to run build in a parent directory, copy build into current directory before deploying to heroku

My friend had a bad folder structure, which he wanted me to contribute to, he had his backend in the client directory.
i wanted to separate the folder, he objected, so as not to loose previous git commit history. so i came up with this approach of running a script before deploying
i.e enters the client removes previous client build, runs new build and copy to backend. in the backend i serve the client build as a static file.
so after committing changes in both client and backend i ran
npm run dev
git push heroku
This approach worked very well, but i want only heroku or npm to run it before deploying, i tried changing my dev script to heroku-postbuild, heroku-prebuild, npm postinstall, and even build, but the build kept failing.
heroku-postbuild
"scripts": {
"start": "node server",
"heroku-postbuild": "cd ../ && rm -rf build && npm run build && cp -rf ./build ./backend/public",
"test": "echo \"Error: no test specified\" && exit 1"
},
heroku-prebuild
"scripts": {
"start": "node server",
heroku-prebuild": "cd ../ && rm -rf build && npm run build && cp -rf ./build ./backend/public",
"test": "echo \"Error: no test specified\" && exit 1"
},
it kept throwing this error,
and i understand why the error occurs, " cd ../" in the heroku-prebuild script refers to the client, which doesnt exist in the heroku repo i am deploying. pls what other approach can you recommend for me.

package.json scripts failing on heroku

So I have a series of scripts that are set up to either dev servers for a React/Node Express application OR a production server on heroku. The structure of the app is as follows:
client/package.json //pckg for react app
package.json //pckg for the node server
these are the scripts in the clients package:
"scripts": {
"start": "if-env NODE_ENV=production && npm run start:prod || npm run start:dev",
"start:prod": "node server.js",
"start:dev": "set NODE_ENV=development&& concurrently \"nodemon --ignore 'client/*'\" \"npm run client\"",
"client": "cd client && npm run start",
"seed": "node scripts/seedDB.js",
"install": "cd client && npm install",
"build": "cd client && npm run build",
"heroku-postbuild": "npm run build"
}
and the only difference between the react apps package.json and the one that is automatically generated with create-react-app is as follows:
"proxy": "http://localhost:3001/",
the way its supposed to run is, install scripts at root folder, run install script to cd into client and install react apps dependencies, then heroku's post-build script should kick in to run the build script which cds into client and builds a production ready react app. finally the start script should see a NODE_ENV of production and run start:prod.
my problem is that for some reason when i push to heroku, it seems to get stuck on an infinite loop on the install script. I have NO clue why, as the exact same scripts work on other projects PERFECTLY.
https://github.com/LordKriegan/reactdbdemo
https://github.com/LordKriegan/reactdbdemo2/ if anyone wants to look at it. (doing a full stack demo for my students :/ not much of a demo if i cant get it deployed)
I got it working. Forgot create-react-app also initializes a git repo...? either that or somewhere along the way i did an extra git init. anyways i had a .git folder in my client folder, which was preventing my client folder from being pushed up. i ended up creating a new repo and pushing everything to that one and now it works. so incase anyone comes here with a similar problem... make sure you didnt end up in some kind of gitception trap. :/

How do I deploy my Typescript Node.js app to Heroku?

When testing locally I was previously running:
"build-live": "nodemon --exec ./node_modules/.bin/ts-node -r dotenv/config -- ./index.ts"
I then figured my Procfile should be something like:
web: ./node_modules/.bin/ts-node -- ./index.ts
But it says module 'typescript' not found, even when it is in package.json. I read in a few places that ts-node is not the way to go to deploy to Heroku, so I am not sure what to do.
UPDATE: I think I am supposed to compile it, so I tried:
web: ./node_modules/.bin/tsc --module commonjs --allowJs --outDir build/ --sourceMap --target es6 index.ts && node build/index.js
This succeeds, however when actually running it, a bunch of the libs I'm using get "Cannot find module '...'".
Alternatively you can have the TypeScript compile as a postinstall hook and run node build/index.js as the only Procfile command:
Your package.json should contain a postinstall hint that gets executed after npm install and before the node process launches:
"scripts": {
"start": "node build/index.js",
"build": "tsc",
"postinstall": "npm run build"
}
You can then leave your Procfile as is:
web: npm start
This 'build on deploy' approach is documented by Heroku here.
The command you've given Heroku is to launch the web "process" by compiling index.ts and dependencies and starting node at index.js. Depending on how things are timed, index.js might or might not exist at the time node starts.
You need to already have your sources compiled by the time you want to start your app. For example, web should just be web: node index.js or similar.
Each build process is different, so you need to figure that out for your own setup. But, suppose you have a classical setup where you push to git and then Heroku picks up that change and updates the app with the new slug. You could just compile things locally and include index.js and any other build output in the repository, for it to be available in the slug for Heroku to use.
A better approach is to use a build server which has an integration with Heroku. After you do the build there, configure it to send the build results to Heroku. Travis has a straighforward setup like this. This way you don't need to include build outputs in your repository, which is considered an anti-pattern.
On a sidenode, try using a tsconfig.json to keep the tsc configuration. It will save you from having to write such long command lines all over the place.
Fabian said that we could do something like:
"scripts": {
"start": "node build/index.js",
"build": "tsc",
"postinstall": "npm run build"
}
As of me writing this, I tested this and can state: postinstall is not required since build script is ran by Heroku. If you want to do it without build script, then you can use heroku-postbuild which will run after dependencies are installed there you run tsc to compile.
My problem was about missing Typescript npm modules. The Typescript compiler tsc was not found when deployed the app to Heroku.
The Heroku deploy process (rightly) does not install development dependencies, in my case the Typescript module was part of devDependencies and thus the tsc command was not running on the Heroku platform.
Solution 1
Add typescript to dependencies: npm i typescript -s
Solution 2
Open Heroku console:
Select console type:
Run the command npm i typescript && npm run tsc
Install typescript as a dev dependency (cf. https://www.typescriptlang.org/download). Once built, your app does not need typescript anymore!
npm install -D typescript
Then in your package.json:
{
"main": "index.js", // <- file will be generated at build time with `tsc`
"scripts": {
"build": "tsc",
"start": "node ."
"start:dev": "ts-node index.ts" // idem, install ts-node as a dev dependency
}
}
The key point here is "build": "tsc".
Why?
Heroku does install all dependencies during build and remove the dev dependencies before the app is deployed (source here).
Node.js deployments will automatically execute an app’s build script during build (since March 11. 2019 source here)
In package.json
"scripts": {
"tsc": "./node_modules/typescript/bin/tsc",
"postinstall": "npm run tsc"
},
Works for me for Heroku deployment.
Installing typescript npm install -D typescript and writing tsc in the build script "build": "tsc", does not work for me. Also, try to run npm i typescript && npm run tsc in the Heroku console which also does not work.
In my case, I remove some dependencies from "devDependencies" to "dependencies", so it goes like this:
"dependencies": {
// The other dependencies goes here, I don't touch them.
// But all TS dependencies I remove to here.
"ts-node": "^9.1.1",
"tsconfig-paths": "^3.9.0",
"typescript": "^4.2.3",
"ts-loader": "^8.0.18"
},

gcloud deploy fails nodejs

I'm trying to deploy to google app engine. When I do so, the initial information all seems correct, the docker build is successful, but after docker it tries the following and fails:
Beginning teardown of remote build environment (this may take a few seconds).
Updating module [default].../Deleted [https://www.googleapis.com/compute/v1/projects/jamsesh-1219/zones/us-central1-f/instances/gae-builder-vm-20160213t114355].
Updating module [default]...failed.
ERROR: (gcloud.preview.app.deploy) Error Response: [13] Timed out when starting VMs. It's possible that the application code is unhealthy. (0/2 ready, 2 still deploying).
I know that I'm running es6 code transpiled to run on node 4.1.2 which is the version I'm told I can use here - I just have to run nvm install v4.1.2 which will switch me over, so I include that in my scripts. Here are my scripts in package.json:
"scripts": {
"test": "test",
"clean": "rm -rf lib",
"watch-js": "./node_modules/.bin/babel --plugins transform-es2015-classes src --presets react -d lib",
"dev-server": "./node_modules/.bin/webpack lib/client/entry.js",
"lint": "./node_modules/.bin/eslint ./src",
"server": "node lib/server/server",
"start-dev": "npm run lint && npm run build && npm run dev-server && npm run server",
"start": "nvm install v4.1.2 && node lib/server/server",
"styles": "./node_modules/.bin/lessc ./public/less/bundle.less ./public/css/bundle.css",
"build": "npm run clean && ./node_modules/.bin/babel --plugins transform-es2015-classes src --presets react -d lib && npm run styles"
},
and my docker file:
# Dockerfile extending the generic Node image with application files for a
# single application.
FROM gcr.io/google_appengine/nodejs
COPY . /app/
# You have to specify "--unsafe-perm" with npm install
# when running as root. Failing to do this can cause
# install to appear to succeed even if a preinstall
# script fails, and may have other adverse consequences
# as well.
# This command will also cat the npm-debug.log file after the
# build, if it exists.
RUN npm install --unsafe-perm || \
((if [ -f npm-debug.log ]; then \
cat npm-debug.log; \
fi) && false)
CMD npm start
So yeah, I've got no clue why the deployment failing. I could definitely benefit from some more thorough documentation on deploying node on GAE. It works on a test repo. Is there something else I need to be doing to run node on GAE with my babel6 transpiled code? Or should something be different in my scripts? In the Google Shell I can run locally fine once I switch to node v4.1.2

Resources