Has anyone successfully deployed a SvelteKit app using npm pg to Netlify/Vercel/Cloudflare? My local dev implementation works just fine with how I have it set up ( db.ts file with a query function in lib/server, and then using endpoint actions and the load function in +page.server.ts files)
My build errors are as follows:
node_modules/pg-connection-string/index.js:3:18: ERROR: Could not resolve "url" node_modules/pg-connection-string/index.js:4:17: ERROR: Could not resolve "fs" node_modules/pg-pool/index.js:2:29: ERROR: Could not resolve "events" node_modules/pg-protocol/dist/parser.js:9:41: ERROR: Could not resolve "assert" node_modules/pg/lib/client.js:3:27: ERROR: Could not resolve "events"
And many of the above display a previous log message with something along the lines of:
✘ [ERROR] Could not resolve "buffer"
node_modules/safe-buffer/index.js:3:21:
3 │ var buffer = require('buffer')
╵ ~~~~~~~~
The package "buffer" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.
I've been trying to get it to just build and it seems like it isn't able to use the pg package because it isn't a true node server environment. For each adapter it attempts to build with (except the node adapter) it refuses to build anything to do with the pg npm package. I could be wrong about the why, but my question about the how remains.
My hope is to avoid something like Prisma (which hasn't been working for me either) and I am trying to do this as "intended" meaning that I want to use SvelteKit as both the front end and the true backend. So an additional express server or the like is not the solution I'm looking for.
EDIT: I have also successfully deployed the app to Azure using the node adapter, but pg AND Postgres.js both do not work.
So, I've spent quite a few hours today trying to put my nodeJS app that's fully using ESM (modules), and I've deployed it via cPanel on a server that's using Node v. 14.20.1. I'm constantly getting an error:
App 1153856 output: internal/modules/cjs/loader.js:948
App 1153856 output: throw new ERR_REQUIRE_ESM(filename);
App 1153856 output: ^
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /<serverlocation>/app.js
App 1153856 output: at new NodeError (internal/errors.js:322:7)
App 1153856 output: at Module.load (internal/modules/cjs/loader.js:948:11)
App 1153856 output: at Function.Module._load (internal/modules/cjs/loader.js:790:12)
App 1153856 output: at Module.require (internal/modules/cjs/loader.js:974:19)
App 1153856 output: at Module.require (/opt/cpanel/ea-ruby27/root/usr/share/passenger/helper-scripts/node-loader.js:80:25)
All files are written as modules, I don't have one "require()" anywhere.
Since the Node started fully supporting JS modules from v14 on, I'm taking a wild guess that the hosting server I'm using (and their Passenger for NodeJS) is using a loader that's using "require()" when calling my app.js file.
I've tried multiple solutions, I've even switched my app.js file to a CommonJS type, but then it required me to switch all other files to CJS as well, which would be too much hassle.
Has anyone managed to find a proper solution to this issue?
For anyone trying to solve this issue, this is how I solved it:
1- Create a loader script: Not necessarily in the same folder as the main app.js file of your app, but that's where I created it. Call it something like loader.cjs. The extension being .cjs is important if you have "type": "module" in your package.json.
As you might've guessed, this will be the new main of your app. Since the passenger's loaders have an issue with ES modules, just let it load a commonjs file instead.
2- Dynamic import of app.js: Did you know that you can still load ES modules in commonjs files? You just need a little extra bit to do so.
Apparently, ES modules are loaded asynchronously, and this doesn't work well with synchronous commonjs files. That's why you got the issue in the first place, right?
Therefore, the solution is: dynamic imports. Just like async functions, treat the imports of ES modules as promises. I don't really like using .then(), so I opted with await:
async function loadApp() {
await import("/path/to/app.js");
}
loadApp();
3- Rename your app.js's extension: I don't exactly know why this is necessary, but I got error along the lines of "couldn't find /path/to/app.mjs" and so I changed it to so. Then it worked. You can keep the name of the file as "app.js" in the path in the previous point, and the import will sstill correctly look for "app.mjs";
There might be more efficient ways to do this, but that's what my 2 bits of brain could come up with. Hopefully it helps others as well.
I'm using the library node-rsa (https://www.npmjs.com/package/node-rsa) in a NuxtJS project. When building for production using the command nuxt build (which includes minification of JS and CSS by default), the build process breaks near the end with the following message:
ERROR in 0.nuxt.bundle.7c6932a7a42bdaaa7fa4.js from UglifyJs
Unexpected token: name (pem) [./~/node-rsa/src/formats/pkcs1.js:55,0][0.nuxt.bundle.7c6932a7a42bdaaa7fa4.js:42640,20]
Error: Webpack build exited with errors
at /home/ubuntu/front-end/node_modules/nuxt/dist/nuxt.js:904:44
at /home/ubuntu/front-end/node_modules/webpack/lib/Compiler.js:267:15
at Compiler.emitRecords (/home/ubuntu/front-end/node_modules/webpack/lib/Compiler.js:362:37)
at /home/ubuntu/front-end/node_modules/webpack/lib/Compiler.js:260:12
at /home/ubuntu/front-end/node_modules/webpack/lib/Compiler.js:355:11
at next (/home/ubuntu/front-end/node_modules/tapable/lib/Tapable.js:154:11)
at Compiler.compiler.plugin (/home/ubuntu/front-end/node_modules/webpack/lib/performance/SizeLimitsPlugin.js:99:4)
at Compiler.applyPluginsAsyncSeries1 (/home/ubuntu/front-end/node_modules/tapable/lib/Tapable.js:158:13)
at Compiler.afterEmit (/home/ubuntu/front-end/node_modules/webpack/lib/Compiler.js:352:8)
at Compiler.<anonymous> (/home/ubuntu/front-end/node_modules/webpack/lib/Compiler.js:347:14)
at /home/ubuntu/front-end/node_modules/async/dist/async.js:460:16
at iteratorCallback (/home/ubuntu/front-end/node_modules/async/dist/async.js:1034:13)
at /home/ubuntu/front-end/node_modules/async/dist/async.js:944:16
at /home/ubuntu/front-end/node_modules/graceful-fs/graceful-fs.js:43:10
at FSReqWrap.oncomplete (fs.js:117:15)
I was able to temporarily mitigate the issue by disabling uglifyJS (solution found at https://github.com/nuxt/nuxt.js/issues/250) but that does not seem like a long term solution. What is the correct way to resolve this?
Potentially related issue: Webpack breaks when i include node-rsa library
The problem is that I was trying to use a node library (not uglify-safe) on the browser without webpacking it. I switched to using pure JS for my crypto needs and everything went fine.
I am trying this article working on a react calculator
I set my package.json with "start": "babel-node ./server/server.js" in the scripts. When I run npm start, errors shows up. yarn start gives more details error message shown there.
max#DESKTOP-4J1U771 MINGW64 ~/Documents/react-calculator (master) $
yarn start yarn run v1.7.0 warning package.json: No license field $
babel-node ./server/server.js
C:\Users\max\Documents\react-calculator\node_modules\babel-core\lib\transformation\file\logger.js:41
throw new Constructor(this._buildMessage(msg));
^
ReferenceError: [BABEL]
C:\Users\max\Documents\react-calculator\server\server.js: Unknown
option: base.0. Check out h ttp://babeljs.io/docs/usage/options/ for
more information about options.
A common cause of this error is the presence of a configuration
options object without the corresponding preset name. Example:
Invalid: { presets: [{option: value}] } Valid: { presets:
[['presetName', {option: value}]] }
For more detailed information on preset configuration, please see
https://babeljs.io/docs/en/plugins#pluginpresets-op tions.
at Logger.error (C:\Users\max\Documents\react-calculator\node_modules\babel-core\lib\transformation\file\logger.j
s:41:11)
at OptionManager.mergeOptions (C:\Users\max\Documents\react-calculator\node_modules\babel-core\lib\transformation
\file\options\option-manager.js:226:20)
at OptionManager.init (C:\Users\max\Documents\react-calculator\node_modules\babel-core\lib\transformation\file\op
tions\option-manager.js:368:12)
at File.initOptions (C:\Users\max\Documents\react-calculator\node_modules\babel-core\lib\transformation\file\inde
x.js:212:65)
at new File (C:\Users\max\Documents\react-calculator\node_modules\babel-core\lib\transformation\file\index.js:135
:24)
at Pipeline.transform (C:\Users\max\Documents\react-calculator\node_modules\babel-core\lib\transformation\pipelin
e.js:46:16)
at Object.transformFileSync (C:\Users\max\Documents\react-calculator\node_modules\babel-core\lib\api\node.js:152:
10)
at compile (C:\Users\max\Documents\react-calculator\node_modules\babel-register\lib\node.js:118:20)
at loader (C:\Users\max\Documents\react-calculator\node_modules\babel-register\lib\node.js:144:14)
at Object.require.extensions.(anonymous function) [as .js] (C:\Users\max\Documents\react-calculator\node_modules\
babel-register\lib\node.js:154:7) error Command failed with exit code
1. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this > command.
Does anyone see what's going on here and how to solve it?
Use create-react-app. It is way simpler than configuring babel and creating React projects the old way. To use it, go to the directory you want your project to be in, and type in your terminal: create-react-app [project-name]. This will do all the configuring for you and make it easier for you. The installation and configuration of React mentioned in the article is outdated, but you can still follow everything else in that tutorial. Good luck!
I have created a project with Angular-CLI. (using command: ng new my-angular-universal).
Then I carefully followed all the instructions from https://github.com/angular/angular-cli/wiki/stories-universal-rendering
It builds for --prod and works fine. But there are no instructions on how I can set up a --dev build and have it served with --watch flag.
I tried removing --prod flags from npm "scripts", and it doesn't even run in dev mode. It builds fine but when I open it in browser this is what I see (directly printed to response):
TypeError: Cannot read property 'moduleType' of undefined
at C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:7069:134
at ZoneDelegate.invoke (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:105076:26)
at Object.onInvoke (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:6328:33)
at ZoneDelegate.invoke (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:105075:32)
at Zone.run (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:104826:43)
at NgZone.run (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:6145:69)
at PlatformRef.bootstrapModuleFactory (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:7068:23)
at Object.renderModuleFactory (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:52132:39)
at View.engine (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:104656:23)
at View.render (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:130741:8)
the versions of npm packages that I use are currently the latest:
#angular/* - #5.2.*
#angular/cli #1.7.3
except for ts-loader, had to downgrade it because it wasn't working:
ts-loader #3.5.0
So if anyone has any info on how to make this work, it would be very appreciated! Or maybe you know some project templates with Angular Universal App configured for both --dev and --prod builds and ability to --watch?
For development, run npm run start which triggers ng serve. The current setup has hot module reloading so it will watch for your changes and update your dev view. I used the same instructions and got it working here https://github.com/ariellephan/angular5-universal-template
In short, for development, run npm run start and look at http://localhost:4200.
For production, run npm run build:ssr and npm run serve:ssrand look at http://localhost:4000
As contributors have pointed out, it might not be the most efficient and fastest way to develop, but nevertheless I did not want to accept workarounds. Besides, hosting front and back on separate servers brings up CORS issues, and I never planned my app to run on separate hosts, I wanted it all on the same host together with API methods.
The problem with --dev build was this:
when building with the following command:
ng build --app 1 --output-hashing=false (note that there is no --prod flag)
AppServerModuleNgFactory turned out missing in the ./dist-server/main.bundle
I imagine that this relates to the ahead of time(--aot) compilation which is the default behavior if you are building for --prod. So the instructions from https://github.com/angular/angular-cli/wiki/stories-universal-rendering included instructions to configure express server for production build only. And since there is no need for server to be able to dynamically render html templates the working --dev build command would be:
ng build --app 1 --output-hashing=false --aot
and this gets rid of the TypeError: Cannot read property 'moduleType' of undefined
Now to watch this whole mess:
run these in separate command windows:
ng build --watch
ng build --app 1 --output-hashing=false --aot --watch
webpack --config webpack.server.config.js --progress --colors --watch
And for the server to restart on change, you have to install nodemon package and run it like this:
nodemon --inspect dist/server (--inspect if you wish to debug server with chrome)
Some other important stuff:
Angular/CLI has a command to generate necessary scaffolding for a universal app:
ng generate universal
and it generates a fixed version of main.ts that avoids client angular bootstrap issue:
document.addEventListener('DOMContentLoaded', () => {
platformBrowserDynamic().bootstrapModule(AppModule)
.catch(err => console.log(err));
});
a problem that I stumbled upon once I implemented TransferState
There are basically two parts - the server and the UI. While developing the UI, I simply use ng serve. That means when I make changes in my code in the IDE, the browser refreshes automatically. And, here the server part is not used.
I do prod build and run the server only for final testing to see if everything works as expected (No error due to any 3PP library DOM manipulation or AOT related issues, etc.)
Here, I have created a skeleton structure of an Angular Universal project. As I extensively use Vagrant and Docker in my projects, I run the server in a Docker container within the Vagrant guest system. And for development of the UI, I don't run the server. Simply, the ng serve is used.
If you look into my structure in the above Github link, you'll find the details as to how to run it for development and production in the Readme file.
The web server handler server.ts uses the server bundle
const { AppServerModuleNgFactory, LAZY_MODULE_MAP } = require('./dist/server/main.bundle');
That's why the server bundle needs to be compiled before you can compile the server.ts file.
So having a watch system would mean
watching/recompiling the client bundle
watching/recompiling the server bundle
recompiling the server.ts once the server bundle is created
All of them take some time (especially if you do it with aot)
I'd recommend, like Saptarshi Basu mentionned, to develop as best as you can with ng serve and check with angular universal every so often.
Otherwise, it should be possible do achieve what you want with some kind of tasks (grunt/gulp/...) which triggers sequentially ng build ... and recompilation of server.ts file.
It is a bit messy no doubt, as we preferably wish for one command to rule them all.
I came up with a somewhat OK solution where my output will be:
dist/browser
dist/ng-server
Using the executable npm-run-all package (I find it working a lot better on windows machines than concurrently does) I run the three watch tasks: browser, ng-server and nodeJS. Watching node has a pre-task defined that simply runs a small utility/helper/file that watches for the existence of a dist/ng-server folder and terminate itself once found.
For all of this to work (based on the universal-starter repo as of november 2018) there's a couple of modifications to package.json required. Primarily, to support the --watch flag on ng run commands we need to update the compiler-cli (if memory serves), ng update --all should take care of that, giving you the latest angular/cli version in the process (assuming you have a recent cli version installed globally).
package.json
ng update --all
angular 6+
angular/cli 7+
yarn add/npm install the following
chokidar
npm-run-all
(runs our tasks in parallel with the -p flag. -p kills all processes, -l gives each running task a specific color and name in the console)
ts-node (runs nodejs in it's ts-format)
nodemon // for restarting ts-node
add something similar to my util/await-file.js (after some consideration I added my own file-watcher code below even though it wasn't exactly written with the intentions to be put up on display...)
modify your package.json scripts like below
modify your angular.json to match your folder names, following my examples, mainly the "server"'s outputPath should be changed from dist/server to dist/ng-server.
package.json scripts
"dev": "npm-run-all -p -r -l watch:ng-server watch:browser watch:node",
"watch:browser": "ng build --prod --progress --watch --delete-output-path",
"watch:ng-server": "ng run ng-universal-demo:server --watch --delete-output-path",
"watch:node": "yarn run watch:file-exist && yarn run ts-node",
"ts-node": "nodemon --exec ts-node server.ts -e ts,js",
"watch:file-exist": "node utils/await-file.js",
util/await-file.js
const chokidar = require('chokidar');
const fs = require('fs');
const path = require('path');
const DIR_NAME = 'ng-server';
const DIST_PATH = './dist';
// creates dist folder if it doesn't exist - prior to adding it to the watcher.
if (!fs.existsSync(DIST_PATH)) {
fs.mkdirSync(DIST_PATH);
}
const watcher = chokidar.watch('file, dir', {
ignored: '*.map',
persistent: true,
awaitWriteFinish: {
stabilityThreshold: 5000,
pollInterval: 100
}
});
const FOLDER_PATH = path.join(process.cwd(), 'dist');
watcher.add(FOLDER_PATH);
console.log(`file-watcher running, waiting for ${DIST_PATH}/${DIR_NAME}`);
function fileFound() {
console.log(`${DIR_NAME} folder found - closing`);
watcher.close();
process.exit();
}
watcher
.on('add', function (filePath) {
const matchWith = path.join('dist', DIR_NAME);
const paths = filePath.split(path.sep);
const fileName = paths[paths.length - 1];
if ((filePath.indexOf(matchWith) >= 0)
&& fileName.indexOf('.js') > fileName.length - 4) {
fileFound();
}
})
.on('error', error => console.log(`Watcher error: ${error}`));
"npm run start" and using "http://localhost:4200" works for me. Even with Angular 10