Astro: How to proxy service calls - node.js

I am setting up an Astro site which will display data fetched from a simple service running on the same host but a different port.
The service is a simple Express app.
server.js:
const express = require('express')
const app = express()
const port = 3010
const response = {
message: "hello"
}
app.get('/api/all', (_req, res) => {
res.send(JSON.stringify(response))
})
app.listen(port, () => {
console.log(`listening on port ${port}`)
})
Since the service is running on port 3010, which is different from the Astro site, I configure a server proxy at the Vite level.
astro.config.mjs:
import { defineConfig } from 'astro/config';
import react from '#astrojs/react';
export default defineConfig({
integrations: [react()],
vite: {
optimizeDeps: {
esbuildOptions: {
define: {
global: 'globalThis'
}
}
},
server: {
proxy: {
'/api/all': 'http://localhost:3010'
}
}
},
});
Here is where I am trying to invoke the service.
index.astro:
---
const response = await fetch('/api/all');
const data = await response.json();
console.log(data);
---
When I run yarn dev I get this console output:
Response {
size: 0,
[Symbol(Body internals)]: {
body: Readable {
_readableState: [ReadableState],
_events: [Object: null prototype],
_eventsCount: 1,
_maxListeners: undefined,
_read: [Function (anonymous)],
[Symbol(kCapture)]: false
},
stream: Readable {
_readableState: [ReadableState],
_events: [Object: null prototype],
_eventsCount: 1,
_maxListeners: undefined,
_read: [Function (anonymous)],
[Symbol(kCapture)]: false
},
boundary: null,
disturbed: false,
error: null
},
[Symbol(Response internals)]: {
type: 'default',
url: undefined,
status: 404,
statusText: '',
headers: { date: 'Tue, 02 Aug 2022 19:41:02 GMT' },
counter: undefined,
highWaterMark: undefined
}
}
It looks like the network request is returning a 404.
I'm not seeing in the doc much more about server configuration.
Am I going about this the right way?
I have this working correctly with a vanilla Vite app and the same config/setup.
How can I proxy local service calls for an Astro application?

Short Answer
You cannot proxy service calls with Astro but also you don't have to
For direct resolution answer see section functional test without proxy
Details
Astro does not forward the server.proxy config to Vite (unless you patch your own version of Astro), the Astro Vite server config can be seen empty
proxy: {
// add proxies here
},
reference https://github.com/withastro/astro/blob/8c100a6fe6cc652c3799d1622e12c2c969f30510/packages/astro/src/core/create-vite.ts#L125
there is a merge of Astro server with Astro vite.server config but it does not take the proxy param. This is not obvious to get from the code, see tests later.
let result = commonConfig;
result = vite.mergeConfig(result, settings.config.vite || {});
result = vite.mergeConfig(result, commandConfig);
reference https://github.com/withastro/astro/blob/8c100a6fe6cc652c3799d1622e12c2c969f30510/packages/astro/src/core/create-vite.ts#L167
Tests
Config tests
I tried all possible combinations of how to input config to Astro and in each location a different port number to show which one takes an override
a vite.config.js file on root with
export default {
server: {
port:6000,
proxy: {
'/api': 'http://localhost:4000'
}
}
}
in two locations in the root file astro.config.mjs
server
vite.server
export default defineConfig({
server:{
port: 3000,
proxy: {
'/api': 'http://localhost:4000'
}
},
integrations: [int_test()],
vite: {
optimizeDeps: {
esbuildOptions: {
define: {
global: 'globalThis'
}
}
},
server: {
port:5000,
proxy: {
'/api': 'http://localhost:4000'
}
}
}
});
in an Astro integration
Astro has a so called integration that helps update the config (sort of Astro plugins) the integration helps identify what was finally kept in the config and also gives a last chance to update the config
integration-test.js
async function config_setup({ updateConfig, config, addPageExtension, command }) {
green_log(`astro:config:setup> running (${command})`)
updateConfig({
server:{proxy : {'/api': 'http://localhost:4000'}},
vite:{server:{proxy : {'/api': 'http://localhost:4000'}}}
})
console.log(config.server)
console.log(config.vite)
green_log(`astro:config:setup> end`)
}
this is the output log
astro:config:setup> running (dev)
{ host: false, port: 3000, streaming: true }
{
optimizeDeps: { esbuildOptions: { define: [Object] } },
server: { port: 5000, proxy: { '/api': 'http://localhost:4000' } }
}
astro:config:setup> end
the proxy parameter is removed from astro server config, the vite config is visible but has no effect as it is overridden, and not forwarded to Vite
test results
dev server runs on port 3000 which is from Astro config server all other configs overridden
the fetch api fails with the error
error Failed to parse URL from /api
File:
D:\dev\astro\astro-examples\24_api-proxy\D:\dev\astro\astro-examples\24_api-proxy\src\pages\index.astro:15:20
Stacktrace:
TypeError: Failed to parse URL from /api
at Object.fetch (node:internal/deps/undici/undici:11118:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
functional test without proxy
Given that Astro front matter runs on the server side, in SSG mode during build and in SSR mode on page load on the server then the server sends the result html, Astro has access to all host ports and can directly use the service port like this
const response = await fetch('http://localhost:4000/api');
const data = await response.json();
console.log(data);
The code above runs as expected without errors
Reference Example
All tests and files mentioned above are available on the reference example github repo : https://github.com/MicroWebStacks/astro-examples/tree/main/24_api-proxy

You can add your own proxy middleware with the astro:server:setup hook.
For example use http-proxy-middleware in the server setup hook.
// plugins/proxy-middleware.mjs
import { createProxyMiddleware } from "http-proxy-middleware"
export default (context, options) => {
const apiProxy = createProxyMiddleware(context, options)
return {
name: 'proxy',
hooks: {
'astro:server:setup': ({ server }) => {
server.middlewares.use(apiProxy)
}
}
}
}
Usage:
// astro.config.mjs
import { defineConfig } from 'astro/config';
import proxyMiddleware from './plugins/proxy-middleware.mjs';
// https://astro.build/config
export default defineConfig({
integrations: [
proxyMiddleware("/api/all", {
target: "http://localhost:3010",
changeOrigin: true,
}),
],
});

Related

Using Spectron to "mock" Electron when testing with Jest

I'm testing library code for an Electron app with Jest. Jest does weird things to require, which is interfering with that Electron needs to do... I think.
Spectron is meant to allow you to access the various Electron bits from within a test framework, by allowing you to create an Electron app via library calls.
Ultimately, I need to be able to mock require('electron') with some real stuff from Electron (like the creation of browser windows), mostly so that various library bits can work as intended.
Here's what it looks like should work:
in package.json:
"jest": {
"moduleNameMapper": {
"^electron$": "<rootDir>/test/mocks/electron.js"
}
}
test/mocks/electron.js:
const Path = require("path")
const Application = require('spectron').Application
const electronPath = Path.join(__dirname, "../../node_modules/electron/dist/Electron.app/Contents/MacOS/Electron")
const app = new Application({ path: electronPath })
module.exports = app.electron
According to the docs, app.electron should give access to the same things as require('electron') does under normal operation.
Some test:
const { BrowserWindow } = require("electron")
test("some test", () => {
const window = new BrowserWindow()
// ...
})
However, this fails because app.electron is undefined, although App itself is defined:
console.log test/mocks/electron.js:58
<ref *1> Application {
host: '127.0.0.1',
port: 9515,
quitTimeout: 1000,
startTimeout: 5000,
waitTimeout: 5000,
connectionRetryCount: 10,
connectionRetryTimeout: 30000,
nodePath: '~/.nvm/versions/node/v13.0.1/bin/node',
path: '~/electron-hello-world/node_modules/electron/dist/Electron.app/Contents/MacOS/Electron',
args: [],
chromeDriverArgs: [],
env: {},
workingDirectory: '~/electron-hello-world',
debuggerAddress: undefined,
chromeDriverLogPath: undefined,
webdriverLogPath: undefined,
webdriverOptions: {},
requireName: 'require',
api: Api { app: [Circular *1], requireName: 'require' },
transferPromiseness: [Function (anonymous)]
}
Not really sure where to go from here. Looking for any solutions

Koa 404 when calling from Vue

I am trying to build an app with Koa and Nuxt. this is what I have:
Define service to retrieve from firestore:
const Firestore = require('#google-cloud/firestore');
const getItems = () => {
const db = new Firestore({
projectId: '*******',
keyFilename: "******"
});
db.collection('items').get()
.then((snapshot) => {
return snapshot;
})
}
Define them in routes.js:
const Router = require('#koa/router');
const articleService = require('./services/itemservice');
const router = new Router();
router.get('/getitems', async(ctx, next) => {
ctx.body = articleService.getItems();
});
module.exports = router;
Add routes to retrieve from routes.js:
app.use(router.routes());
app.use(router.allowedMethods());
And finally call it from a component:
let articles = axios.get('/getitems')
.then(response => {
console.log(response);
})//.....
I am receiving this error:
response:
{ status: 404,
statusText: 'Not Found',
headers:
{ 'content-type': 'text/html; charset=us-ascii',
server: 'Microsoft-HTTPAPI/2.0',
date: 'Fri, 25 Oct 2019 16:08:00 GMT',
connection: 'close',
'content-length': '315' },
config:
{ url: '/getarticles',
method: 'get',
headers: [Object],
transformRequest: [Array],
transformResponse: [Array],
timeout: 0,
adapter: [Function: httpAdapter],
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
maxContentLength: -1,
validateStatus: [Function: validateStatus],
data: undefined },
request:
ClientRequest {
_header:
'GET /getitems HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nUser-Agent: axios/0.19.0\r\nHost: localhost\r\nConnection: close\r\n\r\n',
_onPendingData: [Function: noopPendingOutput],
agent: [Agent],
socketPath: undefined,
timeout: undefined,
method: 'GET',
path: '/getitems',
_ended: true,
res: [IncomingMessage],
aborted: undefined,
timeoutCb: null,
upgradeOrConnect: false,
parser: null,
maxHeadersCount: null,
_redirectable: [Writable],
[Symbol(isCorked)]: false,
[Symbol(outHeadersKey)]: [Object] },
data:
'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">\r\n<HTML><HEAD><TITLE>Not Found</TITLE>\r\n<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>\r\n<BODY><h2>Not Found</h2>\r\n<hr><p>HTTP Error 404. The requested resource is not found.</p>\r\n</BODY></HTML>\r\n' },
isAxiosError: true,
toJSON: [Function] }
Can anyone point me in the right direction?
I've never built an app with Nuxt, but I'll try to help you anyway.
First, I recommend you to read about Promises:
- https://github.com/leonardomso/You-Dont-Know-JS/blob/master/async%20%26%20performance/ch3.md
- https://github.com/leonardomso/You-Dont-Know-JS/blob/master/es6%20%26%20beyond/ch8.md
(those are two chapters of a good JS book series!)
Second, you can try two things in order to find the bug:
- add a .catch block to your thenables, to check if something went wrong;
- add a dummy route that just logs an 'OK', to make sure the routes are be registered and up to respond.
I hope this will help you!
I had this problem with my nuxt / express.js app:
If you would try to type in your browser yourURL/getitems your nuxt app will try to route you to that page instead of just to show u the data.
First thing to do, how to say, you need to define what url your backend should handle.
You go to your nuxt.config.js and add this line of code:
serverMiddleware: ["~/api/index.js"],
That means you have a folder called api and in that folder you have an index.js file and thats your express.js / koa app.
Now in your index.js where your express.js / koa app is you need to add at the end of the line this peace of code:
module.exports = {
path: "/api",
handler: app
};
If everything works fine your URL should have now a prefix api and you should be able to get the data with localhost:PORT/api/getitems
Now nuxt wont try to route you to your url/api because it knows now that this is your backend
If you could provide me your folder structure of your nuxt app i could help you more.
Here is more information about serverMiddleware
https://nuxtjs.org/api/configuration-servermiddleware
EDIT:
somewhere you have a Folder, lets say ist named server or api
in that Folder there should be a index.js file and your routes, model, Controllers etc.
Lets say you have a Folder called server and in that Server you have index.js that should look something like this
const Koa = require('koa');
const app = new Koa();
Import routes from "./routes.js"
app.use(routes)
//here you define now your backend path
module.exports = {
//you can use any path you want
path: "/backend",
handler: app
};
app.listen(3000);
Now you Need to go to your nuxt.config.js file and Point to that index.js File
serverMiddleware: ["~/server/index.js"]
Now you can Access your data with axios:
axios.get("/backend/getitems").then(data => { console.log(data) })
You will Need to add backend to your axios url because thats the path you defined that your Server will handle.

nodejs : unknown error when trying to talk to SignalR api

im trying to use this pakcage
https://www.npmjs.com/package/signalr-client
to talk with SignalR api written in c# , but i get some error just when i trying to create the client
here is my code
var signalR = require('signalr-client');
try
{
var client = new signalR.client(
"https://firouzex.exphoenixtrade.com/realtime",
['GetNewAPIToken' , 'OmsClientHub']
);
}
catch (e) {
console.log('error');
}
but i get this error
Error Message: Protocol Error
Exception: undefined
Error Data: Url {
protocol: 'https:',
slashes: true,
auth: null,
host: 'firouzex.exphoenixtrade.com',
port: null,
hostname: 'firouzex.exphoenixtrade.com',
hash: null,
search:
'?connectionData=%5B%7B%22name%22%3A%22getnewapitoken%22%7D%2C%7B%22name%22%3A%22omsclienthub%22%7D%5D&clientProtocol=1.5',
query:
[Object: null prototype] {
connectionData: '[{"name":"getnewapitoken"},{"name":"omsclienthub"}]',
clientProtocol: '1.5' },
pathname: '/realtime/negotiate',
path:
'/realtime/negotiate?connectionData=%5B%7B%22name%22%3A%22getnewapitoken%22%7D%2C%7B%22name%22%3A%22omsclienthub%22%7D%5D&clientProtocol=1.5',
href:
'https://firouzex.exphoenixtrade.com/realtime/negotiate?connectionData=%5B%7B%22name%22%3A%22getnewapitoken%22%7D%2C%7B%22name%22%3A%22omsclienthub%22%7D%5D&clientProtocol=1.5',
headers: {} }
Known issue "client.Proxy settings currently only work for HTTP and not HTTPS". There is an another package for https https://www.npmjs.com/package/signalrjs. Copied it from npm package for signalr client not getting connected

Accessing Loopback config data from middleware

Say we are in Loopback middleware, such as
app.use(function(req,res,next){
// I am not sure what key to use here in the getter...
const config = app.get('?');
});
I want to access the overall config that Loopback is using.
I tried:
const config = app.get('config');
const config = app.get('env');
const config = app.get('datasources');
nothing gives me quite what I want.
Interestingly, this gives me:
console.log(app.get('restApiRoot'));
=> '/api'
so that's a clue to what's going on, but I want to get the parent object(s) for the above data.
how can we access the configuration that Loopback has loaded. The configuration of course varies by environment variables etc.
I want to log what datasources.x.js file was loaded and what config.x.js file was loaded, and any other server configuration info I can capture.
Having a lot of trouble figuring out how to do this.
This seems to be the same question I have:
https://github.com/strongloop/loopback/issues/1526
but they point me to the void that is Google Groups and I searched through there and couldn't find what the answer to this question.
This behavior is actually inherited from Express.
The entire config is stored in the app.settings object, with app.get(key) and app.set(key,value) just acting as getter/setter.
Doing console.log(app.settings); (in server/server.js for instance) it on a fresh loopback install returns the following:
{ 'x-powered-by': true,
etag: 'weak',
'etag fn': [Function: wetag],
env: 'development',
'query parser': 'extended',
'query parser fn': [Function: parseExtendedQueryString],
'subdomain offset': 2,
'trust proxy': false,
'trust proxy fn': [Function: trustNone],
view: [Function: View],
views: 'C:\\Users\\*******\\Documents\\GitHub\\lbtest\\views',
'jsonp callback name': 'callback',
host: '0.0.0.0',
port: 3000,
restApiRoot: '/api',
remoting:
{ context: { enableHttpContext: false },
rest: { normalizeHttpPath: false, xml: false },
json: { strict: false, limit: '100kb' },
urlencoded: { extended: true, limit: '100kb' },
cors: false,
errorHandler: { disableStackTrace: false } },
legacyExplorer: false,
'loopback-component-explorer': { mountPath: '/explorer' },
url: 'http://localhost:3000/' }

res.render doesn't render correctly after 1023 characters

I have a parent Express app, and a Ghost app as a child app, using Ghost as an npm module here.
I routed Ghost to be rendered at http://localhost:9000/blog. All the configuration works fine (Ghost will throw an error if the basic configuration isn't being provided correctly).
Here is my Ghost startup code
ghost({
config: path.join(__dirname, '/config/ghost.config.js')
}).then(function (ghostServer) {
app.use(ghostServer.config.paths.subdir, ghostServer.rootApp);
ghostServer.start(app);
});
here is my Ghost config
// # Ghost Configuration
// Setup your Ghost install for various [environments](http://support.ghost.org/config/#about-environments).
// Ghost runs in `development` mode by default. Full documentation can be found at http://support.ghost.org/config/
var path = require('path'),
config;
config = {
// ### Production
// When running Ghost in the wild, use the production environment.
// Configure your URL and mail settings here
production: {
url: 'http://my-ghost-blog.com',
mail: {},
database: {
client: 'sqlite3',
connection: {
filename: path.join(__dirname, '/content/data/ghost.db')
},
debug: false
},
server: {
host: '127.0.0.1',
port: '2368'
}
},
// ### Development **(default)**
development: {
// The url to use when providing links to the site, E.g. in RSS and email.
// Change this to your Ghost blog's published URL.
url: 'http://localhost:9000/blog/',
// Example mail config
// Visit http://support.ghost.org/mail for instructions
// ```
// mail: {
// transport: 'SMTP',
// options: {
// service: 'Mailgun',
// auth: {
// user: '', // mailgun username
// pass: '' // mailgun password
// }
// }
// },
// ```
// #### Database
// Ghost supports sqlite3 (default), MySQL & PostgreSQL
database: {
client: 'sqlite3',
connection: {
filename: path.join(__dirname, '../blog/data/ghost-dev.db')
},
debug: false
},
// #### Server
// Can be host & port (default), or socket
server: {
// Host to be passed to node's `net.Server#listen()`
host: '127.0.0.1',
// Port to be passed to node's `net.Server#listen()`, for iisnode set this to `process.env.PORT`
port: '9000'
},
// #### Paths
// Specify where your content directory lives
paths: {
contentPath: path.join(__dirname, '../blog/')
}
},
// **Developers only need to edit below here**
// ### Testing
// Used when developing Ghost to run tests and check the health of Ghost
// Uses a different port number
testing: {
url: 'http://127.0.0.1:2369',
database: {
client: 'sqlite3',
connection: {
filename: path.join(__dirname, '/content/data/ghost-test.db')
}
},
server: {
host: '127.0.0.1',
port: '2369'
},
logging: false
},
// ### Testing MySQL
// Used by Travis - Automated testing run through GitHub
'testing-mysql': {
url: 'http://127.0.0.1:2369',
database: {
client: 'mysql',
connection: {
host : '127.0.0.1',
user : 'root',
password : '',
database : 'ghost_testing',
charset : 'utf8'
}
},
server: {
host: '127.0.0.1',
port: '2369'
},
logging: false
},
// ### Testing pg
// Used by Travis - Automated testing run through GitHub
'testing-pg': {
url: 'http://127.0.0.1:2369',
database: {
client: 'pg',
connection: {
host : '127.0.0.1',
user : 'postgres',
password : '',
database : 'ghost_testing',
charset : 'utf8'
}
},
server: {
host: '127.0.0.1',
port: '2369'
},
logging: false
}
};
module.exports = config;
So basically, when I go to http://localhost:9000/blog, it isn't being rendered at all. Nothing. I was using Chrome and also testing it using Safari. Also tested those two without JavaScript turned on.
And then I try to do curl http://localhost:9000/blog, and try using a requester app (like Postman) and they returned the correct html string. I also tried to do a curl using the user agent as Chrome and as Safari, it also returns the correct html.
I traced down to ghost node_modules, and the renderer is in ghost > core > server > controllers > frontend > index.js in this line res.render(view, result)
I changed the res.render to be like this
res.render(view, result, function(err, string) {
console.log("ERR", err);
console.log("String", string);
res.send(string);
})
and there is no error, it logs the current string, but it doesn't render anything on the browser.
I tried curl, postman, works, but browser doesn't work.
then I tried to send a hello world string, it works, the browser rendered it.
Then I add the string length one by one, and it turns out, any str.length < 1023 will be able to be rendered by the browser, but once it get past that, it doesn't.
And I tried in my parent Express app, it is able to send string which length is more than 1023, and if I use the ghost module as a standalone, it also able to send string more than 1023.
So something must have happened between those two, but I don't know how to debug this.
Please help

Resources