I am using express as a proxy server between a webapp and a remote backend service.
When piping responses through express, the response times are extremely slow.
Example (all services run locally on the same machine, tested with Postman, also running locally)
:8081 is the backend service, :4000 is the express proxy service
Fetching localhost:8081/api/assets/1 takes about 50ms. Absolutley fine.
But fetching localhost:4000/assets/1 takes 5 seconds
The code itself (to me) doesn't seem to indicate why this would take this much longer:
export const proxyApiCall = async (req: Request, res: Response, fullPath: string) => {
try {
const response = await backend.fetch(fullPath); // node-fetch with some additional headers
["content-type", "content-length", "last-modified", "cache-control"].forEach((name) => {
const value = response.headers.get(name);
if (value) {
res.setHeader(name, value);
}
});
res.status(response.status);
response.body.pipe(res);
} catch (err) {
LOG.error(err);
res.sendStatus(500);
}
};
Related
I have an issue where I want to call an external API to verify a certain token coming from my frontend.
i.e
router.post('/api/verifyToken', async (ctx, next) => {
router.post('https://external-site.com/verify').then((response) => {
if (response.success) { console.log('Do some stuff') };
})
})
How would I go about doing this with koa?
You misunderstood with the router itself.
In your router, you define a route where your clients can send HTTP requests and according to your business logic, you return the answers to them.
You can simply imagine router.post('/api/verifyToken' as an event listener. When a request comes in you run whatever is inside of it. It is not an HTTP client though.
If you want to send an external request you have to use an HTTP client for it.
There are a bunch of options:
Built-in http/https package
Axios
Node-fetch
GOT
Request
And many others
One simple example how to do with Axios would be
import axios from 'axios';
router.post('/api/verifyToken', async (ctx, next) => {
try{
const response = await axios.post('https://external-site.com/verify');
// Do your stuff here
console.log(response.data);
ctx.body = response.data;
}catch(e){
ctx.status = 422;
ctx.body = e.message;
}
})
I'm trying to make POST request on front page via 'jquery ajax' to my server, and then with that data from front make POST request to outer server on my server. Using that final response I got from outer request, I wanna render new data into my front using ajax success function.
It seemed to be working well on local server, but when I deploy this project with heroku or azure, this whole process take 1000~2000ms and doesn't seem to be working at all.
what's wrong with my code?
I'm trying to build some detecting system that would notify users if there's a vacancy on wanted course. so I let user to pick a class and at the same time I call a function to check if there's a vacancy on that class via POST request to school server.
//index.html
//in front page, when user pick a course to start observing, I send POST to my server
function scanEmpty(data,cn,elem){
$.ajax({
url: './getLeftSeat',
crossDomain: true,
type: 'POST',
data: data+`&cn=${cn}`,
success: function (data) {
alert('got!')
}
})
}
//app.js
// when I get POST from my server I call scanEmpty()
app.post('/getLeftSeat', async (req,res) => {
scanEmpty(qs.stringify(req.body), req.body["cn"], () => {res.json({success: true})})
})
// and that is like this
const scanEmpty = async(data, CN, cb) => {
if(await parseGetLeftSeat(await getData(data), CN)) cb()
else {
await scanEmpty(data,CN,cb)
}
}
// send POST to school server and get response using axios
async function getData(data) {
return await axios.post(school_server_url, data, {'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8'});
}
// it's just parsing and get data that I want
const parseGetLeftSeat = async (res, CN) => {
return new Promise((resolve, reject) => {
const $ = cheerio.load(res.data);
$("#premier1 > div > table > tbody > tr > td").each((i, e) => {
if (e.firstChild && e.firstChild.data == CN && e.next) {
const tmp = e.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.next.firstChild.data.split('/')
resolve(Number(tmp[1].trim()) - Number(tmp[0].trim()) < 1 ? false : true)
}
})
})
}
It works alright though takes 1000~2000 ms when actually on deploy server while it takes 100~200 ms on local server. I tested some codes and it looks like axios.post() is the one. but even if I change it to node-fetch, the result was the same. I really don't know what to do.
Because of CORS problems, I want to call an external REST API from inside my node express server. That is, I have code like this that obviously does not work because it does not return.
How can I make this work and return the results of my external call?
const server = express();
server.put('/callme',(req,res) => {
axios
('http://weather.com/restapi', 'put', { zip: 10530 })
.then((resp: any) => {
console.log(' success' + resp.data);
})
.catch(function(error: any) {
console.log(error.message);
});
}
Axios returns a Promise which is resolved in the .then(). In order to get the response data back to the client you need to return it with res.send().
const server = express();
server.get('/callme', (req, res) => {
axios
.get('http://weather.com/restapi?zip=10530')
.then((resp: any) => {
res.send(resp.data);
})
.catch(function(error: any) {
console.log(error.message);
});
}
It would be a good idea to cache the weather API response for a period of time and serve the cached response for subsequent requests.
I'm building an electron app and need to call APIs where the API provider has not enabled CORS. The typically proposed solution is to use a reverse proxy which is trivial to do when running locally by using node and cors-anywhere like this:
let port = (process.argv.length > 2) ? parseInt (process.argv[2]) : 8080;
require ('cors-anywhere').createServer ().listen (port, 'localhost');
The app can then be configured to proxy all requests through the reverse proxy on localhost:8080.
So, my questions are:
Is it possible to use node and cors-anywhere in an electron app to create a reverse proxy? I don't want to force the app to make calls to a remote server.
Is there a better or standard way of doing this in an Electron app? I'm assuming I'm not the first to run into CORS issues. :)
Just overide header before send request using webRequest.onBeforeSendHeaders
const filter = {
urls: ['*://*.google.com/*']
};
const session = electron.remote.session
session.defaultSession.webRequest.onBeforeSendHeaders(filter, (details, callback) => {
details.requestHeaders['Origin'] = null;
details.headers['Origin'] = null;
callback({ requestHeaders: details.requestHeaders })
});
put these codes in renderer process
In my application, it wasn't sufficient to remove the Origin header (by setting it to null) in the request. The server I was passing the request to always provided the Access-Control-Allow-Origin header in the response, regardless of it the Origin header is present in the request. So the embedded instance of Chrome did not like that the ACAO header did not match its understanding of the origin.
Instead, I had to change the Origin header on the request and then restore it on the Access-Control-Allow-Origin header on the response.
app.on('ready', () => {
// Modify the origin for all requests to the following urls.
const filter = {
urls: ['http://example.com/*']
};
session.defaultSession.webRequest.onBeforeSendHeaders(
filter,
(details, callback) => {
console.log(details);
details.requestHeaders['Origin'] = 'http://example.com';
callback({ requestHeaders: details.requestHeaders });
}
);
session.defaultSession.webRequest.onHeadersReceived(
filter,
(details, callback) => {
console.log(details);
details.responseHeaders['Access-Control-Allow-Origin'] = [
'capacitor-electron://-'
];
callback({ responseHeaders: details.responseHeaders });
}
);
myCapacitorApp.init();
});
Try this if you are running web apps in localhost
const filter = {
urls: ['http://example.com/*'] // Remote API URS for which you are getting CORS error
}
browserWindow.webContents.session.webRequest.onBeforeSendHeaders(
filter,
(details, callback) => {
details.requestHeaders.Origin = `http://example.com/*`
callback({ requestHeaders: details.requestHeaders })
}
)
browserWindow.webContents.session.webRequest.onHeadersReceived(
filter,
(details, callback) => {
details.responseHeaders['access-control-allow-origin'] = [
'capacitor-electron://-',
'http://localhost:3000' // URL your local electron app hosted
]
callback({ responseHeaders: details.responseHeaders })
}
)
Just had this issue today API calls with axios inside a React app bundled in Electron is returning 400
From what I can see Electron calls act as normal calls to the API urls meaning they are not affected by CORS.
Now when you wrap your calls with a CORS proxy and make a regular call to the proxy, it should error with a 400 error because it's not a CORS call.
This thread explains why cors-anywhere responds like that => https://github.com/Rob--W/cors-anywhere/issues/39
I actually removed my CORS proxies from the app before the Electron build. I still need the CORS proxy for development since I'm testing in the browser.
Hope this helps.
You can have the main process, the NodeJS server running Electron, send the request. This avoids CORS because this is a server-to-server request. You can send an event from the frontend (the render process) to the main process using IPC. In the main process you can listen to this event, send the HTTP request, and return a promise to the frontend.
In main.js (the script where the Electron window is created):
import { app, protocol, BrowserWindow, ipcMain } from ‘electron’
import axios from 'axios'
ipcMain.handle('auth', async (event, ...args) => {
console.log('main: auth', event, args) const result = await axios.post(
'https://api.com/auth',
{
username: args[0].username,
password: args[0].password,
auth_type: args[1],
},
) console.log('main: auth result', result)
console.log('main: auth result.data', result.data) return result.data
})
In your frontend JS:
import { ipcRenderer } from 'electron'
sendAuthRequestUsingIpc() {
return ipcRenderer.invoke('auth',
{
username: AuthService.username,
password: AuthService.password,
},
'password',
).then((data) => {
AuthService.AUTH_TOKEN = data['access_token']
return true
}).catch((resp) => console.warn(resp))
}
I wrote an article that goes into more depth here.
While I have struggled a while with the existing answers I will provide here the solution that finally worked for me, assuming that you are on the main process.
Here are the steps involved:
You need to have access to the session object which can be obtained by one of two ways:
A) via the global session.defaultSession which is available after the app is ready.
const { session } = require('electron');
const curSession = session.defaultSession;
B) The other method is via the session on the BrowserWindow, this assumes that the windnows is already created.
win = new BrowserWindow({});
const curSession = win.webContents.session;
Once you have the session object you set the response header to the site you are sending the request from.
For example, let's say your electron BrowserWindow is loaded from http://localhost:3000 and you are making a request to example.com, here would be some sample code:
const { app, BrowserWindow, session } = require('electron');
app.whenReady().then(_ => {
// If using method B for the session you should first construct the BrowserWindow
const filter = { urls: ['*://*.example.com/*'] };
session.defaultSession.webRequest.onHeadersReceived(filter, (details, callback) => {
details.responseHeaders['Access-Control-Allow-Origin'] = [ 'http://localhost:3000' ];
callback({ responseHeaders: details.responseHeaders });
}
// Construct the BrowserWindow if haven't done so yet...
});
Have you tried using fetch()
Check how to use fetch to make a no-cors request here
https://developers.google.com/web/updates/2015/03/introduction-to-fetch?hl=en
I'm trying to set up HTTP2 for an Express app I've built. As I understand, Express does not support the NPM http2 module, so I'm using SPDY. Here's how I'm thinking to go about it-I'd appreciate advice from people who've implemented something similar.
1) Server setup-I want to wrap my existing app with SPDY, to keep existing routes. Options are just an object with a key and a cert for SSL.
const app = express();
...all existing Express stuff, followed by:
spdy
.createServer(options, app)
.listen(CONFIG.port, (error) => {
if (error) {
console.error(error);
return process.exit(1)
} else {
console.log('Listening on port: ' + port + '.')
}
});
2) At this point, I want to enhance some of my existing routes with a conditional PUSH response. I want to check to see if there are any updates for the client making a request to the route (the client is called an endpoint, and the updates are an array of JSON objects called endpoint changes,) and if so, push to the client.
My idea is that I will write a function which takes res as one of its parameters, save the endpoint changes as a file (I haven't found a way to push non-file data,) and then add them to a push stream, then delete the file. Is this the right approach? I also notice that there is a second parameter that the stream takes, which is a req/res object-am I formatting it properly here?
const checkUpdates = async (obj, res) => {
if(res.push){
const endpointChanges = await updateEndpoint(obj).endpointChanges;
if (endpointChanges) {
const changePath = `../../cache/endpoint-updates${new Date().toISOString()}.json`;
const savedChanges = await jsonfile(changePath, endpointChanges);
if (savedChanges) {
let stream = res.push(changePath, {req: {'accept': '**/*'}, res: {'content-type': 'application/json'}});
stream.on('error', function (err) {
console.log(err);
});
stream.end();
res.end();
fs.unlinkSync(changePath);
}
}
}
};
3) Then, within my routes, I want to call the checkUpdates method with the relevant parameters, like this:
router.get('/somePath', async (req, res) => {
await checkUpdates({someInfo}, res);
ReS(res, {
message: 'keepalive succeeded'
}, 200);
}
);
Is this the right way to implement HTTP2?