I have created a http server which is creating proxy to send requests to target system.
function sendErrorResponse(client_req,client_res){
var options = {
:
};
var proxy = http.request(options, function (res) {
res.pipe(client_res, {
end: true
});
});
client_req.pipe(proxy, {
end: true
});
}
or using http-proxy npm package
function sendErrorResponse(client_req,client_res){
proxy.web(client_req, client_res, { target: 'http://localhost:7777' });
}
Now when client request to my server, I proxy some of the requests to other server and send to client whtever I get from the target server. I want that if target server gives some error (DNS resolution, network errors etc.) I can send same to the client instead of responding client with 500 with some error message.
How can I do that?
Related
I am using this package to get the videos from Tiktok tiktok-package.
It's working fine. But now after some time, I could not get any data maybe my IP blocked. So I used proxy for this which gives me error
Error
Exception thrown in tiktok scraper Error: tunneling socket could not be established, cause=getaddrinfo ENOTFOUND http
I run below lines in my project, but still, it's not working
npm config set proxy http://myproxy:port
npm config set https-proxy http://myproxy:port
Code
'use strict';
const TikTokScraper = require('tiktok-scraper');
var main = async(user, numOfVideos) => {
try {
var data = [];
var myPorxy = 'http://proxy_host:port' //my proxy and port
const posts = await TikTokScraper.user(user, { number: numOfVideos, proxy: myPorxy }
);
if (posts) {
posts.collector.map(post => {
data.push(post);
});
}
//return data;
console.log(data);
} catch (error) {
return error;
}
};
main('zachking', 5);
After taking a look at the Options for the packet your are using, it seems like you don't have to specify the protocol if you are using a http proxy:
// Set proxy {string[] | string default: ''}
// http proxy: 127.0.0.1:8080
// socks proxy: socks5://127.0.0.1:8080
// You can pass proxies as an array and scraper will randomly select a proxy from the array to execute the requests
proxy: '',
Assuming everything else is correct var myPorxy = 'proxy_host:port' should work.
_http_outgoing.js:797
socket.cork();
^
TypeError: socket.cork is not a function
at ClientRequest._flushOutput (_http_outgoing.js:797:10)
at ClientRequest._flush (_http_outgoing.js:776:16)
....
Getting above error when trying to http.get() or request() using SOCKS proxy agent previously created with "proxysocket" library.
I am trying to create working agent to use it socket.io, or ws or http to make connections via SOCKS proxy. I tries "proxysocket" library and its agent gives me error above.
let agent = proxysocket.createAgent("127.0.0.1", 9050);
request("http://www.google.com", {agent: agent}, (err, res, body) => {
if (err){
console.log("Http request error: " + err);
} else{
console.log("Http request connection success!");
console.log('statusCode:', response && response.statusCode);
console.log(body)
}
});
Ok, here is the solution. I tried tons of proxy agents, and the only working one is "socks5-http-client".
Other agents were giving all kinds of errors when using SOCKS5 protocol.
const http = require("http");
const Agent = require("socks5-http-client/lib/Agent"); // Constructor
let agent = new Agent({
socksHost: 'localhost', // Defaults to 'localhost'.
socksPort: 9050 // Defaults to 1080.
});
http.get({
hostname: "www.google.com", // Can be also onion address if used behind TOR
port: 80,
agent: agent,
method: 'GET'
}, (res)=>{
console.log("Connected");
res.pipe(process.stdout);
//Process res
}).on('error', (err) => {
console.error(`Got error: ${err.message}`);
});
Such agent can be passed into any kinds of libraries that use http.Agent such as socket.io, ws, etc.
I'm building an electron app and need to call APIs where the API provider has not enabled CORS. The typically proposed solution is to use a reverse proxy which is trivial to do when running locally by using node and cors-anywhere like this:
let port = (process.argv.length > 2) ? parseInt (process.argv[2]) : 8080;
require ('cors-anywhere').createServer ().listen (port, 'localhost');
The app can then be configured to proxy all requests through the reverse proxy on localhost:8080.
So, my questions are:
Is it possible to use node and cors-anywhere in an electron app to create a reverse proxy? I don't want to force the app to make calls to a remote server.
Is there a better or standard way of doing this in an Electron app? I'm assuming I'm not the first to run into CORS issues. :)
Just overide header before send request using webRequest.onBeforeSendHeaders
const filter = {
urls: ['*://*.google.com/*']
};
const session = electron.remote.session
session.defaultSession.webRequest.onBeforeSendHeaders(filter, (details, callback) => {
details.requestHeaders['Origin'] = null;
details.headers['Origin'] = null;
callback({ requestHeaders: details.requestHeaders })
});
put these codes in renderer process
In my application, it wasn't sufficient to remove the Origin header (by setting it to null) in the request. The server I was passing the request to always provided the Access-Control-Allow-Origin header in the response, regardless of it the Origin header is present in the request. So the embedded instance of Chrome did not like that the ACAO header did not match its understanding of the origin.
Instead, I had to change the Origin header on the request and then restore it on the Access-Control-Allow-Origin header on the response.
app.on('ready', () => {
// Modify the origin for all requests to the following urls.
const filter = {
urls: ['http://example.com/*']
};
session.defaultSession.webRequest.onBeforeSendHeaders(
filter,
(details, callback) => {
console.log(details);
details.requestHeaders['Origin'] = 'http://example.com';
callback({ requestHeaders: details.requestHeaders });
}
);
session.defaultSession.webRequest.onHeadersReceived(
filter,
(details, callback) => {
console.log(details);
details.responseHeaders['Access-Control-Allow-Origin'] = [
'capacitor-electron://-'
];
callback({ responseHeaders: details.responseHeaders });
}
);
myCapacitorApp.init();
});
Try this if you are running web apps in localhost
const filter = {
urls: ['http://example.com/*'] // Remote API URS for which you are getting CORS error
}
browserWindow.webContents.session.webRequest.onBeforeSendHeaders(
filter,
(details, callback) => {
details.requestHeaders.Origin = `http://example.com/*`
callback({ requestHeaders: details.requestHeaders })
}
)
browserWindow.webContents.session.webRequest.onHeadersReceived(
filter,
(details, callback) => {
details.responseHeaders['access-control-allow-origin'] = [
'capacitor-electron://-',
'http://localhost:3000' // URL your local electron app hosted
]
callback({ responseHeaders: details.responseHeaders })
}
)
Just had this issue today API calls with axios inside a React app bundled in Electron is returning 400
From what I can see Electron calls act as normal calls to the API urls meaning they are not affected by CORS.
Now when you wrap your calls with a CORS proxy and make a regular call to the proxy, it should error with a 400 error because it's not a CORS call.
This thread explains why cors-anywhere responds like that => https://github.com/Rob--W/cors-anywhere/issues/39
I actually removed my CORS proxies from the app before the Electron build. I still need the CORS proxy for development since I'm testing in the browser.
Hope this helps.
You can have the main process, the NodeJS server running Electron, send the request. This avoids CORS because this is a server-to-server request. You can send an event from the frontend (the render process) to the main process using IPC. In the main process you can listen to this event, send the HTTP request, and return a promise to the frontend.
In main.js (the script where the Electron window is created):
import { app, protocol, BrowserWindow, ipcMain } from ‘electron’
import axios from 'axios'
ipcMain.handle('auth', async (event, ...args) => {
console.log('main: auth', event, args) const result = await axios.post(
'https://api.com/auth',
{
username: args[0].username,
password: args[0].password,
auth_type: args[1],
},
) console.log('main: auth result', result)
console.log('main: auth result.data', result.data) return result.data
})
In your frontend JS:
import { ipcRenderer } from 'electron'
sendAuthRequestUsingIpc() {
return ipcRenderer.invoke('auth',
{
username: AuthService.username,
password: AuthService.password,
},
'password',
).then((data) => {
AuthService.AUTH_TOKEN = data['access_token']
return true
}).catch((resp) => console.warn(resp))
}
I wrote an article that goes into more depth here.
While I have struggled a while with the existing answers I will provide here the solution that finally worked for me, assuming that you are on the main process.
Here are the steps involved:
You need to have access to the session object which can be obtained by one of two ways:
A) via the global session.defaultSession which is available after the app is ready.
const { session } = require('electron');
const curSession = session.defaultSession;
B) The other method is via the session on the BrowserWindow, this assumes that the windnows is already created.
win = new BrowserWindow({});
const curSession = win.webContents.session;
Once you have the session object you set the response header to the site you are sending the request from.
For example, let's say your electron BrowserWindow is loaded from http://localhost:3000 and you are making a request to example.com, here would be some sample code:
const { app, BrowserWindow, session } = require('electron');
app.whenReady().then(_ => {
// If using method B for the session you should first construct the BrowserWindow
const filter = { urls: ['*://*.example.com/*'] };
session.defaultSession.webRequest.onHeadersReceived(filter, (details, callback) => {
details.responseHeaders['Access-Control-Allow-Origin'] = [ 'http://localhost:3000' ];
callback({ responseHeaders: details.responseHeaders });
}
// Construct the BrowserWindow if haven't done so yet...
});
Have you tried using fetch()
Check how to use fetch to make a no-cors request here
https://developers.google.com/web/updates/2015/03/introduction-to-fetch?hl=en
Sorry, I'm quite new to network programming and nodejs. I'm using nodes in a server with some clients in a local network.
Sometimes I have to ask data to clients via get request:
// somewhere inside server.js
function askDataToClient(ip) {
var options = {
host: String(ip),
port: 80,
path: '/client/read',
auth: 'username:password'
};
var request = http.get(options, function(htres){
var body = "";
htres.on('data', function(data) {
body += data;
});
htres.on('end', function() {
body = JSON.parse(body);
res.json(body);
res.end();
})
htres.on('error', function(e) {
// ...
});
});
}
I'd like to know the server ip used for calling this get request.
I know of this answer but it gives me all the various network active on the server machine:
lo0 127.0.0.1
en1 192.168.3.60
bridge0 192.168.2.1
If I'm querying the client 192.168.3.36 I know it is the second ip, 192.168.3.60 because they are on the same network.. but How to know it programmatically?
You should be able to use htres.socket.address().address to get the IP.
Check out request.connection.remoteAddress property available for the HTTP Request object. This indicates the address of the remote host performing the request.
I'm trying to set a http header when socket.io client makes the connection request. Is there a way to do this?
Here is what i'm doing:
// server side
var io = socketio(server);
io.use(function (socket, next) {
// authorize using authorization header in socket.request.headers
});
// client side
var socket = io(); // i'm trying to set an authorization header in this http reqeust
Any ideas? Thanks.
You can use extraHeaders option, if you are using socket.io-client >= 1.4.
For example:
var socket = io("http://localhost", {
extraHeaders: {
Authorization: "Bearer authorization_token_here"
}
});
engine.io-client, which is a backend of socket.io-client, introduced extraHeaders support on 2015-11-28.
It seems like the client doesn't support setting headers, as not all transports allow for the setting of headers.
This post by facundoolano details a workaround to authentication that doesn't require placing the auth token in the query string.
His workaround module can be found at https://github.com/invisiblejs/socketio-auth.
Makes me wonder why on server-side, socket.io allows for the request headers to be accessed...
There's a new way to do this: https://socket.io/docs/v3/middlewares/. Look under the "Sending Credentials" section.
// client
const socket = io(server, {
transports: ['websocket', 'polling', 'flashsocket'],
auth: {
token: 'abc'
}
});
// server
io.use((socket, next) => {
const token = socket.handshake.auth.token;
if (isValidJwt(token)){
next();
}else{
next(new Error("Socket authentication error"));
}
});
async function isValidJwt(token){
jwt.verify(token, secrets.jwt, function(err, decoded) {
if (err){
console.log(err);
return false;
}else{
//console.log(decoded);
return true;
}
});
}
This following information has been deprecated since socket.io 1.0
There are two methods of authorization: global or namespace (think route). The global method is set on the server with the io.set('authorization', function (handshakeData, callback) configuration call.
The handshakeData object contains the following information:
{
headers: req.headers // <Object> the headers of the request
, time: (new Date) +'' // <String> date time of the connection
, address: socket.address() // <Object> remoteAddress and remotePort object
, xdomain: !!headers.origin // <Boolean> was it a cross domain request?
, secure: socket.secure // <Boolean> https connection
, issued: +date // <Number> EPOCH of when the handshake was created
, url: request.url // <String> the entrance path of the request
, query: data.query // <Object> the result of url.parse().query or a empty object
}
The above information and a deeper explanation is available on this documentation page.
As of version 2.0.0 / 2017-01-22 engine.io-client supports
[feature] Allow extraHeaders to be set for browser clients in XHR requests (#519)
However at this point the socket.io-client is not updated to support this functionality, so couple of days may make this saga end until that time use the following instructions: https://facundoolano.wordpress.com/2014/10/11/better-authentication-for-socket-io-no-query-strings/
"transportOptions" options can be used to send extra headers in socket.io request. I also explained that here :-
Node.js + Socket.io | Set custom headers on the server
For some reason, these request headers are only received if the socket server is also socket.io.
If I connect to a python Websockets server for example I have no luck authenticating.
The only solution that worked for me is to use a different WebSocket client, for example, ws works fine.
import WebSocket from 'ws';
const socket = new WebSocket('wss://example.com/path', {
headers: {
Authorization: 'token'
},
});
Short Answer: It's imposiburu based on spec... if you just need to pass info early... why not query parameters?
socket = io('localhost:5000', {
path: '/mySocketPath',
transports: ['websocket'],
query: {
token:'some-token-value'
}
})
See #satpal-07 in
https://github.com/socketio/socket.io-client/issues/1356#issuecomment-810023635