In java, I can send request via proxy server as
HttpClient clinet = HttpClient.newBuilder()
.proxy(ProxySelector.of(new InetSocketAddress("proxy.server", portNum)))
.build();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("https://server.I.want.to.send.request"))
.POST(HttpRequest.BodyPublishers.noBody())
.build();
HttpResponse<String> response = clinet.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.statusCode());
System.out.println(response.body());
I have tested with code above and checked it works just fine.
But if I want to do the same thing in nodejs (react) it seems like it doesn't work.
const agent = httpsProxyAgent('proxy.server:portNum');
axios.request({
method: 'POST',
url: "https://server.I.want.to.send.request",
data: {body},
httpsAgent: agent
});
As far as I understand, code above is the best I can try with JavaScript in react.
I have tried it, but it fails with timeout error (which means request doesn't go through proxy server I guess)
Could anyone can give me an advise to solve this problem?
i think you can refer to this issue in github:
https://github.com/axios/axios/issues/925#issuecomment-513028175
If this still cannot fix your issue. You might change to use another library, node-fetch. The link is here:
https://www.scrapingbee.com/blog/proxy-node-fetch/
Related
I get Nfts on magic eden with this third party api.
http://api-mainnet.magiceden.io/rpc/getGlobalActivitiesByQuery?q=%7B%22%24match%22%3A%7B%22txType%22%3A%22initializeEscrow%22%2C%22blockTime%22%3A%7B%22%24gt%22%3A1643468983%7D%7D%2C%22%24sort%22%3A%7B%22blockTime%22%3A-1%7D%7D
It responses with results on postman and browser but causes 403 error with axios in node.js.
How can I get data in node.js?
const res = await axios.get(
'http://api-mainnet.magiceden.io/rpc/getGlobalActivitiesByQuery?q=%7B%22%24match%22%3A%7B%22txType%22%3A%22initializeEscrow%22%2C%22blockTime%22%3A%7B%22%24gt%22%3A1643468983%7D%7D%2C%22%24sort%22%3A%7B%22blockTime%22%3A-1%7D%7D',
{
headers : {
"Content-Type": "application/json",
"Access-Control-Allow-Credentials": "*"
}
}
);
return res.data;
Using a proxy here won't help as they have Cloudflare protection set up against bots/scripts and it requires cookies to be present: screenshot 1, screenshot 2
You should get in touch with their support and ask for the API key (they have a public API v2 coming soon), and then use it in Authorization: Bearer <token>
Few things to note here: Their API v2 is still in works and lacks some basic features. Using their current v1 API does not require having an API key (that's what their support personnel said) but it does have bot protection against scripted attacks.
I'm hesitant to go with the API v2 since it lacks even the most basic stuff and I don't expect it to come out anytime soon. Personally, I'm looking to get in touch with people who managed to integrate the v1 into their applications, to see what necessary steps they followed in order to be able to do it.
If you managed to find some new info on that regard let me know. I'll also edit this comment in case I find out how to set up the v1 connection properly.
EDIT: managed to get it working by using the https://github.com/puppeteer/puppeteer library. Started a small headless instance of Chrome and I hit the ME API with that browser like so: screenshot 3
It can be because of CORS error.
You can use Cors proxy to fix it.
Try it,please.
const CORS_PROXY_API = `https://cors.ryanking13.workers.dev/?u=`;
const magicedenAPI = `http://api-mainnet.magiceden.io/rpc/getGlobalActivitiesByQuery?q=%7B%22%24match%22%3A%7B%22txType%22%3A%22initializeEscrow%22%2C%22blockTime%22%3A%7B%22%24gt%22%3A1643468983%7D%7D%2C%22%24sort%22%3A%7B%22blockTime%22%3A-1%7D%7D`
const { data } = await axios({
method: 'get',
url: `${CORS_PROXY_API}${magicedenAPI}`
});
I'm trying to set a cookie in a node request. I have tried using packages like js-cookie, cookie-js, cookie and cookie-manager but none work.
The way I have tried it is very straight-forward, whenever my endpoint gets called i.e. https://develop.api/sess/init, I set the cookie at the very beggining of the endpoint with the following code
import * as Cookies from 'js-cookie';
export const init = async (event: APIGatewayEvent, context: Context) => {
...
Cookies.set('hello', 'hello');
...
}
As my endpoint has an auth header, I can not directly call it into my browser URL due to missing permissions, so I tried generating the fetch function with postman and pasting it into my browser's console. The function is the following
var myHeaders = new Headers();
myHeaders.append("Referer", "accepted.referer.com");
myHeaders.append("key", "somekey");
var requestOptions = {
method: 'GET',
headers: myHeaders,
redirect: 'follow'
};
fetch("https://develop.api/sess/init", requestOptions)
.then(response => response.text())
.then(result => console.log(result))
.catch(error => console.log('error', error));
Once called, my request successfully returns the expected response, but it never shows a Set-Cookie header in the network section, neither shows my cookie in the Application section.
I have to mention that I also tried looking for the cookie when making the call within Postman, but it never sets it neither.
Also, I have tried starting the application in localhost, and I have a successful response, but my cookie is still not being set.
About the package showed in the code, I said I have tried it with different ones and their implementations, so I don't think a broken package is the problem.
I'm starting to think that I have a wrong idea about how cookies work, or that someway I am completely blocking the sending of cookies within my code.
Environment
If it helps in any way, my endpoint is being hosted in a AWS Lambda application.
I know this should be trivial, but being battling with it for a day now.
I finally answered my own issue. The key here is that I'm using AWS lambdas as the proxy, therefore, the headers I were using to send the cookies were wrong, I was sending the cookies with the endpoint instead of within the lambda. Let me explain myself.
I was adding 'Set-Cookie':'cookieKey:cookieVal' in the headers of the Postman Call that I was using to test both my local and develop environments.
Instead of that, I needed to send the request within the response of the lambda for the cookies to be registered.
Please check at the following links for similar cases ->
https://aws.amazon.com/blogs/compute/simply-serverless-using-aws-lambda-to-expose-custom-cookies-with-api-gateway/
https://forum.serverless.com/t/how-to-send-a-cookie-as-a-response/1312/7
I want to get the IP which is used in this request to the URL. I am getting the response required( HTML of the webpage ). Since I want to try to dynamically change Ips using AWS lambda when making a request, I first wanted to see what IP is actually used. Is there a way to get that? I am using 'request-promise' module for this.
NodeJS Code
const options = {
method: 'POST',
"rejectUnauthorized": false,
url: URL,
formData: {
data:data
}
}
const response = await request(options)
I solved this issue by making an API which makes request to my other NodeJS server. and then printing the IP in the second server.
Also AWS Lambda in default state have changing Ips
I'm making an XMLHttpRequest, posting a string to a nodejs server upon certain user events in the browser, but not every event seems to make it to the server.
Here is basically the function I'm calling for each browser event:
function xhr_event(timeStamp){
xhr=new XMLHttpRequest()
xhr.open("POST",'/record_event');
console.log(timeStamp.toString())
xhr.send(timeStamp.toString())
}
where timeStamp=event.timeStamp
On the client side, each event logs to console. On the server side, not all events appear to POST. To the best I can tell, lost events are random.
I read about browser caching but I don't think that can be the problem, since each payload has a unique time stamp? Then again I'm not doing any encoding or setting of headers, so maybe that's the issue?
As #mottek mentioned and explained in the comments, adding a var (or let or const) before xhr solved the problem.
I didn't realize xhr=new XMLHTTPRequest() creates a global variable.
I also got the post to work consistently by using fetch:
fetch('/record_event', {
method: 'post',
headers: {
'Content-Type':'text/plain'
}
body: timeStamp.toString()
}
In my web app project, I came across a cross domain issue. The back-end API was deployed to the server, which providing the IP and port(We can assume the back-end API works well). And I am working on the front-end side, and still in the local development stage. So there is cross domain issue.
so I used a nodejs proxy and try to overcome the cross domain issue. For the nodejs proxy part, I am not the expert on that. Just get it from my teammates, who meet the same issue before.
So in my client side js code, I use axios library to send http request to the nodejs proxy as following:
axios.get('http://127.0.0.1:8888/service/delete/v1?id=5').then((response) => {
console.log(response);
}).catch((error) => {
console.log(error);
});
And the nodejs proxy will replace the 127.0.0.1:8888 part with the real API's IP and port, and send the request. That's my understanding about it.
so I run the nodejs proxy, which is listening on port 8888, and run my frond-end code in another console. But when I send the above mentioned request. I got the error as following:
GET http://127.0.0.1:8888/service/delete/v1?id=5 net::ERR_CONNECTION_TIMED_OUT
Error: Network Error
at createError (createError.js?f777:16)
at XMLHttpRequest.handleError (xhr.js?14ed:87)
and I debug the error deeper in the chrome devtool, and find the Provisional headers are shown in the header as following
I searched some previous article about this issue.It's said that the potential reason is the request is blocked.
But I can send other HTTP request successfully to other third party APIs. For example my mock data service Mockarro as following:
const key = 'mykeyxxx';
const url = `http://www.mockaroo.com/api/generate.json?schema=firstapis&key=${key}`;
axios.get(url).then((response) => {
console.log(response);
});
So my the previous request is blocked. I am very confused. My guess is the issue is from the nodejs proxy part? Right?