Need to access a different domain url - node.js

I need to load page from this url: https://medalist-a805c.web.app/HeardSuggest_Medalist.html which actually loads a page from a express server. I'm able to use cors, the express middleware. But the problem is, even though i need to load this url: https://www.alwaysheard.com/embed/Medalist/brand/$2a$10$Z9ib22YZyRWWZz/Vlzju3u9eZCZM.a8oUYorNPsKGzLPKu6vM696K?uname=embed&orgin=https%3A%2F%2Fwww.alwaysheard.com eventually.
The weird problem i'm facing is it always returns the homepage rather than the url i am trying to get which is: https://www.alwaysheard.com/embed/Medalist/brand/$2a$10$Z9ib22YZyRWWZz/Vlzju3u9eZCZM.a8oUYorNPsKGzLPKu6vM696K?uname=embed&orgin=https%3A%2F%2Fwww.alwaysheard.com
Any help will be highly appreciated. Many Thanks.
This is for a expressjs server.
This is my frotend calling code:
$.ajax({
url: _url,
data: { uname: "embed", orgin: _origin },
// dataType: 'json',
crossDomain: true,
type: "GET",
mode: 'no-cors',
headers: {
'mode': 'no-cors',
// "X-TOKEN": 'xxxxx',
// 'x-Trigger': 'CORS',
// 'Access-Control-Allow-Headers': 'x-requested-with',
// "Access-Control-Allow-Origin": "*",
// 'Authorization': 'Bearer fadsfasf asdfasdf'
},
success: function (data) {
console.log("data: ", data);
$("body").html(data);
}
});
And in backend i'm using app.use(cors());

Your URL is first returning a 302 redirect to the home page and then the $.ajax() system (actually the XMLHttpRequest system underneath it) is following that redirect automatically.
You can see a discussion of that topic here:
How to prevent ajax requests to follow redirects using jQuery
In newer browsers, you can use the fetch() interface instead and it contains a redirect option that allows you to tell it not to follow redirects.
FYI, it's unclear what you're really trying to accomplish here. There is no "page" to retrieve from this URL. That returns a 302 status and a few headers, no page.
In case this isn't clear to you, your long url returns this response:
HTTP/1.1 302 Found
Date: Fri, 18 Oct 2019 17:04:05 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 23
Connection: keep-alive
Server: nginx/1.14.1
Access-Control-Allow-Origin: *
Set-Cookie: _csrf=j6JfsEb_JbmSJlvHDoyhgatS; Path=/
Set-Cookie: connect.sid=s%3A8C1PVNBcm5395dwmPJ4Xbl7BNASxPH4W.WFZ6uz3NklxaObnWvZaZXI%2BoH5VdM9dnOi2VCTw3J%2BI; Path=/; Expires=Mon, 15 Oct 2029 17:04:05 GMT; HttpOnly
Surrogate-Control: no-store
Cache-Control: no-store, no-cache, must-revalidate, proxy-revalidate
Pragma: no-cache
Expires: 0
Location: /
Vary: Accept
This content was retrieved with curl -l -v yourlongurl.

Related

Azure Functions: NodeJS - HTTP Response Renders as XML Rather than HTTP Response

I have an Azure function written in NodeJS where I'm attempting to cause an HTTP Redirect with a 302. The documentation is very sparse on what are valid entries in the response. As a result I've created an object with what I feel should be the correct entries to generate the redirect, but all I get is an XML response. Even items like the status code are shown in the XML rather than changing the real status code.
What am I doing wrong?
My Code:
module.exports = function(context, req){
var url = "https://www.google.com";
context.res = {
status: 302,
headers: {
Location: url
}
}
context.done();
}
This is the response I'm getting in the browser:
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 1164
Content-Type: application/xml; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 18 Jan 2017 00:54:20 GMT
Connection: close
<ArrayOfKeyValueOfstringanyType xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays"><KeyValueOfstringanyType><Key>status</Key><Value xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:int">302</Value></KeyValueOfstringanyType><KeyValueOfstringanyType><Key>headers</Key><Value i:type="ArrayOfKeyValueOfstringanyType"><KeyValueOfstringanyType><Key>Location</Key><Value xmlns:d5p1="http://www.w3.org/2001/XMLSchema" i:type="d5p1:string">https://www.google.com</Value></KeyValueOfstringanyType></Value></KeyValueOfstringanyType></ArrayOfKeyValueOfstringanyType>
The problem is that you're not defining the "body" in the response. This can be set to null, but it must be set for Azure functions to interpret it properly.
e.g. Update your code to:
module.exports = function(context, req){
var url = "https://www.google.com";
context.res = {
status: 302,
headers: {
Location: url
},
body : {}
}
context.done();
}
You will then get the desired response:
HTTP/1.1 302 Found
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 0
Expires: -1
Location: https://www.google.com
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 18 Jan 2017 01:10:13 GMT
Connection: close
Edited 2/16/2017 - Using "null" for the body currently throws an error on Azure. As a result the answer was updated to use {} instead.
This is a bug with azure functions, see this content headers issue.
For now, a workaround is to remove any content related headers if the body of your response is null.

Emotion API Project Oxford base64 image

I am trying to make a call to the Emotion Api via JavaScript with in a PhoneGap app. I encoded the image into base64 and verified that the data can be decoded by one of the online tools. this is the code that i found on the web to use.
var apiKey = "e371fd4333ccad2"; //(you can get a free key on site this is modified for here)
//apiUrl: The base URL for the API. Find out what this is for other APIs via the API documentation
var apiUrl = "https://api.projectoxford.ai/emotion/v1.0/recognize";
"file" is the base64 string.
function CallAPI(file, apiUrl, apiKey)
{
// console.log("file=> " +file);
$.ajax({
url: apiUrl,
beforeSend: function (xhrObj) {
xhrObj.setRequestHeader("Content-Type", "application/octet-stream");
xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key", apiKey);
},
type: "POST",
data: file,
processData: false
})
.done(function (response) {
console.log("in call api a");
ProcessResult(response);
})
.fail(function (error) {
console.log(error.getAllResponseHeaders())
});
}
function ProcessResult(response)
{
console.log("in processrult");
var data = JSON.stringify(response);
console.log(data);
}
I got back this:
Expires: -1
Access-Control-Allow-Origin: *
Pragma: no-cache
Cache-Control: no-cache
Content-Length: 60
Date: Fri, 01 Apr 2016 13:34:32 GMT
Content-Type: application/json; charset=utf-8
X-Powered-By: ASP.NET
So i tried their console test page.
https://dev.projectoxford.ai/docs/services/5639d931ca73072154c1ce89/operations/563b31ea778daf121cc3a5fa/console
I can put in an image like your "example.com/man.jpg" and it works great. but if i take the same image and have it encoded as a base 64 image all i get is "Bad Body" i have tried it both as content type "application/octet-stream" and "application/json" and get the same error. sample of the encoded looks like..and http request
POST https://api.projectoxford.ai/emotion/v1.0/recognize HTTP/1.1
Content-Type: application/octet-stream
Host: api.projectoxford.ai
Ocp-Apim-Subscription-Key: ••••••••••••••••••••••••••••••••
Content-Length: 129420
data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAASABIAAD/...
i get back:
Pragma: no-cache
Cache-Control: no-cache
Date: Fri, 01 Apr 2016 16:23:09 GMT
X-Powered-By: ASP.NET
Content-Length: 60
Content-Type: application/json; charset=utf-8
Expires: -1
{
"error": {
"code": "BadBody",
"message": "Invalid face image."
}
}
I am now not sure if you can send an image like this or not from Javascript. Can anyone tell me if my javascript is correct or if they can send an encoded base64 string image to the site.
thanks for your help
tim
This API does not accept data URIs for images. What you'll need to do is convert it to a binary blob. Though this answer is for a different Project Oxford API, you can apply the same technique.

nodejs https response is not received

I'm using nodejs on windows 7. I'm experiencing strange behavior. The same https request that always gives the same response is most of the time not received by node whereas it is always received by a web browser.
The code is quite simple:
var url1 = 'https:<my url>'
require('request').get(url1).pipe(process.stdout)
The response is actually a chunked response, which is a big json object.
If I try this request on a browser, I always receive the anwser.
If I use the node request module (I also tried with https), the body is most of the time empty. But sometimes it is returned.
I receive status 200 and response headers are:
{ 'content-type': 'application/json',
connection: 'close',
'transfer-encoding': 'chunked',
date: 'Wed, 27 Jan 2016 21:01:59 GMT',
server: 'My Server' }
Any idea?
I found 2 solutions to fix this issue:
1) Set header 'Connection': 'keep-alive'
var opts = { url: 'my url', headers :{ 'Connection':'keep-alive'} }
request(opts).pipe(..)
2) set to true KeepAlive option in https.globalAgent
https.globalAgent.keepAlive=true

Node.js request.js HPE_INVALID_HEADER_TOKEN

I got desperate about one problem and I need some help...
I'm using node.js to crawl a list of websites, some of them gives me this error, for example:
http://www.fz-juelich.de/portal/DE/Home/home_node.html, Parse Error, HPE_INVALID_HEADER_TOKEN
request.get({
url: uri,
timeout: timeout,
headers: {
referer: domain
}
}, (error, response, body) => {
if (error)
console.log(error);
console.log(body);
});
though, curl -i --raw http://www.fz-juelich.de/portal/DE/Home/home_node.html
works just perfect
HTTP/1.1 404 Not Found
Server: Apache-Coyote/1.1
Cache-Control: no-cache
JSESSIONID=E594677A6CCA13BE0338E1D00A729C34; Path=/cae:
Content-Type: text/html;charset=utf-8
Content-Language: de
Set-Cookie: JSESSIONID=E594677A6CCA13BE0338E1D00A729C34; Path=/
Content-Length: 19677
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd" >
Also I'm able to see this website in my chrome browser
Any ideas in which side should I dig to get rid of this errors?
I use quotes in properties and that resolve for me :
request.post(url,{
headers: {
'Authorization': 'Basic onEnAGrosEncodedBase64',
'Content-Type': 'application/x-www-form-urlencoded'
},
form: {
'grant_type': 'client_credentials'
}
})
I hope that can help someone ;)
I the end of this journey I no longer use node.js for crawling and parsing
Go lang crawler fits much better here, more flixibility in http library and easier to write really concurrent stuff

Show image with Express, instead of downloading it

A simple server I'm working can serve images. When browsing to the image URL directly, Chrome offers to download the image, rather than just showing it in the browser. Why is that? Presumably it is something in the headers?
The relevant code is this:
tilestore.getTile(req.param("z"), req.param("x"), req.param("y"), function(err, tile) {
if (!err) {
res.send(tile);
} else {
res.send("Tile rendering error: " + err + "\n");
}
});
Do I need to add something to the headers? They are currently as follows:
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Mon, 01 Sep 2014 07:22:30 GMT
Content-Type: application/octet-stream
Content-Length: 37779
Connection: keep-alive
X-Powered-By: Express
ETag: W/"9393-1937584155"
Ok, it's easy. Just set the content-type:
if (!err) {
res.contentType('image/png');
res.send(tile);
} else {
res.send("Tile rendering error: " + err + "\n");
}
These images are always .png.
New headers:
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Mon, 01 Sep 2014 07:33:22 GMT
Content-Type: image/png
Content-Length: 37779
Connection: keep-alive
X-Powered-By: Express
ETag: W/"9393-1937584155"
Just ran into this same problem, and found that modifying the content type alone was not enough. This answer, attempting to do the opposite, provided the solution. I also needed to add a content-disposition header:
res.set("Content-Disposition", "inline;");
Along with setting the correct content type, this allowed me to display my image rather than downloading.
I had the same problem with svg image files and it turned out that I had to change
'content-type': 'image/svg'
to
'content-type': 'image/svg+xml'

Resources