http.StreamedResponse contentLength returns null - .htaccess

I am using flutter http https://pub.dev/packages/http package to download a file from my own webserver.
Below is the download function:
download(String url, {String filename}) async {
var client = http.Client();
var request = new http.Request('GET', Uri.parse(url));
var response = client.send(request);
String dir = (await getApplicationDocumentsDirectory()).path;
List<List<int>> chunks = new List();
int downloaded = 0;
response.asStream().listen((http.StreamedResponse r) {
r.stream.listen((List<int> chunk) {
// Display percentage of completion
debugPrint('downloadPercentage: ${downloaded / r.contentLength * 100}');
chunks.add(chunk);
downloaded += chunk.length;
}, onDone: () async {
// Display percentage of completion
debugPrint('downloadPercentage: ${downloaded / r.contentLength * 100}');
// Save the file
File file = new File('$dir/$filename');
final Uint8List bytes = Uint8List(r.contentLength);
int offset = 0;
for (List<int> chunk in chunks) {
bytes.setRange(offset, offset + chunk.length, chunk);
offset += chunk.length;
}
await file.writeAsBytes(bytes);
return;
});
});
}
But StreamedResponse r is always null when I download the file from my webserver. I can download files from other web servers and it is not null.
For testing purpose, I used the link https://www.rarlab.com/rar/wrar590.exe
Therefore, I think the problem might be because of the settings on my webserver.
From WinRAR server
I/flutter (18386): IOStreamedResponse
I/flutter (18386): ResponseCode: 200
I/flutter (18386): {last-modified: Thu, 26 Mar 2020 10:03:25 GMT, date: Mon, 04 May 2020 11:47:34 GMT, accept-ranges: bytes, strict-transport-security: max-age=31536000;, content-length: 3007728, etag: "2de4f0-5a1bf18799540", content-type: application/octet-stream, server: Apache}
From my webserver:
I/flutter (29858): IOStreamedResponse
I/flutter (29858): ResponseCode: 200
I/flutter (29858): {connection: keep-alive, last-modified: Thu, 16 Apr 2020 20:22:20 GMT, set-cookie: __cfduid=d2789e7dce3f93d32c76e1d7874ffce1b1588599373; expires=Wed, 03-Jun-20 13:36:13 GMT; path=/; domain=.devoup.com; HttpOnly; SameSite=Lax, cf-request-id: 02817fd5a30000ddc752a1d200000001, transfer-encoding: chunked, date: Mon, 04 May 2020 13:36:13 GMT, access-control-allow-origin: *, vary: Accept-Encoding,User-Agent, content-encoding: gzip, cf-cache-status: DYNAMIC, content-type: application/zip, server: cloudflare, accept-ranges: bytes, cf-ray: 58e29c029952ddc7-SIN}

Sometimes the server doesn’t return contentLength . If that’s the case, then contentLength will be null.
If you have control of the server, like in your case, you can set the x-decompressed-content-length header with the file size before you send it. On the client side, you can retrieve contentLength like this:
final contentLength = double.parse(response.headers['x-decompressed-content-length']);
If you don't have control of the server, you can just show the number of bytes that are being downloaded, or if you know the file size in advance, you can use it to help calculate the % number of bytes downloaded.

Added SetEnv no-gzip 1 to .htaccess file in webserver and it returns content-length

Related

Azure Functions: NodeJS - HTTP Response Renders as XML Rather than HTTP Response

I have an Azure function written in NodeJS where I'm attempting to cause an HTTP Redirect with a 302. The documentation is very sparse on what are valid entries in the response. As a result I've created an object with what I feel should be the correct entries to generate the redirect, but all I get is an XML response. Even items like the status code are shown in the XML rather than changing the real status code.
What am I doing wrong?
My Code:
module.exports = function(context, req){
var url = "https://www.google.com";
context.res = {
status: 302,
headers: {
Location: url
}
}
context.done();
}
This is the response I'm getting in the browser:
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 1164
Content-Type: application/xml; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 18 Jan 2017 00:54:20 GMT
Connection: close
<ArrayOfKeyValueOfstringanyType xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays"><KeyValueOfstringanyType><Key>status</Key><Value xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:int">302</Value></KeyValueOfstringanyType><KeyValueOfstringanyType><Key>headers</Key><Value i:type="ArrayOfKeyValueOfstringanyType"><KeyValueOfstringanyType><Key>Location</Key><Value xmlns:d5p1="http://www.w3.org/2001/XMLSchema" i:type="d5p1:string">https://www.google.com</Value></KeyValueOfstringanyType></Value></KeyValueOfstringanyType></ArrayOfKeyValueOfstringanyType>
The problem is that you're not defining the "body" in the response. This can be set to null, but it must be set for Azure functions to interpret it properly.
e.g. Update your code to:
module.exports = function(context, req){
var url = "https://www.google.com";
context.res = {
status: 302,
headers: {
Location: url
},
body : {}
}
context.done();
}
You will then get the desired response:
HTTP/1.1 302 Found
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 0
Expires: -1
Location: https://www.google.com
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 18 Jan 2017 01:10:13 GMT
Connection: close
Edited 2/16/2017 - Using "null" for the body currently throws an error on Azure. As a result the answer was updated to use {} instead.
This is a bug with azure functions, see this content headers issue.
For now, a workaround is to remove any content related headers if the body of your response is null.

Emotion API Project Oxford base64 image

I am trying to make a call to the Emotion Api via JavaScript with in a PhoneGap app. I encoded the image into base64 and verified that the data can be decoded by one of the online tools. this is the code that i found on the web to use.
var apiKey = "e371fd4333ccad2"; //(you can get a free key on site this is modified for here)
//apiUrl: The base URL for the API. Find out what this is for other APIs via the API documentation
var apiUrl = "https://api.projectoxford.ai/emotion/v1.0/recognize";
"file" is the base64 string.
function CallAPI(file, apiUrl, apiKey)
{
// console.log("file=> " +file);
$.ajax({
url: apiUrl,
beforeSend: function (xhrObj) {
xhrObj.setRequestHeader("Content-Type", "application/octet-stream");
xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key", apiKey);
},
type: "POST",
data: file,
processData: false
})
.done(function (response) {
console.log("in call api a");
ProcessResult(response);
})
.fail(function (error) {
console.log(error.getAllResponseHeaders())
});
}
function ProcessResult(response)
{
console.log("in processrult");
var data = JSON.stringify(response);
console.log(data);
}
I got back this:
Expires: -1
Access-Control-Allow-Origin: *
Pragma: no-cache
Cache-Control: no-cache
Content-Length: 60
Date: Fri, 01 Apr 2016 13:34:32 GMT
Content-Type: application/json; charset=utf-8
X-Powered-By: ASP.NET
So i tried their console test page.
https://dev.projectoxford.ai/docs/services/5639d931ca73072154c1ce89/operations/563b31ea778daf121cc3a5fa/console
I can put in an image like your "example.com/man.jpg" and it works great. but if i take the same image and have it encoded as a base 64 image all i get is "Bad Body" i have tried it both as content type "application/octet-stream" and "application/json" and get the same error. sample of the encoded looks like..and http request
POST https://api.projectoxford.ai/emotion/v1.0/recognize HTTP/1.1
Content-Type: application/octet-stream
Host: api.projectoxford.ai
Ocp-Apim-Subscription-Key: ••••••••••••••••••••••••••••••••
Content-Length: 129420
data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAASABIAAD/...
i get back:
Pragma: no-cache
Cache-Control: no-cache
Date: Fri, 01 Apr 2016 16:23:09 GMT
X-Powered-By: ASP.NET
Content-Length: 60
Content-Type: application/json; charset=utf-8
Expires: -1
{
"error": {
"code": "BadBody",
"message": "Invalid face image."
}
}
I am now not sure if you can send an image like this or not from Javascript. Can anyone tell me if my javascript is correct or if they can send an encoded base64 string image to the site.
thanks for your help
tim
This API does not accept data URIs for images. What you'll need to do is convert it to a binary blob. Though this answer is for a different Project Oxford API, you can apply the same technique.

Node js with express how to enable cache for static files

I have tried all the possibilities. Following are my response headers
HTTP/1.1 200 OK
Expires: Sun, 20 Dec 2015 03:53:42 GMT
Accept-Ranges: bytes
Cache-Control: public, max-age=604800
Last-Modified: Wed, 02 Dec 2015 15:49:21 GMT
Etag: W/"25571-15163623fb3"
Content-Type: image/png
Content-Length: 152945
Date: Sun, 13 Dec 2015 03:53:42 GMT
Connection: keep-alive
But still it's not returning 304 status. If I refresh command refresh in Firefox it's downloading that image again with 200 response code. I don't want that if user click command refresh it's should return 304 status. instead of downloading new file it should take it from browser cache.
Following is my nodejs code
var expireTime = 604800000;
app.all('/css*', function(req, res, next) {
res.header('Expires', new Date(Date.now() + expireTime).toUTCString());
next();
});
app.use('/css',express.static(conf.nodePath + 'css',{ maxAge: expireTime}));
Can anyone guide me how to enable perfect cache. Thanks in advance..

Exact copy of http gzipped response into a string

I need help.
I'm trying to get content of website where Content-encoding is gzip, with dmd v2.066.1 on Windows.
This is my web address for test: "http://diaboli.pl/test2.html".
My HTTP request is:
GET /test2.html HTTP/1.1
Host: diaboli.pl
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: pl,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
User-Agent: My Browser
Referer: http://google.pl
DNT: 1
The server response is:
HTTP/1.1 200 OK
Date: Sat, 24 Jan 2015 23:02:00 GMT
Server: Apache
Last-Modified: Sat, 24 Jan 2015 22:48:44 GMT
ETag: "5c468ad-83f-50d6db511eb00"
Accept-Ranges: bytes
Vary: Accept-Encoding,User-Agent
Content-Encoding: gzip
Content-Length: 942
Content-Type: text/html
.)┘R!SĽ╣ň┌KRB:éş^»{█ĺ.ç}aOě_DźŢ░▼'dĘ$ëĚk\|j\pý§Ěí▀k║Ź■ß♠┐}ú2žŢ ´dĹĺńMłÎ▒└╚‼/§B⌂Ĺ▬°'˘uŕNá☺■█Ór↓m(┘đ▬Ţ┼ńĺ╦⌂
§gŰůqýä╗˘%p▬■&B♂M]§Üú3ý^ý-ÎD`x!Ő╔&M♥~╣y╬uşëňZ#▒]˘ä2}Ś╣xdÄyWüm§?ąě░Äd4,d‼î-▬
┬♣Bön°6{őu└♀☺█UĂ└,aF˘├☼☻OŔ˛mţË▄▀Čó¸ö31ÎňEÖKŮţĄîÔŐ←ôň¸HÉ┌bŤ}Dnń'ń9┌
Îă♠¶U♣VI^▲hËőŃ└_zďĆ6┬6█¨}{╝╦ÄřeđŠoŤčů¤űU´öěŁ*ŠxĂ☻(,─AôlZ»Ú^ß溸ő╬↓M`¬PË═qí¨Ýç▼7╣§y♫<J╬ÓŇëb#PćR§bˇĽ>Ěz╣┴âž7uř┐ `$SřítR¶╗u ź☻‼ĘXçf☺°NH▄˛☻ şp─R­Ą►¬w╬\758GN║K) ;ĺ\ÝŇľ♫╩┼╬|ABYÍţ∟═Yů+╔y?ťkVĐ┼
nş║☼jv¶ĐSô9Dů♠▓Ç˙üK╬2\˝d[☼ <ľ┘Ń↓ü╠âG ˇ¸
ľyŇđd■ß▲e☼¸♣e_ÂśúQ÷śń,ÖŬ[N╝b┼Ř└ŕ↓ÚcS┴3╗╠w▀[ş↕ĺŽCňđś↕⌂═őç˛ţHW∟d=╩║Y►│Ô]sČšX§_ˇ↔ĹCČŤI┬y┤ŕ▲╬Ő↕╩§┌}í m\∟Öç#<W*Ű┐h˘g2SęćĐqš►EËý üXđ.S▀kš2←↑►â☼Ň5Ę╬♀6∟\←B|fđşÚ*ZŽ%▀Î↓#ěEŕ♦TNgcż,→‼│→p-←î˘ă☻p$Ř%ôe
♠♀ŻýŁ8JiŔ▒"L■♀óą↨Č┘´☻«┌:ŰńĹ>♣§╝×░♂öĄT`=BÂ|5mˇ|Ňs)ŐRĹ═▒é┴\yru▬ć=Rďĺ]↔ŰýÉĆ☼─ć↑¬pZÇ▓9PC§ę4 ×#ş Ź☺╬ňLj█Á¨uĄ:│§Bšš∟ďŃ?▼nvO!0↔}î*╠aŢ ţh
Ľ*7Îĺ$vn ŔIŘM¸♀˙¶ÎŞŞb⌂♫äý"´♂çK}⌂Y♀ ♣XŽëM
As You can see, it's a gzip encoded content. The server response is printed out the cmd console with the write() function, character by character.
The problem is, I can't make the exact copy of the response string. If I try, I got this result:
HTTP/1.1 200 OK
Date: Sat, 24 Jan 2015 23:02:00 GMT
Server: Apache
Last-Modified: Sat, 24 Jan 2015 22:48:44 GMT
ETag: "5c468ad-83f-50d6db511eb00"
Accept-Ranges: bytes
Vary: Accept-Encoding,User-Agent
Content-Encoding: gzip
Content-Length: 942
Content-Type: text/html
▼ő
I can determine the length of the content, and it is equal as the HTTP Content-Length header value, but I can see, it's not the same string as the one by one original.
It's also interesting, that I can decompress that bad content string with zlib uncompress() function, and it doesn't return the zlib data error, but the cutted decompressed content.
Ofcourse, the browsers like FF or IE displays the complete decompressed content without problems.
I'm connecting to the server like this:
import std.stdio, import std.string, std.conv, std.socket, std.stream, std.socketstream, std.zlib;
ushort port=80; string domain="diaboli.pl";
string request_uri; int[] pos; string request; string buffer; string znak; string line;
int contentlength=-1; int[] postab; string bodybuffer; string headerbuffer; int readingbody=0;
std.zlib.UnCompress u; const(void)[] udata;
Socket sock = new TcpSocket(new InternetAddress(domain, port));
Stream ss = new SocketStream(sock);
request="GET " ~ request_uri ~ " HTTP/1.1\r\n";
request~="Host: " ~ domain ~ "\r\n";
request~="Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n";
request~="Accept-Language: pl,en-US;q=0.7,en;q=0.3\r\n";
request~="Accept-Encoding: gzip, deflate\r\n";
request~="User-Agent: My Browser\r\n";
request~="Referer: http://google.pl\r\n";
request~="DNT: 1\r\n";
request~="\r\n";
writeln("HTTP request:\n---");
writeln(request);
writeln("---");
ss.writeString(request);
writeln("\nAll response from the server character by character:\n---");
line="";
while (1)
{
if (readingbody==1) readingbody=2; //the way to separate headers and the content - first part.
znak = to!string(ss.getc());
if (ss.eof()) break;
line~=znak;
//if (readingbody==2)
write(znak);
if (znak=="\n")
{
if (strpos(line,"Content-Length: ")>-1)
{
postab ~= strpos(line,"\r");
postab ~= strpos(line,"\n");
contentlength=to!int(substr(line,16,postab.sort[0]-16));
}
if (readingbody==0 && line=="\r\n") readingbody=1;
line="";
}
buffer ~= znak;
//the way to separate headers and the content - second part.
if (readingbody==0 && line=="\r\n") readingbody=1;
if (readingbody==2) bodybuffer ~= znak;
else headerbuffer ~= znak;
}
sock.close();
writeln("\n---");
write("Content-Length="); writeln(contentlength); //This is the Content-Length determined from the HTTP Content-Length header.
write("bodybuffer.length="); writeln(bodybuffer.length); //This the length of the content string
writeln("\nAll response copied into the string:\n---");
writeln(buffer);
writeln("---\nOnly content:\n---");
writeln(bodybuffer);
writeln("---\nUncompressed:\n---");
u = new UnCompress(HeaderFormat.determineFromData);
udata = u.uncompress(bodybuffer);
writeln(cast(string)udata);
//These are my simple text processing functions similar to php.
int strpos(string str,string tofind,int caseinsensitive=0)
{
int pos=-1;
if (caseinsensitive==1)
{
str=toUpper(str);
tofind=toUpper(tofind);
}
if (str.length>=tofind.length)
{
for(int i=0;i<str.length;i++)
{
if (i+tofind.length>str.length) break;
if (str[i..i+tofind.length]==tofind)
{
pos=i;
break;
}
}
}
return pos;
}
string substr(string str,int pos, int offset)
{
string substring="";
if (str.length>0 && pos>-1 && offset>0)
{
substring=str[pos..pos+offset];
}
return substring;
}
There are three problems with your code:
You use Stream.getc, which does newline conversions. This will corrupt binary data. You can fix this by replacing:
znak = to!string(ss.getc());
with:
char c; ss.readBlock(&c, 1); znak = to!string(c);
Although it's better to avoid std.stream entirely, it is ancient code waiting to be replaced.
You specify a HTTP version of 1.1, so the server sends back the conent with Transfer-Encoding: chunked. Your program cannot handle this transfer encoding. You can change the protocol version to 1.0.
When using the std.zlib classes, you must call flush after piping through all the data. Add this line:
udata ~= u.flush();
With these changes, your program works fine for me.

Show image with Express, instead of downloading it

A simple server I'm working can serve images. When browsing to the image URL directly, Chrome offers to download the image, rather than just showing it in the browser. Why is that? Presumably it is something in the headers?
The relevant code is this:
tilestore.getTile(req.param("z"), req.param("x"), req.param("y"), function(err, tile) {
if (!err) {
res.send(tile);
} else {
res.send("Tile rendering error: " + err + "\n");
}
});
Do I need to add something to the headers? They are currently as follows:
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Mon, 01 Sep 2014 07:22:30 GMT
Content-Type: application/octet-stream
Content-Length: 37779
Connection: keep-alive
X-Powered-By: Express
ETag: W/"9393-1937584155"
Ok, it's easy. Just set the content-type:
if (!err) {
res.contentType('image/png');
res.send(tile);
} else {
res.send("Tile rendering error: " + err + "\n");
}
These images are always .png.
New headers:
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Mon, 01 Sep 2014 07:33:22 GMT
Content-Type: image/png
Content-Length: 37779
Connection: keep-alive
X-Powered-By: Express
ETag: W/"9393-1937584155"
Just ran into this same problem, and found that modifying the content type alone was not enough. This answer, attempting to do the opposite, provided the solution. I also needed to add a content-disposition header:
res.set("Content-Disposition", "inline;");
Along with setting the correct content type, this allowed me to display my image rather than downloading.
I had the same problem with svg image files and it turned out that I had to change
'content-type': 'image/svg'
to
'content-type': 'image/svg+xml'

Resources