Is it possible to download a file nested in a zip file, without downloading the entire zip archive?
For example from a url that could look like:
https://www.any.com/zipfile.zip?dir1\dir2\ZippedFileName.txt
Depending on if you are asking whether there is a simple way of implementing this on the server-side or a way of using standard protocols so you can do it from the client-side, there are different answers:
Doing it with the server's intentional support
Optimally, you implement a handler on the server that accepts a query string to any file download similar to your suggestion (I would however include a variable name, example: ?download_partial=dir1/dir2/file
). Then the server can just extract the file from the ZIP archive and serve just that (maybe via a compressed stream if the file is large).
If this is the path you are going and you update the question with the technology used on the server, someone may be able to answer with suggested code.
But on with the slightly more fun way...
Doing it opportunistically if the server cooperates a little
There are two things that conspire to make this a bit feasible, but only worth it if the ZIP file is massive in comparison to the file you want from it.
ZIP files have a directory that says where in the archive each file is. This directory is present at the end of the archive.
HTTP servers optionally allow download of only a range of a response.
So, if we issue a HEAD request for the URL of the ZIP file: HEAD /path/file.zip we may get back a header Accept-Ranges: bytes and a header Content-Length that tells us the length of the ZIP file. If we have those then we can issue a GET request with the header (for example) Range: bytes=1000000-1024000 which would give us part of the file.
The directory of files is towards the end of the archive, so if we request a reasonable block from the end of the file then we will likely get the central directory included. We then look up the file we want, and know where it is located in the large ZIP file.
We can then request just that range from the server, and decompress the result...
Related
I would like to directly compress a directory into a Poco::HTTPServerResponse stream. However, downloading the zip file produced by the following code leads to a corrupt archive. I do know that the below compression approach does work for locally created zip files as I have successfully done that much. What am I missing or is this simply not possible? (Poco v1.6.1)
std::string directory = "/tmp/data";
response.setStatusAndReason(HTTPResponse::HTTPStatus::HTTP_OK);
response.setKeepAlive(true);
response.setContentType("application/zip");
response.set("Content-Disposition","attachment; filename=\"data.zip\"");
Poco::Zip::Compress compress(response.send(),false);
compress.addRecursive(directory,
Poco::Zip::ZipCommon::CompressionMethod::CM_STORE,
Poco::Zip::ZipCommon::CompressionLevel::CL_MAXIMUM,
false, "data");
compress.close();
I use the same technique successfully, with only a slight difference:
The compression method and the compression level (CM and CL).
compress.addFile( cacheFile, Poco::DateTime(), currentFile.GetName(), Poco::Zip::ZipCommon::CM_DEFLATE, Poco::Zip::ZipCommon::CL_SUPERFAST );
A zip file corresponds to the DEFLATE algorithm, so when unzipping, your explorer/archive manager probably doesn't work out.
Either that, or it's pointless to use a MAXIMUM level on a STORE method (STORE non compressing by definition).
EDIT: Just tried it, actually, it's because CM_STORE internally uses headers (probably some kind of tar). Once your files have been added to the zip stream and you close it, Poco tries to order the header, and resets the position of the output stream to the start to write them.
Since it cannont be done on the HTTP output stream (your bytes are already sent!), it fails.
Switching to CM_DEFLATE should fix your problem.
I am using streams to download the excel file. Please find the code below
$response = new \Zend\Http\Response\Stream();
$response->setStream(fopen($myfile, 'r'));
$response->setStatusCode(200);
$headers = new \Zend\Http\Headers();
$headers->addHeaderLine("Content-Type: application/vnd.ms-excel; charset=UTF-8")
->addHeaderLine('Content-Disposition', 'attachment; filename=my.xls')
->addHeaderLine( "Content-Transfer-Encoding: binary" )
->addHeaderLine('Content-Length', filesize($myfile));
$response->setHeaders($headers);
The file generated is proper but when the same file I am enforcing the user to download there are two empty lines coming in the downloaded excel file. I did some researching and thought may it's http version and header lines but I think it's not because if I try otherwise I get those two empty line in start of excel report.
Please note there are no empty space in the start of the content.
Any idea ?? why it's so?
Mind the streamed response does in fact not really provide a stream context to the client. It just buffers the stream internally and sends out the response in one go.
That being said, I have created a controller plugin to send attachments from a file path or directly with the binary data addressed to a variable. It's in my common Soflomo\Common library. I haven't had the issues you described and I use some more headers than you do.
Tell me if that piece of code works for you. One of the differences is you use the size of the original file as the size of the response. I am not sure, as this might cause an indifference with the cached streamed response. Try to just grab the contents and do a strlen() on this content.
I could not believe this: it seems that the zip specification does not allow two different files with the same file name going into one zip file.
In my case I use an external file to specify all the files I wanna zip.
This could look like this:
../Website1/favicon.ico
../Website2/favicon.ico
and there we are, that's not possible, despite keeping the directory structure. You would expect the name to be <../Website1/favicon.ico> rather than but that does not seem to be the case, I get:
"Invalid ZIP request (cannot repeat names in Zip file)"
with WinZip. I tried the same with 7Zip - same result.
Strangely googling did not show many hits that really fit but those I found seem to confirm my findings. That's hard to believe since this limitation is very severe. I actually struggle to understand why this did not hit me a couple of decades earlier.
Am I overlooking something very basic here?
To be precise:
Adding these two files:
C:\Temp\Website1\FavIcon
C:\Temp\Website2\FavIcon
results in a single file; the last Add wins...
This however:
Website1\FavIcon
Website2\FavIcon
results in a zip file that contains both files.
I have a Java Web App running on Tomcat on which I'm supposed to exploit Path traversal vulnerability. There is a section (in the App) at which I can upload a .zip file, which gets extracted in the server's /tmp directory. The content of the .zip file is not being checked, so basically I could put anything in it. I tried putting a .jsp file in it and it extracts perfectly. My problem is that I don't know how to reach this file as a "normal" user from browser. I tried entering ../../../tmp/somepage.jsp in the address bar, but Tomcat just strips the ../ and gives me http://localhost:8080/tmp/ resource not available.
Ideal would be if I could somehow encode ../ in the path of somepage.jsp so that it gets extracted in the web riot directory of the Web App. Is this possible? Are there maybe any escape sequences that would translate to ../ after extracting?
Any ideas would be highly appreciated.
Note: This is a school project in a Security course where I'm supposed to locate vulnerabilities and correct them. Not trying to harm anyone...
Sorry about the downvotes. Security is very important, and should be taught.
Do you pass in the file name to be used?
The check that the server does is probably something something like If location starts with "/tmp" then allow it. So what you want to do is pass `/tmp/../home/webapp/"?
Another idea would be to see if you could craft a zip file that would result in the contents being moved up - like if you set "../" in the filename inside the zip, what would happen? You might need to manually modify things if your zip tools don't allow it.
To protect against this kind of vulnerability you are looking for something like this:
String somedirectory = "c:/fixed_directory/";
String file = request.getParameter("file");
if(file.indexOf(".")>-1)
{
//if it contains a ., disallow
out.print("stop trying to hack");
return;
}
else
{
//load specified file and print to screen
loadfile(somedirectory+file+".txt");
///.....
}
If you just were to pass the variable "file" to your loadfile function without checking, then someone could make a link to load any file they want. See https://www.owasp.org/index.php/Path_Traversal
I am building the server part of a webapp, using node.js. This involves getting data from thetvdb.com (API documentation of thetvdb).
The data comes as a zip file. HTTP download is no problem, however, parsing the file is. I actually never save the file, but just keep it in memory, as suggested in How to download and unzip a zip file in memory in NodeJs?
I have a buffer with valid data (same data as when I download the file with browser/curl...). However, adm-zip (I also tired other zip libraries, some suggest invalid zip length) can't open it. It does not show an error, but the zipEntries in the end have length of 0.
When I write out the buffer to the filesystem and open it with gui or cli tools it works.
I can't give a direkt link to the file, as it would involve my API key, however I re-uploaded it here.
I think I might have an answer for you:
Don't rely on npm install. I just ran the example that you linked to with the zip file you provided, and I get an output of "0".
I saw a comment on that other StackOverflow page, saying that the version of adm-zip on npm is not up to date. I grabbed a fresh copy of adm-zip from github, overwrote the one in my node_modules folder and reran the example code and now get the following:
...
<Actor>
<id>237811</id>
<Image>actors/237811.jpg</Image>
<Name>Peter Pratt</Name>
<Role>The Master</Role>
<SortOrder>3</SortOrder>
</Actor>
<Actor>
<id>23780s/237811.jpg</Image>
Give that a shot!