I'd like to hide my image, so I load images using ASP script. But I see the images are not cached, is there a way to cache this kind of images?
My code:
Response.Buffer = True
response.Expires=240
response.CacheControl="Public"
Response.ContentType = "image/jpeg"
Set http = Server.CreateObject("MSXML2.ServerXMLHTTP")
http.open "GET","http://www.test.net/images/" & request.querystring("ID"), False
http.Send
Response.BinaryWrite http.ResponseBody
Set http = Nothing
Response.Flush
Response.End
This file is named test.asp and I call it via test.asp?ID=12345, like that:
<img src="test.asp?ID=12345">
The firebug shows OK for the image to be loaded not 302 not modified.
Is it possible to cache such kind of images?
There is no reason that ASP page can not be cached, since browser does not know which server side technology generated it. You only need to set proper HTTP header directives. In your case, at the minumum, you need to set Max-Age in the Cache Control directive, for example:
response.CacheControl="public, max-age=68400"
See following article for other useful directives for HTTP case.
Related
Is it possible to cache a file in the browser regardless of its query string parameters?
For instance loading the same cached assets for both:
https://example.com?value=foo
https://example.com?value=bar
Yes, it's part of the options available to the Cache's .match(request, options) method: ignoreSearch.
So when checking your Cache, set this option to true and both requests will be treated the same.
I have a download GET endpoint in my express app. For now it simply reads a file from the file system and streams it after setting some headers.
When i open the endpoint in Chrome, I can see that this is treated as a "document", while in Firefox it is being treated as type png.
I can't seem to understand why it is being treated differently.
Chrome: title bar - "download"
Firefox: title bar - "image name"
In Chrome, this also leads to no caching of the image if I refresh the address bar.
In Firefox it is being cached just fine.
This is my express code:
app.get("/download", function(req, res) {
let file = `${__dirname}/graph-colors.png`;
var mimetype = "image/png";
res.set("Content-Type", mimetype);
res.set("Cache-Control", "public, max-age=1000");
res.set("Content-Disposition", "inline");
res.set("Vary", "Origin");
var filestream = fs.createReadStream(file);
filestream.pipe(res);
});
Also attaching images for Browser network tabs.
This are all to do with the behaviors of Chrome, you can test on another site like Example.png on Wikipedia.
Chrome always treats the "thing" you opened in the address bar as document, ignoring what it really is. You can even test loading a css and it will read document.
For title, it reads download because your path is /download, you cannot change it according to this SO thread.
For caching, Chrome apparently ignores the cache when you are reloading, anything, page or image. You can try using the Wiki example.png, you will get 304 instead of "(from cache)". (304 means the request is sent, and the server has implemented ETag, if-none-match or similar technique)
I'm developing a web crawler in nodejs. I've created a unique list of the urls in the website crawle body. But some of them have extensions like jpg,mp3, mpeg ... I want to avoid crawling those who have extensions. Is there any simple way to do that?
Two options stick out.
1) Use path to check every URL
As stated in comments, you can use path.extname to check for a file extension. Thus, this:
var test = "http://example.com/images/banner.jpg"
path.extname(test); // '.jpg'
This would work, but this feels like you'll wind up having to create a list of file types you can crawl or you must avoid. That's work.
Side note -- be careful using path. Typically, url is your best tool for parsing links because path is aimed at files/directories, not urls. On some systems (Windows), using path to manipulate a url can result in drama because of the slashes involved. Fair warning!
2) Get the HEAD for each link & see if content-type is set to text/html
You may have reasons to avoid making more network calls. If so, this isn't an option. But if it is OK to make additional calls, you could grab the HEAD for each link and check the MIME type stored in content-type.
Something like this:
var headersOptions = {
method: "HEAD",
host: "http://example.com",
path: "/articles/content.html"
};
var req = http.request(headersOptions, function (res) {
// you will probably need to also do things like check
// HTTP status codes so you handle 404s, 301s, and so on
if (res.headers['content-type'].indexOf("text/html") > -1) {
// do something like queue the link up to be crawled
// or parse the link or put it in a database or whatever
}
});
req.end();
One benefit is that you only grab the HEAD, so even if the file is a gigantic video or something, it won't clog things up. You get the HEAD, see the content-type is a video or whatever, then move along because you aren't interested in that type.
Second, you don't have to keep track of file names because you're using a standard MIME type to differentiate html from other data formats.
I'm redesigned a login system where I work and currently if someone know the path to a file they may be able to access it even if they are not logged in at all. So far I've researched and come up with 2 ways to prevent this.
Disable anonymous access and have the file brought in by the webpage. This is what I would prefer to do for now since it wouldn't require moving the files around and it seems to be relatively easy to implement. The problem with this is that if I disable anonymous access the file trying to access the document they requires the username/password. Is it possible to have to username/password as part of the ASP file to eliminate this?
The other option is to move the files outside of the website and elsewhere on the server and have the webpage bring in the file similar the first option. I want to eventually get to this method, currently it would take much more time to do this though since we would have to move the files for all of our users as well as change the programs that generate the different reports to output to their new locations.
You could do this with a rewrite rule in IIS.
http://www.iis.net/downloads/microsoft/url-rewrite
You can also do this with a Request Filtering rule in IIS.
http://www.iis.net/configreference/system.webserver/security/requestfiltering/filteringrules/filteringrule/appliesto
"Is it possible to have to username/password as part of the ASP file to eliminate this?"
Yes, if you only want to restrict access to .asp files Session variables tend to be overused in Classic ASP but this is one situation where it is completely appropriate to use one. At the start of all your restricted pages you could have something like
<% If Session("loggedin") <> 1 then Response.Redirect("login.asp") End If %>
Then you need to find a classic asp login script which sets a session variable. Google or Bing should come up with plenty, but here are a couple of links for you
https://web.archive.org/web/20211020121723/https://www.4guysfromrolla.com/webtech/050499-1.shtml
https://web.archive.org/web/20210323190338/http://www.4guysfromrolla.com/webtech/021600-1.shtml
Edit - http server request code. I haven't tested this
Set xml = Server.CreateObject("Msxml2.ServerXMLHTTP.6.0")
xml.open "GET","http://fullurlto/yourpdffile.pdf", false, "yourusername", "yourpassword"
xml.send
Response.ContentType = "application/pdf"
Response.write xml.responseText
I found a basis for the code elsewhere on this site but it didn't quite work the way I needed it to.
How To Raise a "File Download" in ASP and prevent hotlink
Dim curUser, curDir, curtype
curUser = Request.QueryString("user")
curDir = Request.QueryString("dir")
curType = Request.QueryString("type")
If curUser = Session("homefolder") Then
Set adoStream = CreateObject("ADODB.Stream")
adoStream.Type = 1
adoStream.Open()
adoStream.LoadFromFile "C:/path/to/" & curUser & curDir
Response.Buffer = true
Response.CharSet = "UTF-8"
Response.Clear
If curType = ".TXT" Then
Response.ContentType = "text/plain"
Else
Response.ContentType = "application/pdf"
End If
Response.flush
Do While Not adoStream.eos
Response.BinaryWrite adoStream.Read(1024 * 8)
Response.flush
Loop
Response.End()
adoStream.close
Set adoStream=nothing
Else
Response.Redirect "denied.asp"
End If
The file is brought in and displayed within the browser. If the user tries to see another user's files they are simply redirected. I'm currently only dealing with PDF and TXT file but it will be easy to add new files types if needed.
I'm migrating a series of websites from an existing IIS5 server to a brand new IIS7 web server. One of the pages pulls a data file from a blob in the database and serves it to the end user:
Response.ContentType = rs("contentType")
Response.AddHeader "Content-Disposition", "attachment;filename=" & Trim(rs("docName"))&rs("suffix")' let the browser know the file name
Response.AddHeader "Content-Length", cstr(rs("docsize"))' let the browser know the file size
Testing this in the new IIS7 install, I get a "Connection Reset" error in both Internet Explorer and Firefox. The document is served up correctly if the Content-Length header is removed (but then the user won't get a useful progress bar).
Any ideas on how to correct this; whether it be a server configuration option or via code?
Edit 1: Did a bit more trial and error. The requests will succeed if both "Enable Buffering" and "Enable Chunked Encoding" are false. If either one is enabled the error occurs.
Edit 2: More trial and error testing; turns out that text files will work fine with the script; only binary files (images, pdfs, etc.) will fail. Still completely clueless otherwise.
As already mentioned somewhere else: http://en.wikipedia.org/wiki/Chunked_transfer_encoding
It uses the Transfer-Encoding HTTP response header in place of the Content-Length header, which the protocol would otherwise require. Because the Content-Length header is not used, the server does not need to know the length of the content before it starts transmitting a response to the client (usually a web browser). Web servers can begin transmitting responses with dynamically-generated content before knowing the total size of that content.
In IIS7 this is enabled by default:
http://technet.microsoft.com/en-us/library/cc730855(v=ws.10).aspx
To enable HTTP 1.1 chunked transfer encoding for the World Wide Web
publishing service, use the following syntax:
appcmd set config /section:asp /enableChunkedEncoding:True|False
True enables HTTP 1.1 chunked transfer encoding whereas False disables
HTTP 1.1 chunked transfer encoding. The default value is True.
We had the same problem, our solution: remove AddHeader "Content-Length"
There are two options to make it work:
Output the "Content-Size" header, instead of "Content-Length". Note not all clients will recognise that, but at least it works.
(Preferred) Set Response.Buffer to True, then you can use the "Content-Length" header, and handle the "chunking" yourself (thus not taxing the ASP memory buffer):
The following works for me on IIS7, and seems to send file size info correctly to the browser.
Response.Buffer = True
Response.ContentType = "application/pdf"
Response.AddHeader "Content-Disposition", "attachment; filename=""yourfile.pdf"""
Set objStream = Server.CreateObject("ADODB.Stream")
objStream.Open
objStream.Type = adTypeBinary
objStream.LoadFromFile "yourfile.pdf"
Response.AddHeader "Content-Length", objStream.Size
' Send file in chunks. '
lByteCount = 0
lChunkSize = 100000
While lByteCount < objStream.Size
If lByteCount + lChunkSize > objStream.Size Then lChunkSize = objStream.Size - lByteCount
Response.BinaryWrite objStream.Read(lChunkSize)
Response.Flush ' Flush the buffer every 100KBytes '
lByteCount = lByteCount + lChunkSize
Wend
objStream.Close
Set objStream = Nothing
Encountered this same issue when migrating ASP code from a Windows 2003 server to Windows 2012 with IIS 8.5. The fix was to adjust the ASP code as follows:
From:
Response.Addheader "Content-Length", Size
To:
Response.Addheader "Content-Size", Size