One page with 5 esi:include, each src is one same path with different params like 'm.htm?p=1' & 'm.htm?p=2', but I used varnishlog and see only 1 esi request happended, so don't I use one same path in more than one esi:include tag?
Varnish supports, by default, n esi:include directives per resource, and up to 5 levels of ESI recursion per client request, but that's configurable. Here's a list of steps I would take to debug:
Check to make sure there are no <esi:include /> tags in the response delivered to your client from varnish. Then make sure your back end is generating as many <esi:include /> tags as you expect by hitting it directly. Varnish is to blame only if they exist and varnish is not parsing them. Make sure there are no typos, etc such as <es:include .../>
Request the src urls from the esi include tags directly to your backend server. Does the server error out? The default behavior is to ignore errors on ESI requests, and I don't know if varnishlog will report them in this case.
Check the response headers returned for the esi fragments. If they are also cacheable, varnish will not request them from the backend again until they expire and may not log them in varnishlog (check the docs).
Make sure that requests with different query strings hash to different cache keys. If your VCL attempts to increase hit rate by ignoring query strings, your ESI scheme won't work since each tag is logically including the same cached resource.
Without knowing if you're just concerned with varnishlog output or whether there are actually symptoms of esi failure in the page itself that's all I can muster.
Related
According to : https://developers.docusign.com/docs/esign-rest-api/reference/envelopes/envelopedocuments/list/
The EnvelopeDocument model should have "sizeBytes" property on it.
However, I'm making calls to GET https://demo.docusign.net/restapi/v2.1/accounts/{{accountId}}/envelopes/{{envelopeId}/documents
and the payload does not have it.
I also tried hitting HEAD https://demo.docusign.net/restapi/v2.1/accounts/{{accountId}}/envelopes/{{envelopeId}/documents/{{documentId}
to see if I could read the headers without the actual content but that I receive a 404 NOT FOUND so it must not be supported.
The only way I can see to accomplish this would be to hit the download file endpoint directly, reading the content-length header, and then cancelling the request before streaming any data from it.
Is there a better way to do this?
The only way to do this would be to make a get envelope call
GET https://demo.docusign.net/restapi/v2.1/accounts/{{accountId}}/envelopes/{{envelopeId}/documents
Then in the headers, you will find Content-Length, this is the file size
Reached out to DocuSign and they confirmed there's no way to do this other than to hit the download endpoint and read the content length header.
After some experimentation i found this doesnt really work either because the Content Length header is not always accurate from their api. The only way I can see to do it is to download the entire file into memory and take a count of the bytes, or keep track of the byte count as you stream it to its destination.
I have a rest like API through Node Express.
The ETag is default, not explicitly turned on or off. However whenever I test hitting the server, it always gives me a new ETag, even if the returned JSON/HTML is exactly the same. I also checked the returned header and they look the same. I tested this with two types of content, an API and a static HTML content like a privacy page.
Any idea how to check what's making it different each time?
Express' default behavior is to provide a "strong"-ly validated etag which will only be the same as a previous response if the current response is precisely the same, byte-for-byte.
You could try setting express' etag to weally validate the response, which indicates to the browser that the current response is semantically equivalent as a previous one with the same value, that is, while they might not be byte-for-byte the same, they encapulate or represent the same meaning. To do this, use app.set('etag','weak')
Finally, if this doesn't work for you, you can create your own etag validation function using app.get('etag',function(body,encoding){...}) where you return a hash generated from your content; this allows you to control what express (and thus, the browser) considers being different means in the context of your response.
More than you ever wanted to know about etags can be found at Wikipedi:HTTP_ETag
I'd like to exclude certain pages from the Varnish cache based on the content of the page (for instance if the Form uses a particular hidden field which is a security feature and needs to be unique on every page refresh).
I have dozens of forms, so I don't want to have to exclude each unique page individually from the cache.
Is this possible within the VCL?
No, normally not. The proper way to do it would be to set cache-headers (for instance "Cache-Control: no-cache, must-revalidate") on your pages with the non-cacheable forms that varnish in turn will read.
As a nice side effect that will also cancel most client side caches that also often can cause troubles with CAPTCHAs and the like.
I've got a standard blog-type application with posts and users that can add those posts to their favorites.
Goals
When a user looks at a list of posts, they should see an indication (an image) of whether
each post is a favorite. Anonymous users don't have any favorites.
The list of posts needs to be cached in Varnish (for both anonymous and logged-in users) because it's expensive to calculate.
Ideas
Cache the list page in Varnish and use ESI to fetch the favorites information...
... for each post for the user making the current request. Downside: 50 ESI requests per page (basically the N+1 problem).
... as a JSON object which is then stored on the page. On the client, this object is read and the DOM is manipulated to indicate favorites information. Downside: doesn't work for users without Javascript.
... as a CSS snippet which is stored in the page. The CSS determines what to display for each post. Downside: only works for stylable content (ie, images). Not possible to display text information.
Am I missing any possibilities to accomplish what I want? Idea 3 seems to be the cleverest answer, but it wouldn't work if I also wanted to display the date the user favorited the post.
Answer 2 makes a lot of sense. It makes pages nicely cacheable, and only sacrifices the 'favorite functionality' for people without javascript in their browser.
Who are those people anyway? Still surfing with lynx? ;). And would they accept cookies to make your login mechanism (required for personal favorites) work in the first place, or even login at all?
Is there any data regarding how browsers actually support rest http verbs (especially PUT, DELETE). This question is mostly motivated by the fact that many sources (such as this stackoverflow answer) inform that most browsers don't suport PUT and DELETE but don't say which.
Rails solves this using a patch on the client, and reversing the patch on the server, but I wonder, for instance, which browsers wouldn't need such hack.
When in doubt, ask Anne:
http://annevankesteren.nl/2007/10/http-method-support
It's a couple of years old, but it gives a clean bill of health to PUT and DELETE, but it also tries different cases of more obscure verbs like TRACE and PROPFIND:
Van Kesteren tested Firefox 3, Opera 9.5 and Internet Explorer 7, a mix of pre-release and releases. GET and POST worked as expected, and all supported the standard set of http methods from RFC 2616. There are some casing issues, so you should try to stick with uppercase always; TRACE and OPTIONS are edge cases with problems found in Firefox and Opera, respectively. CONNECT and TRACE and SEARCH are problematic for IE. Bear in mind that this was written in 2007 so YMMV.
I think the whole confusion over browsers not supporting other than GET and POST stems from the fact that the HTML specification lists only these two in the HTML <form> element:
The method attribute of the FORM element specifies the HTTP method used to send the form to the processing agent. This attribute may take two values:
get: With the HTTP "get" method, the form data set is appended to the URI specified by the action attribute (with a question-mark ("?") as separator) and this new URI is sent to the processing agent.
post: With the HTTP "post" method, the form data set is included in the body of the form and sent to the processing agent.
This is the reason why browsers only support GET and POST natively in HTML; since the standard says so.
The current (January 2014) XMLHttpRequest specification on the other hand states explicitly that request methods should be allowed through with some exceptions (CONNECT, TRACE or TRACK), and that RFC2616 methods should be uppercased:
\6. If method is a case-insensitive match for CONNECT, DELETE, GET, HEAD, OPTIONS, POST, PUT, TRACE, or TRACK, subtract 0x20 from each byte in the range 0x61 (ASCII a) to 0x7A (ASCII z).
If it does not match any of the above, it is passed through literally, including in the final request.
EDIT Another source that has come to more or less the same conclusion: That XMLHttpRequest supports PUT and DELETE: http://jshirley.vox.com/library/post/xmlhttprequest-and-rest.html (warning, broken link as of May 2014).
EDIT The question has been asked before of course, but the two year old answers are still valid.
EDIT Short resume of Anne van Kesteren's page from 2007, and added section on HTML vs XMLHttpRequest
See the PutDeleteSupport page of the Atom Wiki. You might be interested in browsing other Atom Wiki pages related to HTTP.