Nginx not Caching my Application - node.js

I'm trying to setup the caching of my ExpressApp and its rendered Jade files with Nginx.
When I have a look at the pm2 logs of my application while running the google Pagespeed Insight test, I see that the Images and HTML are still served by the app instead of the Nginx proxy.
Two Questions:
How do I have to change my configuration to have the desired effect?
How do I effectively validate that the Setup/Caching is working?
Thanks!
My current Nginx Configuration:
proxy_cache_path /var/lib/nginx/cache levels=1:2 keys_zone=backcache:8m max_size=50m;
proxy_cache_key "$scheme$request_method$host$request_uri$is_args$args";
proxy_cache_valid 200 302 10m;
proxy_cache_valid 404 1m;
server {
listen 80 default_server;
listen [::]:80 default_server;
location / {
expires 1w;
proxy_cache backcache;
#proxy_cache_bypass $http_cache_control;
add_header X-Proxy-Cache $upstream_cache_status;
proxy_pass http://localhost:3030;
}
}
Headers:
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
* Server certificate: ###
* Server certificate: COMODO RSA Domain Validation Secure Server CA
* Server certificate: COMODO RSA Certification Authority
* Server certificate: AddTrust External CA Root
> GET / HTTP/1.1
> Host: ###
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.10.0 (Ubuntu)
< Date: Thu, 24 Nov 2016 13:15:37 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 66761
< Connection: keep-alive
< Vary: Accept-Encoding
< X-Powered-By: Express
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET,PUT,POST,DELETE
< Access-Control-Allow-Headers: Origin, x-access-token, X-Requested-With, Content-Type, Accept, un-populated
< ETag: W/"104c9-8+4lo531tUk9k3SJq4sPHg"
< Expires: Thu, 01 Dec 2016 13:15:37 GMT
< Cache-Control: max-age=604800
< X-Proxy-Cache: MISS
Headers of two consecutive Requests to an Image
➜ ~ curl -I **url/img.jpg**
HTTP/1.1 200 OK
Server: nginx/1.10.0 (Ubuntu)
Date: Fri, 25 Nov 2016 03:17:02 GMT
Content-Type: application/octet-stream
Content-Length: 11007
Connection: keep-alive
X-Powered-By: Express
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET,PUT,POST,DELETE
Access-Control-Allow-Headers: Origin, x-access-token, X-Requested-With, Content-Type, Accept, un-populated
Accept-Ranges: bytes
Cache-Control: max-age=604800
Last-Modified: Thu, 17 Nov 2016 16:23:13 GMT
ETag: W/"2aff-158731964ca"
Expires: Fri, 02 Dec 2016 03:17:02 GMT
X-Proxy-Cache: MISS
➜ ~ curl -I **url/img.jpg**
HTTP/1.1 200 OK
Server: nginx/1.10.0 (Ubuntu)
Date: Fri, 25 Nov 2016 03:17:08 GMT
Content-Type: application/octet-stream
Content-Length: 11007
Connection: keep-alive
X-Powered-By: Express
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET,PUT,POST,DELETE
Access-Control-Allow-Headers: Origin, x-access-token, X-Requested-With, Content-Type, Accept, un-populated
Accept-Ranges: bytes
Cache-Control: max-age=604800
Last-Modified: Thu, 17 Nov 2016 16:23:13 GMT
ETag: W/"2aff-158731964ca"
Expires: Fri, 02 Dec 2016 03:17:08 GMT
X-Proxy-Cache: MISS

The headers you show as the request headers are most probably not the original ones, because it says "Provisional headers shown".
The actual request headers probably contain the "cache-control" header and your nginx configuration uses that header to bypass the cache, as indicated by the 'X-Proxy-Cache:BYPASS' response header.
If there is no reason to keep it, you should remove the proxy_cache_bypass-directive from your config.

Related

Cloudflare not caching files even when headers are "Cache-Control: public, max-age=86400"

When i use the curl command all the headers are correct but files are not caching as
CF-Cache-Status: MISS
is shown by curl command .
> < HTTP/1.1 200 OK < Date: Sat, 02 Jun 2018 05:57:28 GMT <
> Content-Type: image/png < Content-Length: 2293 < Connection:
> keep-alive < Set-Cookie:
> __cfduid=d3cf179afd212660124585350d19e88c81527919048; expires=Sun, 02-Jun-19 05:57:28 GMT; path=/; domain=.mayan.cf; HttpOnly <
> X-Powered-By: Express < Accept-Ranges: bytes < Cache-Control: public,
> max-age=86400 < Last-Modified: Fri, 01 Jun 2018 15:39:06 GMT < ETag:
> W/"8f5-163bc009a0c" < CF-Cache-Status: MISS < Expires: Sun, 03 Jun
> 2018 05:57:28 GMT < Server: cloudflare < CF-RAY: 4247aec4300930f6-SIN
> < { [1905 bytes data]
> * Connection #0 to host mayan.cf left intact
Cloudflare uses some heuristics to try to guess, based on the request alone, whether the content is static (cacheable) or dynamic. If it guesses wrong at request time, it won't cache the response. You can disable this heuristic by setting the Page Rule "Cache Level" to "Cache Everything" -- then Cloudflare will assume that the response will always be cacheable.
Try setting that page rule and see if it helps.
If the Cache-Control header is set to "private", "no-store",
"no-cache",  or "max-age=0", or if there is a cookie in the response,
then Cloudflare will not cache the resource.
https://support.cloudflare.com/hc/en-us/articles/202775670-How-Do-I-Tell-Cloudflare-What-to-Cache-

Unable to redirect to other url in apache

I have two Redhat Linux servers, say 'original' and 'demo'. Both have apache running on them. On original server, I have a link 'http://original.com/abc' on a page which I want redirect it to http://demo.com.
What I have done is-
Open httpd.conf on original server and added following lines and restarted apache service:
ProxyPass /abc http://demo.com/
ProxyPassReverse /abc http://demo.com/
But when I am trying to access http://original.com/abc, it is redirecting me to the original server i.e. on http://original.com (not on http://demo.com)
I have tried to find the solutions on various sites but unable to find out what the problem is.
Edit:
Output of curl -v http://original.com/abc:
curl -v http://original.com/abc
* About to connect() to original.com port 80 (#0)
* Trying 10.100.100.100...
* Connected to original.com (10.100.100.100) port 80 (#0)
> GET /demo HTTP/1.1
> User-Agent: curl/7.29.0
> Host: original.com
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Tue, 24 Apr 2018 06:30:23 GMT
< Server: Apache-Coyote/1.1
< X-Frame-Options: SAMEORIGIN
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Location: login.action
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0
< Set-Cookie: JSESSIONID=0BAA246C7F2505D2F5A0335CB0542CAA; Path=/; HttpOnly
<
* Connection #0 to host original.com left intact
Output of curl -v http://demo.com/ :
#curl -v http://demo.com
* About to connect() to 10.100.100.101 port 80 (#0)
* Trying 10.100.100.101...
* Connected to 10.100.100.101 (10.100.100.101) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: demo.com
> Accept: */*
>
< HTTP/1.1 302 Found
< Server: Apache-Coyote/1.1
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Set-Cookie: JSESSIONID=FED19458459131F456D638EC57278C2A; Path=/; HttpOnly
< Location: login.action
< Content-Type: text/html
< Content-Length: 0
< Date: Tue, 24 Apr 2018 06:31:47 GMT
<
* Connection #0 to host demo.com left intact
The textual order of ProxyPass statements is relevant. The first matched entry stops further matching.
ProxyPass / http://some.tomcat:8080/
ProxyPassReverse / http://some.tomcat:8080/
...
ProxyPass /abc http://demo.com/
ProxyPassReverse /abc http://demo.com/
Here, demo.com would be fed nothing. The some.tomcat would be fed both / and /abc

Where's the Correlation ID in the Graph API Response?

I'm trying to provide the correlation id for this issue but I'm not sure how to obtain it. I assume it should be in response, is that correct? Is it the client-request-id maybe?
HTTP Response
HTTP/1.1 403 Forbidden
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json;odata=minimalmetadata;streaming=true;charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.5
ocp-aad-diagnostics-server-name: CGa9FFPZASZM+usTGvTzaOZ4zSMxhpbWh2mO9+eJZWs=
request-id: 6ddd7a86-02c9-4dcf-9932-b17afc976107
client-request-id: 50478d75-542d-4e0e-96d9-de913d632471
x-ms-dirapi-data-contract-version: 1.6
ocp-aad-session-key: some-really-long-value
X-Content-Type-Options: nosniff
DataServiceVersion: 3.0;
Strict-Transport-Security: max-age=31536000; includeSubDomains
Access-Control-Allow-Origin: *
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Duration: 782602
X-Powered-By: ASP.NET
Date: Wed, 09 Aug 2017 14:22:21 GMT
Content-Length: 139
{"odata.error":{"code":"Authorization_RequestDenied","message":{"lang":"en","value":"Insufficient privileges to complete the operation."}}}
Token Request Response
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.5
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
client-request-id: 8c500700-b75c-403a-a2c9-c6dfa27049d3
x-ms-request-id: 75627b63-fbd6-4fa8-95b5-b6a325d60900
P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
Set-Cookie: esctx=some-value; domain=.login.microsoftonline.com; path=/; secure; HttpOnly
Set-Cookie: x-ms-gateway-slice=006; path=/; secure; HttpOnly
Set-Cookie: stsservicecookie=ests; path=/; secure; HttpOnly
X-Powered-By: ASP.NET
Date: Wed, 09 Aug 2017 18:33:33 GMT
Content-Length: 1437
{"token_type":"Bearer","expires_in":"3599","ext_expires_in":"262800","expires_on":"1502307214","not_before":"1502303314","resource":"https://graph.windows.net/","access_token":"some-token-value"}
The correlation ID (also called client-request-id) can be found in interactions with the Azure AD service.
The request immediately before this when you were asking the Azure AD endpoints for tokens will include the ID in the response.
https://login.microsoftonline.com/...
https://login.windows.net/...

JSF-Login-Page in HTTP-Response despite valid JSESSIONID

I think this a HTTP-related problem.
I want to use my (JAX-RS) RESTeasy Service on a (JEE6) JBoss AS 7 Server from an Android Device. The RESTeasy Service is working fine. I am using on the Client-Side the Restlet-Client. This works too - without Security.
I want to use my JAAS-Formbased Security for the Pattern /rest/* in web.xml. So I have to send a HTTP-POST-Request with the Form-Data (j_username and j_password) to /foo/j_security_check.
I get the JSESSIONID from the first Response by the Server:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=uKUqlkUWdhX2l-FihiWyeSJr.undefined; Path=/foo
Pragma: No-cache
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 01:00:00 CET
X-Powered-By: JSF/2.0
Content-Type: text/html;charset=utf-8
Content-Length: 1028
Date: Wed, 15 Aug 2012 11:42:59 GMT
For this anonymous session I am authenticating ...
Header:
POST /foo/j_security_check HTTP/1.1
Date: Wed, 15 Aug 2012 11:42:58 GMT
Accept: text/html
Host: 172.24.47.5:8080
User-Agent: Restlet-Framework/2.0.14
Cookie: JSESSIONID=uKUqlkUWdhX2l-FihiWyeSJr.undefined
Content-Length: 62
Content-Language: *
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content:
j_username=Bob&j_password=a
... and it works: JBoss-Security-TRACE:
2012-08-15 13:22:26,829 TRACE
[org.jboss.security.auth.spi.DatabaseServerLoginModule]
(http-0.0.0.0-0.0.0.0-8080-4) User 'Bob' authenticated, loginOk=true
Now the Problem: In the following request I want to GET the REST-URL (using the Cookie JSESSIONID):
GET /foo/rest/sync/products HTTP/1.1
Date: Wed, 15 Aug 2012 11:42:59 GMT
Accept: application/json
Host: 172.24.47.5:8080
User-Agent: Restlet-Framework/2.0.14
Cookie: JSESSIONID=uKUqlkUWdhX2l-FihiWyeSJr.undefined
Content-Length: 0
But instead of returning the Response with JSON Content, the server is returning the JSF-Login-Page, because it want's me to authenticate again(?):
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Pragma: No-cache
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 01:00:00 CET
X-Powered-By: JSF/2.0
Content-Type: text/html;charset=utf-8
Content-Length: 936
Date: Wed, 15 Aug 2012 11:42:59 GMT
<?xml version="1.0" encoding="utf-8"?> ... ... ... </html>
If I login with the Browser and then open the REST-URL it works fine. This is the GET-Request by the Browser:
GET http://localhost:8080/foo/rest/sync/products HTTP/1.1
Host: localhost:8080
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko)
Chrome/19.0.1084.56 Safari/536.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: JSESSIONID=royq26yLd7REOz2otiZdTl6j.undefined
Anyone has an idea? I think the problem lays in the last request (GET /foo/rest/sync/products), because in the Browser it works fine.
Thanks

Server headers - 302 temporary redirect

I have just checked the server headers for my website and this is what i got:
1 Server Response: http://www.pjnsports.co.uk
HTTP/1.1 302 Moved Temporarily
Content-Length: 0
Location: /?6690d3e0
I havent set up any 302 redirects - i assume this will be my host that is doing this. Is this normal practice?? will it have a negative affect on search results, site load speed etc? basically should i be going to them and telling them to do something about it?
Cheers
Paul
I'm not getting any 302 / 301 going to that site:
GET / HTTP/1.1
Host: www.pjnsports.co.uk
Connection: keep-alive
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.107 Safari/534.13
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: PHPSESSID=emr10g0gs9srtjccadb4k7t846; language=en; currency=GBP; __utmz=239376578.1300041169.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=239376578.1365490247.1300041169.1300041169.1300041169.1; __utmc=239376578; __utmb=239376578.1.10.1300041169
HTTP/1.1 200 OK
Date: Sun, 13 Mar 2011 18:33:41 GMT
Server: Apache
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 6531
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8

Resources