Page exists, google bot returning 404 error - googlebot

I have website which is online. When I'am using it via browser everything is ok and this page is present in browser. When I'm using it as googlebot ( via webmastertools ) i'm getting error
HTTP/1.1 404 Not Found
Date: Mon, 19 Nov 2012 09:57:37 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: symfony=55240a0a341202d07fc96cbc1c1bcca5; path=/
Keep-Alive: timeout=2, max=200
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8
and rest of the html code...
same thing when I'm trying to validate it via wc3 validator.
Please help :( I tryied everything :(
website address is mojaczestochowa.pl
If more info is needed please let me know.

Try to check the pae with web-sniffer and set user agent to google.bot

Here is the exact query, which will simulate server's response to the GoogleBot crawler:
https://websniffer.cc/?url=http://mojaczestochowa.pl/&uak=9

Related

Gmail API throwing 401 Unauthorized

I'm trying to integrate Gmail API in my app in order to send emails to users.
i've created a project on Google Developers Console, enabled Gamil API in it, downloaded the credentials as JSON and followed the instruction provided at https://developers.google.com/gmail/api/quickstart/nodejs
I've also added https://developers.google.com/oauthplayground as a redirect URI for the project.
When I run the code, I get redirected to a consent screen. I choose the account which has the project, then get redirected to "oauthplayground"
However, when I try to exchange authorization code for tokens, I receive 401 unauthorized
the full response is:
HTTP/1.1 401 Unauthorized
Content-length: 75
X-xss-protection: 0
X-content-type-options: nosniff
Transfer-encoding: chunked
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Vary: Origin, X-Origin, Referer
Server: scaffolding on HTTPServer2
-content-encoding: gzip
Pragma: no-cache
Cache-control: no-cache, no-store, max-age=0, must-revalidate
Date: Wed, 21 Oct 2020 07:53:18 GMT
X-frame-options: SAMEORIGIN
Alt-svc: h3-Q050=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Content-type: application/json; charset=utf-8
{
"error_description": "Unauthorized",
"error": "unauthorized_client"
}
Any help is appreciated
Thanks

Express API returns duplicate "100 continue" responses when it should only return one

Problem description
When making a HTTP POST request to my Express app (Node.js) the server responds with two "100 Continue" responses and after that a "200 OK" response, when there should only be one "100 Continue".
Postman manages to ignore the second "100 continue" and reports success but another app I'm building with Mule ESB doesn't accept the extra "100 continue" and fails.
The app otherwise handles the POST request without any problems, data is written to the database and so forth.
About the technical enviroment
The Express app is running on a SUSE-server. Nginx serves as a reverse proxy. I manage the multiple Express apps with PM2.
At first i was using a separate Express app, with a proxy package, to act as a reverse proxy. But i switched it out to using Nginx as the reverse proxy, thinking that might have been the issue. But that made no difference.
I have tried the exact same setup with the express reverse proxy on my local machine and that doesn't return "100 Continue" at all, only "200 OK".
I can't figure out why the exact same app returns different responses when it's running on local/server.
Example response
**REQUEST**
POST /<my-endpoint> HTTP/1.1
Host: api.<my-server>.se:443
User-Agent: AHC/1.0
Connection: keep-alive
Accept: */*
Content-Type: application/json; charset=UTF-8
Content-Length: 123
**RESPONSE**
HTTP/1.1 100 Continue
Strict-Transport-Security: max-age=94608000
Date: Thu, 09 May 2019 10:11:34 GMT
Set-Cookie: WASID_HAG=f88cc3f3525ed06b; path=/; domain=.<my-domain>.se
Set-Cookie: WAAK_HAG=41ef118d9c4608f158cfbdcb0140e652; path=/; domain=.<my-domain>.se; secure
Set-Cookie: UPD=6; path=/; domain=.<my-domain>.se
Set-Cookie: RGSC16972=1030; path=/; domain=.<my-domain>.se
HTTP/1.1 100 Continue
Strict-Transport-Security: max-age=94608000
Date: Thu, 09 May 2019 10:11:34 GMT
Set-Cookie: WASID_HAG=f88cc3f3525ed06b; path=/; domain=.<my-domain>.se
Set-Cookie: WAAK_HAG=41ef118d9c4608f158cfbdcb0140e652; path=/; domain=.<my-domain>.se; secure
Set-Cookie: UPD=6; path=/; domain=.<my-domain>.se
Set-Cookie: RGSC16972=1030; path=/; domain=.<my-domain>.se
HTTP/1.1 200 OK
Server: nginx/1.16.0
Date: Thu, 09 May 2019 10:11:34 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 227
Access-Control-Allow-Origin: *
X-DNS-Prefetch-Control: off
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=15552000; includeSubDomains
X-Download-Options: noopen
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
ETag: W/"e3-aMSDU6iiOn52HqHZuJKLaZNt38k"
Set-Cookie: WASID_HAG=f88cc3f3525ed06b; path=/; domain=.<my-domain>.se
Set-Cookie: WAAK_HAG=41ef118d9c4608f158cfbdcb0140e652; path=/; domain=.<my-domain>.se; secure
Set-Cookie: UPD=6; path=/; domain=.<my-domain>.se
Set-Cookie: RGSC16972=1030; path=/; domain=.<my-domain>.se
Cache-control: no-store
What could be causing the multiple "100 Continue" responses and how can i prevent them?
Since the app doesn't return any "100 Continue" responses at all on my local machine could there be something with the server/firewall that's causing this?

cloudfront Cache-Control headers are different than origin headers

I'm seeing a situation where requests through Cloudfront have a different Cache-Control than my origin. I have Object Caching set to "Use Origin Cache Headers" and (I don't think this is relevant) Compress Objects Automatically set to "No"
I've found that if I change Object Caching to "Customize" and change the value around that does in fact change the headers returned from the CDN. That's okay and all... but I'm curious to know why with my existing settings this header isn't being passed through.
Thanks!
Compressed Request from Origin - shows Cache-Control of '31536000'
(05:34 PM) jsharpe#mbp:~ curl -I https://staging.testing.com/assets/application-0d5691ba401c3f5a305fda52745a831376545a605a6c16e50fc838fdaa567e57.css --compressed
HTTP/1.1 200 OK
Server: Cowboy
Date: Wed, 16 Aug 2017 21:34:22 GMT
Connection: keep-alive
Last-Modified: Wed, 16 Aug 2017 05:05:25 GMT
Content-Type: text/css
Cache-Control: public, max-age=31536000
Content-Encoding: gzip
Vary: Accept-Encoding, Origin
Content-Length: 33563
Via: 1.1 vegur
Compressed Request from CDN - shows Cache-Control of '86400'
(05:34 PM) jsharpe#mbp:~ curl -I https://staging-cdn.testing.com/assets/application-0d5691ba401c3f5a305fda52745a831376545a605a6c16e50fc838fdaa567e57.css --compressed
HTTP/1.1 200 OK
Content-Type: text/css
Content-Length: 33563
Connection: keep-alive
Server: Cowboy
Date: Wed, 16 Aug 2017 05:07:12 GMT
Last-Modified: Wed, 16 Aug 2017 05:05:25 GMT
Cache-Control: public, max-age=86400
Content-Encoding: gzip
Via: 1.1 vegur, 1.1 7d327ef7e21429ba6a44eb6374c976f3.cloudfront.net (CloudFront)
Vary: Accept-Encoding
Age: 59233
X-Cache: Hit from cloudfront
X-Amz-Cf-Id: TEqKbQ5ZYySY7m8rDft_MAlygEiam6gYvzrXBpS7D2DrBNbVUZ1y3Q==

iis 8.5 not caching (azure cloud service)

I am essentially having the same issue as this person here:
IIS 8.5 MVC5 Client Cache is ignored
I assume its more an IIS 8.5 config issue rather than anything to do with Azure.
Any thoughts or suggestions??I also tried setting the cache in c# but that didnt work and also tried to add a custom header through the web.config of type 'cache control' but ended up with a mixed result of:
HTTP/1.1 200 OK
Cache-Control: no-cache,max-age=259200
Content-Type: text/javascript; charset=utf-8
Date: Wed, 18 Mar 2015 11:39:42 GMT
Expires: -1
Pragma: no-cache
Server: Microsoft-IIS/8.5
Set-Cookie: ai_session=24c1f2adcaf647e4b2c3a741d6474618|2015-03-18T11:02:37.3489398+00:00|2015-03-18T11:39:43.2134436+00:00; expires=Wed, 18-Mar-2015 12:09:43 GMT; path=/
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
X-UA-Compatible: IE=Edge
Content-Length: 4956

Foursquare venue photos API only occasionally working with client_id/client_secret?

I've found that some venues will only return photos if I use a signed in user instead of a client_id / client_secret. Is this intentional?
curl -i https://api.foursquare.com/v2/venues/4c36476d93db0f47f6cc1d92/photos?client_id=xxx\&client_secret=xxx\&group=venue\&v=20120304
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: no-cache, private, no-store
Content-Type: application/json; charset=utf-8
Date: Mon, 05 Mar 2012 00:28:34 GMT
Expires: Mon, 5 Mar 2012 00:28:34 GMT
Pragma: no-cache
Server: nginx/0.8.52
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4999
Content-Length: 66
Connection: keep-alive
{"meta":{"code":200},"response":{"photos":{"count":0,"items":[]}}}
curl -i https://api.foursquare.com/v2/venues/4c36476d93db0f47f6cc1d92/photos?group=venue\&v=20120304\&oauth_token=xxx\&v=20120304
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: no-cache, private, no-store
Content-Type: application/json; charset=utf-8
Date: Mon, 05 Mar 2012 00:29:19 GMT
Expires: Mon, 5 Mar 2012 00:29:19 GMT
Pragma: no-cache
Server: nginx/0.8.52
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 1000
Content-Length: 15311
Connection: keep-alive
{"meta":{"code":200},"notifications":[{"type":"notificationTray","item":{"unreadCount":0}}],"response":{"photos":{"count":14,"items":[lots of images here]}}}
I want to fetch a photo to associate with a given place as a background process, not tied to the specific user. Is it intended that this API only functions correctly for signed in users?
Looks like there's a bug in userless access to /venues/photos. The team is investigating. The intended behavior is that userless access of that endpoint returns all public photos attached to that venue.

Resources