I am dealing with the Google Insight Page Speed API and I have come across a weird errors that state "PageSpeed was unable to connect to the server. Ensure that you are using the correct protocol (http vs https), the page loads in a browser, and is accessible on the public Internet." and "Unable to process request. Please wait a while and try again."
I tried finding with its forum but hard luck I didn't found any
Google Insight Page Speed Forum
Also, I have searched for related question which are
PageSpeed was unable to connect to the server
and
Timeout error
You can check my API call.
https://www.googleapis.com/pagespeedonline/v2/runPagespeed?url=http://stackoverflow.com&strategy=Mobile&key=API_KEY
I want to overcome over these errors.
Looking forward for your response.
The Pagespeed API will throw this error when it is being overloaded. If you're making multiple calls to the API you might want to slow down the call rate to implement some type of retry functionality.
Related
I know it sounds a lot like other issues here in Stackoverflow, bear with me, it's not (not that I could tell)
I have a scraping app (using Puppeteer) that I use to scrape an Amazon public page.
It works great, I've debugged it by setting the headless: false and I see it works, and it gives me back the expected result.
The same app fails on Heroku, but the problem is not with launching or using Puppeteer (I have several indications), but probably because I'm being identified as a robot.
The error returned is:
waiting for selector `#link_continue input` failed: timeout 30000ms exceeded
Important to say that the error is a generic Puppeteer error that indicates that the selector I'm waiting for just doesn't appear on-page.
I know it should as it's a selector on the first page I navigate to, and it works locally (as mentioned before) - the selector always exists if the page loads.
I had the exactly same error when I've tried to run the scraping on my local machine before setting a User-Agent header. But at that time I could use the headless:false so I saw in my eyes that I'm being rejected due to illegal operations on their page (robots-like operations) so I was redirected to an error page that didn't contain this selector on it.
For this reason, I suspect it recognizes me as a robot, but I don't know how to debug it, it drives me crazy.
Now, if you'd like to reproduce the problem:
You need to wait for the mentioned selector on this site:
https://sellercentral.amazon.com/hz/fba/profitabilitycalculator/index
and then deploy it to Heroku and try to run it maybe 2-3 times
** Two questions: **
How can I proceed from here, I'm 99.9% sure it's the same issue I had previously, but I can't verify... any suggestions?
Given that this is actually the problem, can anyone suggest an easy-to-use/deploy hosting that also allow easy VPN configuration? I think Heroku doesn't give you to do that unless you have an enterprise account
Thanks
I would like to point out that Amazon is very good at blocking IPs. It is very likely that they already blacklisted IPs of cloud services like Heroku, Azure, etc... Previously I have observed services like Cloudflare, Akamai etc... blacklisting these known IPs.
In this scenario Rotating proxies could help you to avoid getting blocked.
I am using google search console's URL testing tools API for a while, and I am receiving a bunch of 502 bad gateway errors. I had submitted a quota increase request to bump up our default requests per day and described that we needed an increase in requests per minute (default is 60/minute). But unfortunately, google support has not responded yet. We are trying to call this API in parallel.
This is the error :
Request failed with status code 502
Could anyone help me understand this issue? I feel that we are exceeding the quota limits because 502 is usually associated with an issue while communicating with the upstream server.
I have reported the issue in Google's public issue tracker. Please add your feedback there.. https://issuetracker.google.com/issues/194021199
It is broken and only works some times. The worse part is that it reports back as HTTP 200 in the google console without any errors so they probably have no idea that it is even broken.
I have searched all over the internet for an answer and although I can find a million people with the same question I cannot find an official solution to the problem im experiencing.
I always get "Cannot display preview. You can post as is, or try another link." displayed.
I've stripped a page down to only the required open graph meta tags so I know they work (run through multiple OG validators), Ive disabled any kind of robots blocking, any kind of redirects, disabled the firewall on a test server, made sure the LinkedIn bot requests are hitting the server. All I see in the browser console all the time is a status 500 being returned from LinkedIn's preview generator API.
We are hosting on Windows Server in IIS 8.5, it seems if I create a demo and host it somewhere else it works, which makes me think it is server related or IIS settings.
Reading this Linkedin post's picture doesn't appear in summary its seems like a similar issue. We are not serving over SSL so nothing to do with that.
I have already asked this question on LinkedIn's forum but having no luck, so im hoping someone on here can help or someone from LinkedIn's tech team can help.
Thanks
So we had this issue as well and it turns out parts of our system that use user generated themes were not adding the "Content-Type" header to the response.
So examine the response headers coming from your server and make absolutely sure they are correct and that they include the correct "Content-Type" (with correct encoding) and "Content-Length".
I am trying to get data from a website into a table in Excel. I am just using the regular button (get data - from web) in Excel (No code) Works fine for two websites but for a different website I am getting the following error:
Details: "The remote server return an HTTP status code '404' when trying to access 'https://smarkets.com/listing/sport/football/premier-league-2017-2018'."
The webpage certainly exists - I am guessing this is a deliberate strategy by the website to prevent data harvesting.
Anyone have any idea how I can get round it either through the get data route or a VBA approach?
Thanks
JL
I inspected traffic with Fiddler and Postman to no avail and in the end contacted the team direct for an answer.
The short answer, from their API team, is no.
Eventually our API, which may be suitable for your needs, will be
available to everyone.
API is in closed alpha stage as I mentioned in comments. More information here: API feed.
API/Odds Feed We're currently working on a new streaming API that is
faster and more scalable. The API is currently in a closed alpha
stage. Unfortunately there is no timeframe on when we'll be able to
release it to the public.
We will prioritise market makers when issuing streaming API accounts.
If you would like to gain alpha access to this service, you can apply
by outlining your proposal here
You can gain access to their XML feed with odds.smarkets.com/oddsfeed.xml .
The feed is updated every few seconds but the information is delayed
by 30 seconds.
has anyone seen this before so I am getting a 502 bad gateway error on my app, the issue I have is that the detailed error information I am getting says my requested url is https://SOX:80/api however my site is configured to use https://sox.domain.com and the site largely works pulling the various JS files required
my app service name is SOX in the azure dashboard so I assume that is where it is picking up SOX from but I have no idea why it is using this.
So overall the issue had me perplexed... however with more testing I soon figured out what was going on.
my backend is Dotnet core Azure throwing the 502 bad gateway was its way of handling exceptions ultimately the problem was code based.
I am mentioning this purely so that it will help others
my first issue was based on cert handling it seems dotnet runs in a container that is specified by your app name as i mentioned above https://SOX:80
the below was causing my issues
sslPolicyErrors = X509StoreStoreHelper.ValidateSSLPolicy(cert.Thumbprint, cert);
after commenting this out for testing my problem went away(we are putting in a proper fix )
my second issue came from using an unsupported view in Azure SQL master.sys.master_files which again just threw a 502 bad gateway error referencing https://SOX:80
please note I have used https://SOX:80 as a reference to mask the real site.
hope this helps the next person.
Based on your description, I have checked your site (https://sox.azurewebsites.net/) and found that it contains three static files (index.html,generic.html,elements.html). I viewed your website in Chrome incognito window as follows:
I did not find any requests against https://SOX:80/api in your html page or JavaScript files. Please try to access your website in a new incognito window to isolate the cache issue or just press CTRL + F5 to refresh your current page to narrow this issue. Moreover, you need to check whether you have configured URL Rewrite. If you still could not solve this issue, you need to update your question with the details for us to reproduce this issue.