I wrote Python code that retrieves API json data from the haveibeenpwned v2 API. The code has been working flawlessly for a month or so but now when I run the code I get ERROR 503.
I searched online for the error code to my understanding this is usually caused by temporary overloading or maintenance on the server so I tried waiting. I've waited about a week trying 5-10 times a day and get the same error every time.
I used the requests Python module to retrieve the json data and to print the status code. If I get error 404 I know that the information isn't there and it goes through the loop until I get the 200 status code.
print(r.status_code)
Could this be a problem I can solve on my side or should I try to contact the owner of the API.
Edit: I realized that the website recently implemented CloudFlare DDoS protection. Could that impact the way I am trying to retrieve the data?
Related
I am using google search console's URL testing tools API for a while, and I am receiving a bunch of 502 bad gateway errors. I had submitted a quota increase request to bump up our default requests per day and described that we needed an increase in requests per minute (default is 60/minute). But unfortunately, google support has not responded yet. We are trying to call this API in parallel.
This is the error :
Request failed with status code 502
Could anyone help me understand this issue? I feel that we are exceeding the quota limits because 502 is usually associated with an issue while communicating with the upstream server.
I have reported the issue in Google's public issue tracker. Please add your feedback there.. https://issuetracker.google.com/issues/194021199
It is broken and only works some times. The worse part is that it reports back as HTTP 200 in the google console without any errors so they probably have no idea that it is even broken.
I am trying to get data from a website into a table in Excel. I am just using the regular button (get data - from web) in Excel (No code) Works fine for two websites but for a different website I am getting the following error:
Details: "The remote server return an HTTP status code '404' when trying to access 'https://smarkets.com/listing/sport/football/premier-league-2017-2018'."
The webpage certainly exists - I am guessing this is a deliberate strategy by the website to prevent data harvesting.
Anyone have any idea how I can get round it either through the get data route or a VBA approach?
Thanks
JL
I inspected traffic with Fiddler and Postman to no avail and in the end contacted the team direct for an answer.
The short answer, from their API team, is no.
Eventually our API, which may be suitable for your needs, will be
available to everyone.
API is in closed alpha stage as I mentioned in comments. More information here: API feed.
API/Odds Feed We're currently working on a new streaming API that is
faster and more scalable. The API is currently in a closed alpha
stage. Unfortunately there is no timeframe on when we'll be able to
release it to the public.
We will prioritise market makers when issuing streaming API accounts.
If you would like to gain alpha access to this service, you can apply
by outlining your proposal here
You can gain access to their XML feed with odds.smarkets.com/oddsfeed.xml .
The feed is updated every few seconds but the information is delayed
by 30 seconds.
I'm trying to setup a facebook share on https://donate.mozilla.org/en-US/thunderbird/share/
The og:url points to just /thunderbird which is the url I would want shared. Best I can tell the og tags are all there.
When I try to update the data on https://developers.facebook.com/tools/debug/og/object/
When I fetch new scrape information I get one of two errors. Initially, it'll take a long time then respond with a Curl Error : OPERATION_TIMEOUTED Operation timed out after 10000 milliseconds with {some number less than 10000} bytes received then subsequent fetch attempts respond with Curl Error : PARTIAL_FILE transfer closed with 17071 bytes remaining to read
We're using AWS Cloudfront and nodejs with hapijs
It responds with a 206 partial content, which, should be fine. The og tags are all in the beginning of the file.
I found this: docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/RangeGETs.html
There it says a range request is used to get the file in chunks, not to get just the part of the file and give up. So maybe that's causing unexpected behavior. Maybe cloudfront is sending it back in chunks, and facebook stops listening after the first response? I dunno. Just trying to find a theory that fits the facts.
We already have a working share for donate.mozilla.org/en-US/share/ but that might be old data from when we were not using hapijs and instead using expressjs which I don't think was supporting range requests and would instead return a 200.
I'm mostly a front end dev, so a lot of this is out of my comfort zone but I have already learned a lot :)
Edit: I also want to point out we use Heroku for hosting, and if I setup a test with just heroku and without cloudfront: donate.mofostaging.net/en-US/thunderbird/ it fetches the tags successfully. So I suspect it's a bug when facebook and hapijs interact with cloudfront.
I use the Google weather API in my web site, and today I get an error that the API link doesn't return any data.
When I check the link directly I get an (Error 403).
Here is the link.
Can anyone please tell me a solution for this and provide me another link for the API?
Every now and then the API stops working for short periods of time, the last days more often a 403 is trown. For my site, last night it happened 13 times. But the site tries immediately again and the second or third time, the data loads without problems. As the API is unofficial, not sure what’s causing the 403.
Make sure you cache the data as the API will block your IP temporary when you make too much requests. In my case, I cache for 20 minutes and if no data can retrieved, the site will not try more than 10 times to reload the API. Once I forgot to turn caching on after debugging and as my site did many hundred requests (with every visitor), the IP was blocked within an hour. If a remember correct, the error was not a 403. Fortunately, the block lasts for less than a half day.
There is currently an intermittent 403 Forbidden response to the Google Weather API that you are using. See Google Weather API 403 Error
The reason for the intermittent 403 response is not known but has been a problem since the 7th of August 2012.
I issue an authenticated call to /venues/venue_id but I get a 500 error
"Foursquare servers are experiencing problems. Please retry and check
status.foursquare.com for updates."
I make about 60 of these calls, and only ~20 of them fail. It just started failing 3 days ago. Any ideas? What else do I need to provide?
Edit: The call I am making is:
https://api.foursquare.com/v2/venues/4ae2619bf964a520d58d21e3?oauth_token=xxxx&v=20120201
I make about 30 of these, and 10-12 work just fine, but the remaining fail. I tried the Foursquare API Explorer and I still get a 500 response.
My API requests suddenly started working. No word why.