Very strange api throws status 404 due to several features - node.js

Well. Exists something api, and I try get file, but I get 404 status code - why?
I don't know what is, but I found several features.
If use browser and going to path for download file - always ok, him download.
But if use modules(I use superagent) you are get 404 status code.
Ok. I watch on headers request - I copy all headers from firefox and set up in request - anyway 404.
I use Google chrome, and always ok! Him too download file.
I use Sphere(~anonymous browser) - 404.
I go on linux, and try again.
All browsers drop 404. superagent too.
And it's as far as I know work on nginx. (message "404 not found. nginx.")
I search any ideas why it happens.
I use API from bazon.cc

There are several reasons why this might be happening: Cookies, User-Agent based filtering, HTTP-Referrer Header checking and so on. You should use a packet capture tool to make sure the requests are really exactly the same.

Related

IIS 8.0 - Strip html tags from url (XSS protection)

Here's the thing. I'm trying to protect my server from XSS Attacks (And so far with no trouble at all, changing HTTP response Headers and other things) But a Generic vulnerability is still going on, and it happens because in the URL some javascript code can be inserted
(i.e. http://myhost.com/thisfile.jsp?<script>alert("hello")</script> )
when I type this, the response is HTTP 202 OK Status (It redirects to my 404 page). But I need to do one of these actions:
throw another HTTP Status (405, 500, Or any status giving an error)
Throw an error.
What can I do? Is there any way to strip the tags or recognize them via web.config file to throw an error?... I've been trying with the rewrite module and the request filtering with no success at all.
Thank you so much in advance, regards.
Thank you... I solved it and it was so easy (I can't believe i didn't try that first).
I went to IIS manager and then click on request filtering.
Then, on the "Rules" tab I added a "Filtering Rule". Applying to all file extensions and in the field "Deny Query String" I added <script>, <scr+ipt>, etc. So when the URL comes with a tag like that the connection is closed without showing 404 or any error page.
It worked and now the vulnerability scanner doesn't show any risk.

Apigee API end point gives 503 on the browser, but a 200 on Apigee trace and curl

We use Apigee proxy to invoke our API. All works well when we test it out within Apigee trace. Also works fine with curl. But on a browser, it gives a 503. This is not consistent though, sometimes it gives a 200 on the browser too. Tried Chrome and Firefox, same behavior.
Our API still executes well though. We do not return any response, merely set the status. Any ideas on what we could try out to get a 200 on the browser?
Couple of things to check:
Check if your Browser has a DNS entry caching. Sometimes services like ELB changes the actual IPs. So caching DNS entries may result in 503.
Another you may want to check is the difference is in the HTTP Verb used. Browsers send a GET request. But curl commands can do all. So if your service is specifically not serving GET calls you may get some server side errors. Also curl sends certain headers even if you do not explicitly send. E.g., Accept:/ header and user-agent header etc. Check if the server is behaving differently based on those headers.
You should look into using Chrome or Firefox extensions for this. There are two in particular which support a wide range of additional features for API developers.
For Chrome, try Postman.
For Firefox, try RESTClient.
Thanks.

Can't get IIS 7/Coldfusion to deliver 404

So... we have a custom CMS. We have a rewrite rule that any page request (when a file doesn't exist) goes to the root/index.cfm file. There we search our DB for the page in question. If the page exists, we serve up the correct template,etc. If the page doesn't exist I want to server up a 404 page. Now I "think" I cannot do this in IIS since I need to handle the request in CF, so it has to get through. The file will always exist. When the page doesn't exist I've tried using <cfheader statusCode="404" > and then include some html, it puts The resource you are looking for has been removed, had its name changed, or is temporarily unavailable. at the top of the page before my html. In order to get it to display the page I had to remove the 404 status code handler from IIS.
In addition when I fetch as Google, it get's a 301. However when I view response headers in Firefox I get.
Transfer-Encoding: chunked
Content-Type: text/html
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Wed, 16 Jan 2013 21:31:42 GMT
404 Not Found
I've tried a combination of redirecting and all sorts of things. I open to letting IIS handle the 404, if there is a way, but I cannot figure out how to get Coldfusion to correctly deliver a 404 so Google gets it right. Webmaster tools gets mad at me because I am delivering "Soft 404s" before this point, so I am trying to fix that.
I've also tried setting <httpErrors existingResponse="PassThrough" /> whatever the hell that does, but didn't work either. I've been looking up other threads trying to figure this out and just can't.
EDIT: Looking further into this, viewing the header info in both Firebug & Chrome I clearly see the headers say 404. Why would Fetch as Bing and Fetch as Google say differently?
I tested the fact that if I add .cfm to the URLs, it Fetch as Google will deliver see 404. However without the .cfm, it thinks it's 301. Firebug sees both as 404. This seems like a Google issue.
ANSWER Kind of:
So I was doing more testing this morning (Right after I added a bounty actually), and I noticed in webmaster tools, Google correctly noted one of my pages as a 404. So I started looking into it. I have an "Add Trailing Slash Rule". Google notices domain.com/page as a 301 (Correct I guess) to domain.com/page/. But it does notice domain.com/page/ as a 404. I think using the trailing slash rule as I have it is the right way, however, should I be doing something different, or is using the redirect with the slash the "correct" way of doing things, even though Google wants to ding me for it sometimes.
I'm not entirely sure I follow the specifics of your approach, so I will give you a few things that you need to look at in order to get this approach working well (or at least what has worked best for me).
Under "Error Pages", make sure that your 404 error page is set to "Execute a URL on this site" ( I generally set mine to something like "/404.cfm"). This will make sure that your ColdFusion page is called correctly for 404 pages (it sounds like you have this working correctly).
Under "Handler Mappings", double-click on the handler for ".cfm". Then click the "Request Restrictions..." button. It should open to the "mappings" tab. The "Invoke handler only if request is mapped to:" checkbox should NOT be checked.
This can really trip up this sort of operation because it means that IIS won't invoke ColdFusion if the file doesn't exist. This shouldn't be an issue if your 404 is set up correctly, but still something to look into.
While you are in the "Handler Mapping" section, look for the IsapiModule with a path of "*". Mine is always set to ColdFusion - not sure if that makes a difference or not.
The other thing to look at is the "Default Document" setting. Keep in mind that this could impact you when forwarding to a folder.
You might also look at your rewrite rule again and make sure it isn't adding slashes where one already exists.

Is there any way to tell a browser that this is a bad URL to remember?

I'm sending emails to customers, and I'm providing a custom URL for each, which when they go to, will log them in.
This is fine, except if they are using a shared browser that will remember the URL.
Is there any way at all to suggest to the browser that it shouldn't remember a URL?
Edit: This question has nothing to do with caching of the page.
Have the link log them in once. Then make them create credentials that let them access the site in the future. Whats to stop a random person from typing in the url and gaining access to the content?
Yes. You can redirect them with a 301 or 302. Then the browser won't save the URL they went to. At least that work with the Mozilla based browsers and I would imagine others too.
Another way, it is uglier though is to reply with an error and include a body which does a refresh. Whether that works in most browsers, probably not. However, browsers do not cache pages that return an error (404 Page Not Found would work, you could also use 403 Forbidden.)
Other than that, there isn't much you can do. JavaScript does not allow you to temper with the history anymore...

What is the best way to debug a VCL file?

I am writing inline C in my VCL file. More specifically I am using Maxmind's GeoIP database to geocode a visitor's IP. I have everything installed, I have followed all the wiki examples for GeoIP database and everything works swimmingly.
I am trying to now do some magic with GeoIP besides the return country examples. I want to return the visitor's city using the method GeoIP_record_by_addr(), which returns a pointer.
Problem: I cannot seem to correctly cast a GeoIPRecord* to char*. I have tried for hours. I get Varnish to compile my VCL file without any errors or notices, but the varnish server responds with 403.
Question: Anyway I can debug either the inline C or the 403 varnish is responding with?
Generally, Firebug and varnishlog will be your best friends.
If you want to debug pure VCL, the best way is to send data into HTTP headers ([req/bereq/beresp/resp].http.[header name]) and check their value into Firebug (or varnishlog if you have few requests).
If you want to debug inline C, you can also play with headers (VRT_SetHdr()) but if your C code makes varnish crash, you'll see why into /var/log/messages.
You can also check varnishlog to see if varnish crashes...but when varnish crashes, you get timeouts, not 403...
I'd have to see your VCL to understand why you get 403 but technically, it's not an "error", but a "status", meaning that your request has been processed by varnish (and, unfortunately, forbidden somewhrere).
I don't think Varnish would return 403 except if you ask him to do it. So there's a big chance the 403 status comes from your web server (backend).
Anyway, your varnish doesn't seem to crash but rather have behavior issues.

Resources