I am trying to remove the "Server" header from the response.header in Cherrypy via a Custom Tool that I have written. I have hooked this tool to "before_finalize" hook point.
class ChangeHeadersTool(cherrypy.Tool):
def __init__(self):
cherrypy.Tool.__init__(self, 'before_finalize',
self.remove_headers,
priority=100)
def remove_headers(self):
cherrypy.response.headers.pop('Server', None)
The tool is working fine. All the responses are passed through the tool and I am able to pop out other headers from the response. But the 'Server' header getsa added back somewhere down the line I guess. Is there any other way to remove the 'Server' header from the tool
Related
I have studied the code as follow in Cherrypy web development,
if returnpage != '':
raise cherrypy.InternalRedirect(returnpage)
else:
raise cherrypy.HTTPRedirect("/hqc")
Google doesn't help much in this case after I did some research.
I've checked from cherrypy's __doc__, but the documentation there is very terse.
>>>print(cherrypy.InternalRedirect.__doc__)
Exception raised to switch to the handler for a different URL.
This exception will redirect processing to another path within the site
(without informing the client). Provide the new path as an argument when
raising the exception. Provide any params in the querystring for the new URL.
>>> print(cherrypy.HTTPRedirect.__doc__)
Exception raised when the request should be redirected.
This exception will force a HTTP redirect to the URL or URL's you give it.
The new URL must be passed as the first argument to the Exception,
e.g., HTTPRedirect(newUrl). Multiple URLs are allowed in a list.
If a URL is absolute, it will be used as-is. If it is relative, it is
assumed to be relative to the current cherrypy.request.path_info.
If one of the provided URL is a unicode object, it will be encoded
using the default encoding or the one passed in parameter.
There are multiple types of redirect, from which you can select via the
``status`` argument. If you do not provide a ``status`` arg, it defaults to
303 (or 302 if responding with HTTP/1.0).
Examples::
raise cherrypy.HTTPRedirect("")
raise cherrypy.HTTPRedirect("/abs/path", 307)
raise cherrypy.HTTPRedirect(["path1", "path2?a=1&b=2"], 301)
See :ref:`redirectingpost` for additional caveats.
My questions are:
- Why bother with redirect when you can simply invoke another handler?
- What are some practical senarios for the two redirect exception respectively?
InternalRedirect is only handled in the server side, this means that the client would not be aware of that redirection, because in terms of the HTTP protocol that is mediating the session between the client and the server, nothing changed. By server side I mean ONLY CherryPy will be aware of the rediction, if you have some intermediate server (like an nginx reverse proxy) it would not see anything different.
For example if the client visited a url /page_one and then you used raise InternalRedirect('/page_two'), the client (browser) will receive the content from the /page_two handler in the /page_one url. If you raised a regular HTTPRedirect the server would end the first request with an HTTP status code of 303 (or any other status that you passed to the exception) and a Location header to /page_two. Then is the client who will initiate another request to /page_two, basically everybody will be aware of the redirection (more info about HTTP redirection). Most of the time this is the better alternative.
Additionally you could detect if the request came from a previous InternalRedirect by verifying the cherrypy.request.prev property. It will have the previous cherrypy.request object as its value or None.
For the sake of a possible (maybe not the best example) use of an InternalRedirect, checkout this production/beta example page, in addition I added a tool to prohibit the client to reach to handlers directly.
The client will see a different content in the same page /. Note that the access log that CherryPy generates will log the url of the handler that end up handling the request, in this case you will see /_beta or /_production.
import random
import cherrypy
#cherrypy.tools.register('before_handler')
def private_handler():
"""End the request with HTTP 404 not found if the client
tries to reach the handler directly instead of being
internally redirected from other handler.
"""
if cherrypy.request.prev is None:
raise cherrypy.NotFound()
class MainApp:
#cherrypy.expose
def index(self):
# 50/50 change of receiving production or the new SHINY beta page
use_beta = random.randint(0, 1)
if use_beta:
raise cherrypy.InternalRedirect('/_beta')
else:
raise cherrypy.InternalRedirect('/_production')
#cherrypy.tools.private_handler()
#cherrypy.expose
def _production(self):
return (
"<html>"
"<h2>{}</h2>"
"</html>"
).format(
"Welcome to our awesome site!"
)
#cherrypy.tools.private_handler()
#cherrypy.expose
def _beta(self):
return (
"<html>"
'<h1 style="color: blue">{}</h1>'
"<p>{}</p>"
"</html>"
).format(
"Welcome to our awesome site!",
"Here is our new beta content..."
)
cherrypy.quickstart(MainApp())
I am using the Requests library of robot framework to upload files to a server. The file RequestsKeywords.py has a line
logger.info('Post Request using : alias=%s, uri=%s, data=%s, headers=%s, files=%s, allow_redirects=%s '
% (alias, uri, dataStr, headers, files, redir))
This prints out the whole contents of my upload file inside the request in my log file. Now i could get rid of this log by changing the log level however, my goal is to be able to see the log but just truncate it to 80 characters, so I am not browsing through lines of hex values. Any idea how this could be done?
A solution would be to create a wrapper method, that'll temporary disable the logging, and enable it back once completed.
The flow is - get an instance of the RequestsLibrary, call RF's Set Log Level with argument "ERROR" (so at least an error gets through, if needed), call the original keyword, set the log level back to what it was, and return the result.
Here's how it looks like in python:
from robot.libraries.BuiltIn import BuiltIn
def post_request_no_log(*args, **kwargs):
req_lib = BuiltIn().get_library_instance('RequestsLibrary')
current_level = BuiltIn().set_log_level('ERROR')
try:
result = req_lib.post_request(*args, **kwargs)
except Exception as ex:
raise ex
finally:
BuiltIn().set_log_level(current_level)
return result
And the same, in robotframework syntax:
Post Request With No Logging
[Documentation] Runs RequestsLibrary's Post Request, with its logging surpressed
[Arguments] #{args} &{kwargs}
${current level}= Set Log Level ERROR
${result}= Post Request #{args} &{kwargs}
[Return] ${result}
[Teardown] Set Log Level ${current level}
The python's version is bound to be milliseconds faster - no need to parse & match the text in the RF syntax, which on large usage may add up.
Perhaps not the answer you're looking for, but after having looked at the source of the RequestsLibrary I think this is indeed undesirable and should be corrected. It makes sense to have the file contents when running in a debug or trace setting, but not during regular operation.
As I consider this a bug, I'd recommend registering an issue with the GitHub project page or correcting it yourself and providing a pull request. In my opinion the code should be refactored to send the file name under the info setting and the file contents under the trace/debug setting:
logger.info('Post Request using : alias=%s, uri=%s, data=%s, headers=%s, allow_redirects=%s' % ...
logger.trace('Post Request files : files=%s' % ...
In the mean time you have two options. As you correctly said, temporarily reduce the log level settings in Robot Code. If you can't change the script, then using a Robot Framework Listener can help with that. Granted, it would be more work then making the change in the ReqestsLibrary yourself.
An temporary alternative could be to use the RequestLibrary Post, which is deprecated but still present.
If you look at the method in RequestKeywords library, its only calling self. _body_request() at the end. What we ended up doing is writing another keyword that was identical to the original except the part where it called logger.info(). We modified it to log files=%.80s which truncated the file to 80 chars.
def post_request_truncated_logs(
self,
alias,
uri,
data=None,
params=None,
headers=None,
files=None,
allow_redirects=None,
timeout=None):
session = self._cache.switch(alias)
if not files:
data = self._format_data_according_to_header(session, data, headers)
redir = True if allow_redirects is None else allow_redirects
response = self._body_request(
"post",
session,
uri,
data,
params,
files,
headers,
redir,
timeout)
dataStr = self._format_data_to_log_string_according_to_header(data, headers)
logger.info('Post Request using : alias=%s, uri=%s, data=%s, headers=%s, files=%.80s, allow_redirects=%s '
% (alias, uri, dataStr, headers, files, redir))
I'm new with CherryPy web-framework. I would like to know how can
I read POST data that was delivered via HTTP request's body .
Thanks,
Gabriel.
You can either read the body of the request with cherrypy.request.body, cherrypy.request.params or if you use the default handler then, the distinction between GET and POST values is abstracted by cherrypy and you can get the values directly from the arguments:
#cherrypy.expose
def index(self, name, age):
return "My name is %s and age is %s" % (name, age)
Your POST request must have to provide the values for name and age on the traditional forms submission.
If you are planning on using json, then use the cherrypy.tools.json_in decorator and read the cherrypy.request.json property. http://docs.cherrypy.org/en/latest/basics.html#dealing-with-json
If you are using the MethodDispatcher, then there is another way to do it: http://docs.cherrypy.org/en/latest/tutorials.html#tutorial-7-give-us-a-rest
Maybe these posts might help you:
http://blog.joel.mx/posts/cherrypy-101
http://blog.joel.mx/posts/cherrypy-101-method-dispatcher
Cherrypy handles both get and post in a similar way by passing them into the function or method that is handling the request. A good example is located at tut03_get_and_post.py in the tutorial folder within the cherrypy package.
Here is a small portion that specifically speaks to your question...
#cherrypy.expose
def greetUser(self, name=None):
# CherryPy passes all GET and POST variables as method parameters.
# It doesn't make a difference where the variables come from, how
# large their contents are, and so on.
I was wondering if there was any way to re-order HTTP headers that are being sent by our browser, before getting sent back to the web server?
Since the order of the headers leaves some kind of "fingerprinting", see this post and this post, I was thinking about using MITMProxy (with Inline Scripting, I guess) to modify headers on-the-fly. Is this possible?
How would one achieve that?
Note: I'm looking for a method that could be scripted, not a method using a graphical tool like the Burp Suite (although Burp is known to be able to re-order headers)
I'm open to suggestions. Perhaps NGINX might come to the rescue as well?
EDIT: I should be more specific, by giving an example...
Let's say I'm using Firefox. With the use of a funky add-on, I'm spoofing my user-agent to "look" like a Chrome browser. But then if I test my browser with ip-check.info, the "signature" of my browser remains the one of Firefox, even though my spoofed user-agent shows "Chrome".
So the solution, in this specific case, should be to re-order the HTTP headers in the same manner as Chrome does.
How can this be done?
For the record, the order of the HTTP headers should not matter at all according to RFC 7230. But now that you have asked... this can be done in mitmproxy as follows:
import random
def request(context, flow):
# flow.request.headers.fields is a tuple of (name, value) header tuples.
h = list(flow.request.headers.fields)
random.shuffle(h)
flow.request.headers.fields = tuple(h)
See the mitmproxy documentation on netlib.http.Headers for more details.
There are tons of way to reorder them as you wish:
def reorder(headers, header_order=["Host","User-Agent","Accept"]):
lines = []
for name in header_order: # add existing headers in the specified order
if name in headers:
lines.extend(headers.get_all(name))
del headers[name]
lines.extend(headers.fields) # all other headers
return lines
request.headers.fields = reorder(request.headers)
I'm writing a Python 3 (3.5) script that will act as a simple command line interface to a user's stikked install. The API is pretty straight forward, and its documentation is available.
My post function is:
def submit_paste(paste):
global settings
data = {
'title': settings['title'],
'name': settings['author'],
'text': paste,
'lang': settings['language'],
'private': settings['private']
}
data = bytes(urllib.parse.urlencode(data).encode())
print(data)
handler = urllib.request.urlopen(settings['url'], data)
print(handler.read().decode('utf-8'))
When I run the script, I get the printed output of data, and the message returned from the API. The data encoding looks correct to me, and outputs:
b'private=0&text=hello%2C+world%0A&lang=text&title=Untitled&name=jacob'
As you can see, that contains the text= attribute, which is the only one actually required for the API call to successfully work. I've been able to successfully post to the API using curl as shown in that link.
The actual error produced by the API is:
Error: Missing paste text
Is the text attribute somehow being encoded incorrectly?
Turns out the problem wasn't with the post function, but with the URL. My virtual host automatically forwards http traffic to https. Apparently, Apache drops the post variables when it forwards.