When running a CherryPy app it will send server name tag something like CherryPy/version.
Is it possible to rename/overwrite that from the app without modifying CherryPy so it will show something else?
Maybe something like MyAppName/version (CherryPy/version)
This can now be set on a per application basis in the config file/dict
[/]
response.headers.server = "CherryPy Dev01"
Actually asking on IRC on their official channel fumanchu gived me a more clean way to do this (using latest svn):
import cherrypy
from cherrypy import _cpwsgi_server
class HelloWorld(object):
def index(self):
return "Hello World!"
index.exposed = True
serverTag = "MyApp/%s (CherryPy/%s)" % ("1.2.3", cherrypy.__version__)
_cpwsgi_server.CPWSGIServer.environ['SERVER_SOFTWARE'] = serverTag
cherrypy.config.update({'tools.response_headers.on': True,
'tools.response_headers.headers': [('Server', serverTag)]})
cherrypy.quickstart(HelloWorld())
This string appears to be being set in the CherrPy Response class:
def __init__(self):
self.status = None
self.header_list = None
self._body = []
self.time = time.time()
self.headers = http.HeaderMap()
# Since we know all our keys are titled strings, we can
# bypass HeaderMap.update and get a big speed boost.
dict.update(self.headers, {
"Content-Type": 'text/html',
"Server": "CherryPy/" + cherrypy.__version__,
"Date": http.HTTPDate(self.time),
})
So when you're creating your Response object, you can update the "Server" header to display your desired string. From the CherrPy Response Object documentation:
headers
A dictionary containing the headers of the response. You may set values in
this dict anytime before the finalize phase, after which CherryPy switches
to using header_list ...
EDIT: To avoid needing to make this change with every response object you create, one simple way to get around this is to wrap the Response object. For example, you can create your own Response object that inherits from CherryPy's Response and updates the headers key after initializing:
class MyResponse(Response):
def __init__(self):
Response.__init__(self)
dict.update(self.headers, {
"Server": "MyServer/1.0",
})
RespObject = MyResponse()
print RespObject.headers["Server"]
Then you can can call your object for uses where you need to create a Response object, and it will always have the Server header set to your desired string.
Related
Trying to figure out the right mechanism to use here.
I want to modify the flask request coming in every time.
I think the request is immutable, so I am trying to figure out if this mechanism exists.
Basically, I want to append a string onto the end of the request coming in.
I can hook into the request and the right time in a before_request handler like this:
#app.before_app_request
def before_request_custom():
# Get the request
req = flask.request
method = str(req.method)
if method == "GET":
# Do stuff here
pass
But I am not sure what to actually do to add this in, and don't see a way to accomplish it...I guess i could redirect, but that seems silly in this case. Any ideas?
The request object is immutable (https://werkzeug.palletsprojects.com/en/1.0.x/wrappers/#base-wrappers), but request.args or request.form can be set from ImmutableOrderedMultiDict to just OrderedMultiDict using Subclassing on Flask (https://flask.palletsprojects.com/en/1.1.x/patterns/subclassing/). Here's an example of how you could add that filter[is_deleted]=False URL param:
from flask import Flask, request, Request
from werkzeug.datastructures import OrderedMultiDict
class MyRequest(Request):
parameter_storage_class = OrderedMultiDict
class MyApp(Flask):
def __init__(self, import_name):
super(MyApp, self).__init__(import_name)
self.before_request(self.my_before_method)
def my_before_method(self):
if "endpoint" in request.base_url:
request.args["filter[is_deleted]"] = "False"
app = MyApp(__name__)
app.request_class = MyRequest
#app.route('/endpoint/')
def endpoint():
filter = request.args.get('filter[is_deleted]')
return filter
This way you can modify request.args before you actually send the request.
How about this?
from flask import g
#app.before_request
def before_request():
# Get the request
req = flask.request
method = str(req.method)
if method == "GET":
g.my_addon = "secret sauce"
return None
Then, g.my_addon is available in every view function:
from flask import g
#app.route('/my_view')
def my_view():
if g.my_addon == "secret sauce":
print('it worked!')
Using test_request_context() you can make the trick.
Related: https://flask.palletsprojects.com/en/1.1.x/quickstart/#accessing-request-data
I want my client to download (not render) a dynamically generated PDF file via pyramid. Right now I do it like this:
def get_pdf(request):
pdfFile = open('/tmp/example.pdf', "wb")
pdfFile.write(generator.GeneratePDF())
response = FileResponse('/tmp/example.pdf')
response.headers['Content-Disposition'] = ('attachment; filename=example.pdf')
return response
From the client point-of-view it's exactly what I need. However,
It leaves behind an orphaned file
It isn't thread-safe (though I could use random filenames)
The docs say:
class FileResponse
A Response object that can be used to serve a static file from disk simply.
So FileResponse is probably not what I should be using. How would you replace it with something more dynamic but indistinguishable for the client?
Just use a normal response with the same header:
def get_pdf(request):
response = Response(body=generator.GeneratePDF())
response.headers['Content-Disposition'] = ('attachment;filename=example.pdf')
return response
I have a Python (3.6) web server using aiohttp with a lot of routes in it. The handler in my routes usually looks like this:
async def __call__(self, request):
response1 = await call_service_1('aaa')
response2 = await call_service_2('bbb')
response3 = await call_service_3('ccc')
return [response1, response2, response3]
And a service call looks like:
async def call_service1(self, arg):
url = 'http://localhost/' + arg
headers = {'Content-Type': 'application/json'}
async with self.client_session.get(url, headers=headers) as response:
return response
I have a new requirement in which I need to read a header value in the request, and pass that on as a header in my subsequent service requests. I have this requirement for all of my routes and all of my service calls. My initial attempt to fulfil this requirement would be to change the routes to this:
async def __call__(self, request):
my_header_value = request.headers['my_header_value'] # new header value
response1 = call_service_1('aaa', my_header_value)
response2 = call_service_2('bbb', my_header_value)
response3 = call_service_3('ccc', my_header_value)
And to change the service request to this:
async def call_service1(self, arg, my_header_value): # new parameter
url = 'http://localhost/' + arg
headers = {
'Content-Type': 'application/json',
'my_header_value': my_header_value # new header value
}
async with self.client_session.get(url, headers=headers) as response:
return response
I don't think this is the Pythonic way to make a change like this. I think it's a bad design to pass this parameter around like this, from endpoint to endpoint. I know that this is a cross-cutting concern, but I don't know the proper way to design something to handle this in Python. I'd ultimately be copying and pasting code all over the place. There's got to be a better way, but I don't know it and my Python isn't that strong.
For instance, is there some way I can decorate the service calls so that I don't have to add a new parameter all over the place? I don't think I have access to some session variables that I can set so that I don't have to add parameters at all. If I did however, I'd be worried about unit testing.
What do you recommend?
I want to be able to get a body of the specific subrequest using a selenium behind the proxy.
Now I'm using python + selenium + chromedriver. With logging I'm able to get each subrequest's headers but not body. My logging settings:
caps['loggingPrefs'] =
{'performance': 'ALL',
'browser': 'ALL'}
caps['perfLoggingPrefs'] = {"enableNetwork": True,
"enablePage": True,
"enableTimeline": True}
I know there are several options to form a HAR with selenium:
Use geckodriver and har-export-trigger. I tried to make it work with the following code:
window.foo = HAR.triggerExport().then(harLog => { return(harLog); });
return window.foo;
Unfortunately, I don't see the body of the response in the returning data.
Use browsermob proxy. The solution seems totally fine but I didn't find the way to make browsermob proxy work behind the proxy.
So the question is: how can I get the body of the specific network response on the request made during the downloading of the webpage with selenium AND use proxies.
UPD: Actually, with har-export-trigger I get the response bodies, but not all of them: the response body I need is in json, it's MIME type is 'text/html; charset=utf-8' and it is missing from the HAR file I generate, so the solution is still missing.
UPD2: After further investigation, I realized that a response body is missing even on my desktop firefox when the har-export-trigger add-on is turned on, so this solution may be a dead-end (issue on Github)
UPD3: This bug can be seen only with the latest version of har-export-trigger. With version 0.6.0. everything works just fine.
So, for future googlers: you may use har-export-trigger v. 0.6.0. or the approach from the accepted answer.
I have actually just finished to implemented a selenium HAR script with tools you are mentioned in the question. Both HAR getting from har-export-trigger and BrowserMob are verified with Google HAR Analyser.
A class using selenium, gecko driver and har-export-trigger:
class MyWebDriver(object):
# a inner class to implement custom wait
class PageIsLoaded(object):
def __call__(self, driver):
state = driver.execute_script('return document.readyState;')
MyWebDriver._LOGGER.debug("checking document state: " + state)
return state == "complete"
_FIREFOX_DRIVER = "geckodriver"
# load HAR_EXPORT_TRIGGER extension
_HAR_TRIGGER_EXT_PATH = os.path.abspath(
"har_export_trigger-0.6.1-an+fx_orig.xpi")
_PROFILE = webdriver.FirefoxProfile()
_PROFILE.set_preference("devtools.toolbox.selectedTool", "netmonitor")
_CAP = DesiredCapabilities().FIREFOX
_OPTIONS = FirefoxOptions()
# add runtime argument to run with devtools opened
_OPTIONS.add_argument("-devtools")
_LOGGER = my_logger.get_custom_logger(os.path.basename(__file__))
def __init__(self, log_body=False):
self.browser = None
self.log_body = log_body
# return the webdriver instance
def get_instance(self):
if self.browser is None:
self.browser = webdriver.Firefox(capabilities=
MyWebDriver._CAP,
executable_path=
MyWebDriver._FIREFOX_DRIVER,
firefox_options=
MyWebDriver._OPTIONS,
firefox_profile=
MyWebDriver._PROFILE)
self.browser.install_addon(MyWebDriver._HAR_TRIGGER_EXT_PATH,
temporary=True)
MyWebDriver._LOGGER.info("Web Driver initialized.")
return self.browser
def get_har(self):
# JSON.stringify has to be called to return as a string
har_harvest = "myString = HAR.triggerExport().then(" \
"harLog => {return JSON.stringify(harLog);});" \
"return myString;"
har_dict = dict()
har_dict['log'] = json.loads(self.browser.execute_script(har_harvest))
# remove content body
if self.log_body is False:
for entry in har_dict['log']['entries']:
temp_dict = entry['response']['content']
try:
temp_dict.pop("text")
except KeyError:
pass
return har_dict
def quit(self):
self.browser.quit()
MyWebDriver._LOGGER.warning("Web Driver closed.")
A subclass adding BrowserMob proxy for your reference as well:
class MyWebDriverWithProxy(MyWebDriver):
_PROXY_EXECUTABLE = os.path.join(os.getcwd(), "venv", "lib",
"browsermob-proxy-2.1.4", "bin",
"browsermob-proxy")
def __init__(self, url, log_body=False):
super().__init__(log_body=log_body)
self.server = Server(MyWebDriverWithProxy._PROXY_EXECUTABLE)
self.server.start()
self.proxy = self.server.create_proxy()
self.proxy.new_har(url,
options={'captureHeaders': True,
'captureContent': self.log_body})
super()._LOGGER.info("BrowserMob server started")
super()._PROFILE.set_proxy(self.proxy.selenium_proxy())
def get_har(self):
return self.proxy.har
def quit(self):
self.browser.quit()
self.proxy.close()
MyWebDriver._LOGGER.info("BroswerMob server and Web Driver closed.")
I am writing my first Groovy script, where I am calling a REST API.
I have the following call:
def client = new RESTClient( 'http://myServer:9000/api/resources/?format=json' )
That returns:
[[msr:[[data:{"level":"OK"}]], creationDate:2017-02-14T16:44:11+0000, date:2017-02-14T16:46:39+0000, id:136]]
I am trying to get the field level, like this:
def level_value = client.get( path : 'msr/data/level' )
However, when I print the value of the variable obtained:
println level_value.getData()
I get the whole JSON object instead of the field:
[[msr:[[data:{"level":"OK"}]], creationDate:2017-02-14T16:44:11+0000, date:2017-02-14T16:46:39+0000, id:136]]
So, what am I doing wrong?
Haven't looked at the docs for RESTClient but like Tim notes you seem to have a bit of a confusion around the rest client instance vs the respons object vs the json data. Something along the lines of:
def client = new RESTClient('http://myServer:9000/api/resources/?format=json')
def response = client.get(path: 'msr/data/level')
def level = response.data[0].msr[0].data.level
might get you your value. The main point here is that client is an instance of RESTClient, response is a response object representing the http response from the server, and response.data contains the parsed json payload in the response.
You would need to experiment with the expression on the last line to pull out the 'level' value.