HTTP headers format using python's requests - python-3.x

Question

I use python requests to capture a website's http headers. For example, this is a response header:
{'Connection': 'keep-alive',
'Access-Control-Allow-Origin': '*', 'cache-control': 'max-age=600',
'Content-Type': 'text/html; charset=utf-8', 'Expires': 'Fri, 19 Apr
2019 03:16:28 GMT', 'Via': '1.1 varnish, 1.1 varnish', 'X-ESI': 'on',
'Verso': 'false', 'Accept-Ranges': 'none', 'Date': 'Fri, 19 Apr 2019
03:11:12 GMT', 'Age': '283', 'Set-Cookie':
'CN_xid=08f66bff-4001-4173-b4e2-71ac31bb58d7; Expires=Wed, 16 Oct 2019
03:11:12 GMT; path=/;, xid1=1; Expires=Fri, 19 Apr 2019 03:11:27 GMT;
path=/;, verso_bucket=281; Expires=Sat, 18 Apr 2020 03:11:12 GMT;
path=/;', 'X-Served-By': 'cache-iad2133-IAD, cache-gru17122-GRU',
'X-Cache': 'HIT, MISS', 'X-Cache-Hits': '1, 0', 'X-Timer':
'S1555643472.999490,VS0,VE302', 'Content-Security-Policy':
"default-src https: data: 'unsafe-inline' 'unsafe-eval'; child-src
https: data: blob:; connect-src https: data: blob:; font-src https:
data:; img-src https: data: blob:; media-src https: data: blob:;
object-src https:; script-src https: data: blob: 'unsafe-inline'
'unsafe-eval'; style-src https: 'unsafe-inline';
block-all-mixed-content; upgrade-insecure-requests; report-uri
https://l.com/csp/gq",
'X-Fastly-Device-Detect': 'desktop', 'Strict-Transport-Security':
'max-age=7776000; preload', 'Vary': 'Accept-Encoding, Verso,
Accept-Encoding', 'content-encoding': 'gzip', 'transfer-encoding':
'chunked'}
I noted that from several examples I tested, the headers I receive from requests are formatted as 'key':'value' (plz note the single colons surrounding the key and the value). However, when I check the headers from the Firefox-> Web developer -> Inspector, and choose to view the header in raw format, I do not see commas:
HTTP/2.0 200 OK date: Thu, 09 May 2019 18:49:07 GMT expires: -1
cache-control: private, max-age=0 content-type: text/html;
charset=UTF-8 strict-transport-security: max-age=31536000
content-encoding: br server: gws content-length: 55844
x-xss-protection: 0 x-frame-options: SAMEORIGIN set-cookie:
1P_JAR=2019-05-09-18; expires=Sat, 08-Jun-2019 18:49:07 GMT; path=/;
domain=.google.com alt-svc: quic=":443"; ma=2592000; v="46,44,43,39"
X-Firefox-Spdy: h2
I need to know: Does python's requests module always adds single colons? This important from me as I need to include/exclude them in my regex that is used to analyze the headers.

Answer 1

The issue I think you are running into is the request coming back as a dict instead of a value as firefox inspector is giving you. When you do this you could be getting mixed results if one of the value pairs has a numeric or boolean value so when doing your regex you may want to use a Try/Except if you can remove the exterior apostrophes or just use the value given.

Answer 2

It's not the requests module that's adding the colons. Request represents headers as a dict, but you seem to be treating them as a string. When Python converts dicts to strings, they get the colons, the commas, the quotation marks.
The right fix for your program is probably to treat the dictionary as a dictionary, not convert it into a string. But if you really want the headers in string form, you should consider using different tool, such as curl.