I'm writing a program that creates the HTTP Request and sends i securely using SSL. It uses the python library SOCKET to send/receive data. The issue I'm having is that I'm receiving a 200 status back, but the body is not as expected. I've ran the same URI through POSTMAN with no issues.
Here is the full response in bytes (Bytes on wire are smaller than buffer (4096):
b'HTTP/1.1 200 OK\r\nDate: Wed, 20 May 2020 23:42:55 GMT\r\nServer: Apache\r\nStrict-Transport-Security: max-age=315
36000; includeSubDomains\r\nCache-Control: no-cache="Set-Cookie, Set-Cookie2"\r\nAccept-Ranges: bytes\r\nVary: Accep
t-Charset,Accept-Encoding,Accept-Language,Accept\r\nX-Frame-Options: SAMEORIGIN\r\nX-UA-Compatible: IE=edge\r\nTrans
fer-Encoding: chunked\r\nContent-Type: application/json\r\n\r\n845\r\n'
I'm expecting JSON data following 845\r\n like i see in my other responses but nothing.
My code for sending the request is:
'''
for i in nic_list:
try:
create_socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
bind_addr_socket = create_socket.bind((i, 0))
with socket.create_connection((net_loc, port)) as sock:
with context.wrap_socket(sock, server_hostname=net_loc) as ssock:
parse = urlparse(type)
http_type = 'GET {} HTTP/1.1\r\nHost:{}\r\nAuthorization: Basic {}\r\nX-auth-access-token: {}\r\nX-auth-refresh-token: {}\r\nContent-Type: text/html\r\n\r\n'.format(parse[2], net_loc, creds, access_token, refresh_token)
ssock.sendall(http_type.encode())
response = ssock.recv(4096)
except (socket.gaierror, ValueError) as error:
continue
'''
The http_type variable produces the follow HTML, I've modified the output below for confidentiality:
GET /api/fmc_config/v1/domain/ededededed-e0f2-11e3-8169-6d9ed49edef/policy/accesspolicies/000C29F5-FE9F-0ed3-0000-691
489779138/accessrules/0rffr29F5-FE9F-0ed3-0000-derfrfrfr524550 HTTP/1.1
Host:10.1.1.1
Authorization: Basic cfencklnelfkcnelkfnv=
X-auth-access-token: djeidjkfjrjf-frfjirfj
X-auth-refresh-token: frhfuhrufhrifh-fjrifjrifjirfj
Content-Type: text/html
Related
I am sending a file as an object via postman POST or PUT API like below:
How can I in Python -
get this file object
read and save
If you have a working request in Postman, you could copy autogenerated Code Snippet in Python - Requests format:
It might look like this:
import requests
url = "localhost:8080"
payload="<file contents here>"
headers = {
'Content-Type': 'application/octet-stream'
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)
Answer to this question which i finally implemented without url -
#app.route('/uploadFIle', methods=['PUT'])
def uploadFile():
chunk_size = 4096
with open("/Users/xyz/Documents/filename", 'wb') as f:
while True:
chunk = request.stream.read(chunk_size)
if len(chunk) == 0:
break
f.write(chunk)
return jsonify({"success":"File transfer initiated"})
I am loading multiple feed urls every minute
I want to send a http get request, get 200 status code and full data if the data has changed since the last time I loaded it
I want the http status code 304 and no response body if the data has not changed since the last time I loaded it
Python feedparser offers HERE this functionality if I use their library to send GET requests
How do I do this using aiohttp library
How do I send etag and last modified in the GET request
async with session.get(url) as response:
text = await response.text()
print(response.headers.get('etag'), response.headers.get('Last-Modified'))
How do I send etag and last modified and simulate behavior similar to the library above?
UPDATE 1
Here is some detailed code
import asyncio
import aiohttp
async def load_feed(session, url):
# Keep this an empty string for the first request
etag = 'fd31d1100c6390bd8a1f16d2703d56c0'
# Keep this an empty string for the first request
last_modified='Mon, 11 May 2020 22:27:44 GMT'
try:
async with session.get(url, headers={'etag': etag, 'Last-Modified': last_modified}) as response:
t = await response.text()
print(response.headers.get('etag'), response.headers.get('Last-Modified'), response.status, len(t), response.headers)
except Exception as e:
print(e)
async def load_feeds():
try:
async with aiohttp.ClientSession() as session:
tasks = []
for url in ['https://news.bitcoin.com/feed/']:
task = asyncio.ensure_future(load_feed(session, url))
tasks.append(task)
await asyncio.gather(*tasks, return_exceptions=True)
except:
pass
asyncio.get_event_loop().run_until_complete(load_feeds())
Expectation:
Send a request the first time without headers
Get a response code 200 with etag and Last-modified and full response
Send a request again with etag and Last-modified
Get a response code 304 with no response body
What is happening
- I am getting status code 200 and full response body every time
Tried a few other urls too like https://coindesk.com/feed
Last-Modified is a response header, for a request you would use If-Modified-Since:
async def load_feed(session, url):
headers = {
"ETag": "fd31d1100c6390bd8a1f16d2703d56c0",
"If-Modified-Since": "Mon, 11 May 2020 22:27:44 GMT"
}
try:
async with session.get(url, headers=headers) as response:
t = await response.text()
print(response.headers.get("ETag"), response.headers.get('Last-Modified'), response.status, len(t))
except Exception as e:
print(e)
Output (notice status=304):
"fd31d1100c6390bd8a1f16d2703d56c0" Mon, 11 May 2020 22:27:44 GMT 304 0
I have an app on App Engine Flex using the Python 3 runtime. I get the base64 encoded byte string of a resume file from Google Storage with the code below.
storage_client = storage.Client(project=[MYPROJECT])
bucket = storage_client.get_bucket([MYBUCKET])
blob = bucket.blob([MYKEY])
resume_file = blob.download_as_string()
resume_file = str(base64.b64encode(resume_file))[2:-1]
I send that as part of my request parameters like so:
headers = {'Authorization': 'Bearer {}'.format(signed_jwt),
'content-type': 'application/json'}
params = {'inputtype': 'file',
'resume': resume_file}
response = requests.get(HOST, headers=headers, params=params)
However, I get the following error:
Error 413 (Request Entity Too Large)
Your client issued a request that was too large
That's all we know
App Engine has a file size limit of 32MB. However, my file is only 24KB. How can I fix this error?
I had to change my application to accept POST requests instead of GET requests.
Previously, I was sending the resume as a parameter. I'm now sending it as data:
payload = json.dumps({
'inputtype': inputtype,
'resume': inputresume
})
response = requests.post(HOST, headers=headers, data=payload)
I am using Flask. Previously I was reading in the params using:
request.args.get('resume')
I changed it to:
request.get_json().get('resume')
This is an extension of the question/answer here
I am writing a script to make requests from different web services. I am having problem when posting data from the json data below. When I run the
patient_create_bill()function I get the logs below from response.
RESPONSE IS!!!!!
{'Error': 'JSON parse error - Expecting value: line 1 column 1 (char 0)'}
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 0.0.0.0:9000
DEBUG:urllib3.connectionpool:http://0.0.0.0:9000 "POST /api/bill/patient/bills/ HTTP/1.1" 400 72
Creating Patient Bill .....................................
I have attempted to make a POST on post man I get 201 response meaning the payload is fine there is nothing wrong with it.
This is my POST payload.
I have a separate file called mocks.py contains
has PATIENT_BILL_CREATE_PAYLOAD
PATIENT_BILL_CREATE_PAYLOAD = {
"bill_items": [{
"item": "Syringes",
"qty": 2,
"description": "Medicorp syringes"
}],
"bill_services": [{
"service": "Diagnosis",
"service_type": "1",
"duration": 5,
"description": "diagnosis"
}],
"client": "Sandra Hernandez"
}
This is the function
i've imported the PATIENT_BILL_CREATE_PAYLOAD and using it in this function.
def patient_create_bill(headers):
"""This function uses login creds provided and returns token plus logged in user data."""
url = "http://0.0.0.0:9000/api/bill/patient/bills/"
data = PATIENT_BILL_CREATE_PAYLOAD
res = requests.post(url, data=data, headers=headers)
res_data = res.json()
print("Creating Patient Bill .....................................\n")
return res_data
Your own answer is right (encode your data to json), and here is the code fixed. This worked for me:
instead of
res = requests.post(url, data=data, headers=headers)
the correct way to write it is...
import json
...
res = requests.post(url, data=json.dumps(data), headers=headers)
# or
res = requests.post(url, json=data, headers=headers)
More info about this type of requests in requests library docs.
My error was because of using data instead of json in request and also specified the header Content-Type as `application/json :-).
This log tells that you not received body in HTTP reponse (HTTP CODE 400):
DEBUG:urllib3.connectionpool:http://0.0.0.0:9000 "POST /api/bill/patient/bills/ HTTP/1.1" 400 72
Python trying to parse emtry string.
You can run this:
import json
json.loads('')
And this code will raise:
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I think, you should check your URL endpoint to call.
I'm using SoapUI for testing some REST api. As a response I'm getting the URL that should be an endpoint of the next request.
I made the following Property Transfer step
source : myApiCall
property : response
target : myHttpCall
property : endpoint
Everything would be ok, but when transferred the endpoint looks like "www.myurl.com" (with quotes) and thus is invalid. How do I strip the quotes from there?
Raw response :
HTTP/1.1 200 OK
Cache-Control: private
Content-Length: 98
Content-Type: application/json; charset=utf-8
Expires: Tue, 25 Oct 2016 09:04:28 GMT
Server: Microsoft-IIS/8.5
X-Powered-By: ASP.NET
Date: Tue, 25 Oct 2016 09:04:28 GMT
"http://myurl.com/query?queryUid=90e97bdb-00a3-47c2-8809-c15ceec6ea1b"
The problem is that your Raw response include the " quotes in the String. Then you've two possible solutions, remove the " from the Raw response and keep using the same property transfer.
Or if you can not change the response then you can use a Groovy script testStep to get the Raw response and manipulated it to remove the additional " quotes before set the endpoint:
// get your api call
def myApiCall = context.testCase.getTestStepByName('myApiCall')
// get the raw response
def responseUrl = myApiCall.getPropertyValue('Response')
// since your response contains the `"` remove it
responseUrl = responseUrl.replace('"','')
// set the endpoint correctly
def httpCall = context.testCase.getTestStepByName('myHttpCall')
httpCall.setPropertyValue("endpoint",responseUrl)