I managed to get data from the target (see code below), but I want to continuously download as the log file on the server is appended to.
from importlib.resources import path
import requests
import json
import shutil
import logging
logging.captureWarnings(True)
url = "https://<IP address>/wga/reverseproxy_logging/instance/wga/request.log"
headers = {'Accept': 'application/json'}
r = requests.get(url, stream=True, headers=headers, verify=False, auth=('user','password'))
with open('request.log', 'wb') as out_file:
shutil.copyfileobj(r.raw, out_file)
Related
Im trying To Make post requests (multipart/form-data) in this website https://gofile.io/?t=api
every time time i got an error when i try to upload file
my code
import requests
req = requests.session()
files= {'file': open("test.txt", 'rb')}
response = req.post('https://srv-file7.gofile.io/upload', files=files)
print(response.text)
I got error every time ,are smething missing in the code
from requests_toolbelt.multipart.encoder import MultipartEncoder
import requests
import json
mp_encoder = MultipartEncoder(
fields={
'filesUploaded': ('file_name.txt', open('file_name.txt', 'rb'))
}
)
r = requests.post(
'https://srv-file9.gofile.io/upload',
data=mp_encoder,
headers={'Content-Type': mp_encoder.content_type}
)
scrap = r.json()
# send "Token file" 123Abc
Token = scrap['data']['code']
Check this. It will work.
I was using Postman to send post request like on the screenshot
Now I need to implement it in python. This is what i've got for now:
import requests
data = {"sendRequest": {"apiKey": 12345, "field1": "field1value"}}
files = {"attachment": ("file.txt", open("file.txt", "rb"))}
headers = {"Content-type": "multipart/form-data"}
response = requests.post(endpoint, data=data, headers=headers, files=files)
But still it's not working - server doesn't accept it as valid request. I've tried more combinations but without any results and I really couldn't find a solution.
I need this request to be exactly like that one in postman
I finally found a solution. I used MultipartEncoder from requests_toolbelt library.
from requests_toolbelt import MultipartEncoder
import requests
import json
data = {"apiKey": 12345, "field1": "field1value"}}
mp = MultipartEncoder(
fields={
'sendRequest': json.dumps(data), # it is important to convert dict into json
'attachment': ('file.pdf', open('file.pdf', 'rb'), 'multipart/form-data'),
}
)
r = requests.post(ENDPOINT, data=mp, headers={'Content-Type': mp.content_type})
the code can run and debug
but it shows"unresolved import 'requests'Python(unresolved-import)"
is there any other settings what i missed?
import requests
import os
url = 'https://www.google.com'
response = requests.get(url)
response.encoding = 'utf-8'
html = response.text
path = os.path.abspath(os.path.dirname(__file__))+'\\test.txt'
with open(path, 'w', encoding = 'gbk') as f:
f.write(html)
I am logged in in a page and I use a bookmark to download a CSV file. I just click on the link and after some seconds, the file gets downloaded in my PC. I am now trying to automate the "downloading file" process from the URL in Python.
The URL that triggers a file download is the following:
app.example.com/export/org.jsp?media=yes&csv=yes
What I tried in Python3 is shown below:
##First way
import requests
payload = {'inUserName': 'test.test#test.com','inUserPass': 'test'}
with requests.Session() as s:
p = s.post('https://app.example.com/', data=payload)
#print(p.text)
r = s.get('https://app.example.com/export/org.jsp?media=yes&csv=yes')
###Second way
import urllib
import requests
payload = {'inUserName': 'test.test#test.com', 'inUserPass': 'test'}
url = 'https://app.example.com/'
requests.post(url, data=payload)
###Third way
import urllib.request
with urllib.request.urlopen("http://app.example.com/export/org.jsp?media=yes&csv=yes") as url:
s = url.read()
#print(s)
I want to avoid the page scraping technique where I will login in the page and then visit the url. The platform used does not have an API where I can request the file in a different way.
Here is the related code
import logging
logging.getLogger('googleapicliet.discovery_cache').setLevel(logging.ERROR)
import datetime
import json
from flask import Flask, render_template, request
from flask import make_response
from googleapiclient.discovery import build
from googleapiclient.http import MediaIoBaseDownload
from oauth2client.client import AccessTokenCredentials
...
#app.route('/callback_download')
def userselectioncallback_with_drive_api():
"""
Need to make it a background process
"""
logging.info("In download callback...")
code = request.args.get('code')
fileId = request.args.get('fileId')
logging.info("code %s", code)
logging.info("fileId %s", fileId)
credentials = AccessTokenCredentials(
code,
'flex-env/1.0')
http = httplib2.Http()
http_auth = credentials.authorize(http)
# Exports a Google Doc to the requested MIME type and returns the exported content. Please note that the exported content is limited to 10MB.
# v3 does not work? over quota?
drive_service = build('drive', 'v3', http=http_auth)
drive_request = drive_service.files().export(
fileId=fileId,
mimeType='application/pdf')
b = bytes()
fh = io.BytesIO(b)
downloader = MediaIoBaseDownload(fh, drive_request)
done = False
try:
while done is False:
status, done = downloader.next_chunk()
logging.log("Download %d%%.", int(status.progress() * 100))
except Exception as err:
logging.error(err)
logging.error(err.__class__)
response = make_response(fh.getbuffer())
response.headers['Content-Type'] = 'application/pdf'
response.headers['Content-Disposition'] = \
'inline; filename=%s.pdf' % 'yourfilename'
return response
It is based on some code example of drive api. I am trying to export some files from google drive to pdf format.
The exception comes from the line
response = make_response(fh.getbuffer())
It throws the exception:
TypeError: 'memoryview' object is not callable
How can I retrieve the pdf content properly from the fh? Do I need to further apply some base 64 encoding?
My local runtime is python 3.4.3
I have used an incorrect API. I should do this instead:
response = make_response(fh.getvalue())