I'm attempting to write a Twisted Webserver in Python 3.6 that can upload multiple files but being fairly new to both Python and things Web I have ran into an issue I don't understand and I also do not find any good examples relating to multiple file upload.
With the following code I get
from twisted.web import server, resource
from twisted.internet import reactor, endpoints
import cgi
class Counter(resource.Resource):
isLeaf = True
numberRequests = 0
def render_GET(self, request):
print("GET " + str(self.numberRequests))
self.numberRequests += 1
# request.setHeader(b"content-type", b"text/plain")
# content = u"I am request #{}\n".format(self.numberRequests)
content = """<html>
<body>
<form enctype="multipart/form-data" method="POST">
Text: <input name="text1" type="text" /><br />
File: <input name="file1" type="file" multiple /><br />
<input type="submit" />
</form>
</body>
</html>"""
print(request.uri)
return content.encode("ascii")
def render_POST(selfself, request):
postheaders = request.getAllHeaders()
try:
postfile = cgi.FieldStorage(
fp=request.content,
headers=postheaders,
environ={'REQUEST_METHOD': 'POST',
# 'CONTENT_TYPE': postheaders['content-type'], Gives builtins.KeyError: 'content-type'
}
)
except Exception as e:
print('something went wrong: ' + str(e))
filename = postfile["file"].filename #file1 tag also does not work
print(filename)
file = request.args["file"][0] #file1 tag also does not work
endpoints.serverFromString(reactor, "tcp:1234").listen(server.Site(Counter()))
reactor.run()
Error log
C:\Users\theuser\AppData\Local\conda\conda\envs\py36\python.exe C:/Users/theuser/PycharmProjects/myproject/twweb.py
GET 0
b'/'
# My comment POST starts here
Unhandled Error
Traceback (most recent call last):
File "C:\Users\theuser\AppData\Local\conda\conda\envs\py36\lib\site-packages\twisted\web\http.py", line 1614, in dataReceived
finishCallback(data[contentLength:])
File "C:\Users\theuser\AppData\Local\conda\conda\envs\py36\lib\site-packages\twisted\web\http.py", line 2029, in _finishRequestBody
self.allContentReceived()
File "C:\Users\theuser\AppData\Local\conda\conda\envs\py36\lib\site-packages\twisted\web\http.py", line 2104, in allContentReceived
req.requestReceived(command, path, version)
File "C:\Users\theuser\AppData\Local\conda\conda\envs\py36\lib\site-packages\twisted\web\http.py", line 866, in requestReceived
self.process()
--- <exception caught here> ---
File "C:\Users\theuser\AppData\Local\conda\conda\envs\py36\lib\site-packages\twisted\web\server.py", line 195, in process
self.render(resrc)
File "C:\Users\theuser\AppData\Local\conda\conda\envs\py36\lib\site-packages\twisted\web\server.py", line 255, in render
body = resrc.render(self)
File "C:\Users\theuser\AppData\Local\conda\conda\envs\py36\lib\site-packages\twisted\web\resource.py", line 250, in render
return m(request)
File "C:/Users/theuser/PycharmProjects/myproject/twweb.py", line 42, in render_POST
filename = postfile["file"].filename #file1 tag also does not work
File "C:\Users\theuser\AppData\Local\conda\conda\envs\py36\lib\cgi.py", line 604, in __getitem__
raise KeyError(key)
builtins.KeyError: 'file'
I don't understand how to get hold of the file or files so I can save them uploaded after the form Submit in render_POST. This post SO appears to not have this problem. The final app should be able to do this asynchronous for multiple users but before that I would be happy to get just a simple app working.
Using conda on Windows 10 with Python 3.6
FieldStorage in Python 3+ returns a dict with keys as bytes, not strings. So you must access keys like this:
postfile[ b"file" ]
Notice that the key is appended with b"". It's a tad bit confusing if you're new to Python and are unaware of the changes between Python 3 and 2 strings.
Also I answered a similar question a while back but was unable to get it working properly on Python 3.4 but I can't remember what exactly didn't work. Hopefully you don't run into any issues using 3.6.
Related
I have already created an account in picovoice and recieved an access key, but when I try to put the path of my ppn file, it shows an error:
`
[ERROR] loading keyword file at 'C:\Users\Priyam\Desktop\hey keyazip' failed with 'IO_ERROR'
Traceback (most recent call last):
File "e:\Personal Project\import struct.py", line 13, in <module>
porcupine = pvporcupine.create(access_key='access key',
File "C:\Users\Priyam\AppData\Roaming\Python\Python310\site-packages\pvporcupine\__init__.py", line 77, in create
return Porcupine(
File "C:\Users\Priyam\AppData\Roaming\Python\Python310\site-packages\pvporcupine\porcupine.py", line 158, in __init__
raise self._PICOVOICE_STATUS_TO_EXCEPTION[status]()
pvporcupine.porcupine.PorcupineIOError
the code is:
`
porcupine=None
paud=None
audio_stream=None
try:
access_key="access key"
porcupine = pvporcupine.create(access_key='access key',
keyword_paths=['C:\\Users\Priyam\Desktop\hey keyazip'],keywords=['hey keya']) #pvporcupine.KEYWORDS for all keywords
paud=pyaudio.PyAudio()
audio_stream=paud.open(rate=porcupine.sample_rate,channels=1,format=pyaudio.paInt16,input=True,frames_per_buffer=porcupine.frame_length)
while True:
keyword=audio_stream.read(porcupine.frame_length)
keyword=struct.unpack_from("h"*porcupine.frame_length,keyword)
keyword_index=porcupine.process(keyword)
if keyword_index>=0:
print("hotword detected")
finally:
if porcupine is not None:
porcupine.delete()
if audio_stream is not None:
audio_stream.close()
if paud is not None:
paud.terminate()
`
I tried the code above and the code provided by picovoice itself, yet I am facing the same issues
It looks like Porcupine is having trouble finding your keyword file: [ERROR] loading keyword file at 'C:\Users\Priyam\Desktop\hey keyazip' failed with 'IO_ERROR'.
The Picovoice console provides the keyword inside a compressed .zip file. You will need to decompress the file and in your code update the path to lead to the .ppn file located inside. For example: C:\\Users\Priyam\Desktop\hey-keya_en_windows_v2_1_0\hey-keya_en_windows_v2_1_0.ppn
Am just having four weeks of experience in Python. Creating a tool using Tkinter to paste a new company logo on top of the existing images.
The Below method is to, get all images in the given directory and paste the new logo on the initial level. Existing image, edited image, x-position, y-position, a preview of the image and few data's are store in global instance self.images_all_arr.
def get_img_copy(self):
self.images_all_arr = []
existing_img_fldr = self.input_frame.input_frame_data['existing_img_folder']
for file in os.listdir(existing_img_fldr):
img_old = Image.open(os.path.join(existing_img_fldr, file))
img_new_copy = img_old.copy()
self.pasteImage(img_new_copy, initpaste=True) #process to paste new logo.
view_new_img = ImageTk.PhotoImage(img_new_copy)
fname, fext = file.split('.')
formObj = {
"fname": fname,
"fext": fext,
"img_old": img_old,
"img_new": img_new_copy,
"img_new_view": view_new_img,
"add_logo": 1,
"is_default": 1,
"is_opacityImg": 0,
"pos_x": self.defult_x.get(),
"pos_y": self.defult_y.get()
}
self.images_all_arr.append(formObj)
After previewing each image in Tkinter screen, doing some adjustment in position x and y(updating pos_x and pos_y in the list self.images_all_arr) depends upon the necessity.
Well, once all done. Need to save the edited images. Below method to save images, iterating the list self.images_all_arr and call save method as img['img_new'].save(dir_output) since img['img_new'] has updated image.
def generate_imgae(self):
if len(self.images_all_arr):
dir_output = 'xxxxx'
for img in self.images_all_arr:
print(img['img_new'])
img['img_new'].save(dir_output)
print('completed..')
But it returns below error,
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Program Files (x86)\Python38-32\lib\site-packages\PIL\Image.py", line 2138, in save
format = EXTENSION[ext]
KeyError: ''
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Program Files (x86)\Python38-32\lib\tkinter_init_.py", line 1883, in call
return self.func(*args)
File "C:\Users\662828\WORKSPACE\AAA_python_projects\AMI_automation_poc2\position_and_images.py", line 241, in generate_imgae
img['img_new'].save(dir_output)
File "C:\Program Files (x86)\Python38-32\lib\site-packages\PIL\Image.py", line 2140, in save
raise ValueError("unknown file extension: {}".format(ext)) from e
ValueError: unknown file extension:
dir_output doesn't contain the file extension, its just xxxxx. You need to specify what image file format you want. the error tells us this by saying "unknown file format".
Basically, you either need to include the extension in the file name, or pass it as the next parameter in image.save. You can check out the documentation here
eg.
image.save('image.png')
or
image.save('image', 'png')
The below code solved my issue. By giving the exact directory, filename and extension of the image as param to the method image.save(). Here the result of opfile is, C:\Users\WORKSPACE\AAA_python_projects\test\valid.png.
def generate_imgae(self):
if len(self.images_all_arr):
dir_output = r"C:\Users\WORKSPACE\AAA_python_projects\test"
if not os.path.isdir(dir_output):
os.mkdir(dir_output)
for img in self.images_all_arr:
opfile = os.path.join(dir_output, '{}.{}'.format(img['fname'], img['fext'] ))
img['img_new'].save(opfile)
print('completed..')
Thanks #dantechguy
Teams seems to lack any native way of mirroring files to a shared directory. I'm Trying use Python (or another language but python preferred!) to either:
a. Directly pull from microsoft teams into memory using Python to process with Pandas
b. Copy files from teams into a shared network folder (which Python could then read in)
I found this but can't get it to work with teams - the teams URLs don't look anything like these do. How to read SharePoint Online (Office365) Excel files in Python with Work or School Account?
It seems close to what I want to do though. I also found "pymsteams" on PyPi repository. https://pypi.org/project/pymsteams/ which just seems to let you send messages to Teams and nothing else? unless I misunderstand something.
https://pypi.org/project/Office365-REST-Python-Client/
https://pypi.org/project/pymsteams/
from office365.runtime.auth.authentication_context
import AuthenticationContext
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.file import File
url = 'https://teams.microsoft.com/l/file'
username = 'myusername'
password = 'mypassword'
relative_url ='myurl'
ctx_auth = AuthenticationContext(url)
ctx_auth.acquire_token_for_user(username, password)
Trying to run the above code gives
AttributeError: 'NoneType' object has no attribute 'text'
Full stack trace:
runfile('H:/repos/foo/untitled0.py', wdir='H:/repos/foo')
Traceback (most recent call last):
File "<ipython-input-35-314ab7dc63c9>", line 1, in <module>
runfile('H:/repos/foo/untitled0.py', wdir='H:/foo/image_ai')
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
execfile(filename, namespace)
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "H:/repos/image_ai/untitled0.py", line 10, in <module>
ctx_auth.acquire_token_for_user(username, password)
File "C:\ProgramData\Anaconda3\lib\site-packages\office365\runtime\auth\authentication_context.py", line 18, in acquire_token_for_user
return self.provider.acquire_token()
File "C:\ProgramData\Anaconda3\lib\site-packages\office365\runtime\auth\saml_token_provider.py", line 57, in acquire_token
self.acquire_service_token(options)
File "C:\ProgramData\Anaconda3\lib\site-packages\office365\runtime\auth\saml_token_provider.py", line 88, in acquire_service_token
token = self.process_service_token_response(response)
File "C:\ProgramData\Anaconda3\lib\site-packages\office365\runtime\auth\saml_token_provider.py", line 119, in process_service_token_response
return token.text
AttributeError: 'NoneType' object has no attribute 'text'
I've managed to get it working using the Office-365-REST-Python-Client you linked.
If you're using SharePoint Online, you'll need to get an App-Only principle set up and connect using the acquire_token_for_app function instead of acquire_token_for_user, then passing a client_id and client_secret instead of username & password.
from office365.runtime.auth.authentication_context import AuthenticationContext
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.file import File
client_id = 'yourclientid'
client_secret = 'yourclientsecret'
url = 'https://yoursharepointsite.com/teams/yourteam'
relative_url = '/teams/yourteam/Shared%20Documents/yourteamschannel/yourdoc.extension'
ctx_auth = AuthenticationContext(url)
if ctx_auth.acquire_token_for_app(client_id, client_secret):
ctx = ClientContext(url, ctx_auth)
with open(filename, 'wb') as output_file:
response = File.open_binary(ctx, relative_url)
output_file.write(response.content)
else:
print(ctx_auth.get_last_error())
This should download your file to a local drive (specified via the filename variable) and you can then load into pandas etc to process
I am trying to sign in to a website using RoboBrowser and I am stuck with a error message.
My code:
from robobrowser import RoboBrowser
browser = RoboBrowser()
def login():
browser.open('https://www.kijiji.ca/t-login.html')
form = browser.get_form(id="login-form")
form.fields["LoginEmailOrNickname"].value = "an_email_address"
form.fields["login-password"].value = "a_password"
form.fields["login-rememberMe"].value = "true"
browser.submit_form(form)
login()
The error message:
Traceback (most recent call last):
File "/home/rojaslee/Desktop/kijiji_poster/kijiji_poster.py", line 16, in <module>
login()
File "/home/rojaslee/Desktop/kijiji_poster/kijiji_poster.py", line 11, in login
form.fields["LoginEmailOrNickname"].value = ["an_email_address"]
File "/usr/local/lib/python3.4/dist-packages/werkzeug/datastructures.py", line 744, in __getitem__
raise exceptions.BadRequestKeyError(key)
werkzeug.exceptions.BadRequestKeyError: 400: Bad Request
Old thread, but I've been struggling with the same problem. This worked for me:
from robobrowser import Robobrowser
browser = RoboBrowser
def login():
browser.open('https://www.kijiji.ca/t-login.html')
form = browser.get_form(id="login-form")
form["EmailOrNickname"].value = "an_email_address"
form["password"].value = "a_password"
form["rememberMe"].value = "checked"
browser.submit_form(form)
The HTML code from the web site you want to log in is as follows:
<section>
<label for="LoginEmailOrNickname">Email Address or Nickname:</label>
<input id="LoginEmailOrNickname" name="emailOrNickname" req="req" type="text" value="" maxlength="128"><span class="field-message" data-for="LoginEmailOrNickname"></span>
</section>
<section>
<label for="login-password">Password:</label>
<input id="login-password" name="password" req="req" type="password" value="" maxlength="64"><span class="field-message" data-for="login-password"></span>
<a id="LoginForgottenPassword" href="/t-forgot-password.html">Forgot your password?</a>
</section>
To put a value on the form fields you have to get the name attribute, not the id.
This code should work:
form.fields["emailOrNickname"].value = "an_email_address"
form.fields["password"].value = "a_password"
form.fields["rememberMe"].value = "true"
If you need to get the fields of the form you can print them:
print form.fields
But have you tried using a 'try and except'?
I was able to get around this error using:
try:
form['userid'] = 'my_username'
form['password'] = 'my_password'
except Exception:
pass
I used a different website than you so make sure you input the correct form fields.
let me know if this helps!
I am trying to access a website to scrape some information, however I am having trouble posting login information through Python. Here is my code so far:
import requests
c = requests.Session()
url = 'https://subscriber.hoovers.com/H/login/login.html'
USERNAME = 'user'
PASSWORD = 'pass'
c.get(url)
csrftoken = c.cookies['csrftoken']
login_data = dict(j_username=USERNAME, j_password=PASSWORD,
csrfmiddlewaretoken=csrftoken, next='/')
c.post(url, data=login_data, headers=dict(Referer=url))
page = c.get('http://subscriber.hoovers.com/H/home/index.html')
print(page.content)
Here is the form data from the post login page:
j_username:user
j_password:pass
OWASP_CSRFTOKEN:8N0Z-TND5-NV71-C4N4-43BK-B13S-A1MO-NZQC
OWASP_CSRFTOKEN:8N0Z-TND5-NV71-C4N4-43BK-B13S-A1MO-NZQC
Here is the error I receive:
Traceback (most recent call last):
File "C:/Users/10023539/Desktop/pyscripts/webscraper ex.py", line 9, in <module>
csrftoken = c.cookies['csrftoken']
File "C:\Program Files (x86)\Python35-32\Lib\site-packages\requests\cookies.py", line 293, in __getitem__
return self._find_no_duplicates(name)
File "C:\Program Files (x86)\Python35-32\Lib\site-packages\requests\cookies.py", line 351, in _find_no_duplicates
raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path))
KeyError: "name='csrftoken', domain=None, path=None"
I believe the issue has something to do with the 'OWASP_CSRFTOKEN' label? I haven't found any solutions for this specific CSRF name anywhere online. I've also tried removing the c.cookies method and manually typing in the CSRF code into the csrfmiddlewaretoken argument. I've also tried changing the referal URL around, still getting the same error.
Any assistance would be greatly appreciated.
First of all you catch KeyError exception, this mean that cookies dictionary have no key csrftoken.
So you need explore your response for find right CSRF token cookie name.
For example you can print all cookies:
for key in c.cookies.keys():
print('%s: %s' % (key, c.cookies[key]))
UPD: Actually your response have no CSRF cookie.
you need look token in your c.text with pyquery
<input type="hidden" name="OWASP_CSRFTOKEN" class="csrfClass" value="X48L-NEYI-CG18-SJOD-VDW9-FGEB-7WIT-88P4">