Django "Internal Server Error" and "MultiValueDictKeyError" when calling external function - python-3.x

I am running a django project on ubuntu that is supposed to build a website where I can
upload an image
run an external script to modify the image on click when uploaded
The uploading process works fine, but when I try to run the external script I get an the internal server error as seen below.
Is this because of the added b' and \n in the path? If so, how can I solve that, please?
Full Code can be found here https://github.com/hackanons/button-python-click/tree/master/Image%20Edit%20Html%20Button%20Run%20Python%20Script/buttonpython
Thanks a lot for help
image is birdie1.png
file raw url birdie1.png
file full url /home/felix/ucmr/button-python-click/Image_Edit_Html_Button_Run_Python_Script/buttonpython/media/birdie1.png
template url /media/birdie1.png
CompletedProcess(args=['/home/felix/anaconda3/envs/ucmr/bin/python', '//home//felix//ucmr//button-python-click//Image_Edit_Html_Button_Run_Python_Script//test.py', 'upload'], returncode=0, stdout=b'Hi upload welcome to Hackanons & time is 2021-06-01 20:35:53.229957\n')
b'/media/temp.png\n'
[01/Jun/2021 20:35:53] "POST /external/ HTTP/1.1" 200 1074
[01/Jun/2021 20:35:53] "GET /media/birdie1.png HTTP/1.1" 200 103100
Internal Server Error: /external/b'/media/temp.png/n'
Traceback (most recent call last):
File "/home/felix/anaconda3/envs/ucmr/lib/python3.7/site-packages/django/utils/datastructures.py", line 78, in __getitem__
list_ = super().__getitem__(key)
KeyError: 'image'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/felix/anaconda3/envs/ucmr/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/felix/anaconda3/envs/ucmr/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/felix/anaconda3/envs/ucmr/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/felix/ucmr/button-python-click/Image_Edit_Html_Button_Run_Python_Script/buttonpython/buttonpython/views.py", line 19, in external
image=request.FILES['image']
File "/home/felix/anaconda3/envs/ucmr/lib/python3.7/site-packages/django/utils/datastructures.py", line 80, in __getitem__
raise MultiValueDictKeyError(key)
django.utils.datastructures.MultiValueDictKeyError: 'image'
[01/Jun/2021 20:35:53] "GET /external/b'/media/temp.png/n' HTTP/1.1" 500 80417
views.py
from django.shortcuts import render
import requests
import sys
from subprocess import run,PIPE
from django.core.files.storage import FileSystemStorage
def button(request):
return render(request,'home.html')
def output(request):
data=requests.get("https://www.google.com/")
print(data.text)
data=data.text
return render(request,'home.html',{'data':data})
def external(request):
inp=request.POST.get('param', False)
image=request.FILES['image']
print("image is ",image)
fs=FileSystemStorage()
filename=fs.save(image.name, image)
fileurl=fs.open(filename)
templateurl=fs.url(filename)
print("file raw url",filename)
print("file full url", fileurl)
print("template url",templateurl)
out= run([sys.executable,'//home//felix//ucmr//button-python-click//Image_Edit_Html_Button_Run_Python_Script//test.py',inp],shell=False,stdout=PIPE)
image= run([sys.executable,'//home//felix//ucmr//button-python-click//Image_Edit_Html_Button_Run_Python_Script//image.py',str(fileurl),str(filename)],shell=False,stdout=PIPE)
print(out)
print(image.stdout)
return render(request,'home.html',{'data':out.stdout,'raw_url':templateurl,'edit_url':image.stdout})

The problem is that there is no image file in your request. You need to fix that first, make sure that you are getting what you expect.
You should probably start using forms and calling is_valid to do the check for you. If your request does not have the required fields, it will return an error and validation message.
Your first error happens here:
image=request.FILES['image'] # the 'image' key does not exists so you get KeyError
You can fix this using the .get notation to have a default if it doesn't exist.
image=request.FILES.get('image') # this will not throw an error but have `None` as default
After that change, you have the following:
filename=fs.save(image.name, image) # this throws an error if image is None because you can't do None.name
So now you should check that image is not None, and then continue with your logic, or return a BadRequest error response.
if image is not None:
return HttpResponseBadRequest
# <rest of your code>

The error
Internal Server Error: /external/b'/media/temp.png/n'
was caused by a corrupted path from the stdout Process. I added "universal_newlines=True" from subprocess module and everything works fine.
out= run([sys.executable,'//home//felix//ucmr//button-python-click//Image_Edit_Html_Button_Run_Python_Script//test.py',inp],shell=False,stdout=PIPE,universal_newlines=True)
image= run([sys.executable,'//home//felix//ucmr//button-python-click//Image_Edit_Html_Button_Run_Python_Script//image.py',str(fileurl),str(filename)],shell=False,stdout=PIPE,universal_newlines=True)
See also Why does Popen.communicate() return b'hi\n' instead of 'hi'?

Related

get_serving_url silently fails with django-storages, app engine, python 3.x

I am trying to get the seving_url for a project that uses django-storages with google cloud storage for media files.
I am trying to serve the files with get_serving_url, but I get a silent failure here with no text logged in the exception.
The blobkey generates correctly from what I can see
however the image = images.get_serving_url(blobkey, secure_url=True) raises an exception with no error text.
This is what I have done:
#storage_backends.py
class GoogleCloudMediaStorage(GoogleCloudStorage):
"""GoogleCloudStorage suitable for Django's Media files."""
def __init__(self, *args, **kwargs):
if not settings.MEDIA_URL:
raise Exception('MEDIA_URL has not been configured')
kwargs['bucket_name'] = setting('GS_MEDIA_BUCKET_NAME')
super(GoogleCloudMediaStorage, self).__init__(*args, **kwargs)
#this works fine
def url(self, name):
""".url that doesn't call Google."""
return urljoin(settings.MEDIA_URL, name)
#https://programtalk.com/python-examples/google.appengine.api.images.get_serving_url/_
#This does not work yet
def serving_url(self, name):
logging.info('serving url called')
if settings.DEBUG:
return urljoin(settings.MEDIA_URL, name)
else:
# Your app's GCS bucket and any folder structure you have used.
try:
logging.info('trying to get serving_url')
filename = settings.GS_MEDIA_BUCKET_NAME + '/' + name
logging.info(filename)
blobkey = blobstore.create_gs_key('/gs/' + filename)
logging.info('This is a blobkey')
logging.info(blobkey)
image = images.get_serving_url(blobkey, secure_url=True)
return image
except Exception as e:
logging.warn('didnt work')
logging.warn(e)
return urljoin(settings.MEDIA_URL, name)
I have appengine-python-standard installed
and I have wrapped my application
#main.py
from antiques_project.wsgi import application
from google.appengine.api import wrap_wsgi_app
app = wrap_wsgi_app(application)
I also have this in my app.yaml
app_engine_apis: true
I have tried to generate the blobkey in different ways (with and without bucket)
I have also tried secure_url = False and True
So far nothing seems to work
EDIT:
Got a traceback in the logs:
Traceback (most recent call last):
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/appengine/api/images/init.py", line 2013, in get_serving_url_hook
rpc.check_success()
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/appengine/api/apiproxy_stub_map.py", line 614, in check_success
self.__rpc.CheckSuccess()
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/appengine/api/apiproxy_rpc.py", line 149, in CheckSuccess
raise self.exception
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/appengine/runtime/default_api_stub.py", line 276, in _CaptureTrace
f(**kwargs)
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/appengine/runtime/default_api_stub.py", line 269, in _SendRequest
raise self._TranslateToError(parsed_response)
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/appengine/runtime/default_api_stub.py", line 138, in _TranslateToError
raise apiproxy_errors.ApplicationError(response.application_error.code,
google.appengine.runtime.apiproxy_errors.ApplicationError: ApplicationError: 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/srv/config/storage_backends.py", line 50, in serving_url
image = images.get_serving_url(blobkey, secure_url=True)
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/appengine/api/images/init.py", line 1911, in get_serving_url
return rpc.get_result()
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/appengine/api/apiproxy_stub_map.py", line 648, in get_result
return self.__get_result_hook(self)
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/appengine/api/images/init.py", line 2015, in get_serving_url_hook
raise _ToImagesError(e, readable_blob_key)
google.appengine.api.images.TransformationError

The CreateSend API responded with the following error - 301: Invalid CampaignID

I have updated my project to python 3.0 and Django 3.0
Here when i am trying to delete the campaign i am getting this issue
Here is my views.py
def form_valid(self, form):
context = self.get_context_data()
# now push to CM
# if it fails... unschedule it (!!!)
# first update the list so that the group data is correct
cm_campaign = None
cm_list = None
# generates the preview url with campaign monitor and reutrns it
client = self.request.user.client
# get id for client on campaign monitor
client_cm_id = get_client_campaign_monitor_id(client)
# delete first the current campaign
try:
cm_campaign = createsend.Campaign({'api_key': settings.CAMPAIGN_MONITOR_API_KEY}, self.object.campaign_create_send_id)
cm_campaign.delete()
self.object.campaign_create_send_id = None
self.object.save()
except (ClientError, ServerError, BadRequest, Unauthorized, NotFound, Unavailable) as e:
logger.exception(e)
context['errors'].append(CMManager().error_message(e.data.Code))
return self.render_to_response(context)
here is my error traceback
Traceback (most recent call last):
File "/home/project/test project/dev-1.8/mcam/emails/views.py", line 818, in form_valid
cm_campaign.delete()
File "/home/project/test/lib/python3.7/site-packages/createsend/campaign.py", line 110, in delete
response = self._delete("/campaigns/%s.json" % self.campaign_id)
File "/home/project/test/lib/python3.7/site-packages/createsend/createsend.py", line 250, in _delete
return self.make_request(path=path, method="DELETE", params=params)
File "/home/project/test/lib/python3.7/site-packages/createsend/createsend.py", line 215, in make_request
return self.handle_response(response.status, data)
File "/home/project/test/lib/python3.7/site-packages/createsend/createsend.py", line 230, in handle_response
raise Unauthorized(json_data)
createsend.createsend.Unauthorized: The CreateSend API responded with the following error - 301: Invalid CampaignID
How can i solve this issue plese help me

flask test 400 error page does not pass but it is working

I m redirecting 403 error to a personal page. It works fine but test does not pass. I ve other tests that works fine.
this is my code for test:
class TestErrorPages(TestBase):
def test_403_forbidden(self):
# create route to abort the request with the 403 Error
#self.app.route('/403')
def forbidden_error():
abort(403)
Test are not passed and error is:
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
yield
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/case.py", line 628, in run
testMethod()
File "/Users/xxxxx/PycharmProjects/setup/test.py", line 184, in test_403_forbidden
self.assertTrue("403 Error" in response.data)
TypeError: a bytes-like object is required, not 'str'
Where is my error ?
You want to look at response.text
Here's an example:
self.assertTrue("403 Forbidden" in response.text)
A better assertion is to look for the status code.
Here's an example using python requests library
self.assertTrue(403 == response.status_code)

exchangelib KeyError: 'contacts' or KeyError: 'inbox' or

I want to search in contact list of some account and get that contacts details back, but none of folders that told here not work because of KeyError exception .
Somehow i can't access to any of exchange account folders.
Is it permission or ... ?
Code :
from exchangelib import Credentials, Account, Configuration
from exchangelib.protocol import NoVerifyHTTPAdapter, BaseProtocol
BaseProtocol.HTTP_ADAPTER_CLS = NoVerifyHTTPAdapter
credentials = Credentials("YYY#XXX.com", 'PASSWORD')
account = Account(
primary_smtp_address="Account#XXX.com",
autodiscover=True,
credentials=credentials
)
print(account) # work properly with printing my account
print(account.contacts) # not work with KeyError Exception
Error :
Warning (from warnings module):
File "C:\Python\lib\site-packages\urllib3\connectionpool.py", line 857
InsecureRequestWarning)
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
Traceback (most recent call last):
File "C:\Python\lib\site-packages\cached_property.py", line 69, in __get__
return obj_dict[name]
KeyError: 'contacts'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\dvp7\Desktop\ex.py", line 20, in <module>
print(account.contacts)
File "C:\Python\lib\site-packages\cached_property.py", line 73, in __get__
return obj_dict.setdefault(name, self.func(obj))
File "C:\Python\lib\site-packages\exchangelib\account.py", line 169, in contacts
return self.root.get_default_folder(Contacts)
File "C:\Python\lib\site-packages\exchangelib\folders.py", line 965, in get_default_folder
for f in self._folders_map.values():
File "C:\Python\lib\site-packages\exchangelib\folders.py", line 928, in _folders_map
for f in FolderCollection(account=self.account, folders=distinguished_folders).get_folders():
File "C:\Python\lib\site-packages\exchangelib\services.py", line 1053, in call
shape=shape,
File "C:\Python\lib\site-packages\exchangelib\services.py", line 88, in _get_elements
response = self._get_response_xml(payload=payload)
File "C:\Python\lib\site-packages\exchangelib\services.py", line 189, in _get_response_xml
raise rme
File "C:\Python\lib\site-packages\exchangelib\services.py", line 171, in _get_response_xml
res = self._get_soap_payload(soap_response=soap_response_payload)
File "C:\Python\lib\site-packages\exchangelib\services.py", line 227, in _get_soap_payload
cls._raise_soap_errors(fault=fault) # Will throw SOAPError or custom EWS error
File "C:\Python\lib\site-packages\exchangelib\services.py", line 261, in _raise_soap_errors
raise vars(errors)[code](msg)
exchangelib.errors.ErrorInternalServerError: An internal server error occurred. The operation failed.
Build version :
Build=15.0.847.31, API=Exchange2013_SP1, Fullname=Microsoft Exchange Server 2013 SP1
This method is work :
account.root.walk() # output : <exchangelib.folders.FolderCollection object at 0x03ADCA90>
But when i append filter to it above error occurred.
KeyError: 'folders'
only root folder is works fine and there is nothing in it !
print(account.root.all()) # QuerySet(q=Q(), folders=[Root (root)])
For the record, this turned out to be a misbehaving archive inbox folder. Workaround provided in https://github.com/ecederstrand/exchangelib/issues/431#issuecomment-409832287 by telling exchangelib to ignore this folder:
from exchangelib.folders import ArchiveInbox
from exchangelib.version import EXCHANGE_2016
# Set to something newer than your current version
ArchiveInbox.supported_from = EXCHANGE_2016
For those who find this issue by searching for:
exchangelib KeyError: 'inbox'
the fix for us was to upgrade past 1.11.4. The was necessary after 9:15 am PST today

How to response with PIL image in Cherrypy dynamically (Python3)?

It seems the task is easy but...
I have simple PIL.Image object. How to make Cherrypy response with this image dynamically?
def get_image(self, data_id):
cherrypy.response.headers['Content-Type'] = 'image/png'
img = PIL.Image.frombytes(...)
buffer = io.StringIO()
img.save(buffer, 'PNG')
return buffer.getvalue()
This code gives me:
500 Internal Server Error
The server encountered an unexpected condition which prevented it from fulfilling the request.
Traceback (most recent call last):
File "C:\Users\Serge\AppData\Local\Programs\Python\Python36\lib\site-packages\cherrypy\_cprequest.py", line 631, in respond
self._do_respond(path_info)
File "C:\Users\Serge\AppData\Local\Programs\Python\Python36\lib\site-packages\cherrypy\_cprequest.py", line 690, in _do_respond
response.body = self.handler()
File "C:\Users\Serge\AppData\Local\Programs\Python\Python36\lib\site-packages\cherrypy\_cpdispatch.py", line 60, in __call__
return self.callable(*self.args, **self.kwargs)
File "D:\Dev\Bf\webapp\controllers\calculation.py", line 69, in get_image
img.save(buffer, 'PNG')
File "C:\Users\Serge\AppData\Local\Programs\Python\Python36\lib\site-packages\PIL\Image.py", line 1930, in save
save_handler(self, fp, filename)
File "C:\Users\Serge\AppData\Local\Programs\Python\Python36\lib\site-packages\PIL\PngImagePlugin.py", line 731, in _save
fp.write(_MAGIC)
TypeError: string argument expected, got 'bytes'
Can someone help me please?
Use io.BytesIO() instead of io.StringIO(). (From this answer.)

Resources