As docs says https://docs.djangoproject.com/en/2.2/topics/testing/tools/#exceptions;
The only exceptions that are not visible to the test client are Http404, PermissionDenied, SystemExit, and SuspiciousOperation. Django catches these exceptions internally and converts them into the appropriate HTTP response codes. In these cases, you can check response.status_code in your test.
I handle a custom a login mixin for raising an error to user logged in but has no access to that page.
class LoginQuizMarkerMixin(LoginRequiredMixin):
def dispatch(self, *args, **kwargs):
if self.request.user.is_authenticated:
if not self.request.user.has_perm('quizzes.view_sittings'):
raise PermissionDenied("You don't have any access to this page.")
return super().dispatch(*args, **kwargs)
I my unitest, it working fine.
class TestQuestionMarking(TestCase):
def setUp(self):
self.c1 = Category.objects.create_category(name='elderberries')
self.student = User.objects.create_user(username='student',
email='student#rebels.com',
password='top_secret')
self.teacher = User.objects.create_user(username='teacher',
email='teacher#jedis.com',
password='use_d#_force')
self.teacher.user_permissions.add(Permission.objects.get(codename='view_sittings'))
....
def test_paper_marking_list_view(self):
# should be 302 (redirection)
marking_url = reverse('quizzes:quiz_marking')
response = self.client.get(marking_url)
print('response1', response)
# should be 403 (permission denied)
self.client.login(username='student', password='top_secret')
response = self.client.get(marking_url)
print('response2', response)
# should be 200 (allowed)
self.client.login(username='teacher', password='use_d#_force')
response = self.client.get(marking_url)
print('response3', response)
But for somehow, it the error of PermissionDenied it showed an error like this:
docker-compose -f local.yml run --rm django python manage.py test labs.tests.quizzes.test_views.TestQuestionMarking.test_paper_marking_list_view
response1 <HttpResponseRedirect status_code=302, "text/html; charset=utf-8", url="/accounts/login/?next=/quizzes/marking/">
WARNING 2021-02-04 00:28:44,270 log 1 139681767683904 [log.py:222] Forbidden (Permission denied): /quizzes/marking/
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/usr/local/lib/python3.8/site-packages/django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "/app/my_app/quizzes/views.py", line 37, in dispatch
raise PermissionDenied("You don't have any access to this page.")
django.core.exceptions.PermissionDenied: You don't have any access to this page.
response2 <HttpResponseForbidden status_code=403, "text/html; charset=utf-8">
response3 <TemplateResponse status_code=200, "text/html; charset=utf-8">
And what I expected is still capured as an error, not showing up like above. And should be like this below:
response1 <HttpResponseRedirect status_code=302, "text/html; charset=utf-8", url="/accounts/login/?next=/quizzes/marking/">
response2 <HttpResponseForbidden status_code=403, "text/html; charset=utf-8">
response3 <TemplateResponse status_code=200, "text/html; charset=utf-8">
But, in another test when I try to change it with raise Http404, it worked fine:
class LoginQuizMarkerMixin(LoginRequiredMixin):
def dispatch(self, *args, **kwargs):
if self.request.user.is_authenticated:
if not self.request.user.has_perm('quizzes.view_sittings'):
# raise PermissionDenied("You don't have any access to this page.")
raise Http404
return super().dispatch(*args, **kwargs)
The console output:
response1 <HttpResponseRedirect status_code=302, "text/html; charset=utf-8", url="/accounts/login/?next=/quizzes/marking/">
WARNING 2021-02-04 13:39:05,368 log 1 139683990402880 [log.py:222] Not Found: /quizzes/marking/
response2 <HttpResponseNotFound status_code=404, "text/html; charset=utf-8">
response3 <TemplateResponse status_code=200, "text/html; charset=utf-8">
I fixed this by disabling my logging inside settings for test mode.
docker-compose -f local.yml run --rm django python manage.py test --settings=config.settings.test
and inside my config/settings/test.py;
LOGGING = {}
Related issue: How can I disable logging while running unit tests in Python Django?
Related
I used Python3 in windows 10 to write a django2 web app.
I tried to configure the LDAP login, but failed.
When I test using postman, it could get the reply successfully.
That is, I send a request to https://example.com/staff, with some authentication code and payload containing username and password, and it reply me with the LDAP reply.
However, when I tried to using ldap3 in Django, after successfully login, error shows:
AttributeError at /logout/
'tuple' object has no attribute 'backend'
code:
settings.py:
AUTHENTICATION_BACKENDS = (
'app.backends.LDAPBackend',
('django.contrib.auth.backends.ModelBackend'),
)
app/backends.py:
from django.contrib.auth import get_user_model
UserModel = get_user_model()
class LDAPBackend:
def authenticate(self, request, username=None, password=None, **kwargs):
try:
headers = {'Authorization': xxxxx}
body = {"username": username, "password": password}
response = requests.post(url="https://example.com/staff", json=body, headers=headers)
result = response.json()
print(result)
if result['code'] != "OK":
logger.info("Wrong Login Information")
return None
print("connected")
except Exception as e:
print(e)
user = UserModel.objects.update_or_create(username=username)
return user
def get_user(self, user_id):
try:
return UserModel._default_manager.get(pk=user_id)
except UserModel.DoesNotExist:
return None
error shows: AttributeError at /logout/ 'tuple' object has no attribute 'backend', below shows result in console:
connected
Internal Server Error: /logout/
Traceback (most recent call last):
File "C:\Users\software\python\lib\site-packages\django\core\handlers\exception.py", line 34, in inner
response = get_response(request)
File "C:\Users\software\python\lib\site-packages\django\core\handlers\base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "C:\Users\software\python\lib\site-packages\django\core\handlers\base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "C:\Users\software\python\lib\site-packages\django\views\generic\base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "C:\Users\software\python\lib\site-packages\django\utils\decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "C:\Users\software\python\lib\site-packages\django\views\decorators\debug.py", line 76, in sensitive_post_parameters_wrapper
return view(request, *args, **kwargs)
File "C:\Users\software\python\lib\site-packages\django\utils\decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "C:\Users\software\python\lib\site-packages\django\utils\decorators.py", line 142, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "C:\Users\software\python\lib\site-packages\django\utils\decorators.py", line 45, in _wrapper
return bound_method(*args, **kwargs)
File "C:\Users\software\python\lib\site-packages\django\views\decorators\cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "C:\Users\software\python\lib\site-packages\django\contrib\auth\views.py", line 61, in dispatch
return super().dispatch(request, *args, **kwargs)
File "C:\Users\software\python\lib\site-packages\django\views\generic\base.py", line 97, in dispatch
return handler(request, *args, **kwargs)
File "C:\Users\software\python\lib\site-packages\django\views\generic\edit.py", line 141, in post
if form.is_valid():
File "C:\Users\software\python\lib\site-packages\django\forms\forms.py", line 185, in is_valid
return self.is_bound and not self.errors
File "C:\Users\software\python\lib\site-packages\django\forms\forms.py", line 180, in errors
self.full_clean()
File "C:\Users\software\python\lib\site-packages\django\forms\forms.py", line 382, in full_clean
self._clean_form()
File "C:\Users\software\python\lib\site-packages\django\forms\forms.py", line 409, in _clean_form
cleaned_data = self.clean()
File "C:\Users\software\python\lib\site-packages\django\contrib\auth\forms.py", line 205, in clean
self.user_cache = authenticate(self.request, username=username, password=password)
File "C:\Users\software\python\lib\site-packages\django\contrib\auth\__init__.py", line 80, in authenticate
user.backend = backend_path
AttributeError: 'tuple' object has no attribute 'backend'
[22/Mar/2021 09:49:10] "POST /logout/ HTTP/1.1" 500 149071
You write the following line:
user = UserModel.objects.update_or_create(username=username)
The update_or_create method returns a tuple with the object and a boolean with whether the object was created. So you are storing this tuple in the variable user and returning that. But authenticate is supposed to return only the user causing unintended effects leading to an error. Also you should be using get_or_create [Django docs] instead of update_or_create.
So you should change that above line to:
user, created = UserModel.objects.get_or_create(username=username)
I need to access a Twitter user's timeline as a JSON string and return the first 250 chars.
Twitter1.py:
import urllib.request, urllib.parse, urllib.error
import twurl
import ssl
TWITTER_URL = 'https://api.twitter.com/1.1/statuses/user_timeline.json'
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
while True:
print('')
acct = input('Enter Twitter Account:')
if (len(acct) < 1): break
url = twurl.augment(TWITTER_URL,
{'screen_name': acct, 'count': '2'})
print('Retrieving', url)
connection = urllib.request.urlopen(url, context=ctx)
data = connection.read().decode()
print(data[:250])
headers = dict(connection.getheaders())
print('Remaining', headers['x-rate-limit-remaining'])
An error related to urllib occurs in output:
Enter Twitter Account:jack
...
Traceback (most recent call last):
File "C:\Users\User\...\twitter1.py", line 18, in <module>
connection = urllib.request.urlopen(url, context=ctx)
File "C:\Users\User\anaconda3\lib\urllib\request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\User\anaconda3\lib\urllib\request.py", line 531, in open
response = meth(req, response)
File "C:\Users\User\anaconda3\lib\urllib\request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Users\User\anaconda3\lib\urllib\request.py", line 569, in error
return self._call_chain(*args)
File "C:\Users\User\anaconda3\lib\urllib\request.py", line 503, in _call_chain
result = func(*args)
File "C:\Users\User\anaconda3\lib\urllib\request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
HTTPError: Bad Request
I cannot figure out the source of the issue. The syntax appears correct and the correct API information was entered into a separate python file 'hidden.py'. twurl and oauth were imported from twurl.py and oauth.py to access the data (included below). hidden.py simply returns my API info in JSON within a function oauth() and oauth is well known so it is also excluded here. Any guidance would be greatly appreciated.
twurl.py:
import urllib.request, urllib.parse, urllib.error
import oauth
import hidden
def augment(url, parameters):
secrets = hidden.oauth()
consumer = oauth.OAuthConsumer(secrets['consumer_key'],
secrets['consumer_secret'])
token = oauth.OAuthToken(secrets['token_key'], secrets['token_secret'])
oauth_request = oauth.OAuthRequest.from_consumer_and_token(consumer,
token=token, http_method='GET', http_url=url,
parameters=parameters)
oauth_request.sign_request(oauth.OAuthSignatureMethod_HMAC_SHA1(),
consumer, token)
return oauth_request.to_url()
Follow up: was resolved soon after I posted, the issue was regarding a domain being blocked by an antivirus filter.
Getting HTTP 400 bad request from testrail server when i try to post testcase result using urllib in python3. Appreciate if someone help me on this. Thanks!
Below is code,
import urllib.request
import json
import base64
data = {'results':[{'case_id': '123','status_id': '1','comment': 'This test passed', 'version': '0.14.0-W9'}]}
headers = {}
post_data = urllib.parse.urlencode(data).encode()
auth = base64.b64encode(b'user:pass')
auth = auth.decode()
headers['Authorization'] = 'Basic %s' % auth
headers['Content-Type'] = 'application/json'
request = urllib.request.Request("http://testrail.com/index.php?/api/v2/add_results_for_cases/272374", data = post_data, headers = headers)
response = urllib.request.urlopen(request).read()
result = json.loads(response)
print(result)
And error output,
Traceback (most recent call last):
File "p3.py", line 13, in <module>
response = urllib.request.urlopen(request).read()
File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request
Thanks Tomalak and Amiy for quick suggestions.
I have tried testrailAPI library of python3 and it works as expected.
I have a script(test.py) to test some api, like this:
def get_response(fct, data, method=GET):
"""
Performs the query to the server and returns a string containing the
response.
"""
assert(method in (GET, POST))
url = f'http://{hostname}:{port}/{fct}'
if method == GET:
encode_data = parse.urlencode(data)
response = request.urlopen(f'{url}?{encode_data}')
elif method == POST:
response = request.urlopen(url, parse.urlencode(data).encode('ascii'))
return response.read()
In terminal I call:
python test.py -H 0.0.0.0 -P 5000 --add-data
The traceback:
Traceback (most recent call last):
File "test.py", line 256, in <module>
add_plays()
File "test.py", line 82, in add_plays
get_response("add_channel", {"name": channel}, method=POST)
File "test.py", line 43, in get_response
response = request.urlopen(url, parse.urlencode(data).encode('ascii'))
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: BAD REQUEST
The data is {"name": "Channel1"}. I couldn't understand what is wrong. Please can someone give some tip or show whats's wrong?
When I call using curl, works:
curl -X POST -H "Content-Type: application/json" -d '{"name": "Channel1"}' http://0.0.0.0:5000/add_channel
I solved the problem change the test script:
The api was expected a JSON_MIME_TYPE = 'application/json', so I add a header in a request as follow bellow.
The scrit was using a wrong encode because some text in JSON couldn't be encode in Unicode, Eg:"Omö" encode in ascii launch the exception UnicodeEncodeError: 'ascii' codec can't encode character '\xf6' in position 1: ordinal not in range(128). So I changed to utf8.
Here is the fixed code:
def get_response(fct, data, method=GET):
"""
Performs the query to the server and returns a string containing the
response.
"""
assert(method in (GET, POST))
url = f'http://{hostname}:{port}/{fct}'
if method == GET:
encode_data = parse.urlencode(data)
req = request.Request(f'{url}?{encode_data}'
, headers={'content-type': 'application/json'})
response = request.urlopen(req)
elif method == POST:
params = json.dumps(data)
binary_data = params.encode('utf8')
req = request.Request(url
, data= binary_data
, headers={'content-type': 'application/json'})
response = request.urlopen(req)
x = response.read()
return x
I want to create custom request manager for crawler with dynamic waiting.
My crawler need to make requests to sites which prohibit parallel requests from same ip address. If such blocking occurs requests returns with HTTP error codes 403, 503, 429, etc.
In case of error I want to wait some time and repeat request. But for simplicity of parsers they just call get and receive correct page.
I want to use aiohttp and new async with syntax of Python 3.5 so my parsers classes can use async with for my requester class same way if they used aiohttp.ClientSession like this:
# somewhere in a parser
async def get_page(self, requester, page_index):
async with requester.get(URL_FMT.format(page_index)) as response:
html_content = await response.read()
result = self.parsing_page(html_content)
return result
if requester is aiohttp.ClientSession, then response is aiohtpp.ClientResponse which have __aenter__ and __aexit__ methods, so async with working as expected.
But if I put my requester class in the middle it is not working anymore.
Traceback (most recent call last):
File "/opt/project/api/tornado_runner.py", line 6, in <module>
from api import app
File "/opt/project/api/api.py", line 20, in <module>
loop.run_until_complete(session.login())
File "/usr/local/lib/python3.5/asyncio/base_events.py", line 337, in run_until_complete
return future.result()
File "/usr/local/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/usr/local/lib/python3.5/asyncio/tasks.py", line 239, in _step
result = coro.send(None)
File "/opt/project/api/viudata/session.py", line 72, in login
async with self.get('https://www.viudata.com') as resp:
AttributeError: __aexit__
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f44f61ef240>
It is looking like this.
class Requester:
def __init__(self, session: aiohttp.ClientSession):
self.session = session
async def get(self, *args, **kwargs):
is_result_successful = False
while not is_result_successful:
response = await self.session.get(*args, **kwargs)
if response.status in [503, 403, 429]:
await self.wait_some_time()
else:
is_result_successful = True
return response
From my understanding self.session.get is coroutine function so I will await it. Result is aiohttp.ClientResponse which have __aenter__ or __aexit__. But if return it parser's code of async with block return odd error.
Can you say what I need to replace to with my requester class as with aiohttp.ClientSession?
You should write additional code to support async with protocol.
See client.request() and _RequestContextManager for inspiration.