Why is #pytest.fixture passed as an argument to tests but not used - python-3.x

Below is a test file for the Streamlink library. I was reading the test files to understand how testing is done and found a confusing set up for the tests. The test set up a series of #pytest.fixtures to help with testing. The assert_live fixture below is defined and passed into some tests, but never called at all. I dont think this is a mistake because a similar pattern is found throughout the file. I read the docs for fixtures but couldn't find anything that mentions this functionality. I got excited when they talked about using yeild inside a fixture but that turned not to touch on the subject either. Google was even less helpful. I am at a loss, but I am very interested in contributing to this library and need to understand how it works and how they test. Test is at the bottom of the code below. Thank you in advance!
# Other fixtures for context/continuity
#pytest.fixture
def mocker(self):
# The built-in requests_mock fixture is bad when trying to reference the following constants or classes
with requests_mock.Mocker() as mocker:
mocker.register_uri(requests_mock.ANY, requests_mock.ANY, exc=requests_mock.exceptions.InvalidRequest)
yield mocker
#pytest.fixture
def mock(self, request, mocker: requests_mock.Mocker):
mock = mocker.post("https://gql.twitch.tv/gql", **getattr(request, "param", {"json": {}}))
yield mock
assert mock.called_once
payload = mock.last_request.json() # type: ignore[union-attr]
assert tuple(sorted(payload.keys())) == ("extensions", "operationName", "variables")
assert payload.get("operationName") == "PlaybackAccessToken"
assert payload.get("extensions") == {
"persistedQuery": {
"sha256Hash": "0828119ded1c13477966434e15800ff57ddacf13ba1911c129dc2200705b0712",
"version": 1,
},
}
# The confusing fixture
#pytest.fixture
def assert_live(self, mock):
yield
assert mock.last_request.json().get("variables") == { # type: ignore[union-attr]
"isLive": True,
"isVod": False,
"login": "channelname",
"vodID": "",
"playerType": "embed",
}
# The confusing test
# note that assert_live is passed as an arg but is not actually used.
#pytest.mark.parametrize("plugin,mock", [
(
[("api-header", [("Authorization", "OAuth invalid-token")])],
{
"status_code": 401,
"json": {"error": "Unauthorized", "status": 401, "message": "The \"Authorization\" token is invalid."},
},
),
], indirect=True)
def test_auth_failure(self, caplog: pytest.LogCaptureFixture, plugin: Twitch, mock: requests_mock.Mocker, assert_live):
with pytest.raises(NoStreamsError) as cm:
plugin._access_token(True, "channelname")
assert str(cm.value) == "No streams found on this URL: https://twitch.tv/channelname"
assert mock.last_request._request.headers["Authorization"] == "OAuth invalid-token" # type: ignore[union-attr]
assert [(record.levelname, record.module, record.message) for record in caplog.records] == [
("error", "twitch", "Unauthorized: The \"Authorization\" token is invalid."),
]

Related

groovy.lang.MissingMethodException: No signature of method: httpRequest() is applicable for argument types: (java.util.LinkedHashMap)

I have the following piece of code that used to work great in a pipeline. I have to move it into a shared library in Jenkins, so created a class for it and made the necessary adjustments.
def toJson (input) {
return groovy.json.JsonOutput.toJson(input)
}
def void callAPI (args) {
def apiRequestBody = [
"prop1": args.param1,
"prop2": args.param2
]
// Build the request - notice that authentication should happen seamlessly by using Jenkins Credentials
response = httpRequest (authentication: "${CREDENTIALS_STORED_IN_JENKINS}",
consoleLogResponseBody: true,
httpMode: 'POST',
requestBody: toJson(apiRequestBody),
url: "${API_URL}",
customHeaders: [
[
name: 'Content-Type',
value: 'application/json; charset=utf-8'
],
[
name: 'Accept-Charset',
value: 'utf-8'
]
]
)
When I call the callAPI (args) method, I get the following error:
Exception groovy.lang.MissingMethodException: No signature of method: MY_PACKAGE_PATH.MY_CLASS.httpRequest() is applicable for argument types: (java.util.LinkedHashMap) values: [[authentication:MYAPI_UID_PW, consoleLogResponseBody:true, ...]]
What am I missing?
Thanks
httpRequest is a DSL command that's not directly available within the context of a class, much like ou can't use sh, bat, or node.
See https://www.jenkins.io/doc/book/pipeline/shared-libraries/#accessing-steps for more info about this.
You can lift code from within a Jenkinsfile and place it in a "var" (or Global Variable) instead, though.
If you insist on placing the code in a shared library class, refer to the link above, which would transform your code into (notice the "script" parameter and the script.httpRequest syntax):
def void callAPI (script, args) {
def apiRequestBody = [
"prop1": args.param1,
"prop2": args.param2
]
// Build the request
response = script.httpRequest (authentication: "${CREDENTIALS_STORED_IN_JENKINS}",
// ...
}

Falcon application can not render Openapi(swagger) specification

Hello everyone we are running a Falcon application that uses the falcon-apispec library to generate OpenAPI specifications.
Here is our code that initializes the definition:
import falcon
from apispec import APISpec
from falcon_apispec import FalconPlugin
from kubernetes import config
from api.admission_response import AdmissionResponse
from api.health import Health
from api.k8s_config_validator import K8sConfigValidator
from api.middleware.json import RequireJSON, JSONTranslator
from api.apidocs import ApiDocs
def create_app(config_validator):
api = falcon.API(middleware=[
RequireJSON(),
JSONTranslator(),
])
resources ={
'/': AdmissionResponse(config_validator),
'/api-docs': ApiDocs(),
'/health': Health()
}
for r in resources:
api.add_route(r, resources[r])
setup_swagger_documentation(api, resources)
# initialize k8s client
config.load_incluster_config()
return api
def get_app():
return create_app(K8sConfigValidator())
def setup_swagger_documentation(api, resources):
spec = APISpec(
title='Admission Controller API',
version='latest',
openapi_version='2.0',
plugins=[
FalconPlugin(api)
],
info=dict(description="Admission Controller API"),
)
for r in resources:
spec.path(resource=resources[r])
with open('./api/config/openapi/openapi_spec.yaml', 'w') as f:
f.write(spec.to_yaml())
Here is our openapi-spec defintion defined:
openapi: 3.0.0
info:
description: Admission Controller API
title: Admission Controller API
version: latest
paths:
/:
post:
tags:
- "API"
parameters:
- in: "query"
name: "body"
description: "List of user object"
required: true
schema:
type: string
responses:
"200":
description: "Success"
/api-docs:
get:
tags:
- "API Doc Endpoints"
responses:
"200":
description: "Success"
/health:
get:
tags:
- "Health Endpoints"
responses:
"200":
description: "Success"
And here is one of the classes that defines what should be done on a post:
class AdmissionResponse(object):
def __init__(self, k8s_config_validator):
self.k8s_config_validator = k8s_config_validator
#falcon.before(validate_schema)
def on_post(self, req, resp):
"""
---
tags: ['API']
parameters:
- in: "query"
name: "body"
description: "List of user object"
required: true
type: string
responses:
"200":
description: "Success"
"""
admission_review = AdmissionReview(req.context['doc'])
errors = self.k8s_config_validator.validate(admission_review)
if errors:
resp.context['result'] = ResponseBuilder(admission_review).not_allowed(errors)
api.logger.info("Validations for %s of kind %s in %s failed with %s", admission_review.name(), admission_review.kind(), admission_review.namespace(), errors)
else:
resp.context['result'] = ResponseBuilder(admission_review).allowed()
api.logger.info("Validations for %s of kind %s in %s passed", admission_review.name(), admission_review.kind(), admission_review.namespace())
Whenever we try to hit our hosted swagger-ui we run into this error:
Unable to render this definition The provided definition does not specify a valid version field.
Please indicate a valid Swagger or OpenAPI version field. Supported version fields are swagger: "2.0" and those that match openapi: 3.0.n (for example, openapi: 3.0.0).
Does anyone know how we could resolve this? When we paste in our openapi specification to the swagger editor located here: https://editor.swagger.io/ it works just fine. Any help would be awesome!
When calling APISpec, not all strings are supported.
Try something like "0.0.1" (you supplied 'latest').
Also, using falcon_swagger_ui, an error may be reported if the openapi_version format is not correct (in such case, examples of valid formats are provided).
This works for me:
spec = APISpec(
title="My APP",
version="0.0.1",
openapi_version='3.0.0',
plugins=[FalconPlugin(api)]
)

Error on deploying Python app to AWS Lambda

I have built a Python-Tornado app and am trying to deploy it to AWS Lambda using zappa. But, I am getting an error Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.
My folder structure inside the root folder is :
├── amortization.py
├── config.py
├── dmi-amort-dev-1557138776.zip
├── main.py
├── requirements.txt
├── venv
│   ├── bin
│  
└── zappa_settings.json
zappa deploy dev gives me :
Calling deploy for stage dev..
Downloading and installing dependencies..
- pandas==0.24.2: Using locally cached manylinux wheel
- numpy==1.16.3: Using locally cached manylinux wheel
- sqlite==python3: Using precompiled lambda package
Packaging project as zip.
Uploading dmi-amort-dev-1557143681.zip (30.8MiB)..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32.3M/32.3M [00:19<00:00, 1.94MB/s]
Scheduling..
Scheduled dmi-amort-dev-zappa-keep-warm-handler.keep_warm_callback with expression rate(4 minutes)!
Uploading dmi-amort-dev-template-1557143718.json (1.5KiB)..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.56K/1.56K [00:00<00:00, 10.6KB/s]
Waiting for stack dmi-amort-dev to create (this can take a bit)..
75%|█████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 3/4 [00:09<00:04, 5.00s/res]
Deploying API Gateway..
Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.
zappa tail gives me
Traceback (most recent call last):
File "/var/task/handler.py", line 602, in lambda_handler
return LambdaHandler.lambda_handler(event, context)
File "/var/task/handler.py", line 245, in lambda_handler
handler = cls()
File "/var/task/handler.py", line 142, in __init__
wsgi_app_function = getattr(self.app_module, self.settings.APP_FUNCTION)
AttributeError: module 'main' has no attribute 'app'
zappa_settings.json:
{
"dev": {
"app_function": "main.app",
"aws_region": "ap-south-1",
"profile_name": "default",
"project_name": "dmi-amort",
"runtime": "python3.6",
"s3_bucket": "zappa-mekp457ye",
"manage_roles": false,
"role_name": "lambda-role",
}
}
main.py:
import tornado.web
from tornado.ioloop import IOLoop
from tornado.web import MissingArgumentError
from config import get_arguments
from amortization import get_amort_schedule
class MainHandler(tornado.web.RequestHandler):
def prepare(self):
"""Checking if all the required parameters are present."""
if self.request.method != 'POST':
self.write_error(status_code=405, message="Method not allowed")
return
self.parameters = dict()
for key in get_arguments():
try:
self.parameters[key] = self.get_argument(key)
except MissingArgumentError:
self.write_error(status_code=400,
message="Missing Argument(s)")
return
# checking if 'label' is provided
if 'label' in self.request.arguments.keys():
self.parameters['label'] = self.get_argument('label')
# Set up response dictionary.
self.response = dict()
def get(self, *args, **kwargs):
self.write_error(status_code=405, message="Method not allowed")
def post(self, *args, **kwargs):
"""Executes the main logic part."""
self.response = get_amort_schedule(self.parameters)
self.write_json()
def set_default_headers(self):
"""Sets content-type as 'application/json' for response as JSON."""
self.set_header('Content-Type', 'application/json')
def write_error(self, status_code, **kwargs):
"""Invokes when error occurs in processing the request."""
if 'message' not in kwargs:
if status_code == 405:
kwargs['message'] = 'Invalid HTTP method.'
else:
kwargs['message'] = 'Unknown error.'
kwargs["error"] = True
self.set_status(status_code=status_code)
self.response = dict(kwargs)
self.write_json()
def write_json(self):
"""Responsible for writing the response."""
if "status" in self.response:
self.set_status(self.response.get("status"))
self.set_default_headers()
self.write(self.response)
self.finish()
def main():
app = tornado.web.Application([
(r'/', MainHandler),
], debug=True)
# server = HTTPServer(app)
# server.bind(8888)
# server.start(0)
app.listen()
# app.run(host='0.0.0.0')
IOLoop.current().start()
if __name__ == '__main__':
main()
What is the mistake here and how can I fix it?
It looks like the deployment is succeeding, but when Zappa checks to see if the code is working, the return code is 502, which suggests that the lambda function is failing to run in the lambda environment.
Taking a look at the logs, the critical line is:
AttributeError: module 'main' has no attribute 'app'
And this is true, if we look at your code, at no point do you expose an attribute called app in main.py.
I'm not experienced with Tornado, but I suspect that if you move the declaration of app out of the main() function and into the root scope, then the handler should succeed.
For example:
# rest of the file...
self.finish()
app = tornado.web.Application([
(r'/', MainHandler),
], debug=True)
app.listen()
def main():
IOLoop.current().start()
if __name__ == '__main__':
main()

Flask TypeError: argument of type 'NoneType' is not iterable

I am not sure why I am getting this TypeError:
File "C:/Users/PycharmProjects/REST/app.py", line 30, in
valid_book_object
if ("isbn" in book and "name" in book and "price" in book):
TypeError: argument of type 'NoneType' is not iterable
127.0.0.1 - - [12/Nov/2018 14:22:29] "POST /books HTTP/1.1" 500 -
Code:
from flask import Flask, jsonify, request
from test import *
app=Flask(__name__)
books=[
{'name': 'M',
'price': 6.75,
'isbn':123
},
{'name': 'G',
'price': 7.75,
'isbn':456
},
]
#GET /store
#app.route('/books') #first endpoint
def get_books():
return jsonify({'books': books})
# POST /books
#{'name': 'F',
#'price': 7.00,
#'isbn': 789
#},
def valid_book_object(book):
if ("isbn" in book and "name" in book and "price" in book):
return True
print("true")
else:
return False
print("false")
#app.route('/books', methods=['POST'])
def add_book():
#return jsonify(request.get_json())
request_data=request.get_json()
if(valid_book_object(request_data)):
books.insert(0, request_data)
return "True"
else:
return "False"
#GET /books/456
#app.route('/books/<int:isbn>') #second endpoint
def get_book_by_isbn(isbn):
return_value={}
for book in books:
if book["isbn"]==isbn:
return_value={
'name': book["name"],
'price':book["price"]
}
return jsonify(return_value)
app.run(port=5000)
You are not sending JSON data to /books route using POST method.
I tried your solution with postman and it worked. Also, if you want to use some route for GET and POST, don't split them. Use methods=['GET', 'POST']. Here is your code reformatted with PEP 8 standard:
from flask import Flask, jsonify, request
app = Flask(__name__)
books = [
{'name': 'M',
'price': 6.75,
'isbn': 123
},
{'name': 'G',
'price': 7.75,
'isbn': 456
}
]
# POST /books
# {
# "name": "F",
# "price": 7.00,
# "isbn": 789
# }
def valid_book_object(book):
if "isbn" in book and "name" in book and "price" in book:
return True
else:
return False
#app.route('/books', methods=['GET', 'POST'])
def add_book():
# If request is GET, just return JSON data of books.
if request.method == 'GET':
return jsonify({'books': books})
else:
# This is part if it is POST request
request_data = request.get_json()
if valid_book_object(request_data):
books.insert(0, request_data)
return "True"
else:
return "False"
# GET /books/456
#app.route('/books/<int:isbn>') # second endpoint
def get_book_by_isbn(isbn):
return_value = {}
for book in books:
if book["isbn"] == isbn:
return_value = {
'name': book["name"],
'price': book["price"]
}
return jsonify(return_value)
return 'No book with {} isbn value'.format(isbn)
if __name__ == '__main__':
app.run(port=5000)
And here is the output from postman (you can see True at the bottom, that is what you return if POST was successful):
If you use postman, be sure to select application/json content-type.
If you are using JQuery ajax method, please read this answer. But anyway, here is using JQuery (tested):
data = JSON.stringify({
name: "F",
price: 7.00,
isbn: 789
});
$.ajax({
url: '/books',
type: "POST",
data: data,
contentType: "application/json",
dataType: "json",
success: function(){
console.log('Post executed successfully');
}
})

Create Stack Instances Parameter Issue

I'm creating stack instance, using python boto3 SDK. According to the documentation I should be able to use ParameterOverrides but I'm getting following error..
botocore.exceptions.ParamValidationError: Parameter validation failed:
Unknown parameter in input: "ParameterOverrides", must be one of: StackSetName, Accounts, Regions, OperationPreferences, OperationId
Environment :
aws-cli/1.11.172 Python/2.7.14 botocore/1.7.30
imports used
import boto3
import botocore
Following is the code
try:
stackset_instance_response = stackset_client.create_stack_instances(
StackSetName=cloudtrail_stackset_name,
Accounts=[
account_id
],
Regions=[
stack_region
],
OperationPreferences={
'RegionOrder': [
stack_region
],
'FailureToleranceCount': 0,
'MaxConcurrentCount': 1
},
ParameterOverrides=[
{
'ParameterKey': 'CloudtrailBucket',
'ParameterValue': 'test-bucket'
},
{
'ParameterKey': 'Environment',
'ParameterValue': 'SANDBOX'
},
{
'ParameterKey': 'IsCloudTrailEnabled',
'ParameterValue': 'NO'
}
]
)
print("Stackset create Response : " + str(stackset_instance_response))
operation_id = stackset_instance_response['OperationId']
print (operation_id)
except botocore.exceptions.ClientError as e:
print("Stackset creation error : " + str(e))
I'm not sure where I'm doing wrong, any help would be greatly appreciated.
Thank you.
1.8.0 is the first version of Botocore that has parameteroverrides defined.
https://github.com/boto/botocore/blob/1.8.0/botocore/data/cloudformation/2010-05-15/service-2.json#L1087-L1090
1.7.30 doesn't have that defined. https://github.com/boto/botocore/blob/1.7.30/botocore/data/cloudformation/2010-05-15/service-2.json

Resources