Python 3 exception convertion and re-raise - python-3.x

I have the following problem:
my application is a bridge between dbus and json-rpc.
I have a dbus library that in case of Dbus exception is raising a
DBusException("my err msg") exception.
My json-rpc library is catching all the Exception and convert it to a nice generic server error message
{"jsonrpc": "2.0", "error": {"code": -32000, "message": "Server error"}, "id": 2}
The problem is that i want to handle better those exceptions and I can distinguish between them only using the error message:
for example I need to convert the
DBusException("Invalid Parameters")
to
{"jsonrpc": "2.0", "error": {"code": -32001, "message": "Server error", data="Invalid Parameters" }, "id": 2}
that can be done raising in my library an ApiError(msg, code, data) exception.
So summarizing:
- I need to catch a DBusException
- based on the err_msg I need to switch between different exception types and re-raise a modified
ApiError exception
How is the better approach to that? this needs to be done in at least 10 functions that have the same exceptions.
1) using a decorator?
def my_exception_catcher(fun, *args, **kwargv):
try:
fun(args, *argv)
except DBusException as e
err_msg = str(e)
if err_msg == "Invalid Arguments":
raise ApiError("Server Error", code=1, data=err_msg)
else if err_msg == "Connect Error":
raise ApiError("Server Error", code=2, data=err_msg)
else:
raise
#my_exception_catcher
my_fun(*args):
do_something
2) using a function to determine the exception type?
def find_my_dbus_error(err_msg):
if err_msg == "Invalid Arguments":
return ApiError("Server Error", code=1, data=err_msg)
else if err_msg == "Connect Error":
return ApiError("Server Error", code=2, data=err_msg)
else:
return Exception(err_msg)
try:
my_fun(params)
except DBusException as e
raise find_my_dbus_error(err_msg)
3) something else?
Thanks
Nick

Related

GEE python:“ee.batch.Export.image.toAsset” "Request had insufficient authentication scopes."

I am getting an error when using python GEE api to save processed image in Colab. The code and the error are as follow:
# Load a landsat image and select three bands.
landsat = ee.Image('LANDSAT/LC08/C01/T1_TOA/LC08_123032_20140515')\
.select(['B4', 'B3', 'B2'])
# Create a geometry representing an export region.
geometry = ee.Geometry.Rectangle([116.2621, 39.8412, 116.4849, 40.01236])
center = geometry.centroid().getInfo()['coordinates']
center.reverse()
Mapdisplay(center,{'landsat':landsat.getMapId()},zoom_start=7)
# Get band 4 from the Landsat image, copy it.
band4 = landsat.select('B4').rename('b4_mean')\
.addBands(landsat.select('B4').rename('b4_sample'))\
.addBands(landsat.select('B4').rename('b4_max'))\
# Export the image to an Earth Engine asset.
task = ee.batch.Export.image.toAsset(**{
'image': band4,
'description': 'imageToAssetExample',
'assetId': 'users/csaybar/exampleExport',
'scale': 100,
'region': geometry.getInfo()['coordinates']
})
task.start()
This error occurs due to "task.start()"
WARNING:googleapiclient.http:Invalid JSON content from response: b'{\n "error": {\n "code": 403,\n "message": "Request had insufficient authentication scopes.",\n "status": "PERMISSION_DENIED",\n "details": [\n {\n "#type": "type.googleapis.com/google.rpc.ErrorInfo",\n "reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT",\n "domain": "googleapis.com",\n "metadata": {\n "method": "google.earthengine.v1alpha.EarthEngine.ExportImage",\n "service": "earthengine.googleapis.com"\n }\n }\n ]\n }\n}\n'
---------------------------------------------------------------------------
HttpError Traceback (most recent call last)
/usr/local/lib/python3.8/dist-packages/ee/data.py in _execute_cloud_call(call, num_retries)
327 try:
--> 328 return call.execute(num_retries=num_retries)
329 except googleapiclient.errors.HttpError as e:
6 frames
HttpError: <HttpError 403 when requesting https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/image:export?alt=json returned "Request had insufficient authentication scopes.". Details: "[{'#type': 'type.googleapis.com/google.rpc.ErrorInfo', 'reason': 'ACCESS_TOKEN_SCOPE_INSUFFICIENT', 'domain': 'googleapis.com', 'metadata': {'method': 'google.earthengine.v1alpha.EarthEngine.ExportImage', 'service': 'earthengine.googleapis.com'}}]">
During handling of the above exception, another exception occurred:
EEException Traceback (most recent call last)
/usr/local/lib/python3.8/dist-packages/ee/data.py in _execute_cloud_call(call, num_retries)
328 return call.execute(num_retries=num_retries)
329 except googleapiclient.errors.HttpError as e:
--> 330 raise _translate_cloud_exception(e)
331
332
EEException: Request had insufficient authentication scopes.
I tried to look for a GEE service in the google cloud console to enable it, but couldn't find one. And I want to konw how to fix this problem.
The most recent version of the google earth engine API seems to have introduced some issues with gcloud. Try rolling back to an earlier version using conda install -c conda-forge earthengine-api=0.1.320 and then rerunning your process.
For future reference, most GEE-related questions get asked and answered over at gis.stackexchange.com.

Why is #pytest.fixture passed as an argument to tests but not used

Below is a test file for the Streamlink library. I was reading the test files to understand how testing is done and found a confusing set up for the tests. The test set up a series of #pytest.fixtures to help with testing. The assert_live fixture below is defined and passed into some tests, but never called at all. I dont think this is a mistake because a similar pattern is found throughout the file. I read the docs for fixtures but couldn't find anything that mentions this functionality. I got excited when they talked about using yeild inside a fixture but that turned not to touch on the subject either. Google was even less helpful. I am at a loss, but I am very interested in contributing to this library and need to understand how it works and how they test. Test is at the bottom of the code below. Thank you in advance!
# Other fixtures for context/continuity
#pytest.fixture
def mocker(self):
# The built-in requests_mock fixture is bad when trying to reference the following constants or classes
with requests_mock.Mocker() as mocker:
mocker.register_uri(requests_mock.ANY, requests_mock.ANY, exc=requests_mock.exceptions.InvalidRequest)
yield mocker
#pytest.fixture
def mock(self, request, mocker: requests_mock.Mocker):
mock = mocker.post("https://gql.twitch.tv/gql", **getattr(request, "param", {"json": {}}))
yield mock
assert mock.called_once
payload = mock.last_request.json() # type: ignore[union-attr]
assert tuple(sorted(payload.keys())) == ("extensions", "operationName", "variables")
assert payload.get("operationName") == "PlaybackAccessToken"
assert payload.get("extensions") == {
"persistedQuery": {
"sha256Hash": "0828119ded1c13477966434e15800ff57ddacf13ba1911c129dc2200705b0712",
"version": 1,
},
}
# The confusing fixture
#pytest.fixture
def assert_live(self, mock):
yield
assert mock.last_request.json().get("variables") == { # type: ignore[union-attr]
"isLive": True,
"isVod": False,
"login": "channelname",
"vodID": "",
"playerType": "embed",
}
# The confusing test
# note that assert_live is passed as an arg but is not actually used.
#pytest.mark.parametrize("plugin,mock", [
(
[("api-header", [("Authorization", "OAuth invalid-token")])],
{
"status_code": 401,
"json": {"error": "Unauthorized", "status": 401, "message": "The \"Authorization\" token is invalid."},
},
),
], indirect=True)
def test_auth_failure(self, caplog: pytest.LogCaptureFixture, plugin: Twitch, mock: requests_mock.Mocker, assert_live):
with pytest.raises(NoStreamsError) as cm:
plugin._access_token(True, "channelname")
assert str(cm.value) == "No streams found on this URL: https://twitch.tv/channelname"
assert mock.last_request._request.headers["Authorization"] == "OAuth invalid-token" # type: ignore[union-attr]
assert [(record.levelname, record.module, record.message) for record in caplog.records] == [
("error", "twitch", "Unauthorized: The \"Authorization\" token is invalid."),
]

How to escape ":" in Logic Apps Search Query?

I want to get only new emails not reply or forwarded emails.
Here is my query:
(subject:return NOT "re:" OR subject:shipment NOT "re:") AND (attachment: return NOT exception OR attachment:shipment NOT exception) AND (subject:return NOT adjustment) AND (subject:return NOT "fw:" OR subject:shipment NOT "fw:")
But I get these error instead. These query runs fine in MS Outlook application.
{
"status": 400,
"message": "Syntax error: character ':' is not valid at position 24 in '\"(subject:return NOT \"re:\" OR subject:shipment NOT \"re:\") AND (attachment: return NOT exception OR attachment:shipment NOT exception) AND (subject:return NOT adjustment) AND (subject:return NOT \"fw:\" OR subject:shipment NOT \"fw:\")\"'.\r\nclientRequestId: c6f3d1e1-b137-4c5d-acfd-1434ff85a610\r\nserviceRequestId: 2925969d-f4df-4a84-90b1-b9977f08a024",
"error": {
"message": "Syntax error: character ':' is not valid at position 24 in '\"(subject:return NOT \"re:\" OR subject:shipment NOT \"re:\") AND (attachment: return NOT exception OR attachment:shipment NOT exception) AND (subject:return NOT adjustment) AND (subject:return NOT \"fw:\" OR subject:shipment NOT \"fw:\")\"'."
},
"source": "office365-wus2.azconn-wus2.p.azurewebsites.net"
}
You need to use the "/" escape character as explained here
| Escape | Replace: ‘"’, ‘\”‘ |

Not able to upload file using slack api files.upload

This question may seem duplicate but I have tried a lot but did not get success.
I am trying to upload html file using https://slack.com/api/files.upload API but I am getting below error always.
response
{'ok': False, 'error': 'no_file_data'}
I went through documentation [a link]https://api.slack.com/methods/files.upload and tried with different options but still i am getting the same response {'ok': False, 'error': 'no_file_data'}
Also i have seen many similar questions in stack overflow but none of them resolved the problem.
[a link]no_file_data error when using Slack API upload
[a link]How to upload files to slack using file.upload and requests
Below is my code.
import requests
def post_reports_to_slack(html_report):
"""
"""
url = "https://slack.com/api/files.upload"
# my_file = {html_report, open(html_report, 'rb'), 'html'}
data = {
"token": bot_user_token,
"channels": channel_name,
"file": html_report,
"filetype": "html"
}
# data = "token=" + bot_user_token + \
# "&channels=" + channel_name +\
# "&file=" + html_report + "&filetype=" + "html"
response = requests.post(
url=url, data=data,
headers={"Content-Type": "application/x-www-form-urlencoded"})
print("response", response)
print(response.json())
if response.status_code == 200:
print("successfully completed post_reports_to_slack "
"and status code %s" % response.status_code)
else:
print("Failed to post report on slack channel "
"and status code %s" % response.status_code)
Please help to resolve the issue.
I was needed to add "content" argument and "filename" argument instead of "file" argument in files.upload API payload, Now file uploading to slack channel is working fine.
import requests
def post_reports_to_slack(html_report):
url = "https://slack.com/api/files.upload"
with open(html_report) as fh:
html_data = fh.read()
data = {
"token": bot_user_token,
"channels": "#channel_name",
"content": html_data,
"filename": "report.html",
"filetype": "html",
}
response = requests.post(
url=url, data=data,
headers={"Content-Type": "application/x-www-form-urlencoded"})
if response.status_code == 200:
print("successfully completed post_reports_to_slack "
"and status code %s" % response.status_code)
else:
print("Failed to post report on slack channel "
"and status code %s" % response.status_code)

How do I get an authorized user for PostText API call for a Lex bot runtime

Sorry for the long post. I am trying to call a Lex bot with the PostText runtime API with my lambda function. However when I test this call then it returns that the userID is not authorized to use this. This is the error message I receive:
Response:
{
"errorMessage": "An error occurred (AccessDeniedException) when calling the PostText operation: User: arn:aws:sts::981709171824:assumed-role/lambda_basic_execution/OrchestratorAPIApp is not authorized to perform: lex:PostText on resource: arn:aws:lex:us-east-1:981709171824:bot:SupportBot_BookCab:SupportBot_BookCab",
"errorType": "ClientError",
"stackTrace": [
[
"/var/task/lambda_function.py",
18,
"lambda_handler",
"inputText= userInput"
],
[
"/var/runtime/botocore/client.py",
314,
"_api_call",
"return self._make_api_call(operation_name, kwargs)"
],
[
"/var/runtime/botocore/client.py",
612,
"_make_api_call",
"raise error_class(parsed_response, operation_name)"
]
]
}
Request ID:
"677f1820-6ed2-11e8-b891-33ab1951c65f"
Function Logs:
START RequestId: 677f1820-6ed2-11e8-b891-33ab1951c65f Version: $LATEST
An error occurred (AccessDeniedException) when calling the PostText operation: User: arn:aws:sts::981709171824:assumed-role/lambda_basic_execution/OrchestratorAPIApp is not authorized to perform: lex:PostText on resource: arn:aws:lex:us-east-1:981709171824:bot:SupportBot_BookCab:SupportBot_BookCab: ClientError
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 18, in lambda_handler
inputText= userInput
File "/var/runtime/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/runtime/botocore/client.py", line 612, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the PostText operation: User: arn:aws:sts::981709171824:assumed-role/lambda_basic_execution/OrchestratorAPIApp is not authorized to perform: lex:PostText on resource: arn:aws:lex:us-east-1:981709171824:bot:SupportBot_BookCab:SupportBot_BookCab
END RequestId: 677f1820-6ed2-11e8-b891-33ab1951c65f
REPORT RequestId: 677f1820-6ed2-11e8-b891-33ab1951c65f Duration: 325.25 ms Billed Duration: 400 ms Memory Size: 128 MB Max Memory Used: 31 MB
This is my code to calling the API:
import boto3
def lambda_handler(event, context):
responderName = event["DestinationBot"]
userId = event["RecipientID"]
userInput = event["message"]["text"]
client = boto3.client('lex-runtime')
response = client.post_text(
botName=responderName,
botAlias=responderName,
userId=userId,
sessionAttributes={
},
requestAttributes={
},
inputText= userInput
)
This is my sample test input:
{
"DestinationBot": "SupportBot_BookCab",
"RecipientID": "12345",
"message": {
"text": "book me a cab"
}
}
The userID of PostText is the way you persist the conversation back and forth between the user and Lex. It can be anything that you can identify the user by in their incoming request that is consistent and unique to them, at least for that session.
From AWS PostText Docs:
userID
The ID of the client application user. Amazon Lex uses this to identify a user's conversation with your bot. At runtime, each request must contain the userID field.
...
Length Constraints: Minimum length of 2. Maximum length of 100.
Pattern: [0-9a-zA-Z._:-]+
So if a user is using Facebook messenger, they will have a Facebook ID that is passed with their messages and you can use that as their userID.
If a user is using Twilio-SMS, they will have a phone number passed with their messages and you can use that as their userID.
Your code is currently taking event["RecipientID"] and using that as a userID. But the RecipientID from an incoming message is yourself, the receiver of the incoming message.
Your error is telling you that
... User: arn:aws:sts::XXXXXXXXXX:assumed-role/lambda_basic_execution/OrchestratorAPIApp
So the userID = event["RecipientID"] = 'arn:aws:sts::XXXXXXXX:assumed-role/lambda_basic_execution/OrchestratorAPIApp'
You definitely don't want the Recipient ID to be used.
Instead you want the sender's ID to be the userID. Something like:
userId = event["SenderID"]
That might not be the exact code you use, its just an example. You should be able to view the incoming request, and locate something in there to use as a proper userID as I explained above with Facebook and Twilio.

Resources