I am trying to run some Python code in Azure, for the first time ever. I Googled 'how to run Python scripts in Azure' and came across the following link.
https://learn.microsoft.com/en-us/azure/automation/learn/automation-tutorial-runbook-textual-python-3
Essentially, I need to do this.
Open the textual editor by selecting Edit on the MyFirstRunbook-Python3 pane.
Add the following code to authenticate to Azure:
import os
from azure.mgmt.compute import ComputeManagementClient
import azure.mgmt.resource
import automationassets
def get_automation_runas_credential(runas_connection):
from OpenSSL import crypto
import binascii
from msrestazure import azure_active_directory
import adal
# Get the Azure Automation RunAs service principal certificate
cert = automationassets.get_automation_certificate("AzureRunAsCertificate")
pks12_cert = crypto.load_pkcs12(cert)
pem_pkey = crypto.dump_privatekey(crypto.FILETYPE_PEM,pks12_cert.get_privatekey())
# Get run as connection information for the Azure Automation service principal
application_id = runas_connection["ApplicationId"]
thumbprint = runas_connection["CertificateThumbprint"]
tenant_id = runas_connection["TenantId"]
# Authenticate with service principal certificate
resource ="https://management.core.windows.net/"
authority_url = ("https://login.microsoftonline.com/"+tenant_id)
context = adal.AuthenticationContext(authority_url)
return azure_active_directory.AdalAuthentication(
lambda: context.acquire_token_with_client_certificate(
resource,
application_id,
pem_pkey,
thumbprint)
)
# Authenticate to Azure using the Azure Automation RunAs service principal
runas_connection = automationassets.get_automation_connection("AzureRunAsConnection")
azure_credential = get_automation_runas_credential(runas_connection)
When I run that code, I get this error.
Failed
Traceback (most recent call last): File "C:\WPy64-3800\python-3.8.0.amd64\lib\automationassets.py", line 155, in _issue_request response = urllib.request.urlopen(request) File "C:\WPy64-3800\python-3.8.0.amd64\lib\urllib\request.py", line 222, in urlopen return opener.open(url, data, timeout) File "C:\WPy64-3800\python-3.8.0.amd64\lib\urllib\request.py", line 531, in open response = meth(req, response) File "C:\WPy64-3800\python-3.8.0.amd64\lib\urllib\request.py", line 640, in http_response response = self.parent.error( File "C:\WPy64-3800\python-3.8.0.amd64\lib\urllib\request.py", line 569, in error return self._call_chain(*args) File "C:\WPy64-3800\python-3.8.0.amd64\lib\urllib\request.py", line 502, in _call_chain result = func(*args) File "C:\WPy64-3800\python-3.8.0.amd64\lib\urllib\request.py", line 649, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp)urllib.error.HTTPError: HTTP Error 404: Not FoundDuring handling of the above exception, another exception occurred:Traceback (most recent call last): File "C:\Temp\nmcijfgi.bro\a761c289-67d9-493a-99b2-3ba7d46a7cd9", line 36, in <module> runas_connection = automationassets.get_automation_connection("AzureRunAsConnection") File "C:\WPy64-3800\python-3.8.0.amd64\lib\automationassets.py", line 232, in get_automation_connection connection = _client.get_connection_asset(name) File "C:\WPy64-3800\python-3.8.0.amd64\lib\automationassets.py", line 187, in get_connection_asset return self._issue_request(url, method=self._GET) File "C:\WPy64-3800\python-3.8.0.amd64\lib\automationassets.py", line 158, in _issue_request raise AutomationAssetNotFound()automationassets.AutomationAssetNotFound
Do I need 'cert', 'pks12_cert', or 'pem_pkey'? I don't think I have those. I'm not even sure what those things are. Again, I am trying to setup my Azure environment to run a Python script, which works totally fine on my desktop (Spyder).
Most likely you haven't created an Azure Run As Account. Follow the steps here to do so.
Related
I want to send an Email from my outlook account using Python. I used the below code running in an AWS EC2 instance to generate an email.
import smtplib
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
server = smtplib.SMTP('smtp.office365.com', 587)
server.starttls()
server.login("<outlook_mail_id>","<outlook_password>")
server.sendmail("<outlook_mail_id>", "<recipient_email_id> ", "Test Message")
server.quit()
But while running this I get the below error
Traceback (most recent call last):
File "smtp_office_365.py", line 12, in <module>
server.login("<outlook_email_id>","<outlook_pass>")
File "/usr/local/lib/python3.8/smtplib.py", line 734, in login
raise last_exception
File "/usr/local/lib/python3.8/smtplib.py", line 723, in login
(code, resp) = self.auth(
File "/usr/local/lib/python3.8/smtplib.py", line 646, in auth
raise SMTPAuthenticationError(code, resp)
smtplib.SMTPAuthenticationError: (535, b'5.7.3 Authentication unsuccessful [MN2PR15CA0001.namprd15.prod.outlook.com]')
I am able to login into web outlook account using these credentials. Hence to check further I connected with my Admin team and they said that Basic Authentication is disabled for this account and can't be enabled in any case. They suggested me to use modern Authentication method by using Microsoft Online Exchange.
On further exploration I found the below script to send the mail
from exchangelib import DELEGATE, IMPERSONATION, Account, Credentials, Configuration
credentials = Credentials('<outlook_mail_id>', '<outlook_password>')
account = Account('<outlook_mail_id>', credentials=credentials, autodiscover=True)
for item in account.inbox.all().order_by('-datetime_received')[:100]:
print(item.subject, item.sender, item.datetime_received)
But I get the below error while running the script
Traceback (most recent call last):
File "test_exchange.py", line 5, in <module>
account = Account('<outlook_email_id>', credentials=credentials, autodiscover=True)
File "/home/ec2-user/.local/lib/python3.8/site-packages/exchangelib/account.py", line 116, in __init__
self.ad_response, self.protocol = discover(
File "/home/ec2-user/.local/lib/python3.8/site-packages/exchangelib/autodiscover/discovery.py", line 24, in discover
return Autodiscovery(
File "/home/ec2-user/.local/lib/python3.8/site-packages/exchangelib/autodiscover/discovery.py", line 123, in discover
ad_response = self._step_1(hostname=domain)
File "/home/ec2-user/.local/lib/python3.8/site-packages/exchangelib/autodiscover/discovery.py", line 433, in _step_1
return self._step_2(hostname=hostname)
File "/home/ec2-user/.local/lib/python3.8/site-packages/exchangelib/autodiscover/discovery.py", line 451, in _step_2
return self._step_3(hostname=hostname)
File "/home/ec2-user/.local/lib/python3.8/site-packages/exchangelib/autodiscover/discovery.py", line 483, in _step_3
return self._step_4(hostname=hostname)
File "/home/ec2-user/.local/lib/python3.8/site-packages/exchangelib/autodiscover/discovery.py", line 514, in _step_4
return self._step_6()
File "/home/ec2-user/.local/lib/python3.8/site-packages/exchangelib/autodiscover/discovery.py", line 572, in _step_6
raise AutoDiscoverFailed(
exchangelib.errors.AutoDiscoverFailed: All steps in the autodiscover protocol failed for email '<outlook_email_id>'. If you think this is an error, consider doing an official test at https://testconnectivity.microsoft.com
Please let me know if more details are required. Any help would be appreciated.
Thanks
I am trying to create a subscription to a topic using Google's pubsub_v1 library in python. I have successfully created a topic using the library (I can see it in the cloud console after creation). However I am having an issue trying to create a subscription. I tried to solution given in this question to no avail. Here is my subscription code:
from google.cloud import pubsub_v1 as pubsub
topic_name = 'logs'
sub_name = 'logs-consumer'
project_name = 'my-project' # valid project name
subscriber = pubsub.SubscriberClient()
topic_path = subscriber.topic_path(project_name, topic_name)
subscription_path = subscriber.subscription_path(project_name, sub_name)
# Wrap the subscriber in a 'with' block to automatically call close() to
# close the underlying gRPC channel when done.
with subscriber:
subscription = subscriber.create_subscription(
request={"name": subscription_path, "topic": topic_path}
)
Whenever I run this code, I get the following error:
Traceback (most recent call last):
File "/Users/zacharymcgrath/Library/Python/3.7/lib/python/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
return callable_(*args, **kwargs)
File "/Users/zacharymcgrath/Library/Python/3.7/lib/python/site-packages/grpc/_channel.py", line 826, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/Users/zacharymcgrath/Library/Python/3.7/lib/python/site-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "Project 'project:gcp-python-291817' not found or deleted."
debug_error_string = "{"created":"#1607133732.705528000","description":"Error received from peer ipv6:[2607:f8b0:400f:801::200a]:443","file":"src/core/lib/surface/call.cc","file_line":1062,"grpc_message":"Project 'project:gcp-python-291817' not found or deleted.","grpc_status":3}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "logger_consumer_GCP.py", line 28, in <module>
request={"name": subscription_path, "topic": topic_path}
File "/Users/zacharymcgrath/Library/Python/3.7/lib/python/site-packages/google/cloud/pubsub_v1/_gapic.py", line 40, in <lambda>
fx = lambda self, *a, **kw: wrapped_fx(self.api, *a, **kw) # noqa
File "/Users/zacharymcgrath/Library/Python/3.7/lib/python/site-packages/google/pubsub_v1/services/subscriber/client.py", line 526, in create_subscription
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
File "/Users/zacharymcgrath/Library/Python/3.7/lib/python/site-packages/google/api_core/gapic_v1/method.py", line 145, in __call__
return wrapped_func(*args, **kwargs)
File "/Users/zacharymcgrath/Library/Python/3.7/lib/python/site-packages/google/api_core/retry.py", line 286, in retry_wrapped_func
on_error=on_error,
File "/Users/zacharymcgrath/Library/Python/3.7/lib/python/site-packages/google/api_core/retry.py", line 184, in retry_target
return target()
File "/Users/zacharymcgrath/Library/Python/3.7/lib/python/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.InvalidArgument: 400 Project 'project:gcp-python-291817' not found or deleted.
I thought it might be that my project gcloud variable had somehow changed and the library uses environment variables, but I double checked and it was correct. I'm not really sure what I'm doing thats different than the question referred to above. Thanks.
Update
Some helpful information from the comments:
gcp-python-291817 is not the name of the project
the project name is in a config file that both the publisher and the subscriber read from. The publisher did not have any problems when reading the project name from the file and publishing a message
I had a ssh configuration for a VM instance within this project named gcp-python, but was removed a while ago
clearing the gcloud cache and gsutils cache has not fixed the problem either
I found the issue. I re-ran
gcloud auth application-default login
and made sure that GOOGLE_APPLICATION_CREDENTIALS was pointed to the new credentials json file and it worked. I must have messed up the credentials file at some point. Thanks for the help!
I want to search in contact list of some account and get that contacts details back, but none of folders that told here not work because of KeyError exception .
Somehow i can't access to any of exchange account folders.
Is it permission or ... ?
Code :
from exchangelib import Credentials, Account, Configuration
from exchangelib.protocol import NoVerifyHTTPAdapter, BaseProtocol
BaseProtocol.HTTP_ADAPTER_CLS = NoVerifyHTTPAdapter
credentials = Credentials("YYY#XXX.com", 'PASSWORD')
account = Account(
primary_smtp_address="Account#XXX.com",
autodiscover=True,
credentials=credentials
)
print(account) # work properly with printing my account
print(account.contacts) # not work with KeyError Exception
Error :
Warning (from warnings module):
File "C:\Python\lib\site-packages\urllib3\connectionpool.py", line 857
InsecureRequestWarning)
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
Traceback (most recent call last):
File "C:\Python\lib\site-packages\cached_property.py", line 69, in __get__
return obj_dict[name]
KeyError: 'contacts'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\dvp7\Desktop\ex.py", line 20, in <module>
print(account.contacts)
File "C:\Python\lib\site-packages\cached_property.py", line 73, in __get__
return obj_dict.setdefault(name, self.func(obj))
File "C:\Python\lib\site-packages\exchangelib\account.py", line 169, in contacts
return self.root.get_default_folder(Contacts)
File "C:\Python\lib\site-packages\exchangelib\folders.py", line 965, in get_default_folder
for f in self._folders_map.values():
File "C:\Python\lib\site-packages\exchangelib\folders.py", line 928, in _folders_map
for f in FolderCollection(account=self.account, folders=distinguished_folders).get_folders():
File "C:\Python\lib\site-packages\exchangelib\services.py", line 1053, in call
shape=shape,
File "C:\Python\lib\site-packages\exchangelib\services.py", line 88, in _get_elements
response = self._get_response_xml(payload=payload)
File "C:\Python\lib\site-packages\exchangelib\services.py", line 189, in _get_response_xml
raise rme
File "C:\Python\lib\site-packages\exchangelib\services.py", line 171, in _get_response_xml
res = self._get_soap_payload(soap_response=soap_response_payload)
File "C:\Python\lib\site-packages\exchangelib\services.py", line 227, in _get_soap_payload
cls._raise_soap_errors(fault=fault) # Will throw SOAPError or custom EWS error
File "C:\Python\lib\site-packages\exchangelib\services.py", line 261, in _raise_soap_errors
raise vars(errors)[code](msg)
exchangelib.errors.ErrorInternalServerError: An internal server error occurred. The operation failed.
Build version :
Build=15.0.847.31, API=Exchange2013_SP1, Fullname=Microsoft Exchange Server 2013 SP1
This method is work :
account.root.walk() # output : <exchangelib.folders.FolderCollection object at 0x03ADCA90>
But when i append filter to it above error occurred.
KeyError: 'folders'
only root folder is works fine and there is nothing in it !
print(account.root.all()) # QuerySet(q=Q(), folders=[Root (root)])
For the record, this turned out to be a misbehaving archive inbox folder. Workaround provided in https://github.com/ecederstrand/exchangelib/issues/431#issuecomment-409832287 by telling exchangelib to ignore this folder:
from exchangelib.folders import ArchiveInbox
from exchangelib.version import EXCHANGE_2016
# Set to something newer than your current version
ArchiveInbox.supported_from = EXCHANGE_2016
For those who find this issue by searching for:
exchangelib KeyError: 'inbox'
the fix for us was to upgrade past 1.11.4. The was necessary after 9:15 am PST today
I have a simple example of a Python (3.6) Tornado (4.5.2) server and I am attempting to add ssl certs for testing. I have determined it is finding the key and csr files. Here is what my code looks like with a stack trace following detailing the error. Has anyone run into this or solved it?
import tornado.httpserver
import tornado.ioloop
import tornado.web
class indexHandler(tornado.web.RequestHandler):
def get(self):
self.write("hello")
application = tornado.web.Application([
(r'/', indexHandler),
])
if __name__ == '__main__':
http_server = tornado.httpserver.HTTPServer(application, ssl_options={
"certfile": "cert/ig.csr",
"keyfile": "cert/ig.key",
})
http_server.listen(443)
tornado.ioloop.IOLoop.instance().start()
Running on Python 3.6.4 and the server runs but when the page is accessed as https://localhost, it throws the following exception. What am I missing?
ERROR:asyncio:Exception in callback BaseAsyncIOLoop._handle_events(5, 1)
handle: <Handle BaseAsyncIOLoop._handle_events(5, 1)>
Traceback (most recent call last):
File "/<python path>/asyncio/events.py", line 145, in _run
self._callback(*self._args)
File "/<python path>/site-packages/tornado/platform/asyncio.py", line 102, in _handle_events
handler_func(fileobj, events)
File "/<python path>/site-packages/tornado/stack_context.py", line 276, in null_wrapper
return fn(*args, **kwargs)
File "/<python path>/site-packages/tornado/netutil.py", line 252, in accept_handler
callback(connection, address)
File "/<python path>/site-packages/tornado/tcpserver.py", line 264, in _handle_connection
do_handshake_on_connect=False)
File "/<python path>/site-packages/tornado/netutil.py", line 551, in ssl_wrap_socket
context = ssl_options_to_context(ssl_options)
File "/<python path>/site-packages/tornado/netutil.py", line 526, in ssl_options_to_context
context.load_cert_chain(ssl_options['certfile'], ssl_options.get('keyfile', None))
ssl.SSLError: [SSL] PEM lib (_ssl.c:3337)
In above error message, /<python path>/ is equal to:
"/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/"
Its because the signature of the you certificate and the key doesn't match.
OK - I found it!! There are several online resources for determining if your certificate file and key match. I used THIS and they did not match. A quick call to Comodo (cert was thru Namecheap and then thru them) and they fixed it.
Lesson: Validate the key and certificate first!
I'm trying to implement an oauth2 client in Python3 so that I can upload files to github. For a very basic start I'm trying to get a list of authorizations using the API.
This code works:
from subprocess import Popen,PIPE
user = 'MYUSERNAME'
pw = 'MYPASSWORD'
git_url = "https://api.github.com/authorizations"
res = Popen(['curl','--user',user + ':' + pw,git_url],stdout=PIPE,stderr=PIPE).communicate()[0]
print(res)
This code does not work:
user = 'MYUSERNAME'
pw = 'MYPASSWORD'
git_url = "https://api.github.com/authorizations"
import urllib.request
# Create an OpenerDirector with support for Basic HTTP Authentication...
auth_handler = urllib.request.HTTPBasicAuthHandler()
auth_handler.add_password(realm=None,
uri=git_url,
user=user,
passwd=pw)
opener = urllib.request.build_opener(auth_handler)
f = opener.open(git_url)
print(f.read())
In fact, it generates this error:
Traceback (most recent call last):
File "demo.py", line 18, in <module>
f = opener.open("https://api.github.com/authorizations")
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/urllib/request.py", line 375, in open
response = meth(req, response)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/urllib/request.py", line 487, in http_response
'http', request, response, code, msg, hdrs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/urllib/request.py", line 413, in error
return self._call_chain(*args)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/urllib/request.py", line 347, in _call_chain
result = func(*args)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/urllib/request.py", line 495, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
I know that there is an existing Oauth2 implementation in python, but it's python2, not python3, and it does a lot more than I need.
I also know that I could just have my Python program call curl, and that's my fallback.
I'd really like to know what I'm doing wrong.
Thanks.
I have just posted an answer to another question with a full example using urllib2 from python2. Obviously you are interested in python3, but it shouldn't be to difficult to migrate the code.
Hope that helps,