Grabbing files from Microsoft Teams using Python - python-3.x

Teams seems to lack any native way of mirroring files to a shared directory. I'm Trying use Python (or another language but python preferred!) to either:
a. Directly pull from microsoft teams into memory using Python to process with Pandas
b. Copy files from teams into a shared network folder (which Python could then read in)
I found this but can't get it to work with teams - the teams URLs don't look anything like these do. How to read SharePoint Online (Office365) Excel files in Python with Work or School Account?
It seems close to what I want to do though. I also found "pymsteams" on PyPi repository. https://pypi.org/project/pymsteams/ which just seems to let you send messages to Teams and nothing else? unless I misunderstand something.
https://pypi.org/project/Office365-REST-Python-Client/
https://pypi.org/project/pymsteams/
from office365.runtime.auth.authentication_context
import AuthenticationContext
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.file import File
url = 'https://teams.microsoft.com/l/file'
username = 'myusername'
password = 'mypassword'
relative_url ='myurl'
ctx_auth = AuthenticationContext(url)
ctx_auth.acquire_token_for_user(username, password)
Trying to run the above code gives
AttributeError: 'NoneType' object has no attribute 'text'
Full stack trace:
runfile('H:/repos/foo/untitled0.py', wdir='H:/repos/foo')
Traceback (most recent call last):
File "<ipython-input-35-314ab7dc63c9>", line 1, in <module>
runfile('H:/repos/foo/untitled0.py', wdir='H:/foo/image_ai')
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
execfile(filename, namespace)
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "H:/repos/image_ai/untitled0.py", line 10, in <module>
ctx_auth.acquire_token_for_user(username, password)
File "C:\ProgramData\Anaconda3\lib\site-packages\office365\runtime\auth\authentication_context.py", line 18, in acquire_token_for_user
return self.provider.acquire_token()
File "C:\ProgramData\Anaconda3\lib\site-packages\office365\runtime\auth\saml_token_provider.py", line 57, in acquire_token
self.acquire_service_token(options)
File "C:\ProgramData\Anaconda3\lib\site-packages\office365\runtime\auth\saml_token_provider.py", line 88, in acquire_service_token
token = self.process_service_token_response(response)
File "C:\ProgramData\Anaconda3\lib\site-packages\office365\runtime\auth\saml_token_provider.py", line 119, in process_service_token_response
return token.text
AttributeError: 'NoneType' object has no attribute 'text'

I've managed to get it working using the Office-365-REST-Python-Client you linked.
If you're using SharePoint Online, you'll need to get an App-Only principle set up and connect using the acquire_token_for_app function instead of acquire_token_for_user, then passing a client_id and client_secret instead of username & password.
from office365.runtime.auth.authentication_context import AuthenticationContext
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.file import File
client_id = 'yourclientid'
client_secret = 'yourclientsecret'
url = 'https://yoursharepointsite.com/teams/yourteam'
relative_url = '/teams/yourteam/Shared%20Documents/yourteamschannel/yourdoc.extension'
ctx_auth = AuthenticationContext(url)
if ctx_auth.acquire_token_for_app(client_id, client_secret):
ctx = ClientContext(url, ctx_auth)
with open(filename, 'wb') as output_file:
response = File.open_binary(ctx, relative_url)
output_file.write(response.content)
else:
print(ctx_auth.get_last_error())
This should download your file to a local drive (specified via the filename variable) and you can then load into pandas etc to process

Related

Cannot use custom key words with pvporcupine

I have already created an account in picovoice and recieved an access key, but when I try to put the path of my ppn file, it shows an error:
`
[ERROR] loading keyword file at 'C:\Users\Priyam\Desktop\hey keyazip' failed with 'IO_ERROR'
Traceback (most recent call last):
File "e:\Personal Project\import struct.py", line 13, in <module>
porcupine = pvporcupine.create(access_key='access key',
File "C:\Users\Priyam\AppData\Roaming\Python\Python310\site-packages\pvporcupine\__init__.py", line 77, in create
return Porcupine(
File "C:\Users\Priyam\AppData\Roaming\Python\Python310\site-packages\pvporcupine\porcupine.py", line 158, in __init__
raise self._PICOVOICE_STATUS_TO_EXCEPTION[status]()
pvporcupine.porcupine.PorcupineIOError
the code is:
`
porcupine=None
paud=None
audio_stream=None
try:
access_key="access key"
porcupine = pvporcupine.create(access_key='access key',
keyword_paths=['C:\\Users\Priyam\Desktop\hey keyazip'],keywords=['hey keya']) #pvporcupine.KEYWORDS for all keywords
paud=pyaudio.PyAudio()
audio_stream=paud.open(rate=porcupine.sample_rate,channels=1,format=pyaudio.paInt16,input=True,frames_per_buffer=porcupine.frame_length)
while True:
keyword=audio_stream.read(porcupine.frame_length)
keyword=struct.unpack_from("h"*porcupine.frame_length,keyword)
keyword_index=porcupine.process(keyword)
if keyword_index>=0:
print("hotword detected")
finally:
if porcupine is not None:
porcupine.delete()
if audio_stream is not None:
audio_stream.close()
if paud is not None:
paud.terminate()
`
I tried the code above and the code provided by picovoice itself, yet I am facing the same issues
It looks like Porcupine is having trouble finding your keyword file: [ERROR] loading keyword file at 'C:\Users\Priyam\Desktop\hey keyazip' failed with 'IO_ERROR'.
The Picovoice console provides the keyword inside a compressed .zip file. You will need to decompress the file and in your code update the path to lead to the .ppn file located inside. For example: C:\\Users\Priyam\Desktop\hey-keya_en_windows_v2_1_0\hey-keya_en_windows_v2_1_0.ppn

Pandas - Suppress call to _validate_archive? - Python 3.9.13/Pandas 1.4.2/Openpyxl 3.0.10

I have been digging through questions/answers for the BadZipFile exception when calling read_excel() using the openpyxl engine. I looked at my error stack and dug into the Python files and it looks like ZipFile.py is being very strict when validating an archive. It is looking for an EOCD (end of central directory) signature in my XLSX archive file.
When unzipping an archive, if the EOCD cannot be found or validated, there is supposed to be an error thrown when calling unzip in Linux, but I am not seeing it. I am unsure whether the EOCD is there/correct or not (anyone know of a tool to check?).
However, from looking through my stack (below) I am examining what is happening in openpyxl/reader/excel.py. At line 67, the _validate_archive function is defined. I am wondering about the examination for a "file like object".
My use case is an AWS Lambda function which has an HTTP endpoint. I POST an Excel file (I am testing with Postman and using the binary body for the request where I select my Excel file) to the endpoint. The function needs to handle both CSV and XLSX. I include a custom header in which I specify the original file name. I split the filename, look at the extension, and either call read_csv or read_excel. read_csv is working great.
Either way, the file is coming in as base64. For an XLSX file, Pandas handles this OK - up until we get to _validate_archive... What I am unsure of is how the "file like object" check at line 76...
is_file_like = hasattr(filename, 'read')
... interacts with the type by which the Base64 is handled. I am trying straight string (event["body"]), the bytes() object, the BytesIO class, and the StringIO class, all to the same BadZipFile exception.
So... is it possible in Pandas/Openpyxl to suppress the validation of the archive? I want to be able to call read_excel() but not have the archive validated and see what happens.
My error stack:
"Error: (<class 'zipfile.BadZipFile'>, BadZipFile('File is not a zip file'),
<traceback object at 0x7f1019589dc0>)\r\n<class 'zipfile.BadZipFile'>\r\n
File is not a zip file\r\nTraceback (most recent call last):\n
File "/var/task/lambda_function.py", line 20, in lambda_handler\n inv = pd.read_excel( bufferedString, engine='openpyxl', index_col=0 )\n
File "/opt/python/pandas/util/_decorators.py", line 311, in wrapper\n return func(*args, **kwargs)\n
File "/opt/python/pandas/io/excel/_base.py", line 457, in read_excel\n io = ExcelFile(io, storage_options=storage_options, engine=engine)\n
File "/opt/python/pandas/io/excel/_base.py", line 1419, in init\n self._reader = self._engines[engine](self._io, storage_options=storage_options)\n
File "/opt/python/pandas/io/excel/_openpyxl.py", line 525, in init\n super().init(filepath_or_buffer, storage_options=storage_options)\n
File "/opt/python/pandas/io/excel/_base.py", line 518, in init\n self.book = self.load_workbook(self.handles.handle)\n
File "/opt/python/pandas/io/excel/_openpyxl.py", line 536, in load_workbook\n return load_workbook(\n
File "/opt/python/openpyxl/reader/excel.py", line 315, in load_workbook\n reader = ExcelReader(filename, read_only, keep_vba,\n
File "/opt/python/openpyxl/reader/excel.py", line 124, in init\n self.archive = _validate_archive(fn)\n
File "/opt/python/openpyxl/reader/excel.py", line 96, in _validate_archive\n archive = ZipFile(filename, 'r')\n
File "/var/lang/lib/python3.9/zipfile.py", line 1264, in init\n self._RealGetContents()\n
File "/var/lang/lib/python3.9/zipfile.py", line 1331, in _RealGetContents\n
raise BadZipFile("File is not a zip file")\n
zipfile.BadZipFile: File is not a zip file\n"

Snowflake connector.connect breaks with an error message that tells me nothing

I installed the snowflake connector via the command: pip3 install snowflake-connector-python[pandas]==2.3.3 asn1crypto==1.3.0 --user
I attempted to connect via:
from snowflake import connector
con = connector.connect(
host='.snowflakecomputing.com',
user='THE USER I USE FOR LOGGING IN TO MY TRIAL ACCOUNT',
password='THE PASSWORD I USE FOR LOGGING IN TO MY TRIAL ACCOUNT',
account='zka81761.us-east-1',
warehouse='COMPUTE_WH',
database='DEMO_DB',
schema='PUBLIC',
protocol='https',
port=443)
When executing the above code it just hangs for several minutes then I get an error:
snowflake.connector.errors.OperationalError: 250003: Failed to execute request: encoding with 'idna' codec failed (UnicodeError: label empty or too long)
The longer version is:
File "tests/integration_tests/data_sources/test_snowflake_ds.py", line 6, in test_snowflake_ds
ds = SnowflakeDS(query='SELECT * FROM HEALTHCARE_COSTS', host='.snowflakecomputing.com', user='GEORGE3D6', password='a passwordd', account='zka81761.us-east-1', warehouse='COMPUTE_WH', database='DEMO_DB', schema='PUBLIC', protocol='https', port=443)
File "/home/george/mindsdb_native/mindsdb_native/libs/data_types/data_source.py", line 13, in __init__
df, col_map = self._setup(*args, **kwargs)
File "/home/george/mindsdb_native/mindsdb_native/libs/data_sources/snowflake_ds.py", line 21, in _setup
port=port)
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/__init__.py", line 52, in Connect
return SnowflakeConnection(**kwargs)
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/connection.py", line 219, in __init__
self.connect(**kwargs)
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/connection.py", line 414, in connect
self.__open_connection()
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/connection.py", line 613, in __open_connection
self._authenticate(auth_instance)
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/connection.py", line 839, in _authenticate
self.__authenticate(self.__preprocess_auth_instance(auth_instance))
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/connection.py", line 869, in __authenticate
session_parameters=self._session_parameters,
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/auth.py", line 209, in authenticate
socket_timeout=self._rest._connection.login_timeout)
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/network.py", line 509, in _post_request
_include_retry_params=_include_retry_params)
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/network.py", line 586, in fetch
**kwargs)
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/network.py", line 676, in _request_exec_wrapper
conn, full_url, cause)
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/network.py", line 706, in handle_invalid_certificate_error
'errno': ER_FAILED_TO_REQUEST,
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/errors.py", line 128, in errorhandler_wrapper
connection.errorhandler(connection, cursor, error_class, error_value)
File "/home/george/.local/lib/python3.7/site-packages/snowflake/connector/errors.py", line 90, in default_errorhandler
done_format_msg=error_value.get('done_format_msg'))
snowflake.connector.errors.OperationalError: 250003: Failed to execute request: encoding with 'idna' codec failed (UnicodeError: label empty or too long)
This error message tells me nothing, any help would be appreicated
According to the documentation on the python API the host field is no longer used so try removing that. Also, even if it was used, you haven't enclosed it properly in quotes:
You have: host='.snowflakecomputing.com,
Should be: host='.snowflakecomputing.com',
First, I'd see if removing the host completely fixes your issue since it shouldn't be used anyway.
Googling the error, and the error message itself, suggests that the issue is due to the URL being too long so I'd say the error is down to the fact that you haven't enclosed it properly.
I used to connect snowflake from python like below:
import snowflake.connector as sf
sfconnection = sf.connect(
user='THE USER I USE FOR LOGGING IN TO MY TRIAL ACCOUNT',
password='THE PASSWORD I USE FOR LOGGING IN TO MY TRIAL ACCOUNT',
account='zka81761.us-east-1',
warehouse='COMPUTE_WH',
database='DEMO_DB',
schema='PUBLIC')
Apparently the docs are wrong and the host should now be the full URL (so, in my case zka81761.us-east-1.snowflakecomputing.com). That is to say, it should include the account.

Why importing data to Zoho Analytics API causes error?

My goal is to write a script in Python3 that will push data in existing table in Zoho Analytics, the script will be used by a scheduler once a week.
What I have tried to far:
I can successfully import some data using cURL commands. Like so
curl -X POST \ 'https://analyticsapi.zoho.com/api/OwnerEmail/Workspace/TableName?ZOHO_ACTION=IMPORT& ZOHO_OUTPUT_FORMAT=JSON&ZOHO_ERROR_FORMAT=JSON&ZOHO_API_VERSION=1.0&ZOHO_IMPORT_TYPE=APPEND&ZOHO_AUTO_IDENTIFY=True&ZOHO_ON_IMPORT_ERROR=ABORT&ZOHO_CREATE_TABLE=False' \ -H 'Authorization: Zoho-oauthtoken *******' \ -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \ -F ZOHO_FILE='path_to_csv'
What I found out that the ReportClient provided by Zoho Analytics team
Zoho Report Client for Python is not compatible with Python 3. Hence, I installed a wrapped for this ReportClient from [here] (https://pypi.org/project/zoho-analytics-connector).
Following sample examples from Zoho website and tests in github of wrapper for Zoho in Python3, I implement something like this:
Have a class to keep my ENV variables
import os
from zoho_analytics_connector.report_client import ReportClient, ServerError
from zoho_analytics_connector.enhanced_report_client import EnhancedZohoAnalyticsClient
class ZohoTracking:
LOGINEMAILID = os.getenv("ZOHOANALYTICS_LOGINEMAIL")
REFRESHTOKEN = os.getenv("ZOHOANALYTICS_REFRESHTOKEN")
CLIENTID = os.getenv("ZOHOANALYTICS_CLIENTID")
CLIENTSECRET = os.getenv("ZOHOANALYTICS_CLIENTSECRET")
DATABASENAME = os.getenv("ZOHOANALYTICS_DATABASENAME")
OAUTH = True
TABLENAME = "My Table"
Instantiate the Client Class
def get_enhanced_zoho_analytics_client(self) -> EnhancedZohoAnalyticsClient:
assert (not self.OAUTH and self.AUTHTOKEN) or (self.OAUTH and self.REFRESHTOKEN)
rc = EnhancedZohoAnalyticsClient(
// Just setting email, token, etc using class above
...
)
return rc```
Then have a method to upload data to existing table, the data_upload() function has the problem.
def enhanced_data_upload(self):
enhanced_client = self.get_enhanced_zoho_analytics_client()
try:
with open("./import/tracking3.csv", "r") as f:
import_content = f.read()
print(type(import_content))
except Exception as e:
print(f"Error:Check if file exists in the import directory {str(e)}")
return
res = enhanced_client.data_upload(import_content=import_content, table_name=ZohoTracking.TABLENAME)
assert res
Traceback (most recent call last):
File "push2zoho.py", line 106, in <module>
sample.enhanced_data_upload()
File "push2zoho.py", line 100, in enhanced_data_upload
res = enhanced_client.data_upload(import_content=import_content, table_name=ZohoTracking.TABLENAME)
File "/Users/.../zoho_analytics_connector/enhanced_report_client.py", line 99, in data_upload
matching_columns=matching_columns)
File "/Users/.../site-packages/zoho_analytics_connector/report_client.py", line 464, in importData_v2
r=self.__sendRequest(url=url,httpMethod="POST",payLoad=payload,action="IMPORT",callBackData=None)
File "/Users/.../zoho_analytics_connector/report_client.py", line 165, in __sendRequest
raise ServerError(respObj)
File "/Users/.../zoho_analytics_connector/report_client.py", line 1830, in __init__
contHeader = urlResp.headers["Content-Type"]
TypeError: 'NoneType' object is not subscriptable
That is the error I receive. What am I missing in this puzzle? Help is appreciated
In Feb 2021 I changed this inherited Zoho code in my library.
Now it is:
contHeader = urlResp.headers.get("Content-Type",None)
which avoids the final exception you had.

Python image save error - raise ValueError("unknown file extension: {}".format(ext)) from e ValueError: unknown file extension:

Am just having four weeks of experience in Python. Creating a tool using Tkinter to paste a new company logo on top of the existing images.
The Below method is to, get all images in the given directory and paste the new logo on the initial level. Existing image, edited image, x-position, y-position, a preview of the image and few data's are store in global instance self.images_all_arr.
def get_img_copy(self):
self.images_all_arr = []
existing_img_fldr = self.input_frame.input_frame_data['existing_img_folder']
for file in os.listdir(existing_img_fldr):
img_old = Image.open(os.path.join(existing_img_fldr, file))
img_new_copy = img_old.copy()
self.pasteImage(img_new_copy, initpaste=True) #process to paste new logo.
view_new_img = ImageTk.PhotoImage(img_new_copy)
fname, fext = file.split('.')
formObj = {
"fname": fname,
"fext": fext,
"img_old": img_old,
"img_new": img_new_copy,
"img_new_view": view_new_img,
"add_logo": 1,
"is_default": 1,
"is_opacityImg": 0,
"pos_x": self.defult_x.get(),
"pos_y": self.defult_y.get()
}
self.images_all_arr.append(formObj)
After previewing each image in Tkinter screen, doing some adjustment in position x and y(updating pos_x and pos_y in the list self.images_all_arr) depends upon the necessity.
Well, once all done. Need to save the edited images. Below method to save images, iterating the list self.images_all_arr and call save method as img['img_new'].save(dir_output) since img['img_new'] has updated image.
def generate_imgae(self):
if len(self.images_all_arr):
dir_output = 'xxxxx'
for img in self.images_all_arr:
print(img['img_new'])
img['img_new'].save(dir_output)
print('completed..')
But it returns below error,
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Program Files (x86)\Python38-32\lib\site-packages\PIL\Image.py", line 2138, in save
format = EXTENSION[ext]
KeyError: ''
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Program Files (x86)\Python38-32\lib\tkinter_init_.py", line 1883, in call
return self.func(*args)
File "C:\Users\662828\WORKSPACE\AAA_python_projects\AMI_automation_poc2\position_and_images.py", line 241, in generate_imgae
img['img_new'].save(dir_output)
File "C:\Program Files (x86)\Python38-32\lib\site-packages\PIL\Image.py", line 2140, in save
raise ValueError("unknown file extension: {}".format(ext)) from e
ValueError: unknown file extension:
dir_output doesn't contain the file extension, its just xxxxx. You need to specify what image file format you want. the error tells us this by saying "unknown file format".
Basically, you either need to include the extension in the file name, or pass it as the next parameter in image.save. You can check out the documentation here
eg.
image.save('image.png')
or
image.save('image', 'png')
The below code solved my issue. By giving the exact directory, filename and extension of the image as param to the method image.save(). Here the result of opfile is, C:\Users\WORKSPACE\AAA_python_projects\test\valid.png.
def generate_imgae(self):
if len(self.images_all_arr):
dir_output = r"C:\Users\WORKSPACE\AAA_python_projects\test"
if not os.path.isdir(dir_output):
os.mkdir(dir_output)
for img in self.images_all_arr:
opfile = os.path.join(dir_output, '{}.{}'.format(img['fname'], img['fext'] ))
img['img_new'].save(opfile)
print('completed..')
Thanks #dantechguy

Resources