I am trying to create and serve a zip file for user that contains ical files for different workers (each worker has his own ical file)
The problem is that i get the right number of iCal files in my zip but the last file has all the data from previous workers in it. (also the one before etc.)
this is the code I am using
What am I doing wrong?
This is my code
cal = Calendar()
import zipfile, cStringIO
exported_chunks_zip = cStringIO.StringIO()
zipf = zipfile.ZipFile(exported_chunks_zip, "w", compression=zipfile.ZIP_DEFLATED )
for i, rec in enumerate(grouped):
worker = rec['rw_worker_nick'].encode('cp1250')
for rr in rec["allData"]:
startDate = rr['rw_date']
startTime = rr['rw_time_start']
endTime = rr['rw_time_end']
evtstart = datetime.datetime.combine(startDate,startTime)
evtend = datetime.datetime.combine(startDate,endTime)
event = Event()
event.add('summary', rec['rw_worker_nick'])
event.add('dtstart', evtstart)
event.add('dtend', evtend)
cal.add_component(event)
text = cal.to_ical()
zipf.writestr(worker +'.ics', text)
text = ''
any suggestions?
thank you
You create only a single Calendar object outside of the for loop and then keep appending events to it. You should instead create a new Calendar object for each worker within the for loop:
for i, rec in enumerate(grouped):
cal = Calendar()
...
Related
I'm new to Openstack and I'm trying to create a tool so that I can launch any number of instances in an Openstack cloud. This was easily done using the nova-client module of openstacksdk.
Now the problem is that I want to make the instances execute a bash script as they are created by adding it as a userdata file, but it doesn't execute. This is confusing because I don't any error or warning message. Does anyone know what could it be?
Important parts of the code
The most important parts of the Python program are the function which gets the cloud info, the one that creates the instances and the main function, . I'll post them here as #Corey told.
"""
Function that allow us to log at cloud with all the credentials needed.
Username and password are not read from env.
"""
def get_nova_credentials_v2():
d = {}
user = ""
password = ""
print("Logging in...")
user = input("Username: ")
password = getpass.getpass(prompt="Password: ", stream=None)
while (user == "" or password == ""):
print("User or password field is empty")
user = input("Username: ")
password = getpass.getpass(prompt="Password: ", stream=None)
d['version'] = '2.65'
d['username'] = user
d['password'] = password
d['project_id'] = os.environ['OS_PROJECT_ID']
d['auth_url'] = os.environ['OS_AUTH_URL']
d['user_domain_name'] = os.environ['OS_USER_DOMAIN_NAME']
return d
Then we have the create_server function:
"""
This function creates a server using the info we got from JSON file
"""
def create_server(server):
s = {}
print("Creating "+server['compulsory']['name']+"...")
s['name'] = server['compulsory']['name']
s['image'] = server['compulsory']['os']
s['flavor'] = server['compulsory']['flavor']
s['min_count'] = server['compulsory']['copyNumber']
s['max_count'] = server['compulsory']['copyNumber']
s['userdata'] = server['file']
s['key_name'] = server['compulsory']['keyName']
s['availability_zone'] = server['compulsory']['availabilityZone']
s['nics'] = server['compulsory']['network']
print(s['userdata'])
if(exists("instalacion_k8s_docker.sh")):
print("Exists")
s['userdata'] = server['file']
nova.servers.create(**s)
And now the main function:
"""
Main process: First we create a connection to Openstack using our credentials.
Once connected we cal get_serverdata function to get all instance objects we want to be created.
We check that it is not empty and that we are not trying to create more instances than we are allowed.
Lastly we create the instances and the program finishes.
"""
credentials = get_nova_credentials_v2()
nova = client.Client(**credentials)
instances = get_serverdata()
current_instances = len(nova.servers.list())
if not instances:
print("No instance was writen. Check instances.json file.")
exit(3)
num = 0
for i in instances:
create_server(i)
exit(0)
For the rest of the code you can access to this public repo on github.
Thanks a lot!
Problem solved
The problem was the content of the server['file'] as #Corey said. It cannot be the Path to the file where you wrote the data but the content of it or a file type object. In the case of OpenstackSDK it must be base64 encoded but it is not the case in Novaclient.
Thanks a lot to #Corey for all the help! :)
I made a GUI Application which looks like this:
The ones marked red are Tkinter Text widgets and the ones marked yellow are Tkinter Entry widgets
After taking user input, the data is to be added to a PSD file and then rendered as an image. But Lets say, after taking the following data as input:
It renders the following Photoshop file:
How do I fix this issue that it does not recognize "\n" properly and hence the rendered document is rendered useless.
Here is the code which deals with converting of the accepted user data into strings and then adding it to Photoshop template and then rendering it:
def DataAdder2CSV():
global edate, eSNO, eage, egender, ename, ePID, econtact, ecomp, eallergy, ehistory, eR
e=edate.get()
a=eSNO.get()
d=eage.get()
f=egender.get()
b=ename.get()
c=ePID.get()
g=econtact.get()
h=ecomp.get(1.0,END)
i=eallergy.get(1.0,END)
j=ehistory.get(1.0,END)
k=eR.get(1.0,END)
data=[a,b,c,d,e,f,g,h,i,j,k]
file=open("Patient_Data.csv","a", newline="")
writer=csv.writer(file, delimiter=",")
writer.writerow(data)
file.close()
messagebox.showinfo("Prescription Generator", "Data has been saved to the database successfully!")
import win32com.client, os
objShell = win32com.client.Dispatch("WScript.Shell")
UserDocs = objShell.SpecialFolders("MyDocuments")
from tkinter import filedialog
ExpDir=filedialog.askdirectory(initialdir=UserDocs, title="Choose Destination Folder")
psApp = win32com.client.Dispatch("Photoshop.Application")
psApp.Open("D:\Coding\Python Scripts\Dr Nikhil Prescription App\Prescription Generator\Presc_Template.psd")
doc = psApp.Application.ActiveDocument
lf1 = doc.ArtLayers["name"]
tol1 = lf1.TextItem
tol1.contents = b
lf2 = doc.ArtLayers["age"]
tol2 = lf2.TextItem
tol2.contents = d
lf3 = doc.ArtLayers["gender"]
tol3 = lf3.TextItem
tol3.contents = f
lf4 = doc.ArtLayers["pid"]
tol4 = lf4.TextItem
tol4.contents = c
lf4 = doc.ArtLayers["date"]
tol4 = lf4.TextItem
tol4.contents = e
lf5 = doc.ArtLayers["contact"]
tol5 = lf5.TextItem
tol5.contents = g
lf6 = doc.ArtLayers["complaint"]
tol6 = lf6.TextItem
varH=" "+h.rstrip("\n")
tol6.contents =varH
lf7 = doc.ArtLayers["allergy"]
tol7 = lf7.TextItem
tol7.contents = i.rstrip("\n")
lf8 = doc.ArtLayers["history"]
tol8 = lf8.TextItem
varJ=" "+j.rstrip("\n")
tol8.contents =varJ
lf9 = doc.ArtLayers["R"]
tol9 = lf9.TextItem
tol9.contents = k.rstrip("\n")
options = win32com.client.Dispatch('Photoshop.ExportOptionsSaveForWeb')
options.Format = 13
options.PNG8 = False
pngfile =ExpDir+f"/{c}-{b}_({e}).png"
doc.Export(ExportIn=pngfile, ExportAs=2, Options=options)
messagebox.showinfo("Prescription Generator", "Prescription has been saved in the desired location successfully!")
There are 3 ways of expressing new line characters:
MacOS uses \r
Linux uses \n
Windows uses \r\n
Python and tkinter use \n but it looks like psApp.Application uses \r instead. That is why the document isn't rendered properly. For more info read the answers to this question.
Here's my sample code which works:
import os, io, dropbox
def createFolder(dropboxBaseFolder, newFolder):
# creating a temp dummy destination file path
dummyFileTo = dropboxBaseFolder + newFolder + '/' + 'temp.bin'
# creating a virtual in-memory binary file
f = io.BytesIO(b"\x00")
# uploading the dummy file in order to cause creation of the containing folder
dbx.files_upload(f.read(), dummyFileTo)
# now that the folder is created, delete the dummy file
dbx.files_delete_v2(dummyFileTo)
accessToken = '....'
dbx = dropbox.Dropbox(accessToken)
dropboxBaseDir = '/test_dropbox'
dropboxNewSubDir = '/new_empty_sub_dir'
createFolder(dropboxBaseDir, dropboxNewSubDir)
But is there a more efficient/simpler way to do the task ?
Yes, as Ronald mentioned in the comments, you can use the files_create_folder_v2 method to create a new folder.
That would look like this, modifying your code:
import dropbox
accessToken = '....'
dbx = dropbox.Dropbox(accessToken)
dropboxBaseDir = '/test_dropbox'
dropboxNewSubDir = '/new_empty_sub_dir'
res = dbx.files_create_folder_v2(dropboxBaseDir + dropboxNewSubDir)
# access the information for the newly created folder in `res`
I'm in need of optimizing import of .xls files to matlab due to xlsread being very time consuming with large amount of files. Current xlsread script as follows:
scriptName = mfilename('fullpath');
[currentpath, filename, fileextension]= fileparts(scriptName);
xlsnames = dir(fullfile(currentpath,'*.xls'));
xlscount = length(xlsnames);
xlsimportdata = zeros(7,6,xlscount);
for k = 1:xlscount
xlsimport = xlsread(xlsnames(k).name,'D31:I37');
xlsimportdata(:,1:size(xlsimport,2),k) = xlsimport;
end
I have close to 10k files per week that needs processing and with approx. 2sec per file processed on my current workstation, it comes in at about 5½ hours.
I have read that ActiveX can be used for this purpose however that is far beyond my current programming skills and have not been able to find a solution elsewhere. Any help on how to make this would be appreciated.
If it is simple to perform with ActiveX (or other proposed method), I would also be interested in data on cells D5 and G3, which I am currently grabbing from 'xlsnames(k,1).name' and 'xlsnames(k,1).date'
EDIT: updated to reflect the solution
% Get path to .m script
scriptName = mfilename('fullpath');
[currentpath, filename, fileextension]= fileparts(scriptName);
% Generate list of .xls file data
xlsnames = dir(fullfile(currentpath,'*.xls'));
xlscount = length(xlsnames);
SampleInfo = cell(xlscount,2);
xlsimportdata = cell(7,6,xlscount);
% Define xls data ranges to import
SampleID = 'G3';
SampleRuntime = 'D5';
data_range = 'D31:I37';
% Initiate progression bar
h = waitbar(0,'Initiating import...');
% Start actxserver
exl = actxserver('excel.application');
exlWkbk = exl.Workbooks;
for k = 1:xlscount
% Restart actxserver every 100 loops due limited system memory
if mod (k,100) == 0
exl.Quit
exl = actxserver('excel.application');
exlWkbk = exl.Workbooks;
end
exlFile = exlWkbk.Open([dname filesep xlsnames(k).name]);
exlSheet1 = exlFile.Sheets.Item('Page 0');
rngObj1 = exlSheet1.Range(SampleID);
xlsimport_ID = rngObj1.Value;
rngObj2 = exlSheet1.Range(SampleRuntime);
xlsimport_Runtime = rngObj2.Value;
rngObj3 = exlSheet1.Range(data_range);
xlsimport_data = rngObj3.Value;
SampleInfo(k,1) = {xlsimport_ID};
SampleInfo(k,2) = {xlsimport_Runtime};
xlsimportdata(:,:,k) = xlsimport_data;
% Progression bar updater
progress = round((k / xlscount) * 100);
importtext = sprintf('Importing %d of %d', k, xlscount);
waitbar(progress/100,h,sprintf(importtext));
disp(['Import progress: ' num2str(k) '/' num2str(xlscount)]);
end
%close actxserver
exl.Quit
% Close progression bar
close(h)
Give this a try. I am not an ActiveX Excel guru by any means. However, this works for me for my small amount of test XLS files (3). I never close the exlWkbk so I don't know if memory usage is building or if it automatically cleaned up when descoped after the next is opened in its place ... so use at your own risk. I am seeing an almost 2.5x speed increase which seems promising.
>> timeit(#getSomeXLS)
ans =
1.8641
>> timeit(#getSomeXLS_old)
ans =
4.6192
Please leave some feedback if this work on large number of Excel sheets because I am curious how it goes.
function xlsimportdata = getSomeXLS()
scriptName = mfilename('fullpath');
[currentpath, filename, fileextension]= fileparts(scriptName);
xlsnames = dir(fullfile(currentpath,'*.xls'));
xlscount = length(xlsnames);
xlsimportdata = zeros(7,6,xlscount);
exl = actxserver('excel.application');
exlWkbk = exl.Workbooks;
dat_range = 'D31:I37';
for k = 1:xlscount
exlFile = exlWkbk.Open([currentpath filesep xlsnames(k).name]);
exlSheet1 = exlFile.Sheets.Item('Sheet1'); %Whatever your sheet is called.
rngObj = exlSheet1.Range(dat_range);
xlsimport = cell2mat(rngObj.Value);
xlsimportdata(:,:,k) = xlsimport;
end
exl.Quit
Hi I am trying to unload multiple tables from Redshift to a particular S3 bucket getting below error:
psycopg2.InternalError: Specified unload destination on S3 is not empty. Consider using a different bucket / prefix, manually removing the target files in S3, or using the ALLOWOVERWRITE option.
if I add 'allowoverwrite' option on unload_function, it is overwritting before table and unloading last table in S3.
This is the code I have given:
import psycopg2
def unload_data(r_conn, aws_iam_role, datastoring_path, region, table_name):
unload = '''unload ('select * from {}')
to '{}'
credentials 'aws_iam_role={}'
manifest
gzip
delimiter ',' addquotes escape parallel off '''.format(table_name, datastoring_path, aws_iam_role)
print ("Exporting table to datastoring_path")
cur = r_conn.cursor()
cur.execute(unload)
r_conn.commit()
def main():
host_rs = 'dataingestion.*********.us******2.redshift.amazonaws.com'
port_rs = '5439'
database_rs = '******'
user_rs = '******'
password_rs = '********'
rs_tables = [ 'Employee', 'Employe_details' ]
iam_role = 'arn:aws:iam::************:role/RedshiftCopyUnload'
s3_datastoring_path = 's3://mysamplebuck/'
s3_region = 'us_*****_2'
print ("Exporting from source")
src_conn = psycopg2.connect(host = host_rs,
port = port_rs,
database = database_rs,
user = user_rs,
password = password_rs)
print ("Connected to RS")
for i, tabe in enumerate(rs_tables):
if tabe[0] == tabe[-1]:
print("No files to read!")
unload_data(src_conn, aws_iam_role = iam_role, datastoring_path = s3_datastoring_path, region = s3_region, table_name = rs_tables[i])
print (rs_tables[i])
if __name__=="__main__":
main()
It is complaining that you are saving the data to the same destination.
This would be like copying all the files on your computer to the same directory -- there will be files overwritten.
You should change your datastoring_path to be different for each table, such as:
.format(table_name, datastoring_path + '/' + table_name, aws_iam_role)