Hi I have a file name like this. I would like to remove the date part from the file name daily before my load and append the date after the load gets completed. How would I be able to achieve that?.
file name:-
zip_cost_03_08_2018 21_13_04.csv
I need the file name like below before my load starts
zip_cost.csv
I need to append the date back once my load gets completed.
zip_cost_03_08_2018 21_13_04.csv
You can get the timestamp in the format you want by using the date command.
$ date "+%m_%d_%Y_%H_%M_%S"
03_09_2018_09_21_40
So with that, you can just do -
mv "zip_cost_03_08_2018 21_13_04.csv" zip_cost.csv
# Run the load operation
mv zip_cost.csv "zip_cost_$(date '+%m_%d_%Y_%H_%M_%S').csv"
Related
I need to create a python script, which can read 1 hour before and current time data from the log file. And after that I have to search for the matching string and send a mail based on them.
No Idea , Help needed
I frequent a real estate website that shows recent transactions, from which I will download data to parse within a Pandas dataframe. Everything about this dataset remains identical every time I download it (regarding the column names, that is).
The name of the Excel output may change, though. For example, if I already have download a few of these in my Downloads folder, the file that's exported may read "Generic_File_(3)" or "Generic_File_(21)" if I already have a few older "Generic_File" exports in that folder from a previous export.
Ideally, I'd like my workflow to look like this: export this Excel file of real estate sales, then run a Python script to read in the most recent export as a Pandas dataframe. The catch is, I don't want to have to go in and change the filename in the script to match the appending number of the Excel export everytime. I want the pd.read_excel method to simply read the "Generic_File" that is appended with the largest number (which will obviously correspond to the most rent export).
I suppose I could always just delete old exports out of my Downloads folder so the newest, freshest export is always named the same ("Generic_File", in this case), but I'm looking for a way to ensure I don't have to do this. Are wildcards the best path forward, or is there some other method to always read in the most recently downloaded Excel file from my Downloads folder?
I would use the OS package and create a method to read to file names in the downloads folder. Parsing string filenames you could then find the file following your specified format with the highest copy number. Something like the following might help you get started.
import os
downloads = os.listdir('C:/Users/[username here]/Downloads/')
is_file = [True if '.' in item else False for item in downloads]
files = [item for keep, item in zip(is_file, downloads) if keep]
** INSERT CODE HERE TO IDENTIFY THE FILE OF INTEREST **
Regex might be the best way to find matches if you have a diverse listing of files in your downloads folder.
I have a folder which has log files from June to now. I want to retrieve files created in the last week of August and store them at a different path. I can get the creation date from the filename which is in this format:
Run_Merge_BJDSBC_20190901-093
I want to code this in python, so far I have reached here:
for filename in glob.glob(r"C:\Users\chke01\Desktop\PentahoLogs\BAF\*201908[0-9]*.log"):
But I dont know how to select these filenames and write to another folder. Can someone help me here
glob.glob('path') gives you a list of files. You can iterate over that list and use the shutil module to move and rename the files.
for file in glob.glob('*'):
shutil.move(file, new_folder_path)
I have a python code and I am saving the results in a destination with specific file name, this file name will change every time and it is a recurring event.
Here is my code:
import csv
outfile=open('path.macrovariable.csv','w',newline='')
writer=csv.writer(outfile)
writer.writerow(["Jobname","employement","company","descrption","location"])
writer.writerows([job_name])
writer.writerows([emplomnt_type])
writer.writerows([organisation])
writer.writerows([job_descrption])
writer.writerows([job_location])
In the outfile the file name here as "macrovariable" will change every time. I want to create a macrovariable at the top of the program which will be called later in the program in the place of "macrovariable" instead of hardcoding.
Thanks & regards,
Sanjay.
I have qvw file with sql query
Data:
LOAD source, color, date;
select source, color, date
as Mytable;
STORE Data into [..\QV_Data\Data.qvd] (qvd);
Then I export data to excel and save.
I need something to do that automatically instead of me
I need to run query every day and automatically send data to excel but keep old data in excel and append new value.
Can qlikview to do that?
For that you need to create some crazy macro that runs after a reload task in on open-trigger. If you schedule a windows task that execute a bat file with path to qlikview.exe with the filepath as parameters and -r flag for reload(?) you can probably accomplish this... there are a lot of code of similar projects to be found on google.
I suggest adding this to the loadscript instead.
STORE Table into [..\QV_Data\Data.csv] (txt);
and then open that file in excel.
If you need to append data you could concatenate new data onto the previous data.. something like:
Data:
load * from Data.csv;
//add latest data
concatenate(Data)
LOAD source, color, date from ...
STORE Data into [..\QV_Data\Data.csv] (txt);
I assume you have the desktop version so you don't have access to the Qlikview Management Console (if you do, this is obviously the best way).
So, without the Console, you should create a txt file with this command: "C:\Program Files\QlikView\Qv.exe" /r "\\thePathToYourFile\test.qvw". Save this file with .cmd file extension. After that you can schedule this command file with the windows task scheduler.