I'm talking to a server that creates a new zip file daily, ex: (data-1234.zip). Every day the name of the previous zip is removed and a new one is created with an incremented number, ex: (data-1235.zip). The script will be run sporadically throughout the week but it's on a lab system where the user can't manually update the name with what's on the server.
The server only has one zip file in that directory, it's just a matter of getting the correct naming convention. There is, however a "data.ini" file in the folder as well, so something just searching by first name wouldn't necessarily work. I've seen posts similar to This question using regex but the file is currently on 10,609 and I'd rather not use expansion for potentially thousands of calls depending on access to modify the script in the coming years. I've been searching for something similar to "data-*.zip" but haven't had any luck.
Question was solved by changing commands and running
lftp https://download.companyname.com/product/data/ -e "mget data-*.zip; bye"
since lftp allows wildcards in the filename, unlike curl.
I am trying to rename a file from multiple directories
I have a file called slave.log in multiple directories like slave1,slave2......slave17. so daily log rotation happens and creates a new file with current dateformat in it whereas the file data contains a previous day data . I want to rename those files with previous date format .
I have written shell script which works fine but the problem here is , I need to pass the path as parameter but likewise I have 17 directories so i cant schedule 17 cron enteries to run ...I have only basic knowledge about scripting . Please help me with best solution for this scenario
How to get files starting specific name?
I have script like this
open Area_4#10.54.18.18
get -neweronly "/Payload/Daily/2017/Daily_Payload_BH_2*.csv" "D:\FTP-NSQM\2_Payload"
get -neweronly "/Payload/Daily/2017/Daily_Payload_BH_3*.csv" "D:\FTP-NSQM\3_Payload"
I used batch file for execute this script, but nothing happened
If i don't use a specific name (Daily_Payload_BH_2*.csv or Daily_Payload_BH_3*.csv) for get a new files
get -neweronly "/Payload/Daily/2017" "D:\FTP-NSQM\2_Payload"
This script executed properly
Your code is ambiguous.
To get comparable results use:
get -neweronly "/Payload/Daily/2017/Daily_Payload_BH_2*.csv" "D:\FTP-NSQM\2_Payload\"
get -neweronly "/Payload/Daily/2017/Daily_Payload_BH_3*.csv" "D:\FTP-NSQM\3_Payload\"
and
get -neweronly "/Payload/Daily/2017/*" "D:\FTP-NSQM\2_Payload\"
These should get you consistent results.
Note the /* in source paths and the trailing backslash in target paths.
Excerpts from get command documentation (emphasis mine):
Filename can be replaced with wildcard to select multiple files. To download all files in a directory, use mask *.
The last parameter specifies target local directory and optionally operation mask to store file(s) under different name. Destination directory must end with backslash.
If this does not help, we need to see complete log files of both scripts.
How do we connect to a FTP from a shell script and pull the last 14 (or n) modified files by timestamp and place them on one of the folders from the current host. I try to use
mget
, but can we specify to get the files based on the timestamp and the number of files to get... Please advise... Thanks in advance...
You can define an ftp macro (macdef) and automate your login (in your ~/.netrc file) and in youe script fetch a file listing; sort the listing by date with awk or sort -k in your script to build a list of N files that you want to fetch; then simply loop through the list and fire up ftp to fetch them.
It might be easier to use curl. and probably would more portable to use something like perl and Net::FTP.
I have one file delivered in a ftp daily. This file doesn´t have the same name everyday. It has the date and the hour of the creation. For example, today the file has the name 20130814_XX_YY_20130814152345, created at 15:23:45 and tomorrow the file can name 20130815_XX_YY_20130815152421. The _XX_YY_ is always the same but the hour will change everyday.
I want to create a host jcl that gets this file with variable name and rename it to a host file. How can I do this ?
Thank you
Regards
Chuchito
STEP1: You can use LS in FTP to write to disk, so you can have a file with the file-name in it. Then GET that file.
STEP2: Process the contents of your file to generate the FTP Control Cards (at least for the GET). The GET generated will be of the form GET 20130814_XX_YY_20130814152345 'HLQ.MAINFRAM.DATASET', where the server name has come from the file GETted in STEP1 and the local (Mainframe) file can be hard-coded, or supplied to the generation if flexibility is required.
STEP3: Run FTP again with the Control Card(s) generated.
Isn't there anything in the Spec?
Sometimes we create complexities where an "out of the box" solution simplifies life considerably.
After the post updated, I now understand the problem a bit better.
If the name is required to be so specific, then the other suggested solution (if i understand it) is to have a fixed file name on the server that contains a list of file names to be uploaded.
In fact, the server could create a fixed file name that is really the JCL to run on the mainframe!!! This file would include the //SYSIN DD * and GET commands! The mainframe uploads this file and submits it as-is to the job reader, which then runs on the mainframe. The last step of this job (created by the server, but run on the mainframe) is to FTP an empty JCL file back to the server, in this way the server "knows" that the mainframe has uploaded the files.
Alternatively, why does the non-Z\os system need to name the file with time information? If the mainframe processes the file daily then date should be sufficient.
With this change the mainframe can reliably predict the file name for the day, generate the appropriate GET command and run.
With a job scheduler it would be easy to run the upload to the mainframe twice a day. This might address any concerns that are expressed in the desire to include a time in the file's name.
Run a Rexx step via a Background TSO step:
Background TSO step
You can then run a listcat to get all the files. You could either write the listcat output to a file and read it in or trap the output via the Address command
or the OutTrap function.
Then use the standard TSO Rename command.
Alternatively you could run ISPF background rexx program and use the ISPF equivalents to get the file name
(1) The real solution to this should be through a scheduling tool for Mainframe jobs. These tools provide capabilities to take care of formatting like the one you described.
(2) Alternatives: REXX and COBOL
(3) If you don't want to prefer REXX, here's a little brief into how you could create the JCL dynamically using COBOL:
A COBOL program that would read a "template" JCL.
Using INSPECT / REPLACE, you could substitute the prototypes with the string that is populated with the date of your choice (you could supply this as a simple SYSIN parm too, if you want the COBOL code to be flexible on the date selection)
Now that your formatted JCL is ready, you could write it to the output stream
//OUTFILE DD SYSOUT=(INTRDR,)
or
//OUTFILE DD SYSOUT=(,INTRDR)
Anything that is written to INTRDR (Internal Reader), goes straight to JES to submit your job!
Hope this helps.