Search for string in PDS and exclude results using JCL - search

I am trying to do the following:
1) Search members of a PDS for a string.
2) Exclude the members with the above mentioned string and get a list of members.
I want to do this using a JCL or any other mainframe tool or utility.
Any help is appreciated.

I think you're looking for the ISPF Search-For utility, which can also be executed in batch.
You can also use the SRCHFOR command at an ISPF member listing screen, then sort by the prompt column which indicates which members had matches.

There are probably much better ways of doing this but this should work for you.
Copy the following JCL and replace:
your-job-car-info with whatever your job card needs
pds-to-search with the name of the PDS you want to search
temp-pds-name with a valid PDS name that won't duplicate anything you already have
string-to-search-for with the string you want to search PDS members for
The list of PDS members that do not contain string-to-search-for will be listed under DD LISTING,
which you can redirect wherever appropriate.
The first job step (SEARCH) invokes the ISPF SEARCHFOR utility, to identify all the PDS members that contain the string.
The second step (LSTPDS) invokes the TSO LISTDS utility to list all members of the PDS. The third step (REXXPGM) writes a little
REXX procedure to a temporary PDS for use in the next step. The last step (LSTMBR) runs the REXX procedure to build the LISTING dataset
and deletes the temporary files built along the way.
If one were smart enough, I think ICETOOL could probably replace the final 2 steps (but what the heck... Actually I did find a better process, see next JCL after this one)
//LOOKFOR JOB your-job-card-info
//*====================================================================
//SEARCH EXEC PGM=ISRSUPC,PARM=(SRCHCMP,'ANYC')
//NEWDD DD DSN=pds-to-search,DISP=SHR
//OUTDD DD DSN=&&SRCH,DISP=(NEW,PASS,DELETE),
// AVGREC=K,
// SPACE=(133,(5,5)),
// DCB=(LRECL=133,BLKSIZE=133,RECFM=FB)
//SYSIN DD *
SRCHFOR 'string-to-search-for'
/*
//*====================================================================
//LSTPDS EXEC PGM=IKJEFT01
//SYSTSPRT DD DSN=&&MBRS,DISP=(NEW,PASS,DELETE),
// AVGREC=K,
// SPACE=(80,(5,5)),
// DCB=(LRECL=80,BLKSIZE=80,RECFM=FB)
//SYSTSIN DD *
LISTDS 'pds-to-search' MEMBERS
/*
//REXXPDS EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
ALLOCATE DSN('temp-pds-name') NEW KEEP -
DSORG(PO) RECFM(F,B) LRECL(80) -
AVGREC(U) SPACE(80,100) DIR(5) DSNTYPE(PDS)
/*
//*====================================================================
//REXXPGM EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
REPRO INFILE(REXXPROG) OUTDATASET('temp-pds-name(MBRLIST)')
/*
//REXXPROG DD *
'EXECIO * DISKR SRCH(FINIS STEM X.'
FND = ''
DO I = 1 TO X.0
IF WORD(X.I, 3) = 'STRING(S)' THEN FND = FND WORD(X.I, 1)
END
DROP X.
'EXECIO * DISKR MBRS(FINIS STEM X.'
MBRS = 0
DO I = 1 TO X.0 - 2
IF MBRS & WORDPOS(WORD(X.I, 1), FND) = 0 THEN
QUEUE WORD(X.I,1)
ELSE
IF X.I = '--MEMBERS--' THEN MBRS = 1
END
'EXECIO * DISKW LISTING(FINIS'
RETURN
/*
//*====================================================================
//LISTMBR EXEC PGM=IKJEFT01,PARM=('%MBRLIST')
//SYSEXEC DD DSN=NXB123.JUNK,DISP=(OLD,DELETE,DELETE)
//SRCH DD DSN=&&SRCH,DISP=(OLD,DELETE,DELETE)
//MBRS DD DSN=&&MBRS,DISP=(OLD,DELETE,DELETE)
//LISTING DD SYSOUT=A
//SYSTSPRT DD SYSOUT=A
//SYSTSIN DD *
/*
//
And this an even better answer... Just took a while to get there
//MYJOB JOB your-job-card-info
//SEARCH EXEC PGM=ISRSUPC,PARM=(SRCHCMP,'LNFMTO')
//NEWDD DD DSN=pds-to-search,DISP=SHR
//OUTDD DD DSN=&&SRCH,DISP=(NEW,PASS,DELETE),
// AVGREC=K,
// SPACE=(133,(5,5)),
// DCB=(LRECL=133,BLKSIZE=133,RECFM=FB)
//SYSIN DD *
SRCHFOR 'string-to-search-for'
/*
//STEP02 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTMSG DD SYSOUT=*
//SORTIN DD DSN=&&SRCH,DISP=(OLD,DELETE,DELETE)
//SORTOUT DD SYSOUT=*
//SYSIN DD *
SORT FIELDS=COPY -
INCLUDE COND=(30,6,CH,EQ,C' 0 ') -
OUTREC FIELDS=(1,8)
/*
The key is the LNFMTO paramter fed to ISRSUPC. This parameter tells ISRSUPC to list only those members where
the search string was not found. Exactly what was asked for. The next step just
formats the results from the previous step. The list of members that do not contain the search
string are listed in SYSOUT which can be redirected where needed.

Another approach: "punch" the whole PDS, in unsorted order, to a "flat" file; use JOINKEYS with data prepared using SS, for Sub-String, field type.
//PUNCHIT EXEC PGM=IEBPTPCH
//SYSUT1 DD DISP=SHR,DSN=your.pds
//SYSUT2 DD DSN=output.dataset,RECFM=FB,LRECL=81,otherstuffyouneed
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
PUNCH TYPORG=PO
/*
After that, a JOINKEYS step using SORT:
//FINDMISS EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTOUT DD SYSOUT=*
//SYSIN DD *
OPTION COPY
JOINKEYS F1=INA,FIELDS=(1,8,A)
JOINKEYS F2=INB,FIELDS=(82,8,A)
JOIN UNPAIRED,F1,ONLY
/*
//JNF1CNTL DD *
INCLUDE COND=(2,13,CH,EQ,C'MEMBER NAME')
INREC BUILD=(15,8)
/*
//JNF2CNTL DD *
INCLUDE COND=(2,13,CH,EQ,C'MEMBER NAME',
OR,2,72,SS,EQ,C'yoursearchvalue')
INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,13,CH,EQ,C'MEMBER NAME'),
PUSH=(82:15,8)),
IFTHEN=(WHEN=(2,13,CH,EQ,C'MEMBER NAME'),
OVERLAY=(82:8X))
/*
//INA DD DISP=SHR,DSN=YOUR.PUNCHED.FILE
//INB DD DISP=SHR,DSN=YOUR.PUNCHED.FILE (yes, exactly the same one)
Then processes the punched file, reading it twice in the same step, and using JOINKEYS to "match" one extract from the file to another extract from the file.
On INA via JNF1CNTL it extracts all the member names.
On INB via JNF2CNTL it extracts all the member names, plus all data-lines which match the required search. IFTHEN=(WHEN=GROUP is then used to mark each data line that has matched with the member-name they are part of, and the member-name itself gets given a blank member-name.
The JOINKEYS then match on the member-name keys. All those from INA via JNF1CNTL which do not match are those members which do not contain the searched-for string. Both files require sorting, which happens by default for each JOINKEYS statement.
This will work with DFSORT. With SyncSort it will depend whether your installation supports JNFnCNTL files or not (you have to try it, it is not documented). If you do not have JNFnCNTL support, it would require two earlier, separate, steps to extract the data.

Related

JCL - SORTIN added dynamically

Would like to know how to include in a SORT step a file that has the SORTIN content, meaning that the file has been created dynamically and its content is something like this
//SORTIN DD DSN=DES.LQHISE03.ENTRADA,DISP=SHR
Step 1: Please have the following 3 datasets.
Dataset 1: Job card and the EXEC statement.
//JOBNAME JOB ('ACCOUNT INFORMATION'),
// JOBCARD,CLASS=2,MSGCLASS=H,NOTIFY=&SYSUID
//STEP1 EXEC PGM=SORT
Dataset 2: SORTIN content - This should be the dynamically created file.
//SORTIN DD DSN=DES.LQHISE03.ENTRADA,DISP=SHR
Dataset 3: Rest of the content
//SORTOUT DD DSN=OUTPUT.DATASET.NAME,
// DISP=(,CATLG,DELETE),
// SPACE=(CYL,(100,0),RLSE),
// DCB=(LRECL=80,RECFM=FB,BLKSIZE=0)
//SYSOUT DD SYSOUT=*
//SYSIN DD *
SORT FIELDS=COPY
/*
//IDCAMS EXEC PGM=IDCAMS,COND=(0,LT,STEP1)
//SYSPRINT DD SYSOUT=*
//DD1 DD DISP=SHR,DSN=YOUR.PDS
DELETE 'YOUR.PDS(MEMBER)' FILE(DD1)
/*
Step 2: Concatenate the datasets.
//STEP2 EXEC PGM=IEBGENER
//SYSPRINT DD SYSOUT=*
//SYSUT1 DD DSN=Dataset 1,DISP=SHR
// DD DSN=Dataset 2,DISP=SHR
// DD DSN=Dataset 3,DISP=SHR
//SYSUT2 DD DSN=Sort Step,
// DISP=(,CATLG,DELETE),
// SPACE=(CYL,(100,0),RLSE),
// DCB=(LRECL=80,RECFM=FB,BLKSIZE=0)
//SYSIN DD DUMMY
Step 3: Have IEBGENER to write the Sort Step (output from STEP 2) as a member in a PDS.
//STEP3 EXEC PGM=IEBGENER,COND=(0,LT,STEP2)
//SYSUT1 DD DSN=*.STEP2.SYSUT2,DISP=SHR
//SYSUT2 DD DSN=YOUR.PDS(MEMBER),DISP=SHR
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
You will now have the complete JCL in a member inside your PDS (Note: The member inside the PDS will be deleted after the completion of the SORT job). You just need to submit the member. You can make use of the IKJEFT01 utility to run a TSO function, LISTDS. This function displays data set attributes. If a member exists, RC 0 is issued & you may add code to SUBMIT the member, else you may issue some return code (to signify non availability of the member). More details here

How can I email one of the cronjob lines to a different email address?

I have the MAILTO setting set in crontab to a certain email address, eg
a#ab.com
There are about 10 entries in the cron on different lines, running at different times and I want to add an 11th line but have it sent to a different email address.
How can I do this from within the crontab?
awk 'last ~ "<cert>" && $0 ~ "</cert>" {print FILENAME; nextfile} {last=$0}' *
I have tried using an output command but it is only outputting to a file.
Just add a different MAILTO line preceeding your 11th line - that setting will remain in effect for all subsequent cron lines:
MAILTO='a#ab.com'
* * * * * /cmd1
# other crontab jobs here, all emailing to 'a#ab.com'
* * * * * /cmd10
MAILTO='someone_else#ab.com'
# following crontab jobs emailing to 'someone_else#ab.com'
* * * * * /cmd11

how to get current GDG version through jcl

I have a code below to write the properties of the gdg to a PS file.
//STEP10 EXEC PGM=IDCAMS
//DD1 DD DSN='GDGBASE(0)',DISP=SHR
//SYSPRINT DD DSN=GDG.VERSION.PS,
// DISP=(NEW,CATLG,DELETE),
// UNIT=WORK,AVGREC=K,
// SPACE=(108,(5,5),RLSE),
// DCB=(BUFNO=10,RECFM=FB,LRECL=108)
//SYSIN DD *
PRINT INFILE(DD1) COUNT(0) CHAR
/*
But I just need the GDG's latest version name to be written in to the PS file.
I don't think you can do this using pure JCL. As a suggestion to get you started here's a short REXX and some JCL - it's not perfect, but you can tune it to fit your specific needs.
Part 1: a short REXX-script to display the DSN of an allocated file:
/* REXX find DSN for DD */
arg mydd
x= LISTDSI(mydd file)
say sysdsname
Part 2: JCL to invoke it using batch-TSO:
//FINDDSN EXEC PGM=IKJEFT01,PARM='DSNFIND MYFILE'
//MYFILE DD DSN=MY.GDG.FILE(+0),DISP=SHR
//SYSEXEC DD DSN=PDS.CONTAINING.REXX,DISP=SHR
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD DUMMY
where DSNFIND is the membername of the given REXX-procedure.
Output in SYSTSPRT is:
MY.GDG.FILE.G0338V00
READY
END
I haven't found a quick way to suppress the READY and END - but instead of SAY you could e.g. use EXECIO to write to an other file - READY and END would still go to SYSTSPRT.

Broken symlinks and a mysterious (deleted)

I've been doing stuff with the proc filesystem on linux, and I've come across some behavior I'd like to have clarified.
Each process in /proc has a symlink to it's executable file, /proc/{pid}/exe. If a process continues to run after it's executable has been deleted, reading this symlink will return the path to the executable, with (deleted) appended to the end.
Running this command you may even see a few on your system:
grep '(deleted)' <(for dir in $(ls /proc | grep -E '^[0-9]+'); do echo "$dir $(readlink /proc/$dir/exe)"; done)
I tried recreating this behavior with some simple bash commands:
>>> echo "temporary file" >> tmpfile.test
>>> ln -s tmpfile.test tmpfile.link
>>> rm tmpfile.test
>>> readlink tmpfile.link
tmpfile.test
There is no (deleted) appended to the name! Trying a cat tmpfile.link confirms that the link is broken (cat: tmpfile.link: No such file or directory).
However, the other day this same test did result in a (deleted) being appended to the output of readlink. What gives?
Here is what I would like to know:
Is there a sequences of events that guarantees (deleted) will be
appended to the name?
Why does /proc/{pid}/exe show (deleted) for removed executables?
How can I get the name of an executable through /proc/{pid}/exe
without any appended (deleted) and guarantee that the original
executable wasn't just named some_executable (deleted)?
It is not readlink, but Linux changes the symlink to point to <filename> (deleted), i.e., (deleted) gets appended to the target of the link.
FWIW, the special <filename> (deleted) behavior is implemented in the Linux kernel function d_path() here: https://elixir.bootlin.com/linux/v4.1.13/source/fs/dcache.c#L3080.
The source comments (in the snippet below) suggest the special behavior only applies to names (paths) generated on-the-fly for some 'synthetic filesystems' (e.g. procfs) and 'psuedo inodes'.
/**
* d_path - return the path of a dentry
* #path: path to report
* #buf: buffer to return value in
* #buflen: buffer length
*
* Convert a dentry into an ASCII path name. If the entry has been deleted
* the string " (deleted)" is appended. Note that this is ambiguous.
*
* Returns a pointer into the buffer or an error code if the path was
* too long. Note: Callers should use the returned pointer, not the passed
* in buffer, to use the name! The implementation often starts at an offset
* into the buffer, and may leave 0 bytes at the start.
*
* "buflen" should be positive.
*/
char *d_path(const struct path *path, char *buf, int buflen)
{
char *res = buf + buflen;
struct path root;
int error;
/*
* We have various synthetic filesystems that never get mounted. On
* these filesystems dentries are never used for lookup purposes, and
* thus don't need to be hashed. They also don't need a name until a
* user wants to identify the object in /proc/pid/fd/. The little hack
* below allows us to generate a name for these objects on demand:
*
* Some pseudo inodes are mountable. When they are mounted
* path->dentry == path->mnt->mnt_root. In that case don't call d_dname
* and instead have d_path return the mounted path.
*/
if (path->dentry->d_op && path->dentry->d_op->d_dname &&
(!IS_ROOT(path->dentry) || path->dentry != path->mnt->mnt_root))
return path->dentry->d_op->d_dname(path->dentry, buf, buflen);
rcu_read_lock();
get_fs_root_rcu(current->fs, &root);
error = path_with_deleted(path, &root, &res, &buflen);
rcu_read_unlock();
if (error < 0)
res = ERR_PTR(error);
return res;
}

Is there a command to write random garbage bytes into a file?

I am now doing some tests of my application again corrupted files. But I found it is hard to find test files.
So I'm wondering whether there are some existing tools, which can write random/garbage bytes into a file of some format.
Basically, I need this tool to:
It writes random garbage bytes into the file.
It does not need to know the format of the file, just writing random bytes are OK for me.
It is best to write at random positions of the target file.
Batch processing is also a bonus.
Thanks.
The /dev/urandom pseudo-device, along with dd, can do this for you:
dd if=/dev/urandom of=newfile bs=1M count=10
This will create a file newfile of size 10M.
The /dev/random device will often block if there is not sufficient randomness built up, urandom will not block. If you're using the randomness for crypto-grade stuff, you can steer clear of urandom. For anything else, it should be sufficient and most likely faster.
If you want to corrupt just bits of your file (not the whole file), you can simply use the C-style random functions. Just use rnd() to figure out an offset and length n, then use it n times to grab random bytes to overwrite your file with.
The following Perl script shows how this can be done (without having to worry about compiling C code):
use strict;
use warnings;
sub corrupt ($$$$) {
# Get parameters, names should be self-explanatory.
my $filespec = shift;
my $mincount = shift;
my $maxcount = shift;
my $charset = shift;
# Work out position and size of corruption.
my #fstat = stat ($filespec);
my $size = $fstat[7];
my $count = $mincount + int (rand ($maxcount + 1 - $mincount));
my $pos = 0;
if ($count >= $size) {
$count = $size;
} else {
$pos = int (rand ($size - $count));
}
# Output for debugging purposes.
my $last = $pos + $count - 1;
print "'$filespec', $size bytes, corrupting $pos through $last\n";
# Open file, seek to position, corrupt and close.
open (my $fh, "+<$filespec") || die "Can't open $filespec: $!";
seek ($fh, $pos, 0);
while ($count-- > 0) {
my $newval = substr ($charset, int (rand (length ($charset) + 1)), 1);
print $fh $newval;
}
close ($fh);
}
# Test harness.
system ("echo =========="); #DEBUG
system ("cp base-testfile testfile"); #DEBUG
system ("cat testfile"); #DEBUG
system ("echo =========="); #DEBUG
corrupt ("testfile", 8, 16, "ABCDEFGHIJKLMNOPQRSTUVWXYZ ");
system ("echo =========="); #DEBUG
system ("cat testfile"); #DEBUG
system ("echo =========="); #DEBUG
It consists of the corrupt function that you call with a file name, minimum and maximum corruption size and a character set to draw the corruption from. The bit at the bottom is just unit testing code. Below is some sample output where you can see that a section of the file has been corrupted:
==========
this is a file with nothing in it except for lowercase
letters (and spaces and punctuation and newlines).
that will make it easy to detect corruptions from the
test program since the character range there is from
uppercase a through z.
i have to make it big enough so that the random stuff
will work nicely, which is why i am waffling on a bit.
==========
'testfile', 344 bytes, corrupting 122 through 135
==========
this is a file with nothing in it except for lowercase
letters (and spaces and punctuation and newlines).
that will make iFHCGZF VJ GZDYct corruptions from the
test program since the character range there is from
uppercase a through z.
i have to make it big enough so that the random stuff
will work nicely, which is why i am waffling on a bit.
==========
It's tested at a basic level but you may find there are edge error cases which need to be taken care of. Do with it what you will.
Just for completeness, here's another way to do it:
shred -s 10 - > my-file
Writes 10 random bytes to stdout and redirects it to a file. shred is usually used for destroying (safely overwriting) data, but it can be used to create new random files too.
So if you have already have a file that you want to fill with random data, use this:
shred my-existing-file
You could read from /dev/random:
# generate a 50MB file named `random.stuff` filled with random stuff ...
dd if=/dev/random of=random.stuff bs=1000000 count=50
You can specify the size also in a human readable way:
# generate just 2MB ...
dd if=/dev/random of=random.stuff bs=1M count=2
You can also use cat and head. Both are usually installed.
# write 1024 random bytes to my-file-to-override
cat /dev/urandom | head -c 1024 > my-file-to-override

Resources