How can I "debug" JCL? Is there any way to show error or caution messages? When I submit my JOB the output is MAXCC=8. But how can I get why or must I search for where the "error" is?
You don't actually have a JCL problem.
Firstly, to deal with the MAXCC, since that causes a lot of confusion.
For experienced people, MAXCC only exists in IDCAMS (a utility for file and catalog manipulation, defining VSAM files and stuff associated with those types of things).
MAXCC does not exist in JCL, or in any output you will receive from your JOB.
Where the MAXCC comes from is the NOTIFY statement on your JOB card. We've seen the notify arrive so many times, we never read it, but it does indeed show MAXCC. It's just that MAXCC only occurs in IDCAMS and from the NOTIFY, so telling us what it was will confuse many.
The CC bit is the Condition Code. Each jobstep will produce a Condition Code. Usually the CC will be 00, nothing to report, 04, some situation, not necessarily serious, 08, some situation probably serious, 12, some serious situation and 16, something really bad.
Those are only conventional uses, but most utilities from IBM and Independent Software Vendors (ISVs) stick to the convention. You'll occasionally see a 20, perhaps. Probably during an Earthquake.
For instance, if you compile a COBOL program, 00 will mean the compiler spotted no problems. This does not mean your program works. It just means the syntax is OK. 04 means there are "warnings". You should check those always, as some warnings are actually problematic. 08 are serious errors and although object-code is generated (unless not requested through compiler option), it is foolish to try to execute it (except for fun). 12 and 16, the compiler has just given up and will not generate code.
There is a similar hierarchy with the linkeditor/binder. 00 is all ready to run (but no guarantee it will do anything approaching what you want), 04 is a small problem, but you'll want to know what, 08 is a problem which probably means you won't be able to execute the program.
The short of all this is that if you have MAXCC=08 as the result of your NOTIFY then you have at least one step in your JOB which has a CC of 08. You may have more than one step with 08, and you may also have step(s) with 00 or 04.
This is nothing to do with JCL. The CC has been determined by the program you are running, and what the CC means in each case can differ in the specifics (compiler to linkeditor/binder for instance) although the grading will generally be about equivalent.
You can also set the CC from an application program. In COBOL you would use the RETURN-CODE "special register".
MOVE 04 TO RETURN-CODE
When executed, would, assuming RETURN-CODE receives no other value later, get you a CC of 04 from the execution of that program.
You local site standards will dictate how applications indicate errors: it is quite possible you'll never use RETURN-CODE.
The further short is that you have a program error, and you need to do what piet.t has suggested.
You will likely have error messages associated with the step which produces a CC. A quick and easy way to find out more about the message (apart from COBOL compiler messages, which are self-explanatory(!)) is to use the IBM LookAt web-service: http://www-03.ibm.com/systems/z/os/zos/bkserv/lookat/
Paste your message in, click on the GO button. Sometimes you'll need to specify the z/OS version. You can do it always as a beginner.
For debugging JCL itself, to the extent you can, you can use TYPRUN=SCAN on your JOB card, or perhaps your site has a JCL-checker product, although these tend to get a lot of false positives as they are updated years behind for where they try to understand the impact of control cards to standard utilities (TYPRUN=SCAN ignores control cards).
The real test of whether your JCL actually works is to toss it in and run it, creating the conditions which actually test what you have coded. No product can check that the relationships you create in the JCL (through conditional checking of the... CC, it's back again) are correct.
You can't "debug" JCL since there is hardly anything in the JCL that is actually executed - it is mainly for starting a sequence of programs and define their respective environments. The condition-codes result from the programs executed, so to get to the source of your error:
check your joblog which step produces the condition-code 8 - this might depend on your JES-version, in our environment each step produces an IEF142I-message that shows the respective CC.
identify the program that is executed in this step
check the programs output for any error-messages
First of all, you should know return code's means.
knows these,after you can learn how to fix JCL's error(not debug)
if you find error by return code,first,you should check your syntax.
job name's place,add typrun=scan,you can check your syntax.
if TYPRUN=SCAN not return error message,you can see joblog in SDSF.
press S to see detailed information.
because JCL including a or lots steps,you must check which step occur error.ensure which step,you can see detailed information.
in command line input M to reach bottom.
it has a lots of error in area,it shows detailed information of error.
How can I "debug" JCL?
JCL is not easy. There are many, many messages and codes manuals to take a look at for this all encompassing question. Essentially you should do your best to be very familiar with fundamentals. Always be well grounded in the fundamentals of the operating system - z/OS - that you write your JCL in order to control.
Is there any way to show error or caution messages?
Yes indeed. They are ALL over your output - from your operating system to your program to even the messages you yourself can create for debugging purposes.
When I submit my JOB the output is MAXCC=8.
This particular message is obviously from the utility called IDCAMS. That's actually a good thing that you understood that that is the place to begin your troubleshooting. In time you will get better. No need to panic.
But how can I get why or must I search for where the "error" is?
Fundamentals. Always fundamentals. Since this is IDCAMS realize that along with IEFBR14, they NEVER, EVER FAIL once they begin. Now that you know that realize that by never failing they will absolutely NEVER display an ERROR message. An ERROR message is only if it fails. Therefore you will only get return codes. Those return codes from IDCAMS are usually fairly clear. Just do your best to get used to them.
Good luck.
Related
Is there any good reason not to run a brief unknown (30 line) assembly script inline in a usermode c program for dynamic analysis directly on my laptop?
There's only one system call to time, and at this point I can tell that it's a function that takes a c string and it's length, and performs some sort of encryption on it in a loop, which only iterates through the string as long as the length argument tells it.
I know that the script is (supposed to be) from a piece of malicious code, but for the life of me I can't think of any way it could possibly pwn my computer barring some sort of hardware bug (which seems unlikely given that the loop is ~ 7 instructions long and the strangest instruction in the whole script is a shr).
I know it sounds bad running an unknown piece of assembly code directly on the metal, but given my analysis up to this point I can't think of any way it could bite me or escape.
Yes, you can but I won't recommend it.
The problem is not how dangerous is the code this time (assuming you really understand all of the code and you can predict the outcome of any system call), the problem is that it's a slippery slope and it's not worth it considering what's at stake.
I've done quite a few malware analysis and rarely happened that a piece of code caught me off guard but it happened.
Luckily I was working on a virtual machine within an isolated network: I just restored the last snapshot and stepped through the code more carefully.
If you do this analysis on your real machine you may take the habit and one day this will bite you back.
Working with VMs, albeit not as comfortable as using your OS native GUI, is the way to go.
What could go wrong with running a 7 lines assembly snippet?
I don't know, it really depends on the code but a few things to be careful about:
Exceptions. An instruction may intentionally fault to pass the control to an exception handler. This is why it very important that you totally understand the code: both the instruction and the data.
System calls exploits. A specially crafted input to a system call may trigger a 0-day or an unpatched vulnerability in your system. This is why is important that you can predict the outcome of every system call.
Anti debugger techniques. There are a lot of way a piece of code could escape a debugger (I'm thinking Windows debugging here), it's hard to remember them all, be suspicious of everything.
I've just named a few, it's catastrophically possible that an hardware bug could lead to privileged code execution but if that's really a possibility then nothing but a spare sacrificable machine will do.
Finally, if you are going to run the malware (because I assume the work of extracting the code and its context is too much of a burden) up to a breakpoint on your machine, think of what's at stake.
If you place the break point on the wrong spot, if the malware takes another path or if the debugger has a glitchy GUI, you may loose your data or the confidentiality of your machine.
I'n my opinion is not worth it.
I had to make this premise for generality sake but we all sin something, don't we?
I've never run a piece of malware on my machine but I've stepped through some with a virtual machine directly connected on the company network.
It was a controlled move, nothing happened, the competent personnel was advised and it was an happy ending.
This may very well be your case: it can just be a decryption algorithm and nothing more.
However, only you have the final responsibility to judge if it is acceptable to run the piece of code or not.
As I remarked above, in general it is not a good idea and it presupposes that you really understand the code (something that is hard to do and be honest about).
If you think these prerequisites are all satisfied then go ahead and do it.
Before that I would:
Create an unprivileged user and deny it access to my data and common folders (ideally deny it everything but what's is necessary to make the program work).
Backup the the critical data, if any.
Optionally
Make a restore point.
Take an hash of the system folders, a list of installed services and the value of the usual startup registry keys (Sysinternals have a tool to enum them all).
After the analysis, you can check that nothing important system-wide has changed.
It may be helpful to subst a folder and put the malware there so that a dummy path traversal stops in that folder.
Isn't there better solution?
I like using VMs for their snapshotting capabilities, though you may stumble into an anti-VM check (but they are really dumb checks, so it's easy to skip them).
For a 7-line assembly I'd simply rewrite it as a JS function and run it directly in a browser console.
You can simply transform each register in a variable and transcript the code, you don't need to understand it globally but only locally (i.e. each instruction).
JS is handy if you don't have to work with 64-bit quantities because you have an interpreter in front of you right now :)
Alternatively I use any programming language I have at hand (One time even assembly it self, it seems paradoxical but due to a nasty trick I had to convert a 64-bit piece of code to a 32-bit one and patch the malware with it).
You can use Unicorn to easily emulate a CPU (if the architecture is supported) and play with your shellcode without any risk.
Question
What exactly does the link-edit step in my COBOL complier do?
After the code is compiled, there is a link edit step performed. I am not really sure what this step does.
Background Information
Right out of school (3 years ago) I got a job as a mainframe application developer. Having learned nothing about mainframes in school, my knowledge has many gaps. Around my shop, we kind of have a "black box" attitude of we don't need to know how a lot of this stuff works, it just does. I am trying to understand why we need this link-edit step if the program as already compiled successfully.
The linkedit/binderer step makes an executable program out of the output from the compiler (or the Assembler).
If you look at the output data set on SYSLIN from your COBOL compile step (if it is a temporary dataset, you can override it to an FB, LRECL 80 sequential dataset to be able to look at it) you'll see "card images", which contain (amongst some other stuff) the machine-code generated by the compiler.
These card-images are not executable. The code is not even contiguous, and many things like necessary runtime modules are missing.
The Program Binder/Binder (PGM=HEWL) takes the object code (card-images) from the compiler/assembler and does everything necessary (according to the options it was installed with, and further options you provide, and other libraries which many contain object-code or loadmodules or Program Objects) to create an executable program.
There used to be a thing called the Linkage Editor which accomplished this task. Hence linkedit, linkedited. Unfortunately, in English, bind does not conjugate in the same way as edit. There's no good word, so I use Binderer, and Bindered, partly to rail against the establishment which decided to call it the Program Binder (not to be so confused with Binding for DB2, either).
So, today, by linkedit people mean "use of the Program Binder". It is a process to make the output from your compile/assemble into an executable program, which can be a loadmodule, or a Program Object (Enterprise COBOL V5+ can only be bindered into Program Objects, not loadmodules), or a DLL (not to be confused with .dll).
It is worth looking at the output of SYSLIN, the SYSPRINT output from the binder step, and consulting manuals/presentations of the Program Binder which will give you an idea of what goes in, what happens (look up any IEW messages, especially for non-zero-RC executions of the step) by sticking the message in a browser search box. From the documentary material you'll start to get an idea of the breadth of the subject also. The Binder is able to do many useful things.
Here's a link to a useful diagram, some more detailed explanation, and the name of the main reference document for the binder for application programmes: z/OS MVS Program Management: User's Guide and Reference
The program management binder
As an end-note, the reason they are "card images" is because... back in olden times, the object deck from compiler/assembler would be punched onto physical cards. Which would then be used as input cards to the linkage editor. I'm not sorry that I missed out on having to do that ever...
In addition to Bill's (great) answer, I think it is worth to also mention the related topics below ...
Static versus dynamic linking
If a (main) program 'calls' a subprogram, then you can either have such call to happen 'dynamic' or 'static':
dynamic: at run-time (when the main program is executed), the then current content of the subprogram is loaded and performed.
static: at link-time (when the mail program is(re-)linked), the then current content of the subprogram is included (= resolved) in the main program.
Link-edit control cards
The actual creation of the load module (output of the link-edit step) can be controlled by special directives for the link-editor, such as:
Entry points to be created.
Name of the load module to be created.
Includes (of statically linked subprograms) to be performed.
Alias-members to be created.
Storing the link-edit output in PDS or PDSE
The actual output (load module) can be stored in members located in either PDS or PDSE libraries. In doing so, you need to think ahead a bit about which format (PDS or PDSE) best fits your requirements, especially when it comes to concatenating multiple libraries (e.g a preprod environment for testing purposes).
I'm attempting to test an application which has a heavy dependency on the time of day. I would like to have the ability to execute the program as if it was running in normal time (not accelerated) but on arbitrary date/time periods.
My first thought was to abstract the time retrieval function calls with my own library calls which would allow me to alter the behaviour for testing but I wondered whether it would be possible without adding conditional logic to my code base or building a test variant of the binary.
What I'm really looking for is some kind of localised time domain, is this possible with a container (like Docker) or using LD_PRELOAD to intercept the calls?
I also saw a patch that enabled time to be disconnected from the system time using unshare(COL_TIME) but it doesn't look like this got in.
It seems like a problem that must have be solved numerous times before, anyone willing to share their solution(s)?
Thanks
AJ
Whilst alternative solutions and tricks are great, I think you're severely overcomplicating a simple problem. It's completely common and acceptable to include certain command-line switches in a program for testing/evaluation purposes. I would simply include a command line switch like this that accepts an ISO timestamp:
./myprogram --debug-override-time=2014-01-01Z12:34:56
Then at startup, if set, subtract it from the current system time, and indeed make a local apptime() function which corrects the output of regular system for this, and call that everywhere in your code instead.
The big advantage of this is that anyone can reproduce your testing results, without a big readup on custom linux tricks, so also an external testing team or a future co-developer who's good at coding but not at runtime tricks. When (unit) testing, that's a major advantage to be able to just call your code with a simple switch and be able to test the results for equality to a sample set.
You don't even have to document it, lots of production tools in enterprise-grade products have hidden command line switches for this kind of behaviour that the 'general public' need not know about.
There are several ways to query the time on Linux. Read time(7); I know at least time(2), gettimeofday(2), clock_gettime(2).
So you could use LD_PRELOAD tricks to redefine each of these to e.g. substract from the seconds part (not the micro-second or nano-second part) a fixed amount of seconds, given e.g. by some environment variable. See this example as a starting point.
I run the time-consuming code in the background and oftentimes even the 10% of the code is not executed due to a small syntax mistake in the beginning of the do-file.
I would prefer the rest of the do-file to be executed as well since sometimes that mistake in the beginning has no effect on the computations at the end.
(version 2)
Wanting Stata to ignore mistakes can itself be mistaken.
If there is a mistake early in a do-file, it usually has implications for what follows.
Suppose you got Stata to work as you wish. How do you know whether Stata ignored something important or something trivial? If it ignored something trivial, that should be easy to fix. If it ignored something important, that was the wrong decision.
Let's now be more constructive. The help for do tells you that there is a nostop option.
You need to be very careful about how you use it, but it can help here.
The context of do, nostop is precisely that of the OP. People had do-files, often expected to take a long time because of big datasets or lots of heavy computation, and set them going, historically "overnight" or "while you went to lunch". Then they would be irritated to find that the do-file had quickly terminated on the first error, and especially irritated if the error was trivial. So, the idea of do, nostop is to do as much as you can, but as an aid to debugging. For example, suppose you got a variable name wrong in various places; you generate Y but refer to y later, which doesn't exist. You might expect to find corresponding error messages scattered through the file, which you can fix. The error messages are the key here.
The point about do files is that once they are right, you can save yourself lots of time, but no-one ever promised that getting the do-file right in the first place would always be easy.
My firm advice is: Fix the bugs; don't try to ignore them.
P.S. capture was mentioned in another answer. capture may seem similar in spirit, but used in similar style it can be a bad idea.
capture eats error messages, so the user doesn't see them. For debugging, that is the opposite of what is needed.
capture is really a programmer's command, and its uses are where the programmer is being smart on behalf of the user and keeping quiet about it.
Suppose, for example, a variable supplied could be numeric or it could be string. If it's numeric, we need to do A; if it's string we need to do B. (Either A or B could be "nothing".) There could be branches like this.
capture confirm str variable myvar
if _rc { /// it's numeric
<do A>
}
else {
<do B>
}
Otherwise put, capture is for handling predictable problems if and as they arise. It is not for ignoring bugs.
If there are only a couple commands that are hanging up and are otherwise inconsequential to your later calculations, you can always use capture (either as an inline prefix or as a block command, see help capture on use) to force the program to run through commands that stop the program.
But--echoing Nick's general comments about this way of writing and executing do-files--be similarly careful where you apply capture: generally, you should only apply it to commands you are sure would not affect later code or calculations. Or, even better, just remove from the program those lines that are giving problems and you apparently don't need anyway.
On one hand the advice to always close objects is so common that I would feel foolish to ignore it (e.g. VBScript Out Of Memory Error).
However it would be equally foolish to ignore the wisdom of Eric Lippert, who appears to disagree: http://blogs.msdn.com/b/ericlippert/archive/2004/04/28/when-are-you-required-to-set-objects-to-nothing.aspx
I've worked to fix a number of web apps with OOM errors in classic asp. My first (time consuming) task is always to search the code for unclosed objects, and objects not set to nothing.
But I've never been 100% convinced that this has helped. (That said, I have found it hard to pinpoint exactly what DOES help...)
This post by Eric is talking about standalone VBScript files, not classic ASP written in VBScript. See the comments, then Eric's own comment:
Re: ASP -- excellent point, and one that I had not considered. In ASP it is sometimes very difficult to know where you are and what scope you're in.
So from this I can say that everything he wrote isn't relevant for classic ASP i.e. you should always Set everything to Nothing.
As for memory issues, I think that assigning objects (or arrays) to global scope like Session or Application is the main reason for such problems. That's the first thing I would look for and rewrite to hold only single identifider in Session then use database to manage the data.
Basically by setting a COM object to Nothing, you are forcing its terminator to run deterministically, which gives you the opportunity to handle any errors it may raise.
If you don't do it, you can get into a situation like the following:
Your code raises an error
The error isn't handled in your code and therefore ...
other objects instantiated in your code go out of scope, and their terminators run
one of the terminators raises an error
and the error that is propagated is the one from the terminator going out of scope, masking the original error.
I do remember from the dark and distant past that it was specifically recommended to close ADO objects. I'm not sure if this was because of a bug in ADO objects, or simply for the above reason (which applies more generally to any objects that can raise errors in their terminators).
And this recommendation is often repeated, though often without any credible reason. ("While ASP should automatically close and free up all object instantiations, it is always a good idea to explicitly close and free up object references yourself").
It's worth noting that in the article, he's not saying you should never worry about setting objects to nothing - just that it should not be the default behaviour for every object in every script.
Though I do suspect he's a little too quick to dismiss the "I saw this elsewhere" method of coding behaviour, I'm willing to bet that there is a reason Eric didn't consider that has caused this to be passed along as a hard 'n' fast rule - dealing with junior programmers.
When you start looking more closely at the Dreyfus model of skill acquisition, you see that at the beginning levels of acquiring a new skill, learners need simple to follow recipes. They do not yet have the knowledge or ability to make the judgement calls Eric qualifies the recommendation with later on.
Think back to when you first started programming. Could you readily judge if you were "set[tting an] expensive objects to Nothing when you are done with them if you are done with them well before they go out of scope"? Did you really know which objects were expensive or when they truly went out of scope?
Thus, most entry level programmers are simply told "always set every object to Nothing when you are done with it" because it is within their grasp to understand and follow. Unfortunately, not many programmers take the time to self-educate, learn, and grow into the higher-level Dreyfus stages where you can use the more nuanced situational approach.
And then we come back to my earlier statement - even the best of us started out at that earlier stage, where we reflexively closed all objects because that was the best we were capable of. We left large bodies of code that people look at now, and project our current competence backwards to the earlier work and assume we did that for reasons we don't understand.
I've got to get going, but I hope to expand this a little futher...