When you pass "--help" as an argument to a program in command line, generally the program provides standard output designed to help the user. Where is the source of this standard output? Is it within its own file or is it embedded within the object code in a series of print statements?
It's usually within the program itself -- a series of print statements as you mentioned.
Related
I am writing unit tests (using Python 3.7, Pytest 3.6 and tox 3.0) for a function that compiles a series of shell commands as a list of strings, and then executes them using the subprocess module. The shell commands do the following:
creates a filename from the function arguments.
cd into a given directory.
checks for a file with a certain name and removes it if it exist.
creates a new file with the same name.
executes a program and pipes the output into the new file.
My test right now is mocking the subprocess module, and then asserting that it was called with a list of strings that contains all the expected commands given some test arguments.
Is there a way to test that the commands do what they are supposed to? Right now my test is only checking if the list of commands I feed to the subprocess module is the same as the one I have asserted it to be. This does not tell me whether the commands are the right ones for what I am trying to achieve. Rather it only serves as a test on whether I can write down the same string in two different source files.
Can I simulate the side effects that I expect the shell commands to have?
Your question combines two interesting topics: a) Testing code generated from a code generator and b) testing shell code.
For testing code from a generator, you have in principle to do the following: i) test that the generator creates the code that is expected - which you have already done, ii) test that the code snippets / pieces which the generator glues together actually behave (independently and in combination) as it is intended (which in your case are the shell code pieces that in the end together will form a valid shell program) - this is the part about testing shell code that will be adressed below, and iii) test that the inputs that control the generator are correct.
It is comparable with a compiler: i) is the compiler code, ii) are assembly code snippets that the compiler combines to get the resulting assembly program, and iii) is the source code that is given to the compiler to get it compiled. Once i), ii) and iii) are tested, there is only seldom the need to also test the assembly code (that is, on assembly code level). In particular, the source code iii) is ideally tested by test frameworks in the same programming language.
In your case it is not so clear how part iii) looks and how it can be tested, though.
Regarding the testing of shell code / shell code snippets: Shell code is dominated by interactions with other executables or the operating system. The type of problems that lies in interactions in shell code goes in the direction of, am I calling the right executables in the right order with the arguments in the right order with properly formatted argument values, and are the outputs in the form I expect them to be etc. To test all this, you should not apply unit-testing, but integration testing instead.
In your case this means that the shell code snippets should be integration-tested on the different target operating systems (that is, not isolated from the operating system). And, the various ways in which the generator puts these snippets together should also be integration tested to see if they together operate nicely on the operating system.
However, there can be shell code that is suitable for unit-testing. This is, for example, code performing computations within the shell, or string manipulations. I would even consider shell code with calls to certain fundamental tools like basename as suitable for unit-testing (interpreting such tools as being part of the 'standard library' if you like). In your case as you describe the generated shell code
creates a filename from the function arguments.
This sounds like one example of a good candidate for 'unit-testing' shell code: This filename creation functionality could be put into a shell function and then be tested in isolation.
With pytest-mock you can request the mocker fixture and then spy on subprocess functions:
def test_xxx(mocker):
mocker.spy(subprocess, 'call')
subprocess.call(...)
assert subprocess.call.call_count == 1
P.S. Tests with side effects are generally bad practice, so I would recommend running all the shell commands in a tmpdir (pytest fixture which creates a temporary directory).
I use FORTRAN gnu compiler to compile a piece of code written using fortran(.f90). Unlike in other compilers the output of write statement are not displayed in the screen rather written in the output file.
For example I have placed "write(*,*) 'Check it here'" in the middle of the source code so that this message is displayed in the screen when someone runs the compiled version of the code.
I dont understand why this message is not displayed in the terminal window while running the code, but it is written in the output file.
I would appreciate your help to resolve this !!
>
I am compiling these source codes:
https://github.com/firemodels/fds/tree/master/Source
makefile that I am using to compile the code is located here:
https://github.com/firemodels/fds/tree/master/Build/mpi_intel_linux_64
I run the program using a executable that makefile creates
The version of the compiler that I am using is
GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.5) 5.4.0 20160609
>
Thank you.
Way bigger picture: Is there a reason you're building FDS from source rather than downloading binaries directly from NIST i.e. from https://pages.nist.gov/fds-smv/downloads.html ?
Granted, if you're qualifying the code for safety-related use, you may need to compile from source rather than use someone else's binaries. You may need to add specific info to a header page such as code version, date of run, etc. to satisfy QA requirements.
If you're just learning about FDS (practicing fire analysis, learning about CFD, evaluating the code), I'd strongly suggest using NIST's binaries. If you need/want to compile it from source, we'll need more info to diagnose the problem.
That said, operating on the assumption that you have a use case that requires that you build the code, your specific problem seems to be that writing to the default output unit * isn't putting the output where you expect.
Modern Fortran provides the iso_fortran_env module which formalizes a lot of the obscure trivia of Fortran, in this case, default input and output units.
In the module you're editing, look for something like:
use iso_fortran_env
or
use iso_fortran_env, only: output_unit
or
use, intrinsic:: iso_fortran_env, only: STDOUT => output_unit
If you see an import of output_unit or (as in the last case) an alias to it, write to that unit instead of to *.
If you don't an import from iso_fortran_env, add the last line above to the routine or module you're printing from and write to STDOUT instead of *.
That may or may not fix things, depending on if the FDS authors do something strange to redirect IO. They might; I'm not sure how writing to screen works in an MPI environment where the code may run in parallel on a number of networked machines (I'd write to a networked logging system in that case, but that's just me). But in a simple case of a single instance of the code running, writing to output_unit is more precise than writing to * and more portable and legible than writing to 6.
Good luck with FDS; I tried using it briefly to model layer formation from a plume of hydrogen gas in air. FDS brought my poor 8 CPU machine to its knees so I went back to estimating it by hand instead of trying to make CFD work...
Quick backstory: I'm a graduate student, and I know very little (read: almost nothing) about batch files. A collaborator at another university came to me and told me to create a batch file for a supercomputer which runs on a Linux system. After googling, it looks like a batch file is technically only for Windows systems, and the Linux equivalent is a "shell script". I talked to my collaborator about this, and he's insistent that it should be a batch file, not a shell script, even though it's a Linux system.
Is there something I'm missing here, or is there some way to make a batch file for Linux? There is a language barrier, so I wonder if that's part of the problem. Thanks, and sorry for such an elementary question.
The term "batch" may refer to two different things, maybe that is the issue here:
the MS-Windows batch command processor, some very primitive and limited, non-interactive shell environment
"batch processing" which simply means non-interactive processing of work items or jobs
You are right that the (very) rough equivalent to an MS-Windows batch script is a shell script in unixoid systems (so Linux too). However it should be pointed out that there are many very different types of shell environments you can use, so you have a huge flexibility here.
Considering the two alternative meanings above I could imagine that what is meant is "a script that does batch processing". Usually it is of less importance which specific type of language is chosen for that.
The *nix equivalent would be a shell script. For example, Ubuntu's default shell is bash.
BASH Programming Introduction might prove a worthy read.
As another answer already noted, scripts of command line commands are called "batch files" on Windows, but the term also means non-interactive processing in general.
Since the context was some sort of a supercomputer, there's a possibility that a "batch file" means something specific to the system, not just a generic shell script/program.
Either that, or they are just very insistent on using uncommon terminology.
I am trying to write a script where it will cross check to things:
The architecture for which the setup file was intended (32 or 64 bit)
The Architecture of the system.
The second part is quite easy and can be figured out using commands like lscpu and then extracting that specific line using combination of grep and awk or sed. However the first part is proving out to be a complicated one. I tried using the file command but it has a very irregular output. Hence it becomes very difficult extracting a specific column from it. I also tried using objdump though traditionally not used for things like this. However as expected, due to its limitations, it does not recognize most of the file types.
The rest part of the script is dead simple where I would be comparing these values and proceeding with my intended tasks. I would like your help with the Point 1 mentioned above.
I recently got into Stata coming from a procedural/OO/functional background, and am having trouble understanding the basic elements of the language.
For example, I discovered that there is a syntax command which "allows programs to interpret the arguments the user types according to a grammar, such as standard Stata syntax". I infer this is the reason why some command require a list of variables given as arguments to be separated by whitespaces while others require a comma-separated list. But the idea of a program defining its own syntax instead of the (parameter) syntax being enforced seems plain weird.
Another quite interesting construct is the syntax for macro definition and expansion (`macro') and the apparent absence of local variables as known in other languages.
Is there something like a "Stata for Java developers" document explaining the basic concepts of the language to people with my background?
PS: Apologies if this question seems unclear. Unfortunately, I can't formulate more concrete/clear questions at this point :(
I'm not exactly sure what you are looking for... but here's a few related points. Stata is kind of like writing a Unix shell script or a Windows batch file. Each line executes a command, and the first word is the command name. By convention, most commands have the following structure:
command [varlist] [=exp] [if expression] [in range] [weight] [using filename] [, options]
Brackets [.] means it's optional (or unavailable, depending on the command). Some commands can be prefixed (such as by:, xi:, or svy:) The syntax of commands by Stata Corp and experienced users are pretty consistent. But, because Stata users also write commands, you occasionally see things that are wacky.
When Stata users write commands, they are saved in .ado files (not .do) and are defined using the program command. (See help program and the "Ado files" section of the manual.) Writing a command is akin to writing a function in other languages (e.g., MatLab)
The syntax command is used to help you write your own command. When you execute a command, everything following the command's name (command above) is passed to the program in the local macro `0'. The syntax command parses this local macro, so that you can reference `varlist' or `if' and so on. In theory, you could parse `0' yourself, but the syntax command makes it much easier for you and your users (as long as you are following the conventional syntax). I put an example at the bottom.
I don't know exactly what you mean by "apparent absence of local variables as known in other languages." Macros store a single string or a single number in memory. Here's a comment I wrote about Stata's local/global macros. They are indeed a unique feature of Stata's programming language. As their names imply, "local" macros are only available within a specify program (command) or .do file while "global" macros are available throughout a Stata session.
I found that, once I got used to macros in Stata, I started to miss them in other languages. They are pretty handy. In addition to (local/global) macros and the main data set, you can also store "things" in memory with the scalar and matrix commands (and one or two other obscure things).
I hope that helps. Here's a list resources that might help.
Example:
program define myprogram
syntax varlist [if], [hello(string) yes]
macro list _0 _varlist _if _hello _yes
summarize `varlist' `if'
display "Here's the string in my hello option: `hello'"
if !missing("`yes'") di "Yes is on"
else di "Yes is off"
end
sysuse auto.dta
myprogram rep78 headroom if price > 5000 , hello("world") yes
A few books offer an "X for Y users" approach, but generally between stats software solutions. Regarding your question, I would recommend using instinct first.
I started reading (programming and markup) code about ten years ago, and even though I cannot code in a large number of languages, I can read a few languages rather easily. I found Stata easy because most of its core commands are straightforward, with recurrent optional statements like over, if or replace (to take a voluntarily diverse set of statements) that are easy to understand and then apply.
When I teach Stata, I always have problems getting students to use the help pages as much as I do (and I love the fact they can be accessed so easily, just like in R). I explain the paradox by considering the fact that I can read the syntax indications straightaway. Syntax is very well covered by the previous reply to your question.
The extra mile consists in opening the [R], [U] and especially [P] handbooks that come with Stata in the utilities folder. There is a wealth of details there, which will interest both programmers and training statisticians. This is where I learnt to use macros and loops, beyond the obvious logic of commands like local/global and foreach/while (if I understand the term correctly, Stata is Turing-complete).
Stata is sometimes a bit of a pain when it comes to using single/double quotes in macro loops, but it's pretty straightforward otherwise. Have fun!