Why would this parameter be ignored if provided in Python StreamReader? - python-3.x

In python, I am reading a file in order to process it.
My goal is to read X lines of that file at once, process the batch of lines and repeat until there is no line left to process in the file.
The StreamReader I am using is of type: <class 'encodings.cp437.StreamReader'>
It is the result of:
with hdfs.read("myfile", encoding="cp437") as reader:
where hdfs is InsecureClient from there
I found this documentation about the cp437 encoded StreamReader readlines.
The documentation says:
sizehint, if given, is ignored since there is no efficient way to
finding the true end-of-line.
What does that actually means ? I do not understand why you would allow a parameter that will be ignored.
What is it even supposed to do ?

According to the documentation it should work as expected, the documentation link you provided is incorrect.
Correct link:https://docs.python.org/3/library/codecs.html#codecs.StreamReader.readlines.
Quoting:
readlines([sizehint[, keepends]])¶ Read all lines
available on the input stream and return them as a list of lines.
Line-endings are implemented using the codec’s decode() method and are included in the
list entries if keepends is true. sizehint,
if given, is passed as the size argument to the stream’s read()
method.

Related

Is there a way to read a command line dictionary or json arguments in python

I want to read the command line argument as json format, or the key:value pair and I want to use the Key as variable name in the program and value is equal to variable value, for example, if user submitted, rank:1, than my program will use rank as variable name and number 1 as its value, that is rank = 1 in program.
Could someone please help me to achieve this using multiple key:value pairs as command line arguments.
Also is there a way to change the field separator from space to comma or anything else?
Please take a look at argparse library: it should help you to achieve your goal by declaring keywords as command line arguments.
This library is very useful and offers a lot of possibilities.

Save responses to a file in JMeter with PostProcessor

I try to use following code in JSR223 PostProcessor to save response to CSV file in each iteration.
if(${__groovy(ctx.getPreviousResult().getResponseCode(),)} == "200"){
vars.put("response", prev.getResponseDataAsString());
String res="${response}";
FileWriter fstream = new FileWriter("logresult.csv",true);
fstream.write(res+"\n");
fstream.close();
}
I assume csv file should be created in same folder as jmx, but it isn't present after execution. I execute test in one thread.
Don't use ${} syntax in JSR223 script as part of JMeter best practices
if(prev.getResponseCode() == "200"){
String res=prev.getResponseDataAsString();
vars.put("response", res);
FileWriter fstream = new FileWriter("logresult.csv",true);
fstream.write(res+"\n");
fstream.close();
}
ensure the script does not use any variable using ${varName} as caching would take only first value of ${varName}. Instead use :
vars.get("varName")
Your syntax is not correct, you need to amend the first line to look like:
if(prev.getResponseCode().equals('200')){
also you don't need this line at all
String res="${response}";
instead use the following:
fstream.write(vars.get('response')+"\n");
see JSR223 Sampler documentation for comprehensive explanation, the relevant quote:
JMeter processes function and variable references before passing the script field to the interpreter, so the references will only be resolved once. Variable and function references in script files will be passed verbatim to the interpreter, which is likely to cause a syntax error.
In general you're going into wrong direction, in case if you run your test with > 1 thread you might run into the race condition when 2 threads will be writing into the same file resulting in garbage if not worse.
If you have to store the responses into a separate file I would recommend:
Extracting the whole response into a JMeter Variable using i.e. Regular Expression Extractor
Defining Sample Variable property to hold this value
Write the value into the file using Flexible File Writer

Is there a way to compare the format in which a line of a text file was written in Fortran?

I'm developing a Fortran program that must obtain some data from a text file and generate another text file using specific data from the first one.
The input file have many lines written in several specific formats which I know of. Although I know the formats, the lines in this file are generated in a "random way".
It would be much easier to generate the output file if I could compare the format in which each line was written, then I would know exactly what data I can get from that line of the input file to use it in the output file.
What I need is something like, for example, knowing that the format of the line read and stored in the LINHA variable is described in the FORMATO variable, do something like:
    
IF (FORMATO = '(1X, 15,3F8.1,2 (5A, 1X))') THEN
READ (LINHA, '(6X, F8.1)') my_variable
END IF
Because there might be another format such as
'(6A, 2F8.1, F8.6,2 (6A))'
in which, if I use the same READ statement, I will read an F8.1 variable in my_variable, however this value is not the correct one.
A (not so elegant) work-around that I can think of is to read the entire line using the advance = no option of read() and parse each character in the line separately. While doing so, you may count white spaces or other specific characters that you know of and then identify the different formats from there.
It would be helpful if you could give more specifications of the nature of the task.
The best option is to read without format, keeping each line in a character array. Then read the line variable as an internal file with the required format using the variable IOSTAT in order to check if the format is the correct.
INT max_size = 80
CHARACTER(LEN=max_size) :: line
READ(*,*) line
READ(line,'(1X, 15,3F8.1,2 (5A, 1X))',IOSTAT=ios) var1, var2, ...
Problem solved using a mixture of some of the suggestions posted.
I read each of the lines of the input file in an internal variable (RLINFILE) in the format '(A165)'. After that, I read all the contents of the string that I put in this internal variable in several dummy variables, using the format I knew of the lines from where I wanted to get some information (read all the information of the line in the desired and get IOSTAT = 0 guarantee that this is the correct line), so if the result of the reading is ok (IOSTAT = 0), it is because the line I just read was the correct one for the information I wanted, so I store the contents of some of the dummy variables that represent the values that interest me. In the code, the solution looked something like this:
OPEN(UNIT=LU1,FILE=RlinName,STATUS='OLD')
ilin = 0
formato = '(14X,A,1X,F7.1,1X,F7.1,5X,A,1X,A,1X,A,5X,A,I5,1X,A,I3,3F8.1,A,A,A,1X,A,2(1X,F8.2),1X,A,1X,A)'
DO WHILE (.TRUE.)
READ(LU1,'(A165)',END=300) RLINFILE
READ(RLINFILE,formato,IOSTAT=linhaok) dum2_a1,dum2_f1,dum2_f2,dum2_a2,dum2_a3,dum2_a4,dum2_a5,dum2_i1,dum2_a6,dum2_i2,dum2_f3,dum2_f4,dum2_f5,dum2_a7,dum2_a8,dum2_a9,dum2_a10,dum2_f6,dum2_f7,dum2_a11,dum2_a12
IF(linhaok.EQ.0) THEN
ilin = ilin+1
rlin_lshu(ilin) = dum2_a4
rlin_nbpa(ilin) = dum2_i1
rlin_ncir(ilin) = dum2_i2
rlin_ppij(ilin) = dum2_f3
rlin_pqij(ilin) = dum2_f4
rlin_tapn(ilin) = dum2_a7
END IF
END DO
300 CLOSE(UNIT=LU1)
The description of the problem you are trying to solve is a bit vague to me, but the simplest solutions that comes to my mind, given the description of the problem, is to modify the original code that generates the input data file, to write the used Fortran READ format before the data line in the input file. This way, you can read the format as a string and use it in the subsequent data IO in your second code.
If you describe the specific task your tryting to accomplish in more details, perhaps more experienced Fortranners could help.

Migrate from Python2 to Python 3.6.2 with TypeError

I want to migrate from python2 to python3.
In code, it opens a file in binary mode like this.
f = open('test', 'rb+')
Because, it needs file seeking like this.
f.seek(-26, 1)
And, the code writes a formatted string to file.
f.writelines("%20s,%04d\n" % (varStr, varInt))
f.writelines('{0:>20s},{1:04d}\n'.format(varStr, varInt))
Upper code is original code and lower code is edited by me.
But, When I run a code, there is an error.
TypeError: a bytes like object is required, not 'int'
Please, help me out from this hell.
There are two mistakes:
You need encode your strings when opening the file in binary mode (to change them in bytes)
writelines must be used with a list instead of a single item (the function has an 's' at the end). In your case, you can use 'write'
Maybe something like this will work:
f.write(("%20s,%04d\n" % (varStr, varInt)).encode())
f.write(('{0:>20s},{1:04d}\n'.format(varStr, varInt)).encode())

need guidance with basic function creation in MATLAB

I have to write a MATLAB function with the following description:
function counts = letterStatistics(filename, allowedChar, N)
This function is supposed to open a text file specified by filename and read its entire contents. The contents will be parsed such that any character that isn’t in allowedChar is removed. Finally it will return a count of all N-symbol combinations in the parsed text. This function should be stored in a file name “letterStatistics.m” and I made a list of some commands and things of how the function should be organized according to my professors' lecture notes:
Begin the function by setting the default value of N to 1 in case:
a. The user specifies a 0 or negative value of N.
b. The user doesn’t pass the argument N into the function, i.e., counts = letterStatistics(filename, allowedChar)
Using the fopen function, open the file filename for reading in text mode.
Using the function fscanf, read in all the contents of the opened file into a string variable.
I know there exists a MATLAB function to turn all letters in a string to lower case. Since my analysis will disregard case, I have to use this function on the string of text.
Parse this string variable as follows (use logical indexing or regular expressions – do not use for loops):
a. We want to remove all newline characters without this occurring:
e.g.
In my younger and more vulnerable years my father gave me some advice that I've been turning over in my mind ever since.
In my younger and more vulnerableyears my father gave me some advicethat I’ve been turning over in my mindever since.
Replace all newline characters (special character \n) with a single space: ' '.
b. We will treat hyphenated words as two separate words, hence do the same for hyphens '-'.
c. Remove any character that is not in allowedChar. Hint: use regexprep with an empty string '' as an argument for replace.
d. Any sequence of two or more blank spaces should be replaced by a single blank space.
Use the provided permsRep function, to create a matrix of all possible N-symbol combinations of the symbols in allowedChar.
Using the strfind function, count all the N-symbol combinations in the parsed text into an array counts. Do not loop through each character in your parsed text as you would in a C program.
Close the opened file using fclose.
HERE IS MY QUESTION: so as you can see i have made this list of what the function is, what it should do, and using which commands (fclose etc.). the trouble is that I'm aware that closing the file involves use of 'fclose' but other than that I'm not sure how to execute #8. Same goes for the whole function creation. I have a vague idea of how to create a function using what commands but I'm unable to produce the actual code.. how should I begin? Any guidance/hints would seriously be appreciated because I'm having programmers' block and am unable to start!
I think that you are new to matlab, so the documentation may be complicated. The root of the problem is the basic understanding of file I/O (input/output) I guess. So the thing is that when you open the file using fopen, matlab returns a pointer to that file, which is generally called a file ID. When you call fclose you want matlab to understand that you want to close that file. So what you have to do is to use fclose with the correct file ID.
fid = open('test.txt');
fprintf(fid,'This is a test.\n');
fclose(fid);
fid = 0; % Optional, this will make it clear that the file is not open,
% but it is not necessary since matlab will send a not open message anyway
Regarding the function creation the syntax is something like this:
function out = myFcn(x,y)
z = x*y;
fprintf('z=%.0f\n',z); % Print value of z in the command window
out = z>0;
This is a function that checks if two numbers are positive and returns true they are. If not it returns false. This may not be the best way to do this test, but it works as example I guess.
Please comment if this is not what you want to know.

Resources