How to write multiple variables to a file using MPI I/O using Fortran - io

The below code gives me 4 variables saving 4 different rows with 10 colums, whereas I need to save like 4 columns in 10 rows. I'm using hexdump syntax to extract from the file
program main
use mpi
integer :: wsize,wrank,ierr,i,fh,offset
integer , parameter :: count = 10
integer :: buf1(count),buf2(count),buf3(count),buf4(count)
integer , dimension(10,2) :: buf
call MPI_Init(ierr)
call MPI_Comm_rank(MPI_COMM_WORLD,wrank,ierr)
call MPI_Comm_size(MPI_COMM_WORLD,wsize,ierr)
offset = 0;
call MPI_File_open(MPI_COMM_WORLD, "test1.dat", MPI_MODE_RDWR + MPI_MODE_CREATE, MPI_INFO_NULL, fh, ierr)
do i = 1,count
buf1(i) = 1*i
buf2(i) = 2*i
buf3(i) = 3*i
buf4(i) = 4*i
end do
call MPI_FILE_WRITE_AT(fh, offset, /buf1,buf2,buf3,buf4/), 4*count, MPI_INTEGER, mpi_status_ignore, ierr)
call MPI_File_close(fh,ierr)
call MPI_FINALIZE(ierr)
end program main
using Hexdump command:
hexdump -v -e ' "%10d" ' -e ' "\n"' test1.dat > hextest1.dat`
If you try to open hextest1.dat after conversions it seems to be like what I posted.
1 2 3 4 5 6 7 8 9 10
2 4 6 8 10 12 14 16 18 20
3 6 9 12 15 18 21 24 27 30
4 8 12 16 20 24 28 32 36 40

Related

What do "!" and "." mean in BASIC?

Trying to translate BASIC code written in the 1990's to Python. I keep coming across two symbols, ! (exclamation mark) and . (period). I can't find any documentation online on what they do.
I have the code running but some of the outputs are not as expected - I am wondering if these might be the issue as I previously thought that the period may just be a typo for a multiplication.
Examples:
|
v
QWLOST = (((TW-TDAO)/(TWRT-TDAOR))^1.25)*((VISR/VIS)^0.25).(PW+PE)*DT
TFAVE = (TTO+TBO)/2!
^
|
In case anyone else in the future needs to know this.
! - defines a single
. - Was just a typo for * (multiplication)
I tried a few things in bwBasic (in Linux, in case that's relevant!).
bwBASIC: list
10: for i = 1 to 20
20: print i, ., . - i
30: next i
40: print ".="; .
This gave me:
bwBASIC: run
1 20 19
2 20 18
3 20 17
4 20 16
5 20 15
6 20 14
7 20 13
8 20 12
9 20 11
10 20 10
11 20 9
12 20 8
13 20 7
14 20 6
15 20 5
16 20 4
17 20 3
18 20 2
19 20 1
20 20 0
.= 20
Which would suggest that . (in bwBasic in any case) is the max number in a for loop.

Efficient Reading of Input File

Currently for a task, I am working with input files which give Matrix related test cases (Matrix Multiplication) i.e., example of an input file ->
N M
1 3 5 ... 6 (M columns)
....
5 4 2 ... 1 (N rows)
I was using simple read() to access them till now, but this is not efficient for large files of size > 10^2.
So I wanted to know is there some way to use processes to do this in parallel.
Also I was thinking of using multiple IO readers based on line, so then each process could read different segments of the file but couldn't find any helpful resources.
Thank you.
PS: Current code is using this:
io:fread(IoDev, "", "~d")
Did you consider to use re module? I did not make a performance test, but it may be efficient. In the following example I do not use the first "M N" line. So I did not put it in the matrix.txt file.
matrix file:
1 2 3 4 5 6 7 8 9
11 12 13 14 15 16 17 18 19
21 22 23 24 25 26 27 28 29
31 32 33 34 35 36 37 38 39
I made the conversion in the shell
1> {ok,B} = file:read_file("matrix.txt"). % read the complete file and store it in a binary
{ok,<<"1 2 3 4 5 6 7 8 9\r\n11 12 13 14 15 16 17 18 19\r\n21 22 23 24 25 26 27 28 29\r\n31 32 33 34 35 36 37 38 39">>}
2> {ok,ML} = re:compile("[\r\n]+"). % to split the complete binary in a list a binary, one for each line
{ok,{re_pattern,0,0,0,
<<69,82,67,80,105,0,0,0,0,0,0,0,1,8,0,0,255,255,255,255,
255,255,...>>}}
3> {ok,MN} = re:compile("[ ]+"). % to split the line into binaries one for each integer
{ok,{re_pattern,0,0,0,
<<69,82,67,80,73,0,0,0,0,0,0,0,17,0,0,0,255,255,255,255,
255,255,...>>}}
4> % a function to split a line and convert each chunk into integer
4> F = fun(Line) -> Nums = re:split(Line,MN), [binary_to_integer(N) || N <- Nums] end.
#Fun<erl_eval.7.126501267>
5> Lines = re:split(B,ML). % split the file into lines
[<<"1 2 3 4 5 6 7 8 9">>,<<"11 12 13 14 15 16 17 18 19">>,
<<"21 22 23 24 25 26 27 28 29">>,
<<"31 32 33 34 35 36 37 38 39">>]
6> lists:map(F,Lines). % map the function to each lines
[[1,2,3,4,5,6,7,8,9],
[11,12,13,14,15,16,17,18,19],
[21,22,23,24,25,26,27,28,29],
[31,32,33,34,35,36,37,38,39]]
7>
if you want to check the matrix size, you can replace the last line with:
[[NbRows,NbCols]|Matrix] = lists:map(F,Lines),
case (length(Matrix) == NbRows) andalso
lists:foldl(fun(X,Acc) -> Acc andalso (length(X) == NbCols) end,true,Matrix) of
true -> {ok,Matrix};
_ -> {error_size,Matrix}
end.
is there some way to use processes to do this in parallel.
Of course.
Also I was thinking of using multiple IO readers based on line, so
then each process could read different segments of the file but
couldn't find any helpful resources.
You don't seek to positions in a file by line, rather you seek to byte positions. While a file may look like a bunch of lines, a file is actually just one long sequence of characters. Therefore, you will need to figure out what byte positions you want to seek to in the file.
Check out file:position, file:pread.

How to generate 3 natural number that sum to 60 using awk

I am trying to write awk script that generate 3 natural numbers that sum to 60. I am trying with rand function but I`ve got problem with sum to 60
Here is one way:
awk -v n=60 'BEGIN{srand();a=int(rand()*n);b=int(rand()*(n-a));c=n-a-b;
print a,b,c}'
Idea is:
generate random number a :0=<a<60
generate random number b :0=<b<60-a
c=60-a-b
here, I set a variable n=60, to make it easy if you have other sum.
If we run this one-liner 10 times, we get output:
kent$ awk 'BEGIN{srand();for(i=1;i<=10;i++){a=int(rand()*60);b=int(rand()*(60-a));c=60-a-b;print a,b,c}}'
46 7 7
56 1 3
26 15 19
14 12 34
44 6 10
1 36 23
32 1 27
41 0 19
55 1 4
54 1 5

How to set an arbitrary seed to the --random-sort option of Linux SORT?

In man page of SORT, it says you can set a random source like:
$ sort some.txt --random-sort --random-source=/dev/urandom
I want to an standard output text to the source like:
$ sort some.txt --random-sort --random-source=`date +"%m%d%H%M"`
But this only says:
open failed: 11021103: No such file or directory
How can I do this?
Here's a simple python script that takes a seed and outputs random bytes:
> cat rand_bits.py
import random
import sys
if len(sys.argv) > 1:
rng = random.Random(int(sys.argv[-1]))
else:
rng = random.Random(0xBA5EBA11)
try:
while True:
sys.stdout.write(chr(rng.getrandbits(8)))
except (IOError, KeyboardInterrupt):
pass
sys.stdout.close()
You can just feed those bytes straight into sort:
> sort <(seq 25) -R --random-source=<(python rand_bits.py 5)
8
2
4
7
10
19
17
11
3
20
14
18
1
16
25
12
5
21
24
23
22
9
15
13
6
By the way, the input can be any file, but the file better be long enough!
> sort <(seq 25) -R --random-source=<(date +"%m%d%H%M")
sort: /dev/fd/12: end of file
> sort <(seq 25) -R --random-source=/dev/sda1
3
13
24
5
10
16
4
17
12
18
14
2
6
15
23
21
19
11
9
1
20
25
22
8
7

File Reading problems in Python

While Reading the files in python using
f = open ("filename.txt")
and accessing the data with
f.read(1)
and finally finding the position of stream usibg
f.tell()
for every step; We get a continous numbering starting from 0 to the current position.
The problem i am facing is that i am actually getting a random number as f.tell() for some positions and then continung the numbers.
For examle, the f.tell() outputs look something ike the following
0
1
2
3
133454568679978
6
7
8...
Any idea why this is happening?
My Code :
f=open("temp_mcompress.cpp")
current = ' '
while current != '' :
print(f.tell())
current = f.read(1)
f.close()
Temp_mcompress.cpp file :
#include <iostream>
int main(int a)
{
}
OUtput :
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
18446744073709551636
18446744073709551638
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
18446744073709551655
40
41
43
44
It seems I might have found the problem which may still be applicable to python 3.x:
source: http://docs.python.org/2.4/lib/bltin-file-objects.html
tell()
Return the file's current position, like stdio's ftell().
Note: On Windows, tell() can return illegal values (after an fgets())
when reading files with Unix-style line-endings. Use binary mode
('rb') to circumvent this problem.

Resources