Currently for a task, I am working with input files which give Matrix related test cases (Matrix Multiplication) i.e., example of an input file ->
N M
1 3 5 ... 6 (M columns)
....
5 4 2 ... 1 (N rows)
I was using simple read() to access them till now, but this is not efficient for large files of size > 10^2.
So I wanted to know is there some way to use processes to do this in parallel.
Also I was thinking of using multiple IO readers based on line, so then each process could read different segments of the file but couldn't find any helpful resources.
Thank you.
PS: Current code is using this:
io:fread(IoDev, "", "~d")
Did you consider to use re module? I did not make a performance test, but it may be efficient. In the following example I do not use the first "M N" line. So I did not put it in the matrix.txt file.
matrix file:
1 2 3 4 5 6 7 8 9
11 12 13 14 15 16 17 18 19
21 22 23 24 25 26 27 28 29
31 32 33 34 35 36 37 38 39
I made the conversion in the shell
1> {ok,B} = file:read_file("matrix.txt"). % read the complete file and store it in a binary
{ok,<<"1 2 3 4 5 6 7 8 9\r\n11 12 13 14 15 16 17 18 19\r\n21 22 23 24 25 26 27 28 29\r\n31 32 33 34 35 36 37 38 39">>}
2> {ok,ML} = re:compile("[\r\n]+"). % to split the complete binary in a list a binary, one for each line
{ok,{re_pattern,0,0,0,
<<69,82,67,80,105,0,0,0,0,0,0,0,1,8,0,0,255,255,255,255,
255,255,...>>}}
3> {ok,MN} = re:compile("[ ]+"). % to split the line into binaries one for each integer
{ok,{re_pattern,0,0,0,
<<69,82,67,80,73,0,0,0,0,0,0,0,17,0,0,0,255,255,255,255,
255,255,...>>}}
4> % a function to split a line and convert each chunk into integer
4> F = fun(Line) -> Nums = re:split(Line,MN), [binary_to_integer(N) || N <- Nums] end.
#Fun<erl_eval.7.126501267>
5> Lines = re:split(B,ML). % split the file into lines
[<<"1 2 3 4 5 6 7 8 9">>,<<"11 12 13 14 15 16 17 18 19">>,
<<"21 22 23 24 25 26 27 28 29">>,
<<"31 32 33 34 35 36 37 38 39">>]
6> lists:map(F,Lines). % map the function to each lines
[[1,2,3,4,5,6,7,8,9],
[11,12,13,14,15,16,17,18,19],
[21,22,23,24,25,26,27,28,29],
[31,32,33,34,35,36,37,38,39]]
7>
if you want to check the matrix size, you can replace the last line with:
[[NbRows,NbCols]|Matrix] = lists:map(F,Lines),
case (length(Matrix) == NbRows) andalso
lists:foldl(fun(X,Acc) -> Acc andalso (length(X) == NbCols) end,true,Matrix) of
true -> {ok,Matrix};
_ -> {error_size,Matrix}
end.
is there some way to use processes to do this in parallel.
Of course.
Also I was thinking of using multiple IO readers based on line, so
then each process could read different segments of the file but
couldn't find any helpful resources.
You don't seek to positions in a file by line, rather you seek to byte positions. While a file may look like a bunch of lines, a file is actually just one long sequence of characters. Therefore, you will need to figure out what byte positions you want to seek to in the file.
Check out file:position, file:pread.
I am new to LabVIEW programming, and the problem is, that I need index numbers of each element in a array, and also select values by those indexes into another array, like so:
Array 13 15 16 17 18 19 12 17 17 162 626 36 828 463 565 665 565 32 587
Index 5 6 7 8 9 12 13 14 20 22 23 24 25 26 27 28 29 30 21
In the result program should get these index values .
Thanks in advance!
Upd: here is code snippet of how to get indexes of elements in the array.
Also, there is OpenG Array library (you could install it via VI Package Manager), it has built-in function of how to get all entries of element (see it on the picture).
Below is the code snippet for search in 2D array:
Update: search 2D array in 2D array, LV2013 snippet
It seems to happen after I installed YCM. I also have Python-mode plugin installed because I like the lint and "run inside vim" feature, but I turned off "rope". My YCM settings are like below.
31 " Use gcc syntastic checker
30 let g:ycm_show_diagnostics_ui = 0
29
28 " Skip checking of ycm config file at vim startup
27 let g:ycm_confirm_extra_conf=0
26
25 " Includes tags in completion
24 let g:ycm_collect_identifiers_from_tags_files=1
23
22 " Include text in comments and strings
21 let g:ycm_collect_identifiers_from_comments_and_strings = 0
20
19 " Start completion from second char
18 let g:ycm_min_num_of_chars_for_completion=2
17
16 " Disable chache, rebuild every time
15 let g:ycm_cache_omnifunc=1
14
13 " Enable syntax based completion
12 let g:ycm_seed_identifiers_with_syntax=1
11
10 " Enable completion while in comments
9 let g:ycm_complete_in_comments = 1
8
7 " Enable completion while in strings
6 let g:ycm_complete_in_strings = 1
I tried to profile vim, but I don't get a clue from the result. Below is my profiling report.
0 FUNCTIONS SORTED ON SELF TIME
1 count total (s) self (s) function
2 60 0.883352 0.782989 <SNR>93_GetCompletions()
3 2967 0.905869 0.579009 pymode#folding#expr()
4 1997 0.275032 <SNR>110_BlockStart()
5 302 0.168611 <SNR>93_Pyeval()
6 120 0.968933 0.066387 youcompleteme#Complete()
7 1997 0.051828 <SNR>110_BlockEnd()
8 76 0.030649 0.028957 <SNR>97_GetNearbyTag()
9 154 0.026513 <SNR>69_Highlight_Matching_Pair()
10 61 0.025747 <SNR>93_InsideCommentOrString()
11 61 0.057955 0.016587 <SNR>93_IdentifierFinishedOperations()
12 6 0.013762 0.013707 <SNR>93_OnFileReadyToParse()
13 60 0.012566 <SNR>93_OnCompleteDone()
14 4 0.011961 <SNR>143_SearchParensPair()
15 74 0.040687 0.010170 Powerline#Functions#fugitive#GetBranch()
16 76 0.114571 0.009806 <SNR>93_OnCursorMovedInsertMode()
17 2 0.006829 0.006792 SyntasticRefreshCursor()
18 61 0.040498 0.005477 <SNR>93_InvokeCompletion()
19 74 0.005211 308()
20 91 0.004580 Pl#Statusline()
21 74 0.005954 0.004241 <SNR>59_buffer_spec()
Vim doesn't stand asynchronous job control and is basically mono-thread, so he can only do one thing after the other.
You can't type text while YCM is working.
You should try NeoVim who stand it, it won't solve your problem but will minimize it.
I have this data in Excel.
A B C
--------------------------------------
Line Number Value #1 Value #2
1 21 35
2 21 27
3 21 18
4 10 47
5 50 5
6 37 68
7 10 21
8 75 21
I tried to calculate the total "21" based on odd line number. In this situation, the answer should be 3. However, neither" IF(MOD(A1:A8,2)=1,COUNTIF(B1:C8,21)) " nor " {IF(MOD(A1:A8,2)=1,COUNTIF(B1:C8,21))} "worked and Google didn't yield anything helpful. Could anyone help me? Thanks!!
This works for odd lines:
=SUM(COUNTIF(A:B,21)-SUMPRODUCT((A:B=21)*(MOD(ROW(A:B),2)=0)))
there may be a better way of writing this formula.
Use this to count even lines:
=SUMPRODUCT((A:B=21)*(MOD(ROW(A:B),2)=0))
I've already checked a similarly existing topic (How to read numeric data from a string in FORTRAN), but I'm not being able to do what I want.
I need to open a file and read a numeric value from a string. Bellow there's a section of the file in question. I want to read the integer next to 'ELEMENTS:', but so far I'm not being able to do so.
ELEMENT GROUP 2.4.6
GROUP: 1 ELEMENTS: 187169 MATERIAL: 2 NFLAGS: 1
fluid
0
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30
31 32 33 34 35 36 37 38 39 40
Can someone please help me here?
Ok guys, thanks to your answers the program is working!
For further reference, here's the reading part of the code:
READ(77,'(A)') str
ipos = INDEX(str,"ELEMENTS:",back=.true.) + 9
READ (str(1+ipos:),*) k
PRINT*, k
Thank for the answers.