I am trying to parallelize a fairly complicated simulation code used for oil field calculations.
My question is, if I can declare a variable or some allocatable arrays and a few subroutines in a module, then use this module in another module/subroutine which contains the parallel region, will those variables and arrays be considered private to each thread (i.e. they will have separate copies of those variables and changes made to a variables in a thread won't be seen by other threads) or they'll be shared?
Like this:
module m2
implicit none
integer :: global
contains
subroutine s2()
implicit none
integer :: tid
tid = ! Some calculation
end subroutine s2
end module m2
module m1
use m2
implicit none
contains
subroutine s1
!$omp parallel do
do i=1, 9
call s2()
end do
!$omp end parallel do
end subroutine s1
end module m1
Will tid and global be private or shared?
Any help is greatly appreciated!
Module variables are always shared in OpenMP unless you use the threadprivate directive. See Difference between OpenMP threadprivate and private for detailed description of threadprivate. So global will be shared.
The local variable tid is declared in the subroutine and called from the parallel region. Therefore it will be private unless it has the save attribute.
(Note that initialization like integer :: tid = 0 also adds the save implicitly, so be careful.)
Related
I am working on a f90 code that I didn't write. I am not a very experienced fortran user.
There is a part that bothers me a bit and I am not sure if it is a normal practice. Can someone help me to clarify a bit please ?
I have a structure with an allocatable array defined in a module.
This variable is passed un-allocated to a subroutine.
The subroutine then allocates its corresponding local variable.
In the main, the output of the passed structure variable is allocated.
What I am not sure I understand is how the main program is handling the size of the return variable as it is not defined before. Is it a common practice?
Personally, I would have think that a variable with a defined size should have been passed to the subroutine.
If I resume coarsly the code:
modulus_mod.f90:
module modulus_mod
public :: mod
type mod
real,allocatable,dimension(:,:) :: rec
end type mod
end module modulus_mod
subtoto.f90:
subroutine subtoto(recloc)
implicit none
real,allocatable,dimension(:,:) :: recloc
WRITE(*,*) 'in'
ALLOCATE( recloc(10,10) )
WRITE(*,*) 'inout'
recloc(:,:)=1.
WRITE(*,*) 'out'
endsubroutine subtoto
toto.f90:
program toto
use modulus_mod
implicit none
type(mod) :: model
!>>> Here not allocated if tested
if(allocated(model%rec)) WRITE(*,*) 'allocated bf'
if(.NOT.allocated(model%rec)) WRITE(*,*) 'NOT allocated bf'
CALL subtoto(model%rec)
WRITE(*,*) 'out sub'
!>>>Here it should be allocated correctly if tested
if(allocated(model%rec)) WRITE(*,*) 'allocated af'
if(.NOT.allocated(model%rec)) WRITE(*,*) 'NOT allocated af'
end program toto
Trying to make a hands-on summary here:
Variable is an allocatable dummy argument in a subroutine, for example:
subroutine sub(variable)
real, allocatable, [intent(...)] :: variable(:)
here, status is controlled by the intent keyword:
intent(out): initially not allocated, when the subroutine is called
intent(in), intent(inout) or [no intent declared]: initial allocation status depends on what the variable was like outside of this routine
At any times, you can check both the status and the size of this variable by
allocated(variable) and size(variable).
Note that size is undefined if the variable is not allocated, so you may want to use something like:
safe_size = merge(size(variable),-1,allocated(variable))
to prevent the undefined behavior.
Variable is in a module: it is saved. Its status is not allocated at the beginning of the program; both status and contents can change during runtime (for example being changed by a subroutine or function, with the rules of 1.), but nothing is modified between these calls.
In your case, if you want your variable to always be allocated by the routine, you should specify an intent(out), like:
subroutine subtoto(recloc)
implicit none
real,allocatable,dimension(:,:), intent(out) :: recloc
[...]
end subroutine subtoto
Multithreaded FFTW can be implemented as in this from FFTW homepage. Instead, we want to call the serial FFTW routines within OpenMP parallel environment using multiple threads. We want variable and fourier_variable to be thread-safe. This could be done by using PURE subroutines and declaring variable and fourier_variable inside it. The question here is related to calling FFTW routines like fftw_execute_dft_r2c from within a PURE subroutine.
A stripped-down version of the code is presented here just for reference (the full code is an optimisation solver involving many FFTW calls).
PROGRAM main
USE fft_call
REAL(8), DIMENSION(1:N, 1:N) :: variable
COMPLEX(C_DOUBLE_COMPLEX), DIMENSION(1:N/2+1, 1:N) :: fourier_variable
INTEGER :: JJ
!$OMP PARALLEL
!$OMP DO
DO JJ = 1, 5
call fourier_to_physical(fourier_variable, variable)
END DO
!$OMP END DO
!$OMP END PARALLEL
END PROGRAM main
MODULE fft_call
contains
PURE SUBROUTINE fourier_to_physical( fourier_variable, variable)
IMPLICIT NONE
REAL(8), DIMENSION(1:N, 1:N) :: variable
COMPLEX(C_DOUBLE_COMPLEX), DIMENSION(1:N/2+1, 1:N), INTENT(OUT) :: fourier_variable
CALL fftw_execute_dft_r2c (plan_fwd, variable, fourier_variable)
END SUBROUTINE fourier_to_physical
END MODULE fft_call
The error while calling fftw_plan_dft_r2c_2d from the PURE subroutine fourier_to_physical:
Error: Function reference to 'fftw_plan_dft_r2c' at (1) is to a non-PURE procedure within a PURE procedure
The question: is there a way to call FFTW routines like fftw_execute_dft_r2c from within a PURE subroutine in Fortran90?
Or, in other words, are their PURE versions of fftw_execute_dft_r2c such that we can call them from PURE procedures? We are beginners to OpenMP.
I came across a question when I was learning about how to avoid a data conflict with multiple threads potential reading and writing using the OpenMP directive !$atomic.
Shown in the text below is the code snippet made up for my question. I am wondering if it is legal in FORTRAN to use a different index (here is j) for !$atomic than the loop index variable i, which is the one immediately following the directive !$omp parallel do private(a,b) ? Thanks.
program main
...
integer :: i,j
integer, dimension(10000) :: vx,vy,va,vb
...
va=0
!$omp parallel do private(j)
do i=1,10000
j=merge(vx(i),vy(i),mod(i,2)==1)
!$omp atomic update
va(j)=va(j)+vb(j)
end do
!$omp end parallel do
...
end program
Furthermore, is it OK to loop on an atomic directive?
program main
...
integer :: i,j
integer, dimension(10000) :: vx,vy
integer, dimension(12,10000) :: va,vb
...
va=0
!$omp parallel do private(j,k)
do i=1,10000
j=merge(vx(i),vy(i),mod(i,2)==1)
do k=1,12
!$omp atomic update
va(k,j)=va(k,j)+vb(k,j)
enddo
end do
!$omp end parallel do
...
end program
Yes, why not? It is just update of a memory address, there is no difference. There even wouldn't be much sense in using atomic with i in your case, as different threads have different values of i.
BUT, be aware of your race condition with j you are writing to it from more thread, it should be private.
Your second example adds nothing new, it is the same situation, still legal.
I encountered a problem when I port my Fortran project to OpenMP. In my original code, there are two functions named add and mpy being passed to a threaded subroutine submodel that throws respective function into another subroutine defined in a module toolbox.
Now, for my new code, I am wondering whether there is a way to produce exactly the same outcome as with my original code but with a tiny twist that moves the two functions add and mpy to be hosted (i.e., contained) within the subroutine submodel.
Thanks.
Lee
--- My original code consists of four files: MAIN.F90, MODEL.F90, VARIABLE.F90, and TOOLBOX.F90
OUTPUT:
--- addition ---
3 7 11 15
--- multiplication ---
2 12 30 56
Press any key to continue . . .
MAIN.F90
program main
use model
implicit none
call sandbox()
end program main
MODEL.F90
module model
use omp_lib
use variable
implicit none
contains
subroutine submodel(func,x,y)
implicit none
interface
function func(z)
implicit none
integer :: z,func
end function func
end interface
integer :: x,y
call tool(func,x,y)
end subroutine submodel
function add(a)
implicit none
integer :: a,add
add=a+thread_private
end function add
function mpy(m)
implicit none
integer :: m,mpy
mpy=m*thread_private
end function mpy
subroutine sandbox()
implicit none
integer :: a(4),b(4),c(4),i
a=[((i),i=1,7,2)]
b=[((i),i=2,8,2)]
!$omp parallel do
do i=1,4
thread_private=b(i)
call submodel(add,a(i),c(i))
enddo
!$omp end parallel do
write(6,'(a)') '--- addition ---'
write(6,'(4(i5))') c
!$omp parallel do
do i=1,4
thread_private=b(i)
call submodel(mpy,a(i),c(i))
enddo
!$omp end parallel do
write(6,'(a)') '--- multiplication ---'
write(6,'(4(i5))') c
end subroutine sandbox
end module model
TOOLBOX.F90
module toolbox
implicit none
contains
subroutine tool(funct,input,output)
implicit none
interface
function funct(x)
implicit none
integer :: x,funct
end function funct
end interface
integer :: input,output
output = funct(input)
end subroutine tool
end module toolbox
VARIABLE.F90
module variable
use toolbox
implicit none
integer :: thread_private
!$omp threadprivate(thread_private)
end module variable
Is it possible to simply rearrange them in this way? (I have tried and apparently it failed):
subroutine submodel(func,x,y)
implicit none
interface
function func(z)
implicit none
integer :: z,func
end function func
end interface
integer :: x,y
call tool(func,x,y)
contains
function add(a)
implicit none
integer :: a,add
add=a+thread_private
end function add
function mpy(m)
implicit none
integer :: m,mpy
mpy=m*thread_private
end function mpy
end subroutine submodel
You can make the two procedures internal to the subroutine submodel exactly as you did in your last code snippet. The problem is you cannot pass these two subroutines as actual arguments from outside of the subroutine, because you have no access to them there.
Even if you have procedure pointers to them stored somewhere, these would be invalid as soon as the original run of submodel that could have created them ended.
I would think about using some switch:
subroutine submodel(switch,x,y)
implicit none
integer :: switch,x,y
select case(switch)
case(USE_ADD)
call tool(add,x,y)
case(USE_MPY)
call tool(mpy,x,y)
case default
stop "unknown switch value"
end select
contains
function add(a)
implicit none
integer :: a,add
add=a+thread_private
end function add
function mpy(m)
implicit none
integer :: m,mpy
mpy=m*thread_private
end function mpy
end subroutine submodel
Another option is to keep your original design.
I have these three functions:
When I run the first 2 functions, There's no problem, but when I run the last function (LMTD), It says 'Division by zero' yet when I debug some of the arguments have values, some don't. I know what I have to do, but I want to know why I have to do it, because it makes no sense to me.
Tinn-function doesn't have Tut's arguments, so I have to add them to Tinn-function's arguments. Same goes for Tut, that doesn't know all of Tinn's arguments, and LMTD has to have both of Tinn and Tut's arguments. If I do that, it all runs smoothly. Why do I have to do this?
Public Function Tinn(Tw, Qw, Qp, Q, deltaT)
Tinn = (((Tw * Qw) + (Tut(Q, fd, mix) * Q)) / Qp) + deltaT
End Function
Public Function Tut(Q, fd, mix)
Tut = Tinn(Tw, Qw, Qp, Q, deltaT) _
- (avgittEffektAiUiLMTD() / ((Q * fd * mix) / 3600))
End Function
Public Function LMTD(Tsjo)
LMTD = ((Tinn(Tw, Qw, Qp, Q, deltaT) - Tsjo) - (Tut(Q, fd, mix) - Tsjo)) _
/ (WorksheetFunction.Ln((Tinn(Tw, Qw, Qp, Q, deltaT) - Tsjo) _
/ (Tut(Q, fd, mix) - Tsjo)))
End Function
I will try to give a useful and complete explanation on how arguments are being passed:
As far as I can tell, LMTD is the main function calling the other function.
Each time a new function is called, it is placed on top of what they call the "stack";
The principle of Stack involves that memory is allocated and deallocated at one end of the memory (top of the stack): memory is allocated to those local variables declared and used in the function on top of the stack (function that is called gets in scope and forms a new layer on top of the stack) while these local variables are being released as soon as the function goes out of scope (when the value is returned). Something generally referred to as "Last In First Out" (LIFO).
So if you consider LMTD the base (which is probably not the ultimate base, since it is must be called by another sub routine or function), Tinn and Tut are placed on top of the stack whenever these functions are being called.
However (and here is the point),
Variables not locally declared in functions and passed as argument are standard passed by Reference, they are pointer variables containing the memory address of the arguments sent by the function (or sub) on the lower layer of the stack.
When a function takes parameters by reference (default), it can change the values contained by the memory addresses that are passed and thus the original variable value can be changed when the called function is returned.
This example illustrates it:
Sub Base_Sub()
Dim i as single
Dim c as single
Dim d as single
c = 5
d = 6
i = Function_1(c, d)
End Sub
Function Function_1(c, d)
c = 7 'Notice that the variables c and d are also changed in the Base_sub
d = 5
Function_1 = c + d
End Function
On the contrary, if you would send variable by value (byVal keyword), this would mean that a copy of the original variable (that is passed as argument) is made and the original variable remains untouched while the copy is being manipulated in the function. In other words, this copy would become a local variable on top of the stack and released as soon as the function goes out of scope.
So without looking into dept into your code to deep, when you call many functions in one routine, it may help you to keep this general concept of the different layers in mind.
In order to keep an eye on your local variables, use the "locals" window in VBA for follow-up or use debug.print to follow up in the immediate window.
What could help you gain more transparency regarding the error is by performing a check. For example
for Tinn function:
If QP = 0 then
'Notify problem at QP.
end if
I'm sorry if my explanation was more than you expected, but I tried to be as complete as possible on this one.