Easiest way to make basic OpenMP like library - multithreading

Question

I would like to make a basic library with some very basic features of OpenMP. For example, just to be able to write a code like below. I wonder how I can use LLVM and pthreads to accomplish this.
I guess there are two steps:
Preprocessing to figure out the parallel task (parallel vs parallel-for)
Convert appropriate code blocks to a void* function needed for pthreads
Automation to create, run and join threads
Example code:
#Our_Own parallel
{
printf("hello");
}
#Our_Own parallel for
for (int i =0; i < 1000; i++)
{
printf(i);
}

Answer 1

There is no need to use pragmas to implement parallel loops (or tasks) in C++. You can see this in the implementation of Threading Building Blocks (TBB), or Kokkos, both of which provide parallelism without using pragmas. As a consequence, there is no need to make any compiler modifications to do this!
The key observation here is that C++ provides lambdas which allow you to abstract a chunk of code into an anonymous function and to bind the appropriate variables from the context so that it can later be invoked in some other thread.
Even if you do want to map to pragmas, for instance to provide your own, "improved" version of OpenMP, you can do that without using anything more than C macros, by using the _Pragma directive, which can be placed inside a macro, something like this:-
#include <stdio.h>
#include <omp.h>
#define STRINGIFY1(...) #__VA_ARGS__
#define STRINGIFY(...) STRINGIFY1(__VA_ARGS__)
#define MY_PARALLEL _Pragma("omp parallel")
#define my_threadID() omp_get_thread_num()
int main (int, char **)
{
MY_PARALLEL
{
printf ("Hello from thread %d\n", my_threadID());
}
return 0;
}
However, we're rather in the dark about what you are really trying to achieve, and in what context:
Since OpenMP implementations almost all sit on top of pthreads, why do you need something different?
Which language is this for? (C, C++, other?)
What is the reason to avoid using existing implementations (such as TBB, RAJA, Kokkos, C++ Parallel Algorithms)?
Remember, "The best code is the code I do not have to write".
(P.s. if you want to see the type of thing you are taking on, look at the Little OpenMP runtime which implements some (not all) of the CPU OpenMP requirements, and the associated book.)

Answer 2

DISCLAIMER: This is a very hard task, and there are many hidden tasks involved (how to access variables and set up their sharing attributes, implicit barriers, synchronization, etc.) I do not claim that you can solve all of them using the idea described below.
If you use a (python) script to preprocess your code, here is a very minimal example how to start in C language (C++ may be a bit easier because of lambda functions).
Use macro definitions (preferably put them in a header file) :
// Macro definitions (in own.h)
#define OWN_PARALLEL_BLOCK_START(f) \
void *f(void * data) \
{ \
#define OWN_PARALLEL_BLOCK_END \
return NULL; \
}
#define OWN_PARALLEL_MAIN(f) \
{ \
int THREAD_NUM=4; \
pthread_t thread_id[THREAD_NUM]; \
for(int i=0;i<THREAD_NUM;i++) { \
pthread_create(&thread_id[i], NULL, f, NULL); \
} \
for(int i=0;i<THREAD_NUM;i++) { \
pthread_join(thread_id[i], NULL); \
} \
}
Your (python) script should convert this:
int main(){
#pragma own parallel
{
printf("Hello word\n");
}
}
to the following:
OWN_PARALLEL_BLOCK_START(ThreadBlock_1)
{
printf("Hello word\n");
}
OWN_PARALLEL_BLOCK_END
int main(){
OWN_PARALLEL_MAIN(ThreadBlock_1)
}
Check it on Compiler Explorer

Answer 3

I would like to make a basic library with some very basic features of OpenMP.
No, you really wouldn't. At minimum, because nothing of the sort you are considering can accurately be described as "basic" or as (only) a "library".
There are multiple good alternatives already available, not least OpenMP itself. Use one of those. Or if you insist on rolling your own then do so with the understanding that you are taking on a major project.
For example, just to be able to write a code like below. I wonder how I can use LLVM and pthreads to accomplish this. I guess there are two steps:
Preprocessing to figure out the parallel task (parallel vs parallel-for)
Yes, but not with the C preprocessor. It is not nearly powerful enough. You need a source-to-source translator that has a sufficient semantic understanding of the target language (C? C++?) to recognize your annotations and the source constructs to which they apply, and to apply the needed transformations and tooling.
I think the LLVM project contains pieces that would help with this (so you probably don't need to write a C parser from scratch, for example).
Convert appropriate code blocks to a void* function needed for pthreads
Yes, that would be among the transformations required.
Automation to create, run and join threads
Yes, that would also be among the transformations required.
But those are not all. For the parallel for case, for example, you also need to account for splitting the loop iterations among multiple threads. You probably also need a solution for recognizing and managing access to shared variables.