I have to submit a class project for a perl script that run in a multithread and validates MX record against a list. I think I have the script ready but I just don't know how to implement multithreading to make it run, for example, in 50 threads. Kindly examine script below and help. SO basically, what I want it to do is to make the script run in a multithreaded way to speed it up and make it complete faster.
Your help is very much appreciated.
Edit: solved
When I get home, I'll write a better manual. Including code excerpts
Step 1: Create a sequence with all important calls as a separate function
Step 2: Use the function from step 1 to create the individual threads and specialize this function with the individual parameters
Step 3: After the sequence is finish detach the thread.
Maybe do Step 2 as a Loop/Daemon
Here you can read something about threads and how do you use them in your code.
mathematik.uni-ulm.de - English Tutorial Modul threads
The Perldoc have a good example too:
perldoc - threads
Related
I got a question for you guys and its not as specific as usual, which could make it a little annoying to answer.
The tool i'm working with is Camunda in combination with Groovy scripts and the goal is to reduce the maximum cpu load (or peak load). I'm doing this by "stretching" the work load over a certain time frame since the platform seems to be unhappy with huge work load inputs in a short amount of time. The resulting problem is that Camunda wont react smoothly when someone tries to operate it at the UI - Level.
So i wrote a small script which basically just lets each individual process determine his own "time to sleep" before running, if a certain threshold is exceeded. This is based on how many processes are trying to run at the same time as the individual process.
It looks like:
Process wants to start -> Process asks how many other processes are running ->
waitingTime = numberOfProcesses * timeToSleep * iterationOfMeasures
CPU-Usage Curve 1,3 without the Script. Curve 2,4 With the script
Testing it i saw that i could stretch the work load and smoothe out the UI - Levels. But now i need to describe why this is working exactly.
The Questions are:
What does a sleep method do exactly ?
What does the sleep method do on CPU - Level?
How does an OS-Scheduler react to a Sleep Method?
Namely: Does the scheduler reschedule or just simply "wait" for the time given?
How can i recreate and test the question given above?
The main goal is not for you to answer this, but could you give me a hint for finding the right Literature to answer these questions? Maybe you remember a book which helped you understand this kind of things or a Professor recommended something to you. (Mine wont answer, and i cant blame him)
I'm grateful for hints and or recommendations !
i'm sure you could use timer event
https://docs.camunda.org/manual/7.15/reference/bpmn20/events/timer-events/
it allows to postpone next task trigger for some time defined by expression.
about sleep in java/groovy: https://www.javamex.com/tutorials/threads/sleep.shtml
using sleep is blocking current thread in groovy/java/camunda.
so instead of doing something effective it's just blocked.
I was trying to check response messages written in perl which takes requests through Amazon API and returns responses..How to run parallel fork as single thread in perl?. I'm using LWP::UserAgent module and I want to debug HTTP requests.
As a word of warning - threads and forks are different things in perl. Very different.
However the long and short of it is - you can't, at least not trivially - a fork is a separate process. It actually happens when you run -any- external command in perl, it's just by default perl sits and waits for that command to finish and return output.
However if you've got access to the code, you can amend it to run single threaded - sometimes that's as simple as reducing the paralleism with a config parameter. (In fact quite often - debugging parallel code is a much more complicated task than sequential, so getting it working before running parallel is really important).
You might be able to embed a waitpid into your primary code so you've only got one thing running at once. Without a code example though, it's impossible to say for sure.
I'm trying to figure out exactly how nodejs tasks are run. I understand that there is a main loop that takes requests and then queues them up and moves on. What exactly then executes those queued up events/tasks?
Update:
Can somebody actually please explain it? I appreciate people wanting me to script it and figure it out myself, but sometimes it's better to just have it explained rather than creating barriers to learning simple concepts.
you can follow https://github.com/node-inspector/node-inspector
use can use node-inspector to select a script and set some breakpoints,help you to understand event loop,
I'm implementing a simple job scheduler, which spans a new process for every job to run. When a job exits, I'd like it to report the number of actions executed to the scheduler.
The simplest way I could find, is to exit with the number of actions as a return code. The process would for example exit with return code 3 for "3 actions executed".
But the standard (AFAIK) being to use the return code 0 when a process exited successfully, and any other value when there was en error, would this approach risk to create any problem?
Note: the child process is not an executable script, but a fork of the parent, so not accessible from the outside world.
What you are looking for is inter process communication - and there are plenty ways to do it:
Sockets
Shared memory
Pipes
Exclusive file descriptors (to some extend, rather go for something else if you can)
...
Return convention changes are not something a regular programmer should dare to violate.
The only risk is confusing a calling script. What you describe makes sense, since what you want really is the count. As Joe said, use negative values for failures, and you should consider including a --help option that explains the return values ... so you can figure out what this code is doing when you try to use it next month.
I would use logs for it: log the number of actions executed to the scheduler. This way you can also log datetimes and other extra info.
I would not change the return convention...
If the scheduler spans a child and you are writing that you could also open a pipe per child, or a named pipes or maybe unix domain sockets, and use that for inter process communication and writing the processed jobs there.
I would stick with conventions, namely returning 0 for success, expecially if your program is visible/usable around by other people, or anyway document well those decisions.
Anyway apart from conventions there are also standards.
I have a perl script which is running correct but it is only using 1 core of my 2 core CPU, how can i make it utilise all cores.
I know that i can create threads using threads->new(); but how do i fit that into something like:
my $twig= new XML::Twig::XPath(TwigRoots => {TrdCaptRpt => \&top_level});
$twig->parsefile($file);
where the subroutine is being called by something else.
The standard approach with Perl is to not try to use multiple cores with one invocation of the script, but instead to run jobs in parallel on separate cores.
Yes, you can use threading with Perl, but Perl's threading is (very) heavyweight. To avoid potential race conditions, when you spawn a thread Perl simply copies everything that it does not want to explicitly share. Therefore using threading is likely to be much slower than not.
You would need to modify the code of XML::Twig. There is no canned answer of what would need to be done. if you find yourself having to run this script for multiple files, a better and very simple option, is to write your script so it can run for more than 1 file at the same time. You could do that with threads or you could do that with a wrapper script that executes 2 copies of your script at the same time (perhaps with xargs?).