When using libcurl multithreaded there is a long list of potential problems that can come and bite you in the ass if you're not careful:
https://curl.haxx.se/libcurl/c/threadsafe.html
What I wonder is simply if the same applies when using the libcurl multi-interface?
https://curl.haxx.se/libcurl/c/libcurl-multi.html
I would assume that is not the case as the multi-interface runs on the same thread, but I'm not sure, and I can't find it mentioned in the documentation.
No, it doesn't apply exactly for the reasons you state: the multi interface is not using multi-threading by itself.
Related
So, I want to avoid future problems when using __sync_fetch_and_add in a context of Boost-based multithreaded application.
Any chance that a low-level threading implementation used by Boost (pthreads here) would affect the functionality of buitins?
The builtins are the intrinsics.
They don't make assumptions about the libraries that will be used in applications.
There is no way it can interfere.
(On a tangent: Some libraries, like Boost Asio, optionally can use C++11 atomics instead of boost::detail::atomic_count (doc))
I'm looking for a way to run an arbitrary Haskell code safely (or refuse to run unsafe code).
Must have:
module/function whitelist
timeout on execution
memory usage restriction
Capabilities I would like to see:
ability to kill thread
compiling the modules to native code
caching of compiled code
running several interpreters concurrently
complex datatype for compiler errors (insted of simple message in String)
With that sort of functionality it would be possible to implement a browser plugin capable of running arbitrary Haskell code, which is the idea I have in mind.
EDIT: I've got two answers, both great. Thanks! The sad part is that there doesn't seem to be ready-to-go library, just a similar program. It's a useful resource though. Anyway I think I'll wait for 7.2.1 to be released and try to use SafeHaskell in my own program.
We've been doing this for about 8 years now in lambdabot, which supports:
a controlled namespace
OS-enforced timeouts
native code modules
caching
concurrent interactive top-levels
custom error message returns.
This series of rules is documented, see:
Safely running untrusted Haskell code
mueval, an alternative implementation based on ghc-api
The approach to safety taken in lambdabot inspired the Safe Haskell language extension work.
For approaches to dynamic extension of compiled Haskell applications, in Haskell, see the two papers:
Dynamic Extension of Typed Functional Languages, and
Dynamic applications from the ground up.
GHC 7.2.1 will likely have a new facility called SafeHaskell which covers some of what you want. SafeHaskell ensures type-safety (so things like unsafePerformIO are outlawed), and establishes a trust mechanism, so that a library with a safe API but implemented using unsafe features can be trusted. It is designed exactly for running untrusted code.
For the other practical aspects (timeouts and so on), lambdabot as Don says would be a great place to look.
I have written a little python application and here you can see how Task Manager looks during a typical run.
(source: weinzierl.name)
While the application is perfectly multithreaded, unsurprisingly it uses only one CPU core.
Regardless of the fact that most modern scripting languages support multithreading, scripts can run on one CPU core only.
Ruby, Python, Lua, PHP all can only run on a single core.
Even Erlang, which is said to be especially good for concurrent programming, is affected.
Is there a scripting language that has built in
support for threads that are not confined to a single core?
WRAP UP
Answers were not quite what I expected, but the TCL answer comes close.
I'd like to add perl, which (much like TCL) has interpreter-based threads.
Jython, IronPython and Groovy fall under the umbrella of combining a proven language with the proven virtual machine of another language. Thanks for your hints in this
direction.
I chose Aiden Bell's answer as Accepted Answer.
He does not suggest a particular language but his remark was most insightful to me.
You seem use a definition of "scripting language" that may raise a few eyebrows, and I don't know what that implies about your other requirements.
Anyway, have you considered TCL? It will do what you want, I believe.
Since you are including fairly general purpose languages in your list, I don't know how heavy an implementation is acceptable to you. I'd be surprised if one of the zillion Scheme implementations doesn't to native threads, but off the top of my head, I can only remember the MzScheme used to but I seem to remember support was dropped. Certainly some of the Common LISP implementations do this well. If Embeddable Common Lisp (ECL) does, it might work for you. I don't use it though so I'm not sure what the state of it's threading support is, and this may of course depend on platform.
Update Also, if I recall correctly, GHC Haskell doesn't do quite what you are asking, but may do effectively what you want since, again, as I recall, it will spin of a native thread per core or so and then run its threads across those....
You can freely multi-thread with the Python language in implementations such as Jython (on the JVM, as #Reginaldo mention Groovy is) and IronPython (on .NET). For the classical CPython implementation of the Python language, as #Dan's comment mentions, multiprocessing (rather than threading) is the way to freely use as many cores as you have available
Thread syntax may be static, but implementation across operating systems and virtual machines may change
Your scripting language may use true threading on one OS and fake-threads on another.
If you have performance requirements, it might be worth looking to ensure that the scripted threads fall through to the most beneficial layer in the OS. Userspace threads will be faster, but for largely blocking thread activity kernel threads will be better.
As Groovy is based on the Java virtual machine, you get support for true threads.
F# on .NET 4 has excellent support for parallel programming and extremely good performance as well as support for .fsx files that are specifically designed for scripting. I do all my scripting using F#.
An answer for this question has already been accepted, but just to add that besides tcl, the only other interpreted scripting language that I know of that supports multithreading and thread-safe programming is Qore.
Qore was designed from the bottom up to support multithreading; every aspect of the language is thread-safe; the language was designed to support SMP scalability and multithreading natively. For example, you can use the background operator to start a new thread or the ThreadPool class to manage a pool of threads. Qore will also throw exceptions with common thread errors so that threading errors (like potential deadlocks or errors with threading APIs like trying to grab a lock that's already held by the current thread) are immediately visible to the programmer.
Qore additionally supports and thread resources; for example, a DatasourcePool allocation is treated as a thread-local resource; if you forget to commit or roll back a transaction before you end your thread, the thread resource handling for the DatasourcePool class will roll back the transaction automatically and throw an exception with user-friendly information about the problem and how it was solved.
Maybe it could be useful for you - an overview of Qore's features is here: Why use Qore?.
CSScript in combination with Parallel Extensions shouldn't be a bad option. You write your code in pure C# and then run it as a script.
It is not related to the threading mechanism. The problem is that (for example in python) you have to get interpreter instance to run the script. To acquire the interpreter you have to lock it as it is going to keep the reference count and etc and need to avoid concurrent access to this objects. Python uses pthread and they are real threads but when you are working with python objects just one thread is running an others waiting. They call this GIL (Global Interpreter Lock) and it is the main problem that makes real parallelism impossible inside a process.
https://wiki.python.org/moin/GlobalInterpreterLock
The other scripting languages may have kind of the same problem.
Guile supports POSIX threads which I believe are hardware threads.
I received a library from an external developer in form of a well defined API(in C++ and Java). What can be some tests to check if the library is thread-safe ?
Basically you can't, it's more or less impossible to test for thread-safety.
And also if you don't have the author's guarantee that the library is thread-safe then they aren't going to fix threading issues, so future versions might be less thread-safe.
If you've got the source code, then you can investigate common thread-safety issues: shared state, locks etc. But if you've only got binaries, then the best you can hope is to show that the library is not thread-safe. Even then reproducing the problems reliably might be extremely difficult.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I'm wondering which languages support (or don't support) native multithreading, and perhaps get some details about the implementation. Hopefully we can produce a complete overview of this specific functionality.
Erlang has built-in support for concurrent programming.
Strictly speaking, Erlang processe are greenlets. But the language and virtual machine are designed from the ground up to support concurrency. The language has specific control structures for asynchronous inter-process messaging.
In Python, greenlet is a third-party package that provides lightweight threads and channel-based messaging. But it does not bear the comparison with Erlang.
I suppose that the list of languages that are higher-level than Haskell is pretty short, and it has pretty good support for concurrency and parallelism.
With CPython, one has to remember about the GIL. To summarize: only one processor is used, even on multiprocessor machines. There are multiple ways around this, as the comment shows.
Older versions of C and C++ (namely, C89, C99, C++98, and C++03) have no support at all in the core language, although libraries such as POSIX threads are available for pretty much every platform in common user today.
The newest versions of C and C++, C11 and C++11, do have built-in threading support in the language, but it's an optional feature of C11, so implementations such as single-core embedded systems can choose not to support it while supporting the rest of C11 if they desire.
Delphi/FreePascal also has support for threads.
I'll assume, from other answers, that it's only native on the Windows platforms.
Some nice libraries that implement better features on top of the TThread Object:
OmniThreadLibrary
BMThread
Clojure is an up and coming Lisp-dialect for the JVM that is specifically designed to handle concurrency well.
It features a functional style API, some very efficient implementations of various immutable data structures, and agent system (bit like actors in Scala and processes in Erlang). It even has software transactional memory.
All in all, Clojure goes to great lenght to help you write correct multithreaded and concurrent code.
I believe that the official squeak VM does not support native (OS) threads, but that the Gemstone version does.
(Feel free to edit this if not correct).
You need to define "native" in this context.
Java claims some sort of built-in multithreading, but is just based on coarse grained locking and some library support. At this moment, it is not more 'native' than C with the POSIX threads. The next version of C++ (0x) will include a threading library as well.
I know Java and C# support multithreading and that the next version of C++ will support it directly... (The planned implementation is available as part of the boost.org libraries...)
Boost::thread is great, I'm not sure whether you can say its part of the language though. It depends if you consider the CRT/STL/Boost to be 'part' of C++, or an optional add-on library.
(otherwise practically no language has native threading as they're all a feature of the OS).
This question doesn't make sense: whether a particular implementation chooses to implement threads as native threads or green threads has nothing to do with the language, that is an internal implementation detail.
There are Java implementations that use native threads and Java implementations that use green threads.
There are Ruby implementations that use native threads and Ruby implementations that use green threads.
There are Python implementations that use native threads and Python implementations that use green threads.
There are even POSIX Thread implementations that use green threads, e.g. the old LinuxThreads library or the GNU pth library.
And just because an implementation uses native threads doesn't mean that these threads can actually run in parallel; many implementations use a Global Interpreter Lock to ensure only one thread can run at a time. On the other hand, using green threads doesn't mean that they can't run in parallel: the BEAM Erlang VM for example can schedule its green threads (more precisely green processes) across mulitple CPU cores, the same is planned for the Rubinius Ruby VM.
Perl doesn't usefully support native threads.
Yes, there is a Perl threads module, and yes it uses native platform threads in its implementation. The problem is, it isn't very useful in the general case.
When you create a new thread using Perl threads, it copies the entire state of the Perl interpreter. This is very slow and uses lots of RAM. In fact it's probably slower than using fork() on Unix, as the latter uses copy-on-write and Perl threads do not.
But in general each language has its own threading model, some are different from others. Python (mostly) uses native platform threads but has a big lock which ensures that only one runs (Python code) at once. This actually has some advantages.
Aren't threads out of fashion these days in favour of processes? (Think Google Chrome, IE8)
I made a multithreading extension for Lua recently, called Lua Lanes. It merges multithreading concepts so naturally to the language, that I would not see 'built in' multithreading being any better.
For the record, Lua's built in co-operative multithreading (coroutines) can often also be used. With or without Lanes.
Lanes has no GIL, and runs code in separate Lua universes per thread. Thus, unless your C libraries would crash, it is immune to the problems associated with thread usage. In fact, the concept is more like processes and message passing, although only one OS process is used.
Finally Go is here with multi-threading with its own pkg Goroutine.
People say it is on the structure of C-language.
It is also easy to use and understand .
Perl and Python do. Ruby is working on it, but the threads in Ruby 1.8 are not really threads.