In blog posts and examples by Mark Seemann, I got a first glimpse of free monads as a way to structure the boundary between pure code and IO code. My basic understanding is that a free monad lets you build a program (an abstract syntax tree - AST) of pure functions that an interpreter then translates into a sequence of impure procedure calls. Hence, this interpreter turns the pure operations of the AST into a sequence of monadic IO actions.
I'm wondering if this is duplicating what the Haskell runtime is already doing with the IO monad. If I view IO as nothing special, but a regular Monad whose bind function >>= sequences the state of the "Real World" through all monadic operations in IO, then this sequencing does not provide any computation on its own (as explained for free monads in the excellent answer here). I can then view all IO actions like getLine, writeFile and the like as operations in the free IO monad, and the Haskell runtime as the interpreter. The runtime interprets each IO action by means of some underlying system call, C FFI call or the like, which is obviously impure.
So, in this view, functions that return IO actions are simply building up the AST that then gets interpreted by the Haskell runtime. But up to this point, everything is pure. In this view, a function a -> IO b is not impure, in the same way that an operation of in a free monad is not impure.
Is this intuition correct? If not, where does it fall short?
Your intuition is correct: the IO-typed functions do indeed build up a tree of actions, which is then interpreted by the runtime. Well, at least this is a valid way of looking at it (see also Will Ness's comment).
The difference from a free monad is that there is just one interpreter. You can't pick another one, and you can't implement your own if you wanted to.
The AST of the free monad has two principal properties: first, it's compositional; second, it's analyzable. The interpreter can analyze the AST by matching on its constructors, and perform the interpretation accordingly.
The IO monad shares the first of these properties, but not the second. If you have a value IO String, there is no way to tell if it was created by calling readLn or pure "foo" or something else.
Related
What mechanism does Haskell use to actually decide to invoke the 4 actions below?
main :: IO ()
main = getLine >>= putStrLn >> getLine >>= putStrLn
Initially I thought it was to do with lazy evaluation, but... as from Real Word Haskell, about IO actions, they
produce an effect when performed, but not when evaluated
So I suspect it's some other mechanism rather than the system wanting to "evaluate" main. What is this mechanism? Or if it is evaluation, what is Haskell "wanting" to evaluate that causes it to execute the chain of actions?
As a first order approximation, the only source of evaluation in a Haskell program is main. What that means is that:
IO actions can be assembled and composed through >>=, >>, <*>, fmap, etc to produce any other IO actions but
only the main IO action will ever produce effects.
In a sense all a Haskell program ever does is run main :: IO (). For anything to be evaluated, it has to stand in the way of running the IO action (this is where laziness fits). That begs the question: what does it mean to actually run an IO action?
Under the hood, IO ends up behaving like a (strict) State monad that threads through it a RealWorld state (which contains no information - it is symbolic of the state that side-effects encompass on the world), so "running" IO (sort of equivalent to State RealWorld) is like calling runState. Naturally, this runState can occur only once for any program - and this is exactly what main does (and what makes it magical)!
It may seem strange, but running IO actions is actually outside the scope of ordinary Haskell language!1
The Haskell built in libraries provide "basic" IO actions like getLine :: IO String, functions that return IO actions like putStrLn :: String -> IO (), and ways of building IO actions out of other IO actions (mostly by providing a Monad interface, so anything that works on any monad like all the stuff in Control.Monad is a way of working with IO).
All of that is pure and lazy, in exactly the same way that non-IO Haskell code is. IO is not a special case for anything you can do with ordinary Haskell code (which is why you can use Monad-generic code on IO; all that code is written and compiled without any knowledge of any special rules that IO has, so it could only work if there aren't any).
But none of that actually ever performs an IO action; it just makes new IO actions out of other ones. This is what people mean when they talk about how "evaluating an IO action doesn't produce an effect". A value "apple" ++ "banana" of type String can be represented by an unevaluated thunk; when it gets evaluated to "applebanana" it still represents exactly the same value, the system just has it recorded as data in memory rather than a pointer to some code that could be run to produce it1. In exactly the same way a value putStrLn "apple" >> putStrLn "banana" of type IO () can be represented by an unevaluated thunk, and when it gets evaluated all that means is that the system is now representing that same value with a data structure instead of a pointer to code that will run the (pure, lazy) function >> on two other IO actions. But we've only talked about the system's in-memory representation of the IO action, nothing about actually running them to produce some side effects.
And there actually are no language features of Haskell that talk about how IO actions are performed. The runtime system "just knows" how to execute the main IO action from the Main module3. The Haskell language has no way of talking about how or whether that happens; that's all handled by the system that provides Haskell to you (GHC, or another Haskell system). The only option the Haskell language gives you is that main is defined as a Haskell action; any IO actions that you incorporate as part of the definition of main will get run.
1 I'm pretending that things like unsafePerformIO do not exist for the purpose of this discussion. As the name implies, it deliberately breaks the normal rules. It's also not intended for introducing "performing IO actions" as a normal part of the Haskell language, but only for use in the internals of something that presents a "normal Haskell" interface.
2 Usually this happens partially: only very basic types like Int are "all-or-nothing" evaluated. Most can be partially evaluated to data structures that contain thunks deeper down (which may or may not themselves get evaluated later).
3 Or GHCi "just knows" how to execute IO actions that you enter at its prompt.
According to https://wiki.haskell.org/IO_inside#Welcome_to_the_RealWorld.2C_baby, there is a "fake" type that represents the real world, RealWorld, and IO (a) is actually a function.
type IO a = RealWorld -> (a, RealWorld)
So main, as you might expect in other languages, is actually a function
main :: RealWorld -> ((), RealWorld)
that is called when the program runs. So to evaluate the final output, which is of type ((), RealWorld), Haskell needs to get the value of the RealWorld component, and in order to do that, it must run the main function. Note: it's the runtime that cause this function to run. There is no way in Haskell to trigger the execution of this function.
In the case of
main = getLine >>= putStrLn >> getLine >>= putStrLn
each of the actions are actually functions, and to work out the RealWorld value output at the end of the final putStrLn, it would need to run it, and all the actions leading up to it.
So it is lazy evaluation, but of the hidden RealWorld value.
After beginning to learn Haskell, there something I don't understand in Haskell even after reading a lot of documentation.
I understand that to perform IO operation you have to use a "IO monad" that wrapped a value in a kind of "black box" so the any function that uses the IO monad is still purely functional. OK, fine, but then where does the IO operation actually happen?
Does it mean that a Monad itself is not purely functional? Or is the IO operation implemented, let's say in C, and "embedded" inside the Haskell compiler?
Can I write, in pure Haskell, with or without a Monad, something that will do an IO operation? And if not, where does this ability come from if it's not possible inside the language itself? If it's embedded/linked inside the Haskell compiler to chunks of C code, will that eventually be called by the IO Monad to do the "dirty job"?
As a preface, it's not "the IO Monad", despite what many poorly-written introductions say. It's just "the IO type". There's nothing magical about monads. Haskell's Monad class is a pretty boring thing - it's just unfamiliar and more abstract than what most languages can support. You don't ever see anyone call IO "the IO Alternative," even though IO implements Alternative. Focusing too much on Monad only gets in the way of learning.
The conceptual magic for principled handling of effects (not side-effects!) in pure languages is the existence of an IO type at all. It's a real type. It's not some flag saying "This is impure!". It's a full Haskell type of kind * -> *, just like Maybe or []. Type signatures accepting IO values as arguments, like IO a -> IO (Maybe a), make sense. Type signatures with nested IO make sense, like IO (IO a).
So if it's a real type, it must have a concrete meaning. Maybe a as a type represents a possibly-missing value of type a. [a] means 0 or more values of type a. IO a means a sequence of effects that produce a value of type a.
Note that the entire purpose of IO is to represent a sequence of effects. As I said above, they're not side effects. They can't be hidden away in an innocuous-looking leaf of the program and mysteriously change things behind the back of other code. Instead, effects are rather explicitly called out by the fact that they're an IO value. This is why people attempt to minimize the part of their program using IO types. The less that you do there, the fewer ways there are for spooky action at a distance to interfere with your program.
As to the main thrust of your question, then - a complete Haskell program is an IO value called main, and the collection of definitions it uses. The compiler, when it generates code, inserts a decidedly non-Haskell block of code that actually runs the sequence of effects in an IO value. In some sense, this is what Simon Peyton Jones (one of the long-time authors of GHC) was getting at in his talk Haskell is useless.
It's true that whatever actually executes the IO action cannot remain conceptually pure. (And there is that very impure function that runs IO actions exposed within the Haskell language. I won't say more about it than it was added to support the foreign function interface, and using it improperly will break your program very badly.) But the point of Haskell is to provide a principled interface to the effect system, and hide away the unprincipled bits. And it does that in a way that's actually quite useful in practice.
After diving into monads I understand that they are a general concept to allow chaining computations inside some context (failing, non-determinism, state, etc) and there is no magic behind them.
Still IO monad feels even if not magic, but special.
you cannot escape IO monad like you can with other monads
IO action can only be run by the main function
IO is always at the bottom of a monad transformers chain
implementation of IO monad is unclear and source code shows some Haskell internals
What are the reasons for the points above? What makes IO so special?
Update: in pure code evaluation order doesn't matter. But it does matter when doing IO (we want to save customer before we read it). From what I understand IO monad gives us such ordering guarantees. Is it a property of a monad in general or it is something specific to IO monad?
you cannot escape IO monad like you can with other monads
I'm not sure what you mean by “escape”. If you mean, somehow unwrap the internal representation of the monadic values (e.g. list -> sequence of cons-cells) – that is an implementation detail. In fact, you can define an IO emulation in pure Haskell – basically just a State over a lot of globally-available data. That would have all the semantics of IO, but without actually interacting with the real world, only a simulation thereof.
If you mean, you can “extract values” from within the monad – nope, that's not in general possible, even for most pure-haskell monads. For instance, you can't extract a value from Maybe a (could be Nothing) or Reader b a (what if b is uninhabited?)
IO action can only be run by the main function
Well, in a sense, everything can only be run by the main function. Code that's not in some way invoked from main will only sit there, you could replace it with undefined without changing anything.
IO is always at the bottom of a monad transformers chain
True, but that's also the case for e.g. ST.
Implementation of IO monad is unclear and source code shows some Haskell internals
Again: implementation is just, well, an implementation detail. The complexity of IO implementations actually has a lot to do with it being highly optimised; the same is also true for specialised pure monads (e.g. attoparsec).
As I already said), much simpler implementations are possible, they just wouldn't be as useful as the full-fledged optimised real-world IO type.
Fortunately, implementation needn't really bother you; the inside of IO may be unclear but the outside, the actual monadic interface, is pretty simple.
in pure code evaluation order doesn't matter
First of all – evaluation order can matter in pure code!
Prelude> take 10 $ foldr (\h t -> h `seq` (h:t)) [] [0..]
[0,1,2,3,4,5,6,7,8,9]
Prelude> take 10 $ foldr (\h t -> t `seq` (h:t)) [] [0..]
^CInterrupted.
But indeed you can never get a wrong, non-⊥ result due to misordered pure-code evaluation. That actually doesn't apply to reordering monadic actions (IO or otherwise) though, because changing the sequence order changes the actual structure of the resultant action, not just the evaluation order that the runtime will use to construct this structure.
For example (list monad):
Prelude> [1,2,3] >>= \e -> [10,20,30] >>= \z -> [e+z]
[11,21,31,12,22,32,13,23,33]
Prelude> [10,20,30] >>= \z -> [1,2,3] >>= \e -> [e+z]
[11,12,13,21,22,23,31,32,33]
All that said, certainly IO is quite special, indeed I think some people hesitate to call it a monad (it's a bit unclear what it's actually supposed to mean for IO to fulfill the monad laws). In particular, lazy IO is one massive troublemaker (and best just avoided, at all times).
Similar statements could be made about the ST monad, or arguably the STM monad (although you can actually implement that one on top of IO).
Basically things like the Reader monad, Error monad, Writer monad, etc., are all just pure code. The ST and IO monads are the only ones that really do impure things (state mutation, etc), so they aren't definable in pure Haskell. They have to be "hard-wired" into the compiler somewhere.
Given a Haskell value (edit per Rein Heinrich's comment) f:
f :: IO Int
f = ... -- ignoring its implementation
Quoting "Type-Driven Development with Idris,"
The key property of a pure function is that the same inputs always produce the same result. This property is known as referential transparency
Is f, and, namely all IO ... functions in Haskell, pure? It seems to me that they are not since, lookInDatabase :: IO DBThing, won't always return the same value since:
at t=0, the DB might be down
at t=1, the DB might be up and return MyDbThing would result
In short, is f (and IO ... functions in general) pure? If yes, then please correct my incorrect understanding given my attempt to disprove the functional purity of f with my t=... examples.
IO is really a separate language, conceptually. It's the language of the Haskell RTS (runtime system). It's implemented in Haskell as a (relatively simple) embedded DSL whose "scripts" have the type IO a.
So Haskell functions that return values of type IO a, are actually not the functions that are being executed at runtime — what gets executed is the IO a value itself. So these functions actually are pure but their return values represent non-pure computations.
From a language design point of view, IO is a really elegant hack to keep the non-pure ugliness completely isolated away while at the same integrating it tightly into its pure surroundings, without resorting to special casing. In other words, the design does not solve the problems caused by impure IO but it does a great job of at least not affecting the pure parts of your code.
The next step would be to look into FRP — with FRP you can make the layer that contains IO even thinner and move even more of non-pure logic into pure logic.
You might also want to read John Backus' writings on the topic of Function Programming, the limitations of the Von Neumann architecture etc. Conal Elliott is also a name to google if you're interested in the relationship between purity and IO.
P.S. also worth noting is that while IO is heavily reliant on monads to work around an aspect of lazy evaluation, and because monads are a very nice way of structuring embedded DSLs (of which IO is just a single example), monads are much more general than IO, so try not to think about IO and monads in the same context too much — they are two separate things and both could exist without the other.
First of all, you're right in noticing that I/O actions are not pure. That's impossible. But, purity in all functions is one of Haskell's promising points, so what's happening?
Whether you like it or not, a function that applies into a (may also be incorrectly said "returns a") IO Something with some arguments will always return the same IO Something with the same arguments. The IO monad allows you to "hide" actions inside of the container the monad acts like. When you have a IO String, that function/object does not contain a String/[Char], but rather sort of a promise that you'll get that String somehow in the future. Thus, IO contains information of what to do when the impure I/O action needs to be performed.
After all, the only way for an IO action to be performed is by it having the name main, or be a dependency of main thereof. Because of the flexibility of monads, you can "concatenate" IO actions. A program like this... (note: this code is not a good idea)
main = do
input <- getLine
putStrLn input
Is syntatic sugar for...
main =
getLine >>= (\input -> putStrLn input)
That would state that main is the I/O action resulting from printing to standard output a string read from standard input, followed by a newline character. Did you saw the magic? IO is just a wrapper representing what to do, in an impure context, to produce some given output, but not the result of that operation, because that would need the Haskell language to admit impure code.
Think of it as sort of a receipe. If you have a receipe (read: IO monad) for a cake (read: Something in IO Something), you know how to make the cake, but you can't make the cake (because you could screw that masterpiece). Instead, the master chief (read: the most basic parts of the Haskell runtime system, responsible for applying main) does the dirty work for you (read: doing impure/illegal stuff), and, the best of all, he won't commit any mistakes (read: breaking code purity)... unless the oven breaks of course (read: System.IO.Error), but he knows how to clean that up (read: code will always remain pure).
This is one of the reasons that IO is an opaque type. It's implementation is somewhat controversial (until you read GHC's source code), and is better of to be left as implementation-defined.
Just be happy, because you've been illuminated by purity. A lot of programmers don't even know of Haskell's existence!
I hope this has led some light on you!
Haskell is pulling a trick here. IO both is and isn't pure, depending on how you look at it.
On the "IO is pure" side, you're fallen into the very common error of thinking of a function returning an IO DBThing as of it were returning a DBThing. When someone claims that a function with type Stuff -> IO DBThing is pure they are not saying that you can feed it the same Stuff and always get the same DBThing; as you correctly note that is impossible, and also not very useful! What they're saving is that given particular Stuff you'll always get back the same IO DBThing.
You actually can't get a DBThing out of an IO DBThing at all, so Haskell don't ever have to worry about the database containing different values (or being unavailable) at different times. All you can do with an IO DBThing is combine it with something else that needs a DBThing and produces some other kind of IO thing; the result of such a combination is an IO thing.
What Haskell is doing here is building up a correspondence between manipulation of pure Haskell values and changes that would happen out in the world outside the program. There are things you can do with some ordinary pure values that don't make any sense with impure operations like altering the state of a database. So using the correspondence between IO values and the outside world, Haskell simply doesn't provide you with any operations on IO values that would correspond to things that don't make sense in the real world.
There are several ways to explain how you're "purely" manipulating the real world. One is to say that IO is just like a state monad, only the state being threaded through is the entire world outside your program;= (so your Stuff -> IO DBThing function really has an extra hidden argument that receives the world, and actually returns a DBThing along with another world; it's always called with different worlds, and that's why it can return different DBThing values even when called with the same Stuff). Another explanation is that an IO DBThing value is itself an imperative program; your Haskell program is a totally pure function doing no IO, which returns an impure program that does IO, and the Haskell runtime system (impurely) executes the program it returns.
But really these are both simply metaphors. The point is that the IO value simply has a very limited interface which doesn't allow you to do anything that doesn't make sense as a real world action.
Note that the concept of monad hasn't actually come into this. Haskell's IO system really doesn't depend on monads; Monad is just a convenient interface which is sufficiently limited that if you're only using the generic monad interface you also can't break the IO limitations (even if you don't know your monad is actually IO). Since the Monad interface is also interesting enough to write a lot of useful programs, the fact that IO forms a monad allows a lot of code that's useful on other types to be generically reused on IO.
Does this mean you actually get to write pure IO code? Not really. This is the "of course IO isn't pure" side of the coin. When you're using the fancy "combining IO functions together" you still have to think about your program executing steps one after the other (or in parallel), affecting and being affected by outside conditions and systems; in short exactly the same kind of reasoning you have to use to write IO code in an imperative language (only with a nicer type system than most of them). Making IO pure doesn't really help you banish impurity from the way you have to think about your code.
So what's the point? Well for one, it gets us a compiler-enforced demarcation of code that can do IO and code that can't. If there's no IO tag on the type then impure IO isn't involved. That would be useful in any language just on its own. And the compiler knows this too; Haskell compilers can apply optimizations to non-IO code that would be invalid in most other languages because it's often impossible to know that a given section of code doesn't have side effects (unless you can see the full implementation of everything the code calls, transitively).
Also, because IO is pure, code analysis tools (including your brain) don't have to treat IO-code specially. If you can pick out a code transformation that would be valid on pure code with the same structure as the IO code, you can do it on the IO code. Compilers make use of this. Many transformations are ruled out by the structure that IO code must use (in order to stay within the bounds of things that have a sensible correspondence to things in the outside world) but they would also be ruled out by any pure code that used the same structure; the careful construction of the IO interface makes "execution order dependency" look like ordinary "data dependency", so you can just use the rules of data dependency to determine the rules of using IO.
Short answer: Yes, that f is referential transparent.
Whenever you look at it, it equals the same value.
But that doesn't mean it will always bind the same value.
In short, is f (and IO ... functions in general) pure?
So what you're really asking is:
Are IO definitions in Haskell pure?
You're really not going to like it.
Deep Thought.
It depends on what you mean by "pure".
From section 6.1.7 (page 75) of the Haskell 2010 report:
The IO type serves as a tag for operations (actions) that interact with the outside world. The IO type is abstract: no constructors are visible to the user. IO is an instance of the Monad and Functor classes.
the crucial point being:
The IO type is abstract
If Haskell's FFI was sufficiently-enhanced, IO could be as simple as:
data IO a -- a tag type: no visible constructors
instance Monad IO where
return = unitIO
(>>=) = bindIO
foreign import ccall "primUnitIO" unitIO :: a -> IO a
foreign import ccall "primBindIO" bindIO :: IO a -> (a -> IO b) -> IO b
⋮
No Haskell definitions whatsoever: all I/O-related activity is performed by calls to foreign code, usually written in the same language as the Haskell implementation. A variation of this approach
is used in Agda:
4 Compiling Agda programs
This section deals with the topic of getting Agda programs to interact
with the real world. Type checking Agda programs requires evaluating
arbitrary terms, ans as long as all terms are pure and normalizing this is
not a problem, but what happens when we introduce side effects? Clearly,
we don't want side effects to happen at compile time. Another question is
what primitives the language should provide for constructing side effecting
programs. In Agda, these problems are solved by allowing arbitrary
Haskell functions to be imported as axioms. At compile time, these imported
functions have no reduction behaviour, only at run time is the
Haskell function executed.
(emphasis by me.)
By moving the problem of I/O outside of Haskell or Agda, questions of "purity" are now a matter for that other language (or languages!).
Given these circumstances, there can be no "standard definition" for IO, so there's no common way to determine such a property for that type, let alone any of its expressions. We can't even provide a simple proof that IO is monadic (i.e. it satisfies the monad laws) as return and (>>=) simply cannot be defined in standard Haskell 2010.
To get some idea on how this affects the determining of various IO-related properties, see:
Semantics of fixIO by Levent Erkok, John Launchbury and Andrew Moran.
Tackling the Awkward Squad: … by Simon Peyton Jones (starting from section 3.2 on page 20).
So when you next hear or read about Haskell being "referentially transparent" or "purely functional", you now know that (at least for I/O) they're just conjectures - no actual standard definition means there's no way to prove or disprove them.
(If you're now wondering how Haskell got into this state, I provide some more details here.)
I've had a look at the algo.monads and fluokitten documentation. I've also read through monad blog entries by Jim Duey, Konrad Hinsen and Leonardo Borges.
The closest I can find is Konrad Hinsen's library Monadic IO streams - but this doesn't appear to 'implement the monad interface' (for want of a better phrasing)
This is example using ST in Haskell
oneST :: ST s Int -- note that this works correctly for any s
oneST = do var <- newSTRef 0
modifySTRef var (+1)
readSTRef var
one :: Int
one = runST oneST
My question is: Is it possible to do the IO Monad from Haskell in Clojure? Could you provide an example?
There are a few ways to answer this question.
Yes
Trivially:
Think of IO as a monad transformer that grants the special permission of working with side effects. Then any monad in Clojure is an IO monad, as performing side effects is not a privileged operation in Clojure.
Fatuously: Clojure is Turing-complete, so you could implement all of Haskell, including the IO monad in Clojure. Haskell is Turing-complete, so you could implement all of Clojure in Haskell and expose the IO monad.
No
Philosophically: The essence of the IO monad, preserving purity while integrating with the type system, is incompatible with Clojure's impurity and dynamic typing. Any attempt to shoehorn in the IO monad would either be at odds with Clojure's philosophy or fail to capture the essential point of having an IO monad.
Maybe
Partially: The monadic-io-streams library linked to in the question is intended for use with the algo.monads library, or its predecessor. The monadic interface is the state monad. Monadic-io-streams provides some jailed IO monadic functions to work with it. This does not prevent you from using any other functions with side effects, and without a type system integrating IO there is no systematic way to say which is which. This is not the IO monad; it just does a few things similar to the IO monad. This is interesting but of dubious utility.
Someday: There is interest in Typed Clojure. If side effects are added to the type system, then it may become desirable to isolate them in a structured manner for some purposes and give reason for the existence of something like an IO monad in Typed Clojure.