After diving into monads I understand that they are a general concept to allow chaining computations inside some context (failing, non-determinism, state, etc) and there is no magic behind them.
Still IO monad feels even if not magic, but special.
you cannot escape IO monad like you can with other monads
IO action can only be run by the main function
IO is always at the bottom of a monad transformers chain
implementation of IO monad is unclear and source code shows some Haskell internals
What are the reasons for the points above? What makes IO so special?
Update: in pure code evaluation order doesn't matter. But it does matter when doing IO (we want to save customer before we read it). From what I understand IO monad gives us such ordering guarantees. Is it a property of a monad in general or it is something specific to IO monad?
you cannot escape IO monad like you can with other monads
I'm not sure what you mean by “escape”. If you mean, somehow unwrap the internal representation of the monadic values (e.g. list -> sequence of cons-cells) – that is an implementation detail. In fact, you can define an IO emulation in pure Haskell – basically just a State over a lot of globally-available data. That would have all the semantics of IO, but without actually interacting with the real world, only a simulation thereof.
If you mean, you can “extract values” from within the monad – nope, that's not in general possible, even for most pure-haskell monads. For instance, you can't extract a value from Maybe a (could be Nothing) or Reader b a (what if b is uninhabited?)
IO action can only be run by the main function
Well, in a sense, everything can only be run by the main function. Code that's not in some way invoked from main will only sit there, you could replace it with undefined without changing anything.
IO is always at the bottom of a monad transformers chain
True, but that's also the case for e.g. ST.
Implementation of IO monad is unclear and source code shows some Haskell internals
Again: implementation is just, well, an implementation detail. The complexity of IO implementations actually has a lot to do with it being highly optimised; the same is also true for specialised pure monads (e.g. attoparsec).
As I already said), much simpler implementations are possible, they just wouldn't be as useful as the full-fledged optimised real-world IO type.
Fortunately, implementation needn't really bother you; the inside of IO may be unclear but the outside, the actual monadic interface, is pretty simple.
in pure code evaluation order doesn't matter
First of all – evaluation order can matter in pure code!
Prelude> take 10 $ foldr (\h t -> h `seq` (h:t)) [] [0..]
[0,1,2,3,4,5,6,7,8,9]
Prelude> take 10 $ foldr (\h t -> t `seq` (h:t)) [] [0..]
^CInterrupted.
But indeed you can never get a wrong, non-⊥ result due to misordered pure-code evaluation. That actually doesn't apply to reordering monadic actions (IO or otherwise) though, because changing the sequence order changes the actual structure of the resultant action, not just the evaluation order that the runtime will use to construct this structure.
For example (list monad):
Prelude> [1,2,3] >>= \e -> [10,20,30] >>= \z -> [e+z]
[11,21,31,12,22,32,13,23,33]
Prelude> [10,20,30] >>= \z -> [1,2,3] >>= \e -> [e+z]
[11,12,13,21,22,23,31,32,33]
All that said, certainly IO is quite special, indeed I think some people hesitate to call it a monad (it's a bit unclear what it's actually supposed to mean for IO to fulfill the monad laws). In particular, lazy IO is one massive troublemaker (and best just avoided, at all times).
Similar statements could be made about the ST monad, or arguably the STM monad (although you can actually implement that one on top of IO).
Basically things like the Reader monad, Error monad, Writer monad, etc., are all just pure code. The ST and IO monads are the only ones that really do impure things (state mutation, etc), so they aren't definable in pure Haskell. They have to be "hard-wired" into the compiler somewhere.
Related
Regarding Haskell's monadic IO construct:
Is it just a convention or is there is a implementation reason for it?
Could you not just FFI into libc.so instead to do your I/O, and skip the whole IO-monad thing?
Would it work anyway, or is the outcome nondeterministic because of:
(a) Haskell's lazy evaluation?
(b) another reason, like that the GHC is pattern-matching for the IO monad and then handling it in a special way (or something else)?
What is the real reason - in the end you end up using a side effect, so why not do it the simple way?
Yes, monadic I/O is a consequence of Haskell being lazy. Specifically, though, monadic I/O is a consequence of Haskell being pure, which is effectively necessary for a lazy language to be predictable.†
This is easy to illustrate with an example. Imagine for a moment that Haskell were not pure, but it was still lazy. Instead of putStrLn having the type String -> IO (), it would simply have the type String -> (), and it would print a string to stdout as a side-effect. The trouble with this is that this would only happen when putStrLn is actually called, and in a lazy language, functions are only called when their results are needed.
Here’s the trouble: putStrLn produces (). Looking at a value of type () is useless, because () means “boring”. That means that this program would do what you expect:
main :: ()
main =
case putStr "Hello, " of
() -> putStrLn " world!"
-- prints “Hello, world!\n”
But I think you can agree that programming style is pretty odd. The case ... of is necessary, however, because it forces the evaluation of the call to putStr by matching against (). If you tweak the program slightly:
main :: ()
main =
case putStr "Hello, " of
_ -> putStrLn " world!"
…now it only prints world!\n, and the first call isn’t evaluated at all.
This actually gets even worse, though, because it becomes even harder to predict as soon as you start trying to do any actual programming. Consider this program:
printAndAdd :: String -> Integer -> Integer -> Integer
printAndAdd msg x y = putStrLn msg `seq` (x + y)
main :: ()
main =
let x = printAndAdd "first" 1 2
y = printAndAdd "second" 3 4
in (y + x) `seq` ()
Does this program print out first\nsecond\n or second\nfirst\n? Without knowing the order in which (+) evaluates its arguments, we don’t know. And in Haskell, evaluation order isn’t even always well-defined, so it’s entirely possible that the order in which the two effects are executed is actually completely impossible to determine!
This problem doesn’t arise in strict languages with a well-defined evaluation order, but in a lazy language like Haskell, we need some additional structure to ensure side-effects are (a) actually evaluated and (b) executed in the correct order. Monads happen to be an interface that elegantly provide the necessary structure to enforce that order.
Why is that? And how is that even possible? Well, the monadic interface provides a notion of data dependency in the signature for >>=, which enforces a well-defined evaluation order. Haskell’s implementation of IO is “magic”, in the sense that it’s implemented in the runtime, but the choice of the monadic interface is far from arbitrary. It seems to be a fairly good way to encode the notion of sequential actions in a pure language, and it makes it possible for Haskell to be lazy and referentially transparent without sacrificing predictable sequencing of effects.
It’s worth noting that monads are not the only way to encode side-effects in a pure way—in fact, historically, they’re not even the only way Haskell handled side-effects. Don’t be misled into thinking that monads are only for I/O (they’re not), only useful in a lazy language (they’re plenty useful to maintain purity even in a strict language), only useful in a pure language (many things are useful monads that aren’t just for enforcing purity), or that you needs monads to do I/O (you don’t). They do seem to have worked out pretty well in Haskell for those purposes, though.
† Regarding this, Simon Peyton Jones once noted that “Laziness keeps you honest” with respect to purity.
Could you just FFI into libc.so instead to do IO and skip the IO Monad thing?
Taking from https://en.wikibooks.org/wiki/Haskell/FFI#Impure_C_Functions, if you declare an FFI function as pure (so, with no reference to IO), then
GHC sees no point in calculating twice the result of a pure function
which means the the result of the function call is effectively cached. For example, a program where a foreign impure pseudo-random number generator is declared to return a CUInt
{-# LANGUAGE ForeignFunctionInterface #-}
import Foreign
import Foreign.C.Types
foreign import ccall unsafe "stdlib.h rand"
c_rand :: CUInt
main = putStrLn (show c_rand) >> putStrLn (show c_rand)
returns the same thing every call, at least on my compiler/system:
16807
16807
If we change the declaration to return a IO CUInt
{-# LANGUAGE ForeignFunctionInterface #-}
import Foreign
import Foreign.C.Types
foreign import ccall unsafe "stdlib.h rand"
c_rand :: IO CUInt
main = c_rand >>= putStrLn . show >> c_rand >>= putStrLn . show
then this results in (probably) a different number returned each call, since the compiler knows it's impure:
16807
282475249
So you're back to having to use IO for the calls to the standard libraries anyway.
Let's say using FFI we defined a function
c_write :: String -> ()
which lies about its purity, in that whenever its result is forced it prints the string. So that we don't run into the caching problems in Michal's answer, we can define these functions to take an extra () argument.
c_write :: String -> () -> ()
c_rand :: () -> CUInt
On an implementation level this will work as long as CSE is not too aggressive (which it is not in GHC because that can lead to unexpected memory leaks, it turns out). Now that we have things defined this way, there are many awkward usage questions that Alexis points out—but we can solve them using a monad:
newtype IO a = IO { runIO :: () -> a }
instance Monad IO where
return = IO . const
m >>= f = IO $ \() -> let x = runIO m () in x `seq` f x
rand :: IO CUInt
rand = IO c_rand
Basically, we just stuff all of Alexis's awkward usage questions into a monad, and as long as we use the monadic interface, everything stays predictable. In this sense IO is just a convention—because we can implement it in Haskell there is nothing fundamental about it.
That's from the operational vantage point.
On the other hand, Haskell's semantics in the report are specified using denotational semantics alone. And, in my opinion, the fact that Haskell has a precise denotational semantics is one of the most beautiful and useful qualities of the language, allowing me a precise framework to think about abstractions and thus manage complexity with precision. And while the usual abstract IO monad has no accepted denotational semantics (to the lament of some of us), it is at least conceivable that we could create a denotational model for it, thus preserving some of the benefits of Haskell's denotational model. However, the form of I/O we have just given is completely incompatible with Haskell's denotational semantics.
Simply put, there are only supposed to be two distinguishable values (modulo fatal error messages) of type (): () and ⊥. If we treat FFI as the fundamentals of I/O and use the IO monad only "as a convention", then we effectively add a jillion values to every type—to continue having a denotational semantics, every value must be adjoined with the possibility of performing I/O prior to its evaluation, and with the extra complexity this introduces, we essentially lose all our ability to consider any two distinct programs equivalent except in the most trivial cases—that is, we lose our ability to refactor.
Of course, because of unsafePerformIO this is already technically the case, and advanced Haskell programmers do need to think about the operational semantics as well. But most of the time, including when working with I/O, we can forget about all that and refactor with confidence, precisely because we have learned that when we use unsafePerformIO, we must be very careful to ensure it plays nicely, that it still affords us as much denotational reasoning as possible. If a function has unsafePerformIO, I automatically give it 5 or 10 times more attention than regular functions, because I need to understand the valid patterns of use (usually the type signature tells me everything I need to know), I need to think about caching and race conditions, I need to think about how deep I need to force its results, etc. It's awful[1]. The same care would be necessary of FFI I/O.
In conclusion: yes it's a convention, but if you don't follow it then we can't have nice things.
[1] Well actually I think it's pretty fun, but it's surely not practical to think about all those complexities all the time.
That depends on what the meaning of "is" is—or at least what the meaning of "convention" is.
If a "convention" means "the way things are usually done" or "an agreement among parties covering a particular matter" then it is easy to give a boring answer: yes, the IO monad is a convention. It is the way the designers of the language agreed to handle IO operations and the way that users of the language usually perform IO operations.
If we are allowed to choose a more interesting definition of "convention" then we can get a more interesting answer. If a "convention" is a discipline imposed on a language by its users in order to achieve a particular goal without assistance from the language itself, then the answer is no: the IO monad is the opposite of a convention. It is a discipline enforced by the language that assists its users in constructing and reasoning about programs.
The purpose of the IO type is to create a clear distinction between the types of "pure" values and the types of values which require execution by the runtime system to generate a meaningful result. The Haskell type system enforces this strict separation, preventing a user from (say) creating a value of type Int which launches the proverbial missiles. This is not a convention in the second sense: its entire goal is to move the discipline required to perform side effects in a safe and consistent way from the user and onto the language and its compiler.
Could you just FFI into libc.so instead to do IO and skip the IO Monad thing?
It is, of course, possible to do IO without an IO monad: see almost every other extant programming language.
Would it work anyway or is the outcome undeterministic because of Haskell evaluating lazy or something else, like that the GHC is pattern matching for IO Monad and then handling it in a special way or something else.
There is no such thing as a free lunch. If Haskell allowed any value to require execution involving IO then it would have to lose other things that we value. The most important of these is probably referential transparency: if myInt could sometimes be 1 and sometimes be 5 depending on external factors then we would lose most of our ability to reason about our programs in a rigorous way (known as equational reasoning).
Laziness was mentioned in other answers, but the issue with laziness would specifically be that sharing would no longer be safe. If x in let x = someExpensiveComputationOf y in x * x was not referentially transparent, GHC would not be able to share the work and would have to compute it twice.
What is the real reason?
Without the strict separation of effectful values from non-effectful values provided by IO and enforced by the compiler, Haskell would effectively cease to be Haskell. There are plenty of languages that don't enforce this discipline. It would be nice to have at least one around that does.
In the end you end you endup in a sideeffect. So why not do it the simple way?
Yes, in the end your program is represented by a value called main with an IO type. But the question isn't where you end up, it's where you start: If you start by being able to differentiate between effectful and non-effectful values in a rigorous way then you gain a lot of advantages when constructing that program.
What is the real reason - in the end you end up using a side effect, so why not do it the simple way?
...you mean like Standard ML? Well, there's a price to pay - instead of being able to write:
any :: (a -> Bool) -> [a] -> Bool
any p = or . map p
you would have to type out this:
any :: (a -> Bool) -> [a] -> Bool
any p [] = False
any p (y:ys) = y || any p ys
Could you not just FFI into libc.so instead to do your I/O, and skip the whole IO-monad thing?
Let's rephrase the question:
Could you not just do I/O like Standard ML, and skip the whole IO-monad thing?
...because that's effectively what you would be trying to do. Why "trying"?
SML is strict, and relies on sytactic ordering to specify the order of evaluation everywhere;
Haskell is non-strict, and relies on data dependencies to specify the order of evaluation for certain expressions e.g. I/O actions.
So:
Would it work anyway, or is the outcome nondeterministic because of:
(a) Haskell's lazy evaluation?
(a) - the combination of non-strict semantics and visible effects is generally useless. For an amusing exhibition of just how useless this combination can be, watch this presentation by Erik Meiyer (the slides can be found here).
I feel I have a good understanding of monads, but I'm not too sure what is referred to by 'monadic effects'? Is this the evaluation of a monad? Does it have something to do with IO?
If you have a value of type M a with M a Monad (or Applicative for applicative effects), then by effects we mean the information that is not contained in the a part. For example with IO it is very clear. A value of IO Int is an Int with some IO effects like writing to a file or firing missiles. A value of type Maybe Int is an Int with the effect of maybe actually not containing an Int. For [Int] the effect is, that you actually have multiple Ints.
We call this an effect because you can think of Monads and Applicatives as notions of computation with certain effects. For Maybe the effects are that you can abort the computation prematurely, for [] you can split the computation.
I'm going to try to talk at the question several ways, and hopefully it's helpful.
An "effect" (as in "side-effects") refer to the behaviors of a specific instance of Monad, so e.g. the State monad expresses the effect of "stateful computation" with get and put. Monad transformer libraries like mtl can be thought of as ways of "composing effects".
Without knowing the types (or in fact reading the docs) for foo and bar here, we can't say anything about what "monadic effects" are happening here, even though we can say quite a few other things about this code:
do a <- fmap bar $ foo x
b <- baz
return (a,b)
The do block above has a type of the form SomeMonad m=> m (a,b). That tuple (a,b) that is "returned", and the way it can be passed to another "effectful computation" with >>=, is not what we're talking about when we talk about "effects".
Monadic effects always actually "happen" when you run them (by calling runState for State for instance).
In the case of IO only the runtime has access to the particular run function for IO, so the nonexistent runIO function calls main to run your program. For IO the "monadic effects" are truly the same as what in other languages you'd call "side-effects", i.e. just about anything that might change the state of the world.
When I was reading the documentations of SDL in haskell, I found that some functions inevitably modifies its input. For example, blitSurface has destination surface as input, but it is updated within the function. Now, generalizing the problem, if I have a function f :: a -> IO a, does it break composition if I modify a inside the function? What about f :: IO a -> IO a? What about a -> IO ()? And what about IO a -> IO ()?
Considering the case that blitSurface is actually a foreign function, and making a new surface every frame doesn't sound very efficient, these functions are hard to avoid. Will such functions cause problems in a larger scale? For example, using fModifySurface :: Surface -> IO () which does destructive update as an example:
main = do
w <- ... -- The window surface
-- Do something here
s <- someFuncGetSurface -- We get a surface here
fModifySurface s -- Destructively update s
blitSurface ...... -- Ignore the actual API, but destructively updates w
Are there any unexpected semantics in the code above? If so, what is the best way to make use of foreign functions that changes the input?
I observe that f a b and flip f b a are beta-equivalent terms. On the other hand, the straightforward IO version of these, namely, f <$> a <*> b and flip f <$> b <*> a, are certainly not beta-equivalent; and even using the equivalence from "Tackling the Awkward Squad", which makes many more IO actions equivalent, these two terms are not equivalent.
At a high level, what this means is that if you prove something about the behavior of pure terms, then you can re-use that proof even when the pure computation is used as part of a larger program. On the other hand, there is not a corresponding way to uniformly lift a local proof about an IO term into a proof about a larger IO-based program; if you wish to do so, you must invoke some global properties about the particular IO actions you plan to use together.
This is the impetus behind the common advice to lift as much of your computation as possible out of IO and into the pure world -- indeed, it is one of the primary motivations for doing pure functional programming in the first place, namely, that imperative programs do not compose well.
However, none of this discussion is specific to the FFI or to functions whose IO actions update the values referenced by one of their inputs. blitSurface is no worse or better in this regard than basically any of the "sin bin" we toss into IO. Imperative programs simply aren't compositional in the same sense that pure ones are.
I've had a look at the algo.monads and fluokitten documentation. I've also read through monad blog entries by Jim Duey, Konrad Hinsen and Leonardo Borges.
The closest I can find is Konrad Hinsen's library Monadic IO streams - but this doesn't appear to 'implement the monad interface' (for want of a better phrasing)
This is example using ST in Haskell
oneST :: ST s Int -- note that this works correctly for any s
oneST = do var <- newSTRef 0
modifySTRef var (+1)
readSTRef var
one :: Int
one = runST oneST
My question is: Is it possible to do the IO Monad from Haskell in Clojure? Could you provide an example?
There are a few ways to answer this question.
Yes
Trivially:
Think of IO as a monad transformer that grants the special permission of working with side effects. Then any monad in Clojure is an IO monad, as performing side effects is not a privileged operation in Clojure.
Fatuously: Clojure is Turing-complete, so you could implement all of Haskell, including the IO monad in Clojure. Haskell is Turing-complete, so you could implement all of Clojure in Haskell and expose the IO monad.
No
Philosophically: The essence of the IO monad, preserving purity while integrating with the type system, is incompatible with Clojure's impurity and dynamic typing. Any attempt to shoehorn in the IO monad would either be at odds with Clojure's philosophy or fail to capture the essential point of having an IO monad.
Maybe
Partially: The monadic-io-streams library linked to in the question is intended for use with the algo.monads library, or its predecessor. The monadic interface is the state monad. Monadic-io-streams provides some jailed IO monadic functions to work with it. This does not prevent you from using any other functions with side effects, and without a type system integrating IO there is no systematic way to say which is which. This is not the IO monad; it just does a few things similar to the IO monad. This is interesting but of dubious utility.
Someday: There is interest in Typed Clojure. If side effects are added to the type system, then it may become desirable to isolate them in a structured manner for some purposes and give reason for the existence of something like an IO monad in Typed Clojure.
I don't understand the exact algebra and theory behind Haskell's monads. However, when I think about functional programming in general I get the impression that state would be modelled by taking an initial state and generating a copy of it to represent the next state. This is like when one list is appended to another; neither list gets modified, but a third list is created and returned.
Is it therefore valid to think of monadic operations as implicitly taking an initial state object as a parameter and implicitly returning a final state object? These state objects would be hidden so that the programmer doesn't have to worry about them and to control how they gets accessed. So, the programmer would not try to copy the object representing the IO stream as it was ten minutes ago.
In other words, if we have this code:
main = do
putStrLn "Enter your name:"
name <- getLine
putStrLn ( "Hello " ++ name )
...is it OK to think of the IO monad and the "do" syntax as representing this style of code?
putStrLn :: IOState -> String -> IOState
getLine :: IOState -> (IOState, String)
main :: IOState -> IOState
-- main returns an IOState we can call "state3"
main state0 = putStrLn state2 ("Hello " ++ name)
where (state2, name) = getLine state1
state1 = putStrLn state0 "Enter your name:"
No, that's not what monads in general do. However, your analogy is in fact exactly correct with regards to the data type State s a, which happens to be a monad. State is defined like this:
newtype State s a = State { runState :: s -> (a, s) }
...where the type variable s is the state value and a is the "regular" value that you use. So a value in "the State monad" is just a function from an initial state, to a return value and final state. The monadic style, as applied to State, does nothing more than automatically thread a state value through a sequence of functions.
The ST monad is superficially similar, but uses magic to allow computations with real side-effects, but only such that the side effects can't be observed from outside particular uses of ST.
The IO monad is essentially an ST monad set to "more magic", with side effects that touch the outside world and only a single point where IO computations are run, namely the entry point for the entire program. Still, on some conceptual level, you can still think of it as threading a "state" value through functions the way regular State does.
However, other monads don't necessarily have anything whatsoever to do with threading state, or sequencing functions, or whatnot. The operations needed for something to be a monad are incredibly general and abstract. For instance, using Maybe or Either as monads lets you use functions that might return errors, with the monadic style handling escaping from the computation when an error occurs the same way that State threads a state value. Using lists as a monad gives you nondeterminism, letting you simultaneously apply functions to multiple inputs and see all possible outputs, with the monadic style automatically applying the function to each argument and collecting all the outputs.
Is it therefore valid to think of monadic operations as implicitly taking an initial state object as a parameter and implicitly returning a final state object?
This seems to be a common sticking point for learning monads, i.e., trying to figure out how a single magical monad primordial soup is simultaneously useful for representing stateful computations, computations that can fail, nondeterministic computations, exceptions, continuations, sequencing side effects, and so on.
Threading state through a sequence of stateful computations is one single example of an operation that satisfies the monad laws.
You're correct in observing that the State and IO monads are close cousins, but your analogy will fall apart if you try inserting, say, the list monad.
Not monads in general, but for the IO monad, yes — in fact, the type IO a is often defined as the function type RealWorld -> (RealWorld, a). So in this view, the desugared type of putStrLn is String -> RealWorld -> (RealWorld, ()) and getChar is RealWorld -> (RealWorld, Char) — and we only partially apply it, with the monadic bind taking care of fully evaluating it and passing the RealWorld around. (GHC's ST library actually includes a very real RealWorld type, though it's described as "deeply magical" and not for actual use.)
There are many other monads that don't have this property, though. There's no RealWorld being passed around with, for example, the monads [1,2,3,4] or Just "Hello".
Absolutely not. This is not what monads are in general. You can use monads to implicitly pass around data, but that is just one application. If you use this model of monads then you are going to miss out on a lot of the really cool things monads can do.
Instead, think about monads as being pieces of data that represent a computation. For example, there is a sense in which implicitly passing around data isn't pure because pure languages insist that you be explicit about all of your arguments and return types. So if you want to pass around data implicitly you can do this: define a new data type that is a representation of doing something impure, and then write a piece of code to work with that.
An extreme example (just a theoretical example, you're unlikely to want to do this) would be this: C allows impure computations, so you could define a type that represents a piece of C code. You can then write an interpreter that takes one of these C structures and interprets it. All monads are like this, though usually much simpler than a C interpreter. But this view is more powerful because it also explains the monads that aren't about passing around hidden state.
You should probably also try to resist the temptation to see IO as passing around a hidden world state. The internal implementation of IO has nothing to do with how you should think about IO. The IO monad is about building representations of the I/O you want to perform. You return one of these representations to the Haskell run-time system and it's up to that system to implement your representation however it wants.
Any time you want to do something, and you can't see how to directly implement it in a pure way, but you can see how to build a pure data structure to describe want you want, you may have an application for a monad, especially if your structure is naturally a tree of some sort.
I prefer to think about monads as objects that represents delayed actions (runXXX, main) with result which can be combined according to that result.
return "somthing"
actionA >>= \x -> makeActionB x
And those action not necessary to be state-full. I.e. you can think about monad of function construction like this:
instance Monad ((->) a) where
m >>= fm = \x -> fm (m x) x
return = const
sqr = \x -> x*x
cube = \x -> x*x*x
weird = do
a <- sqr
b <- cube
return (a+b)