Sorry newb question here, but how does Haskell know not to apply referential transparency to e.g. readLn or when putStrLn-ing a same string twice? Is it because IO is involved? IOW, will not the compiler apply referential transparency to functions returning IO?
You need to distinguish between evaluation and execution.
If you evaluate 2 + 7, the result is 9. If you replace one expression that evaluates to 9 with another, different expression that also evaluates to 9, then the meaning of the program has not changed. This is what referential transparency guarantees. We could common up several shared expressions that reduce to 9, or duplicate a shared expression into multiple copies, and the meaning of the program does not change. (Performance might, but not the end result.)
If you evaluate readLn, it evaluates to an I/O command object. You can imagine it as being a data structure that describes what I/O operation(s) you want performed. But the object itself is just data. If you evaluate readLn twice, it returns the same I/O command object twice. You can merge several copies into one; you can split one copy into several. It doesn't change the meaning of the program.
Now if you want to execute the I/O action, that's a different thing. Clearly, I/O operations need to be executed in exactly the way the program specifies, without being randomly duplicated or rearranged. But that's OK, because it's not the Haskell expression evaluation engine that does that. You can pretend that the Haskell runtime runs main, which builds a giant I/O command object representing the entire program. The Haskell runtime then reads this data structure and executes the I/O operations it requests, in the order specified. (Not actually how it works, but a useful mental model.)
Ordinarily you don't need to bother thinking about this strict separation between evaluating readLn to get an I/O command object and then executing the resulting I/O command object to get a result. But strictly that's notionally what it does.
(You may also have heard that I/O "forms a monad". That's merely a fancy way of saying that there's a particular set of operators for changing I/O command objects together into bigger I/O command objects. It's not central to understanding the separation between evaluate and execute.)
The IO type is defined as:
newtype IO a = IO (State# RealWorld -> (# State# RealWorld, a #))
Notice is very similar to the State Monad where the state is the state of the real world, so imho you can think of IO as referentially transparent and pure, what's impure is the Haskell runtime (interpreter) that runs your IO actions (the algebra).
Take a look at the Haskell wiki, it explains IO in greater detail: IO Inside
Due to return values are wrapped into IO, you can't reuse them unless you "pull" them out, effectively running the IO action:
readLn :: IO String
twoLines = readLn ++ readLn -- can't do this, because (++) works on String's, not IO String's
twoLines' = do
l1 <- readLn
l2 <- readLn -- "pulling" out of readLn causes console input to be read again
return (l1 ++ l2) -- so l1 and l2 have different values, and this works
Sort of. You’ve probably heard that IO is a monad, which means a value wrapped in it has to use monadic operations, such as bind (>>=), sequential composition (>>) and return. So, you could write a greeting program like this:
prompt :: IO String
prompt = putStrLn "Who are you?" >>
getLine >>=
\name ->
return $ "Hello, " ++ name ++ "!"
main :: IO ()
main = prompt >>=
putStrLn
You’re more likely to see this with the equivalent do notation, which is just another way of writing the exact same program. In this case, though, I think the unsugared version makes it clearer that the computation is a series of statements chained together with >> and >>=, where we use >> when we want to throw out the result of the previous step, and >>= when we want to pass a result to the next function in the chain. If we need to give that result a name, we can capture it as a parameter, like in the lambda expression \name -> inside prompt. If we need to lift a simple String into an IO String, we use return.
The equivalent in do notation, by the way, is:
prompt :: IO String
prompt = do
putStrLn "Who are you?"
name <- getLine
return $ "Hello, " ++ name ++ "!"
main :: IO ()
main = do
message <- prompt
putStrLn message
So how does it know that main, which returns nothing, is not referentially transparent, and that prompt, which returns an IO String, is not either? There is something special to IO, or at least something IO lacks: with many other monads, such as State and Maybe, there is a way to do a lazy computation inside the monad and discard the wrapper, getting a pure value back out. You can declare a State Int monad, do deterministic, sequential, stateful computations inside it for a while, then use evalState to get the pure Int result back. You can do a Maybe Char computation, like searching for a character in a string, check that it worked, and if so, read the pure Char back out.
With IO, you cannot do this. If you have an IO String, all you can do with it is bind it to an IO function that takes a String argument, such as PutStrLn, or pass it to a function that takes an IO String argument. If you call prompt a second time, it won’t silently give you the same result; it will actually run again. If you tell it to pause for a few milliseconds, it won’t lazily wait until you need some return value later in the program to do that. If it returns an empty value like IO (), the compiler will not optimize it by just returning that constant.
The way this works internally is to wrap the object together with a state-of-the-world parameter that is different for each call. That means that two different calls to getLine depend on different states of the world, and the return value of main requires computing the final state of the world, which depends on all previous IO operations.
Related
What mechanism does Haskell use to actually decide to invoke the 4 actions below?
main :: IO ()
main = getLine >>= putStrLn >> getLine >>= putStrLn
Initially I thought it was to do with lazy evaluation, but... as from Real Word Haskell, about IO actions, they
produce an effect when performed, but not when evaluated
So I suspect it's some other mechanism rather than the system wanting to "evaluate" main. What is this mechanism? Or if it is evaluation, what is Haskell "wanting" to evaluate that causes it to execute the chain of actions?
As a first order approximation, the only source of evaluation in a Haskell program is main. What that means is that:
IO actions can be assembled and composed through >>=, >>, <*>, fmap, etc to produce any other IO actions but
only the main IO action will ever produce effects.
In a sense all a Haskell program ever does is run main :: IO (). For anything to be evaluated, it has to stand in the way of running the IO action (this is where laziness fits). That begs the question: what does it mean to actually run an IO action?
Under the hood, IO ends up behaving like a (strict) State monad that threads through it a RealWorld state (which contains no information - it is symbolic of the state that side-effects encompass on the world), so "running" IO (sort of equivalent to State RealWorld) is like calling runState. Naturally, this runState can occur only once for any program - and this is exactly what main does (and what makes it magical)!
It may seem strange, but running IO actions is actually outside the scope of ordinary Haskell language!1
The Haskell built in libraries provide "basic" IO actions like getLine :: IO String, functions that return IO actions like putStrLn :: String -> IO (), and ways of building IO actions out of other IO actions (mostly by providing a Monad interface, so anything that works on any monad like all the stuff in Control.Monad is a way of working with IO).
All of that is pure and lazy, in exactly the same way that non-IO Haskell code is. IO is not a special case for anything you can do with ordinary Haskell code (which is why you can use Monad-generic code on IO; all that code is written and compiled without any knowledge of any special rules that IO has, so it could only work if there aren't any).
But none of that actually ever performs an IO action; it just makes new IO actions out of other ones. This is what people mean when they talk about how "evaluating an IO action doesn't produce an effect". A value "apple" ++ "banana" of type String can be represented by an unevaluated thunk; when it gets evaluated to "applebanana" it still represents exactly the same value, the system just has it recorded as data in memory rather than a pointer to some code that could be run to produce it1. In exactly the same way a value putStrLn "apple" >> putStrLn "banana" of type IO () can be represented by an unevaluated thunk, and when it gets evaluated all that means is that the system is now representing that same value with a data structure instead of a pointer to code that will run the (pure, lazy) function >> on two other IO actions. But we've only talked about the system's in-memory representation of the IO action, nothing about actually running them to produce some side effects.
And there actually are no language features of Haskell that talk about how IO actions are performed. The runtime system "just knows" how to execute the main IO action from the Main module3. The Haskell language has no way of talking about how or whether that happens; that's all handled by the system that provides Haskell to you (GHC, or another Haskell system). The only option the Haskell language gives you is that main is defined as a Haskell action; any IO actions that you incorporate as part of the definition of main will get run.
1 I'm pretending that things like unsafePerformIO do not exist for the purpose of this discussion. As the name implies, it deliberately breaks the normal rules. It's also not intended for introducing "performing IO actions" as a normal part of the Haskell language, but only for use in the internals of something that presents a "normal Haskell" interface.
2 Usually this happens partially: only very basic types like Int are "all-or-nothing" evaluated. Most can be partially evaluated to data structures that contain thunks deeper down (which may or may not themselves get evaluated later).
3 Or GHCi "just knows" how to execute IO actions that you enter at its prompt.
According to https://wiki.haskell.org/IO_inside#Welcome_to_the_RealWorld.2C_baby, there is a "fake" type that represents the real world, RealWorld, and IO (a) is actually a function.
type IO a = RealWorld -> (a, RealWorld)
So main, as you might expect in other languages, is actually a function
main :: RealWorld -> ((), RealWorld)
that is called when the program runs. So to evaluate the final output, which is of type ((), RealWorld), Haskell needs to get the value of the RealWorld component, and in order to do that, it must run the main function. Note: it's the runtime that cause this function to run. There is no way in Haskell to trigger the execution of this function.
In the case of
main = getLine >>= putStrLn >> getLine >>= putStrLn
each of the actions are actually functions, and to work out the RealWorld value output at the end of the final putStrLn, it would need to run it, and all the actions leading up to it.
So it is lazy evaluation, but of the hidden RealWorld value.
What is the difference between pure and impure in haskell?
When doing IO in haskell, what does it mean to keep the pure and impure items separate?
Essentially you want to keep as little code as possible in the "impure section". Code written in the IO monad is forever tainted with insecurity. A function with the signature IO Int will return an integer in the IO monad, but it could on top of that send nuclear missiles to the moon. We have no way of knowing without studying every line of the code.
For example, let's say you want to write a program that takes a string and appends ", dude" to it.
main = do
line <- getLine
putStrLn $ line ++ ", dude"
Some of the parts of the code are required to be in the IO monad, because they have side effects. This includes getLine and putStrLn. However, putting the two strings together does not.
main = do
line <- getLine
putStrLn $ addDude line
addDude input = input ++ ", dude"
The signature of addDude shows that it is pure: String -> String. No IO here. This means we can assume that addDude will behave at least in that way. It will take one string and return one string. It is impossible for it to have side effects. It is impossible for it to blow up the moon.
Purity simply means causing no side-effects (reading from a disk, moving a robot arm, etc.)
Separating pure from impure functions means that you can know more about what your code is going to do. For example, when you say 1 + 2, you know for sure by its type (Int -> Int -> Int) that the only thing it does is take two numbers and produce a third number. If its type were Int -> Int -> IO Int, it might move a robot arm each time it adds the two numbers.
A good starting place for the basics of Haskell can be found here:
http://learnyouahaskell.com/introduction#so-whats-haskell
Everything in Haskell is pure. What you are reading about is likely about code inside the IO mondad vs. outside. Once you put something into the IO monad, it can never "escape" -- you have to stay in the IO monad. Therefore, the IO monad has a tendency to "invade" your code -- if you have a thing that returns IO, then any code that calls that also has to return IO, and so on. Therefore it is best to use the IO monad only where it is necessary, at as top level of the program as possible, and separate out any parts of the computation that are pure into pure functions.
I cant figure out why I get two different results but I'm sure it has to do with IO, which I am beginning to hate!
For example:
ghci> x <- readFile "foo.txt"
ghci> let y = read x :: [Int]
ghci> :t y
y :: [Int]
Now when I create that file and do the same thing it comes out as IO [Int] ?
foo.txt is a txt file containing only this: 12345
Someone that can explain this to me? As I'm about to snap it!
Thanks for any insight!
Read about ghci. To quote
The syntax of a statement accepted at the GHCi prompt is exactly the same as the syntax of a statement in a Haskell do expression. However, there's no monad overloading here: statements typed at the prompt must be in the IO monad.
Basically you are inside the IO Monad when you are writing anything in ghci.
Main idea
Be clear on the distinction in Haskell between an IO operation that produces a value, and the value itself.
Recommended Reading
A good reference to IO in Haskell that doesn't expect you to want to know the theoretical basis for monads is sigfpe's
The IO Monad for People who Simply Don't Care.
Note it's the monad-theoretical bit that he's assuming you don't care about. He's assuming you do care about doing IO.
It's not very long, and I think it's very much worth a read for you, because it makes explicit some 'rules' that you're not aware of so are causing you irritation.
Your code
Anyway, in your code
x <- readFile "foo.txt"
the readFile "foo.txt" bit has type IO String, which means it's an operation that produces a String.
When you do x <- readFile "foo.txt", you use x to refer to the String it produces.
Notice the distinction between the output, x and the operation that produced it, readFile "foo.txt".
Next let's look at y. You define let y = read x :: [Int], so y is a list of Ints, as you specified.
However, y isn't the same as the whole chunk that defines it.
example = do
x <- readFile "foo.txt"
let y = read x :: [Int]
return y
Here example :: IO [Int], whereas y itself has type [Int].
The cause of your frustration
If you come from an imperative language, this is frustrating at first -
you're used to being able to use functions that produce values wherever you'd use values,
but you're used to those functions also being allowed to execute arbitrary IO operations.
In Haskell, you can do whatever you like with 'pure' functions (that don't use IO) but not IO operations.
Haskell programmers see a whole world of difference between an IO operation that returns a value,
which can only be reused in other IO operations, and a pure function which can be used anywhere.
This means that you can end up trapped in the awkward IO monad all the time,
and all your functions are full of IO datatypes. This is inconvenient and you write messy code.
How to avoid IO mess
First solve the problem you have in its entirety without using external (file or user) data:
Write sample data that you normally read from a file or user as values in your source code.
Write your functions with any data you need in the definition as parameters to the function.
This is the only way you can get data when you're writing pure code. Write pure code first.
Test your functions on the specimen data. (If you like, you can reload in ghci every time you write a new function, making sure it does what you expect.)
Once your program is complete without the IO, you can introduce it as a wrapper around the pure code at the end.
This means that in your program, I don't think you should be writing any readFile or other IO code until you're nearly finished.
It's a completely different workflow - in an imperative language you'd write code to read your data, then do stuff, then write your data.
In Haskell it's better to write the code that does stuff first, then the code to read and write the data at the end, once you know the functionality is right.
You can't actually do exactly the same thing in a Haskell source file, so I suspect what you did actually looks like this:
readFoo = do
x <- readFile "foo.txt"
let y = read x :: [Int]
return y
And are surprised that the type of readFoo comes out as IO [Int], even though it's returning y which is of type [Int].
If this is the case, the source of your confusion is that return in Haskell isn't a return statement from imperative languages.
return in Haskell is a function. A perfectly ordinary function. Like any other function it takes a value of some type as input and gives you a value of some other type as output. Specialised to the case of IO (return can be used with any monad, but we'll keep it simple here), it has this type:
a -> IO a
So it takes a value of any type and gives you a value in the same type wrapped up in the IO monad. So if y has type [Int], then return y has type IO [Int], and that's what you get as the result of readFoo.
There's no way to get the [Int] "out" of the IO [Int]. This is deliberate. The whole point of IO is that any value which is dependent on anything "outside" the program can only appear in an IO x type. So a function which internally reads "foo.txt" and returns a list of the integers in it must be impossible to write in Haskell, or the whole house of cards falls down. If we saw a function with a type like readFoo :: [Int], as you were trying to write, then we know that readFoo can only be one particular list of integers; it can't be a list that depends on the contents of a file on disk.
GHCi does a little bit of magic to make it easier to quickly test stuff at the command prompt. If this were part of a program, then IO [Int] would indeed be the correct type. GHCi is letting you treat it as just Int to save you a bit of typing.
So that's the "why".
Now, if you have a specific question like "how do I do X given that the type signature is this?"...
I don't understand the exact algebra and theory behind Haskell's monads. However, when I think about functional programming in general I get the impression that state would be modelled by taking an initial state and generating a copy of it to represent the next state. This is like when one list is appended to another; neither list gets modified, but a third list is created and returned.
Is it therefore valid to think of monadic operations as implicitly taking an initial state object as a parameter and implicitly returning a final state object? These state objects would be hidden so that the programmer doesn't have to worry about them and to control how they gets accessed. So, the programmer would not try to copy the object representing the IO stream as it was ten minutes ago.
In other words, if we have this code:
main = do
putStrLn "Enter your name:"
name <- getLine
putStrLn ( "Hello " ++ name )
...is it OK to think of the IO monad and the "do" syntax as representing this style of code?
putStrLn :: IOState -> String -> IOState
getLine :: IOState -> (IOState, String)
main :: IOState -> IOState
-- main returns an IOState we can call "state3"
main state0 = putStrLn state2 ("Hello " ++ name)
where (state2, name) = getLine state1
state1 = putStrLn state0 "Enter your name:"
No, that's not what monads in general do. However, your analogy is in fact exactly correct with regards to the data type State s a, which happens to be a monad. State is defined like this:
newtype State s a = State { runState :: s -> (a, s) }
...where the type variable s is the state value and a is the "regular" value that you use. So a value in "the State monad" is just a function from an initial state, to a return value and final state. The monadic style, as applied to State, does nothing more than automatically thread a state value through a sequence of functions.
The ST monad is superficially similar, but uses magic to allow computations with real side-effects, but only such that the side effects can't be observed from outside particular uses of ST.
The IO monad is essentially an ST monad set to "more magic", with side effects that touch the outside world and only a single point where IO computations are run, namely the entry point for the entire program. Still, on some conceptual level, you can still think of it as threading a "state" value through functions the way regular State does.
However, other monads don't necessarily have anything whatsoever to do with threading state, or sequencing functions, or whatnot. The operations needed for something to be a monad are incredibly general and abstract. For instance, using Maybe or Either as monads lets you use functions that might return errors, with the monadic style handling escaping from the computation when an error occurs the same way that State threads a state value. Using lists as a monad gives you nondeterminism, letting you simultaneously apply functions to multiple inputs and see all possible outputs, with the monadic style automatically applying the function to each argument and collecting all the outputs.
Is it therefore valid to think of monadic operations as implicitly taking an initial state object as a parameter and implicitly returning a final state object?
This seems to be a common sticking point for learning monads, i.e., trying to figure out how a single magical monad primordial soup is simultaneously useful for representing stateful computations, computations that can fail, nondeterministic computations, exceptions, continuations, sequencing side effects, and so on.
Threading state through a sequence of stateful computations is one single example of an operation that satisfies the monad laws.
You're correct in observing that the State and IO monads are close cousins, but your analogy will fall apart if you try inserting, say, the list monad.
Not monads in general, but for the IO monad, yes — in fact, the type IO a is often defined as the function type RealWorld -> (RealWorld, a). So in this view, the desugared type of putStrLn is String -> RealWorld -> (RealWorld, ()) and getChar is RealWorld -> (RealWorld, Char) — and we only partially apply it, with the monadic bind taking care of fully evaluating it and passing the RealWorld around. (GHC's ST library actually includes a very real RealWorld type, though it's described as "deeply magical" and not for actual use.)
There are many other monads that don't have this property, though. There's no RealWorld being passed around with, for example, the monads [1,2,3,4] or Just "Hello".
Absolutely not. This is not what monads are in general. You can use monads to implicitly pass around data, but that is just one application. If you use this model of monads then you are going to miss out on a lot of the really cool things monads can do.
Instead, think about monads as being pieces of data that represent a computation. For example, there is a sense in which implicitly passing around data isn't pure because pure languages insist that you be explicit about all of your arguments and return types. So if you want to pass around data implicitly you can do this: define a new data type that is a representation of doing something impure, and then write a piece of code to work with that.
An extreme example (just a theoretical example, you're unlikely to want to do this) would be this: C allows impure computations, so you could define a type that represents a piece of C code. You can then write an interpreter that takes one of these C structures and interprets it. All monads are like this, though usually much simpler than a C interpreter. But this view is more powerful because it also explains the monads that aren't about passing around hidden state.
You should probably also try to resist the temptation to see IO as passing around a hidden world state. The internal implementation of IO has nothing to do with how you should think about IO. The IO monad is about building representations of the I/O you want to perform. You return one of these representations to the Haskell run-time system and it's up to that system to implement your representation however it wants.
Any time you want to do something, and you can't see how to directly implement it in a pure way, but you can see how to build a pure data structure to describe want you want, you may have an application for a monad, especially if your structure is naturally a tree of some sort.
I prefer to think about monads as objects that represents delayed actions (runXXX, main) with result which can be combined according to that result.
return "somthing"
actionA >>= \x -> makeActionB x
And those action not necessary to be state-full. I.e. you can think about monad of function construction like this:
instance Monad ((->) a) where
m >>= fm = \x -> fm (m x) x
return = const
sqr = \x -> x*x
cube = \x -> x*x*x
weird = do
a <- sqr
b <- cube
return (a+b)
I'm trying a little experiment in haskell, wondering if it is possible to exploit laziness to process IO. I'd like to write a function that takes a String (a list of Chars) and produces a string, lazily. I would like then to be abily to lazily feed it characters from IO, so each character would be processed as soon as it was available, and the output would be produced as the characters necessary became available. However, I'm not quite sure if/how I can produce a lazy list of characters from input inside the IO monad.
Regular String IO in Haskell is lazy. So your example should just work out of the box.
Here's an example, using the 'interact' function, which applies a function to a lazy stream of characters:
interact :: (String -> String) -> IO ()
Let's filter out the letter 'e' from the input stream, lazily (i.e. run in constant space):
main = interact $ filter (/= 'e')
You could also use getContents and putStr if you like. They're all lazy.
Running it to filter the letter 'e' from the dictionary:
$ ghc -O2 --make A.hs
$ ./A +RTS -s < /usr/share/dict/words
...
2 MB total memory in use (0 MB lost due to fragmentation)
...
so we see that it ran in a constant 2M footprint.
The simplest method of doing lazy IO involves functions such as interact, readFile, hGetContents, and such, as dons says; there's a more extended discussion of these in the book Real World Haskell that you might find useful. If memory serves me, all such functions are eventually implemented using the unsafeInterleaveIO that ephemient mentions, so you can also build your own functions that way if you want.
On the other hand, it might be wise to note that unsafeInterleaveIO is exactly what it says on the tin: unsafe IO. Using it--or functions based on it--breaks purity and referential transparency. This allows apparently pure functions (that is, that do not return an IO action) to effect the outside world when evaluated, produce different results from the same arguments, and all those other unpleasant things. In practice, most sensible ways of using unsafeInterleaveIO won't cause problems, and simple mistakes will usually result in obvious and easily diagnosed bugs, but you've lost some nice guarantees.
There are alternatives, of course; you can find assorted libraries on Hackage that provide restricted, safer lazy IO or conceptually different approaches. However, given that problems arise only rarely in practical use, I think most people are inclined to stick with the built-in, technically unsafe functions.
unsafeInterleaveIO :: IO a -> IO a
unsafeInterleaveIO allos IO computation to be deferred lazily. When passed a value of type IO a, the IO will only be performed when the value of a is demanded. This is used to implement lazy file reading, see System.IO.hGetContents.
For example, main = getContents >>= return . map Data.Char.toUpper >>= putStr is lazy; as you feed characters to stdin, you will get characters on stdout.
(This is the same as writing main = interact $ map Data.Char.toUpper, as in dons's answer.)