I've looked over https://www.fpcomplete.com/blog/2017/06/tale-of-two-brackets, though skimming some parts, and I still don't quite understand the core issue "StateT is bad, IO is OK", other than vaguely getting the sense that Haskell allows one to write bad StateT monads (or in the ultimate example in the article, MonadBaseControl instead of StateT, I think).
In the haddocks, the following law must be satisfied:
askUnliftIO >>= (\u -> liftIO (unliftIO u m)) = m
So this appears to be saying that state is not mutated in the monad m when using askUnliftIO. But to my mind, in IO, the entire world can be the state. I could be reading and writing to a text file on disk, for instance.
To quote another article by Michael,
False purity We say WriterT and StateT are pure, and technically they
are. But let's be honest: if you have an application which is entirely
living within a StateT, you're not getting the benefits of restrained
mutation that you want from pure code. May as well call a spade a
spade, and accept that you have a mutable variable.
This makes me think this is indeed the case: with IO we are being honest, with StateT, we are not being honest about mutability ... but that seems another issue than what the law above is trying to show; after all, MonadUnliftIO is assuming IO. I'm having trouble understanding conceptually how IO is more restrictive than something else.
Update 1
After sleeping (some), I am still confused but am gradually getting less so as the day wears on. I worked out the law proof for IO. I realized the presence of id in the README. In particular,
instance MonadUnliftIO IO where
askUnliftIO = return (UnliftIO id)
So askUnliftIO would appear to return an IO (IO a) on an UnliftIO m.
Prelude> fooIO = print 5
Prelude> :t fooIO
fooIO :: IO ()
Prelude> let barIO :: IO(IO ()); barIO = return fooIO
Prelude> :t barIO
barIO :: IO (IO ())
Back to the law, it really appears to be saying that state is not mutated in the monad m when doing a round trip on the transformed monad (askUnliftIO), where the round trip is unLiftIO -> liftIO.
Resuming the example above, barIO :: IO (), so if we do barIO >>= (u -> liftIO (unliftIO u m)), then u :: IO () and unliftIO u == IO (), then liftIO (IO ()) == IO (). **So since everything has basically been applications of id under the hood, we can see that no state was changed, even though we are using IO. Crucially, I think, what is important is that the value in a is never run, nor is any other state modified, as a result of using askUnliftIO. If it did, then like in the case of randomIO :: IO a, we would not be able to get the same value had we not run askUnliftIO on it. (Verification attempt 1 below)
But, it still seems like we could do the same for other Monads, even if they do maintain state. But I also see how, for some monads, we may not be able to do so. Thinking of a contrived example: each time we access the value of type a contained in the stateful monad, some internal state is changed.
Verification attempt 1
> fooIO >> askUnliftIO
5
> fooIOunlift = fooIO >> askUnliftIO
> :t fooIOunlift
fooIOunlift :: IO (UnliftIO IO)
> fooIOunlift
5
Good so far, but confused about why the following occurs:
> fooIOunlift >>= (\u -> unliftIO u)
<interactive>:50:24: error:
* Couldn't match expected type `IO b'
with actual type `IO a0 -> IO a0'
* Probable cause: `unliftIO' is applied to too few arguments
In the expression: unliftIO u
In the second argument of `(>>=)', namely `(\ u -> unliftIO u)'
In the expression: fooIOunlift >>= (\ u -> unliftIO u)
* Relevant bindings include
it :: IO b (bound at <interactive>:50:1)
"StateT is bad, IO is OK"
That's not really the point of the article. The idea is that MonadBaseControl permits some confusing (and often undesirable) behaviors with stateful monad transformers in the presence of concurrency and exceptions.
finally :: StateT s IO a -> StateT s IO a -> StateT s IO a is a great example. If you use the "StateT is attaching a mutable variable of type s onto a monad m" metaphor, then you might expect that the finalizer action gets access to the most recent s value when an exception was thrown.
forkState :: StateT s IO a -> StateT s IO ThreadId is another one. You might expect that the state modifications from the input would be reflected in the original thread.
lol :: StateT Int IO [ThreadId]
lol = do
for [1..10] $ \i -> do
forkState $ modify (+i)
You might expect that lol could be rewritten (modulo performance) as modify (+ sum [1..10]). But that's not right. The implementation of forkState just passes the initial state to the forked thread, and then can never retrieve any state modifications. The easy/common understanding of StateT fails you here.
Instead, you have to adopt a more nuanced view of StateT s m a as "a transformer that provides a thread-local immutable variable of type s which is implicitly threaded through a computation, and it is possible to replace that local variable with a new value of the same type for future steps of the computation." (more or less a verbose english retelling of the s -> m (a, s)) With this understanding, the behavior of finally becomes a bit more clear: it's a local variable, so it does not survive exceptions. Likewise, forkState becomes more clear: it's a thread-local variable, so obviously a change to a different thread won't affect any others.
This is sometimes what you want. But it's usually not how people write code IRL and it often confuses people.
For a long time, the default choice in the ecosystem to do this "lowering" operation was MonadBaseControl, and this had a bunch of downsides: hella confusing types, difficult to implement instances, impossible to derive instances, sometimes confusing behavior. Not a great situation.
MonadUnliftIO restricts things to a simpler set of monad transformers, and is able to provide relatively simple types, derivable instances, and always predictable behavior. The cost is that ExceptT, StateT, etc transformers can't use it.
The underlying principle is: by restricting what is possible, we make it easier to understand what might happen. MonadBaseControl is extremely powerful and general, and quite difficult to use and confusing as a result. MonadUnliftIO is less powerful and general, but it's much easier to use.
So this appears to be saying that state is not mutated in the monad m when using askUnliftIO.
This isn't true - the law is stating that unliftIO shouldn't do anything with the monad transformer aside from lowering it into IO. Here's something that breaks that law:
newtype WithInt a = WithInt (ReaderT Int IO a)
deriving newtype (Functor, Applicative, Monad, MonadIO, MonadReader Int)
instance MonadUnliftIO WithInt where
askUnliftIO = pure (UnliftIO (\(WithInt readerAction) -> runReaderT 0 readerAction))
Let's verify that this breaks the law given: askUnliftIO >>= (\u -> liftIO (unliftIO u m)) = m.
test :: WithInt Int
test = do
int <- ask
print int
pure int
checkLaw :: WithInt ()
checkLaw = do
first <- test
second <- askUnliftIO >>= (\u -> liftIO (unliftIO u test))
when (first /= second) $
putStrLn "Law violation!!!"
The value returned by test and the askUnliftIO ... lowering/lifting are different, so the law is broken. Furthermore, the observed effects are different, which isn't great either.
Related
There seems to be some undocumented knowledge about the difference between Monad IO and IO. Remarks here and here) hint that IO a can be used in negative position but may have unintended consequences:
Citing Snoyman 1:
However, we know that some control flows (such as exception handling)
are not being used, since they are not compatible with MonadIO.
(Reason: MonadIO requires that the IO be in positive, not negative,
position.) This lets us know, for example, that foo is safe to use in
a continuation-based monad like ContT or Conduit.
And Kmett 2:
I tend to export functions with a MonadIO constraint... whenever it
doesn't have to take an IO-like action in negative position (as an
argument).
When my code does have to take another monadic action as an argument,
then I usually have to stop and think about it.
Is there danger in such functions that programmers should know about?
Does it for example mean that running arbitrary continuation-based action may redefine control flow giving unexpected results in ways that Monad IO based interface are safe from?
Is there danger in such functions that programmers should know about?
There is not danger. Quite the opposite, the point Snoyman and Kmett are making is that Monad IO doesn't let you lift through things with IO in a negative positive.
Suppose you want to generalize putStrLn :: String -> IO (). You can, because the IO is in a positive position:
putStrLn' :: MonadIO m => String -> m ()
putStrLn' str = liftIO (putStrLn str)
Now, suppose you want to generalize handle :: Exception e => (e -> IO a) -> IO a -> IO a. You can't (at least not with just MonadIO):
handle' :: (MonadIO m, Exception e) => (e -> m a) -> m a -> m a
handle' handler act = liftIO (handle (handler . unliftIO) (unliftIO act))
unliftIO :: MonadIO m => m a -> IO a
unliftIO = error "MonadIO isn't powerful enough to make this implementable!"
You need something more. If you're curious about how you'd do that, take a look at the implementation of functions in lifted-base. For instance: handle :: (MonadBaseControl IO m, Exception e) => (e -> m a) -> m a -> m a.
Though it is okay to bind IO [[Char]] and IO () but its not allowed to bind Maybe with IO. Can someone give an example how this relaxation would lead to a bad design? Why freedom in the polymorphic type of Monad is allowed though not the Monad itself?
There are a lot of good theoretical reasons, including "that's not what Monad is." But let's step away from that for a moment and just look at the implementation details.
First off - Monad isn't magic. It's just a standard type class. Instances of Monad only get created when someone writes one.
Writing that instance is what defines how (>>) works. Usually it's done implicitly through the default definition in terms of (>>=), but that just is evidence that (>>=) is the more general operator, and writing it requires making all the same decisions that writing (>>) would take.
If you had a different operator that worked on more general types, you have to answer two questions. First, what would the types be? Second, how would you go about providing implementations? It's really not clear what the desired types would be, from your question. One of the following, I guess:
class Poly1 m n where
(>>) :: m a -> n b -> m b
class Poly2 m n where
(>>) :: m a -> n b -> n b
class Poly3 m n o | m n -> o where
(>>) :: m a -> n b -> o b
All of them could be implemented. But you lose two really important factors for using them practically.
You need to write an instance for every pair of types you plan to use together. This is a massively more complex undertaking than just an instance for each type. Something about n vs n^2.
You lose predictability. What does the operation even do? Here's where theory and practice intersect. The theory behind Monad places a lot of restrictions on the operations. Those restrictions are referred to as the "monad laws". They are beyond the ability to verify in Haskell, but any Monad instance that doesn't obey them is considered to be buggy. The end result is that you quickly can build an intuition for what the Monad operations do and don't do. You can use them without looking up the details of every type involved, because you know properties that they obey. None of those possible classes I suggested give you any kind of assurances like that. You just have no idea what they do.
I’m not sure that I understand your question correctly, but it’s definitely possible to compose Maybe with IO or [] in the same sense that you can compose IO with [].
For example, if you check the types in GHCI using :t,
getContents >>= return . lines
gives you an IO [String]. If you add
>>= return . map Text.Read.readMaybe
you get a type of IO [Maybe a], which is a composition of IO, [] and Maybe. You could then pass it to
>>= return . Data.Maybe.catMaybes
to flatten it to an IO [a]. Then you might pass the list of parsed valid input lines to a function that flattens it again and computes an output.
Putting this together, the program
import Text.Read (readMaybe)
import Data.Maybe (catMaybes)
main :: IO ()
main = getContents >>= -- IO String
return . lines >>= -- IO [String]
return . map readMaybe >>= -- IO [Maybe Int]
return . catMaybes >>= -- IO [Int]
return . (sum :: [Int] -> Int) >>= -- IO Int
print -- IO ()
with the input:
1
2
Ignore this!
3
prints 6.
It would also be possible to work with an IO (Maybe [String]), a Maybe [IO String], etc.
You can do this with >> as well. Contrived example: getContents >> (return . Just) False reads the input, ignores it, and gives you back an IO (Maybe Bool).
mtl doesn't care that much about the order of the monads in the monad stack, as long as there is a viable implementation. For example:
foo :: (MonadError () m, MonadState Int m) => m ()
foo = (put 1 >> throwError ()) `catchError` return
main :: IO ()
main = do
print $ runExcept $ flip execStateT 0 $ foo -- returns Right 0
print $ flip execState 0 $ runExceptT $ foo -- returns 1
Suppose I wanted to create a mini-mtl framework (for the sake of simplicity, one which covered only ExceptT and StateT) that was sensitive to the order of composition, and let me express things like
this function requires an error effect with priority over the
state effect, but I'm letting you choose the exact error and state
implementations.
or
this function requires a state effect with priority over the
error effect, but I'm letting you choose the exact error and state
implementations.
in such a way that only the stacks with the correct order had the required instances.
I don't know how to embed effect priority in the typeclass constraints. Would I need some kind of "constraint transformer" of kind Constraint -> Constraint? Perhaps some constraint parameterized by a type-level list of constraints?
Does s in STRef s a get instantiated with a concrete type? One could easily imagine some code where STRef is used in a context where the a takes on Int. But there doesn't seem to be anything for the type inference to give s a concrete type.
Imagine something in pseudo Java like MyList<S, A>. Even if S never appeared in the implementation of MyList instantiating a concrete type like MyList<S, Integer> where a concrete type is not used in place of S would not make sense. So how can STRef s a work?
tl;dr - in practice it seems it always gets initialised to RealWorld in the end
The source notes that s can be instantiated to RealWorld inside invocations of stToIO, but is otherwise uninstantiated:
-- The s parameter is either
-- an uninstantiated type variable (inside invocations of 'runST'), or
-- 'RealWorld' (inside invocations of 'Control.Monad.ST.stToIO').
Looking at the actual code for ST however it seems runST uses a specific value realWorld#:
newtype ST s a = ST (STRep s a)
type STRep s a = State# s -> (# State# s, a #)
runST :: (forall s. ST s a) -> a
runST st = runSTRep (case st of { ST st_rep -> st_rep })
runSTRep :: (forall s. STRep s a) -> a
runSTRep st_rep = case st_rep realWorld# of
(# _, r #) -> r
realWorld# is defined as a magic primitive inside the GHC source code:
realWorldName = mkWiredInIdName gHC_PRIM (fsLit "realWorld#")
realWorldPrimIdKey realWorldPrimId
realWorldPrimId :: Id -- :: State# RealWorld
realWorldPrimId = pcMiscPrelId realWorldName realWorldStatePrimTy
(noCafIdInfo `setUnfoldingInfo` evaldUnfolding
`setOneShotInfo` stateHackOneShot)
You can also confirm this in ghci:
Prelude> :set -XMagicHash
Prelude> :m +GHC.Prim
Prelude GHC.Prim> :t realWorld#
realWorld# :: State# RealWorld
From your question I can not see if you understand why the phantom s type is there at all. Even if you did not ask for this explicitly, let me elaborate on that.
The role of the phantom type
The main use of the phantom type is to constrain references (aka pointers) to stay "inside" the ST monad. Roughly, the dynamically allocated data must end its life when runST returns.
To see the issue, let's pretend that the type of runST were
runST :: ST s a -> a
Then, consider this:
data Dummy
let var :: STRef Dummy Int
var = runST (newSTRef 0)
change :: () -> ()
change = runST (modifySTRef var succ)
access :: () -> Int
result :: (Int, ())
result = (access() , change())
in result
(Above I added a few useless () arguments to make it similar to imperative code)
Now what should be the result of the code above? It could be either (0,()) or (1,()) depending on the evaluation order. This is a big no-no in the pure Haskell world.
The issue here is that var is a reference which "escaped" from its runST. When you escape the ST monad, you are no longer forced to use the monad operator >>= (or equivalently, the do notation to sequentialize the order of side effects. If references are still around, then we can still have side effects around when there should be none.
To avoid the issue, we restrict runST to work on ST s a where a does not depend on s. Why this? Because newSTRef returns a STRef s a, a reference to a tagged with the phantom type s, hence the return type depends on s and can not be extracted from the ST monad through runST.
Technically, this restriction is done by using a rank-2 type:
runST :: (forall s. ST s a) -> a
the "forall" here is used to implement the restriction. The type is saying: choose any a you wish, then provide a value of type ST s a for any s I wish, then I will return an a. Mind that s is chosen by runST, not by the caller, so it could be absolutely anything. So, the type system will accept an application runST action only if action :: forall s. ST s a where s is unconstrained, and a does not involve s (recall that the caller has to choose a before runST chooses s).
It is indeed a slightly hackish trick to implement the independence constraint, but it does work.
On the actual question
To connect this to your actual question: in the implementation of runST, s will be chosen to be any concrete type. Note that, even if s were simply chosen to be Int inside runST it would not matter much, because the type system has already constrained a to be independent from s, hence to be reference-free. As #Ganesh pointed out, RealWorld is the type used by GHC.
You also mentioned Java. One could attempt to play a similar trick in Java as follows: (warning, overly simplified code follows)
interface ST<S,A> { A call(); }
interface STAction<A> { <S> ST<S,A> call(S dummy); }
...
<A> A runST(STAction<A> action} {
RealWorld dummy = new RealWorld();
return action.call(dummy).call();
}
Above in STAction parameter A can not depend on S.
I have some code that currently uses a ST monad for evaluation. I like not putting IO everywhere because the runST method produces a pure result, and indicates that such result is safe to call (versus unsafePerformIO). However, as some of my code has gotten longer, I do want to put debugging print statements in.
Is there any class that provides a dual-personality monad [or typeclass machinery], one which can be a ST or an IO (depending on its type or a "isDebug" flag)? I recall SPJ introduced a "Mutation" class in his "Fun with Type Functions" paper, which used associative types to relate IO to IORef and ST to STRef. Does such exist as a package somewhere?
Edit / solution
Thanks very much [the nth time], C.A. McCann! Using that solution, I was able to introduce an additional class for monads which support a pdebug function. The ST monad will ignore these calls, whereas IO will run putStrLn.
class DebugMonad m where
pdebug :: String -> m ()
instance DebugMonad (ST s) where
pdebug _ = return ()
instance DebugMonad IO where
pdebug = putStrLn
test initV = do
v <- newRef initV
modifyRef v (+1)
pdebug "debug"
readRef v
testR v = runST $ test v
This has a very fortunate consequence in ghci. Since it expects expressions to be IO types by default, running something like "test 3" will result in the IO monad being run, so you can debug it easily, and then call it with something like "testR" when you actually want to run it.
If you want a unified interface to IORef and STRef, have you looked at the stateref package? It has type classes for "references to mutable data", separated for readable, writable, etc., with instances for IORef and STRef, as well as things like TVar, MVar, ForeignPtr, etc.
Have you considered Debug.Trace.trace instead?
http://www.haskell.org/haskellwiki/Debugging