There seems to be some undocumented knowledge about the difference between Monad IO and IO. Remarks here and here) hint that IO a can be used in negative position but may have unintended consequences:
Citing Snoyman 1:
However, we know that some control flows (such as exception handling)
are not being used, since they are not compatible with MonadIO.
(Reason: MonadIO requires that the IO be in positive, not negative,
position.) This lets us know, for example, that foo is safe to use in
a continuation-based monad like ContT or Conduit.
And Kmett 2:
I tend to export functions with a MonadIO constraint... whenever it
doesn't have to take an IO-like action in negative position (as an
argument).
When my code does have to take another monadic action as an argument,
then I usually have to stop and think about it.
Is there danger in such functions that programmers should know about?
Does it for example mean that running arbitrary continuation-based action may redefine control flow giving unexpected results in ways that Monad IO based interface are safe from?
Is there danger in such functions that programmers should know about?
There is not danger. Quite the opposite, the point Snoyman and Kmett are making is that Monad IO doesn't let you lift through things with IO in a negative positive.
Suppose you want to generalize putStrLn :: String -> IO (). You can, because the IO is in a positive position:
putStrLn' :: MonadIO m => String -> m ()
putStrLn' str = liftIO (putStrLn str)
Now, suppose you want to generalize handle :: Exception e => (e -> IO a) -> IO a -> IO a. You can't (at least not with just MonadIO):
handle' :: (MonadIO m, Exception e) => (e -> m a) -> m a -> m a
handle' handler act = liftIO (handle (handler . unliftIO) (unliftIO act))
unliftIO :: MonadIO m => m a -> IO a
unliftIO = error "MonadIO isn't powerful enough to make this implementable!"
You need something more. If you're curious about how you'd do that, take a look at the implementation of functions in lifted-base. For instance: handle :: (MonadBaseControl IO m, Exception e) => (e -> m a) -> m a -> m a.
Related
I've looked over https://www.fpcomplete.com/blog/2017/06/tale-of-two-brackets, though skimming some parts, and I still don't quite understand the core issue "StateT is bad, IO is OK", other than vaguely getting the sense that Haskell allows one to write bad StateT monads (or in the ultimate example in the article, MonadBaseControl instead of StateT, I think).
In the haddocks, the following law must be satisfied:
askUnliftIO >>= (\u -> liftIO (unliftIO u m)) = m
So this appears to be saying that state is not mutated in the monad m when using askUnliftIO. But to my mind, in IO, the entire world can be the state. I could be reading and writing to a text file on disk, for instance.
To quote another article by Michael,
False purity We say WriterT and StateT are pure, and technically they
are. But let's be honest: if you have an application which is entirely
living within a StateT, you're not getting the benefits of restrained
mutation that you want from pure code. May as well call a spade a
spade, and accept that you have a mutable variable.
This makes me think this is indeed the case: with IO we are being honest, with StateT, we are not being honest about mutability ... but that seems another issue than what the law above is trying to show; after all, MonadUnliftIO is assuming IO. I'm having trouble understanding conceptually how IO is more restrictive than something else.
Update 1
After sleeping (some), I am still confused but am gradually getting less so as the day wears on. I worked out the law proof for IO. I realized the presence of id in the README. In particular,
instance MonadUnliftIO IO where
askUnliftIO = return (UnliftIO id)
So askUnliftIO would appear to return an IO (IO a) on an UnliftIO m.
Prelude> fooIO = print 5
Prelude> :t fooIO
fooIO :: IO ()
Prelude> let barIO :: IO(IO ()); barIO = return fooIO
Prelude> :t barIO
barIO :: IO (IO ())
Back to the law, it really appears to be saying that state is not mutated in the monad m when doing a round trip on the transformed monad (askUnliftIO), where the round trip is unLiftIO -> liftIO.
Resuming the example above, barIO :: IO (), so if we do barIO >>= (u -> liftIO (unliftIO u m)), then u :: IO () and unliftIO u == IO (), then liftIO (IO ()) == IO (). **So since everything has basically been applications of id under the hood, we can see that no state was changed, even though we are using IO. Crucially, I think, what is important is that the value in a is never run, nor is any other state modified, as a result of using askUnliftIO. If it did, then like in the case of randomIO :: IO a, we would not be able to get the same value had we not run askUnliftIO on it. (Verification attempt 1 below)
But, it still seems like we could do the same for other Monads, even if they do maintain state. But I also see how, for some monads, we may not be able to do so. Thinking of a contrived example: each time we access the value of type a contained in the stateful monad, some internal state is changed.
Verification attempt 1
> fooIO >> askUnliftIO
5
> fooIOunlift = fooIO >> askUnliftIO
> :t fooIOunlift
fooIOunlift :: IO (UnliftIO IO)
> fooIOunlift
5
Good so far, but confused about why the following occurs:
> fooIOunlift >>= (\u -> unliftIO u)
<interactive>:50:24: error:
* Couldn't match expected type `IO b'
with actual type `IO a0 -> IO a0'
* Probable cause: `unliftIO' is applied to too few arguments
In the expression: unliftIO u
In the second argument of `(>>=)', namely `(\ u -> unliftIO u)'
In the expression: fooIOunlift >>= (\ u -> unliftIO u)
* Relevant bindings include
it :: IO b (bound at <interactive>:50:1)
"StateT is bad, IO is OK"
That's not really the point of the article. The idea is that MonadBaseControl permits some confusing (and often undesirable) behaviors with stateful monad transformers in the presence of concurrency and exceptions.
finally :: StateT s IO a -> StateT s IO a -> StateT s IO a is a great example. If you use the "StateT is attaching a mutable variable of type s onto a monad m" metaphor, then you might expect that the finalizer action gets access to the most recent s value when an exception was thrown.
forkState :: StateT s IO a -> StateT s IO ThreadId is another one. You might expect that the state modifications from the input would be reflected in the original thread.
lol :: StateT Int IO [ThreadId]
lol = do
for [1..10] $ \i -> do
forkState $ modify (+i)
You might expect that lol could be rewritten (modulo performance) as modify (+ sum [1..10]). But that's not right. The implementation of forkState just passes the initial state to the forked thread, and then can never retrieve any state modifications. The easy/common understanding of StateT fails you here.
Instead, you have to adopt a more nuanced view of StateT s m a as "a transformer that provides a thread-local immutable variable of type s which is implicitly threaded through a computation, and it is possible to replace that local variable with a new value of the same type for future steps of the computation." (more or less a verbose english retelling of the s -> m (a, s)) With this understanding, the behavior of finally becomes a bit more clear: it's a local variable, so it does not survive exceptions. Likewise, forkState becomes more clear: it's a thread-local variable, so obviously a change to a different thread won't affect any others.
This is sometimes what you want. But it's usually not how people write code IRL and it often confuses people.
For a long time, the default choice in the ecosystem to do this "lowering" operation was MonadBaseControl, and this had a bunch of downsides: hella confusing types, difficult to implement instances, impossible to derive instances, sometimes confusing behavior. Not a great situation.
MonadUnliftIO restricts things to a simpler set of monad transformers, and is able to provide relatively simple types, derivable instances, and always predictable behavior. The cost is that ExceptT, StateT, etc transformers can't use it.
The underlying principle is: by restricting what is possible, we make it easier to understand what might happen. MonadBaseControl is extremely powerful and general, and quite difficult to use and confusing as a result. MonadUnliftIO is less powerful and general, but it's much easier to use.
So this appears to be saying that state is not mutated in the monad m when using askUnliftIO.
This isn't true - the law is stating that unliftIO shouldn't do anything with the monad transformer aside from lowering it into IO. Here's something that breaks that law:
newtype WithInt a = WithInt (ReaderT Int IO a)
deriving newtype (Functor, Applicative, Monad, MonadIO, MonadReader Int)
instance MonadUnliftIO WithInt where
askUnliftIO = pure (UnliftIO (\(WithInt readerAction) -> runReaderT 0 readerAction))
Let's verify that this breaks the law given: askUnliftIO >>= (\u -> liftIO (unliftIO u m)) = m.
test :: WithInt Int
test = do
int <- ask
print int
pure int
checkLaw :: WithInt ()
checkLaw = do
first <- test
second <- askUnliftIO >>= (\u -> liftIO (unliftIO u test))
when (first /= second) $
putStrLn "Law violation!!!"
The value returned by test and the askUnliftIO ... lowering/lifting are different, so the law is broken. Furthermore, the observed effects are different, which isn't great either.
Real World Haskell states that "Transformer stacking order is important". However, I can't seem to figure out if there's a difference between ExceptT (ResourceT m) a and ResourceT (ExceptT m) a. Will they interfere with each other?
In this example, there is no real difference between both orders. The reason being: unlike many transformers including ExceptT, the resource transformer does not “inject” its own doings into the base monad you apply it to, but rather start off the entire action with passing in the release references.
If you write out the types (I'll refer to MaybeT instead of ExceptT for the sake of simplicity; they're obviously equivalent for the purpose of this question) then you have basically
type MaybeResourceT m a = MaybeT (IORef RelMap -> m a)
= IORef RelMap -> m (Maybe a)
type ResourceMaybeT m a = ResourceT (m (Maybe a))
= IORef RelMap -> m (Maybe a)
i.e. actually equivalent types. I suppose you could also show that for the operations.
I'm testing a REST server. I hit it in the IO monad and simulate it in State Db where Db tracks the supposed state of the server. The following function is supposed to run both versions and compare the results...
check :: (Eq a, MonadState d s) => s a -> IO a -> s (IO Bool)
-- or: check :: (Eq a, MonadState d s, MonadIO i) => s a -> i a -> s (i Bool)
check _ _ = (return.return) False -- for now
but when I try it with these simplest possible functions ...
simReset :: State Db ()
realReset :: IO ()
reset :: StateT Db IO Bool
reset = check simReset realReset
I get this error:
Couldn't match expected type `Bool' with actual type `IO Bool'
Expected type: StateT Db IO Bool
Actual type: StateT Db IO (IO Bool)
In the return type of a call of `check'
In the expression: check simReset realReset
Why? And how do I fix it?
(This topic started here: Lift to fix the *inside* of a monad transformer stack)
In your implementation, check is going to return an IO Bool regardless of what the state monad s is. So, when you pass simReset to check, which is a monadic action in the State Db monad, the return value is going to be State Db (IO Bool).
How to fix it depends on what you're trying to do. Based on your implementation of reset it seems like you're trying to interface with a transformer stack of the form StateT Db IO a. In this case, you're describing a kind of program in the context of StateT Db IO. There are two ways to go about this:
You can upgrade simReset to have type MonadState Db s => s () (This shouldn't actually require any implementation change).
You can define an auxiliary function that "hoists" it into the proper monad
For example:
hoistState :: Monad m => State s a -> StateT s m a
hoistState prg = StateT $ \st -> return $ runState prg st
In either case, you'll probably want to keep the action that you're performing and the result in the same monad:
check :: (Eq a, MonadIO s, MonadState d s) => s a -> IO a -> s Bool
Then, with solution one, you have
reset = check simReset realReset
And with solution two you have:
reset = check (hoistState simReset) realReset
In automated theorem proving (proof search), I compose transformers of type
t :: Claim -> IO (Maybe (Claim, Proof -> Proof))
such that: when t c returns Just (c', f), then c' implies c and a proof for c is obtained from a proof p' for c' by computing f p'.
Is this a lens, somehow? (If yes, what would it help?)
There's also a more general case (for several, or zero, subgoals)
ts :: Claim -> IO (Maybe ([Claim], [Proof] -> Proof))
The IO part is important because these transformers do substantial work (calling external processes), and I might want to impose timeouts.
I can't readily see how lenses could help with that. However, reordering your monadic stack and using monad transformers should make composition much easier, and also open up the possibility of abstracting from IO in the cases you don't need impurity:
t' :: Claim -> MaybeT IO (Claim, Proof -> Proof)
If you want or need to keep using an existing implementation of t in spite of the more cumbersome type, you can lift its result to MaybeT with:
(MaybeT . return =<<) . lift :: m (Maybe b) -> MaybeT m b
It is worth noting that Claim -> (Claim, Proof -> Proof) is equivalent to State Claim (Proof -> Proof), so it might be possible to go even further:
t'' :: StateT Claim (MaybeT IO) (Proof -> Proof)
While hacking something up earlier, I created the following code:
newtype Callback a = Callback { unCallback :: a -> IO (Callback a) }
liftCallback :: (a -> IO ()) -> Callback a
liftCallback f = let cb = Callback $ \x -> (f x >> return cb) in cb
runCallback :: Callback a -> IO (a -> IO ())
runCallback cb =
do ref <- newIORef cb
return $ \x -> readIORef ref >>= ($ x) . unCallback >>= writeIORef ref
Callback a represents a function that handles some data and returns a new callback that should be used for the next notification. A callback which can basically replace itself, so to speak. liftCallback just lifts a normal function to my type, while runCallback uses an IORef to convert a Callback to a simple function.
The general structure of the type is:
data T m a = T (a -> m (T m a))
It looks much like this could be isomorphic to some well-known mathematical structure from category theory.
But what is it? Is it a monad or something? An applicative functor? A transformed monad? An arrow, even? Is there a search engine similar Hoogle that lets me search for general patterns like this?
The term you are looking for is free monad transformer. The best place to learn how these work is to read the "Coroutine Pipelines" article in issue 19 of The Monad Reader. Mario Blazevic gives a very lucid description of how this type works, except he calls it the "Coroutine" type.
I wrote up his type in the transformers-free package and then it got merged into the free package, which is its new official home.
Your Callback type is isomorphic to:
type Callback a = forall r . FreeT ((->) a) IO r
To understand free monad transformers, you need to first understand free monads, which are just abstract syntax trees. You give the free monad a functor which defines a single step in the syntax tree, and then it creates a Monad from that Functor that is basically a list of those types of steps. So if you had:
Free ((->) a) r
That would be a syntax tree that accepts zero or more as as input and then returns a value r.
However, usually we want to embed effects or make the next step of the syntax tree dependent on some effect. To do that, we simply promote our free monad to a free monad transformer, which interleaves the base monad between syntax tree steps. In the case of your Callback type, you are interleaving IO in between each input step, so your base monad is IO:
FreeT ((->) a) IO r
The nice thing about free monads is that they are automatically monads for any functor, so we can take advantage of this to use do notation to assemble our syntax tree. For example, I can define an await command that will bind the input within the monad:
import Control.Monad.Trans.Free
await :: (Monad m) => FreeT ((->) a) m a
await = liftF id
Now I have a DSL for writing Callbacks:
import Control.Monad
import Control.Monad.Trans.Free
printer :: (Show a) => FreeT ((->) a) IO r
printer = forever $ do
a <- await
lift $ print a
Notice that I never had to define the necessary Monad instance. Both FreeT f and Free f are automatically Monads for any functor f, and in this case ((->) a) is our functor, so it automatically does the right thing. That's the magic of category theory!
Also, we never had to define a MonadTrans instance in order to use lift. FreeT f is automatically a monad transformer, given any functor f, so it took care of that for us, too.
Our printer is a suitable Callback, so we can feed it values just by deconstructing the free monad transformer:
feed :: [a] -> FreeT ((->) a) IO r -> IO ()
feed as callback = do
x <- runFreeT callback
case x of
Pure _ -> return ()
Free k -> case as of
[] -> return ()
b:bs -> feed bs (k b)
The actual printing occurs when we bind runFreeT callback, which then gives us the next step in the syntax tree, which we feed the next element of the list.
Let's try it:
>>> feed [1..5] printer
1
2
3
4
5
However, you don't even need to write all this up yourself. As Petr pointed out, my pipes library abstracts common streaming patterns like this for you. Your callback is just:
forall r . Consumer a IO r
The way we'd define printer using pipes is:
printer = forever $ do
a <- await
lift $ print a
... and we can feed it a list of values like so:
>>> runEffect $ each [1..5] >-> printer
1
2
3
4
5
I designed pipes to encompass a very large range of streaming abstractions like these in such a way that you can always use do notation to build each streaming component. pipes also comes with a wide variety of elegant solutions for things like state and error handling, and bidirectional flow of information, so if you formulate your Callback abstraction in terms of pipes, you tap into a ton of useful machinery for free.
If you want to learn more about pipes, I recommend you read the tutorial.
The general structure of the type looks to me like
data T (~>) a = T (a ~> T (~>) a)
where (~>) = Kleisli m in your terms (an arrow).
Callback itself doesn't look like an instance of any standard Haskell typeclass I can think of, but it is a Contravariant Functor (also known as Cofunctor, misleadingly as it turns out). As it is not included in any of the libraries that come with GHC, there exist several definitions of it on Hackage (use this one), but they all look something like this:
class Contravariant f where
contramap :: (b -> a) -> f a -> f b
-- c.f. fmap :: (a -> b) -> f a -> f b
Then
instance Contravariant Callback where
contramap f (Callback k) = Callback ((fmap . liftM . contramap) f (f . k))
Is there some more exotic structure from category theory that Callback possesses? I don't know.
I think that this type is very close to what I have heard called a 'Circuit', which is a type of arrow. Ignoring for a moment the IO part (as we can have this just by transforming a Kliesli arrow) the circuit transformer is:
newtype CircuitT a b c = CircuitT { unCircuitT :: a b (c, CircuitT a b c) }
This is basicall an arrow that returns a new arrow to use for the next input each time. All of the common arrow classes (including loop) can be implemented for this arrow transformer as long as the base arrow supports them. Now, all we have to do to make it notionally the same as the type you mention is to get rid of that extra output. This is easily done, and so we find:
Callback a ~=~ CircuitT (Kleisli IO) a ()
As if we look at the right hand side:
CircuitT (Kleisli IO) a () ~=~
(Kliesli IO) a ((), CircuitT (Kleisli IO) a ()) ~=~
a -> IO ((), CircuitT (Kliesli IO) a ())
And from here, you can see how this is similar to Callback a, except we also output a unit value. As the unit value is in a tuple with something else anyway, this really doesn't tell us much, so I would say they're basically the same.
N.B. I used ~=~ for similar but not entirely equivalent to, for some reason. They are very closely similar though, in particular note that we could convert a Callback a into a CircuitT (Kleisli IO) a () and vice-versa.
EDIT: I would also fully agree with the ideas that this is A) a monadic costream (monadic operation expecitng an infinite number of values, I think this means) and B) a consume-only pipe (which is in many ways very similar to the circuit type with no output, or rather output set to (), as such a pipe could also have had output).
Just an observation, your type seems quite related to Consumer p a m appearing in the pipes library (and probably other similar librarties as well):
type Consumer p a = p () a () C
-- A Pipe that consumes values
-- Consumers never respond.
where C is an empty data type and p is an instance of Proxy type class. It consumes values of type a and never produces any (because its output type is empty).
For example, we could convert a Callback into a Consumer:
import Control.Proxy
import Control.Proxy.Synonym
newtype Callback m a = Callback { unCallback :: a -> m (Callback m a) }
-- No values produced, hence the polymorphic return type `r`.
-- We could replace `r` with `C` as well.
consumer :: (Proxy p, Monad m) => Callback m a -> () -> Consumer p a m r
consumer c () = runIdentityP (run c)
where
run (Callback c) = request () >>= lift . c >>= run
See the tutorial.
(This should have been rather a comment, but it's a bit too long.)