Using pipes, I'm trying to write an instance of MonadTransControl for the ProxyFast or ProxyCorrect type. This is what I've got:
instance MonadTransControl (ProxyFast a' a b' b) where
data StT (ProxyFast a' a b' b) a = StProxy { unStProxy :: ProxyFast a' a b' b Identity a}
liftWith = undefined
restoreT = undefined
I have no idea how to write liftWith or restoreT. The instances for the other monad transformers all use a function that "swaps" the monads, for example EitherT e m a -> m (EitherT e Identity a), but I couldn't find any such function in pipes. How does the instance for MonadTransControl for ProxyCorrect / ProxyFast look like? Or is it impossible to write one? (If yes, is it possible in pipes 4.0?)
Thanks for the link, and now I can give a better answer.
No, there is no way to implement this, using either version of pipes. The reason why is that MonadTransControl expects a monad transformer to be built on top of a single layer of the underlying base monad. This is true for all the monad transformers that MonadTransControl currently implements, such as:
ErrorT ~ m (Either e r)
StateT ~ s -> m (r, s)
WriterT ~ m (r, w)
ReaderT ~ i -> m r
ListT ~ m [r] -- This version of ListT is wrong, and the true ListT
-- would not work for `MonadTransControl`
However, a Proxy does not wrap a single layer of the base monad. This is true for both pipes versions where you can nest as many layers of the base monad as you want.
In fact, any monad transformer that nests the base monad multiple times will defy a MonadTransControl instance, such as:
FreeT -- from the `free` package
ListT -- when done "right"
ConduitM -- from the `conduit` package
However, just because pipes does not implement MonadTransControl doesn't mean that all hope is lost. pipes-safe implements many of the operations that one would typically expect from MonadTransControl, such as bracketing resource acquisitions, so if you can elaborate on your specific use case I can tell you more if there is an appropriate pipes-based solution for your problem.
Related
In many cases, it isn't clear to me what is to be gained by combining two monads with a transformer rather than using two separate monads. Obviously, using two separate monads is a hassle and can involve do notation inside do notation, but are there cases where it just isn't expressive enough?
One case seems to be StateT on List: combining monads doesn't get you the right type, and if you do obtain the right type via a stack of monads like Bar (where Bar a = (Reader r (List (Writer w (Identity a))), it doesn't do the right thing.
But I'd like a more general and technical understanding of exactly what monad transformers are bringing to the table, when they are and aren't necessary, and why.
To make this question a little more focused:
What is an actual example of a monad with no corresponding transformer (this would help illustrate what transformers can do that just stacking monads can't).
Are StateT and ContT the only transformers that give a type not equivalent to the composition of them with m, for an underlying monad m (regardless of which order they're composed.)
(I'm not interested in particular implementation details as regards different choices of libraries, but rather the general (and probably Haskell independent) question of what monad transformers/morphisms are adding as an alternative to combining effects by stacking a bunch of monadic type constructors.)
(To give a little context, I'm a linguist who's doing a project to enrich Montague grammar - simply typed lambda calculus for composing word meanings into sentences - with a monad transformer stack. It would be really helpful to understand whether transformers are actually doing anything useful for me.)
Thanks,
Reuben
To answer you question about the difference between Writer w (Maybe a) vs MaybeT (Writer w) a, let's start by taking a look at the definitions:
newtype WriterT w m a = WriterT { runWriterT :: m (a, w) }
type Writer w = WriterT w Identity
newtype MaybeT m a = MaybeT { runMaybeT :: m (Maybe a) }
Using ~~ to mean "structurally similar to" we have:
Writer w (Maybe a) == WriterT w Identity (Maybe a)
~~ Identity (Maybe a, w)
~~ (Maybe a, w)
MaybeT (Writer w) a ~~ (Writer w) (Maybe a)
== Writer w (Maybe a)
... same derivation as above ...
~~ (Maybe a, w)
So in a sense you are correct -- structurally both Writer w (Maybe a) and MaybeT (Writer w) a
are the same - both are essentially just a pair of a Maybe value and a w.
The difference is how we treat them as monadic values.
The return and >>= class functions do very different things depending
on which monad they are part of.
Let's consider the pair (Just 3, []::[String]). Using the association
we have derived above here's how that pair would be expressed in both monads:
three_W :: Writer String (Maybe Int)
three_W = return (Just 3)
three_M :: MaybeT (Writer String) Int
three_M = return 3
And here is how we would construct a the pair (Nothing, []):
nutin_W :: Writer String (Maybe Int)
nutin_W = return Nothing
nutin_M :: MaybeT (Writer String) Int
nutin_M = MaybeT (return Nothing) -- could also use mzero
Now consider this function on pairs:
add1 :: (Maybe Int, String) -> (Maybe Int, String)
add1 (Nothing, w) = (Nothing w)
add1 (Just x, w) = (Just (x+1), w)
and let's see how we would implement it in the two different monads:
add1_W :: Writer String (Maybe Int) -> Writer String (Maybe Int)
add1_W e = do x <- e
case x of
Nothing -> return Nothing
Just y -> return (Just (y+1))
add1_M :: MaybeT (Writer String) Int -> MaybeT (Writer String) Int
add1_M e = do x <- e; return (e+1)
-- also could use: fmap (+1) e
In general you'll see that the code in the MaybeT monad is more concise.
Moreover, semantically the two monads are very different...
MaybeT (Writer w) a is a Writer-action which can fail, and the failure is
automatically handled for you. Writer w (Maybe a) is just a Writer
action which returns a Maybe. Nothing special happens if that Maybe value
turns out to be Nothing. This is exemplified in the add1_W function where
we had to perform a case analysis on x.
Another reason to prefer the MaybeT approach is that we can write code
which is generic over any monad stack. For instance, the function:
square x = do tell ("computing the square of " ++ show x)
return (x*x)
can be used unchanged in any monad stack which has a Writer String, e.g.:
WriterT String IO
ReaderT (WriterT String Maybe)
MaybeT (Writer String)
StateT (WriterT String (ReaderT Char IO))
...
But the return value of square does not type check against Writer String (Maybe Int) because square does not return a Maybe.
When you code in Writer String (Maybe Int), you code explicitly reveals
the structure of monad making it less generic. This definition of add1_W:
add1_W e = do x <- e
return $ do
y <- x
return $ y + 1
only works in a two-layer monad stack whereas a function like square
works in a much more general setting.
What is an actual example of a monad with no corresponding transformer (this would help illustrate what transformers can do that just stacking monads can't).
IO and ST are the canonical examples here.
Are StateT and ContT the only transformers that give a type not equivalent to the composition of them with m, for an underlying monad m (regardless of which order they're composed.)
No, ListT m a is not (isomorphic to) [m a]:
newtype ListT m a =
ListT { unListT :: m (Maybe (a, ListT m a)) }
To make this question a little more focused:
What is an actual example of a monad with no corresponding transformer (this would help illustrate what transformers can do that just stacking monads can't).
There are no known examples of a monad that lacks a transformer, as long as the monad is defined explicitly as a pure lambda-calculus term, with no side effects and no external libraries being used. The Haskell monads such as IO and ST are essentially interfaces to an external library defined by low-level code. Those monads cannot be defined by pure lambda-calculus, and their monad transformers probably do not exist.
Even though there are no known explicit examples of monads without transformers, there is also no known general method or algorithm for obtaining a monad transformer for a given monad. If I define some complicated monad, for example like this code in Haskell:
type D a = Either a ((a -> Bool) -> Maybe a)
then it is far from obvious how to define a transformer for the monad D.
This D a may look a contrived and artificial example (and it's also not obvious why it is a monad) but there might be legitimate cases for using that monad, which is a "free pointed monad on the Search monad on Maybe".
To clarify: A "search monad on n" is the type S n q a = (a -> n q) -> n a where n is another monad and q is a fixed type.
A "free pointed monad on M" is the type P a = Either a (M a) where M is another monad.
In any case, I just want to illustrate the point. I don't think it would be easy for anyone to come up with the monad transformer for D and then to prove that it satisfies the laws of monad transformers. There is no known algorithm that takes the code of D and outputs the code of its transformer.
Are StateT and ContT the only transformers that give a type not equivalent to the composition of them with m, for an underlying monad m (regardless of which order they're composed.)
Monad transformers are necessary because stacking two monads is not always a monad. Most "simple" monads, like Reader, Writer, Maybe, etc., stack with other monads in a particular order. But the result of stacking, say, Writer + Reader + Maybe, is a more complicated monad that no longer allows stacking with new monads.
There are several examples of monads that do not stack at all: State, Cont, List, Free monads, the Codensity monad, and a few other, less well known monads, like the "free pointed" monad shown above.
For each of those "non-stacking" monads, one needs to guess the correct monad transformer somehow.
I have studied this question for a while and I have assembled a list of techniques for creating monad transformers, together with full proofs of all laws. There doesn't seem to be any system to creating a monad transformer for a specific monad. I even found a couple of monads that have two inequivalent transformers.
Generally, monad transformers can be classified in 6 different families:
Functor composition in one or another order: EitherT, WriterT, ReaderT and a generalization of Reader to a special class of monads, called "rigid" monads. An example of a "rigid" monad is Q a = (H a) -> a where H is an arbitrary (but fixed) contravariant functor.
The "adjunction recipe": StateT, ContT, CodensityT, SearchT, which gives transformers that are not functorial.
The "recursive recipe": ListT, FreeT
Cartesian product of monads: If M and N are monads then their Cartesian product, type P a = (M a, N a) is also a monad whose transformer is the Cartesian product of transformers.
The free pointed monad: P a = Either a (M a) where M is another monad. For that monad, the transformer's type is m (Either a (MT m a)) where MT is the monad M's transformer.
Monad stacks, that is, monads obtained by applying one or more monad transformers to some other monad. A monad stack's transformer is build via a special recipe that uses all the transformers of the individual monads in the stack.
There may be monads that do not fit into any of these cases, but I have seen no examples so far.
Details and proofs of these constructions of monad transformers are in my draft book here https://github.com/winitzki/sofp
Real World Haskell states that "Transformer stacking order is important". However, I can't seem to figure out if there's a difference between ExceptT (ResourceT m) a and ResourceT (ExceptT m) a. Will they interfere with each other?
In this example, there is no real difference between both orders. The reason being: unlike many transformers including ExceptT, the resource transformer does not “inject” its own doings into the base monad you apply it to, but rather start off the entire action with passing in the release references.
If you write out the types (I'll refer to MaybeT instead of ExceptT for the sake of simplicity; they're obviously equivalent for the purpose of this question) then you have basically
type MaybeResourceT m a = MaybeT (IORef RelMap -> m a)
= IORef RelMap -> m (Maybe a)
type ResourceMaybeT m a = ResourceT (m (Maybe a))
= IORef RelMap -> m (Maybe a)
i.e. actually equivalent types. I suppose you could also show that for the operations.
I'm having a bit of difficulty with monad transformers at the moment. I'm defining a few different non-deterministic relations which make use of transformers. Unfortunately, I'm having trouble understanding how to translate cleanly from one effectful model to another.
Suppose these relations are "foo" and "bar". Suppose that "foo" relates As and Bs to Cs; suppose "bar" relates Bs and Cs to Ds. We will define "bar" in terms of "foo". To make matters more interesting, the computation of these relations will fail in different ways. (Since the bar relation depends on the foo relation, its failure cases are a superset.) I therefore give the following type definitions:
data FooFailure = FooFailure String
data BarFailure = BarSpecificFailure | BarFooFailure FooFailure
type FooM = ListT (EitherT FooFailure (Reader Context))
type BarM = ListT (EitherT BarFailure (Reader Context))
I would then expect to be able to write the relations with the following function signatures:
foo :: A -> B -> FooM C
bar :: B -> C -> BarM D
My problem is that, when writing the definition for "bar", I need to be able to receive errors from the "foo" relation and properly represent them in "bar" space. So I'd be fine with a function of the form
convert :: (e -> e') -> ListT (EitherT e (Reader Context) a
-> ListT (EitherT e' (Reader Context) a
I can even write that little beast by running the ListT, mapping on EitherT, and then reassembling the ListT (because it happens that m [a] can be converted to ListT m a). But this seems... messy.
There's a good reason I can't just run a transformer, do some stuff under it, and generically "put it back"; the transformer I ran might have effects and I can't magically undo them. But is there some way in which I can lift a function just far enough into a transformer stack to do some work for me so I don't have to write the convert function shown above?
I think convert is a good answer, and using Control.Monad.Morph and Control.Monad.Trans.Either it's (almost) really simple to write:
convert :: (Monad m, Functor m, MFunctor t)
=> (e -> e')
-> t (EitherT e m) b -> t (EitherT e' m) b
convert f = hoist (bimapEitherT f id)
the slight problem is that ListT isn't an instance of MFunctor. I think this is the author boycotting ListT because it doesn't follow the monad transformer laws though because it's easy to write a type-checking instance
instance MFunctor ListT where hoist nat (ListT mas) = ListT (nat mas)
Anyway, generally take a look at Control.Monad.Morph for dealing with natural transformations on (parts of) transformer stacks. I'd say that fits the definition of lifting a function "just enough" into a stack.
For example, having...
consumer :: Proxy p => () -> Consumer p a (EitherT String IO) ()
producer :: Proxy p => () -> Producer p a (EitherT ByteString IO) r
... how do I make this work?
session :: EitherT ByteString (EitherT String IO) ()
session = runProxy $ producer >-> consumer
Note: I've read Mixing Base Monads in Control.Proxy.Tutorial. I get the first example but can't understand the contrived example. More examples, not so obvious but not so contrived, might clarify how to use hoist and lift to match any combination of base monads.
Suppose you have a monad transformer stack like MT1 MT2 MT3 M a where M is the base monad.
Using lift, you can add a new monad transformer to the left. It can be any transformer, so let's symbolize it by ?.
lift :: MT1 MT2 MT3 M a -> ? MT1 MT2 MT3 M a
Using hoist, you can manipulate the monad stack to the right of the leftmost element. Manipulate it how? For example by supplying a lift:
hoist lift :: MT1 MT2 MT3 M a -> MT1 ? MT2 MT3 M a
Using combinations of hoist and lift, you can insert these "wildcards" at any point in the monad transformer stack.
hoist (hoist lift) :: MT1 MT2 MT3 M a -> MT1 MT2 ? MT3 M a
hoist (hoist (hoist lift)) :: MT1 MT2 MT3 M a -> MT1 MT2 MT3 ? M a
This technique can be used to equalize the two monad stacks from your example.
The either package (where I guess your EitherT type is coming from) provide several functions for modifying the first argument, e.g.
bimapEitherT :: Functor m => (e -> f) -> (a -> b) -> EitherT e m a -> EitherT f m b
You can use this, along with some appropriate encoding (or decoding), to turn an EitherT String IO a into an EitherT ByteString IO a (or vice versa), then hoist that transformation into the Consumer or Producer monad transformer.
There are actually two solutions.
The first solution is the one Daniel Wagner proposed: you modify the two base monads to use the same Left type. For example, we could normalize them to both use ByteString. To do this, we first take ByteString's pack function:
pack :: String -> ByteString
Then we lift it to work on the left value of an EitherT:
import Control.Error (fmapLT) -- from the 'errors' package
fmapLT pack :: (Monad m) => EitherT String m r -> EitherT ByteString m r
Now we need to target that transformation to your Consumer's base monad, using hoist:
hoist (fmapLT pack)
:: (Monad m, Proxy p)
=> Consumer p a (EitherT String m) r -> Consumer p a (EitherT ByteString m) r
Now you can compose your consumer directly with your producer since they have the same base monad.
The second solution is the one Daniel Diaz Carrete proposed. You instead get your two pipes to agree on a common monad transformer stack that contains both EitherT layers. All you have to do is decide in which order to nest those two layers.
Let's imagine that you choose to layer the EitherT String transformer outside of the EitherT ByteString transformer. That would mean that your final target monad transformer stack would be:
(Proxy p) => Session (EitherT String (EitherT ByteString p)) IO r
Now you need to promote both of your pipes to target that transformer stack.
For your Consumer, you need to insert an EitherT ByteString layer in between EitherT String and IO if you want to match that final monad transformer stack. Creating the layer is easy: you just use lift, but you need to target that lift in between those two specific layers, so you use hoist, twice, because you need to skip over both the proxy monad transformer and the EitherT String monad transformer:
hoist (hoist lift) . consumer
:: Proxy p => () -> Consumer p a (EitherT String (EitherT ByteString IO)) ()
For your Producer, you need to insert an EitherT String layer in between the proxy monad transformer and the EitherT ByteString transformer if you want to match the final monad transformer stack. Again, creating the layer is easy: we just use lift, but you need to target that lift in between those two specific layers. You just hoist, but this time you only use it once, since you only need to skip over the proxy monad transformer to nestle the lift in the right spot:
hoist lift . producer
:: Proxy p => () -> Producer p a (EitherT String (EitherT ByteString IO)) r
Now your producer and consumer have the same monad transformer stack and you can compose them directly.
Now, you might wonder: Is this process of hoisting lifts doing the "right thing"? The answer is yes. Part of the magic of category theory is that we can rigorously define what it means to correctly insert an "empty monad transformer layer" using lift and we can similarly rigorously define what it means to "target something in between two monad transformers" using hoist by specifying several theoretically-inspired laws and verifying that lift and hoist observe those laws.
Once we satisfy those laws, we can just ignore all the nitty gritty details of what exactly lift and hoist do. Category theory frees us to work at a very high abstraction level where we just think in terms of "inserting lifts" spatially between monad transformers and the code magically translates our spatial intuition into the rigorously correct behavior.
My guess is that you probably want the first solution since you can then share error-handling between the producer and consumer in a single EitherT layer.
While hacking something up earlier, I created the following code:
newtype Callback a = Callback { unCallback :: a -> IO (Callback a) }
liftCallback :: (a -> IO ()) -> Callback a
liftCallback f = let cb = Callback $ \x -> (f x >> return cb) in cb
runCallback :: Callback a -> IO (a -> IO ())
runCallback cb =
do ref <- newIORef cb
return $ \x -> readIORef ref >>= ($ x) . unCallback >>= writeIORef ref
Callback a represents a function that handles some data and returns a new callback that should be used for the next notification. A callback which can basically replace itself, so to speak. liftCallback just lifts a normal function to my type, while runCallback uses an IORef to convert a Callback to a simple function.
The general structure of the type is:
data T m a = T (a -> m (T m a))
It looks much like this could be isomorphic to some well-known mathematical structure from category theory.
But what is it? Is it a monad or something? An applicative functor? A transformed monad? An arrow, even? Is there a search engine similar Hoogle that lets me search for general patterns like this?
The term you are looking for is free monad transformer. The best place to learn how these work is to read the "Coroutine Pipelines" article in issue 19 of The Monad Reader. Mario Blazevic gives a very lucid description of how this type works, except he calls it the "Coroutine" type.
I wrote up his type in the transformers-free package and then it got merged into the free package, which is its new official home.
Your Callback type is isomorphic to:
type Callback a = forall r . FreeT ((->) a) IO r
To understand free monad transformers, you need to first understand free monads, which are just abstract syntax trees. You give the free monad a functor which defines a single step in the syntax tree, and then it creates a Monad from that Functor that is basically a list of those types of steps. So if you had:
Free ((->) a) r
That would be a syntax tree that accepts zero or more as as input and then returns a value r.
However, usually we want to embed effects or make the next step of the syntax tree dependent on some effect. To do that, we simply promote our free monad to a free monad transformer, which interleaves the base monad between syntax tree steps. In the case of your Callback type, you are interleaving IO in between each input step, so your base monad is IO:
FreeT ((->) a) IO r
The nice thing about free monads is that they are automatically monads for any functor, so we can take advantage of this to use do notation to assemble our syntax tree. For example, I can define an await command that will bind the input within the monad:
import Control.Monad.Trans.Free
await :: (Monad m) => FreeT ((->) a) m a
await = liftF id
Now I have a DSL for writing Callbacks:
import Control.Monad
import Control.Monad.Trans.Free
printer :: (Show a) => FreeT ((->) a) IO r
printer = forever $ do
a <- await
lift $ print a
Notice that I never had to define the necessary Monad instance. Both FreeT f and Free f are automatically Monads for any functor f, and in this case ((->) a) is our functor, so it automatically does the right thing. That's the magic of category theory!
Also, we never had to define a MonadTrans instance in order to use lift. FreeT f is automatically a monad transformer, given any functor f, so it took care of that for us, too.
Our printer is a suitable Callback, so we can feed it values just by deconstructing the free monad transformer:
feed :: [a] -> FreeT ((->) a) IO r -> IO ()
feed as callback = do
x <- runFreeT callback
case x of
Pure _ -> return ()
Free k -> case as of
[] -> return ()
b:bs -> feed bs (k b)
The actual printing occurs when we bind runFreeT callback, which then gives us the next step in the syntax tree, which we feed the next element of the list.
Let's try it:
>>> feed [1..5] printer
1
2
3
4
5
However, you don't even need to write all this up yourself. As Petr pointed out, my pipes library abstracts common streaming patterns like this for you. Your callback is just:
forall r . Consumer a IO r
The way we'd define printer using pipes is:
printer = forever $ do
a <- await
lift $ print a
... and we can feed it a list of values like so:
>>> runEffect $ each [1..5] >-> printer
1
2
3
4
5
I designed pipes to encompass a very large range of streaming abstractions like these in such a way that you can always use do notation to build each streaming component. pipes also comes with a wide variety of elegant solutions for things like state and error handling, and bidirectional flow of information, so if you formulate your Callback abstraction in terms of pipes, you tap into a ton of useful machinery for free.
If you want to learn more about pipes, I recommend you read the tutorial.
The general structure of the type looks to me like
data T (~>) a = T (a ~> T (~>) a)
where (~>) = Kleisli m in your terms (an arrow).
Callback itself doesn't look like an instance of any standard Haskell typeclass I can think of, but it is a Contravariant Functor (also known as Cofunctor, misleadingly as it turns out). As it is not included in any of the libraries that come with GHC, there exist several definitions of it on Hackage (use this one), but they all look something like this:
class Contravariant f where
contramap :: (b -> a) -> f a -> f b
-- c.f. fmap :: (a -> b) -> f a -> f b
Then
instance Contravariant Callback where
contramap f (Callback k) = Callback ((fmap . liftM . contramap) f (f . k))
Is there some more exotic structure from category theory that Callback possesses? I don't know.
I think that this type is very close to what I have heard called a 'Circuit', which is a type of arrow. Ignoring for a moment the IO part (as we can have this just by transforming a Kliesli arrow) the circuit transformer is:
newtype CircuitT a b c = CircuitT { unCircuitT :: a b (c, CircuitT a b c) }
This is basicall an arrow that returns a new arrow to use for the next input each time. All of the common arrow classes (including loop) can be implemented for this arrow transformer as long as the base arrow supports them. Now, all we have to do to make it notionally the same as the type you mention is to get rid of that extra output. This is easily done, and so we find:
Callback a ~=~ CircuitT (Kleisli IO) a ()
As if we look at the right hand side:
CircuitT (Kleisli IO) a () ~=~
(Kliesli IO) a ((), CircuitT (Kleisli IO) a ()) ~=~
a -> IO ((), CircuitT (Kliesli IO) a ())
And from here, you can see how this is similar to Callback a, except we also output a unit value. As the unit value is in a tuple with something else anyway, this really doesn't tell us much, so I would say they're basically the same.
N.B. I used ~=~ for similar but not entirely equivalent to, for some reason. They are very closely similar though, in particular note that we could convert a Callback a into a CircuitT (Kleisli IO) a () and vice-versa.
EDIT: I would also fully agree with the ideas that this is A) a monadic costream (monadic operation expecitng an infinite number of values, I think this means) and B) a consume-only pipe (which is in many ways very similar to the circuit type with no output, or rather output set to (), as such a pipe could also have had output).
Just an observation, your type seems quite related to Consumer p a m appearing in the pipes library (and probably other similar librarties as well):
type Consumer p a = p () a () C
-- A Pipe that consumes values
-- Consumers never respond.
where C is an empty data type and p is an instance of Proxy type class. It consumes values of type a and never produces any (because its output type is empty).
For example, we could convert a Callback into a Consumer:
import Control.Proxy
import Control.Proxy.Synonym
newtype Callback m a = Callback { unCallback :: a -> m (Callback m a) }
-- No values produced, hence the polymorphic return type `r`.
-- We could replace `r` with `C` as well.
consumer :: (Proxy p, Monad m) => Callback m a -> () -> Consumer p a m r
consumer c () = runIdentityP (run c)
where
run (Callback c) = request () >>= lift . c >>= run
See the tutorial.
(This should have been rather a comment, but it's a bit too long.)