Working on permuted monad transformer stack - haskell

One of the problems with monad transformers I find is the need to lift the operations into the right monad. A single lift here and there isn't bad, but sometimes there are functions that looks like this:
fun = do
lift a
lift b
c
lift d
lift e
f
I would like to be able to write this function thus:
fun = monadInvert $ do
a
b
lift c
d
e
lift f
This halves the number of lifts and makes the code cleaner.
The question is: for what monads is monadInvert possible? How should one create this function?
Bonus points: define it for monad m which is an instance of MonadIO.
The title of this question speaks of permutations: indeed, how can we deal with arbitrary permutations of a monad tranformer stack?

Well, first of all, you don't actually need so much lifting. For monad transformers the following identities hold:
lift c >>= lift . f = lift (c >>= f)
lift c1 >> lift c2 = lift (c1 >> c2)
It's not uncommon to write:
x <- lift $ do
{- ... -}
Next is: When you use libraries like mtl or monadLib (i.e. type class based libraries instead of transformers directly), you actually can access most underlying monads directly:
c :: StateT MyState (ReaderT MyConfig SomeOtherMonad) Result
c = do
x <- ask
y <- get
{- ... -}
Finally, if you really need a lot of lifting despite these two points, you should consider writing a custom monad or even use an entirely different abstraction. I find myself using the automaton arrow for stateful computations instead of a state monad.

You might be interested in Monads, Zippers and Views, Virtualizing the Monad Stack by Tom Schrijvers and Bruno Oliveira.
This doesn't address your point about reducing the lifts, but it's an interesting approach to your "monad permutations" problem.
Here's the abstract:
This work aims at making monadic components more reusable and robust
to changes by employing two new techniques for virtualizing the monad
stack: the monad zipper and monad views. The monad zipper is a monad
transformer that creates virtual monad stacks by ignoring particular
layers in a concrete stack. Monad views provide a general framework
for monad stack virtualization: they take the monad zipper one step
further and integrate it with a wide range of other virtualizations.
For instance, particular views allow restricted access to monads in
the stack. Furthermore, monad views can be used by components to
provide a call-by-reference-like mechanism to access particular layers
of the monad stack. With these two mechanisms component requirements
in terms of the monad stack shape no longer need to be literally
reflected in the concrete monad stack, making these components more
reusable and robust to changes.

I am mostly sure that what you are decribing is impossible for IO, which is always the innermost monad:
From Martin Grabmüller: Monad Transformers Step by Step, available at
http://www.grabmueller.de/martin/www/pub/
Down in this document, we call liftIO in eval6 to perform I/O actions.
Why do we need to lift in this case? Because there is no IO class for
which we can instantiate a type as. Therefore, for I/O actions, we
have to call lift to send the commands inwards
In general, for monads less restrictive than IO, (such as Error and State) order still matters for semantics, so you can't change the order of the stack just to make syntax more convenient.
For some monads, like Reader, that are central (in that they do commute in the stack), your idea seems not clearly impossible. I don't know how to write it though. I guess it would be a type class CentralMonad that ReaderT is an instance of, with some implementation...

Related

In Haskell terminology, what are monadic effects?

I feel I have a good understanding of monads, but I'm not too sure what is referred to by 'monadic effects'? Is this the evaluation of a monad? Does it have something to do with IO?
If you have a value of type M a with M a Monad (or Applicative for applicative effects), then by effects we mean the information that is not contained in the a part. For example with IO it is very clear. A value of IO Int is an Int with some IO effects like writing to a file or firing missiles. A value of type Maybe Int is an Int with the effect of maybe actually not containing an Int. For [Int] the effect is, that you actually have multiple Ints.
We call this an effect because you can think of Monads and Applicatives as notions of computation with certain effects. For Maybe the effects are that you can abort the computation prematurely, for [] you can split the computation.
I'm going to try to talk at the question several ways, and hopefully it's helpful.
An "effect" (as in "side-effects") refer to the behaviors of a specific instance of Monad, so e.g. the State monad expresses the effect of "stateful computation" with get and put. Monad transformer libraries like mtl can be thought of as ways of "composing effects".
Without knowing the types (or in fact reading the docs) for foo and bar here, we can't say anything about what "monadic effects" are happening here, even though we can say quite a few other things about this code:
do a <- fmap bar $ foo x
b <- baz
return (a,b)
The do block above has a type of the form SomeMonad m=> m (a,b). That tuple (a,b) that is "returned", and the way it can be passed to another "effectful computation" with >>=, is not what we're talking about when we talk about "effects".
Monadic effects always actually "happen" when you run them (by calling runState for State for instance).
In the case of IO only the runtime has access to the particular run function for IO, so the nonexistent runIO function calls main to run your program. For IO the "monadic effects" are truly the same as what in other languages you'd call "side-effects", i.e. just about anything that might change the state of the world.

Flatten monad stack

So I have this sort of code all over my first serious haskell project:
f :: (MonadTrans t) => ExceptT () (t (StateT A B)) C
f = do mapExceptT lift $ do
lift $ do
...
lift $ do
...
r <- ...
...
return r
>>= \r -> ...
There definitely may be something wrong about how I try to achieve my goals (there might be simpler ways how to do it) but currently I am interested in learning how to handle a stack of monad transformers in some nicer way, if there is one. This is the only way I figured out how to get r in the context of B and lift it to a monad higher in the stack. Lifting whole blocks instead of initial statements is as far as I could get on my own.
What I also often end up with are chains of lift which I found out can be avoided with liftIO if the deep monad is IO. I am not aware of a generic way for other monads though.
Is there a pattern that one can follow when he ends up dealing with such stacks, and having to extract one value at some level, a different value at a different level, combine these and affect any of the two levels or maybe yet another one?
Can the stack be manipulated somehow without neither lifting whole blocks (which causes let and bound variables to be scoped and restricted to the inner block) nor having to lift . lift . ... lift individual actions?
This is a well-known problem with monad transformers in general. Researchers have devised various ways of handling it, none of which is clearly "best". Some of the known solutions include:
The mtl approach, which automatically lifts type classes of monads over its built-in monad transformers (and only its built-in monad transformers). This allows you to just write f :: (MonadState A m, MonadError () m) => m C if those are the only features of the monad that your function is using. Due to its extreme non-portability and a few other reasons, mtl is generally considered pseudo-deprecated. See this page and this question for the gory details.
If you have a particular monad stack you are using over and over again, you can wrap it in a newtype and write instances of the various monad type classes it supports manually. For Functor, Applicative, Monad, and any other type classes implemented by the top-level transformer in your stack, you can use GeneralizedNewtypeDeriving to have the compiler write the instances for you automatically; for other type classes, you will have to insert the appropriate number of lift calls for each method. The advantage of this approach is that it's more general and simpler to understand while giving you the same flexibility at the call site as mtl. The big problem with this approach is that it encourages using a single "mega-monad" for all operations rather than specifying only the needed operations, since adding any new monad transformer to the stack requires writing a whole new list of instances.
In most cases, you don't really want a monad that has "some arbitrary state of type A" and "some arbitrary exception-throwing capability". Rather, the different features offered by your monad stack have some semantic meaning in your mental model of your program. A variation of the previous approach is to create custom type classes for the effects beyond the basic Functor, Applicative, and Monad and write instances for the custom type classes on your newtype'd monad instead. This has a major advantage over the other approaches listed here: you can have a stack with multiple copies of the same monad transformer in it at different positions. This is the strategy I've used the most in my own programs so far.
A completely different approach is effect systems. Normally, an effect system has to be built in to a language's type system, but it's possible to encode an effect system in Haskell's type system. See the effect-monads package.
The usual approach is to use the mtl library rather than using transformers directly. I'm not sure what the story behind your t is, but the usual mtl approach is to use very general type signatures at definition sites, like
foo :: (MonadError e m, MonadState s m) => m Int
Then fix the actual transformer stacks at the call sites. A common recommendation is to wrap up the stack in a newtype to avoid muddying things up where they're used.
If this isn't your style (and it's not for everyone), you can still use the mtl methods to perform operations, while giving an explicit transformer stack. This should cut down on the manual lifting substantially. The advantage of this approach is that it gives you a better view of the interactions of effects at the definition sites; the disadvantage is that more code needs all the information.

lift, return, and a transformer type constructor

For well over a year, I have been intensely using lift, return, and constructors such as EitherT, ReaderT, and so forth. I've read Real World Haskell, Learn You a Haskell, almost every monad tutorial out there, and tried writing my own. Yet, I constantly remain confused about these three operations. Any time I am writing new code I try to figure out which of the three to use, and it almost always takes me an hour or more on the first function in a particular block of code.
What is an intuitive understanding of the three? Simple types are insufficient, as in all three cases I can instantly recite the types to you. What is a meaning for what these do that is consistent across all of the standard monad transformers?
(Unfortunately, if you respond in math terms, I'm still not going to understand you. While I can write code to solve math problems and can set up time complexity based on the code I see, I cannot after many years of trying to work in Haskell relate math terms to programming terms.)
return takes a pure computation and turns it into a computation which claims to have some monad-y side-effects, but doesn't.
lift takes a computation that has some side-effects, and adds more.
EitherT, ReaderT, and so on take a computation that already has all the side-effects you're interested in and "spells them differently" -- for example, where before your state was spelled as a function that returns an updated value, it is now spelled as a State(T)-ful computation.
So let's say you have a computation. In a lazy language like Haskell you'd write
comp1 :: a
and know that this computation will be performed upon request and result in a value of type a.
Let's say you have a similar computation, but in addition to computing a value of type a, it might "fail" for some reason or another. For example, a might be Integer and this computation will "fail" if its a division by zero. We're write this now as
comp2 :: Maybe a
where the Maybe constructor "tags" the a to indicate failure.
Let's say we have a similar computation as before, but now we are allowed to fail, but also collect a log during the computation. "Log collecting" is called Writer so we'd like to tag our type with Writer as well as Maybe. Unfortunately
comp3_bad :: (Writer String) Maybe a
doesn't make any sense. The definition of writer allows for a single parameter, not two. We can consider a bit of what the underlying mechanics of this combined effect would be, though—it needs to return a Maybe paired with the log... or perhaps if the computation fails, the log is discarded. There are two options
comp3_1 :: (String, Maybe a)
comp3_2 :: Maybe (String, a)
If we unpack the Writer, we can see that these are equivalent to
comp3_1' :: Writer String (Maybe a)
comp3_2' :: Maybe (Writer String a)
This pattern of nesting is called composition. If you want to combine the effects of two monads then you'd like to compose them. For some monads this works directly, though it's a little cumbersome.
Unfortunately, some monads start to break the monad laws once they are composed. They can still be "stacked" but not in the normal way. So, we allow each type to determine its stacking method by creating the transformer version <monad>T.
newtype WriterT w m a = WriterT { runWriterT :: m (w, a) }
newtype MaybeT m a = MaybeT { runMaybeT :: m (Maybe a) }
-- note that
WriterT String Maybe a == Maybe (String, a)
MaybeT (Writer String) a == (String, Maybe a)
These composed stacks of monads are called monad transformer stacks and they allow you to assemble side effects in layers.
So what happens if we have two different, but similar stacks that we'd like to use together. For instance, we can consider Maybe to be a monad... or a monad transformer stack of a single layer. Compare that to WriterT String Maybe which is a monad transformer stack of two layers, the bottom of which is Maybe.
These two stacks are very similar, but we cannot transport computations from one to the other. Or rather, we can, but it's fairly annoying
transport :: Maybe a -> WriterT String Maybe a
transport Nothing = WriterT Nothing
transport (Just a) = WriterT (Just ("", a))
this transport forms a general pattern where we "add another layer" onto a stack. This general pattern is called lift
lift :: Maybe a -> WriterT String Maybe a
Or, written polymorphically we see the extra layer t being prepended.
lift :: MonadTrans t => m a -> t m a
Finally, we've come a long way from our pure computation at the beginning
comp1 :: a
and demonstrated that we can lift simple transformer stacks into more complex ones. Can we consider comp1 to be living in the very simplest of transformer stacks—the empty stack?
It turns out that this is actually a really valid point of view. We can even "lift" comp1 into a more sophisticated transformer stack... but the terminology changes slightly.
return :: Monad m => a -> m a
So, it's valid to think of return as lifting a pure computation into a basic monad. This is a foundational principle of monads even—that they can embed pure computations within them.

Is 'Chaining operations' the "only" thing that the Monad class solves?

To clarify the question: it is about the merits of the monad type class (as opposed to just its instances without the unifying class).
After having read many references (see below),
I came to the conclusion that, actually, the monad class is there to solve only one, but big and crucial, problem: the 'chaining' of functions on types with context. Hence, the famous sentence "monads are programmable semicolons".
In fact, a monad can be viewed as an array of functions with helper operations.
I insist on the difference between the monad class, understood as a general interface for other types; and these other types instantiating the class (thus, "monadic types").
I understand that the monad class by itself, only solves the chaining of operators because mainly, it only mandates its type instances
to have bind >>= and return, and tell us how they must behave. And as a bonus, the compiler greatyly helps the coding providing do notation for monadic types.
On the other hand,
it is each individual type instantiating the monad class which solves each concrete problem, but not merely for being a instance of Monad. For instance Maybe solves "how a function returns a value or an error", State solves "how to have functions with global state", IO solves "how to interact with the outside world", and so on. All theses classes encapsulate a value within a context.
But soon or later, we will need to chain operations on such context-types. I.e., we will need to organize calls to functions on these types in a particular sequence (for an example of such a problem, please read the example about multivalued functions in You could have invented monads).
And you get solved the problem of chaining, if you have each type be an instance of the monad class.
For the chaining to work you need >>= just with the exact signature it has, no other. (See this question).
Therefore, I guess that the next time you define a context data type T for solving something, if you need to sequence calls of functions (on values of T) consider making T an instance of Monad (if you need "chaining with choice" and if you can benefit from the do notation). And to make sure you are doing it right, check that T satisfies the monad laws
Then, I ask two questions to the Haskell experts:
A concrete question: is there any other problem that the monad class solves by ifself (leaving apart monadic classes)? If any, then, how it compares in relevance to the problem of chaining operations?
An optional general question: are my conclusions right, am I misunderstanding something?
References
Tutorials
Monads in pictures Definitely worth it; read this one first.
Fistful of monads
You could have invented monads
Monads are trees (pdf)
StackOverflow Questions & Answers
How to detect a monad
On the signature of >>= monad operator
You're definitely on to something in the way that you're stating this—there are many things that Monad means and you've separated them out well.
That said, I would definitely say that chaining operations is not the primary thing solved by Monads. That can be solved using plain Functors (with some trouble) or easily with Applicatives. You need to use the full monad spec when "chaining with choice". In particular, the tension between Applicative and Monad comes from Applicative needing to know the entire structure of the side-effecting computation statically. Monad can change that structure at runtime and thus sacrifices some analyzability for power.
To make the point more clear, the only place you deal with a Monad but not any specific monad is if you're defining something with polymorphism constrained to be a Monad. This shows up repeatedly in the Control.Monad module, so we can examine some examples from there.
sequence :: [m a] -> m [a]
forever :: m a -> m b
foldM :: (a -> b -> m a) -> a -> [b] -> m a
Immediately, we can throw out sequence as being particular to Monad since there's a corresponding function in Data.Traversable, sequenceA which has a type slightly more general than Applicative f => [f a] -> f [a]. This ought to be a clear indicator that Monad isn't the only way to sequence things.
Similarly, we can define foreverA as follows
foreverA :: Applicative f => f a -> f b
foreverA f = flip const <$> f <*> foreverA f
So more ways to sequence non-Monad types. But we run into trouble with foldM
foldM :: (Monad m) => (a -> b -> m a) -> a -> [b] -> m a
foldM _ a [] = return a
foldM f a (x:xs) = f a x >>= \fax -> foldM f fax xs
If we try to translate this definition to Applicative style we might write
foldA :: (Applicative f) => (a -> b -> f a) -> a -> [b] -> f a
foldA _ a [] = pure a
foldA f a (x:xs) = foldA f <$> f a x <*> xs
But Haskell will rightfully complain that this doesn't typecheck--each recursive call to foldA tries to put another "layer" of f on the result. With Monad we could join those layers down, but Applicative is too weak.
So how does this translate to Applicatives restricting us from runtime choices? Well, that's exactly what we express with foldM, a monadic computation (a -> b -> m a) which depends upon its a argument, a result from a prior monadic computation. That kind of thing simply doesn't have any meaning in the more purely sequential world of Applicative.
To solve the problem of chaining operations on an individual monadic type, it's not at all necessary to make it an instance of Monad and be sure the monad laws are satisfied. You could just implement a chaining operation directly on your type.
It would probably be very similar to the monadic bind, but not necessarily exactly the same (recall that bind for lists is concatMap, a function that exists anyway, but with the arguments in a different order). And you wouldn't have to worry about the monad laws, because you would have a slightly different interface for each type, so they wouldn't have any common requirements.
To ask what problem the Monad type class itself solves, look at all the functions (in Control.Monad and else where) that work on values in any monadic type. The problem solved is code reuse! Monad is exactly the part of all the monadic types that is common to each and every one of them. That part is sufficient on its own to write useful computations. All of these functions could be implemented for any individual monadic type (often more directly), but they've already been implemented for all monadic types, even the ones that don't exist yet.
You don't write a Monad instance so that you can chain operations on your type (often you already have a way of chaining, in fact). You write a Monad instance for all the code that automatically comes along with the Monad instance. Monad isn't about solving any problem for any single type, it's about a way of viewing many disparate types as instances of a single unifying concept.

Avoiding lift with monad transformers

I have a problem to which a stack of monad transformers (or even one monad transformer) over IO. Everything is good, except that using lift before every action is terribly annoying! I suspect there is really nothing to do about that, but I thought I'd ask anyway.
I am aware of lifting entire blocks, but what if the code is really of mixed types? Would it not be nice if GHC threw in some syntactic sugar (for example, <-$ = <- lift)?
For all the standard mtl monads, you don't need lift at all. get, put, ask, tell — they all work in any monad with the right transformer somewhere in the stack. The missing piece is IO, and even there liftIO lifts an arbitrary IO action down an arbitrary number of layers.
This is done with typeclasses for each "effect" on offer: for example, MonadState provides get and put. If you want to create your own newtype wrapper around a transformer stack, you can do deriving (..., MonadState MyState, ...) with the GeneralizedNewtypeDeriving extension, or roll your own instance:
instance MonadState MyState MyMonad where
get = MyMonad get
put s = MyMonad (put s)
You can use this to selectively expose or hide components of your combined transformer, by defining some instances and not others.
(You can easily extend this approach to all-new monadic effects you define yourself, by defining your own typeclass and providing boilerplate instances for the standard transformers, but all-new monads are rare; most of the time, you'll get by simply composing the standard set offered by mtl.)
You can make your functions monad-agnostic by using typeclasses instead of concrete monad stacks.
Let's say that you have this function, for example:
bangMe :: State String ()
bangMe = do
str <- get
put $ str ++ "!"
-- or just modify (++"!")
Of course, you realize that it works as a transformer as well, so one could write:
bangMe :: Monad m => StateT String m ()
However, if you have a function that uses a different stack, let's say ReaderT [String] (StateT String IO) () or whatever, you'll have to use the dreaded lift function! So how is that avoided?
The trick is to make the function signature even more generic, so that it says that the State monad can appear anywhere in the monad stack. This is done like this:
bangMe :: MonadState String m => m ()
This forces m to be a monad that supports state (virtually) anywhere in the monad stack, and the function will thus work without lifting for any such stack.
There's one problem, though; since IO isn't part of the mtl, it doesn't have a transformer (e.g. IOT) nor a handy type class per default. So what should you do when you want to lift IO actions arbitrarily?
To the rescue comes MonadIO! It behaves almost identically to MonadState, MonadReader etc, the only difference being that it has a slightly different lifting mechanism. It works like this: you can take any IO action, and use liftIO to turn it into a monad agnostic version. So:
action :: IO ()
liftIO action :: MonadIO m => m ()
By transforming all of the monadic actions you wish to use in this way, you can intertwine monads as much as you want without any tedious lifting.

Resources