So I have this sort of code all over my first serious haskell project:
f :: (MonadTrans t) => ExceptT () (t (StateT A B)) C
f = do mapExceptT lift $ do
lift $ do
...
lift $ do
...
r <- ...
...
return r
>>= \r -> ...
There definitely may be something wrong about how I try to achieve my goals (there might be simpler ways how to do it) but currently I am interested in learning how to handle a stack of monad transformers in some nicer way, if there is one. This is the only way I figured out how to get r in the context of B and lift it to a monad higher in the stack. Lifting whole blocks instead of initial statements is as far as I could get on my own.
What I also often end up with are chains of lift which I found out can be avoided with liftIO if the deep monad is IO. I am not aware of a generic way for other monads though.
Is there a pattern that one can follow when he ends up dealing with such stacks, and having to extract one value at some level, a different value at a different level, combine these and affect any of the two levels or maybe yet another one?
Can the stack be manipulated somehow without neither lifting whole blocks (which causes let and bound variables to be scoped and restricted to the inner block) nor having to lift . lift . ... lift individual actions?
This is a well-known problem with monad transformers in general. Researchers have devised various ways of handling it, none of which is clearly "best". Some of the known solutions include:
The mtl approach, which automatically lifts type classes of monads over its built-in monad transformers (and only its built-in monad transformers). This allows you to just write f :: (MonadState A m, MonadError () m) => m C if those are the only features of the monad that your function is using. Due to its extreme non-portability and a few other reasons, mtl is generally considered pseudo-deprecated. See this page and this question for the gory details.
If you have a particular monad stack you are using over and over again, you can wrap it in a newtype and write instances of the various monad type classes it supports manually. For Functor, Applicative, Monad, and any other type classes implemented by the top-level transformer in your stack, you can use GeneralizedNewtypeDeriving to have the compiler write the instances for you automatically; for other type classes, you will have to insert the appropriate number of lift calls for each method. The advantage of this approach is that it's more general and simpler to understand while giving you the same flexibility at the call site as mtl. The big problem with this approach is that it encourages using a single "mega-monad" for all operations rather than specifying only the needed operations, since adding any new monad transformer to the stack requires writing a whole new list of instances.
In most cases, you don't really want a monad that has "some arbitrary state of type A" and "some arbitrary exception-throwing capability". Rather, the different features offered by your monad stack have some semantic meaning in your mental model of your program. A variation of the previous approach is to create custom type classes for the effects beyond the basic Functor, Applicative, and Monad and write instances for the custom type classes on your newtype'd monad instead. This has a major advantage over the other approaches listed here: you can have a stack with multiple copies of the same monad transformer in it at different positions. This is the strategy I've used the most in my own programs so far.
A completely different approach is effect systems. Normally, an effect system has to be built in to a language's type system, but it's possible to encode an effect system in Haskell's type system. See the effect-monads package.
The usual approach is to use the mtl library rather than using transformers directly. I'm not sure what the story behind your t is, but the usual mtl approach is to use very general type signatures at definition sites, like
foo :: (MonadError e m, MonadState s m) => m Int
Then fix the actual transformer stacks at the call sites. A common recommendation is to wrap up the stack in a newtype to avoid muddying things up where they're used.
If this isn't your style (and it's not for everyone), you can still use the mtl methods to perform operations, while giving an explicit transformer stack. This should cut down on the manual lifting substantially. The advantage of this approach is that it gives you a better view of the interactions of effects at the definition sites; the disadvantage is that more code needs all the information.
Related
I have a question about MonadError, namely instead of using MonadError, it is better to use Except?
MonadError and Except are not mutually exclusive. They are things of different kinds.
MonadError is a type class (i.e. an "interface") that defines certain operations (throwError and catchError), which may be implemented by different monads. If a function has a type signature requiring MonadError, it means "I will work in any monad, as long as it implements MonadError, because I make use of the operations defined in it", for example:
f :: MonadError MyErrorType m => Int -> m String
Except, on the other hand, is a specific monad that has a specific underlying implementation. This specific monad happens to implement the operations from MonadError, which means that any function that requires MonadError may be used in Except:
runExcept $ f 42 -- using the definition of `f` above
You could also declare your function to be used specifically in Except, as opposed to "in anything that implements MonadError":
g :: Int -> Except MyErrorType String
Such function can still be used in Except like f can:
runExcept $ g 42
But it cannot be used in other monads that implement MonadError.
The practical consequence of this is that if you're writing some local, small, simple program, it's ok to write all your functions in Except, but if you're writing a bigger system with a lot of modules, or better yet, a reusable library, you're better off writing your functions in MonadError, because then your consumers will have more freedom in consuming your library. In OO world this idea is known as "programming to interface".
One possible drawback of this is that the code might (not always) come out less performant, because the compiler may not be able to do enough inlining without knowing the final types of everything. In practice, however, the performance penalty is trivial in most applications. The usual maxim applies: first measure, then optimize.
I'm currently building a new API, and one of the functions it currently provides is:
inSpan :: Tracer -> Text -> IO a -> IO a
I'm looking to move that Tracer into a monad, giving me a signature more like
inSpan :: MonadTracer m => Text -> m a -> m a
The implementation of inSpan uses bracket, which means I have two main options:
class MonadUnliftIO m => MonadTracer m
or
class MonadMask m => MonadTracer m
But which should I prefer? Note that I'm in control of all the types I've mentioned, which makes me slightly lean towards MonadMask as it doesn't enforce IO at the bottom (that is, we could perhaps have a pure MonadTracer instance).
Is there anything else I should consider?
Let's lay out the options first (repeating some of your question in the process):
MonadMask from the exceptions library. This can work on a wide range of monads and transformers, and does not require that the base monad be IO.
MonadUnliftIO from the unliftio-core (or unliftio) library. This library only works for monads with IO at their base, and which is somehow isomorphic to ReaderT env IO.
MonadBaseControl from the monad-control library. This library will require IO at the base, but will allow non-ReaderT.
Now the tradeoffs. MonadUnliftIO is the newest addition to the fray, and has the least developed library support. This means that, in addition to the limitations of what monads can be instances, many good instances just haven't been written yet.
The important question is: why does MonadUnliftIO make this seemingly arbitrary requirement around ReaderT-like things? This is to prevent issues with lost monadic state. For example, the semantics of bracket_ (put 1) (put 2) (put 3) are not really clear, and therefore MonadUnliftIO disallows a StateT instance.
MonadBaseControl relaxes the ReaderT restriction and has wider library support. It's also considered more complicated internally than the other two, but for your usages that shouldn't really matter. And it allows you to make mistakes with the monadic state as mentioned above. If you're careful in your usage, this won't matter.
MonadMask allows totally pure transformer stacks. I think there's a good argument to be had around the usefulness of modeling asynchronous exceptions in a pure stack, but I understand this kind of approach is something people want to do sometimes. In exchange for getting more instances, you still have the limitations around monadic state, plus the inability to lift some IO control actions, like timeout or forkIO.
My recommendation:
If you want to match the way most people are doing things today, it's probably best to choose MonadMask, it's the most well adopted solution.
If you want that goal, but you also need to do a timeout or withMVar or something, use MonadBaseControl.
And if you know there's a specific set of monads you need compatibility with, and want compile time guarantees about the correctness of your code vis-a-vis monadic state, use MonadUnliftIO.
I am a long time monad transformer user, first time monad transformer writer.... And I feel like I've done something unnecessary.
We are working on a project that has multiple DB tables, and hardcoding the set into different monad stacks was becoming unwieldy, so we decided to break it into different pluggable monad transformers, allowing us to pick and choose at the function type level, like this
doSomething::(HasUserTable m, HasProductTable m)=>Int->m String
(HasXTable is the class, XTableT is the concrete monad transformer). These separate monad transformers could be inserted or removed in a fully modular fashion, and would store the DB handles, require ResourceT, etc....
My first attempt was to just wrap around ReaderT, which would be used to hold the DB handle. It became immediately apparent that this would not work, as ReaderT (and StateT, etc) can not be stacked without using chains of hardcoded "lift"s, thus breaking the pluggable modularity of the stack elements.
The only solution seemed to be to write completely separate copies of the ReaderT monad, each allowing access to the others at a lower level. This works, but the solution is filled with boilerplate code, something like this
class HasUserTable m where
getUser::String->m User
newtype UserTableT m r = UserTableT{runUserTableT::String->m r}
--Standard monad instance stuff, biolerplate copy of ReaderT
instance Functor m=>Functor (UserTableT m) where....
instance Applicative m=>Applicative (UserTableT m) where....
instance Monad m=>Monad (UserTableT m) where....
instance Monad m=>HasUserTable (UserTableT m) where....
--Gotta hardcode passthrough rules to every other monad transformer
--in the world, mostly using "lift"....
instance MonadTrans BlockCacheT where....
instance (HasUserTable m, Monad m)=>HasUserTable (StateT a m)....
instance (HasUserTable m, Monad m)=>HasUserTable (ResourceT m)....
.... etc for all other monad transformers
--Similarly, need to hardcode passthrough rules for all other monads
--through the newly created one
instance MonadResource m=>MonadResource (UserTableT m) where....
instance MonadState a m=>MonadState a (UserTableT m) where....
instance (MonadBaseControl IO m) => MonadBaseControl IO (UserTableT m)....
.... etc for all other monad transformers
What makes this even worse is that we need to add even more passthrough rules for each new monad transformer we add (ie- each new table we add needs to passthrough all the other table monad transformers, so we need n^2 instance declarations!)
Is there a cleaner way to do this?
Yep, that's one of the problems with monad transformers: when you add a new transformer, you have to write an ever-increasing number of boilerplate instances. It's n instances each time, for a total of O(n^2) instances. You can observe this scaling issue in the mtl source code, for example. Monad transformers are not readily extensible.
Now, a good percentage of the monads we use on a daily basis can be expressed as some combination of the transformers provided by mtl, which means that someone else has already done the work of writing all of those boring instances. But those transformers certainly don't cover every monad, and you'll get bitten whenever you need to write your own.
That's why there's ongoing effort being put into devising new approaches to effect typing. A good example in Haskell is Kiselyov et al's extensible-effects library, which takes an algebraic approach to effect typing, based on free monads. The design of this library is described in two articles: An Alternative to Monad Transformers, which spends some time describing problems with the mtl approach, and More Extensible Effects, describing an updated and optimised implementation of the library.
If you want to see how far safe and extensible effect typing can be taken, see Edwin Brady's effects library for the Idris language. There exist quite a lot of resources explaining effects: a tutorial, the original Programming and Reasoning with Algebraic Effects article, and Resource-dependent Algebraic Effects describing some newer features of effects. There's probably some more resources that I've forgotten in this list.
I read an article which said:
Providing instances for the many standard type-classes [Functors] will immediately give you a lot of functionality for practically free
My question is: what is this functionality that you get for free (for functors or other type-classes)? I know what the definition of a functor is, but what do I get for free by defining something as a functor/other type-class. Something other than a prettier syntax. Ideally this would be general and useful functions that operate on functors/other type-classes.
My imagination (could be wrong) of what free means is functions of this sort: TypeClass x => useful x y = ..
== Edit/Additition ==
I guess I'm mainly asking about the more abstract (and brain boggling) type-classes, like the ones in this image. For less abstract classes like Ord, my object oriented intuition understands.
Functors are simple and probably not the best example. Let's look at Monads instead:
liftM - if something is a Monad, it is also a Functor where liftM is fmap.
>=>, <=<: you can compose a -> m b functions for free where m is your monad.
foldM, mapM, filterM... you get a bunch of utility functions that generalize existing functions to use your monad.
when, guard* and unless -- you also get some control functions for free.
join -- this is actually fairly fundamental to the definition of a monad, but you don't need to define it in Haskell since you've defined >>=.
transformers -- ErrorT and stuff. You can bolt error handling onto your new type, for free (give or take)!
Basically, you get a wide variety of standard functions "lifted" to use your new type as soon as you make it a Monad instance. It also becomes trivial (but alas not automatic) to make it a Functor and Applicative as well.
However, these are all "symptoms" of a more general idea. You can write interesting, nontrivial code that applies to all monads. You might find some of the functions you wrote for your type--which are useful in your particular case, for whatever reason--can be generalized to all monads. Now you can suddenly take your function and use it on parsers, and lists, and maybes and...
* As Daniel Fischer helpfully pointed out, guard requires MonadPlus rather than Monad.
Functors are not very interesting by themselves, but they are a necessary stepping stone to get into applicative functors and Traversables.
The main property which makes applicative functors useful is that you can use fmap with the applicative operator <*> to "lift" any function of any arity to work with applicative values. I.e. you can turn any a -> b -> c -> d into Applicative f => f a -> f b -> f c -> f d. You can also take a look at Data.Traversable and Data.Foldable which contain several general purpose functions that involve applicative functors.
Alternative is a specialized applicative functor which supports choice between alternatives that can "fail" (the exact meaning of "empty" depends in the applicative instance). Applicative parsers are one practical example where the definitions of some and many are very intuitive (e.g. match some pattern zero-or-more times or one-or-more times).
Monads are one of the most interesting and useful type-classes, but they are already well covered by the other answers.
Monoid is another type-class that is both simple and immediately useful. It basically defines a way to add two pieces of data together, which then gives you a generic concat as well as functionality in the aforementioned Foldable module and it also enables you to use the Writer monad with the data type.
There are many of the standard functions in haskell that require that their arguments implement one or more type-classes. Doing so in your code allows other developers (or yourself) to use your data in ways they are already familiar with, without having to write additional functions.
As an example, implementing the Ord type-class will allow you to use things like sort, min, max, etc. Where otherwise, you would need sortBy and the like.
Yes, it means that implementing the type class Foo gives you all the other functions that have a Foo constraint "for free".
The Functor type class isn't too interesting in that regard, as it doesn't give you a lot.
A better example is monads and the functions in the Control.Monad module. Once you've defined the two Monad functions (>>=) and return for your type, you get another thirty or so functions that can then be used on your type.
Some of the more useful ones include: mapM, sequence, forever, join, foldM, filterM, replicateM, when, unless and liftM. These show up all the time in Haskell code.
As others have said, Functor itself doesn't actually get you much for free. Basically, the more high-level or general a typeclass is (meaning the more things fit that description), then the less "free" functionality you are going to get. So for example, Functor, and Monoid don't provide you with much, but Monad and Arrow provide you with a lot of useful functions for free.
In Haskell, it's still a good idea to write an instance for Functor and Monoid though (if your data type is indeed a functor or a monoid), because we almost always try to use the most general interface possible when writing functions. If you are writing a new function that can get away with only using fmap to operate on your data type, then there is no reason to artificially restrict that function to to Monads or Applicatives, since it might be useful later for other things.
Your object-oriented intuition carries across, if you read "interface and implementation" for "typeclass and instance". If you make your new type C an instance of a standard typeclass B, then you get for free that your type will work with all existing code A that depends on B.
As others have said, when the typeclass is something like Monad, then the freebies are the many library functions like foldM and when.
One of the problems with monad transformers I find is the need to lift the operations into the right monad. A single lift here and there isn't bad, but sometimes there are functions that looks like this:
fun = do
lift a
lift b
c
lift d
lift e
f
I would like to be able to write this function thus:
fun = monadInvert $ do
a
b
lift c
d
e
lift f
This halves the number of lifts and makes the code cleaner.
The question is: for what monads is monadInvert possible? How should one create this function?
Bonus points: define it for monad m which is an instance of MonadIO.
The title of this question speaks of permutations: indeed, how can we deal with arbitrary permutations of a monad tranformer stack?
Well, first of all, you don't actually need so much lifting. For monad transformers the following identities hold:
lift c >>= lift . f = lift (c >>= f)
lift c1 >> lift c2 = lift (c1 >> c2)
It's not uncommon to write:
x <- lift $ do
{- ... -}
Next is: When you use libraries like mtl or monadLib (i.e. type class based libraries instead of transformers directly), you actually can access most underlying monads directly:
c :: StateT MyState (ReaderT MyConfig SomeOtherMonad) Result
c = do
x <- ask
y <- get
{- ... -}
Finally, if you really need a lot of lifting despite these two points, you should consider writing a custom monad or even use an entirely different abstraction. I find myself using the automaton arrow for stateful computations instead of a state monad.
You might be interested in Monads, Zippers and Views, Virtualizing the Monad Stack by Tom Schrijvers and Bruno Oliveira.
This doesn't address your point about reducing the lifts, but it's an interesting approach to your "monad permutations" problem.
Here's the abstract:
This work aims at making monadic components more reusable and robust
to changes by employing two new techniques for virtualizing the monad
stack: the monad zipper and monad views. The monad zipper is a monad
transformer that creates virtual monad stacks by ignoring particular
layers in a concrete stack. Monad views provide a general framework
for monad stack virtualization: they take the monad zipper one step
further and integrate it with a wide range of other virtualizations.
For instance, particular views allow restricted access to monads in
the stack. Furthermore, monad views can be used by components to
provide a call-by-reference-like mechanism to access particular layers
of the monad stack. With these two mechanisms component requirements
in terms of the monad stack shape no longer need to be literally
reflected in the concrete monad stack, making these components more
reusable and robust to changes.
I am mostly sure that what you are decribing is impossible for IO, which is always the innermost monad:
From Martin Grabmüller: Monad Transformers Step by Step, available at
http://www.grabmueller.de/martin/www/pub/
Down in this document, we call liftIO in eval6 to perform I/O actions.
Why do we need to lift in this case? Because there is no IO class for
which we can instantiate a type as. Therefore, for I/O actions, we
have to call lift to send the commands inwards
In general, for monads less restrictive than IO, (such as Error and State) order still matters for semantics, so you can't change the order of the stack just to make syntax more convenient.
For some monads, like Reader, that are central (in that they do commute in the stack), your idea seems not clearly impossible. I don't know how to write it though. I guess it would be a type class CentralMonad that ReaderT is an instance of, with some implementation...