Related
I've recently realized this thing:
On the one hand:
Constraints specified on a class header, must be specified again on an instance of that class, but any use of that class as a constraint somewhere else does not need to reimport the class constraints. They are implicitly satisfied.
class (Ord a) => ClassA a where methodA :: a -> Bool -- i decided to put constraint (Ord a) in the class header
instance (Ord a) => ClassA a where methodA x = x <= x -- compiler forces me to add (Ord a) => in the front
class OtherClassA a where otherMethodA :: a -> Bool
instance (ClassA a) => OtherClassA a where otherMethodA x = x <= x && methodA x -- i don't need to specify (Ord a) so it must be brought implicitly in context
On the other hand:
A constraint specified in a class method, does not need to be specified again on an instance of that class, but any use of that class as a constraint somewhere else, needs to reimport specific constraints for method used.
class ClassB a where methodB :: (Ord a) => a -> Bool -- i decided to put constraint (Ord a) in the method
instance ClassB a where methodB x = x <= x -- i don't need to specify (Ord a) so it must be implicitly in context
class OtherClassB a where otherMethodB :: a -> Bool
instance (ClassB a, Ord a) => OtherClassB a where otherMethodB = methodB -- compiler forced me to add (Ord a)
What is the motivation for those behaviors? Would it not be preferrable to be explicit about constraints at all times?
More concretely when I have a set of conditions I know all methods in a type class should satisfy, should I write the conditions, in the type class header, or in front of each method?
Should I write constraints in a type class definition at all?
Short Answer
Here's my general advice on constraints in class declarations and instance definitions. See below for a longer explanation and a detailed description of your examples.
If you have classes with a logical relationship such that it is logically impossible for a type to belong to class Base without belonging to class Super, use a constraint in the class declaration, like so:
class Super a => Base a where ...
Some example:
-- all Applicatives are necessarily Functors
class Functor f => Applicative f where ...
-- All orderable types can also be tested for equality
class Eq f => Ord f where ...
-- Every HTMLDocument also supports Document methods
class Document doc => HTMLDocument doc where ...
Avoid writing instances that apply to all types, with or without constraints. With a few exceptions, these usually point to a design flaw:
-- don't do this
instance SomeClass1 a
-- or this
instance (Eq a) => SomeClass1 a
Instances for higher-order types make sense though, and use whatever constraints are necessary for the instance to compile:
instance (Ord a, Ord b) => Ord (a, b) where
compare (x1,x2) (y1,y2) = case compare x1 x2 of
LT -> LT
GT -> GT
EQ -> compare x2 y2
Don't use constraints on class methods, except where a class should support different subsets of methods on different types, depending on the constraints available.
Long Answer
Constraints in class declarations and instance definitions have different meanings and different purposes. A constraint in a class declaration like:
class (Big a) => Small a
defines Big as a "superclass" of Small and represents a type-level claim of a logical necessity: any type of class Small is necessarily also a type of class Big. Having such a constraint improves type correctness (since any attempt to define a Small instance for a type a that doesn't also have a Big instance -- a logical inconsistency -- will be rejected by the compiler) and convenience, since a Small a constraint will automatically make available the Big class interface in addition to the Small interface.
As a concrete example, in modern Haskell, Functor is a superclass of Applicative which is a superclass of Monad. All Monads are Applicatives and all Applicatives are Functors, so this superclass relationship reflects a logical relationship between these collections of types, and also provides the convenience of being able to use the monad (do-notation, >>=, and return), applicative (pure and <*>) and functor (fmap or <$>) interfaces using only a Monad m constraint.
A consequence of this superclass relationship is that any Monad instance must also be accompanied by an Applicative and Functor instance to provide evidence to the compiler that the necessary superclass constraints are satisfied.
In contrast, a constraint in an instance definition introduces a dependency of the specific, defined instance on another instance. Most commonly, I see this used to define instances for classes of high-order types, like the Ord instance for lists:
instance Ord a => Ord [a] where ...
That is, an Ord [a] instance can be defined for any type a using lexicographic ordering of the list, provided that the type a itself can be ordered. The constraint here does not (and indeed could not) apply to all Ord types. Rather, the instance definition provides an instance for all lists by introducing a dependency on an instance for the element type -- it says that an Ord [a] instance is available for any type a that has an Ord a instance available.
Your examples are somewhat unusual, as one doesn't normally define an instance:
instance SomeClass a where ...
that applies to all types a, with or without additional constraints.
Nonetheless, what's happening is that:
class (Ord a) => ClassA a
is introducing a logical type-level fact, namely that all types of class ClassA are also of class Ord. Then, you are presenting an instance of ClassA applicable to all types:
instance ClassA a
But, this presents a problem to the compiler. Your class declaration has stated that is it logically necessary that all types of ClassA also belong to class Ord, and the compiler requires proof of the Ord a constraint for any instance ClassA a that you define. By writing instance ClassA a, you are making the bold claim that all types are of ClassA, but the compiler has no evidence that all classes have the necessary Ord a instances. For this reason, you must write:
instance (Ord a) => ClassA a
which, in words, says "all types a have an instance of ClassA provided an Ord a instance is also available". The compiler accepts this as proof that you are only defining instances for the those types a that have the requisite Ord a instance.
When you go on to define:
class OtherClassA a where
otherMethodA :: a -> Bool
instance (ClassA a) => OtherClassA a where
otherMethodA x = x <= x && methodA x
since OtherClassA has no superclass, there is no logical necessity that types of this class are also of class Ord, and the compiler will require no proof of this. In your instance definition however, you define an instance applicable to all types whose implementation requires Ord a, as well as ClassA a. Fortunately, you've provided a ClassA a constraint, and because Ord is a superclass of ClassA, it is a logical necessity that any a with a ClassA a constraint also has an Ord a constraint, so the compiler is satisfied that a has both required instances.
When you write:
class ClassB a where
methodB :: (Ord a) => a -> Bool
you are doing something unusual, and the compiler tries to warn by refusing to compile this unless you enable the extension ConstrainedClassMethods. What this definition says is that there is no logical necessity that types of class ClassB also be of class Ord, so you're free to define instances that lack the require instance. For example:
instance ClassB (Int -> Int) where
methodB _ = False
which defines an instance for functions Int -> Int (and this type has no Ord instance). However, any attempt to use methodB on such a type will demand an Ord instance:
> methodB (*(2::Int))
... • No instance for (Ord (Int -> Int)) ...
This can be useful if there are several methods and only some of them require constraints. The GHC manual gives the following example:
class Seq s a where
fromList :: [a] -> s a
elem :: Eq a => a -> s a -> Bool
You can define sequences Seq s a with no logical necessity that the elements a be comparable. But, without Eq a, you're only allowed to use a subset of the methods. If you try to use a method that needs Eq a with a type a that doesn't have such an instance, you'll get an error.
Anyway, your instance:
instance ClassB a where
methodB x = x <= x
defines an instance for all types (without requiring any evidence of Ord a, since there's no logical necessity here), but you can only use methodB on the subset of types with an Ord instance.
In your final example:
class OtherClassB a where
otherMethodB :: a -> Bool
there's no logical necessity that a type of class OtherClassB also be a type of class Ord, and there's no requirement that otherMethodB only be used with types having an Ord a instance. You could, if you wanted, define the instance:
instance OtherClassB a where
otherMethodB _ = False
and it would compile fine. However, by defining the instance:
instance OtherClassB a where
otherMethodB = methodB
you are providing an instance for all types whose implementation uses methodB and so requires ClassB. If you modify this to read:
instance (ClassB a) => OtherClassB a where
otherMethodB = methodB
the compiler still isn't satified. The specific method methodB requires an Ord a instance, but since Ord is not a superclass of ClassB, there is no logical necessity that the constraint ClassB a implies Ord a, so you must provide additionl evidence to the compiler that an Ord a instance is available. By writing:
instance (ClassB a, Ord a) => OtherClassB a where
otherMethodB = methodB
you are providing an instance that requires ClassB a (to run methodB) and Ord a (because methodB has it as an additional requirement), so you need to tell the compiler that this instance applies to all types a provided both ClassB a and Ord a instances are available. The compiler is satisfied with this.
You Don't Need Type Classes to Delay Concrete Types
From your examples and follow-up comments, it sounds like you're (mis)using type classes to support a particular style of programming that avoids committing to concrete types until absolutely necessary.
(As an aside, I used to think this style was a good idea, but I've gradually come around to thinking it's mostly pointless. Haskell's type system makes refactoring so easy that there's little risk to committing to concrete types, and concrete programs tend to be mostly easier to read and write than abstract programs. However, many people have used this style of programming profitably, and I can think of at least one high-quality library (lens) that takes it to absolute extremes very effectively. So, no judgement!)
Anyway, this style of programming is generally better supported by writing top-level polymorphic functions and placing the needed constraints on the functions. There is usually no need (and no point) in defining new type classes. This was what #duplode was saying in the comments. You can replace:
class (Ord a) => ClassA where method :: a -> Bool
instance (Ord a) => ClassA where methodA x = x <= x
with the simpler top-level function definition:
methodA :: (Ord a) => a -> Bool
methodA x = x <= x
because the class and instance serve no purpose. The main point of type classes is to provide ad hoc polymorphism, to allow you to have a single function (methodA) that has different implementations for different types. If there's only one implementation for all types, that's just a plain old parametric polymorphic function, and no type class is needed.
Nothing changes if there are multiple methods, and usually nothing changes if there are multiple constraints. If your philosophy is that data types should be characterized only by the properties they satisfy rather than by what they are, then the flip side of that is that functions should be typed to demand of their argument types only the properties that they need. If they demand more than they need, they are prematurely committing to a more concrete type than necessary.
So, a class for, say, an orderable numeric key type with a printable representation:
class (Ord a, Num a, Show a) => Key a where
firstKey :: a
nextKey :: a -> a
sortKeys :: [a] -> [a]
keyLength :: a -> Int
and a single instance:
instance (Ord a, Num a, Show a) => Key a where
firstKey = 1
nextKey x = x + 1
sortKeys xs = sort xs
keyLength k = length (show k)
is more idiomatically written as a set of functions that constrain the type only based on the properties they require:
firstKey :: (Num key) => key
firstKey = 1
nextKey :: (Num key) => key -> key
nextKey = (+1)
sortKeys :: (Ord key) => [key] -> [key]
sortKeys = sort
keyLength :: (Show key) => key -> Int
keyLength = length . show
On the other hand, if you find it helpful to have a formal "name" for an abstract type and prefer the compiler's help in enforcing use of this type instead of just using type variables like "key" with evocative names, I guess you can use type classes for this purpose. However, your type classes probably shouldn't have any methods. You want to write:
class (Ord a, Num a, Show a) => Key a
and then a bunch of top-level functions that use the type class.
firstKey :: (Key k) => k
firstKey = 1
nextKey :: (Key k) => k -> k
nextKey = (+1)
sortKeys :: (Key k) => [k] -> [k]
sortKeys = sort
keyLength :: (Show k) => k -> Int
keyLength = length . show
Your entire program can be written this way, and no instances are actually needed until you get down to choosing your concrete types and documenting them all in one place. For example, in your Main.hs program, you could commit to an Int key by giving an instance for a concrete type and using it:
instance Key Int
main = print (nextKey firstKey :: Int)
This concrete instance also avoids the need for extensions like undecidable instances and warning about fragile bindings.
I've just read this paper ("Type classes: an exploration of the design space" by Peyton Jones & Jones), which explains some challenges with the early typeclass system of Haskell, and how to improve it.
Many of the issues that they raise are related to context reduction which is a way to reduce the set of constraints over instance and function declarations by following the "reverse entailment" relationship.
e.g. if you have somewhere instance (Ord a, Ord b) => Ord (a, b) ... then within contexts, Ord (a, b) gets reduced to {Ord a, Ord b} (reduction does not always shrink the number of constrains).
I did not understand from the paper why this reduction was necessary.
Well, I gathered it was used to perform some form of type checking. When you have your reduced set of constraint, you can check that there exist some instance that can satisfy them, otherwise it's an error. I'm not too sure what the added value of that is, since you would notice the problem at the use site, but okay.
But even if you have to do that check, why use the result of reduction inside inferred types? The paper points out it leads to unintuitive inferred types.
The paper is quite ancient (1997) but as far as I can tell, context reduction is still an ongoing concern. The Haskell 2010 spec does mention the inference behaviour I explain above (link).
So, why do it this way?
I don't know if this is The Reason, necessarily, but it might be considered A Reason: in early Haskell, type signatures were only permitted to have "simple" constraints, namely, a type class name applied to a type variable. Thus, for example, all of these were okay:
Ord a => a -> a -> Bool
Eq a => a -> a -> Bool
Graph gr => gr n e -> [n]
But none of these:
Ord (Tree a) => Tree a -> Tree a -> Bool
Eq (a -> b) => (a -> b) -> (a -> b) -> Bool
Graph Gr => Gr n e -> [n]
I think there was a feeling then -- and still today, as well -- that allowing the compiler to infer a type which one couldn't write manually would be a bit unfortunate. Context reduction was a way of turning the above signatures either into ones that could be written by hand as well or an informative error. For example, since one might reasonably have
instance Ord a => Ord (Tree a)
in scope, we could turn the illegal signature Ord (Tree a) => ... into the legal signature Ord a => .... On the other hand, if we don't have any instance of Eq for functions in scope, one would report an error about the type which was inferred to require Eq (a -> b) in its context.
This has a couple of other benefits:
Intuitively pleasing. Many of the context reduction rules do not change whether the type is legal, but do reflect things humans would do when writing the type. I'm thinking here of the de-duplication and subsumption rules that let you turn, e.g. (Eq a, Eq a, Ord a) into just Ord a -- a transformation one definitely would want to do for readability.
This can frequently catch stupid errors; rather than inferring a type like Eq (Integer -> Integer) => Bool which can't be satisfied in a law-abiding way, one can report an error like Perhaps you did not apply a function to enough arguments?. Much friendlier!
It becomes the compiler's job to pinpoint what went wrong. Instead of inferring a complicated context like Eq (Tree (Grizwump a, [Flagle (Gr n e) (Gr n' e') c])) and complaining that the context is not satisfiable, it instead is forced to reduce this to the constituent constraints; it will instead complain that we couldn't determine Eq (Grizwump a) from the existing context -- a much more precise and actionable error.
I think this is indeed desirable in a dictionary passing implementation. In such an implementation, a "dictionary", that is, a tuple or record of functions is passed as implicit argument for every type class constraint in the type of the applied function.
Now, the question is simply when and how those dictionaries are created. Observe that for simple types like Int by necessity all dictionaries for whatever type class Int is an instance of will be a constant.
Not so in the case of parameterized types like lists, Maybe or tuples. It is clear that to show a tuple, for instance, the Show instances of the actual tuple elements need to be known. Hence such a polymorphic dictionary cannot be a constant.
It appears that the principle guiding the dictionary passing is such that only dictionaries for types that appear as type variables in the type of the applied function are passed. Or, to put it differently: no redundant information is replicated.
Consider this function:
f :: (Show a, Show b) => (a,b) -> Int
f ab = length (show ab)
The information that a tuple of show-able components is also showable, thus a constraint like Show (a,b) needs not to appear when we already know (Show a, Show b).
An alternative implementation would be possible, though, where the caller .would be responsible to create and pass dictionaries. This could work without context reduction, such that the type of f would look like:
f :: Show (a,b) => (a,b) -> Int
But this would mean that the code to create the tuple dictionary would have to be repeated on every call site. And it is easy to come up with examples where the number of necessary constraints actually increases, like in:
g :: (Show (a,a), Show(b,b), Show (a,b), Show (b, a)) => a -> b -> Int
g a b = maximum (map length [show (a,a), show (a,b), show (b,a), show(b,b)])
It is instructive to implement a type class/instance system with actual records that are explicitly passed. For example:
data Show' a = Show' { show' :: a -> String }
showInt :: Show' Int
showInt = Show' { show' = intshow } where
intshow :: Int -> String
intshow = show
Once you do this you will probably easily recognize the need for "context reduction".
I need to write a Serialize instance for the following data type:
data AnyNode = forall n . (Typeable n, Serialize n) => AnyNode n
Serializing this is no problem, but I can't implement deserialization, since the compiler has no way to resolve the specific instance of Serialize n, since the n is isolated from the outer scope.
There's been a related discussion in 2006. I am now wondering whether any sort of solution or a workaround has arrived today.
You just tag the type when you serialize, and use a dictionary to untag the type when you deserialize. Here's some pseudocode omitting error checking etc:
serialAnyNode (AnyNode x) = serialize (typeOf n, serialize x)
deserialAnyNode s = case deserialize s of
(typ,bs) -> case typ of
"String" -> AnyNode (deserialize bs :: String)
"Int" -> AnyNode (deserialize bs :: Int)
....
Note that you can only deserialize a closed universe of types with your function. With some extra work, you can also deserialize derived types like tuples, maybes and eithers.
But if I were to declare an entirely new type "Gotcha" deriving Typeable and Serialize, deserialAnyNode of course couldn't deal with it without extension.
You need to have some kind of centralised "registry" of deserialization functions so you can dispatch on the actual type (extracted from the Typeable information). If all types you want to deserialize are in the same module this is pretty easy to set up. If they are in multiple modules you need to have one module that has the mapping.
If your collection of types is more dynamic and not easily available at compile time, you can perhaps use the dynamic linking to gain access to the deserializers. For each type that you want to deserialize you export a C callable function with a name derived from the Typeable information (you could use TH to generate these). Then at runtime, when you want to deserialize a type, generate the same name and the use the dynamic linker to get hold of the address of the function and then an FFI wrapper to get a Haskell callable function. This is a rather involved process, but it can be wrapped up in a library. No, sorry, I don't have such a library.
It's hard to tell what you're asking here, exactly. You can certainly pick a particular type T, deserialize a ByteString to it, and store it in an AnyNode. That doesn't do the user of an AnyNode much good, though -- you still picked T, after all. If it wasn't for the Typeable constraint, the user wouldn't even be able to tell what the type is (so let's get rid of the Typeable constraint because it makes things messier). Maybe what you want is a universal instead of an existential.
Let's split Serialize up into two classes -- call them Read and Show -- and simplify them a bit (so e.g. read can't fail).
So we have
class Show a where show :: a -> String
class Read a where read :: String -> a
We can make an existential container for a Show-able value:
data ShowEx where
ShowEx :: forall a. Show a => a -> ShowEx
-- non-GADT: data ShowEx = forall a. Show a => ShowEx a
But of course ShowEx is isomorphic to String, so there isn't a whole lot point to this. But note that an existential for Read is has even less point:
data ReadEx where
ReadEx :: forall a. Read a => a -> ReadEx
-- non-GADT: data ReadEx = forall a. Read a => ReadEx a
When I give you a ReadEx -- i.e. ∃a. Read a *> a -- it means that you have a value of some type, and you don't know what the type is, but you can a String into another value of the same type. But you can't do anything with it! read only produces as, but that doesn't do you any good when you don't know what a is.
What you might want with Read would be a type that lets the caller choose -- i.e., a universal. Something like
newtype ReadUn where
ReadUn :: (forall a. Read a => a) -> ReadUn
-- non-GADT: newtype ReadUn = ReadUn (forall a. Read a => a)
(Like ReadEx, you could make ShowUn -- i.e. ∀a. Show a => a -- and it would be just as useless.)
Note that ShowEx is essentially the argument to show -- i.e. show :: (∃a. Show a *> a) -> String -- and ReadUn is essentially the return value of read -- i.e. read :: String -> (∀a. Read a => a).
So what are you asking for, an existential or a universal? You can certainly make something like ∀a. (Show a, Read a) => a or ∃a. (Show a, Read a) *> a, but neither does you much good here. The real issue is the quantifier.
(I asked a question a while ago where I talked about some of this in another context.)
In Scrap your boilerplate reloaded, the authors describe a new presentation of Scrap Your Boilerplate, which is supposed to be equivalent to the original.
However, one difference is that they assume a finite, closed set of "base" types, encoded with a GADT
data Type :: * -> * where
Int :: Type Int
List :: Type a -> Type [a]
...
In the original SYB, type-safe cast is used, implemented using the Typeable class.
My questions are:
What is the relationship between these two approaches?
Why was the GADT representation chosen for the "SYB Reloaded" presentation?
[I am one of the authors of the "SYB Reloaded" paper.]
TL;DR We really just used it because it seemed more beautiful to us. The class-based Typeable approach is more practical. The Spine view can be combined with the Typeable class and does not depend on the Type GADT.
The paper states this in its conclusions:
Our implementation handles the two central ingredients of generic programming differently from the original SYB paper: we use overloaded functions with
explicit type arguments instead of overloaded functions based on a type-safe
cast 1 or a class-based extensible scheme [20]; and we use the explicit spine
view rather than a combinator-based approach. Both changes are independent
of each other, and have been made with clarity in mind: we think that the structure of the SYB approach is more visible in our setting, and that the relations
to PolyP and Generic Haskell become clearer. We have revealed that while the
spine view is limited in the class of generic functions that can be written, it is
applicable to a very large class of data types, including GADTs.
Our approach cannot be used easily as a library, because the encoding of
overloaded functions using explicit type arguments requires the extensibility of
the Type data type and of functions such as toSpine. One can, however, incorporate Spine into the SYB library while still using the techniques of the SYB
papers to encode overloaded functions.
So, the choice of using a GADT for type representation is one we made mainly for clarity. As Don states in his answer, there are some obvious advantages in this representation, namely that it maintains static information about what type a type representation is for, and that it allows us to implement cast without any further magic, and in particular without the use of unsafeCoerce. Type-indexed functions can also be implemented directly by using pattern matching on the type, and without falling back to various combinators such as mkQ or extQ.
Fact is that I (and I think the co-authors) simply were not very fond of the Typeable class. (In fact, I'm still not, although it is finally becoming a bit more disciplined now in that GHC adds auto-deriving for Typeable, makes it kind-polymorphic, and will ultimately remove the possibility to define your own instances.) In addition, Typeable wasn't quite as established and widely known as it is perhaps now, so it seemed appealing to "explain" it by using the GADT encoding. And furthermore, this was the time when we were also thinking about adding open datatypes to Haskell, thereby alleviating the restriction that the GADT is closed.
So, to summarize: If you actually need dynamic type information only for a closed universe, I'd always go for the GADT, because you can use pattern matching to define type-indexed functions, and you do not have to rely on unsafeCoerce nor advanced compiler magic. If the universe is open, however, which is quite common, certainly for the generic programming setting, then the GADT approach might be instructive, but isn't practical, and using Typeable is the way to go.
However, as we also state in the conclusions of the paper, the choice of Type over Typeable isn't a prerequisite for the other choice we're making, namely to use the Spine view, which I think is more important and really the core of the paper.
The paper itself shows (in Section 8) a variation inspired by the "Scrap your Boilerplate with Class" paper, which uses a Spine view with a class constraint instead. But we can also do a more direct development, which I show in the following. For this, we'll use Typeable from Data.Typeable, but define our own Data class which, for simplicity, just contains the toSpine method:
class Typeable a => Data a where
toSpine :: a -> Spine a
The Spine datatype now uses the Data constraint:
data Spine :: * -> * where
Constr :: a -> Spine a
(:<>:) :: (Data a) => Spine (a -> b) -> a -> Spine b
The function fromSpine is as trivial as with the other representation:
fromSpine :: Spine a -> a
fromSpine (Constr x) = x
fromSpine (c :<>: x) = fromSpine c x
Instances for Data are trivial for flat types such as Int:
instance Data Int where
toSpine = Constr
And they're still entirely straightforward for structured types such as binary trees:
data Tree a = Empty | Node (Tree a) a (Tree a)
instance Data a => Data (Tree a) where
toSpine Empty = Constr Empty
toSpine (Node l x r) = Constr Node :<>: l :<>: x :<>: r
The paper then goes on and defines various generic functions, such as mapQ. These definitions hardly change. We only get class constraints for Data a => where the paper has function arguments of Type a ->:
mapQ :: Query r -> Query [r]
mapQ q = mapQ' q . toSpine
mapQ' :: Query r -> (forall a. Spine a -> [r])
mapQ' q (Constr c) = []
mapQ' q (f :<>: x) = mapQ' q f ++ [q x]
Higher-level functions such as everything also just lose their explicit type arguments (and then actually look exactly the same as in original SYB):
everything :: (r -> r -> r) -> Query r -> Query r
everything op q x = foldl op (q x) (mapQ (everything op q) x)
As I said above, if we now want to define a generic sum function summing up all Int occurrences, we cannot pattern match anymore, but have to fall back to mkQ, but mkQ is defined purely in terms of Typeable and completely independent of Spine:
mkQ :: (Typeable a, Typeable b) => r -> (b -> r) -> a -> r
(r `mkQ` br) a = maybe r br (cast a)
And then (again exactly as in original SYB):
sum :: Query Int
sum = everything (+) sumQ
sumQ :: Query Int
sumQ = mkQ 0 id
For some of the stuff later in the paper (e.g., adding constructor information), a bit more work is needed, but it can all be done. So using Spine really does not depend on using Type at all.
Well, obviously the Typeable use is open -- new variants can be added after the fact, and without modifying the original definitions.
The important change though is that in that TypeRep is untyped. That is, there is no connection between the runtime type , TypeRep, and the static type it encodes. With the GADT approach we can encode the mapping between a type a and its Type, given by the GADT Type a.
We thus bake in evidence for the type rep being statically linked to its origin type, and can write statically typed dynamic application (for example) using Type a as evidence that we have a runtime a.
In the older TypeRep case, we have no such evidence and it comes down to runtime string equality, and a coerce and hope for the best through fromDynamic.
Compare the signatures:
toDyn :: Typeable a => a -> TypeRep -> Dynamic
versus GADT style:
toDyn :: Type a => a -> Type a -> Dynamic
I can't fake my type evidence, and I can use that later when reconstructing things, to e.g. lookup the type class instances for a when all I have is a Type a.
When metaprogramming, it may be useful (or necessary) to pass along to Haskell's type system information about types that's known to your program but not inferable in Hindley-Milner. Is there a library (or language extension, etc) that provides facilities for doing this—that is, programmatic type annotations—in Haskell?
Consider a situation where you're working with a heterogenous list (implemented using the Data.Dynamic library or existential quantification, say) and you want to filter the list down to a bog-standard, homogeneously typed Haskell list. You can write a function like
import Data.Dynamic
import Data.Typeable
dynListToList :: (Typeable a) => [Dynamic] -> [a]
dynListToList = (map fromJust) . (filter isJust) . (map fromDynamic)
and call it with a manual type annotation. For example,
foo :: [Int]
foo = dynListToList [ toDyn (1 :: Int)
, toDyn (2 :: Int)
, toDyn ("foo" :: String) ]
Here foo is the list [1, 2] :: [Int]; that works fine and you're back on solid ground where Haskell's type system can do its thing.
Now imagine you want to do much the same thing but (a) at the time you write the code you don't know what the type of the list produced by a call to dynListToList needs to be, yet (b) your program does contain the information necessary to figure this out, only (c) it's not in a form accessible to the type system.
For example, say you've randomly selected an item from your heterogenous list and you want to filter the list down by that type. Using the type-checking facilities supplied by Data.Typeable, your program has all the information it needs to do this, but as far as I can tell—this is the essence of the question—there's no way to pass it along to the type system. Here's some pseudo-Haskell that shows what I mean:
import Data.Dynamic
import Data.Typeable
randList :: (Typeable a) => [Dynamic] -> IO [a]
randList dl = do
tr <- randItem $ map dynTypeRep dl
return (dynListToList dl :: [<tr>]) -- This thing should have the type
-- represented by `tr`
(Assume randItem selects a random item from a list.)
Without a type annotation on the argument of return, the compiler will tell you that it has an "ambiguous type" and ask you to provide one. But you can't provide a manual type annotation because the type is not known at write-time (and can vary); the type is known at run-time, however—albeit in a form the type system can't use (here, the type needed is represented by the value tr, a TypeRep—see Data.Typeable for details).
The pseudo-code :: [<tr>] is the magic I want to happen. Is there any way to provide the type system with type information programatically; that is, with type information contained in a value in your program?
Basically I'm looking for a function with (pseudo-) type ??? -> TypeRep -> a that takes a value of a type unknown to Haskell's type system and a TypeRep and says, "Trust me, compiler, I know what I'm doing. This thing has the value represented by this TypeRep." (Note that this is not what unsafeCoerce does.)
Or is there something completely different that gets me the same place? For example, I can imagine a language extension that permits assignment to type variables, like a souped-up version of the extension enabling scoped type variables.
(If this is impossible or highly impractical,—e.g., it requires packing a complete GHCi-like interpreter into the executable—please try to explain why.)
No, you can't do this. The long and short of it is that you're trying to write a dependently-typed function, and Haskell isn't a dependently typed language; you can't lift your TypeRep value to a true type, and so there's no way to write down the type of your desired function. To explain this in a little more detail, I'm first going to show why the way you've phrased the type of randList doesn't really make sense. Then, I'm going to explain why you can't do what you want. Finally, I'll briefly mention a couple thoughts on what to actually do.
Existentials
Your type signature for randList can't mean what you want it to mean. Remembering that all type variables in Haskell are universally quantified, it reads
randList :: forall a. Typeable a => [Dynamic] -> IO [a]
Thus, I'm entitled to call it as, say, randList dyns :: IO [Int] anywhere I want; I must be able to provide a return value for all a, not simply for some a. Thinking of this as a game, it's one where the caller can pick a, not the function itself. What you want to say (this isn't valid Haskell syntax, although you can translate it into valid Haskell by using an existential data type1) is something more like
randList :: [Dynamic] -> (exists a. Typeable a => IO [a])
This promises that the elements of the list are of some type a, which is an instance of Typeable, but not necessarily any such type. But even with this, you'll have two problems. First, even if you could construct such a list, what could you do with it? And second, it turns out that you can't even construct it in the first place.
Since all that you know about the elements of the existential list is that they're instances of Typeable, what can you do with them? Looking at the documentation, we see that there are only two functions2 which take instances of Typeable:
typeOf :: Typeable a => a -> TypeRep, from the type class itself (indeed, the only method therein); and
cast :: (Typeable a, Typeable b) => a -> Maybe b (which is implemented with unsafeCoerce, and couldn't be written otherwise).
Thus, all that you know about the type of the elements in the list is that you can call typeOf and cast on them. Since we'll never be able to usefully do anything else with them, our existential might just as well be (again, not valid Haskell)
randList :: [Dynamic] -> IO [(TypeRep, forall b. Typeable b => Maybe b)]
This is what we get if we apply typeOf and cast to every element of our list, store the results, and throw away the now-useless existentially typed original value. Clearly, the TypeRep part of this list isn't useful. And the second half of the list isn't either. Since we're back to a universally-quantified type, the caller of randList is once again entitled to request that they get a Maybe Int, a Maybe Bool, or a Maybe b for any (typeable) b of their choosing. (In fact, they have slightly more power than before, since they can instantiate different elements of the list to different types.) But they can't figure out what type they're converting from unless they already know it—you've still lost the type information you were trying to keep.
And even setting aside the fact that they're not useful, you simply can't construct the desired existential type here. The error arises when you try to return the existentially-typed list (return $ dynListToList dl). At what specific type are you calling dynListToList? Recall that dynListToList :: forall a. Typeable a => [Dynamic] -> [a]; thus, randList is responsible for picking which a dynListToList is going to use. But it doesn't know which a to pick; again, that's the source of the question! So the type that you're trying to return is underspecified, and thus ambiguous.3
Dependent types
OK, so what would make this existential useful (and possible)? Well, we actually have slightly more information: not only do we know there's some a, we have its TypeRep. So maybe we can package that up:
randList :: [Dynamic] -> (exists a. Typeable a => IO (TypeRep,[a]))
This isn't quite good enough, though; the TypeRep and the [a] aren't linked at all. And that's exactly what you're trying to express: some way to link the TypeRep and the a.
Basically, your goal is to write something like
toType :: TypeRep -> *
Here, * is the kind of all types; if you haven't seen kinds before, they are to types what types are to values. * classifies types, * -> * classifies one-argument type constructors, etc. (For instance, Int :: *, Maybe :: * -> *, Either :: * -> * -> *, and Maybe Int :: *.)
With this, you could write (once again, this code isn't valid Haskell; in fact, it really bears only a passing resemblance to Haskell, as there's no way you could write it or anything like it within Haskell's type system):
randList :: [Dynamic] -> (exists (tr :: TypeRep).
Typeable (toType tr) => IO (tr, [toType tr]))
randList dl = do
tr <- randItem $ map dynTypeRep dl
return (tr, dynListToList dl :: [toType tr])
-- In fact, in an ideal world, the `:: [toType tr]` signature would be
-- inferable.
Now, you're promising the right thing: not that there exists some type which classifies the elements of the list, but that there exists some TypeRep such that its corresponding type classifies the elements of the list. If only you could do this, you would be set. But writing toType :: TypeRep -> * is completely impossible in Haskell: doing this requires a dependently-typed language, since toType tr is a type which depends on a value.
What does this mean? In Haskell, it's perfectly acceptable for values to depend on other values; this is what a function is. The value head "abc", for instance, depends on the value "abc". Similarly, we have type constructors, so it's acceptable for types to depend on other types; consider Maybe Int, and how it depends on Int. We can even have values which depend on types! Consider id :: a -> a. This is really a family of functions: id_Int :: Int -> Int, id_Bool :: Bool -> Bool, etc. Which one we have depends on the type of a. (So really, id = \(a :: *) (x :: a) -> x; although we can't write this in Haskell, there are languages where we can.)
Crucially, however, we can never have a type that depends on a value. We might want such a thing: imagine Vec 7 Int, the type of length-7 lists of integers. Here, Vec :: Nat -> * -> *: a type whose first argument must be a value of type Nat. But we can't write this sort of thing in Haskell.4 Languages which support this are called dependently-typed (and will let us write id as we did above); examples include Coq and Agda. (Such languages often double as proof assistants, and are generally used for research work as opposed to writing actual code. Dependent types are hard, and making them useful for everyday programming is an active area of research.)
Thus, in Haskell, we can check everything about our types first, throw away all that information, and then compile something that refers only to values. In fact, this is exactly what GHC does; since we can never check types at run-time in Haskell, GHC erases all the types at compile-time without changing the program's run-time behavior. This is why unsafeCoerce is easy to implement (operationally) and completely unsafe: at run-time, it's a no-op, but it lies to the type system. Consequently, something like toType is completely impossible to implement in the Haskell type system.
In fact, as you noticed, you can't even write down the desired type and use unsafeCoerce. For some problems, you can get away with this; we can write down the type for the function, but only implement it with by cheating. That's exactly how fromDynamic works. But as we saw above, there's not even a good type to give to this problem from within Haskell. The imaginary toType function allows you to give the program a type, but you can't even write down toType's type!
What now?
So, you can't do this. What should you do? My guess is that your overall architecture isn't ideal for Haskell, although I haven't seen it; Typeable and Dynamic don't actually show up that much in Haskell programs. (Perhaps you're "speaking Haskell with a Python accent", as they say.) If you only have a finite set of data types to deal with, you might be able to bundle things into a plain old algebraic data type instead:
data MyType = MTInt Int | MTBool Bool | MTString String
Then you can write isMTInt, and just use filter isMTInt, or filter (isSameMTAs randomMT).
Although I don't know what it is, there's probably a way you could unsafeCoerce your way through this problem. But frankly, that's not a good idea unless you really, really, really, really, really, really know what you're doing. And even then, it's probably not. If you need unsafeCoerce, you'll know, it won't just be a convenience thing.
I really agree with Daniel Wagner's comment: you're probably going to want to rethink your approach from scratch. Again, though, since I haven't seen your architecture, I can't say what that will mean. Maybe there's another Stack Overflow question in there, if you can distill out a concrete difficulty.
1 That looks like the following:
{-# LANGUAGE ExistentialQuantification #-}
data TypeableList = forall a. Typeable a => TypeableList [a]
randList :: [Dynamic] -> IO TypeableList
However, since none of this code compiles anyway, I think writing it out with exists is clearer.
2 Technically, there are some other functions which look relevant, such as toDyn :: Typeable a => a -> Dynamic and fromDyn :: Typeable a => Dynamic -> a -> a. However, Dynamic is more or less an existential wrapper around Typeables, relying on typeOf and TypeReps to know when to unsafeCoerce (GHC uses some implementation-specific types and unsafeCoerce, but you could do it this way, with the possible exception of dynApply/dynApp), so toDyn doesn't do anything new. And fromDyn doesn't really expect its argument of type a; it's just a wrapper around cast. These functions, and the other similar ones, don't provide any extra power that isn't available with just typeOf and cast. (For instance, going back to a Dynamic isn't very useful for your problem!)
3 To see the error in action, you can try to compile the following complete Haskell program:
{-# LANGUAGE ExistentialQuantification #-}
import Data.Dynamic
import Data.Typeable
import Data.Maybe
randItem :: [a] -> IO a
randItem = return . head -- Good enough for a short and non-compiling example
dynListToList :: Typeable a => [Dynamic] -> [a]
dynListToList = mapMaybe fromDynamic
data TypeableList = forall a. Typeable a => TypeableList [a]
randList :: [Dynamic] -> IO TypeableList
randList dl = do
tr <- randItem $ map dynTypeRep dl
return . TypeableList $ dynListToList dl -- Error! Ambiguous type variable.
Sure enough, if you try to compile this, you get the error:
SO12273982.hs:17:27:
Ambiguous type variable `a0' in the constraint:
(Typeable a0) arising from a use of `dynListToList'
Probable fix: add a type signature that fixes these type variable(s)
In the second argument of `($)', namely `dynListToList dl'
In a stmt of a 'do' block: return . TypeableList $ dynListToList dl
In the expression:
do { tr <- randItem $ map dynTypeRep dl;
return . TypeableList $ dynListToList dl }
But as is the entire point of the question, you can't "add a type signature that fixes these type variable(s)", because you don't know what type you want.
4 Mostly. GHC 7.4 has support for lifting types to kinds and for kind polymorphism; see section 7.8, "Kind polymorphism and promotion", in the GHC 7.4 user manual. This doesn't make Haskell dependently typed—something like TypeRep -> * example is still out5—but you will be able to write Vec by using very expressive types that look like values.
5 Technically, you could now write down something which looks like it has the desired type: type family ToType :: TypeRep -> *. However, this takes a type of the promoted kind TypeRep, and not a value of the type TypeRep; and besides, you still wouldn't be able to implement it. (At least I don't think so, and I can't see how you would—but I am not an expert in this.) But at this point, we're pretty far afield.
What you're observing is that the type TypeRep doesn't actually carry any type-level information along with it; only term-level information. This is a shame, but we can do better when we know all the type constructors we care about. For example, suppose we only care about Ints, lists, and function types.
{-# LANGUAGE GADTs, TypeOperators #-}
import Control.Monad
data a :=: b where Refl :: a :=: a
data Dynamic where Dynamic :: TypeRep a -> a -> Dynamic
data TypeRep a where
Int :: TypeRep Int
List :: TypeRep a -> TypeRep [a]
Arrow :: TypeRep a -> TypeRep b -> TypeRep (a -> b)
class Typeable a where typeOf :: TypeRep a
instance Typeable Int where typeOf = Int
instance Typeable a => Typeable [a] where typeOf = List typeOf
instance (Typeable a, Typeable b) => Typeable (a -> b) where
typeOf = Arrow typeOf typeOf
congArrow :: from :=: from' -> to :=: to' -> (from -> to) :=: (from' -> to')
congArrow Refl Refl = Refl
congList :: a :=: b -> [a] :=: [b]
congList Refl = Refl
eq :: TypeRep a -> TypeRep b -> Maybe (a :=: b)
eq Int Int = Just Refl
eq (Arrow from to) (Arrow from' to') = liftM2 congArrow (eq from from') (eq to to')
eq (List t) (List t') = liftM congList (eq t t')
eq _ _ = Nothing
eqTypeable :: (Typeable a, Typeable b) => Maybe (a :=: b)
eqTypeable = eq typeOf typeOf
toDynamic :: Typeable a => a -> Dynamic
toDynamic a = Dynamic typeOf a
-- look ma, no unsafeCoerce!
fromDynamic_ :: TypeRep a -> Dynamic -> Maybe a
fromDynamic_ rep (Dynamic rep' a) = case eq rep rep' of
Just Refl -> Just a
Nothing -> Nothing
fromDynamic :: Typeable a => Dynamic -> Maybe a
fromDynamic = fromDynamic_ typeOf
All of the above is pretty standard. For more on the design strategy, you'll want to read about GADTs and singleton types. Now, the function you want to write follows; the type is going to look a bit daft, but bear with me.
-- extract only the elements of the list whose type match the head
firstOnly :: [Dynamic] -> Dynamic
firstOnly [] = Dynamic (List Int) []
firstOnly (Dynamic rep v:xs) = Dynamic (List rep) (v:go xs) where
go [] = []
go (Dynamic rep' v:xs) = case eq rep rep' of
Just Refl -> v : go xs
Nothing -> go xs
Here we've picked a random element (I rolled a die, and it came up 1) and extracted only the elements that have a matching type from the list of dynamic values. Now, we could have done the same thing with regular boring old Dynamic from the standard libraries; however, what we couldn't have done is used the TypeRep in a meaningful way. I now demonstrate that we can do so: we'll pattern match on the TypeRep, and then use the enclosed value at the specific type the TypeRep tells us it is.
use :: Dynamic -> [Int]
use (Dynamic (List (Arrow Int Int)) fs) = zipWith ($) fs [1..]
use (Dynamic (List Int) vs) = vs
use (Dynamic Int v) = [v]
use (Dynamic (Arrow (List Int) (List (List Int))) f) = concat (f [0..5])
use _ = []
Note that on the right-hand sides of these equations, we are using the wrapped value at different, concrete types; the pattern match on the TypeRep is actually introducing type-level information.
You want a function that chooses a different type of values to return based on runtime data. Okay, great. But the whole purpose of a type is to tell you what operations can be performed on a value. When you don't know what type will be returned from a function, what do you do with the values it returns? What operations can you perform on them? There are two options:
You want to read the type, and perform some behaviour based on which type it is. In this case you can only cater for a finite list of types known in advance, essentially by testing "is it this type? then we do this operation...". This is easily possible in the current Dynamic framework: just return the Dynamic objects, using dynTypeRep to filter them, and leave the application of fromDynamic to whoever wants to consume your result. Moreover, it could well be possible without Dynamic, if you don't mind setting the finite list of types in your producer code, rather than your consumer code: just use an ADT with a constructor for each type, data Thing = Thing1 Int | Thing2 String | Thing3 (Thing,Thing). This latter option is by far the best if it is possible.
You want to perform some operation that works across a family of types, potentially some of which you don't know about yet, e.g. by using type class operations. This is trickier, and it's tricky conceptually too, because your program is not allowed to change behaviour based on whether or not some type class instance exists – it's an important property of the type class system that the introduction of a new instance can either make a program type check or stop it from type checking, but it can't change the behaviour of a program. Hence you can't throw an error if your input list contains inappropriate types, so I'm really not sure that there's anything you can do that doesn't essentially involve falling back to the first solution at some point.