I am trying to use an exception to skip parts of the code here. Instead of getting caught by catcheE and resuming normal behavior all following actions in the mapM_ chain get skipped.
I looked at this question and it appears that catchE ~ main and checkMaybe ~ intercept.
I also checked the implementation of mapM_to be sure it does what i want it to, but i don't understand how the Left value can escape dlAsset to affect the behavior of mapM_.
I refactored this from a version where i simply used an empty string as an exception marker for the failed lookup. In that version checkMaybe just returned a Right value immediately and it worked (matching on "" to 'catch')
import Data.HashMap.Strict as HM hiding (map)
import qualified Data.ByteString.Lazy as BS
import qualified Data.ByteString.Char8 as BSC8
import qualified JSONParser as P -- my module
retrieveAssets :: (Text -> Text) -> ExceptT Text IO ()
retrieveAssets withName = withManager $ (lift ((HM.keys . P.assets)
<$> P.raw) ) >>= mapM_ f
where
f = \x -> dlAsset x "0.1246" (withName x)
dlAsset :: Text -> Text -> Text -> ReaderT Manager (ExceptT Text IO) ()
dlAsset name size dest = do
req <- lift $ (P.assetLookup name size <$> P.raw) >>= checkMaybe
name >>= parseUrl . unpack -- lookup of a url
res <- httpLbs req
lift $ (liftIO $ BS.writeFile (unpack dest) $ responseBody res)
`catchE` (\_ -> return ()) -- always a Right value?
where
checkMaybe name a = case a of
Nothing -> ExceptT $ fmap Left $ do
BSC8.appendFile "./resources/images/missingFiles.txt" $
BSC8.pack $ (unpack name) ++ "\n"
putStrLn $ "lookup of " ++ (unpack name) ++ " failed"
return name
Just x -> lift $ pure x
(had to reformat to become somewhat readable here)
edit: i'd like to understand what actually happens here, that would probably help me more than knowing which part of the code is wrong.
The problem is that your call to catchE only covered the very last line of dlAsset. It needs to be moved to the left of the do-notation indentation level to cover all of the do notation.
Related
I am trying to get a good grip on the do notation in Haskell.
I could use it with Maybe and then print the result. Like this:
maybeAdd :: Maybe Integer
maybeAdd = do one <- maybe1
two <- maybe2
three <- maybe3
return (one + two + three)
main :: IO ()
main = putStr (show $ fromMaybe 0 maybeAdd)
But instead of having a separate function I am trying to use the do notation with the Maybe inside the main function. But I am not having any luck. The various attempts I tried include:
main :: IO ()
main = do one <- maybe1
two <- maybe2
three <- maybe3
putStr (show $ fromMaybe 0 $ return (one + two + three))
main :: IO ()
main = do one <- maybe1
two <- maybe2
three <- maybe3
putStr (show $ fromMaybe 0 $ Just (one + two + three))
main :: IO ()
main = do one <- maybe1
two <- maybe2
three <- maybe3
putStr (show $ (one + two + three))
All of these leads to various types of compilation errors, which unfortunately I failed to decipher to get the correct way to do it.
How do I achieve the above? And perhaps maybe an explanation of why the approaches I tried were wrong also?
Each do block must work within a single monad. If you want to use multiple monads, you could use multiple do blocks. Trying to adapt your code:
main :: IO ()
main = do -- IO block
let x = do -- Maybe block
one <- maybe1
two <- maybe2
three <- maybe3
return (one + two + three)
putStr (show $ fromMaybe 0 x)
You could even use
main = do -- IO block
putStr $ show $ fromMaybe 0 $ do -- Maybe block
one <- maybe1
two <- maybe2
three <- maybe3
return (one + two + three)
-- other IO actions here
but it could be less readable in certain cases.
The MaybeT monad transformer would come handy in this particular case. MaybeT monad transformer is just a type defined something like;
newtype MaybeT m a = MaybeT {runMaybeT :: m (Maybe a)}
Actually transformers like MaybeT, StateT etc, are readily available in Control.Monad.Trans.Maybe, Control.Monad.Trans.State... For illustration purposes it' Monad instance could be something like shown below;
instance Monad m => Monad (MaybeT m) where
return = MaybeT . return . Just
x >>= f = MaybeT $ runMaybeT x >>= g
where
g Nothing = return Nothing
g (Just x) = runMaybeT $ f x
so as you will notice the monadic f function takes a value that resides in the Maybe monad which itself is in another monad (IO in our case). The f function does it's thing and wraps the result back into MaybeT m a.
Also there is a MonadTrans class where you can have some common functionalities those are used by the transformer types. One such is lift which is used to lift the value into a transformer according to that particular instance's definition. For MaybeT it should look like
instance MonadTrans MaybeT where
lift = MaybeT . (liftM Just)
Lets perform your task with monad transformers.
addInts :: MaybeT IO ()
addInts = do
lift $ putStrLn "Enter two integers.."
i <- lift getLine
guard $ test i
j <- lift getLine
guard $ test j
lift . print $ (read i :: Int) + (read j :: Int)
where
test = and . (map isDigit)
So when called like
λ> runMaybeT addInts
Enter two integers..
1453
1571
3024
Just ()
The catch is, since a monad transformer is also a member of Monad typeclass, one can nest them indefinitelly and still do things under a singe do notation.
Edit: answer gets downvoted but it is unclear to me why. If there is something wrong with the approach please care to elaborate me so that it helps people including me to learn something better.
Taking the opportunity of being on the edit session, i would like to add a better code since i think Char based testing might not be the best idea as it will not take negative Ints into account. So let's try using readMaybe from the Text.Read package while we are doing things with the Maybe type.
import Control.Monad.Trans.Maybe
import Control.Monad.Trans.Class (lift)
import Text.Read (readMaybe)
addInts :: MaybeT IO ()
addInts = do
lift $ putStrLn "Enter two integers.."
i <- lift getLine
MaybeT $ return (readMaybe i :: Maybe Int)
j <- lift getLine
MaybeT $ return (readMaybe j :: Maybe Int)
lift . print $ (read i :: Int) + (read j :: Int)
I guess now it works better...
λ> runMaybeT addInts
Enter two integers..
-400
500
100
Just ()
λ> runMaybeT addInts
Enter two integers..
Not an Integer
Nothing
I'm using pipes, attoparsec, and pipes-attoparsec to write a database dump file converter. The general format of the file is to have a create table command followed by an optional insert command. In addition to transforming the statements in place, the table definitions have to be held in memory until the very end for additional processing (indexes, constraints, etc.).
This works fine, but now I need to allow some of my internal parsers to have access to my Producer's State in order to determine which parser needs to be run while processing the values from the insert command.
I tried something like this:
-- IO
import qualified Data.ByteString.Char8 as BS (putStrLn)
import System.Exit (ExitCode (..), exitSuccess, exitFailure)
import System.IO (hPutStrLn, stderr)
-- Pipes
import Pipes (runEffect, for, liftIO, Producer, Effect)
import Pipes.Attoparsec (parsed, ParsingError)
import Pipes.Lift (runStateP)
import Pipes.Safe (runSafeT)
import qualified Pipes.ByteString as PBS (stdin)
-- State
import Control.Monad.Trans.Class (lift)
import Control.Monad.Trans.State.Strict
dump' :: StateT ParserState Parser Command
dump' = fmap Create createStatements' <|> fmap Insert justData'
doStuff :: MonadIO m => Effect m (Either (ParsingError, Producer ByteString (StateT ParserState m) ()) (), ParserState)
doStuff = runStateP defaultParserState theStuff
theStuff :: MonadIO m => Effect (StateT ParserState m) (Either (ParsingError, Producer ByteString (StateT ParserState m) ()) ())
theStuff = for runParser (liftIO . BS.putStrLn <=< lift . processCommand)
runParser :: MonadIO m => Producer Command (StateT ParserState m) (Either (ParsingError, Producer ByteString (StateT ParserState m) ()) ())
runParser = do
s <- lift get
liftIO $ putStrLn "runParser"
liftIO $ putStrLn $ show s
parsed (evalStateT dump' s) PBS.stdin
processCommand :: MonadIO m => Command -> StateT ParserState m ByteString
processCommand (Create xs) = do
currentState <- get
liftIO $ putStrLn "processCommand"
liftIO $ putStrLn $ show currentState
_ <- put (currentState { constructs = xs ++ (constructs currentState)})
return $ P.firstPass $ P.transformConstructs xs
processCommand (Insert x) = return x
Complete source (including parsers): https://github.com/cimmanon/mysqlnothx/blob/parser-state/src/Main.hs
When I run it, I get a result that looks something like this:
runParser
ParserState {constructs = []}
processCommand
ParserState {constructs = []}
processCommand
ParserState {constructs = [ ... ]}
processCommand
ParserState {constructs = [ ..... ]}
I was expecting runParser (which would grab the latest contents from State) to be run every time processCommand runs, but that's clearly not the case based on the output. When I check the contents of State within the parser, it's always empty no matter how many commands are parsed.
How can I extend State from my Producers to my Parser (dump') so that they share the same State? If my Producer has 4 values in State, the Parser should also see those same 4 values.
I was expecting runParser (which would grab the latest contents from State) to be run every time processCommand runs, but that's clearly not the case.
Your main effect is for runParser (liftIO . BS.putStrLn <=< lift . processCommand). To understand what this effect does you need to understand what for does:
(for p body) loops over p replacing each yield with body
"Loops over p" is accurate if a bit confusing. It doesn't run p once for each value produced by p; that would explode! Instead for replaces every yield in p with body. By replacing yield with body it runs body once for every yielded value. Running the body once for each produced value is similar to how in other languages a for-loop over a list runs the body once for each value in the list.
Your runParser is
runParser = do
s <- lift get
liftIO $ putStrLn "runParser"
liftIO $ putStrLn $ show s
parsed (evalStateT dump' s) PBS.stdin
It reads the state, outputs it, and produces the Commands parsed from stdin. Pipes-autoparsec's parsed parses the source and yields once for each completely successfully parsed value. Your for then replaces each of parsed's yields with liftIO . BS.putStrLn <=< lift . processCommand. The complete effect runs runParser once and processCommand once for each yield, which is what you're observing in the output.
I have revisited Haskell lateley and constructed a toy programming language parser/interpreter. Using Parsec for lexing and parsing and a separate interpreter. I'm running in to some issues with feeding the result from the parser to my interpreter and handle the potential error from both the interpreter and parser. I end up with something like this:
main = do
fname <- getArgs
input <- readFile (head fname)
case lparse (head fname) input of
Left msg -> putStrLn $ show msg
Right p -> case intrp p of
Left msg -> putStrLn $ show msg
Right r -> putStrLn $ show r
This dosn't look pretty at all. My problem is that lparse returns Either ParseError [(String, Stmt)] and itrp returns the type Either ItrpError Stmt so I'm having a real hard time feeding the Right result from the parser to the interpreter and at the same time bail and print the possible ParseError or IntrpError.
The closest to what i want is something like this
main = do
fname <- getArgs
input <- readFile (head fname)
let prog = lparse (head fname) input
(putStrLn . show) (intrp <$> prog)
But this will not surprisingly yield a nested Either and not print pretty either.
So are there any nice Haskell ideomatic way of doing this threading results from one computation to another and handling errors (Lefts) in a nice way without nesting cases?
Edit
adding types of lparse and itrp
lparse :: Text.Parsec.Pos.SourceName -> String -> Either Text.Parsec.Error.ParseError [([Char], Stmt)]
intrp :: [([Char], Stmt)] -> Either IntrpError Stmt
While not perfect, I'd create a helper function for embedding any Showable error from Either into MonadError:
{-# LANGUAGE FlexibleContexts #-}
import Control.Monad.Except
strErr :: (MonadError String m, Show e) => Either e a -> m a
strErr = either (throwError . show) return
Then if you have a computation that can fail with errors, like
someFn :: ExceptT String IO ()
someFn = strErr (Left 42)
you can run it (printing errors to stdout) as
main :: IO ()
main = runExceptT someFn >>= either putStrLn return
In your case it'd be something like
main = either putStrLn return <=< runExceptT $ do
fname <- liftIO getArgs
input <- liftIO $ readFile (head fname)
prog <- strErr $ lparse (head fname) input
r <- strErr $ interp prog
print r
Well, if you want to chain successful computations, you can always use >>= to do that. For instance in your case:
lparse (head fname) input >>= intrp
And if you want to print out either your error message you can use the either class that takes two handler functions, one for the case when you have Left a (error in your case) and another for Right b (in your case a successful thing). An example:
either (putStrLn . show) (putStrLn . show) (lparse (head fname) input >>= intrp)
And if anything fails in your chain (any step of your monadic chain becomes Left a) it stops and can for instance print out the error message in the above case.
I am writing simple sitemap.xml crawler. The code is below. My question is why the code in the end of main does not print anything. I suspect it's because haskell's lazyness but don't know how to deal with it here:
import Network.HTTP.Conduit
import qualified Data.ByteString.Lazy as L
import Text.XML.Light
import Control.Monad.Trans (liftIO)
import Control.Monad
import Data.String.Utils
import Control.Exception
download :: Manager -> Request -> IO (Either HttpException L.ByteString)
download manager req = do
try $
fmap responseBody (httpLbs req manager)
downloadUrl :: Manager -> String -> IO (Either HttpException L.ByteString)
downloadUrl manager url = do
request <- parseUrl url
download manager request
getPages :: Manager -> [String] -> IO [Either HttpException L.ByteString]
getPages manager urls =
sequence $ map (downloadUrl manager) urls
main = withManager $ \ manager -> do
-- I know simpleHttp is bad here
mapSource <- liftIO $ simpleHttp "http://example.com/sitemap.xml"
let elements = (parseXMLDoc mapSource) >>= Just . findElements (mapElement "loc")
Just urls = liftM (map $ (replace "/#!" "?_escaped_fragment_=") . strContent) elements
mapElement name = QName name (Just "http://www.sitemaps.org/schemas/sitemap/0.9") Nothing
return $
getPages manager urls >>= \ pages -> do
print "evaluate me!"
sequence $ map print pages
You're running into the same problem I describe here, at least as far as having incorrect code that typechecks when it should actually give a type error: Why is the type of "Main.main", "IO ()" and not "IO a"?. This is why you should always give main the type signature main :: IO () explicitly.
To fix the problem, you will want to replace return with lift (see http://hackage.haskell.org/package/transformers/docs/Control-Monad-Trans-Class.html#v:lift) and replace sequence $ map ... with mapM_. mapM_ f is equivalent to sequence_ . map f.
Substitute your last return with runResourceT (http://hackage.haskell.org/package/resourcet-1.1.1/docs/Control-Monad-Trans-Resource.html#v:runResourceT). As it's type suggests, it would turn ResourceT into IO action.
Yesterday i tried to write a simple rss downloader in Haskell wtih hte help of the Network.HTTP and Feed libraries. I want to download the link from the rss item and name the downloaded file after the title of the item.
Here is my short code:
import Control.Monad
import Control.Applicative
import Network.HTTP
import Text.Feed.Import
import Text.Feed.Query
import Text.Feed.Types
import Data.Maybe
import qualified Data.ByteString as B
import Network.URI (parseURI, uriToString)
getTitleAndUrl :: Item -> (Maybe String, Maybe String)
getTitleAndUrl item = (getItemTitle item, getItemLink item)
downloadUri :: (String,String) -> IO ()
downloadUri (title,link) = do
file <- get link
B.writeFile title file
where
get url = let uri = case parseURI url of
Nothing -> error $ "invalid uri" ++ url
Just u -> u in
simpleHTTP (defaultGETRequest_ uri) >>= getResponseBody
getTuples :: IO (Maybe [(Maybe String, Maybe String)])
getTuples = fmap (map getTitleAndUrl) <$> fmap (feedItems) <$> parseFeedString <$> (simpleHTTP (getRequest "http://index.hu/24ora/rss/") >>= getResponseBody)
I reached a state where i got a list which contains tuples, which contains name and the corresponding link. And i have a downloadUri function which properly downloads the given link to a file which has the name of the rss item title.
I already tried to modify downloadUri to work on (Maybe String,Maybe String) with fmap- ing on get and writeFile but failed with it horribly.
How can i apply my downloadUri function to the result of the getTuples function. I want to implement the following main function
main :: IO ()
main = some magic incantation donwloadUri more incantation getTuples
The character encoding of the result of getItemTitle broken, it puts code points in the places of the accented characters. The feed is utf8 encoded, and i thought that all haskell string manipulation functions are defaulted to utf8. How can i fix this?
Edit:
Thanks for you help, i implemented successfully my main and helper functions. Here comes the code:
downloadUri :: (Maybe String,Maybe String) -> IO ()
downloadUri (Just title,Just link) = do
item <- get link
B.writeFile title item
where
get url = let uri = case parseURI url of
Nothing -> error $ "invalid uri" ++ url
Just u -> u in
simpleHTTP (defaultGETRequest_ uri) >>= getResponseBody
downloadUri _ = print "Somewhere something went Nothing"
getTuples :: IO (Maybe [(Maybe String, Maybe String)])
getTuples = fmap (map getTitleAndUrl) <$> fmap (feedItems) <$> parseFeedString <$> decodeString <$> (simpleHTTP (getRequest "http://index.hu/24ora/rss/") >>= getResponseBody)
downloadAllItems :: Maybe [(Maybe String, Maybe String)] -> IO ()
downloadAllItems (Just feedlist) = mapM_ downloadUri $ feedlist
downloadAllItems _ = error "feed does not get parsed"
main = getTuples >>= downloadAllItems
The character encoding issue has been partially solved, i put decodeString before the feed parsing, so the files get named properly. But if i want to print it out, the issue still happens. Minimal working example:
main = getTuples
It sounds like it's the Maybes that are giving you trouble. There are many ways to deal with Maybe values, and some useful library functions like fromMaybe and fromJust. However, the simplest way is to do pattern matching on the Maybe value. We can tweak your downloadUri function to work with the Maybe values. Here's an example:
downloadUri :: (Maybe String, Maybe String) -> IO ()
downloadUri (Just title, Just link) = do
file <- get link
B.writeFile title file
where
get url = let uri = case parseURI url of
Nothing -> error $ "invalid uri" ++ url
Just u -> u in
simpleHTTP (defaultGETRequest_ uri) >>= getResponseBody
downloadUri _ = error "One of my parameters was Nothing".
Or maybe you can let the title default to blank, in which case you could insert this just before the last line in the previous example:
downloadUri (Nothing, Just link) = downloadUri (Just "", Just link)
Now the only Maybe you need to work with is the outer one, applied to the array of tuples. Again, we can pattern match. It might be clearest to write a helper function like this:
downloadAllItems (Just ts) = ??? -- hint: try a `mapM`
downloadAllItems Nothing = ??? -- don't do anything, or report an error, or...
As for your encoding issue, my guesses are:
You're reading the information from a file that isn't UTF-8 encoded, or your system doesn't realise that it's UTF-8 encoded.
You are reading the information correctly, but it gets messed up when you output it.
In order to help you with this problem, I need to see a full code example, which shows how you're reading the information and how you output it.
Your main could be something like the shown below. There may be some more concise way to compose these two operations though:
main :: IO ()
main = getTuples >>= process
where
process (Just lst) = foldl (\s v -> do {t <- s; download v}) (return ()) lst
process Nothing = return ()
download (Just t, Just l) = downloadUri (t,l)
download _ = return ()