I'm working on this project to get feet wet with Haskell and struggling with finding simple examples.
In this instance, I would like to have a web request handler for Snap, which returns a list of files in a directory.
I believe I'm trying to get the return of getDirectoryContents into a Bytestring which Snap wants.
I am most confused about what to do with the return value I get at the line filenames <- getDirectoryContents "data" below:
import Control.Applicative
import Snap.Core
import Snap.Util.FileServe
import Snap.Http.Server
import System.Directory (getDirectoryContents)
main :: IO ()
main = quickHttpServe site
site :: Snap ()
site =
ifTop (writeBS "hello world") <|>
route [ ("foo", writeBS "bar")
, ("echo/:echoparam", echoHandler)
, ("view_root_json_files", listRootFilesHandler)
] <|>
dir "static" (serveDirectory ".")
echoHandler :: Snap ()
echoHandler = do
param <- getParam "echoparam"
maybe (writeBS "must specify echo/param in URL")
writeBS param
listRootFilesHandler :: Snap ()
listRootFilesHandler = do
-- read all filenames in /data folders
filenames <- getDirectoryContents "data"
writeText filenames
Since you want to use writeText, you need to convert [FilePath] to Text. Luckily, Text is an instance of Monoid, and a list is a instance of Foldable, so we can simply use foldMap pack filenames to get a single text:
-- import Data.Foldable (foldMap)
-- import Data.Text (pack, Text)
toText :: [FilePath] -> Text
toText = foldMap pack
Note that you'll need to use liftIO to actually use a IO a in Snap b, since Snap is an instance of MonadIO:
listRootFilesHandler :: Snap ()
listRootFilesHandler = do
filenames <- liftIO $ getDirectoryContents "data"
writeText $ toText filenames
If you want to add a newline (or <br/>) after each FilePath, add flip snoc '\n':
toText = foldMap (flip snoc '\n' . pack)
-- toText = foldMap (flip append (pack "<br/>") . pack)
You can't "convert" an IO action to a string. Of course not, it's a completely different thing, conceptually. What you can do is extract / "focus" a value in a monad, such as IO. That's what the val <- action syntax in a do block is used for, you have that quite correct.
The only problem with your current implementation is that you have an IO-monad action, but want to execute it in the Snap monad. Well, actually Snap is deep down the IO monad, with a whole lot of extra stuff attached via monad transformers. But you can always use Snap as if it were IO. That's true for a whole bunch of monads (all that are created by stacking transformers on IO), so there's a dedicated class for "monads that can do IO".
listRootFilesHandler = do
-- read all filenames in /data folders
filenames <- liftIO $ getDirectoryContents "data"
...
The other, rather easier thing is flattening a [FilePath] list to a single string. I suppose you know how to do that.
Related
Hello community thank you for your time.
I have an error and I am not sure what the error is, but what I think the problem is:
There is no IO transformer from ext-1.2.4.1:Data.Text.Internal.Lazy.Text IO) to Web.Scotty.Internal.Types.ScottyT.
But I wondering why the compiler works with ext-1.2.4.1:Data.Text.Internal.Lazy.Text IO). That's why I am working just with String and I removed all occurrences of {-# LANGUAGE OverloadedStrings #-} but still get the error. On the other hand, this should be IO [String], shouldn't it?
And as you can mention I don't really know what ext-1.2.4.1:Data.Text.Internal.Lazy.Text IO) is.
At another place, I already use liftIO successfully for an a -> IO String function. And I think I use them the same way.
I think I get slowly a feeling for what a monad is, but not quite sure. I don't really know why I have to use a lift function at all.
Error message:
• No instance for (MonadIO
(Web.Scotty.Internal.Types.ScottyT
text-1.2.4.1:Data.Text.Internal.Lazy.Text IO))
arising from a use of ‘liftIO’
• In a stmt of a 'do' block:
paths <- liftIO $ getAllFilePaths2 path
In the expression:
do paths <- liftIO $ getAllFilePaths2 path
pathsToScotty paths
In an equation for ‘pathsToScotty2’:
pathsToScotty2 path
= do paths <- liftIO $ getAllFilePaths2 path
pathsToScotty paths
|
49 | paths <- liftIO $ getAllFilePaths2 path
Where the error occurred:
import Control.Monad.IO.Class
...
pathsToScotty2 :: String -> ScottyM ()
pathsToScotty2 path = do
paths <- liftIO $ getAllFilePaths2 path
pathsToScotty paths
getAllFilePaths2 :: String -> IO [String]
getAllFilePaths2 dir = do
putStrLn dir
isFile <- doesFileExist dir
if isFile
then return [dir]
else do
dirs <- listDirectory dir
foldl foldHelper2 (return []) $ map (\d -> show $ mconcat [dir, "/",d ]) dirs
foldHelper2 :: IO [String] -> String -> IO [String]
foldHelper2 ps path = do
paths <- ps
newPaths <- getAllFilePaths2 path
return (paths ++ newPaths)
Truly understanding monads takes time, practice, and patience, but it shouldn't be too hard to understand the need for liftIO by examining your types.
First off, the type of liftIO is MonadIO m => IO a -> m a. This means that the function can convert any IO action into an action in the monad m so long as m has an instance of MonadIO. In theory, this can only be implemented if m has some way of processing IO actions, so this function is embedding the given action into the m monad.
You're definitely in the right sort of place to use liftIO, so why isn't it working? That is, you have a value getAllFilePaths2 path of type IO [String], and you'd like it to be a value of type ScottyM [String] — this indeed seems like a good place to use liftIO. However, ScottyM is not an instance of MonadIO, as that error message you saw is trying to tell you, so you can't use liftIO.
This may seem crazy—can you really not embed IO actions into ScottyM?—but there's actually a good reason for this. What happens if the IO action throws an error? Does your whole web app crash? It would if you naively used liftIO. Instead, scotty provides the function liftAndCatchIO, which, as the docs describe, is "Like liftIO, but catch any IO exceptions and turn them into Scotty exceptions." This is the preferred way to embed IO actions into Scotty.
And here comes the final gotcha: Note that liftAndCatchIO actually produces values of type ActionM a, not ScottyM a. Additionally, there's no way to take a value in the ActionM monad and get it into the ScottyM monad. Instead, you need to use that value as an action. So, I'm not sure what pathsToScotty does, but it's very likely that you'll need to rewrite it.
I'm using pipes, attoparsec, and pipes-attoparsec to write a database dump file converter. The general format of the file is to have a create table command followed by an optional insert command. In addition to transforming the statements in place, the table definitions have to be held in memory until the very end for additional processing (indexes, constraints, etc.).
This works fine, but now I need to allow some of my internal parsers to have access to my Producer's State in order to determine which parser needs to be run while processing the values from the insert command.
I tried something like this:
-- IO
import qualified Data.ByteString.Char8 as BS (putStrLn)
import System.Exit (ExitCode (..), exitSuccess, exitFailure)
import System.IO (hPutStrLn, stderr)
-- Pipes
import Pipes (runEffect, for, liftIO, Producer, Effect)
import Pipes.Attoparsec (parsed, ParsingError)
import Pipes.Lift (runStateP)
import Pipes.Safe (runSafeT)
import qualified Pipes.ByteString as PBS (stdin)
-- State
import Control.Monad.Trans.Class (lift)
import Control.Monad.Trans.State.Strict
dump' :: StateT ParserState Parser Command
dump' = fmap Create createStatements' <|> fmap Insert justData'
doStuff :: MonadIO m => Effect m (Either (ParsingError, Producer ByteString (StateT ParserState m) ()) (), ParserState)
doStuff = runStateP defaultParserState theStuff
theStuff :: MonadIO m => Effect (StateT ParserState m) (Either (ParsingError, Producer ByteString (StateT ParserState m) ()) ())
theStuff = for runParser (liftIO . BS.putStrLn <=< lift . processCommand)
runParser :: MonadIO m => Producer Command (StateT ParserState m) (Either (ParsingError, Producer ByteString (StateT ParserState m) ()) ())
runParser = do
s <- lift get
liftIO $ putStrLn "runParser"
liftIO $ putStrLn $ show s
parsed (evalStateT dump' s) PBS.stdin
processCommand :: MonadIO m => Command -> StateT ParserState m ByteString
processCommand (Create xs) = do
currentState <- get
liftIO $ putStrLn "processCommand"
liftIO $ putStrLn $ show currentState
_ <- put (currentState { constructs = xs ++ (constructs currentState)})
return $ P.firstPass $ P.transformConstructs xs
processCommand (Insert x) = return x
Complete source (including parsers): https://github.com/cimmanon/mysqlnothx/blob/parser-state/src/Main.hs
When I run it, I get a result that looks something like this:
runParser
ParserState {constructs = []}
processCommand
ParserState {constructs = []}
processCommand
ParserState {constructs = [ ... ]}
processCommand
ParserState {constructs = [ ..... ]}
I was expecting runParser (which would grab the latest contents from State) to be run every time processCommand runs, but that's clearly not the case based on the output. When I check the contents of State within the parser, it's always empty no matter how many commands are parsed.
How can I extend State from my Producers to my Parser (dump') so that they share the same State? If my Producer has 4 values in State, the Parser should also see those same 4 values.
I was expecting runParser (which would grab the latest contents from State) to be run every time processCommand runs, but that's clearly not the case.
Your main effect is for runParser (liftIO . BS.putStrLn <=< lift . processCommand). To understand what this effect does you need to understand what for does:
(for p body) loops over p replacing each yield with body
"Loops over p" is accurate if a bit confusing. It doesn't run p once for each value produced by p; that would explode! Instead for replaces every yield in p with body. By replacing yield with body it runs body once for every yielded value. Running the body once for each produced value is similar to how in other languages a for-loop over a list runs the body once for each value in the list.
Your runParser is
runParser = do
s <- lift get
liftIO $ putStrLn "runParser"
liftIO $ putStrLn $ show s
parsed (evalStateT dump' s) PBS.stdin
It reads the state, outputs it, and produces the Commands parsed from stdin. Pipes-autoparsec's parsed parses the source and yields once for each completely successfully parsed value. Your for then replaces each of parsed's yields with liftIO . BS.putStrLn <=< lift . processCommand. The complete effect runs runParser once and processCommand once for each yield, which is what you're observing in the output.
This question already has answers here:
How to flatten IO [[String]]?
(2 answers)
Closed 7 years ago.
I would like to look in my current directory and only print .zip files.
My strategy (shown below) is to get the FilePaths as an IO [FilePath]. I thought I could lift the IO so that it would be possible to filter on string elements.
What is wrong in my thinking? I wonder if it is a problem that I use liftIO on an IO [FilePath] instead of IO FilePath.
import System.Directory
import System.FilePath.Glob
import Control.Monad.IO.Class
main :: IO()
listCompressedImages folder =
filter (match (compile ".zip")) (liftIO (getDirectoryContents folder))
main = listCompressedImages "." >>= print
You don't want to use liftIO here, that's for lifting an IO action to a more complex monad, not for extracting a value from an IO action. In short, you can't turn IO a into a. The whole point of IO is to prevent you from doing this. You can work with the a value directly using do notation, though:
listCompressedImages :: FilePath -> IO [FilePath]
listCompressedImages folder = do
-- getDirectoryContents :: FilePath -> IO [FilePath]
-- contents :: [FilePath]
contents <- getDirectoryContents folder
-- filter (match (compile ".zip")) :: [FilePath] -> [FilePath]
return $ filter (match (compile ".zip")) contents
main :: IO ()
main = do
-- compressedImages :: [FilePath]
compresssedImages <- listCompressedImages "."
print compressedImages
Whenever you have something with the type IO a and you want to get the value of type a from it, use do notation and extract it using <-. For a more in-depth explanation I'll defer to Learn You a Haskell.
You can not extract anything from IO, but you can adapt the other functions to work on IO values
listCompressedImages folder =
filter (match (compile ".zip")) `fmap` getDirectoryContents folder
The above fmap applies a pure function (as filter ...) to some IO value. Note that the resulting type will still be IO -- again, you can never escape the IO monad.
I am writing simple sitemap.xml crawler. The code is below. My question is why the code in the end of main does not print anything. I suspect it's because haskell's lazyness but don't know how to deal with it here:
import Network.HTTP.Conduit
import qualified Data.ByteString.Lazy as L
import Text.XML.Light
import Control.Monad.Trans (liftIO)
import Control.Monad
import Data.String.Utils
import Control.Exception
download :: Manager -> Request -> IO (Either HttpException L.ByteString)
download manager req = do
try $
fmap responseBody (httpLbs req manager)
downloadUrl :: Manager -> String -> IO (Either HttpException L.ByteString)
downloadUrl manager url = do
request <- parseUrl url
download manager request
getPages :: Manager -> [String] -> IO [Either HttpException L.ByteString]
getPages manager urls =
sequence $ map (downloadUrl manager) urls
main = withManager $ \ manager -> do
-- I know simpleHttp is bad here
mapSource <- liftIO $ simpleHttp "http://example.com/sitemap.xml"
let elements = (parseXMLDoc mapSource) >>= Just . findElements (mapElement "loc")
Just urls = liftM (map $ (replace "/#!" "?_escaped_fragment_=") . strContent) elements
mapElement name = QName name (Just "http://www.sitemaps.org/schemas/sitemap/0.9") Nothing
return $
getPages manager urls >>= \ pages -> do
print "evaluate me!"
sequence $ map print pages
You're running into the same problem I describe here, at least as far as having incorrect code that typechecks when it should actually give a type error: Why is the type of "Main.main", "IO ()" and not "IO a"?. This is why you should always give main the type signature main :: IO () explicitly.
To fix the problem, you will want to replace return with lift (see http://hackage.haskell.org/package/transformers/docs/Control-Monad-Trans-Class.html#v:lift) and replace sequence $ map ... with mapM_. mapM_ f is equivalent to sequence_ . map f.
Substitute your last return with runResourceT (http://hackage.haskell.org/package/resourcet-1.1.1/docs/Control-Monad-Trans-Resource.html#v:runResourceT). As it's type suggests, it would turn ResourceT into IO action.
I want to create a Happstack application with lots of access to a database. I think that a Monad Stack with IO at the bottom and a Database Write-like monad on top (with log writer in the middle) will work to have a clear functions in each access, example:
itemsRequest :: ServerConfig -> ServerPart Response
itemsRequest cf = dir "items" $ do
methodM [GET,HEAD]
liftIO $ noticeM (scLogger cf) "sended job list"
items <- runDBMonad (scDBConnString cf) $ getItemLists
case items of
(Right xs) -> ok $ toResponse $ show xs
(Left err) -> internalServerError $ toResponse $ show err
With:
getItemList :: MyDBMonad (Error [Item])
getItemList = do
-- etc...
But I have little knowledge of Monad and Monad Transformers (I see this question as an exercise to learn about it), and I have no idea how to begin the creation of Database Monad, how to lift the IO from happstack to the Database Stack,...etc.
Here is some minimal working code compiled from snippets above for confused newbies like me.
You put stuff into AppConfig type and grab it with ask inside your response makers.
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Happstack.Server
import Control.Monad.Reader
import qualified Data.ByteString.Char8 as C
myApp :: AppMonad Response
myApp = do
-- access app config. look mom, no lift!
test <- ask
-- try some happstack funs. no lift either.
rq <- askRq
bs <- lookBS "lol"
-- test IO please ignore
liftIO . print $ test
liftIO . print $ rq
liftIO . print $ bs
-- bye
ok $ toResponse ("Oh, hi!" :: C.ByteString)
-- Put your stuff here.
data AppConfig = AppConfig { appSpam :: C.ByteString
, appEggs :: [C.ByteString] } deriving (Eq, Show)
config = AppConfig "THIS. IS. SPAAAAAM!!1" []
type AppMonad = ReaderT AppConfig (ServerPartT IO)
main = simpleHTTP (nullConf {port=8001}) $ runReaderT myApp config {appEggs=["red", "gold", "green"]}
You likely want to use 'ReaderT':
type MyMonad a = ReaderT DbHandle ServerPart a
The Reader monad transformer makes a single value accessible using the ask function - in this case, the value we want everyone to get at is the database connection.
Here, DbHandle is some connection to your database.
Because 'ReaderT' is already an instance of all of the happstack-server type-classes all normal happstack-server functions will work in this monad.
You probably also want some sort of helper to open and close the database connection:
runMyMonad :: String -> MyMonad a -> ServerPart a
runMyMonad connectionString m = do
db <- liftIO $ connect_to_your_db connectionString
result <- runReaderT m db
liftIO $ close_your_db_connection db
(It might be better to use a function like 'bracket' here, but I don't know that there is such an operation for the ServerPart monad)
I don't know how you want to do logging - how do you plan to interact with your log-file? Something like:
type MyMonad a = ReaderT (DbHandle, LogHandle) ServerPart a
and then:
askDb :: MyMonad DbHandle
askDb = fst <$> ask
askLogger :: MyMonad LogHandle
askLogger = snd <$> ask
might be enough. You could then build on those primitives to make higher-level functions. You would also need to change runMyMonad to be passed in a LogHandle, whatever that is.
Once you get more than two things you want access to it pays to have a proper record type instead of a tuple.