How to properly close network connections with network-conduit? - haskell

To learn the basics of conduit library, I used network-conduit to make a simple echo server:
import Control.Monad.IO.Class
import qualified Data.ByteString.Char8 as BS
import Data.Conduit
import Data.Conduit.Network
-- A conduit that print the input it receives on the console
-- and passes it through.
echo :: (MonadIO m) => Conduit BS.ByteString m BS.ByteString
echo = do
yield (BS.pack "Anything you type will be echoed back.\n")
-- Print the received data to the console as well:
awaitForever (\x -> liftIO (BS.putStr x) >> yield x)
echoApp :: (MonadIO m) => Application m
echoApp appData = appSource appData $= echo $$ appSink appData
-- Listen on port 4545:
main :: IO ()
main = runTCPServer (serverSettings 4545 HostAny) echoApp
It does what I wanted, but when the client closes its part of the connection, the server is still waiting for input instead of writing out any remaining data and closing its sending part of the connection too:
$ nc localhost 4545 <<<"Hello world!"
Anything you type will be echoed back.
Hello world!
I tried removing echo and do just
echoApp appData = appSource appData $$ appSink appData
but the problem is still there. What am I doing wrong?

I'm not sure what you mean by "the server won't respond to it"? I'd guess you're expecting the server to shut down after the client disconnects. If so, that's not the intention of the library: it continues to server connections in an infinite loop as long as they continue coming in. Using addCleanup, you can see that the individual connection handlers do in fact terminate, e.g.:
echo :: (MonadIO m) => Conduit BS.ByteString m BS.ByteString
echo = addCleanup (const $ liftIO $ putStrLn "Stopping") $ do
yield (BS.pack "Anything you type will be echoed back.\n")
-- Print the received data to the console as well:
awaitForever (\x -> liftIO (BS.putStr x) >> yield x)

It turned out that the problem wasn't with network-conduit, that part works correctly. The problem was with nc, which doesn't close its sending part of the socket when it sends all the data. I made a testing python script and it works against the server as expected:
#!/usr/bin/env python
import socket
HOST = 'localhost'
PORT = 4545
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall('Hello, world')
# This was missing in `nc`:
s.shutdown(socket.SHUT_WR);
print 'Received'
data = s.recv(1024)
while data:
print data,
data = s.recv(1024)
s.close()

Related

How do I shut down `runTCPServer`?

I'm writing a socket server with runTCPServer from conduit-extra (formerly known as network-conduit). My goal is to interact with my editor using this server --- activate the server from the editor (most likely just by calling external command), use it, and terminate the server when the work is done.
For simplicity, I start with a simple echo server, and let's say I'd like to shut down the whole process when the connection is closed.
So I tried:
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Data.Conduit
import Data.Conduit.Network
import Data.ByteString (ByteString)
import Control.Monad.IO.Class (liftIO)
import System.Exit (exitSuccess)
import Control.Exception
defaultPort :: Int
defaultPort = 4567
main :: IO ()
main = runTCPServer (serverSettings defaultPort "*") $ \ appData ->
appSource appData $$ conduit =$= appSink appData
conduit :: ConduitM ByteString ByteString IO ()
conduit = do
msg <- await
case msg of
Nothing -> liftIO $ do
putStrLn "Nothing left"
exitSuccess
-- I'd like the server to shut down here
(Just s) -> do
yield s
conduit
But this doesn't work -- the program continues to accept new connections. If I am not mistaken, this is because the thread listening to the connection we're dealing with exits with exitSuccess, but the entire process doesn't. So this is totally understandable, but I haven't been able to find a way to exit the whole process.
How do I terminate a server run by runTCPServer? Is runTCPServer something that's supposed to serve forever?
Here's a simple implementation of the idea described in comments:
main = do
mv <- newEmptyMVar
tid <- forkTCPServer (serverSettings defaultPort "*") $ \ appData ->
appSource appData $$ conduit mv =$= appSink appData
() <- takeMVar mv -- < -- wait for done signal
return ()
conduit :: MVar () -> ConduitM ByteString ByteString IO ()
conduit mv = do
msg <- await
case msg of
Nothing -> liftIO $ do
putStrLn "Nothing left"
putMVar mv () -- < -- signal that we're done
(Just s) -> do
yield s
conduit mv

Extending the IRC bot from wiki.haskell.org with state

Problem
I'm attempting to extend the IRC bot from https://wiki.haskell.org/Roll_your_own_IRC_bot with some state that's updated every time the bot posts a message in the channel it's connected to.
The feature is: every time the command !last said is issued in the IRC channel, the bot should respond with a time stamp. To support this, the privmsg function needs to update the bot's state -- specifically the lastPosted record -- with a new timestamp every time it is called.
Work so far
I took the code from the bottom of the Haskell wiki page (which used a ReaderT to access information about the bot's environment) and tried to change out the ReaderT for a State Transformer (StateT). The results are below and as you can see, I didn't get very far.
import Data.List
import Network
import System.IO
import System.Exit
import System.Time
import Control.Arrow
import Control.Monad.State
import Control.Exception
import Text.Printf
server = "irc.freenode.org"
port = 6667
chan = "#testbot-test"
nick = "testbottest"
-- The 'Net' monad, a wrapper over IO, carrying the bot's immutable state.
type Net = StateT Bot IO
data Bot = Bot { socket :: Handle, lastPosted :: ClockTime }
-- Set up actions to run on start and end, and run the main loop
main :: IO ()
main = bracket connect disconnect loop
where
disconnect = hClose . socket
loop st = runStateT run st
-- Connect to the server and return the initial bot state
connect :: IO Bot
connect = notify $ do
h <- connectTo server (PortNumber (fromIntegral port))
t <- getClockTime
hSetBuffering h NoBuffering
return (Bot h t)
where
notify a = bracket_
(printf "Connecting to %s ... " server >> hFlush stdout)
(putStrLn "done.")
a
-- We're in the Net monad now, so we've connected successfully
-- Join a channel, and start processing commands
run :: Net ()
run = do
write "NICK" nick
write "USER" (nick ++ " 0 * :test bot")
write "JOIN" chan
gets socket >>= listen
-- Process each line from the server
listen :: Handle -> Net ()
listen h = forever $ do
s <- init `fmap` liftIO (hGetLine h)
liftIO (putStrLn s)
if ping s then pong s else eval (clean s)
where
forever a = a >> forever a
clean = drop 1 . dropWhile (/= ':') . drop 1
ping x = "PING :" `isPrefixOf` x
pong x = write "PONG" (':' : drop 6 x)
-- Dispatch a command
eval :: String -> Net ()
eval "!quit" = write "QUIT" ":Exiting" >> liftIO (exitWith ExitSuccess)
-- Posting when something was last posted shouldn't count as last posted.
eval "!last said" = getLastPosted >>= (\t -> write "PRIVMSG" (chan ++ " :" ++ t))
eval x | "!id " `isPrefixOf` x = privmsg (drop 4 x)
eval _ = return () -- ignore everything else
getLastPosted :: Net String
getLastPosted = do
t <- gets lastPosted
return $ show t
-- Send a privmsg to the current chan + server
privmsg :: String -> Net ()
privmsg s = write "PRIVMSG" (chan ++ " :" ++ s)
-- Send a message out to the server we're currently connected to
write :: String -> String -> Net ()
write s t = do
h <- gets socket
liftIO $ hPrintf h "%s %s\r\n" s t
liftIO $ printf "> %s %s\n" s t
Other support avenues explored
spent a couple of days reading up on ReaderT, StateT and their non-transformer friends Reader and State,
checking Stack Overflow for anyone with a similar problem, but the only other IRC bot question threaded the socket as an argument to every function that needed it (instead of using a ReaderT),
Tweeted Don S. the original author of the wiki page
asked in the Haskell IRC channel.
Question
How can the Haskell wiki IRC bot be extended to post a message, containing the date and time stamp of the last message posted? Preferably using an abstraction like ReaderT (only allowing mutable state) rather than passing state around in function arguments.
I got your code to compile by simply adding a >> return () to the definition of loop in your main:
main :: IO ()
main = bracket connect disconnect loop
where
disconnect = hClose . socket
loop st = (runStateT run st) >> return ()
This effectively ignores the return value of runStateT. Here are all of the variants of runState/runStateT:
runStateT - return both the final state and returned value
evalStateT - return only the final value
execStateT - return only the final state
Your original definition of loop was returning a pair (from runStateT), and this didn't type check since main wants a computation which returns just ().
To update the lastPosted field, consider this addition to the eval function which is triggered when the bot is sent the message !update time:
eval "!update time"
= do t <- liftIO getClockTime
bot <- get
put (bot { lastPosted = t })
We need to liftIO getClockTime since we are operating in the Net monad.
Then we get the old state and put the updated state. You can add this logic wherever you want to update the lastPosted time in the Net monad.
Full code is available at: http://lpaste.net/142931

Why does hIsEOF not return?

I am creating a simple chat server using Haskell's Network library. The server has to do two things:
Broadcast a message every n ticks to all of the currently connected clients
Receive any messages from the clients sent to the server.
The server and client are defined as
data Server = Server {
sSocket :: Socket,
sPort :: Port,
sClients :: MVar (Set.Set ClientAddress)
}
newtype ClientAddress = ClientAddress (Handle, HostName, PortNumber)
deriving (Show)
A server is constructed by running the following function
startServer :: Port -> IO ThreadId --
startServer port = withSocketsDo $ do
socket <- listenOn $ PortNumber $ fromIntegral port
clients <- newMVar Set.empty
let server = Server socket port clients
forkIO $ forever $ do
client#(handle, host, port) <- accept socket
modifyMVar_ clients (\cs -> return $ Set.insert (ClientAddress client) cs)
forkIO $ forever $ serve $ ClientAddress client
forkIO $ forever $ sendServerUpdates 1000000 server
Note that the last two lines fork two different threads: the first for handling client connections and "serving" their messages, and the second for sending server broadcasts to the clients.
Broadcasting to the clients works as follows
sendServerUpdates :: Microsecond -> Server -> IO ()
sendServerUpdates frequency server = do
withMVar (sClients server) (mapM_ sendServerUpdate)
threadDelay frequency
sendServerUpdate :: ClientAddress -> IO ()
sendServerUpdate (ClientAddress (handle, host, port)) = do
putStrLn "Sending update."
The issue I have, is that receiving messages from the clients seem to block. I receive messages by checking if the handle has contents
serve :: ClientAddress -> IO ()
serve (ClientAddress (handle, host, port)) = do
b <- hIsEOF handle
putStrLn $ show $ b -- <-- It never makes it this far...
Unfortunately, the code never makes it to the second line to call the putStrLn. It seems like hIsEOF encounters some exception, although the documentation doesn't seem to mention it.
Why does my code block on hIsEOF forever?
In hIsEOF's documentation I found the following:
NOTE: hIsEOF may block, because it has to attempt to read from the
stream to determine whether there is any more data to be read.
I wouldn't have expected this. I wonder if hReady or hGetBufNonBlocking are any better? I've never tried the whole non-blocking IO bit in Haskell.

Best way to send 10GB vector across network in Haskell

I am curious to know what the best way to send 10 GB once over the network in Haskell? I don't want to send it as binary without having to write a decode/encoder. Is it best to keep it strict or lazy?
You can also use Pipes. A zlib compressed network example can be:
module Main where
--------------------------------------------------------------------------------
import Pipes
import qualified Pipes.Prelude as P
import Pipes.Network.TCP
import Data.ByteString
import Pipes.ByteString as PB
import System.Environment
import Pipes.Zlib
--------------------------------------------------------------------------------
myReceive :: (MonadIO m) => m ()
myReceive = serve (Host "127.0.0.1") "8000" $ \(sock, remote) -> do
Prelude.putStrLn $ "TCP connection established from " ++ show remote
runEffect $ decompress defaultWindowBits (fromSocket sock 4096) >-> PB.stdout
mySend :: IO ()
mySend = connect "127.0.0.1" "8000" $ \(sock, remote) -> do
Prelude.putStrLn $ "Connection established to " ++ show remote
runEffect $ compress bestCompression defaultWindowBits (PB.stdin) >-> toSocket sock
main = do
a:[] <- getArgs
case a of
"send" -> mySend
"receive" -> myReceive
_ -> Prelude.putStrLn "Usage: netsend <send|receive>"

Using a monad inside the IO monad

Is there something that is like the opposite of liftIO? I'm using websockets, and I want to be able to listen for messages from the server in a separate thread. Here's what I'm doing:
import Network.WebSockets
import qualified Data.Text as T
import Control.Monad.IO.Class
import Control.Monad
import Control.Concurrent
import Control.Applicative
printMessages :: WebSockets Hybi00 ()
printMessages = forever $ do
resp <- receiveDataMessage
liftIO $ print resp
run :: WebSockets Hybi00 ()
run = do
liftIO . forkIO $ printMessages
forever $ do
line <- liftIO getLine
sendTextData . T.pack $ line
main = connect "0.0.0.0" 8080 "/" run
So printMessages listens for messages from the server and keeps printing them out. The problem is, forkIO expects a function that returns IO (). Is there any way for me to run printMessages in the IO monad?
If I'm understanding this right, the reason you want to receive messages in another thread is because the main thread will be waiting for user input to send.
From a look at the documentation, it seems like you'll have an easier time if you reverse the roles of the threads: receive in the main thread, and send asynchronously from the other.
Then you can use getSink :: Protocol p => WebSockets p (Sink p) to grab a sink before forking, which you can then use with sendSink :: Sink p -> Message p -> IO () which lives in IO, avoiding the whole problem of mixing monads.
In other words, restructure your code to something like this:
sendMessages :: Sink Hybi00 -> IO ()
sendMessages sink = forever $ do
line <- getLine
let msg = textData . T.pack $ line
sendSink sink msg
run :: WebSockets Hybi00 ()
run = do
sink <- getSink
liftIO . forkIO $ sendMessages sink
forever $ do
resp <- receiveDataMessage
liftIO $ print resp
main = connect "0.0.0.0" 8080 "/" run

Resources