Gmail TLS hanshake failure in haskell - haskell

I am trying to write a little script to send an email via gmail.
I can connect to gmail but when I try to hanshake it fails.
Any pointers to get the handshake a go?
Here is the code:
import Network.Socket
import Network
import Network.TLS
import Network.TLS.Extra
import Crypto.Random
import Data.CertificateStore -- to remove
import System.Certificate.X509 --to use I think
import System.IO
import Text.Printf
import Control.Monad (forever)
import qualified Data.ByteString.Char8 as B
main :: IO ()
main = emailGmail2
tListen :: Context -> IO ()
tListen ctx =
forever $ recvData ctx >>= B.putStrLn
cWrite :: Handle -> String -> IO ()
cWrite h s = do
hPrintf h "%s\r\n" s
printf "> %s\n" s
cListen :: Handle -> IO ()
cListen h =
forever $ hGetLine h >>= putStrLn
emailGmail2 = do
let
host = "smtp.gmail.com"
port = 587
params = defaultParamsClient
g <- newGenIO :: IO SystemRandom
h <- connectTo host (PortNumber (fromIntegral port))
hSetBuffering h LineBuffering
cWrite h "EHLO"
cWrite h "STARTTLS"
--cListen h
con <- contextNewOnHandle h params g
handshake con
tListen con
And here's the error:
HandshakeFailed (Error_Packet_Parsing "Failed reading: invalid header
type: 50\nFrom:\theader\n\n")

This error is partially to do with handling the SMTP protocol. Looking at the RFC for Secure SMTP over TLS RFC 2487 there is an example client-server dialogue.
S: <waits for connection on TCP port 25>
C: <opens connection>
S: 220 mail.imc.org SMTP service ready
C: EHLO mail.ietf.org
S: 250-mail.imc.org offers a warm hug of welcome
S: 250 STARTTLS
C: STARTTLS
S: 220 Go ahead
C: <starts TLS negotiation>
C & S: <negotiate a TLS session>
C & S: <check result of negotiation>
C: <continues by sending an SMTP command>
. . .
In your code you are sending EHLO and STARTTLS, and then immediately beginning the handshake negotiation. I believe what is happening is that the server is still sending some of the 250 and 220 codes above and the TLS library is trying to interpret these as TLS messages which is causing the problem.
Indeed if I open another terminal and listen to port 587 with netcat and change the program to connect to localhost I get the same error if I reply with "250 STARTTLS"
Changing this got the program to work for me:
import Network.Socket
import Network
import Network.TLS
import Network.TLS.Extra
import Crypto.Random
import qualified Crypto.Random.AESCtr as RNG
import System.IO
import Text.Printf
import Control.Monad (when)
import Data.List (isPrefixOf)
ciphers :: [Cipher]
ciphers =
[ cipher_AES128_SHA1
, cipher_AES256_SHA1
, cipher_RC4_128_MD5
, cipher_RC4_128_SHA1
]
main :: IO ()
main = emailGmail2
cWrite :: Handle -> String -> IO ()
cWrite h s = do
hPrintf h "%s\r\n" s
printf "> %s\n" s
cWaitFor :: Handle -> String -> IO ()
cWaitFor h str = do
ln <- hGetLine h
putStrLn ln
when (not $ str `isPrefixOf` ln) (cWaitFor h str)
emailGmail2 = do
let
host = "smtp.gmail.com"
port = 587
params = defaultParamsClient{pCiphers = ciphers}
g <- RNG.makeSystem
h <- connectTo host (PortNumber (fromIntegral port))
hSetBuffering h LineBuffering
cWrite h "EHLO"
cWaitFor h "250-STARTTLS"
cWrite h "STARTTLS"
cWaitFor h "220"
con <- contextNewOnHandle h params g
handshake con
bye con
Also note that I needed to add some ciphers from Network.TLS.Extras as otherwise I got an Illegal Parameter error. I found the code for this and to add logging from Vincent's tests on the Github page.
Another note: In case you run into more issues I should point out that I used the command line programs gnutls-cli and ssldump to debug the issue with ciphers mentioned above.

Related

Really weird issue with ByteString socket

I had a weird problem with ByteString socket programming. I have minimized the issue to the following short scripts:
-- ByteString.hs
{-# LANGUAGE OverloadedStrings #-}
import Network.Socket hiding (send, sendTo, recv, recvFrom)
import Network.Socket.ByteString
import Network.BSD
import System.IO
import qualified Data.ByteString as BS
import qualified Data.ByteString.Char8 as C8
connectMud :: HostName
-> String
-> IO Handle
connectMud hostname port = do
addrinfos <- getAddrInfo Nothing (Just hostname) (Just port)
let serveraddr = head addrinfos
sock <- socket (addrFamily serveraddr) Stream defaultProtocol
setSocketOption sock KeepAlive 1
connect sock (addrAddress serveraddr)
h <- socketToHandle sock ReadWriteMode
hSetBuffering h (BlockBuffering Nothing)
return h
main:: IO ()
main =
C8.putStrLn "connecting, please wait" >> connectMud "aardwolf.org" "4000"
>>= BS.hGetContents >>= C8.putStrLn
and
--ByteString_Lazy.hs
{-# LANGUAGE OverloadedStrings #-}
import Network.Socket hiding (send, sendTo, recv, recvFrom)
import Network.Socket.ByteString
import Network.BSD
import System.IO
import qualified Data.ByteString.Lazy as BS
import qualified Data.ByteString.Lazy.Char8 as C8
connectMud :: HostName
-> String
-> IO Handle
connectMud hostname port = do
addrinfos <- getAddrInfo Nothing (Just hostname) (Just port)
let serveraddr = head addrinfos
sock <- socket (addrFamily serveraddr) Stream defaultProtocol
setSocketOption sock KeepAlive 1
connect sock (addrAddress serveraddr)
h <- socketToHandle sock ReadWriteMode
hSetBuffering h (BlockBuffering Nothing)
return h
main:: IO ()
main =
C8.putStrLn "connecting, please wait" >> connectMud "aardwolf.org" "4000"
>>= BS.hGetContents >>= C8.putStrLn
The only difference between these two short scripts is that one is using Data.ByteString and the other one Data.ByteString.Lazy. The lazy version works fine but the strict version does not seem to receive any output at all. Any thoughts?
I would guess that it has to do with the differing behavior of the two hGetContents functions in question (the lazy version vs the strict version).
The strict version "reads a handle's entire contents strictly into a ByteString." But you're trying to read from a stream, so the EOF is never encountered.
Compare with the behavior of the lazy version:
Read entire handle contents lazily into a ByteString. Chunks are read on demand, using the default chunk size.
Once EOF is encountered, the Handle is closed.

connecting to server with haskell

Suppose in my program I need to connect to the server (using only Network, not Network.Socket!) and post, get some data (think about it as telnet analog). I need to receive messages from server in separate thread (because I can send something on server and get something seperatly). So the question is how to receive messages and and send it in two separate threads? I dont know how to use forkIO and how to use Handle
For now I wrote something like this:
sender :: Handle -> IO ()
sender h = do
toSend <- getContents
hPutStr h toSend
sender h
receiver :: Handle -> IO ()
receiver h = do
response <- hGetContents h
putStrLn $ "the response is: " ++ response
receiver h
main :: IO ()
main = do
let host = "haskell.org"
let port = 40
h <- connectTo host (PortNumber $ fromIntegral i)
forkIO $ receiver h
sender h
return ()
As I understood, this code works quite well. The main problem was with port I used. haskell.org's port 40 is not open (found out using nmap). So the connection just freezed. Only few changes I did in sender and receiver:
sender :: Handle -> IO ()
sender h = getContents >>= hPutStrLn h
receiver :: Handle -> IO ()
receiver h = hGetContents h >>= putStrLn
So the final code is
import Network
import Control.Concurrent(forkIO)
import System.IO
import System.Environment
import Control.Monad
sender :: Handle -> IO ()
sender h = getContents >>= hPutStrLn h
receiver :: Handle -> IO ()
receiver h = forever $ hGetLine h >>= putStrLn
main :: IO ()
main = do
[host, port] <- getArgs
h <- connectTo host (Service port)
forkIO $ receiver h
sender h
return ()

Reading from a Socket while keeping the connection open

currently I have some code which creates a server and a client for some testing purposes.
Server code
import Network.Socket
import Network.Socket.ByteString as NSB
import Network.Socket.ByteString.Lazy as NSBL
import Data.ByteString.Lazy as BSL
import Data.ByteString as BS
import Data.HashMap as HM
import System.IO as IO
main = withSocketsDo $ do
sock <- socket AF_INET Stream 0
setSocketOption sock ReuseAddr 1
bindSocket sock (SockAddrInet (fromInteger 8585) iNADDR_ANY)
listen sock 100
servLoop sock
servLoop sock = do
client = accept sock
IO.putStrLn "Got a connection"
h <- SocketToHandle client ReadWriteMode
hSetBuffering h NoBuffering
req <- BSL.hGet h 1024
IO.putStrLn "Got some contents:"
IO.putStrLn $ show req
Client Code
import Network.Socket
import Network.BSD
import Control.Monad
import System.IO as IO
import Data.Text as T
import Data.ByteString.Lazy as BSL
import Data.MessagePack as MP
main = withSocketsDo $ do
sock <- socket AF_INET Stream 0
setSocketOption sock ReuseAddr 1
addr <- liftM hostAddresses $ getHostByName "localhost"
connect sock $ SockAddrInet (fromInteger 8585) (Prelude.head addr)
handle <- socketToHandle sock ReadWriteMode
replicateM_ 5 $ BSL.hPut handle $ MP.pack ("Hello host" :: Text)
hFlush handle
getLine
replicateM_ 5 $ BSL.hPut handle $ MP.pack ("Hello host" :: Text)
hFlush handle
hClose handle
The observed behaviour is that the message is not sent until hClose handle is called on the client side. However, I would like to keep the handle open for sending more requests and receiving responses rapidly. Am I going about this the right way? And if so, is there a way to keep the handle open but read and write the socket?
hGet keeps trying to read until it has as many bytes as you asked for, namely 1024. You probably want to use hGetNonBlocking instead. Or something like:
do
r1 <- BSL.hGet h 1
rRest <- BSL.hGetNonBlocking h 1024
return (r1 BS.append rRest)

Best way to send 10GB vector across network in Haskell

I am curious to know what the best way to send 10 GB once over the network in Haskell? I don't want to send it as binary without having to write a decode/encoder. Is it best to keep it strict or lazy?
You can also use Pipes. A zlib compressed network example can be:
module Main where
--------------------------------------------------------------------------------
import Pipes
import qualified Pipes.Prelude as P
import Pipes.Network.TCP
import Data.ByteString
import Pipes.ByteString as PB
import System.Environment
import Pipes.Zlib
--------------------------------------------------------------------------------
myReceive :: (MonadIO m) => m ()
myReceive = serve (Host "127.0.0.1") "8000" $ \(sock, remote) -> do
Prelude.putStrLn $ "TCP connection established from " ++ show remote
runEffect $ decompress defaultWindowBits (fromSocket sock 4096) >-> PB.stdout
mySend :: IO ()
mySend = connect "127.0.0.1" "8000" $ \(sock, remote) -> do
Prelude.putStrLn $ "Connection established to " ++ show remote
runEffect $ compress bestCompression defaultWindowBits (PB.stdin) >-> toSocket sock
main = do
a:[] <- getArgs
case a of
"send" -> mySend
"receive" -> myReceive
_ -> Prelude.putStrLn "Usage: netsend <send|receive>"

How to properly close network connections with network-conduit?

To learn the basics of conduit library, I used network-conduit to make a simple echo server:
import Control.Monad.IO.Class
import qualified Data.ByteString.Char8 as BS
import Data.Conduit
import Data.Conduit.Network
-- A conduit that print the input it receives on the console
-- and passes it through.
echo :: (MonadIO m) => Conduit BS.ByteString m BS.ByteString
echo = do
yield (BS.pack "Anything you type will be echoed back.\n")
-- Print the received data to the console as well:
awaitForever (\x -> liftIO (BS.putStr x) >> yield x)
echoApp :: (MonadIO m) => Application m
echoApp appData = appSource appData $= echo $$ appSink appData
-- Listen on port 4545:
main :: IO ()
main = runTCPServer (serverSettings 4545 HostAny) echoApp
It does what I wanted, but when the client closes its part of the connection, the server is still waiting for input instead of writing out any remaining data and closing its sending part of the connection too:
$ nc localhost 4545 <<<"Hello world!"
Anything you type will be echoed back.
Hello world!
I tried removing echo and do just
echoApp appData = appSource appData $$ appSink appData
but the problem is still there. What am I doing wrong?
I'm not sure what you mean by "the server won't respond to it"? I'd guess you're expecting the server to shut down after the client disconnects. If so, that's not the intention of the library: it continues to server connections in an infinite loop as long as they continue coming in. Using addCleanup, you can see that the individual connection handlers do in fact terminate, e.g.:
echo :: (MonadIO m) => Conduit BS.ByteString m BS.ByteString
echo = addCleanup (const $ liftIO $ putStrLn "Stopping") $ do
yield (BS.pack "Anything you type will be echoed back.\n")
-- Print the received data to the console as well:
awaitForever (\x -> liftIO (BS.putStr x) >> yield x)
It turned out that the problem wasn't with network-conduit, that part works correctly. The problem was with nc, which doesn't close its sending part of the socket when it sends all the data. I made a testing python script and it works against the server as expected:
#!/usr/bin/env python
import socket
HOST = 'localhost'
PORT = 4545
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall('Hello, world')
# This was missing in `nc`:
s.shutdown(socket.SHUT_WR);
print 'Received'
data = s.recv(1024)
while data:
print data,
data = s.recv(1024)
s.close()

Resources