basic mqtt with haskell - haskell

I moved mqtt-hs to LTS-5.13 and compiled just fine with stack. Then I created the following subscriber to listen on a topic hierarchy. This is the subscriber code
{-# LANGUAGE OverloadedStrings #-}
module Main(main) where
import Control.Concurrent (threadWaitRead)
import System.Posix.Types
import Network.MQTT
import Network.MQTT.Logger
main :: IO ()
main = do
conn <- connect defaultConfig { cLogger = warnings stdLogger, cHost = "192.168.0.1" }
qos <- subscribe conn Confirm "/something" callback
putStrLn "control-c to finish"
threadWaitRead $ Fd 1
callback topic payload = putStrLn $ "A message was published to " ++ show topic ++ ": " ++ show payload
When I send something to the topic (using a simple Haskell publisher too), I am getting this in the console of this program:
control-c to finish
[Error] {handle: <socket: 3>}: recvLoop: end of file
[Error] recvLoop: No reconnect, terminating.
but no other output.
Both publisher and subscriber connect to a broker (Mosquitto). The subscriber seems to be Ok, the above happens only when receiving the message from the publisher. The above message also occurs when I stop the broker.
Any ideas?
Update
I cloned most recent mqtt-hs (0.3.0) and introduced this change
--- a/Network/MQTT.hs
+++ b/Network/MQTT.hs
## -448,9 +448,10 ## recvLoop :: MQTT -> IO ()
recvLoop m = loopWithReconnect m "recvLoop" $ \mqtt -> do
h <- readMVar (handle mqtt)
eof <- hIsEOF h
- if eof
- then ioError $ mkIOError eofErrorType "" (Just h) Nothing
- else getMessage mqtt >>= dispatchMessage mqtt
+ getMessage mqtt >>= dispatchMessage mqtt
+ -- if eof
+ -- then ioError $ mkIOError eofErrorType "" (Just h) Nothing
+ -- else getMessage mqtt >>= dispatchMessage mqtt
`catch`
\e -> logWarning mqtt $ "recvLoop: Caught " ++ show (e :: MQTTException)
i.e, I disabled the EOF checking. Now the message is printed to the console, but the subscriber enters into a loop throwing [Warning] recvLoop: Caught EOF as fast as it can.
Is this a mosquito error or bug in mqtt-hs?
Update 2
I can confirm it works with ActiveMQ without the hack. Having said this, it is better if mqtt-hs could recover from broker closing connection to a subscriber.

I believe that you might have it misconfigured. I'm currently using mqtt-hs in production and have almost no issues with it (barring a bug that has already been corrected in the project's github). The new update uses a TChan to have the messages delivered correctly as in a persistent connection, there is a chance the that some messages may be lost as the handler has not yet been made available.
You define the connection to the broker, but you don't tell the client about how the reconnections should be handled. For reference, I'm using something like the following at the moment:
mqttConfig :: HostName -> MQTTConfig
mqttConfig host = defaultConfig { cClean = False
, cClientID = "myClientId"
, cHost = host
, cUsername = Just "username"
, cPassword = Just "password"
, cKeepAlive = Just 10
, cReconnPeriod = Just 1
, cLogger = stdLogger }
Even though the version in use here is slightly updated, very little needed to change and reconnctions happen as needed when the broker (mosquitto as well here) goes down and then later recovers.

Related

"Error No session in progress" error for simple example using webdriver-w3c

{-# LANGUAGE OverloadedStrings #-}
module Main where
import Web.Api.WebDriver.Endpoints
import Web.Api.WebDriver
main :: IO ()
main = do
x <- execWebDriverT defaultWebDriverConfig doLogin
print x
doLogin :: WebDriverT IO ()
doLogin = do
fullscreenWindow
navigateTo "http://localhost:8001/login"
z <- findElements CssSelector "div.alert"
assertEqual [] z "Errors found"
The above code results in an error of:
2021-12-13 16:32:39 ERROR Error No session in progress (Left (E
NoSession),S {_httpOptions = Options { manager = Left _, proxy =
Nothing, auth = Nothing, headers = [("User-Agent","haskell
wreq-0.5.3.2")], params = [], redirects = 10, cookies = Just (CJ
{expose = []}) }, _httpSession = Nothing, _userState = WDState
{_sessionId = Nothing, _breakpoints = BreakpointsOff}},W {unW =
[LogEntry {_logEntryTimestamp = 2021-12-13 16:32:39.03282154 UTC,
_logEntryUID = "", _logEntrySeverity = LogError, _logEntry = L_Error NoSession}]})
What does this error indicate? What am I missing?
Full example project can be found here: https://github.com/chrissound/434/commit/ea1f3d840b64093b40ebba0e3dfceaacd4b36716
The error indicates no session has been initialized on the 'webdriver'.
A session is equivalent to a single instantiation of a particular user agent, including all its child browsers. WebDriver gives each session a unique session ID that can be used to differentiate one session from another, allowing multiple user agents to be controlled from a single HTTP server, and allowing sessions to be routed via a multiplexer (known as an intermediary node).
https://www.w3.org/TR/webdriver/#sessions
A session can be created/managed with:
x <- execWebDriverT defaultWebDriverConfig $
runIsolated defaultFirefoxCapabilities doLogin
Instead of x <- execWebDriverT defaultWebDriverConfig doLogin.
Seems like runIsolated takes care of all the session handling.

Warp / Scotty not terminating thread / resources at end of request

I'm running into an issue where my Scotty app does not seem to terminate old HTTP request threads. And eventually, after a large number (10-20) of concurrent requests, I run into an error with too many DB connections libpq: failed (FATAL: sorry, too many clients already).
{-# LANGUAGE OverloadedStrings #-}
import Web.Scotty
import Database.PostgreSQL.Simple
import Control.Monad.IO.Class
connection :: IO Connection
connection = connect defaultConnectInfo
{ connectHost = "localhost", connectUser="postgres", connectPassword="mysecretpassword" }
main :: IO ()
main = scotty 8000 $ do
get "/" $ do
c <- liftIO $ connection
text "test"
This also happens with a Warp application (which Scotty):
{-# LANGUAGE OverloadedStrings #-}
import Network.Wai
import Network.Wai.Handler.Warp (run)
import Network.HTTP.Types (status200)
import Network.HTTP.Types.Header (hContentType)
import Database.PostgreSQL.Simple
import Control.Monad.IO.Class
connection :: IO Connection
connection = connect defaultConnectInfo
{ connectHost = "localhost", connectUser="postgres", connectPassword="mysecretpassword" }
main = run 8000 app
app :: Application
app req respond = do
respond $ responseStream status200 [] $ \write flush -> do
print "test"
con <- connection
flush
write $ "World\n"
Why is this happening? Is there an simple way to "finalize" the request at the end?
I can manually close the connection, but ideally I think killing the thread with any other related resources would be ideal.
I've verified that it keeps the connection open by running the following in postgres:
SELECT sum(numbackends) FROM pg_stat_database;
Scotty seems to take a few seconds until it closes it automatically (after the request is completed).
postgresql-simple provides the close :: Connection -> IO () function which closes the connection and free associated resources. You need to close the connection after you are done with it.
But a common problem is the following: what happens if the code between opening and closing the connection throws an exception? How to ensure that the connection is closed even in that case, so that we don't leak connections?
In Haskell, this is solved using the bracket :: IO a -> (a -> IO b) -> (a -> IO c) -> IO c function. You pass it an IO action that allocates some resource (in this case, a connection) a function that frees the allocated resource once we are we are done with it (in our case, the close function) and a function that says what we actually want to do with the allocated resource (in this case, perform a query).
bracket is somewhat similar to try-with-resources in Java or the "using" statement in C#.
Instead of opening and closing a connection on each request, a better approach would be so use some kind of connection pool shared between request threads. persistent-postgresql uses resource-pool for example.

Do i need a scheduler to create daily log files with the hslogger library of Haskell?

First of all, this is my first experience ever with logs; and i don't really know what a scheduler is, only heard about them.
Now, as a first approach, i want to make a file with only important logs (WARNING or above), and daily files (like 2017-7-20.log) for all possible logged messages that day.
This is my code atm:
module Utilities.Loggers where
import System.Log.Logger
-- import System.Log.Handler.Syslog (TODO: remote logs)
import System.Log.Handler.Simple
import System.Log.Handler (LogHandler, setFormatter)
import System.Log.Formatter
import Data.Time
import GHC.IO.Handle (Handle)
--------------------------------------------
importantLogsPath :: FilePath
importantLogsPath = "app/logs/prioritary.log"
logsFolder :: FilePath
logsFolder = "app/logs/allLogs/"
defaultFormat :: LogFormatter a
defaultFormat = simpleLogFormatter "[$utcTime : $tid : $loggername : $prio] $msg"
applyFormat :: LogHandler a => a -> a
applyFormat h = setFormatter h defaultFormat
--------------------------------------------
prioritaryHandler :: IO (GenericHandler Handle)
prioritaryHandler = applyFormat <$> prioritaryHandler'
where
prioritaryHandler' = fileHandler importantLogsPath WARNING
todaysHandler :: UTCTime -> IO (GenericHandler Handle)
todaysHandler = (applyFormat <$>) . todaysHandler'
where
todaysHandler' time = fileHandler (mkPath time) DEBUG
mkPath time = logsFolder ++ show (utctDay time) ++ ".log"
--------------------------------------------
initLogger :: IO ()
initLogger = do
h1 <- prioritaryHandler
timeNow <- getCurrentTime
h2 <- todaysHandler timeNow
updateGlobalLogger "MyApp" (setHandlers [h2,h1])
If i understood correctly, each handle "listens" for possible messages that are above its priority level (daily handles have the DEBUG priority, the lowest possible, so they should catch every message)
So i need something that everyday at 00:00 utctime removes the handle for the passing day and adds the handle for the new day, right?
Also i'm a bit puzzled with the naming issues for loggers/messages, but that's just me (althout any insight will be welcome)
Edit: it's a web server

Where is the breaking change?

I wrote a CRUD application to interface with JIRA. I ended up upgrading my haskell enviornment, because cabal-dev doesn't solve everything. As a result, I've got some breakage, with this error anytime I try to use any code that interfaces with JIRA.
Spike: HandshakeFailed (Error_Misc "user error (unexpected type received. expecting
handshake and got: Alert [(AlertLevel_Warning,UnrecognizedName)])")
After a little googling, I think this either has to do with tls or http-conduit which uses tls.
I'm currently using tls-1.1.2 and http-conduit-1.8.7.1
previously I was using
tls-0.9.11 and http-conduit >= 1.5 && < 1.7 (not sure which exactly, old install is gone.
This is where I believe the break is happening
manSettings :: ManagerSettings
manSettings = def { managerCheckCerts = \ _ _ _-> return CertificateUsageAccept }
this is what it used to look like
manSettings :: ManagerSettings
manSettings = def { managerCheckCerts = \ _ _ -> return CertificateUsageAccept }
Here's the code that uses it
initialRequest :: forall (m :: * -> *). URI -> IO (Request m,Manager)
initialRequest uri = do
initReq <- parseUrl uri -- let the server tell you what the request header
-- should look like
manager <- newManager manSettings -- a Manager manages http connections
-- we mod the settings to handle
-- the SSL cert. See manSettings below.
return (modReq initReq,manager)
where modReq initReq = applyBasicAuth username password initReq
Let me know if I'm left something out. I'm not sure at this point what broke between then and now.
It's a good guess about the error source, but very unlikely: managerCheckCerts simply uses the certificate package to inspect certificates for validity. The error message you're seeing seems to be coming from tls itself and indicates a failure in the data transport. It's probably a good idea to file a bug report with tls, preferably first by narrowing down the issue to a single HTTPS call that fails (or even better, using tls alone and demonstrating the same failure).

Haskell ZeroMQ binding not working for REQ socket

So here i was, barely able to install the libzmq on a windows desktop and then zeromq-haskell with cabal. I wanted to test the api by binding a python program with a haskell program in a hello-world type application.
So the most basic pattern i see is the request-reply pattern . First i tried to make the server in haskell (REP) and the client in python (REQ), witch failed miserably no matter what i did. The generated exception message was Exception: receive: failed (No error).
So i look inside the System.ZMQ and System.ZMQ.Base source code and i see that receive throws an error on calling c_zmq_recv , witch in turn maps directly to a ffi (?) call to the C api. So i think perhaps i didn't do the installation properly , but then i try to make the client in Haskell and the server in python and i notice it works without any problem, so perhaps the recv interface isn't the problem here.
Here is the haskell code below , with both client and server functions
import System.ZMQ
import Control.Monad (forM_,forever)
import Data.ByteString.Char8 (pack,unpack)
import Control.Concurrent (threadDelay)
clientMain :: IO ()
clientMain = withContext 1 (\context->do
putStrLn "Connecting to server"
withSocket context Req $ (\socket-> do
connect socket "tcp://127.0.0.1:5554"
putStrLn $ unwords ["Sending request"]
send socket (pack "Hello...") []
threadDelay (1*1000*1000)
reply<-receive socket []
putStrLn $ unwords ["Received response : ",unpack reply]))
serverMain :: IO ()
serverMain = withContext 1 (\context-> do
putStrLn "Listening at 5554"
withSocket context Rep $ (\socket-> do
connect socket "tcp://127.0.0.1:5554"
forever $ do
message<-receive socket [] -- this throws an IO Exception
putStrLn $ unwords ["Received request : ",unpack message]
threadDelay (1*1000*1000)
send socket (pack "World") [] ))
main :: IO ()
main = serverMain -- replace with clientMain and it works
Now i really didn't get around to testing all other modes of communication (push/pull, subscribe/publish, pair etc.) and for what i need the python server/haskell client is probably better but i am curious about weather i'm doing something wrong or if any part of my code is broken in any way.
Thanks in advance
You need to make one of the sockets (usually the server) bind, you seem to have them both connecting.
Try changing connect socket "tcp://127.0.0.1:5554" to bind socket "tcp://127.0.0.1:5554" in the serverMain function.

Resources