Mule ESB: Retrieving email messages from Gmail using IMAP connector - gmail

I am new to Mule and im trying to create a Mule configuration that pulls sent emails from a GMail account via imap and pushes them to a php script that processes and stores them in a custom CRM that i've built.
For starters, i'm just trying to get the inbox emails dumped into a text files and i plan to work from there.
As new messages are received by the mailbox, Mule should pick up the new messages and process them automatically.
The Mule config looks like this so far:
<imaps:connector name="IMAP" mailboxFolder="INBOX" validateConnections="false" doc:name="IMAP" />
<flow name="flows1Flow1" doc:name="flows1Flow1">
<imaps:inbound-endpoint host="imap.gmail.com" port="993" user="[[username]]%40gmail.com" password="[[password]]" connector-ref="IMAP" doc:name="IMAP"/>
<file:outbound-endpoint path="D:\mailflow" outputPattern="msg_#[function:date].txt" doc:name="File"/>
</flow>
The program runs and gets to this point:
INFO 2012-01-12 13:51:06,606 [main] org.mule.DefaultMuleContext:
**********************************************************************
* Application: mailflow *
* OS encoding: Cp1252, Mule encoding: UTF-8 *
* *
* Agents Running: *
* JMX Agent *
**********************************************************************
INFO 2012-01-12 13:51:06,606 [main] org.mule.module.launcher.DeploymentService:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Started app 'mailflow' +
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
And then just sits there indefinitely, doing nothing?!
There is documentation suggesting that because i'm using IMAPS i need to add a TLS client and TLS key store to the imaps connector. I'm not sure what these are or how to use them though and the documentation is highly specialized and hard to understand.
I'm also not sure that that's what the problem is in the first place as the app doesn't crash at any point.
Has anyone had success creating an imap flow with GMail?
Please Help?!

Just create the connector like this:
<imaps:connector name="IMAP">
<imaps:tls-client/>
<imaps:tls-trust-store/>
</imaps:connector>
And that should do the trick.
Also, I'd remove the "#gmail" from the user's definition, since it's not necessary.
Bye!
German

just put in a * and you wont see the error and will still work fine.
<imaps:tls-client path="*" storePassword="*"/>
<imaps:tls-trust-store path="*" storePassword="*"/>

You have to change imap:connector and imap:inbound-endpoint to imaps:connector and imaps:inbound-endpoint.
It is working fine for me. I have the same issued and now it is fixed with this little change.

Only non-deleted and unread messages are dispatched as messages (RetrieveMessageReceiver.java:148 and 149)
if (!messages[i].getFlags().contains(Flags.Flag.DELETED)
&& !messages[i].getFlags().contains(Flags.Flag.SEEN))
If the folder is big, it will take some time (potentially even hours) to get to the point to process unread messages.

Settings for the IMAP
Use the app password
https://security.google.com/settings/security/apppasswords
and finally use the correct path for the mail to be saved.

Related

Connection configuration loops - Prosys OPC UA Client

I'm using sample codes from documentation and I'm trying to connect to server using Prosys OPC UA Client. I have tried opcua-commander and integration objects opc ua client and it looks like server works just fine.
Here's what is happening:
After entering endpointUrl, client adds to url -- urn:NodeOPCUA-Server-default.
Client asks to specify security settings.
Client asks to choose server - only 1 option and it's urn:NodeOPCUA-Server-default.
And it goes back to step 2 and 3 over and over.
If I just minimize prosys client without closing configuration after some time I get this info in terminal:
Server: closing SESSION new ProsysOpcUaClient Session15 because of timeout = 300000 has expired without a keep alive
\x1B[46mchannel = \x1B[49m ::ffff:10.10.13.2 port = 51824
I have tried this project and it works -> node-opcua-htmlpanel. What's missing in sample code then?
After opening debugger I have noticed that each Time I select security settings and hit OK, server_publish_engine reports:
server_publish_engine:179 Cencelling pending PublishRequest with statusCode BadSecureChannelClosed (0x80860000) length = 0
This is due to a specific interoperability issue that was introduced in node-opcua#0.2.2. this will be fixed in next version of node-opcua. The resolution can be tracked here https://github.com/node-opcua/node-opcua/issues/464
The issue has been handled at the Prosys OPC Forum:
The error happens because the server sends different
EndpointDescriptions in GetEndpointsResponse and
CreateSessionResponse.
In GetEndpoints, the returned EndpointDescriptions contain
TransportProfileUri=http://opcfoundation.org/UA-Profile/Transport/uatcp-uasc-uabinary.
In CreateSessionResponse, the corresponding TransportProfileUri is
empty.
In principle, the server application is not working according to
specification. The part 4 of the OPC UA specification states that “The
Server shall return a set of EndpointDescriptions available for the
serverUri specified in the request. … The Client shall verify this
list with the list from a DiscoveryEndpoint if it used a
DiscoveryEndpoint to fetch the EndpointDescriptions. It is recommended
that Servers only include the server.applicationUri, endpointUrl,
securityMode, securityPolicyUri, userIdentityTokens,
transportProfileUri and securityLevel with all other parameters set to
null. Only the recommended parameters shall be verified by the
client.”

JMS Message body is null when publishing with jms:publish-subscribe-channel

I am trying to use jms:publish-subscribe-channel to pub/sub on a single ActiveMQ topic. I am able to receive messages from ActiveMQ on the channel just fine, however when I publish to the channel the message body is null (when received by another application listening on the ActiveMQ topic). I was able to recreate the problem using spring-integration-samples->basic->jms. I modified outboundChannelAdapter.xml to use jms:publish-subscribe-channel instead of jms:outbound-channel-adapter. Is there another step needed in order to publish a simple string message? Here's my code change to outboundChannelAdapter.xml:
<stream:stdin-channel-adapter id="stdin" channel="stdinToJmsoutChannel"/>
<jms:publish-subscribe-channel id="stdinToJmsoutChannel" topic="requestTopic" />
<stream:stdout-channel-adapter id="stdout" channel="stdinToJmsoutChannel" append-newline="true"/>
I am not sure what you mean by "the message body is null".
I just made the exact same change to the sample and it worked fine for me...
Please type something and hit <enter>
foo
foo
I had to add -Dorg.apache.activemq.SERIALIZABLE_PACKAGES=* to the command line because activemq needs whitelisting for classes (the whole message is serialized in jms-backed channels).

IBM WebSphere / MQTT.js : Convert MQHRF2 to MQSTR format

I have a client in NodeJS using the MQTT module (mqtt.js) to communicate with a topic of WebSphere MQ, redirected to a queue : I need the result to be on a queue so I use a topic between the source and the destination (since MQTT protocol communicate only with topics).
So we have :
NodeJS (source) -> Topic of Websphere MQ (intermediate) -> Queue of Websphere MQ (destination)
The problem is that the received messages on the queue of WebsphereMQ are MQHRF2 format but I need MQSTR.
I did not find any property in mqtt.js side to specify the format.
I supposed we can force it with IBM Websphere MQ, but how ? Can I create a channel or anything for this use, that's to say convert the receiving format ?
Thanks in advance for your help !
Configuration :
MQTT.js v3
WebSphere MQ v7.5
Constrains :
WebSphere MQ must be v7.5 (so I can't upgrade to v8 to use mqlight)
Better stay with the module MQTT.js
It works correctly when I publish directly on a topic via the WebSphere MQ explorer. However, it fails when I publish from the NodeJS application.
I tried both methods listing in the comments but I found a weird thing : The result is different according to if I use MQExplorer (what I always used) or the MQ console :
On the MQ Explorer :
On the MQ Console, with the command : amqsbcg QUEUE_MQTT_VERIF MQTTVerification :
As you can see, in the console, the format field is empty, whereas on the MQExplorer it is MQHRF2. The messages are also different.
So the result on the MQConsole is right whereas the MQExplorer still shows a wrong result after the changes.
Note that the result is right only if changes has been done on both :
Subscription : ALTER SUB(APPLE.PRICES) PSPROP(NONE)
Queue : ALTER QLOCAL(PRICES) PROPCTL(NONE)
Maybe the source of the problem is just the display on MQExplorer software ?
EDIT:
The difference between MQExplorer and MQConsole was due to the parameter in MQ Explorer : Window -> Preferences -> WebSphereMQ Explorer -> Message properties -> Uncheck "Display message properties".
When redirecting publications made on a topic to a queue you probably have some definitions like this:-
DEFINE QLOCAL(PRICES)
DEFINE TOPIC(APPLES) TOPICSTR('Price/Fruit/Apples')
DEFINE SUB(APPLE.PRICES) TOPICOBJ(APPLES) DESTCLAS(PROVIDED) DEST(PRICES)
which redirects publications made on the topic string 'Price/Fruit/Apples' to the queue PRICES.
What you will find with such a set up however, is that the topic string is added to the message by the queue manager and thus adds an MQRFH2 header onto the front of your published message.
Message Descriptor (MQMD)
Report :00000000
Message Type :8 (Datagram)
Format :'MQHRF2 '
Priority :0
Persistence :0 (Not Persistent)
Message Id :414D51204D51473120202020202020202D77835720003702
'AMQ MQG1 -w.W .7.'
ReplyToQ :' '
ReplyToQMgr :'MQG1 '
[ 102 bytes] Message Content
<mqps>
<Top>
Price/Fruit/Apples
</Top>
</mqps>
Apples are $2/kilo
To remove this without changing the application reading from the queue, make this alteration to the subscription.
ALTER SUB(APPLE.PRICES) PSPROP(NONE)
This stops the queue manager even putting the topic string into the message in the first place.
From your MQ Explorer screenshot, it can be seen that the MQRFH2 is still present after the SUB is changed to PSPROP(NONE) because there is another property present - mqtt.clientid. Since there are other items in the MQRFH2 as well as the topic string, then the SUB change will not remove those - only the topic string added by the queue manager. In that case, you should try the following.
If alternatively you want to have the topic string there sometimes and only remove it for applications that don't want to see it, you can make a similar change to the queue, which can be over-ridden in application code to force the properties to be delivered to the application, but otherwise they won't be. This would also allow you to read any existing messages that had already been published (the change to the SUB isn't retrospectively applied to messages that are already on the queue).
ALTER QLOCAL(PRICES) PROPCTL(NONE)
This will mean that applications that don't specifically request MQGMO_FORCE_RFH2 will see the message without the MQRFH2. The amqsbcg sample is one such application.
If an application codes MQGMO_FORCE_RFH2 it will still be able to see the properties in an MQRFH2 header because it has explicitly asked for that. You cannot stop that by changing the queue.
Your problem is only now with the way MQ Explorer chooses to display the message. There is a parameter in MQ Explorer : Window -> Preferences -> WebSphereMQ Explorer -> Message properties -> Uncheck "Display message properties" which will stop it forcing them to be an MQRFH2, and then you'll be all good.

OpenDeploy Base Server hangs up with starting deployment

I'm managing Autonomy/Interwoven OpenDeploy Base Server.
OS:RHEL5.3
TSVer.:TeamSite6.7.2
ODVer:OpenDeploy7.1
OpenDeploy Base Server process sometimes hangs up when starting deployment.
following message are out in [od.log]
* glibc detected /var/od-home/OpenDeployNG/jre/bin/java: double free or corruption (!prev): 0xb01721d8 **
does anyone know about this problem?
Teamsite 6.7.1+ doesn't support Vintela (VAS) for users or groups on Linux.
I've seen same error in the iwtrace.log, the fix is to make sure that VAS (Vintela) is not used for authentication of users and groups.

IIS Application pool identity

I am attempting to obtain a data feed from yahoo finance. I am doing this with the following code:
System.Net.WebRequest request = System.Net.WebRequest.Create(http://download.finance.yahoo.com/download/quotes.csv?format=sl&ext=.csv&symbols=^ftse,^ftmc,^ftas,^ftt1x,^dJA);
request.UseDefaultCredentials = true;
// set properties of the request
using (System.Net.WebResponse response = request.GetResponse())
{
using (System.IO.StreamReader reader = new System.IO.StreamReader(response.GetResponseStream()))
{
return reader.ReadToEnd();
}
}
I have placed this code into a console application and, using Console.WriteLine on the output I receive the information I require. I have used the 'Run as..' command to execute this using a specific domain account.
When I use this code from within a Page load I receive the following error message "No connection could be made because the target machine actively refused it 76.13.114.90:80".
This seems to suggest that the call is reaching yahoo (is this true?) and that there is something missing.
This would suggest there is an identity difference in the calls between the console application and application pool.
Environment is: Windows Server 2003, IIS 6.0, .net 4.0
"Target machine actively refused it" indicates that the TCP connection itself is not succeeding. This could be due to the fact that the Proxy settings when run under IIS are not the same as those that apply when you run in the console.
You can fix this by setting a WebProxy on your request, that points to the proxy server being used in the environment.
Yes, an active refusal is indication that the target machine is receiving the request and the information in the headers is either incorrect or insufficient to process the request. It is entirely possible that if you had to run this call using a "run as" command in console that the application pool's identity user does not have the appropriate permission or username. You can attempt to change the identity user to this specific domain account to see if that alleviates the problem, but you may have to isolate this particular function into its own application pool in order to protect the rest of the website from having this specification.

Resources