FTP issues with FTP inbound channel adapter - spring-integration

Our project we are using ftp:inbound-channel-adapter to poll files from the FTP server.it working fine.But in between the polling is not working.when i see the FTP server logs i see "425 Can't open data connection." now when i restart or stop and start the ftp:inbound-channel-adapter again its polling properly.This issue is repeatedly occurring to solve i need to stop/start the ftp:inbound-channel-adapter.ftp:inbound-channel-adapter is running in linux OS.
Am using spring-integration 3 just to more clear i have included the xsd info
(spring-integration-3.0.xsd,spring-integration-ftp-3.0.xsd)
is there any specific client mode i need to set for FTP i.e Active(local/remote) /Passive(local/remote) etc?
below my ftp:inbound-channel-adapter configuration
<bean id="ftpClientFactory" class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
<property name="host" value="abcd.com"/>
<property name="port" value="21"/>
<property name="username" value="userid"/>
<property name="password" value="password"/>
</bean>
<int-ftp:inbound-channel-adapter id="ftpInbound"
channel="ftpChannel"
session-factory="ftpClientFactory"
auto-create-local-directory="true"
delete-remote-files="true"
remote-directory="/"
local-filename-generator-expression="new java.text.SimpleDateFormat('yyyy-MM-dd-hhmmssSSS').format(new java.util.Date()) + '.'+ #this"
local-directory="${ftp.sync.folder}"
remote-file-separator="/">
</int-ftp:inbound-channel-adapter>
so not sure i can do something in the FTP server.but i like to see is there any option in ftp:inbound-channel-adapter or any thing you guy suggest so that whenever FTP server throws "425 Can't open data connection." instead of manually stop/start the ftp:inbound-channel-adapter is there any option or automatic way to make this work.Thanks
Added info on spring integration version and ftp session factory.

There are 2 ways to connect to the FTP server active and passive mode.
ActiveMode : where FTP server has to made Data Connection with the port mentioned by the Client
(firewall issues if port is blocked by fire wall and you will get 425 Data Connection error)
Passivemode : Where client has to made Data connection with the port mentioned by the FTP server.
(no fairwall issues in the client side.Also we can configure the passvieports in FTP server and made these ports not block by FTP servers firewall.)
If you not specify any clientmode in ftpsessionfactory it defaults to the Active mode i.e clientMode=0.
So i have firewall issue which causes 425 data connection issue.After i OFF the firewall its worked well.So now i changed my FTPsessionfactory to use Passivemode so FTP server never cares about clients Firewall
<bean id="ftpClientFactory" class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
<property name="host" value="abcd.com"/>
<property name="port" value="21"/>
<property name="username" value="userid"/>
<property name="password" value="password"/>
<!-- 2 passive mode -->
<property name="clientMode" value="2"/>
</bean>
This way never cares about client's firewall.
very good post about FTP http://slacksite.com/other/ftp.html

Related

Remove jetty version in embedded Jetty in ActiveMQ

We have not seen anywhere in the documentation of ActiveMQ how to hide the embedded jetty version. This is marked as a security thread by our penetration testers when we are using a web sockets transport on port 80. We have been playing around with the configuration file jetty.xml and the parameters, but no success. It has been addressed for other projects (see https://issues.apache.org/jira/browse/HADOOP-13414) by a patch in the code,
So far we have been trying to change the configuration in jetty.xml.
After searching for jetty properties, this is how we configured the property for the property:
<bean id="httpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
<property name="sendServerVersion" value="false">
</property>
</bean>
However, this has no effect in the exposing of the version. We tried further with a connection factory, but this also had no effect:
<bean id="invokeConnectors" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="Server" />
<property name="targetMethod" value="setConnectors" />
<property name="arguments">
<list>
<bean id="Connector" class="org.eclipse.jetty.server.ServerConnector">
<constructor-arg ref="Server" />
<constructor-arg>
<list>
<bean id="httpConnectionFactory" class="org.eclipse.jetty.server.HttpConnectionFactory">
<constructor-arg ref="httpConfig"/>
</bean>
</list>
</constructor-arg>
<!-- see the jettyPort bean -->
<property name="host" value="#{systemProperties['jetty.host']}" />
<property name="port" value="#{systemProperties['jetty.port']}" />
</bean>
</list>
</property>
</bean>
Are we on the right track, or does it need to be addressed by the codebase of ActiveMQ?
This is how we show the version:
#nmap -sV -p80 localhost
Starting Nmap 7.70 ( https://nmap.org ) at 2018-04-23 18:16 CEST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000098s latency).
PORT STATE SERVICE VERSION
80/tcp open http Jetty 9.2.22.v20170606
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 11.34 seconds
Edit: we received an answer in the ActiveMQ JIRA that indicates that it will need some code modification in the ActiveMQ project. We will update this question if the current state changes. Current ActiveMQ is 5.15.3.
Quoting Tim Bain from the linked issue:
Since org.apache.activemq.transport.discovery.http.EmbeddedJettyServer.start() creates its Server by calling new (no Spring injection), and the code of that method doesn't provide any means to specify an HttpConfiguration, the ability to change this behavior requires a code change. There is similar code in org.apache.activemq.transport.WebTransportServerSupport, also without any ability to specify an HttpConfiguration that would disable the sending of the version number.
The use of ...
org.eclipse.jetty.server.HttpConfiguration.setSendServerVersion(false);
... is the correct setting.
However, you cannot change this AFTER the server has been started.
The startup of the server itself, is the only place you can make this change.
When the Connectors are created/initialized is the only place this setting is used.
This is because that setting is propagated into many components during startup (the Http/1 Generator, the Http/2 Generator, the ErrorPage generator, the WebSocket Upgrade process, etc)

connect Logstash 1.5.0 with log4j of several servers

I'm trying to connect logstash (Version 1.5.0) to get logs of services (that run on apache-tomcat). These logs are log4j.
I use this config for logstash:
input {
log4j {
mode => server
host => localhost
port => 4560
type => "log4j"
}
}...
and in my service' log4j.xml I've set my SocketAppender:
<appender name="OHADS" class="org.apache.log4j.net.SocketAppender">
<param name="port" value="4560" />
<param name="remoteHost" value="localhost" />
</appender>
It works fine.
The questions:
I want logstash to collect logs not from my 'localhost', but from other tomcats, from other machines, as well. How can I do that? when I tried to put in the "host" (in logstash config) something other than localhost (or the IP of the local machine), i got error on startup:
"Cannot assign requested address - bind - Cannot assign requested
address".
How can I connect it to several IPs simultanously?
any ideas?

How to start hazelcast client from console

How do I run a Hazelcast client from the console?
I have this config in xml:
<hazelcast-client xsi:schemaLocation="http://www.hazelcast.com/schema/client-config hazelcast-client-config-3.3.xsd"
xmlns="http://www.hazelcast.com/schema/client-config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<network>
<port auto-increment="true" port-count="100">5701</port>
<outbound-ports>
<!--
Allowed port range when connecting to other nodes.
0 or * means use system provided port.
-->
<ports>0</ports>
</outbound-ports>
<join>
<multicast enabled="true">
<multicast-group>224.2.2.3</multicast-group>
<multicast-port>54327</multicast-port>
</multicast>
</join>
</network>
</hazelcast-client>
and I need a console like this when I run run.sh or run.bat, but I want it to be a client not a member.
You can use the clientConsole.bat given by Hazelcast to connect to any Hazelcast server and get a console for that. Using the console you can query any existing map or queue in your Hazelcast server.
I use hazelcast-3.8.1.
In the demo directory you can find clientConsole.bat/clientConsole.sh.
To connect to an existing server you need to provide the details in a "hazelcast-client.xml" file. Here is the content of "hazelcast-client.xml" file:
<hazelcast-client xmlns="http://www.hazelcast.com/schema/client-config">
<group>
<name>[GROUP_NAME]</name>
<password>[GROUP_PASSWORD]</password>
</group>
<network>
<cluster-members>
<address>[HAZELCAST_SERVER_HOST_NAME_OR_IP]</address>
</cluster-members>
</network>
</hazelcast-client>
Above [HAZELCAST_SERVER_HOST_NAME_OR_IP] is the ip or hostname of hazelcast server which you want to connect and get client console.
[GROUP_NAME] and [GROUP_PASSWORD] should be same what is mentioned in "hazelcast.xml" file on Hazelcast server.

jBoss7 no remote access

i setup a jboss7 server including the standalon.xml for remote access on a debian machine.
But I cant get external remote access.
This is my interface part of the standalone.xml:
<interfaces>
<interface name="management">
<inet-address value="192.168.3.182"/>
</interface>
<interface name="public">
<inet-address value="192.168.3.182"/>
</interface>
</interfaces>
I also tried:
<interfaces>
<interface name="management">
<inet-address value="127.0.0.1"/>
</interface>
<interface name="public">
<any-ipv4-address/>
</interface>
</interfaces>
I tried a lot of possibilities to setting up the standalone.xml but I must forgot something.
I see the jboss7 homepage when I type the ip address in my browser but Im note able to connect with my java test client on the remote port(4447).
The test client works if the client runs local on the machine.
Do you have an idea?
Edit: telnet 192.168.3.182 4447 gives me the ip back with a blink cursor, so i guess waiting for response additional i see a log entry in my jboss server log -> ) JBREM000200: Remote connection failed: java.io.IOException: Received an invalid message length of 1728579688
I would check if the port is listening with the following command under linux "netstat -tulpn | grep -i 4447"
Good luck,
that "Received an invalid message length of 1728579688" error indicates that jboss is listening on the port but that the connector is not using the right kind of protocol.
of course, if you are connecting with telnet then you would expect to see that error since telnet is not a JMX client.
JBoss uses JBoss Remoting for JMX connections. So if you are using an external client, you need to make sure the JBoss Remoting libraries are available to your client.
This is a good description of the issue using VisualVM as the client:
http://blog.akquinet.de/2012/11/01/connecting-visualvm-with-a-remote-jboss-as-7-eap6-jvm-process/

How to force open webrole on port 80 in azure dev fabric(azure sdk 1.5)

Recently, I upgraded my dev env from ver1.1 to ver 1.5 of the Azure SDK (I know - am a little too late :))
What I noticed was that my webrole was opening up at the port 81 always. Is there a way for me to force open the azure webrole on port 80 in my dev env?
The reason I need this is :
I have a browser extension which connects to my webrole - and it expects the webrole to be on port 80; Until now, testing on the dev env was easy - I just need to do an etc/host redirection and my regular browser plugin would connect to my dev fabric.
On my website, I also provide open-auth authentication from google/facebook. I would not be able to test that on my dev env if I access it as www.mywebsite.com:81/ instead of www.mywebsite.com
Anyone has a pointer?
Kapil
Just make sure port 80 is available. The compute emulator takes the port you asked for or the first available port above that.
If you are using emulator - this will be an issues, cause it opens first free port.
If you will host your role on azure, you can configure your port in config file.
For example , in this configuration:
<WebRole name="TestApplication1">
<Sites>
<Site name="Web">
<Bindings>
<Binding name="Endpoint1" endpointName="Endpoint1" />
</Bindings>
</Site>
</Sites>
<Endpoints>
<InputEndpoint name="Endpoint1" protocol="http" port="80" />
</Endpoints>
<Imports>
<Import moduleName="Diagnostics" />
</Imports>
<ConfigurationSettings>
<Setting name="MyCustomSettingInAzure" />
</ConfigurationSettings>
You can see that for endpoint1 we configured port to use with number 80.
Update:
I've searched a bit, and found this post:
http://social.msdn.microsoft.com/Forums/en/windowsazuredevelopment/thread/ae2df7e0-5005-4bcd-8b69-bb53323eb589
There are some ideas which i believe can help you. It will require adding some commands to your pre-build actions.
One more update
Please run command : Netstat -a -n -o
This will show who is using port 80
If you can't force it to use port 80 in stead of port 81 in your development environment even if it the port is available, then you could install Fiddler2 and try to add the following Fiddler rule at the bottom of OnBeforeRequest():
// Windows Azure force socket 80
if (oSession.host == "app.dev.com:81") { oSession.host = "127.0.0.1:80"; }
if (oSession.host == "127.0.0.1:81") { oSession.host = "127.0.0.1:80"; }
if (oSession.url == "127.0.0.1:81") { oSession.url = "127.0.0.1:80"; }
I haven't tested it, but I think that could give you a pointer of how to do it.
Okay - the issue has been solved. Please see this thread for more details.
http://social.msdn.microsoft.com/Forums/en-US/windowsazuredevelopment/thread/5447f16e-2eed-4170-9771-17c7c9e7e570?prof=required
Basically I pointed my default site in the IIS to point to a different port and that worked
Kapil

Resources