SSHJ: exec hangs or loses data - sshj

I have used Trilead and Jsch to use as a SSH client for Hg, and now I'm trying it with SSHJ because it seems to provide more modern key support. The sketched code reads as
SSHClient client = createClientAndAuthenticate();
Session session = client.startSession();
Command cmd = session.exec(command); // command usually is "hg -R /path/to/hg/repository serve --stdio"
startThreadToCopyInputStream(cmd.getOutputStream());
startThreadToCopyOutputStream(cmd.getInputStream());
startThreadToCopyOutputStream(cmd.getErrorStream());
cmd.join(); // here it hangs endlessly
The startThreadToCopyInputStream method copies all bytes from the local Hg process to cmd.getOutputStream() and then finishes the input stream. But in contrast to Trilead and JSch the cmd.getInputStream() and cmd.getErrorStream() remain open endlessly.
I've now changed the code to something like:
SSHClient client = createClientAndAuthenticate();
Session session = client.startSession();
Command cmd = session.exec(command); // command usually is "hg -R /path/to/hg/repository serve --stdio"
startThreadToCopyInputStream(cmd.getOutputStream());
startThreadToCopyOutputStream(cmd.getInputStream());
startThreadToCopyOutputStream(cmd.getErrorStream());
waitUntilInputStreamIsClosed();
cmd.close();
This works fine in 90% of the cases, but sometimes Hg complains that the server closed the connection to early. How should I know when to be able to call cmd.close() because the server process has finished?

It looks like this is a bug in SSHJ (see ticket 143). The work-around is to not use cmd.getOutputStream() directly, but instead:
OutputStream cmdOutputStream = new FilterOutputStream(cmd.getOutputStream()) {
#Override
public void close() throws IOException {
try {
super.close();
}
finally {
client.getTransport().write(new SSHPacket(Message.CHANNEL_EOF).putUInt32(session.getRecipient()));
}
}
};
before passing to startThreadToCopyInputStream(cmdOutputStream);.

Related

How can I call Shell Script on remote machine from Browser or Azure Functions

I have a shell file on the remote machine which will perform certain required actions. Can I call this shell from outside of the VM.
Like by using Azure functions or browser itself.
Here is the Snapshot for shell.
According your needs,I suggest you connecting to a remote server using SSH and execute commands.
I'm not sure which language you are using. So,I just offer java sample code for you here.
You could use SSH component JCraft for remote connection and shell commands invocations.
JSch jsch = new JSch();
String command = "/tmp/myscript.sh";
Session session = jsch.getSession(user, host, 22);
session.connect();
Channel channel = session.openChannel("exec");
((ChannelExec)channel).setCommand(command);
channel.setInputStream(null);
((ChannelExec)channel).setErrStream(System.err);
InputStream in = channel.getInputStream();
channel.connect();
byte[] tmp = new byte[1024];
while (true) {
while (in.available() > 0) {
int i = in.read(tmp, 0, 1024);
if (i < 0) {
break;
}
System.out.print(new String(tmp, 0, i));
}
if (channel.isClosed()) {
if (channel.getExitStatus() == 0) {
System.out.println("Command executed successully.");
}
break;
}
}
channel.disconnect();
session.disconnect();
Also,you could refer to this thread How do I run SSH commands on remote system using Java?.
Hope it helps you. Any concern,please feel free to let me kown.

Reading data from StreamSocket only works on desktop, not on phone (Universal Windows App)

Having a weird problem with an UWP app I'm trying to create when creating a socket and reading back data from it. Here's the basic scenario:
_commandSocket = new StreamSocket();
await _commandSocket.ConnectAsync(new HostName("192.168.1.1"), "15740", SocketProtectionLevel.PlainSocket);
_commandOutput = _commandSocket.OutputStream;
_commandInput = _commandSocket.InputStream;
await _commandOutput.WriteAsync(initCommandBuffer);
var lengthBytes = new byte[4].AsBuffer();
await input.ReadAsync(lengthBytes, 4, InputStreamOptions.None);
etc..etc...etc...
I'm basically opening a socket and sending an initialization command to my device. The idea is that it'll reply by first sending me the length of the reply as an uint.
This all works perfectly well when I debug/run the app on my desktop, but if I try to run this on my Window 10 Mobile device (Nokia 930) it seems not a single byte is being read (I've tried reading 1 byte instead of 4). Since app development should be 'Universal' and it's all working perfectly on my desktop I have no idea where to start fixing this.
Opening the connection on the mobile device works, I'm also not getting any errors when sending the initialization command, it just keeps blocking on the await ReadAsync command (as if nothing is being sent back).
I don't have any influence on the device that I'm connecting to, but again since this is all working flawlessly on the desktop and the device doesn't know who's asking I'm pretty clueless.
Cannot confirm my answer for sure, becasue I only tested in on an emulator, but I used stream socket listener.
private async void StartServer()
{
int Port = 80;
try
{
//Bound to port 80
await listener.BindServiceNameAsync(Port.ToString());
}
catch (System.Runtime.InteropServices.COMException)
{
//Port 80 for communication is blocked.
}
listener.ConnectionReceived += async (s, e) =>
{
using (IInputStream input = e.Socket.InputStream)
{
var buffer = new Windows.Storage.Streams.Buffer(200);
await input.ReadAsync(buffer, buffer.Capacity, InputStreamOptions.Partial);
//reads incoming stream
string data = Encoding.ASCII.GetString(buffer.ToArray(), 0, (int) buffer.Length);
}
using (IOutputStream output = e.Socket.OutputStream)
{
using (Stream response = output.AsStreamForWrite())
{
response.Write(Encoding.ASCII.GetBytes("Response."), 0, 1);
}
}
};
}

Redis Connections May Not be Closing with c#

I'm connecting to Azure Redis and they show me the number of open connections to my redis server. I've got the following c# code that encloses all my Redis sets and gets. Should this be leaking connections?
using (var connectionMultiplexer = ConnectionMultiplexer.Connect(connectionString))
{
lock (Locker)
{
redis = connectionMultiplexer.GetDatabase();
}
var o = CacheSerializer.Deserialize<T>(redis.StringGet(cacheKeyName));
if (o != null)
{
return o;
}
lock (Locker)
{
// get lock but release if it takes more than 60 seconds to complete to avoid deadlock if this app crashes before release
//using (redis.AcquireLock(cacheKeyName + "-lock", TimeSpan.FromSeconds(60)))
var lockKey = cacheKeyName + "-lock";
if (redis.LockTake(lockKey, Environment.MachineName, TimeSpan.FromSeconds(10)))
{
try
{
o = CacheSerializer.Deserialize<T>(redis.StringGet(cacheKeyName));
if (o == null)
{
o = func();
redis.StringSet(cacheKeyName, CacheSerializer.Serialize(o),
TimeSpan.FromSeconds(cacheTimeOutSeconds));
}
redis.LockRelease(lockKey, Environment.MachineName);
return o;
}
finally
{
redis.LockRelease(lockKey, Environment.MachineName);
}
}
return o;
}
}
}
You can keep connectionMultiplexer in a static variable and not create it for every get/set. That will keep one connection to Redis always opening and proceed your operations faster.
Update:
Please, have a look at StackExchange.Redis basic usage:
https://github.com/StackExchange/StackExchange.Redis/blob/master/Docs/Basics.md
"Note that ConnectionMultiplexer implements IDisposable and can be disposed when no longer required, but I am deliberately not showing using statement usage, because it is exceptionally rare that you would want to use a ConnectionMultiplexer briefly, as the idea is to re-use this object."
It works nice for me, keeping single connection to Azure Redis (sometimes, create 2 connections, but this by design). Hope it will help you.
I was suggesting try using Close (or CloseAsync) method explicitly. In a test setting you may be using different connections for different test cases and not want to share a single multiplexer. A search for public code using Redis client shows a pattern of Close followed by Dispose calls.
Noting in the XML method documentation of Redis client that close method is described as doing more:
//
// Summary:
// Close all connections and release all resources associated with this object
//
// Parameters:
// allowCommandsToComplete:
// Whether to allow all in-queue commands to complete first.
public void Close(bool allowCommandsToComplete = true);
//
// Summary:
// Close all connections and release all resources associated with this object
//
// Parameters:
// allowCommandsToComplete:
// Whether to allow all in-queue commands to complete first.
[AsyncStateMachine(typeof(<CloseAsync>d__183))]
public Task CloseAsync(bool allowCommandsToComplete = true);
...
//
// Summary:
// Release all resources associated with this object
public void Dispose();
And then I looked up the code for the client, found it here:
https://github.com/StackExchange/StackExchange.Redis/blob/master/src/StackExchange.Redis/ConnectionMultiplexer.cs
And we can see Dispose method calling Close (not the usual override-able protected Dispose(bool)), further more with the wait for connections to close set to true. It appears to be an atypical dispose pattern implementation in that by trying all the closure and waiting on them it is chancing to run into exception while Dispose method contract is supposed to never throw one.

RabbitMQ messages disappearing mysteriously

I have a node worker waiting for RabbitMQ messages. Today I made some changes to it, and now when I connect the dev instance, everything works—it appears to connect—but then the consume functions don't fire. Messages still disappear though like they've been consumed.
Rabbit.queue('consume', function (q) {
q.bind('consume');
// This fires
console.log('Listening to queue...');
q.subscribe(function (message) {
// This doesn't fire but message gets removed from queue
console.log(message);
init(message);
});
});
I had this same behavior using the RabbitMQ client in a C# console application:
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
//Various RabbitMQ client magic that doesn't work
}
}
Console.WriteLine(" Press [enter] to exit.");
Console.ReadLine();
Note those last two lines. They keep the console open and running. Otherwise the app would have instantly closed. The problem was I put those two lines of code outside the using statement. Once you step passed the closing bracket of the using statement, the "used" object is disposed. So I was closing my connection and channel almost immediately after opening them. The solution was to put the Console lines INSIDE the using brackets:
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
//Various RabbitMQ client magic that astounds and amazes
Console.WriteLine(" Press [enter] to exit.");
Console.ReadLine();
}
}
Obvious and trivial once you see it, but I wasted an hour on it. I don't know if this is analogous to anything you are doing in NodeJS, but perhaps it will help some other frustrated coder.

ReceiveFromAsync leaking SocketAsyncEventArgs?

I have a client application that receives video stream from a server via UDP or TCP socket.
Originally, when it was written using .NET 2.0 the code was using BeginReceive/EndReceive and IAsyncResult.
The client displays each video in it's own window and also using it's own thread for communicating with the server.
However, since the client is supposed to be up for a long period of time, and there might be 64 video streams simultaneously, there is a "memory leak" of IAsyncResult objects that are allocated each time the data receive callback is called.
This causes the application eventually to run out of memory, because the GC can't handle releasing of the blocks in time. I verified this using VS 2010 Performance Analyzer.
So I modified the code to use SocketAsyncEventArgs and ReceiveFromAsync (UDP case).
However, I still see a growth in memory blocks at:
System.Net.Sockets.Socket.ReceiveFromAsync(class System.Net.Sockets.SocketAsyncEventArgs)
I've read all the samples and posts about implementing the code, and still no solution.
Here's how my code looks like:
// class data members
private byte[] m_Buffer = new byte[UInt16.MaxValue];
private SocketAsyncEventArgs m_ReadEventArgs = null;
private IPEndPoint m_EndPoint; // local endpoint from the caller
Initializing:
m_Socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
m_Socket.Bind(m_EndPoint);
m_Socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, MAX_SOCKET_RECV_BUFFER);
//
// initalize the socket event args structure.
//
m_ReadEventArgs = new SocketAsyncEventArgs();
m_ReadEventArgs.Completed += new EventHandler<SocketAsyncEventArgs>(readEventArgs_Completed);
m_ReadEventArgs.SetBuffer(m_Buffer, 0, m_Buffer.Length);
m_ReadEventArgs.RemoteEndPoint = new IPEndPoint(IPAddress.Any, 0);
m_ReadEventArgs.AcceptSocket = m_Socket;
Starting the read process:
bool waitForEvent = m_Socket.ReceiveFromAsync(m_ReadEventArgs);
if (!waitForEvent)
{
readEventArgs_Completed(this, m_ReadEventArgs);
}
Read completion handler:
private void readEventArgs_Completed(object sender, SocketAsyncEventArgs e)
{
if (e.BytesTransferred == 0 || e.SocketError != SocketError.Success)
{
//
// we got error on the socket or connection was closed
//
Close();
return;
}
try
{
// try to process a new video frame if enough data was read
base.ProcessPacket(m_Buffer, e.Offset, e.BytesTransferred);
}
catch (Exception ex)
{
// log and error
}
bool willRaiseEvent = m_Socket.ReceiveFromAsync(e);
if (!willRaiseEvent)
{
readEventArgs_Completed(this, e);
}
}
Basically the code works fine and I see the video streams perfectly, but this leak is a real pain.
Did I miss anything???
Many thanks!!!
Instead of recursively calling readEventArgs_Completed after !willRaiseEvent use goto to return to the top of the method. I noticed I was slowly chewing up stack space when I had a pattern similar to yours.

Resources