I'm struggling to find a good name for a configurable parameter in a testbench.
The parameter is used to control how many downstream requests can be queued until we stall the upstream module.
Any ideas?
How about clearanceNum ?
Clearance is mean official authorization for something to proceed ...
Related
I am aware that there is a probe available at /hazelcast/health/ready at port 5701.
However I need to do this progamatically through code as I am using embedded hazelcast deployed on a kubernetes cluster and all communication should pass through the main application (that means hazelcast cannot expose that endpoint, using http requests through localhost would not suffice). I tried looking into the documentation but I found no help in this.
Only thing that I found is to use instance.getServer().getPartitionService().isLocalMemberSafe() but I got no evidence that this is effectively the same as checking the readiness probe.
Any help would be appreciated, thanks!
The exact logic for /ready endpoint is:
node.isRunning() && node.getNodeExtension().isStartCompleted()
I guess you can't use exactly the same from the code, but fairly good approximations are:
instance.getLifecycleService().isRunning() (the only difference is that it won't wait with being ready for joining other members)
instance.getPartitionService().isClusterSafe() (the difference is that it will wait for all the Hazelcast migration to finish)
You can use any of them. If you want to be really sure that Hazelcast member can receive the traffic when its ready, then the second option is totally safe.
In my spring integration application i have several stored-proc-outbound-gateway, i would to log how much time each call is taking, any help would be appreciated.
I would ideally like to be able to enable/disable logging for the parameters used, time taken and total rows retrieved (returning-resultset) to monitor and performance tuning purpose.
Thanks
You can add a ChannelInterceptor (subclass of ChannelInterceptorAdapter) to the request channel which will give you raw timing (preSend/postSend), but the time will include any processing downstream of the gateway (on direct channels).
Since you want to examine the results too, you could start a timer (e.g. Spring StopWatch) in the interceptor (preSend) on the request channel and stop the timer in an interceptor on the reply channel). If you use the same interceptor bean you can store the timer in a ThreadLocal.
You can turn on/off collection using a boolean property on the interceptor.
Alternatively, you can add a custom advice to the gateway.
EDIT
The advice is probably the best approach because with a ThreadLocal you will need to add code to the first interceptor to handle failures and clean up. With an around advice, the timer would just be a local method variable.
I've a couple of questions regarding the (excellent) CouchDB .NET client MyCouch:
Is there some built-in retry policy in case of "transient" failure (like the server responding 503)?
Should instances of MyCouchClient or MyCouchStore be cached to be reused? Right now I'm creating one for each incoming request, but I'm wondering if that incurs a performance penalty.
I would like to customize the configuration of Json.NET as used by MyCouch, like adding a new StringEnumConverter { CamelCaseText = true } to the list of Converters. Is there a way to achieve that through the API?
Thanks
1) There's no magic in the MyCouchClient, it's just simple requests and responses. The MyCouchStore how-ever, I would gladly accept pull request to have options for retries or e.g. auto-batching queries.
2) Here are some links to information that would help you to decide on per request or per application.
Is async HttpClient from .Net 4.5 a bad choice for intensive load applications?
http://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.defaultconnectionlimit.aspx
So doing one per application would probably need a reconf of the connection limit.
I have this centralized in my IoC-config, and by default I'm not doing per application. The first "connection" can take a bit longer, but the second has been measured down to milliseconds against Cloudant by other users so that should in general not be an issue.
3)
You can configure the serializer by providing a custom MyCouchClientBootstrapper and providing a custom implementation of: https://github.com/danielwertheim/mycouch/blob/master/source/projects/MyCouch.Net45/MyCouchClientBootstrapper.cs#L170
And you also have to extend this guy: https://github.com/danielwertheim/mycouch/blob/master/source/projects/MyCouch.Net45/Serialization/SerializationConfiguration.cs#L9
Feel free to suggest changes that makes this process simpler for you.
I read through the official documentation and the official whitepaper, but I couldn't find a satisfying answer to how Thrift handles failures in the following scenario:
Say you have a client sending a method call to a server to insert an entry in some data structure residing in that server (it doesn't really matter what it is). Suppose the server has processed the call and inserted the entry but the client couldn't receive a response due to a network failure. In such a case, how should the client handle this? A simple retry of sending the call would possibly result in a duplicate entry being inserted. Does the Thrift library persist the response somewhere so that it can resend to the client when it is back online? Or is it the application's responsibility to do so?
Would appreciate it if someone could point out the details of how it works, besides directing to its source code.
The question is an interesting one, but it is by no means limited to Thrift. A better name would be
Handling failures in asynchronous or remote calls in general
because that's in essence, what it is. Altough in the specific case of an RPC-style API like, for example, a Thrift service, the client blocks and it seems to be an synchronous call, it really isn't that way.
The whole problem can be rephrased to the more general question about
Designing robust distributed systems
So what is the main problem, that we have to deal with? We have to assume that every call we do may fail. In particular, it can fail in three ways:
request died
request sent, server processing successful, response died
request sent, server processing failed, response died
In some cases, this is not a big deal, regardless of the exact case we have. If the client just wants to retrieve some values, he can simply re-query and will get some results eventually if he tries often enough.
In other cases, especially when the client modifies data on the server, it may become more problematic. The general recommendation in such cases is to make the service calls idempotent, meaning: regardless, how often I do the same call, the end result is always the same. This could be achieved by various means and more or less depends on the use case.
For example, one method is it to send some logical "ticket" values along with each request to filter out doubled or outdated requests on the server. The server keeps track and/or checks these tickets, before the processing starts eventually. But again, if that method suits your needs depends on your use case.
The Command and Query Responsibility Segregation (CQRS) pattern is another approach to deal with the complexity. It basically breaks the API into setters and getters. I'd recommend to look into that topic, but it is not useful for every scenario. I'd also recommend to look at the Data Consistency Primer article. Last not least the CAP theorem is always a good read.
Good Service/API design is not simple, and the fact, that we have to deal with a distributed parallel system does not make it easier, quite the opposite.
Let me try to give a straight answer.
... is it the application's responsibility to do so?
Yes.
There're 4 types of Exceptions involved in Thrift RPC, including TTransportException, TProtocolException, TApplicationException, and User-defined exceptions.
Based on the book Programmer's Guide to Apache Thrift, the former 2 are local exceptions, while the latter 2 are not.
As the names imply, TTransportException includes exceptions like NOT_OPEN, TIMED_OUT, and TProtocolException includes INVALID_DATA, BAD_VERSION, etc. These exceptions are not propagated from the server the the client and act much like normal language exceptions.
TApplicationExceptions involve problems such as calling a method that isn’t implemented or failing to provide the necessary arguments to a method.
User-defined Exceptions are defined in IDL files and raised by the user code.
For all of these exceptions, no retry operations are done by Thrift RPC framework itself. Instead, they should be handled properly by the application code.
I'm writing a server application and its client counterpart that both use Netty for the network layer. I find myself facing typical safety concerns about sending a password from a client to the server so I decided SSL was the safest way of doing this.
I know of the securechat example and will use this to modify my pipelines accordingly. However, I would also like to disable SSL after password transmission and acknowledge to save a few precious CPU cycles on server side, which may be busy with many other clients. The ChannelPipeline documentation states that:
"Once attached, the coupling between the channel and the pipeline is permanent; the channel cannot attach another pipeline to it nor detach the current pipeline from it."
The idea is then to not change the pipeline on-the-fly, which is prohibited, but to somehow tell the SslHandler in the pipeline that it should stop encrypting messages at some point. I was thinking of creating a class inheriting from SslHandler, overriding its handleDownstream function to call context.sendDownstream(evt) after some point in the communication.
Question 1: Is this a bad idea, that is, disabling SSL at some point ?
To allow a block in the pipeline (say a Decoder) telling another block (say SslHandler) that it should change its behaviour from now on, I thought I could create, say, an AtomicBoolean in my ChannelPipelineFactory's getPipeline() and pass it to the constructor of both the Decoder and the SslHandler.
Question 2: Is this a bad idea, that is, sharing state between pipeline blocks ? I'm worried I might screw up the multithreading of Netty here: are the blocks of a pipeline working on a single message, one at a time ? i.e.: does the first block wait for the completion of the last block before pulling the next message ?
EDIT:
Oh my bad, this is from the ChannelPipeline page I had been visiting many times and quoting in this very question:
"A ChannelHandler can be added or removed at any time because a ChannelPipeline is thread safe. For example, you can insert a SslHandler when sensitive information is about to be exchanged, and remove it after the exchange."
So this answers question 2 about modifying the pipeline's content on-the-fly, and not the pipeline reference itself.
I'm not sure about the efficacy of turning off SSL once established, but I think you have misinterpreted the mutability of the pipeline. Once a given channel is associated with a pipeline, that association is immutable. However, the handlers in the pipeline can be safely modified. That is to say, you can add and remove handlers as your protocol requires. Accordingly,you should be able to remove the SSL handler once it has served its purpose.
You can remove SslHandler from the pipeline with ChannelPipeline.remove(..) then it should turn your connection to plaintext. Please file a bug if it does not work - we actually have not tried that scenario in production :-)
I'm not sure about Netty, but in principle, you could indeed carry on with plain traffic on the same TCP connection. There are a few downsides:
Only the authentication would be secured. A MITM could perform actions other than those intended by the user. (This is similar to using HTTP Digest to some extent: the credentials are protected, but the request/response entities aren't.)
From an implementation point of view, this is tricky to get right. The TLS specification says:
If the application protocol using TLS provides that any data may be
carried over the underlying transport after the TLS connection is
closed, the TLS implementation must receive the responding
close_notify alert before indicating to the application layer that
the TLS connection has ended.
This implies that you're going to synchronise your stream somehow to wait for the close_notify response, before carrying on with your plain traffic.
The SSLEngine programming model is rather complex, and you may find that the Netty API isn't necessary handling this situation.
While it may make sense to want to save a few CPU cycles, most of the SSL/TLS overhead is in the handshake, which you'll be doing anyway. The symmetric cryptographic operations used for the actual encryption of the data are much less expensive. (You should try to measure this overhead to see if it really is a problem.)