I have two different machine. One is configured with IP 192.168.2.100 and the other one with 192.168.2.101.
This is the code of the first verticle :
public class Sender extends AbstractVerticle {
public static void main(String... args) {
// Configuration du cluster manager
Config config = new Config();
config.getNetworkConfig().getJoin().getTcpIpConfig().addMember("192.168.2.101");
VertxOptions options = new VertxOptions();
options.setClusterManager(new HazelcastClusterManager());
options.setClusterHost("192.168.2.100");
options.setClustered(true);
options.setHAEnabled(true);
Vertx.clusteredVertx(options, vertx ->
vertx.result().deployVerticle(Sender.class.getName(), new DeploymentOptions().setHa(true))
);
}
#Override
public void start() throws Exception {
vertx.setPeriodic(5000, id -> {
vertx.eventBus().send("Address", "message",rep->{
System.out.println("response : "+rep.result().body());
});
});
}
}
And this is the code of the second verticle:
package com.vetx.Consumer;
import com.hazelcast.config.Config;
public class Consumer extends AbstractVerticle {
private String name = null;
public Consumer(String name) {
this.name = name;
}
public Consumer(){
}
public static void main(String... args) {
// Configuration du cluster manager
Config config = new Config();
config.getNetworkConfig().getJoin().getTcpIpConfig().addMember("192.168.2.100");
VertxOptions options = new VertxOptions();
options.setClusterManager(new HazelcastClusterManager());
options.setClusterHost("192.168.2.101");
options.setClustered(true);
options.setHAEnabled(true);
Vertx.clusteredVertx(options, vertx ->
vertx.result().deployVerticle(Consumer.class.getName(), new DeploymentOptions().setHa(true))
);
}
#Override
public void start() throws Exception {
vertx.eventBus().consumer("Address", message -> {
System.out.println(" received message: " +message.body());
message.reply("Success");
});
}
}
I try to use the high availability with cluster to implement a consumer to consume a message and a sender to send the message.When a try to kill the sender in order to redeploy verticle after failover, i got the following exception :
SEVERE: Failed to redeploy verticle after failover
java.lang.ClassNotFoundException: com.vetx.Sender.Sender
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at io.vertx.core.impl.JavaVerticleFactory.createVerticle(JavaVerticleFactory.java:37)
at io.vertx.core.impl.DeploymentManager.createVerticles(DeploymentManager.java:229)
at io.vertx.core.impl.DeploymentManager.lambda$doDeployVerticle$2(DeploymentManager.java:202)
at io.vertx.core.impl.FutureImpl.setHandler(FutureImpl.java:76)
at io.vertx.core.impl.DeploymentManager.doDeployVerticle(DeploymentManager.java:171)
at io.vertx.core.impl.DeploymentManager.doDeployVerticle(DeploymentManager.java:143)
at io.vertx.core.impl.DeploymentManager.deployVerticle(DeploymentManager.java:131)
at io.vertx.core.impl.HAManager.doDeployVerticle(HAManager.java:281)
at io.vertx.core.impl.HAManager.processFailover(HAManager.java:553)
at io.vertx.core.impl.HAManager.checkFailover(HAManager.java:489)
at io.vertx.core.impl.HAManager.nodeLeft(HAManager.java:309)
at io.vertx.core.impl.HAManager.access$100(HAManager.java:102)
at io.vertx.core.impl.HAManager$1.nodeLeft(HAManager.java:152)
at io.vertx.spi.cluster.hazelcast.HazelcastClusterManager.memberRemoved(HazelcastClusterManager.java:325)
at com.hazelcast.internal.cluster.impl.ClusterServiceImpl.dispatchEvent(ClusterServiceImpl.java:916)
at com.hazelcast.internal.cluster.impl.ClusterServiceImpl.dispatchEvent(ClusterServiceImpl.java:88)
at com.hazelcast.spi.impl.eventservice.impl.LocalEventDispatcher.run(LocalEventDispatcher.java:56)
at com.hazelcast.util.executor.StripedExecutor$Worker.process(StripedExecutor.java:217)
at com.hazelcast.util.executor.StripedExecutor$Worker.run(StripedExecutor.java:200)
Both of the cluster members should have the class com.vetx.Sender.Sender in their classpath since it will be serialized/deserialized on both sides. Seems like one of the members doesn't have it.
Also, did you make sure members can form a cluster of 2? I see your network config has one ip on each host getTcpIpConfig().addMember("192.168.2.101"), best practice is to add all ip's on all hosts i.e. to have an identical network config on all hosts to avoid confusions.
Related
I'm having trouble grasping the concepts behind multithreading in netty, EventLoopGroup (MultithreadEventLoopGroup), MultithreadEventExecutorGroup, DefaultEventExecutorGroup
I am trying to understand how the server handles multiple clients simultaneously sending requests that will execute some business logic and CRUD operations that add to RTT. Below is my netty server code which works, but I am trying to understand exactly how it will work with concurrent users and multiple open channels.
I have a simple ServerBootstrap
#Component
#RequiredArgsConstructor
public class SocketServer {
private final ContextAwareLogger logger;
private final ServerInitializer serverInitializer;
private final NioEventLoopGroup bossGroup;
private final NioEventLoopGroup workerGroup;
private Channel mainChannel;
#PostConstruct
public void start() {
try {
ServerBootstrap bootstrap = init();
mainChannel = bootstrap.bind(8484).sync().channel(); // save the main channel so we can cleanly close it when app is shutdown
logger.info("Netty server started...");
} catch (Exception e) {
e.printStackTrace();
}
}
#PreDestroy
public void stop() throws InterruptedException {
logger.info("Shutting down Netty server");
bossGroup.shutdownGracefully().sync();
workerGroup.shutdownGracefully().sync();
mainChannel.closeFuture().sync();
logger.info("Netty Server shutdown complete.");
}
private ServerBootstrap init() {
return new ServerBootstrap()
.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.option(ChannelOption.SO_BACKLOG, 5000)
.option(ChannelOption.TCP_NODELAY, true)
.option(ChannelOption.SO_KEEPALIVE, true)
.childHandler(serverInitializer);
}
}
ChannelInitializer:
#Component
#RequiredArgsConstructor
public class ServerInitializer extends ChannelInitializer<SocketChannel> {
private final PacketDecoder packetDecoder;
private final ServerHandler serverHandler;
private final PacketEncoder packetEncoder;
#Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
socketChannel.pipeline()
.addLast("decoder", packetDecoder) // ByteArrayDecoder
.addLast("encoder", packetEncoder) // ByteArrayEncoder
.addLast("inbound", serverHandler); // ChannelInboundHandlerAdapter
}
}
ChannelInboundHandlerAdapter:
#Component
#Sharable
public class ServerHandler extends ChannelInboundHandlerAdapter {
#Autowired
private SomeService someService;
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
// contains db access
byte[] accept = someService.validateClient(ctx.channel());
ctx.channel().writeAndFlush(accept);
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
// may contain db access
byte[] response = someService.processPacket(ctx.channel(), msg));
ctx.channel().writeAndFlush(response);
}
}
Now when a client connects, I understand that a new Channel will opened and the handlers will be reused. Requirement is each client request/response needs to process immediately without waiting for some other client's CRUD operations to finish.
Are my channelRead and channelActive, etc, async because I am using NioEventLoopGroup (ie will each client's channel operations be run independent of each other)?
If a single client sends multiple requests in series, are they guaranteed to be handled in the same order?
Do I need to specific DefaultEventExecutorGroup for my inbound handler? (https://stackoverflow.com/a/28305019/1738539)
You either would need to use a DefaultEventExecutorGroup for your ServerHandler or dispatch the validateClient(...) / processPacket(...) to your own ThreadPool. Failing todo so will cause the EventLoop thread to block and so no other IO can be processed for this EventLoop until the blocking operation completes.
I have to do some extra logic layer on existing classes. I'm using autofac.
The project is Windows service having also Kestrel server. Program.cs e.g.
void Main(string[] args) {
var customQueue = new CustomQueue();
var someClass = new SomeClass(customQueue);
var randomClass = new RandomClass();
// do some logic here with using declared instances
var server = new Server(someClass, randomClass);
server.Start();
}
And here is the Server.cs
class Server {
private IWebHost _host;
public Server(SomeClass cls, RandomClass cls1) {
// set to fields
}
void Start() {
_host = new WebHostBuilder()
.UseKestrel()
.ConfigureServices(services =>
{
services.AddAutoFac(); // first as per doc in order to scaffold 'ConfigureContainer'?
services.AddSingleton(someClass); // fields
services.AddSingleton(randomClass); // fields
})
.UseContentRoot(Directory.GetCurrentDirectory())
.ConfigureAppConfiguration(...)
.ConfigureLogging(...);
.UseStartup<Startuo>()
.Build();
_host.StartAsync();
}
}
Startup.cs
public class Startup
{
public ILifetimeScope AutofacContainer { get; private set; }
public IServiceProvider ConfigureServices(IServiceCollection services)
{
// create a container-builder and register dependencies
var builder = new ContainerBuilder();
// populate the service-descriptors added to `IServiceCollection`
// BEFORE you add things to Autofac so that the Autofac
// registrations can override stuff in the `IServiceCollection`
// as needed
builder.Populate(services);
builder.RegisterType<SomeClass>()
.As<ISomeClass>()
.EnableInterfaceInterceptors()
.InterceptedBy(typeof(SomeClassInterceptor));
builder.Register(_ => new SomeClassInterceptor());
AutofacContainer = builder.Build();
return new AutofacServiceProvider(AutofacContainer);
}
}
And last SomeClassInterceptor.cs
public class SomeClassInterceptor : IInterceptor
{
public void Intercept(IInvocation invocation)
{
try
{
invocation.Proceed();
if (invocation.ReturnValue is Task taskResult)
{
taskResult.ContinueWith(
t =>
{
Console.WriteLine("OOHH YEAHH");
}, TaskContinuationOptions.None);
}
else
{
Console.WriteLine("WOW");
}
}
catch (Exception ex)
{
Console.WriteLine("EXCEPTIOOOOOOOOOOOOONNNNNNNNNNNNNNNNNNNN");
}
}
}
I've tried with Named and Typed registration with having Intercept(...) attribute but still no success. Nothing gets triggered.
Also ISomeClass is inheriting other interfaces, I've tried setting .As<>() also with those but no.
What am I missing?
In order to let the interceptor works. The class should be build by Autofac.
services.AddAutofac();
services.AddSingleton(someClass);
services.AddSingleton(randomClass);
In your case you configure Autofac using the AddAutofac method then add SomeClass as a singleton using the AddSingleton which will override the Autofac configuration. There is no way that Autofac can inject the interceptor in it.
If you want to register SomeClass as singleton you should register it using the SingleInstance() method
builder.RegisterType<SomeClass>()
.As<ISomeClass>()
.SingleInstance()
.EnableInterfaceInterceptors()
.InterceptedBy(typeof(SomeClassInterceptor));
I have developed own idGenerator based on Hazelcast IdGenerator class (with storing each last_used_id into db). Now I want to run hazelcast cluster as a single java application and my web-application as other app (web-application restart shouldn't move id values to next block). I move MyIdGeneratorProxy and MyIdGeneratorService to new application, run it, run web-application as a hazelcast-client and get
IllegalArgumentException: No factory registered for service: ecs:impl:idGeneratorService
It was okay when client and server were the same application.
It seems it's unable to process without some clientProxy. I have compared IdGeneratorProxy and ClientIdGeneratorProxy and it looks the same. What is the idea? How to write client proxy for services? I have found no documentation yet. Is direction of investigations correct? I thought it is possible to divide hazelcast inner services (like a id generator service) and my business-processes. Should I store custom ClientProxy (for custom spi) in my web-application?
This is a demo how to create a client proxy, the missing part CustomClientProxy function call, is quit complicated(more like a server proxy,here is called ReadRequest, the server is called Operation), you can find a how AtomicLong implement.For every client proxy method you have to make a request.
#Test
public void client() throws InterruptedException, IOException
{
ClientConfig cfg = new XmlClientConfigBuilder("hazelcast-client.xml").build();
ServiceConfig serviceConfig = new ServiceConfig();
serviceConfig.setName(ConnectorService.NAME)
.setClassName(ConnectorService.class.getCanonicalName())
.setEnabled(true);
ProxyFactoryConfig proxyFactoryConfig = new ProxyFactoryConfig();
proxyFactoryConfig.setService(ConnectorService.NAME);
proxyFactoryConfig.setClassName(CustomProxyFactory.class.getName());
cfg.addProxyFactoryConfig(proxyFactoryConfig);
HazelcastInstance hz = HazelcastClient.newHazelcastClient(cfg);
Thread.sleep(1000);
for (int i = 0; i < 10; i++)
{
Connector c = hz.getDistributedObject(ConnectorService.NAME, "Connector:" + ThreadLocalRandom.current()
.nextInt(10000));
System.out.println(c.snapshot());
}
}
private static class CustomProxyFactory implements ClientProxyFactory
{
#Override
public ClientProxy create(String id)
{
return new CustomClientProxy(ConnectorService.NAME, id);
}
}
private static class CustomClientProxy extends ClientProxy implements Connector
{
protected CustomClientProxy(String serviceName, String objectName)
{
super(serviceName, objectName);
}
#Override
public ConnectorState snapshot()
{
return null;
}
#Override
public void loadState(ConnectorState state)
{
}
#Override
public boolean reconnect(HostNode node)
{
return false;
}
#Override
public boolean connect()
{
return false;
}
}
EDIT
In hazelcast the IdGenerate is implemented as a wrapper for AtomicLong, you should implement you IdGenerate by you own, instead of extend IdGenerate.
So you have to implement these(more like a todo list XD):
API
interface MyIdGenerate
Server
MyIdGenerateService
MyIdGenerateProxy
MyIdGenerateXXXOperation
Client
ClientMyIdGenerateFactory
ClientMyIdGenerateProxy
MyIdGenerateXXXRequest
I also made a sequence(same as IdGenerate) here, this is backed by zookeeper or redis,also it's easy to add a db backend,too.I will integrate to hazelcast if I got time.
I am receiving an error when using NServiceBus 4.0.3 with NHibernate 3.3.1 when it's trying to process a message
INFO NServiceBus.Unicast.Transport.TransportReceiver [(null)] <(null)> - Failed to process message
Autofac.Core.Registration.ComponentNotRegisteredException: The requested service 'NServiceBus.Impersonation.ExtractIncomingPrincipal' has not been registered. To avoid this exception, either register a component to provide the service, check for service registration using IsRegistered(), or use the ResolveOptional() method to resolve an optional dependency.
at NServiceBus.Unicast.Transport.TransportReceiver.ProcessMessage(TransportMessage message) in c:\BuildAgent\work\d4de8921a0aabf04\src\NServiceBus.Core\Unicast\Transport\TransportReceiver.cs:line 353
at NServiceBus.Unicast.Transport.TransportReceiver.TryProcess(TransportMessage message) in c:\BuildAgent\work\d4de8921a0aabf04\src\NServiceBus.Core\Unicast\Transport\TransportReceiver.cs:line 233
at NServiceBus.Transports.Msmq.MsmqDequeueStrategy.ProcessMessage(TransportMessage message) in c:\BuildAgent\work\d4de8921a0aabf04\src\NServiceBus.Core\Transports\Msmq\MsmqDequeueStrategy.cs:line 262
at NServiceBus.Transports.Msmq.MsmqDequeueStrategy.Action() in c:\BuildAgent\work\d4de8921a0aabf04\src\NServiceBus.Core\Transports\Msmq\MsmqDequeueStrategy.cs:line 197
2013-08-30 09:35:02,508 [9] WARN NServiceBus.Faults.Forwarder.FaultManager [(null)] <(null)> - Message has failed FLR and will be handed over to SLR for retry attempt: 1, MessageID=8aaed043-b744-49c2-965d-a22a009deb32.
I think it's fairly obvious what that I need to implement or register an "ExtractIncomingPrincipal", but I can't seem to find any documentation on how or whether there is a default one that I can use. I wouldn't have figured that I would have had to register any of the NServiceBus-related services as many of them are already being registered in my IoC implementation.
As requested, here is the EndpointConfig and supporting code I have currently:
[EndpointSLA("00:00:30")]
public class EndpointConfig : IConfigureThisEndpoint, AsA_Server, IWantCustomInitialization {
public void Init() {
Configure.With().ObjectBuilderAdapter().UseInMemoryTimeoutPersister().UseInMemoryGatewayPersister().InMemorySagaPersister().InMemorySubscriptionStorage();
}
}
//public class PrincipalExtractor : ExtractIncomingPrincipal {
// public IPrincipal GetPrincipal(TransportMessage message) {
// return Thread.CurrentPrincipal;
// }
//}
public class ObjectBuilderAdapter : IContainer {
readonly IDependencyInjector injector;
public ObjectBuilderAdapter(IDependencyInjectionBuilder dependencyInjectionBuilder) {
injector = dependencyInjectionBuilder.Create(); //This method does all the common service registrations that I am trying to re-use
//injector.RegisterType<ExtractIncomingPrincipal, PrincipalExtractor>();
}
public void Dispose() {
injector.Dispose();
}
public object Build(Type typeToBuild) {
return injector.Resolve(typeToBuild);
}
public IContainer BuildChildContainer() {
return new ObjectBuilderAdapter(new DependencyInjectorBuilder());
}
public IEnumerable<object> BuildAll(Type typeToBuild) {
return injector.ResolveAll(typeToBuild);
}
public void Configure(Type component, DependencyLifecycle dependencyLifecycle) {
injector.RegisterType(component);
}
public void Configure<T>(Func<T> component, DependencyLifecycle dependencyLifecycle) {
injector.RegisterType(component);
}
public void ConfigureProperty(Type component, string property, object value) {
if (injector is AutofacDependencyInjector) {
((AutofacDependencyInjector)injector).ConfigureProperty(component, property, value);
} else {
Debug.WriteLine("Configuring {0} for property {1} but we don't handle this scenario.", component.Name, property);
}
}
public void RegisterSingleton(Type lookupType, object instance) {
injector.RegisterInstance(lookupType, instance);
}
public bool HasComponent(Type componentType) {
return injector.IsRegistered(componentType);
}
public void Release(object instance) { }
}
public static class Extensions {
public static Configure ObjectBuilderAdapter(this Configure config) {
ConfigureCommon.With(config, new ObjectBuilderAdapter(new DependencyInjectorBuilder()));
return config;
}
}
I removed the IWantCustomInitialization (left over from something else I had tried earlier) interface implementation on the class and my service now processes the message. There are errors still (relating to trying to connect to Raven [even though I thought I am using everything in-memory), but it's processing the message.
I'am playing around with WebSocketServlet (tomcat) and I have some question about doing it properly without race condition problems.
I have an instance variable (so non thread-safe) that will keep track of all the websocket connections
HashMap<String,MyServers> myNonThreadSafeVariable = HashMap<String,MyServers>
This is what the HashMap will contain (roughly...)
private final class MyServers extends MessageInbound {
final Set<MyClients> clients = new CopyOnWriteArraySet<MyClients>();
private String serverName;
#Override
protected void onOpen(WsOutbound outbound) {}
#Override
protected void onClose(WsOutbound outbound) {}
#Override
protected void onText(WsOutbound outbound) {}
}
private final class Clients extends MessageInbound {
private int clientID;
#Override
protected void onOpen(WsOutbound outbound) {}
#Override
protected void onClose(WsOutbound outbound) {}
#Override
protected void onText(WsOutbound outbound) {}
}
So now.. during my servlet life time, I am looping through myNonThreadSafeVariable and then maybe also will loop through myNonThreadSafeVariable.clients and then maybe also modified or add a clients or server etc...
For example when a server connect, in his onOpen there will be something like
myNonThreadSafeVariable.put(key,this);
or When a client connects in his onOpen (quit concern about this one)
server = myNonThreadSafeVariable,get(key);
sever.clients.add(this);
Or sometimes when I have to ping all the clients of all the servers:
for (Entry<String, MyServers> entry : myNonThreadSafeVariable.entrySet()) {
MyServers server = entry.getValue();
server.sendMessage("ping||");
for (MyClients member : entry.getValue().clients) {
client.sendMessage("")
}
}
So If I undertand correctly as myNonThreadSafeVariable is global so will myNonThreadSafeVariable.clients etc..
So my question is what is a good practice to avoid race condition in this scenario ?
Using mutex and synchronized on them when access ether the myNonThreadSafeVariable and myNonThreadSafeVariable.clients ? Or should I avoid using an instance variable at all ? But how ?
thanks !
You could use a ReadWriteLock: you block readers and writers when writing, you block only writers when reading:
private final ReadWriteLock lock = new ReentrantReadWriteLock();
private final Lock readLock = lock.readLock();
private final Lock writeLock = lock.writeLock();
...
writeLock.lock();
try
{
myNonThreadSafeVariable.put(key,this);
}
finally
{
writeLock.unlock();
}
...
writeLock.lock();
try
{
server = myNonThreadSafeVariable,get(key);
sever.clients.add(this);
}
finally
{
writeLock.unlock();
}
...
readLock.lock();
try
{
for (Entry<String, MyServers> entry : myNonThreadSafeVariable.entrySet())
{
MyServers server = entry.getValue();
server.sendMessage("ping||");
for (MyClients member : entry.getValue().clients)
{
client.sendMessage("")
}
}
}
finally
{
readLock.unlock();
}
Moreover if you want to avoid the read lock you can copy the whole collection and scan the copy, letting the possibility for the original collection to be changed while notifying.