rabbitmq con connect after published on iis - iis

I am hosting asp.net core 2.0 webapi application on iis. but after host my application, rabbitmq host Broker unreachable. i have test this with vs debug mode and it's working fine. so my uri is correct.
here is my code.
services.AddSingleton(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://192.168.100.239:15672/"), "/", h =>
{
h.Username("vinit");
h.Password("vinit");
h.Heartbeat(10);
});
cfg.ReceiveEndpoint(host, "TestQ", e =>
{
e.LoadFrom(provider);
});
}));
in iis i am getting this error.
Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor[9]
An error occurred starting the application
System.AggregateException: One or more errors occurred. (Broker unreachable: vinit#192.168.100.239:15672/) ---> MassTransit.RabbitMqTransport.RabbitMqConnectionException: Broker unreachable: vinit#192.168.100.239:15672/ ---> RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable ---> System.IO.IOException: connection.start was never received, likely due to a network timeout
at RabbitMQ.Client.Framing.Impl.Connection.StartAndTune()
at RabbitMQ.Client.Framing.Impl.Connection.Open(Boolean insist)
at RabbitMQ.Client.Framing.Impl.ProtocolBase.CreateConnection(IConnectionFactory factory, Boolean insist, IFrameHandler frameHandler, String clientProvidedName)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
--- End of inner exception stack trace ---
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
at MassTransit.RabbitMqTransport.Integration.ConnectionContextFactory.<CreateConnection>d__9.MoveNext()
--- End of inner exception stack trace ---
at MassTransit.RabbitMqTransport.Integration.ConnectionContextFactory.<CreateConnection>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
is i am missing some thing or need to do any additional setting?
Please suggest me.
Thank you

After reading RabbitMQ Document, i got answer that we can manage port from rabbitmq.config file in our system, and there are different port for transport.
Thank you

Related

Connectivity problem using Spring Data Mongo and Spring Integration with Azure CosmosDB (MongoDB)

I have the following setup:
Spring Integration with Spring Data Mongo repositories
Above application running as a pod on kubernetes (Azure Kubernetes Service)
Above application connects to Azure CosmosDB (MongoDB)
Above application connects to Azure SQL Server
The application should fetch information from MongoDB and start processing. It works all fine while running locally and within a docker container. The problem occurs when it's deployed to AKS as it can't establish connection to Mongo and Azure SQL while bootstrapping. The connection will be successfully opened after another 3 seconds or so which you can see here: https://gist.github.com/nadworny/c69659e65a7d6e8d96573db13d1f1095
For comparison here is bootstrapping log from localhost: https://gist.github.com/nadworny/c04d6baa571e5b7ddcbd8856cf22a390
What I also can't explain is what happens afterwards. I have a Mongo inbound adapter that looks as follow:
return IntegrationFlows.from(MongoDb.reactiveInboundChannelAdapter(mongoDbFactory,
new Query().addCriteria(Criteria.where("status").is(ProcessingStatus.PROCESSED))
.with(Sort.by(Sort.Direction.DESC, "modifiedDate")).limit(1))
.collectionName("processingMetadata")
.entityClass(ProcessingMetadata.class)
.expectSingleResult(true),
e -> e.poller(Pollers.fixedDelay(Duration.ofSeconds(pollingIntervalSeconds))))
.<Mono<ProcessingMetadata>>handle((p, h) -> {
// do something
})
It should poll Mongo every 3 seconds but it actually never does (again it works locally without a problem).
The connectivity works fine after that as I have another process that is triggered by a file upload and it works correctly (data is written to MongoDB).
To be honest I'm a little bit confused which of the components is making troubles here so I hope you can help me narrow it down.
UPDATE1
I did some further debugging. Actually it's not a problem of mongo but the next handler after it which is Jpa Gateway:
.handle(Jpa.retrievingGateway(this.sourceEntityManagerFactory)
.entityClass(DocumentHeader.class)
.jpaQuery("from DocumentHeader d where d.modifiedDate > :modified")
.parameterExpression("modified", "payload")
.maxResults(maxResults), e -> e.id("retrieveDocumentHeader"))
.<List>handle((p, h) -> {
if (p.isEmpty())
this.advices.waitUntilCompletedAdvice().setWait(false);
return p;
})
.channel(Channels.DOCUMENT_HEADER.name())
For some reason, while executing on AKS, the Jpa Handler won't be invoked and Spring Integration jumps directly to the next step:
2020-09-25 08:35:07.771 DEBUG 1 --- [ask-scheduler-3] o.s.d.m.core.ReactiveMongoTemplate : find using query: { "status" : "PROCESSED"} fields: Document{{}} for class: class com.zurich.ccmc.data.orchestrator.domain.targetdb.ProcessingMetadata in collection: processingMetadata
2020-09-25 08:35:07.771 DEBUG 1 --- [ask-scheduler-3] o.s.i.e.SourcePollingChannelAdapter : Poll resulted in Message: GenericMessage [payload=MonoNext, headers={mongo_collectionName=processingMetadata, id=4a26e221-deb9-f1af-1c35-7ba0d47042b1, timestamp=1601022907771}]
2020-09-25 08:35:07.778 DEBUG 1 --- [ask-scheduler-3] o.s.i.splitter.DefaultMessageSplitter : handler 'bean 'setProcessingStatusToOpen.splitter#0' for component 'setProcessingStatusToOpen.org.springframework.integration.config.ConsumerEndpointFactoryBean#0'; defined in: 'class path resource [xxx/processing/ProcessingMetadataFlow.class]'; from source: 'bean method setProcessingStatusToOpen'' produced no reply for request Message: GenericMessage [payload=[], headers={mongo_collectionName=processingMetadata, id=0a446b4c-12fc-d72d-d4ad-8453eb51227c, timestamp=1601022907778}]
Found the problem. The spring.datasource.jdbc-url parameter was wrong (jdbc:sqlserver:// was missing at the beginning when deployed to AKS). I would expect some kind of a proper exception but there is only this warning:
2020-09-24 14:33:04.710 WARN 1 --- [ task-1] o.h.e.j.e.i.JdbcEnvironmentInitiator : HHH000342: Could not obtain connection to query metadata : Driver com.microsoft.sqlserver.jdbc.SQLServerDriver
accept jdbcUrl, xxx.database.windows.net
What I also don't understand is why the Jpa.retrievingGateway didn't throw any exception and jump over to the next handler...

Why does kestrel keep throwing this debug ssl exception?

I have a Service Fabric Setup and I have a microservice that registers on port 443 and uses https. I have a reverse proxy in my cluster setup. The reverse proxy is secured with a certificate.
I also use the same certificate when starting up my microservice:
new ServiceInstanceListener(serviceContext =>
new KestrelCommunicationListener(serviceContext, "EndpointHttps", (url, listener) =>
{
ServiceEventSource.Current.ServiceMessage(serviceContext, $"Starting Kestrel on {url}");
return new WebHostBuilder()
.UseKestrel(x =>
{
int port = serviceContext.CodePackageActivationContext.GetEndpoint("EndpointHttps").Port;
x.Listen(IPAddress.IPv6Any, port, listenOptions =>
{
listenOptions.UseHttps(transportCertificate);
listenOptions.NoDelay = true;
});
})
.ConfigureServices(
services => services
.AddSingleton<StatelessServiceContext>(serviceContext))
.UseContentRoot(Directory.GetCurrentDirectory())
.UseEnvironment(environment)
.UseStartup<Startup>()
.UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
.UseUrls(url)
.UseSerilog(Logger.Serilog)
.Build();
}))
};
Everything seems to work fine and my site is secure in the browser and the api works. However my log is getting filled up with the following statements (these are just debug messages but they are filling up my log):
133649 Failed to authenticate HTTPS connection. Debug System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
133643 Connection id ""0HLPCETMP8DKM"" started. key='SourceContext'>Microsoft.AspNetCore.Server.Kestrel
133642 Connection id ""0HLPCETMP8DKL"" received FIN. key='SourceContext'>Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets
133641 Connection id ""0HLPCETLEPLQ7"" stopped. key='SourceContext'>Microsoft.AspNetCore.Server.Kestrel
133640 Connection id ""0HLPCETLEPLQ7"" sending FIN. key='SourceContext'>Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets
The exception is:
System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Security.SslState.ThrowIfExceptional()
at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
at System.Net.Security.SslStream.EndAuthenticateAsServer(IAsyncResult asyncResult)
at System.Net.Security.SslStream.<>c.<AuthenticateAsServerAsync>b__51_1(IAsyncResult iar)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionAdapter.InnerOnConnectionAsync(ConnectionAdapterContext context)
Does anyone have any idea why it would do this? Any other logs I can look at or suggestions on how to debug?
Thanks.
This is very likely due to the fact that you're running Service Fabric with the standard template, which sets up a Azure Load Balancer with a health probe for the ports that you've set up to use. The standard probe is generally TCP and will probe the endpoint every 5 seconds. You can remove the probe if that's a viable option. I personally use a less aggressive probe interval.
Note that changing the load balancer rules generally takes a long time, anything from a few minutes to near half an hour.

Need help investigating Azure App Service .NET Core 2.1 ArgumentOutOfRangeException and System.OverflowException

Today I've adjusted the App Service Log settings for my Azure App Serivce: a .NET Core 2.1 Web API. Soon after I changed the Application Logging (file system) setting from Information to Warning, my application stopped responding. I immediately kicked of a .NET Trace using the Diagnose and solve problems feature.
This Trace report showed the following:
system.private.corelib!
microsoft.aspnetcore.hosting!
System.Private.CoreLib!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start
microsoft.aspnetcore.hosting.il!Microsoft.AspNetCore.Hosting.Views.ErrorPage.ExecuteAsync
microsoft.aspnetcore.hosting!
System.Private.CoreLib!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start
microsoft.aspnetcore.hosting.il!Microsoft.Extensions.RazorViews.BaseView.ExecuteAsync
microsoft.aspnetcore.hosting!
microsoft.aspnetcore.hosting!
System.Private.CoreLib!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1+<>c[System.Threading.Tasks.VoidTaskResult,Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__188`1[Microsoft.AspNetCore.Hosting.Internal.HostingApplication+Context]].<.cctor>b__9_0
system.private.corelib!
System.Private.CoreLib!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[System.Threading.Tasks.VoidTaskResult,Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__188`1[Microsoft.AspNetCore.Hosting.Internal.HostingApplication+Context]].MoveNext
system.private.corelib!
system.private.corelib!
system.private.corelib!
system.private.corelib!
system.private.corelib!
Same time, the Azure Blob log showed these lines:
2019-07-06 14:04:06.476 +00:00 [Error] Microsoft.AspNetCore.Server.Kestrel: Connection id "0HLO233NHLK6Q", Request id "0HLO233NHLK6Q:000003BF": An unhandled exception was thrown by the application.
System.OverflowException: Arithmetic operation resulted in an overflow.
at System.IO.StreamWriter.Write(String value)
at Microsoft.Extensions.RazorViews.BaseView.Write(String value)
at Microsoft.AspNetCore.Hosting.Views.ErrorPage.ExecuteAsync()
at Microsoft.Extensions.RazorViews.BaseView.ExecuteAsync(HttpContext context)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
2019-07-06 14:04:19.245 +00:00 [Error] Microsoft.AspNetCore.Server.Kestrel: Connection id "0HLO233NHLK6T", Request id "0HLO233NHLK6T:000000EF": An unhandled exception was thrown by the application.
System.ArgumentOutOfRangeException: Index and count must refer to a location within the buffer.
Parameter name: chars
at System.Text.EncoderNLS.GetBytes(Char[] chars, Int32 charIndex, Int32 charCount, Byte[] bytes, Int32 byteIndex, Boolean flush)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.Write(String value)
at Microsoft.AspNetCore.Hosting.Views.ErrorPage.ExecuteAsync()
at Microsoft.Extensions.RazorViews.BaseView.ExecuteAsync(HttpContext context)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
These stacktraces don't show any of my own code, they seem framework related.
Is there a message or a hint in these error stack traces that I can use to prevent these situations?
Many thanks in advance
I know it's too late but maybe my answer will help someone else.
I received the same exceptions as you. But it was not Azure App Service but IIS. The problem was in incorrect configuration somewhere in Startup.cs. You webhost tried to show error page but at the same time App Service (or IIS in my case) tried to shut down or restart it.
This thread helped me a lot https://github.com/aspnet/KestrelHttpServer/issues/1737

how to debug error 500 Internal Server Error on an Azure App?

I got an "500 Internal Server Error - An error occurred while starting the application" after deploying my application: https://iidapp.azurewebsites.net/
I keep finding the following error message but I am unable to find out on msdn websites any information describing how I can specify the SAS URL
INFO: The app was working for a long period and I didn't have to set the SAS URL; I wonder why suddenly Azure is generating exceptions
INFO2: the app works perfectly on my local machine
Any help is welcomed as I couldn't find any solution by reading the related topics on stackoverflow
2017-04-05T18:51:32
System.ApplicationException: The trace listener AzureBlobTraceListener is disabled. ---> System.InvalidOperationException: The SAS URL for the cloud storage account is not specified. Use the environment variable 'DIAGNOSTICS_AZUREBLOBCONTAINERSASURL' to define it.
at Microsoft.WindowsAzure.WebSites.Diagnostics.AzureBlobTraceListener.RefreshConfig()
--- End of inner exception stack trace ---
An error was triggered by W3SVC-WP : {app name}02000780
I found out on eventid that the code 02000780 meant that a file was missing
I eventually found out that it was possible to log further information by enabling the stdoulog inside the web.config. Read When a .NET Core Azure App Service won’t start: 502.5 Process Failure
I opened the debug console and found out that a directory wasn't found https://{app_id}.scm.azurewebsites.net/DebugConsole
Voilà! I corrected the code and the app is up and running!

Azure Web App - HTTP time out

After a deploy to our Azure Web App, we are getting 500 timeouts on any request to the service:
500 - The request timed out.
The web server failed to respond within the specified time.
This has come out of the blue and we cannot determine what's causing it. It seems to take around 230s consistently to time out.
I've enabled all the diagnostic logs in the portal:
But I honestly don't quite know what to look for in the logs. I've scoured through all the files in the following folders but nothing jumps out.
How can I troubleshoot this problem?
The trick to get debug messages is to set stdoutLogFile="D:\home\LogFiles\stdout.log" in your config, instead of the ..\logs path that you had. After changing that, you get an error file under D:\home\LogFiles. Here is the error you get:
Application startup exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentNullException: SMTP server password cannot be null or empty.
Parameter name: smtpPassword
at TransitApi.Infrastructure.Modules.Logging.EmailOutput.EmailLogger..ctor(String recipient, String sender, String smtpUsername, String smtpPassword, String smtpHost, Int32 smtpPort, String environmentName, LogLevel minimumLevel)
at TransitApi.Infrastructure.Modules.Logging.EmailOutput.EmailLoggerProvider.CreateLogger(String name)
at Microsoft.Extensions.Logging.Logger.AddProvider(ILoggerProvider provider)
at Microsoft.Extensions.Logging.LoggerFactory.AddProvider(ILoggerProvider provider)
at TransitApi.Api.Startup.Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
So some kind of issue setting up the mail server. That causes the process to crash, and that it behaves poorly.
But I highly suggest that you upgrade to Core RC2, as RC1 is quite obsolete.

Resources