Security issue while writing data to HBase table from SPARK using Java - apache-spark

JavaDStream<FreshInput> inputDStream = kafkaInputDStream
.map(new Function<Tuple2<String, FreshInput>, FreshInput>() {
private static final long serialVersionUID = 1L;
#Override
public FreshInput call(Tuple2<String, FreshInput> tuple) throws Exception {
if(tuple._2!=null)
{
FreshLogger.errorLog_and_Console("***************************************************");
FreshLogger.errorLog_and_Console("INSIDE FRESH INPUT CALL");
FreshLogger.errorLog_and_Console(tuple._1.toString());
FreshLogger.errorLog_and_Console(tuple._2.toString());
FreshLogger.errorLog_and_Console("***************************************************");
Configuration hconf=HBaseConfiguration.create();
hconf.addResource(new Path("/etc/hbase/conf/core-site.xml"));
hconf.addResource(new Path("/etc/hbase/conf/hbase-site.xml"));
UserGroupInformation.setConfiguration(hconf);
FreshLogger.errorLog_and_Console("hconf is "+hconf.toString());
//UserGroupInformation.loginUserFromKeytabAndReturnUGI("gffshnee", "/etc/krb5.keytab");
Connection connection = ConnectionFactory.createConnection(hconf);
Table table = connection.getTable(TableName.valueOf("gfttsdgn:FRESHHBaseRushi"));
Put p = new Put(Bytes.toBytes("row1"));
p.add(Bytes.toBytes("c1"), Bytes.toBytes("output"), Bytes.toBytes("rushi"));
table.put(p);
}
return tuple._2();
}
});
Getting following Exception:
2017-02-24 09:09:33 WARN -FRESH Executor task launch worker-0 -UserGroupInformation : PriviledgedActionException as:gffshnee (auth:SIMPLE) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
2017-02-24 09:09:33 WARN -FRESH Executor task launch worker-0 -RpcClientImpl$Connection$1 : Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
2017-02-24 09:09:33 ERROR-FRESH Executor task launch worker-0 -RpcClientImpl$Connection$1 : SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:181)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1783)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1242)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:34070)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1589)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1398)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1199)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:395)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:344)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:238)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1495)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1086)
at com.citi.fresh.freshcore.util.ProcessRDD$1.call(ProcessRDD.java:198)
at com.citi.fresh.freshcore.util.ProcessRDD$1.call(ProcessRDD.java:160)
at org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1.apply(JavaPairRDD.scala:1015)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.convert.Wrappers$IteratorWrapper.next(Wrappers.scala:30)
at com.citi.fresh.freshcore.util.ProcessRDD$3$1.call(ProcessRDD.java:361)
at com.citi.fresh.freshcore.util.ProcessRDD$3$1.call(ProcessRDD.java:352)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:225)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:225)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1869)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1869)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
Please suggest.

Related

Embedded apache drill - unable to start with security configuration

We are fetching password with custom authentication provider and that will fetch the password from secured service. with below configuration when we start it, it is throwing error. The same configuration is working for cluster mode.
drill.exec: {
sys.store.provider.local.path="/home/user/somedatafolder",
security.user.auth:
{ enabled: true, packages += "com.app", impl: "myCustomAuthenticatorType" // custom auth provider }
},
// below properties are used by the custom authenticator to fetch the password from external service
drill.username: "username",
drill.password.key: "passkey",
drill.password.service.url: "https://somehost/api",
drill.exec.options:
{ security.admin.users: "username" }
Error :
Error: Failure in connecting to Drill: org.apache.drill.exec.rpc.NonTransientRpcException: javax.security.sasl.SaslException: Server requires authentication using [PLAIN]. Insufficient credentials?. [Details: Encryption: disabled , MaxWrappedSize: 65536 , WrapSizeLimit: 0]. (state=,code=0)
java.sql.SQLNonTransientConnectionException: Failure in connecting to Drill: org.apache.drill.exec.rpc.NonTransientRpcException: javax.security.sasl.SaslException: Server requires authentication using [PLAIN]. Insufficient credentials?. [Details: Encryption: disabled , MaxWrappedSize: 65536 , WrapSizeLimit: 0].
at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:178)
at org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:67)
at org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:67)
at org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
at org.apache.drill.jdbc.Driver.connect(Driver.java:75)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
at sqlline.Commands.connect(Commands.java:1364)
at sqlline.Commands.connect(Commands.java:1244)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
at sqlline.SqlLine.dispatch(SqlLine.java:730)
at sqlline.SqlLine.initArgs(SqlLine.java:410)
at sqlline.SqlLine.begin(SqlLine.java:515)
at sqlline.SqlLine.start(SqlLine.java:267)
at sqlline.SqlLine.main(SqlLine.java:206)
Caused by: org.apache.drill.exec.rpc.NonTransientRpcException: javax.security.sasl.SaslException: Server requires authentication using [PLAIN]. Insufficient credentials?. [Details: Encryption: disabled , MaxWrappedSize: 65536 , WrapSizeLimit: 0].
at org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:207)
at org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:458)
at org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:402)
at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:169)
... 18 more

Webflux test in error: ApplicationContextException: Unable to start ReactiveWebApplicationContext due to missing ReactiveWebServerFactory bean

I am trying to factor out some mock bean configuration.
Here is the mock configuration:
#Configuration
public class PvgCacheMockConfiguration {
#Primary
#Bean
public PvgProjectClient pvgProjectClient() {
PvgProjectClient mock = Mockito.mock(PvgProjectClient.class);
when(mock.getAll()).thenReturn(generateMockProjects());
return mock;
}
private ValidationProjects generateMockProjects() {
ValidationProject validationProject = ValidationProject
.builder()
.id("anId")
.code("aCode")
.name("aName")
.projectResponsibleIpn("anIpn")
.batchForMilestone(MilestoneName.PPC)
.type(ValidationProjectType.VEHICLE)
.company(Company.builder().code("aCode").label("aLabel").build())
.withValidationPlanModule(false)
.build();
return ValidationProjects.builder().validationProjects(singletonList(validationProject)).build();
}
}
Here is how I try to import my configuration:
#RunWith(SpringRunner.class)
#SpringBootTest(
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
properties = {"spring.main.web-application-type=reactive"})
#ActiveProfiles("test")
#Sql(scripts = {"classpath:reset.sql",
"classpath:sql/validationSupportNeedRepositoryTest/ValidationSupportNeedEntity.sql"})
#ContextConfiguration(classes = {PvgCacheMockConfiguration.class})
#AutoConfigureWebTestClient
public class ValidationSupportNeedControllerIntegrationTest {
...
Here is the stack trace when I run any test from the above test suite:
java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:125)
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:108)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:117)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
at org.springframework.boot.test.autoconfigure.SpringBootDependencyInjectionTestExecutionListener.prepareTestInstance(SpringBootDependencyInjectionTestExecutionListener.java:44)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:246)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:227)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:289)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:291)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:246)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:116)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:59)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:39)
at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy1.processTestClass(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:146)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:128)
at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.context.ApplicationContextException: Unable to start reactive web server; nested exception is org.springframework.context.ApplicationContextException: Unable to start ReactiveWebApplicationContext due to missing ReactiveWebServerFactory bean.
at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.onRefresh(ReactiveWebServerApplicationContext.java:76)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:544)
at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:61)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:395)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:327)
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:139)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:117)
... 48 more
Caused by: org.springframework.context.ApplicationContextException: Unable to start ReactiveWebApplicationContext due to missing ReactiveWebServerFactory bean.
at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.getWebServerFactory(ReactiveWebServerApplicationContext.java:125)
at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.createWebServer(ReactiveWebServerApplicationContext.java:99)
at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.onRefresh(ReactiveWebServerApplicationContext.java:73)
... 56 more
From STDOUT:
2018-09-17 14:54:19.759 WARN 12777 --- [ Test worker] onfigReactiveWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start reactive web server; nested exception is org.springframework.context.ApplicationContextException: Unable to start ReactiveWebApplicationContext due to missing ReactiveWebServerFactory bean.
From STDERR:
sept. 17, 2018 2:53:36 PM org.springframework.test.context.support.AbstractContextLoader generateDefaultLocations
INFOS: Could not detect default resource locations for test class [com.renault.digital.pval.prototype.api.rest.controllers.ValidationSupportNeedControllerIntegrationTest]: no resource found for suffixes {-context.xml, Context.groovy}.
sept. 17, 2018 2:53:36 PM org.springframework.boot.test.context.SpringBootTestContextBootstrapper getDefaultTestExecutionListenerClassNames
INFOS: Loaded default TestExecutionListener class names from location [META-INF/spring.factories]: [org.springframework.security.test.context.support.WithSecurityContextTestExecutionListener, org.springframework.security.test.context.support.ReactorContextTestExecutionListener, org.springframework.boot.test.autoconfigure.restdocs.RestDocsTestExecutionListener, org.springframework.boot.test.autoconfigure.web.client.MockRestServiceServerResetTestExecutionListener, org.springframework.boot.test.autoconfigure.web.servlet.MockMvcPrintOnlyOnFailureTestExecutionListener, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverTestExecutionListener, org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener, org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener, org.springframework.test.context.web.ServletTestExecutionListener, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener, org.springframework.test.context.support.DependencyInjectionTestExecutionListener, org.springframework.test.context.support.DirtiesContextTestExecutionListener, org.springframework.test.context.transaction.TransactionalTestExecutionListener, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener]
sept. 17, 2018 2:53:36 PM org.springframework.boot.test.context.SpringBootTestContextBootstrapper getTestExecutionListeners
INFOS: Using TestExecutionListeners: [org.springframework.test.context.web.ServletTestExecutionListener#43f48770, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener#32d9e79d, org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener#69dc5744, org.springframework.boot.test.autoconfigure.SpringBootDependencyInjectionTestExecutionListener#11a74db, org.springframework.test.context.support.DirtiesContextTestExecutionListener#6632bc8f, org.springframework.test.context.transaction.TransactionalTestExecutionListener#1baa4adb, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener#10c7489d, org.springframework.security.test.context.support.WithSecurityContextTestExecutionListener#42a94af2, org.springframework.security.test.context.support.ReactorContextTestExecutionListener#310c50ec, org.springframework.boot.test.autoconfigure.restdocs.RestDocsTestExecutionListener#78393884, org.springframework.boot.test.autoconfigure.web.client.MockRestServiceServerResetTestExecutionListener#521004be, org.springframework.boot.test.autoconfigure.web.servlet.MockMvcPrintOnlyOnFailureTestExecutionListener#5ae62ad4, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverTestExecutionListener#93e9a54, org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener#38f27eef]

Spring boot integration mail - authorization failed error

Following is my spring configuration bean for pop3 connection,
#Bean
public IntegrationFlow mailListener() {
return IntegrationFlows.from(Mail.pop3InboundAdapter("pop3://sample.test:Sample2test_1#xxx.xx.x.xx/INBOX")
.shouldDeleteMessages(true).get(),
e -> e.poller(Pollers.fixedRate(5000).maxMessagesPerPoll(1)))
.<Message>handle((payload, header) -> logMail(payload))
.get();
}
Also double checked the credentials that looks good.
Getting the below exception,
2018-06-22 19:27:54.351 ERROR 2092 --- [ask-scheduler-2]
o.s.integration.handler.LoggingHandler :
org.springframework.messaging.MessagingException: failure occurred
while polling for mail; nested exception is
javax.mail.AuthenticationFailedException: authorization failed at
org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:131)
at
org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:224)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:245)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:58)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:190)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:186)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:353)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55)
at
org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:51)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:344)
at
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at
org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
Source) at java.util.concurrent.FutureTask.run(Unknown Source) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown
Source) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source) at java.lang.Thread.run(Unknown Source) Caused by:
javax.mail.AuthenticationFailedException: authorization failed at
com.sun.mail.pop3.POP3Store.protocolConnect(POP3Store.java:208) at
javax.mail.Service.connect(Service.java:295) at
javax.mail.Service.connect(Service.java:176) at
javax.mail.Service.connect(Service.java:125) at
org.springframework.integration.mail.AbstractMailReceiver.connectStoreIfNecessary(AbstractMailReceiver.java:286)
at
org.springframework.integration.mail.AbstractMailReceiver.openFolder(AbstractMailReceiver.java:293)
at
org.springframework.integration.mail.AbstractMailReceiver.receive(AbstractMailReceiver.java:319)
at
org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:112)
... 19 more
There is an option like this:
/**
* The Java Mail properties.
* #param javaMailProperties the javaMailProperties.
* #return the spec.
* #see AbstractMailReceiver#setJavaMailProperties(Properties)
*/
public S javaMailProperties(Properties javaMailProperties) {
You can specify there mail.debug = true to trace what's going on with mail protocol during interaction.
UPDATE
We have this in your logs:
DEBUG POP3: connecting to host "172.16.1.45", port 995, isSSL false
19:54:46.746 [task-scheduler-1] DEBUG org.springframework.beans.factory.support.DefaultListableBeanFactory - Returning cached instance of singleton bean 'errorChannel'
19:54:46.753 [task-scheduler-1] DEBUG org.springframework.integration.channel.PublishSubscribeChannel - preSend on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.messaging.MessagingException: failure occurred while polling for mail; nested exception is javax.mail.MessagingException: Connect failed;
nested exception is:
java.net.ConnectException: Connection refused: connect, headers={id=290f4c5a-2dfe-d7cb-0637-a0e9f0394daa, timestamp=1529677486753}]
So, it can't just connect to the 172.16.1.45:995. There is no authentication handshake yet.

Obtaining Azure Access Token for German Cloud

I have a small program which fails while trying to obtain the Access Token for the Azure German Cloud
public static String generateAccessToken(AzureAccount accountValue) throws MalformedURLException, InterruptedException,
ExecutionException, ServiceUnavailableException {
AuthenticationContext context;
ExecutorService service = null;
try {
String tenantId = accountValue.getTenant();
String ClientID = accountValue.getClient();
String secretKey = accountValue.getKey();
service = Executors.newFixedThreadPool(1);
context = new AuthenticationContext(
"https://login.microsoftonline.de/" + tenantId
+ "/oauth2/authorize", false,
service);
ClientCredential cred = new ClientCredential(ClientID, secretKey);
Future<AuthenticationResult> future =
context.acquireToken("https://management.microsoftazure.de", cred,
null);
AuthenticationResult authenticationResult = future.get();
if (authenticationResult == null) {
throw new ServiceUnavailableException("authentication result was null");
}
System.out.println("Bearer " + authenticationResult.getAccessToken());
return "Bearer " + authenticationResult.getAccessToken();
}
finally {
service.shutdown();
}
}
The code works correctly for the Azure General Cloud but is failing for the German Cloud.
This is the stackTrace
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:914)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1283)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1258)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
at com.microsoft.aad.adal4j.AdalOAuthRequest.configureHeaderAndExecuteOAuthCall(AdalOAuthRequest.java:140)
at com.microsoft.aad.adal4j.AdalOAuthRequest.send(AdalOAuthRequest.java:83)
at com.microsoft.aad.adal4j.AdalTokenRequest.executeOAuthRequestAndProcessResponse(AdalTokenRequest.java:80)
at com.microsoft.aad.adal4j.AuthenticationContext.acquireTokenCommon(AuthenticationContext.java:818)
at com.microsoft.aad.adal4j.AuthenticationContext.access$100(AuthenticationContext.java:66)
at com.microsoft.aad.adal4j.AuthenticationContext$1.call(AuthenticationContext.java:174)
at com.microsoft.aad.adal4j.AuthenticationContext$1.call(AuthenticationContext.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491)
... 23 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
... 29 more
java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.vmturbo.mediation.azure.Runner.generateAccessToken(Runner.java:127)
at com.vmturbo.mediation.azure.Runner.main(Runner.java:79)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:914)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1283)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1258)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
at com.microsoft.aad.adal4j.AdalOAuthRequest.configureHeaderAndExecuteOAuthCall(AdalOAuthRequest.java:140)
at com.microsoft.aad.adal4j.AdalOAuthRequest.send(AdalOAuthRequest.java:83)
at com.microsoft.aad.adal4j.AdalTokenRequest.executeOAuthRequestAndProcessResponse(AdalTokenRequest.java:80)
at com.microsoft.aad.adal4j.AuthenticationContext.acquireTokenCommon(AuthenticationContext.java:818)
at com.microsoft.aad.adal4j.AuthenticationContext.access$100(AuthenticationContext.java:66)
at com.microsoft.aad.adal4j.AuthenticationContext$1.call(AuthenticationContext.java:174)
at com.microsoft.aad.adal4j.AuthenticationContext$1.call(AuthenticationContext.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491)
... 23 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
It looks like it is looking for some certificate file which I have no idea about. Does anybody have any idea about this ?
Also is there a way of obtaining the Azure Access token using the Azure Java SDK so that I don't have to make a REST API call individually ?
This is because you were using a VM not in the German IP range. You have two options:
Deploy a VM in Azure Germany cloud or in AWS Frankfurt
In order to deploy from a non-German location: update the Java SDK you have (I had 1.8.0_92 and updated to 1.8.0_144) and this should work

Connecting to cassandra cluster with kerberos using java driver

I followed the instructions from the following datastax post - Accessing secure DSE clusters
This is my code snippet -
public static void main(String[] args) {
KerberosAuthenticatedClient client = new KerberosAuthenticatedClient();
System.setProperty("java.security.krb5.conf","C:/Users/ADMIN/Desktop/krb5.config");
System.setProperty("java.security.auth.login.config","C:/Users/ADMIN/Desktop/DseClient.config");
cluster = Cluster.builder().addContactPoint(node).withAuthProvider(new DseAuthProvider()).build();
session = cluster.connect();
This is my DseClient File -
DseClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
doNotPrompt=true
keyTab="C:/Users/ADMIN/Desktop/dse.keytab"
principal="rock#MY.COM";
};
And This is my krb5.conf -
[libdefaults]
default_realm = MY.COM
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
[realms]
MY.COM = {
kdc = x.x.x.x
admin_server = x.x.x.x
}
[domain_realm]
.my.com = MY.COM
my.com = MY.COM
I have installed kerberos on my client machine and also on the servers too. But when I run I get the following error :-
Exception in thread "main" java.lang.RuntimeException: javax.security.auth.login.LoginException: Unable to obtain password from user
at com.datastax.driver.core.sasl.KerberosAuthenticator.loginSubject(KerberosAuthenticator.java:113)
at com.datastax.driver.core.sasl.KerberosAuthenticator.<init>(KerberosAuthenticator.java:94)
at com.datastax.driver.core.sasl.DseAuthProvider.newAuthenticator(DseAuthProvider.java:52)
at com.datastax.driver.core.Connection.initializeTransport(Connection.java:163)
at com.datastax.driver.core.Connection.<init>(Connection.java:131)
at com.datastax.driver.core.Connection.<init>(Connection.java:59)
at com.datastax.driver.core.Connection$Factory.open(Connection.java:444)
at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:205)
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:168)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:81)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:812)
at com.datastax.driver.core.Cluster$Manager.access$100(Cluster.java:739)
at com.datastax.driver.core.Cluster.<init>(Cluster.java:82)
at com.datastax.driver.core.Cluster.<init>(Cluster.java:67)
at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:708)
at KerberosAuthenticatedClient.connect(KerberosAuthenticatedClient.java:19)
at KerberosAuthenticatedClient.main(KerberosAuthenticatedClient.java:45)
Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user
at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Unknown Source)
at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Unknown Source)
at com.sun.security.auth.module.Krb5LoginModule.login(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at javax.security.auth.login.LoginContext.invoke(Unknown Source)
at javax.security.auth.login.LoginContext.access$000(Unknown Source)
at javax.security.auth.login.LoginContext$4.run(Unknown Source)
at javax.security.auth.login.LoginContext$4.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(Unknown Source)
at javax.security.auth.login.LoginContext.login(Unknown Source)
at com.datastax.driver.core.sasl.KerberosAuthenticator.loginSubject(KerberosAuthenticator.java:109)
... 16 more
The error in the stacktrace is complaining that it is unable to find any credentials for the OS user executing the request. This is usually cause by one of three conditions:
not setting the location of the JAAS config correctly (which causes us to fall back to the default of using the TGT cache & often leading to 2.)
an empty local ticket cache (when not using a keytab)
credentials for the specified principal missing from the keytab
The first thing I'd check is that you have set the java.security.auth.login.config system property to the location of your JAAS config file (the DseClient file) correctly?
If you have and are still seeing the error, you can check the contents of the keytab with klist -e -t -k /path/to/keytab
Another useful tip for debugging is to set -Dsun.security.krb5.debug=true which will dump lots of detailed info to stdout.
I changed the service principle to dse/hostname#REALM from cassandra/hostname#REALM and it worked. I am not sure why the java program which I ran from my windows machine was taking the service principle as dse/hostname instead of cassandra/hostname

Resources