I am trying to override the transactional behaviour for a service method(someService.updateSomething() in the example) annotated with #Transactional annotation in Spring. To do so, from other class, I am using programmatic transactional code like the next:
#Service
public class MyServiceClass {
private TransactionTemplate transactionTemplate;
public MyClass (PlatformTransactionManager transactionManager) {
transactionTemplate = new TransactionTemplate(transactionManager);
}
#Transactional
public void someMethod(){
transactionTemplate.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
protected void doInTransactionWithoutResult(TransactionStatus status){
try{
someService.updateSomething();
}catch(Exception e){
LOGGER.error("Error has ocurred");
}
}
});
}
}
My problem is that someService.updateSomething() does not run in a new Transaction and I dont understand why. So:
If I call a proxied service method with transactional behaviour like someService.updateSomething() but in the call I create a new transaction like in the example, when the code hits to the proxied method, it will take the new transaction created and not the transaction already running for the someMethod() method, right?
Thanks!
Related
In my Spring Boot project, I have two JMS listeners listening to one queue. All messages received from the queue have to be processed in the same way and persisted / updated in the database (Oracle). Currently, I have a synchronized method in a class that is doing the parsing of the messages. As expected, all thread read messages simultaneously, but parsing is done one by one as the method (parseMessage()) is synchronized. What I want is to parse the messages simultaneously and do database operations as well.
How can I solve this?
I don't want to create two different classes with the same code and use #Qualifier to call different classes in each listener, as the code for parsing the message is the same.
The ideal solution, I think, is to do database operations using a new synchronized method in a new class, but parsing the message in a multi-threaded way. So, at a time only one thread can say persist / update. When a thread is not waiting to persist / update, it continues the parsing on its own thread.
Please correct me if I am wrong or if you find the optimal solution. Let me know if any other info is needed.
JMS Controller Class
#RestController
#EnableJms
public class JMSController {
#Autowired
private IParseMapXml iParseMapXml;
#JmsListener(destination = "${app.jms_destinaltion}")
public void receiveMessage1(String recvMsg) {
try {
InputSource is = new InputSource(new StringReader(recvMsg.replaceAll("&", "&")));
Document doc = new SAXReader().read(is);
iParseMapXml.parseMessage(doc);
} catch (Exception e) {
}
}
#JmsListener(destination = "${app.jms_destinaltion}")
public void receiveMessage2(String recvMsg) {
try {
InputSource is = new InputSource(new StringReader(recvMsg.replaceAll("&", "&")));
Document doc = new SAXReader().read(is);
iParseMapXml.parseMessage(doc);
} catch (Exception e) {
}
}
}
Parse XML Interface
public interface IParseMapXml {
public void parseMessage(Document doc);
}
Parsing Implementation
public class ParsingMessageClass implements IParseMapXml{
#Override
#Transactional
synchronized public void parseMessage(Document doc) {
// TODO Auto-generated method stub
....
PROCESS DATA/MESSAGE
....
DO DB OPERATIONS
}
}
The task is to call a database, retrieve certain records update and save them.
As the amount of records if fairly large we want to do this Async, however, this doesn't seem to be implemented correctly.
The main class:
#SpringBootApplication
#EnableAsync
MainApplication() {
#Bean("threadPoolExecutor")
public TaskExecutor getAsyncExecutor(){
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(DataSourceConfig.getTHREAD_POOL_SIZE());
executor.setMaxPoolSize(DataSourceConfig.getTHREAD_POOL_SIZE());
executor.setWaitForTasksToCompleteOnShutdown(true);
executor.setThreadNamePrefix("RetryEnhancement-");
return executor;
}
}
Method in the first service:
#Service
public class FirstService() {
#Transactional
public void fullProcess() {
for(int counter = 0; counter < ConfigFile.getTHREADS(); counter++){
secondaryService.threads();
}
}
}
Method in the second service:
#Service
public class SecondService () {
#Async("threadPoolExecutor")
public void threads() {
while(thirdService.threadMethod()) {
//doNothing
}
}
}
Method in the third service:
#Service
public class ThirdService() {
#Transactional
public boolean threads() {
Record record = repository.fetchRecord();
if(record!=null) {
updateRecord(record);
saveRecord(record);
return true;
} else {
return false;
}
}
}
Repository:
public interface repository extends CrudRepository<Record, long> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
Record fetchRecord();
}
The issue I'm finding is that, while the code executes perfectly fine, it seems to have a Synchronous execution (found by adding a .sleep and watching the execution in the logger).
The seperate threads seem to be waiting until the other is executed.
I'm probably doing something wrong and if another thread already explains the issue, than please refer it, though I have not been able to find this issue in a different thread.
Your solution is way to complex. Ditch all of that and just inject the TaskExecutor and do the updateRecord in a separate thread (you might need to retrieve it again as you are now using a different thread and thus connection.
Something like this should do the trick
private final TaskExecutor executor; // injected through constructor
public void process() {
Stream<Record> records = repository.fetchRecords(); // Using a stream gives you a lazy cursor!
records.forEach(this::processRecord);
}
private void processRecord(Record record) {
executor.submit({
updateRecord(record);
saveRecord(record);
});
}
You might want to put the processRecord into another object and make it #Transactional or wrap it in a TransactionTemplate to get that behavior.
When I run the integration test for code which calls JPA repository within a new thread, I'm getting data that was populated during starting PostgreSQLContainer and I can't receive data from the script above class test( #Sql(scripts ="data.sql").
But when I remove #Transactional annotation above the test I can get data both from SQL script from test and test container.
My question is it possible to get data in a multithreading environment from test script without removing #Transactional annotation?
Thank you for your answer!
Application stack: Spring boot 2.1v+ test containers PostgreSQL 1.10.3v+ JUnit 4.12v
DB testcontainers config
#TestConfiguration
public class DatabaseTestConfig {
private static JdbcDatabaseContainer PSQL;
static {
PSQL = (PostgreSQLContainer) new PostgreSQLContainer("mdillon/postgis:9.4").withUsername("test")
.withPassword("test")
.withDatabaseName("test");
PSQL.start();
Arrays.asList("main_data.sql")
.forEach(DatabaseTestConfig::restoreDump);
/*
set db properties
*/
}
public void restoreDump(String fileName){
/*
insert sql data
PSQL.copyFileToContainer(fileName)...
*/
}
}
Base Integration Test class
#RunWith(SpringRunner.class)
#SpringBootTest(classes = { DatabaseTestConfig.class, ProjectApplication.class })
#ActiveProfiles("test-int")
#AutoConfigureMockMvc
#Sql(scripts = "classpath:extra_data.sql") // insert some extra data for all integration tests
public abstract class AbstractIntTest {
#Autowired
protected MockMvc mockMvc;
Integration Test that calls service where everething happenes
#Transactional
public class SomeIntegrationTest extends AbstractIntTest {
#Before
public void setUp() throws IOException {
//...
}
#Test
public void callServiceTest() throws Exception {
//mockMvc.perform(post(ENDPOINT_URL)
}
Service with simplified logic
#Service
#AllArgsConstructor
public class SomeService {
private final SomeJpaReporistory repo;
private final ExecutorService executor;
#Override
#Transactional
public SomeData call(){
return CompletableFuture.supplyAsync(() -> {
return repo.findAll();
}, executor).exceptionally(e -> {
throw new BadRequestException(e.getMessage());
});
}
When you make the test transactional, the SQL queries in extra_data.sql are performed in a transaction. That transaction is bound to a particular thread and is begun before execution of the test method and rolled back after the test method has completed:
Begin transaction
Execute extra_data.sql
Invoke test method
Roll back transaction
In step 3 you are calling repo.findAll() on a separate thread due to your service's use of supplyAsync. As a transaction is bound to a particular thread, this findAll() call is not part of the transaction in which extra_data.sql was executed. To be able to read the data added by extra_data.sql, it would have to be able to read uncommitted changes and perform a dirty read. Postgres does not support the read uncommitted isolation level so this isn't possible.
You'll need to revisit how you're populating your database with test data or your use of transactions in your tests. Perhaps you could apply extra_data.sql to the database in the same manner as main_data.sql so that it's always in place before any tests are executed and before any transactions are begun.
This is how I've solved this problem:
#Test
#Transactional
#Sql(scripts = "/db/extra_data.sql",
config = #SqlConfig(transactionMode = SqlConfig.TransactionMode.ISOLATED))
void test() {
// extra_data.sql are executed before this test is run.
}
I want to clear one doubt. I am creating my own service class to manipulate data. Should i create this service instance at class level(in MVCPortlet) or create new instance in my processAction/doView method.
Is there is any issue of thread safety while using instance level.
Fg:
public class MvcCycle extends MVCPortlet {
int counter;
LdapService ldapservice;
#Override
public void init() throws PortletException {
counter=0;
ldapservice = new LdapService(); // Option 1
super.init();
}
#Override
public void doView(RenderRequest renderRequest, RenderResponse renderResponse) throws IOException, PortletException {
// TODO Auto-generated method stub
System.out.println("Counter hits "+ ++counter);
//
LdapService ldapservice = new LdapService(); // Option 2
ldapservice.authUser(request.getParameter("email"));
// -- some code--
super.doView(renderRequest, renderResponse);
}
#Override
public void destroy() {
System.out.println("Last counter "+counter);
counter=0;
super.destroy();
}
}
class LdapService{
public boolean authUser(String email){
if(//logic to authenticate user){
return true;
}else{
return false;
}
}
}
Here, my ldapservice initiate only once. So when multiple hits come. which code is beneficial on this time either Option1 or Option 2.
Hope i have cleared my problem.
Following up on your comment: in general: it depends. We don't have enough information to give you blank architectural recommendations.
In the context of Liferay it could make sense to utilize service builder, even if you don't have database interaction. Just create an empty entity, you'll have a service with no persistence.
<entity name="MyService" local="true"/>
That's it. Now implement MyServiceLocalServiceImpl added you're done
Edit, after your clarification: it doesn't make a difference. I'd still recommend to not instantiate at all in your portlet, because that makes your portlet dependent on the service implementation. E.g. I still go with service builder.
You're also asking about thread safety and that depends on your implementation as well. Your sample code is thread safe, but the actual code might not be. Your judgement.
I try to realize the following workflow with Spring Integration:
1) Poll REST API
2) store the POJO in Cassandra cluster
It's my first try with Spring Integration, so I'm still a bit overwhelmed about the mass of information from the reference. After some research, I could make the following work.
1) Poll REST API
2) Transform mapped POJO JSON result into a string
3) save string into file
Here's the code:
#Configuration
public class ConsulIntegrationConfig {
#InboundChannelAdapter(value = "consulHttp", poller = #Poller(maxMessagesPerPoll = "1", fixedDelay = "1000"))
public String consulAgentPoller() {
return "";
}
#Bean
public MessageChannel consulHttp() {
return MessageChannels.direct("consulHttp").get();
}
#Bean
#ServiceActivator(inputChannel = "consulHttp")
MessageHandler consulAgentHandler() {
final HttpRequestExecutingMessageHandler handler =
new HttpRequestExecutingMessageHandler("http://localhost:8500/v1/agent/self");
handler.setExpectedResponseType(AgentSelfResult.class);
handler.setOutputChannelName("consulAgentSelfChannel");
LOG.info("Created bean'consulAgentHandler'");
return handler;
}
#Bean
public MessageChannel consulAgentSelfChannel() {
return MessageChannels.direct("consulAgentSelfChannel").get();
}
#Bean
public MessageChannel consulAgentSelfFileChannel() {
return MessageChannels.direct("consulAgentSelfFileChannel").get();
}
#Bean
#ServiceActivator(inputChannel = "consulAgentSelfFileChannel")
MessageHandler consulAgentFileHandler() {
final Expression directoryExpression = new SpelExpressionParser().parseExpression("'./'");
final FileWritingMessageHandler handler = new FileWritingMessageHandler(directoryExpression);
handler.setFileNameGenerator(message -> "../../agent_self.txt");
handler.setFileExistsMode(FileExistsMode.APPEND);
handler.setCharset("UTF-8");
handler.setExpectReply(false);
return handler;
}
}
#Component
public final class ConsulAgentTransformer {
#Transformer(inputChannel = "consulAgentSelfChannel", outputChannel = "consulAgentSelfFileChannel")
public String transform(final AgentSelfResult json) throws IOException {
final String result = new StringBuilder(json.toString()).append("\n").toString();
return result;
}
This works fine!
But now, instead of writing the object to a file, I want to store it in a Cassandra cluster with spring-data-cassandra. For that, I commented out the file handler in the config file, return the POJO in transformer and created the following, :
#MessagingGateway(name = "consulCassandraGateway", defaultRequestChannel = "consulAgentSelfFileChannel")
public interface CassandraStorageService {
#Gateway(requestChannel="consulAgentSelfFileChannel")
void store(AgentSelfResult agentSelfResult);
}
#Component
public final class CassandraStorageServiceImpl implements CassandraStorageService {
#Override
public void store(AgentSelfResult agentSelfResult) {
//use spring-data-cassandra repository to store
LOG.info("Received 'AgentSelfResult': {} in Cassandra cluster...");
LOG.info("Trying to store 'AgentSelfResult' in Cassandra cluster...");
}
}
But this seems to be a wrong approach, the service method is never triggered.
So my question is, what would be a correct approach for my usecase? Do I have to implement the MessageHandler interface in my service component, and use a #ServiceActivator in my config. Or is there something missing in my current "gateway-approach"?? Or maybe there is another solution, that I'm not able to see..
Like mentioned before, I'm new to SI, so this may be a stupid question...
Nevertheless, thanks a lot in advance!
It's not clear how you are wiring in your CassandraStorageService bean.
The Spring Integration Cassandra Extension Project has a message-handler implementation.
The Cassandra Sink in spring-cloud-stream-modules uses it with Java configuration so you can use that as an example.
So I finally made it work. All I needed to do was
#Component
public final class CassandraStorageServiceImpl implements CassandraStorageService {
#ServiceActivator(inputChannel="consulAgentSelfFileChannel")
#Override
public void store(AgentSelfResult agentSelfResult) {
//use spring-data-cassandra repository to store
LOG.info("Received 'AgentSelfResult': {}...");
LOG.info("Trying to store 'AgentSelfResult' in Cassandra cluster...");
}
}
The CassandraMessageHandler and the spring-cloud-streaming seemed to be a to big overhead to my use case, and I didn't really understand yet... And with this solution, I keep control over what happens in my spring component.