When I run the integration test for code which calls JPA repository within a new thread, I'm getting data that was populated during starting PostgreSQLContainer and I can't receive data from the script above class test( #Sql(scripts ="data.sql").
But when I remove #Transactional annotation above the test I can get data both from SQL script from test and test container.
My question is it possible to get data in a multithreading environment from test script without removing #Transactional annotation?
Thank you for your answer!
Application stack: Spring boot 2.1v+ test containers PostgreSQL 1.10.3v+ JUnit 4.12v
DB testcontainers config
#TestConfiguration
public class DatabaseTestConfig {
private static JdbcDatabaseContainer PSQL;
static {
PSQL = (PostgreSQLContainer) new PostgreSQLContainer("mdillon/postgis:9.4").withUsername("test")
.withPassword("test")
.withDatabaseName("test");
PSQL.start();
Arrays.asList("main_data.sql")
.forEach(DatabaseTestConfig::restoreDump);
/*
set db properties
*/
}
public void restoreDump(String fileName){
/*
insert sql data
PSQL.copyFileToContainer(fileName)...
*/
}
}
Base Integration Test class
#RunWith(SpringRunner.class)
#SpringBootTest(classes = { DatabaseTestConfig.class, ProjectApplication.class })
#ActiveProfiles("test-int")
#AutoConfigureMockMvc
#Sql(scripts = "classpath:extra_data.sql") // insert some extra data for all integration tests
public abstract class AbstractIntTest {
#Autowired
protected MockMvc mockMvc;
Integration Test that calls service where everething happenes
#Transactional
public class SomeIntegrationTest extends AbstractIntTest {
#Before
public void setUp() throws IOException {
//...
}
#Test
public void callServiceTest() throws Exception {
//mockMvc.perform(post(ENDPOINT_URL)
}
Service with simplified logic
#Service
#AllArgsConstructor
public class SomeService {
private final SomeJpaReporistory repo;
private final ExecutorService executor;
#Override
#Transactional
public SomeData call(){
return CompletableFuture.supplyAsync(() -> {
return repo.findAll();
}, executor).exceptionally(e -> {
throw new BadRequestException(e.getMessage());
});
}
When you make the test transactional, the SQL queries in extra_data.sql are performed in a transaction. That transaction is bound to a particular thread and is begun before execution of the test method and rolled back after the test method has completed:
Begin transaction
Execute extra_data.sql
Invoke test method
Roll back transaction
In step 3 you are calling repo.findAll() on a separate thread due to your service's use of supplyAsync. As a transaction is bound to a particular thread, this findAll() call is not part of the transaction in which extra_data.sql was executed. To be able to read the data added by extra_data.sql, it would have to be able to read uncommitted changes and perform a dirty read. Postgres does not support the read uncommitted isolation level so this isn't possible.
You'll need to revisit how you're populating your database with test data or your use of transactions in your tests. Perhaps you could apply extra_data.sql to the database in the same manner as main_data.sql so that it's always in place before any tests are executed and before any transactions are begun.
This is how I've solved this problem:
#Test
#Transactional
#Sql(scripts = "/db/extra_data.sql",
config = #SqlConfig(transactionMode = SqlConfig.TransactionMode.ISOLATED))
void test() {
// extra_data.sql are executed before this test is run.
}
Related
i am making cron job like loop to do something using new thread.
when module stop, this thread keeps running, so when i deployed updated module, i'm afraid it will make duplicate thread doing similar task
#Component(immediate = true, service = ExportImportLifecycleListener.class)
public class StaticUtils extends Utils{
private StaticUtils() {}
private static class SingletonHelper{
private static final StaticUtils INSTANCE = new StaticUtils();
}
public static StaticUtils getInstance() {
return SingletonHelper.INSTANCE;
}
}
public class Utils extends BaseExportImportLifecycleListener{
public Utils() {
startTask();
}
protected Boolean CRON_START = true;
private void startTask() {
new Thread(new Runnable() {
public void run() {
while (CRON_START) {
System.out.println("test naon bae lah ");
}
}
}).start();
}
#Deactivate
protected void deactivate() {
CRON_START = false;
System.out.println(
"cron stop lah woooooooooooooooooy");
}
}
i'm using liferay 7
I have populated task that i store from db, so this thread is checking is there a task that it must do, then if it exist execute it.
I'm quite new in osgi and liferay. i've try to use scheduler and failed and also exportimportlifecycle listener but dont really get it yet
think again: Do you really need something to run all the time in the background, or do you just need some asynchronous processing in the background, when triggered? It might be better to start a background task as a one-off, that automatically terminates
Liferay provides an internal MessageBus, that you can utilize to listen to events and implement background processing, without the need for a custom thread
You're in the OSGi world, so you can utilize #Activate, #Modified, #Deactivate (from org.osgi.service.component.annotations) or use a org.osgi.framework.BundleActivator.
But, in general, it's preferable if you don't start your own thread
I am trying to override the transactional behaviour for a service method(someService.updateSomething() in the example) annotated with #Transactional annotation in Spring. To do so, from other class, I am using programmatic transactional code like the next:
#Service
public class MyServiceClass {
private TransactionTemplate transactionTemplate;
public MyClass (PlatformTransactionManager transactionManager) {
transactionTemplate = new TransactionTemplate(transactionManager);
}
#Transactional
public void someMethod(){
transactionTemplate.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
protected void doInTransactionWithoutResult(TransactionStatus status){
try{
someService.updateSomething();
}catch(Exception e){
LOGGER.error("Error has ocurred");
}
}
});
}
}
My problem is that someService.updateSomething() does not run in a new Transaction and I dont understand why. So:
If I call a proxied service method with transactional behaviour like someService.updateSomething() but in the call I create a new transaction like in the example, when the code hits to the proxied method, it will take the new transaction created and not the transaction already running for the someMethod() method, right?
Thanks!
The task is to call a database, retrieve certain records update and save them.
As the amount of records if fairly large we want to do this Async, however, this doesn't seem to be implemented correctly.
The main class:
#SpringBootApplication
#EnableAsync
MainApplication() {
#Bean("threadPoolExecutor")
public TaskExecutor getAsyncExecutor(){
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(DataSourceConfig.getTHREAD_POOL_SIZE());
executor.setMaxPoolSize(DataSourceConfig.getTHREAD_POOL_SIZE());
executor.setWaitForTasksToCompleteOnShutdown(true);
executor.setThreadNamePrefix("RetryEnhancement-");
return executor;
}
}
Method in the first service:
#Service
public class FirstService() {
#Transactional
public void fullProcess() {
for(int counter = 0; counter < ConfigFile.getTHREADS(); counter++){
secondaryService.threads();
}
}
}
Method in the second service:
#Service
public class SecondService () {
#Async("threadPoolExecutor")
public void threads() {
while(thirdService.threadMethod()) {
//doNothing
}
}
}
Method in the third service:
#Service
public class ThirdService() {
#Transactional
public boolean threads() {
Record record = repository.fetchRecord();
if(record!=null) {
updateRecord(record);
saveRecord(record);
return true;
} else {
return false;
}
}
}
Repository:
public interface repository extends CrudRepository<Record, long> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
Record fetchRecord();
}
The issue I'm finding is that, while the code executes perfectly fine, it seems to have a Synchronous execution (found by adding a .sleep and watching the execution in the logger).
The seperate threads seem to be waiting until the other is executed.
I'm probably doing something wrong and if another thread already explains the issue, than please refer it, though I have not been able to find this issue in a different thread.
Your solution is way to complex. Ditch all of that and just inject the TaskExecutor and do the updateRecord in a separate thread (you might need to retrieve it again as you are now using a different thread and thus connection.
Something like this should do the trick
private final TaskExecutor executor; // injected through constructor
public void process() {
Stream<Record> records = repository.fetchRecords(); // Using a stream gives you a lazy cursor!
records.forEach(this::processRecord);
}
private void processRecord(Record record) {
executor.submit({
updateRecord(record);
saveRecord(record);
});
}
You might want to put the processRecord into another object and make it #Transactional or wrap it in a TransactionTemplate to get that behavior.
I have a strange problem:
Say I have two Entity classes, e.g. Container and ContainedObject. Container has a #OneToMany relationship to ContainedObject.
When getting the contained objects directly, everything works fine, but when running from a background thread, I get "could not initialize proxy - no Session".
Example:
#Component
public class Something {
#Autowired
private ContainerRepository _repo;
public void doInForeground() {
Container container = _repo.findOne(42L); // suppose 42 exists
container.getContainedObjects().size(); // succeeds
}
public void submitToBackground() {
CompletableFuture<Void> f = CompletableFuture.supplyAsync(() -> doInBackground());
}
private Void doInBackground() {
Container container = _repo.findOne(42L);
container.getContainedObjects().size(); // throws LazyInitializationException
}
}
The ContainerRepository is a PagingAndSortingRepository. Running doInForegroundsucceeds, but submitToBackground throws the well-known LazyInitializationException "could not initialize proxy - no Session"
It seems as if no transaction is started or it is closed too fast. Annotating any of the methods with #Transactional does not help either.
Is it simply not possible to use Spring-JPA in background threads or is any additional magic needed?
1) You need a readOnly transaction to get lazy collections.
2) I suppose that #Transactional doesn't work on your local methods because when you call a local method you call the implementation directly, not a proxy object created by Spring.
If it's the case there are at least 2 choices:
1) Switch to Aspectj
<tx:annotation-driven mode="aspectj" proxy-target-class="true"/>
2) Create a transaction manually using TransactionTemplate inside doInForeground():
http://simplespringtutorial.com/springProgrammaticTransactions.html
i'm just running into a complicated Problem as i began to unit-Test some Controller-Methods in a Spring FW4 based Java Application.
My ApplicationConfig.java is annotated with #Configuration and #EnableTransactionManagement(proxyTargetClass = true) and a public Controller method, which i created to save a new object of a simple entity-class is testet with the following ControllerTestClass
#RunWith(SpringJUnit4ClassRunner.class)
#WebAppConfiguration
#ContextConfiguration(locations = {"classpath:/test-context.xml"})
#TransactionConfiguration(defaultRollback = true, transactionManager = "annotationDrivenTransactionManager")
public class TestController
#Autowired
public MyClassService myClassServiceMock;
protected MockMvc mockMvc;
#Autowired
protected WebApplicationContext webApplicationContext;
#Before
public void setUp() throws Exception {
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext).build();
};
#org.junit.Before
public void reset() {
Mockito.reset(myClassServiceMock);
}
#org.junit.After
public void after() {
verifyNoMoreInteractions(myClassServiceMock);
}
#Test
public void testSaveObject() throws Exception {
MyObject object = new MyObjectBuilder().withName("object").withDate("2014-08-15").build();
when(myClassServiceMock.createObject(objectName, objectDate)).thenReturn(object);
[.. mockMvcTest which works ... ]
verify(myclassServiceMock, times(1)).createObject(objectName, objectDate);
}
}
The following part of the debug.log is something I can't figure out the reason for the problem, but when i remove the #EnableTransactionManager-Annotation, no error occures...
2014-08-15_17:25:59.608 [main] DEBUG o.s.orm.jpa.JpaTransactionManager - Creating new transaction with name [a.b.c.MyClassService$$EnhancerByMockitoWithCGLIB$$cf62a86c.saveObject]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT; ''
2014-08-15_17:25:59.608 [main] DEBUG o.s.orm.jpa.JpaTransactionManager - Opened new EntityManager [org.hibernate.jpa.internal.EntityManagerImpl#513f39c] for JPA transaction
2014-08-15_17:25:59.616 [main] DEBUG o.s.orm.jpa.JpaTransactionManager - Exposing JPA transaction as JDBC transaction [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#320cac01]
2014-08-15_17:25:59.618 [main] DEBUG o.s.orm.jpa.JpaTransactionManager - Initiating transaction commit
2014-08-15_17:25:59.618 [main] DEBUG o.s.orm.jpa.JpaTransactionManager - Committing JPA transaction on EntityManager [org.hibernate.jpa.internal.EntityManagerImpl#513f39c]
2014-08-15_17:25:59.633 [main] DEBUG o.s.orm.jpa.JpaTransactionManager - Closing JPA EntityManager [org.hibernate.jpa.internal.EntityManagerImpl#513f39c] after transaction
2014-08-15_17:25:59.633 [main] DEBUG o.s.o.jpa.EntityManagerFactoryUtils - Closing JPA EntityManager
2014-08-15_17:25:59.635 [main] DEBUG o.s.t.c.s.DirtiesContextTestExecutionListener - After test method: context [DefaultTestContext#8f72029 testClass = MyControllerTest, testInstance = a.b.c.MyControllerTest#453204e6, testMethod = testSaveObject#MyClassControllerTest, testException = org.mockito.exceptions.misusing.UnfinishedVerificationException:
Missing method call for verify(mock) here:
-> at a.b.c.MyClassService$$FastClassBySpringCGLIB$$809f2bf.invoke(<generated>)
Example of correct verification:
verify(mock).doSomething()
I would really appreciate some help, because my oddisee and research already is going on for a couple of days :(
Thanks
Firstly, you aren't initializing Mockito in your tests. Read this post
You need to call MockitoAnnotations.initMocks() because you are already using #RunWith(SpringJUnit4ClassRunner.class) and you can only specify one runner on a Class.
#Before
public void reset() {
MockitoAnnotations.initMocks(this);
// Mockito.reset(myClassServiceMock); <= remove this line
}
I think you also want to use #Mock instead of #Autowired for this mock so that you have an instance of a Mockito mock that you can then call verify() on later. You will also have to inject myClassServiceMock into your class under test (i.e. the Controller)
#Mock
public MyClassService myClassServiceMock;
You can remove the call to Mockito.reset() as #Mock will create a new instance for each test method.
If you were intending to use #Autowired and retrieve the instance of MyClassService from your application context then you won't be able to call any Mockito methods like verify() on it.
I would also expect that #TransactionConfiguration is not required because you are never hitting your database (because you are mocking out your service layer), so you can remove it. If you are hitting your database in your test, then that's a different story, but I can't tell that from the code you have provided.