Is there anywhere to report bugs/ request features in ServiceStack?
While using ServiceStack, my ServiceStack.ServiceInterface.Service object was throwing this error:
ExecuteReader requires an open and available Connection. The connection's current state is closed.
The Service class includes a Db property (used in examples), which is a IDbConnection - db connections are not thread safe.
I'm interested to know why this non thread safe method of access a database is included in the Service class. It's no good for servicing multiple web service requests.
Service.cs will try to resolve an IDbConnectionFactory that will create a new IDbConnection for you, so there isn't a thread safety issue here.
If you'd like to handle it differently, you can override it.
private IDbConnection db;
public virtual IDbConnection Db
{
get { return db ?? (db = TryResolve<IDbConnectionFactory>().OpenDbConnection()); }
}
Source:
https://github.com/ServiceStack/ServiceStack/blob/ada0f43012610dc9ee9ae863e77dfa36b7abea28/src/ServiceStack/Service.cs#L68
Edit:
Maybe it's not clear that OrmLiteConnectionFactories automatically create a new connection in conjunction with an OpenDbConnection call, but they do:
Source:
https://github.com/ServiceStack/ServiceStack.OrmLite/blob/db40347532a14441eba32e575bcf07f3b2f45cef/src/ServiceStack.OrmLite/OrmLiteConnectionFactory.cs#L72
Related
I have set my Domain Model objects to be independent of any service and infrastructure logic.
I am also using Domain Events to react to some changes in Domain Models.
Now my problem is how to raise those events from the Domain Model objects itself.
Currently I am using Udi Dahan's DomainEvents static class for this (I need evens to be handled exactly when they happen and not at a latter time).
The events are used for many things, like logging, updating the data in related services and other Domain Model objects and db, publishing messages to the MassTransit bus etc.
The DomainEvents static class uses Autofac scope that I inject at some point in it, to find the IMediatr instance and to publish the events, like this:
public static class DomainEvents
{
private static ILifetimeScope Scope;
public async static Task RaiseAsync<TDomainEvent>(TDomainEvent #event) where TDomainEvent : IDomainEvent
{
var mediator = Scope?.Resolve<IMediatorBus>();
if (mediator != null)
{
await mediator!.Publish(#event).ConfigureAwait(false);
}
else
{
Debug.WriteLine("Mediator not set for DomainEvents!");
}
}
public static void SetScope(ILifetimeScope scope)
{
Scope = scope;
}
}
This all works ok in a single-threaded environment, but the method DomainEvents.SetScope() is a possible racing problem in multhi-threaded environment.
Ie. When I introduce MassTransit and create message consumers, each Message consumer will set the current LifetimeScope to DomainEvents by that method, and here is the problem, each consumer will overwrite the lifetime scope with the new one.
Why I use DomainEvents static class? Because I don't want to pollute my Domain Model Objects with infrastructure stuff.
I thought about making DomainEvents non static (define an interface), but then I need them injected in every Domain Model Object and I'm still thinking about this, but maybe there is a better way.
I want to know if there is a better way to handle this?
Maybe some change in DomainEvents class? Or maybe remove the DomainEvents static class end use an interface or DomainService to do this.
The problem is I don't like static classes, but I also don't like pushing non domain-specific dependencies into Domain Model Objects.
Please help.
UPDATE
To better clarify the process and for what I use DomainEvents...
I have a long-running process that can take from few minutes to few hours/days to complete.
So the process is going like this:
I receive an message from MassTransit ie ProcessStartMessage(processId)
Get the ProcessData for (processId) from Db.
Construct an in-memory Domain Model ProcessTracker (singleton) and put all the data I loaded from DB in it. (in-memory cache)
I receive another message from Masstransit ie. ProcessStatusChanged(processId, data).
Forward this message data to in-memory singleton ProcessTracker to process.
ProcessTracker process the data.
For ProcessTracker to be able to process this data it instantiates many Domain Model Objects, each responsible to process some part of the data. (Note there is NO more db calls and entity hydration from db, it all happens in memory, also Domain Model is not mapped to any entity, it is not connected to any db object).
At some point I need to log what a Domain Model object in the chain has done, has it work finished or started, has reached some milestone etc. This is done by raising DomainEvents. I also need to notify the GUI of those events, so they are used to send Masstransit messages too.
Ie.(pseudo code):
public class ProcessTracker
{
private Step _currentStep;
public void ProcessData(data)
{
_currentStep.ProcessData(data);
DomainEvents.Raise(new ProcesTrackerDataProcessed());
...
}
}
public class Step
{
public Phase _currentPhase;
public void ProcessData(data)
{
if (data.IsManual && _someOtherCondition())
{
DomainEvents.Raise(new StepDataEvent1());
...
}
if(data.CanTransition)
{
DomainEvents.Raise(new TransitionToNewPhase(this, data));
}
_currentPhase.DoSomeWork(data);
DomainEvents.Raise(new StepDataProcessed(this, data));
...
}
}
About db updates, those are not transactional and not important to the process and the Domain Model Object state is kept only in memory, if the process crash the process MUST begin from the start (there is NO recovery).
To end the process:
I receive ProcessEnd from the MassTransit
The message data is forwarded to the ProcessTracker
ProcessTracker handles the data an nets a result of the proceess
The result of the process is saved to db
A message is sent to other parties in the process that notifies them of a process completion.
Ask yourself first what are you going to do when you raise an event from your domain model?
Normally it works like this:
Get a command
Load a domain object from a repository
Execute behaviour
(here probably) Raise an event
Persist the new domain object state
So, where your extra domain event handlers would fit? Are you going to execute some other database calls, send an email? Remember that it all happens now, when you haven't even persisted the changed state of your domain object. What if your persistence fails? It will happen after you executed all the domain handlers.
You should not execute more than one transaction when you handle a single command. The Aggregate pattern clearly tells you that the aggregate is the transaction boundary. You should raise domain events after you complete the transaction, or within the same technical transaction, but it should only persist the aggregate state and the event. Domain event reactions potentially trigger transactions for other domain objects, and it should be done outside of the scope of handling the current command.
The issue is not at all technical, it is a design problem.
If you use MassTransit, you can only make it (relatively) reliable if you handle the command in a message consumer. Then, you can use in-memory outbox, which will not send an event unless the consumer succeeded. It is still not guaranteed that the event will be published in case of the broker failure.
Unless you go to Event Sourcing, you have two 100% reliable options:
Use a transactional outbox pattern (NServiceBus has one and it's quite complex). It has limitations on what type of database you use.
Store the event to the same database as the domain object, in a different table, within the same transaction. Poll the table with DELETE INTO and emit events to the broker from there.
I have a quarkus application with an async endpoint that creates an entity with default properties, starts a new thread within the request method and executes a long running job and then returns the entity as a response for the client to track.
#POST
#Transactional
public Response startJob(#NonNull JsonObject request) {
// create my entity
JobsRecord job = new JobsRecord();
// set default properties
job.setName(request.getString("name"));
// make persistent
jobsRepository.persist(job);
// start the long running job on a different thread
Executor.execute(() -> longRunning(job));
return Response.accepted().entity(job).build();
}
Additionally, the long running job will make updates to the entity as it runs and so it must also be transactional. However, the database entity just doesn't get updated.
These are the issues I am facing:
I get the following warnings:
ARJUNA012094: Commit of action id 0:ffffc0a80065:f2db:5ef4e1c7:0 invoked while multiple threads active within it.
ARJUNA012107: CheckedAction::check - atomic action 0:ffffc0a80065:f2db:5ef4e1c7:0 commiting with 2 threads active!
Seems like something that should be avoided.
I tried using #Transaction(value = TxType.REQUIRES_NEW) to no avail.
I tried using the API Approach instead of the #Transactional approach on longRunning as mentioned in the guide as follows:
#Inject UserTransaction transaction;
.
.
.
try {
transaction.begin();
jobsRecord.setStatus("Complete");
jobsRecord.setCompletedOn(new Timestamp(System.currentTimeMillis()));
transaction.commit();
} catch (Exception e) {
e.printStackTrace();
transaction.rollback();
}
but then I get the errors: ARJUNA016051: thread is already associated with a transaction! and ARJUNA016079: Transaction rollback status is:ActionStatus.COMMITTED
I tried both the declarative and API based methods again this time with context propagation enabled. But still no luck.
Finally, based on the third approach, I thought keeping the #Transactional on the Http request handler and leaving longRunning as is without declarative or API based transaction approaches would work. However the database still does not get updated.
Clearly I am misunderstanding how JTA and context propagation works (among other things).
Is there a way (or even a design pattern) that allows me to update database entities asynchronously in a quarkus web application? Also why wouldn't any of the approaches I took have any effect?
Using quarkus 1.4.1.Final with ext: [agroal, cdi, flyway, hibernate-orm, hibernate-orm-panache, hibernate-validator, kubernetes-client, mutiny, narayana-jta, rest-client, resteasy, resteasy-jackson, resteasy-mutiny, smallrye-context-propagation, smallrye-health, smallrye-openapi, swagger-ui]
You should return an async type from your JAX-RS resource method, the transaction context will then be available when the async stage executes. There is some relevant documentation in the quarkus guide on context propagation.
I would start by looking at the one of the reactive examples such as the getting started quickstart. Try annotating each resource endpoint with #Transactional and the async code will run with a transaction context.
I use Async with a method that call a remote service with Feign and I need to append an oauth2 token to the request, for that I use a RequestInterceptor.
#Bean
public RequestInterceptor requestTokenBearerInterceptor() {
return requestTemplate -> {
Object principal = SecurityContextHolder
.getContext()
.getAuthentication()
.getPrincipal();
if (!principal.equals("anonymousUser")) {
OAuth2AuthenticationDetails details = (OAuth2AuthenticationDetails)
SecurityContextHolder.getContext().getAuthentication().getDetails();
requestTemplate.header("Authorization", "bearer " + details.getTokenValue());
}
};
}
But when the requestInterceptor is used in another thread, I don't have acces to the same security context so getAuhentication return null.
I try to fix it in the executor configuration, I create a DelegatingSecurityContextExecutor wrapping the executor and the security context. But it seems that the bean is created in the 'main' thread and the security context is not the same used then, when a RestController method is executed, so the getAuthentication() still return null.
#Bean(name = "asyncExecutor")
public Executor asyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(3);
executor.setMaxPoolSize(3);
executor.setQueueCapacity(100);
executor.setThreadNamePrefix("AsynchThread-");
executor.initialize();
Executor wrappedExecutor = new DelegatingSecurityContextExecutor(executor, SecurityContextHolder.getContext());
return wrappedExecutor;
}
How can I configure the executor the right way ?
I finally found the solution, it is possible to propagate the security context automaticly to the other threads.
Just add this line of code in the static main method of your spring boot application :
SecurityContextHolder.setStrategyName(SecurityContextHolder.MODE_INHERITABLETHREADLOCAL);
The solution is well explained here : https://www.baeldung.com/spring-security-async-principal-propagation?fbclid=IwAR1zeGKvRvBb7GG8SmxO4x8-NlKkG39Q29WoLKxZ8NRzyKEcnDWx4Q6EUk0
!! WARNING !! : I noticed an unexpected behaviour with that solution, at least on my local dev environment. I'm connected to my app with two different accounts using sessionbox tool of chrome (same with different browsers), and it seems that when I 'm connected with user A SecurityContextHolder.getContext().getAuthentication().getPrincipal() return the security context of user B ... So Huge security problem ! I'm looking for a solution at the moment.
Reading this post : How to set up Spring Security SecurityContextHolder strategy? the solution seems to be here Spring Security and #Async (Authenticated Users mixed up)
It seems to me you can not use RequestInterceptor here. As far as I know when you use #Async you lose request context in the method which you want to execute in asynchronous way. To do so you have to explicitly pass access token to asynchronous method and provide it as request header:
#FeignClient(name = "userClient", url ="${userService.hostname}")
public interface MyFeignClient {
String AUTH_TOKEN = “Authorization”;
#GetMapping(“/users”)
List<User> findUsers(#RequestHeader(AUTH_TOKEN) String bearerToken);
}
I have an azure function that runs off of a queue trigger. The repository has method to grab the connection string from the ConnectionStrings collection.
return System.Configuration.ConfigurationManager.ConnectionStrings["MyDataBase"].ToString();
This works great for the most part but I see intermittently that this returns a null exception error.
Is there a way I can make this more robust?
Do azure functions sometimes fail to get the settings?
Should I store the setting in a different section?
I also want to say that this runs thousands of times a day but I see this popup about a 100 times.
Runtime version: 1.0.12299.0
Are you reading the configuration for every function call? You should consider reading it once (e.g. using a Lazy<string> and static) and reusing it for all function invocations.
Maybe there is a concurrency issue when multiple threads access the code. Putting a lock around the code could help as well. ConfigurationManager.ConnectionStrings should be tread-safe, but maybe it isn't in the V1 runtime.
A similar problem was posted here, but this concerned app settings and not connection strings. I don't think using CloudConfigurationManager should be the correct solution.
You can also try putting the connection string into the app settings, unless you are using Entity Framework.
Connection strings should only be used with a function app if you are using entity framework. For other scenarios use App Settings. Click to learn more.
(via Azure Portal)
Not sure if this applies to the V1 runtime as well.
The solution was to add a private static string for the connection string. Then only read from the configuration if it fails. I then added a retry that paused half a second. This basically removed this from happening.
private static string connectionString = String.Empty;
private string getConnectionString(int retryCount)
{
if (String.IsNullOrEmpty(connectionString))
{
if (System.Configuration.ConfigurationManager.ConnectionStrings["MyEntity"] != null)
{
connectionString = System.Configuration.ConfigurationManager.ConnectionStrings["MyEntity"].ToString();
}
else
{
if (retryCount > 2)
{
throw new Exception("Failed to Get Connection String From Application Settings");
}
retryCount++;
getConnectionString(retryCount);
}
}
return connectionString;
}
I don't know if this perfect but it works. I went from seeing this exception 30 times a day to none.
I have been trying to add parallel foreach in existing application and faced an odd issue
Application Architecture
Controller resolve BO layer
BO Layer resolve Service Layer
Service Layer resolve UOW and Repository Layer
UOW layer resolve DB Connection
BO Layer
private IUserService _userService;
public BOUser(IUserService userService) => _userService=userService;
public void AddUser(User user) => _userService.addUser(user);
Service Layer
private IUnitofWork _uow;
private IUserRepository _userRepo;
UserService(IUOW _uow, IUserRepository _userRepo)
{
uow = _uow;
_userRepo.uow = _uow;
}
public void AddUser (User user) {
_uow.BeginTransaction();
_userRepo.Add(user);
_uow.CommitTransaction();
}
Repo Layer
public IUnitOfWork UnitOfWork { get; set; }
public void Add(user){
UnitOfWork.Connection.Insert<UserContact>(userContact, UnitOfWork.Transaction);
}
Unit of Work
public UnitOfWork(IDbConnection connection)
{
Connection = connection; // responsible for creating new connection
}
This works fine as of today but as i tried to add multiple user using parallel.foreach in BO layer i got one failed and one success
Reason behind is service layer is initiated when BO layer initiated and UOW.connection remains to be 1 for whole process
when i tried multiple threads because Connection was one object that was shared between multiple threads
it failed because one thread completed the work and closed Connection object
i have solution in mind by removing UOW from Constructor and by using service location pattern that would be
IUserSrevice userService = new UserService();
that will create multiple connection object based on threads.
its not rite way of doing it. Any expert opinion will help
Connection is not a thread safe object because the idea is to create a pool of connection objects and have each thread borrow a connection from the pool, do its work and return it to the pool once done.
Thus no more than one thread can access a connection object at a time.
You should inject the connection pool and not the individual connection objects.