Increase Servicestack Concurrent Process - servicestack

Question

I have following Service:
public class ServerApp : AppHostHttpListenerPoolBase
{
public ServerApp() : base("Server", 500,
typeof(TestService).Assembly)
{
}
public override void Configure(Container container)
{
ThreadsPerProcessor = 50;
}
}
public class TestService : Service
{
private readonly Logger _logger = LogManager.GetCurrentClassLogger();
public object Any(Hello hello)
{
_logger.Info("Received: " + hello.Name);
var waitingMinutes = new Random().Next(1, 10);
Thread.Sleep(TimeSpan.FromMinutes(waitingMinutes));
_logger.Info("Response: " + hello.Name);
return new GoodBye(){Message = "Bye Bye " + hello.Name};
}
}
}
and I have simple test project to push Parallel request to the Service (and push all is ok), but Service only process 2 requests at a time. When a request has been processed, the next request should be processed.
How can I increase the concurrent process?

Answer 1

ServiceStack's AppHostHttpListenerPoolBase does execute Service concurrently by executing Services on new ThreadPool Threads.
To demonstrate this I've created a stand-alone example which executes 50 concurrent requests sleeping between 30-60s that calls the stand-alone test based on your Example in ConcurrencyTest.cs calling this Service:
public class SleepTest : IReturn<SleepTestResponse>
{
public string Name { get; set; }
public int WaitingSecs { get; set; }
}
public class SleepTestResponse
{
public string Message { get; set; }
public ResponseStatus ResponseStatus { get; set; }
}
public class TestConcurrencyService : Service
{
public object Any(SleepTest request)
{
var sw = Stopwatch.StartNew();
Thread.Sleep(TimeSpan.FromSeconds(request.WaitingSecs));
return new SleepTestResponse
{
Message = $"{request.Name} took {sw.Elapsed.TotalSeconds} secs",
};
}
}
Which is called with this client load test executing 50 concurrent requests waiting between 30-60s:
var rand = new Random();
var client = new JsonHttpClient(Config.AbsoluteBaseUri);
client.GetHttpClient().Timeout = TimeSpan.FromMinutes(5);
long responsesReceived = 0;
long totalSecondsWaited = 0;
var sw = Stopwatch.StartNew();
const int ConcurrentRequests = 50;
ConcurrentRequests.Times(i =>
{
Interlocked.Increment(ref responsesReceived);
ThreadPool.QueueUserWorkItem(async _ => {
var request = new SleepTest {
Name = $"Request {i+1}",
WaitingSecs = rand.Next(30, 60),
};
Interlocked.Add(ref totalSecondsWaited, request.WaitingSecs);
log.Info($"[{DateTime.Now.TimeOfDay}] Sending {request.Name} to sleep for {request.WaitingSecs} seconds...");
try
{
var response = await client.GetAsync(request);
log.Info($"[{DateTime.Now.TimeOfDay}] Received {request.Name}: {response.Message}");
}
catch (Exception ex)
{
log.Error($"[{DateTime.Now.TimeOfDay}] Error Response: {ex.UnwrapIfSingleException().Message}", ex);
}
finally
{
Interlocked.Decrement(ref responsesReceived);
}
});
});
Which waits for all threads to finish with:
while (Interlocked.Read(ref responsesReceived) > 0)
{
Thread.Sleep(10);
}
log.Info($"Took {sw.Elapsed.TotalSeconds} to execute {ConcurrentRequests} Concurrent Requests waiting a total of {totalSecondsWaited} seconds.");
Which results in:
INFO: Took 246.4556327 to execute 50 Concurrent Requests waiting a total of 2228 seconds.
i.e a lot less than it would've taken if the requests weren't executed concurrently.
The full log for all requests showing the requests were executed just marginally longer than the server was requested to wait for.
The 2 concurrency limit you're indicating suggests it's a http client connection limit per domain. You should make sure you're using a load testing tool that doesn't have these limits like wrk or apacahe bench.
Use .NET Core for best performance
Note if you want maximum throughput for Self Hosting ServiceStack Services we recommend running ServiceStack on .NET Core, preferably on the .NET Core runtime, but if you need to, you can also run ServiceStack ASP.NET Core on the .NET Framework.