I'm want to make multiple requests to a remote service and I'm trying an asynchronous/multithread approach.
For the concurrent part, I'll try this http://coding-time.blogspot.pt/2008/03/implement-your-own-parallelfor-in-c.html
IList<string> returnedValues = new List<string>();
int numberOfRequests = 5;
object sync = new object();
ParallelUtilities.For(0, numberOfRequests, delegate(int i)
{
string response = string.Empty;
try
{
try
{
Func<int, string> caller = CallService;
IAsyncResult result = caller.BeginInvoke(i, null, null);
Thread.Sleep(0);
string returnValue = caller.EndInvoke(result);
lock (sync)
{
returnedValues.Add(returnValue);
}
}
catch (Exception ex)
{
Console.WriteLine(string.Concat("ex: ", ex.Message, "stack:", ex.StackTrace));
}
});
}
And the service call
private string CallService(int index)
{
string value;
using (var channel = GetClientChannel<IService>("http://localhost:51383/Service/Service.svc"))
{
value = channel.GetValue(index);
}
return value + "for this index:" + index;
}
Is there a better way to do this?
How do IAsyncResult works in multithreading?
Replace
IAsyncResult result = caller.BeginInvoke(i, null, null);
Thread.Sleep(0);
string returnValue = caller.EndInvoke(result);
with
string returnValue = CallService(result);
There's no point in starting code on the thread-pool and then waiting for it immediately. What would you gain by doing that? The Sleep is especially suspicious to me. You try to initiate a context switch which just adds additional overhead.
How do IAsyncResult works in multithreading?
The two are not related. What do you mean?
Related
I would like to ask how to fetch all doc from solr collection using solrJ.
I have written one code but getting error
Exception in thread "main" org.apache.solr.client.solrj.SolrServerException: No collection param specified on request and no default collection has been set.
String zkHostString = "linux152:2181,linuxUL:2181,linux170:2181/solr";
CloudSolrClient server = new CloudSolrClient(zkHostString);
SolrQuery parameters = new SolrQuery();
public void cursorMark() throws IOException, SolrServerException {
SolrQuery parameters = new SolrQuery();
QueryResponse response = new QueryResponse();
response = server.query(parameters);
parameters.set("q",":");
parameters.set("qt","/select");
parameters.setParam("wt","json");
parameters.set("collection", "RetailDev_Protocol");
int fetchSize = 2;
parameters.setRows(fetchSize);
String cursorMark = CursorMarkParams.CURSOR_MARK_START;
boolean done = false;
while (! done) {
parameters.set(CursorMarkParams.CURSOR_MARK_PARAM, cursorMark);
long offset = 0;
long totalResults = response.getResults().getNumFound();
while (offset < totalResults)
{
parameters.setStart((int) offset);
try {
for (SolrDocument doc : server.query(parameters).getResults())
{
log.info((String) doc.getFieldValue("title"));
}
} catch (SolrServerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
offset += fetchSize;
}
String nextCursorMark = (response).getNextCursorMark();
}
SolrDocumentList list = response.getResults();
System.out.println(list.toString());
}
You need to set your collection in the following way:
server.setDefaultCollection("<MY_COLLECTION");
otherwise you get the error that you specified in your question.
please have a look at below code.
public static class DbModel
{
public static readonly int TableID = 0;
static DbModel()
{
DbModel.PodID = FetchTableID().PodID;
}
public static Pod FetchTableID()
{
Pod result = null;
try
{
//Control never comes back from this line. What am I missing?
var searchResult = apiPod.SearchTableAsync(1).Result;
result = searchResult.First();
}
catch (Exception ex)
{
Helpers.TraceException(PageName,"FEtchPodID","Unable to fetch PodID",ex);
}
return result;
}
}
Signature of SearchTableAsync looks like this
public async Task<List<Pod>> SearchTableAsync(int i)
{
try
{
using (var client = new HttpClient())
{
//deleted - connecting to server, constructing query string etc.
var response = await client.GetAsync(ApiBaseUrl + "api/Pod/Search" + queryString);
if (response.IsSuccessStatusCode)
{
var podList = await response.Content.ReadAsAsync<List<Pod>>();
return podList;
}
else
{
//log error
}
}
}
catch (Exception ex)
{
Logger.TraceError(null, ex);
}
return null;
}
Call to SearchTableAsync never returns back. Am I missing anything? Or its because I am invoking this from Static constructor?
The problem is likely due to the use of Task.Result property. this is a blocking property and can cause a deadlock. You can simply await the task which will return the result, but you need to make the method async.
public static async Pod FetchTableID()
{
Pod result = null;
try
{
var searchResult = await apiPod.SearchTableAsync(1);
result = searchResult.First();
}
catch (Exception ex)
{
Helpers.TraceException(PageName,"FEtchPodID","Unable to fetch PodID",ex);
}
return result;
}
searchResult = apiPod.SearchTableAsync(1).excute.get;
use this instead of
var searchResult = apiPod.SearchTableAsync(1).Result;
This post explains why it was blocking.
Cheers,
Hemant
I'm trying to remove items from a list until its empty with multithreading.
Code:
public void testUsers() {
final List<User> users = userDao.findAll();
final int availableProcessors = Runtime.getRuntime().availableProcessors() * multiplier;
final List<String> loggingList = Lists.newArrayList();
final List<Integer> sizeChecked = Lists.newArrayList();
int totalSizeChecked = 0;
int sizeList = users.size();
ExecutorService executorService = Executors.newFixedThreadPool(availableProcessors);
for (int i = 0; i < availableProcessors; i++) {
createThread(executorService, users, loggingList, sizeChecked);
}
executorService.shutdown();
try {
// wait for all threads to die
executorService.awaitTermination(1, TimeUnit.HOURS);
} catch (InterruptedException ex) {
}
for (Integer count : sizeChecked) {
totalSizeChecked += count;
}
Assert.assertTrue(totalSizeChecked==sizeList);
}
private void createThread(ExecutorService executorService, final List<User> users,
final Collection<String> loggingList, final List<Integer> sizeChecked) {
executorService.execute(new Runnable() {
#Override
public void run() {
int totalChecked = 0;
while (!users.isEmpty()) {
User user = null;
synchronized (users) {
if (!users.isEmpty()) {
user = users.remove(0);
}
}
totalChecked++;
if (user != null) {
String reason = checkUser(user);
if (reason != null) {
loggingList.add(reason);
}
} else {
LOGGER.info("user is null");
}
}
sizeChecked.add(totalChecked);
}
});
}
Now I was thinking this couldn't be so wrong cause I made the list synchronised for removing the first item.
I'm testing with a multiplier of 6.(on prod it will be lowered to 1-2)
I get this in the email :
The batch was not correctly executed.
Size of accounts that must be checked : 28499. Size of accounts that have been checked: 25869
What do I wrong to get it threadsafe?
List<Integer> sizeChecked is not thread safe. Therefore you cannot add elements in parallel in it.
Synchronize your add operation or use a thread-safe structure. If sizeChecked is just a counter, use an AtomicLong instead and make each thread increment it.
I need to test a Data Context and see what behavior it has under multiple simultaneous requests, for that I made a simple console application that [in theory] would send these requests:
private static DateTime startTime = DateTime.Now.AddSeconds(5);
public static Random rand = new Random();
static void Main(string[] args)
{
const byte testThreads = 10;
ThreadStart[] threadStarts = new ThreadStart[testThreads];
Thread[] threads = new Thread[testThreads];
for (byte i = 0; i < testThreads; i++)
{
threadStarts[i] = new ThreadStart(ExecutePOST);
threads[i] = new Thread(threadStarts[i]);
}
for (byte i = 0; i < testThreads; i++){ threads[i].Start(); }
for (byte i = 0; i < testThreads; i++){ threads[i].Join(); }
}
The called function is
private static void ExecutePOST()
{
while (DateTime.Now < startTime) { }
Console.WriteLine("{0} STARTING TEST", DateTime.Now.Millisecond);
WebRequest webRequest = WebRequest.Create(/*URL*/);
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.Method = "POST";
string name = string.Format("Test {0}", Program.rand.Next(1000));
byte[] bytes = Encoding.ASCII.GetBytes(/*PARAMETERS*/);
Stream output = null;
try
{
webRequest.ContentLength = bytes.Length;
output = webRequest.GetRequestStream();
output.Write(bytes, 0, bytes.Length);
Console.WriteLine("{0}:{1}", DateTime.Now.Millisecond, name);
}
catch (WebException ex)
{
Console.WriteLine(ex.Message);
}
finally
{
if (output != null)
{
output.Close();
}
}
}
The output I get is:
Can anyone please explain this behavior? Why is it stopping after two requests?
Thank you
Yes, this is because the number of connections per URL is limited to 2 by default - the connections are pooled.
You're hogging the connection by writing data to the request stream, but then never getting the response. A simple:
using (webRequest.GetResponse()) {}
at the end of the method is likely to sort it out. That will finish the request and release the connection to be used by another request.
Also note that a using statement for the output stream would make your code simpler too.
I'm trying to implement a simple TCP connection between Client/Server. I made the Server multithreaded so that it can take either multiple requests (such as finding the sum, max, min of a string of numbers provided by the user) from a single client or accept multiple connections from different clients. I'm running both of them on my machine, but the server doesn't seem to push out an answer. Not sure what I'm doing wrong here --
public final class CalClient {
static final int PORT_NUMBER = 6789;
public static void main (String arg[]) throws Exception
{
String serverName;
#SuppressWarnings("unused")
String strListOfNumbers = null;
int menuIndex;
boolean exit = false;
BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Please enter host name...");
System.out.print("> ");
serverName = inFromUser.readLine();
Socket clientSocket = new Socket(serverName, PORT_NUMBER);
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
//outToServer.writeBytes(serverName + '\n');
System.out.println("");
System.out.println("Enter 1 to enter the list of numbers");
System.out.println("Enter 2 to perform Summation");
System.out.println("Enter 3 to calculate Maximum");
System.out.println("Enter 4 to calculate Minimum");
System.out.println("Enter 5 to Exit");
while (!exit) {
System.out.print(">");
menuIndex = Integer.parseInt(inFromUser.readLine());
if (menuIndex == 1) {
System.out.println("Please enter the numbers separated by commas.");
System.out.print(">");
strListOfNumbers = inFromUser.readLine();
outToServer.writeBytes("List" + strListOfNumbers);
//continue;
}
else if (menuIndex == 2) {
outToServer.writeBytes("SUM");
System.out.println(inFromServer.readLine());
}
else if (menuIndex == 3) {
outToServer.writeBytes("MAX");
System.out.println(inFromServer.readLine());
}
else if (menuIndex == 4) {
outToServer.writeBytes("MIN");
System.out.println(inFromServer.readLine());
}
else if (menuIndex == 5) {
outToServer.writeBytes("EXIT");
exit = true;
}
}
}
}
public final class CalServer
{
static final int PORT_NUMBER = 6789;
public static void main(String[] args) throws IOException
{
try {
ServerSocket welcomeSocket = new ServerSocket(PORT_NUMBER);
System.out.println("Listening");
while (true) {
Socket connectionSocket = welcomeSocket.accept();
if (connectionSocket != null) {
CalRequest request = new CalRequest(connectionSocket);
Thread thread = new Thread(request);
thread.start();
}
}
} catch (IOException ioe) {
System.out.println("IOException on socket listen: " + ioe);
ioe.printStackTrace();
}
}
}
final class CalRequest implements Runnable
{
Socket socket;
BufferedReader inFromClient;
DataOutputStream outToClient;
TreeSet<Integer> numbers = new TreeSet<Integer>();
int sum = 0;
public CalRequest(Socket socket)
{
this.socket = socket;
}
#Override
public void run()
{
try {
inFromClient = new BufferedReader(new InputStreamReader(socket.getInputStream()));
outToClient = new DataOutputStream(socket.getOutputStream());
while(inFromClient.readLine()!= null) {
processRequest(inFromClient.readLine());
}
} catch (IOException e) {
e.printStackTrace();
}
}
public void processRequest(String string) throws IOException
{
String strAction = string.substring(0,3);
if (strAction.equals("LIS")) {
String strNumbers = string.substring(5);
String[] strNumberArr;
strNumberArr = strNumbers.split(",");
// convert each element of the string array to type Integer and add it to a treeSet container.
for (int i=0; i<strNumberArr.length; i++)
numbers.add(new Integer(Integer.parseInt(strNumberArr[i])));
}
else if (strAction.equals("SUM")) {
#SuppressWarnings("rawtypes")
Iterator it = numbers.iterator();
int total = 0;
while (it.hasNext()) {
total += (Integer)(it.next());
}
}
else if (strAction.equals("MAX")) {
outToClient.writeBytes("The max is: " + Integer.toString(numbers.last()));
}
else if (strAction.equals("MIN")) {
outToClient.writeBytes("The max is: " + Integer.toString(numbers.first()));
}
}
}
Since you are using readLine(), I would guess that you actually need to send line terminators.
My experience with TCP socket communications uses ASCII data exclusively, and my code reflects that I believe. If that's the case for you, you may want to try this:
First, try instantiating your data streams like this:
socket = new Socket (Dest, Port);
toServer = new PrintWriter (socket.getOutputStream(), true);
fromServer = new BufferedReader (new InputStreamReader
(socket.getInputStream()), 8000);
The true at the end the printWriter constructor tells it to auto flush (lovely term) the buffer when you issue a println.
When you actually use the socket, use the following:
toServer.println (msg.trim());
resp = fromServer.readLine().trim();
I don't have to append the \n to the outgoing text myself, but this may be related to my specific situation (more on that below). The incoming data needs to have a \n at its end or readLine doesn't work. I assume there are ways you could read from the socket byte by byte, but also that the code would not be nearly so simple.
Unfortunately, the TCP server I'm communicating with is a C++ program so the way we ensure the \n is present in the incoming data isn't going to work for you (And may not be needed in the outgoing data).
Finally, if it helps, I built my code based on this web example:
http://content.gpwiki.org/index.php/Java:Tutorials:Simple_TCP_Networking
Edit: I found another code example that uses DataOutputStream... You may find it helpful, assuming you haven't already seen it.
http://systembash.com/content/a-simple-java-tcp-server-and-tcp-client/