get result from multithread for the same method - multithreading

I would like to parallelize my program to be fast, so my program is like that:
Sim1 sim1 = new Sim1();
for(Entry<Integer, HashSet<String>> entry : map_topics_words.entrySet()) {
Integer k = entry.getKey();
Double sim = sim1.prob(word_m, entry.getValue());
sim_avg.put(k, sim);
score += sim;
}
and prob in the method in class Sim1 like that
public double prob(String w_i, HashSet<String> set_i){
Similarity sim = new Similarity();
double score = 1;
Iterator<String> it = set_i.iterator();
while (it.hasNext()) {
score += sim.computeSim(w_i, it.next());
}
score = score/set_i.size();
return score;
}
and computeSim in the method in classe Similarity like that :
public double computeSim(String w_1, String w_2){
return cmp(w_1,w_2);
}
So I would like to use thread for the first method and thread for the second method, I tried different ways but I failed
Any help, please
Thank you

You can change the first method code as below. We can try this by using Executor framework by submitting the work of prob() method as Callable task so that It can be executed in different thread, and then by using Future we can get the result for that particular call, for this we need to maintain one more map of key and corresponding Future object, Please see below code to understand it better, hope it may help you.
Sim1 sim1 = new Sim1();
Map<Integer, Future<Double>> workerMap = new HashMap<>();
ExecutorService exe = Executors.newCachedThreadPool();
for(Map.Entry<Integer, HashSet<String>> entry : map_topics_words.entrySet()) {
Integer k = entry.getKey();
workerMap.put(k, exe.submit(()->{ //Java 8 lamda
return sim1.prob(word_m, entry.getValue());
}));
}
//This loop is to get the result of prob() method for all the keys and process them further
for(Map.Entry<Integer, HashSet<String>> entry : map_topics_words.entrySet()) {
Integer k = entry.getKey();
try {
Double sim = workerMap.get(k).get();
sim_avg.put(k, sim);
score += sim;
} catch (Exception e) {
e.printStackTrace();
}
}
I will suggest you to google the working of Callable and Future in java, You can see this link also.

Related

How to define Task with parameters and return value in c++\cli?

i have a class in a cs file:
public class ThreadData
{
private int index;
public ThreadData(int index)
{
this.index = index;
}
public static ThreadDataOutput DoWork(ThreadDataInput input)
{
return new ThreadDataOutput();
}
}
now, i have c++ code that tries to init a new task and to us the above function:
int numOfThread = 2;
array<Task^>^ taskArr = gcnew array<Task^>(numOfThread);
for (int i = 0; i < numOfThread; i++)
{
ThreadData^ td = gcnew ThreadData(i);
ThreadDataInput^ input = gcnew ThreadDataInput(i);
Task<ThreadDataOutput^>^ task = gcnew Task<ThreadDataOutput^>(td->DoWork, input);
taskArr[i] = task;
taskArr[i]->Start();
}
Task::WaitAll(taskArr, 300 * 1000);
the following code return 2 errors at compile time:
can't take address of 'ThreadData::DoWork' unless creating delegate instance
cannot convert argument 1 from 'AmadeusWS::ThreadDataOutput ^(__clrcall *)(AmadeusWS::ThreadDataInput ^)' to 'System::Func ^
i also tried to declare a delegate like this in the cs file:
public static Func<ThreadDataInput, ThreadDataOutput> DoWork2 = delegate(ThreadDataInput taskDataInput)
{
return new ThreadDataOutput();
};
but i don't know how to call it from the c++\cli code
can anyone assist me to understand how to define cli delegate that can take parametr ?
thanks
In order to create a delegate instance in C++/CLI, you need to construct it explicitly, and specify the object that it will be called on separately from the class & method to be called.
gcnew Func<TInput, TOutput>(theObject, &TheClass::MethodToInvoke)
Note that the method to be called is specified in the C++ style.
Substituting that in to your task creation, I believe this statement will work for you:
Task<ThreadDataOutput^>^ task = gcnew Task<ThreadDataOutput^>(
gcnew Func<ThreadDataInput^, ThreadDataOutput^>(td, &ThreadData::DoWork),
input);
Edit
In the code you posted in your comment, you missed the object to invoke the delegate on.
gcnew Func<Object^, Object^>(td, &ThreadData::DoWork)
^^

Sentiment Analysis(SentiWordNet) - Judging the context of a sentence

I am trying to find whether a sentence is Positive or Negative in the following steps:
1.) Retrieving the Parts of speech(verbs, nouns, adjectives etc) from the sentence using the Stanford NLP parser.
2.) Using the SentiWordNet to find the Positive and Negative values related to each Part of Speech.
3.) Summing up the Positive and Negative values obtained to calculate a Net Positive and Net Negative value related to a sentence.
But the problem is that, the SentiWordNet return a list of Positive/Negative values based on different senses/contexts. Is it possible to pass a particular sentence along with the part of speech to the SentiWordNet parser, so that it can judge the sense/context automatically and returns only one pair of Positive and Negative value?
Or is there any other alternate solution to this problem?
Thanks.
SentoWordNet Demo Code
This may help you.
// Copyright 2013 Petter Törnberg
//
// This demo code has been kindly provided by Petter Törnberg <pettert#chalmers.se>
// for the SentiWordNet website.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
public class SentiWordNetDemoCode {
private Map<String, Double> dictionary;
public SentiWordNetDemoCode(String pathToSWN) throws IOException {
// This is our main dictionary representation
dictionary = new HashMap<String, Double>();
// From String to list of doubles.
HashMap<String, HashMap<Integer, Double>> tempDictionary = new HashMap<String, HashMap<Integer, Double>>();
BufferedReader csv = null;
try {
csv = new BufferedReader(new FileReader(pathToSWN));
int lineNumber = 0;
String line;
while ((line = csv.readLine()) != null) {
lineNumber++;
// If it's a comment, skip this line.
if (!line.trim().startsWith("#")) {
// We use tab separation
String[] data = line.split("\t");
String wordTypeMarker = data[0];
// Example line:
// POS ID PosS NegS SynsetTerm#sensenumber Desc
// a 00009618 0.5 0.25 spartan#4 austere#3 ascetical#2
// ascetic#2 practicing great self-denial;...etc
// Is it a valid line? Otherwise, through exception.
if (data.length != 6) {
throw new IllegalArgumentException(
"Incorrect tabulation format in file, line: "
+ lineNumber);
}
// Calculate synset score as score = PosS - NegS
Double synsetScore = Double.parseDouble(data[2])
- Double.parseDouble(data[3]);
// Get all Synset terms
String[] synTermsSplit = data[4].split(" ");
// Go through all terms of current synset.
for (String synTermSplit : synTermsSplit) {
// Get synterm and synterm rank
String[] synTermAndRank = synTermSplit.split("#");
String synTerm = synTermAndRank[0] + "#"
+ wordTypeMarker;
int synTermRank = Integer.parseInt(synTermAndRank[1]);
// What we get here is a map of the type:
// term -> {score of synset#1, score of synset#2...}
// Add map to term if it doesn't have one
if (!tempDictionary.containsKey(synTerm)) {
tempDictionary.put(synTerm,
new HashMap<Integer, Double>());
}
// Add synset link to synterm
tempDictionary.get(synTerm).put(synTermRank,
synsetScore);
}
}
}
// Go through all the terms.
for (Map.Entry<String, HashMap<Integer, Double>> entry : tempDictionary
.entrySet()) {
String word = entry.getKey();
Map<Integer, Double> synSetScoreMap = entry.getValue();
// Calculate weighted average. Weigh the synsets according to
// their rank.
// Score= 1/2*first + 1/3*second + 1/4*third ..... etc.
// Sum = 1/1 + 1/2 + 1/3 ...
double score = 0.0;
double sum = 0.0;
for (Map.Entry<Integer, Double> setScore : synSetScoreMap
.entrySet()) {
score += setScore.getValue() / (double) setScore.getKey();
sum += 1.0 / (double) setScore.getKey();
}
score /= sum;
dictionary.put(word, score);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (csv != null) {
csv.close();
}
}
}
public double extract(String word, String pos) {
return dictionary.get(word + "#" + pos);
}
public static void main(String [] args) throws IOException {
if(args.length<1) {
System.err.println("Usage: java SentiWordNetDemoCode <pathToSentiWordNetFile>");
return;
}
String pathToSWN = args[0];
SentiWordNetDemoCode sentiwordnet = new SentiWordNetDemoCode(pathToSWN);
System.out.println("good#a "+sentiwordnet.extract("good", "a"));
System.out.println("bad#a "+sentiwordnet.extract("bad", "a"));
System.out.println("blue#a "+sentiwordnet.extract("blue", "a"));
System.out.println("blue#n "+sentiwordnet.extract("blue", "n"));
}
}
We can pass the pos to sentiwordnet parser.
Download pattern python module
from pattern.en import wordnet
print wordnet.synsets("kill",pos="VB")[0].weight
wordnet.synsets returns list of synsets
and from that we are selecting 1st item
Output will be a tuple of (polarity,subjectivity)
Hope this helps...

Exception in thread "Thread-4" java.lang.NullPointerException?

i have a problem with this exception.
I'm trying to make a function to display data from database on a table, every keystroke i made at jtextfield. So its like the table automatically refreshed with new data everytime i type in the jtextfield.
Here's the code :
First i have this variable :
private Statement stmt;
List<Barang> dataSBarang =new LinkedList();
boolean searchBarang=true;
Searching sBarang;
And this is how i call the function :
private void inputkodeTFMouseClicked(java.awt.event.MouseEvent evt){
sBarang = new Searching( stmt, dataSBarang, modelDetail, tabelDetailBarang, inputkodeTF, searchBarang);
sBarang.start();
}
And this is the Searching Object
public class Searching extends Thread{
private Statement stmt;
private List<Barang> dataBarang;
private JTable jTabelBarang;
private JTextField tf;
private boolean cari;
private DefaultTableModel modelBarang;
public Searching(Statement stmt, List<Barang> dataBarang, DefaultTableModel tm, JTable jTabelBarang, JTextField tf, boolean cari){
this.stmt=stmt;
this.dataBarang=dataBarang;
this.modelBarang=tm;
this.jTabelBarang=jTabelBarang;
this.tf=tf;
this.cari=cari;
}
#Override
public void run(){
String temp="";
while(cari==true){
//System.out.println("jalan");
try{
String masukan = tf.getText();
System.out.println(masukan);
if(!masukan.equals("")&&!masukan.equals(temp)){
clearTableBarang();
//System.out.println("Mencari "+ masukan);
ResultSet rs = stmt.executeQuery("select kode_barang, nama_barang, jumlah_stok, " +
"minimal_stok, harga_jual, deskripsi_barang from BARANG WHERE (kode_barang LIKE " +
"'"+masukan+"%')");
System.out.println(rs);
while(rs.next()){
String kode_barang = rs.getString ("kode_barang");
String nama_barang = rs.getString ("nama_barang");
int jumlah_stok = rs.getInt("jumlah_stok");
int minimal_stok = rs.getInt("minimal_stok");
int harga_jual = rs.getInt("harga_jual");
String deskripsi_barang = rs.getString ("deskripsi_barang");
//System.out.println(kode_barang+" "+deskripsi_barang);
dataBarang.add(new Barang(kode_barang,nama_barang,jumlah_stok,minimal_stok,harga_jual,deskripsi_barang));
((DefaultTableModel) jTabelBarang.getModel()).insertRow(jTabelBarang.getRowCount(), new Object[]{kode_barang, nama_barang, jumlah_stok, minimal_stok, harga_jual, deskripsi_barang});
}
temp = masukan;
}
else if(masukan.equals("")&&!masukan.equals(temp)) {
clearTableBarang();
showTableBarang();
temp = masukan;
}
} catch(SQLException s){s.printStackTrace();}
catch(ArrayIndexOutOfBoundsException s){s.printStackTrace();}
try {
sleep(500);
} catch(InterruptedException e){}
}
}
public void clearTableBarang(){
int numrows = modelBarang.getRowCount();
for(int i = numrows - 1; i >=0; i--){
modelBarang.removeRow(i);
}
dataBarang.clear();
}
public void showTableBarang(){
try{
ResultSet rs = stmt.executeQuery("select kode_barang, nama_barang, jumlah_stok, minimal_stok, harga_jual, deskripsi_barang from barang");
while(rs.next()){
String kode_barang = rs.getString ("kode_barang");
String nama_barang = rs.getString ("nama_barang");
int jumlah_stok = rs.getInt("jumlah_stok");
int minimal_stok = rs.getInt("minimal_stok");
int harga_jual = rs.getInt("harga_jual");
String deskripsi_barang = rs.getString ("deskripsi_barang");
//System.out.println(kode_barang+" "+deskripsi_barang);
dataBarang.add(new Barang(kode_barang,nama_barang,jumlah_stok,minimal_stok,harga_jual,deskripsi_barang));
((DefaultTableModel)jTabelBarang.getModel()).insertRow(jTabelBarang.getRowCount(), new Object[]{kode_barang, nama_barang, jumlah_stok, minimal_stok, harga_jual, deskripsi_barang});
}
} catch(SQLException s){s.printStackTrace();}
}
public void delay(){
try {
sleep(1000000000);
} catch(InterruptedException e){}
}
}
This is the error :
Exception in thread "Thread-4" java.lang.NullPointerException
at mypkg.Searching.run(Searching.java:47)
FYI : Line 47 is pointing to
ResultSet rs = stmt.executeQuery("select kode_barang, nama_barang, jumlah_stok, " +
"minimal_stok, harga_jual, deskripsi_barang from BARANG WHERE (kode_barang LIKE " +
"'"+masukan+"%')");
Please help me solve the problem. Thank you very much. :D
NullPointerExceptions are the most easy ones to fix with a debugger. Just place a breakpoint on that line and see what is null.
If the line you posted is correct, you do not even need a debugger since the only thing that can throw the exception is stmt which will be null.
Note:
It is a good thing to run your DB query in a separate Thread to avoid blocking the UI. However, in your case you are updating the UI from that Thread which is not allowed and will cause weird issues. All Swing components must be accessed on the Event Dispatch Thread (EDT). Consult the Swing concurrency tutorial for more information
I do hope you are not starting up a separate Thread on each keystroke in the textfield as you indicated in your question. Looking at the code in the Thread, you remove all elements from the table and then re-add rows. So if a users types in 5 characters at a normal typing speed, you will launch 5 threads which most likely run all at the same time (since a DB connection might not be that fast if your network is lagging). That means that with your current code 5 Threads are, at the same time, removing the table model and adding rows. Even if you put all the Swing code on the EDT (see my first point), you still end up with 5 threads posting runnables on the EDT messing with your table model. I do not know what the resulting table model will be, but probably not what you want
Clearly stmt is null.
You declare it here:
private Statement stmt;
List<Barang> dataSBarang =new LinkedList();
boolean searchBarang=true;
Searching sBarang;
But I don't see any initialization code anywhere, so most likely it remains null for this method call:
private void inputkodeTFMouseClicked(java.awt.event.MouseEvent evt){
sBarang = new Searching( stmt, dataSBarang, modelDetail, tabelDetailBarang, inputkodeTF, searchBarang);
sBarang.start();
}

Limitation in Cassandra-0.8.1 when using batch mutation

I found some exceptions from cassandra when I do batch mutation, it said "already has modifications in this mutation", but the info given are two different operations.
I use Super column with counters in this case, it's like
Key: md5 of urls, utf-8
SuperColumnName: date, utf-8
ColumnName: Counter name is a random number from 1 to 200,
ColumnValue:1L
L
public void SuperCounterMutation(ArrayList<String> urlList) {
LinkedList<HCounterSuperColumn<String, String>> counterSuperColumns;
for(String line : urlList) {
String[] ele = StringUtils.split(StringUtils.strip(line), ':');
String key = ele[0];
String SuperColumnName = ele[1];
LinkedList<HCounterColumn<String>> ColumnList = new LinkedList<HCounterColumn<String>>();
for(int i = 2; i < ele.length; ++i) {
ColumnList.add(HFactory.createCounterColumn(ele[i], 1L, ser));
}
mutator.addCounter(key, ColumnFamilyName, HFactory.createCounterSuperColumn(SuperColumnName, ColumnList, ser, ser));
++count;
if(count >= BUF_MAX_NUM) {
try {
mutator.execute();
} catch(Exception e) {
e.printStackTrace();
}
mutator = HFactory.createMutator(keyspace, ser);
count = 0;
}
}
return;
}
Error info from cassandra log showed that the duplicated operations have the same key only, SuperColumnName are not the same, and for counter name set, some conflicts have intersects and some not.
I'm using Cassandra 0.8.1 with hector 0.8.0-rc2
Can anyone tell me the reason of this problem? Thanks in advance!
Error info from cassandra log showed that the duplicated operations have the same key
Bingo. You'll need to combine operations from the same key into a single mutation.

How to create a method at runtime using Reflection.emit

I'm creating an object at runtime using reflection emit. I successfully created the fields, properties and get set methods.
Now I want to add a method. For the sake of simplicity let's say the method just returns a random number. How do I define the method body?
EDIT:
Yes, I've been looking at the msdn documentation along with other references and I'm starting to get my head wrapped around this stuff.
I see how the example above is adding and/or multplying, but what if my method is doing other stuff. How do I define that "stuff"
Suppose I was generating the class below dynamically, how would I create the body of GetDetails() method?
class TestClass
{
public string Name { get; set; }
public int Size { get; set; }
public TestClass()
{
}
public TestClass(string Name, int Size)
{
this.Name = Name;
this.Size = Size;
}
public string GetDetails()
{
string Details = "Name = " + this.Name + ", Size = " + this.Size.ToString();
return Details;
}
}
You use a MethodBuilder to define methods. To define the method body, you call GetILGenerator() to get an ILGenerator, and then call the Emit methods to emit individual IL instructions. There is an example on the MSDN documentation for MethodBuilder, and you can find other examples of how to use reflection emit on the Using Reflection Emit page:
public static void AddMethodDynamically(TypeBuilder myTypeBld,
string mthdName,
Type[] mthdParams,
Type returnType,
string mthdAction)
{
MethodBuilder myMthdBld = myTypeBld.DefineMethod(
mthdName,
MethodAttributes.Public |
MethodAttributes.Static,
returnType,
mthdParams);
ILGenerator ILout = myMthdBld.GetILGenerator();
int numParams = mthdParams.Length;
for (byte x = 0; x < numParams; x++)
{
ILout.Emit(OpCodes.Ldarg_S, x);
}
if (numParams > 1)
{
for (int y = 0; y < (numParams - 1); y++)
{
switch (mthdAction)
{
case "A": ILout.Emit(OpCodes.Add);
break;
case "M": ILout.Emit(OpCodes.Mul);
break;
default: ILout.Emit(OpCodes.Add);
break;
}
}
}
ILout.Emit(OpCodes.Ret);
}
It sounds like you're looking for resources on writing MSIL. One important resource is the OpCodes class, which has a member for every IL instruction. The documentation describes how each instruction works. Another important resource is either Ildasm or Reflector. These will let you see the IL for compiled code, which will help you understand what IL you want to write. Running your GetDetailsMethod through Reflector and setting the language to IL yields:
.method public hidebysig instance string GetDetails() cil managed
{
.maxstack 4
.locals init (
[0] string Details,
[1] string CS$1$0000,
[2] int32 CS$0$0001)
L_0000: nop
L_0001: ldstr "Name = "
L_0006: ldarg.0
L_0007: call instance string ConsoleApplication1.TestClass::get_Name()
L_000c: ldstr ", Size = "
L_0011: ldarg.0
L_0012: call instance int32 ConsoleApplication1.TestClass::get_Size()
L_0017: stloc.2
L_0018: ldloca.s CS$0$0001
L_001a: call instance string [mscorlib]System.Int32::ToString()
L_001f: call string [mscorlib]System.String::Concat(string, string, string, string)
L_0024: stloc.0
L_0025: ldloc.0
L_0026: stloc.1
L_0027: br.s L_0029
L_0029: ldloc.1
L_002a: ret
}
To generate a method like that dynamically, you will need to call ILGenerator.Emit for each instruction:
ilGen.Emit(OpCodes.Nop);
ilGen.Emit(OpCodes.Ldstr, "Name = ");
ilGen.Emit(OpCodes.Ldarg_0);
ilGen.Emit(OpCodes.Call, nameProperty.GetGetMethod());
// etc..
You may also want to look for introductions to MSIL, such as this one: Introduction to IL Assembly Language.

Resources