How to integrate the outputs from Callbacks in C++/CPLEX - visual-c++

I used macro Callbacks in my code and I want to integrate outputs with together. At each node I need the value of Pseudocosts, slacks, variableBranch and etc. But I don’t know how to integrate these data that I retrieve with different Callbacks. I don’t run all Callbacks with together. Each time I run the code with different Callbacks the values of NodeID or Node aren’t equal. For example in pic1 I run BranchCallback to retrieve Pseudocosts and in pic3 I used UserCutCallback to retrieve the values of variables at each nodes. As can be seen in pic1 the last node is 126 but in pic3 the last node is 164. I want to create data structure in excel with each nodes but I don’t know which number of nodes I have to consider?126 or 164?
For example in pic1, can I say all information(values of pseudocosts ) about node10 is belong to node10 in pic3? And in pic3, all information(values of slacks ) about node10 is belong to node10 in pic1?
ILOUSERCUTCALLBACK1(Myvalue, IloArray<IloNumVarArray>, vars) {
for (int i = 0; i < nbworkers; ++i) {
for (int j = 0; j < nbmachines; j++) {
cout << "getvalue(" << vars[i][j] << ") = "
<< getValue(vars[i][j]) << endl;
}
}
}

You ask many things at the same time. I am going to answer the question that is related to the subject. For everything else please create a new question and show your code, your actual output and explain how that differs from the expected output.
The IloCplex class has a function out(). That function returns a reference to the stream to which CPLEX sends all its output. You can pass that reference to your callbacks and then write to that stream from your callbacks.
For example:
ILOUSERCUTCALLBACK1(MyCallback, std::ostream &, output) {
output << "Message from callback" << std::endl;
}
and then
cplex.use(MyCallback(cplex.getEnv(), cplex.out()));
to create and register the callback.
UPDATE After you edited your question, it seems your problem is not to print output from a callback but something else.
First of all note that it is expected to get different search paths if you do one run with a user cut callback and another run with a branch callback. There is no way to relate the node numbers or node ids from one run to the other. The statistics you want to obtain must be acquired with one single run.
Moreover, in order to identify a node you should not use the node number or the number of nodes processed (the number in the left-most column in the log). Instead you should use the ID of a node. This is what is known as sequence number in the C++ API. This is the only thing you can use to identify a node. These ids should match with the node ids shown at the very right of the log in case dynamic search is disabled (which happens automatically if you use control callbacks). These node IDs are available from all callbacks and you can use them collect information collected from different callbacks for the same node.
Update 2: Actually, I was wrong. The number returned by getNodeId() and the number shown in the NodeID column of the log are different, see my answer to this question. There is no way to relate these two numbers. Sorry for the confusion. I think you asked a similar question in another forum as well and I claimed the two numbers to be the same. That was wrong as well. Sorry again.
So basically your only option to relate things in a callback to things in a log is to perform a single-threaded run and then look at the order in which things are printed.
However, in order to trace the tree (along with pseudo costs etc.) you don't need the log. You can do everything from a callback by just using sequence numbers. The most difficult part is tracking the parent/child relationship which can be done like this (not that this is not thread safe):
struct Parent {
typedef IloCplex::MIPCallbackI::NodeId NodeId;
struct Less {
bool operator()(NodeId const &n1, NodeId const &n2) const {
return n1._id < n2._id;
}
};
typedef std::map<NodeId,NodeId,Less> MapType;
MapType parents;
void set(NodeId child, NodeId parent) { parents[child] = parent; }
IloCplex::MIPCallbackI::NodeId get(NodeId child) const {
MapType::const_iterator it = parents.find(child);
return (it == parents.end()) ? NodeId() : it->second;
}
};
Parent parent;
ILOBRANCHCALLBACK0(BranchCallback) {
std::cout << "CALLBACK[B]: " << getNodeId()
<< " (" << parent.get(getNodeId()) << ")" << std::endl;
int const n = getNbranches();
for (int i = 0; i < n; ++i) {
NodeId id = makeBranch(i);
parent.set(id, getNodeId());
}
}

Related

std::map insert thread safe in c++11?

I have very simple code in which multiple threads are trying to insert data in std::map and as per my understanding this should led to program crash because this is data race
std::map<long long,long long> k1map;
void Ktask()
{
for(int i=0;i<1000;i++)
{
long long random_variable = (std::rand())%1000;
std::cout << "Thread ID -> " << std::this_thread::get_id() << " with looping index " << i << std::endl;
k1map.insert(std::make_pair(random_variable, random_variable));
}
}
int main()
{
std::srand((int)std::time(0)); // use current time as seed for random generator
for (int i = 0; i < 1000; ++i)
{
std::thread t(Ktask);
std::cout << "Thread created " << t.get_id() << std::endl;
t.detach();
}
return 0;
}
However i ran it multiple time and there is no application crash and if run same code with pthread and c++03 application is crashing so I am wondering is there some change in c++11 that make map insert thread safe ?
No, std::map::insert is not thread-safe.
There are many reasons why your example may not crash. Your threads may be running in a serial fashion due to the system scheduler, or because they finish very quickly (1000 iterations isn't that much). Your map will fill up quickly (only having 1000 nodes) and therefore later insertions won't actually modify the structure and reduce possibility of crashes. Or perhaps the implementation you're using IS thread-safe.
For most standard library types, the only thread safety guarantee you get is that it is safe to use separate object instances in separate threads. That's it.
And std::map is not one of the exceptions to that rule. An implementation might offer you more of a guarantee, or you could just be getting lucky.
And when it comes to fixing threading bugs, there's only one kind of luck.

Filling QGraphicsScene with items in thread

I have the thread where the worker object is running infinite cycle. Here I have the following code that I would like it to read coordinates from list and place QGraphicsEllipseItem on these coordinates. The list can be updated by another thread, so I protect it by mutex. But sometimes the size of list may grow up so I would like to create new QGraphicsEllipse items for it if needed.
int meter_to_pixel_ratio = 20;
int x_pixel, y_pixel;
int i;
forever {
visualizationDataMutex->lock();
while(ellipseList->count()<visualizationData->count())
{
qDebug() << "Creating new visual item...";
ellipseList->append(new QGraphicsEllipseItem(0.0, 0.0, 10.0, 10.0));
ellipseList->last()->setVisible(false);
visualizationScene->addItem(ellipseList->last());
}
for(i=0; i<visualizationData->count(); i++)
{
x_pixel = meter_to_pixel_ratio*visualizationData->at(i)->x();
y_pixel = meter_to_pixel_ratio*visualizationData->at(i)->y();
ellipseList->at(i)->setPos(x_pixel, y_pixel);
ellipseList->at(i)->setBrush(QBrush(*visualizationColor->at(i)));
if(!ellipseList->at(i)->isVisible()) ellipseList->at(i)->setVisible(true);
}
visualizationDataMutex->unlock();
// repaint scene
visualizationScene->update();
QThread::msleep(100);
}
The problem I have is, that when I try to run the program I´ll obtain a runtime error. Tried to qDebug() the ellipseList->count() and seems to have the exactly same number of elements as needed (as visualizationData->count()). When commented these three lines:
//ellipseList->at(i)->setPos(x_pixel, y_pixel);
//ellipseList->at(i)->setBrush(QBrush(*visualizationColor->at(i)));
//if(!ellipseList->at(i)->isVisible()) ellipseList->at(i)->setVisible(true);
program can run without crashing. I do not understand why is this happening since there is no other function working with QGraphicsView/QGraphicsScene. (QGraphicsView was added from Qt Designer environment into mainwindow).

How to parallelize "while" loop by the using of PPL

I need to parallelize "while" loop by the means of PPL. I have the following code in Visual C++ in MS VS 2013.
int WordCount::CountWordsInTextFiles(basic_string<char> p_FolderPath, vector<basic_string<char>>& p_TextFilesNames)
{
// Word counter in all files.
atomic<unsigned> wordsInFilesTotally = 0;
// Critical section.
critical_section cs;
// Set specified folder as current folder.
::SetCurrentDirectory(p_FolderPath.c_str());
// Concurrent iteration through p_TextFilesNames vector.
parallel_for(size_t(0), p_TextFilesNames.size(), [&](size_t i)
{
// Create a stream to read from file.
ifstream fileStream(p_TextFilesNames[i]);
// Check if the file is opened
if (fileStream.is_open())
{
// Word counter in a particular file.
unsigned wordsInFile = 0;
// Read from file.
while (fileStream.good())
{
string word;
fileStream >> word;
// Count total number of words in all files.
wordsInFilesTotally++;
// Count total number of words in a particular file.
wordsInFile++;
}
// Verify the values.
cs.lock();
cout << endl << "In file " << p_TextFilesNames[i] << " there are " << wordsInFile << " words" << endl;
cs.unlock();
}
});
// Destroy critical section.
cs.~critical_section();
// Return total number of words in all files in the folder.
return wordsInFilesTotally;
}
This code does parallel iteration through std::vector in outer loop. Parallelism is provided by concurrency::parallel_for() algorithm. But this code also has nested "while" loop that executes reading from file. I need to parallelize this nested "while" loop. How can this nested "while" loop can be parallelized by the means of PPL. Please help.
As user High Performance Mark hints in his comment, parallel reads from the same ifstream instance will cause undefined and incorrect behavior. (For some more discussion, see question "Is std::ifstream thread-safe & lock-free?".) You're basically at the parallelization limit here with this particular algorithm.
As a side note, even reading multiple different file streams in parallel will not really speed things up if they are all being read from the same physical volume. The disk hardware can only actually support so many parallel requests (typically not more than one at a time, queuing up any requests that come in while it is busy). For some more background, you might want to check out Mark Friedman's Top Six FAQs on Windows 2000 Disk Performance; the performance counters are Windows-specific, but most of the information is of general use.

C# - Matrix Printing Incorrectly on a Separate Thread

I'm currently making a Console Application game, and am working on the multiplayer version.
When I am receiving packets, I create a new thread to handle the packets, in order to return to the ReceiveFrom command as soon as possible.
The newly created thread should reprint the whole Matrix, using the new information it got regarding the changes in the matrix (it should print the Matrix with updated player positions).
The problem is, when the Print method is called on the new thread, it is badly performed. It prints the matrix very inaccurately, and many characters of the matrix are just in a mess on the Console Screen.
Here are the methods:
void Receive()
{
while (true)
{
byte[] msg = new byte[1024];
client.Receive(msg);
Thread handle = new Thread(() => HandleInput(msg));
handle.Start();
}
}
void HandleInput(byte[] msgX)
{
string data = Encoding.ASCII.GetString(msgX);
data = data.Replace("\0", "");
if (data.Contains('*') || data.Contains('\0') || data.Contains(' ')) // Move Packet (moving objects in a matrice)
{
// for example, a move packet can be: '*'!12!10
char ToMove = char.Parse(data.Split('!')[0]);
int X = int.Parse(data.Split('!')[1]);
int Y = int.Parse(data.Split('!')[2]);
grid[X, Y] = ToMove;
Print();
}
}
void Print() // print a 20x50 matrix
{
Console.Clear();
for (int y = 0; y < 52; y++)
{
Console.Write('-');
}
Console.WriteLine();
for (int i = 0; i < 20; i++)
{
Console.Write('|');
for (int j = 0; j < 50; j++)
{
Console.Write(grid[i,j]);
}
Console.WriteLine('|');
}
for (int x = 0; x < 52; x++)
{
Console.Write('-');
}
}
But, when I tried to treat the input and print the matrix on the same thread of the Receive method, it worked fine. The problem printing it only came when I printed the matrix on a separate thread.
So why does this happen? Why can't a separate thread just print the matrix correctly?
You create a new thread for every received kilobyte, but those threads are not guaranteed to start running in sequence, to wait one another, nor to finish running in sequence.
There is also no guarantee that network client will receive all needed bytes in one call of Receive method. It may happen in one, but it may happen in several calls. It may also happen that two Send methods from the other side of the connection are merged into one message. All that is guaranteed in network communication is the ordering of the bytes in the stream, and nothing else.
Your next question will probably be "so how do I make it work"? I can't do the job for you, but this is in general how I would do it:
Read chunk of bytes from the stream and store it in some buffer.
Check in the loop for existence of a complete message in the buffer. If there is such a message remove that message from the buffer, and process the message. I would process it in the same thread, and only if there are problems with that approach I would consider additional threads (or rather just one additional thread to whom I would be sending all the messages).
Repeat the loop until all complete messages are removed from the buffer, and continue reading the stream.

Using MultiMap to store Financial Market Data in C++

I would like to understand if MultiMap is the best container in STL to store financial market data with a format like "date","price" (for example 07/10/2013 1000).
I tried to make an simple example, just to understand which could be the implementation but i got an horrible errors when i tried to print them out.
class Date {
int day;
int month;
int year;
int value_of_date;
public:
Date(int d, int m, int y):
day(d),month(m),year(y){
value_of_date=year*10000 + month*100 + day;
}
friend ostream & operator<< (ostream &out, const Date &date);
};
ostream & operator<< (ostream &out, const Date &date) {
out << "(" << date.day << ", " <<
date.month << ", " <<
date.year << ")";
return out;
}
int main () {
std::multimap<Date,int> first;
first.insert(std::pair<Date,int>(Date(01,01,2000),1000));
first.insert(std::pair<Date,int>(Date(01,02,2000),1010));
first.insert(std::pair<Date,int>(Date(01,03,2000),1020));
first.insert(std::pair<Date,int>(Date(01,04,2000),1030));
for(auto i = first.cbegin(); i != first.cend(); i++) {
std::cout << i->first << " " << i->second << std::endl;
}
return 0;
}
Is the comparison < operator() my problem here ? How do i implement < operator() to sort the date.
is there a more elegant solution for a type date instead of using Class Date ?
If this is the best CONTAINER for financial market data ?
Thank you very much for any help
Do you HAVE to do this in C++? It's a HORRIBLE language to do financial stuff in. Trust me, I've been there.
If you're trying to add a custom class to a container, you'll need to study the container's requirements for elements. For example, containers often need to be able to create, delete, compare, sort, and assign values to the things they contain, so you need to implement comparison operators (yes, operator < () is part of it; just implement an operator < (const YourClass& other) const, but read up on your container class), assignment operators, copy constructors, etc.
Which is part of why another language would be easier. Python, for example, would automatically derive most of those operators for you, AND it was approved by the SEC in the last couple of years as the official language for financial market data. Python, Java, Ruby, Perl, or just about any other mainstream language, would be a better choice -- especially if it comes with a decimal class. If you're worried about performance, look into NumPy, and use a database backend like MySQL or mongodb.
But if you DO (believe) you need to use C++ for whatever reason (probably huge data volumes and performance), at least do yourself the courtesy of using boost - especially its variant type, and considering STXXL. If you want performance, though, go would be a more sensible choice lately. Really, the only reason to use C++ for this is if you work for a crazy company that forces you to... which does happen.

Resources