I have spent the last day looking for a memory leak in my code. For each frame in my loop I can see in the task manager that about 200 Mb of memory is lost. Eventually the program of course crashes.
The pseudo-code looks like this:
for frame = 0:NBROFFRAMES
cv::Mat = getImage(frame);
cout<<"Before optimization"<<endl;
getchar();
optFunction.optimizeColor(img);
cout<<"After optimization"<<endl;
getchar();
Do other things
end
Just looking in the task manager shows that in optimizeColor approximately 200 Mb are lost for every frame.
In optimizeColorI run Eigens ConjugateGradient several times. When I changed to Cholesky Factorization, SimplicialLDLT, the memory consumption is between 4.5-4.7 Gb over the entire sequence. Using ConjugateGradient it starts with 4.5 Gb and after 10 images the amount of memory used according to my Task Manager is 11.2 Gb.
I have also run Valgrind and one of the messages I get is the following:
==22576== 110,430,720 bytes in 18 blocks are possibly lost in loss record 30,186 of 30,189
==22576== at 0x4C2E80F: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==22576== by 0x913CCD: Eigen::internal::scoped_array<int>::scoped_array(long) (Memory.h:666)
==22576== by 0x910329: Eigen::internal::CompressedStorage<float, int>::reallocate(long) (CompressedStorage.h:230)
==22576== by 0x910695: Eigen::internal::CompressedStorage<float, int>::resize(long, double) (CompressedStorage.h:96)
==22576== by 0x91CC41: Eigen::SparseMatrix<float, 0, int>& Eigen::SparseMatrix<float, 0, int>::operator=<Eigen::SparseMatrix<float, 1, int> >(Eigen::SparseMatrixBase<Eigen::SparseMatrix<float, 1, int> > const&) (SparseMatrix.h:1085)
==22576== by 0x91A911: Eigen::SparseMatrix<float, 0, int>::SparseMatrix<Eigen::SparseMatrix<float, 1, int> >(Eigen::SparseMatrixBase<Eigen::SparseMatrix<float, 1, int> > const&) (SparseMatrix.h:674)
==22576== by 0x91837C: void Eigen::Ref<Eigen::SparseMatrix<float, 0, int> const, 0, Eigen::OuterStride<-1> >::construct<Eigen::SparseMatrix<float, 1, int> >(Eigen::SparseMatrix<float, 1, int> const&, Eigen::internal::false_type) (SparseRef.h:217)
==22576== by 0x91631C: Eigen::Ref<Eigen::SparseMatrix<float, 0, int> const, 0, Eigen::OuterStride<-1> >::Ref<Eigen::SparseMatrix<float, 1, int> >(Eigen::SparseMatrixBase<Eigen::SparseMatrix<float, 1, int> > const&) (SparseRef.h:184)
==22576== by 0x913801: void Eigen::internal::generic_matrix_wrapper<Eigen::SparseMatrix<float, 0, int>, false>::grab<Eigen::SparseMatrix<float, 1, int> >(Eigen::EigenBase<Eigen::SparseMatrix<float, 1, int> > const&) (IterativeSolverBase.h:81)
==22576== by 0x90FEFE: void Eigen::IterativeSolverBase<Eigen::ConjugateGradient<Eigen::SparseMatrix<float, 0, int>, 3, Eigen::DiagonalPreconditioner<float> > >::grab<Eigen::SparseMatrix<float, 1, int> >(Eigen::SparseMatrix<float, 1, int> const&) (IterativeSolverBase.h:375)
==22576== by 0x90C03D: Eigen::ConjugateGradient<Eigen::SparseMatrix<float, 0, int>, 3, Eigen::DiagonalPreconditioner<float> >& Eigen::IterativeSolverBase<Eigen::ConjugateGradient<Eigen::SparseMatrix<float, 0, int>, 3, Eigen::DiagonalPreconditioner<float> > >::analyzePattern<Eigen::SparseMatrix<float, 1, int> >(Eigen::EigenBase<Eigen::SparseMatrix<float, 1, int> > const&) (IterativeSolverBase.h:199)
==22576== by 0x9077F3: OptimizeAlbedo::optimize_rho(cv::Mat const&, std::vector<Eigen::Matrix<float, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<float, -1, 1, 0, -1, 1> > > const&, std::vector<Eigen::Matrix<int, 2, 1, 0, 2, 1>, std::allocator<Eigen::Matrix<int, 2, 1, 0, 2, 1> > > const&, Eigen::Matrix<float, -1, 1, 0, -1, 1> const&, float, float, Eigen::Matrix<float, -1, 1, 0, -1, 1> const&) (OptimizeAlbedo.cpp:586)
The only thing I do that allocates memory is that I define the sizes of some matrices when I create the instance optFunction and reserve space for the entries. That is of course only done once and that I have checked several times. The only explanation I see is that ConjugateGradient is consuming memory.
Here is my code to call ConjugateGradient:
function optmize()
Eigen::ConjugateGradient<Eigen::SparseMatrix<float>, Eigen::Lower | Eigen::Upper> cg;
cg.setTolerance(0.001);
cg.setMaxIterations(200);
cg.analyzePattern(A_tot);
cg.factorize(A_tot);
Eigen::VectorXf opt = cg.solveWithGuess(b_tot, rho_current);
end
I would have expected that cg is cleared when the function goes out of scope, but it seems that is not the case.
I think I use Eigen version: 3.2.92, this is in the Macros.h:
#define EIGEN_WORLD_VERSION 3
#define EIGEN_MAJOR_VERSION 2
#define EIGEN_MINOR_VERSION 92
I am using Ubuntu 16.04 and g++ 5.4.0.
EDIT: As pointed out by Avi Ginsburg, I was using a beta-version of Eigen. By upgrading to 3.3.2 it works without memory leaks.
Related
I want assign tasks to workers in a way that cost is minimized given a time constraint.
Suppose:
Labor Cost: $20/hr and handles 10Kg
Machine Cost: $30/hr and handles 20Kg and requires a Machine Operator
Machine Operator Cost: 15$/hr
Constraints:
Total Available Labor = 10
Total Available Machines = 5
Total Available Machine Operator = 3
Total Required Load to Handle = 1000Kg
Total Available Hours to complete Task = 8
How can I solve this problem using Python? Are there any libraries to solve such problems?
Edit:
Objective:
Minimize:
Cost = 20 * ith_labor_time + 30 * jth_Machine_time + 15 * kth_Machine_Operator_time
Now the constraints:
1) 0 <= i(total_labor) <= 10
2) 0 <= j(total_machine) <= 5 (Not 5 because only 3 operators available and machine is dependent on operator)
3) 0 <= k(total_machine_operator_time) <= 3
4) sum(ith_labor_time <= 80)
5) sum(ith_machine_time <= 40)
5) sum(ith_Operator_time) - sum(ith_Machine_time) = 0
6) constraint that total weight handled is 1000kg (unable to make this constraint)
7) machine_time and operator_time are dependent on each other as 1 machine requires 1 operator (I believe this is handled in constraint 5)
Practical Example:
Suppose we have total_labor = 3, total_machines = 3 and total_operators = 3
Cost per labor/hr = $20
Cost per machinery/hr = $30
Cost per operator/hr = $15
Labor process rate/hr = 10 Kg
Machine process rate/hr = 20 Kg
Next each labor can work for 8 hours similarly each machinery can work for 8 hours and same for the machine operator they can work for 8 hours.
Task needs to be completed in 8 hours and 1000kg load needs to be processed.
My formulation:
Min:
Cost = 20*L1 + 20*L2 + 20*L3 + 30*M1 + 30*M2 + 30*M3 + 15*O1 + 15*O2 + 15*O3
Constraints:
L1 + L2 + L3 <= 24 (if each labor work simultaneously they can total up to 24hrs)
L1 <= 8
L2 <= 8
L3 <= 8
M1 + M2 + M3 <= 24 (if each machinery works simultaneously they can total up to 24hrs)
M1 <= 8
M2 <= 8
M3 <= 8
M1 + M2 + M3 - O1 - O2 - O3 = 0 (Each machinery requires an operator)
10*L1 + 10*L2 + 10*L3 + 20*M1 + 20*M2 + 20*M3 = 1000 (Total load to be processed = 1000 Kg)
Code
from scipy.optimize import linprog
c = [20, 20, 20, 30, 30, 30, 15, 15, 15]
A = [[1, 1, 1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 1, 1, 0, 0, 0],
[1, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 1, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 1, 0, 0, 0],
[0, 0, 0, 1, 1, 1, -1, -1, -1]]
A1 = [[10, 10, 10, 20, 20, 20, 0, 0, 0]]
b = [24, 24, 8, 8, 8, 8, 8, 8, 0]
b2 = [1000]
res = linprog(c, A_ub=A, A_eq=A1, b_ub=b, b_eq=b2, method='revised simplex')
I have a B&W image filled with white pixels (255). How can I obtain all the white pixel coordinates present in the image on Vivado HLS?
I'm using hls::Mat to store images.
Here's my top-level function on Vivado HLS:
#include "top.h"
#include <iostream>
void dust_detect(AXI_STREAM& input_data, AXI_STREAM& output_data, int m, int n)
{
auto int pixel;
#pragma HLS DATAFLOW
//Create AXI streaming interfaces for the core
#pragma HLS INTERFACE axis port=input_data
#pragma HLS INTERFACE axis port=output_data
#pragma HLS INTERFACE ap_ctrl_none port=return
/************* Arrays used ***************/
gray_IMAGE img_0;
#pragma HLS STREAM variable=img_0
gray_IMAGE img_1;
#pragma HLS STREAM variable=img_1
gray_IMAGE img_2;
#pragma HLS STREAM variable=img_2
gray_IMAGE img_2a;
#pragma HLS STREAM variable=img_2a
gray_IMAGE img_2b;
#pragma HLS STREAM variable=img_2b
gray_IMAGE img_3;
#pragma HLS STREAM variable=img_3
gray_IMAGE img_4;
#pragma HLS STREAM variable=img_4
gray_IMAGE img_5;
#pragma HLS STREAM variable=img_5
gray_IMAGE img_6;
#pragma HLS STREAM variable=img_6
gray_IMAGE img_7;
#pragma HLS STREAM variable=img_7
gray_IMAGE img_7a;
#pragma HLS STREAM variable=img_7a
gray_IMAGE img_7b;
#pragma HLS STREAM variable=img_7b
const char coefficients1[7][10] = { { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1} };
hls::Window<7,10,char> erodewindow;
for (int i=0;i<7;i++){
for (int j=0;j<10;j++){
erodewindow.val[i][j]=coefficients1[i][j];
}
}
const char coefficients2[9][12] = { { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
{ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1} };
hls::Window<9,12,char> dilatewindow;
for (int i=0;i<9;i++){
for (int j=0;j<12;j++){
dilatewindow.val[i][j]=coefficients2[i][j];
}
}
hls::AXIvideo2Mat(input_data, img_0);
hls::Threshold(img_0,img_1,80,255,HLS_THRESH_BINARY);//OTSU THRESHOLDING
hls::Threshold(img_1,img_2,80,255,HLS_THRESH_BINARY_INV);//Invert the Thresholded output
hls::Duplicate(img_2,img_2a,img_2b);
hls::Erode<4,4>(img_2a,img_3,erodewindow);
hls::Dilate<6,6>(img_3,img_4,dilatewindow);
hls::Threshold(img_4,img_5,100,255,HLS_THRESH_BINARY_INV);//Invert the Dilated output
hls::Threshold(img_5,img_6,100,1,HLS_THRESH_BINARY);
hls::Mul(img_2b,img_6,img_7);
hls::Duplicate(img_7,img_7a,img_7b);
for(m=0; m<MAX_HEIGHT; m++) {
for(n=0; n<MAX_WIDTH; n++) {
#pragma HLS PIPELINE IT=1
auto pixel = img_7a.read();
if(pixel != 0)
{
printf("White pixel found at x: " + m + "\ty: " + n) ; // White pixel found at (x,y)
}
}
}
hls::Mat2AXIvideo(img_7b,output_data);
}
I need help with the part of the code after hls::Mul that's used to find the white pixel coordinates in image img_7a.
The question is fairly broad, as there can be many ways to achieve you're goal depending on your setup and requirements.
However, if you're using an hls::Mat type for storing your image, the actual image is stored into a FIFO. So for identifying all the white pixels, you would be forced to scan all the pixels of the image in sequence, something like:
for (i = 0; i < img.rows; ++i) {
for (j = 0; j < img.cols; ++j) {
#pragma HLS PIPELINE II=1
auto pixel = img.read();
if (pixel != 0) {
// White pixel found: use i and j (the coordinates) or store them somewhere.
}
// Eventually write back the pixel into the image: img.write(pixel);
}
}
If the image is instead stored in a buffer like registers or BRAM, then the above loop over the rows and columns of the image can be parallelized. For instance:
const int kUnrollFactor = 4;
pixel_t img[H][W];
#pragma HLS ARRAY_PARTITION variable=img block factor=kUnrollFactor dim=2
for (i = 0; i < H; ++i) {
for (j = 0; j < W; ++j) {
#pragma HLS PIPELINE II=1
#pragma HLS UNROLL factor=kUnrollFactor
if (img[i][j] != 0) {
// White pixel found: use i and j (the coordinates) or store them somewhere.
}
}
}
As for eventually storing your result: since there can be at most W * H white pixels, you might need a buffer of W * H * log2(max(W, H)) bits.
Edit
Based on the updated question, the last part of the code might give some problems.
Since a definitions of AXI_STREAM and gray_IMAGE are not given, the data type of the stream (and in turn of the internal streams img_*) might not be comparable with 255 (the white pixel value).
Suppose the pixel is defined as a struct, then doing pixel == 255 would simply not compile. On the other end, if it is a "flatten" RGB array like ap_uint<24> (three 8bit pixels), comparing with 255 might be meaningless (you would instead need to compare it with 0xFFFFFF).
Otherwise, if the pixel type is integer or unsigned char, then comparing it with 255 or 0 would give no problems.
I'm trying to make some operations on Pointcloud but when calling VoxelDownSample function i'm getting Segmentation fault signal. I see that the problem is that program want to allocate some astronomic amount of data but i don't know where it come from as actuall PointCloud object size is 120.
Code:
open3d::geometry::PointCloud open3d_source_original;
open3d::geometry::PointCloud open3d_source_down;
createPointCloud(open3d_source_original);
open3d_source_down = *open3d_source_original.VoxelDownSample(0.01);
createPointCloud is my function which creates PointCloud from csv file and it works fine.
gdb output:
#0 __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff6eb4921 in __GI_abort () at abort.c:79
#2 0x00007ffff7ad6957 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff7adcae6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007ffff7adcb21 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007ffff7adcd54 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00007ffff7b05012 in std::__throw_bad_alloc() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x0000555555563476 in __gnu_cxx::new_allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> >::allocate(unsigned long, void const*) ()
#8 0x0000555555562bec in std::allocator_traits<std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::allocate(std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> >&, unsigned long) ()
#9 0x0000555555562088 in std::_Vector_base<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::_M_allocate(unsigned long) ()
#10 0x00005555555613ac in Eigen::Matrix<double, 3, 1, 0, 3, 1>* std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::_M_allocate_and_copy<__gnu_cxx::__normal_iterator<Eigen::Matrix<double, 3, 1, 0, 3, 1> const*, std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > > >(unsigned long, __gnu_cxx::__normal_iterator<Eigen::Matrix<double, 3, 1, 0, 3, 1> const*, std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > >, __gnu_cxx::__normal_iterator<Eigen::Matrix<double, 3, 1, 0, 3, 1> const*, std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > >) ()
#11 0x00005555555606e2 in std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::operator=(std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > const&) ()
#12 0x000055555555fcfa in open3d::geometry::PointCloud::operator=(open3d::geometry::PointCloud const&) ()
#13 0x000055555555f400 in main ()
Valgrind output:
==14534== Invalid read of size 8
==14534== at 0x114D59: std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::begin() const (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534== by 0x1147EA: std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::operator=(std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > const&) (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534== by 0x113CF9: open3d::geometry::PointCloud::operator=(open3d::geometry::PointCloud const&) (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534== by 0x1133FF: main (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534== Address 0x61da940 is 0 bytes after a block of size 112 alloc'd
==14534== at 0x4C3217F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14534== by 0x12C358: allocate (new_allocator.h:111)
==14534== by 0x12C358: allocate (alloc_traits.h:436)
==14534== by 0x12C358: __allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<open3d::geometry::PointCloud, std::allocatoropen3d::geometry::PointCloud, (__gnu_cxx::_Lock_policy)2> > > (allocated_ptr.h:104)
==14534== by 0x12C358: __shared_count<open3d::geometry::PointCloud, std::allocatoropen3d::geometry::PointCloud > (shared_ptr_base.h:635)
==14534== by 0x12C358: __shared_ptr<std::allocatoropen3d::geometry::PointCloud > (shared_ptr_base.h:1295)
==14534== by 0x12C358: shared_ptr<std::allocatoropen3d::geometry::PointCloud > (shared_ptr.h:344)
==14534== by 0x12C358: allocate_shared<open3d::geometry::PointCloud, std::allocatoropen3d::geometry::PointCloud > (shared_ptr.h:691)
==14534== by 0x12C358: make_sharedopen3d::geometry::PointCloud (shared_ptr.h:707)
==14534== by 0x12C358: open3d::geometry::PointCloud::VoxelDownSample(double) const (PointCloud.cpp:306)
==14534== by 0x1133DB: main (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534==
==14534==
==14534== HEAP SUMMARY:
==14534== in use at exit: 8 bytes in 1 blocks
==14534== total heap usage: 19 allocs, 18 frees, 107,576 bytes allocated
==14534==
==14534== LEAK SUMMARY:
==14534== definitely lost: 0 bytes in 0 blocks
==14534== indirectly lost: 0 bytes in 0 blocks
==14534== possibly lost: 0 bytes in 0 blocks
==14534== still reachable: 8 bytes in 1 blocks
==14534== suppressed: 0 bytes in 0 blocks
==14534== Rerun with --leak-check=full to see details of leaked memory
==14534==
==14534== For counts of detected and suppressed errors, rerun with: -v
==14534== Use --track-origins=yes to see where uninitialised values come from
==14534== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0)
Operating system: Ubuntu 18.04
Open3D build from source
Compiler version: gcc 7.5
I tried to change macros from Eigen and change compilers but nothing worked.
This part shows a SIGABRT due to too much memory, not a SIGSEGV:
#0 __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff6eb4921 in __GI_abort () at abort.c:79
Valgrind is indicating that uninitialized data was used, not unmapped memory:
==14534== Invalid read of size 8
It appears to be pointing at the internal pointer to the start of the array which underlies the vector:
==14534== at 0x114D59: std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::begin() const (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
Then it says that the vector was created by make_shared inside voxelDownSample:
==14534== by 0x12C358: make_sharedopen3d::geometry::PointCloud (shared_ptr.h:707)
==14534== by 0x12C358: open3d::geometry::PointCloud::VoxelDownSample(double) const (PointCloud.cpp:306)
My intuition says,
If a weak_ptr is involved, then perhaps the vector was destroyed by losing all its shared_ptr references. Valgrind can't see this because it wouldn't be deallocated until the weak_ptr references are gone, too.
Otherwise, there's some other kind of memory error, such as
trying to use the vector before it has actually been constructed
using a raw pointer after the shared_ptr is destroyed
a buffer overflow, or other randomly spraying memory (rather less likely to hit a shared_ptr object, but you never know)
In any case, it looks like a library bug, if this object was created after the last time any of your code ran, and your parameters were numerically valid.
I'm trying to somehow test my rbd storage with random read, random write, mixed randrw, but the output is not correct, it is a sequential growing number.
What is wrong with my steps?
This is the fio file that I ran:
; fio-rand-write.job for fiotest
[global]
name=fio-rand-write
filename=fio-rand-write
rw=randwrite
bs=4K
direct=1
write_iops_log=rand-read
[file1]
size=1G
ioengine=libaio
iodepth=16
And the result is this:
head rand-read_iops.1.log
2, 1, 1, 4096, 0
2, 1, 1, 4096, 0
2, 1, 1, 4096, 0
2, 1, 1, 4096, 0
2, 1, 1, 4096, 0
3, 1, 1, 4096, 0
4, 1, 1, 4096, 0
5, 1, 1, 4096, 0
5, 1, 1, 4096, 0
5, 1, 1, 4096, 0
tail rand-read_iops.1.log
30700, 1, 1, 4096, 0
30700, 1, 1, 4096, 0
30700, 1, 1, 4096, 0
30700, 1, 1, 4096, 0
30700, 1, 1, 4096, 0
30700, 1, 1, 4096, 0
30700, 1, 1, 4096, 0
30700, 1, 1, 4096, 0
30700, 1, 1, 4096, 0
30700, 1, 1, 4096, 0
I'm using fio 3.18.
Why I don't get the iops that is the real one?
(Note this isn't really a programming question so Stackoverflow is the wrong place to ask this... Maybe Super User or Serverfault would be a better choice and get faster answers?)
but the output is not correct, it is a sequential growing number
Which column are you referring to? If you mean the left most column then isn't that time per the fio Log File Formats documentation?
Fio supports a variety of log file formats, for logging latencies, bandwidth, and IOPS. The logs share a common format, which looks like this:
time (msec), value, data direction, block size (bytes), offset (bytes)
Doesn't time generally monotonically increase relative to prior readings (accounting for precision)?
Also see the documentation for write_iops_log that says:
Because fio defaults to individual I/O logging, the value entry in the IOPS log will be 1 unless windowed logging (see log_avg_msec) has been enabled
i'm an old glBegin() programmer and i need to port this structure
struct Line { float color[3]; float vertices[2][3];};
to a more new drawing paradigm.
how to draw this with glDrawArrays or glDrawElements, supposed that shaders are ok?
thank you, mic.
glEnableVertexAttribArray(0);
glEnableVertexAttribArray(1);
glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, sizeof(float) * 3, lines->vertices);
glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, sizeof(float) * 3, lines->color);
glDrawElements(GL_LINES, 2, GL_UNSIGNED_SHORT, line_indices);
Assuming you've assigned your vertex shader's 'position' and 'color' vertex attributes to 0 and 1. Indices array should be something like GLshort indices[] = { 0, 1 };.