Related
I recently started a job, where the requirements for coding were not made clear to me before I took it on. I'm enjoying it, but the learning curve has been massive as I've never used OOP before. This problem has me stumped so any help would be gratefully received - even if it just shows I'm missing the obvious!
I'm currently trying to update code on a Raspberry Pi 3B running an old version of Raspbian Jessie so that it will run on Buster. The code is over 10,000 lines long but this simplified snippet illustrates the problem that is the root cause of many problems when running on Buster. The code communicates with another chip on a pcb using SPI. The other chip is programmed with registers and CUSTAT is one of them. All registers exhibit the same problem. I'm confident it's not an SPI issue - communications have been checked and work.
The code is:
#!/usr/bin/python3
import spidev, time
import RPi.GPIO as GPIO
DEV_OE = 18 #GPIO24
SPI_MOSI = 19 #GPIO10
SPI_MISO = 21 #GPIO9
SPI_SCK = 23 #GPIO11
SPI_CE0 = 24 #GPIO8
SPI_CE1 = 26 #GPIO7
PROFILER = 40 #GPIO21
PINS_OUT = [DEV_OE, PROFILER]
def rpi_init():
GPIO.setwarnings(False)
GPIO.setmode(GPIO.BOARD)
GPIO.setup(PINS_OUT, GPIO.OUT)
GPIO.output(DEV_OE, 1)
def SPI_init():
global spi
spi = spidev.SpiDev(0,0)
spi.max_speed_hz=(500000)
def SPI_xfer(bytelist):
rxed = spi.xfer(bytelist)
return rxed
class Reg8_Variable():
def __init__(self, name, spi_seq):
self._spi_seq = spi_seq
self._value = "None"
self.name = name
def read_from_CPLD(self):
#byteseq = [0x01, 0x00]
byteseq = self._spi_seq
print ("byteseq", byteseq)
rxed = SPI_xfer(byteseq)
print ("SPI received ", rxed)
self._value = rxed[1]
CUSTAT = Reg8_Variable("CUSTAT", [0x01, 0x00])
if __name__ == "__main__":
print("start")
rpi_init() #sets up GPIO pins
SPI_init() #initialises SPI comms protocol
for i in range(5):
print (i, CUSTAT.name, CUSTAT._value, CUSTAT._spi_seq)
CUSTAT.read_from_CPLD()
print (i, CUSTAT.name, CUSTAT._value, CUSTAT._spi_seq)
time.sleep(2)
When it runs on Buster using python 3.7.3 the value of CUSTAT._spi_seq is changed to the value of the read SPI when read_from_CPLD is run. Expected behaviour is that CUSTAT._spi_seq stays at [0x01, 0x00]. CUSTAT.value should either read 0x01 or 0x05 depending upon whether a switch on the pcb is pressed. Output is
start
0 CUSTAT None [1, 0]
byteseq [1, 0]
SPI received [0, 5]
0 CUSTAT 5 [0, 5]
1 CUSTAT 5 [0, 5]
byteseq [0, 5]
SPI received [0, 0]
1 CUSTAT 0 [0, 0]
2 CUSTAT 0 [0, 0]
byteseq [0, 0]
SPI received [0, 0]
2 CUSTAT 0 [0, 0]
3 CUSTAT 0 [0, 0]
byteseq [0, 0]
SPI received [0, 0]
3 CUSTAT 0 [0, 0]
4 CUSTAT 0 [0, 0]
byteseq [0, 0]
SPI received [0, 0]
4 CUSTAT 0 [0, 0]
Commenting out byteseq = self._spi_seq and replacing it with byteseq = [0x01, 0x00] gives the correct output below:
start
0 CUSTAT None [1, 0]
byteseq [1, 0]
SPI received [0, 5]
0 CUSTAT 5 [1, 0]
1 CUSTAT 5 [1, 0]
byteseq [1, 0]
SPI received [0, 5]
1 CUSTAT 5 [1, 0]
2 CUSTAT 5 [1, 0]
byteseq [1, 0]
SPI received [0, 1]
2 CUSTAT 1 [1, 0]
3 CUSTAT 1 [1, 0]
byteseq [1, 0]
SPI received [0, 5]
3 CUSTAT 5 [1, 0]
4 CUSTAT 5 [1, 0]
byteseq [1, 0]
SPI received [0, 5]
4 CUSTAT 5 [1, 0]
Running the original code (with byteseq = self._spi_seq) on Jessie with python 3.4.2 gives the correct output below:
start
0 CUSTAT None [1, 0]
byteseq [1, 0]
SPI received [0, 5]
0 CUSTAT 5 [1, 0]
1 CUSTAT 5 [1, 0]
byteseq [1, 0]
SPI received [0, 5]
1 CUSTAT 5 [1, 0]
2 CUSTAT 5 [1, 0]
byteseq [1, 0]
SPI received [0, 5]
2 CUSTAT 5 [1, 0]
3 CUSTAT 5 [1, 0]
byteseq [1, 0]
SPI received [0, 1]
3 CUSTAT 1 [1, 0]
4 CUSTAT 1 [1, 0]
byteseq [1, 0]
SPI received [0, 5]
4 CUSTAT 5 [1, 0]
The hardware is the same - I only switched out the SD card on the Pi. What causes the difference in behaviour between python 3.4.2 and 3.7.3?
Sorry for the long question and thanks in advance for any help.
EDIT:
So I have a workaround. Changing the code to:
def read_from_CPLD(self):
byteseq = self._spi_seq.copy()
print ("byteseq", byteseq)
rxed = SPI_xfer(self._spi_seq)
print ("SPI received ", rxed)
self._value = rxed[1]
self._spi_seq = byteseq.copy()
gives the expected response. However, I still don't understand what caused the _spi_seq list to be changed and why only in 3.7.3 and not 3.4.2.
Any explanations gratefully received.
Root cause is bug in spidev - see my 2nd comment.
I'm trying to make some operations on Pointcloud but when calling VoxelDownSample function i'm getting Segmentation fault signal. I see that the problem is that program want to allocate some astronomic amount of data but i don't know where it come from as actuall PointCloud object size is 120.
Code:
open3d::geometry::PointCloud open3d_source_original;
open3d::geometry::PointCloud open3d_source_down;
createPointCloud(open3d_source_original);
open3d_source_down = *open3d_source_original.VoxelDownSample(0.01);
createPointCloud is my function which creates PointCloud from csv file and it works fine.
gdb output:
#0 __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff6eb4921 in __GI_abort () at abort.c:79
#2 0x00007ffff7ad6957 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff7adcae6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007ffff7adcb21 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007ffff7adcd54 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00007ffff7b05012 in std::__throw_bad_alloc() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x0000555555563476 in __gnu_cxx::new_allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> >::allocate(unsigned long, void const*) ()
#8 0x0000555555562bec in std::allocator_traits<std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::allocate(std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> >&, unsigned long) ()
#9 0x0000555555562088 in std::_Vector_base<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::_M_allocate(unsigned long) ()
#10 0x00005555555613ac in Eigen::Matrix<double, 3, 1, 0, 3, 1>* std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::_M_allocate_and_copy<__gnu_cxx::__normal_iterator<Eigen::Matrix<double, 3, 1, 0, 3, 1> const*, std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > > >(unsigned long, __gnu_cxx::__normal_iterator<Eigen::Matrix<double, 3, 1, 0, 3, 1> const*, std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > >, __gnu_cxx::__normal_iterator<Eigen::Matrix<double, 3, 1, 0, 3, 1> const*, std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > >) ()
#11 0x00005555555606e2 in std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::operator=(std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > const&) ()
#12 0x000055555555fcfa in open3d::geometry::PointCloud::operator=(open3d::geometry::PointCloud const&) ()
#13 0x000055555555f400 in main ()
Valgrind output:
==14534== Invalid read of size 8
==14534== at 0x114D59: std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::begin() const (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534== by 0x1147EA: std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::operator=(std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > const&) (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534== by 0x113CF9: open3d::geometry::PointCloud::operator=(open3d::geometry::PointCloud const&) (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534== by 0x1133FF: main (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534== Address 0x61da940 is 0 bytes after a block of size 112 alloc'd
==14534== at 0x4C3217F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14534== by 0x12C358: allocate (new_allocator.h:111)
==14534== by 0x12C358: allocate (alloc_traits.h:436)
==14534== by 0x12C358: __allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<open3d::geometry::PointCloud, std::allocatoropen3d::geometry::PointCloud, (__gnu_cxx::_Lock_policy)2> > > (allocated_ptr.h:104)
==14534== by 0x12C358: __shared_count<open3d::geometry::PointCloud, std::allocatoropen3d::geometry::PointCloud > (shared_ptr_base.h:635)
==14534== by 0x12C358: __shared_ptr<std::allocatoropen3d::geometry::PointCloud > (shared_ptr_base.h:1295)
==14534== by 0x12C358: shared_ptr<std::allocatoropen3d::geometry::PointCloud > (shared_ptr.h:344)
==14534== by 0x12C358: allocate_shared<open3d::geometry::PointCloud, std::allocatoropen3d::geometry::PointCloud > (shared_ptr.h:691)
==14534== by 0x12C358: make_sharedopen3d::geometry::PointCloud (shared_ptr.h:707)
==14534== by 0x12C358: open3d::geometry::PointCloud::VoxelDownSample(double) const (PointCloud.cpp:306)
==14534== by 0x1133DB: main (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
==14534==
==14534==
==14534== HEAP SUMMARY:
==14534== in use at exit: 8 bytes in 1 blocks
==14534== total heap usage: 19 allocs, 18 frees, 107,576 bytes allocated
==14534==
==14534== LEAK SUMMARY:
==14534== definitely lost: 0 bytes in 0 blocks
==14534== indirectly lost: 0 bytes in 0 blocks
==14534== possibly lost: 0 bytes in 0 blocks
==14534== still reachable: 8 bytes in 1 blocks
==14534== suppressed: 0 bytes in 0 blocks
==14534== Rerun with --leak-check=full to see details of leaked memory
==14534==
==14534== For counts of detected and suppressed errors, rerun with: -v
==14534== Use --track-origins=yes to see where uninitialised values come from
==14534== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0)
Operating system: Ubuntu 18.04
Open3D build from source
Compiler version: gcc 7.5
I tried to change macros from Eigen and change compilers but nothing worked.
This part shows a SIGABRT due to too much memory, not a SIGSEGV:
#0 __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff6eb4921 in __GI_abort () at abort.c:79
Valgrind is indicating that uninitialized data was used, not unmapped memory:
==14534== Invalid read of size 8
It appears to be pointing at the internal pointer to the start of the array which underlies the vector:
==14534== at 0x114D59: std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > >::begin() const (in /home/rufus/CLionProjects/isolatedSegFault/build/isolatedSegFault)
Then it says that the vector was created by make_shared inside voxelDownSample:
==14534== by 0x12C358: make_sharedopen3d::geometry::PointCloud (shared_ptr.h:707)
==14534== by 0x12C358: open3d::geometry::PointCloud::VoxelDownSample(double) const (PointCloud.cpp:306)
My intuition says,
If a weak_ptr is involved, then perhaps the vector was destroyed by losing all its shared_ptr references. Valgrind can't see this because it wouldn't be deallocated until the weak_ptr references are gone, too.
Otherwise, there's some other kind of memory error, such as
trying to use the vector before it has actually been constructed
using a raw pointer after the shared_ptr is destroyed
a buffer overflow, or other randomly spraying memory (rather less likely to hit a shared_ptr object, but you never know)
In any case, it looks like a library bug, if this object was created after the last time any of your code ran, and your parameters were numerically valid.
Sorry if this is a noob question.
i have a list, for exemple "a" which has two or tree list inside a1,a2,a3 like :
[ [1,2], [0,0,0], [1,2,3,4,5] ]
[ [1,2,3,4], [0,0], [1,2,3] ]
etc...
Each list inside doesn't has the same length so when I make a simple
for i in range(len(a)):
print(a[i])
the result is :
[[1,2],[0,0,0],[1,2,3,4,5]]
[[1,2,3,4],[0,0],[1,2,3]]
While I'm waiting for this result where elements in list are aligned in columns :
[[1,2] [0,0,0] [1,2,3,4,5]]
[[1,2,3,4] [0,0] [1,2,3]]
How can I do that?
thanks in advance for any help.
You can use string's ljust method to align the lists according to the largest:
a = [[[1, 2], [0, 0, 0], [1, 2, 3, 4, 5]],
[[1, 2, 3, 4], [0, 0], [1, 2, 3]]]
size = max([len(str(x)) for sub in a for x in sub]) + 4
for sub in a:
for x in sub:
print(str(x).ljust(size), end="")
print()
This Gives:
[1, 2] [0, 0, 0] [1, 2, 3, 4, 5]
[1, 2, 3, 4] [0, 0] [1, 2, 3]
Another option using one loop and the join method could be:
for sub in a:
print(''.join((str(x).ljust(size) for x in sub)))
Another quick-and-dirty approach, using str.format:
a = [[ [1,2], [0,0,0], [1,2,3,4,5] ],
[ [1,2,3,4], [0,0], [1,2,3] ],
[ [1], [2,2]],
[]]
max_sublist = max(a, key=len)
max_len = len(str(max((i for subl in a for i in subl), key=lambda k: len(str(k)))))
for subl in a:
print('[{}]'.format(('{!s:<{width}}' * len(max_sublist)).format(*subl, *' '*len(max_sublist), width=max_len)))
Prints:
[[1, 2] [0, 0, 0] [1, 2, 3, 4, 5]]
[[1, 2, 3, 4] [0, 0] [1, 2, 3] ]
[[1] [2, 2] ]
[ ]
Consider a 2D numpy array, a 1D numpy array, and a constant:
arr1 = [[ 4 4] arr2 = [ 1 7] k = 2
[ 3 6]
[ 7 10]
[-2 6]
[-1 6]
[-8 8]]
Here's what I need to do: If the absolute value of the values in arr1[:,0] are in arr2, then I need to subtract k from the corresponding values in arr1[:,1]. The final output should be:
arr1 = [[ 4 4]
[ 3 6]
[ 7 8]
[-2 6]
[-1 4]
[-8 8]]
Thank you.
Still learning, but this seems to work:
print(arr1[np.in1d(abs[:,0]), arr2), 1] -= k)
How can I create and array of arrays from individual arrays?
input:
print(shingle_set1)
print(shingle_set2)
output:
[12 14]
[11 16]
input:
total_shingle_set = numpy.append(shingle_set1, shingl_set2)
total_shingle_set = numpy.concatenate((shingle_set1, shingle_set2))
output for both statements:
[12 14 11 16]
The output above is one long array, which I don't want. I want to preserve each array in an array of arrays, like the following output:
[[12 14] [11 16]]
Experimented a bit more and the following gave me the answer:
total_shingle_set = numpy.concatenate(([shingle_set1], [shingle_set2]))
output:
[[12 14] [11 16]]