convert JSON buffer to string in perl - node.js

Issue: Trying to write json buffer data to a file using Perl script.
Scenario:
I will have binary data in Postgres.
So to recreate the scenario, I converted "geeks" into binary format and stored in Postgres.
Input : geeks
Output : 1100111 1100101 1100101 1101011 1110011
Nodejs query the binary data from the postgres and send the response as below in json format.
[
{
"lo_get": {
"type": "Buffer",
"data": [
49,
49,
48,
48,
49,
49,
49,
32,
49,
49,
48,
48,
49,
48,
49,
32,
49,
49,
48,
48,
49,
48,
49,
32,
49,
49,
48,
49,
48,
49,
49,
32,
49,
49,
49,
48,
48,
49,
49
]
}
}
]
Below is the Perl script I have written but I am not sure how to write the original string i.e. "geeks" to the file ..
$req = HTTP::Request->new(GET => $url);
$res = $ua->request($req);
$response = decode_json($res->content);
if ($res->is_success & #$response) {
$string = #$response[0]->{Output}->{data};
print ($string);
}
open my $fh, ">", "data_out.json";
print $fh encode_json($string);
close $fh;
Am not sure how can i get the file with 'geeks' as content.

my $binary =
pack 'C*',
#{ $response->[0]{lo_get}{data} };
say $binary; # '1100111 1100101 1100101 1101011 1110011
my $bytes =
pack 'C*',
map { oct("0b$_") }
split(' ', $binary);
say $bytes; # geeks
It is converted into binary
This makes no sense. In addition to the needless extra work converting the bytes into their binary representation or back when accessing the database, it multiplies the storage needed by 9 for nothing!

Algorithm to solution of the problem:
Obtain data into hash ref
use operator map with split to convert numbers into bin representation
join array of converted with chr and oct bin numbers into a string
process string as you desire
use strict;
use warnings;
use feature 'say';
use JSON;
use Data::Dumper;
my $fname = 'data.txt';
my $input = do { local $/; <DATA> };
my $data = from_json($input);
say 'Read: ' . Dumper($data);
my $str = join '', map { chr } #{$data->[0]{lo_get}{data}};
my #bin = map { '0b' . $_ } split ' ', $str;
$str = join '', map { chr oct($_) } #bin;
say 'Out: ' . $str;
open my $fh, '>', $fname
or die "Couldn't open $fname";
say $fh $str;
close $fh;
__DATA__
[
{
"lo_get": {
"type": "Buffer",
"data": [
49,
49,
48,
48,
49,
49,
49,
32,
49,
49,
48,
48,
49,
48,
49,
32,
49,
49,
48,
48,
49,
48,
49,
32,
49,
49,
48,
49,
48,
49,
49,
32,
49,
49,
49,
48,
48,
49,
49
]
}
}
]
Run as script.pl
Read: $VAR1 = [
{
'lo_get' => {
'type' => 'Buffer',
'data' => [
49,
49,
48,
48,
49,
49,
49,
32,
49,
49,
48,
48,
49,
48,
49,
32,
49,
49,
48,
48,
49,
48,
49,
32,
49,
49,
48,
49,
48,
49,
49,
32,
49,
49,
49,
48,
48,
49,
49
]
}
}
];
Out: geeks
Content of the file data.txt
geeks

Related

DOMException: Unable to set device configuration Chrome 86

WebUSB has very limited coverage, but that's okay for this project.
The navigator.usb.requestDevice method succeeds:
navigator.usb.requestDevice({
filters: [{
vendorId: RECEIVER_VENDOR_ID,
productId: RECEIVER_PRODUCT_ID
}]
})
Chrome receives permission (via modal interaction) to access the device.
To the browser, the device exists:
console.log(device)
USBDevice {usbVersionMajor: 2, usbVersionMinor: 0, usbVersionSubminor: 0, deviceClass: 2, deviceSubclass: 2, …}
configuration: null
configurations: [USBConfiguration]
deviceClass: 2
deviceProtocol: 0
deviceSubclass: 2
deviceVersionMajor: 2
deviceVersionMinor: 0
deviceVersionSubminor: 0
manufacturerName: "STMicroelectronics"
opened: true
productId: 22336
productName: "STM32 Virtual ComPort"
serialNumber: "355837673037"
usbVersionMajor: 2
usbVersionMinor: 0
usbVersionSubminor: 0
vendorId: 1155
__proto__: etc...
The example receiver I am following echoes the Mozilla docs:
await usbDevice.selectConfiguration(1)
(ReceiverParcer is here)
Making my way through a Web.dev article on Accessing USB via the web, and hoping, meanwhile, for some SO Magic.
What am I not seeing/doing?
Thanks.
Update
Checking chrome://device-log (level Debug) returns:
USBUser[14:51:38] USB device added: vendor=1155 "STMicroelectronics", product=22336 "STM32 Virtual ComPort", serial="355837673037", guid=2bd99a9a-75bf-4782-8a0d-2f9e13b06e86
USBUser[14:51:30] USB device removed: guid=1b763328-45db-4152-9c56-9b592d614c3e
USBEvent[14:51:30] Failed to clear halt: Pipe error
Because USB–and even more so WebUSB–seems to be more fragile than the http connections I'm used, parallel to playing with the Javascript interface, I also worked at pulling in the data with PyUSB. The goal is to write data from the device to a CSV file so either could work.
In this related post I outline some additional steps in the process including:
rebooting the machine
with the device connected
a good USB cable
checking chrome://device-log at DEBUG level
using a demo of the Serial App
checking status with kexstat, "unclaiming" with kextunload
looking at kernel log with dmesg
I would love to accept a more robust "answer" than this, but I am finally seeing some data:
let rawStream = new Uint8Array(data.buffer)
console.log(rawstream)
Uint8Array(69) [51, 48, 49, 32, 48, 48, 48, 48, 48, 48, 48, 48, 32, 48, 48, 48, 48, 48, 48, 48, 48, 32, 48, 48, 48, 48, 48, 48, 48, 48, 32, 48, 48, 48, 48, 48, 48, 48, 48, 32, 48, 48, 48, 48, 48, 48, 48, 48, 32, 48, 48, 48, 48, 48, 48, 48, 48, 32, 48, 48, 48, 48, 48, 48, 48, 48, 32, 13, 10]
And
let segments = String.fromCharCode.apply(null, rawStream).split(' ')
console.log(segments)
 ["300", "V1.06", "585AEEDD", "00000000", "00000000", "00000000", "00000000", "00000000", "00000000", "
↵"]

Converting a .mat file to cv image

I have a .mat file and want to convert it into a CV image format such that I can use it for a CNN model.
I am trying to obtain an RGB/ other colored image and not gray.
I tried doing the following(below) but I get a grayscale image, but when I plot the actual mat file using matplotlib it is not grayscale. Also, the .mat file has a px_spacing array apart from the image array. I am not sure how this is helpful.
def mat_to_image(mat_image):
f = loadmat(mat_image,appendmat=True)
image = np.array(f.get('I')).astype(np.float32)
mean = image.mean()
std= image.std()
print(mean, std)
hi = np.max(image)
lo = np.min(image)
image = (((image - lo)/(hi-lo))*255).astype(np.uint8)
im = Image.fromarray(image,mode='RGB')
return im
images=mat_to_image(dir/filename)
cv_img = cv2.cvtColor(np.array(images), cv2.COLOR_GRAY2RGB)
Normally plotting the .mat file fetches a non-grayscale(RGB image)
imgplot= plt.imshow(loadmat(img,appendmat=True).get('I'))
plt.show()
Here is how the mat file looks after print(loadmat('filename'))
{'__header__': b'MATLAB 5.0 MAT-file, Platform: PCWIN64, Created on: Mon Sep 9 11:32:54 2019',
'__version__': '1.0',
'__globals__': [],
'I': array([
[ 81, 75, 74, 75, -11, 14, 49, 37, 29, -24, -183, -349, -581, -740],
[ 51, 33, 67, 36, 1, 42, 30, 49, 47, 42, 14, -85, -465, -727],
[ 23, 31, 36, 20, 54, 70, 44, 56, 56, 79, 62, 19, -204, -595],
[ 7, 12, 36, 47, 59, 68, 74, 56, 59, 100, 74, 34, -3, -353],
[ 23, 19, 51, 87, 86, 79, 91, 76, 96, 95, 52, 51, 74, -141],
[ 18, 51, 54, 97, 93, 94, 98, 83, 119, 71, 36, 69, 50, -16],
[ -10, 5, 53, 92, 69, 87, 103, 114, 118, 77, 51, 68, 30, 0],
[ -24, 11, 74, 80, 49, 68, 106, 129, 107, 63, 57, 70, 39, -1],
[ -45, 43, 83, 69, 43, 64, 98, 108, 90, 35, 27, 55, 31, -13],
[ -9, 32, 83, 78, 66, 106, 89, 85, 58, 43, 31, 39, 28, 7],
[ 45, 35, 76, 45, 51, 84, 55, 66, 49, 41, 39, 28, 13, -7],
[ 85, 67, 61, 45, 69, 53, 23, 32, 31, -12, -34, -182, -376, -425],
[ 136, 93, 71, 54, 30, 39, 17, -21, -29, -43, -101, -514, -792, -816]
], dtype=int16),
'px_spacing': array([[0.78125]])}

Generate 2 random integer list of different size

I am trying to generate 2 list with different size consisting with random numbers. I can generate 2 list with random numbers, but how to achieve 2 different length of lists?
import random
list1 = random.sample(xrange(100), 10)
list2 = random.sample(xrange(100), 10)
print(list1)
print(list2)
Need to generate the lists with 2 random different sizes as well, as if both the lists are completely random.
Try the below code. Hope this would help.
If you want to create random number list of two different sizes. Then you can explicitly, pass the size of the list as a second argument, as given below.
import random
list1 = random.sample(xrange(100), 100)
list2 = random.sample(xrange(100), 10)
print(list1)
print(list2)
Ouput will be :
[46, 73, 13, 89, 44, 23, 74, 8, 19, 79, 36, 80, 85, 42, 82, 39, 61, 15, 27, 68, 67, 30, 11, 21, 86, 16, 63, 95, 17, 90, 37, 81, 20, 71, 93, 99, 40, 6, 47, 92, 58, 35, 12, 2, 10, 98, 87, 50, 51, 97, 70, 65, 78, 22, 72, 45, 59, 0, 52, 14, 1, 84, 43, 24, 54, 31, 18, 69, 7, 75, 53, 25, 57, 94, 83, 66, 3, 5, 88, 32, 4, 28, 29, 55, 9, 77, 60, 62, 41, 76, 48, 56, 34, 91, 33, 96, 49, 38, 26, 64]
[82, 58, 74, 61, 21, 77, 53, 35, 44, 59]
Now if you want to randomly decide the size of the list, the pass a random number as a second argument, by using randint function
import random
list1 = random.sample(range(100), random.randint(1,101))
list2 = random.sample(range(100), random.randint(1,101))
print(list1)
print(list2)
Output would be:
[93, 60, 82, 53, 16, 42, 0, 68, 88, 11, 89, 62, 38, 14, 27, 8, 45, 25, 83, 97, 94]
[30, 5, 19, 11, 14, 6, 7, 86, 16, 53, 71, 12, 90, 32]
You can try something like this, which would randomly generate the size between 1 and 10.
import random
list1 = random.sample(range(100), random.randint(1,10))
list2 = random.sample(range(100), random.randint(1,10))
print(list1)
print(list2)
This will generate random length of the lists. Hope it helps !
You need to randomize the second Parameter as well to become lists of random size:
import random
list1 = random.sample(range(100), random.randint(1,10))
list2 = random.sample(range(100), random.randint(1,10))
print(list1)
print(list2)

code execution using if elif statement

interface port-channel 1
ALLOWED_VLAN 2,4-7,27,30-31,38-39,41-42,48-50
ALLOWED_VLAN 74,678,1101-1102,1201-1202
interface port-channel 2
ALLOWED_VLAN 37,51-73,75-76,1051-1052,2001
interface port-channel 101
ALLOWED_VLAN 10,18-19,37,39,51-52,75-76,901-902
ALLOWED_VLAN 2901-2902,3204,3305
import re
import itertools
fileOpen3 = open('C:\\Python36\\execrice\\inter.txt')
list3 = []
for line in fileOpen3.readlines():
if line.startswith('ALLOWED_VLAN'):
allowedVlan = re.compile(r'\d+\S+')
list1 = allowedVlan.findall(line)
st1 = list1[0]
pv1 = st1.split(',')
list3.append(pv1)
merged = list(itertools.chain.from_iterable(list3))
singleVlanDigit = []
expandedVlan1 = []
for i in merged:
rangeOfVlan = []
if '-' in i:
rangeOfVlan.append(i)
else:
singleVlanDigit.append(i)
singleVlanDigit = list(map(int,singleVlanDigit))
for j in rangeOfVlan:
l = j.split('-')
startVlan = int(l[0])
endVlan = int(l[1])
for k in range(startVlan,endVlan):
expandedVlan1.append(k)
vlanallowed = singleVlanDigit + expandedVlan1
vlanallowed = list(map(int,vlanallowed))
print (vlanallowed)
elif line.startswith('interface port-channel'):
list3=[]
print ("interface port-channel")
fileOpen3.close()
my program combines all the digits in one single list where as i want it to stop when it reads the "interface port-channel 2" and so on
i want the output of this program to be as below
interface port-channel
[2, 27, 4, 5, 6, 30, 38, 41, 48, 49, 74, 678, 1101, 1201]
interface port-channel
[37, 2001, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 75, 1051]
interface port-channel
[10, 37, 39, 18, 51, 75, 901, 3204, 3305, 2901]
but it produces the below output
interface port-channel
[2, 27, 4, 5, 6, 30, 38, 41, 48, 49]
[2, 27, 74, 678, 4, 5, 6, 30, 38, 41, 48, 49, 1101, 1201]
interface port-channel
[2, 27, 74, 678, 37, 2001, 4, 5, 6, 30, 38, 41, 48, 49, 1101, 1201, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 75, 1051]
interface port-channel
[2, 27, 74, 678, 37, 2001, 10, 37, 39, 4, 5, 6, 30, 38, 41, 48, 49, 1101, 1201, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 75, 1051, 18, 51, 75, 901]
[2, 27, 74, 678, 37, 2001, 10, 37, 39, 3204, 3305, 4, 5, 6, 30, 38, 41, 48, 49, 1101, 1201, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 75, 1051, 18, 51, 75, 901, 2901]
after reinitialising the list3 in elif block as suggested by Boar i got the below results which is very to my end results
interface port-channel
[2, 27, 4, 5, 6, 30, 38, 41, 48, 49]
[2, 27, 74, 678, 4, 5, 6, 30, 38, 41, 48, 49, 1101, 1201]
interface port-channel
[37, 2001, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 75, 1051]
interface port-channel
[10, 37, 39, 18, 51, 75, 901]
[10, 37, 39, 3204, 3305, 18, 51, 75, 901, 2901]
but i want the results to be like this
interface port-channel
[2, 27, 74, 678, 4, 5, 6, 30, 38, 41, 48, 49, 1101, 1201]
interface port-channel
[37, 2001, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 75, 1051]
interface port-channel
[10, 37, 39, 3204, 3305, 18, 51, 75, 901, 2901]
I've had to guess at where the line breaks fall in your data but I think I have it right.
The repeated data is because you are initializing list3 in the wrong place. It should be after if line.startswith('ALLOWED_VLAN'):
With that fix, your program does this:
interface port-channel
[2, 27, 4, 5, 6, 30, 38, 41, 48, 49]
[74, 678, 1101, 1201]
interface port-channel
[37, 2001, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 75, 1051]
interface port-channel
[10, 37, 39, 18, 51, 75, 901]
[3204, 3305, 2901]
which is close to what you want. You just need to combine the lists of integers before printing them. Combine the values of vnlanallowed in the if part of the loop, then print it out and reinitialize it in the elif part.
To get the exact format of the output you want, you need to wait until you know you have all of the lists of port-channel numbers before printing them. You only know that when you see a new list start, or you reach end-of-file.
import re
import itertools
fileOpen3 = open(r'E:\...\inter.txt')
channel_list = []
for line in fileOpen3.readlines():
if line.startswith('ALLOWED_VLAN'):
list3 = []
allowedVlan = re.compile(r'\d+\S+')
list1 = allowedVlan.findall(line)
st1 = list1[0]
pv1 = st1.split(',')
list3.append(pv1)
merged = list(itertools.chain.from_iterable(list3))
singleVlanDigit = []
expandedVlan1 = []
for i in merged:
rangeOfVlan = []
if '-' in i:
rangeOfVlan.append(i)
else:
singleVlanDigit.append(i)
singleVlanDigit = list(map(int,singleVlanDigit))
for j in rangeOfVlan:
l = j.split('-')
startVlan = int(l[0])
endVlan = int(l[1])
for k in range(startVlan,endVlan):
expandedVlan1.append(k)
vlanallowed = singleVlanDigit + expandedVlan1
vlanallowed = list(map(int,vlanallowed))
#print (vlanallowed)
channel_list.extend(vlanallowed)
elif line.startswith('interface port-channel'):
if channel_list:
print ("interface port-channel", channel_list)
channel_list = []
print("interface port-channel", channel_list)
fileOpen3.close()

Percent groovy list slicing

my code:
def originalList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100]
def newList = orginalList.percent(0.05,0.95) //I have no idea what I'm doing here
println newList
I have an original list of numbers, they are 1 - 100 and i want to make a new list from the original list however the new list must only have data that belongs to the sub-range 5%- 95% of the original list
so the new list must be like [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18....95]. How do i do that? i know my newList code is wrong
You mean like:
originalList[ 4..94 ] // zero starting pos
Or do you need percentages?
You could do:
originalList[ (originalList.size() * 0.05 - 1)..<(originalList.size() * 0.95) ]
You could also use the metaClass:
List.metaClass.percent { double lower, double upper ->
int d = lower * delegate.size() - 1
int t = upper * delegate.size()
delegate.take( t ).drop( d )
}
originalList.percent( 0.05, 0.95 )

Resources