When generating sad files using the python sandbox the connections do not appear to be added to the xml generated using sb.generateSADXML, conversely when you load a sad file containing connections they are not made in the sandbox. Tested in REDHAWK 2.2 & 2.1.3
In [3]: myreader = sb.launch('rh.FileReader')
2018-10-10 11:50:24 INFO FileReader_i:352 - Using sample rate of 2.5e+07 Sps
In [4]: mywriter = sb.launch('rh.FileWriter')
2018-10-10 11:50:39 INFO FileWriter_i:122 - Cannot determine domain, defaulting to local $SDRROOT filesystem
In [5]: myreader.connect(mywriter,usesPortName='dataOctet_out')
In [6]: sb.show()
Components Running:
------------------
rh_FileReader_1 <local component 'rh_FileReader_1' at 0x7fe3396b9c10>
rh_FileWriter_2 <local component 'rh_FileWriter_2' at 0x7fe3395e1d10>
Services Running:
----------------
Component Connections:
---------------------
rh_FileReader_1/dataOctet_out [IDL:BULKIO/dataOctet:1.0] -> rh_FileWriter_2/dataOctet_in [IDL:BULKIO/dataOctet:1.0]
Event Channels:
--------------
SDRROOT:
-------
/var/redhawk/sdr
In [7]: sb.ge
sb.generateSADXML sb.getDEBUG sb.getIDE_REF sb.getService
sb.getComponent sb.getEventChannel sb.getSDRROOT
In [7]: sb.generateSADXML('andy')
Out[7]: '<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE softwareassembly PUBLIC "-//JTRS//DTD SCA V2.2.2 SAD//EN" "softwareassembly.dtd">\n<softwareassembly id="DCE:58dd2953-95e8-42f7-a420-f1c9323b57e0" name="andy">\n <componentfiles>\n\n <componentfile id="rh.FileReader_e4cebf50-be28-4cc6-a73f-4fd22a8a65f0" type="SPD">\n <localfile name="/components/rh/FileReader/FileReader.spd.xml"/>\n </componentfile>\n\n <componentfile id="rh.FileWriter_0d2dd048-2c35-4616-be5c-02810427f2e4" type="SPD">\n <localfile name="/components/rh/FileWriter/FileWriter.spd.xml"/>\n </componentfile>\n </componentfiles>\n <partitioning>\n\n <componentplacement>\n <componentfileref refid="rh.FileReader_e4cebf50-be28-4cc6-a73f-4fd22a8a65f0"/>\n <componentinstantiation id="DCE:83124a9b-195b-4b6b-a638-308b643779e6">\n <usagename>rh_FileReader_1</usagename>\n <findcomponent>\n <namingservice name="rh_FileReader_1"/>\n </findcomponent>\n </componentinstantiation>\n </componentplacement>\n\n <componentplacement>\n <componentfileref refid="rh.FileWriter_0d2dd048-2c35-4616-be5c-02810427f2e4"/>\n <componentinstantiation id="DCE:7511beb1-2b07-48b2-845d-27f6e252fe40">\n <usagename>rh_FileWriter_2</usagename>\n <findcomponent>\n <namingservice name="rh_FileWriter_2"/>\n </findcomponent>\n </componentinstantiation>\n </componentplacement>\n </partitioning>\n\n <assemblycontroller>\n <componentinstantiationref refid="DCE:83124a9b-195b-4b6b-a638-308b643779e6"/>\n </assemblycontroller>\n <connections>\n </connections>\n</softwareassembly>\n'
In [8]:
This issue will be resolved in REDHAWK 2.2.1
Related
the title. I have copied the code as is from google's website but it is not working:
import io
from google.cloud import vision_v1
def sample_batch_annotate_files(file_path="path/to/your/document.pdf"):
"""Perform batch file annotation."""
client = vision_v1.ImageAnnotatorClient()
# Supported mime_type: application/pdf, image/tiff, image/gif
mime_type = "application/pdf"
with io.open(file_path, "rb") as f:
content = f.read()
input_config = {"mime_type": mime_type, "content": content}
features = [{"type_": vision_v1.Feature.Type.DOCUMENT_TEXT_DETECTION}]
# The service can process up to 5 pages per document file. Here we specify
# the first, second, and last page of the document to be processed.
pages = [1, 2, -1]
requests = [{"input_config": input_config, "features": features, "pages": pages}]
response = client.batch_annotate_files(requests=requests)
for image_response in response.responses[0].responses:
print(u"Full text: {}".format(image_response.full_text_annotation.text))
for page in image_response.full_text_annotation.pages:
for block in page.blocks:
print(u"\nBlock confidence: {}".format(block.confidence))
for par in block.paragraphs:
print(u"\tParagraph confidence: {}".format(par.confidence))
for word in par.words:
print(u"\t\tWord confidence: {}".format(word.confidence))
Yet it is not working and giving me the following error:
File "---", line 16, in sample_batch_annotate_files
features = [{"type_": vision_v1.Feature.Type.DOCUMENT_TEXT_DETECTION}]
AttributeError: module 'google.cloud.vision_v1' has no attribute 'Feature'
I am using a conda environment and this is the .yml - as I read in another posts, I have installed the Google-api-python-client and google-cloud-vision as recommended. Might it be related to the 'google-cloud-vision' version 1.0.1 when it should be 3.5.1 ? how to update it? I installed it with: conda install -c conda-forge google-cloud-vision
channels:
- conda-forge
- defaults
dependencies:
- aiohttp=3.8.1=py39hb82d6ee_1
- aiosignal=1.2.0=pyhd8ed1ab_0
- async-timeout=4.0.2=pyhd8ed1ab_0
- attrs=22.1.0=pyh71513ae_1
- brotlipy=0.7.0=py39hb82d6ee_1004
- ca-certificates=2022.9.24=h5b45459_0
- cachetools=5.2.0=pyhd8ed1ab_0
- certifi=2022.9.24=pyhd8ed1ab_0
- cffi=1.15.1=py39h0878f49_0
- charset-normalizer=2.1.1=pyhd8ed1ab_0
- cryptography=37.0.4=py39h7bc7c5c_0
- frozenlist=1.3.1=py39hb82d6ee_0
- google-api-core=2.10.1=pyhd8ed1ab_0
- google-api-core-grpc=2.10.1=hd8ed1ab_0
- google-api-python-client=2.64.0=pyhd8ed1ab_0
- google-auth=2.12.0=pyh1a96a4e_0
- google-auth-httplib2=0.1.0=pyhd8ed1ab_1
- google-cloud-core=2.3.2=pyhd8ed1ab_0
- google-cloud-storage=2.5.0=pyh6c4a22f_0
- google-cloud-vision=1.0.1=pyhd8ed1ab_0
- google-crc32c=1.1.2=py39h3fc79e4_3
- google-resumable-media=2.4.0=pyhd8ed1ab_0
- googleapis-common-protos=1.56.4=py39h35db3c3_0
- grpcio=1.46.0=py39hb76b349_1
- grpcio-status=1.41.1=pyhd3eb1b0_0
- httplib2=0.20.4=pyhd8ed1ab_0
- idna=3.4=pyhd8ed1ab_0
- libcrc32c=1.1.2=h0e60522_0
- libprotobuf=3.20.1=h7755175_1
- libzlib=1.2.12=h8ffe710_2
- multidict=6.0.2=py39hb82d6ee_1
- openssl=1.1.1q=h8ffe710_0
- pip=22.2.2=py39haa95532_0
- protobuf=3.20.1=py39hcbf5309_0
- pyasn1=0.4.8=py_0
- pyasn1-modules=0.2.7=py_0
- pycparser=2.21=pyhd8ed1ab_0
- pyopenssl=22.0.0=pyhd8ed1ab_1
- pyparsing=3.0.9=pyhd8ed1ab_0
- pysocks=1.7.1=pyh0701188_6
- python=3.9.13=h6244533_1
- python_abi=3.9=2_cp39
- pyu2f=0.1.5=pyhd8ed1ab_0
- requests=2.28.1=pyhd8ed1ab_1
- rsa=4.9=pyhd8ed1ab_0
- setuptools=63.4.1=py39haa95532_0
- six=1.16.0=pyh6c4a22f_0
- sqlite=3.39.3=h2bbff1b_0
- typing-extensions=4.4.0=hd8ed1ab_0
- typing_extensions=4.4.0=pyha770c72_0
- tzdata=2022c=h04d1e81_0
- uritemplate=4.1.1=pyhd8ed1ab_0
- urllib3=1.26.11=pyhd8ed1ab_0
- vc=14.2=h21ff451_1
- vs2015_runtime=14.27.29016=h5e58377_2
- wheel=0.37.1=pyhd3eb1b0_0
- win_inet_pton=1.1.0=py39hcbf5309_4
- wincertstore=0.2=py39haa95532_2
- yarl=1.7.2=py39hb82d6ee_2
- zlib=1.2.12=h8ffe710_2
Manually installed google-cloud-vision downloading the tar.bz2 from Anaconda.
activate [env]
conda install path-to-tar
By doing that I had my google-cloud-vision package update to the version I chose, but then I had an issue with PROTO. The error said that could find the module "import proto".
What I did is to update: conda install -c conda-forge google-api-python-client
and that solved the issue.
emphasized textHi , I am currently Porting a Piece of code from python 2.7 to python 3.6 , The Code involves dumping data from xml into a list the codes looks as below
import os
import xml.etree.ElementTree as ET
regs_list = []
tree = ET.parse("test.xml")
root = tree.getroot()
for reg in root:
regs_list.append(reg)
for i in range(len(regs_list)):
print (regs_list[i].attrib["name"])
if not (regs_list[i].find("field") == None):
for regs_list[i].field in regs_list[i]:
print (regs_list[i].field.attrib["first_bit"])
The XML looks like this
<register offset="0x4" width="4" defaultValue="0x100000" name="statuscommand" desc="STATUSCOMMAND- Status and Command ">
<field first_bit="30" last_bit="31" WH="ROOO" flask="0xc0000000" name="reserved0" desc=""/>
<field first_bit="29" last_bit="29" WH="1CWRH flask="0x20000000" name="rma" desc=""/>
<field first_bit="28" last_bit="28" WH="1CWRH flask="0x10000000" name="rta" desc=""/>
</register>
<register offset="0x8" width="4" defaultValue="0x8050100" name="reve" desc="REVCLASSCODE - Revision ID and Class Code">
<field first_bit="8" last_bit="31" WH="ROOO" flask="0xffffff00" name="class_codes" desc=""/>
<field first_bit="0" last_bit="7" WH="ROOO" flask="0xff" name="rid" desc=""/>
</register>
This Code works perfectly fine in python 2.7 , we are able to dump both parent (Register) and child (field) into the list regs_list , But in 3.6 we get an error as below
3.6 output "
for regs_list[i].field in regs_list[i]:
AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'field'
2.7 output
Parent and child parsed and dumped in the list without error
Is there a difference in the way xml.etree.ElementTree.Element and lists work in 3.6 and 2.7 ??
This question explains how to read the TLS section of a packet using scapy.
However, my program is not able to read it. All it returns is a bunch of hexadecimal characters
>>> from scapy.all import *
>>> load_layer('tls')
>>> cap = rdpcap('tls.pcap')
>>> p1=cap[0]
>>> p1
<Ether dst=14:cc:20:51:33:ea src=f4:f2:6d:93:51:f1 type=0x800 |<IP version=4 ihl=5 tos=0x0 len=146 id=62970 flags=DF frag=0 ttl=64 proto=tcp chksum=0x50a0 src=192.168.1.143 dst=54.254.250.149 |<TCP sport=49335 dport=50443 seq=549695462 ack=200962336 dataofs=5 reserved=0 flags=PA window=4380 chksum=0xb0ac urgptr=0 |<Raw load="\x17\x03\x01\x00 \xf2\x10\xfd\x95N'\xf2\xaf\x99tp\x93\xbc\xe9\x81w\x91\x1b\xe0\xc9M:\x9a!]\xb0!\xae\xd2\x86\xb0>\x17\x03\x01\x00#d>\x0b\xee\xf0\xab\xded\x02E)\x0e0\xbb\xe6\x82uU\xb22\x87\xd6\xe4n[\x1d\x18\xe8\xd6\x1c\x00N_C\xe6\xdd\xbe\x89#6p\xd9\xaf\x19\xb3s\x07H\xdeF\x88\xdar\x0f\x8a\n!4\xeb\xd3F\xefgH" |>>>>
I want to get the tls record version, tls record length and the tls record content type.
This is screenshot of the packet opened in wireshark.
Can somepne please show me what I am doing wrong and how to read the tls content properly?
I am using Python3.6, and thus am not able to use stable scapy-ssl_tls, which is currently limited to Python 2.
You are so close. You just need to use TLS(pkt.load).
Download a TLS Capture
For this example, use this tls capture from Wireshark's Bugzilla.
We can see that packet 4 is the TLS Client Hello:
tshark -r DNS-over-TLS.pcapng -Y "frame.number==4"
4 0.122267 133.93.28.45 → li280-151.members.linode.com TLSv1 384 Client
Hello 00:00:5e:00:01:18 ← 48:d7:05:df:86:0b
Load with Scapy
Make sure that you have the cryptography library installed, as it's required for loading TLS captures.
>>> import cryptography
>>> # No errors
Reproducing what you have so far with this capture:
>>> from scapy.all import *
>>> load_layer('tls')
>>> cap = rdpcap('DNS-over-TLS.pcapng')
>>> tls_client_hello=cap[3] # Wireshark numbers packets starting at 1, scapy at 0
>>> tls_client_hello
<Ether dst=14:cc:20:51:33:ea src=f4:f2:6d:93:51:f1 type=0x800 |<IP version=4
ihl=5 tos=0x0 len=146 id=62970 flags=DF frag=0 ttl=64 proto=tcp chksum=0x50a0
src=192.168.1.143 dst=54.254.250.149 |<TCP sport=49335 dport=50443 seq=549695462
ack=200962336 dataofs=5 reserved=0 flags=PA window=4380 chksum=0xb0ac urgptr=0 |
<Raw load="\x17\x03\x01\x00
\xf2\x10\xfd\x95N'\xf2\xaf\x99tp\x93\xbc\xe9\x81w\x91\x1b\xe0\xc9M:\x9a!]\xb0!\xa
e\xd2\x86\xb0>\x17\x03\x01\x00#d>\x0b\xee\xf0\xab\xded\x02E)\x0e0\xbb\xe6\x82uU\x
b22\x87\xd6\xe4n[\x1d\x18\xe8\xd6\x1c\x00N_C\xe6\xdd\xbe\x89#6p\xd9\xaf\x19\xb3s\
x07H\xdeF\x88\xdar\x0f\x8a\n!4\xeb\xd3F\xefgH" |>>>>
Note that the part that we want to view is called Raw load. To access this part of the packet, you use tls_client_hello.load. Keep in mind that TLS will take a bytes object that contains the data, but not an entire packet.
>>> TLS(tls_client_hello.load)
<TLS type=handshake version=TLS 1.0 len=313 iv=b'' msg=[<TLSClientHello
msgtype=client_hello msglen=309 version=TLS 1.2 gmt_unix_time=Tue, 18 May 2077
23:20:52 +0000 (3388605652)
random_bytes=d6d533aca04dca42db8b123b0a143dcd580079147122e4de095c15cf sidlen=0
sid='' cipherslen=182 ciphers=[TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
< TLS output truncated ... >
Further Reading
I highly recommend looking at Scapy TLS Notebooks that do a good job of documenting scapy+TLS usage.
Aim: Get the arrival time from the pcap files
Language: python3.7
Tools: Scapy.sniff
Above all ,i want get the arrival time data,in the .pcap ,when i use wireshark ,i saw the data in the Ethernet Frame,but when i use
#Scapy.sniff(offline='.pcap') ,i just get the Ether,TCP,IP and others ,so how can i get that data?
Thanx alot!
>>from scapy.all import *
>>a = sniff(offline = '***.pcap')
>>a[0]
[out]:
<Ether dst=*:*:*:*:*:* src=*:*:*:*:*:* type=** |<IP version=4 ihl=5 tos=0x20 len=52 id=14144 flags=DF frag=0 ttl=109 proto=tcp chksum=0x5e3b src=*.*.*.* dst=*.*.*.* |<TCP sport=gcsp dport=http seq=1619409885 ack=1905830025 dataofs=8 reserved=0 flags=A window=65535 chksum=0xfdb5 urgptr=0 options=[('NOP', None), ('NOP', None), ('SAck', (1905831477, 1905831485))] |>>>
[ ]:
The packet time from the pcap is available in the time member:
print(a[0].time)
It's kept as a floating point value (the standard python "timestamp" format). To get it in a form more easily understandable, you may want to use the datetime module:
>>> from datetime import datetime
>>> dt = datetime.fromtimestamp(a[0].time)
>>> print(dt)
2018-11-12 03:03:00.259780
The scapy documentation isn't great. It can be very instructive to use the interactive help facility. For example, in the interpreter:
$ python
>>> from scapy.all import *
>>> a = sniff(offline='mypcap.pcap')
>>> help(a[0])
This will show you all the methods and attributes of the object represented by a[0]. In your case, that is an instance of class Ether(scapy.packet.Packet).
I am trying to extract stock prices from the website 'iex' and everything works fine until I try to plot my data into a visualization model. Can anyone take a look and see what I am doing wrong? This happens when I put the coding for 'open' as well as 'volume' Thank you!
msft['Open'].plot(label='MSFT',figsize=(16,8),title='Open Title')
gm['Open'].plot(label='gm')
ford['Open'].plot(label='ford')plt.legend()
I get the follow error
KeyError: 'Open'
pandas datareader uses lower case "open":
In [11]: from pandas_datareader import data as web
In [12]: msft = web.DataReader('MSFT', 'iex', "2019-01-01", "2019-01-31")
In [13]: msft.head()
Out[13]:
open high low close volume
date
2019-01-02 99.1266 101.3173 98.5192 100.6899 35329345
2019-01-03 99.6743 99.7589 96.7866 96.9858 42578410
2019-01-04 99.2959 102.0740 98.5093 101.4965 44060620
2019-01-07 101.2077 102.8289 100.5505 101.6259 35656136
2019-01-08 102.6018 103.5278 101.2808 102.3628 31514415
In [14]: msft["open"].head()
Out[14]:
date
2019-01-02 99.1266
2019-01-03 99.6743
2019-01-04 99.2959
2019-01-07 101.2077
2019-01-08 102.6018
Name: open, dtype: float64