"installing" Kaitai Struct Python - python-3.x

I need help with installing Kaitai Struct on my Laptop.
I installed python-kaitaistruct and compiled the network files.
But i get an import error:
Traceback (most recent call last):
File "test2.py", line 1, in <module>
from ethernet_frame import *
File "/home/bene/python/ethernet_frame.py", line 15, in <module>
from ipv6_packet import Ipv6Packet
File "/home/bene/python/ipv6_packet.py", line 17, in <module>
from ipv4_packet import Ipv4Packet
File "/home/bene/python/ipv4_packet.py", line 17, in <module>
from ipv6_packet import Ipv6Packet
ImportError: cannot import name 'Ipv6Packet
My folder looks like this:
insgesamt 76K
drwxr-xr-x 3 bene bene 330 19. Jan 16:03 .
drwx------ 24 bene bene 4,0K 19. Jan 16:06 ..
-rw-r--r-- 1 bene bene 42 5. Jan 12:38 country.py
-rw-r--r-- 1 bene bene 8,0K 5. Jan 12:09 dns_packet.py
-rw-r--r-- 1 bene bene 1,6K 5. Jan 12:09 ethernet_frame.py
-rw-r--r-- 1 bene bene 3,0K 5. Jan 12:09 icmp_packet.py
-rw-r--r-- 1 bene bene 7,7K 5. Jan 12:09 ipv4_packet.py
-rw-r--r-- 1 bene bene 2,7K 5. Jan 12:09 ipv6_packet.py
-rw-r--r-- 1 bene bene 6,4K 5. Jan 12:09 microsoft_network_monitor_v2.py
-rw-r--r-- 1 bene bene 7,0K 5. Jan 12:09 pcap.py
drwxr-xr-x 2 bene bene 180 5. Jan 12:12 __pycache__
-rw-r--r-- 1 bene bene 1,1K 5. Jan 12:09 tcp_segment.py
-rw-r--r-- 1 bene bene 518 5. Jan 12:32 test1.py
-rw-r--r-- 1 bene bene 596 19. Jan 15:56 test2.py
-rw-r--r-- 1 bene bene 667 5. Jan 12:38 test.py
-rw-r--r-- 1 bene bene 880 5. Jan 12:09 udp_datagram.py
-rw-r--r-- 1 bene bene 986 5. Jan 12:09 windows_systemtime.py
and the file i executed:
from ethernet_frame import *
import socket
s = socket.socket(socket.AF_PACKET,socket.SOCK_RAW,socket.ntohs(3))
def network(buf):
io = BytesIO(buf)
ksio = KaitaiStream(io)
pkt = EthernetFrame(ksio)
dummy = pkt.ipv4_body.src_ip_addr
print(dummy)
while True:
p = s.recvfrom(65565)
network(p)
Can someone help me maybe i installed it wrong?
Or a Full Guide how to install and use it would be cool :DD
Thank you <3

I had a similar issue and after checking the .py file created by the compiler I found editing the import sequence solved the issue. Example:
import ipv4_packet was causing the same error as you have.
I checked ipv4_packet.py and it had an import statement from ipv6_packet import Ipv6Packet.
# This is a generated file! Please edit source .ksy file and use kaitai-struct-compiler to rebuild
from pkg_resources import parse_version
from kaitaistruct import __version__ as ks_version, KaitaiStruct, KaitaiStream, BytesIO
from enum import Enum
if parse_version(ks_version) < parse_version('0.7'):
raise Exception("Incompatible Kaitai Struct Python API: 0.7 or later is required, but you have %s" % (ks_version))
from udp_datagram import UdpDatagram
from tcp_segment import TcpSegment
from ipv6_packet import Ipv6Packet
from icmp_packet import IcmpPacket
class Ipv4Packet(KaitaiStruct):
class ProtocolEnum(Enum):
ipv6_packet inturn tries to import the class from ipv4_packet: from ipv4_packet import Ipv4Packet, and causes the import error. If the lines (ipv4_packet .py):
from udp_datagram import UdpDatagram
from tcp_segment import TcpSegment
from ipv6_packet import Ipv6Packet
from icmp_packet import IcmpPacket
are moved to after the class definition there is no error.
Just playing around with Kaitai Struct for the first time this morning, I am sure this does not need to be done manually and there is an issue in the compilation I/we are doing, but this works if you just want to have a quick play.

I believe you should put the files that Kaitai Struct generated for you somewhere where Python will find it. Probably the simplest solution so far would be to just keep it in current directory and launch test2.py with something like:
PYTHONPATH=. python ./test2.py
Alternatively, if you want to keep everything in the same directory, newer compilers allow you to specify --python-package . to generate package imports which seek stuff in current directory.

Related

"OSError: Starting path not found" when importing spotifycharts

I get the following error when importing spotifycharts:
OSError: Starting path not found
When I do pip freeze it's listed.
Full error:
OSError Traceback (most recent call last)
/var/folders/fm/ck9x_7l55y93t7gkxl93g_q40000gn/T/ipykernel_57786/3799758561.py in <module>
1 import environs
----> 2 import spotifycharts as sc
/opt/anaconda3/envs/python39/lib/python3.9/site-packages/spotifycharts/__init__.py in <module>
9 from tqdm import auto
10
---> 11 from spotifycharts import classes
12 from spotifycharts import settings
13 from spotifycharts import exceptions
/opt/anaconda3/envs/python39/lib/python3.9/site-packages/spotifycharts/classes.py in <module>
10
11 import spotifycharts as sc
---> 12 from spotifycharts import settings
13 from spotifycharts import exceptions
14 from spotifycharts.logging import logger
/opt/anaconda3/envs/python39/lib/python3.9/site-packages/spotifycharts/settings.py in <module>
1 import environs
2 configuration = environs.Env()
----> 3 configuration.read_env('spotifycharts/configuration.env')
4
5
/opt/anaconda3/envs/python39/lib/python3.9/site-packages/environs/__init__.py in read_env(path, recurse, verbose, override)
413 if not start_dir: # Only a filename was given
414 start_dir = os.getcwd()
--> 415 for dirname in _walk_to_root(start_dir):
416 check_path = Path(dirname) / env_name
417 if check_path.exists():
/opt/anaconda3/envs/python39/lib/python3.9/site-packages/dotenv/main.py in _walk_to_root(path)
244 """
245 if not os.path.exists(path):
--> 246 raise IOError('Starting path not found')
247
248 if os.path.isfile(path):
OSError: Starting path not found
How do I solve this?
Was able to find a workaround.
Open the settings.py file located in /opt/anaconda3/envs/python39/lib/python3.9/site-packages/spotifycharts/ in above case.
Add the full path to the read_env() method
Change
configuration.read_env('spotifycharts/configuration.env')
to
configuration.read_env('/opt/anaconda3/envs/python39/lib/python3.9/site-packages/spotifycharts/configuration.env')

No module named keras.api Mask R CNN

I am in the demo file of the Mask R CNN repo from matterport. Trying to run the first cell but encountering the following error. I have keras 2.3.0 installed. Running Python 3.8. The below is a trace of the error from the model.py file inside the Mask R CNN repo, which came with the clone. Thanks!
The repo referred to is here: https://github.com/matterport/Mask_RCNN.git
Thank you all for your kind support.
ModuleNotFoundError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_3220/983756133.py in <module>
7 import matplotlib
8 import matplotlib.pyplot as plt
----> 9 import keras
10
11 # Root directory of the project
~\OneDrive\New Project\myenv1\lib\site-packages\keras\__init__.py in <module>
1 from __future__ import absolute_import
2
----> 3 from . import utils
4 from . import activations
5 from . import applications
~\OneDrive\New Project\myenv1\lib\site-packages\keras\utils\__init__.py in <module>
24 from .layer_utils import get_source_inputs
25 from .layer_utils import print_summary
---> 26 from .vis_utils import model_to_dot
27 from .vis_utils import plot_model
28 from .np_utils import to_categorical
~\OneDrive\New Project\myenv1\lib\site-packages\keras\utils\vis_utils.py in <module>
5
6 import os
----> 7 from ..models import Model
8 from ..layers.wrappers import Wrapper
9
~\OneDrive\New Project\myenv1\lib\site-packages\keras\models.py in <module>
10 from .engine.input_layer import Input
11 from .engine.input_layer import InputLayer
---> 12 from .engine.training import Model
13 from .engine.sequential import Sequential
14 from .engine.saving import save_model
~\OneDrive\New Project\myenv1\lib\site-packages\keras\engine\__init__.py in <module>
6 from .base_layer import Layer
7 from .network import get_source_inputs
----> 8 from .training import Model
~\OneDrive\New Project\myenv1\lib\site-packages\keras\engine\training.py in <module>
12 from .network import Network
13 from .base_layer import Layer
---> 14 from . import training_utils
15 from . import training_arrays
16 from . import training_generator
~\OneDrive\New Project\myenv1\lib\site-packages\keras\engine\training_utils.py in <module>
15 from .. import backend as K
16 from .. import losses
---> 17 from .. import metrics as metrics_module
18 from ..utils import Sequence
19 from ..utils import generic_utils
~\OneDrive\New Project\myenv1\lib\site-packages\keras\metrics.py in <module>
1848 import tensorflow as tf
1849 if tf.__version__ >= '2.0.0':
-> 1850 BaseMeanIoU = tf.keras.metrics.MeanIoU
1851
1852
~\OneDrive\New Project\myenv1\lib\site-packages\tensorflow\python\util\lazy_loader.py in __getattr__(self, item)
60
61 def __getattr__(self, item):
---> 62 module = self._load()
63 return getattr(module, item)
64
~\OneDrive\New Project\myenv1\lib\site-packages\tensorflow\python\util\lazy_loader.py in _load(self)
43 """Load the module and insert it into the parent's globals."""
44 # Import the target module and insert it into the parent's namespace
---> 45 module = importlib.import_module(self.__name__)
46 self._parent_module_globals[self._local_name] = module
47
~\AppData\Local\Programs\Python\Python38\lib\importlib\__init__.py in import_module(name, package)
125 break
126 level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)
128
129
ModuleNotFoundError: No module named 'keras.api'
I read the repo that you have referred. You maybe try run your code with environment below:
Python 3.4, TensorFlow 1.3, Keras 2.0.8

unable to import pytorch-lightning

I installed pytorch-lightning using pip, and I'm running on Mac.
I tried:
! pip install pytorch-lightning --upgrade
! pip install pytorch-lightning-bolts
(finished successfully)
and then:
import pytorch_lightning as pl
and what I get is:
--
-------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-3-f3b4217dcea1> in <module>
7 from torchvision.datasets import MNIST
8 from torchvision import transforms
----> 9 import pytorch_lightning as pl
10 from pytorch_lightning.metrics.functional import accuracy
11 tmpdir = os.getcwd()
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/__init__.py in <module>
60 # We are not importing the rest of the lightning during the build process, as it may not be compiled yet
61 else:
---> 62 from pytorch_lightning import metrics
63 from pytorch_lightning.callbacks import Callback
64 from pytorch_lightning.core import LightningDataModule, LightningModule
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/metrics/__init__.py in <module>
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
---> 14 from pytorch_lightning.metrics.classification import ( # noqa: F401
15 Accuracy,
16 AUC,
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/metrics/classification/__init__.py in <module>
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
---> 14 from pytorch_lightning.metrics.classification.accuracy import Accuracy # noqa: F401
15 from pytorch_lightning.metrics.classification.auc import AUC # noqa: F401
16 from pytorch_lightning.metrics.classification.auroc import AUROC # noqa: F401
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/metrics/classification/accuracy.py in <module>
16 import torch
17
---> 18 from pytorch_lightning.metrics.functional.accuracy import _accuracy_compute, _accuracy_update
19 from pytorch_lightning.metrics.metric import Metric
20
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/metrics/functional/__init__.py in <module>
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
---> 14 from pytorch_lightning.metrics.functional.accuracy import accuracy # noqa: F401
15 from pytorch_lightning.metrics.functional.auc import auc # noqa: F401
16 from pytorch_lightning.metrics.functional.auroc import auroc # noqa: F401
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/metrics/functional/accuracy.py in <module>
16 import torch
17
---> 18 from pytorch_lightning.metrics.classification.helpers import _input_format_classification, DataType
19
20
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/metrics/classification/helpers.py in <module>
17 import torch
18
---> 19 from pytorch_lightning.metrics.utils import select_topk, to_onehot
20 from pytorch_lightning.utilities import LightningEnum
21
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/metrics/utils.py in <module>
16 import torch
17
---> 18 from pytorch_lightning.utilities import rank_zero_warn
19
20 METRIC_EPS = 1e-6
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/utilities/__init__.py in <module>
16 import numpy
17
---> 18 from pytorch_lightning.utilities.apply_func import move_data_to_device # noqa: F401
19 from pytorch_lightning.utilities.distributed import ( # noqa: F401
20 AllGatherGrad,
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/utilities/apply_func.py in <module>
23
24 from pytorch_lightning.utilities.exceptions import MisconfigurationException
---> 25 from pytorch_lightning.utilities.imports import _TORCHTEXT_AVAILABLE
26
27 if _TORCHTEXT_AVAILABLE:
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/utilities/imports.py in <module>
54 _TORCH_GREATER_EQUAL_1_7 = _compare_version("torch", operator.ge, "1.7.0")
55 _TORCH_QUANTIZE_AVAILABLE = bool([eg for eg in torch.backends.quantized.supported_engines if eg != 'none'])
---> 56 _APEX_AVAILABLE = _module_available("apex.amp")
57 _BOLTS_AVAILABLE = _module_available('pl_bolts')
58 _DEEPSPEED_AVAILABLE = not _IS_WINDOWS and _module_available('deepspeed')
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pytorch_lightning/utilities/imports.py in _module_available(module_path)
32 """
33 try:
---> 34 return find_spec(module_path) is not None
35 except AttributeError:
36 # Python 3.6
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/util.py in find_spec(name, package)
92 parent_name = fullname.rpartition('.')[0]
93 if parent_name:
---> 94 parent = __import__(parent_name, fromlist=['__path__'])
95 try:
96 parent_path = parent.__path__
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/apex/__init__.py in <module>
11 ISessionFactory)
12 from pyramid.security import NO_PERMISSION_REQUIRED
---> 13 from pyramid.session import UnencryptedCookieSessionFactoryConfig
14 from pyramid.settings import asbool
15
ImportError: cannot import name 'UnencryptedCookieSessionFactoryConfig' from 'pyramid.session' (unknown location
I guess this is an outdated issue as we have cut out TorchMetrics to a standalone package. Please, check out the latest PytorchLightning.
Try installing it from the GitHub repository first before importing it in the notebook.
Run the following command in the Notebook:
!pip install git+https://github.com/PyTorchLightning/pytorch-lightning

Python logging call crashes at usesTime

Every once in 50-100 calls to the logger, the program crashes with the following trace messages:
Mar 20 07:10:14 service.bash[7693]: Fatal Python error: Cannot recover from stack overflow.
Mar 20 07:10:14 service.bash[7693]: Current thread 0x76fa3010 (most recent call first):
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 381 in usesTime
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 537 in usesTime
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 569 in format
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 831 in format
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 981 in emit
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 856 in handle
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 1488 in callHandlers
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 1426 in handle
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 1416 in _log
Mar 20 07:10:14 service.bash[7693]: File "/usr/lib/python3.5/logging/__init__.py", line 1280 in info
Mar 20 07:10:14 service.bash[7693]: File "*****************_app/base.py", line 63 in log_this
Any idea what could be causing this crash?
Don't see similar or other logging calls elsewhere in the program crashing it.
Here is the stack of the calls made to the logger:
self.info("cs={} miso={} mosi{} clk{}".format( self.csPin, self.misoPin, self.mosiPin, self.clkPin))
|
self.log_this("info", msg)
|
self.log.info(msg)
The logger is setup in the base class initialization routine in the following way:
# Global logger is declared as a class attribute
cls.log = logging.getLogger(cls.args["app"])
c_handler = logging.StreamHandler()
f_handler = logging.handlers.RotatingFileHandler(
cls.args["--log"],
maxBytes=(10**6)*int(cls.args["--logsize"]), # CLI is in MB
backupCount=1)
# Create handlers
if cls.args["--debug"]:
cls.log.setLevel(logging.DEBUG)
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.DEBUG)
else:
cls.log.setLevel(logging.INFO)
c_handler.setLevel(logging.INFO)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s: %(levelname)s: %(message)s')
f_format = logging.Formatter('%(asctime)s: %(name)s: %(levelname)s: %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
cls.log.addHandler(c_handler)
cls.log.addHandler(f_handler)
cls.log.info("Logger initialized")
Thank you.
Are you using Windows and switching to an UTF8 locale within Python? There is a Python bug in this specific scenario (similar issue: https://bugs.python.org/issue36792). Here's a minimal code sample that reproduces the bug on my machine:
import locale
import logging
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter('%(asctime)s'))
logging.getLogger().setLevel(logging.DEBUG)
logging.getLogger().addHandler(handler)
logging.info(1)
locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
logging.info(2)
Output:
2019-10-25 21:20:15,657
Process finished with exit code -1073740940 (0xC0000374)
If you are unsure if this might be related to your problem, try adding the following line in front of your call to the logging module and see if it solves the problem:
locale.setlocale(locale.LC_ALL, 'en_US')

Cannot import tensorflow in IPyhon3

I am just pasting the output of my bash shell which could be quite explicit. Running IPython:
petrux#orion:~/Projects/dket$ ipython
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.0.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import tensorflow as tf
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-41389fad42b5> in <module>()
----> 1 import tensorflow as tf
/home/petrux/.local/lib/python3.5/site-packages/tensorflow/__init__.py in <module>()
22
23 # pylint: disable=wildcard-import
---> 24 from tensorflow.python import *
25 # pylint: enable=wildcard-import
26
/home/petrux/.local/lib/python3.5/site-packages/tensorflow/python/__init__.py in <module>()
73
74 # Protocol buffers
---> 75 from tensorflow.core.framework.graph_pb2 import *
76 from tensorflow.core.framework.node_def_pb2 import *
77 from tensorflow.core.framework.summary_pb2 import *
/home/petrux/.local/lib/python3.5/site-packages/tensorflow/core/framework/graph_pb2.py in <module>()
4 import sys
5 _b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
----> 6 from google.protobuf import descriptor as _descriptor
7 from google.protobuf import message as _message
8 from google.protobuf import reflection as _reflection
ImportError: No module named 'google.protobuf'; 'google' is not a package
while running the regular python interpreter:
petrux#orion:~/Projects/dket$ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>>
I am running Ubuntu 16.04 as OS.
Thanks in advance.

Resources