Related
Hi I am trying to create a verilog register that outputs its value only when the write signal is high else it is high impedance. Is the following synthesizable?
module R(data_from_bus,data_to_bus,clk,read,write);
input [7:0]data_from_bus;
input clk,read,write;
output reg[7:0] data_to_bus;
reg[7:0] r_reg;
always#(posedge clk)
begin
if (read==1)
r_reg<=data_from_bus;
end
always#(write)
begin
if (write==1)
data_to_bus=r_reg;
else
data_to_bus=8'bz;
end
endmodule
yes, it is synthesizable, but not necessarily doing what you want because of the questionable format.
here's a better (safer) version:
module R(data_from_bus,data_to_bus,clk,read,write);
input [7:0]data_from_bus;
input clk,read,write;
output data_to_bus;
reg[7:0] r_reg;
always#(posedge clk) begin
if (read)
r_reg<=data_from_bus;
else
r_reg<=r_reg;
end
wire[7:0] r_reg_wire;
assign r_reg_wire = r_reg;
assign data_to_bus = write ? r_reg_wire : 8'bz;
endmodule
the main problem of the one you posted is that you are not having an else statement for the first non-blocking assignment: (if (read == 1))
This might result in inferring a latch (but tools are most likely smart enough to fix it implicitly), which does the same thing in simulation as a flip-flop in simulation, but will mess with timing in real life deployment
a really good approach is to use 'always_ff' for registers assignment, 'always_comb' for combinational logic assignment, and 'always_latch' for intended latch (which is rarely used apart from really fishy timing case such as clock gating); but these keyword are only supported in SystemVerilog
Yes.
Here is the result of synthesizing the posted code in the free online tools available at the EDA Playground website, using Mentor Precision.
Please add r_reg to the sensitivity list for the combinational logic to assure the simulation and synthesis results agree. Use always #(*) to accomplish the same thing using a wildcard style approach.
Synthesis ran and produced no errors.
The log is shown below.
The last part of the log is a post synthesis Verilog netlist.
Note the tool used the FDRE primitive to implement the registers bits.
To repeat this process, see the reference design at:
https://www.edaplayground.com/x/2BmJ
Copy the reference design to your EDA Playground account (assuming you have one;you should its free and helpfu) using the copy button.
Paste the design you want to synthesize into the design.v tab.
Run it by clicking the run button.
Log file
[2022-05-08 23:57:07 UTC] precision -shell -file run.do -fileargs "design.sv" && sed 's-$-<br>-g' precision.v > tmp.html && echo '<!DOCTYPE html> <html> <head> <style> body {font-family: monospace;} </style> </head> <body>' > tmp2.html && echo '</body> </html> ' > tmp3.html && cat tmp2.html tmp.html tmp3.html > precision.html
precision: Setting MGC_HOME to /usr/share/precision/Mgc_home ...
precision: Executing on platform: Derived from Red Hat Enterprise Linux 7.1 (Source) -- 5.4.0-107-generic -- x86_64
// Precision RTL Synthesis 64-bit 2021.1.0.4 (Production Release) Tue Jul 20 01:22:31 PDT 2021
//
// Copyright (c) Mentor Graphics Corporation, 1996-2021, All Rights Reserved.
// Portions copyright 1991-2008 Compuware Corporation
// UNPUBLISHED, LICENSED SOFTWARE.
// CONFIDENTIAL AND PROPRIETARY INFORMATION WHICH IS THE
// PROPERTY OF MENTOR GRAPHICS CORPORATION OR ITS LICENSORS
//
// Running on Linux runner#eaa22c631d4a #121-Ubuntu SMP Thu Mar 24 16:04:27 UTC 2022 5.4.0-107-generic x86_64
//
// Start time Sun May 8 19:57:09 2022
# -------------------------------------------------
# Info: [9569]: Logging session transcript to file /home/runner/precision.log
# Warning: [9508]: Results directory is not set. Use new_project, open_project, or set_results_dir.
# Info: [9577]: Input directory: /home/runner
# Info: [9572]: Moving session transcript to file /home/runner/precision.log
# Info: [9558]: Created project /home/runner/project_1.psp in folder /home/runner.
# Info: [9531]: Created directory: /home/runner/impl_1.
# Info: [9557]: Created implementation impl_1 in project /home/runner/project_1.psp.
# Info: [9578]: The Results Directory has been set to: /home/runner/impl_1/
# Info: [9569]: Logging project transcript to file /home/runner/impl_1/precision.log
# Info: [9569]: Logging suppressed messages transcript to file /home/runner/impl_1/precision.log.suppressed
# Info: [9552]: Activated implementation impl_1 in project /home/runner/project_1.psp.
# Info: [20026]: MultiProc: Precision will use a maximum of 8 logical processors.
# Info: [15302]: Setting up the design to use synthesis library "xca7.syn"
# Info: [585]: The global max fanout is currently set to 10000 for Xilinx - ARTIX-7.
# Info: [15328]: Setting Part to: "7A100TCSG324".
# Info: [15329]: Setting Process to: "1".
# Info: [7513]: The default input to Vivado place and route has been set to "Verilog".
# Info: [7512]: The place and route tool for current technology is Vivado.
# Info: [3052]: Decompressing file : /usr/share/precision/Mgc_home/pkgs/psr/techlibs/xca7.syn in /home/runner/impl_1/synlib.
# Info: [3022]: Reading file: /home/runner/impl_1/synlib/xca7.syn.
# Info: [645]: Loading library initialization file /usr/share/precision/Mgc_home/pkgs/psr/userware/xilinx_rename.tcl
# Info: [40000]: hdl-analyze, Release RTLC-Precision 2021a.12
# Info: [42003]: Starting analysis of files in library "work"
# Info: [41002]: Analyzing input file "/home/runner/design.sv" ...
# Info: [670]: Top module of the design is set to: R.
# Info: [668]: Current working directory: /home/runner/impl_1.
# Info: [40000]: RTLC-Driver, Release RTLC-Precision 2021a.12
# Info: [40000]: Last compiled on Jul 2 2021 08:23:33
# Info: [44512]: Initializing...
# Info: [44504]: Partitioning design ....
# Info: [40000]: RTLCompiler, Release RTLC-Precision 2021a.12
# Info: [40000]: Last compiled on Jul 2 2021 08:49:53
# Info: [44512]: Initializing...
# Info: [44522]: Root Module R: Pre-processing...
# Info: [44523]: Root Module R: Compiling...
# Warning: [45784]: "/home/runner/design.sv", line 11: Module R, Net(s) r_reg[7:0]: Although this signal is not part of the sensitivity list of this block, it is being read. This may lead to simulation mismatch.
# Info: [44842]: Compilation successfully completed.
# Info: [44856]: Total lines of RTL compiled: 17.
# Info: [44835]: Total CPU time for compilation: 0.0 secs.
# Info: [44513]: Overall running time for compilation: 1.0 secs.
# Info: [668]: Current working directory: /home/runner/impl_1.
# Info: [15334]: Doing rtl optimizations.
# Info: [671]: Finished compiling design.
# Info: [668]: Current working directory: /home/runner/impl_1.
# Info: [20026]: MultiProc: Precision will use a maximum of 8 logical processors.
# Info: [15002]: Optimizing design view:.work.R.INTERFACE
# Info: [15002]: Optimizing design view:.work.R.INTERFACE
# Info: [8010]: Gated clock transformations: Begin...
# Info: [8010]: Gated clock transformations: End...
# Info: [8053]: Added global buffer BUFGP for Port port:clk
# Info: [3027]: Writing file: /home/runner/impl_1/R.edf.
# Info: [3027]: Writing file: /home/runner/impl_1/R.xdc.
# Info: -- Writing file /home/runner/impl_1/R.tcl
# Info: [3027]: Writing file: /home/runner/impl_1/R.v.
# Info: -- Writing file /home/runner/impl_1/R.tcl
# Info: [671]: Finished synthesizing design.
# Info: [11019]: Total CPU time for synthesis: 0.8 s secs.
# Info: [11020]: Overall running time for synthesis: 1.0 s secs.
# Info: /home/runner/impl_1/precision_tech.sdc
# Info: [3027]: Writing file: /home/runner/precision.v.
# Info: [3027]: Writing file: /home/runner/precision.xdc.
# Info: -- Writing file /home/runner/impl_1/R.tcl
# Info: Info, Command 'auto_write' finished successfully
# Info: Num File Type Path
# Info: --------------------------------------------------------
# Info: 0 /home/runner/impl_1/R_area.rep
# Info: 1 /home/runner/impl_1/R_con_rep.sdc
# Info: 2 /home/runner/impl_1/R_tech_con_rep.sdc
# Info: 3 /home/runner/impl_1/R_fsm.rep
# Info: 4 /home/runner/impl_1/R_dsp_modes.rep
# Info: 5 /home/runner/impl_1/R_ram_modes.rep
# Info: 6 /home/runner/impl_1/R_env.htm
# Info: 7 /home/runner/impl_1/R.edf
# Info: 8 /home/runner/impl_1/R.v
# Info: 9 /home/runner/impl_1/R.xdc
# Info: 10 /home/runner/impl_1/R.tcl
# Info: ***************************************************************
# Info: Device Utilization for 7A100TCSG324
# Info: ***************************************************************
# Info: Resource Used Avail Utilization
# Info: ---------------------------------------------------------------
# Info: IOs 19 210 9.05%
# Info: Global Buffers 1 32 3.12%
# Info: LUTs 1 63400 0.00%
# Info: CLB Slices 1 15850 0.01%
# Info: Dffs or Latches 8 126800 0.01%
# Info: Block RAMs 0 135 0.00%
# Info: DSP48E1s 0 240 0.00%
# Info: ---------------------------------------------------------------
# Info: *****************************************************
# Info: Library: work Cell: R View: INTERFACE
# Info: *****************************************************
# Info: Number of ports : 19
# Info: Number of nets : 40
# Info: Number of instances : 29
# Info: Number of references to this view : 0
# Info: Total accumulated area :
# Info: Number of Dffs or Latches : 8
# Info: Number of LUTs : 1
# Info: Number of Primitive LUTs : 1
# Info: Number of accumulated instances : 29
# Info: *****************************
# Info: IO Register Mapping Report
# Info: *****************************
# Info: Design: work.R.INTERFACE
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | Port | Direction | INFF | OUTFF | TRIFF |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_from_bus(7) | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_from_bus(6) | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_from_bus(5) | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_from_bus(4) | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_from_bus(3) | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_from_bus(2) | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_from_bus(1) | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_from_bus(0) | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_to_bus(7) | Output | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_to_bus(6) | Output | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_to_bus(5) | Output | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_to_bus(4) | Output | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_to_bus(3) | Output | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_to_bus(2) | Output | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_to_bus(1) | Output | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | data_to_bus(0) | Output | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | clk | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | read | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: | write | Input | | | |
# Info: +---------------------+-----------+----------+----------+----------+
# Info: Total registers mapped: 0
# Info: [12022]: Design has no timing constraint and no timing information.
# Info: //
# Info: // Verilog description for cell R,
# Info: // Sun May 8 19:57:18 2022
# Info: //
# Info: // Precision RTL Synthesis, 64-bit 2021.1.0.4//
# Info: module R ( data_from_bus, data_to_bus, clk, read, write ) ;
# Info: input [7:0]data_from_bus ;
# Info: output [7:0]data_to_bus ;
# Info: input clk ;
# Info: input read ;
# Info: input write ;
# Info: wire [7:0]data_from_bus_int;
# Info: wire clk_int;
# Info: wire read_int, write_int, nx57998z1, nx198;
# Info: wire [7:0]r_reg;
# Info: OBUFT \data_to_bus_triBus1(0) (.O (data_to_bus[0]), .I (r_reg[0]), .T (
# Info: nx57998z1)) ;
# Info: OBUFT \data_to_bus_triBus1(1) (.O (data_to_bus[1]), .I (r_reg[1]), .T (
# Info: nx57998z1)) ;
# Info: OBUFT \data_to_bus_triBus1(2) (.O (data_to_bus[2]), .I (r_reg[2]), .T (
# Info: nx57998z1)) ;
# Info: OBUFT \data_to_bus_triBus1(3) (.O (data_to_bus[3]), .I (r_reg[3]), .T (
# Info: nx57998z1)) ;
# Info: OBUFT \data_to_bus_triBus1(4) (.O (data_to_bus[4]), .I (r_reg[4]), .T (
# Info: nx57998z1)) ;
# Info: OBUFT \data_to_bus_triBus1(5) (.O (data_to_bus[5]), .I (r_reg[5]), .T (
# Info: nx57998z1)) ;
# Info: OBUFT \data_to_bus_triBus1(6) (.O (data_to_bus[6]), .I (r_reg[6]), .T (
# Info: nx57998z1)) ;
# Info: OBUFT \data_to_bus_triBus1(7) (.O (data_to_bus[7]), .I (r_reg[7]), .T (
# Info: nx57998z1)) ;
# Info: IBUF write_ibuf (.O (write_int), .I (write)) ;
# Info: IBUF read_ibuf (.O (read_int), .I (read)) ;
# Info: IBUF \data_from_bus_ibuf(0) (.O (data_from_bus_int[0]), .I (
# Info: data_from_bus[0])) ;
# Info: IBUF \data_from_bus_ibuf(1) (.O (data_from_bus_int[1]), .I (
# Info: data_from_bus[1])) ;
# Info: IBUF \data_from_bus_ibuf(2) (.O (data_from_bus_int[2]), .I (
# Info: data_from_bus[2])) ;
# Info: IBUF \data_from_bus_ibuf(3) (.O (data_from_bus_int[3]), .I (
# Info: data_from_bus[3])) ;
# Info: IBUF \data_from_bus_ibuf(4) (.O (data_from_bus_int[4]), .I (
# Info: data_from_bus[4])) ;
# Info: IBUF \data_from_bus_ibuf(5) (.O (data_from_bus_int[5]), .I (
# Info: data_from_bus[5])) ;
# Info: IBUF \data_from_bus_ibuf(6) (.O (data_from_bus_int[6]), .I (
# Info: data_from_bus[6])) ;
# Info: IBUF \data_from_bus_ibuf(7) (.O (data_from_bus_int[7]), .I (
# Info: data_from_bus[7])) ;
# Info: INV ix57998z1315 (.O (nx57998z1), .I (write_int)) ;
# Info: BUFGP clk_ibuf (.O (clk_int), .I (clk)) ;
# Info: GND ps_gnd (.G (nx198)) ;
# Info: FDRE \reg_r_reg(7) (.Q (r_reg[7]), .C (clk_int), .CE (read_int), .D (
# Info: data_from_bus_int[7]), .R (nx198)) ;
# Info: FDRE \reg_r_reg(6) (.Q (r_reg[6]), .C (clk_int), .CE (read_int), .D (
# Info: data_from_bus_int[6]), .R (nx198)) ;
# Info: FDRE \reg_r_reg(5) (.Q (r_reg[5]), .C (clk_int), .CE (read_int), .D (
# Info: data_from_bus_int[5]), .R (nx198)) ;
# Info: FDRE \reg_r_reg(4) (.Q (r_reg[4]), .C (clk_int), .CE (read_int), .D (
# Info: data_from_bus_int[4]), .R (nx198)) ;
# Info: FDRE \reg_r_reg(3) (.Q (r_reg[3]), .C (clk_int), .CE (read_int), .D (
# Info: data_from_bus_int[3]), .R (nx198)) ;
# Info: FDRE \reg_r_reg(2) (.Q (r_reg[2]), .C (clk_int), .CE (read_int), .D (
# Info: data_from_bus_int[2]), .R (nx198)) ;
# Info: FDRE \reg_r_reg(1) (.Q (r_reg[1]), .C (clk_int), .CE (read_int), .D (
# Info: data_from_bus_int[1]), .R (nx198)) ;
# Info: FDRE \reg_r_reg(0) (.Q (r_reg[0]), .C (clk_int), .CE (read_int), .D (
# Info: data_from_bus_int[0]), .R (nx198)) ;
# Info: endmodule
I decode h.264 on Jetson Nano using Open-cv.
I use this Code:
import cv2
try:
cap = cv2.VideoCapture('udp://234.0.0.0:46002', cv2.CAP_FFMPEG)
print(f"cap = {cap}")
except Exception as e:
print(f"Error: {e}")
if not cap.isOpened():
print('VideoCapture not opened')
exit(-1)
while True:
ret, frame = cap.read()
# print(f"frame = {frame}")
try:
cv2.imshow('Image', frame)
except Exception as e:
print(e)
if cv2.waitKey(1) & 0XFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
everything works fine.
now I won't try to optimize my code by decoding using GPU my question is how can I do this?
I see this option:
cap = cv2.VideoCapture('filesrc location=sample2.mp4 ! qtdemux ! queue ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw,format=BGRx ! queue ! videoconvert ! queue ! video/x-raw, format=BGR ! appsink', cv2.CAP_GSTREAMER)
but my source is URL.
I would be happy to any help how to decode h.264 from URL in python using GPU.
I use the FFmpeg command on my computer to get information about the video and I get this plot:
ffmpeg -hide_banner -loglevel debug -i udp://127.0.0.0:46002 -f xv display
Splitting the commandline.
Reading option '-hide_banner' ... matched as option 'hide_banner' (do not
show program banner) with argument '1'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging
level) with argument 'debug'.
Reading option '-i' ... matched as input url with argument
'udp://127.0.0.0:46002'.
Reading option '-f' ... matched as option 'f' (force format) with argument
'xv'.
Reading option 'display' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option hide_banner (do not show program banner) with argument 1.
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url udp://127.0.0.0:46002.
Successfully parsed a group of options.
Opening an input file: udp://127.0.0.0:46002.
[NULL # 0000020a7c5ded80] Opening 'udp://127.0.0.0:46002' for reading
[udp # 0000020a7c5cb700] No default whitelist set
[udp # 0000020a7c5cb700] end receive buffer size reported is 393216
[h264 # 0000020a7c5ded80] Format h264 probed with size=32768 and score=51
[h264 # 0000020a7c5ded80] Before avformat_find_stream_info() pos: 0 bytes
read:33339 seeks:0 nb_streams:1
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[extract_extradata # 0000020a7c60eec0] nal_unit_type: 1(Coded slice of a
non-IDR picture), nal_ref_idc: 2
Last message repeated 1 times
[h264 # 0000020a7c631340] nal_unit_type: 1(Coded slice of a non-IDR
picture), nal_ref_idc: 2
Last message repeated 1 times
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[h264 # 0000020a7c631340] decode_slice_header error
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[h264 # 0000020a7c631340] decode_slice_header error
[h264 # 0000020a7c631340] no frame!
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[extract_extradata # 0000020a7c60eec0] nal_unit_type: 1(Coded slice of a
non-IDR picture), nal_ref_idc: 2
Last message repeated 1 times
[h264 # 0000020a7c631340] nal_unit_type: 1(Coded slice of a non-IDR
picture), nal_ref_idc: 2
Last message repeated 1 times
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[h264 # 0000020a7c631340] decode_slice_header error
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[h264 # 0000020a7c631340] decode_slice_header error
[h264 # 0000020a7c631340] no frame!
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[extract_extradata # 0000020a7c60eec0] nal_unit_type: 1(Coded slice of a
non-IDR picture), nal_ref_idc: 2
Last message repeated 1 times
[h264 # 0000020a7c631340] nal_unit_type: 1(Coded slice of a non-IDR
picture), nal_ref_idc: 2
Last message repeated 1 times
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[h264 # 0000020a7c631340] decode_slice_header error
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[h264 # 0000020a7c631340] decode_slice_header error
[h264 # 0000020a7c631340] no frame!
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[extract_extradata # 0000020a7c60eec0] nal_unit_type: 1(Coded slice of a
non-IDR picture), nal_ref_idc: 2
Last message repeated 1 times
[h264 # 0000020a7c631340] nal_unit_type: 1(Coded slice of a non-IDR
picture), nal_ref_idc: 2
Last message repeated 1 times
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[h264 # 0000020a7c631340] decode_slice_header error
[h264 # 0000020a7c631340] non-existing PPS 0 referenced
[h264 # 0000020a7c631340] decode_slice_header error
[h264 # 0000020a7c631340] no frame!
[extract_extradata # 0000020a7c60eec0] nal_unit_type: 7(SPS), nal_ref_idc:3
[extract_extradata # 0000020a7c60eec0] nal_unit_type: 8(PPS), nal_ref_idc:3
[extract_extradata # 0000020a7c60eec0] nal_unit_type: 5(IDR), nal_ref_idc:3
Last message repeated 1 times
[h264 # 0000020a7c631340] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 # 0000020a7c631340] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 # 0000020a7c631340] nal_unit_type: 5(IDR), nal_ref_idc: 3
Last message repeated 1 times
[h264 # 0000020a7c631340] Format yuv420p chosen by get_format().
[h264 # 0000020a7c631340] Reinit context to 720x576, pix_fmt: yuv420p
[h264 # 0000020a7c631340] nal_unit_type: 1(Coded slice of a non-IDR
picture), nal_ref_idc: 2
Last message repeated 11 times
[h264 # 0000020a7c5ded80] max_analyze_duration 5000000 reached at 5000000
microseconds st:0
[h264 # 0000020a7c5ded80] After avformat_find_stream_info() pos: 971047
bytes read:971495 seeks:0 frames:128
Input #0, h264, from 'udp://127.0.0.0:46002':
Duration: N/A, bitrate: N/A
Stream #0:0, 128, 1/1200000: Video: h264 (Constrained Baseline), 1
reference frame, yuv420p(progressive, left), 720x576, 0/1, 25 fps, 25 tbr,
1200k tbn, 50 tbc
Successfully opened the file.
Parsing a group of options: output url display.
Applying option f (force format) with argument xv.
Successfully parsed a group of options.
Opening an output file: display.
[NULL # 0000020a7ce73000] Requested output format 'xv' is not a suitable
output format
display: Invalid argument
[AVIOContext # 0000020a7c610300] Statistics: 971495 bytes read, 0 seeks
You would use uridecodebin that can decode various types of urls, containers, protocols and codecs.
With Jetson, the decoder selected by uridecodebin for h264 would be nvv4l2decoder, that doesn't use GPU but better dedicated HW decoder NVDEC.
nvv4l2decoder outputs into NVMM memory in NV12 format, while opencv appsink expects BGR format in system memory. So you would use HW converter nvvidconv for converting and copying into system memory. Unfortunately, nvvidconv doesn't support BGR format, so first convert into supported BGRx format with nvvidconv, and finally use CPU plugin videoconvert for BGRx -> BGR conversion such as:
pipeline='uridecodebin uri=rtsp://127.0.0.1:8554/test ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1'
cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
This is for the general way.
Though, for some streaming protocols it may not be so simple.
For RTP-H264/UDP, ffmpeg backend may only work with a SDP file.
For gstreamer backend you would instead use a pipeline such as:
pipeline='udpsrc port=46002 multicast-group=234.0.0.0 ! application/x-rtp,encoding-name=H264 ! rtpjitterbuffer latency=500 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1'
As you can use FFMPEG, I'd speculate that received stream is using RTP-MP2T. So you would try:
# Using NVDEC, but this may fail depending on sender side's codec:
cap = cv2.VideoCapture('udpsrc multicast-group=234.0.0.0 port=46002 ! application/x-rtp,media=video,encoding-name=MP2T,clock-rate=90000,payload=33 ! rtpjitterbuffer latency=300 ! rtpmp2tdepay ! tsdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1', cv2.CAP_GSTREAMER)
# Or using CPU (may not support high pixel rate with Nano):
cap = cv2.VideoCapture('udpsrc multicast-group=234.0.0.0 port=46002 ! application/x-rtp,media=video,encoding-name=MP2T,clock-rate=90000,payload=33 ! rtpjitterbuffer latency=300 ! rtpmp2tdepay ! tsdemux ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1', cv2.CAP_GSTREAMER)
[Note that I'm not familiar with 234.0.0.0, so unsure if multicast-group should be used as I did].
If this doesn't work, you may try to get more information about received stream. You may try working ffmpeg such as:
ffmpeg -hide_banner -loglevel debug -i udp://234.0.0.0:46002 -f xv display
If you see:
Stream #0:0, 133, 1/1200000: Video: h264 (Constrained Baseline), 1 reference frame, yuv420p(progressive, left), 720x576, 0/1, 25 fps, 25 tbr, 1200k tbn, 50 tbc
you may have to change clock-rate to 1200000 (default value is 90000):
application/x-rtp,media=video,encoding-name=MP2T,clock-rate=1200000
This is assuming the stream is mpeg2 ts. In this case, first lines show:
...
Opening an input file: udp://127.0.0.1:5002.
[NULL # 0x55761c4690] Opening 'udp://127.0.0.1:5002' for reading
[udp # 0x55761a27c0] No default whitelist set
[udp # 0x55761a27c0] end receive buffer size reported is 131072
[mpegts # 0x55761c4690] Format mpegts probed with size=2048 and score=47
[mpegts # 0x55761c4690] stream=0 stream_type=1b pid=41 prog_reg_desc=HDMV
[mpegts # 0x55761c4690] Before avformat_find_stream_info() pos: 0 bytes read:26560 seeks:0 nb_streams:1
...
ffmpeg tries to guess and here found the stream was in mpegts format. You would check in your case what ffmpeg finds. Note that first guess may not be correct, you would have to check the whole log and see what it finds working.
Another speculation would be that your stream is not RTP, but rather raw h264 stream. In such case you may be able to decode with something like:
gst-launch-1.0 udpsrc port=46002 multicast-group=234.0.0.0 ! h264parse ! nvv4l2decoder ! autovideosink
If this works, for opencv you would use:
pipeline='udpsrc port=46002 multicast-group=234.0.0.0 ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1'
I have an audio file, that have some silences, which I am detecting with ffmpeg detectsilence and then trying to remove with removesilence, however there is some strange behavior. Specifically:
1) File's Basic info based on ffprobe show_streams
Input #0, mp3, from 'my_file.mp3':
Metadata:
encoder : Lavf58.64.100
Duration: 00:00:25.22, start: 0.046042, bitrate: 32 kb/s
Stream #0:0: Audio: mp3, 24000 Hz, mono, fltp, 32 kb/s
2) Using detectsilence
ffmpeg -i my_file.mp3 -af silencedetect=noise=-50dB:d=0.2 -f null -
I get this result
[mp3float # 000001ee50074280] overread, skip -7 enddists: -1 -1
[silencedetect # 000001ee5008a1c0] silence_start: 6.21417
[silencedetect # 000001ee5008a1c0] silence_end: 6.91712 | silence_duration: 0.702958
[silencedetect # 000001ee5008a1c0] silence_start: 16.44
[silencedetect # 000001ee5008a1c0] silence_end: 17.1547 | silence_duration: 0.714708
[mp3float # 000001ee50074280] overread, skip -10 enddists: -3 -3
[mp3float # 000001ee50074280] overread, skip -5 enddists: -4 -4
[silencedetect # 000001ee5008a1c0] silence_start: 24.4501
size=N/A time=00:00:25.17 bitrate=N/A speed=1.32e+03x
video:0kB audio:1180kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[silencedetect # 000001ee5008a1c0] silence_end: 25.176 | silence_duration: 0.725917
That also match the values and points based on Adobe Audition
So far all good.
3) Now, based on some calculations (which is based on application's logic on what should be the final duration of the audio) I am trying to delete the silence with "0.725917"s duration. For that, based on ffmpeg docs (https://ffmpeg.org/ffmpeg-filters.html#silencedetect)
Trim all silence encountered from beginning to end where there is more
than 1 second of silence in audio:
silenceremove=stop_periods=-1:stop_duration=1:stop_threshold=-90dB
I run this command
ffmpeg -i my_file.mp3 -af silenceremove=stop_periods=-1:stop_threshold=-50dB:stop_duration=0.72 result1.mp3
So, I am expecting that it should delete only the silence with "0.725917" duration (the last one in the above image), however it is deleting the silence that starts at 16.44s with duration of "0.714708"s. Please see the following comparison:
4) Running detectsilence on result1.mp3 with same options gives even stranger results
ffmpeg -i result1.mp3 -af silencedetect=noise=-50dB:d=0.2 -f null -
result
[mp3float # 0000017723404280] overread, skip -5 enddists: -4 -4
[silencedetect # 0000017723419540] silence_start: 6.21417
[silencedetect # 0000017723419540] silence_end: 6.92462 | silence_duration: 0.710458
[mp3float # 0000017723404280] overread, skip -7 enddists: -6 -6
[mp3float # 0000017723404280] overread, skip -7 enddists: -2 -2
[mp3float # 0000017723404280] overread, skip -6 enddists: -1 -1
Last message repeated 1 times
[silencedetect # 0000017723419540] silence_start: 23.7308
size=N/A time=00:00:24.45 bitrate=N/A speed=1.33e+03x
video:0kB audio:1146kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[silencedetect # 0000017723419540] silence_end: 24.456 | silence_duration: 0.725167
So, the results are:
With command to remove silences that are longer than "0.72 second", a silence that was "0.714708"s, got removed and - a silence with "0.725917"s remained as is (well, actually changed a little - as per 3rd point)
The first silence that had started at "6.21417" and had a duration of "0.702958"s, suddenly now has a duration of "0.710458"s
The 3rd silence that had started at "24.4501" (which now starts at 23.7308 - obviously because the 2nd silence was removed) and had a duration of "0.725917", now suddenly is "0.725167"s (this one is not a big difference, but still why even removing other silence, this silence's duration should change at all).
Accordingly the expected results are:
Only the silences that match the provided condition (stop_duration=0.72) should be removed. In this specific example only the last one, but in general any silence that matches the condition of the length - irrelevant of their positioning (start, end or in the middle)
Other silences should remain with same exact duration they were before
FFMpeg: 4.2.4-1ubuntu0.1, Ubuntu: 20.04.2
Some attempts and results, while playing with ffmpeg options
a)
ffmpeg -i my_file.mp3 -af silenceremove=stop_periods=-1:stop_threshold=-50dB:stop_duration=0.72:detection=peak tmp1.mp3
result:
First and second silences are removed, 3rd silence's duration remains exactly the same
b)
ffmpeg -i my_file.mp3 -af silenceremove=stop_periods=-1:stop_threshold=-50dB:stop_duration=0.71 tmp_0.71.mp3
result:
First and second silences are removed, 3rd silence remains, but the duration becomes "0.72075"s
c)
ffmpeg -i my_file.mp3 -af silenceremove=stop_periods=-1:stop_threshold=-50dB:stop_duration=0.7 tmp_0.7.mp3
result:
all 3 silence are removed
d) the edge case
this command still removes the second silence (after which the first silence become exactly as in point #4 and last silence becomes "0.721375")
ffmpeg -i my_file.mp3 -af silenceremove=stop_periods=-1:stop_threshold=-50dB:stop_duration=0.72335499999 tmp_0.72335499999.mp3
but this one, again does not remove any silence:
ffmpeg -i my_file.mp3 -af silenceremove=stop_periods=-1:stop_threshold=-50dB:stop_duration=0.723355 tmp_0.723355.mp3
e) window param case 0.03
ffmpeg -i my_file.mp3 -af silenceremove=stop_periods=-1:stop_threshold=-50dB:stop_duration=0.72:window=0.03 window_0.03.mp3
does not remove any silence, but the detect silence
ffmpeg -i window_0.03.mp3 -af silencedetect=noise=-50dB:d=0.2 -f null -
gives this result (compare with silences in result1.mp3 - from point #4 )
[mp3float # 000001c5c8824280] overread, skip -5 enddists: -4 -4
[silencedetect # 000001c5c883a040] silence_start: 6.21417
[silencedetect # 000001c5c883a040] silence_end: 6.92462 | silence_duration: 0.710458
[mp3float # 000001c5c8824280] overread, skip -7 enddists: -6 -6
[mp3float # 000001c5c8824280] overread, skip -7 enddists: -2 -2
[silencedetect # 000001c5c883a040] silence_start: 16.4424
[silencedetect # 000001c5c883a040] silence_end: 17.1555 | silence_duration: 0.713167
[mp3float # 000001c5c8824280] overread, skip -6 enddists: -1 -1
Last message repeated 1 times
[silencedetect # 000001c5c883a040] silence_start: 24.4508
size=N/A time=00:00:25.17 bitrate=N/A speed=1.24e+03x
video:0kB audio:1180kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[silencedetect # 000001c5c883a040] silence_end: 25.176 | silence_duration: 0.725167
f) window case 0.01
ffmpeg -i my_file.mp3 -af silenceremove=stop_periods=-1:stop_threshold=-50dB:stop_duration=0.72:window=0.01 window_0.01.mp3
removes first and second silences, the detect silence with same params has the following result
[mp3float # 000001ea631d4280] overread, skip -5 enddists: -4 -4
Last message repeated 1 times
[mp3float # 000001ea631d4280] overread, skip -7 enddists: -2 -2
[mp3float # 000001ea631d4280] overread, skip -6 enddists: -1 -1
Last message repeated 1 times
[silencedetect # 000001ea631ea1c0] silence_start: 23.0108
size=N/A time=00:00:23.73 bitrate=N/A speed=1.2e+03x
video:0kB audio:1113kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[silencedetect # 000001ea631ea1c0] silence_end: 23.736 | silence_duration: 0.725167
Any thoughts, ideas, points are much appreciated.
You're suffering from two things:
You are converting back to an mp3 (a lossy format), which is causing result1.mp3 to be reencoded and become slightly different than a perfect cut. The fix for this is to use .wav's (a lossless format).
The silenceremove function is using a window and you need to set it to 0 to do sample-by-sample.
ffmpeg -i my_file.mp3 my_file.wav
ffmpeg -i my_file.wav -af silencedetect=noise=-50dB:d=0.2 -f null -
ffmpeg -i my_file.wav -af silenceremove=stop_periods=-1:stop_threshold=-50dB:stop_duration=0.72:window=0 result1.wav
ffmpeg -i result1.wav -af silencedetect=noise=-50dB:d=0.2 -f null -
Final output of the last line. I would consider this a solid solution, because the silence starts and durations match up perfectly with their values before the cut:
[silencedetect # 0x5570a855b400] silence_start: 6.21417
[silencedetect # 0x5570a855b400] silence_end: 6.91712 | silence_duration: 0.702958
[silencedetect # 0x5570a855b400] silence_start: 16.44
[silencedetect # 0x5570a855b400] silence_end: 17.1547 | silence_duration: 0.714708
size=N/A time=00:00:24.45 bitrate=N/A speed=4.49e+03x
You can then reencode it to .mp3 if you want.
File1.log
207.46.13.90 37556
157.55.39.51 34268
40.77.167.109 21824
157.55.39.253 19683
File2.log
207.46.13.90 62343
157.55.39.51 58451
157.55.39.200 37675
40.77.167.109 21824
Below should be expected
Output.log
207.46.13.90 37556 62343
157.55.39.51 34268 58451
157.55.39.200 ----- 37675
40.77.167.109 21824 21824
157.55.39.253 19683 -----
I tried with the below 'join' command - but it skips the missing line
join --nocheck-order File1.log File2.log
outputting like below (not as expected)
207.46.13.90 37556 62343
157.55.39.51 34268 58451
40.77.167.109 21824 21824
Could someone please help with the proper command for the desired output. Thanks in advance
Could you please try following.
awk '
FNR==NR{
a[$1]=$2
next
}
($1 in a){
print $0,a[$1]
b[$1]
next
}
{
print $1,$2 " ----- "
}
END{
for(i in a){
if(!(i in b)){
print i" ----- "a[i]
}
}
}
' Input_file2 Input_file1
Output will be as follows.
207.46.13.90 37556 62343
157.55.39.51 34268 58451
40.77.167.109 21824 21824
157.55.39.253 19683 -----
157.55.39.200 ----- 37675
The following is just enough if you don't care about sorting order of the output:
join -a1 -a2 -e----- -oauto <(sort file1.log) <(sort file2.log) |
column -t -s' ' -o' '
with recreation of the input files:
cat <<EOF >file1.log
207.46.13.90 37556
157.55.39.51 34268
40.77.167.109 21824
157.55.39.253 19683
EOF
cat <<EOF >file2.log
207.46.13.90 62343
157.55.39.51 58451
157.55.39.200 37675
40.77.167.109 21824
EOF
outputs:
157.55.39.200 ----- 37675
157.55.39.253 19683 -----
157.55.39.51 34268 58451
207.46.13.90 37556 62343
40.77.167.109 21824 21824
join by default joins by the first columns. The -a1 -a2 make it print the unmatched lines from both inputs. The -e----- prints unknown columns as dots. The -oauto determinates the output from the columns of the inputs. Because we want to sort on the first column, we don't need to specif -k1 to sort, but sort -s -k1 could speed things up. To match the expected output, I also piped to column.
You can sort the output by ports by pipeing it to for example to sort -rnk2,3.
I have been using Snort for my school project.
My problem is that the log files are in binary format and I am not able to read them using less/cat/vi. How do I do this?
I have specified in my snort.conf file unified2 format.
Here is my snort.conf file
--------------------------------------------------
# VRT Rule Packages Snort.conf
#
# For more information visit us at:
# http://www.snort.org Snort Website
# http://vrt-blog.snort.org/ Sourcefire VRT Blog
#
# Mailing list Contact: snort-sigs#lists.sourceforge.net
# False Positive reports: fp#sourcefire.com
# Snort bugs: bugs#snort.org
#
# Compatible with Snort Versions:
# VERSIONS : 2.9.6.2
#
# Snort build options:
# OPTIONS : --enable-gre --enable-mpls --enable-targetbased --enable-ppm --enable-perfprofiling --enable-zlib --enable-active-response --enable-normalizer --enable-reload --enable-react --enable-flexresp3
#
# Additional information:
# This configuration file enables active response, to run snort in
# test mode -T you are required to supply an interface -i <interface>
# or test mode will fail to fully validate the configuration and
# exit with a FATAL error
#--------------------------------------------------
###################################################
# This file contains a sample snort configuration.
# You should take the following steps to create your own custom configuration:
#
# 1) Set the network variables.
# 2) Configure the decoder
# 3) Configure the base detection engine
# 4) Configure dynamic loaded libraries
# 5) Configure preprocessors
# 6) Configure output plugins
# 7) Customize your rule set
# 8) Customize preprocessor and decoder rule set
# 9) Customize shared object rule set
###################################################
###################################################
# Step #1: Set the network variables. For more information, see README.variables
###################################################
# Setup the network addresses you are protecting
ipvar HOME_NET any
# Set up the external network addresses. Leave as "any" in most situations
ipvar EXTERNAL_NET any
# List of DNS servers on your network
ipvar DNS_SERVERS $HOME_NET
# List of SMTP servers on your network
ipvar SMTP_SERVERS $HOME_NET
# List of web servers on your network
ipvar HTTP_SERVERS $HOME_NET
# List of sql servers on your network
ipvar SQL_SERVERS $HOME_NET
# List of telnet servers on your network
ipvar TELNET_SERVERS $HOME_NET
# List of ssh servers on your network
ipvar SSH_SERVERS $HOME_NET
# List of ftp servers on your network
ipvar FTP_SERVERS $HOME_NET
# List of sip servers on your network
ipvar SIP_SERVERS $HOME_NET
# List of ports you run web servers on
portvar HTTP_PORTS [36,80,81,82,83,84,85,86,87,88,89,90,311,383,555,591,593,631,801,808,818,901,972,1158,1220,1414,1533,1741,1830,1942,2231,2301,2381,2809,2980,3029,3037,3057,3128,3443,3702,4000,4343,4848,5000,5117,5250,5600,6080,6173,6988,7000,7001,7071,7144,7145,7510,7770,7777,7778,7779,8000,8008,8014,8028,8080,8081,8082,8085,8088,8090,8118,8123,8180,8181,8222,8243,8280,8300,8333,8344,8500,8509,8800,8888,8899,8983,9000,9060,9080,9090,9091,9111,9290,9443,9999,10000,11371,12601,13014,15489,29991,33300,34412,34443,34444,41080,44449,50000,50002,51423,53331,55252,55555,56712]
# List of ports you want to look for SHELLCODE on.
portvar SHELLCODE_PORTS !80
# List of ports you might see oracle attacks on
portvar ORACLE_PORTS 1024:
# List of ports you want to look for SSH connections on:
portvar SSH_PORTS 22
# List of ports you run ftp servers on
portvar FTP_PORTS [21,2100,3535]
# List of ports you run SIP servers on
portvar SIP_PORTS [5060,5061,5600]
# List of file data ports for file inspection
portvar FILE_DATA_PORTS [$HTTP_PORTS,110,143]
# List of GTP ports for GTP preprocessor
portvar GTP_PORTS [2123,2152,3386]
# other variables, these should not be modified
ipvar AIM_SERVERS [64.12.24.0/23,64.12.28.0/23,64.12.161.0/24,64.12.163.0/24,64.12.200.0/24,205.188.3.0/24,205.188.5.0/24,205.188.7.0/24,205.188.9.0/24,205.188.153.0/24,205.188.179.0/24,205.188.248.0/24]
# Path to your rules files (this can be a relative path)
# Note for Windows users: You are advised to make this an absolute path,
# such as: c:\snort\rules
var RULE_PATH /etc/snort/rules/rules
var SO_RULE_PATH /etc/snort/rules/so_rules
var PREPROC_RULE_PATH /etc/snort/rules/preproc_rules
# If you are using reputation preprocessor set these
var WHITE_LIST_PATH /etc/snort/rules/rules
var BLACK_LIST_PATH /etc/snort/rules/rules
###################################################
# Step #2: Configure the decoder. For more information, see README.decode
###################################################
# Stop generic decode events:
config disable_decode_alerts
# Stop Alerts on experimental TCP options
config disable_tcpopt_experimental_alerts
# Stop Alerts on obsolete TCP options
config disable_tcpopt_obsolete_alerts
# Stop Alerts on T/TCP alerts
config disable_tcpopt_ttcp_alerts
# Stop Alerts on all other TCPOption type events:
config disable_tcpopt_alerts
# Stop Alerts on invalid ip options
config disable_ipopt_alerts
# Alert if value in length field (IP, TCP, UDP) is greater th elength of the packet
# config enable_decode_oversized_alerts
# Same as above, but drop packet if in Inline mode (requires enable_decode_oversized_alerts)
# config enable_decode_oversized_drops
# Configure IP / TCP checksum mode
config checksum_mode: all
# Configure maximum number of flowbit references. For more information, see README.flowbits
# config flowbits_size: 64
# Configure ports to ignore
# config ignore_ports: tcp 21 6667:6671 1356
# config ignore_ports: udp 1:17 53
# Configure active response for non inline operation. For more information, see REAMDE.active
# config response: eth0 attempts 2
# Configure DAQ related options for inline operation. For more information, see README.daq
#
# config daq: <type>
# config daq_dir: <dir>
# config daq_mode: <mode>
# config daq_var: <var>
#
# <type> ::= pcap | afpacket | dump | nfq | ipq | ipfw
# <mode> ::= read-file | passive | inline
# <var> ::= arbitrary <name>=<value passed to DAQ
# <dir> ::= path as to where to look for DAQ module so's
# Configure specific UID and GID to run snort as after dropping privs. For more information see snort -h command line options
#
# config set_gid:
# config set_uid:
# Configure default snaplen. Snort defaults to MTU of in use interface. For more information see README
#
# config snaplen:
#
# Configure default bpf_file to use for filtering what traffic reaches snort. For more information see snort -h command line options (-F)
#
# config bpf_file:
#
# Configure default log directory for snort to log to. For more information see snort -h command line options (-l)
#
# config logdir:
config logdir:/var/log/snort/
###################################################
# Step #3: Configure the base detection engine. For more information, see README.decode
###################################################
# Configure PCRE match limitations
config pcre_match_limit: 3500
config pcre_match_limit_recursion: 1500
# Configure the detection engine See the Snort Manual, Configuring Snort - Includes - Config
config detection: search-method ac-split search-optimize max-pattern-len 20
# Configure the event queue. For more information, see README.event_queue
config event_queue: max_queue 8 log 5 order_events content_length
###################################################
## Configure GTP if it is to be used.
## For more information, see README.GTP
####################################################
# config enable_gtp
###################################################
# Per packet and rule latency enforcement
# For more information see README.ppm
###################################################
# Per Packet latency configuration
#config ppm: max-pkt-time 250, \
# fastpath-expensive-packets, \
# pkt-log
# Per Rule latency configuration
#config ppm: max-rule-time 200, \
# threshold 3, \
# suspend-expensive-rules, \
# suspend-timeout 20, \
# rule-log alert
###################################################
# Configure Perf Profiling for debugging
# For more information see README.PerfProfiling
###################################################
#config profile_rules: print all, sort avg_ticks
#config profile_preprocs: print all, sort avg_ticks
###################################################
# Configure protocol aware flushing
# For more information see README.stream5
###################################################
config paf_max: 16000
###################################################
# Step #4: Configure dynamic loaded libraries.
# For more information, see Snort Manual, Configuring Snort - Dynamic Modules
###################################################
# path to dynamic preprocessor libraries
dynamicpreprocessor directory /usr/lib64/snort-2.9.6.2_dynamicpreprocessor/
# path to base preprocessor engine
dynamicengine /usr/lib64/snort-2.9.6.2_dynamicengine/libsf_engine.so
# path to dynamic rules libraries
dynamicdetection directory /usr/local/lib/snort_dynamicrules
###################################################
# Step #5: Configure preprocessors
# For more information, see the Snort Manual, Configuring Snort - Preprocessors
###################################################
# GTP Control Channle Preprocessor. For more information, see README.GTP
# preprocessor gtp: ports { 2123 3386 2152 }
# Inline packet normalization. For more information, see README.normalize
# Does nothing in IDS mode
preprocessor normalize_ip4
preprocessor normalize_tcp: ips ecn stream
preprocessor normalize_icmp4
preprocessor normalize_ip6
preprocessor normalize_icmp6
# Target-based IP defragmentation. For more inforation, see README.frag3
preprocessor frag3_global: max_frags 65536
preprocessor frag3_engine: policy windows detect_anomalies overlap_limit 10 min_fragment_length 100 timeout 180
# Target-Based stateful inspection/stream reassembly. For more inforation, see README.stream5
preprocessor stream5_global: track_tcp yes, \
track_udp yes, \
track_icmp no, \
max_tcp 262144, \
max_udp 131072, \
max_active_responses 2, \
min_response_seconds 5
preprocessor stream5_tcp: policy windows, detect_anomalies, require_3whs 180, \
overlap_limit 10, small_segments 3 bytes 150, timeout 180, \
ports client 21 22 23 25 42 53 70 79 109 110 111 113 119 135 136 137 139 143 \
161 445 513 514 587 593 691 1433 1521 1741 2100 3306 6070 6665 6666 6667 6668 6669 \
7000 8181 32770 32771 32772 32773 32774 32775 32776 32777 32778 32779, \
ports both 36 80 81 82 83 84 85 86 87 88 89 90 110 311 383 443 465 563 555 591 593 631 636 801 808 818 901 972 989 992 993 994 995 1158 1220 1414 1533 1741 1830 1942 2231 2301 2381 2809 2980 3029 3037 3057 3128 3443 3702 4000 4343 4848 5000 5117 5250 5600 6080 6173 6988 7907 7000 7001 7071 7144 7145 7510 7802 7770 7777 7778 7779 \
7801 7900 7901 7902 7903 7904 7905 7906 7908 7909 7910 7911 7912 7913 7914 7915 7916 \
7917 7918 7919 7920 8000 8008 8014 8028 8080 8081 8082 8085 8088 8090 8118 8123 8180 8181 8222 8243 8280 8300 8333 8344 8500 8509 8800 8888 8899 8983 9000 9060 9080 9090 9091 9111 9290 9443 9999 10000 11371 12601 13014 15489 29991 33300 34412 34443 34444 41080 44449 50000 50002 51423 53331 55252 55555 56712
preprocessor stream5_udp: timeout 180
# performance statistics. For more information, see the Snort Manual, Configuring Snort - Preprocessors - Performance Monitor
# preprocessor perfmonitor: time 300 file /var/snort/snort.stats pktcnt 10000
# HTTP normalization and anomaly detection. For more information, see README.http_inspect
preprocessor http_inspect: global iis_unicode_map unicode.map 1252 compress_depth 65535 decompress_depth 65535
preprocessor http_inspect_server: server default \
http_methods { GET POST PUT SEARCH MKCOL COPY MOVE LOCK UNLOCK NOTIFY POLL BCOPY BDELETE BMOVE LINK UNLINK OPTIONS HEAD DELETE TRACE TRACK CONNECT SOURCE SUBSCRIBE UNSUBSCRIBE PROPFIND PROPPATCH BPROPFIND BPROPPATCH RPC_CONNECT PROXY_SUCCESS BITS_POST CCM_POST SMS_POST RPC_IN_DATA RPC_OUT_DATA RPC_ECHO_DATA } \
chunk_length 500000 \
server_flow_depth 0 \
client_flow_depth 0 \
post_depth 65495 \
oversize_dir_length 500 \
max_header_length 750 \
max_headers 100 \
max_spaces 200 \
small_chunk_length { 10 5 } \
ports { 36 80 81 82 83 84 85 86 87 88 89 90 311 383 555 591 593 631 801 808 818 901 972 1158 1220 1414 1533 1741 1830 1942 2231 2301 2381 2809 2980 3029 3037 3057 3128 3443 3702 4000 4343 4848 5000 5117 5250 5600 6080 6173 6988 7000 7001 7071 7144 7145 7510 7770 7777 7778 7779 8000 8008 8014 8028 8080 8081 8082 8085 8088 8090 8118 8123 8180 8181 8222 8243 8280 8300 8333 8344 8500 8509 8800 8888 8899 8983 9000 9060 9080 9090 9091 9111 9290 9443 9999 10000 11371 12601 13014 15489 29991 33300 34412 34443 34444 41080 44449 50000 50002 51423 53331 55252 55555 56712 } \
non_rfc_char { 0x00 0x01 0x02 0x03 0x04 0x05 0x06 0x07 } \
enable_cookie \
extended_response_inspection \
inspect_gzip \
normalize_utf \
unlimited_decompress \
normalize_javascript \
apache_whitespace no \
ascii no \
bare_byte no \
directory no \
double_decode no \
iis_backslash no \
iis_delimiter no \
iis_unicode no \
multi_slash no \
utf_8 no \
u_encode yes \
webroot no
# ONC-RPC normalization and anomaly detection. For more information, see the Snort Manual, Configuring Snort - Preprocessors - RPC Decode
preprocessor rpc_decode: 111 32770 32771 32772 32773 32774 32775 32776 32777 32778 32779 no_alert_multiple_requests no_alert_large_fragments no_alert_incomplete
# Back Orifice detection.
preprocessor bo
# FTP / Telnet normalization and anomaly detection. For more information, see README.ftptelnet
preprocessor ftp_telnet: global inspection_type stateful encrypted_traffic no check_encrypted
preprocessor ftp_telnet_protocol: telnet \
ayt_attack_thresh 20 \
normalize ports { 23 } \
detect_anomalies
preprocessor ftp_telnet_protocol: ftp server default \
def_max_param_len 100 \
ports { 21 2100 3535 } \
telnet_cmds yes \
ignore_telnet_erase_cmds yes \
ftp_cmds { ABOR ACCT ADAT ALLO APPE AUTH CCC CDUP } \
ftp_cmds { CEL CLNT CMD CONF CWD DELE ENC EPRT } \
ftp_cmds { EPSV ESTA ESTP FEAT HELP LANG LIST LPRT } \
ftp_cmds { LPSV MACB MAIL MDTM MIC MKD MLSD MLST } \
ftp_cmds { MODE NLST NOOP OPTS PASS PASV PBSZ PORT } \
ftp_cmds { PROT PWD QUIT REIN REST RETR RMD RNFR } \
ftp_cmds { RNTO SDUP SITE SIZE SMNT STAT STOR STOU } \
ftp_cmds { STRU SYST TEST TYPE USER XCUP XCRC XCWD } \
ftp_cmds { XMAS XMD5 XMKD XPWD XRCP XRMD XRSQ XSEM } \
ftp_cmds { XSEN XSHA1 XSHA256 } \
alt_max_param_len 0 { ABOR CCC CDUP ESTA FEAT LPSV NOOP PASV PWD QUIT REIN STOU SYST XCUP XPWD } \
alt_max_param_len 200 { ALLO APPE CMD HELP NLST RETR RNFR STOR STOU XMKD } \
alt_max_param_len 256 { CWD RNTO } \
alt_max_param_len 400 { PORT } \
alt_max_param_len 512 { SIZE } \
chk_str_fmt { ACCT ADAT ALLO APPE AUTH CEL CLNT CMD } \
chk_str_fmt { CONF CWD DELE ENC EPRT EPSV ESTP HELP } \
chk_str_fmt { LANG LIST LPRT MACB MAIL MDTM MIC MKD } \
chk_str_fmt { MLSD MLST MODE NLST OPTS PASS PBSZ PORT } \
chk_str_fmt { PROT REST RETR RMD RNFR RNTO SDUP SITE } \
chk_str_fmt { SIZE SMNT STAT STOR STRU TEST TYPE USER } \
chk_str_fmt { XCRC XCWD XMAS XMD5 XMKD XRCP XRMD XRSQ } \
chk_str_fmt { XSEM XSEN XSHA1 XSHA256 } \
cmd_validity ALLO < int [ char R int ] > \
cmd_validity EPSV < [ { char 12 | char A char L char L } ] > \
cmd_validity MACB < string > \
cmd_validity MDTM < [ date nnnnnnnnnnnnnn[.n[n[n]]] ] string > \
cmd_validity MODE < char ASBCZ > \
cmd_validity PORT < host_port > \
cmd_validity PROT < char CSEP > \
cmd_validity STRU < char FRPO [ string ] > \
cmd_validity TYPE < { char AE [ char NTC ] | char I | char L [ number ] } >
preprocessor ftp_telnet_protocol: ftp client default \
max_resp_len 256 \
bounce yes \
ignore_telnet_erase_cmds yes \
telnet_cmds yes
# SMTP normalization and anomaly detection. For more information, see README.SMTP
preprocessor smtp: ports { 25 465 587 691 } \
inspection_type stateful \
b64_decode_depth 0 \
qp_decode_depth 0 \
bitenc_decode_depth 0 \
uu_decode_depth 0 \
log_mailfrom \
log_rcptto \
log_filename \
log_email_hdrs \
normalize cmds \
normalize_cmds { ATRN AUTH BDAT CHUNKING DATA DEBUG EHLO EMAL ESAM ESND ESOM ETRN EVFY } \
normalize_cmds { EXPN HELO HELP IDENT MAIL NOOP ONEX QUEU QUIT RCPT RSET SAML SEND SOML } \
normalize_cmds { STARTTLS TICK TIME TURN TURNME VERB VRFY X-ADAT X-DRCP X-ERCP X-EXCH50 } \
normalize_cmds { X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \
max_command_line_len 512 \
max_header_line_len 1000 \
max_response_line_len 512 \
alt_max_command_line_len 260 { MAIL } \
alt_max_command_line_len 300 { RCPT } \
alt_max_command_line_len 500 { HELP HELO ETRN EHLO } \
alt_max_command_line_len 255 { EXPN VRFY ATRN SIZE BDAT DEBUG EMAL ESAM ESND ESOM EVFY IDENT NOOP RSET } \
alt_max_command_line_len 246 { SEND SAML SOML AUTH TURN ETRN DATA RSET QUIT ONEX QUEU STARTTLS TICK TIME TURNME VERB X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \
valid_cmds { ATRN AUTH BDAT CHUNKING DATA DEBUG EHLO EMAL ESAM ESND ESOM ETRN EVFY } \
valid_cmds { EXPN HELO HELP IDENT MAIL NOOP ONEX QUEU QUIT RCPT RSET SAML SEND SOML } \
valid_cmds { STARTTLS TICK TIME TURN TURNME VERB VRFY X-ADAT X-DRCP X-ERCP X-EXCH50 } \
valid_cmds { X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \
xlink2state { enabled }
# Portscan detection. For more information, see README.sfportscan
# preprocessor sfportscan: proto { all } memcap { 10000000 } sense_level { low }
# ARP spoof detection. For more information, see the Snort Manual - Configuring Snort - Preprocessors - ARP Spoof Preprocessor
# preprocessor arpspoof
# preprocessor arpspoof_detect_host: 192.168.40.1 f0:0f:00:f0:0f:00
# SSH anomaly detection. For more information, see README.ssh
preprocessor ssh: server_ports { 22 } \
autodetect \
max_client_bytes 19600 \
max_encrypted_packets 20 \
max_server_version_len 100 \
enable_respoverflow enable_ssh1crc32 \
enable_srvoverflow enable_protomismatch
# SMB / DCE-RPC normalization and anomaly detection. For more information, see README.dcerpc2
preprocessor dcerpc2: memcap 102400, events [co ]
preprocessor dcerpc2_server: default, policy WinXP, \
detect [smb [139,445], tcp 135, udp 135, rpc-over-http-server 593], \
autodetect [tcp 1025:, udp 1025:, rpc-over-http-server 1025:], \
smb_max_chain 3, smb_invalid_shares ["C$", "D$", "ADMIN$"]
# DNS anomaly detection. For more information, see README.dns
preprocessor dns: ports { 53 } enable_rdata_overflow
# SSL anomaly detection and traffic bypass. For more information, see README.ssl
preprocessor ssl: ports { 443 465 563 636 989 992 993 994 995 7801 7802 7900 7901 7902 7903 7904 7905 7906 7907 7908 7909 7910 7911 7912 7913 7914 7915 7916 7917 7918 7919 7920 }, trustservers, noinspect_encrypted
# SDF sensitive data preprocessor. For more information see README.sensitive_data
preprocessor sensitive_data: alert_threshold 25
# SIP Session Initiation Protocol preprocessor. For more information see README.sip
preprocessor sip: max_sessions 40000, \
ports { 5060 5061 5600 }, \
methods { invite \
cancel \
ack \
bye \
register \
options \
refer \
subscribe \
update \
join \
info \
message \
notify \
benotify \
do \
qauth \
sprack \
publish \
service \
unsubscribe \
prack }, \
max_uri_len 512, \
max_call_id_len 80, \
max_requestName_len 20, \
max_from_len 256, \
max_to_len 256, \
max_via_len 1024, \
max_contact_len 512, \
max_content_len 2048
# IMAP preprocessor. For more information see README.imap
preprocessor imap: \
ports { 143 } \
b64_decode_depth 0 \
qp_decode_depth 0 \
bitenc_decode_depth 0 \
uu_decode_depth 0
# POP preprocessor. For more information see README.pop
preprocessor pop: \
ports { 110 } \
b64_decode_depth 0 \
qp_decode_depth 0 \
bitenc_decode_depth 0 \
uu_decode_depth 0
# Modbus preprocessor. For more information see README.modbus
preprocessor modbus: ports { 502 }
# DNP3 preprocessor. For more information see README.dnp3
preprocessor dnp3: ports { 20000 } \
memcap 262144 \
check_crc
# Reputation preprocessor. For more information see README.reputation
preprocessor reputation: \
memcap 500, \
priority whitelist, \
nested_ip inner, \
whitelist $WHITE_LIST_PATH/white_list.rules, \
blacklist $BLACK_LIST_PATH/black_list.rules
###################################################
# Step #6: Configure output plugins
# For more information, see Snort Manual, Configuring Snort - Output Modules
###################################################
# unified2
# Recommended for most installs
# output unified2: filename merged.log, limit 128, nostamp, mpls_event_types, vlan_event_types
output unified2: filename snort.log, limit 128
# Additional configuration for specific types of installs
# output alert_unified2: filename snort.alert, limit 128, nostamp
# output log_unified2: filename snort.log, limit 128, nostamp
# output unified2: filename snort.log, limit 128
# syslog
# output alert_syslog: LOG_AUTH LOG_ALERT
# pcap
# output log_tcpdump: tcpdump.log
# metadata reference data. do not modify these lines
include classification.config
include reference.config
###################################################
# Step #7: Customize your rule set
# For more information, see Snort Manual, Writing Snort Rules
#
# NOTE: All categories are enabled in this conf file
###################################################
# site specific rules
# include $RULE_PATH/local.rules
include $RULE_PATH/app-detect.rules
include $RULE_PATH/attack-responses.rules
include $RULE_PATH/backdoor.rules
include $RULE_PATH/bad-traffic.rules
include $RULE_PATH/blacklist.rules
include $RULE_PATH/botnet-cnc.rules
include $RULE_PATH/browser-chrome.rules
include $RULE_PATH/browser-firefox.rules
include $RULE_PATH/browser-ie.rules
include $RULE_PATH/browser-other.rules
include $RULE_PATH/protocol-telnet.rules
include $RULE_PATH/protocol-tftp.rules
include $RULE_PATH/protocol-voip.rules
include $RULE_PATH/pua-adware.rules
include $RULE_PATH/pua-other.rules
include $RULE_PATH/pua-p2p.rules
include $RULE_PATH/pua-toolbars.rules
include $RULE_PATH/web-coldfusion.rules
include $RULE_PATH/web-frontpage.rules
include $RULE_PATH/web-iis.rules
include $RULE_PATH/web-misc.rules
include $RULE_PATH/web-php.rules
include $RULE_PATH/x11.rules
# test rule for ping
include $RULE_PATH/testping.rules
###################################################
# Step #8: Customize your preprocessor and decoder alerts
# For more information, see README.decoder_preproc_rules
###################################################
# decoder and preprocessor event rules
# include $PREPROC_RULE_PATH/preprocessor.rules
# include $PREPROC_RULE_PATH/decoder.rules
# include $PREPROC_RULE_PATH/sensitive-data.rules
###################################################
# Step #9: Customize your Shared Object Snort Rules
# For more information, see http://vrt-blog.snort.org/2009/01/using-vrt-certified-shared-object-rules.html
###################################################
# dynamic library rules
# include $SO_RULE_PATH/bad-traffic.rules
# include $SO_RULE_PATH/browser-ie.rules
# include $SO_RULE_PATH/chat.rules
# include $SO_RULE_PATH/dos.rules
# include $SO_RULE_PATH/exploit.rules
# include $SO_RULE_PATH/file-flash.rules
# include $SO_RULE_PATH/icmp.rules
# include $SO_RULE_PATH/imap.rules
# include $SO_RULE_PATH/misc.rules
# include $SO_RULE_PATH/multimedia.rules
# include $SO_RULE_PATH/netbios.rules
# include $SO_RULE_PATH/nntp.rules
# include $SO_RULE_PATH/p2p.rules
# include $SO_RULE_PATH/smtp.rules
# Event thresholding or suppression commands. See threshold.conf
include threshold.conf
Apparently, I have not heard about u2spewfoo, which is a dumping tool used for dumping the content of unified2 log files to stdout.
I ran command u2spewfoo snort.log and was able to read the log.
I hope this helps!