Python socket server: can't compare input from client - python-3.x

I'm practicing a little bit with constructing a socket server on python. I'm currently running it locally on my linux mint system, and testing it with telnet localhost 20000.
The idea is quite simple. For now, I want the client(myself) to send a "hello" message to the server on which the server responses back with another "hello" message. Then the client can send any message on which the server does not respond, until the client says "bye". Then the server responds with another "bye", and the socket is closed.
I implemented this in the following way (in python 3):
1 from socket import *
2 serverPort = 20000
3 serverSocket = socket(AF_INET, SOCK_STREAM)
4 serverSocket.bind(('', serverPort))
5 serverSocket.listen(1)
6
7 print('The echo server is ready to receive')
8 while 1:
9 connectionSocket, addr = serverSocket.accept()
10 print('Processing client ', addr)
11 sentence = ""
12
13 try:
14 sentence = connectionSocket.recv(1024)
15 while sentence != "hello":
16 sentence = connectionSocket.recv(1024)
17 connectionSocket.send("hello")
18
19 sentence = connectionSocket.recv(1024)
20 while sentence != "bye":
21 sentence = connectionSocket.recv(1024)
22 connectionSocket.send("bye")
23 except error:
24 pass
25
26 print('Client closed ', addr)
27 connectionSocket.close()
It's a very simple program, in which I can't find any bug, so I was very surprised to see it not work. When I type in "hello" after running the server with python3 server.py and establishing a connection trough telnet localhost 20000 I simply get no response. I first thought that the problem lies in the equality test in line number 15, so I tested a few things with prints, and instead of "hello" The message received by the server was: b'hello\r\n. I understand the \n, because I type Enter after the "hello" message, but it's not clear to me why the b' and the \r appear.
Do you know what the problem is here and how I can fix it? On other examples on stack overflow the string comparision just works fine, so I can't figure out why it is not working for me.
Thanks in advance!
Update:
I use nc now instead of telnet, and changed every occurence of connectionSocket.recv(1024) into str(connectionSocket.recv(1024), 'utf-8'). The server gets the correct string now, but it still does not echo "hello" back to me, nor does it close the socket when I type "bye".

Use netcat instead of telnet to avoid the telnet protocol bytes.
Like this: nc localhost 20000 and type your input then.
Also, socket I/O is always bytes in Python 3, for instance see https://docs.python.org/3/library/socket.html#socket.socket.recv So you have to compare with b'hello' (or decode the bytes to a string first)

Related

SED style Multi address in Python?

I have an app that parses multiple Cisco show tech files. These files contain the output of multiple router commands in a structured way, let me show you an snippet of a show tech output:
`show clock`
20:20:50.771 UTC Wed Sep 07 2022
Time source is NTP
`show callhome`
callhome disabled
Callhome Information:
<SNIPET>
`show module`
Mod Ports Module-Type Model Status
--- ----- ------------------------------------- --------------------- ---------
1 52 16x10G + 32x10/25G + 4x100G Module N9K-X96136YC-R ok
2 52 16x10G + 32x10/25G + 4x100G Module N9K-X96136YC-R ok
3 52 16x10G + 32x10/25G + 4x100G Module N9K-X96136YC-R ok
4 52 16x10G + 32x10/25G + 4x100G Module N9K-X96136YC-R ok
21 0 Fabric Module N9K-C9504-FM-R ok
22 0 Fabric Module N9K-C9504-FM-R ok
23 0 Fabric Module N9K-C9504-FM-R ok
<SNIPET>
My app currently uses both SED and Python scripts to parse these files. I use SED to parse the show tech file looking for a specific command output, once I find it, I stop SED. This way I don't need to read all the file (these can get to be very big files). This is a snipet of my SED script:
sed -E -n '/`show running-config`|`show running`|`show running config`/{
p
:loop
n
p
/`show/q
b loop
}' $1/$file
As you can see I am using a multi address range in SED. My question specifically is, how can I achieve something similar in python? I have tried multiple combinations of flags: DOTALL and MULTILINE but I can't get the result I'm expecting, for example, I can get a match for the command I'm looking for, but python regex wont stop until the end of the file after the first match.
I am looking for something like this
sed -n '/`show clock`/,/`show/p'
I would like the regex match to stop parsing the file and print the results, immediately after seeing `show again , hope that makes sense and thank you all for reading me and for your help
You can use nested loops.
import re
def process_file(filename):
with open(filename) as f:
for line in f:
if re.search(r'`show running-config`|`show running`|`show running config`', line):
print(line)
for line1 in f:
print(line1)
if re.search(r'`show', line1):
return
The inner for loop will start from the next line after the one processed by the outer loop.
You can also do it with a single loop using a flag variable.
import re
def process_file(filename):
in_show = False
with open(filename) as f:
for line in f:
if re.search(r'`show running-config`|`show running`|`show running config`', line):
in_show = True
if in_show
print(line)
if re.search(r'`show', line1):
return

ACK packets forged issues: "This frame is a (suspected) retransmission"

I'm playing with scapy. I'm trying to forge JUST PSH/ACK and ACK packets in sequence
I coded two tools: A which sends PSH/ACK packets and then sniffs the resulting ACK, writing the sequence in a file to use it later
.....
bitack = random.randrange(1,656787969)
bitseq = random.randrange(1,4294967295)
if os.path.exists('test.txt'):
with open('test.txt','r') as f:
bitseq = int(f.read())
else:
with open('test.txt','w') as f:
f.write(str(bitseq))
.....
text = "Ok"
TSval = int(time.time())
TSecr = TSval
acker = IP(src="127.0.0.1",dst=destinazione"127.0.0.1")/TCP(sport=88,dport=8888,
flags="PA", seq=bitseq, ack=bitack, options=[('Timestamp', (TSval, TSecr))])/text
send(acker)
.....
rx = sniff(filter="host 127.0.0.1 and src port 8888", iface="lo", count=1)
seqcc = rx[0].getlayer(TCP).seq
ackcc = rx[0].getlayer(TCP).ack
with open('test.txt','w') as f:
f.write(str(ackcc))
print("SEQFINALE=", ackcc)
B: which sends ACK packets AFTER it sniffs a PSH/ACK packet from A. I know the ack packets contain text ( in this example the same of A), but this is what I want
....
rx = sniff(filter="host 127.0.0.1 and dst port 8888", iface="lo", count=1)
seqcc = rx[0].getlayer(TCP).seq
print("seq:", seqcc)
ackcc = rx[0].getlayer(TCP).ack
print("ack:", ackcc)
var = rx[0][Raw].load.decode(encoding='utf-8', errors='ignore')
acker = IP(src="127.0.0.1",dst="127.0.0.1")/TCP(sport=8888,dport=88, flags="A",
seq=ackcc, ack=seqcc + int(len(var)), options=[('Timestamp', (TSval, TSecr))])/var
send(acker)
.....
Everything works fine expect that wireshark gives some warning and I don't understand why:
"Expert Info (Note/Sequence): This frame is a (suspected) retransmission"
The first two packets are perfect:
Is there any issue in how I handle the sequence number/ ack number?
This makes me crazy
It is a retransmission. Your capture shows a frame from 8888 to 88 at seq=1 with 52 bytes of data (len=52). If you ever send another frame from 8888 to 88 at seq=1, it's a retransmission. TCP streams are in a single direction: A sends to B, B ACK's what A sent. (in this case, there should be an ACK=53 in a frame from 88 to 8888, either alone or piggybacking data.)

How to write Chinese characters to file based on unicode code point in Python3

I am trying to write Chinese characters to a CSV file based on their Unicode code points found in a text file in unicode.org/Public/zipped/13.0.0/Unihan.zip. For instance, one example character is U+9109.
In the example below I can get the correct output by hard coding the value (line 8), but keep getting it wrong with every permutation I've tried at generating the bytes from the code point (lines 14-16).
I'm running this in Python 3.8.3 on a Debian-based Linux distro.
Minimal working (broken) example:
1 #!/usr/bin/env python3
2
3 def main():
4
5 output = open("test.csv", "wb")
6
7 # Hardcoded values work just fine
8 output.write('\u9109'.encode("utf-8"))
9
10 # Comma separation
11 output.write(','.encode("utf-8"))
12
13 # Problem is here
14 codepoint = '9109'
15 u_str = '\\' + 'u' + codepoint
16 output.write(u_str.encode("utf-8"))
17
18 # End with newline
19 output.write('\n'.encode("utf-8"))
20
21 output.close()
22
23 if __name__ == "__main__":
24 main()
Executing and viewing results:
example $
example $./test.py
example $
example $cat test.csv
鄉,\u9109
example $
The expected output would look like this (Chinese character occurring on both sides of the comma):
example $
example $./test.py
example $cat test.csv
鄉,鄉
example $
chr is used to convert integers to code points in Python 3. Your code could use:
output.write(chr(0x9109).encode("utf-8"))
But if you specify the encoding in the open instead of using binary mode you don't have to manually encode everything. print to a file handles newlines for you as well.
with open("test.txt",'w',encoding='utf-8') as output:
for i in range(0x4e00,0x4e10):
print(f'U+{i:04X} {chr(i)}',file=output)
Output:
U+4E00 一
U+4E01 丁
U+4E02 丂
U+4E03 七
U+4E04 丄
U+4E05 丅
U+4E06 丆
U+4E07 万
U+4E08 丈
U+4E09 三
U+4E0A 上
U+4E0B 下
U+4E0C 丌
U+4E0D 不
U+4E0E 与
U+4E0F 丏

Buffered UDP packets during shortage time

The situation i have is Linux client is using UDP Socket. Client is sending message and in case no response within 10 seconds , Client will retry again in 10 seconds intervals.
The case is when connection is down , many trials are sent from client at the same time nothing was received on server side . Once the connection is up i found all previous messages are received at the same moment on server which means it was buffered and it causes a lot of problems because of duplicated messages received on the same moment on server side .
TCPDUMP ON client Side:
21:01:14.691903 IP 172.123.13211 > 172.34.13211: length 88 "1st at second 14"
21:01:24.692791 IP 172.123.13211 > 172.34.13211: length 88 "2nd at second 24"
21:01:34.694930 IP 172.123.13211 > 172.34.13211: length 88 "3rd at second 34"
21:01:44.696020 IP 172.123.13211 > 172.34.13211: length 88 "4th ate second 44"
Server TCPDUMP once the connection is up :
12:02:01.509518 IP 172.123.13211 > 13211: length 88 "Received 1st at second 1"
12:02:01.517841 IP 172.123.13211 > 13211: length 88 "Received 2nd at second 1"
12:02:01.543759 IP 172.123.13211 > 13211 length 88 "Received 3rd at second 1"
12:02:01.550741 IP 13211 > 172.123.13211: length 36
12:02:01.567948 IP 172.123.13211 > .13211: length 88
I need to understand in case of UDP socket is used and Connection is down .
how to avoid buffering packets during shortage
Client Code is in C++
Thank you
you might be looking for this :
How to flush Input Buffer of an UDP Socket in C?
Also the language you have used in the question is wrong. Please be more clean and precise and use relevant terminology

MicroPython client not receive text file larger than 4kb (4096 bytes) from Python Server

I have an micropython client on esp32 board, and Python on linux server. I am trying send 5.5kb text file from Python Server to MicroPython client. It sends successfully but MicroPython client does not receive all data. Codes as follows;
Python Server:
with open('downloads/%s' % (request_path), 'rb') as f:
data = f.read()
self.wfile.write(data) #data is 5.5kb
MicroPython Client
recvData = sock.read(4096).decode('utf-8').split("\r\n")
print("Response_Received:: %s" % recvData)
sock.close()
Response_Received:: ['HTTP/1.0 200 OK', 'Server: SimpleHTTP/0.6 Python/3.5.3', 'Date: Sat, 09 Jun 2018 09:29:41 GMT', '', '# Ity: asdasd\n# ksduygfkhsgdkjfksjdhfg\n kjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy98\n 47y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhs\n gdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349rio\n t34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r3\n 49riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkv\n nvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogijiksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweu\n oiruy9847y397r349riot34jt;o\n giji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbs\n djkvbjcxbvhweioufhoiweuoiruy9847y397\n r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufh\n oiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduyg\n fkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhwei\n oufhoiweuoiruy9847y397r349riot\n 34jt;ogiji4vuijo4vjlkvnvl;kksduyg\n fkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhw\n eioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjd\n hfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiwe\n uoiruy9847y397r349riot34jt;ogiji4vuij\n o4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhwe\n ioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfk\n hsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;o\n giji4vuijo4vjlkvnvl;kksduygfkhsgdk\n jfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvh\n weioufhoiweuoiruy9847y397r349riot34jt;ogiji\n 4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiru\n y9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;k4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduyg\n fkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcx\n bvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhs\n gdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdj\n nvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogijiksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9bfkjcbsdjkvbjcxbvhweioufhoi847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweu\nnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogijiksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweu\nnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogijiksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufnvl;k']
Client receives only 4140 bytes of the array data in due to buffer size(4096), 4th element of the recvData is lost. MicroPython does not accept over this Buffer size. How can i receive all my data (5.5kb) in 4th element of recvData array without any loss?
I have tried to fragment the received data, but it was not successful.
while True:
chunck = s.recv(4096)
if not chunck:
break
fragments.append(chunck)
Since your goal is to write the file to the filesystem, the simplest solution is to stop trying to hold the entire file in memory. Instead of building up your fragments array, just write the received chunks to a file:
with open('datafile', 'w') as fd:
while True:
chunk = s.recv(4096)
if not chunk:
break
fd.write(chunk)
This requires a constant amount of memory and can be used to receive
files of arbitrary size.

Resources