Telnet.read.until function doesn't work - python-3.x

Trying to telnet to a Brocade router and the python script is sending out error.... Not sure what is wrong here. Have tried a debug but cant make it working. I believe it's prompt issue. I appreciate if anyone has suggestion how to get it work.
Note: This is Python 3.0
import getpass
import sys
import telnetlib
HOST = "1.1.1.1"
user = "admin"
password = "password"
port = "23"
telnet = telnetlib.Telnet(HOST)
telnet.read_until("sw0 login:,3")
telnet.write(admin + "\r")
if password:
telnet.read_until("Password: ")
telnet.write(password + "\n")
tn.write("term len 0" + "\n")
telnet.write("sh ver br\n")
telnet.write("exit\n")
ERROR:
Traceback (most recent call last):
File "C:\Users\milan\Desktop\telnetNew.py", line 13, in <module>
telnet.read_until("Username :,3")
File "C:\Python33\lib\telnetlib.py", line 299, in read_until
return self._read_until_with_select(match, timeout)
File "C:\Python33\lib\telnetlib.py", line 352, in _read_until_with_select
i = self.cookedq.find(match)
TypeError: Type str doesn't support the buffer API
This is my prompt after logging manually using telnet port 23 and this is what i expected command to work.
_________________________________________________________________________________
Network OS (sw0)
xxxxxxxxxx
sw0 login: xxxxx
Password: xxxxx
WARNING: The default password of 'admin' and 'user' accounts have not been changed.
Welcome to the Brocade Network Operating System Software
admin connected from 10.100.131.18 using console on sw0
sw0# sh ver
sw0#

In looking at the docs, it appears that telnetlib want a bytestr, not a Str. so try this., which should convert everything to bytes as opposed to Str
import sys
import telnetlib
HOST = "1.1.1.1"
user = "admin"
password = "password"
port = "23"
telnet = telnetlib.Telnet(HOST,port)
telnet.read_until(b"login: ")
telnet.write(admin.encode('ascii') + b"\n")
telnet.read_until(b"Password: ")
telnet.write(password.encode('ascii') + b"\n")
tn.write(b"term len 0\n")
telnet.write(b"sh ver br\n")
telnet.write(b"exit\n")
--edit-- I installed python and tried this against one of my routers. Changing the username/password to my credentials I was able to login fine. ( I removed the password check and the getpass as they where not being used, being that your code has them hardcoded). It looks like you copied the 2.x example, but the 3.x requires the buffer compatible ones without the b" i Get this
Traceback (most recent call last):
File "foo.py", line 5, in <module>
telnet.read_until("login: ")
File "/usr/local/Cellar/python32/3.2.5/Frameworks/Python.framework/Versions/3.2/lib/python3.2/telnetlib.py", line 293, in read_until
return self._read_until_with_poll(match, timeout)
File "/usr/local/Cellar/python32/3.2.5/Frameworks/Python.framework/Versions/3.2/lib/python3.2/telnetlib.py", line 308, in _read_until_with_poll
i = self.cookedq.find(match)
TypeError: expected an object with the buffer interface
with the b" I get
[~] /usr/local/bin/python3.2 foo.py
b'\r\n\r\nThis computer system including all related equipment, network devices\r\n(specifically including Internet access), are provided only for\r\nauthorized use. All computer
which shows it is working.. what errors are you getting now

import telnetlib , socket
class TELNET(object):
def __init__(self):
self.tn = None
self.username = "root"
self.password = "12345678"
self.host = "10.1.1.1"
self.port = 23
self.timeout = 5
self.login_prompt = b"login: "
self.password_prompt = b"Password: "
def connect(self):
try :
self.tn = telnetlib.Telnet(self.host,self.port,self.timeout)
except socket.timeout :
print("TELNET.connect() socket.timeout")
self.tn.read_until(self.login_prompt, self.timeout)
self.tn.write(self.username.encode('ascii') + b"\n")
if self.password :
self.tn.read_until(self.password_prompt,self.timeout)
self.tn.write(self.password.encode('ascii') + b"\n")
def write(self,msg):
self.tn.write(msg.encode('ascii') + b"\n")
return True
def read_until(self,value):
return self.tn.read_until(value)
def read_all(self):
try :
return self.tn.read_all().decode('ascii')
except socket.timeout :
print("read_all socket.timeout")
return False
def close(self):
self.tn.close()
return True
def request(self,msg):
self.__init__()
self.connect()
if self.write(msg) == True :
self.close()
resp = self.read_all()
return resp
else :
return False
telnet = TELNET()
#call request function
resp = telnet.request('ps www') # it will be return ps output
print(resp)
That code works on python3

Most of these answers are great in-depth explanations and I found them a little hard to follow. The short answer is that the Sample code is wrong on the Python 3x site.
to fix this, instead of
' telnet.read_until("Username :,3")'
use
' telnet.read_until(b"Username :,3")'

Like the other answers say, the problem is that Telnetlib expects byte strings. In Python 2 the str type is a byte string (a string of binary data) whereas it's a unicode string in Python 3 and a new bytes type that represents binary data in the same was as str did in Python 2.
The upshot of this is that you need to convert your str data into bytes data when using Telnetlib in Python 3. I have some code that works in both Python2 and Python3 that I thought worth sharing.
class Telnet(telnetlib.Telnet,object):
if sys.version > '3':
def read_until(self,expected,timeout=None):
expected = bytes(expected, encoding='utf-8')
received = super(Telnet,self).read_until(expected,timeout)
return str(received, encoding='utf-8')
def write(self,buffer):
buffer = bytes(buffer, encoding='utf-8')
super(Telnet,self).write(buffer)
def expect(self,list,timeout=None):
for index,item in enumerate(list):
list[index] = bytes(item, encoding='utf-8')
match_index, match_object, match_text = super(Telnet,self).expect(list,timeout)
return match_index, match_object, str(match_text, encoding='utf-8')
You then instantiate the new Telnet class instead of telnetlib.Telnet. The new class overrides methods in Telnetlib to perform conversion from str to bytes and bytes to str so that the problem just goes away.
Of course, I've only overriden the methods that I was using, but it should be clear from the example how to extend the idea to other methods. Basically, you need to use
bytes(str_string, encoding='utf-8')
and
str(bytes_string, encoding='utf-8')
My code is valid Python 2 and 3 so will work unmodified with either interpreter. When in Python2 the code that overrides the methods is skipped over, effectively ignored. If you don't care for backwards compatibility you can remove the condition that checks for the Python version.

Related

Please , can you help me get rid of this error?

I am trying to create a code for a random password generator with python and flask. I have created both the front end and the backend yet every time i am trying to run it i get a json error:
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
this is my front end code:
import requests
import json
session=requests.session()
data={"letters":0,"symbols":0,"numbers":0}
header={"content-type": "application/json"}
print("Welcome to the PyPassword generator!")
while True:
data ["letters"]=input("how many lettres would you like in your password? > ")
data ["symbols"]=input("how many symbols would you like? > ")
data ["numbers"]=input("how many numbers would you like? > ")
print(data)
r=session.post("http://127.0.0.1:1337/api/PyPassword_generator", data=json.dumps(data), headers=header)
if r:
break
json_data=r.json()
print(json_data)
EDIT: Ading the response from the server component
Traceback (most recent call last):
File "C:\Users\User\PycharmProjects\pythonProject\venv\lib\site-packages\requests\models.py", line 972, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\User\PycharmProjects\pythonProject\venv\lib\site-packages\simplejson\__init__.py", line 525, in loads
return _default_decoder.decode(s)
File "C:\Users\User\PycharmProjects\pythonProject\venv\lib\site-packages\simplejson\decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "C:\Users\User\PycharmProjects\pythonProject\venv\lib\site-packages\simplejson\decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\User\Downloads\client.py", line 19, in <module>
json_data = r.json()
File "C:\Users\User\PycharmProjects\pythonProject\venv\lib\site-packages\requests\models.py", line 976, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
b'<!doctype html>\n<html lang=en>\n<title>405 Method Not Allowed</title>\n<h1>Method Not Allowed</h1>\n<p>The method is not allowed for the requested URL.</p>\n'
Process finished with exit code 1ode here
so this is the server code
from flask import Flask, request, session, jsonify,json
import random
class game:
def __init__(self,lettres,symbols,numbers):
self.letters=lettres
self.symbols=symbols
self.numbers=numbers
self.app = Flask("Shmuel Api server",static_url_path='')
self.app.secret_key="Secret"
#self.app.route('/api/ PyPassword_generator', methods=['Get',"Post"])
def PyPassword_generator():
print(request.data)
print(json.loads(request.data.decode("ascii")))
a=int(request.json["letters"])
b=int(request.json["symbols"])
c=int(request.json["numbers"])
password = self.test(a,b,c)
jsonify({f"your password is : {password}"})
def run(self):
debug=True
self.app.run(host='10.0.0.10', port='5000', debug=debug)
password_list=[]
for char in range(a):
password_list.append(random.choice(self.letters))
for char in range(b):
password_list.append(random.choice(self.symbols))
for char in range(c):
password_list.append(random.choice(self.numbers))
random.shuffle(password_list)
password="".join(password_list)
return password
mp=game(["A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z"],["0","1","2","3","4","5","6","7","8","9"],["!","#","$","%","&","(",")","*","+"])
mp.run()
and this is the response I get
* Serving Flask app 'Shmuel Api server' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://10.0.0.10:5000 (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 446-336-887
10.0.0.10 - - [11/Jun/2022 21:23:52] "POST /api/PyPassword_generator HTTP/1.1" 405 -
There's a space in the URL of the server component, and the methods must be in caps...
#self.app.route('/api/ PyPassword_generator', methods=['Get',"Post"])
should look like:
#self.app.route('/api/PyPassword_generator', methods=['GET',"POST"])
Defining Flask routes inside an object like this looks wrong though. I would probably simplify it:
from flask import Flask, request, session, jsonify,json
import random
app = Flask("__name__")
app.config['SECRET_KEY']='make me super secret'
#app.route('/api/PyPassword_generator', methods=['GET',"POST"])
def PyPassword_generator():
# print(request.data)
# print(json.loads(request.data.decode("ascii")))
try:
submitted_data = request.get_json()
a=int(submitted_data["letters"])
b=int(submitted_data["symbols"])
c=int(submitted_data["numbers"])
except ValueError: # Some issue with the input
return {'error': 'invalid input data'}, 400
password = generate_password(a,b,c)
return jsonify({'message': f"your password is : {password}"})
Then on the client end, simplify things, by using requests native json argument, which means you don't need to use the json.dumps or pass the headers either, as this arg automatically sets the Content-Type...
import requests
session=requests.session()
data={"letters":0,"symbols":0,"numbers":0}
print("Welcome to the PyPassword generator!")
while True:
data["letters"]=input("how many lettres would you like in your password? > ")
data["symbols"]=input("how many symbols would you like? > ")
data["numbers"]=input("how many numbers would you like? > ")
print(data)
r = session.post("http://127.0.0.1:5005/api/PyPassword_generator", json=data)
json_data=r.json()
print(json_data)
if r.ok:
break
Of course generating passwords comes with its own security considerations, and I assume this is just for learning python. Transmitting passwords over the wire comes with security considerations you should understand first.
With that in mind, instead of random.shuffle for this, perhaps investigate the python secrets module. Let's implement the generate_password function, and instead only work with the input a to set the length of the overall password. Using their example, you could have something like:
import secrets
import string
def generate_password(a,b,c):
alphabet = string.ascii_letters + string.digits + string.punctuation
password = ''.join(secrets.choice(alphabet) for i in range(a))
return password
Now this all works like:
Welcome to the PyPassword generator!
how many lettres would you like in your password? > 20
how many symbols would you like? > 1
how many numbers would you like? > 1
{'letters': '20', 'symbols': '1', 'numbers': '1'}
{'message': 'your password is : R#!o]KFnqk`,+Wg{JQ:#'}
Of course some obvious changes required on your part, but hope this helps.

How can I use/authenticate msfrpc with python3.x?

Edited: The code below works, and the changes are commented. As noted, with python3 one must prefix the string literals with a "b" to produce an instance of the byte type instead of a Unicode str type.
I'm trying to use msfrpc (written in Python 2) in Python 3, and I'm encountering authentication errors. The code I'm using is below; see the comments in the code for the changes I made.
The program runs successfully in python2 (when using httplib rather than http.client), with what appears to be the same authentication exchange as seen when using python3.
import msgpack
import http.client #Changed from httplib
class Msfrpc:
class MsfError(Exception):
def __init__(self,msg):
self.msg = msg
def __str__(self):
return repr(self.msg)
class MsfAuthError(MsfError):
def __init__(self,msg):
self.msg = msg
def __init__(self,opts=[]):
self.host = opts.get('host') or "127.0.0.1"
self.port = opts.get('port') or 55552
self.uri = opts.get('uri') or "/api/"
self.ssl = opts.get('ssl') or False
self.authenticated = False
self.token = False
self.headers = {"Content-type" : "binary/message-pack" }
if self.ssl:
self.client = http.client.HTTPSConnection(self.host,self.port) #Changed httplib -> http.client
else:
self.client = http.client.HTTPConnection(self.host,self.port) #Changed httplib -> http.client
def encode(self,data):
return msgpack.packb(data)
def decode(self,data):
return msgpack.unpackb(data)
def call(self,meth,opts = []):
if meth != "auth.login":
if not self.authenticated:
raise self.MsfAuthError("MsfRPC: Not Authenticated")
if meth != "auth.login":
opts.insert(0,self.token)
opts.insert(0,meth)
params = self.encode(opts)
self.client.request("POST",self.uri,params,self.headers)
resp = self.client.getresponse()
return self.decode(resp.read())
def login(self,user,password):
ret = self.call('auth.login',[user,password])
if ret.get(b'result') == b'success': #Added b
self.authenticated = True
self.token = ret.get(b'token') #Added b
return True
else:
raise self.MsfAuthError("MsfRPC: Authentication failed")
if __name__ == '__main__':
# Create a new instance of the Msfrpc client with the default options
client = Msfrpc({})
# Login to the msfmsg server using the password "abc123"
client.login('msf','abc123')
# Get a list of the exploits from the server
mod = client.call('module.exploits')
# Grab the first item from the modules value of the returned dict
print ("Compatible payloads for : %s\n" % mod[b'modules'][0]) #Added () #Added b
# Get the list of compatible payloads for the first option
ret = client.call('module.compatible_payloads',[mod[b'modules'][0]]) #Added b
for i in (ret.get(b'payloads')): #Added b
print ("\t%s" % i) #Added ()
When running the program, the result is:
root#kali:~/Dropbox/PythonStuff/python-nmap-test# python3 test3.py
Traceback (most recent call last):
File "test3.py", line 20, in <module>
client.login('msf','abc123')
File "/usr/local/lib/python3.7/dist-packages/msfrpc.py", line 64, in login
raise self.MsfAuthError("MsfRPC: Authentication failed")
msfrpc.MsfAuthError: 'MsfRPC: Authentication failed'
Curiously, running tcpdump while the program is executing shows a successful authentication and a token being granted:
When the program executes successfully with python2 the authentication exchange appears identical, but if someone feels posting the packet capture of the program (running in python2) completing successfully would help to answer the question I can easily add it.
There wasn't much interest in this but I'm posting the answer should anyone else have the question; the solution was pretty straightforward and I noted the changes in the code posted above.
The problem was that msfrpc was originally written in Python 2, and therefore the prefix of 'b' in the msgpack RPC responses from the Metasploit host was ignored but it is required in Python 3 to indicate that the literal should become a bytes literal instead of a string type.
For more information, check out the link below; once I read it the answer was pretty obvious:
https://timothybramlett.com/Strings_Bytes_and_Unicode_in_Python_2_and_3.html
The above code works fine, but really a better solution is to use the msfrpc module written by Dan McInerney which instead uses the 'requests' package.
https://github.com/DanMcInerney/msfrpc

How to solve the problem with the error: UnicodeDecodeError: 'utf-8'?

The program should output the results executed in the user's console and send the result to me by mail. Commands from the console work fine, except for "ipconfig".
Code that works correctly:
import subprocess, time, os, smtplib, re
from subprocess import check_output
serverIP = "iperf.net.ru"
def startShell(serverIP):
"""Start windows command line, measuring Internet channel"""
print("Measuring external channel, please wait...")
try:
information = check_output("iperf3 -c %s -i 11" % serverIP, shell = True).decode()
return information
except:
print("There was an error connecting to the server!")
The code in which the error occurs:
def netSettings():
"""Show network settings"""
try:
netSet = check_output("ipconfig", shell = True).decode()
return netSet
except:
print("An error occurred while displaying the settings!")
Error Type:
>>> netSet = check_output("ipconfig", shell = True).decode()
Traceback (most recent call last):
File "<pyshell#39>", line 1, in <module>
netSet = check_output("ipconfig", shell = True).decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8d in position 2: invalid start byte
Output without decode function ()
>>> netSet = check_output("ipconfig", shell = True)
>>> print(netSet)
b'\r\n\x8d\xa0\xe1\xe2\xe0\xae\xa9\xaa\xa0 \xaf\xe0\xae\xe2\xae\xaa\xae\xab\xa0 IP
Really looking forward to your advice! Honestly searched the forum before asking!
The shell used to capture the output is using a Russian OEM encoding of cp866, so you're probably using a Russian-localized version of Windows:
b = b'\r\n\x8d\xa0\xe1\xe2\xe0\xae\xa9\xaa\xa0 \xaf\xe0\xae\xe2\xae\xaa\xae\xab\xa0 IP'
print(b.decode('cp866'))
Output:
Настройка протокола IP
You could parse the output of chcp before ipconfig to get the code page.
It seems like the return of check_output isn't encoded using utf-8. Here's a list of standard encodings that you can try putting into decode: https://docs.python.org/3/library/codecs.html
netSet = check_output("ipconfig", shell = True).decode(encoding="866")
Mark Tolonen helped me solve the problem!

python3 OSError: [Errno -9999] Unanticipated host error with pyaudio in speech recognition AI

I am working on an AI, like Jarvis in python3. I am using the python speech_recognition module and pyaudio and everything else required acording to this page.
https://pypi.python.org/pypi/SpeechRecognition/
I have it on a raspberry pi now, before i was using my mac which was working fine. Now sometimes i get an error when running my Jarvis code on my Raspberry pi! Not always but frequetly enough to put a wrench in our progress. And not knowing when the error will come is a big problem and we need to get rid of it. Iḿ using a blue Snowball mic. Here is my code and my error if you could help, that would be great thanks!
Traceback (most recent call last):
File "/media/pi/TRAVELDRIVE/Jarvis(10.0).py", line 172, in <module>
with m as source: r.adjust_for_ambient_noise(source)
File "/usr/local/lib/python3.4/dist-packages/speech_recognition/__init__.py", line 140, in __enter__
input=True, # stream is an input stream
File "/usr/local/lib/python3.4/dist-packages/pyaudio.py", line 750, in open
stream = Stream(self, *args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/pyaudio.py", line 441, in __init__
self._stream = pa.open(**arguments)
OSError: [Errno -9999] Unanticipated host error
Jarvis.py
#JARVIS mark 10. python 3.5.1 version
#JUST.A.RATHER.VERY.INTELEGENT.SYSTEM.
##import speech_recognition
##import datetime
##import os
##import random
##import datetime
##import webbrowser
##import time
##import calendar
from difflib import SequenceMatcher
import nltk
from nltk.tokenize import sent_tokenize, word_tokenize
from nltk.tokenize import PunktSentenceTokenizer
import speech_recognition as sr
import sys
from time import sleep
import os
import random
r = sr.Recognizer()
m = sr.Microphone()
#Brain functions, vocab!
what_i_should_call_someone = [""]
Good_Things = ["love","sweet","nice","happy","fun","awesome","great"]
Bad_Things = ["death","kill","hurt","harm","discomfort","rape","pain","sad","depression","depressed","angry","mad","broken","raging","rage"]
# Words that you might says in the beginning of your input, for example: "um hey where are we!?!"
Slang_Words = ["um","uh","hm","eh"]
# Put all greetings in here
Static_Greetings = ["Hey","Hi","Hello"]
# Put your AIs Name and other names just in case.
Name = ["jarvis"]
posible_answer_key_words = ["becuase","yes","no"]
Chance_that_question_was_asked_1 = 0
Chance_that_question_was_asked_2 = 0
certainty_question_was_asked = 0
Me_statment_keywords = ["you","your","yours"]
You_statment_keywords = ["i","i'm","me"]
global certainty_person_is_talking_to_me
what_i_said = ("")
Just_asked_querstion = False
the_last_thing_i_said = ("")
the_last_thing_person_said = ("")
what_person_said = ("")
what_person_said_means = [""]
what_im_about_to_say = [""]
why_im_about_to_say_it = [""]
who_im_talking_to = [""]
how_i_feel = [""]
why_do_i_feel_the_way_i_do = [""]
what_i_am_thinking = ("")
# ways to describe the nouns last said
it_pronouns = ["it","they","she","he"]
# last person place or thing described spoken or descussed!
last_nouns = [""]
# Sample of random questions so Jarvis has somthing to index to know what a question is!
Sample_Questions = ["what is the weather like","where are we today","why did you do that","where is the dog","when are we going to leave","why do you hate me","what is the Answer to question 8",
"what is a dinosour","what do i do in an hour","why do we have to leave at 6.00", "When is the apointment","where did you go","why did you do that","how did he win","why won’t you help me",
"when did he find you","how do you get it","who does all the shipping","where do you buy stuff","why don’t you just find it in the target","why don't you buy stuff at target","where did you say it was",
"when did he grab the phone","what happened at seven am","did you take my phone","do you like me","do you know what happened yesterday","did it break when it dropped","does it hurt everyday",
"does the car break down often","can you drive me home","where did you find me"
"can it fly from here to target","could you find it for me"]
Sample_Greetings = ["hey","hello","hi","hey there","hi there","hello there","hey jarvis","hey dude"]
Question_Keyword_Answer = []
Int_Question_Keywords_In_Input = []
Possible_Question_Key_Words = ["whats","what","where","when","why","isn't","whats","who","should","would","could","can","do","does","can","can","did"]
Possible_Greeting_Key_Words = ["hey","hi","hello",Name]
# In this function: Analyze the user input find out if it's (Question, Answer, Command. Etc) and what is being: Asked, Commanded, ETC.
def Analyze():
def Analyze_For_Greeting():
def Greeting_Keyword_Check():
global Possible_Greeting_Key_Words
Int_Greeting_Keywords_In_Input = []
for words in what_person_said_l_wt:
if words in Possible_Greeting_Key_Words:
Int_Greeting_Keywords_In_Input.append(words)
Amount_Greeting_Keywords = (len(Int_Greeting_Keywords_In_Input))
if Amount_Greeting_Keywords > 0:
return True
def Greeting_Sentence_Match():
for Ran_Greeting in Sample_Greetings:
Greeting_Matcher = SequenceMatcher(None, Ran_Greeting, what_person_said_l).ratio()
if Greeting_Matcher > 0.5:
print (Greeting_Matcher)
print ("Similar to Greeting: "+Ran_Greeting)
return True
Greeting_Keyword_Check()
Greeting_Sentence_Match()
#In this function: determin if the input is a question or not.
def Analyze_For_Question():
# In this function: if there is atleast one question keyword in the user input then return true.
def Question_Keyword_Check():
global Possible_Question_Key_Words
Int_Question_Keywords_In_Input = []
for words in what_person_said_l_wt:
if words in Possible_Question_Key_Words:
Int_Question_Keywords_In_Input.append(words)
Amount_Question_keywords = (len(Int_Question_Keywords_In_Input))
if Amount_Question_keywords > 0:
return True
# In this function: if the users input is simular to other sample questions, return true.
def Question_Sentence_Match():
for Ran_Question in Sample_Questions:
Question_Matcher = SequenceMatcher(None, Ran_Question, what_person_said_l).ratio()
if Question_Matcher > 0.5:
print (Question_Matcher)
print ("Similar to Question: "+Ran_Question)
return True
# In this function: if the first word of the users input is a question keyword and there is a different question keyword in the input return true.
def Question_Verb_Noun_Check():
#if you say "hey jarvis" before somthing like a question or command it will still understand
try:
for word in what_person_said_l_wt:
if word in Static_Greetings or word in Name:
print (word)
Minus_Begin_Greet1 = what_person_said_l_wt.remove(word)
print (Minus_Begin_Greet1)
return True
except IndexError:
pass
Question_Keyword_Check()
Question_Sentence_Match()
Question_Verb_Noun_Check()
if Question_Keyword_Check()==True and Question_Sentence_Match()==True and Question_Verb_Noun_Check()==True:
return True
else:
return False
# All the funtions in Analyze
Analyze_For_Greeting()
Analyze_For_Question()
Conversation=True
Conversation_Started=False
while Conversation==True:
try:
if Conversation_Started==False:
#Greeting()
Conversation_Started=True
with m as source: r.adjust_for_ambient_noise(source)
print(format(r.energy_threshold))
print("Say something!") # just here for now and testing porposes so we know whats happening
with m as source: audio = r.listen(source)
print("Got it! Now to recognize it...")
try:
# recognize speech using Google Speech Recognition
value = r.recognize_google(audio)
# we need some special handling here to correctly print unicode characters to standard output
if str is bytes: # this version of Python uses bytes for strings (Python 2)
print(u"You said {}".format(value).encode("utf-8"))
else: # this version of Python uses unicode for strings (Python 3+)
print("You said {}".format(value))
what_person_said_l = value.lower()
what_person_said_l_wt = word_tokenize(what_person_said_l)
Analyze()
except sr.UnknownValueError:
print ("what was that?")
except sr.RequestError as e:
print("Uh oh! Sorry sir Couldn't request results from Google Speech Recognition service; {0}".format(e))
except KeyboardInterrupt:
pass

How convert an mbox to a JSON structure?

I am trying to convert an mbox to a JSON structure suitable for import into MongoDB i.e.
I am using mining social web second edition mailbox chapter but its not working properly.
I am trying to convert an mbox to a JSON structure suitable for import into MongoDB i.e.
I am using mining social web second edition mailbox chapter but its not working properly.
import sys
import mailbox
import email
import quopri
import json
import time
from BeautifulSoup import BeautifulSoup
from dateutil.parser import parse
MBOX = 'resources/ch06-mailboxes/data/enron.mbox'
OUT_FILE = MBOX + '.json'
def cleanContent(msg):
# Decode message from "quoted printable" format, but first
# re-encode, since decodestring will try to do a decode of its own
msg = quopri.decodestring(msg.encode('utf-8'))
# Strip out HTML tags, if any are present.
# Bail on unknown encodings if errors happen in BeautifulSoup.
try:
soup = BeautifulSoup(msg)
except:
return ''
return ''.join(soup.findAll(text=True))
# There's a lot of data to process, and the Pythonic way to do it is with a
# generator. See http://wiki.python.org/moin/Generators.
# Using a generator requires a trivial encoder to be passed to json for object
# serialization.
class Encoder(json.JSONEncoder):
def default(self, o): return list(o)
# The generator itself...
def gen_json_msgs(mb):
while 1:
msg = mb.next()
if msg is None:
break
yield jsonifyMessage(msg)
def jsonifyMessage(msg):
json_msg = {'parts': []}
for (k, v) in msg.items():
json_msg[k] = v.decode('utf-8', 'ignore')
# The To, Cc, and Bcc fields, if present, could have multiple items.
# Note that not all of these fields are necessarily defined.
for k in ['To', 'Cc', 'Bcc']:
if not json_msg.get(k):
continue
json_msg[k] = json_msg[k].replace('\n', '').replace('\t', '').replace('\r', '')\
.replace(' ', '').decode('utf-8', 'ignore').split(',')
for part in msg.walk():
json_part = {}
if part.get_content_maintype() != 'text':
print >> sys.stderr, "Skipping MIME content in JSONification
({0})".format(part.get_content_maintype())
continue
json_part['contentType'] = part.get_content_type()
content = part.get_payload(decode=False).decode('utf-8', 'ignore')
json_part['content'] = cleanContent(content)
json_msg['parts'].append(json_part)
# Finally, convert date from asctime to milliseconds since epoch using the
# $date descriptor so it imports "natively" as an ISODate object in MongoDB
then = parse(json_msg['Date'])
millis = int(time.mktime(then.timetuple())*1000 + then.microsecond/1000)
json_msg['Date'] = {'$date' : millis}
return json_msg
mbox = mailbox.UnixMailbox(open(MBOX, 'rb'), email.message_from_file)
# Write each message out as a JSON object on a separate line
# for easy import into MongoDB via mongoimport
f = open(OUT_FILE, 'w')
for msg in gen_json_msgs(mbox):
if msg != None:
f.write(json.dumps(msg, cls=Encoder) + '\n')
f.close()
print "All done"
getting error:
80 # for easy import into MongoDB via mongoimport
81
---> 82 f = open(OUT_FILE, 'w')
83 for msg in gen_json_msgs(mbox):
84 if msg != None:
IOError: [Errno 13] Permission denied: 'resources/ch06-mailboxes/data/enron.mbox.json'
The code you mentioned became obsolete in Third Edition of Mining Social Web
I tried making a workable script that not just converts MBOX to JSON, but even extracts the Attachments to usable formats.
Link to the repo -
https://github.com/PS1607/mbox-to-json
Read the README file for usage instructions.
It seems that your problem is related to user permissions instead of Python. Line 82 tries to open a file in the "data" folder, but permission was denied. You should try executing your script using the sudo command from a terminal:
sudo python3 <your script name>
This should take care of the error you pointed out.
PS: Python 3 uses print as a function; line 88 should read
print('All done')

Resources