I am using pycharm to 2018.3.4 to develop python scripts and I am still pretty new in this language, I normally write PowerShell code. I have a question concerning debugging. Take this unfinished code as an example.
#! python3
# phoneAndEmail.py - Finds phone numbers and email addresses on the clipboard.
import pyperclip, re, sys
phoneRegex = re.compile(r'''(
(\(?([\d \-\)\–\+\/\(]+)\)?([ .-–\/]?)([\d]+))
)''', re.VERBOSE)
mailRegex = re.compile(r'''(
\w+#\w+\.\w+
)''', re.VERBOSE)
text = pyperclip.paste()
def getMatches(text, regex) :
return regex.findall(text)
Emails = getMatches(text,mailRegex) #I want to play with this variable at runtime
phone = getMatches(text,phoneRegex)
I am at the stage where I want to analyze the variable Emails at runtime. So I set a breakpoint and can view the contents of the varieble just fine. However I also want to run some methods and play with their input parameters at runtime. Does someone know how this is possible? If this is possible with another IDE then this would be fine too.
In Pycharm 2018.3.4 you simply do this by clicking on the console tab and then on the show command prompt button.
Now you can type in Emails or Emails.pop() for example in order to manipulate variables:
You can also refer to this post, where this was discussed for an older version: change variable in Pycharm debugger
Related
i am using a tkinter Text widget to display the content of gerber-code files.
the program runs on a raspberry pi and send code over serial to a machine one line of text at at time.
i set the current active line as follows:
class TextEditor(tkinter.Text):
def __init__(self, tkRoot):
...
self.tag_configure("activeLine", background="#87e8ed")# set the colour used for activeLine
def setLine(self, lineNumber):
self.tag_remove("activeLine", "1.0", "end")
self.tag_add("activeLine", str(lineNumber)+".0 linestart", str(lineNumber)+".0 lineend+1c")
def getLine(self):
pass # need to return the activeLine line number
there should only ever be one line at a time highlighted with "activeLine" so the first instance would be fine.
i could store a variable in the call to setLine and read it back in getLine but i would prefer not to as any edits to the text it could go out of sink
i notice using IDLE that the debugger uses what looks the same principle as i am trying to achieve here to set breakpoints, is it possible and if so where would i start looking for the IDLE source code to look into how it is achieved there, i am currently writing this on a Ubuntu 18.04 desktop i would like to no best ways to search IDLE source from
any help would be greatly appreciated, i am quite new to python and tkinter as i am generally a windows dot.net programmer but i am now learning to use Linux
i have now found an answer to my own question
listing all the functions of the text widget that start with "tag_" like this:
d = dir(self.tkRoot.text)
for dv in d:
s = str(dv)
if s.startswith("tag_"):
print(dv)
i found the method "tag_ranges(name)" that returns me this
(<textindex object: '5.0'>, <textindex object: '6.0'>)
at the time of calling the current line was 5
I've a quick question about "pandas_profiling" .So basically i'm trying to use the pandas 'profiling' but instead of showing the output it says something like this:
<pandas_profiling.ProfileReport at 0x23c02ed77b8>
Where i'm making the mistake?? or Does it have anything to do with Ipython?? Because i'm using Ipython in Anaconda.
try this
pfr = pandas_profiling.ProfileReport(df)
pfr.to_notebook_iframe()
pandas_profiling creates an object that then needs to be displayed or output. One standard way of doing so is to save it as an HTML:
profile.to_file(outputfile="sample_file_name.html")
("profile" being the variable you used to save the profile itself)
It doesn't have to do with ipython specifically - the difference is that because you're going line by line (instead of running a full block of code, including the reporting step) it's showing you the object itself. The code above should allow you to see the report once you open it up.
I am writing an program for a web automation in python. Is here a ways to hide the geckodriver? So that the console (see picture) won't show up when I start the program.
console of geckodriver
here is a fraction of my code:
from selenium import webdriver
from selenium import *
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC`
driver=webdriver.Firefox()
wait=WebDriverWait(driver,120)
url = r"http://google.com"
driver.get(url) #This line starts the console (see picture)
To prevent geckodriver from displaying any windows, you need to pass -headless argument:
from selenium import webdriver
options = webdriver.FirefoxOptions()
options.add_argument('-headless')
driver = webdriver.Firefox(options=options)
driver.get('https://www.example.com')
...
driver.quit()
This worked for me in C#. It blocks both geckodriver and firefox window
FirefoxOptions f = new FirefoxOptions();
f.AddArgument("-headless");
var ffds = FirefoxDriverService.CreateDefaultService();
ffds.HideCommandPromptWindow = true;
driver = new FirefoxDriver(ffds,f);
I was able to do that after implementing PyVirtualDisplay
sudo pip install pyvirtualdisplay # Install it into your Virtual Environment
Then just import Display as follows:
from pyvirtualdisplay import Display
Then, before fetching, start the virtual display as follows:
# initiate virtual display with 'visible=0' activated
# this way you will hide the browser
display = Display(visible=0, size=(800, 600))
# Start Display
display.start()
...
# Do your fetching/scrapping
...
# Stop Display
display.stop()
I hope it helps
Here the approach which has solved it in my case. I used #thekingofravens suggestion but realized that it is just enough to create the Run.bat file which he mentions in his post. Previously I ran my programms in IDLE with F5. Because of this the Geckodriver popped up every few seconds.
My Solution: I just made the Run.bat file with the same code:
cd C:\PathToYourFileWhichIsCausingTheGeckodriverToPopUp
python FileWhichIsCausingTheGeckodriverToPopUp.py
And thats it. Just start this file when you want to run your code and the Geckodriver won't pop up. (That it works without the whole path to you program, Python has to be in PATH.)
Also, of course you can run your program from the command line with the same commands like above and without creating an extra bat file.
You have to take care of a couple of stuffs here:
Keep the useful imports. Unusual imports like from selenium import * (in your code) must be avoided.
Keep your code minimal. wait=WebDriverWait(driver,120) have no usage in your code block.
When you use the raw r switch remember to use single quotes '...'
If you initialize the webdriver instance remember to call the quit() method.
Be careful about indentation while using Python.
Here is the minimal code which uses geckodriver to open the url http://www.google.com, print the Page Title and quits the driver instance.
from selenium import webdriver
driver=webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
url = r'http://www.google.com'
driver.get(url)
print("Page Title is : %s" %driver.title)
driver.quit()
Console Output:
Page Title is : Google
So I found a very generic workaround to solve this on Windows (in Linux it doesn't seem to be an issue in the first place, can't speak for OSX).
So you need to make three files and its very awkward.
First, you make a file. call it start.bat (or anything) and put the following code in it:
wscript.exe "C:\Wherever\invisible.vbs" "C:\Some Other Place\Run.bat"
This will be the top level batch script. It will be visible for a split second while it launches a visual basic script and passes a batch script as an argument. The purpose of this is to make the next console invisible. Next we make the VBscript, invisible.vbs:
CreateObject("Wscript.Shell").Run """" & WScript.Arguments(0) & """", 0, False
Finally, we make the script that invisible.vbs is supposed to hide, we could call it Run.bat
cd C:\Wherever
python script_using_geckodriver.py
What happens is, as follows:
The first .bat file launches invisible.vbs
invisible.vbs launches the second .bat file without showing it on screen.
The second .bat file then launches the python program. Python (and geckodriver) output to the invisible cmd therefore hiding the geckodriver console window.
P.S. all of this works with PyInstaller to produce a single redistributable package the user can just click on.
Credit harrymc # superuser.com for this solution, which I found when trying to solve an unrelated problem. I tested and realized it was cross applicable to this.
https://superuser.com/questions/62525/run-a-batch-file-in-a-completely-hidden-way
When the lib click detects that the runtime is python3 but the encoding is ASCII then it ends the python program abruptly:
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Either switch to Python 2 or consult http://click.pocoo.org/python3/ for mitigation steps.
I found the cause of this issue in my case, when I connect to my Linux host from my Mac, the Terminal.app set the SSH session locale to my Mac locale (es_ES.UTF-8) However my Linux host hasn't installed such locale (only en_US.utf-8).
I applied an initial workaround to fix it (but It had many issues, see accepted answer):
import locale, codecs
# locale.getpreferredencoding() == 'ANSI_X3.4-1968'
if codecs.lookup(locale.getpreferredencoding()).name == 'ascii':
os.environ['LANG'] = 'en_US.utf-8'
EDIT: For a better patch see my accepted answer.
All my linux hosts have installed 'en_US.utf-8' as locale (Fedora uses it as default).
My question is: Is there a better (more robust) way to choose/force the locale in a python3 script ? For instance, setting one of the available locales in the system.
Maybe there is a different approach to fix this issue but I didn't find it.
If you have python version >= 3.7, then you should not need to do anything. If you have python 3.6 see the original solution.
EDIT 2017-12-08
I've seen that there is a PEP 538 for py3.7, that will change the entire behavior of python3 encoding management during startup, I think that the new approach will fix the original problem: https://www.python.org/dev/peps/pep-0538/
IMHO the changes targeted to python 3.7 for encoding issues, should have been planed years ago, but better late than never, I guess.
EDIT 2015-09-01
There is an opened issue (enhancement), http://bugs.python.org/issue15216, that will allow to change the encoding in a created (not-used) stream easily (sys.std*). But is targeted to python 3.7 So, we'll have to wait for a while.
Original solution that targets python version 3.6
NOTE: this solution should not be needed for anyone running python version >= 3.7 see PEP 538
Well, my initial workaround had many flaws, I got to pass the click library check about the encoding, but the encoding itself was not fixed, so I get exceptions when the input parameters or output had non-ascii characters.
I had to implement a more complex method, with 3 steps: set locale, correct encoding in std in/out and re-encode the command line parameters, besides I've added a "friendly" exit if the first try to set the locale doesn't work as expected:
def prevent_ascii_env():
"""
To avoid issues reading unicode chars from stdin or writing to stdout, we need to ensure that the
python3 runtime is correctly configured, if not, we try to force to utf-8,
but It isn't possible then we exit with a more friendly message that the original one.
"""
import locale, codecs, os, sys
# locale.getpreferredencoding() == 'ANSI_X3.4-1968'
if codecs.lookup(locale.getpreferredencoding()).name == 'ascii':
os.environ['LANG'] = 'en_US.utf-8'
if codecs.lookup(locale.getpreferredencoding()).name == 'ascii':
print("The current locale is not correctly configured in your system")
print("Please set the LANG env variable to the proper value before to call this script")
sys.exit(-1)
#Once we have the proper locale.getpreferredencoding() We can change current stdin/out streams
_, encoding = locale.getdefaultlocale()
import io
sys.stderr = io.TextIOWrapper(sys.stderr.detach(), encoding=encoding, errors="replace", line_buffering=True)
sys.stdout = io.TextIOWrapper(sys.stdout.detach(), encoding=encoding, errors="replace", line_buffering=True)
sys.stdin = io.TextIOWrapper(sys.stdin.detach(), encoding=encoding, errors="replace", line_buffering=True)
# And finally we need to re-encode the input parameters
for i, p in enumerate(sys.argv):
sys.argv[i] = os.fsencode(p).decode()
This patch solves almost all issues, however it has a caveat, the method shutils.get_terminal_size() raises a ValueError because the sys.__stdout__ has been detached, click lib uses that method to print the help, to fix it I had to apply a monkey-patch on click lib
def wrapper_get_terminal_size():
"""
Replace the original function termui.get_terminal_size (click lib) by a new one
that uses a fallback if ValueError exception has been raised
"""
from click import termui, formatting
old_get_term_size = termui.get_terminal_size
def _wrapped_get_terminal_size():
try:
return old_get_term_size()
except ValueError:
import os
sz = os.get_terminal_size()
return sz.columns, sz.lines
termui.get_terminal_size = _wrapped_get_terminal_size
formatting.get_terminal_size = _wrapped_get_terminal_size
With this changes all my scripts work fine now when the environment has a wrong locale configured but the system supports en_US.utf-8 (It's the Fedora default locale).
If you find any issue on this approach or have a better solution, please add a new answer.
It's an aged thread, however this answer might help other in the future or myself. If it's *nux
env | grep LC_ALL
if it's set, do the follows. That's all of it.
unset LC_ALL
If you are running python 3.6 then you will still get this error. Here is a simple solution that the authors of click recommend:
#!/bin/bash
# before your python code executes set two environment variables
export LANG=en_US.utf8
export LC_ALL=en_US.utf8
NOTE: replace the values with whatever your locale is configured to
NOTE: this solution is even given in the PEP 538 document seen here.
I haven't found this simple method (re-exec script with proper environment before doing anything) so I'll add it for future travellers using old Python version for some reason. Add it bellow imports to be that first :
if os.environ["LC_ALL"] != "C.UTF-8" or os.environ["LANG"] != "C.UTF-8":
os.execve(sys.executable,
[os.path.realpath(__file__)] + sys.argv,
{"LC_ALL": "C.UTF-8", "LANG": "C.UTF-8"})
I am trying to use jedi to complete python code inside a PyQt application, using QCompleter and QStringListModel to store the possible completion.
Here's a simple working demo:
#!/usr/bin/env python3
from PyQt5.QtCore import *
from PyQt5.QtWidgets import *
import jedi
import sys
class JediEdit(QLineEdit):
def __init__(self, parent=None):
super().__init__(parent)
self._model = QStringListModel()
self._compl = QCompleter()
self._compl.setModel(self._model)
self.setCompleter(self._compl)
self.textEdited.connect(self.update_model)
def update_model(self, cur_text):
script = jedi.Script(cur_text)
compl = script.completions()
strings = list(cur_text + c.complete for c in compl)
self._model.setStringList(strings)
if __name__ == '__main__':
app = QApplication(sys.argv)
line = JediEdit()
line.show()
sys.exit(app.exec_())
If you run the application and write a code which is not completing anything (e.g. or foo =), the completion will actually show all the possible tokens that can go in that position.
So, if I run and write a space in the field, lots of things pops up, from abs to __version__.
I would like to prevent this: is it possible to query jedi.Script to understand if the token is being completed or if a completely new token is starting?
EDIT: another little question: say that I am running an interpreter which is detached from jedi current state. How can I provide local and global variables to jedi.Script so that it will take into account those, instead of its own completions?
Autocompletion
Jedi's autocompletion will always show all possible tokens in a place. That's the whole point in autocompletion.
If you don't want that behavior just scan the last few characters for whitespace and certain other characters like = or :, it would be a very simple regex command. (You could also try to look up Jedi's internals and use the way how Jedi knows about this context. However I'm not going to tell you, because it's not a public API and IMHO regex calls suffice.)
In the future something like that might be possible. (See https://github.com/davidhalter/jedi/issues/253).
Now that I think about it, there might be another way that you could experiment with this: You can try to play with Completion.name and Completion.complete. The latter only gives you what could come after the current token, while the name would be the full thing. So you can compare and if they are equal than you might not want to display anything.
Have fun playing with the API :-)
Interpreter
If you're running an interpreter, you can use jedi.Interpreter to combine code with actual Python objects. It's pretty flexible. But please note that the current Interpreter (0.8.1) is very buggy. Please use the master branch from Github (0.9.0).