The question is: How to apply PyCharm inspection quick-fixes in bulk? [Presumably the same features are also in IntelliJ, CLion, et al.]
I'm porting a Python 2.7 code base to Python 3.8 and want to make one kind of change at a time [e.g. replace raise Exception, "message" with raise Exception("message")], review it, and test it. Eventually I'll try using python-modernize, but can PyCharm make specific types of compatibility fixes in bulk?
You can configure PyCharm's Code compatibility inspection to check for, say, Python 2.7 + 3.7 + and 3.8, then run that inspection on your entire project or a smaller scope. Or use the Code > Run Inspection by Name... menu command to run the Code compatibility inspection with a temporary choice of options and scope. The warnings will list items like these:
Python version 3.7, 3.8 do not support this syntax
Python version 3.7, 3.8 do not have module __builtin__
Python version 3.7, 3.8 do not have method xrange
Python version 3.7, 3.8 do not have module cPickle
dict.iterkeys(), dict.iteritems() and dict.itervalues() methods are not available in py3
basestring type is not available in py3
If you select an example of these first two types of warnings, PyCharm can apply a "quick-fix" like Convert raise statement to supported form. For the other warnings listed above, it can only offer to suppress the warning :-(
I have hundreds of these! How to fix them in bulk?
There's a cute trick.
In the Inspection Results tool window, click the Expand All button (or type Command-+ or whatever the equivalent is in your keymap).
Type Command-F (or the equivalent) to Find an example of your chosen type of inspection warning, e.g. search for syntax.
(PyCharm won't open a "Find" dialog for this but you can still type the search text and it will display that in an overlay.)
Do Edit > Select All (or type Command-A or equivalent).
Now click the button to apply the quick-fix!
(If you know how to make PyCharm fix the other compatibility warnings, please add more Answers. Otherwise, it's on to modernize.)
EDITED POST
When I first wrote this, I was mystified by the varied behaviors of the "imagesnap" USB cam image capture program on MacOS. Sometimes it took pictures and sometimes it didn't, and this seemed to vary with the environment where it was called: such as directly in a Terminal window, in a shell program, in a Python program running in the Terminal, in a Python program running in the Pycharm IDE, called with os.popen(), subprocess.run(), subprocess.call(), subprocess.popen(), etc.
Sometimes imagesnap would work just fine and take images, and sometimes it would fail silently. I was really having a hard time figuring out why the behaviors were varied, and how to get picture taking using imagesnap on my mac to work reliably when called from a Python program.
I searched on keywords like MacOS, Catalina, imagesnap, USB camera, webcam, Python, PyCharm IDE, shell command, os.popen(), subprocess.run(), subprocess.call(), subprocess.Popen(), and more. I did not find the solution anywhere, and I didn't get an answer here.
I've finally understood the problem better, and that's why I've rewritten this question.
What's behind the inconsistent behavior of imagesnap when called from different environments, and how can I call it from a Linux program and get it to reliably take pictures?
I now understand that MacOS X's privacy settings in System Preferences control which environments are allowed or silently denied to access USB cameras.
For more detail, and a workaround, see my own answer below. I hope that others who understand the issues even more will add improved answers.
ORIGINAL POST:
Below is a simple Python test program. It uses os.popen() and
captures an image from a selected USB webcam using imagesnap -d and
saves an image in image.jpg.
But the imagesnap -d command behaves differently when running in the
PyCharm environment, returning a blank response (' ') and failing to save the requested image.
Submitting this same code WITHIN PyCharm's terminal window fails in
exactly the same way.
But when the same python commands are submitted to the terminal
outside Pycharm, the program takes the image successfully and reports back as shown in the output below.
I'm running Mac OS X Catalina, and using PyCharm 2019.2.3
Community Edition for development. Python version is 3.7.4
Sample code:
import os
return_string = os.popen("imagesnap -d 'DEVICE_NAME' 'image.jpg'").read()
print("'",return_string,"'")
Within the PyCharm development environment this program returns the following and does not create the jpg file:
/Users/mcgregor94086/PycharmProjects/SonaScannerGUI/venv/bin/python
/Users/mcgregor94086/Library/Preferences/PyCharmCE2019.2/scratches/scratch_5.py
' '
Process finished with exit code 0
Outside PyCharm this same code returns:
$ python3
Python 3.7.4 (v3.7.4:e09359112e, Jul 8 2019, 14:54:52)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> return_string = os.popen("imagesnap -d 'DEVICE_NAME' 'image.jpg'").read()
>>> print("'",return_string,"'")
' Capturing image from device "<AVCaptureDALDevice: 0x7fa9bc203330 [DEVICE_NAME][0x141717501bcf0b09]>"... '
>>>
After digging into this problem a great deal, and without any responses, I started to search the web more broadly for answers. And although I found almost no information on the web to guide me what was happening, I now believe that I know what is preventing me from taking pictures programmatically within a python program. I also believe that I now have a reasonable hypothesis as to why I am seeing the results I was seeing.
I have a work around that is currently working, but which I fear may not in the future.
The believe the problem has to do with MacOS and I believe it may vary with the specific release of MacOS. I am currently using MacOS X 10.15.1 Catalina. earlier MacOS X releases might not be as strict.
In the Catalina release at a minimum, ANY connected USB Camera is considered a special device governed by special security, and management of access to the camera device is controlled in the Systems Preferences -> Security & Privacy panel.
When you click the Camera row in the menu on the left of the panel you will see a checkbox list of apps which are allowed to programmatically access you camera. If you unlock the page (lower left), you can change which apps are allowed to access the camera by checking or unchecking them in the list.
On the machines I have available, there is a list of 4 apps in this panel: Terminal, UberConference, Google Chrome, Skype.
The list of available apps varies for other resources, such as Contacts, Calendars, Accessibility, Full Disk Access, etc.
Some of these resources (such as Accessibility and Full Disk Access) selection lists have a pair of "+|-" buttons below the app list that allow you to add, or permanently remove, apps from accessing that resource.
But Camera. Microphone and some other resource do not have such a pair of buttons, and for these devices, the list appear unchangeable by the user.
The panel suggests that the apps get on the list by "requesting access" to the camera, but I have not found an explanation anywhere on line about how to add an app to this access list.
One of the predefined apps in the list is "Terminal". That was checked, so when I ran imagesnap from within a terminal window the program had the necessary privacy privileges and the camera took pictures just fine.
In contrast, when I tried to run a python program within the PyCharm IDE, the PyCharm IDE did not identify itself as a terminal and the request silently failed.
Python has a number of different ways to launch an independent program such as imagesnap. These methods include os.popen(), subprocess.run() and subprocess.Popen(). Some of these methods are deprecated, so we can't rely on them working in the future. Other methods() have additional options which control whether a new "shell" is created or not, and these may complicate determining what environment the app will see.
I found it confusing to try to predict which of these python methods
would succeed in opening the camera and which would not, as well as
their behavior when run inside the IDE vs. in a terminal window.
Perhaps someone else can compile such a table.
Work around: For the present, I was able to work around this problem by using os.popen, by having it open a new terminal with an initial string. This seems to ensure that Apple security sees the parent as the "Terminal" app which we specifically authorized in the control panel.
I then found
os.popen("open -a Terminal.app imagesnap.sh")
worked.
While this work around is working for me at present, I was not happy about using os.popen() instead of one of the subprocess methods, as I worry it may be deprecated. Second, this worked with a shell script ("imagesnap.sh") but I have not figured out how to pass parameters or run executables.
Perhaps a knowledgable reader can help with better suggestions.
I want to set Spyder to use a non-interactive matplotlib backend, so that plot figures do not pop up but I can still save figures to file, like this hint.
However, it looks like Spyder automatically imports matplotlib and sets the backend internally somewhere, so any call I make to matplotlib.use() pops an error saying that MPL has already been called.
Prefs show that "automatically import PyLab" is disabled. Also there is no 'Agg' backend in the dropdown list.
How do I use a non-interactive backend?
Or preferably: how can I allow command-line setting of the backend again? (I would like my script to choose whether to use the interactive/non-interactive backends depending on some variables, before MPL import.)
Spyder prefs Screenshot:
Versions/attempt, on Mac OS 10.13.6 (shows MPL already imported!):
Python 3.6.6 |Anaconda custom (64-bit)| (default, Jun 28 2018, 11:07:29)
IPython 6.5.0 -- An enhanced Interactive Python.
>> matplotlib
Using matplotlib backend: Qt5Agg
(Spyder maintainer here) As mentioned by #ImportanceOfBeingErnest in his comment above, you simply need to turn off the option called Activate Support in the screenshot you posted to be able to set by yourself the Matplotlib backend you want to use in your code.
I have built a simple GUI in tkinter that goes through an algorithm to fit Radial Velocity data. Everything works fine in Spyder, however when I try to run the file in command prompt the GUI's dimensions change drastically. This is how it is supposed to look and how it looks in Spyder. This is how it looks when it's run in command prompt. The plots that are produced show the same behavior described. I've tried setting all the windows in Tkinter so that they are unable to be resized. I'm not sure what else to try. Is there a way to fix this? I am on Windows 10 64bit, Python 3.5.4, and Spyder 3.2.5.
I found out what the problem was. It's due to the fact that Spyder automatically overrides high DPI scaling, whereas both Python.exe and Pythonw.exe do not. The solution was to check the 'Override High DPI Scaling behavior' option under the compatibility tab for both python.exe and pythonw.exe.
I have an iPython Notebook with an interactive slider widget:
To share the notebook, I would like to use the link provided by NBViewer. This would allow me to share the notebook with people that don't use Python. Unfortunately, the widget does not show when viewing the NBViewer link:
Is there something I need to do to make widgets work with NBViewer?
Widgets are not shown in nbviewer with current ipython 2.x.
IPython 3 is supposed to add persistent widgets that can be shown by nbviewer.
The release of ipython 3 was scheduled for this fall but it is likely to happen early next year.
I had to downgrade ipywidget from version 7.0.3 to get nbviewer showing ipywidget.
pip install ipywidgets==6.0.0