We are new to using AI Platform. Following the documentation for creating a new notebook seems to work fine. I am able to launch JupyterLab and navigate through examples. However, there is no output or error/warning messages when cells in the notebook are executed (using tutorial notebooks). The cell gets marked as * however there is nothing happening.
The status in lower left corner goes 'starting' / 'unknown' and then stays on "Reconnecting".. using Python 3 setup (using defaults for all options when creating the notebook). I can stop and restart the kernal but same message. I have also tried changing to the Python 2 kernal, same results.
Is this related to the lack of output/warnings, do I have one problem or two? and how do I know what it is preventing the connection.
I try to open a terminal window and am not able to type any commands.
how do you debug something that isn't reporting errors?
Related
I have started using VS Code for work instead of Anaconda and there are some weird observations which I am not able to figure out.
If I try to execute some code in a cell in the interactive window, many-a-times, only output remains available and the input code automatically gets hidden. For eg. I type the following and execute:-
And this is what I see at the window:-
Similarly, there are some variables which I have already defined but they don't show up in the Jupyer:Variables tab. For eg. there is a variable _link_name_to_index which is not visible in the tab as shown below:-
However, when I execute the same in the window, I can see the values of the dictionary as expected:-
Maybe it is just a matter of some settings, but I still couldn't find out which one. Also, I know I have put a lot of images, because it is not a code doubt as such, rather more of a tool doubt. I have taken only one particular example here, but I face this issue often. I am using VS Code version 1.75.0 in Ubuntu 20.04, using Python 3.9.12 in a virtual environment, if it helps.
This is a bug in the latest Version of VSCode (1.75). It is tracked in this issue. For now, the only option seems to be downgrading to v1.74.x
I've lost a day's work in Jupyter Notebook because it didn't autosave. As I was working, I would click on File > Save and Checkpoint to be safe. I closed the notebook when I finished working and when I tried reopening it, I see that essentially all my work is gone and it says Last Checkpoint: a day ago (autosaved) next to the filename. I now realize that I had been working for a long time on the notebook with an orange box with some kind of error message. I'm sorry that I can't remember the specific message right now, but I think it was something involving POS or POST or POSIT ...?. I saw Trusted next to the error message, so I must've thought things were OK.
I tried opening the filename-checkpoint.ipynb file within the .ipynb_checkpoints folder but it's blank. I looked up a possible solution on Recovering from a Jupyter Disaster, but it requires SQL, which I don't know. Is there any hope for recovering my work? I realize that this is probably a rookie mistake, but I'm pretty new at this.
Thanks
PS: I'm running Python3 on MacOS
Hope you've already found how to recover lost work from Jupyter notebook work. If not, try the following:
Go to Anaconda Navigator (the green circle!)
Launch a Jupyter Lab
In Jupyter Lab, open a Terminal window
Launch iPython in the terminal by typing ipython and hitting enter
Hit Up arrow
All your code are stored in history and each cell compilation that you would've done in the past shows up there.
Copy+Paste it back to a new Jupyter notebook and you are ready to go again!
If you want to copy/paste.
After running terminal and ipython, page_up yo code you want. Click right button on a mouse, then Watch the elements code. Now you can select code, copy and paste it.
If you don't have Anaconda Navigator you can do access the same data the manual way:
open a new ipython terminal
type %history -g -f history.txt
This will output the entire history of edits which are saved in history.sqlite in your IPython profile folder. You can scroll down to the bottom to find your most recent edits.
You can also just explore the history.sqlite directly, which is located in your IPython profile. The profile should be inside .ipython in your user directory (windows and linux).
Thanks to Christian Long's answer on another question for providing this info.
EDITED POST
When I first wrote this, I was mystified by the varied behaviors of the "imagesnap" USB cam image capture program on MacOS. Sometimes it took pictures and sometimes it didn't, and this seemed to vary with the environment where it was called: such as directly in a Terminal window, in a shell program, in a Python program running in the Terminal, in a Python program running in the Pycharm IDE, called with os.popen(), subprocess.run(), subprocess.call(), subprocess.popen(), etc.
Sometimes imagesnap would work just fine and take images, and sometimes it would fail silently. I was really having a hard time figuring out why the behaviors were varied, and how to get picture taking using imagesnap on my mac to work reliably when called from a Python program.
I searched on keywords like MacOS, Catalina, imagesnap, USB camera, webcam, Python, PyCharm IDE, shell command, os.popen(), subprocess.run(), subprocess.call(), subprocess.Popen(), and more. I did not find the solution anywhere, and I didn't get an answer here.
I've finally understood the problem better, and that's why I've rewritten this question.
What's behind the inconsistent behavior of imagesnap when called from different environments, and how can I call it from a Linux program and get it to reliably take pictures?
I now understand that MacOS X's privacy settings in System Preferences control which environments are allowed or silently denied to access USB cameras.
For more detail, and a workaround, see my own answer below. I hope that others who understand the issues even more will add improved answers.
ORIGINAL POST:
Below is a simple Python test program. It uses os.popen() and
captures an image from a selected USB webcam using imagesnap -d and
saves an image in image.jpg.
But the imagesnap -d command behaves differently when running in the
PyCharm environment, returning a blank response (' ') and failing to save the requested image.
Submitting this same code WITHIN PyCharm's terminal window fails in
exactly the same way.
But when the same python commands are submitted to the terminal
outside Pycharm, the program takes the image successfully and reports back as shown in the output below.
I'm running Mac OS X Catalina, and using PyCharm 2019.2.3
Community Edition for development. Python version is 3.7.4
Sample code:
import os
return_string = os.popen("imagesnap -d 'DEVICE_NAME' 'image.jpg'").read()
print("'",return_string,"'")
Within the PyCharm development environment this program returns the following and does not create the jpg file:
/Users/mcgregor94086/PycharmProjects/SonaScannerGUI/venv/bin/python
/Users/mcgregor94086/Library/Preferences/PyCharmCE2019.2/scratches/scratch_5.py
' '
Process finished with exit code 0
Outside PyCharm this same code returns:
$ python3
Python 3.7.4 (v3.7.4:e09359112e, Jul 8 2019, 14:54:52)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> return_string = os.popen("imagesnap -d 'DEVICE_NAME' 'image.jpg'").read()
>>> print("'",return_string,"'")
' Capturing image from device "<AVCaptureDALDevice: 0x7fa9bc203330 [DEVICE_NAME][0x141717501bcf0b09]>"... '
>>>
After digging into this problem a great deal, and without any responses, I started to search the web more broadly for answers. And although I found almost no information on the web to guide me what was happening, I now believe that I know what is preventing me from taking pictures programmatically within a python program. I also believe that I now have a reasonable hypothesis as to why I am seeing the results I was seeing.
I have a work around that is currently working, but which I fear may not in the future.
The believe the problem has to do with MacOS and I believe it may vary with the specific release of MacOS. I am currently using MacOS X 10.15.1 Catalina. earlier MacOS X releases might not be as strict.
In the Catalina release at a minimum, ANY connected USB Camera is considered a special device governed by special security, and management of access to the camera device is controlled in the Systems Preferences -> Security & Privacy panel.
When you click the Camera row in the menu on the left of the panel you will see a checkbox list of apps which are allowed to programmatically access you camera. If you unlock the page (lower left), you can change which apps are allowed to access the camera by checking or unchecking them in the list.
On the machines I have available, there is a list of 4 apps in this panel: Terminal, UberConference, Google Chrome, Skype.
The list of available apps varies for other resources, such as Contacts, Calendars, Accessibility, Full Disk Access, etc.
Some of these resources (such as Accessibility and Full Disk Access) selection lists have a pair of "+|-" buttons below the app list that allow you to add, or permanently remove, apps from accessing that resource.
But Camera. Microphone and some other resource do not have such a pair of buttons, and for these devices, the list appear unchangeable by the user.
The panel suggests that the apps get on the list by "requesting access" to the camera, but I have not found an explanation anywhere on line about how to add an app to this access list.
One of the predefined apps in the list is "Terminal". That was checked, so when I ran imagesnap from within a terminal window the program had the necessary privacy privileges and the camera took pictures just fine.
In contrast, when I tried to run a python program within the PyCharm IDE, the PyCharm IDE did not identify itself as a terminal and the request silently failed.
Python has a number of different ways to launch an independent program such as imagesnap. These methods include os.popen(), subprocess.run() and subprocess.Popen(). Some of these methods are deprecated, so we can't rely on them working in the future. Other methods() have additional options which control whether a new "shell" is created or not, and these may complicate determining what environment the app will see.
I found it confusing to try to predict which of these python methods
would succeed in opening the camera and which would not, as well as
their behavior when run inside the IDE vs. in a terminal window.
Perhaps someone else can compile such a table.
Work around: For the present, I was able to work around this problem by using os.popen, by having it open a new terminal with an initial string. This seems to ensure that Apple security sees the parent as the "Terminal" app which we specifically authorized in the control panel.
I then found
os.popen("open -a Terminal.app imagesnap.sh")
worked.
While this work around is working for me at present, I was not happy about using os.popen() instead of one of the subprocess methods, as I worry it may be deprecated. Second, this worked with a shell script ("imagesnap.sh") but I have not figured out how to pass parameters or run executables.
Perhaps a knowledgable reader can help with better suggestions.
Response times in a PyCharm Python Console are unbearably slow when the console is configured to use IPython. This wasn't the case in a previous installation. When the Python Console is not configured to use IPython, and uses IDLE, it performs normally.
The poor response time seems to be linked to the REPL response function, as depicted in the image below. When the REPL response dialog completes, a background process continues to run for several minutes.
Since this was not a problem in a previous installation, the issue must be a result of a configuration change. However, the only thing I changed was to utilize a Conda env instead of a virtualenv.
Any ideas?
This issue was reported to JetBrains. The issue was duplicated and identified as a bug. The status of the issue can be viewed at:
https://youtrack.jetbrains.com/issue/PY-24880
Here is how to apply the "answer" found in the JetBrains bug filed by #vlmercado https://youtrack.jetbrains.com/issue/PY-24880:
Disable auto popup completions (in Editor->General->Code Completions)
Posting my comment as an answer, I have tried it now for a few days and had 0 REPL hangups.
Ubuntu but should work on every machine.
Go to: File----> Settings---->Editor---->General---->Code Completion
Uncheck Rank completion suggestions based on Machine Learning
That's it. Happy Coding.
Today I had the same problem in Pycharm2019.3 with Python3.8. It may have been caused by using miniconda instead of the full anaconda installation, because after installing the ipython package (with -conda install ipython) the long REPL waiting times have dissappeared.
I am not sure if it is solved by jetbrains, for me, it just randomly starts even after 1-2 hrs of coding,
I have tried every options found in Jet brains as well as Stack Overflow like checking and unchecking of 1,2 in combination as shown in image. Have experienced same with the code completion option as well,
These options might help:
Settings: File > Settings > Build, Execution, Deployment > Console
Ipython Setting option:
Run with Console option:
In the PyUnit view in Pydev...
Suppose the view currently shows 5 tests, 1 of which fails.
When I right click on the failed test, and then run it (with either Run or Debug), all the other tests disappear from the view. How do I stop it from removing all the other tests?
Next question.....
Also, Pydev then creates a run configuration for the test that I just ran. Which means that if I right click on the test module in the PyDev Package Explorer, and then try to "Run As" "Python Unit Test", I then have to select which run configuration. How do I stop it creating a new run configuration?
When you right-click a failed test and run it, you did a new run configuration that just ran it alone (so, you get the results just showing that one).
What is implemented is that you have a 'pin' icon in the pyunit view. You can click it to 'bookmark' the run, so, when you press the 'reload', which is the icon in the right of the pin, it'll reload those results (or you can manually choose the results from any run from the test run history dropdown).
As for running, there's no way to stop it from creating a new run configuration, but what you can do is just re-run the last one with Ctrl+F11 (provided you configured it properly as explained in http://pydev.org/manual_101_run.html).
You can also access old runs with Alt+R, T > number from 1 to 9 an old configuration you want to run.
Even though it is possibly not the exact approach you may expect, one option is to start the Unittest from the command line and attach the debugger by RemoteDebugServer via 'pydevd.py'.
This is now a fully automated option of ePyUnit which includes the automation of remote debugging with PyDev and Eclipse by 'pydevd.py'. This works seamlessly for the 'subprocess' call as well as independently started command line processes.
See:
https://pypi.python.org/pypi/epyunit
https://pythonhosted.org/epyunit/
For basics of remote debugging:
http://www.pydev.org/manual_adv_remote_debugger.html
Also enhanced unittest integration into PyUnit.
Comments and fixes are welcome.
Have fun..