Is it possible for a Python script to write into a iPython Notebook?
with open("my_notebook.ipynb", "w") as jup:
jup.write("print(\"Hello there!\")")
If there's some package for doing so, can I also control the way cells are split in the notebook?
I'm designing a software tool (that carries out some optimization) to prepare an iPython notebook that can be run on some server performing scientific computations.
I understand that a related solution is to output to a Python script and load it within a iPython Notebook using %load my_python_script.py. However, that involves a user to type stuff that I would ideally like to avoid.
Look at the nbformat repo on Github. The reference implementation is shown there.
From their docs
Jupyter (né IPython) notebook files are simple JSON documents, containing text, source code, rich media output, and metadata. Each segment of the document is stored in a cell.
It also sounds like you want to create the notebook programmatically, so you should use the NotebookNode object.
For the code, something like, should get you what you need. new_cell_code should be used if you have code cells versus just plain text cells. Text cells should use the existing markdown formatting.
import nbformat
notebook = nbformat.v4.new_notebook()
text = """Hello There """
notebook['cells'] = [nbformat.v4.new_markdown_cell(text)]
notebook= nbformat.v4.new_notebook()
nbformat.write(notebook,'filename.ipynb')
Related
With IPython/Jupyter it's possible to output markdown using the IPython display module and its MarkDownclass.
Question
How can I accomplish this with Azure Databricks?
What I tried
Databricks display
Tried using Databrick's display with the IPython Markdown class:
from IPython.display import Markdown
display(Markdown('*some markdown* test'))
but this results in the following error:
Exception: Cannot call display(<class 'IPython.core.display.Markdown'>)
IPython display
I then tried to use IPython's display:
from IPython.display import display, Markdown
display(Markdown('*some markdown* test'))
but this just displays the text:
<IPython.core.display.Markdown object>
IPython display_markdown
Tried using IPython's display_markdown:
from IPython.display import display_markdown
display_markdown('# Markdown is here!\n*some markdown*\n- and\n- some\n- more')
but this results in nothing showing up:
Looking up documentation
Also tried checking Azure Databricks documentation. At first I visited https://www.databricks.com/databricks-documentation which leads me to https://learn.microsoft.com/en-ca/azure/databricks/ but I wasn't able to find anything via searching or clicking the links and I usually find Microsoft documentation quite good.
Checking Databrick's display source
As Saideep Arikontham mentioned in the comments, Databricks version 11 and above is using IPython kernel so I dug a bit deeper.
According to Databrick's source for the display function, it will readily render any object that implements _repr_html().
However I'm having a hard time being able to get the raw html output that I'm assuming IPython.display.Markdown should be able to output. I can only find _repr_markdown_() and _data_and_metadata() where the former just calls the latter and the output, at least in Databricks, is just the original raw markdown string.
Markdown and display_markdown are not giving desired output when used in Azure Databricks. I have done the following in Databricks 11.1 runtime.
Taking inputs from the question, I understood that when a class has _repr_html(), it is able to output the desired result. But when this method is absent in class, it is returning an object.
So, for Markdown to work, I have written my own Markdown class where I used Python's markdown library.
from IPython.display import DisplayObject, TextDisplayObject
class Markdown(TextDisplayObject):
def __init__(self,TextDisplayObject):
import markdown as md
#converting markdown to html
self.html = md.markdown(TextDisplayObject)
def _repr_html_(self):
return self.html
Now, this class is not completely same as the IPython.display.Markdown. I have formatted your sample markdown
'# Markdown is here!\n*some markdown*\n- and\n- some\n- more' as following to get the desired result.
Markdown('''# Markdown is here!\n
*some markdown*\n
- and\n
- some\n
- more''')
NOTE:
For display_markdown() to display output, we must specify another argument raw as True (display_markdown(<markdown_str>, raw=True)). However, in Databricks it is returning undefined (NoneType).
Please do install markdown library first using %pip install markdown in Databricks cell.
I'm trying to create nice slides using jupyter notebook and RISE. One of my objectives is to display a pandas-dataframe in a Markdown cell in order to have some styling flexibility.
I am using the following code to display my dataframe in a Markdown cell:
{{Markdown(display(df_x))}}
After running this line, I get the following result:
image of dataframe displayed
I would like to get rid of the text printed below my dataframe (<IPython.core.display.Markdown object>).
I still haven't found a way to achieve this. Could someone give me a hand?
This is the library I'm working with:
from IPython.display import display
Not familiar with Markdown class so not sure why you need that but this text printed in the output cell is coming from the fact that this Markdown class is returning and object and since you're not assigning it to any variable the default behavior for the notebook is to run something like str(your_object) which correctly returns <IPython.core.display.Markdown object>.
So the easiest workaround would be to just assign it to some variable like this:
dummy_var = Markdown(display(df_x))
# or better yet:
_ = Markdown(display(df_x))
I have been using Paraview to visualize and analyse VTU files. I find the calculate gradient filter quite useful. I would like to know if there is a python API for Paraview which I can use to use this filter.
I'm looking for something like this.
import paraview as pv
MyFile = "Myfile0001.vtu"
Divergence = pv.filters.GradientOfUnstructuredDataset.(Myfile)
ParaView is fully scriptable in python. Each part of this doc has a 'do it in python' version.
Whereas API doc does not necessary exist, you can use the Python Trace (in Tool menu), that records action from the GUI and save it as a python script.
EDIT
To get back data as an array, it needs some additional steps as ParaView works on a client/server mode. You should Fetch the data and then you can manipulate the vtkObject, extract the array and convert it to numpy.
Something like
from paraview.simple import *
from vtk.numpy_interface import dataset_adapter as dsa
gridvtu = XMLUnstructuredGridReader(registrationName='grid', FileName=['grid.vtu'])
gradient = GradientOfUnstructuredDataSet(registrationName='Gradient', Input=gridvtu)
vtk_grid = servermanager.Fetch(gradient)
wraped_grid = dsa.WrapObject(vtk_grid)
divergence_array = wraped_grid.PointData["Divergence"]
Note that divergence_array is a numpy.ndarray
You also can write pure vtk code, as in this example on SO
I am using Google Colabs for my research in machine learning.
I do many variations on a network and run them saving the results.
I have a part of my notebook that used to be separate file (network.py) At the start of a training session I used to save this file in a directory that has the results and logs etc. Now that this part of the code is in the notebook it is easier to edit etc, BUT I do not have a file to copy to the output directory that describes the model. how to i take a section of a google colab notebook and save the raw code as a python file?
Things I have tried:
%%writefile "my_file.py" - is able to write the file however the classes are not available to the runtime.
I've a quick question about "pandas_profiling" .So basically i'm trying to use the pandas 'profiling' but instead of showing the output it says something like this:
<pandas_profiling.ProfileReport at 0x23c02ed77b8>
Where i'm making the mistake?? or Does it have anything to do with Ipython?? Because i'm using Ipython in Anaconda.
try this
pfr = pandas_profiling.ProfileReport(df)
pfr.to_notebook_iframe()
pandas_profiling creates an object that then needs to be displayed or output. One standard way of doing so is to save it as an HTML:
profile.to_file(outputfile="sample_file_name.html")
("profile" being the variable you used to save the profile itself)
It doesn't have to do with ipython specifically - the difference is that because you're going line by line (instead of running a full block of code, including the reporting step) it's showing you the object itself. The code above should allow you to see the report once you open it up.