Sphinx documentation and links to Markdown - python-3.x

Question

I'm trying to use Sphinx to build some documentation from Markdown source. My conf.py is as follows...
conf.py
from recommonmark.parser import CommonMarkParser
project = 'DS'
copyright = '2018, DS'
author = 'DS, Work'
version = ''
release = ''
extensions = []
templates_path = ['_templates']
source_suffix = ['.rst', '.md']
master_doc = 'index'
language = None
exclude_patterns = []
pygments_style = 'sphinx'
html_theme = 'classic'
html_static_path = ['_static']
source_parsers = {
'.md': CommonMarkParser,
}
htmlhelp_basename = 'DSDocumentationdoc'
latex_elements = {
}
latex_documents = [
(master_doc, 'DSDocumentation.tex', 'DS Documentation',
'DS, Work', 'manual'),
]
man_pages = [
(master_doc, 'dsdocumentation', 'DS Documentation',
[author], 1)
]
texinfo_documents = [
(master_doc, 'DSDocumentation', 'DS Documentation',
author, 'DSDocumentation', 'One line description ofproject.',
'Miscellaneous'),
]
index.rst
Welcome to DS Documentation!
======================================
The following documentation is produced and maintained by the Data Science team.
Contents:
.. toctree::
:maxdepth: 2
:glob:
README.md
documentation.md
getting_started/*
how-tos/*
statistics_data_visualisation.md
The documents build and html output is generated, however README.md has links to other markdown documents in the two sub-directories such as the following...
... [this document](./getting_started/setting_your_machine_up.md)...
...which in the translated README.html document the target has not been converted to the translated HTML target as its been recognised as reference external...
...<a class="reference external" href="./getting_started/setting_your_machine_up.md">this document</a>...
...I was half-expecting/hoping it would output as reference internal and convert the file extension approrpiately...
...<a class="reference internal" href="./getting_started/setting_your_machine_up.html">this document</a>...
...so that links worked in the same vein as the Table of Contents does in the sidebar.
Any suggestions as to whether this can be achieved would be appreciated.
Cheers.
EDIT
Trying out the solution suggested by #waylan I have added the following to by conf.py to enable_auto_doc_ref...
def setup(app):
app.add_config_value('recommonmark_config', {
'enable_auto_doc_ref': True,
}, True)
app.add_transform(AutoStructify)
...and on running make html I get the following error.....
❱ cat /tmp/sphinx-err-57rejer3.log
# Sphinx version: 1.8.0
# Python version: 3.6.6 (CPython)
# Docutils version: 0.14
# Jinja2 version: 2.10
# Last messages:
# building [mo]: targets for 0 po files that are out of date
#
# building [html]: targets for 16 source files that are out of date
#
# updating environment:
#
# 16 added, 0 changed, 0 removed
#
# reading sources... [ 6%] README
#
# Loaded extensions:
# sphinx.ext.mathjax (1.8.0) from /home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/ext/math
jax.py
# alabaster (0.7.11) from /home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/alabaster/__init__.py
Traceback (most recent call last):
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/cmd/build.py", line 304, in build_ma
in
app.build(args.force_all, filenames)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/application.py", line 341, in build
self.builder.build_update()
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 347, in
build_update
len(to_build))
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 360, in
build
updated_docnames = set(self.read())
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 468, in
read
self._read_serial(docnames)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 490, in
_read_serial
self.read_doc(docname)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 534, in
read_doc
doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/io.py", line 318, in read_doc
pub.publish()
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/docutils/core.py", line 218, in publish
self.apply_transforms()
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/docutils/core.py", line 199, in apply_trans
forms
self.document.transformer.apply_transforms()
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/sphinx/transforms/__init__.py", line 90, in
apply_transforms
Transformer.apply_transforms(self)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/docutils/transforms/__init__.py", line 171,
in apply_transforms
transform.apply(**kwargs)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/recommonmark/transform.py", line 325, in ap
ply
self.traverse(self.document)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/recommonmark/transform.py", line 297, in tr
averse
self.traverse(child)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/recommonmark/transform.py", line 297, in tr
averse
self.traverse(child)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/recommonmark/transform.py", line 297, in tr
averse
self.traverse(child)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/recommonmark/transform.py", line 287, in tr
averse
newnode = self.find_replace(c)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/recommonmark/transform.py", line 267, in fi
nd_replace
newnode = self.auto_doc_ref(node)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/recommonmark/transform.py", line 175, in au
to_doc_ref
return self.state_machine.run_role('doc', content=content)
File "/home/neil.shephard#samba.sheffield.thefloow.com/.local/lib/python3.6/site-packages/recommonmark/states.py", line 134, in run_r
ole
content=content)
TypeError: 'NoneType' object is not callable
I've looked through the last two calls and I think this might be down to content not being set, which may be something to do with my index.rst but I'm really out of my depth here.

Answer 1

The recommonmark documentation suggests enabling AutoStructify by adding the following to your config.py file:
from recommonmark.transform import AutoStructify
github_doc_root = 'https://github.com/rtfd/recommonmark/tree/master/doc/'
def setup(app):
app.add_config_value('recommonmark_config', {
'url_resolver': lambda url: github_doc_root + url,
'auto_toc_tree_section': 'Contents',
}, True)
app.add_transform(AutoStructify)
This will give you the following features:
enable_auto_toc_tree: whether enable Auto Toc Tree feature.
auto_toc_tree_section: when enabled, Auto Toc Tree will only be enabled on section that matches the title.
enable_auto_doc_ref: whether enable Auto Doc Ref feature.
enable_math: whether enable Math Formula
enable_inline_math: whether enable Inline Math
enable_eval_rst: whether Embed reStructuredText is enabled.
url_resolver: a function that maps a existing relative position in the document to a http link
Of note is the Auto Doc Ref feature:
It is common to refer to another document page in one document. We
usually use reference to do that. AutoStructify will translate these
reference block into a structured document reference. For example
[API Reference](api_ref.md)
will be translated to the AST of following reStructuredText code
:doc:`API Reference </api_ref>`
And it will be rendered as API Reference
Why is this necessary? Because, unlike Rst, Markdown does not have any knowledge of anything outside of the given document and has no support for Rst style directives. Therefore, there is no mechanism to transform a URL.
Instead, AutoStructify waits until after the recommonmark bridge converts the Markdown to Sphinx's underlying document structure (docutils document object), then it runs a series of transformers on it to provide limited Rst like functionality. Even with AutoStructify, you will never get full feature support when using Markdown. That would require Markdown to have native support for directives, which is not likely to ever happen.