Handling of indented fenced input/output code blocks (e.g. within an itemize)

[nthiery]

Markdown supports fenced code blocks within itemize or other markdown construct denoted by indentation, as in the following example:

# Test file for notedown's handling of indented fenced code blocks

Those may typically occur within an itemize or so.

Normal fenced input/output code blocks:
```{.python .input}
1+1
```
```{.json .output}
[{"execution_count": 1, "output_type": "execute_result", "data": {"text/plain": "2"}, "metadata": {}}]
```

1.  Start of an itemize  
    Indented fenced input/output code blocks

    ```{.python .input}
    1+1
    ```
    ```{.json .output}
    [{"execution_count": 1, "output_type": "execute_result", "data": {"text/plain": "2"}, "metadata": {}}]
    ```

    Some more:

    ```{.python .input}
    1+1
    ```
    ```{.json .output}
    [{"execution_count": 1, "output_type": "execute_result", "data": {"text/plain": "2"}, "metadata": {}}]
    ```

1.  Next item (should be numbered 2.)

Would it be possible to have notedown support them as well? In particular, is there a way to translate such code blocks in the Jupyter notebook model?

Currently notedown gets confused and treat the the fenced code blocks and everything indented after as a single raw codeblock. I tried the different values of --match without success.

[nthiery]

@minrk: you may have insight about this last aspect?

[aaren]

I think this comes down to supporting an indented fenced code block. Technically this should be possible on markdown -> notebook conversion.

As for notebook -> markdown conversion: I don't think that there is a way to represent this in the notebook model. There isn't a way to nest a code cell within a markdown cell (which is what I see this is). As a result, the conversion of your test input would be one way only.

I could add support for this (indented fences) in the markdown -> notebook conversion, but have concerns about my code block matching regex getting much more complex. At some point I'd like to have an external library (likely mistune) doing the markdown parsing, but last time I checked it wasn't simple.

[nthiery]

On Tue, Oct 06, 2015 at 09:10:20AM -0700, aaren wrote:

I think this comes down to supporting an indented fenced code block.
Technically this should be possible on markdown -> notebook conversion.

This would be very much appreciated!

As for notebook -> markdown conversion: I don't think that there is a
way to represent this in the notebook model. There isn't a way to nest
a code cell within a markdown cell (which is what I see this is).

That's my understanding too. I can imagine that there are good reasons
for this design choice, in particular keeping things simple; however --
just my 2 cents -- I see it as a serious limitation of the notebook
model.

As a result, the conversion of your test input would be one way
only.

For my use cases (being able to author structured documents in ReST --
in particular the Sage documentation --, and convert them with [1] to
IPython notebooks for interactive browsing), that's totally fine.

I could add support for this (indented fences) in the markdown ->
notebook conversion, but have concerns about my code block
matching regex getting much more complex. At some point I'd like
to have an external library (likely mistune) doing the markdown
parsing, but last time I checked it wasn't simple.

Ok.

Just being curious -- and slightly off topic -- would it make sense to
extend pandoc with a reader and a writer for the IPython notebook
format to benefit from its infrastructure?

Kind regards,
Nicolas

[1] https://github.com/nthiery/rst-to-ipynb/

[aaren]

@nthiery can you link me to a real example of the input that you would like to process? And corresponding output? I'd like to understand a bit better.

R.e. notebook model: yes I think a tree is a more natural model for the notebook and perhaps that will appear in future formats. The simplicity of the flat model is a major plus though.

R.e. pandoc notebook reader / writer: I've thought about this before. The pandoc document model is nice to work with. I've thought about using pandoc for notedown but it is a heavy dependency.

[minrk]

Feel free to blame the notebook format for not supporting code cells within a bulleted list. We are aware of the issue, but haven't found the complexity trade off to make sense yet. Perhaps in the future when we have fewer other problems to solve.

I think it's ambiguous whether code blocks in bullets should be converted to code cells, since it breaks document formatting. You could, for instance, leave them as code blocks in markdown cells, which would remain perfectly valid. It is not true in general that all code blocks should be converted to code cells.

R.e. pandoc notebook reader / writer: I've thought about this before. The pandoc document model is nice to work with. I've thought about using pandoc for notedown but it is a heavy dependency.

It sure is. I think nbconvert would be a great deal nicer if it were restructured entirely to be pandoc filters/readers/writers, but the dependency is rough for a lot of people, especially on conservative linux environments.