Remove Null?

[jgm]

We have a Block constructor Null. I think this was added before we were using Builder in the readers; I wonder if there's any reason for it to exist. Note that there's nothing like this for Inline.

[jgm]

I decided to wait on this until it can be packaged with other breaking API changes. Otherwise this causes too much disruption for what it gains us.

[nichtich]

Null is useful in filters that remove block elements by replacing them with a Null element. Pandoc should not throw an error when a Null element is found in JSON input, but silently ignore it.

[jgm]

@nichtich I guess there are two potentially separate issues here:

• should the be a Null constructor in the Haskell type Block?
• in reading the JSON serialization, what should be done with JSON null elements?

We could, in principle, ignore JSON nulls without having a Null constructor in Block (though there may be issues of detail I'm not thinking of, especially since we're using an automatically derived FromJSON instance).

[jgm]

Note also that one can always write a filter that is capable of removing Blocks by making the type of the underlying transformation [Block] -> [Block]; in fact I often use a Block -> [Block] and promote it to [Block] -> [Block] using concatMap.

[nichtich]

should the be a Null constructor in the Haskell type Block?

No, there is no need to have it.

in reading the JSON serialization, what should be done with JSON null elements?

They can be ignored but should be documented. Right now the JSON format is only described implicitly by Haskell types, so removal would also make existing AST structure invalid. Confusion such as #75 (table format changed two times from 1.20 to 1.21 to 1.22) and any breaking of backwards compatibility should be avoided.

[jgm]

That's true, explicitly ignoring null in parsing JSON would keep a degree of backwards compatibility for JSON filters that use Null. (As long as they aren't written in Haskell using the Null constructor.) Doing this, however, might cause a lot of additional code complexity if it means not using automatic derivation of the FromJSON instance. In that case it may not be worth it -- and maybe we should just keep Null, as ugly but harmless.

[zkamvar]

What is the appropriate replacement pattern for using pandoc.Null()? I don't speak enough Lua to understand what this means:

Note also that one can always write a filter that is capable of removing Blocks by making the type of the underlying transformation [Block] -> [Block]; in fact I often use a Block -> [Block] and promote it to [Block] -> [Block] using concatMap.

The solution I've come up with is to replace pandoc.Null() with pandoc.RawBlock("text", "") as in:

::::: removeme

This should disappear

:::::

::::: keep

This should stay

:::::

Div = function(el)
  v,i = el.classes:find("removeme")
  if i ~= nil then
    return pandoc.RawBlock("text", "")
  else
    return el
  end
end

Is this correct?

[jgm]

In a Lua filter you can return {} and it will just omit the element.