Feature request: environment that expands LaTeX macros before writing python code

[matterhorn103]

Hi!

I am basically trying to do the same thing as the user on SE here:
https://tex.stackexchange.com/questions/515463/pythontex-using-latex-macros-in-pyblock?newreg=08b9004544364c2683c20a8737bc7586
i.e. pass the output of a LaTeX macro to the python side.

You say in your response on SE that it would be possible to make an environment that expands macros and passes the results to the python code. It is also listed as a possible future development in the documentation. Did you ever look into it? Is it possible that something like that could be implemented?

Using \pyc like in your comment also doesn't work for me.

I also assume that implementation of a new environment like this would require TeX coding rather than python coding, and would thus not be something I could implement myself?

Many thanks.

---

Purely for the sake of completeness, some info on what I am trying to do:

I am using the package chemnum, which generates a list of key:value pairs. The value ("label") for a key ("id") can be accessed within LaTeX using the macro \cmpd{key}, but I want to pass the pairs to pythontex, manipulate them, and then generate LaTeX code after manipulation. Specifically, I want to order the keys/IDs in ascending order of their values/labels. I think this would be a challenge in TeX, and I certainly don't know how to do it - but I do know how I would do it in python. To do this I would use the following code (of which the first block is the relevant one):

\begin{pycode}
dict1 = {}
for item in some_list:
    dict1[item] = \cmpd{item}
\end{pycode}

\begin{pycode}
dict2 = {value: key for key, value in dict1.items()}
for label in range(500):
    if label in dict2:
        print(fr"\input{{chapters/cmpds/{dict2[label]}}}")
\end{pycode}

So if the key value pairs are {"216": 2, "450": 1, "378": 3} output would be for example:

\input{chapters/cmpds/450}
\input{chapters/cmpds/216}
\input{chapters/cmpds/378}

It seems this is not possible currently with pythontex. In the linked StackExchange question you suggest that the one-line pythontex structures do indeed expand macros but even if I replace my first pycode block with the suggested code and remove the loop it still doesn't work:

\pyc{dict1["450"] = """\cmpd{450}"""

Instead, the value in the dictionary for the key "450" is just "\cmpd{450}", not the result of that macro, which would be 2.

[gpoore]

An environment probably wouldn't help a lot in this case. Something like

\begin{pycode}
dict1 = {}
for item in some_list:
    dict1[item] = \cmpd{item}
\end{pycode}

would not work with such an environment, because the environment contents would be passed to Python with LaTeX macro expansions, and \cmpd{item} does not exist. That is, it is impossible to have a loop in Python that assembles macros and then gets their values. The closest approximation to this would be to create \pyc commands in a Python loop, and then print them back to LaTeX, and then LaTeX will send them back to Python, and then Python can pick them up on the next run. But that get complicated fast, although it can work with careful design.

Your \pyc code isn't working because of the way that \cmpd is designed. It doesn't simply expand to another value; rather, it does additional LaTeX things in the background, and all of this interferes with it being expanded inside \pyc (it's probably robust/protected/etc.). I'd suggest trying \cmpdplain or one of the other plain commands instead. So maybe something like \pyc{dict1["450"] = """\cmpdplain{450}"""}. If you need to send a lot of these from LaTeX to Python, you'll probably want to write a loop in LaTeX, and then use this sort of \pyc command inside the loop. There's a lot of ways to do this; you might look at the pgffor package to get started.

[matterhorn103]

Good point, that hadn't occurred to me. Clearly the loop could never work as I envisaged. Thanks for the quick response.

I did try \cmpdplain as well as you describe, but what gets passed to python is not the result of \cmpdplain but rather the string """\cmpdplain{450}""" itself. Even though it gives a simple string as an output in LaTeX, perhaps even \cmpdplain is what you describe as "protected etc.".

Still, I can imagine a general environment where macros are expanded first might be useful for various users in order to pass data and variables to the python side. I guess it remains unsupported though?

[gpoore]

It looks like \cmpdplain needs more processing/expansion. You may want something like this:

\documentclass{article}
\usepackage{chemnum}
\usepackage{pythontex}

\makeatletter
\def\cmpdtopy#1{%
  \edef\cmpdtopy@tmp{\cmpdplain{#1}}%
  \expandafter\cmpdtopy@i\expandafter{\cmpdtopy@tmp}{#1}}
\def\cmpdtopy@i#1#2{%
  \pyc{dict1["#2"] = """#1"""}}
\makeatother

\begin{document}

\cmpd{x}
\cmpd{450}


init:

\begin{pycode}
dict1 = {}
\end{pycode}

to py:

\cmpdtopy{450}

value:

\begin{pycode}
print(f'literal:  \\detokenize{{{dict1["450"]}}}')
print(f'LaTeX:  {dict1["450"]}')
\end{pycode}

\end{document}

There is still no support for an environment that expands macros. It would definitely be useful, and should be possible for simple cases. I've had limited time for working on LaTeX, and am currently trying to decide if I want to replace PythonTeX with a largely compatible LaTeX interface to Codebraid, which is like PythonTeX but has more code execution capabilities and currently only supports Pandoc Markdown and not LaTeX.