Alternative formulation

Question

Alternative formulation

gvanrossum opened this issue 5 years ago · 10 comments

Let's argue about the details of this version. If the callback needs access to the FL object the caller can pass a callback constructed using functools.partial() or a lambda.

from typing import Callable

CallbackType = Callable[[object, str], str]


class FL:
    __slots__ = ["raw", "call"]

    def __init__(self, raw: str, call: Callable[[CallbackType], str]):
        self.raw = raw
        self.call = call

    def __call__(self, callback: CallbackType = format) -> str:
        return self.call(callback)

    def __str__(self) -> str:
        return self()

    def __repr__(self) -> str:
        return f"FL({self.raw!r}, {self.call!r})"


width = 3.14
height = 42
label = "narrow box"

# Emulate s = fl"W={width:.3f}, H={height:.3f}, area={width*height:.2f}  # {label}"
s = FL("W={width:.3f}, H={height:.3f}, area={width*height:.2f}  # {label}",
       lambda cb:
       f"W={cb(width, '.3f')}, H={cb(height, '.3f')}, area={cb(width*height, '.2f')}  # {cb(label, '')}")

print(s)
print(s(lambda val, spec: repr(val)))
print(repr(s))

Answer 1 · 2020-06-14T21:25:51.000Z

Translation example (without CLDR plural rules):

dutch = {
    "{person} invites {num_guests} guests to their party":
    "{person} nodigt {num_guests} gasten uit op hun feest",
}
languages = {"nl": dutch}

def translate(fl: FL, lang: str, **params: object) -> str:
    langdb = languages.get(lang, {})
    translated = langdb.get(fl.raw, fl.raw)
    return translated.format(**params)

# Note that the lambda here is invalid -- person and num_guests don't exist!
sentence = FL("{person} invites {num_guests} guests to their party",
              lambda cb: f"{cb(person, '')} invites {cb(num_guests, '')} guests to their party")
print(translate(sentence, "en", person="Jim", num_guests=2))
print(translate(sentence, "nl", person="Guido", num_guests=42))

Answer 2 · 2020-06-14T21:28:27.000Z

Hm, my translation example doesn't need FL strings at all. What am I missing?

Answer 3 · 2020-06-14T22:13:13.000Z

Part of the puzzle is the example in PEP 501:

  >>> import datetime
  >>> name = 'Jane'
  >>> age = 50
  >>> anniversary = datetime.date(1991, 10, 12)
  >>> format(i'My name is {name}, my age next year is {age+1}, my anniversary is {anniversary:%A, %B %d, %Y}.')
  'My name is Jane, my age next year is 51, my anniversary is Saturday, October 12, 1991.'

(Again, the idea of using __str__() to render the template was apparently unknown at the time PEP 501 was written.)

So the person and num_guests values shouldn't be passed as keywords to translate(), but should be implicit at the location where the FL-string is created, as if it were an F-string. Rewriting my example:

def translate(fl: FL, lang: str) -> str:
    langdb = languages.get(lang, {})
    translated = langdb.get(fl.raw, fl.raw)
    ???

person = "Guido"
num_guests = 42
sentence = fl"{person} invites {num_guests} guests to their party"
print(sentence)  # English
print(translate(sentence, "nl"))  # Dutch

The question is now what the ??? in translate() should be. Clearly it should call fl.call() with some callback that gathers the formatted values and sticks them into a table, and then call (FIXED:) translated.format(**table). And now we get to the crux of the matter, which is that it needs access to the variable names (or more general: the text of the expressions) used in the interpolations, to be used as keys in that table. Put together:

def translate(fl: FL, lang: str) -> str:
    langdb = languages.get(lang, {})
    translated = langdb.get(fl.raw, fl.raw)
    table = {}
    def callback(value: object, spec: str, text: str) -> str:
        table[text] = value
    fl.call(callback)
    return translated.format(**table)

Also the lambda passed to the FL constructor must pass this extra argument to the callback:

sentence = FL("{person} invites {num_guests} guests to their party",
    lambda cb:
    f"{cb(person, '', 'person')} invites {cb(num_guests, '', 'num_guests")} guests to their party")

I now see that you solved this by adding an index field to the callback signature instead of the text of the expression. That would require the translation framework to parse the raw string looking for interpolations, rather than using my solution. And technically the index can be recovered by counting how often the callback has been called, so we wouldn't need to pass it (so I get my default callback of format back :-).

But clearly we're into bikeshed mode now.

Answer 4 · 2020-06-14T22:34:16.000Z

Complete version:

from typing import Callable, Dict

CallbackType = Callable[[object, str, str], str]


class FL:
    __slots__ = ["raw", "call"]

    def __init__(self, raw: str, call: Callable[[CallbackType], str]):
        self.raw = raw
        self.call = call

    def __call__(self, callback: CallbackType) -> str:
        return self.call(callback)

    def __str__(self) -> str:
        return self(lambda value, spec, text: format(value, spec))

    def __repr__(self) -> str:
        return f"FL({self.raw!r}, {self.call!r})"


width = 3.14
height = 42
label = "narrow box"

# Emulate s = fl"W={width:.3f}, H={height:.3f}, area={width*height:.2f}  # {label}"
s = FL("W={width:.3f}, H={height:.3f}, area={width*height:.2f}  # {label}",
       lambda cb:
       f"W={cb(width, '.3f', 'width')}, "
       f"H={cb(height, '.3f', 'height')}, "
       f"area={cb(width*height, '.2f', 'width*height')}  "
       f"# {cb(label, '', 'label')}")

print(s)
print(s(lambda val, spec, text: repr(val)))
print(repr(s))

# Translation example

dutch = {
    "{person} invites {num_guests} guests to their party":
    "{person} nodigt {num_guests} gasten uit op hun feest",
}
languages = {"nl": dutch}

def translate(fl: FL, lang: str) -> str:
    langdb = languages.get(lang, {})
    translated = langdb.get(fl.raw, fl.raw)
    table: Dict[str, object] = {}
    def callback(value: object, spec: str, text: str) -> str:
        table[text] = value
        return "{}"
    fl(callback)
    return translated.format(**table)

person = "Guido"
num_guests = 42
sentence = FL("{person} invites {num_guests} guests to their party",
    lambda cb:
    f"{cb(person, '', 'person')} invites {cb(num_guests, '', 'num_guests')} guests to their party")
print(translate(sentence, "en"))
print(translate(sentence, "nl"))

Answer 5 · 2020-06-14T22:40:57.000Z

These are some great observations and examples! Also , it's not bikeshedding in my mind if we are simplifying the code and corresponding API. In particular:

We can remove index from the callback API as you mentioned. Any relatively complex callback scheme that uses the raw attribute and needs an index is going to have some sort of corresponding state machine, or equivalently an object to manage this functionality.
Likewise, we don't need to have a list of the computed expression values for a computed formatspec. Since the formatspec is defined by a formal language, it can be parsed. Arguably even this is unnecessary for real usage - either the formatspec is directly used, or it is going to be ignored.

Answer 6 · 2020-06-15T00:00:29.000Z

Sounds like we agree that we can remove self and index from the callback API. Do you also agree that we ought to add text though?

Where in your current proposal is the computed expression? Or is that just from PEP 501's _field_values? And I guess your "computed formatspec" is its _format_specifiers?

Answer 7 · 2020-06-15T05:18:41.000Z

So I had been thinking that raw was sufficient, because the text field can be derived from it. But clearly it's inconvenient for such a common usage.

I suggest we use expr instead of text - I think the term "text" could be confusing to be honest. Some precedent: https://docs.python.org/3/whatsnew/3.8.html#f-strings-support-for-self-documenting-expressions-and-debugging

Re "computed formatspec" - this is the spec, but taking into account any computed expressions. So in this example from PEP 498

>>> width = 10
>>> precision = 4
>>> value = decimal.Decimal('12.34567')
>>> f'result: {value:{width}.{precision}}'
'result:      12.35'

the callback is called once with these args:

callback(value=Decimal('12.34567'), spec="10.4", expr='value')

Answer 8 · 2020-06-15T05:21:59.000Z

One last bit of potential naming bikeshedding. I suggest format_spec, since that's used by both PEP 498 (https://www.python.org/dev/peps/pep-0498/#specification) and the ref doc (https://docs.python.org/3/reference/datamodel.html#object.__format__)

Answer 9 · 2020-06-15T05:52:57.000Z

Thanks for the clarification. 'expr' is fine. But these should be positional-only.

Answer 10 · 2020-06-15T18:40:02.000Z

Agreed about positional only.

The code example is quite clear here, and can be readily modified to use "expr":

def callback(value: object, spec: str, expr: str) -> str: ...

I don't know if it's possible for CallbackType if we can also specify the same signature for it as well.

With that, we can document typical usage, eg something like the following as the call:

callback(Decimal('12.34567'), "10.4", 'value')

(I do think it's helpful in the PEP to show concrete examples like this, much as we see in PEP 498, https://www.python.org/dev/peps/pep-0498/#format-specifiers, from which this exact example is drawn.)