keyboard.code() and keyboard.char() clarifications
niksilver opened this issue · 6 comments
A question: What are the intended behaviours of keyboard.code() and keyboard.char()? I am willing to clarify the documenation and make any code changes (if we decide there should be any), but importantly I'd like to understand the intended behaviour of each.
tl;dr - Maybe keyboard.code() should be recommended for non-visible characters only (maybe not); keyboard.char() should be recommended for visible characters only; it would be useful to add a code-to-character converter function, but its logic will depend on the intended logic of keyboard.code().
Observations
Here are some specific observations...
keyboard.code()
This is intended to capture individual key presses. For example, when the key for letter A is pressed then this function will receive the same code regardless of whether the SHIFT key is being pressed to capitalise it. But this function also captures whether the key has gone up or down or is held (via the second parameter being 0, 1 or 2 respectively).
keyboard.char()
This is intended to capture the character generated from key presses, and is only called for visible characters. For example, if the key for letter A is pressed then this function will receive the character A or a depending on whether SHIFT is currently pressed. But this function is only called when the key is pushed down or held. We cannot use this function to see when a key is released.
Both functions
When a key is pushed down, and also when it is held, both the above functions are called.
Key remapping
Keyboards will get remapped for different countries' keyboard layouts. By looking at keyboard.process() we learn that this key mapping happens after keyboard.code() is called, and therefore is only relevant for keyboard.char().
This therefore leads to a question: What should we expect for keyboard.code() if someone presses the key for letter A? Currently, if the user has a US keyboard this function receives code = 30, but if they have a French keyboard it receives code = 16. This kind of difference also exists for non-alphabetic characters (ie *, #, <, etc).
And the reverse is therefore also true. For example, if keyboard.code() receives code = 30 then it's not at all clear what character has been pressed. But we sort-of know the physical placing of the key on the keyboard!
So in general we must presume that all keys for visible characters may get remapped. I'm going to assume that country-specific layouts do not remap non-visible keys (ie ENTER, LEFTSHIFT, F7, etc).
So as it stands keyboard.code() is not good for acting on visible characters unless we allow it to its own remapping by using whatever keyboard layout the user has chosen. It is good for non-visible keys such as LEFTSHIFT, F7, etc.
The "press A for two seconds" problem
Currently we can't write a script which requires the user to (for example) "press A for two seconds" - because we don't have a function which tells us when the A key is pressed and when it's lifted. Maybe this isn't a strong need, but it's a shame to not have the capability.
Conclusions
Alternative 1
Let's assume for the moment that key remapping is happening at the correct place in keyboard.process(). That means we should recommend that script authors use keyboard.code() only for non-visible keys, and use keyboard.char() only for visible characters.
In this situation we could solve the "press A for two seconds" problem by introducing a new public function which does the key remapping - takes a key code and returns the character that the user thinks they pressed. This logic is already in keyboard.process() but it's not exposed in public as a separate function. Even so, there is a wrinkle here. Suppose a key can produce 1 and (when shifted) !. Do we return the raw, unmodified key (ie 1, regardless of any modifiers) or either 1 or ! depending on modifiers? Perhaps we should give the function's user both options.
Also, if we decide we should have this function then keyboard.code() becomes more useful. Users will be able to use the given code for non-visible keys and apply the remapping function if they want a visible character.
Alternative 2
Alternatively we could decide that key remapping should move, and happen before keyboard.code(). In this case keyboard.char() should still be documented as for visible characters only, but the guidance for keyboard.code() will be different. Here we'll say it is for any key and any character.
But here, if keyboard.code() is called with, say code = 30 we still want to know what key is pressed in a human readable form - is it 1 or ! or LEFTSHIFT etc? And - again - we should probably give the option of taking into account modifiers. This function wouldn't be a remapping function, it would be a "code to char" function. This will solve the "press A for two seconds" problem.
One problem with this alternative is there is a chance some scripts would break, because we're changing what codes they get in keyboard.code() - for keys with visible characters. I'm thinking particularly of the amazing zxcvbn. It's possible this would fix bugs more than create them - as suggeted by discussions here and here - but still, changing the logic of an API function is poor practice.
Another problem is a logic mismatch. Key remapping is a process that takes a key + modifiers and returns a character. But if we want to remap for keyboard.code() then we need to take a key, ignore modifiers, and return a code. This isn't the same thing, my head hurts thinking about it, and I worry it could result in really problematic results.
The more I think about these problems the more I dislike this alternative. But maybe you think differently.
Phew
Phew. Apologies for the length of this, and thanks for sticking with me. I think the conclusion of the conclusion is:
- Maybe do key remapping so that it's already done for
keyboard.code(). - Clarify
keyboard.code()andkeyboard.char()documentation so API users know what to expect and when to use them. - May add a function to the module that convert from a code to character, optionally considering modifiers.
Thoughts definitely welcome.
thanks for the detailed analysis. as evidenced by its length, it's not a simple question.
i don't think i have a simple answer but maybe some quick observations.
first here's a link to the probably most relevant bit of keyboard.process():
https://github.com/monome/norns/blob/main/lua/core/keyboard.lua#L141-L154
and a caveat: i added the initial raw HID support, and then all this other stuff was built up on that as different needs arose. so indeed "intention" became a little muddled.
but it seems basically clear that:
- handling keycodes on press/lift is more suited for "musical typing" applications
- handling characters is more suited for actual text entry.
couple more thoughts / suggestions:
-
we could add a
keyboard.char_lift()function that fires on lifts only and sends only the printable character. that would solve the "hold A for 2sec" use case without breaking any APIs. it would also be in line with how other environments like SuperCollider handle key input. -
we could offer a reverse lookup API call for scripts, returning the keycode corresponding to a given character under the selected keymap. (ignoring shift/alt of course.)
i'm a little ignorant on these topics, but i don't know of any way we can really compensate for the fact that different kbs have different physical code layouts.
Many thanks for taking the time to read all of that and for the thoughtful reply. Regarding the "intention" of the APIs - yes, I guess it's often only with use and experience that we get to refine the idea of what we orginally needed or intended. What's important now is how we intend for these functions to be used from now on.
I like your two suggestions for additional functions. I thought I'd summarise the suggestions so far in a little table ("Y" = Yes, "-" = No).
In this table, things we might want to do are:
- VC2. Enable "press [visible character] for 2 seconds" Eg, "Press
*for two seconds". - NK2. Enable "press [non-printing key] for 2 seconds" problem. Eg, "Hold
SHIFTfor two seconds". - COCH. Translate from key code to visible character or non-printing key, perhaps taking into account any modifier keys. Eg, in
keyboard.code()see if the user has hitAorF2. - CHCO. Translate from visible character or non-printing key to key code. Eg, in
keyboard.code()see if what the user has pressed is the key forAor is the key forF2. This is a different way to do the thing in the previous point.
The function names below are just placeholders.
| VC2 | NK2 | COCH | CHCO | |
|---|---|---|---|---|
| Current functions only | - | Y | - | - |
code_to_char() & current |
Y | Y | Y | - |
char_to_code() & current |
Y | Y | - | Y |
char_lift() & current |
Y | Y | - | - |
What this table doesn't show is that different functions have different ease of use. In particular, VC2 and NK2 are both possible with any one of the last three functions alone (plus the current functionality), but some of those suggested functions will make it easier or harder depending on the need and any surrounding logic. So I think there's a case for having all of them.
I'm assuming that while something like char_to_code() could be called with, say, char_to_code("A") but also be called with char_to_code("F7"). So it would be useful for printable characters and non-visible keys.
(By the way, I am now ignoring my "Alternative 2", which involved changing when the key remapping happens in keyboard.process(). The more I think about it the less I like it; I'm sorry I even introduced the idea.)
i'm up for additional functions in the keyboard API, though i haven't surveyed the current usage of the API in user scripts so i'm wary of breaking changes.
i do think the functionality you described would be appreciated. @p3r7 has been a contributor to the current state of keyboard handling.
hey!
my contribution on this part of the system has only been the introduction of the international layout feature.
i also thought about similar use-cases at the time but didn't want to embark too many changes at once as it was already a big chunk (involving re-coding a good chunk of the devices menu).
from my experience porting several "tricky" scripts (nisp, orca...) to the keyboard lib i'd vouch for the code_to_char approach.
as far as documentation goes, there would be 2 main use cases:
- basic text input: use
keyboard.charfor imputing characters & check for modifier keys using keyboard.state.. usekeyboard.codefor DELETE/ENTER. - keyboard as "buttons" (aka any key is a "modifier"): use
keyboard.code+code_to_char.
Thanks, @p3r7. (And there is no "only" about the introduction of the international layout feature!)
I think the consensus then is (a) some kind of code_to_char() function, and (b) documenting use of keyboard.char() and keyboard.code() as you indicate.
I will create a pull request for consideration and link it to this issue. I will include a char_to_code() for symmetry, but will leave out char_lift() for the moment. We can add it (and remove char_to_code()) after discussion, if we feel the need.
Thanks, all, for your help so far.