/kb-layout-evaluation

Evaluate ergonomic keyboard layouts over multiple languages

Primary LanguagePythonMIT LicenseMIT

Keyboard layout evaluation

An evaluation of existing keyboard layouts over multiple languages, focused on ergonomic keyboards.

Many keyboard layouts are designed (by hand or generated by an algorithm) to improve the ergonomics of Qwerty. However, they are typically assessed for typing in a single language, and on a standard keyboard. This analysis evaluates those layouts over several languages, and for an ergonomic keyboard.

The method uses statistics of bigram use (sets of 2 letters) for each language, and grades them according to subjective "weights" (depending on the keys positions), to calculate a comparative difficulty between layouts.

It is focused on writing text, not code. Programming requires special characters more suited to an additional layer; this script focuses on the base layer. Also, no attempt was made to generate a new "optimum" layout, as there would be a lot of variation depending on subjective parameters. Existing layouts already perform very well here, including some generated ones.

The results show that any alternative layout gives a significant ergonomic advantage over Qwerty. Several options give good results, in particular Colemak DHm which also brings good familiarity, accessible shortcuts, and positive user feedback.

results

This project has also been modified and used here.

Table of contents

Character statistics

Contained in folder character_stats.

The layout evaluation needs bigram frequencies (sets of 2 letters) for each language.

The frequencies come from Practical Cryptography for English, French, Spanish, German, and Swedish; and from Norvig for English (to compare).

For comparison, my own corpus is also analyzed (for English and French); made of my emails, some texts from free books, and some internet articles.

count.py

Requirements: Python 3, Pandas.

The script count.py takes the text files in the data folder and outputs the character counts in chars.csv, and the bigram counts in bigrams.csv. Upper case is converted to lower case.

The list of characters to take into account is configurable in the code, in the list chars. Currently it takes the basic alphabet, plus éèêàçâîô.,-'/.

The provided chars.csv and bigrams.csv files were generated with a personal corpus of emails (mails_en and mails_fr, 300~400kB of raw text each) and various free books and articles (vrac_en and vrac_fr, 200~400kB each).

Spreadsheet analysis

This analysis is done in the Libreoffice spreadsheet stats.ods.

Character counts

The character frequencies for both English and French are quite consistent between the sources and my own corpus.

chars_en

Here is the same chart for French.

Bigram counts

The bigram counts show more discrepancies. The charts below show the top 80 bigrams (sorted by the average of both sources for English).

bigram_en

Here is the same chart for French.

Punctuation

The "theory" numbers from Practical Cryptography and Norvig do not contain data on punctuation characters such as .,-'/. However while the use of . and , depend a lot on each author's style, taking into account a non-zero frequency is essential due to their frequency.

The relevant bigram frequencies are not present in any existing statistics, but for French the Bépo layout project gives character frequencies from a Wikipedia 2008 dump.

For English, Vivian Cook's analysis gives frequencies by word. This University of Maryland paper (Table 1) gives a frequency for . of 1.151% (from Google Ngram data).

The frequencies from the personal corpus contain those characters so it can be used as a control. However, it is useful to have more realistic "theory" results. The approach is to "fix" the missing frequencies by copying the relevant ones from the personal corpus and normalizing the column.

To check that those inputs make sense, they can be compared to the few data available.

For English, we can estimate by dividing the frequencies from Vivian Cook by the average word length of 4.79 letters (from Norvig).

For English Personal corpus Frequency / Word length Google Ngram
Period . 1.6 % 1.36 % 1.151 %
Comma , 1.2 % 1.29 %
For French Personal corpus Wikipedia dump
Period . 1.2 % 0.83 %
Comma , 1.5 % 1.02 %

Some variation can be observed, but the numbers from the personal corpus pass that sanity check. It seems a bit more punctuation is used compared to the average literature, which isn't bad to consider.

Takeaway

For the evaluation, the "theory" numbers will be the average from both sources for English, and the only source I have for other languages. But the differences with my own corpus show the sensitivity of those inputs, therefore the results should be taken with some tolerance.

As the "theory" numbers do not contain characters such as .,-'/, those will be copied from the personal corpus data then normalized.

Layout evaluation

Contained in folder layout_evaluation.

Focus definition

The influence of the physical keyboard is on the weights and penalties. The algorithm is more or less the same otherwise.

The chosen weights are for an ergonomic keyboard, so they are symmetrical. It would be similar for any non-staggered keyboard (no horizontal shift between rows).

Only the keys on the main 3×12 matrix are taken into account, which are reachable by a finger easily. The "numbers" row is ignored as we focus on the alphabetical layout.

Thumb keys available on ergonomic keyboards are ignored as I arbitrarily prefer not to place any alphanumeric character on them.

layout_matrix

The keys are designated by a code (hand, row, column). The numbering includes space for some currently-unused keys, in case of evolution (like for an Ergodox-like keyboard).

Evaluation principle

For each language, the bigram frequencies are imported from the character statistics, as a percentage of use.

The principle is to assign a difficulty (weight) to a bigram (two keys typed consecutively). The bigram weight is multiplied by its frequency, and all the results are summed up to get a general difficulty value of the layout.

Weightlayout = sum( Weightbigram × Probabilitybigram )

The bigram weight is composed of:

  • The weights assigned to the two keys, representing the relative difficulty to push them individually
  • A penalty, representing the added difficulty of pushing those 2 keys one after the other

Weightbigram = Weightkey1 + Weightkey2 + Penaltykey1 & key2

The results for all layouts and languages are finally normalized compared to Qwerty in English (at 100%).

Key base weights

The base weights are shown below. The home row is identified by a red border.

They represent the relative difficulty to hit a single key. The proposed values are for an ergonomic keyboard, with vertical columns and a comfortable home row position.

weights

Penalties

The penalties represent the additional difficulty of hitting 2 keys consecutively. They come on top of the base weight. They are only taken into account if the 2 keys are hit by the same hand (and are not the same key, like "aa"). By default, they are given a slightly positive value in order to favor hand alternation.

Generally, the penalties are higher if the same finger is used, and the more rows separate the 2 keys. They can be negative if the relative position makes the motion easy, such as a close "inward roll" (like "sd" on Qwerty).

First finger Second finger Same row 1 row jump 2 rows jump Comment
Index Index 2.5 3.5 4.5 Same finger
Index Middle 0.5 1.0 2.0
Index Ring 0.5 0.8 1.5
Index Pinky 0.5 0.8 1.1
Middle Index -1.5 -0.5 1.5 Inward roll
Middle Middle N/A 3.5 4.5 Same finger
Middle Ring 0.5 1.0 2.0
Middle Pinky 0.5 0.8 1.5
Ring Index -1.5 -0.5 1.5 Inward roll
Ring Middle -2.0 -0.5 1.2 Inward roll
Ring Ring N/A 3.5 4.5 Same finger
Ring Pinky 1.0 1.5 2.5
Pinky Index -1.0 0.0 1.0 Inward roll
Pinky Middle -1.0 0.0 1.5 Inward roll
Pinky Ring -1.0 0.0 1.5 Inward roll
Pinky Pinky 3.0 4.0 5.5 Same finger

Limits

Bigram frequencies variations

The results are approximate as the bigram frequencies aren't a precise and objective number for everyone.

However, the results for English and French show very little difference between the "theory" values and my personal corpus. Therefore it seems the variation in bigram use doesn't affect the final grade very significantly.

Accented characters

The results for languages outside English are slightly off because most accented characters are not taken into account.

Currently, the ignored characters are êàçâîôñäöüß/å, mainly because those characters are absent from most considered layouts. The characters é and è were added manually to the layouts (on unused keys, on the vowel side if there's one) because I particularly care about French, and due to their high frequency (2.85%).

The characters ' and - were also added when missing, on unused keys.

The issue mainly affects German (äöüß, 1.56% of the characters), Swedish (äöå, 4.45%), but also French (êàçâîô, 0.75%) and Spanish (ñ, 0.22%).

To mitigate this, the bigram frequencies are normalized after removing the ignored characters, so the summed grade is still calculated over 100%.

script.py

Requirements: Python 3, Pandas, Matplotlib.

script.py uses the bigram statistics from stats.csv, and config.txt (key weights, penalties, and layouts definitions) to generate the results (table and plot).

To customize the script, edit config.txt and have a look at the main() function.

The code isn't very efficient as it iterates through dataframes to generate the results. In practice it executes in ~10s so it doesn't really matter.

Results

Here are the full results.

results

Results for English only.

results

Results for English only sorted by no punctuation.

results

Average of English results.

results

Average of all results.

results

Here is the complete results list. The layouts can be seen in config.txt.

Layout English English perso English No Punctuation English Avg French French perso Spanish German Swedish Full Avg
APTAP Final 63.23 62.20 62.20 62.543333 65.27 65.64 65.48 63.55 65.28 64.461667
APT 63.50 62.43 62.47 62.800000 65.84 66.41 65.94 63.26 65.79 64.783000
Colemak DHv 64.04 63.75 62.94 63.576667 63.78 63.53 63.28 64.65 67.05 64.442333
CTGAP 2.0 64.17 62.92 63.13 63.406667 66.02 66.49 66.76 65.69 67.05 65.832333
MTGAP Mod2 64.23 62.88 63.80 63.636667 65.92 65.99 66.32 67.33 67.56 66.160333
MTGAP 64.26 62.91 63.80 63.656667 66.21 66.29 66.32 67.33 67.56 66.223333
CTGAP 64.36 63.56 63.85 63.923333 65.06 64.45 65.12 66.57 67.90 65.653667
BEAKL 19bis 64.41 63.36 63.97 63.913333 65.14 64.31 64.62 68.10 69.18 66.107667
Colemak DHm 64.45 64.08 63.61 64.046667 64.33 64.30 64.35 64.87 68.34 65.184333
MTGAP Mod1 64.46 63.17 64.03 63.886667 66.73 67.09 67.26 67.72 67.65 66.685333
Engram 2.0/Engram Mod 64.47 63.17 63.78 63.806667 67.66 67.41 66.70 69.30 69.65 67.398333
Heart 64.60 64.05 63.86 64.170000 69.00 69.27 67.63 67.47 67.16 67.113000
ColemaQ 64.66 64.18 63.58 64.140000 64.56 64.49 64.35 64.79 68.08 65.177000
APTAP Initial 64.70 63.70 63.13 63.843333 67.22 67.82 66.94 63.66 65.23 65.438667
The-1 64.70 63.06 63.49 63.750000 69.47 69.17 69.15 67.71 70.45 68.076000
APTv3 64.70 63.56 64.08 64.113333 65.99 66.58 66.74 64.85 66.03 65.603667
ColemaQ-F 64.75 64.37 63.70 64.273333 64.53 64.59 64.16 64.36 67.62 64.994667
Rsi-terminated 64.81 64.21 64.11 64.376667 64.56 64.21 63.67 66.20 67.64 65.254333
MTGAP 2.0 64.83 64.34 63.46 64.210000 63.83 62.94 64.18 65.15 66.95 64.775000
Colemak Qi x 64.87 64.54 63.84 64.416667 64.93 64.53 64.68 64.44 67.98 65.249333
MTGAP "ergonomic" 64.90 64.85 63.83 64.526667 65.94 65.98 64.98 66.41 66.90 65.755333
Hieamtsrn 64.90 64.07 64.48 64.483333 67.00 66.28 66.03 67.42 68.48 66.610667
ISRT 64.93 64.22 64.22 64.456667 63.76 63.03 63.27 65.07 65.72 64.382333
Balance-12 64.94 64.14 64.51 64.530000 67.02 66.21 65.97 67.29 68.67 66.615000
White 65.01 63.88 64.35 64.413333 70.24 71.06 68.27 66.68 69.73 67.948667
hyperroll 65.01 64.20 64.51 64.573333 66.40 65.56 66.49 68.09 69.26 66.878667
Colemak Qi 65.02 64.78 64.26 64.686667 65.40 65.12 65.69 64.80 68.45 65.777333
Kaehi 65.36 63.95 64.50 64.603333 66.10 65.62 66.09 68.01 68.12 66.536667
Arensito 65.39 64.92 64.82 65.043333 65.03 64.21 66.23 65.81 69.08 66.156667
x1 65.41 64.61 64.96 64.993333 66.33 66.34 66.45 68.20 68.09 66.813667
Colemak DHk 65.51 65.07 64.68 65.086667 65.82 65.77 65.84 65.61 68.64 66.194333
Workman 65.62 65.28 64.82 65.240000 67.82 67.61 67.02 67.11 71.50 67.717000
rollmak 65.67 65.41 64.67 65.250000 64.89 65.45 64.59 64.78 67.96 65.550000
Soul mod 65.69 65.31 64.84 65.280000 65.79 65.33 64.88 64.55 68.55 65.764000
Capewell Mod 65.75 65.45 64.71 65.303333 65.53 64.71 65.97 67.04 67.80 66.246667
MTGAP "standard" 65.76 65.14 64.69 65.196667 65.11 64.57 64.60 66.95 66.91 65.699333
APTv3 Angle Mod 65.78 64.80 65.17 65.250000 67.32 67.40 68.46 65.53 65.60 66.440000
Capewell 65.84 65.68 64.79 65.436667 65.47 64.62 66.02 67.06 67.95 66.302333
full-maks-minimo 65.84 65.10 64.78 65.240000 67.87 67.55 68.45 67.98 67.46 67.368000
ARTS 65.87 65.18 64.95 65.333333 68.23 68.53 68.94 65.51 68.50 67.332667
BEAKL 19 65.89 64.89 64.92 65.233333 66.21 64.90 66.30 67.17 69.04 66.659667
MTGAP "shortcuts" 65.94 65.49 64.91 65.446667 64.44 64.14 62.88 65.61 66.52 64.949333
boo 66.13 65.39 65.22 65.580000 66.84 66.14 65.55 67.74 68.77 66.826000
BEAKL 19 Opt French 66.23 65.13 65.49 65.616667 65.81 64.69 66.02 65.48 69.50 66.373333
Oneproduct 66.32 65.93 65.55 65.933333 69.27 69.06 68.25 68.63 69.40 68.275667
stndc 66.37 66.21 65.46 66.013333 68.81 70.25 66.51 65.30 66.04 66.678667
Uciea 66.39 66.08 65.94 66.136667 65.90 65.79 65.19 66.16 68.50 66.366333
Hands down 66.43 65.97 65.68 66.026667 65.05 63.98 66.27 63.31 66.83 65.390333
ISRT angle mod 66.48 65.89 65.79 66.053333 64.89 64.22 64.72 66.57 67.32 65.843667
Colemak Angle Mod ANSI 66.64 66.77 65.81 66.406667 65.03 64.37 65.42 66.84 68.28 66.329333
MTGAP "Easy" 66.68 66.32 65.88 66.293333 65.31 64.43 64.72 65.15 65.52 65.310667
Colemak Angle Mod ISO 66.70 66.82 65.87 66.463333 65.09 64.44 65.77 67.19 68.32 66.501667
Hands Down 66.71 66.46 65.78 66.316667 63.48 62.16 65.33 65.44 68.37 65.655333
Colemak 66.96 66.87 66.14 66.656667 64.88 64.08 65.54 67.95 69.01 66.727333
Colman 67.12 66.40 66.28 66.600000 68.59 68.26 69.22 69.66 72.89 69.359000
uciea-pcrw 67.19 67.09 66.78 67.020000 66.25 65.97 66.24 66.79 68.98 67.028000
Niro mod 67.22 67.18 66.54 66.980000 67.10 66.54 66.76 67.88 70.60 67.808000
uciea-gcsv 67.36 67.16 66.90 67.140000 67.02 66.44 66.73 68.27 71.58 68.090000
BEAKL 15 67.37 66.90 66.66 66.976667 67.61 66.33 66.82 69.55 68.77 67.817333
Halmak 67.76 67.25 66.94 67.316667 67.85 67.49 67.04 69.78 70.86 68.533333
APTx 67.87 67.50 66.96 67.443333 69.54 69.15 69.06 66.48 70.52 68.569667
SIND 67.92 67.56 67.26 67.580000 67.88 69.32 67.18 69.57 68.62 68.310000
Rolll 68.02 67.15 67.49 67.553333 69.67 69.55 71.10 67.92 71.77 69.590667
Three 68.12 67.39 67.19 67.566667 70.35 70.03 69.64 71.19 71.14 69.945333
Norman 68.17 67.54 67.51 67.740000 70.92 69.92 71.43 70.05 72.29 70.386000
Optimal-trigram 68.44 68.13 67.59 68.053333 67.01 67.42 65.66 70.07 67.81 67.761667
Typehack 68.49 68.14 67.99 68.206667 67.38 67.30 68.75 68.18 66.92 67.879333
Real 68.59 68.31 67.78 68.226667 67.59 66.90 67.97 66.08 66.83 67.270333
Semimak JQ 68.59 68.64 67.86 68.363333 68.55 69.22 68.33 64.81 66.89 67.455667
Pine 68.63 68.08 67.44 68.050000 69.63 70.24 67.42 69.27 70.64 69.063000
Semimak 68.66 68.66 67.93 68.416667 68.41 69.23 68.39 64.92 67.12 67.533333
ASSET 68.70 68.08 68.03 68.270000 66.00 64.69 66.87 70.54 71.21 68.447000
sertain 68.72 68.36 67.86 68.313333 70.81 70.89 69.93 68.16 70.47 69.544667
Notarize 69.28 68.64 68.67 68.863333 67.46 66.06 67.86 69.40 68.11 68.198667
RTNA 70.00 69.92 69.78 69.900000 73.29 73.11 72.32 69.69 73.36 71.694000
Capewell-Dvorak 70.03 69.08 69.49 69.533333 74.36 73.78 74.55 71.10 76.19 73.088667
Optimal digram 70.40 69.96 69.82 70.060000 68.65 68.46 69.12 70.39 70.29 69.683000
Whorf 70.48 70.88 69.35 70.236667 73.67 74.09 70.48 69.69 72.09 71.275333
FLAW 70.56 69.97 69.73 70.086667 73.63 74.19 73.18 72.85 74.61 72.927333
qgmlwyfub 70.68 70.22 70.04 70.313333 71.86 72.89 70.43 72.35 73.91 71.875667
Dwarf 70.72 70.88 69.82 70.473333 72.08 71.92 72.05 68.42 70.60 70.708667
qgmlwbyuv 70.87 70.53 70.29 70.563333 72.24 73.41 71.05 74.19 74.72 72.669667
g-ansi-maks 70.93 70.68 70.78 70.796667 70.82 70.85 72.58 69.10 72.93 71.248333
Dvormax 71.05 70.50 70.92 70.823333 68.32 69.06 69.18 70.90 73.21 70.560667
Klausler 71.29 70.37 70.65 70.770000 73.26 72.25 74.27 73.34 74.15 73.057000
Qwpr 71.52 71.11 70.97 71.200000 70.07 70.23 69.62 73.26 71.88 71.222000
c-qwerty n 71.77 71.62 71.22 71.536667 69.49 68.15 70.26 73.02 75.48 71.823333
ADNW-english-bigram 72.07 71.50 71.89 71.820000 70.26 70.19 71.04 71.33 73.05 71.493000
Coeur 72.27 72.43 71.86 72.186667 64.89 63.67 67.25 71.23 73.67 69.723333
Bepo keyberon 72.59 72.50 72.00 72.363333 66.58 65.30 68.71 72.26 73.69 70.592667
Minimak-8key 72.77 72.17 72.08 72.340000 71.48 70.74 72.13 75.40 73.74 72.944000
qgmlwbyuob 72.79 71.68 72.16 72.210000 76.36 77.29 75.66 72.42 74.02 74.227000
404 72.91 72.26 72.12 72.430000 69.41 68.80 67.81 71.90 73.67 70.983000
Bepo 40% 73.13 73.11 72.55 72.930000 66.61 65.35 68.72 73.29 73.76 70.936000
Dvorak 73.59 72.61 73.16 73.120000 74.98 73.39 75.54 74.79 78.74 75.275000
Neo 76.19 75.36 75.42 75.656667 73.48 72.95 74.75 71.81 74.35 73.956333
Qwkrfy 78.45 77.54 77.91 77.966667 75.61 76.30 73.80 82.63 78.04 77.678333
abKey 78.54 78.64 76.82 78.000000 77.43 78.68 74.23 79.46 79.87 77.923000
c-qwerty 79.35 78.61 79.09 79.016667 77.08 75.57 78.21 84.85 84.70 80.620333
qwerf 81.27 80.42 80.90 80.863333 79.97 78.82 79.04 87.53 84.61 82.287667
Engram 82.88 80.94 83.16 82.326667 88.13 88.39 89.56 89.71 95.44 89.059333
Foalmak 83.03 81.04 82.78 82.283333 86.24 84.86 82.64 87.82 87.67 85.192667
c-qwerty 0.5 87.58 86.71 87.26 87.183333 85.42 83.16 84.04 89.88 91.07 87.292667
Qwertz 98.70 97.76 98.69 98.383333 96.16 96.03 93.91 98.57 95.55 96.501667
Qwerty Angle Mod 99.68 98.93 99.64 99.416667 96.55 96.58 92.79 100.24 95.76 96.954333
Qwerty 100.00 99.15 99.97 99.706667 96.71 96.87 92.69 100.00 96.33 97.103333
Azerty 105.45 104.62 105.74 105.270000 100.91 101.33 102.67 103.08 102.99 103.026000
onehand 2 107.40 107.81 107.58 107.596667 108.35 110.96 111.29 110.60 105.73 108.974333
ABCDE 107.44 107.75 107.49 107.560000 106.34 106.70 107.32 103.63 105.00 106.006000
onehand 115.82 115.61 115.77 115.733333 118.12 119.76 119.28 108.36 110.73 114.608667
TNWMLC 129.02 130.26 129.61 129.630000 125.43 125.71 125.64 122.20 125.92 125.792000
Vitrimak Ortho 138.89 140.48 140.08 139.816667 130.61 132.99 132.36 138.38 136.83 135.837333
Vitrimak 139.77 142.06 140.94 140.923333 132.75 134.50 134.62 133.69 133.50 135.271667

Conclusion

All the alternative layouts perform very significantly better than the traditional ones (Qwerty and similar). The biggest interest for ergonomics is using an alternative layout at all, the selected choice matters a lot less.

results

The options that perform the best are some Colemak variants like DHm (image below), the obscure BEAKL 19bis, the Engram, as well as the layouts generated by Mathematical multicore (MTGAP).

Colemak DHm

In conclusion, the Colemak DHm layout is particularly recommended for its good results and better shortcuts access, familiarity with Qwerty, and larger positive user feedback.