Poor editor performance with soql query with hundreds of fields

[superfell]

As the number of fields in the soql query grow, the responsiveness of the editor gets slowly worse, at ~300 fields, there's > 100ms delay for each keypress (on Simon's 2015 iMac)

There's 2 parts that are contributing to this.
the resolveTokens step takes anywhere upto 50ms. It's not too bad for any single token, but there's just so many of them. This needs some more work with a profiler.
the applyTokens steps takes anywhere from 50 to 200ms. The main culprit seems to be the addition of the completions as attribute values

if (t.completions.count > 0) {
            [txt addAttribute:KeyCompletions value:t.completions range:t.loc];
}

this call to addAttribute gets slow and slower, by the time it's adding the last of the completions, its ~4ms for that one call.
But the other calls such as the addAttributes:range: to color the field don't exhibit the same behavior.

Some approaches to address this could be

1. store the completions in a custom structure outside of the attributed string, we may be able to optimize the storing/lookup of these given that we don't need to deal with overlaps.
2. diff the tokens and apply the difference. This is surely non-trivial
3. for the resolveTokens problem if the individual calls can't be improved then can we parallelize them? can we keep the previous tokens and just resolve any token over the edited text?