Full soft-wrap implementation
adsr opened this issue · 7 comments
First of all, thanks for this nice editor!
I have written a short introduction to mle here /1/
I have some questions regarding usage:
- Would it be possible to catch S-home key event?
- I would like to insert the content of a file into the currently edited file: currently I load the file into a buffer, copy it, press M-1, then uncut it. Though, once I have cut text in my 1st buffer, then the buffer from M-2 is not available. I guess, I do something wrong here.
- Would it be possible to make mle faster during startup? It seems
bline_count_chars()is a little bit slow, see /1/ - When opening a PHP source code file then leading tabs are shown as underscore. When opening a regular Markdown file with leading tabs, they are shown as space, i.e., no underscore. I wonder why this is.
- Line-wrap has been already mentioned.
Hi, thank you for writing up that blog post. It's better than the official help!
- Would it be possible to catch S-home key event?
For this key press, termbox2 currently understands xterm style ("\x1b[1;2H") and rxvt style ("\x1b[7$"). It's likely your terminal emulator is not sending a distinct sequence for S-home vs home. What terminal are you using? If by chance you're using the macOS Terminal app, I believe there's a way to configure it to emit custom sequences. I helped someone do that a while back.
- I would like to insert the content of a file into the currently edited file: currently I load the file into a buffer, copy it, press M-1, then uncut it. Though, once I have cut text in my 1st buffer, then the buffer from M-2 is not available. I guess, I do something wrong here.
Yes cut/copy buffers are local to current buffer. But there is also a global buffer that contains the last cut/copy performed in any buffer. That command (cmd_uncut_last) is bound to CM-u by default. So (with default key bindings) you can M-2, C-d q (cut all), M-1, CM-u (uncut last).
By the way, to insert the content of a file into a buffer, it may be easier to cat ... it with cmd_shell (M-e).
- Would it be possible to make mle faster during startup? It seems bline_count_chars() is a little bit slow, see /1/
I'm very curious about this. What kind of machine are you running this on? On my machine (Linux 6.x x86_64), mle is still a bit faster than vim latest. It seems more recent versions of vim became more efficient since the last time I tested this! I do see most of the time spent in buffer_set_mmapped but bline_count_chars is near zero. I wonder if this shortcut is being skipped on your setup for some reason. If you care to, can you compile with CFLAGS="-O2 -g -pg", run the large file test, and share output of gprof -l ./mle ./gmon.out? That will do line-by-line profiling with gprof which may hint where it's spending time in that function.
- When opening a PHP source code file then leading tabs are shown as underscore. When opening a regular Markdown file with leading tabs, they are shown as space, i.e., no underscore. I wonder why this is.
mle has 1 built-in syntax called syn_generic which applies to several common source file types including PHP. As a personal preference, I included a rule in it to underline all tabs in red. syn_generic doesn't apply to markdown (*.md) files, so you won't see red tabs nor any other styles. You can define other syntaxes to override syn_generic if you prefer. It's on my backlog to add proper syntax definitions for each language individually. In my mlerc, I do define a markdown-specifc syntax rule as well as some other rules that syn_generic doesn't cover.
- Line-wrap has been #75 mentioned.
Indeed, I have a branch with this that I need to revisit. It will most likely be in the next version.
Thanks for the elaborate answer.
- Indeed, the problem with S-home is unrelated to mle, but is due to
st, the so called suckless terminal. I can verify that the problem does not occur inxterm. - Using the shell looks like the best approach -- I knew that I was missing something ;-)
- Below you'll find the output of
gproffor the smaller and larger file, see /1/ and /2/. I am using Arch Linux with kernel 6.6.2, see /3/ for all the details and mle 1.7.3-dev (mle -v). The test is withmle -NQq, so startup file should be ignored. One note: I tested this on/tmpon Arch Linux, which is essentially a RAM disk, thereby eliminating all harddisk/SSD access. Therefore any advantages, which are only due tommap()vs.read(), will not matter much, see /4/ - I commented line 2179 in
editor.c, and can verify that it's the syntax highlighting that "confused" me.
/1/ "Small" file, i.e., seq 999000
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls Ts/call Ts/call name
66.67 0.04 0.04 bline_count_chars (buffer.c:1076 @ b328)
8.33 0.05 0.01 bline_count_chars (buffer.c:1053 @ b2ac)
8.33 0.05 0.01 bline_count_chars (buffer.c:1056 @ b2b7)
8.33 0.06 0.01 buffer_set_mmapped (buffer.c:370 @ e1ac)
8.33 0.06 0.01 buffer_set_mmapped (buffer.c:378 @ e1bb)
0.00 0.06 0.00 999007 0.00 0.00 buffer_add_mark (buffer.c:224 @ c4f0)
/2/ "Large" file, i.e., seq 9999000
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ns/call ns/call name
51.79 0.29 0.29 bline_count_chars (buffer.c:1076 @ b328)
6.25 0.33 0.04 buffer_set_mmapped (buffer.c:370 @ e1bf)
5.36 0.36 0.03 buffer_set_mmapped (buffer.c:378 @ e1bb)
4.46 0.38 0.03 buffer_set_mmapped (buffer.c:370 @ e1ac)
3.57 0.40 0.02 bline_count_chars (buffer.c:1083 @ b33c)
3.57 0.42 0.02 buffer_destroy (buffer.c:205 @ c610)
3.57 0.44 0.02 buffer_set_mmapped (buffer.c:378 @ e1a9)
2.68 0.46 0.02 9999001 1.50 1.50 bline_count_chars (buffer.c:1044 @ b280)
2.68 0.47 0.02 bline_count_chars (buffer.c:1053 @ b297)
1.79 0.48 0.01 bline_count_chars (buffer.c:1077 @ b300)
1.79 0.49 0.01 bline_count_chars (buffer.c:1085 @ b32c)
1.79 0.50 0.01 bline_count_chars (buffer.c:1076 @ b341)
1.79 0.51 0.01 buffer_add_mark (buffer.c:225 @ c4fe)
1.79 0.52 0.01 buffer_set_mmapped (buffer.c:365 @ e166)
/3/ Uses
/4/ Performance Comparison of mmap() versus read() versus fread()
Interesting. I wonder why bline_count_chars is getting called for every line in the file. It's supposed to be called lazily. With no syntax definition applied I'd expect it to be called only for the lines that are rendered to the terminal. Also unsure about buffer_add_mark for the same reason.
Can you run it through a debugger or perf record -g and see where they are getting called?
Results for small file seq 999000 are as below:
Samples: 275 of event 'cycles:Pu', Event count (approx.): 152924128
Children Self Command Shared Object Symbol
+ 99.16% 0.00% mle libc.so.6 [.] 0x00007f7b22ed2cd0 ◆
+ 99.16% 0.00% mle mle [.] main ▒
+ 77.60% 0.00% mle mle [.] editor_init ▒
+ 77.25% 0.00% mle mle [.] editor_open_bview ▒
+ 77.25% 0.00% mle mle [.] bview_new ▒
+ 50.43% 0.00% mle mle [.] _bview_init ▒
+ 50.43% 0.00% mle mle [.] bview_set_syntax.part.0 ▒
+ 49.42% 0.34% mle mle [.] buffer_set_tab_width ▒
+ 26.94% 25.43% mle mle [.] bline_count_chars ▒
+ 26.83% 0.00% mle mle [.] _bview_open_buffer ▒
+ 26.83% 0.00% mle mle [.] buffer_new_open ▒
+ 26.13% 0.00% mle mle [.] buffer_open ▒
+ 21.56% 0.00% mle mle [.] editor_deinit ▒
+ 20.83% 0.00% mle mle [.] bview_destroy ▒
+ 20.83% 0.00% mle mle [.] _bview_deinit ▒
+ 20.11% 2.47% mle mle [.] buffer_destroy ▒
+ 18.51% 18.51% mle libc.so.6 [.] _mcount ▒
+ 9.55% 9.29% mle mle [.] buffer_set_mmapped ▒
+ 8.15% 8.15% mle libc.so.6 [.] 0x0000000000113ce4 ▒
+ 8.15% 0.00% mle libc.so.6 [.] 0x00007f7b22fbece4 ▒
+ 8.04% 8.04% mle libc.so.6 [.] 0x00000000001577b4 ▒
+ 8.04% 0.00% mle libc.so.6 [.] 0x00007f7b230027b4 ▒
+ 3.98% 3.98% mle mle [.] _buffer_bline_free.isra.0 ▒
+ 3.29% 3.29% mle libc.so.6 [.] 0x0000000000113e3d ▒
+ 3.29% 0.00% mle libc.so.6 [.] 0x00007f7b22fbee3d ▒
+ 3.26% 3.26% mle libc.so.6 [.] 0x00000000001577b9 ▒
+ 3.26% 0.00% mle libc.so.6 [.] 0x00007f7b230027b9 ▒
+ 2.95% 2.95% mle libc.so.6 [.] 0x00000000001577a8 ▒
+ 2.95% 0.00% mle libc.so.6 [.] 0x00007f7b230027a8 ▒
+ 2.53% 2.53% mle libc.so.6 [.] 0x0000000000113d30 ▒
+ 2.53% 0.00% mle libc.so.6 [.] 0x00007f7b22fbed30 ▒
+ 1.54% 1.54% mle libc.so.6 [.] 0x00000000001577bc ▒
+ 1.54% 0.00% mle libc.so.6 [.] 0x00007f7b230027bc ▒
+ 1.51% 1.51% mle libc.so.6 [.] 0x0000000000113cfe ▒
+ 1.51% 0.00% mle libc.so.6 [.] 0x00007f7b22fbecfe ▒
+ 1.06% 1.06% mle libc.so.6 [.] 0x0000000000113d23 ▒
+ 1.06% 0.00% mle libc.so.6 [.] 0x00007f7b22fbed23 ▒
+ 0.83% 0.00% mle [unknown] [.] 0xffffffff914012c6 ▒
+ 0.74% 0.74% mle libc.so.6 [.] cfree ▒
+ 0.74% 0.00% mle mle [.] _editor_destroy_kmap ▒
+ 0.70% 0.00% mle [unknown] [.] 0xffffffff914000ea
Large file seq 9999000:
Samples: 3K of event 'cycles:Pu', Event count (approx.): 1481260200
Children Self Command Shared Object Symbol
+ 99.91% 0.00% mle libc.so.6 [.] 0x00007fc55e698cd0
+ 99.91% 0.00% mle mle [.] main
+ 80.84% 0.00% mle mle [.] editor_init
+ 80.84% 0.00% mle mle [.] editor_open_bview
+ 80.84% 0.00% mle mle [.] bview_new
+ 53.66% 0.00% mle mle [.] _bview_init
+ 53.66% 0.00% mle mle [.] bview_set_syntax.part.0
+ 53.25% 1.09% mle mle [.] buffer_set_tab_width
+ 33.47% 33.35% mle mle [.] bline_count_chars
+ 27.18% 0.00% mle mle [.] _bview_open_buffer
+ 27.18% 0.00% mle mle [.] buffer_new_open
+ 26.40% 0.00% mle mle [.] buffer_open
+ 19.04% 0.03% mle mle [.] editor_deinit
+ 19.01% 0.00% mle mle [.] bview_destroy
+ 19.01% 0.00% mle mle [.] _bview_deinit
+ 18.86% 3.53% mle mle [.] buffer_destroy
+ 12.00% 11.95% mle mle [.] buffer_set_mmapped
+ 10.10% 10.10% mle libc.so.6 [.] _mcount
+ 8.67% 0.00% mle libc.so.6 [.] 0x00007fc55e784ce4
+ 8.64% 8.64% mle libc.so.6 [.] 0x0000000000113ce4
+ 5.63% 0.00% mle libc.so.6 [.] 0x00007fc55e7c87b4
+ 5.61% 5.61% mle libc.so.6 [.] 0x00000000001577b4
+ 5.29% 5.29% mle libc.so.6 [.] 0x0000000000113d30
+ 5.29% 0.00% mle libc.so.6 [.] 0x00007fc55e784d30
+ 3.64% 0.00% mle libc.so.6 [.] 0x00007fc55e7c87a8
+ 3.61% 3.61% mle libc.so.6 [.] 0x00000000001577a8
+ 3.00% 3.00% mle mle [.] _buffer_bline_free.isra.0
+ 2.16% 0.00% mle libc.so.6 [.] 0x00007fc55e7c87b9
+ 2.14% 2.14% mle libc.so.6 [.] 0x00000000001577b9
+ 2.07% 2.07% mle libc.so.6 [.] 0x0000000000113e3d
+ 2.07% 0.00% mle libc.so.6 [.] 0x00007fc55e784e3d
+ 1.40% 1.40% mle libc.so.6 [.] 0x00000000001577bc
+ 1.40% 0.00% mle libc.so.6 [.] 0x00007fc55e7c87bc
+ 1.37% 1.37% mle libc.so.6 [.] 0x0000000000113d23
+ 1.37% 0.00% mle libc.so.6 [.] 0x00007fc55e784d23
+ 0.97% 0.97% mle libc.so.6 [.] 0x0000000000157787
OK I see a bug in _editor_should_skip_rc. Is it faster if you run mle -N -Qq large (instead of mle -NQq large)?
Yes, this change in the command line makes a difference of a factor of more than two!
| File | Command | real time in s |
|---|---|---|
seq 999000 |
time mle -NQq file |
0.08 |
seq 999000 |
time mle -N -Qq file |
0.04 |
seq 9999000 |
time mle -NQq file |
0.82 |
seq 9999000 |
time mle -N -Qq file |
0.36 |
Profiling mle 1.7.3-dev and the "small" files gives:
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
66.67 0.02 0.02 1 20.00 20.00 buffer_set_mmapped
33.34 0.03 0.01 999007 0.00 0.00 buffer_add_mark
0.00 0.03 0.00 2044 0.00 0.00 bytebuf_nputs
0.00 0.03 0.00 919 0.00 0.00 send_attr
0.00 0.03 0.00 349 0.00 0.00 convert_num
0.00 0.03 0.00 335 0.00 0.00 cap_trie_add
0.00 0.03 0.00 275 0.00 0.00 utf8_char_to_unicode
0.00 0.03 0.00 226 0.00 0.00 _editor_key_to_input
0.00 0.03 0.00 181 0.00 0.00 _editor_init_kmap_add_binding
0.00 0.03 0.00 181 0.00 0.00 _editor_init_kmap_add_binding_to_trie
0.00 0.03 0.00 178 0.00 0.00 tb_set_cell
0.00 0.03 0.00 126 0.00 0.00 tb_print
0.00 0.03 0.00 126 0.00 0.00 tb_print_ex
0.00 0.03 0.00 126 0.00 0.00 tb_printf_rect
0.00 0.03 0.00 111 0.00 0.00 editor_register_cmd
0.00 0.03 0.00 66 0.00 0.00 send_cursor_if
Results for small file with perf report:
Samples: 118 of event 'cycles:Pu', Event count (approx.): 75812993
Children Self Command Shared Object Symbol
+ 96.75% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff0b8cd0
+ 96.75% 0.00% mle.gprof mle.gprof [.] main
+ 54.64% 0.00% mle.gprof mle.gprof [.] editor_init
+ 54.09% 0.00% mle.gprof mle.gprof [.] editor_open_bview
+ 54.09% 0.00% mle.gprof mle.gprof [.] bview_new
+ 54.09% 0.00% mle.gprof mle.gprof [.] _bview_open_buffer
+ 54.09% 0.00% mle.gprof mle.gprof [.] buffer_new_open
+ 52.65% 0.00% mle.gprof mle.gprof [.] buffer_open
+ 41.02% 0.00% mle.gprof mle.gprof [.] editor_deinit
+ 41.02% 0.00% mle.gprof mle.gprof [.] bview_destroy
+ 41.02% 0.00% mle.gprof mle.gprof [.] _bview_deinit
+ 38.10% 10.31% mle.gprof mle.gprof [.] buffer_destroy
+ 18.90% 16.98% mle.gprof mle.gprof [.] buffer_set_mmapped
+ 11.47% 11.47% mle.gprof libc.so.6 [.] 0x00000000001577b4
+ 11.47% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1e87b4
+ 10.36% 10.36% mle.gprof mle.gprof [.] _buffer_bline_free.isra.0
+ 9.38% 9.38% mle.gprof libc.so.6 [.] 0x00000000001577a8
+ 9.38% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1e87a8
+ 9.27% 9.27% mle.gprof libc.so.6 [.] _mcount
+ 6.80% 6.80% mle.gprof libc.so.6 [.] 0x00000000001577b9
+ 6.80% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1e87b9
+ 4.42% 4.42% mle.gprof libc.so.6 [.] 0x0000000000113ce4
+ 4.42% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1a4ce4
+ 3.52% 3.52% mle.gprof libc.so.6 [.] 0x0000000000157787
+ 3.52% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1e8787
+ 2.40% 2.40% mle.gprof libc.so.6 [.] 0x0000000000113d20
+ 2.40% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1a4d20
+ 1.50% 1.50% mle.gprof libc.so.6 [.] cfree
+ 1.50% 1.50% mle.gprof libc.so.6 [.] 0x0000000000113d9a
+ 1.50% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1a4d9a
+ 1.44% 0.00% mle.gprof [unknown] [.] 0xffffffffb6e000ea
+ 1.44% 0.00% mle.gprof [unknown] [.] 0xffffffffb6d93470
+ 1.40% 1.40% mle.gprof libc.so.6 [.] 0x0000000000113cdb
+ 1.40% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1a4cdb
+ 1.39% 1.39% mle.gprof [unknown] [k] 0xffffffffb6e01c60
+ 1.36% 1.36% mle.gprof libc.so.6 [.] 0x0000000000113d30
+ 1.36% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1a4d30
+ 1.09% 0.00% mle.gprof mle.gprof [.] editor_run
+ 1.09% 0.00% mle.gprof mle.gprof [.] _editor_loop
+ 1.08% 1.08% mle.gprof [unknown] [k] 0xffffffffb6e012a0
+ 1.07% 1.07% mle.gprof libc.so.6 [.] 0x00000000001577b0
+ 1.07% 0.00% mle.gprof libc.so.6 [.] 0x00007fd1ff1e87b0
+ 0.97% 0.97% mle.gprof libc.so.6 [.] 0x00000000001577bc