universal-ctags/libreadtags

Do mmap whole tags file

Closed this issue · 3 comments

Do mmap whole tags file

We can start by adding a thin abstraction layer.

[yamato@control]~/var/libreadtags% nm readtags.o | grep U
                 U calloc
                 U __ctype_b_loc
                 U __ctype_toupper_loc
                 U __errno_location
                 U fclose
                 U feof
                 U fgetpos
                 U fgets
                 U fopen
                 U free
                 U fseek
                 U fsetpos
                 U ftell
                 U malloc
                 U memmove
                 U perror
                 U realloc
                 U rewind
                 U strchr
                 U strcmp
                 U strdup
                 U strlen
                 U strncpy
                 U strtol

fclose, feof, fgetpos, fgets, fopen, fseek, fsetpos, ftell, rewind.

fopen of glibc (>=2.3) accepts "m" as a part of mode argument.

       m (since glibc 2.3)
              Attempt to access the file using mmap(2), rather than I/O system
              calls (read(2), write(2)).  Currently, use  of  mmap(2)  is  at‐
              tempted only for a file opened for reading.

Implementing mmap backend in libreadtags, providing the compile-time option to passing "m" will be better.

I observed "m" flag improves the performance.

% ls -lh ~/.citre/upstream-linux.tags 
-rw-r--r--. 1 yamato yamato 3.0G Sep 22 07:01 /home/yamato/.citre/upstream-linux.tags

% time ./readtags-normal -t ~/.citre/upstream-linux.tags  -l > /dev/null 
./readtags-normal -t ~/.citre/upstream-linux.tags -l > /dev/null  14.17s user 1.03s system 99% cpu 15.253 total
% time ./readtags-normal -t ~/.citre/upstream-linux.tags  -l > /dev/null
./readtags-normal -t ~/.citre/upstream-linux.tags -l > /dev/null  15.38s user 1.12s system 98% cpu 16.706 total
% time ./readtags+m -t ~/.citre/upstream-linux.tags  -l > /dev/null
./readtags+m -t ~/.citre/upstream-linux.tags -l > /dev/null  14.13s user 0.39s system 99% cpu 14.569 total
% time ./readtags+m -t ~/.citre/upstream-linux.tags  -l > /dev/null
./readtags+m -t ~/.citre/upstream-linux.tags -l > /dev/null  14.11s user 0.43s system 99% cpu 14.602 total