HarrisonGrodin/Rewrite.jl

Term printing is not unicode compatible

MasonProtter opened this issue · 0 comments

Currently, the printing of Terms assumes that the string representation of the term may be sanely indexed which is not actually the case whenever there is unicode in the Term.

MWE:

julia> using Terms

julia> @term 
Error showing value of type Term:
ERROR: StringIndexError(":(@term :ω)", 11)
Stacktrace:
 [1] string_index_err(::String, ::Int64) at ./strings/string.jl:12
 [2] getindex at ./strings/string.jl:248 [inlined]
 [3] show(::IOContext{REPL.Terminals.TTYTerminal}, ::Term) at /Users/mason/.julia/packages/Terms/rb9Kk/src/term.jl:58
 [4] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::Term) at ./sysimg.jl:194
 [5] display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any) at /Users/mason/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:131
 [6] display(::REPL.REPLDisplay, ::Any) at /Users/mason/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:135
 [7] display(::Any) at ./multimedia.jl:287
 [8] #invokelatest#1 at ./essentials.jl:742 [inlined]
 [9] invokelatest at ./essentials.jl:741 [inlined]
 [10] print_response(::IO, ::Any, ::Any, ::Bool, ::Bool, ::Any) at /Users/mason/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:155
 [11] print_response(::REPL.AbstractREPL, ::Any, ::Any, ::Bool, ::Bool) at /Users/mason/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:140
 [12] (::getfield(REPL, Symbol("#do_respond#38")){Bool,getfield(REPL, Symbol("##48#57")){REPL.LineEditREPL,REPL.REPLHistoryProvider},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at /Users/mason/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:714
 [13] #invokelatest#1 at ./essentials.jl:742 [inlined]
 [14] invokelatest at ./essentials.jl:741 [inlined]
 [15] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /Users/mason/julia/usr/share/julia/stdlib/v1.1/REPL/src/LineEdit.jl:2273
 [16] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /Users/mason/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:1035
 [17] run_repl(::REPL.AbstractREPL, ::Any) at /Users/mason/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:192
 [18] (::getfield(Base, Symbol("##734#736")){Bool,Bool,Bool,Bool})(::Module) at ./client.jl:362
 [19] #invokelatest#1 at ./essentials.jl:742 [inlined]
 [20] invokelatest at ./essentials.jl:741 [inlined]
 [21] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:346
 [22] exec_options(::Base.JLOptions) at ./client.jl:284
 [23] _start() at ./client.jl:436

Here is some stuff to show some intuition for the problem:

str = "a ω b"

julia> length(str)
5

julia> str[1]
'a': ASCII/Unicode U+0061 (category Ll: Letter, lowercase)

julia> str[2]
' ': ASCII/Unicode U+0020 (category Zs: Separator, space)

julia> str[3]
'ω': Unicode U+03c9 (category Ll: Letter, lowercase)

julia> str[4]
ERROR: StringIndexError("a ω b", 4)
Stacktrace:
 [1] string_index_err(::String, ::Int64) at ./strings/string.jl:12
 [2] getindex_continued(::String, ::Int64, ::UInt32) at ./strings/string.jl:218
 [3] getindex(::String, ::Int64) at ./strings/string.jl:211
 [4] top-level scope at none:0

julia> eachindex(str)
Base.EachStringIndex{String}("a ω b")

julia> for i in eachindex(str)
       println(str[i])
       end
a

ω

b