elixir-lang/elixir

Support Erlang 20 new features

Closed this issue · 27 comments

The following are chores that be done in order to properly support 20. Those must be backported:

  • The compiler has new warnings for repeated identical map keys
  • No longer use regexes in Elixir as they are not portable from 19->20
  • Fix logger messages
  • Check for OTP version in Mix manifest

The following changes can be done as soon as the first RC is out:

  • Support UTF-8 encoding for atoms and variables
  • Use the new debug_info compiler options
  • Use the new extra_chunks compiler options
  • Support shell history

The following tasks requires us to support at least Erlang 20:

  • Trim down the Unicode modules as Erlang provides NFC/NFD conversion and deprecate functions from String
  • There are two new guard BIFs floor/1 and ceil/1
  • No longer use try/catch in Task (this requirement was dropped from proc_lib)
  • No longer use outdated rand algorithms (exs1024 -> exs1024s)
  • Use map instructions in comprehensions #7032
  • Optimize maps:put and maps:merge into instructions #7353
  • Consistently use :second, :nanosecond as time units and deprecate the old ones
  • Update parse transforms warning to say they are no longer applied

Please do not send PR for those issues as they cannot be tackled right now but rather long term.

List updated with Erlang 20 RC1 notes.

c0b commented

About unicode atoms, with erlang 20.0-rc1 I found only the quoted syntax is supported:

Erlang/OTP 20 [RELEASE CANDIDATE 1] [erts-9.0] [source-9ac8bdb] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.0  (abort with ^G)
1> 'hello_юникод_世界'.
'hello_юникод_世界'
2> hello_юникод_世界.
* 1: illegal character

am not sure will the unquoted unicode atom be supported in erlang?
when run iex on top of, it's similar that the quoted syntax is already supported, so wonder from here will the unquoted syntax be supported?

Erlang/OTP 20 [RELEASE CANDIDATE 1] [erts-9.0] [source-9ac8bdb] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false]

Interactive Elixir (1.4.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> :"hello_юникод_世界" 
:"hello_юникод_世界"
iex(2)> :hello_юникод_世界
** (SyntaxError) iex:2: unexpected token: "ю" (column 8, codepoint U+044E) 

The plan is exactly to support the unquoted syntax in Elixir.

gmile commented

@josevalim regarding this:

No longer use regexes in Elixir as they are not portable from 19->20

Can you point to a regex usage example in Elixir source code, the usage of which would have to be cleaned?

I saw support for shell history is checked, is it already part of 1.4.5?

@imranismail there is an open pull request to improve the experience under 1.5 but it does work on 1.4.5 as long as you set an environment variable. See the docs here: #6242.

Yep just saw a tweet by @ferd about the environment variable

Thank you, will give this a shot

@josevalim why adding the two new guards require only Erlang 20? Shouldn't it be possible to define the functions with some annotation that they can be used in guards only in Erlang 20? Outside of guards we could simulate them with other calls.

No longer use outdated rand algorithms (exs1024 -> exs1024s)

Regarding that item should usages of :exsplus also be replaced? If so should they be changed to :exsp or :exrop?

I ask because :exsplus is not listed as one of the three built-in algorithms in the :rand docs.

@TattdCodeMonkey so we would migrate them all, yeah.

@josevalim what about OTP-14296?

OTP-14296 Application(s): debugger, stdlib
               The Erlang shell, qlc:string_to_handle(), and the
               Debugger (the Evaluator area and Edit variable window
               of the Bindings area) can parse pids, ports,
               references, and external funs, as long as they can be
               created in the running system.

If I understand, that makes erlang parse PID<0.85.0> and so on correctly. Seems like that could be used in iex once we require OTP 20

@archseer we don't need to wait for Erlang to support this feature but it is not a feature we are interested into bringing into IEx. Elixir allows any custom data type to be printed as #XYZ<...>, Erlang only handles the built-in types like that and does not allow custom representations, so it is easier for them to tackle this.

@archseer you can create pids and refs with the pid and ref helpers in IEx.

Closing this as we created separate issues for what is pending.

Environment

  • Elixir & Erlang versions (elixir --version):
    Erlang/OTP 21 [erts-10.0.1] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1]
    Elixir 1.6.6 (compiled with OTP 19)
  • Operating system:
    Windows 10
  • Client (editor or IDE):
    vscode

###############
I get the message below on opening elixir phoenix project in vscode:

Dialyzer is disabled because core Elixir modules are missing debug info. You may need to recompile Elixir with Erlang >= OTP 20

I've tried to update elixir but still no luck.

What is recommended? Do I need to rollback Erlang to OTP 19?

NobbZ commented

You need to recompile your elixir with erlang OTP 20 or newer. Thats all written in the message.

Alternatively you can of course download a version prebuilt for OTP 20+.

But if you do it or not, you can totally work with your current setup. The warning you see from your editor is just about some ELS-stuff that enables you to get quicker feedback from dialyzer in the editor.

It does not influence your ability to manually run dialyzer. Also it will not affect other functionality of your editor or the language server.

Well, these are both fresh installed latest oficially released windows versions for erlang and elixir. I just was not sure that everything except the dializer will work as expected. So it will, thanks..

NobbZ commented

dialyzer itself will work perfectly fine, only the editor integration does not work. Thats again has nothing to do with elixir. If the message shown is unclear please report this to the vscode-elixir-ls project.

@igormisha each Elixir version is precompiled with the lowest supported Erlang/OTP version. In this case, Elixir v1.7 is precompiled with Erlang/OTP 19. This is rarely a problem but it is a problem on Erlang/OTP 21 which expects the dialyzer information elsewhere. You can downgrade to Erlang/OTP 19 or 20. Also note this issue won't happen in the next Elixir version, as it requires Erlang/OTP 20+.

Thanks for the explanation, Jose!

@josevalim WOuld it be possible to provide two precompiled versions? (the newest and the oldest otp versions)

This breaks dialyzer in all 1.7 elixir docker images.

Hex.pm team uses bob to provide all elixir versions compiled against all supported major erlang versions. You could use their packages.

https://github.com/hexpm/bob#elixir-builds

@michalmuskala Thanks, I'll have a look and provide a PR for the image.

Just want to comment here, if someone will get:

Dialyzer is disabled because core Elixir modules are missing debug info. You may need to recompile Elixir with Erlang >= OTP 20

error in Elixir Language Server with Elixir 1.7 from ASDF and this will be the first link.


As said, Elixir by default ships precompiled with earliest Erlang version - 19.
BUT, we can use any Bob version from asdf Elixir plugin.

Instead:

$ asdf install elixir 1.7

we can use:

$ asdf install elixir 1.7-otp-21

Then in asdf config:

# .tools-versions

elixir 1.7.3-otp-21

This will solve the problem.

For windows users

go to this link
download master-otp-20.zip

in you folder instalation
C:\Program Files (x86)\Elixir

remove bin, lib, man

and paste master-otp-20.zip files