Ejabberd 19.x and 20.01 does not support external HTTP auth
adiii717 opened this issue · 27 comments
Environment
- ejabberd version: 19.x
- Erlang version:
Erlang/OTP 21 [erts-10.2.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] - OS: Linux (Debian)
- Installed from: Offical Docker image
Configuration
auth_method: http
auth_opts:
host: "http://localhost:12000"
connection_pool_size: 10
connection_opts: []
basic_auth: ""
path_prefix: "/"
...
Errors from error.log/crash.log
06:30:30.437 [critical] Failed to start ejabberd application: Invalid value of option auth_method->1: unsupported database: http
Bug description
The ejabberd external HTTP auth module stop working with 19.x and the config not allow me to use auth_method:http, it crashes with the mentioned error, previously we were able to authenticate users using this module.
https://github.com/processone/ejabberd-contrib/tree/master/ejabberd_auth_http
Thanks
Failed to start ejabberd application: Invalid value of option auth_method->1: unsupported database: http
In general, for contributed modules, the process is: first download and install the module:
$ ejabberdctl modules_update_specs
$ ejabberdctl module_install ejabberd_auth_http
And then enable the module in ejabberd.yml and restart ejabberd.
However, this is not enough for this ejabberd_auth_http contribution, because it is an authentication method, and requires more integration with ejabberd. Now I've noticed it requires different steps to install it, not yet documented in README.txt Please try those steps and comment if they work, then I'll update the documentation.
- If not already done: Download the ejabberd-contrib repository: ejabberdctl modules_update_specs
- Copy ejabberd_auth_http.erl and scram2.erl from $HOME/.ejabberd-modules/sources/ejabberd-contrib/ejabberd_auth_http/src/ to your ejabberd/src/
- Apply this small change:
--- a/src/ejabberd_options.erl
+++ b/src/ejabberd_options.erl
@@ -33,6 +33,8 @@
%%% API
%%%===================================================================
-spec opt_type(atom()) -> econf:validator().
+opt_type(auth_opts) ->
+ econf:any();
opt_type(access_rules) ->
acl:validator(access_rules);
opt_type(acl) ->
@@ -472,6 +474,7 @@ options() ->
{queue_type, ram},
{version, ejabberd_config:version()},
%% Other options
+ {auth_opts, []},
{acl, []},
{access_rules, []},
{acme, #{}},
- Run "make options"
- Recompile ejabberd as usual, and reinstall it
- Now you can configure ejabberd as mentioned in the documentation
- Restart ejabberd
A already tried the first option which did not worked.
> In general, for contributed modules, the process is: first download and install the module:
> $ ejabberdctl modules_update_specs
> $ ejabberdctl module_install ejabberd_auth_http
> And then enable the module in ejabberd.yml and restart ejabberd.
It will be fine to work with 18.x, but above 18.04 I got below error.
13:35:22.610 [error] Unknown option 'auth_opts'
13:35:22.610 [error] Failed to load configuration file /home/ejabberd/conf/ejabberd.yml
BTW external auth is good to run in a production environment? seems like piping to stdin.
BTW external auth is good to run in a production environment? seems like piping to stdin.
It mostly depends on efficiency of your external authentication script/program: stdin/stdout serialization between processes is not that expensive. For example, Postfix uses the same design and nobody complains.
I see thanks for the clarification, but We had a discussion with Jérôme Sautret last year regarding ejabberd server optimization and He suggested us to use HTTP based auth or internal authentication instead of the external script but we can not use internal auth as we are interested to use custom authentication. As we faced some issue in past where external auth script was not responding to the ejabberd during peak time or underload. But Today I have seen some nodejs based script that also able to work with external script.
https://www.npmjs.com/package/ejabberd-auth
So I am still confused to for external auth or do the above workaround with ejabberd HTTP auth?
Thanks
Unknown option 'auth_opts'
That's solved by the method with 6 steps that I mentioned before ;)
So I am still confused to for external auth or do the above workaround with ejabberd HTTP auth?
Then it's probably better to keep using existing solution. As @badlop told already the code can be ported to new ejabberd versions.
@badlop seems like the changes does not resolve the error, now getting this error
[critical] Failed to set logging: {error,
{handler_not_added,
{invalid_config,logger_std_h,
#{file =>
"/home/ejabberd/logs/ejabberd.log"}}}}
This is another problem. You messed with configure/compile process. What's your Erlang version?
Erlang/OTP 21 [erts-10.2.1] [source] [64-bit]
as I mentioned I am using Docker, so this what I am using for the complication,
I have cloned the repo and did the above mentioned changes.
COPY ejabberd ejabberd
WORKDIR /ejabberd
COPY vars.config .
COPY rel/*exs rel/
RUN mix deps.get \
&& (cd deps/eimp; ./configure)
Well your ejabberd was compiled as if you were using Erlang/OTP 22, hence this error.
So I need to use Erlang/OTP 22?
Yes, you can simply upgrade to the latest Erlang to get rid of this problem fast.
But don't forget to recompile ejabberd after the upgrade.
Note with thanks, let me try this.
With ejabberd 19.09.1 and following the instructions in this comment:
Following the steps in order, make options runs fine, and then a make afterwards results in:
src/ejabberd_option.erl:231: spec for auth_opts/0 already defined
src/ejabberd_option.erl:232: function auth_opts/0 already defined
src/ejabberd_option.erl:234: spec for auth_opts/1 already defined
src/ejabberd_option.erl:235: function auth_opts/1 already defined
src/ejabberd_option.erl:19: Warning: function auth_opts/0 already exported
src/ejabberd_option.erl:19: Warning: function auth_opts/1 already exported
Compiling src/ejabberd_option.erl failed:
ERROR: compile failed while processing /srv/ejabberd/ejabberd-19.09.1: rebar_abort
make: *** [Makefile:112: src] Error 1
Copying the .erl files, running make options, patching src/ejabberd_options.erl, then running make; make install works fine, but upon attempted authentication:
03:57:24.138 [error] gen_server <0.692.0> terminated with reason: no case clause matching true in ejabberd_auth:'-db_check_password/7-fun-0-'/5 line 661
03:57:24.138 [error] CRASH REPORT Process <0.692.0> with 0 neighbours exited with reason: no case clause matching true in ejabberd_auth:'-db_check_password/7-fun-0-'/5 line 661 in p1_server:terminate/7 line 878
03:57:24.139 [error] Supervisor ejabberd_c2s_sup had child undefined started with {ejabberd_c2s,start_link,undefined} at <0.692.0> exit with reason no case clause matching true in ejabberd_auth:'-db_check_password/7-fun-0-'/5 line 661 in context child_terminated
Issues are also present in ejabberd 20.01. Both on Erlang/OTP 22.
You are right, ejabberd considers opt_type only when it is in ejabberd_options, and in fact complains when it's double defined. So I've removed it from ejabberd_auth_http.erl and updated the patch. In summary, the updated steps are:
- Download or update the ejabberd-contrib repository: ejabberdctl modules_update_specs
- Copy ejabberd_auth_http.erl and scram2.erl from $HOME/.ejabberd-modules/sources/ejabberd-contrib/ejabberd_auth_http/src/ to your ejabberd/src/
- Apply this patch to ejabberd source code:
diff --git a/src/ejabberd_options.erl b/src/ejabberd_options.erl
index 3af793a91..e8b8cb890 100644
--- a/src/ejabberd_options.erl
+++ b/src/ejabberd_options.erl
@@ -62,6 +62,21 @@ opt_type(auth_cache_size) ->
econf:pos_int(infinity);
opt_type(auth_method) ->
econf:list_or_single(econf:db_type(ejabberd_auth));
+opt_type(auth_opts) ->
+ fun(L) when is_list(L) ->
+ lists:map(
+ fun({host, V}) when is_binary(V) ->
+ {host, V};
+ ({connection_pool_size, V}) when is_integer(V) ->
+ {connection_pool_size, V};
+ ({connection_opts, V}) when is_list(V) ->
+ {connection_opts, V};
+ ({basic_auth, V}) when is_binary(V) ->
+ {basic_auth, V};
+ ({path_prefix, V}) when is_binary(V) ->
+ {path_prefix, V}
+ end, L)
+ end;
opt_type(auth_password_format) ->
econf:enum([plain, scram]);
opt_type(auth_use_cache) ->
@@ -443,6 +458,7 @@ opt_type(jwt_auth_only_rule) ->
{disable_sasl_mechanisms, [binary()]} |
{s2s_zlib, boolean()} |
{loglevel, ejabberd_logger:loglevel()} |
+ {auth_opts, [{any(), any()}]} |
{listen, [ejabberd_listener:listener()]} |
{modules, [{module(), gen_mod:opts(), integer()}]} |
{ldap_uids, [{binary(), binary()}]} |
@@ -493,6 +509,7 @@ options() ->
fun(Host) -> ejabberd_config:get_option({cache_size, Host}) end},
{auth_method,
fun(Host) -> [ejabberd_config:default_db(Host, ejabberd_auth)] end},
+ {auth_opts, []},
{auth_password_format, plain},
{auth_use_cache,
fun(Host) -> ejabberd_config:get_option({use_cache, Host}) end},
- Run "make options"
- Recompile ejabberd as usual, and reinstall it
- Now you can configure ejabberd as mentioned in the documentation
- Restart ejabberd
I was able to make the above changes and compiled the ejabberd, now its crash with this error
@zinid my EJabberd version is 20.01 and Erlang version info
Erlang/OTP 22 [erts-10.4.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]
here is the Error logs
2020-02-06 09:15:35.257134+00:00 [critical] Failed to start ejabberd application: Configuration error: Duplicated option: auth_method
whenever I use auth_method: http
auth_method: http
auth_opts:
host: "http://localhost:3000"
connection_pool_size: 10
connection_opts: []
basic_auth: ""
path_prefix: "/"
When I removed ejabberd node modules it throw this err
2020-02-06 09:17:33.088428+00:00 [error] SUPERVISOR REPORT:
supervisor: {local,ejabberd_http_sup}
errorContext: child_terminated
reason: {undef,
[{cuesport,get_worker,[ejabberd_auth_http_localhost],[]},
{ejabberd_auth_http,make_req,5,
[{file,"src/ejabberd_auth_http.erl"},{line,203}]},
{ejabberd_auth_http,check_password,4,
[{file,"src/ejabberd_auth_http.erl"},{line,63}]},
{ejabberd_auth,'-db_check_password/7-fun-0-',5,
[{file,"src/ejabberd_auth.erl"},{line,661}]},
{ets_cache,update,5,
[{file,"/ejabberd/deps/cache_tab/src/ets_cache.erl"},
{line,208}]},
{ejabberd_auth,db_check_password,7,
[{file,"src/ejabberd_auth.erl"},{line,658}]},
{ejabberd_auth,'-check_password_with_authmodule/6-fun-0-',8,
[{file,"src/ejabberd_auth.erl"},{line,243}]},
{lists,foldl,3,[{file,"lists.erl"},{line,1263}]}]}
offender: [{pid,<0.761.0>},
{id,undefined},
{mfargs,{ejabberd_http,start_link,undefined}},
{restart_type,temporary},
{shutdown,5000},
{child_type,worker}]
Failed to start ejabberd application: Invalid value of option auth_method->1: unsupported database: http
In general, for contributed modules, the process is: first download and install the module:
$ ejabberdctl modules_update_specs
$ ejabberdctl module_install ejabberd_auth_http
And then enable the module in ejabberd.yml and restart ejabberd.However, this is not enough for this ejabberd_auth_http contribution, because it is an authentication method, and requires more integration with ejabberd. Now I've noticed it requires different steps to install it, not yet documented in README.txt Please try those steps and comment if they work, then I'll update the documentation.
- If not already done: Download the ejabberd-contrib repository: ejabberdctl modules_update_specs
- Copy ejabberd_auth_http.erl and scram2.erl from $HOME/.ejabberd-modules/sources/ejabberd-contrib/ejabberd_auth_http/src/ to your ejabberd/src/
- Apply this small change:
--- a/src/ejabberd_options.erl +++ b/src/ejabberd_options.erl @@ -33,6 +33,8 @@ %%% API %%%=================================================================== -spec opt_type(atom()) -> econf:validator(). +opt_type(auth_opts) -> + econf:any(); opt_type(access_rules) -> acl:validator(access_rules); opt_type(acl) -> @@ -472,6 +474,7 @@ options() -> {queue_type, ram}, {version, ejabberd_config:version()}, %% Other options + {auth_opts, []}, {acl, []}, {access_rules, []}, {acme, #{}},
- Run "make options"
- Recompile ejabberd as usual, and reinstall it
- Now you can configure ejabberd as mentioned in the documentation
- Restart ejabberd
with this option, we also getting this error
[critical] Failed to start ejabberd application: Configuration error: Duplicated option: auth_method
@adiii717 Do you have multiple definitions in your .yml or includes?
No, here is my HTTP auth config
auth_method: http
auth_opts:
host: "http://localhost:3000"
connection_pool_size: 10
connection_opts: []
basic_auth: ""
path_prefix: "/"
whenever I use auth_method: http
The error message clearly indicates that the option auth_method is defined twice.
Maybe you define once in ejabberd.yml, and the other definition is in $HOME/.ejabberd-modules/ejabberd_auth_http/conf/ejabberd_auth_http.yml ? You can remove that conf file, as nowadays you configure it directly in the main ejabberd.yml
Thanks, but now getting this error when try to login
[error] SUPERVISOR REPORT:
supervisor: {local,ejabberd_http_sup}
errorContext: child_terminated
reason: {badarg,
[{ets,lookup,[ejabberd_auth_http_localhost,pool_size],[]},
{cuesport,get_worker,1,
[{file,
"/home/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/ejabberd_auth_http/deps/cuesport/src/cuesport.erl"},
{line,55}]},
{ejabberd_auth_http,make_req,5,
[{file,"src/ejabberd_auth_http.erl"},{line,203}]},
{ejabberd_auth_http,check_password,4,
[{file,"src/ejabberd_auth_http.erl"},{line,63}]},
{ejabberd_auth,'-db_check_password/7-fun-0-',5,
[{file,"src/ejabberd_auth.erl"},{line,661}]},
{ets_cache,update,5,
[{file,"/ejabberd/deps/cache_tab/src/ets_cache.erl"},
{line,208}]},
{ejabberd_auth,db_check_password,7,
[{file,"src/ejabberd_auth.erl"},{line,658}]},
{ejabberd_auth,'-check_password_with_authmodule/6-fun-0-',8,
[{file,"src/ejabberd_auth.erl"},{line,243}]}]}
offender: [{pid,<0.842.0>},
{id,undefined},
{mfargs,{ejabberd_http,start_link,undefined}},
{restart_type,temporary},
{shutdown,5000},
{child_type,worker}]
Ok, it's taking some time, but it seems this time we're closer to the final instructions. Starting from the beginning:
- Download or update the ejabberd-contrib repository:
ejabberdctl modules_update_specs - Compile and install the module and its dependencies:
ejabberdctl module_install ejabberd_auth_http - Configure the module in your main ejabberd.yml probably with something like this:
## auth_method: internal
auth_method: http
auth_opts:
host: "http://localhost:3000"
connection_pool_size: 10
connection_opts: []
basic_auth: ""
path_prefix: "/"
- Remove the small configuration file here, which has duplicated options: $HOME/.ejabberd-modules/ejabberd_auth_http/conf/*
- Apply this patch to ejabberd source code:
diff --git a/src/ejabberd_options.erl b/src/ejabberd_options.erl
index 3af793a91..e8b8cb890 100644
--- a/src/ejabberd_options.erl
+++ b/src/ejabberd_options.erl
@@ -62,6 +62,21 @@ opt_type(auth_cache_size) ->
econf:pos_int(infinity);
opt_type(auth_method) ->
econf:list_or_single(econf:db_type(ejabberd_auth));
+opt_type(auth_opts) ->
+ fun(L) when is_list(L) ->
+ lists:map(
+ fun({host, V}) when is_binary(V) ->
+ {host, V};
+ ({connection_pool_size, V}) when is_integer(V) ->
+ {connection_pool_size, V};
+ ({connection_opts, V}) when is_list(V) ->
+ {connection_opts, V};
+ ({basic_auth, V}) when is_binary(V) ->
+ {basic_auth, V};
+ ({path_prefix, V}) when is_binary(V) ->
+ {path_prefix, V}
+ end, L)
+ end;
opt_type(auth_password_format) ->
econf:enum([plain, scram]);
opt_type(auth_use_cache) ->
@@ -443,6 +458,7 @@ opt_type(jwt_auth_only_rule) ->
{disable_sasl_mechanisms, [binary()]} |
{s2s_zlib, boolean()} |
{loglevel, ejabberd_logger:loglevel()} |
+ {auth_opts, [{any(), any()}]} |
{listen, [ejabberd_listener:listener()]} |
{modules, [{module(), gen_mod:opts(), integer()}]} |
{ldap_uids, [{binary(), binary()}]} |
@@ -493,6 +509,7 @@ options() ->
fun(Host) -> ejabberd_config:get_option({cache_size, Host}) end},
{auth_method,
fun(Host) -> [ejabberd_config:default_db(Host, ejabberd_auth)] end},
+ {auth_opts, []},
{auth_password_format, plain},
{auth_use_cache,
fun(Host) -> ejabberd_config:get_option({use_cache, Host}) end},
- Run "make options"
- Recompile ejabberd, and reinstall it
- Restart ejabberd
again crash during login, but in this case, the auth server at least received the call, but ejabberd crash with this error
2020-02-07 18:50:21.975556+00:00 [info] (<0.886.0>) Accepted connection [::ffff:172.17.0.1]:33436 -> [::ffff:172.17.0.5]:5280
2020-02-07 18:50:21.980539+00:00 [error] CRASH REPORT:
crasher:
initial call: ejabberd_http:init/3
pid: <0.886.0>
registered_name: []
exception error: no case clause matching true
in function ejabberd_auth:'-db_check_password/7-fun-0-'/5 (src/ejabberd_auth.erl, line 661)
in call from ets_cache:update/5 (/ejabberd/deps/cache_tab/src/ets_cache.erl, line 208)
in call from ejabberd_auth:db_check_password/7 (src/ejabberd_auth.erl, line 658)
in call from ejabberd_auth:'-check_password_with_authmodule/6-fun-0-'/8 (src/ejabberd_auth.erl, line 243)
in call from lists:foldl/3 (lists.erl, line 1263)
in call from ejabberd_auth:check_password_with_authmodule/6 (src/ejabberd_auth.erl, line 241)
in call from ejabberd_auth:check_password/6 (src/ejabberd_auth.erl, line 218)
in call from ejabberd_web_admin:get_auth_account2/5 (src/ejabberd_web_admin.erl, line 251)
ancestors: [ejabberd_http_sup,ejabberd_sup,<0.130.0>]
message_queue_len: 0
messages: []
links: [<0.846.0>,#Port<0.45>]
dictionary: []
trap_exit: false
status: running
heap_size: 1598
stack_size: 27
reductions: 3152
neighbours:
2020-02-07 18:50:21.983956+00:00 [error] SUPERVISOR REPORT:
supervisor: {local,ejabberd_http_sup}
errorContext: child_terminated
reason: {{case_clause,true},
[{ejabberd_auth,'-db_check_password/7-fun-0-',5,
[{file,"src/ejabberd_auth.erl"},{line,661}]},
{ets_cache,update,5,
[{file,"/ejabberd/deps/cache_tab/src/ets_cache.erl"},
{line,208}]},
{ejabberd_auth,db_check_password,7,
[{file,"src/ejabberd_auth.erl"},{line,658}]},
{ejabberd_auth,'-check_password_with_authmodule/6-fun-0-',8,
[{file,"src/ejabberd_auth.erl"},{line,243}]},
{lists,foldl,3,[{file,"lists.erl"},{line,1263}]},
{ejabberd_auth,check_password_with_authmodule,6,
[{file,"src/ejabberd_auth.erl"},{line,241}]},
{ejabberd_auth,check_password,6,
[{file,"src/ejabberd_auth.erl"},{line,218}]},
{ejabberd_web_admin,get_auth_account2,5,
[{file,"src/ejabberd_web_admin.erl"},
{line,251}]}]}
offender: [{pid,<0.886.0>},
{id,undefined},
{mfargs,{ejabberd_http,start_link,undefined}},
{restart_type,temporary},
{shutdown,5000},
{child_type,worker}]
I had the same problem
ejabberd version 19.09.1
I ran Dialyzer, which reports inconsistencies between function calls, and it reported several problems in ejabberd_auth_http.erl when used with recent ejabberd. I've fixed most of them and committed to ejabberd-contrib. So, that is another step forward.
Please update the module: remove it reverting step 2, then repeat steps 1, 2, 4 and 8. Let's see what results in your new tests.
The bad news is that I don't have any server where I can test the module in practice...
Thanks to your help, it seems the module now finally can be compiled and installed correctly. So, I've appplied the changes to ejabberd.
From now on, if you find problems in the module, please open a new ticket to investigate them.