ether/etherpad-lite

Can't retrive admin panel behind Nginx reverse proxy under subdirectory (possible upstream experimental)

Closed this issue · 9 comments

EL-File4138 commented

Describe the bug
Set up an instance behind Nginx reverse proxy under the subdirectory with methods from wiki. The main project works fine, but the admin panel failed to load.
Upon further investigation, found that the assets are loaded relative to the webroot(<host>/admin/assets/...), but not to the subdirectory, which results in a failed rewrite catch for nginx.
Checking for code, admin seems like a separate Vite project. Its build option base specified absolute one level to root, which would not work aforementioned. Other options are not available either, ./ would just result in resolving to webroot without admin level, ./admin is not supported. Seems like an experimental method described in Vite project document would work but with tweaking.
Recommendation:

  • Make a fix to the path problem using the provided feature (experimental);
    • Should be the best to do, still using the current method to treat subdirectory build, but still experimental;
  • Provide a configuration to declare the use of subdirectory, rewriting the base path on the fly, for the admin panel only (Just adding a workaround instruction calling people to manually write would be helpful);
    • Quickest to do but not really recommended, more quirk to do between the main project and the admin panel;
  • Other possible fixes?

To Reproduce
Steps to reproduce the behavior:

  1. Manual install and start the instance as described in README;
  2. Configure reverse proxy with the provided config in the wiki;
  3. Open the admin panel.

Expected behavior
Admin panel should load.

Screenshots
Incorrect loading when using subdirectory deployment

Server (please complete the following information):

  • Etherpad version: v2.0.3
  • OS: CentOS 9 Stream
  • Node.js version (node --version): v18.19.1
  • npm version (npm --version): 10.2.0
  • Is the server free of plugins: It's free of plugins.

Desktop (please complete the following information):
Not relevant but here you go:

  • OS: Fedora
  • Browser [e.g. chrome, safari]: Mozilla Firefox
  • Version [e.g. 22]: 125.0.3

Smartphone (please complete the following information):
Not relevant.

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

SamTV12345 commented

Describe the bug Set up an instance behind Nginx reverse proxy under the subdirectory with methods from wiki. The main project works fine, but the admin panel failed to load. Upon further investigation, found that the assets are loaded relative to the webroot(<host>/admin/assets/...), but not to the subdirectory, which results in a failed rewrite catch for nginx. Checking for code, admin seems like a separate Vite project. Its build option base specified absolute one level to root, which would not work aforementioned. Other options are not available either, ./ would just result in resolving to webroot without admin level, ./admin is not supported. Seems like an experimental method described in Vite project document would work but with tweaking. Recommendation:

  • Make a fix to the path problem using the provided feature (experimental);

    • Should be the best to do, still using the current method to treat subdirectory build, but still experimental;

  • Provide a configuration to declare the use of subdirectory, rewriting the base path on the fly, for the admin panel only (Just adding a workaround instruction calling people to manually write would be helpful);

    • Quickest to do but not really recommended, more quirk to do between the main project and the admin panel;

  • Other possible fixes?

To Reproduce Steps to reproduce the behavior:

  1. Manual install and start the instance as described in README;
  2. Configure reverse proxy with the provided config in the wiki;
  3. Open the admin panel.

Expected behavior Admin panel should load.

Screenshots Incorrect loading when using subdirectory deployment

Server (please complete the following information):

  • Etherpad version: v2.0.3
  • OS: CentOS 9 Stream
  • Node.js version (node --version): v18.19.1
  • npm version (npm --version): 10.2.0
  • Is the server free of plugins: It's free of plugins.

Desktop (please complete the following information): Not relevant but here you go:

  • OS: Fedora
  • Browser [e.g. chrome, safari]: Mozilla Firefox
  • Version [e.g. 22]: 125.0.3

Smartphone (please complete the following information): Not relevant.

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context Add any other context about the problem here.

Oh that is a regression. Thanks for the report. I'll try to find a solution. Maybe something with JS that replaces the url with window.href at runtime but that is something I need to check first.

Uatschitchun commented

I'll add myself to this one:

I'm running EP on Uberspace even behind a nginx proxy which adds

    location /pad/ {
        rewrite ^/pad/?(.*) /$1 break;
        proxy_pass http://server.local.uberspace.de:9001;

to nginx.conf.

Admin panel was working well with 1.9.7 but it fails with 2.x

What can be done here?

EL-File4138 commented

@Uatschitchun The nginx config has been updated too, see Wiki. Please try modifying your nginx config.

Uatschitchun commented

@EL-File4138 It's sadly not possible to setup own nginx.conf on uberspace ;(

Don't know if this helps, but calling domain.de/pad/admin/login gives this in access.log:

server.uber.space:443 88.78.0.0 - - [25/May/2024:14:27:32 +0200] "GET /pad/admin/login HTTP/2.0" 200 492 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 OPR/109.0.0.0"
server.uber.space:443 88.78.0.0 - - [25/May/2024:14:27:32 +0200] "GET /admin/assets/index-BkfU4Tdl.js HTTP/2.0" 404 196 "https://server.uber.space/pad/admin/login" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 OPR/109.0.0.0"
server.uber.space:443 88.78.0.0 - - [25/May/2024:14:27:32 +0200] "GET /admin/assets/index-E-lmtrZj.css HTTP/2.0" 404 196 "https://server.uber.space/pad/admin/login" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 OPR/109.0.0.0"

While opening domain.de/pad/admin-auth gives this:

server.uber.space:443 88.78.0.0 - - [25/May/2024:14:29:39 +0200] "GET /pad/admin-auth HTTP/2.0" 401 23 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 OPR/109.0.0.0"

Result:
Authentication Required

/pad/socket.io/ is also working:

server.uber.space:443 88.78.0.0 - - [25/May/2024:14:32:53 +0200] "GET /pad/socket.io/ HTTP/2.0" 400 40 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 OPR/109.0.0.0"

Result:
{"code":0,"message":"Transport unknown"}

So is there any re-writing going on regarding /admin/assests preventing Admin UI from working?

EL-File4138 commented

@Uatschitchun This fix utilizes an HTTP Header X-Proxy-Path to derive the correct admin panel path, which is set in the related nginx config. In other words, you have to refer to your hosting provider for the related change.

Uatschitchun commented

@EL-File4138
Hm, ok. I've set X-Proxy-Path to /pad:

$ curl -I https://server.uber.space/pad
HTTP/2 200 
date: Sat, 25 May 2024 16:19:56 GMT
content-type: text/html; charset=utf-8
content-length: 6042
vary: Accept-Encoding
x-ua-compatible: IE=Edge,chrome=1
referrer-policy: strict-origin-when-cross-origin
etag: W/"179a-DDyWCOW/5aauII2xK/KNIA4OLTg"
x-xss-protection: 1; mode=block
x-proxy-path: /pad
x-content-type-options: nosniff
strict-transport-security: max-age=31536000
x-frame-options: SAMEORIGIN

So if I get your fix correct, the paths should be created dynamically from adding x-proxy-path to /admin & /socket.io

Did the following:

  1. Freshly checked out the devel branch
  2. just run bin/installDeps.sh
  3. edit settings.json to have admin user
  4. start EP: pnpm run prod (do NOT create admin UI with bin/run.sh or pnpm run build-copy in admin/

URL/pad => works (worked also before setting X-Proxy-Path)
URL/pad/socket.io => works (worked also before setting X-Proxy-Path)
URL/pad/admin/plugins => crashes:

[ERROR] server - Error: ENOENT: no such file or directory, stat '/home/user/etherpad/src/templates/admin/index.html'
    at Object.statSync (node:fs:1659:25)
    at <anonymous> (/home/user/etherpad/src/node/hooks/express/admin.ts:57:14)
    at suppressedCallback (node:fs:267:5)
    at FSReqCallback.oncomplete (node:fs:194:23)
[ERROR] server - Error occurred while stopping Etherpad
[ERROR] server - Error in process exit TypeError: Cannot read properties of null (reading 'metrics')
    at Gauge._readFn (/home/user/etherpad/src/node/db/DB.ts:45:57)
    at Gauge.toJSON (/home/user/etherpad/node_modules/.pnpm/measured-core@2.0.0/node_modules/measured-core/lib/metrics/Gauge.js:24:17)
    at /home/user/etherpad/node_modules/.pnpm/measured-core@2.0.0/node_modules/measured-core/lib/Collection.js:62:46
    at Array.forEach (<anonymous>)
    at Collection.toJSON (/home/user/etherpad/node_modules/.pnpm/measured-core@2.0.0/node_modules/measured-core/lib/Collection.js:60:32)
    at Object.exports.exit (/home/user/etherpad/src/node/server.ts:248:75)
    at Object.exports.stop (/home/user/etherpad/src/node/server.ts:232:26)
    at Object.exports.exit (/home/user/etherpad/src/node/server.ts:262:7)

access.log states:

"GET /pad/socket.io/ HTTP/2.0" 400 40 "-" "
"GET /pad/admin/plugins HTTP/2.0" 502 552 "-" "

URL is correct: /pad/admin/plugins

After running bin/run.sh:
URL/pad => works (worked also before setting X-Proxy-Path)
URL/pad/socket.io => works (worked also before setting X-Proxy-Path)
URL/pad/admin/plugins => does not work (X-Proxy-Path is set but not used)

"GET /pad/socket.io/ HTTP/2.0" 400 40 "-" "

"GET /pad/admin/plugins HTTP/2.0" 200 492 "-" "
"GET /admin/assets/index-E-lmtrZj.css HTTP/2.0" 404 196 "https://server.uber.space/pad/admin/plugins" 
"GET /admin/assets/index-BkfU4Tdl.js HTTP/2.0" 404 196 "https://server.uber.space/pad/admin/plugins"
  • Although X-Proxy-Path is set, it seems to not be taken into account
  • If Admin UI isn't built, EP crashes. The console error in src/node/hooks/express/admin.ts:22 doesn't show

With 2.0.3 (prior to the fix) the missing Admin UI is just noted as:

[ERROR] settings - Error: ENOENT: no such file or directory, stat '/home/user/etherpad/src/templates/admin/index.html'

and EP does not crash!

access.log gives:

"GET /pad/socket.io/ HTTP/2.0" 400 40 "-" 
"GET /pad/admin/plugins HTTP/2.0" 500 1250 "-"

which are the correct paths!

EL-File4138 commented

@Uatschitchun
Some possible problems:

  1. You may need to reread the nginx config. The header should be targeted to /admin, /socket.io, and /admin-auth, with rewrite and redirect. The problem here is the resource path directed in HTML, not the targeted URL, for example:
"GET /admin/assets/index-E-lmtrZj.css HTTP/2.0" 404 196 "https://server.uber.space/pad/admin/plugins" 
"GET /admin/assets/index-BkfU4Tdl.js HTTP/2.0" 404 196 "https://server.uber.space/pad/admin/plugins"
  1. Try cherry-picking the related fix without other non-tested commits (i.e., don't pull the fresh devel right off the git). The crash might be the result of other commits.

If the problem persists, I recommend you first test under the given environment (A fully controlled nginx and webroot), then match your configuration with the given config.

SamTV12345 commented

@EL-File4138 Hm, ok. I've set X-Proxy-Path to /pad:

$ curl -I https://server.uber.space/pad
HTTP/2 200 
date: Sat, 25 May 2024 16:19:56 GMT
content-type: text/html; charset=utf-8
content-length: 6042
vary: Accept-Encoding
x-ua-compatible: IE=Edge,chrome=1
referrer-policy: strict-origin-when-cross-origin
etag: W/"179a-DDyWCOW/5aauII2xK/KNIA4OLTg"
x-xss-protection: 1; mode=block
x-proxy-path: /pad
x-content-type-options: nosniff
strict-transport-security: max-age=31536000
x-frame-options: SAMEORIGIN

So if I get your fix correct, the paths should be created dynamically from adding x-proxy-path to /admin & /socket.io

Did the following:

  1. Freshly checked out the devel branch
  2. just run bin/installDeps.sh
  3. edit settings.json to have admin user
  4. start EP: pnpm run prod (do NOT create admin UI with bin/run.sh or pnpm run build-copy in admin/

URL/pad => works (worked also before setting X-Proxy-Path) URL/pad/socket.io => works (worked also before setting X-Proxy-Path) URL/pad/admin/plugins => crashes:

[ERROR] server - Error: ENOENT: no such file or directory, stat '/home/user/etherpad/src/templates/admin/index.html'
    at Object.statSync (node:fs:1659:25)
    at <anonymous> (/home/user/etherpad/src/node/hooks/express/admin.ts:57:14)
    at suppressedCallback (node:fs:267:5)
    at FSReqCallback.oncomplete (node:fs:194:23)
[ERROR] server - Error occurred while stopping Etherpad
[ERROR] server - Error in process exit TypeError: Cannot read properties of null (reading 'metrics')
    at Gauge._readFn (/home/user/etherpad/src/node/db/DB.ts:45:57)
    at Gauge.toJSON (/home/user/etherpad/node_modules/.pnpm/measured-core@2.0.0/node_modules/measured-core/lib/metrics/Gauge.js:24:17)
    at /home/user/etherpad/node_modules/.pnpm/measured-core@2.0.0/node_modules/measured-core/lib/Collection.js:62:46
    at Array.forEach (<anonymous>)
    at Collection.toJSON (/home/user/etherpad/node_modules/.pnpm/measured-core@2.0.0/node_modules/measured-core/lib/Collection.js:60:32)
    at Object.exports.exit (/home/user/etherpad/src/node/server.ts:248:75)
    at Object.exports.stop (/home/user/etherpad/src/node/server.ts:232:26)
    at Object.exports.exit (/home/user/etherpad/src/node/server.ts:262:7)

access.log states:

"GET /pad/socket.io/ HTTP/2.0" 400 40 "-" "
"GET /pad/admin/plugins HTTP/2.0" 502 552 "-" "

URL is correct: /pad/admin/plugins

After running bin/run.sh: URL/pad => works (worked also before setting X-Proxy-Path) URL/pad/socket.io => works (worked also before setting X-Proxy-Path) URL/pad/admin/plugins => does not work (X-Proxy-Path is set but not used)

"GET /pad/socket.io/ HTTP/2.0" 400 40 "-" "

"GET /pad/admin/plugins HTTP/2.0" 200 492 "-" "
"GET /admin/assets/index-E-lmtrZj.css HTTP/2.0" 404 196 "https://server.uber.space/pad/admin/plugins" 
"GET /admin/assets/index-BkfU4Tdl.js HTTP/2.0" 404 196 "https://server.uber.space/pad/admin/plugins"
  • Although X-Proxy-Path is set, it seems to not be taken into account
  • If Admin UI isn't built, EP crashes. The console error in src/node/hooks/express/admin.ts:22 doesn't show

With 2.0.3 (prior to the fix) the missing Admin UI is just noted as:

[ERROR] settings - Error: ENOENT: no such file or directory, stat '/home/user/etherpad/src/templates/admin/index.html'

and EP does not crash!

access.log gives:

"GET /pad/socket.io/ HTTP/2.0" 400 40 "-" 
"GET /pad/admin/plugins HTTP/2.0" 500 1250 "-"

which are the correct paths!

If it might help I have my configuration posted over here: #6402
Somehow the commenter used another rewrite rule that did not work out for me but it seems for him. Quite confusing the proxy thing. Let me know if I can help you any further.

Uatschitchun commented

Ok, so the X-Proxy-Path header isn't correctly received here. Have to check further. It works when "hardcoding" the subfolder path:

            const proxyHeaderValue = "/pad";
            if (proxyHeaderValue) {
              let string = data.toString();
              dataToSend = string.replaceAll("/admin", proxyHeaderValue + "/admin");
              dataToSend = dataToSend.replaceAll("/socket.io", proxyHeaderValue + "/socket.io");

@EL-File4138 & @SamTV12345
could you both please check if viewing a pad from within Admin UI is working with subfolders? On my side the pad is openend without the subfolder path:
https://server.uber.space/p/ABCDE
instead of
https://server.uber.space/pad/p/ABCDE

and the ADMIN_PATH check in src/node/hooks/express/admin.ts:21 is wrong. I commented in #6388

P.S.: Wouldn't it be easier to just add a setting in settings.json instead of adding the header in nginx.conf and rewriting URLs accordingly?!