Tablet syncs but shows no connection to cloud

Question

Tablet syncs but shows no connection to cloud

giovi321 opened this issue 10 months ago · 11 comments

I'm trying to sync an existing rmfakecloud account with a new tablet, but I keep getting the following error when I issue systemctl status xochitl -l:

Mar 06 10:25:35 reMarkable xochitl[596]: 09:25:35.055 rm.network.notifications Notifications socket is not OK: UnconnectedState (checkIfShouldConnect /home/runner/work/xochitl/xochitl/src/notifications/src/notifications.cpp:190)

and the cloud icon shows that the server is offline. Everything was working with the previous tablet.

rmfakecloud version: last, 0.0.17
xochitl version: 3.9.5.2026
rmfakecloud-proxy installation method: manual
rmfakecloud server installation: built from the source

Answer 1 · 2024-03-06T13:34:56.000Z

Ok, I managed to sync, but the cloud connection always appears as offline. I'm using a reverse proxy with Apache 2 on the server, could that be the cause? How is the connectivity checked by xochitl?

I see the following error on the proxy server:

192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "GET /integrations/v1/ HTTP/1.1" 200 4905 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 4843 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 4843 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 4843 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 4843 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "GET /integrations/v1/ HTTP/1.1" 200 4905 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 4843 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"
192.168.1.1 - - [06/Mar/2024:15:11:46 +0100] "POST /v2/events HTTP/1.1" 404 136 "-" "xochitl/3.9.5.837 (codex 4.0.367)"

and the following on rmfakecloud server:

mar 06 14:56:09 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:09+01:00" level=info msg="Requestng blob: root"
mar 06 14:56:09 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:09+01:00" level=info msg="[GIN] 2024/03/06 - 14:57:11 | 200 |     189.011<C2><B5>s |  192.168.56.105 | GET      \"/blobstorage?blobid=root&exp=REDACTED&scope=read&signature=REDACTED>
mar 06 14:56:16 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:16+01:00" level=info msg="[auth-middleware] UserId: giovi deviceId: RM110-325-38011 newSync: true"
mar 06 14:56:16 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:16+01:00" level=info msg="connecting websocket from: giovi"
mar 06 14:56:16 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:16+01:00" level=warning msg="can't upgrade websocket to ws websocket: the client is not using the websocket protocol: 'upgrade' token not found in 'Connection' header"
mar 06 14:56:16 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:16+01:00" level=info msg="[GIN] 2024/03/06 - 14:57:17 | 400 |     170.506<C2><B5>s |  192.168.56.105 | GET      \"/notifications/ws/json/1\""
mar 06 14:56:16 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:16+01:00" level=info msg="[auth-middleware] UserId: giovi deviceId: RM110-325-38011 newSync: true"
mar 06 14:56:16 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:16+01:00" level=info msg="[GIN] 2024/03/06 - 14:57:31 | 200 |     296.412<C2><B5>s |  192.168.56.105 | POST     \"/sync/v2/signed-urls/downloads\""
mar 06 14:56:18 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:18+01:00" level=info msg="[auth-middleware] UserId: giovi deviceId: RM110-325-38011 newSync: true"
mar 06 14:56:18 GC01remarkableSRVR rmfakecloud[527]: time="2024-03-06T14:56:18+01:00" level=info msg="[GIN] 2024/03/06 - 14:57:31 | 200 |     210.953<C2><B5>s |  192.168.56.105 | GET      \"/blobstorage?blobid=root&exp=REDACTED&scope=read&signature=REDACTED""

Answer 2 · 2024-10-24T08:18:45.000Z

Same problem here! Can sync documents but under storage -> Check Sync it says "There are some issues with the cloud connection. Please try again later."

rmfakecloud version: last, 0.0.19
xochitl version: 3.3.2.1666
rmfakecloud-proxy installation method: Toltec
rmfakecloud server installation: Docker

Answer 3 · 2024-10-24T14:50:25.000Z

Is the reverse proxy in front of the container configured to allow websockets? The error log suggests it isn't.

Answer 4 · 2024-10-24T15:41:40.000Z

In my case yes, there is an apache2 reverse proxy. Is there there something that is not being proxied maybe?

Answer 5 · 2024-10-24T15:45:19.000Z

In my case yes, there is an apache2 reverse proxy.

And the apache2 reverse proxy is properly configured to handle websockets? See the nginx configuration for some example information on what is needed, although apache2's configuration is different: https://ddvk.github.io/rmfakecloud/install/reverse-proxy/nginx/

Is there there something that is not being proxied maybe?

You would not see logs in the container if it wasn't being proxied.

Answer 6 · 2024-10-24T19:50:55.000Z

Thank you for the input. I don't see any particular configuration for the websocket in the nginx config at that link, apache2 supports WS reverse proxy but i need an address of the WS for the config. Do you have an idea of what it could be, please?

Answer 7 · 2024-10-24T19:58:32.000Z

Thank you for the input. I don't see any particular configuration for the websocket in the nginx config at that link

proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";

proxy_read_timeout 1d;
proxy_send_timeout 1d;

apache2 supports WS reverse proxy but i need an address of the WS for the config. Do you have an idea of what it could be, please?

From https://github.com/ddvk/rmfakecloud/blob/master/internal/app/routes.go#L111C19-L111C43 the /notifications/ws/json/1 path would be the one that is a websocket.

Answer 8 · 2024-10-24T20:53:01.000Z

I did some testing and adding this seems to make the ws work (at least from the logs of apache2), but the tablet still says there are problems with the WS.

    ProxyPass "/notifications/ws/json/1/" "ws://192.168.56.102:88/notifications/ws/json/1/"
    ProxyPassReverse "/notifications/ws/json/1/" "ws://192.168.56.102:88/notifications/ws/json/1/"
<Location "/notifications/ws/json/1">
   RewriteEngine On
   RewriteCond %{HTTP:Upgrade} =websocket [NC]
   RewriteRule /(.*) ws://192.168.56.102:88/$1 [P,L]
   RewriteCond %{HTTP:Upgrade} !=websocket [NC]
   RewriteRule /(.*) http://192.168.56.102:88/$1 [P,L]

   ProxyAddHeaders Off
   RequestHeader set Connection "upgrade"
   RequestHeader set Upgrade "websocket"
</Location>

apache log:
192.168.1.1 - - [24/Oct/2024:22:26:17 +0200] "GET /notifications/ws/json/1 HTTP/1.1" 200 4085 "-" "xochitl/3.11.0.849 (codex 4.0.447-3-g19f36ec9-dirty)"

Using curl i get obviously unauthorized but this makes me think it works correclty:

HTTP/1.1 401 Unauthorized
Date: Thu, 24 Oct 2024 21:06:53 GMT
Server: Apache/2.4.62 (Debian)
Strict-Transport-Security: max-age=15768000; includeSubDomains; preload
Content-Type: text/plain; charset=utf-8
Content-Length: 12

Unauthorized

Answer 9 · 2024-10-24T22:13:26.000Z

SOLUTION FOUND!
Now it syncs and correctly signals that it is connected to the cloud.

Maybe it would make sense to add it to the documentation.

Here's my apache2 configuration:


<VirtualHost *:80>
        ServerName remarkable.mydomain.com
        ServerAlias www.remarkable.mydomain.com
	RewriteEngine On
	RewriteCond %{HTTPS} !=on
	RewriteCond %{SERVER_NAME} =remarkable.mydomain.com
	RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>


<VirtualHost *:443>
	ServerName remarkable.mydomain.com
	ServerAlias www.remarkable.mydomain.com
	SSLEngine on
	ServerAdmin unavailable
	CustomLog ${APACHE_LOG_DIR}/vhosts/remarkable_access.log combined
	ErrorLog ${APACHE_LOG_DIR}/vhosts/remarkable_error.log
	<IfModule mod_headers.c>
	    Header always add Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
	</IfModule>

        ProxyPass "/notifications/ws/json/1" "ws://myIP:myPORT/notifications/ws/json/1"
        ProxyPassReverse "/notifications/ws/json/1" "ws://myIP:myPORT/notifications/ws/json/1"

	ProxyTimeout 86400

	# WebSocket specific handling
	RewriteEngine On
	RewriteCond %{HTTP:Upgrade} websocket [NC]
	RewriteCond %{HTTP:Connection} upgrade [NC]
	RewriteRule ^/notifications/ws/json/1/(.*) ws://myIP:myPORT/notifications/ws/json/1/$1 [P,L]

	RequestHeader set Connection "upgrade"
	RequestHeader set Upgrade "websocket"

        ProxyPass / http://myIP:myPORT/
        ProxyPassReverse / http://myIP:myPORT/
        ProxyRequests Off
        ProxyPreserveHost On

	Include /etc/letsencrypt/options-ssl-apache.conf
	SSLCertificateFile /etc/letsencrypt/live/remarkable.mydomain.com/fullchain.pem
	SSLCertificateKeyFile /etc/letsencrypt/live/remarkable.mydomain.com/privkey.pem

</VirtualHost>

Answer 10 · 2024-10-25T07:33:36.000Z

I'm using HAproxy within pfsense. What should I configure there?

Answer 11 · 2024-10-25T16:20:08.000Z

I'm using HAproxy within pfsense. What should I configure there?

You would need to look at the examples above for other technologies, and then reference the documentation for HAproxy to see how to mimic the configuration.