Event messageDeleted not raised

[rdonadono]

Hi,

I set up a Proton Mail account and created a local instance of EmailEngine with docker.

Everything seems to work correctly except for the messageDeleted event not being raised when a message is deleted.

From the log it appears that EmailEngine realizes that something has been deleted but nothing else happens at the webhook level.

emailengine-1                  | {"level":20,"time":1720044502256,"pid":6,"hostname":"bd8744bbd129","tid":4,"component":"imap-client","account":"test-account","cid":"eemctr5q4n7mmt5cbimy","sub":"imap-connection","channel":"primary","src":"s","msg":"* 345 EXPUNGE","cid":"eemctr5q4n7mmt5cbimy:m:0"}
emailengine-1                  | {"level":10,"time":1720044502257,"pid":6,"hostname":"bd8744bbd129","tid":4,"component":"imap-client","account":"test-account","cid":"eemctr5q4n7mmt5cbimy","sub":"mailbox","path":"INBOX","msg":"Untagged EXPUNGE","path":"INBOX","seq":345,"vanished":false}

Events such as messageNew are correctly raised instead and I have correctly enabled them all from the admin panel.

Is there something I'm missing?

[andris9]

Each email server has its quirks. ProtonMail does not have an official IMAP server, so EmailEngine is not tested against it, and thus, it is not "officially supported."

[andris9]

I just tested it, and it did not seem to have any issues. EmailEngine detected the expunged message and generated a webhook for it. Maybe you have message deletion webhooks disabled?

Here are EmailEngine logs for the expunged email:

{
  "level": 20,
  "time": 1720077335542,
  "pid": 32489,
  "hostname": "andrismacbook",
  "tid": 2,
  "component": "imap-client",
  "account": "proton",
  "cid": "13u4d4fux1x2v4r083xi:m:0",
  "sub": "imap-connection",
  "channel": "primary",
  "src": "s",
  "msg": "* 3 EXPUNGE"
}
{
  "level": 10,
  "time": 1720077335543,
  "pid": 32489,
  "hostname": "andrismacbook",
  "tid": 2,
  "component": "imap-client",
  "account": "proton",
  "cid": "13u4d4fux1x2v4r083xi",
  "sub": "mailbox",
  "path": "INBOX",
  "msg": "Untagged EXPUNGE",
  "seq": 3,
  "vanished": false
}
{
  "level": 20,
  "time": 1720077335547,
  "pid": 32489,
  "hostname": "andrismacbook",
  "tid": 2,
  "component": "imap-client",
  "account": "proton",
  "cid": "13u4d4fux1x2v4r083xi",
  "sub": "mailbox",
  "path": "INBOX",
  "msg": "Deleted",
  "uid": 4
}
{
  "level": 10,
  "time": 1720077335555,
  "pid": 32489,
  "hostname": "andrismacbook",
  "tid": 3,
  "msg": "Processing webhook",
  "action": "webhook",
  "queue": "notify",
  "code": "processing",
  "job": "162",
  "webhooks": "https://webhook.site/***",
  "accountWebhooks": false,
  "event": "messageDeleted",
  "data": {
    "serviceUrl": "https://emailengine",
    "account": "proton",
    "date": "2024-07-04T07:15:35.549Z",
    "path": "INBOX",
    "specialUse": "\\Inbox",
    "event": "messageDeleted",
    "data": {
      "id": "AAAAAQAAAAQ",
      "uid": 4
    },
    "eventId": "2d4c1719-5c5a-41d7-986b-531a54975cac"
  },
  "account": "proton"
}
{
  "level": 10,
  "time": 1720077335871,
  "pid": 32489,
  "hostname": "andrismacbook",
  "tid": 3,
  "msg": "Webhook posted",
  "action": "webhook",
  "queue": "notify",
  "code": "result_success",
  "job": "162",
  "webhooks": "https://webhook.site/***",
  "requestBodySize": 195,
  "accountWebhooks": false,
  "event": "messageDeleted",
  "status": 200,
  "account": "proton"
}

[rdonadono]

The version I use is v2.42.0 on Docker and the events are all subscribed.

The version of the Proton Mail Bridge is v3.8.1.

Is there anything I can test that can help me understand why it is not being raised?

Thanks!

[andris9]

The main reason might be that the EmailEngine's internal index gets into an inconsistent state (maybe bugs in the bridge, etc.) and cannot determine which email corresponds to the EXPUNGE sequence number. IMAP server always informs the client that message number X was deleted, but the email client must know what that X actually means. EmailEngine uses an internal index that should match message sequence numbering in the server at all times, but if the server is providing inconsistent notifications, then, over time, that index might go out of sync with the state of the server.

What you can do:

1. Set up eerawlog utility to log actual IMAP transactions between EmailEngine and IMAP server
2. Flush the index of the account in question (Request account flush)
3. Wait until the index has been resynced
4. Delete an email in the Protonmail account and see if it triggers the event
5. If it does not, then check the IMAP log from eerawlog tool to see if all operations from the server look consistent

Also, check other kinds of accounts. Does EmailEngine trigger deleted message webhooks for other accounts and only fail to do so for the Proton account, or is it not sending these at all?