Add the ability to send notifications to telegram topics

[almostinf]

FEATURE

Summary

Now Moira supports telegram sender, which knows how to send notifications to personal, private, public channels. We need to add the ability to specify specific channel topics

Research

As one option, add another way to enter a custom value in contact and on backend parse the channel id and a specific topic from it

Need to see how to send notifications by topic in the Telegram client we are working with

[Gaspero]

Some implementation considerations I came with during research:

• Telebot package should be updated to v3 (gopkg.in/telebot.v3) - update shouldn't be too hard.
• In telebot v3 message.ThreadID parameter is available. It stores message_id which indicated the first message in topic.
• ThreadID should be then stored Moira Contacts. It value should be used during message sending (SendOptions.ThreadID)
• Not sure, but it looks like thread name could be fetched via https://core.telegram.org/method/channels.getForumTopics method

[Gaspero]

During further research I found out that there is no method for fetching group topic name in Telegram Bot API and previously mentioned channels.getForumTopics is only available in TdLib (telegram client API) and thus not supported by telebot package.

I think that {groupName}/{threadID} name pattern for Moira delivery channel might be misleading and unobvious for Moira users.

I propose following implementation design:

• Moira user sends /start@MyMoiraBot {CustomName} command to the target Telegram group's topic where {CustomName} is either the full name of Moira delivery channel or postfix after groupName e.g. moira test group // my custom topic name
• Bot replies to the topic with the full name of Moira delivery channel
• Moire user then enters the full name of Moira delivery channel to the web app.

Waiting for the core development team decision.
Original question link in community chat: https://t.me/moira_alert/5606

[almostinf]

Hi, as far as we understood you correctly, you are confused by the non-obvious naming of contacts in UI, for example @test_channel/1234, by which it is difficult to understand to which topic the notifications will go, but this problem exists not only in topics, but also, for example, in private channels in Telegram, where the value of a contact can be, for example, 1494975744. Besides, custom aliases can be useful for other senders as well

I propose to solve this problem for both private channels and topics by adding an additional Name field in the contact structure, which will be used on the front end to display the custom name, thus solving the problem with non-obvious contacts, preserving backwards compatibility and not having to support mapping the topics name to the topics id. If you go this way, it would be better to do it in a separate PR, besides it would require some minor changes on the frontend, since now the default UI always shows contact values. If you want, you can take care of this problem as well, but we also can solve this quickly as well

The {group}/{threadID} convention in the contact values looks good

How do you like the idea? Did we understand the problem correctly?

[Gaspero]

I did some further research and I would like to clarify one question before we continue discussion on feature implementation details.

@almostinf could you please explain the background of the decision to first store telegram chatId via DataBase.SetUsernameID method as "moira-telegram-users:{ContactData.Value}":"{Telegram ChatID}" DB record instead of storing ChatID as ContactData.Value?

This requires additional additional DB call in sender.SendEvents to retrieve Telegram ChatID from ContactData.Value.

At first, I attempted to use this existing solution to enable custom channel names. But adding ContactData.Name for this purpose is obviously a clearer solution. However, it seems like we should keep existing logic for storing Telegram ChatID's anyways to preserve backwards compatibility.

[almostinf]

As far as I can see from the code, the DataBase.SetUsernameID method is used for mapping contact.Value to chatID of Telegram, it can be convenient for users, for example, to create a contact for personal chat, because instead of chatID it will be enough to specify @name, it would be good to keep this logic for backward compatibility, but we can consider other options with migrations if we can't do without it

[Gaspero]

I propose to solve this problem for both private channels and topics by adding an additional Name field in the contact structure, which will be used on the front end to display the custom name, thus solving the problem with non-obvious contacts, preserving backwards compatibility and not having to support mapping the topics name to the topics id

I agree that adding an additional Name field in ContactData struct is an optimal solution for supporting custom aliases.

Nevertheless, some additional changes should be introduced to be able to store Thread ID alongside Telegram Chat ID. And in my opinion current approach for storing Telegram Chat IDs could benefit from a more streamlined and organized structure.

I have vizualized current db schema and proposed design to be more clear.

Current db schema

Proposed db schema

Details

v1 will continue to use moira-telegram-users records to store Telegram Chat IDs

In v2 moira-telegram-users records will not be used. Instead, ContactData.Value will be used to store an object containing all the necessary contact information. Including Telegram (Telebot.ChatType), immutable Telegram ChatID and optional ChatID parameter.

This solution provides an easy way to differentiate between v1 and v2 data in ContactData.Value field, while preserving backwards compatibility, avoiding additional DB calls and unnecessary complexity in sender code.

I suppose that json object for ContactData.Value will be generated by Moira bot. Either by running /start command in target chat or by forwarding a message from the target chat (or private channel) to the bot.

The only drawback is that if ContactData.Name will not be set by the user, then json object form ContactData.Value field will be displayed as a notification channel name.

What do you think?

[Gaspero]

We had a call with Moira core development team to discuss proposed db schema.

Summary:

1. Adding Name field for Moira Notification channels is out of the scope of this feature - I've created a separate issue #1018
2. We have agreed upon the following db schema
   
   Json object will be stored as a value of moira-telegram-users record. Implementation should also include db migrations for all moira-telegram-users records to have the same structure.

[Gaspero]

Wrapping up my research on how Telegram chat information is currently stored.

I've created 4 delivery channels:

• group chat
• personal chat
• private channel
• public channel

Delivery channel display name	Delivery channel type	moira-telegram-users	moira-contact
my group chat	group chat	key: moira-telegram-users:someteam / testmoira value: -10021****6791	key: moira-contact:38b74b16-fa5c-47b8-b9a5-fe2060893710 value: {"type":"telegram","value":"my group chat","id":"38b74b16-fa5c-47b8-b9a5-fe2060893710","user":"anonymous","team":""}
@my_tg_username	personal chat	key: moira-telegram-users:@my_tg_username value: 74****70	key:moira-contact:2ea0f19e-925d-48c4-8c3f-37ec0eb5cac2 value: {"type":"telegram","value":"@my_tg_username","id":"2ea0f19e-925d-48c4-8c3f-37ec0eb5cac2","user":"anonymous","team":""}
%21*****532	private channel	N/A	key: moira-contact:0536705c-3bbc-40c1-b047-caf3ff2b3861 value: {"type":"telegram","value":"%21*****532","id":"0536705c-3bbc-40c1-b047-caf3ff2b3861","user":"anonymous","team":""}
#my_public_channel	public channel	N/A	key: moira-contact:0536705c-3bbc-40c1-b047-caf3ff2b3861 value: {\"type\":\"telegram\",\"value\":\"#my_public_channel\",\"id\":\"e38631bd-2ee9-44e2-ae6b-0b47f6cd8d52\",\"user\":\"anonymous\",\"team\":\"\"}

This demonstrates that:

• both group chat and personal chat chatIds are stored as moira-telegram-users records
• private channel values are mapped to Chat IDs directly (see send.go) and are not stored at moira-telegram-users records.
• public channel values are used "as is" and their chat Ids are not stored at moira-telegram-users records as well.

[Gaspero]

@almostinf I've implemented the feature, but I still have to fix tests in senders/telegram/send_test.go.

Before opening PR, I wanted to clarify if it should include migrations: as far as I can see from the code, migrations are bound to particular release version. Should I create migration file like cmd/cli/from_2.11_to_2.12.go and include it in PR along with necessary cli package changes?

[almostinf]

Yeah, you can do that