[Feature Idea] Working with calls

Question

[Feature Idea] Working with calls

spontanurlaub opened this issue 4 years ago · 13 comments

It would be nice to work with calls using the bot api. Because streaming the data live would be quite complicated, I think it would be nice if you would be able to play audio (/video) files in a call and record the incoming audio (/video). That would allow a wide variety of interesting bots like a mailbox (if you don't answer the call within x seconds, it asks the user to leave a message that will then be sent into the chat as voice message), web-radio, voice assistant, alarm clocks with a real ringtone and much more

We would need to add libtgvoip to this project to support calls. Please leave your feedback.

Answer 1 · 2020-12-25T12:52:10.000Z

no libtgvoip should never be used programmatically outside the official clients because one wrong comma and everything crashes (in fact the dev doesn't work for telegram anymore), we should only implement the new library (https://github.com/TelegramMessenger/tgcalls)

Answer 2 · 2020-12-25T12:53:28.000Z

Okay, I was not up to date with that, using tgcalls would be fine

Answer 3 · 2020-12-25T14:38:46.000Z

We would need to stream the audio/video content from and to the server, that would put a lot of stress on the botapi and a lot of work

Answer 4 · 2020-12-25T14:45:24.000Z

It would be disabled by default of course and can only be activated with a command line comment. Allowing calls would let this api really stand out compared to most other userbot implementations

Answer 5 · 2020-12-25T17:40:09.000Z

@code1mountain How would you handle the communication between the client and the (web) server?

Answer 6 · 2020-12-25T17:40:43.000Z

websocket?

Answer 7 · 2020-12-25T17:43:42.000Z

I would not have any live communication from the server to the client. Just normal requests to play audio or start/stop/download the recording of the incoming audio/video stream.

Answer 8 · 2020-12-25T17:46:55.000Z

@code1mountain That is correct. AFAIK the only well-integrated and really usable implementations reside in MadelineProto and tdlib

Answer 9 · 2021-01-04T19:28:19.000Z

Okey, maybe we should see what use cases we could have?
Especially with those new group calls.

I can think of those:

Sending
- Internet Radio
- Sound boards
Receiving
- Recorde as MP3

I think maybe a mp3 upload for sending and an internet-radio like stream for download could be sufficient for a first version?
It wouldn't really support real time sending, but my thought would be that maybe with file_ids the playing could at least be timed properly enough?

Do we know what the resource usage could look like?
I imagine that continuous streams would be a whole different deal for the server, as that'll mean more CPU, network activity and assigned ports and all that.

Answer 10 · 2021-01-04T19:32:11.000Z

@luckydonald noice

Answer 11 · 2021-01-04T19:44:51.000Z

Alarms (scheduled wake up calls)
High priority notifications
Get-me-outta-here calls (fake getting an important call by push of a button)

All those just need the ability to trigger a call, not to actually transmit audio.

Answer 12 · 2021-01-04T20:18:27.000Z

We should also think about call events.

Currently that's just "text": "Voice Chat started" , and "text": "Voice Chat ended". Which I'd bet is an old tl-layer compatibility thing.
Maybe that's why #36 or rather the upstream update to the newest version is taking a bit, and we get that for free then.
Or they'll filter out those events. Let's see.

Answer 13 · 2021-02-08T13:46:29.000Z

As video chat is a thing, too, probably RTMP could make sense.
That's what big sites like Facebook, YouTube, Twitch and Periscope do for their live-streaming, so having something compatible with that would make integrations very easy.