[Privacy Issue] RAW recordings are created and stored, even if the meeting isn't recorded
ichdasich opened this issue · 59 comments
Describe the bug
Raw recordings are stored even if no recording button is pressed.
To Reproduce
Steps to reproduce the behavior:
- Start a meeting
- DO NOT Click on 'Record meeting'
- End the meeting
- Find RAW files of the session on the BBB server
Expected behavior
Sessions during which the recording button is not pressed are not stored.
Actual behavior
Sessions during which the recording button is not pressed are stored
Additional context
I am not sure if I am holding it wrong, but i migrated to a new BBB setup, and moved recordings. There, I found that reencoding all recordings would make recordings of non-recorded meetings show up in my users' accounts. This obviously caused unrest and is an (unintended) privacy violation on my side.
This should:
- Not be default behavior
- Be clearly documented
- Possible to configure off
To clarify: I understand that there might be a reason to do this temporarily; However, if there is no recording-press during a session, the RAW data should be deleted when the session ends.
This is not a bug, but instead this is a documented feature and design decision. See https://docs.bigbluebutton.org/dev/recording.html
Related to bigbluebutton/greenlight#1163 (comment)
You can drop a script in the post_archive dir to check if there are recording marks. If none, delete the recording.
/usr/local/bigbluebutton/core/scripts/post_archive
Here is how to check.
This is still on purpose (although not a good default in my opinion).
There already exists a script in /etc/cron.daily/bigbluebutton which deletes the raw recording data 14 days after the recording ended by default. This allows one to create a recording of the session even when the record button was not clicked. Anyways, when starting a new session, the bigbluebutton API call specifies if recording is desired or not. This allows the frontend to handle notifying the user of data being recorded / ask for permission or not enable the recording feature at all. To be GDPR compliant on a public server which allows guests, you have to set disableRecordingDefault to true anyways (Edit, this was true back in 2020, but not anymore nowadays, when the system is properly set up/adjusted).
Btw, closing this as duplicate of bigbluebutton/greenlight#1163
@ichdasich : Please consider reopening this issue.
IMHO this issue relates to the UX of bbb's HTML5 client. Typically, recoding user content (e.g. webcams) requires consent in many situations, whereby users has to be informed about recordings going on.
However, the current UX of the HTML5 client suggests, that no recording is done if the record button is grey / white.
Based on the old red recording light in a TV studio, users expect a recording to be present, when the button is red, whereas a grey / white button suggests, that no recording is taking place. But: In the background, recording takes place nevertheless, as long as record=true is specified when creating the room.
I'd suggest to change the UI of the HTML5 client.
- The record button has to be red and looking activylish when recording is enabled on a session basis.
- In addition, the the insert begin/end marker action should be addressed by a button labeled as "Insert Begin Marker" (or "insert end marker" resp.)
Hey @yanosz,
I agree with you there. I though the other ticket was indeed also in BBB, not in GL; For greenlight, there should be a specific interaction. I will open a new ticket in the GL repo for the interaction changes for joining and starting rooms.
Reopened. UX for GL considered in bigbluebutton/greenlight#1395
For BBB, it might be nice to change the meaning of the recording button:
make it always red (different shades for set-start-marker/set-end-marker) and reword the recording notion around "Setting start-marker/setting end-marker"
User stories:
As a moderator, i want to use a button labled 'start recording marker' to set a start recording marker if none has been set yet.
As a moderator, i want to use a button labled 'stop recording marker' to set a stop recording marker, if a start recording marker has been set and no stop recording marker is present thereafter.
I want to second this strongly, since the GDPR (DSGVO in german) requires everyone to specifically consent for userdependent data to be saved.
Saving the videostream even temporarily, when the UI suggests that nothing(!) is being actively recorded, could (and IMHO should) be interpreted as an active lie.
@marcarneg regarding consent for video recording, please see bigbluebutton/greenlight#1163 - if you use some other frontend, the consent-text/check has to be done in that specific frontend before the user joins the session.
Actually bigbluebutton/greenlight#1163 bigbluebutton/greenlight#1395 has been closed as a duplicate of that.
@basisbit This is still impossible to sell to people, when security of personal data is important to them. Yes, recordings of session should be possible (and therefore greenlight should correctly point that out before the session starts), but if the moderator promises (and the html5 frontend signals), that the session is not being recorded, it's very important to make sure, thats totally true.
In our case, we even don't want participants to create a greenlight-account on our server (they're joining via url+pin)- a saved video on the harddisk without a direct consent or at least notification to the users could send a complicated (and destructive) message.
@marcarneg As a hot fix in your situation: If you do not need recordings, globally disabling should be relatively straight forward (which then also prevents the recording of the temp-files.) This then only might (didn't check) leave temp files in, e.g., freeswitch.
In general, I fully agree with your comment. My todo currently has implementing a consent switch on the room-join form (for my own fork of GL; Not really prod-level code.), also detailing this.
If you don't need recording, set disableRecordingDefault to true in /usr/share/bbb-web/WEB-INF/classes/bigbluebutton.properties
If you need recording capability, you could have your teachers / presenters ask every participant of the session, and if they agree, then the presenter can use OBS to record the screen.
(Personally, I'd say that every public instance of BBB that uses Greenlight and has not disabled recording either in Greenlight code or in BBB config, is not in compliance with GDPR. There was plenty of discussion about this already in the previously mentioned issue, so let's not continue it here, please.)
In our case, we even don't want participants to create a greenlight-account [...]
@marcarneg They don't have to create a greenlight account. The session-join-link shows a "join this session" page. Exactly this page of your frontend should be extended to ask the participant to accept the recording legal information if record=true is set for this session.
A naïve question: I get that BBB records everything temporarily because it makes recording easier. But why does it hang on to these temporary files for 14 days? I think a way saner, and more defendable default, would be to allow recordings, using the same infrastructure as now, but immediately cleanse the temporary files after the postprocessing after the end of a meeting is finished.
I actually configured my installation like that, and put all temporary directories/the recording directories on TMPFS. (very nasty solution adding sudo access to bbb-record for the bbb-user and calling a record-delete if there were no recording markers.):
/etc/sudoers:
bigbluebutton ALL = NOPASSWD: /usr/bin/bbb-record
and
/usr/local/bigbluebutton/core/scripts/archive/archive.rb (after line 242):
BigBlueButton.logger.info("There's no recording marks for #{meeting_id}, deleting the recording.")
system('sudo', 'bbb-record', '--delete', "#{meeting_id}") || raise('Failed to delete local recording')
I suppose (but don't know) that this feature comes from an environment where people occassionally forget to hit record, and then panic-y write to the IT that they forgot things... and demand it to be restored. ;-)
I've created a more extensive another version of the change in archive.rb. I didn't want to keep the temporary files anywhere so the changes to tmpfs are not required in my version. [Disregard that. I should've read the bbb-record code beforehand]
It's not pretty but it works on our server. It also allows automatic cleaning of previously generated archives.
A naïve question: I get that BBB records everything temporarily because it makes recording easier. But why does it hang on to these temporary files for 14 days? I think a way saner, and more defendable default, would be to allow recordings, using the same infrastructure as now, but immediately cleanse the temporary files after the postprocessing after the end of a meeting is finished.
I've been using BigBlueButton since 0.81 and one of the most common requests I get is "I forgot to click (or misclicked) on the recording icon, can you still retrieve it?". Usually followed by a "It is a really important meeting".
So yeah, I love that it keep temporary raw files for a potential rebuilding. I don't wish this to go away.
But, sure, a better indication that the session is being recording regardless is welcome. (Maybe when you activate your webcam?)
I agree that having the ability to record post-meeting is a nice option – but it should be that, an option, in my humble opinion.
And it has to be implemented so that it actually is easily legal usable in most of the world: that requires asking for permission before the user joins the meeting. Thus, asking just before webcam is activated is not a suitable solution.
And it has to be implemented so that it actually is easily legal usable in most of the world: that requires asking for permission before the user joins the meeting. Thus, asking just before webcam is activated is not a suitable solution.
Indeed, asking before joining makes way more sense.
I would really appreciate if this is NOT going to be solved simply by putting various consent stuff around. People are asked for consent for each and every mouse click in the internet nowadays, nobody will ever read all that shit.
At the moment, the administrator has only two options: record all and everything or disable recording altogether. If there would be a third option: "record when record button is pressed", this would be a great thing.
In terms of recordings: I think recording (and asking for consent) if the session is set for record=true, and not asking for consent and not recording when the room is set to record=false is the best way to go. Recording everything and then cutting later should be fine, as the temp-data is needed for technical reasons, as long as it is deleted when it is no longer technically needed, i.e., when the recording has been processed and cut. So I doubt that the architecture has to be changed.
Same with logs. The log level, retention etc. can be controled by the operator of a BBB instance, so I don't thing that this needs changes in BBB.
What would, in general, be nice would be a documentation piece on 'running BBB GDPR compliant', that details all the necessary steps, as already now most of the considerations are in how you configure it. Could also come with a sample privacy policy; I will try to drag in experts who can help on that. :-)
100% agree to all what @ichdasich wrote there. +1 We definitely need the Greenlight changes soon, to be able to keep customers in Europe or providers of international services happy.
just created an issue in the repository of docs.bigbluebutton.org so that we can create a documentation piece on 'running BBB GDPR compliant'
bigbluebutton/bigbluebutton.github.io#136
Please help by adding your best practices suggestions there!
Dear all,
I am following this issue as well. You can imagine that many data protection authorities evaluate these days video conferencing solutions for their privacy properties. NGOs were first to publish comparisons.
When EU private and public sector operate BBB, they are the controller and responsible for the recording and storing of BBB. The current recording behaviour is indeed problematic:
- data protection by default (GDPR art. 25): why isn't BBB’s default to not record sessions
- consent for recording (GDPR art. 7): If consent is the legal basis used for recording sessions, then the controller must be able to explain how consent was obtained. Perfect behaviour would be to record only from those who consented and let others still join the room. Otherwise, people can argue they were forced to consent to join the room.
- personal data in log files is a different story: GDPR gives some options when some safeguards are in place.
- transparency is key (GDPR art. 12 and art. 13): of course also without recording sessions, personal data is processed. Before people enter the room, they should be informed about it and give consent. Best practice is to use layered information: have the essence in one paragraph and offer a link to the full data protection notice.
Unfortunately, I cannot give legal advice and answer specific follow-up questions in this issue. Though there is a mailing list with knowledgeable people: EDPS IPEN Network
@ffdixon fyi, this topic is getting lots of attention here ein Europe. Could you maybe help improve the documentation on this matter?
(I am not a lawyer, I can not give legal advice, this is just my understanding of the matter. Also, I am not an official member of the BigBlueButton project, instead I am only a contributor)
@rriemann-eu thank you a lot for your help regarding this important topic and the provided information!
In general, BigBlueButton is a server software, not a service, thus every operator of a service (which might be based on the BigBlueButton software) has to operate their service in compliance to whatever laws are to be applied for their service (for example the various local variants of GDPR).
The default of BigBlueButton is to not create or store any recordings. When a room/session is created, the Frontend (Moodle or Greenlight or other plugins) can specify if recording should be done for the room or not. For the Greenlight Frontend (which by default instructs BBB to create recordings for every room), there are improvements currently being done to make it easier for server operators to add consent-checking and displaying details before a user joins the room and also to have the person who created the room specify if recordings are needed or not. That is part of the Greenlight software project and is tracked here: bigbluebutton/greenlight#1163
All the parts of BigBlueButton which create server-logs can be changed to comply with GDPR by the server operator - for example logging can be limited to only log technical critical problems or not create any logs at all. There also exist lots of solutions for anonymizing logs if necessary for the service operator.
Of course, the easiest way to operate BigBlueButton in compliance with GDPR is to not "store" any personal data (or other data which would require consent or laws permitting the storage) on the servers by disabling all logging and disabling the recording feature for the BBB server.
Anyways, here are some steps which in my opinion should be done in the next days/weeks to improve the current situation:
- add best practices and easy to follow instructions on how to adjust logging in BigBlueButton, how to add Terms of Service and similar legally required information to the Greenlight frontend
- add a note in the install instructions that the server operator has to consider
- create documentation on where, how, for what and for how long personal data might be stored / processed in a default installation of BigBlueButton, so that server operators can use this documentation to create their own service documentation much easier.
Hey guys I strongly agree with @rriemann-eu . Being a GDPR consultant and (quite) happy user of BBB I think this is quite an urgent issue.
Just adding a consent banner to "legalize" the current status quo is not a good move as I can think of many use-cases where no consent would be required to participate in the meeting at all. Now adding a consent box just for this purpose feels redundant - especially if no recording is intended by the host.
And let me add that even with a consent box implemented this processing is still at least problematic if you ask me. I can think of many situations where I'd actually object that the consent you plan to implement might be legally binding. Also one would have to take transparency problems into account (Art. 12 ff GDPR). And we haven't even looked into the principles of privacy by-design and by-default ...
Given that many organisations in the EU are using BBB (not to include the ones abroad processing data of EU citicens) this thing is quite huge. Also I would expect that many users are not aware of the problematic and probably unlawful processing happening in the background of their self-hosted BBB server. So I think this is nothing that just affects some crazy EU privacy guys ...
Don't get me wrong, I can fully understand that you probably had good reasons for this design decision and I'm pretty sure that this feature might be really handy and probably also required for operators in some organisations. Still I think that this should not be the default.
I would love to see an option that would just record audio/video when the record button is actually activated. Alternatively at least purge the session data when the meeting has ended. I think that this sort of temporary "caching" could probably be justified even though it would be better if no data would be stored at all (unless required for meeting operation, logging/auditing and things like that of course).
And for those who do actually need the post-meeting recording the mentioned terms and consent window could be added... So I think the proposed greenlight changes are a step in the right direction.
Regarding logging let me add that I can think of situations where one may be legally obliged to store logs to ensure secure operation of the service under GDPR. Personally I think that there should be just the logging you need to ensure you can control attacks, abuse, brute-force attempts and the like.
To add on this: It seems like if recordings are disabled (record=false) bbb still stores the presentations of sessions.
There now exists documentation on what configuration can/should be changed on BigBlueButton servers to make it easier to operate a GDPR compliant service: https://docs.bigbluebutton.org/admin/privacy.html
Further improvements that are being worked on:
- simplify installation of BigBlueButton servers which don't store logs or temporary files that might include private data (bigbluebutton/bbb-install#202)
- make recordings of a session started by Greenlight optional (bigbluebutton/greenlight#1163)
- improve configuration defaults of other software projects which BigBlueButton uses
- add best practices to the Privacy documentation
- improve default configuration of BigBlueButton
@ffdixon Apart from GDPR violations, recordings of video and audio without consent could also possibly be considered a criminal offence in Germany according to § 201 StGB or §201a StGB (https://twitter.com/sayho said so in a twitter conversation and I personally agree with him).
So anybody running a BBB server in Germany who has not disabled all recordings could potentially commit a criminal offense.
As a user I expect that I am not recorded unless the (visible) recordings button actively shows me that a recording is in progress. A consent button where users agree to recordings of a meeting even though the recordings button hasn't been activated is a workaround. And not a good one.
Taking Art. 25 GDPR into account as well, I think you should re-consider the way recordings are handled in BBB. A few users asking to retrieve recordings because they forgot to click on the record button is not a good enough reason for the current recordings implementation.
Why not give admins the option to only start recordings when the recordings button was pressed? Yes, that may cause some delays in the recording but it makes it much easier to comply with legal requirements.
@SiljaJ of course operating any service which is not GDPR compliant is a crime here in the EU. You get that problem with most server software solutions in default configuration. If you set up your own service, you should always know what you are doing or you risk huge fines. There exists documentation for server operators on how to set up your server in compliance with GDPR which can be followed within 15 minutes.
Regarding your suggestion about giving server administrators a configuration option for how the recording feature should be done: This is a free/libre open source project. If you need this feature with high priority and just waiting for some other developer to implement this in their free time is not good enough, then you should hire a software developer to do the desired changes and then submit these changes to the project so other users can also use this new feature.
Is this setting also used on demo.bigbluebutton.org?
I did not a find a document describing the privacy policy (at least prior to registration).
If everything on demo.bigbluebutton.org is recorded in a covert way this is a major legal and trust issue. Especially as it isn't hosted in Canada but obviously in the US on Amazon Servers.
@bpcurse a) there is no recording enabled on the demo server, b) it is a demo server, c) do not use the demo server for production, d) this is a demo server of a canadian company
Please stop trolling and actually help solve the problem by providing code. BigBlueButton is a free/libre open source software development project.
@bpcurse a) there is no recording enabled on the demo server, b) it is a demo server, c) do not use the demo server for production, d) this is a demo server of a canadian company
Please stop trolling and actually help solve the problem by providing code. BigBlueButton is a free/libre open source software development project.
Glad to hear that default settings are not applied to the demo server 👍
Please don't act like a fool and accuse me falsely of trolling. I really like this great open source software and will contribute code if I am able to. The aim of my comment was to point out that there might be a yet unseen problem and keep the company behind it out of trouble.
From a legal point of view it makes no difference how you call it, or how it is used (production system or not). Covert recording is a crime in many countries and european GDPR regulation sets fines up to 20 Mio. EUR or up to 4 % of the total worldwide annual turnover of the preceding financial year, whichever is higher. Really nothing to joke about.
Please stop trolling and actually help solve the problem by providing code. BigBlueButton is a free/libre open source software development project.
@basisbit Nobody is attacking anyone. I think it's great that you contribute to BBB. Thank you for that! Everyone who participates in any project in their free time has my absolute respect.
However, BBB is not a small hobby (side) project but has a business behind it. Blindside Networks offer professional services and hosting. Therefore they have a responsibility to make sure their software complies with legal regulations. The fact that Fred Dixon is named as the data protection officer for Blindside Networks tells me that there might be some misunderstandings in GDPR requirements. A CEO can never be the DPO of their own company.
People who criticize BBB in this issues don't just troll but point out flaws and try to offer constructive solutions. From what I understand the way recordings are handled is a design decision. From a legal perspective I'd wish that this decision was revised or at least another configuration option would be considered.
Yes, forking the project is an option. But if changes to the code are not pulled into main code because of a design decision like this one, it doesn't change the fundamental issues.
@SiljaJ Thank you for your feedback! Could you please send this information to Fred Dixon and resolve the hosting issue outside of this project here? This issue tracker is for the BBB software development project only and thus we should at least try to keep focused on the software development project, not the hosting solution which Blindsidenetworks does. Also, this is getting off-topic, please let's try to keep focused on the main issue which was posted by ichdasich
@basisbit I think the majority of participants here love BBB and want it to be as good as it possibly can. And yes, if we don't receive an official response here, we will try to contact Fred Dixon. The issue is too important to be neglected. Nor is disabling recordings globally an acceptable solution. However, I can imagine that the company is currently under heavy stress due to the global situation and the rocketing demand of meeting solutions. Hopefully, they can find the time to respond to this before public awareness of this problem becomes even bigger. (There has already been a discussion about this on twitter between some German IT lawyers.)
It would be nice to receive some acknowledgment from @ffdixon concerning this issue. If Blindside Networks also sees a necessity to improve this but has no time to provide a quick resolution, we are willing to let our devs look into possible technical solutions. But since we are also running a business, we are only willing to commit resources if Blindeside Networks is also interested in finding a solution for this issue.
The current recording implementation is sub-optimal and can be seen as disappointing or misleading users. Instead the decision should be less restrictive and delegated to the operators. They should be able to choose between the current method and an on-demand method. Currently having to disable recordings for GDPR reasons is frustrating due to the loss of this valuable feature.
If GDPR compliance is required, I think anyone affected can compromise by loosing a few seconds during the recording startup.
To not disappoint the recording users expectation, the potential frustration could be omitted/calmed by providing a countdown. For instance when pressing the record button it could show a countdown reflecting the remaining seconds until the recording will begin. This may also give the attendees time to "adjust" and "warn" them of an impending recording.
Later, when the various recorded items/streams are processed by the server, initial seconds where not all streams are available yet, could be removed. Hope this makes sense...
I am the Product Manager for BigBlueButton and have been managing the project since 2008.
Thanks for all the comments in this thread. We care about data privacy and, as an open source project, its clear that others do so as well and want to help to improve BigBlueButton, both from a compliance and code perspective.
Right now, as pointed out, BigBlueButton either (a) records the content shared in the session, and gives the instructor the ability to mark segments of the conference (by pressing the Start/Resume record button) for processing into recordings, or (b) does not record the session.
For (a), the content can be automatically removed at some point after the session ends. Nonetheless, content is recorded and retained on the server until removal.
The better solution would be to start with (b) and when Start/Resume record button is first pressed then, and only then, start recording content. I don't yet have a sense yet of the scope of changes needed to support this approach.
Today, if you don't want content recorded, the API call that creates a meeting can p[ass record=false. Currently, we don't expose this option in Greenlight, but that would be easy to add. But once the meeting starts, this value is set in stone -- it's either (a) or (b).
We care about privacy of data. There is more work to be done. We welcome contributions -- both discussions and code -- from others in this area.
@ffdixon Thank you for responding so fast!
Does this mean the issue can be reduced to the usage of Greenlight? If third-party implementations initiate a meeting via the API with record=false it actually obeys this and does not silently record the meeting in the background? If so, can we still mix third-party implementations and the usage of Greenlight without reintroducing the issue in meetings which have been created by third-party solutions?
If third-party implementations initiate a meeting via the API with record=false it actually obeys this and does not silently record the meeting in the background?
This is correct.
There already exists an issue for changing that behavior of Greenlight: bigbluebutton/greenlight#1163
BBB only records the meeting if the record button (pressed or unpressed) is "visible" (for all participants).
It depends on how the third-party implementations create rooms. Rooms are stateless, so even if you use greenlight, and a third party app creates rooms with record=false, those rooms wont be recorded (and neither will have the option to be recorded at all). However, if these third-party apps use record=true, they will lead to recordings, even if greenlight is not used.
The code basibit referenced for greenlight actually works rather well. I am running it in production for some time now, and it, well, works.
@basisbit Thanks for clarifying. I did read the sections before but was unsure until now. Since we are integrating BBB meeting management into our main software, this eases our pain.
I assume it should be possible to add an option to Greenlight which allows us to disable recordings for a specific room. Without having looked into the Greenlight code this may be the quickest solution. Has anyone created a pull request for such a change yet? If not, I think we will look into it.
As @basisbit said, there is bigbluebutton/greenlight#1163
Ok, this seems to be the closest match: bigbluebutton/greenlight#1296. If finished, it may help a bit. However, we still need to find a solution that only records when users actively click the "Start recording" button. Pausing also needs to physically stop recording. I assume that some content will also be missing whenever the recording is resumed...
I do not understand ichdasich's suggestion to add the mentioned lines to archive.rb after line 242. My line 242 is
else
and that else relates to line 226
if not archive_has_recording_marks?(meeting_id, raw_archive_dir, break_timestamp)
So, should the suggested lines not be inserted BEFORE line 242?
Please update the documetation, so that the european customers can make changes in their /usr/local/bigbluebutton/core/scripts/archive/archive.rb. My ruby experience is pretty bad.
Thank you :)
Are you still facing this issue? Nearly been a year, Please let me know. We might ship the fixed version (Releasing version 3.4 by 1st October 2021) of the BigBlueButton which has been re-branded and is still under process... On our Fork, we have seemed to fixed this issue. In case, you need help , I am happily willing to contribute my code.
@vagbox ??? I suggest reading the last couple of messages instead of randomly mentioning a seemingly non-existing fork from a github account with close to zero participation in anything BigBlueButton related.
The typical frontend (Greenlight) has been adjusted so that a session creator can choose if a session should be recorded or not. There exist adjustments for showing a notification text before joining a recorded session in order to comply with certain legal requirements that exist in some countries. This issue here is not closed because a rewrite of the recording feature would still be a nice and desirable improvement, so that it is easier to use the recording feature in countries with very strict data protection requirements.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This should really be handled as a critical security issue (partly to you, bot!).
@rugk see discussion above. This issue was already handled quite a while ago and is not critical any more at all. The current implementation is just not yet as good as desired, thus it is kept open until a bigger rewrite of it is done.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.