Feature request/doc request : Play all segments of a given speaker

Question

Feature request/doc request : Play all segments of a given speaker

PaulLerner opened this issue 5 years ago · 18 comments

Hi,
I'd like the "play" button of a given speaker in the "Segment Labeling" section to play all the segments of that speaker in a given order.
I should be able to implement it myself but I'm having problems diving into the code given the lack of technical documentation.

Bests,

Answer 1 · 2019-10-23T16:12:54.000Z

Hi,

You can do it in this way:

First, filter the regions with given speaker - you can do it with iterateRegions function, any region has a speaker param inside
Subscribe those regions to out event (wavesurfer regions doc)
Start playback from first region in filtered regions, and on out event move a playback to next region

I don't think this is the best solution, but i doesn't see others now

Answer 2 · 2019-10-24T09:25:16.000Z

Hi, thank you for your answer,

I'm having trouble with associating a function to to that play button. I've created a playSpeaker() function in the MainController class in app/controller.js. So I changed ng-click="playStop()" to ng-click="ctrl.playSpeaker()" but it doesn't have any effect (for now the function just logs a debug message). It's strange as any other change I make on app/controller.js code or app/playPartDirective.js is effective.

Answer 3 · 2019-10-24T14:11:30.000Z

ng-click="playStop()" relates to app/playPartDirective.js so you need to add playSpeaker() function to playPartDirective, not to MainController, and use it without ctrl. so it will be used like ng-click="playSpeaker()" - we don't use a controller-as syntax in this directive

Answer 4 · 2019-10-29T09:18:39.000Z

Ok so I replaced playStop by playSpeaker using the same syntax:

scope.playSpeaker = function (){
          console.log("in playSpeaker function ");
          var ctrl = scope.$parent.ctrl;
          var speaker_id=scope.$parent.speaker;
          console.log("speaker_id",speaker_id);
          ctrl.iterateRegions(function (region) {
            let current_speaker = region.data.speaker;
            if (current_speaker[0]==speaker_id){
              console.log("region of the same speaker:",region);
              region.on('out', function(e) {
                console.log("logging ",region," on out event");

                scope.rep.start=region.start;
                scope.rep.end=region.end;
              });
            }

          });
          scope.isPlaying ? stop() : play();

I'm not sure this is what you thought about but anyway I noticed a problem with the playStop function (i.e. the above function last line):

It doesn't work unless you first click somewhere
It behaves in a similary way as ctrl.playRegion(): no matter which speaker I click on, it only plays the currently selected region.
Only difference with ctrl.playRegion() I can see is that the cursor does not move so the 'out' event is never triggered.

Answer 5 · 2019-10-29T14:18:28.000Z

Hi Paul. Please try following code

scope.playSpeaker = function () {
    console.log("in playSpeaker function ");
                var ctrl = scope.$parent.ctrl;
                var speaker_id=scope.$parent.speaker;
                console.log("speaker_id",speaker_id);
                let firstRegion = null
                ctrl.iterateRegions(function (region) {
                    let current_speaker = region.data.speaker;
                    if (current_speaker[0]==speaker_id){
                        if (!firstRegion) {
                            firstRegion = region
                            region.play()
                        }
                    console.log("region of the same speaker:",region);
                    region.on('out', function(e) {
                        console.log("logging ",region," on out event");
                    });
                    }
                });
 }

For now, it will plays a first region of selected speaker. You can catch 'out' event and play a next region. If you need more info, please lmk. Thanks

Answer 6 · 2019-10-29T15:41:32.000Z

Hi, thank you for your answer.
The 'out' event doesn't fire sometime, I've read about a similar issue here

So when the region is done playing I have to use the ctrl.playPause() button. However, using this ctrl.playPause() button I almost have the behaviour I want after playing the next region (stored in a list) in the 'out' event as this :

scope.playSpeaker = function () {
    console.log("in playSpeaker function ");
                var ctrl = scope.$parent.ctrl;
                var speaker_id=scope.$parent.speaker;
                console.log("speaker_id",speaker_id);
                let firstRegion = null
                var speakers_regions=[]
                var i =0
                ctrl.iterateRegions(function (region) {
                    let current_speaker = region.data.speaker;
                    if (current_speaker[0]==speaker_id){
                        if (!firstRegion) {
                            firstRegion = region
                            region.play()
                        }
                        speakers_regions.push(region)
                    region.on('out', function(e) {
                        i+=1;
                        console.log(i);
                        speakers_regions[i].play()
                        console.log("logging ",region," on out event");
                    });
                    }
                });
 }

What's the difference between region.on('out' and source.addEventListener('out' ?

Answer 7 · 2019-10-31T15:33:33.000Z

Hi, thank you for your answer.
The 'out' event doesn't fire sometime, I've read about a similar issue here

So when the region is done playing I have to use the ctrl.playPause() button. However, using this ctrl.playPause() button I almost have the behaviour I want after playing the next region (stored in a list) in the 'out' event as this :

scope.playSpeaker = function () {
    console.log("in playSpeaker function ");
                var ctrl = scope.$parent.ctrl;
                var speaker_id=scope.$parent.speaker;
                console.log("speaker_id",speaker_id);
                let firstRegion = null
                var speakers_regions=[]
                var i =0
                ctrl.iterateRegions(function (region) {
                    let current_speaker = region.data.speaker;
                    if (current_speaker[0]==speaker_id){
                        if (!firstRegion) {
                            firstRegion = region
                            region.play()
                        }
                        speakers_regions.push(region)
                    region.on('out', function(e) {
                        i+=1;
                        console.log(i);
                        speakers_regions[i].play()
                        console.log("logging ",region," on out event");
                    });
                    }
                });
 }

What's the difference between region.on('out' and source.addEventListener('out' ?

Hi Paul

source here is a AudioBufferSourceNode, and it has only ended event listener (doc)

Answer 8 · 2019-11-04T11:29:36.000Z

Hi,
Thanks for the clarification. I was still not able to solve that out event that doesn't fire though, have you looked into the similar issue on wavesurfer git ?

I have a similar issue where I want to play a region in loop, I've modified the playRegion like this:

playRegion() {
        if (this.selectedRegion) {
            this.selectedRegion.play();
            var region= this.selectedRegion;
            region.on('out', function(e) {
                region.play();
                console.log("logging ",region," on out event");
            });
        }
        // play silence region
        else {
            var silence = this.calcSilenceRegion();
            this.wavesurfer.play(silence.start, silence.end);
        }
    }

But it behaves the same was I've described above : the 'out' event doesn't fire unless I press the play button manually.

Answer 9 · 2019-11-06T01:13:37.000Z

Hi Paul

I can't reproduce it on my machine

Please, give me a sample data (regions & audio file), or you have this issue on demo regions and audio?

Answer 10 · 2019-11-06T08:14:50.000Z

I have this issue on demo regions and audio...
What happens on your machine ?

I'm running on ubuntu 16.04 with npm 6.11.3 and node v12.11.1

running npm install gives me :

npm WARN speech-recognition@1.0.0 No repository field.
npm WARN speech-recognition@1.0.0 scripts['server'] should probably be scripts['start'].
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 15927 packages in 10.604s
found 0 vulnerabilities

Answer 11 · 2019-11-09T10:40:29.000Z

Hi Paul

On my machine (Ubuntu 18.04, npm 6.13.0 and node v9.6.1) i have a next:

npm WARN speech-recognition@1.0.0 No repository field.
npm WARN speech-recognition@1.0.0 scripts['server'] should probably be scripts['start'].
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 1063 packages from 536 contributors and audited 15927 packages in 37.103s

Answer 12 · 2019-11-12T14:47:10.000Z

Ok so the warnings look similar.
Have you tried the above code in your machine ?
Does the 'out' event fires consistently ?

Bests,

Answer 13 · 2019-11-21T12:52:15.000Z

Hi Paul

Sorry for a long answer

I get this bug too and found a solution, the code above can be used a start point for play all regions of selected speaker:

scope.speakerRegions = []

const playRegion = (currentRegion, nextRegionIndex) => {
	currentRegion.on('out', null)
	if (nextRegionIndex < scope.speakerRegions.length - 1) {
		const nextRegion = scope.speakerRegions[nextRegionIndex]
		nextRegion.on('out', () => playRegion(nextRegion, nextRegionIndex + 1))
		seek(nextRegion.start)
	}
}

const seek = (time) => {
	scope.wavesurfer.seekTo((time) / scope.wavesurfer.getDuration());
}

function play() {
	console.log('play', scope.wavesurfer)
	console.log("in playSpeaker function ");
	var ctrl = scope.$parent.ctrl;
	var speaker_id=scope.$parent.speaker;
	console.log("speaker_id",speaker_id);
	ctrl.iterateRegions(function (region) {
		let current_speaker = region.data.speaker;
		if (current_speaker[0] === speaker_id) {
			scope.speakerRegions.push(region)
		}
	});

	scope.speakerRegions[0].on('out', () => playRegion(scope.speakerRegions[0], 1))
	scope.wavesurfer.play()
}

You need to pass a wavesurfer from main controller, i.e. :

<play-speaker wavesurfer="ctrl.wavesurfer"></play-speaker>

We can't use a region play method, because it stops before an out event. When i tested it previous time, looks like i make a mistake, sorry.

Answer 14 · 2019-12-06T18:00:25.000Z

Hi, thank you for your answer.

I'm sorry but some things are unclear to me: where do I put all of this code ? In playPartDirective ?
The function play in there looks very different...
Also before we were talking about implementing the function as scope.playSpeaker = function () {}
not as function playSpeaker() {}

I don't understand what you mean by

You need to pass a wavesurfer from main controller, i.e. :
<play-speaker wavesurfer="ctrl.wavesurfer"></play-speaker>

Currently, the button is located in playPart and is like

<button class="btn btn-primary btn-sm" ng-click="playFirstRegion()" style="vertical-align: top; height: 23px">
        <i class="glyphicon glyphicon-play" ng-show="!isPlaying" style="vertical-align: top;"></i>
        <i class="glyphicon glyphicon-pause" ng-show="isPlaying" style="vertical-align: top;"></i>
</button>

Answer 15 · 2019-12-08T06:43:53.000Z

Hi Paul

Can you please give a link to your code (i mean an your fork), with your directive? I'll show where you will need to insert a code

re You need to pass a wavesurfer from main controller, i.e. - you need to pass a wavesurfer instance into your directive, because you need a direct access to wavesurfer object - you'll need to use play() and seek() functions

Answer 16 · 2019-12-09T08:28:35.000Z

Hi,

Here's the link to the current playFirstRegion function : https://github.com/PaulLerner/gecko/blob/pyannote/app/playPartDirective.js#L99
(Don't mind the playSpeaker and playStop functions)

Thank you for your help :)

Answer 17 · 2019-12-10T14:31:32.000Z

Hi Paul

Please check this patch, nvm re changes in config.js - i made it for our current regions format

Answer 18 · 2019-12-10T17:42:38.000Z

Thanks !
It needed some tweaks but it works 😁