How does matching work?

[RWOverdijk]

This might be a dumb question, if so I am sorry.

I looked through the code and the example and am now properly confused.

What I'm doing to test (albeit a bit awkwardly) is piping my microphone input using ffmpeg, to an example file with node, but I notice that the fingerprints very rarely match those that I generated based on the source file.

The way I thought it worked was:

1. Create fingerprints for a source file (I used the example in the readme)
2. Create fingerprints for the input (for example microphone) and match against against previously generated fingerprints.

Like I said, I rarely have any results. Do I need to make it more sensitive? If so, how would I do that? Or am I not supposed to use my microphone as input with this lib?

Thanks!

Update:

If it helps, here's the code I'm using (a mess because it's just me going nuts):

var decoder = require('child_process').spawn('ffmpeg', [
	'-i', 'pipe:0',
	'-acodec', 'pcm_s16le',
	'-ar', 22050,
	'-ac', 1,
	'-f', 'wav',
	'-v', 'fatal',
	'pipe:1'
], { stdio: ['pipe', 'pipe', process.stderr] });

process.stdin.pipe(decoder.stdin);

var Codegen = require("stream-audio-fingerprint");
var fingerprinter = new Codegen();
decoder.stdout.pipe(fingerprinter);

const buildDB = process.argv[2] === 'db';
const live = process.argv[2] === 'live';
const database = [];
const db = buildDB ? null : require('./db.json');

fingerprinter.on("data", function(data) {
  const suspects = [];

  for (var i=0; i < data.tcodes.length; i++) {
    if (buildDB) {
      database.push({ time: data.tcodes[i], fingerprint: data.hcodes[i] })
    } else if (live) {
      suspects.push(data.hcodes[i]);
    } else {
      database.push(data.hcodes[i]);
    }
  }

  if (live) {
    const res = db.filter(x => suspects.indexOf(x.fingerprint) > -1);

    console.log({ tada: res });
  }
}).on('end', () => {
  if (buildDB) {
    console.log(JSON.stringify(database));
  } else {
    const db = require('./db.json');
    const results = [];

    const res = db.filter(x => database.indexOf(x.fingerprint) > -1);

    console.log({ tada: res });
  }
});

The way I run it is:

1. Store above file as index.js in a directory.
2. Make sure there's a source .wav file in the same directory. Let's call it my_file.wav
3. Make sure you have this file on an external device for playing. I have it on my phone (iPhone 11 pro)
4. "Index" the audio file: cat my_file.wav | node index.js db > db.json
5. Pipe your mic into the file: ffmpeg -f avfoundation -i ":0" -ar 22050 -acodec pcm_s16le -v fatal -f wav pipe:1 | node index.js live
6. Start playing the source audio on the external device (again, I'm playing it on my phone).
7. Observe some matches, very rarely (example: { tada: [ { time: 19433, fingerprint: 139300 } ] })

[bioleyl]

Hello RWOverfijk

Have you continue working on the code? I tried it but it's hard to find a concrete match on an audio track.

[dest4]

If I understand correctly, you play audio on your phone and record what its microphone catches. This degrades a lot the audio!
You will want to tune the fingerprint parameters so that the algorithm generates more fingerprints, e.g. increase the size of the frequency bins (i.e. reduce NFFT).
Also try to have DO_PLOT set to true to check that what you are recording is visually similar to the original file.

[RWOverdijk]

@dest4 Yeah. I want to index the source file, and then use the microphone to find matches as I want to sync on a phone.

I'll try playing with that parameter. Thanks :)

[RWOverdijk]

Hei, past me, it's you from the future. Don't worry, you still look... Average.

I have no idea how you managed to not get it working, but here's what I did:

• Going for 22050
• Lowering NFFT to 256
• Manually making your samples mono (2 bytes left, 2 bytes right, add, average)

Okay bye