Strange matching problem (may be M4a related)
Opened this issue · 6 comments
Whatever I do, I cannot get these to match. The music is there at Google end, but it shows no match and tries to upload (the upload intentionally fails, as I have the retry count set to 0, I don't want dupes).
Tried allsorts, even copying the tags from Google Music web interface into MP3Tag and recreating the ID3 info. It's not having it, totally refusing to match these files. If you need any other details, please let me know.
Turned on logging:
Uploading /usb/smb/Audio/Playlists/4StarPlus_Rock_And_Metal.m3u
Synching playlist: /usb/smb/Audio/Playlists/4StarPlus_Rock_And_Metal.m3u
483 songs already in Google Music playlist
486 songs in local playlist
Adding: 05 smash it up.m4a
[{u'comment': u'', u'rating': 0, u'lastPlayed': 1309567095069875L, u'disc': 0, u'matchedId': u'T3hmvfcaysgxgbgtc2sjr2schwi', u'composer': u'Algy Ward/Captain Sensible/Dave Vanian/Rat Scabies', u'year': 1994, u'id': u'd8ddf9f0-bad2-3f3f-89aa-bcad64b052c5', u'subjectToCuration': False, u'album': u'Sessions of the Damned', u'title': u'Smash It Up', u'deleted': False, u'albumArtist': u'The Damned', u'albumArtUrl': u'//lh5.googleusercontent.com/512zAzfTHd527dV6FnJaDp3qIlSBW4K10bsY55sZen4H4XMhQpbFmgQ2tm4=s130-c-e100', u'type': 2, u'titleNorm': u'smash it up', u'track': 15, u'albumArtistNorm': u'the damned', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Alternative', u'playCount': 0, u'creationDate': 1308997468462075L, u'name': u'Smash It Up', u'albumNorm': u'sessions of the damned', u'artist': u'The Damned', u'url': u'', u'totalDiscs': 0, u'durationMillis': 242346, u'artistNorm': u'the damned'}, {u'comment': u'', u'rating': 0, u'lastPlayed': 1309568640825704L, u'disc': 0, u'matchedId': u'Tx7lmkil3jenzsxw6vl67yluoli', u'composer': u'', u'year': 1979, u'id': u'25bee812-dbf6-3bcb-b6a6-5ef47ccc1ef0', u'subjectToCuration': False, u'album': u'The Best Of The Damned', u'title': u'smash it up', u'deleted': False, u'albumArtist': u'The Damned', u'albumArtUrl': u'//lh4.googleusercontent.com/WI2J5Vj0uZquE4NcPdQfOAnIT5klA8FZCdPTsZpShwFm0cXZz6V6n7IT12JW=s130-c-e100', u'type': 2, u'titleNorm': u'smash it up', u'track': 5, u'albumArtistNorm': u'the damned', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Punk', u'playCount': 0, u'creationDate': 1308997468462075L, u'name': u'smash it up', u'albumNorm': u'the best of the damned', u'artist': u'The Damned', u'url': u'', u'totalDiscs': 0, u'durationMillis': 173011, u'artistNorm': u'the damned'}, {u'comment': u'', u'rating': 0, u'lastPlayed': 1309563691570519L, u'disc': 0, u'matchedId': u'', u'composer': u'Algy Ward/Chris Millar/Dave Vanian/Ray Burns', u'year': 2004, u'id': u'a5f01a93-5f92-3842-9f27-b82133b24707', u'subjectToCuration': False, u'album': u'Machine Gun Etiquette [25th Anniversary Edition]', u'title': u'Smash It Up, Pt. 2 [Backing Track - Singalonga Damned][#][]', u'deleted': False, u'albumArtist': u'The Damned', u'albumArtUrl': u'//lh3.googleusercontent.com/Hwlf-9z--38HBfsMTXV8sL7xf5lmcWvxkhXPg4F3q3nZagMoov0KsLp6Vxji=s130-c-e100', u'type': 2, u'titleNorm': u'smash it up, pt. 2 [backing track - singalonga damned][#][]', u'track': 15, u'albumArtistNorm': u'the damned', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Alternative', u'playCount': 0, u'creationDate': 1308997457824269L, u'name': u'Smash It Up, Pt. 2 [Backing Track - Singalonga Damned][#][]', u'albumNorm': u'machine gun etiquette [25th anniversary edition]', u'artist': u'The Damned', u'url': u'', u'totalDiscs': 0, u'durationMillis': 176541, u'artistNorm': u'the damned'}, {u'comment': u'', u'rating': 0, u'lastPlayed': 1309563677920313L, u'disc': 0, u'matchedId': u'Tzjycf4dqb3vavgfjddcs6u2uli', u'composer': u'Ray Burns', u'year': 2004, u'id': u'54eab7f9-c625-3d99-b264-481171105c45', u'subjectToCuration': False, u'album': u'Machine Gun Etiquette [25th Anniversary Edition]', u'title': u'Smash It Up, Pt. 4 [#][]', u'deleted': False, u'albumArtist': u'The Damned', u'albumArtUrl': u'//lh3.googleusercontent.com/Hwlf-9z--38HBfsMTXV8sL7xf5lmcWvxkhXPg4F3q3nZagMoov0KsLp6Vxji=s130-c-e100', u'type': 2, u'titleNorm': u'smash it up, pt. 4 [#][]', u'track': 16, u'albumArtistNorm': u'the damned', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Alternative', u'playCount': 0, u'creationDate': 1308997457824269L, u'name': u'Smash It Up, Pt. 4 [#][]', u'albumNorm': u'machine gun etiquette [25th anniversary edition]', u'artist': u'The Damned', u'url': u'', u'totalDiscs': 0, u'durationMillis': 117701, u'artistNorm': u'the damned'}, {u'comment': u'', u'rating': 0, u'lastPlayed': 1309563289883808L, u'disc': 0, u'matchedId': u'T25uygggdd2ogxsfgjhz4nnb7vu', u'composer': u'Alasdair Ward/Christopher Millar/David Vanian/Raymond Burns', u'year': 2004, u'id': u'43b41b61-2b8d-32a4-b5cd-146a5dd4a1f6', u'subjectToCuration': False, u'album': u'Machine Gun Etiquette [25th Anniversary Edition]', u'title': u'Smash It Up, Pts. 1-2', u'deleted': False, u'albumArtist': u'The Damned', u'albumArtUrl': u'//lh3.googleusercontent.com/Hwlf-9z--38HBfsMTXV8sL7xf5lmcWvxkhXPg4F3q3nZagMoov0KsLp6Vxji=s130-c-e100', u'type': 2, u'titleNorm': u'smash it up, pts. 1-2', u'track': 11, u'storeId': u'T25uygggdd2ogxsfgjhz4nnb7vu', u'albumArtistNorm': u'the damned', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Alternative', u'playCount': 0, u'creationDate': 1308997351657660L, u'name': u'Smash It Up, Pts. 1-2', u'albumNorm': u'machine gun etiquette [25th anniversary edition]', u'artist': u'The Damned', u'url': u'', u'totalDiscs': 0, u'durationMillis': 313771, u'artistNorm': u'the damned'}]
Smash It Up ][ The Damned ][ The Best Of The Damned ][ 5
upload failed - skipping
Adding: 12 - Reward.m4a
[{u'comment': u'', u'rating': 0, u'lastPlayed': 1355180352422837L, u'disc': 0, u'matchedId': u'', u'composer': u'', u'year': 1978, u'id': u'0aaf1f5b-1be1-314c-9754-81803dc610d2', u'subjectToCuration': False, u'album': u'Sound Of The Suburbs', u'title': u'Reward', u'deleted': False, u'albumArtist': u'Various Artists', u'albumArtUrl': u'//lh5.googleusercontent.com/U9GWqcXAjmngn_9s3h1G3WkCRD3jGTFAVdn1Uwxzid66dTCu2KwOp8XRQXaTZA=s130-c-e100', u'type': 2, u'titleNorm': u'reward', u'track': 12, u'albumArtistNorm': u'various artists', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Pop', u'playCount': 0, u'creationDate': 1355174269392382L, u'name': u'Reward', u'albumNorm': u'sound of the suburbs', u'artist': u'Various Artists', u'url': u'', u'totalDiscs': 0, u'durationMillis': 162144, u'artistNorm': u'various artists'}]
Reward ][ The Teardrop Explodes ][ The Sound of the Suburbs ][ 12
upload failed - skipping
Adding: 01 new rose.m4a
[{u'comment': u'', u'rating': 0, u'lastPlayed': 1311119432972179L, u'disc': 0, u'matchedId': u'T6qj7l2f5aqsshrkkex3tkuik54', u'composer': u'', u'year': 1993, u'id': u'519ef715-cf8f-35a9-96cc-c27f72f2541a', u'subjectToCuration': False, u'album': u'The Spaghetti Incident', u'title': u'New Rose', u'deleted': False, u'albumArtist': u"Guns N' Roses", u'albumArtUrl': u'//lh5.googleusercontent.com/GPWJJsmap_iNDfPZkOFyfM_FJXFPtC_ELn9wm65ctkrUWthgdmDt1gbmQ6EWdQ=s130-c-e100', u'type': 2, u'titleNorm': u'new rose', u'track': 2, u'storeId': u'T6qj7l2f5aqsshrkkex3tkuik54', u'albumArtistNorm': u"guns n' roses", u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Hard Rock', u'playCount': 0, u'creationDate': 1310765698383335L, u'name': u'New Rose', u'albumNorm': u'the spaghetti incident', u'artist': u"Guns N' Roses", u'url': u'', u'totalDiscs': 0, u'durationMillis': 158429, u'artistNorm': u"guns n' roses"}, {u'comment': u'', u'rating': 0, u'lastPlayed': 1309568198589664L, u'disc': 0, u'matchedId': u'Tz7f7s7pofftrbf7wx2cfafapli', u'composer': u'', u'year': 1979, u'id': u'e46786d2-70c7-3898-9c11-5af186143120', u'subjectToCuration': False, u'album': u'The Best Of The Damned', u'title': u'new rose', u'deleted': False, u'albumArtist': u'The Damned', u'albumArtUrl': u'//lh4.googleusercontent.com/WI2J5Vj0uZquE4NcPdQfOAnIT5klA8FZCdPTsZpShwFm0cXZz6V6n7IT12JW=s130-c-e100', u'type': 2, u'titleNorm': u'new rose', u'track': 1, u'storeId': u'Tz7f7s7pofftrbf7wx2cfafapli', u'albumArtistNorm': u'the damned', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Punk', u'playCount': 0, u'creationDate': 1308997468462075L, u'name': u'new rose', u'albumNorm': u'the best of the damned', u'artist': u'The Damned', u'url': u'', u'totalDiscs': 0, u'durationMillis': 164141, u'artistNorm': u'the damned'}, {u'comment': u'', u'rating': 0, u'lastPlayed': 1309565728906475L, u'disc': 0, u'matchedId': u'T2zo2buxv5sayavrafzmb6shhce', u'composer': u'Brian James', u'year': 1994, u'id': u'77b72ac9-d985-30d3-a188-16666f4fa72f', u'subjectToCuration': False, u'album': u'Sessions of the Damned', u'title': u'New Rose', u'deleted': False, u'albumArtist': u'The Damned', u'albumArtUrl': u'//lh5.googleusercontent.com/512zAzfTHd527dV6FnJaDp3qIlSBW4K10bsY55sZen4H4XMhQpbFmgQ2tm4=s130-c-e100', u'type': 2, u'titleNorm': u'new rose', u'track': 3, u'albumArtistNorm': u'the damned', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Alternative', u'playCount': 0, u'creationDate': 1308997457824269L, u'name': u'New Rose', u'albumNorm': u'sessions of the damned', u'artist': u'The Damned', u'url': u'', u'totalDiscs': 0, u'durationMillis': 161773, u'artistNorm': u'the damned'}, {u'comment': u'', u'rating': 0, u'lastPlayed': 1309562360265906L, u'disc': 0, u'matchedId': u'Tz7f7s7pofftrbf7wx2cfafapli', u'composer': u'Brian James', u'year': 1977, u'id': u'14ac239d-cd3b-3a37-b224-5240d09a248d', u'subjectToCuration': False, u'album': u'Damned Damned Damned', u'title': u'New Rose', u'deleted': False, u'albumArtist': u'The Damned', u'albumArtUrl': u'//lh3.googleusercontent.com/4nc4g8lSiVA4alcy68xpvKW5vSlpX6Q5v7wYvHEAzpRJ--aYMZ0WI6nqns2Q=s130-c-e100', u'type': 2, u'titleNorm': u'new rose', u'track': 7, u'storeId': u'Tz7f7s7pofftrbf7wx2cfafapli', u'albumArtistNorm': u'the damned', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Alternative', u'playCount': 0, u'creationDate': 1308997351657660L, u'name': u'New Rose', u'albumNorm': u'damned damned damned', u'artist': u'The Damned', u'url': u'', u'totalDiscs': 0, u'durationMillis': 164699, u'artistNorm': u'the damned'}]
New Rose ][ The Damned ][ The Best Of The Damned ][ 1
upload failed - skipping
Adding: 4 - 'Another Girl, Another Planet'.m4a
[{u'comment': u'', u'rating': 0, u'lastPlayed': 1355180830473009L, u'disc': 0, u'matchedId': u'', u'composer': u'Perrett', u'year': 1978, u'id': u'61e0e3ae-532d-3521-8239-b251679f245f', u'subjectToCuration': False, u'album': u'Sound Of The Suburbs', u'title': u'Another Girl Another Planet', u'deleted': False, u'albumArtist': u'Various Artists', u'albumArtUrl': u'//lh6.googleusercontent.com/dcTpnLzSbIX0Etip6i2hu1k316YEKrGOX-lkrQazh5Ap_DQXhFKXf_92stz9Xg=s130-c-e100', u'type': 2, u'titleNorm': u'another girl another planet', u'track': 4, u'albumArtistNorm': u'various artists', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'Pop', u'playCount': 0, u'creationDate': 1355174269392382L, u'name': u'Another Girl Another Planet', u'albumNorm': u'sound of the suburbs', u'artist': u'The Only Ones', u'url': u'', u'totalDiscs': 0, u'durationMillis': 179977, u'artistNorm': u'the only ones'}, {u'comment': u'', u'rating': 0, u'lastPlayed': 1312003647961141L, u'disc': 0, u'matchedId': u'Tz7sj2haodlqcy2poe3iioayl2m', u'composer': u'', u'year': 1978, u'id': u'c7b1ff6e-6b26-32d7-8107-6df5add4700c', u'subjectToCuration': False, u'album': u'The Only Ones', u'title': u'Another Girl, Another Planet', u'deleted': False, u'albumArtist': u'The Only Ones', u'albumArtUrl': u'//lh4.googleusercontent.com/JeYwuBASTcGWpLmJbiubdlfIL4URVY5sEahZIFoatUvNdD-B6zr0grGTI7Bp=s130-c-e100', u'type': 2, u'titleNorm': u'another girl, another planet', u'track': 2, u'storeId': u'Tz7sj2haodlqcy2poe3iioayl2m', u'albumArtistNorm': u'the only ones', u'totalTracks': 0, u'beatsPerMinute': 0, u'genre': u'New Wave', u'playCount': 0, u'creationDate': 1310768074996073L, u'name': u'Another Girl, Another Planet', u'albumNorm': u'the only ones', u'artist': u'The Only Ones', u'url': u'', u'totalDiscs': 0, u'durationMillis': 180001, u'artistNorm': u'the only ones'}]
Another Girl Another Planet ][ The Only Ones ][ The Sound of the Suburbs ][ 4
upload failed - skipping
482 songs unmodified
0 songs added
4 songs failed
0 songs removed
Smash It Up and New Rose
There are two matches (same song - different album), the one that seems to be an accurate match is the wrong case for the title. Fix would be to .lower() all the comparisons in tag compare. Though I'm not really sure why the values would be different (did you upload this song previously with another method?). Maybe get an ID3 tag editor and have a look at the file directly (including if it contains different ID3 data versions). It could be that Google defaults to using a different version of ID3 data than what this script does...
Reward and Another Girl Another Planet
The artist doesn't match ('Various Artists' compared to 'The Teardrop Explodes'). Same questions as above.
I think comparing using tolower() makes sense makes sense... though I'm interested to know why there are two copies of those songs and why the second has a lower case name...
For the various artists one; if you can let me know about the ID3 data I might be able to force the script to use the same ID3 that Google must be looking at (or get it to compare both).
For what its worth - Even if you do re-upload the same file, I think Google is smart enough to not duplicate it (I was doing some testing earlier, and when I searched for the duplicates I'd supposedly uploaded there was only ever one copy).
Problem is, upload is broken in the API for me. (there is an open issue in the gmusicapi about failure to upload non mp3 files), and my NAS takes forever to transcode and upload over ADSL :-(
All the music was originally uploaded using Windows Music Manager from a shared drive on my Linux NAS, I am truing to keep it all in sync including playlists.
Will investigate the files ID3 and see if I can get any more clues...
Still having some problems. Had a go at hacking the source, but not a python programmer, and not that familiar with mutegen library.
What I have tried to do, is add an interactive mode, where when it can't figure out the match, prompt the possibilities, and whichever match is selected, it writes a new metadata tag that references the google music id field. When run in either interactive or regular mode, any matches between the previously selected local files's "id" field and the google music id field overrides all other matching tests.
I got some of it working, but can't get the mutagen working in "easy mode" to read an "id" tag (it seems in easy mode, it can only read a known set of tags, and I can't get "hard mode" reading unknown tags either....
I've pushed case-insensitive tag comparison if that helps.
When are you prompting interactive mode? Any time there are matches but none of them are "perfect"?
If you want to store data in the ID3... I guess you'd use TXXX (user data).
See https://code.google.com/p/mutagen/wiki/Tutorial#Hard_Examples:_ID3 for example on how to read/write ID3 with mutagen (my mutagen knowledge is very limited I'm afraid).
Alternatively of course you could just write all the data to a separate file in the same folder as the playlist and store any match-data in there.
If you could get an ID3 dump of the files as requested above (all the ID3 data in the files - potentially multiple versions) then I might be able to fix the underlying problem (which may mean getting a different library to pull out ID3 data of a specific version).
Something very strange is happening Google side. I removed my problematic albums from Google Music, reuploaded them, and still it failed on the same old tracks...
In the end, I got it working by doing lowercase comparisons and doing an "or" on the albumArtist as follows:
return g_song['title'].lower() == tag['title'].lower() and\
g_song['artist'].lower() == tag['artist'].lower() or\
g_song['albumArtist'].lower() == tag['artist'].lower() and\
g_song['album'].lower() == tag['album'].lower() and\
g_song['track'] == tag['track']
It now syncs my playlists correctly. It does start throwing protocol errors when you try and upload over 1000 playlist items (as Google Music sadly only supports playlists upto 1000 songs).
Thanks for the info. I might try to see if I can have a play around with figuring out which version of ID3 tags GM prioritises.
I'll throw in a warning/check for the 1000 limit too I suppose.