Encountering error in immediate mode. ChunkParser?

Question

Encountering error in immediate mode. ChunkParser?

btc opened this issue 9 years ago · 30 comments

Hi, there. Thanks for the great software. I'm hoping you can help me to understand an error I am encountering while using the program in immediate mode (-i).

As you can see from the output below, the program is reading from session file 334268.session.

The problem is: when I invoke the command to run the program, sometimes it starts up correctly, and sometimes it errors out. When the program errors out, it shows the errors I have pasted below. When it starts up correctly, it only functions for one or two tracks before eventually erroring out the same way.

I am not an expert, but it appears the program is very sensitive to the current state of the session file.

While the source code is really well-written and I have managed to explore it with some success, this error seems to be related to the binary file. The binary file, unfortunately, is opaque to me.

User:sslscrobbler Administrator$ ./historyreader.php -i --dir /Volumes/Macintosh\ HD-1/Users/RemoteUser/Music/_Serato_/History/Sessions/
Using default config. Customise by creating config.php (based on config.php-default)
Using file /Volumes/Macintosh HD-1/Users/RemoteUser/Music/_Serato_/History/Sessions/334268.session ...
2016-07-08 20:28:48 WARNING: SignalHandler - PCNTL extension not installed! Cannot trap Ctrl-C / SIGKILL
2016-07-08 20:28:48 INFO: PluginWrapper - 0: GrowlPlugin installed
2016-07-08 20:28:48 INFO: PluginWrapper - 1: NowPlayingLoggerPlugin installed

1: PLAYING    [--:--] [Vybz Kartel - Western Union] [prev: Masicka - Hardball]
2: NEW        [--:--] [Alkaline - Formula] [prev: Alkaline - Champion boy]
Date: 2016-07-08 20:28:48 Memory Usage: 16,944,960 bytes
2016-07-08 20:28:48 INFO: NowPlayingModel - enqueued track Vybz Kartel - Western Union
2016-07-08 20:28:48 INFO: NowPlayingModel - enqueued track Alkaline - Formula
2016-07-08 20:28:48 INFO: SSLEventGrowlRenderer - >> Now Playing: >> Vybz Kartel - Western Union
2016-07-08 20:28:48 INFO: SSLEventGrowlRenderer - >> Track change: >> Track started: Vybz Kartel - Western Union
2016-07-08 20:28:48 INFO: SSLEventGrowlRenderer - >> Track change: >> Track started: Alkaline - Formula

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

… (hundreds of the above/below lines omitted)

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109
No more data (read 3 bytes)
Try ./historyreader.php --help

User:sslscrobbler-master Administrator$ php -v
PHP 5.3.29 (cli) (built: Mar 19 2015 14:04:17) 
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2014 Zend Technologies

Here's the output of php -i: https://gist.github.com/briantigerchow/3413441b73da93659ffc826a1edf37be

Answer 1 · 2016-07-08T21:02:11.000Z

Hi there!

Indeed - it basically works by constantly reading the session file.

I've not seen that error myself but it should be quite easy to fix. I'll
try to take a look some time this weekend.

Does it happen when you play a particular track?
On Fri, 8 Jul 2016 at 21:52, Brian Tiger Chow notifications@github.com
wrote:

Hi, there. Thanks for the great software. I'm hoping you can help me to
understand an error I am encountering while using the program in immediate
mode (-i).

As you can see from the output below, the program is reading from session
file 334268.session.

The problem is: when I invoke the command to run the program, sometimes it
starts up correctly, and sometimes it errors out. When the program errors
out, it shows the errors I have pasted below. When it starts up correctly,
it only functions for one or two tracks before eventually erroring out the
same way.

I am not an expert, but it appears the program is very sensitive to the
current state of the session file.

While the source code is really well-written and I have managed to explore
it with some success, this error seems to be related to the binary file.
The binary file, unfortunately, is opaque to me.

User:sslscrobbler Administrator$ ./historyreader.php -i --dir /Volumes/Macintosh\ HD-1/Users/RemoteUser/Music/Serato/History/Sessions/
Using default config. Customise by creating config.php (based on config.php-default)
Using file /Volumes/Macintosh HD-1/Users/RemoteUser/Music/Serato/History/Sessions/334268.session ...
2016-07-08 20:28:48 WARNING: SignalHandler - PCNTL extension not installed! Cannot trap Ctrl-C / SIGKILL
2016-07-08 20:28:48 INFO: PluginWrapper - 0: GrowlPlugin installed
2016-07-08 20:28:48 INFO: PluginWrapper - 1: NowPlayingLoggerPlugin installed

1: PLAYING [--:--] [Vybz Kartel - Western Union] [prev: Masicka - Hardball]
2: NEW [--:--] [Alkaline - Formula] [prev: Alkaline - Champion boy]
Date: 2016-07-08 20:28:48 Memory Usage: 16,944,960 bytes
2016-07-08 20:28:48 INFO: NowPlayingModel - enqueued track Vybz Kartel - Western Union
2016-07-08 20:28:48 INFO: NowPlayingModel - enqueued track Alkaline - Formula
2016-07-08 20:28:48 INFO: SSLEventGrowlRenderer - >> Now Playing: >> Vybz Kartel - Western Union
2016-07-08 20:28:48 INFO: SSLEventGrowlRenderer - >> Track change: >> Track started: Vybz Kartel - Western Union
2016-07-08 20:28:48 INFO: SSLEventGrowlRenderer - >> Track change: >> Track started: Alkaline - Formula

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

… (hundreds of the above/below lines omitted)

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109

Warning: fread(): Length parameter must be greater than 0 in /Users/Administrator/src/sslscrobbler-master/SSL/SSLChunkParser.php on line 109
No more data (read 3 bytes)
Try ./historyreader.php --help

User:sslscrobbler-master Administrator$ php -v
PHP 5.3.29 (cli) (built: Mar 19 2015 14:04:17)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2014 Zend Technologies

Here's the output of php -i

https://gist.github.com/briantigerchow/3413441b73da93659ffc826a1edf37be

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#20, or mute the thread
https://github.com/notifications/unsubscribe/AAEoVoEhuHnGCM1UL3j3tyTz4kLsC9jNks5qTriCgaJpZM4JIYzJ
.

Answer 2 · 2016-07-08T21:09:53.000Z

Though it is possibly related to a particular track, the error occurs often enough that this seems unlikely.

To aid in debugging, I've captured session files in both ok and broken states. The ok file is taken during normal operation. The broken file is taken just seconds after a crash. Loading the broken file into the program reproduces the crash.

I've included both files in the attached zip.

session.zip

Answer 3 · 2016-07-09T18:07:56.000Z

Please make sure you copy the tile whilst Serato is still open as it
rewrites the file after you quit to remove all the gaps and other changes
(e.g adding and removing tracks you didn't play)

Yes. I copied the file during an active session.

Answer 4 · 2016-07-10T22:05:43.000Z

So, this broken file and the normal file are identical apart from one thing - the broken file has about 2800 bytes of empty/zeroes at the end.

I've never seen that before, and I can only guess why the file is like that.

Does Serato continue to behave normally after this? If you continued to play another song after this happens, could you send me that file as well? (i.e. the equivalent of ".ok", ".broken" and ".broken_plus1song" or whatever.

Answer 5 · 2016-07-10T22:39:15.000Z

Yes. Serato continues to operate normally. This is taken from a serato instance that runs 24/7.

And yes, I can do that for you. Unfortunately, it'll take me a few hours to get back to the office, though.

Also, I should note: the "ok" file is, itself, a "broken plus n songs" file. The session file oscillates between broken and ok continuously during the session.

Answer 6 · 2016-07-10T23:02:23.000Z

Hey @briantigerchow, I have addressed this in the latest master - please give it a go and let me know if it fixes your problem.

Answer 7 · 2016-07-10T23:05:22.000Z

Just saw your last reply @briantigerchow - my hunch is that Serato is allocating extra space to the history file in a new way that I haven't seen before, by extending it with blank space - and your last comment correlates with that. So I hope my fix works for you!

Answer 8 · 2016-07-10T23:13:19.000Z

Out of interest, which operating system do you run Serato on?

On Sun, 10 Jul 2016 at 23:39 Brian Tiger Chow notifications@github.com
wrote:

Yes. Serato continues to operate normally. This is taken from a serato
instance that runs 24/7.

And yes, I can do that for you. Unfortunately, it'll take me a few hours
to get back to the office, though.

Also, I should note: the "ok" file is, itself, a "broken plus n songs"
file. The session file oscillates between broken and ok continuously during
the session.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAEoVtr6yOD2nEUoS2P_LiN-6XtE75y2ks5qUXSTgaJpZM4JIYzJ
.

Answer 9 · 2016-07-11T00:02:23.000Z

Out of interest, which operating system do you run Serato on?

OS X

Hey @briantigerchow, I have addressed this in the latest master - please give it a go and let me know if it fixes your problem.

I'll try this out in the morning and let you know. Thanks for the quick turnaround.

Answer 10 · 2016-07-11T20:26:17.000Z

Hi there. The issue reported last week is no longer occurring. Now, there is a different error. This error comes and goes. Also, many times, songs play but the program does not report them.

User:sslscrobbler-master Administrator$ ./historyreader.php -i 336235.session 
Using default config. Customise by creating config.php (based on config.php-default)
2016-07-11 20:21:00 WARNING: SignalHandler - PCNTL extension not installed! Cannot trap Ctrl-C / SIGKILL
2016-07-11 20:21:00 INFO: PluginWrapper - 0: GrowlPlugin installed
2016-07-11 20:21:00 INFO: PluginWrapper - 1: NowPlayingLoggerPlugin installed
Found chunk claiming to be enormous (1,712.02 MiB); are you reading the right file?

I've attached the file.

enormous_chunk.session.zip

Answer 11 · 2016-07-11T20:28:13.000Z

I'll try to investigate some time in the week!

Usually that means it's trying to interpret junk as data, so I've got
something wrong somewhere...

On Mon, 11 Jul 2016 at 21:26, Brian Tiger Chow notifications@github.com
wrote:

Hi there. The issue reported last week is no longer occurring. Now, there
is a different error. This error comes and goes. Also, many times, songs
play but the program does not report them.

User:sslscrobbler-master Administrator$ ./historyreader.php -i 336235.session
Using default config. Customise by creating config.php (based on config.php-default)
2016-07-11 20:21:00 WARNING: SignalHandler - PCNTL extension not installed! Cannot trap Ctrl-C / SIGKILL
2016-07-11 20:21:00 INFO: PluginWrapper - 0: GrowlPlugin installed
2016-07-11 20:21:00 INFO: PluginWrapper - 1: NowPlayingLoggerPlugin installed
Found chunk claiming to be enormous (1,712.02 MiB); are you reading the right file?

I've attached the file.

enormous_chunk.session.zip
https://github.com/ben-xo/sslscrobbler/files/358014/enormous_chunk.session.zip

—
You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAEoVpJpZanDMHyrEoW8CnYEPg3mPzU1ks5qUqbpgaJpZM4JIYzJ
.

Answer 12 · 2016-07-11T20:30:26.000Z

Thanks! To be clear, there are two observable issues at the moment:

enormous chunk
songs play in Serato DJ but do not appear in program

Answer 13 · 2016-07-11T20:33:04.000Z

Thanks. I expect they're related, we'll see!
On Mon, 11 Jul 2016 at 21:30, Brian Tiger Chow notifications@github.com
wrote:

Thanks! To be clear, there are two observable issues at the moment:

enormous chunk

songs play in Serato DJ but do not appear in program

—
You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAEoVsLScS39ilrI1hjeQ2nO54dA_CBIks5qUqfigaJpZM4JIYzJ
.

Answer 14 · 2016-07-12T12:58:27.000Z

I've done some more research.

Your latest file has a huge gaping hole in the middle - and the end of the track just before the hole is repeated just after the hole. This means I would have to jump through some hoops to resync the stream. Possible, but nasty.

This looks very strange to me - it looks like a bug in Serato. Can you confirm that, if you quit serato, then reload it, the entire tracklist stas intact inside the serato history tab? If it does, I'll endeavour to fix it; if it doesn't, you should report a bug to Serato.

Exactly which version of Serato is this? (Product and version number)

Answer 15 · 2016-07-12T13:21:24.000Z

I've just pushed a version which jumps through hoops to try to solve your problem :o) Give that one a go…

Answer 16 · 2016-07-13T03:58:42.000Z

I've done some more research.

Thanks!

Your latest file has a huge gaping hole in the middle - and the end of the track just before the hole is repeated just after the hole. This means I would have to jump through some hoops to resync the stream. Possible, but nasty.

That's crazy. I hope you don't have to jeopardize the integrity of the codebase to fit this bizarre use case.

I've just pushed a version which jumps through hoops to try to solve your problem :o) Give that one a go…

Unfortunately it didn't work. The program repeatedly prints Hit unallocated blank space in file and new tracks still aren't showing up.

Exactly which version of Serato is this? (Product and version number)

It's Scratch Live 2.5.0. This could be the problem.

(The DJs here prefer it to Serato DJ. I'll have to do some investigation and lobbying to see if Serato DJ is an option for us.)

This looks very strange to me - it looks like a bug in Serato. Can you confirm that, if you quit serato, then reload it, the entire tracklist stas intact inside the serato history tab? If it does, I'll endeavour to fix it; if it doesn't, you should report a bug to Serato.

I'll confirm this tomorrow for you. Thanks again.

Answer 17 · 2016-07-13T06:01:09.000Z

I use Scratch Live 2.5.0 myself. However, I've never had it open for more
than about 10 hours in one go.

I tried to fix the "hit unallocated blank space" message last night.

Last night I successfully did a show with this version (only 2 hours, mind
you). This version also successfully reads from all three of the files you
provided me with, all the way through to the end.

You never answered me if the broken history file shows the correct history
in Serato after quitting and restarting Serato.

One more question: you have your history file on an external drive,
correct? What file system is that formatted as? HFS+?

On Wed, 13 Jul 2016 at 04:58, Brian Tiger Chow notifications@github.com
wrote:

I've done some more research.

Thanks!

Your latest file has a huge gaping hole in the middle - and the end of the
track just before the hole is repeated just after the hole. This means I
would have to jump through some hoops to resync the stream. Possible, but
nasty.

That's crazy. I hope you don't have to jeopardize the integrity of the
codebase to fit this bizarre use case.

I've just pushed a version which jumps through hoops to try to solve your
problem :o) Give that one a go…

Unfortunately it didn't work. The program repeatedly prints Hit
unallocated blank space in file and new tracks still aren't showing up.

Exactly which version of Serato is this? (Product and version number)

It's Scratch Live 2.5.0. This could be the problem.

(The DJs here prefer it to Serato DJ. I'll have to do some investigation
and lobbying to see if Serato DJ is an option for us.)

This looks very strange to me - it looks like a bug in Serato. Can you
confirm that, if you quit serato, then reload it, the entire tracklist stas
intact inside the serato history tab? If it does, I'll endeavour to fix it;
if it doesn't, you should report a bug to Serato.

I'll confirm this tomorrow for you. Thanks again.

—
You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAEoVrvxGgNlGUAHx4vYhfyL7GecSvmuks5qVGJygaJpZM4JIYzJ
.

Answer 18 · 2016-07-13T07:36:22.000Z

So - I've made a couple more debugging changes - and I've examined the "enormous" file you sent me more thoroughly.

The "enormous" file has a few tracks played at 6pm GMT, and then a 2 hour gap (when it hits the blank space), and then 4 more tracks at the end at 8pm GMT.

There's nothing else in the file - so there's nothing I can do to read the info in that "missing" period.

These session files are huge; can I suggest (for example) that you use the "end session" button inside serato between DJs? this will close off the current session file and open a new one, and SSLScrobbler should detect that.

Answer 19 · 2016-07-14T20:06:37.000Z

Unfortunately…

These session files are huge; can I suggest (for example) that you use the "end session" button inside serato between DJs? this will close off the current session file and open a new one, and SSLScrobbler should detect that.

The program detects the new file, but the behaviour is the same:

new tracks aren't detected
Constantly emits "Hit unallocated blank space in file."

The program exhibits this behaviour on Scratch Live (2.5.0) and Serato DJ (checking version now...).

Please see attached session file for more info.

340595.session.zip

Answer 20 · 2016-07-14T20:31:06.000Z

Just to see if I can replicate what's going on:

Exactly which version of OSX?
Using the default PHP?
Can you tell me more about the external drive? Is it SSD? What filesystem is it formatted with? (type "mount" in the terminal - you should see something like "/dev/disk8 on /Volumes/Home (hfs, local, journaled)" which would tell me it's hfs. I want to know if it's hfs, or exfat, or something else.)

Answer 21 · 2016-07-14T20:53:55.000Z

Exactly which version of OSX?

Yosemite 10.10.3

Using the default PHP?

Yes

Can you tell me more about the external drive? Is it SSD? What filesystem is it formatted with? (type "mount" in the terminal - you should see something like "/dev/disk8 on /Volumes/Home (hfs, local, journaled)" which would tell me it's hfs. I want to know if it's hfs, or exfat, or something else.)

It's not running on an external drive. It's a late 2014 Retina iMac with its disk formatted as Mac OS Extended (Journaled).

Answer 22 · 2016-07-14T20:59:27.000Z

ah, I got thought it was because of the --dir patch you submitted - that's fine!

I think OSX 10.10 goes up to 10.10.5, which is what I use when streaming.

I gather from the sample file you sent that you use a Rane Sixty-One? - I don't see why it should make a difference, though. (I use an SL2 and have never been able to test with anything else).

I don't mean to be difficult, but trying to narrow the possibilities down - have you tried substituting in a different Mac (or would you be able to test at home for example?)

I do appreciate any help you can give improving the software, of course

Answer 23 · 2016-07-14T22:06:15.000Z

I gather from the sample file you sent that you use a Rane Sixty-One? - I don't see why it should make a difference, though. (I use an SL2 and have never been able to test with anything else).

Just tested on...

Rane 62, OS X 10.10.5, Serato DJ 1.9.0

No luck. Same problem.

I gather from the sample file you sent that you use a Rane Sixty-One? - I don't see why it should make a difference, though. (I use an SL2 and have never been able to test with anything else).

It's a possibility I'm forced to consider. About to test on an SL1 now.

Answer 24 · 2016-07-14T22:31:00.000Z

Tried the SL1. Same problem.

Answer 25 · 2016-07-22T10:53:19.000Z

Ok. So we've ruled out:

the specific Rane hardware
the specific Serato software
the filesystem of your drive
version of OSX (i think?)

We haven't ruled out:

this specific Mac
some strange property of your MP3 collection, or Serato history

If you get to the bottom of it or can isolate any conditions which cause the bug to stop occurring, I am interested to know - but at this point I'm fairly certain the problem is not with SSLScrobbler.

Answer 26 · 2016-07-22T15:17:46.000Z

Thank you for your dedication to this issue.

version of OSX (i think?)

Yes. We've ruled this out.

this specific Mac, some strange property of your MP3 collection

I've tried this on three different Macs with three different iTunes collections.

If you get to the bottom of it or can isolate any conditions which cause the bug to stop occurring, I am interested to know - but at this point I'm fairly certain the problem is not with SSLScrobbler.

I'm continuing to investigate on my end. This week, I will have a Mac Mini dedicated to processing history files and scrobbling. I should have a heap of new information for you by Friday 29 July, so look out for that.

Answer 27 · 2021-01-13T08:59:51.000Z

I wonder if your problem ever got resolved?

Answer 28 · 2021-01-14T15:45:39.000Z

We gave up on the use-case, unfortunately. 🙏🏽

Answer 29 · 2021-01-14T16:05:22.000Z

We gave up on the use-case, unfortunately. 🙏🏽

alright fair enough!!! I still use this app every week :)

Answer 30 · 2021-01-14T16:07:11.000Z

I do as well! 👍