Is output of "status" parseable?

[bokutin]

Hello.

In relation to the below, I suspect that the current Administrative Protocol ”status” output cannot be parsed.
netdata/netdata#9604 (comment)
p-alik/perl-Gearman#46 (comment)

In most cases I accept that function names are simple strings, but is it possible to write an exact parser?

Thanks,

[esabol]

Seeing as how function names have no restrictions other than being a NUL-terminated string, I think the answer is no in that most general case.

If you make sure to only use function names which don't include tabs, I think the answer is yes? I think tabs are what are tripping you up?

That said, I think it would make sense to add a JSON format mode to the status response. That would give us flexibility to add more information to the output, which is something others have requested.

[bokutin]

Thanks for the answer.

I'm just using prefix. https://metacpan.org/pod/Gearman::Objects#prefix([$prefix])
Then the tab of prefix_sepaleter will be included in the function name. https://metacpan.org/pod/Gearman::Objects#prefix_separator([$separator])

In the case of netdata above, it doesn't work just because tab is included in the function name, so I posted an issue.
Further research into writing patches revealed that it was logically impossible to write the correct parser.

Personally I don't use function names like \n or space, but I'm stuck on how to answer the above issue.

[bokutin]

The reason why it's logically impossible to write a parser is that the following code is allowed:

$worker->register_function( "func2\t0\t0\t0\nfunc3", sub {} );

# If you do the following, "gearadmin --status" is indistinguishable from the above.
$worker->register_function( "func2", sub {} );
$worker->register_function( "func3", sub {} );

# Both the code above and below produce the same result.
% gearadmin --status
func2   0       0       0
func3   0       0       0
.

http://gearman.org/protocol/
Administrative Protocol status
FUNCTION\tTOTAL\tRUNNING\tAVAILABLE_WORKERS

[p-alik]

Then the tab of prefix_sepaleter will be included in the function name.

The prefix - and separator also - exists only in perl-Gearman and gearmand doesn't know anything about it.

Since 2.004.001 it is possible to provide desirable separator value.
The tab as default separator value remains for backwards compatibility.

[p-alik]

The reason why it's logically impossible to write a parser is that the following code is allowed:

$worker->register_function( "func2\t0\t0\t0\nfunc3", sub {} );

I think it is an issue for perl-Gearman only.

[bokutin]

Thank you for your reply.

Even with the gearadmin, Can the same thing.

% gearadmin --status
.
% perl -e 'system qw(gearman -w -f), "func2\t0\t0\t0\nfunc3"'
% gearadmin --status                                         
func2   0       0       0
func3   0       0       0
.

If the spec prohibited the possible \n in function names, the parser would have written it.
Or, in the output of "status", escape the delimiter included in the function name.

I've found that I can't write a parser with the correct "status" output, so I'd like to end this issue.

Thank you for your quick response.

There can also be strange state like this.

#!/usr/local/bin/perl

use rlib;
use Modern::Perl;
use Data::Dumper;
use Gearman::Client;
use Gearman::Worker;
use JSON::MaybeXS;
use URI::Escape;

my $bin = "\xFF"; # binary, not ascii

my $worker = Gearman::Worker->new;
$worker->job_servers( '127.0.0.1' );

$worker->register_function( "func1",         sub {}) or die;
$worker->register_function( "func1\0",       sub {}) or die;
$worker->register_function( "func1\0\0",     sub {}) or die;
$worker->register_function( "func1\0\0\0",   sub {}) or die;
$worker->register_function( "func1\0\0\0\0", sub {}) or die;

my $client = Gearman::Client->new;
$client->job_servers( '127.0.0.1' );

my $text = "";
$client->_job_server_status_command( # same as "echo status | nc 127.0.0.1 4730"
    "status\n",
    sub {
        my ($js, $line) = @_;
        $text .= $line."\n";
    },
);

my @func;
push @func, { job => $1, queued => $2, running => $3, capable => $4 }
    while $text =~ m{^(.*?)\t(\d+)\t(\d+)\t(\d+)$}msg;
$_->{job_hex} = uri_escape $_->{job} for @func;
@func = sort { $a->{job_hex} cmp $b->{job_hex} } @func;

say JSON::MaybeXS->new->canonical->pretty->encode( \@func );

__END__

% ./test3.pl | cat -v > a
[
   {
      "capable" : "1",
      "job" : "func1",
      "job_hex" : "func1",
      "queued" : "0",
      "running" : "0"
   },
   {
      "capable" : "1",
      "job" : "func1",
      "job_hex" : "func1",
      "queued" : "0",
      "running" : "0"
   },
   {
      "capable" : "1",
      "job" : "func1",
      "job_hex" : "func1",
      "queued" : "0",
      "running" : "0"
   },
   {
      "capable" : "1",
      "job" : "func1",
      "job_hex" : "func1",
      "queued" : "0",
      "running" : "0"
   },
   {
      "capable" : "1",
      "job" : "func1",
      "job_hex" : "func1",
      "queued" : "0",
      "running" : "0"
   }
]

% gearadmin --status
func1   0       0       0
func1   0       0       0
func1   0       0       0
func1   0       0       0
func1   0       0       0
.

[bokutin]

Thanks for the quick answer!