This repository contains a few command-line utilities for OpenSplice. In principle, they are source compatible with OpenSplice V6 (and should be easily ported to V5 in a pinch) but the compatibility with old versions is not regularly checked and some minor issue may be encountered.
The primary tool is "pubsub", a generic reader-writer program with far too many command-line options. The "lsbuiltin" tool offers a simple way to get the contents of the built-in topics containing discovery information in a human-readable form (but you can get the same data using "pubsub", even if the formatting is a little bit harder on the eyes).
Then there is "idl2md" (and "simpleidlpp"). These scripts work together with the OpenSplice IDL preprocessor "idlpp" to easily construct input files for defining topics/types in "pubsub" that are not yet present in the system.
The "lsbuiltin" tool and "idl2md" script are simple enough to not require additional documentation, but "pubsub" is another story altogether.
The model of "pubsub" is to have one participant, with one publisher and one subscriber, and N readers and writers (though typically only 1 reader/writer). One can suppress readers and/or writers, and a publisher/subscriber is only created when needed.
A number of topic types are embedded in "pubsub", and for these topics it can automatically write data, check sequence numbers, measure throughput. For all topics, it offers the ability to write/dispose arbitrary data and the ability to print the received data with a configurable amount of meta-data.
pubsub [OPTIONS] PARTITION...
The options control topics, QoS, input selection, printing modes, &c. The remaining arguments are interpreted as partition names to be used when creating the publisher and subscriber entities. Without options, it will use a standard topic, embedded type, standard QoS (though different from the DCPS default QoS), &c., to provide the simplest possible way to quickly read and write some data and verify basic system operation.
The partition names are subjected to environment variable expansion, with two environment variables automatically defined by pubsub:
- SYSTEMID is set to the OpenSplice system id (in decimal) for the federation that
pubsubis in; - NODE_BUILTIN_PARTITION is set to the node-specific built-in partition (__NODE_x_BUILT-IN PARTITION__) for the federation.
When invoked as pub (e.g., via a soft-link), the default reader mode specification is changed to -m0 (see below), disabling the default creation of readers. Similarly, when invoked as sub the default writer mode is changed to -w0 (again, see below), disabling the default creation of writers.
There are so many options that without doubt some interact in unfortunate ways, but a surprising number of them combine nicely.
Topic and type to read/write are controlled by two options, -T and -K. Specifying a topic with -T when one has already been specified for the current reader/writer pair introduces a new reader/writer pair; similarly, specifying a type with -K when one has already been specified also introduces a new pair.
| option | effect |
|---|---|
-K TYPE |
selects a type to use, either one of the predefined ones, an arbitrary one from a lookup of the topic, or a file providing metadata. |
-T TOPIC |
selects a topic name (and possibly sets a timeout for the find_topic operation and a content-filtered topic expression) |
The TYPE argument of -K is interpreted as follows:
| TYPE | type name | default topic name | resulting type |
|---|---|---|---|
KS |
KeyedSeq |
PubSub |
key (32-bit int), sequence number (unsigned 32-bit int), octet sequence |
K32 |
Keyed32 |
PubSub32 |
key (32-bit int), sequence number (unsigned 32-bit int), octet array of length 32 |
K64 |
Keyed64 |
PubSub64 |
as K32, length 64 |
K128 |
Keyed128 |
PubSub128 |
as K32, length 128 |
K256 |
Keyed256 |
PubSub256 |
as K32, length 256 |
OU |
OneULong |
PubSubOU |
sequence number only |
ARB |
n/a | no default | uses find_topic to look up type and topic QoS, no topic QoS override possible |
| FILE | n/a | no default | reads typename, keylist and type metadata from FILE and defines a topic with the specified name |
The default used depends on what has and hasn't been specified:
| specified | default |
|---|---|
neither -K nor -T |
type KS with topic name from above table |
only -K |
typic name from above table (or an error for the ARB and FILE variants) |
only -T |
type ARB |
The TOPIC argument of the -T option has 3 components, of which the first is required and the others are optional, and which are separated by a colon: TOPIC[:TO][:EXPR]. Here TOPIC is the DDS topic name, TO is a time-out for find_topic (i.e., when using -K ARB), and EXPR is an optional content filter. The timeout is specified in seconds (or the word "inf" for an infinite timeout) and has a default of 10s. Specifying a content filter causes a content-filtered topic with a name "cft_N_" (with N an integer) to be defined.
The content-filtered topic expression is subjected to environment-variable expansion with the variables SYSTEMID and NODE_BUILTIN_PARTITION set automatically, just like the partition names.
QoS can be configured for the topics (other than those looked up via find_topic), subscriber, publisher, readers and writers, but not (yet) independently for the individual entities. QoS defaults are slightly different from those of DCPS, and the readers and writers inherit automatically from the topic QoS.
In ARB mode, the topic QoS is set by the existing topic's definition. In KS, K32, ..., K256, OU and FILE modes (of the -K option), default topic QoS is the DCPS default except:
| QoS setting | value |
|---|---|
| reliability kind | reliable |
| max blocking time | 1s |
| destination order | by source timestamp |
Default subscriber and publisher QoS are the DCPS default , except that the partition is always explicitly set from the command-line arguments (and at least one must be given -- the default partition is specified by giving an empty string as partition name).
Default reader and writer QoS are determined by applying (in this order) the DCPS default, the topic QoS and the built-in default overrides of:
| QoS setting | value |
|---|---|
| history kind | keep all |
| resource limits (reader) | max_samples = 10000 |
| resource limits (writer) | max_samples = 100 |
| auto-dispose unregistered instances | false |
The -q option allows the specification of overrides applied after setting the defaults. It takes an argument in either of three forms: FS:QOS to set QoS directly from the command-line, provider=[PROFILE,]URI to select a QoS provider, or anything accepted by the selected QoS provider, if any.
Here, FS is a set of one or more flags (list them without separators) indicating to which types of entities this QoS setting is to be applied:
| flag | entity type |
|---|---|
t |
topic |
p |
publisher |
s |
subscriber |
w |
writer |
r |
reader |
a |
all of the above |
Specifying a QoS that is inapplicable to the entity type causes a warning, but is otherwise ignored. The QOS field is then interpreted as a comma-separated list of QoS specifications, each of the form Q=S. In the below table, "{a,b}" means either of a and b, and "[a]" means a is optional:
| Q | S | applicability | meaning |
|---|---|---|---|
A |
{a,p:S,w:S} | t, r, w | set liveliness to: automatic, manual-per-participant, or manual-per-writer; S is the lease duration |
d |
{v,tl,t,p} | t, r, w | set the durability kind: volatile, transient-local, transient, or persistent |
D |
P | t, r, w | set deadline to P |
k |
{all,N} | t, r, w | set history kind to keep all, or to keep-last N |
l |
D | r, w | set latency budget to D |
L |
D | w | set lifespan to D |
o |
{r,s} | t, r, w | set destination order, by-reception or by-source. |
O |
{s,x[:S]} | t, r, w | set ownership to shared, or to exclusive with ownership strength S (defaults to 0) |
p |
P | w | set transport priority to P |
P |
{i,t,g} | p, s | set presentation QoS: by-instance, by-topic, or by-group; for by-instance, coherent_access is set to false, for by-topic and by-group, coherent_access is set to true; ordered access is always false |
r |
{y[:T],s[:T],n} | t, r, w | set reliability: reliable, reliable + synchronous, best-effort; T is the max blocking time |
R |
S/I/SpI | r, w | set resource limits: max samples, max instances, and max samples per instance |
S |
C[/H[/S/I/SpI]] | t | set durability service: cleanup delay C, history setting H (same as k setting), resource limits (same as R) |
u |
{y,n} | w | set autodispose unregistered instances to true/false |
U |
T | r, w | set user data to T, C-style escape-sequence processing is applied |
V |
K0[:K1[:...]] | r | set subscriber-defined keys to fields K0, K1, &c. |
x |
D | r | set auto-purge disposed instances delay |
All durations are in seconds and may be floating-point numbers. Durations and limits also accept "inf" for infinite/unlimited.
If a QoS provider has been configured, then the third form is passed through to the QoS provider unchanged, except that an empty string is interpreted to mean the default value. Note that FS:QOS form allows updating individual QoS settings, but that the QoS provider always sets all QoS settings for the entity. Hence, the QoS provider is really only useful for overriding the default QoS used by pubsub.
Whether or not a reader (for a topic) is created, and in what mode it operates is primarily controlled by the -m option, which has various possible argument forms:
| argument | meaning |
|---|---|
0 |
no reader for the current topic specification |
p |
print received data, reading/taking alive and not-alive-no-writers data and taking disposed data -- whether it reads or takes is controlled by the -R option |
pp |
as p, except that it polls rather than waits for data to arrive and always takes data |
c[:N] |
"check" mode, applicable only to KS, K32, ..., K256 modes. Takes all incoming samples on a data_available trigger, and prints the number of received samples, out-of-order samples and throughput, when triggered and when the previous line was output >= 1s ago. The out-of-order checking is based on expecting the topic's sequence number field to increment for each sample and the keys to cycle from 0 to N-1 (exactly corresponding to the -wN writer mode). N is a global setting, the last one set wins, the default is 1. |
cp[:N] |
polling variant of the above |
d |
uses simpler triggering than p, reads/takes everything and prints it |
dp |
polling variant of the above, always takes data |
z |
zero-load mode; creates a reader but does nothing with it; changes the (pubsub) default history setting to keep-last 1, but still allows the -q option to override it |
In addition, there are the following options affecting behaviour:
| option | argument | per-reader | meaning |
|---|---|---|---|
-n |
N | yes | limit read/take to N samples |
-s |
T | yes | sleep T ms after each read/take (default: 0) |
-$ |
yes | perform one final, unlimited take-all just before stopping in p and d modes |
|
-F |
no | explicitly set line-buffering mode, useful when piping the output into a script that filters and outputs the results | |
-W |
T | no | call wait_for_historical_data with timeout T |
-O |
no | "once" mode, triggers termination after reading data once immediately (though after having called wait_for_historical_data if requested) -- this is the same take that otherwise is enabled with -$ as the "final take"; exist status is 0 if data was read, 1 if not |
Then, there is a global -P option controlling what gets printed. The -P option takes a comma-separated list of keywords turning on/off various outputs. In the three tables below, those marked with "yes" in the second column can be inverted by prefixing them with "no":
| keyword | "no" prefix? | meaning |
|---|---|---|
| meta | yes | enable printing of all sample metadata |
| trad | yes | "traditional" (i.e., older pubsub versions) flags: pid, time, phandle, stime, state |
| pid | yes | process id of pubsub |
| topic | yes | which topic |
| time | yes | read time relative to program start |
| phandle | yes | publication handle |
| ihandle | yes | instance handle |
| stime | yes | source timestamp |
| rtime | yes | reception timestamp |
| dgen | yes | disposed generation count |
| nwgen | yes | no-writers generation count |
| ranks | yes | sample, generation, absolute generation ranks |
| state | yes | instance/sample/view states |
The sample meta data is always printed in the above order, with spaces (and in some cases, space-colon-space) separating the fields. For ARB mode and FILE modes of the -K option, it is possible to control whether or not the topic type is printed and as well as the formatting of the sample contents:
| keyword | "no" prefix? | meaning |
|---|---|---|
| type | no | print type definition at start up |
| dense | no | no additional white space, no field names |
| fields | no | field names (C99-style designated initializers), some white space |
| multiline | no | field names (C99-style designated initializers), one field per line |
For the KS, K32, ... K256 modes, a sample is always printed as the sequence number followed by the key value (separated by a space); in the case of invalid samples (those that do not have the "valid_data" flag in the sample info set), it prints "NA" for the sequence number. For the OU mode, it is just the sequence number or "NA".
Finally, there are the ones that do not fit anywhere else:
| keyword | "no" prefix? | meaning |
|---|---|---|
| finaltake | yes | print a "final take" notice before the results of the optional final take (see -$ option above) just before stopping. |
The default is "nometa,state,fields,finaltake".
Each reader runs in a separate thread with its own waitsets and mode settings (except for some settings, such as the number of key values in checking mode). Output always goes to stdout and uses file locking to keep the lines separate.
Analogous to the reader mode selection option, there is the -w writer mode selection option (-w is probably more mnemonic of "writer" than -m is of "reader", but hey, it grew over time). The -w option argument is interpreted as follows:
| argument | auto | meaning |
|---|---|---|
0 |
n/a | no writer for the current topic specification |
| N | yes | cycle through N key values as fast as possible, incrementing the sequence number for each sample (see also -m option) |
| N:R | yes | as N but at a rate of R samples/second (rate in floating-point) |
| N:R*B | yes | as above, but writing bursts of B samples at a rate of R bursts/second (rate in floating-point) |
- |
no | (a hyphen) read from stdin, the default |
| :P | no | listen on port P for a TCP connection, then read from that connection until closed and repeat |
| H:P | no | establish a TCP connection to host H, port P and read from it |
Each writer in "automatic" mode (the "auto" column above) runs in a separate thread until pubsub termination is triggered, either by end-of-input on a non-automatic writer or by reaching the time limit set by -D. Note that the TCP-server mode is an odd one as it automatically repeats when the client drops the connection, rather than triggering termination.
The non-automatic modes all read from the same input and in a single thread. Only the last input specification given is actually used. If invoked as pubsub or pub the default writer mode is - (read from stdin), hence specifying a different input source once changes all (non-auto) writers to use the alternative one. However, if invoked as sub writers are only created if explicitly requested, and in that case it can be convenient to specify mode - for the first writers that you do want created, and the actual input source for the final writer that you do want created.
The automatic modes are available only for the KS, K32, ..., K256 and OU types, and for the OU type, the actual value of the N argument is irrelevant as long as it is positive (as it doesn't have a key). The non-auto ones are available for all types. The automatic modes report once per 4s how many samples they have written, plus an ASCII-art single-line histogram showing the distribution of the time it takes to perform the write operation.
Behaviour can be further configured with the following options:
| option | argument | applicability | meaning |
|---|---|---|---|
-D |
D | auto | run for D seconds (D may be floating-point); this option affects all automatic writers |
-r |
auto | pre-register instances, than write using the instance handles | |
-z |
N | auto KS | set the size of the octet sequence to N-12 bytes in the KS mode (12 bytes is occupied by key, sequence number and sequence length, so this gives an N-byte sample) |
-@ |
non-auto | write an exact copy of everything on a duplicate writer |
In principle, the input is interpreted as a white-space separated sequence of commands. This is strictly the case for the KS, K32, ..., K256 and OU modes, and mostly the case for arbitrary-type mode. In the latter mode, command-processing is line-based and some commands take the remainder of the line.
The commands and their arguments are:
| command | argument | meaning |
|---|---|---|
| V | synonym for the w command |
|
w |
V | write sample with value V |
d |
V | dispose V |
D |
V | write-dispose V |
u |
V | unregister V |
r |
V | register V |
s |
N | sleep for N seconds (N an integer) |
n |
N | sleep ("nap") for N microseconds (N an integer) |
z |
N | set sequence size to N-12 (KS only, see also -z option) |
p |
PS | set publisher partitions to the comma-separated list PS (note that QoS changes are not supported by the DDSI2 service at the time of writing) |
Y |
invoke the "dispose_all" operation on the topic | |
B |
begin coherent changes | |
E |
end coherent changes | |
W |
wait_for_acknowledgements with timeout set to infinite | |
S |
P;T;U | make a persistent snapshot for partition expression P, topic expression T, and destination URI U |
C |
T;F;D | write DDSI control topic (if feature enabled in config): T = {self,all,id} systemId of target; F = {d,m,dm} whether deaf, mute or deafmute; D = duration in seconds (floating or inf) |
: |
N | select the non-auto writer number N (0-based), if N has a sign, move towards the right (+) or the left (-) N-th non-auto writer |
: |
NAME | select the unique writer of which the topic name starts with NAME |
The value V is a single integer in non-arbitrary-type modes, a C99-style (designated) initializer in arbitrary-type modes. Typically that means the value starts with a '{' in arbitrary-type mode, as typically the topic type is a struct.
The initializers in arbitrary-type mode are mostly, but not exactly C99 (designated) initializers. The most important difference is that IDL unions are discriminated, and in pubsub the base format for union initialization is the discriminant value followed by a colon, followed by the value. However, if the value is specified with a member name, the discriminant value will be set automatically.
The write/dispose/&c. commands can optionally specify the seconds component of the source timestamp by appending an @ to the command letter, e.g., w@T. If the timestamp starts with a =, it is an absolute timestamp, otherwise relative to the current time. So w@-3 24 in non-arbitrary-type mode writes key value 24 with a timestamp 3s in the past, and w@=10 314159 would write with a timestamp of 10.0s since the epoch of 1 Jan 1970.
In arbitrary-type mode, the p, S and : commands interpret the remainder of the line, for all other commands, interpretation continues on the same line. In non-arbitrary type modes, the argument always ends at the first white-space character, and hence in non-arbitrary type modes, the possible partition names, URIs, &c. are limited.
| option | argument | meaning |
|---|---|---|
-M |
T:U | wait at most T seconds for a each writer to detect a matching reader with user data U and not owned by this instance of pubsub; terminates on time-out |
-S |
ES | set listeners for the events in ES (see below) |
-* |
N | sleep for N seconds just before returning from main(), after deleting all entities |
-! |
disable built-in signal handlers for the INT and TERM signals that trigger graceful termination of pubsub, just like end-of-file does for non-automatic writers |
The events for which listeners can be set with the -S option are specified as a comma-separated list of keywords, either the abbreviated form or the full form:
| abbreviated form | full form |
|---|---|
pr |
pre-read |
sl |
sample-lost |
sr |
sample-rejected |
lc |
liveliness-changed |
sm |
subscription-matched |
riq |
requested-incompatible-qos |
rdm |
requested-deadline-missed |
ll |
liveliness-lost |
pm |
publication-matched |
oiq |
offered-incompatible-qos |
odm |
offered-deadline-missed |
The pre-read event is not a listener, just a line printed when a pubsub waitset has been triggered and data is about to be read. The listeners all print decoded information, which for the "incompatible QoS" listeners includes the names of the incompatible QoS.
pubsub testcreates a reader and a writer in partition "test" with standard QoS (see below for the actual QoS used), for typeKSand with a topic name ofPubSub.pubsub -KOU -Tfoo bar bazcreates a topic "foo" for the OU type in the partitions bar and bazpubsub -TDCPSHeartbeat:'id.systemId<>$SYSTEMID' '__BUILT-IN PARTITION__'looks up the "DCPSHeartbeat" topic (which is an OpenSplice built-in topic and hence always present), derives a content-filtered topic that selects only those for which the systemId differs from that of PubSub itself and creates a reader and a writer with the topic QoS in the "__BUILT-IN PARTITION__" partition. Each node joining/leaving the network will be printed.pubsub -KKS -TA -KKS -TA ""will create twoKSreaders/writers both for topic A and A in the default partition, and consequently any data written will be received (and printed) by both readers.pubsub -qrw:k=1 testuses topic "PubSub" of typed "KeyedSeq" (KS), by-source ordering, reliability kind reliable, &c., but with a keep-last 1 history rather than the (pubsub) default of keep-all.echo '{.deaf_mute=true, .duration=30}' | pubsub -m0 -qps:P=t -Tq_ddsiControl '$NODE_BUILTIN_PARTITION'creates a by-topic coherent publisher with a writer for the q_ddsiControl topic (pre-defined by DDSI2 if the control interface is enabled) in the node-local built-in partition, and publishes a command to make DDSI2 deaf-mute for the next 30s.pubsub -w0 -mcp:1 -z1024 testandpubsub -w1 -r -m0 testgive a simple throughput test with 1kB-byte large samples of the KeyedSeq type (with reliable delivery and by-source timestamp ordering)pubsub -w0 -K<(./idl2md ShapeType.idl ShapeType) -TCircle -Ptype,multiline -Ssm,riq -qrw:o=r,r=n ''defines the Circle topic using the type and key definition from the output of the "idl2md" script and creates a best-effort, order-by-reception-timestamp reader in the default partition. It prints the type definition, it prints the samples one field per line, and it has listeners set that show when matching writers come and go, and when incompatible writers are detected.pubsub -qps:P=g -qt:d=p -w:12345 -KKS -TA -KKS -TB -Pmeta barfandecho B 1 2 :B 3 4 E | nc localhost 12345uses "netcat" to pipe a group transaction containing two samples of topic A and two samples of topic B into pubsub for publication; and has pubsub printing these with full sample meta-data. (The readers are independent in pubsub, so the output is not necessarily a coherent set.)- A simple persistently-written RnR scenario:
echo '{.scenarioName="xyzzy", .rnrId="rnr", .kind=ADD_RECORD_COMMAND:{ .storage="st", .interestExpr
={"*.PubSub"} }}
{.scenarioName="BuiltinScenario", .rnrId="rnr", .kind=START_SCENARIO_COMMAND:"xyzzy"}' | \
./pub -qw:d=p -Trr_scenario RecordAndReplay
(sleep 1 ; for x in 1 2 3 4 5 6 ; do echo $x ; sleep 1 ; done ; sleep 1) | ./pubsub test
echo '{.scenarioName="BuiltinScenario", .rnrId="rnr", .kind=STOP_SCENARIO_COMMAND:"xyzzy"}
d{.scenarioName="xyzzy", .rnrId="rnr"}' | \
./pub -qw:d=p -Trr_scenario RecordAndReplay