Reuse a common block of fields in multiple messages

[donmendelson]

Currently, the schema only allows blocks of fields to be defined inline as the root of a message or as a repeating group. However, blocks of fields are often reused in multiple FIX messages. Repeating groups belong to multiple messages. For example, InstrumentLegGrp is used in many message types, including SecurityStatus and MarketDataStatisticsReport. Additionally, the FIX Repository (from which message specs are generated) also has a concept of a common component, even when it is not part of repeating group. For example, Instrument component is group of fields that hold all the attributes of a traded instrument. It is used the majority of FIX message types but only needed to be defined the Repository once.

SBE schema should have equivalent features to define repeating groups and blocks once and reference them in any number of messages.

[donmendelson]

We already have an XML type to support reusable blocks of fields, named "blockType". (Not a coincidence--this feature request was anticipated.) It is the same type currently used for the root of a message and repeating groups. Thus, messages, reusable blocks and repeating group entries can be treated polymorphically by a schema parser, and also recursively, since blocks may contain nested groups.

I suggest we use new XML element <block> for reusable blocks.

Two questions about XSD design for reusable blocks.

1. Where to put reusable blocks? A <block> could be another child of <types>, or we could create a new container <blocks>. (Aside: should we have created <messages> to contain instances of <message>?)
2. How to refer to a reusable block in a message template? We have already defined a token "ref" for reference to an existing encoding type from a composite. Reuse the same token to refer to <block>, or use a different one? In other words, are blocks and types in the same namespace or different namespaces?

[donmendelson]

Prototype example:

    <blocks>
        <block id="1003" name="Instrument">
            <field id="55" name="Symbol" type="idString" />
            <field id="65" name="SymbolSfx" type="idString" />
            <field id="48" name="SecurityId" type="idString" />
            <field id="22" name="SecurityIdSource" type="securityIdSourceEnum" />
        </block>
    </blocks>

    <message id="14" name="NewOrderSingle">
        <field name="ClOrdId" id="11" type="idString" />
        <field name="Account" id="1" type="idString" />
        <field name="Side" id="54" type="sideEnum" />
        <field name="OrderQty" id="38" type="qtyEncoding"  />
        <field name="OrdType" id="40" type="ordTypeEnum"  />
        <field name="Price" id="44" type="decimal" />
        <!-- Use the Instrument block here -->
        <ref name="Product" id="1003" />
    </message>

The block is cross-referenced by its unique "id", not name.

[donmendelson]

Implications:

• Fields in a reusable block may have offset attribute, but it would only represent offset from the beginning of the block, not offset in a message that uses the block.
• A message may have multiple instances of a block, e.g. instrument and underlying instrument if instrument is a derivative. Therefore, a block reference needs a name to distinguish usage.
• SBE optimizes direct access by placing all fixed-length elements before all variable-length elements in a message, i.e. fields before repeating groups and data. However, if a block contains a repeating group and multiple blocks are used to form a message, it is possible to violate this rule.

[mjpt777]

If composites are full fledged types then what value do blocks bring? A field can a type that points to a composite in the example above.

The issue I come across is that composites don't have fields in them, only other type declarations. When building a system like this I like to think of things are fully fledged types or they are macros. Hybrids are a little weird.

A type can be mapped to a class in an OO language, e.g. BaseInstrument that could have the fields of your instrumentBlock example.

If you want the fields to be common and expanded in place without semantic scoping then the ref in your case could substitute the block into its place.

The main question is do we want to have a semantic scope for the block/type/component that can be mapped to an high level language and thus be treated as a type passed to a method so behaviour can be generalised.

[donmendelson]

Solution deferred until it can be prototyped and thoroughly reviewed.