Opentype STAT table: stuck on pointer to array of pointers to structs
Closed this issue · 3 comments
(please pardon the length and difficulty of this report, but it is a complicated data structure they've defined in Opentype, and it just may be beyond what restructure was defined to do, at least directly?)
I am interested in working up a parser for Opentype STAT Style Attributes Tables for use in Fontkit, and thus I am using restructure in the structure definitions.
I'm stuck on a complication with one part of the STAT specification and need suggestions, and/or the restructure code may need fixes or new features to handle this scenario:
- an array size plus pointer to an array of pointers to structures
I have everything working in one self-contained Mocha test (attached) versionSTAT01.js.txt for running within a restructure git clone, using sample data embedded in the file, but with one remaining problem.
STAT field axisValueCount has the length of the linked array. The field offsetToAxisValueOffsets points to an array. That array then contains pointers to the desired axis value tables. So there are three levels of data - initial STAT table, intermediate separate array of pointers, and separate axis value tables.
All that would seem to translate to be something like:
Pointer(uint, Array( Pointer(uint, valueStruct), 'theSize') )
But I am unable to use Pointer() twice in the same line. I am also unable to acquire the array count from the initial table to size the array. I can't find any combination of definitions that allows for describing
the situation as found in the STAT table.
.
Below I describe (at length) the things I've tried. It's a lot of description. You may want to instead first peek at the STAT table spec and play with the attached Mocha test file.
.
In the same STAT structure there is another pointer, to an array of structs, and the below code works for that:
DesignAxisCount: r.uint16,
OffsetToDesignAxes:
new r.Pointer(r.uint32,
new r.Array(
AxisRecord,
'DesignAxisCount')),
For the above "pointer to array of structs" the array count is accessed correctly and the definition for this simpler setup works great.
The problem is with a pointer that links to an array, but that array itself contains pointers to the desired final structures. The array size comes from the original struct:
AxisValueCount: r.uint16,
OffsetToAxisValueOffsets:
new r.Pointer(r.uint32,
new r.Array(
new r.Pointer(r.uint16, AxisValue),
'AxisValueCount')),
For a "pointer to array of pointers to structs" the array count can't be picked up from the initial struct.
If I try to reference the array size using the preceding field,
AxisValueCount: r.uint16,
OffsetToAxisValueOffsets:
new r.Pointer(r.uint32,
new r.Array(
new r.Pointer(r.uint16, AxisValue),
'AxisValueCount')
),
I get the error reason: "Error: Unknown version 6"
C:\Toms\Study\fonts\javascript\restructure-0.5.4\src\VersionedStruct.js:37 throw new Error("Unknown version " + res.version);
If I try to hard-code the array count to 6, this code also fails
OffsetToAxisValueOffsets:
new r.Pointer(r.uint32,
new r.Array(
new r.Pointer(r.uint16, AxisValue),
6)
),
with the exact same reason. From these two tests it looks like mentioning r.Pointer() twice in one line causes confusion...
If I separate the pointer and array definitions, like so:
AxisValueArray = new r.Struct( {
AxisValues: new r.Array( new r.Pointer(r.uint16, AxisValue), 6),
})
. . . . .
OffsetToAxisValueOffsets:
new r.Pointer(r.uint32, AxisValueArray),
the correct result is obtained (it works). But note I'm hard-coding the array size to 6.
If I try to reference the array size count from the original struct,
AxisValueArray = new r.Struct( {
AxisValues: new r.Array( new r.Pointer(r.uint16, AxisValue), 'AxisValueCount'),
})
. . . . .
OffsetToAxisValueOffsets:
new r.Pointer(r.uint32, AxisValueArray),
this fails with "Error: Not a fixed size"
C:\Toms\Study\fonts\javascript\restructure-0.5.4\src\utils.js:19 throw new Error('Not a fixed size');
Continuing to research possibilities I thought to check using functions for array size:
AxisValueArray = new r.Struct( {
AxisValues: new r.Array( new r.Pointer(r.uint16, AxisValue),
function(parent){
console.log("**D this: %j", this)
console.log("**D parent: %j", parent)
return 6
}),
})
as maybe I could chain upwards to find the array size value in the original first-level structure. But this displayed the astonishing:
**D this: {}
**D parent: {}
No parent?
I'm stuck. At this point it looks like extra code in two steps would be needed to handle the "pointer to array to pointers" situation. It looks like I'd have to use
OffsetToAxisValueOffsets: r.uint32,
then figure out the correct data offset to the intermediate array, and then again call restructure to separately parse the "array of pointers to structs", stuffing that result back into the first result structure. Foo.
Is this impossible given the existing restructure feature set? Ideas?
Attached test file: versionSTAT01.js.txt
Glad to hear you're playing around with this!
You have the right idea that it is a pointer to an array of pointers. I think the issue here has to do with which structures the offsets are relative to.
The spec says
Array of offsets to axis value tables, in bytes from the start of the axis value offsets array.
The outer pointer is relative to the start of the STAT table, but the inner pointers are relative to the start of the array.
restructure's pointers are relative to the nearest Struct, and so the inner pointers you've defined will also be relative to the STAT table instead of the array. To work around this you could do something like this:
let AxisValueArray = new r.Struct({
values: new r.Array(new r.Pointer(r.uint16, AxisValue), t => t.parent.axisValueCount)
});
let STAT = new r.Struct({
// ...
axisValueCount: r.uint16,
axisValues: new r.Pointer(r.uint32, AxisValueArray)
});There is an extra struct in there, which ensures that the pointers are relative to the start of the array (which equals the start of the struct) instead of the STAT table.
We could also add an option to the Array type in restructure to do this. It already does what we need in the case where the length is a Number type, but we could add an option to do this more generally.
Ahh, thank you, I was misleading myself when trying to check using the array count function because the parent property was not showing up in %j displays, cuz it's hidden (duh). So now using
var AxisValueArray = new r.Struct( {
AxisValues: new r.Array(new r.Pointer(r.uint16, AxisValue),
function(parent){ return parent.parent.AxisValueCount }),
})
together with
OffsetToAxisValueOffsets: new r.Pointer(r.uint32, AxisValueArray),
I'm working through the STAT table data as found in 4 different test font files (non-proprietary), which exercise varying data formatting, even found some that had obsolete deprecated STAT version numbers. So far I've seen these three different consequences of the above code:
Non-zero count and non-zero pointer:
"AxisValueCount": 6,
"OffsetToAxisValueOffsets": {
"AxisValues": [
{ ......
"Value": 600.0, // 600.0 39321600
},
Zero count and zero pointer:
"AxisValueCount": 0,
"OffsetToAxisValueOffsets": null
Zero count but non-zero pointer:
"AxisValueCount": 0,
"OffsetToAxisValueOffsets": {
"AxisValues": []
}
The output is always faithful to the input - yay!
I'll work towards cleaning up and contributing over at fontkit a new table parser. Could you review this test code and see if you'd like it contributed here?
versionSTAT01.js.txt
Glad it's working! I think tests for font specific things should go in fontkit. We can add a font with a STAT table and check that we get what we expected.