larq/zookeeper

Support accurate dataset length computation in `TFDSDataset` components in TF2.3

AdamHillier opened this issue · 0 comments

From the TF 2.3 release notes:

  • tf.data.experimental.cardinality is now a method on tf.data.Dataset.
  • tf.data.Dataset now supports len(Dataset) when the cardinality is finite.

We should make use of these functions here:

def num_examples(self, split) -> int:
"""Compute the number of examples in a given split."""
return sum(self.splits[s].num_examples for s in base_splits(split))