cldf/csvw

Support zipped data files

Closed this issue · 0 comments

It should be possible to read data from zipped CSV files and write data to zipped files transparently. I.e. something along the lines of

  • looking up fname.parent / (fname.name + '.zip') here:

    csvw/src/csvw/metadata.py

    Lines 645 to 672 in 58be38c

    def iterdicts(self, log=None, with_metadata=False, fname=None, _Row=collections.OrderedDict):
    """Iterate over the rows of the table
    Create an iterator that maps the information in each row to a `dict` whose keys are
    the column names of the table and whose values are the values in the corresponding
    table cells, or for virtual columns (which have no values) the valueUrl for that
    column. This includes columns not specified in the table specification.
    :param log: Logger object (default None) The object that reports parsing errors.\
    If none is given, parsing errors raise ValueError instead.
    :param bool with_metadata: (default False) Also yield fname and lineno
    :param fname: file-like, pathlib.Path, or str (default None)\
    The file to be read. Defaults to inheriting from a parent object, if one exists.
    :return: A generator of dicts or triples (fname, lineno, dict) if with_metadata
    """
    dialect = self._get_dialect()
    fname = fname or self.url.resolve(self.base)
    colnames, virtualcols, requiredcols = [], [], set()
    for col in self.tableSchema.columns:
    if col.virtual:
    if col.valueUrl:
    virtualcols.append((col.header, col.valueUrl))
    else:
    colnames.append(col.header)
    if col.required:
    requiredcols.add(col.header)
    with UnicodeReaderWithLineNumber(fname, dialect=dialect) as reader:
  • supporting a zip=False keyword argument here
    def write(self, items, fname=DEFAULT, base=None):