icelake-io/icelake

Can't get `TableMetadata` file location from `Table`

Closed this issue · 2 comments

nooberfsh commented

Hi.
I'm currently implementing a PostgreSQL Catalog, the database has a table iceberg_tables which keeps the location of
the current table's the metadata file, when I make commit to a table, I need the original metadata location to perform
an atomic swap. Table doesn't store metadata file location, is it feasible to store it in Table?

Xuanwo commented

I don't think storing the table metadata file location in Table is necessary. I have made similar comments regarding this issue on iceberg-rust: apache/iceberg-rust#75 (comment).

The catalog should be responsible for maintaining the correct location and updating it using Catalog::update_table.

nooberfsh commented

I think it is necessary. The JDBC Catalog stores metadata_location in a table which has the schema:

catalog_name table_namespace table_name metadata_location previous_metadata_location

When you commit some change to a table you should make sure the table is not changed. So the Catalog should check the metadata_location remains the same otherwise it should abort the commit or retry it.