Column families

Question

Column families

adamjabone opened this issue 6 years ago · 12 comments

I can't find out how to pass arguments to level(location[, options[, callback]]) so I can open column families.
I`am still getting error:

(node:22916) UnhandledPromiseRejectionWarning: OpenError: Invalid argument: You have to open all column families. Column families not opened: col0, col1, col2, col3, col4, col5, col6, col7
    at C:...project\node_modules\levelup\lib\levelup.js:87:23
    at C:...project\node_modules\abstract-leveldown\abstract-leveldown.js:41:14
    at C:...project\node_modules\deferred-leveldown\deferred-leveldown.js:20:21
    at C:...project\node_modules\abstract-leveldown\abstract-leveldown.js:41:14
    at C:...project\node_modules\abstract-leveldown\abstract-leveldown.js:41:14
(node:22916) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:22916) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

How can I open column families?

Answer 1 · 2018-09-04T07:55:13.000Z

Can you share code to reproduce? I didn't know what column families were and would have guessed it's not exposed in the [level-]rocksdb API - but the error message you posted suggests otherwise.

Answer 2 · 2018-09-04T09:08:47.000Z

Code is very simple:

var level = require('level-rocksdb')
var dbPath = 'C:\\Users\\User\\AppData\\Local\\Parity\\Ethereum\\chains\\ethereum\\db\\906a34e69aec8c0d\\overlayrecent\\db';  // database path
var db = level(dbPath);

And error:

(node:22916) UnhandledPromiseRejectionWarning: OpenError: Invalid argument: You have to open all column families. Column families not opened: col0, col1, col2, col3, col4, col5, col6, col7
at C:...project\node_modules\levelup\lib\levelup.js:87:23
at C:...project\node_modules\abstract-leveldown\abstract-leveldown.js:41:14
at C:...project\node_modules\deferred-leveldown\deferred-leveldown.js:20:21
at C:...project\node_modules\abstract-leveldown\abstract-leveldown.js:41:14
at C:...project\node_modules\abstract-leveldown\abstract-leveldown.js:41:14
(node:22916) UnhandledPromiseRejectionWarning: Unhandled promise rejection.
This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:22916) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

It is rocksdb created and used by program Parity. I try to use the database with JavaScript code.

Answer 3 · 2018-09-04T09:34:24.000Z

I see. The database wasn't created by level-rocksdb, but by another program that does support column families? And it wrote data that level-rocksdb can't read.

To support it, we need to pass column family options from JS -> binding -> RocksDB. Would you be up for making a PR?

Answer 4 · 2018-09-04T09:59:04.000Z

Yes, I will wait for PR. First I have tried to open the database as leveldb with levelup(leveldown(dbpath), hope this didn`t mess anything.

Answer 5 · 2018-09-04T11:33:57.000Z

I have same error using Java:

import org.rocksdb.RocksDB;
import org.rocksdb.Options;
import org.rocksdb.RocksDBException;

public class Main {
    static String dbPath = "C:/Users/ja1/AppData/Local/Parity/Ethereum/chains/ethereum/db/906a34e69aec8c0d/overlayrecent/db";

    public static void main(String[] args)
    {
        System.out.println("Hello World!");
        // a static method that loads the RocksDB C++ library.
        RocksDB.loadLibrary();

        // the Options class contains a set of configurable DB options
        // that determines the behaviour of the database.
        try (final Options options = new Options().setCreateIfMissing(false)) {

            // a factory method that returns a RocksDB instance
            try (final RocksDB db = RocksDB.open(options, dbPath)) {
                System.out.println("Opened");

                // do something
            }
        } catch (RocksDBException e) {
            // do some error handling
            System.out.println("Did not opened");
            System.out.println(e);

        }
    }
}

Error:

Hello World!
Did not opened
org.rocksdb.RocksDBException: You have to open all column families. Column families not opened: col0, col1, col2, col3, col4, col5, col6, col7

So source of problem can be elsewhere.

Answer 6 · 2018-09-04T11:36:09.000Z

May help you: https://github.com/facebook/rocksdb/wiki/RocksJava-Basics#opening-a-database-with-column-families

Answer 7 · 2018-09-04T11:42:15.000Z

Thank you, this will help me.

Answer 8 · 2018-09-04T12:54:55.000Z

It helped, I opened family col0, col1, etc. and it works, do you know how to pass same options in level-rocksdb?

Answer 9 · 2018-09-04T13:16:38.000Z

This is not currently supported. Apologies if that wasn't clear.

Answer 10 · 2018-09-04T13:18:11.000Z

That was clear, I wanted to assert, thanks.

Answer 11 · 2019-01-01T13:27:48.000Z

See Level/rocksdb#13 for passing options to the RocksDB binding.