FirebaseExtended/firebase-import

file smaller than 400mb but can't import at "/" path, failed: write_too_big

barust-a opened this issue · 2 comments

Hello I have problem to import firebase realtime database (360mb), When i put export.json to root of my database i got this error (below) but when i choose a path other than "/" like "--path /test" it work, I don't understand why it doesn't work at the root.

`Alexandre:Documents alex$ firebase-import --database_url https://url.firebaseio.com/ --path / --json export.json --service_account /path/to/key/my_key.json 

All data at https://url.firebaseio.com/ will be overwritten.
Press <enter> to proceed, Ctrl-C to abort.
Reading /Users/alex/Documents/export.json... (may take a minute)

Preparing JSON for import... (may take a minute)
[2019-07-09T18:53:23.902Z]  @firebase/database: FIREBASE WARNING: set at / failed: write_too_big 
[2019-07-09T18:53:23.907Z]  @firebase/database: FIREBASE WARNING: Exception was thrown by user callback. Error: WRITE_TOO_BIG: Data to write exceeds the maximum size that can be modified with a single request.
    at /usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:13139:33
    at exceptionGuard (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:695:9)
    at Repo.callOnCompleteCallback (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:13130:13)
    at /usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:12907:19
    at /usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:12078:17
    at PersistentConnection.onDataMessage_ (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:12111:17)
    at Connection.onDataMessage_ (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:11394:14)
    at Connection.onPrimaryMessageReceived_ (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:11388:18)
    at WebSocketConnection.onMessage (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:11289:27)
    at WebSocketConnection.appendFrame_ (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:10892:18) 
/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:706
            throw e;
            ^

Error: WRITE_TOO_BIG: Data to write exceeds the maximum size that can be modified with a single request.
    at /usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:13139:33
    at exceptionGuard (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:695:9)
    at Repo.callOnCompleteCallback (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:13130:13)
    at /usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:12907:19
    at /usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:12078:17
    at PersistentConnection.onDataMessage_ (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:12111:17)
    at Connection.onDataMessage_ (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:11394:14)
    at Connection.onPrimaryMessageReceived_ (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:11388:18)
    at WebSocketConnection.onMessage (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:11289:27)
    at WebSocketConnection.appendFrame_ (/usr/local/lib/node_modules/firebase-import/node_modules/@firebase/database/dist/index.node.cjs.js:10892:18)
`

@barust-a Sorry for the delay. I think this is likely because you have a lot of data in your database that would need to be deleted in order to import the new data at the root. This existing data makes the overall write operation too big. You could likely use the Firebase CLI (or maybe the Firebase console but it might hit the same issue) to delete the data, and then perform the import.

See
https://firebase.google.com/docs/cli#install_the_firebase_cli
https://firebase.google.com/docs/cli#rtdb-commands

FYI, here is a relevant blog post:
https://firebase.googleblog.com/2019/03/large-deletes-in-realtime-database.html
You can disable the write size limit protection temporarily.

firebase database:settings:set defaultWriteSizeLimit unlimited