Fatal database error after updating to latest jar
Closed this issue · 8 comments
leijurv commented
.08:20:33.606 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 115000 08:20:33.740 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:33.740 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 120000
08:20:33.875 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:33.875 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 125000
08:20:34.010 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:34.010 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 130000
08:20:34.148 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:34.149 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 135000
08:20:34.283 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:34.283 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 140000 08:20:34.418 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:34.418 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 145000
08:20:34.552 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:34.552 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 150000
08:20:34.687 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:34.687 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 155000 08:20:34.821 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN () .08:20:34.821 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 160000 08:20:34.955 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:34.956 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 165000
08:20:35.089 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:35.089 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 170000
08:20:35.222 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:35.222 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 175000
08:20:35.357 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:35.357 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 180000
08:20:35.494 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:35.494 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 185000
08:20:35.628 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.08:20:35.629 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 190000
08:20:35.762 DEBUG DatabaseUpdate.execute : Executing: DELETE FROM messages WHERE id IN ()
.
08:20:35.763 INFO DB_Update_9._doUpdate : Converted 91171 of 192686 messages.
Cleaning up the database (this might also take some time, sorry)...
08:20:35.763 DEBUG DatabaseUpdate.execute : Executing: VACUUM
An error occured!
java.lang.RuntimeException: org.sqlite.SQLiteException: [SQLITE_ERROR] SQL error or missing database (cannot V
ACUUM - SQL statements in progress)
at de.fabianonline.telegram_backup.DatabaseUpdates.doUpdates(DatabaseUpdates.kt:107)
at de.fabianonline.telegram_backup.Database.<init>(Database.kt:234)
at de.fabianonline.telegram_backup.Database.<init>(Database.kt:50)
at de.fabianonline.telegram_backup.Database$Companion.init(Database.kt:766)
at de.fabianonline.telegram_backup.CommandLineController.<init>(CommandLineController.kt:102)
at de.fabianonline.telegram_backup.CommandLineRunnerKt.main(CommandLineRunner.kt:41)
Caused by: org.sqlite.SQLiteException: [SQLITE_ERROR] SQL error or missing database (cannot VACUUM - SQL statements in progress)
at org.sqlite.core.DB.newSQLException(DB.java:909)
at org.sqlite.core.DB.newSQLException(DB.java:921)
at org.sqlite.core.DB.throwex(DB.java:886)
at org.sqlite.core.NativeDB._exec_utf8(Native Method)
at org.sqlite.core.NativeDB._exec(NativeDB.java:87)
at org.sqlite.jdbc3.JDBC3Statement.executeUpdate(JDBC3Statement.java:116)
at de.fabianonline.telegram_backup.DatabaseUpdate.execute(DatabaseUpdates.kt:168)
at de.fabianonline.telegram_backup.DB_Update_9._doUpdate(DatabaseUpdates.kt:445)
at de.fabianonline.telegram_backup.DatabaseUpdate.doUpdate(DatabaseUpdates.kt:155)
at de.fabianonline.telegram_backup.DatabaseUpdates.doUpdates(DatabaseUpdates.kt:104)
... 5 more
08:20:35.768 ERROR CommandLineController.<init> : Exception caught!
java.lang.RuntimeException: org.sqlite.SQLiteException: [SQLITE_ERROR] SQL error or missing database (cannot VACUUM - SQL statements in progress)
at de.fabianonline.telegram_backup.DatabaseUpdates.doUpdates(DatabaseUpdates.kt:107)
at de.fabianonline.telegram_backup.Database.<init>(Database.kt:234)
at de.fabianonline.telegram_backup.Database.<init>(Database.kt:50)
at de.fabianonline.telegram_backup.Database$Companion.init(Database.kt:766)
at de.fabianonline.telegram_backup.CommandLineController.<init>(CommandLineController.kt:102)
at de.fabianonline.telegram_backup.CommandLineRunnerKt.main(CommandLineRunner.kt:41)
Caused by: org.sqlite.SQLiteException: [SQLITE_ERROR] SQL error or missing database (cannot VACUUM - SQL statements in progress)
at org.sqlite.core.DB.newSQLException(DB.java:909)
at org.sqlite.core.DB.newSQLException(DB.java:921)
at org.sqlite.core.DB.throwex(DB.java:886)
at org.sqlite.core.NativeDB._exec_utf8(Native Method)
at org.sqlite.core.NativeDB._exec(NativeDB.java:87)
at org.sqlite.jdbc3.JDBC3Statement.executeUpdate(JDBC3Statement.java:116)
at de.fabianonline.telegram_backup.DatabaseUpdate.execute(DatabaseUpdates.kt:168)
at de.fabianonline.telegram_backup.DB_Update_9._doUpdate(DatabaseUpdates.kt:445)
at de.fabianonline.telegram_backup.DatabaseUpdate.doUpdate(DatabaseUpdates.kt:155)
at de.fabianonline.telegram_backup.DatabaseUpdates.doUpdates(DatabaseUpdates.kt:104)
... 5 common frames omitted
----- EXIT -----
leijurv commented
It seems to have run some extremely long sql commands, lists of thousands and thousands of message ids
leijurv commented
This is what the beginning of the output looks like:
leijurv@swamp:/mnt/data/Dropbox$ java -jar telegram_backup.jar --with-supergroups --with-channels --account +16503876684 --debug --target /mnt/tgdata/
08:30:40.351 DEBUG Utils.getNewestVersion : Requesting current release info from https://api.github.com/repos/fabianonline/telegram_backup/releases
08:30:41.581 DEBUG Utils.getNewestVersion : Found current release version 1.1.3
08:30:41.581 DEBUG Utils.compareVersions : Comparing versions 1.1.3-3-gad6f57d and 1.1.3.
08:30:41.600 DEBUG Utils.compareVersions : v1 has a suffix, v2 not.
08:30:41.601 INFO CommandLineController.<init> : CommandLineController started. App version 1.1.3-3-gad6f57d
Telegram_Backup version 1.1.3-3-gad6f57d, Copyright (C) 2016, 2017 Fabian Schlenz
Telegram_Backup comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions; run it with '--license' for details.
08:30:41.602 DEBUG CommandLineController.setupFileBase: Target dir at startup: /home/leijurv/.telegram_backup
08:30:41.602 DEBUG CommandLineController.setupFileBase: Target dir after options: /mnt/tgdata/
Base directory for files: /mnt/tgdata/
08:30:41.602 DEBUG CommandLineController.<init> : Initializing TelegramApp
08:30:41.602 DEBUG CommandLineController.selectAccount: Account requested: +1REDACTEDXX
08:30:41.602 DEBUG CommandLineController.selectAccount: accounts.size: 1
08:30:41.602 DEBUG CommandLineController.selectAccount: account: +1REDACTEDXX
08:30:41.602 DEBUG CommandLineController.<init> : CommandLineOptions.cmd_login: false
08:30:41.602 INFO CommandLineController.<init> : Initializing ApiStorage
08:30:41.603 INFO CommandLineController.<init> : Initializing TelegramUpdateHandler
08:30:41.603 INFO CommandLineController.<init> : Creating Client
08:30:42.194 INFO CommandLineController.<init> : Initializing UserManager
08:30:42.195 DEBUG UserManager.<init> : Calling getFullUser
08:30:42.323 DEBUG CommandLineController.<init> : CommandLineOptions.cmd_login: false
Opening database...
08:30:42.376 DEBUG DatabaseUpdates.<init> : Registering Database Updates...
08:30:42.376 DEBUG DatabaseUpdates.register : Registering class de.fabianonline.telegram_backup.DB_Update_1 as update to version 1
08:30:42.377 DEBUG DatabaseUpdates.register : Registering class de.fabianonline.telegram_backup.DB_Update_2 as update to version 2
08:30:42.377 DEBUG DatabaseUpdates.register : Registering class de.fabianonline.telegram_backup.DB_Update_3 as update to version 3
08:30:42.377 DEBUG DatabaseUpdates.register : Registering class de.fabianonline.telegram_backup.DB_Update_4 as update to version 4
08:30:42.377 DEBUG DatabaseUpdates.register : Registering class de.fabianonline.telegram_backup.DB_Update_5 as update to version 5
08:30:42.377 DEBUG DatabaseUpdates.register : Registering class de.fabianonline.telegram_backup.DB_Update_6 as update to version 6
08:30:42.378 DEBUG DatabaseUpdates.register : Registering class de.fabianonline.telegram_backup.DB_Update_7 as update to version 7
08:30:42.378 DEBUG DatabaseUpdates.register : Registering class de.fabianonline.telegram_backup.DB_Update_8 as update to version 8
08:30:42.378 DEBUG DatabaseUpdates.register : Registering class de.fabianonline.telegram_backup.DB_Update_9 as update to version 9
08:30:42.378 DEBUG DatabaseUpdates.doUpdates : DatabaseUpdate.doUpdates running
08:30:42.378 DEBUG DatabaseUpdates.doUpdates : Getting current database version
08:30:42.378 DEBUG DatabaseUpdates.doUpdates : Checking if table database_versions exists
08:30:42.379 DEBUG DatabaseUpdates.doUpdates : Table exists. Checking max version
08:30:42.379 DEBUG DatabaseUpdates.doUpdates : version: 8
Database version: 8
08:30:42.379 DEBUG DatabaseUpdates.doUpdates : Max available database version is 9
08:30:42.379 DEBUG DatabaseUpdates.doUpdates : Update is necessary. 8 => 9.
08:30:42.379 DEBUG DatabaseUpdates.doUpdates : Update to version 9 needs a backup
08:30:42.379 DEBUG DatabaseUpdates.doUpdates : Performing backup
Creating a backup of your database as database.version_8.backup.sqlite
08:30:42.379 DEBUG Database.backupDatabase : Copying /mnt/tgdata//+1REDACTEDXX/database.sqlite to /mnt/tgdata//+1REDACTEDXX/database.version_8.backup.sqlite
08:30:42.490 DEBUG DatabaseUpdates.doUpdates : Applying updates
08:30:42.491 DEBUG DatabaseUpdate.doUpdate : Applying update to version 9
Updating to version 9...
Updating supergroup channel message data (this might take some time)...
08:30:42.562 DEBUG DB_Update_9._doUpdate : Found 192686 candidates for conversion
08:30:42.562 DEBUG DB_Update_9._doUpdate : Querying with limit 5000 and offset 0
leijurv commented
This could be because my database is very big and I have a lot of messages and media:
4.0K auth.dat
335M database.sqlite
48K database.sqlite-journal
335M database.version_8.backup.sqlite
4.0K dc.dat
75G files
itskenny0 commented
leijurv commented
Thank you, that fixes it
fabianonline commented
A real bugfix is in the works. Sorry for taking so long. :-(
fabianonline commented
1.1.4 is out.