storesafe/cordova-sqlite-storage

Android not saving the inserts [workaround needed]

chintanop opened this issue · 37 comments

@brodybits Seems like a bunch of folks are facing this issue: on Android: https://groups.google.com/forum/#!topic/Cordova-SQLitePlugin/oq8wMDSH4M4

Cordova: 3.5.0
Android API Level: 19

Any idea?

It sounds like an issue introduced by recent updates to the Android sqlite classes. SQLite itself should never do this: https://www.sqlite.org/transactional.html

From December 2014:

On Tue, Dec 16, 2014 at 1:48 PM, mladenpetrovic88@gmail.com wrote:

I have been using this plugin for some time now, and recently i noticed that
sometimes plugin just doesn't insert anything in database. From the several
things that need to be initialized on app start, nothing is inserted, and of
course my whole app breaks.
I noticed this after updates to android support jar, cordova and few other
things.
I can't think of any reason why something like this would occur so please
any tips would be helpful.
Thank you

On Wed, Dec 17, 2014 at 1:59 PM, mladenpetrovic88@gmail.com wrote:

After hours of trying to figure out why/when does db fails i finally figured
it out.
When i start the app then exit it (not close/swipe it) and return back to
app sqlite plugin notifies me again in console that database is opened
again

SQLitePlugin openargs: {"name":"insideoutdb","bgType":1}
I am using cordova 4.0.0, Ionic 1.2.8 and sqlite 3.8.6 on lollipop and
kitkat.
My understanding is that database isn't closed until you explicitly close
the app ( or android kills it ) so when i come back to my app it tries to
open already opened db and it overwrites it or crash.

From February 2015:

On Sun, Feb 15, 2015 at 12:48 AM, Cesar Fernandes cesar.fa@gmail.com wrote:

I have the same problem.... I am working with latest versions of cordova
and plugin...

I manage 3 sqlite tables. I make some inserts in one of the tables, and when
i close the app and come back, the inserts get lost... then for some reason
that i can not find out, i populate the second table with lots of entries
from one web service, and this one does not get lost.... i already tried
lots of things to solve this problem, but i am almost givin up...

I alway get messages that database is locked , or database was closed bu
remained 1 without closing....

On Sun, Mar 1, 2015 at 1:54 AM, COP chintan@trialx.com wrote:

Hi, I'm also having the same problem on Android

The inserts work fine till the APP is on, i.e. you can insert and select the
data.

Once, you KILL the app. All the inserts are gone! Works fine in iOS - the
Android is not working.

What I am suspecting now is the following commit in the Android database classes: aosp-mirror/platform_external_sqlite@d4f30d0

It references the following sqlite fossil commit: http://www.sqlite.org/src/info/6c4c2b7dba

I made a possible workaround in the branch xb-android-lock-workaround. Can you please let us know if this will solve the problem?

@brodybits Thanks a lot for quick turn-around! this worked!

Happy that it solves your problem. But it points to an issue with the Android sqlite library. My idea is to make this workaround an option for those who need it.

The workaround as proposed was validated by @chintanop and is now optional, as described in README.md.

What files should I overwrite from the workaround branch? I replaced only SQLitePlugin.java and SQLitePlugin.js , but seems to be still wrong

The workaround branch is no longer supported. Please completely remove your app from the Android simulator or device, install the latest update from the master branch of this plugin, and use the androidLockWorkaround option in sqlitePlugin.openDatabase, as described in the readme.

Ok! thank you very much! I will play with that now...

Also note that this is really a bug in the Android database libraries and not this project. Unfortunately I have not reproduced this issue myself. I have implemented a very crude workaround but this should really be fixed in the Android code itself!

As I said before, I have a theory that the issue was introduced in the following commit in the Android database classes: android/platform_external_sqlite@d4f30d0, which references the following sqlite fossil commit: http://www.sqlite.org/src/info/6c4c2b7dba

It would really help if someone could file a bug on the Android project itself at: https://code.google.com/p/android/issues/list

I really appreciate your help. I will write to Android developers like you
told...

Tks again
Em 20/03/2015 22:19, "Chris Brody" notifications@github.com escreveu:

Also note that this is really a bug in the Android database libraries and
not this project. Unfortunately I have not reproduced this issue
myself. I have implemented a very crude workaround but this should
really be fixed in the Android code itself!

As I said before, I have a theory that the issue was introduced in the
following commit in the Android database classes:
aosp-mirror/platform_external_sqlite@d4f30d0
aosp-mirror/platform_external_sqlite@d4f30d0,
which references the following sqlite fossil commit:
http://www.sqlite.org/src/info/6c4c2b7dba

It would really help if someone could file a bug on the Android project
itself at: https://code.google.com/p/android/issues/list

Reply to this email directly or view it on GitHub
#193 (comment)
.

Yeah, i think i ll have to write to android developers.. sorry because i
was hopefull.. i have to finish this project soon... :/

I re-installed the plugin it is still not working.. i can see in the catlog
the workaround is being called "Android db closing/locking workaround
applied" ...

I use a moto G android 4.4 ...

When i emulate it in eclipse, it is working , but not on my device.

On Fri, Mar 20, 2015 at 10:18 PM, a Cesar Fernandes cesar.fa@gmail.com
wrote:

I really appreciate your help. I will write to Android developers like you
told...

Tks again
Em 20/03/2015 22:19, "Chris Brody" notifications@github.com escreveu:

Also note that this is really a bug in the Android database libraries and

not this project. Unfortunately I have not reproduced this issue
myself. I have implemented a very crude workaround but this should
really be fixed in the Android code itself!

As I said before, I have a theory that the issue was introduced in the
following commit in the Android database classes:
aosp-mirror/platform_external_sqlite@d4f30d0
aosp-mirror/platform_external_sqlite@d4f30d0,
which references the following sqlite fossil commit:
http://www.sqlite.org/src/info/6c4c2b7dba

It would really help if someone could file a bug on the Android project
itself at: https://code.google.com/p/android/issues/list

Reply to this email directly or view it on GitHub
#193 (comment)
.

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

I just had an idea. The project at https://github.com/brodysoft/Cordova-sqlcipher-adaptor includes SQLCipher, which is based off an older of sqlite and should give you the functionality without this issue lurking. If you omit the key option (or set it to '') when opening a database, it will create or open your database without any encryption. Please let me know if this will work for you or not.

So let me see if i got it... I must remove SQLitePlugin and install
SQLCipher?

tks again for your help

On Sat, Mar 21, 2015 at 1:04 PM, Chris Brody notifications@github.com
wrote:

I just had an idea. The project at
https://github.com/brodysoft/Cordova-sqlcipher-adaptor includes
SQLCipher, which is based off an older of sqlite and should give you the
functionality without this issue lurking. If you omit the key option (or
set it to '') when opening a database, it will create or open your database
without any encryption. Please let me know if this will work for you or not.

Reply to this email directly or view it on GitHub
#193 (comment)
.

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

👍 yes!

@cesar1983 how is it going?

Well. Actually it did not work..

after calling :
SQLiteDatabase mydb = SQLiteDatabase.openOrCreateDatabase(dbfile, key,
null);

it hides the exception
net.sqlcipher.database.DatabaseObjectNotClosedException: Application did
not close the cursor or database object that was opened here

so I guess it is the same problem again...

I started upgrading my Córdova version that was 3.2 to give a try... But
i had to go out ... I AM coming back home and as soon i finish all my tries
i let you know...

Thank you very much and if you have any new idea let me know please.

@cesar1983 https://github.com/cesar1983 how is it going?

Reply to this email directly or view it on GitHub
#193 (comment)
.

If you switch between the regular SQLitePlugin version and the SQLCipher version, please make sure you completely uninstall the app. I should have told you earlier.

I think the error message shows that the built-in Android sqlite library holds onto the lock!

If you did already uninstall the old version, then we have to think further.

Well, i really think i did, but i am starting a new project from zero to
make sure....

On Sat, Mar 21, 2015 at 7:15 PM, Chris Brody notifications@github.com
wrote:

If you switch between the regular SQLitePlugin version and the SQLCipher
version, please make sure you completely uninstall the app. I should have
told you earlier.

I think the error message shows that the built-in Android sqlite library
holds onto the lock!

If you did already uninstall the old version, then we have to think
further.

Reply to this email directly or view it on GitHub
#193 (comment)
.

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

I started the project from zero, and added cipher plugin.

my openDatabase call is like this:

db = window.sqlitePlugin.openDatabase({name: "test.db", key: ""});

the app started, and first thigs i do are :

1 - get product list from a webservice - that went ok
2 - get client list from ws - that went ok, but i get a different message
in catlog at this time, for avery insert i make.. i took a screenshot ...
even though the inserts were effective.

[image: Inline image 1]

I go on, register one order, and close the app.. i come back, and i see
everything is there.. i make a new order, leave the app, close it, and
when i come back , there is nothing there anymore...

I just realized now, that onDestroy() method, from

On Sat, Mar 21, 2015 at 9:26 PM, a Cesar Fernandes cesar.fa@gmail.com
wrote:

Well, i really think i did, but i am starting a new project from zero to
make sure....

On Sat, Mar 21, 2015 at 7:15 PM, Chris Brody notifications@github.com
wrote:

If you switch between the regular SQLitePlugin version and the SQLCipher
version, please make sure you completely uninstall the app. I should have
told you earlier.

I think the error message shows that the built-in Android sqlite library
holds onto the lock!

If you did already uninstall the old version, then we have to think
further.

Reply to this email directly or view it on GitHub
#193 (comment)
.

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

I am sorry, a sent msg before finishing it..

.... I just realized now, that onDestroy() method, from java class, is
never called, when i close the app in my device. Do you think there is
any relation?

On Sun, Mar 22, 2015 at 10:31 AM, a Cesar Fernandes cesar.fa@gmail.com
wrote:

I started the project from zero, and added cipher plugin.

my openDatabase call is like this:

db = window.sqlitePlugin.openDatabase({name: "test.db", key: ""});

the app started, and first thigs i do are :

1 - get product list from a webservice - that went ok
2 - get client list from ws - that went ok, but i get a different message
in catlog at this time, for avery insert i make.. i took a screenshot ...
even though the inserts were effective.

[image: Inline image 1]

I go on, register one order, and close the app.. i come back, and i see
everything is there.. i make a new order, leave the app, close it, and
when i come back , there is nothing there anymore...

I just realized now, that onDestroy() method, from

On Sat, Mar 21, 2015 at 9:26 PM, a Cesar Fernandes cesar.fa@gmail.com
wrote:

Well, i really think i did, but i am starting a new project from zero to
make sure....

On Sat, Mar 21, 2015 at 7:15 PM, Chris Brody notifications@github.com
wrote:

If you switch between the regular SQLitePlugin version and the SQLCipher
version, please make sure you completely uninstall the app. I should have
told you earlier.

I think the error message shows that the built-in Android sqlite library
holds onto the lock!

If you did already uninstall the old version, then we have to think
further.

Reply to this email directly or view it on GitHub
#193 (comment)
.

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

I am sorry, when I said there was nothing there anymore, I meant, only the
new order.... Everything that was inserted at the first time is there....
but that is all... everything i insert after that do not keep in database
when i close the app

On Sun, Mar 22, 2015 at 10:33 AM, a Cesar Fernandes cesar.fa@gmail.com
wrote:

I am sorry, a sent msg before finishing it..

.... I just realized now, that onDestroy() method, from java class, is
never called, when i close the app in my device. Do you think there is
any relation?

On Sun, Mar 22, 2015 at 10:31 AM, a Cesar Fernandes cesar.fa@gmail.com
wrote:

I started the project from zero, and added cipher plugin.

my openDatabase call is like this:

db = window.sqlitePlugin.openDatabase({name: "test.db", key: ""});

the app started, and first thigs i do are :

1 - get product list from a webservice - that went ok
2 - get client list from ws - that went ok, but i get a different message
in catlog at this time, for avery insert i make.. i took a screenshot ...
even though the inserts were effective.

[image: Inline image 1]

I go on, register one order, and close the app.. i come back, and i see
everything is there.. i make a new order, leave the app, close it, and
when i come back , there is nothing there anymore...

I just realized now, that onDestroy() method, from

On Sat, Mar 21, 2015 at 9:26 PM, a Cesar Fernandes cesar.fa@gmail.com
wrote:

Well, i really think i did, but i am starting a new project from zero to
make sure....

On Sat, Mar 21, 2015 at 7:15 PM, Chris Brody notifications@github.com
wrote:

If you switch between the regular SQLitePlugin version and the
SQLCipher version, please make sure you completely uninstall the app. I
should have told you earlier.

I think the error message shows that the built-in Android sqlite
library holds onto the lock!

If you did already uninstall the old version, then we have to think
further.

Reply to this email directly or view it on GitHub
#193 (comment)
.

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

Café Hostel
Rua Agissê, 152 - Vl. Madalena
+55 11 2649-7217
http://www.cafehostel.com.br
https://www.facebook.com/cafehostelsp

On Mar 22, 2015 2:40 PM, "César Fernandes de Almeida" <
notifications@github.com> wrote:

I am sorry, when I said there was nothing there anymore, I meant, only the
new order.... Everything that was inserted at the first time is there....
but that is all... everything i insert after that do not keep in database

That sounds really bad!

.... I just realized now, that onDestroy() method, from java class, is
never called, when i close the app in my device. Do you think there is
any relation?

Should not be! SQLite is ACID compliant, which means it should never lose
your data this way.

OTOH it is always good to clean up files and other resources whenever
possible.

I am busy helping with church now, will take a better look tonight or
tomorrow. I will try to make a better test for this issue.

Chris

Tks. I ll try more things today...

I will try to make a more simple APP ... I can even send it to you and
explain better my steps...
Anything that could help...

Cheers
Em 22/03/2015 14:04, "Chris Brody" notifications@github.com escreveu:

On Mar 22, 2015 2:40 PM, "César Fernandes de Almeida" <
notifications@github.com> wrote:

I am sorry, when I said there was nothing there anymore, I meant, only
the
new order.... Everything that was inserted at the first time is there....
but that is all... everything i insert after that do not keep in database

That sounds really bad!

.... I just realized now, that onDestroy() method, from java class, is
never called, when i close the app in my device. Do you think there is
any relation?

Should not be! SQLite is ACID compliant, which means it should never lose
your data this way.

OTOH it is always good to clean up files and other resources whenever
possible.

I am busy helping with church now, will take a better look tonight or
tomorrow. I will try to make a better test for this issue.

Chris

Reply to this email directly or view it on GitHub
#193 (comment)
.

That would really help me.

Does this issue occurs only when you use db.transaction or on db.executeSql too?

Does this issue occurs only when you use db.transaction or on db.executeSql too?

Unfortunately I have not reproduced this issue myself so far. The workaround is only implemented in the case of using db.transaction().

mdbw3 commented

I'm running into this issue also. Does it also occur when opening the database with androidDatabaseImplementation: 2?

As it occurs occasionally, testing / debugging is quite a challenge.

I'm running into this issue also. Does it also occur when opening the database with androidDatabaseImplementation: 2?

As it occurs occasionally, testing / debugging is quite a challenge.

This should only be an issue if you open the database with androidDatabaseImplementation: 2 setting. If you see this issue with the default settings then please raise a new issue along with some details that can help me reproduce the problem.

As documented, there is an androidLockWorkaround that you can use in case you encounter this issue. Please read and follow the documentation very carefully if you need to use this setting.

mdbw3 commented

Ok, thanks. I'll look into it and open a new issue if I can't find a related item or solution.

Edit: the example code at the 'Workaround for Android db locking issue' section is a little bit confusing:
var db = window.sqlitePlugin.openDatabase({name: 'my.db', location: 'default', androidLockWorkaround: 1});

The 'androidDatabaseImplementation: 2 ' setting isn't set, so the example code doesn't make sense, right?

The 'androidDatabaseImplementation: 2 ' setting isn't set, so the example code doesn't make sense, right?

@mdbw3 the README.md has been updated, thanks.

I'm having this problem, but I don't use androidDatabaseImplementation: 2, and the data is lost without closing the app. Specifically, I insert a row and get back the inserted id, then the next insert to that table gets back the same insert id.

One potential cause is that I'm opening the database read-only in some Android plugin code. My understanding was that read-only wouldn't affect a read-write connection, but maybe that's just for Sqlite itself and not for something that wraps it (Android code or this plugin).

What info would be useful in tracking this down? Or is there already a workaround (can I use androidLockWorkaround: 1 without androidDatabaseImplementation: 2)? The docs seem to say that won't help.

Hi @craig-at-rsg,

That really sounds like a different problem. I suggest we discuss this in a new issue. Thanks!

@brodybits Thanks for getting back to me so fast. I just opened #601 for this.

I am writing through a translator. Hello, I will raise this problem again. It occurs when UPDATE records in the database. I would like to use the solution proposed in the documentation in the "Workaround for Android db locking issue" section, but the constructor lacks androidDatabaseProvider and androidLockWorkaround. Are there any other solutions to this problem?
I am sure that the method works without exception and the input data is correct, but in practice nothing happens. After UPDATE it follows: .then, I don’t know what it returns ... I know that catch does not work out. It is strange that this does not happen for all records, and only after the first successful UPDATE record. Other records work fine, but favorites will work until the first change, the second and the next ones no longer work.

Please raise a new issue with some information about what is happening, how you open the database, how you store data, and what kind of devices you see the the issue on. I generally do not support discussions in closed issues.