The memory is never release after call the API saveDocument() of Database to save MutableDocument object
heliangzhang opened this issue · 10 comments
Hi,
I have a application need to save a lot of documents in a short time.
I have write the code follow https://docs.couchbase.com/couchbase-lite/current/swift.html
But the memory is increase continuous until memory warning and the memory never released after all document have saved.
My code is like bellow.
@IBAction func startAdd(_ sender: Any) {
// Get the database (and create it if it doesn’t exist).
let database: Database
do {
database = try Database(name: "mydb")
} catch {
fatalError("Error opening database")
}
DispatchQueue.global().async {
let startTime = Date()
DispatchQueue.main.async {
self.startTimeLab.text = "\(startTime)"
}
for index in 0 ... self.addCount {
// Create a new document (i.e. a record) in the database.
let mutableDoc = MutableDocument()
mutableDoc.setInt(81, forKey: "fromUserOid")
mutableDoc.setInt(0, forKey: "markDelete")
mutableDoc.setInt(2, forKey: "readCount")
mutableDoc.setInt(2, forKey: "retryCount")
mutableDoc.setInt64(1591691473246, forKey: "sendingTimestamp")
mutableDoc.setInt64(1591772518435, forKey: "sortId")
mutableDoc.setInt(2, forKey: "status")
mutableDoc.setInt64(1591691473246, forKey: "timestamp")
mutableDoc.setInt(31465, forKey: "toUserOid")
mutableDoc.setInt(1, forKey: "type")
mutableDoc.setInt64(1591772518435, forKey: "updateTime")
mutableDoc.setString("aa6ad9yrqiaa6ad9yrqiaa6ad9yrqi", forKey: "fromJid")
mutableDoc.setString(self.image, forKey: "image")
mutableDoc.setString("aa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqi", forKey: "imageCacheUri")
mutableDoc.setString("aa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqi", forKey: "message")
mutableDoc.setString("fffB1B2555B-2672-46B3-869D-7FE00D5961CC", forKey: "messageId")
mutableDoc.setString(self.notifiJson, forKey: "notifyJson")
mutableDoc.setString(self.stanza, forKey: "stanza")
mutableDoc.setString("aawnmft2", forKey: "tid")
mutableDoc.setBoolean(false, forKey: "isOffline")
// Save it to the database.
do {
try database.saveDocument(mutableDoc)
} catch {
fatalError("Error saving document")
}
DispatchQueue.main.async {
self.currentCountLab.text = "\(index)"
}
}
let endTime = Date()
DispatchQueue.main.async {
self.endTimeLab.text = "\(endTime)"
self.totalTime.text = "\(endTime.timeIntervalSince(startTime))"
}
// Create a query to fetch documents of type SDK.
let query = QueryBuilder
.select(SelectResult.all())
.from(DataSource.database(database))
.where(Expression.property("type").equalTo(Expression.string("SDK")))
// Run the query.
do {
let result = try query.execute()
print("Number of rows :: \(result.allResults().count)")
} catch {
fatalError("Error running the query")
}
}
}
The memory allocation is like this:
Is there any problem when I used the framework.
Waiting for your reply ,Thank you ~
My Xcode version is 11.5.
Target version is iOS 13.0
I used the couchBaseLite version is Couchbase-lite-swift-enterprise-2.7.1(Down load from https://packages.couchbase.com/releases/couchbase-lite-ios/2.7.1/couchbase-lite-swift_enterprise_2.7.1.zip )
The follow attach is my demo without Couchbase Lite framework for it is too big .
CouchbaseLiteSwiftDemo.zip
When processing a lot of data/objects in the loop, you could add @autoreleasepool {} block inside the for-loop to drain the autoreleased objects before they get piling up and cause the memory issue.
Some info about @heliangzhang autoreleasepool:
If @autoreleasepool doesn't help, please reopen the issue.
Hi, @pasin
Thanks for your reply ~
I have add the autoreleasepool out of for loop.
But it does not work for me.The memory is increase continuous.
If I move the create database and save document operation in the same block .The memory can be release when save finish.
But the maximum memory is too high and cause crash for memory warning when I migrate a lot of record from CoreData .
My code like this
@IBAction func startAdd(_ sender: Any) {
DispatchQueue.global().async {
// Get the database (and create it if it doesn’t exist).
let database: Database
do {
database = try Database(name: "mydb")
} catch {
fatalError("Error opening database")
}
let startTime = Date()
DispatchQueue.main.async {
self.startTimeLab.text = "\(startTime)"
}
autoreleasepool {
for index in 0 ... self.addCount {
// Create a new document (i.e. a record) in the database.
let mutableDoc = MutableDocument()
mutableDoc.setInt(81, forKey: "fromUserOid")
mutableDoc.setInt(0, forKey: "markDelete")
mutableDoc.setInt(2, forKey: "readCount")
mutableDoc.setInt(2, forKey: "retryCount")
mutableDoc.setInt64(1591691473246, forKey: "sendingTimestamp")
mutableDoc.setInt64(1591772518435, forKey: "sortId")
mutableDoc.setInt(2, forKey: "status")
mutableDoc.setInt64(1591691473246, forKey: "timestamp")
mutableDoc.setInt(31465, forKey: "toUserOid")
mutableDoc.setInt(1, forKey: "type")
mutableDoc.setInt64(1591772518435, forKey: "updateTime")
mutableDoc.setString("aa6ad9yrqiaa6ad9yrqiaa6ad9yrqi", forKey: "fromJid")
mutableDoc.setString(self.image, forKey: "image")
mutableDoc.setString("aa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqi", forKey: "imageCacheUri")
mutableDoc.setString("aa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqiaa6ad9yrqi", forKey: "message")
mutableDoc.setString("fffB1B2555B-2672-46B3-869D-7FE00D5961CC", forKey: "messageId")
mutableDoc.setString(self.notifiJson, forKey: "notifyJson")
mutableDoc.setString(self.stanza, forKey: "stanza")
mutableDoc.setString("aawnmft2", forKey: "tid")
mutableDoc.setBoolean(false, forKey: "isOffline")
// Save it to the database.
do {
try database.saveDocument(mutableDoc)
} catch {
fatalError("Error saving document")
}
DispatchQueue.main.async {
self.currentCountLab.text = "\(index)"
}
}
}
let endTime = Date()
DispatchQueue.main.async {
self.endTimeLab.text = "\(endTime)"
self.totalTime.text = "\(endTime.timeIntervalSince(startTime))"
}
// Create a query to fetch documents of type SDK.
let query = QueryBuilder
.select(SelectResult.all())
.from(DataSource.database(database))
.where(Expression.property("type").equalTo(Expression.string("SDK")))
// Run the query.
do {
let result = try query.execute()
print("Number of rows :: \(result.allResults().count)")
} catch {
fatalError("Error running the query")
}
}
}
You need to put the autoreleasepool block in the for-loop otherwise the objects will be released after the for-loop is done.
for index in 0 ... self.addCount {
autoreleasepool {
...
}
}
You can also improve the performance by processing docs in batch so that the objects will be released in batch instead of in every iteration.
@pasin
Hi, Pasin
It worked now.Thank you very much ~~
@pasin
Hi,Pasin
It was work for me with the CouchbaseLiteSwift.framework I build with master branch.
It was not work for the couchbase-lite-swift_enterprise_2.7.1 framework download from https://packages.couchbase.com/releases/couchbase-lite-ios/2.7.1/couchbase-lite-swift_enterprise_2.7.1.zip
Thank you very much.
@jayahariv Can you check the @heliangzhang's last comment? I don't remember that we have bugs or fixes anything in that area.
@heliangzhang Could you please try using 2.8.1? looks like the big increase in memory is not happening with 2.8.1. I have tried the sample project you attached, 2.7.1 will increase the memory usage to ~573MBs, whereas 2.8.1 is only increasing to ~22MBs and staying more stable(~2MBs increase) from there on.
If you are an enterprise customer and not able to update to 2.8.x, please open a support ticket for this issue. We can look further into the issue.
