LMDB is throwing an EPERM error when trying to create the database.
Closed this issue · 8 comments
Looks promising, but I'm curious if you've managed to get this working on macOS. Specifically, in a sandboxed applications using a standard Xcode project.
When I try to create the Database within the initial Transaction, I get:
Error: other(returnCode: 1)
This maps to the EPERM that LMDB throws.
This is the same problem that is present in another Swift wrapper around LLMD when attempting to use it on macOS. It's likely related to how LMDB uses Semaphores. There was some discussion awhile back here:
If you're able to get this working on macOS, are you using any additional build flags?
In order to reproduce this, just create a new Xcode project for macOS, add the QuickLMDB package, then try and open a database from within a transaction. It will likely throw an exception and the error will be EPERM.
Hey thanks for bringing this up and referencing the discussion in the SwiftLMDB project.
My years of experience in Swift and LMDB is mostly based on Linux, the platform where this framework was forged and tested.
I recently began working on a new iOS and MacOS project (my first frontend project in years) - I hit this exact issue when I "broke ground" about a month ago. I was not able to get it working with a native MacOS (sandboxed) build, but I have since (for many reasons) reduced my deployment target to just iOS, and its working great (as it does in non-sandbox MacOS).
As your reference to issue 23 in SwiftLMDB mentions - its super weird that Apple (supposedly) allows use of these semaphores on the iOS sandbox, but not the MacOS sandbox?
I'll try submitting a bug report to Symas and seeing if this is something they are willing to look at.
@kennycarruthers if you are using QuickLMDB in a sandboxed setup, its safe to assume that you are not concerned with multi-process access on the database. You may be able to hack together a compromise if it makes sense for your needs.
If you are willing to give up the 1-writer, n-readers relationship that LMDB offers with its transaction engine, you can initialize an Environment with Environment.Flags.noLock, which should (theoretically) stop the semaphores from being used.
In a setup like this, you can use a Swift actor as the locking & access control mechanism for the transactions. You would be forced to only one run transaction at a time (since Swift actors are serialized) for your entire process across all threads - however, LMDB is so damn fast, I could imagine a lot of medium to low-end data needs where this could be adequate.
Thankfully I didn't have a gun to my head to get a MacOS build working for my current frontend project - however - if I found myself in this situation, I would probably try the idea I've suggested here.
Environment.Flags.noLock seems to be the only way right now.
Thanks for the follow-up. Running LLMD in noLock mode wasn't something I explored at the time but might be a viable workaround. My use-case at the time was mostly read-heavy and involved doing a lot of "full table scans", so I was interesting in seeing if I could use multiple cursors to parallelize that in some way.
You're welcome to close this issue because, like with the other projects, it's more of an LMDB issue than anything else. I was just curious if you had found a working build configuration or not. Thanks again for sharing this project.
Appreciate that! I'll leave this ticket open (for now) since sandbox deployments on MacOS is nothing radical in Swift land - this is probably something people should know about until I can figure out where Symas stands on this.
As an aside:
If concurrent reads is your primary need, you can definitely compromise towards that with .noLock as well - so long as you have confidence in your ability to control when the writes are happening. It sounds unsafe but it doesn't necessarily need to be - you can use Environment.Flags.noLock and Environment.Flags.readOnly together to guarantee complete safety in concurrent reading if you can manage writes through other means. Far from ideal but certainly possible.
If I hear anything noteworthy from Symas I'll give you a ping.
Good luck!
Please provide a simple test case to reproduce this problem. Note that I don't usually spend any time with MacOS dev tools, so you will need to be explicit in every step otherwise I won't be able to investigate.
After seeing some additional feedback from @hyc in issue 23 of SwiftLMDB and in the ticket, I am thinking this one will get closed out and I'll need to adjust my support claims in the README to note this exception. QuickLMDB is meant to be a rather pure interpretation of a wrapper and this issue gets kinda messy.
That being said, it is very easy to swap a different (or modified) version of LMDB into QuickLMDB if someone is so inclined to make these changes.
As an aside,
I've entertained building an additional library on top of QuickLMDB that integrates natively with Swift's concurrency model (in place of the locking mechanisms), but never justified building it for many obvious reasons.
This might be an interesting avenue to explore in the future, but unfortunately, cant be on my immediate roadmap at this time.
Thanks for following up on this and exploring what the options where.