Support for concurrent read/ writes

[Strengthless]

Hi thank you for the great work!

I'm fairly new to DBF files, and unfortunately I can't find much documentation online about DBF specifications.

Does this package (or DBF files themselves) come in support for concurrent read/ writes?

For context, I'm currently working on an API integration for a library system where they use DBF files on a network drive as their database. When reading the DBF files, will it cause issues to the existing terminals (responsible for checking in and out books)?

Thank you once again.

[yortus]

Does this package (or DBF files themselves) come in support for concurrent read/ writes?

This package does not provide any protections that would be needed in concurrent environments. It was originally designed just to bulk-extract data from old DBF files into something more modern. It has evolved from there to support more field types and operations, but nothing has been added to support concurrent reads/writes.

I'm not planning to add such support, but PRs from the community are always considered.

Is there any way the library system could migrate to a more modern database? That's the approach I'd recommend. Otherwise perhaps there's a way to serialise all reads/writes to the DBF files, which should guard against concurrency issues if done correctly.

[lordrip]

If I understood correctly, the way concurrency works with DBF files, at least with Microsoft Visual FoxPro, is by setting flags in the headers (mark the entire file as exclusive) or at the beginning of each row (row lock).

So if a DBF file consumer wants exclusive access, it should attempt to open the file and enable a flag signaling to other users that they can't write yet. Likewise, when updating a row, a flag should be set at the row level and once that's done, remove it.

I totally get your point @yortus, implementing this is not trivial 😄, as it would probably mean that we would need to write into the files differently, so instead of dumping a file, the library should allow updating individual records.

@Strengthless, maybe there are alternatives that can help here, for instance, there's the Apache Camel project with the purpose of interconnecting systems a different levels.

[Strengthless]

Thank you @lordrip and @yortus for the valuable insights!

Does any of you happen to know the behaviour of DBF files when they're being read? To be precise, when the files are being read, will they be put into lock preventing further read/ write actions?

For more context, there are multiple instances of check-in and check-out terminals around the library, and they seem to handle concurrent read/ writes without a problem (though the system is pretty legacy and closed source, which means I cannot determine the exact mechanism behind). The API system I'm integrating will only invoke reading without writing, therefore it shouldn't cause any problem to the existing terminals as long as the files aren't put into lock mode when reading, I suppose? 🤔

[lordrip]

It will mostly depend on how are the files being read or in which mode, for that matter.

For instance, using Microsoft Visual FoxPro, a SELECT clause will copy the records into a dedicated cursor, leaving the file opened in shared mode, without affecting other users. That being said, there's another mode that revolves around navigating in the file, record by record, and in this case, the engine can be set to automatically lock the record in use.

I think for reading-only purposes, there shouldn't be any problem with concurrent users.