Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

"Near-realtime" isn't good enough for SQLite. It has a strong dependency on the consistency guarantees provided by a POSIX-like filesystem.

The approach described in this page does make some attempt to provide consistency. The master generates a single stream of file updates, and all of the replicas consistently apply those changes in the same order. But rsync is not guaranteed to observe changes to files on the master in the same order that they were written.

In particular, SQLite (by default) uses a rollback journal to recover from failures. While a transaction is in process and some parts of the database might have been partially updated, the journal stores the old contents of the modified pages. Since the old data is fsync'ed to the journal before the new data is written to the database file, the data on disk at any moment in time is recoverable.

But there's no clean way for rsync to read an atomic snapshot of both the database file and the journal at the same instant in time. If it reads them at different times, a database change might be partially applied or partially rolled back, which has a high likelihood of corrupting the database.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: