Disk caching in the year 2020

In the previous blog post I went over the design of storage systems and how they protect themselves from data loss in the face of disk failure or unclean shutdowns. If you’re putting together the hardware yourself modern Linux systems and modern hardware give you endless cost, performance and durability tradeoffs for whatever system you want. But 2020 also gives us the cloud, and storage systems in the cloud whose durability far exceeds whatever you could build yourself.

Durability in the year 2020

It is the year 2020 and we still don’t have great answers to data durability in the face of unclean shutdowns. Unclean shutdowns are things like power outages, system faults and unlucky kernel panics and preventing data loss when they happen is a hard problem. I’m going to talk through a few ways these can cause data loss in this blog post but you can probably come up with new ones on your own - exotic failures that no system will ever handle correctly. In the year 2020, the conventional wisdom is still true. Always take backups, and RAID is no substitute for backups.