It’s interesting to see how databases have come a long way and have clearly out-shadowed file-systems for storing structured or unstructured information.Technically, both of them support the basic features necessary for data access.
For example, both of them:
- Ensure data is managed to ensure its integrity and quality
- Allow shared access by a community of users
- Use well-defined schema for data-access
- Support a query language
Download Executive Brief: Choosing the Right Model for Enterprise Backup & Recovery
But file systems seriously lack some of the critical features necessary for managing data. Let’s take a look at some of these features.
Transaction support
Atomic transactions guarantee complete failure or success of an operation. This is especially needed when there is concurrent access to the same data set. This is one of the basic features provided by all databases.
But most file-systems don’t have this feature. Only the lesser known file systems, such as Transactional NTFS(TxF), Sun ZFS, and Veritas VxFS, support this feature. Most of the popular open source file systems (including ext3, xfs, reiserfs) are not even POSIX compliant.
Fast Indexing
Databases allow indexing based on any attribute or data property (i.e. SQL columns). This helps fast retrieval of data, based on the indexed attribute. This functionality is not offered by most file systems, i.e. you can’t quickly access “all files created after 2 PM today.”
The desktop search tools like Google desktop or MAC spotlight offer this functionality. But for this, they have to scan and index the complete file system and store the information in an internal relational database.
Snapshots
Snapshot is a point-in-time copy/view of the data. Snapshots are needed for backup applications, which need consistent point-in-time copies of data.
The transactional and journaling capabilities enable most of the databases to offer snapshots without shopping access to the data. Most file systems however, don’t provide this feature (ZFS and VxFS being only exceptions). The backup software has to either depend on the running application or underlying storage for snapshots.
Clustering
Advanced databases like MySQL also offer clustering capabilities. MySQL offers shared-nothing clusters using synchronous replication. This helps the databases scale up and support larger & more fault tolerant production environments.
File systems still don’t support this option. The only exceptions are Veritas CFS and GFS (Open Source).
Replication
Replication is a commodity with databases and forms the basis for disaster-recovery plans. File-systems still have to evolve to handle it.
Relational View of Data
File systems store files and other objects only as a stream of bytes, and have little or no information about the data stored in the files. Such file systems also provide only a single way of organizing the files, namely via directories and file names. The associated attributes are also limited in number, e.g. type, size, author, creation time, etc. This does not help in managing related data, as disparate items do not have any relationships defined.
Databases, on the other hand, offer easy means to relate stored data. It also offers a flexible query language (SQL) to retrieve the data. For example, it is possible to query a database for “contacts of all persons who live in Acapulco and sent emails yesterday”, but impossible in the case of a file system.
File systems need to evolve and provide capabilities to relate different data sets. This will help the application writers to make use of native file system capabilities to relate data. A good effort in this direction has been Microsoft WinFS.
Conclusion
The only disadvantage with using the databases as the primary storage option, seems to be the additional cost associated. But, I see no reason why file systems in the future will not borrow features from databases.
Disclosure
Druva inSync uses a proprietary file system to store and index the backed up data. The meta-data for the file system is stored in an embedded MySQL database. The database-driven model was chosen to store additional identifiers with each block – size, hash and time. This helps the file system with:
- Block Size: Divide files into variable sized blocks
- Data deduplication: Store single copy of duplicate blocks
- Temporal File System: Store time information with each block. This enables faster time-based restores.
Find out more about to pros and cons of different architecture models by accessing this report: Choosing the Right Model for Enterprise Backup & Recovery