From personal computers to embedded devices, storing data for later is an essential part of most modern system. When storing that data, most often a filesystem (FS) is must-have for organizing that data to help find what you are looking for later.
Usually the filesystem is provided by the operating system, so applications don’t have to worry about it. Some examples of this are the Linux filesystems ext4, or the windows filesystems NTFS. But maybe you are building an operating system? Or running a device without one? Then you have a choice:
If you have the choice, you may choose not to build your own filesystem for multiple reasons:
Despite these advantages of a COTS FS, there are other circumstances where building your own may make more sense:
If you must build your own filesystem, here are some things to consider.
Your system may not need all the features common in most filesystems. Things you might not need include:
Limiting your filesystem to what you do need will reduce the complexity of your filesystem design, and therefore reduce development time and potentially improve performance.
One place to start when looking at features and interfaces you need is the POSIX standard, as many filesystems are POSIX compliant.
Users of a filesystem mainly just think of a file as the data it contains. But additional data about the file must be stored. The size, name and location of the file are usually crucial data to store. Information about what space has been used and what is available for use is also a common piece of metadata.
When designing your metadata, there is choice of how much metadata you want. Storing more metadata may make it faster to parse/access the data at the cost of less disk space for actual data and more time spent updating the metadata.
It would be convenient to keep files entirely contiguous, but often not possible. This leads to file fragmentation.
File fragmentation decreases performance as it prevents consecutive reads and adds additional metadata that must be stored for the file. Your design will therefore want to reduce fragmentation when possible.
Filesystems must be suited to the hardware they are running on.
Especially in embedded, flash storage is quite common but presents additional complexity. For example, in most flash devices, after writing data to a memory page, new data cannot be written again to that page until a larger memory block is erased. Due to this mechanism, filesystems must often write the data elsewhere instead of rewriting data to the same location.
Additionally, characteristics of the hardware such as the page size in a flash device should be considered. How well the filesystem is aligned with the hardware can have a large impact on the performance of the filesystem.
A robust filesystem takes time and resources to build. If you haven’t budgeted for a filesystem while facing an aggressive schedule, you could also find yourself up against a wall. In contrast, consider your filesystem early on in the development process and you’ll reach your goals with greater confidence.
DornerWorks has years of experience building complex embedded systems that require every bit of space to be used efficiently. Whether it’s a finicky filesystem, or a completely new device built from the ground up, we can guide your next project to success. Contact us today and set up a free consultation.
