Skip to main content Roast my custom file system design : r/osdev

Log In

Go to osdev
r/osdev • 10 mo. ago
JakeStBu

Roast my custom file system design

I've been working on a custom file system, SpecFS, for SpecOS, after looking at how other file
systems work. I've been refining this for a couple of days and I'm honestly pretty happy with it.
Please have a look at my fs design and tell me what's wrong with it that I missed on (it's designed
right now only for 28 bit LBA):

 No boot sector data (information is largely assumed. I'm not really trying to make this
cross-compatible with anything)
 First 1,000 sectors are reserved for kernel image and the sector map, explained later (this
may be increased if needed)
 Two types of sectors (besides reserved) which share the data section:
 Directory sector
 File data sector
 The last 28 bits of each sector is reserved for pointing to the next sector of the directory
or file
 If it's the end of the file/directory, the last 28 bits should be the NULL byte (0x00).
 If it's not the end of the file/directory, the whole thing can be used (except for the last
byte, which must be 0x10)
 The first 28 bits of each folder sector is an LBA which points to the folder's parent
directory. If it is root, then this should point to itself.

Directory sector - entry data:

 File name (13 bytes, shared between file name and extension)
 File attributes (1 byte: read only = 0x01, hidden = 0x02, system = 0x03)
 Type (f or d, depending on if it's a directory or file. 1 byte.)
 File name length (1 byte. More about long file entries soon.)
 Time created (5 bit hour, 6 bit minute, 5 bit seconds - 2 bytes total, double seconds)
 Date created (7 bit year, 4 bit month, 5 bit day - 2 bytes total)
 Time last edited (same format as time created, 2 bytes total)
 Date last edited (same format as date created, 2 bytes total)
 LBA of first sector of this entry (28 bits = 4 bytes)
 File size in sectors (always 0x00 for folders, 4 bytes)

= 32 bytes
Sector map:

The sector takes up the first 900 sectors, but the next 100 of reserved space are used for the
sector map. This is basically a bitmap of every sector in the data section.

This is used when files are created or expanded so that the kernel knows where a sector is
avaliable to write to.

Long file entries:

If a file name is longer than the allocated 13 bytes (the length is stored in the main entry), then
add another entry after the main one containing it's full file name, of the length allocated by the
main entry. This does not include the first 13 characters, which are obviously defined by the
main entry.

Limits:

 Partition can be maximum 2 ^ 28 sectors (assuming 512 byte sector size, that's
approximately 137.4 GB. The reserved space for the next sector pointer can be changed
for lower efficiency, but higher disk size support). This is because the file system is built
for a disk driver using 28 bit LBA. This can be modified to a 48 bit LBA support, which
would allow for 2 ^ 48 sectors (assuming 512 byte sector size again, that's about 550
gigabytes).
 Basically nothing else. Files can be any size, and folders can be any size, obviously up to
partition size.

I'd love to know your thoughts on this. Thanks!

Sort by:
someidiot332
• 10mo ago

why would someone choose to use this over a simpler and easier filesystem like FAT? are you
limited to only 512 byte sectors? how will you support special file types like pipes and links?

JakeStBu
• 10mo ago

This was mostly because I wanted an extra challenge to create my own file system, it's not really
meant for others to use. I'm not limited to 512 byte sectors, but I really wanted to try to make this
file system as simple and easy to implement as possible. Links will literally just be normal files
with the contents being the path of what it points to, nothing fancy.
Octocontrabass
• 10mo ago

The last 28 bits of each sector is reserved for pointing to the next sector of the directory or file

Awful. You've made it impossible to do any decent caching on top of this filesystem.

File name (13 bytes, shared between file name and extension)

How are these file names encoded?

File size in sectors

What happens when a file doesn't fill a whole number of sectors?

100 of reserved space are used for the sector map. This is basically a bitmap of every sector in
the data section.

With 512-byte sectors, your bitmap has enough bits for 200MiB.

the file system is built for a disk driver using 28 bit LBA.

NVMe has 64-bit LBA. (USB does too, kinda.)

I'd love to know your thoughts on this.

I think it's worse than Macintosh File System.

JakeStBu
• 10mo ago

Awful. You've made it impossible to do any decent caching on top of this filesystem.
Yeah, that's still kinda an issue. It's meant to be quite basic right now and easy to implement.

How are these file names encoded?

Just ASCII.

What happens when a file doesn't fill a whole number of sectors?

Maybe a little inefficient. I'll work on this.

With 512-byte sectors, your bitmap has enough bits for 200MiB.

Yeah, I noticed this. Wondering how I could fix this. Perhaps bigger clusters are needed than 512
byte sectors...

NVMe has 64-bit LBA. (USB does too, kinda.)

This is built for my kernel's ATA PIO mode driver which supports 28 bit LBA which I find to be
easier.

I think it's worse than Macintosh File System

I'll have to have a look into that - it needs some refinement but I would like to give a version of
this a shot, since I want to not use an existing fs.

BGBTech
• 10mo ago

Agreed, it does not seem like a good idea, has no real obvious advantage (and some notable
drawbacks) if compared with FAT.
Admittedly, I had on/off also been considering a possible filesystem design (as a possible
alternative to FAT, which I am using now), but more because both EXTn and NTFS displease
me.

Though, my considered design would kinda resemble a hybrid of EXT2 and NTFS:

 Inode table is itself an Inode, supporting extending the table as needed;

 Inodes are built from multiple tagged structures (like the MFT in NTFS);
 Encodes references to blocks in a similar way to EXT2;
 Directory entries are fixed length and organized into an AVL tree.

The idea in my case was to have 64-byte directory entries with a 48 byte name field, left/right
sub-nodes, and an inode number, etc.

 48 bytes is enough to cover many filenames directly.

 AVL tree allows semi-efficient lookup and listing names in sorted order.
 AVL trees are less overkill than B+Trees.
 They seem less annoying than the scheme used by EXT2.
 The first dirent points to the root of the AVL tree.

I wanted to avoid needless complexity, which seemed particularly rampant in NTFS. A 48-byte
name does still mean it needs multi-part entries to deal with longer names, but probably still less
annoying than fully variable-length directory entries (like in EXT2), and less complicated than a
B+Tree or similar (though, rebalancing when inserting into an AVL tree is a little awkward).

Most information about a file will be held in its Inode. It may support file compression but
specifics here are still TBD.

darkslide3000
• 10mo ago

You have basically reinvented FAT but worse. Chaining sectors like a linked list is the most
basic and most inefficient way to map a file onto sectors, serious file systems nowadays have
mechanisms that allow allocating larger contiguous ranges of sectors where possible and
generally try to avoid making the random access lookup time O(n) with a large n.

But you managed to do even worse because at least for FAT the entire list is out of band in a
relatively small number of sectors that fit well in the cache... in your case, you're actually forcing
the system to read every sector in order to find the next one, not just the metadata. Sectors can
only be read in full from a disk, so in essence you've designed a system where it is impossible to
open a file and read the last byte without reading every single byte before it. That's terrible for
many common use cases. (And if your file system driver doesn't spend a lot of effort and
memory on caching sector lists, you're gonna keep paying that price again and again for every
seek.)
Another really bad consequence of your design is that the actual amount of bytes usable for the
file data per sector is 4 bytes smaller than the sector size (you said the last 28 bits are reserved
but I assume you meant that actually 32 bit are reserved and only 28 of those are the next sector
number, because otherwise you'd be splitting a single logical byte across two sectors which is
even more insane). Many programs know and make use of the fact that file systems group things
along power-of-two alignment boundaries, and arrange their data such that the stuff that needs to
be read/written together is aligned to those boundaries. By taking 4 bytes away from every sector
you shift all the rest of the data around so that that doesn't work anymore, and write operations
that the application intended (for performance) to only overwrite a single sector will end up
overwriting two in your system.

JakeStBu
• 10mo ago

Yeah... Perhaps I'll just use exFAT

eteran
• 10mo ago

My main thought is that if I've understood it correctly, your directories and files are basically a
linked list of sectors.whixh will make random access wildly inefficient.

I think you'd be better off creating a structure similar to paging where "blocks" which are some
number of contiguous sectors represent an array of block pointers.

Let's say a block is 4K, that means even with a basic setup, you can support files or directories
up to 4GB in size.

This way you get random access very efficiently.

austroalex
• 10mo ago

I skimmed over this for about 69 microseconds and I think you have managed to make another
FAT clone but somehow worse.

Rating FAT/10, would not mkfs.specfs again

New to Reddit?

Create your account and connect with a world of communities.

Continue With Phone Number

By continuing, you agree to our User Agreement and acknowledge that you understand the
Privacy Policy.
More posts you may like


r/programminghumor

• 9 mo. ago

This is what developer says when their code doesn't work ..

237 upvotes · 16 comments

r/xkcd

• 2 mo. ago

Can someone implement this in a web extension? 1288

 1.3K upvotes · 46 comments

r/dwarffortress

• 1 yr. ago

Dwarf Therapist - Update Memory Layout

36 upvotes · 7 comments

r/arch

• 3 mo. ago

Funny error message

 60 upvotes · 10 comments

r/programminghumor

• 10 mo. ago

HTML

314 upvotes · 17 comments


r/DHExchange
• 2 mo. ago

Archive of abandoned large-system software?

24 upvotes · 8 comments

r/dji
• 1 yr. ago

DJI bad file system format

4 upvotes · 1 comment


r/armadev
• 1 yr. ago

uiNamespace setVariable for custom HUD elements

2 upvotes · 1 comment

r/Python
• 1 yr. ago

ConfigClass - simple dataclass inspired configuration

23 upvotes · 5 comments


r/mcresourcepack
• 1 yr. ago

Texture not rendering on custom block model

1 upvote · 6 comments

r/Progressbar95
• 2 mo. ago

Recently discovered the file extension for TXT files doesn't actually need to
match (just needs to be three characters)

31 upvotes · 7 comments

r/arch
• 3 mo. ago

This is driving me nuts, my webbrowser does not respect my custom cursor. How
do I fix this?

 0:07
34 upvotes · 17 comments

r/EnterTheGungeon
• 4 mo. ago

So the game is easier on new files?

200 upvotes · 21 comments

r/programminghumor
• 5 mo. ago

databases

435 upvotes · 18 comments

r/PBSOD
• 1 mo. ago

Depressed POS terminal

180 upvotes · 29 comments

r/YoungPeopleShorts
• 2 mo. ago

Your pc ran into a problem

35 upvotes · 23 comments

r/FrutigerAero
• 2 mo. ago

This is how my webpage it's going!

 2
202 upvotes · 39 comments

r/RPGMaker
• 1 mo. ago

Finally I got all UI draggable! Building UI won't be a chore anymore

 0:24
427 upvotes · 32 comments

r/ZanClan
• 3 mo. ago

I don’t know how to code

442 upvotes · 13 comments

r/CivVII
• 2 mo. ago

Edge scrolling

9 upvotes · 6 comments


r/StableDiffusion
• 2 mo. ago

Open source is the way

 0:23
201 upvotes · 6 comments

r/comfyui
• 1 yr. ago

XY Plot - Saving checkpoint name in filename or metada

6 upvotes · 1 comment

r/DistroTube
• 1 yr. ago

Topic suggestion: keyboard customization by software

2 upvotes

r/idm
• 4 mo. ago
DS Glitch

 0:51
24 upvotes · 12 comments

r/Titanfall_2_
• 3 mo. ago

Everything is unlocked for no reason??? GLITCH (UPDATE)

 4

174 upvotes · 8 comments

Top 4% Rank by size

Public

Top Posts



Reddit

reReddit: Top posts of June 7, 2024


Reddit

reReddit: Top posts of June 2024

Reddit

reReddit: Top posts of 2024

Reddit Rules Privacy Policy User Agreement Reddit, Inc. © 2025. All rights reserved.