Outline

n Wrap up discussion on permissions:

Access control lists and capabilities
File System Performance
n

n File system performance:

n Why is the original Unix file-system slow?
What changes could be made?
Arvind Krishnamurthy
n

n What did the BSD 4.2 folks do?

Spring 2001
n File system consistency issues in the presence of failures:
n Transactions

Access control lists vs. capability

lists A combined approach: Unix
n Access control lists (ACL): keep lists of access for each domain with n Intermediate domain: groups
each object: n Users belong to groups
File3: User A: rwx
Group B: rw n Access control list is small:
………… n One user
n Capability lists (CAP): keep lists of access rights for each object with n One “group”
each domain
User A: File1: rw n Change permissions on an object by:
File2: r n Changing the user who can access
…………
n Changing the group which can access
n Which is better?
n ACLs allow easy changing of an object’s permissions
n Change user privileges by:
n Capability lists allow easy changing of a domain’s permissions
n Changing the user’s group
n Which is faster?
n Which is easier to revoke?
n Performance: fast and reasonably flexible

Disk organization Disk performance

n Disk surface
n Circular disk coated with n Seek
magnetic material n Position heads over cylinder, typically 5.3 − 8 ms
n Tracks n Rotational delay
n Concentric rings around disk
surface, bits laid out serially n Wait for a sector to rotate underneath the heads
along each track n Typically 8.3 − 6.0 ms (7,200 – 10,000RPM) or ½ rotation takes
n Sectors sector 4.15-3ms
n Each track is split into arc of n Transfer bytes
track (min unit of transfer) n Average transfer bandwidth (15-37 Mbytes/sec)
n magnetic disks come n Performance of transfer 1 Kbytes (or 512 bytes)?
organized in a disk pack
n What is the effective bandwidth?
n Disk arm: seek to the right
cylinder
seek a cylinder

1
 Unix lookup: /a/b/c Unix file system is slow!
“.” Name space Physical organization
a disk
n What are the problems?
“..” 2
b 3 Inode table
“.” 4
5 How many disk I/O’s to n How can we fix them?
... access first byte of c?
<a,3>
c File system for BSD 4.2 (Fast File System or FFS)
Blocks are 512 bytes n

<c, 14> n Kirk McKusick, Bill Joy, Samuel Leffler, Robert Fabry

<b,5> How much does it cost to

access the next block?

Block Size Question:

n Use a bigger block size:
n 4k or 8k instead of 512 bytes

n Just big blocks? n Why don’t we have 1MB block sizes and 1KB fragment
n Most files are small sizes?
n Experiments: 4K block size resulted in 50% waste

n Introduce smaller “fragments” (0.5 to 1k)

n File size < block size: a number of contiguous fragments
n File size > block size: a number of blocks plus a number of contiguous
fragments

n Good bandwidth for large files, pretty good disk utilization for small
files

BSD 4.2 Improvements Cylinder Groups

n Increase block sizes è improve disk bandwidth n Each cylinder group contains:
Inodes, indirect blocks, data blocks
Increase locality è improve disk bandwidth
n
n
n Seek within a cylinder group is small (usually a few tracks)
n Allocation of “related” info within a physical region
n Locality: original Unix system used free lists: n Locality:
n Initially everything is allocated contiguously n Inodes close to data blocks
n However, free list gets jumbled up very fast n Data blocks close to each other
How to get locality?
Locality: original Unix allocated I-nodes at the beginning of
n
n

the disk n Keep spare capacity

n Inodes are not allocated close to data n Lie if necessary (limit: 90% fullness)
n Improvement: allocate Inodes in the middle n Trade-off between capacity and performance
n Even better: use notion of “cylinder groups” n If you want to keep some things together, you gotta spread other
things apart!

2
 Near and Far File System Consistency
n Keep a directory’s contents within a cylinder group n File systems have lots of data structures
n Spread out sub-directories n Bitmap for free blocks
n Try to allocate file blocks in the same cylinder groups n Directory
n Spread out “medium” to “big” files n File header
n First 50K within the same cylinder group n Indirect blocks
n And switch cylinder groups every 1MB n Data blocks

n Rotationally optimal local allocation:

n Skip sectors n For performance, all must be cached!
n Rationale for 90% fullness: how hard it is to find a rotationally good
spot This is OK for reads but what about writes ?
n Search order: rotationally closest in current cylinder, current
cylinder group, hash to another cylinder group, exhaustive search
n Current disks: track buffers, fewer platters

Modified data in memory Multiple updates

n Options for writing data: n If multiple updates needed to perform some operations,
crash can occur between them!
Write-through: write change immediately to disk n Moving a file between directories:
n Delete file from old directory

Problem: slow! Have to wait for write to complete n Add file to new directory

before you go on
n Create new file
n Allocate space on disk for header, data
Write-back: delay writing modified data back to disk (for
example, until replaced) n Write new header to disk

n Add the new file to directory

Problem: can lose data on a crash!

What if there is a crash in the middle? Even with write-through it can
still have problems

Unix approach (ad-hoc) Meta-data consistency

n For meta-data, Unix uses “synchronous write through”.
n Meta-data: synchronous writes, data: asynchronous writes n If multiple updates needed, Unix does them in specific order
n If it crashes, run the special program “fsck” which scans the entire
disk for internal consistency to check for “in progress” operations
n Meta-data: needed to keep file system logically consistent and then fixes up anything in progress.
n Directories
n Bitmap Example:
n File headers
n Indirect blocks In what order do you do file creation?
n ……
In what order do you do block allocation?
n Data: user bytes

3
 User data consistency Transaction concept
n For user data, Unix uses “write back” --- forced to disk every 30 n Transactions: group actions together so that they are
seconds (or user can call “sync” to force to disk immediately). n Atomic: either happens or it does not (no partial operations)
n Serializable: transactions appear to happen one after the other
No guarantee blocks are written to disk in any order. n Durable: once it happens, stays happened
Sometimes meta-data consistency is good enough
Critical sections are atomic and serializable, but not durable
How should vi save changes to a file to disk ?
Need two more items:
Write new version in temp file
Commit --- when transaction is done (durable)
Move old version to other temp file
Rollback --- if failure during a transaction (means it didn’t happen at all)
Move new version into real file
Unlink old version
n Do a set of operations tentatively. If you get to commit, ok. Otherwise,
roll back the operations as if the transaction never happened.
If crash, look at temp area; if any files out there, send email to user that
there might be a problem.

Transaction implementation
Transaction implementation (cont’d)
n Key idea: fix problem of how you make multiple updates to disk, by Disk
turning multiple updates into a single disk write! Memory cache

n Example: money transfer from account x to account y: X: 0 X: 0

Y: 2 Y: 2
Begin transaction
x =x+1
y = y–1
Commit Sequence of steps to execute transaction: X=1 Y=1 commit
1. Write new value of X to log
n Keep “write-ahead” (or “redo”) log on disk of all changes in 2. Write new value of Y to log write-ahead log (on disk or
transaction. 3. Write commit tape or non-volatile RAM)
n A log is like a journal, never erased, record of everything you’ve done 4. Write x to disk
n Once both changes are on log, transaction is committed. 5. Write y to disk
n Then can “write behind” changes to disk --- if crash after commit, replay
log to make sure updates get to disk 6. Reclaim space on log

Transaction implementation Transaction implementation

(cont’d) (cont’d)
u What if we crash after 1?
n Can we write X back to disk before commit ?
X=1 Y=1 commit
u No commit, nothing on disk, so
just ignore changes
1. Write new value of X to log
n Yes: Keep an “undo log”
u What if we crash after 2? Ditto
2. Write new value of Y to log u What if we crash after 3 before
3. Write commit 4 or 5? n Save old value along with new value
4. Write x to disk u Commit written to log, so replay n If transaction does not commit, “undo change”!
5. Write y to disk those changes back to disk
6. Reclaim space on log
u What if we crash while we are
writing “commit” ?
u As with concurrency, we need
some primitive atomic operation
or else can’t build anything. (e.g.,
writing a single sector on disk is
atomic!)

4
 Transactions under multiple
threads Two-phase locking
n What if two threads run same transaction at same time? use locks! n Don’t allow “unlock” before commit.
Begin transaction
Lock x, y
x=x+1
n First phase: only allowed to acquire locks (this avoids
y=y–1 deadlock concerns).
Unlock x, y
Commit
n Second phase: all unlocks happen at commit
n Problematic scenario:
n Thread A grabs locks, modifies x, y, writes to the log, unlocks
n Before A commits, thread B grabs lock, writes x, y, unlocks, and does n Thread B can’t see any of A’s changes, until A commits and
commit. releases locks. This provides serializability.
n Before A commits, it crashes!

B commits values for x and y that depends on A committing!

Transactions in file systems

n Write-ahead logging
n Almost all file systems built after 1985 (NT, Solaris) uses “write
ahead logging”
n Write all changes in a transaction to log (update directory, allocate
block, etc.) before sending any changes to disk.
n Example transactions: “Create file”, “Delete file”, “Move file”

This eliminates any need for file system check (fsck) after crash
If crash, read log:
n If log is not complete, no change!

n If log is completely written, apply all changes to disk

n If log is zero, then all updates have gotten to disk.

n Pros: reliability Cons: all data written twice