1 Notes on Filesystem Layout
2 --------------------------
3
4 These notes describe what mkcramfs generates. Kernel requirements are
5 a bit looser, e.g. it doesn't care if the <file_data> items are
6 swapped around (though it does care that directory entries (inodes) in
7 a given directory are contiguous, as this is used by readdir).
8
9 All data is in host-endian format; neither mkcramfs nor the kernel
10 ever do swabbing. (See section `Block Size' below.)
11
12 <filesystem>:
13 <superblock>
14 <directory_structure>
15 <data>
16
17 <superblock>: struct cramfs_super (see cramfs.h).
18
19 <directory_structure>:
20 For each file:
21 struct cramfs_inode (see cramfs.h).
22 Filename. Not generally null-terminated, but it is
23 null-padded to a multiple of 4 bytes.
24
25 The order of inode traversal is described as "width-first" (not to be
26 confused with breadth-first); i.e. like depth-first but listing all of
27 a directory's entries before recursing down its subdirectories: the
28 same order as `ls -AUR' (but without the /^\..*:$/ directory header
29 lines); put another way, the same order as `find -type d -exec
30 ls -AU1 {} \;'.
31
32 <data>:
33 One <file_data> for each file that's either a symlink or a
34 regular file of non-zero st_size.
35
36 <file_data>:
37 nblocks * <block_pointer>
38 (where nblocks = (st_size - 1) / blksize + 1)
39 nblocks * <block>
40 padding to multiple of 4 bytes
41
42 The i'th <block_pointer> for a file stores the byte offset of the
43 *end* of the i'th <block> (i.e. one past the last byte, which is the
44 same as the start of the (i+1)'th <block> if there is one). The first
45 <block> immediately follows the last <block_pointer> for the file.
46 <block_pointer>s are each 32 bits long.
47
48 The order of <file_data>'s is a depth-first descent of the directory
49 tree, i.e. the same order as `find -size +0 \( -type f -o -type l \)
50 -print'.
51
52
53 <block>: The i'th <block> is the output of zlib's compress function
54 applied to the i'th blksize-sized chunk of the input data.
55 (For the last <block> of the file, the input may of course be smaller.)
56 Each <block> may be a different size. (See <block_pointer> above.)
57 <block>s are merely byte-aligned, not generally u32-aligned.
58
59
60 Holes
61 -----
62
63 This kernel supports cramfs holes (i.e. [efficient representation of]
64 blocks in uncompressed data consisting entirely of NUL bytes), but by
65 default mkcramfs doesn't test for & create holes, since cramfs in
66 kernels up to at least 2.3.39 didn't support holes. Compile mkcramfs
67 with -DDO_HOLES if you want it to create files that can have holes in
68 them.
69
70
71 Tools
72 -----
73
74 If you're hacking on cramfs, you might find useful some tools for
75 testing cramfs at <http://cvs.bofh.asn.au/cramfs/>, including a
76 rudimentary fsck for cramfs.
77
78
79 Future Development
80 ==================
81
82 Block Size
83 ----------
84
85 (Block size in cramfs refers to the size of input data that is
86 compressed at a time. It's intended to be somewhere around
87 PAGE_CACHE_SIZE for cramfs_readpage's convenience.)
88
89 The superblock ought to indicate the block size that the fs was
90 written for, since comments in <linux/pagemap.h> indicate that
91 PAGE_CACHE_SIZE may grow in future (if I interpret the comment
92 correctly).
93
94 Currently, mkcramfs #define's PAGE_CACHE_SIZE as 4096 and uses that
95 for blksize, whereas Linux-2.3.39 uses its PAGE_CACHE_SIZE, which in
96 turn is defined as PAGE_SIZE (which can be as large as 32KB on arm).
97 This discrepancy is a bug, though it's not clear which should be
98 changed.
99
100 One option is to change mkcramfs to take its PAGE_CACHE_SIZE from
101 <asm/page.h>. Personally I don't like this option, but it does
102 require the least amount of change: just change `#define
103 PAGE_CACHE_SIZE (4096)' to `#include <asm/page.h>'. The disadvantage
104 is that the generated cramfs cannot always be shared between different
105 kernels, not even necessarily kernels of the same architecture if
106 PAGE_CACHE_SIZE is subject to change between kernel versions.
107
108
109 The remaining options try to make cramfs more sharable.
110
111 One part of that is addressing endianness. The two options here are
112 `always use little-endian' (like ext2fs) or `writer chooses
113 endianness; kernel adapts at runtime'. Little-endian wins because of
114 code simplicity and little CPU overhead even on big-endian machines.
115
116 The cost of swabbing is changing the code to use the le32_to_cpu
117 etc. macros as used by ext2fs. We don't need to swab the compressed
118 data, only the superblock, inodes and block pointers.
119
120
121 The other part of making cramfs more sharable is choosing a block
122 size. The options are:
123
124 1. Always 4096 bytes.
125
126 2. Writer chooses blocksize; kernel adapts but rejects blocksize >
127 PAGE_CACHE_SIZE.
128
129 3. Writer chooses blocksize; kernel adapts even to blocksize >
130 PAGE_CACHE_SIZE.
131
132 It's easy enough to change the kernel to use a smaller value than
133 PAGE_CACHE_SIZE: just make cramfs_readpage read multiple blocks.
134
135 The cost of option 1 is that kernels with a larger PAGE_CACHE_SIZE
136 value don't get as good compression as they can.
137
138 The cost of option 2 relative to option 1 is that the code uses
139 variables instead of #define'd constants. The gain is that people
140 with kernels having larger PAGE_CACHE_SIZE can make use of that if
141 they don't mind their cramfs being inaccessible to kernels with
142 smaller PAGE_CACHE_SIZE values.
143
144 Option 3 is easy to implement if we don't mind being CPU-inefficient:
145 e.g. get readpage to decompress to a buffer of size MAX_BLKSIZE (which
146 must be no larger than 32KB) and discard what it doesn't need.
147 Getting readpage to read into all the covered pages is harder.
148
149 The main advantage of option 3 over 1, 2, is better compression. The
150 cost is greater complexity. Probably not worth it, but I hope someone
151 will disagree. (If it is implemented, then I'll re-use that code in
152 e2compr.)
153
154
155 Another cost of 2 and 3 over 1 is making mkcramfs use a different
156 block size, but that just means adding and parsing a -b option.
157
158
159 Inode Size
160 ----------
161
162 Given that cramfs will probably be used for CDs etc. as well as just
163 silicon ROMs, it might make sense to expand the inode a little from
164 its current 12 bytes. Inodes other than the root inode are followed
165 by filename, so the expansion doesn't even have to be a multiple of 4
166 bytes.
This page was automatically generated by the
LXR engine.
Visit the LXR main site for more
information.