« back to all changes in this revision
Viewing changes to bzrlib/dirstate.py
- browse files at revision 5895
- compare with another revision
- compare with revision 6059
- download diff
- download tarball
- view history from revision 5895
- Committer: Canonical.com Patch Queue Manager
- Date: 2011-05-19 09:19:47 UTC
- mfrom: (5891.1.2 api-docs)
- Revision ID: pqm@pqm.ubuntu.com-20110519091947-5u0csgddxucufsrm
(spiv) Update 'api-docs' make target to use pydoctor,
and fix the formatting of many docstrings. (Andrew Bennetts)
and fix the formatting of many docstrings. (Andrew Bennetts)
- files modified:
- Makefile
added
removed
bzrlib/dirstate.py
20
20
lines by NL. The field delimiters are ommitted in the grammar, line delimiters
21
21
are not - this is done for clarity of reading. All string data is in utf8.
22
22
23
MINIKIND = "f" | "d" | "l" | "a" | "r" | "t";
24
NL = "\n";
25
NULL = "\0";
26
WHOLE_NUMBER = {digit}, digit;
27
BOOLEAN = "y" | "n";
28
REVISION_ID = a non-empty utf8 string;
29
30
dirstate format = header line, full checksum, row count, parent details,
31
ghost_details, entries;
32
header line = "#bazaar dirstate flat format 3", NL;
33
full checksum = "crc32: ", ["-"], WHOLE_NUMBER, NL;
34
row count = "num_entries: ", WHOLE_NUMBER, NL;
35
parent_details = WHOLE NUMBER, {REVISION_ID}* NL;
36
ghost_details = WHOLE NUMBER, {REVISION_ID}*, NL;
37
entries = {entry};
38
entry = entry_key, current_entry_details, {parent_entry_details};
39
entry_key = dirname, basename, fileid;
40
current_entry_details = common_entry_details, working_entry_details;
41
parent_entry_details = common_entry_details, history_entry_details;
42
common_entry_details = MINIKIND, fingerprint, size, executable
43
working_entry_details = packed_stat
44
history_entry_details = REVISION_ID;
45
executable = BOOLEAN;
46
size = WHOLE_NUMBER;
47
fingerprint = a nonempty utf8 sequence with meaning defined by minikind.
48
49
Given this definition, the following is useful to know:
50
entry (aka row) - all the data for a given key.
51
entry[0]: The key (dirname, basename, fileid)
52
entry[0][0]: dirname
53
entry[0][1]: basename
54
entry[0][2]: fileid
55
entry[1]: The tree(s) data for this path and id combination.
56
entry[1][0]: The current tree
57
entry[1][1]: The second tree
58
59
For an entry for a tree, we have (using tree 0 - current tree) to demonstrate:
60
entry[1][0][0]: minikind
61
entry[1][0][1]: fingerprint
62
entry[1][0][2]: size
63
entry[1][0][3]: executable
64
entry[1][0][4]: packed_stat
65
OR (for non tree-0)
66
entry[1][1][4]: revision_id
23
::
24
25
MINIKIND = "f" | "d" | "l" | "a" | "r" | "t";
26
NL = "\\n";
27
NULL = "\\0";
28
WHOLE_NUMBER = {digit}, digit;
29
BOOLEAN = "y" | "n";
30
REVISION_ID = a non-empty utf8 string;
31
32
dirstate format = header line, full checksum, row count, parent details,
33
ghost_details, entries;
34
header line = "#bazaar dirstate flat format 3", NL;
35
full checksum = "crc32: ", ["-"], WHOLE_NUMBER, NL;
36
row count = "num_entries: ", WHOLE_NUMBER, NL;
37
parent_details = WHOLE NUMBER, {REVISION_ID}* NL;
38
ghost_details = WHOLE NUMBER, {REVISION_ID}*, NL;
39
entries = {entry};
40
entry = entry_key, current_entry_details, {parent_entry_details};
41
entry_key = dirname, basename, fileid;
42
current_entry_details = common_entry_details, working_entry_details;
43
parent_entry_details = common_entry_details, history_entry_details;
44
common_entry_details = MINIKIND, fingerprint, size, executable
45
working_entry_details = packed_stat
46
history_entry_details = REVISION_ID;
47
executable = BOOLEAN;
48
size = WHOLE_NUMBER;
49
fingerprint = a nonempty utf8 sequence with meaning defined by minikind.
50
51
Given this definition, the following is useful to know::
52
53
entry (aka row) - all the data for a given key.
54
entry[0]: The key (dirname, basename, fileid)
55
entry[0][0]: dirname
56
entry[0][1]: basename
57
entry[0][2]: fileid
58
entry[1]: The tree(s) data for this path and id combination.
59
entry[1][0]: The current tree
60
entry[1][1]: The second tree
61
62
For an entry for a tree, we have (using tree 0 - current tree) to demonstrate::
63
64
entry[1][0][0]: minikind
65
entry[1][0][1]: fingerprint
66
entry[1][0][2]: size
67
entry[1][0][3]: executable
68
entry[1][0][4]: packed_stat
69
70
OR (for non tree-0)::
71
72
entry[1][1][4]: revision_id
67
73
68
74
There may be multiple rows at the root, one per id present in the root, so the
69
in memory root row is now:
70
self._dirblocks[0] -> ('', [entry ...]),
71
and the entries in there are
72
entries[0][0]: ''
73
entries[0][1]: ''
74
entries[0][2]: file_id
75
entries[1][0]: The tree data for the current tree for this fileid at /
76
etc.
77
78
Kinds:
79
'r' is a relocated entry: This path is not present in this tree with this id,
80
but the id can be found at another location. The fingerprint is used to
81
point to the target location.
82
'a' is an absent entry: In that tree the id is not present at this path.
83
'd' is a directory entry: This path in this tree is a directory with the
84
current file id. There is no fingerprint for directories.
85
'f' is a file entry: As for directory, but it's a file. The fingerprint is the
86
sha1 value of the file's canonical form, i.e. after any read filters have
87
been applied to the convenience form stored in the working tree.
88
'l' is a symlink entry: As for directory, but a symlink. The fingerprint is the
89
link target.
90
't' is a reference to a nested subtree; the fingerprint is the referenced
91
revision.
75
in memory root row is now::
76
77
self._dirblocks[0] -> ('', [entry ...]),
78
79
and the entries in there are::
80
81
entries[0][0]: ''
82
entries[0][1]: ''
83
entries[0][2]: file_id
84
entries[1][0]: The tree data for the current tree for this fileid at /
85
etc.
86
87
Kinds::
88
89
'r' is a relocated entry: This path is not present in this tree with this
90
id, but the id can be found at another location. The fingerprint is
91
used to point to the target location.
92
'a' is an absent entry: In that tree the id is not present at this path.
93
'd' is a directory entry: This path in this tree is a directory with the
94
current file id. There is no fingerprint for directories.
95
'f' is a file entry: As for directory, but it's a file. The fingerprint is
96
the sha1 value of the file's canonical form, i.e. after any read
97
filters have been applied to the convenience form stored in the working
98
tree.
99
'l' is a symlink entry: As for directory, but a symlink. The fingerprint is
100
the link target.
101
't' is a reference to a nested subtree; the fingerprint is the referenced
102
revision.
92
103
93
104
Ordering:
94
105
95
The entries on disk and in memory are ordered according to the following keys:
106
The entries on disk and in memory are ordered according to the following keys::
96
107
97
108
directory, as a list of components
98
109
filename
99
110
file-id
100
111
101
112
--- Format 1 had the following different definition: ---
102
rows = dirname, NULL, basename, NULL, MINIKIND, NULL, fileid_utf8, NULL,
103
WHOLE NUMBER (* size *), NULL, packed stat, NULL, sha1|symlink target,
104
{PARENT ROW}
105
PARENT ROW = NULL, revision_utf8, NULL, MINIKIND, NULL, dirname, NULL,
106
basename, NULL, WHOLE NUMBER (* size *), NULL, "y" | "n", NULL,
107
SHA1
113
114
::
115
116
rows = dirname, NULL, basename, NULL, MINIKIND, NULL, fileid_utf8, NULL,
117
WHOLE NUMBER (* size *), NULL, packed stat, NULL, sha1|symlink target,
118
{PARENT ROW}
119
PARENT ROW = NULL, revision_utf8, NULL, MINIKIND, NULL, dirname, NULL,
120
basename, NULL, WHOLE NUMBER (* size *), NULL, "y" | "n", NULL,
121
SHA1
108
122
109
123
PARENT ROW's are emitted for every parent that is not in the ghosts details
110
124
line. That is, if the parents are foo, bar, baz, and the ghosts are bar, then
135
149
----
136
150
137
151
Design priorities:
138
1) Fast end to end use for bzr's top 5 uses cases. (commmit/diff/status/merge/???)
139
2) fall back current object model as needed.
140
3) scale usably to the largest trees known today - say 50K entries. (mozilla
152
1. Fast end to end use for bzr's top 5 uses cases. (commmit/diff/status/merge/???)
153
2. fall back current object model as needed.
154
3. scale usably to the largest trees known today - say 50K entries. (mozilla
141
155
is an example of this)
142
156
143
157
144
158
Locking:
159
145
160
Eventually reuse dirstate objects across locks IFF the dirstate file has not
146
161
been modified, but will require that we flush/ignore cached stat-hit data
147
162
because we won't want to restat all files on disk just because a lock was
148
163
acquired, yet we cannot trust the data after the previous lock was released.
149
164
150
Memory representation:
165
Memory representation::
166
151
167
vector of all directories, and vector of the childen ?
152
168
i.e.
153
169
root_entrie = (direntry for root, [parent_direntries_for_root]),
167
183
- What's the risk of error here? Once we have the base format being processed
168
184
we should have a net win regardless of optimality. So we are going to
169
185
go with what seems reasonable.
186
170
187
open questions:
171
188
172
189
Maybe we should do a test profile of the core structure - 10K simulated
1205
1222
def _fields_per_entry(self):
1206
1223
"""How many null separated fields should be in each entry row.
1207
1224
1208
Each line now has an extra '\n' field which is not used
1225
Each line now has an extra '\\n' field which is not used
1209
1226
so we just skip over it
1210
entry size:
1227
1228
entry size::
1211
1229
3 fields for the key
1212
1230
+ number of fields per tree_data (5) * tree count
1213
1231
+ newline
Loggerhead is a web-based interface for Breezy
Version: 2.0.1