1
=============================
2
Bazaar Architectural Overview
3
=============================
5
This document describes the key classes and concepts within Bazaar. It is
6
intended to be useful to people working on the Bazaar codebase, or to
7
people writing plugins. People writing plugins may also like to read the
8
guide to `Integrating with Bazaar <integration.html>`_ for some specific recipes.
10
There's some overlap between this and the `Core Concepts`_ section of the
11
user guide, but this document is targetted to people interested in the
12
internals. In particular the user guide doesn't go any deeper than
13
"revision", because regular users don't care about lower-level details
14
like inventories, but this guide does.
16
If you have any questions, or if something seems to be incorrect, unclear
17
or missing, please talk to us in ``irc://irc.freenode.net/#bzr``, write to
18
the Bazaar mailing list, or simply file a bug report.
27
All IDs are globally unique identifiers. Inside bzrlib they are almost
28
always represented as UTF-8 encoded bytestrings (i.e. ``str`` objects).
32
:Revision IDs: The unique identifier of a single revision, such as
33
``pqm@pqm.ubuntu.com-20110201161347-ao76mv267gc1b5v2``
34
:File IDs: The unique identifier of a single file.
36
By convention, in the bzrlib API, parameters of methods that are expected
37
to be IDs (as opposed to keys, revision numbers, or some other handle)
38
will end in ``id``, e.g. ``revid`` or ``file_id``.
40
Ids may be stored directly or they can be inferred from other
41
data. Native Bazaar formats store ids directly; foreign VCS
42
support usually generates them somehow. For example, the
43
Git commit with SHA ``fb235a3be6372e40ff7f7ebbcd7905a08cb04444``
44
is referred to with the revision ID
45
``git-v1:fb235a3be6372e40ff7f7ebbcd7905a08cb04444``. IDs are expected
51
File ids are unique identifiers for files. There are three slightly different
52
categories of file ids.
57
Tree file ids are used in the ``Tree`` API and can either be UTF-8 encoded
58
bytestrings or tuples of UTF-8 encoded bytestrings. Plain bytestrings
59
are considered to be the equivalent of a 1-tuple containing that
62
Tree file ids should be considered valid only for a specific tree context.
63
Note that this is a stricter interpretation than what the current bzr
64
format implementation provides - its file ids are persistent across runs
67
For some formats (most notably bzr's own formats) it's possible for the
68
implementation to specify the file id to use. In other case the tree
69
mandates a specific file id.
74
Inventories are specific to the bzr native format and are the main layer
75
below the ``Tree`` implementation of bzr. File ids in inventories can
76
only be UTF-8 encoded bytestrings. A single Tree object can be associated
77
with multiple inventories if there are nested trees.
79
Tree file ids for bzr formats are a tuple of inventory file ids for the file
80
in question. Each non-last item in the tuple refers to the tree
81
reference of an inner tree. The last item in the tuple refers to the
82
actual file. This means that lookups of file ids doesn't scale with
83
the number of nested trees.
85
Inventory file ids are only relevant for native Bazaar formats; foreign
86
formats don't use inventories.
91
Transform ids are used during tree transform operations (used by e.g. merge).
92
The same transform id is expected to be used for two instances of the
93
same file. At the moment transform ids are directly derived from file
94
ids, but in the future they could be based on other data too (e.g.
95
automatic rename detection or format-specific rules).
100
A composite of one or more ID elements. E.g. a (file-id, revision-id)
101
pair is the key to the "texts" store, but a single element key of
102
(revision-id) is the key to the "revisions" store.
111
The ``Transport`` layer handles access to local or remote directories.
112
Each Transport object acts as a logical connection to a particular
113
directory, and it allows various operations on files within it. You can
114
*clone* a transport to get a new Transport connected to a subdirectory or
117
Transports are not used for access to the working tree. At present
118
working trees are always local and they are accessed through the regular
119
Python file I/O mechanisms.
124
Transports work in terms of URLs. Take note that URLs are by definition
125
only ASCII - the decision of how to encode a Unicode string into a URL
126
must be taken at a higher level, typically in the Store. (Note that
127
Stores also escape filenames which cannot be safely stored on all
128
filesystems, but this is a different level.)
130
The main reason for this is that it's not possible to safely roundtrip a
131
URL into Unicode and then back into the same URL. The URL standard
132
gives a way to represent non-ASCII bytes in ASCII (as %-escapes), but
133
doesn't say how those bytes represent non-ASCII characters. (They're not
134
guaranteed to be UTF-8 -- that is common but doesn't happen everywhere.)
136
For example, if the user enters the URL ``http://example/%e0``, there's no
137
way to tell whether that character represents "latin small letter a with
138
grave" in iso-8859-1, or "latin small letter r with acute" in iso-8859-2,
139
or malformed UTF-8. So we can't convert the URL to Unicode reliably.
141
Equally problematic is if we're given a URL-like string containing
142
(unescaped) non-ASCII characters (such as the accented a). We can't be
143
sure how to convert that to a valid (i.e. ASCII-only) URL, because we
144
don't know what encoding the server expects for those characters.
145
(Although it is not totally reliable, we might still accept these and
146
assume that they should be put into UTF-8.)
148
A similar edge case is that the URL ``http://foo/sweet%2Fsour`` contains
149
one directory component whose name is "sweet/sour". The escaped slash is
150
not a directory separator, but if we try to convert the URL to a regular
151
Unicode path, this information will be lost.
153
This implies that Transports must natively deal with URLs. For simplicity
154
they *only* deal with URLs; conversion of other strings to URLs is done
155
elsewhere. Information that Transports return, such as from ``list_dir``,
156
is also in the form of URL components.
163
* `Developer guide to bzrlib transports <transports.html>`_
164
* API docs for ``bzrlib.transport.Transport``
169
Each control directory (such as ".bzr/") can contain zero or one
170
repositories, zero or one working trees and zero or more branches.
172
The ``BzrDir`` class is the ``ControlDir`` implementation that is
173
responsible for the ".bzr/" directory and its implementation. Plugins
174
that provide support for other version control systems can provide
175
other subclasses of ``ControlDir``.
180
A representation of a directory of files (and other directories and
181
symlinks etc). The most important kinds of Tree are:
183
:WorkingTree: the files on disk editable by the user
184
:RevisionTree: a tree as recorded at some point in the past
186
Trees can map file paths to file-ids and vice versa (although trees such
187
as WorkingTree may have unversioned files not described in that mapping).
188
Trees have an inventory and parents (an ordered list of zero or more
191
The implementation of ``Tree`` for Bazaar's own formats is based around
192
``Inventory`` objects which describe the shape of the tree. Each tree has
193
at least one inventory associated with it, which is available as the
194
``root_inventory`` attribute on tree. The tree can have more inventories
195
associated with it if there are references to other trees in it. These
196
references are indicated with ``tree-reference`` inventory entry at the
197
point where the other tree is nested. The tree reference entry contains
198
sufficient information for looking up the inventory associated with the
199
nested tree. There can be multiple layers of nesting.
201
Not each ``Tree`` implementation will necessarily have an associated
202
``root_inventory``, as not all implementations of ``Tree`` are based
203
around inventories (most notably, implementations of foreign VCS file
209
A workingtree is a special type of Tree that's associated with a working
210
directory on disk, where the user can directly modify the files.
214
* Maintaining a WorkingTree on disk at a file path.
215
* Maintaining the basis inventory (the inventory of the last commit done)
216
* Maintaining the working inventory.
217
* Maintaining the pending merges list.
218
* Maintaining the stat cache.
219
* Maintaining the last revision the working tree was updated to.
220
* Knows where its Branch is located.
225
* local access to the working tree
231
A Branch is a key user concept - its a single line of history that one or
232
more people have been committing to.
234
A Branch is responsible for:
236
* Holding user preferences that are set in a Branch.
237
* Holding the 'tip': the last revision to be committed to this Branch.
238
(And the revno of that revision.)
239
* Knowing how to open the Repository that holds its history.
240
* Allowing write locks to be taken out to prevent concurrent alterations to the branch.
244
* URL access to its base directory.
245
* A Transport to access its files.
246
* A Repository to hold its history.
252
Repositories store committed history: file texts, revisions, inventories,
253
and graph relationships between them. A repository holds a bag of
254
revision data that can be pointed to by various branches:
256
* Maintains storage of various history data at a URL:
258
* Revisions (Must have a matching inventory)
260
* Inventories for each Revision. (Must have all the file texts available).
263
* Synchronizes concurrent access to the repository by different
264
processes. (Most repository implementations use a physical mutex only
265
for a short period, and effectively support multiple readers and
271
A repository can be configured to refer to a list of "fallback"
272
repositories. If a particular revision is not present in the original
273
repository, it refers the query to the fallbacks.
275
Compression deltas don't span physical repository boundaries. So the
276
first commit to a new, empty repository with fallback repositories will
277
store a full text of the inventory, and of every new file text.
279
At runtime, repository stacking is actually configured by the branch, not
280
the repository. So doing ``a_bzrdir.open_repository()``
281
gets you just the single physical repository, while
282
``a_bzrdir.open_branch().repository`` gets one configured with a stacking.
283
Therefore, to permanently change the fallback repository stored on disk,
284
you must use ``Branch.set_stacked_on_url``.
286
Changing away from an existing stacked-on URL will copy across any
287
necessary history so that the repository remains usable.
289
A repository opened from an HPSS server is never stacked on the server
290
side, because this could cause complexity or security problems with the
291
server acting as a proxy for the client. Instead, the branch on the
292
server exposes the stacked-on URL and the client can open that.
298
This section describes the model for how bzr stores its data. The
299
representation of that data on disk varies considerable depending on the
300
format of the repository (and to a lesser extent the format of the branch
301
and working tree), but ultimately the set of objects being represented is
307
A branch directly contains:
309
* the ID of the current revision that branch (a.k.a. the “tip”)
310
* some settings for that branch (the values in “branch.conf”)
311
* the set of tags for that branch (not supported in all formats)
313
A branch implicitly references:
315
* A repository. The repository might be colocated in the same directory
316
as the branch, or it might be somewhere else entirely.
322
A repository contains:
329
A store is a key-value mapping. This says nothing about the layout on
330
disk, just that conceptually there are distinct stores, each with a
331
separate namespace for the keys. Internally the repository may serialize
332
stores in the same file, and/or e.g. apply compression algorithms that
333
combine records from separate stores in one block, etc.
335
You can consider the repository as a single key space, with keys that look
336
like *(store-name, ...)*. For example, *('revisions',
337
revision-id)* or *('texts', revision-id, file-id)*.
342
Stores revision objects. The keys are GUIDs. The value is a revision
343
object (the exact representation on disk depends on the repository
346
As described in `Core Concepts`_ a revision describes a snapshot of the
347
tree of files and some metadata about them.
351
* parent revisions (an ordered sequence of zero or more revision IDs)
355
* (and all other revision properties)
357
* an inventory ID (that inventory describes the tree contents). Is often
358
the same as the revision ID, but doesn't have to be (e.g. if no files
359
were changed between two revisions then both revisions will refer to
366
Stores inventory objects. The keys are GUIDs. (Footnote: there will
367
usually be a revision with the same key in the revision store, but there
368
are rare cases where this is not true.)
370
An inventory object contains:
372
* a set of inventory entries
374
An inventory entry has the following attributes
376
* a file-id (a GUID, or the special value TREE_ROOT for the root entry of
377
inventories created by older versions of bzr)
378
* a revision-id, a GUID (generally corresponding to the ID of a
379
revision). The combination of (file-id, revision-id) is a key into the
381
* a kind: one of file, directory, symlink, tree-reference (tree-reference
382
is only supported in unsupported developer formats)
383
* parent-id: the file-id of the directory that contains this entry (this
384
value is unset for the root of the tree).
385
* name: the name of the file/directory/etc in that parent directory
386
* executable: a flag indicating if the executable bit is set for that
389
An inventory entry will have other attributes, depending on the kind:
408
For some more details see `Inventories <inventory.html>`_.
414
Stores the contents of individual versions of files. The keys are pairs
415
of (file-id, revision-id), and the values are the full content (or
416
"text") of a version of a file.
418
For consistency/simplicity text records exist for all inventory entries,
419
but in general only entries with of kind "file" have interesting records.
425
Stores cryptographic signatures of revision contents. The keys match
426
those of the revision store.
428
.. _Core Concepts: http://doc.bazaar.canonical.com/latest/en/user-guide/core_concepts.html
added
removed
doc/developers/overview.txt
