~bzr-pqm/bzr/bzr.dev

==============================
Development repository formats
==============================

.. contents::

Using development repository formats
====================================

Motivation
----------

We believe that we can continue to gain substantial performance benefits
by altering the repository storage in bzr. The more feedback we can get
on the changes during the development process the better.

To make it possible to get more feedback we are going to expose the
current development formats to the users of our development trunk
'bzr.dev'. The technical details of the individual formats are at the
end of this document.

Format names
------------

The current development format will be called 'development'. Each time
the development format changes, the prior development format will be
renamed to e.g. 'development0', 'development1' etc.

When a release of bzr is done, all the older numbered development
formats will be removed from 'bzr.dev', so we will not be carrying the
code for them around indefinately.

Support for upgrade and migration
---------------------------------

The preservation and renaming policy makes it quite safe for users to
test out development formats (though we cannot guarantee bugs of course
- it is development code):

 - users of a given development format can always get back onto regular
   formats by switching to the next bzr released version which is
   guaranteed to be able to upgrade from that development format.
 - users that routinely use bzr.dev should upgrade to the most recent
   development version available before pulling in bzr.dev changes
   around release time, as that is when old format cleanups will occur.

We cannot guarantee backwards compatability though, because some of the
planned work may be 'upgrade only'. Please see ``bzr help formats`` for
the text of the 'development' format which will indicate its
compatability with other formats if you need to interoperate with
users or services that do not have bzr.dev.

Before converting to a development format
-----------------------------------------

Run a ``bzr check`` with the version of bzr that you will be using.
``bzr check`` gets updated as we find new things that are inconsistent
with existing repositories. While only a small number of repositories
are likely to have any given error, it is best to check just in case.

If ``bzr check`` reports a problem, run this command::

  bzr reconcile

Note that reconcile can take many hours, particularly if you are
reconciling one of the 'knit' or 'dirstate' format repositories. If you
have such a repository, consider upgrading it to 'pack-0.92' first,
which will perform reconcile significantly faster.

Creating a new development format branch
----------------------------------------

If you're starting a project from scratch, it's easy to make it a
``development`` one. Here's how::

  cd my-stuff
  bzr init --development
  bzr add
  bzr commit -m "initial import"

In other words, use the normal sequence of commands but add the
``--development`` option to the ``init`` command.


Creating a new development format repository
--------------------------------------------

If you're starting a project from scratch and wish to use a shared repository
for branches, you can make it a ``development`` repository like this::

  cd my-repo
  bzr init-repo --development .
  cd my-stuff
  bzr init
  bzr add
  bzr commit -m "initial import"

In other words, use the normal sequence of commands but add the
``--development`` option to the ``init-repo`` command.

Upgrading an existing branch or repository to development
---------------------------------------------------------

If you have an existing branch and wish to migrate it to
a ``development`` format, use the ``upgrade`` command like this::

  bzr upgrade --development path-to-my-branch

If you are using a shared repository, run::

  bzr upgrade --development ROOT_OF_REPOSITORY

to upgrade the history database. Note that this will not
alter the branch format of each branch, so
you will need to also upgrade each branch individually
if you are upgrading from an old (e.g. < 0.17) bzr.
More modern bzr's will already have the branch format at
our latest branch format which adds support for tags.

Starting a new development format branch from one in an older format
--------------------------------------------------------------------

This can be done in one of several ways:

1. Create a new branch and pull into it
2. Create a standalone branch and upgrade its format
3. Create a knitpack shared repository and branch into it

Here are the commands for using the ``pull`` approach::

    bzr init --development my-new-branch
    cd my-new-branch
    bzr pull my-source-branch

Here are the commands for using the ``upgrade`` approach::

    bzr branch my-source-branch my-new-branch
    cd my-new-branch
    bzr upgrade --development .

Here are the commands for the shared repository approach::

  cd my-repo
  bzr init-repo --development .
  bzr branch my-source-branch my-new-branch
  cd my-new-branch

As a reminder, any of the above approaches can fail if the source branch
has inconsistent data within it and hasn't been reconciled yet. Please
be sure to check that before reporting problems.

Develoment formats for bzr-svn users
------------------------------------

If you are using ``bzr-svn`` or are testing the prototype subtree support,
you can still use and assist in testing the development formats. The
commands to use are identical to the ones given above except that the
name of the format to use is ``development-subtree``.

**WARNING**: Note that bzr only supports one-way conversion **to** the
subtree format ``development-subtree``. Once you are using
``development-subtree`` you cannot pull or merge back into a regular
format such as ``pack-0.92``, ``development`` etc.

The ``development-subtree`` format is required for the bzr-svn
plug-in but should otherwise not be used until the subtree feature is
complete within bzr.

Reporting problems
------------------

If you need any help or encounter any problems, please contact the developers
via the usual ways, i.e. chat to us on IRC or send a message to our mailing
list. See http://wiki.bazaar.canonical.com/BzrSupport for contact details.


Technical notes
===============

When to create a new development format
---------------------------------------

Whenever a code change will result in incorrect behaviour with existing
``development`` repositories. Changes in push/pull/init/commit/merge
have all been known to do this in the past.

How to create a new development format
--------------------------------------

1. Register two new formats with the next available sequence number.
   e.g. ``development1`` and ``development1-subtree``. (You can see the
   current development format for an example.
   These should:

   * Use your new development repository/branch/tree classes
   * Have really bare bones help - something like 'changes X to be Y
     see ...developers/development-repo.html'
   * Be hidden and experimental.
2. Change the repository class (or branch or tree) in the
   ``development`` and ``development-subtree`` formats to point to the
   new class you are creating.
3. Add a new development format (and tests!). Repository formats are in
   ``bzrlib.repofmt``. You probably want to reproduce the current
   development format from ``bzrlib.repofmt.pack_repo`` with just new
   disk format strings, ``_get_matching_bzrdir`` and help.
4. Register your development format with the various registries. At the
   moment you need to update:

    1. ``bzrlib/bzrdir.py`` to register the WT/Branch/Repository
       collection.

    2. ``bzrlib/workingtree.py``, ``bzrlib/branch.py``,
       ``bzrlib/repository.py``, each one maintains a direct list of
       their respective formats.

    3. For repositories, you also need to update the InterKnit1and2
       class. This is responsible for converting between rich-root and
       non-rich-root repositories.

    4. For repositories based on KnitPackRepository, you need to update
       ``bzrlib/tests/test_pack_repository.py`` to add the class to the
       tested permutations.

5. Alter any other things that do class based tests. The easiest way
   to find these is a grep for Development in bzrlib - and please
   refactor as you find these to reduce the relevance this step has,
   as it should not need to exist.
6. Now subclass/create from scratch/whatever the live object code you
   need to change to introduce your new format. Keep in mind that
   eventually it will become the default format, so please don't keep
   subclassing the last releases code, rather consider making the last
   releases code a subclass of your new code (if there is a lot in
   common) so that we can eventually remove that format once it becomes
   ancient (or relegate it to a plugin).
7. Once you have made the changes that required a new disk format, you
   should submit the resulting branch to be merged. Other changes - to
   take advantage of whatever new feature you have added - should be
   sent in separately, because the disk level changes are a contention
   point between multiple developers.

Format Details
==============

development
-----------

Not currently available, as our development formats are all rich root or
subtrees now.

development-rich-root
---------------------

Currently an alias for Development6Subtree

development-subtree
-------------------

Currently an alias for Development6Subtree

Development6RichRoot[Subtree]
-----------------------------

These formats use the new groupcompress delta compress and a CHK(Content
Hash Key) based inventory store which is much faster at incremental
operations than the prior XML based store.
*Note* Converting from a non-rich-root to a rich-root format is a
one-way upgrade, and you cannot merge back afterwards: using this format
for everyday use is likely to cause all developers of a project to
upgrade to a rich-root format themselves. This is fine, as bzr is moving
to make rich-root formats the default and to get all users to upgrade,
but we have not finalised the migration process, and until we do do not
recomment that casual users upgrade. Users of bzr-svn are already using
rich-root formats and can test with this with impunity.


..
   vim: tw=72 ft=rst expandtab