~bzr-pqm/bzr/bzr.dev

.. -*- mode: indented-text; compile-command: "make -C doc" -*-

*******************

Things to do in bzr

*******************

See also various low-level TODOs in the source code.  Try looking in

the list archive or on gmane.org for previous discussion of these

issues.

These are classified by approximate size: an hour or less, a day or

less, and several days or more.

Small things

------------

* Fix tests so that import errors caused by modules don't produce false reports

  that the tests themselves don't exist.

* Fix tests so that one test failure doesn't prevent other tests from running

* print a message at the end of running the tests telling them that the

  test log and output exists but can be removed

* tests for running the commit editor, and fix problem of not passing in 

  multiple arguments

* Merging add of a new file clashing with an existing file doesn't

  work; add gets an error that it's already versioned and the merge

  aborts.

* Merge should ignore the destination's working directory, otherwise

  we get an error about the statcache when pulling from a remote

  branch.

* Add of a file that was present in the base revision should put back

  the previous file-id.

* Not sure I'm happy with needing to pass a root id to EmptyTree;

  comparing anything against an EmptyTree with no root should have the

  same effect(?)

* Handle diff of files which do not have a trailing newline; probably

  requires patching difflib to get it exactly right, or otherwise

  calling out to GNU diff.

* Should be able to copy files between branches to preserve their

  file-id (and perhaps eventually parentage.)

* -r option should take a revision-id as well as a revno.

* ``bzr info`` should count only people with distinct email addresses as

  different committers.  (Or perhaps only distinct userids?)

* On Windows, command-line arguments should be `glob-expanded`__,

  because the shell doesn't do this.  However, there are probably some

  commands where this shouldn't be done, such as 'bzr ignore', because

  we want to accept globs.

* ``bzr ignore`` command that just adds a line to the ``.bzrignore`` file

  and makes it versioned.  Fix this to break symlinks.

* Any useful sanity checks in 'bzr ignore'?  Perhaps give a warning if

  they try to add a single file which is already versioned, or if they

  add a pattern which already exists, or if it looks like they gave an

  unquoted glob.

__ http://mail.python.org/pipermail/python-list/2001-April/037847.html

* Separate read and write version checks?

* ``bzr status DIR`` should give status on all files under that

  directory.

* ``bzr log DIR`` should give changes to any files within DIR; at the

  moment it only lists things which modify the specific named file

  (and not its contents)

* ``bzr inventory -r REV`` and perhaps unify this with ``bzr ls``,

  giving options to display ids, types, etc.

* RemoteBranch could maintain a cache either in memory or on disk.  We

  know more than an external cache might about which files are

  immutable and which can vary.  On the other hand, it's much simpler

  to just use an external proxy cache.

  Perhaps ~/.bzr/http-cache.  Baz has a fairly simple cache under

  ~/.arch-cache, containing revision information encoded almost as a

  bunch of archives.  Perhaps we could simply store full paths.

* Maybe also store directories in the statcache so that we can quickly

  identify that they still exist.

* Diff should show timestamps; for files from the working directory we

  can use the file itself; for files from a revision we should use the

  commit time of the revision.

* Perhaps split command infrastructure from the actual command

  definitions.

* Cleaner support for negative boolean options like --no-recurse.

* Statcache should possibly map all file paths to / separators

* quotefn doubles all backslashes on Windows; this is probably not the

  best thing to do.  What would be a better way to safely represent

  filenames?  Perhaps we could doublequote things containing spaces,

  on the principle that filenames containing quotes are unlikely?

  Nice for humans; less good for machine parsing.

* Patches should probably use only forward slashes, even on Windows,

  otherwise Unix patch can't apply them. (?)

* Branch.update_revisions() inefficiently fetches revisions from the

  remote server twice; once to find out what text and inventory they

  need and then again to actually get the thing.  This is a bit

  inefficient.

  One complicating factor here is that we don't really want to have

  revisions present in the revision-store until all their constituent

  parts are also stored.

  The basic problem is that RemoteBranch.get_revision() and similar

  methods return object, but what we really want is the raw XML, which

  can be popped into our own store.  That needs to be refactored.

* Guard against repeatedly merging any particular patch.

* More options for diff:

  - diff two revisions of the same tree

  - diff two different branches, optionally at different revisions

  - diff a particular file in another tree against the corresponding

    version in this tree (which should be the default if the second

    parameter is a tree root)

  - diff everything under a particular directory, in any of the above

    ways

  - diff two files inside the same tree, even if they have different

    ids

  - and, of course, tests for all this

* Reproducible performance benchmark to measure whether performance is 

  getting better or worse.

* ``bzr log -m foo`` should perhaps error if nothing matches?

* ``bzr diff -r 30 -r 40 foo.c`` or ``bzr diff -r30..40 foo.c``

  If diffing between two branches then we probably want two -r

  options, since the revisions don't form a range that can be

  evaluated on either one.

* bzr diff shouldn't diff binary files

* setup.py install when run from a bzr tree should freeze the tree

  revision-id into the installed bzr.

* bzr script should trap ImportError and perhaps give a better error

  message?

* revert after a merge should possibly remove all the BASE/THIS/OTHER

  files to get you back to where you were.

* files that are added and then deleted are still reported as added

* stores should raise KeyError, not IndexError

* merging from a remote branch seems to sometimes raise errors not

  present locally

* should be possible to give a related branch when pulling from a

  remote branch to make things faster

* sometimes gives "conflicting add" even when the contents are in fact

  the same???

* BZRDIR should be in branch.py not __init__.py.

Medium things

-------------

* merge should add all revision and inventory XML to the local store.

* check should give a warning for revisions that are named in the

  chain but not actually present in the store.

* remove anything outside of the branch implementation that directly

  accesses the stores.

* More efficient diff of only selected files.  We should be able to

  just get the id for the selected files, look up their location and

  diff just those files.  No need to traverse the entire inventories.

* Fix up Inventory objects to represent root object as an entry.

* Don't convert entire entry from ElementTree to an object when it is

  read in, but rather wait until the program actually wants to know

  about that node.

* Extract changes from one revision to the next to a text form

  suitable for transmission over email.

* More test cases.

  - ``missing`` command

  - Selected-file commit

  - Impossible selected-file commit: adding things in non-versioned

    directories, crossing renames, etc.

* Write a reproducible benchmark, perhaps importing various kernel versions.

* Directly import diffs!  It seems a bit redundant to need to rescan

  the directory to work out what files diff added/deleted/changed when

  all the information is there in the diff in the first place.

  Getting the exact behaviour for added/deleted subdirectories etc

  might be hard.

  At the very least we could run diffstat over the diff, or perhaps

  read the status output from patch.  Just knowing which files might

  be modified would be enough to guide the add and commit.

  Given this we might be able to import patches at 1/second or better.

* revfile compression.

* Split inventory into per-directory files?

* Fix ignore file parsing:

  - fnmatch is not the same as unix patterns

  - perhaps add extended globs from rsh/rsync

  - perhaps a pattern that matches only directories or non-directories

* Consider using Python logging library as well as/instead of

  bzrlib.trace.

* Commands should give some progress indication by default.

  - But quieten this with ``--silent``.

* Change to using gettext message localization.

* Make a clearer separation between internal and external bzrlib

  interfaces.  Make internal interfaces use protected names.  Write at

  least some documentation for those APIs, probably as docstrings.

  Consider using ZopeInterface definitions for the external interface;

  I think these are already used in PyBaz.  They allow automatic

  checking of the interface but may be unfamiliar to general Python

  developers, so I'm not really keen.

* Commands to dump out all command help into a manpage or HTML file or

  whatever.

* Handle symlinks in the working directory; at the very least it

  should be possible for them to be present and ignored/unknown

  without causing assertion failures.

  Eventually symlinks should be versioned.

* Allow init in a subdirectory to create a nested repository, but only

  if the subdirectory is not already versioned.   Perhaps also require

  a ``--nested`` to protect against confusion.

* Branch names?

* More test framework:

  - Class that describes the state of a working tree so we can just

    assert it's equal.

* Try using XSLT to add some formatting to REST-generated HTML.  Or

  maybe write a small Python program that specifies a header and foot

  for the pages and calls into the docutils libraries.

* --format=xml for log, status and other commands.

* Attempting to explicitly add a file that's already added should give

  a warning; however there should be no warning for directories (since

  we scan for new children) or files encountered in a directory that's

  being scanned.

* Better handling of possible collisions on case-losing filesystems;

  make sure a single file does not get added twice under different

  names.

* Clean up XML inventory:

  - Use nesting rather than parent_id pointers.

  - Hold the ElementTree in memory in the Inventory object and work

    directly on that, rather than converting into Python objects every

    time it is read in.  Probably still expose it through some kind of

    object interface though, but perhaps that should just be a proxy

    for the elements.

  - Less special cases for the root directory.

* Perhaps inventories should remember the revision in which each file

  was last changed, as well as its current state?  This is a bit

  redundant but might often be interested to know.

* stat cache should perhaps only stat files as necessary, rather than

  doing them all up-front.  On the other hand, that disallows the

  optimization of stating them in inode order.

* It'd be nice to pipeline multiple HTTP requests.  Often we can

  predict what will be wanted in future: all revisions, or all texts

  in a particular revision, etc.

  urlgrabber's docs say they are working on batched downloads; we

  could perhaps ride on that or just create a background thread (ew).

* Paranoid mode where we never trust SHA-1 matches.

* --dry-run mode for commit?  (Or maybe just run with

  check-command=false?)

* Generally, be a bit more verbose unless --silent is specified.

* Function that finds all changes to files under a given directory;

  perhaps log should use this if a directory is given.

* XML attributes might have trouble with filenames containing \n and

  \r.  Do we really want to support this?  I think perhaps not.

* Unify smart_add and plain Branch.add(); perhaps smart_add should

  just build a list of files to add and pass that to the regular add

  function.

* Function to list a directory, saying in which revision each file was

  last modified.  Useful for web and GUI interfaces, and slow to

  compute one file at a time.

  This will be done when we track file texts by referring to the

  version that created them. 

* Check locking is correct during merge-related operations.

* Perhaps attempts to get locks should timeout after some period of

  time, or at least display a progress message.

* Don't pass around command classes but rather pass objects.  This'd

  make it cleaner to construct objects wrapping external commands.

* Track all merged-in revisions in a versioned add-only metafile.

* ``uncommit`` command that removes a revision from the end of the

  revision-history; just doing this is enough to remove the commit,

  and a new commit will automatically be made against the

  predecessor.  This can be repeated.

  It only makes sense to delete from the tail of history.

  This has been implemented, but it does not remove the texts from

  the store.

Large things

------------

* Generate annotations from current file relative to previous

  annotations.

  - Is it necessary to store any kind of annotation where data was

    deleted?

* Update revfile_ format and make it active:

  - Texts should be identified by something keyed on the revision, not

    an individual text-id.  This is much more useful for annotate I

    think; we want to map back to the revision that last changed it.

  - Access revfile revisions through the Tree/Store classes.

  - Check them from check commands.

  - Store annotations.

.. _revfile: revfile.html

* Hooks for pre-commit, post-commit, etc.

  Consider the security implications; probably should not enable hooks

  for remotely-fetched branches by default.

* Pre-commit check.  If this hook is defined, it needs to be handled

  specially: create a temporary directory containing the tree as it

  will be after the commit.  This means excluding any ignored/unknown

  files, and respecting selective commits.  Run the pre-commit check

  (e.g. compile and run test suite) in there.

  Possibly this should be done by splitting the commit function into

  several parts (under a single interface).  It is already rather

  large.  Decomposition:

   - find tree modifications and prepare in-memory inventory

   - export that inventory to a temporary directory

   - run the test in that temporary directory

   - if that succeeded, continue to actually finish the commit

  What should be done with the text of modified files while this is

  underway?  I don't think we want to count on holding them in memory

  and we can't trust the working files to stay in one place so I

  suppose we need to move them into the text store, or otherwise into

  a temporary directory.

  If the commit does not actually complete, we would rather the

  content was not left behind in the stores.

* Web interface

* GUI (maybe in Python GTK+?)

* C library interface

* Expansion of $Id$ keywords within working files.  Perhaps do this in

  exports first as a simpler case because then we don't need to deal

  with removing the tags on the way back in.

* ``bzr find``

Selftest

--------

* Write some benchmark tests

* we could display the ids of some tests selected by time to rune, or cputime,

  rusage. 

* We could run tests multiple times and report deviation/slowest/fastest.

* Gather lsprofile output for each test individually - log that somewhere.

* Should plugins be able to offer benchmark tests ?