~bzr-pqm/bzr/bzr.dev

.. -*- mode: indented-text; compile-command: "make -C doc" -*-


*******************
Things to do in bzr
*******************


See also various low-level TODOs in the source code.  Try looking in
the list archive or on gmane.org for previous discussion of these
issues.

These are classified by approximate size: an hour or less, a day or
less, and several days or more.


Small things
------------

* Fix tests so that import errors caused by modules don't produce false reports
  that the tests themselves don't exist.

* Fix tests so that one test failure doesn't prevent other tests from running

* print a message at the end of running the tests telling them that the
  test log and output exists but can be removed

* tests for running the commit editor, and fix problem of not passing in 
  multiple arguments

* Merging add of a new file clashing with an existing file doesn't
  work; add gets an error that it's already versioned and the merge
  aborts.

* Merge should ignore the destination's working directory, otherwise
  we get an error about the statcache when pulling from a remote
  branch.

* Add of a file that was present in the base revision should put back
  the previous file-id.

* Not sure I'm happy with needing to pass a root id to EmptyTree;
  comparing anything against an EmptyTree with no root should have the
  same effect(?)

* Handle diff of files which do not have a trailing newline; probably
  requires patching difflib to get it exactly right, or otherwise
  calling out to GNU diff.

* Should be able to copy files between branches to preserve their
  file-id (and perhaps eventually parentage.)

* -r option should take a revision-id as well as a revno.

* ``bzr info`` should count only people with distinct email addresses as
  different committers.  (Or perhaps only distinct userids?)

* On Windows, command-line arguments should be `glob-expanded`__,
  because the shell doesn't do this.  However, there are probably some
  commands where this shouldn't be done, such as 'bzr ignore', because
  we want to accept globs.

* ``bzr ignore`` command that just adds a line to the ``.bzrignore`` file
  and makes it versioned.  Fix this to break symlinks.

* Any useful sanity checks in 'bzr ignore'?  Perhaps give a warning if
  they try to add a single file which is already versioned, or if they
  add a pattern which already exists, or if it looks like they gave an
  unquoted glob.

__ http://mail.python.org/pipermail/python-list/2001-April/037847.html

* Separate read and write version checks?

* ``bzr status DIR`` should give status on all files under that
  directory.

* ``bzr log DIR`` should give changes to any files within DIR; at the
  moment it only lists things which modify the specific named file
  (and not its contents)

* ``bzr inventory -r REV`` and perhaps unify this with ``bzr ls``,
  giving options to display ids, types, etc.

* RemoteBranch could maintain a cache either in memory or on disk.  We
  know more than an external cache might about which files are
  immutable and which can vary.  On the other hand, it's much simpler
  to just use an external proxy cache.

  Perhaps ~/.bzr/http-cache.  Baz has a fairly simple cache under
  ~/.arch-cache, containing revision information encoded almost as a
  bunch of archives.  Perhaps we could simply store full paths.

* Maybe also store directories in the statcache so that we can quickly
  identify that they still exist.

* Diff should show timestamps; for files from the working directory we
  can use the file itself; for files from a revision we should use the
  commit time of the revision.

* Perhaps split command infrastructure from the actual command
  definitions.

* Cleaner support for negative boolean options like --no-recurse.

* Statcache should possibly map all file paths to / separators

* quotefn doubles all backslashes on Windows; this is probably not the
  best thing to do.  What would be a better way to safely represent
  filenames?  Perhaps we could doublequote things containing spaces,
  on the principle that filenames containing quotes are unlikely?
  Nice for humans; less good for machine parsing.

* Patches should probably use only forward slashes, even on Windows,
  otherwise Unix patch can't apply them. (?)

* Branch.update_revisions() inefficiently fetches revisions from the
  remote server twice; once to find out what text and inventory they
  need and then again to actually get the thing.  This is a bit
  inefficient.

  One complicating factor here is that we don't really want to have
  revisions present in the revision-store until all their constituent
  parts are also stored.

  The basic problem is that RemoteBranch.get_revision() and similar
  methods return object, but what we really want is the raw XML, which
  can be popped into our own store.  That needs to be refactored.

* Guard against repeatedly merging any particular patch.

* More options for diff:

  - diff two revisions of the same tree

  - diff two different branches, optionally at different revisions

  - diff a particular file in another tree against the corresponding
    version in this tree (which should be the default if the second
    parameter is a tree root)

  - diff everything under a particular directory, in any of the above
    ways

  - diff two files inside the same tree, even if they have different
    ids

  - and, of course, tests for all this

* Reproducible performance benchmark to measure whether performance is 
  getting better or worse.

* ``bzr log -m foo`` should perhaps error if nothing matches?

* ``bzr diff -r 30 -r 40 foo.c`` or ``bzr diff -r30..40 foo.c``

  If diffing between two branches then we probably want two -r
  options, since the revisions don't form a range that can be
  evaluated on either one.

* bzr diff shouldn't diff binary files

* setup.py install when run from a bzr tree should freeze the tree
  revision-id into the installed bzr.

* bzr script should trap ImportError and perhaps give a better error
  message?

* revert after a merge should possibly remove all the BASE/THIS/OTHER
  files to get you back to where you were.

* files that are added and then deleted are still reported as added

* stores should raise KeyError, not IndexError

* merging from a remote branch seems to sometimes raise errors not
  present locally

* should be possible to give a related branch when pulling from a
  remote branch to make things faster

* sometimes gives "conflicting add" even when the contents are in fact
  the same???

* BZRDIR should be in branch.py not __init__.py.

* bzrlib.delta.compare_trees() shouldn't need to sort all of the result
  lists, since they are being added in alphabetical order. Write tests
  to guarantee this, and remove the sort() calls.

Medium things
-------------

* merge should add all revision and inventory XML to the local store.

* check should give a warning for revisions that are named in the
  chain but not actually present in the store.

* remove anything outside of the branch implementation that directly
  accesses the stores.

* More efficient diff of only selected files.  We should be able to
  just get the id for the selected files, look up their location and
  diff just those files.  No need to traverse the entire inventories.

* Fix up Inventory objects to represent root object as an entry.

* Don't convert entire entry from ElementTree to an object when it is
  read in, but rather wait until the program actually wants to know
  about that node.

* Extract changes from one revision to the next to a text form
  suitable for transmission over email.

* More test cases.

  - ``missing`` command

  - Selected-file commit

  - Impossible selected-file commit: adding things in non-versioned
    directories, crossing renames, etc.

* Write a reproducible benchmark, perhaps importing various kernel versions.

* Directly import diffs!  It seems a bit redundant to need to rescan
  the directory to work out what files diff added/deleted/changed when
  all the information is there in the diff in the first place.
  Getting the exact behaviour for added/deleted subdirectories etc
  might be hard.

  At the very least we could run diffstat over the diff, or perhaps
  read the status output from patch.  Just knowing which files might
  be modified would be enough to guide the add and commit.

  Given this we might be able to import patches at 1/second or better.

* revfile compression.

* Split inventory into per-directory files?

* Fix ignore file parsing:

  - fnmatch is not the same as unix patterns

  - perhaps add extended globs from rsh/rsync

  - perhaps a pattern that matches only directories or non-directories

* Consider using Python logging library as well as/instead of
  bzrlib.trace.

* Commands should give some progress indication by default.

  - But quieten this with ``--silent``.

* Change to using gettext message localization.

* Make a clearer separation between internal and external bzrlib
  interfaces.  Make internal interfaces use protected names.  Write at
  least some documentation for those APIs, probably as docstrings.

  Consider using ZopeInterface definitions for the external interface;
  I think these are already used in PyBaz.  They allow automatic
  checking of the interface but may be unfamiliar to general Python
  developers, so I'm not really keen.

* Commands to dump out all command help into a manpage or HTML file or
  whatever.

* Handle symlinks in the working directory; at the very least it
  should be possible for them to be present and ignored/unknown
  without causing assertion failures.

  Eventually symlinks should be versioned.

* Allow init in a subdirectory to create a nested repository, but only
  if the subdirectory is not already versioned.   Perhaps also require
  a ``--nested`` to protect against confusion.

* Branch names?

* More test framework:

  - Class that describes the state of a working tree so we can just
    assert it's equal.

* Try using XSLT to add some formatting to REST-generated HTML.  Or
  maybe write a small Python program that specifies a header and foot
  for the pages and calls into the docutils libraries.

* --format=xml for log, status and other commands.

* Attempting to explicitly add a file that's already added should give
  a warning; however there should be no warning for directories (since
  we scan for new children) or files encountered in a directory that's
  being scanned.

* Better handling of possible collisions on case-losing filesystems;
  make sure a single file does not get added twice under different
  names.

* Clean up XML inventory:

  - Use nesting rather than parent_id pointers.

  - Hold the ElementTree in memory in the Inventory object and work
    directly on that, rather than converting into Python objects every
    time it is read in.  Probably still expose it through some kind of
    object interface though, but perhaps that should just be a proxy
    for the elements.

  - Less special cases for the root directory.

* Perhaps inventories should remember the revision in which each file
  was last changed, as well as its current state?  This is a bit
  redundant but might often be interested to know.

* stat cache should perhaps only stat files as necessary, rather than
  doing them all up-front.  On the other hand, that disallows the
  optimization of stating them in inode order.

* It'd be nice to pipeline multiple HTTP requests.  Often we can
  predict what will be wanted in future: all revisions, or all texts
  in a particular revision, etc.

  urlgrabber's docs say they are working on batched downloads; we
  could perhaps ride on that or just create a background thread (ew).

* Paranoid mode where we never trust SHA-1 matches.

* --dry-run mode for commit?  (Or maybe just run with
  check-command=false?)

* Generally, be a bit more verbose unless --silent is specified.

* Function that finds all changes to files under a given directory;
  perhaps log should use this if a directory is given.

* XML attributes might have trouble with filenames containing \n and
  \r.  Do we really want to support this?  I think perhaps not.

* Unify smart_add and plain Branch.add(); perhaps smart_add should
  just build a list of files to add and pass that to the regular add
  function.

* Function to list a directory, saying in which revision each file was
  last modified.  Useful for web and GUI interfaces, and slow to
  compute one file at a time.
  
  This will be done when we track file texts by referring to the
  version that created them. 

* Check locking is correct during merge-related operations.

* Perhaps attempts to get locks should timeout after some period of
  time, or at least display a progress message.

* Don't pass around command classes but rather pass objects.  This'd
  make it cleaner to construct objects wrapping external commands.

* Track all merged-in revisions in a versioned add-only metafile.

* ``uncommit`` command that removes a revision from the end of the
  revision-history; just doing this is enough to remove the commit,
  and a new commit will automatically be made against the
  predecessor.  This can be repeated.

  It only makes sense to delete from the tail of history.

  This has been implemented, but it does not remove the texts from
  the store.


Large things
------------

* Generate annotations from current file relative to previous
  annotations.

  - Is it necessary to store any kind of annotation where data was
    deleted?

* Update revfile_ format and make it active:

  - Texts should be identified by something keyed on the revision, not
    an individual text-id.  This is much more useful for annotate I
    think; we want to map back to the revision that last changed it.

  - Access revfile revisions through the Tree/Store classes.

  - Check them from check commands.

  - Store annotations.

.. _revfile: revfile.html

* Hooks for pre-commit, post-commit, etc.

  Consider the security implications; probably should not enable hooks
  for remotely-fetched branches by default.

* Pre-commit check.  If this hook is defined, it needs to be handled
  specially: create a temporary directory containing the tree as it
  will be after the commit.  This means excluding any ignored/unknown
  files, and respecting selective commits.  Run the pre-commit check
  (e.g. compile and run test suite) in there.

  Possibly this should be done by splitting the commit function into
  several parts (under a single interface).  It is already rather
  large.  Decomposition:

   - find tree modifications and prepare in-memory inventory

   - export that inventory to a temporary directory

   - run the test in that temporary directory

   - if that succeeded, continue to actually finish the commit

  What should be done with the text of modified files while this is
  underway?  I don't think we want to count on holding them in memory
  and we can't trust the working files to stay in one place so I
  suppose we need to move them into the text store, or otherwise into
  a temporary directory.

  If the commit does not actually complete, we would rather the
  content was not left behind in the stores.

* Web interface

* GUI (maybe in Python GTK+?)

* C library interface

* Expansion of $Id$ keywords within working files.  Perhaps do this in
  exports first as a simpler case because then we don't need to deal
  with removing the tags on the way back in.

* ``bzr find``

Selftest
--------

* Write some benchmark tests

* we could display the ids of some tests selected by time to rune, or cputime,
  rusage. 

* We could run tests multiple times and report deviation/slowest/fastest.

* Gather lsprofile output for each test individually - log that somewhere.

* Should plugins be able to offer benchmark tests ?