Viewing all changes in revision 4634.71.1.
- Committer: John Arbash Meinel
- Date: 2009-09-09 18:52:56 UTC
- mto: (4634.52.16 2.0)
- mto: This revision was merged to the branch mainline in revision 4738.
- Revision ID: john@arbash-meinel.com-20090909185256-rdaxy872xauoem46
Work around bug #402623 by allowing BTreeGraphIndex(...,unlimited_cache=True).
The basic issue is that the access pattern for chk pages is fully random,
because the keys are 'sha1' handles. As such, we have no locality of
reference, and downloading a large project over HTTP can cause us to
redownload all of the .cix pages multiple times. The bug report
noticed the pages getting downloaded 4-5 times.
This was causing a significant increase in the total bytes downloaded.
(For Launchpad, downloading the 10MB cix file 5 times was 50MB, out of
around 160MB total download.)
The basic issue is that the access pattern for chk pages is fully random,
because the keys are 'sha1' handles. As such, we have no locality of
reference, and downloading a large project over HTTP can cause us to
redownload all of the .cix pages multiple times. The bug report
noticed the pages getting downloaded 4-5 times.
This was causing a significant increase in the total bytes downloaded.
(For Launchpad, downloading the 10MB cix file 5 times was 50MB, out of
around 160MB total download.)
expand all
collapse all
added
removed
