~bzr-pqm/bzr/bzr.dev : changes

~bzr-pqm/bzr/bzr.dev » Changes from revision 0.23.49

Rev	Summary	Authors	Date
0.23.9	We now basically have full support for using diff-delta as t... We now basically have full support for using diff-delta as the compressor. Will still need some tuning/tweaking to see how we want to proceed.	John Arbash Meinel	15 years ago
0.23.8	Add another test text. Add another test text.	John Arbash Meinel	15 years ago
0.23.7	Add a apply_delta2 function, just in case it matters. Add a apply_delta2 function, just in case it matters.	John Arbash Meinel	15 years ago
0.23.6	Start stripping out the actual GroupCompressor Start stripping out the actual GroupCompressor in preparation for using the diff-delta code. Add some tests that we can generate and apply diff deltas. We need to start adding some exceptions, and consider moving the core of the patch-delta loop back into a pure C function, as the generated code is very messy.	John Arbash Meinel	15 years ago
0.23.5	Minor changes to get diff-delta.c and patch-delta.c to compi... Minor changes to get diff-delta.c and patch-delta.c to compile. This includes bringing in 'delta.h'	John Arbash Meinel	15 years ago
0.23.4	Merge in the latest updates to the gc trunk. Merge in the latest updates to the gc trunk.	John Arbash Meinel	15 years ago
0.23.3	Fix a couple more locations. Fix a couple more locations.	John Arbash Meinel	15 years ago
0.23.2	Add the diff-delta.c and patch-delta.c files. Add the diff-delta.c and patch-delta.c files.	Nicolas Pitre	15 years ago
0.23.1	Start a quick experimentation with a different 'diff' algori... Start a quick experimentation with a different 'diff' algorithm.	John Arbash Meinel	15 years ago
0.20.20	Setting _fetch_order='topological' gives sub-optimal orderin... Setting _fetch_order='topological' gives sub-optimal ordering for gc=>gc fetches. This is because the 'autopack' code will convert to 'gc-optimal', which means that 'unordered' will then continue the 'gc-optimal' route.	John Arbash Meinel	15 years ago
0.20.19	Groupcompress now supports 'autopack' and 'pack'. Groupcompress now supports 'autopack' and 'pack'. It does this by just creating a new pack file, wrapping a GCVersionedFiles around it, and streaming in the data in 'gc-optimal' ordering. This actually seems to work fairly well.	John Arbash Meinel	15 years ago
0.20.18	Implement new handling of get_bytes_as(), and get_missing_co... Implement new handling of get_bytes_as(), and get_missing_compression_parent_keys() Now works on bzr.dev's new streaming code.	John Arbash Meinel	15 years ago
0.20.17	Fix the test suite now that we don't match short lines Fix the test suite now that we don't match short lines	John Arbash Meinel	15 years ago
0.20.16	Adding a 'soft' flag, to make the minimum match 200 bytes Adding a 'soft' flag, to make the minimum match 200 bytes comp time is 9m46s, comp size is improved across the board 11.3MB. So max group 8MB, max inter-file-id 4MB, 'soft' matching with a new file_id gives good compression at equivalent speed.	John Arbash Meinel	15 years ago
0.20.15	Change so that regions that have lots of copies get converte... Change so that regions that have lots of copies get converted back into an insertion. This does get triggered, but it doesn't help. The total compression is 17MB, and the conversion time is 10min. Which is equivalent to the original values. Even further, don't match blocks that are shorter than XX bytes (currently 10). With a value of 5, we still get trivial blocks inserted. With a value of 10, everything changes to copies. Dropping the max block size to 8MB decreases the total bytes to 14MB (presumably because the copy records now have 1 fewer byte per record). It also makes it 9m versus 10m. Preferentially splitting based on file-id (at >= 4MB) stays at 9min, but drops it to 13MB	John Arbash Meinel	15 years ago
0.20.14	Factor out _get_group_and_delta_lines. Factor out _get_group_and_delta_lines. The previous change (to ignore empty texts, and start new compressors) dropped the conversion time to 11m43s at a modest expansion to 13.4MB. The time difference is surprising, we should check if it is the no-newlines or the new-compressors. (my guess is the latter).	John Arbash Meinel	15 years ago
0.20.13	Play around a bit. Play around a bit. 1) Empty texts are no-op inserted, to avoid ever trying to match against their text. 2) If we find a new file-id and the compressor is more than half full, we go ahead and start a new compressor.	John Arbash Meinel	15 years ago
0.20.12	Change the code a little bit. Change the code a little bit. If a given text has not been seen before, insert all lines for that text. At present, we are doing worse than knit compression, because we have so many matching groups from various locations. Which causes us to just have huge swaths of copies. By inserting the full lines, we get more regions that we are able to generate a larger match against. This slows down the processing (10m => 24m), but improves compression (16MB => 12MB).	John Arbash Meinel	15 years ago
0.20.11	start experimenting with gc-optimal ordering. start experimenting with gc-optimal ordering.	John Arbash Meinel	15 years ago
0.20.10	Change the extraction ordering for 'unordered'. Change the extraction ordering for 'unordered'. Instead of using a random ordering, use the ordering defined by the index memos. This should give us the best group-locality. This gives a rather large performance improvement. Like 30s versus 10min.	John Arbash Meinel	15 years ago