~bzr-pqm/bzr/bzr.dev : changes

~bzr-pqm/bzr/bzr.dev » Changes from revision 3641.5.12

Rev	Summary	Authors	Date
3641.5.12	Play around with max_repack=0 and limiting work done based o... Play around with max_repack=0 and limiting work done based on the number of ZSYNC_FLUSH calls. We can shave 1s/7.5s by doing this, though we end up with probably a 20% increase in storage size (for the reasonable values, for Z_SYNC=0 we can get 3x storage size)	John Arbash Meinel	16 years ago
3641.5.11	Some small test cleanup Some small test cleanup	John Arbash Meinel	16 years ago
3641.5.10	Only Z_SYNC_FLUSH when we have extra bytes. Only Z_SYNC_FLUSH when we have extra bytes.	John Arbash Meinel	16 years ago
3641.5.9	Shrink the page size so that the three_level and iter_all Shrink the page size so that the three_level and iter_all tests don't have to use so many keys to get a three-level index.	John Arbash Meinel	16 years ago
3641.5.8	Fix up the test suite, now that we pack more efficiently. Fix up the test suite, now that we pack more efficiently. Unfortunately, that means upping the node count for the iter_all and three_level tests.	John Arbash Meinel	16 years ago
3641.5.7	Merge in bzr.dev to get the pyrex update Merge in bzr.dev to get the pyrex update	John Arbash Meinel	16 years ago
3641.5.6	Include the benchmarks for the new estimator. Include the benchmarks for the new estimator. All around, it seems to do a much better job. A bit faster, and a bit smaller.	John Arbash Meinel	16 years ago
3641.5.5	Document my attempt to use copy() as a look-ahead. Document my attempt to use copy() as a look-ahead.	John Arbash Meinel	16 years ago
3641.5.4	Using a different safety margin for the first repack, Using a different safety margin for the first repack, and using 2 repacks gives us effectively the same result, while still making it safe for arbitary data. (With 1-repack, it does effect the results 3-5%, and with 2-repacks the second margin gives the same results. Also, we now can get about 2-3:1 of lines that are 'blindly' added versus ones which are added with a SYNC.	John Arbash Meinel	16 years ago
3641.5.3	If we repack earlier, it catches this case. If we repack earlier, it catches this case. Still need to fix the other tests, but at least the too_much test passes now. Impact on real-world results is measurable (2-3% final compression). Is it worth it?	John Arbash Meinel	16 years ago
3641.5.2	(broken, but hopeful) Change the compact logic. (broken, but hopeful) Change the compact logic. Instead of only paying attention to the total bytes read, use the fact that we know some of that is already compressed well. So instead, we just pay attention to the bytes that are added since the last sync. This means we Z_SYNC_FLUSH much less often. (After syncing, we end up with more room to add without syncing.) This improves both the time to compress and the final compressed size. Need to update tests with the new offsets. Also, we seem to have found a case where using Z_SYNC_FLUSH in the middle of a stream will actually generate better compression than compressing the whole stream in one pass. test_too_much_data_does_not_exceed_size triggers this. It can be packed in more than 100 bytes less than the amount given by a full compress.	John Arbash Meinel	16 years ago
3641.5.1	Update the stats for the current code layout. Update the stats for the current code layout. This shows why I like _max_repack=2 so much. It is the highest value that has 'no waste'. At _max_repack=2, you can always sneak in 1 more line, which avoids triggering an extra repack. Also, updating the timings with the current tuning.	John Arbash Meinel	16 years ago
3641.3.31	NEWS NEWS	John Arbash Meinel	16 years ago
3641.3.30	Rename _parse_btree to _btree_serializer Rename _parse_btree to _btree_serializer	John Arbash Meinel	16 years ago
3641.3.29	Cleanup the copyright headers Cleanup the copyright headers	John Arbash Meinel	16 years ago
3641.3.28	Remove FixedMemoryGraphIndex Remove FixedMemoryGraphIndex	John Arbash Meinel	16 years ago
3641.3.27	Bringing reserved in as a keyword to write() also saves some... Bringing reserved in as a keyword to write() also saves some time. ChunkWriter.write() is called for every key, so having an extra redirection The was a measurable loss. Also, a small tuneup for the compressor that we access often. Unfortunately, there isn't much more tuning that we can do in that function.	John Arbash Meinel	16 years ago
3641.3.26	A couple small tweaks. A couple small tweaks.	John Arbash Meinel	16 years ago
3641.3.25	Shave off some more time by using exact accessors. Shave off some more time by using exact accessors. Basically, the inner access is using PyTuple and PyString functions instead of their PyObject counterparts.	John Arbash Meinel	16 years ago
3641.3.24	Use the compiled flatten function. Use the compiled flatten function. Shows a 3-5% gain in testing and real-world conditions.	John Arbash Meinel	16 years ago

Older »