30
30
down the track do not break new features or bug fixes that you are
31
31
contributing today.
33
As of May 2008, Bazaar ships with a test suite containing over 12000 tests
34
and growing. We are proud of it and want to remain so. As community
35
members, we all benefit from it. Would you trust version control on
36
your project to a product *without* a test suite like Bazaar has?
33
As of September 2009, Bazaar ships with a test suite containing over
34
23,000 tests and growing. We are proud of it and want to remain so. As
35
community members, we all benefit from it. Would you trust version control
36
on your project to a product *without* a test suite like Bazaar has?
39
39
Running the Test Suite
154
154
cmd_object.run() method directly. This is a lot faster than
155
155
subprocesses and generates the same logging output as running it in a
156
156
subprocess (which invoking the method directly does not).
158
158
3. Only test the one command in a single test script. Use the bzrlib
159
159
library when setting up tests and when evaluating the side-effects of
160
160
the command. We do this so that the library api has continual pressure
197
197
__ http://docs.python.org/lib/module-doctest.html
203
``bzrlib/tests/script.py`` allows users to write tests in a syntax very close to a shell session,
204
using a restricted and limited set of commands that should be enough to mimic
205
most of the behaviours.
207
A script is a set of commands, each command is composed of:
209
* one mandatory command line,
210
* one optional set of input lines to feed the command,
211
* one optional set of output expected lines,
212
* one optional set of error expected lines.
214
Input, output and error lines can be specified in any order.
216
Except for the expected output, all lines start with a special
217
string (based on their origin when used under a Unix shell):
219
* '$ ' for the command,
221
* nothing for output,
224
Comments can be added anywhere, they start with '#' and end with
227
The execution stops as soon as an expected output or an expected error is not
230
When no output is specified, any ouput from the command is accepted
231
and execution continue.
233
If an error occurs and no expected error is specified, the execution stops.
235
An error is defined by a returned status different from zero, not by the
236
presence of text on the error stream.
238
The matching is done on a full string comparison basis unless '...' is used, in
239
which case expected output/errors can be less precise.
243
The following will succeeds only if 'bzr add' outputs 'adding file'::
248
If you want the command to succeed for any output, just use::
252
The following will stop with an error::
256
If you want it to succeed, use::
259
2> bzr: ERROR: unknown command "not-a-command"
261
You can use ellipsis (...) to replace any piece of text you don't want to be
264
$ bzr branch not-a-branch
265
2>bzr: ERROR: Not a branch...not-a-branch/".
267
This can be used to ignore entire lines too::
273
# And here we explain that surprising fourth line
280
You can check the content of a file with cat::
285
You can also check the existence of a file with cat, the following will fail if
286
the file doesn't exist::
251
343
The test exists but is known to fail, for example this might be
252
344
appropriate to raise if you've committed a test for a bug but not
253
345
the fix for it, or if something works on Unix but not on Windows.
255
347
Raising this allows you to distinguish these failures from the
256
348
ones that are not expected to fail. If the test would fail
257
349
because of something we don't expect or intend to fix,
261
353
KnownFailure should be used with care as we don't want a
262
354
proliferation of quietly broken tests.
356
ModuleAvailableFeature
357
A helper for handling running tests based on whether a python
358
module is available. This can handle 3rd-party dependencies (is
359
``paramiko`` available?) as well as stdlib (``termios``) or
360
extension modules (``bzrlib._groupcompress_pyx``). You create a
361
new feature instance with::
363
MyModuleFeature = ModuleAvailableFeature('bzrlib.something')
366
def test_something(self):
367
self.requireFeature(MyModuleFeature)
368
something = MyModuleFeature.module
264
371
We plan to support three modes for running the test suite to control the
265
372
interpretation of these results. Strict mode is for use in situations
266
373
like merges to the mainline and releases where we want to make sure that
277
384
UnavailableFeature fail pass pass
278
385
KnownFailure fail pass pass
279
386
======================= ======= ======= ========
282
389
Test feature dependencies
283
390
-------------------------
added
removed
doc/developers/testing.txt
