« back to all changes in this revision
Viewing changes to bzrlib/transport/smart.py
- browse files at revision 1997.1.6
- compare with another revision
- compare with revision 6486
- download diff
- download tarball
- view history from revision 1997.1.6
- Committer: Robert Collins
- Date: 2006-09-17 21:03:04 UTC
- mfrom: (2018 +trunk)
- mto: This revision was merged to the branch mainline in revision 2019.
- Revision ID: robertc@robertcollins.net-20060917210304-3a697132f5fb68ac
Merge bzr.dev.
- files added:
- bzrlib/inspect_for_copy.py
- files modified:
- BRANCH.TODO
added
removed
bzrlib/transport/smart.py
1
# Copyright (C) 2006 Canonical Ltd
2
#
3
# This program is free software; you can redistribute it and/or modify
4
# it under the terms of the GNU General Public License as published by
5
# the Free Software Foundation; either version 2 of the License, or
6
# (at your option) any later version.
7
#
8
# This program is distributed in the hope that it will be useful,
9
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11
# GNU General Public License for more details.
12
#
13
# You should have received a copy of the GNU General Public License
14
# along with this program; if not, write to the Free Software
15
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
16
17
"""Smart-server protocol, client and server.
18
19
Requests are sent as a command and list of arguments, followed by optional
20
bulk body data. Responses are similarly a response and list of arguments,
21
followed by bulk body data. ::
22
23
SEP := '\001'
24
Fields are separated by Ctrl-A.
25
BULK_DATA := CHUNK+ TRAILER
26
Chunks can be repeated as many times as necessary.
27
CHUNK := CHUNK_LEN CHUNK_BODY
28
CHUNK_LEN := DIGIT+ NEWLINE
29
Gives the number of bytes in the following chunk.
30
CHUNK_BODY := BYTE[chunk_len]
31
TRAILER := SUCCESS_TRAILER | ERROR_TRAILER
32
SUCCESS_TRAILER := 'done' NEWLINE
33
ERROR_TRAILER :=
34
35
Paths are passed across the network. The client needs to see a namespace that
36
includes any repository that might need to be referenced, and the client needs
37
to know about a root directory beyond which it cannot ascend.
38
39
Servers run over ssh will typically want to be able to access any path the user
40
can access. Public servers on the other hand (which might be over http, ssh
41
or tcp) will typically want to restrict access to only a particular directory
42
and its children, so will want to do a software virtual root at that level.
43
In other words they'll want to rewrite incoming paths to be under that level
44
(and prevent escaping using ../ tricks.)
45
46
URLs that include ~ should probably be passed across to the server verbatim
47
and the server can expand them. This will proably not be meaningful when
48
limited to a directory?
49
"""
50
51
52
53
# TODO: A plain integer from query_version is too simple; should give some
54
# capabilities too?
55
56
# TODO: Server should probably catch exceptions within itself and send them
57
# back across the network. (But shouldn't catch KeyboardInterrupt etc)
58
# Also needs to somehow report protocol errors like bad requests. Need to
59
# consider how we'll handle error reporting, e.g. if we get halfway through a
60
# bulk transfer and then something goes wrong.
61
62
# TODO: Standard marker at start of request/response lines?
63
64
# TODO: Make each request and response self-validatable, e.g. with checksums.
65
#
66
# TODO: get/put objects could be changed to gradually read back the data as it
67
# comes across the network
68
#
69
# TODO: What should the server do if it hits an error and has to terminate?
70
#
71
# TODO: is it useful to allow multiple chunks in the bulk data?
72
#
73
# TODO: If we get an exception during transmission of bulk data we can't just
74
# emit the exception because it won't be seen.
75
# John proposes: I think it would be worthwhile to have a header on each
76
# chunk, that indicates it is another chunk. Then you can send an 'error'
77
# chunk as long as you finish the previous chunk.
78
#
79
# TODO: Clone method on Transport; should work up towards parent directory;
80
# unclear how this should be stored or communicated to the server... maybe
81
# just pass it on all relevant requests?
82
#
83
# TODO: Better name than clone() for changing between directories. How about
84
# open_dir or change_dir or chdir?
85
#
86
# TODO: Is it really good to have the notion of current directory within the
87
# connection? Perhaps all Transports should factor out a common connection
88
# from the thing that has the directory context?
89
#
90
# TODO: Pull more things common to sftp and ssh to a higher level.
91
#
92
# TODO: The server that manages a connection should be quite small and retain
93
# minimum state because each of the requests are supposed to be stateless.
94
# Then we can write another implementation that maps to http.
95
#
96
# TODO: What to do when a client connection is garbage collected? Maybe just
97
# abruptly drop the connection?
98
#
99
# TODO: Server in some cases will need to restrict access to files outside of
100
# a particular root directory. LocalTransport doesn't do anything to stop you
101
# ascending above the base directory, so we need to prevent paths
102
# containing '..' in either the server or transport layers. (Also need to
103
# consider what happens if someone creates a symlink pointing outside the
104
# directory tree...)
105
#
106
# TODO: Server should rebase absolute paths coming across the network to put
107
# them under the virtual root, if one is in use. LocalTransport currently
108
# doesn't do that; if you give it an absolute path it just uses it.
109
#
110
# XXX: Arguments can't contain newlines or ascii; possibly we should e.g.
111
# urlescape them instead. Indeed possibly this should just literally be
112
# http-over-ssh.
113
#
114
# FIXME: This transport, with several others, has imperfect handling of paths
115
# within urls. It'd probably be better for ".." from a root to raise an error
116
# rather than return the same directory as we do at present.
117
#
118
# TODO: Rather than working at the Transport layer we want a Branch,
119
# Repository or BzrDir objects that talk to a server.
120
#
121
# TODO: Probably want some way for server commands to gradually produce body
122
# data rather than passing it as a string; they could perhaps pass an
123
# iterator-like callback that will gradually yield data; it probably needs a
124
# close() method that will always be closed to do any necessary cleanup.
125
#
126
# TODO: Split the actual smart server from the ssh encoding of it.
127
#
128
# TODO: Perhaps support file-level readwrite operations over the transport
129
# too.
130
#
131
# TODO: SmartBzrDir class, proxying all Branch etc methods across to another
132
# branch doing file-level operations.
133
#
134
# TODO: jam 20060915 _decode_tuple is acting directly on input over
135
# the socket, and it assumes everything is UTF8 sections separated
136
# by \001. Which means a request like '\002' Will abort the connection
137
# because of a UnicodeDecodeError. It does look like invalid data will
138
# kill the SmartStreamServer, but only with an abort + exception, and
139
# the overall server shouldn't die.
140
141
from cStringIO import StringIO
142
import errno
143
import os
144
import socket
145
import sys
146
import tempfile
147
import threading
148
import urllib
149
import urlparse
150
151
from bzrlib import (
152
bzrdir,
153
errors,
154
revision,
155
transport,
156
trace,
157
urlutils,
158
)
159
from bzrlib.bundle.serializer import write_bundle
160
from bzrlib.trace import mutter
161
from bzrlib.transport import local
162
163
# must do this otherwise urllib can't parse the urls properly :(
164
for scheme in ['ssh', 'bzr', 'bzr+loopback', 'bzr+ssh']:
165
transport.register_urlparse_netloc_protocol(scheme)
166
del scheme
167
168
169
def _recv_tuple(from_file):
170
req_line = from_file.readline()
171
return _decode_tuple(req_line)
172
173
174
def _decode_tuple(req_line):
175
if req_line == None or req_line == '':
176
return None
177
if req_line[-1] != '\n':
178
raise errors.SmartProtocolError("request %r not terminated" % req_line)
179
return tuple((a.decode('utf-8') for a in req_line[:-1].split('\x01')))
180
181
182
def _send_tuple(to_file, args):
183
# XXX: this will be inefficient. Just ask Robert.
184
to_file.write('\x01'.join((a.encode('utf-8') for a in args)) + '\n')
185
to_file.flush()
186
187
188
class SmartProtocolBase(object):
189
"""Methods common to client and server"""
190
191
def _send_bulk_data(self, body):
192
"""Send chunked body data"""
193
assert isinstance(body, str)
194
self._out.write('%d\n' % len(body))
195
self._out.write(body)
196
self._out.write('done\n')
197
self._out.flush()
198
199
# TODO: this only actually accomodates a single block; possibly should support
200
# multiple chunks?
201
def _recv_bulk(self):
202
chunk_len = self._in.readline()
203
try:
204
chunk_len = int(chunk_len)
205
except ValueError:
206
raise errors.SmartProtocolError("bad chunk length line %r" % chunk_len)
207
bulk = self._in.read(chunk_len)
208
if len(bulk) != chunk_len:
209
raise errors.SmartProtocolError("short read fetching bulk data chunk")
210
self._recv_trailer()
211
return bulk
212
213
def _recv_tuple(self):
214
return _recv_tuple(self._in)
215
216
def _recv_trailer(self):
217
resp = self._recv_tuple()
218
if resp == ('done', ):
219
return
220
else:
221
self._translate_error(resp)
222
223
224
class SmartStreamServer(SmartProtocolBase):
225
"""Handles smart commands coming over a stream.
226
227
The stream may be a pipe connected to sshd, or a tcp socket, or an
228
in-process fifo for testing.
229
230
One instance is created for each connected client; it can serve multiple
231
requests in the lifetime of the connection.
232
233
The server passes requests through to an underlying backing transport,
234
which will typically be a LocalTransport looking at the server's filesystem.
235
"""
236
237
def __init__(self, in_file, out_file, backing_transport):
238
"""Construct new server.
239
240
:param in_file: Python file from which requests can be read.
241
:param out_file: Python file to write responses.
242
:param backing_transport: Transport for the directory served.
243
"""
244
self._in = in_file
245
self._out = out_file
246
self.smart_server = SmartServer(backing_transport)
247
# server can call back to us to get bulk data - this is not really
248
# ideal, they should get it per request instead
249
self.smart_server._recv_body = self._recv_bulk
250
251
def _recv_tuple(self):
252
"""Read a request from the client and return as a tuple.
253
254
Returns None at end of file (if the client closed the connection.)
255
"""
256
return _recv_tuple(self._in)
257
258
def _send_tuple(self, args):
259
"""Send response header"""
260
return _send_tuple(self._out, args)
261
262
def _send_error_and_disconnect(self, exception):
263
self._send_tuple(('error', str(exception)))
264
self._out.flush()
265
## self._out.close()
266
## self._in.close()
267
268
def _serve_one_request(self):
269
"""Read one request from input, process, send back a response.
270
271
:return: False if the server should terminate, otherwise None.
272
"""
273
req_args = self._recv_tuple()
274
if req_args == None:
275
# client closed connection
276
return False # shutdown server
277
try:
278
response = self.smart_server.dispatch_command(req_args[0], req_args[1:])
279
self._send_tuple(response.args)
280
if response.body is not None:
281
self._send_bulk_data(response.body)
282
except KeyboardInterrupt:
283
raise
284
except Exception, e:
285
# everything else: pass to client, flush, and quit
286
self._send_error_and_disconnect(e)
287
return False
288
289
def serve(self):
290
"""Serve requests until the client disconnects."""
291
# Keep a reference to stderr because the sys module's globals get set to
292
# None during interpreter shutdown.
293
from sys import stderr
294
try:
295
while self._serve_one_request() != False:
296
pass
297
except Exception, e:
298
stderr.write("%s terminating on exception %s\n" % (self, e))
299
raise
300
301
302
class SmartServerResponse(object):
303
"""Response generated by SmartServer."""
304
305
def __init__(self, args, body=None):
306
self.args = args
307
self.body = body
308
309
310
class SmartServer(object):
311
"""Protocol logic for smart server.
312
313
This doesn't handle serialization at all, it just processes requests and
314
creates responses.
315
"""
316
317
# TODO: Better way of representing the body for commands that take it,
318
# and allow it to be streamed into the server.
319
320
def __init__(self, backing_transport):
321
self._backing_transport = backing_transport
322
323
def do_hello(self):
324
"""Answer a version request with my version."""
325
return SmartServerResponse(('ok', '1'))
326
327
def do_has(self, relpath):
328
r = self._backing_transport.has(relpath) and 'yes' or 'no'
329
return SmartServerResponse((r,))
330
331
def do_get(self, relpath):
332
backing_bytes = self._backing_transport.get_bytes(relpath)
333
return SmartServerResponse(('ok',), backing_bytes)
334
335
def _deserialise_optional_mode(self, mode):
336
if mode == '':
337
return None
338
else:
339
return int(mode)
340
341
def do_append(self, relpath, mode):
342
old_length = self._backing_transport.append_bytes(
343
relpath, self._recv_body(), self._deserialise_optional_mode(mode))
344
return SmartServerResponse(('appended', '%d' % old_length))
345
346
def do_delete(self, relpath):
347
self._backing_transport.delete(relpath)
348
349
def do_iter_files_recursive(self, abspath):
350
# XXX: the path handling needs some thought.
351
#relpath = self._backing_transport.relpath(abspath)
352
transport = self._backing_transport.clone(abspath)
353
filenames = transport.iter_files_recursive()
354
return SmartServerResponse(('names',) + tuple(filenames))
355
356
def do_list_dir(self, relpath):
357
filenames = self._backing_transport.list_dir(relpath)
358
return SmartServerResponse(('names',) + tuple(filenames))
359
360
def do_mkdir(self, relpath, mode):
361
self._backing_transport.mkdir(relpath,
362
self._deserialise_optional_mode(mode))
363
364
def do_move(self, rel_from, rel_to):
365
self._backing_transport.move(rel_from, rel_to)
366
367
def do_put(self, relpath, mode):
368
self._backing_transport.put_bytes(relpath,
369
self._recv_body(),
370
self._deserialise_optional_mode(mode))
371
372
def do_rename(self, rel_from, rel_to):
373
self._backing_transport.rename(rel_from, rel_to)
374
375
def do_rmdir(self, relpath):
376
self._backing_transport.rmdir(relpath)
377
378
def do_stat(self, relpath):
379
stat = self._backing_transport.stat(relpath)
380
return SmartServerResponse(('stat', str(stat.st_size), oct(stat.st_mode)))
381
382
def do_get_bundle(self, path, revision_id):
383
# open transport relative to our base
384
t = self._backing_transport.clone(path)
385
control, extra_path = bzrdir.BzrDir.open_containing_from_transport(t)
386
repo = control.open_repository()
387
tmpf = tempfile.TemporaryFile()
388
base_revision = revision.NULL_REVISION
389
write_bundle(repo, revision_id, base_revision, tmpf)
390
tmpf.seek(0)
391
return SmartServerResponse((), tmpf.read())
392
393
def dispatch_command(self, cmd, args):
394
func = getattr(self, 'do_' + cmd, None)
395
if func is None:
396
raise errors.SmartProtocolError("bad request %r" % (cmd,))
397
try:
398
result = func(*args)
399
if result is None:
400
result = SmartServerResponse(('ok',))
401
return result
402
except errors.NoSuchFile, e:
403
return SmartServerResponse(('NoSuchFile', e.path))
404
except errors.FileExists, e:
405
return SmartServerResponse(('FileExists', e.path))
406
except errors.DirectoryNotEmpty, e:
407
return SmartServerResponse(('DirectoryNotEmpty', e.path))
408
except UnicodeError, e:
409
# If it is a DecodeError, than most likely we are starting
410
# with a plain string
411
str_or_unicode = e.object
412
if isinstance(str_or_unicode, unicode):
413
val = u'u:' + str_or_unicode
414
else:
415
val = u's:' + str_or_unicode.encode('base64')
416
# This handles UnicodeEncodeError or UnicodeDecodeError
417
return SmartServerResponse((e.__class__.__name__,
418
e.encoding, val, str(e.start), str(e.end), e.reason))
419
420
421
class SmartTCPServer(object):
422
"""Listens on a TCP socket and accepts connections from smart clients"""
423
424
def __init__(self, backing_transport=None, host='127.0.0.1', port=0):
425
"""Construct a new server.
426
427
To actually start it running, call either start_background_thread or
428
serve.
429
430
:param host: Name of the interface to listen on.
431
:param port: TCP port to listen on, or 0 to allocate a transient port.
432
"""
433
if backing_transport is None:
434
backing_transport = memory.MemoryTransport()
435
self._server_socket = socket.socket()
436
self._server_socket.bind((host, port))
437
self.port = self._server_socket.getsockname()[1]
438
self._server_socket.listen(1)
439
self._server_socket.settimeout(1)
440
self.backing_transport = backing_transport
441
442
def serve(self):
443
# let connections timeout so that we get a chance to terminate
444
# Keep a reference to the exceptions we want to catch because the socket
445
# module's globals get set to None during interpreter shutdown.
446
from socket import timeout as socket_timeout
447
from socket import error as socket_error
448
self._should_terminate = False
449
while not self._should_terminate:
450
try:
451
self.accept_and_serve()
452
except socket_timeout:
453
# just check if we're asked to stop
454
pass
455
except socket_error, e:
456
trace.warning("client disconnected: %s", e)
457
pass
458
459
def get_url(self):
460
"""Return the url of the server"""
461
return "bzr://%s:%d/" % self._server_socket.getsockname()
462
463
def accept_and_serve(self):
464
conn, client_addr = self._server_socket.accept()
465
conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
466
from_client = conn.makefile('r')
467
to_client = conn.makefile('w')
468
handler = SmartStreamServer(from_client, to_client,
469
self.backing_transport)
470
connection_thread = threading.Thread(None, handler.serve, name='smart-server-child')
471
connection_thread.setDaemon(True)
472
connection_thread.start()
473
474
def start_background_thread(self):
475
self._server_thread = threading.Thread(None,
476
self.serve,
477
name='server-' + self.get_url())
478
self._server_thread.setDaemon(True)
479
self._server_thread.start()
480
481
def stop_background_thread(self):
482
self._should_terminate = True
483
# self._server_socket.close()
484
# we used to join the thread, but it's not really necessary; it will
485
# terminate in time
486
## self._server_thread.join()
487
488
489
class SmartTCPServer_for_testing(SmartTCPServer):
490
"""Server suitable for use by transport tests.
491
492
This server is backed by the process's cwd.
493
"""
494
495
def __init__(self):
496
self._homedir = os.getcwd()
497
# The server is set up by default like for ssh access: the client
498
# passes filesystem-absolute paths; therefore the server must look
499
# them up relative to the root directory. it might be better to act
500
# a public server and have the server rewrite paths into the test
501
# directory.
502
SmartTCPServer.__init__(self, transport.get_transport("file:///"))
503
504
def setUp(self):
505
"""Set up server for testing"""
506
self.start_background_thread()
507
508
def tearDown(self):
509
self.stop_background_thread()
510
511
def get_url(self):
512
"""Return the url of the server"""
513
host, port = self._server_socket.getsockname()
514
# XXX: I think this is likely to break on windows -- self._homedir will
515
# have backslashes (and maybe a drive letter?).
516
# -- Andrew Bennetts, 2006-08-29
517
return "bzr://%s:%d%s" % (host, port, urlutils.escape(self._homedir))
518
519
def get_bogus_url(self):
520
"""Return a URL which will fail to connect"""
521
return 'bzr://127.0.0.1:1/'
522
523
524
class SmartStat(object):
525
526
def __init__(self, size, mode):
527
self.st_size = size
528
self.st_mode = mode
529
530
531
class SmartTransport(transport.Transport):
532
"""Connection to a smart server.
533
534
The connection holds references to pipes that can be used to send requests
535
to the server.
536
537
The connection has a notion of the current directory to which it's
538
connected; this is incorporated in filenames passed to the server.
539
540
This supports some higher-level RPC operations and can also be treated
541
like a Transport to do file-like operations.
542
543
The connection can be made over a tcp socket, or (in future) an ssh pipe
544
or a series of http requests. There are concrete subclasses for each
545
type: SmartTCPTransport, etc.
546
"""
547
548
def __init__(self, url, clone_from=None, client=None):
549
"""Constructor.
550
551
:param client: ignored when clone_from is not None.
552
"""
553
### Technically super() here is faulty because Transport's __init__
554
### fails to take 2 parameters, and if super were to choose a silly
555
### initialisation order things would blow up.
556
if not url.endswith('/'):
557
url += '/'
558
super(SmartTransport, self).__init__(url)
559
self._scheme, self._username, self._password, self._host, self._port, self._path = \
560
transport.split_url(url)
561
if clone_from is None:
562
if client is None:
563
self._client = SmartStreamClient(self._connect_to_server)
564
else:
565
self._client = client
566
else:
567
# credentials may be stripped from the base in some circumstances
568
# as yet to be clearly defined or documented, so copy them.
569
self._username = clone_from._username
570
# reuse same connection
571
self._client = clone_from._client
572
573
def abspath(self, relpath):
574
"""Return the full url to the given relative path.
575
576
@param relpath: the relative path or path components
577
@type relpath: str or list
578
"""
579
return self._unparse_url(self._remote_path(relpath))
580
581
def clone(self, relative_url):
582
"""Make a new SmartTransport related to me, sharing the same connection.
583
584
This essentially opens a handle on a different remote directory.
585
"""
586
if relative_url is None:
587
return self.__class__(self.base, self)
588
else:
589
return self.__class__(self.abspath(relative_url), self)
590
591
def is_readonly(self):
592
"""Smart server transport can do read/write file operations."""
593
return False
594
595
def get_smart_client(self):
596
return self._client
597
598
def _unparse_url(self, path):
599
"""Return URL for a path.
600
601
:see: SFTPUrlHandling._unparse_url
602
"""
603
# TODO: Eventually it should be possible to unify this with
604
# SFTPUrlHandling._unparse_url?
605
if path == '':
606
path = '/'
607
path = urllib.quote(path)
608
netloc = urllib.quote(self._host)
609
if self._username is not None:
610
netloc = '%s@%s' % (urllib.quote(self._username), netloc)
611
if self._port is not None:
612
netloc = '%s:%d' % (netloc, self._port)
613
return urlparse.urlunparse((self._scheme, netloc, path, '', '', ''))
614
615
def _remote_path(self, relpath):
616
"""Returns the Unicode version of the absolute path for relpath."""
617
return self._combine_paths(self._path, relpath)
618
619
def has(self, relpath):
620
"""Indicate whether a remote file of the given name exists or not.
621
622
:see: Transport.has()
623
"""
624
resp = self._client._call('has', self._remote_path(relpath))
625
if resp == ('yes', ):
626
return True
627
elif resp == ('no', ):
628
return False
629
else:
630
self._translate_error(resp)
631
632
def get(self, relpath):
633
"""Return file-like object reading the contents of a remote file.
634
635
:see: Transport.get_bytes()/get_file()
636
"""
637
remote = self._remote_path(relpath)
638
resp = self._client._call('get', remote)
639
if resp != ('ok', ):
640
self._translate_error(resp, relpath)
641
return StringIO(self._client._recv_bulk())
642
643
def _serialise_optional_mode(self, mode):
644
if mode is None:
645
return ''
646
else:
647
return '%d' % mode
648
649
def mkdir(self, relpath, mode=None):
650
resp = self._client._call('mkdir',
651
self._remote_path(relpath),
652
self._serialise_optional_mode(mode))
653
self._translate_error(resp)
654
655
def put_file(self, relpath, upload_file, mode=None):
656
# its not ideal to seek back, but currently put_non_atomic_file depends
657
# on transports not reading before failing - which is a faulty
658
# assumption I think - RBC 20060915
659
pos = upload_file.tell()
660
try:
661
return self.put_bytes(relpath, upload_file.read(), mode)
662
except:
663
upload_file.seek(pos)
664
raise
665
666
def put_bytes(self, relpath, upload_contents, mode=None):
667
# FIXME: upload_file is probably not safe for non-ascii characters -
668
# should probably just pass all parameters as length-delimited
669
# strings?
670
resp = self._client._call_with_upload(
671
'put',
672
(self._remote_path(relpath), self._serialise_optional_mode(mode)),
673
upload_contents)
674
self._translate_error(resp)
675
676
def append_file(self, relpath, from_file, mode=None):
677
return self.append_bytes(relpath, from_file.read(), mode)
678
679
def append_bytes(self, relpath, bytes, mode=None):
680
resp = self._client._call_with_upload(
681
'append',
682
(self._remote_path(relpath), self._serialise_optional_mode(mode)),
683
bytes)
684
if resp[0] == 'appended':
685
return int(resp[1])
686
self._translate_error(resp)
687
688
def delete(self, relpath):
689
resp = self._client._call('delete', self._remote_path(relpath))
690
self._translate_error(resp)
691
692
def rename(self, rel_from, rel_to):
693
self._call('rename',
694
self._remote_path(rel_from),
695
self._remote_path(rel_to))
696
697
def move(self, rel_from, rel_to):
698
self._call('move',
699
self._remote_path(rel_from),
700
self._remote_path(rel_to))
701
702
def rmdir(self, relpath):
703
resp = self._call('rmdir', self._remote_path(relpath))
704
705
def _call(self, method, *args):
706
resp = self._client._call(method, *args)
707
self._translate_error(resp)
708
709
def _translate_error(self, resp, orig_path=None):
710
"""Raise an exception from a response"""
711
what = resp[0]
712
if what == 'ok':
713
return
714
elif what == 'NoSuchFile':
715
if orig_path is not None:
716
error_path = orig_path
717
else:
718
error_path = resp[1]
719
raise errors.NoSuchFile(error_path)
720
elif what == 'error':
721
raise errors.SmartProtocolError(unicode(resp[1]))
722
elif what == 'FileExists':
723
raise errors.FileExists(resp[1])
724
elif what == 'DirectoryNotEmpty':
725
raise errors.DirectoryNotEmpty(resp[1])
726
elif what in ('UnicodeEncodeError', 'UnicodeDecodeError'):
727
encoding = str(resp[1]) # encoding must always be a string
728
val = resp[2]
729
start = int(resp[3])
730
end = int(resp[4])
731
reason = str(resp[5]) # reason must always be a string
732
if val.startswith('u:'):
733
val = val[2:]
734
elif val.startswith('s:'):
735
val = val[2:].decode('base64')
736
if what == 'UnicodeDecodeError':
737
raise UnicodeDecodeError(encoding, val, start, end, reason)
738
elif what == 'UnicodeEncodeError':
739
raise UnicodeEncodeError(encoding, val, start, end, reason)
740
else:
741
raise errors.SmartProtocolError('unexpected smart server error: %r' % (resp,))
742
743
def _send_tuple(self, args):
744
self._client._send_tuple(args)
745
746
def _recv_tuple(self):
747
return self._client._recv_tuple()
748
749
def disconnect(self):
750
self._client.disconnect()
751
752
def delete_tree(self, relpath):
753
raise errors.TransportNotPossible('readonly transport')
754
755
def stat(self, relpath):
756
resp = self._client._call('stat', self._remote_path(relpath))
757
if resp[0] == 'stat':
758
return SmartStat(int(resp[1]), int(resp[2], 8))
759
else:
760
self._translate_error(resp)
761
762
## def lock_read(self, relpath):
763
## """Lock the given file for shared (read) access.
764
## :return: A lock object, which should be passed to Transport.unlock()
765
## """
766
## # The old RemoteBranch ignore lock for reading, so we will
767
## # continue that tradition and return a bogus lock object.
768
## class BogusLock(object):
769
## def __init__(self, path):
770
## self.path = path
771
## def unlock(self):
772
## pass
773
## return BogusLock(relpath)
774
775
def listable(self):
776
return True
777
778
def list_dir(self, relpath):
779
resp = self._client._call('list_dir',
780
self._remote_path(relpath))
781
if resp[0] == 'names':
782
return [name.encode('ascii') for name in resp[1:]]
783
else:
784
self._translate_error(resp)
785
786
def iter_files_recursive(self):
787
resp = self._client._call('iter_files_recursive',
788
self._remote_path(''))
789
if resp[0] == 'names':
790
return resp[1:]
791
else:
792
self._translate_error(resp)
793
794
795
class SmartStreamClient(SmartProtocolBase):
796
"""Connection to smart server over two streams"""
797
798
def __init__(self, connect_func):
799
self._connect_func = connect_func
800
self._connected = False
801
802
def __del__(self):
803
self.disconnect()
804
805
def _ensure_connection(self):
806
if not self._connected:
807
self._in, self._out = self._connect_func()
808
self._connected = True
809
810
def _send_tuple(self, args):
811
self._ensure_connection()
812
_send_tuple(self._out, args)
813
814
def _send_bulk_data(self, body):
815
self._ensure_connection()
816
SmartProtocolBase._send_bulk_data(self, body)
817
818
def _recv_bulk(self):
819
self._ensure_connection()
820
return SmartProtocolBase._recv_bulk(self)
821
822
def _recv_tuple(self):
823
self._ensure_connection()
824
return SmartProtocolBase._recv_tuple(self)
825
826
def _recv_trailer(self):
827
self._ensure_connection()
828
return SmartProtocolBase._recv_trailer(self)
829
830
def disconnect(self):
831
"""Close connection to the server"""
832
if self._connected:
833
self._out.close()
834
self._in.close()
835
836
def _call(self, *args):
837
self._send_tuple(args)
838
return self._recv_tuple()
839
840
def _call_with_upload(self, method, args, body):
841
"""Call an rpc, supplying bulk upload data.
842
843
:param method: method name to call
844
:param args: parameter args tuple
845
:param body: upload body as a byte string
846
"""
847
self._send_tuple((method,) + args)
848
self._send_bulk_data(body)
849
return self._recv_tuple()
850
851
def query_version(self):
852
"""Return protocol version number of the server."""
853
# XXX: should make sure it's empty
854
self._send_tuple(('hello',))
855
resp = self._recv_tuple()
856
if resp == ('ok', '1'):
857
return 1
858
else:
859
raise errors.SmartProtocolError("bad response %r" % (resp,))
860
861
862
class SmartTCPTransport(SmartTransport):
863
"""Connection to smart server over plain tcp"""
864
865
def __init__(self, url, clone_from=None):
866
super(SmartTCPTransport, self).__init__(url, clone_from)
867
try:
868
self._port = int(self._port)
869
except (ValueError, TypeError), e:
870
raise errors.InvalidURL(path=url, extra="invalid port %s" % self._port)
871
self._socket = None
872
873
def _connect_to_server(self):
874
self._socket = socket.socket()
875
self._socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
876
result = self._socket.connect_ex((self._host, int(self._port)))
877
if result:
878
raise errors.ConnectionError("failed to connect to %s:%d: %s" %
879
(self._host, self._port, os.strerror(result)))
880
# TODO: May be more efficient to just treat them as sockets
881
# throughout? But what about pipes to ssh?...
882
to_server = self._socket.makefile('w')
883
from_server = self._socket.makefile('r')
884
return from_server, to_server
885
886
def disconnect(self):
887
super(SmartTCPTransport, self).disconnect()
888
# XXX: Is closing the socket as well as closing the files really
889
# necessary?
890
if self._socket is not None:
891
self._socket.close()
892
893
try:
894
from bzrlib.transport import sftp
895
except errors.ParamikoNotPresent:
896
# no paramiko, no SSHTransport.
897
pass
898
else:
899
class SmartSSHTransport(SmartTransport):
900
"""Connection to smart server over SSH."""
901
902
def __init__(self, url, clone_from=None):
903
# TODO: all this probably belongs in the parent class.
904
super(SmartSSHTransport, self).__init__(url, clone_from)
905
try:
906
if self._port is not None:
907
self._port = int(self._port)
908
except (ValueError, TypeError), e:
909
raise errors.InvalidURL(path=url, extra="invalid port %s" % self._port)
910
911
def _connect_to_server(self):
912
# XXX: don't hardcode vendor
913
# XXX: cannot pass password to SSHSubprocess yet
914
if self._password is not None:
915
raise errors.InvalidURL("SSH smart transport doesn't handle passwords")
916
self._ssh_connection = sftp.SSHSubprocess(self._host, 'openssh',
917
port=self._port, user=self._username,
918
command=['bzr', 'serve', '--inet'])
919
return self._ssh_connection.get_filelike_channels()
920
921
def disconnect(self):
922
super(SmartSSHTransport, self).disconnect()
923
self._ssh_connection.close()
924
925
926
def get_test_permutations():
927
"""Return (transport, server) permutations for testing"""
928
return [(SmartTCPTransport, SmartTCPServer_for_testing)]
Loggerhead is a web-based interface for Breezy
Version: 2.0.1