3331.3.5
by Martin Pool
Move physical lock hooks onto new PhysicalLock class variable |
1 |
# Copyright (C) 2006, 2007, 2008 Canonical Ltd
|
1887.1.1
by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines, |
2 |
#
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
3 |
# This program is free software; you can redistribute it and/or modify
|
4 |
# it under the terms of the GNU General Public License as published by
|
|
5 |
# the Free Software Foundation; either version 2 of the License, or
|
|
6 |
# (at your option) any later version.
|
|
1887.1.1
by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines, |
7 |
#
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
8 |
# This program is distributed in the hope that it will be useful,
|
9 |
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
10 |
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
11 |
# GNU General Public License for more details.
|
|
1887.1.1
by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines, |
12 |
#
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
13 |
# You should have received a copy of the GNU General Public License
|
14 |
# along with this program; if not, write to the Free Software
|
|
4183.7.1
by Sabin Iacob
update FSF mailing address |
15 |
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
16 |
|
1553.5.31
by Martin Pool
LockDir doc corrections |
17 |
"""On-disk mutex protecting a resource
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
18 |
|
1553.5.33
by Martin Pool
LockDir review comment fixes |
19 |
bzr on-disk objects are locked by the existence of a directory with a
|
20 |
particular name within the control directory. We use this rather than OS
|
|
21 |
internal locks (such as flock etc) because they can be seen across all
|
|
22 |
transports, including http.
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
23 |
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
24 |
Objects can be read if there is only physical read access; therefore
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
25 |
readers can never be required to create a lock, though they will
|
26 |
check whether a writer is using the lock. Writers can't detect
|
|
27 |
whether anyone else is reading from the resource as they write.
|
|
28 |
This works because of ordering constraints that make sure readers
|
|
29 |
see a consistent view of existing data.
|
|
30 |
||
31 |
Waiting for a lock must be done by polling; this can be aborted after
|
|
32 |
a timeout.
|
|
33 |
||
34 |
Locks must always be explicitly released, typically from a try/finally
|
|
35 |
block -- they are not released from a finalizer or when Python
|
|
36 |
exits.
|
|
37 |
||
38 |
Locks may fail to be released if the process is abruptly terminated
|
|
39 |
(machine stop, SIGKILL) or if a remote transport becomes permanently
|
|
40 |
disconnected. There is therefore a method to break an existing lock.
|
|
41 |
This should rarely be used, and generally only with user approval.
|
|
42 |
Locks contain some information on when the lock was taken and by who
|
|
43 |
which may guide in deciding whether it can safely be broken. (This is
|
|
44 |
similar to the messages displayed by emacs and vim.) Note that if the
|
|
45 |
lock holder is still alive they will get no notification that the lock
|
|
46 |
has been broken and will continue their work -- so it is important to be
|
|
47 |
sure they are actually dead.
|
|
48 |
||
49 |
A lock is represented on disk by a directory of a particular name,
|
|
50 |
containing an information file. Taking a lock is done by renaming a
|
|
51 |
temporary directory into place. We use temporary directories because
|
|
52 |
for all known transports and filesystems we believe that exactly one
|
|
53 |
attempt to claim the lock will succeed and the others will fail. (Files
|
|
54 |
won't do because some filesystems or transports only have
|
|
55 |
rename-and-overwrite, making it hard to tell who won.)
|
|
56 |
||
57 |
The desired characteristics are:
|
|
58 |
||
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
59 |
* Locks are not reentrant. (That is, a client that tries to take a
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
60 |
lock it already holds may deadlock or fail.)
|
61 |
* Stale locks can be guessed at by a heuristic
|
|
62 |
* Lost locks can be broken by any client
|
|
63 |
* Failed lock operations leave little or no mess
|
|
64 |
* Deadlocks are avoided by having a timeout always in use, clients
|
|
65 |
desiring indefinite waits can retry or set a silly big timeout.
|
|
66 |
||
67 |
Storage formats use the locks, and also need to consider concurrency
|
|
68 |
issues underneath the lock. A format may choose not to use a lock
|
|
69 |
at all for some operations.
|
|
70 |
||
71 |
LockDirs always operate over a Transport. The transport may be readonly, in
|
|
72 |
which case the lock can be queried but not acquired.
|
|
73 |
||
74 |
Locks are identified by a path name, relative to a base transport.
|
|
75 |
||
76 |
Calling code will typically want to make sure there is exactly one LockDir
|
|
77 |
object per actual lock on disk. This module does nothing to prevent aliasing
|
|
78 |
and deadlocks will likely occur if the locks are aliased.
|
|
79 |
||
80 |
In the future we may add a "freshen" method which can be called
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
81 |
by a lock holder to check that their lock has not been broken, and to
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
82 |
update the timestamp within it.
|
83 |
||
1553.5.18
by Martin Pool
LockDir should use Transport.rename when releasing locks too. |
84 |
Example usage:
|
85 |
||
86 |
>>> from bzrlib.transport.memory import MemoryTransport
|
|
87 |
>>> # typically will be obtained from a BzrDir, Branch, etc
|
|
88 |
>>> t = MemoryTransport()
|
|
89 |
>>> l = LockDir(t, 'sample-lock')
|
|
1553.5.61
by Martin Pool
Locks protecting LockableFiles must now be explicitly created before use. |
90 |
>>> l.create()
|
2555.3.11
by Martin Pool
fix up doctest example for token being returned from wait_lock |
91 |
>>> token = l.wait_lock()
|
1553.5.18
by Martin Pool
LockDir should use Transport.rename when releasing locks too. |
92 |
>>> # do something here
|
93 |
>>> l.unlock()
|
|
94 |
||
1553.5.12
by Martin Pool
New LockDir locking mechanism |
95 |
"""
|
96 |
||
2555.3.10
by Martin Pool
doc |
97 |
|
98 |
# TODO: We sometimes have the problem that our attempt to rename '1234' to
|
|
99 |
# 'held' fails because the transport server moves into an existing directory,
|
|
100 |
# rather than failing the rename. If we made the info file name the same as
|
|
101 |
# the locked directory name we would avoid this problem because moving into
|
|
102 |
# the held directory would implicitly clash. However this would not mesh with
|
|
103 |
# the existing locking code and needs a new format of the containing object.
|
|
104 |
# -- robertc, mbp 20070628
|
|
105 |
||
1553.5.12
by Martin Pool
New LockDir locking mechanism |
106 |
import os |
107 |
import time |
|
108 |
||
2055.2.1
by John Arbash Meinel
Make LockDir less sensitive to invalid configuration of email |
109 |
from bzrlib import ( |
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
110 |
debug, |
2055.2.1
by John Arbash Meinel
Make LockDir less sensitive to invalid configuration of email |
111 |
errors, |
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
112 |
lock, |
2055.2.1
by John Arbash Meinel
Make LockDir less sensitive to invalid configuration of email |
113 |
)
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
114 |
import bzrlib.config |
115 |
from bzrlib.errors import ( |
|
116 |
DirectoryNotEmpty, |
|
117 |
FileExists, |
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
118 |
LockBreakMismatch, |
1553.5.23
by Martin Pool
Start LockDir.confirm method and LockBroken exception |
119 |
LockBroken, |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
120 |
LockContention, |
2872.5.1
by Martin Pool
Avoid internal error tracebacks on failure to lock on readonly transport (#129701). |
121 |
LockFailed, |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
122 |
LockNotHeld, |
123 |
NoSuchFile, |
|
1662.1.12
by Martin Pool
Translate unknown sftp errors to PathError, no NoSuchFile |
124 |
PathError, |
1558.10.1
by Aaron Bentley
Handle lockdirs over NFS properly |
125 |
ResourceBusy, |
2872.5.1
by Martin Pool
Avoid internal error tracebacks on failure to lock on readonly transport (#129701). |
126 |
TransportError, |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
127 |
)
|
1957.1.1
by John Arbash Meinel
Report to the user when we are spinning on a lock |
128 |
from bzrlib.trace import mutter, note |
3224.5.40
by Andrew Bennetts
Keep importing format_delta into lockdir.py (why does pyflakes say it is unused?) |
129 |
from bzrlib.osutils import format_delta, rand_chars, get_host_name |
3224.5.31
by Andrew Bennetts
A couple more lazy imports, helps 'bzr log --line -r -1' a little. |
130 |
import bzrlib.ui |
131 |
||
132 |
from bzrlib.lazy_import import lazy_import |
|
133 |
lazy_import(globals(), """ |
|
3224.5.39
by Andrew Bennetts
Tweak lazy_import in lockdir.py to avoid confusing DocTestSuite |
134 |
from bzrlib import rio
|
3224.5.31
by Andrew Bennetts
A couple more lazy imports, helps 'bzr log --line -r -1' a little. |
135 |
""") |
1957.1.15
by John Arbash Meinel
Review feedback from Robert |
136 |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
137 |
# XXX: At the moment there is no consideration of thread safety on LockDir
|
138 |
# objects. This should perhaps be updated - e.g. if two threads try to take a
|
|
139 |
# lock at the same time they should *both* get it. But then that's unlikely
|
|
140 |
# to be a good idea.
|
|
141 |
||
142 |
# TODO: Perhaps store some kind of note like the bzr command line in the lock
|
|
143 |
# info?
|
|
144 |
||
145 |
# TODO: Some kind of callback run while polling a lock to show progress
|
|
146 |
# indicators.
|
|
147 |
||
1553.5.59
by Martin Pool
Pass file/mode bits through to creation of lock files/dirs |
148 |
# TODO: Make sure to pass the right file and directory mode bits to all
|
149 |
# files/dirs created.
|
|
150 |
||
1957.1.15
by John Arbash Meinel
Review feedback from Robert |
151 |
|
152 |
_DEFAULT_TIMEOUT_SECONDS = 300 |
|
1957.1.9
by John Arbash Meinel
Change default timeouts, and report differently the first failure |
153 |
_DEFAULT_POLL_SECONDS = 1.0 |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
154 |
|
1957.1.15
by John Arbash Meinel
Review feedback from Robert |
155 |
|
3724.1.1
by Martin Pool
Move Lock hooks onto a base Lock class and make them more consistent with other lock classes |
156 |
class LockDir(lock.Lock): |
3331.3.5
by Martin Pool
Move physical lock hooks onto new PhysicalLock class variable |
157 |
"""Write-lock guarding access to data.
|
158 |
"""
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
159 |
|
1553.5.33
by Martin Pool
LockDir review comment fixes |
160 |
__INFO_NAME = '/info' |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
161 |
|
1553.5.59
by Martin Pool
Pass file/mode bits through to creation of lock files/dirs |
162 |
def __init__(self, transport, path, file_modebits=0644, dir_modebits=0755): |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
163 |
"""Create a new LockDir object.
|
164 |
||
165 |
The LockDir is initially unlocked - this just creates the object.
|
|
166 |
||
167 |
:param transport: Transport which will contain the lock
|
|
168 |
||
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
169 |
:param path: Path to the lock within the base directory of the
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
170 |
transport.
|
171 |
"""
|
|
172 |
self.transport = transport |
|
173 |
self.path = path |
|
174 |
self._lock_held = False |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
175 |
self._locked_via_token = False |
1553.5.54
by Martin Pool
Add LockDir.read_lock fake method |
176 |
self._fake_read_lock = False |
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
177 |
self._held_dir = path + '/held' |
178 |
self._held_info_path = self._held_dir + self.__INFO_NAME |
|
1553.5.59
by Martin Pool
Pass file/mode bits through to creation of lock files/dirs |
179 |
self._file_modebits = file_modebits |
180 |
self._dir_modebits = dir_modebits |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
181 |
|
1957.1.1
by John Arbash Meinel
Report to the user when we are spinning on a lock |
182 |
self._report_function = note |
183 |
||
1553.5.12
by Martin Pool
New LockDir locking mechanism |
184 |
def __repr__(self): |
185 |
return '%s(%s%s)' % (self.__class__.__name__, |
|
186 |
self.transport.base, |
|
187 |
self.path) |
|
188 |
||
1553.5.13
by Martin Pool
New Transport.rename that mustn't overwrite |
189 |
is_held = property(lambda self: self._lock_held) |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
190 |
|
1666.1.4
by Robert Collins
* 'Metadir' is now the default disk format. This improves behaviour in |
191 |
def create(self, mode=None): |
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
192 |
"""Create the on-disk lock.
|
193 |
||
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
194 |
This is typically only called when the object/directory containing the
|
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
195 |
directory is first created. The lock is not held when it's created.
|
196 |
"""
|
|
2555.3.13
by Martin Pool
LockDir cleanups |
197 |
self._trace("create lock directory") |
2872.5.1
by Martin Pool
Avoid internal error tracebacks on failure to lock on readonly transport (#129701). |
198 |
try: |
199 |
self.transport.mkdir(self.path, mode=mode) |
|
200 |
except (TransportError, PathError), e: |
|
201 |
raise LockFailed(self, e) |
|
202 |
||
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
203 |
|
2555.3.20
by Martin Pool
rename _lock_core to _attempt_lock |
204 |
def _attempt_lock(self): |
2555.3.13
by Martin Pool
LockDir cleanups |
205 |
"""Make the pending directory and attempt to rename into place.
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
206 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
207 |
If the rename succeeds, we read back the info file to check that we
|
208 |
really got the lock.
|
|
209 |
||
210 |
If we fail to acquire the lock, this method is responsible for
|
|
211 |
cleaning up the pending directory if possible. (But it doesn't do
|
|
212 |
that yet.)
|
|
213 |
||
214 |
:returns: The nonce of the lock, if it was successfully acquired.
|
|
215 |
||
216 |
:raises LockContention: If the lock is held by someone else. The exception
|
|
217 |
contains the info of the current holder of the lock.
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
218 |
"""
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
219 |
self._trace("lock_write...") |
220 |
start_time = time.time() |
|
2872.5.1
by Martin Pool
Avoid internal error tracebacks on failure to lock on readonly transport (#129701). |
221 |
try: |
222 |
tmpname = self._create_pending_dir() |
|
223 |
except (errors.TransportError, PathError), e: |
|
224 |
self._trace("... failed to create pending dir, %s", e) |
|
225 |
raise LockFailed(self, e) |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
226 |
try: |
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
227 |
self.transport.rename(tmpname, self._held_dir) |
2872.5.1
by Martin Pool
Avoid internal error tracebacks on failure to lock on readonly transport (#129701). |
228 |
except (errors.TransportError, PathError, DirectoryNotEmpty, |
229 |
FileExists, ResourceBusy), e: |
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
230 |
self._trace("... contention, %s", e) |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
231 |
self._remove_pending_dir(tmpname) |
232 |
raise LockContention(self) |
|
233 |
except Exception, e: |
|
234 |
self._trace("... lock failed, %s", e) |
|
235 |
self._remove_pending_dir(tmpname) |
|
236 |
raise
|
|
237 |
# We must check we really got the lock, because Launchpad's sftp
|
|
238 |
# server at one time had a bug were the rename would successfully
|
|
239 |
# move the new directory into the existing directory, which was
|
|
240 |
# incorrect. It's possible some other servers or filesystems will
|
|
241 |
# have a similar bug allowing someone to think they got the lock
|
|
242 |
# when it's already held.
|
|
243 |
info = self.peek() |
|
244 |
self._trace("after locking, info=%r", info) |
|
245 |
if info['nonce'] != self.nonce: |
|
246 |
self._trace("rename succeeded, " |
|
247 |
"but lock is still held by someone else") |
|
248 |
raise LockContention(self) |
|
249 |
self._lock_held = True |
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
250 |
self._trace("... lock succeeded after %dms", |
251 |
(time.time() - start_time) * 1000) |
|
2555.3.6
by Martin Pool
Restore disabled LockDir trace message |
252 |
return self.nonce |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
253 |
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
254 |
def _remove_pending_dir(self, tmpname): |
255 |
"""Remove the pending directory
|
|
256 |
||
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
257 |
This is called if we failed to rename into place, so that the pending
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
258 |
dirs don't clutter up the lockdir.
|
259 |
"""
|
|
260 |
self._trace("remove %s", tmpname) |
|
2555.3.18
by Martin Pool
Don't fail hard when removing pending lockdir |
261 |
try: |
262 |
self.transport.delete(tmpname + self.__INFO_NAME) |
|
263 |
self.transport.rmdir(tmpname) |
|
264 |
except PathError, e: |
|
265 |
note("error removing pending lock: %s", e) |
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
266 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
267 |
def _create_pending_dir(self): |
268 |
tmpname = '%s/%s.tmp' % (self.path, rand_chars(10)) |
|
269 |
try: |
|
270 |
self.transport.mkdir(tmpname) |
|
271 |
except NoSuchFile: |
|
272 |
# This may raise a FileExists exception
|
|
273 |
# which is okay, it will be caught later and determined
|
|
274 |
# to be a LockContention.
|
|
275 |
self._trace("lock directory does not exist, creating it") |
|
276 |
self.create(mode=self._dir_modebits) |
|
277 |
# After creating the lock directory, try again
|
|
278 |
self.transport.mkdir(tmpname) |
|
279 |
self.nonce = rand_chars(20) |
|
280 |
info_bytes = self._prepare_info() |
|
281 |
# We use put_file_non_atomic because we just created a new unique
|
|
282 |
# directory so we don't have to worry about files existing there.
|
|
283 |
# We'll rename the whole directory into place to get atomic
|
|
284 |
# properties
|
|
285 |
self.transport.put_bytes_non_atomic(tmpname + self.__INFO_NAME, |
|
286 |
info_bytes) |
|
287 |
return tmpname |
|
288 |
||
1553.5.12
by Martin Pool
New LockDir locking mechanism |
289 |
def unlock(self): |
1553.5.30
by Martin Pool
docstring for LockDir.unlock |
290 |
"""Release a held lock
|
291 |
"""
|
|
1553.5.54
by Martin Pool
Add LockDir.read_lock fake method |
292 |
if self._fake_read_lock: |
293 |
self._fake_read_lock = False |
|
294 |
return
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
295 |
if not self._lock_held: |
296 |
raise LockNotHeld(self) |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
297 |
if self._locked_via_token: |
298 |
self._locked_via_token = False |
|
299 |
self._lock_held = False |
|
300 |
else: |
|
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
301 |
old_nonce = self.nonce |
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
302 |
# rename before deleting, because we can't atomically remove the
|
303 |
# whole tree
|
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
304 |
start_time = time.time() |
305 |
self._trace("unlocking") |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
306 |
tmpname = '%s/releasing.%s.tmp' % (self.path, rand_chars(20)) |
307 |
# gotta own it to unlock
|
|
308 |
self.confirm() |
|
309 |
self.transport.rename(self._held_dir, tmpname) |
|
310 |
self._lock_held = False |
|
311 |
self.transport.delete(tmpname + self.__INFO_NAME) |
|
2555.3.14
by Martin Pool
Better handling in LockDir of rename that moves one directory within another |
312 |
try: |
313 |
self.transport.rmdir(tmpname) |
|
314 |
except DirectoryNotEmpty, e: |
|
315 |
# There might have been junk left over by a rename that moved
|
|
316 |
# another locker within the 'held' directory. do a slower
|
|
317 |
# deletion where we list the directory and remove everything
|
|
318 |
# within it.
|
|
319 |
#
|
|
320 |
# Maybe this should be broader to allow for ftp servers with
|
|
321 |
# non-specific error messages?
|
|
322 |
self._trace("doing recursive deletion of non-empty directory " |
|
323 |
"%s", tmpname) |
|
324 |
self.transport.delete_tree(tmpname) |
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
325 |
self._trace("... unlock succeeded after %dms", |
326 |
(time.time() - start_time) * 1000) |
|
3331.3.2
by Robert Collins
Polish on lock hooks to be easier to use. |
327 |
result = lock.LockResult(self.transport.abspath(self.path), |
328 |
old_nonce) |
|
3331.3.6
by Martin Pool
merge trunk |
329 |
for hook in self.hooks['lock_released']: |
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
330 |
hook(result) |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
331 |
|
1687.1.5
by Robert Collins
Add break_lock utility function to LockDir. |
332 |
def break_lock(self): |
333 |
"""Break a lock not held by this instance of LockDir.
|
|
334 |
||
335 |
This is a UI centric function: it uses the bzrlib.ui.ui_factory to
|
|
336 |
prompt for input if a lock is detected and there is any doubt about
|
|
337 |
it possibly being still active.
|
|
338 |
"""
|
|
1687.1.6
by Robert Collins
Extend LockableFiles to support break_lock() calls. |
339 |
self._check_not_locked() |
1687.1.5
by Robert Collins
Add break_lock utility function to LockDir. |
340 |
holder_info = self.peek() |
341 |
if holder_info is not None: |
|
1957.1.5
by John Arbash Meinel
Create a helper function for formatting lock information |
342 |
lock_info = '\n'.join(self._format_lock_info(holder_info)) |
343 |
if bzrlib.ui.ui_factory.get_boolean("Break %s" % lock_info): |
|
1687.1.5
by Robert Collins
Add break_lock utility function to LockDir. |
344 |
self.force_break(holder_info) |
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
345 |
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
346 |
def force_break(self, dead_holder_info): |
1553.5.25
by Martin Pool
New LockDir.force_break and simple test case |
347 |
"""Release a lock held by another process.
|
348 |
||
349 |
WARNING: This should only be used when the other process is dead; if
|
|
350 |
it still thinks it has the lock there will be two concurrent writers.
|
|
351 |
In general the user's approval should be sought for lock breaks.
|
|
352 |
||
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
353 |
dead_holder_info must be the result of a previous LockDir.peek() call;
|
354 |
this is used to check that it's still held by the same process that
|
|
355 |
the user decided was dead. If this is not the current holder,
|
|
356 |
LockBreakMismatch is raised.
|
|
357 |
||
1553.5.25
by Martin Pool
New LockDir.force_break and simple test case |
358 |
After the lock is broken it will not be held by any process.
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
359 |
It is possible that another process may sneak in and take the
|
1553.5.25
by Martin Pool
New LockDir.force_break and simple test case |
360 |
lock before the breaking process acquires it.
|
361 |
"""
|
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
362 |
if not isinstance(dead_holder_info, dict): |
363 |
raise ValueError("dead_holder_info: %r" % dead_holder_info) |
|
1687.1.6
by Robert Collins
Extend LockableFiles to support break_lock() calls. |
364 |
self._check_not_locked() |
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
365 |
current_info = self.peek() |
366 |
if current_info is None: |
|
1553.5.26
by Martin Pool
Breaking an already-released lock should just succeed |
367 |
# must have been recently released
|
368 |
return
|
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
369 |
if current_info != dead_holder_info: |
370 |
raise LockBreakMismatch(self, current_info, dead_holder_info) |
|
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
371 |
tmpname = '%s/broken.%s.tmp' % (self.path, rand_chars(20)) |
372 |
self.transport.rename(self._held_dir, tmpname) |
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
373 |
# check that we actually broke the right lock, not someone else;
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
374 |
# there's a small race window between checking it and doing the
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
375 |
# rename.
|
1553.5.33
by Martin Pool
LockDir review comment fixes |
376 |
broken_info_path = tmpname + self.__INFO_NAME |
377 |
broken_info = self._read_info_file(broken_info_path) |
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
378 |
if broken_info != dead_holder_info: |
379 |
raise LockBreakMismatch(self, broken_info, dead_holder_info) |
|
380 |
self.transport.delete(broken_info_path) |
|
1553.5.25
by Martin Pool
New LockDir.force_break and simple test case |
381 |
self.transport.rmdir(tmpname) |
382 |
||
1687.1.6
by Robert Collins
Extend LockableFiles to support break_lock() calls. |
383 |
def _check_not_locked(self): |
384 |
"""If the lock is held by this instance, raise an error."""
|
|
385 |
if self._lock_held: |
|
386 |
raise AssertionError("can't break own lock: %r" % self) |
|
387 |
||
1553.5.20
by Martin Pool
Start adding LockDir.confirm() method |
388 |
def confirm(self): |
389 |
"""Make sure that the lock is still held by this locker.
|
|
390 |
||
391 |
This should only fail if the lock was broken by user intervention,
|
|
392 |
or if the lock has been affected by a bug.
|
|
393 |
||
394 |
If the lock is not thought to be held, raises LockNotHeld. If
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
395 |
the lock is thought to be held but has been broken, raises
|
1553.5.20
by Martin Pool
Start adding LockDir.confirm() method |
396 |
LockBroken.
|
397 |
"""
|
|
398 |
if not self._lock_held: |
|
399 |
raise LockNotHeld(self) |
|
1553.5.23
by Martin Pool
Start LockDir.confirm method and LockBroken exception |
400 |
info = self.peek() |
401 |
if info is None: |
|
402 |
# no lock there anymore!
|
|
403 |
raise LockBroken(self) |
|
404 |
if info.get('nonce') != self.nonce: |
|
405 |
# there is a lock, but not ours
|
|
406 |
raise LockBroken(self) |
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
407 |
|
1553.5.33
by Martin Pool
LockDir review comment fixes |
408 |
def _read_info_file(self, path): |
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
409 |
"""Read one given info file.
|
410 |
||
411 |
peek() reads the info file of the lock holder, if any.
|
|
412 |
"""
|
|
1553.5.33
by Martin Pool
LockDir review comment fixes |
413 |
return self._parse_info(self.transport.get(path)) |
1553.5.20
by Martin Pool
Start adding LockDir.confirm() method |
414 |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
415 |
def peek(self): |
416 |
"""Check if the lock is held by anyone.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
417 |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
418 |
If it is held, this returns the lock info structure as a rio Stanza,
|
419 |
which contains some information about the current lock holder.
|
|
420 |
Otherwise returns None.
|
|
421 |
"""
|
|
422 |
try: |
|
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
423 |
info = self._read_info_file(self._held_info_path) |
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
424 |
self._trace("peek -> held") |
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
425 |
return info |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
426 |
except NoSuchFile, e: |
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
427 |
self._trace("peek -> not held") |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
428 |
|
1993.1.2
by John Arbash Meinel
Use Stanza.to_string() rather than a RioWriter |
429 |
def _prepare_info(self): |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
430 |
"""Write information about a pending lock to a temporary file.
|
431 |
"""
|
|
432 |
# XXX: is creating this here inefficient?
|
|
433 |
config = bzrlib.config.GlobalConfig() |
|
2055.2.1
by John Arbash Meinel
Make LockDir less sensitive to invalid configuration of email |
434 |
try: |
435 |
user = config.user_email() |
|
2055.2.2
by John Arbash Meinel
Switch extract_email_address() to use a more specific exception |
436 |
except errors.NoEmailInUsername: |
2055.2.1
by John Arbash Meinel
Make LockDir less sensitive to invalid configuration of email |
437 |
user = config.username() |
3224.5.39
by Andrew Bennetts
Tweak lazy_import in lockdir.py to avoid confusing DocTestSuite |
438 |
s = rio.Stanza(hostname=get_host_name(), |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
439 |
pid=str(os.getpid()), |
440 |
start_time=str(int(time.time())), |
|
441 |
nonce=self.nonce, |
|
2055.2.1
by John Arbash Meinel
Make LockDir less sensitive to invalid configuration of email |
442 |
user=user, |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
443 |
)
|
1993.1.2
by John Arbash Meinel
Use Stanza.to_string() rather than a RioWriter |
444 |
return s.to_string() |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
445 |
|
446 |
def _parse_info(self, info_file): |
|
3224.5.39
by Andrew Bennetts
Tweak lazy_import in lockdir.py to avoid confusing DocTestSuite |
447 |
return rio.read_stanza(info_file.readlines()).as_dict() |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
448 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
449 |
def attempt_lock(self): |
450 |
"""Take the lock; fail if it's already held.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
451 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
452 |
If you wish to block until the lock can be obtained, call wait_lock()
|
453 |
instead.
|
|
454 |
||
455 |
:return: The lock token.
|
|
456 |
:raises LockContention: if the lock is held by someone else.
|
|
457 |
"""
|
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
458 |
if self._fake_read_lock: |
459 |
raise LockContention(self) |
|
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
460 |
result = self._attempt_lock() |
3331.3.2
by Robert Collins
Polish on lock hooks to be easier to use. |
461 |
hook_result = lock.LockResult(self.transport.abspath(self.path), |
462 |
self.nonce) |
|
3331.3.6
by Martin Pool
merge trunk |
463 |
for hook in self.hooks['lock_acquired']: |
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
464 |
hook(hook_result) |
465 |
return result |
|
2555.3.13
by Martin Pool
LockDir cleanups |
466 |
|
467 |
def wait_lock(self, timeout=None, poll=None, max_attempts=None): |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
468 |
"""Wait a certain period for a lock.
|
469 |
||
470 |
If the lock can be acquired within the bounded time, it
|
|
471 |
is taken and this returns. Otherwise, LockContention
|
|
472 |
is raised. Either way, this function should return within
|
|
473 |
approximately `timeout` seconds. (It may be a bit more if
|
|
474 |
a transport operation takes a long time to complete.)
|
|
2555.3.5
by Martin Pool
Return token directly from LockDir.acquire to avoid unnecessary peek() |
475 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
476 |
:param timeout: Approximate maximum amount of time to wait for the
|
477 |
lock, in seconds.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
478 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
479 |
:param poll: Delay in seconds between retrying the lock.
|
480 |
||
481 |
:param max_attempts: Maximum number of times to try to lock.
|
|
482 |
||
2555.3.5
by Martin Pool
Return token directly from LockDir.acquire to avoid unnecessary peek() |
483 |
:return: The lock token.
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
484 |
"""
|
1957.1.17
by John Arbash Meinel
Change tests that expect locking to fail to timeout sooner. |
485 |
if timeout is None: |
486 |
timeout = _DEFAULT_TIMEOUT_SECONDS |
|
487 |
if poll is None: |
|
488 |
poll = _DEFAULT_POLL_SECONDS |
|
2555.3.13
by Martin Pool
LockDir cleanups |
489 |
# XXX: the transport interface doesn't let us guard against operations
|
490 |
# there taking a long time, so the total elapsed time or poll interval
|
|
491 |
# may be more than was requested.
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
492 |
deadline = time.time() + timeout |
1957.1.13
by John Arbash Meinel
Change to reporting the time when we will stop trying to grab the lock |
493 |
deadline_str = None |
1957.1.7
by John Arbash Meinel
Add the ability to report if the lock changes from underneath you |
494 |
last_info = None |
2555.3.13
by Martin Pool
LockDir cleanups |
495 |
attempt_count = 0 |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
496 |
while True: |
2555.3.13
by Martin Pool
LockDir cleanups |
497 |
attempt_count += 1 |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
498 |
try: |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
499 |
return self.attempt_lock() |
500 |
except LockContention: |
|
501 |
# possibly report the blockage, then try again
|
|
502 |
pass
|
|
503 |
# TODO: In a few cases, we find out that there's contention by
|
|
504 |
# reading the held info and observing that it's not ours. In
|
|
505 |
# those cases it's a bit redundant to read it again. However,
|
|
506 |
# the normal case (??) is that the rename fails and so we
|
|
507 |
# don't know who holds the lock. For simplicity we peek
|
|
508 |
# always.
|
|
509 |
new_info = self.peek() |
|
510 |
if new_info is not None and new_info != last_info: |
|
511 |
if last_info is None: |
|
512 |
start = 'Unable to obtain' |
|
1957.1.9
by John Arbash Meinel
Change default timeouts, and report differently the first failure |
513 |
else: |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
514 |
start = 'Lock owner changed for' |
515 |
last_info = new_info |
|
516 |
formatted_info = self._format_lock_info(new_info) |
|
517 |
if deadline_str is None: |
|
518 |
deadline_str = time.strftime('%H:%M:%S', |
|
519 |
time.localtime(deadline)) |
|
3441.4.1
by Martin Albisetti
Change locked error to help the user resolve the situation |
520 |
lock_url = self.transport.abspath(self.path) |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
521 |
self._report_function('%s %s\n' |
522 |
'%s\n' # held by |
|
523 |
'%s\n' # locked ... ago |
|
3441.4.1
by Martin Albisetti
Change locked error to help the user resolve the situation |
524 |
'Will continue to try until %s, unless ' |
525 |
'you press Ctrl-C\n' |
|
526 |
'If you\'re sure that it\'s not being ' |
|
527 |
'modified, use bzr break-lock %s', |
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
528 |
start, |
529 |
formatted_info[0], |
|
530 |
formatted_info[1], |
|
531 |
formatted_info[2], |
|
3441.4.1
by Martin Albisetti
Change locked error to help the user resolve the situation |
532 |
deadline_str, |
533 |
lock_url) |
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
534 |
|
535 |
if (max_attempts is not None) and (attempt_count >= max_attempts): |
|
536 |
self._trace("exceeded %d attempts") |
|
537 |
raise LockContention(self) |
|
538 |
if time.time() + poll < deadline: |
|
539 |
self._trace("waiting %ss", poll) |
|
540 |
time.sleep(poll) |
|
541 |
else: |
|
542 |
self._trace("timeout after waiting %ss", timeout) |
|
543 |
raise LockContention(self) |
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
544 |
|
2018.5.75
by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place. |
545 |
def leave_in_place(self): |
546 |
self._locked_via_token = True |
|
547 |
||
548 |
def dont_leave_in_place(self): |
|
549 |
self._locked_via_token = False |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
550 |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
551 |
def lock_write(self, token=None): |
552 |
"""Wait for and acquire the lock.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
553 |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
554 |
:param token: if this is already locked, then lock_write will fail
|
555 |
unless the token matches the existing lock.
|
|
556 |
:returns: a token if this instance supports tokens, otherwise None.
|
|
557 |
:raises TokenLockingNotSupported: when a token is given but this
|
|
558 |
instance doesn't support using token locks.
|
|
2279.7.7
by Andrew Bennetts
LockDir, Repository and Branch lock token changes from the hpss branch. |
559 |
:raises MismatchedToken: if the specified token doesn't match the token
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
560 |
of the existing lock.
|
2018.5.145
by Andrew Bennetts
Add a brief explanation of what tokens are used for to lock_write docstrings. |
561 |
|
562 |
A token should be passed in if you know that you have locked the object
|
|
563 |
some other way, and need to synchronise this object's state with that
|
|
564 |
fact.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
565 |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
566 |
XXX: docstring duplicated from LockableFiles.lock_write.
|
567 |
"""
|
|
568 |
if token is not None: |
|
569 |
self.validate_token(token) |
|
2018.5.75
by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place. |
570 |
self.nonce = token |
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
571 |
self._lock_held = True |
572 |
self._locked_via_token = True |
|
2018.5.78
by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the |
573 |
return token |
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
574 |
else: |
2555.3.5
by Martin Pool
Return token directly from LockDir.acquire to avoid unnecessary peek() |
575 |
return self.wait_lock() |
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
576 |
|
577 |
def lock_read(self): |
|
1759.2.1
by Jelmer Vernooij
Fix some types (found using aspell). |
578 |
"""Compatibility-mode shared lock.
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
579 |
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
580 |
LockDir doesn't support shared read-only locks, so this
|
1553.5.54
by Martin Pool
Add LockDir.read_lock fake method |
581 |
just pretends that the lock is taken but really does nothing.
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
582 |
"""
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
583 |
# At the moment Branches are commonly locked for read, but
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
584 |
# we can't rely on that remotely. Once this is cleaned up,
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
585 |
# reenable this warning to prevent it coming back in
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
586 |
# -- mbp 20060303
|
587 |
## warn("LockDir.lock_read falls back to write lock")
|
|
1553.5.54
by Martin Pool
Add LockDir.read_lock fake method |
588 |
if self._lock_held or self._fake_read_lock: |
589 |
raise LockContention(self) |
|
590 |
self._fake_read_lock = True |
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
591 |
|
1957.1.5
by John Arbash Meinel
Create a helper function for formatting lock information |
592 |
def _format_lock_info(self, info): |
593 |
"""Turn the contents of peek() into something for the user"""
|
|
594 |
lock_url = self.transport.abspath(self.path) |
|
595 |
delta = time.time() - int(info['start_time']) |
|
596 |
return [ |
|
597 |
'lock %s' % (lock_url,), |
|
598 |
'held by %(user)s on host %(hostname)s [process #%(pid)s]' % info, |
|
599 |
'locked %s' % (format_delta(delta),), |
|
600 |
]
|
|
601 |
||
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
602 |
def validate_token(self, token): |
603 |
if token is not None: |
|
604 |
info = self.peek() |
|
605 |
if info is None: |
|
606 |
# Lock isn't held
|
|
607 |
lock_token = None |
|
608 |
else: |
|
609 |
lock_token = info.get('nonce') |
|
610 |
if token != lock_token: |
|
611 |
raise errors.TokenMismatch(token, lock_token) |
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
612 |
else: |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
613 |
self._trace("revalidated by token %r", token) |
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
614 |
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
615 |
def _trace(self, format, *args): |
616 |
if 'lock' not in debug.debug_flags: |
|
617 |
return
|
|
618 |
mutter(str(self) + ": " + (format % args)) |