5599.2.1
by John Arbash Meinel
Fix breaking corrupt lock files on Windows. |
1 |
# Copyright (C) 2006-2011 Canonical Ltd
|
1887.1.1
by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines, |
2 |
#
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
3 |
# This program is free software; you can redistribute it and/or modify
|
4 |
# it under the terms of the GNU General Public License as published by
|
|
5 |
# the Free Software Foundation; either version 2 of the License, or
|
|
6 |
# (at your option) any later version.
|
|
1887.1.1
by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines, |
7 |
#
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
8 |
# This program is distributed in the hope that it will be useful,
|
9 |
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
10 |
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
11 |
# GNU General Public License for more details.
|
|
1887.1.1
by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines, |
12 |
#
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
13 |
# You should have received a copy of the GNU General Public License
|
14 |
# along with this program; if not, write to the Free Software
|
|
4183.7.1
by Sabin Iacob
update FSF mailing address |
15 |
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
16 |
|
1553.5.31
by Martin Pool
LockDir doc corrections |
17 |
"""On-disk mutex protecting a resource
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
18 |
|
1553.5.33
by Martin Pool
LockDir review comment fixes |
19 |
bzr on-disk objects are locked by the existence of a directory with a
|
20 |
particular name within the control directory. We use this rather than OS
|
|
21 |
internal locks (such as flock etc) because they can be seen across all
|
|
22 |
transports, including http.
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
23 |
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
24 |
Objects can be read if there is only physical read access; therefore
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
25 |
readers can never be required to create a lock, though they will
|
26 |
check whether a writer is using the lock. Writers can't detect
|
|
27 |
whether anyone else is reading from the resource as they write.
|
|
28 |
This works because of ordering constraints that make sure readers
|
|
29 |
see a consistent view of existing data.
|
|
30 |
||
31 |
Waiting for a lock must be done by polling; this can be aborted after
|
|
32 |
a timeout.
|
|
33 |
||
34 |
Locks must always be explicitly released, typically from a try/finally
|
|
35 |
block -- they are not released from a finalizer or when Python
|
|
36 |
exits.
|
|
37 |
||
38 |
Locks may fail to be released if the process is abruptly terminated
|
|
39 |
(machine stop, SIGKILL) or if a remote transport becomes permanently
|
|
40 |
disconnected. There is therefore a method to break an existing lock.
|
|
41 |
This should rarely be used, and generally only with user approval.
|
|
42 |
Locks contain some information on when the lock was taken and by who
|
|
43 |
which may guide in deciding whether it can safely be broken. (This is
|
|
44 |
similar to the messages displayed by emacs and vim.) Note that if the
|
|
45 |
lock holder is still alive they will get no notification that the lock
|
|
46 |
has been broken and will continue their work -- so it is important to be
|
|
47 |
sure they are actually dead.
|
|
48 |
||
49 |
A lock is represented on disk by a directory of a particular name,
|
|
50 |
containing an information file. Taking a lock is done by renaming a
|
|
51 |
temporary directory into place. We use temporary directories because
|
|
52 |
for all known transports and filesystems we believe that exactly one
|
|
53 |
attempt to claim the lock will succeed and the others will fail. (Files
|
|
54 |
won't do because some filesystems or transports only have
|
|
55 |
rename-and-overwrite, making it hard to tell who won.)
|
|
56 |
||
57 |
The desired characteristics are:
|
|
58 |
||
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
59 |
* Locks are not reentrant. (That is, a client that tries to take a
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
60 |
lock it already holds may deadlock or fail.)
|
61 |
* Stale locks can be guessed at by a heuristic
|
|
62 |
* Lost locks can be broken by any client
|
|
63 |
* Failed lock operations leave little or no mess
|
|
64 |
* Deadlocks are avoided by having a timeout always in use, clients
|
|
65 |
desiring indefinite waits can retry or set a silly big timeout.
|
|
66 |
||
67 |
Storage formats use the locks, and also need to consider concurrency
|
|
68 |
issues underneath the lock. A format may choose not to use a lock
|
|
69 |
at all for some operations.
|
|
70 |
||
71 |
LockDirs always operate over a Transport. The transport may be readonly, in
|
|
72 |
which case the lock can be queried but not acquired.
|
|
73 |
||
74 |
Locks are identified by a path name, relative to a base transport.
|
|
75 |
||
76 |
Calling code will typically want to make sure there is exactly one LockDir
|
|
77 |
object per actual lock on disk. This module does nothing to prevent aliasing
|
|
78 |
and deadlocks will likely occur if the locks are aliased.
|
|
79 |
||
80 |
In the future we may add a "freshen" method which can be called
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
81 |
by a lock holder to check that their lock has not been broken, and to
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
82 |
update the timestamp within it.
|
83 |
||
1553.5.18
by Martin Pool
LockDir should use Transport.rename when releasing locks too. |
84 |
Example usage:
|
85 |
||
86 |
>>> from bzrlib.transport.memory import MemoryTransport
|
|
87 |
>>> # typically will be obtained from a BzrDir, Branch, etc
|
|
88 |
>>> t = MemoryTransport()
|
|
89 |
>>> l = LockDir(t, 'sample-lock')
|
|
1553.5.61
by Martin Pool
Locks protecting LockableFiles must now be explicitly created before use. |
90 |
>>> l.create()
|
2555.3.11
by Martin Pool
fix up doctest example for token being returned from wait_lock |
91 |
>>> token = l.wait_lock()
|
1553.5.18
by Martin Pool
LockDir should use Transport.rename when releasing locks too. |
92 |
>>> # do something here
|
93 |
>>> l.unlock()
|
|
94 |
||
5425.6.1
by Martin Pool
Only detect as stale locks from the same user |
95 |
Some classes of stale locks can be predicted by checking: the host name is the
|
96 |
same as the local host name; the user name is the same as the local user; the
|
|
97 |
process id no longer exists. The check on user name is not strictly necessary
|
|
98 |
but helps protect against colliding host names.
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
99 |
"""
|
100 |
||
2555.3.10
by Martin Pool
doc |
101 |
|
102 |
# TODO: We sometimes have the problem that our attempt to rename '1234' to
|
|
103 |
# 'held' fails because the transport server moves into an existing directory,
|
|
104 |
# rather than failing the rename. If we made the info file name the same as
|
|
105 |
# the locked directory name we would avoid this problem because moving into
|
|
106 |
# the held directory would implicitly clash. However this would not mesh with
|
|
107 |
# the existing locking code and needs a new format of the containing object.
|
|
108 |
# -- robertc, mbp 20070628
|
|
109 |
||
5425.4.8
by Martin Pool
Basic detection of known dead lock holders |
110 |
import errno |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
111 |
import os |
112 |
import time |
|
113 |
||
2055.2.1
by John Arbash Meinel
Make LockDir less sensitive to invalid configuration of email |
114 |
from bzrlib import ( |
5425.4.15
by Martin Pool
Add steal_dead_locks option letting automatic lock-breaking be turned off |
115 |
config, |
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
116 |
debug, |
2055.2.1
by John Arbash Meinel
Make LockDir less sensitive to invalid configuration of email |
117 |
errors, |
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
118 |
lock, |
4852.1.1
by John Arbash Meinel
Change the LockDir code to use get_bytes rather than get(). |
119 |
osutils, |
5425.4.13
by Martin Pool
Steal locks of dead local processes, with a user warning |
120 |
ui, |
121 |
urlutils, |
|
2055.2.1
by John Arbash Meinel
Make LockDir less sensitive to invalid configuration of email |
122 |
)
|
4634.85.9
by Andrew Bennetts
Add some experimental decorators: @only_raises(..) and @cleanup_method. |
123 |
from bzrlib.decorators import only_raises |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
124 |
from bzrlib.errors import ( |
125 |
DirectoryNotEmpty, |
|
126 |
FileExists, |
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
127 |
LockBreakMismatch, |
1553.5.23
by Martin Pool
Start LockDir.confirm method and LockBroken exception |
128 |
LockBroken, |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
129 |
LockContention, |
4634.161.1
by Andrew Bennetts
Add LockCorrupt error, and use it to provide nicer handling of unparseable lock/held/info files. |
130 |
LockCorrupt, |
2872.5.1
by Martin Pool
Avoid internal error tracebacks on failure to lock on readonly transport (#129701). |
131 |
LockFailed, |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
132 |
LockNotHeld, |
133 |
NoSuchFile, |
|
1662.1.12
by Martin Pool
Translate unknown sftp errors to PathError, no NoSuchFile |
134 |
PathError, |
1558.10.1
by Aaron Bentley
Handle lockdirs over NFS properly |
135 |
ResourceBusy, |
2872.5.1
by Martin Pool
Avoid internal error tracebacks on failure to lock on readonly transport (#129701). |
136 |
TransportError, |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
137 |
)
|
1957.1.1
by John Arbash Meinel
Report to the user when we are spinning on a lock |
138 |
from bzrlib.trace import mutter, note |
3224.5.40
by Andrew Bennetts
Keep importing format_delta into lockdir.py (why does pyflakes say it is unused?) |
139 |
from bzrlib.osutils import format_delta, rand_chars, get_host_name |
3224.5.31
by Andrew Bennetts
A couple more lazy imports, helps 'bzr log --line -r -1' a little. |
140 |
|
141 |
from bzrlib.lazy_import import lazy_import |
|
142 |
lazy_import(globals(), """ |
|
3224.5.39
by Andrew Bennetts
Tweak lazy_import in lockdir.py to avoid confusing DocTestSuite |
143 |
from bzrlib import rio
|
3224.5.31
by Andrew Bennetts
A couple more lazy imports, helps 'bzr log --line -r -1' a little. |
144 |
""") |
1957.1.15
by John Arbash Meinel
Review feedback from Robert |
145 |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
146 |
# XXX: At the moment there is no consideration of thread safety on LockDir
|
147 |
# objects. This should perhaps be updated - e.g. if two threads try to take a
|
|
148 |
# lock at the same time they should *both* get it. But then that's unlikely
|
|
149 |
# to be a good idea.
|
|
150 |
||
151 |
# TODO: Perhaps store some kind of note like the bzr command line in the lock
|
|
152 |
# info?
|
|
153 |
||
154 |
# TODO: Some kind of callback run while polling a lock to show progress
|
|
155 |
# indicators.
|
|
156 |
||
1553.5.59
by Martin Pool
Pass file/mode bits through to creation of lock files/dirs |
157 |
# TODO: Make sure to pass the right file and directory mode bits to all
|
158 |
# files/dirs created.
|
|
159 |
||
1957.1.15
by John Arbash Meinel
Review feedback from Robert |
160 |
|
5284.6.7
by Parth Malwankar
set _DEFAULT_TIMEOUT_SECONDS to 30s based on discussion on bug #592148 |
161 |
_DEFAULT_TIMEOUT_SECONDS = 30 |
1957.1.9
by John Arbash Meinel
Change default timeouts, and report differently the first failure |
162 |
_DEFAULT_POLL_SECONDS = 1.0 |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
163 |
|
1957.1.15
by John Arbash Meinel
Review feedback from Robert |
164 |
|
3724.1.1
by Martin Pool
Move Lock hooks onto a base Lock class and make them more consistent with other lock classes |
165 |
class LockDir(lock.Lock): |
3331.3.5
by Martin Pool
Move physical lock hooks onto new PhysicalLock class variable |
166 |
"""Write-lock guarding access to data.
|
167 |
"""
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
168 |
|
1553.5.33
by Martin Pool
LockDir review comment fixes |
169 |
__INFO_NAME = '/info' |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
170 |
|
5425.4.10
by Martin Pool
Allow storing extra info into lockdir info files to aid testing |
171 |
def __init__(self, transport, path, file_modebits=0644, dir_modebits=0755, |
172 |
extra_holder_info=None): |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
173 |
"""Create a new LockDir object.
|
174 |
||
175 |
The LockDir is initially unlocked - this just creates the object.
|
|
176 |
||
177 |
:param transport: Transport which will contain the lock
|
|
178 |
||
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
179 |
:param path: Path to the lock within the base directory of the
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
180 |
transport.
|
5425.4.10
by Martin Pool
Allow storing extra info into lockdir info files to aid testing |
181 |
|
182 |
:param extra_holder_info: If passed, {str:str} dict of extra or
|
|
183 |
updated information to insert into the info file when the lock is
|
|
184 |
taken.
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
185 |
"""
|
186 |
self.transport = transport |
|
187 |
self.path = path |
|
188 |
self._lock_held = False |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
189 |
self._locked_via_token = False |
1553.5.54
by Martin Pool
Add LockDir.read_lock fake method |
190 |
self._fake_read_lock = False |
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
191 |
self._held_dir = path + '/held' |
192 |
self._held_info_path = self._held_dir + self.__INFO_NAME |
|
1553.5.59
by Martin Pool
Pass file/mode bits through to creation of lock files/dirs |
193 |
self._file_modebits = file_modebits |
194 |
self._dir_modebits = dir_modebits |
|
1957.1.1
by John Arbash Meinel
Report to the user when we are spinning on a lock |
195 |
self._report_function = note |
5425.4.10
by Martin Pool
Allow storing extra info into lockdir info files to aid testing |
196 |
self.extra_holder_info = extra_holder_info |
5425.4.25
by Martin Pool
Support auto-stealing dead locks, but turn it off by default. |
197 |
self._warned_about_lock_holder = None |
1957.1.1
by John Arbash Meinel
Report to the user when we are spinning on a lock |
198 |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
199 |
def __repr__(self): |
200 |
return '%s(%s%s)' % (self.__class__.__name__, |
|
4556.2.6
by Andrew Bennetts
More conservative fix for LockContention serialisation. |
201 |
self.transport.base, |
202 |
self.path) |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
203 |
|
1553.5.13
by Martin Pool
New Transport.rename that mustn't overwrite |
204 |
is_held = property(lambda self: self._lock_held) |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
205 |
|
1666.1.4
by Robert Collins
* 'Metadir' is now the default disk format. This improves behaviour in |
206 |
def create(self, mode=None): |
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
207 |
"""Create the on-disk lock.
|
208 |
||
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
209 |
This is typically only called when the object/directory containing the
|
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
210 |
directory is first created. The lock is not held when it's created.
|
211 |
"""
|
|
2555.3.13
by Martin Pool
LockDir cleanups |
212 |
self._trace("create lock directory") |
2872.5.1
by Martin Pool
Avoid internal error tracebacks on failure to lock on readonly transport (#129701). |
213 |
try: |
214 |
self.transport.mkdir(self.path, mode=mode) |
|
215 |
except (TransportError, PathError), e: |
|
216 |
raise LockFailed(self, e) |
|
217 |
||
2555.3.20
by Martin Pool
rename _lock_core to _attempt_lock |
218 |
def _attempt_lock(self): |
2555.3.13
by Martin Pool
LockDir cleanups |
219 |
"""Make the pending directory and attempt to rename into place.
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
220 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
221 |
If the rename succeeds, we read back the info file to check that we
|
222 |
really got the lock.
|
|
223 |
||
224 |
If we fail to acquire the lock, this method is responsible for
|
|
225 |
cleaning up the pending directory if possible. (But it doesn't do
|
|
226 |
that yet.)
|
|
227 |
||
228 |
:returns: The nonce of the lock, if it was successfully acquired.
|
|
229 |
||
5425.6.3
by Martin Pool
PEP8 cleanups |
230 |
:raises LockContention: If the lock is held by someone else. The
|
231 |
exception contains the info of the current holder of the lock.
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
232 |
"""
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
233 |
self._trace("lock_write...") |
234 |
start_time = time.time() |
|
2872.5.1
by Martin Pool
Avoid internal error tracebacks on failure to lock on readonly transport (#129701). |
235 |
try: |
236 |
tmpname = self._create_pending_dir() |
|
237 |
except (errors.TransportError, PathError), e: |
|
238 |
self._trace("... failed to create pending dir, %s", e) |
|
239 |
raise LockFailed(self, e) |
|
5425.4.13
by Martin Pool
Steal locks of dead local processes, with a user warning |
240 |
while True: |
241 |
try: |
|
242 |
self.transport.rename(tmpname, self._held_dir) |
|
243 |
break
|
|
244 |
except (errors.TransportError, PathError, DirectoryNotEmpty, |
|
245 |
FileExists, ResourceBusy), e: |
|
246 |
self._trace("... contention, %s", e) |
|
247 |
other_holder = self.peek() |
|
248 |
self._trace("other holder is %r" % other_holder) |
|
5425.4.25
by Martin Pool
Support auto-stealing dead locks, but turn it off by default. |
249 |
try: |
250 |
self._handle_lock_contention(other_holder) |
|
251 |
except: |
|
5425.4.13
by Martin Pool
Steal locks of dead local processes, with a user warning |
252 |
self._remove_pending_dir(tmpname) |
5425.4.25
by Martin Pool
Support auto-stealing dead locks, but turn it off by default. |
253 |
raise
|
5425.4.13
by Martin Pool
Steal locks of dead local processes, with a user warning |
254 |
except Exception, e: |
255 |
self._trace("... lock failed, %s", e) |
|
256 |
self._remove_pending_dir(tmpname) |
|
257 |
raise
|
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
258 |
# We must check we really got the lock, because Launchpad's sftp
|
259 |
# server at one time had a bug were the rename would successfully
|
|
260 |
# move the new directory into the existing directory, which was
|
|
261 |
# incorrect. It's possible some other servers or filesystems will
|
|
262 |
# have a similar bug allowing someone to think they got the lock
|
|
263 |
# when it's already held.
|
|
4634.114.1
by Martin Pool
Better message when lockdir goes missing after rename succeeds |
264 |
#
|
5243.1.2
by Martin
Point launchpad links in comments at production server rather than edge |
265 |
# See <https://bugs.launchpad.net/bzr/+bug/498378> for one case.
|
4634.114.1
by Martin Pool
Better message when lockdir goes missing after rename succeeds |
266 |
#
|
267 |
# Strictly the check is unnecessary and a waste of time for most
|
|
268 |
# people, but probably worth trapping if something is wrong.
|
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
269 |
info = self.peek() |
270 |
self._trace("after locking, info=%r", info) |
|
4634.114.1
by Martin Pool
Better message when lockdir goes missing after rename succeeds |
271 |
if info is None: |
272 |
raise LockFailed(self, "lock was renamed into place, but " |
|
273 |
"now is missing!") |
|
4634.138.2
by Martin Pool
Copy with lock info file being empty |
274 |
if info.get('nonce') != self.nonce: |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
275 |
self._trace("rename succeeded, " |
276 |
"but lock is still held by someone else") |
|
277 |
raise LockContention(self) |
|
278 |
self._lock_held = True |
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
279 |
self._trace("... lock succeeded after %dms", |
280 |
(time.time() - start_time) * 1000) |
|
2555.3.6
by Martin Pool
Restore disabled LockDir trace message |
281 |
return self.nonce |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
282 |
|
5425.4.25
by Martin Pool
Support auto-stealing dead locks, but turn it off by default. |
283 |
def _handle_lock_contention(self, other_holder): |
284 |
"""A lock we want to take is held by someone else.
|
|
285 |
||
286 |
This function can: tell the user about it; possibly detect that it's
|
|
287 |
safe or appropriate to steal the lock, or just raise an exception.
|
|
288 |
||
289 |
If this function returns (without raising an exception) the lock will
|
|
290 |
be attempted again.
|
|
291 |
||
292 |
:param other_holder: A LockHeldInfo for the current holder; note that
|
|
293 |
it might be None if the lock can be seen to be held but the info
|
|
294 |
can't be read.
|
|
295 |
"""
|
|
296 |
if (other_holder is not None): |
|
297 |
if other_holder.is_lock_holder_known_dead(): |
|
298 |
if self.get_config().get_user_option_as_bool( |
|
299 |
'locks.steal_dead', |
|
300 |
default=False): |
|
301 |
ui.ui_factory.show_user_warning( |
|
302 |
'locks_steal_dead', |
|
303 |
lock_url=urlutils.join(self.transport.base, self.path), |
|
304 |
other_holder_info=unicode(other_holder)) |
|
305 |
self.force_break(other_holder) |
|
306 |
self._trace("stole lock from dead holder") |
|
307 |
return
|
|
308 |
raise LockContention(self) |
|
309 |
||
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
310 |
def _remove_pending_dir(self, tmpname): |
311 |
"""Remove the pending directory
|
|
312 |
||
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
313 |
This is called if we failed to rename into place, so that the pending
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
314 |
dirs don't clutter up the lockdir.
|
315 |
"""
|
|
316 |
self._trace("remove %s", tmpname) |
|
2555.3.18
by Martin Pool
Don't fail hard when removing pending lockdir |
317 |
try: |
318 |
self.transport.delete(tmpname + self.__INFO_NAME) |
|
319 |
self.transport.rmdir(tmpname) |
|
320 |
except PathError, e: |
|
321 |
note("error removing pending lock: %s", e) |
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
322 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
323 |
def _create_pending_dir(self): |
324 |
tmpname = '%s/%s.tmp' % (self.path, rand_chars(10)) |
|
325 |
try: |
|
326 |
self.transport.mkdir(tmpname) |
|
327 |
except NoSuchFile: |
|
328 |
# This may raise a FileExists exception
|
|
329 |
# which is okay, it will be caught later and determined
|
|
330 |
# to be a LockContention.
|
|
331 |
self._trace("lock directory does not exist, creating it") |
|
332 |
self.create(mode=self._dir_modebits) |
|
333 |
# After creating the lock directory, try again
|
|
334 |
self.transport.mkdir(tmpname) |
|
5425.4.10
by Martin Pool
Allow storing extra info into lockdir info files to aid testing |
335 |
info = LockHeldInfo.for_this_process(self.extra_holder_info) |
336 |
self.nonce = info.get('nonce') |
|
2555.3.13
by Martin Pool
LockDir cleanups |
337 |
# We use put_file_non_atomic because we just created a new unique
|
338 |
# directory so we don't have to worry about files existing there.
|
|
339 |
# We'll rename the whole directory into place to get atomic
|
|
340 |
# properties
|
|
341 |
self.transport.put_bytes_non_atomic(tmpname + self.__INFO_NAME, |
|
5425.4.6
by Martin Pool
Move preparation of new locks into LockHeldInfo |
342 |
info.to_bytes()) |
2555.3.13
by Martin Pool
LockDir cleanups |
343 |
return tmpname |
344 |
||
4634.85.9
by Andrew Bennetts
Add some experimental decorators: @only_raises(..) and @cleanup_method. |
345 |
@only_raises(LockNotHeld, LockBroken) |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
346 |
def unlock(self): |
1553.5.30
by Martin Pool
docstring for LockDir.unlock |
347 |
"""Release a held lock
|
348 |
"""
|
|
1553.5.54
by Martin Pool
Add LockDir.read_lock fake method |
349 |
if self._fake_read_lock: |
350 |
self._fake_read_lock = False |
|
351 |
return
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
352 |
if not self._lock_held: |
4509.3.25
by Martin Pool
Add an option for unlock errors to be non-fatal |
353 |
return lock.cant_unlock_not_held(self) |
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
354 |
if self._locked_via_token: |
355 |
self._locked_via_token = False |
|
356 |
self._lock_held = False |
|
357 |
else: |
|
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
358 |
old_nonce = self.nonce |
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
359 |
# rename before deleting, because we can't atomically remove the
|
360 |
# whole tree
|
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
361 |
start_time = time.time() |
362 |
self._trace("unlocking") |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
363 |
tmpname = '%s/releasing.%s.tmp' % (self.path, rand_chars(20)) |
364 |
# gotta own it to unlock
|
|
365 |
self.confirm() |
|
366 |
self.transport.rename(self._held_dir, tmpname) |
|
367 |
self._lock_held = False |
|
368 |
self.transport.delete(tmpname + self.__INFO_NAME) |
|
2555.3.14
by Martin Pool
Better handling in LockDir of rename that moves one directory within another |
369 |
try: |
370 |
self.transport.rmdir(tmpname) |
|
371 |
except DirectoryNotEmpty, e: |
|
372 |
# There might have been junk left over by a rename that moved
|
|
373 |
# another locker within the 'held' directory. do a slower
|
|
374 |
# deletion where we list the directory and remove everything
|
|
375 |
# within it.
|
|
376 |
#
|
|
377 |
# Maybe this should be broader to allow for ftp servers with
|
|
378 |
# non-specific error messages?
|
|
379 |
self._trace("doing recursive deletion of non-empty directory " |
|
380 |
"%s", tmpname) |
|
381 |
self.transport.delete_tree(tmpname) |
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
382 |
self._trace("... unlock succeeded after %dms", |
383 |
(time.time() - start_time) * 1000) |
|
3331.3.2
by Robert Collins
Polish on lock hooks to be easier to use. |
384 |
result = lock.LockResult(self.transport.abspath(self.path), |
4327.1.2
by Vincent Ladeuil
Introduce a new lock_broken hook. |
385 |
old_nonce) |
3331.3.6
by Martin Pool
merge trunk |
386 |
for hook in self.hooks['lock_released']: |
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
387 |
hook(result) |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
388 |
|
1687.1.5
by Robert Collins
Add break_lock utility function to LockDir. |
389 |
def break_lock(self): |
390 |
"""Break a lock not held by this instance of LockDir.
|
|
391 |
||
5425.4.13
by Martin Pool
Steal locks of dead local processes, with a user warning |
392 |
This is a UI centric function: it uses the ui.ui_factory to
|
1687.1.5
by Robert Collins
Add break_lock utility function to LockDir. |
393 |
prompt for input if a lock is detected and there is any doubt about
|
5425.4.13
by Martin Pool
Steal locks of dead local processes, with a user warning |
394 |
it possibly being still active. force_break is the non-interactive
|
395 |
version.
|
|
5416.2.3
by Martin Pool
Better test for break-lock; show lock URLs as they're broken |
396 |
|
397 |
:returns: LockResult for the broken lock.
|
|
1687.1.5
by Robert Collins
Add break_lock utility function to LockDir. |
398 |
"""
|
1687.1.6
by Robert Collins
Extend LockableFiles to support break_lock() calls. |
399 |
self._check_not_locked() |
4634.161.1
by Andrew Bennetts
Add LockCorrupt error, and use it to provide nicer handling of unparseable lock/held/info files. |
400 |
try: |
401 |
holder_info = self.peek() |
|
402 |
except LockCorrupt, e: |
|
403 |
# The lock info is corrupt.
|
|
5425.4.23
by Martin Pool
Resolve conflicts against trunk |
404 |
if ui.ui_factory.get_boolean(u"Break (corrupt %r)" % (self,)): |
4634.161.1
by Andrew Bennetts
Add LockCorrupt error, and use it to provide nicer handling of unparseable lock/held/info files. |
405 |
self.force_break_corrupt(e.file_data) |
406 |
return
|
|
1687.1.5
by Robert Collins
Add break_lock utility function to LockDir. |
407 |
if holder_info is not None: |
5425.4.13
by Martin Pool
Steal locks of dead local processes, with a user warning |
408 |
if ui.ui_factory.confirm_action( |
5425.6.2
by Martin Pool
Lock holder human version is now unicode, not str |
409 |
u"Break %(lock_info)s", |
5425.4.30
by Martin Pool
Restore confirmation name that was used to force breaking |
410 |
'bzrlib.lockdir.break', |
5425.6.2
by Martin Pool
Lock holder human version is now unicode, not str |
411 |
dict(lock_info=unicode(holder_info))): |
5416.2.3
by Martin Pool
Better test for break-lock; show lock URLs as they're broken |
412 |
result = self.force_break(holder_info) |
5425.4.13
by Martin Pool
Steal locks of dead local processes, with a user warning |
413 |
ui.ui_factory.show_message( |
5416.2.3
by Martin Pool
Better test for break-lock; show lock URLs as they're broken |
414 |
"Broke lock %s" % result.lock_url) |
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
415 |
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
416 |
def force_break(self, dead_holder_info): |
1553.5.25
by Martin Pool
New LockDir.force_break and simple test case |
417 |
"""Release a lock held by another process.
|
418 |
||
419 |
WARNING: This should only be used when the other process is dead; if
|
|
420 |
it still thinks it has the lock there will be two concurrent writers.
|
|
421 |
In general the user's approval should be sought for lock breaks.
|
|
422 |
||
423 |
After the lock is broken it will not be held by any process.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
424 |
It is possible that another process may sneak in and take the
|
1553.5.25
by Martin Pool
New LockDir.force_break and simple test case |
425 |
lock before the breaking process acquires it.
|
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
426 |
|
5425.6.3
by Martin Pool
PEP8 cleanups |
427 |
:param dead_holder_info:
|
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
428 |
Must be the result of a previous LockDir.peek() call; this is used
|
429 |
to check that it's still held by the same process that the user
|
|
430 |
decided was dead. If this is not the current holder,
|
|
431 |
LockBreakMismatch is raised.
|
|
5425.4.4
by Martin Pool
merge trunk |
432 |
|
5416.2.3
by Martin Pool
Better test for break-lock; show lock URLs as they're broken |
433 |
:returns: LockResult for the broken lock.
|
1553.5.25
by Martin Pool
New LockDir.force_break and simple test case |
434 |
"""
|
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
435 |
if not isinstance(dead_holder_info, LockHeldInfo): |
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
436 |
raise ValueError("dead_holder_info: %r" % dead_holder_info) |
1687.1.6
by Robert Collins
Extend LockableFiles to support break_lock() calls. |
437 |
self._check_not_locked() |
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
438 |
current_info = self.peek() |
439 |
if current_info is None: |
|
1553.5.26
by Martin Pool
Breaking an already-released lock should just succeed |
440 |
# must have been recently released
|
441 |
return
|
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
442 |
if current_info != dead_holder_info: |
443 |
raise LockBreakMismatch(self, current_info, dead_holder_info) |
|
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
444 |
tmpname = '%s/broken.%s.tmp' % (self.path, rand_chars(20)) |
445 |
self.transport.rename(self._held_dir, tmpname) |
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
446 |
# check that we actually broke the right lock, not someone else;
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
447 |
# there's a small race window between checking it and doing the
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
448 |
# rename.
|
1553.5.33
by Martin Pool
LockDir review comment fixes |
449 |
broken_info_path = tmpname + self.__INFO_NAME |
450 |
broken_info = self._read_info_file(broken_info_path) |
|
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
451 |
if broken_info != dead_holder_info: |
452 |
raise LockBreakMismatch(self, broken_info, dead_holder_info) |
|
453 |
self.transport.delete(broken_info_path) |
|
1553.5.25
by Martin Pool
New LockDir.force_break and simple test case |
454 |
self.transport.rmdir(tmpname) |
4327.1.2
by Vincent Ladeuil
Introduce a new lock_broken hook. |
455 |
result = lock.LockResult(self.transport.abspath(self.path), |
456 |
current_info.get('nonce')) |
|
457 |
for hook in self.hooks['lock_broken']: |
|
458 |
hook(result) |
|
5416.2.3
by Martin Pool
Better test for break-lock; show lock URLs as they're broken |
459 |
return result |
1553.5.25
by Martin Pool
New LockDir.force_break and simple test case |
460 |
|
4634.161.1
by Andrew Bennetts
Add LockCorrupt error, and use it to provide nicer handling of unparseable lock/held/info files. |
461 |
def force_break_corrupt(self, corrupt_info_lines): |
462 |
"""Release a lock that has been corrupted.
|
|
5425.6.3
by Martin Pool
PEP8 cleanups |
463 |
|
4634.161.1
by Andrew Bennetts
Add LockCorrupt error, and use it to provide nicer handling of unparseable lock/held/info files. |
464 |
This is very similar to force_break, it except it doesn't assume that
|
465 |
self.peek() can work.
|
|
5425.6.3
by Martin Pool
PEP8 cleanups |
466 |
|
4634.161.1
by Andrew Bennetts
Add LockCorrupt error, and use it to provide nicer handling of unparseable lock/held/info files. |
467 |
:param corrupt_info_lines: the lines of the corrupted info file, used
|
468 |
to check that the lock hasn't changed between reading the (corrupt)
|
|
469 |
info file and calling force_break_corrupt.
|
|
470 |
"""
|
|
471 |
# XXX: this copes with unparseable info files, but what about missing
|
|
472 |
# info files? Or missing lock dirs?
|
|
473 |
self._check_not_locked() |
|
474 |
tmpname = '%s/broken.%s.tmp' % (self.path, rand_chars(20)) |
|
475 |
self.transport.rename(self._held_dir, tmpname) |
|
476 |
# check that we actually broke the right lock, not someone else;
|
|
477 |
# there's a small race window between checking it and doing the
|
|
478 |
# rename.
|
|
479 |
broken_info_path = tmpname + self.__INFO_NAME |
|
5599.2.1
by John Arbash Meinel
Fix breaking corrupt lock files on Windows. |
480 |
broken_content = self.transport.get_bytes(broken_info_path) |
481 |
broken_lines = osutils.split_lines(broken_content) |
|
4634.161.1
by Andrew Bennetts
Add LockCorrupt error, and use it to provide nicer handling of unparseable lock/held/info files. |
482 |
if broken_lines != corrupt_info_lines: |
483 |
raise LockBreakMismatch(self, broken_lines, corrupt_info_lines) |
|
484 |
self.transport.delete(broken_info_path) |
|
485 |
self.transport.rmdir(tmpname) |
|
486 |
result = lock.LockResult(self.transport.abspath(self.path)) |
|
487 |
for hook in self.hooks['lock_broken']: |
|
488 |
hook(result) |
|
489 |
||
1687.1.6
by Robert Collins
Extend LockableFiles to support break_lock() calls. |
490 |
def _check_not_locked(self): |
491 |
"""If the lock is held by this instance, raise an error."""
|
|
492 |
if self._lock_held: |
|
493 |
raise AssertionError("can't break own lock: %r" % self) |
|
494 |
||
1553.5.20
by Martin Pool
Start adding LockDir.confirm() method |
495 |
def confirm(self): |
496 |
"""Make sure that the lock is still held by this locker.
|
|
497 |
||
498 |
This should only fail if the lock was broken by user intervention,
|
|
499 |
or if the lock has been affected by a bug.
|
|
500 |
||
501 |
If the lock is not thought to be held, raises LockNotHeld. If
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
502 |
the lock is thought to be held but has been broken, raises
|
1553.5.20
by Martin Pool
Start adding LockDir.confirm() method |
503 |
LockBroken.
|
504 |
"""
|
|
505 |
if not self._lock_held: |
|
506 |
raise LockNotHeld(self) |
|
1553.5.23
by Martin Pool
Start LockDir.confirm method and LockBroken exception |
507 |
info = self.peek() |
508 |
if info is None: |
|
509 |
# no lock there anymore!
|
|
510 |
raise LockBroken(self) |
|
511 |
if info.get('nonce') != self.nonce: |
|
512 |
# there is a lock, but not ours
|
|
513 |
raise LockBroken(self) |
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
514 |
|
1553.5.33
by Martin Pool
LockDir review comment fixes |
515 |
def _read_info_file(self, path): |
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
516 |
"""Read one given info file.
|
517 |
||
518 |
peek() reads the info file of the lock holder, if any.
|
|
519 |
"""
|
|
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
520 |
return LockHeldInfo.from_info_file_bytes( |
521 |
self.transport.get_bytes(path)) |
|
1553.5.20
by Martin Pool
Start adding LockDir.confirm() method |
522 |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
523 |
def peek(self): |
524 |
"""Check if the lock is held by anyone.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
525 |
|
4634.138.2
by Martin Pool
Copy with lock info file being empty |
526 |
If it is held, this returns the lock info structure as a dict
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
527 |
which contains some information about the current lock holder.
|
528 |
Otherwise returns None.
|
|
529 |
"""
|
|
530 |
try: |
|
1553.5.58
by Martin Pool
Change LockDirs to format "lock-name/held/info" |
531 |
info = self._read_info_file(self._held_info_path) |
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
532 |
self._trace("peek -> held") |
1553.5.27
by Martin Pool
Confirm that only the intended holder of a lock was broken. |
533 |
return info |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
534 |
except NoSuchFile, e: |
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
535 |
self._trace("peek -> not held") |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
536 |
|
1993.1.2
by John Arbash Meinel
Use Stanza.to_string() rather than a RioWriter |
537 |
def _prepare_info(self): |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
538 |
"""Write information about a pending lock to a temporary file.
|
539 |
"""
|
|
540 |
||
2555.3.13
by Martin Pool
LockDir cleanups |
541 |
def attempt_lock(self): |
542 |
"""Take the lock; fail if it's already held.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
543 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
544 |
If you wish to block until the lock can be obtained, call wait_lock()
|
545 |
instead.
|
|
546 |
||
547 |
:return: The lock token.
|
|
548 |
:raises LockContention: if the lock is held by someone else.
|
|
549 |
"""
|
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
550 |
if self._fake_read_lock: |
551 |
raise LockContention(self) |
|
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
552 |
result = self._attempt_lock() |
3331.3.2
by Robert Collins
Polish on lock hooks to be easier to use. |
553 |
hook_result = lock.LockResult(self.transport.abspath(self.path), |
554 |
self.nonce) |
|
3331.3.6
by Martin Pool
merge trunk |
555 |
for hook in self.hooks['lock_acquired']: |
3331.3.1
by Robert Collins
* ``LockDir`` lock acquisition and release now trigger hooks allowing |
556 |
hook(hook_result) |
557 |
return result |
|
2555.3.13
by Martin Pool
LockDir cleanups |
558 |
|
5609.30.1
by Martin Pool
Avoid UnboundLocalError reporting url when wait_lock fails |
559 |
def lock_url_for_display(self): |
560 |
"""Give a nicely-printable representation of the URL of this lock."""
|
|
561 |
# As local lock urls are correct we display them.
|
|
562 |
# We avoid displaying remote lock urls.
|
|
563 |
lock_url = self.transport.abspath(self.path) |
|
564 |
if lock_url.startswith('file://'): |
|
565 |
lock_url = lock_url.split('.bzr/')[0] |
|
566 |
else: |
|
567 |
lock_url = '' |
|
568 |
return lock_url |
|
569 |
||
2555.3.13
by Martin Pool
LockDir cleanups |
570 |
def wait_lock(self, timeout=None, poll=None, max_attempts=None): |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
571 |
"""Wait a certain period for a lock.
|
572 |
||
573 |
If the lock can be acquired within the bounded time, it
|
|
574 |
is taken and this returns. Otherwise, LockContention
|
|
575 |
is raised. Either way, this function should return within
|
|
576 |
approximately `timeout` seconds. (It may be a bit more if
|
|
577 |
a transport operation takes a long time to complete.)
|
|
2555.3.5
by Martin Pool
Return token directly from LockDir.acquire to avoid unnecessary peek() |
578 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
579 |
:param timeout: Approximate maximum amount of time to wait for the
|
580 |
lock, in seconds.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
581 |
|
2555.3.13
by Martin Pool
LockDir cleanups |
582 |
:param poll: Delay in seconds between retrying the lock.
|
583 |
||
584 |
:param max_attempts: Maximum number of times to try to lock.
|
|
585 |
||
2555.3.5
by Martin Pool
Return token directly from LockDir.acquire to avoid unnecessary peek() |
586 |
:return: The lock token.
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
587 |
"""
|
1957.1.17
by John Arbash Meinel
Change tests that expect locking to fail to timeout sooner. |
588 |
if timeout is None: |
589 |
timeout = _DEFAULT_TIMEOUT_SECONDS |
|
590 |
if poll is None: |
|
591 |
poll = _DEFAULT_POLL_SECONDS |
|
2555.3.13
by Martin Pool
LockDir cleanups |
592 |
# XXX: the transport interface doesn't let us guard against operations
|
593 |
# there taking a long time, so the total elapsed time or poll interval
|
|
594 |
# may be more than was requested.
|
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
595 |
deadline = time.time() + timeout |
1957.1.13
by John Arbash Meinel
Change to reporting the time when we will stop trying to grab the lock |
596 |
deadline_str = None |
1957.1.7
by John Arbash Meinel
Add the ability to report if the lock changes from underneath you |
597 |
last_info = None |
2555.3.13
by Martin Pool
LockDir cleanups |
598 |
attempt_count = 0 |
5609.30.1
by Martin Pool
Avoid UnboundLocalError reporting url when wait_lock fails |
599 |
lock_url = self.lock_url_for_display() |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
600 |
while True: |
2555.3.13
by Martin Pool
LockDir cleanups |
601 |
attempt_count += 1 |
1553.5.12
by Martin Pool
New LockDir locking mechanism |
602 |
try: |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
603 |
return self.attempt_lock() |
604 |
except LockContention: |
|
605 |
# possibly report the blockage, then try again
|
|
606 |
pass
|
|
607 |
# TODO: In a few cases, we find out that there's contention by
|
|
608 |
# reading the held info and observing that it's not ours. In
|
|
609 |
# those cases it's a bit redundant to read it again. However,
|
|
610 |
# the normal case (??) is that the rename fails and so we
|
|
611 |
# don't know who holds the lock. For simplicity we peek
|
|
612 |
# always.
|
|
613 |
new_info = self.peek() |
|
614 |
if new_info is not None and new_info != last_info: |
|
615 |
if last_info is None: |
|
616 |
start = 'Unable to obtain' |
|
1957.1.9
by John Arbash Meinel
Change default timeouts, and report differently the first failure |
617 |
else: |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
618 |
start = 'Lock owner changed for' |
619 |
last_info = new_info |
|
5425.4.18
by Martin Pool
Cleaner and consistent user-orientend representation of LockHeldInfo |
620 |
msg = u'%s lock %s %s.' % (start, lock_url, new_info) |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
621 |
if deadline_str is None: |
622 |
deadline_str = time.strftime('%H:%M:%S', |
|
5425.4.18
by Martin Pool
Cleaner and consistent user-orientend representation of LockHeldInfo |
623 |
time.localtime(deadline)) |
5284.6.3
by Parth Malwankar
fixed tests. closed review comments by mgz. |
624 |
if timeout > 0: |
625 |
msg += ('\nWill continue to try until %s, unless ' |
|
5425.4.18
by Martin Pool
Cleaner and consistent user-orientend representation of LockHeldInfo |
626 |
'you press Ctrl-C.'
|
627 |
% deadline_str) |
|
5284.6.3
by Parth Malwankar
fixed tests. closed review comments by mgz. |
628 |
msg += '\nSee "bzr help break-lock" for more.' |
5425.4.18
by Martin Pool
Cleaner and consistent user-orientend representation of LockHeldInfo |
629 |
self._report_function(msg) |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
630 |
if (max_attempts is not None) and (attempt_count >= max_attempts): |
631 |
self._trace("exceeded %d attempts") |
|
632 |
raise LockContention(self) |
|
633 |
if time.time() + poll < deadline: |
|
634 |
self._trace("waiting %ss", poll) |
|
635 |
time.sleep(poll) |
|
636 |
else: |
|
5284.6.4
by Parth Malwankar
local urls are now shows as they are valid. |
637 |
# As timeout is always 0 for remote locks
|
638 |
# this block is applicable only for local
|
|
639 |
# lock contention
|
|
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
640 |
self._trace("timeout after waiting %ss", timeout) |
5284.6.9
by Parth Malwankar
cleaner handling of lock_url display. |
641 |
raise LockContention('(local)', lock_url) |
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
642 |
|
2018.5.75
by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place. |
643 |
def leave_in_place(self): |
644 |
self._locked_via_token = True |
|
645 |
||
646 |
def dont_leave_in_place(self): |
|
647 |
self._locked_via_token = False |
|
1553.5.12
by Martin Pool
New LockDir locking mechanism |
648 |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
649 |
def lock_write(self, token=None): |
650 |
"""Wait for and acquire the lock.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
651 |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
652 |
:param token: if this is already locked, then lock_write will fail
|
653 |
unless the token matches the existing lock.
|
|
654 |
:returns: a token if this instance supports tokens, otherwise None.
|
|
655 |
:raises TokenLockingNotSupported: when a token is given but this
|
|
656 |
instance doesn't support using token locks.
|
|
2279.7.7
by Andrew Bennetts
LockDir, Repository and Branch lock token changes from the hpss branch. |
657 |
:raises MismatchedToken: if the specified token doesn't match the token
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
658 |
of the existing lock.
|
2018.5.145
by Andrew Bennetts
Add a brief explanation of what tokens are used for to lock_write docstrings. |
659 |
|
660 |
A token should be passed in if you know that you have locked the object
|
|
661 |
some other way, and need to synchronise this object's state with that
|
|
662 |
fact.
|
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
663 |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
664 |
XXX: docstring duplicated from LockableFiles.lock_write.
|
665 |
"""
|
|
666 |
if token is not None: |
|
667 |
self.validate_token(token) |
|
2018.5.75
by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place. |
668 |
self.nonce = token |
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
669 |
self._lock_held = True |
670 |
self._locked_via_token = True |
|
2018.5.78
by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the |
671 |
return token |
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
672 |
else: |
2555.3.5
by Martin Pool
Return token directly from LockDir.acquire to avoid unnecessary peek() |
673 |
return self.wait_lock() |
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
674 |
|
675 |
def lock_read(self): |
|
1759.2.1
by Jelmer Vernooij
Fix some types (found using aspell). |
676 |
"""Compatibility-mode shared lock.
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
677 |
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
678 |
LockDir doesn't support shared read-only locks, so this
|
1553.5.54
by Martin Pool
Add LockDir.read_lock fake method |
679 |
just pretends that the lock is taken but really does nothing.
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
680 |
"""
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
681 |
# At the moment Branches are commonly locked for read, but
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
682 |
# we can't rely on that remotely. Once this is cleaned up,
|
3943.8.1
by Marius Kruger
remove all trailing whitespace from bzr source |
683 |
# reenable this warning to prevent it coming back in
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
684 |
# -- mbp 20060303
|
685 |
## warn("LockDir.lock_read falls back to write lock")
|
|
1553.5.54
by Martin Pool
Add LockDir.read_lock fake method |
686 |
if self._lock_held or self._fake_read_lock: |
687 |
raise LockContention(self) |
|
688 |
self._fake_read_lock = True |
|
1553.5.44
by Martin Pool
LockableFiles can now call LockDir directly |
689 |
|
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
690 |
def validate_token(self, token): |
691 |
if token is not None: |
|
692 |
info = self.peek() |
|
693 |
if info is None: |
|
694 |
# Lock isn't held
|
|
695 |
lock_token = None |
|
696 |
else: |
|
697 |
lock_token = info.get('nonce') |
|
698 |
if token != lock_token: |
|
699 |
raise errors.TokenMismatch(token, lock_token) |
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
700 |
else: |
2555.3.16
by Martin Pool
Clean up patch to make divergence from mainline smaller. |
701 |
self._trace("revalidated by token %r", token) |
2279.7.1
by Andrew Bennetts
``LockableFiles.lock_write()`` now accepts a ``token`` keyword argument, so that |
702 |
|
2555.3.3
by Martin Pool
Simple lock tracing in LockDir |
703 |
def _trace(self, format, *args): |
704 |
if 'lock' not in debug.debug_flags: |
|
705 |
return
|
|
706 |
mutter(str(self) + ": " + (format % args)) |
|
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
707 |
|
5425.4.15
by Martin Pool
Add steal_dead_locks option letting automatic lock-breaking be turned off |
708 |
def get_config(self): |
709 |
"""Get the configuration that governs this lockdir."""
|
|
710 |
# XXX: This really should also use the locationconfig at least, but
|
|
711 |
# that seems a bit hard to hook up at the moment. -- mbp 20110329
|
|
712 |
return config.GlobalConfig() |
|
713 |
||
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
714 |
|
715 |
class LockHeldInfo(object): |
|
716 |
"""The information recorded about a held lock.
|
|
717 |
||
718 |
This information is recorded into the lock when it's taken, and it can be
|
|
5425.6.3
by Martin Pool
PEP8 cleanups |
719 |
read back by any process with access to the lockdir. It can be used, for
|
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
720 |
example, to tell the user who holds the lock, or to try to detect whether
|
721 |
the lock holder is still alive.
|
|
722 |
||
723 |
Prior to bzr 2.4 a simple dict was used instead of an object.
|
|
724 |
"""
|
|
725 |
||
726 |
def __init__(self, info_dict): |
|
727 |
self.info_dict = info_dict |
|
728 |
||
5425.4.8
by Martin Pool
Basic detection of known dead lock holders |
729 |
def __repr__(self): |
5425.4.18
by Martin Pool
Cleaner and consistent user-orientend representation of LockHeldInfo |
730 |
"""Return a debugging representation of this object."""
|
5425.4.8
by Martin Pool
Basic detection of known dead lock holders |
731 |
return "%s(%r)" % (self.__class__.__name__, self.info_dict) |
732 |
||
5425.4.18
by Martin Pool
Cleaner and consistent user-orientend representation of LockHeldInfo |
733 |
def __unicode__(self): |
734 |
"""Return a user-oriented description of this object."""
|
|
735 |
d = self.to_readable_dict() |
|
736 |
return ( |
|
737 |
u'held by %(user)s on %(hostname)s (process #%(pid)s), ' |
|
738 |
u'acquired %(time_ago)s' % d) |
|
739 |
||
740 |
def to_readable_dict(self): |
|
741 |
"""Turn the holder info into a dict of human-readable attributes.
|
|
742 |
||
743 |
For example, the start time is presented relative to the current time,
|
|
744 |
rather than as seconds since the epoch.
|
|
5425.6.3
by Martin Pool
PEP8 cleanups |
745 |
|
746 |
Returns a list of [user, hostname, pid, time_ago] all as readable
|
|
5425.4.18
by Martin Pool
Cleaner and consistent user-orientend representation of LockHeldInfo |
747 |
strings.
|
748 |
"""
|
|
749 |
start_time = self.info_dict.get('start_time') |
|
750 |
if start_time is None: |
|
751 |
time_ago = '(unknown)' |
|
752 |
else: |
|
753 |
time_ago = format_delta( |
|
754 |
time.time() - int(self.info_dict['start_time'])) |
|
755 |
user = self.info_dict.get('user', '<unknown>') |
|
756 |
hostname = self.info_dict.get('hostname', '<unknown>') |
|
757 |
pid = self.info_dict.get('pid', '<unknown>') |
|
758 |
return dict( |
|
759 |
user=user, |
|
760 |
hostname=hostname, |
|
761 |
pid=pid, |
|
762 |
time_ago=time_ago) |
|
763 |
||
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
764 |
def get(self, field_name): |
765 |
"""Return the contents of a field from the lock info, or None."""
|
|
766 |
return self.info_dict.get(field_name) |
|
767 |
||
768 |
@classmethod
|
|
5425.4.10
by Martin Pool
Allow storing extra info into lockdir info files to aid testing |
769 |
def for_this_process(cls, extra_holder_info): |
5425.4.6
by Martin Pool
Move preparation of new locks into LockHeldInfo |
770 |
"""Return a new LockHeldInfo for a lock taken by this process.
|
771 |
"""
|
|
5425.4.10
by Martin Pool
Allow storing extra info into lockdir info files to aid testing |
772 |
info = dict( |
5425.4.6
by Martin Pool
Move preparation of new locks into LockHeldInfo |
773 |
hostname=get_host_name(), |
774 |
pid=str(os.getpid()), |
|
5425.4.10
by Martin Pool
Allow storing extra info into lockdir info files to aid testing |
775 |
nonce=rand_chars(20), |
5425.4.6
by Martin Pool
Move preparation of new locks into LockHeldInfo |
776 |
start_time=str(int(time.time())), |
5425.4.7
by Martin Pool
Add and test is_locked_by_this_process |
777 |
user=get_username_for_lock_info(), |
5425.4.10
by Martin Pool
Allow storing extra info into lockdir info files to aid testing |
778 |
)
|
779 |
if extra_holder_info is not None: |
|
780 |
info.update(extra_holder_info) |
|
781 |
return cls(info) |
|
5425.4.6
by Martin Pool
Move preparation of new locks into LockHeldInfo |
782 |
|
783 |
def to_bytes(self): |
|
784 |
s = rio.Stanza(**self.info_dict) |
|
785 |
return s.to_string() |
|
786 |
||
787 |
@classmethod
|
|
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
788 |
def from_info_file_bytes(cls, info_file_bytes): |
789 |
"""Construct from the contents of the held file."""
|
|
790 |
lines = osutils.split_lines(info_file_bytes) |
|
791 |
try: |
|
792 |
stanza = rio.read_stanza(lines) |
|
793 |
except ValueError, e: |
|
794 |
mutter('Corrupt lock info file: %r', lines) |
|
795 |
raise LockCorrupt("could not parse lock info file: " + str(e), |
|
796 |
lines) |
|
797 |
if stanza is None: |
|
798 |
# see bug 185013; we fairly often end up with the info file being
|
|
799 |
# empty after an interruption; we could log a message here but
|
|
800 |
# there may not be much we can say
|
|
801 |
return cls({}) |
|
802 |
else: |
|
803 |
return cls(stanza.as_dict()) |
|
804 |
||
805 |
def __cmp__(self, other): |
|
5425.4.7
by Martin Pool
Add and test is_locked_by_this_process |
806 |
"""Value comparison of lock holders."""
|
5425.4.3
by Martin Pool
Represent lock held info as an object, not just a dict |
807 |
return ( |
808 |
cmp(type(self), type(other)) |
|
809 |
or cmp(self.info_dict, other.info_dict)) |
|
5425.4.7
by Martin Pool
Add and test is_locked_by_this_process |
810 |
|
811 |
def is_locked_by_this_process(self): |
|
5425.6.3
by Martin Pool
PEP8 cleanups |
812 |
"""True if this process seems to be the current lock holder."""
|
5425.4.7
by Martin Pool
Add and test is_locked_by_this_process |
813 |
return ( |
814 |
self.get('hostname') == get_host_name() |
|
815 |
and self.get('pid') == str(os.getpid()) |
|
816 |
and self.get('user') == get_username_for_lock_info()) |
|
817 |
||
5425.4.8
by Martin Pool
Basic detection of known dead lock holders |
818 |
def is_lock_holder_known_dead(self): |
819 |
"""True if the lock holder process is known to be dead.
|
|
820 |
||
821 |
False if it's either known to be still alive, or if we just can't tell.
|
|
822 |
||
823 |
We can be fairly sure the lock holder is dead if it declared the same
|
|
824 |
hostname and there is no process with the given pid alive. If people
|
|
825 |
have multiple machines with the same hostname this may cause trouble.
|
|
826 |
||
827 |
This doesn't check whether the lock holder is in fact the same process
|
|
828 |
calling this method. (In that case it will return true.)
|
|
829 |
"""
|
|
830 |
if self.get('hostname') != get_host_name(): |
|
831 |
return False |
|
5425.4.9
by Martin Pool
Defensively handle 'localhost' in detecting dead locks |
832 |
if self.get('hostname') == 'localhost': |
833 |
# Too ambiguous.
|
|
834 |
return False |
|
5425.6.1
by Martin Pool
Only detect as stale locks from the same user |
835 |
if self.get('user') != get_username_for_lock_info(): |
836 |
# Could well be another local process by a different user, but
|
|
837 |
# just to be safe we won't conclude about this either.
|
|
838 |
return False |
|
5425.4.8
by Martin Pool
Basic detection of known dead lock holders |
839 |
pid_str = self.info_dict.get('pid', None) |
840 |
if not pid_str: |
|
841 |
mutter("no pid recorded in %r" % (self, )) |
|
842 |
return False |
|
843 |
try: |
|
844 |
pid = int(pid_str) |
|
845 |
except ValueError: |
|
5425.6.3
by Martin Pool
PEP8 cleanups |
846 |
mutter("can't parse pid %r from %r" |
5425.4.8
by Martin Pool
Basic detection of known dead lock holders |
847 |
% (pid_str, self)) |
848 |
return False |
|
5425.5.5
by Martin
Quick implementation of dead process detection on win32 |
849 |
return osutils.is_local_pid_dead(pid) |
5425.4.8
by Martin Pool
Basic detection of known dead lock holders |
850 |
|
5425.4.7
by Martin Pool
Add and test is_locked_by_this_process |
851 |
|
852 |
def get_username_for_lock_info(): |
|
853 |
"""Get a username suitable for putting into a lock.
|
|
854 |
||
855 |
It's ok if what's written here is not a proper email address as long
|
|
856 |
as it gives some clue who the user is.
|
|
857 |
"""
|
|
858 |
try: |
|
5425.4.15
by Martin Pool
Add steal_dead_locks option letting automatic lock-breaking be turned off |
859 |
return config.GlobalConfig().username() |
5425.4.7
by Martin Pool
Add and test is_locked_by_this_process |
860 |
except errors.NoWhoami: |
861 |
return osutils.getuser_unicode() |