1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
//! Mutexes can deadlock each other, but you can avoid this by always acquiring your locks in a
//! consistent order. This crate provides tracing to ensure that you do.
//!
//! This crate tracks a virtual "stack" of locks that the current thread holds, and whenever a new
//! lock is acquired, a dependency is created from the last lock to the new one. These dependencies
//! together form a graph. As long as that graph does not contain any cycles, your program is
//! guaranteed to never deadlock.
//!
//! # Panics
//!
//! The primary method by which this crate signals an invalid lock acquisition order is by
//! panicking. When a cycle is created in the dependency graph when acquiring a lock, the thread
//! will instead panic. This panic will not poison the underlying mutex.
//!
//! This conflicting dependency is not added to the graph, so future attempts at locking should
//! succeed as normal.
//!
//! # Structure
//!
//! Each module in this crate exposes wrappers for a specific base-mutex with dependency trakcing
//! added. This includes [`stdsync`] which provides wrappers for the base locks in the standard
//! library, and more depending on enabled compile-time features. More back-ends may be added as
//! features in the future.
//!
//! # Feature flags
//!
//! `tracing-mutex` uses feature flags to reduce the impact of this crate on both your compile time
//! and runtime overhead. Below are the available flags. Modules are annotated with the features
//! they require.
//!
//! - `backtraces`: Enables capturing backtraces of mutex dependencies, to make it easier to
//!   determine what sequence of events would trigger a deadlock. This is enabled by default, but if
//!   the performance overhead is unaccceptable, it can be disabled by disabling default features.
//!
//! - `lockapi`: Enables the wrapper lock for [`lock_api`][lock_api] locks
//!
//! - `parkinglot`: Enables wrapper types for [`parking_lot`][parking_lot] mutexes
//!
//! # Performance considerations
//!
//! Tracing a mutex adds overhead to certain mutex operations in order to do the required
//! bookkeeping. The following actions have the following overhead.
//!
//! - **Acquiring a lock** locks the global dependency graph temporarily to check if the new lock
//!   would introduce a cyclic dependency. This crate uses the algorithm proposed in ["A Dynamic
//!   Topological Sort Algorithm for Directed Acyclic Graphs" by David J. Pearce and Paul H.J.
//!   Kelly][paper] to detect cycles as efficently as possible. In addition, a thread local lock set
//!   is updated with the new lock.
//!
//! - **Releasing a lock** updates a thread local lock set to remove the released lock.
//!
//! - **Allocating a lock** performs an atomic update to a shared counter.
//!
//! - **Deallocating a mutex** temporarily locks the global dependency graph to remove the lock
//!   entry in the dependency graph.
//!
//! These operations have been reasonably optimized, but the performance penalty may yet be too much
//! for production use. In those cases, it may be beneficial to instead use debug-only versions
//! (such as [`stdsync::Mutex`]) which evaluate to a tracing mutex when debug assertions are
//! enabled, and to the underlying mutex when they're not.
//!
//! For ease of debugging, this crate will, by default, capture a backtrace when establishing a new
//! dependency between two mutexes. This has an additional overhead of over 60%. If this additional
//! debugging aid is not required, it can be disabled by disabling default features.
//!
//! [paper]: https://whileydave.com/publications/pk07_jea/
//! [lock_api]: https://docs.rs/lock_api/0.4/lock_api/index.html
//! [parking_lot]: https://docs.rs/parking_lot/0.12.1/parking_lot/
#![cfg_attr(docsrs, feature(doc_cfg))]
use std::cell::RefCell;
use std::fmt;
use std::marker::PhantomData;
use std::ops::Deref;
use std::ops::DerefMut;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering;
use std::sync::Mutex;
use std::sync::MutexGuard;
use std::sync::OnceLock;
use std::sync::PoisonError;

#[cfg(feature = "lockapi")]
#[cfg_attr(docsrs, doc(cfg(feature = "lockapi")))]
pub use lock_api;
#[cfg(feature = "parkinglot")]
#[cfg_attr(docsrs, doc(cfg(feature = "parkinglot")))]
pub use parking_lot;
use reporting::Dep;
use reporting::Reportable;

use crate::graph::DiGraph;

mod graph;
#[cfg(feature = "lockapi")]
#[cfg_attr(docsrs, doc(cfg(feature = "lockapi")))]
pub mod lockapi;
#[cfg(feature = "parkinglot")]
#[cfg_attr(docsrs, doc(cfg(feature = "parkinglot")))]
pub mod parkinglot;
mod reporting;
pub mod stdsync;

thread_local! {
    /// Stack to track which locks are held
    ///
    /// Assuming that locks are roughly released in the reverse order in which they were acquired,
    /// a stack should be more efficient to keep track of the current state than a set would be.
    static HELD_LOCKS: RefCell<Vec<usize>> = RefCell::new(Vec::new());
}

/// Dedicated ID type for Mutexes
///
/// # Unstable
///
/// This type is currently private to prevent usage while the exact implementation is figured out,
/// but it will likely be public in the future.
struct MutexId(usize);

impl MutexId {
    /// Get a new, unique, mutex ID.
    ///
    /// This ID is guaranteed to be unique within the runtime of the program.
    ///
    /// # Panics
    ///
    /// This function may panic when there are no more mutex IDs available. The number of mutex ids
    /// is `usize::MAX - 1` which should be plenty for most practical applications.
    pub fn new() -> Self {
        // Counter for Mutex IDs. Atomic avoids the need for locking.
        static ID_SEQUENCE: AtomicUsize = AtomicUsize::new(0);

        ID_SEQUENCE
            .fetch_update(Ordering::SeqCst, Ordering::SeqCst, |id| id.checked_add(1))
            .map(Self)
            .expect("Mutex ID wraparound happened, results unreliable")
    }

    pub fn value(&self) -> usize {
        self.0
    }

    /// Get a borrowed guard for this lock.
    ///
    /// This method adds checks adds this Mutex ID to the dependency graph as needed, and adds the
    /// lock to the list of
    ///
    /// # Panics
    ///
    /// This method panics if the new dependency would introduce a cycle.
    pub fn get_borrowed(&self) -> BorrowedMutex {
        self.mark_held();
        BorrowedMutex {
            id: self,
            _not_send: PhantomData,
        }
    }

    /// Mark this lock as held for the purposes of dependency tracking.
    ///
    /// # Panics
    ///
    /// This method panics if the new dependency would introduce a cycle.
    pub fn mark_held(&self) {
        let opt_cycle = HELD_LOCKS.with(|locks| {
            if let Some(&previous) = locks.borrow().last() {
                let mut graph = get_dependency_graph();

                graph.add_edge(previous, self.value(), Dep::capture).err()
            } else {
                None
            }
        });

        if let Some(cycle) = opt_cycle {
            panic!("{}", Dep::panic_message(&cycle))
        }

        HELD_LOCKS.with(|locks| locks.borrow_mut().push(self.value()));
    }

    pub unsafe fn mark_released(&self) {
        HELD_LOCKS.with(|locks| {
            let mut locks = locks.borrow_mut();

            for (i, &lock) in locks.iter().enumerate().rev() {
                if lock == self.value() {
                    locks.remove(i);
                    return;
                }
            }

            // Drop impls shouldn't panic but if this happens something is seriously broken.
            unreachable!("Tried to drop lock for mutex {:?} but it wasn't held", self)
        });
    }
}

impl Default for MutexId {
    fn default() -> Self {
        Self::new()
    }
}

impl fmt::Debug for MutexId {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(f, "MutexID({:?})", self.0)
    }
}

impl Drop for MutexId {
    fn drop(&mut self) {
        get_dependency_graph().remove_node(self.value());
    }
}

/// `const`-compatible version of [`crate::MutexId`].
///
/// This struct can be used similarly to the normal mutex ID, but to be const-compatible its ID is
/// generated on first use. This allows it to be used as the mutex ID for mutexes with a `const`
/// constructor.
///
/// This type can be largely replaced once std::lazy gets stabilized.
struct LazyMutexId {
    inner: OnceLock<MutexId>,
}

impl LazyMutexId {
    pub const fn new() -> Self {
        Self {
            inner: OnceLock::new(),
        }
    }
}

impl fmt::Debug for LazyMutexId {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(f, "{:?}", self.deref())
    }
}

impl Default for LazyMutexId {
    fn default() -> Self {
        Self::new()
    }
}

impl Deref for LazyMutexId {
    type Target = MutexId;

    fn deref(&self) -> &Self::Target {
        self.inner.get_or_init(MutexId::new)
    }
}

/// Borrowed mutex ID
///
/// This type should be used as part of a mutex guard wrapper. It can be acquired through
/// [`MutexId::get_borrowed`] and will automatically mark the mutex as not borrowed when it is
/// dropped.
///
/// This type intentionally is [`!Send`](std::marker::Send) because the ownership tracking is based
/// on a thread-local stack which doesn't work if a guard gets released in a different thread from
/// where they're acquired.
#[derive(Debug)]
struct BorrowedMutex<'a> {
    /// Reference to the mutex we're borrowing from
    id: &'a MutexId,
    /// This value serves no purpose but to make the type [`!Send`](std::marker::Send)
    _not_send: PhantomData<MutexGuard<'static, ()>>,
}

/// Drop a lock held by the current thread.
///
/// # Panics
///
/// This function panics if the lock did not appear to be handled by this thread. If that happens,
/// that is an indication of a serious design flaw in this library.
impl<'a> Drop for BorrowedMutex<'a> {
    fn drop(&mut self) {
        // Safety: the only way to get a BorrowedMutex is by locking the mutex.
        unsafe { self.id.mark_released() };
    }
}

/// Get a reference to the current dependency graph
fn get_dependency_graph() -> impl DerefMut<Target = DiGraph<usize, Dep>> {
    static DEPENDENCY_GRAPH: OnceLock<Mutex<DiGraph<usize, Dep>>> = OnceLock::new();

    DEPENDENCY_GRAPH
        .get_or_init(Default::default)
        .lock()
        .unwrap_or_else(PoisonError::into_inner)
}

#[cfg(test)]
mod tests {
    use rand::seq::SliceRandom;
    use rand::thread_rng;

    use super::*;

    #[test]
    fn test_next_mutex_id() {
        let initial = MutexId::new();
        let next = MutexId::new();

        // Can't assert N + 1 because multiple threads running tests
        assert!(initial.0 < next.0);
    }

    #[test]
    fn test_lazy_mutex_id() {
        let a = LazyMutexId::new();
        let b = LazyMutexId::new();
        let c = LazyMutexId::new();

        let mut graph = get_dependency_graph();
        assert!(graph.add_edge(a.value(), b.value(), Dep::capture).is_ok());
        assert!(graph.add_edge(b.value(), c.value(), Dep::capture).is_ok());

        // Creating an edge c → a should fail as it introduces a cycle.
        assert!(graph.add_edge(c.value(), a.value(), Dep::capture).is_err());

        // Drop graph handle so we can drop vertices without deadlocking
        drop(graph);

        drop(b);

        // If b's destructor correctly ran correctly we can now add an edge from c to a.
        assert!(get_dependency_graph()
            .add_edge(c.value(), a.value(), Dep::capture)
            .is_ok());
    }

    /// Test creating a cycle, then panicking.
    #[test]
    #[should_panic]
    fn test_mutex_id_conflict() {
        let ids = [MutexId::new(), MutexId::new(), MutexId::new()];

        for i in 0..3 {
            let _first_lock = ids[i].get_borrowed();
            let _second_lock = ids[(i + 1) % 3].get_borrowed();
        }
    }

    /// Fuzz the global dependency graph by fake-acquiring lots of mutexes in a valid order.
    ///
    /// This test generates all possible forward edges in a 100-node graph consisting of natural
    /// numbers, shuffles them, then adds them to the graph. This will always be a valid directed,
    /// acyclic graph because there is a trivial order (the natural numbers) but because the edges
    /// are added in a random order the DiGraph will still occassionally need to reorder nodes.
    #[test]
    fn fuzz_mutex_id() {
        const NUM_NODES: usize = 100;

        let ids: Vec<MutexId> = (0..NUM_NODES).map(|_| Default::default()).collect();

        let mut edges = Vec::with_capacity(NUM_NODES * NUM_NODES);
        for i in 0..NUM_NODES {
            for j in i..NUM_NODES {
                if i != j {
                    edges.push((i, j));
                }
            }
        }

        edges.shuffle(&mut thread_rng());

        for (x, y) in edges {
            // Acquire the mutexes, smallest first to ensure a cycle-free graph
            let _ignored = ids[x].get_borrowed();
            let _ = ids[y].get_borrowed();
        }
    }
}