This year I had the opportunity to present a talk at RailsConf 2021!
The conference was 100% remote, so I recorded everything in advance, and then had a live Q&A session over discord. As of May 2021, the conference-branded version of the talk is now available on YouTube:
Click here to download my slides (with code errors corrected):
At 20:00 I demonstrate the use of
lock!on two account records. A couple of conference-goers rightly pointed out that such a strategy could be subject to a deadlock, a topic that I cut for time. In practice, deadlocks are one of many low-probability reasons a method might fail, so thankfully the resiliency strategies I cover should help mitigate them.
I'd also add that it's not actually the explicit use of
lock!so much as the
transactionthat is the source of the deadlock. Most SQL databases will implicitly lock rows on
DELETE, and by eagerly locking both records up-front, we may actually be reducing the overall chance of a deadlock occurring.
At 24:30, the slide should read
At 27:39, the slide should read
At 35:01 I state that most job backends guarantee "at-least-once execution," but on second thought, I'm not actually sure this is true. My understanding is that Redis-backed queues tend to use a delete-on-pickup strategy, which means that if the server process crashes or is halted, these jobs may never have a chance to complete.
The pro version of
sidekiqclaims to optionally support at-least-once durability, by relying on Redis' atomic
RPOPLPUSHto move jobs to an in-progress queue, deleting them only after they complete. (Similar to the way that most DB-backed queues work by default.)
Of course, if you can't guarantee message delivery at the time of enqueue, then you effectively don't have at-least-once execution guarantees, regardless of your queue backend.
At 36:57, the upper right
savemethod should read something like
if !order.in_progress? ...instead of
if !order.completed? ...because the subsequent update sets
in_progress: true, and we must check that state in order to avoid race conditions. (I had been planning on adding a segment covering state machines, but it got cut for time. If you're interested in how I track state across parts of persistence operations, check out the steady_state gem!)