38 - Journal Club / RFD / Oxide

Viewing public RFDs.Learn more

published<br>RFD 38<br>RFD 38 Journal Club

This RFD can be accessed by the following groups:[public]

State<br>published

RFD<br>38

Updated

Origins<br>Journal clubs<br>trace their origins to the 19th century, when they were used by physicians<br>to keep current on medical literature. Since then, they have played a<br>pivotal role for physicians in particular, and for the biosciences more<br>generally. The literature on journal clubs themselves cites three essential<br>purposes, concisely summarized by Mark Linzer in a<br>survey of journal club history[linzer]:

It is remarkable to note that all three of the historical goals of medical<br>journal clubs (to 'keep up' with the literature, to impact on clinical<br>practice and to teach critical reading skills) were mentioned in our survey<br>with almost equal frequencies.

Goals<br>For Oxide, a journal club serves similar purposes to those distilled by<br>Linzer: to keep us abreast of the other work in our domain, to keep our own<br>reading skills sharp (and to model them for more junior engineers), and<br>(especially) to broaden and inform our own engineering thinking, as<br>described in [rfd5].

Experience<br>The literature on journal clubs (and in particular Deendadaylan<br>et al.'s survey)<br>cites some common problems with journal<br>clubs: keeping discussion germane, keeping attendance up, and making sure<br>that everyone has read the papers being discussed. While these problems<br>seem to transcend medicine, the specific solutions used over the years in<br>the health sciences may have less relevance for us in software engineering.<br>To that end, it’s worth discussing previous experiences in software<br>engineering in particular.

Experience at Joyent<br>In experimenting<br>with a journal club at Joyent, we had a modicum of success, but it<br>ultimately fizzled: after having had monthly cadence for most of a year,<br>Joyent’s journal club slowly became more and more irregular until disappearing<br>entirely.

There were a couple of clear challenges that mirror those seen elsewhere:

Lack of discussion leadership. We tried to rotate leadership of a<br>discussion, but when "looking for volunteers", it seemed to always fall to<br>the same names. This problem was exacerbated by the fact that discussion<br>leadership had become burdensome due to in part to lack of preparation of<br>participants (described below).

Fixed monthly cadence. The monthly cadence was seemingly never the right<br>interval: at times it was too infrequent (in particular, with the<br>benefit of hindsight, a more timely journal club at some key junctures might<br>have resulted in more informed development), and at times too frequent. The<br>lesson is that the cadence should be — as much as possible — dynamic:<br>there are times that a team will want to be reading and discussing the<br>literature very intensely, and times that they will naturally be focused on<br>other things.

Lack of preparation. Regrettably, we found lack of preparation to become<br>rampant: participants would arrive at journal club having not read the paper<br>in advance. This wasn’t misbehavior per se in that the meeting was<br>deliberately not exclusive to those having read the paper (and participants<br>would be upfront about having not read the paper but wishing to benefit from<br>the discussion), but it would invariably result in the leader of the<br>discussion summarizing the work — which itself would take much of the<br>alloted time (and effectively punished those who had read the paper). This<br>reduced journal club to being more of a lecture than a conversation, which<br>in turn made leading a discussion overly burdensome.

Experience at Delphix<br>One additional lesson from Delphix was that keeping the burden low for the<br>prospective coordinators is essential. We drifted into paralysis in a surprising<br>way. We had a few organizers in a row who were extremely diligent in their<br>preparation. They presented excellent slides that thoroughly and helpfully<br>explained the content of the papers discussed. Each successive presentation was<br>of higher and higher quality. As a result…​

the space for discussion decreased

the burden on participants to have read the paper decreased

and the expectations for what it took to coordinate became insurmountable and<br>no one was willing to coordinate.

It may feel like it runs counter to our value of rigor, but having the<br>coordinator not be more prepared for the discussion than anyone else is<br>important to encourage all forms of participation.

Experience at Transposit<br>The additional lesson from Transposit was to cultivate a culture of curiosity<br>even in the face of an urgent need to ship. We only made space for papers that<br>were highly related to areas of the product; the result was that often<br>interest was too narrow and often the discussion happened too late to impact the<br>product.

Experience with Papers We Love<br>Several of us have been involved with (that is, attended and/or presented at)<br>Papers We Love. This is a kind of internet<br>journal club, and has resulted in many fruitful...