Creating effective community doc events (a case study)
About a month ago, I helped run a highly successful community docs jam for the OSS Salt Project (sponsored by my employer VMware): the Salt User Guide docs jam. This was the second docs jam we ran since our previous one in October and it lasted a full working day. The Salt User Guide docs jam was a gigantic improvement over our previous docs jam. We exceeded all of the metrics we were using to track the success of the event:
- We had great attendance. 35 participants attended the Salt User Guide docs jam and 31 participants attended the keynote address given by Salt founder Tom Hatch.
- Attendees came from diverse geographical and professional backgrounds. Attendees came from Europe, India, and throughout the continental U.S. 3 participants were from the general Salt community and at least 4 of the participants were technical writers, which is often quite rare to see in OSS projects. We had a nearly perfect 50-50 gender split (also sometimes rare for open source). And at least 2 of our participants told us they specifically took work off to come to our community jam to work on our project, which honestly impressed me.
- The event completed the work it set out to accomplish. The Salt User Guide docs jam resulted in 17 merge requests from 8 unique contributors, of which all but one were new. All 17 open issues that were earmarked for work during the jam were completed, meaning the project was completed in 1-2 days.
- We had a fairly high conversion rate. One of our goals was to generate some community interest in joining our documentation working group, which means these participants are interested in making long-term contributions to the documentation and the project overall. At least 3 individuals who participated in the docs jam have attended a docs working group meeting since the docs jam.
- The event generated a lot of positive buzz. The next day at the Salt Open Hour, community members gave nothing but rave reviews for the jam. For example, one of our community members who participated said: “What I thought was especially great was getting one-on-one help setting up an environment. This is the kind of thing people attend courses and pay money for, so it was a huge value to contributors.”
- I received multiple requests to hold a similar event again. For about two weeks after the event, I received at least one inquiry a day asking when we’d hold another docs jam event like this one from people who were not able to make it.
For comparison, the previous docs jam had 24 attendees, 20 of whom came to the keynote (most of whom were employees), and only about 4 people came for the working portion of the docs jam. We had only 1 pull request that resulted from the docs jam and no conversions. So, the Salt User Guide docs jam was a huge improvement over our previous docs jam.
But why was it such a success? What lessons can we learn from this jam that we can apply to future community docs events (both internal-facing and external-facing)? I’ll explore possible answers in this blog entry.
A meaningful project with clear tasks and goals
As I’ve written about before in my reflections on Nadia Eghabl’s book, one of the key motivators for people who become active contributors to an OSS project is the sense that they can make a meaningful contribution. We worked hard from the beginning to make this a key element of our second docs jam.
Our first docs jam wasn’t as successful in terms of participation and productivity. It was our first docs jam so we were really just learning how to run a docs jam in the first place. When we held a retrospective in the first documentation working group meeting immediately after our first docs jam, we talked about how we could improve. Sage (one of our project leaders) rightfully suggested that the next event should be more task-focused and project-oriented.
From that suggestion, we came up with the idea to create a high quality Getting Started guide for new users. I instantly thought of a high-quality instructional guide that had been written in the previous year by Alan Cugler (an employee of one of SaltStack’s partners). Up to that point, this guide was only available to SaltStack customers who enrolled in a paid course about Salt basics. I thought it would be great to open-source this document and make it available for free to the community. We decided that for the community docs jam, we could work on converting the Salt User Guide from a Google Doc into reStructured Text (rST), which is the form of Markdown we use for Salt’s documentation.
As I look back on Sage’s original suggestion, I think it was absolutely the right direction to go. Having a project with a clear goal and tasks was one of the reasons why this docs jam was successful.
High quality community documentation to support the event
After we announced the future doc jam project at SaltConf20, our documentation working group spent the next four months building out the infrastructure for the project. This involved:
- Getting permission from the internal copyright holders and from Alan to open source the document.
- Creating a repository and setting it up with our baseline theming, content, CI/CD pipeline, licensing, and permissions. My coworker Derek (docs tool guru extraordinaire) worked tirelessly to set up the new repository in GitLab and used all of the great knowledge he’d gleaned on past docs projects to deploy the new repository. (Thanks, Derek!)
- Coordinating Salt experts to review Alan’s content to ensure it was accurate.
- Preparing each chapter so that it was publicly accessible and cleaning up any issues in the original files.
- Downloading all the images in the chapters and putting them in a document so that they were available in the repository and could easily be referenced.
- Coordinating logistics with project leaders to get the event officially on the calendar, ensure there was a full-day Zoom room, and ensure there was a keynote speaker.
In the 3-4 weeks leading up to the docs jam, Derek and I began to build out the core supporting documents needed to run the event. We wanted to make sure there was clear guidance at every step along the way that enabled docs jam contributors to help themselves as much as possible:
- The participation guidelines – This document was the keystone document for the project. This document explained the project’s goals, schedule, and participation instructions. We provided this document to potential participants both before and on the day of the event so that people who were interested could learn more about what the project entailed. Even more importantly, building out this document helped me think through all of the event logistics from the participant’s perspective, which got me into the right frame of mind to ensure that we had everything we needed. To reference this archived document, see Documentation Jam Guidelines.
- Detailed list of issues – The part that took me the longest was building out the issues for the jam: one for each chapter that needed to be converted to rST (17 in total). Each issue was written in a way that guided individuals through the process of converting their docs. To view an example of one of these issues, see Ch. 1 Overview – Convert to rST (closed issue).
- A spreadsheet tracking the work to be done – This spreadsheet helped us coordinate who was working on what during the docs jam. It was the source of truth for who was working on what element on the day of the event to ensure we weren’t duplicating each other’s efforts. It was also bolstered by our project board in GitLab that showed the 17 issues and where they were at in the process by their status labels: (unclaimed, in progress, MR submitted, complete). The board no longer exists, so I can’t link to it, but you can still access the archived spreadsheet: Salt User Guide Chapter Spreadsheet.
- The repo README – Derek created the landing page for the repo itself, which described the repository and linked out to other supporting documents. This document is still being used for the project today: Salt User Guide README.
- The contributing guide – Derek mostly authored this guide to help our participants set up their environment. This document is also still being used for the project today: Salt User Guide Contributing.
- An rST guide – Lastly, we created this guide to help participants learn about the conventions we wanted them to use when formatting their rST. See Writing Salt Documentation (rST guide).
- A style guide – I created this guide to answer questions that could come up about how to refer to certain Salt terms and other conventions in case users noticed a style inconsistency in the original document. See Salt Doc Style Guide.
Arguably, these documents were necessary to reduce the coordination costs for the event and ensure that participants could help themselves as much as possible, freeing the event leaders to focus on more high priority requests for help.
Testing, testing, testing
Prior to the event, Derek and I tested our docs jam workflow and process by converting one of the chapters from Google Docs into rST ourselves. Doing this dry run allowed us to work through unseen kinks in the process and ensure they were ironed out either in the technology we were using or in the supporting community docs that we created for the event.
Publicity that aligned incentives
Our publicity campaign was kind of an afterthought in that we only put it together a week before the event, but it actually worked incredibly well.
We created two different types of marketing messages: one aimed at technical writers and one aimed at Salt community members. Both versions gave event details and explained the project. Then each one had a message explaining what that different group would get out of it.
For the community version, I had a section that included:
Why should you consider participating?
–Docs jam contributors will be given a Salt Project store credit to buy some swag!
–If you’re new to Salt, you’ll get some one-on-one help setting up your environment to contribute to the Salt docs and Salt in general. Plus, this new guide is geared for new Salt users and will be a great way to learn more.
–You’ll make a meaningful contribution to the community by expanding access to a high-quality resource that teaches some fundamentals and advanced concepts about Salt. This guide was authored by Alan Cugler (Terminal Labs) and has been vetted by the core Salt team and other Salt experts. It’s also been tested in the field to teach new users about Salt and has received positive reviews. In short, we’re super excited to make this resource available to our community!
For the technical writer version, I mentioned what my connection to the project was and billed it as an opportunity to learn more about open source toolchains. Here’s the full announcement:
I work as a technical writer for an open-source project called Salt, a configuration management and automation tool. Are you new to open source and would like some 1:1 mentorship as you learn how to author in markdown, work in Git and GitLab, and see what it’s like to contribute to OSS? If so, I’d encourage you to come to our community Salt Docs jam on Wednesday, February 10!
The jam will be held from 9a.m. to 3p.m. Mountain Standard Time and will feature a keynote from Tom Hatch (the founder of Salt) at noon. For this community jam, we’re going to work on converting our new Salt User Guide from a Google Doc format into reStructured Text (rSt), a version of Markdown that is ideal for Python-based projects. Feel free to come and participate for an hour or two or stay for the whole day as your schedule allows!
Participants who submit a merge request will be given a store credit to buy some Salt Project swag. We’ll be available to answer questions and mentor you as you set up your environment, learn rST, and submit your first merge request. See our Salt Docs Jam event page and our Salt User Guide Docs Jam guide for the official schedule and more information about participating.
As I mentioned above, we had a really strong showing of technical writers at our event, so this messaging really worked! I posted it on Write the Docs and also in the technical writing subreddit where it generated a ton of buzz and interest.
As my publicity notices indicated, on the day of the actual docs jam, we spent the morning providing dedicated assistance to the participants if they needed help setting up their environment to contribute. We knew this would be the biggest hurdle for most participants, so it was important to spend as much time helping people as needed.
We started by having everyone join our all-day Zoom call. After our initial kickoff where we encouraged everyone to introduce ourselves, we went to different breakout rooms for participants based on which operating systems they were using. In each breakout room, we had a dedicated project leader who helped participants set up their environments. (I helped the Windows folks and Derek helped the Linux folks. We pulled in one of the other project leaders to help Mac folks.) We basically helped them work their way through the Contributing guide and helped them troubleshoot along the way.
This took most of the time in the morning up until the midday keynote, so the afternoon was when people actually started working on their tasks. After that, most people co-worked together on the Zoom call and we fielded questions as they were working on the project and working on their merge requests. Derek and I ended up working 1:1 with a few participants late in the evening or in the days following to get their merge requests to pass.
The day after the retrospective was over, the docs jam leaders met and had a retrospective to discuss what went well and what we’d do differently the next time. Retrospectives are valuable because they help us learn from our successes as well as our failures. As I mentioned earlier, the Salt User Guide docs jam was the direct result of our retrospective on our previous docs jam and it was a gigantic improvement because of that initial meeting.
All in all, the Salt User Guide docs jam was a tremendous success that generated a lot of interest and enthusiasm in the Salt project and in the documentation. To get a little academic for a moment, I think the docs jam was successful because it had each of the four elements that the theorist Yochai Benkler identified as necessary to successfully pull off commons-based peer production:
- Intrinsic motivation – The project created a resource that would make a big difference to the community and to new users who wanted to learn about Salt. The way we marketed the event also highlighted intrinsically reasons why people should consider participating because it emphasized the skills and professional development that participants would gain by participating.
- Modularity – The project was broken up into clear subcomponents that could easily fit back together: everyone could work individually to convert one Google Docs chapter into rST and then bring it back into the larger project: the Salt User Guide.
- Granularity – After setting up their environments, most contributors could complete a chapter within an hour. They could complete the tasks without too much pre-existing knowledge.
- Low coordination costs – We worked really hard to reduce coordination costs and provide users with lots of support so that they could succeed on their own. This ensured that the docs jam leaders could focus on the most important requests during the day.
The dirty secret is that in the amount of time we spent preparing for the documentation jam, we could have probably just converted all the rST files ourselves. But we chose to invest our time building this infrastructure as an investment in our larger community. Events like these ensure that we are breathing new life into our community by engaging new community members in our project. And it’s giving back to the community in terms of mentorship and resources. And that’s the beauty of participating in open source projects!