MDN Product Talk: The Case for Experiments

This is the third post in a series that is leading up to a roadmap of sorts — a set of experiments I will recommend to help MDN both deepen and widen its service to web developers.

The first post in the series introduced MDN as a product ecosystem. The second post in the series explained the strategic context for MDN’s product work. In this post I will explain how an experimental approach can help us going forward, and I’ll talk about the two very different kinds of experiments we’re undertaking in 2015.

In the context of product development, “experiments” is shorthand for market-validated learning — a product discovery and optimization activity advocated by practitioners of lean product development and many others. Product development teams run experiments by testing hypotheses against reality. The primary benefit of this methodology is eliminating wasted effort by building things that people demonstrably want.

Now my method, though hard to practise, is easy to explain; and it is this. I propose to establish progressive stages of certainty.

-Francis Bacon (introducing the scientific method in Novum Organum)

Product experiments all…

  • have a hypothesis
  • test the hypothesis by exposing it to reality (in other words, introducing it to the market)
  • deliver some insight to drive further product development

Here is a concrete example of how MDN will benefit from an experimental approach. We have an outstanding bug, “Kuma: Optional IRC Widget“, opened 3 years ago and discussed at great length on numerous occasions. This bug, like so many enhancement requests, is really a hypothesis in disguise: It asserts that MDN would attract and retain more contributors if they had an easier way to discuss contribution in realtime with other contributors.

That hypothesis is untested. We don’t have an easy way to discuss contribution in realtime now. In order to test the bug’s hypothesis we propose to integrate a 3rd-party IRC widget into a specific subset of MDN pages and measure the result. We will undoubtedly learn something from this experiment: We will learn something about the specific solution or something about the problem itself, or both.

Understanding the actual problem to be solved (and for who) is a critical element of product experimentation. In this case, we do not assert that anonymous MDN visitors need a realtime chat feature, and we do not assert that MDN contributors specifically want to use IRC. We assert that contributors need a way to discuss and ask questions about contribution, and by giving them such a facility we’ll increase the quality of contribution. Providing this facility via IRC widget is an implementation detail.

This experiment is an example of optimization. We already know that contribution is a critical factor in the quality of documentation in the documentation wiki. This is because we already understand the business model and key metrics of the documentation wiki. The MDN documentation wiki is a very successful product, and our focus going forward should be on improving and optimizing it. We can do that with experiments like the one above.

In order to optimize anything, though, we need better measurements than we have now. Here’s an illustration of the key components of MDN’s documentation wiki:

metrics_status Visitors come to the wiki from search, by way of events, or through links in online social activity. If they create an account they become users and we notify them that they can contribute. If they follow up on that notification they become returners. If they contribute they become contributors. If they stop visiting they become disengaged users. Users can request content (in Bugzilla and elsewhere). Users can share content (manually).

All the red and orange shapes in the picture above represent things we’re measuring imperfectly or not at all. So we track the number of visitors and the number of users, but we don’t measure the rate by which visitors become users (or any other conversion rate). We measure the rates of content production and content consumption, but we don’t measure the helpfulness of content. And so forth.

If we wanted to add a feature to the wiki that might impact one of these numbers, how would we measure the before and after states? We couldn’t. If we wanted to choose between features that might affect these numbers, how would we decide which metric needed the most attention? We couldn’t. So in 2015 we must prioritize building enough measurements into the MDN platform that we can see what needs optimization and which optimizations make a difference. In particular, considering the size of content’s role in our ecosystem, we must prioritize features that help us better understand the impact of our content.

Once we have proper measurements, we have a huge backlog of optimization opportunities to consider for the documentation wiki. Experiments will help us prioritize them and implement them.

As we do so, we are also simultaneously engaged in a completely different kind of experimentation. Steve Blank describes the difference in his recent post, “Fear of Failure and Lack of Speed In a Large Corporation”. To paraphrase him: A successful product organization that has already found market fit (i.e. MDN’s documentation wiki) properly seeks to maximize the value of its existing fit — it optimizes. But a fledgling product organization has no idea what the fit is, and so properly seeks to discover it.

This second kind of experimentation is not for optimization, it is for discovery. MDN’s documentation wiki clearly solves a problem and there is clearly value in solving that problem, but MDN’s audience has plenty of problems to solve (and more on the way), and new audiences similar to MDN’s current audience have similar problems to solve, too. We can see far enough ahead to advance some broad hypotheses about solving these problems, and we now need to learn how accurate those hypotheses are.

Here is an illustration of the different kinds of product experiments we’re running in the context of the overall MDN ecosystem:

optimization_vs_learningThe left bubble represents the existing documentation wiki: It’s about content and contribution; we have a lot of great optimizations to build there. The right bubble represents our new product/market areas: We’re exploring new products for web developers (so far, in services) and we’re serving a new audience of web learners (so far, by adding some new areas to our existing product).

The right bubble is far less knowable than the left. We need to conduct experiments as quickly as possible to learn whether any particular service or teaching material resonates with its audience. Our experiments with new products and audiences will be more wide-ranging than our experiments to improve the wiki; they will also be measured in smaller numbers. These new initiatives have the possibility to grow into products as successful as the documentation wiki, but our focus in 2015 is to validate that these experiments can solve real problems for any part of MDN’s audience.

As Marty Cagan says, “Good [product] teams are skilled in the many techniques to rapidly try out product ideas to determine which ones are truly worth building.  Bad teams hold meetings to generate prioritized roadmaps.” On MDN we have an incredible opportunity to develop our product team by taking a more experimental approach to our work. Developing our product team will improve the quality of our products and help us serve more web developers better.

In an upcoming post I will talk about how our 2015 focus areas will help us meet the future. And of course I will talk about specific experiments soon, too.

MDN Product Talk: Business Context

This post is intended to frame the business context that MDN operates in. This is not a product strategy. It is the second in a series of posts begun here.

One of the questions I asked in my introductory post was, “What does solving problems for MDN’s audience accomplish for Mozilla?”. In other words, what opportunities make this investment worthwhile?

It’s not always comfortable to ask such questions, but they’re important. We operate within constraints; everything we do has an opportunity cost. Why MDN? I’m glad to say it is easy to answer.

MDN is a public service sponsored by Mozilla, a mission-driven nonprofit. Mozilla does not measure success by revenue, we measure success by our global impact. Products at Mozilla are expected to deliver impact on Mozilla’s mission and support at least one of Mozilla’s main product lines.

Mission Impact

Mozilla’s mission is to build the internet as a global public resource open and accessible to all. Web developers are critical to the mission. They choose the platforms and technologies they use to build products:

  • Deploying to the web is a developer choice.
  • Using open web standards is a developer choice.
  • Building content that works across a multitude of devices is a developer choice.
  • Making content accessible to a broad, global audience is a developer choice.

The success of Mozilla’s mission depends on these developer choices. Developers make choices pragmatically — they choose the possible. MDN makes building open and accessible on the web more possible. MDN empowers all developers to make choices aligned with Mozilla’s mission and thereby create the web the world needs.

How do we measure this impact? Look to future posts for more discussion of this.

Top-line Organizational & Product Support

MDN supports Mozilla’s top-line goal to create meaningful relationships with Mozilla product users worldwide. MDN’s current audience contributes approximately 1-2% toward Mozilla’s overall relationship goals; this number is naturally limited by the size of MDN’s market.

MDN is naturally aligned with Mozilla Learning. Mozilla’s learning products build contributors to the web. MDN empowers expert and professional web makers and is uniquely positioned to address those who aspire to be expert or professional web developers. MDN’s alignment with Mozilla’s learning initiatives is obvious, but only recently intentional; in 2015 we will experiment to learn more about how to directly support Mozilla Learning.

Market

In 2015 MDN will focus on three segments — two of them new.

MDN historically has served web developers who want to build standards-based web sites. After a decade of providing high-quality information about web technologies on its documentation wiki, MDN is considered an authority among these developers. (For instance: Instructors at several “web developer bootcamps” in my town tell me they actively guide students toward MDN, away from MDN’s competitors, because they only trust MDN.) MDN serves more of these developers every month and now serves 25-50% of the global market. These developers are MDN’s core users and the choices they make directly impact Mozilla’s mission.

MDN’s authority, expertise and capabilities position it uniquely to serve web learners who aspire to be web developers. The job market for developers is strong and growing. The number of jobs in both web development and software development is expected to grow by 20% or more in the U.S. by 2020. Global growth is expected to follow a similar trend. MDN can match these new web developers with product features that will help them become web developers building standards-based web sites.

MDN’s authority, scale and unique content position it to serve professional web developers who want to build standards-based web sites professionally. Recent surveys indicate that more than half of MDN’s visitors meet that description. A large number of these professionals say they are motivated to build web products that embody Mozilla values like accessibility, compatibility, and security, but are under-served by tools to make doing so feasible. MDN can build new products and features to fill the gap, increasing these professionals’ ability to build standards-based web sites on deadline.

2015 Goals

Considering those opportunities, in 2015 the MDN team has committed to…

  1. “Relationships”: Support Mozilla’s overall goals by continuing the (double digit per year) organic growth of our original segment, web developers who want to build standards-based web sites.
  2. “Teaching”: Support Mozilla’s learning initiatives by exploring and implementing new capabilities that will serve a new market and segment, web learners who aspire to be web developers.
  3. “Services”: Add a new product, “Services” to serve a new segment, professional web developers who want to build standards-based web sites professionally.

KPIs for these live on the Mozilla wiki with other information about the MDN team’s overall vision, mission and objectives.

Summary

Can you put a TL;DR at the bottom of a post? Let’s find out. After this post and yesterday’s, we can now say…

MDN is a product ecosystem that serves an audience whose choices are critical to the success of Mozilla’s mission. MDN serves them at a scale unique within Mozilla and in the market. In 2015 MDN will continue serving web developers who want to build standards-based web sites while building prototypes and experiments to solve problems for web learners and professional web developers.

Next: Products! Stay tuned…

MDN Product Talk: Introduction

In coming days I will post a series of blog posts about MDN, the product I am product manager for. I will talk about MDN’s business case, product strategy, and a series of experiments we can run on MDN in 2015 (and beyond) to help it continue to serve its audience and the web.

Many people familiar with MDN will consider some of the following posts obvious; to them I say, “stay with me.” Not all of this will be obvious to everyone. Some may be novel to everyone. Any of it may need clarification or improvement, which I will learn from gentle comments.

mdn_logo-wordmark-full_colorAs a new member of the MDN team, I taxed my colleagues in pursuit of answers to such questions as…

  • What is MDN?
  • Whom is MDN for?
  • What are their problems?
  • How does MDN solve those problems?
  • What does solving those problems accomplish for Mozilla?

I posed such questions to almost everyone I could corner, from members of Mozilla’s steering committee to random web developers I met at kids’ birthday parties. I interrogated historic mailing list threads. I scanned through hundreds of enhancement requests in our bug backlog. I read books about product management. I doodled architectural diagrams and flowcharts. And finally I think I came to an understanding. I hope sharing it is helpful.

So what is MDN? Is it a documentation site? A developer program? A wiki? A network? A suite of products? A single product? A brand? A railway station? It depends on whom you ask and when. Few of the people I cornered could say exactly what MDN is or ought to be. Most had an idea of what MDN ought to accomplish, but their ideas varied wildly.

One thing is clear: MDN is successfully serving a large portion of its potential market and has been doing so in its present form for nearly 10 years. Which is great. But… what is MDN?

For most of MDN’s primary audience — web developers — MDN is a documentation wiki (i.e. a user-generated content site or UGC) serving trustworthy information about open web technologies (primarily HTML, JavaScript and CSS). And of course the audience is correct.

But MDN is also a brand that resonates with its audience and fans. And MDN is also a community of contributors who care about MDN. And MDN is also a platform where all these things mingle and mix.

All told, MDN includes (at least) 6 product offerings in operation or development and (at least) 8 significant enhancement projects underway, as well as numerous activities and projects that sustain the documentation wiki. The interplay between these activities is complex.

OK, great; but what is MDN?!? Considering its scope — the variety of its critical stakeholders, the complexity of its value chain, the number of supporting activities underway — I think MDN can only be considered a product ecosystem.

MDN_product_ecosystem

  • MDN is a strong brand supported a number of products and activities.
  • Chief among these is a platform, the documentation wiki, which is also called MDN.
  • Within the platform content is huge: MDN’s primary audience visits solely for the content, because the content is valuable. The MDN brand’s authority and the MDN platform’s scale both depend on the MDN content’s quality.
  • As an open-source web application serving user-generated content about open standards, contribution overlaps almost every aspect MDN — especially content creation. The MDN content’s quality depends in large part on contribution.
  • Various marketing efforts support MDN, strengthening its brand, attracting new visitors, and activating contributors. MDN marketing efforts among developers also support Mozilla’s brand.
  • In response to a clear market need, the MDN team is experimenting with some developer-facing services that will be partly supported by the content and platform.

That’s a lot of moving pieces — and we haven’t even begun talking about whom MDN serves and what problems MDN solves for them! Look for answers to those questions and much more in coming posts.

As we go I’ll use the above diagram to help contextualize things. More to come!

:wq

Report: Web Compatibility Summit

Last week I spoke at Mozilla’s Compatibility Summit. I spoke about compatibility data. Here are my slides.

Compatibility data…

  • is information curated by web developers about the cross-browser compatibility of the web’s features.
  • documents browser implementation of standards, browser anomalies (both bugs and features), and workarounds for common cross-browser bugs. It theoretically covers hundreds of browser/version combinations, but naturally tends to focus on browsers/versions with the most use.
  • is partly structured (numbers and booleans can answer the question, “does Internet Explorer 10 support input type=email?”) and partly unstructured (numbers and booleans cannot say, “Safari Mobile for iOS applies a default style of opacity: 0.4 to disabled textual <input> elements.”).

Compatibility data changes all the time. As a colleague pointed out last week, “Every six weeks we have all new browsers” — each with an imperfect implementation of web standards, each introducing new incompatibilities into the web.

Web developers are often the first to discover and document cross-browser incompatibilities since their work product is immediately impacted. And they do a pretty good job of sharing compatibility data. But as I said in my talk: Compatibility data is an oral tradition. Web developers gather at the town pump and share secrets for making boxes with borders float in Browser X. Web developers sit at the feet of venerated elders and hear how to make cross-browser compatible CSS transitions. We post our discoveries and solutions in blogs; in answers on StackOverflow; in GitHub repositories. We find answers on old blogs; in countless abandoned PHP forums; on the third page of search results.

There are a small number of truly canonical sources for compatibility data. We surveyed MDN visitors in January and learned that they refer to two sources far more than any others: MDN and caniuse.com. MDN has more comprehensive information — more detailed, for more browser versions, accompanied by encyclopedic reference materials. caniuse has a much better interface and integrates with browser market share data. Both have good communities of contributors. Together, they represent the canon of compatibility data.

Respondents to our recent survey said they use the two sites differently: They use caniuse for planning new sites and debugging existing issues, and use MDN for exploring new features and when answering questions that come up when writing code.

On MDN, we’re working on a new database of compatibility data with a read/write API. This project solves some maintenance issues for us, and promises to create more opportunities for web developers to build automation around web compatibility data (for an example of this, see doiuse.com, which scans your CSS for features covered by caniuse and returns a browser coverage report).

Of course the information in MDN and caniuse is only one tool for improving web compatibility, which is why the different perspectives at the Compat Summit were so valuable.

concentric_compatibility

If we think of web compatibility as a set of concentric circles, MDN and caniuse (and the entire, sprawling web compatibility data corpus) occupy a middle ring. In the illustration above, the rings get more diffuse as they get larger, representing the increasing challenge of finding and solving incompatibility as it moves from vendor to web developer to site.

  • By the time most developers encounter cross-browser compatibility issues, those issues have been deployed in browser versions. So browser vendors have a lot of responsibility to make web standards possible; to deploy as few standards-breaking features and bugs as possible. Jacob Rossi from Microsoft invited Compat Summit participants to collaborate on a framework that would allow browser vendors to innovate and push the web forward without creating durable incompatibility issues in deployed websites.
  • When incompatibilities land in browser releases, web developers find them, blog about them, and build their websites around them. At the Compat Summit, Alex McPherson from Quickleft presented his clever work quantifying some of these issues, and I invited all present to start treating compatibility data like an important public resource (as described above).
  • Once cross-browser incompatibilities are discussed in blog posts and deployed on the web, the only way to address incompatibilities is to politely ask web developers to fix them. Mike Taylor and Colleen Williams talked about Mozilla’s and Microsoft’s developer outreach activities — efforts like Webcompat.com, “bug reporting for the internet”.

At the end of the Compat Summit, Mike Taylor asked the participants whether we should have another. My answer takes the form of two questions:

  1. Should someone work to keep the importance of cross-browser compatibility visible among browser vendors and web developers?
  2. If we do not, who will?

I think the answer is clear.

Special thanks to Jérémie Patonnier for helping me get up to speed on web compatibility data.

Learning Experiments on MDN

At the bottom of this post, I ask: Who should I connect with to learn more about the state of the art in professional web developer education? Read on to see why, and answer if you can.

Mark Surman, Executive Director of the Mozilla Foundation, wrote on his blog recently about the Foundation’s ambitious plans to make Mozilla “a global classroom and lab for the citizens of the web”. He briefly mentioned MDN, the product I manage:

Given what we’re already doing, being bold doesn’t need to involve huge new investments. In fact, it can start simply with being more clear and assertive the work we already do. Not just with Webmaker, Hive and Maker Party, but also with user education in Firefox, Mozilla Developer Network, ReMo program and our research and fellowships programs. What I’m talking about starts with making these things stronger — and then telling a clear story to ourselves and the world about how they add up to a coherent whole. That’s what I want us to start doing in 2015.

He’s right: Mozilla has been teaching the web for years through numerous channels. Along with our explicit efforts, Mozilla’s policy, community development, content, and product features all help people learn how the web works and how to use it effectively. We can confidently state our intent to do more of that.

MDN has always – explicitly – been a learning resource. When the MDN wiki launched in 2005 it was billed as “a Mozilla project dedicated to providing documentation, education, and community for developers of all types.” Now MDN helps millions of developers every month to understand the core technologies used for building web sites.

Mozilla’s other efforts in education to date have focused on building local networks, teaching web literacy and creating enthusiasm for web technology among young people and non-developers. MDN complements this work because it provides similar (but distinct) opportunities to an audience of intermediate and advanced web developers. A significant majority of MDN visitors – 70% in recent surveys – are professionals in the field.

Along with the Foundation group, MDN will spend much of 2015 making our educational role explicit. For example, last week the MDN content team quietly launched a new learning area on MDN. This learning content represents a large and ongoing effort by many volunteers and staff contributors to create new pathways into the advanced web topics MDN  covers. Like most wiki content, it is a work in progress; in 2015 the learning content will make great leaps forward. (Also like most wiki content, you can contribute your expertise to make it better.)

We recognize that learners learn in very different ways. Some easily absorb the technical documentation MDN has always provided; others need a different format, or different materials, to help them grasp these complicated subjects. But what additional formats and materials can MDN contribute to these learning experiences? Like the  learning area, this is new territory for MDN. So in 2015 we’ll undertake a variety of experiments to help understand how MDN can best help people grow their web development skills and become professionals and adepts at making the web the world needs.

Here are some naive examples to show what I mean by “experiments”:

  • Videos are a common medium among sites teaching things. Are they effective? Would MDN’s audience appreciate them? Should we experiment with integrating or linking to more video content on MDN?
  • There are lots of online schools teaching web development. Are any of them doing an incredible job of it? Should we talk to that school about collaborating to give MDN users access to its programs?
  • MDN can facilitate local learning groups. Would that be valuable? How could we do that? How would we know it was working?
  • etc.

I will solicit and explore such experiments here, on this blog, and will recommend the most compelling opportunities for more work in 2015. I encourage anyone with feedback or questions to discuss in comments, reach out to me directly, or respond in kind on their own blog (and let me know about it).

Here is my first question for all the learners and teachers out there: Who should I connect with to learn more about the state of the art in professional web developer education? Twitter handles, blog URLs, and introductions welcome.

Vouched Improvements on Mozillians.org

Back in October I wrote a few blog posts describing a significant problem with the way we admit new members into the Mozillians.org community platform. Yesterday the Mozillians.org team fixed it!

Before yesterday, everyone with a “vouched” account in Mozillians.org was empowered to vouch others. But we never explained what it meant to vouch someone: What it implied, what it granted. As a result, the standard for being vouched was arbitrary, the social significance of being vouched was diluted, and the privileges granted to vouched users were distributed more widely than they ought to be.

Yesterday the Mozillians.org development team released a major refactor of the vouching system. For the first time we have a shared definition and understanding of vouching: A vouch signals participation and contribution in Mozilla’s community, and grants access to content and systems not available to the general public.

The new vouch system includes features that…

  • ask a “voucher” to explain to the community why they are vouching someone
  • grant the “vouching” privilege only to people who have themselves been vouched multiple times
  • remove “legacy vouches” from accounts that were vouched before we agreed what vouching meant and whose current contributor status can’t be easily verified

It is much clearer now who can access non-public information using Mozillians.org (people who have been vouched because they participate and contribute to Mozilla) and how that list of people can grow (through individual judgments by people who have themselves been vouched numerous times).

When we know the composition of a network and understand how it will grow, we can make better decisions about sharing things with the network. We can confidently choose to share some things because we understand whom we’re sharing with. And we can reasonably choose to withhold some things for the very same reason. Understanding a network simultaneously encourages more sharing and reduces inadvertent disclosure.

Thanks to the Mozillians.org development team for making these excellent improvements!

Site launch: Affiliates 2.0!

Cross-posted from http://blog.mozilla.org/webdev/2014/05/21/site-launch-affiliates-2-0/.

We just launched a redesigned website for Firefox Affiliates — check it out!

New Affiliates web site!

The Affiliates website is a portal for almost 100,000 Mozilla supporters. Affiliates believe in Mozilla’s cause and want to show their support by placing an affiliate link on their website. Here’s mine:

If you use Adblock Plus you probably see nothing here. And that’s a shame.

The original Affiliates website launched in 2011. Its branding recalls an earlier era in Mozilla style. I will miss those fuzzy monsters, but (for now at least) I know I can still find them on Mozilla.org’s 404 page. And seriously, after three years, the site needed more than a brand refresh. As part of this launch we…

Image from the original front page.
The Affiliates 1.0 front page included fuzzy monsters.
  • Upgraded to the latest version of Playdoh
  • Removed lots of old code and subsystems no longer in use
  • Reconsidered and redesigned the entire user experience
  • Updated the pages to a beautiful take on Mozilla’s Sandstone theme
  • Added an affiliate leaderboard
  • Added user profiles and a personalized dashboard
  • Added space for localized news and announcements
  • Much more!

If you’re already a Firefox Affiliate, we* hope you enjoy these improvements. And if you’re not, why not sign up now? Being an affiliate is an easy way to support the efforts of a non-profit that strives every day to build the web the world needs.

We’d like to hear from you. Please file a bug if you find one or reach out in the #affiliates channel on irc.mozilla.org to say “hi”. And keep your eye out for more new features — there’s another batch of Affiliates Awesome just around the corner!

* The Affiliates 2.0 team included dozens of people spanning the globe — too many to list here. This site redesign was a collective effort of Mozilla’s amazing affiliates, volunteers, localizers, designers, copywriters, engineers, quality assurers, contractors,  and many others. Thank you all.

A new Mozillians.org signup process

I previously wrote about some issues with the Mozillians.org authorization system. In a nutshell: The language we use to talk about signing up for (“vouching”) and accessing the platform through the API (“corporate vs. consumer”) doesn’t match our present and future needs, and also doesn’t match some of our recent practices. This disconnect has a variety of problematic ramifications. We want to fix it.

The resolution for these dual concerns is related, but we don’t need to resolve them simultaneously. Below, I describe a proposal for resolving concerns related to signing up for the platform. Many of these ideas came from williamr, jmenon, davidwboswell, giorgos, and others across Mozilla.

The problem we need to resolve is ambiguity about what it means to have an account in Mozillians.org. And, with a nod to Mozilla’s ambitious plans for community building, we need to resolve this problem in a way that makes it easy for the right people to join Mozillians.org.

Goal: Eliminate confusion about what having an account in Mozillians.org means. Make it easy for the right people to join Mozillians.org.

Those two requirements are quite complex. Opinions differ about what it means to be a Mozillian, and about how that relates to the systems that a Mozillian might join. We reached out to the community in a variety of forums to help us understand the scope of this conversation (here and here, for example). After discussing for several months, we believe the community has provided sufficient guidance for us to write code. Specifically, the community said (to use Mitchell Baker’s words),

‘[We need] to be inclusive, and provide welcome, encouragement and legitimacy to people across a range of different levels of engagement. At the same
time, we want a way to identify the set of people who are actively committed and engaged in a community of shared effort. A single “yes/no” decision — yes, you’re a Mozillian, or “no, you’re not” can’t capture all of this well.’

It’s clear that being a Mozillian is a journey. In order to support this journey, we want people who are just starting out to have access to a basic set of resources within Mozilla’s contributor ecosystem.

Mozillians.org is in a position to provide this basic level of service; for example, having an account in Mozillians.org allows Mozillians to connect with one another, lets them access certain protected resources, and makes possible a unified view of an individual’s contributions across myriad contribution pathways. Therefore, we want to build a Mozillians.org signup process that allows people to join Mozillians.org as soon as possible — at the beginning of their journey.

We want to build a Mozillians.org signup process that allows people to join Mozillians.org as soon as possible — at the beginning of their journey.

However, not just anyone can join. Mozillians do share certain characteristics and we want our signup process to capture these. The community has converged around a small set of criteria that distinguish a Mozillian. According to that consensus, in order to be a Mozillian, someone should…

  1. Be active, now or historically
  2. Self-identify with Mozilla’s mission
  3. Engage with other Mozillians in the community

We can capture all of these criteria in a signup process. In other words, we can build a harmonic coherence, building the community’s definition of Mozillian right into Mozillians.org.

cohere
Obligatory doge graphic

We’ll do this by replacing the current signup process, which depends on a vouching step, with a signup process driven entirely by invitations. Here are the specific features we propose to implement in the first iteration of this work:

  • In order to join Mozillians.org, a person must have an invitation.
  • Invites can be sent by people who already have accounts in Mozillians.org.
  • Inviting will be rate-limited (e.g. every Mozillian gets 5 invites per month).
  • The signup process will be clearly explained to logged-out users anyplace it makes sense to do so. For example, “Think you have what it takes to be a Mozillian? Great! Just find a Mozillian to invite you.”
  • The invitation process will include specific instructions including the criteria for inviting someone. For example, “Mozillians are active. Please invite people who are actively contributing.
  • All language about vouching will be replaced with language about inviting, site-wide.
  • Upon accepting an invitation, a user will be presented with the Mozilla Manifesto. Clicking “I Support Mozilla’s Mission” lets them continue to the profile creation screen.
  • The profile creation/editing screen will include links to specific information about how particular information added to Mozillians.org profiles is shared (e.g. “Information in your profile will be viewable by other members of Mozillians.org”).
  • Profiles will include an “Invited by” link to the person who invited them (people who were vouched in the prior system will be “invited by” their voucher).
  • A formal announcement of this change will be made in advance, giving all users 30 days to change or remove their accounts if they wish to.

Together, these features address the overarching goal of this effort because they clarify the makeup of our community. The new interactions and language clearly explain that Mozillians.org relies on the good judgment of each community member, and can grow organically based on the actions of any existing member.

These features also address the criteria that define a Mozillian:

  1. They specifically ask existing members to consider contribution activity when inviting new members.
  2. They require each new Mozillian to read and support the Mozilla Manifesto, self-identifying with the mission.
  3. They ensure that new members of Mozillians.org are already engaged with someone in the community.

In the future, we can imagine extending this functionality in a few ways:

  • We can allow certain API accounts to create invitations on behalf of a specific pathway. For example, when someone submits their first PR to a Mozilla repository in Github, they might get an automatic invitation to join Mozillians.org.
  • We can prompt existing Mozillians to read/agree to the Manifesto on a regular basis — perhaps annually?

We are eager to hear feedback about this proposal because we intend to start coding a new signup process in early 2014. Please reach out in the developer’s mailing list or in the comments.

A Weird Ritual

I stumbled across the short form creative brief by way of a tweet from @jmspool and I had to share it. It reminds me of something we built at dojo4 when I was CEO there, but it includes a Very Important Addition.

Technology and design work is expensive, far too expensive to do without some anticipated return (unless you work in a sector that allows you to light bricks of technology dollars afire on a hot summer day “just for the ambiance”).

When I was at dojo4, upon signing a contract with a customer, I tried to capture a couple sentences about their anticipated return on a 3″ by 5″ notecard. I pinned the notecard to a corkboard where everyone could see it (and I mean everyone: our employees, our guests, our other customers). We called the notecard the Project Compass.

The project compass was intended to be a guide, an arbiter in times of uncertainty. Whenever a question arose about project scope or direction, we could look at the project compass for clarity. A typical project compass might be “Redesign the website and add a feature allowing customers to create and manage their own profiles.”

Astute readers may notice the above compass doesn’t describe a return on investment at all. That’s quite common, unfortunately. It is HARD to achieve clarity about a project’s anticipated return. Often, a project’s sponsor has already done some initial analysis and design and is giving implementation experts the output of the initial analysis (“Redesign the website”) instead of the input (“Increase signups and improve retention”). Sometimes the sponsor can’t articulate what they hope to achieve. Just as often the implementation team can’t hear it.

At dojo4 we assumed imperfection in our project compass. Every project required a compass to move forward, but we agreed (and said out loud) that the compass might change. We even asked @anthonydimitre to draw us a classy graphic explaining exactly how this process would work:

dojo4's Project Compass, circa 2011
dojo4’s Project Compass, circa 2011

We usually dove into implementation as soon as a compass was written and pinned to the corkboard. For the minimum-viable-product startups dojo4 worked with, “implementation” was practically synonymous with “changing the compass”. But “implementation” also always meant “charging hard toward maximal features in minimal time”.

The little black “No” in the middle of the illustration above is what we envisioned happening if the project was discovered to be out of alignment with the compass. We’d look at stories and check them against the compass and change one or the other as necessary. But delivering code at breakneck pace to customers with rapidly changing goals was totally orthogonal to thoughtfully reviewing and making adjustments to paper-based project artifacts. So, our project compasses often went stale.

This is common in every kind of project everywhere. Sometimes it is a problem; sometimes not. When a project sponsor and all the project’s implementers have fantastic rapport and constant engagement, they can happily forget whatever they wrote in the brief three months ago. They’re grooving. But sometimes, the project sponsor and the implementers will carry divergent ideas of the project’s purpose all the way to launch day. I know at least a few people who’ve changed careers after pulling a week of all-nighters to deliver something that nobody wants.

That was precisely what the project compass was designed to help us avoid. But a corkboard full of stale project compasses didn’t help anything. Which is why the short form creative brief caught my eye. The document itself resembles the project compass — more verbose, still quite brief. But unlike the compass, the short form creative brief is imbued with longevity through a “weird ritual at the start of every meeting”:

One of the team members, always a different person, would read the exact same document out loud, word for word. The document, about three–quarters of a printed page, contained a tiny creative brief about the design they were working on. Reading it out loud was how they started every design meeting, whether it was a brainstorming meeting or a design review….[then] the project’s leader would turn to the group and ask the same question, “Everyone agree that this is what we’re working on today?”

Many times this exercise has no obvious impact: Everyone simply nods and the meeting moves forward. But occasionally, someone asks for clarification. They ask because they’re new to the project; or they ask because they’ve been assigned a task that doesn’t seem aligned; or they ask because they sponsored the project and no longer agree with something in the brief. When someone asks, the group discusses and updates the brief as needed.

Ritual is the perfect word for this exercise because the magic only happens if you do it religiously. You read the brief every time. You read the brief even when it feels silly to read the brief. Even — no, especially — when the meeting is about something urgent or tense. Because reading the brief puts the project’s critical facts right where they belong: At the forefront of everyone’s mind, in consensus terms freshly aligned with the effort actually underway, for the entire duration of the project.

I suspect the shape of the brief (or compass) is not nearly as important as its frequent review. Of course, it should contain enough information to explain why project participants keep meeting and working together, instead of playing pinball or hoarding shoes or visiting every county in Texas. That could be one terse sentence. The important thing is that the brief continues to explain where the group is headed, even if the group changes direction.

At Mozilla we use etherpad for planning meetings and taking notes during meetings. I have begun adding a “theme” to the top of etherpad agendas as a gentle way to remind people of the big reason we’re having yet another weekly meeting. For example, on Mozillians.org right now, the big reason we’re having yet another weekly meeting is to discuss the Curated Groups feature that we’ve been working on all quarter.

After reading up on the short form creative brief, I think I may take a moment at every meeting to speak our current theme out loud, too. Does everyone agree that this is our focus right now? Are there any questions about what it means?

Proposal: The Future of Contributor Tools

I work as a web product engineer for various web products at Mozilla, including several platforms designed to serve Mozilla’s contributor community. I team up with product stakeholders to design products and I work with implementation experts to build them. I’m responsible for achieving vision.

This role gives me a good perspective on what we say we want to do and how we do it. And I see an opportunity for us to increase the scope and ambition of both in our contributor tools efforts.

In September, I collaborated with David Boswell and Chris More on a proposal explaining the opportunity. Since then, inspired by great conversations at the 2013 Mozilla Summit, Mozillians have discussed and converged on a few foundational definitions that will inform our subsequent conversations about contributors. Simultaneously, the proposal (which has been in Google Documents until now) has slowly made its way around the organization by way of document invites.

Since there seems to be some interest in it, and since invite-only isn’t really the Mozilla way, I’m sharing the most recent version of the proposal here.

Please note: This is a proposal, it’s not a plan. If you support it, your advocacy will help! If not, your challenging questions will help, too!

Contributor Tools Program

Definition and Scope

The Contributor Tools program builds and maintains systems that connect contributors to contribution opportunities and helps keep them actively involved. These systems serve the entire project and are not specific to a given functional area or product. This includes tools that:

  • Offer people both online and offline ways to get involved with the project (such as a call-to-action on http://www.mozilla.org or an organization-wide events system)
  • Match new contributors with tasks and teams appropriate to their skills and interests (such as a task board or a tool to help contributors find bugs)
  • Provide visibility into the activities contributors are involved in (such as a contribution logger or a dashboard)
  • Recognize people for their contribution to the project (such as a badging system or a profile management system)

This does not include tools that:

  • Serve the needs of just one functional area (such as a localization workflow tool)
  • Serve as a platform for interacting with an audience other than new and active contributors (such as a product site, a blog aggregator or an affiliate marketing program)
  • Would be required by Mozilla even in the absence of a vibrant contributor community (such as a bug tracker or a customer support website)

Problems Addressed

Connecting with and relating to contributors is an effort that enjoys broad support across Mozilla. But the tools we currently depend on to connect with and relate to Mozilla’s contributors are fragmented. As a result:

  • We do not undertake tools efforts according to a unified strategy
  • We do not measure the value or impact of these efforts in any comprehensive way
  • We do not take advantage of the efficiencies or provide the cohesive experiences that shared technologies, data, processes and resources could deliver

The Contributor Tools program will address these problems by unifying various tools efforts in a single program, creating a new structure for strategy, measurement and performance.

Stakeholders

The stakeholders for these tools include the staff and volunteers actively involved with building communities around their projects and the contributors who devote effort to furthering Mozilla’s mission. These stakeholders are represented by the Systems and Data Working Group of the Community Builders team.

The Systems and Data Working Group will meet regularly to establish decision-making structures for the program, set requirements for each system, create criteria of success for the program and advocate for and secure resources for those initiatives. In 2013 the group identified the systems required to complete Mozilla’s contributor tools suite.

Implementation

The Contributor Tools program should optimally have enough resources to maintain existing systems (providing security and stability fixes but few enhancements) while undertaking one major new effort at a time (such as building a new platform or redesigning an existing platform). This would require a small team of implementation staff plus the option to bring on contractors for burst efforts.

The Web Productions team has successfully built a program similar to the Contributor Tools program to deliver continuous service and improvement on http://www.mozilla.org. That program is accountable to a group of stakeholders from around the organization who are represented by a product owner. The product owner works closely with a program manager who helps guide the efforts of a cross-functional implementation team including some dedicated staff, some shared staff, some contractors and some contributors.

Implementing a similar approach for the Contributor Tools program would mean identifying individuals to occupy all of those roles. The exact number of individuals would depend on the scope of the program’s accountability. Our initial analysis suggests that existing staff and contractors might be able to provide most or all of the resources initially needed by this program.

Leverage

This program will also be able to gain additional leverage in two ways:

  1. Current community tools efforts are fragmented with teams building siloed functionality that could be extended to support the entire project (e.g. an events manager in reps.mozilla.org, contribution dashboards in support.mozilla.org). Numerous staff throughout the organization contribute, or are ready to contribute, to these fragmented efforts. Guiding their efforts with a unified strategy will allow for smarter allocation of existing resources.
  2. These tools will also serve as featured web development projects for new contributors. The program’s charter could include a mandate to maximize contributor involvement in system development.

Next Steps

The most minimal implementation of this program would require some realignment of product and engineering teams to cover the broader scope contemplated here:

  • Product teams would drive products forward based on the priorities of the Systems and Data Working Group of the Community Building team.
  • Implementation teams would shift between projects as necessary to achieve the program’s vision.

This realignment would enable the program to undertake Contributor Tools efforts according to the priorities established by a broad coalition of stakeholders.

A more ambitious program implementation might also include:

  • Adding a full-time Community Builder to the program team to maximize contributor involvement in product implementation.
  • Enlisting engineers, designers and other implementation experts from elsewhere within the organization who are already working on this effort.

These additions would enable the program to develop new features more rapidly and/or support more systems. Furthermore, the full-time Community Builder’s role could be charged with establishing a model for other web projects to use, thereby increasing project capacity across the entire organization.