Measuring dev rel programs far beyond marketing activities

Ana Jimenez Santamaria of Bitergia shares metrics and KPIs for dev rel programs that focus on developer engagement, developer acquisition, and developer satisfaction.

Video

Summary:

• Ways to measure your developer programs in terms of acquisition, engagement and satisfaction, but far away from marketing.

  • More focused on a community perspective.

• Dev rel is about building relationships.

  • Department you came from but always have in mind the real mission and why you were hired as a dev rel and not as a marketing specialist.

  • ‘Cause dev rel is about building relationships, and it’s about community.

• It’s really really nice that you track things related to marketing but don’t only do that.

• Be near your developers and go to the same places with them.

  • Try to get the whole picture.

• Understanding metrics take a lot of time.

• Needs to get the domain knowledge

• DevRels need to interpret numbers to get insight for your community.

Scribbles:

Dev roles are really diverse.

• A dev rel can be hired by a soft foundation.

• They can be hired by a company.

• They can be hired because they want to manage open source projects.

• Or just proprietary software.

• Makes it really, really difficult to bring up a standard way of metrics.

• Dev rel roles can report to many different departments and those departments have different ways, and different KPIs to measure success.

Dev rel roles are connected

• They’re hired by a foundation.

• They're hired for a company.

• They report to marketing

• They report to engineering.

• Mission of a dev rel is always to build relationships with developers.

• Developers can be seen in many different profiles or personas.

• At the end, everything is related to community issues and ways to measure community.

Measuring acquisition

• We can measure the developer growth by software product/project.

• Real examples from the CHAOSS community.

• Try to analyze the developer retention rate or the developer bounce rate.

• Example - Software GrimoireLab, Open source

  • Activity -> community -> Attraction and retention.

  • Measure Git, ’cause we are fond of our community, it’s really engaged, it does a lot of Git code, for instance, and that’s a lot of Git commits.

  • You can see the attracted developers and the last committed developers over a period of time.

Measuring engagement

• Take a look at the data sources about all the different challenges the developer community faces.

• All the different data sources are aggregated, all the different channels where our community is.

• We can define if my community is a contributors community, is a user community, is a maintainer community, or any other type of community you would like to profile.

• Depends on a lot of domain knowledge.

• Define a personas pattern.

Developer engagement by software product or project

• See the most active community and the most engaged community project among the other ones.

• Identify which projects had at some point some activity and then were left like the one on the prospect.

• What happened then?

• What did we do wrong?

• Or what can we do better in order to improve that engagement with the project itself?

Measuring developer satisfaction

• Uber OPSO use case

• OPSO - open source program office.

  • Where open source and all the related activities related with open source are centralized and growing inside a company.

• Uber wanted to know how efficient they were in answering, enclosing, and merging pull requests.

• They wanted to measure GitHub in this case.

• They wanted to know how fast they were when measuring Uber developers versus non-Uber developers.

• Turns out that they tend to merge non-Uber developers took them more time when, rather than when, they wanted to merit a Uber developer pull request.