These Community Benchmarks Will Help You Build A Better Strategy

We seem to be doing a lot of community benchmarking at the moment.

Benchmarking is essentially comparing one community to similar communities or best practices.

When it’s done well, benchmarking shines a bright light on how you’re doing today, where you need to improve, and shows the path for your community strategy. It’s great ammunition too for showing the value of your work to others.

Without benchmarks, you don’t have much visibility about the big picture. It’s like running a race with a blindfold on. You have no idea if you’re running in the right direction let alone beating the competition.

Benchmarking helps establish reasonable goals, identify opportunities, and priorities.

Sadly most organizations don’t have anything close to the data they need to build out a decent community strategy.

 

Three Types of Community Benchmarking

We split benchmarking into three categories. These are:

1) Benchmarking against competitors. This is where we compare a community against an organization’s competitors.

2) Benchmarking against alternatives. This is where we compare the benefit a community offers an audience against anywhere else a member can get that benefit.

3) Benchmarking against ‘best in class’ standards. This is where we begin with the best standards in each category and benchmark a community against them.

For the majority of communities, I’d recommend doing the first two. However, if you’re managing a mature, established, community trying to figure out what to do next, then a ‘best in class’ analysis can be helpful (but we’re not going to cover it here).

 

What Is Competitor Benchmarking?

Competitor benchmarking is where we draw up a list of an organization’s competitors (or community’s competitors) and compare a community against them.

Competitor benchmarking is critical when you want your community to be seen as a competitive advantage. For example, if your community is better than their community, that’s another reason to use your brand vs. theirs.

This is especially important for companies where a supportive community is a critical added value to the product (e.g. a thriving developer community).

In our experience, we’ve also found ‘beating competitors’ to be the single most persuasive argument for attracting more resources for a community. If you don’t have competitor benchmarks, you can’t say where you are today or where you need to go next.

 

Use Competitor Benchmarks To Set Realistic Expectations

We also use competitor benchmarking to set realistic expectations.

For example, if it took a competitor 3 years to get 100k registered members, you’re not likely to match that within 1 year. A more realistic target might be:

• Year 1: Attract 10k registrations.
• Year 2: Attract 30k registrations.
• Year 3: Attract 50k registrations.

There are a lot of variables here (are they of similar size etc…?), but you get the idea. You can learn from how other communities developed and set realistic targets.

 

Competitor benchmarks show where you can improve

Finally, we use competitor benchmarks to identify where a client is and isn’t doing well. You should be able to look at the table and easily see the features where your community is beating the competition vs. where it’s struggling.

This lets you either double down on what makes you stand out or try to compensate for your weaknesses.

 

Criteria For Competitive Benchmarking

You can come up with any criteria you like. But be mindful that benchmarking a community takes a lot of time and it’s not always easy to compare one type of community to another.

We’ve done some deeper investigative work in the past trying to build up a more detailed community picture than is visible on the surface. This includes interviewing members of the community team, snooping on LinkedIn, browsing their job descriptions etc…to build a picture of their budget, team structure, ROI etc….but it’s time-consuming.

In most cases, we want to evaluate a community without needing to have any unique/special access to it. This means we typically look at things we can find in the public sphere.

This typically includes a combination of:

• Date founded. This helps you set realistic targets (find the oldest post or use archive.org).
• Platform(s). This is the community’s technology stack (use builtwith.com or look at the source code).
• Lifecycle stage. Where are they in the community lifecycle?
• Access. Is the community public or private? Do you need special permissions/approval to participate? This has a big impact on the quantity and quality of participation. Try registering and see what happens.
• No. Registered members. Look at listed registered members or member profile URL numbers to find the highest-end number.
• Search. Do they use the platform search tool or a federated/cognitive search tool? Use builtwith.com or look at the source code. Coveo and SearchUnify are common options.
• Level of activity. No. questions per day is a common metric. Either manually count posts for a day or count for a few hours and average over a day (note: some platforms let you use numbers in discussion post URLs so you can work this out).
• Quality of response. Are the responses good and informative or uninformative, unhelpful, responses? Use this benchmarking template.
• Time to first response, response rate, accepted solution rate. If you can’t scrape this data, you can take a random sample of 50 to 100 posts and calculate this for yourself.
• No. groups. Sub-groups are a good indicator of the maturity of a community (but only if they’re active). Manually count these where possible.
• Gamification. Does the community use gamification tools? Are they basic or advanced? I.e. Is the community using the default gamification settings or have they developed advanced, custom, missions/point systems for their audience?
• MVP/Superuser program. How many superusers does the community have? How well documented and detailed is the program?
• CMS. Does the community integrate with a blog/CMS? Is it widely used/kept up to date?
• Knowledge base. Does the community host/use a knowledge base? How widely used it is? Is knowledge kept up to date?
• Social Media. Usually pull metrics Twitter, Facebook, Instagram, and YouTube. This gives you a signal if the community is overperforming or underperforming against other platforms which target similar audiences.
• Events. Does the community host events? How many? What kind etc? How well attended do they seem to be?

This is a typical list, you can often add or remove those which are and aren’t relevant to you. However, it’s a good place to begin.

 

Example Of Competitor Benchmarks

Let’s use a typical example of competitor benchmarks here:

Now you have the data, it’s important to use it the right way.

Be mindful of the following:

1) Every community lives within its own unique set of circumstances. Some advantageous, others disadvantageous. It’s a lot easier to have 500k members if you have 100 million customers. It’s a lot harder if you have 1 million. It’s often interesting to use a ratio of comparing community engagement (and registrations) to social media followings to see if the ratio is similar to others.

2) Every community has different objectives. Salesforce clearly has a strong community effort – but that doesn’t mean every community should be like Salesforce. You might not want to integrate your community deeply with am LMS. The level of activity for example, often depends upon things you can’t control. So use your judgment and knowledge of your strategy.

The purpose of this process is to give you more insights you can use to build a better community strategy – not provide a set of definitive answers.

If you’re Tableau for example, you might wonder why forum activity is low when so many things are so strong. If you’re ServiceNow, you might start thinking about improving the MVP program and facilitating more events in the community.

If you’re Atlassian, you might start looking at what Tableau does in their superuser program and see what you can improve.

When done well, competitive benchmarking gives you a rich dataset you can use to develop a better community strategy

 

Data Scraping – Hard To Do, But Tremendously Insightful

Another thing we do for clients (where possible) is scrape data from communities to compare them by size, speed of response, response rate, and accepted solution rate.

This isn’t always possible (Salesforce communities are notoriously painful in this regard). But when it works, the data (as you can see below) is illuminating. It becomes clear, as you can see below, which are the top performers for their size and which need to be improved:

Once you can see data like this, you can find communities of a similar size, on a similar platform, or in a similar sector, and nudge yourself more towards them.

Looking at the above, you can see some obvious areas of improvement:

• UiPath and Dell: Improve the average time to first response.
• eBay, TomTom, and Square: Improve the accepted solution rate
• AWS & ARM: Improve the response rate etc…

Using your competitive benchmarks, you should be able to set yourself realistic targets and get inspiration for what to do next. Knowing industry averages is really useful here.

 

What Is Alternative Benchmarking?

Alternative benchmarking compares the potential benefits of a community with other places members can go to get those benefits. It’s critical in establishing the unique positioning of your community.

It essentially acknowledges your community isn’t the only game in town.

Sure, people can go to your community to ask questions and get help. But they can also go to Google, browse documentation/knowledge articles, call the customer support line and ask friends. If the goal of your community is to resolve problems, alternative benchmarking recognises that there are plenty of other places people can go to resolve problems.

The same is true with any benefit your community offers your audience. Whether it’s belonging, a chance to explore a topic with like-minded others, or influence – there are plenty of other places members can go to get those things. Why would they visit your community? What is the tremendous unique value your community offers?

For example, if you’re building a community for exports to proactively share advice and expertise (a ‘success’ community), why would your target audience when they can share advice on social media and build their own following? What can your community offer that they don’t?

This is where alternative benchmarking matters.

 

Criteria For Alternatives Benchmarking

There are plenty of ways to benchmark alternatives. We usually begin by identifying the value a community is trying to offer a community. This usually falls into four categories: support, exploration, influence, and belonging.

Which of these four (you can pick more than one) are the primary benefits of the community to an audience?

Next use the table below to go deeper and find out specifically which of these members care more about:

How To Discover What Matters Most

It’s easy to build a list like this above. But you want to know which of these your audience cares about the most.

Surveys tend to be the best way of gathering this data (but if that’s not possible, you can simply ask members). Here’s a client example below:

It’s worth noting that member research isn’t perfect. In my experience reducing effort to get a response is usually what members like the most. But that rarely ranks highly on surveys. Over time you get the hang of these nuances.

As we can see from the above, there are three broad groups of things members want here. Detailed responses, followed by access and speed. Personalisation and responses from experts don’t rank highly.

 

Example Of Competitor Benchmarks

You can find a (slightly) altered client version of these benchmarks below:

Looking at this it becomes clear the community has a problem in the eyes of members. It doesn’t do anything particularly well. However, by seeing how other platforms are doing we can identify two clear opportunities here. These are:

• The community could aim to improve the speed of response. It won’t compete with the WhatsApp group, but WhatsApp is a tiny group. This is probably the biggest opportunity today.

• The community could improve convenience. No other widely used channel does this extremely well. LinkedIn is hardly a threat and WhatsApp is too small to matter. By making the community more convenient (improving the login/registration process) etc…the community becomes the most convenient place to get support.

 

Using Benchmarking Data To Shape Strategy

The real magic however is combining the benchmarking data with the data about what members want to figure out exactly what you should be doing next.

You’re looking to answer four questions here.

1) Where should you double-down on what’s going well? (i.e. where is your community relatively strong vs. the competition and where is the level of audience interest high?)

2) Where should you level up? (i.e. Where is your community relatively weak vs. the competition and where is the level of audience interest high?)

3) What should you stop doing? (i.e. Where is your community relatively strong in areas with a low level of audience interest?)

4) What should you avoid doing? (i.e. Where is your community weak in areas where audience interest is weak?)

This might make more sense in a simple chart shown below:

Let’s use (a different client example) to see how this might look in practice.

Notice now we have data like this we can start to build a clearer sense of tactical priorities.

p.s. It’s also good to have a ‘costly mistakes’ box when a colleague asks you to ‘just create a wiki’.

 

What Should You Work On Next?

Now we have this rich data, we can help clients create data-driven community strategies with short-term action plans.

This might look something like this.

Objective 1: Improve staff accessibility within the community.

• Activity 1: Create a weekly session for PMs and rotating senior executives to take questions from the community.
• Activity 2: Set a target for % of questions touched by staff. Set a target for staff to engage in an increasing number of questions within the community.
• Activity 3: Create a framework and provide training for staff to engage in any question.

Objective 2: Reduce the effort involved to engage with the community

• Activity 4: Pre-popular question text with common questions and show related search results. This helps answers to show up without members having to ask the question.
• Activity 5: Integrate the community with the product. This means enabling questions to appear alongside the product as members go about using it.

Objective 3: Bring speed of response up to the industry average.
(this is usually around 48 to 72 hours).

• Activity 6: Automatically escalate unanswered questions to a community MVP channel via Zapier after 24 hours. This ensures more members will get a response to the question.
• Activity 7: Automatically escalate unanswered questions to product experts after 48 hours.

Objective 4: Bring the accuracy of responses up to industry average.

• Activity 8: Evaluate accuracy and train. Pull a sample of posts provided by community MVPs each month and check for accuracy. Those with the lowest accuracy are invited to an exclusive training session. If they don’t attend, address the issue directly.

 

How Frequently Should You Do Benchmarking?

Once a year feels about the right time frame for me.

We have some clients we do benchmarking for on an annual basis.

The great thing about independent benchmarking is you get to see all the improvements you’ve made in the year beyond those which can only be captured in a graph showing engagement or the number of active members.

If you do your benchmarking this year, I’d set a calendar invitation to do it around the same time next year too.

 

Why It’s Important To Get Your Benchmarking Done

Benchmarking gives you a comprehensive picture of how your community is doing within its environment. It gives you an incredibly rich dataset filled with information you can use to build your strategy and action plans.

We’ve only really scratched the surface in the insights benchmarking can provide you.

Once you’ve got the full dataset, you can do awesome things like:

• Track progress against competitors over several years.
• Run user experience testing against competitor sites and make quick adaptations.
• Set realistic target ranges and achieve them by incorporating the features which have worked elsewhere.
• Spot new competitors and adapt quickly to prevent losing members.
• Invest your time in the areas which have the biggest impact.
• Check and compare ratios of engagement to social media followers to see if your community is performing at the right level.

You can use the framework above and do the benchmarking yourself, or you can get benchmarking done professionally (by firms like FeverBee). But whichever road you go, make sure you get your community benchmarks. You will find them tremendously useful when deciding what you should do next.

Resources

• Get FeverBee To Do Your Benchmarks.
• Builtwith.com (to discover who is using which platform).
• Archive.org (to find out when a community was launched.
• The Community Lifecycle.
• Benchmarking templates.
• Community Positioning.