As a plugin developer myself I read the reviews on the official WordPress plugin directory with great interest. Not only of my own plugins but of all others as well. Doing that I have noticed an interesting pattern in regards to how people rate plugins in the repository.
Simply put: Most people give either 1 or 5 stars. The typical plugin has between 5% – 30% of 1 star reviews and then most of the rest filled up with 5 star reviews. There is never much in the middle except for some amount of 4 star reviews, which I guess are left by people harder to please or those who generally don’t give a “perfect” rating. Just browse through the most popular plugins and you will have a hard time finding one that deviates from this pattern (I did and could not find a single one on the first 4 pages).
As it is now 2 and 3 star ratings could as well not exist. 4 and 5 star ratings could be merged and we get to the boolean operation which the large majority of people use to rate plugins: Does it work for me? Yes or No.
That sounded familiar. And sure enough, when scrolling down enough on a plugins directory page you get to the “Compatibility” box. This underused feature lets a user select whether a plugin works for them or not, which is exactly what most people also use the rating system for!
So my next thought was: Why not combine the two?
And that brings us to…
My Suggestion
Combine the current “Ratings” and “Compatibility” features into one new 2-step rating process.
The two steps would then look like this:
- Step 1: Does the plugin work for you?
The simple Yes / No question that is currently the “Compatibility” feature will be converted into the first part of all plugin reviews. - Step 2: How well does the plugin work for you?
This is the usual 1 to 5 star rating but only if the user said “Yes” in step 1 he can then rate the functionality of the plugin. If he answered “No” he gets taken to the support forum of the plugin with the suggestion to post more details about the issue there.
Instead of by selecting a star rating the plugin review process is now started by the user choosing whether the plugin works for him or not with the respective button.
After clicking “Yes” the user gets taken to a page where he can select his WordPress version. After making his choice the compatibility vote counts immediately. Optionally the user can leave a functionality review on the same page with the usual star rating form and by leaving a comment (just like now).
If the user pressed “No” he also gets taken to a page where he can select his WordPress version and afterwards the compatibility rating counts. Optionally he can start a new thread in the plugin support forum right away by providing more information about the issue he has with the plugin.
Advantages
I think this change would lead to the compatibility rating feature getting used more frequently and at the same time make the star ratings more useful for comparing plugins! To put it into an easy-to-digest list:
- We get more data as to whether a plugin works in the latest WordPress version. The “Compatibility” feature is rarely used but if it becomes the main ingredient of the review process this will certainly change.
- The star rating becomes more meaningful. Since it is now given only by people who the plugin actually works for it will better reflect its features and make it easier to compare to similar plugins. In a diverse ecosystem like WordPress a plugin not working is not necessarily the fault of the plugin itself (but might be caused by hosting, another plugin or other variables) and because of that I think it makes sense to separate “not-working” ratings and ratings of the functionality of a plugin (by people it works for).
- It encourages users to post their problem to the plugin’s support forum instead of just leaving a frustrated 1-star review if a plugin does not work.
- Fairer for plugin developers because if they get many “not working” compatibility reviews they have a chance to remedy that in a future release. As it is now 1-star ratings for a bad release or update (which might have had good intentions) stay forever.
Last but not least I think it simply makes sense! Whether it works is the most integral part of a plugin rating, so I don’t know why we would want or need a separated compatibility feature instead of merging both together.
Disadvantages
I can’t think of anything really. One could argue that there would be less reviews because the process of leaving one becomes more complicated but I would not necessarily agree with that. My suggestion would mean users are asked to fill out the current “Compatibility” form before proceeding to the current “Ratings” form. The difference is only 2 simply mouse clicks.
What do you think?
I would love to hear your opinion on this. Simply post a comment below with your thoughts.
If you like the idea I would also appreciate if you could share it. This blog does not get too many visitors, so your voice is required to get this heard by the right people (whoever that may be) if you want to see it happen.
Permalink
I like it! One other thought that came to mind regarding your observation that votes seem to be polarized at the ends of the spectrum, 1 star and 5 stars. It seems to me like the rating system essentially works as a thumbs up/down voting system, as opposed to a more gradated one. My theory on this is that, when faced with general question, “how good is a plugin?”, most people tend not to break their assessment down any more than just “good” or “not so good.” If we’re looking for a more evenly distributed rating system, then perhaps a more granular approach might work better.
One suggestion might be break the rating system into a few criteria. Keep the 5 stars, but have users rank a few different items. For example:
– It just works
– Documentation
– Performance
– Support (might not always be applicable, and is somewhat obvious from the support tab)
– etc
As far as display goes, you can show:
– The overall rating (the average of all the individual ratings)
– Each individual rating (perhaps via an accordion link to save on space)
Thoughts? Ideas? Suggestions?
Permalink
Hi Mickey and thanks for your comment! I agree with your theory and suppose most people search the plugin directory for something they need to complete a specific task. That would explain the polarized ratings. A plugin that gets the task done deserves 5 stars in the eyes of the user, one that does not 1 star. There is no “it does what I want a little. 3 stars.”
Regarding your suggestion: In general I think that would be great, more detailed data is always better. The problem I see is that plugins are very diverse things nowadays. Many simple ones don’t really need any documentation, performance is something many users (maybe everyone) would have a hard time to judge and put into a star value, etc. Maybe it could be made optional and added to the end of the review process like this:
1. Does the plugin work? (Yes / No)
2. How well does the plugin work? (1 to 5 stars)
3. Additional information survey: How is the documentation? How was you experience with support? etc etc. (optional)
Basically that would make it an extension of my idea above. Each step after the first would be optional, e.g. a user could only say a plugin works for him or he could leave a star rating in addition or do all three.