[FOR ENVATO STAFF] The market needs a rating process improvement NOW!

Dear Envato staff, we work together since 2011, and we have 3 profiles with hundreds of products on the market. If i have to point at the one biggest problem of this market is the rating system.

As matter of fact, we take good care of our product quality and customer support.

But sometimes we get bad ratings.

Every time we go through those ratings they are left from people who:

  • Don’t have a clue what a computer is

  • Don’t read the manual AT ALL

  • Use the ratings instead of using the helpdesk, comments or simply emailing us

Also, the following problems occurs as subsequence:

  1. We can only post 1 reply, and can’t edit that
  2. The people who understand it was their fault, don’t go back changing a bad review because that would proof they are dumb, and they are scared of appearing dumb
  3. We have NO WAY AT ALL to contact the customer, as our data is public while the customer is like a ninja in the shadow, so we can’t even help them fix their problem.

These pictures are from the last of our cases: WordPress theme, 1 star rating, the customer was trying to upload the full zipped product, while the theme of course explains it correctly.
Thank God he sent a screenshot and replied to our rating feedback asking us help

As you can see he is uploading under PLUGINS (not even under themes!!!) the big product zip file

[screenshot removed for privacy]

After explaining the mistake and linking the documentation page (http://www.qantumthemes.com/manuals/onair2/knowledge-base/theme-installation/)

he then replied it was still not working.

The following screenshots are even more spine-chilling: he is trying to upload a Photoshop zip file as plugin and an HTML zip template as plugin

TRYING TO UPLOAD A PHOTOSHOP AS PLUGIN!!
[screenshot removed for privacy]

TRYING TO UPLOAD AN HTML PRODUCT VERSION AS PLUGIN
[screenshot removed for privacy]

We also explained this person that the rating was unjustified because he was not following the right procedure and it was not a product fault, but despite we are still helping him, he didn’t change the rating, because he is afraid to appear incompetent and still tries to blame us.

Now, what I would ask is, why Envato doesn’t adopt the same system as Amazon (which are not the first idiots on the planet) and put a filter on the ratings where a reviewer from Envato takes the rating and analyze them before posting, trying to get in touch with the developers first?

To handle customers like this, not only an author loses a lot of sales and reputation because of bad customers, but also loses a lot of time to still try to make the customer happy, and sometimes as now we are forced to perform the installation to the customer for free to try to make him happy, which means we are again losing time and money.

Why does this ragint system has to be so unfiltered and primitive at cost of authors dealing with incompetent people?

Why not scheduling a deep review of the system as the world already found solutions to avoid these problems?

Do you want a simple solution? here it is:

Bad rating request > alert the author and see if he answers and what.
The author asks to be put in touch with the customer using a, Envato supervised discussion thread > So Envato can see is the freaking customer is uploading his grandma on the wheelchair as zipped theme.
The rating is held on moderation.
CASE A: The Author provides a good service > the rating is not compliant, deleted
CASE B: The Author gives a poor or no service? > The rating is published

This is the Amazon approach and would DRAMATICALLY increase the market efficiency and author/customers satisfaction, ad a very small cost by Envato work time, for s drastic overall market improvement.

You guys already have threads full of people mentioning to leave Envato, and threads about this same subject.

I don’t want to pust a destructive complaint, instead I’m offering a solution for a HUGE problem.

You want more authors as you are also making programs about this? Start by making the marketplace better with those improvements.

Thanks for your time and collaboration.

22 Likes

Also to mention, yes I know that we can contact Envato authors support, but it seems to be kind of a weekly routine now. What I’m mentioning is an overall better Envato supervision and management of those cases, to prevent them at the beginning. This would increase sales, save tons of time, identify bad customers (who don’t have the skills to do what they try to do) and separate those cases from bad products.

It is not about the specific case, is about the fact that similar cases are a very annoying 0.5% of customers that steals 20% of the total working time of the week.

Please consider seriously similar improvements.
Thanks

1 Like

The rating system is also used as a tool for customers to force unjustified refunds. Only if a refund is given will a review be removed. This allows the customer to bully sellers into refunds;

The customer leaves a one star review then requests a refund. You either give the refund to have the review removed or refuse the refund and live with the 1 star review.

On a side note, I would like to see envato follow up purchases with a ‘don’t forget to rate your download’ email, sent a few days later to encourage users to leave reviews.

Thanks

3 Likes

Thanks for your feedback, yes ratings are often used to blackmail authors into providing free code customizations, installations, fixes on third party plugins issues and similar.
The response time of the Envato staff became lately very long (talking about 2 days to answer to authors).
What if we put stars rating to Envato authors support? Just saying…

1 Like

I’m not sure what the benefit would be for the increased time, cost and effort? Everyone’s in the same boat, so if you remove reviews relating to the issues you’ve mentioned, it just means that everyone will have 5 stars rather than 4.5 stars… not sure how that would change anything when it comes to sales. And everyone having 5 stars would look a bit suspicious!

Hello @SpaceStockFootage and thanks for taking time to present your feedback.
I’m glad your presented this point of view because actually this is exactly the opposite of what I was meaning.

The largest rating systems of the world such as TripAdvisor, Booking.com or Amazon just to quote some, all have a review of the rating before publishing.

The review is not meant to prevent bad rating, but to evaluate their content and to prevent honorable business to be harassed by people blackmailing to receive discounts, free stuff or simply unable to properly use a product.

I make you an example translated to TripAdvisor for a WordPress theme review.

Imagine a customer goes to a restaurant, orders a pizza and, isntead of cutting and eating it, tries to stuck the entire pizza in his mouth all at once.
When he fails in his crazy task (which is literally the same thing my customer was doing while uploading a photoshop as plugin) he writes a 1 star review to this restaurant saying the pizza was horrible.

He also puts a picture of what he did, so the pizzeria can see what the problem is, but nobody is there to prevent this kind of people leaving reviews.

This is how the best restaurant in town gets a bad reputation because crazy people tries to eat a full pizza in a bite instead of cutting it (bad product usage) despite the waiter tells them to eat properly.

But, lucily for every tripadvisor user, such crazy things don’t happen, and the same is for amazon and many others.
Why? Because allowing this means simply making the entire market worst.

Now, you probably make amazing space pictures, imagine I take a picture from you, try to upload a 30 Mb picture in my wordpress pretending is a plugin, and write a 1 star review saying yur photo is bad and not working.

And nobody is there filtering this crazy stuff.

Now you have to get in touch with me because you understand i’m making it wrong.

But you cant!!! Because user profile doesn’t provide email of the customer ok?

Then you write to envato asking help, and they reply 3 days later, time in which you lost 20 sales because people were basing their choice to the useless review I left.

It we were on Amazon or Booking, a man in the middle can see the crazyness of the rating and prevent its publication, asking to the customer if he wants to get in touch witht he author.

Well, THIS is the missing step in Envato.
We just get the crazy stuff, unfiltered.

Can you see what I mean?

Then another case is a product is broken, a customer tries to use a product and is broken completely, old and not working, so he writes this in the review. The review like this means the product SHOULD NOT BE ON THE MARKET and should trigger an envato warning. Take the theme / plugin, try to install it and see if is broken for real.

If is broken, the review gets published.

if it works, the customer is not allowed to publish this review and invited to get in touch with the author.

PROBLEM. SOLVED. PERIOD.

4 Likes

I had started a detailed thread a while ago and also made some suggestions.

4 Likes

And please prevent rating from out-of-support-time-customers
And please show the user that they’re out of support time when they come to comment & send support message :frowning:

1 Like

This goes both ways. Staff removes legit negative feedbacks in the same way.

Just an example:

  1. Vendor sells theme claiming that’s RTL
  2. Turns out that the theme is actually not RTL

Vendor admits in the comments section that indeed it’s not RTL but they will upgrade this with the next updates… The “next updates” have not been there for 6 months - For whole this time the Vendor has continued to sell the theme as RTL with many many other issues.

This is actually happening. Seller flags the review and keeps selling.

For the past 6 months the negative reviews appear and then disappear constantly over and over again - The seller just flags them one by one, what he offers is false and he is allowed to sell and flag the legit reviews.

So how is this possible?

What this marketplace needs is a verification system / team not for the sellers and buyers but for the things that are being sold.

1 Like

I know, I probably replied it too. It just didn’t get enough attention and I’m glad you joined the discussion too. Any additional input is welcome, as we are just trying to make the marketplace better for the customers, for authors and for Envato itself.

Kind of “first brick” of a review update, because 1 stars 50% comes from expired supports to blackmail authors and receive help for free.

If you give someone a gun, he will use it, and most of the times not for the good.

Intelligent customers use ratings to thank authors.

Stupid customers use ratings to cover their ignorance.

Bad customers use ratings to blackmail authors and receive free stuff.

2 Likes

“So, you are right, and the other side of your consideration is that negative ratings for undeclared functionalities are simply bs.”

Let me get that straight - You claim that your theme has certain functionalities, I will pay you for those functionalities and then you say that well, the theme actually doesn’t have them - And my feedback would be considered as a BS because you lied about those things on your sales copy?

This is what you just actually said.

1 Like

No, you got it wrong :smiley:
If the theme declares a functionality, it HAS to be in the theme.
If I say the theme is compatible with WPMU, it has to be so, or a bad rating is 100% legitimate.

If instead nowhere is written that the theme does something, for instance the theme is not declared rtl (which is not a theme requirement) and the customer leaves 1 star because the theme is NOT rtl, this is an illegitimate rating.

As it would be like you buy some shoes and say they are s*t because they do not make coffee.

As simple as this.

4 Likes

Yeah was a small misunderstanding there but i agree with everything you said.

1 Like

It seems like you are doing yourself a disservice here. Calling customers stupid, ignorant, and blackmailing certainly won’t be good for business.

This issue is common in every review system I’m aware of including Airbnb, Uber, Restaurants, Hotels, and so on. The focal point as always should be on your top 80%, which is what everyone looks at anyway.

The bottom 20% is not likely affecting sales, brand, or even remotely impacting your business. However, you certainly could be doing all of these things to yourself at this very moment.

[https://gyazo.com/27d728f7b6b6c1cba6cf9da84c06f12a](http://Perfect Example)
This did not affect sales as with most 1 star reviews. And oddly this is precisely what you are attempting to fight and waste time on.

Personally, I was not a fan when Envato introduced off-site support. This opened the door for just as many bad developers as it did bad buyers. I’ve permanently left Envato myself and do not buy here at this point for this exact reason. Sending support emails through a completely separate system only to get answers weeks later asking me the same question I asked to support. And that includes sending screenshots and outlining the issue in depth as a full stack developer/server engineer. I have even taken the time to fix and send an authors code to fix it for everyone. Was it ever fixed, nope. While this varies between each author, bad patterns are forming in many aspects of Envato.

As far as I can tell, Envato does not care. The rating system and off-site support was all done to eliminate them as the issue and offset the workload to authors to handle. The staff they would have to hire to accomplish a 1:1 review system would immediately get rejected. At this stage, they cannot even keep up with reviewing new submissions (from what I’ve heard) and the code quality coming out now vs a few years back is extraordinarily different.

The worst thing for me as a developer is the response from authors saying, “contact support.” when 10, 25, 50+ people have asked the same question in comments. Just answer the question and let users utilize the search built into Envato. You would instantly have a huge drop in support requests over trivial and common issues. Plus it also drops your comment count :wink:

Best case scenario I’d focus on the 80% and move on. Anyone who is serious about buying will review comments “not 1 star ratings” and make a decision based on the last update date and the comments over the last two weeks.

As for improving the system my suggestion was as follows:
Avada: 368,229 sales / 27,766 comments / ratio 13.26 sales per comment
ThemeHatz: 5,209 sales / 372 comments / ratio 14.00 sales per comment
ThemeStarz: 1774 sales / 189 comments / ratio 9.83 sales per comment
ChimpStudio: 4,014 sales / 4,639 comments / ratio 0.86 sales per comment
shared with Envato on Nov 9, 2017, 2:06 PM help@market.envato.com
above numbers are from 2017

The above is my rule of thumb when it comes to spotting duds. Personally I think anyone with a ratio under 1.00 should be banned by Envato or restricted. All they do is
try to sell as many lemons as possible while trying to keep the sheep calm with reassurances that improvements/updates are coming.

ChimpStudios - Current
Home Villas 1,277 sales / 1,239 comments / ratio 1.03
FoodBakery 1,572 sales / 1,644 comments / ratio 0.95
Travel Advisor 179 sales / 113 comments / ratio 1.58
JobCareer 6,133 sales / 5220 comments / ratio 1.17
consistently bad themes, consistently bad support

Avada - Current
517,685 sales / 31,922 comments / 16.21 (+2.95) since 2017
clearly this theme works, clearly they make an effort

ThemeHats - Current
6,288 sales / 425 comments / 14.75 (+0.75) ratio
notice a trend yet?

This has helped me in the past and I suspect it still hold true now without even looking at reviews. Perhaps this could be used as a base to develop something new not entirely based on reviews but rather quality as a whole.

Hello degrama, your deep analysis and suggestions are a real and interesting step for the discussion. I strongly agree with most of your points, and kindly dissent with others.

In the specific, I’d like to consider the average statistical distribution and weight of ratings for the products, in relationship with their impact on the single businesses of the authors.

Explaining better:
Examples like the top selling themes as Avada or X, are a slice quite far from the normal distribution of average authors, as the amunt of sales is not representative of the 80% of the other themes, that have between 500 and 10000 sales.

The very smart insight from your intervention resides, on my behalf, on the % proportion between ratings and sales.

The rason of its relevance is in the amount of normal review compared with reviews of incompetent customers, blaming a product for their failures.

If you are an average seller with 2000 sales, and 100 reviews, a single bad Rating causes a dramatic decrease of the average value.
So, while a more capillar review system of the ratings can be impossible for a few hyper sellers, it results quite a small job if you consider only the reviews of every product and exclude the ones with 100K+ sales.

Also, I’d like to see things from a less abstract perspective, as everybody is human, a product is a product and a company is just a group of people providing the work.

By this point of view, when you look at small developers based on 3-4 people teams, it’s easy to understand how a big effort can easily be harmed by a single bad customer.
Where this logic makes sence, is still a big question mark for me.

My harsh words are aimed in particular to some old “customers” we had who did not even download the product and left bad reviews because they didn’t know how to do the first click.
Customers who aggressibely wrote top offensive words ritch of explicit falsities, like that they contacted support or products didn’t work, and for instance they didn’t even know how to login in WordPress.

back to our company grond compariso, imagine you are a big chef, with your elite restaurant in new york, where you invested all your life.
The restaurant is very new, and you need good reviews.
Then one guy used to eat rats comes there and writes a super insulting reviews because the food didn’t taste like the rats he was eating before.

Then boom, your 20 years study, your billionar investment and your restaurant goes to hell.

Is this fair?

No, so this is why Yelp, tripadvisor and booking, as any other rating system, has a tight regulation about rating contents.

Ar they all idiot companies or are instead protecting the companies that are constituting theyr base of incomes, considering that the users are sometimes unpredictable and often frustrated by a sufficient lack of knowledge?

Let’s be realistic, and consider a wider paradigm of more representative authors, including the impact of unjustified ratings on perfectly working and blazing new products.

Also because, we have to face another reality: you may probably know how hard it is to publish a product, and if a reviewer approves its quality, if we believe that a customer negative rating is an actual proof of poor products, it means the reviewer from envato did a bad job, which is instead the opposite exactly because reviewers are expert, skilled and trained developers, while customers have sometimes a user-oriented point of view.

" the code quality coming out now vs a few years back is extraordinarily different."

Exactly the point in fact, as I found myself a huge coding improvement thanks to updated guidelines and suggestions, and that’s why the possibility to actually have bad products out is so low that it should act as base for a quality standard. This means that bad reviews have a much higher possibility to be just lack of knowledge/reading and effort.

In fact your conclusion is correct:

try to sell as many lemons as possible while trying to keep the sheep calm with reassurances that improvements/updates are coming.

There are quite many similar cases on any market, in or out envato, even on Amazon, and these are the cases when a supervision sould clean up the overall market quality and let the customer understand why items here are better than other market without control

Also, think we are not allowed to quote other products and you should mark the names :smiley:

Thanks again for your kind partecipation, hope other authors may join our conversation.

Wish you a great day!

Stellar reply and well thought out.

Just for the sake of this discussion, lets consider the themes I mentioned at the top have a 14 - 16 ratio of sales / comments.

Traveler with 6,123 sales / 6,460 comments / 0.94 ratio
Before even looking at the comments, I know they are on the bottom end in my experience. One comment immediately stands out, “I have opened a ticket 10 days ago”, “Don’t buy this template it contains lots of bugs”, even after support many users complain it is not working. Notice how the ratio is below 1.00 and notice the vibe in the comments section.

I firmly believe there is a large gap between top / bottom. While everything else falls in-between. Which is perfectly fine provided what is being provided generally works as advertised and authors genuinely make an effort

Feel free to browse the themes section and check for yourself. You will likely find a pretty close connection between sales / comments and ratio. Then checkout the comments for the last few days and gauge the overall vibe.

Obviously this does not apply across the board, I can’t see this really applying to design or work which does not incorporate functionality. But perhaps this can be scaled and leveled off through existing products in each category.

Looking at code/PHP I immediately see one with a ratio of 0.90 that is over a year old. And the initial comments for today include, “it does not work” and “i am having trouble”. The atmosphere/vibe is definitely off.

And the second one I opened has a ratio of 1.62 and the overall atmosphere/vibe is substantially better. You can tell people are using it and it at least works.

Given reviews are still a big part of Envato’s platform and many platforms like Airbnb. I will talk about Airbnb quick as I think the relation is similar. My girlfriend has an MBA and often talks about how the first days/months for anything new is critical. You need 5 star reviews for your first 10 bookings (typically at very discounted rates) to even break into the market. So yes when something new hits the market a) it should be ready b) you will have to take extra care with each new client.

However you quickly reach a point where the focus shifts to focusing on that 80% and trying not to let the bottom 20% impact you personally. Because honestly these lower end reviews say more about the person reviewing then the product itself.

As it stands, almost every product on Envato is rated 4 stars or higher. Even the products with a sales / comment ratio below one have a 4.5 star rating. And I know they are absolutely terrible.

As per the master chef, this is also a topic of interest as I am a foodie myself. It all relates back to my first comment, 80/20 and to focus on the top 80% and ignore the bottom 20% which also applies here. One famous chef recently said if you get caught up in a review by someone who left a bad review or rating because their preferences are for street food then yes you are doomed. No energy should be invested into scenarios like this and these reviews ultimately reflect the person reviewing rather then what or who they are reviewing.

While supervision would help it likely wont happen here. I presented my case to Envato in 2017 and support would not budge at all.

At the end of the day rating are useless here given everything is 4+ stars. It would at least be interesting to see a different metric that at least had some range between lows and high. Which I’m certain could easily be converted into a rating as well.

Browse the site in Themes/WP, code/PHP and let me know! And lastly try not to sweat the trash reviews if they truly do not apply.

For a long time, I written that the rating system is a generator of abuse!
Very often ratings are based on absurdity
For example:
I get two bad ratings (1) for this, that the author of the models I’ve used in my work has removed the photos or moved to another page and the link is not working!
I bought pictures four years ago and I have no control over it. But sometimes bad rating is given by incompetence, envy or does not read the rules of the market!
For example:
“Why is the model not in the download package”, when in the product description says that the package does not contain a model image!
In both cases, Envato hard refused to change ratings, and humiliation remains to me!
Stop this madness!

1 Like

Rembassio, I think your work speaks for itself and the same applies. Focus on the top 80% and keep doing what you do. At the end of the day people can right through these. Someone’s poor review reflect them more than it does you.

Hello and thanks for joining the discussion. You are actually bringing another example of what I mean. Is the equivalend of when, on Amazon, you try to leave a bad rating because the mailman throws a package in the garden, in fact Amazon deletes such ratings as are not about the product nor the seller is responsible.

Exactly as in your case.

If a product was approved from a professional, except if it’s super old, it is ALREADY judged as technically compliant with the requirements, and working, or it was not in the market.

As @degrama says, a 1star tells more about the customer than it tells bout the product, but as you correctly mention, people looks at the rating, and often don’t even read the content of the rating.

PLUS another BIG BIG problem is that ratings affects your overall author score.
On top of this, ratings for deleted items still makes average with your global author rating.