State of the Union: ThemeForest review queues.

Below is the first update on this topic. Further updates can be found in the links below:

Hey guys, Matt here.

Apologies are due here. There’s been a great deal of pain and frustration as a result of the current waiting times. We’ve decided that Jarel’s time is best spent on working to fix the issue. After having a number of discussions with him, I felt I could share with folks the state of the union.

Firstly, some context is required about the system that works to review our items. When we talk about a review queue, it’s natural that those words conjure up images of something a bit like this:

However, in reality the system is a great deal larger, with a lot more moving pieces:

Each month, our review team inspects over 500,000 unique items, not including updates. Also, the review queue is far from a straight line ending in a reviewer. There is an absolutely huge, and ever evolving level of technical infrastructure (engineering work) that sends the items to different places at different times and supports our reviewers to ensure that their time is spent as efficiently as possible.

So how did we get to where we are now?

As many of you are aware, there are three different outcomes from the review queue.

  1. Hard Rejection
  2. Soft Rejection
  3. Acceptance

Where this began was within an in-depth discussion of the differences between a Hard Rejection and a Soft Rejection. Frankly, our team felt we were Hard Rejecting too many items, and so we had an internal workshop to clarify the differences between a soft and hard rejections. Here’s where we landed:

For an Item that has been hard rejected to be accepted, the item would essentially have to be changed so majorly that it would in essence be a completely different item altogether. Therefore, when we review, we wanted to be asking ourselves some important questions: Does this item have potential? Could it get over the line? How can we help get it there?

The way we wanted to move was clear. We wanted more soft rejections and less hard rejections. The problem however is that soft rejections require far higher levels of reviewer attention. Additionally themes are significantly more complicated now than they were a few years ago, resulting in greater length of review time. Soft rejections require us to know the item in higher level of detail, and at their best, deliver key insights to an author on how they might improve their work. We don’t always get it 100% correct, but when we do pull it off, the result is a big win-win for the author and for Envato. Change was required.

Changes to big systems like ours take time and work. Recruiting new reviewers is just one part of that puzzle. The bigger challenge is actually in altering the infrastructure that supports our reviewers, one which was built to support the old way of working. We held off on the recruitment element because we chose to work on our infrastructure first, rather than induct new reviewers to two systems. However, with the attention of Envato’s engineers pulled in many different directions, the work was put in a queue of it’s own. This leading us to where we are now.

Let me be clear, we are at fault. Balls were dropped in our transition from one world to another. We don’t write this post with a view towards asking forgiveness as we don’t believe the frustrations of our community are to be brushed aside lightly. We write this instead because i’ve been in enough queues myself where I’ve sighed in frustration, clenched my fist and just wished I knew what on earth was going on. Well, the above is what’s going on, and the below is what we’re going to do about it.

As Collis has already stated, we’re in the process of recruiting more reviewers. More essentially though, the whistle has been blown and we’ve now got a crack team of engineers working on improving the all important infrastructure that support them. This will take a little bit of time, but we’ve got a roadmap for what needs to be done (we’ll be sharing pieces of this soon) and the wheels are now moving in the right direction. We’re confident that soon, we will have made headway in bringing the queues down to levels that are sustainable and manageable for everyone.

20 Likes

Thank you for updates.

@matthewcoxy, well that was a long overdue update and thanks for that!

It’s highly important that authors are kept in the loop of what’s going on so explaining a bit of the process and implications within Envato helps put some worries to rest. Looking forward, unfortunately we still only know that you guys are working on it and we have to be patient, so there’s no change here. And the frustration will keep on building as there are no clear timelines for this issue to be resolved, nor what being “resolved” means. What should we expect, and when? 7-days tops queues for WP themes? When, by the end of June? July?

Would be great if you guys communicate more about TRUE objectives, something palpable. Trust me, everyone here is still affectionate toward Envato and in most cases, the frustration is sort of like feeling cheated by the person you’re in a relationship with. What we need is real communication, 100% void of truisms and political-like speeches.

Yes, we know we need to be patient. Yes, we know we should expect changes. Yes, we know it can take time. But what no one really knows is where exactly are we in “the loop”. Stages, expected finish line.

When you run a presentation by your CEO and board, you never go and tell them: we’re working on it, be patient. You’d get fired in a second. They expect targets, timelines and results. Now I’m not suggesting the community is or should be your board, but they are the ones that enable your business and play a major role in its income and progress. Communication without stereotypes, but with real facts is just respectful and common sense.

That being said, good luck and godspeed with the process and keep us in the loop!

Alex

12 Likes

Thank you for the update, but this does not help, no deadlines, no compensations, so excuse me but this post did absolutely nothing. All you sad yea sorry we messed up but you will still have to wait and suffer. I really do not know if you realise what all of this means? Certain author rely on this to survive!

Yet again another meaningless post. As I sad do not charge you provision until you fix it so you feel the suffering also!

14 Likes

Although I appreciate the update no one has addressed why there is no submission requirements?

The current submission guidelines bare little resemblance to the soft rejections. Without clear up to date guidelines we are shooting in the dark, wasting our own development time and making the review process longer.

7 Likes

Thank you for the update. Even though we have a WP Theme in the review queue, we prefer waiting and getting some guidelines to improve our theme (in case of soft-rejection), rather than a painful hard-rejection. :slight_smile:

2 Likes

Wow, chill man.

As it was already mentioned, it’s work in progress.

I don’t get it, if you’re suffering this badly, and it’s affecting your survival, just go somewhere else or start your own shop, it’s that simple.

Leaving the anger behind, think about how lucky we are that Envato exists and it’s still a bootstrapped company. You should be more grateful.

3 Likes

^ I hope Envato can see now that this was a poor choice. We needed more reviewers years ago - as you were told by the community of authors many times over and over. Ineffective transition of reviewers to a new system would be the least of your worry now.

Thanks for the update though.

2 Likes

So in the short term nothing will improve.
The wait for a review will continue to increase.

Can you please tell how many months will take to rebuild your entire review infrastructure? And, how much time it will take for the review process to become normal?

1 Like

Hope to see the updates soon…

So from my understanding, the reason the review times got in the state they are is because the minimum requirements ( hard vs soft ) were lowered.

And because of that those items that would previously get a hard reject now get FREE design/functionality advise from a reviewer on what to do to make it better which takes up a lot of time.

Here’s my thoughts.

Requirements should be adjusted periodically to keep the amount of new releases at a healthy level at which a newly released item has enough time to be seen by potential customers.

So if there aren’t enough items fitting the requirements, the requirements go down. If there are more than enough items the requirements go up.

And the current state here is that there’s a lot of items being released, the requirements should go up.

So I don’t understand why the requirements adjustment went in the opposite direction. Increase in amount of releases is not what’s needed here.

Also, a reviewer shouldn’t spend his valuable time thinking about how an item that has potential can be improved to make it more than just potential, that’s the author’s job. If it only has potential then hard reject and say it needs to be greatly improved before resubmitting ( as it used to be ).

Reviewer’s job is to review, not to actively participate in the item’s creation process and hold the author’s hand guiding him to item release.

So, instead of big system changes and bigger review teams that will “take a little bit of time”, it can be fixed right now by simply adjusting the requirements and not making the reviewers do what the authors should be doing.

The current road being taken is a long long one, it will cost Envato more, it will cause sales per item to decrease ( due to more items being released ), lower the overall quality of items…

But just take a few steps back and take the other road, it will not cost Envato more, it will cause more sales per item, it will increase overall quality of items…

Anyway, that’s just my thoughts, I’d love to hear what other authors think.

6 Likes

If this is the case then it is wrong aproach. I already said that HTML and PSD should be design base for CMS as it was before, that means if HTML/PSD is approved then WP is approved also without exceptions. Why would they lose time to check WP’s design? Also authors would be sure to rely on already approved PSD before they make WP version.

Just thought about something and throwing an idea. The absolute first review & impression is checking the design first, then it’s code quality.

I’m thinking of a system like:

  • Hard reject - it’s a no!
  • Design - Soft reject (lack of features, design needs improvements etc.)
  • Design Approved
  • Code quality - Soft reject (actual code review, bad practices etc.)
  • Coding quality Approved and item approved for sale.

If we think about it, it’s easier to find design reviewers AND/OR code reviewers, but harder for reviewers knowing both. Maybe the process would still be a bit slower, but soft rejects would still get feedback (there were always complaints about this) and the queue wouldn’t be flooded.

3 Likes

Thank you for update @matthewcoxy this is the level of comunication I like!

I would love to be part of Envato review system.

again no clear time-frame, would be better if you people can provide a time-frame when this issue will be resolve rather then explaining us how envato infrastructure works.

1 Like

Thanks for the update. But we need deadlines

2 Likes

App store review times decreased from average of 8.8 days to 20 hours in just few months since Phil Schiller took over - http://www.bloomberg.com/news/articles/2016-05-12/apple-shortens-app-review-times-in-push-to-boost-service-sales. Aim is to boost revenue from services and developer loyalty. Some new apps are being approved in just few hours time and developers feel this is awesome since it reduces development cycle and plan their work better.

The job of a reviewer at Apple is probably a little different the one at Envato with a few more quality checks, lot more rejections here but still great to hear some good news.

Probably a good idea to plan on developing an app in between theme soft rejections :slight_smile:

16 Likes

Excellent link, @LiveMesh!

Month ago I have uploaded for review a brand new education theme – Smarty School – and I still have not received your feedback. I contacted twice the Envato support and they said the review now can take up to 2 months. Before it was no longer than 2 weeks – that was ok.

I think this is absolutely unfair to authors, who build their business around Themeforest & other Envato marketplaces. For us it’s vital important to get the fastest response on the item reviews. We currently have 10 people in team, working on WordPress themes, updating, supporting, etc.

We don’t create multi-purpose WordPress themes – we only create micro-niche themes and we really enjoy creating micro-niche themes. The only bad side of micro-niche is the life time of the theme. If we don’t publish new theme, we reduce in sales. If we earn less, we can’t pay our team, can’t pay office rent on time, which is not good.

What I would like to suggest – it’s to provide privileges to Elite Authors in item review process and reduce the review time up to 1 week. I think the biggest amount of revenue generate only Elite & Power Elite Authors on Themeforest. I think it’s the same as in App Store, where only 5% of authors generate 90% of sales.

New authors on Themeforest should go through the normal process of review, because new authors don’t know all the requirements on code quality, design, etc. When we started on Themeforest in 2011 we also didn’t know much about the quality of themes, quality of design for Themeforest, many other small things.

If Envato will not solve the issue with the item review timing, it will get the following issues:

  • Authors will upload very raw themes just to get soft-rejection. Because authors know that they have 4-8 weeks more to complete the theme and after soft-rejection, immediately upload the update.
  • For Reviewers – it will mean more work & time to provide the feedback on raw themes. So the review timing will grow up like it does now.
  • Elite & Power Elite Authors may look for other marketplaces to stabilize their business and income flow. They may co-operate or go their way to sell the WordPress themes directly like Themezilla (Orman Clark) does.
  • Item support will become worse. Most new authors need time to build the right item support system – customer will be unhappy with Themeforest, and will probably lean to other marketplaces looking for better quality and response in support.
  • Quality of new themes may become amateurish. Most of new themes created by new authors are not compatible with popular and necessary WordPress plugins like Slider Revolution, Gravity Forms, WPML. Even Visual Composer modules are not styled and designed the right way – the authors style only that modules which they use in demo.

I might be wrong, but this is what I think. The only solution I see here is to work closely with Elite Authors and Power Elite Authors – providing more attention and dedication.

I hope Envato or Themeforest management will read my note about the current situation with Item Review process.

Thanks for reading,

Igor Ligay
StylemixThemes

8 Likes