State of the Union: ThemeForest review queues.

Has to be more than 3 on Wordpress surely

So, what do you expect? :smiley:

And now, the most important question. When Envato hires enough reviewers? Is it too difficult to hire some reviewers? Or whether we should actually be less productive? :slight_smile:

It should be easy. I’ll give you a happiness formula:

More products = more reviewers. More reviewers = fast review times. Fast review times = more happy authors. More happy authors = everyone is happy. … Almost :smiley:

The truth is probably “uncomfortable” and “against the forum rules”. :slight_smile:

1 Like

June is about to end. Clock is ticking! Official words?

TL;DR:

  • The review queues continue to be the focus of a great deal of work:
  • We have made inroads into improving the capacity of our team and infrastructure to be able to deal with higher volumes of items
  • The rate at which new items are being submitted also continues to rise meaning that wait times for new submissions continues to increase.
  • We have a roadmap of several initiatives that we detail below to bring the queues down further. Specifics relating to these initiatives and challenges can be found below.

Hi everyone,

We’re now approaching the end of June, so we wanted to jump into this thread and give you all an update as to where we’re at, some of the changes we’ve made, the issues we’re facing and the roadmap moving forward.

Firstly, let’s talk about the current state of our queues. For the purposes of giving you context around what we’re doing, we wanted to paint a clearer picture of how the queue works:

Item updates, resubmissions and submissions from Power Elite Authors are moved to the front of the queue. I’ve listed the reasons why we assign high priority to these types of submissions below.

Updates: This comes from a need to maintain the quality of current items on the market. Updates address a range of important things, from ensuring items are compatible with Wordpress Updates, to fixing a bug, or remedying a security vulnerability. We prioritise these because our first responsibility needs to be ensuring that the items that are on our shelves are safely maintained for our authors and our customers.

Resubmissions: If we’ve reviewed and soft rejected an item with feedback, then we need to be able to get to that item again as quickly as possible. Note: We recently increased the number of soft rejections.

Power Elites: This has historically been a reward we provided to our authors once they reached the 2M+ mark to recognise their high value contributions. It has since expanded to include the 1M+ authors as well. These submissions account for a very, very small number of items overall.

So how does this look in practice? Below are some graphs that illustrate the WordPress Category on ThemeForest.*

*Note, “Reviewing Actions” refers to the number of reviews undertaken. As “New Items” is the only category in which demand exceeds supply, we have broken this into “new items” and “new items submitted”. The former is the number of new items reviewed, the latter is the number of new item submissions.

Firstly, let’s begin with the obvious; the wait time for new WordPress items is averaging sixty days. We’re a long way from where we want to be in this area, and there is a fair amount of work in our path before we arrive.

We need to openly face an uncomfortable truth: Whilst we are doing everything we can to try and reduce the wait times for new items, the actions we are taking will not have immediate impact. Things will improve, but it will take time, and things will likely get worse before they improve. It could be another couple of months before we see a reduction in wait times for new items.

We know many authors are waiting for new submissions to be approved, and in some cases, have a great deal riding on those submissions. For that, we can only offer our heartfelt and sincere apologies.

We are now focused on doing everything we can towards repairing the situation. Below, I’m going to talk in specifics about the work that we’ve done thus far, and what our roadmap is moving forward.

1. We’ve more than doubled the size of the ThemeForest review team.
We now have a fresh group of reviewers in the process of being onboarded to our team who are learning the ropes.

2. We’ve increased the size of the AudioJungle review team by 36%, to reach almost 50% over the course of the next month.
They too are in their early days of learning how our system works, but as they get up to speed, our capacity to review greater numbers of items will increase.

3. Our engineers have made some headway into resolving the enormous infrastructure challenges that we mentioned in earlier posts. Let’s get specific:

  • Auto-approval for qualified authors (due to be re-instated in the next few days)
  • Getting content to our reviewers faster to speed up review times. We’re close to completing a test of this, which is intended to speed up the average time to review an item. We expect to have some results in July
  • Trying ways to re-prioritize the queue (This didn’t work too well, see below for the challenges with the code)
  • Created tools to speed up review of updates to WP themes by detecting changes between versions. These are already in use by our review teams.
  • We’ve included all Power Elite Authors (1M+) in priority reviews.

We are also facing some serious challenges.

  • The code that our team is trying to work with, is, putting it mildly: incredibly complex. In some places, it’s nearly 10 years old. To give just one example: When a new preview is uploaded for a VideoHive item, there are four totally different sub-tasks that the system creates and has to keep track of, and three of them are tracked in totally different ways. Working with this stuff is incredibly complex and time consuming for our engineers. Many of you have already noticed that within the scope of our work we’ve encountered problems that have triggered unintentional side-effects.
  • Whilst our capacity to manage the number of submissions is increasing, so too is the number of submissions that we’re dealing with.
  • In some categories such as WordPress, items are becoming increasingly more complex and multi-purpose. This increases the time it takes us to review both new submissions and updates to existing items.

Which brings me to the roadmap of how we’re going to tackle this moving forward and the specific initiatives we have planned:

1. We will continue the process of hiring new reviewers to better bring supply to the level of demand.

We need to ensure our reviewers are supported and given the time they need to get up to speed. As they learn more, and become more efficient, the rate at which we can review items will increase. Quality control here is incredibly important, both in the recruitment of the reviewers (we’re quite picky) and the training we provide them.

2. We are continuing to focus on improving the tools and infrastructure that support our reviewers to do their work.

Much of this is around bringing in more automated checks on items to ensure our reviewers can get through more items, faster. These will steadily chip away and bring incremental improvement over time.

3. Encouraging higher quality submissions of items

One of our biggest challenges is the amount of time we spend reviewing items that are never approved for sale. To illustrate, approximately 75% of new items submitted to ThemeForest never reach a state of being approved by our Quality Team. We’re proud to be a marketplace that allows anyone to have a chance to succeed, but want to look towards ways to raise the quality level of items before they reach the point of being submitted. Over coming months we’ll be bringing in changes to the author experience to encourage submission of higher quality items, priming authors for success.

5. Better reporting on queue wait times

Although having better reports does not in itself improve the wait times, we want to improve the way we communicate wait times to our authors. Currently, we only display queue averages on our http://quality.market.envato.com portal. Whilst the average wait time is useful, it risks generalising as depending on the nature of the item and the author, there is variance. We know that better context gives our authors the power to make better decisions.

It is the unfortunate reality of our work that we cannot clearly say when the review queues will begin to drop. There are just too many unknowns. All we can do is focus our energy and attention on improving our team’s capacity to manage the queues.

We read each forum post and know well the pain that review queues are causing many authors, but we also wanted to take a moment to give heartfelt thanks to the members of our community who taken the time to express their support for us. Whether through applying to become a reviewer, a supportive email or simply a forum post, we are grateful to have you with us.

We will continue to keep you updated.

9 Likes

Thanks for being transparent. This should help me and many other authors. I know exactly now what to do. Thanks again!

And I hope review times will be back to normal some awesome day.

If is this is one of your biggest challenges why not also move elite authors up the queue too? Elite authors are already proven to submit approvable quality items and are already established in the marketplace. Why should they be in the same boat as a new unproven author? Your priority right now should be to review quality items from established authors.

11 Likes

Please set elite authors right after power elites in queue, i’ve sold more than $500,000 and working with envato last 4 years. I guess It’s not fair if elite authors will be in regular queue with new authors. For me this is not hobby it’s my only income.

18 Likes

So basically you are saying that only power elites means something, and the rest of us elite authors are not??? That’s really nice from you! Unbelievable…

6 Likes

As well as power elite authors can you prioritise submissions from users whos username begins with the letter ‘s’? That would be pretty swell.

3 Likes

None of this addresses the fact that many new authors were having their themes reviewed and often accepted in 30 days. Is it really on to prioritise them over established authors whose livelihood revolves around theme-forest and do not suddenly expect to be two months in a review queue when it was previously 7 days.

Its sad to see the June date of normalising the review queue completely go out the window but I suspected this would be the case.

One of our biggest challenges is the amount of time we spend reviewing items that are never approved for sale.

Automated checks so if a theme does not meet certain minimum requirements it does not have time wasted by the review checking it. I contacted yourselves about something like this I’d created but was told it was decided against to go down this route!

2 Likes

Maybe 4-5 years ago. I think two-three months is ok taking into consideration how big the market is. Also, you all complain about this queue, but in the same time you all submit themes with hundreds of demos, different WP or PHP frameworks and expect reviewers to know all your coding habits, not taking into consideration that some new authors don’t even know how to enqueue a script. This is the new norm, you should get used to it, it will not get any better.

That is not true at all about 5-7 years ago, even my last item was 30 days not 70 and that was the middle of may. My item before that was 2 weeks and that was last October.

Envato have openly talked about the reasons behind why the review queue is long. Have a read through the announcements so you are more aware of what’s happening.

Also whilst many people do submit 100’s of demos I don’t and do not agree with doing it either.

Hello Matthew,

Thanks for the “update” even if it doesn’t really update the current situation. You just finally admitted what we already knew.

Sorry to say this, but your post is just “Yet another false promise we made, friends, but Hey - let’s face the unpleasant truth - you should be used to it by now.”.

Let me ask you just two questions**, please:

  1. On average ~8-12 new WordPress themes are approved every day. Does this mean your 3-4 senior/experienced reviewers capacity is up to 3,33 themes/person/day + whatever the number of reviewed updates? If so - how do you plan to catch up with the pile of new submissions? Hire another 341 reviewers?

  2. Some days (e.g. yesterday) 3 of the 10 approved themes are made by Power Elites, which forms solid 30%. Are you planning to create a private theme club and let us “mortals” hit the road and find/form another big marketplace. If that’s the plan, would you, please be kind enough to inform us directly.


** - may contain irony

Last, but not least - Sorry for being “not supportive” lately, but you left us no choice. I can assure you that being rude and not supportive doesn’t give us ANY pleasure. Guaranteed! Most of us are grown, intelligent people. Anyways, being an Envato author is supposed to be a business venture. “We feel your pain, but can’t do better” would be more appropriate for a friendly meeting, not business operation. Just face and admit it - We are NOT FRIENDS! We are business partners! Do something. Really! Please!

6 Likes

That’s what they say on the surface, those are not all the reasons (that’s what I think). Because of this pandemic with multi purpose themes/frameworks and no guidelines, each reviewer needs to review hundreds of files and some crazy code.

I actually don’t blame theme authors, I actually want to blame Envato because they didn’t stop this when they should have.

I feel your pain, but let’s be realistic, the Power Elite Authors are Envato’s business partners, everyone else is just the unpaid intern trying to make themselves known. I’m not bitter about it though, it’s the way the World works, not just Envato.

1 Like

We need to openly face an uncomfortable truth

Why not at least be honest on the average review times page then, the 35 days things is really irritating, would much rather know the truth even if it is 60, 70 or 80 days.

Are there different levels of reviewers because I don’t feel it needs a fully trained coding genius reviewer to have the first look of a submitted theme. A much less trained reviewer could be used to quickly filter some of the obviously not up to design standards themes and significantly reduce the queue surely.

Also the power elite authors are the ones that are probably financially stable and can more than likely afford to wait in the queue more than everyone else because they have a ton of products which will keep them ticking over. A long review time is not going to affect them half as much as it will the authors with just a few themes trying to do this full time :frowning:

2 Likes

Knowing Envato, we all knew things weren’t going to get any better before late August at least. Which is all fine. However, prioritizing new authors over Elite authors is a bad idea.

Non-elite authors are still getting priority over Elites: (Elites are at ~60 days to get reviewed)

I can add more if you want.

1 Like

If you are making differences there are different stages of elite authors also :stuck_out_tongue:
The main point is, the first time actually the part that everyone is quoting was honest.

It will get worst before it gets better, meaning it will not get better all to the end of the year or longer. Couple of month always means more… and envato did state in the beginning that it could go into third quarter.

Meaning we will keep on loosing money. Who survives, survives.
Lets just hope envato will learn from this disaster to listen to early signs from small guys. We have been warning about this the day review times went up from 2 days to 5 days, but nobody listened.

The only question here is in my book, who is legally responsible for so many authors that will get financially ruined, and what will that side do to remedy or help those authors.

In comparison maybe: If i have a store and because of my mistake i close the door and clients cannot buy the product any more am i responsible for your damage and what do i have to do to ease it.

In the end things will get back to track quite a few authors will suffer/ have to close the shop go into dept etc in that time, those are the ones i fear for.

Cheers and Good luck everyone.

5 Likes