Below is the first update on this topic. Further updates can be found in the links below:
Hey guys, Matt here.
Apologies are due here. There’s been a great deal of pain and frustration as a result of the current waiting times. We’ve decided that Jarel’s time is best spent on working to fix the issue. After having a number of discussions with him, I felt I could share with folks the state of the union.
Firstly, some context is required about the system that works to review our items. When we talk about a review queue, it’s natural that those words conjure up images of something a bit like this:
However, in reality the system is a great deal larger, with a lot more moving pieces:
Each month, our review team inspects over 500,000 unique items, not including updates. Also, the review queue is far from a straight line ending in a reviewer. There is an absolutely huge, and ever evolving level of technical infrastructure (engineering work) that sends the items to different places at different times and supports our reviewers to ensure that their time is spent as efficiently as possible.
So how did we get to where we are now?
As many of you are aware, there are three different outcomes from the review queue.
- Hard Rejection
- Soft Rejection
- Acceptance
Where this began was within an in-depth discussion of the differences between a Hard Rejection and a Soft Rejection. Frankly, our team felt we were Hard Rejecting too many items, and so we had an internal workshop to clarify the differences between a soft and hard rejections. Here’s where we landed:
For an Item that has been hard rejected to be accepted, the item would essentially have to be changed so majorly that it would in essence be a completely different item altogether. Therefore, when we review, we wanted to be asking ourselves some important questions: Does this item have potential? Could it get over the line? How can we help get it there?
The way we wanted to move was clear. We wanted more soft rejections and less hard rejections. The problem however is that soft rejections require far higher levels of reviewer attention. Additionally themes are significantly more complicated now than they were a few years ago, resulting in greater length of review time. Soft rejections require us to know the item in higher level of detail, and at their best, deliver key insights to an author on how they might improve their work. We don’t always get it 100% correct, but when we do pull it off, the result is a big win-win for the author and for Envato. Change was required.
Changes to big systems like ours take time and work. Recruiting new reviewers is just one part of that puzzle. The bigger challenge is actually in altering the infrastructure that supports our reviewers, one which was built to support the old way of working. We held off on the recruitment element because we chose to work on our infrastructure first, rather than induct new reviewers to two systems. However, with the attention of Envato’s engineers pulled in many different directions, the work was put in a queue of it’s own. This leading us to where we are now.
Let me be clear, we are at fault. Balls were dropped in our transition from one world to another. We don’t write this post with a view towards asking forgiveness as we don’t believe the frustrations of our community are to be brushed aside lightly. We write this instead because i’ve been in enough queues myself where I’ve sighed in frustration, clenched my fist and just wished I knew what on earth was going on. Well, the above is what’s going on, and the below is what we’re going to do about it.
As Collis has already stated, we’re in the process of recruiting more reviewers. More essentially though, the whistle has been blown and we’ve now got a crack team of engineers working on improving the all important infrastructure that support them. This will take a little bit of time, but we’ve got a roadmap for what needs to be done (we’ll be sharing pieces of this soon) and the wheels are now moving in the right direction. We’re confident that soon, we will have made headway in bringing the queues down to levels that are sustainable and manageable for everyone.
