A Transparent Treatise of Inconsistency in AJ Review and Author Processes

Thank you Adrien for going above and beyond your job responsibilities and offering insight into a part of the process that we can only guess about at times. The realization and awareness that AJ has increased its review standards has made me more focused on improving my products. I believe this is a healthy way for the community to think about it as well.

Nice article

Yes, but will the item price increase follow? Good quality costs a lot of money for both this marketplace (more rejected items) and the authors (higher chance of rejected items). So, I think the item price increase is justified.

Thanks for the post…very interesting…!

WorldBeyond said

Yes, but will the item price increase follow? Good quality costs a lot of money for both this marketplace (more rejected items) and the authors (higher chance of rejected items). So, I think the item price increase is justified.

+1 it would be nice to fight the devaluation of music if only a little bit.

Hi again everyone,

Glad this topic had a positive impact and resonated well with so many of you. All the feedback is very much appreciated by everyone, and just wanted to answer a few of the standout questions and comments here. Hope everything is more clarified now. :slight_smile:

MindTransmission said

Hello, I don’t know if this has been covered before, but I’m wondering if the reviewers work independently in their own studios (using their own monitoring setups) or is there a central office at which they all work?

Reviewers are spread out all over the globe, but they also work together in realtime.

LeatherwingStudios said

But given how successful Envato has been I would love to see you add reviewers so borderline rejects would always be reviewed by a second person without that second person knowing it was already rejected. And it does not have to impact the queue if you simply absorb the additional cost of additional reviewers out of your ample profits!

Also, add a tiny bit of time on each reject to tell the author why the item is being rejected. Again, this will not slow down the queue if you simply add reviewers and absorb the additional cost out of profits! If a reviewer goes through a process to reject why can they not have a simple but complete form to check off the criteria for the author? Yes, I know it is not your job to teach, blah, blah. But these authors provide the means for Envato to succeed! They deserve the respect of knowing why their submission was rejected in my opinion!

Obviously you do not have to do either of these things because you are successful enough as it is. But these things would not cost a lot and would greatly improve the experience for your suppliers and the community in general!

That is asking for the company to make a very substantial investment of time and resources. It would be a lot more than you might think. Keep in mind that this used to be the approach in the past. Even though Hard Rejections have no “intrinsic” value, as they do not exist as items in the library, and cannot be sold, they used to get a lot of very specific feedback to attempt to steer or guide the rejected tracks into something more salvageable.

However, over the years, careful analysis has shown that the vast majority of hard rejected items, even when they receive substantial and detailed feedback are exceedingly less likely and unable to be reworked into an acceptance. This isn’t an anecdotal speculation. It’s what really happened over several years. So this is why today we have to reserve feedback for soft rejections only.

||+1250096|Sky-Productions said-|| Maybe the best thing to do Adrien is implement a new rule...any threads that are clearly a complaint against a hard rejection will be deleted or locked...just like self promotion or spam.

Well, at the outset we don’t really want to muzzle the community’s freedom to discuss item rejection topics so long as ideas not are brought forth in a discourteous or unmannerly way. We also appreciate to be able to monitor the discussions, as it helps us measure the quality of the work we’re doing. It would be a bit heavy-handed to formally forbid discussion of rejections, as MusicBox mentioned because of the values Envato strives to hold at its core. Whether or not any rejection chatter impacts the impressions people get of authors questioning or contesting their submission results, it’s probably something that can remain in the item discussion for now.

But yeah if posts do get out of hand, they’re certainly going to be addressed, case by case or otherwise.

FASSounds said

Great info,it’s nice to know about the borderline things. So I think it’s still matter of luck if we just got a middle quality track

Mostly no, but a little bit of yes. That’s where the potential inconsistency stems from, inherently. As Andy mentioned, reviewers are not rolling dice. There is a concerted effort to minimize subjective decisions in the review process. But if a track is very, very borderline, i.e. really right in that theoretical middle area, then it can be either an acceptance or a rejection, potentially, depending on various factors. If it gets rejected, the types/number of issues that make it borderline will determine if it is a soft or a hard rejection.

SmartOwlMusic said

Why are tracks in the grey or borderline “hard rejected” when a soft rejection sounds like it could be more appropriate? If a track could go either way, wouldn’t it be more appropriate to soft reject with a little detail to improve the track?

As above, it depends on the type/number of issues. E.g. If there are numerous smaller or subtle but important issues on a borderline track, deemed too diverse to explain individually, or due to considerable details needed to be conveyed separately, then it will more likely be hard rejected. On the other hand, a borderline track that has a single major issue, for example, it would be more prone to be sorted with a single directive, reliably, so that would be more likely to be soft rejected.

BrancoG said

When are we going to talk about the fact that non exclusive authors have no chance of getting trending or featured? That bothers me a lot more than hard reject from time to time…

That’s outside the scope of this post, but it’s the company’s prerogative to moreso feature content that is not found on other competing libraries or licensable elsewhere online.

Gorynich said

So many words.There must be also a result, I hope.

We all hope so. The truth is, it also depends on you. :slight_smile:

WorldBeyond said

Yes, but will the item price increase follow? Good quality costs a lot of money for both this marketplace (more rejected items) and the authors (higher chance of rejected items). So, I think the item price increase is justified.

That’s a notion that may have conceptual merit, but it’s outside the capacity of Quality & Review Teams to comment or speculate on the matter, at this point.

lemega said

My tracks are not borderline, they are histrionic. SCNR

You win, Lemega, you win. :smiley:

Thank you for the detailed reply, Adrien! It’s nice to feel like our voices are heard and it’s very appreciated that you took your time and made such a well thought out reply to some of our questions. And it’s interesting to know that our reviewers are working their own version of 50 Shades of Grey. :wink: I’ll take everything you posted into consideration and of course continue trying hard to strive for success!

I agree with some of the others on the subject of rejection posts. I don’t always post about my rejections, but it is nice to be able to do so if I want to, and I’m glad AJ still plans on letting us keep our community vibe where we can meet up and discuss our wins and losses.

ADG3studios said
MindTransmission said

Hello, I don’t know if this has been covered before, but I’m wondering if the reviewers work independently in their own studios (using their own monitoring setups) or is there a central office at which they all work?

Reviewers are spread out all over the globe, but they also work together in realtime.

Thank you for answering my question!

Thanks Adrien, for taking the time to explain to all of us how things work, it sounds very fair to me.

+1 - well written with humble heart. Well done, Adrien!

FlossieWood said

Thanks Adrien, for taking the time to explain to all of us how things work, it sounds very fair to me.

Thanks Adrien,really appreciate your time to read and answer all those questions. Keep the good works and spirits!!! :smiley:

Yet still my question is why are good quality, original sfx hard rejected while hundreds of similar, many of low fi quality,over populated sfx (thunder, rain, etc etc) are being accepted en mass every day ?

+1, thanks!

FlossieWood said

Thanks Adrien, for taking the time to explain to all of us how things work, it sounds very fair to me.

Adrien, I can’t thank you enough for taking the time to post this. I really believe this needs to be a stick and required reading. In fact, if it were me, I would make this a yearly review item for all authors and capture that each and every author has reviewed it.

I may be one of the few, but infield the review process one of the finest and fairest I have encountered. If something gets rejected, I have complete freedom to choose how to respond. I can try to improve and resubmit, i can submit elsewhere, etc. I submitted the track meaning that, basd on my research, I believe that AJ is a viable consignment space for my goods. But even in a consignment store, the proprietor reserves the right to reject some products for inclusion.

I was shocked to find out the the review team has 11 members. I thought the number was much, much higher. Kudos for doing so much with so few resources.

Thanks again!
Michael

PS - Now can I be a featured author? :wink:

PPS - Just kidding. My response was sincere.

PPPS - But can I, huh, can I??

ADG3studios said

We’d really love to give you examples of the “best of the worst” we hear, for your elucidation, if anything, but for privacy reasons we simply can’t. But we have a feeling most of you wouldn’t believe your ears.

If you can picture as an example, a track that sounds like a locomotive having intimate relations with a chainsaw while a music box sample plays in the background in a completely different key, with birds singing here and there, and random atonal electronic vocal accents… Well, you get the picture. Very creative perhaps, but AudioJungle is not the right library for that! :slight_smile:

Hi Adrien. Thanks again for the post but you’ve (once again) failed to acknowledge an important topic that I’ve attempted to address several times now. I’ve contacted support but still haven’t received a reply. To cut to the chase…

When is my garden tools with music box + trains of the world with birds sample pack going to be approved…!!! (By the way it was a lawn edger, not a chainsaw! There’s a difference you know.) I’ve checked sfx category and there’s nothing like what I submitted currently available. I know it’s a niche market I’m aiming at, but I’m just trying to ensure diversity within AJ. Maybe New #1 sfx author has all the power tools/locomotives/chainsaws/birds you need? (Well whoopdie doo - good for them…)

I appreciate all the fancy mumbo-jumbo about reviewer subjectivity and commercial “utility”. But simple reality check, pal…riddle me this…what’s a buyer going to do when he or she needs the sound of a train approaching a music box while sparrows fly in background…with a chainsaw operating nearby…(with implied sexual/romantic overtones)…? Well?

I’ll tell you… nothing. That’s right. Nothing. 'Cause your review team thinks it’s not worth anything, or lacks “utility”. You know how many locomotives I had to record? You know how many birds? You think I like hanging around hardware stores recording power tools? You ever talk to a power tool salesman, or try and convince them to do you a favour? Do you have any idea of the long term consequences of making deals with some of these kinds of people?? You think implying romantic relations between a train and a garden edger is easy???

Next time I hope you consider how much work goes into this stuff before rudely dismissing my submissions as fodder for your “humour”. You’re a Canadian, right? Figures…

:wink:

Thanks Adrien, very well said post that is invaluable to AJ authors. What really surprised me was only 11 reviewers, I thought that number would’ve been a lot higher.

I totally understand the human nature in choosing what is good and what is bad within the creative/art world… which makes the creativity process so exciting. Within my career as an Art Director I have to judge submissions at an equal level and not let my personal tastes make my decision. You need to understand the clients needs and wants, which I would imagine the 11 reviewers take into consideration which allows them to make the best choices possible for RF sales.

Unless AJ introduces a computer generated option for reviewing, there is no way we will make every reviewer agree to what we think will sell.

Take a deep breath, accept your rejected track and improve upon it and appreciate the reviewers decisions and comments. Keep creating and lets hope there will never be a computerized review system.

11 is the right number of reviewers for audio jungle. Most marketplaces have 10 reviewers, but these guys can go one louder if they have to.

ToivoMedia said

11 is the right number of reviewers for audio jungle. Most marketplaces have 10 reviewers, but these guys can go one louder if they have to.

))))))))))) lol))
Excuse me for offtopic) could not resist)

ToivoMedia said

11 is the right number of reviewers for audio jungle. Most marketplaces have 10 reviewers, but these guys can go one louder if they have to.

close the contest…You Win The Internet with that one!!! :smiley:

ToivoMedia said

11 is the right number of reviewers for audio jungle. Most marketplaces have 10 reviewers, but these guys can go one louder if they have to.

Haaaaaa, thats awesome!