Yes, removing all music will surely increase your sales! Honestly sounds like an emotional decision coming from a feeling of frustration and maybe anger. So yes, it sucks obviously… but instead of going all nuclear, perhaps consider becoming non-exclusive and branching out into other marketplaces? If you get no sales or only a few in a month, the extra dollars you make per sale as an exclusive author most probably doesn’t outweigh the money you could make elsewhere.
And by being non exclusive and keeping your tracks here you will still get some money from Envato, while building up your portfolio and increasing your chances of transitioning to Elements at some time, instead of eliminating these income sources completely + will have an opportunity to earn extra money elsewhere. Sounds like a win win to me. But obviously it’s up to you.