A reminder that you can check for any issues across the Envato sites at http://status.envato.com/.
Over the last week or so there have been some issues affecting the upload process across Envato Market. We needed to disable the upload feature at times in order to protect the broader system and prevent any impacts on sales and other customer features across Market.
As well as the downtime for uploads, the issues also impacted some other processes including trusted approvals of updates. We’re working hard to catch up on these now and a majority of the queued transactions have now been processed.
As you can imagine, we process, move, delete and update A LOT of files across Envato Market. With more than 11 million items for sale (plus the millions of files that are archived, replicated, disabled or deleted) we need some robust disk management in the back-end of market. We’ve made some massive improvements to the infrastructure across Envato Market over the last 12 to 18 months and these have brought huge performance and stability improvements to the 10 year old platform.
Unfortunately over the last week, some of our disk infrastructure had some issues - we had a node in our storage cluster fail. While we worked to fully restore the system, our performance was heavily degraded so we decided to suspend uploads to ensure customers could keep buying without interruption.
These issues were on the back of some other instabilities in some of our shared file systems that resulted in some corrupted directories which have since been isolated and recovered.
We’ve now taken a number of actions to try to ensure these issues don’t happen again, including more and better monitoring and reconfiguration of some of our disk infrastructure.
To be clear - no existing files were lost or permanently corrupted as a result of this downtime. We have backups in place to ensure we can always recover if something goes wrong.
We’d like to take this opportunity to apologize to anyone affected by these issues, and reinforce our commitment to improving performance across the board.