Rollout of HTTPS proxy for author hosted imagery

Envato Announcements
1

Hi there!

Starting today we are gradually rolling out a feature to a small subset of users to proxy some of the author hosted images via our content delivery network. We’re doing this because we’ve identified the slowest part of our page render time is spent fetching third party resources for author hosted images (such as demo images in item descriptions). Often these assets are stored on shared hosting and not utilising a content delivery network which means that while your browser is attempting to render the page, it needs to fetch these resources from a single remote origin which could be on the other side of the world.

The way we are achieving this is by rewriting the content at render time to replace all asset links with a URL that we have generated based on the checksum of the remote file. The newly written URL is proxied through our CDN and outputted to your browser ensuring it only hits our CDN for the asset (and caching it for future requests). Here is an example of what it would look like using a HTML image tag in an item description.

Before

<img src="http://mywebsite.com/images/banner.jpg" />

After

<img src="https://camo.envatousercontent.com/7y65d6d7g998jt98gh4t8/bvknvkvbnnbvvhjk7" />

Here is also a visual of the request flow with the image proxying introduced.

A drawback of this approach is that authors who use some third party scripts to count impressions on the item pages will notice they are getting more concentrated hits from a single IP (our CDN nodes) instead of the client IP. The reason for this is that the tracking service does not use the X-Forwarded-For HTTP header and instead relies on another HTTP header for the client IP.

If you update images on your image host end, remember to also update the name used so that it will result in a new image being generated on our CDN and it won’t end up with an incorrect cached version. Failing to do this will result in the old image being served until the CDN time to live (or TTL) has expired.

From a buyer perspective, they will see these pages rendered faster thanks to being able to fetch the asset from a CDN node closer to their location - hopefully leading to a higher conversion rate! Aside from the performance improvement, this is a major step forward towards our goal of running the marketplaces over HTTPS everywhere instead of being limited to pages where author hosted imagery isn’t present such as the checkout process.

If you have any questions or concerns, please don’t hesitate to get in touch (via a support ticket or forum reply) and I’m more than happy to explain anything that may have been missed or not explained clearly enough.

8 Likes
2

Awesome stuff!

3

Does the author have some functionality to purge CDN cache for images on sales page?

In our scenario - We updated an image on sales page but the file name was still the same because of that the CDN was not updating the new image.

~ Nik

4

Hey,
As mentioned above if you update your images, you will need to update the filename used so that the image hash is recalculated otherwise it will continue serving the old asset. Purging isn’t something we’re looking to add for this rollout.

5

Its time to allow authors to upload image directly to Themeforest. @jacobbednarz

Why should we still upload to 3rd party sites?

WYSIWYG Editor FTW!!

3 Likes
6

I already upload my images on CDN (Amazon S3), and they are all with HTTPS. Is there a way to opt out of this system, there is no need to force it if it is already done with CDN and HTTPS?

7

@surjithctly this is something we have tossed around internally as a solution however we haven’t pursued that avenue yet. I don’t think we would implement a WYSIWYG editor without overhauling the whole user input side of things as we are running quite a frankenstein setup for input rendering. Right now we run a (very) custom version of textile and that in itself is difficult to maintain so we’d need to look at an upgrade path for the data so every author didn’t need to update all of their items when we made the swap. In an ideal world, we’d probably move to something better supported like a standardised version of Markdown and just use an editor that supported the functionality on offer.

@SMARTPlugins we only send HTTP assets via our asset proxy. Everything else will stay as is :smile:

1 Like
8

@jacobbednarz Yeah… Even if we don’t have more controls, We just wanted a quick way to use headings, anchor tags and images and a preview along side. This will save a lot of time for authors.

Thanks for the consideration.

9

Thanks for the feedback! I can’t promise anything but what I can do is feed this onto our product team and see if we can get some love for the author/item description and it’s attached functionality as this would not only help authors but also solve some of our underlying maintenance issues we have in this area.

1 Like
10

Hi,
I have a simple suggestion for author hosted images. You should offer a separate CDN like website where authors can upload extra images. You can also allot fixed storage space Eg: 40-50 MB.
Additionally, you can configure that CDN images can be hot linked only on Envato market sites, to prevent bandwidth leakage.
Thanks

11

Hey I have issues with the GIF files of some of my items you sent to cloudfront !

2 possible issues :

12

I am on Chrome - Windows 8.1

And it seems I am not alone :
http://videohive.net/item/minimalism-2/13979481

This problem seems to impact GIF only and maybe heavy ones ? …

Found the problem : It seems it is coming from my internet provider. This is very strange. Hope I am the only one.

14

As simple as an image hosting service sounds, it would require quite a bit of effort from both Envato and it’s authors in order to rollout (not only technically but also in terms of business logic around author limits, features, access, etc). Due to the complexity we seen up front with a solution like this we took the route of simplicity and what met our needs at the time.

There is nothing to say we won’t change this in the future but at the moment we don’t need a full blown image hosting service in order to achieve our current goals. Thanks for the feedback though!

15

I’m already using S3 for hosting images and was thinking about plugging in CloudFront.
You guys are using CloudFront, right?
How would that work with images that already use CDN?

16

Providing you use HTTPS links, it won’t send them through our asset proxy and will use your designated origin. I’d highly recommend cloudfront (if you can afford it) as it will speed up your page load times by a sizeable amount if they are continuously getting viewed.

1 Like
17

Hi Jacob!

We are currently having 40+ products at marketplaces. We have feature to generate changelog image for every product. With your caching solution, this will not work anymore, because the image name has to be same. Any solution for this? For example sending HTTP headers with expiration, or creating special img attr nocache=“true” ?

cheers, Thomas from freshface team

18

@FRESHFACE just host your image on SSL already and it will bypass the CDN (that may change later though)

Also try setting correct no cache headers.

19

@dtbaker is spot on. If your HTML image element is using HTTPS already, it won’t be sent via the HTTPS asset proxy (as I’ve mentioned before here). Setting no-cache headers will also help as we respect origin cache control however this is something that we may change in the future depending on a few factors we are currently looking at.

Regarding your image change log, I would really plead with your team to rethink the strategy there as it doesn’t seem like a good use of an image and it will be counting towards your page load times. We are working on some tooling to help authors identify issues like this as the item page response times are some of the worst in our application due to the amount of imagery and third party calls being made. One alternative could be storing a text generated version on your website and just linking to it form the item itself.

20

I’ve bundled text change log generation into the Grunt task that packs up my WordPress theme for distribution. Grunt spits out a new item description page with each new version and I just copy/paste that over.

However it would be WONDERFUL to have a public change log / update section on certain items, so authors don’t have to come up with their own fancy way of displaying a change log each time.

This could even be used as a basic bug tracker, where support staff could add a “buyer reported bug” to the system and mark if for fixing, or reviewers can add soft-rejection/soft-disable notes as individual line items that can be ticked off as completed. Or even with bulk changes like adding a “make sure your theme supports WordPress 4.5” to all themes and having authors tick that off, then soft-disabling all others that don’t comply. </brain-dump>

21

Hi Jacob!

thanks for providing your answers :slight_smile: Anyway we dont have SSL certificate at our server in current time, so this will add additional hustle for us.

About the changelog strategy, thank you for your suggestions, we already though about this, because we have like 40 items at envato marketplaces, so you can imagine the pain when we have to edit anything. Easiest way to solve this would be to allow changing the item page trough API. I suppose, that you are in different development team, but just in case, do you think this feature will come anyhow soon?

cheers, FRESHFACE