Register Members List Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
  #1  
Old 05 Aug 2009, 13:58
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
/clientscripts over to Amazon S3/Cloudfront?

We have moved most of our static images over to Amazon Cloudfront/S3 content delivery network (CDN) over the past few months with good results.

Now, I am considering moving all the vB clientside Javascript in the /clientscript directory over to Cloudfront as well.

Anyone else using a CDN for the JS in the /clientscript directory?

Any issues other than keeping in sync with updates anyone can think of?
Reply With Quote
  #2  
Old 05 Aug 2009, 14:26
R1lover's Avatar
R1lover R1lover is offline
 
Join Date: Apr 2006
Location: Northern Ca
Why would you want to do this?
Reply With Quote
  #3  
Old 05 Aug 2009, 14:32
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by R1lover View Post
Why would you want to do this?
Because these (clientside) scripts account for (approx) 1/3 of the traffic, so moving them to a CDN will reduce the load on the server and also increase performance (just as it did when we moved most of all the static images).
Reply With Quote
  #4  
Old 06 Aug 2009, 14:31
kmike kmike is offline
 
Join Date: Oct 2002
You should consider adding "Expires" HTTP header to .js files, so the user browsers won't have to re-check these files for updates on every page load. And by the way, the same could be done for the static images.

For the record, .js files (with the Expires header) account for 0.9% of our bandwidth usage. Forum images take another 1.20%.
Reply With Quote
  #5  
Old 06 Aug 2009, 14:45
R1lover's Avatar
R1lover R1lover is offline
 
Join Date: Apr 2006
Location: Northern Ca
I would think putting your money in a better setup/server is better then putting your money into another host, in this case the cloud setup.

I agree that static images and .js have little to no impact on any of my servers or sites.

There is more to be had with a good server and proper setup.
Reply With Quote
  #6  
Old 06 Aug 2009, 14:56
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by R1lover View Post
I would think putting your money in a better setup/server is better then putting your money into another host, in this case the cloud setup.

I agree that static images and .js have little to no impact on any of my servers or sites.

There is more to be had with a good server and proper setup.
We have a good server and have run benchmarks with Amazon S3/CF. You cannot get the same global performance from a single server or local cluster than you can with a global CDN.

While your "opinion" is valuable. It is also incorrect because we have alread run the performance numbers and know the gain realized from the global Amazon CDN.

OBTW, Amazon CloudFront services are not "another host", so it seems you enjoy an "opinion" about something you do not understand. CloudFront is a Content Delivery Network with (seven I think, at last count) global geographic locations. It is not a hosting solutions.

We run over 500 metrics on our configuration and know the performance better than an uninformed third party who, frankly speaking, is just "hip shooting".

In fact, in the vBSEO forum, there are a number of users, like us, who server static content off the main server. Some use other hosts, others, like us, use a global CDN.

Sorry to be so direct. Facts are facts.

Cheers.
Reply With Quote
  #7  
Old 06 Aug 2009, 15:06
R1lover's Avatar
R1lover R1lover is offline
 
Join Date: Apr 2006
Location: Northern Ca
Not wanting to argue here, but if they are storing your images and then serving them and you are paying for this service, then in fact they are a host... lol

The only benifit I an see is one that you mentioned, being globally located for faster service to people on the other side of the world.

Other then that you are still paying for something you should be able to do yourself.

This all changes as the size of the forum has a large impact on these things as well, but since you didn't mentione this I went with what the aerage user would need and this service for the average user is useless imo.
Reply With Quote
  #8  
Old 06 Aug 2009, 16:01
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Let's not argue, because you are shooting from the hip and will just keep shooting yourself in the foot my friend :-) I posted here, not do debate performance, but to look for configuration issues related to vBulletin, but I was able to work with the folks on the vBSEO board to fine tune all those issues.

Here are some published data related to CF/S3 performance, you can review yourself.

Amazon CloudFront / S3 Small Object Test Result

Huge client side gains for using CloudFront (across the board), some as high as 95% improvement.

On a different note, related to server side apache GETs (which effects apache workers, load, etc.)

As I posted earlier in this ill-fated thread, our /clientscript traffic accounted for 30% of all our GET requests (before I moved them to S3/CF yesterday). (The static gifs are mostly gone off the server and were already served from CF globally).

Regarding the reason I posted here, I was not looking for a performance discussion, as I know the performance intimately, serving millions of PVs per month.

We are all set now..... We had a very useful discussion over at vBSEO on this.

Cheers.
Reply With Quote
  #9  
Old 07 Aug 2009, 02:51
motowebmaster motowebmaster is offline
 
Join Date: Feb 2006
I would be interested in the outcome of this strategy.
__________________
Shawn - MotoWebmaster
Motorcycles International
Reply With Quote
  #10  
Old 07 Aug 2009, 13:27
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by motowebmaster View Post
I would be interested in the outcome of this strategy.
The basic outcome has already been published:

Amazon CloudFront / S3 Small Object Test Result

I can confirm that moving the /clientscript also results in better server performance, especially if you have a lot of .js code (from mods, plugins, etc.).

Globally, pulling static objects like images and Javascript results in faster response time for end users (unless they are next to the server, of course, see results above), decreased Apache workers, decreased bandwidth (out from the main server, increased at CF/S3 of course) and a decrease in load average (load results depend on a number of factors and is not easily qualified).

Please note that vB/Jelsoft uses the same strategy (for a long time) in the Server Settings and Optimization Options:

Use Remote YUI
YUI (Yahoo! User Interface Library) script files are hosted locally on your server, you may however have them served from Yahoo's own servers, saving you some bandwidth and potentially decreasing load times.
So, you can see that Jelsoft/vB already use this strategy for the YUI scripts to increase performance, etc. Using a global CDN (like Amazon S3/CF, Limelight, etc). for static content (images, clientside scripts, etc) is the same strategy (but on a larger scale).

:-) Maybe we can convince Jelsoft to serve their scripts in S3/CF and permit all vB customers to pull from the global CDN as part of the license :-)
Attached Images
File Type: png Picture 12.png (18.0 KB, 124 views)

Last edited by imported_silkroad; 07 Aug 2009 at 13:44.
Reply With Quote
  #11  
Old 23 Aug 2009, 17:56
Webcams Webcams is offline
 
Join Date: Sep 2007
Not knowing all of the fine details involved, is it difficult to set-up to work through vbulletin?
Is the process very intense and is it easy to set-up & test before you actually 'launch' the cloud?
I have set-up the vbulletin to work on my shared server with very few problems I just don't know if this is going to be very intense, or a matter of simply uploading the files to amazon and it will serve them.

Are there timeout issues when the local amazon server is serving images and such quicker then the main server is handling the rest of the process? I am afraid of users timing out on posting new threads and replies.
Sorry if this is to simple minded but I want the best experience for my visitors even if I don't possess the knowledge others do.
Reply With Quote
  #12  
Old 23 Aug 2009, 18:02
royo royo is offline
 
Join Date: Jan 2005
There's no advantage to the cloud when compared to regular servers, in fact you will overcomplicate yourself having to learn how that specific cloud works and adapt to it's limitations, and learn how to scale with it. Pricing wise, the bandwidth is very expensive, and will be cheaper to go with a server.
Reply With Quote
  #13  
Old 23 Aug 2009, 19:15
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by royo View Post
There's no advantage to the cloud when compared to regular servers, in fact you will overcomplicate yourself having to learn how that specific cloud works and adapt to it's limitations, and learn how to scale with it. Pricing wise, the bandwidth is very expensive, and will be cheaper to go with a server.
This is definately not the case.

We use S3/CF on a site with around 4M PVs per month, and found using S3/CF was so easy to set it, it was almost trivial, and our users notice faster downloads.

In addition, a good server, no matter how gigantic, cannot outperform a globally distributed content delivery network (CDN) which is what Amazon CloudFront is.

I think Mr. Royo is confusing "cloud computing" with Amazon's CloudFront, which is a global CDN, not a "cloud computing infrastructure".

In addition, I am not posting from "theory", we actually run it, serving millions of users each month from over 200 countries
Reply With Quote
  #14  
Old 23 Aug 2009, 22:43
tpearl5's Avatar
tpearl5 tpearl5 is offline
 
Join Date: Nov 2001
Real name: John
silkroad - I recently moved just about all my js files and images to amazon s3 w/ cloudfront - results are good so far. This took an additional 4-5gb a day off of the database server (where they were previously being served)

Basically all I did is search templates for "clientscript" and added the CNAME I created for cloudfront in front of /clientscript

I would like to figure out how to move avatars and other 'dynamic' images over to s3. I read through this thread wich looks like didn't result in much: http://www.vbulletin.com/forum/showthread.php?t=302300
__________________
John
Reply With Quote
  #15  
Old 24 Aug 2009, 01:28
RedWingFan's Avatar
RedWingFan RedWingFan is offline
 
Join Date: Oct 2004
I posted my experiences with CDN in that thread.

I used replacement variables rather than change the templates, and am using SimpleCDN on a test basis for now. I've noticed the bandwidth usage drop, so I know it's working.

On our server, avatars are in /forums/images/customavatars, so they are served by the CDN thanks to my directing the entire /images directory to SimpleCDN. The difference, though, is that SimpleCDN uses a "mirror" type of delivery where, if an image is requested from the CDN, and it is not cached there, it will grab it from the server. That is why I can safely push the avatars to visitors via SimpleCDN.

I don't know if Amazon has any mechanism like that, or if you could use something like rsync with your avatar directory to an extent where new avatars would appear after a short delay. (If I did it, I would change text to tell visitors that their new avatar would be active within five minutes, and have cron run rsync every five minutes to push the files out to the CDN.) Not ideal, of course.

Just some random thoughts (from a mind that is currently half awake ).
__________________
-= N =-
Reply With Quote
Reply



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


New To Site? Need Help?

All times are GMT. The time now is 13:34.

Layout Options | Width: Wide Color: