Register Members List Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
  #61  
Old 15 Nov 2009, 12:28
RCA RCA is offline
 
Join Date: Mar 2003
move completed:




is easy code a php script to capture files from one server and move to others (i do in original upload process), vb 3.x only needs avatars and profilepicture if remember correctly, all other files and images are full static.
Attached Images
File Type: gif scdn1.gif (13.9 KB, 243 views)
Reply With Quote
  #62  
Old 15 Nov 2009, 17:12
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by RCA View Post
is easy code a php script to capture files from one server and move to others (i do in original upload process), vb 3.x only needs avatars and profilepicture if remember correctly, all other files and images are full static.
That is not what I asked about.

I specifically said in my post "capture a 404 error on one web server and the retrieve the missing file from another web server...." (and then serve it on the first server, of course).

This is non-trivial.
Reply With Quote
  #63  
Old 22 Nov 2009, 19:07
Kaelon's Avatar
Kaelon Kaelon is offline
 
Join Date: Jan 2002
This is the first time I have heard about SimpleCDN and its game-changing "mirror bucket" feature. I definitely am eager to try this out, as I have just switched our Big Board (1.5 million posts, 40,000 users, around 200-300 concurrent a peak) from a dedicated colo over to the Cloud, and would love to migrate all of our static content off of the server.
Reply With Quote
  #64  
Old 22 Nov 2009, 19:15
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by Kaelon View Post
This is the first time I have heard about SimpleCDN and its game-changing "mirror bucket" feature. I definitely am eager to try this out, as I have just switched our Big Board (1.5 million posts, 40,000 users, around 200-300 concurrent a peak) from a dedicated colo over to the Cloud, and would love to migrate all of our static content off of the server.
Give it a try! We get around 4,000 to 4,200 peak users these days, and are reasonably happy with SimpleCDN, but they do "hiccup" from time to time. In the beginning, their team told me it was because of some DDOS attacks against their EU nodes. They told me all that would be resolved in Sept.

September has come to pass, SimpleCDN will not respond alas!

Wake me up, when November ends :-)
Reply With Quote
  #65  
Old 23 Nov 2009, 14:09
Kaelon's Avatar
Kaelon Kaelon is offline
 
Join Date: Jan 2002
I am definitely going to try an upload bucket and see how things go in the short-term before delving in too deeply.

Do you have a back-up plan for when SimpleCDN is on the fritz? Or does your site suffer, and then basically you're stuck with what has come to pass?
Reply With Quote
  #66  
Old 23 Nov 2009, 15:46
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by Kaelon View Post
Do you have a back-up plan for when SimpleCDN is on the fritz? Or does your site suffer, and then basically you're stuck with what has come to pass?
We CNAME alias our Simple CDN mirror buckets and use a short TTL in the DNS. So, if there seems to be a prolonged hiccup with Simple CDN we just repoint the CNAME to our origin server and bypass SimpleCDN until the problem is fixed.

If you host your own nameserver, you could easily automate this.
Reply With Quote
  #67  
Old 23 Nov 2009, 20:09
Kaelon's Avatar
Kaelon Kaelon is offline
 
Join Date: Jan 2002
Originally Posted by imported_silkroad View Post
We CNAME alias our Simple CDN mirror buckets and use a short TTL in the DNS. So, if there seems to be a prolonged hiccup with Simple CDN we just repoint the CNAME to our origin server and bypass SimpleCDN until the problem is fixed.

If you host your own nameserver, you could easily automate this.
This is fascinating. I've already gotten a single Upload Bucket and created a CNAME alias to point to it, for the specific static stuff that I want to serve. I'm thinking of creating a Mirror Bucket for the dynamic-CDN, but I have a lot of directories that have dynamic content in them:

/gallery/files <-- all of my user uploaded galleries
/forums/customavatars <-- custom avatars
/forums/customprofilepics <-- custom profilepics
/forums/signaturepics <-- custom signaturepics
/forums/thumbs <-- generated thumbnails
/forums/albums <-- uploaded albums
/links/banners <-- uploaded banner images
/forums/images <-- all of my other static forum images

What do you think the best way to create a dynamic-CDN "Mirror Bucket" for these would be? Should I change my hierarchies? So far, SimpleCDN is working like a charm.
Reply With Quote
  #68  
Old 23 Nov 2009, 20:18
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by Kaelon View Post
What do you think the best way to create a dynamic-CDN "Mirror Bucket" for these would be? Should I change my hierarchies? So far, SimpleCDN is working like a charm.
We have to be "speaking the same language" first. I don't know what you mean by "Dynamic CDN" when discussing in terms of Mirror Bucket. Every file can be updated based on the cache configuration of the CDN, and any file not found in the CDN cache will go to the original server to get it.

So, your directory for the CDN mirror will MIRROR your origin site.

We use two Mirror Buckets, one for images and the other for clientscript and CSS files, but that is not necessary and we paid for an extra CNAME alias. That way we could track the usage for the clientscript differently from the images.

It is simply a Mirror, so you can Mirror in as many virtual buckets as you like.

Also, if you need to update some javascript, you can delete the script from the CDN cache in the user admin control panel.....

Last edited by imported_silkroad; 23 Nov 2009 at 20:21. Reason: Auto-Merged DoublePost
Reply With Quote
  #69  
Old 23 Nov 2009, 20:29
Kaelon's Avatar
Kaelon Kaelon is offline
 
Join Date: Jan 2002
Okay, I think I understand this. So, for example, I could have one mirror bucket that basically mirrors my entire domain, but it will only fetch the content that is requested of it. So, does this mean if it is asked to produce /forums/images/21.gif, it will create the forums and images directories for me, and handle everything on its own? If so, then, I suppoes I should be fine with just one mirror for my entire domain given that I have so much user-generated content spread throughout all of my services.

1. Is there any security with SimpleCDN to prevent hot-linking (and thus, raising your bandwidth bills)?

2. How often do Mirror Buckets refresh or update images? For example, if a user uploads a new signature that overwrites the old one, will it be updated?

3. How do you recommend using the no-cache header linking options for your buckets?

Last edited by Kaelon; 23 Nov 2009 at 21:05.
Reply With Quote
  #70  
Old 23 Nov 2009, 23:22
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by Kaelon View Post
1. Is there any security with SimpleCDN to prevent hot-linking (and thus, raising your bandwidth bills)?
No, not at this time.

2. How often do Mirror Buckets refresh or update images? For example, if a user uploads a new signature that overwrites the old one, will it be updated?
If a user uploads a signature, the signature resides in the database, so that is not a CDN issue. The CDN does not work for PHP files that access the database. If you are talking about images in the signature, I think you will find that new signatures, like avatars, get a new (version) number appended in the name of the gif.

3. How do you recommend using the no-cache header linking options for your buckets?
We cache all our image and client content to the maximum number of days. This is configurable in the CDN admin panel. This overrides the no-cache header information in your files, as I recall; I suggest you open a ticket with SimpleCDN and ask them questions about how they deal with headers, cache, etc. That way you get the info directly from them.
Reply With Quote
  #71  
Old 10 Dec 2009, 20:27
RedWingFan's Avatar
RedWingFan RedWingFan is offline
 
Join Date: Oct 2004
Since our $15 "play money" ran out, I disabled using SimpleCDN for now. I have been monitoring our server CPU loads using Cacti, and there has been NO change in loads since we started serving images from our own server again. BUT, I am positive we would still see a result due to decreased disk I/O and a few CPU cycles, so we may revisit this in the near future when we raise the funds to pay into it. Our $15 lasts at least six weeks, maybe more.

All the while, though, I must say that SimpleCDN worked flawlessly. And my replacement variables are easy to reinstall, so I can be serving images from them again with a minimum of work.

I just hope vB4 lets us change things this easily.
__________________
-= N =-
Reply With Quote
  #72  
Old 11 Dec 2009, 12:34
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
Originally Posted by RedWingFan View Post
..... there has been NO change in loads since we started serving images from our own server again. BUT, I am positive we would still see a result due to decreased disk I/O and a few CPU cycles, so we may revisit this in the near future when we raise the funds to pay into it.
That all depends on your traffic. We have 4 to 5 M PVs a month.

If we turn off SimpleCDN, which we can easily do in our DNS if there is an issue with the CDN, we see huge load increases on our server and a very large difference in bandwidth status.

It's all relative, isn't it?
Reply With Quote
  #73  
Old 11 Dec 2009, 12:50
RedWingFan's Avatar
RedWingFan RedWingFan is offline
 
Join Date: Oct 2004
Originally Posted by imported_silkroad View Post
That all depends on your traffic. We have 4 to 5 M PVs a month.
We certainly don't have THAT much! I do see the difference in our bandwidth usage stats though, so I know I had SimpleCDN working correctly. Just eyeballing the stats, we are serving out about 2.5GB to 3GB more per day than when we were using SimpleCDN. (We are still using only about 25% of the bandwidth quota we have on our account, so that's not an issue.)

I estimate our static files (other than attachments) are using about 1/3 of our bandwidth, overall.

Maybe I'm trying to talk myself back into using it...
__________________
-= N =-
Reply With Quote
  #74  
Old 11 Dec 2009, 13:18
imported_silkroad imported_silkroad is offline
 
Join Date: Dec 2003
OBTW, I am working on a PHP script that checks the performance of the CDN, and if it not performing, it changes our $vbphrase[image_server] and $vbphrase{client_scriptserver] and the RRs to another server (or back to the original server).

I already have this script working for our OpenX servers, and when there is any issue on the net, it automatically switches over to another server (we replicate the database), or back again.

It is not that much work to even check one CDNs performance and if not up to some standard, switch to another CDN. Then, when the cheaper CDN is performing again, switch back over to that one.

Reply With Quote
  #75  
Old 13 Dec 2009, 18:29
Kaelon's Avatar
Kaelon Kaelon is offline
 
Join Date: Jan 2002
Generally speaking, I have only had a couple of pronounced moments when SimpleCDN stopped performing in the past three weeks. How has everyone else's SimpleCDN performance been throughout late November and early December?

Also, how are you guys changing your image variables? I noticed that imported_silkroad is using a $vbphrase[image_server] phrase (which sounds like a great idea, actually). I am using a CNAME alias and mapping all of my image calls to the cdn.domain.com to point to SimpleCDN's servers. Someone mentioned that simply remapping the CNAME back to our primary server might not be such a bad idea. How are you guys doing this?

Great thoughts all around - please keep them coming. I'd love to use this as a long-term solution to load-balancing on the basis of mitigating requests.
Reply With Quote
Reply



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


New To Site? Need Help?

All times are GMT. The time now is 12:22.

Layout Options | Width: Wide Color: