Results 1 to 10 of 10
  1. #1
    Full Member
    Join Date
    January 18th, 2005
    Posts
    396
    Caching thumbnails on the fly -- Scott (Snib)
    Scott - in one of your 2007 postings you mentioned that you cached your data thumbnails on your server. Could you please explain a bit about this process?

    How you download the original larger pictures from all your vendors

    What technique you use to do the actual resize

    How you index and store such a large number of objects

    What date flag triggers you to add a thumbnail

    Is this process done separately with your main update process

    Thank you - Charles

  2. #2
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    I do the same thing as Scott, but not in php, and I use the CDN layer from Softlayer to avoid the disk space issues. The number of images I have to store is too large and would require many terabytes of data.

    My image urls look like this:

    http://cdce.internapcdn.net/xx/myImageHandler.aspx?originalImageUrl=http%3A%2F%2Fsomeaffilateimageserver.com%2Fimage678.jpg&width=90

    The CDN layer knows that xx is my server so on the first request, they substitute my domain and call myImageHandler.aspx

    myImageHandler turns around and requests the image from the originalImageUrl query parameter, then does an in-memory resize using graphics code that I wrote. I resize it to width of the width query parameter, keeping the aspect ratio the same. I have safeguards in place to make sure the width parameter is within a valid range so that someone couldn't bring my servers down my requesting a million pixel image.

    I then write the re-sized image to the output stream, after setting the cache headers so that the CDN layer knows how long to store the image in the cloud before requesting another one.

    So, my resizes take place the first time the user requests a page. Subsequent requests will be served by the content delivery network. I've got several monster servers, but the first couple days after the image handler goes live can put the server under heavy load. Using the CDN layer means I never have to write the image to disk, everything is done in memory.

    I'm doing everything in c# and .NET. I assume you are more interested in PHP methodologies?
    Last edited by isellstuff; February 18th, 2010 at 09:52 AM. Reason: err. hotlinking...
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  3. #3
    Moderator MichaelColey's Avatar
    Join Date
    January 18th, 2005
    Location
    Mansfield, TX
    Posts
    16,232
    Wow! Nice technology. I may have to look into CDN myself. Seems affordably priced, as well ($0.20 per GB or less).
    Michael Coley
    Amazing-Bargains.com
     Affiliate Tips | Merchant Best Practices | Affiliate Friendly? | Couponing | CPA Networks? | ABW Tips | Activating Affiliates
    "Education is the most powerful weapon which you can use to change the world." Nelson Mandela

  4. #4
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by MichaelColey
    Wow! Nice technology. I may have to look into CDN myself. Seems affordably priced, as well ($0.20 per GB or less).
    Yes, the CDN is very affordable compared to all of the competitors. One thing to add, if anyone gets serious about Softlayer CDN. Softlayer's DNS servers were under DDOS earlier in the year and Softlayer has their own alias for cdce.internapcdn.net. The DDOS brought down all of my images because my image urls used the Softlayer alias, so now I use "cdce.internapcdn.net" to avoid the Softlayer name servers.

    Maybe Softlayer fixed their names server DDOS vulnerability, but for now I don't rely on them for DNS. I use DNSMadeEasy for my own domains and the internapcdn.net workaround for CDN.
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  5. #5
    Full Member
    Join Date
    January 18th, 2005
    Posts
    396
    Wow - a bigger step for me than I imagined - thank you for giving me some directions and some pitfalls to watch for. Charles

  6. #6
    ABW Ambassador isellstuff's Avatar
    Join Date
    November 9th, 2005
    Location
    Virginia
    Posts
    1,659
    Quote Originally Posted by micheck
    Wow - a bigger step for me than I imagined - thank you for giving me some directions and some pitfalls to watch for. Charles
    One other suggestion. I don't currently specify a maximum height allowed. This has resulted in some really tall images, even in my thumbnails. I noticed I had some tennis racket images that were pushing html down on the page, for instance because they were tall and skinny to begin with. I need to enhance my image resize code with a maximum height and if, resizing while respecting the aspect ratio is going to result in an image that is taller than the maximum height, I need to shrink the image down further.

    This is actually on my todo list, along with providing mouse-over image enlargements.
    Merchants, any data you provide to Google Shopping should also be in your affiliate network datafeed. More data means more sales!

  7. #7
    Moderator MichaelColey's Avatar
    Join Date
    January 18th, 2005
    Location
    Mansfield, TX
    Posts
    16,232
    What I do in a similar application is to specify both a maximum width and height, and I size down to fit within that rectangle. For instance, if I specify that I want a 100x100 image, the following transformations would happen:

    200x200 -> 100x100
    100x200 -> 50x100
    200x100 -> 100x50
    Michael Coley
    Amazing-Bargains.com
     Affiliate Tips | Merchant Best Practices | Affiliate Friendly? | Couponing | CPA Networks? | ABW Tips | Activating Affiliates
    "Education is the most powerful weapon which you can use to change the world." Nelson Mandela

  8. #8
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    My solution is similar to isellstuff's, but I just use my server to store the images and data. It's a PHP script that downloads the image and resizes it based on the width and height given. For example /image.php?w=50&h=80&url=http://www.url.to/image.jpg. Originally I generated a random filename for the image, but this caused issues with duplicate filenames. I got around this by stripping all non-alphanumeric characters from the image URL to be used as the filename.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  9. #9
    ABW Ambassador Doug247's Avatar
    Join Date
    January 18th, 2005
    Location
    DE USA
    Posts
    931
    Yeah caching images is great. I do it on a few of my sites.
    Thanks,
    Doug

  10. #10
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Quote Originally Posted by itsupportnotes
    Yeah caching images is great. I do it on a few of my sites.
    Agreed, it's really the only way to get consistent image sizes without using the browser to resize.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  11. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Geo Caching
    By Doug247 in forum Midnight Cafe'
    Replies: 3
    Last Post: September 5th, 2007, 12:10 AM
  2. Caching Amazon
    By Snib in forum Amazon
    Replies: 5
    Last Post: August 10th, 2004, 03:36 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •