Results 1 to 12 of 12
  1. #1
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    Los Angeles
    Posts
    4,053
    How big is a datafeed?
    I know, I know - how long is a piece of string?

    But I'm serious, I've just taken new web space and thinking of getting a dedicated IP for that site, and having feeds FTP'd to it. The size of the web space is 100Meg, with the site's files taking up about 20-25 max.

    Is 75-80 enough for feeds?

  2. #2
    What's the word? Rhia7's Avatar
    Join Date
    January 13th, 2006
    Posts
    9,578
    Wow, you need dedicated hosting for datafeeds in order for them to work?
    ~Rhia7 -- Remember the 7
    Twitter me

  3. #3
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    Los Angeles
    Posts
    4,053
    Rhia, I'm not talking about a dedicated box, just a unique IP number for the site, which is on shared hosting.

  4. #4
    The "other" left wing davidh's Avatar
    Join Date
    January 18th, 2005
    Location
    Boston
    Posts
    3,492
    Just multiply the number of items in a feed by the number of bytes + 1k in the template you are going to use.

    Chances are you'll have enough space to slap a handful of fairly large feeds.
    CUSTOM BANNERS by GRAPHICS CANDY ~ Banner Sets and Website Graphics ~ Professional design, reasonable rates
    DESIGNER DOG CHECKS ~ We double-dog dare ya to write one!

  5. #5
    notary sojac Herb ԿԬ's Avatar
    Join Date
    January 18th, 2005
    Location
    Central/Western NY State
    Posts
    7,741
    Lightbulb
    Quote Originally Posted by webworker
    I know, I know - how long is a piece of string?

    But I'm serious, I've just taken new web space and thinking of getting a dedicated IP for that site, and having feeds FTP'd to it. The size of the web space is 100Meg, with the site's files taking up about 20-25 max.

    Is 75-80 enough for feeds?
    the servers I use cost me the typical $9.95/mo, allow me some extra domains, don't worry (supposedly) if my space goes above 400 megs (supposedly I can ask for more).

    however, I try to keep the size total down to 300 megs in case some spider wants to get in there and increase my bandwidth too much.

    I'd want to have a gig or so available if I was going to go out of the cheapie mode I'm in now.

  6. #6
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    My CJ datafeeds are over 50 megs zipped up. Unzipped I'm looking at over 200 megs of data and all together I would estimate my feeds add up to over 500 to 700 megs. This is just for one particular site that uses about 40 datafeeds. This is just the raw data, not including templates, PHP code, site images, mySQL databases and site log files. Since I write all of this data to mySQL that's another 700 megs or so just for my database files. It all adds up pretty quick.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  7. #7
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    What Snib said as to overall size. My biggest site is nearly a gig altogether. And that's with the feed itself residing outside that folder and therefore not being counted in the interface's site-size calculation. If it was all in one spot I'm sure it'd pass a gig.

    Also, when considering server requirements, don't forget the RAM and CPU requirements of processing the data. For one site, it may or may not take much more, depending on the size of it. A small site shouldn't need much. But for a megasite, it can easily pass the weenie limits of most shared hosting. For those big sites, you want to be able to chew through the RAM like it's candy if need be, not worry about whether you are slowing down the response rate for some 1-page wonder who'll never get hit the first.

  8. #8
    What's the word? Rhia7's Avatar
    Join Date
    January 13th, 2006
    Posts
    9,578
    Quote Originally Posted by Snib
    200 megs of data ... up to over 500 to 700 megs.
    - Scott
    Those megs are from a combination of multiple merchants, right?

    What if a webmaster only wanted to use one merchant's data feed?

    How many megs is a single data feed from a single merchant?
    ~Rhia7 -- Remember the 7
    Twitter me

  9. #9
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    Los Angeles
    Posts
    4,053
    I'll just be FTP'ing it to the site, downloading to my own machine and can just nuke the one that's online, as far as I'm figuring now. There won't be anything processed on the site so hogging CPU resources isn't an issue.

    It sounds like there might be more than 100 meg needed, especially if I use 2 or 3 feeds, which would probably be the maximum. Basically, it's for a Performics feed or two because I can't deal with how you have to make links there and I need product links for a couple of their merchants.

  10. #10
    Resident Genius and Staunch Capitalist Leader's Avatar
    Join Date
    January 18th, 2005
    Location
    Florida
    Posts
    12,817
    Quote Originally Posted by Rhia7
    Those megs are from a combination of multiple merchants, right?

    What if a webmaster only wanted to use one merchant's data feed?

    How many megs is a single data feed from a single merchant?
    Depends on the merchant! With only 21 or 22 items, GoodBulbs' is just 9.87 Kb! (I think a meg is something like 1020 Kb...)

    On the other hand, a merchant with tens of thousands of items is probably more than 100Mb.
    There is no knowledge that is not power. ~Hemingway

  11. #11
    ABW Ambassador Akiva's Avatar
    Join Date
    January 18th, 2005
    Location
    New Jersey
    Posts
    3,266
    webworker, it really depends on which merchants. Some merchants at Performics have feeds of 200MB+. However most are in the 25-70MB range unzipped.
    Akiva Bergstrom | akiva@affsolutions.com | 718-871-8286

    Affiliate Marketing Solutions by affSolutions - Creator of the Product Showcase Creator

    Managed Programs: EssentialApparel.com (Join) | SportsFanfare.com (Join)


    Affiliates: Product Showcase Creator Directory | Merchants: License the Product Showcase Creator!

  12. #12
    ABW Ambassador Mike O's Avatar
    Join Date
    January 18th, 2005
    Location
    Los Angeles area
    Posts
    843
    My rule of thumb is that a typical DF takes about 1 kilobyte per item.

    So a 1,000-item feed would be about a megabyte, a 10,000-item feed would be about 10 megs, etc.

    That depends strongly on the size of the product descriptions, how many other fields -- price, product category, etc. -- are also populated, and so on.
    A joy shared is a joy doubled.
    A burden shared is a burden lightened.

  13. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. Beware of a BIG F-up on this datafeed...
    By Mr. Sal in forum ShareASale - SAS
    Replies: 8
    Last Post: June 15th, 2012, 01:33 AM
  2. Big Datafeed Updates to Shops
    By JoyUnltd in forum PopShops
    Replies: 2
    Last Post: March 21st, 2011, 09:40 AM
  3. CJ Needs a BIG Improvement in Datafeed Management
    By Uncle Rico in forum Commission Junction - CJ
    Replies: 13
    Last Post: March 30th, 2009, 01:27 PM
  4. Big FJB Joy - Datafeed is up!
    By Geno Prussakov in forum ShareASale - SAS
    Replies: 0
    Last Post: March 1st, 2006, 06:03 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •