Results 1 to 20 of 20
  1. #1
    Newbie
    Join Date
    January 17th, 2005
    Posts
    36
    I've been trying for the past couple days to get a script working that will download a tigerdirect datafeed (~7 mb). The code below works perfectly for smaller files that I've tried (6 kb) but I get an error when I try to download the larger files through the script. My eventual goal is to automate the downloading, and the loading of the datafeed into my database a couple times a week so I don't have to be bothered with it.

    Response.Buffer = True
    Server.ScriptTimeout = 3600
    Set xml = Server.CreateObject("MSXML2.XMLHTTP")
    xml.Open "GET", "ftp://usernameassword@ip.address/feed.txt", false
    xml.Send()
    If err.Number <> 0 then
    xml.abort()
    Set xml = Nothing
    Else
    fl1 = xml.responseText
    Set xml = Nothing
    fl1 = Replace(fl1,"ID_HERE","XXXX")
    Set fs = Server.CreateObject("Scripting.FileSystemObject")
    Set fl = fs.CreateTextFile("C:\Path\feed.txt",True,False)
    fl.WriteLine fl1
    fl.Close
    Set fl = Nothing
    Set fs = Nothing
    End If

    The error I get when this is executed is (BTW, it's actually loading the 7 MB file before the error):

    Microsoft VBScript runtime error '800a0005'
    Invalid procedure call or argument
    /tdftp.asp, line 18

    Line 18 is: fl.WriteLine fl1

    If I change the last parameter of line 17 to True (to save the file as unicode instead of ASCII) it saves it with no error, but the output in the file is unreadable, ex: ÿþ atrldpo|cttpalraanec aam sezc kaa 4nihr Pmf

    If anybody has a solution to fix this script so it will work with the larger files I would appreciate it, also any better solutions for downloading the datafeeds from a FTP site, I would appreciate hearing about them as well.

    [This message was edited by Haiko on July 30, 2003 at 01:28 PM.]

  2. #2
    ABW Ambassador FFoc's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,015
    if you are just using it to download, use wget or ncftpget on the command line...

    syntax:
    <pre class="ip-ubbcode-code-pre">wget -q http://user:pass@site.com/path/to/datafeed.txt</pre>
    (-q means quiet)
    for ncftpget, the syntax is the same.

    --
    "The greatest good you can do for another is not just to share your riches, but to reveal to him his own." – Benjamin Disraeli
    --
    Ford Fox-body Owners Club -- http://www.ford-fox.org

  3. #3
    Newbie
    Join Date
    January 17th, 2005
    Posts
    36
    Thanks FFoc, If I go an alternate route I may use your suggestion. But for now the point of what I'm trying to do is to have the retrieval and processing be completely unassisted through the server-side scripting.

  4. #4
    ABW Ambassador cditty's Avatar
    Join Date
    January 17th, 2005
    Location
    Memphis TN
    Posts
    1,434
    There may be some sort of file size limitation in play. PHP has a 2 meg buffer pre-configured in the php.ini file. Perhaps there is something similar in asp.

    Just a thought.

    Chris

    ----------------------------
    Scriptsforyoursite.com - Featuring datafeed import scripts for Backcountry Store, Coldwater Creek, Mondera.com and many more.....

  5. #5
    Newbie
    Join Date
    January 17th, 2005
    Posts
    36
    I agree cditty, I do think it could be a buffer size limitation, and I've searched for hours on the topic w/ no solid leads for ASP. I think I'll try to do the same things in ASP.NET, maybe it's more tolerant.

  6. #6
    Affiliate Manager bevmoam's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    273
    Hi Mikey,

    We had the same problem with our sample script for affiliates. The problem are special characters in the Response Text.

    We changed the Script to use ADODB.Stream instead of WriteLine and it worked.

    Here is the Code:


    Dim strFullURL
    Dim objSrvHTTP
    Dim strText
    Dim strResultFilename
    Dim objFso

    'Declare constants
    strFullURL = "ftp://usernameassword@ip.address/feed.txt"
    strResultFilename = "C:\Path\feed.txt"

    'Use Microsoft XMLHTTP to get the Data
    Set objSrvHTTP = CreateObject("MSXML2.XMLHTTP")
    objSrvHTTP.Open "GET", strFullURL, False
    objSrvHTTP.Send ""
    strText = objSrvHTTP.ResponseText
    Set objSrvHTTP = Nothing

    'Delete (old) target file on local hard drive
    Set objFso = CreateObject("Scripting.FilesystemObject")
    If objFso.FileExists(strResultFilename) Then
    objFso.DeleteFile strResultFilename
    End If

    'Create the stream
    Set objStream = CreateObject("ADODB.Stream")

    'Initialize the stream
    objStream.Open

    'Reset the position and indicate the character encoding
    objStream.Position = 0
    objStream.Charset = "ascii"

    'Write data to file
    objStream.WriteText strText

    'Save the stream to a file
    objStream.SaveToFile strResultFilename

    Set objFso = Nothing

    Hope that will help.

    CC

    ___________________________
    Bevmo.com Affiliate Manager
    5%-10% Commission, 60 day cookie, 2 AM's
    http://www.bevmo.com
    https://www.bevmo.com/affiliates/
    http://www.bevmoaffiliates.com

  7. #7
    ABW Ambassador FFoc's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,015
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by mikey9:
    ...to have the retrieval and processing be completely unassisted through the server-side scripting.<HR></BLOCKQUOTE>
    Yep, we have a thread going on the subject here: http://abw.infopop.cc/6/ubb.x?a=tpc&...9&m=9426003283

    --
    "The greatest good you can do for another is not just to share your riches, but to reveal to him his own." – Benjamin Disraeli
    --
    Ford Fox-body Owners Club -- http://www.ford-fox.org

  8. #8
    Newbie
    Join Date
    January 17th, 2005
    Posts
    36
    <BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by AM - Bevmo.com:
    We changed the Script to use ADODB.Stream instead of WriteLine and it worked.
    <HR></BLOCKQUOTE>

    CC,
    You're awesome! I can't tell you how much I appreciate your insight into this problem, especially since you were dead on, and with your suggestion my script was working in less than a minute. Thanks again.
    Mike

  9. #9
    Affiliate Manager bevmoam's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    273
    Mike,

    You are very welcome.

    You are also welcome to join our affiliate program... now that you got the TigerDirect Datafeed to work. Just leave me a PM or e-mail me.

    We run our Affiliate Program through CJ, but provide Product Datafeeds and more via our In-house Affiliate Intranet.

    We have a Forum here at ABW as well:
    http://abw.infopop.cc/6/ubb.x?a=frm&...9&f=6046025762

    CC

    ___________________________
    Bevmo.com Affiliate Manager
    5%-10% Commission, 60 day cookie, 2 AM's
    http://www.bevmo.com
    https://www.bevmo.com/affiliates/
    http://www.bevmoaffiliates.com

  10. #10
    ABW Ambassador FFoc's Avatar
    Join Date
    January 18th, 2005
    Posts
    1,015
    BTW, the Bevmo "backroom" and feed system are VERY well done - I only wish that all the other merchants whose feeds I used could be even HALF as advanced.. (only ones that even come close are Mondera, followed by Irvs/EE)

    --
    "The greatest good you can do for another is not just to share your riches, but to reveal to him his own." – Benjamin Disraeli
    --
    Ford Fox-body Owners Club -- http://www.ford-fox.org

  11. #11
    Newbie
    Join Date
    January 17th, 2005
    Posts
    36
    CC it's a very tempting invitation more than ever. Your generosity in helping me get the script working was wonderful, however the very program I set it up for stabbed their datafeed affiliates in the back by revoking all use of their datafeed this morning. If I wasn't LDS (Mormon) I'd promote your stuff in a heartbeat because it looks like you have a very organized, and well treated affiliate community.

  12. #12
    Affiliate Manager bevmoam's Avatar
    Join Date
    January 18th, 2005
    Location
    Fresno/CA
    Posts
    273
    I understand, no problem.
    Btw. BevMo sells Glassware as well .

    CC

    ___________________________
    Bevmo.com Affiliate Manager
    5%-10% Commission, 60 day cookie, 2 AM's
    http://www.bevmo.com
    https://www.bevmo.com/affiliates/
    http://www.bevmoaffiliates.com

  13. #13
    ABW Ambassador
    Join Date
    January 17th, 2005
    Posts
    594
    Do not forget object cleanup....

    add:

    Set objStream = Nothing

    right before:

    "Set objFso = Nothing"

    IamJaloppy

  14. #14
    Full Member
    Join Date
    January 17th, 2005
    Posts
    273
    Does the ftp server I'm grabbing the info from register the IP of the server or the IP address of the client? My guess is that it's the IP of the server which would be great for Linkshare's ridiculous requirement of having a static IP.

  15. #15
    ABW Ambassador
    Join Date
    January 17th, 2005
    Location
    sTrl
    Posts
    1,372
    Yes, I use the IP of my server for my linkshare merchandiser account.
    Use the server to grab the file then. much quicker & easier once you figure out how to do it.

    I use either wget or NET::FTP in a perl script to grab & process mine.

  16. #16
    Full Member
    Join Date
    January 17th, 2005
    Posts
    273
    Can you post how to use Wget or a link to good instructions?

  17. #17
    Full Member
    Join Date
    January 17th, 2005
    Posts
    273
    Actually on second thought, the above script gives me more flexibility simply because I know Vbscript.

    So those who have used the vbscript above, is the IP address of the server indeeded the one that the FTP site recognizes? Anyone successfully used it with Linkshare?

  18. #18
    Full Member
    Join Date
    January 17th, 2005
    Posts
    273
    um, ok, So I do this and get a file not found error:

    ftp://meritline:affiliate@66.206.24....tafeed1012.txt

    Any ideas why this doesn't work. Copy and paste this into your browser and it will also fail. If I leave off the text file at the end it logs in fine and I can manually download the file. Any ideas?

  19. #19
    ABW Ambassador sjangro's Avatar
    Join Date
    January 17th, 2005
    Location
    Boston
    Posts
    1,529
    Looks like you got the filename wrong, Steele, put this in the URL:

    20031102-M-Datafeed.txt

    --
    scott@befree.com

  20. #20
    Full Member
    Join Date
    January 17th, 2005
    Posts
    273
    They changed the filename over night. However the new filename does not work either:

    ftp://meritline:affiliate@66.206.24....M-Datafeed.txt

+ Reply to Thread

Similar Threads

  1. asp.net script to download data feed
    By magantivenkat in forum Programming / Datafeeds / Tools
    Replies: 0
    Last Post: September 30th, 2008, 09:50 AM
  2. Datafeed download/process/update-to-dB scripts - suggestions, endorsements?
    By markwelch in forum Marketing Resources & Power Tools
    Replies: 1
    Last Post: January 28th, 2006, 05:16 PM
  3. does anyone have CJ, Performics & BeFree automated datafeed download scripts?
    By walkman in forum Programming / Datafeeds / Tools
    Replies: 0
    Last Post: April 4th, 2005, 07:59 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •