Welcome, guest! Login Join
Close

Login to Your Account


  |  » Forgot your password?

Notices

ASP Scripts to Download Datafeed

 
Tools Search
  #1  
Old
Newbie
Join Date: January 18th, 2005
Posts: 36
I've been trying for the past couple days to get a script working that will download a tigerdirect datafeed (~7 mb). The code below works perfectly for smaller files that I've tried (6 kb) but I get an error when I try to download the larger files through the script. My eventual goal is to automate the downloading, and the loading of the datafeed into my database a couple times a week so I don't have to be bothered with it.

Response.Buffer = True
Server.ScriptTimeout = 3600
Set xml = Server.CreateObject("MSXML2.XMLHTTP")
xml.Open "GET", "ftp://usernameassword@ip.address/feed.txt", false
xml.Send()
If err.Number <> 0 then
xml.abort()
Set xml = Nothing
Else
fl1 = xml.responseText
Set xml = Nothing
fl1 = Replace(fl1,"ID_HERE","XXXX")
Set fs = Server.CreateObject("Scripting.FileSystemObject")
Set fl = fs.CreateTextFile("C:\Path\feed.txt",True,False)
fl.WriteLine fl1
fl.Close
Set fl = Nothing
Set fs = Nothing
End If

The error I get when this is executed is (BTW, it's actually loading the 7 MB file before the error):

Microsoft VBScript runtime error '800a0005'
Invalid procedure call or argument
/tdftp.asp, line 18

Line 18 is: fl.WriteLine fl1

If I change the last parameter of line 17 to True (to save the file as unicode instead of ASCII) it saves it with no error, but the output in the file is unreadable, ex: ÿþ atrldpo|cttpalraanec aam sezc kaa 4nihr Pmf

If anybody has a solution to fix this script so it will work with the larger files I would appreciate it, also any better solutions for downloading the datafeeds from a FTP site, I would appreciate hearing about them as well.

[This message was edited by Haiko on July 30, 2003 at 01:28 PM.]
  #2  
Old
ABW Ambassador
Join Date: January 18th, 2005
Posts: 1,015
if you are just using it to download, use wget or ncftpget on the command line...

syntax:
<pre class="ip-ubbcode-code-pre">wget -q http://user:pass@site.com/path/to/datafeed.txt</pre>
(-q means quiet)
for ncftpget, the syntax is the same.

--
"The greatest good you can do for another is not just to share your riches, but to reveal to him his own." – Benjamin Disraeli
--
Ford Fox-body Owners Club -- http://www.ford-fox.org
  #3  
Old
Newbie
Join Date: January 18th, 2005
Posts: 36
Thanks FFoc, If I go an alternate route I may use your suggestion. But for now the point of what I'm trying to do is to have the retrieval and processing be completely unassisted through the server-side scripting.
Join ABW to remove this sponsored message.
  #4  
Old
ABW Ambassador
Join Date: January 18th, 2005
Location: Memphis TN
Posts: 1,434
There may be some sort of file size limitation in play. PHP has a 2 meg buffer pre-configured in the php.ini file. Perhaps there is something similar in asp.

Just a thought.

Chris

----------------------------
Scriptsforyoursite.com - Featuring datafeed import scripts for Backcountry Store, Coldwater Creek, Mondera.com and many more.....
  #5  
Old
Newbie
Join Date: January 18th, 2005
Posts: 36
I agree cditty, I do think it could be a buffer size limitation, and I've searched for hours on the topic w/ no solid leads for ASP. I think I'll try to do the same things in ASP.NET, maybe it's more tolerant.
  #6  
Old
Affiliate Manager
Join Date: January 18th, 2005
Location: Fresno/CA
Posts: 273
Hi Mikey,

We had the same problem with our sample script for affiliates. The problem are special characters in the Response Text.

We changed the Script to use ADODB.Stream instead of WriteLine and it worked.

Here is the Code:


Dim strFullURL
Dim objSrvHTTP
Dim strText
Dim strResultFilename
Dim objFso

'Declare constants
strFullURL = "ftp://usernameassword@ip.address/feed.txt"
strResultFilename = "C:\Path\feed.txt"

'Use Microsoft XMLHTTP to get the Data
Set objSrvHTTP = CreateObject("MSXML2.XMLHTTP")
objSrvHTTP.Open "GET", strFullURL, False
objSrvHTTP.Send ""
strText = objSrvHTTP.ResponseText
Set objSrvHTTP = Nothing

'Delete (old) target file on local hard drive
Set objFso = CreateObject("Scripting.FilesystemObject")
If objFso.FileExists(strResultFilename) Then
objFso.DeleteFile strResultFilename
End If

'Create the stream
Set objStream = CreateObject("ADODB.Stream")

'Initialize the stream
objStream.Open

'Reset the position and indicate the character encoding
objStream.Position = 0
objStream.Charset = "ascii"

'Write data to file
objStream.WriteText strText

'Save the stream to a file
objStream.SaveToFile strResultFilename

Set objFso = Nothing

Hope that will help.

CC

___________________________
Bevmo.com Affiliate Manager
5%-10% Commission, 60 day cookie, 2 AM's
http://www.bevmo.com
https://www.bevmo.com/affiliates/
http://www.bevmoaffiliates.com
Join ABW to remove this sponsored message.
  #7  
Old
ABW Ambassador
Join Date: January 18th, 2005
Posts: 1,015
<BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by mikey9:
...to have the retrieval and processing be completely unassisted through the server-side scripting.<HR></BLOCKQUOTE>
Yep, we have a thread going on the subject here: http://abw.infopop.cc/6/ubb.x?a=tpc&...9&m=9426003283

--
"The greatest good you can do for another is not just to share your riches, but to reveal to him his own." – Benjamin Disraeli
--
Ford Fox-body Owners Club -- http://www.ford-fox.org
  #8  
Old
Newbie
Join Date: January 18th, 2005
Posts: 36
<BLOCKQUOTE class="ip-ubbcode-quote"><font size="-1">quote:</font><HR>Originally posted by AM - Bevmo.com:
We changed the Script to use ADODB.Stream instead of WriteLine and it worked.
<HR></BLOCKQUOTE>

CC,
You're awesome! I can't tell you how much I appreciate your insight into this problem, especially since you were dead on, and with your suggestion my script was working in less than a minute. Thanks again.
Mike
  #9  
Old
Affiliate Manager
Join Date: January 18th, 2005
Location: Fresno/CA
Posts: 273
Mike,

You are very welcome.

You are also welcome to join our affiliate program... now that you got the TigerDirect Datafeed to work. Just leave me a PM or e-mail me.

We run our Affiliate Program through CJ, but provide Product Datafeeds and more via our In-house Affiliate Intranet.

We have a Forum here at ABW as well:
http://abw.infopop.cc/6/ubb.x?a=frm&...9&f=6046025762

CC

___________________________
Bevmo.com Affiliate Manager
5%-10% Commission, 60 day cookie, 2 AM's
http://www.bevmo.com
https://www.bevmo.com/affiliates/
http://www.bevmoaffiliates.com
Join ABW to remove this sponsored message.
  #10  
Old
ABW Ambassador
Join Date: January 18th, 2005
Posts: 1,015
BTW, the Bevmo "backroom" and feed system are VERY well done - I only wish that all the other merchants whose feeds I used could be even HALF as advanced.. (only ones that even come close are Mondera, followed by Irvs/EE)

--
"The greatest good you can do for another is not just to share your riches, but to reveal to him his own." – Benjamin Disraeli
--
Ford Fox-body Owners Club -- http://www.ford-fox.org
  #11  
Old
Newbie
Join Date: January 18th, 2005
Posts: 36
CC it's a very tempting invitation more than ever. Your generosity in helping me get the script working was wonderful, however the very program I set it up for stabbed their datafeed affiliates in the back by revoking all use of their datafeed this morning. If I wasn't LDS (Mormon) I'd promote your stuff in a heartbeat because it looks like you have a very organized, and well treated affiliate community.
  #12  
Old
Affiliate Manager
Join Date: January 18th, 2005
Location: Fresno/CA
Posts: 273
I understand, no problem.
Btw. BevMo sells Glassware as well .

CC

___________________________
Bevmo.com Affiliate Manager
5%-10% Commission, 60 day cookie, 2 AM's
http://www.bevmo.com
https://www.bevmo.com/affiliates/
http://www.bevmoaffiliates.com
Join ABW to remove this sponsored message.
  #13  
Old
ABW Ambassador
Join Date: January 18th, 2005
Posts: 594
Do not forget object cleanup....

add:

Set objStream = Nothing

right before:

"Set objFso = Nothing"

IamJaloppy
  #14  
Old
Full Member
Join Date: January 18th, 2005
Posts: 273
Does the ftp server I'm grabbing the info from register the IP of the server or the IP address of the client? My guess is that it's the IP of the server which would be great for Linkshare's ridiculous requirement of having a static IP.
  #15  
Old
ABW Ambassador
Join Date: January 18th, 2005
Location: sTrl
Posts: 1,372
Yes, I use the IP of my server for my linkshare merchandiser account.
Use the server to grab the file then. much quicker & easier once you figure out how to do it.

I use either wget or NET::FTP in a perl script to grab & process mine.
Join ABW to remove this sponsored message.
  #16  
Old
Full Member
Join Date: January 18th, 2005
Posts: 273
Can you post how to use Wget or a link to good instructions?
  #17  
Old
Full Member
Join Date: January 18th, 2005
Posts: 273
Actually on second thought, the above script gives me more flexibility simply because I know Vbscript.

So those who have used the vbscript above, is the IP address of the server indeeded the one that the FTP site recognizes? Anyone successfully used it with Linkshare?
  #18  
Old
Full Member
Join Date: January 18th, 2005
Posts: 273
um, ok, So I do this and get a file not found error:

ftp://meritline:affiliate@66.206.24....tafeed1012.txt

Any ideas why this doesn't work. Copy and paste this into your browser and it will also fail. If I leave off the text file at the end it logs in fine and I can manually download the file. Any ideas?
Join ABW to remove this sponsored message.
  #19  
Old
ABW Ambassador
Join Date: January 18th, 2005
Location: Boston
Posts: 1,526
Looks like you got the filename wrong, Steele, put this in the URL:

20031102-M-Datafeed.txt

--
scott@befree.com
  #20  
Old
Full Member
Join Date: January 18th, 2005
Posts: 273
They changed the filename over night. However the new filename does not work either:

ftp://meritline:affiliate@66.206.24....M-Datafeed.txt
Reply
 

Tools Search
Search:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
CJ Product Datafeed File .. continued cumbrowski Commission Junction - CJ 15 January 17th, 2006 05:43 AM
Datafeed Download w/ script??? cditty Commission Junction - CJ 12 October 27th, 2003 05:58 PM
Datafeed download problem? cditty BettyMills 9 October 3rd, 2003 08:22 PM

X

Welcome to ABestWeb.com

Create your username to jump into the discussion!

ABestWeb.com is the largest affiliate marketing community on the Internet. Join us by filling in the form below.


(4 digit year)

Already a member?



Content Relevant URLs by vBSEO ©2011, Crawlability, Inc.