Page 1 of 2 12 LastLast
Results 1 to 25 of 29
  1. #1
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Linkshare Datafeed Download Script
    I was helping out a fellow affiliate with a Linkshare datafeed download script, and I thought I'd share this with everybody. This can be run from your server and will download the feeds you specify. It allows you to maintain your static server IP with Linkshare so you don't need to keep changing it to accommodate your local dynamic IP.

    Code:
    <?php
    
    // define some variables
    $ftp_server = 'aftp.linksynergy.com';
    $ftp_user_name = 'yourlogin';
    $ftp_user_pass = 'yourpass';
    
    $local_path = ".";
    
    // Change this array to include the datafeeds you have access to.
    
    $server_files = array(
    	1 => "/1243_1398932_mp.txt.gz",
    	2 => "/1234_1398932_mp.txt.gz",
    	3 => "/123_1398932_mp.txt.gz"
    );	
    
    // set up basic connection
    $conn_id = ftp_connect($ftp_server);
    
    // login with username and password
    $login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);
    
    foreach($server_files as $server_file)
    {
    	$local_file = $local_path.$server_file;
    
    	// try to download $server_file and save to $local_file
    	if (ftp_get($conn_id, $local_file, $server_file, FTP_BINARY)) {
       		echo "Successfully written to $local_file\n";
    	} else {
       		echo "There was a problem\n";
    	}
    }	
    
    // close the connection
    ftp_close($conn_id);
    
    ?>
    You just need to insert your own login and password as well as a list of the datafeeds you want it to download. I left a couple test entries in there so you can see how to format the $server_files array.

    This is strongly based on the ftp_get page on PHP.net:

    http://us2.php.net/manual/en/function.ftp-get.php

    Just make sure Linkshare has your server's IP address configured with your merchandiser account.

    Enjoy!
    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  2. #2
    What's the word? Rhia7's Avatar
    Join Date
    January 13th, 2006
    Posts
    9,578
    Smile
    Scott,

    It's really nice of you to share this Linkshare datafeed download script
    ~Rhia7 -- Remember the 7
    Twitter me

  3. #3
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    It's best that you run this from your shell account. If you want to run it via the web you need to make sure the directory the datafeeds will be written to is is writable with 777 permissions.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  4. #4
    What's the word? Rhia7's Avatar
    Join Date
    January 13th, 2006
    Posts
    9,578
    Quote Originally Posted by Snib
    It's best that you run this from your shell account. If you want to run it via the web you need to make sure the directory the datafeeds will be written to is is writable with 777 permissions.

    - Scott
    Does the Shell account refer to the Secure Shell (SSH) Access and
    Proxy SSL Encryption? Two things my hosting account comes with but I have not used yet.

    I don't use datafeeds yet, but your instructions will come in handy when I do.

    Thanks, Scott
    ~Rhia7 -- Remember the 7
    Twitter me

  5. #5
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Quote Originally Posted by Rhia7
    Does the Shell account refer to the Secure Shell (SSH) Access and
    Proxy SSL Encryption? Two things my hosting account comes with but I have not used yet.
    You just need SSH. It lets you log into your server and run command line utilities. It's similar to DOS except much nicer and more powerful.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  6. #6
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    It's too late for me to go back and edit the snippet, but this is also acceptable for the datafeed array:
    Code:
    $server_files = array(
    	"/1243_1398932_mp.txt.gz",
    	"/1234_1398932_mp.txt.gz",
    	"/123_1398932_mp.txt.gz"
    );
    The numbers proceeding each datafeed file name aren't necessary.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  7. #7
    Member SeanW's Avatar
    Join Date
    July 28th, 2005
    Posts
    69
    What a coincidence... I've been writing a batch system to help with the downloading and processing of feeds. Doing perl instead of php, YAML for config files.

    My big problem was wanting to make sure that step N+1 only executed if step N failed, so my code checks the errorlevel based on what the config tells it:
    Code:
    --- !vendor.com
    environment:
        PATH: +:/home/httpd/vhosts/hostname
    vars:
        merchantid: 5
    process:
      - run: wget ftp://userid:password@datafeeds.shar...xxx/xxx.txt.gz -O %OUTFILE%.gz -q
        0: success
        1: fail
      - run: gunzip %OUTFILE%.gz
        0: success
        1: fail
      - run: shareasale.pl --merchant=%merchantid% --feed=%OUTFILE%
        0: success
    The code:

    Code:
    #!/usr/bin/perl
    
    use YAML;
    use Data::Dumper;
    
    my $configfile;
    
    my @jobs = YAML::LoadFile($configfile||"jobs.yaml");
    
    foreach my $job (@jobs) {
            print "----------------------\n";
            # Load in vars
            my $oldenv = {};
    
            while (my ($k, $v) = each (%{$job->{'environment'}})) {
                    $oldenv->{$k} = $ENV{$k};
                    # If it starts with a +, add it on
                    if ($v =~ /^\+(.*)/) {
                            $ENV{$k} .= $1;
                    } else {
                            $ENV{$k} = $v;
                    }
            }
    
    
            # Execute commands
    
            foreach $run (@{$job->{'process'}}) {
                    my $thecommand = $run->{'run'};
                    $thecommand =~ s/%OUTFILE%/\/tmp\/job.out/g;
                    while (my ($k, $v) = each (%{$job->{'vars'}})) {
                            $thecommand =~ s/%$k%/$v/g;
                    }
                    print $thecommand . "\n";
                    system($thecommand);
                    if (defined $run->{$?}) {
                            print "--" . $run->{$?} . "--\n";
                            last if ($run->{$?} eq "fail");
                    } else {
                            print "Unknown status $?\n";
                    }
            }
    
            # Cleanup
            while (my ($k, $v) = each (%$oldenv)) {
                    $ENV{$k} = $v;
            }
            unlink "/tmp/job.out";
    }
    The system lends itself to other keywords (ie skip, goto, alert), I just haven't yet found a need to implement them.

    Sean

  8. #8
    Moderator MichaelColey's Avatar
    Join Date
    January 18th, 2005
    Location
    Mansfield, TX
    Posts
    16,232
    Here's the one I wrote:

    http://forum.abestweb.com/showthread...6507#post56507

    It basically gets mirrors everything out there, only downloading the ones that changed.
    Michael Coley
    Amazing-Bargains.com
     Affiliate Tips | Merchant Best Practices | Affiliate Friendly? | Couponing | CPA Networks? | ABW Tips | Activating Affiliates
    "Education is the most powerful weapon which you can use to change the world." Nelson Mandela

  9. #9
    Newbie
    Join Date
    March 25th, 2006
    Posts
    8
    Snib,
    Thank you for the information. I cant yet send PMs but sent you an email for little bit more information for my specific web site.

  10. #10
    Member
    Join Date
    January 18th, 2005
    Posts
    64
    Thanks for the nice script. WIll definitely make my life easier.

  11. #11
    Newbie
    Join Date
    January 18th, 2005
    Posts
    14
    Thanks Scott, I have battled with the problem on and off for a while and this was by far the simplest. Changed to user, password and files names, saved as a file and put into a cron job and worked the very first time. Getting something to work first time is a rarity for me.

  12. #12
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Quote Originally Posted by Jarvi
    Thanks Scott, I have battled with the problem on and off for a while and this was by far the simplest. Changed to user, password and files names, saved as a file and put into a cron job and worked the very first time. Getting something to work first time is a rarity for me.
    Great to hear! I'm glad you were able to get it going on the first try!

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  13. #13
    Newbie
    Join Date
    January 18th, 2005
    Posts
    46
    Snib: For some reason the datafeed download script that you had published here has started giving errors. It now doesn't logs you in. It was working fine. The error messages:

    Warning: ftp_login() expects parameter 1 to be resource, boolean given in /home/xxxx/public_html/feed.php on line 22

    Warning: ftp_get() expects parameter 1 to be resource, boolean given in /home/xxxx/public_html/feed.php on line 29
    There was a problem

    Warning: ftp_close() expects parameter 1 to be resource, boolean given in /home/xxxx/public_html/feed.php on line 37

  14. #14
    ABW Ambassador Snib's Avatar
    Join Date
    January 18th, 2005
    Location
    Virginia
    Posts
    5,303
    Quote Originally Posted by ssgupta
    Snib: For some reason the datafeed download script that you had published here has started giving errors. It now doesn't logs you in. It was working fine.
    Sometimes Linkshare has connection problems with their FTP. If it worked previously that's most likely the problem. See if it works after a day or two.

    - Scott
    Hatred stirs up strife, But love covers all transgressions.

  15. #15
    Newbie
    Join Date
    January 18th, 2005
    Posts
    46
    Has not worked for the last 10 days or so.

  16. #16
    Animal Lover
    Join Date
    January 18th, 2005
    Location
    oz
    Posts
    1,210
    Has it worked before for that particular IP that you are using the script from? LS has strict rules about downloading and they require that every IP address you ftp from be on their list of 'approved' IP addresses.

    Just a thought.
    Oscar
    My DataFeed Scripts - php datafeed scripts for your site
    Shareasale datafeed scripts - to display Shareasale datafeeds
    Linkshare datafeed scripts - for multiple Linkshare merchants

  17. #17
    Newbie
    Join Date
    January 18th, 2005
    Posts
    46
    Yes it was working for a fairly long time. My IPs are with LS and they haven't changed. I can login manually and download using CuteFTP. Is the script working for you?

  18. #18
    Crazy Cat Lady Heidi's Avatar
    Join Date
    January 18th, 2005
    Location
    Rochester, NY
    Posts
    1,685
    its still working for me but occassionally i do have issues with it and if i wait a few hours and try again, all is well.
    Heidi
    "Happy are those who dream dreams and are willing to pay the price to make them come true"

  19. #19
    Crazy Cat Lady Heidi's Avatar
    Join Date
    January 18th, 2005
    Location
    Rochester, NY
    Posts
    1,685
    ah just thought i would also add - i have to login via ssh and run it - it won't work from the browser or via a cron job for me.
    Heidi
    "Happy are those who dream dreams and are willing to pay the price to make them come true"

  20. #20
    Newbie
    Join Date
    January 18th, 2005
    Posts
    46
    It worked from browser, cron as well ssh. Now works from nowhere.

  21. #21
    Animal Lover
    Join Date
    January 18th, 2005
    Location
    oz
    Posts
    1,210
    Has your host changed the permissions for the server lately? Sometimes in an upgrade they will change that and disable outgoing ftp connects via php. Check with your host if that's the case.

    You can check it by copying this bit of code and saving it as phpdetails.php (or whatever) then running it and seeing what it says under ftp support (ie whether it's enabled or not).

    PHP Code:
    <?php  phpinfo();  ?>
    Oscar
    Last edited by oscar; July 31st, 2006 at 05:03 PM. Reason: added code
    My DataFeed Scripts - php datafeed scripts for your site
    Shareasale datafeed scripts - to display Shareasale datafeeds
    Linkshare datafeed scripts - for multiple Linkshare merchants

  22. #22
    Member
    Join Date
    June 10th, 2006
    Posts
    112
    Thanks for sharing. However my server is a sharing host. So my server IP is not static right? Thanks.
    [LEFT]-[URL=http://www.ableaffiliate.com]www.AbleAffiliate.com[/URL]-
    :approval:
    Free Affiliate Program Directory for Submission[/LEFT]

  23. #23
    Newbie
    Join Date
    January 18th, 2005
    Posts
    46
    FTP support is enabled on the server. But as I stated earlier, it is now not working from my browser either.

  24. #24
    Full Member
    Join Date
    May 19th, 2005
    Posts
    200
    how can I unzip the file? I try:
    <?php
    $zip = zip_open("./datafeed/111_mp.txt.gz");

    if ($zip) {

    while ($zip_entry = zip_read($zip)) {
    echo "Name: " . zip_entry_name($zip_entry) . "\n";
    echo "Actual Filesize: " . zip_entry_filesize($zip_entry) . "\n";
    echo "Compressed Size: " . zip_entry_compressedsize($zip_entry) . "\n";
    echo "Compression Method: " . zip_entry_compressionmethod($zip_entry) . "\n";

    if (zip_entry_open($zip, $zip_entry, "r")) {
    echo "File Contents:\n";
    $buf = zip_entry_read($zip_entry, zip_entry_filesize($zip_entry));
    echo "$buf\n";

    zip_entry_close($zip_entry);
    }
    echo "\n";

    }

    zip_close($zip);

    }
    ?>

    It shows error
    Warning: zip_open() Cannot open zip archive ./datafeed/111_mp.txt.gz
    in /unzip.php on line 2
    I need traffic. I need traffic.I need traffic. I need traffic.I need traffic. I need traffic.I need traffic. I need traffic.I need traffic. I need traffic. That's what I need. I need it.

  25. #25
    Full Member
    Join Date
    May 19th, 2005
    Posts
    200
    got it, use gzopen(),
    I need traffic. I need traffic.I need traffic. I need traffic.I need traffic. I need traffic.I need traffic. I need traffic.I need traffic. I need traffic. That's what I need. I need it.

+ Reply to Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Does Linkshare now limit datafeed download speed?
    By teezone in forum Rakuten LinkShare - LS
    Replies: 13
    Last Post: December 12th, 2012, 08:16 AM
  2. Linkshare Download Script
    By chrisk in forum Programming / Datafeeds / Tools
    Replies: 41
    Last Post: March 18th, 2007, 10:24 AM
  3. Problems with php datafeed download script
    By steven420 in forum Programming / Datafeeds / Tools
    Replies: 11
    Last Post: November 29th, 2006, 08:17 AM
  4. Datafeed Download w/ script???
    By cditty in forum Commission Junction - CJ
    Replies: 12
    Last Post: October 27th, 2003, 05:58 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •