Results 1 to 5 of 5
  1. #1
    Full Member
    Join Date
    January 18th, 2005
    Posts
    390
    Could somebody explain to me what the "MSIECrawler" spider is? I see appear a lot in my site reports, but I have no idea what it is or who it is reporting to.

    Thanks!

  2. #2
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    Yeah, that's Microsoft's new crawler. They are developing their own search engine - no idea when it will be ready, but it should add another interesting twist to the SEO game.

    Search Engine Positioning - 1 Design 4 Life

  3. #3
    ABW Ambassador
    Join Date
    January 18th, 2005
    Location
    United Kingdom
    Posts
    1,797
    Oh...I'm talking rubbish. MSIECrawler isn't the new crawler - it's a Microsoft spider used to spider and download websites and has been around ages. Basically, someone is crawling (and downloading) your site.

    It is supposed to obey robots.txt though.

    Just came off the phone with my brother (who works for Microsoft) and have the new Microsoft search engine on the brain at the moment. Sorry.

    Search Engine Positioning - 1 Design 4 Life

  4. #4
    Full Member
    Join Date
    January 18th, 2005
    Posts
    390
    Thanks for the quick response.

    Any idea what MSFT does with the results from the spider? Are they filtered into the MSN search engine?

  5. #5
    ABW Ambassador Steveinid's Avatar
    Join Date
    January 18th, 2005
    Posts
    827
    Here's a thought and a question at the same time.

    MSFrontpage can 'crawl' a site and download it to your computer. It'll do it for the whole domain and outgoing links up to 5 levels (I believe) Depending on the settings you specifiy. I've used it before to recapture sites that I've totally messed up before. You can also use it to completely copy another persons site.

    Would this show up in your logs as a crawl or would it be considered normal traffic?

    "Bueller...? Bueller...?"

  6. Newsletter Signup

+ Reply to Thread

Similar Threads

  1. good spider or bad spider?
    By Gordon in forum Newbie Affiliate FAQs & Helpful Articles
    Replies: 3
    Last Post: September 3rd, 2004, 09:38 AM
  2. New spider?
    By womanht in forum Midnight Cafe'
    Replies: 1
    Last Post: August 8th, 2004, 06:00 AM
  3. Who is this spider IP?
    By eggerda in forum Search Engine Optimization
    Replies: 1
    Last Post: June 8th, 2004, 10:05 AM
  4. What spider is this? Is it a SE?
    By loxly in forum Search Engine Optimization
    Replies: 4
    Last Post: March 9th, 2004, 11:43 PM
  5. CJ Spider?
    By appbizz in forum Midnight Cafe'
    Replies: 1
    Last Post: April 7th, 2003, 10:12 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •