we imagine, when the spider is happy to visit our website, we crawl through the source website each link, the thought to be able to find some good content fresh, but a lot of dead link waiting for it, so once, two times, three times, each time to have such a situation, do you think the spider on your site has a good evaluation, will once again grab your content? I like website www.qqya.cc was because of a large number of 404 pages is a spider, causing the web content is not indexed by search engines, so small in later observation site the log found spider crawling pages contain a status code of 404 page immediately to see light suddenly, so these 404 pages are clear, shielding, and in At that time a lot of the chain, then web spider crawling is normal.
believes that many webmaster with small, every day to view the site log and then through the analysis of habits, log on our website to understand the spider crawling situation, of course in the process of small and medium-sized series often simply look at the number of spiders crawling my site every day, from this point can see the visual health website but in the situation, for many webmaster, their website feel very perfect, but it is not a spider crawling, this is undoubtedly a great blow to the webmaster, so today Xiaobian specially arranged a few reasons why some spiders don’t visit our website, here to share with everyone.
spiders can often visit our website, I believe that a large part of the chain are attractive, so for the friendliness of the chain, we also need regular observation, so as not to do in vain. I believe that many owners at this time should be asked, why, how.
(two) site there are a lot of dead link.
have to say, each big search engine development up to now, has been regarded as quite intelligent, especially in recent years the search engine update algorithm, for example, Google launched in 2011 when the image search function, we can find the source web page address from the pictures, but after all, is the search engine and he still has not a small gap. However, at this time many owners, especially the enterprise station, in order to highlight the products on the website is inlaid with a large number of flash and pictures, but we all know that the search engine spiders can grab flash, and picture files, so even if our content in good, are also not let spiders crawl crawl. So for the existence of a large number of pictures and flash sites in those websites, small series suggest that some of the spider crawling simulation tools site check for yourself, see whether it is because of this situation caused by spiders don’t visit our website.
(a) flash website, too many pictures.
(three) site outside the chain with nofollow tags.