» Yahoo! Sitemap Feed Submission... worth the Effort?

Looking to make strategic use of user search behavior...

...in your keyword selection and search engine optimization? Let Antezeta provide you with Search Engine Optimization and Web Marketing services.

In the past year, Yahoo! has reaffirmed its commitment to organic search by releasing a series of tools for web site administrators and Search Engine Marketing professionals. In this article, we take a look at Yahoo!'s site map submission services, part of the Yahoo! Site Explorer program. We will make the inevitable comparison to Google, the current innovators in this area.

When Google launched their Sitemaps search engine submission service in June 2005, Yahoo! quickly responded by adding a brief note to their site submission form indicating that sites could also provide a text file of site URLs, urllist.txt. The program was quickly dubbed Yahoo URL list.

Search Engine Optimization (SEO) experts who noticed responded with a collective yawn. The service seemed to be more of a marketing reaction to Google's sitemaps program (now called Google Webmaster Tools) rather than true innovation in Yahoo crawling. The file format didn't provide for page meta data, such as modification date, required for optimization of search engine crawling. Nor did Yahoo check for updated versions of this file during regular crawling. For the vast majority of sites, simple website submission doesn't provide any added value. Pages which are not found during regular crawling are unlikely to rank well and are indicative of on and off site web architecture issues which need to be addressed. Finally, Yahoo! did not provide any feedback on the status of submitted URLs.

The true innovation in site map submission is the dialog which ensues between a web site administrator and the Search Engine. A web master has the possibility to tell a search engine how fresh (last modified date) and how important (Google's priority) a page is, allowing the search engine to optimize site crawling. The search engine in turn can provide valuable feedback on the status of the crawling process to the site's webmaster.

In conjunction with the San Jose Search Engine Strategies 2006 conference, Yahoo! developers have released a combination of features which make Yahoo! site map submission worth consideration, especially for sites already generating a site map for Google. Sites integrating Search Engine Optimization into their overall site usability and quality processes will want to keep an eye on both current and future crawling information Yahoo! makes available to sites providing site maps.

The Site Map Management Life Cycle

Yahoo! Site Map Feed Generation

Yahoo! began accepting a number of well defined XML formats late last year (RSS 0.9, 1.0, 2.0 and Atom 0.3). While this is a big step in the right direction, we would prefer that the big four search engines (Google, Yahoo!, Microsoft and Ask) agree on an industry wide sitemap standard. While the XML formats support document Title meta information not currently in Google's sitemaps specification, it is unclear if Yahoo! actually uses this data. Non of the Yahoo! supported XML formats provide for the document priority information in Google's specification.

In comparison to Google, Yahoo! did not provide site administrators with a tool to generate a compatible xml file, nor does Yahoo! even list options on their help pages. While it is true that many blog based sites have integrated RSS feed generation, most web sites are not blogs.

The best current solution is to either modify the Google sitemap generator to produce a Yahoo! compatible XML file in addition to Google's specified format, or to translate Google's sitemap into a RSS compatible format suitable for Yahoo!.

Free Sitemap Software from Antezeta

Antezeta has taken on this challenge. We provide a free Yahoo! sitemap generator which will generate both Google and Yahoo! site maps simultaneously. It is based on a modified version of Google's sitemap_gen.py Python program.

Site Map Feed Submission to Yahoo!

Yahoo! offers three possibilities to submit a site map. The first is through their standard site submission form. The second is through a new Yahoo! Site Explorer webmaster console interface and the third is through the Yahoo! web services API notification scheme.

Yahoo! supports gzipped files for all formats, which is a nice way to save bandwidth. While the notification API is a nice touch, Google provides this functionality as an integrated part of it's site map generator program.

The new Yahoo! Site Explorer interface still has some bugs to work out; some feeds are rejected with a rather ambiguous, and apparently erroneous, system message:

Site Map Download Monitoring

Sites may want to track downloads of their Yahoo! site map as part of normal Search Engine Optimization measurement activity. We noticed that Google will automatically download a sitemaps file as part of normal crawling activity, once Google has been notified to check for a sitemap. Yahoo! requires notification every time a site map changes.

Search Engine Crawler Feedback

On the occasion of the San Jose Search Engine Strategies 2006 conference, Yahoo! announced the availability in Yahoo! Site Explorer of crawling feedback data such as document last crawl date and human language detected. This feedback is independent of sitemap submission – http data such as pages not found is not yet offered. While a nice beginning, many sites already have access to the last crawl date through their web analytics tools. It will be interesting to see how accurate Yahoo!s document language detection is; webmasters should monitor this as part of their SEO activity.

While Yahoo! offers excellent API support for automatic Site Explorer queries, the API has not yet been extended to support the two crawler feedback data points (language and crawl date) currently offered.

So is Yahoo! Site Explorer Sitemap Feed submission worth the effort?

In a word, yes. But only if you are already summiting to Google and you add Yahoo! submission to that process. Until Yahoo! adds additional crawling statistics to their Site Explorer, Yahoo! sitemap submission won't add much value for most sites.

What about Ask and Microsoft MSN?

Neither Ask nor Microsoft's MSN currently provide web master dashboards nor site map submission services.

Resources

Official Documentation

Discussion Groups

Site Maps Generation Software

Was this document useful?

Was this resource helpful? If so, feel free to put a link to this page on your site! Just copy this code:

Bookmark this page with your bookmark service (hover over a logo to see service name):

Looking for consulting?

To better understand the nuances of Search Engine Optimization, let Antezeta help you with your Search Engine Marketing Needs!

Contact us today to find out more about this topic and the rest of the Web Ecosystem!

Sitemap Feed Submission in Yahoo!... worth the Effort?