In part one of this article, we set out to document the little known Ask web search API by providing background information. In this continuation, we’ll look at the actual API details.
Update: Ask disabled access to their API on 6 March 2007. We are working on obtaining additional information. Write us if you would like to be notified of further developments.
The following information was determined by observation and conjecture. Write us if you want to be notified when we update this page with more complete information. We are assuming the reader has already worked with REST queries and is familiar with parsing XML data.
Request URL
The request URL is formed by adding query parameter and their values to a base URL using the format query parameter=values. Successive parameters are added using a & before each parameter.
Base URL: http://xml.teoma.com/e?
Request URL parameters should be URL encoded.
Before considering the data elements in detail, try a query using the ASK Web Search API. This example will return the first 20 results for “Italy”. If you have trouble viewing the xml results in your browser, try Firefox.
Unless indicated otherwise, Ask parameters, tag attributes and the tags themselves are suppressed when their values are 0 or empty.
Request Parameters
| Parameter | Value | Description |
|---|---|---|
| a | integer | ? |
| f | integer; default 0 | First result number (offset) to return. Default is 0. Used to “page” through results. |
| i | integer | ? |
| p | integer; default 0 | Treat query words as a phrase |
| t | string. required. | Query terms. |
| u | integer; default 10. Takes values 1 to 200. | Maximum number of results to return |
| y | integer | ? |
Ask advanced query parameter modifiers
The following advanced query modifiers are documented for general use with ask search queries. They appear to work with the web services API as well.
| Parameter | Value | Description |
|---|---|---|
| site: | string, e.g. domain name | Restrict query to specific site. Must be used with another query term. To query an entire site, use site:www.mydomain.com inurl:www.mydomain.com. |
| intitle: | string | limit query to page titles containing text |
| inurl: | string | limit query to page URLs |
| lang: | language code. (NL Dutch; EN English; FR French; DE German; IT Italian; PT Portuguese; ES Spanish) | limit query to specific language. |
| geoloc: | Region code. (CA Central America; EU Europe; IA India / Asia; NA North America; OC Oceania; SA South America) | limit query to specific geographic region. We noticed that Ask’s spelling system is not aware of geoloc as a query parameter. Ask suggests an alternate spelling! |
| inlink: | string | limit query to links containing a string. |
| last: | week, 2weeks, month, 3months, 6months, year, 2years | limit query to pages indexed in a specific time frame. |
| afterdate: | afterdate:yyyymmdd | limit query to pages modified after date e.g. afterdate:20061015 |
| beforedate: | beforedate:yyyymmdd | limit query to pages modified before date, e.g. beforedate:20060312 |
| betweendate: | betweendate:yyyymmdd,yyyymmdd | limit query to pages modified during a date range, e.g. betweendate: 19960115,20030412 |
Response Fields
The response is delivered as a xml version 1.0 file using UTF-8 encoding. The entire response is wrapped in a SEARCHRESULTS tag. Tags may contain data, attributes or both.
| Tag | Attributes | Value | Description |
|---|---|---|---|
| RESULTSET | Wrapper tag for both topic and web search result sets. | ||
| QUERY | string | Contains the processed query string. | |
| ESTIMATERESULT | integer | Estimate of total matching results in ASK. | |
| TOTALRESULT | integer | Total results available for queries. Maximum of 200. | |
| FIRSTRESULT | integer | 0 based offset, used for looping through “pages” of results. | |
| NUMRESULTS | integer | Count of results in result set. Default is 10. maximum is TOTALRESULT, i.e. 200. | |
| SORT | string; default: rank | Sort order. Other values? | |
| MORERESULTS | string; true or false | ||
| STOPWORDS | string | List of stop words (i.e. the, a, for) filtered from query | |
| COUNT | integer | Number of stop words excluded from query |
| Tag | Attributes | Value | Description |
|---|---|---|---|
| RESULTSET | Wrapper tag for both topic and web search result sets. | ||
| QUERY | string | Contains the processed query string. | |
| ESTIMATERESULT | integer | Estimate of total matching results in ASK. | |
| TOTALRESULT | integer | Total results available for queries. Maximum of 200. | |
| FIRSTRESULT | integer | 0 based offset, used for looping through “pages” of results. | |
| NUMRESULTS | integer | Count of results in result set. Default is 10. maximum is TOTALRESULT, i.e. 200. | |
| SORT | string; default: rank | Sort order. Other values? | |
| MORERESULTS | string; true or false | ||
| STOPWORDS | string | List of stop words (i.e. the, a, for) filtered from query | |
| COUNT | integer | Number of stop words excluded from query |
Topics
In the relentless pursuit of relevant results, search engines strive to find patterns and relationships in the relatively unstructured data that is the web. Most search engines are able to group similar results from the same site, displaying just one or two results with an option to see more results from the same site. This grouping is called site level topic clustering.
The Teoma search engine, integrated into the current Ask, was one of the first to apply topic clustering to the greater web. Teoma determined clusters by identifying communities, web pages which link to each other, automatically naming the clusters based common phrases appearing in the group. The clustering was dynamic, performed for each query (although presumably search engines cache popular queries).
At one point, Teoma displayed topic clusters as folders above the main search results under the heading Web Pages Grouped by Topic. Later they were called “Refine”. Users could click on a topic to refine their search by organizing results into specific sub-topic categories based on the search query.
Topics are not currently displayed in Ask search query results. Fortunately, this information is still available via the web services API.
At most 10 topics are returned.
Topic Response Header
| Tag | Attributes | Value | Description |
|---|---|---|---|
| TOPICGROUPS | NUMGROUPS | integer; 1 to 10 | Number of topic clusters returned. |
| MORERESULTS | string; true or false | More results are available than those returned? |
Topic Response Details
| Tag | Attributes | Value | Description |
|---|---|---|---|
| TOPICGROUP | ID | integer, 0 based | Detail item wrapper tag. Contains item id, an integer starting at 0. |
| NAME | string | Contains keyword or keyword phrase used to name topic group. | |
| URL | string; partial query URL | A concatenation of t= and the words appearing in name, each separated with “+” as required to create a URL |
Web Search
Web Search Response Header
| Tag | Attributes | Value | Description |
|---|---|---|---|
| PREVWEBPAGE | string | Tag appears if prior “pages” of results are available. Contains query string to append to base URL which will return previous results. | |
| A | integer; default 1. | ? | |
| F | integer | First result number (offset) to return in the next result set. Is the sum of the current first result ID and the number of records requested. | |
| P | integer; default: 0. 1 if phrase. | Treat query words as a phase for proximity searching. | |
| U | integer; default 10. Values 1 to 200. | Number of records requested/ to request. | |
| MOREWEBPAGE | string | Tag appears if more “pages” of results are available. Contains query string to append to base URL which will return next results. | |
| A | integer; default 1. | ? | |
| F | integer | First result number (offset) to return in the next result set. Is the sum of the current first result ID and the number of records requested. | |
| P | integer; default: 0. 1 if phrase. | Treat query words as a phase for proximity searching. | |
| U | integer; default 10. Values 1 to 200. | Number of records requested/ to request. | |
| RELATED | string | Query string to perform related query | |
| A | integer; default 1. | ? | |
| I | integer; default 0. Not suppressed as with other attributes. | ? | |
| Y | integer; default 1. | ? | |
| WEBPAGE | Wrapper tag for web search results detail records |
Web Search Response Details
| Tag | Attributes | Value | Description |
|---|---|---|---|
| RESPONSE | ID | integer | 0 based offset |
| SCORE | decimal; values 0.01 to 1.00 | Site rank for query | |
| PARTITION | x,y integer pair | ? …worth some reflection. | |
| IND | integer, tag not present or 1 | Appears to be a presentation tag, meaning “indent”. Appears when a site cluster result is present. | |
| DOCTYPE | string, default: tag not present. PDF appears to be the only currently used value. | mime type. Ask officially supports Flash (swf) and PDF documents in addition to standard text. In reality, flash documents are not highlighted in Ask search results nor is the DOCTYPE tag populated. Ask does not yet support a doctype search filter, although it was promised in the past. inurl:pdf or inurl:swf work to a degree as a workaround. | |
| TITLE | string (Truncated after the first ~65 characters) | Document title | |
| URL | string | Document URL | |
| ABSTRACT | string (upto ~140 characters) | Document Abstract. Based on user query. | |
| SITE | string, default not present. Value is a URL | More results from current site. Present when IND is 1, is URL to query site for given keywords | |
| CACHED | KEY | string; e.g. 00*knpldsckmkez | Teoma 3.0 introduced cached versions of “popular” sites. A document may not be cached if the site is not popular or the document has used the robots “noachieve” meta tag. |
| URL | string | complete URL to access cached document. |
Teoma Experts’ Links / Resources
Teoma offered Expert Links, later called Resources, which we have not encountered in the xml API.
Expert links are Web sites created by individual enthusiasts, or “fans”, containing lists of resources relevant to the search topic. For example, an amateur golfer might have created a page devoted to his personal collection of favorite golfing sites. Teoma’s expert identification technology discovers and presents these pages as “Expert Links.”
Query the Ask Web Search API with our Perl example program
We have created a sample Perl program to query Ask’s Web Search API, saving the returned xml file and placing the results in a spreadsheet file for analysis. Note that this program is only a sample; it is not intended for production use. Be kind to Ask: don’t kill their servers with incessant automated queries.
Download and uncompress ask-search-1.0.pl.tgz. Read the licence terms and warnings at the beginning of the program. Note that the name of this file will change. Link to this page, not the file! To use, type
perl ask-search.pl <query terms>
i.e.
perl ask-search.pl the rolling stones site:bbc.co.uk
In this case, Ask will ignore the, a stop word. All sites with the domain suffix bbc.co.uk will be queried for rolling stones. Three files will be created:
- ask-query-response.xml – contains the raw results from Ask, in XML.
- ask-query-results.txt – contains a processed view of the XML, used by the perl module XML::Simple
- ask-search.xls – contains the query results in an Excel format spreadsheet. The first tab contains the search results. The second tab contains up to 10 topics.
Known Limitations
- “Encyclopedia” entries, such as those from Wikipedia, are not integrated in this data.
- “Narrow your search” options, as in the current Ask.com interface, are not the same as the topic group options.
- Sponsored ads are not present in the xml data.
- The binocular preview option URL is not present.
Unfortunately Ask’s current crawling frequency of web sites appears to be much lower than Ask’s competitors Google, Yahoo! and Microsoft’s MSN Search and Windows Live. Consequently, the base data processed by Ask’s algorithms is more likely to be out of date, clearly impacting the quality of search results.
Ask Maintains Separate Regional Search Engines?
Our initial analysis indicates that the results available from xml.teoma.com are the same as those from the US version of Ask, www.ask.com. It appears that localized versions of Ask, such as Ask Italia, are probably using a different underlying data base. We see richer results for Italian sites in Ask Italia compared to Ask.com. With other search engines, a simply modification of language and/ or region code, in this case lang and geoloc, usually solves the problem. Not with Ask. We also note the lack of an advanced query options page for Ask Italia.
External Resources
Last updated: 2009-06-17
Similar Posts:
- Tracking Search Engine Cache Page Views with Web Analytics
- Google Squared – expanded search engine results in a table
- Decrypting Ask’s Web Search API
- Joomla SEO: Removing useless meta tags
- Search engine Teoma is back. But will anybody notice, much less care?
Registration is now open for the next SEO Course (May 14 and 15) and Google Analytics Course (May 9 and 10) in Milan. Don’t miss the opportunity!
Sean Carlos is a web marketing consultant & teacher, assisting companies with their Search (SEO + PPC = SEM), Social Media & Digital Media Measurement strategies. Sean first worked with text indexing in 1990 in a project for the Los Angeles County Museum of Art. Since then he worked for Hewlett-Packard Consulting and later as IT Manager of a real estate website before founding Antezeta in 2006. Sean is an 
the script doen’t work since http://xml.teoma.com/ is unavailable
yea, it is not working. So, currently no any websearch api or others api from ask.com right? I could not search for it.