SharePoint 2010 Search : Replacing the SharePoint Search Engine (part 2) - FAST Search Server 2010 for SharePoint

3. FAST Search Server 2010 for SharePoint

In Wave 14, Microsoft has decided to attack alternative enterprise search engines head-on. In early 2008, Microsoft acquired the Norwegian enterprise search vendor FAST Search and Transfer (FAST). Established nearly a decade earlier, FAST was one of the world's leading providers of high-end search and business intelligence solutions. Its platform, FAST ESP, is a highly customizable and scalable search engine capable of meeting a seemingly endless array of search- and intelligence-based business needs. By acquiring FAST, Microsoft took a significant step toward alleviating its deficiencies in the search marketplace.

Since the acquisition, Microsoft and the FAST Group have been hard at work to integrate FAST's search technologies into Microsoft's more mainstream infrastructure solutions. While some of these technologies are available in native SharePoint 2010 Search, the release of FAST Search Server for SharePoint 2010 established Microsoft as a real player in the enterprise search space. FAST Search Server for SharePoint 2010 is a replacement for the SharePoint search architecture. It integrates seamlessly into SharePoint 2010 with a familiar interface and by picking up settings, permissions, and metadata. FAST Search Server for SharePoint 2010 is not an extension to SharePoint 2010 Search, however; it is a replacement, since it does require dedicated servers and adds a completely different index pipeline.

FAST Search Server for SharePoint 2010 can greatly expand the user interface and the search back end for enterprise SharePoint deployments. The platform includes all of the features of standard SharePoint 2010 Search and adds additional user interface features such as advanced sorting, document previews, deep numbered refiners, and visual best bets. Advanced sorting allows for a broad range of additional search sorting options beyond SharePoint 2010's standard relevancy. This feature is similar to the "display relevance view option" and "display modified date view option" in standard SharePoint 2010, but provides more extensive sorting such as size, document rank, created date, and modified date. These sorting options can also be customized to meet the specific needs of a deployment. Figure 3 shows the default options made available with this feature.

Figure 3. FAST Search Server for SharePoint 2010 advanced sorting

One of the most popular features of FAST Search Server for SharePoint 2010 is the document thumbnail and preview feature. FAST Search Server for SharePoint 2010's document preview is made available through the use of SharePoint Office Web Applications (OWA), so previews are available only for Office documents. More specifically, first-page thumbnails are available on the search results page for Microsoft Word and PowerPoint files. Full document previews are currently available only for PowerPoint files (Figure 4). For FAST's thumbnail and document preview to function, the document must be stored on a SharePoint 2010 site that has OWA enabled. While this does provide a nice visualization enhancement for SharePoint, clients using the search replacement should be aware that previews for Office files other than Word and PowerPoint are not supported and users must have OWA enabled on their browsers. This does lead to the restriction that users cannot preview PDF, Autodesk, or other Microsoft Office file formats. Caution should also be used when deploying this feature to public sites or those that are expected to be accessed via mobile devices, where the installation of OWA cannot be reasonably assured.

Figure 4. FAST Search Server for SharePoint 2010 PowerPoint preview

FAST Search Server for SharePoint 2010 greatly enhances the Best Bets feature to allow for multiple visual best bets. Unlike SharePoint 2010 Search, which allowed for only one best bet per keyword, multiple suggestions can be tied to a single keyword using FAST. In addition, with FAST, Best Bets can use HTML for graphical best bets similar to those found on web sites instead of the simple text suggestions provided with SharePoint. This is especially beneficial to organizations that wish to use SharePoint 2010 on public-facing deployments. Figure 5 shows the difference between best bets in SharePoint 2010 and those available in FAST for SharePoint. The visual banner is a FAST best bet, whereas the second item titled sharepoint with a star is a standard SPS2010 best bet.

Figure 5. FAST Search Server 2010 for SharePoint best bets

The popular search refinement Web Part is also greatly enhanced in FAST for SharePoint. Instead of providing plain, shallow refinements that can analyze only the first 50 items in a result set by default, FAST's refinements are deep and numbered (Figure 6). This allows for refiners based on the aggregation of managed properties for the entire result set. This has the significant benefit of refinement options reflecting all items matching a query. It does, however, take a significant amount of memory from the query server for large result sets, so enabling this feature should be done with caution. For deployments of FAST for SharePoint with more limited hardware or a high query frequency, the option remains available to display refinements based only on the aggregation of properties for the top 100 search results.

Figure 6. FAST Search Server for SharePoint 2010 deep refiners

FAST Search Server for SharePoint 2010 can also enhance the user interface by providing different result relevancy, best bets, and refinement options based on user context. This allows the search interface to be catered toward user groups. For example, when searching for the keyword "expansion," people logged in from the sales group may get a best bet for documents outlining the strategy to increase sales figures in a new geography. People in this department may also get search refinements based on product groups or sales territories. If a user from the human resources department executes the same query, FAST Search Server for SharePoint 2010 can be configured to provide a different experience for that user based on his or her department. The user in the human resources department that searches for the keyword "expansion" could get a visual best bet for hiring policies in a newly established department. The search refinements may include fields such as department, clearance level, or policy type.

All of these user-side features can be added through significantly less expensive search extensions. The features that make FAST Search Server for SharePoint 2010 stand out are those on the back end. FAST Search Server for SharePoint 2010 replaces SharePoint 2010's search architecture with an extremely scalable search engine. While SharePoint 2010's native search index can handle up to 100 million items, FAST Search Server for SharePoint 2010 can be scaled to index billions of items while retaining sub-second query times. Like the Google Search Appliance, scalability is dependent on licensing and hardware. FAST Search Server 2010 for SharePoint requires its own dedicated hardware. It cannot be installed on the same server as SharePoint Server 2010. The minimum hardware requirements include a dedicated server with 4GB RAM, 4 CPU cores, and 50GB of disk space, but 16GB RAM, 8 CPU cores, and 1TB of disk space on RAID are highly recommended to achieve the intended performance. Due to the enterprise nature of the search engine, however, most deployments contain about four servers dedicated to FAST Search Server 2010 for SharePoint.

In addition to an extremely scalable index, FAST provides much greater control over the index pipeline. SharePoint 2010 Search was designed to make access to powerful search relatively simple. As a result, some limitations were implemented in what can be manipulated in regards to relevancy during indexing. SharePoint 2010's index pipeline can be considered a black box, with very little ability for manipulation. SharePoint 2010 and FAST's default relevancy is designed to provide accurate results on intranets against large sets of different data. Each organization's needs are different, and because relevancy is perceived differently among different users and organizations, it may be helpful to tune relevancy according to specific needs. Organizations that wish to directly influence relevancy algorithms can do so using FAST Search Server for SharePoint 2010. Administrators with programming and Windows PowerShell experience can blend multiple rank components such as content context, authority, term proximity, and quality. Keyword-driven, statistic, dynamic, and linguistic relevance can all be manipulated to affect the order of items in a result set.

FAST allows administrators to extract metadata at index time, execute business intelligence activities on the fly, and build custom queries using FAST's proprietary FQL query language. FAST's metadata extraction is similar to the auto-classification solutions , although it does require a bit more manual attention. Metadata can be extracted from the plain text within items as they are indexed, even if no formal structure exists within the file. This allows for significantly improved connection of structured concepts and properties to documents. This metadata can then be used to enhance search relevancy and provide more accurate refinement options to users.

Finally, much of the real potential for large enterprises that choose to implement FAST Search Server for SharePoint 2010 is tied to the ability to use it for a core to custom search-based application (SBA) development. Microsoft has taken the first step to pulling search technologies out of the search center and into other applications. For example, FAST can be connected to business intelligence solutions such as Excel workbooks and Reporting Services reports. This allows for advanced discovery of statistical information that may be useful for management teams to understand how employees are working and utilizing data. It also provides the ability for marketing teams to use search to understand how clients may be navigating a public SharePoint site or what best bets are most effective in converting site browsers to clients. Developers and consultants can utilize the FAST Search Server for SharePoint 2010 as the core infrastructure for advanced systems or portals that rely on information access and reporting. Examples include search-based applications such as research and development portals, product support dashboards, supply chain management trackers, and compliance tools.

As mentioned earlier, FAST Search Server for SharePoint 2010 is licensed separately from SharePoint 2010. Unlike the Google Search Appliance, which includes everything needed in one package, organizations that wish to implement FAST must purchase hardware, Windows Server, SQL, and FAST Search Server licensing. Users must have the enterprise client access license (ECAL) to access the FAST features. At the end of 2010, pricing per server for FAST Search Server 2010 for SharePoint started at about US$22,000.

4. Choosing the Best Option

This section was meant to provide only a high-level overview of the most popular options for replacing SharePoint 2010's native search engine. Considering the financial and time investments required to implement enterprise search engines, even those that are marketed to be simple, a thorough analysis of the best solution to meet an organization's particular needs should be conducted. A hasty jump into a mixed platform may prove to be a costly waste of resources if better hardware or user interface extensions would meet needs with greater efficiency. Table 1 will assist with an initial analysis of options.