SPS NH: Search Driven Design Patterns for SharePoint 2013

Event:
SharePoint Saturday NH
Saturday, September 21, 2013

Session Title:
Search Driven Design Patterns for SharePoint 2013

Abstract:
First, we will discuss SharePoint portal design flaws, related deficiencies, and challenges that exist with SharePoint in the enterprise.  We will discuss the Search platform and redefine what it means to us and how we leverage it.  We will take a tour of a Search Driven portal and review the configuration of Managed Properties, Result Sources, Query Rules, Search Results Web Parts, Display Templates, Search Boxes, and Search Pages.  Next, we will construct Search Driven components using creative techniques and reflect on the benefits of the overall approach that we followed, comparing and contrasting with other design patterns.  Finally, we will take a step back and discuss the future of Search and how to align SharePoint Search with the overarching Enterprise Architecture strategy.

This session provides practical guidance and best practices for SharePoint 2013 and SharePoint 2010 portal design, content management design, and search configuration based on real world lessons learned.  The demonstrations and technical deep dives showcase Search Driven Design Patterns in motion while providing instructions for how to build a Search Driven portal.

Slides:

Search Driven Design Patterns for SharePoint 2013.pdf

SPS NYC: Search Driven Design Patterns for SharePoint 2013

Event:
SharePoint Saturday NYC
Saturday, July 27, 2013

Session Title:
Search Driven Design Patterns for SharePoint 2013

Abstract:
First, we will step back and examine some SharePoint related deficiencies, common portal design flaws, and challenges that exist in the enterprise.  Next, we will align ourselves with an overarching Enterprise Architecture strategy and redefine the role of SharePoint Search in the organization.  From there, we will design a SharePoint 2013 portal that is optimized and ready for Search Driven design patterns.  Finally, we will construct Search Driven components using creative techniques and reflect on the benefits of the overall approach that we followed.

Slides:

Search Driven Design Patterns for SharePoint 2013.pdf

SPS Boston: Search Driven Design Patterns for SharePoint 2013

Event:
SharePoint Saturday Boston
Saturday, April 27, 2013

Session Title:
Search Driven Design Patterns for SharePoint 2013

Abstract:
First, we will step back and examine some SharePoint related deficiencies, common portal design flaws, and challenges that exist in the enterprise.  Next, we will align ourselves with an overarching Enterprise Architecture strategy and redefine the role of SharePoint Search in the organization.  From there, we will design a SharePoint 2013 portal that is optimized and ready for Search Driven design patterns.  Finally, we will construct Search Driven components using creative techniques and reflect on the benefits of the overall approach that we followed.

Slides:

Search Driven Design Patterns for SharePoint 2013.pdf

NYESUG: Search Driven Design Patterns for SharePoint 2013

Event:
The NY Enterprise Search User Group
Wednesday, January 9, 2013

Session Title:
Search Driven Design Patterns for SharePoint 2013

Abstract:
First, we will step back and examine some SharePoint related deficiencies, common portal design flaws, and challenges that exist in the enterprise.  Next, we will align ourselves with an overarching Enterprise Architecture strategy and redefine the role of SharePoint Search in the organization.  From there, we will design a SharePoint 2013 portal that is optimized and ready for Search Driven design patterns.  Finally, we will construct Search Driven components using creative techniques and reflect on the benefits of the overall approach that we followed.

Slides:

Search Driven Design Patterns for SharePoint 2013.pdf

SharePoint 4.0: How Search Indexing Works

The crawl process is essential to the Indexing side of SharePoint 2010 Search. Exploring the Search Service Application screens provides some insight to the configurable components that make up Search in SharePoint 2010, but you may actually have to dive deep into the system documentation and other resources to gain an understanding for the steps that occur within the subsystems, when SharePoint is instructed to crawl content.

The crawl process goes something like this:
1. Full crawl started
2. Start address moved to queue
3. Protocol determined
4.Connector selected
5. iFilter opens files
6. Content index created on crawl server
7. Index moved in batches to query server
8. Data written to Crawl and Property databases
(Microsoft, 2010)

One myth about SharePoint 2010 crawl process is that when content is crawled it never touches the file system of the Crawl Server and goes directly to Query Server...this isn't true. The way it works in environments that have crawler and query roles on separate servers is; a content index is built on the crawl server initially, but is moved in batches to the query server(s). The point when all batches have propogated to the query server, then it may appear as if there is no foot print on the Crawl server's file system...this is because all the batches have been moved up.

From a server topology perspective, Search requires three different server roles. These include Crawl Server, Query Server, and Database Server. Each of these roles plays a role in the crawl process. For scalability and availability, the architecture supports configurations that include one or many of each of the server roles. Determining the best topology is a balancing act of capacity requirements with available resources.

Reference

Microsoft (2010). Microsoft SharePoint 2010 Product Information Capabilities Search. Retrieved September 22, 2010 from
http://sharepoint.microsoft.com/en-us/product/capabilities/search/Pages/default.aspx.

MSDN: Enterprise Search URL Syntax Reference

One of the most simple, yet powerful features of Microsoft SharePoint Search is the URL syntax. Using URL syntax you can submit a query to a search results page without requiring the use of a SharePoint search box. This means that search results can be rendered on a search results page any way that the proper URL string can be built.

For example, search queries may be submitted implicitly throughout the user experience as a user is navigating a SharePoint Web site. Hyperlinks (dynamic or static) containing search query parameters do all of the work.

The MSDN article, "Enterprise Search URL Syntax Reference, http://msdn.microsoft.com/en-us/library/aa637082.aspx," explains the proper format for submitting keywords, scope, view, and start page information through the query string.

SharePoint 3.0: How to Crawl Case Sensitive URLs

By default, MOSS will not crawl URLs which are case sensitive. However, Microsoft Help and Support Article ID 932619 provides instructions for making a registry modification to the Index server which allows case sensitive URLs to be crawled.

Microsoft Help and Support Article ID 932619:
You cannot crawl case-sensitive Web content in SharePoint Server 2007
http://support.microsoft.com/kb/932619

It is important to note that a content index reset is required. For this reason, this modification may require a maintenance window for production environments, especially large environments with lots of items in the content index that will need to be crawled again.

Like most system changes, it is a good idea to test this first in Development or QA environments, record the change, and then schedule a time to deploy it to production. The instructions are straightforward, however, the first time that I made the modification, the search service didn't start properly so I had to stop it and start it a second time. Then it was fine. Running through the steps in Development or QA first will allow you to work the kinks out.

SharePoint 3.0: Error: The Backup/Restore Job Failed

Description:


In this scenario, a full farm backup failed because the backup of the "SharedSearchIndex" failed.

Error: The backup/restore job failed. In order to rerun the backup/restore the timer job must be deleted from the Timer Job Definitions page. Failure Message: The backup job failed. For more information, see the error log that is located in the backup directory.

Central Administration > Operations (Tab) > Backup and Restore (Heading) > Backup and restore job status

Error: Object Shared Search Index failed in event OnBackupComplete. For more information, see the error log located in the backup directory. WebException: The current operation timed-out after 3600 seconds.

Central Administration > Operations (Tab) > Backup and Restore (Heading) > Backup and restore job status



Error: The Backup/Restore job has the status of "Failed"

Central Administration > Operations (Tab) > Global Configuration (Heading) > Timer job status



Solution:


This issue may be caused by a timeout. To resolve this issue, there are a few steps. First, any time a SharePoint Backup/Restore job fails, you must delete the failed timer job definition, otherwise subsequent backup attempts will also fail.

Second, take an inventory of scheduled jobs that are running on the farm, including incremental crawl schedules, SharePoint backup scheduled tasks, and Enterprise Backup jobs (e.g. Symantec BackupExec, Networker, etc..). Measure how much time incremental crawls are taking to run. Also, measure how much time the backup is taking to run. Determine what time the backup job needs to start. Keep in mind that if an enterprise backup solution is backing up the SharePoint backup files, then the SharePoint backup should COMPLETE before the tape backup job begins.

If necessary, adjust the scheduled jobs so that the crawls are not interferring with the backup job.


Steps:

1. Delete the Backup/Restore timer job definition.

Central Administration > Operations (Tab) > Global Configuration (Heading) > Timer job definition

2. Adjust the start time of the incremental crawl. Make sure the schedule is set up in a way that the incremental crawls complete at a time that is earlier than the start time of the scheduled task that runs the backup job.

For example, if the incremental crawl takes less than an hour to run, and the the backup job is scheduled to run at 8:00 PM, then schedule the last incremental crawl to occur at 7:00PM.

SSP Administration > Search (Heading) > Search Settings > Content sources and crawl schedulesContent Source > Edit > Crawl schedules


3. If necessary, adjust the start time of the scheduled task that runs the backup

Start > Settings > Control Panel > Scheduled Tasks

SharePoint 3.0: Search Results: How to Add a Document Properties Hyperlink

Introduction

As for a document appearing in SharePoint 3.0 search results; the standard formatting includes a hyperlink to open the document. However, you may want your search results to also include a link to open the display form or edit form of the document's associated list item. This way users have the options of opening a document as well as viewing and editing document properties all from the search results page.


Step 1. Crawl the Item ID Crawled Property

* SharePoint Shared Services Administration > Search Settings > Metadata Property Mappings > Crawled Properties
* Search for ows_ID(Integer)
* Edit/Map Property: ows_ID
* Mappings to managed properties: Check to "Include values for this property in the search index"
* Click OK


Step 2. Create a Managed Property for the Document's List Item ID

* SharePoint Shared Services Administration > Search Settings > Metadata Property Mappings > Managed Properties
* New Managed Property
Name and type
Property Name: OWSIDInteger
Description:
The type of information in this property: Integer
Mappings to crawled properties: Select - Include values from a single crawled property based on the order specified
Crawled properties mapped to this managed property: OWS_ID(Integer)
Use in scopes: No - Do not allow this property to be used in scopes

Step 3. Run a Full Crawl

* SharePoint Shared Services Administration > Search Settings > Content sources and crawl schedules > For each applicable content source, invoke the context menu and Start Full Crawl


Step 4. Make the New Managed Property Part of the Search Query

* Search Center site collection > search results page > search results web part > edit web part > Modify Shared Web Part
* Web part tool pane > Results Query Options > Selected Columns:
Modify the XML of Selected Columns to include the new managed property, OWSIDInteger

Reference:
How to: Change the Properties Returned in the Core Search Results
http://msdn2.microsoft.com/en-us/library/ms560074.aspx

Step 6. Test the Search Query to Ensure the New Managed Property is Returning Values

* Search Center site collection > search results page > search results web part > edit web part > Modify Shared Web Part
* Web part tool pane >Miscellaneous > XSL Editor:
Modify the XML so it returns raw XML results

Reference:
How to: View Search Results XML Data
http://msdn2.microsoft.com/en-us/library/ms546985.aspx


Step 5. Make the Managed Property Display in the Search Results

5A. Create an XML File Containing the Raw XML Representation of the Search Results
* Open an XML editor such as Visual Studio
* File > New > File > XML File
* Search Center > Perform a search (the results should still be coming back as raw XML)
* Copy the search results into the XML file
* Save the XML file to be used in Step 5B

5B. Create a new XSL Stylesheet
* Create a test site
* Add a Data View Web Part (DVWP) to the Test Site
* SharePoint Designer > Open test site > Open default.aspx
* Split View
* Menu > Insert > SharePoint Controls > Data View
* Data Source Library > XML Files > Add an XML file:
Upload the XML file from Step 5A
* Drag the content from the new data source to the DVWP
* Format the DVWP accoding to the desired look and feel for search results
* When look and feel is complete, open the XML editor and create a new XML file:
Visual Studio > File > New > File > XML File
* SharePoint Designer > Highlight the DVWP in the Design window
* Copy the code for the DVWP from the Code view
* Paste the XML into the new XML file
* Identify the XSL Stylesheet portion of the XML
* Cut away any unnecessary XML from the file, leaving only the XSL Stylesheet
* Save the XML file to be used in Step 5C

5C. Apply the new XSL Stylesheet to the Search Results Web Part
* Visual Studio > Open file:
Open the XML file created in Step 5B
* Search Center site collection > search results page > search results web part > edit web part > Modify Shared Web Part
* Web part tool pane >Miscellaneous > XSL Editor:
Open the XSL Editor
* Paste the XML from Visual Studio into the XSL Stylesheet, overwriting the existing stylesheet
* Click OK

SharePoint 3.0: Error: Crawl Log "The object was not found."

Description:

Unable to crawl a web site content source.

Error: "The object was not found. (The item was deleted because it was either not found or the crawler was denied access to it.)"

Solution:

Determine if the web site is case sensitive. If so, implement the configuration changes described in the following KB article:

You cannot crawl case-sensitive Web content in SharePoint Server 2007
http://support.microsoft.com/kb/932619


Keywords:
Crawling case sensitive web sites; MOSS 2007; Crawl jsp pages; How to crawl .jsp pages with SharePoint; Case-Sensitive; sharepoint search jsp; sharepoint search .jsp

SharePoint V3: Search: “The search request was unable to connect to the Search Service”

Description:

Error: “The search request was unable to connect to the Search Service”

Event Log has errors referencing a corrupt index.


Solution:

* Reset index
SSP Administration > Search Settings > Reset all crawled content

* Perform a full crawl
SSP Administration > Search Settings > Content Sources > Perform a full crawl

SharePoint 3.0: Default Search Relevance by File Type

The Search engine in SharePoint 3.0 considers certain file types more relevant than others. Below is a list of file types in sequence of most relevant to least relevant.

• Web Pages
• PowerPoint Presentations
• Word Documents
• Emails
• XML Files
• Excel Files
• Text Files
• SharePoint List Items

SharePoint V3: Error: No results matching your search were found

Description:

When performing a contextual search (WSS Search) from a SharePoint 3.0 site, no results are returned. Error: No results matching your search were found.





Are you using SSL?
Are you using a network load balancer?
Are there multiple domains in the environment?
Are the web applications accessible from outside the organization?



Solution:

Central Administration > Operations > Alternate Access Mappings

* Ensure that the Pubic URL for the web application if fully qualified

* Ensure that all possible URLs for the web application are configured in the default (or appropriate) zone

e.g.
https://sharepoint.domain.com
http://sharepoint.domain.com
https://sharepoint
http://sharepoint

SharePoint V3: Search Considerations for Choosing Yes No Column Type

I was working on a search customization a while back, displaying search results in a grid view. I encountered a design decision that required that I step back and change the column type I was using for some of the Yes/No columns in the site collection.

One lesson I learned is that when you are deciding on a which column types to use, especially Yes/No columns, keep search close at mind because it will impact search downstream. Ideally, if your columns will be leveraged by managed properties and used in search scopes to help users fine tune their queries, then keep in mind how users think.

For example, I had some Yes/No columns that I originally set up as Yes/No Checkboxes. I created managed properties for these columns but then realized...nobody is going to search on 0 and 1, they are going to search on yes and no. E.g. Give me all my ice cream sundae options where vanilla ice cream = yes...not where vanilla ice cream = 1.

Anyway, there are at least four common ways to achieve Yes/No in a SharePoint column:

Choice Radio
Choice Drop Down
Choice Checkbox
Yes/No Checkbox

The table below compares each of these:



As a result of doing this comparison, I have been using choice drop down more often than not in lists and libraries where Yes/No values are required for metadata. Even though the Yes/No checkbox may more appealing to users filling out the metadata, I think the drop down wins when you weigh in the search considerations.

SharePoint V3: Search: “The search request was unable to connect to the Search Service”

Description:

Error: “The search request was unable to connect to the Search Service”


Solution:

Step 1

* Central Admin > Application Management > Office SharePoint Server Shared Services > Create or Configure this farm's shared services
* Shared Services Provider (drop down) > Edit Properties
* Edit Shared Services Provider > Index Server: Verify the index server is set

Step 2

Single Server Farm

On the MOSS 2007 server:
* Start > Run > CMD > stsadm.exe -o osearch -action stop
* Start > Run > CMD > stsadm.exe -o osearch -action start -role queryindex

Multiple Server Farm

On the MOSS 2007 Front End Web/Query server:
* Start > Run > CMD > stsadm.exe -o osearch -action stop

On the MOSS 2007 Index server:
* Start > Run > CMD > stsadm.exe -o osearch -action stop

On the MOSS 2007 Index server:
* Start > Run > CMD > stsadm.exe -o osearch -action start -role index

On the MOSS 2007 Front End Web/Query server:
* Start > Run > CMD > stsadm.exe -o osearch -action start -role query

SharePoint V3: Modify Advanced Search Box Result Types

Description:

By default, the Advanced Search Box offers a Result Type filter that allows users to select which type of documents they want returned in their search query.

By default, the options are:
* All Results
* Documents
* Word Documents
* Excel Documents
* Presentations





You would like users to be able to query for PDF Documents, too.


Solution:

* Site Actions > Edit Page > Advanced Search Box > Edit > Modify Shared Web Part
* Expand the Properties node > Click on Properties elipses
* Open Visual Studio > File > New File > XML File
* Copy the XML into Visual Studio
* Locate the Result Type node for Word Documents, Copy it, Modify it for PDF

* Copy the XML back into the Advanced Search Box Properties Editor, Click OK
* Publish the page

Links: SharePoint V3: Search: BDC

Enabling Business Data Search
http://msdn2.microsoft.com/en-us/library/ms492695.aspx

SharePoint 2007: BDC - User Profiles
http://blah.winsmarts.com/2007-4-SharePoint_2007__BDC_-_User_Profiles.aspx

Walkthrough: Configuring Search for the AdventureWorks Business Data Application Sample
http://msdn2.microsoft.com/en-us/library/ms493671.aspx

Links: SharePoint V3: Search: Search Center

Adding Properties to MOSS Advanced Search
http://www.codeproject.com/spoint/MOSS_Advanced_Search.asp?print=true

How to: Change the Properties Returned in the Core Search Results
http://msdn2.microsoft.com/en-us/library/ms560074.aspx

How to: Display a Search Scope in the Search Box and Advanced Search Web Parts
http://msdn2.microsoft.com/en-us/library/ms498872.aspx

Walkthrough: Add a Tab and Custom Search Page with Enterprise Search Web Parts to the Search Center
http://msdn2.microsoft.com/en-us/library/ms584031.aspx

Links: SharePoint V3: Search: Planning

Plan search (Office SharePoint Server)
http://technet2.microsoft.com/Office/en-us/library/e8c0fccd-8364-4352-8778-c9c46a668b701033.mspx

Plan for administrative and service accounts (Office SharePoint Server
http://technet2.microsoft.com/Office/en-us/library/f07768d4-ca37-447a-a056-1a67d93ef5401033.mspx?mfr=true

MOSS Search - Customer Proof of Concept Scoping and Typical Tasks
http://blogs.msdn.com/miketag/archive/2007/01/18/moss-search-customer-proof-of-concept-scoping-and-typical-tasks.aspx

SharePoint Deployment Capacity & Performance Planning 2003 & 2007
http://blogs.msdn.com/joelo/archive/2006/07/19/671789.aspx

Estimate performance and capacity requirements for search environments
http://technet2.microsoft.com/Office/en-us/library/5465aa2b-aec3-4b87-bce0-8601ff20615e1033.mspx?mfr=true

Estimate performance and capacity requirements for Internet environments (Office SharePoint Server)
http://technet2.microsoft.com/Office/en-us/library/45997bad-da15-4442-8727-78e0f2dc5a9c1033.mspx?mfr=true

Estimate MOSS Search Disk Space Requirements
http://blogs.msdn.com/miketag/archive/2007/07/05/estimate-moss-search-disk-space-requirements.aspx

SharePoint V3: MOSS 2007 Search: How to Index Lotus Notes Content

I recently configured MOSS Search to index Lotus Notes. Here is an overview of the process.

Preparation:
* Lotus Notes server name
* Lotus Notes database file name
* List of fields you want to display in search results
* Notes user logon file (e.g. user.id)
* User credentials for installing Notes client

Steps:
* Install Lotus Notes Client 6.0 to the SharePoint Index Server
* From the index server, share the c:\program files\lotus\notes\data directory allowing WSS_WPG group access
* From an existing workstation, copy a valid user id file to index server c:\program files\lotus\notes\data
* Download the Lotus C++ API Toolkit 3.0I* Extract the files* Copy lcppn30i.dll to the index server, c:\program files\microsoft office servers\12.0\bin
* Rename 1cppn30i.dll to 1cppn30.dll
* From index server, c:\program files\microsoft office servers\12.0\bin, execute notessetup.exe
* Specify the server name, database file, view, etc.
* From index server, cmd prompt, net stop osearch, then net start osearch
* SSP Admin > Search Settings > Content Sources: Configure content source for Lotus Notes
* SSP Admin > Search Settings > Scopes: Configure scope for Lotus Notes content source
* SSP Admin > Search Settings > MetaData Property Mappings: Create managed properties for those Lotus Notes properties that you want to search or display in search results
* Search Center: Configure the search web parts and search results web parts (and related xml files) to display useful field when the search result item is coming from the Lotus content source

Lessons:
* The notessetup.exe won't work with the 1cppn30i.dll file name, need to rename it to 1cppn30.dll
* I used 6.0 Notes Client, because Notes Client 6.5 did not create a data directory

References:
* Configure Office SharePoint Server Search to crawl Lotus Notes (Office SharePoint Server 2007 for Search)
* Using SharePoint 2007 to Index Lotus
* Secure crawls of Lotus Notes with SharePoint
* How to Configure Search to Honor Lotus Notes Security Settings
* MOSS2007 Search and Index Lotus Notes
* IBM software downloads (for Lotus C++ API ToolKit)