Living a SharePoint life

Tuesday, November 19, 2013

Removing a corrupted search service from SharePoint 2010

Removing the SharePoint 2010 Search Service can become quite a pain. Especially when the removal hasn’t work out the way it should have. So what do you do when the removal failed and your attempts to reinstall the search service are failing as well? I suffered from the same pains and therefor I’ll go thru the step I took to get my search service back online in this blog article.




The First thing to know is you do not simply remove the search service from a SharePoint Farm.

The second thing is how to clean up your mess....

The search service in SharePoint 2010 is, let’s say, different. It is integrated as a service application into SharePoint but the administration is done thru its own interface. For instance starting or stopping the service on a farm server cannot be managed thru the “Manage services on server” setting in the central administration. Instead you must change the topology of the search infrastructure. But you probably know this already.

So, in my case the customers search infrastructure was ‘a little bit messed up’. In fact it didn’t work at all and the attempts to recover search from the backup wasn’t helping either. Because the search index was very small and recreating the index wouldn’t take too long, the decision was made to recreate the search service.

Words of advice


You should be very confident in working with SharePoint. A good understanding and experience how to work with the Powershell and the Internet Information Server are necessary.

Before you continue reading and recreating the steps I did, you really should make sure you have a working backup you can actually restore! Even if the backup restores to a point where the search service is corrupted but the rest will still work. I did some bad things to my farm. You shouldn’t do this, under any circumstances, in productive environments unless you really have to. If you screw up, you might have to reinstall your farm. Don’t blame me if you wreck your installation.

Removing the service application from the farm


There are a lot of different blog articles out there explaining how to remove the search service application from the farm. In basic there are 3 ways to do so, with more or less success.
  • Removing thru the central administration
  • By the Powershell
  • With STSADM.EXE
I’ve tried all of them and only the STSADM.EXE had some effect. But in the end none of the three where really successfully for me and chances are neither for you. In the end I had the search service removed, but after recreating the service application SharePoint the service wasn’t functional and spitted exceptions at me. 

So here is how I got it working again:

Finding the ID for the service application


Open the PowerShell for SharePoint and enter the following command:

Get-SPServiceApplication

We are looking for the ID with the TypeName “Search Service Application”. If you have installed more than one search service application on your farm, make sure you choose the right ID. You don’t want to kill the wrong service application in the end.


The red marked Id is the one we are looking for.

Now start a Windows command shell as an Administrator. Navigate to the 14 Hive and use the STSADM.EXE command to execute the following:

STSADM.EXE -o deleteconfigurationobject -id <GUID of the service>

After this command, your service application is gone and with it the search administration service as well.

These are the steps I found everywhere on the web how to remove the search service. Unfortunately I belief there is a lot of information missing behind this point. For instance in my environment, removing the service application leaves the service application proxy untouched, so you must take care of that. My other findings where, that the WCF endpoints in the IIS where not removed. Neither have the Databases nor the Index files. These are things we must take care of our self.

Cleaning the proxy connections


Deleting an old proxy connection is pretty simple.

  1. Go to the Central Administration into the Application Management
  2. Choose Manage service applications
  3. Look for the old Search Service Application Proxy entry in the list and if it still exists, just delete it.
    Make sure it is the one from the service application you deleted in advance.

Cleaning up the service pool


The steps to clean out the IIS application pool should only be performed when your search service application was exclusive connected to its own application pool in IIS. If you reused the application pool for other services in your SharePoint farm you must not follow the next steps. When you have doubts, you should lean back, take a deep breath and reconsider your setup to be sure it’s ok to delete the application pool from IIS. It should only be removed when no other service is connected to the pool than the search service.

SharePoint Service Application Pool


Said so, we’ll take a look at the Service application pool next. Return to your Powershell and execute the following command:

Get-SPServiceApplicationPool

The Names returned in the list are the ones used when you create new service applications and you choose to reuse an existing application pool instead of creating a new one. In my case I found the name of the application pool I used for search as well as two old entries I deleted also.



To delete the Service application pool from the list you use the following command:


Remove-SPServiceApplicationPool <Name of the application pool>


IIS Application Pool


To remove the application pool from the IIS we’ll open the Internet Information Services Manager in Windows and select the Application Pools in the navigation pane. Finding the right application pool can be a bit tricky because the Name is a Guid. There are a few hints e.g. if you used a dedicated account for the search service application pool or the low number of connected applications to that pool. To be sure right click on an application pool entry and select View Applications. Look at the Physical Path and see if its location is pointing to the search web service directory. If you found the right application pool you can delete it. But before doing so, we’ll remove the WCF endpoints first.

Deleting the WCF service endpoint


In the IIS Manager Navigation pane, open the SharePoint Web Services application and look for the Guid that is connected to application pool. Validate you have chosen the correct WCF endpoint by selecting the Basic Settings action in the Action pane. Look at the Physical Path and see if its location is pointing to the search web service directory. When you have the right WCF endpoint for the search service, delete it. After you have removed the search service WCF endpoint you can continue to remove the application pool as well.

Deleting the search index

To delete the search index you only need the Windows explorer or a command line. Navigate to the location where the index files are stored. The names of the folders should be represented by Guid. Select the folders and delete them.

The default location for the index files is C:\ProgramFiles\Microsoft Office Servers\14.0\Data. For you the location may have been relocated to an other destination.

Stoping the search service instances


Our final step is to make sure all search service instances are stopped in the farm. To find all instances return to the Powershell and use the following command:
Get-SPEnterpriseSearchServiceInstance
This returns a list of all servers in your farm hosting the search service. For all entries in the list the Status should be Disabled instead of Online. If one or more of the instances are online we must shut them down. To do so we need the Id that can be seen in the list returned as well. Use the following command inside the Powershell:

Stop-SPEnterpriseSearchServiceInstance -Identity <Enterprise Search Service Instance ID>

Recreating the search service application

When you removed everything from the server you can start recreating the search service application in your farm. I recommend you do this with the Powershell. That way you will gain more control over the naming of the databases used to create the topology. Microsoft provides a good article on this topic in the Technet library how to do this:
http://technet.microsoft.com/en-us/library/cc262839(v=office.14).aspx

Well I hope this helped someone out there to get their search service running again. For me it sure did.



This post is

11 comments:

  1. Thanks a lot. The "Deleting the WCF service endpoint" was the thing I was looking for. Thanks a lot

    ReplyDelete
  2. I had the following error after corrupted Search Service deletion:

    Event ID: 3760
    Event Description: SQL Database 'Search_Service_Application_CrawlStoreDB' on SQL Server instance 'SQLServer' not found.


    Take a look to the following post of Steve Magid (http://social.technet.microsoft.com/Forums/en-US/78214c39-50e5-42e7-a7a6-29530bc96b41/crawlstore-database-which-was-deleted-causing-event-id-3760-and-5586?forum=sharepointgeneralprevious)


    ==================================================
    CrawlStore Database which was deleted causing event ID 3760 and 5586

    =================================================

    I finally figured this out.

    Even though I deleted the databases in SQL, I ran get-spdatabase in Powershell and noticed my old search databases were still showing.

    I then proceeded to delete these databases in Powershell

    $oldDB = get-spdatabase id

    $oldDB.delete()

    With thanks to this kind blogger

    http://msmvps.com/blogs/shane/archive/2010/12/06/search-database-guid-issue.aspx

    ==================================================

    ReplyDelete
  3. Thanks a million, though i had to restart the server for it to work after using your guide.

    Thanks again

    ReplyDelete
    Replies
    1. Glad to hear you got the service back working again.

      Delete
  4. Do you have any advice in a case where trying to delete a service application (project server in my case) fails with a null object error?
    "Object reference not set to an instance of an object."
    Trying to delete via CA, Powershell remove-spserviceapplication, and stsadm delete object all fail with this null error, though Get-SPServiceapplication shows the application.
    I'm not finding this issue or how to resolve anywhere I search for weeks now.

    ReplyDelete
    Replies
    1. I normaly have these problems with the SP Timer. Use Get-SPTimerJob on the PowerShell and look for anytimer job that has null entries.
      https://technet.microsoft.com/en-us/library/ff607914.aspx
      This article might help as well. Use the Bing Translator for english, please.
      http://blog.greenbrain.de/2013/04/keine-verbindung-mit-dem-sharepoint.html

      Delete
  5. Thanks for the good hints. There is just one Little mistake: It should be Get-SPEnterpriseSearchServiceInstance... ("Enterprise" is missing)

    ReplyDelete
    Replies
    1. You're right. I've changed it now. Thank's for the hint.

      Delete
  6. ok, I had deleted the search service application from CA. The search service was still running in windows services and get-spenterprisesearchserviceinstance showed a status for some components as running. I followed the article steps for deleting app pools and executed Get-SPEnterpriseSearchServiceInstance|Stop-SPEnterpriseSearchServiceInstance

    Good sauce ;)

    ReplyDelete
  7. After all these years you are still helping!

    thanks alot :)

    ReplyDelete

Featured Post

The Retro Powershell - Looking good in 8-Bit | Part 1

I wrote a little script that, when placed in your PowerShell Profile, will print a message similar to the old boot message you got from you...