built in function

I'm not sure if it will be a good idea to process big text collection using C# or any dot net code.

Can't you leave it to SQLServer?

You can use SQL Server Full text feature

Create full text index on the table containing your data, it will automatically create a search terms catalog which you can use

Fol. SQL query can give you search terms

Also have a look at fol. links

http://msdn.microsoft.com/en-us/library/cc280607.aspx

http://technet.microsoft.com/en-us/library/cc280607.aspx