SharePoint Workflow: "Failed on Start"

I ran into this infamous "failed on start" error when running SharePoint OOB workflow, and it took me quite a while figuring out before I nearly gave up. So I think it is worth a blog.

Sympton:

1) any publishing workflow failed with the same error, but none of non-publish workflow failed
2) they failed on every site collection in the web application
3) they did not fail to another web application even when the site is also a publishing template

So those symptons make me think the problem is at web application level, and maybe publishing features related. For that reason, I deactived and reactived all features at applicaion level, and also tried deactivate/reactivate publishing features at site collection level (even it doesn;t make sense, but you are willing to try everything when desparate, aren't you?). I also made both applications share same application pool. But none of those effort as well as server reboots helped me find any clue.

The ULS viewer showing the following 2 errors:

RunWorkflow: Microsoft.SharePoint.SPException: error compilererror Line="1" Column="1" Text="The root activity type is invalid." /Error
at Microsoft.SharePoint.Workflow.SPNoCodeXomlCompiler.LoadXomlAssembly

Microsoft.SharePoint.SPException: error compilererror Line="1" Column="1" Text="The root activity type is invalid." /Error

Which doesn't really help after spending hours on Bin.com.


Finally another ULS error shed light on me:

Microsoft.SharePoint.SPException: An error occurred creating the configuration section handler for System.Workflow.ComponentModel.WorkflowCompiler/authorizedTypes: Could not load type 'System.Workflow.ComponentModel.Compiler.A uthorizedTypesSectionHandler' from assembly 'System.Workflow.ComponentModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. (C:\inetpub\wwwroot\wss\VirtualDirectories\80\web.config line 20)

The resolution is then very simple: open web.config and fix the typo there.

The question I need to ask myself is, how can I miss this ULS error earlier? The reason turns out to be, this error occurs when a workflow is created, not when the workflow is running.

SSL certificate handshake and web service error: The Underlying Connection Was Closed. Could Not Establish Trust Relationship with Remote Server

In SSL, Client and Server authenticate each other's certificate. The handshake process is illustrated as follows:

Note: It is optional for Server to authenticate Client's certificate. It is configurable in IIS site->SSL Settings. By default it is "ignore". In a rare occasion client certificate is "required", the handshake will fail if  client certificate is not present and you will get "403.7 Forbidden " error in browser. In ASP .Net Web code, some people suggest getting around this by sending client certificate to server, but I have not tried it.

On the other side, Client is always required to authenticate server's certificate. Browser will popup a security alert when  server certificate is not valid/trusted. you can depress the alert by installing the certificate into both "Personal" and "Trusted Root Certificate Authority" locations in client machine's certificate store.

you can check the store by MMC->Add Snap-in->Certificate-> My User(Computer) Account.

Things are different if you call SSL web service from .Net ASP Web. you often get this error:

The Underlying Connection Was Closed. Could Not Establish Trust Relationship with Remote Server

The reason is the computer maintains two different certificate stores:

• The local machine store: An ASP.NET Web application looks in this store to locate client certificates.
• The local user store: An interactive user application looks in this store to locate client certificates.

So, in order to solve above error in .Net code, you will need to import the same server certificate into Local Machine Store on both "Personal" and "Trusted Root Certificate Authority" location. Also you need to grant  the service account running ASP .Net Web has sufficient right to access the Local Machine Store. ( using local admin right is not good idea, but good enough to run a test). See here for details.


Another workaround  which are quite "popular", but bad, is to bypass security handshake totally. See here for details. the killer is System.Net.ServicePointManager.ServerCertificateValidationCallback += delegate { return true; }; 

Another SharePoint 2010 deployment error

If you run into this error when deploying a sharepoint solution from Visual Studio 2010:

Error occurred in deployment step 'Retract Solution': The language-neutral solution package was not found.

The chance is this solution wasn't properly retracted, and SharePoint still thinks it is installed. A simple way to fix is to change solution GUID. The GUID and wsp name is defined in the Package\Package.package, but you need to open this file in text editor such as Notepad (not  in Visual Studio), and restart Visual Studio.

what rights needed to view sharepoint sites associated with TFS Team Projects

Given the scenario that you have TFS integrated with SharePoint and Report Server, each TFS team site can have an associated SharePoint site which display info such as reports and Team projects work items. To view this sharepoint site, what rights need to be granted:

• sharepoint permission
• TFS team project readers right (via TFS Administration Console)
• Report Server "Browser" right (via Report Manager site)

One of very confusing service account required for TFS is Report Reader Account. The MSFT document says :

The report reader account is the identity that is used to gather information for reports

It has nothing to do with Report Server access, it is the account that will be granted access to one of database called TFS_WareHouse. So it is the identity utilized to retrieve data for reports, not access reports themselves.

Redesign an infopath template from sharepoint site

If you publish an infopath template directly to a sharepoint library, you can design (redesign) it later with InfoPath client, but what if you publish the template as a content type or as administrator approved form? you will NOT be able to do the same thing as direct publish by browsing sharepoint library.

Instead, go to Site Settings->Site Content Type Gallery ->select content type->Advanced Settings -> Edit Template:

access denied error with SPD workflow

Lately running into a problem when any user with "designer" permission level failed to save workflow from SPD with an error saying " Server Error: Access Deny". Further, users with designer permission can't manually start this workflow from browser with a similar access deny error. The workflow startup page is one of four files listed for each workflow in SPD, and it is missing from designer view:

Check out and check this file back in, the error goes away.

Implement AJAX & Backend Services

Way to implement Ajax:

• use Sys.Net.WebRequest to call backend service (ASPX, ASMX or ASHX)
•  server side ajax enabled WCF service
• use asp:servicereference to emit script proxy for client script to call WCF service
• client side Ajax Library:  http://microsoftpdc.com/Sessions/FT29
• Ajax Toolkit (server side Ajax controls)

What backend services to call?

WCF or ASMX?

• WCF/REST: Async friendly, abstract complexity (serialization/deserialization etc) web friendly, Binary, format of choice (Json/xml/image etc.), end to end
• Session Http cookies
• SOAP based web service asmx: overhead, xml only, computing distribution

configure ASP.NET impersonation authentication in IIS 7 and forward user credentials

If you just add  in  web.config aiming to configure impersonation for a IIS 7 web site, the site will be broken immediately with a HTTP 500 error.

The reason is that Application Pools in IIS 7  have  "Managed Pipeline Mode" defaulted as "Integrated", under which impersonation doesn't work. Changing from "Integrated" to "Classic" is the fix. If you check all application pools for sharepoint application, they are all set as "Classic".

Since IIS 7 use Kernel Mode for authentication, and if you are using server BIOS name as URL, you get Kerberos authentication for free (see this blog for details). Does this mean, with impersonation in place,  you can forward logon user's credential to next hop? for example, setting credential before calling web service such as:

svc.Credentials = System.Net.CredentialCache.DefaultCredentials; 
 
You still need delegation right for your service account (or machine account?) to make that hop happen. Otherwise you simply forward a empty credential to the web service, and if the web service is not anonymous, the call will fail.
 
 
In case of NTLM authentication, logon users' credential can not be forwarded by any means.

Change SharePoint Application Binding, easier than thought

Recently I need to install Fiddle on one of my test servers to trace down an authentication issue, the server has SSP hosting application running at 8888, which blocks Fiddler. So I want to see how difficult to move this application to a different port.

I know I have to change port number on IIS and  also have to change AAM, but surprisingly that is all needs to be done. SSP picks up automatically its new hosting app and everything runs like a champion.

Browser-enabled InfoPath Form with web services

It is common for InfoPath Form to utilize web services as secondary data sources. Normally there is no problem if InfoPath forms are opened in a InfoPath Client application, but problems occur when InfoPath forms are opened in browser, i,e, using InfoPath Form Service.

There are 2 options to workaround brows-enabled Form with web services: one is to set form "Full Trust":

This option will require Administrator-approved publishing option.

and the other is to convert data sources into data connection:

 This option will allow form trust at Domain level, and with other publishing options available, but this option will require "Cross-Domain Access" checked for InfoPath Service from Central Admin:

For browser-enabled Infopath form, web services are called from sharepoint server, sometimes the network communication problem between sharepoint server and web service server can block services for browser-enabled forms, but could have no impact when InfoPath Client applications open the same form, or have no impact on web service configuration at design time.

SharePoint 2007 List Event ItemAdding fire twice

when  you cancel the event in ItemAdding Handler, like:

public override void ItemAdding(SPItemEventProperties properties)

{
                SPUtility.SendEmail(properties.OpenWeb(), true, true, to, subject, body);           properties.Cancel = true;
}
you will get 2 email messages.
Other senario that list events fire more than once, largely becuase the event handlers are registered more than once. Use SPM to find out, and write code to un-regisgter. http://blogs.msdn.com/b/ketaanhs/archive/2008/04/01/how-to-un-register-an-event-handler-on-a-list-sharepoint-2007-moss.aspx

Beware of SPWeb.GetFolder(URL)

After a call to SPWeb.GetFolder(URL), you better check if it exists by spFolder.Exists(), since the GetFolder() call won't return error even if the folder doesn't exist. What is more, even if it doesn't exist, you can still get some properties (like spFolder.Url etc) out of it. But all of sudden, you get the following error when you try to get Item out of the folder object:

The object specified does not belong to a list. at Microsoft.SharePoint.SPWeb.GetItem(String strUrl, Boolean bFile, Boolean cacheRowsetAndId)


at Microsoft.SharePoint.SPFolder.get_Item()

This seems to me a bug, but is documented in http://msdn.microsoft.com/en-us/library/ms461547.aspx

If the folder does not exist, an SPFolder object is returned, but any attempt to access its properties throws a FileNotFoundException exception.

Create a single Search Scope for SharePoint site, File Shares and People

You might wonder why not just use "All Sites"? A large SharePoint Farm normally includes multiple sharepoint applications with distinct users, users in one application are only interested in finding contents from their own application, not all contents in the farm. Also SharePont search scope generally confuse regular users even though it sounds well technically. So it is often desired to use only single scope which can cover all and only contents for a particular business unit, and those contents normally include file shares and also people in the whole organization.

How to do this?

• First create a content source from Search Service Application for file share
• Create a search scope at Search Service Application Level with a custom "target results page", like  http://yourSharePointSite/SearchCenter/Pages/SearchTransport.aspx
• Add the following rules for the scope created above:
• ContentSource = file shares
• Folder = http://yoursharepointsite/
• contentclass = urn:content-class:SPSPeople

this scope has to be created at service level, rather than site collection level since it includes file shares

•    In Site collection, Search Setting, choose "enable Custom scope" and drop down Mode as:

• In Site Collection, Search Scope, Modify Search Dropdown Display Group to display only one single search scope created in the above step
• Create a Search Result Pages:
• use Search Result Page Layout (only avaiable in Enterprise Search center site template) , which gives you most web parts on the page
• Edit "Search Box" web part as follows:

• Edit "Search Core Results" as follows:

• insert "Refinement Panel" Web part and Customize it. see this blog for details
• insert "People Search Core Results" and modify "cross-web part query id" property

• insert another "Search Paging"web part and modify its "cross-web query id" property
• Modify Tab in search results
• go to http://yourSharePointSite/SearchCenter/SearchResults/AllItems.aspx
• modify "All Site" tab to use search result page created in the above step and also modify Tab name to reflect your scope name.

By now, your single search goal is accomplished, but you might notice that the search center has no branding and navigation, as it is using minimal.master. Don't try to replace its master page as described in this blog.

Some notes on SharePoint 2010 Session State

SharePoint 2010 have Session State disable by default, in contrast, SharePoint 2007 has Session State enabled by default.

why? does not SharePoint 2010 need any session state?

but first, what is the session state? Session state is a part of state management. State management is the information that ASP .Net need before processing page request, i,e, what is the values of controls or variable on the page? Those infomation is normally handled by mechanics such as View State, Query String, Hidden Fields.. Those are called client side mechanics because the data is stored at client side. Session State is different in that it is server side (only session id is sent to client via cookie or query string) and its main purpose is to remember if a request is a new or existing one by session ID.

So why SharePoint 2010 don't need session id? SharePont Form service needs session state (SP State Service) in the scenario of multi-page forms, other than that SharePoint doesn't need to track session ID in general. Turning on Session state can potentially degrade performance as SharePoint does not automatically remove old session state records from the session state database tables, and there is only one session database for sharepoint farmwise. see Todd Carter's blog for cautions while using SharePoint Session.

what is the implication of this new setting in sharepoint 2010? It now becomes recommendation that affinity or Sticky session be set for Load Balancer. See this blog from SharePoint Joe.

data in Managed Meta Column won't update

If you change managed meta data term in the central admin, and you will find the data in Managed Meta Column won;t update until up to one hour. By now,you should be easily guess that it is controlled by a hourly timer job. Yes and its name is Taxonomy Update Scheduler. This did take me some time finding out, so I hope blogging it here can save someone one hour.

Visual Studio 2010 deployment: Error occurred in deployment step 'Activate Features': Feature with Id '320ed40e-cf38-493b-9bc3-7fd3b01e8524' is not installed in this farm, and cannot be added to this scope.

If you get this error while deploying from Visual  Studio 2010, the chances are you have a multiple-servers farm, and Visual Studio can only deploy the solution to its local server.

Debug PowerPivot Errors

1) "Unknown Error" on Server Health:

This is a excel calculation service problem: its service account is not granted a db_owner role in Central Admin Content Database.

 See this post for explanation why this needs to be done.

2) Window Authentication Error:

This error occurs randomly after Excel Server reboot. It is because Claim to Window Token Service is not running, starting it from Service Management Console will make this error go away.



3) Red X Error in the bubble chart web part
check the version of Microsoft® SQL Server® 2008 R2 ADOMD.NET

notice: sometimes the version from control panel is misleading, so it is better to check physical file's version:

To get this version, download SQL 2008 R2 Feature Pack.

4) Red X error on PowerPivot Workbook in PowerPivot Gallery

Disabling loopback on the WFE should resolve this problem, this is best available fix so far. See this KB for details.

5) Window Event Log error on Orphan database associated with PowerPivot Application

This one happens if you created a powerpivot application and deleted it later. Even though the database was deleted, but somehow it still stays in sharepoint metadata. Use PowerShell get-spdatabase to verify and delete it.

6)PowerPivot Workbook Data Refresh "Access Denied" error

this one occurs when

• Unattended Account is not granted "contributor" permission from sharepoint site, or
• SSS application's owner is not set as Farm Account (don't know why....)

Data Refresh is triggered by Timer Job, which first fire request to SSS for unattended account credential, and on success, use the unattended account to open data source. In case datasource is sharepoint resource such as a integrated mode report, unattended account need sharepoint permissoin to do that. Notice, all those happen without PowerPivot service getting involved at all at this point.

7) Data Refresh Error when use SSRS report as datasource:

Errors in the high-level relational engine. The following exception occurred while the managed IDbCommand interface was being used: The remote server returned an error: (403) Forbidden..

This error occurs when powerpivot Analysis Service try to access embedded SSRS report data. This call is made directly to SSRS by PowerPivot Analysis Service. As SSRS needs to check sharepoint permission of caller credential (which credential? when kerberos is implemented, it is PowerPivot Analysis Service Account. see this post for details), it throws the above error when the PowerPivot AS service account doesn't have sharepoint read permission. Notice: in this case, it is PowerPivot Analysis Service Account that will be ultimately passed to SSRS reporting data, not unattended account.

PowerPivot Data Refresh and Excel Refresh

PowerPivot Workbook has 2 kinds of data sources, one is from its Pivot table cache, and the other is from PowerPivot Analysis Cubs. Then what  about Data Refresh? PowerPivot Data Refresh is to refresh Analysis Cubs from external data sources, and as part of this refresh process, the workbook is updated as well.

The credential used for data refresh can be: Unattended account, embedded window credential, or SSS

Notice, there is no window authentication listed above, which means the logon user credential is never used for data refresh. So obviously data in Analysis Service is not security trimmed.

Also notice, it is PowerPivot System Service which retrieves data refresh credential and then sends to Analysis Service, since both PowerPivot and Analysis Service are guaranteed running on the same server, Analysis Service can delegate this credential further to external datasource without need of kerberos delegation.

Not to confuse PowerPivot data refresh with Excel refresh. Excel refresh happens when users click slicer or manually do data refresh. The result is, its cached pivot table get refreshed from .abf file or AS cubs ultimately. PowerPivot functions as Excel data source in this case.

As both Excel Service and PowerPivot service are claim aware, it is claim token that flow from Excel to PowerPivot System service. From PowerPivot to Analysis Service which is not claim aware, it is window token that flows thanks to Claim to Window Token Service. But the most interesting thing is, PowerPivot use PowerPivot Service Application account, not logon user account to connect to Analysis Service:

This is why there is no need to configure security in PowerPivot Analysis Service, but the question is, what is the security enforcement at Analysis Service level even with the note "PowerPivot System Service immediately downgrade the connection"?

the answer is it doesn't need to be, since the data refresh is just to refresh data in AS cubs, and they don't need to be security trimmed. Excel Service will enforce security when it connects to AS Cubs to fetch data.

The follow-up question is, does Excel Service account need kerberos delegation to AS? The answer depends, if Excel Service and Power Pivot run on the same server as they should, there is no hop and Kerberos is not required, if Excel Service and Power Pivot run on different server, Excel Service account should be granted a delegation right to AS and logon users should have read access to AS Cubs if window authentication in Excel workbook is chosen.

Dilema: which account to run SPUserCodeV4?

If you want to run sandbox solution, you have to run "Microsoft SharePoint Foundation Sandboxed Code Service", which is a SharePoint service wrapper for window service "SharePoint 2010 User Code Host", and has no service application associated with it.

By default, this service is run under sharepoint farm account, and results in a warning from SharePoint Health Analyzer.

You can certainly change it to run as a different managed account to suppress the warning, but you should be aware of its ramification: the managed account running this sandbox service will be granted some privileges similar to farm account, such as  db_owner roles for all WSS Content databases! And this role won't be reversed on later service account changes. This certainly violate least privilege rule.

other service accounts' database right and sharepoint permission: issues related with Excel, PerformancePoint and People Search

As recommended by this technet article, running sharepoint 2010 portal application under one service account, and running all other service applications such as Excel, PPS etc under other application pool accounts, if so, does any service application pool account have access to the portal's Content Database? Does any of them even have permission to sharepoint portal site?

Quoted in the same technet article:

Other application pool accounts

The other application pool account must be a domain user account. This account must not be a member of the administrators group on any computer in the server farm.


The following machine-level permission is configured automatically: This account is a member of WSS_WPG.


The following SQL Server and database permissions are configured automatically:
This account is assigned to the db_owner role for the content databases.

This account is assigned to the db_owner role for search databases associated with the Web application.
This account must have read and write access to the associated service application database.
This account is assigned to the WSS_CONTENT_APPLICATION_POOLS role associated with the farm configuration database.
This account is assigned to the WSS_CONTENT_APPLICATION_POOLS role associated with the SharePoint_Admin content database

The one in red is questionable: actually none of application service accounts has been granted db_owner for Content Database! This is the root problem of "The workbook cannot be opened" Error as described in this blog.

The same problem occurs to Performance Point Service, but PPS even has another anomaly: its service account has to be granted a "read" permission for "Data Connections" as described in this blog. Somehow PPS invoke RunWithElevatedPrivilge call to check user's permission on "Data Connections" library, and get access denied error. The error tells us that those service accounts don't even have sharepoint portal access!

Search Service Account and default content Account(crawl account) are generally granted full read access in the web application's "user policy", but even so, it doesn't mean either of them has content database owner right. Also notice that default content account needs to have "Retrieve People Data for Search Crawlers" right on Profile Service Application in order to crawl people profile or sps3://mysite. If you change the default content account, this right needs to be update manually.