The service did not respond to the start or control request in a timely fashion.

When creating a new FS farm or joining a new node to an existing farm, i.e. running FSCONFIG.EXE or FSCONFIGWIZARD.EXE, or configuring an FS-P, i.e. running FSPCONFIGWIZARD.EXE, the process might fail with the resultant error being that the service did not respond to the start or control request in a timely fashion.

I have a screenshot from a simple script that is just a wrapper around FSCONFIG.  The output is from FSCONFIGs verbose switch.

clip_image002

As you can see everything succeeded except the service didn’t start quickly enough so the overall process fails.  If you try to start the the service it will start and all appears to be well.

The first couple of times I saw this issue was on VMs on my laptop.  I was writing a deployment guide and simply put it down to my machine being not exactly enterprise production class (five VMs, twenty instances of IE, a bunch of Word and Visio instances, Spotify, OneNote, etc. –you get the picture).  However I’ve now hit it in two different customer environments. 

Looks like the reason is related to the .NET Authenticode signature verification.  I’m not going to pretend I understand what all of this means to the CLR but after reading the remarks in the Publisher class documentation a couple of times the following seems to hold true:

  • Default behaviour within the .NET runtime is that code access security (CAS) does not check for Publisher evidence, therefore if you are not using a custom code group based on the PublisherMembershipCondition class, which AD FS binaries do not, disable Authenticode signature verification by configuring the runtime to not provide Publisher evidence for CAS and increase start up time.

How do we disable this?  Using the <generatePublisherEvidence> element of the Runtime settings schema (according to the aforementioned MSDN link).

This is actually already documented on the TechNet wiki.  Although for whatever reason I never stumbled upon this wiki article when searching for my error, hence this post.  I also feel I have a little more info. on the choices available to you –asking around my immediate community of AD FS folk and one or two .NET folk and it seems that the general consensus is that the better of the two options presented (turn off Authenticode signature verification or increase the Service Control Manager (SCM) timeout) is to disable Authenticode signature verification.  Obviously with this advice we assume that these are dedicated boxes, i.e. your FS is only an FS and not running a bunch of other managed code that might actually require the Publisher evidence!

And on this same subject a question I’ve fielded from two different customers is whether or not we implement this throughout the farm.  Your opinion on this matter might differ to mine but I like things to be consistent.  If I’m implementing a 2 x 2 farm (2 FS, 2 FS-P), for example, and I need to enable this setting on one node all nodes get it even if they succeeded.  I hate the idea of having multiple nodes in a farm with different configurations in place (other than those mandated on you like FIMs single point of failure -EWS polling- for example).

At this point my post is pretty much complete, however, even though I’ve liked you to the TechNet wiki I might as well define it here for completeness too.  To turn off Authenticode signature verification add the generatePublisherEvidence element with a property of enabled = FALSE to the .NET Framework runtime’s machine.config file:

Open \Windows\Microsoft.NET\Framework\v2.0.50727\CONFIG\machine.config from an elevated NOTEPAD and paste the following:

<configuration>
	<runtime>
		<generatepublisherevidence enabled="false" />
	</runtime>
</configuration>

In both of my cases there was an empty <runtime /> element that needs to be replaced by the above XML.  It was half way down underneath </configProtectedData> and above <connectionStrings>.

Advertisements

About Paul Williams

IT consultant working for Microsoft specialising in Identity Management and Directory Services.
This entry was posted in AD FS and tagged , , , , , , . Bookmark the permalink.

5 Responses to The service did not respond to the start or control request in a timely fashion.

  1. Paul, I’m currently setting up some ADFS farms (2node) and I also stumbled upon this in one of the environments. I myself am considering to actually allow servers/service accounts through the proxy towards these CRL urls. Wouldn’t that be a more long term solution? I think more and more products will suffer this kind of issues and in the end you’ll end up with an environment which is patched together with this kind of workarounds… Or don’t you see allowing them through a proxy as a viable solution?

    • I agree that there are scenarios whereby allowing the FS nodes to communicate on the Internet via proxy is desirable, i.e. establishing trust via metadata and the subsequent polling and management via metadata. I have a draft post on this very subject that is awaiting some results from debugging an issue with a proxy at a customer site.

      For this particular issue I don’t see it as a solution however as the AD FS assemblies never create a demand for the Authenticode signature, so I think it’s better to simply turn it off…

      I’m obviously interested in any and all arguments to the contrary.

  2. Mark Taylor says:

    I experienced the same issue of the service failing to start after creating a SQL based farm via FsConfig.exe CreateSQLFarm …

    After reading this blog and a few other sources pointing to authenticode issues (likely all stemming from the same CSS escalation) we had an internal debate about the validity of the cause with the prevailing belief being that failure to validate authenticode signing would be encountered at installation time as opposed to starting the service. Instead it was the belief of the group that a CRL lookup against the issuing root cert for our SSL cert was occurring and failing due to the restricted internet access in our environment.

    Netmon captures revealed that our STS server was indeed attempting to request HTTP resources from a remote location, bolstering the assumption that a CRL lookup was occuring. Enabling CAPI2 logging on our STS (Server 2008R2) gave us a URL for the desired CRL file. I browsed to the CRL from an internet facing workstation, downloaded it, and then installed it to the local store on our STS server manually. Once that was complete, the service started after a slight delay (it still attempts a fresh CRL update, then fails back to the cert store.) Upon confirming this worked, we performed a clean uninstall of ADFS, followed by reinstalling and recreating the farm to ensure everything was properly configured.

    Drawbacks to this workaround mean that service admins will need to establish a regular process of manually updating the CRL for their root issuing cert. However it does not require the modification of the authenticode service which could lead to other undesired issues.

    • Excellent comment Mark!

      I assume that the validity period of the CRL is weeks or months as opposed to days?

      Have you considered modifying the WinHTTP settings such that the FS nodes can access the Internet via your proxy? Or enabling a firewall rule that allows the nodes to bypass the proxy for the specific CRL location(s)? Or is Internet access from a production server never going to happen and not worth considering?

      I think I’m going to try and do some digging to see if there are alternative approaches to CRL validation in non-Internet environments.

      Re. the authenticode signature check – I asked a similar question of some colleagues however I heedlessly deferred to the advice on MSDN! 🙂

      • Mark Taylor says:

        The life of the CRL is contained within itself. In my case it was 3 months.

        In another environment where I can directly modify the firewalls specific outbound access would be a consideration. In my case, each change requires a certain amount of bureaucracy so we just open it directly although using a proxy such as ISA is a consideration.

        Lastly, you can turn off CRL verification in ADFS but that’s not exactly best practices.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s