Saturday, April 2, 2016

TFS Build Issue: - Builds will be in queued state forever and will not build unless the priority of build is manually set to above normal i.e High or Above Normal.

Hi All,
I want to share my experience in fixing one of the build issue stated above.
TFS Build Issue: - Builds will be in queued state forever and will not build unless the priority of build is manually set to above normal i.e High or Above Normal.

Recently one of our customer started facing issue that the Continuous Integration (CI) build queued forever and was able to build the application only my manually setting the priority of the queued build to High or Above normal. Since this required manual intervention and customer was quite upset that they have to set priority of the build to “High or Above normal” for all CI builds.
When we connected to the customer and checked there were no issues in build server as there were enough Build controller and available agents to handle build requests. We also checked restarting the build service, but it did not resolve the issue.
As we already observed, since build works fine when the priority of the build is changed, we suspected that there may be some old builds which may be in queued state and TFS is waiting for those builds to get completed first.
So to clarify the above doubt we executed below query against the “Tfs_<Collection_Name>” database to know what is the build queue state
select * from tbl_buildQueue
Surprisingly we found that there were many builds in State = 2 i.e. “In progress” or “Queued” State but there were no builds existing for those Build definitions for that Team Projects under that collection, and this was the actual cause of the issue, since TFS still thinks that there are builds yet to be completed and all new builds are set in queued state and build remains in this state forever since there is no way to complete the previous builds as the Build Definitions or Builds would have been deleted.
We can also execute below query to get the build status along with the Build Definition Name
select a.DefinitionId,a.Status,b.DefinitionName from tbl_buildQueue a inner join tbl_BuildDefinition b on a.DefinitionId=b.DefinitionId
To solve this, we executed below query which will change the state of all build runs into completed State i.e. to 16
update tbl_BuildQueue set status=16 
Post to this change CI build started working as expected J 

Hope this post helped.

1 comment:
Write comments