cancel
Showing results for 
Search instead for 
Did you mean: 
Workflow Hero

version update takes forever

Hi

I have done 2 updates to the latest Jan 2016 version of Forms and Workflow.  One went through without incident - probably 20 minutes overalll - on our local dev server.

The other, at a client, took forever.  In fact the GUI installer never finished - I killed the task after 14 hours.

I then ran the installation again with the PowerShell scripts and it did eventually finish, but still took an enormously long time.  I could not time the finish, but it was more than 4.5 hours.

Unfortunately, I could not find anything that showed me what it was doing whilst it was 'busy', but from the script output, I can see some indications of which stages took so long.

Note that the start time here was around 11:45 am.

Detected SharePoint 2013 is installed

Stopping Live Relay on server XXXXXX-SPAP.

Stopping Live Relay on server XXXXXX-SPFE.

Resetting Server(s):

        XXXXXX-SPAP

        XXXXXX-SPFE

Stopping Timer service on server XXXXXX-SPAP.

Starting Timer service on server XXXXXX-SPAP.

Stopping Timer service on server XXXXXX-SPFE.

Starting Timer service on server XXXXXX-SPFE.

Updating Solution Core Components

Upgrade job for solution Core Components (NintexWorkflow2013Core.wsp) is not created yet (time waiting=3 seconds)

Upgrade job for solution Core Components (NintexWorkflow2013Core.wsp) in server XXXXXX-SPAP is not finished yet

Upgrade job for solution Core Components (NintexWorkflow2013Core.wsp) in server XXXXXX-SPFE is not finished yet

Upgrade timer job for solution: Core Components (NintexWorkflow2013Core.wsp) in server XXXXXX-SPAP finished successfu

lly at: 02/06/2016 13:12:54

Upgrade timer job for solution: Core Components (NintexWorkflow2013Core.wsp) in server XXXXXX-SPFE finished successfu

lly at: 02/06/2016 13:12:09

Updating Solution Web Front End Components

Upgrade job for solution Web Front End Components (NintexWorkflow2013.wsp) is not created yet (time waiting=3 seconds)

Upgrade job for solution Web Front End Components (NintexWorkflow2013.wsp) in server XXXXXX-SPFE is not finished yet

Upgrade timer job for solution: Web Front End Components (NintexWorkflow2013.wsp) in server XXXXXX-SPFE finished succ

essfully at: 02/06/2016 14:43:26

Updating Solution Enterprise Features

Installing Solution Backwards Compatibility UI

Installing Solution Backwards Compatibility Enterprise Features

Restarting SharePoint Timer Services.

Stopping Timer service on server XXXXXX-SPAP.

Starting Timer service on server XXXXXX-SPAP.

Stopping Timer service on server XXXXXX-SPFE.

Starting Timer service on server XXXXXX-SPFE.

Already installed Certificate: Baltimore CyberTrust Root.crt

Already installed Certificate: GTE CyberTrust Global Root.cer

Already installed Certificate: Microsoft Internet Authority.cer

Already installed Certificate: Microsoft Secure Server Authority.cer

Already installed Certificate: Thawte Primary Root CA.cer

Already installed Certificate: Thawte SSL CA.cer

Already installed Certificate: Thawte SSL CA_SHA2.cer

Updating Solution Nintex Live

      ReInstalling version 3.0.3.2

Resetting Server(s):

        XXXXXX-SPAP

        XXXXXX-SPFE

Upgrade job for solution Nintex Live (NintexLiveCore.wsp) is not created yet (time waiting=3 seconds)

Upgrade job for solution Nintex Live (NintexLiveCore.wsp) in server XXXXXX-SPAP is not finished yet

Upgrade job for solution Nintex Live (NintexLiveCore.wsp) in server XXXXXX-SPFE is not finished yet

Upgrade timer job for solution: Nintex Live (NintexLiveCore.wsp) in server XXXXXX-SPAP finished successfully at: 02/0

6/2016 16:20:49

Upgrade timer job for solution: Nintex Live (NintexLiveCore.wsp) in server XXXXXX-SPFE finished successfully at: 02/0

6/2016 16:20:48

WARNING: Existing version of Nintex Live detected, an Upgrade was performed

Stopping Timer service on server XXXXXX-SPAP.

Starting Timer service on server XXXXXX-SPAP.

Stopping Timer service on server XXXXXX-SPFE.

Starting Timer service on server XXXXXX-SPFE.

DisplayName                    Id                                       CompatibilityLevel   Scope

-----------                    --                                       ------------------   -----

NintexLiveAdminLinks           29e9a673-31a4-46a3-b0d2-d8e1db1dbd92     14                   Web

Installing Nintex Workflow Start Service on: XXXXXX-SPAP

Installing Nintex Workflow Start Service on: XXXXXX-SPFE

Installing Nintex Live Workflow Queue Service on: XXXXXX-SPAP

Installing Nintex Live Workflow Queue Service on: XXXXXX-SPFE

Resetting Server(s):

        XXXXXX-SPAP

        XXXXXX-SPFE

The customer has three Nintex Content databases, with sizes of 5GB, 2GB and 200MB.

My questions are:

  • is this an expected duration for an update on a live site?
  • is there a way to arrange things so that it can go through quicker?
  • is there a safe point, before the completion, at which the customer may start using the product?
0 Kudos
Reply
10 Replies
Workflow Hero

Re: version update takes forever

Hi andrew,

that is an extremely long time and I wouldn't expect it to take that long.

Were you upgrading from a very old version?

I would highly recommend you reach out to the Nintex support team, since they may have some steps on another way to install the latest build.

Vadim

Accept as Solution Reply
Workflow Hero

Re: version update takes forever

Hi Vadim

Not an old version, they had previously been updated around the middle of 2015.  I have taken your advice and forwarded my questions on to the support team.

Thanks for the response.

0 Kudos
Accept as Solution Reply
Not applicable

Re: version update takes forever

Hi, andrew van Renen​, Please let us know how it worked out.

Good luck!

Frank

Your community manager

0 Kudos
Accept as Solution Reply
Workflow Hero

Re: version update takes forever

May be a network issue (bad route or server not in the same VLAN for example).

What was the performance of the servers during installarion (memory, CPU)? Not a process which took long time or ressources?

Accept as Solution Reply
Workflow Hero

Re: version update takes forever

Network issues may be a factor - it is a hosted environment and the hosters don't take much care over environment (just a small customers among many larger ones).

The servers had decent RAM (32GB) and 4 CPUs and no real user activity at the time.  I have no visibility into what might have been happening on the underlying host servers (its all HyperV based), or the SAN, so there may have been something there too (backups even?)

It would be nice to have better visibility into what was happening during the long-running sections of the update, though.

Support referred me to this document Troubleshooting Nintex Deployments​, and while the points there are valid, they are quite general.

Accept as Solution Reply
Workflow Hero

Re: version update takes forever

hi andrew van Renen​,

If you turn on Nintex WF logging to 'verbose' in Central Admin -> Monitoring-> Configure Diagnostic Logging, then view them with ULS viewer while it installed. This could give you some insight.

Other option could be clearing your configuration cache in SharePoint and/or taking a look in event viewer of the servers.

Hope this helps.

-Jesse

Accept as Solution Reply
Not applicable

Re: version update takes forever

Andrew,

The Nintex installer is handled by a combination of Powershell and SharePoint. Most of the Powershell used in the installer creates/updates services and deploys solutions. All the heavy lifting is handled by SharePoint via the deployment timer job.

As you can see from the install logs you have provided, the process that is taking the longest is solution deployment piece of the installer. This process is handled exclusively by SharePoint. Microsoft writes a very clear overview of what happens during deployment of solutions found here: Installation and Deployment of a Farm Solution in SharePoint 2010

Particularly this piece:

Details of the Deployment Step

https://msdn.microsoft.com/library/aa544500(office.14).aspx#Anchor_2

The deployment step for a farm solution creates a timer job. This timer job is used by the timer service on each web server in the server farm. The timer job also uses the SharePoint Foundation Administrative web service to access appropriate privileges to deploy solution files to each computer, so both services must be running on all servers for the deployment to succeed.

Initially, the package manifest is parsed to find assemblies, application pages, JavaScript, and other files that are not part of a Feature. These are copied to the locations specified in the manifest. All files contained within a Feature are copied to the Feature directory, a subdirectory of %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\FEATURES. After solution files are copied to the target computers, a configuration reset is scheduled for all front-end web servers; the reset then deploys the files and restarts Internet Information Services (IIS). Farm administrators can specify when this occurs.

Finally, farm solution Features are registered, and schema and definition files are committed to the configuration store.

Farm administrators can choose to deploy a solution on only some web applications in the farm.

This article also provides you with ways to troubleshoot and research failures during solution deployment:

Installation Failures

https://msdn.microsoft.com/library/aa544500(office.14).aspx#Anchor_4

During installation to the front-end web servers, the following failures can occur:

  • If the timer service is not activated on a front-end web server, the deployment job remains stopped. On the pending jobs page in the user interface, the job appears as pending but not being serviced. The administrator must either fix the timer service or cancel the deployment job.
  • If the SharePoint Foundation Administrative service is not activated on a particular computer, an error code is set in the SPRunningJob object that marks the stage as failed and prevents any further operation. The failed deployment is converted to an administrative alert that notifies the administrator that the job failed due to a SharePoint Foundation Administrative web service that was not running.
  • If the extraction of a solution package (.wsp) fails on any particular server, the stage is marked as having failed and processing stops. "Error" will appear in red in the Status field for the solution in the Solution Management page of Central Administration.
  • If one or more files fail to copy—for example, an existing file is marked as read-only—the stage is marked as failed, and processing stops.
  • If the deployment stage code causes an exception, an administrative alert is created with the exception and deployment stops. The underlying job definitions are removed.
  • If an external failure occurs (for example, a power outage), final deployment stops, but can then be rerun.

Also for those who are curious you can see what happens when we upgrade solutions:

Upgrading a Farm Solution in SharePoint 2010

Let me know if this helps answer your questions.

Cheers,

Andrew Beals

Accept as Solution Reply
Workflow Hero

Re: version update takes forever

Hi Andrew

That is a very comprehensive answer, thank you.  It maybe does not directly answer the question, but I will know where to look next time to see what is actually going on,

Regards,

Andrew

0 Kudos
Accept as Solution Reply
Not applicable

Re: version update takes forever

Andrew,

Always glad to help.

Unfortunately, there's not much we can really do to troubleshoot your install after its already happened outside of running it again and cranking up ULS logging to verbose to watch what SharePoint is doing during the deployment.

However, once you start to troubleshoot install issues as a solution deployment error and not a 3rd party installer things become a bit more manageable.

Good luck out there and remember knowing is half the battle!

Cheers,

Andrew Beals

0 Kudos
Accept as Solution Reply