Skip to main content

Hi All, 

 

 

Wishing everyone a very HAPPY and prosperous new year 2017!

 

 

Background: 
Company hired a partner to automate their paper based system into electronic forms. 

Partner used SharePoint Online and used Nintex forms and workflows. However partner are using a single list with 255 columns and at times forms and workflow shows following error screenshot:

 

 

 

Current: 

Now, company approached us and we are in process of fixing the issue.

 

We found that it that list has some workflow is too slow due to number of columns and workflow at times shows this error.

We raised a  support ticket with Nintex and they asked to export form and import the form to a new list.  While doing so we got the following error:


 
Nintex also concluded that because of large columns company should reduce no of columns. However, the company cant reduce the no of columns as they are important columns.      

We are in process to re-develop the solution. 

 

We are thinking to split the that lists and planning to Nintex form to query other lists within the form

 

Q: Is it possible to have  a Nintex form to query different lists and display those columns within the form

 

Vadim TabakmanFernando HunthFranck Cornu

Hi Sandy,

Happy New Year :)

You could use the List Lookup control on the form to do a query on another list and display the information.

Do you know why they have so many fields on that list?  Is it absolutely necessary?

You can have a bunch of controls on the form that aren't linked to fields and where needed, a workflow can query the data inside the form, rather than having so many fields on the list itself.

Vadim


Thanks ‌ for your quick reply. I am also thinking on the same lines. However, company mentioned that they have different sections which are required.

We are in very initial stage and will make a final decision in 2-3 days. 
But I am very happy that you replied to my query !!  


What Vadim Tabakman‌ wrote is a really good solution. You have to review the columns list and decide if some of them can be considered as dictionary values for example. For such columns you should then create separate lists and use the lookup. This will also makes your workflow/ process data easier to maintain.

Maybe you can also find separable sub-processes in your workflow? Then you could create a separate list for each sub-process and trigger them one after another, using the "Start workflow" action.

Regardm,

Tomasz


Depending on how much the data within the list is going to grow, I would question whether SharePoint is the correct solution for this.

255 columns in a normal database table would be one very big, flat and unwieldy table, put that in SharePoint and it becomes more so.

Sounds to me like what you are now doing by breaking up the list into smaller lists is normalizing it and as we know SharePoint is not a relational database tool.

Have a good think about where this might head in the future. Whilst you might be able to manage the relationships between the tables now, will this be possible if the solution develops further. It might be worth looking at this as a database solution rather than a SharePoint solution.

Also, check out the column type limits here. This is for SharePoint 2013, but I believe similar limitations exist in O365


This is an interesting problem to say the least, because no matter what you do you will tend to run into one size limitation or another - even if you do resort to pulling in data from other places into a single form, your form could crash due to the amount of data.

You might want to consider building a series of forms, connected by a workflow, and then you could use task forms along the way to collect data and stash that data along the way (to either different lists, or to a data base).   A data base table might be ideal, because you could then use a reporting tool that would be better suited to produce output that could be visually reviewed.

SP is great at a lot of things, but there are a few cases that make fitting it to a solution challenging.  

Here is an article that describes the size limitations (pertaining to items) and some other interesting stats:

SharePoint Server 2010 capacity management: Software boundaries and limits 


Hi Mike M‌  Ryan GreenawayTomasz PoszytekVadim Tabakman

Thanks a ton for your valuable inputs.

I had to study this complicated form due to no of columns. 

I found that 67 columns were the primary columns. 
When a drop down item is selected (such as specific Database), other selections within a form with more fields such as SQL Server, Oracle, MySql etc as options  is shown.  

I need to make some more lists with lookup fields for this primary list with 67 columns and make solution more easier to maintain and scalable.

Any suggestion from experts?

Thanks.  
 


I would break down the form into sections/tabs. Each section would be a different list. The first section that is created would have the automated ID that would be used in the other lists to pull the information together. I definitely would still try to reduce any fields that aren't necessary because that is still a lot of fields.


Thanks  burkslm‌ for inputs, 

I found that there are redundant data in one of major section in that huge 255 columns list.

List stores a specific configuration for product.

For e.g. Product A has specific configuration and each configuration has exact same 25 columns (column Color of Prod A: “Green”, Material used: “Laminate Flooring” etc.) Product B has specific configuration and each configuration has 25 columns (column Color of Product B: “Blue”, Material used: “Hardwood Flooring” etc.)

I am making 2 lists to separate.

a) One list for Products with one column. For e.g. Product list with 4 products. Product A, B, C and D.

b) Second list with all Products Configuration and Product lookup.


However, in the Main form (Project Name, Customer Name, Product Category etc.)  I need to present all the Products and their related configuration to end users.

For e.g. Two entries in current form  looks this in Main form:  
1) Project Name: Riverside View Condo , Customer Name: John Smith , Product Category (Drop Down): Duplex house, Product: A,  Color: Green, Material: Laminate Flooring etc.

2) Project Name: Skyline Condo , Customer Name: Alex Yew , Product Category (Drop Down): Penthouse , Product: D,  Color: Blue, Material: Hardwood Flooring etc.  

I am thinking to use concatenate “Products” and “Products Configuration in the main  form.  For e.g.  Material_Hardwood_Flooring_ProductA in the main form.

Using this technique I will be significantly reduce the number of columns.

Any suggestions or recommendations would be greatly appreciated. 

Hi Guys,

 

The environment is Office 365 and I guess there is no way to touch the access the database. #greenawayr  The only option is have some kind of “relational database” using a couple of lists with list lookup. The solution design is pretty bad by previous partner and we have to re-design the form.


Current list has 255 columns and I found that there are redundant columns which could be reduced.

Current list has linked to 8 simple approval workflows that run in sequence. Mike M  you suggested to have built a series of forms.  The similar advise is given by Sean Feeney  Nintex video as well in terms of optimizing the performance of the Nintex form.  List lookup limit is 8 for a list.  The form should NOT be larger than 500K in size.

  1. Sean suggested if have a very lengthy form and break into individual task forms & then collect that data. Data can be viewed by others to see at different portion of workflow.
  2. If it’s a single user, he further explains that although its large a FORM, it’s a right way to build the form.

  3. If there are certain portion that has to be filled by individuals, then we need integrate forms with workflow


In our new form re-design, staff members will be large part of form with 67 columns. So I will choose 3 option. I can’t reduce the columns further as these information are key columns for the Form and then they go for further approvals for different departments such as

Project office manager: with exact same 25 columns. Previous partner has named columns approver comment, Database etc. as Approver_Comment_A, Database_A etc.  

Requestor Review: with exact same 25 columns. Previous partner has named columns approver comment, Database etc. as Approver_Comment_B, Database_B  

Program manager: with exact same 25 columns. Previous partner has named columns approver comment, Database etc. as Approver_Comment_C, Database_C etc.

Closure approval: : with exact same 25 columns. Previous partner has named columns approver comment, Database etc. as Approver_Comment_D, Database_D etc. 

  I want to use a main list with 67 columns to be main form and other lists (don’t finalized the list names yet). The reporting can be used.

I will planning to use 4 new lists like

  • Project office manager with 25 columns with lookup within the main list.
  • Requestor Review with 25 columns with lookup within the main list.
  • Program manager with 25 columns with lookup within the main list.
  • Closure approval: with 25 columns with lookup within the main list.

Q: Is above list design okay or you experts have better advice on list design?     
Q: How do I create task forms for new lists? Any walk through would beneficial

Thanks a ton for your help in advance. 


‌ 

I need your expert comments on this. 


Hi Sandy, 

To offer a quick reply for question two, you design the task forms on the task actions themselves from within the workflow.   Assign a Task for example will allow you to edit the associated Nintex form from within its configuration.   So unlike a list form where you are editing the form directly on top of the list, you will instead edit the Nintex form from within the workflow itself, on the Assign a Task action itself.  Let me know if you need further instruction.

As for the overall architecture, I'd have to take a closer look and examine it a bit more in depth.   I've got to run at the minute but hopefully I can get back to this, if not someone else may chime in.

Thanks,
Mike


Mike M‌ 

Thanks for your valuable insights.

  1.  "So unlike a list form where you are editing the form directly on top of the list, you will instead edit the Nintex form from within the workflow itself, on the Assign a Task action itself.  Let me know if you need further instruction."


    I am not able to visualize the approach. Is it possible to for some simple instructions for me? Sorry about that.  

  2. Mike M‌ primarily, I would be have more lists in my design. The project has been already started. I came with the following lists for far:
    1. Main List (with 67 Columns)
    2. Database (25 columns and lookup to the main list)
    3. Project office manager with 25 columns and lookup to the main list.
    4. Requester Review with 25 columns and lookup to the the main list.
    5. Program manager with 25 columns and lookup to the the main list.
    6. Closure approval: with 25 columns with lookup within the main list.

Any suggestions would be highly appreciated. 

Vadim TabakmanTomasz PoszytekburkslmRyan Greenaway


Hi guys,

 

I was talking to a senior engineer who has in Nintex more experience than me today. He suggested to use repeating section for this kind of scenario. So, I will have   following form design

 

  1. Main List (with 67 Columns)
  2. Repeating section Database column with 25 controls. 
  3. Repeating section Project office manager column with 25 controls
  4. Repeating section  Requester Review column with 25 controls.
  5. Repeating section Program manager  column with 25 controls
  6. Repeating section Closure approval  column with 25 controls.

Please let me this approach with repeating sections with 72 columns within a Nintex form.



Thank you!!


You'll need to think about how much data will get entered into these repeating sections. We had a form which contained a large amount of data in repeating sections and we experienced performance issues to the extent where we had to increase a timeout limit in the web.config which is definitely not ideal.


Ryan Greenaway‌ Thanks a ton for your insight and about performance. Company is using SharePoint Online.  Last partner had exactly the same performance issue due to number of columns. its MY FIRST Nintex project and want to deliver the project successfully. 

"You can have a bunch of controls on the form that aren't linked to fields and where needed, a workflow can query the data inside the form, rather than having so many fields on the list itself"

Vadim Tabakman‌ Do you mean I use Repeating section and not linked to fields? Can you please advise if using splitting lists and using repeating sections for could be potential solution??

Please advice experts!.    


What Ryan mentioned about repeating section size is correct.   I just ran a test the other day to see how much data it took in order to get a form to fail and so I placed 4 text boxes inside a repeating section and it handled 200 rows OK (though it slowed down a bit).  Then I placed another repeating section on the form with a choice, two check boxes, and a yes/no control and gave them all default values for 300 rows - between the two repeating sections that was 500 rows and the form completely crashed and would not load.    I know this isn't scientific benchmarking as the kinds of controls and data within them will affect this number but I thought it was interesting non the less.

Just as an aside, when your controls are inside of a repeating section they do not hook up to a backing list column.   You will need to parse the data out of the repeating sections in order to retrieve the individual values from the XML that is stored.   This is not to say that it isn't the right approach, only wanted to make you aware of this fact.

Thanks,

Mike


We had more controls in our repeating sections and far fewer rows required to crash it.


That would do the trick the as well.   I've seen an on-prem one crash with 17 rows with quite a few controls so good point.


Hi experts, 

I am deep diving the current Nintex form (developed by past partner) which as more than 250+ columns and loading time is around 30 seconds.

 

 

There are 7 sections in the farm.

 

  1. General: 8 columns (Request Number is main column, Requested Date, Requested by (Current user), Current Date etc. )
  2. Project Info: 16 columns (Project description, Project Category, Project Category etc. )
  3. Commercial Info: 19 columns (Product Life, Yearly Quantity, Customer Information etc. )
  4. Supply Chain Info: 5 Columns (Delivery type, customer location etc. )+ Base Option (25 Columns, lighting, connectivity, database etc. )

 

  1. Product Specifications: 4 Columns (Battery type, battery size etc.)
  2. Other Info: 5 Columns (Certification, Protocol etc. )
  3. Requestor Comments: 2 Columns

The minimum columns for the form are 84 columns.


However, Supply Chain Info section has 5 columns and base information product with 25 columns. Columns types includes 5 radio buttons, 7 drop down lists, 2 checkboxes and remaining are single line of text.

User also have different options within the form and can select Option A, B and C. All the options have exactly same information as base option with 25 columns as base information with(5 radio buttons, 7 drop down lists, 2 checkboxes and remaining are single line of text)



Past partner has created columns like this:


Option A (25 Columns)

Display_A

Database_A

Protocol_A etc.

Option B (25 Columns)

Display_B

Database_B

Protocol_B etc.

Option C (25 Columns)

Display_C

Database_C

Protocol_C etc.



Furthermore, if a user select Option A and selects a specific database, then that database itself has 30 more columns subsequent configuration about the product.

For e.g.

User select Option A from the Supply Chain Info  section>> Selects few items within form  >> Selects a Database named “ABC” from dropdown >> Nintex form shows 30 more columns for that database.      

 

Approvals are done by different users across the globe in an approval chain and there are 5 approvals. Each approver has 6 columns (Date time, radio button, 2 People and Groups, 2 multiple line).

They are indicated Part 2 Approvers, Part 3 Approvers etc. within the form for each 7 sections as described above.  

So 30 more columns for approval sections as well

This is how the list is designed by last partner. I need to make the form more manageable so that current customer does face the error with improved performance.

This is my design:

  1. Split the lists.
  2. Main list with Request Number, Requester etc.
  3. I guess splitting the bigger sections such as Supply Chain Info, Project specification (Option A, Option B, Option C)
  4. Splitting Project Category and Commercial Info. Use lookups for main list.
  5. Separate list for Database as might can reach 90 columns if use selects all the Options A, B, and C.
  6. Still considering which form I can use Repeating section.   


Any guidance in the better approach of splitting the lists/repeating section would be highly appreciated.


Hi Experts ..

 

Having 25 controls (5 Radio Button, 7 Single Line, 10 Drop down, 3 Checkboxes etc )  in a repeating section will slow the Form's performance?

With design this could be maximum 4 repeating sections i.e. 100 controls in 4 repeating section wthin a single form. Not all repeating sections will be used and dependent on product design

Anyone has experience in these large forms or any advice to manage lists or repeating section?

Thanks in advance. 

Mike MRyan Greenaway


Those particular controls won't be particularly resource hungry, but lookups and people pickers will be more so.

It does sound like a form that could hit performance issues.

‌ you created a troublesome form didn't you? Anything more to add to this?


Thanks a ton Ryan Greenaway‌ 

Paul Crawford‌ can you please advice on this? 

I came up with this list design 

Lists Design

 

List Names

Columns

Main List

8 Columns

Project Information and lookup to main list

16 Columns

Commercial Information and lookup to main list

19 Columns

Supplier Chain Information and lookup to main list

5 Columns

Options and lookup to main list

·         4 Columns  (Basic Option, Option A, Option B, Option C).

·         Using repeated sections.

·         Each Repeating section will have  30 controls

·         Not all the Repeated sections will be used.

Database and lookup to main list

30 Columns

 Any suggestions/advice this list design? 
Ryan Greenaway Mike M Tomasz Poszytek Paul Crawford Vadim Tabakman


‌  

Can you please advice on the list designs that I have prepared? 

Which is the best approach to handle the kind of form? 

Thanks!


burkslm‌ 

Thanks for your wonderful idea. That's my initial design to split the lists.  I came up with following list design

List Names

Columns

Main List (General Information, Request Number is main column, Requested Date, Requested by (Current user), Current Date etc. )

8 Columns

Project Information and lookup to main list

16 Columns

Commercial Information and lookup to main list

19 Columns

Supplier Chain Information

5 Columns

Options and lookup to main list

      • 4 Columns  (Basic Option, Option A, Option B, Option C).
      • Using repeated sections.
      • Each Repeating section will have  25 controls
      • Not all the Repeated sections will be used.

Database and lookup to main list

30 Columns

Other’s Information and lookup to main list

6 columns

 

In their paper form, it’s a 6 page and when a user checks the specific options such as Option A (each one has exactly same configuration ), Option B and Option C. Past partner created 25 columns each for options that amounts 75 columns.

That’s why I am planning to use repeating section and get just 4 columns instead of 75 columns.

Can you please suggest, how could I use 
automated ID which you suggested for all the lists? In my design each list will have its own ID.    

Thanks!!


Mike MRyan Greenaway

Is it possible to save Nintex Form data rather than custom listI will be saving minimum within list such as Requestor Date, Approved by fields etc and store max info inside the Nintex form.  
How it can be done?

Any guidance will be greatly appreciated.  


Reply