Skip to main content

So...

I have a blank list (I delete all records every night to make it clean for the next day) with just headers. We will call this Report1. I run a report every morning and need to import this data into Report1. We'll call this new daily report Import 1. After I import all data from Import 1, I want to delete Import 1 from the site so that tomorrow I can upload a new Import 1 with the same name (I haven't figured out how to overwrite). This will be for an end user that does not have SharePoint experience so I am trying to make it as seamless as possible and not have to change actions in workflows.

I have a few questions here...

1) On Import 1, I do not have a commonality field to do a lookup off of so I created a column called Import Trigger and put move in that field for every line item. I then created my workflow to query the whole list and store everything in a collection variable. Then run if that collection variable into an List ID. I then did a create item to Report1 by doing a lookup to Import1 when Import Trigger field equals "Move".I am missing something here because it only brought one line item over.

2) At the end of this workflow, I have a delete multiple items action, which works fine, but I want it to delete the whole list. Not just the line contents. In relation to this, every time I delete the Import1 from the site, and upload the new Import1 for the day, the workflow needs updated again manually.

Any tricks or tips to help me through this one?

Thanks!

Hi Josh,

You can use web services to do the operations

http://{site Url}/_vti_bin/NintexWorkflow/Workflow.asmx

And

http://<Site url>/_vti_bin/Lists.asmx


Josh Shutts​ -

This seems very confusing and complex for no reason. You are importing data (which you delete) into a list (which you also delete) and do this everyday.

In regards to your questions:

1) Sounds like you need to loop through your resulting data set and not just the first one in the collection.

2) Why not just import the data directly into the list and purge out the contents each night (if needed). Why delete the entire list and then recreate?


Hey Jesse

Yes, it's a complex daily initiative.

I have tried to copy and paste the data from excel into the list but it's 12000 records and times out after 2 hours of running. Thought a workflow would be the best option.How else can I import into an existing list? Maybe this is what will solve my problem?


Josh,

In order to copy from Excel into a list, you have to copy 99 rows at a time. It gives the system time to process the data. It is the same for a workflow. In a workflow, it is best practices to Commit Pending Data after each insert and have a counter that when it reaches 100 pause a minute or so before updating an item. This takes longer, but it will not time out.


Josh Shutts​ -

Have you looked that the Import Spreadsheet custom list template? It might help you in the import aspect of it, but I do not think it can be automated because there is no Template ID associated to it. You would have to do it manually, but if it is all setup in the excel file as a table, should be easy enough.


I've done that too....but again, it will time out with huge amounts of data. Maybe it could be imported in smaller chunks.


Hello Josh,

What technology are you using for reporting?   It sounds like this might be something that could be done much more quickly and efficiently by using a SQL table along with reporting.   It'll be many times faster than trying to read and write to a SP list.   Because  I don't know your process this is  only a suggestion based on what I can only imagine what might be happening and there may be excellent reasons that  my suggestion isn't a viable solution, but did want to throw it  out there to consider.

Thanks,

Mike


Reply