canRetrieveMoreRows / loadNextOffsetPage not working as expected

Hi Kartik,

It looks like there are 2 issues here and I’ll try to explain them both.

1. The first issue is that you’re not getting all 12201 records. Under the hood, Skuid is using Salesforce’s StandardSetController to handle loading data when you use loadNextOffsetPage().  As you can see here, they only support loading in up to 10,000 rows.

2. The second issue is the inconsistent results.  I’ve seen this before and it seems to be a quirk with Salesforce’s StandardSetController.  When Skuid paginates through the StandardSetController, sometimes the StandardSetController returns rows that it had already returned on the previous page.  Skuid models are smart enough to not let 2 records with the same id into them, but that’s what is causing the less than expected numbers for your models.  One way that I’ve found to work around this issue is to put a good explicit order by on your model.  If your model had no order by originally, order by something like “SystemModStamp, Id”.  If your model already had an order by on it, try adding “Id” to the end of your order by statement.  I have found this to fix the issue.

Just out of curiosity, are you using Javascript to call loadNextOffsetPage() after each chunk is returned until you get the full data set?  Unfortunately, because of Salesforce limitations I don’t think you could get more than 10,000 rows this way.  One thing I have thought about though is changing a condition on your model such that you could divide your full data set into segments that would be less than 10,000 rows each.  For example you could get the accounts that start with the letter A-H first, then when you finished that query, go ahead and get I-Z.  This way you would work around Salesforce’s limitations.

Hi Ben,

Thanks for your response. I am using Javascript to call loadNextOffsetPage() until the full dataset is loaded, for the sole purpose of performing an export of the entire dataset. It’s good to know that it’s a Salesforce limitation that’s limiting the number of records returned to 10000.


The second issue turned out to have two underlying problems. The first was the one you mentioned, and adding Id at the end of the order by seems to resolve that issue.

The other part was that the code I wrote increased the recordsLimit to 2000 (from 250) before loading the next offset page. I expected that that the StandardSetController would return records 251-2250 on the first load, when it actually loaded in rows 2001-4000. I altered my code to run updateData() once before loading in any of the offset pages and that resolved the problem.

Thanks for the help!
Kartik

Glad to know you got this worked out… 

I do not understand the “second issue” reported and resolved here. Running skuid 11.2.11 in early 2019 we found a model retrieving 20 rows on initial load, ordering by a custom date field (Note_Date__c). These settings are defined in the skuid page. The last record displayed on the page is from 1/4/2019. When we load 20 more records, the next record is also from 1/4/2019.

There should be a third record from 1/4/2019. It is not there.

The page has a date-range filter on Note_Date__c. If we set the filter to 1/3/2019 through 1/5/2019 – and this is the only change – All three of the 1/4/2019 records are displayed!

Furthermore, if I go to the developer tools in the browser and grab the soql for the unfiltered model (skuid.$M(“Notes”).soql), and run the soql in the Salesforce developer console, 29 rows are retrieved (including three dated 1/4/2019.)  With the browser developer tools, skuid.$M(“Notes”).data.length reads 28.

Why?

I did add Id to the order by setting on the model, and the problem went away.