Implementing native paging support in a K2 service broker

The problem:

I wrote a custom K2 service broker that exposes List methods to retrieve records from a back end system, and return them as results. I had implemented a MaxRecords parameter to be passed to each List method to limit the number of records returned. During testing I noticed that there was a direct relationship between performance, and the number of records being returned. The more records returned, the slower the performance. Another issue with this implementation is that it would be impossible to get to any records beyond the MaxRecords value.

For example, if I set MaxRecords to 250, I would never be able to see record # 251 without increasing the MaxRecords value.

Enter Paging:

I dug around a bit, and found that the ServicePackage property of the ServiceAssemblyBase object exposes some paging properties.

Then a light bulb went off. If I have access to that information, I should be able to use it to grab only a page at a time from the back end system. I could make the back end system do the heavy lifting of getting only a page of data. This was a much better solution than just using the max records which limited the number of records the List method could return at all. 

I coded my broker to read the PageSize and Pagenumber properties from ServiceAssemblyBase.ServicePackage and if they were valid (not set to 0) I used the information to write paged queries against the back end system.  If they were set to 0, would fall back on my MaxRecords implementation.  When I ran some tests, I noticed that even though I enabled paging in K2, and set the page size properly in the UI, these values were always 0 within the broker. Another thing I noticed is that even though my broker was returning say 250 records every time the method was called, in the K2 UI, it showed only a single page. The K2 infrastructure was paging over the 250 records that I returned. When page 2 was requested, the broker returned the same 250 records, and then K2 would show page 2 from that. If I had say millions of records, I could see how much of a performance issue this would be. 

After some more digging around, I found that there’s a magic property that you can set in your broker code to tell K2 that this broker handles its own paging. I found that if I set the HandledPaging property in my broker (on the ServiceAssemblyBase object) to true, the PageSize, and Pagenumber  properties were populated properly, and my code started working, and I was able to get only a page of data from the back end system.  When page 2 was requested, my broker could see that and only get page 2.

Caveats

Sorting

It appears that even though I could now page within my broker, sorting became an issue. I thought perhaps that there’s a property that I could read from the K2 infrastructure from within my broker to determine how the data is to be sorted.  I could then use that information to tell the back end system to sort  the results before returning a page. Unfortunately K2 doesn’t expose that information where a custom service broker would have access to it. K2 apparently sorts the data after it is returned from a broker, so if you implement paging in this way, when a user sorts the data it will only be sorting the data in that particular page.