6.4 Query Paging

SData provides a paging mechanism so that a service consumer can retrieve large collections without using too much memory and without timing out. Two paging modes are supported:

A sequential mode where the consumer can move forwards and backwards through the pages
An indexed mode where the consumer can directly request a specific page.

Sequential paging

The sequential mode relies on four feed level links: first, last, previous and next:

<link rel="first" type="application/atom+xml; type=feed" title="First Page" 
      href="http://www.example.com/sdata/myApp/myContract/-/salesOrders?startIndex=1&amp;count=10" />
<link rel="last" type="application/atom+xml; type=feed" title="Last Page" 
      href="http://www.example.com/sdata/myApp/myContract/-/salesOrders?startIndex=31461&amp;count=10" />
<link rel="previous" type="application/atom+xml; type=feed" title="Previous Page" 
      href="http://www.example.com/sdata/myApp/myContract/-/salesOrders?startIndex=11&amp;count=10" />
<link rel="next" type="application/atom+xml; type=feed" title="Next Page" 
      href="http://www.example.com/sdata/myApp/myContract/-/salesOrders?startIndex=31&amp;count=10" />

The consumer can iterate through the pages by following the next link until it reaches a page that does not have any next link. It can also move backwards by following the previous link. The first and last links allow the consumer to obtain the first and last page, as expected.

In the href attribute value, the query parameters are separated by & rather than &. This is because the ampersand character is a special character in XML and it needs to be specially marked in this context. Also, the URLs returned by the provider SHOULD have their non-ASCII characters encoded, as described in the Internationalized URLs section.

Indexed paging

The second mode is the indexed mode where the consumer requests a specific page. This mode relies on the startIndex and count query parameters. For example, the following query returns elements 21 to 30 of the salesOrders collection:

GET /sdata/myApp/myContract/-/salesOrders?startIndex=21&count=10

The returned feed contains information that the feed consumer can use to update a progress bar or set up paging links, if this is an interactive interface:

<opensearch:totalResults>31465</opensearch:totalResults>
<opensearch:startIndex>21</opensearch:startIndex>
<opensearch:itemsPerPage>10</opensearch:itemsPerPage>

The returned itemsPerPage value MAY be different from the value of the count parameter passed in the request. For example, the provider may impose a limit on the page size. If you request count=1000, the provider may only return 200 entries. In this case, the provider MUST set itemsPerPage to 200.

When the consumer does not specify the count parameter, the provider SHOULD still page its response using its default page size. As a general rule, the service SHOULD always page and impose a reasonable limit on page size to avoid timeouts and memory problems.

URL variants for paging

In the previous example, the sequential paging mechanism uses indexed paging for the next and previous links. This is not a requirement, though. Some feeds may only support sequential paging, or use a different set of parameters to express the next and previous links. For example:

<link rel="next" type="application/atom+xml; type=feed" title="Next Page" 
      href="http://www.example.com/sdata/myApp/myContract/-/salesOrders?orderBy=id asc&where=id gt 3547" />

Here, paging is based on a test on the resource id rather than on an index. This is perfectly valid because the service consumer SHOULD use the next link “as is” to page forwards. It SHOULD NOT interpret the link.

Paging can be lossy as it works on a snapshot of the database. If information is inserted or deleted in the database while the consumer is reading the pages then the consumer will miss a resource or get the same resource twice. This is usually not a problem for interactive applications but it will be problematic for processes that have transactional requirements.
The format of the next URL can have an influence here. For example, the earlier format (orderBy=id asc&where=id gt 3547) is more robust than the indexed format (startIndex=31&count=10) because it will not skip resources and will not return duplicates. But it still doesn't have “repeatable read” semantics.

At this stage, SData doesn't provide any support for transactions that span several requests because this would go against the “stateless” nature of the protocol. This doesn't prevent specific applications from exposing transactions as resources and provide transacted paging (the transaction id could be passed via a query parameter). SData does not try to standardize this at this stage.

Metadata

SData provides sme:canPageXxx attributes to advertise supported paging modes in the schema. These attributes apply to resource kind definitions. For example:

  <xs:element name="salesOrder" type="tns:salesOrderType" 
    sme:role="resourceKind" sme:pluralName="salesOrders" sme:label="Sales Order"
    sme:canGet="true" sme:canPost="true" sme:canPut="true" sme:canDelete="true"
    sme:canPageNext="true" sme:canPagePrevious="true" sme:canPageIndex="true" 
    sme:supportsETag="true" sme:batchingMode="syncOrAsync" />

These attributes are defined in the following table:

Attribute	Value	Description
canPagePrevious	true false (default)	Does the resource kind support sequential paging in backwards direction? When this attribute is true, feeds should contain `previous` and `last` links.
canPageNext	true false (default)	Does the resource kind support sequential paging in forwards direction? When this attribute is true, feeds should contain `next` and `first` links.
canPageIndex	true false (default)	Does the resource kind support indexed paging? When this attribute is true, feeds should contain `opensearch` elements. Also, `startIndex` and `count` parameters should be supported in query URLs.

SData providers MUST provide at least sequential paging in the forwards direction collections that are not bounded by small values (100 or less). They SHOULD provide index paging whenever practical. The paging protocol MUST conform to the rules above.