Pagation approach for large result sets

added new article explaining the new approach.
updated the existing doc about startrow limit.

Author: leonardean

docs/general-development/pagination-for-large-result-sets.md

@@ -0,0 +1,56 @@
+---
+title: Pagination for large result sets
+ms.date: 10/14/2020
+ms.prod: sharepoint
+localization_priority: Priority
+---
+
+# Pagination for large result sets
+
+If you have a large number of pages (for example, over 50,000) to page through in a query, it is recommended to use the approach explained in this article instead of the approach of [StartRow](https://docs.microsoft.com/sharepoint/dev/general-development/sharepoint-search-rest-api-overview#startrow). This approach uses sorting on `[docid]` in ascending order and uses a query restriction on the `IndexDocId` value with an increasing value of `IndexDocID` for each new page.
+
+The advantages of this approach are:
+
+- It provides pagination with better performance
+- It does not have a limit on the number of pages 
+
+Here is an example of using this approach:
+
+For Page 1, issue a query with sort on `[docid]` in ascending order:
+
+```http
+GET http:// _server_/_api/search/query?querytext='sharepoint'&sortlist='[docid]:ascending'
+```
+
+The result of this query should contain the following content:
+
+```xml
+...
+<d:element m:type="SP.SimpleDataRow">
+    <d:Cells>
+        ...
+        <d:element m:type="SP.KeyValue">
+            <d:Key>DocId</d:Key>
+            <d:Value>10</d:Value>
+            <d:ValueType>Edm.Int64</d:ValueType>
+        </d:element>
+        ...
+```
+
+Get the `DocId` value of the last entry in the result. You should be able to find `DocId` under `SP.SimpleDataRow`. Let's say the `DocId` value is `10`. We will use that as the `DocID` restriction for page 2:
+
+For Page 2, we use the following query, where we continue using sortlist on `DocId` in ascending order, but also add an `IndexDocId` restriction:
+
+```http
+GET http:// _server_/_api/search/query?querytext='sharepoint indexdocid>10'&amp;sortlist='[docid]:ascending'
+```
+
+Let's say the `DocId` value of the last entry in the result is 20.
+
+For Page 3, we continue the query with the same pattern as previous page:
+
+```http
+GET http:// _server_/_api/search/query?querytext='sharepoint indexdocid>20'&amp;sortlist='[docid]:ascending'
+```
+
+And so on for the rest of the pages.

docs/general-development/sharepoint-search-rest-api-overview.md

@@ -239,6 +239,9 @@ GET http:// _server_/_api/search/query?querytext='sharepoint'&startrow=10
 }
 ```
 
+> [!NOTE]
+> Please be aware that in order to provide search experience with high performance, we limit the maximum supported value of `StartRow` to be **50,000**. If you need to page through larger result sets, please see [Pagination for large result sets.](https://docs.microsoft.com/sharepoint/dev/general-development/pagination-for-large-result-sets)
+
 <a name="bk_RowLimit"> </a>
 
 ### RowLimit

docs/toc.yml

@@ -1098,6 +1098,8 @@
         items:
         - name: Retrieve query suggestions using the Search REST service
           href: general-development/retrieving-query-suggestions-using-the-search-rest-service.md
+        - name: Pagination for large result sets
+          href: general-development/pagination-for-large-result-sets.md
       - name: Search add-ins
         href: general-development/search-add-ins-in-sharepoint.md
       - name: Customize search results