Pagination Strategies
Read large result sets in stable, manageable pages instead of one enormous response.
What you'll learn
- Explain why large collections are paginated
- Compare offset-based and cursor-based pagination
- Traverse all pages without missing or duplicating records
- Handle data that changes mid-traversal
- Combine pagination with rate-limit awareness
6 min
Why paginate
Returning a huge collection in one response is bad for everyone. It is slow to generate, heavy to transfer, and can exhaust memory on both ends. Pagination breaks the result into pages, letting a client fetch a manageable slice at a time and stop early if it has what it needs.
From the client's side, pagination means a loop: request a page, process it, request the next, until there are no more. The two common schemes — offset and cursor — differ in how a page is identified, and that difference has real consequences for correctness when the underlying data is changing. Choosing the right scheme up front saves painful bugs later.
Offset pagination
Offset pagination identifies a page by position: skip the first 40 records, give me the next 20. It is simple and lets a client jump to an arbitrary page.
GET /resource?limit=20&offset=40Its weakness shows when the underlying set changes during traversal. If a record is inserted or removed between page requests, the positions shift: you can skip a record or see one twice. Offset also gets slower deep into a large set, since the server must count past everything skipped. It is fine for small, stable collections, but risky for large or actively changing ones.
Cursor pagination
Cursor pagination identifies the next page by a stable pointer — typically everything after this record — rather than by a numeric position. The server returns a cursor with each page, and the client passes it back to get the next.
GET /resource?limit=20&after=eyJpZCI6MTIzfQ
// response includes: 'next_cursor': '...'Because the cursor anchors to a record rather than a count, inserts and deletes elsewhere do not cause skips or duplicates, and performance stays steady regardless of depth. The trade-off is you cannot jump to an arbitrary page — you must walk forward. For large or changing collections, cursors are the safer choice. Treat the cursor as opaque and pass it back unchanged.
Traversing safely
However you paginate, write the loop defensively:
- Stop on the documented signal — an empty page or a null next cursor, not a guessed count.
- Pass the cursor opaque — never parse or fabricate it.
- Pace the loop — fetching every page back to back can hit rate limits.
- Expect change — totals may shift between pages, especially under offset.
For very large traversals, persist your progress so an interrupted run can resume rather than starting over. Combining steady pacing with cursor pagination gives you a traversal that is both correct and kind to the API.
Key takeaways
- Pagination splits large collections into manageable pages
- Offset pagination is simple but skips or duplicates when data changes mid-traversal
- Cursor pagination anchors to a record, staying correct and fast at any depth
- Stop on the documented end signal and treat cursors as opaque
- Pace the traversal to stay within rate limits
FAQ
When should I use cursor over offset pagination?
Use cursors for large or actively changing collections. They avoid the skips and duplicates that offset suffers when records are inserted or removed mid-traversal, and they keep performance steady at any depth.
Can I jump to a specific page with cursors?
Generally no. Cursor pagination is designed for sequential forward traversal, so you walk page by page. Offset pagination allows arbitrary jumps but at the cost of correctness on changing data.
How do I know when to stop?
Use the API's documented end signal — an empty page or a null next cursor — rather than relying on a total count, which may shift while you traverse. Treat the cursor itself as opaque and pass it back unchanged.
Ready to build?
Read the API reference, grab the OpenAPI spec, and ship a resilient integration.