An efficient alternative to paging with SQL OFFSETs

Developer Resources

Challenge

Running WordPress.com means having multimillion-record database tables. Tables which we often need to batch-query.

Provided we could hardly select (or update, etc) millions of records at once and expect speed, we commonly have to “page” our scripts to only handle a limited number of records at once, then move on to the next batch.

Classic, but inefficient, solution

The usual way of paging result sets in most SQL RDMS is to use the OFFSET option (or LIMIT [offset], [limit], which is the same).

But on a performance level, this means you’re asking your DB engine to figure out where to start from all on its own, every time. Which then means it must be aware of every record before the queried offset, because they could be different between queries (deletes, etc). So the higher your offset number, the longer the overall query will take.

Alternative solution

Instead, of…

View original post 126 more words

Tinggalkan Jawapan

Please log in using one of these methods to post your comment:

WordPress.com Logo

Anda sedang menulis komen melalui akaun WordPress.com anda. Log Out / Tukar )

Twitter picture

Anda sedang menulis komen melalui akaun Twitter anda. Log Out / Tukar )

Facebook photo

Anda sedang menulis komen melalui akaun Facebook anda. Log Out / Tukar )

Google+ photo

Anda sedang menulis komen melalui akaun Google+ anda. Log Out / Tukar )

Connecting to %s