Skip to content
Muhammet Şafak
tr
Asked by: Naz Answered:

Should I use chunkById instead of chunk when the same job also updates the rows it iterates?


Question

In Laravel I'm processing a large table with a batch job. I filter records with `where('status', 'pending')`, iterate them with `chunk()`, and on each row I update the `status` column from `pending` to `processed`. The problem: some rows never get processed and are skipped. I think that as I update rows they drop out of the filter, the OFFSET shifts, and the records beneath it get passed over. Should I use `chunkById()` instead of `chunk()`? Does that actually fix the skip problem?

Answer

Short answer: yes, use chunkById(). The skipping you see is a well-known trap of chunk()’s OFFSET-based pagination, and yours is the textbook case. There’s one condition: never update the cursor column (usually id).

  1. Why chunk() skips. chunk() runs a separate query per page with LIMIT/OFFSET. Once you process the first 500 rows and set status to processed, those rows fall out of the where status = pending filter; on the second page OFFSET 500 is now applied to a shrunken result set, and a slice right in the middle is skipped entirely.
  2. How chunkById() fixes it. Instead of OFFSET it does keyset (seek) pagination: WHERE id > ? ORDER BY id LIMIT n. The next page always continues from the last processed id; since processed rows are already behind the cursor, there’s no “shift” to speak of.
    Order::where('status', 'pending')
        ->chunkById(500, function ($orders) {
            foreach ($orders as $order) {
                $order->update(['status' => 'processed']);
            }
        });
  3. Critical rule: don’t update the cursor column. Updating status is fine, because it’s not the cursor. But if you change the column you order by (id) inside the loop, the seek logic breaks and you’re back to inconsistency.
  4. Use lazyById() for memory. It uses the same seek logic but returns a single LazyCollection instead of a closure; if you prefer writing a fluent foreach — and without inflating RAM on huge tables — it’s cleaner.
  5. The fastest path often isn’t iteration at all. If the update is deterministic (no per-row computation), you don’t need to loop; a single set-based query beats all of them and is atomic:
    UPDATE orders SET status = 'processed' WHERE status = 'pending';
  6. Watch locks and deadlocks. In large batch updates keep the chunk size reasonable (500–1000 is a good start) and, if needed, process each chunk in its own transaction; one giant transaction inflates both lock duration and deadlock risk.

Bottom line: personally, if there’s real per-row work I’d use chunkById() (or lazyById() for fluency) and never touch the cursor column. But if the work is just updating a column, I’d drop the iteration entirely and switch to a single set-based UPDATE — it’s faster and the skip risk is zero.

Share:

Comments

Sign in with your GitHub account to join the discussion. Comments are stored in GitHub Discussions.

More Questions

All questions

Search the site

Start typing to search posts, projects and pages.

Esc to close Powered by Pagefind