Frontend Development 11 min read

How to Efficiently Render 100,000 Records in the Frontend: Tips and Code

This article walks through creating a simple Node server, building a basic HTML/JS front‑end, and applying performance techniques such as pagination with setTimeout, requestAnimationFrame, document fragments, and lazy‑loading to render massive data sets smoothly.

Programmer DD

Jun 15, 2022

How to Efficiently Render 100,000 Records in the Frontend: Tips and Code

During a recent interview a candidate was asked how to handle a backend API that returns 100,000 records at once. The following discussion explores several front‑end performance‑optimization strategies.

Create Server

Use Node to start a lightweight HTTP server that returns a JSON array of 100,000 objects.

const http = require('http')
const port = 8000;
let list = []
let num = 0
// create 100,000 records
for (let i = 0; i < 100_000; i++) {
  num++
  list.push({
    src: 'https://miro.medium.com/fit/c/64/64/1*XYGoKrb1w5zdWZLOIEevZg.png',
    text: `hello world ${num}`,
    tid: num
  })
}
http.createServer(function (req, res) {
  res.writeHead(200, {
    'Access-Control-Allow-Origin': '*',
    "Access-Control-Allow-Methods": "DELETE,PUT,POST,GET,OPTIONS",
    'Access-Control-Allow-Headers': 'Content-Type'
  })
  res.end(JSON.stringify(list));
}).listen(port, function () {
  console.log('server is listening on port ' + port);
});

Start the server with node server.js or nodemon server.js.

Create Front‑End Template

The front‑end consists of an index.html file and an index.js file.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Document</title>
  <style>
    * {padding:0;margin:0;}
    #container {height:100vh;overflow:auto;}
    .sunshine {display:flex;padding:10px;}
    img {width:150px;height:150px;}
  </style>
</head>
<body>
  <div id="container"></div>
  <script src="./index.js"></script>
</body>
</html>

// fetch data from the server
const getList = () => {
  return new Promise((resolve, reject) => {
    var ajax = new XMLHttpRequest();
    ajax.open('get', 'http://127.0.0.1:8000');
    ajax.send();
    ajax.onreadystatechange = function () {
      if (ajax.readyState == 4 && ajax.status == 200) {
        resolve(JSON.parse(ajax.responseText))
      }
    }
  })
}

const container = document.getElementById('container')

Direct Rendering (Slow)

Render all items at once. This takes about 12 seconds for 100,000 records and is unacceptable.

const renderList = async () => {
  const list = await getList()
  list.forEach(item => {
    const div = document.createElement('div')
    div.className = 'sunshine'
    div.innerHTML = `<img src="${item.src}" /><span>${item.text}</span>`
    container.appendChild(div)
  })
}
renderList()

Pagination with setTimeout

Split the data into pages (e.g., 200 items per page) and render each page sequentially using setTimeout to keep the UI responsive.

const renderList = async () => {
  const list = await getList()
  const total = list.length
  const limit = 200
  const totalPage = Math.ceil(total / limit)
  const render = (page) => {
    if (page >= totalPage) return
    setTimeout(() => {
      for (let i = page * limit; i < page * limit + limit; i++) {
        const item = list[i]
        const div = document.createElement('div')
        div.className = 'sunshine'
        div.innerHTML = `<img src="${item.src}" /><span>${item.text}</span>`
        container.appendChild(div)
      }
      render(page + 1)
    }, 0)
  }
  render(0)
}

Using requestAnimationFrame

Replace setTimeout with requestAnimationFrame to reduce reflow and improve performance.

const renderList = async () => {
  const list = await getList()
  const total = list.length
  const limit = 200
  const totalPage = Math.ceil(total / limit)
  const render = (page) => {
    if (page >= totalPage) return
    requestAnimationFrame(() => {
      for (let i = page * limit; i < page * limit + limit; i++) {
        const item = list[i]
        const div = document.createElement('div')
        div.className = 'sunshine'
        div.innerHTML = `<img src="${item.src}" /><span>${item.text}</span>`
        container.appendChild(div)
      }
      render(page + 1)
    })
  }
  render(0)
}

Document Fragment

Build a DocumentFragment for each page, append all child nodes to the fragment first, then attach the fragment to the container, which reduces the number of expensive DOM insertions.

const renderList = async () => {
  console.time('time')
  const list = await getList()
  const total = list.length
  const limit = 200
  const totalPage = Math.ceil(total / limit)
  const render = (page) => {
    if (page >= totalPage) return
    requestAnimationFrame(() => {
      const fragment = document.createDocumentFragment()
      for (let i = page * limit; i < page * limit + limit; i++) {
        const item = list[i]
        const div = document.createElement('div')
        div.className = 'sunshine'
        div.innerHTML = `<img src="${item.src}" /><span>${item.text}</span>`
        fragment.appendChild(div)
      }
      container.appendChild(fragment)
      render(page + 1)
    })
  }
  render(0)
  console.timeEnd('time')
}

Lazy Loading (Vue Example)

Because the viewport can only display a limited number of items, add a blank node at the bottom and load more data when it becomes visible.

<script setup lang="ts">
import { onMounted, ref, computed } from 'vue'
const getList = () => { /* same as before */ }
const container = ref<HTMLElement>()
const blank = ref<HTMLElement>()
const list = ref<any[]>([])
const page = ref(1)
const limit = 200
const maxPage = computed(() => Math.ceil(list.value.length / limit))
const showList = computed(() => list.value.slice(0, page.value * limit))
const handleScroll = () => {
  if (page.value > maxPage.value) return
  const clientHeight = container.value?.clientHeight
  const blankTop = blank.value?.getBoundingClientRect().top
  if (clientHeight === blankTop) {
    page.value++
  }
}
onMounted(async () => {
  const res = await getList()
  list.value = res
})
</script>

<template>
  <div id="container" @scroll="handleScroll" ref="container">
    <div class="sunshine" v-for="item in showList" :key="item.tid">
      <img :src="item.src" />
      <span>{{ item.text }}</span>
    </div>
    <div ref="blank"></div>
  </div>
</template>

Conclusion

Starting from an interview question, we examined several front‑end performance‑optimization techniques: direct rendering, pagination with setTimeout, using requestAnimationFrame, leveraging document fragments, and lazy loading with Vue. These approaches help keep the UI responsive even when the backend returns massive data sets.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.

frontend lazy loading requestAnimationFrame

Written by

Programmer DD

A tinkering programmer and author of "Spring Cloud Microservices in Action"

0 followers

Reader feedback

How this landed with the community

Rate this article

Was this worth your time?

Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.