How to Build a Scalable Asynchronous Excel Export System for Large E‑Commerce

Introduction

In large e‑commerce systems, exporting massive data sets synchronously often leads to time‑outs and memory overflow, harming user experience.

System Architecture

Overall Architecture Diagram

Core Components

Annotation Layer : Uses @ExcelExport to enable declarative programming.

Aspect Layer : CommonTaskAspect intercepts calls and creates tasks.

Task Management Layer : ExcelExportTask executes export logic.

Scheduling Engine : Quartz schedules periodic scans.

Message Notification Layer : RocketMQ + WebSocket for asynchronous notifications.

Storage Layer : MySQL stores task status; OSS stores generated files.

Asynchronous Export Process

Step 1 – Annotation‑Driven Task Creation

@ExcelExport(ExcelBizTypeEnum.USER)
@ApiOperation(notes = "Export user data", value = "Export user data")
@PostMapping("/export")
public void export(HttpServletResponse response, UserConditionEntity userConditionEntity) {
    // method body can be empty; aspect handles it
}

Step 2 – Aspect Interception and Task Creation

@Aspect
@Component
public class CommonTaskAspect {
    @Before("@annotation(cn.net.susan.annotation.ExcelExport)")
    public void before(JoinPoint joinPoint) throws Throwable {
        // obtain annotation, create task entity, persist to DB
    }
}

Step 3 – Scheduled Task Execution

@Component
public class CommonTaskJob extends BaseJob {
    @Override
    public JobResult doRun(String params) {
        // fetch waiting/running tasks, execute via strategy, return SUCCESS
    }
}

Step 4 – Asynchronous Task Handler

@AsyncTask(TaskTypeEnum.EXPORT_EXCEL)
@Service
public class ExcelExportTask implements IAsyncTask {
    @Override
    public void doTask(CommonTaskEntity commonTaskEntity) {
        // update status, invoke service to generate Excel, upload to OSS, notify
    }
}

Step 5 – Message Notification

@RocketMQMessageListener(topic = "${mall.mgt.excelExportTopic:EXCEL_EXPORT_TOPIC}",
                         consumerGroup = "${mall.mgt.excelExportGroup:EXCEL_EXPORT_GROUP}")
@Component
public class ExcelExportConsumer implements RocketMQListener<MessageExt> {
    @Override
    public void onMessage(MessageExt message) {
        // parse message and push WebSocket notification
    }
}

Design Highlights

Declarative Programming : Annotation‑driven, concise business code.

Zero Intrusion : No changes to business methods.

Type Safety : Enum ensures correct business type.

Strategy + Factory : Extensible task handling.

Quartz Scheduling : Periodic task scanning.

Concurrent Execution : Supports multiple tasks in parallel.

Advantages

User Experience

Immediate response, no waiting.

Real‑time progress via WebSocket.

Clear error messages on failure.

System Performance

High concurrency through async processing.

Memory‑optimized pagination and streaming.

Load balancing via task queue.

Scalable horizontally and vertically.

Development & Maintenance

Simplified code thanks to annotations.

Easy to add new export types.

Unified task monitoring and logging.

Robust error handling and retry.

Conclusion

The Excel asynchronous export mechanism combines annotation‑driven declarative programming, AOP interception, Quartz scheduling, RocketMQ messaging, and OSS storage to deliver a high‑performance, reliable, and extensible solution for large‑scale data export in enterprise e‑commerce applications.