Design and Implementation of Asynchronous File Export/Import Architecture

In many business systems, file import and export are common requirements; while synchronous export satisfies most scenarios, large data volumes and complex business logic often cause timeouts, making asynchronous processing essential to keep the user experience responsive.

Typical asynchronous export scenarios include statistics, imports/exports, template message sending, and complex post‑order processing, all sharing characteristics such as long processing time, low business priority, high timeout risk, and large data size.

Advantages of asynchronous export include solving timeout issues, greatly improving request handling speed, increasing throughput, and enhancing overall system performance. Disadvantages are higher complexity, more interfaces, additional server resources, and increased learning and development costs.

Architecture diagram :

Architecture analysis (process flow) :

1. Frontend sends export request.

2. API server validates parameters.

3. If validation fails, return error and end.

4. Create a task entity, set status to "processing", store it in Redis, start an asynchronous thread to call the export service, and return a "processing" status to the frontend.

5. Frontend receives the processing status and starts AJAX polling every 10 seconds to query task status.

6. Export service assembles data.

7. Data is written to a file server, which returns a URL.

8. Retrieve the task from Redis, update the URL and status to "completed", and persist back to Redis.

9. Frontend detects the "completed" status, obtains the URL, and stops polling.

10. Frontend calls an endpoint to clear the task entity from Redis.

11. Frontend displays the export result, allowing automatic download or manual click.

Additional page‑initialisation flow:

When the page loads, the export button is disabled.

AJAX queries the task status; if no task is running, the button becomes enabled; if a task is completed, a download button is shown.

To avoid Redis dead‑locks, each task object has a 30‑minute expiration, after which the task is discarded and the user can retry.

Data source diagram :

API design

Interface List

1. Get Task Status – getTaskStatus

Returns TaskResultOut entity:

Field

Type

Nullable

Comment

status

int

No, default 0

Task status (0=not started, 1=processing, 2=completed)

message

string

Yes

Prompt message (success or error)

url

string

Yes

File address

state

bool

No, default false

Whether the task succeeded

2. Register Export Task – registerExportTask

Returns true/false indicating registration success.

3. Clear Task Info – clearTaskInfo

Returns true/false indicating whether the task was cleared.

Frontend design

The frontend JavaScript is written by the frontend team; the detailed code is omitted, but the essential API calls are:

export: function(e) {
    return t.post(r.api.form["export"], e, {});
},
getTaskStatus: function(e) {
    return t.post(r.api.form.getTaskStatus, e, {});
},
registerExportTask: function(e) {
    return t.post(r.api.form.registerExportTask, e, {});
},
clearTaskInfo: function(e) {
    return t.post(r.api.form.clearTaskInfo, e, {});
}

Project actual application (screenshots omitted for brevity):

After clicking cancel, the UI updates accordingly (screenshot omitted).

File import design

Due to space constraints, the import flow mirrors the export flow and is illustrated by a design diagram:

Supplementary notes

The system requires at least one web server, one API server, one Redis server, and one file server. In a Java ecosystem with sufficient resources, the architecture can be split into frontend server, node‑layer cluster, backend API cluster, Redis, and file server.

For small companies, limited server resources may make implementation challenging; for large enterprises, mature asynchronous frameworks (e.g., task centers) already provide high‑availability solutions, making this design more suitable for medium‑scale companies.

Layer responsibilities are illustrated by additional diagrams (images omitted).

Source: http://www.cnblogs.com/jingch/p/8832161.html . Content is reproduced with attribution; copyright belongs to the original author.