Collect URLs and Async Convert to PDF for ZIP Download (Spring Boot + Feign)

Scenario Description

An online education platform allows students to batch‑download course materials that originate from three different sources: self‑hosted courses (preview page URLs that need PDF conversion), partner institutions (URLs obtained via a partner API and also need conversion), and historical archives (already stored as PDFs on OSS and can be downloaded directly).

When a student selects multiple resources and clicks “Batch Download”, the backend must collect all URLs, submit them to a file service for asynchronous PDF conversion and ZIP packaging, and return a task ID for the frontend to poll.

Data Structure Definitions

/**
 * Batch download request parameters
 */
@Data
public class BatchDownloadForm {
    @ApiModelProperty("Self‑hosted course material ID list")
    private List<String> selfCourseIds;
    @ApiModelProperty("Partner institution material ID list")
    private List<String> partnerCourseIds;
    @ApiModelProperty("Historical archive material ID list")
    private List<String> archiveIds;
    @ApiModelProperty("Student ID")
    @NotBlank(message = "学生ID不能为空")
    private String studentId;
}

/**
 * Unified file URL structure submitted to the file service
 */
@Data
public class FileUrl {
    @ApiModelProperty("File address (preview page URL or direct download URL)")
    @NotBlank(message = "url不能为空")
    private String url;
    @ApiModelProperty("File name inside the ZIP package")
    @NotBlank(message = "文件名不能为空")
    private String fileName;
    @ApiModelProperty("Whether to download directly (true skips PDF conversion)")
    private boolean directDownload = false;
}

/**
 * Asynchronous task submission parameters
 */
@Data
public class AsyncPdfTaskForm {
    @ApiModelProperty("System source")
    @NotBlank(message = "系统来源不能为空")
    private String source;
    @ApiModelProperty("Business type (custom identifier for different download tasks)")
    private String businessType;
    @ApiModelProperty("Business code (e.g., student ID for task correlation)")
    @NotBlank(message = "业务编码不能为空")
    private String businessCode;
    @ApiModelProperty("URL list")
    @Size(min = 1, message = "URL列表不能为空")
    private List<FileUrl> urls;
}

Database Table (Self‑Hosted Course Material)

CREATE TABLE `t_course_material` (
  `id` bigint(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  `course_id` varchar(50) NOT NULL COMMENT '课程ID',
  `title` varchar(200) NOT NULL COMMENT '资料标题',
  `preview_url` varchar(500) DEFAULT NULL COMMENT '预览页面URL',
  `source_type` varchar(20) NOT NULL COMMENT '来源类型：SELF自建/PARTNER合作/ARCHIVE归档',
  `pdf_url` varchar(500) DEFAULT NULL COMMENT '已归档的PDF地址（仅ARCHIVE类型有值）',
  `create_time` datetime DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='课程资料表';

Feign Interface Definition

/**
 * Partner institution material interface
 * Calls partner system to obtain preview URLs
 */
@FeignClient(name = "partnerFeign", url = "{feign.file-service.url}")
public interface FileServiceFeign {
    /**
     * Submit asynchronous PDF conversion and packaging task
     * @param form task parameters (including URL list)
     * @return JSON string containing task ID
     */
    @PostMapping("/v2/export/asyncUrlToPdf")
    String asyncUrlToPdf(@RequestBody AsyncPdfTaskForm form);
}

Service Implementation

@Slf4j
@Service
public class MaterialDownloadServiceImpl {
    private final CourseMaterialMapper courseMaterialMapper;
    private final PartnerFeign partnerFeign;
    private final FileServiceFeign fileServiceFeign;
    @Value("{course.preview.base-url}")
    private String selfPreviewBaseUrl;
    @Value("${spring.application.name}")
    private String serviceName;

    public MaterialDownloadServiceImpl(CourseMaterialMapper courseMaterialMapper,
                                      PartnerFeign partnerFeign,
                                      FileServiceFeign fileServiceFeign) {
        this.courseMaterialMapper = courseMaterialMapper;
        this.partnerFeign = partnerFeign;
        this.fileServiceFeign = fileServiceFeign;
    }

    /**
     * Batch download course materials
     * 1. Process three source types and collect all file URLs
     * 2. Submit to file service for async PDF conversion and packaging
     * 3. Return task ID for frontend polling
     */
    public String batchDownload(BatchDownloadForm form) {
        List<FileUrl> allFileUrls = new ArrayList<>();
        // 1. Self‑hosted courses
        if (CollUtil.isNotEmpty(form.getSelfCourseIds())) {
            List<FileUrl> selfUrls = buildSelfCourseUrls(form.getSelfCourseIds());
            allFileUrls.addAll(selfUrls);
        }
        // 2. Partner institution materials
        if (CollUtil.isNotEmpty(form.getPartnerCourseIds())) {
            List<FileUrl> partnerUrls = buildPartnerCourseUrls(form.getPartnerCourseIds());
            allFileUrls.addAll(partnerUrls);
        }
        // 3. Historical archives
        if (CollUtil.isNotEmpty(form.getArchiveIds())) {
            List<FileUrl> archiveUrls = buildArchiveUrls(form.getArchiveIds());
            allFileUrls.addAll(archiveUrls);
        }
        // 4. Validate and submit async task
        if (CollUtil.isEmpty(allFileUrls)) {
            throw new BusinessException("没有可下载的资料");
        }
        return submitAsyncTask(form.getStudentId(), allFileUrls);
    }

    private List<FileUrl> buildSelfCourseUrls(List<String> courseIds) {
        List<CourseMaterialEntity> materials = courseMaterialMapper.selectBatchIds(courseIds);
        return materials.stream().map(material -> {
            FileUrl fileUrl = new FileUrl();
            long timestamp = System.currentTimeMillis();
            String sign = DigestUtils.md5Hex(material.getCourseId() + "EDU" + timestamp);
            String url = String.format("%s?courseId=%s×tamp=%d&sign=%s",
                    selfPreviewBaseUrl, material.getCourseId(), timestamp, sign);
            fileUrl.setUrl(url);
            fileUrl.setFileName(material.getTitle() + ".pdf");
            fileUrl.setDirectDownload(false);
            return fileUrl;
        }).collect(Collectors.toList());
    }

    private List<FileUrl> buildPartnerCourseUrls(List<String> courseIds) {
        List<FileUrl> urls = new ArrayList<>();
        for (String courseId : courseIds) {
            try {
                long timestamp = System.currentTimeMillis();
                String signature = DigestUtils.md5Hex(partnerSecretKey + timestamp + ":" + courseId);
                String result = partnerFeign.getMaterialPreviewUrl(courseId, timestamp, signature);
                JSONObject json = JSONUtil.parseObj(result);
                if (!"200".equals(String.valueOf(json.get("code")))) {
                    log.error("获取合作方资料URL失败, courseId={}, msg={}", courseId, json.get("message"));
                    continue; // skip single failure
                }
                FileUrl fileUrl = new FileUrl();
                fileUrl.setUrl(json.getStr("data"));
                fileUrl.setFileName("合作课程_" + courseId + ".pdf");
                fileUrl.setDirectDownload(false);
                urls.add(fileUrl);
            } catch (Exception e) {
                log.error("调用合作方接口异常, courseId={}, error={}", courseId, e.getMessage());
            }
        }
        return urls;
    }

    private List<FileUrl> buildArchiveUrls(List<String> archiveIds) {
        List<CourseMaterialEntity> materials = courseMaterialMapper.selectBatchIds(archiveIds);
        return materials.stream()
                .filter(m -> StrUtil.isNotBlank(m.getPdfUrl()))
                .map(material -> {
                    FileUrl fileUrl = new FileUrl();
                    fileUrl.setUrl(material.getPdfUrl());
                    fileUrl.setFileName(material.getTitle() + ".pdf");
                    fileUrl.setDirectDownload(true);
                    return fileUrl;
                })
                .collect(Collectors.toList());
    }

    private String submitAsyncTask(String studentId, List<FileUrl> urls) {
        AsyncPdfTaskForm taskForm = new AsyncPdfTaskForm();
        taskForm.setSource(serviceName);
        taskForm.setBusinessType("course_material_download");
        taskForm.setBusinessCode(studentId);
        taskForm.setUrls(urls);
        String result = fileServiceFeign.asyncUrlToPdf(taskForm);
        JSONObject json = JSONUtil.parseObj(result);
        if (!"1".equals(String.valueOf(json.get("code")))) {
            throw new BusinessException("提交下载任务失败：" + json.getStr("msg"));
        }
        return json.getStr("msg"); // task ID
    }
}

Controller

@Api(tags = "课程资料下载")
@RestController
@RequestMapping("/material")
public class MaterialDownloadController {
    private final MaterialDownloadServiceImpl materialDownloadService;
    public MaterialDownloadController(MaterialDownloadServiceImpl materialDownloadService) {
        this.materialDownloadService = materialDownloadService;
    }
    @ApiOperation("批量下载课程资料")
    @PostMapping("/batchDownload")
    public R<String> batchDownload(@RequestBody @Validated BatchDownloadForm form) {
        String taskId = materialDownloadService.batchDownload(form);
        return new R<>(taskId);
    }
}

Full Process Flow

学生勾选资料 → 点击 "批量下载"
│
├── 前端发送请求
│   {
│     "selfCourseIds": ["C001", "C002"],
│     "partnerCourseIds": ["P001"],
│     "archiveIds": ["A001", "A002"],
│     "studentId": "STU_2024001"
│   }
│
├── 后端 Service 处理
│   ├── 自建课程 C001, C002
│   │   ├── 查库获取资料信息
│   │   ├── 拼接签名 URL: https://edu.com/preview?courseId=C001×tamp=xxx&sign=xxx
│   │   └── FileUrl { url=..., fileName="高等数学第一章.pdf", directDownload=false }
│   ├── 合作机构 P001
│   │   ├── 生成签名: MD5(secretKey + timestamp + ":P001")
│   │   ├── 调用 partnerFeign.getMaterialPreviewUrl("P001", timestamp, signature)
│   │   ├── 获取返回的预览 URL
│   │   └── FileUrl { url=..., fileName="合作课程_P001.pdf", directDownload=false }
│   ├── 历史归档 A001, A002
│   │   ├── 查库获取已归档的 PDF 地址
│   │   └── FileUrl { url="https://oss.com/xxx.pdf", fileName="线性代数.pdf", directDownload=true }
│   └── 合并所有 FileUrl → 提交异步任务
│
├── 文件服务异步处理
│   ├── C001: 打开 URL → 渲染页面 → 截图/打印 → 生成 PDF
│   ├── C002: 同上
│   ├── P001: 同上
│   ├── A001: 直接从 OSS 下载 PDF 原文件
│   ├── A002: 同上
│   └── 5 个文件打包 → 上传 ZIP 到 OSS → 生成下载链接
│
├── 后端返回任务 ID: "task_20240601_001"
│
└── 前端轮询 GET /file/task/status?taskId=task_20240601_001
    第1次: { "status": "processing", "progress": "3/5" }
    第2次: { "status": "processing", "progress": "5/5" }
    第3次: { "status": "completed", "downloadUrl": "https://oss.com/download/task_xxx.zip" }
    → 弹出下载

Key Design Points

Multi‑source handling : Different methods are invoked per source type to fetch URLs, keeping channel logic independent.

Unified collection : All URLs are gathered into a List<FileUrl> so downstream services have a single interface.

directDownload flag : Distinguishes files that need PDF conversion from those that can be downloaded directly, avoiding redundant processing.

Failure isolation : Partner API calls are wrapped in try‑catch; a single failure is logged and skipped without aborting the whole batch.

URL signing : MD5(secretKey + timestamp + bizId) is added to URLs to prevent tampering.

Asynchronous task : The service returns a task ID immediately, allowing large batches to be processed without client timeout.

Frontend polling : The client periodically queries task status until completion, decoupling request submission from processing.

Applicable Scenarios

This pattern fits any situation that requires collecting files from multiple origins and packaging them together, such as batch downloading contracts, exporting heterogeneous reports, printing shipping labels from different carriers, downloading electronic invoices, or exporting user data from various modules.