Laravel Excel 导出大量数据,服务器超时
1. 运行环境
2). 当前使用的版本
Laravel 版本:8.38.0
PHP 版本:7.4.3
mysql: 5.7
3). 当前系统
Windows 10
4). 业务环境
导出大量csv格式数据数据
2. 问题描述?
使用dcat/easy-excel 倒出数据是应为导出的字段需要大量查询和逻辑,导致数据量上万条时服务器请求超时,之后我使用chunkSize方法分块导出,经测试最大值为100,这样下来导致php脚本运行超时,数据整合下拉大约两万条数据,本地测试需要十几分钟,应为线上服务器打开了安全模式,所以不能设置set_time_limit(0)。之后我想到了队列,然后进行测试,在第一步直接被卡死 (队列使用插件Maatwebsite/Laravel-Excel 3)
我写的导出类
<?php
namespace App\Tenant\Extensions;
use Dcat\Admin\Grid\Exporters\AbstractExporter;
use Illuminate\Support\Str;
use Maatwebsite\Excel\Concerns\Exportable;
use Maatwebsite\Excel\Concerns\FromCollection;
use Maatwebsite\Excel\Concerns\WithHeadings;
use Maatwebsite\Excel\Concerns\WithMapping;
use PhpOffice\PhpSpreadsheet\Worksheet\Drawing;
use Maatwebsite\Excel\Concerns\WithStrictNullComparison;
use Maatwebsite\Excel\Concerns\ShouldAutoSize;
class ExcelExpoter extends AbstractExporter implements WithMapping, WithHeadings, FromCollection, WithStrictNullComparison,ShouldAutoSize,FromQuery
{
use Exportable;
protected $fileName = '表格导出测试';
protected $titles = [];
public function __construct()
{
$this->fileName = $this->fileName.'_'.Str::random(6).'.csv';//拼接下载文件名称
$this->titles = $title = [
'coures_id' =>'Course ID',
'lesson_id' =>'Lesson ID',
'subject_id' =>'Subject ID',
'subject_name' =>'Subject Name',
'level_name' =>'Level Name',
'progress' =>'Progress',
'lesson_start_at' =>'Lesson Start Datetime',
'lesson_end_at' =>'Lesson End Datetime',
'duration' =>'Duration',
'max_student' =>'Max Student',
'of_student' =>'of Student',
'identity' =>'Identity',
'name' =>'Name',
'phone' =>'Phone Number',
'present' =>'Show Up',
'preset_time' =>'Actual Duration(m)',
'late' =>'Late',
'leave_early' =>'Leavel Early',
'first_entry_datetime' =>'First Entry Datetime',
'last_exit_datetime' =>'Last Exit Datetime'
];
parent::__construct();
}
public function export()
{
// TODO: Implement export() method.
$this->queue($this->fileName)->onQueue('exports');
return back()->withSuccess('Export started!');
}
public function collection()
{
// TODO: Implement collection() method.
return collect($this->buildData());
}
public function headings(): array
{
// TODO: Implement headings() method.
return $this->titles();
}
public function map($row): array
{
// TODO: Implement map() method.
return $row;
}
public function drawings()
{
$drawing = new Drawing();
$drawing->setName('Logo');
$drawing->setDescription('This is my logo');
$drawing->setPath(public_path('/img/logo.jpg'));
$drawing->setHeight(90);
$drawing->setCoordinates('B3');
return $drawing;
}
public function query()
{
return Invoice::query();
}
}
报错信息
Serialization of 'Closure' is not allowed
希望各位大佬给个建议哈
Serialization of 'Closure' is not allowed 报错是 PHP无法序列化闭包,应该是队列引用的代码里包含了闭包,根据Stack trace 报错信息找一下问题,我遇到过向Job里边传参传了整个request实例,包含了上传文件的属性,报了这个错,最后按需传参,传输组解决了。 队列导出的话,可以试试github.com/Maatwebsite/Laravel-Exc... 的队列导出功能,文件生成成功后可以用websocket传输数据通知前端下载。