PHP中循环使用curl调用接口导致内存溢出如何解决 
                            
                                                    
                        
                    
                    
  
                    
                    1. 运行环境
lnmp
1). 当前使用的 Laravel 版本?
laravel 版本:5.4
2). 当前使用的 php版本?
PHP 版本:7.1
3). 当前系统 ubuntu
2. 问题描述?
在laravel中循环调用接口拉取抖音数据导致内存溢出,每次内存溢出都是在curl_exec函数中,问一下这种情况有什么好的解决方法吗?循环调用是不可避免的
private function curl($url, $postData)
    {
        $ch = curl_init($url);
        $options = [
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_POST           => true,
            CURLOPT_SSL_VERIFYPEER => false,
            CURLOPT_SSL_VERIFYHOST => false,
            CURLOPT_POSTFIELDS     => $postData
        ];
        curl_setopt_array($ch, $options);
        $response = curl_exec($ch);
        if (curl_errno($ch)) {
            Log::info(curl_error($ch));
        } else {
            $data = json_decode($response, true);
        }
        curl_reset($ch);
        curl_close($ch);
        unset($ch);
        return $data;
    }贴一下目前的代码
$pullDay = date('Y-m-d', strtotime('-3 day'));
        $pullDay = '2023-10-26';
        $pageSize = 10000; //单次查询数量 抖音平台支持最大10000
        $url = 'https://developer.toutiao.com/api/apps/taskbox/query_task_video_daily_data/';
        //从数据库获取查询的记录
        $record = DistributionDyTaskPullRecord::query()
            ->where('billing_date', $pullDay)
            ->orderByDesc('id')
            ->first();
        $lastRequestEndTime = strtotime($pullDay) + 86399; //视频发布结束参数的最大时间
        if (!$record) {
            $pageNo = 1;
            $publishStartTime = strtotime(date('Y-m-d', strtotime('-81 day')));
            $publishEndTime = strtotime('+1 week -1 second', $publishStartTime);
        } else {
            if ($record->is_need_page) {
                $pageNo = $record->page + 1;
                $publishStartTime = $record->publish_start_time;
                $publishEndTime = $record->publish_end_time;
            } else {
                $pageNo = 1;
                $publishStartTime = $record->publish_end_time + 1;
                $publishEndTime = strtotime('+1 week -1 second', $publishStartTime);
            }
            if ($record->status == 504 || $record->error_code == 4001010) {
                $pageNo = $record->page;
                $publishStartTime = $record->publish_start_time;
                $publishEndTime = $record->publish_end_time;
            }
            //当所有条件满足的时候 就不在继续拉取
            if ($record->status == 1 && $record->error_code == 0
                && $record->publish_end_time >= $lastRequestEndTime
                && $record->is_need_page == 0) return;
        }
        if ($publishEndTime > $lastRequestEndTime) $publishEndTime = $lastRequestEndTime;
//        DistributionDyTaskSubmissionVideoDetail::query()->where('date', $pullDay)->delete();
        $disTask = DistributionTask::all()->pluck('payment_allocate_ratio', 'task_id');
        $packSetting = PackSetting::query()->where('id', 34)->first();
        $post_data = [
            'access_token'             => $packSetting->access_token,
            'appid'                    => $packSetting->appid,
            'page_no'                  => $pageNo,
            'page_size'                => $pageSize,
            'video_publish_start_time' => $publishStartTime,
            'video_publish_end_time'   => $publishEndTime,
            'billing_date'             => $pullDay,
        ];
        $insertRecord = [
            'page'               => $pageNo,
            'publish_start_time' => $publishStartTime,
            'publish_end_time'   => $publishEndTime,
            'billing_date'       => $pullDay,
        ];
        try {
            $data = GuzzleHelper::post($url, [ 'query' => $post_data ]);
            $insertRecord['error_code'] = $data['error'] ?? 0;
            $insertRecord['data_count'] = count($data['results']);
            $insertData = [];
            if ($data['error'] == 0 && count($data['results']) > 0) {
                foreach ($data['results'] as $val) {
                    $new = [
                        'video_id'               => $val['VideoId'] ?? 0,
                        'task_id'                => $val['TaskId'] ?? 0,
                        'task_name'              => $val['TaskName'] ?? '',
                        'publish_time'           => $val['PublishTime'] ?? 0,
                        'author'                 => $val['Author'] ?? '',
                        'douyin_id'              => $val['DouyinID'] ?? '',
                        'video_title'            => $val['VideoTitle'] ?? '',
                        'micro_app_title'        => $val['MicroAppTitle'] ?? '',
                        'video_views'            => $val['video_views'] ?? 0,
                        'likes'                  => $val['Likes'] ?? 0,
                        'comments'               => $val['Comments'] ?? 0,
                        'shares'                 => $val['Shares'] ?? 0,
                        'video_link'             => $val['VideoLink'] ?? '',
                        'expected_profit'        => $val['ExpectedProfit'] ?? 0,
                        'gmv_1d'                 => $val['GMV1d'] ?? 0,
                        'refund_gmv_1d'          => $val['RefundGMV1d'] ?? 0,
                        'billing_gmv_1d'         => $val['BillingGMV1d'] ?? 0,
                        'billing_refund_gmv_1d'  => $val['BillingRefundGMV1d'] ?? 0,
                        'ad_share_cost_1d'       => $val['AdShareCost1d'] ?? 0,
                        'feed_ad_share_cost_1d'  => $val['FeedAdShareCost1d'] ?? 0,
                        'active_cnt_1d'          => $val['ActiveCnt1d'] ?? 0,
                        'date'                   => $val['date'] ?? '',
                        'client_name'            => $val['ClientName'] ?? '',
                        'payment_allocate_ratio' => $disTask[$val['TaskId']] ?? 0,
                    ];
                    $insertData[] = $new;
                }
                $batches = array_chunk($insertData, 1000);
                foreach ($batches as $batch) {
                    DistributionDyTaskSubmissionVideoDetail::insert($batch);
                }
            }
            if (count($data['results']) < $pageSize) {
                $insertRecord['is_need_page'] = 0;
            } else {
                $insertRecord['is_need_page'] = 1;
            }
        } catch (\Exception $e) {
            Log::info('抖音数据拉取异常,拉取参数为' . json_encode($post_data) . '。异常信息' . $e->getMessage());
            if ($e->getCode() == 504) {
                $insertRecord['status'] = $e->getCode();
            }
        }
        $insertRecord['created_at'] = Carbon::now();
        DistributionDyTaskPullRecord::insert($insertRecord);
    }之前分页是从1-N 的样子和$pullDay这个参数是递增的类似10-21,10-22这样循环
 
           
         
                     
                    
 是我框架他们封装的sql日志记录的原因,这里一直往$sqlMessage中存 mysql信息 导致内存一直增长
是我框架他们封装的sql日志记录的原因,这里一直往$sqlMessage中存 mysql信息 导致内存一直增长 
            
 
             
             
         
         内存基本上会越来越大@
 内存基本上会越来越大@ 
             
         
             
             
             艰难找到了一个遇到同样问题的情况
 艰难找到了一个遇到同样问题的情况 ,这样,然后通过脚本每分钟去调用一下接口,这样能很有效的去解决内存溢出的问题
,这样,然后通过脚本每分钟去调用一下接口,这样能很有效的去解决内存溢出的问题 
             
             
             
             
             
             
           
           关于 LearnKu
                关于 LearnKu
               
                     
                     
                     粤公网安备 44030502004330号
 粤公网安备 44030502004330号 
 
推荐文章: