Laravel Excel upload and progressbar - excel

Question

I have a website where I can upload a .xlsx file which contains some rows of information for my database. I read the documentation for laravel-excel but it looks like it only works with progress bar if you use the console method; which I don't.
I currently just use a plain HTML upload form, no ajax yet.
But to create this progress bar for this I need to convert it to ajax, which is no hassle, that I can do.
But how would I create the progress bar when uploading the file and iterating through each row in the Excel file?
This is the controller and method where the upload gets done:
/**
* Import companies
*
* #param Import $request
* #return \Illuminate\Routing\Redirector|\Illuminate\Http\RedirectResponse
*/
public function postImport(Import $request)
{
# Import using Import class
Excel::import(new CompaniesImport, $request->file('file'));
return redirect(route('dashboard.companies.index.get'))->with('success', 'Import successfull!');
}
And this is the import file:
public function model(array $row)
{
# Don't create or validate on empty rows
# Bad workaround
# TODO: better solution
if (!array_filter($row)) {
return null;
}
# Create company
$company = new Company;
$company->crn = $row['crn'];
$company->name = $row['name'];
$company->email = $row['email'];
$company->phone = $row['phone'];
$company->website = (!empty($row['website'])) ? Helper::addScheme($row['website']) : '';
$company->save();
# Everything empty.. delete address
if (!empty($row['country']) || !empty($row['state']) || !empty($row['postal']) || !empty($row['address']) || !empty($row['zip'])) {
# Create address
$address = new CompanyAddress;
$address->company_id = $company->id;
$address->country = $row['country'];
$address->state = $row['state'];
$address->postal = $row['postal'];
$address->address = $row['address'];
$address->zip = $row['zip'];
$address->save();
# Attach
$company->addresses()->save($address);
}
return $company;
}
I know this is not much at this point. I just need some help figuring out how I would create this progress bar, because I'm pretty stuck.
My thought is to create a ajax upload form though, but from there I don't know.

Answer 1

Just an idea, but you could use the Laravel session to store the total_row_count and processed_row_count during the import execution. Then, you could create a separate AJAX call on a setInterval() to poll those session values (e.g., once per second). This would allow you to calculate your progress as processed_row_count / total_row_count, and output to a visual progress bar. – matticustard
Putting #matticustard comment into practice. Below is just sample of how things could be implemented, and maybe there are areas to improve.
1. Routes
import route to initialize Excel import.
import-status route will be used to get latest import status
Route::post('import', [ProductController::class, 'import']);
Route::get('import-status', [ProductController::class, 'status']);
2. Controller
import action will validate uploaded file, and pass $id to ProductsImport class. As it will be queued and run in the background, there is no access to current session. We will use cache in the background. It will be good idea to generate more randomized $id if more concurrent imports will be processed, for now just unix date to keep simple.
You currently cannot queue xls imports. PhpSpreadsheet's Xls reader contains some non-utf8 characters, which makes it impossible to queue.
XLS imports could not be queued
public function import()
{
request()->validate([
'file' => ['required', 'mimes:xlsx'],
]);
$id = now()->unix()
session([ 'import' => $id ]);
Excel::queueImport(new ProductsImport($id), request()->file('file')->store('temp'));
return redirect()->back();
}
Get latest import status from cache, passing $id from session.
public function status()
{
$id = session('import');
return response([
'started' => filled(cache("start_date_$id")),
'finished' => filled(cache("end_date_$id")),
'current_row' => (int) cache("current_row_$id"),
'total_rows' => (int) cache("total_rows_$id"),
]);
}
3. Import class
Using WithEvents BeforeImport we set total rows of our excel file to the cache. Using onRow we set currently processing row to the cache. And AfterReset clear all the data.
<?php
namespace App\Imports;
use App\Models\Product;
use Maatwebsite\Excel\Row;
use Maatwebsite\Excel\Concerns\OnEachRow;
use Maatwebsite\Excel\Events\AfterImport;
use Maatwebsite\Excel\Events\BeforeImport;
use Maatwebsite\Excel\Concerns\WithEvents;
use Illuminate\Contracts\Queue\ShouldQueue;
use Maatwebsite\Excel\Concerns\WithStartRow;
use Maatwebsite\Excel\Concerns\WithChunkReading;
use Maatwebsite\Excel\Concerns\WithMultipleSheets;
class ProductsImport implements OnEachRow, WithEvents, WithChunkReading, ShouldQueue
{
public $id;
public function __construct(int $id)
{
$this->id = $id;
}
public function chunkSize(): int
{
return 100;
}
public function registerEvents(): array
{
return [
BeforeImport::class => function (BeforeImport $event) {
$totalRows = $event->getReader()->getTotalRows();
if (filled($totalRows)) {
cache()->forever("total_rows_{$this->id}", array_values($totalRows)[0]);
cache()->forever("start_date_{$this->id}", now()->unix());
}
},
AfterImport::class => function (AfterImport $event) {
cache(["end_date_{$this->id}" => now()], now()->addMinute());
cache()->forget("total_rows_{$this->id}");
cache()->forget("start_date_{$this->id}");
cache()->forget("current_row_{$this->id}");
},
];
}
public function onRow(Row $row)
{
$rowIndex = $row->getIndex();
$row = array_map('trim', $row->toArray());
cache()->forever("current_row_{$this->id}", $rowIndex);
// sleep(0.2);
Product::create([ ... ]);
}
}
4. Front end
On the front-end side this is just sample how things could be handled. Here I used vuejs, ant-design-vue and lodash.
After uploading file handleChange method is called
On successful upload trackProgress method is called for the first time
trackProgress method is recursive function, calling itself on complete
with lodash _.debounce method we can prevent calling it too much
export default {
data() {
this.trackProgress = _.debounce(this.trackProgress, 1000);
return {
visible: true,
current_row: 0,
total_rows: 0,
progress: 0,
};
},
methods: {
handleChange(info) {
const status = info.file.status;
if (status === "done") {
this.trackProgress();
} else if (status === "error") {
this.$message.error(_.get(info, 'file.response.errors.file.0', `${info.file.name} file upload failed.`));
}
},
async trackProgress() {
const { data } = await axios.get('/import-status');
if (data.finished) {
this.current_row = this.total_rows
this.progress = 100
return;
};
this.total_rows = data.total_rows;
this.current_row = data.current_row;
this.progress = Math.ceil(data.current_row / data.total_rows * 100);
this.trackProgress();
},
close() {
if (this.progress > 0 && this.progress < 100) {
if (confirm('Do you want to close')) {
this.$emit('close')
window.location.reload()
}
} else {
this.$emit('close')
window.location.reload()
}
}
},
};
<template>
<a-modal
title="Upload excel"
v-model="visible"
cancel-text="Close"
ok-text="Confirm"
:closable="false"
:maskClosable="false"
destroyOnClose
>
<a-upload-dragger
name="file"
:multiple="false"
:showUploadList="false"
:action="`/import`"
#change="handleChange"
>
<p class="ant-upload-drag-icon">
<a-icon type="inbox" />
</p>
<p class="ant-upload-text">Click to upload</p>
</a-upload-dragger>
<a-progress class="mt-5" :percent="progress" :show-info="false" />
<div class="text-right mt-1">{{ this.current_row }} / {{ this.total_rows }}</div>
<template slot="footer">
<a-button #click="close">Close</a-button>
</template>
</a-modal>
</template>
<script>
export default {
data() {
this.trackProgress = _.debounce(this.trackProgress, 1000);
return {
visible: true,
current_row: 0,
total_rows: 0,
progress: 0,
};
},
methods: {
handleChange(info) {
const status = info.file.status;
if (status === "done") {
this.trackProgress();
} else if (status === "error") {
this.$message.error(_.get(info, 'file.response.errors.file.0', `${info.file.name} file upload failed.`));
}
},
async trackProgress() {
const { data } = await axios.get('/import-status');
if (data.finished) {
this.current_row = this.total_rows
this.progress = 100
return;
};
this.total_rows = data.total_rows;
this.current_row = data.current_row;
this.progress = Math.ceil(data.current_row / data.total_rows * 100);
this.trackProgress();
},
close() {
if (this.progress > 0 && this.progress < 100) {
if (confirm('Do you want to close')) {
this.$emit('close')
window.location.reload()
}
} else {
this.$emit('close')
window.location.reload()
}
}
},
};
</script>