https://picperf.io/https://laravelnews.s3.amazonaws.com/featured-images/Laravel-Ingest-LN.png
Laravel Ingest by Robin Kopp is a configuration-driven ETL (Extract, Transform, Load) package that replaces one-off import scripts with declarative importer classes. It handles files from a few hundred to tens of millions of rows by processing them through PHP Generators and Laravel Queues, keeping memory usage consistent regardless of file size.
Main Features
- Declarative importer classes using a fluent
IngestConfigbuilder - Automatic resolution of
BelongsToandBelongsToManyrelationships - Duplicate handling strategies:
SKIP,CREATE,UPDATE, andUPDATE_IF_NEWER - Dry-run mode to validate imports before writing to the database
- Failed row tracking with downloadable CSV exports
- Column aliasing to map varying header names to a single field
- Dynamic model resolution based on row data
- Import sources: file upload, filesystem disks (including S3), URL, FTP, and SFTP
- Auto-generated Artisan commands and REST API endpoints per importer
Defining an Importer
After installing the package and running migrations, you create an importer class that implements IngestDefinition and returns an IngestConfig. By convention, these live in the App\Ingest namespace:
namespace App\Ingest;
use App\Models\Product;
use LaravelIngest\Contracts\IngestDefinition;
use LaravelIngest\DTOs\IngestConfig;
use LaravelIngest\Enums\DuplicateStrategy;
use LaravelIngest\Enums\SourceType;
class ProductImporter implements IngestDefinition
{
public function getConfig(): IngestConfig
{
return IngestConfig::for(Product::class)
->fromSource(SourceType::UPLOAD)
->keyedBy('sku')
->onDuplicate(DuplicateStrategy::UPDATE)
->map('Product Name', 'name')
->relate('Category', 'category', Category::class, 'slug')
->validate([
'sku' => 'required|string',
'Product Name' => 'required|string|min:3',
]);
}
}
Register the importer in your AppServiceProvider using the package’s tag:
use LaravelIngest\IngestServiceProvider;
$this->app->tag([ProductImporter::class], IngestServiceProvider::INGEST_DEFINITION_TAG);
Running Imports
Once registered, the package exposes both an Artisan command and an HTTP endpoint for each importer.
Via CLI:
php artisan ingest:run product-importer --file=products.csv
Via API (multipart form upload):
POST /api/v1/ingest/upload/product-importer
For dry runs, append the --dry-run flag to the Artisan command to validate the file and surface any errors without touching the database.
Monitoring
The package includes several Artisan commands for checking on running or completed imports:
php artisan ingest:list # List registered importers
php artisan ingest:status {id} # Show progress and row statistics
php artisan ingest:cancel {id} # Stop an in-progress import
php artisan ingest:retry {id} # Reprocess only the failed rows
Equivalent REST endpoints are also available:
GET /api/v1/ingest— recent runsGET /api/v1/ingest/{id}— status and statisticsGET /api/v1/ingest/{id}/errors/summary— aggregated error breakdownGET /api/v1/ingest/{id}/failed-rows/download— CSV of rows that failed
Events
The package dispatches events throughout the import lifecycle — IngestRunStarted, ChunkProcessed, RowProcessed, IngestRunCompleted, and IngestRunFailed — which you can listen to for notifications or custom side effects.
You can find Laravel Ingest on GitHub and read the full documentation at the Laravel Ingest docs.
Laravel News
