2 years ago

If you've built a product big enough or for long enough, chances are you've had to offer export functionality that delivers a ZIP file to the user containing their data. In our case this was a media export containing videos, photos and audio clips that were stored in an S3 bucket. Our first implementation used an asynchronous background job to download each media file to disk and then used PHP's https://www.php.net/manual/en/book.zip.php to build the final archive to send to the user.

We'd just stream each of the files off S3 in turn and send them straight back to S3 inside the ZIP file, never keeping an entire file in memory.

Exports are finishing faster than ever, and we're building these ZIP files without ever saving the input files or the resulting ZIP file to disk.