Why is storing files directly in a database considered a poor design choice?

Storing binary data in a database causes page size explosion, as large files span multiple pages, which increases costs and decreases efficiency for both reading and writing.

What is the purpose of using pre-signed URLs in this system?

Pre-signed URLs allow clients to interact directly with the object store securely. This prevents the API from becoming a bottleneck and removes the need for the server to buffer large file uploads in memory.

How can we perform heavy operations like virus scanning without slowing down the user experience?

By utilizing a message queue, the system can trigger asynchronous tasks. Once a file is uploaded, a message is sent to the queue, and background workers process the file while the user is notified only when the work is complete.

When should a Content Delivery Network (CDN) be used in this architecture?

A CDN is ideal for static or public files, such as documentation or website assets, because it is globally distributed and routes user requests to the geographically closest server.

What are some open-source alternatives if one chooses not to use cloud-based object storage?

The presenter mentions options such as RustFS and SeaweedFS as S3-compatible alternatives for self-hosting object storage.

Design a File Upload Service Like a Senior Engineer

System Architecture Evolution
📌 Starting with a primitive API and relational database approach leads to major bottlenecks, as the API becomes a single point of failure and databases struggle with binary data page size limits.
☁️ Introduce an Object Store (e.g., S3, Azure Blob Storage) to handle files, using pre-signed URLs to allow clients to upload data directly, offloading traffic from your primary API.
🚀 Modern object stores support multipart uploads, which enable breaking large files into smaller chunks for efficient transmission and support features like pause/resume and deduplication.

Asynchronous Processing
⚙️ Offload background tasks—such as virus scanning, thumbnail generation, or OCR—to separate worker services to prevent blocking the main API.
📩 Utilize a message queue to trigger these tasks; the object store can trigger an event or message once a file is successfully uploaded, creating a decoupled and highly scalable pipeline.

Download Security & Performance
🌐 Integrate a Content Delivery Network (CDN) to store and serve static or public assets globally, reducing latency by routing users to the geographically closest node.
🔐 For private or secure downloads, continue to use pre-signed URLs to ensure that access is authenticated and authorized before the object store serves the file.

Key Points & Insights
➡️ Avoid storing binary files directly in relational databases to prevent performance degradation and costly database page explosions.
➡️ Always leverage pre-signed URLs for secure, direct client-to-storage communication, which keeps the API lightweight and secure.
➡️ Consider self-hosted open-source alternatives like SeaweedFS or RustFS if you require S3-compatible object storage without relying on cloud-specific providers.
➡️ Design for modularity by separating metadata storage (database) from actual file content (object store) to ensure the system is horizontally scalable.

📸 Video summarized with SummaryTube.com on Jun 15, 2026, 08:37 UTC

Design a File Upload Service Like a Senior Engineer

Loading Similar Videos...

Recently Summarized Videos

📜Transcript

📄Video Description

Loading Similar Videos...

Recently Summarized Videos

Get the Chrome Extension