Skip to main content

Storage API

Taruvi's Storage API provides a powerful cloud file storage system. Store and manage files with advanced features like batch operations, path-based access, and fine-grained bucket controls.

Overview​

The Storage API is designed for:

  • Scalable File Storage: Reliable cloud storage for unlimited files
  • Multi-Tenant: Organize files by apps and buckets for isolation
  • Secure Access: Public and private buckets with authentication
  • Advanced Filtering: 40+ query filters with sub-second performance on 100K+ objects
  • Advanced Features: Batch uploads, metadata search, copy/move operations
  • Flexible Upload Methods: Support for both multipart and raw binary uploads

Key Concepts​

Buckets​

Buckets are top-level containers for organizing files.

Features:

  • Public or private access control
  • File size limits
  • MIME type restrictions
  • Per-app isolation

Objects​

Objects are files stored within buckets. Each object has:

  • Path: User-defined path for organization (e.g., users/avatar.jpg)
  • Unique ID: Auto-generated UUID for reliable references
  • Metadata: Size, MIME type, and custom JSON metadata
  • Tracking: Created/modified timestamps for auditing

Storage Paths​

Objects are stored using user-friendly paths that you define:

Examples:
  users/user-123/profile.jpg
  documents/2025/report.pdf
  media/thumbnails/image-001.jpg

Benefits:

  • Intuitive Organization: Create folder-like structures
  • Secure Storage: Files are stored securely in the cloud
  • Easy Renaming: Change paths without re-uploading files

How It Works​

The Storage API provides a simple REST interface to manage files in the cloud:

  1. Create Buckets: Organize files into logical containers with access controls
  2. Upload Files: Store files using PUT/POST requests with raw binary or form-data
  3. Download Files: Retrieve files via direct URLs or streaming downloads
  4. Manage Objects: Search, copy, move, and delete files with simple API calls

API Patterns​

The Storage API provides multiple upload methods:

1. Path-Based Upload API ⭐ Recommended​

Upload files using path-based PUT/POST requests:

PUT    /api/apps/{app_slug}/storage/buckets/{bucket}/objects/{path/to/file.jpg}
POST   /api/apps/{app_slug}/storage/buckets/{bucket}/objects/{path/to/file.jpg}

Features:

  • ✅ Object path directly in URL
  • ✅ Raw binary data in body
  • ✅ Metadata via headers (X-Metadata-*)
  • ✅ Clean, RESTful design
  • ✅ App-scoped for multi-tenancy

Both PUT and POST support:

  • Raw binary (recommended): Send file bytes directly with metadata in headers
  • Multipart form: Also accepts file field for compatibility

2. Form-Based Upload API (Legacy)​

Traditional multipart form uploads:

POST   /api/apps/{app_slug}/storage/buckets/{bucket_slug}/objects/
  • Uses multipart/form-data
  • File in file form field
  • Path in path form field

Bucket & Object Management​

Standard REST endpoints for managing buckets and objects (all scoped to app):

# Buckets
GET    /api/apps/{app_slug}/storage/buckets/
POST   /api/apps/{app_slug}/storage/buckets/
GET    /api/apps/{app_slug}/storage/buckets/{bucket_slug}/
PATCH  /api/apps/{app_slug}/storage/buckets/{bucket_slug}/
DELETE /api/apps/{app_slug}/storage/buckets/{bucket_slug}/

# Objects
GET    /api/apps/{app_slug}/storage/buckets/{bucket_slug}/objects/              # List
GET    /api/apps/{app_slug}/storage/buckets/{bucket_slug}/objects/{key}/        # Download (default)
GET    /api/apps/{app_slug}/storage/buckets/{bucket_slug}/objects/{key}/?metadata=true  # Get metadata (JSON)
PATCH  /api/apps/{app_slug}/storage/buckets/{bucket_slug}/objects/{key}/        # Update metadata
DELETE /api/apps/{app_slug}/storage/buckets/{bucket_slug}/objects/{key}/        # Delete

Example - Raw Binary Upload:

curl -X PUT \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: image/jpeg" \
  -H "X-Metadata-User-Id: 123" \
  --data-binary "@avatar.jpg" \
  https://api.example.com/api/apps/my-app/storage/buckets/avatars/objects/users/123/avatar.jpg

Common Use Cases​

User Avatars​

Bucket: avatars
Files: users/{user_id}/avatar.jpg
Access: Public
Limits: 5MB, images only

Document Management​

Bucket: documents
Files: projects/{project_id}/docs/{filename}
Access: Private
Limits: 50MB, documents only

Media Library​

Bucket: media
Files: {year}/{month}/{filename}
Access: Public
Limits: 100MB, images/videos

Security Features​

Access Control​

  • Authentication Required: All endpoints require authentication
  • Per-Bucket Control: Public vs private buckets
  • User Tracking: Created/modified by fields

File Validation​

  • Size Limits: Configurable per bucket
  • MIME Type Filtering: Whitelist allowed file types
  • Wildcard Patterns: Support for image/*, video/*, etc.

Storage Security​

  • Secure Storage: Files are stored securely in the cloud
  • Unique Identifiers: Each file has a unique ID preventing conflicts
  • Access Control: Public/private buckets with authentication requirements

Performance Considerations​

Efficient Operations​

  • Streaming Downloads: Fast, efficient file delivery
  • Batch Operations: Delete multiple files efficiently
  • Advanced Filtering: Sub-second queries on 100K+ objects with 40+ filter options
  • Optimized Indexes: Composite indexes for common query patterns

Best Practices​

  • Use bucket-relative path prefixes for folder-like organization (?prefix=users/123/ or ?file__startswith=users/123/)
  • Configure bucket file_size_limit appropriately for your use case (default: 50MB)
  • Enable public buckets for static assets
  • Use advanced query filters for complex searches (range, metadata, user filters)
  • Apply range filters for file size/date filtering (?size__gte=1048576&size__lte=10485760)
  • Filter by MIME category for faster queries (?mimetype_category=image)

Error Handling​

The API uses standard HTTP status codes:

CodeMeaning
200Success
201Created
204Deleted successfully
400Validation error (size, MIME type)
404Bucket or object not found
500Server error (storage issues)

All errors return JSON:

{
  "error": "File size exceeds bucket limit",
  "detail": "File size (10485760 bytes) exceeds bucket limit (5MB)"
}

Rate Limits & Quotas​

Storage operations have the following limits:

  • Request Rate: High throughput for thousands of requests per second
  • File Size: Default 50MB per file (configurable per bucket)
  • Batch Upload: Max 10 files per request, 100MB total size limit

Authentication​

All API operations require authentication using Taruvi JWT tokens:

-H "Authorization: Bearer YOUR_TOKEN"

Public Buckets: Files in public buckets can be accessed without authentication for read operations.

Next Steps​

API Reference Summary​

Buckets​

  • Create, list, update, delete buckets
  • Configure access control and file limits

Objects​

  • Upload files with multipart/form-data or raw binary (PUT/POST)
  • Download files with streaming responses
  • Advanced filtering with 40+ query parameters:
    • Range filters (size, dates)
    • Multi-value MIME type filtering
    • Metadata search (JSON fields)
    • Visibility and user/ownership filters
    • Full-path search capabilities
  • Copy and move operations

Advanced​

  • Batch upload/delete operations
  • Path-based file access
  • Advanced search and filtering
  • Custom metadata storage