Data Service API Endpoint Reference

Complete reference for all Data Service API endpoints with request/response examples.

Authentication

All endpoints require JWT authentication. Include the access token in the Authorization header:

Authorization: Bearer YOUR_ACCESS_TOKEN

Apps Endpoints

List Apps

GET /api/apps/

Response:

[
  {
    "id": 1,
    "name": "My Application",
    "slug": "my-application",
    "description": "Application description",
    "datasource_count": 2,
    "created_at": "2024-10-30T12:00:00Z",
    "updated_at": "2024-10-30T12:00:00Z"
  }
]

Create App

POST /api/apps/
Content-Type: application/json

{
  "name": "New Application",
  "description": "Optional description"
}

Response: 201 Created

{
  "id": 2,
  "name": "New Application",
  "slug": "new-application",
  "description": "Optional description",
  "datasource_count": 0,
  "table_count": 0,
  "datasources": [],
  "created_at": "2024-10-30T12:00:00Z",
  "updated_at": "2024-10-30T12:00:00Z",
  "created_by": 1,
  "modified_by": null
}

Get App Details

GET /api/apps/{app_slug}/

Response:

{
  "id": 1,
  "name": "My Application",
  "slug": "my-application",
  "description": "Application description",
  "datasource_count": 2,
  "table_count": 5,
  "datasources": [
    {
      "id": 1,
      "name": "Users DB",
      "slug": "users-db",
      "provider_type": "flat_table",
      "table_count": 3
    },
    {
      "id": 2,
      "name": "Analytics DB",
      "slug": "analytics-db",
      "provider_type": "jsonb",
      "table_count": 2
    }
  ],
  "created_at": "2024-10-30T12:00:00Z",
  "updated_at": "2024-10-30T12:00:00Z",
  "created_by": 1,
  "modified_by": null
}

Update App

PUT /api/apps/{app_slug}/
Content-Type: application/json

{
  "name": "Updated Name",
  "description": "Updated description"
}

PATCH /api/apps/{app_slug}/
Content-Type: application/json

{
  "description": "Just update description"
}

Delete App (Soft Delete)

DELETE /api/apps/{app_slug}/

Response: 204 No Content

Note: This marks the app, all its datasources, and all datatables as inactive.

Get App Statistics

GET /api/apps/{app_slug}/stats/

Response:

{
  "app_name": "My Application",
  "app_slug": "my-application",
  "total_datasources": 2,
  "total_tables": 5,
  "materialized_tables": 3,
  "unmaterialized_tables": 2,
  "datasources": [
    {
      "name": "Users DB",
      "slug": "users-db",
      "provider_type": "flat_table",
      "total_tables": 3,
      "materialized_tables": 2
    },
    {
      "name": "Analytics DB",
      "slug": "analytics-db",
      "provider_type": "jsonb",
      "total_tables": 2,
      "materialized_tables": 1
    }
  ]
}

Restore App

POST /api/apps/{app_slug}/restore/

Response:

{
  "message": "App restored successfully"
}

DataSource Endpoints

List DataSources

GET /api/apps/{app_slug}/datasources/

Response:

[
  {
    "id": 1,
    "name": "Users DB",
    "slug": "users-db",
    "app_name": "My Application",
    "app_slug": "my-application",
    "provider_type": "flat_table",
    "description": "User data storage",
    "is_active": true,
    "table_count": 3,
    "created_at": "2024-10-30T12:00:00Z",
    "updated_at": "2024-10-30T12:00:00Z"
  }
]

Create DataSource

POST /api/apps/{app_slug}/datasources/
Content-Type: application/json

{
  "name": "Analytics DB",
  "slug": "analytics-db",
  "provider_type": "jsonb",
  "provider_config": {
    "compression": "lz4",
    "indexing": "gin"
  },
  "description": "Analytics data storage"
}

Provider Types:

flat_table: Traditional PostgreSQL tables
jsonb: PostgreSQL JSONB storage
mongodb: MongoDB collections

Response: 201 Created

{
  "id": 2,
  "app": 1,
  "app_name": "My Application",
  "app_slug": "my-application",
  "name": "Analytics DB",
  "slug": "analytics-db",
  "provider_type": "jsonb",
  "provider_config": {
    "compression": "lz4",
    "indexing": "gin"
  },
  "description": "Analytics data storage",
  "is_active": true,
  "table_count": 0,
  "materialized_count": 0,
  "created_at": "2024-10-30T12:00:00Z",
  "updated_at": "2024-10-30T12:00:00Z",
  "created_by": 1,
  "modified_by": null
}

Get DataSource Details

GET /api/apps/{app_slug}/datasources/{datasource_slug}/

Update DataSource

PUT /api/apps/{app_slug}/datasources/{datasource_slug}/
PATCH /api/apps/{app_slug}/datasources/{datasource_slug}/

Delete DataSource

DELETE /api/apps/{app_slug}/datasources/{datasource_slug}/

Get DataSource Tables

GET /api/apps/{app_slug}/datasources/{datasource_slug}/tables/

Get DataSource Statistics

GET /api/apps/{app_slug}/datasources/{datasource_slug}/stats/

Response:

{
  "total_tables": 3,
  "materialized_tables": 2,
  "unmaterialized_tables": 1,
  "provider_type": "flat_table",
  "is_active": true
}

DataTable Endpoints

List DataTables

GET /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/

Response:

[
  {
    "id": 1,
    "name": "users",
    "datasource_name": "Users DB",
    "datasource_slug": "users-db",
    "app_name": "My Application",
    "app_slug": "my-application",
    "physical_table_name": "myapp_users",
    "is_materialized": true,
    "is_active": true,
    "field_count": 5,
    "created_at": "2024-10-30T12:00:00Z",
    "updated_at": "2024-10-30T12:00:00Z"
  }
]

Create DataTable

POST /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/
Content-Type: application/json

{
  "name": "users",
  "description": "User accounts table",
  "json_schema": {
    "fields": [
      {
        "name": "id",
        "type": "integer",
        "constraints": {
          "required": true,
          "unique": true
        }
      },
      {
        "name": "email",
        "type": "string",
        "format": "email",
        "constraints": {
          "required": true,
          "unique": true
        }
      },
      {
        "name": "name",
        "type": "string",
        "constraints": {
          "required": true
        }
      },
      {
        "name": "is_active",
        "type": "boolean"
      },
      {
        "name": "created_at",
        "type": "datetime"
      }
    ],
    "primaryKey": ["id"]
  }
}

Response: 201 Created

{
  "id": 1,
  "data_source": 1,
  "datasource_name": "Users DB",
  "datasource_slug": "users-db",
  "app_name": "My Application",
  "app_slug": "my-application",
  "name": "users",
  "json_schema": {
    "fields": [...],
    "primaryKey": ["id"]
  },
  "physical_table_name": "myapp_users",
  "is_materialized": false,
  "is_active": true,
  "description": "User accounts table",
  "schema_hash": "a7f3b8c...",
  "field_count": 5,
  "field_names": ["id", "email", "name", "is_active", "created_at"],
  "primary_key": ["id"],
  "created_at": "2024-10-30T12:00:00Z",
  "updated_at": "2024-10-30T12:00:00Z",
  "created_by": 1,
  "modified_by": null
}

Get DataTable Details

GET /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/

Update DataTable

PUT /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/
PATCH /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/
Content-Type: application/json

{
  "description": "Updated description",
  "json_schema": {
    // Updated schema
  }
}

Note: Schema updates automatically recalculate the schema_hash.

Delete DataTable

DELETE /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/

Get Schema Only

GET /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/schema/

Response:

{
  "name": "users",
  "schema": {
    "fields": [...],
    "primaryKey": ["id"]
  },
  "schema_hash": "a7f3b8c...",
  "field_count": 5
}

Validate Data Against Schema

POST /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/validate_data/
Content-Type: application/json

{
  "data": [
    {
      "id": 1,
      "email": "user@example.com",
      "name": "John Doe",
      "is_active": true,
      "created_at": "2024-10-30T12:00:00Z"
    },
    {
      "id": 2,
      "email": "invalid-email",
      "name": "Jane Smith"
    }
  ]
}

Response:

{
  "is_valid": false,
  "errors": [
    "Row 1: Missing required field 'email'"
  ],
  "validated_rows": 2
}

Cross-DataSource Endpoints

List All DataTables for App

GET /api/apps/{app_slug}/datatables/

Lists all datatables across all datasources for the app.

Query Parameters:

datasource: Filter by datasource slug
is_materialized: Filter by materialization status (true/false)

Examples:

# All tables
GET /api/apps/my-app/datatables/

# Tables in specific datasource
GET /api/apps/my-app/datatables/?datasource=users-db

# Only materialized tables
GET /api/apps/my-app/datatables/?is_materialized=true

Get Cross-DataSource Statistics

GET /api/apps/{app_slug}/datatables/stats/

Response:

{
  "total_tables": 5,
  "materialized_tables": 3,
  "unmaterialized_tables": 2,
  "datasource_count": 2,
  "datasources": {
    "users-db": {
      "name": "Users DB",
      "provider_type": "flat_table",
      "table_count": 3,
      "materialized_count": 2
    },
    "analytics-db": {
      "name": "Analytics DB",
      "provider_type": "jsonb",
      "table_count": 2,
      "materialized_count": 1
    }
  }
}

Error Responses

Validation Errors

{
  "json_schema": [
    "Invalid Frictionless schema: Field 'age' has invalid constraints"
  ]
}

Not Found

HTTP/1.1 404 Not Found

{
  "detail": "Not found."
}

Permission Denied

HTTP/1.1 403 Forbidden

{
  "detail": "You do not have permission to perform this action."
}

Rate Limit Exceeded

HTTP/1.1 429 Too Many Requests

{
  "detail": "Request was throttled. Expected available in 45 seconds."
}

Pagination

List endpoints support pagination:

GET /api/apps/?page=2&page_size=20

Response:

{
  "count": 50,
  "next": "http://api.example.com/api/apps/?page=3",
  "previous": "http://api.example.com/api/apps/?page=1",
  "results": [...]
}

Complete Workflow Example

1. Create App

curl -X POST https://api.example.com/api/apps/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "E-Commerce Platform",
    "description": "Main e-commerce application"
  }'

2. Create DataSource

curl -X POST https://api.example.com/api/apps/e-commerce-platform/datasources/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Product Database",
    "slug": "products-db",
    "provider_type": "flat_table"
  }'

3. Create DataTable

curl -X POST https://api.example.com/api/apps/e-commerce-platform/datasources/products-db/datatables/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "products",
    "json_schema": {
      "fields": [
        {"name": "id", "type": "integer", "constraints": {"required": true, "unique": true}},
        {"name": "name", "type": "string", "constraints": {"required": true}},
        {"name": "price", "type": "number", "constraints": {"required": true, "minimum": 0}},
        {"name": "in_stock", "type": "boolean"}
      ],
      "primaryKey": ["id"]
    }
  }'

4. Get Statistics

curl https://api.example.com/api/apps/e-commerce-platform/stats/ \
  -H "Authorization: Bearer YOUR_TOKEN"

Best Practices

Always validate schemas before deployment
Use slug-based lookups for stable URLs
Implement pagination for large result sets
Handle rate limits gracefully with exponential backoff
Monitor schema_hash for cache invalidation
Use soft deletes to maintain data history
Document your schemas with title and description fields

Rate Limiting Strategy

import time
import requests

def api_call_with_retry(url, headers, data, max_retries=3):
    for attempt in range(max_retries):
        response = requests.post(url, headers=headers, json=data)

        if response.status_code == 429:  # Rate limited
            retry_after = int(response.headers.get('Retry-After', 60))
            time.sleep(retry_after)
            continue

        return response

    raise Exception("Max retries exceeded")

Authentication​

Apps Endpoints​

List Apps​

Create App​

Get App Details​

Update App​

Delete App (Soft Delete)​

Get App Statistics​

Restore App​

DataSource Endpoints​

List DataSources​

Create DataSource​

Get DataSource Details​

Update DataSource​

Delete DataSource​

Get DataSource Tables​

Get DataSource Statistics​

DataTable Endpoints​

List DataTables​

Create DataTable​

Get DataTable Details​

Update DataTable​

Delete DataTable​

Get Schema Only​

Validate Data Against Schema​

Cross-DataSource Endpoints​

List All DataTables for App​

Get Cross-DataSource Statistics​

Error Responses​

Validation Errors​

Not Found​

Permission Denied​

Rate Limit Exceeded​

Pagination​

Complete Workflow Example​

1. Create App​

2. Create DataSource​

3. Create DataTable​

4. Get Statistics​

Best Practices​

Rate Limiting Strategy​

Authentication

Apps Endpoints

List Apps

Create App

Get App Details

Update App

Delete App (Soft Delete)

Get App Statistics

Restore App

DataSource Endpoints

List DataSources

Create DataSource

Get DataSource Details

Update DataSource

Delete DataSource

Get DataSource Tables

Get DataSource Statistics

DataTable Endpoints

List DataTables

Create DataTable

Get DataTable Details

Update DataTable

Delete DataTable

Get Schema Only

Validate Data Against Schema

Cross-DataSource Endpoints

List All DataTables for App

Get Cross-DataSource Statistics

Error Responses

Validation Errors

Not Found

Permission Denied

Rate Limit Exceeded

Pagination

Complete Workflow Example

1. Create App

2. Create DataSource

3. Create DataTable

4. Get Statistics

Best Practices

Rate Limiting Strategy