Scaling Content Moderation

A moderation setup that works at 1,000 messages a day breaks at a million: rate limits bite, per-call latency stacks up, and costs balloon. This guide shows how to wire Discuse moderation into high-throughput pipelines — when to block, when to check in the background, what to cache, and how to route around peak load.

What breaks when moderation scales up?

Each scaling problem has a specific cause and a specific fix:

Challenge	Impact	Solution
Volume growth	API rate limits, increased costs	Async processing, caching
Latency requirements	Poor user experience	Pre-moderation, queuing
Cost management	Budget constraints	Smart routing, caching
Consistency	Uneven enforcement	Centralized configuration
Peak handling	Service degradation	Auto-scaling, queues

Which architecture pattern should I use?

Pattern 1: Synchronous pre-moderation

Best for: Low-volume, high-stakes content (payments, legal)

User Input → API Call → Moderation → Decision → Response
                ↓
           (Blocking call)

// Simple synchronous flow
async function createPost(content) {
  // Check moderation before saving
  const result = await checkModeration(content);

  if (result.has_violations) {
    throw new ModerationError(result.message);
  }

  // Safe to publish
  return await savePost(content);
}

Pros: Simple, immediate feedback Cons: Adds latency, limits throughput

Pattern 2: Async post-moderation

Best for: High-volume, lower-stakes content (comments, messages)

User Input → Save (Pending) → Publish → Background Check → Action
                                              ↓
                                    ┌────────┴────────┐
                                    ↓                 ↓
                                  Safe            Violation
                                    ↓                 ↓
                                  Keep             Remove

// Async flow with queue
async function createPost(content) {
  // Save immediately with pending status
  const post = await savePost(content, { status: 'pending' });

  // Queue for async moderation
  await moderationQueue.add('check-content', {
    postId: post.id,
    content: content
  });

  return post;
}

// Background worker
moderationQueue.process('check-content', async (job) => {
  const result = await checkModeration(job.data.content);

  if (result.has_violations) {
    await removePost(job.data.postId);
    await notifyUser(job.data.postId, 'content_removed');
  } else {
    await updatePost(job.data.postId, { status: 'approved' });
  }
});

Pros: Non-blocking, handles volume Cons: Harmful content briefly visible

Pattern 3: Tiered moderation

Best for: Balanced approach with cost optimization

User Input → Quick Check → High Confidence? ──Yes──► Auto-decision
                              │
                              No
                              ↓
                         Full Analysis ──► Decision

async function tieredModeration(content) {
  // Tier 1: Fast local checks (regex, blocklists)
  const localResult = await quickLocalCheck(content);
  if (localResult.definiteViolation) {
    return { action: 'block', source: 'local' };
  }

  // Tier 2: Cached API results
  const cacheKey = hashContent(content);
  const cached = await cache.get(cacheKey);
  if (cached) {
    return { action: cached.action, source: 'cache' };
  }

  // Tier 3: Full API check
  const apiResult = await checkModeration(content);
  await cache.set(cacheKey, apiResult, TTL);

  return { action: determineAction(apiResult), source: 'api' };
}

Pattern 4: Fan-out with aggregation

Best for: Multiple check types with independent scaling

                    ┌──► Text Check ──┐
                    │                  │
User Input ──► Router ──► Image Check ──► Aggregator ──► Decision
                    │                  │
                    └──► Link Check ──┘

async function parallelModeration(content) {
  const checks = [];

  if (content.text) {
    checks.push(checkText(content.text));
  }

  if (content.images?.length) {
    checks.push(checkImages(content.images));
  }

  if (content.links?.length) {
    checks.push(checkLinks(content.links));
  }

  // Run all checks in parallel
  const results = await Promise.all(checks);

  // Aggregate results
  return aggregateResults(results);
}

function aggregateResults(results) {
  // Most severe result wins
  const hasViolation = results.some(r => r.has_violations);
  const maxScore = Math.max(...results.map(r => r.max_score || 0));

  return {
    has_violations: hasViolation,
    max_score: maxScore,
    details: results
  };
}

Caching strategies

Content-based caching

Cache results for identical content:

const cache = new Redis();

async function checkWithCache(content) {
  const hash = crypto.createHash('sha256')
    .update(content.text || '')
    .update(JSON.stringify(content.images || []))
    .digest('hex');

  // Check cache first
  const cached = await cache.get(`mod:${hash}`);
  if (cached) {
    return { ...JSON.parse(cached), cached: true };
  }

  // API call
  const result = await callModerationAPI(content);

  // Cache for 24 hours
  await cache.setex(`mod:${hash}`, 86400, JSON.stringify(result));

  return { ...result, cached: false };
}

User-based caching

Cache decisions for repeat offenders:

async function checkUserHistory(userId) {
  const recentViolations = await cache.get(`violations:${userId}`);

  if (recentViolations > 5) {
    // Flag for enhanced scrutiny
    return { enhancedModeration: true };
  }

  return { enhancedModeration: false };
}

async function recordViolation(userId) {
  await cache.incr(`violations:${userId}`);
  await cache.expire(`violations:${userId}`, 86400 * 7); // 7 days
}

Queue management

Priority queues

Handle different content types with different priorities:

const queues = {
  critical: new Queue('moderation-critical'),  // Reports, appeals
  high: new Queue('moderation-high'),          // Public posts
  normal: new Queue('moderation-normal'),      // Comments, DMs
  low: new Queue('moderation-low')             // Profile updates
};

async function queueForModeration(content, priority = 'normal') {
  const queue = queues[priority] || queues.normal;

  return queue.add('check', {
    contentId: content.id,
    type: content.type,
    data: content
  }, {
    priority: getPriorityNumber(priority),
    timeout: getTimeout(priority)
  });
}

Respecting rate limits

Each API key has a requests-per-minute limit; exceeding it makes the API reject the call with a "rate limit exceeded" error. Throttle on your side to stay under that limit and to avoid burning quota on redundant calls. Set the limiter's per-minute ceiling to match your key's configured RPM:

const Bottleneck = require('bottleneck');

const limiter = new Bottleneck({
  maxConcurrent: 50,                   // Cap parallel requests
  minTime: 20,                         // Min 20ms between requests
  reservoir: YOUR_KEY_RPM,             // Match your API key's per-minute limit
  reservoirRefreshAmount: YOUR_KEY_RPM,
  reservoirRefreshInterval: 60 * 1000  // Refill each minute
});

async function rateLimitedCheck(content) {
  return limiter.schedule(() => checkModeration(content));
}

Quota is separate from the rate limit: each call counts against your plan's monthly API-request allowance, and the response's usage object reports api_requests_used, api_requests_limit, and api_requests_remaining so you can watch headroom.

Dead-letter queues

Handle failed moderation attempts:

moderationQueue.process('check-content', async (job) => {
  try {
    const result = await checkModeration(job.data.content);
    await applyDecision(job.data.contentId, result);
  } catch (error) {
    if (job.attemptsMade < 3) {
      throw error; // Retry
    }

    // Move to dead letter queue after 3 attempts
    await deadLetterQueue.add('failed-moderation', {
      ...job.data,
      error: error.message,
      attempts: job.attemptsMade
    });

    // Apply safe default (hold for review)
    await holdForManualReview(job.data.contentId);
  }
});

Cost optimization

Smart routing

Only call paid APIs when necessary:

async function smartModeration(content) {
  // Step 1: Free local checks
  const localResult = runLocalFilters(content);
  if (localResult.isDefinitelySpam) {
    return { action: 'block', cost: 0 };
  }

  // Step 2: Check cache (free)
  const cached = await getCachedResult(content);
  if (cached) {
    return { action: cached.action, cost: 0 };
  }

  // Step 3: Risk-based API call
  const riskScore = calculateRiskScore(content, localResult);

  if (riskScore < 0.2) {
    // Low risk: approve without API call
    return { action: 'approve', cost: 0 };
  }

  // Step 4: API call for uncertain content
  const apiResult = await checkModeration(content);
  return {
    action: determineAction(apiResult),
    cost: calculateApiCost(content)
  };
}

Concurrency control

The /api/v2/check endpoint scores one submission per call. To "batch" work, group items in your worker and fan them out concurrently under a bounded pool, rather than expecting a single multi-item request. A check that includes several media URLs is itself one call covering all of them — one image-heavy call is cheaper than many single-image calls.

// Bounded concurrency: many items, capped parallel calls.
async function processBatch(items, concurrency = 10) {
  const results = [];
  for (let i = 0; i < items.length; i += concurrency) {
    const slice = items.slice(i, i + concurrency);
    const settled = await Promise.allSettled(
      slice.map(item => checkModeration(item.content))
    );
    results.push(...settled);
  }
  return results;
}

A single request can carry up to 10 image URLs, 5 GIF URLs, 3 video URLs, 5 document URLs, and 20 links, plus text up to 10,000 characters — so consolidate a submission's media into one call instead of splitting it.

Monitoring and alerting

Key metrics dashboard

const metrics = {
  // Volume metrics
  total_requests: new Counter('moderation_requests_total'),
  requests_per_second: new Gauge('moderation_rps'),

  // Latency metrics
  latency_p50: new Histogram('moderation_latency_p50'),
  latency_p99: new Histogram('moderation_latency_p99'),

  // Queue metrics
  queue_depth: new Gauge('moderation_queue_depth'),
  processing_time: new Histogram('moderation_processing_time'),

  // Cost metrics
  api_calls: new Counter('moderation_api_calls'),
  estimated_cost: new Counter('moderation_estimated_cost'),

  // Accuracy metrics
  violations_detected: new Counter('moderation_violations'),
  false_positives: new Counter('moderation_false_positives')
};

Alerting rules

# Alert on queue backup
- alert: ModerationQueueBackup
  expr: moderation_queue_depth > 10000
  for: 5m
  labels:
    severity: warning
  annotations:
    summary: Moderation queue backing up

# Alert on high latency
- alert: ModerationLatencyHigh
  expr: moderation_latency_p99 > 5000
  for: 2m
  labels:
    severity: critical
  annotations:
    summary: Moderation latency exceeding 5 seconds

# Alert on high error rate
- alert: ModerationErrorRate
  expr: rate(moderation_errors_total[5m]) / rate(moderation_requests_total[5m]) > 0.01
  for: 5m
  labels:
    severity: critical
  annotations:
    summary: Moderation error rate above 1%

Infrastructure scaling

Horizontal scaling

# Kubernetes HPA for moderation workers
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: moderation-worker
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: moderation-worker
  minReplicas: 3
  maxReplicas: 50
  metrics:
  - type: External
    external:
      metric:
        name: queue_depth
      target:
        type: AverageValue
        averageValue: 100

Reducing latency for global users

The Discuse API is reached at a single base URL, https://api.discuse.com. To keep latency low for global traffic, run your moderation workers close to your users and let them call the API, rather than calling it synchronously from a user-facing request path in a distant region. Combine this with the async post-moderation and caching patterns above so a slow round-trip never blocks a user.

Best practices summary

1. Cache aggressively: identical content returns a cached result. The API also deduplicates identical content per project within a short window and returns cached: true.
2. Use queues: async processing for high volume.
3. Implement tiers: local → cache → API.
4. Monitor everything: queue depth, latency, and usage against your quota.
5. Plan for failure: dead-letter queues and safe fallbacks.
6. Scale horizontally: more workers, not bigger machines.
7. Optimize cost: smart routing, fewer redundant calls, and consolidated media per request.

Next steps

• AI Content Moderation Guide - Understanding AI moderation
• Configuring Thresholds - Fine-tune detection
• Quick Start Guide - Basic implementation