Try Demo Sign In

Content Moderation API - A Developer's Guide

January 7, 2023 • By discuse

Content Moderation API Integration - A Developer’s Guide

Implementing effective content moderation isn’t just about choosing the right service—it’s about seamless technical integration that balances protection, performance, and user experience. This comprehensive developer’s guide walks through the practical aspects of integrating content moderation APIs into your application architecture. We’ll explore everything from authentication strategies and request optimization to handling edge cases and scaling for growth. Whether you’re building a social platform, marketplace, dating app, or any service with user-generated content, you’ll discover concrete code examples, architectural patterns, and performance optimization techniques specific to content moderation workflows. Learn how to implement pre-submission filtering, post-upload verification, batch processing, and real-time moderation with minimal latency impact. By following these implementation best practices, you’ll create content safety systems that protect users without compromising your application’s performance or developer velocity.

Understanding Moderation API Architecture

Before diving into implementation, it’s crucial to understand how moderation APIs typically work and where they fit in your application architecture.

Core Components of Content Moderation APIs

Modern content moderation systems like Discuse’s API consist of several key components:

  • Authentication Layer: Secures access through API keys or OAuth tokens
  • Request Handling: Processes incoming content analysis requests
  • Content Analysis Engine: The core AI/ML systems that evaluate content
  • Decision Engine: Applies your policies to the analysis results
  • Response Handling: Returns structured decisions and metadata
  • Webhook System: Provides asynchronous notifications for long-running processes

Integration Patterns

There are four primary patterns for integrating moderation into your application flow:

  1. Pre-submission Filtering: Content is analyzed before it enters your system
  2. Post-submission Analysis: Content is stored but flagged as unverified until moderation completes
  3. Background Moderation: Content is immediately visible but may be removed if flagged
  4. Hybrid Approaches: Combining patterns based on risk level and content type

Each pattern offers different tradeoffs between safety, user experience, and system complexity.

API Integration Basics

Authentication and Security

Secure integration begins with proper authentication:

// Example: Setting up authenticated requests with Discuse API
const discuseApi = axios.create({
  baseURL: 'https://api.discuse.com/v1',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${API_KEY}`,
    'X-Client-ID': CLIENT_ID
  }
});

Security Best Practices:

  • Store API keys in environment variables, never in source code
  • Implement key rotation schedules to minimize risk from compromised credentials
  • Use IP whitelisting where available to restrict API access to your servers
  • Create separate API keys for staging and production environments

Basic Request Structure

A typical moderation request contains:

// Example: Basic text moderation request
const moderationRequest = {
  content: {
    text: userSubmittedContent,
    type: 'plain_text'
  },
  context: {
    userId: user.id,
    contentId: post.id,
    locale: user.locale || 'en-US'
  },
  moderationProfile: 'standard', // or custom profile name
  synchronous: true
};

const response = await discuseApi.post('/moderate', moderationRequest);

Key Request Fields:

  • content: The actual text, image, video, or other media to moderate
  • context: Metadata to help with moderation decisions and tracking
  • moderationProfile: Which rule set to apply
  • synchronous: Whether to wait for results or handle via webhook

Handling API Responses

Moderation API responses typically include:

// Example response structure
{
  "requestId": "mod_1234567890",
  "status": "success",
  "decision": {
    "result": "approved", // or "rejected", "flagged"
    "confidence": 0.98,
    "categories": {
      "harassment": 0.01,
      "violence": 0.005,
      "hate_speech": 0.002,
      "sexual": 0.001
    }
  },
  "actionRecommendation": "allow",
  "processingTimeMs": 78,
  "timestamp": "2023-06-12T15:23:47Z"
}

Implementing Decision Handling:

// Example: Processing moderation decisions
function handleModerationResponse(response, content) {
  switch(response.decision.result) {
    case 'approved':
      publishContent(content);
      break;
    case 'rejected':
      notifyUserOfRejection(content, response.decision.categories);
      storeRejectionForAnalysis(content, response);
      break;
    case 'flagged':
      publishWithRestrictions(content);
      queueForHumanReview(content, response);
      break;
    default:
      handleError('Unknown moderation result');
  }
}

Implementing Common Moderation Workflows

Let’s explore concrete implementation approaches for the most common moderation scenarios.

Pre-submission Text Moderation

This pattern checks content before it’s submitted, providing immediate feedback:

// Client-side implementation
async function checkTextBeforeSubmission(text) {
  const checkResult = await apiClient.post('/pre-check', { text });
  
  if (checkResult.isPotentiallyProblematic) {
    showWarningToUser("This content may violate our community guidelines");
    requireConfirmation();
  }
}

// Server-side validation
async function handlePostSubmission(req, res) {
  try {
    // First validate the submission normally
    validateSubmissionFormat(req.body);
    
    // Then check content
    const moderationResult = await discuseApi.post('/moderate', {
      content: { text: req.body.content, type: 'plain_text' },
      context: { userId: req.user.id }
    });
    
    if (moderationResult.decision.result === 'rejected') {
      return res.status(400).json({
        error: 'Content violates community guidelines',
        details: mapCategoriesToUserFriendlyMessages(moderationResult.decision.categories)
      });
    }
    
    // Continue with normal submission flow
    const post = await database.posts.create({...});
    return res.status(201).json({ post });
  } catch (error) {
    handleErrorAndResponse(error, res);
  }
}

Image Moderation with Upload Flow

For image moderation, consider this implementation pattern:

// Server-side implementation
async function handleImageUpload(req, res) {
  try {
    // First store the image with a pending status
    const imageFile = req.files.image;
    const uploadResult = await storageService.store(imageFile, {
      visibility: 'private', // Keep private until moderated
      metadata: { moderationStatus: 'pending' }
    });
    
    // Then send for moderation
    const moderationResult = await discuseApi.post('/moderate', {
      content: { 
        image: {
          url: uploadResult.privateUrl, // Temporary signed URL for the API to access
          type: imageFile.mimetype
        }
      },
      context: { userId: req.user.id, contentId: uploadResult.id }
    });
    
    if (moderationResult.decision.result === 'approved') {
      // Update visibility and status
      await storageService.update(uploadResult.id, {
        visibility: 'public',
        metadata: { moderationStatus: 'approved' }
      });
      
      return res.status(201).json({
        image: { id: uploadResult.id, url: uploadResult.publicUrl }
      });
    } else {
      // Clean up rejected image
      await storageService.delete(uploadResult.id);
      
      return res.status(400).json({
        error: 'Image violates community guidelines',
        details: mapCategoriesToUserFriendlyMessages(moderationResult.decision.categories)
      });
    }
  } catch (error) {
    handleErrorAndResponse(error, res);
  }
}

Asynchronous Video Moderation

For longer-form content like videos, asynchronous processing is preferred:

// Initial submission
async function submitVideoForModeration(videoId) {
  // Update video status
  await database.videos.update(videoId, { moderationStatus: 'processing' });
  
  // Request moderation
  await discuseApi.post('/moderate/async', {
    content: { 
      video: {
        url: getSignedUrl(videoId),
        type: 'video/mp4',
        duration: videoDuration
      }
    },
    context: { contentId: videoId },
    webhook: {
      url: `${API_BASE_URL}/webhooks/moderation-callback`,
      headers: { 'X-Webhook-Secret': WEBHOOK_SECRET }
    }
  });
}

// Webhook handler
async function handleModerationWebhook(req, res) {
  // Validate webhook signature
  if (!isValidWebhookSignature(req)) {
    return res.status(401).send('Invalid signature');
  }
  
  const { contentId, decision } = req.body;
  
  // Update content status based on decision
  if (decision.result === 'approved') {
    await database.videos.update(contentId, { 
      moderationStatus: 'approved',
      isPublic: true
    });
    
    // Notify user of successful processing
    await notificationService.notify(video.userId, {
      type: 'video_approved',
      videoId: contentId
    });
  } else {
    await database.videos.update(contentId, { 
      moderationStatus: 'rejected',
      rejectionReason: mapDecisionToReason(decision)
    });
    
    // Notify user of rejection
    await notificationService.notify(video.userId, {
      type: 'video_rejected',
      videoId: contentId,
      reason: mapDecisionToUserFriendlyMessage(decision)
    });
  }
  
  return res.status(200).send('Webhook processed');
}

Performance Optimization Strategies

Moderation API integration requires thoughtful performance optimization to maintain responsiveness.

Batch Processing

For applications that need to check multiple content items:

// Batch processing example
async function moderateBatch(contentItems) {
  const batchRequest = {
    items: contentItems.map(item => ({
      id: item.id,
      content: { 
        text: item.text,
        type: 'plain_text'
      },
      context: { userId: item.userId }
    })),
    moderationProfile: 'standard'
  };
  
  const batchResult = await discuseApi.post('/moderate/batch', batchRequest);
  
  // Process each result
  return batchResult.items.map(result => ({
    contentId: result.id,
    approved: result.decision.result === 'approved',
    decision: result.decision
  }));
}

Parallel Processing

For multi-part content like a post with text and images:

// Parallel moderation processing
async function moderateComplexPost(post) {
  // Start all moderation requests in parallel
  const [textResult, imageResults] = await Promise.all([
    discuseApi.post('/moderate', {
      content: { text: post.text, type: 'plain_text' }
    }),
    
    // Process all images in parallel too
    Promise.all(post.images.map(image => 
      discuseApi.post('/moderate', {
        content: { 
          image: { url: image.url, type: image.type } 
        }
      })
    ))
  ]);
  
  // Combine results
  const allApproved = textResult.decision.result === 'approved' && 
    imageResults.every(result => result.decision.result === 'approved');
    
  return {
    approved: allApproved,
    textDecision: textResult.decision,
    imageDecisions: imageResults.map(r => r.decision)
  };
}

Caching Strategies

Implement caching to avoid re-moderating identical content:

// Caching implementation
const moderationCache = new LRUCache({
  max: 10000,
  ttl: 1000 * 60 * 60 * 24 // 24 hour cache
});

async function moderateWithCaching(content) {
  // Generate cache key based on content hash
  const contentHash = createHash('sha256')
    .update(JSON.stringify(content))
    .digest('hex');
    
  // Check cache first
  const cachedResult = moderationCache.get(contentHash);
  if (cachedResult) {
    recordCacheHit(contentHash);
    return cachedResult;
  }
  
  // If not in cache, perform moderation
  const result = await discuseApi.post('/moderate', {
    content: content
  });
  
  // Cache the result
  moderationCache.set(contentHash, result);
  
  return result;
}

Error Handling and Resilience

Robust error handling is critical for moderation systems.

Graceful Degradation

Implement fallback options when moderation services are unavailable:

async function moderateWithFallback(content) {
  try {
    return await discuseApi.post('/moderate', { content });
  } catch (error) {
    logModerationError(error);
    
    if (isServiceUnavailable(error)) {
      // Implement fallback strategy based on risk tolerance:
      
      // Option 1: Fail closed - reject content
      // return { decision: { result: 'rejected', reason: 'service_unavailable' } };
      
      // Option 2: Fail open but mark for later review
      return { 
        decision: { 
          result: 'flagged',
          reason: 'manual_review_required' 
        }
      };
    }
    
    // Rethrow other errors
    throw error;
  }
}

Retry Strategies

Implement exponential backoff for transient failures:

async function moderateWithRetry(content, maxRetries = 3) {
  let attempt = 0;
  
  while (attempt < maxRetries) {
    try {
      return await discuseApi.post('/moderate', { content });
    } catch (error) {
      attempt++;
      
      if (!isRetryableError(error) || attempt >= maxRetries) {
        throw error;
      }
      
      // Exponential backoff with jitter
      const delayMs = Math.min(
        100 * Math.pow(2, attempt) + Math.random() * 100,
        2000 // Max 2 second delay
      );
      
      await new Promise(resolve => setTimeout(resolve, delayMs));
    }
  }
}

Circuit Breaker Pattern

Protect your systems when moderation services experience prolonged issues:

// Example using Resilience4js in Node.js
const circuitBreaker = new CircuitBreaker('moderation', {
  failureThreshold: 0.3, // 30% failures trigger open circuit
  resetTimeout: 30000,   // Try again after 30 seconds
  fallback: content => ({ 
    decision: { 
      result: 'flagged',
      confidence: 0,
      reason: 'circuit_open'
    }
  })
});

async function moderateWithCircuitBreaker(content) {
  return circuitBreaker.execute(() => 
    discuseApi.post('/moderate', { content })
  );
}

Monitoring and Observability

Effective monitoring is essential for moderation systems.

Key Metrics to Track

Set up dashboards tracking these critical metrics:

// Example instrumentation middleware
function instrumentModerationRequest(req, res, next) {
  const startTime = Date.now();
  
  // Add to request counter
  metrics.increment('moderation.requests.total', {
    contentType: req.body.content.type
  });
  
  // Original handler
  const originalEnd = res.end;
  res.end = function(...args) {
    const duration = Date.now() - startTime;
    
    // Track latency
    metrics.timing('moderation.requests.duration', duration, {
      contentType: req.body.content.type
    });
    
    // Track results
    if (res.statusCode >= 200 && res.statusCode < 300) {
      metrics.increment('moderation.requests.success');
      
      // Track decision types
      if (res.locals.moderationResult) {
        metrics.increment(`moderation.decision.${res.locals.moderationResult.decision.result}`);
      }
    } else {
      metrics.increment('moderation.requests.error', {
        statusCode: res.statusCode
      });
    }
    
    originalEnd.apply(res, args);
  };
  
  next();
}

Logging Strategy

Implement structured logging for moderation:

function logModerationActivity(userContext, content, moderationResult) {
  logger.info('Content moderation decision', {
    userId: userContext.userId,
    contentId: userContext.contentId,
    contentType: content.type,
    decision: moderationResult.decision.result,
    confidence: moderationResult.decision.confidence,
    processingTimeMs: moderationResult.processingTimeMs,
    categories: Object.entries(moderationResult.decision.categories)
      .filter(([_, score]) => score > 0.1)
      .map(([category, score]) => ({ category, score }))
  });
}

Security Considerations

Content moderation systems handle sensitive data and need specific security protections.

Data Protection

Implement proper data handling practices:

// Example: Data minimization
function prepareContentForModeration(userContent) {
  return {
    // Include only what's needed for moderation
    text: userContent.text,
    
    // Anonymize where possible
    context: {
      // Use hashed or tokenized identifiers when possible
      userIdentifier: hashUserId(userContent.userId),
      
      // Include only relevant metadata
      contentType: userContent.type,
      locale: userContent.locale
    }
  };
}

Webhook Security

Secure webhook endpoints:

// Example: Webhook signature validation
function verifyWebhookSignature(req) {
  const signature = req.headers['x-discuse-signature'];
  const timestamp = req.headers['x-discuse-timestamp'];
  
  // Check timestamp freshness (prevent replay attacks)
  const now = Math.floor(Date.now() / 1000);
  if (Math.abs(now - parseInt(timestamp)) > 300) { // 5 minute tolerance
    return false;
  }
  
  // Calculate expected signature
  const payload = timestamp + '.' + JSON.stringify(req.body);
  const expectedSignature = createHmac('sha256', WEBHOOK_SECRET)
    .update(payload)
    .digest('hex');
    
  // Constant-time comparison to prevent timing attacks
  return timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(expectedSignature)
  );
}

Real-World Integration Examples

Web Application Example

// Express middleware for moderation
function moderationMiddleware(req, res, next) {
  // Skip moderation for certain trusted users if appropriate
  if (req.user && req.user.isTrusted && req.user.noModerationViolations) {
    return next();
  }
  
  // Check if content requires moderation
  if (!contentRequiresModeration(req.body)) {
    return next();
  }
  
  // Extract content for moderation
  const content = extractModerableContent(req.body);
  
  // Perform moderation
  discuseApi.post('/moderate', {
    content: content,
    context: {
      userId: req.user?.id || 'anonymous',
      contentId: req.body.id,
      clientIp: getClientIp(req),
      userAgent: req.headers['user-agent']
    }
  })
  .then(result => {
    if (result.decision.result === 'rejected') {
      // Log rejection
      logContentRejection(req.user?.id, content, result);
      
      // Return appropriate error
      return res.status(400).json({
        error: 'Content violates community guidelines',
        details: mapDecisionToUserFriendlyMessage(result.decision)
      });
    }
    
    // For approved or flagged content, continue but store decision
    req.moderationResult = result;
    next();
  })
  .catch(error => {
    // Log error
    logModerationError(error);
    
    // Determine fallback behavior based on error type
    if (shouldBlockOnError(error)) {
      return res.status(500).json({
        error: 'Content could not be processed at this time'
      });
    }
    
    // Flag for manual review but allow to continue
    req.moderationResult = {
      decision: {
        result: 'flagged',
        reason: 'moderation_service_error'
      }
    };
    next();
  });
}

Mobile Application Example

// React Native example for client-side pre-check
function ContentSubmissionScreen() {
  const [content, setContent] = useState('');
  const [isChecking, setIsChecking] = useState(false);
  const [contentWarning, setContentWarning] = useState(null);
  
  // Debounced pre-check function
  const checkContent = useCallback(
    debounce(async (text) => {
      if (text.length < 5) return;
      
      setIsChecking(true);
      try {
        const result = await apiClient.post('/pre-check', { text });
        
        if (result.isPotentiallyProblematic) {
          setContentWarning({
            message: 'This content may violate our community guidelines.',
            categories: result.categories
          });
        } else {
          setContentWarning(null);
        }
      } catch (error) {
        console.error('Pre-check failed:', error);
      } finally {
        setIsChecking(false);
      }
    }, 500),
    []
  );
  
  // Call check when content changes
  useEffect(() => {
    checkContent(content);
  }, [content, checkContent]);
  
  async function handleSubmit() {
    // Final server-side check happens during submission API call
    // ...
  }
  
  return (
    <View>
      <TextInput
        value={content}
        onChangeText={setContent}
        placeholder="Write your post..."
        multiline
      />
      
      {isChecking && <ActivityIndicator size="small" />}
      
      {contentWarning && (
        <View style={styles.warningContainer}>
          <Text style={styles.warningText}>{contentWarning.message}</Text>
          {/* Display specific category warnings */}
        </View>
      )}
      
      <Button 
        title="Submit" 
        onPress={handleSubmit} 
        disabled={isChecking || !content.trim()}
      />
    </View>
  );
}

Conclusion: Building a Moderation System That Scales

Implementing content moderation is a multifaceted challenge that requires balancing technical performance, user experience, and safety. The core principles to remember:

  1. Design for resilience. Moderation failures should never bring down your entire system.

  2. Layer your approach. Combine pre-submission guidance, real-time filtering, and post-publication monitoring for comprehensive protection.

  3. Optimize for performance. Use caching, batching, and asynchronous processing where appropriate to minimize latency impact.

  4. Be transparent with users. Clear explanations for moderation decisions improve the user experience and reduce frustration.

  5. Instrument everything. Comprehensive logging and metrics are essential for optimizing moderation effectiveness.

Discuse’s content moderation API is designed specifically for developer-friendly integration, with thorough documentation, reference implementations, and a support team experienced in helping platforms implement effective content safety systems.

Our API’s consistent response formats, reliable performance, and comprehensive error handling make it the ideal foundation for building scalable content moderation systems that grow with your platform.

Contact our developer relations team today to discuss your specific moderation requirements and get personalized integration guidance for your platform.

Continue Reading

Back to all articles

Browse our complete collection of content moderation resources

Share this article
Twitter LinkedIn Email

Ready to see Discuse in action?

Experience the industry's most powerful content moderation API with a personalized demo.

Request a demo View documentation