Building a RESTful API with Node.js and Express: What 3 Years of Production Taught Me

Last March, our startup's API went from handling 500,000 requests per day to over 10 million practically overnight. We'd landed a major client, and their integration team was already building against our endpoints. I remember sitting in our office at 2 AM on a Friday, watching our response times climb from 200ms to 8 seconds, then watching the entire API collapse under load.

That night changed everything I thought I knew about building APIs with Node.js and Express.

I'd built dozens of REST APIs before. I knew the basics—routes, middleware, controllers, all that stuff. But there's a massive gap between building an API that works on your laptop and building one that survives real production traffic. The tutorials don't tell you about connection pool exhaustion. The documentation doesn't mention that bodyParser has a default limit that'll bite you when clients start sending larger payloads. Nobody warns you that your error handling strategy will determine whether you spend weekends debugging or actually sleeping.

Here's what I learned building and scaling RESTful APIs that now serve over 50 million requests daily, support 200,000+ active users, and maintain 99.97% uptime. This isn't theory—these are the patterns, mistakes, and hard-won lessons from three years in the trenches.

The Problem with Most Express API Tutorials

Most Express tutorials follow the same pattern. They show you how to set up a basic server, add a few routes, maybe throw in some middleware, and call it done. You end up with something like this:

const express = require('express');
const app = express();

app.use(express.json());

app.get('/api/users', (req, res) => {
  res.json({ users: [] });
});

app.listen(3000);

This works fine for demos. It completely falls apart in production.

When we first deployed our API, we used this exact pattern. Within two weeks, we had our first major incident. A client accidentally sent a request with a 50MB JSON payload. Our server tried to parse it, ran out of memory, and crashed. The process manager restarted it, but the client retried immediately. Crash, restart, retry, crash. We were stuck in a crash loop for 20 minutes before I figured out what was happening.

The fix was simple—add a payload size limit:

app.use(express.json({ limit: '1mb' }));

But that incident taught me something crucial: production APIs need to be defensive from day one. You can't assume clients will behave nicely. You can't assume network conditions will be perfect. You can't assume your database will always respond quickly.

Starting with the Right Foundation

Let me show you how I structure Express APIs now. This is the foundation that's survived three years of production use, multiple scaling challenges, and enough edge cases to fill a book.

First, the project structure matters more than you think. Here's what we use:

src/
  config/
    database.js
    redis.js
    logger.js
  middleware/
    auth.js
    errorHandler.js
    rateLimiter.js
    requestLogger.js
    validator.js
  models/
    User.js
    Product.js
  routes/
    index.js
    users.js
    products.js
  services/
    UserService.js
    ProductService.js
  utils/
    errors.js
    response.js
  app.js
  server.js
tests/
  integration/
  unit/

This structure emerged after we hit 1 million users. Before that, we had everything in one giant index.js file. Bad idea. When you need to add caching, implement rate limiting, or debug a production issue at 3 AM, you'll thank yourself for organizing properly.

Here's the server.js that starts everything:

const app = require('./app');
const logger = require('./config/logger');
const config = require('./config');

const PORT = config.port || 3000;
const ENV = config.env || 'development';

const server = app.listen(PORT, () => {
  logger.info(`API server started on port ${PORT} in ${ENV} mode`);
  logger.info(`Worker process ${process.pid} is running`);
});

// Graceful shutdown handling - this saved us during deployments
process.on('SIGTERM', () => {
  logger.info('SIGTERM received, starting graceful shutdown');
  
  server.close(() => {
    logger.info('HTTP server closed');
    
    // Close database connections
    require('./config/database').close();
    
    // Close Redis connections
    require('./config/redis').quit();
    
    process.exit(0);
  });
  
  // Force shutdown after 30 seconds
  setTimeout(() => {
    logger.error('Forced shutdown after timeout');
    process.exit(1);
  }, 30000);
});

process.on('unhandledRejection', (reason, promise) => {
  logger.error('Unhandled Rejection at:', promise, 'reason:', reason);
  // Don't exit in production - log and continue
  if (ENV !== 'production') {
    process.exit(1);
  }
});

process.on('uncaughtException', (error) => {
  logger.error('Uncaught Exception:', error);
  // Exit process - let process manager restart it
  process.exit(1);
});

That graceful shutdown handling? We added it after a deployment caused active requests to fail mid-processing. Users were getting 500 errors during our deploy window. Now we wait for active requests to complete before shutting down. Our deploy error rate dropped from 2.3% to 0.01%.

The Middleware Stack That Actually Works in Production

Middleware order matters. I learned this the hard way when our authentication middleware was running after our rate limiter. Unauthenticated requests were consuming our rate limit quota. We were essentially DDoS-ing ourselves.

Here's the middleware stack we use now, in the exact order that matters:

const express = require('express');
const helmet = require('helmet');
const cors = require('cors');
const compression = require('compression');
const morgan = require('morgan');
const rateLimit = require('express-rate-limit');

const errorHandler = require('./middleware/errorHandler');
const requestLogger = require('./middleware/requestLogger');
const { authenticate } = require('./middleware/auth');

const app = express();

// 1. Security headers - ALWAYS FIRST
app.use(helmet({
  contentSecurityPolicy: {
    directives: {
      defaultSrc: ["'self'"],
      styleSrc: ["'self'", "'unsafe-inline'"]
    }
  },
  hsts: {
    maxAge: 31536000,
    includeSubDomains: true,
    preload: true
  }
}));

// 2. CORS - before any routes
app.use(cors({
  origin: process.env.ALLOWED_ORIGINS?.split(',') || '*',
  credentials: true,
  maxAge: 86400 // 24 hours
}));

// 3. Compression - early for all responses
app.use(compression({
  filter: (req, res) => {
    if (req.headers['x-no-compression']) {
      return false;
    }
    return compression.filter(req, res);
  },
  threshold: 1024 // Only compress responses > 1KB
}));

// 4. Body parsing with limits
app.use(express.json({ 
  limit: '1mb',
  verify: (req, res, buf) => {
    req.rawBody = buf.toString('utf8');
  }
}));
app.use(express.urlencoded({ 
  extended: true, 
  limit: '1mb' 
}));

// 5. Request logging
app.use(requestLogger);

// 6. Rate limiting - BEFORE authentication
const limiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // limit each IP to 100 requests per windowMs
  message: 'Too many requests from this IP',
  standardHeaders: true,
  legacyHeaders: false,
  // Skip rate limiting for specific IPs
  skip: (req) => {
    const whitelist = process.env.RATE_LIMIT_WHITELIST?.split(',') || [];
    return whitelist.includes(req.ip);
  }
});

app.use('/api/', limiter);

// 7. Routes
app.use('/api', require('./routes'));

// 8. 404 handler
app.use((req, res) => {
  res.status(404).json({
    error: 'Not Found',
    message: `Route ${req.method} ${req.path} not found`,
    timestamp: new Date().toISOString()
  });
});

// 9. Error handler - ALWAYS LAST
app.use(errorHandler);

module.exports = app;

Let me explain why this order matters, because we got it wrong for six months.

Security headers first: We had a security audit in Q3 2024 that flagged missing security headers. Adding Helmet was easy, but we initially put it after body parsing. The auditor pointed out that certain attacks could exploit the parser before security headers were set. Moving Helmet to position #1 was a one-line change that closed several attack vectors.

CORS before routes: We spent two days debugging why our frontend couldn't access the API. Turns out our CORS middleware was after the routes, so preflight requests were hitting our 404 handler. Moving CORS up fixed it immediately.

Compression early: We added compression after noticing our bandwidth costs were climbing. Our API responses were averaging 50KB uncompressed. With compression, they dropped to 8KB—an 84% reduction. But we initially added compression late in the middleware stack, after authentication. This meant unauthenticated requests weren't being compressed. Moving it earlier saved us about $800/month in bandwidth costs.

Rate limiting before authentication: This was the big one. We were rate-limiting AFTER authentication, which meant attackers could hammer our database with authentication attempts before hitting rate limits. Moving rate limiting before auth reduced our database load by 40% during attack attempts.

Error Handling That Actually Helps You Debug

Here's something nobody tells you: your error handling strategy determines how fast you can fix production issues.

We used to have error handling like this:

app.use((err, req, res, next) => {
  console.error(err);
  res.status(500).json({ error: 'Something went wrong' });
});

This is useless in production. When something breaks at 2 AM, you need to know exactly what happened, where it happened, and what the user was doing. Generic error messages don't cut it.

Here's our production error handler:

// utils/errors.js
class AppError extends Error {
  constructor(message, statusCode, isOperational = true) {
    super(message);
    this.statusCode = statusCode;
    this.isOperational = isOperational;
    this.timestamp = new Date().toISOString();
    Error.captureStackTrace(this, this.constructor);
  }
}

class ValidationError extends AppError {
  constructor(message, errors = []) {
    super(message, 400);
    this.errors = errors;
  }
}

class AuthenticationError extends AppError {
  constructor(message = 'Authentication required') {
    super(message, 401);
  }
}

class AuthorizationError extends AppError {
  constructor(message = 'Insufficient permissions') {
    super(message, 403);
  }
}

class NotFoundError extends AppError {
  constructor(resource = 'Resource') {
    super(`${resource} not found`, 404);
  }
}

class ConflictError extends AppError {
  constructor(message) {
    super(message, 409);
  }
}

class RateLimitError extends AppError {
  constructor(message = 'Rate limit exceeded') {
    super(message, 429);
  }
}

module.exports = {
  AppError,
  ValidationError,
  AuthenticationError,
  AuthorizationError,
  NotFoundError,
  ConflictError,
  RateLimitError
};

And the error handler middleware:

// middleware/errorHandler.js
const logger = require('../config/logger');
const { AppError } = require('../utils/errors');

module.exports = (err, req, res, next) => {
  // Default to 500 if status code not set
  err.statusCode = err.statusCode || 500;
  err.status = err.status || 'error';

  // Log error details
  const errorLog = {
    timestamp: new Date().toISOString(),
    method: req.method,
    url: req.originalUrl,
    ip: req.ip,
    userId: req.user?.id,
    statusCode: err.statusCode,
    message: err.message,
    stack: err.stack,
    body: req.body,
    query: req.query,
    params: req.params
  };

  if (err.statusCode >= 500) {
    logger.error('Server Error:', errorLog);
  } else {
    logger.warn('Client Error:', errorLog);
  }

  // Don't leak error details in production
  if (process.env.NODE_ENV === 'production') {
    // Operational errors - safe to send to client
    if (err.isOperational) {
      return res.status(err.statusCode).json({
        status: err.status,
        message: err.message,
        ...(err.errors && { errors: err.errors })
      });
    }

    // Programming or unknown errors - don't leak details
    logger.error('Non-operational error:', err);
    return res.status(500).json({
      status: 'error',
      message: 'Internal server error'
    });
  }

  // Development - send full error details
  res.status(err.statusCode).json({
    status: err.status,
    message: err.message,
    stack: err.stack,
    error: err,
    request: {
      body: req.body,
      query: req.query,
      params: req.params
    }
  });
};

This error handling saved us countless hours. When we had an issue where certain product searches were failing, the logs showed us:

{
  "timestamp": "2024-03-15T14:23:45.123Z",
  "method": "GET",
  "url": "/api/products/search",
  "ip": "192.168.1.100",
  "userId": "user_12345",
  "statusCode": 500,
  "message": "Invalid regular expression",
  "query": { "q": "laptop [unclosed" },
  "stack": "Error: Invalid regular expression..."
}

We immediately saw the issue: users were entering search queries with unescaped regex characters. We added input sanitization and solved it in 15 minutes. With generic error handling, we'd have spent hours reproducing the issue.

Database Integration That Doesn't Fall Over

Our first production database integration was a disaster. We were using a simple connection pattern:

// DON'T DO THIS
const { Pool } = require('pg');
const pool = new Pool({
  connectionString: process.env.DATABASE_URL
});

app.get('/api/users', async (req, res) => {
  const result = await pool.query('SELECT * FROM users');
  res.json(result.rows);
});

This worked fine in development. In production, it failed spectacularly.

The problem? Connection pool exhaustion. Under load, we'd open connections faster than we could close them. Eventually, we'd hit the connection limit and new requests would hang indefinitely. Users would see timeouts. Our monitoring showed connection pool utilization at 100%.

Here's what actually works:

// config/database.js
const { Pool } = require('pg');
const logger = require('./logger');

const pool = new Pool({
  host: process.env.DB_HOST,
  port: process.env.DB_PORT || 5432,
  database: process.env.DB_NAME,
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  
  // Connection pool settings - these numbers came from load testing
  max: 20, // Maximum connections
  min: 5,  // Minimum connections
  idleTimeoutMillis: 30000, // Close idle connections after 30s
  connectionTimeoutMillis: 5000, // Fail fast if can't get connection
  
  // Statement timeout - kill queries after 30s
  statement_timeout: 30000,
  
  // Query timeout - kill queries after 30s
  query_timeout: 30000
});

// Monitor pool health
pool.on('connect', (client) => {
  logger.debug('New database client connected');
});

pool.on('error', (err, client) => {
  logger.error('Unexpected database error:', err);
});

pool.on('remove', (client) => {
  logger.debug('Database client removed from pool');
});

// Health check query
const healthCheck = async () => {
  try {
    const result = await pool.query('SELECT NOW()');
    return { healthy: true, timestamp: result.rows[0].now };
  } catch (error) {
    logger.error('Database health check failed:', error);
    return { healthy: false, error: error.message };
  }
};

// Graceful shutdown
const close = async () => {
  logger.info('Closing database pool');
  await pool.end();
  logger.info('Database pool closed');
};

module.exports = {
  query: (text, params) => pool.query(text, params),
  getClient: () => pool.connect(),
  healthCheck,
  close
};

Those pool settings came from painful experience. Initially, we had max: 100 because we thought "more is better." Wrong. Our database server had a hard limit of 100 connections, and we were running 4 API servers. Do the math: 4 servers × 100 connections = 400 connections, but the database only allowed 100. Connections would hang waiting for availability.

We load-tested different pool sizes and found that max: 20 gave us the best throughput without overwhelming the database. With 4 servers, that's 80 total connections—well under the database limit with room for other services.

The statement timeout is crucial. We had a bug where a poorly written query would scan the entire users table (2 million rows) without an index. The query would run for 5+ minutes, holding a connection and blocking other requests. Adding statement timeouts meant these queries would fail fast instead of degrading the entire API.

Here's how we use this in routes:

// routes/users.js
const express = require('express');
const router = express.Router();
const db = require('../config/database');
const { NotFoundError, ValidationError } = require('../utils/errors');
const { authenticate } = require('../middleware/auth');

// Get user by ID
router.get('/:id', authenticate, async (req, res, next) => {
  try {
    const { id } = req.params;
    
    // Validate ID format
    if (!id.match(/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i)) {
      throw new ValidationError('Invalid user ID format');
    }
    
    const result = await db.query(
      'SELECT id, email, name, created_at FROM users WHERE id = $1',
      [id]
    );
    
    if (result.rows.length === 0) {
      throw new NotFoundError('User');
    }
    
    res.json({
      status: 'success',
      data: result.rows[0]
    });
  } catch (error) {
    next(error);
  }
});

// Create user with transaction
router.post('/', authenticate, async (req, res, next) => {
  const client = await db.getClient();
  
  try {
    const { email, name, password } = req.body;
    
    // Validate input
    if (!email || !name || !password) {
      throw new ValidationError('Missing required fields', [
        { field: 'email', message: 'Email is required' },
        { field: 'name', message: 'Name is required' },
        { field: 'password', message: 'Password is required' }
      ]);
    }
    
    await client.query('BEGIN');
    
    // Check if user exists
    const existingUser = await client.query(
      'SELECT id FROM users WHERE email = $1',
      [email]
    );
    
    if (existingUser.rows.length > 0) {
      throw new ConflictError('User with this email already exists');
    }
    
    // Hash password (using bcrypt in real app)
    const hashedPassword = await hashPassword(password);
    
    // Insert user
    const result = await client.query(
      `INSERT INTO users (email, name, password_hash, created_at) 
       VALUES ($1, $2, $3, NOW()) 
       RETURNING id, email, name, created_at`,
      [email, name, hashedPassword]
    );
    
    // Create user profile
    await client.query(
      'INSERT INTO user_profiles (user_id, bio, avatar_url) VALUES ($1, $2, $3)',
      [result.rows[0].id, '', null]
    );
    
    await client.query('COMMIT');
    
    res.status(201).json({
      status: 'success',
      data: result.rows[0]
    });
  } catch (error) {
    await client.query('ROLLBACK');
    next(error);
  } finally {
    client.release();
  }
});

module.exports = router;

Notice the transaction handling? We learned this after a bug where user creation would succeed but profile creation would fail, leaving orphaned user records. Wrapping both operations in a transaction ensures they either both succeed or both fail.

Also notice we're always releasing the client in the finally block. We had a leak where errors would prevent client release, slowly exhausting the connection pool. The finally block ensures cleanup happens even if an error occurs.

Authentication and Authorization That Scales

We started with session-based authentication using express-session. It worked until we scaled to multiple servers. Sessions were stored in memory, so users would be logged in on server A but not server B. Load balancer would route them to different servers randomly. Nightmare.

We switched to JWT tokens and haven't looked back:

// middleware/auth.js
const jwt = require('jsonwebtoken');
const { AuthenticationError, AuthorizationError } = require('../utils/errors');
const redis = require('../config/redis');

const JWT_SECRET = process.env.JWT_SECRET;
const JWT_EXPIRY = process.env.JWT_EXPIRY || '24h';

// Generate JWT token
const generateToken = (user) => {
  return jwt.sign(
    {
      id: user.id,
      email: user.email,
      role: user.role
    },
    JWT_SECRET,
    {
      expiresIn: JWT_EXPIRY,
      issuer: 'api.yourapp.com',
      audience: 'yourapp.com'
    }
  );
};

// Verify JWT token
const verifyToken = (token) => {
  try {
    return jwt.verify(token, JWT_SECRET, {
      issuer: '

api.yourapp.com',
      audience: 'yourapp.com'
    });
  } catch (error) {
    if (error.name === 'TokenExpiredError') {
      throw new AuthenticationError('Token has expired');
    }
    if (error.name === 'JsonWebTokenError') {
      throw new AuthenticationError('Invalid token');
    }
    throw new AuthenticationError('Token verification failed');
  }
};

// Authentication middleware
const authenticate = async (req, res, next) => {
  try {
    // Extract token from header
    const authHeader = req.headers.authorization;
    
    if (!authHeader || !authHeader.startsWith('Bearer ')) {
      throw new AuthenticationError('No token provided');
    }
    
    const token = authHeader.substring(7);
    
    // Check if token is blacklisted (for logout functionality)
    const isBlacklisted = await redis.get(`blacklist:${token}`);
    if (isBlacklisted) {
      throw new AuthenticationError('Token has been revoked');
    }
    
    // Verify token
    const decoded = verifyToken(token);
    
    // Attach user to request
    req.user = decoded;
    req.token = token;
    
    next();
  } catch (error) {
    next(error);
  }
};

// Authorization middleware - check user roles
const authorize = (...allowedRoles) => {
  return (req, res, next) => {
    if (!req.user) {
      return next(new AuthenticationError());
    }
    
    if (!allowedRoles.includes(req.user.role)) {
      return next(new AuthorizationError('You do not have permission to access this resource'));
    }
    
    next();
  };
};

// Logout - blacklist token
const logout = async (req, res, next) => {
  try {
    const token = req.token;
    const decoded = req.user;
    
    // Calculate remaining TTL
    const expiresIn = decoded.exp - Math.floor(Date.now() / 1000);
    
    // Blacklist token until it expires
    await redis.setex(`blacklist:${token}`, expiresIn, '1');
    
    res.json({
      status: 'success',
      message: 'Logged out successfully'
    });
  } catch (error) {
    next(error);
  }
};

module.exports = {
  generateToken,
  verifyToken,
  authenticate,
  authorize,
  logout
};

The token blacklisting saved us when we discovered a security issue. A user's account was compromised, and we needed to immediately invalidate all their tokens. With session-based auth, we could just delete the session. With JWT, tokens are stateless—they're valid until they expire. The blacklist lets us revoke tokens when needed.

We use Redis for the blacklist because it's fast and supports automatic expiration. When a token expires naturally, it's automatically removed from the blacklist. No cleanup needed.

Here's how we use these auth middlewares in routes:

// Routes with different permission levels
router.get('/api/products', authenticate, getProducts); // Any authenticated user
router.post('/api/products', authenticate, authorize('admin', 'manager'), createProduct); // Admin or manager only
router.delete('/api/products/:id', authenticate, authorize('admin'), deleteProduct); // Admin only

Caching Strategy That Actually Improves Performance

We added Redis caching after our database became the bottleneck. Our most common query was fetching product details—the same products were being queried thousands of times per minute. Each query hit the database, even though the data rarely changed.

Here's our caching implementation:

// config/redis.js
const Redis = require('ioredis');
const logger = require('./logger');

const redis = new Redis({
  host: process.env.REDIS_HOST || 'localhost',
  port: process.env.REDIS_PORT || 6379,
  password: process.env.REDIS_PASSWORD,
  db: process.env.REDIS_DB || 0,
  retryStrategy: (times) => {
    const delay = Math.min(times * 50, 2000);
    return delay;
  },
  maxRetriesPerRequest: 3,
  enableReadyCheck: true,
  enableOfflineQueue: false
});

redis.on('connect', () => {
  logger.info('Redis client connected');
});

redis.on('error', (err) => {
  logger.error('Redis error:', err);
});

redis.on('close', () => {
  logger.warn('Redis connection closed');
});

module.exports = redis;

// utils/cache.js
const redis = require('../config/redis');
const logger = require('../config/logger');

class CacheManager {
  constructor() {
    this.defaultTTL = 3600; // 1 hour
  }

  // Generate cache key
  generateKey(prefix, identifier) {
    return `${prefix}:${identifier}`;
  }

  // Get from cache
  async get(key) {
    try {
      const data = await redis.get(key);
      if (data) {
        return JSON.parse(data);
      }
      return null;
    } catch (error) {
      logger.error('Cache get error:', error);
      return null; // Fail gracefully
    }
  }

  // Set cache with TTL
  async set(key, value, ttl = this.defaultTTL) {
    try {
      await redis.setex(key, ttl, JSON.stringify(value));
      return true;
    } catch (error) {
      logger.error('Cache set error:', error);
      return false;
    }
  }

  // Delete from cache
  async del(key) {
    try {
      await redis.del(key);
      return true;
    } catch (error) {
      logger.error('Cache delete error:', error);
      return false;
    }
  }

  // Delete multiple keys by pattern
  async delPattern(pattern) {
    try {
      const keys = await redis.keys(pattern);
      if (keys.length > 0) {
        await redis.del(...keys);
      }
      return true;
    } catch (error) {
      logger.error('Cache delete pattern error:', error);
      return false;
    }
  }

  // Cache-aside pattern helper
  async getOrSet(key, fetchFunction, ttl = this.defaultTTL) {
    // Try to get from cache
    let data = await this.get(key);
    
    if (data !== null) {
      return { data, fromCache: true };
    }

    // Cache miss - fetch from source
    data = await fetchFunction();
    
    // Store in cache
    await this.set(key, data, ttl);
    
    return { data, fromCache: false };
  }
}

module.exports = new CacheManager();

Here's how we use caching in our routes:

// routes/products.js
const cache = require('../utils/cache');

router.get('/:id', async (req, res, next) => {
  try {
    const { id } = req.params;
    const cacheKey = cache.generateKey('product', id);

    // Try cache first
    const { data: product, fromCache } = await cache.getOrSet(
      cacheKey,
      async () => {
        // Fetch from database if not in cache
        const result = await db.query(
          'SELECT * FROM products WHERE id = $1',
          [id]
        );
        
        if (result.rows.length === 0) {
          throw new NotFoundError('Product');
        }
        
        return result.rows[0];
      },
      3600 // Cache for 1 hour
    );

    res.set('X-Cache', fromCache ? 'HIT' : 'MISS');
    res.json({
      status: 'success',
      data: product
    });
  } catch (error) {
    next(error);
  }
});

// Update product - invalidate cache
router.put('/:id', authenticate, authorize('admin'), async (req, res, next) => {
  try {
    const { id } = req.params;
    const { name, price, description } = req.body;

    const result = await db.query(
      `UPDATE products 
       SET name = $1, price = $2, description = $3, updated_at = NOW()
       WHERE id = $4
       RETURNING *`,
      [name, price, description, id]
    );

    if (result.rows.length === 0) {
      throw new NotFoundError('Product');
    }

    // Invalidate cache
    const cacheKey = cache.generateKey('product', id);
    await cache.del(cacheKey);
    
    // Also invalidate product list caches
    await cache.delPattern('products:list:*');

    res.json({
      status: 'success',
      data: result.rows[0]
    });
  } catch (error) {
    next(error);
  }
});

This caching strategy reduced our database load by 70%. Our average response time dropped from 180ms to 45ms for cached requests. The X-Cache header helps us monitor cache hit rates—we're currently at 85% cache hits for product queries.

The key lesson: always invalidate cache when data changes. We had a bug where product updates weren't clearing the cache, so users would see stale prices. The delPattern function helps us invalidate related caches—when a product updates, we clear both the specific product cache and all product list caches.

Request Validation That Prevents Bad Data

Input validation is where most APIs fail. We learned this after a client sent a request with price: "free" instead of a number. It crashed our API because we were doing math on a string.

Here's our validation approach using Joi:

// middleware/validator.js
const Joi = require('joi');
const { ValidationError } = require('../utils/errors');

const validate = (schema) => {
  return (req, res, next) => {
    const { error, value } = schema.validate(req.body, {
      abortEarly: false, // Return all errors, not just the first one
      stripUnknown: true // Remove unknown fields
    });

    if (error) {
      const errors = error.details.map(detail => ({
        field: detail.path.join('.'),
        message: detail.message
      }));

      return next(new ValidationError('Validation failed', errors));
    }

    // Replace req.body with validated and sanitized data
    req.body = value;
    next();
  };
};

module.exports = { validate };

// Example validation schemas
const Joi = require('joi');

const productSchemas = {
  create: Joi.object({
    name: Joi.string().min(3).max(200).required(),
    description: Joi.string().max(2000).allow(''),
    price: Joi.number().positive().precision(2).required(),
    category: Joi.string().valid('electronics', 'clothing', 'books', 'other').required(),
    stock: Joi.number().integer().min(0).default(0),
    tags: Joi.array().items(Joi.string().max(50)).max(10).default([]),
    metadata: Joi.object().unknown(true).default({})
  }),

  update: Joi.object({
    name: Joi.string().min(3).max(200),
    description: Joi.string().max(2000),
    price: Joi.number().positive().precision(2),
    category: Joi.string().valid('electronics', 'clothing', 'books', 'other'),
    stock: Joi.number().integer().min(0),
    tags: Joi.array().items(Joi.string().max(50)).max(10),
    metadata: Joi.object().unknown(true)
  }).min(1), // At least one field required

  query: Joi.object({
    page: Joi.number().integer().min(1).default(1),
    limit: Joi.number().integer().min(1).max(100).default(20),
    sort: Joi.string().valid('name', 'price', 'created_at', '-name', '-price', '-created_at').default('-created_at'),
    category: Joi.string().valid('electronics', 'clothing', 'books', 'other'),
    minPrice: Joi.number().positive(),
    maxPrice: Joi.number().positive().greater(Joi.ref('minPrice')),
    search: Joi.string().max(200)
  })
};

module.exports = { productSchemas };

Using validation in routes:

const { validate } = require('../middleware/validator');
const { productSchemas } = require('../schemas/product');

router.post('/', 
  authenticate, 
  authorize('admin'), 
  validate(productSchemas.create),
  async (req, res, next) => {
    try {
      // req.body is now validated and sanitized
      const { name, description, price, category, stock, tags, metadata } = req.body;
      
      const result = await db.query(
        `INSERT INTO products (name, description, price, category, stock, tags, metadata, created_at)
         VALUES ($1, $2, $3, $4, $5, $6, $7, NOW())
         RETURNING *`,
        [name, description, price, category, stock, JSON.stringify(tags), JSON.stringify(metadata)]
      );

      res.status(201).json({
        status: 'success',
        data: result.rows[0]
      });
    } catch (error) {
      next(error);
    }
  }
);

Validation catches issues before they hit your database or business logic. It also provides clear error messages to clients:

{
  "status": "error",
  "message": "Validation failed",
  "errors": [
    {
      "field": "price",
      "message": "\"price\" must be a number"
    },
    {
      "field": "name",
      "message": "\"name\" is required"
    }
  ]
}

Monitoring and Logging That Actually Helps

We use Winston for structured logging and track key metrics:

// config/logger.js
const winston = require('winston');
const path = require('path');

const logFormat = winston.format.combine(
  winston.format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss' }),
  winston.format.errors({ stack: true }),
  winston.format.json()
);

const logger = winston.createLogger({
  level: process.env.LOG_LEVEL || 'info',
  format: logFormat,
  defaultMeta: { 
    service: 'api',
    environment: process.env.NODE_ENV 
  },
  transports: [
    // Write errors to error.log
    new winston.transports.File({ 
      filename: path.join('logs', 'error.log'), 
      level: 'error',
      maxsize: 10485760, // 10MB
      maxFiles: 5
    }),
    // Write all logs to combined.log
    new winston.transports.File({ 
      filename: path.join('logs', 'combined.log'),
      maxsize: 10485760,
      maxFiles: 5
    })
  ]
});

// Console logging for development
if (process.env.NODE_ENV !== 'production') {
  logger.add(new winston.transports.Console({
    format: winston.format.combine(
      winston.format.colorize(),
      winston.format.simple()
    )
  }));
}

module.exports = logger;

Request logging middleware:

// middleware/requestLogger.js
const logger = require('../config/logger');

module.exports = (req, res, next) => {
  const startTime = Date.now();

  // Log when response finishes
  res.on('finish', () => {
    const duration = Date.now() - startTime;
    
    const logData = {
      method: req.method,
      url: req.originalUrl,
      statusCode: res.statusCode,
      duration: `${duration}ms`,
      ip: req.ip,
      userAgent: req.get('user-agent'),
      userId: req.user?.id
    };

    if (res.statusCode >= 500) {
      logger.error('Request failed', logData);
    } else if (res.statusCode >= 400) {
      logger.warn('Client error', logData);
    } else {
      logger.info('Request completed', logData);
    }

    // Track slow requests
    if (duration > 1000) {
      logger.warn('Slow request detected', {
        ...logData,
        threshold: '1000ms'
      });
    }
  });

  next();
};

Testing Strategy for Production APIs

We test at three levels:

// tests/unit/services/UserService.test.js
const UserService = require('../../../src/services/UserService');
const db = require('../../../src/config/database');

jest.mock('../../../src/config/database');

describe('UserService', () => {
  beforeEach(() => {
    jest.clearAllMocks();
  });

  describe('getUserById', () => {
    it('should return user when found', async () => {
      const mockUser = { id: '123', email: 'test@example.com', name: 'Test User' };
      db.query.mockResolvedValue({ rows: [mockUser] });

      const result = await UserService.getUserById('123');

      expect(result).toEqual(mockUser);
      expect(db.query).toHaveBeenCalledWith(
        expect.stringContaining('SELECT'),
        ['123']
      );
    });

    it('should throw NotFoundError when user not found', async () => {
      db.query.mockResolvedValue({ rows: [] });

      await expect(UserService.getUserById('999'))
        .rejects
        .toThrow('User not found');
    });
  });
});

// tests/integration/routes/users.test.js
const request = require('supertest');
const app = require('../../../src/app');
const db = require('../../../src/config/database');
const { generateToken } = require('../../../src/middleware/auth');

describe('User Routes', () => {
  let authToken;

  beforeAll(async () => {
    // Setup test database
    await db.query('BEGIN');
    
    // Create test user
    const result = await db.query(
      'INSERT INTO users (email, name, role) VALUES ($1, $2, $3) RETURNING *',
      ['test@example.com', 'Test User', 'user']
    );
    
    authToken = generateToken(result.rows[0]);
  });

  afterAll(async () => {
    await db.query('ROLLBACK');
    await db.close();
  });

  describe('GET /api/users/:id', () => {
    it('should return user when authenticated', async () => {
      const response = await request(app)
        .get('/api/users/123')
        .set('Authorization', `Bearer ${authToken}`)
        .expect(200);

      expect(response.body.status).toBe('success');
      expect(response.body.data).toHaveProperty('id');
      expect(response.body.data).toHaveProperty('email');
    });

    it('should return 401 when not authenticated', async () => {
      await request(app)
        .get('/api/users/123')
        .expect(401);
    });
  });
});

Performance Optimization Lessons

1. Database Indexes: We added indexes after noticing slow queries. A simple index on products.category reduced query time from 800ms to 12ms:

CREATE INDEX idx_products_category ON products(category);
CREATE INDEX idx_products_created_at ON products(created_at DESC);
CREATE INDEX idx_users_email ON users(email); -- For login queries

2. Pagination: Always paginate list endpoints. We learned this when a client requested /api/products and our API tried to return 500,000 products:

router.get('/', async (req, res, next) => {
  try {
    const page = parseInt(req.query.page) || 1;
    const limit = Math.min(parseInt(req.query.limit) || 20, 100); // Max 100 items
    const offset = (page - 1) * limit;

    const [products, countResult] = await Promise.all([
      db.query(
        'SELECT * FROM products ORDER BY created_at DESC LIMIT $1 OFFSET $2',
        [limit, offset]
      ),
      db.query('SELECT COUNT(*) FROM products')
    ]);

    const totalItems = parseInt(countResult.rows[0].count);
    const totalPages = Math.ceil(totalItems / limit);

    res.json({
      status: 'success',
      data: products.rows,
      pagination: {
        page,
        limit,
        totalItems,
        totalPages,
        hasNextPage: page < totalPages,
        hasPrevPage: page > 1
      }
    });
  } catch (error) {
    next(error);
  }
});

3. Response Compression: Already mentioned, but worth repeating—compression reduced our bandwidth by 84%.

4. Connection Pooling: Properly configured pools prevent connection exhaustion.

5. Async/Await Everywhere: Never use callbacks in production. Async/await makes error handling cleaner and prevents callback hell.

Deployment and DevOps Practices

We deploy using PM2 in cluster mode:

// ecosystem.config.js
module.exports = {
  apps: [{
    name: 'api',
    script: './src/server.js',
    instances: 'max', // Use all CPU cores
    exec_mode: 'cluster',
    env: {
      NODE_ENV: 'production',
      PORT: 3000
    },
    error_file: './logs/pm2-error.log',
    out_file: './logs/pm2-out.log',
    merge_logs: true,
    max_memory_restart: '1G', // Restart if memory exceeds 1GB
    exp_backoff_restart_delay: 100,
    listen_timeout: 10000,
    kill_timeout: 5000
  }]
};

Health check endpoint for load balancers:

router.get('/health', async (req, res) => {
  const health = {
    uptime: process.uptime(),
    timestamp: Date.now(),
    status: 'ok',
    checks: {}
  };

  // Check database
  try {
    const dbHealth = await db.healthCheck();
    health.checks.database = dbHealth.healthy ? 'ok' : 'error';
  } catch (error) {
    health.checks.database = 'error';
    health.status = 'error';
  }

  // Check Redis
  try {
    await redis.ping();
    health.checks.redis = 'ok';
  } catch (error) {
    health.checks.redis = 'error';
    health.status = 'error';
  }

  const statusCode = health.status === 'ok' ? 200 : 503;
  res.status(statusCode).json(health);
});

Conclusion

Building production-ready RESTful APIs with Node.js and Express isn't about knowing the framework—it's about understanding how systems fail under real-world conditions and designing for those failures from day one.

The patterns I've shared aren't theoretical best practices from documentation. They're battle-tested solutions that emerged from three years of production incidents, scaling challenges, and 2 AM debugging sessions. Every configuration value, every middleware ordering decision, every error handling strategy came from a real problem we had to solve.

Key Takeaways

1. Structure Matters from Day One: Don't wait until you have a 5,000-line index.js file to organize your code. Start with a clear separation of concerns: routes, middleware, services, configuration, and utilities. Your future self will thank you when you need to debug a production issue at 3 AM.

2. Error Handling Determines Your Sleep Schedule: Generic error messages are useless in production. Implement structured error classes, comprehensive logging, and operational/non-operational error distinction. The difference between "Something went wrong" and detailed error context is the difference between 15-minute fixes and 3-hour debugging sessions.

3. Database Connections Will Exhaust: Connection pooling isn't optional. Configure reasonable pool sizes based on your infrastructure, implement statement timeouts, and always release clients in finally blocks. Connection leaks will take down your API faster than any other issue.

4. Authentication Must Scale: Session-based auth doesn't work across multiple servers. Use JWT tokens with proper expiration, implement token blacklisting for logout functionality, and always validate tokens on every request. Security isn't negotiable.

5. Cache Aggressively, Invalidate Carefully: Caching can reduce database load by 70%+ and improve response times dramatically. But stale cache data will erode user trust. Always invalidate related caches when data changes, and make cache failures non-critical—your API should work even if Redis is down.

6. Validate Everything: Never trust client input. Use schema validation libraries like Joi to catch bad data before it reaches your business logic. Return clear, actionable error messages that help developers integrate with your API correctly.

7. Monitor What Matters: Log structured data with context—request details, user IDs, timing information, and error stacks. Track slow queries, cache hit rates, error rates, and response times. You can't fix what you can't measure.

8. Test at Multiple Levels: Unit tests verify individual functions, integration tests verify API endpoints and database interactions, and load tests verify performance under realistic traffic. All three are necessary.

9. Deploy with Confidence: Use process managers like PM2 in cluster mode, implement graceful shutdowns, add health check endpoints, and monitor your applications in production. Zero-downtime deployments are achievable with the right setup.

10. Performance Optimization Is Continuous: Database indexes, pagination, compression, and proper async handling aren't optional extras—they're fundamental requirements. Profile your API under load and optimize the bottlenecks.

The gap between a working API and a production-ready API is enormous. It's the difference between code that runs on your laptop and code that serves millions of requests reliably. These patterns won't prevent all issues—production systems are inherently complex—but they'll give you the foundation to handle whatever comes your way.

Start with solid fundamentals, monitor everything, iterate based on real data, and never assume your API is "done." Production is where the real learning happens. The API you deploy today will teach you how to build a better one tomorrow.

Now go build something that scales.