Caching Best Practices

Learn how to optimize your API usage with effective caching strategies.

Why Caching Matters

Benefits of Caching
  • Cost Efficiency: Reduce API calls and associated costs
  • Performance: Faster response times for cached data
  • Reliability: Less dependency on external website availability
  • Rate Limits: Stay within your rate limits more easily

Free Tier Strategies

Working with 1-Day Cache

✅ Do

  • • Plan your requests around the 1-day cache window
  • • Batch requests for URLs you need to check
  • • Store cached responses locally if possible
  • • Check cache status headers to avoid unnecessary requests

❌ Don't

  • • Make repeated requests for the same URL within 24 hours
  • • Ignore the X-Cache header
  • • Assume every request triggers a fresh scrape

Premium Tier Strategies

Optimizing Custom Cache

Choosing Cache Duration

  • Static Content (5-15 minutes): For content that rarely changes
  • Dynamic Content (30-60 minutes): For content that updates periodically
  • Frequently Updated (2-6 hours): For content that changes often
  • Rarely Updated (12-24 hours): For stable content

Example: Short Cache for News Sites

Loading code...

News sites update frequently, so a 10-minute cache is appropriate.

Example: Long Cache for Documentation

Loading code...

Documentation sites change infrequently, so a 24-hour cache works well.

Revalidation Guidelines

When to Revalidate
  • • User explicitly requests fresh data
  • • You know content was recently updated
  • • Debugging scraping issues
  • • Testing website changes
When NOT to Revalidate
  • • On every API call
  • • In automated background jobs
  • • For high-frequency requests
  • • Without user consent

Monitoring Cache Performance

Checking Cache Status

JavaScript Example

Loading code...

Reading Cache Headers

Loading code...

Pro Tips

PageSight | PageSight