PageSightCaching System
Learn how PageSight's intelligent caching system works and how to use it effectively.
How Caching Works
PageSight automatically caches API responses to reduce server load and improve response times. Each request generates a unique cache key based on the URL, requested categories, and response format.
Unique key created from URL + categories + format
System checks for existing cached data before processing
Successful responses stored in MongoD with expiration time
Cache Duration
Cache Duration: 1 day (fixed)
Free users get a fixed 1-day cache duration. The cacheTime parameter is ignored.
Cache revalidation is not available on the free plan.
Default Cache: 1 day (if not specified)
Custom Cache: Minimum 5 minutes
Pro users can set custom cache duration per request using the cacheTime parameter.
Cache revalidation available with revalidate=true.
Basic Caching
By default, all successful API responses are cached. The first request performs a fresh scrape, and subsequent requests within the cache period return cached data.
Cache Headers
X-Cache header to see if the response came from cache (HIT) or was freshly scraped (MISS).Custom Cache Duration (Pro Only)
Pro users can set a custom cache duration using the cacheTime parameter. The minimum cache time is 5 minutes.
Use Cases
Short cache (5-30 minutes): For frequently updated websites like news sites, social media
Medium cache (1-6 hours): For blogs, documentation sites
Long cache (1 day+): For static sites, rarely updated content
Cache Revalidation (Pro Only)
Force a fresh scrape by adding revalidate=true to your request. This bypasses the cache and performs a new scrape.
When to Use Revalidation
- When you know a website has been updated
- For real-time data requirements
- When testing or debugging
- After making changes to the target website
Caching Best Practices
• Use appropriate cache duration for your use case
• Check X-Cache header to understand cache behavior
• Use revalidation when you need fresh data
• Monitor cache hit rates for optimization
• Set shorter cache for frequently updated sites
• Don't set cache time too short (wastes resources)
• Don't always use revalidate (defeats caching purpose)
• Don't ignore cache headers
• Don't assume data is always fresh
• Don't use free plan cacheTime parameter (ignored)
Cache Response Indicators
The API response includes indicators to help you understand cache status.