Caching

Understand and optimize the LRU caching system for maximum performance.

Overview

Large File MCP Server uses an intelligent LRU (Least Recently Used) cache to dramatically improve performance for repeated access patterns. Understanding how caching works enables you to optimize your workflows for maximum efficiency.

Cache Architecture

What Gets Cached

The server caches three types of data:

1. File Chunks: Results from read_large_file_chunk
2. File Structure: Metadata from get_file_structure
3. Search Results: Recent search patterns and results

Cache Keys

Each cache entry uses a composite key:

// Chunk cache key
const key = `${filePath}:chunk:${chunkIndex}`;

// Structure cache key
const key = `${filePath}:structure`;

// Search cache key
const key = `${filePath}:search:${pattern}`;

Cache Configuration

Default configuration:

{
  maxSize: 100,        // Maximum cached chunks
  ttl: undefined,      // No time-based expiration
  eviction: 'lru'      // Least Recently Used eviction
}

Performance Impact

Cache Hit vs Miss

Operation	Cache Hit	Cache Miss	Improvement
Read chunk (< 1MB)	~5ms	~50ms	10x faster
Read chunk (1-10MB)	~5ms	~200ms	40x faster
Read chunk (10-100MB)	~5ms	~800ms	160x faster
File structure	~1ms	~500ms	500x faster
Search (simple)	~2ms	~300ms	150x faster

Typical Hit Rates

Access Pattern	Expected Hit Rate	Notes
Sequential reading	20-30%	Low reuse
Random access	60-70%	Moderate reuse
Code navigation	80-90%	High reuse
Log analysis	85-95%	Very high reuse

Optimizing for Cache

1. Sequential Access with Overlap

Read chunks in sequence but allow overlap:

// Good: Sequential with potential reuse
for (let i = 0; i < totalChunks; i++) {
  const chunk = await read_large_file_chunk({
    filePath: file,
    chunkIndex: i
  });

  // If you need to reference previous chunk
  if (i > 0) {
    const prevChunk = await read_large_file_chunk({
      filePath: file,
      chunkIndex: i - 1
    }); // Cache hit!
  }
}

2. Batch Related Operations

Group operations on the same file:

// Good: Batch operations on same file
const file = "/var/log/app.log";

// All these benefit from cache
const structure = await get_file_structure({ filePath: file });
const firstChunk = await read_large_file_chunk({ filePath: file, chunkIndex: 0 });
const search = await search_in_large_file({ filePath: file, pattern: "ERROR" });
const navigation = await navigate_to_line({ filePath: file, lineNumber: 100 });

// Bad: Interleaved operations on different files
const file1Chunk = await read_large_file_chunk({ filePath: file1, chunkIndex: 0 });
const file2Chunk = await read_large_file_chunk({ filePath: file2, chunkIndex: 0 });
const file1Search = await search_in_large_file({ filePath: file1, pattern: "ERROR" });
// file1Chunk may be evicted by now

3. Reuse File Structure

Cache structure information:

// Application-level caching
const structureCache = new Map<string, FileStructure>();

async function getStructureCached(filePath: string) {
  if (!structureCache.has(filePath)) {
    const structure = await get_file_structure({ filePath });
    structureCache.set(filePath, structure);
  }
  return structureCache.get(filePath)!;
}

// Use cached structure
const structure = await getStructureCached(file);
console.log(`File has ${structure.totalChunks} chunks`);

// Later: still cached
const structure2 = await getStructureCached(file); // Instant

4. Navigate Near Cached Chunks

When navigating, stay near cached regions:

// If chunk 5 is cached, navigate nearby
const chunk5 = await read_large_file_chunk({
  filePath: file,
  chunkIndex: 5
}); // Cache miss

// These are likely cache hits (if line is in chunk 5)
await navigate_to_line({
  filePath: file,
  lineNumber: chunk5.startLine + 10
}); // Cache hit!

await navigate_to_line({
  filePath: file,
  lineNumber: chunk5.endLine - 10
}); // Cache hit!

Cache Monitoring

Check Cache Performance

Track cache hits and misses:

class CacheMonitor {
  private hits = 0;
  private misses = 0;

  recordHit() {
    this.hits++;
  }

  recordMiss() {
    this.misses++;
  }

  getStats() {
    const total = this.hits + this.misses;
    const hitRate = total > 0 ? (this.hits / total) * 100 : 0;

    return {
      hits: this.hits,
      misses: this.misses,
      total,
      hitRate: hitRate.toFixed(2) + "%"
    };
  }

  reset() {
    this.hits = 0;
    this.misses = 0;
  }
}

const monitor = new CacheMonitor();

// Wrap operations
async function readChunkMonitored(filePath: string, chunkIndex: number) {
  const startTime = Date.now();
  const result = await read_large_file_chunk({ filePath, chunkIndex });
  const duration = Date.now() - startTime;

  // Heuristic: < 20ms likely a cache hit
  if (duration < 20) {
    monitor.recordHit();
  } else {
    monitor.recordMiss();
  }

  return result;
}

// After operations
console.log("Cache statistics:", monitor.getStats());
// Output: { hits: 85, misses: 15, total: 100, hitRate: "85.00%" }

Cache Warmup

Pre-load Frequently Accessed Data

Warm up cache before heavy operations:

async function warmupCache(filePath: string) {
  console.log("Warming up cache...");

  // Load file structure (always useful)
  await get_file_structure({ filePath });

  // Load first few chunks (common access)
  const warmupChunks = 5;
  for (let i = 0; i < warmupChunks; i++) {
    await read_large_file_chunk({
      filePath,
      chunkIndex: i
    });
  }

  console.log("Cache warmed up");
}

// Use before operations
await warmupCache("/var/log/app.log");

// Now operations are faster
await search_in_large_file({
  filePath: "/var/log/app.log",
  pattern: "ERROR"
}); // Benefits from cached chunks

Cache Invalidation

When Cache is Invalidated

The cache is automatically invalidated when:

1. File Modified: File modification time changes
2. File Size Changes: File grows or shrinks
3. Cache Full: LRU eviction when max size reached

Manual Cache Control

While there's no direct API for cache control, you can influence it:

// Force cache refresh by accessing with different pattern
async function refreshCache(filePath: string, chunkIndex: number) {
  // Read chunk (may be cached)
  const chunk1 = await read_large_file_chunk({
    filePath,
    chunkIndex
  });

  // Small delay
  await new Promise(resolve => setTimeout(resolve, 100));

  // Read again (will use cache if still valid)
  const chunk2 = await read_large_file_chunk({
    filePath,
    chunkIndex
  });

  return chunk2;
}

Memory Management

Cache Memory Usage

Each cached chunk uses approximately:

// Average chunk memory
const chunkSize = 500; // lines
const avgLineLength = 100; // characters
const charsPerByte = 2; // UTF-16

const memoryPerChunk = chunkSize * avgLineLength * charsPerByte;
// ~100KB per chunk

// With max cache size of 100
const totalCacheMemory = memoryPerChunk * 100;
// ~10MB total cache memory

Optimize for Memory

If running in memory-constrained environments:

// Use smaller chunk sizes
const chunk = await read_large_file_chunk({
  filePath: file,
  chunkIndex: 0
  // Uses file type default, but you can process in smaller batches
});

// Process and release
processChunk(chunk);
chunk = null; // Allow garbage collection

Best Practices

1. Group File Operations

// Good: Operations on same file together
async function analyzeFile(filePath: string) {
  const structure = await get_file_structure({ filePath });
  const firstChunk = await read_large_file_chunk({ filePath, chunkIndex: 0 });
  const errors = await search_in_large_file({ filePath, pattern: "ERROR" });

  // All benefit from cache
  return { structure, firstChunk, errors };
}

// Bad: Interleaved operations
async function analyzeBad() {
  await get_file_structure({ filePath: file1 });
  await get_file_structure({ filePath: file2 });
  await read_large_file_chunk({ filePath: file1, chunkIndex: 0 }); // file1 cache may be evicted
}

2. Access Patterns Matter

// Good: Sequential or localized access
for (let i = 0; i < 10; i++) {
  await read_large_file_chunk({ filePath: file, chunkIndex: i });
}

// Bad: Random access across entire file
const randomIndices = [0, 50, 25, 75, 10, 60, ...]; // Random order
for (const index of randomIndices) {
  await read_large_file_chunk({ filePath: file, chunkIndex: index });
  // Poor cache utilization
}

3. Reuse Structure Information

// Good: Get structure once, use many times
const structure = await get_file_structure({ filePath: file });

const chunkSize = structure.chunkSize;
const totalChunks = structure.totalChunks;
const fileSize = structure.fileSize;

// Use this information for planning

// Bad: Repeated structure calls
for (let i = 0; i < someLimit; i++) {
  const structure = await get_file_structure({ filePath: file }); // Wasteful
  // ...
}

4. Monitor Cache Effectiveness

// Track performance to optimize
const stats = {
  operations: 0,
  totalTime: 0,
  cacheHits: 0
};

async function monitoredRead(filePath: string, chunkIndex: number) {
  const start = Date.now();
  const result = await read_large_file_chunk({ filePath, chunkIndex });
  const duration = Date.now() - start;

  stats.operations++;
  stats.totalTime += duration;

  if (duration < 20) stats.cacheHits++;

  return result;
}

// Periodically review
setInterval(() => {
  const avgTime = stats.totalTime / stats.operations;
  const hitRate = (stats.cacheHits / stats.operations) * 100;

  console.log(`Avg time: ${avgTime.toFixed(2)}ms, Hit rate: ${hitRate.toFixed(1)}%`);
}, 60000);

Advanced Patterns

Cache-Aware Pagination

Optimize pagination for cache:

async function paginateWithCache(
  filePath: string,
  currentPage: number,
  pageSize: number = 500
) {
  const structure = await get_file_structure({ filePath });
  const chunkIndex = Math.floor((currentPage * pageSize) / structure.chunkSize);

  // Read current chunk (may be cached)
  const chunk = await read_large_file_chunk({
    filePath,
    chunkIndex
  });

  // Pre-load next chunk (cache warmup)
  if (chunkIndex + 1 < structure.totalChunks) {
    read_large_file_chunk({
      filePath,
      chunkIndex: chunkIndex + 1
    }); // Fire and forget
  }

  return chunk;
}

Multi-File Cache Strategy

Manage cache across multiple files:

class MultiFileCacheManager {
  private fileAccessCount = new Map<string, number>();

  async read(filePath: string, chunkIndex: number) {
    // Track access
    this.fileAccessCount.set(
      filePath,
      (this.fileAccessCount.get(filePath) || 0) + 1
    );

    return await read_large_file_chunk({ filePath, chunkIndex });
  }

  getMostAccessed(limit: number = 5): string[] {
    return Array.from(this.fileAccessCount.entries())
      .sort(([, a], [, b]) => b - a)
      .slice(0, limit)
      .map(([file]) => file);
  }

  async warmupMostAccessed() {
    const topFiles = this.getMostAccessed(3);

    for (const file of topFiles) {
      await get_file_structure({ filePath: file });
      await read_large_file_chunk({ filePath: file, chunkIndex: 0 });
    }
  }
}

Troubleshooting

Low Hit Rate

If hit rate < 60%:

1. Check access pattern: Random access has lower hit rates
2. Too many files: Accessing too many files evicts cache
3. Large chunks: Fewer chunks fit in cache

High Memory Usage

If memory is an issue:

1. Reduce concurrent operations: Process files one at a time
2. Use streaming: stream_large_file instead of chunks
3. Application-level limits: Limit cache size in your application

Cache Thrashing

If cache constantly evicts:

1. Too many files: Focus on fewer files at a time
2. Access pattern: Improve sequential access
3. Chunk size: Consider file type optimization

See Also

• Performance Guide - Overall performance optimization
• Best Practices - Usage recommendations
• API Reference - All available tools
• Troubleshooting - Common issues