Skip to content

Legacy library

This page describes the old TypeScript @sylphx/coderag library. It is not the Locus MCP server. The live server is local Rust BM25 in @sylphx/locus. Start at the quickstart.

Performance Tuning ​

Locus is optimized for speed and memory efficiency. This guide covers optimization strategies for large codebases.

Memory Optimization ​

Locus provides multiple strategies to reduce memory usage.

Low Memory Mode ​

Low memory mode uses SQL-based search instead of loading the entire index into RAM.

Enabling:

typescript
import { CodebaseIndexer } from '@sylphx/coderag'
import { PersistentStorage } from '@sylphx/coderag/storage-persistent'

const storage = new PersistentStorage()

const indexer = new CodebaseIndexer({
  storage,
  lowMemoryMode: true  // Default: true with PersistentStorage
})

Memory comparison:

Codebase SizeIn-MemoryLow MemoryReduction
1k chunks10 MB5 MB50%
10k chunks100 MB10 MB90%
100k chunks1 GB15 MB98.5%

Trade-offs:

  • Memory: 90% reduction for large codebases
  • Search speed: ~50% slower (15-30ms vs 10-20ms)
  • Indexing speed: Unchanged (same time to build index)

When to use:

  • Large codebases (10k+ files)
  • Memory-constrained environments
  • Long-running processes (servers, daemons)

When to avoid:

  • Small codebases (<1k files) where memory isn't a concern
  • Latency-critical applications requiring <10ms search
  • Development environments with plenty of RAM

Chunk-Based Indexing ​

Locus indexes at the chunk level rather than file level, reducing memory for large files.

Example:

typescript
// Large file: 10,000 lines, 500 KB

// File-level indexing:
// - 1 document vector (500 KB content in memory)
// - Large TF-IDF vector (many unique terms)

// Chunk-level indexing:
// - 50 chunks (average 200 lines each)
// - 50 smaller TF-IDF vectors (fewer terms per chunk)
// - More precise search (function-level granularity)

Memory savings:

For a 1 MB file split into 10 chunks:

File-level: 1 MB content + 100 KB TF-IDF vector = 1.1 MB
Chunk-level: 1 MB content + 10 * 10 KB TF-IDF vectors = 1.1 MB (similar)

But with low memory mode:
File-level: 100 KB TF-IDF vector in RAM
Chunk-level: Only query candidates loaded (typically 10-50 chunks)

Streaming Indexing ​

Locus processes files in batches to avoid loading everything at once.

Batch processing:

typescript
const indexer = new CodebaseIndexer({
  indexingBatchSize: 50  // Process 50 files at a time (default)
})

Memory profile during indexing:

Without batching (10k files):
  Peak memory: ~2 GB (all files loaded)

With batching (10k files, batch=50):
  Peak memory: ~100 MB (50 files at a time)

Batch size tuning:

Batch SizeMemory UsageIndexing SpeedRecommendation
10Very lowSlow (many DB writes)Memory-critical
50LowFastDefault
100MediumFasterHigh-memory systems
500HighFastestRAM-abundant

Example:

typescript
// Low memory environment (512 MB RAM)
const indexer = new CodebaseIndexer({
  indexingBatchSize: 10
})

// High memory environment (16 GB RAM)
const indexer = new CodebaseIndexer({
  indexingBatchSize: 200
})

Batch Sizes ​

Locus uses batching at multiple levels for efficiency.

File Batch Size ​

Controls how many files are processed together during indexing.

Configuration:

typescript
const indexer = new CodebaseIndexer({
  indexingBatchSize: 50  // Files per batch
})

Impact:

  • Memory: Smaller batches = lower peak memory
  • Speed: Larger batches = fewer DB transactions (faster)
  • I/O: Larger batches = better disk cache utilization

Tuning guide:

typescript
// For 1 GB RAM or less
indexingBatchSize: 10

// For 4 GB RAM (typical laptop)
indexingBatchSize: 50  // Default

// For 16 GB RAM or more
indexingBatchSize: 100-200

// For 64 GB RAM (server)
indexingBatchSize: 500

Vector Batch Size ​

Controls how many embeddings are generated at once.

Configuration:

typescript
const indexer = new CodebaseIndexer({
  embeddingProvider: provider,
  vectorBatchSize: 10  // Embeddings per API call (default)
})

Impact:

  • API calls: Larger batches = fewer API requests (lower cost)
  • Latency: Larger batches = longer per-request latency
  • Rate limits: Too large batches may hit rate limits

Tuning guide:

typescript
// Conservative (avoid rate limits)
vectorBatchSize: 5

// Default (good balance)
vectorBatchSize: 10

// Aggressive (faster but may hit limits)
vectorBatchSize: 50

// Maximum (check provider limits)
vectorBatchSize: 100

OpenAI rate limits:

text-embedding-3-small:
  - Tier 1: 3,000 RPM, 1M TPM
  - Tier 2: 5,000 RPM, 5M TPM
  - Tier 3: 5,000 RPM, 5M TPM

Recommended batch size:
  - Tier 1: 10 (safe)
  - Tier 2+: 20-50 (faster)

SQL Batch Size ​

Internal batch sizes for database operations.

Document vectors:

typescript
// Internal constant in storage-persistent.ts
const BATCH_SIZE = 199  // SQLite variable limit: ~999 / 5 fields = 199 rows

SQLite has a limit of ~999 bind variables. With 5 fields per row (chunkId, term, tf, tfidf, rawFreq), the maximum batch is 199 rows.

IDF scores:

typescript
// Internal constant
const BATCH_SIZE = 300  // 3 fields per row: term, idf, documentFrequency

These are internal optimizations and cannot be configured.

Caching Strategies ​

Locus uses multiple cache layers for performance.

Query Token Cache ​

Caches tokenized queries to avoid re-tokenization.

Configuration:

typescript
// Internal cache in tfidf.ts
const queryTokenCache = new Map<string, string[]>()
const QUERY_CACHE_MAX_SIZE = 100

Behavior:

  • Stores up to 100 unique queries
  • LRU eviction (oldest removed when full)
  • No TTL (persists until evicted or cleared)

Performance impact:

Without cache:
  Query "async function" → tokenize (~5ms) → search (15ms) = 20ms total

With cache (hit):
  Query "async function" → cache hit (~0.01ms) → search (15ms) = 15ms total

Speedup: 25% for repeated queries

Search Result Cache ​

Caches complete search results including scores and snippets.

Configuration:

typescript
// In CodebaseIndexer
this.searchCache = new LRUCache<SearchResult[]>(100, 5)
// 100 entries, 5 minute TTL

Parameters:

  • Max size: 100 queries
  • TTL: 5 minutes
  • Eviction: LRU (least recently used)

Cache key:

Cache key includes all search parameters:

typescript
const cacheKey = createCacheKey(query, {
  limit: 10,
  fileExtensions: ['.ts'],
  pathFilter: 'src/',
  excludePaths: ['node_modules/']
})

Different options = different cache entry.

Invalidation:

Cache is invalidated on index updates:

typescript
// File changed
await storage.storeFile(file)
searchCache.invalidate()  // Clear all cached results

Performance impact:

Cache miss (first query):
  "async function" → search (20ms)

Cache hit (repeated query):
  "async function" → cache (<1ms)

Speedup: 20x for repeated queries

Vector Storage Cache ​

LanceDB (vector storage) has internal caching.

Behavior:

  • Recently accessed vectors cached in memory
  • Automatic cache management (no configuration needed)
  • Typical cache size: 10-50 MB

No user configuration required.

Optimization Checklist ​

For large codebases (10k+ files):

  • [x] Enable low memory mode
  • [x] Use persistent storage (PersistentStorage)
  • [x] Tune indexing batch size (10-50 files)
  • [x] Reduce vector batch size if hitting rate limits (5-10)
  • [x] Monitor memory usage and adjust accordingly

For memory-constrained environments:

  • [x] lowMemoryMode: true
  • [x] indexingBatchSize: 10
  • [x] Skip vector search (no embeddingProvider)
  • [x] Use BM25-only search (faster, no embeddings)

For high-performance requirements:

  • [x] Increase indexing batch size (100-200)
  • [x] Increase vector batch size (20-50)
  • [x] Use in-memory mode if RAM permits (lowMemoryMode: false)
  • [x] Enable hybrid search with tuned vectorWeight

Performance Benchmarks ​

Indexing speed:

FilesChunksTimeThroughput
1005002s50 files/s
1,0005,00015s67 files/s
10,00050,000180s56 files/s
100,000500,0002,400s42 files/s

Performance degrades slightly for very large codebases due to IDF recalculation.

Search speed:

ModeChunksTimeNotes
BM25 (in-memory)10k10msFastest
BM25 (low-memory)10k20msSQL overhead
Vector10k40msEmbedding + search
Hybrid (0.7)10k50msBoth searches

Memory usage:

ModeChunksRAM
In-memory1k10 MB
In-memory10k100 MB
In-memory100k1 GB
Low-memory1k5 MB
Low-memory10k10 MB
Low-memory100k15 MB

Example Configurations ​

Default (balanced):

typescript
const indexer = new CodebaseIndexer({
  storage: new PersistentStorage(),
  lowMemoryMode: true,      // Use SQL-based search
  indexingBatchSize: 50,    // 50 files per batch
  vectorBatchSize: 10,      // 10 embeddings per API call
  embeddingProvider: provider
})

Memory-optimized:

typescript
const indexer = new CodebaseIndexer({
  storage: new PersistentStorage(),
  lowMemoryMode: true,
  indexingBatchSize: 10,     // Smaller batches
  vectorBatchSize: 5,        // Fewer API calls
  maxFileSize: 524288        // 512 KB limit (skip large files)
})

Speed-optimized:

typescript
const indexer = new CodebaseIndexer({
  storage: new PersistentStorage(),
  lowMemoryMode: false,      // In-memory search
  indexingBatchSize: 200,    // Large batches
  vectorBatchSize: 50,       // Large embedding batches
  embeddingProvider: provider
})

BM25-only (no embeddings):

typescript
const indexer = new CodebaseIndexer({
  storage: new PersistentStorage(),
  lowMemoryMode: true,
  indexingBatchSize: 50
  // No embeddingProvider = BM25-only search
})

// Fast keyword search
const results = await indexer.search('async function')

Hybrid with custom weights:

typescript
import { hybridSearch } from '@sylphx/coderag/hybrid-search'

const indexer = new CodebaseIndexer({
  storage: new PersistentStorage(),
  embeddingProvider: provider,
  vectorBatchSize: 20
})

// Tune vectorWeight for your use case
const results = await hybridSearch(query, indexer, {
  vectorWeight: 0.8,  // Favor semantic search
  limit: 20
})

Profiling ​

Measure indexing time:

typescript
const start = Date.now()

await indexer.index({
  onProgress: (current, total, file) => {
    const elapsed = Date.now() - start
    const rate = current / (elapsed / 1000)
    console.log(`[${current}/${total}] ${file} (${rate.toFixed(1)} files/s)`)
  }
})

const elapsed = Date.now() - start
console.log(`Indexing complete: ${elapsed}ms`)

Measure search time:

typescript
const start = Date.now()
const results = await indexer.search('async function')
const elapsed = Date.now() - start

console.log(`Search time: ${elapsed}ms (${results.length} results)`)

Monitor cache hit rate:

typescript
const stats = indexer.searchCache.stats()
console.log(`Cache hit rate: ${(stats.hitRate * 100).toFixed(1)}%`)
console.log(`Cache size: ${stats.size}/${stats.maxSize}`)

Memory profiling:

typescript
// Node.js memory usage
const used = process.memoryUsage()
console.log(`Heap used: ${(used.heapUsed / 1024 / 1024).toFixed(2)} MB`)
console.log(`Heap total: ${(used.heapTotal / 1024 / 1024).toFixed(2)} MB`)
console.log(`RSS: ${(used.rss / 1024 / 1024).toFixed(2)} MB`)

Released under the MIT License.