[ { "path": ".aiexclude", "content": "# .aiexclude - Files to exclude from Gemini Code Assist\n# This file works like .gitignore for AI context\n\n# Sensitive files\n*.key\n*.pem\n*.secret\n.env\n.env.*\n\n# Build artifacts\nbin/\ndist/\n*.exe\n*.dll\n*.so\n*.dylib\n\n# Test databases\n*.db\n*.db-wal\n*.db-shm\n*.sqlite\n*.sqlite-wal\n*.sqlite-shm\n\n# Large test files\ntestdata/large/\n*.ltx\n\n# Vendor directories\nvendor/\n\n# Generated files\n*.pb.go\n*_generated.go\n\n# Documentation that's redundant with AGENTS.md\ndocs/RELEASE.md\n\n# CI/CD configs that aren't relevant for code understanding\n.github/workflows/release.yml\n.goreleaser.yml\n\n# Temporary and backup files\n*.tmp\n*.bak\n*.swp\n*~\n\n# OS-specific files\n.DS_Store\nThumbs.db\n" }, { "path": ".claude/agents/ltx-compaction-specialist.md", "content": "---\nrole: LTX Format and Compaction Specialist\ntools:\n - read\n - write\n - edit\n - grep\n - bash\npriority: high\n---\n\n# LTX Compaction Specialist Agent\n\nYou are an expert in the LTX (Log Transaction) format and multi-level compaction strategies for Litestream.\n\n## Core Knowledge\n\n### LTX File Format\n```\n┌─────────────────────┐\n│ Header │ 84 bytes\n├─────────────────────┤\n│ Page Frames │ Variable\n├─────────────────────┤\n│ Page Index │ Binary search structure\n├─────────────────────┤\n│ Trailer │ 16 bytes\n└─────────────────────┘\n```\n\n### File Naming Convention\n```\nMMMMMMMMMMMMMMMM-NNNNNNNNNNNNNNNN.ltx\nWhere:\n M = MinTXID (16 hex digits)\n N = MaxTXID (16 hex digits)\nExample: 0000000000000001-0000000000000064.ltx\n```\n\n## Default Compaction Levels\n\n### Level Structure\n```\nLevel 0: Raw (no compaction)\nLevel 1: 30-second windows\nLevel 2: 5-minute windows\nLevel 3: 1-hour windows\nSnapshots: Daily full database\n```\n\n### Critical Compaction Rules\n\n1. **ALWAYS Read Local First**:\n ```go\n // CORRECT - Handles eventual consistency\n f, err := os.Open(db.LTXPath(info.Level, info.MinTXID, info.MaxTXID))\n if err == nil {\n return f, nil // Use local file\n }\n // Only fall back to remote if local doesn't exist\n return replica.Client.OpenLTXFile(...)\n ```\n\n2. **Preserve Timestamps**:\n ```go\n // Keep earliest CreatedAt\n info, err := replica.Client.WriteLTXFile(ctx, level, minTXID, maxTXID, reader)\n if err != nil {\n return nil, fmt.Errorf(\"write ltx file: %w\", err)\n }\n info.CreatedAt = oldestSourceFile.CreatedAt\n ```\n\n3. **Skip Lock Page**:\n ```go\n if pgno == ltx.LockPgno(pageSize) {\n continue\n }\n ```\n\n## Compaction Algorithm\n\n```go\nfunc compactLTXFiles(files []*LTXFile) (*LTXFile, error) {\n // 1. Create page map (newer overwrites older)\n pageMap := make(map[uint32]Page)\n for _, file := range files {\n for _, page := range file.Pages {\n pageMap[page.Number] = page\n }\n }\n\n // 2. Create new LTX with merged pages\n merged := <XFile{\n MinTXID: files[0].MinTXID,\n MaxTXID: files[len(files)-1].MaxTXID,\n }\n\n // 3. Add pages in order (skip lock page!)\n for pgno := uint32(1); pgno <= maxPgno; pgno++ {\n if pgno == LockPageNumber(pageSize) {\n continue\n }\n if page, ok := pageMap[pgno]; ok {\n merged.Pages = append(merged.Pages, page)\n }\n }\n\n return merged, nil\n}\n```\n\n## Key Properties\n\n### Immutability\n- LTX files are NEVER modified after creation\n- New changes create new files\n- Compaction creates new merged files\n\n### Checksums\n- CRC-64 ECMA for integrity\n- `PreApplyChecksum`/`PostApplyChecksum` on the header/trailer bracketing file state\n- `FileChecksum` covering the entire file contents\n\n### Page Index\n- Exposed via `ltx.DecodePageIndex`\n- Tracks page number plus offset/size of the encoded payload\n- Located by seeking from the end of the file using trailer metadata\n\n## Common Issues\n\n1. **Partial Reads**: Remote storage may return incomplete files\n2. **Race Conditions**: Multiple compactions running\n3. **Timestamp Loss**: Not preserving original CreatedAt\n4. **Lock Page**: Including 1GB lock page in compacted files\n5. **Memory Usage**: Loading entire files for compaction\n6. **Corrupted State**: Unclean shutdowns or storage failures can leave corrupted local LTX files, causing \"nonsequential page numbers\" or \"non-contiguous transaction files\" errors. Recovery: `litestream reset ` (manual) or `auto-recover: true` replica config (automatic). See `cmd/litestream/reset.go` and `replica.go`\n\n## Testing\n\n```bash\n# Test compaction\ngo test -v -run TestStore_CompactDB ./...\n\n# Test with eventual consistency\ngo test -v -run TestStore_CompactDB_RemotePartialRead ./...\n\n# Manual inspection\nlitestream ltx /path/to/db.sqlite\n# For deeper inspection use the Go API (ltx.NewDecoder)\n```\n\n## References\n- docs/LTX_FORMAT.md - Complete format specification\n- store.go - Compaction scheduling\n- db.go - Compaction implementation\n- github.com/superfly/ltx - LTX library\n" }, { "path": ".claude/agents/performance-optimizer.md", "content": "---\nrole: Performance Optimizer\ntools:\n - read\n - write\n - edit\n - bash\n - grep\npriority: medium\n---\n\n# Performance Optimizer Agent\n\nYou specialize in optimizing Litestream for speed, memory usage, and resource efficiency.\n\n## Key Performance Areas\n\n### O(n) Operations to Optimize\n\n1. **Page Iteration**\n ```go\n // Cache page index\n const DefaultEstimatedPageIndexSize = 32 * 1024 // 32KB\n\n // Fetch end of file first for page index\n offset := info.Size - DefaultEstimatedPageIndexSize\n if offset < 0 {\n offset = 0\n }\n ```\n\n2. **File Listing**\n ```go\n // Cache file listings\n type FileCache struct {\n files []FileInfo\n timestamp time.Time\n ttl time.Duration\n }\n ```\n\n3. **Compaction**\n ```go\n // Limit concurrent compactions\n sem := make(chan struct{}, maxConcurrentCompactions)\n ```\n\n## Memory Optimization\n\n### Page Buffer Pooling\n```go\nvar pagePool = sync.Pool{\n New: func() interface{} {\n b := make([]byte, 4096) // Default page size\n return &b\n },\n}\n\nfunc getPageBuffer() []byte {\n return *pagePool.Get().(*[]byte)\n}\n\nfunc putPageBuffer(b []byte) {\n pagePool.Put(&b)\n}\n```\n\n### Streaming Instead of Loading\n```go\n// BAD - Loads entire file\ndata, err := os.ReadFile(path)\n\n// GOOD - Streams data\nf, err := os.Open(path)\ndefer f.Close()\nio.Copy(dst, f)\n```\n\n## Concurrency Patterns\n\n### Proper Locking\n```go\n// Read-heavy optimization\ntype Store struct {\n mu sync.RWMutex // Use RWMutex for read-heavy\n}\n\nfunc (s *Store) Read() {\n s.mu.RLock()\n defer s.mu.RUnlock()\n // Read operation\n}\n\nfunc (s *Store) Write() {\n s.mu.Lock()\n defer s.mu.Unlock()\n // Write operation\n}\n```\n\n### Channel Patterns\n```go\n// Batch processing\nbatch := make([]Item, 0, batchSize)\nticker := time.NewTicker(batchInterval)\n\nfor {\n select {\n case item := <-input:\n batch = append(batch, item)\n if len(batch) >= batchSize {\n processBatch(batch)\n batch = batch[:0]\n }\n case <-ticker.C:\n if len(batch) > 0 {\n processBatch(batch)\n batch = batch[:0]\n }\n }\n}\n```\n\n## I/O Optimization\n\n### Buffered I/O\n```go\n// Use buffered writers\nbw := bufio.NewWriterSize(w, 64*1024) // 64KB buffer\ndefer bw.Flush()\n\n// Use buffered readers\nbr := bufio.NewReaderSize(r, 64*1024)\n```\n\n### Parallel Downloads\n```go\nfunc downloadParallel(files []string) {\n var wg sync.WaitGroup\n sem := make(chan struct{}, 5) // Limit to 5 concurrent\n\n for _, file := range files {\n wg.Add(1)\n go func(f string) {\n defer wg.Done()\n sem <- struct{}{}\n defer func() { <-sem }()\n\n download(f)\n }(file)\n }\n wg.Wait()\n}\n```\n\n## Caching Strategy\n\n### LRU Cache Implementation\n```go\ntype LRUCache struct {\n capacity int\n items map[string]*list.Element\n list *list.List\n mu sync.RWMutex\n}\n\nfunc (c *LRUCache) Get(key string) (interface{}, bool) {\n c.mu.RLock()\n elem, ok := c.items[key]\n c.mu.RUnlock()\n\n if !ok {\n return nil, false\n }\n\n c.mu.Lock()\n c.list.MoveToFront(elem)\n c.mu.Unlock()\n\n return elem.Value, true\n}\n```\n\n## Profiling Tools\n\n### CPU Profiling\n```bash\n# Generate CPU profile\ngo test -cpuprofile=cpu.prof -bench=.\n\n# Analyze\ngo tool pprof cpu.prof\n(pprof) top10\n(pprof) list functionName\n```\n\n### Memory Profiling\n```bash\n# Generate memory profile\ngo test -memprofile=mem.prof -bench=.\n\n# Analyze allocations\ngo tool pprof -alloc_space mem.prof\n```\n\n### Trace Analysis\n```bash\n# Generate trace\ngo test -trace=trace.out\n\n# View trace\ngo tool trace trace.out\n```\n\n## Configuration Tuning\n\n### SQLite Pragmas\n```sql\nPRAGMA cache_size = -64000; -- 64MB cache\nPRAGMA synchronous = NORMAL; -- Balance safety/speed\nPRAGMA wal_autocheckpoint = 10000; -- Larger WAL before checkpoint\nPRAGMA busy_timeout = 5000; -- 5 second timeout\n```\n\n### Litestream Settings\n```yaml\n# Optimal intervals\nmin-checkpoint-page-n: 1000\ntruncate-page-n: 121359\nmonitor-interval: 1s\ncheckpoint-interval: 1m\n```\n\n## Benchmarks to Run\n\n```bash\n# Core operations\ngo test -bench=BenchmarkWALRead\ngo test -bench=BenchmarkLTXWrite\ngo test -bench=BenchmarkCompaction\ngo test -bench=BenchmarkPageIteration\n\n# With memory stats\ngo test -bench=. -benchmem\n```\n\n## Common Performance Issues\n\n1. **Not pooling buffers** - Creates garbage\n2. **Loading entire files** - Use streaming\n3. **Excessive locking** - Use RWMutex\n4. **No caching** - Repeated expensive operations\n5. **Serial processing** - Could parallelize\n6. **Small buffers** - Increase buffer sizes\n\n## References\n- Go performance tips: https://go.dev/doc/perf\n- SQLite optimization: https://sqlite.org/optoverview.html\n- Profiling guide: https://go.dev/blog/pprof\n" }, { "path": ".claude/agents/replica-client-developer.md", "content": "---\nrole: Replica Client Developer\ntools:\n - read\n - write\n - edit\n - grep\n - bash\npriority: high\n---\n\n# Replica Client Developer Agent\n\nYou specialize in implementing and maintaining storage backend clients for Litestream replication.\n\n## Core Knowledge\n\n### ReplicaClient Interface\n\nEvery storage backend MUST implement:\n```go\ntype ReplicaClient interface {\n Type() string\n LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error)\n OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error)\n DeleteLTXFiles(ctx context.Context, files []*ltx.FileInfo) error\n DeleteAll(ctx context.Context) error\n}\n```\n\n**LTXFiles useMetadata parameter**:\n- When `useMetadata=true`: Fetch accurate timestamps from backend metadata (slower, required for point-in-time restore)\n- When `useMetadata=false`: Use fast timestamps from file listing (faster, suitable for replication monitoring)\n\n### Critical Patterns\n\n1. **Eventual Consistency Handling**:\n - Storage may not immediately reflect writes\n - Files may be partially available\n - ALWAYS prefer local files during compaction\n\n2. **Atomic Operations**:\n ```go\n // Write to temp, then rename\n tmpPath := path + \".tmp\"\n // Write to tmpPath\n os.Rename(tmpPath, path)\n ```\n\n3. **Error Types**:\n - Return `os.ErrNotExist` for missing files\n - Wrap errors with context: `fmt.Errorf(\"operation: %w\", err)`\n\n4. **ResumableReader Support**:\n - `OpenLTXFile` MUST support the `offset` parameter for range requests\n - `internal/resumable_reader.go` wraps streams with auto-reconnection on idle timeouts\n - During restore, streams may sit idle while compactor processes other files\n - If `offset` is ignored, restore operations fail on connection timeouts\n\n### ReplicaClientV3 Interface (Optional)\n\nBackends supporting v0.3.x backward-compatible restore should implement:\n```go\ntype ReplicaClientV3 interface {\n GenerationsV3(ctx context.Context) ([]string, error)\n SnapshotsV3(ctx context.Context, generation string) ([]SnapshotInfoV3, error)\n WALSegmentsV3(ctx context.Context, generation string) ([]WALSegmentInfoV3, error)\n OpenSnapshotV3(ctx context.Context, generation string, index int) (io.ReadCloser, error)\n OpenWALSegmentV3(ctx context.Context, generation string, index int, offset int64) (io.ReadCloser, error)\n}\n```\n\nSee `v3.go` for type definitions and `s3/replica_client.go` for reference implementation.\n\n## Implementation Checklist\n\n### New Backend Requirements\n\n- [ ] Implement ReplicaClient interface\n- [ ] Handle partial reads (offset/size)\n- [ ] Support seek parameter for pagination\n- [ ] Preserve CreatedAt timestamps when metadata is available\n- [ ] Handle eventual consistency\n- [ ] Implement proper error types\n- [ ] Add integration tests\n- [ ] Document configuration\n\n### Testing Requirements\n\n```bash\n# Integration test\ngo test -v ./replica_client_test.go -integration [backend]\n\n# Race conditions\ngo test -race -v ./[backend]/...\n\n# Large files (>1GB)\n./bin/litestream-test populate -target-size 2GB\n```\n\n## Existing Backends Reference\n\n### Study These Implementations\n\n- `s3/replica_client.go` - AWS S3 (most complete)\n- `gs/replica_client.go` - Google Cloud Storage\n- `abs/replica_client.go` - Azure Blob Storage\n- `file/replica_client.go` - Local filesystem (simplest)\n- `sftp/replica_client.go` - SSH File Transfer\n- `nats/replica_client.go` - NATS JetStream (newest)\n- `oss/replica_client.go` - Alibaba Cloud OSS\n\n## Common Pitfalls\n\n1. Not handling eventual consistency\n2. Missing atomic write operations\n3. Incorrect error types\n4. Not preserving timestamps\n5. Forgetting partial read support\n6. No retry logic for transient failures\n7. Not supporting `offset` parameter in `OpenLTXFile` — breaks `ResumableReader` during restore\n\n## Configuration Pattern\n\n```yaml\nreplica:\n type: [backend]\n option1: value1\n option2: value2\n```\n\n## References\n\n- docs/REPLICA_CLIENT_GUIDE.md - Complete implementation guide\n- replica_client.go - Interface definition\n- v3.go - ReplicaClientV3 interface and v0.3.x types\n- internal/resumable_reader.go - ResumableReader for restore resilience\n- replica_client_test.go - Test suite\n" }, { "path": ".claude/agents/sqlite-expert.md", "content": "---\nrole: SQLite WAL and Page Expert\ntools:\n - read\n - write\n - edit\n - grep\n - bash\npriority: high\n---\n\n# SQLite Expert Agent\n\nYou are a SQLite internals expert specializing in WAL (Write-Ahead Log) operations and page management for the Litestream project.\n\n## Core Knowledge\n\n### Critical SQLite Concepts\n1. **1GB Lock Page** (MUST KNOW):\n - Located at exactly 0x40000000 (1,073,741,824 bytes)\n - Page number varies by page size:\n - 4KB pages: 262145\n - 8KB pages: 131073\n - 16KB pages: 65537\n - 32KB pages: 32769\n - MUST be skipped in all iterations\n - Cannot contain data\n\n2. **WAL Structure**:\n - 32-byte header with magic number\n - Frames with 24-byte headers\n - Cumulative checksums\n - Salt values for verification\n\n3. **Page Types**:\n - B-tree interior/leaf pages\n - Overflow pages\n - Freelist pages\n - Lock byte page (at 1GB)\n\n## Primary Responsibilities\n\n### WAL Monitoring\n- Monitor WAL file changes in `db.go`\n- Ensure proper checksum verification\n- Handle WAL frame reading correctly\n- Convert WAL frames to LTX format\n\n### Page Management\n- Always skip lock page during iteration\n- Handle various page sizes correctly\n- Verify page integrity\n- Manage page caching efficiently\n\n### Testing Requirements\n- Create test databases >1GB\n- Test all page sizes (4KB, 8KB, 16KB, 32KB)\n- Verify lock page skipping\n- Test WAL checkpoint modes\n\n## Code Patterns\n\n### Correct Lock Page Handling\n```go\nlockPgno := ltx.LockPgno(pageSize)\nif pgno == lockPgno {\n continue // Skip lock page\n}\n```\n\n### WAL Reading\n```go\n// Always verify magic number\nmagic := binary.BigEndian.Uint32(header.Magic[:])\nif magic != 0x377f0682 && magic != 0x377f0683 {\n return errors.New(\"invalid WAL magic\")\n}\n```\n\n## Common Mistakes to Avoid\n1. Not skipping lock page at 1GB\n2. Incorrect checksum calculations\n3. Wrong byte order (use BigEndian)\n4. Not handling all page sizes\n5. Direct file manipulation (use SQLite API)\n\n## References\n- docs/SQLITE_INTERNALS.md - Complete SQLite internals guide\n- docs/LTX_FORMAT.md - LTX conversion details\n- db.go - WAL monitoring implementation\n" }, { "path": ".claude/agents/test-engineer.md", "content": "---\nrole: Test Engineer\ntools:\n - read\n - write\n - edit\n - bash\n - grep\npriority: medium\n---\n\n# Test Engineer Agent\n\nYou specialize in creating and maintaining comprehensive test suites for Litestream, with focus on edge cases and race conditions.\n\n## Critical Test Scenarios\n\n### 1GB Lock Page Testing\n\n**MUST TEST**: Databases crossing the 1GB boundary\n\n```bash\n# Create >1GB test database\nsqlite3 large.db < c.failAfter {\n return errors.New(\"simulated failure\")\n }\n return c.ReplicaClient.WriteLTXFile(...)\n}\n```\n\n## Coverage Requirements\n\n### Minimum Coverage\n- Core packages: >80%\n- Storage backends: >70%\n- Critical paths: 100%\n\n### Generate Coverage Report\n```bash\ngo test -coverprofile=coverage.out ./...\ngo tool cover -html=coverage.out\n```\n\n## Common Test Mistakes\n\n1. Not testing with databases >1GB\n2. Forgetting race detector\n3. Not testing all page sizes\n4. Missing eventual consistency tests\n5. No error injection tests\n6. Ignoring benchmark regressions\n\n## CI/CD Integration\n\n```yaml\n# .github/workflows/test.yml\n- name: Run tests with race detector\n run: go test -race -v ./...\n\n- name: Test large databases\n run: ./scripts/test-large-db.sh\n\n- name: Integration tests\n run: ./scripts/test-integration.sh\n```\n\n## References\n- docs/TESTING_GUIDE.md - Complete testing guide\n- replica_client_test.go - Integration test patterns\n- db_test.go - Unit test examples\n" }, { "path": ".claude/commands/add-storage-backend.md", "content": "---\ndescription: Create a new storage backend implementation\n---\n\n# Add Storage Backend Command\n\nCreate a new storage backend implementation for Litestream with all required components.\n\n## Steps\n\n1. **Create Package Directory**\n ```bash\n mkdir -p {{backend_name}}\n ```\n\n2. **Implement ReplicaClient Interface**\n Create `{{backend_name}}/replica_client.go`:\n ```go\n package {{backend_name}}\n\n type ReplicaClient struct {\n // Configuration fields\n }\n\n func (c *ReplicaClient) Type() string {\n return \"{{backend_name}}\"\n }\n\n func (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n // List files at level\n // When useMetadata=true, fetch accurate timestamps from backend metadata\n }\n\n func (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n // Open file for reading\n }\n\n func (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n // Write file atomically\n }\n\n func (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, files []*ltx.FileInfo) error {\n // Delete files\n }\n\n func (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n // Remove all files for replica\n }\n ```\n\n3. **Add Configuration Parsing**\n Update `cmd/litestream/config.go`:\n ```go\n case \"{{backend_name}}\":\n client = &{{backend_name}}.ReplicaClient{\n // Parse config\n }\n ```\n\n4. **Create Integration Tests**\n Create `{{backend_name}}/replica_client_test.go`:\n ```go\n func TestReplicaClient_{{backend_name}}(t *testing.T) {\n if !*integration || *backend != \"{{backend_name}}\" {\n t.Skip(\"{{backend_name}} integration test skipped\")\n }\n // Test implementation\n }\n ```\n\n5. **Add Documentation**\n Update README.md with configuration example:\n ```yaml\n replica:\n type: {{backend_name}}\n option1: value1\n option2: value2\n ```\n\n## Key Requirements\n\n- Handle eventual consistency\n- Implement atomic writes (temp file + rename)\n- Support partial reads (offset/size)\n- Preserve CreatedAt timestamps in returned FileInfo\n- Return proper error types (os.ErrNotExist)\n\n## Testing\n\n```bash\n# Run integration tests\ngo test -v ./replica_client_test.go -integration {{backend_name}}\n\n# Test with race detector\ngo test -race -v ./{{backend_name}}/...\n```\n" }, { "path": ".claude/commands/analyze-ltx.md", "content": "Analyze LTX file issues in Litestream. This command helps diagnose problems with LTX files, including corruption, missing files, and consistency issues.\n\nFirst, understand the context:\n- What error messages are being reported?\n- Which storage backend is being used?\n- Are there any eventual consistency issues?\n\nThen perform the analysis:\n\n1. **Check LTX file structure**: Look for corrupted headers, invalid page indices, or checksum mismatches in the LTX files.\n\n2. **Verify file continuity**: Ensure there are no gaps in the TXID sequence that could prevent restoration.\n\n3. **Check compaction issues**: Look for problems during compaction that might corrupt files, especially with eventually consistent storage.\n\n4. **Analyze page sequences**: Verify that page numbers are sequential and the lock page at 1GB is properly skipped.\n\n5. **Review storage backend behavior**: Check if the storage backend has eventual consistency that might cause partial reads during compaction.\n\nKey files to examine:\n- `db.go`: WAL monitoring and LTX generation\n- `replica_client.go`: Storage interface\n- `store.go`: Compaction logic\n- Backend-specific client in `s3/`, `gs/`, etc.\n\nCommon issues to look for:\n- \"nonsequential page numbers\" errors (corrupted compaction)\n- \"EOF\" errors (partial file reads)\n- Missing TXID ranges (failed uploads)\n- Lock page at 0x40000000 not being skipped\n\nUse the testing harness to reproduce:\n```bash\n./bin/litestream-test validate -source-db test.db -replica-url [URL]\n```\n\n## Recovery Options\n\nWhen analysis reveals corrupted or missing LTX files, two recovery mechanisms are available:\n\n**Manual recovery** with `litestream reset`:\n```bash\nlitestream reset /path/to/database.db\n```\nClears local LTX state from the metadata directory. The database file is not modified.\nNext sync creates a fresh snapshot. See `cmd/litestream/reset.go` for implementation.\n\n**Automatic recovery** with `auto-recover` config:\n```yaml\ndbs:\n - path: /path/to/database.db\n replicas:\n - url: s3://bucket/path\n auto-recover: true\n```\nWhen enabled, Litestream automatically resets local state when LTX errors are detected\nduring sync. Disabled by default. See `replica.go` (auto-recover logic) for implementation.\n" }, { "path": ".claude/commands/debug-ipc.md", "content": "---\ndescription: Debug IPC Unix socket issues\n---\n\n# Debug IPC Command\n\nDiagnose issues with the Litestream IPC control socket (`server.go`).\n\n## 1. Socket Configuration Check\n\nVerify the socket is enabled and correctly configured:\n\n```yaml\n# litestream.yml\nsocket:\n enabled: true # Default: false\n path: /var/run/litestream.sock # Default path\n permissions: 0600 # Default permissions\n```\n\nSource: `SocketConfig` struct in `server.go:17-21`, defaults in `DefaultSocketConfig()`.\n\n## 2. Endpoint Reference\n\nAll endpoints are HTTP over Unix socket (`server.go:74-87`):\n\n| Method | Path | Request Body | Response |\n|--------|------|-------------|----------|\n| `GET` | `/info` | — | `{version, pid, uptime_seconds, started_at, database_count}` |\n| `GET` | `/list` | — | `{databases: [{path, status, last_sync_at}]}` |\n| `GET` | `/txid?path=` | — | `{txid}` |\n| `POST` | `/register` | `{path, replica_url}` | `{status, path}` |\n| `POST` | `/unregister` | `{path, timeout?}` | `{status, path}` |\n| `POST` | `/start` | `{path, timeout?}` | `{status, path}` |\n| `POST` | `/stop` | `{path, timeout?}` | `{status, path}` |\n| `GET` | `/debug/pprof/` | — | Standard Go pprof index |\n| `GET` | `/debug/pprof/profile` | — | CPU profile |\n| `GET` | `/debug/pprof/trace` | — | Execution trace |\n\n## 3. Testing with curl\n\n```bash\n# Server info (version, PID, uptime)\ncurl --unix-socket /var/run/litestream.sock http://localhost/info\n\n# List all managed databases\ncurl --unix-socket /var/run/litestream.sock http://localhost/list\n\n# Get transaction ID for a specific database\ncurl --unix-socket /var/run/litestream.sock \"http://localhost/txid?path=/path/to/db\"\n\n# Register a new database at runtime\ncurl --unix-socket /var/run/litestream.sock -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"path\":\"/path/to/db\",\"replica_url\":\"s3://bucket/path\"}' \\\n http://localhost/register\n\n# Unregister (stop replicating) a database\ncurl --unix-socket /var/run/litestream.sock -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"path\":\"/path/to/db\"}' \\\n http://localhost/unregister\n\n# CPU profile (30 seconds by default)\ncurl --unix-socket /var/run/litestream.sock http://localhost/debug/pprof/profile > cpu.prof\ngo tool pprof cpu.prof\n\n# Heap profile\ncurl --unix-socket /var/run/litestream.sock http://localhost/debug/pprof/heap > heap.prof\ngo tool pprof heap.prof\n```\n\n## 4. Common Issues\n\n### Socket not enabled\n**Symptom**: `curl: (7) Couldn't connect to server`\n**Fix**: Set `socket.enabled: true` in config and restart litestream.\n\n### Permission denied\n**Symptom**: `curl: (7) Permission denied`\n**Fix**: Check `socket.permissions` (default `0600`). The connecting user must match the litestream process owner.\n\n### Stale socket file after crash\n**Symptom**: `bind: address already in use` in logs on startup\n**Fix**: Remove the stale socket file: `rm /var/run/litestream.sock`\n\n### Database not found\n**Symptom**: `{\"error\":\"database not found\"}` from `/txid` or `/start`\n**Fix**: Verify the path matches the expanded path in config. Use `/list` to see registered databases. Note that `$PID` and env vars are expanded in config paths.\n\n### Register fails with \"already exists\"\n**Symptom**: `{\"error\":\"database already registered\"}`\n**Fix**: The database path is already being replicated. Use `/unregister` first, then `/register` with new settings.\n\n## 5. pprof Debugging\n\nAvailable pprof endpoints on the IPC socket:\n\n```bash\n# Interactive CPU profile analysis\ncurl -s --unix-socket /var/run/litestream.sock \\\n http://localhost/debug/pprof/profile?seconds=30 > cpu.prof\ngo tool pprof -http=:8080 cpu.prof\n\n# Goroutine dump (useful for deadlock investigation)\ncurl --unix-socket /var/run/litestream.sock \\\n http://localhost/debug/pprof/goroutine?debug=2\n\n# Memory allocation profile\ncurl -s --unix-socket /var/run/litestream.sock \\\n http://localhost/debug/pprof/heap > heap.prof\ngo tool pprof -http=:8080 heap.prof\n\n# Execution trace (captures scheduler, GC, goroutine events)\ncurl -s --unix-socket /var/run/litestream.sock \\\n http://localhost/debug/pprof/trace?seconds=5 > trace.out\ngo tool trace trace.out\n```\n" }, { "path": ".claude/commands/debug-wal.md", "content": "Debug WAL monitoring issues in Litestream. This command helps diagnose problems with WAL change detection, checkpointing, and replication triggers.\n\nFirst, understand the symptoms:\n- Is replication not triggering on changes?\n- Are checkpoints failing or not happening?\n- Is the WAL growing unbounded?\n\nThen debug the monitoring system:\n\n1. **Check monitor goroutine** (db.go:1499):\n```go\n// Verify monitor is running\nfunc (db *DB) monitor() {\n ticker := time.NewTicker(db.MonitorInterval) // Default: 1s\n // Check if ticker is firing\n // Verify checkWAL() is being called\n}\n```\n\n2. **Verify WAL change detection**:\n```go\n// Check if WAL changes are detected\nfunc (db *DB) checkWAL() (bool, error) {\n // Get WAL size and checksum\n // Compare with previous values\n // Should return true if changed\n}\n```\n\n3. **Debug checkpoint triggers**:\n```go\n// Check checkpoint thresholds\nMinCheckpointPageN int // Default: 1000 pages\nTruncatePageN int // Default: 121359 pages\n\n// Verify WAL page count\nwalPageCount := db.WALPageCount()\nif walPageCount > db.MinCheckpointPageN {\n // Should trigger passive checkpoint\n}\nif walPageCount > db.TruncatePageN {\n // Should trigger truncate checkpoint (emergency brake)\n}\n```\n\n4. **Check long-running read transaction**:\n```go\n// Ensure rtx is maintained\nif db.rtx == nil {\n // Read transaction lost - replication may fail\n}\n```\n\n5. **Monitor notification channel**:\n```go\n// Check if replicas are notified\nselect {\ncase <-db.notify:\n // WAL change detected\ndefault:\n // No changes\n}\n```\n\nCommon issues to check:\n- MonitorInterval too long (default 1s)\n- Checkpoint failing due to active transactions\n- Read transaction preventing checkpoint\n- Notify channel not triggering replicas\n- WAL file permissions issues\n\nDebug commands:\n```sql\n-- Check WAL status\nPRAGMA wal_checkpoint;\nPRAGMA journal_mode;\nPRAGMA page_count;\nPRAGMA wal_autocheckpoint;\n\n-- Check for locks\nSELECT * FROM pragma_lock_status();\n```\n\nTesting WAL monitoring:\n```go\nfunc TestDB_WALMonitoring(t *testing.T) {\n db := setupTestDB(t)\n\n // Set fast monitoring for test\n db.MonitorInterval = 10 * time.Millisecond\n\n // Write data\n writeTestData(t, db, 100)\n\n // Wait for notification\n select {\n case <-db.notify:\n // Success\n case <-time.After(1 * time.Second):\n t.Error(\"WAL change not detected\")\n }\n}\n```\n\nMonitor with logging:\n```go\nslog.Debug(\"wal check\",\n \"size\", walInfo.Size,\n \"checksum\", walInfo.Checksum,\n \"pages\", walInfo.PageCount)\n```\n" }, { "path": ".claude/commands/fix-common-issues.md", "content": "---\ndescription: Fix common Litestream issues\n---\n\n# Fix Common Issues Command\n\nDiagnose and fix common issues in Litestream deployments.\n\n## Issue 1: Lock Page Not Being Skipped\n\n**Symptom**: Errors or corruption with databases >1GB\n\n**Check**:\n```bash\n# Find lock page references\ngrep -r \"LockPgno\" --include=\"*.go\"\n```\n\n**Fix**:\n```go\n// Ensure all page iterations skip lock page\nlockPgno := ltx.LockPgno(pageSize)\nif pgno == lockPgno {\n continue\n}\n```\n\n## Issue 2: Race Condition in Replica Position\n\n**Symptom**: Data races detected, inconsistent position tracking\n\n**Check**:\n```bash\ngo test -race -v -run TestReplica_Sync ./...\n```\n\n**Fix**:\n```go\n// Change from RLock to Lock for writes\nfunc (r *Replica) SetPos(pos ltx.Pos) {\n r.mu.Lock() // NOT RLock!\n defer r.mu.Unlock()\n r.pos = pos\n}\n```\n\n## Issue 3: Eventual Consistency Issues\n\n**Symptom**: Compaction failures, partial file reads\n\n**Check**:\n```bash\n# Look for remote reads during compaction\ngrep -r \"OpenLTXFile\" db.go | grep -v \"os.Open\"\n```\n\n**Fix**:\n```go\n// Always try local first\nf, err := os.Open(db.LTXPath(info.Level, info.MinTXID, info.MaxTXID))\nif err == nil {\n return f, nil\n}\n// Only fall back to remote if local doesn't exist\nreturn replica.Client.OpenLTXFile(...)\n```\n\n## Issue 4: CreatedAt Timestamp Loss\n\n**Symptom**: Point-in-time recovery lacks accurate timestamps\n\n**Check**:\n```go\ninfo, err := client.WriteLTXFile(ctx, level, minTXID, maxTXID, r)\nif err != nil {\n t.Fatal(err)\n}\nif info.CreatedAt.IsZero() {\n t.Fatal(\"CreatedAt not set\")\n}\n```\n\n**Fix**:\n```go\n// Ensure storage metadata is copied into the returned FileInfo\nmodTime := resp.LastModified\ninfo.CreatedAt = modTime\n```\n\n## Issue 5: Non-Atomic File Writes\n\n**Symptom**: Partial files, corruption on crash\n\n**Check**:\n```bash\n# Find direct writes without temp files\ngrep -r \"os.Create\\|os.WriteFile\" --include=\"*.go\"\n```\n\n**Fix**:\n```go\n// Write to temp, then rename\ntmpPath := path + \".tmp\"\nif err := os.WriteFile(tmpPath, data, 0644); err != nil {\n return err\n}\nreturn os.Rename(tmpPath, path)\n```\n\n## Issue 6: WAL Checkpoint Blocking\n\n**Symptom**: WAL grows indefinitely, database locks\n\n**Check**:\n```sql\n-- Check WAL size\nPRAGMA wal_checkpoint(PASSIVE);\nSELECT page_count * page_size FROM pragma_page_count(), pragma_page_size();\n```\n\n**Fix**:\n```go\n// Release read transaction periodically\ndb.rtx.Rollback()\ndb.rtx = nil\n// Checkpoint\ndb.db.Exec(\"PRAGMA wal_checkpoint(RESTART)\")\n// Restart read transaction\ndb.initReadTx()\n```\n\n## Issue 7: Memory Leaks\n\n**Symptom**: Growing memory usage over time\n\n**Check**:\n```bash\n# Generate heap profile\ngo test -memprofile=mem.prof -run=XXX -bench=.\ngo tool pprof -top mem.prof\n```\n\n**Fix**:\n```go\n// Use sync.Pool for buffers\nvar pagePool = sync.Pool{\n New: func() interface{} {\n b := make([]byte, pageSize)\n return &b\n },\n}\n\n// Close resources properly\ndefer func() {\n if f != nil {\n f.Close()\n }\n}()\n```\n\n## Issue 8: Corrupted or Missing LTX Files\n\n**Symptom**: Sync failures with errors like \"ltx validation failed\", \"nonsequential page numbers\",\n\"non-contiguous transaction files\", or persistent sync retry backoff loops after unclean shutdowns.\n\n**Check**:\n```bash\n# Look for LTXError messages in logs\nrg -i \"ltx.*error|reset.*local|auto.recover\" /var/log/litestream.log\n\n# Check if meta directory has corrupted state (note dot prefix)\nls -la /path/to/.database.db-litestream/ltx/\n```\n\n**Fix - Manual Reset**:\n```bash\n# Clears local LTX state, forces fresh snapshot on next sync\n# Database file is NOT modified\nlitestream reset /path/to/database.db\n\n# With explicit config file\nlitestream reset -config /etc/litestream.yml /path/to/database.db\n```\n\n**Fix - Automatic Recovery**:\n```yaml\n# Add to replica config in litestream.yml\ndbs:\n - path: /path/to/database.db\n replicas:\n - url: s3://bucket/path\n auto-recover: true # Automatically resets on LTX errors\n```\n\n**When to use which**: Use `auto-recover` for unattended deployments where automatic recovery\nis preferred over manual intervention. Use manual `reset` when you want to investigate the\ncorruption first. `auto-recover` is disabled by default because resetting discards local LTX\nhistory, which may reduce point-in-time restore granularity.\n\n**Reference**: `cmd/litestream/reset.go`, `replica.go` (auto-recover logic), `db.go` (`ResetLocalState`)\n\n## Issue 9: IPC Socket Connection Failures\n\n**Symptom**: `connection refused` or `no such file or directory` when using the control socket\n\n**Check**:\n```bash\n# Verify socket exists and has correct permissions\nls -la /var/run/litestream.sock\n\n# Verify socket is enabled in config\nrg -A3 'socket:' /etc/litestream.yml\n\n# Check if litestream process is running\npgrep -a litestream\n```\n\n**Fix**:\n```yaml\n# Enable socket in litestream.yml\nsocket:\n enabled: true\n path: /var/run/litestream.sock\n permissions: 0600\n```\n\n**Stale socket**: If litestream crashed, the socket file may still exist. The process creates a new socket on startup and will fail if the stale file exists. Remove it manually:\n```bash\nrm /var/run/litestream.sock\n```\n\n**Reference**: `server.go` (`SocketConfig`, `Server.Start`)\n\n## Issue 10: Backup Files Accumulating (Retention Disabled)\n\n**Symptom**: Storage usage growing unbounded, old LTX files never deleted\n\n**Check**:\n```bash\n# Look for retention warning in logs\nrg \"retention disabled\" /var/log/litestream.log\n```\n\n**Fix**:\n```yaml\n# Option 1: Re-enable Litestream retention (default)\nretention:\n enabled: true\n\n# Option 2: Keep disabled, but configure cloud lifecycle policies\n# Example: S3 lifecycle rule to expire objects in ltx/ prefix after 30 days\n```\n\n**Reference**: `store.go` (`SetRetentionEnabled`), `compactor.go` (`RetentionEnabled`), `cmd/litestream/replicate.go:295`\n\n## Issue 11: v0.3.x Restore Not Finding Backups\n\n**Symptom**: Restore fails with \"no snapshots available\" but v0.3.x backups exist in storage\n\n**Check**:\n```bash\n# Verify v0.3.x backup structure exists\naws s3 ls s3://bucket/path/generations/ --recursive | head -20\n```\n\n**Fix**: Ensure the backend implements `ReplicaClientV3`. The S3 backend supports this automatically. The restore process checks for v0.3.x backups when no v0.4.x+ backup is found.\n\n**Reference**: `v3.go` (`ReplicaClientV3` interface), `s3/replica_client.go`\n\n## Diagnostic Commands\n\n```bash\n# Check database integrity\nsqlite3 database.db \"PRAGMA integrity_check;\"\n\n# List replicated LTX files\nlitestream ltx /path/to/db.sqlite\n\n# Check replication status\nlitestream databases\n\n# Reset corrupted local state\nlitestream reset /path/to/database.db\n\n# Test restoration\nlitestream restore -o test.db [replica-url]\n\n# IPC socket diagnostics (requires socket.enabled: true)\ncurl --unix-socket /var/run/litestream.sock http://localhost/info\ncurl --unix-socket /var/run/litestream.sock http://localhost/list\ncurl --unix-socket /var/run/litestream.sock \"http://localhost/txid?path=/path/to/db\"\n```\n\n## Prevention Checklist\n\n- [ ] Always test with databases >1GB\n- [ ] Run with race detector in CI\n- [ ] Test all page sizes (4KB, 8KB, 16KB, 32KB)\n- [ ] Verify eventual consistency handling\n- [ ] Check for proper locking (Lock vs RLock)\n- [ ] Ensure atomic file operations\n- [ ] Preserve timestamps in compaction\n" }, { "path": ".claude/commands/run-comprehensive-tests.md", "content": "---\ndescription: Run comprehensive test suite for Litestream\n---\n\n# Run Comprehensive Tests Command\n\nExecute a full test suite including unit tests, integration tests, race detection, and large database tests.\n\n## Quick Test Suite\n\n```bash\n# Basic tests with race detection\ngo test -race -v ./...\n\n# With coverage\ngo test -race -cover -v ./...\n```\n\n## Full Test Suite\n\n### 1. Unit Tests\n```bash\necho \"=== Running Unit Tests ===\"\ngo test -v ./... -short\n```\n\n### 2. Race Condition Tests\n```bash\necho \"=== Testing for Race Conditions ===\"\ngo test -race -v -run TestReplica_Sync ./...\ngo test -race -v -run TestDB_Sync ./...\ngo test -race -v -run TestStore_CompactDB ./...\ngo test -race -v ./...\n```\n\n### 3. Integration Tests\n```bash\necho \"=== Running Integration Tests ===\"\n\n# S3 (requires AWS credentials)\nAWS_ACCESS_KEY_ID=xxx AWS_SECRET_ACCESS_KEY=yyy \\\n go test -v ./replica_client_test.go -integration s3\n\n# Google Cloud Storage (requires credentials)\nGOOGLE_APPLICATION_CREDENTIALS=/path/to/creds.json \\\n go test -v ./replica_client_test.go -integration gcs\n\n# Azure Blob Storage\nAZURE_STORAGE_ACCOUNT=xxx AZURE_STORAGE_KEY=yyy \\\n go test -v ./replica_client_test.go -integration abs\n\n# SFTP (requires SSH server)\ngo test -v ./replica_client_test.go -integration sftp\n\n# File system (always available)\ngo test -v ./replica_client_test.go -integration file\n```\n\n### 4. Large Database Tests (>1GB)\n```bash\necho \"=== Testing Large Databases ===\"\n\n# Create test database for each page size\nfor pagesize in 4096 8192 16384 32768; do\n echo \"Testing page size: $pagesize\"\n\n # Create >1GB database\n sqlite3 test-${pagesize}.db <70% for core packages\n✅ Lock page correctly skipped for all page sizes\n✅ Restoration works for databases >1GB\n✅ No memory leaks in benchmarks\n" }, { "path": ".claude/commands/test-compaction.md", "content": "Test Litestream compaction logic. This command helps test and debug compaction issues, especially with eventually consistent storage backends.\n\nFirst, understand the test scenario:\n- What storage backend needs testing?\n- What size database is involved?\n- Are there eventual consistency concerns?\n\nThen create comprehensive tests:\n\n1. **Test basic compaction**:\n```go\nfunc TestCompaction_Basic(t *testing.T) {\n // Create multiple LTX files at level 0\n // Run compaction to level 1\n // Verify merged file is correct\n}\n```\n\n2. **Test with eventual consistency**:\n```go\nfunc TestStore_CompactDB_RemotePartialRead(t *testing.T) {\n // Use mock client that returns partial data initially\n // Verify compaction prefers local files\n // Ensure no corruption occurs\n}\n```\n\n3. **Test lock page handling during compaction**:\n```go\nfunc TestCompaction_LockPage(t *testing.T) {\n // Create database > 1GB\n // Compact with data around lock page\n // Verify lock page is skipped (page at 0x40000000)\n}\n```\n\n4. **Test timestamp preservation**:\n```go\nfunc TestCompaction_PreserveTimestamps(t *testing.T) {\n // Compact files with different CreatedAt times\n // Verify earliest timestamp is preserved\n}\n```\n\nKey areas to test:\n- Reading from local files first (db.go:1280-1294)\n- Skipping lock page at 1GB boundary\n- Preserving CreatedAt timestamps\n- Handling partial/incomplete remote files\n- Concurrent compaction safety\n\nRun with race detector:\n```bash\ngo test -race -v -run TestStore_CompactDB ./...\n```\n\nUse the test harness for large databases:\n```bash\n./bin/litestream-test populate -db test.db -target-size 1.5GB\n./bin/litestream-test validate -source-db test.db -replica-url file:///tmp/replica\n```\n" }, { "path": ".claude/commands/trace-replication.md", "content": "Trace the complete replication flow in Litestream. This command helps understand how changes flow from SQLite through to storage backends.\n\nFollow the replication path step by step:\n\n1. **Application writes to SQLite**:\n```sql\n-- Application performs write\nINSERT INTO table VALUES (...);\n-- SQLite appends to WAL file\n```\n\n2. **DB.monitor() syncs the shadow WAL** (db.go:1499):\n```go\nticker := time.NewTicker(db.MonitorInterval) // default 1s\nfor {\n select {\n case <-db.ctx.Done():\n return\n case <-ticker.C:\n }\n\n if err := db.Sync(db.ctx); err != nil && !errors.Is(err, context.Canceled) {\n db.Logger.Error(\"sync error\", \"error\", err)\n }\n}\n```\n\n3. **Replica.monitor() responds** (replica.go):\n```go\nticker := time.NewTicker(r.SyncInterval)\ndefer ticker.Stop()\n\nnotify := r.db.Notify()\nfor {\n select {\n case <-ctx.Done():\n return\n case <-ticker.C:\n // Enforce minimum sync interval\n case <-notify:\n // WAL changed, time to sync\n }\n\n notify = r.db.Notify()\n\n if err := r.Sync(ctx); err != nil && !errors.Is(err, context.Canceled) {\n r.Logger().Error(\"monitor error\", \"error\", err)\n }\n}\n```\n\n4. **Replica.Sync() uploads new L0 files** (replica.go):\n```go\n// Determine local database position\ndpos, err := r.db.Pos()\nif err != nil {\n return err\n}\nif dpos.IsZero() {\n return fmt.Errorf(\"no position, waiting for data\")\n}\n\n// Upload each unreplicated L0 file\nfor txID := r.Pos().TXID + 1; txID <= dpos.TXID; txID = r.Pos().TXID + 1 {\n if err := r.uploadLTXFile(ctx, 0, txID, txID); err != nil {\n return err\n }\n r.SetPos(ltx.Pos{TXID: txID})\n}\n```\n\n5. **ReplicaClient uploads to storage**:\n```go\nfunc (c *S3Client) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n // Stream LTX data to storage and return metadata (size, CreatedAt, checksums)\n}\n```\n\n6. **Checkpoint when thresholds are hit**:\n```go\nif walPageCount > db.MinCheckpointPageN {\n db.Checkpoint(ctx, litestream.CheckpointModePassive)\n}\nif walPageCount > db.TruncatePageN {\n db.Checkpoint(ctx, litestream.CheckpointModeTruncate)\n}\n```\n\nKey synchronization points:\n- WAL monitoring (1s intervals)\n- Replica sync (configurable, default 1s)\n- Checkpoint triggers (page thresholds)\n- Compaction (hourly/daily)\n\nTrace with logging:\n```go\n// Enable debug logging\nslog.SetLogLevel(slog.LevelDebug)\n\n// Key log points:\nslog.Debug(\"wal changed\", \"size\", walSize)\nslog.Debug(\"syncing replica\", \"pos\", r.Pos())\nslog.Debug(\"ltx uploaded\", \"txid\", maxTXID)\nslog.Debug(\"checkpoint complete\", \"mode\", mode)\n```\n\nPerformance metrics to monitor:\n- WAL growth rate\n- Sync latency\n- Upload throughput\n- Checkpoint frequency\n- Compaction duration\n\nCommon bottlenecks:\n1. Slow storage uploads\n2. Large transactions causing big LTX files\n3. Long-running read transactions blocking checkpoints\n4. Eventual consistency delays\n5. Network latency to storage\n\nTest replication flow:\n```bash\n# Start replication with verbose logging\nlitestream replicate -v\n\n# In another terminal, write to database\nsqlite3 test.db \"INSERT INTO test VALUES (1, 'data');\"\n\n# Watch logs for flow:\n# - WAL change detected\n# - Replica sync triggered\n# - LTX file uploaded\n# - Position updated\n```\n\nVerify replication:\n```bash\n# List replicated files\naws s3 ls s3://bucket/path/ltx/0000/\n\n# Restore and verify\nlitestream restore -o restored.db s3://bucket/path\nsqlite3 restored.db \"SELECT * FROM test;\"\n```\n" }, { "path": ".claude/commands/validate-replica.md", "content": "Validate a ReplicaClient implementation in Litestream. This command helps ensure a replica client correctly implements the interface and handles edge cases.\n\nFirst, identify what needs validation:\n- Which replica client implementation?\n- What storage backend specifics?\n- Any known issues or concerns?\n\nThen validate the implementation:\n\n1. **Interface compliance check**:\n\n ```go\n // Ensure all methods are implemented\n var _ litestream.ReplicaClient = (*YourClient)(nil)\n ```\n\n1. **Verify error types**:\n\n ```go\n // OpenLTXFile must return os.ErrNotExist for missing files\n _, err := client.OpenLTXFile(ctx, 0, 999, 999, 0, 0)\n if !errors.Is(err, os.ErrNotExist) {\n t.Errorf(\"Expected os.ErrNotExist, got %v\", err)\n }\n ```\n\n1. **Test partial reads**:\n\n ```go\n // Must support offset and size parameters\n rc, err := client.OpenLTXFile(ctx, 0, 1, 100, 50, 25)\n data, _ := io.ReadAll(rc)\n if len(data) != 25 {\n t.Errorf(\"Expected 25 bytes, got %d\", len(data))\n }\n ```\n\n1. **Verify timestamp preservation**:\n\n ```go\n // CreatedAt should reflect remote object metadata (or upload time)\n start := time.Now()\n info, _ := client.WriteLTXFile(ctx, 0, 1, 100, reader)\n if info.CreatedAt.IsZero() || info.CreatedAt.Before(start.Add(-time.Second)) {\n t.Error(\"unexpected CreatedAt timestamp\")\n }\n ```\n\n1. **Test eventual consistency handling**:\n - Implement retry logic for transient failures\n - Handle partial file availability\n - Verify write-after-write consistency\n\n1. **Validate cleanup**:\n\n ```go\n // DeleteAll must remove everything\n err := client.DeleteAll(ctx)\n files, _ := client.LTXFiles(ctx, 0, 0, false)\n if files.Next() {\n t.Error(\"Files remain after DeleteAll\")\n }\n ```\n\nKey validation points:\n- Proper error types (os.ErrNotExist, os.ErrPermission)\n- Context cancellation handling\n- Concurrent operation safety\n- Iterator doesn't load all files at once\n- Proper path construction for storage backend\n\nRun integration tests:\n```bash\ngo test -v ./[backend]/replica_client_test.go -integration\n```\n" }, { "path": ".claude/settings.json", "content": "{\n \"project_name\": \"Litestream\",\n \"description\": \"Standalone disaster recovery tool for SQLite\",\n\n \"auto_formatting\": {\n \"enabled\": true,\n \"markdown\": {\n \"enabled\": true,\n \"tool\": \"markdownlint\"\n },\n \"json\": {\n \"enabled\": true,\n \"tool\": \"jq\"\n }\n },\n\n \"file_permissions\": {\n \"read_only_patterns\": [\n \"*.pdf\"\n ]\n }\n}\n" }, { "path": ".github/ISSUE_TEMPLATE/bug_report.md", "content": "---\nname: Bug report\nabout: Create a report to help us improve Litestream\ntitle: ''\nlabels: bug\nassignees: ''\n\n---\n\n\n\n## Bug Description\n\nA clear and concise description of the bug.\n\n## Environment\n\n**Litestream version:**\n\n```text\npaste output here\n```\n\n**Operating system & version:**\n\n\n**Installation method:**\n\n\n**Storage backend:**\n\n\n## Steps to Reproduce\n\n\n1. Step one\n2. Step two\n3. Step three\n\n**Expected behavior:**\n\n\n**Actual behavior:**\n\n\n## Configuration\n\n\n
litestream.yml\n\n```yaml\n# Paste your configuration here\n```\n\n
\n\n## Logs\n\n\n\n
Log output\n\n```text\n# Paste relevant logs here\n```\n\n
\n\n## Additional Context\n\nAdd any other context about the problem here (e.g., recent changes, related issues, workarounds you've tried).\n" }, { "path": ".github/ISSUE_TEMPLATE/config.yml", "content": "blank_issues_enabled: true\ncontact_links:\n - name: Documentation Issues\n url: https://github.com/benbjohnson/litestream.io/issues/new/choose\n about: Report documentation bugs or suggest improvements\n - name: Documentation\n url: https://litestream.io\n about: Check our documentation for setup and usage guides\n" }, { "path": ".github/ISSUE_TEMPLATE/feature_request.md", "content": "---\nname: Feature request\nabout: Suggest an idea for Litestream\ntitle: ''\nlabels: enhancement\nassignees: ''\n\n---\n\n\n\n\n## Feature Description\n\n\n## Use Cases\n\n\n\n## Attempted Solutions\n\n\n\n## Proposal\n\n\n" }, { "path": ".github/pull_request_template.md", "content": "## Description\n\n\n## Motivation and Context\n\n\nFixes #(issue number)\n\n## How Has This Been Tested?\n\n\n\n## Types of changes\n\n- [ ] Bug fix (non-breaking change which fixes an issue)\n- [ ] New feature (non-breaking change which adds functionality)\n- [ ] Breaking change (would cause existing functionality to not work as expected)\n\n## Checklist\n\n- [ ] My code follows the code style of this project (`go fmt`, `go vet`)\n- [ ] I have tested my changes (`go test ./...`)\n- [ ] I have updated the documentation accordingly (if needed)\n" }, { "path": ".github/workflows/commit.yml", "content": "on:\n push:\n branches:\n - main\n pull_request:\n types:\n - opened\n - synchronize\n - reopened\n\nname: Commit\njobs:\n lint:\n name: Lint\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - run: |\n go install golang.org/x/tools/cmd/goimports@latest\n go install honnef.co/go/tools/cmd/staticcheck@latest\n export PATH=\"$HOME/go/bin:$PATH\"\n\n - uses: pre-commit/action@v3.0.0\n\n vfs-build-test:\n name: VFS Build Test (macOS)\n runs-on: macos-latest\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Build VFS shared library\n run: make vfs\n\n - name: Verify shared library created\n run: file dist/litestream-vfs.so\n\n vfs-build-test-linux:\n name: VFS Build Test (Linux)\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Build VFS Linux AMD64\n run: make vfs-linux-amd64\n\n - name: Verify shared library created\n run: file dist/litestream-vfs-linux-amd64.so\n\n build-windows:\n name: Build Windows\n runs-on: ubuntu-latest\n steps:\n - run: sudo apt-get install -y mingw-w64\n\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - run: |\n go build ./cmd/litestream/\n file ./litestream.exe\n env:\n CGO_ENABLED: \"1\"\n GOOS: windows\n GOARCH: amd64\n CC: x86_64-w64-mingw32-gcc\n\n docker-smoke-test:\n name: Docker Smoke Test\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: docker/setup-buildx-action@v3\n\n - name: Build default image\n uses: docker/build-push-action@v6\n with:\n context: .\n target: default\n push: false\n load: true\n tags: litestream:default\n\n - name: Build hardened image\n uses: docker/build-push-action@v6\n with:\n context: .\n target: hardened\n push: false\n load: true\n tags: litestream:hardened\n\n - name: \"Default: verify version command\"\n run: docker run --rm litestream:default version\n\n - name: \"Hardened: verify version command\"\n run: docker run --rm litestream:hardened version\n\n - name: \"Hardened: verify non-root user (UID 65532)\"\n run: |\n user=$(docker inspect --format='{{.Config.User}}' litestream:hardened)\n echo \"Container user: $user\"\n if [ \"$user\" != \"nonroot:nonroot\" ]; then\n echo \"FAIL: Expected user 'nonroot:nonroot', got '$user'\"\n exit 1\n fi\n docker create --name uid-check litestream:hardened version\n uid=$(docker export uid-check | tar -xf - --to-stdout etc/passwd | grep '^nonroot:' | cut -d: -f3)\n docker rm uid-check\n echo \"nonroot UID: $uid\"\n if [ \"$uid\" != \"65532\" ]; then\n echo \"FAIL: Expected UID 65532, got '$uid'\"\n exit 1\n fi\n\n - name: \"Hardened: verify no shell exists\"\n run: |\n if docker run --rm --entrypoint /bin/sh litestream:hardened -c \"echo hello\" 2>/dev/null; then\n echo \"FAIL: Shell should not exist in scratch image\"\n exit 1\n fi\n echo \"PASS: No shell in scratch image\"\n\n - name: \"Hardened: verify CA certificates present\"\n run: |\n docker create --name cert-check litestream:hardened version\n docker export cert-check | tar -tf - | grep -q \"etc/ssl/certs/ca-certificates.crt\"\n echo \"PASS: CA certificates found\"\n docker rm cert-check\n\n build:\n name: Build & Unit Test\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - run: go env\n\n - run: go install ./cmd/litestream\n\n - run: go build -o /dev/null ./_examples/library/basic\n - run: go build -o /dev/null ./_examples/library/s3\n - run: go test -v ./_examples/library\n\n - run: go test -v .\n - run: go test -v ./internal\n - run: go test -v ./abs\n - run: go test -v ./file\n - run: go test -v ./gs\n - run: go test -v ./nats\n - run: go test -v ./s3\n - run: go test -v ./sftp\n - run: go test -v ./cmd/litestream\n\n\n# long-running-test:\n# name: Run Long Running Unit Test\n# runs-on: ubuntu-latest\n# steps:\n# - uses: actions/checkout@v2\n# - uses: actions/setup-go@v2\n# with:\n# go-version: '1.24'\n# - uses: actions/cache@v2\n# with:\n# path: ~/go/pkg/mod\n# key: ${{ inputs.os }}-go-${{ hashFiles('**/go.sum') }}\n# restore-keys: ${{ inputs.os }}-go-\n#\n# - run: go install ./cmd/litestream\n# - run: go test -v -run=TestCmd_Replicate_LongRunning ./integration -long-running-duration 1m\n\n s3-mock-test:\n name: Run S3 Mock Tests\n runs-on: ubuntu-latest\n needs: build\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-python@v5\n with:\n python-version: '3.12'\n# cache: 'pip'\n - run: pip install moto[s3,server]\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - run: go env\n\n - run: go install ./cmd/litestream\n\n - run: ./etc/s3_mock.py go test -v ./replica_client_test.go -integration s3\n\n minio-integration-test:\n name: Run MinIO Integration Tests\n runs-on: ubuntu-latest\n needs: build\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Start MinIO Server\n run: |\n docker run -d \\\n --name minio-test \\\n -p 9000:9000 \\\n -p 9001:9001 \\\n -e MINIO_ROOT_USER=minioadmin \\\n -e MINIO_ROOT_PASSWORD=minioadmin \\\n quay.io/minio/minio server /data --console-address \":9001\"\n\n # Wait for MinIO to be ready\n echo \"Waiting for MinIO server to be ready...\"\n for i in {1..30}; do\n if docker exec minio-test mc alias set local http://localhost:9000 minioadmin minioadmin 2>/dev/null; then\n echo \"MinIO server is ready\"\n break\n fi\n echo \"Waiting for MinIO server... ($i/30)\"\n sleep 1\n done\n\n # Create test bucket\n docker exec minio-test mc mb local/testbucket\n\n - run: go install ./cmd/litestream\n\n - name: Test Query Parameter Support\n run: |\n # Create test database\n sqlite3 /tmp/test.db \"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT); INSERT INTO users (name) VALUES ('Alice'), ('Bob');\"\n\n # Test replicate with query parameters\n export AWS_ACCESS_KEY_ID=minioadmin\n export AWS_SECRET_ACCESS_KEY=minioadmin\n\n # Run replication for 5 seconds\n timeout 5 litestream replicate /tmp/test.db \\\n \"s3://testbucket/test.db?endpoint=localhost:9000&forcePathStyle=true\" || true\n\n # Verify files were uploaded\n docker exec minio-test mc ls local/testbucket/test.db/\n\n # Test restore with query parameters\n rm -f /tmp/restored.db\n litestream restore -o /tmp/restored.db \\\n \"s3://testbucket/test.db?endpoint=localhost:9000&forcePathStyle=true\"\n\n # Verify restored data\n if [ \"$(sqlite3 /tmp/restored.db 'SELECT COUNT(*) FROM users;')\" != \"2\" ]; then\n echo \"ERROR: Restored database does not have expected data\"\n exit 1\n fi\n\n echo \"MinIO integration test with query parameters passed!\"\n\n - name: Cleanup\n if: always()\n run: |\n docker stop minio-test || true\n docker rm minio-test || true\n\n nats-docker-test:\n name: Run NATS Integration Tests\n runs-on: ubuntu-latest\n needs: build\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Start NATS Server with JetStream\n run: |\n docker run -d \\\n --name nats-test \\\n -p 4222:4222 \\\n -p 8222:8222 \\\n nats:latest \\\n -js \\\n -DV\n\n # Wait for NATS to be ready\n echo \"Waiting for NATS server to be ready...\"\n for i in {1..30}; do\n if nc -z localhost 4222; then\n echo \"NATS server is ready\"\n break\n fi\n echo \"Waiting for NATS server... ($i/30)\"\n sleep 1\n done\n\n - name: Create NATS Object Store Bucket\n run: |\n # Install NATS CLI - get latest version using jq for JSON parsing\n NATS_VERSION=$(curl -fsSL -H \"Authorization: token ${{ secrets.GITHUB_TOKEN }}\" \\\n https://api.github.com/repos/nats-io/natscli/releases/latest \\\n | jq -r '.tag_name' \\\n | sed 's/^v//')\n if [ -z \"$NATS_VERSION\" ] || [ \"$NATS_VERSION\" = \"null\" ]; then\n NATS_VERSION=\"0.3.1\"\n echo \"Warning: Failed to fetch latest NATS CLI version, using fallback v${NATS_VERSION}\"\n fi\n wget \"https://github.com/nats-io/natscli/releases/download/v${NATS_VERSION}/nats-${NATS_VERSION}-linux-amd64.zip\" -O nats.zip\n unzip nats.zip\n sudo mv \"nats-${NATS_VERSION}-linux-amd64/nats\" /usr/local/bin/\n sudo chmod +x /usr/local/bin/nats\n\n # Create the object store bucket\n nats object add litestream-test --max-bucket-size=100M --replicas=1\n\n - run: go env\n\n - run: go install ./cmd/litestream\n\n - run: go test -v ./replica_client_test.go -integration nats\n env:\n LITESTREAM_NATS_URL: \"nats://localhost:4222\"\n LITESTREAM_NATS_BUCKET: \"litestream-test\"\n\n - name: Cleanup\n if: always()\n run: |\n docker stop nats-test || true\n docker rm nats-test || true\n\n s3-integration-test:\n name: Run S3 Integration Tests\n runs-on: ubuntu-latest\n needs: build\n if: github.ref == 'refs/heads/main'\n concurrency:\n group: integration-test-s3\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - run: go env\n\n - run: go install ./cmd/litestream\n\n - run: go test -v ./replica_client_test.go -integration s3\n env:\n LITESTREAM_S3_ACCESS_KEY_ID: ${{ secrets.LITESTREAM_S3_ACCESS_KEY_ID }}\n LITESTREAM_S3_SECRET_ACCESS_KEY: ${{ secrets.LITESTREAM_S3_SECRET_ACCESS_KEY }}\n LITESTREAM_S3_REGION: us-east-1\n LITESTREAM_S3_BUCKET: integration.litestream.io\n\n gcp-integration-test:\n name: Run GCP Integration Tests\n runs-on: ubuntu-latest\n needs: build\n if: github.ref == 'refs/heads/main'\n concurrency:\n group: integration-test-gcp\n steps:\n - name: Extract GCP credentials\n run: 'echo \"$GOOGLE_APPLICATION_CREDENTIALS\" > /opt/gcp.json'\n shell: bash\n env:\n GOOGLE_APPLICATION_CREDENTIALS: ${{secrets.GOOGLE_APPLICATION_CREDENTIALS}}\n\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - run: go env\n\n - run: go install ./cmd/litestream\n\n - run: go test -v ./replica_client_test.go -integration gs\n env:\n GOOGLE_APPLICATION_CREDENTIALS: /opt/gcp.json\n LITESTREAM_GS_BUCKET: integration.litestream.io\n\n abs-integration-test:\n name: Run Azure Blob Store Integration Tests\n runs-on: ubuntu-latest\n needs: build\n if: github.ref == 'refs/heads/main'\n concurrency:\n group: integration-test-abs\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - run: go env\n\n - run: go install ./cmd/litestream\n\n - run: go test -v ./replica_client_test.go -integration abs\n env:\n LITESTREAM_ABS_ACCOUNT_NAME: ${{ secrets.LITESTREAM_ABS_ACCOUNT_NAME }}\n LITESTREAM_ABS_ACCOUNT_KEY: ${{ secrets.LITESTREAM_ABS_ACCOUNT_KEY }}\n LITESTREAM_ABS_BUCKET: integration\n\n r2-integration-test:\n name: Run Cloudflare R2 Integration Tests\n runs-on: ubuntu-latest\n needs: build\n if: github.ref == 'refs/heads/main'\n concurrency:\n group: integration-test-r2\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - run: go env\n\n - run: go install ./cmd/litestream\n\n - run: go test -v ./replica_client_test.go -integration -replica-clients=r2\n env:\n LITESTREAM_R2_ACCESS_KEY_ID: ${{ secrets.LITESTREAM_R2_ACCESS_KEY_ID }}\n LITESTREAM_R2_SECRET_ACCESS_KEY: ${{ secrets.LITESTREAM_R2_SECRET_ACCESS_KEY }}\n LITESTREAM_R2_ENDPOINT: ${{ secrets.LITESTREAM_R2_ENDPOINT }}\n LITESTREAM_R2_BUCKET: ${{ secrets.LITESTREAM_R2_BUCKET }}\n\n sftp-integration-test:\n name: Run SFTP Integration Tests\n runs-on: ubuntu-latest\n needs: build\n steps:\n - name: Prepare OpenSSH server\n run: |-\n sudo mkdir -p /test/etc/ssh /test/home /run/sshd /test/data/\n sudo ssh-keygen -t ed25519 -f /test/etc/ssh/id_ed25519_host -N \"\"\n sudo ssh-keygen -t ed25519 -f /test/etc/ssh/id_ed25519 -N \"\"\n sudo chmod 0600 /test/etc/ssh/id_ed25519_host /test/etc/ssh/id_ed25519\n sudo chmod 0644 /test/etc/ssh/id_ed25519_host.pub /test/etc/ssh/id_ed25519.pub\n sudo cp /test/etc/ssh/id_ed25519 /test/id_ed25519\n sudo chown $USER /test/id_ed25519\n sudo tee /test/etc/ssh/sshd_config < /tmp/sftp-concurrent.yml < /tmp/sftp-sequential.yml </dev/null 2>&1; then\n echo \"WebDAV server is ready\"\n break\n fi\n sleep 1\n done\n\n # Verify WebDAV is accessible\n curl -u testuser:testpass http://localhost:8080/ || (docker logs webdav-test && exit 1)\n\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - run: go env\n\n - run: go install ./cmd/litestream\n\n - run: go test -v ./replica_client_test.go -integration webdav\n env:\n LITESTREAM_WEBDAV_URL: \"http://localhost:8080\"\n LITESTREAM_WEBDAV_USERNAME: \"testuser\"\n LITESTREAM_WEBDAV_PASSWORD: \"testpass\"\n LITESTREAM_WEBDAV_PATH: \"/testdata\"\n\n - name: Cleanup\n if: always()\n run: |\n docker stop webdav-test || true\n docker rm webdav-test || true\n" }, { "path": ".github/workflows/integration-tests.yml", "content": "name: Integration Tests\n\non:\n pull_request:\n paths:\n - '**.go'\n - 'go.mod'\n - 'go.sum'\n - 'tests/integration/**'\n - '.github/workflows/integration-tests.yml'\n workflow_dispatch:\n inputs:\n test_type:\n description: 'Test type to run'\n required: false\n default: 'quick'\n type: choice\n options:\n - 'quick'\n - 'all'\n - 'long'\n\npermissions:\n contents: read\n\njobs:\n quick-tests:\n name: Quick Integration Tests\n runs-on: ubuntu-latest\n if: github.event_name == 'pull_request' || inputs.test_type == 'quick' || inputs.test_type == 'all'\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Build binaries\n run: |\n go build -o bin/litestream ./cmd/litestream\n go build -o bin/litestream-test ./cmd/litestream-test\n\n - name: Run quick integration tests\n run: |\n go test -v -tags=integration -timeout=30m ./tests/integration/... \\\n -run=\"TestFreshStart|TestDatabaseIntegrity|TestRapidCheckpoints|TestS3AccessPointLocalStack|TestRestore_|TestBinaryCompatibility|TestCompaction_Compatibility|TestVersionMigration|TestUpgrade|TestLockPage|TestDirectoryWatcher|TestDatabaseDeletion|TestWALGrowth|TestBusyTimeout|TestConcurrentOperations\"\n env:\n CGO_ENABLED: 1\n\n - name: Upload test logs\n if: failure()\n uses: actions/upload-artifact@v4\n with:\n name: quick-test-logs\n path: |\n /tmp/litestream-*/*.log\n /tmp/*-test.log\n\n scenario-tests:\n name: Scenario Integration Tests\n runs-on: ubuntu-latest\n if: github.event_name == 'workflow_dispatch' && (inputs.test_type == 'all' || inputs.test_type == 'long')\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Build binaries\n run: |\n go build -o bin/litestream ./cmd/litestream\n go build -o bin/litestream-test ./cmd/litestream-test\n\n - name: Run all scenario tests\n run: |\n go test -v -tags=integration -timeout=1h ./tests/integration/... \\\n -run=\"Test(FreshStart|DatabaseIntegrity|DatabaseDeletion|RapidCheckpoints|WALGrowth|ConcurrentOperations|BusyTimeout)\"\n env:\n CGO_ENABLED: 1\n\n - name: Upload test logs\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: scenario-test-logs\n path: |\n /tmp/litestream-*/*.log\n /tmp/*-test.log\n\n long-running-tests:\n name: Long-Running Integration Tests\n runs-on: ubuntu-latest\n if: github.event_name == 'workflow_dispatch' && inputs.test_type == 'long'\n timeout-minutes: 600\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Build binaries\n run: |\n go build -o bin/litestream ./cmd/litestream\n go build -o bin/litestream-test ./cmd/litestream-test\n\n - name: Run long tests\n run: |\n go test -v -tags=\"integration,long\" -timeout=10h ./tests/integration/... \\\n -run=\"TestOvernight|Test1GBBoundary\"\n env:\n CGO_ENABLED: 1\n\n - name: Upload test logs\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: long-test-logs\n path: |\n /tmp/litestream-*/*.log\n /tmp/*-test.log\n\n summary:\n name: Test Summary\n runs-on: ubuntu-latest\n needs: [quick-tests]\n if: always() && (github.event_name == 'pull_request' || inputs.test_type == 'quick' || inputs.test_type == 'all')\n steps:\n - name: Generate summary\n run: |\n echo \"## Integration Test Results\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n\n if [ \"${{ needs.quick-tests.result }}\" == \"success\" ]; then\n echo \"✅ **Quick Tests:** Passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.quick-tests.result }}\" == \"failure\" ]; then\n echo \"❌ **Quick Tests:** Failed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.quick-tests.result }}\" == \"skipped\" ]; then\n echo \"⏭️ **Quick Tests:** Skipped\" >> $GITHUB_STEP_SUMMARY\n fi\n\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"---\" >> $GITHUB_STEP_SUMMARY\n echo \"**Triggered by:** @${{ github.actor }}\" >> $GITHUB_STEP_SUMMARY\n\n # Note: Scenario and long-running tests run independently on workflow_dispatch.\n # Check individual job results for those test suites.\n" }, { "path": ".github/workflows/manual-integration-tests.yml", "content": "name: Manual Integration Tests\n\npermissions:\n issues: write\n pull-requests: write\n\non:\n workflow_dispatch:\n inputs:\n pr_number:\n description: 'Pull Request number to test (optional)'\n required: false\n type: string\n branch:\n description: 'Branch name to test (optional, ignored if PR number is provided)'\n required: false\n type: string\n default: ''\n test_s3:\n description: 'Run S3 integration tests (AWS)'\n required: false\n default: true\n type: boolean\n test_gcs:\n description: 'Run Google Cloud Storage integration tests'\n required: false\n default: false\n type: boolean\n test_abs:\n description: 'Run Azure Blob Storage integration tests'\n required: false\n default: false\n type: boolean\n test_tigris:\n description: 'Run Fly.io Tigris integration tests'\n required: false\n default: false\n type: boolean\n test_r2:\n description: 'Run Cloudflare R2 integration tests'\n required: false\n default: false\n type: boolean\n test_b2:\n description: 'Run Backblaze B2 integration tests'\n required: false\n default: false\n type: boolean\n test_multipart:\n description: 'Run multipart upload stress tests (5MB-50MB files)'\n required: false\n default: false\n type: boolean\n\njobs:\n setup:\n name: Setup and Validation\n runs-on: ubuntu-latest\n outputs:\n ref: ${{ steps.get-ref.outputs.ref }}\n ref_name: ${{ steps.get-ref.outputs.ref_name }}\n steps:\n - name: Determine checkout ref\n id: get-ref\n run: |\n if [ -n \"${{ github.event.inputs.pr_number }}\" ]; then\n echo \"ref=refs/pull/${{ github.event.inputs.pr_number }}/head\" >> $GITHUB_OUTPUT\n echo \"ref_name=PR #${{ github.event.inputs.pr_number }}\" >> $GITHUB_OUTPUT\n echo \"::notice::Testing PR #${{ github.event.inputs.pr_number }}\"\n elif [ -n \"${{ github.event.inputs.branch }}\" ]; then\n echo \"ref=refs/heads/${{ github.event.inputs.branch }}\" >> $GITHUB_OUTPUT\n echo \"ref_name=branch ${{ github.event.inputs.branch }}\" >> $GITHUB_OUTPUT\n echo \"::notice::Testing branch ${{ github.event.inputs.branch }}\"\n else\n echo \"ref=${{ github.ref }}\" >> $GITHUB_OUTPUT\n echo \"ref_name=branch ${{ github.ref_name }}\" >> $GITHUB_OUTPUT\n echo \"::notice::Testing current branch ${{ github.ref_name }}\"\n fi\n\n - name: Validate inputs\n run: |\n if [ \"${{ github.event.inputs.test_s3 }}\" != \"true\" ] && \\\n [ \"${{ github.event.inputs.test_gcs }}\" != \"true\" ] && \\\n [ \"${{ github.event.inputs.test_abs }}\" != \"true\" ] && \\\n [ \"${{ github.event.inputs.test_tigris }}\" != \"true\" ] && \\\n [ \"${{ github.event.inputs.test_r2 }}\" != \"true\" ] && \\\n [ \"${{ github.event.inputs.test_b2 }}\" != \"true\" ] && \\\n [ \"${{ github.event.inputs.test_multipart }}\" != \"true\" ]; then\n echo \"::error::At least one test type must be selected\"\n exit 1\n fi\n\n echo \"### Test Configuration\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"**Target:** ${{ steps.get-ref.outputs.ref_name }}\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"**Integration Tests:**\" >> $GITHUB_STEP_SUMMARY\n echo \"- S3 (AWS): ${{ github.event.inputs.test_s3 }}\" >> $GITHUB_STEP_SUMMARY\n echo \"- Google Cloud Storage: ${{ github.event.inputs.test_gcs }}\" >> $GITHUB_STEP_SUMMARY\n echo \"- Azure Blob Storage: ${{ github.event.inputs.test_abs }}\" >> $GITHUB_STEP_SUMMARY\n echo \"- Fly.io Tigris: ${{ github.event.inputs.test_tigris }}\" >> $GITHUB_STEP_SUMMARY\n echo \"- Cloudflare R2: ${{ github.event.inputs.test_r2 }}\" >> $GITHUB_STEP_SUMMARY\n echo \"- Backblaze B2: ${{ github.event.inputs.test_b2 }}\" >> $GITHUB_STEP_SUMMARY\n echo \"- Multipart Stress Tests: ${{ github.event.inputs.test_multipart }}\" >> $GITHUB_STEP_SUMMARY\n\n s3-integration:\n name: S3 Integration Tests (AWS)\n runs-on: ubuntu-latest\n needs: setup\n if: github.event.inputs.test_s3 == 'true'\n concurrency:\n group: integration-test-s3-manual\n cancel-in-progress: false\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_S3_ACCESS_KEY_ID }}\" ]; then\n echo \"::notice title=Skipped::S3 integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n ref: ${{ needs.setup.outputs.ref }}\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Show Go environment\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go env\n\n - name: Install litestream\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go install ./cmd/litestream\n\n - name: Run S3 integration tests\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go test -v ./replica_client_test.go -integration -replica-clients=s3\n env:\n LITESTREAM_S3_ACCESS_KEY_ID: ${{ secrets.LITESTREAM_S3_ACCESS_KEY_ID }}\n LITESTREAM_S3_SECRET_ACCESS_KEY: ${{ secrets.LITESTREAM_S3_SECRET_ACCESS_KEY }}\n LITESTREAM_S3_REGION: us-east-1\n LITESTREAM_S3_BUCKET: integration.litestream.io\n\n - name: Create test results directory\n if: always()\n run: |\n mkdir -p test-results\n echo \"Test completed at $(date)\" > test-results/s3-test.log\n\n - name: Upload test results\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: s3-test-results\n path: |\n *.log\n test-results/\n\n tigris-integration:\n name: Tigris Integration Tests\n runs-on: ubuntu-latest\n needs: setup\n if: github.event.inputs.test_tigris == 'true'\n concurrency:\n group: integration-test-tigris-manual\n cancel-in-progress: false\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_TIGRIS_ACCESS_KEY_ID }}\" ]; then\n echo \"::notice title=Skipped::Tigris integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n ref: ${{ needs.setup.outputs.ref }}\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Show Go environment\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go env\n\n - name: Install litestream\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go install ./cmd/litestream\n\n - name: Run Tigris integration tests\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go test -v ./replica_client_test.go -integration -replica-clients=tigris\n env:\n LITESTREAM_TIGRIS_ACCESS_KEY_ID: ${{ secrets.LITESTREAM_TIGRIS_ACCESS_KEY_ID }}\n LITESTREAM_TIGRIS_SECRET_ACCESS_KEY: ${{ secrets.LITESTREAM_TIGRIS_SECRET_ACCESS_KEY }}\n\n - name: Create test results directory\n if: always()\n run: |\n mkdir -p test-results\n echo \"Test completed at $(date)\" > test-results/tigris-test.log\n\n - name: Upload test results\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: tigris-test-results\n path: |\n *.log\n test-results/\n\n r2-integration:\n name: Cloudflare R2 Integration Tests\n runs-on: ubuntu-latest\n needs: setup\n if: github.event.inputs.test_r2 == 'true'\n concurrency:\n group: integration-test-r2-manual\n cancel-in-progress: false\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_R2_ACCESS_KEY_ID }}\" ]; then\n echo \"::notice title=Skipped::R2 integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n ref: ${{ needs.setup.outputs.ref }}\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Show Go environment\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go env\n\n - name: Install litestream\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go install ./cmd/litestream\n\n - name: Run R2 integration tests\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go test -v ./replica_client_test.go -integration -replica-clients=r2\n env:\n LITESTREAM_R2_ACCESS_KEY_ID: ${{ secrets.LITESTREAM_R2_ACCESS_KEY_ID }}\n LITESTREAM_R2_SECRET_ACCESS_KEY: ${{ secrets.LITESTREAM_R2_SECRET_ACCESS_KEY }}\n LITESTREAM_R2_ENDPOINT: ${{ secrets.LITESTREAM_R2_ENDPOINT }}\n LITESTREAM_R2_BUCKET: ${{ secrets.LITESTREAM_R2_BUCKET }}\n\n - name: Create test results directory\n if: always()\n run: |\n mkdir -p test-results\n echo \"Test completed at $(date)\" > test-results/r2-test.log\n\n - name: Upload test results\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: r2-test-results\n path: |\n *.log\n test-results/\n\n b2-integration:\n name: Backblaze B2 Integration Tests\n runs-on: ubuntu-latest\n needs: setup\n if: github.event.inputs.test_b2 == 'true'\n concurrency:\n group: integration-test-b2-manual\n cancel-in-progress: false\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_B2_KEY_ID }}\" ]; then\n echo \"::notice title=Skipped::B2 integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n ref: ${{ needs.setup.outputs.ref }}\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Show Go environment\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go env\n\n - name: Install litestream\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go install ./cmd/litestream\n\n - name: Run B2 integration tests\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go test -v ./replica_client_test.go -integration -replica-clients=b2\n env:\n LITESTREAM_B2_KEY_ID: ${{ secrets.LITESTREAM_B2_KEY_ID }}\n LITESTREAM_B2_APPLICATION_KEY: ${{ secrets.LITESTREAM_B2_APPLICATION_KEY }}\n LITESTREAM_B2_ENDPOINT: ${{ secrets.LITESTREAM_B2_ENDPOINT }}\n LITESTREAM_B2_BUCKET: ${{ secrets.LITESTREAM_B2_BUCKET }}\n\n - name: Create test results directory\n if: always()\n run: |\n mkdir -p test-results\n echo \"Test completed at $(date)\" > test-results/b2-test.log\n\n - name: Upload test results\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: b2-test-results\n path: |\n *.log\n test-results/\n\n multipart-stress:\n name: Multipart Upload Stress Tests\n runs-on: ubuntu-latest\n needs: setup\n if: github.event.inputs.test_multipart == 'true'\n concurrency:\n group: integration-test-multipart-manual\n cancel-in-progress: false\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_S3_ACCESS_KEY_ID }}\" ]; then\n echo \"::notice title=Skipped::Multipart stress tests skipped - S3 credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n ref: ${{ needs.setup.outputs.ref }}\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Show Go environment\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go env\n\n - name: Install litestream\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go install ./cmd/litestream\n\n - name: Run multipart threshold tests (AWS S3)\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: |\n go test -v ./replica_client_test.go -integration -replica-clients=s3 \\\n -run \"TestReplicaClient_S3_Multipart|TestReplicaClient_S3_Concurrency\" \\\n -timeout 30m\n env:\n LITESTREAM_S3_ACCESS_KEY_ID: ${{ secrets.LITESTREAM_S3_ACCESS_KEY_ID }}\n LITESTREAM_S3_SECRET_ACCESS_KEY: ${{ secrets.LITESTREAM_S3_SECRET_ACCESS_KEY }}\n LITESTREAM_S3_REGION: us-east-1\n LITESTREAM_S3_BUCKET: integration.litestream.io\n\n - name: Create test results directory\n if: always()\n run: |\n mkdir -p test-results\n echo \"Test completed at $(date)\" > test-results/multipart-test.log\n\n - name: Upload test results\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: multipart-test-results\n path: |\n *.log\n test-results/\n\n gcs-integration:\n name: Google Cloud Storage Integration Tests\n runs-on: ubuntu-latest\n needs: setup\n if: github.event.inputs.test_gcs == 'true'\n concurrency:\n group: integration-test-gcp-manual\n cancel-in-progress: false\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}\" ]; then\n echo \"::notice title=Skipped::GCS integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - name: Extract GCP credentials\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: 'echo \"$GOOGLE_APPLICATION_CREDENTIALS\" > /opt/gcp.json'\n shell: bash\n env:\n GOOGLE_APPLICATION_CREDENTIALS: ${{secrets.GOOGLE_APPLICATION_CREDENTIALS}}\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n ref: ${{ needs.setup.outputs.ref }}\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Show Go environment\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go env\n\n - name: Install litestream\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go install ./cmd/litestream\n\n - name: Run GCS integration tests\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go test -v ./replica_client_test.go -integration -replica-clients=gs\n env:\n GOOGLE_APPLICATION_CREDENTIALS: /opt/gcp.json\n LITESTREAM_GS_BUCKET: litestream-github-workflows\n\n - name: Create test results directory\n if: always()\n run: |\n mkdir -p test-results\n echo \"Test completed at $(date)\" > test-results/gcs-test.log\n\n - name: Upload test results\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: gcs-test-results\n path: |\n *.log\n test-results/\n\n abs-integration:\n name: Azure Blob Storage Integration Tests\n runs-on: ubuntu-latest\n needs: setup\n if: github.event.inputs.test_abs == 'true'\n concurrency:\n group: integration-test-abs-manual\n cancel-in-progress: false\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_ABS_ACCOUNT_NAME }}\" ]; then\n echo \"::notice title=Skipped::Azure Blob Storage integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n ref: ${{ needs.setup.outputs.ref }}\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Show Go environment\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go env\n\n - name: Install litestream\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go install ./cmd/litestream\n\n - name: Run Azure Blob Storage integration tests\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: go test -v ./replica_client_test.go -integration -replica-clients=abs\n env:\n LITESTREAM_ABS_ACCOUNT_NAME: ${{ secrets.LITESTREAM_ABS_ACCOUNT_NAME }}\n LITESTREAM_ABS_ACCOUNT_KEY: ${{ secrets.LITESTREAM_ABS_ACCOUNT_KEY }}\n LITESTREAM_ABS_BUCKET: integration\n\n - name: Create test results directory\n if: always()\n run: |\n mkdir -p test-results\n echo \"Test completed at $(date)\" > test-results/abs-test.log\n\n - name: Upload test results\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: abs-test-results\n path: |\n *.log\n test-results/\n\n summary:\n name: Test Summary\n runs-on: ubuntu-latest\n needs: [setup, s3-integration, gcs-integration, abs-integration, tigris-integration, r2-integration, b2-integration, multipart-stress]\n if: always()\n steps:\n - name: Download all artifacts\n uses: actions/download-artifact@v4\n continue-on-error: true\n\n - name: Generate summary\n run: |\n echo \"## Integration Test Results\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"### Target: ${{ needs.setup.outputs.ref_name }}\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n\n echo \"### Integration Tests\" >> $GITHUB_STEP_SUMMARY\n\n if [ \"${{ needs.s3-integration.result }}\" == \"success\" ]; then\n echo \"✅ **S3 (AWS):** Passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.s3-integration.result }}\" == \"failure\" ]; then\n echo \"❌ **S3 (AWS):** Failed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.s3-integration.result }}\" == \"skipped\" ]; then\n echo \"⏭️ **S3 (AWS):** Skipped\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.gcs-integration.result }}\" == \"success\" ]; then\n echo \"✅ **Google Cloud Storage:** Passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.gcs-integration.result }}\" == \"failure\" ]; then\n echo \"❌ **Google Cloud Storage:** Failed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.gcs-integration.result }}\" == \"skipped\" ]; then\n echo \"⏭️ **Google Cloud Storage:** Skipped\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.abs-integration.result }}\" == \"success\" ]; then\n echo \"✅ **Azure Blob Storage:** Passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.abs-integration.result }}\" == \"failure\" ]; then\n echo \"❌ **Azure Blob Storage:** Failed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.abs-integration.result }}\" == \"skipped\" ]; then\n echo \"⏭️ **Azure Blob Storage:** Skipped\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.tigris-integration.result }}\" == \"success\" ]; then\n echo \"✅ **Fly.io Tigris:** Passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.tigris-integration.result }}\" == \"failure\" ]; then\n echo \"❌ **Fly.io Tigris:** Failed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.tigris-integration.result }}\" == \"skipped\" ]; then\n echo \"⏭️ **Fly.io Tigris:** Skipped\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.r2-integration.result }}\" == \"success\" ]; then\n echo \"✅ **Cloudflare R2:** Passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.r2-integration.result }}\" == \"failure\" ]; then\n echo \"❌ **Cloudflare R2:** Failed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.r2-integration.result }}\" == \"skipped\" ]; then\n echo \"⏭️ **Cloudflare R2:** Skipped\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.b2-integration.result }}\" == \"success\" ]; then\n echo \"✅ **Backblaze B2:** Passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.b2-integration.result }}\" == \"failure\" ]; then\n echo \"❌ **Backblaze B2:** Failed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.b2-integration.result }}\" == \"skipped\" ]; then\n echo \"⏭️ **Backblaze B2:** Skipped\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.multipart-stress.result }}\" == \"success\" ]; then\n echo \"✅ **Multipart Stress Tests:** Passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.multipart-stress.result }}\" == \"failure\" ]; then\n echo \"❌ **Multipart Stress Tests:** Failed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.multipart-stress.result }}\" == \"skipped\" ]; then\n echo \"⏭️ **Multipart Stress Tests:** Skipped\" >> $GITHUB_STEP_SUMMARY\n fi\n\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"---\" >> $GITHUB_STEP_SUMMARY\n echo \"**Triggered by:** @${{ github.actor }}\" >> $GITHUB_STEP_SUMMARY\n echo \"**Run ID:** [${{ github.run_id }}](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})\" >> $GITHUB_STEP_SUMMARY\n echo \"**Workflow:** ${{ github.workflow }}\" >> $GITHUB_STEP_SUMMARY\n\n - name: Comment on PR (if applicable)\n if: github.event.inputs.pr_number != ''\n continue-on-error: true\n uses: actions/github-script@v7\n with:\n script: |\n const pr_number = ${{ github.event.inputs.pr_number }};\n const run_url = `https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}`;\n\n // Build status emoji based on results\n let statusEmoji = '✅';\n const failedJobs = [];\n\n if ('${{ needs.s3-integration.result }}' === 'failure') failedJobs.push('S3');\n if ('${{ needs.gcs-integration.result }}' === 'failure') failedJobs.push('GCS');\n if ('${{ needs.abs-integration.result }}' === 'failure') failedJobs.push('Azure');\n if ('${{ needs.tigris-integration.result }}' === 'failure') failedJobs.push('Tigris');\n if ('${{ needs.r2-integration.result }}' === 'failure') failedJobs.push('R2');\n if ('${{ needs.b2-integration.result }}' === 'failure') failedJobs.push('B2');\n if ('${{ needs.multipart-stress.result }}' === 'failure') failedJobs.push('Multipart');\n\n if (failedJobs.length > 0) {\n statusEmoji = '❌';\n }\n\n let body = `${statusEmoji} **Manual integration tests** have been run by @${{ github.actor }}\\n\\n`;\n\n if (failedJobs.length > 0) {\n body += `Failed jobs: ${failedJobs.join(', ')}\\n\\n`;\n }\n\n body += `[View test results](${run_url})`;\n\n try {\n await github.rest.issues.createComment({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: pr_number,\n body: body\n });\n console.log(`Successfully commented on PR #${pr_number}`);\n } catch (error) {\n console.log(`Failed to comment on PR #${pr_number}: ${error.message}`);\n // Don't fail the workflow if commenting fails\n }\n" }, { "path": ".github/workflows/pr-metrics.yml", "content": "name: PR Build Metrics\n\non:\n pull_request:\n types: [opened, synchronize, reopened]\n\nconcurrency:\n group: pr-metrics-${{ github.event.pull_request.number }}\n cancel-in-progress: true\n\npermissions:\n contents: read\n pull-requests: write\n\njobs:\n build-base:\n runs-on: ubuntu-latest\n steps:\n - name: Checkout base branch\n uses: actions/checkout@v4\n with:\n ref: ${{ github.event.pull_request.base.sha }}\n\n - name: Set up Go\n id: setup-go\n uses: actions/setup-go@v5\n with:\n go-version-file: go.mod\n\n - name: Cache base binary\n id: cache-base\n uses: actions/cache@v4\n with:\n path: litestream-base\n key: pr-metrics-base-${{ github.event.pull_request.base.sha }}-${{ steps.setup-go.outputs.go-version }}\n\n - name: Build base binary\n if: steps.cache-base.outputs.cache-hit != 'true'\n run: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags \"-s -w\" -o litestream-base ./cmd/litestream\n\n - name: Record base binary size\n run: stat --format=%s litestream-base > base-size.txt\n\n - name: Upload base size\n uses: actions/upload-artifact@v4\n with:\n name: base-size\n path: base-size.txt\n\n build-pr:\n runs-on: ubuntu-latest\n steps:\n - name: Checkout PR branch\n uses: actions/checkout@v4\n\n - name: Set up Go\n uses: actions/setup-go@v5\n with:\n go-version-file: go.mod\n\n - name: Build PR binary\n run: |\n START=$SECONDS\n CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags \"-s -w\" -o litestream-pr ./cmd/litestream\n echo \"$((SECONDS - START))\" > build-time.txt\n go version | awk '{print $3}' > go-version.txt\n\n - name: Record PR binary size\n run: stat --format=%s litestream-pr > pr-size.txt\n\n - name: Upload PR size\n uses: actions/upload-artifact@v4\n with:\n name: pr-size\n path: pr-size.txt\n\n - name: Upload build info\n uses: actions/upload-artifact@v4\n with:\n name: build-info\n path: |\n build-time.txt\n go-version.txt\n\n analyze-deps:\n runs-on: ubuntu-latest\n steps:\n - name: Checkout PR branch\n uses: actions/checkout@v4\n with:\n path: pr\n\n - name: Checkout base branch\n uses: actions/checkout@v4\n with:\n ref: ${{ github.event.pull_request.base.sha }}\n path: base\n\n - name: Set up Go\n uses: actions/setup-go@v5\n with:\n go-version-file: pr/go.mod\n\n - name: Diff go.mod dependencies\n run: |\n extract_require_deps() {\n local gomod=\"$1\"\n awk '/^require \\(/{found=1; next} found && /^\\)/{found=0} found && /^\\t/' \"$gomod\" | sed 's/^\\t//' | sort\n }\n extract_require_deps base/go.mod > /tmp/base-deps.txt\n extract_require_deps pr/go.mod > /tmp/pr-deps.txt\n\n {\n echo \"## Added\"\n comm -13 /tmp/base-deps.txt /tmp/pr-deps.txt\n echo \"## Removed\"\n comm -23 /tmp/base-deps.txt /tmp/pr-deps.txt\n } > deps-diff.txt\n\n - name: Module graph size\n run: |\n cd base && go mod graph | wc -l | tr -d ' ' > ../base-graph-size.txt && cd ..\n cd pr && go mod graph | wc -l | tr -d ' ' > ../pr-graph-size.txt && cd ..\n\n - name: Run govulncheck\n run: |\n go install golang.org/x/vuln/cmd/govulncheck@latest\n cd pr\n govulncheck ./... > ../vulncheck-results.txt 2>&1 || true\n\n - name: Check Go toolchain freshness\n run: |\n CURRENT=$(grep '^toolchain ' pr/go.mod | awk '{print $2}' | sed 's/^go//')\n if [ -z \"$CURRENT\" ]; then\n CURRENT=$(grep '^go ' pr/go.mod | awk '{print $2}')\n fi\n MINOR=$(echo \"$CURRENT\" | grep -oE '^[0-9]+\\.[0-9]+')\n\n LATEST=$(curl -sf 'https://go.dev/dl/?mode=json&include=all' | \\\n python3 -c \"import sys,json; releases=json.load(sys.stdin); print(next(r['version'] for r in releases if r['stable'] and r['version'].startswith('go${MINOR}.')))\" 2>/dev/null | sed 's/^go//')\n\n if [ -z \"$LATEST\" ]; then\n echo \"current=${CURRENT}\" > go-toolchain.txt\n echo \"latest=unknown\" >> go-toolchain.txt\n echo \"stale=false\" >> go-toolchain.txt\n elif [ \"$CURRENT\" = \"$LATEST\" ]; then\n echo \"current=${CURRENT}\" > go-toolchain.txt\n echo \"latest=${LATEST}\" >> go-toolchain.txt\n echo \"stale=false\" >> go-toolchain.txt\n else\n echo \"current=${CURRENT}\" > go-toolchain.txt\n echo \"latest=${LATEST}\" >> go-toolchain.txt\n echo \"stale=true\" >> go-toolchain.txt\n fi\n\n - name: Upload dependency analysis\n uses: actions/upload-artifact@v4\n with:\n name: dep-analysis\n path: |\n deps-diff.txt\n base-graph-size.txt\n pr-graph-size.txt\n vulncheck-results.txt\n go-toolchain.txt\n\n post-comment:\n if: github.event.pull_request.head.repo.full_name == github.repository\n runs-on: ubuntu-latest\n needs: [build-base, build-pr, analyze-deps]\n steps:\n - name: Download all artifacts\n uses: actions/download-artifact@v4\n\n - name: Post PR comment and manage labels\n uses: actions/github-script@v7\n with:\n script: |\n const fs = require('fs');\n const owner = context.repo.owner;\n const repo = context.repo.repo;\n const issue_number = context.issue.number;\n\n const baseSize = parseInt(fs.readFileSync('base-size/base-size.txt', 'utf8').trim());\n const prSize = parseInt(fs.readFileSync('pr-size/pr-size.txt', 'utf8').trim());\n const buildTime = fs.readFileSync('build-info/build-time.txt', 'utf8').trim();\n const goVersion = fs.readFileSync('build-info/go-version.txt', 'utf8').trim();\n const depsDiff = fs.readFileSync('dep-analysis/deps-diff.txt', 'utf8').trim();\n const baseGraphSize = fs.readFileSync('dep-analysis/base-graph-size.txt', 'utf8').trim();\n const prGraphSize = fs.readFileSync('dep-analysis/pr-graph-size.txt', 'utf8').trim();\n let vulncheck = fs.readFileSync('dep-analysis/vulncheck-results.txt', 'utf8').trim();\n const toolchainData = fs.readFileSync('dep-analysis/go-toolchain.txt', 'utf8').trim();\n\n const MAX_VULNCHECK_LEN = 10000;\n if (vulncheck.length > MAX_VULNCHECK_LEN) {\n vulncheck = vulncheck.substring(0, MAX_VULNCHECK_LEN) + '\\n... (truncated, see workflow run for full output)';\n }\n\n // --- Parse Go toolchain freshness ---\n const tcLines = Object.fromEntries(toolchainData.split('\\n').map(l => l.split('=')));\n const goCurrent = tcLines.current || 'unknown';\n const goLatest = tcLines.latest || 'unknown';\n const goStale = tcLines.stale === 'true';\n\n // --- Compute metrics ---\n const diff = prSize - baseSize;\n const absPct = baseSize > 0 ? Math.abs((diff / baseSize) * 100) : 0;\n const pct = baseSize > 0 ? ((diff / baseSize) * 100).toFixed(2) : 'N/A';\n const sign = diff > 0 ? '+' : '';\n const fmt = (bytes) => (bytes / 1024 / 1024).toFixed(2) + ' MB';\n\n const addedSection = depsDiff.split('## Removed')[0].replace('## Added', '').trim();\n const removedSection = depsDiff.split('## Removed')[1]?.trim() || '';\n const addedDeps = addedSection ? addedSection.split('\\n').filter(l => l.trim()) : [];\n const removedDeps = removedSection ? removedSection.split('\\n').filter(l => l.trim()) : [];\n const depsChanged = addedDeps.length + removedDeps.length;\n\n const graphDiff = parseInt(prGraphSize) - parseInt(baseGraphSize);\n const graphSign = graphDiff > 0 ? '+' : '';\n\n const hasVulns = vulncheck.includes('Vulnerability #') || vulncheck.includes('GO-');\n\n // --- Determine status ---\n const flags = [];\n let sizeIcon = '✅';\n if (diff > 0 && absPct >= 10) { sizeIcon = '🚨'; flags.push('binary size >10%'); }\n else if (diff > 0 && absPct >= 5) { sizeIcon = '⚠️'; flags.push('binary size >5%'); }\n else if (diff < 0) { sizeIcon = '📉'; }\n\n const vulnIcon = hasVulns ? '⚠️' : '✅';\n if (hasVulns) flags.push('vulnerabilities found');\n\n const goIcon = goStale ? '⚠️' : '✅';\n if (goStale) flags.push(`Go toolchain outdated (${goCurrent} → ${goLatest})`);\n\n const depsIcon = depsChanged > 0 ? 'ℹ️' : '✅';\n\n const overallIcon = flags.length > 0 ? '⚠️' : '✅';\n const overallText = flags.length > 0\n ? `**Attention needed** — ${flags.join(', ')}`\n : '**All clear** — no issues detected';\n\n // --- Build compact comment ---\n let depsDetail = 'No dependency changes.';\n if (depsChanged > 0) {\n const parts = [];\n if (addedDeps.length > 0) parts.push('**Added:**\\n' + addedDeps.map(d => `- \\`${d}\\``).join('\\n'));\n if (removedDeps.length > 0) parts.push('**Removed:**\\n' + removedDeps.map(d => `- \\`${d}\\``).join('\\n'));\n depsDetail = parts.join('\\n\\n');\n }\n\n const body = `## PR Build Metrics\n\n ${overallIcon} ${overallText}\n\n | Check | Status | Summary |\n |:------|:------:|:--------|\n | Binary size | ${sizeIcon} | ${fmt(prSize)} (${sign}${(diff / 1024).toFixed(1)} KB / ${sign}${pct}%) |\n | Dependencies | ${depsIcon} | ${depsChanged > 0 ? `${addedDeps.length} added, ${removedDeps.length} removed` : 'No changes'} |\n | Vulnerabilities | ${vulnIcon} | ${hasVulns ? 'Issues found — expand details below' : 'None detected'} |\n | Go toolchain | ${goIcon} | ${goCurrent}${goStale ? ` → ${goLatest} available` : ' (latest)'} |\n | Module graph | ✅ | ${prGraphSize} edges (${graphSign}${graphDiff}) |\n\n ### Binary Size\n\n | | Size | Change |\n |---|---:|---:|\n | Base (\\`${context.payload.pull_request.base.sha.substring(0, 7)}\\`) | ${fmt(baseSize)} | |\n | PR (\\`${context.sha.substring(0, 7)}\\`) | ${fmt(prSize)} | ${sign}${(diff / 1024).toFixed(1)} KB (${sign}${pct}%) |\n\n ### Dependency Changes\n\n ${depsDetail}\n\n ### govulncheck Output\n\n \\`\\`\\`\n ${vulncheck}\n \\`\\`\\`\n\n ### Build Info\n\n | Metric | Value |\n |---|---|\n | Build time | ${buildTime}s |\n | Go version | \\`${goVersion}\\` |\n | Commit | \\`${context.sha.substring(0, 7)}\\` |\n\n \n ---\n 🤖 Updated on each push.`.replace(/^ /gm, '');\n\n // --- Post or update comment (with history) ---\n const marker = '## PR Build Metrics';\n const historyMarker = '';\n let existing = null;\n for await (const response of github.paginate.iterator(\n github.rest.issues.listComments,\n { owner, repo, issue_number, per_page: 100 }\n )) {\n existing = response.data.find(c => c.body?.startsWith(marker));\n if (existing) break;\n }\n\n if (existing) {\n // Extract previous summary line and history from existing comment\n const prevBody = existing.body;\n const prevStatusMatch = prevBody.match(/\\| Binary size \\|[^\\n]+/);\n const prevCommitMatch = prevBody.match(/\\| Commit \\| `([^`]+)` \\|/);\n const prevSummary = prevStatusMatch ? prevStatusMatch[0] : null;\n const prevCommit = prevCommitMatch ? prevCommitMatch[1] : '?';\n\n // Extract existing history entries\n let historyEntries = '';\n const historyIdx = prevBody.indexOf(historyMarker);\n if (historyIdx !== -1) {\n const afterMarker = prevBody.substring(historyIdx + historyMarker.length);\n // Support both old
format and new plain format\n const detailsMatch = afterMarker.match(/
[\\s\\S]*?<\\/details>/);\n if (detailsMatch) {\n const innerMatch = detailsMatch[0].match(/[^<]*<\\/summary>([\\s\\S]*?)<\\/details>/);\n if (innerMatch) historyEntries = innerMatch[1].trim();\n } else {\n // Plain format: extract table rows after the header\n const tableMatch = afterMarker.match(/\\| Commit \\| Updated[\\s\\S]*?(?=\\n---|\\n$|$)/);\n if (tableMatch) historyEntries = tableMatch[0].trim();\n }\n }\n\n // Build new history (most recent first, cap at 10)\n const now = new Date().toISOString().replace('T', ' ').substring(0, 16) + ' UTC';\n const newEntry = prevSummary\n ? `| \\`${prevCommit}\\` | ${now} | ${prevSummary.replace(/\\| Binary size \\|/, '').trim().replace(/^\\||\\|$/g, '').trim()} |`\n : null;\n\n let historyRows = '';\n if (newEntry || historyEntries) {\n const existingRows = historyEntries\n .split('\\n')\n .filter(l => l.startsWith('|') && !l.startsWith('| Commit') && !l.startsWith('|:'))\n .slice(0, 9);\n const allRows = newEntry ? [newEntry, ...existingRows] : existingRows;\n if (allRows.length > 0) {\n historyRows = `\\n### History (${allRows.length} previous)\\n\\n| Commit | Updated | Status | Summary |\\n|:-------|:--------|:------:|:--------|\\n${allRows.join('\\n')}`;\n }\n }\n\n // Insert history into body\n const finalBody = body.replace(historyMarker, historyMarker + historyRows);\n await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body: finalBody });\n } else {\n await github.rest.issues.createComment({ owner, repo, issue_number, body });\n }\n\n // --- Manage labels ---\n const LABELS = {\n SIZE_WARNING: 'metrics: size-warning',\n SIZE_ALERT: 'metrics: size-alert',\n VULNS: 'metrics: vulns-found',\n GO_UPDATE: 'metrics: go-update',\n };\n\n const ensureLabel = async (name, color, description) => {\n try {\n await github.rest.issues.getLabel({ owner, repo, name });\n } catch {\n await github.rest.issues.createLabel({ owner, repo, name, color, description });\n }\n };\n\n const addLabel = async (name) => {\n await github.rest.issues.addLabels({ owner, repo, issue_number, labels: [name] });\n };\n\n const removeLabel = async (name) => {\n try {\n await github.rest.issues.removeLabel({ owner, repo, issue_number, name });\n } catch { /* label not present, ignore */ }\n };\n\n // Size labels\n await ensureLabel(LABELS.SIZE_WARNING, 'fbca04', 'Binary size increased 5-10%');\n await ensureLabel(LABELS.SIZE_ALERT, 'e11d48', 'Binary size increased >10%');\n await ensureLabel(LABELS.VULNS, 'e11d48', 'govulncheck found vulnerabilities');\n await ensureLabel(LABELS.GO_UPDATE, 'fbca04', 'Go toolchain has a newer patch release');\n\n if (diff > 0 && absPct >= 10) {\n await addLabel(LABELS.SIZE_ALERT);\n await removeLabel(LABELS.SIZE_WARNING);\n } else if (diff > 0 && absPct >= 5) {\n await addLabel(LABELS.SIZE_WARNING);\n await removeLabel(LABELS.SIZE_ALERT);\n } else {\n await removeLabel(LABELS.SIZE_WARNING);\n await removeLabel(LABELS.SIZE_ALERT);\n }\n\n // Vuln label\n if (hasVulns) {\n await addLabel(LABELS.VULNS);\n } else {\n await removeLabel(LABELS.VULNS);\n }\n\n // Go toolchain label\n if (goStale) {\n await addLabel(LABELS.GO_UPDATE);\n } else {\n await removeLabel(LABELS.GO_UPDATE);\n }\n" }, { "path": ".github/workflows/pre-release-checklist.yml", "content": "name: Pre-Release Checklist\n\n# Advisory workflow to verify release readiness\n# This workflow reports results but does NOT block releases\n# Run manually before tagging a new release\n\npermissions:\n contents: read\n issues: write\n pull-requests: write\n\non:\n workflow_dispatch:\n inputs:\n version:\n description: 'Version being released (e.g., v0.5.6)'\n required: true\n type: string\n create_issue:\n description: 'Create a GitHub issue with results'\n required: false\n default: true\n type: boolean\n test_cloud_providers:\n description: 'Run cloud provider integration tests (S3, GCS, ABS, R2)'\n required: false\n default: true\n type: boolean\n test_multipart:\n description: 'Run multipart upload stress tests'\n required: false\n default: true\n type: boolean\n\njobs:\n unit-tests:\n name: Unit Tests\n runs-on: ubuntu-latest\n outputs:\n result: ${{ steps.test.outcome }}\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Run unit tests\n id: test\n run: go test -v -race ./...\n\n build-verification:\n name: Build Verification\n runs-on: ubuntu-latest\n outputs:\n result: ${{ steps.build.outcome }}\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Build main binary\n id: build\n run: |\n go build -o bin/litestream ./cmd/litestream\n ./bin/litestream version\n\n - name: Verify binary runs\n run: |\n ./bin/litestream --help\n ./bin/litestream databases --help\n ./bin/litestream replicate --help\n ./bin/litestream restore --help\n\n s3-integration:\n name: S3 Integration (AWS)\n runs-on: ubuntu-latest\n if: github.event.inputs.test_cloud_providers == 'true'\n outputs:\n result: ${{ steps.test.outcome }}\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_S3_ACCESS_KEY_ID }}\" ]; then\n echo \"::notice title=Skipped::S3 integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Run S3 integration tests\n id: test\n if: steps.check-secrets.outputs.has_secrets == 'true'\n continue-on-error: true\n run: go test -v ./replica_client_test.go -integration -replica-clients=s3\n env:\n LITESTREAM_S3_ACCESS_KEY_ID: ${{ secrets.LITESTREAM_S3_ACCESS_KEY_ID }}\n LITESTREAM_S3_SECRET_ACCESS_KEY: ${{ secrets.LITESTREAM_S3_SECRET_ACCESS_KEY }}\n LITESTREAM_S3_REGION: us-east-1\n LITESTREAM_S3_BUCKET: integration.litestream.io\n\n gcs-integration:\n name: GCS Integration\n runs-on: ubuntu-latest\n if: github.event.inputs.test_cloud_providers == 'true'\n outputs:\n result: ${{ steps.test.outcome }}\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}\" ]; then\n echo \"::notice title=Skipped::GCS integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - name: Extract GCP credentials\n if: steps.check-secrets.outputs.has_secrets == 'true'\n run: 'echo \"$GOOGLE_APPLICATION_CREDENTIALS\" > /opt/gcp.json'\n shell: bash\n env:\n GOOGLE_APPLICATION_CREDENTIALS: ${{secrets.GOOGLE_APPLICATION_CREDENTIALS}}\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Run GCS integration tests\n id: test\n if: steps.check-secrets.outputs.has_secrets == 'true'\n continue-on-error: true\n run: go test -v ./replica_client_test.go -integration -replica-clients=gs\n env:\n GOOGLE_APPLICATION_CREDENTIALS: /opt/gcp.json\n LITESTREAM_GS_BUCKET: litestream-github-workflows\n\n abs-integration:\n name: Azure Blob Storage Integration\n runs-on: ubuntu-latest\n if: github.event.inputs.test_cloud_providers == 'true'\n outputs:\n result: ${{ steps.test.outcome }}\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_ABS_ACCOUNT_NAME }}\" ]; then\n echo \"::notice title=Skipped::Azure Blob Storage integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Run ABS integration tests\n id: test\n if: steps.check-secrets.outputs.has_secrets == 'true'\n continue-on-error: true\n run: go test -v ./replica_client_test.go -integration -replica-clients=abs\n env:\n LITESTREAM_ABS_ACCOUNT_NAME: ${{ secrets.LITESTREAM_ABS_ACCOUNT_NAME }}\n LITESTREAM_ABS_ACCOUNT_KEY: ${{ secrets.LITESTREAM_ABS_ACCOUNT_KEY }}\n LITESTREAM_ABS_BUCKET: integration\n\n r2-integration:\n name: Cloudflare R2 Integration\n runs-on: ubuntu-latest\n if: github.event.inputs.test_cloud_providers == 'true'\n outputs:\n result: ${{ steps.test.outcome }}\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_R2_ACCESS_KEY_ID }}\" ]; then\n echo \"::notice title=Skipped::R2 integration tests skipped - credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Run R2 integration tests\n id: test\n if: steps.check-secrets.outputs.has_secrets == 'true'\n continue-on-error: true\n run: go test -v ./replica_client_test.go -integration -replica-clients=r2\n env:\n LITESTREAM_R2_ACCESS_KEY_ID: ${{ secrets.LITESTREAM_R2_ACCESS_KEY_ID }}\n LITESTREAM_R2_SECRET_ACCESS_KEY: ${{ secrets.LITESTREAM_R2_SECRET_ACCESS_KEY }}\n LITESTREAM_R2_ENDPOINT: ${{ secrets.LITESTREAM_R2_ENDPOINT }}\n LITESTREAM_R2_BUCKET: ${{ secrets.LITESTREAM_R2_BUCKET }}\n\n multipart-stress:\n name: Multipart Upload Stress Tests\n runs-on: ubuntu-latest\n if: github.event.inputs.test_multipart == 'true'\n outputs:\n result: ${{ steps.test.outcome }}\n steps:\n - name: Check for required secrets\n id: check-secrets\n run: |\n if [ -z \"${{ secrets.LITESTREAM_S3_ACCESS_KEY_ID }}\" ]; then\n echo \"::notice title=Skipped::Multipart stress tests skipped - S3 credentials not configured\"\n echo \"has_secrets=false\" >> $GITHUB_OUTPUT\n else\n echo \"has_secrets=true\" >> $GITHUB_OUTPUT\n fi\n\n - uses: actions/checkout@v4\n if: steps.check-secrets.outputs.has_secrets == 'true'\n\n - uses: actions/setup-go@v5\n if: steps.check-secrets.outputs.has_secrets == 'true'\n with:\n go-version-file: \"go.mod\"\n\n - name: Run multipart stress tests\n id: test\n if: steps.check-secrets.outputs.has_secrets == 'true'\n continue-on-error: true\n run: |\n go test -v ./replica_client_test.go -integration -replica-clients=s3 \\\n -run \"TestReplicaClient_S3_Multipart|TestReplicaClient_S3_Concurrency\" \\\n -timeout 30m\n env:\n LITESTREAM_S3_ACCESS_KEY_ID: ${{ secrets.LITESTREAM_S3_ACCESS_KEY_ID }}\n LITESTREAM_S3_SECRET_ACCESS_KEY: ${{ secrets.LITESTREAM_S3_SECRET_ACCESS_KEY }}\n LITESTREAM_S3_REGION: us-east-1\n LITESTREAM_S3_BUCKET: integration.litestream.io\n\n generate-checklist:\n name: Generate Release Checklist\n runs-on: ubuntu-latest\n needs:\n - unit-tests\n - build-verification\n - s3-integration\n - gcs-integration\n - abs-integration\n - r2-integration\n - multipart-stress\n if: always()\n steps:\n - name: Generate checklist summary\n id: checklist\n run: |\n echo \"## Pre-Release Checklist for ${{ github.event.inputs.version }}\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"**Date:** $(date -u '+%Y-%m-%d %H:%M:%S UTC')\" >> $GITHUB_STEP_SUMMARY\n echo \"**Triggered by:** @${{ github.actor }}\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n\n echo \"### Core Tests\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n\n if [ \"${{ needs.unit-tests.result }}\" == \"success\" ]; then\n echo \"- [x] Unit tests passed\" >> $GITHUB_STEP_SUMMARY\n else\n echo \"- [ ] Unit tests **FAILED**\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.build-verification.result }}\" == \"success\" ]; then\n echo \"- [x] Build verification passed\" >> $GITHUB_STEP_SUMMARY\n else\n echo \"- [ ] Build verification **FAILED**\" >> $GITHUB_STEP_SUMMARY\n fi\n\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"### Cloud Provider Integration Tests\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n\n if [ \"${{ needs.s3-integration.result }}\" == \"success\" ]; then\n echo \"- [x] AWS S3 integration passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.s3-integration.result }}\" == \"skipped\" ]; then\n echo \"- [ ] AWS S3 integration (skipped)\" >> $GITHUB_STEP_SUMMARY\n else\n echo \"- [ ] AWS S3 integration **FAILED**\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.gcs-integration.result }}\" == \"success\" ]; then\n echo \"- [x] Google Cloud Storage integration passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.gcs-integration.result }}\" == \"skipped\" ]; then\n echo \"- [ ] Google Cloud Storage integration (skipped)\" >> $GITHUB_STEP_SUMMARY\n else\n echo \"- [ ] Google Cloud Storage integration **FAILED**\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.abs-integration.result }}\" == \"success\" ]; then\n echo \"- [x] Azure Blob Storage integration passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.abs-integration.result }}\" == \"skipped\" ]; then\n echo \"- [ ] Azure Blob Storage integration (skipped)\" >> $GITHUB_STEP_SUMMARY\n else\n echo \"- [ ] Azure Blob Storage integration **FAILED**\" >> $GITHUB_STEP_SUMMARY\n fi\n\n if [ \"${{ needs.r2-integration.result }}\" == \"success\" ]; then\n echo \"- [x] Cloudflare R2 integration passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.r2-integration.result }}\" == \"skipped\" ]; then\n echo \"- [ ] Cloudflare R2 integration (skipped)\" >> $GITHUB_STEP_SUMMARY\n else\n echo \"- [ ] Cloudflare R2 integration **FAILED**\" >> $GITHUB_STEP_SUMMARY\n fi\n\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"### Stress Tests\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n\n if [ \"${{ needs.multipart-stress.result }}\" == \"success\" ]; then\n echo \"- [x] Multipart upload stress tests passed\" >> $GITHUB_STEP_SUMMARY\n elif [ \"${{ needs.multipart-stress.result }}\" == \"skipped\" ]; then\n echo \"- [ ] Multipart upload stress tests (skipped)\" >> $GITHUB_STEP_SUMMARY\n else\n echo \"- [ ] Multipart upload stress tests **FAILED**\" >> $GITHUB_STEP_SUMMARY\n fi\n\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"---\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"**Note:** This checklist is advisory only.\" >> $GITHUB_STEP_SUMMARY\n echo \"Review failed items before proceeding with the release.\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"[View full run details](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})\" >> $GITHUB_STEP_SUMMARY\n\n # Create issue body for later use\n cat > /tmp/issue_body.md << 'ISSUE_EOF'\n ## Pre-Release Checklist for ${{ github.event.inputs.version }}\n\n **Date:** $(date -u '+%Y-%m-%d %H:%M:%S UTC')\n **Triggered by:** @${{ github.actor }}\n **Workflow Run:** https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}\n\n ### Core Tests\n\n | Test | Status |\n |------|--------|\n | Unit Tests | ${{ needs.unit-tests.result == 'success' && '✅ Passed' || '❌ Failed' }} |\n | Build Verification | ${{ needs.build-verification.result == 'success' && '✅ Passed' || '❌ Failed' }} |\n\n ### Cloud Provider Integration Tests\n\n | Provider | Status |\n |----------|--------|\n | AWS S3 | ${{ needs.s3-integration.result == 'success' && '✅ Passed' || needs.s3-integration.result == 'skipped' && '⏭️ Skipped' || '❌ Failed' }} |\n | Google Cloud Storage | ${{ needs.gcs-integration.result == 'success' && '✅ Passed' || needs.gcs-integration.result == 'skipped' && '⏭️ Skipped' || '❌ Failed' }} |\n | Azure Blob Storage | ${{ needs.abs-integration.result == 'success' && '✅ Passed' || needs.abs-integration.result == 'skipped' && '⏭️ Skipped' || '❌ Failed' }} |\n | Cloudflare R2 | ${{ needs.r2-integration.result == 'success' && '✅ Passed' || needs.r2-integration.result == 'skipped' && '⏭️ Skipped' || '❌ Failed' }} |\n\n ### Stress Tests\n\n | Test | Status |\n |------|--------|\n | Multipart Uploads | ${{ needs.multipart-stress.result == 'success' && '✅ Passed' || needs.multipart-stress.result == 'skipped' && '⏭️ Skipped' || '❌ Failed' }} |\n\n ---\n\n **Note:** This checklist is advisory only. Review any failed items before proceeding with the release.\n ISSUE_EOF\n\n - name: Create GitHub Issue\n if: github.event.inputs.create_issue == 'true'\n continue-on-error: true\n uses: actions/github-script@v7\n with:\n script: |\n const version = '${{ github.event.inputs.version }}';\n const runUrl = `https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}`;\n\n // Determine overall status\n const results = {\n unit: '${{ needs.unit-tests.result }}',\n build: '${{ needs.build-verification.result }}',\n s3: '${{ needs.s3-integration.result }}',\n gcs: '${{ needs.gcs-integration.result }}',\n abs: '${{ needs.abs-integration.result }}',\n r2: '${{ needs.r2-integration.result }}',\n multipart: '${{ needs.multipart-stress.result }}'\n };\n\n const failed = Object.entries(results)\n .filter(([_, v]) => v === 'failure')\n .map(([k, _]) => k);\n\n const status = failed.length === 0 ? '✅' : '⚠️';\n const title = `${status} Pre-Release Checklist: ${version}`;\n\n let body = `## Pre-Release Checklist for ${version}\\n\\n`;\n body += `**Workflow Run:** [View Details](${runUrl})\\n\\n`;\n\n body += `### Results Summary\\n\\n`;\n body += `| Test | Status |\\n`;\n body += `|------|--------|\\n`;\n body += `| Unit Tests | ${results.unit === 'success' ? '✅' : '❌'} |\\n`;\n body += `| Build | ${results.build === 'success' ? '✅' : '❌'} |\\n`;\n body += `| AWS S3 | ${results.s3 === 'success' ? '✅' : results.s3 === 'skipped' ? '⏭️' : '❌'} |\\n`;\n body += `| GCS | ${results.gcs === 'success' ? '✅' : results.gcs === 'skipped' ? '⏭️' : '❌'} |\\n`;\n body += `| Azure | ${results.abs === 'success' ? '✅' : results.abs === 'skipped' ? '⏭️' : '❌'} |\\n`;\n body += `| R2 | ${results.r2 === 'success' ? '✅' : results.r2 === 'skipped' ? '⏭️' : '❌'} |\\n`;\n body += `| Multipart | ${results.multipart === 'success' ? '✅' : results.multipart === 'skipped' ? '⏭️' : '❌'} |\\n\\n`;\n\n if (failed.length > 0) {\n body += `### ⚠️ Failed Tests\\n\\n`;\n body += `The following tests failed and should be reviewed:\\n`;\n failed.forEach(f => body += `- ${f}\\n`);\n body += `\\n`;\n }\n\n body += `---\\n`;\n body += `*This issue was automatically created by the pre-release checklist workflow.*\\n`;\n\n try {\n await github.rest.issues.create({\n owner: context.repo.owner,\n repo: context.repo.repo,\n title: title,\n body: body,\n labels: ['release-checklist']\n });\n console.log('Created release checklist issue');\n } catch (error) {\n console.log(`Failed to create issue: ${error.message}`);\n }\n" }, { "path": ".github/workflows/release.docker.yml", "content": "on:\n release:\n types:\n - published\n# pull_request:\n# types:\n# - opened\n# - synchronize\n# - reopened\n# branches-ignore:\n# - \"dependabot/**\"\n\nname: Release (Docker)\njobs:\n docker:\n runs-on: ubuntu-latest\n env:\n # CGO cross-compilation supported platforms\n PLATFORMS: \"linux/amd64,linux/arm64\"\n VERSION: \"${{ github.event_name == 'release' && github.event.release.tag_name || github.sha }}\"\n\n steps:\n - uses: actions/checkout@v4\n - uses: docker/setup-qemu-action@v3\n - uses: docker/setup-buildx-action@v3\n - uses: docker/login-action@v3\n with:\n username: benbjohnson\n password: ${{ secrets.DOCKERHUB_TOKEN }}\n\n # Default (Debian) image\n - id: meta-default\n uses: docker/metadata-action@v5\n with:\n images: litestream/litestream\n tags: |\n type=ref,event=branch\n type=ref,event=pr\n type=sha\n type=sha,format=long\n type=semver,pattern={{version}}\n type=semver,pattern={{major}}.{{minor}}\n\n - uses: docker/build-push-action@v6\n with:\n context: .\n target: default\n push: true\n provenance: true\n sbom: true\n platforms: ${{ env.PLATFORMS }}\n tags: ${{ steps.meta-default.outputs.tags }}\n labels: ${{ steps.meta-default.outputs.labels }}\n build-args: |\n LITESTREAM_VERSION=${{ env.VERSION }}\n\n # Hardened (Scratch) image\n - id: meta-scratch\n uses: docker/metadata-action@v5\n with:\n images: litestream/litestream\n flavor: |\n latest=auto\n suffix=-scratch\n tags: |\n type=ref,event=branch\n type=ref,event=pr\n type=sha\n type=sha,format=long\n type=semver,pattern={{version}}\n type=semver,pattern={{major}}.{{minor}}\n\n - uses: docker/build-push-action@v6\n with:\n context: .\n target: hardened\n push: true\n provenance: true\n sbom: true\n platforms: ${{ env.PLATFORMS }}\n tags: ${{ steps.meta-scratch.outputs.tags }}\n labels: ${{ steps.meta-scratch.outputs.labels }}\n build-args: |\n LITESTREAM_VERSION=${{ env.VERSION }}\n" }, { "path": ".github/workflows/release.yml", "content": "name: Release\n\non:\n push:\n tags:\n - 'v*'\n workflow_dispatch:\n inputs:\n tag:\n description: 'Release tag (e.g., v0.3.14)'\n required: true\n type: string\n\npermissions:\n contents: write\n packages: write\n id-token: write\n\njobs:\n goreleaser:\n runs-on: ubuntu-latest\n steps:\n - name: Checkout\n uses: actions/checkout@v4\n with:\n fetch-depth: 0\n\n - name: Set up Go\n uses: actions/setup-go@v5\n with:\n go-version-file: go.mod\n cache: true\n\n - name: Install cross-compilers\n run: |\n sudo apt-get update\n sudo apt-get install -y gcc-aarch64-linux-gnu gcc-arm-linux-gnueabihf gcc-arm-linux-gnueabi\n\n - name: Set up QEMU\n uses: docker/setup-qemu-action@v3\n\n - name: Install Syft for SBOM generation\n uses: anchore/sbom-action/download-syft@v0\n with:\n syft-version: latest\n\n - name: Import GPG key\n if: ${{ env.GPG_PRIVATE_KEY != '' }}\n id: import_gpg\n uses: crazy-max/ghaction-import-gpg@v6\n with:\n gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}\n passphrase: ${{ secrets.GPG_PASSPHRASE }}\n\n - name: Run GoReleaser\n uses: goreleaser/goreleaser-action@v6\n with:\n version: latest\n args: release --clean\n env:\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n GPG_FINGERPRINT: ${{ steps.import_gpg.outputs.fingerprint }}\n HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }}\n\n vfs-build-linux:\n name: Build VFS (Linux ${{ matrix.arch }})\n runs-on: ubuntu-22.04 # glibc 2.35 (ubuntu-20.04 runners deprecated)\n needs: goreleaser\n strategy:\n matrix:\n arch: [amd64, arm64]\n steps:\n - name: Checkout\n uses: actions/checkout@v4\n with:\n fetch-depth: 0\n\n - name: Set up Go\n uses: actions/setup-go@v5\n with:\n go-version-file: go.mod\n cache: true\n\n - name: Install cross-compiler (arm64)\n if: matrix.arch == 'arm64'\n run: |\n sudo apt-get update\n sudo apt-get install -y gcc-aarch64-linux-gnu\n\n - name: Get version\n id: version\n run: |\n if [ \"${{ github.event_name }}\" == \"workflow_dispatch\" ]; then\n echo \"version=${{ inputs.tag }}\" >> $GITHUB_OUTPUT\n else\n echo \"version=${GITHUB_REF#refs/tags/}\" >> $GITHUB_OUTPUT\n fi\n\n - name: Build VFS extension\n run: make vfs-linux-${{ matrix.arch }}\n\n - name: Verify artifact\n run: |\n file dist/litestream-vfs-linux-${{ matrix.arch }}.so\n # For native arch, verify it can be loaded by SQLite\n if [ \"${{ matrix.arch }}\" == \"amd64\" ]; then\n sqlite3 ':memory:' \".load dist/litestream-vfs-linux-amd64.so\" \".exit\" && echo \"Extension loads successfully\"\n fi\n # For cross-compiled arch, verify ELF header and architecture\n if [ \"${{ matrix.arch }}\" == \"arm64\" ]; then\n readelf -h dist/litestream-vfs-linux-arm64.so | grep -E \"(Class|Machine)\"\n echo \"ARM64 ELF header verified\"\n fi\n\n - name: Create archive\n run: |\n cd dist\n cp litestream-vfs-linux-${{ matrix.arch }}.so litestream.so\n tar -czvf litestream-vfs-${{ steps.version.outputs.version }}-linux-${{ matrix.arch }}.tar.gz \\\n litestream.so\n\n - name: Generate checksum\n run: |\n cd dist\n sha256sum litestream-vfs-${{ steps.version.outputs.version }}-linux-${{ matrix.arch }}.tar.gz \\\n > litestream-vfs-${{ steps.version.outputs.version }}-linux-${{ matrix.arch }}.tar.gz.sha256\n\n - name: Upload to release\n env:\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n run: |\n gh release upload ${{ steps.version.outputs.version }} \\\n dist/litestream-vfs-${{ steps.version.outputs.version }}-linux-${{ matrix.arch }}.tar.gz \\\n dist/litestream-vfs-${{ steps.version.outputs.version }}-linux-${{ matrix.arch }}.tar.gz.sha256 \\\n --clobber\n\n - name: Upload artifact for packaging\n uses: actions/upload-artifact@v4\n with:\n name: vfs-linux-${{ matrix.arch }}\n path: dist/litestream-vfs-linux-${{ matrix.arch }}.so\n\n vfs-build-darwin:\n name: Build VFS (macOS ${{ matrix.arch }})\n runs-on: ${{ matrix.runner }}\n needs: goreleaser\n strategy:\n matrix:\n include:\n - arch: amd64\n runner: macos-15\n - arch: arm64\n runner: macos-15\n steps:\n - name: Checkout\n uses: actions/checkout@v4\n with:\n fetch-depth: 0\n\n - name: Set up Go\n uses: actions/setup-go@v5\n with:\n go-version-file: go.mod\n cache: true\n\n - name: Get version\n id: version\n run: |\n if [ \"${{ github.event_name }}\" == \"workflow_dispatch\" ]; then\n echo \"version=${{ inputs.tag }}\" >> $GITHUB_OUTPUT\n else\n echo \"version=${GITHUB_REF#refs/tags/}\" >> $GITHUB_OUTPUT\n fi\n\n - name: Build VFS extension\n run: make vfs-darwin-${{ matrix.arch }}\n\n - name: Verify artifact\n run: |\n file dist/litestream-vfs-darwin-${{ matrix.arch }}.dylib\n # Verify architecture matches target\n lipo -info dist/litestream-vfs-darwin-${{ matrix.arch }}.dylib\n # Verify the expected symbol exists (macOS sqlite3 doesn't support .load)\n nm -gU dist/litestream-vfs-darwin-${{ matrix.arch }}.dylib | grep sqlite3_litestreamvfs_init && echo \"Entry point symbol found\"\n\n - name: Create archive\n run: |\n cd dist\n cp litestream-vfs-darwin-${{ matrix.arch }}.dylib litestream.dylib\n tar -czvf litestream-vfs-${{ steps.version.outputs.version }}-darwin-${{ matrix.arch }}.tar.gz \\\n litestream.dylib\n\n - name: Generate checksum\n run: |\n cd dist\n shasum -a 256 litestream-vfs-${{ steps.version.outputs.version }}-darwin-${{ matrix.arch }}.tar.gz \\\n > litestream-vfs-${{ steps.version.outputs.version }}-darwin-${{ matrix.arch }}.tar.gz.sha256\n\n - name: Upload to release\n env:\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n run: |\n gh release upload ${{ steps.version.outputs.version }} \\\n dist/litestream-vfs-${{ steps.version.outputs.version }}-darwin-${{ matrix.arch }}.tar.gz \\\n dist/litestream-vfs-${{ steps.version.outputs.version }}-darwin-${{ matrix.arch }}.tar.gz.sha256 \\\n --clobber\n\n - name: Upload artifact for packaging\n uses: actions/upload-artifact@v4\n with:\n name: vfs-darwin-${{ matrix.arch }}\n path: dist/litestream-vfs-darwin-${{ matrix.arch }}.dylib\n\n # macos-sign:\n # runs-on: macos-latest\n # needs: goreleaser\n # strategy:\n # matrix:\n # arch: [amd64, arm64]\n # steps:\n # - name: Checkout\n # uses: actions/checkout@v4\n\n # - name: Set up Go\n # uses: actions/setup-go@v5\n # with:\n # go-version-file: go.mod\n\n # - name: Download release artifacts\n # uses: actions/download-artifact@v4\n # with:\n # name: litestream-darwin-${{ matrix.arch }}\n # path: dist/\n\n # - name: Import Apple Developer Certificate\n # env:\n # MACOS_CERTIFICATE: ${{ secrets.MACOS_CERTIFICATE_P12 }}\n # MACOS_CERTIFICATE_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PASSWORD }}\n # run: |\n # echo \"$MACOS_CERTIFICATE\" | base64 --decode > certificate.p12\n # security create-keychain -p actions temp.keychain\n # security default-keychain -s temp.keychain\n # security unlock-keychain -p actions temp.keychain\n # security import certificate.p12 -k temp.keychain -P \"$MACOS_CERTIFICATE_PASSWORD\" -T /usr/bin/codesign\n # security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k actions temp.keychain\n\n # - name: Sign and Notarize\n # env:\n # APPLE_API_KEY: ${{ secrets.APPLE_API_KEY_P8 }}\n # APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }}\n # APPLE_API_ISSUER_ID: ${{ secrets.APPLE_API_ISSUER_ID }}\n # AC_PASSWORD: ${{ secrets.AC_PASSWORD }}\n # run: |\n # gon etc/gon-${{ matrix.arch }}.hcl\n\n # - name: Upload signed binary\n # env:\n # GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n # run: |\n # gh release upload ${{ github.ref_name }} dist/litestream-*-darwin-${{ matrix.arch }}.zip\n\n # windows-sign:\n # runs-on: windows-latest\n # needs: goreleaser\n # strategy:\n # matrix:\n # arch: [amd64, arm64]\n # steps:\n # - name: Checkout\n # uses: actions/checkout@v4\n #\n # - name: Download release artifacts\n # uses: actions/download-artifact@v4\n # with:\n # name: litestream-windows-${{ matrix.arch }}\n # path: dist/\n #\n # - name: Sign Windows binary\n # env:\n # WINDOWS_CERTIFICATE_PFX: ${{ secrets.WINDOWS_CERTIFICATE_PFX }}\n # WINDOWS_CERTIFICATE_PASSWORD: ${{ secrets.WINDOWS_CERTIFICATE_PASSWORD }}\n # run: |\n # echo \"$env:WINDOWS_CERTIFICATE_PFX\" | base64 -d > cert.pfx\n # & signtool sign /f cert.pfx /p \"$env:WINDOWS_CERTIFICATE_PASSWORD\" /fd SHA256 /td SHA256 /tr http://timestamp.digicert.com dist\\litestream.exe\n # Remove-Item cert.pfx\n #\n # - name: Upload signed binary\n # env:\n # GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n # run: |\n # gh release upload ${{ github.ref_name }} dist\\litestream-*-windows-${{ matrix.arch }}.zip\n\n publish-pypi:\n name: Publish to PyPI (${{ matrix.os }}-${{ matrix.arch }})\n runs-on: ubuntu-latest\n needs: [vfs-build-linux, vfs-build-darwin]\n environment: release\n permissions:\n id-token: write\n strategy:\n matrix:\n include:\n - os: linux\n arch: amd64\n ext: so\n src_name: litestream-vfs-linux-amd64.so\n dest_name: litestream-vfs.so\n platform_tag: manylinux_2_35_x86_64\n - os: linux\n arch: arm64\n ext: so\n src_name: litestream-vfs-linux-arm64.so\n dest_name: litestream-vfs.so\n platform_tag: manylinux_2_35_aarch64\n - os: darwin\n arch: amd64\n ext: dylib\n src_name: litestream-vfs-darwin-amd64.dylib\n dest_name: litestream-vfs.dylib\n platform_tag: macosx_11_0_x86_64\n - os: darwin\n arch: arm64\n ext: dylib\n src_name: litestream-vfs-darwin-arm64.dylib\n dest_name: litestream-vfs.dylib\n platform_tag: macosx_11_0_arm64\n steps:\n - name: Checkout\n uses: actions/checkout@v4\n\n - name: Set up Python\n uses: actions/setup-python@v5\n with:\n python-version: \"3.12\"\n\n - name: Download VFS artifact\n uses: actions/download-artifact@v4\n with:\n name: vfs-${{ matrix.os }}-${{ matrix.arch }}\n path: packages/python/litestream_vfs/\n\n - name: Rename binary\n run: |\n mv packages/python/litestream_vfs/${{ matrix.src_name }} \\\n packages/python/litestream_vfs/${{ matrix.dest_name }}\n\n - name: Get version\n id: version\n run: |\n if [ \"${{ github.event_name }}\" == \"workflow_dispatch\" ]; then\n VERSION=\"${{ inputs.tag }}\"\n else\n VERSION=\"${GITHUB_REF#refs/tags/}\"\n fi\n echo \"version=${VERSION#v}\" >> $GITHUB_OUTPUT\n\n - name: Build wheel\n run: |\n cd packages/python\n pip install setuptools wheel\n LITESTREAM_VERSION=${{ steps.version.outputs.version }} python setup.py bdist_wheel\n\n - name: Rename wheel with platform tag\n run: |\n cd packages/python\n python scripts/rename_wheel.py dist ${{ matrix.platform_tag }}\n\n - name: Publish to PyPI\n uses: pypa/gh-action-pypi-publish@release/v1\n with:\n packages-dir: packages/python/dist/\n\n publish-npm:\n name: Publish to npm\n runs-on: ubuntu-latest\n needs: [vfs-build-linux, vfs-build-darwin]\n steps:\n - name: Checkout\n uses: actions/checkout@v4\n\n - name: Set up Node.js\n uses: actions/setup-node@v4\n with:\n node-version: \"20\"\n registry-url: \"https://registry.npmjs.org\"\n\n - name: Download all VFS artifacts\n uses: actions/download-artifact@v4\n with:\n path: artifacts/\n\n - name: Copy binaries to platform packages\n run: |\n cp artifacts/vfs-linux-amd64/litestream-vfs-linux-amd64.so \\\n packages/npm/litestream-vfs-linux-amd64/litestream-vfs.so\n cp artifacts/vfs-linux-arm64/litestream-vfs-linux-arm64.so \\\n packages/npm/litestream-vfs-linux-arm64/litestream-vfs.so\n cp artifacts/vfs-darwin-amd64/litestream-vfs-darwin-amd64.dylib \\\n packages/npm/litestream-vfs-darwin-amd64/litestream-vfs.dylib\n cp artifacts/vfs-darwin-arm64/litestream-vfs-darwin-arm64.dylib \\\n packages/npm/litestream-vfs-darwin-arm64/litestream-vfs.dylib\n\n - name: Get version\n id: version\n run: |\n if [ \"${{ github.event_name }}\" == \"workflow_dispatch\" ]; then\n VERSION=\"${{ inputs.tag }}\"\n else\n VERSION=\"${GITHUB_REF#refs/tags/}\"\n fi\n echo \"version=${VERSION#v}\" >> $GITHUB_OUTPUT\n\n - name: Set versions\n run: |\n VERSION=${{ steps.version.outputs.version }}\n for dir in packages/npm/litestream-vfs-*/; do\n jq --arg v \"$VERSION\" '.version = $v' \"$dir/package.json\" > tmp.json && mv tmp.json \"$dir/package.json\"\n done\n jq --arg v \"$VERSION\" '\n .version = $v |\n .optionalDependencies = (.optionalDependencies | to_entries | map(.value = $v) | from_entries)\n ' packages/npm/litestream-vfs/package.json > tmp.json && mv tmp.json packages/npm/litestream-vfs/package.json\n\n - name: Publish platform packages\n env:\n NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}\n run: |\n for dir in packages/npm/litestream-vfs-*/; do\n cd \"$dir\"\n npm publish --access public\n cd -\n done\n\n - name: Wait for registry propagation\n run: sleep 30\n\n - name: Publish main package\n env:\n NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}\n run: |\n cd packages/npm/litestream-vfs\n npm publish --access public\n\n publish-gem:\n name: Publish to RubyGems (${{ matrix.os }}-${{ matrix.arch }})\n runs-on: ubuntu-latest\n needs: [vfs-build-linux, vfs-build-darwin]\n strategy:\n matrix:\n include:\n - os: linux\n arch: amd64\n ext: so\n src_name: litestream-vfs-linux-amd64.so\n dest_name: litestream-vfs.so\n platform: x86_64-linux\n - os: linux\n arch: arm64\n ext: so\n src_name: litestream-vfs-linux-arm64.so\n dest_name: litestream-vfs.so\n platform: aarch64-linux\n - os: darwin\n arch: amd64\n ext: dylib\n src_name: litestream-vfs-darwin-amd64.dylib\n dest_name: litestream-vfs.dylib\n platform: x86_64-darwin\n - os: darwin\n arch: arm64\n ext: dylib\n src_name: litestream-vfs-darwin-arm64.dylib\n dest_name: litestream-vfs.dylib\n platform: arm64-darwin\n steps:\n - name: Checkout\n uses: actions/checkout@v4\n\n - name: Set up Ruby\n uses: ruby/setup-ruby@v1\n with:\n ruby-version: \"3.3\"\n\n - name: Download VFS artifact\n uses: actions/download-artifact@v4\n with:\n name: vfs-${{ matrix.os }}-${{ matrix.arch }}\n path: packages/ruby/lib/\n\n - name: Rename binary\n run: |\n mv packages/ruby/lib/${{ matrix.src_name }} \\\n packages/ruby/lib/${{ matrix.dest_name }}\n\n - name: Get version\n id: version\n run: |\n if [ \"${{ github.event_name }}\" == \"workflow_dispatch\" ]; then\n VERSION=\"${{ inputs.tag }}\"\n else\n VERSION=\"${GITHUB_REF#refs/tags/}\"\n fi\n echo \"version=${VERSION#v}\" >> $GITHUB_OUTPUT\n\n - name: Build gem\n run: |\n cd packages/ruby\n LITESTREAM_VERSION=${{ steps.version.outputs.version }} \\\n PLATFORM=${{ matrix.platform }} \\\n gem build litestream-vfs.gemspec\n\n - name: Publish to RubyGems\n env:\n GEM_HOST_API_KEY: ${{ secrets.RUBYGEMS_API_KEY }}\n run: |\n cd packages/ruby\n gem push litestream-vfs-*.gem\n" }, { "path": ".github/workflows/stale-issues.yml", "content": "name: Stale Issue Manager\n\non:\n schedule:\n - cron: '0 0 * * *'\n workflow_dispatch:\n\npermissions:\n issues: write\n\njobs:\n stale:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/stale@v9\n with:\n repo-token: ${{ secrets.GITHUB_TOKEN }}\n\n # Issues: Label after 90 days, close after 30 more days\n days-before-stale: 90\n days-before-close: 30\n stale-issue-label: 'stale'\n\n # Exempt these labels from stale marking\n exempt-issue-labels: 'Soon,release-blocker'\n\n # Message posted when issue becomes stale\n stale-issue-message: >\n This issue has been inactive for 90 days and will be automatically\n closed in 30 days if there is no further activity. If this issue is\n still relevant, please add a comment to keep it open. Thank you for\n your contribution!\n\n # Message posted when issue is closed\n close-issue-message: >\n This issue has been automatically closed due to inactivity. If you\n believe this issue is still relevant, please reopen it or create a\n new issue with updated information. Thank you!\n\n # Disable pull request processing\n days-before-pr-stale: -1\n days-before-pr-close: -1\n\n # Limit operations per run\n operations-per-run: 100\n\n # Remove stale label when there is activity\n remove-stale-when-updated: true\n" }, { "path": ".github/workflows/upgrade-tests.yml", "content": "name: Upgrade Tests\n\non:\n pull_request:\n paths:\n - '**.go'\n - 'go.mod'\n - 'go.sum'\n - 'tests/integration/**'\n - '.github/workflows/upgrade-tests.yml'\n workflow_dispatch:\n\npermissions:\n contents: read\n\njobs:\n upgrade-test:\n name: v0.3.x to v0.5.x Upgrade Test\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n\n - uses: actions/setup-go@v5\n with:\n go-version-file: \"go.mod\"\n\n - name: Download Litestream v0.3.13\n run: |\n mkdir -p /tmp/litestream-v3\n gh release download v0.3.13 --repo benbjohnson/litestream --pattern 'litestream-v0.3.13-linux-amd64.tar.gz' --dir /tmp\n tar -xzf /tmp/litestream-v0.3.13-linux-amd64.tar.gz -C /tmp/litestream-v3\n chmod +x /tmp/litestream-v3/litestream\n /tmp/litestream-v3/litestream version\n env:\n GH_TOKEN: ${{ github.token }}\n\n - name: Build current binaries\n run: go build -o bin/litestream ./cmd/litestream && go build -o bin/litestream-test ./cmd/litestream-test\n\n - name: Run upgrade integration tests\n run: go test -v -tags=integration -timeout=10m ./tests/integration/... -run=TestUpgrade\n env:\n CGO_ENABLED: \"1\"\n LITESTREAM_V3_BIN: /tmp/litestream-v3/litestream\n\n - name: Upload test logs\n if: failure()\n uses: actions/upload-artifact@v4\n with:\n name: upgrade-test-logs\n path: /tmp/litestream-*/*.log\n" }, { "path": ".gitignore", "content": ".DS_Store\n/src/litestream-vfs.h\n/dist\n.vscode\n.sprite\n\n# Claude-related files (force include despite global gitignore)\n!.claude/\n!.claude/**\n# But ignore logs, hooks, and local settings\n.claude/logs/\n.claude/hooks/\n.claude/settings.local.json\n# Keep CLAUDE.md ignored as it's auto-loaded by Claude Code\nCLAUDE.md\n\n# Binary\nbin/\n\n# Cached binaries for integration tests\n.cache/\n.claude/*.loop.local.md\n.claude/worktrees/\nCLAUDE.local.md\n\n# Package manager build artifacts\npackages/python/dist/\npackages/python/build/\npackages/python/*.egg-info/\n" }, { "path": ".goreleaser.yml", "content": "version: 2\n\nproject_name: litestream\n\nbefore:\n hooks:\n - go mod tidy\n\nbuilds:\n - id: litestream\n main: ./cmd/litestream\n binary: litestream\n env:\n - CGO_ENABLED=0\n goos:\n - linux\n - darwin\n - windows\n goarch:\n - amd64\n - arm64\n - arm\n goarm:\n - \"6\"\n - \"7\"\n ldflags:\n - -s -w -X main.Version={{.Version}}\n ignore:\n - goos: windows\n goarch: arm\n - goos: darwin\n goarch: arm\n\narchives:\n - id: main\n formats:\n - tar.gz\n format_overrides:\n - goos: windows\n formats:\n - zip\n name_template: >-\n {{ .ProjectName }}-\n {{- .Version }}-\n {{- .Os }}-\n {{- if eq .Arch \"amd64\" }}x86_64\n {{- else if eq .Arch \"386\" }}i386\n {{- else }}{{ .Arch }}{{ end }}\n {{- if .Arm }}v{{ .Arm }}{{ end }}\n files:\n - etc/litestream.yml\n - etc/litestream.service\n - README.md\n - LICENSE\n\nnfpms:\n - vendor: Litestream\n homepage: https://litestream.io\n maintainer: Litestream Contributors \n description: Streaming replication for SQLite databases\n license: Apache 2.0\n formats:\n - deb\n - rpm\n contents:\n - src: etc/litestream.yml\n dst: /etc/litestream.yml\n type: config\n - src: etc/litestream.service\n dst: /lib/systemd/system/litestream.service\n type: config\n bindir: /usr/bin\n file_name_template: >-\n {{ .ProjectName }}-\n {{- .Version }}-\n {{- .Os }}-\n {{- if eq .Arch \"amd64\" }}x86_64\n {{- else if eq .Arch \"386\" }}i386\n {{- else }}{{ .Arch }}{{ end }}\n {{- if .Arm }}v{{ .Arm }}{{ end }}\n\nbrews:\n - name: litestream\n homepage: https://litestream.io\n description: Streaming replication for SQLite databases\n license: Apache-2.0\n repository:\n owner: benbjohnson\n name: homebrew-litestream\n branch: main\n token: \"{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}\"\n install: |\n bin.install \"litestream\"\n etc.install \"etc/litestream.yml\" => \"litestream.yml\"\n test: |\n system \"#{bin}/litestream\", \"version\"\n commit_author:\n name: goreleaser\n email: bot@goreleaser.com\n\nchecksum:\n name_template: 'checksums.txt'\n algorithm: sha256\n\nsnapshot:\n version_template: \"{{ .Tag }}-next\"\n\nchangelog:\n sort: asc\n filters:\n exclude:\n - '^docs:'\n - '^test:'\n - '^chore:'\n - 'Merge pull request'\n - 'Merge branch'\n\nrelease:\n github:\n owner: benbjohnson\n name: litestream\n draft: false\n prerelease: auto\n mode: replace\n header: |\n ## Platform Support\n\n ⚠️ **Windows Notice**: Windows binaries are provided for convenience but Windows is NOT an officially supported platform. Use at your own risk. Community contributions for Windows improvements are welcome.\n\n ✅ **Supported Platforms**: Linux (amd64, arm64, armv6, armv7), macOS (amd64, arm64)\n\n ## Installation\n\n ### Homebrew (macOS and Linux)\n ```bash\n brew tap benbjohnson/litestream\n brew install litestream\n ```\n\n ### Debian/Ubuntu\n Download the `.deb` file for your architecture and install:\n ```bash\n sudo dpkg -i litestream-*.deb\n ```\n\n ### RPM-based systems\n Download the `.rpm` file for your architecture and install:\n ```bash\n sudo rpm -i litestream-*.rpm\n ```\n\n ### Binary installation\n Download the appropriate archive for your platform, extract, and move to your PATH.\n\n ## VFS Extension (Experimental)\n\n SQLite loadable extensions for read-only access to Litestream replicas are available for supported platforms:\n\n | Platform | File |\n |----------|------|\n | Linux x86_64 | `litestream-vfs-v{{.Version}}-linux-amd64.tar.gz` |\n | Linux ARM64 | `litestream-vfs-v{{.Version}}-linux-arm64.tar.gz` |\n | macOS Intel | `litestream-vfs-v{{.Version}}-darwin-amd64.tar.gz` |\n | macOS Apple Silicon | `litestream-vfs-v{{.Version}}-darwin-arm64.tar.gz` |\n\n Install via package managers:\n ```bash\n pip install litestream-vfs # Python\n npm install litestream-vfs # Node.js\n gem install litestream-vfs # Ruby\n ```\n\n # Signing configuration\n # signs:\n # - id: macos\n # cmd: gon\n # args:\n # - \"{{ .ProjectPath }}/gon-sign.hcl\"\n # artifacts: archive\n # ids:\n # - main\n # signature: \"${artifact}.zip\"\n # output: true\n # env:\n # - APPLE_DEVELOPER_ID_APPLICATION={{ .Env.APPLE_DEVELOPER_ID }}\n # - APPLE_DEVELOPER_TEAM_ID={{ .Env.APPLE_TEAM_ID }}\n # - AC_PASSWORD={{ .Env.AC_PASSWORD }}\n\nsboms:\n - artifacts: archive\n" }, { "path": ".markdownlint.json", "content": "{\n \"default\": true,\n \"MD013\": false,\n \"MD024\": false,\n \"MD026\": false,\n \"MD031\": false,\n \"MD032\": false,\n \"MD033\": {\n \"allowed_elements\": [\"br\", \"kbd\", \"sub\", \"sup\"]\n },\n \"MD041\": false\n}\n" }, { "path": ".pre-commit-config.yaml", "content": "repos:\n - repo: https://github.com/pre-commit/pre-commit-hooks\n rev: v4.1.0\n hooks:\n - id: trailing-whitespace\n exclude_types: [markdown]\n - id: end-of-file-fixer\n - id: check-yaml\n - id: check-added-large-files\n\n - repo: https://github.com/tekwizely/pre-commit-golang\n rev: v1.0.0-beta.5\n hooks:\n - id: go-imports-repo\n args:\n - \"-local\"\n - \"github.com/benbjohnson/litestream\"\n - \"-w\"\n - id: go-vet-repo-mod\n - id: go-staticcheck-repo-mod\n" }, { "path": "AGENTS.md", "content": "# AGENTS.md - Litestream AI Agent Guide\n\nLitestream is a disaster recovery tool for SQLite that runs as a background process, monitors the WAL, converts changes to immutable LTX files, and replicates them to cloud storage. It uses `modernc.org/sqlite` (pure Go, no CGO required).\n\n## Before You Start\n\n1. Read [AI_PR_GUIDE.md](AI_PR_GUIDE.md) for contribution requirements\n2. Check [CONTRIBUTING.md](CONTRIBUTING.md) for what we accept (bug fixes welcome, features need discussion)\n3. Review recent PRs for current patterns\n\n## Critical Rules\n\n- **Lock page at 1GB**: SQLite reserves page at 0x40000000. Always skip it. See [docs/SQLITE_INTERNALS.md](docs/SQLITE_INTERNALS.md)\n- **LTX files are immutable**: Never modify after creation. See [docs/LTX_FORMAT.md](docs/LTX_FORMAT.md)\n- **Single replica per database**: Each DB replicates to exactly one destination\n- **Use `litestream ltx`**: Not `litestream wal` (deprecated)\n- **Use `litestream reset`**: Clears corrupted local LTX state for a database. See `cmd/litestream/reset.go`\n- **`auto-recover` config**: Replica option that automatically resets local state on LTX errors. Disabled by default. See `replica.go`\n- **Retention enabled by default**: `Store.RetentionEnabled` is `true` by default. Disable only when cloud lifecycle policies handle cleanup. See `store.go`\n- **IPC socket disabled by default**: Control socket is off by default. Enable with `socket.enabled: true` in config. See `server.go`\n- **`$PID` config expansion**: Config files support `$PID` to expand to the current process ID, plus standard `$ENV_VAR` expansion. See `cmd/litestream/main.go`\n- **`litestream ltx -level`**: Use `-level 0`–`9` or `-level all` to inspect specific compaction levels. See `cmd/litestream/ltx.go`\n- **Return errors, don't log them**: Always return errors to callers. Never `log.Printf(err)` and continue — this silently hides failures in a disaster recovery tool. Only use DEBUG log for best-effort operations where failure doesn't affect correctness and a valid fallback exists (e.g., reading SHM mxFrame optimization hint). See [docs/PATTERNS.md](docs/PATTERNS.md#error-handling)\n\n## Layer Boundaries\n\n| Layer | File | Responsibility |\n|-------|------|----------------|\n| DB | `db.go` | Database state, restoration, WAL monitoring, library API (`SyncStatus`, `SyncAndWait`, `EnsureExists`) |\n| Replica | `replica.go` | Replication mechanics only |\n| Storage | `**/replica_client.go` | Backend implementations (includes `ReplicaClientV3` for v0.3.x restore) |\n| IPC | `server.go` | Unix socket control API (register/unregister, /txid, pprof) |\n| Leasing | `leaser.go`, `s3/leaser.go` | Distributed lease acquisition via conditional writes |\n\nDatabase state logic belongs in DB layer, not Replica layer.\n\n## Quick Reference\n\n**Build:**\n\n```bash\ngo build -o bin/litestream ./cmd/litestream\ngo test -race -v ./...\n```\n\n**Code quality:**\n\n```bash\npre-commit run --all-files\n```\n\n## Documentation\n\n| Document | When to Read |\n|----------|--------------|\n| [docs/PATTERNS.md](docs/PATTERNS.md) | Code patterns and anti-patterns |\n| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Deep component details |\n| [docs/SQLITE_INTERNALS.md](docs/SQLITE_INTERNALS.md) | WAL format, lock page |\n| [docs/LTX_FORMAT.md](docs/LTX_FORMAT.md) | Replication format |\n| [docs/TESTING_GUIDE.md](docs/TESTING_GUIDE.md) | Test strategies |\n| [docs/REPLICA_CLIENT_GUIDE.md](docs/REPLICA_CLIENT_GUIDE.md) | Adding storage backends |\n| [docs/PROVIDER_COMPATIBILITY.md](docs/PROVIDER_COMPATIBILITY.md) | Provider-specific S3/cloud configs |\n\n## Checklist\n\nBefore submitting changes:\n\n- [ ] Read relevant docs above\n- [ ] Follow patterns in [docs/PATTERNS.md](docs/PATTERNS.md)\n- [ ] Test with race detector (`go test -race`)\n- [ ] Run `pre-commit run --all-files`\n- [ ] For page iteration: test with >1GB databases\n- [ ] Show investigation evidence in PR (see [AI_PR_GUIDE.md](AI_PR_GUIDE.md))\n" }, { "path": "AI_PR_GUIDE.md", "content": "# AI-Assisted Contribution Guide\n\nThis guide helps AI assistants (and humans using them) submit high-quality PRs to Litestream.\n\n## TL;DR Checklist\n\nBefore submitting a PR:\n\n- [ ] **Show your investigation** - Include logs, file patterns, or debug output proving the problem\n- [ ] **Define scope clearly** - State what this PR does AND does not do\n- [ ] **Include runnable test commands** - Not just descriptions, actual `go test` commands\n- [ ] **Reference related issues/PRs** - Show awareness of related work\n- [ ] **Error handling**: Does the code return errors to callers? Watch for `log.Printf(err)` followed by `continue` or `return nil` — this silently swallows failures.\n\n## What Makes PRs Succeed\n\nAnalysis of recent PRs shows successful submissions share these patterns:\n\n### 1. Investigation Artifacts\n\nShow evidence, don't just describe the fix.\n\n**Good:**\n\n```markdown\n## Problem\nFile patterns show excessive snapshot creation after checkpoint:\n- 21:43 5.2G snapshot.ltx\n- 21:47 5.2G snapshot.ltx (after checkpoint - should not trigger new snapshot)\n\nDebug logs show `verify()` incorrectly detecting position mismatch...\n```\n\n**Bad:**\n\n```markdown\n## Problem\nSnapshots are created too often. This PR fixes it.\n```\n\n### 2. Clear Scope Definition\n\nExplicitly state boundaries.\n\n**Good:**\n\n```markdown\n## Scope\nThis PR adds the lease client interface only.\n\n**In scope:**\n- LeaseClient interface definition\n- Mock implementation for testing\n\n**Not in scope (future PRs):**\n- Integration with Store\n- Distributed coordination logic\n```\n\n**Bad:**\n\n```markdown\n## Changes\nAdded leasing support and also fixed a checkpoint bug I noticed.\n```\n\n### 3. Runnable Test Commands\n\n**Good:** Include actual commands that can be run:\n\n```bash\n# Unit tests\ngo test -race -v -run TestDB_CheckpointDoesNotTriggerSnapshot ./...\n\n# Integration test with file backend\ngo test -v ./replica_client_test.go -integration file\n```\n\n**Bad:** Vague descriptions like \"Manual testing with file backend\" or \"Verified it works\"\n\n### 4. Before/After Comparison\n\nFor behavior changes, show the difference:\n\n**Good:**\n\n```markdown\n## Behavior Change\n\n| Scenario | Before | After |\n|----------|--------|-------|\n| Checkpoint with no changes | Creates snapshot | No snapshot |\n| Checkpoint with changes | Creates snapshot | Creates snapshot |\n```\n\n## Common Mistakes\n\n### Scope Creep\n\n**Problem:** Mixing unrelated changes in one PR.\n\n**Example:** PR titled \"Add lease client\" also includes a fix for checkpoint timing.\n\n**Fix:** Split into separate PRs. Reference them: \"This PR adds the lease client. The checkpoint fix is in #XXX.\"\n\n### Missing Root Cause Analysis\n\n**Problem:** Implementing a fix without proving the problem exists.\n\n**Example:** \"Add exponential backoff\" without showing what's filling disk.\n\n**Fix:** Include investigation showing the actual cause before proposing solution.\n\n### Vague Test Plans\n\n**Problem:** \"Tested manually\" or \"Verified it works.\"\n\n**Fix:** Include exact commands:\n\n```bash\ngo test -race -v -run TestSpecificFunction ./...\n```\n\n### No Integration Context\n\n**Problem:** Large features without explaining how they fit.\n\n**Fix:** For multi-PR work, explain the phases:\n\n```markdown\nThis is Phase 1 of 3 for distributed leasing:\n1. **This PR**: Lease client interface\n2. Future: Store integration\n3. Future: Distributed coordination\n```\n\n## PR Description Template\n\nUse this structure for PR descriptions:\n\n```text\n## Summary\n[1-2 sentences: what this PR does]\n\n## Problem\n[Evidence of the problem - logs, file patterns, user reports]\n\n## Solution\n[Brief explanation of the approach]\n\n## Scope\n**In scope:**\n- [item]\n\n**Not in scope:**\n- [item]\n\n## Test Plan\n[Include actual go test commands here]\n\n## Related\n- Fixes #XXX\n- Related to #YYY\n```\n\n## What We Accept\n\nFrom [CONTRIBUTING.md](CONTRIBUTING.md):\n\n- **Bug fixes** - Welcome, especially with evidence\n- **Small improvements** - Performance, code cleanup\n- **Documentation** - Always welcome\n- **Features** - Discuss in issue first; large features typically implemented internally\n\n## Resources\n\n- [AGENTS.md](AGENTS.md) - Project overview and checklist\n- [docs/PATTERNS.md](docs/PATTERNS.md) - Code patterns\n- [CONTRIBUTING.md](CONTRIBUTING.md) - Contribution guidelines\n" }, { "path": "CONTRIBUTING.md", "content": "# Contributing to Litestream\n\nThank you for your interest in contributing to Litestream! We value community contributions and appreciate your help in making Litestream better.\n\n## Types of Contributions We Accept\n\n### ✅ We Encourage and Accept\n\n- **Bug fixes and patches**: If you've found a bug and have a fix, we welcome your contribution\n- **Security vulnerability reports**: Please report security issues responsibly (see Security section below)\n- **Documentation improvements**: Help make our docs clearer and more comprehensive\n- **Testing and feedback**: Report issues, test new features, and provide feedback\n- **Small code improvements**: Performance optimizations, code cleanup, and minor enhancements\n\n### ⚠️ Discuss First\n\n- **Feature requests**: Please open an issue to discuss new features before implementing them\n- **Large changes**: For significant modifications, please discuss your approach in an issue first\n\n### ❌ Generally Not Accepted\n\n- **Large external feature contributions**: Features carry a long-term maintenance burden. To reduce burnout and maintain code quality, we typically implement major features internally. This allows us to ensure consistency with the overall architecture and maintain the high reliability that Litestream users depend on for disaster recovery\n- **Breaking changes**: Changes that break backward compatibility require extensive discussion\n\n## AI-Assisted Contributions\n\nWe welcome AI-assisted contributions for bug fixes and small improvements. Whether you're using Claude, Copilot, Cursor, or other AI tools:\n\n**Requirements:**\n\n- **Show your investigation** - Include evidence (logs, file patterns, debug output) proving the problem exists\n- **Define scope clearly** - State what the PR does and does not do\n- **Include runnable test commands** - Actual `go test` commands, not just descriptions\n- **Human review before submission** - You're responsible for the code you submit\n\n**Resources:**\n\n- [AI_PR_GUIDE.md](AI_PR_GUIDE.md) - Detailed guide with templates and examples\n- [AGENTS.md](AGENTS.md) - Project overview for AI assistants\n\n## How to Contribute\n\n### Reporting Bugs\n\nBefore reporting a bug:\n\n1. Check the [existing issues](https://github.com/benbjohnson/litestream/issues) to avoid duplicates\n2. Verify you're using the latest version of Litestream\n3. Gather diagnostic information (OS, version, configuration, error messages)\n\nWhen reporting a bug, please use our issue template and include:\n\n- Your operating system and version\n- Litestream version (`litestream version`)\n- Relevant configuration (sanitized of sensitive data)\n- Steps to reproduce the issue\n- Expected vs actual behavior\n- Any error messages or logs\n\n### Submitting Pull Requests\n\n1. **Fork the repository** and create a new branch from `main`\n2. **Make your changes** following our code style (see Development section)\n3. **Add or update tests** as appropriate\n4. **Update documentation** if you're changing behavior\n5. **Run tests and linters** locally:\n\n ```bash\n go test -v ./...\n go vet ./...\n go fmt ./...\n goimports -local github.com/benbjohnson/litestream -w .\n pre-commit run --all-files\n ```\n\n6. **Submit a pull request** with a clear description of your changes\n\n### Pull Request Guidelines\n\nYour PR should:\n\n- Have a clear, descriptive title\n- Reference any related issues (e.g., \"Fixes #123\")\n- Include tests for bug fixes and new features\n- Pass all CI checks\n- Have a focused scope (one bug fix or feature per PR)\n\n## Development Setup\n\n### Prerequisites\n\n- Go 1.24 or later\n- CGO enabled (for SQLite integration)\n- Git\n- Pre-commit (optional but recommended): `pip install pre-commit`\n\n### Building from Source\n\n```bash\n# Clone the repository\ngit clone https://github.com/benbjohnson/litestream.git\ncd litestream\n\n# Build the binary\ngo build ./cmd/litestream\n\n# Run tests\ngo test -v ./...\n\n# Install pre-commit hooks (recommended)\npre-commit install\n```\n\n### Code Style\n\n- Follow standard Go conventions\n- Use `gofmt` and `goimports` for formatting\n- Run `go vet` and `staticcheck` for static analysis\n- Keep functions focused and well-documented\n- Add comments for exported types and functions\n\n### Testing\n\n- Write unit tests for new functionality\n- Ensure existing tests pass before submitting PRs\n- Integration tests require specific environment setup (see test files for details)\n\n## Security\n\nIf you discover a security vulnerability, please:\n\n1. **DO NOT** open a public issue\n2. Email the maintainers directly with details\n3. Allow time for the issue to be addressed before public disclosure\n\n## Code of Conduct\n\nWe expect all contributors to:\n\n- Be respectful and inclusive\n- Welcome newcomers and help them get started\n- Focus on constructive criticism\n- Respect differing viewpoints and experiences\n\n## Getting Help\n\n- **Documentation**: [litestream.io](https://litestream.io)\n- **Issues**: [GitHub Issues](https://github.com/benbjohnson/litestream/issues)\n\n## License\n\nBy contributing to Litestream, you agree that your contributions will be licensed under the Apache License 2.0, the same as the project.\n\n## Acknowledgments\n\nThank you to all our contributors! Your efforts help make Litestream a reliable disaster recovery tool for the SQLite community.\n" }, { "path": "Dockerfile", "content": "FROM golang:1.25 AS builder\n\n# Install build dependencies for VFS extension\nRUN apt-get update && apt-get install -y gcc libc6-dev && rm -rf /var/lib/apt/lists/*\n\nWORKDIR /src/litestream\nCOPY . .\n\nARG LITESTREAM_VERSION=latest\n\n# Build litestream binary\nRUN --mount=type=cache,target=/root/.cache/go-build \\\n\t--mount=type=cache,target=/go/pkg \\\n\tgo build -ldflags \"-s -w -X 'main.Version=${LITESTREAM_VERSION}' -extldflags '-static'\" -tags osusergo,netgo,sqlite_omit_load_extension -o /usr/local/bin/litestream ./cmd/litestream\n\n# Build VFS loadable extension\nRUN --mount=type=cache,target=/root/.cache/go-build \\\n\t--mount=type=cache,target=/go/pkg \\\n\tmkdir -p dist && \\\n\tCGO_ENABLED=1 go build \\\n\t-tags \"vfs,SQLITE3VFS_LOADABLE_EXT\" \\\n\t-buildmode=c-archive \\\n\t-o dist/litestream-vfs.a ./cmd/litestream-vfs && \\\n\tmv dist/litestream-vfs.h src/litestream-vfs.h && \\\n\tgcc -DSQLITE3VFS_LOADABLE_EXT -g -fPIC -shared \\\n\t-o dist/litestream-vfs.so \\\n\tsrc/litestream-vfs.c \\\n\tdist/litestream-vfs.a \\\n\t-lpthread -ldl -lm\n\n# --- Hardened image (Scratch) ---\nFROM alpine:3.21 AS certs\nRUN apk --update add ca-certificates && \\\n\techo \"nonroot:x:65532:65532:nonroot:/home/nonroot:/sbin/nologin\" > /etc/minimal-passwd && \\\n\techo \"nonroot:x:65532:\" > /etc/minimal-group\n\nFROM scratch AS hardened\nCOPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt\nCOPY --from=certs /etc/minimal-passwd /etc/passwd\nCOPY --from=certs /etc/minimal-group /etc/group\nCOPY --from=builder /usr/local/bin/litestream /usr/local/bin/litestream\nUSER nonroot:nonroot\nENTRYPOINT [\"/usr/local/bin/litestream\"]\nCMD []\n\n# --- Default image (Debian) ---\nFROM debian:bookworm-slim AS default\n\nRUN apt-get update && \\\n\tapt-get install -y ca-certificates sqlite3 && \\\n\trm -rf /var/lib/apt/lists/*\n\nCOPY --from=builder /usr/local/bin/litestream /usr/local/bin/litestream\nCOPY --from=builder /src/litestream/dist/litestream-vfs.so /usr/local/lib/litestream-vfs.so\n\nENTRYPOINT [\"/usr/local/bin/litestream\"]\nCMD []\n" }, { "path": "GEMINI.md", "content": "# GEMINI.md - Gemini Code Assist Configuration\n\nGemini-specific configuration for Litestream. See [AGENTS.md](AGENTS.md) for project documentation.\n\n## Before Contributing\n\n1. Read [AI_PR_GUIDE.md](AI_PR_GUIDE.md) - PR quality requirements\n2. Read [AGENTS.md](AGENTS.md) - Project overview and checklist\n3. Check [CONTRIBUTING.md](CONTRIBUTING.md) - What we accept\n\n## File Exclusions\n\nCheck `.aiexclude` for patterns of files that should not be shared with Gemini.\n\n## Gemini Strengths for This Project\n\n- **Test generation** - Creating comprehensive test suites\n- **Documentation** - Generating and updating docs\n- **Code review** - Identifying issues and security concerns\n- **Local codebase awareness** - Enable for full repository understanding\n\n## Documentation\n\nLoad as needed:\n\n- [docs/PATTERNS.md](docs/PATTERNS.md) - Code patterns when writing code\n- [docs/SQLITE_INTERNALS.md](docs/SQLITE_INTERNALS.md) - For WAL/page work\n- [docs/TESTING_GUIDE.md](docs/TESTING_GUIDE.md) - For test generation\n\n## Critical Rules\n\n- **Lock page at 1GB** - Skip page at 0x40000000\n- **LTX files are immutable** - Never modify after creation\n- **Layer boundaries** - DB handles state, Replica handles replication\n\n## Quick Commands\n\n```bash\ngo build -o bin/litestream ./cmd/litestream\ngo test -race -v ./...\npre-commit run --all-files\n```\n" }, { "path": "LICENSE", "content": "\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright [yyyy] [name of copyright owner]\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n" }, { "path": "Makefile", "content": "default:\n\ndocker:\n\tdocker build -t litestream .\n\n# VFS build configuration\nVFS_BUILD_TAGS := vfs,SQLITE3VFS_LOADABLE_EXT\nVFS_SRC := ./cmd/litestream-vfs\nVFS_C_SRC := src/litestream-vfs.c\nMACOSX_MIN_VERSION := 11.0\nDARWIN_LDFLAGS := -framework CoreFoundation -framework Security -lresolv -mmacosx-version-min=$(MACOSX_MIN_VERSION)\nLINUX_LDFLAGS := -lpthread -ldl -lm\n\n.PHONY: vfs\nvfs:\n\tmkdir -p dist\n\tgo build -tags vfs,SQLITE3VFS_LOADABLE_EXT -o dist/litestream-vfs.a -buildmode=c-archive ./cmd/litestream-vfs\n\tmv dist/litestream-vfs.h src/litestream-vfs.h\n\tgcc -DSQLITE3VFS_LOADABLE_EXT -framework CoreFoundation -framework Security -lresolv -g -fPIC -shared -o dist/litestream-vfs.so src/litestream-vfs.c dist/litestream-vfs.a\n\n.PHONY: vfs-linux-amd64\nvfs-linux-amd64:\n\tmkdir -p dist\n\tCGO_ENABLED=1 GOOS=linux GOARCH=amd64 \\\n\t\tgo build -tags $(VFS_BUILD_TAGS) -o dist/litestream-vfs-linux-amd64.a -buildmode=c-archive $(VFS_SRC)\n\tcp dist/litestream-vfs-linux-amd64.h src/litestream-vfs.h\n\tgcc -DSQLITE3VFS_LOADABLE_EXT -g -fPIC -shared -o dist/litestream-vfs-linux-amd64.so \\\n\t\t$(VFS_C_SRC) dist/litestream-vfs-linux-amd64.a $(LINUX_LDFLAGS)\n\n.PHONY: vfs-linux-arm64\nvfs-linux-arm64:\n\tmkdir -p dist\n\tCGO_ENABLED=1 GOOS=linux GOARCH=arm64 CC=aarch64-linux-gnu-gcc \\\n\t\tgo build -tags $(VFS_BUILD_TAGS) -o dist/litestream-vfs-linux-arm64.a -buildmode=c-archive $(VFS_SRC)\n\tcp dist/litestream-vfs-linux-arm64.h src/litestream-vfs.h\n\taarch64-linux-gnu-gcc -DSQLITE3VFS_LOADABLE_EXT -g -fPIC -shared -o dist/litestream-vfs-linux-arm64.so \\\n\t\t$(VFS_C_SRC) dist/litestream-vfs-linux-arm64.a $(LINUX_LDFLAGS)\n\n.PHONY: vfs-darwin-amd64\nvfs-darwin-amd64:\n\tmkdir -p dist\n\tCGO_ENABLED=1 GOOS=darwin GOARCH=amd64 \\\n\t\tgo build -tags $(VFS_BUILD_TAGS) -o dist/litestream-vfs-darwin-amd64.a -buildmode=c-archive $(VFS_SRC)\n\tcp dist/litestream-vfs-darwin-amd64.h src/litestream-vfs.h\n\tclang -DSQLITE3VFS_LOADABLE_EXT -arch x86_64 -g -fPIC -shared -o dist/litestream-vfs-darwin-amd64.dylib \\\n\t\t$(VFS_C_SRC) dist/litestream-vfs-darwin-amd64.a $(DARWIN_LDFLAGS)\n\n.PHONY: vfs-darwin-arm64\nvfs-darwin-arm64:\n\tmkdir -p dist\n\tCGO_ENABLED=1 GOOS=darwin GOARCH=arm64 \\\n\t\tgo build -tags $(VFS_BUILD_TAGS) -o dist/litestream-vfs-darwin-arm64.a -buildmode=c-archive $(VFS_SRC)\n\tcp dist/litestream-vfs-darwin-arm64.h src/litestream-vfs.h\n\tclang -DSQLITE3VFS_LOADABLE_EXT -arch arm64 -g -fPIC -shared -o dist/litestream-vfs-darwin-arm64.dylib \\\n\t\t$(VFS_C_SRC) dist/litestream-vfs-darwin-arm64.a $(DARWIN_LDFLAGS)\n\nvfs-test:\n\tgo test -v -tags=vfs ./cmd/litestream-vfs\n\n.PHONY: clean\nclean:\n\trm -rf dist\n\nmcp-wrap:\n\tfly mcp wrap --mcp=\"./dist/litestream\" --bearer-token=$(FLY_MCP_BEARER_TOKEN) --debug -- mcp --debug\n\nmcp-inspect:\n\tfly mcp proxy -i --url http://localhost:8080/ --bearer-token=$(FLY_MCP_BEARER_TOKEN)\n\n\n.PHONY: default clean mcp-wrap mcp-inspect\n" }, { "path": "README.md", "content": "Litestream\n![GitHub release (latest by date)](https://img.shields.io/github/v/release/benbjohnson/litestream)\n![Status](https://img.shields.io/badge/status-beta-blue)\n![GitHub](https://img.shields.io/github/license/benbjohnson/litestream)\n[![Docker Pulls](https://img.shields.io/docker/pulls/litestream/litestream.svg?maxAge=604800)](https://hub.docker.com/r/litestream/litestream/)\n==========\n\nLitestream is a standalone disaster recovery tool for SQLite. It runs as a\nbackground process and safely replicates changes incrementally to another file\nor S3. Litestream only communicates with SQLite through the SQLite API so it\nwill not corrupt your database.\n\nIf you need support or have ideas for improving Litestream, please visit\n[GitHub Issues](https://github.com/benbjohnson/litestream/issues).\nPlease visit the [Litestream web site](https://litestream.io) for installation\ninstructions and documentation.\n\nIf you find this project interesting, please consider starring the project on\nGitHub.\n\nContributing\n------------\n\nWe welcome bug reports, fixes, and patches! Please see our [Contributing Guide](CONTRIBUTING.md) for details on how to contribute.\n\nAcknowledgements\n----------------\n\nI want to give special thanks to individuals who invest much of their time and\nenergy into the project to help make it better:\n\n- Thanks to [Cory LaNou](https://twitter.com/corylanou) for giving early feedback and testing when Litestream was still pre-release.\n- Thanks to [Michael Lynch](https://github.com/mtlynch) for digging into issues and contributing to the documentation.\n- Thanks to [Kurt Mackey](https://twitter.com/mrkurt) for feedback and testing.\n- Thanks to [Sam Weston](https://twitter.com/cablespaghetti) for figuring out how to run Litestream on Kubernetes and writing up the docs for it.\n- Thanks to [Rafael](https://github.com/netstx) & [Jungle Boogie](https://github.com/jungle-boogie) for helping to get OpenBSD release builds working.\n- Thanks to [Simon Gottschlag](https://github.com/simongottschlag), [Marin](https://github.com/supermarin),[Victor Björklund](https://github.com/victorbjorklund), [Jonathan Beri](https://twitter.com/beriberikix) [Yuri](https://github.com/yurivish), [Nathan Probst](https://github.com/nprbst), [Yann Coleu](https://github.com/yanc0), and [Nicholas Grilly](https://twitter.com/ngrilly) for frequent feedback, testing, & support.\n\nHuge thanks to fly.io for their support and for contributing credits for testing and development!\n" }, { "path": "SECURITY.md", "content": "# Security Policy\n\n## Reporting Security Vulnerabilities\n\nWe take security issues seriously. If you discover a security vulnerability in Litestream, please report it responsibly.\n\n**Please DO NOT open a public GitHub issue for security vulnerabilities.**\n\nInstead, please report security issues via email to the maintainers. This allows us to assess the issue and release a fix before the vulnerability is publicly disclosed.\n\n## What to Include\n\nWhen reporting a security issue, please include:\n\n- Description of the vulnerability\n- Steps to reproduce the issue\n- Potential impact\n- Any suggested fixes (if you have them)\n\n## Response Timeline\n\nWe will make our best effort to acknowledge receipt of your report as soon as possible and keep you informed of our progress. Please understand that as an open source project, response times may vary.\n\nThank you for helping keep Litestream and its users safe!\n" }, { "path": "_examples/library/README.md", "content": "# Litestream Library Usage Examples\n\nThese examples demonstrate how to use Litestream as a Go library instead of as a\nstandalone CLI tool.\n\n## API Stability Warning\n\nThe Litestream library API is not considered stable and may change between\nversions. The CLI interface is more stable for production use. Use the library\nAPI at your own risk, and pin to specific versions.\n\n**Note (POSIX platforms):** All POSIX platforms (Linux, macOS, BSD, etc.) use\nper-process locks for SQLite, not per-handle locks. If you open the same\ndatabase with two different SQLite driver implementations in the same process\nand close one of them, you can hit locking issues. You **must** use\n`modernc.org/sqlite` for your app since Litestream uses it internally.\n\n## Important Constraints\n\nWhen using Litestream as a library, be aware of these critical requirements:\n\n1. **Required Driver**: You must use `modernc.org/sqlite`. Litestream uses this\n driver internally, and mixing drivers causes lock conflicts on POSIX systems.\n\n2. **Lifecycle Management**: You cannot call `litestream.DB.Close()` or\n `Replica.Stop(true)` while your application still has open database\n connections. Either close all your app's database connections first, or only\n close Litestream when your process is shutting down.\n\n3. **PRAGMA Configuration**: Use DSN parameters (e.g.,\n `?_pragma=busy_timeout(5000)`) instead of `PRAGMA` statements via\n `ExecContext`. An `sql.DB` is a connection pool, and `ExecContext` only\n applies the PRAGMA to one random connection from the pool.\n\n## Examples\n\n### Basic (File Backend)\n\nThe simplest example using local filesystem replication.\n\n```bash\ncd basic\ngo run main.go\n```\n\nThis creates:\n\n- `myapp.db` - The SQLite database\n- `replica/` - Directory containing replicated LTX files\n\n### S3 Backend\n\nA more complete example showing the restore-on-startup pattern with S3.\n\n```bash\ncd s3\n\n# Set required environment variables\nexport AWS_ACCESS_KEY_ID=\"your-access-key\"\nexport AWS_SECRET_ACCESS_KEY=\"your-secret-key\"\nexport LITESTREAM_BUCKET=\"your-bucket-name\"\nexport LITESTREAM_PATH=\"databases/myapp\" # optional, defaults to \"litestream\"\nexport AWS_REGION=\"us-east-1\" # optional, defaults to \"us-east-1\"\n\ngo run main.go\n```\n\nThis example:\n\n1. Checks if the local database exists\n2. If not, attempts to restore from S3\n3. Starts background replication to S3\n4. Inserts sample data every 2 seconds\n5. Gracefully shuts down on Ctrl+C\n\n## Core API Pattern\n\n```go\nimport (\n \"context\"\n \"database/sql\"\n \"github.com/benbjohnson/litestream\"\n \"github.com/benbjohnson/litestream/file\" // or s3, gs, abs, etc.\n _ \"modernc.org/sqlite\"\n)\n\n// 1. Create database wrapper\ndb := litestream.NewDB(\"/path/to/db.sqlite\")\n\n// 2. Create replica client\nclient := file.NewReplicaClient(\"/path/to/replica\")\n// OR from URL:\n// client, _ := litestream.NewReplicaClientFromURL(\"s3://bucket/path\")\n\n// 3. Attach replica to database\nreplica := litestream.NewReplicaWithClient(db, client)\ndb.Replica = replica\nclient.Replica = replica // file backend only; preserves ownership/permissions\n\n// 4. Create compaction levels (L0 required, plus at least one more)\nlevels := litestream.CompactionLevels{\n {Level: 0},\n {Level: 1, Interval: 10 * time.Second},\n}\n\n// 5. Create Store to manage DB and background compaction\nstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\n// 6. Open Store (opens all DBs, starts background monitors)\nif err := store.Open(ctx); err != nil { ... }\ndefer store.Close(context.Background())\n\n// 7. Open your app's SQLite connection for normal database operations\n// Use DSN params for PRAGMAs to ensure they apply to all connections in the pool\ndsn := fmt.Sprintf(\"file:%s?_pragma=busy_timeout(5000)&_pragma=journal_mode(wal)\", \"/path/to/db.sqlite\")\nsqlDB, err := sql.Open(\"sqlite\", dsn)\nif err != nil { ... }\n```\n\n## Restore Pattern\n\n```go\n// Create replica without database for restore\nreplica := litestream.NewReplicaWithClient(nil, client)\n\nopt := litestream.NewRestoreOptions()\nopt.OutputPath = \"/path/to/restored.db\"\n// Optional: point-in-time restore\n// opt.Timestamp = time.Now().Add(-1 * time.Hour)\n\nif err := replica.Restore(ctx, opt); err != nil {\n if errors.Is(err, litestream.ErrTxNotAvailable) || errors.Is(err, litestream.ErrNoSnapshots) {\n // No backup available, create fresh database\n }\n return err\n}\n```\n\n## Supported Backends\n\n- `file` - Local filesystem\n- `s3` - AWS S3 and S3-compatible storage\n- `gs` - Google Cloud Storage\n- `abs` - Azure Blob Storage\n- `oss` - Alibaba Cloud OSS\n- `sftp` - SFTP servers\n- `nats` - NATS JetStream\n- `webdav` - WebDAV servers\n\n## Key Configuration Options\n\n### Store Settings\n\n```go\nstore.SnapshotInterval = 24 * time.Hour // How often to create snapshots\nstore.SnapshotRetention = 24 * time.Hour // How long to keep snapshots\nstore.L0Retention = 5 * time.Minute // How long to keep L0 files after compaction\n```\n\n### DB Settings\n\n```go\ndb.MonitorInterval = 1 * time.Second // How often to check for changes\ndb.CheckpointInterval = 1 * time.Minute // Time-based checkpoint interval\ndb.MinCheckpointPageN = 1000 // Page threshold for checkpoint\ndb.BusyTimeout = 1 * time.Second // SQLite busy timeout\n```\n\n### Replica Settings\n\n```go\nreplica.SyncInterval = 1 * time.Second // Time between syncs\nreplica.MonitorEnabled = true // Auto-sync in background\n```\n\n## Resources\n\n- [Litestream Documentation](https://litestream.io)\n- [GitHub Repository](https://github.com/benbjohnson/litestream)\n" }, { "path": "_examples/library/basic/main.go", "content": "// Example: Basic Litestream Library Usage\n//\n// This example demonstrates the simplest way to use Litestream as a Go library.\n// It replicates a SQLite database to the local filesystem.\n//\n// Run: go run main.go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\t\"os/signal\"\n\t\"syscall\"\n\t\"time\"\n\n\t_ \"modernc.org/sqlite\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n)\n\nfunc main() {\n\tif err := run(context.Background()); err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc run(ctx context.Context) error {\n\t// Paths for this example\n\tdbPath := \"./myapp.db\"\n\treplicaPath := \"./replica\"\n\n\t// 1. Create the Litestream DB wrapper\n\tdb := litestream.NewDB(dbPath)\n\n\t// 2. Create a replica client (file-based for this example)\n\tclient := file.NewReplicaClient(replicaPath)\n\n\t// 3. Create a replica and attach it to the database\n\treplica := litestream.NewReplicaWithClient(db, client)\n\tdb.Replica = replica\n\tclient.Replica = replica\n\n\t// 4. Create compaction levels (L0 is required, plus at least one more level)\n\tlevels := litestream.CompactionLevels{\n\t\t{Level: 0},\n\t\t{Level: 1, Interval: 10 * time.Second},\n\t}\n\n\t// 5. Create a Store to manage the database and background compaction\n\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\n\t// 6. Open the store (opens all DBs and starts background monitors)\n\tif err := store.Open(ctx); err != nil {\n\t\treturn fmt.Errorf(\"open store: %w\", err)\n\t}\n\tdefer func() {\n\t\tif err := store.Close(context.Background()); err != nil {\n\t\t\tlog.Printf(\"close store: %v\", err)\n\t\t}\n\t}()\n\n\t// 7. Open your app's SQLite connection for normal database operations\n\tsqlDB, err := openAppDB(ctx, dbPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open app db: %w\", err)\n\t}\n\tdefer sqlDB.Close()\n\tif err := initSchema(ctx, sqlDB); err != nil {\n\t\treturn fmt.Errorf(\"init schema: %w\", err)\n\t}\n\n\t// Insert some test data periodically\n\tticker := time.NewTicker(2 * time.Second)\n\tdefer ticker.Stop()\n\n\t// Handle shutdown gracefully\n\tsigCh := make(chan os.Signal, 1)\n\tsignal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)\n\n\tfmt.Println(\"Writing data every 2 seconds. Press Ctrl+C to stop.\")\n\tfmt.Printf(\"Database: %s\\n\", dbPath)\n\tfmt.Printf(\"Replica: %s\\n\", replicaPath)\n\n\tfor {\n\t\tselect {\n\t\tcase <-ticker.C:\n\t\t\tif err := insertRow(ctx, sqlDB); err != nil {\n\t\t\t\tlog.Printf(\"insert row: %v\", err)\n\t\t\t}\n\t\tcase <-sigCh:\n\t\t\tfmt.Println(\"\\nShutting down...\")\n\t\t\treturn nil\n\t\t}\n\t}\n}\n\nfunc openAppDB(_ context.Context, path string) (*sql.DB, error) {\n\tdsn := fmt.Sprintf(\"file:%s?_pragma=busy_timeout(5000)&_pragma=journal_mode(wal)\", path)\n\treturn sql.Open(\"sqlite\", dsn)\n}\n\nfunc initSchema(ctx context.Context, db *sql.DB) error {\n\t_, err := db.ExecContext(ctx, `\n\t\tCREATE TABLE IF NOT EXISTS events (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tmessage TEXT NOT NULL,\n\t\t\tcreated_at TEXT NOT NULL\n\t\t)\n\t`)\n\treturn err\n}\n\nfunc insertRow(ctx context.Context, db *sql.DB) error {\n\tmsg := fmt.Sprintf(\"Event at %s\", time.Now().Format(time.RFC3339))\n\tresult, err := db.ExecContext(ctx,\n\t\t`INSERT INTO events (message, created_at) VALUES (?, ?)`,\n\t\tmsg, time.Now().Format(time.RFC3339))\n\tif err != nil {\n\t\treturn err\n\t}\n\tid, _ := result.LastInsertId()\n\tfmt.Printf(\"Inserted row %d: %s\\n\", id, msg)\n\treturn nil\n}\n" }, { "path": "_examples/library/library_example_test.go", "content": "package library_test\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"errors\"\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"modernc.org/sqlite\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n)\n\nfunc TestLibraryExampleFileBackend(t *testing.T) {\n\tctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)\n\tdefer cancel()\n\n\trootDir := t.TempDir()\n\tdbPath := filepath.Join(rootDir, \"example.db\")\n\treplicaPath := filepath.Join(rootDir, \"replica\")\n\n\tdb := litestream.NewDB(dbPath)\n\tclient := file.NewReplicaClient(replicaPath)\n\treplica := litestream.NewReplicaWithClient(db, client)\n\tdb.Replica = replica\n\tclient.Replica = replica\n\n\tlevels := litestream.CompactionLevels{\n\t\t{Level: 0},\n\t\t{Level: 1, Interval: 10 * time.Second},\n\t}\n\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\n\tclosed := false\n\tt.Cleanup(func() {\n\t\tif !closed {\n\t\t\t_ = store.Close(context.Background())\n\t\t}\n\t})\n\n\tif err := store.Open(ctx); err != nil {\n\t\tt.Fatalf(\"open store: %v\", err)\n\t}\n\n\tsqlDB, err := openAppDB(ctx, dbPath)\n\tif err != nil {\n\t\tt.Fatalf(\"open app db: %v\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tif _, err := sqlDB.ExecContext(ctx, `\n\t\tCREATE TABLE IF NOT EXISTS events (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tmessage TEXT NOT NULL\n\t\t)\n\t`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\tif _, err := sqlDB.ExecContext(ctx, `INSERT INTO events (message) VALUES ('hello');`); err != nil {\n\t\tt.Fatalf(\"insert row: %v\", err)\n\t}\n\n\tif err := waitForLTXFiles(replicaPath, 5*time.Second); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := sqlDB.Close(); err != nil {\n\t\tt.Fatalf(\"close app db: %v\", err)\n\t}\n\n\tif err := store.Close(ctx); err != nil && !errors.Is(err, sql.ErrTxDone) {\n\t\tt.Fatalf(\"close store: %v\", err)\n\t}\n\tclosed = true\n\n\trestoreClient := file.NewReplicaClient(replicaPath)\n\trestoreReplica := litestream.NewReplicaWithClient(nil, restoreClient)\n\n\trestorePath := filepath.Join(rootDir, \"restored.db\")\n\topt := litestream.NewRestoreOptions()\n\topt.OutputPath = restorePath\n\tif err := restoreReplica.Restore(ctx, opt); err != nil {\n\t\tt.Fatalf(\"restore: %v\", err)\n\t}\n}\n\nfunc openAppDB(ctx context.Context, path string) (*sql.DB, error) {\n\tdb, err := sql.Open(\"sqlite\", path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif _, err := db.ExecContext(ctx, `PRAGMA journal_mode = wal;`); err != nil {\n\t\t_ = db.Close()\n\t\treturn nil, err\n\t}\n\tif _, err := db.ExecContext(ctx, `PRAGMA busy_timeout = 5000;`); err != nil {\n\t\t_ = db.Close()\n\t\treturn nil, err\n\t}\n\treturn db, nil\n}\n\nfunc waitForLTXFiles(replicaPath string, timeout time.Duration) error {\n\tdeadline := time.Now().Add(timeout)\n\tfor time.Now().Before(deadline) {\n\t\tmatches, err := filepath.Glob(filepath.Join(replicaPath, \"ltx\", \"0\", \"*.ltx\"))\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"glob ltx files: %w\", err)\n\t\t}\n\t\tif len(matches) > 0 {\n\t\t\treturn nil\n\t\t}\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\treturn fmt.Errorf(\"timeout waiting for ltx files in %s\", replicaPath)\n}\n" }, { "path": "_examples/library/s3/main.go", "content": "// Example: Litestream Library Usage with S3 and Restore-on-Startup\n//\n// This example demonstrates a production-like pattern for using Litestream:\n// - Check if local database exists\n// - If not, restore from S3 backup (if available)\n// - Start replication to S3\n// - Graceful shutdown\n//\n// Environment variables:\n// - AWS_ACCESS_KEY_ID: AWS access key\n// - AWS_SECRET_ACCESS_KEY: AWS secret key\n// - LITESTREAM_BUCKET: S3 bucket name (e.g., \"my-backup-bucket\")\n// - LITESTREAM_PATH: Path within bucket (e.g., \"databases/myapp\")\n// - AWS_REGION: AWS region (default: us-east-1)\n//\n// Run: go run main.go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"errors\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\t\"os/signal\"\n\t\"syscall\"\n\t\"time\"\n\n\t_ \"modernc.org/sqlite\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/s3\"\n)\n\nconst dbPath = \"./myapp.db\"\n\nfunc main() {\n\tif err := run(context.Background()); err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc run(ctx context.Context) error {\n\t// Load configuration from environment\n\tbucket := os.Getenv(\"LITESTREAM_BUCKET\")\n\tif bucket == \"\" {\n\t\treturn fmt.Errorf(\"LITESTREAM_BUCKET environment variable required\")\n\t}\n\tpath := os.Getenv(\"LITESTREAM_PATH\")\n\tif path == \"\" {\n\t\tpath = \"litestream\"\n\t}\n\tregion := os.Getenv(\"AWS_REGION\")\n\tif region == \"\" {\n\t\tregion = \"us-east-1\"\n\t}\n\n\t// 1. Create S3 replica client\n\tclient := s3.NewReplicaClient()\n\tclient.Bucket = bucket\n\tclient.Path = path\n\tclient.Region = region\n\tclient.AccessKeyID = os.Getenv(\"AWS_ACCESS_KEY_ID\")\n\tclient.SecretAccessKey = os.Getenv(\"AWS_SECRET_ACCESS_KEY\")\n\n\t// 2. Restore from S3 if local database doesn't exist\n\tif err := restoreIfNotExists(ctx, client, dbPath); err != nil {\n\t\treturn fmt.Errorf(\"restore: %w\", err)\n\t}\n\n\t// 3. Create the Litestream DB wrapper\n\tdb := litestream.NewDB(dbPath)\n\n\t// 4. Create replica and attach to database\n\treplica := litestream.NewReplicaWithClient(db, client)\n\tdb.Replica = replica\n\n\t// 5. Create compaction levels (L0 is required, plus at least one more level)\n\tlevels := litestream.CompactionLevels{\n\t\t{Level: 0},\n\t\t{Level: 1, Interval: 10 * time.Second},\n\t}\n\n\t// 6. Create a Store to manage the database and background compaction\n\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\n\t// 7. Open store (opens all DBs and starts background monitors)\n\tif err := store.Open(ctx); err != nil {\n\t\treturn fmt.Errorf(\"open store: %w\", err)\n\t}\n\tdefer func() {\n\t\tlog.Println(\"Closing store...\")\n\t\tif err := store.Close(context.Background()); err != nil {\n\t\t\tlog.Printf(\"close store: %v\", err)\n\t\t}\n\t}()\n\n\t// 8. Open your app's SQLite connection for normal operations\n\tsqlDB, err := openAppDB(ctx, dbPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open app db: %w\", err)\n\t}\n\tdefer sqlDB.Close()\n\tif err := initSchema(ctx, sqlDB); err != nil {\n\t\treturn fmt.Errorf(\"init schema: %w\", err)\n\t}\n\n\t// Start the application\n\tlog.Printf(\"Database: %s\", dbPath)\n\tlog.Printf(\"Replicating to: s3://%s/%s\", bucket, path)\n\tlog.Println(\"Writing data every 2 seconds. Press Ctrl+C to stop.\")\n\n\tticker := time.NewTicker(2 * time.Second)\n\tdefer ticker.Stop()\n\n\tsigCh := make(chan os.Signal, 1)\n\tsignal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)\n\n\tfor {\n\t\tselect {\n\t\tcase <-ticker.C:\n\t\t\tif err := insertRow(ctx, sqlDB); err != nil {\n\t\t\t\tlog.Printf(\"insert row: %v\", err)\n\t\t\t}\n\t\tcase <-sigCh:\n\t\t\tlog.Println(\"Shutting down...\")\n\t\t\treturn nil\n\t\t}\n\t}\n}\n\n// restoreIfNotExists restores the database from S3 if it doesn't exist locally.\nfunc restoreIfNotExists(ctx context.Context, client *s3.ReplicaClient, dbPath string) error {\n\t// Check if database already exists\n\tif _, err := os.Stat(dbPath); err == nil {\n\t\tlog.Println(\"Local database found, skipping restore\")\n\t\treturn nil\n\t} else if !os.IsNotExist(err) {\n\t\treturn err\n\t}\n\n\tlog.Println(\"Local database not found, attempting restore from S3...\")\n\n\t// Initialize the client\n\tif err := client.Init(ctx); err != nil {\n\t\treturn fmt.Errorf(\"init s3 client: %w\", err)\n\t}\n\n\t// Create a replica (without DB) for restore\n\treplica := litestream.NewReplicaWithClient(nil, client)\n\n\t// Set up restore options\n\topt := litestream.NewRestoreOptions()\n\topt.OutputPath = dbPath\n\n\t// Attempt restore\n\tif err := replica.Restore(ctx, opt); err != nil {\n\t\t// If no backup exists, that's OK - we'll create a fresh database\n\t\tif errors.Is(err, litestream.ErrTxNotAvailable) || errors.Is(err, litestream.ErrNoSnapshots) {\n\t\t\tlog.Println(\"No backup found in S3, will create new database\")\n\t\t\treturn nil\n\t\t}\n\t\treturn err\n\t}\n\n\tlog.Println(\"Database restored from S3\")\n\treturn nil\n}\n\nfunc openAppDB(_ context.Context, path string) (*sql.DB, error) {\n\tdsn := fmt.Sprintf(\"file:%s?_pragma=busy_timeout(5000)&_pragma=journal_mode(wal)\", path)\n\treturn sql.Open(\"sqlite\", dsn)\n}\n\nfunc initSchema(ctx context.Context, db *sql.DB) error {\n\t_, err := db.ExecContext(ctx, `\n\t\tCREATE TABLE IF NOT EXISTS events (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tmessage TEXT NOT NULL,\n\t\t\tcreated_at TEXT NOT NULL\n\t\t)\n\t`)\n\treturn err\n}\n\nfunc insertRow(ctx context.Context, db *sql.DB) error {\n\tmsg := fmt.Sprintf(\"Event at %s\", time.Now().Format(time.RFC3339))\n\tresult, err := db.ExecContext(ctx,\n\t\t`INSERT INTO events (message, created_at) VALUES (?, ?)`,\n\t\tmsg, time.Now().Format(time.RFC3339))\n\tif err != nil {\n\t\treturn err\n\t}\n\tid, _ := result.LastInsertId()\n\tlog.Printf(\"Inserted row %d: %s\", id, msg)\n\treturn nil\n}\n" }, { "path": "abs/replica_client.go", "content": "package abs\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"os\"\n\t\"path\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/to\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azidentity\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/bloberror\"\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\nfunc init() {\n\tlitestream.RegisterReplicaClientFactory(\"abs\", NewReplicaClientFromURL)\n}\n\n// ReplicaClientType is the client type for this package.\nconst ReplicaClientType = \"abs\"\n\n// MetadataKeyTimestamp is the metadata key for storing LTX file timestamps in Azure Blob Storage.\n// Azure metadata keys cannot contain hyphens, so we use litestreamtimestamp (C# identifier rules).\nconst MetadataKeyTimestamp = \"litestreamtimestamp\"\n\nvar _ litestream.ReplicaClient = (*ReplicaClient)(nil)\n\n// ReplicaClient is a client for writing LTX files to Azure Blob Storage.\ntype ReplicaClient struct {\n\tmu sync.Mutex\n\tclient *azblob.Client\n\tlogger *slog.Logger\n\n\t// Azure credentials\n\tAccountName string\n\tAccountKey string\n\tSASToken string // SAS token for container-level access\n\tEndpoint string\n\n\t// Azure Blob Storage container information\n\tBucket string\n\tPath string\n}\n\n// NewReplicaClient returns a new instance of ReplicaClient.\nfunc NewReplicaClient() *ReplicaClient {\n\treturn &ReplicaClient{\n\t\tlogger: slog.Default().WithGroup(ReplicaClientType),\n\t}\n}\n\nfunc (c *ReplicaClient) SetLogger(logger *slog.Logger) {\n\tc.logger = logger.WithGroup(ReplicaClientType)\n}\n\n// NewReplicaClientFromURL creates a new ReplicaClient from URL components.\n// This is used by the replica client factory registration.\n// URL format: abs://[account-name@]container/path\nfunc NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error) {\n\tclient := NewReplicaClient()\n\n\t// Extract account name from userinfo if present (abs://account@container/path)\n\tif userinfo != nil {\n\t\tclient.AccountName = userinfo.Username()\n\t}\n\n\tclient.Bucket = host\n\tclient.Path = urlPath\n\n\tif client.Bucket == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for abs replica URL\")\n\t}\n\n\treturn client, nil\n}\n\n// Type returns \"abs\" as the client type.\nfunc (c *ReplicaClient) Type() string {\n\treturn ReplicaClientType\n}\n\n// Init initializes the connection to Azure. No-op if already initialized.\nfunc (c *ReplicaClient) Init(ctx context.Context) (err error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tif c.client != nil {\n\t\treturn nil\n\t}\n\n\t// Validate required configuration\n\tif c.Bucket == \"\" {\n\t\treturn fmt.Errorf(\"abs: container name is required\")\n\t}\n\n\t// Construct & parse endpoint unless already set.\n\tendpoint := c.Endpoint\n\tif endpoint == \"\" {\n\t\tif c.AccountName == \"\" {\n\t\t\treturn fmt.Errorf(\"abs: account name is required when endpoint is not specified\")\n\t\t}\n\t\tendpoint = fmt.Sprintf(\"https://%s.blob.core.windows.net\", c.AccountName)\n\t}\n\n\t// Configure client options with retry policy\n\tclientOptions := &azblob.ClientOptions{\n\t\tClientOptions: azcore.ClientOptions{\n\t\t\tRetry: policy.RetryOptions{\n\t\t\t\tMaxRetries: 10,\n\t\t\t\tRetryDelay: time.Second,\n\t\t\t\tMaxRetryDelay: 30 * time.Second,\n\t\t\t\tTryTimeout: 15 * time.Minute, // Reasonable timeout for blob operations\n\t\t\t\tStatusCodes: []int{\n\t\t\t\t\thttp.StatusRequestTimeout,\n\t\t\t\t\thttp.StatusTooManyRequests,\n\t\t\t\t\thttp.StatusInternalServerError,\n\t\t\t\t\thttp.StatusBadGateway,\n\t\t\t\t\thttp.StatusServiceUnavailable,\n\t\t\t\t\thttp.StatusGatewayTimeout,\n\t\t\t\t},\n\t\t\t},\n\t\t\tTelemetry: policy.TelemetryOptions{\n\t\t\t\tApplicationID: \"litestream\",\n\t\t\t},\n\t\t},\n\t}\n\n\t// Check for SAS token first (highest priority for explicit credentials)\n\tsasToken := c.SASToken\n\tif sasToken == \"\" {\n\t\tsasToken = os.Getenv(\"LITESTREAM_AZURE_SAS_TOKEN\")\n\t}\n\n\t// Check if we have explicit credentials or should use default credential chain\n\taccountKey := c.AccountKey\n\tif accountKey == \"\" {\n\t\taccountKey = os.Getenv(\"LITESTREAM_AZURE_ACCOUNT_KEY\")\n\t}\n\n\t// Create Azure Blob Storage client with appropriate authentication\n\t// Priority: SAS token > Shared key > Default credential chain\n\tvar client *azblob.Client\n\tif sasToken != \"\" {\n\t\t// SAS token authentication - append token to endpoint URL\n\t\tif accountKey != \"\" {\n\t\t\tslog.Warn(\"both SAS token and account key configured, using SAS token\")\n\t\t} else {\n\t\t\tslog.Debug(\"using SAS token authentication\")\n\t\t}\n\t\t// Strip leading \"?\" if present to avoid double \"?\"\n\t\tendpointWithSAS := fmt.Sprintf(\"%s?%s\", endpoint, strings.TrimPrefix(sasToken, \"?\"))\n\t\tclient, err = azblob.NewClientWithNoCredential(endpointWithSAS, clientOptions)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"abs: cannot create azure blob client with SAS token: %w\", err)\n\t\t}\n\t} else if accountKey != \"\" && c.AccountName != \"\" {\n\t\t// Use shared key authentication (existing behavior)\n\t\tslog.Debug(\"using shared key authentication\")\n\t\tcredential, err := azblob.NewSharedKeyCredential(c.AccountName, accountKey)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"abs: cannot create shared key credential: %w\", err)\n\t\t}\n\t\tclient, err = azblob.NewClientWithSharedKeyCredential(endpoint, credential, clientOptions)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"abs: cannot create azure blob client with shared key: %w\", err)\n\t\t}\n\t} else {\n\t\t// Use default credential chain (similar to AWS SDK default credential chain)\n\t\t// This includes:\n\t\t// - Environment variables (AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID)\n\t\t// - Managed Identity (for Azure VMs, App Service, etc.)\n\t\t// - Azure CLI credentials\n\t\t// - Visual Studio Code credentials\n\t\tslog.Debug(\"using default credential chain (managed identity, Azure CLI, environment variables, etc.)\")\n\t\tcredential, err := azidentity.NewDefaultAzureCredential(nil)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"abs: cannot create default azure credential: %w\", err)\n\t\t}\n\t\tclient, err = azblob.NewClient(endpoint, credential, clientOptions)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"abs: cannot create azure blob client with default credential: %w\", err)\n\t\t}\n\t}\n\n\tc.client = client\n\treturn nil\n}\n\n// LTXFiles returns an iterator over all available LTX files.\n// Azure always uses accurate timestamps from metadata since they're included in LIST operations at zero cost.\n// The useMetadata parameter is ignored.\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\treturn newLTXFileIterator(ctx, c, level, seek), nil\n}\n\n// WriteLTXFile writes an LTX file to remote storage.\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, rd io.Reader) (info *ltx.FileInfo, err error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tkey := litestream.LTXFilePath(c.Path, level, minTXID, maxTXID)\n\n\t// Use TeeReader to peek at LTX header while preserving data for upload\n\tvar buf bytes.Buffer\n\tteeReader := io.TeeReader(rd, &buf)\n\n\t// Extract timestamp from LTX header\n\thdr, _, err := ltx.PeekHeader(teeReader)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"extract timestamp from LTX header: %w\", err)\n\t}\n\ttimestamp := time.UnixMilli(hdr.Timestamp).UTC()\n\n\t// Combine buffered data with rest of reader\n\trc := internal.NewReadCounter(io.MultiReader(&buf, rd))\n\n\t// Upload blob with proper content type, access tier, and metadata\n\t// Azure metadata keys cannot contain hyphens, so use litestreamtimestamp\n\t_, err = c.client.UploadStream(ctx, c.Bucket, key, rc, &azblob.UploadStreamOptions{\n\t\tHTTPHeaders: &blob.HTTPHeaders{\n\t\t\tBlobContentType: to.Ptr(\"application/octet-stream\"),\n\t\t},\n\t\tAccessTier: to.Ptr(blob.AccessTierHot), // Use Hot tier as default\n\t\tMetadata: map[string]*string{\n\t\t\tMetadataKeyTimestamp: to.Ptr(timestamp.Format(time.RFC3339Nano)),\n\t\t},\n\t})\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"abs: cannot upload ltx file %q: %w\", key, err)\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Add(float64(rc.N()))\n\n\treturn <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tSize: rc.N(),\n\t\tCreatedAt: timestamp,\n\t}, nil\n}\n\n// OpenLTXFile returns a reader for an LTX file.\n// Returns os.ErrNotExist if no matching min/max TXID is not found.\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tkey := litestream.LTXFilePath(c.Path, level, minTXID, maxTXID)\n\tresp, err := c.client.DownloadStream(ctx, c.Bucket, key, &azblob.DownloadStreamOptions{\n\t\tRange: blob.HTTPRange{\n\t\t\tOffset: offset,\n\t\t\tCount: size,\n\t\t},\n\t})\n\n\tif isNotExists(err) {\n\t\treturn nil, os.ErrNotExist\n\t} else if err != nil {\n\t\treturn nil, fmt.Errorf(\"abs: cannot start new reader for %q: %w\", key, err)\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"GET\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, \"GET\").Add(float64(*resp.ContentLength))\n\n\treturn resp.Body, nil\n}\n\n// DeleteLTXFiles deletes LTX files.\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\tfor _, info := range a {\n\t\tkey := litestream.LTXFilePath(c.Path, info.Level, info.MinTXID, info.MaxTXID)\n\n\t\tc.logger.Debug(\"deleting ltx file\", \"level\", info.Level, \"minTXID\", info.MinTXID, \"maxTXID\", info.MaxTXID, \"key\", key)\n\n\t\t_, err := c.client.DeleteBlob(ctx, c.Bucket, key, nil)\n\t\tif isNotExists(err) {\n\t\t\tcontinue\n\t\t} else if err != nil {\n\t\t\treturn fmt.Errorf(\"abs: cannot delete ltx file %q: %w\", key, err)\n\t\t}\n\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\t}\n\n\treturn nil\n}\n\n// DeleteAll deletes all LTX files.\nfunc (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\t// List all blobs with the configured path prefix\n\tprefix := \"/\"\n\tif c.Path != \"\" {\n\t\tprefix = strings.TrimSuffix(c.Path, \"/\") + \"/\"\n\t}\n\n\tpager := c.client.NewListBlobsFlatPager(c.Bucket, &azblob.ListBlobsFlatOptions{\n\t\tPrefix: &prefix,\n\t\tInclude: azblob.ListBlobsInclude{Metadata: true},\n\t})\n\n\tfor pager.More() {\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"LIST\").Inc()\n\n\t\tresp, err := pager.NextPage(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"abs: cannot list blobs: %w\", err)\n\t\t}\n\n\t\tfor _, item := range resp.Segment.BlobItems {\n\t\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\n\t\t\t_, err := c.client.DeleteBlob(ctx, c.Bucket, *item.Name, nil)\n\t\t\tif isNotExists(err) {\n\t\t\t\tcontinue\n\t\t\t} else if err != nil {\n\t\t\t\treturn fmt.Errorf(\"abs: cannot delete blob %q: %w\", *item.Name, err)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\ntype ltxFileIterator struct {\n\tctx context.Context\n\tcancel context.CancelFunc\n\tclient *ReplicaClient\n\tlevel int\n\tseek ltx.TXID\n\n\tpager *runtime.Pager[azblob.ListBlobsFlatResponse]\n\tpageItems []*ltx.FileInfo\n\tpageIndex int\n\n\tclosed bool\n\terr error\n\tinfo *ltx.FileInfo\n}\n\nfunc newLTXFileIterator(ctx context.Context, client *ReplicaClient, level int, seek ltx.TXID) *ltxFileIterator {\n\tctx, cancel := context.WithCancel(ctx)\n\n\titr := <xFileIterator{\n\t\tctx: ctx,\n\t\tcancel: cancel,\n\t\tclient: client,\n\t\tlevel: level,\n\t\tseek: seek,\n\t}\n\n\t// Create paginator for listing blobs with level prefix\n\tdir := litestream.LTXLevelDir(client.Path, level)\n\tprefix := dir + \"/\"\n\tif seek != 0 {\n\t\tprefix += seek.String()\n\t}\n\n\titr.pager = client.client.NewListBlobsFlatPager(client.Bucket, &azblob.ListBlobsFlatOptions{\n\t\tPrefix: &prefix,\n\t\tInclude: azblob.ListBlobsInclude{Metadata: true},\n\t})\n\n\treturn itr\n}\n\nfunc (itr *ltxFileIterator) Close() (err error) {\n\titr.closed = true\n\titr.cancel()\n\treturn itr.err\n}\n\nfunc (itr *ltxFileIterator) Next() bool {\n\tif itr.closed || itr.err != nil {\n\t\treturn false\n\t}\n\n\t// Process blobs until we find a valid LTX file\n\tfor {\n\t\t// Load next page if needed\n\t\tif itr.pageItems == nil || itr.pageIndex >= len(itr.pageItems) {\n\t\t\tif !itr.loadNextPage() {\n\t\t\t\treturn false\n\t\t\t}\n\t\t}\n\n\t\t// Process current item from page\n\t\tif itr.pageIndex < len(itr.pageItems) {\n\t\t\titr.info = itr.pageItems[itr.pageIndex]\n\t\t\titr.pageIndex++\n\t\t\treturn true\n\t\t}\n\t}\n}\n\n// loadNextPage loads the next page of blobs and extracts valid LTX files\nfunc (itr *ltxFileIterator) loadNextPage() bool {\n\tif !itr.pager.More() {\n\t\treturn false\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"LIST\").Inc()\n\n\tresp, err := itr.pager.NextPage(itr.ctx)\n\tif err != nil {\n\t\titr.err = fmt.Errorf(\"abs: cannot list blobs: %w\", err)\n\t\treturn false\n\t}\n\n\t// Extract blob items directly from the response\n\titr.pageItems = nil\n\titr.pageIndex = 0\n\n\tfor _, item := range resp.Segment.BlobItems {\n\t\tkey := path.Base(*item.Name)\n\t\tminTXID, maxTXID, err := ltx.ParseFilename(key)\n\t\tif err != nil {\n\t\t\tcontinue // Skip non-LTX files\n\t\t}\n\n\t\t// Build file info\n\t\tinfo := <x.FileInfo{\n\t\t\tLevel: itr.level,\n\t\t\tMinTXID: minTXID,\n\t\t\tMaxTXID: maxTXID,\n\t\t\tSize: *item.Properties.ContentLength,\n\t\t}\n\n\t\t// Skip if below seek TXID\n\t\tif info.MinTXID < itr.seek {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Skip if wrong level\n\t\tif info.Level != itr.level {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Always use accurate timestamp from metadata since it's zero-cost\n\t\t// Azure includes metadata in LIST operations, so no extra API call needed\n\t\tinfo.CreatedAt = item.Properties.CreationTime.UTC()\n\t\tif item.Metadata != nil {\n\t\t\tif ts, ok := item.Metadata[MetadataKeyTimestamp]; ok && ts != nil {\n\t\t\t\tif parsed, err := time.Parse(time.RFC3339Nano, *ts); err == nil {\n\t\t\t\t\tinfo.CreatedAt = parsed\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\titr.pageItems = append(itr.pageItems, info)\n\t}\n\n\treturn len(itr.pageItems) > 0 || itr.pager.More()\n}\n\nfunc (itr *ltxFileIterator) Err() error { return itr.err }\n\nfunc (itr *ltxFileIterator) Item() *ltx.FileInfo {\n\treturn itr.info\n}\n\nfunc isNotExists(err error) bool {\n\tvar respErr *azcore.ResponseError\n\tif errors.As(err, &respErr) {\n\t\treturn respErr.ErrorCode == string(bloberror.BlobNotFound) || respErr.ErrorCode == string(bloberror.ContainerNotFound)\n\t}\n\treturn false\n}\n" }, { "path": "cmd/litestream/databases.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"flag\"\n\t\"fmt\"\n\t\"os\"\n\t\"text/tabwriter\"\n)\n\n// DatabasesCommand is a command for listing managed databases.\ntype DatabasesCommand struct{}\n\n// Run executes the command.\nfunc (c *DatabasesCommand) Run(_ context.Context, args []string) (err error) {\n\tfs := flag.NewFlagSet(\"litestream-databases\", flag.ContinueOnError)\n\tconfigPath, noExpandEnv := registerConfigFlag(fs)\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t} else if fs.NArg() != 0 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\n\t// Load configuration.\n\tif *configPath == \"\" {\n\t\t*configPath = DefaultConfigPath()\n\t}\n\tconfig, err := ReadConfigFile(*configPath, !*noExpandEnv)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// List all databases.\n\tw := tabwriter.NewWriter(os.Stdout, 0, 8, 2, ' ', 0)\n\tdefer w.Flush()\n\n\tfmt.Fprintln(w, \"path\\treplica\")\n\tfor _, dbConfig := range config.DBs {\n\t\tdb, err := NewDBFromConfig(dbConfig)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfmt.Fprintf(w, \"%s\\t%s\\n\",\n\t\t\tdb.Path(),\n\t\t\tdb.Replica.Client.Type())\n\t}\n\n\treturn nil\n}\n\n// Usage prints the help screen to STDOUT.\nfunc (c *DatabasesCommand) Usage() {\n\tfmt.Printf(`\nThe databases command lists all databases in the configuration file.\n\nUsage:\n\n\tlitestream databases [arguments]\n\nArguments:\n\n\t-config PATH\n\t Specifies the configuration file.\n\t Defaults to %s\n\n\t-no-expand-env\n\t Disables environment variable expansion in configuration file.\n\n`[1:],\n\t\tDefaultConfigPath(),\n\t)\n}\n" }, { "path": "cmd/litestream/directory_watcher.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/fsnotify/fsnotify\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nconst debounceInterval = 250 * time.Millisecond\n\n// DirectoryMonitor watches a directory tree for SQLite databases and dynamically\n// manages database instances within the store as files are created or removed.\ntype DirectoryMonitor struct {\n\tstore *litestream.Store\n\tconfig *DBConfig\n\tdirPath string\n\tpattern string\n\trecursive bool\n\n\twatcher *fsnotify.Watcher\n\tctx context.Context\n\tcancel context.CancelFunc\n\n\tlogger *slog.Logger\n\n\tmu sync.Mutex\n\tdbs map[string]*litestream.DB\n\twatchedDirs map[string]struct{}\n\n\t// Only accessed from the run() goroutine, so no mutex is needed.\n\tpendingEvents map[string]fsnotify.Op\n\tdebounceActive bool\n\n\twg sync.WaitGroup\n}\n\n// NewDirectoryMonitor returns a new monitor for directory-based replication.\nfunc NewDirectoryMonitor(ctx context.Context, store *litestream.Store, dbc *DBConfig, existing []*litestream.DB) (*DirectoryMonitor, error) {\n\tif dbc == nil {\n\t\treturn nil, errors.New(\"database config required\")\n\t}\n\tif store == nil {\n\t\treturn nil, errors.New(\"store required\")\n\t}\n\n\tdirPath, err := expand(dbc.Dir)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif _, err := os.Stat(dirPath); err != nil {\n\t\treturn nil, err\n\t}\n\n\twatcher, err := fsnotify.NewWatcher()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmonitorCtx, cancel := context.WithCancel(ctx)\n\tdm := &DirectoryMonitor{\n\t\tstore: store,\n\t\tconfig: dbc,\n\t\tdirPath: dirPath,\n\t\tpattern: dbc.Pattern,\n\t\trecursive: dbc.Recursive,\n\t\twatcher: watcher,\n\t\tctx: monitorCtx,\n\t\tcancel: cancel,\n\t\tlogger: slog.With(\"dir\", dirPath),\n\t\tdbs: make(map[string]*litestream.DB),\n\t\twatchedDirs: make(map[string]struct{}),\n\t\tpendingEvents: make(map[string]fsnotify.Op),\n\t}\n\n\tfor _, db := range existing {\n\t\tdm.dbs[db.Path()] = db\n\t}\n\n\tif err := dm.addInitialWatches(); err != nil {\n\t\twatcher.Close()\n\t\tcancel()\n\t\treturn nil, err\n\t}\n\n\tdm.scanDirectory(dm.dirPath)\n\n\tdm.wg.Add(1)\n\tgo dm.run()\n\n\treturn dm, nil\n}\n\n// Close stops the directory monitor and releases resources.\nfunc (dm *DirectoryMonitor) Close() {\n\tdm.cancel()\n\t_ = dm.watcher.Close()\n\tdm.wg.Wait()\n}\n\nfunc (dm *DirectoryMonitor) run() {\n\tdefer dm.wg.Done()\n\n\t// Debounce timer lives in this goroutine so shutdown via dm.wg.Wait() is clean.\n\tdebounceTimer := time.NewTimer(debounceInterval)\n\tdebounceTimer.Stop()\n\tdefer debounceTimer.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-dm.ctx.Done():\n\t\t\treturn\n\t\tcase event, ok := <-dm.watcher.Events:\n\t\t\tif !ok {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif dm.handleEvent(event) && !dm.debounceActive {\n\t\t\t\tdebounceTimer.Reset(debounceInterval)\n\t\t\t\tdm.debounceActive = true\n\t\t\t}\n\t\tcase <-debounceTimer.C:\n\t\t\tdm.flushPendingEvents()\n\t\tcase err, ok := <-dm.watcher.Errors:\n\t\t\tif !ok {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tdm.logger.Error(\"directory watcher error\", \"error\", err)\n\t\t}\n\t}\n}\n\nfunc (dm *DirectoryMonitor) addInitialWatches() error {\n\tif dm.recursive {\n\t\treturn filepath.WalkDir(dm.dirPath, func(path string, d os.DirEntry, err error) error {\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif !d.IsDir() {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\treturn dm.addDirectoryWatch(path)\n\t\t})\n\t}\n\n\treturn dm.addDirectoryWatch(dm.dirPath)\n}\n\nfunc (dm *DirectoryMonitor) addDirectoryWatch(path string) error {\n\tabspath := filepath.Clean(path)\n\n\tdm.mu.Lock()\n\tdefer dm.mu.Unlock()\n\n\tif _, ok := dm.watchedDirs[abspath]; ok {\n\t\treturn nil\n\t}\n\tdm.watchedDirs[abspath] = struct{}{}\n\n\tif err := dm.watcher.Add(abspath); err != nil {\n\t\tdelete(dm.watchedDirs, abspath)\n\t\treturn err\n\t}\n\n\tdm.logger.Debug(\"watching directory\", \"path\", abspath)\n\treturn nil\n}\n\nfunc (dm *DirectoryMonitor) removeDirectoryWatch(path string) {\n\tabspath := filepath.Clean(path)\n\n\tdm.mu.Lock()\n\tdefer dm.mu.Unlock()\n\n\tif _, ok := dm.watchedDirs[abspath]; !ok {\n\t\treturn\n\t}\n\tdelete(dm.watchedDirs, abspath)\n\n\tif err := dm.watcher.Remove(abspath); err != nil {\n\t\tdm.logger.Debug(\"remove directory watch\", \"path\", abspath, \"error\", err)\n\t}\n}\n\n// handleEvent processes a single fsnotify event. Returns true if pending events\n// were queued and the debounce timer should be reset.\nfunc (dm *DirectoryMonitor) handleEvent(event fsnotify.Event) bool {\n\tpath := filepath.Clean(event.Name)\n\tif path == \"\" {\n\t\treturn false\n\t}\n\n\tif dm.shouldSkipPath(path) {\n\t\treturn false\n\t}\n\n\tinfo, statErr := os.Stat(path)\n\tisDir := statErr == nil && info.IsDir()\n\n\t// Early pattern check for create/write only. Rename must pass through\n\t// to removal handling below, since the old path may be a tracked DB.\n\tif !isDir && event.Op&(fsnotify.Create|fsnotify.Write) != 0 {\n\t\tif !dm.matchesPattern(path) {\n\t\t\treturn false\n\t\t}\n\t}\n\n\tdm.mu.Lock()\n\t_, wasWatchedDir := dm.watchedDirs[path]\n\tdm.mu.Unlock()\n\n\tif isDir && event.Op&(fsnotify.Create|fsnotify.Rename) != 0 {\n\t\tif dm.recursive {\n\t\t\tif err := dm.addDirectoryWatch(path); err != nil {\n\t\t\t\tdm.logger.Error(\"add directory watch\", \"path\", path, \"error\", err)\n\t\t\t}\n\t\t\tdm.scanDirectory(path)\n\t\t}\n\t}\n\n\tif (isDir || wasWatchedDir) && event.Op&(fsnotify.Remove|fsnotify.Rename) != 0 {\n\t\tdm.removeDirectoryWatch(path)\n\t\tdm.removeDatabasesUnder(path)\n\t\treturn false\n\t}\n\n\tif isDir {\n\t\treturn false\n\t}\n\n\tif statErr != nil && !os.IsNotExist(statErr) {\n\t\tdm.logger.Debug(\"stat event path\", \"path\", path, \"error\", statErr)\n\t\treturn false\n\t}\n\n\tif event.Op&(fsnotify.Remove|fsnotify.Rename) != 0 {\n\t\tdm.removeDatabase(path)\n\t\treturn false\n\t}\n\n\tif event.Op&(fsnotify.Create|fsnotify.Write|fsnotify.Rename) != 0 {\n\t\tdm.pendingEvents[path] |= event.Op\n\t\treturn true\n\t}\n\n\treturn false\n}\n\n// flushPendingEvents processes all queued potential database events.\nfunc (dm *DirectoryMonitor) flushPendingEvents() {\n\tfor path := range dm.pendingEvents {\n\t\tselect {\n\t\tcase <-dm.ctx.Done():\n\t\t\treturn\n\t\tdefault:\n\t\t}\n\t\tdm.handlePotentialDatabase(path)\n\t}\n\tdm.pendingEvents = make(map[string]fsnotify.Op)\n\tdm.debounceActive = false\n}\n\nfunc (dm *DirectoryMonitor) handlePotentialDatabase(path string) {\n\tif !dm.matchesPattern(path) {\n\t\treturn\n\t}\n\n\tdm.mu.Lock()\n\tif _, exists := dm.dbs[path]; exists {\n\t\tdm.mu.Unlock()\n\t\treturn\n\t}\n\tdm.dbs[path] = nil\n\tdm.mu.Unlock()\n\n\tvar db *litestream.DB\n\tsuccess := false\n\tdefer func() {\n\t\tif !success {\n\t\t\tif db != nil {\n\t\t\t\t_ = dm.store.UnregisterDB(dm.ctx, db.Path())\n\t\t\t}\n\t\t\tdm.mu.Lock()\n\t\t\tdelete(dm.dbs, path)\n\t\t\tdm.mu.Unlock()\n\t\t}\n\t}()\n\n\tif !IsSQLiteDatabase(path) {\n\t\treturn\n\t}\n\n\tvar err error\n\tdb, err = newDBFromDirectoryEntry(dm.config, dm.dirPath, path)\n\tif err != nil {\n\t\tdm.logger.Error(\"configure database\", \"path\", path, \"error\", err)\n\t\treturn\n\t}\n\n\tif err := dm.store.RegisterDB(db); err != nil {\n\t\tdm.logger.Error(\"register database with store\", \"path\", path, \"error\", err)\n\t\treturn\n\t}\n\n\tdm.mu.Lock()\n\tdm.dbs[path] = db\n\tdm.mu.Unlock()\n\n\tsuccess = true\n\tdm.logger.Info(\"added database to replication\", \"path\", path)\n}\n\nfunc (dm *DirectoryMonitor) removeDatabase(path string) {\n\tdm.mu.Lock()\n\tdb := dm.dbs[path]\n\tdm.mu.Unlock()\n\n\tif db == nil {\n\t\treturn\n\t}\n\n\tif err := dm.store.UnregisterDB(dm.ctx, db.Path()); err != nil {\n\t\tdm.logger.Error(\"unregister database from store\", \"path\", path, \"error\", err)\n\t\treturn\n\t}\n\n\tdm.mu.Lock()\n\tdelete(dm.dbs, path)\n\tdm.mu.Unlock()\n\n\tdm.logger.Info(\"removed database from replication\", \"path\", path)\n}\n\nfunc (dm *DirectoryMonitor) removeDatabasesUnder(dir string) {\n\tprefix := dir + string(os.PathSeparator)\n\n\tdm.mu.Lock()\n\tvar toClose []*litestream.DB\n\tvar toClosePaths []string\n\tfor path, db := range dm.dbs {\n\t\tif path == dir || strings.HasPrefix(path, prefix) {\n\t\t\ttoClose = append(toClose, db)\n\t\t\ttoClosePaths = append(toClosePaths, path)\n\t\t}\n\t}\n\tdm.mu.Unlock()\n\n\tfor i, db := range toClose {\n\t\tif db == nil {\n\t\t\tdm.mu.Lock()\n\t\t\tdelete(dm.dbs, toClosePaths[i])\n\t\t\tdm.mu.Unlock()\n\t\t\tcontinue\n\t\t}\n\t\tif err := dm.store.UnregisterDB(dm.ctx, db.Path()); err != nil {\n\t\t\tdm.logger.Error(\"unregister database from store\", \"path\", db.Path(), \"error\", err)\n\t\t\tcontinue\n\t\t}\n\n\t\tdm.mu.Lock()\n\t\tdelete(dm.dbs, toClosePaths[i])\n\t\tdm.mu.Unlock()\n\t}\n}\n\nfunc (dm *DirectoryMonitor) matchesPattern(path string) bool {\n\tmatched, err := filepath.Match(dm.pattern, filepath.Base(path))\n\tif err != nil {\n\t\tdm.logger.Error(\"pattern match failed\", \"pattern\", dm.pattern, \"path\", path, \"error\", err)\n\t\treturn false\n\t}\n\treturn matched\n}\n\n// shouldSkipPath returns true for SQLite auxiliary files (WAL, SHM, journal)\n// that generate many events but are never databases themselves.\nfunc (dm *DirectoryMonitor) shouldSkipPath(path string) bool {\n\tbase := filepath.Base(path)\n\treturn strings.HasSuffix(base, \"-wal\") ||\n\t\tstrings.HasSuffix(base, \"-shm\") ||\n\t\tstrings.HasSuffix(base, \"-journal\")\n}\n\n// scanDirectory discovers pre-existing databases and is also called when new\n// directories appear to close the race window between watch registration and file creation.\nfunc (dm *DirectoryMonitor) scanDirectory(dir string) {\n\tif !dm.recursive {\n\t\tentries, err := os.ReadDir(dir)\n\t\tif err != nil {\n\t\t\tif !os.IsNotExist(err) {\n\t\t\t\tdm.logger.Debug(\"read directory\", \"path\", dir, \"error\", err)\n\t\t\t}\n\t\t\treturn\n\t\t}\n\n\t\tfor _, entry := range entries {\n\t\t\tif entry.IsDir() {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tpath := filepath.Join(dir, entry.Name())\n\t\t\tdm.handlePotentialDatabase(path)\n\t\t}\n\t\treturn\n\t}\n\n\terr := filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error {\n\t\tif err != nil {\n\t\t\tif os.IsNotExist(err) {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tdm.logger.Debug(\"scan directory entry\", \"path\", path, \"error\", err)\n\t\t\treturn nil\n\t\t}\n\n\t\tif d.IsDir() {\n\t\t\tif path != dir {\n\t\t\t\tif err := dm.addDirectoryWatch(path); err != nil {\n\t\t\t\t\tdm.logger.Error(\"add directory watch\", \"path\", path, \"error\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\n\t\tdm.handlePotentialDatabase(path)\n\t\treturn nil\n\t})\n\tif err != nil && !os.IsNotExist(err) {\n\t\tdm.logger.Debug(\"scan directory\", \"path\", dir, \"error\", err)\n\t}\n}\n" }, { "path": "cmd/litestream/directory_watcher_stress_test.go", "content": "//go:build stress\n\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nvar dbCounts = []int{100, 250, 500, 1000, 2500}\n\nfunc TestDirectoryWatcher_PreCreated(t *testing.T) {\n\tfor _, count := range dbCounts {\n\t\tcount := count\n\t\tt.Run(fmt.Sprintf(\"%d\", count), func(t *testing.T) {\n\t\t\tdbDir := t.TempDir()\n\t\t\treplicaDir := t.TempDir()\n\n\t\t\tt.Logf(\"Creating %d databases...\", count)\n\t\t\tdbs := createTestDatabases(t, dbDir, count)\n\t\t\tdefer closeTestDatabases(dbs)\n\n\t\t\tstore, monitors := startDirectoryMonitor(t, dbDir, replicaDir)\n\t\t\tdefer stopDirectoryMonitor(store, monitors)\n\n\t\t\ttimeout := 3*time.Minute + time.Duration(count/100)*time.Minute\n\t\t\tt.Logf(\"Waiting for detection (timeout: %v)...\", timeout)\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), timeout)\n\t\t\tdefer cancel()\n\n\t\t\tif err := waitForDBCount(ctx, store, count); err != nil {\n\t\t\t\tt.Fatalf(\"Failed to detect all databases: %v (got %d, expected %d)\",\n\t\t\t\t\terr, len(store.DBs()), count)\n\t\t\t}\n\n\t\t\tt.Logf(\"All %d databases detected successfully\", count)\n\t\t})\n\t}\n}\n\nfunc TestDirectoryWatcher_DynamicScaling(t *testing.T) {\n\tfor _, finalCount := range dbCounts {\n\t\tfinalCount := finalCount\n\t\tt.Run(fmt.Sprintf(\"%d\", finalCount), func(t *testing.T) {\n\t\t\tbatchSize := finalCount / 10\n\t\t\tif batchSize < 10 {\n\t\t\t\tbatchSize = 10\n\t\t\t}\n\t\t\tbatchTimeout := 60*time.Second + time.Duration(batchSize/50)*30*time.Second\n\n\t\t\tdbDir := t.TempDir()\n\t\t\treplicaDir := t.TempDir()\n\n\t\t\tinitialDBs := batchSize\n\t\t\tt.Logf(\"Creating initial %d databases...\", initialDBs)\n\t\t\tdbs := createTestDatabases(t, dbDir, initialDBs)\n\n\t\t\tstore, monitors := startDirectoryMonitor(t, dbDir, replicaDir)\n\t\t\tdefer stopDirectoryMonitor(store, monitors)\n\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), batchTimeout)\n\t\t\tif err := waitForDBCount(ctx, store, initialDBs); err != nil {\n\t\t\t\tcancel()\n\t\t\t\tcloseTestDatabases(dbs)\n\t\t\t\tt.Fatalf(\"Failed to detect initial databases: %v\", err)\n\t\t\t}\n\t\t\tcancel()\n\t\t\tt.Logf(\"Initial %d databases detected\", initialDBs)\n\n\t\t\tcurrentCount := initialDBs\n\t\t\tfor currentCount < finalCount {\n\t\t\t\taddCount := batchSize\n\t\t\t\tif currentCount+addCount > finalCount {\n\t\t\t\t\taddCount = finalCount - currentCount\n\t\t\t\t}\n\n\t\t\t\tt.Logf(\"Adding batch: %d -> %d databases\", currentCount, currentCount+addCount)\n\t\t\t\tnewDBs := createTestDatabasesBatch(t, dbDir, currentCount, addCount)\n\t\t\t\tdbs = append(dbs, newDBs...)\n\n\t\t\t\tctx, cancel := context.WithTimeout(context.Background(), batchTimeout)\n\t\t\t\texpectedCount := currentCount + addCount\n\t\t\t\tif err := waitForDBCount(ctx, store, expectedCount); err != nil {\n\t\t\t\t\tcancel()\n\t\t\t\t\tcloseTestDatabases(dbs)\n\t\t\t\t\tt.Fatalf(\"Failed to detect batch (expected %d, got %d): %v\",\n\t\t\t\t\t\texpectedCount, len(store.DBs()), err)\n\t\t\t\t}\n\t\t\t\tcancel()\n\n\t\t\t\tcurrentCount += addCount\n\t\t\t\ttime.Sleep(500 * time.Millisecond)\n\t\t\t}\n\n\t\t\tcloseTestDatabases(dbs)\n\t\t\tt.Logf(\"Successfully scaled to %d databases\", finalCount)\n\t\t})\n\t}\n}\n\nfunc TestDirectoryWatcher_ConcurrentWrites(t *testing.T) {\n\tfor _, count := range dbCounts {\n\t\tcount := count\n\t\tt.Run(fmt.Sprintf(\"%d\", count), func(t *testing.T) {\n\t\t\tconst writeDuration = 10 * time.Second\n\t\t\tconst writesPerDBPerSec = 5\n\n\t\t\tdbDir := t.TempDir()\n\t\t\treplicaDir := t.TempDir()\n\n\t\t\tt.Logf(\"Creating %d databases...\", count)\n\t\t\tdbs := createTestDatabases(t, dbDir, count)\n\t\t\tdefer closeTestDatabases(dbs)\n\n\t\t\tstore, monitors := startDirectoryMonitor(t, dbDir, replicaDir)\n\t\t\tdefer stopDirectoryMonitor(store, monitors)\n\n\t\t\ttimeout := 3*time.Minute + time.Duration(count/100)*time.Minute\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), timeout)\n\t\t\tif err := waitForDBCount(ctx, store, count); err != nil {\n\t\t\t\tcancel()\n\t\t\t\tt.Fatalf(\"Failed to detect databases: %v\", err)\n\t\t\t}\n\t\t\tcancel()\n\n\t\t\tt.Logf(\"Starting concurrent writes for %v...\", writeDuration)\n\t\t\tvar totalWrites int64\n\t\t\tvar wg sync.WaitGroup\n\n\t\t\twriteCtx, writeCancel := context.WithTimeout(context.Background(), writeDuration)\n\t\t\tdefer writeCancel()\n\n\t\t\tfor i, db := range dbs {\n\t\t\t\twg.Add(1)\n\t\t\t\tgo func(idx int, db *sql.DB) {\n\t\t\t\t\tdefer wg.Done()\n\t\t\t\t\tticker := time.NewTicker(time.Second / time.Duration(writesPerDBPerSec))\n\t\t\t\t\tdefer ticker.Stop()\n\n\t\t\t\t\tfor {\n\t\t\t\t\t\tselect {\n\t\t\t\t\t\tcase <-writeCtx.Done():\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\tcase <-ticker.C:\n\t\t\t\t\t\t\t_, err := db.Exec(\"INSERT INTO data (value) VALUES (?)\",\n\t\t\t\t\t\t\t\tfmt.Sprintf(\"db%d-%d\", idx, time.Now().UnixNano()))\n\t\t\t\t\t\t\tif err == nil {\n\t\t\t\t\t\t\t\tatomic.AddInt64(&totalWrites, 1)\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}(i, db)\n\t\t\t}\n\n\t\t\twg.Wait()\n\t\t\tt.Logf(\"Completed %d total writes across %d databases\", totalWrites, count)\n\n\t\t\tif totalWrites == 0 {\n\t\t\t\tt.Fatal(\"Expected at least some writes to succeed\")\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc createTestDatabases(t *testing.T, dir string, count int) []*sql.DB {\n\treturn createTestDatabasesBatch(t, dir, 0, count)\n}\n\nfunc createTestDatabasesBatch(t *testing.T, dir string, startIdx, count int) []*sql.DB {\n\tt.Helper()\n\tdbs := make([]*sql.DB, 0, count)\n\n\tfor i := 0; i < count; i++ {\n\t\tidx := startIdx + i\n\t\tdbPath := filepath.Join(dir, fmt.Sprintf(\"test_%04d.db\", idx))\n\n\t\tdb, err := sql.Open(\"sqlite\", dbPath)\n\t\tif err != nil {\n\t\t\tcloseTestDatabases(dbs)\n\t\t\tt.Fatalf(\"Failed to open database %d: %v\", idx, err)\n\t\t}\n\n\t\t_, err = db.Exec(`\n\t\t\tPRAGMA journal_mode=WAL;\n\t\t\tCREATE TABLE IF NOT EXISTS data (\n\t\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\t\tvalue TEXT,\n\t\t\t\tcreated_at DATETIME DEFAULT CURRENT_TIMESTAMP\n\t\t\t);\n\t\t`)\n\t\tif err != nil {\n\t\t\tdb.Close()\n\t\t\tcloseTestDatabases(dbs)\n\t\t\tt.Fatalf(\"Failed to initialize database %d: %v\", idx, err)\n\t\t}\n\n\t\tdbs = append(dbs, db)\n\t}\n\n\treturn dbs\n}\n\nfunc closeTestDatabases(dbs []*sql.DB) {\n\tfor _, db := range dbs {\n\t\tif db != nil {\n\t\t\tdb.Close()\n\t\t}\n\t}\n}\n\nfunc startDirectoryMonitor(t *testing.T, dbDir, replicaDir string) (*litestream.Store, []*DirectoryMonitor) {\n\tt.Helper()\n\n\tsyncInterval := time.Second\n\tdbConfig := &DBConfig{\n\t\tDir: dbDir,\n\t\tPattern: \"*.db\",\n\t\tRecursive: false,\n\t\tWatch: true,\n\t\tReplica: &ReplicaConfig{\n\t\t\tType: \"file\",\n\t\t\tPath: replicaDir,\n\t\t\tReplicaSettings: ReplicaSettings{\n\t\t\t\tSyncInterval: &syncInterval,\n\t\t\t},\n\t\t},\n\t}\n\n\tdbs, err := NewDBsFromDirectoryConfig(dbConfig)\n\tif err != nil && !strings.Contains(err.Error(), \"no SQLite databases found\") {\n\t\tt.Fatalf(\"Failed to create DBs from directory config: %v\", err)\n\t}\n\n\tstore := litestream.NewStore(dbs, litestream.DefaultCompactionLevels)\n\tif err := store.Open(context.Background()); err != nil {\n\t\tt.Fatalf(\"Failed to open store: %v\", err)\n\t}\n\n\tmonitor, err := NewDirectoryMonitor(context.Background(), store, dbConfig, dbs)\n\tif err != nil {\n\t\tstore.Close(context.Background())\n\t\tt.Fatalf(\"Failed to create directory monitor: %v\", err)\n\t}\n\n\treturn store, []*DirectoryMonitor{monitor}\n}\n\nfunc stopDirectoryMonitor(store *litestream.Store, monitors []*DirectoryMonitor) {\n\tfor _, m := range monitors {\n\t\tm.Close()\n\t}\n\tstore.Close(context.Background())\n}\n\nfunc waitForDBCount(ctx context.Context, store *litestream.Store, expected int) error {\n\tticker := time.NewTicker(100 * time.Millisecond)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn ctx.Err()\n\t\tcase <-ticker.C:\n\t\t\tif len(store.DBs()) >= expected {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n}\n" }, { "path": "cmd/litestream/directory_watcher_test.go", "content": "package main\n\nimport (\n\t\"testing\"\n\n\t\"github.com/fsnotify/fsnotify\"\n)\n\nfunc TestDirectoryMonitor_shouldSkipPath(t *testing.T) {\n\tdm := &DirectoryMonitor{}\n\n\ttests := []struct {\n\t\tname string\n\t\tpath string\n\t\texpected bool\n\t}{\n\t\t// Should skip SQLite auxiliary files\n\t\t{\"skip WAL file\", \"/path/to/db.sqlite-wal\", true},\n\t\t{\"skip SHM file\", \"/path/to/db.sqlite-shm\", true},\n\t\t{\"skip journal file\", \"/path/to/db.sqlite-journal\", true},\n\t\t{\"skip WAL file simple\", \"test.db-wal\", true},\n\t\t{\"skip SHM file simple\", \"test.db-shm\", true},\n\t\t{\"skip journal file simple\", \"test.db-journal\", true},\n\n\t\t// Should not skip actual database files\n\t\t{\"allow .db file\", \"/path/to/test.db\", false},\n\t\t{\"allow .sqlite file\", \"/path/to/test.sqlite\", false},\n\t\t{\"allow .sqlite3 file\", \"/path/to/test.sqlite3\", false},\n\t\t{\"allow arbitrary file\", \"/path/to/data.dat\", false},\n\n\t\t// Edge cases\n\t\t{\"allow file ending in wal (not -wal)\", \"/path/to/withdrawal\", false},\n\t\t{\"allow file ending in shm (not -shm)\", \"/path/to/rhythm\", false},\n\t\t{\"allow file ending in journal (not -journal)\", \"/path/to/myjournal\", false},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot := dm.shouldSkipPath(tt.path)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"shouldSkipPath(%q) = %v, want %v\", tt.path, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDirectoryMonitor_matchesPattern(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpattern string\n\t\tpath string\n\t\texpected bool\n\t}{\n\t\t// *.db pattern\n\t\t{\"matches .db\", \"*.db\", \"/path/to/test.db\", true},\n\t\t{\"no match .sqlite\", \"*.db\", \"/path/to/test.sqlite\", false},\n\t\t{\"no match .db.backup\", \"*.db\", \"/path/to/test.db.backup\", false},\n\n\t\t// *.sqlite pattern\n\t\t{\"matches .sqlite\", \"*.sqlite\", \"/path/to/test.sqlite\", true},\n\t\t{\"no match .db with sqlite pattern\", \"*.sqlite\", \"/path/to/test.db\", false},\n\n\t\t// * (match all) pattern\n\t\t{\"match all pattern\", \"*\", \"/path/to/anything.db\", true},\n\t\t{\"match all pattern sqlite\", \"*\", \"/path/to/anything.sqlite\", true},\n\n\t\t// prefix patterns\n\t\t{\"prefix match\", \"app_*.db\", \"/path/to/app_users.db\", true},\n\t\t{\"prefix no match\", \"app_*.db\", \"/path/to/users.db\", false},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tdm := &DirectoryMonitor{pattern: tt.pattern}\n\t\t\tgot := dm.matchesPattern(tt.path)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"matchesPattern(%q) with pattern %q = %v, want %v\", tt.path, tt.pattern, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDirectoryMonitor_pendingEvents(t *testing.T) {\n\tt.Run(\"coalesces multiple events for same path\", func(t *testing.T) {\n\t\tdm := &DirectoryMonitor{\n\t\t\tpendingEvents: make(map[string]fsnotify.Op),\n\t\t}\n\n\t\tdm.pendingEvents[\"/path/to/test.db\"] |= fsnotify.Create\n\t\tdm.pendingEvents[\"/path/to/test.db\"] |= fsnotify.Write\n\t\tdm.pendingEvents[\"/path/to/test.db\"] |= fsnotify.Write\n\n\t\tif len(dm.pendingEvents) != 1 {\n\t\t\tt.Errorf(\"expected 1 pending event, got %d\", len(dm.pendingEvents))\n\t\t}\n\n\t\top := dm.pendingEvents[\"/path/to/test.db\"]\n\t\tif op&fsnotify.Create == 0 {\n\t\t\tt.Error(\"expected Create op to be set\")\n\t\t}\n\t\tif op&fsnotify.Write == 0 {\n\t\t\tt.Error(\"expected Write op to be set\")\n\t\t}\n\t})\n\n\tt.Run(\"queues different paths separately\", func(t *testing.T) {\n\t\tdm := &DirectoryMonitor{\n\t\t\tpendingEvents: make(map[string]fsnotify.Op),\n\t\t}\n\n\t\tdm.pendingEvents[\"/path/to/db1.db\"] |= fsnotify.Create\n\t\tdm.pendingEvents[\"/path/to/db2.db\"] |= fsnotify.Create\n\t\tdm.pendingEvents[\"/path/to/db3.db\"] |= fsnotify.Write\n\n\t\tif len(dm.pendingEvents) != 3 {\n\t\t\tt.Errorf(\"expected 3 pending events, got %d\", len(dm.pendingEvents))\n\t\t}\n\t})\n}\n" }, { "path": "cmd/litestream/info.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\n// InfoCommand represents the command to show daemon information.\ntype InfoCommand struct{}\n\n// Run executes the info command.\nfunc (c *InfoCommand) Run(_ context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-info\", flag.ContinueOnError)\n\tsocketPath := fs.String(\"socket\", \"/var/run/litestream.sock\", \"control socket path\")\n\ttimeout := fs.Int(\"timeout\", 10, \"timeout in seconds\")\n\tjsonOutput := fs.Bool(\"json\", false, \"output raw JSON\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif fs.NArg() > 0 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\n\tif *timeout <= 0 {\n\t\treturn fmt.Errorf(\"timeout must be greater than 0\")\n\t}\n\n\tclientTimeout := time.Duration(*timeout) * time.Second\n\tclient := &http.Client{\n\t\tTimeout: clientTimeout,\n\t\tTransport: &http.Transport{\n\t\t\tDialContext: func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\t\treturn net.DialTimeout(\"unix\", *socketPath, clientTimeout)\n\t\t\t},\n\t\t},\n\t}\n\n\tresp, err := client.Get(\"http://localhost/info\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to connect to control socket: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read response: %w\", err)\n\t}\n\n\tif resp.StatusCode != http.StatusOK {\n\t\tvar errResp litestream.ErrorResponse\n\t\tif err := json.Unmarshal(body, &errResp); err == nil && errResp.Error != \"\" {\n\t\t\treturn fmt.Errorf(\"info failed: %s\", errResp.Error)\n\t\t}\n\t\treturn fmt.Errorf(\"info failed: %s\", string(body))\n\t}\n\n\tvar result litestream.InfoResponse\n\tif err := json.Unmarshal(body, &result); err != nil {\n\t\treturn fmt.Errorf(\"failed to parse response: %w\", err)\n\t}\n\n\tif *jsonOutput {\n\t\toutput, err := json.MarshalIndent(result, \"\", \" \")\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to format response: %w\", err)\n\t\t}\n\t\tfmt.Println(string(output))\n\t} else {\n\t\tuptime := time.Duration(result.UptimeSeconds) * time.Second\n\t\tfmt.Printf(\"Litestream %s\\n\", result.Version)\n\t\tfmt.Printf(\" PID: %d\\n\", result.PID)\n\t\tfmt.Printf(\" Uptime: %s\\n\", uptime)\n\t\tfmt.Printf(\" Started at: %s\\n\", result.StartedAt.Format(time.RFC3339))\n\t\tfmt.Printf(\" Databases: %d\\n\", result.DatabaseCount)\n\t}\n\n\treturn nil\n}\n\n// Usage prints the help text for the info command.\nfunc (c *InfoCommand) Usage() {\n\tfmt.Println(`\nusage: litestream info [OPTIONS]\n\nShow daemon information from a running Litestream instance.\n\nOptions:\n -json\n Output raw JSON instead of human-readable text.\n\n -socket PATH\n Path to control socket (default: /var/run/litestream.sock).\n\n -timeout SECONDS\n Maximum time to wait in seconds (default: 10).\n`[1:])\n}\n" }, { "path": "cmd/litestream/info_test.go", "content": "package main_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"sync/atomic\"\n\t\"testing\"\n\n\t\"github.com/benbjohnson/litestream\"\n\tmain \"github.com/benbjohnson/litestream/cmd/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nvar testSocketCounter uint64\n\nfunc testSocketPath(t *testing.T) string {\n\tt.Helper()\n\tn := atomic.AddUint64(&testSocketCounter, 1)\n\tpath := fmt.Sprintf(\"/tmp/ls-cmd-test-%d.sock\", n)\n\tt.Cleanup(func() { os.Remove(path) })\n\treturn path\n}\n\nfunc TestInfoCommand_Run(t *testing.T) {\n\tt.Run(\"TooManyArguments\", func(t *testing.T) {\n\t\tcmd := &main.InfoCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"extra-arg\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for too many arguments\")\n\t\t}\n\t\tif err.Error() != \"too many arguments\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ConnectionError\", func(t *testing.T) {\n\t\tcmd := &main.InfoCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/nonexistent/socket.sock\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for socket connection failure\")\n\t\t}\n\t})\n\n\tt.Run(\"CustomTimeout\", func(t *testing.T) {\n\t\tcmd := &main.InfoCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/nonexistent/socket.sock\", \"-timeout\", \"1\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for socket connection failure\")\n\t\t}\n\t})\n\n\tt.Run(\"InvalidTimeoutZero\", func(t *testing.T) {\n\t\tcmd := &main.InfoCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-timeout\", \"0\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for zero timeout\")\n\t\t}\n\t\tif err.Error() != \"timeout must be greater than 0\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"InvalidTimeoutNegative\", func(t *testing.T) {\n\t\tcmd := &main.InfoCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-timeout\", \"-1\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for negative timeout\")\n\t\t}\n\t\tif err.Error() != \"timeout must be greater than 0\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(context.Background())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tserver.Version = \"v1.0.0-test\"\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\tcmd := &main.InfoCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", server.SocketPath})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"JSONOutput\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(context.Background())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tserver.Version = \"v1.0.0-test\"\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\tcmd := &main.InfoCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", server.SocketPath, \"-json\"})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n}\n" }, { "path": "cmd/litestream/list.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\n// ListCommand represents the command to list all managed databases.\ntype ListCommand struct{}\n\n// Run executes the list command.\nfunc (c *ListCommand) Run(_ context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-list\", flag.ContinueOnError)\n\tsocketPath := fs.String(\"socket\", \"/var/run/litestream.sock\", \"control socket path\")\n\ttimeout := fs.Int(\"timeout\", 10, \"timeout in seconds\")\n\tjsonOutput := fs.Bool(\"json\", false, \"output raw JSON\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif fs.NArg() > 0 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\n\tif *timeout <= 0 {\n\t\treturn fmt.Errorf(\"timeout must be greater than 0\")\n\t}\n\n\tclientTimeout := time.Duration(*timeout) * time.Second\n\tclient := &http.Client{\n\t\tTimeout: clientTimeout,\n\t\tTransport: &http.Transport{\n\t\t\tDialContext: func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\t\treturn net.DialTimeout(\"unix\", *socketPath, clientTimeout)\n\t\t\t},\n\t\t},\n\t}\n\n\tresp, err := client.Get(\"http://localhost/list\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to connect to control socket: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read response: %w\", err)\n\t}\n\n\tif resp.StatusCode != http.StatusOK {\n\t\tvar errResp litestream.ErrorResponse\n\t\tif err := json.Unmarshal(body, &errResp); err == nil && errResp.Error != \"\" {\n\t\t\treturn fmt.Errorf(\"list failed: %s\", errResp.Error)\n\t\t}\n\t\treturn fmt.Errorf(\"list failed: %s\", string(body))\n\t}\n\n\tvar result litestream.ListResponse\n\tif err := json.Unmarshal(body, &result); err != nil {\n\t\treturn fmt.Errorf(\"failed to parse response: %w\", err)\n\t}\n\n\tif *jsonOutput {\n\t\toutput, err := json.MarshalIndent(result, \"\", \" \")\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to format response: %w\", err)\n\t\t}\n\t\tfmt.Println(string(output))\n\t} else {\n\t\tif len(result.Databases) == 0 {\n\t\t\tfmt.Println(\"No databases configured\")\n\t\t} else {\n\t\t\tfor _, db := range result.Databases {\n\t\t\t\tsyncInfo := \"never\"\n\t\t\t\tif db.LastSyncAt != nil {\n\t\t\t\t\tsyncInfo = db.LastSyncAt.Format(time.RFC3339)\n\t\t\t\t}\n\t\t\t\tfmt.Printf(\"%s [%s] (last sync: %s)\\n\", db.Path, db.Status, syncInfo)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// Usage prints the help text for the list command.\nfunc (c *ListCommand) Usage() {\n\tfmt.Println(`\nusage: litestream list [OPTIONS]\n\nList all managed databases from a running daemon.\n\nOptions:\n -json\n Output raw JSON instead of human-readable text.\n\n -socket PATH\n Path to control socket (default: /var/run/litestream.sock).\n\n -timeout SECONDS\n Maximum time to wait in seconds (default: 10).\n`[1:])\n}\n" }, { "path": "cmd/litestream/list_test.go", "content": "package main_test\n\nimport (\n\t\"context\"\n\t\"testing\"\n\n\t\"github.com/benbjohnson/litestream\"\n\tmain \"github.com/benbjohnson/litestream/cmd/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestListCommand_Run(t *testing.T) {\n\tt.Run(\"TooManyArguments\", func(t *testing.T) {\n\t\tcmd := &main.ListCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"extra-arg\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for too many arguments\")\n\t\t}\n\t\tif err.Error() != \"too many arguments\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ConnectionError\", func(t *testing.T) {\n\t\tcmd := &main.ListCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/nonexistent/socket.sock\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for socket connection failure\")\n\t\t}\n\t})\n\n\tt.Run(\"CustomTimeout\", func(t *testing.T) {\n\t\tcmd := &main.ListCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/nonexistent/socket.sock\", \"-timeout\", \"1\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for socket connection failure\")\n\t\t}\n\t})\n\n\tt.Run(\"InvalidTimeoutZero\", func(t *testing.T) {\n\t\tcmd := &main.ListCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-timeout\", \"0\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for zero timeout\")\n\t\t}\n\t\tif err.Error() != \"timeout must be greater than 0\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"InvalidTimeoutNegative\", func(t *testing.T) {\n\t\tcmd := &main.ListCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-timeout\", \"-1\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for negative timeout\")\n\t\t}\n\t\tif err.Error() != \"timeout must be greater than 0\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(context.Background())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\tcmd := &main.ListCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", server.SocketPath})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"SuccessEmpty\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(context.Background())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\tcmd := &main.ListCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", server.SocketPath})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"JSONOutput\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(context.Background())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\tcmd := &main.ListCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", server.SocketPath, \"-json\"})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n}\n" }, { "path": "cmd/litestream/ltx.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"flag\"\n\t\"fmt\"\n\t\"os\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\n// LTXCommand represents a command to list LTX files for a database.\ntype LTXCommand struct{}\n\n// Run executes the command.\nfunc (c *LTXCommand) Run(ctx context.Context, args []string) (err error) {\n\tfs := flag.NewFlagSet(\"litestream-ltx\", flag.ContinueOnError)\n\tconfigPath, noExpandEnv := registerConfigFlag(fs)\n\tvar level levelVar\n\tfs.Var(&level, \"level\", \"compaction level (0-9 or \\\"all\\\")\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t} else if fs.NArg() == 0 || fs.Arg(0) == \"\" {\n\t\treturn fmt.Errorf(\"database path required\")\n\t} else if fs.NArg() > 1 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\n\tvar r *litestream.Replica\n\tif litestream.IsURL(fs.Arg(0)) {\n\t\tif *configPath != \"\" {\n\t\t\treturn fmt.Errorf(\"cannot specify a replica URL and the -config flag\")\n\t\t}\n\t\tif r, err = NewReplicaFromConfig(&ReplicaConfig{URL: fs.Arg(0)}, nil); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tinitLog(os.Stdout, \"INFO\", \"text\")\n\t} else {\n\t\tif *configPath == \"\" {\n\t\t\t*configPath = DefaultConfigPath()\n\t\t}\n\n\t\t// Load configuration.\n\t\tconfig, err := ReadConfigFile(*configPath, !*noExpandEnv)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Lookup database from configuration file by path.\n\t\tpath, err := expand(fs.Arg(0))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdbc := config.DBConfig(path)\n\t\tif dbc == nil {\n\t\t\treturn fmt.Errorf(\"database not found in config: %s\", path)\n\t\t}\n\n\t\tdb, err := NewDBFromConfig(dbc)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t} else if db.Replica == nil {\n\t\t\treturn fmt.Errorf(\"database has no replica\")\n\t\t}\n\t\tr = db.Replica\n\t}\n\n\t// List LTX files.\n\tw := tabwriter.NewWriter(os.Stdout, 0, 8, 2, ' ', 0)\n\tdefer w.Flush()\n\n\tfmt.Fprintln(w, \"level\\tmin_txid\\tmax_txid\\tsize\\tcreated\")\n\n\t// Determine which levels to iterate.\n\tvar levels []int\n\tif int(level) == levelAll {\n\t\tfor lvl := 0; lvl <= litestream.SnapshotLevel; lvl++ {\n\t\t\tlevels = append(levels, lvl)\n\t\t}\n\t} else {\n\t\tlevels = []int{int(level)}\n\t}\n\n\tfor _, lvl := range levels {\n\t\titr, err := r.Client.LTXFiles(ctx, lvl, 0, false)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tfor itr.Next() {\n\t\t\tinfo := itr.Item()\n\t\t\tfmt.Fprintf(w, \"%d\\t%s\\t%s\\t%d\\t%s\\n\",\n\t\t\t\tlvl,\n\t\t\t\tinfo.MinTXID,\n\t\t\t\tinfo.MaxTXID,\n\t\t\t\tinfo.Size,\n\t\t\t\tinfo.CreatedAt.Format(time.RFC3339),\n\t\t\t)\n\t\t}\n\t\tif err := itr.Close(); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\n// Usage prints the help screen to STDOUT.\nfunc (c *LTXCommand) Usage() {\n\tfmt.Printf(`\nThe ltx command lists all LTX files available for a database.\n\nUsage:\n\n\tlitestream ltx [arguments] DB_PATH\n\n\tlitestream ltx [arguments] REPLICA_URL\n\nArguments:\n\n\t-config PATH\n\t Specifies the configuration file.\n\t Defaults to %s\n\n\t-no-expand-env\n\t Disables environment variable expansion in configuration file.\n\n\t-replica NAME\n\t Optional, filter by a specific replica.\n\n\t-level LEVEL\n\t Compaction level to list (0-9 or \"all\").\n\t Defaults to 0.\n\nExamples:\n\n\t# List all LTX files for a database.\n\t$ litestream ltx /path/to/db\n\n\t# List all LTX files on S3\n\t$ litestream ltx -replica s3 /path/to/db\n\n\t# List all LTX files for replica URL.\n\t$ litestream ltx s3://mybkt/db\n\n\t# List LTX files at snapshot level (level 9).\n\t$ litestream ltx -level 9 /path/to/db\n\n\t# List LTX files across all compaction levels.\n\t$ litestream ltx -level all /path/to/db\n\n`[1:],\n\t\tDefaultConfigPath(),\n\t)\n}\n" }, { "path": "cmd/litestream/ltx_test.go", "content": "package main\n\nimport (\n\t\"testing\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nfunc TestTXIDVarParsing(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinput string\n\t\twant ltx.TXID\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"valid hex string\",\n\t\t\tinput: \"0000000000000002\",\n\t\t\twant: ltx.TXID(2),\n\t\t},\n\t\t{\n\t\t\tname: \"valid hex string with letters\",\n\t\t\tinput: \"00000000000000ff\",\n\t\t\twant: ltx.TXID(255),\n\t\t},\n\t\t{\n\t\t\tname: \"uppercase hex\",\n\t\t\tinput: \"00000000000000FF\",\n\t\t\twant: ltx.TXID(255),\n\t\t},\n\t\t{\n\t\t\tname: \"invalid - too short\",\n\t\t\tinput: \"ff\",\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid - too long\",\n\t\t\tinput: \"00000000000000001\",\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid - non-hex characters\",\n\t\t\tinput: \"000000000000000g\",\n\t\t\twantErr: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar v txidVar\n\t\t\terr := v.Set(tt.input)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"Set(%q) = nil, want error\", tt.input)\n\t\t\t\t}\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"Set(%q) error = %v, want nil\", tt.input, err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif ltx.TXID(v) != tt.want {\n\t\t\t\tt.Errorf(\"Set(%q) = %v, want %v\", tt.input, ltx.TXID(v), tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestTXIDVarString(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tvalue ltx.TXID\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"zero\",\n\t\t\tvalue: ltx.TXID(0),\n\t\t\twant: \"0000000000000000\",\n\t\t},\n\t\t{\n\t\t\tname: \"small number\",\n\t\t\tvalue: ltx.TXID(2),\n\t\t\twant: \"0000000000000002\",\n\t\t},\n\t\t{\n\t\t\tname: \"larger number\",\n\t\t\tvalue: ltx.TXID(255),\n\t\t\twant: \"00000000000000ff\",\n\t\t},\n\t\t{\n\t\t\tname: \"max value\",\n\t\t\tvalue: ltx.TXID(^uint64(0)),\n\t\t\twant: \"ffffffffffffffff\",\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tv := txidVar(tt.value)\n\t\t\tif got := v.String(); got != tt.want {\n\t\t\t\tt.Errorf(\"String() = %q, want %q\", got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestLevelVarParsing(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinput string\n\t\twant int\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"level 0\",\n\t\t\tinput: \"0\",\n\t\t\twant: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"level 5\",\n\t\t\tinput: \"5\",\n\t\t\twant: 5,\n\t\t},\n\t\t{\n\t\t\tname: \"level 9 (snapshot level)\",\n\t\t\tinput: \"9\",\n\t\t\twant: litestream.SnapshotLevel,\n\t\t},\n\t\t{\n\t\t\tname: \"all levels\",\n\t\t\tinput: \"all\",\n\t\t\twant: levelAll,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid - negative\",\n\t\t\tinput: \"-1\",\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid - too high\",\n\t\t\tinput: \"10\",\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid - non-numeric\",\n\t\t\tinput: \"abc\",\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid - empty\",\n\t\t\tinput: \"\",\n\t\t\twantErr: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar v levelVar\n\t\t\terr := v.Set(tt.input)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"Set(%q) = nil, want error\", tt.input)\n\t\t\t\t}\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"Set(%q) error = %v, want nil\", tt.input, err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif int(v) != tt.want {\n\t\t\t\tt.Errorf(\"Set(%q) = %v, want %v\", tt.input, int(v), tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestLevelVarString(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tvalue int\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"level 0\",\n\t\t\tvalue: 0,\n\t\t\twant: \"0\",\n\t\t},\n\t\t{\n\t\t\tname: \"level 9\",\n\t\t\tvalue: 9,\n\t\t\twant: \"9\",\n\t\t},\n\t\t{\n\t\t\tname: \"all levels\",\n\t\t\tvalue: levelAll,\n\t\t\twant: \"all\",\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tv := levelVar(tt.value)\n\t\t\tif got := v.String(); got != tt.want {\n\t\t\t\tt.Errorf(\"String() = %q, want %q\", got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "cmd/litestream/main.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"math\"\n\t\"net/url\"\n\t\"os\"\n\t\"os/user\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/dustin/go-humanize\"\n\t\"github.com/superfly/ltx\"\n\t_ \"golang.org/x/crypto/x509roots/fallback\"\n\t\"gopkg.in/yaml.v2\"\n\t_ \"modernc.org/sqlite\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/abs\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/gs\"\n\t\"github.com/benbjohnson/litestream/internal\"\n\t\"github.com/benbjohnson/litestream/nats\"\n\t\"github.com/benbjohnson/litestream/oss\"\n\t\"github.com/benbjohnson/litestream/s3\"\n\t\"github.com/benbjohnson/litestream/sftp\"\n\t\"github.com/benbjohnson/litestream/webdav\"\n)\n\n// Build information.\nvar (\n\tVersion = \"(development build)\"\n)\n\n// errStop is a terminal error for indicating program should quit.\nvar errStop = errors.New(\"stop\")\n\n// Sentinel errors for configuration validation\nvar (\n\tErrInvalidSnapshotInterval = errors.New(\"snapshot interval must be greater than 0\")\n\tErrInvalidSnapshotRetention = errors.New(\"snapshot retention must be greater than 0\")\n\tErrInvalidCompactionInterval = errors.New(\"compaction interval must be greater than 0\")\n\tErrInvalidSyncInterval = errors.New(\"sync interval must be greater than 0\")\n\tErrInvalidL0Retention = errors.New(\"l0 retention must be greater than 0\")\n\tErrInvalidL0RetentionCheckInterval = errors.New(\"l0 retention check interval must be greater than 0\")\n\tErrInvalidShutdownSyncTimeout = errors.New(\"shutdown-sync-timeout must be >= 0\")\n\tErrInvalidShutdownSyncInterval = errors.New(\"shutdown sync interval must be greater than 0\")\n\tErrInvalidHeartbeatURL = errors.New(\"heartbeat URL must be a valid HTTP or HTTPS URL\")\n\tErrInvalidHeartbeatInterval = errors.New(\"heartbeat interval must be at least 1 minute\")\n\tErrConfigFileNotFound = errors.New(\"config file not found\")\n)\n\n// ConfigValidationError wraps a validation error with additional context\ntype ConfigValidationError struct {\n\tErr error\n\tField string\n\tValue interface{}\n}\n\nfunc (e *ConfigValidationError) Error() string {\n\tif e.Value != nil {\n\t\treturn fmt.Sprintf(\"%s: %v (got %v)\", e.Field, e.Err, e.Value)\n\t}\n\treturn fmt.Sprintf(\"%s: %v\", e.Field, e.Err)\n}\n\nfunc (e *ConfigValidationError) Unwrap() error {\n\treturn e.Err\n}\n\nfunc main() {\n\tinitLog(os.Stdout, \"INFO\", \"text\")\n\n\tm := NewMain()\n\tif err := m.Run(context.Background(), os.Args[1:]); errors.Is(err, flag.ErrHelp) || errors.Is(err, errStop) {\n\t\tos.Exit(1)\n\t} else if err != nil {\n\t\tslog.Error(\"failed to run\", \"error\", err)\n\t\tos.Exit(1)\n\t}\n}\n\n// Main represents the main program execution.\ntype Main struct{}\n\n// NewMain returns a new instance of Main.\nfunc NewMain() *Main {\n\treturn &Main{}\n}\n\n// Run executes the program.\nfunc (m *Main) Run(ctx context.Context, args []string) (err error) {\n\t// Execute replication command if running as a Windows service.\n\tif isService, err := isWindowsService(); err != nil {\n\t\treturn err\n\t} else if isService {\n\t\treturn runWindowsService(ctx)\n\t}\n\n\t// Copy \"LITESTEAM\" environment credentials.\n\tapplyLitestreamEnv()\n\n\t// Extract command name.\n\tvar cmd string\n\tif len(args) > 0 {\n\t\tcmd, args = args[0], args[1:]\n\t}\n\n\tswitch cmd {\n\tcase \"databases\":\n\t\treturn (&DatabasesCommand{}).Run(ctx, args)\n\tcase \"replicate\":\n\t\tc := NewReplicateCommand()\n\t\tif err := c.ParseFlags(ctx, args); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Setup signal handler.\n\t\tsignalCh := signalChan()\n\n\t\t// Create done channel for interrupt during shutdown. It will be\n\t\t// closed when a second signal arrives, allowing each database's\n\t\t// retry loop to observe the interrupt.\n\t\tdone := make(chan struct{})\n\t\tc.SetDone(done)\n\n\t\tif err := c.Run(ctx); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Wait for signal to stop program.\n\t\tselect {\n\t\tcase err = <-c.execCh:\n\t\t\tif c.cmd != nil {\n\t\t\t\tslog.Info(\"subprocess exited, litestream shutting down\")\n\t\t\t} else {\n\t\t\t\tslog.Info(\"replication complete, litestream shutting down\")\n\t\t\t}\n\t\tcase sig := <-signalCh:\n\t\t\tslog.Info(\"signal received, litestream shutting down\", \"signal\", sig)\n\n\t\t\tif c.cmd != nil {\n\t\t\t\tslog.Info(\"sending signal to exec process\")\n\t\t\t\tif err := c.cmd.Process.Signal(sig); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"cannot signal exec process: %w\", err)\n\t\t\t\t}\n\n\t\t\t\tslog.Info(\"waiting for exec process to close\")\n\t\t\t\tif err := <-c.execCh; err != nil && !strings.HasPrefix(err.Error(), \"signal:\") {\n\t\t\t\t\treturn fmt.Errorf(\"cannot wait for exec process: %w\", err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Listen for a second signal to close done and interrupt retry loops.\n\t\t\tgo func() {\n\t\t\t\t<-signalCh\n\t\t\t\tclose(done)\n\t\t\t}()\n\t\t}\n\n\t\t// Gracefully close.\n\t\tif e := c.Close(ctx); e != nil && err == nil {\n\t\t\terr = e\n\t\t}\n\t\tslog.Info(\"litestream shut down\")\n\t\treturn err\n\n\tcase \"start\":\n\t\treturn (&StartCommand{}).Run(ctx, args)\n\tcase \"stop\":\n\t\treturn (&StopCommand{}).Run(ctx, args)\n\tcase \"register\":\n\t\treturn (&RegisterCommand{}).Run(ctx, args)\n\tcase \"unregister\":\n\t\treturn (&UnregisterCommand{}).Run(ctx, args)\n\tcase \"reset\":\n\t\treturn (&ResetCommand{}).Run(ctx, args)\n\tcase \"restore\":\n\t\treturn (&RestoreCommand{}).Run(ctx, args)\n\tcase \"status\":\n\t\treturn (&StatusCommand{}).Run(ctx, args)\n\tcase \"sync\":\n\t\treturn (&SyncCommand{}).Run(ctx, args)\n\tcase \"list\":\n\t\treturn (&ListCommand{}).Run(ctx, args)\n\tcase \"info\":\n\t\treturn (&InfoCommand{}).Run(ctx, args)\n\tcase \"version\":\n\t\treturn (&VersionCommand{}).Run(ctx, args)\n\tcase \"ltx\":\n\t\treturn (<XCommand{}).Run(ctx, args)\n\tcase \"wal\":\n\t\t// Deprecated: Keep for backward compatibility\n\t\tfmt.Fprintln(os.Stderr, \"Warning: 'wal' command is deprecated, please use 'ltx' instead\")\n\t\treturn (<XCommand{}).Run(ctx, args)\n\tdefault:\n\t\tif cmd == \"\" || cmd == \"help\" || strings.HasPrefix(cmd, \"-\") {\n\t\t\tm.Usage()\n\t\t\treturn flag.ErrHelp\n\t\t}\n\t\treturn fmt.Errorf(\"litestream %s: unknown command\", cmd)\n\t}\n}\n\n// Usage prints the help screen to STDOUT.\nfunc (m *Main) Usage() {\n\tfmt.Println(`\nlitestream is a tool for replicating SQLite databases.\n\nUsage:\n\n\tlitestream [arguments]\n\nThe commands are:\n\n\tdatabases list databases specified in config file\n\tinfo show daemon information\n\tlist list all managed databases\n\tltx list available LTX files for a database\n\tregister register a database for replication\n\treplicate runs a server to replicate databases\n\treset reset local state for a database\n\trestore recovers database backup from a replica\n\tstart start replication for a database\n\tstatus display replication status for databases\n\tstop stop replication for a database\n\tsync force an immediate sync for a database\n\tunregister unregister a database from replication\n\tversion prints the binary version\n`[1:])\n}\n\n// Config represents a configuration file for the litestream daemon.\ntype Config struct {\n\t// Global replica settings that serve as defaults for all replicas\n\tReplicaSettings `yaml:\",inline\"`\n\n\t// Bind address for serving metrics.\n\tAddr string `yaml:\"addr\"`\n\n\t// Socket configuration for control commands.\n\tSocket litestream.SocketConfig `yaml:\"socket\"`\n\n\t// List of stages in a multi-level compaction.\n\t// Only includes L1 through the last non-snapshot level.\n\tLevels []*CompactionLevelConfig `yaml:\"levels\"`\n\n\t// Snapshot configuration\n\tSnapshot SnapshotConfig `yaml:\"snapshot\"`\n\n\t// Validation configuration\n\tValidation ValidationConfig `yaml:\"validation\"`\n\n\t// L0 retention settings\n\tL0Retention *time.Duration `yaml:\"l0-retention\"`\n\tL0RetentionCheckInterval *time.Duration `yaml:\"l0-retention-check-interval\"`\n\n\t// Verify TXID consistency at destination level after each compaction.\n\t// When enabled, logs warnings if gaps or overlaps are detected.\n\tVerifyCompaction bool `yaml:\"verify-compaction\"`\n\n\t// Retention configuration\n\tRetention RetentionConfig `yaml:\"retention\"`\n\n\t// Heartbeat settings (global defaults)\n\tHeartbeatURL string `yaml:\"heartbeat-url\"`\n\tHeartbeatInterval *time.Duration `yaml:\"heartbeat-interval\"`\n\n\t// List of databases to manage.\n\tDBs []*DBConfig `yaml:\"dbs\"`\n\n\t// Subcommand to execute during replication.\n\t// Litestream will shutdown when subcommand exits.\n\tExec string `yaml:\"exec\"`\n\n\t// Logging\n\tLogging LoggingConfig `yaml:\"logging\"`\n\n\t// MCP server options\n\tMCPAddr string `yaml:\"mcp-addr\"`\n\n\t// Shutdown sync retry settings\n\tShutdownSyncTimeout *time.Duration `yaml:\"shutdown-sync-timeout\"`\n\tShutdownSyncInterval *time.Duration `yaml:\"shutdown-sync-interval\"`\n\n\t// Path to the config file\n\t// This is only used internally to pass the config path to the MCP tool\n\tConfigPath string `yaml:\"-\"`\n}\n\n// SnapshotConfig configures snapshots.\ntype SnapshotConfig struct {\n\tInterval *time.Duration `yaml:\"interval\"`\n\tRetention *time.Duration `yaml:\"retention\"`\n}\n\n// RetentionConfig configures retention enforcement behavior.\ntype RetentionConfig struct {\n\tEnabled *bool `yaml:\"enabled\"`\n}\n\n// ValidationConfig configures periodic validation checks.\ntype ValidationConfig struct {\n\tInterval *time.Duration `yaml:\"interval\"`\n}\n\n// LoggingConfig configures logging.\ntype LoggingConfig struct {\n\tLevel string `yaml:\"level\"`\n\tType string `yaml:\"type\"`\n\tStderr bool `yaml:\"stderr\"`\n}\n\n// propagateGlobalSettings copies global replica settings to individual replica configs.\nfunc (c *Config) propagateGlobalSettings() {\n\tfor _, dbc := range c.DBs {\n\t\t// Handle both old-style 'replicas' and new-style 'replica'\n\t\tif dbc.Replica != nil {\n\t\t\tdbc.Replica.SetDefaults(&c.ReplicaSettings)\n\t\t}\n\t\tfor _, rc := range dbc.Replicas {\n\t\t\trc.SetDefaults(&c.ReplicaSettings)\n\t\t}\n\t}\n}\n\n// DefaultConfig returns a new instance of Config with defaults set.\nfunc DefaultConfig() Config {\n\tdefaultSnapshotInterval := 24 * time.Hour\n\tdefaultSnapshotRetention := 24 * time.Hour\n\tdefaultL0Retention := litestream.DefaultL0Retention\n\tdefaultL0RetentionCheckInterval := litestream.DefaultL0RetentionCheckInterval\n\tdefaultShutdownSyncTimeout := litestream.DefaultShutdownSyncTimeout\n\tdefaultShutdownSyncInterval := litestream.DefaultShutdownSyncInterval\n\treturn Config{\n\t\tLevels: []*CompactionLevelConfig{\n\t\t\t{Interval: litestream.DefaultCompactionLevels[1].Interval},\n\t\t\t{Interval: litestream.DefaultCompactionLevels[2].Interval},\n\t\t\t{Interval: litestream.DefaultCompactionLevels[3].Interval},\n\t\t},\n\t\tSnapshot: SnapshotConfig{\n\t\t\tInterval: &defaultSnapshotInterval,\n\t\t\tRetention: &defaultSnapshotRetention,\n\t\t},\n\t\tSocket: litestream.DefaultSocketConfig(),\n\t\tL0Retention: &defaultL0Retention,\n\t\tL0RetentionCheckInterval: &defaultL0RetentionCheckInterval,\n\t\tShutdownSyncTimeout: &defaultShutdownSyncTimeout,\n\t\tShutdownSyncInterval: &defaultShutdownSyncInterval,\n\t}\n}\n\n// Validate returns an error if config contains invalid settings.\nfunc (c *Config) Validate() error {\n\t// Validate snapshot intervals\n\tif c.Snapshot.Interval != nil && *c.Snapshot.Interval <= 0 {\n\t\treturn &ConfigValidationError{\n\t\t\tErr: ErrInvalidSnapshotInterval,\n\t\t\tField: \"snapshot.interval\",\n\t\t\tValue: *c.Snapshot.Interval,\n\t\t}\n\t}\n\tif c.Snapshot.Retention != nil && *c.Snapshot.Retention <= 0 {\n\t\treturn &ConfigValidationError{\n\t\t\tErr: ErrInvalidSnapshotRetention,\n\t\t\tField: \"snapshot.retention\",\n\t\t\tValue: *c.Snapshot.Retention,\n\t\t}\n\t}\n\tif c.L0Retention != nil && *c.L0Retention <= 0 {\n\t\treturn &ConfigValidationError{\n\t\t\tErr: ErrInvalidL0Retention,\n\t\t\tField: \"l0-retention\",\n\t\t\tValue: *c.L0Retention,\n\t\t}\n\t}\n\tif c.L0RetentionCheckInterval != nil && *c.L0RetentionCheckInterval <= 0 {\n\t\treturn &ConfigValidationError{\n\t\t\tErr: ErrInvalidL0RetentionCheckInterval,\n\t\t\tField: \"l0-retention-check-interval\",\n\t\t\tValue: *c.L0RetentionCheckInterval,\n\t\t}\n\t}\n\tif c.ShutdownSyncTimeout != nil && *c.ShutdownSyncTimeout < 0 {\n\t\treturn &ConfigValidationError{\n\t\t\tErr: ErrInvalidShutdownSyncTimeout,\n\t\t\tField: \"shutdown-sync-timeout\",\n\t\t\tValue: *c.ShutdownSyncTimeout,\n\t\t}\n\t}\n\tif c.ShutdownSyncInterval != nil && *c.ShutdownSyncInterval <= 0 {\n\t\treturn &ConfigValidationError{\n\t\t\tErr: ErrInvalidShutdownSyncInterval,\n\t\t\tField: \"shutdown-sync-interval\",\n\t\t\tValue: *c.ShutdownSyncInterval,\n\t\t}\n\t}\n\n\t// Validate global heartbeat settings\n\tif c.HeartbeatURL != \"\" && !isValidHeartbeatURL(c.HeartbeatURL) {\n\t\treturn &ConfigValidationError{\n\t\t\tErr: ErrInvalidHeartbeatURL,\n\t\t\tField: \"heartbeat-url\",\n\t\t\tValue: c.HeartbeatURL,\n\t\t}\n\t}\n\tif c.HeartbeatInterval != nil && *c.HeartbeatInterval < litestream.MinHeartbeatInterval {\n\t\treturn &ConfigValidationError{\n\t\t\tErr: ErrInvalidHeartbeatInterval,\n\t\t\tField: \"heartbeat-interval\",\n\t\t\tValue: *c.HeartbeatInterval,\n\t\t}\n\t}\n\n\t// Validate compaction level intervals\n\tfor i, level := range c.Levels {\n\t\tif level.Interval <= 0 {\n\t\t\treturn &ConfigValidationError{\n\t\t\t\tErr: ErrInvalidCompactionInterval,\n\t\t\t\tField: fmt.Sprintf(\"levels[%d].interval\", i),\n\t\t\t\tValue: level.Interval,\n\t\t\t}\n\t\t}\n\t}\n\n\t// Validate database configs\n\tfor idx, db := range c.DBs {\n\t\t// Validate that either path or dir is specified, but not both\n\t\tif db.Path != \"\" && db.Dir != \"\" {\n\t\t\treturn fmt.Errorf(\"database config #%d: cannot specify both 'path' and 'dir'\", idx+1)\n\t\t}\n\t\tif db.Path == \"\" && db.Dir == \"\" {\n\t\t\treturn fmt.Errorf(\"database config #%d: must specify either 'path' or 'dir'\", idx+1)\n\t\t}\n\n\t\t// When using dir, pattern must be specified\n\t\tif db.Dir != \"\" && db.Pattern == \"\" {\n\t\t\treturn fmt.Errorf(\"database config #%d: 'pattern' is required when using 'dir'\", idx+1)\n\t\t}\n\t\tif db.Watch && db.Dir == \"\" {\n\t\t\treturn fmt.Errorf(\"database config #%d: 'watch' can only be enabled with a directory\", idx+1)\n\t\t}\n\n\t\t// Use path or dir for identifying the config in error messages\n\t\tdbIdentifier := db.Path\n\t\tif dbIdentifier == \"\" {\n\t\t\tdbIdentifier = db.Dir\n\t\t}\n\n\t\t// Validate sync intervals for replicas\n\t\tif db.Replica != nil && db.Replica.SyncInterval != nil && *db.Replica.SyncInterval <= 0 {\n\t\t\treturn &ConfigValidationError{\n\t\t\t\tErr: ErrInvalidSyncInterval,\n\t\t\t\tField: fmt.Sprintf(\"dbs[%s].replica.sync-interval\", dbIdentifier),\n\t\t\t\tValue: *db.Replica.SyncInterval,\n\t\t\t}\n\t\t}\n\t\tfor i, replica := range db.Replicas {\n\t\t\tif replica.SyncInterval != nil && *replica.SyncInterval <= 0 {\n\t\t\t\treturn &ConfigValidationError{\n\t\t\t\t\tErr: ErrInvalidSyncInterval,\n\t\t\t\t\tField: fmt.Sprintf(\"dbs[%s].replicas[%d].sync-interval\", dbIdentifier, i),\n\t\t\t\t\tValue: *replica.SyncInterval,\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// CompactionLevels returns a full list of compaction levels include L0.\nfunc (c *Config) CompactionLevels() litestream.CompactionLevels {\n\tlevels := litestream.CompactionLevels{\n\t\t{Level: 0},\n\t}\n\n\tfor i, lvl := range c.Levels {\n\t\tlevels = append(levels, &litestream.CompactionLevel{\n\t\t\tLevel: i + 1,\n\t\t\tInterval: lvl.Interval,\n\t\t})\n\t}\n\n\treturn levels\n}\n\n// DBConfig returns database configuration by path.\nfunc (c *Config) DBConfig(configPath string) *DBConfig {\n\tfor _, dbConfig := range c.DBs {\n\t\tif dbConfig.Path == configPath {\n\t\t\treturn dbConfig\n\t\t}\n\t}\n\treturn nil\n}\n\n// OpenConfigFile opens a configuration file and returns a reader.\n// Expands the filename path if needed.\nfunc OpenConfigFile(filename string) (io.ReadCloser, error) {\n\t// Expand filename, if necessary.\n\tfilename, err := expand(filename)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Open configuration file.\n\tf, err := os.Open(filename)\n\tif os.IsNotExist(err) {\n\t\treturn nil, fmt.Errorf(\"%w: %s\", ErrConfigFileNotFound, filename)\n\t} else if err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn f, nil\n}\n\n// ReadConfigFile unmarshals config from filename. Expands path if needed.\n// If expandEnv is true then environment variables are expanded in the config.\nfunc ReadConfigFile(filename string, expandEnv bool) (Config, error) {\n\tf, err := OpenConfigFile(filename)\n\tif err != nil {\n\t\treturn DefaultConfig(), err\n\t}\n\tdefer f.Close()\n\n\treturn ParseConfig(f, expandEnv)\n}\n\n// ParseConfig unmarshals config from a reader.\n// If expandEnv is true then environment variables are expanded in the config.\nfunc ParseConfig(r io.Reader, expandEnv bool) (_ Config, err error) {\n\tconfig := DefaultConfig()\n\n\t// Read configuration.\n\tbuf, err := io.ReadAll(r)\n\tif err != nil {\n\t\treturn config, err\n\t}\n\n\t// Expand environment variables, if enabled.\n\tif expandEnv {\n\t\tbuf = []byte(os.Expand(string(buf), func(key string) string {\n\t\t\tif key == \"PID\" {\n\t\t\t\treturn strconv.Itoa(os.Getpid())\n\t\t\t}\n\t\t\treturn os.Getenv(key)\n\t\t}))\n\t}\n\n\t// Save defaults before unmarshaling\n\tdefaultSnapshotInterval := config.Snapshot.Interval\n\tdefaultSnapshotRetention := config.Snapshot.Retention\n\tdefaultL0Retention := config.L0Retention\n\tdefaultL0RetentionCheckInterval := config.L0RetentionCheckInterval\n\n\tif err := yaml.Unmarshal(buf, &config); err != nil {\n\t\treturn config, err\n\t}\n\n\t// Restore defaults if they were overwritten with nil by empty YAML sections\n\tif config.Snapshot.Interval == nil {\n\t\tconfig.Snapshot.Interval = defaultSnapshotInterval\n\t}\n\tif config.Snapshot.Retention == nil {\n\t\tconfig.Snapshot.Retention = defaultSnapshotRetention\n\t}\n\tif config.L0Retention == nil {\n\t\tconfig.L0Retention = defaultL0Retention\n\t}\n\tif config.L0RetentionCheckInterval == nil {\n\t\tconfig.L0RetentionCheckInterval = defaultL0RetentionCheckInterval\n\t}\n\n\t// Normalize paths.\n\tfor _, dbConfig := range config.DBs {\n\t\tif dbConfig.Path == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tif dbConfig.Path, err = expand(dbConfig.Path); err != nil {\n\t\t\treturn config, err\n\t\t}\n\t}\n\n\t// Propagate settings from global config to individual configs.\n\tconfig.propagateGlobalSettings()\n\n\t// Validate configuration\n\tif err := config.Validate(); err != nil {\n\t\treturn config, err\n\t}\n\n\t// Configure logging.\n\tlogOutput := os.Stdout\n\tif config.Logging.Stderr {\n\t\tlogOutput = os.Stderr\n\t}\n\tif v := os.Getenv(\"LOG_LEVEL\"); v != \"\" {\n\t\tconfig.Logging.Level = v\n\t}\n\tinitLog(logOutput, config.Logging.Level, config.Logging.Type)\n\n\treturn config, nil\n}\n\n// CompactionLevelConfig the configuration for a single level of compaction.\ntype CompactionLevelConfig struct {\n\tInterval time.Duration `yaml:\"interval\"`\n}\n\n// DBConfig represents the configuration for a single database or directory of databases.\ntype DBConfig struct {\n\tPath string `yaml:\"path\"`\n\tDir string `yaml:\"dir\"` // Directory to scan for databases\n\tPattern string `yaml:\"pattern\"` // File pattern to match (e.g., \"*.db\", \"*.sqlite\")\n\tRecursive bool `yaml:\"recursive\"` // Scan subdirectories recursively\n\tWatch bool `yaml:\"watch\"` // Enable directory monitoring for changes\n\tMetaPath *string `yaml:\"meta-path\"`\n\tMonitorInterval *time.Duration `yaml:\"monitor-interval\"`\n\tCheckpointInterval *time.Duration `yaml:\"checkpoint-interval\"`\n\tBusyTimeout *time.Duration `yaml:\"busy-timeout\"`\n\tMinCheckpointPageN *int `yaml:\"min-checkpoint-page-count\"`\n\tTruncatePageN *int `yaml:\"truncate-page-n\"`\n\n\tRestoreIfDBNotExists bool `yaml:\"restore-if-db-not-exists\"`\n\n\tReplica *ReplicaConfig `yaml:\"replica\"`\n\tReplicas []*ReplicaConfig `yaml:\"replicas\"` // Deprecated\n}\n\n// NewDBFromConfig instantiates a DB based on a configuration.\nfunc NewDBFromConfig(dbc *DBConfig) (*litestream.DB, error) {\n\tconfigPath, err := expand(dbc.Path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Initialize database with given path.\n\tdb := litestream.NewDB(configPath)\n\n\t// Override default database settings if specified in configuration.\n\tif dbc.MetaPath != nil {\n\t\texpandedMetaPath, err := expand(*dbc.MetaPath)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to expand meta path: %w\", err)\n\t\t}\n\t\tdbc.MetaPath = &expandedMetaPath\n\t\tdb.SetMetaPath(expandedMetaPath)\n\t}\n\tif dbc.MonitorInterval != nil {\n\t\tdb.MonitorInterval = *dbc.MonitorInterval\n\t}\n\tif dbc.CheckpointInterval != nil {\n\t\tdb.CheckpointInterval = *dbc.CheckpointInterval\n\t}\n\tif dbc.BusyTimeout != nil {\n\t\tdb.BusyTimeout = *dbc.BusyTimeout\n\t}\n\tif dbc.MinCheckpointPageN != nil {\n\t\tdb.MinCheckpointPageN = *dbc.MinCheckpointPageN\n\t}\n\tif dbc.TruncatePageN != nil {\n\t\tdb.TruncatePageN = *dbc.TruncatePageN\n\t}\n\n\t// Instantiate and attach replica.\n\t// v0.3.x and before supported multiple replicas but that was dropped to\n\t// ensure there's a single remote data authority.\n\tswitch {\n\tcase dbc.Replica == nil && len(dbc.Replicas) == 0:\n\t\treturn nil, fmt.Errorf(\"must specify replica for database\")\n\tcase dbc.Replica != nil && len(dbc.Replicas) > 0:\n\t\treturn nil, fmt.Errorf(\"cannot specify 'replica' and 'replicas' on a database\")\n\tcase len(dbc.Replicas) > 1:\n\t\treturn nil, fmt.Errorf(\"multiple replicas on a single database are no longer supported\")\n\t}\n\n\tvar rc *ReplicaConfig\n\tif dbc.Replica != nil {\n\t\trc = dbc.Replica\n\t} else {\n\t\trc = dbc.Replicas[0]\n\t}\n\n\tr, err := NewReplicaFromConfig(rc, db)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdb.Replica = r\n\n\treturn db, nil\n}\n\n// NewDBsFromDirectoryConfig scans a directory and creates DB instances for all SQLite databases found.\nfunc NewDBsFromDirectoryConfig(dbc *DBConfig) ([]*litestream.DB, error) {\n\tif dbc.Dir == \"\" {\n\t\treturn nil, fmt.Errorf(\"directory path is required for directory replication\")\n\t}\n\n\tif dbc.Pattern == \"\" {\n\t\treturn nil, fmt.Errorf(\"pattern is required for directory replication\")\n\t}\n\n\tdirPath, err := expand(dbc.Dir)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Find all SQLite databases in the directory\n\tdbPaths, err := FindSQLiteDatabases(dirPath, dbc.Pattern, dbc.Recursive)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to scan directory %s: %w\", dirPath, err)\n\t}\n\n\tif len(dbPaths) == 0 && !dbc.Watch {\n\t\treturn nil, fmt.Errorf(\"no SQLite databases found in directory %s with pattern %s\", dirPath, dbc.Pattern)\n\t}\n\n\t// Create DB instances for each found database\n\tvar dbs []*litestream.DB\n\tmetaPaths := make(map[string]string)\n\n\tfor _, dbPath := range dbPaths {\n\t\tdb, err := newDBFromDirectoryEntry(dbc, dirPath, dbPath)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to create DB for %s: %w\", dbPath, err)\n\t\t}\n\n\t\t// Validate unique meta-path to prevent replication state corruption\n\t\tif mp := db.MetaPath(); mp != \"\" {\n\t\t\tif existingDB, exists := metaPaths[mp]; exists {\n\t\t\t\treturn nil, fmt.Errorf(\"meta-path collision: databases %s and %s would share meta-path %s, causing replication state corruption\", existingDB, dbPath, mp)\n\t\t\t}\n\t\t\tmetaPaths[mp] = dbPath\n\t\t}\n\n\t\tdbs = append(dbs, db)\n\t}\n\n\treturn dbs, nil\n}\n\n// newDBFromDirectoryEntry creates a DB instance for a database discovered via directory replication.\nfunc newDBFromDirectoryEntry(dbc *DBConfig, dirPath, dbPath string) (*litestream.DB, error) {\n\t// Calculate relative path from directory root\n\trelPath, err := filepath.Rel(dirPath, dbPath)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to calculate relative path for %s: %w\", dbPath, err)\n\t}\n\n\t// Create a copy of the config for the discovered database\n\tdbConfigCopy := *dbc\n\tdbConfigCopy.Path = dbPath\n\tdbConfigCopy.Dir = \"\" // Clear dir field for individual DB\n\tdbConfigCopy.Pattern = \"\" // Clear pattern field\n\tdbConfigCopy.Recursive = false // Clear recursive flag\n\tdbConfigCopy.Watch = false // Individual DBs do not watch directories\n\n\t// Ensure every database discovered beneath a directory receives a unique\n\t// metadata path. Without this, all databases share the same meta-path and\n\t// clobber each other's replication state.\n\tif dbc.MetaPath != nil {\n\t\tbaseMetaPath, err := expand(*dbc.MetaPath)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to expand meta path for %s: %w\", dbPath, err)\n\t\t}\n\t\tmetaPathCopy := deriveMetaPathForDirectoryEntry(baseMetaPath, relPath)\n\t\tdbConfigCopy.MetaPath = &metaPathCopy\n\t}\n\n\t// Deep copy replica config and make path unique per database.\n\t// This prevents all databases from writing to the same replica path.\n\tif dbc.Replica != nil {\n\t\treplicaCopy, err := cloneReplicaConfigWithRelativePath(dbc.Replica, relPath)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to configure replica for %s: %w\", dbPath, err)\n\t\t}\n\t\tdbConfigCopy.Replica = replicaCopy\n\t}\n\n\t// Also handle deprecated 'replicas' array field.\n\tif len(dbc.Replicas) > 0 {\n\t\tdbConfigCopy.Replicas = make([]*ReplicaConfig, len(dbc.Replicas))\n\t\tfor i, replica := range dbc.Replicas {\n\t\t\treplicaCopy, err := cloneReplicaConfigWithRelativePath(replica, relPath)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"failed to configure replica %d for %s: %w\", i, dbPath, err)\n\t\t\t}\n\t\t\tdbConfigCopy.Replicas[i] = replicaCopy\n\t\t}\n\t}\n\n\treturn NewDBFromConfig(&dbConfigCopy)\n}\n\n// cloneReplicaConfigWithRelativePath returns a copy of the replica configuration with the\n// database-relative path appended to either the replica path or URL, depending on how the\n// replica was configured.\nfunc cloneReplicaConfigWithRelativePath(base *ReplicaConfig, relPath string) (*ReplicaConfig, error) {\n\tif base == nil {\n\t\treturn nil, nil\n\t}\n\n\treplicaCopy := *base\n\trelPath = filepath.ToSlash(relPath)\n\tif relPath == \"\" || relPath == \".\" {\n\t\treturn &replicaCopy, nil\n\t}\n\n\tif replicaCopy.URL != \"\" {\n\t\tu, err := url.Parse(replicaCopy.URL)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"parse replica url: %w\", err)\n\t\t}\n\t\tappendRelativePathToURL(u, relPath)\n\t\treplicaCopy.URL = u.String()\n\t\treturn &replicaCopy, nil\n\t}\n\n\tswitch base.ReplicaType() {\n\tcase \"file\":\n\t\trelOSPath := filepath.FromSlash(relPath)\n\t\tif replicaCopy.Path != \"\" {\n\t\t\treplicaCopy.Path = filepath.Join(replicaCopy.Path, relOSPath)\n\t\t} else {\n\t\t\treplicaCopy.Path = relOSPath\n\t\t}\n\tdefault:\n\t\t// Normalize to forward slashes for cloud/object storage backends.\n\t\tbasePath := filepath.ToSlash(replicaCopy.Path)\n\t\tif basePath != \"\" {\n\t\t\treplicaCopy.Path = path.Join(basePath, relPath)\n\t\t} else {\n\t\t\treplicaCopy.Path = relPath\n\t\t}\n\t}\n\n\treturn &replicaCopy, nil\n}\n\n// deriveMetaPathForDirectoryEntry returns a unique metadata directory for a\n// database discovered through directory replication by appending the database's\n// relative path and the standard Litestream suffix to the configured base path.\nfunc deriveMetaPathForDirectoryEntry(basePath, relPath string) string {\n\trelPath = filepath.Clean(relPath)\n\tif relPath == \".\" || relPath == \"\" {\n\t\treturn basePath\n\t}\n\n\trelDir, relFile := filepath.Split(relPath)\n\tif relFile == \"\" || relFile == \".\" {\n\t\treturn filepath.Join(basePath, relPath)\n\t}\n\n\tmetaDirName := \".\" + relFile + litestream.MetaDirSuffix\n\treturn filepath.Join(basePath, relDir, metaDirName)\n}\n\n// appendRelativePathToURL appends relPath to the URL's path component, ensuring\n// the result remains rooted and uses forward slashes.\nfunc appendRelativePathToURL(u *url.URL, relPath string) {\n\tcleanRel := strings.TrimPrefix(relPath, \"/\")\n\tif cleanRel == \"\" || cleanRel == \".\" {\n\t\treturn\n\t}\n\n\tbasePath := u.Path\n\tvar joined string\n\tif basePath == \"\" {\n\t\tjoined = cleanRel\n\t} else {\n\t\tjoined = path.Join(basePath, cleanRel)\n\t}\n\n\tjoined = \"/\" + strings.TrimPrefix(joined, \"/\")\n\tu.Path = joined\n}\n\n// FindSQLiteDatabases recursively finds all SQLite database files in a directory.\n// Exported for testing.\nfunc FindSQLiteDatabases(dir string, pattern string, recursive bool) ([]string, error) {\n\tvar dbPaths []string\n\n\terr := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Skip directories unless recursive\n\t\tif info.IsDir() {\n\t\t\tif !recursive && path != dir {\n\t\t\t\treturn filepath.SkipDir\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\n\t\t// Check if file matches pattern\n\t\tmatched, err := filepath.Match(pattern, filepath.Base(path))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif !matched {\n\t\t\treturn nil\n\t\t}\n\n\t\t// Check if it's a SQLite database\n\t\tif IsSQLiteDatabase(path) {\n\t\t\tdbPaths = append(dbPaths, path)\n\t\t}\n\n\t\treturn nil\n\t})\n\n\treturn dbPaths, err\n}\n\n// IsSQLiteDatabase checks if a file is a SQLite database by reading its header.\n// Exported for testing.\nfunc IsSQLiteDatabase(path string) bool {\n\tfile, err := os.Open(path)\n\tif err != nil {\n\t\treturn false\n\t}\n\tdefer file.Close()\n\n\t// SQLite files start with \"SQLite format 3\\x00\"\n\theader := make([]byte, 16)\n\tif _, err := file.Read(header); err != nil {\n\t\treturn false\n\t}\n\n\treturn string(header) == \"SQLite format 3\\x00\"\n}\n\n// ByteSize is a custom type for parsing byte sizes from YAML.\n// It supports both SI units (KB, MB, GB using base 1000) and IEC units\n// (KiB, MiB, GiB using base 1024) as well as short forms (K, M, G).\ntype ByteSize int64\n\n// UnmarshalYAML implements yaml.Unmarshaler for ByteSize.\nfunc (b *ByteSize) UnmarshalYAML(unmarshal func(interface{}) error) error {\n\tvar s string\n\tif err := unmarshal(&s); err != nil {\n\t\treturn err\n\t}\n\n\tsize, err := ParseByteSize(s)\n\tif err != nil {\n\t\treturn err\n\t}\n\t*b = ByteSize(size)\n\treturn nil\n}\n\n// ParseByteSize parses a byte size string using github.com/dustin/go-humanize.\n// Supports both SI units (KB=1000, MB=1000², etc.) and IEC units (KiB=1024, MiB=1024², etc.).\n// Examples: \"1MB\", \"5MiB\", \"1.5GB\", \"100B\", \"1024KB\"\nfunc ParseByteSize(s string) (int64, error) {\n\ts = strings.TrimSpace(s)\n\tif s == \"\" {\n\t\treturn 0, fmt.Errorf(\"empty size string\")\n\t}\n\n\t// Use go-humanize to parse the byte size string\n\tbytes, err := humanize.ParseBytes(s)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"invalid size format: %w\", err)\n\t}\n\n\t// Check that the value fits in int64\n\tif bytes > math.MaxInt64 {\n\t\treturn 0, fmt.Errorf(\"size %d exceeds maximum allowed value (%d)\", bytes, int64(math.MaxInt64))\n\t}\n\n\treturn int64(bytes), nil\n}\n\n// ReplicaSettings contains settings shared across replica configurations.\n// These can be set globally in Config or per-replica in ReplicaConfig.\ntype ReplicaSettings struct {\n\tSyncInterval *time.Duration `yaml:\"sync-interval\"`\n\tValidationInterval *time.Duration `yaml:\"validation-interval\"`\n\n\t// If true, automatically reset local state when LTX errors are detected.\n\t// This allows recovery from corrupted/missing LTX files by forcing a fresh sync.\n\t// Disabled by default to prevent silent data loss scenarios.\n\tAutoRecover *bool `yaml:\"auto-recover\"`\n\n\t// S3 settings\n\tAccessKeyID string `yaml:\"access-key-id\"`\n\tSecretAccessKey string `yaml:\"secret-access-key\"`\n\tRegion string `yaml:\"region\"`\n\tBucket string `yaml:\"bucket\"`\n\tEndpoint string `yaml:\"endpoint\"`\n\tForcePathStyle *bool `yaml:\"force-path-style\"`\n\tSignPayload *bool `yaml:\"sign-payload\"`\n\tRequireContentMD5 *bool `yaml:\"require-content-md5\"`\n\tSkipVerify bool `yaml:\"skip-verify\"`\n\tPartSize *ByteSize `yaml:\"part-size\"`\n\tConcurrency *int `yaml:\"concurrency\"`\n\n\t// S3 Server-Side Encryption (SSE-C: Customer-provided keys)\n\tSSECustomerAlgorithm string `yaml:\"sse-customer-algorithm\"`\n\tSSECustomerKey string `yaml:\"sse-customer-key\"`\n\tSSECustomerKeyPath string `yaml:\"sse-customer-key-path\"`\n\n\t// S3 Server-Side Encryption (SSE-KMS: AWS Key Management Service)\n\tSSEKMSKeyID string `yaml:\"sse-kms-key-id\"`\n\n\t// ABS settings\n\tAccountName string `yaml:\"account-name\"`\n\tAccountKey string `yaml:\"account-key\"`\n\tSASToken string `yaml:\"sas-token\"`\n\n\t// SFTP settings\n\tHost string `yaml:\"host\"`\n\tUser string `yaml:\"user\"`\n\tPassword string `yaml:\"password\"`\n\tKeyPath string `yaml:\"key-path\"`\n\tConcurrentWrites *bool `yaml:\"concurrent-writes\"`\n\tHostKey string `yaml:\"host-key\"`\n\n\t// WebDAV settings\n\tWebDAVURL string `yaml:\"webdav-url\"`\n\tWebDAVUsername string `yaml:\"webdav-username\"`\n\tWebDAVPassword string `yaml:\"webdav-password\"`\n\n\t// NATS settings\n\tJWT string `yaml:\"jwt\"`\n\tSeed string `yaml:\"seed\"`\n\tCreds string `yaml:\"creds\"`\n\tNKey string `yaml:\"nkey\"`\n\tUsername string `yaml:\"username\"`\n\tToken string `yaml:\"token\"`\n\tTLS bool `yaml:\"tls\"`\n\tRootCAs []string `yaml:\"root-cas\"`\n\tClientCert string `yaml:\"client-cert\"`\n\tClientKey string `yaml:\"client-key\"`\n\tMaxReconnects *int `yaml:\"max-reconnects\"`\n\tReconnectWait *time.Duration `yaml:\"reconnect-wait\"`\n\tTimeout *time.Duration `yaml:\"timeout\"`\n\n\t// Encryption identities and recipients\n\tAge struct {\n\t\tIdentities []string `yaml:\"identities\"`\n\t\tRecipients []string `yaml:\"recipients\"`\n\t} `yaml:\"age\"`\n}\n\n// SetDefaults merges default settings from src into the current ReplicaSettings.\n// Individual settings override defaults when already set.\nfunc (rs *ReplicaSettings) SetDefaults(src *ReplicaSettings) {\n\tif src == nil {\n\t\treturn\n\t}\n\n\t// Timing settings\n\tif rs.SyncInterval == nil && src.SyncInterval != nil {\n\t\trs.SyncInterval = src.SyncInterval\n\t}\n\tif rs.ValidationInterval == nil && src.ValidationInterval != nil {\n\t\trs.ValidationInterval = src.ValidationInterval\n\t}\n\n\t// Recovery settings\n\tif rs.AutoRecover == nil && src.AutoRecover != nil {\n\t\trs.AutoRecover = src.AutoRecover\n\t}\n\n\t// S3 settings\n\tif rs.AccessKeyID == \"\" {\n\t\trs.AccessKeyID = src.AccessKeyID\n\t}\n\tif rs.SecretAccessKey == \"\" {\n\t\trs.SecretAccessKey = src.SecretAccessKey\n\t}\n\tif rs.Region == \"\" {\n\t\trs.Region = src.Region\n\t}\n\tif rs.Bucket == \"\" {\n\t\trs.Bucket = src.Bucket\n\t}\n\tif rs.Endpoint == \"\" {\n\t\trs.Endpoint = src.Endpoint\n\t}\n\tif rs.ForcePathStyle == nil {\n\t\trs.ForcePathStyle = src.ForcePathStyle\n\t}\n\tif rs.SignPayload == nil {\n\t\trs.SignPayload = src.SignPayload\n\t}\n\tif rs.RequireContentMD5 == nil {\n\t\trs.RequireContentMD5 = src.RequireContentMD5\n\t}\n\tif src.SkipVerify {\n\t\trs.SkipVerify = true\n\t}\n\n\t// S3 SSE settings\n\tif rs.SSECustomerAlgorithm == \"\" {\n\t\trs.SSECustomerAlgorithm = src.SSECustomerAlgorithm\n\t}\n\tif rs.SSECustomerKey == \"\" {\n\t\trs.SSECustomerKey = src.SSECustomerKey\n\t}\n\tif rs.SSECustomerKeyPath == \"\" {\n\t\trs.SSECustomerKeyPath = src.SSECustomerKeyPath\n\t}\n\tif rs.SSEKMSKeyID == \"\" {\n\t\trs.SSEKMSKeyID = src.SSEKMSKeyID\n\t}\n\n\t// ABS settings\n\tif rs.AccountName == \"\" {\n\t\trs.AccountName = src.AccountName\n\t}\n\tif rs.AccountKey == \"\" {\n\t\trs.AccountKey = src.AccountKey\n\t}\n\tif rs.SASToken == \"\" {\n\t\trs.SASToken = src.SASToken\n\t}\n\n\t// SFTP settings\n\tif rs.Host == \"\" {\n\t\trs.Host = src.Host\n\t}\n\tif rs.User == \"\" {\n\t\trs.User = src.User\n\t}\n\tif rs.Password == \"\" {\n\t\trs.Password = src.Password\n\t}\n\tif rs.KeyPath == \"\" {\n\t\trs.KeyPath = src.KeyPath\n\t}\n\tif rs.ConcurrentWrites == nil {\n\t\trs.ConcurrentWrites = src.ConcurrentWrites\n\t}\n\n\t// NATS settings\n\tif rs.JWT == \"\" {\n\t\trs.JWT = src.JWT\n\t}\n\tif rs.Seed == \"\" {\n\t\trs.Seed = src.Seed\n\t}\n\tif rs.Creds == \"\" {\n\t\trs.Creds = src.Creds\n\t}\n\tif rs.NKey == \"\" {\n\t\trs.NKey = src.NKey\n\t}\n\tif rs.Username == \"\" {\n\t\trs.Username = src.Username\n\t}\n\tif rs.Token == \"\" {\n\t\trs.Token = src.Token\n\t}\n\tif !rs.TLS {\n\t\trs.TLS = src.TLS\n\t}\n\tif len(rs.RootCAs) == 0 {\n\t\trs.RootCAs = src.RootCAs\n\t}\n\tif rs.ClientCert == \"\" {\n\t\trs.ClientCert = src.ClientCert\n\t}\n\tif rs.ClientKey == \"\" {\n\t\trs.ClientKey = src.ClientKey\n\t}\n\tif rs.MaxReconnects == nil {\n\t\trs.MaxReconnects = src.MaxReconnects\n\t}\n\tif rs.ReconnectWait == nil {\n\t\trs.ReconnectWait = src.ReconnectWait\n\t}\n\tif rs.Timeout == nil {\n\t\trs.Timeout = src.Timeout\n\t}\n\n\t// Age encryption settings\n\tif len(rs.Age.Identities) == 0 {\n\t\trs.Age.Identities = src.Age.Identities\n\t}\n\tif len(rs.Age.Recipients) == 0 {\n\t\trs.Age.Recipients = src.Age.Recipients\n\t}\n}\n\n// ReplicaConfig represents the configuration for a single replica in a database.\ntype ReplicaConfig struct {\n\tReplicaSettings `yaml:\",inline\"`\n\n\tType string `yaml:\"type\"` // \"file\", \"s3\"\n\tName string `yaml:\"name\"` // Deprecated\n\tPath string `yaml:\"path\"`\n\tURL string `yaml:\"url\"`\n}\n\n// NewReplicaFromConfig instantiates a replica for a DB based on a config.\nfunc NewReplicaFromConfig(c *ReplicaConfig, db *litestream.DB) (_ *litestream.Replica, err error) {\n\t// Ensure user did not specify URL in path.\n\tif litestream.IsURL(c.Path) {\n\t\treturn nil, fmt.Errorf(\"replica path cannot be a url, please use the 'url' field instead: %s\", c.Path)\n\t}\n\n\t// Reject age encryption configuration as it's currently non-functional.\n\t// Age encryption support was removed during the LTX storage layer refactor\n\t// and has not been reimplemented. Accepting this config would silently\n\t// write plaintext data to remote storage instead of encrypted data.\n\t// See: https://github.com/benbjohnson/litestream/issues/790\n\tif len(c.Age.Identities) > 0 || len(c.Age.Recipients) > 0 {\n\t\treturn nil, fmt.Errorf(\"age encryption is not currently supported, if you need encryption please revert back to Litestream v0.3.x\")\n\t}\n\n\t// Build replica.\n\tr := litestream.NewReplica(db)\n\tif v := c.SyncInterval; v != nil {\n\t\tr.SyncInterval = *v\n\t}\n\tif v := c.AutoRecover; v != nil {\n\t\tr.AutoRecoverEnabled = *v\n\t}\n\n\t// Build and set client on replica.\n\tswitch c.ReplicaType() {\n\tcase \"file\":\n\t\tif r.Client, err = newFileReplicaClientFromConfig(c, r); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\tcase \"s3\":\n\t\tif r.Client, err = NewS3ReplicaClientFromConfig(c, r); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\tcase \"gs\":\n\t\tif r.Client, err = newGSReplicaClientFromConfig(c, r); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\tcase \"abs\":\n\t\tif r.Client, err = newABSReplicaClientFromConfig(c, r); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\tcase \"sftp\":\n\t\tif r.Client, err = newSFTPReplicaClientFromConfig(c, r); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\tcase \"webdav\":\n\t\tif r.Client, err = newWebDAVReplicaClientFromConfig(c, r); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\tcase \"nats\":\n\t\tif r.Client, err = newNATSReplicaClientFromConfig(c, r); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\tcase \"oss\":\n\t\tif r.Client, err = newOSSReplicaClientFromConfig(c, r); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unknown replica type in config: %q\", c.Type)\n\t}\n\n\tr.Client.SetLogger(r.Logger())\n\n\treturn r, nil\n}\n\n// newFileReplicaClientFromConfig returns a new instance of file.ReplicaClient built from config.\nfunc newFileReplicaClientFromConfig(c *ReplicaConfig, r *litestream.Replica) (_ *file.ReplicaClient, err error) {\n\t// Ensure URL & path are not both specified.\n\tif c.URL != \"\" && c.Path != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & path for file replica\")\n\t}\n\n\t// Parse configPath from URL, if specified.\n\tconfigPath := c.Path\n\tif c.URL != \"\" {\n\t\tif _, _, configPath, err = litestream.ParseReplicaURL(c.URL); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\t// Ensure path is set explicitly or derived from URL field.\n\tif configPath == \"\" {\n\t\treturn nil, fmt.Errorf(\"file replica path required\")\n\t}\n\n\t// Expand home prefix and return absolute path.\n\tif configPath, err = expand(configPath); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Instantiate replica and apply time fields, if set.\n\tclient := file.NewReplicaClient(configPath)\n\tclient.Replica = r\n\treturn client, nil\n}\n\n// NewS3ReplicaClientFromConfig returns a new instance of s3.ReplicaClient built from config.\n// Exported for testing.\nfunc NewS3ReplicaClientFromConfig(c *ReplicaConfig, _ *litestream.Replica) (_ *s3.ReplicaClient, err error) {\n\t// Ensure URL & constituent parts are not both specified.\n\tif c.URL != \"\" && c.Path != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & path for s3 replica\")\n\t} else if c.URL != \"\" && c.Bucket != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & bucket for s3 replica\")\n\t}\n\n\tbucket, configPath := c.Bucket, c.Path\n\tregion, endpoint, skipVerify := c.Region, c.Endpoint, c.SkipVerify\n\tsignSetting := newBoolSetting(true)\n\tif v := c.SignPayload; v != nil {\n\t\tsignSetting.Set(*v)\n\t}\n\trequireSetting := newBoolSetting(true)\n\tif v := c.RequireContentMD5; v != nil {\n\t\trequireSetting.Set(*v)\n\t}\n\n\t// Use path style if an endpoint is explicitly set. This works because the\n\t// only service to not use path style is AWS which does not use an endpoint.\n\tforcePathStyle := (endpoint != \"\")\n\tif v := c.ForcePathStyle; v != nil {\n\t\tforcePathStyle = *v\n\t}\n\n\t// Apply settings from URL, if specified.\n\tvar (\n\t\tendpointWasSet bool\n\t\tusignPayload bool\n\t\tusignPayloadSet bool\n\t\turequireContentMD5 bool\n\t\turequireContentMD5Set bool\n\t)\n\tif endpoint != \"\" {\n\t\tendpointWasSet = true\n\t}\n\n\tif c.URL != \"\" {\n\t\t_, host, upath, query, _, err := litestream.ParseReplicaURLWithQuery(c.URL)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tvar (\n\t\t\tubucket string\n\t\t\turegion string\n\t\t\tuendpoint string\n\t\t\tuforcePathStyle bool\n\t\t)\n\n\t\tif strings.HasPrefix(host, \"arn:\") {\n\t\t\tubucket = host\n\t\t\turegion = litestream.RegionFromS3ARN(host)\n\t\t} else {\n\t\t\tubucket, uregion, uendpoint, uforcePathStyle = s3.ParseHost(host)\n\t\t}\n\n\t\t// Override with query parameters if provided\n\t\tif qEndpoint := query.Get(\"endpoint\"); qEndpoint != \"\" {\n\t\t\t// Ensure endpoint has a scheme (defaults to https:// for cloud, http:// for local)\n\t\t\tqEndpoint, _ = litestream.EnsureEndpointScheme(qEndpoint)\n\t\t\tuendpoint = qEndpoint\n\t\t\t// Default to path style for custom endpoints unless explicitly set to false\n\t\t\tif query.Get(\"forcePathStyle\") != \"false\" {\n\t\t\t\tuforcePathStyle = true\n\t\t\t}\n\t\t\tendpointWasSet = true\n\t\t}\n\t\tif qRegion := query.Get(\"region\"); qRegion != \"\" {\n\t\t\turegion = qRegion\n\t\t}\n\t\tif qForcePathStyle := query.Get(\"forcePathStyle\"); qForcePathStyle != \"\" {\n\t\t\tuforcePathStyle = qForcePathStyle == \"true\"\n\t\t}\n\t\tif qSkipVerify := query.Get(\"skipVerify\"); qSkipVerify != \"\" {\n\t\t\tskipVerify = qSkipVerify == \"true\"\n\t\t}\n\t\tif v, ok := litestream.BoolQueryValue(query, \"signPayload\", \"sign-payload\"); ok {\n\t\t\tusignPayload = v\n\t\t\tusignPayloadSet = true\n\t\t}\n\t\tif v, ok := litestream.BoolQueryValue(query, \"requireContentMD5\", \"require-content-md5\"); ok {\n\t\t\turequireContentMD5 = v\n\t\t\turequireContentMD5Set = true\n\t\t}\n\n\t\t// Only apply URL parts to field that have not been overridden.\n\t\tif configPath == \"\" {\n\t\t\tconfigPath = upath\n\t\t}\n\t\tif bucket == \"\" {\n\t\t\tbucket = ubucket\n\t\t}\n\t\tif region == \"\" {\n\t\t\tregion = uregion\n\t\t}\n\t\tif endpoint == \"\" {\n\t\t\tendpoint = uendpoint\n\t\t}\n\t\tif !forcePathStyle {\n\t\t\tforcePathStyle = uforcePathStyle\n\t\t}\n\t\tif !signSetting.set && usignPayloadSet {\n\t\t\tsignSetting.Set(usignPayload)\n\t\t}\n\t\tif !requireSetting.set && urequireContentMD5Set {\n\t\t\trequireSetting.Set(urequireContentMD5)\n\t\t}\n\t}\n\n\t// Ensure required settings are set.\n\tif bucket == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for s3 replica\")\n\t}\n\n\t// Detect S3-compatible provider endpoints for applying appropriate defaults.\n\t// These providers require specific settings to work correctly with AWS SDK v2.\n\tisTigris := litestream.IsTigrisEndpoint(endpoint)\n\tif !isTigris && !endpointWasSet && litestream.IsTigrisEndpoint(c.Endpoint) {\n\t\tisTigris = true\n\t}\n\tisDigitalOcean := litestream.IsDigitalOceanEndpoint(endpoint)\n\tisBackblaze := litestream.IsBackblazeEndpoint(endpoint)\n\tisFilebase := litestream.IsFilebaseEndpoint(endpoint)\n\tisScaleway := litestream.IsScalewayEndpoint(endpoint)\n\tisMinIO := litestream.IsMinIOEndpoint(endpoint)\n\tisCloudflareR2 := litestream.IsCloudflareR2Endpoint(endpoint)\n\tisSupabase := litestream.IsSupabaseEndpoint(endpoint)\n\n\t// Track if forcePathStyle was explicitly set by user (config or URL query param).\n\tforcePathStyleSet := c.ForcePathStyle != nil\n\n\t// Apply provider-specific defaults for S3-compatible providers.\n\t// These settings ensure compatibility with each provider's S3 implementation.\n\tif isTigris {\n\t\t// Tigris: requires signed payloads, no MD5\n\t\tsignSetting.ApplyDefault(true)\n\t\trequireSetting.ApplyDefault(false)\n\t}\n\tif isDigitalOcean || isBackblaze || isFilebase || isScaleway || isCloudflareR2 || isMinIO || isSupabase {\n\t\t// All these providers require signed payloads (don't support UNSIGNED-PAYLOAD)\n\t\tsignSetting.ApplyDefault(true)\n\t}\n\tif !forcePathStyleSet {\n\t\t// Filebase, Backblaze B2, MinIO, and Supabase require path-style URLs\n\t\tif isFilebase || isBackblaze || isMinIO || isSupabase {\n\t\t\tforcePathStyle = true\n\t\t}\n\t}\n\n\t// Build replica.\n\tclient := s3.NewReplicaClient()\n\tclient.AccessKeyID = c.AccessKeyID\n\tclient.SecretAccessKey = c.SecretAccessKey\n\tclient.Bucket = bucket\n\tclient.Path = configPath\n\tclient.Region = region\n\tclient.Endpoint = endpoint\n\tclient.ForcePathStyle = forcePathStyle\n\tclient.SkipVerify = skipVerify\n\n\tclient.SignPayload = signSetting.value\n\tclient.RequireContentMD5 = requireSetting.value\n\n\tif isCloudflareR2 {\n\t\tclient.Concurrency = s3.DefaultR2Concurrency\n\t}\n\n\t// Apply upload configuration if specified.\n\tif c.PartSize != nil {\n\t\tclient.PartSize = int64(*c.PartSize)\n\t}\n\tif c.Concurrency != nil {\n\t\tclient.Concurrency = *c.Concurrency\n\t}\n\n\t// Apply SSE-C configuration if specified.\n\tif c.SSECustomerKey != \"\" || c.SSECustomerKeyPath != \"\" {\n\t\tclient.SSECustomerAlgorithm = c.SSECustomerAlgorithm\n\t\tif client.SSECustomerAlgorithm == \"\" {\n\t\t\tclient.SSECustomerAlgorithm = \"AES256\"\n\t\t}\n\n\t\t// Read key from file if path is specified, otherwise use direct value.\n\t\tif c.SSECustomerKeyPath != \"\" {\n\t\t\tkeyPath := c.SSECustomerKeyPath\n\t\t\t// Expand ~ to home directory\n\t\t\tif strings.HasPrefix(keyPath, \"~\") {\n\t\t\t\thome, err := os.UserHomeDir()\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, fmt.Errorf(\"cannot expand home directory for sse-customer-key-path: %w\", err)\n\t\t\t\t}\n\t\t\t\tkeyPath = home + keyPath[1:]\n\t\t\t}\n\t\t\tkeyData, err := os.ReadFile(keyPath)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"cannot read sse-customer-key-path %q: %w\", c.SSECustomerKeyPath, err)\n\t\t\t}\n\t\t\tclient.SSECustomerKey = strings.TrimSpace(string(keyData))\n\t\t} else {\n\t\t\tclient.SSECustomerKey = c.SSECustomerKey\n\t\t}\n\t}\n\n\t// Apply SSE-KMS configuration if specified.\n\tif c.SSEKMSKeyID != \"\" {\n\t\tclient.SSEKMSKeyID = c.SSEKMSKeyID\n\t}\n\n\treturn client, nil\n}\n\n// newGSReplicaClientFromConfig returns a new instance of gs.ReplicaClient built from config.\nfunc newGSReplicaClientFromConfig(c *ReplicaConfig, _ *litestream.Replica) (_ *gs.ReplicaClient, err error) {\n\t// Ensure URL & constituent parts are not both specified.\n\tif c.URL != \"\" && c.Path != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & path for gs replica\")\n\t} else if c.URL != \"\" && c.Bucket != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & bucket for gs replica\")\n\t}\n\n\tbucket, configPath := c.Bucket, c.Path\n\n\t// Apply settings from URL, if specified.\n\tif c.URL != \"\" {\n\t\t_, uhost, upath, err := litestream.ParseReplicaURL(c.URL)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Only apply URL parts to field that have not been overridden.\n\t\tif configPath == \"\" {\n\t\t\tconfigPath = upath\n\t\t}\n\t\tif bucket == \"\" {\n\t\t\tbucket = uhost\n\t\t}\n\t}\n\n\t// Ensure required settings are set.\n\tif bucket == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for gs replica\")\n\t}\n\n\t// Build replica.\n\tclient := gs.NewReplicaClient()\n\tclient.Bucket = bucket\n\tclient.Path = configPath\n\treturn client, nil\n}\n\n// newABSReplicaClientFromConfig returns a new instance of abs.ReplicaClient built from config.\nfunc newABSReplicaClientFromConfig(c *ReplicaConfig, _ *litestream.Replica) (_ *abs.ReplicaClient, err error) {\n\t// Ensure URL & constituent parts are not both specified.\n\tif c.URL != \"\" && c.Path != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & path for abs replica\")\n\t} else if c.URL != \"\" && c.Bucket != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & bucket for abs replica\")\n\t}\n\n\t// Build replica.\n\tclient := abs.NewReplicaClient()\n\tclient.AccountName = c.AccountName\n\tclient.AccountKey = c.AccountKey\n\tclient.SASToken = c.SASToken\n\tclient.Bucket = c.Bucket\n\tclient.Path = c.Path\n\tclient.Endpoint = c.Endpoint\n\n\t// Apply settings from URL, if specified.\n\tif c.URL != \"\" {\n\t\tu, err := url.Parse(c.URL)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif client.AccountName == \"\" && u.User != nil {\n\t\t\tclient.AccountName = u.User.Username()\n\t\t}\n\t\tif client.Bucket == \"\" {\n\t\t\tclient.Bucket = u.Host\n\t\t}\n\t\tif client.Path == \"\" {\n\t\t\tclient.Path = strings.TrimPrefix(path.Clean(u.Path), \"/\")\n\t\t}\n\t}\n\n\t// Ensure required settings are set.\n\tif client.Bucket == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for abs replica\")\n\t}\n\n\treturn client, nil\n}\n\n// newSFTPReplicaClientFromConfig returns a new instance of sftp.ReplicaClient built from config.\nfunc newSFTPReplicaClientFromConfig(c *ReplicaConfig, _ *litestream.Replica) (_ *sftp.ReplicaClient, err error) {\n\t// Ensure URL & constituent parts are not both specified.\n\tif c.URL != \"\" && c.Path != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & path for sftp replica\")\n\t} else if c.URL != \"\" && c.Host != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & host for sftp replica\")\n\t}\n\n\thost, user, password, path := c.Host, c.User, c.Password, c.Path\n\n\t// Apply settings from URL, if specified.\n\tif c.URL != \"\" {\n\t\tu, err := url.Parse(c.URL)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Only apply URL parts to field that have not been overridden.\n\t\tif host == \"\" {\n\t\t\thost = u.Host\n\t\t}\n\t\tif user == \"\" && u.User != nil {\n\t\t\tuser = u.User.Username()\n\t\t}\n\t\tif password == \"\" && u.User != nil {\n\t\t\tpassword, _ = u.User.Password()\n\t\t}\n\t\tif path == \"\" {\n\t\t\tpath = u.Path\n\t\t}\n\t}\n\n\t// Ensure required settings are set.\n\tif host == \"\" {\n\t\treturn nil, fmt.Errorf(\"host required for sftp replica\")\n\t} else if user == \"\" {\n\t\treturn nil, fmt.Errorf(\"user required for sftp replica\")\n\t}\n\n\t// Build replica.\n\tclient := sftp.NewReplicaClient()\n\tclient.Host = host\n\tclient.User = user\n\tclient.Password = password\n\tclient.Path = path\n\tclient.KeyPath = c.KeyPath\n\tclient.HostKey = c.HostKey\n\n\t// Set concurrent writes if specified, otherwise use default (true)\n\tif c.ConcurrentWrites != nil {\n\t\tclient.ConcurrentWrites = *c.ConcurrentWrites\n\t}\n\n\treturn client, nil\n}\n\n// newWebDAVReplicaClientFromConfig returns a new instance of webdav.ReplicaClient built from config.\nfunc newWebDAVReplicaClientFromConfig(c *ReplicaConfig, _ *litestream.Replica) (_ *webdav.ReplicaClient, err error) {\n\t// Ensure URL & constituent parts are not both specified.\n\tif c.URL != \"\" && c.Path != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & path for webdav replica\")\n\t} else if c.URL != \"\" && c.WebDAVURL != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & webdav-url for webdav replica\")\n\t}\n\n\twebdavURL, username, password, path := c.WebDAVURL, c.WebDAVUsername, c.WebDAVPassword, c.Path\n\n\t// Apply settings from URL, if specified.\n\tif c.URL != \"\" {\n\t\tu, err := url.Parse(c.URL)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Build WebDAV URL from scheme and host\n\t\tscheme := \"http\"\n\t\tif u.Scheme == \"webdavs\" {\n\t\t\tscheme = \"https\"\n\t\t}\n\t\tif webdavURL == \"\" && u.Host != \"\" {\n\t\t\twebdavURL = fmt.Sprintf(\"%s://%s\", scheme, u.Host)\n\t\t}\n\n\t\t// Extract credentials from URL\n\t\tif username == \"\" && u.User != nil {\n\t\t\tusername = u.User.Username()\n\t\t}\n\t\tif password == \"\" && u.User != nil {\n\t\t\tpassword, _ = u.User.Password()\n\t\t}\n\t\tif path == \"\" {\n\t\t\tpath = u.Path\n\t\t}\n\t}\n\n\t// Ensure required settings are set.\n\tif webdavURL == \"\" {\n\t\treturn nil, fmt.Errorf(\"webdav-url required for webdav replica\")\n\t}\n\n\t// Build replica.\n\tclient := webdav.NewReplicaClient()\n\tclient.URL = webdavURL\n\tclient.Username = username\n\tclient.Password = password\n\tclient.Path = path\n\n\treturn client, nil\n}\n\n// newNATSReplicaClientFromConfig returns a new instance of nats.ReplicaClient built from config.\nfunc newNATSReplicaClientFromConfig(c *ReplicaConfig, _ *litestream.Replica) (_ *nats.ReplicaClient, err error) {\n\t// Parse URL if provided to extract bucket name and server URL\n\tvar url, bucket string\n\tif c.URL != \"\" {\n\t\tscheme, host, bucketPath, err := litestream.ParseReplicaURL(c.URL)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"invalid NATS URL: %w\", err)\n\t\t}\n\t\tif scheme != \"nats\" {\n\t\t\treturn nil, fmt.Errorf(\"invalid scheme for NATS replica: %s\", scheme)\n\t\t}\n\n\t\t// Reconstruct URL without bucket path\n\t\tif host != \"\" {\n\t\t\turl = fmt.Sprintf(\"nats://%s\", host)\n\t\t}\n\n\t\t// Extract bucket name from path\n\t\tif bucketPath != \"\" {\n\t\t\tbucket = strings.Trim(bucketPath, \"/\")\n\t\t}\n\t}\n\n\t// Use bucket from config if not extracted from URL\n\tif bucket == \"\" {\n\t\tbucket = c.Bucket\n\t}\n\n\t// Ensure required settings are set\n\tif bucket == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for NATS replica\")\n\t}\n\n\t// Validate TLS configuration\n\t// Both client cert and key must be specified together\n\tif (c.ClientCert != \"\") != (c.ClientKey != \"\") {\n\t\treturn nil, fmt.Errorf(\"client-cert and client-key must both be specified for mutual TLS authentication\")\n\t}\n\n\t// Build replica client\n\tclient := nats.NewReplicaClient()\n\tclient.URL = url\n\tclient.BucketName = bucket\n\n\t// Set authentication options\n\tclient.JWT = c.JWT\n\tclient.Seed = c.Seed\n\tclient.Creds = c.Creds\n\tclient.NKey = c.NKey\n\tclient.Username = c.Username\n\tclient.Password = c.Password\n\tclient.Token = c.Token\n\n\t// Set TLS options\n\tclient.RootCAs = c.RootCAs\n\tclient.ClientCert = c.ClientCert\n\tclient.ClientKey = c.ClientKey\n\n\t// Set connection options with defaults\n\tif c.MaxReconnects != nil {\n\t\tclient.MaxReconnects = *c.MaxReconnects\n\t}\n\tif c.ReconnectWait != nil {\n\t\tclient.ReconnectWait = *c.ReconnectWait\n\t}\n\tif c.Timeout != nil {\n\t\tclient.Timeout = *c.Timeout\n\t}\n\n\treturn client, nil\n}\n\n// newOSSReplicaClientFromConfig returns a new instance of oss.ReplicaClient built from config.\nfunc newOSSReplicaClientFromConfig(c *ReplicaConfig, _ *litestream.Replica) (_ *oss.ReplicaClient, err error) {\n\t// Ensure URL & constituent parts are not both specified.\n\tif c.URL != \"\" && c.Path != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & path for oss replica\")\n\t} else if c.URL != \"\" && c.Bucket != \"\" {\n\t\treturn nil, fmt.Errorf(\"cannot specify url & bucket for oss replica\")\n\t}\n\n\tbucket, configPath := c.Bucket, c.Path\n\tregion, endpoint := c.Region, c.Endpoint\n\n\t// Apply settings from URL, if specified.\n\tif c.URL != \"\" {\n\t\t_, host, upath, err := litestream.ParseReplicaURL(c.URL)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tvar (\n\t\t\tubucket string\n\t\t\turegion string\n\t\t)\n\n\t\tubucket, uregion, _ = oss.ParseHost(host)\n\n\t\t// Only apply URL parts to fields that have not been overridden.\n\t\tif configPath == \"\" {\n\t\t\tconfigPath = upath\n\t\t}\n\t\tif bucket == \"\" {\n\t\t\tbucket = ubucket\n\t\t}\n\t\tif region == \"\" {\n\t\t\tregion = uregion\n\t\t}\n\t}\n\n\t// Ensure required settings are set.\n\tif bucket == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for oss replica\")\n\t}\n\n\t// Build replica client.\n\tclient := oss.NewReplicaClient()\n\tclient.AccessKeyID = c.AccessKeyID\n\tclient.AccessKeySecret = c.SecretAccessKey\n\tclient.Bucket = bucket\n\tclient.Path = configPath\n\tclient.Region = region\n\tclient.Endpoint = endpoint\n\n\t// Apply upload configuration if specified.\n\tif c.PartSize != nil {\n\t\tclient.PartSize = int64(*c.PartSize)\n\t}\n\tif c.Concurrency != nil {\n\t\tclient.Concurrency = *c.Concurrency\n\t}\n\n\treturn client, nil\n}\n\n// applyLitestreamEnv copies \"LITESTREAM\" prefixed environment variables to\n// their AWS counterparts as the \"AWS\" prefix can be confusing when using a\n// non-AWS S3-compatible service.\nfunc applyLitestreamEnv() {\n\tif v, ok := os.LookupEnv(\"LITESTREAM_ACCESS_KEY_ID\"); ok {\n\t\tif _, ok := os.LookupEnv(\"AWS_ACCESS_KEY_ID\"); !ok {\n\t\t\tos.Setenv(\"AWS_ACCESS_KEY_ID\", v)\n\t\t}\n\t}\n\tif v, ok := os.LookupEnv(\"LITESTREAM_SECRET_ACCESS_KEY\"); ok {\n\t\tif _, ok := os.LookupEnv(\"AWS_SECRET_ACCESS_KEY\"); !ok {\n\t\t\tos.Setenv(\"AWS_SECRET_ACCESS_KEY\", v)\n\t\t}\n\t}\n}\n\ntype boolSetting struct {\n\tvalue bool\n\tset bool\n}\n\nfunc newBoolSetting(defaultValue bool) boolSetting {\n\treturn boolSetting{value: defaultValue}\n}\n\nfunc (s *boolSetting) Set(value bool) {\n\ts.value = value\n\ts.set = true\n}\n\nfunc (s *boolSetting) ApplyDefault(value bool) {\n\tif !s.set {\n\t\ts.value = value\n\t}\n}\n\n// ReplicaType returns the type based on the type field or extracted from the URL.\nfunc (c *ReplicaConfig) ReplicaType() string {\n\tif replicaType := litestream.ReplicaTypeFromURL(c.URL); replicaType != \"\" {\n\t\treturn replicaType\n\t} else if c.Type != \"\" {\n\t\treturn c.Type\n\t}\n\treturn \"file\"\n}\n\n// DefaultConfigPath returns the default config path.\nfunc DefaultConfigPath() string {\n\tif v := os.Getenv(\"LITESTREAM_CONFIG\"); v != \"\" {\n\t\treturn v\n\t}\n\treturn defaultConfigPath\n}\n\nfunc registerConfigFlag(fs *flag.FlagSet) (configPath *string, noExpandEnv *bool) {\n\treturn fs.String(\"config\", \"\", \"config path\"),\n\t\tfs.Bool(\"no-expand-env\", false, \"do not expand env vars in config\")\n}\n\n// isValidHeartbeatURL checks if the URL is a valid HTTP or HTTPS URL.\nfunc isValidHeartbeatURL(u string) bool {\n\treturn strings.HasPrefix(u, \"http://\") || strings.HasPrefix(u, \"https://\")\n}\n\n// expand returns an absolute path for s.\n// It also strips SQLite connection string prefixes (sqlite://, sqlite3://).\nfunc expand(s string) (string, error) {\n\t// Strip SQLite connection string prefixes if present.\n\ts = StripSQLitePrefix(s)\n\n\t// Just expand to absolute path if there is no home directory prefix.\n\tprefix := \"~\" + string(os.PathSeparator)\n\tif s != \"~\" && !strings.HasPrefix(s, prefix) {\n\t\treturn filepath.Abs(s)\n\t}\n\n\t// Look up home directory.\n\tu, err := user.Current()\n\tif err != nil {\n\t\treturn \"\", err\n\t} else if u.HomeDir == \"\" {\n\t\treturn \"\", fmt.Errorf(\"cannot expand path %s, no home directory available\", s)\n\t}\n\n\t// Return path with tilde replaced by the home directory.\n\tif s == \"~\" {\n\t\treturn u.HomeDir, nil\n\t}\n\treturn filepath.Join(u.HomeDir, strings.TrimPrefix(s, prefix)), nil\n}\n\n// StripSQLitePrefix removes SQLite connection string prefixes (sqlite://, sqlite3://)\n// from the given path. This allows users to use standard connection string formats\n// across their tooling while Litestream extracts just the file path.\nfunc StripSQLitePrefix(s string) string {\n\tif len(s) < 9 || s[0] != 's' {\n\t\treturn s\n\t}\n\tfor _, prefix := range []string{\"sqlite3://\", \"sqlite://\"} {\n\t\tif strings.HasPrefix(s, prefix) {\n\t\t\treturn strings.TrimPrefix(s, prefix)\n\t\t}\n\t}\n\treturn s\n}\n\n// txidVar allows the flag package to parse index flags as hex-formatted TXIDs\ntype txidVar ltx.TXID\n\n// Ensure type implements interface.\nvar _ flag.Value = (*txidVar)(nil)\n\n// String returns an 8-character hexadecimal value.\nfunc (v *txidVar) String() string {\n\treturn ltx.TXID(*v).String()\n}\n\n// Set parses s into an integer from a hexadecimal value.\nfunc (v *txidVar) Set(s string) error {\n\ttxID, err := ltx.ParseTXID(s)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"invalid txid format\")\n\t}\n\t*v = txidVar(txID)\n\treturn nil\n}\n\n// levelAll is a sentinel value indicating all compaction levels should be shown.\nconst levelAll = -1\n\n// levelVar allows the flag package to parse compaction level flags.\n// Accepts integers 0-9 or \"all\" for all levels.\ntype levelVar int\n\nvar _ flag.Value = (*levelVar)(nil)\n\nfunc (v *levelVar) String() string {\n\tif *v == levelAll {\n\t\treturn \"all\"\n\t}\n\treturn strconv.Itoa(int(*v))\n}\n\nfunc (v *levelVar) Set(s string) error {\n\tif s == \"all\" {\n\t\t*v = levelAll\n\t\treturn nil\n\t}\n\tn, err := strconv.Atoi(s)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"invalid level: must be 0-%d or \\\"all\\\"\", litestream.SnapshotLevel)\n\t}\n\tif n < 0 || n > litestream.SnapshotLevel {\n\t\treturn fmt.Errorf(\"level must be between 0 and %d\", litestream.SnapshotLevel)\n\t}\n\t*v = levelVar(n)\n\treturn nil\n}\n\nfunc initLog(w io.Writer, level, typ string) {\n\tlogOptions := slog.HandlerOptions{\n\t\tLevel: slog.LevelInfo,\n\t\tReplaceAttr: internal.ReplaceAttr,\n\t}\n\n\t// Read log level from environment, if available.\n\tif v := os.Getenv(\"LOG_LEVEL\"); v != \"\" {\n\t\tlevel = v\n\t}\n\n\tswitch strings.ToUpper(level) {\n\tcase \"TRACE\":\n\t\tlogOptions.Level = internal.LevelTrace\n\tcase \"DEBUG\":\n\t\tlogOptions.Level = slog.LevelDebug\n\tcase \"INFO\":\n\t\tlogOptions.Level = slog.LevelInfo\n\tcase \"WARN\", \"WARNING\":\n\t\tlogOptions.Level = slog.LevelWarn\n\tcase \"ERROR\":\n\t\tlogOptions.Level = slog.LevelError\n\t}\n\n\tvar logHandler slog.Handler\n\tswitch typ {\n\tcase \"json\":\n\t\tlogHandler = slog.NewJSONHandler(w, &logOptions)\n\tcase \"text\", \"\":\n\t\tlogHandler = slog.NewTextHandler(w, &logOptions)\n\t}\n\n\t// Set global default logger.\n\tslog.SetDefault(slog.New(logHandler))\n}\n" }, { "path": "cmd/litestream/main_notwindows.go", "content": "//go:build !windows\n\npackage main\n\nimport (\n\t\"context\"\n\t\"os\"\n\t\"os/signal\"\n\t\"syscall\"\n)\n\nconst defaultConfigPath = \"/etc/litestream.yml\"\n\nfunc isWindowsService() (bool, error) {\n\treturn false, nil\n}\n\nfunc runWindowsService(ctx context.Context) error {\n\tpanic(\"cannot run windows service as unix process\")\n}\n\nfunc signalChan() <-chan os.Signal {\n\tch := make(chan os.Signal, 2)\n\tsignal.Notify(ch, syscall.SIGINT, syscall.SIGTERM)\n\treturn ch\n}\n" }, { "path": "cmd/litestream/main_test.go", "content": "package main_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"crypto/x509\"\n\t\"errors\"\n\t\"os\"\n\t\"os/exec\"\n\t\"os/user\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n\tmain \"github.com/benbjohnson/litestream/cmd/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/gs\"\n\t\"github.com/benbjohnson/litestream/s3\"\n\t\"github.com/benbjohnson/litestream/sftp\"\n)\n\nfunc TestOpenConfigFile(t *testing.T) {\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\t// Create a temporary file with test content\n\t\tdir := t.TempDir()\n\t\ttestContent := \"test: content\\n\"\n\t\tconfigPath := filepath.Join(dir, \"test.yml\")\n\t\tif err := os.WriteFile(configPath, []byte(testContent), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Open the file\n\t\trc, err := main.OpenConfigFile(configPath)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"failed to open config file: %v\", err)\n\t\t}\n\t\tdefer rc.Close()\n\n\t\t// Read and verify the content\n\t\tbuf := new(bytes.Buffer)\n\t\tif _, err := buf.ReadFrom(rc); err != nil {\n\t\t\tt.Fatalf(\"failed to read from config file: %v\", err)\n\t\t}\n\n\t\tif got := buf.String(); got != testContent {\n\t\t\tt.Errorf(\"content mismatch: got %q, want %q\", got, testContent)\n\t\t}\n\t})\n\n\tt.Run(\"FileNotFound\", func(t *testing.T) {\n\t\t_, err := main.OpenConfigFile(\"/nonexistent/file.yml\")\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for nonexistent file\")\n\t\t} else if !errors.Is(err, main.ErrConfigFileNotFound) {\n\t\t\tt.Errorf(\"expected ErrConfigFileNotFound, got: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestReadConfigFile(t *testing.T) {\n\t// Ensure global AWS settings are propagated down to replica configurations.\n\tt.Run(\"PropagateGlobalSettings\", func(t *testing.T) {\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\naccess-key-id: XXX\nsecret-access-key: YYY\n\ndbs:\n - path: /path/to/db\n replicas:\n - url: s3://foo/bar\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, true)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := config.AccessKeyID, `XXX`; got != want {\n\t\t\tt.Fatalf(\"AccessKeyID=%v, want %v\", got, want)\n\t\t} else if got, want := config.SecretAccessKey, `YYY`; got != want {\n\t\t\tt.Fatalf(\"SecretAccessKey=%v, want %v\", got, want)\n\t\t} else if got, want := config.DBs[0].Replicas[0].AccessKeyID, `XXX`; got != want {\n\t\t\tt.Fatalf(\"Replica.AccessKeyID=%v, want %v\", got, want)\n\t\t} else if got, want := config.DBs[0].Replicas[0].SecretAccessKey, `YYY`; got != want {\n\t\t\tt.Fatalf(\"Replica.SecretAccessKey=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\t// Ensure environment variables are expanded.\n\tt.Run(\"ExpandEnv\", func(t *testing.T) {\n\t\tos.Setenv(\"LITESTREAM_TEST_0129380\", \"/path/to/db\")\n\t\tos.Setenv(\"LITESTREAM_TEST_1872363\", \"s3://foo/bar\")\n\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: $LITESTREAM_TEST_0129380\n replicas:\n - url: ${LITESTREAM_TEST_1872363}\n - url: ${LITESTREAM_TEST_NO_SUCH_ENV}\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, true)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := config.DBs[0].Path, `/path/to/db`; got != want {\n\t\t\tt.Fatalf(\"DB.Path=%v, want %v\", got, want)\n\t\t} else if got, want := config.DBs[0].Replicas[0].URL, `s3://foo/bar`; got != want {\n\t\t\tt.Fatalf(\"Replica[0].URL=%v, want %v\", got, want)\n\t\t} else if got, want := config.DBs[0].Replicas[1].URL, ``; got != want {\n\t\t\tt.Fatalf(\"Replica[1].URL=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\t// Ensure environment variables are not expanded.\n\tt.Run(\"NoExpandEnv\", func(t *testing.T) {\n\t\tos.Setenv(\"LITESTREAM_TEST_9847533\", \"s3://foo/bar\")\n\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /path/to/db\n replicas:\n - url: ${LITESTREAM_TEST_9847533}\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := config.DBs[0].Replicas[0].URL, `${LITESTREAM_TEST_9847533}`; got != want {\n\t\t\tt.Fatalf(\"Replica.URL=%v, want %v\", got, want)\n\t\t}\n\t})\n}\n\nfunc TestNewDBFromConfig_MetaPathExpansion(t *testing.T) {\n\tu, err := user.Current()\n\tif err != nil {\n\t\tt.Skipf(\"user.Current failed: %v\", err)\n\t}\n\tif u.HomeDir == \"\" {\n\t\tt.Skip(\"no home directory available for expansion test\")\n\t}\n\n\ttmpDir := t.TempDir()\n\tdbPath := filepath.Join(tmpDir, \"db.sqlite\")\n\treplicaPath := filepath.Join(tmpDir, \"replica\")\n\tif err := os.MkdirAll(replicaPath, 0o755); err != nil {\n\t\tt.Fatalf(\"failed to create replica directory: %v\", err)\n\t}\n\n\tmetaPath := filepath.Join(\"~\", \"litestream-meta\")\n\tconfig := &main.DBConfig{\n\t\tPath: dbPath,\n\t\tMetaPath: &metaPath,\n\t\tReplica: &main.ReplicaConfig{\n\t\t\tType: \"file\",\n\t\t\tPath: replicaPath,\n\t\t},\n\t}\n\n\tdb, err := main.NewDBFromConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBFromConfig failed: %v\", err)\n\t}\n\n\texpectedMetaPath := filepath.Join(u.HomeDir, \"litestream-meta\")\n\tif got := db.MetaPath(); got != expectedMetaPath {\n\t\tt.Fatalf(\"MetaPath not expanded: got %s, want %s\", got, expectedMetaPath)\n\t}\n\tif config.MetaPath == nil || *config.MetaPath != expectedMetaPath {\n\t\tt.Fatalf(\"config MetaPath not updated: got %v, want %s\", config.MetaPath, expectedMetaPath)\n\t}\n}\n\nfunc TestNewFileReplicaFromConfig(t *testing.T) {\n\tr, err := main.NewReplicaFromConfig(&main.ReplicaConfig{Path: \"/foo\"}, nil)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t} else if client, ok := r.Client.(*file.ReplicaClient); !ok {\n\t\tt.Fatal(\"unexpected replica type\")\n\t} else if got, want := client.Path(), \"/foo\"; got != want {\n\t\tt.Fatalf(\"Path=%s, want %s\", got, want)\n\t}\n}\n\nfunc TestNewS3ReplicaFromConfig(t *testing.T) {\n\tt.Run(\"URL\", func(t *testing.T) {\n\t\tr, err := main.NewReplicaFromConfig(&main.ReplicaConfig{URL: \"s3://foo/bar\"}, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if client, ok := r.Client.(*s3.ReplicaClient); !ok {\n\t\t\tt.Fatal(\"unexpected replica type\")\n\t\t} else if got, want := client.Bucket, \"foo\"; got != want {\n\t\t\tt.Fatalf(\"Bucket=%s, want %s\", got, want)\n\t\t} else if got, want := client.Path, \"bar\"; got != want {\n\t\t\tt.Fatalf(\"Path=%s, want %s\", got, want)\n\t\t} else if got, want := client.Region, \"\"; got != want {\n\t\t\tt.Fatalf(\"Region=%s, want %s\", got, want)\n\t\t} else if got, want := client.Endpoint, \"\"; got != want {\n\t\t\tt.Fatalf(\"Endpoint=%s, want %s\", got, want)\n\t\t} else if got, want := client.ForcePathStyle, false; got != want {\n\t\t\tt.Fatalf(\"ForcePathStyle=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"MinIO\", func(t *testing.T) {\n\t\tr, err := main.NewReplicaFromConfig(&main.ReplicaConfig{URL: \"s3://foo.localhost:9000/bar\"}, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if client, ok := r.Client.(*s3.ReplicaClient); !ok {\n\t\t\tt.Fatal(\"unexpected replica type\")\n\t\t} else if got, want := client.Bucket, \"foo\"; got != want {\n\t\t\tt.Fatalf(\"Bucket=%s, want %s\", got, want)\n\t\t} else if got, want := client.Path, \"bar\"; got != want {\n\t\t\tt.Fatalf(\"Path=%s, want %s\", got, want)\n\t\t} else if got, want := client.Region, \"us-east-1\"; got != want {\n\t\t\tt.Fatalf(\"Region=%s, want %s\", got, want)\n\t\t} else if got, want := client.Endpoint, \"http://localhost:9000\"; got != want {\n\t\t\tt.Fatalf(\"Endpoint=%s, want %s\", got, want)\n\t\t} else if got, want := client.ForcePathStyle, true; got != want {\n\t\t\tt.Fatalf(\"ForcePathStyle=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"Backblaze\", func(t *testing.T) {\n\t\tr, err := main.NewReplicaFromConfig(&main.ReplicaConfig{URL: \"s3://foo.s3.us-west-000.backblazeb2.com/bar\"}, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if client, ok := r.Client.(*s3.ReplicaClient); !ok {\n\t\t\tt.Fatal(\"unexpected replica type\")\n\t\t} else if got, want := client.Bucket, \"foo\"; got != want {\n\t\t\tt.Fatalf(\"Bucket=%s, want %s\", got, want)\n\t\t} else if got, want := client.Path, \"bar\"; got != want {\n\t\t\tt.Fatalf(\"Path=%s, want %s\", got, want)\n\t\t} else if got, want := client.Region, \"us-west-000\"; got != want {\n\t\t\tt.Fatalf(\"Region=%s, want %s\", got, want)\n\t\t} else if got, want := client.Endpoint, \"https://s3.us-west-000.backblazeb2.com\"; got != want {\n\t\t\tt.Fatalf(\"Endpoint=%s, want %s\", got, want)\n\t\t} else if got, want := client.ForcePathStyle, true; got != want {\n\t\t\tt.Fatalf(\"ForcePathStyle=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"AccessPointARN\", func(t *testing.T) {\n\t\turl := \"s3://arn:aws:s3:us-east-2:123456789012:accesspoint/stream-replica\"\n\t\tr, err := main.NewReplicaFromConfig(&main.ReplicaConfig{URL: url}, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tclient, ok := r.Client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatal(\"unexpected replica type\")\n\t\t}\n\t\tif got, want := client.Bucket, \"arn:aws:s3:us-east-2:123456789012:accesspoint/stream-replica\"; got != want {\n\t\t\tt.Fatalf(\"Bucket=%s, want %s\", got, want)\n\t\t}\n\t\tif got, want := client.Path, \"\"; got != want {\n\t\t\tt.Fatalf(\"Path=%s, want %s\", got, want)\n\t\t}\n\t\tif got, want := client.Region, \"us-east-2\"; got != want {\n\t\t\tt.Fatalf(\"Region=%s, want %s\", got, want)\n\t\t}\n\t\tif got, want := client.Endpoint, \"\"; got != want {\n\t\t\tt.Fatalf(\"Endpoint=%s, want %s\", got, want)\n\t\t}\n\t\tif got, want := client.ForcePathStyle, false; got != want {\n\t\t\tt.Fatalf(\"ForcePathStyle=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"AccessPointARNWithPrefix\", func(t *testing.T) {\n\t\turl := \"s3://arn:aws:s3:us-west-1:123456789012:accesspoint/stream-replica/backups/primary\"\n\t\tr, err := main.NewReplicaFromConfig(&main.ReplicaConfig{URL: url}, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tclient, ok := r.Client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatal(\"unexpected replica type\")\n\t\t}\n\t\tif got, want := client.Bucket, \"arn:aws:s3:us-west-1:123456789012:accesspoint/stream-replica\"; got != want {\n\t\t\tt.Fatalf(\"Bucket=%s, want %s\", got, want)\n\t\t}\n\t\tif got, want := client.Path, \"backups/primary\"; got != want {\n\t\t\tt.Fatalf(\"Path=%s, want %s\", got, want)\n\t\t}\n\t\tif got, want := client.Region, \"us-west-1\"; got != want {\n\t\t\tt.Fatalf(\"Region=%s, want %s\", got, want)\n\t\t}\n\t\tif got, want := client.Endpoint, \"\"; got != want {\n\t\t\tt.Fatalf(\"Endpoint=%s, want %s\", got, want)\n\t\t}\n\t\tif got, want := client.ForcePathStyle, false; got != want {\n\t\t\tt.Fatalf(\"ForcePathStyle=%v, want %v\", got, want)\n\t\t}\n\t})\n}\n\nfunc TestNewGSReplicaFromConfig(t *testing.T) {\n\tr, err := main.NewReplicaFromConfig(&main.ReplicaConfig{URL: \"gs://foo/bar\"}, nil)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t} else if client, ok := r.Client.(*gs.ReplicaClient); !ok {\n\t\tt.Fatal(\"unexpected replica type\")\n\t} else if got, want := client.Bucket, \"foo\"; got != want {\n\t\tt.Fatalf(\"Bucket=%s, want %s\", got, want)\n\t} else if got, want := client.Path, \"bar\"; got != want {\n\t\tt.Fatalf(\"Path=%s, want %s\", got, want)\n\t}\n}\n\nfunc TestNewSFTPReplicaFromConfig(t *testing.T) {\n\thostKey := \"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAnK0+GdwOelXlAXdqLx/qvS7WHMr3rH7zW2+0DtmK5r\"\n\tr, err := main.NewReplicaFromConfig(&main.ReplicaConfig{\n\t\tURL: \"sftp://user@example.com:2222/foo\",\n\t\tReplicaSettings: main.ReplicaSettings{\n\t\t\tHostKey: hostKey,\n\t\t},\n\t}, nil)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t} else if client, ok := r.Client.(*sftp.ReplicaClient); !ok {\n\t\tt.Fatal(\"unexpected replica type\")\n\t} else if got, want := client.HostKey, hostKey; got != want {\n\t\tt.Fatalf(\"HostKey=%s, want %s\", got, want)\n\t} else if got, want := client.Host, \"example.com:2222\"; got != want {\n\t\tt.Fatalf(\"Host=%s, want %s\", got, want)\n\t} else if got, want := client.User, \"user\"; got != want {\n\t\tt.Fatalf(\"User=%s, want %s\", got, want)\n\t} else if got, want := client.Path, \"/foo\"; got != want {\n\t\tt.Fatalf(\"Path=%s, want %s\", got, want)\n\t}\n}\n\n// TestNewReplicaFromConfig_AgeEncryption verifies that age encryption configuration is rejected.\n// Age encryption is currently non-functional and would silently write plaintext data.\n// See: https://github.com/benbjohnson/litestream/issues/790\nfunc TestNewReplicaFromConfig_AgeEncryption(t *testing.T) {\n\tt.Run(\"RejectIdentities\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://foo/bar\",\n\t\t}\n\t\tconfig.Age.Identities = []string{\"AGE-SECRET-KEY-1EXAMPLE\"}\n\n\t\t_, err := main.NewReplicaFromConfig(config, nil)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when age identities are configured\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"age encryption is not currently supported\") {\n\t\t\tt.Errorf(\"expected age encryption error, got: %v\", err)\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"revert back to Litestream v0.3.x\") {\n\t\t\tt.Errorf(\"expected error to reference v0.3.x, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"RejectRecipients\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://foo/bar\",\n\t\t}\n\t\tconfig.Age.Recipients = []string{\"age1example\"}\n\n\t\t_, err := main.NewReplicaFromConfig(config, nil)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when age recipients are configured\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"age encryption is not currently supported\") {\n\t\t\tt.Errorf(\"expected age encryption error, got: %v\", err)\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"revert back to Litestream v0.3.x\") {\n\t\t\tt.Errorf(\"expected error to reference v0.3.x, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"RejectBoth\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://foo/bar\",\n\t\t}\n\t\tconfig.Age.Identities = []string{\"AGE-SECRET-KEY-1EXAMPLE\"}\n\t\tconfig.Age.Recipients = []string{\"age1example\"}\n\n\t\t_, err := main.NewReplicaFromConfig(config, nil)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when both age identities and recipients are configured\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"age encryption is not currently supported\") {\n\t\t\tt.Errorf(\"expected age encryption error, got: %v\", err)\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"revert back to Litestream v0.3.x\") {\n\t\t\tt.Errorf(\"expected error to reference v0.3.x, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"AllowEmpty\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://foo/bar\",\n\t\t}\n\n\t\t_, err := main.NewReplicaFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error when age configuration is not present: %v\", err)\n\t\t}\n\t})\n}\n\n// TestConfig_Validate_SnapshotIntervals tests validation of snapshot intervals\nfunc TestConfig_Validate_SnapshotIntervals(t *testing.T) {\n\tt.Run(\"ValidInterval\", func(t *testing.T) {\n\t\tyaml := `\nsnapshot:\n interval: 1h\n retention: 24h\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// Verify the values were set as expected\n\t\tif config.Snapshot.Interval == nil {\n\t\t\tt.Fatal(\"expected snapshot interval to be set\")\n\t\t}\n\t\tif *config.Snapshot.Interval != 1*time.Hour {\n\t\t\tt.Errorf(\"expected snapshot interval of 1h, got %v\", *config.Snapshot.Interval)\n\t\t}\n\n\t\tif config.Snapshot.Retention == nil {\n\t\t\tt.Fatal(\"expected snapshot retention to be set\")\n\t\t}\n\t\tif *config.Snapshot.Retention != 24*time.Hour {\n\t\t\tt.Errorf(\"expected snapshot retention of 24h, got %v\", *config.Snapshot.Retention)\n\t\t}\n\t})\n\n\tt.Run(\"ZeroInterval\", func(t *testing.T) {\n\t\tyaml := `\nsnapshot:\n interval: 0s\n retention: 24h\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for zero interval\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidSnapshotInterval) {\n\t\t\tt.Errorf(\"expected ErrInvalidSnapshotInterval, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ZeroRetention\", func(t *testing.T) {\n\t\tyaml := `\nsnapshot:\n interval: 1h\n retention: 0s\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for zero retention\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidSnapshotRetention) {\n\t\t\tt.Errorf(\"expected ErrInvalidSnapshotRetention, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NegativeInterval\", func(t *testing.T) {\n\t\tyaml := `\nsnapshot:\n interval: -1h\n retention: 24h\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for negative interval\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidSnapshotInterval) {\n\t\t\tt.Errorf(\"expected ErrInvalidSnapshotInterval, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NotSpecified\", func(t *testing.T) {\n\t\tyaml := `\n# snapshot section not specified\ndbs:\n - path: /tmp/test.db\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// When snapshot section is not specified, defaults should be applied\n\t\tif config.Snapshot.Interval == nil {\n\t\t\tt.Fatal(\"expected snapshot interval to have default value\")\n\t\t}\n\t\tif *config.Snapshot.Interval != 24*time.Hour {\n\t\t\tt.Errorf(\"expected default snapshot interval of 24h, got %v\", *config.Snapshot.Interval)\n\t\t}\n\n\t\tif config.Snapshot.Retention == nil {\n\t\t\tt.Fatal(\"expected snapshot retention to have default value\")\n\t\t}\n\t\tif *config.Snapshot.Retention != 24*time.Hour {\n\t\t\tt.Errorf(\"expected default snapshot retention of 24h, got %v\", *config.Snapshot.Retention)\n\t\t}\n\t})\n\n\tt.Run(\"EmptySection\", func(t *testing.T) {\n\t\tyaml := `\nsnapshot:\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// When snapshot section is empty, defaults should be preserved\n\t\tif config.Snapshot.Interval == nil {\n\t\t\tt.Fatal(\"expected snapshot interval to have default value\")\n\t\t}\n\t\tif *config.Snapshot.Interval != 24*time.Hour {\n\t\t\tt.Errorf(\"expected default snapshot interval of 24h, got %v\", *config.Snapshot.Interval)\n\t\t}\n\n\t\tif config.Snapshot.Retention == nil {\n\t\t\tt.Fatal(\"expected snapshot retention to have default value\")\n\t\t}\n\t\tif *config.Snapshot.Retention != 24*time.Hour {\n\t\t\tt.Errorf(\"expected default snapshot retention of 24h, got %v\", *config.Snapshot.Retention)\n\t\t}\n\t})\n}\n\nfunc TestConfig_Validate_ValidationInterval(t *testing.T) {\n\tt.Run(\"ValidInterval\", func(t *testing.T) {\n\t\tyaml := `\nvalidation:\n interval: 5m\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\tif config.Validation.Interval == nil {\n\t\t\tt.Fatal(\"expected validation interval to be set\")\n\t\t}\n\t\tif *config.Validation.Interval != 5*time.Minute {\n\t\t\tt.Errorf(\"expected validation interval of 5m, got %v\", *config.Validation.Interval)\n\t\t}\n\t})\n\n\tt.Run(\"NotSpecified\", func(t *testing.T) {\n\t\tyaml := `\ndbs:\n - path: /tmp/test.db\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// When validation section is not specified, interval should be nil (disabled)\n\t\tif config.Validation.Interval != nil {\n\t\t\tt.Errorf(\"expected validation interval to be nil, got %v\", *config.Validation.Interval)\n\t\t}\n\t})\n}\n\nfunc TestParseReplicaURL_AccessPoint(t *testing.T) {\n\tt.Run(\"WithPrefix\", func(t *testing.T) {\n\t\tscheme, host, urlPath, err := litestream.ParseReplicaURL(\"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/db-access/backups/prod\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif scheme != \"s3\" {\n\t\t\tt.Fatalf(\"scheme=%s, want s3\", scheme)\n\t\t}\n\t\tif host != \"arn:aws:s3:us-east-1:123456789012:accesspoint/db-access\" {\n\t\t\tt.Fatalf(\"host=%s, want arn:aws:s3:us-east-1:123456789012:accesspoint/db-access\", host)\n\t\t}\n\t\tif urlPath != \"backups/prod\" {\n\t\t\tt.Fatalf(\"path=%s, want backups/prod\", urlPath)\n\t\t}\n\t})\n\n\tt.Run(\"Invalid\", func(t *testing.T) {\n\t\tif _, _, _, err := litestream.ParseReplicaURL(\"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/\"); err == nil {\n\t\t\tt.Fatal(\"expected error\")\n\t\t}\n\t})\n}\n\nfunc TestConfig_Validate_L0Retention(t *testing.T) {\n\tt.Run(\"ZeroRetention\", func(t *testing.T) {\n\t\tyaml := `\nl0-retention: 0s\ndbs:\n - path: /tmp/test.db\n replica:\n url: file:///tmp/replica\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for zero l0 retention\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidL0Retention) {\n\t\t\tt.Errorf(\"expected ErrInvalidL0Retention, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ZeroCheckInterval\", func(t *testing.T) {\n\t\tyaml := `\nl0-retention-check-interval: 0s\ndbs:\n - path: /tmp/test.db\n replica:\n url: file:///tmp/replica\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for zero l0 retention check interval\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidL0RetentionCheckInterval) {\n\t\t\tt.Errorf(\"expected ErrInvalidL0RetentionCheckInterval, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"DefaultsApplied\", func(t *testing.T) {\n\t\tyaml := `\ndbs:\n - path: /tmp/test.db\n replica:\n url: file:///tmp/replica\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif config.L0Retention == nil {\n\t\t\tt.Fatal(\"expected default l0 retention to be set\")\n\t\t}\n\t\tif *config.L0Retention != litestream.DefaultL0Retention {\n\t\t\tt.Errorf(\"expected default l0 retention %v, got %v\", litestream.DefaultL0Retention, *config.L0Retention)\n\t\t}\n\t\tif config.L0RetentionCheckInterval == nil {\n\t\t\tt.Fatal(\"expected default l0 retention check interval to be set\")\n\t\t}\n\t\tif *config.L0RetentionCheckInterval != litestream.DefaultL0RetentionCheckInterval {\n\t\t\tt.Errorf(\"expected default l0 retention check interval %v, got %v\", litestream.DefaultL0RetentionCheckInterval, *config.L0RetentionCheckInterval)\n\t\t}\n\t})\n}\n\n// TestConfig_Validate_SyncIntervals tests validation of replica sync intervals\nfunc TestConfig_Validate_SyncIntervals(t *testing.T) {\n\tt.Run(\"ValidSyncInterval\", func(t *testing.T) {\n\t\tyaml := `\ndbs:\n - path: /tmp/test.db\n replica:\n url: file:///tmp/replica\n sync-interval: 30s\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// Verify the sync interval was set correctly\n\t\tif len(config.DBs) != 1 {\n\t\t\tt.Fatal(\"expected one database\")\n\t\t}\n\t\tif config.DBs[0].Replica == nil {\n\t\t\tt.Fatal(\"expected replica to be set\")\n\t\t}\n\t\tif config.DBs[0].Replica.SyncInterval == nil {\n\t\t\tt.Fatal(\"expected sync interval to be set\")\n\t\t}\n\t\tif *config.DBs[0].Replica.SyncInterval != 30*time.Second {\n\t\t\tt.Errorf(\"expected sync interval of 30s, got %v\", *config.DBs[0].Replica.SyncInterval)\n\t\t}\n\t})\n\n\tt.Run(\"ZeroSyncInterval\", func(t *testing.T) {\n\t\tyaml := `\ndbs:\n - path: /tmp/test.db\n replica:\n url: file:///tmp/replica\n sync-interval: 0s\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for zero sync interval\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidSyncInterval) {\n\t\t\tt.Errorf(\"expected ErrInvalidSyncInterval, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NegativeSyncInterval\", func(t *testing.T) {\n\t\tyaml := `\ndbs:\n - path: /tmp/test.db\n replica:\n url: file:///tmp/replica\n sync-interval: -30s\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for negative sync interval\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidSyncInterval) {\n\t\t\tt.Errorf(\"expected ErrInvalidSyncInterval, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NotSpecifiedSyncInterval\", func(t *testing.T) {\n\t\tyaml := `\ndbs:\n - path: /tmp/test.db\n replica:\n url: file:///tmp/replica\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// When sync-interval is not specified, it should remain nil\n\t\t// The default will be applied when the replica is created\n\t\tif len(config.DBs) != 1 {\n\t\t\tt.Fatal(\"expected one database\")\n\t\t}\n\t\tif config.DBs[0].Replica == nil {\n\t\t\tt.Fatal(\"expected replica to be set\")\n\t\t}\n\t\tif config.DBs[0].Replica.SyncInterval != nil {\n\t\t\tt.Errorf(\"expected sync-interval to be nil when not specified, got %v\", *config.DBs[0].Replica.SyncInterval)\n\t\t}\n\t})\n\n\tt.Run(\"MultipleReplicasWithZero\", func(t *testing.T) {\n\t\tyaml := `\ndbs:\n - path: /tmp/test.db\n replicas:\n - url: file:///tmp/replica1\n sync-interval: 30s\n - url: file:///tmp/replica2\n sync-interval: 0s\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for zero sync interval in second replica\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidSyncInterval) {\n\t\t\tt.Errorf(\"expected ErrInvalidSyncInterval, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ValidMultipleReplicas\", func(t *testing.T) {\n\t\tyaml := `\ndbs:\n - path: /tmp/test.db\n replicas:\n - url: file:///tmp/replica1\n sync-interval: 30s\n - url: file:///tmp/replica2\n sync-interval: 1m\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// Verify both replicas have correct intervals\n\t\tif len(config.DBs) != 1 {\n\t\t\tt.Fatal(\"expected one database\")\n\t\t}\n\t\tif len(config.DBs[0].Replicas) != 2 {\n\t\t\tt.Fatal(\"expected two replicas\")\n\t\t}\n\n\t\t// Check first replica\n\t\tif config.DBs[0].Replicas[0].SyncInterval == nil {\n\t\t\tt.Fatal(\"expected first replica sync interval to be set\")\n\t\t}\n\t\tif *config.DBs[0].Replicas[0].SyncInterval != 30*time.Second {\n\t\t\tt.Errorf(\"expected first replica sync interval of 30s, got %v\", *config.DBs[0].Replicas[0].SyncInterval)\n\t\t}\n\n\t\t// Check second replica\n\t\tif config.DBs[0].Replicas[1].SyncInterval == nil {\n\t\t\tt.Fatal(\"expected second replica sync interval to be set\")\n\t\t}\n\t\tif *config.DBs[0].Replicas[1].SyncInterval != 1*time.Minute {\n\t\t\tt.Errorf(\"expected second replica sync interval of 1m, got %v\", *config.DBs[0].Replicas[1].SyncInterval)\n\t\t}\n\t})\n}\n\n// TestConfig_Validate_CompactionLevels tests validation of compaction level intervals\nfunc TestConfig_Validate_CompactionLevels(t *testing.T) {\n\tt.Run(\"ValidLevels\", func(t *testing.T) {\n\t\tyaml := `\nlevels:\n - interval: 5m\n - interval: 1h\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// Verify the levels were set correctly\n\t\tif len(config.Levels) != 2 {\n\t\t\tt.Fatalf(\"expected 2 compaction levels, got %d\", len(config.Levels))\n\t\t}\n\t\tif config.Levels[0].Interval != 5*time.Minute {\n\t\t\tt.Errorf(\"expected level[0] interval of 5m, got %v\", config.Levels[0].Interval)\n\t\t}\n\t\tif config.Levels[1].Interval != 1*time.Hour {\n\t\t\tt.Errorf(\"expected level[1] interval of 1h, got %v\", config.Levels[1].Interval)\n\t\t}\n\t})\n\n\tt.Run(\"ZeroLevelInterval\", func(t *testing.T) {\n\t\tyaml := `\nlevels:\n - interval: 0s\n - interval: 1h\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for zero level interval\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidCompactionInterval) {\n\t\t\tt.Errorf(\"expected ErrInvalidCompactionInterval, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NegativeLevelInterval\", func(t *testing.T) {\n\t\tyaml := `\nlevels:\n - interval: 5m\n - interval: -1h\n`\n\t\t_, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for negative level interval\")\n\t\t}\n\t\tif !errors.Is(err, main.ErrInvalidCompactionInterval) {\n\t\t\tt.Errorf(\"expected ErrInvalidCompactionInterval, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NotSpecified\", func(t *testing.T) {\n\t\tyaml := `\n# levels section not specified, should use defaults\ndbs:\n - path: /tmp/test.db\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// When levels are not specified, defaults should be applied\n\t\tif len(config.Levels) != 3 {\n\t\t\tt.Fatalf(\"expected 3 default compaction levels, got %d\", len(config.Levels))\n\t\t}\n\n\t\t// Check default intervals: 30s, 5m and 1h\n\t\tif config.Levels[0].Interval != 30*time.Second {\n\t\t\tt.Errorf(\"expected default level[0] interval of 5m, got %v\", config.Levels[0].Interval)\n\t\t}\n\t\tif config.Levels[1].Interval != 5*time.Minute {\n\t\t\tt.Errorf(\"expected default level[0] interval of 5m, got %v\", config.Levels[0].Interval)\n\t\t}\n\t\tif config.Levels[2].Interval != 1*time.Hour {\n\t\t\tt.Errorf(\"expected default level[1] interval of 1h, got %v\", config.Levels[1].Interval)\n\t\t}\n\t})\n\n\tt.Run(\"CustomLevels\", func(t *testing.T) {\n\t\tyaml := `\nlevels:\n - interval: 10m\n - interval: 30m\n - interval: 2h\n`\n\t\tconfig, err := main.ParseConfig(strings.NewReader(yaml), false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// Verify three custom levels\n\t\tif len(config.Levels) != 3 {\n\t\t\tt.Fatalf(\"expected 3 compaction levels, got %d\", len(config.Levels))\n\t\t}\n\t\tif config.Levels[0].Interval != 10*time.Minute {\n\t\t\tt.Errorf(\"expected level[0] interval of 10m, got %v\", config.Levels[0].Interval)\n\t\t}\n\t\tif config.Levels[1].Interval != 30*time.Minute {\n\t\t\tt.Errorf(\"expected level[1] interval of 30m, got %v\", config.Levels[1].Interval)\n\t\t}\n\t\tif config.Levels[2].Interval != 2*time.Hour {\n\t\t\tt.Errorf(\"expected level[2] interval of 2h, got %v\", config.Levels[2].Interval)\n\t\t}\n\t})\n}\n\n// TestConfig_DefaultValues tests that default values are properly set\nfunc TestConfig_DefaultValues(t *testing.T) {\n\t// Test empty config\n\tconfig, err := main.ParseConfig(strings.NewReader(\"\"), false)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Check snapshot defaults\n\tif config.Snapshot.Interval == nil {\n\t\tt.Error(\"expected snapshot interval to have default value\")\n\t} else if *config.Snapshot.Interval != 24*time.Hour {\n\t\tt.Errorf(\"expected default snapshot interval of 24h, got %v\", *config.Snapshot.Interval)\n\t}\n\n\tif config.Snapshot.Retention == nil {\n\t\tt.Error(\"expected snapshot retention to have default value\")\n\t} else if *config.Snapshot.Retention != 24*time.Hour {\n\t\tt.Errorf(\"expected default snapshot retention of 24h, got %v\", *config.Snapshot.Retention)\n\t}\n}\n\n// TestParseByteSize tests the ParseByteSize function with various inputs,\n// including IEC units (MiB, GiB) and decimal values that require proper rounding.\nfunc TestParseByteSize(t *testing.T) {\n\ttests := []struct {\n\t\tinput string\n\t\twant int64\n\t\twantErr bool\n\t}{\n\t\t// IEC units (base 1024) - the most important fix for AWS/B2 docs compatibility\n\t\t{\"1MiB\", 1024 * 1024, false},\n\t\t{\"5MiB\", 5 * 1024 * 1024, false},\n\t\t{\"1GiB\", 1024 * 1024 * 1024, false},\n\t\t{\"1TiB\", 1024 * 1024 * 1024 * 1024, false},\n\t\t{\"1024KiB\", 1024 * 1024, false},\n\n\t\t// SI units (base 1000) - traditional metric units\n\t\t{\"1MB\", 1000 * 1000, false},\n\t\t{\"5MB\", 5 * 1000 * 1000, false},\n\t\t{\"1GB\", 1000 * 1000 * 1000, false},\n\t\t{\"1TB\", 1000 * 1000 * 1000 * 1000, false},\n\t\t{\"1000KB\", 1000 * 1000, false},\n\n\t\t// Short forms (base 1000 - SI units without the 'B')\n\t\t{\"1M\", 1000 * 1000, false},\n\t\t{\"1K\", 1000, false},\n\t\t{\"1G\", 1000 * 1000 * 1000, false},\n\t\t{\"1T\", 1000 * 1000 * 1000 * 1000, false},\n\n\t\t// Decimal values with proper rounding (no more truncation issues)\n\t\t{\"1.5MB\", 1500000, false}, // 1.5 * 1000 * 1000\n\t\t{\"1.5MiB\", 1572864, false}, // 1.5 * 1024 * 1024\n\t\t{\"0.5MB\", 500000, false}, // Should round properly, not truncate\n\t\t{\"2.5GiB\", 2684354560, false}, // 2.5 * 1024^3\n\t\t{\"100.5KB\", 100500, false}, // Decimals work with any unit\n\n\t\t// Basic units\n\t\t{\"100B\", 100, false},\n\t\t{\"100\", 100, false}, // No unit defaults to bytes\n\n\t\t// Case insensitive\n\t\t{\"1mib\", 1024 * 1024, false},\n\t\t{\"5MIB\", 5 * 1024 * 1024, false},\n\t\t{\"1gib\", 1024 * 1024 * 1024, false},\n\n\t\t// With spaces (go-humanize handles this)\n\t\t{\"1 MiB\", 1024 * 1024, false},\n\t\t{\"5 MB\", 5 * 1000 * 1000, false},\n\t\t{\"10 GiB\", 10 * 1024 * 1024 * 1024, false},\n\n\t\t// Real-world examples from AWS/Backblaze documentation\n\t\t{\"5MB\", 5000000, false}, // AWS SDK default\n\t\t{\"100MB\", 100000000, false}, // B2 recommended size\n\t\t{\"5MiB\", 5242880, false}, // The value from the original error report\n\t\t{\"1MiB\", 1048576, false}, // B2 minimum (though actually they require 5MB)\n\n\t\t// Invalid inputs\n\t\t{\"\", 0, true},\n\t\t{\"MB\", 0, true},\n\t\t{\"invalid\", 0, true},\n\t\t{\"1XB\", 0, true},\n\t\t{\"notanumber\", 0, true},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.input, func(t *testing.T) {\n\t\t\tgot, err := main.ParseByteSize(tt.input)\n\t\t\tif (err != nil) != tt.wantErr {\n\t\t\t\tt.Errorf(\"ParseByteSize(%q) error = %v, wantErr %v\", tt.input, err, tt.wantErr)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif !tt.wantErr && got != tt.want {\n\t\t\t\tt.Errorf(\"ParseByteSize(%q) = %d, want %d\", tt.input, got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestParseByteSizeOverflow tests that values larger than int64 are rejected.\nfunc TestParseByteSizeOverflow(t *testing.T) {\n\t// 10 EB (exabytes) = 10,000,000,000,000,000,000 bytes, which exceeds int64 max (9,223,372,036,854,775,807)\n\t_, err := main.ParseByteSize(\"10EB\")\n\tif err == nil {\n\t\tt.Error(\"expected error for value exceeding int64 max, got nil\")\n\t}\n\tif !strings.Contains(err.Error(), \"exceeds maximum\") {\n\t\tt.Errorf(\"expected overflow error, got: %v\", err)\n\t}\n}\n\n// TestS3ReplicaConfig_PartSizeAndConcurrency tests that part-size and concurrency\n// configuration values are properly parsed from YAML and applied to the S3 client.\n// This test addresses issue #747 where Backblaze B2's 1MB chunk size limit was\n// being exceeded due to part-size not being honored.\nfunc TestS3ReplicaConfig_PartSizeAndConcurrency(t *testing.T) {\n\tt.Run(\"WithPartSize_IEC\", func(t *testing.T) {\n\t\t// Test IEC unit (MiB) - the main fix addressing PR feedback\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /path/to/db\n replicas:\n - type: s3\n bucket: mybucket\n path: mypath\n region: us-east-1\n part-size: 5MiB\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 || len(config.DBs[0].Replicas) != 1 {\n\t\t\tt.Fatal(\"expected one database with one replica\")\n\t\t}\n\n\t\treplicaConfig := config.DBs[0].Replicas[0]\n\t\tif replicaConfig.PartSize == nil {\n\t\t\tt.Fatal(\"expected part-size to be set\")\n\t\t}\n\t\t// 5 MiB = 5 * 1024 * 1024 = 5242880 bytes\n\t\tif got, want := int64(*replicaConfig.PartSize), int64(5*1024*1024); got != want {\n\t\t\tt.Errorf(\"PartSize = %d, want %d\", got, want)\n\t\t}\n\n\t\t// Test that the value is properly applied to the client\n\t\tr, err := main.NewReplicaFromConfig(replicaConfig, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tclient, ok := r.Client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatal(\"expected S3 replica client\")\n\t\t}\n\n\t\tif got, want := client.PartSize, int64(5*1024*1024); got != want {\n\t\t\tt.Errorf(\"client.PartSize = %d, want %d\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"WithPartSize_SI\", func(t *testing.T) {\n\t\t// Test SI unit (MB) - uses base 1000\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /path/to/db\n replicas:\n - type: s3\n bucket: mybucket\n path: mypath\n region: us-east-1\n part-size: 5MB\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 || len(config.DBs[0].Replicas) != 1 {\n\t\t\tt.Fatal(\"expected one database with one replica\")\n\t\t}\n\n\t\treplicaConfig := config.DBs[0].Replicas[0]\n\t\tif replicaConfig.PartSize == nil {\n\t\t\tt.Fatal(\"expected part-size to be set\")\n\t\t}\n\t\t// 5 MB = 5 * 1000 * 1000 = 5000000 bytes (SI units use base 1000)\n\t\tif got, want := int64(*replicaConfig.PartSize), int64(5*1000*1000); got != want {\n\t\t\tt.Errorf(\"PartSize = %d, want %d\", got, want)\n\t\t}\n\n\t\t// Test that the value is properly applied to the client\n\t\tr, err := main.NewReplicaFromConfig(replicaConfig, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tclient, ok := r.Client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatal(\"expected S3 replica client\")\n\t\t}\n\n\t\tif got, want := client.PartSize, int64(5*1000*1000); got != want {\n\t\t\tt.Errorf(\"client.PartSize = %d, want %d\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"WithConcurrency\", func(t *testing.T) {\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /path/to/db\n replicas:\n - type: s3\n bucket: mybucket\n path: mypath\n region: us-east-1\n concurrency: 10\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 || len(config.DBs[0].Replicas) != 1 {\n\t\t\tt.Fatal(\"expected one database with one replica\")\n\t\t}\n\n\t\treplicaConfig := config.DBs[0].Replicas[0]\n\t\tif replicaConfig.Concurrency == nil {\n\t\t\tt.Fatal(\"expected concurrency to be set\")\n\t\t}\n\t\tif got, want := *replicaConfig.Concurrency, 10; got != want {\n\t\t\tt.Errorf(\"Concurrency = %d, want %d\", got, want)\n\t\t}\n\n\t\t// Test that the value is properly applied to the client\n\t\tr, err := main.NewReplicaFromConfig(replicaConfig, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tclient, ok := r.Client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatal(\"expected S3 replica client\")\n\t\t}\n\n\t\tif got, want := client.Concurrency, 10; got != want {\n\t\t\tt.Errorf(\"client.Concurrency = %d, want %d\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"WithBoth\", func(t *testing.T) {\n\t\t// Test both part-size (using IEC unit) and concurrency together\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /path/to/db\n replicas:\n - type: s3\n bucket: mybucket\n path: mypath\n region: us-east-1\n part-size: 10MiB\n concurrency: 10\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 || len(config.DBs[0].Replicas) != 1 {\n\t\t\tt.Fatal(\"expected one database with one replica\")\n\t\t}\n\n\t\treplicaConfig := config.DBs[0].Replicas[0]\n\n\t\t// Verify both values are parsed\n\t\tif replicaConfig.PartSize == nil {\n\t\t\tt.Fatal(\"expected part-size to be set\")\n\t\t}\n\t\t// 10 MiB = 10 * 1024 * 1024 = 10485760 bytes\n\t\tif got, want := int64(*replicaConfig.PartSize), int64(10*1024*1024); got != want {\n\t\t\tt.Errorf(\"PartSize = %d, want %d\", got, want)\n\t\t}\n\n\t\tif replicaConfig.Concurrency == nil {\n\t\t\tt.Fatal(\"expected concurrency to be set\")\n\t\t}\n\t\tif got, want := *replicaConfig.Concurrency, 10; got != want {\n\t\t\tt.Errorf(\"Concurrency = %d, want %d\", got, want)\n\t\t}\n\n\t\t// Test that both values are properly applied to the client\n\t\tr, err := main.NewReplicaFromConfig(replicaConfig, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tclient, ok := r.Client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatal(\"expected S3 replica client\")\n\t\t}\n\n\t\tif got, want := client.PartSize, int64(10*1024*1024); got != want {\n\t\t\tt.Errorf(\"client.PartSize = %d, want %d\", got, want)\n\t\t}\n\t\tif got, want := client.Concurrency, 10; got != want {\n\t\t\tt.Errorf(\"client.Concurrency = %d, want %d\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"NotSpecified\", func(t *testing.T) {\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /path/to/db\n replicas:\n - type: s3\n bucket: mybucket\n path: mypath\n region: us-east-1\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 || len(config.DBs[0].Replicas) != 1 {\n\t\t\tt.Fatal(\"expected one database with one replica\")\n\t\t}\n\n\t\treplicaConfig := config.DBs[0].Replicas[0]\n\n\t\t// When not specified, should be nil\n\t\tif replicaConfig.PartSize != nil {\n\t\t\tt.Errorf(\"expected PartSize to be nil when not specified, got %v\", *replicaConfig.PartSize)\n\t\t}\n\t\tif replicaConfig.Concurrency != nil {\n\t\t\tt.Errorf(\"expected Concurrency to be nil when not specified, got %v\", *replicaConfig.Concurrency)\n\t\t}\n\n\t\t// Test that the client is created successfully without these values\n\t\tr, err := main.NewReplicaFromConfig(replicaConfig, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tclient, ok := r.Client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatal(\"expected S3 replica client\")\n\t\t}\n\n\t\t// When not specified, client should have default (0) values\n\t\t// The AWS SDK will use its own defaults\n\t\tif got, want := client.PartSize, int64(0); got != want {\n\t\t\tt.Errorf(\"client.PartSize = %d, want %d (AWS SDK default will be used)\", got, want)\n\t\t}\n\t\tif got, want := client.Concurrency, 0; got != want {\n\t\t\tt.Errorf(\"client.Concurrency = %d, want %d (AWS SDK default will be used)\", got, want)\n\t\t}\n\t})\n}\n\n// TestDBConfig_CheckpointFields tests that checkpoint-related configuration fields\n// are properly parsed from YAML and applied to the DB instance.\nfunc TestDBConfig_CheckpointFields(t *testing.T) {\n\tt.Run(\"MinCheckpointPageN\", func(t *testing.T) {\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /tmp/test.db\n min-checkpoint-page-count: 2000\n replica:\n url: file:///tmp/replica\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 {\n\t\t\tt.Fatal(\"expected one database config\")\n\t\t}\n\n\t\tdbc := config.DBs[0]\n\t\tif dbc.MinCheckpointPageN == nil {\n\t\t\tt.Fatal(\"expected min-checkpoint-page-count to be set\")\n\t\t}\n\t\tif got, want := *dbc.MinCheckpointPageN, 2000; got != want {\n\t\t\tt.Errorf(\"MinCheckpointPageN = %d, want %d\", got, want)\n\t\t}\n\n\t\t// Test that the value is properly applied to the DB\n\t\tdb, err := main.NewDBFromConfig(dbc)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif got, want := db.MinCheckpointPageN, 2000; got != want {\n\t\t\tt.Errorf(\"db.MinCheckpointPageN = %d, want %d\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"TruncatePageN\", func(t *testing.T) {\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /tmp/test.db\n truncate-page-n: 100000\n replica:\n url: file:///tmp/replica\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 {\n\t\t\tt.Fatal(\"expected one database config\")\n\t\t}\n\n\t\tdbc := config.DBs[0]\n\t\tif dbc.TruncatePageN == nil {\n\t\t\tt.Fatal(\"expected truncate-page-n to be set\")\n\t\t}\n\t\tif got, want := *dbc.TruncatePageN, 100000; got != want {\n\t\t\tt.Errorf(\"TruncatePageN = %d, want %d\", got, want)\n\t\t}\n\n\t\t// Test that the value is properly applied to the DB\n\t\tdb, err := main.NewDBFromConfig(dbc)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif got, want := db.TruncatePageN, 100000; got != want {\n\t\t\tt.Errorf(\"db.TruncatePageN = %d, want %d\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"BothCheckpointFields\", func(t *testing.T) {\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /tmp/test.db\n min-checkpoint-page-count: 2000\n truncate-page-n: 100000\n replica:\n url: file:///tmp/replica\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 {\n\t\t\tt.Fatal(\"expected one database config\")\n\t\t}\n\n\t\tdbc := config.DBs[0]\n\t\tif dbc.MinCheckpointPageN == nil {\n\t\t\tt.Fatal(\"expected min-checkpoint-page-count to be set\")\n\t\t}\n\t\tif got, want := *dbc.MinCheckpointPageN, 2000; got != want {\n\t\t\tt.Errorf(\"MinCheckpointPageN = %d, want %d\", got, want)\n\t\t}\n\n\t\tif dbc.TruncatePageN == nil {\n\t\t\tt.Fatal(\"expected truncate-page-n to be set\")\n\t\t}\n\t\tif got, want := *dbc.TruncatePageN, 100000; got != want {\n\t\t\tt.Errorf(\"TruncatePageN = %d, want %d\", got, want)\n\t\t}\n\n\t\t// Test that both values are properly applied to the DB\n\t\tdb, err := main.NewDBFromConfig(dbc)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif got, want := db.MinCheckpointPageN, 2000; got != want {\n\t\t\tt.Errorf(\"db.MinCheckpointPageN = %d, want %d\", got, want)\n\t\t}\n\t\tif got, want := db.TruncatePageN, 100000; got != want {\n\t\t\tt.Errorf(\"db.TruncatePageN = %d, want %d\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"NotSpecified_UsesDefaults\", func(t *testing.T) {\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: /tmp/test.db\n replica:\n url: file:///tmp/replica\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 {\n\t\t\tt.Fatal(\"expected one database config\")\n\t\t}\n\n\t\tdbc := config.DBs[0]\n\t\tif dbc.MinCheckpointPageN != nil {\n\t\t\tt.Errorf(\"expected MinCheckpointPageN to be nil when not specified, got %v\", *dbc.MinCheckpointPageN)\n\t\t}\n\t\tif dbc.TruncatePageN != nil {\n\t\t\tt.Errorf(\"expected TruncatePageN to be nil when not specified, got %v\", *dbc.TruncatePageN)\n\t\t}\n\n\t\t// Test that the DB uses default values\n\t\tdb, err := main.NewDBFromConfig(dbc)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif got, want := db.MinCheckpointPageN, litestream.DefaultMinCheckpointPageN; got != want {\n\t\t\tt.Errorf(\"db.MinCheckpointPageN = %d, want default %d\", got, want)\n\t\t}\n\t\tif got, want := db.TruncatePageN, litestream.DefaultTruncatePageN; got != want {\n\t\t\tt.Errorf(\"db.TruncatePageN = %d, want default %d\", got, want)\n\t\t}\n\t})\n}\n\nfunc TestFindSQLiteDatabases(t *testing.T) {\n\t// Create a temporary directory using t.TempDir() - automatically cleaned up\n\ttmpDir := t.TempDir()\n\n\t// Create test files\n\ttestFiles := []struct {\n\t\tpath string\n\t\tisSQLite bool\n\t\tshouldFind bool\n\t}{\n\t\t{\"test1.db\", true, true},\n\t\t{\"test2.sqlite\", true, true},\n\t\t{\"test3.db\", false, false}, // Not a SQLite file\n\t\t{\"test.txt\", false, false},\n\t\t{\"subdir/test4.db\", true, true},\n\t\t{\"subdir/test5.sqlite\", true, true},\n\t\t{\"subdir/deep/test6.db\", true, true},\n\t}\n\n\t// Create test files\n\tfor _, tf := range testFiles {\n\t\tfullPath := filepath.Join(tmpDir, tf.path)\n\t\tdir := filepath.Dir(fullPath)\n\t\tif err := os.MkdirAll(dir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tfile, err := os.Create(fullPath)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif tf.isSQLite {\n\t\t\t// Write SQLite header\n\t\t\tif _, err := file.Write([]byte(\"SQLite format 3\\x00\")); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t} else {\n\t\t\t// Write non-SQLite content\n\t\t\tif _, err := file.Write([]byte(\"not a sqlite file\")); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t\tif err := file.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tt.Run(\"non-recursive *.db pattern\", func(t *testing.T) {\n\t\tdbs, err := main.FindSQLiteDatabases(tmpDir, \"*.db\", false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Should only find test1.db in root directory\n\t\tif len(dbs) != 1 {\n\t\t\tt.Errorf(\"expected 1 database, got %d\", len(dbs))\n\t\t}\n\t})\n\n\tt.Run(\"recursive *.db pattern\", func(t *testing.T) {\n\t\tdbs, err := main.FindSQLiteDatabases(tmpDir, \"*.db\", true)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Should find test1.db, test4.db, and test6.db\n\t\tif len(dbs) != 3 {\n\t\t\tt.Errorf(\"expected 3 databases, got %d\", len(dbs))\n\t\t}\n\t})\n\n\tt.Run(\"recursive *.sqlite pattern\", func(t *testing.T) {\n\t\tdbs, err := main.FindSQLiteDatabases(tmpDir, \"*.sqlite\", true)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Should find test2.sqlite and test5.sqlite\n\t\tif len(dbs) != 2 {\n\t\t\tt.Errorf(\"expected 2 databases, got %d\", len(dbs))\n\t\t}\n\t})\n\n\tt.Run(\"recursive * pattern\", func(t *testing.T) {\n\t\tdbs, err := main.FindSQLiteDatabases(tmpDir, \"*\", true)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Should find all 5 SQLite databases\n\t\tif len(dbs) != 5 {\n\t\t\tt.Errorf(\"expected 5 databases, got %d\", len(dbs))\n\t\t}\n\t})\n}\n\nfunc TestParseReplicaURLWithQuery(t *testing.T) {\n\tt.Run(\"S3WithEndpoint\", func(t *testing.T) {\n\t\turl := \"s3://mybucket/path/to/db?endpoint=localhost:9000®ion=us-east-1&forcePathStyle=true\"\n\t\tscheme, host, path, query, _, err := litestream.ParseReplicaURLWithQuery(url)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif scheme != \"s3\" {\n\t\t\tt.Errorf(\"expected scheme 's3', got %q\", scheme)\n\t\t}\n\t\tif host != \"mybucket\" {\n\t\t\tt.Errorf(\"expected host 'mybucket', got %q\", host)\n\t\t}\n\t\tif path != \"path/to/db\" {\n\t\t\tt.Errorf(\"expected path 'path/to/db', got %q\", path)\n\t\t}\n\t\tif query.Get(\"endpoint\") != \"localhost:9000\" {\n\t\t\tt.Errorf(\"expected endpoint 'localhost:9000', got %q\", query.Get(\"endpoint\"))\n\t\t}\n\t\tif query.Get(\"region\") != \"us-east-1\" {\n\t\t\tt.Errorf(\"expected region 'us-east-1', got %q\", query.Get(\"region\"))\n\t\t}\n\t\tif query.Get(\"forcePathStyle\") != \"true\" {\n\t\t\tt.Errorf(\"expected forcePathStyle 'true', got %q\", query.Get(\"forcePathStyle\"))\n\t\t}\n\t})\n\n\tt.Run(\"S3WithoutQuery\", func(t *testing.T) {\n\t\turl := \"s3://mybucket/path/to/db\"\n\t\tscheme, host, path, query, _, err := litestream.ParseReplicaURLWithQuery(url)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif scheme != \"s3\" {\n\t\t\tt.Errorf(\"expected scheme 's3', got %q\", scheme)\n\t\t}\n\t\tif host != \"mybucket\" {\n\t\t\tt.Errorf(\"expected host 'mybucket', got %q\", host)\n\t\t}\n\t\tif path != \"path/to/db\" {\n\t\t\tt.Errorf(\"expected path 'path/to/db', got %q\", path)\n\t\t}\n\t\tif len(query) != 0 {\n\t\t\tt.Errorf(\"expected no query parameters, got %v\", query)\n\t\t}\n\t})\n\n\tt.Run(\"FileURL\", func(t *testing.T) {\n\t\turl := \"file:///path/to/db\"\n\t\tscheme, host, path, query, _, err := litestream.ParseReplicaURLWithQuery(url)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif scheme != \"file\" {\n\t\t\tt.Errorf(\"expected scheme 'file', got %q\", scheme)\n\t\t}\n\t\tif host != \"\" {\n\t\t\tt.Errorf(\"expected empty host, got %q\", host)\n\t\t}\n\t\tif path != \"/path/to/db\" {\n\t\t\tt.Errorf(\"expected path '/path/to/db', got %q\", path)\n\t\t}\n\t\tif query != nil {\n\t\t\tt.Errorf(\"expected nil query for file URL, got %v\", query)\n\t\t}\n\t})\n\n\tt.Run(\"BackwardCompatibility\", func(t *testing.T) {\n\t\t// Test that ParseReplicaURL still works as before\n\t\turl := \"s3://mybucket/path/to/db?endpoint=localhost:9000\"\n\t\tscheme, host, path, err := litestream.ParseReplicaURL(url)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif scheme != \"s3\" {\n\t\t\tt.Errorf(\"expected scheme 's3', got %q\", scheme)\n\t\t}\n\t\tif host != \"mybucket\" {\n\t\t\tt.Errorf(\"expected host 'mybucket', got %q\", host)\n\t\t}\n\t\tif path != \"path/to/db\" {\n\t\t\tt.Errorf(\"expected path 'path/to/db', got %q\", path)\n\t\t}\n\t})\n\n\tt.Run(\"S3TigrisExample\", func(t *testing.T) {\n\t\turl := \"s3://mybucket/db?endpoint=fly.storage.tigris.dev®ion=auto\"\n\t\tscheme, host, path, query, _, err := litestream.ParseReplicaURLWithQuery(url)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif scheme != \"s3\" {\n\t\t\tt.Errorf(\"expected scheme 's3', got %q\", scheme)\n\t\t}\n\t\tif host != \"mybucket\" {\n\t\t\tt.Errorf(\"expected host 'mybucket', got %q\", host)\n\t\t}\n\t\tif path != \"db\" {\n\t\t\tt.Errorf(\"expected path 'db', got %q\", path)\n\t\t}\n\t\tif query.Get(\"endpoint\") != \"fly.storage.tigris.dev\" {\n\t\t\tt.Errorf(\"expected endpoint 'fly.storage.tigris.dev', got %q\", query.Get(\"endpoint\"))\n\t\t}\n\t\tif query.Get(\"region\") != \"auto\" {\n\t\t\tt.Errorf(\"expected region 'auto', got %q\", query.Get(\"region\"))\n\t\t}\n\t})\n\n\tt.Run(\"S3WithSkipVerify\", func(t *testing.T) {\n\t\turl := \"s3://mybucket/db?endpoint=self-signed.local&skipVerify=true\"\n\t\t_, _, _, query, _, err := litestream.ParseReplicaURLWithQuery(url)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif query.Get(\"skipVerify\") != \"true\" {\n\t\t\tt.Errorf(\"expected skipVerify 'true', got %q\", query.Get(\"skipVerify\"))\n\t\t}\n\t})\n}\n\nfunc TestIsSQLiteDatabase(t *testing.T) {\n\t// Create temporary test files using t.TempDir() - automatically cleaned up\n\ttmpDir := t.TempDir()\n\n\tt.Run(\"valid SQLite file\", func(t *testing.T) {\n\t\tpath := filepath.Join(tmpDir, \"valid.db\")\n\t\tfile, err := os.Create(path)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := file.Write([]byte(\"SQLite format 3\\x00\")); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := file.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif !main.IsSQLiteDatabase(path) {\n\t\t\tt.Error(\"expected file to be identified as SQLite database\")\n\t\t}\n\t})\n\n\tt.Run(\"invalid SQLite file\", func(t *testing.T) {\n\t\tpath := filepath.Join(tmpDir, \"invalid.db\")\n\t\tfile, err := os.Create(path)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := file.Write([]byte(\"not a sqlite file\")); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := file.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif main.IsSQLiteDatabase(path) {\n\t\t\tt.Error(\"expected file to NOT be identified as SQLite database\")\n\t\t}\n\t})\n\n\tt.Run(\"non-existent file\", func(t *testing.T) {\n\t\tpath := filepath.Join(tmpDir, \"doesnotexist.db\")\n\t\tif main.IsSQLiteDatabase(path) {\n\t\t\tt.Error(\"expected non-existent file to NOT be identified as SQLite database\")\n\t\t}\n\t})\n}\n\nfunc TestDBConfigValidation(t *testing.T) {\n\tt.Run(\"both path and dir specified\", func(t *testing.T) {\n\t\tconfig := main.Config{\n\t\t\tDBs: []*main.DBConfig{\n\t\t\t\t{\n\t\t\t\t\tPath: \"/path/to/db.sqlite\",\n\t\t\t\t\tDir: \"/path/to/dir\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\terr := config.Validate()\n\t\tif err == nil {\n\t\t\tt.Error(\"expected validation error when both path and dir are specified\")\n\t\t}\n\t})\n\n\tt.Run(\"neither path nor dir specified\", func(t *testing.T) {\n\t\tconfig := main.Config{\n\t\t\tDBs: []*main.DBConfig{\n\t\t\t\t{},\n\t\t\t},\n\t\t}\n\n\t\terr := config.Validate()\n\t\tif err == nil {\n\t\t\tt.Error(\"expected validation error when neither path nor dir are specified\")\n\t\t}\n\t})\n\n\tt.Run(\"dir without pattern\", func(t *testing.T) {\n\t\tconfig := main.Config{\n\t\t\tDBs: []*main.DBConfig{\n\t\t\t\t{\n\t\t\t\t\tDir: \"/path/to/dir\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\terr := config.Validate()\n\t\tif err == nil {\n\t\t\tt.Error(\"expected validation error when dir is specified without pattern\")\n\t\t}\n\t})\n\n\tt.Run(\"valid path configuration\", func(t *testing.T) {\n\t\tconfig := main.DefaultConfig()\n\t\tconfig.DBs = []*main.DBConfig{\n\t\t\t{\n\t\t\t\tPath: \"/path/to/db.sqlite\",\n\t\t\t},\n\t\t}\n\n\t\terr := config.Validate()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected validation error for valid path config: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"valid directory configuration\", func(t *testing.T) {\n\t\tconfig := main.DefaultConfig()\n\t\tconfig.DBs = []*main.DBConfig{\n\t\t\t{\n\t\t\t\tDir: \"/path/to/dir\",\n\t\t\t\tPattern: \"*.db\",\n\t\t\t\tRecursive: true,\n\t\t\t},\n\t\t}\n\n\t\terr := config.Validate()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected validation error for valid directory config: %v\", err)\n\t\t}\n\t})\n}\n\n// TestNewDBsFromDirectoryConfig_UniquePaths verifies that each database discovered\n// in a directory gets a unique replica path to prevent data corruption.\nfunc TestNewDBsFromDirectoryConfig_UniquePaths(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\t// Create multiple databases\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"db1.db\"))\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"db2.db\"))\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"db3.db\"))\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.db\",\n\t\tReplica: &main.ReplicaConfig{\n\t\t\tType: \"file\",\n\t\t\tPath: \"/backup/base\",\n\t\t},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\n\tif len(dbs) != 3 {\n\t\tt.Fatalf(\"expected 3 databases, got %d\", len(dbs))\n\t}\n\n\t// Verify each has unique replica path\n\tpaths := make(map[string]bool)\n\tfor _, db := range dbs {\n\t\tif db.Replica == nil {\n\t\t\tt.Fatalf(\"database %s has no replica\", db.Path())\n\t\t}\n\t\treplicaPath := db.Replica.Client.(*file.ReplicaClient).Path()\n\t\tif paths[replicaPath] {\n\t\t\tt.Errorf(\"duplicate replica path: %s\", replicaPath)\n\t\t}\n\t\tpaths[replicaPath] = true\n\n\t\t// Verify path includes database name\n\t\tdbName := filepath.Base(db.Path())\n\t\tif !strings.Contains(replicaPath, dbName) {\n\t\t\tt.Errorf(\"replica path %s does not contain database name %s\", replicaPath, dbName)\n\t\t}\n\t}\n\n\t// Verify all paths are different\n\tif len(paths) != 3 {\n\t\tt.Errorf(\"expected 3 unique paths, got %d\", len(paths))\n\t}\n}\n\n// TestNewDBsFromDirectoryConfig_MetaPathPerDatabase ensures that each database\n// discovered via a directory config receives a unique metadata directory when a\n// base meta-path is provided.\nfunc TestNewDBsFromDirectoryConfig_MetaPathPerDatabase(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\trootDB := filepath.Join(tmpDir, \"primary.db\")\n\tcreateSQLiteDB(t, rootDB)\n\n\tnestedDir := filepath.Join(tmpDir, \"team\", \"nested\")\n\tif err := os.MkdirAll(nestedDir, 0o755); err != nil {\n\t\tt.Fatalf(\"failed to create nested directory: %v\", err)\n\t}\n\tnestedDB := filepath.Join(nestedDir, \"analytics.db\")\n\tcreateSQLiteDB(t, nestedDB)\n\n\tu, err := user.Current()\n\tif err != nil {\n\t\tt.Skipf(\"user.Current failed: %v\", err)\n\t}\n\tif u.HomeDir == \"\" {\n\t\tt.Skip(\"no home directory available for expansion test\")\n\t}\n\n\tmetaRoot := filepath.Join(\"~\", \"meta-root\")\n\texpandedMetaRoot := filepath.Join(u.HomeDir, \"meta-root\")\n\treplicaDir := filepath.Join(t.TempDir(), \"replica\")\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.db\",\n\t\tRecursive: true,\n\t\tMetaPath: &metaRoot,\n\t\tReplica: &main.ReplicaConfig{\n\t\t\tType: \"file\",\n\t\t\tPath: replicaDir,\n\t\t},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\tif len(dbs) != 2 {\n\t\tt.Fatalf(\"expected 2 databases, got %d\", len(dbs))\n\t}\n\n\texpectedMetaPaths := map[string]string{\n\t\trootDB: filepath.Join(expandedMetaRoot, \".primary.db\"+litestream.MetaDirSuffix),\n\t\tnestedDB: filepath.Join(expandedMetaRoot, \"team\", \"nested\", \".analytics.db\"+litestream.MetaDirSuffix),\n\t}\n\n\tmetaSeen := make(map[string]struct{})\n\tfor _, db := range dbs {\n\t\tmetaPath := db.MetaPath()\n\t\twant, ok := expectedMetaPaths[db.Path()]\n\t\tif !ok {\n\t\t\tt.Fatalf(\"unexpected database path returned: %s\", db.Path())\n\t\t}\n\t\tif metaPath != want {\n\t\t\tt.Fatalf(\"database %s meta path mismatch: got %s, want %s\", db.Path(), metaPath, want)\n\t\t}\n\t\tif _, dup := metaSeen[metaPath]; dup {\n\t\t\tt.Fatalf(\"duplicate meta path detected: %s\", metaPath)\n\t\t}\n\t\tmetaSeen[metaPath] = struct{}{}\n\t}\n}\n\n// TestNewDBsFromDirectoryConfig_SubdirectoryPaths verifies that the relative\n// directory structure is preserved in replica paths when using recursive scanning.\nfunc TestNewDBsFromDirectoryConfig_SubdirectoryPaths(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\t// Create databases in subdirectories\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"db1.db\"))\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"team-a\", \"db2.db\"))\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"team-b\", \"nested\", \"db3.db\"))\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.db\",\n\t\tRecursive: true,\n\t\tReplica: &main.ReplicaConfig{\n\t\t\tType: \"file\",\n\t\t\tPath: \"/backup\",\n\t\t},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\n\tif len(dbs) != 3 {\n\t\tt.Fatalf(\"expected 3 databases, got %d\", len(dbs))\n\t}\n\n\t// Build expected path mappings\n\texpectedPaths := map[string]string{\n\t\tfilepath.Join(tmpDir, \"db1.db\"): \"/backup/db1.db\",\n\t\tfilepath.Join(tmpDir, \"team-a\", \"db2.db\"): \"/backup/team-a/db2.db\",\n\t\tfilepath.Join(tmpDir, \"team-b\", \"nested\", \"db3.db\"): \"/backup/team-b/nested/db3.db\",\n\t}\n\n\tfor _, db := range dbs {\n\t\texpectedPath, ok := expectedPaths[db.Path()]\n\t\tif !ok {\n\t\t\tt.Errorf(\"unexpected database path: %s\", db.Path())\n\t\t\tcontinue\n\t\t}\n\n\t\treplicaPath := db.Replica.Client.(*file.ReplicaClient).Path()\n\t\tif replicaPath != expectedPath {\n\t\t\tt.Errorf(\"database %s: expected replica path %s, got %s\", db.Path(), expectedPath, replicaPath)\n\t\t}\n\t}\n}\n\n// TestNewDBsFromDirectoryConfig_DuplicateFilenames verifies that databases with\n// the same filename in different subdirectories get unique replica paths.\nfunc TestNewDBsFromDirectoryConfig_DuplicateFilenames(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\t// Create databases with same name in different directories\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"team-a\", \"db.sqlite\"))\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"team-b\", \"db.sqlite\"))\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.sqlite\",\n\t\tRecursive: true,\n\t\tReplica: &main.ReplicaConfig{\n\t\t\tType: \"s3\",\n\t\t\tPath: \"backups\",\n\t\t\tReplicaSettings: main.ReplicaSettings{\n\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\tRegion: \"us-east-1\",\n\t\t\t},\n\t\t},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\n\tif len(dbs) != 2 {\n\t\tt.Fatalf(\"expected 2 databases, got %d\", len(dbs))\n\t}\n\n\t// Verify paths are unique despite duplicate filenames\n\tpaths := make(map[string]bool)\n\tfor _, db := range dbs {\n\t\treplicaPath := db.Replica.Client.(*s3.ReplicaClient).Path\n\t\tif paths[replicaPath] {\n\t\t\tt.Errorf(\"duplicate replica path found: %s\", replicaPath)\n\t\t}\n\t\tpaths[replicaPath] = true\n\t}\n\n\tif len(paths) != 2 {\n\t\tt.Errorf(\"expected 2 unique paths, got %d\", len(paths))\n\t}\n\n\t// Verify paths contain subdirectory to disambiguate\n\tfor _, db := range dbs {\n\t\treplicaPath := db.Replica.Client.(*s3.ReplicaClient).Path\n\t\tif !strings.Contains(replicaPath, \"team-a\") && !strings.Contains(replicaPath, \"team-b\") {\n\t\t\tt.Errorf(\"replica path %s does not contain team subdirectory\", replicaPath)\n\t\t}\n\t}\n}\n\n// TestNewDBsFromDirectoryConfig_S3URL verifies that replica URLs receive a\n// per-database suffix so multiple databases do not overwrite one another.\nfunc TestNewDBsFromDirectoryConfig_S3URL(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"team-a\", \"db.sqlite\"))\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"team-b\", \"nested\", \"db.sqlite\"))\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.sqlite\",\n\t\tRecursive: true,\n\t\tReplica: &main.ReplicaConfig{\n\t\t\tURL: \"s3://test-bucket/backups\",\n\t\t},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\n\tif len(dbs) != 2 {\n\t\tt.Fatalf(\"expected 2 databases, got %d\", len(dbs))\n\t}\n\n\texpectedPaths := map[string]string{\n\t\tfilepath.Join(tmpDir, \"team-a\", \"db.sqlite\"): \"backups/team-a/db.sqlite\",\n\t\tfilepath.Join(tmpDir, \"team-b\", \"nested\", \"db.sqlite\"): \"backups/team-b/nested/db.sqlite\",\n\t}\n\n\tfor _, db := range dbs {\n\t\texpectedPath, ok := expectedPaths[db.Path()]\n\t\tif !ok {\n\t\t\tt.Errorf(\"unexpected database path: %s\", db.Path())\n\t\t\tcontinue\n\t\t}\n\n\t\tclient := db.Replica.Client.(*s3.ReplicaClient)\n\t\tif client.Path != expectedPath {\n\t\t\tt.Errorf(\"database %s: expected replica path %s, got %s\", db.Path(), expectedPath, client.Path)\n\t\t}\n\t}\n}\n\n// TestNewDBsFromDirectoryConfig_ReplicasArrayURL verifies URL handling when\n// using the deprecated replicas array form.\nfunc TestNewDBsFromDirectoryConfig_ReplicasArrayURL(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"db1.sqlite\"))\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"subs\", \"db2.sqlite\"))\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.sqlite\",\n\t\tRecursive: true,\n\t\tReplicas: []*main.ReplicaConfig{\n\t\t\t{\n\t\t\t\tURL: \"s3://legacy-bucket/base\",\n\t\t\t},\n\t\t},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\n\tif len(dbs) != 2 {\n\t\tt.Fatalf(\"expected 2 databases, got %d\", len(dbs))\n\t}\n\n\texpectedPaths := map[string]string{\n\t\tfilepath.Join(tmpDir, \"db1.sqlite\"): \"base/db1.sqlite\",\n\t\tfilepath.Join(tmpDir, \"subs\", \"db2.sqlite\"): \"base/subs/db2.sqlite\",\n\t}\n\n\tfor _, db := range dbs {\n\t\texpectedPath, ok := expectedPaths[db.Path()]\n\t\tif !ok {\n\t\t\tt.Errorf(\"unexpected database path: %s\", db.Path())\n\t\t\tcontinue\n\t\t}\n\n\t\tclient := db.Replica.Client.(*s3.ReplicaClient)\n\t\tif client.Path != expectedPath {\n\t\t\tt.Errorf(\"database %s: expected replica path %s, got %s\", db.Path(), expectedPath, client.Path)\n\t\t}\n\t}\n}\n\n// TestNewDBsFromDirectoryConfig_SpecialCharacters verifies that special characters\n// in database filenames are handled correctly in replica paths.\nfunc TestNewDBsFromDirectoryConfig_SpecialCharacters(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\t// Create databases with special characters\n\tspecialNames := []string{\n\t\t\"my database.db\",\n\t\t\"user@example.com.db\",\n\t\t\"tenant#1.db\",\n\t}\n\n\tfor _, name := range specialNames {\n\t\tcreateSQLiteDB(t, filepath.Join(tmpDir, name))\n\t}\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.db\",\n\t\tReplica: &main.ReplicaConfig{\n\t\t\tType: \"file\",\n\t\t\tPath: \"/backup\",\n\t\t},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\n\tif len(dbs) != len(specialNames) {\n\t\tt.Fatalf(\"expected %d databases, got %d\", len(specialNames), len(dbs))\n\t}\n\n\t// Verify each special name is in a replica path\n\tfor _, db := range dbs {\n\t\treplicaPath := db.Replica.Client.(*file.ReplicaClient).Path()\n\t\tdbName := filepath.Base(db.Path())\n\n\t\tif !strings.Contains(replicaPath, dbName) {\n\t\t\tt.Errorf(\"replica path %s does not contain database name %s\", replicaPath, dbName)\n\t\t}\n\t}\n}\n\n// TestNewDBsFromDirectoryConfig_EmptyBasePath verifies that an empty base path\n// results in the database relative path being used as the entire replica path.\nfunc TestNewDBsFromDirectoryConfig_EmptyBasePath(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"test.db\"))\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.db\",\n\t\tReplica: &main.ReplicaConfig{\n\t\t\tType: \"file\",\n\t\t\tPath: \"\", // Empty base path\n\t\t},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\n\tif len(dbs) != 1 {\n\t\tt.Fatalf(\"expected 1 database, got %d\", len(dbs))\n\t}\n\n\treplicaPath := dbs[0].Replica.Client.(*file.ReplicaClient).Path()\n\t// When base path is empty, the relative path (just filename) is used\n\t// But it's still expanded to absolute path by the file backend\n\tif !strings.HasSuffix(replicaPath, \"test.db\") {\n\t\tt.Errorf(\"expected replica path to end with 'test.db', got %s\", replicaPath)\n\t}\n}\n\n// TestNewDBsFromDirectoryConfig_ReplicasArray verifies that the deprecated\n// 'replicas' array field is handled correctly with unique paths.\nfunc TestNewDBsFromDirectoryConfig_ReplicasArray(t *testing.T) {\n\ttmpDir := t.TempDir()\n\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"db1.db\"))\n\tcreateSQLiteDB(t, filepath.Join(tmpDir, \"db2.db\"))\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.db\",\n\t\tReplicas: []*main.ReplicaConfig{\n\t\t\t{\n\t\t\t\tType: \"file\",\n\t\t\t\tPath: \"/backup\",\n\t\t\t},\n\t\t},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\n\tif len(dbs) != 2 {\n\t\tt.Fatalf(\"expected 2 databases, got %d\", len(dbs))\n\t}\n\n\t// Verify each has unique replica path\n\tpaths := make(map[string]bool)\n\tfor _, db := range dbs {\n\t\tif db.Replica == nil {\n\t\t\tt.Fatalf(\"database %s has no replica\", db.Path())\n\t\t}\n\t\treplicaPath := db.Replica.Client.(*file.ReplicaClient).Path()\n\t\tif paths[replicaPath] {\n\t\t\tt.Errorf(\"duplicate replica path: %s\", replicaPath)\n\t\t}\n\t\tpaths[replicaPath] = true\n\t}\n}\n\nfunc TestNewDBsFromDirectoryConfig_EmptyDirectoryRequiresDatabases(t *testing.T) {\n\ttmpDir := t.TempDir()\n\treplicaDir := filepath.Join(tmpDir, \"replica\")\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.db\",\n\t\tReplica: &main.ReplicaConfig{Type: \"file\", Path: replicaDir},\n\t}\n\n\tif _, err := main.NewDBsFromDirectoryConfig(config); err == nil {\n\t\tt.Fatalf(\"expected error for empty directory when watch disabled\")\n\t}\n}\n\nfunc TestNewDBsFromDirectoryConfig_EmptyDirectoryWithWatch(t *testing.T) {\n\ttmpDir := t.TempDir()\n\treplicaDir := filepath.Join(tmpDir, \"replica\")\n\n\tconfig := &main.DBConfig{\n\t\tDir: tmpDir,\n\t\tPattern: \"*.db\",\n\t\tWatch: true,\n\t\tReplica: &main.ReplicaConfig{Type: \"file\", Path: replicaDir},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t}\n\tif len(dbs) != 0 {\n\t\tt.Fatalf(\"expected 0 databases, got %d\", len(dbs))\n\t}\n}\n\nfunc TestDirectoryMonitor_DetectsDatabaseLifecycle(t *testing.T) {\n\tctx := context.Background()\n\n\trootDir := t.TempDir()\n\treplicaDir := filepath.Join(t.TempDir(), \"replicas\")\n\n\tinitialPath := filepath.Join(rootDir, \"initial.db\")\n\tcreateSQLiteDB(t, initialPath)\n\n\tconfig := &main.DBConfig{\n\t\tDir: rootDir,\n\t\tPattern: \"*.db\",\n\t\tReplica: &main.ReplicaConfig{Type: \"file\", Path: replicaDir},\n\t}\n\n\tdbs, err := main.NewDBsFromDirectoryConfig(config)\n\tif err != nil {\n\t\tt.Fatalf(\"NewDBsFromDirectoryConfig failed: %v\", err)\n\t}\n\n\tstoreConfig := main.DefaultConfig()\n\tstore := litestream.NewStore(dbs, storeConfig.CompactionLevels())\n\tstore.CompactionMonitorEnabled = false\n\n\tif err := store.Open(ctx); err != nil {\n\t\tt.Fatalf(\"unexpected error opening store: %v\", err)\n\t}\n\tdefer func() {\n\t\tif err := store.Close(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"unexpected error closing store: %v\", err)\n\t\t}\n\t}()\n\n\tmonitor, err := main.NewDirectoryMonitor(ctx, store, config, dbs)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to initialize directory monitor: %v\", err)\n\t}\n\tdefer monitor.Close()\n\n\tnewPath := filepath.Join(rootDir, \"new.db\")\n\tcreateSQLiteDB(t, newPath)\n\n\tif !waitForCondition(5*time.Second, func() bool { return hasDBPath(store.DBs(), newPath) }) {\n\t\tt.Fatalf(\"expected new database %s to be detected\", newPath)\n\t}\n\n\tif err := os.Remove(newPath); err != nil {\n\t\tt.Fatalf(\"failed to remove database: %v\", err)\n\t}\n\n\tif !waitForCondition(5*time.Second, func() bool { return !hasDBPath(store.DBs(), newPath) }) {\n\t\tt.Fatalf(\"expected database %s to be removed\", newPath)\n\t}\n}\n\nfunc TestDirectoryMonitor_RecursiveDetectsNestedDatabases(t *testing.T) {\n\tctx := context.Background()\n\trootDir := t.TempDir()\n\treplicaDir := filepath.Join(t.TempDir(), \"replicas\")\n\n\tconfig := &main.DBConfig{\n\t\tDir: rootDir,\n\t\tPattern: \"*.db\",\n\t\tRecursive: true,\n\t\tWatch: true,\n\t\tReplica: &main.ReplicaConfig{Type: \"file\", Path: replicaDir},\n\t}\n\n\tstoreConfig := main.DefaultConfig()\n\tstore := litestream.NewStore(nil, storeConfig.CompactionLevels())\n\tstore.CompactionMonitorEnabled = false\n\tif err := store.Open(ctx); err != nil {\n\t\tt.Fatalf(\"unexpected error opening store: %v\", err)\n\t}\n\tdefer func() {\n\t\tif err := store.Close(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"unexpected error closing store: %v\", err)\n\t\t}\n\t}()\n\n\tmonitor, err := main.NewDirectoryMonitor(ctx, store, config, nil)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to initialize directory monitor: %v\", err)\n\t}\n\tdefer monitor.Close()\n\n\tdeepDir := filepath.Join(rootDir, \"tenant\", \"nested\", \"deeper\")\n\tif err := os.MkdirAll(deepDir, 0755); err != nil {\n\t\tt.Fatalf(\"failed to create nested directories: %v\", err)\n\t}\n\tdeepDB := filepath.Join(deepDir, \"deep.db\")\n\tcreateSQLiteDB(t, deepDB)\n\n\tif !waitForCondition(5*time.Second, func() bool { return hasDBPath(store.DBs(), deepDB) }) {\n\t\tt.Fatalf(\"expected nested database %s to be detected\", deepDB)\n\t}\n}\n\n// createSQLiteDB creates a minimal SQLite database file for testing\nfunc createSQLiteDB(t *testing.T, path string) {\n\tt.Helper()\n\n\tdir := filepath.Dir(path)\n\tif err := os.MkdirAll(dir, 0755); err != nil {\n\t\tt.Fatalf(\"failed to create directory %s: %v\", dir, err)\n\t}\n\n\tfile, err := os.Create(path)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to create file %s: %v\", path, err)\n\t}\n\tdefer file.Close()\n\n\t// Write SQLite header\n\tif _, err := file.Write([]byte(\"SQLite format 3\\x00\")); err != nil {\n\t\tt.Fatalf(\"failed to write SQLite header: %v\", err)\n\t}\n}\n\nfunc TestNewS3ReplicaClientFromConfig(t *testing.T) {\n\tt.Run(\"URLWithEndpointQuery\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://mybucket/path/to/db?endpoint=localhost:9000®ion=us-west-2&forcePathStyle=true&skipVerify=true\",\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif client.Bucket != \"mybucket\" {\n\t\t\tt.Errorf(\"expected bucket 'mybucket', got %q\", client.Bucket)\n\t\t}\n\t\tif client.Path != \"path/to/db\" {\n\t\t\tt.Errorf(\"expected path 'path/to/db', got %q\", client.Path)\n\t\t}\n\t\tif client.Endpoint != \"http://localhost:9000\" {\n\t\t\tt.Errorf(\"expected endpoint 'http://localhost:9000', got %q\", client.Endpoint)\n\t\t}\n\t\tif client.Region != \"us-west-2\" {\n\t\t\tt.Errorf(\"expected region 'us-west-2', got %q\", client.Region)\n\t\t}\n\t\tif !client.ForcePathStyle {\n\t\t\tt.Error(\"expected ForcePathStyle to be true\")\n\t\t}\n\t\tif !client.SkipVerify {\n\t\t\tt.Error(\"expected SkipVerify to be true\")\n\t\t}\n\t})\n\n\tt.Run(\"URLWithoutQuery\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://mybucket.s3.amazonaws.com/path/to/db\",\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif client.Bucket != \"mybucket\" {\n\t\t\tt.Errorf(\"expected bucket 'mybucket', got %q\", client.Bucket)\n\t\t}\n\t\tif client.Path != \"path/to/db\" {\n\t\t\tt.Errorf(\"expected path 'path/to/db', got %q\", client.Path)\n\t\t}\n\t\t// Should use default AWS settings\n\t\tif client.Endpoint != \"\" {\n\t\t\tt.Errorf(\"expected empty endpoint for AWS S3, got %q\", client.Endpoint)\n\t\t}\n\t\tif client.ForcePathStyle {\n\t\t\tt.Error(\"expected ForcePathStyle to be false for AWS S3\")\n\t\t}\n\t})\n\n\tt.Run(\"ConfigOverridesQuery\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://mybucket/path?endpoint=from-query®ion=us-east-1\",\n\t\t\tReplicaSettings: main.ReplicaSettings{\n\t\t\t\tEndpoint: \"from-config\",\n\t\t\t\tRegion: \"us-west-1\",\n\t\t\t},\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Config values should take precedence over query params\n\t\tif client.Endpoint != \"from-config\" {\n\t\t\tt.Errorf(\"expected endpoint from config 'from-config', got %q\", client.Endpoint)\n\t\t}\n\t\tif client.Region != \"us-west-1\" {\n\t\t\tt.Errorf(\"expected region from config 'us-west-1', got %q\", client.Region)\n\t\t}\n\t})\n\n\tt.Run(\"TigrisExample\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://my-tigris-bucket/db.sqlite?endpoint=fly.storage.tigris.dev®ion=auto\",\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif client.Bucket != \"my-tigris-bucket\" {\n\t\t\tt.Errorf(\"expected bucket 'my-tigris-bucket', got %q\", client.Bucket)\n\t\t}\n\t\tif client.Endpoint != \"https://fly.storage.tigris.dev\" {\n\t\t\tt.Errorf(\"expected Tigris endpoint with https scheme, got %q\", client.Endpoint)\n\t\t}\n\t\tif client.Region != \"auto\" {\n\t\t\tt.Errorf(\"expected region 'auto' for Tigris, got %q\", client.Region)\n\t\t}\n\t\tif !client.ForcePathStyle {\n\t\t\tt.Error(\"expected ForcePathStyle to be true for custom endpoint\")\n\t\t}\n\t\tif !client.SignPayload {\n\t\t\tt.Error(\"expected SignPayload to be true for Tigris\")\n\t\t}\n\t\tif client.RequireContentMD5 {\n\t\t\tt.Error(\"expected RequireContentMD5 to be false for Tigris\")\n\t\t}\n\t})\n\n\tt.Run(\"TigrisConfigEndpoint\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tPath: \"path\",\n\t\t\tReplicaSettings: main.ReplicaSettings{\n\t\t\t\tBucket: \"mybucket\",\n\t\t\t\tEndpoint: \"https://fly.storage.tigris.dev\",\n\t\t\t},\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif !client.SignPayload {\n\t\t\tt.Error(\"expected SignPayload to be true for config-based Tigris endpoint\")\n\t\t}\n\t\tif client.RequireContentMD5 {\n\t\t\tt.Error(\"expected RequireContentMD5 to be false for config-based Tigris endpoint\")\n\t\t}\n\t})\n\n\tt.Run(\"HTTPSEndpoint\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://mybucket/path?endpoint=https://secure.storage.com®ion=us-east-1\",\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif client.Endpoint != \"https://secure.storage.com\" {\n\t\t\tt.Errorf(\"expected endpoint 'https://secure.storage.com', got %q\", client.Endpoint)\n\t\t}\n\t\tif !client.ForcePathStyle {\n\t\t\tt.Error(\"expected ForcePathStyle to be true for custom endpoint\")\n\t\t}\n\t})\n\n\tt.Run(\"QuerySigningOptions\", func(t *testing.T) {\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://bucket/db?sign-payload=true&require-content-md5=false\",\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !client.SignPayload {\n\t\t\tt.Error(\"expected SignPayload to be true when query parameter is set\")\n\t\t}\n\t\tif client.RequireContentMD5 {\n\t\t\tt.Error(\"expected RequireContentMD5 to be false when disabled via query\")\n\t\t}\n\t})\n\n\tt.Run(\"ConfigOverridesQuerySigning\", func(t *testing.T) {\n\t\tsignTrue := true\n\t\trequireFalse := false\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://bucket/db?sign-payload=false&require-content-md5=true\",\n\t\t\tReplicaSettings: main.ReplicaSettings{\n\t\t\t\tSignPayload: &signTrue,\n\t\t\t\tRequireContentMD5: &requireFalse,\n\t\t\t},\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !client.SignPayload {\n\t\t\tt.Error(\"expected config SignPayload to override query parameter\")\n\t\t}\n\t\tif client.RequireContentMD5 {\n\t\t\tt.Error(\"expected config RequireContentMD5=false to override query parameter\")\n\t\t}\n\t})\n\n\tt.Run(\"TigrisManualOverride\", func(t *testing.T) {\n\t\tsignFalse := false\n\t\trequireTrue := true\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tURL: \"s3://bucket/db?endpoint=fly.storage.tigris.dev®ion=auto\",\n\t\t\tReplicaSettings: main.ReplicaSettings{\n\t\t\t\tSignPayload: &signFalse,\n\t\t\t\tRequireContentMD5: &requireTrue,\n\t\t\t},\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.SignPayload {\n\t\t\tt.Error(\"expected manual SignPayload override to take precedence\")\n\t\t}\n\t\tif !client.RequireContentMD5 {\n\t\t\tt.Error(\"expected manual RequireContentMD5 override to take precedence\")\n\t\t}\n\t})\n\n\tt.Run(\"ProviderDefaultsParity\", func(t *testing.T) {\n\t\ttests := []struct {\n\t\t\tname string\n\t\t\tendpoint string\n\t\t}{\n\t\t\t{\"Tigris\", \"https://fly.storage.tigris.dev\"},\n\t\t\t{\"DigitalOcean\", \"https://nyc3.digitaloceanspaces.com\"},\n\t\t\t{\"Backblaze\", \"https://s3.us-west-002.backblazeb2.com\"},\n\t\t\t{\"Filebase\", \"https://s3.filebase.com\"},\n\t\t\t{\"Scaleway\", \"https://s3.fr-par.scw.cloud\"},\n\t\t\t{\"CloudflareR2\", \"https://accountid.r2.cloudflarestorage.com\"},\n\t\t\t{\"MinIO\", \"http://localhost:9000\"},\n\t\t\t{\"Supabase\", \"https://myproject.supabase.co/storage/v1/s3\"},\n\t\t}\n\n\t\tfor _, tt := range tests {\n\t\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t\turlClient, err := litestream.NewReplicaClientFromURL(\n\t\t\t\t\t\"s3://mybucket/path?endpoint=\" + tt.endpoint + \"®ion=us-east-1\",\n\t\t\t\t)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"URL factory error: %v\", err)\n\t\t\t\t}\n\t\t\t\tuc := urlClient.(*s3.ReplicaClient)\n\n\t\t\t\tcc, err := main.NewS3ReplicaClientFromConfig(&main.ReplicaConfig{\n\t\t\t\t\tPath: \"path\",\n\t\t\t\t\tReplicaSettings: main.ReplicaSettings{\n\t\t\t\t\t\tBucket: \"mybucket\",\n\t\t\t\t\t\tEndpoint: tt.endpoint,\n\t\t\t\t\t\tRegion: \"us-east-1\",\n\t\t\t\t\t},\n\t\t\t\t}, nil)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"Config factory error: %v\", err)\n\t\t\t\t}\n\n\t\t\t\tif uc.SignPayload != cc.SignPayload {\n\t\t\t\t\tt.Errorf(\"SignPayload: URL=%v, Config=%v\", uc.SignPayload, cc.SignPayload)\n\t\t\t\t}\n\t\t\t\tif uc.RequireContentMD5 != cc.RequireContentMD5 {\n\t\t\t\t\tt.Errorf(\"RequireContentMD5: URL=%v, Config=%v\", uc.RequireContentMD5, cc.RequireContentMD5)\n\t\t\t\t}\n\t\t\t\tif uc.ForcePathStyle != cc.ForcePathStyle {\n\t\t\t\t\tt.Errorf(\"ForcePathStyle: URL=%v, Config=%v\", uc.ForcePathStyle, cc.ForcePathStyle)\n\t\t\t\t}\n\t\t\t\tif uc.Concurrency != cc.Concurrency {\n\t\t\t\t\tt.Errorf(\"Concurrency: URL=%v, Config=%v\", uc.Concurrency, cc.Concurrency)\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"R2ConfigExplicitOverride\", func(t *testing.T) {\n\t\tsignFalse := false\n\t\tconfig := &main.ReplicaConfig{\n\t\t\tPath: \"path\",\n\t\t\tReplicaSettings: main.ReplicaSettings{\n\t\t\t\tBucket: \"mybucket\",\n\t\t\t\tEndpoint: \"https://accountid.r2.cloudflarestorage.com\",\n\t\t\t\tSignPayload: &signFalse,\n\t\t\t},\n\t\t}\n\n\t\tclient, err := main.NewS3ReplicaClientFromConfig(config, nil)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif client.SignPayload {\n\t\t\tt.Error(\"expected explicit SignPayload=false to override R2 default\")\n\t\t}\n\t})\n}\n\nfunc TestGlobalDefaults(t *testing.T) {\n\t// Test comprehensive global defaults functionality\n\tt.Run(\"GlobalReplicaDefaults\", func(t *testing.T) {\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\t\tsyncInterval := \"30s\"\n\t\tvalidationInterval := \"1h\"\n\n\t\tif err := os.WriteFile(filename, []byte(`\n# Global defaults for all replicas\naccess-key-id: GLOBAL_ACCESS_KEY\nsecret-access-key: GLOBAL_SECRET_KEY\nregion: us-west-2\nendpoint: custom.s3.endpoint.com\nsync-interval: `+syncInterval+`\nvalidation-interval: `+validationInterval+`\n\ndbs:\n # Database 1: Uses all global defaults\n - path: /tmp/db1.sqlite\n replica:\n type: s3\n bucket: my-bucket-1\n\n # Database 2: Overrides some defaults\n - path: /tmp/db2.sqlite\n replica:\n type: s3\n bucket: my-bucket-2\n region: us-east-1 # Override global region\n access-key-id: CUSTOM_KEY # Override global access key\n\n # Database 3: Uses legacy replicas format\n - path: /tmp/db3.sqlite\n replicas:\n - type: s3\n bucket: my-bucket-3\n # Should inherit all other global settings\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, true)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Test global settings were parsed correctly\n\t\tif got, want := config.AccessKeyID, \"GLOBAL_ACCESS_KEY\"; got != want {\n\t\t\tt.Errorf(\"config.AccessKeyID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := config.SecretAccessKey, \"GLOBAL_SECRET_KEY\"; got != want {\n\t\t\tt.Errorf(\"config.SecretAccessKey=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := config.Region, \"us-west-2\"; got != want {\n\t\t\tt.Errorf(\"config.Region=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := config.Endpoint, \"custom.s3.endpoint.com\"; got != want {\n\t\t\tt.Errorf(\"config.Endpoint=%v, want %v\", got, want)\n\t\t}\n\n\t\t// Parse expected intervals\n\t\texpectedSyncInterval, err := time.ParseDuration(syncInterval)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\texpectedValidationInterval, err := time.ParseDuration(validationInterval)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif config.SyncInterval == nil || *config.SyncInterval != expectedSyncInterval {\n\t\t\tt.Errorf(\"config.SyncInterval=%v, want %v\", config.SyncInterval, expectedSyncInterval)\n\t\t}\n\t\tif config.ValidationInterval == nil || *config.ValidationInterval != expectedValidationInterval {\n\t\t\tt.Errorf(\"config.ValidationInterval=%v, want %v\", config.ValidationInterval, expectedValidationInterval)\n\t\t}\n\n\t\t// Test Database 1: Should inherit all global defaults\n\t\tdb1 := config.DBs[0]\n\t\tif db1.Replica == nil {\n\t\t\tt.Fatal(\"db1.Replica is nil\")\n\t\t}\n\t\treplica1 := db1.Replica\n\n\t\tif got, want := replica1.AccessKeyID, \"GLOBAL_ACCESS_KEY\"; got != want {\n\t\t\tt.Errorf(\"replica1.AccessKeyID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica1.SecretAccessKey, \"GLOBAL_SECRET_KEY\"; got != want {\n\t\t\tt.Errorf(\"replica1.SecretAccessKey=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica1.Region, \"us-west-2\"; got != want {\n\t\t\tt.Errorf(\"replica1.Region=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica1.Endpoint, \"custom.s3.endpoint.com\"; got != want {\n\t\t\tt.Errorf(\"replica1.Endpoint=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica1.Bucket, \"my-bucket-1\"; got != want {\n\t\t\tt.Errorf(\"replica1.Bucket=%v, want %v\", got, want)\n\t\t}\n\t\tif replica1.SyncInterval == nil || *replica1.SyncInterval != expectedSyncInterval {\n\t\t\tt.Errorf(\"replica1.SyncInterval=%v, want %v\", replica1.SyncInterval, expectedSyncInterval)\n\t\t}\n\t\tif replica1.ValidationInterval == nil || *replica1.ValidationInterval != expectedValidationInterval {\n\t\t\tt.Errorf(\"replica1.ValidationInterval=%v, want %v\", replica1.ValidationInterval, expectedValidationInterval)\n\t\t}\n\n\t\t// Test Database 2: Should override some defaults\n\t\tdb2 := config.DBs[1]\n\t\tif db2.Replica == nil {\n\t\t\tt.Fatal(\"db2.Replica is nil\")\n\t\t}\n\t\treplica2 := db2.Replica\n\n\t\tif got, want := replica2.AccessKeyID, \"CUSTOM_KEY\"; got != want {\n\t\t\tt.Errorf(\"replica2.AccessKeyID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica2.SecretAccessKey, \"GLOBAL_SECRET_KEY\"; got != want {\n\t\t\tt.Errorf(\"replica2.SecretAccessKey=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica2.Region, \"us-east-1\"; got != want {\n\t\t\tt.Errorf(\"replica2.Region=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica2.Endpoint, \"custom.s3.endpoint.com\"; got != want {\n\t\t\tt.Errorf(\"replica2.Endpoint=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica2.Bucket, \"my-bucket-2\"; got != want {\n\t\t\tt.Errorf(\"replica2.Bucket=%v, want %v\", got, want)\n\t\t}\n\n\t\t// Test Database 3: Legacy replicas format should work\n\t\tdb3 := config.DBs[2]\n\t\tif len(db3.Replicas) != 1 {\n\t\t\tt.Fatalf(\"db3.Replicas length=%v, want 1\", len(db3.Replicas))\n\t\t}\n\t\treplica3 := db3.Replicas[0]\n\n\t\tif got, want := replica3.AccessKeyID, \"GLOBAL_ACCESS_KEY\"; got != want {\n\t\t\tt.Errorf(\"replica3.AccessKeyID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica3.SecretAccessKey, \"GLOBAL_SECRET_KEY\"; got != want {\n\t\t\tt.Errorf(\"replica3.SecretAccessKey=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica3.Region, \"us-west-2\"; got != want {\n\t\t\tt.Errorf(\"replica3.Region=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica3.Endpoint, \"custom.s3.endpoint.com\"; got != want {\n\t\t\tt.Errorf(\"replica3.Endpoint=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := replica3.Bucket, \"my-bucket-3\"; got != want {\n\t\t\tt.Errorf(\"replica3.Bucket=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\t// Test different replica types inherit appropriate defaults\n\tt.Run(\"MultipleReplicaTypes\", func(t *testing.T) {\n\t\tfilename := filepath.Join(t.TempDir(), \"litestream.yml\")\n\n\t\tif err := os.WriteFile(filename, []byte(`\n# Global defaults that apply to all supported replica types\naccess-key-id: GLOBAL_S3_KEY\nsecret-access-key: GLOBAL_S3_SECRET\nregion: global-region\nendpoint: global.endpoint.com\naccount-name: global-abs-account\naccount-key: global-abs-key\nhost: global.sftp.host\nuser: global-sftp-user\npassword: global-sftp-pass\nsync-interval: 45s\n\ndbs:\n - path: /tmp/s3.sqlite\n replica:\n type: s3\n bucket: s3-bucket\n\n - path: /tmp/abs.sqlite\n replica:\n type: abs\n bucket: abs-container\n\n - path: /tmp/sftp.sqlite\n replica:\n type: sftp\n path: /backup/path\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, true)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\texpectedSyncInterval, _ := time.ParseDuration(\"45s\")\n\n\t\t// Test S3 replica inherits S3-specific defaults\n\t\ts3Replica := config.DBs[0].Replica\n\t\tif got, want := s3Replica.AccessKeyID, \"GLOBAL_S3_KEY\"; got != want {\n\t\t\tt.Errorf(\"s3Replica.AccessKeyID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := s3Replica.SecretAccessKey, \"GLOBAL_S3_SECRET\"; got != want {\n\t\t\tt.Errorf(\"s3Replica.SecretAccessKey=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := s3Replica.Region, \"global-region\"; got != want {\n\t\t\tt.Errorf(\"s3Replica.Region=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := s3Replica.Endpoint, \"global.endpoint.com\"; got != want {\n\t\t\tt.Errorf(\"s3Replica.Endpoint=%v, want %v\", got, want)\n\t\t}\n\t\tif s3Replica.SyncInterval == nil || *s3Replica.SyncInterval != expectedSyncInterval {\n\t\t\tt.Errorf(\"s3Replica.SyncInterval=%v, want %v\", s3Replica.SyncInterval, expectedSyncInterval)\n\t\t}\n\n\t\t// Test ABS replica inherits ABS-specific defaults\n\t\tabsReplica := config.DBs[1].Replica\n\t\tif got, want := absReplica.AccountName, \"global-abs-account\"; got != want {\n\t\t\tt.Errorf(\"absReplica.AccountName=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := absReplica.AccountKey, \"global-abs-key\"; got != want {\n\t\t\tt.Errorf(\"absReplica.AccountKey=%v, want %v\", got, want)\n\t\t}\n\t\tif absReplica.SyncInterval == nil || *absReplica.SyncInterval != expectedSyncInterval {\n\t\t\tt.Errorf(\"absReplica.SyncInterval=%v, want %v\", absReplica.SyncInterval, expectedSyncInterval)\n\t\t}\n\n\t\t// Test SFTP replica inherits SFTP-specific defaults\n\t\tsftpReplica := config.DBs[2].Replica\n\t\tif got, want := sftpReplica.Host, \"global.sftp.host\"; got != want {\n\t\t\tt.Errorf(\"sftpReplica.Host=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := sftpReplica.User, \"global-sftp-user\"; got != want {\n\t\t\tt.Errorf(\"sftpReplica.User=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := sftpReplica.Password, \"global-sftp-pass\"; got != want {\n\t\t\tt.Errorf(\"sftpReplica.Password=%v, want %v\", got, want)\n\t\t}\n\t\tif sftpReplica.SyncInterval == nil || *sftpReplica.SyncInterval != expectedSyncInterval {\n\t\t\tt.Errorf(\"sftpReplica.SyncInterval=%v, want %v\", sftpReplica.SyncInterval, expectedSyncInterval)\n\t\t}\n\t})\n}\n\nfunc TestStripSQLitePrefix(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinput string\n\t\twant string\n\t}{\n\t\t{\"sqlite3 prefix\", \"sqlite3:///path/to/db.sqlite\", \"/path/to/db.sqlite\"},\n\t\t{\"sqlite prefix\", \"sqlite:///path/to/db.sqlite\", \"/path/to/db.sqlite\"},\n\t\t{\"sqlite3 relative path\", \"sqlite3://./data/db.sqlite\", \"./data/db.sqlite\"},\n\t\t{\"sqlite relative path\", \"sqlite://./data/db.sqlite\", \"./data/db.sqlite\"},\n\t\t{\"sqlite3 tilde path\", \"sqlite3://~/db.sqlite\", \"~/db.sqlite\"},\n\t\t{\"sqlite tilde path\", \"sqlite://~/db.sqlite\", \"~/db.sqlite\"},\n\t\t{\"no prefix\", \"/path/to/db.sqlite\", \"/path/to/db.sqlite\"},\n\t\t{\"relative no prefix\", \"./data/db.sqlite\", \"./data/db.sqlite\"},\n\t\t{\"tilde no prefix\", \"~/db.sqlite\", \"~/db.sqlite\"},\n\t\t{\"empty string\", \"\", \"\"},\n\t\t{\"sqlite3 windows path\", \"sqlite3://C:/data/db.sqlite\", \"C:/data/db.sqlite\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot := main.StripSQLitePrefix(tt.input)\n\t\t\tif got != tt.want {\n\t\t\t\tt.Errorf(\"StripSQLitePrefix(%q) = %q, want %q\", tt.input, got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReadConfigFile_SQLiteConnectionString(t *testing.T) {\n\tt.Run(\"ConfigWithSQLitePrefix\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"test.db\")\n\n\t\tfilename := filepath.Join(dir, \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: sqlite3://`+dbPath+`\n replicas:\n - url: file://`+filepath.Join(dir, \"replica\")+`\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, true)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"ReadConfigFile failed: %v\", err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 {\n\t\t\tt.Fatalf(\"expected 1 database, got %d\", len(config.DBs))\n\t\t}\n\n\t\tif got := config.DBs[0].Path; got != dbPath {\n\t\t\tt.Errorf(\"DBs[0].Path = %q, want %q\", got, dbPath)\n\t\t}\n\t})\n\n\tt.Run(\"ConfigWithSQLite3Prefix\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"test.db\")\n\n\t\tfilename := filepath.Join(dir, \"litestream.yml\")\n\t\tif err := os.WriteFile(filename, []byte(`\ndbs:\n - path: sqlite://`+dbPath+`\n replicas:\n - url: file://`+filepath.Join(dir, \"replica\")+`\n`[1:]), 0666); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tconfig, err := main.ReadConfigFile(filename, true)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"ReadConfigFile failed: %v\", err)\n\t\t}\n\n\t\tif len(config.DBs) != 1 {\n\t\t\tt.Fatalf(\"expected 1 database, got %d\", len(config.DBs))\n\t\t}\n\n\t\tif got := config.DBs[0].Path; got != dbPath {\n\t\t\tt.Errorf(\"DBs[0].Path = %q, want %q\", got, dbPath)\n\t\t}\n\t})\n}\n\nfunc TestX509FallbackRoots(t *testing.T) {\n\tif runtime.GOOS != \"linux\" {\n\t\tt.Skip(\"x509 fallback test requires Linux (macOS uses system keychain)\")\n\t}\n\n\tif os.Getenv(\"GO_X509_FALLBACK_TEST\") == \"1\" {\n\t\tpool, err := x509.SystemCertPool()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"SystemCertPool() error: %v\", err)\n\t\t}\n\t\tif pool.Equal(x509.NewCertPool()) {\n\t\t\tt.Fatal(\"SystemCertPool() returned empty pool; x509roots/fallback not providing certificates\")\n\t\t}\n\t\treturn\n\t}\n\n\tcmd := exec.Command(os.Args[0], \"-test.run=^TestX509FallbackRoots$\", \"-test.v\")\n\tcmd.Env = []string{\n\t\t\"GO_X509_FALLBACK_TEST=1\",\n\t\t\"SSL_CERT_FILE=/nonexistent/cert.pem\",\n\t\t\"SSL_CERT_DIR=/nonexistent/certs\",\n\t\t\"HOME=\" + t.TempDir(),\n\t}\n\tout, err := cmd.CombinedOutput()\n\tt.Logf(\"subprocess output:\\n%s\", out)\n\tif err != nil {\n\t\tt.Fatalf(\"x509 fallback roots verification failed: %v\", err)\n\t}\n}\n\nfunc hasDBPath(dbs []*litestream.DB, path string) bool {\n\tfor _, db := range dbs {\n\t\tif db.Path() == path {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc waitForCondition(timeout time.Duration, fn func() bool) bool {\n\tdeadline := time.Now().Add(timeout)\n\tfor {\n\t\tif fn() {\n\t\t\treturn true\n\t\t}\n\t\tif time.Now().After(deadline) {\n\t\t\treturn fn()\n\t\t}\n\t\ttime.Sleep(50 * time.Millisecond)\n\t}\n}\n" }, { "path": "cmd/litestream/main_windows.go", "content": "//go:build windows\n\npackage main\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"os/signal\"\n\n\t\"golang.org/x/sys/windows\"\n\t\"golang.org/x/sys/windows/svc\"\n\t\"golang.org/x/sys/windows/svc/eventlog\"\n)\n\nconst defaultConfigPath = `C:\\Litestream\\litestream.yml`\n\n// serviceName is the Windows Service name.\nconst serviceName = \"Litestream\"\n\n// isWindowsService returns true if currently executing within a Windows service.\nfunc isWindowsService() (bool, error) {\n\treturn svc.IsWindowsService()\n}\n\nfunc runWindowsService(ctx context.Context) error {\n\t// Attempt to install new log service. This will fail if already installed.\n\t// We don't log the error because we don't have anywhere to log until we open the log.\n\t_ = eventlog.InstallAsEventCreate(serviceName, eventlog.Error|eventlog.Warning|eventlog.Info)\n\n\telog, err := eventlog.Open(serviceName)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer elog.Close()\n\n\t// Set eventlog as log writer while running.\n\tslog.SetDefault(slog.New(slog.NewTextHandler((*eventlogWriter)(elog), nil)))\n\tdefer slog.SetDefault(slog.New(slog.NewTextHandler(os.Stderr, nil)))\n\n\tslog.Info(\"Litestream service starting\")\n\n\tif err := svc.Run(serviceName, &windowsService{ctx: ctx}); err != nil {\n\t\treturn errStop\n\t}\n\n\tslog.Info(\"Litestream service stopped\")\n\treturn nil\n}\n\n// windowsService is an interface adapter for svc.Handler.\ntype windowsService struct {\n\tctx context.Context\n}\n\nfunc (s *windowsService) Execute(args []string, r <-chan svc.ChangeRequest, statusCh chan<- svc.Status) (svcSpecificEC bool, exitCode uint32) {\n\tvar err error\n\n\t// Notify Windows that the service is starting up.\n\tstatusCh <- svc.Status{State: svc.StartPending}\n\n\t// Instantiate replication command and load configuration.\n\tc := NewReplicateCommand()\n\tif c.Config, err = ReadConfigFile(DefaultConfigPath(), true); err != nil {\n\t\tslog.Error(\"cannot load configuration\", \"error\", err)\n\t\treturn true, 1\n\t}\n\n\t// Execute replication command.\n\tif err := c.Run(s.ctx); err != nil {\n\t\tslog.Error(\"cannot replicate\", \"error\", err)\n\t\tstatusCh <- svc.Status{State: svc.StopPending}\n\t\treturn true, 2\n\t}\n\n\t// Notify Windows that the service is now running.\n\tstatusCh <- svc.Status{State: svc.Running, Accepts: svc.AcceptStop}\n\n\tfor {\n\t\tselect {\n\t\tcase req := <-r:\n\t\t\tswitch req.Cmd {\n\t\t\tcase svc.Stop:\n\t\t\t\tc.Close(s.ctx)\n\t\t\t\tstatusCh <- svc.Status{State: svc.StopPending}\n\t\t\t\treturn false, windows.NO_ERROR\n\t\t\tcase svc.Interrogate:\n\t\t\t\tstatusCh <- req.CurrentStatus\n\t\t\tdefault:\n\t\t\t\tslog.Error(\"Litestream service received unexpected change request\", \"cmd\", req.Cmd)\n\t\t\t}\n\t\t}\n\t}\n}\n\n// Ensure implementation implements io.Writer interface.\nvar _ io.Writer = (*eventlogWriter)(nil)\n\n// eventlogWriter is an adapter for using eventlog.Log as an io.Writer.\ntype eventlogWriter eventlog.Log\n\nfunc (w *eventlogWriter) Write(p []byte) (n int, err error) {\n\telog := (*eventlog.Log)(w)\n\treturn 0, elog.Info(1, string(p))\n}\n\nfunc signalChan() <-chan os.Signal {\n\tch := make(chan os.Signal, 2)\n\tsignal.Notify(ch, os.Interrupt)\n\treturn ch\n}\n" }, { "path": "cmd/litestream/mcp.go", "content": "package main\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"log/slog\"\n\t\"net/http\"\n\t\"os/exec\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/MadAppGang/httplog\"\n\t\"github.com/mark3labs/mcp-go/mcp\"\n\t\"github.com/mark3labs/mcp-go/server\"\n)\n\ntype MCPServer struct {\n\tctx context.Context\n\tmux *http.ServeMux\n\thttpServer *http.Server\n\tconfigPath string\n}\n\nfunc NewMCP(ctx context.Context, configPath string) (*MCPServer, error) {\n\ts := &MCPServer{\n\t\tctx: ctx,\n\t\tconfigPath: configPath,\n\t}\n\n\tmcpServer := server.NewMCPServer(\n\t\t\"Litestream MCP Server\",\n\t\t\"1.0.0\",\n\t\tserver.WithToolCapabilities(false),\n\t\tserver.WithRecovery(),\n\t\tserver.WithLogging(),\n\t)\n\t// Add the tools to the server\n\tmcpServer.AddTool(InfoTool(configPath))\n\tmcpServer.AddTool(DatabasesTool(configPath))\n\tmcpServer.AddTool(RestoreTool(configPath))\n\tmcpServer.AddTool(LTXTool(configPath))\n\tmcpServer.AddTool(VersionTool())\n\tmcpServer.AddTool(StatusTool(configPath))\n\tmcpServer.AddTool(ResetTool(configPath))\n\n\ts.mux = http.NewServeMux()\n\ts.mux.Handle(\"/\", httplog.Logger(server.NewStreamableHTTPServer(mcpServer)))\n\treturn s, nil\n}\n\nfunc (s *MCPServer) Start(addr string) {\n\ts.httpServer = &http.Server{\n\t\tAddr: addr,\n\t\tHandler: s.mux,\n\t\tReadHeaderTimeout: 30 * time.Second,\n\t}\n\tgo func() {\n\t\tslog.Info(\"Starting MCP Streamable HTTP server\", \"addr\", addr)\n\t\tif err := s.httpServer.ListenAndServe(); err != nil {\n\t\t\tslog.Error(\"MCP server error\", \"error\", err)\n\t\t}\n\t}()\n}\n\nfunc (s *MCPServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\ts.mux.ServeHTTP(w, r)\n}\n\n// Close attempts to gracefully shutdown the server.\nfunc (s *MCPServer) Close() error {\n\tctx, cancel := context.WithTimeout(s.ctx, 10*time.Second)\n\tdefer cancel()\n\treturn s.httpServer.Shutdown(ctx)\n}\n\n// isReplicaURL returns true if the path looks like a replica URL (s3://, gs://, etc.)\n// rather than a local database path. The CLI rejects -config when using replica URLs.\nfunc isReplicaURL(path string) bool {\n\treturn strings.Contains(path, \"://\")\n}\n\nfunc DatabasesTool(configPath string) (mcp.Tool, server.ToolHandlerFunc) {\n\ttool := mcp.NewTool(\"litestream_databases\",\n\t\tmcp.WithDescription(\"List databases and their replicas as defined in the Litestream config file. The default path is /etc/litestream.yml but is not required.\"),\n\t\tmcp.WithString(\"config\", mcp.Description(\"Path to the Litestream config file. Optional.\")),\n\t)\n\n\treturn tool, func(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {\n\t\targs := []string{\"databases\"}\n\t\tconfig := configPath\n\t\tif configVal, err := req.RequireString(\"config\"); err == nil {\n\t\t\tconfig = configVal\n\t\t}\n\t\targs = append(args, \"-config\", config)\n\t\tcmd := exec.CommandContext(ctx, \"litestream\", args...)\n\t\toutput, err := cmd.CombinedOutput()\n\t\tif err != nil {\n\t\t\treturn mcp.NewToolResultError(strings.TrimSpace(string(output)) + \": \" + err.Error()), nil\n\t\t}\n\t\treturn mcp.NewToolResultText(string(output)), nil\n\t}\n}\n\nfunc InfoTool(configPath string) (mcp.Tool, server.ToolHandlerFunc) {\n\ttool := mcp.NewTool(\"litestream_info\",\n\t\tmcp.WithDescription(\"Get a comprehensive summary of Litestream's current status including databases, LTX files, and version information.\"),\n\t\tmcp.WithString(\"config\", mcp.Description(\"Path to the Litestream config file. Optional.\")),\n\t)\n\n\treturn tool, func(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {\n\t\tvar summary strings.Builder\n\t\tsummary.WriteString(\"=== Litestream Status Report ===\\n\\n\")\n\n\t\t// Get version info\n\t\tversionCmd := exec.CommandContext(ctx, \"litestream\", \"version\")\n\t\tversionOutput, err := versionCmd.CombinedOutput()\n\t\tif err != nil {\n\t\t\tslog.Error(\"Failed to get version info\", \"error\", err)\n\t\t\treturn mcp.NewToolResultError(\"Failed to get version info: \" + err.Error()), nil\n\t\t}\n\t\tsummary.WriteString(\"Version Information:\\n\")\n\t\tsummary.WriteString(string(versionOutput))\n\t\tsummary.WriteString(\"\\n\")\n\n\t\t// Get databases info\n\t\targs := []string{\"databases\"}\n\t\tconfig := configPath\n\t\tif configVal, err := req.RequireString(\"config\"); err == nil {\n\t\t\tconfig = configVal\n\t\t}\n\t\tsummary.WriteString(\"Current Config Path:\\n\")\n\t\tsummary.WriteString(config + \"\\n\\n\")\n\n\t\targs = append(args, \"-config\", config)\n\t\tdbCmd := exec.CommandContext(ctx, \"litestream\", args...)\n\t\tdbOutput, err := dbCmd.CombinedOutput()\n\t\tif err != nil {\n\t\t\tslog.Error(\"Failed to get databases info\", \"error\", err)\n\t\t\treturn mcp.NewToolResultError(\"Failed to get databases info: \" + err.Error()), nil\n\t\t}\n\n\t\tsummary.WriteString(\"Databases:\\n\")\n\t\tsummary.WriteString(string(dbOutput))\n\t\tsummary.WriteString(\"\\n\")\n\n\t\t// Parse database paths from output\n\t\tscanner := bufio.NewScanner(strings.NewReader(string(dbOutput)))\n\t\t// Skip header line\n\t\tscanner.Scan()\n\t\tvar dbPaths []string\n\t\tfor scanner.Scan() {\n\t\t\tfields := strings.Fields(scanner.Text())\n\t\t\tif len(fields) > 0 {\n\t\t\t\tdbPaths = append(dbPaths, fields[0])\n\t\t\t}\n\t\t}\n\n\t\t// Get LTX files info for each database\n\t\tsummary.WriteString(\"LTX Files:\\n\")\n\t\tfor _, dbPath := range dbPaths {\n\t\t\tltxArgs := []string{\"ltx\"}\n\t\t\tif config != \"\" {\n\t\t\t\tltxArgs = append(ltxArgs, \"-config\", config)\n\t\t\t}\n\t\t\tltxArgs = append(ltxArgs, dbPath)\n\t\t\tltxCmd := exec.CommandContext(ctx, \"litestream\", ltxArgs...)\n\t\t\tltxOutput, err := ltxCmd.CombinedOutput()\n\t\t\tif err != nil {\n\t\t\t\tsummary.WriteString(\"Failed to get LTX files for \" + dbPath + \": \" + err.Error() + \"\\n\")\n\t\t\t\tsummary.WriteString(string(ltxOutput))\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tsummary.WriteString(\"Database: \" + dbPath + \"\\n\")\n\t\t\tsummary.WriteString(string(ltxOutput))\n\t\t\tsummary.WriteString(\"\\n\")\n\t\t}\n\n\t\treturn mcp.NewToolResultText(summary.String()), nil\n\t}\n}\n\nfunc RestoreTool(configPath string) (mcp.Tool, server.ToolHandlerFunc) {\n\ttool := mcp.NewTool(\"litestream_restore\",\n\t\tmcp.WithDescription(\"Restore a database from a Litestream replica.\"),\n\t\tmcp.WithString(\"path\", mcp.Required(), mcp.Description(\"Database path or replica URL.\")),\n\t\tmcp.WithString(\"o\", mcp.Description(\"Output path for the restored database. Optional.\")),\n\t\tmcp.WithString(\"config\", mcp.Description(\"Path to the Litestream config file. Optional.\")),\n\t\tmcp.WithString(\"txid\", mcp.Description(\"Restore up to a specific transaction ID. Optional.\")),\n\t\tmcp.WithString(\"timestamp\", mcp.Description(\"Restore to a specific point-in-time (RFC3339). Optional.\")),\n\t\tmcp.WithString(\"parallelism\", mcp.Description(\"Number of WAL files to download in parallel. Optional.\")),\n\t\tmcp.WithBoolean(\"if_db_not_exists\", mcp.Description(\"Return 0 if the database already exists. Optional.\")),\n\t\tmcp.WithBoolean(\"if_replica_exists\", mcp.Description(\"Return 0 if no backups found. Optional.\")),\n\t)\n\n\treturn tool, func(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {\n\t\targs := []string{\"restore\"}\n\t\tif o, err := req.RequireString(\"o\"); err == nil {\n\t\t\targs = append(args, \"-o\", o)\n\t\t}\n\n\t\t// Get path first to determine if it's a replica URL\n\t\tpath, _ := req.RequireString(\"path\")\n\n\t\t// Only add -config for database paths, not replica URLs\n\t\t// The CLI rejects -config when restoring from a replica URL\n\t\tif !isReplicaURL(path) {\n\t\t\tconfig := configPath\n\t\t\tif configVal, err := req.RequireString(\"config\"); err == nil {\n\t\t\t\tconfig = configVal\n\t\t\t}\n\t\t\tif config != \"\" {\n\t\t\t\targs = append(args, \"-config\", config)\n\t\t\t}\n\t\t}\n\n\t\tif txid, err := req.RequireString(\"txid\"); err == nil {\n\t\t\targs = append(args, \"-txid\", txid)\n\t\t}\n\t\tif timestamp, err := req.RequireString(\"timestamp\"); err == nil {\n\t\t\targs = append(args, \"-timestamp\", timestamp)\n\t\t}\n\t\tif parallelism, err := req.RequireString(\"parallelism\"); err == nil {\n\t\t\targs = append(args, \"-parallelism\", parallelism)\n\t\t}\n\t\tif ifDBNotExists, err := req.RequireBool(\"if_db_not_exists\"); err == nil {\n\t\t\targs = append(args, \"-if-db-not-exists\", strconv.FormatBool(ifDBNotExists))\n\t\t}\n\t\tif ifReplicaExists, err := req.RequireBool(\"if_replica_exists\"); err == nil {\n\t\t\targs = append(args, \"-if-replica-exists\", strconv.FormatBool(ifReplicaExists))\n\t\t}\n\t\tif path != \"\" {\n\t\t\targs = append(args, path)\n\t\t}\n\t\tcmd := exec.CommandContext(ctx, \"litestream\", args...)\n\t\toutput, err := cmd.CombinedOutput()\n\t\tif err != nil {\n\t\t\treturn mcp.NewToolResultError(strings.TrimSpace(string(output)) + \": \" + err.Error()), nil\n\t\t}\n\t\treturn mcp.NewToolResultText(string(output)), nil\n\t}\n}\n\nfunc VersionTool() (mcp.Tool, server.ToolHandlerFunc) {\n\ttool := mcp.NewTool(\"litestream_version\",\n\t\tmcp.WithDescription(\"Print the Litestream binary version.\"),\n\t)\n\n\treturn tool, func(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {\n\t\tcmd := exec.CommandContext(ctx, \"litestream\", \"version\")\n\t\toutput, err := cmd.CombinedOutput()\n\t\tif err != nil {\n\t\t\treturn mcp.NewToolResultError(strings.TrimSpace(string(output)) + \": \" + err.Error()), nil\n\t\t}\n\t\treturn mcp.NewToolResultText(string(output)), nil\n\t}\n}\n\nfunc LTXTool(configPath string) (mcp.Tool, server.ToolHandlerFunc) {\n\ttool := mcp.NewTool(\"litestream_ltx\",\n\t\tmcp.WithDescription(\"List all LTX files for a database or replica URL.\"),\n\t\tmcp.WithString(\"path\", mcp.Required(), mcp.Description(\"Database path or replica URL.\")),\n\t\tmcp.WithString(\"config\", mcp.Description(\"Path to the Litestream config file. Optional, ignored for replica URLs.\")),\n\t)\n\n\treturn tool, func(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {\n\t\targs := []string{\"ltx\"}\n\n\t\t// Get path first to determine if it's a replica URL\n\t\tpath, _ := req.RequireString(\"path\")\n\n\t\t// Only add -config for database paths, not replica URLs\n\t\t// The CLI rejects -config when using a replica URL\n\t\tif !isReplicaURL(path) {\n\t\t\tconfig := configPath\n\t\t\tif configVal, err := req.RequireString(\"config\"); err == nil {\n\t\t\t\tconfig = configVal\n\t\t\t}\n\t\t\tif config != \"\" {\n\t\t\t\targs = append(args, \"-config\", config)\n\t\t\t}\n\t\t}\n\n\t\tif path != \"\" {\n\t\t\targs = append(args, path)\n\t\t}\n\t\tcmd := exec.CommandContext(ctx, \"litestream\", args...)\n\t\toutput, err := cmd.CombinedOutput()\n\t\tif err != nil {\n\t\t\treturn mcp.NewToolResultError(strings.TrimSpace(string(output)) + \": \" + err.Error()), nil\n\t\t}\n\t\treturn mcp.NewToolResultText(string(output)), nil\n\t}\n}\n\nfunc StatusTool(configPath string) (mcp.Tool, server.ToolHandlerFunc) {\n\ttool := mcp.NewTool(\"litestream_status\",\n\t\tmcp.WithDescription(\"Display replication status including database path, status, local transaction ID, and WAL size.\"),\n\t\tmcp.WithString(\"config\", mcp.Description(\"Path to the Litestream config file. Optional.\")),\n\t\tmcp.WithString(\"path\", mcp.Description(\"Filter to a specific database path. Optional.\")),\n\t)\n\n\treturn tool, func(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {\n\t\targs := []string{\"status\"}\n\t\tconfig := configPath\n\t\tif configVal, err := req.RequireString(\"config\"); err == nil {\n\t\t\tconfig = configVal\n\t\t}\n\t\targs = append(args, \"-config\", config)\n\t\tif path, err := req.RequireString(\"path\"); err == nil {\n\t\t\targs = append(args, path)\n\t\t}\n\t\tcmd := exec.CommandContext(ctx, \"litestream\", args...)\n\t\toutput, err := cmd.CombinedOutput()\n\t\tif err != nil {\n\t\t\treturn mcp.NewToolResultError(strings.TrimSpace(string(output)) + \": \" + err.Error()), nil\n\t\t}\n\t\treturn mcp.NewToolResultText(string(output)), nil\n\t}\n}\n\nfunc ResetTool(configPath string) (mcp.Tool, server.ToolHandlerFunc) {\n\ttool := mcp.NewTool(\"litestream_reset\",\n\t\tmcp.WithDescription(\"Clear local Litestream state for a database. Removes local LTX files, forcing fresh snapshot on next sync. Database file is not modified.\"),\n\t\tmcp.WithString(\"path\", mcp.Required(), mcp.Description(\"Database path to reset.\")),\n\t\tmcp.WithString(\"config\", mcp.Description(\"Path to the Litestream config file. Optional.\")),\n\t)\n\n\treturn tool, func(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {\n\t\targs := []string{\"reset\"}\n\t\tconfig := configPath\n\t\tif configVal, err := req.RequireString(\"config\"); err == nil {\n\t\t\tconfig = configVal\n\t\t}\n\t\targs = append(args, \"-config\", config)\n\t\tif path, err := req.RequireString(\"path\"); err == nil {\n\t\t\targs = append(args, path)\n\t\t}\n\t\tcmd := exec.CommandContext(ctx, \"litestream\", args...)\n\t\toutput, err := cmd.CombinedOutput()\n\t\tif err != nil {\n\t\t\treturn mcp.NewToolResultError(strings.TrimSpace(string(output)) + \": \" + err.Error()), nil\n\t\t}\n\t\treturn mcp.NewToolResultText(string(output)), nil\n\t}\n}\n" }, { "path": "cmd/litestream/register.go", "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\ntype RegisterCommand struct{}\n\nfunc (c *RegisterCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-register\", flag.ContinueOnError)\n\ttimeout := fs.Int(\"timeout\", 30, \"timeout in seconds\")\n\tsocketPath := fs.String(\"socket\", \"/var/run/litestream.sock\", \"control socket path\")\n\treplicaFlag := fs.String(\"replica\", \"\", \"replica URL (e.g., s3://bucket/prefix, file:///backup/path)\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif fs.NArg() == 0 {\n\t\treturn fmt.Errorf(\"database path required\")\n\t}\n\tif fs.NArg() > 1 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\tif *replicaFlag == \"\" {\n\t\treturn fmt.Errorf(\"replica URL required (use -replica flag)\")\n\t}\n\tif *timeout <= 0 {\n\t\treturn fmt.Errorf(\"timeout must be greater than 0\")\n\t}\n\n\tdbPath := fs.Arg(0)\n\treplicaURL := *replicaFlag\n\n\t// Create HTTP client that connects via Unix socket with timeout.\n\tclientTimeout := time.Duration(*timeout) * time.Second\n\tclient := &http.Client{\n\t\tTimeout: clientTimeout,\n\t\tTransport: &http.Transport{\n\t\t\tDialContext: func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\t\treturn net.DialTimeout(\"unix\", *socketPath, clientTimeout)\n\t\t\t},\n\t\t},\n\t}\n\n\treq := litestream.RegisterDatabaseRequest{\n\t\tPath: dbPath,\n\t\tReplicaURL: replicaURL,\n\t}\n\treqBody, err := json.Marshal(req)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to marshal request: %w\", err)\n\t}\n\n\tresp, err := client.Post(\"http://localhost/register\", \"application/json\", bytes.NewReader(reqBody))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to connect to control socket: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read response: %w\", err)\n\t}\n\n\tif resp.StatusCode != http.StatusOK {\n\t\tvar errResp litestream.ErrorResponse\n\t\tif err := json.Unmarshal(body, &errResp); err == nil && errResp.Error != \"\" {\n\t\t\treturn fmt.Errorf(\"register failed: %s\", errResp.Error)\n\t\t}\n\t\treturn fmt.Errorf(\"register failed: %s\", string(body))\n\t}\n\n\tvar result litestream.RegisterDatabaseResponse\n\tif err := json.Unmarshal(body, &result); err != nil {\n\t\treturn fmt.Errorf(\"failed to parse response: %w\", err)\n\t}\n\n\toutput, err := json.MarshalIndent(result, \"\", \" \")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to format response: %w\", err)\n\t}\n\tfmt.Println(string(output))\n\n\treturn nil\n}\n\nfunc (c *RegisterCommand) Usage() {\n\tfmt.Println(`\nusage: litestream register [OPTIONS] DB_PATH\n\nRegister a database for replication.\n\nArguments:\n DB_PATH Path to the SQLite database file.\n\nOptions:\n -replica URL\n Replica destination URL (e.g., s3://bucket/prefix, file:///backup/path).\n Required.\n\n -timeout SECONDS\n Maximum time to wait in seconds (default: 30).\n\n -socket PATH\n Path to control socket (default: /var/run/litestream.sock).\n`[1:])\n}\n" }, { "path": "cmd/litestream/register_test.go", "content": "package main_test\n\nimport (\n\t\"context\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/benbjohnson/litestream\"\n\tmain \"github.com/benbjohnson/litestream/cmd/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestRegisterCommand_Run(t *testing.T) {\n\tt.Run(\"MissingDBPath\", func(t *testing.T) {\n\t\tcmd := &main.RegisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/tmp/test.sock\", \"-replica\", \"file:///tmp/backup\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for missing database path\")\n\t\t}\n\t\tif err.Error() != \"database path required\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"MissingReplicaFlag\", func(t *testing.T) {\n\t\tcmd := &main.RegisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/tmp/test.sock\", \"/tmp/test.db\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for missing replica flag\")\n\t\t}\n\t\tif err.Error() != \"replica URL required (use -replica flag)\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"TooManyArguments\", func(t *testing.T) {\n\t\tcmd := &main.RegisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/tmp/test.sock\", \"-replica\", \"file:///tmp/backup\", \"/tmp/test.db\", \"extra\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for too many arguments\")\n\t\t}\n\t\tif err.Error() != \"too many arguments\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"InvalidTimeoutZero\", func(t *testing.T) {\n\t\tcmd := &main.RegisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-timeout\", \"0\", \"-replica\", \"file:///tmp/backup\", \"/tmp/test.db\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for zero timeout\")\n\t\t}\n\t\tif err.Error() != \"timeout must be greater than 0\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"SocketConnectionError\", func(t *testing.T) {\n\t\tcmd := &main.RegisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/nonexistent/socket.sock\", \"-replica\", \"file:///tmp/backup\", \"/tmp/test.db\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for socket connection failure\")\n\t\t}\n\t})\n\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(context.Background())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\t// Create a temporary database file.\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\ttestingutil.MustCloseDBs(t, db, sqldb)\n\t\tdbPath := db.Path()\n\n\t\t// Create a temp directory for backup.\n\t\tbackupDir := filepath.Join(t.TempDir(), \"backup\")\n\n\t\tcmd := &main.RegisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", server.SocketPath, \"-replica\", \"file://\" + backupDir, dbPath})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// Verify database was registered with store.\n\t\tif len(store.DBs()) != 1 {\n\t\t\tt.Errorf(\"expected 1 database in store, got %d\", len(store.DBs()))\n\t\t}\n\t})\n\n\tt.Run(\"AlreadyExists\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(context.Background())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\t// Create a temp directory for backup.\n\t\tbackupDir := filepath.Join(t.TempDir(), \"backup\")\n\n\t\tcmd := &main.RegisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", server.SocketPath, \"-replica\", \"file://\" + backupDir, db.Path()})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// Still only 1 database - didn't register a duplicate.\n\t\tif len(store.DBs()) != 1 {\n\t\t\tt.Errorf(\"expected 1 database in store, got %d\", len(store.DBs()))\n\t\t}\n\t})\n}\n" }, { "path": "cmd/litestream/replicate.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"flag\"\n\t\"fmt\"\n\t\"log/slog\"\n\t\"net\"\n\t\"net/http\"\n\t_ \"net/http/pprof\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\n\t\"github.com/mattn/go-shellwords\"\n\t\"github.com/prometheus/client_golang/prometheus/promhttp\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/abs\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/gs\"\n\t\"github.com/benbjohnson/litestream/nats\"\n\t\"github.com/benbjohnson/litestream/oss\"\n\t\"github.com/benbjohnson/litestream/s3\"\n\t\"github.com/benbjohnson/litestream/sftp\"\n)\n\n// ReplicateCommand represents a command that continuously replicates SQLite databases.\ntype ReplicateCommand struct {\n\tcmd *exec.Cmd // subcommand\n\texecCh chan error // subcommand error channel\n\n\t// One-shot replication flags\n\tonce bool // replicate once and exit\n\tforceSnapshot bool // force snapshot to all replicas\n\tenforceRetention bool // enforce retention of old snapshots\n\n\tConfig Config\n\n\t// MCP server\n\tMCP *MCPServer\n\n\t// Server for IPC control commands.\n\tServer *litestream.Server\n\n\t// Manages the set of databases & compaction levels.\n\tStore *litestream.Store\n\n\t// Directory monitors for dynamic database discovery.\n\tdirectoryMonitors []*DirectoryMonitor\n\n\t// Done channel for interrupt handling during shutdown. When closed,\n\t// the shutdown sync retry loop exits and any in-flight sync is cancelled.\n\tdone <-chan struct{}\n}\n\nfunc NewReplicateCommand() *ReplicateCommand {\n\treturn &ReplicateCommand{\n\t\texecCh: make(chan error),\n\t}\n}\n\n// ParseFlags parses the CLI flags and loads the configuration file.\nfunc (c *ReplicateCommand) ParseFlags(_ context.Context, args []string) (err error) {\n\tfs := flag.NewFlagSet(\"litestream-replicate\", flag.ContinueOnError)\n\texecFlag := fs.String(\"exec\", \"\", \"execute subcommand\")\n\tlogLevelFlag := fs.String(\"log-level\", \"\", \"log level (trace, debug, info, warn, error)\")\n\trestoreIfDBNotExists := fs.Bool(\"restore-if-db-not-exists\", false, \"restore from replica if database doesn't exist\")\n\tonceFlag := fs.Bool(\"once\", false, \"replicate once and exit\")\n\tforceSnapshotFlag := fs.Bool(\"force-snapshot\", false, \"force snapshot when replicating once\")\n\tenforceRetentionFlag := fs.Bool(\"enforce-retention\", false, \"enforce retention of old snapshots when replicating once\")\n\tconfigPath, noExpandEnv := registerConfigFlag(fs)\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\t// Load configuration or use CLI args to build db/replica.\n\tswitch fs.NArg() {\n\tcase 0:\n\t\t// No arguments provided, use config file\n\t\tif *configPath == \"\" {\n\t\t\t*configPath = DefaultConfigPath()\n\t\t}\n\t\tif c.Config, err = ReadConfigFile(*configPath, !*noExpandEnv); err != nil {\n\t\t\treturn err\n\t\t}\n\t\t// Override log level if CLI flag provided (takes precedence over env var)\n\t\tif *logLevelFlag != \"\" {\n\t\t\tc.Config.Logging.Level = *logLevelFlag\n\t\t\t// Set env var so initLog sees CLI flag as highest priority\n\t\t\tos.Setenv(\"LOG_LEVEL\", *logLevelFlag)\n\t\t\tlogOutput := os.Stdout\n\t\t\tif c.Config.Logging.Stderr {\n\t\t\t\tlogOutput = os.Stderr\n\t\t\t}\n\t\t\tinitLog(logOutput, c.Config.Logging.Level, c.Config.Logging.Type)\n\t\t}\n\n\tcase 1:\n\t\t// Only database path provided, missing replica URL\n\t\treturn fmt.Errorf(\"must specify at least one replica URL for %s\", fs.Arg(0))\n\n\tdefault:\n\t\t// Database path and replica URLs provided via CLI\n\t\tif *configPath != \"\" {\n\t\t\treturn fmt.Errorf(\"cannot specify a replica URL and the -config flag\")\n\t\t}\n\n\t\t// Initialize config with defaults when using command-line arguments\n\t\tc.Config = DefaultConfig()\n\t\tlogLevel := \"INFO\"\n\t\tif *logLevelFlag != \"\" {\n\t\t\tlogLevel = *logLevelFlag\n\t\t\t// Set env var so initLog sees CLI flag as highest priority\n\t\t\tos.Setenv(\"LOG_LEVEL\", *logLevelFlag)\n\t\t}\n\t\tc.Config.Logging.Level = logLevel\n\t\tinitLog(os.Stdout, logLevel, \"text\")\n\n\t\tdbConfig := &DBConfig{\n\t\t\tPath: fs.Arg(0),\n\t\t\tRestoreIfDBNotExists: *restoreIfDBNotExists,\n\t\t}\n\t\tfor _, u := range fs.Args()[1:] {\n\t\t\t// Check if this looks like a flag that was placed after positional arguments\n\t\t\tif strings.HasPrefix(u, \"-\") {\n\t\t\t\treturn fmt.Errorf(\"flag %q must be positioned before DB_PATH and REPLICA_URL arguments\", u)\n\t\t\t}\n\t\t\tsyncInterval := litestream.DefaultSyncInterval\n\t\t\tdbConfig.Replicas = append(dbConfig.Replicas, &ReplicaConfig{\n\t\t\t\tURL: u,\n\t\t\t\tReplicaSettings: ReplicaSettings{\n\t\t\t\t\tSyncInterval: &syncInterval,\n\t\t\t\t},\n\t\t\t})\n\t\t}\n\t\tc.Config.DBs = []*DBConfig{dbConfig}\n\t}\n\n\tc.Config.ConfigPath = *configPath\n\n\t// Override config exec command, if specified.\n\tif *execFlag != \"\" {\n\t\tc.Config.Exec = *execFlag\n\t}\n\n\t// Apply restore-if-db-not-exists flag to all databases if specified.\n\t// This allows the CLI flag to work with config files.\n\tif *restoreIfDBNotExists {\n\t\tfor _, dbConfig := range c.Config.DBs {\n\t\t\tdbConfig.RestoreIfDBNotExists = true\n\t\t}\n\t}\n\n\t// Set one-shot replication flags and validate their usage.\n\tc.once = *onceFlag\n\tc.forceSnapshot = *forceSnapshotFlag\n\tc.enforceRetention = *enforceRetentionFlag\n\n\t// Validate flag combinations.\n\tif c.once && c.Config.Exec != \"\" {\n\t\treturn fmt.Errorf(\"cannot specify -once flag with -exec\")\n\t}\n\tif c.forceSnapshot && !c.once {\n\t\treturn fmt.Errorf(\"cannot specify -force-snapshot flag without -once\")\n\t}\n\tif c.enforceRetention && !c.once {\n\t\treturn fmt.Errorf(\"cannot specify -enforce-retention flag without -once\")\n\t}\n\n\treturn nil\n}\n\n// Run loads all databases specified in the configuration.\nfunc (c *ReplicateCommand) Run(ctx context.Context) (err error) {\n\t// Display version information.\n\tslog.Info(\"litestream\", \"version\", Version, \"level\", c.Config.Logging.Level)\n\n\t// Start MCP server if enabled\n\tif c.Config.MCPAddr != \"\" {\n\t\tc.MCP, err = NewMCP(ctx, c.Config.ConfigPath)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tgo c.MCP.Start(c.Config.MCPAddr)\n\t}\n\n\t// Setup databases.\n\tif len(c.Config.DBs) == 0 {\n\t\tslog.Error(\"no databases specified in configuration\")\n\t}\n\n\t// Attempt restore for databases that need it (before creating DB objects)\n\tfor _, dbConfig := range c.Config.DBs {\n\t\tif dbConfig.RestoreIfDBNotExists && dbConfig.Path != \"\" {\n\t\t\tif err := c.restoreIfNeeded(ctx, dbConfig); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\tvar dbs []*litestream.DB\n\tvar watchables []struct {\n\t\tconfig *DBConfig\n\t\tdbs []*litestream.DB\n\t}\n\tfor _, dbConfig := range c.Config.DBs {\n\t\t// Handle directory configuration\n\t\tif dbConfig.Dir != \"\" {\n\t\t\tdirDbs, err := NewDBsFromDirectoryConfig(dbConfig)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdbs = append(dbs, dirDbs...)\n\t\t\tslog.Info(\"found databases in directory\", \"dir\", dbConfig.Dir, \"count\", len(dirDbs), \"watch\", dbConfig.Watch)\n\t\t\tif dbConfig.Watch {\n\t\t\t\twatchables = append(watchables, struct {\n\t\t\t\t\tconfig *DBConfig\n\t\t\t\t\tdbs []*litestream.DB\n\t\t\t\t}{config: dbConfig, dbs: dirDbs})\n\t\t\t}\n\t\t} else {\n\t\t\t// Handle single database configuration\n\t\t\tdb, err := NewDBFromConfig(dbConfig)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdbs = append(dbs, db)\n\t\t}\n\t}\n\n\tlevels := c.Config.CompactionLevels()\n\tc.Store = litestream.NewStore(dbs, levels)\n\n\t// Only override default snapshot interval if explicitly set in config\n\tif c.Config.Snapshot.Interval != nil {\n\t\tc.Store.SnapshotInterval = *c.Config.Snapshot.Interval\n\t}\n\t// Only override default snapshot retention if explicitly set in config\n\tif c.Config.Snapshot.Retention != nil {\n\t\tc.Store.SnapshotRetention = *c.Config.Snapshot.Retention\n\t}\n\tif c.Config.L0Retention != nil {\n\t\tc.Store.SetL0Retention(*c.Config.L0Retention)\n\t}\n\tif c.Config.L0RetentionCheckInterval != nil {\n\t\tc.Store.L0RetentionCheckInterval = *c.Config.L0RetentionCheckInterval\n\t}\n\tif c.Config.ShutdownSyncTimeout != nil {\n\t\tc.Store.SetShutdownSyncTimeout(*c.Config.ShutdownSyncTimeout)\n\t}\n\tif c.Config.ShutdownSyncInterval != nil {\n\t\tc.Store.SetShutdownSyncInterval(*c.Config.ShutdownSyncInterval)\n\t}\n\tif c.Config.VerifyCompaction {\n\t\tc.Store.SetVerifyCompaction(true)\n\t}\n\tif c.Config.Retention.Enabled != nil && !*c.Config.Retention.Enabled {\n\t\tc.Store.SetRetentionEnabled(false)\n\t}\n\tif c.Config.Validation.Interval != nil {\n\t\tc.Store.ValidationInterval = *c.Config.Validation.Interval\n\t}\n\tif c.done != nil {\n\t\tc.Store.SetDone(c.done)\n\t}\n\tif c.Config.HeartbeatURL != \"\" {\n\t\tinterval := litestream.DefaultHeartbeatInterval\n\t\tif c.Config.HeartbeatInterval != nil {\n\t\t\tinterval = *c.Config.HeartbeatInterval\n\t\t}\n\t\tc.Store.Heartbeat = litestream.NewHeartbeatClient(c.Config.HeartbeatURL, interval)\n\t}\n\n\t// Disable all background monitors when running once.\n\t// This must be done after config settings are applied.\n\tif c.once {\n\t\tc.Store.CompactionMonitorEnabled = false\n\t\tc.Store.L0RetentionCheckInterval = 0\n\t\tfor _, db := range dbs {\n\t\t\tdb.MonitorInterval = 0\n\t\t\tif db.Replica != nil {\n\t\t\t\tdb.Replica.MonitorEnabled = false\n\t\t\t}\n\t\t}\n\t}\n\n\tif err := c.Store.Open(ctx); err != nil {\n\t\treturn fmt.Errorf(\"cannot open store: %w\", err)\n\t}\n\n\tif !c.Store.RetentionEnabled {\n\t\tslog.Warn(\"retention disabled; cloud provider lifecycle policies must handle retention\",\n\t\t\t\"hint\", \"idle databases that stop receiving writes will not generate new snapshots and may lose backup coverage if cloud retention expires\")\n\t}\n\n\t// Start control server if socket is enabled\n\tif c.Config.Socket.Enabled {\n\t\tc.Server = litestream.NewServer(c.Store)\n\t\tc.Server.SocketPath = c.Config.Socket.Path\n\t\tc.Server.SocketPerms = c.Config.Socket.Permissions\n\t\tc.Server.PathExpander = expand\n\t\tc.Server.Version = Version\n\t\tif err := c.Server.Start(); err != nil {\n\t\t\tslog.Warn(\"failed to start control server\", \"error\", err)\n\t\t}\n\t}\n\n\tfor _, entry := range watchables {\n\t\tmonitor, err := NewDirectoryMonitor(ctx, c.Store, entry.config, entry.dbs)\n\t\tif err != nil {\n\t\t\tfor _, m := range c.directoryMonitors {\n\t\t\t\tm.Close()\n\t\t\t}\n\t\t\tif closeErr := c.Store.Close(ctx); closeErr != nil {\n\t\t\t\tslog.Error(\"failed to close store after monitor failure\", \"error\", closeErr)\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"start directory monitor for %s: %w\", entry.config.Dir, err)\n\t\t}\n\t\tc.directoryMonitors = append(c.directoryMonitors, monitor)\n\t}\n\n\t// Notify user that initialization is done.\n\tfor _, db := range c.Store.DBs() {\n\t\tr := db.Replica\n\t\tslog.Info(\"initialized db\", \"path\", db.Path())\n\t\tslogWith := slog.With(\"type\", r.Client.Type(), \"sync-interval\", r.SyncInterval)\n\t\tswitch client := r.Client.(type) {\n\t\tcase *file.ReplicaClient:\n\t\t\tslogWith.Info(\"replicating to\", \"path\", client.Path())\n\t\tcase *s3.ReplicaClient:\n\t\t\tslogWith.Info(\"replicating to\", \"bucket\", client.Bucket, \"path\", client.Path, \"region\", client.Region, \"endpoint\", client.Endpoint)\n\t\tcase *gs.ReplicaClient:\n\t\t\tslogWith.Info(\"replicating to\", \"bucket\", client.Bucket, \"path\", client.Path)\n\t\tcase *abs.ReplicaClient:\n\t\t\tslogWith.Info(\"replicating to\", \"bucket\", client.Bucket, \"path\", client.Path, \"endpoint\", client.Endpoint)\n\t\tcase *sftp.ReplicaClient:\n\t\t\tslogWith.Info(\"replicating to\", \"host\", client.Host, \"user\", client.User, \"path\", client.Path)\n\t\tcase *nats.ReplicaClient:\n\t\t\tslogWith.Info(\"replicating to\", \"bucket\", client.BucketName, \"url\", client.URL)\n\t\tcase *oss.ReplicaClient:\n\t\t\tslogWith.Info(\"replicating to\", \"bucket\", client.Bucket, \"path\", client.Path, \"region\", client.Region)\n\t\tdefault:\n\t\t\tslogWith.Info(\"replicating to\")\n\t\t}\n\t}\n\n\t// Serve metrics over HTTP if enabled.\n\tif c.Config.Addr != \"\" {\n\t\thostport := c.Config.Addr\n\t\tif host, port, _ := net.SplitHostPort(c.Config.Addr); port == \"\" {\n\t\t\treturn fmt.Errorf(\"must specify port for bind address: %q\", c.Config.Addr)\n\t\t} else if host == \"\" {\n\t\t\thostport = net.JoinHostPort(\"localhost\", port)\n\t\t}\n\n\t\tslog.Info(\"serving metrics on\", \"url\", fmt.Sprintf(\"http://%s/metrics\", hostport))\n\t\tgo func() {\n\t\t\thttp.Handle(\"/metrics\", promhttp.Handler())\n\t\t\tif err := http.ListenAndServe(c.Config.Addr, nil); err != nil {\n\t\t\t\tslog.Error(\"cannot start metrics server\", \"error\", err)\n\t\t\t}\n\t\t}()\n\t}\n\n\t// Parse exec commands args & start subprocess.\n\tif c.Config.Exec != \"\" {\n\t\texecArgs, err := shellwords.Parse(c.Config.Exec)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"cannot parse exec command: %w\", err)\n\t\t}\n\n\t\tc.cmd = exec.CommandContext(ctx, execArgs[0], execArgs[1:]...)\n\t\tc.cmd.Env = os.Environ()\n\t\tc.cmd.Stdout = os.Stdout\n\t\tc.cmd.Stderr = os.Stderr\n\t\tif err := c.cmd.Start(); err != nil {\n\t\t\treturn fmt.Errorf(\"cannot start exec command: %w\", err)\n\t\t}\n\t\tgo func() { c.execCh <- c.cmd.Wait() }()\n\t} else if c.once {\n\t\t// Run one-shot replication in a goroutine so the caller can wait on execCh.\n\t\tgo c.runOnce(ctx)\n\t}\n\n\treturn nil\n}\n\n// runOnce performs one-shot replication for all databases.\n// It syncs all databases, optionally takes snapshots, and enforces retention.\nfunc (c *ReplicateCommand) runOnce(ctx context.Context) {\n\tvar err error\n\tdefer func() { c.execCh <- err }()\n\n\tfor _, db := range c.Store.DBs() {\n\t\tslog.Info(\"syncing database\", \"path\", db.Path())\n\n\t\t// Sync the database to process any pending WAL changes.\n\t\tif err = db.Sync(ctx); err != nil {\n\t\t\terr = fmt.Errorf(\"sync database %s: %w\", db.Path(), err)\n\t\t\treturn\n\t\t}\n\n\t\t// Sync the replica to upload any pending LTX files.\n\t\tif err = db.Replica.Sync(ctx); err != nil {\n\t\t\terr = fmt.Errorf(\"sync replica for %s: %w\", db.Path(), err)\n\t\t\treturn\n\t\t}\n\n\t\t// Force a snapshot if requested.\n\t\tif c.forceSnapshot {\n\t\t\tslog.Info(\"taking snapshot\", \"path\", db.Path())\n\t\t\tif _, err = db.Snapshot(ctx); err != nil {\n\t\t\t\terr = fmt.Errorf(\"snapshot %s: %w\", db.Path(), err)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\n\t\t// Enforce retention if requested.\n\t\tif c.enforceRetention {\n\t\t\tslog.Info(\"enforcing retention\", \"path\", db.Path())\n\t\t\tif err = c.Store.EnforceSnapshotRetention(ctx, db); err != nil {\n\t\t\t\terr = fmt.Errorf(\"enforce retention for %s: %w\", db.Path(), err)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}\n\n\tslog.Info(\"one-shot replication complete\")\n}\n\n// Close closes all open databases.\nfunc (c *ReplicateCommand) Close(ctx context.Context) error {\n\tfor _, monitor := range c.directoryMonitors {\n\t\tmonitor.Close()\n\t}\n\tc.directoryMonitors = nil\n\n\tif c.Server != nil {\n\t\tif err := c.Server.Close(); err != nil {\n\t\t\tslog.Error(\"error closing control server\", \"error\", err)\n\t\t}\n\t}\n\tif c.Store != nil {\n\t\tif err := c.Store.Close(ctx); err != nil {\n\t\t\tif errors.Is(err, litestream.ErrShutdownInterrupted) {\n\t\t\t\tslog.Warn(\"shutdown sync skipped by user interrupt\", \"error\", err)\n\t\t\t} else {\n\t\t\t\tslog.Error(\"failed to close database\", \"error\", err)\n\t\t\t}\n\t\t}\n\t}\n\tif c.Config.MCPAddr != \"\" && c.MCP != nil {\n\t\tif err := c.MCP.Close(); err != nil {\n\t\t\tslog.Error(\"error closing MCP server\", \"error\", err)\n\t\t}\n\t}\n\treturn nil\n}\n\n// SetDone sets the done channel used for interrupt handling during shutdown.\n// When the channel is closed, the shutdown sync retry loop exits.\nfunc (c *ReplicateCommand) SetDone(done <-chan struct{}) {\n\tc.done = done\n}\n\n// restoreIfNeeded restores a database from its replica if the database doesn't\n// exist and the RestoreIfDBNotExists option is enabled. If no backup exists\n// (first start scenario), it returns nil to allow fresh replication to begin.\nfunc (c *ReplicateCommand) restoreIfNeeded(ctx context.Context, dbConfig *DBConfig) error {\n\tdbPath, err := expand(dbConfig.Path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Skip if database already exists\n\tif _, err := os.Stat(dbPath); !os.IsNotExist(err) {\n\t\tslog.Info(\"database exists, skipping restore\", \"path\", dbPath)\n\t\treturn nil\n\t}\n\n\t// Get replica config (handles both Replica and Replicas fields)\n\tvar rc *ReplicaConfig\n\tif dbConfig.Replica != nil {\n\t\trc = dbConfig.Replica\n\t} else if len(dbConfig.Replicas) > 0 {\n\t\trc = dbConfig.Replicas[0]\n\t} else {\n\t\treturn fmt.Errorf(\"no replica configured for database: %s\", dbPath)\n\t}\n\n\t// Create replica from config (nil db since we're just restoring)\n\tr, err := NewReplicaFromConfig(rc, nil)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot create replica for restore: %w\", err)\n\t}\n\n\t// Attempt restore\n\topt := litestream.NewRestoreOptions()\n\topt.OutputPath = dbPath\n\n\tslog.Info(\"attempting restore before replication\", \"path\", dbPath)\n\tif err := r.Restore(ctx, opt); errors.Is(err, litestream.ErrTxNotAvailable) {\n\t\t// No backup exists yet (first start) - this is OK\n\t\tslog.Info(\"no backup found, starting fresh\", \"path\", dbPath)\n\t\treturn nil\n\t} else if err != nil {\n\t\treturn fmt.Errorf(\"restore failed: %w\", err)\n\t}\n\n\tslog.Info(\"restore completed\", \"path\", dbPath)\n\treturn nil\n}\n\n// Usage prints the help screen to STDOUT.\nfunc (c *ReplicateCommand) Usage() {\n\tfmt.Printf(`\nThe replicate command starts a server to monitor & replicate databases.\nYou can specify your database & replicas in a configuration file or you can\nreplicate a single database file by specifying its path and its replicas in the\ncommand line arguments.\n\nUsage:\n\n\tlitestream replicate [arguments]\n\n\tlitestream replicate [arguments] DB_PATH REPLICA_URL [REPLICA_URL...]\n\nArguments:\n\n\t-config PATH\n\t Specifies the configuration file.\n\t Defaults to %s\n\n\t-exec CMD\n\t Executes a subcommand. Litestream will exit when the child\n\t process exits. Useful for simple process management.\n\n\t-once\n\t Replicate once and exit. This performs a single sync of all\n\t databases and their replicas, then exits. Cannot be used with -exec.\n\n\t-force-snapshot\n\t Force a snapshot to be taken for all databases. Requires -once.\n\t This is useful for creating a complete backup before maintenance\n\t or when migrating databases between hosts.\n\n\t-enforce-retention\n\t Enforce retention policies for old snapshots. Requires -once.\n\t This removes snapshots that are older than the configured\n\t snapshot retention period.\n\n\t-log-level LEVEL\n\t Sets the log level. Overrides the config file setting.\n\t Valid values: trace, debug, info, warn, error\n\n\t-no-expand-env\n\t Disables environment variable expansion in configuration file.\n\n\t-restore-if-db-not-exists\n\t Restores the database from the replica if it doesn't exist.\n\t On first start with no backup, proceeds normally.\n\n`[1:], DefaultConfigPath())\n}\n" }, { "path": "cmd/litestream/replicate_test.go", "content": "package main_test\n\nimport (\n\t\"context\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\n\tmain \"github.com/benbjohnson/litestream/cmd/litestream\"\n)\n\nfunc TestReplicateCommand_ParseFlags_OnceFlags(t *testing.T) {\n\tt.Run(\"OnceFlag\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-once\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"OnceWithForceSnapshot\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-once\", \"-force-snapshot\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"OnceWithEnforceRetention\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-once\", \"-enforce-retention\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"OnceWithAllFlags\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-once\", \"-force-snapshot\", \"-enforce-retention\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ForceSnapshotRequiresOnce\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-force-snapshot\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when -force-snapshot is used without -once\")\n\t\t}\n\t\texpectedError := \"cannot specify -force-snapshot flag without -once\"\n\t\tif !strings.Contains(err.Error(), expectedError) {\n\t\t\tt.Errorf(\"expected error message to contain %q, got %q\", expectedError, err.Error())\n\t\t}\n\t})\n\n\tt.Run(\"EnforceRetentionRequiresOnce\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-enforce-retention\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when -enforce-retention is used without -once\")\n\t\t}\n\t\texpectedError := \"cannot specify -enforce-retention flag without -once\"\n\t\tif !strings.Contains(err.Error(), expectedError) {\n\t\t\tt.Errorf(\"expected error message to contain %q, got %q\", expectedError, err.Error())\n\t\t}\n\t})\n\n\tt.Run(\"OnceAndExecMutuallyExclusive\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-once\", \"-exec\", \"echo test\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when -once and -exec are both specified\")\n\t\t}\n\t\texpectedError := \"cannot specify -once flag with -exec\"\n\t\tif !strings.Contains(err.Error(), expectedError) {\n\t\t\tt.Errorf(\"expected error message to contain %q, got %q\", expectedError, err.Error())\n\t\t}\n\t})\n}\n\nfunc TestReplicateCommand_ParseFlags_FlagPositioning(t *testing.T) {\n\tt.Run(\"ExecFlagAfterPositionalArgs\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\n\t\t// Test the scenario from issue #245: -exec flag after positional arguments\n\t\targs := []string{\"test.db\", \"s3://bucket/test.db\", \"-exec\", \"echo test\"}\n\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when -exec flag is positioned after positional arguments\")\n\t\t}\n\n\t\texpectedError := `flag \"-exec\" must be positioned before DB_PATH and REPLICA_URL arguments`\n\t\tif !strings.Contains(err.Error(), expectedError) {\n\t\t\tt.Errorf(\"expected error message to contain %q, got %q\", expectedError, err.Error())\n\t\t}\n\t})\n\n\tt.Run(\"ExecFlagBeforePositionalArgs\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\n\t\t// Test the correct usage: -exec flag before positional arguments\n\t\targs := []string{\"-exec\", \"echo test\", \"test.db\", \"s3://bucket/test.db\"}\n\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error when -exec flag is positioned correctly: %v\", err)\n\t\t}\n\n\t\t// Verify the exec command was set correctly\n\t\tif cmd.Config.Exec != \"echo test\" {\n\t\t\tt.Errorf(\"expected exec command to be %q, got %q\", \"echo test\", cmd.Config.Exec)\n\t\t}\n\t})\n\n\tt.Run(\"ConfigFlagAfterPositionalArgs\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\n\t\t// Test other flags after positional arguments\n\t\targs := []string{\"test.db\", \"s3://bucket/test.db\", \"-config\", \"/path/to/config\"}\n\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when -config flag is positioned after positional arguments\")\n\t\t}\n\n\t\texpectedError := `flag \"-config\" must be positioned before DB_PATH and REPLICA_URL arguments`\n\t\tif !strings.Contains(err.Error(), expectedError) {\n\t\t\tt.Errorf(\"expected error message to contain %q, got %q\", expectedError, err.Error())\n\t\t}\n\t})\n\n\tt.Run(\"MultipleFlags\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\n\t\t// Test multiple flags in correct position\n\t\targs := []string{\"-exec\", \"echo test\", \"-no-expand-env\", \"test.db\", \"s3://bucket/test.db\"}\n\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error with multiple flags positioned correctly: %v\", err)\n\t\t}\n\n\t\t// Verify the exec command was set correctly\n\t\tif cmd.Config.Exec != \"echo test\" {\n\t\t\tt.Errorf(\"expected exec command to be %q, got %q\", \"echo test\", cmd.Config.Exec)\n\t\t}\n\t})\n\n\tt.Run(\"OnlyDatabasePathProvided\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\n\t\t// Test with only database path (should error but for different reason)\n\t\targs := []string{\"test.db\"}\n\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when only database path is provided without replica URL\")\n\t\t}\n\n\t\t// Should get the \"must specify at least one replica URL\" error, not the flag positioning error\n\t\texpectedError := \"must specify at least one replica URL\"\n\t\tif !strings.Contains(err.Error(), expectedError) {\n\t\t\tt.Errorf(\"expected error message to contain %q, got %q\", expectedError, err.Error())\n\t\t}\n\t})\n}\n\nfunc TestReplicateCommand_ParseFlags_LogLevel(t *testing.T) {\n\tt.Run(\"LogLevelWithCLIArgs\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-log-level\", \"debug\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif cmd.Config.Logging.Level != \"debug\" {\n\t\t\tt.Errorf(\"expected log level to be %q, got %q\", \"debug\", cmd.Config.Logging.Level)\n\t\t}\n\t})\n\n\tt.Run(\"LogLevelTrace\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-log-level\", \"trace\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif cmd.Config.Logging.Level != \"trace\" {\n\t\t\tt.Errorf(\"expected log level to be %q, got %q\", \"trace\", cmd.Config.Logging.Level)\n\t\t}\n\t})\n\n\tt.Run(\"LogLevelError\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-log-level\", \"error\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif cmd.Config.Logging.Level != \"error\" {\n\t\t\tt.Errorf(\"expected log level to be %q, got %q\", \"error\", cmd.Config.Logging.Level)\n\t\t}\n\t})\n\n\tt.Run(\"LogLevelDefaultsToInfo\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif cmd.Config.Logging.Level != \"INFO\" {\n\t\t\tt.Errorf(\"expected log level to default to %q, got %q\", \"INFO\", cmd.Config.Logging.Level)\n\t\t}\n\t})\n\n\tt.Run(\"LogLevelWithOtherFlags\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-log-level\", \"warn\", \"-once\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif cmd.Config.Logging.Level != \"warn\" {\n\t\t\tt.Errorf(\"expected log level to be %q, got %q\", \"warn\", cmd.Config.Logging.Level)\n\t\t}\n\t})\n\n\tt.Run(\"LogLevelAfterPositionalArgs\", func(t *testing.T) {\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"test.db\", \"file:///tmp/replica\", \"-log-level\", \"debug\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when -log-level flag is positioned after positional arguments\")\n\t\t}\n\t\texpectedError := `flag \"-log-level\" must be positioned before DB_PATH and REPLICA_URL arguments`\n\t\tif !strings.Contains(err.Error(), expectedError) {\n\t\t\tt.Errorf(\"expected error message to contain %q, got %q\", expectedError, err.Error())\n\t\t}\n\t})\n\n\tt.Run(\"LogLevelFlagOverridesEnvVar\", func(t *testing.T) {\n\t\t// Set LOG_LEVEL env var to a different value\n\t\toldEnv := os.Getenv(\"LOG_LEVEL\")\n\t\tos.Setenv(\"LOG_LEVEL\", \"error\")\n\t\tdefer func() {\n\t\t\tif oldEnv == \"\" {\n\t\t\t\tos.Unsetenv(\"LOG_LEVEL\")\n\t\t\t} else {\n\t\t\t\tos.Setenv(\"LOG_LEVEL\", oldEnv)\n\t\t\t}\n\t\t}()\n\n\t\tcmd := main.NewReplicateCommand()\n\t\targs := []string{\"-log-level\", \"debug\", \"test.db\", \"file:///tmp/replica\"}\n\t\terr := cmd.ParseFlags(context.Background(), args)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// CLI flag should take precedence, setting LOG_LEVEL env var to \"debug\"\n\t\tif got := os.Getenv(\"LOG_LEVEL\"); got != \"debug\" {\n\t\t\tt.Errorf(\"expected LOG_LEVEL env var to be %q (CLI flag), got %q\", \"debug\", got)\n\t\t}\n\t\tif cmd.Config.Logging.Level != \"debug\" {\n\t\t\tt.Errorf(\"expected config log level to be %q, got %q\", \"debug\", cmd.Config.Logging.Level)\n\t\t}\n\t})\n}\n" }, { "path": "cmd/litestream/reset.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"flag\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\n// ResetCommand is a command for resetting local Litestream state for a database.\ntype ResetCommand struct{}\n\n// Run executes the command.\nfunc (c *ResetCommand) Run(ctx context.Context, args []string) (err error) {\n\tfs := flag.NewFlagSet(\"litestream-reset\", flag.ContinueOnError)\n\tconfigPath, noExpandEnv := registerConfigFlag(fs)\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\t// Validate arguments - need exactly one database path\n\tif fs.NArg() == 0 {\n\t\treturn fmt.Errorf(\"database path required\")\n\t} else if fs.NArg() > 1 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\n\tdbPath := fs.Arg(0)\n\n\t// Make absolute if needed\n\tif !filepath.IsAbs(dbPath) {\n\t\tif dbPath, err = filepath.Abs(dbPath); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// Load configuration to find the database (if config exists)\n\tvar dbConfig *DBConfig\n\tif *configPath != \"\" {\n\t\tconfig, configErr := ReadConfigFile(*configPath, !*noExpandEnv)\n\t\tif configErr != nil {\n\t\t\treturn fmt.Errorf(\"cannot read config: %w\", configErr)\n\t\t}\n\n\t\t// Find database config\n\t\tfor _, dbc := range config.DBs {\n\t\t\texpandedPath := dbc.Path\n\t\t\tif !filepath.IsAbs(expandedPath) {\n\t\t\t\texpandedPath, _ = filepath.Abs(expandedPath)\n\t\t\t}\n\t\t\tif expandedPath == dbPath {\n\t\t\t\tdbConfig = dbc\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t}\n\n\t// If no config found, check if database file exists\n\tif _, err := os.Stat(dbPath); os.IsNotExist(err) {\n\t\treturn fmt.Errorf(\"database does not exist: %s\", dbPath)\n\t} else if err != nil {\n\t\treturn fmt.Errorf(\"cannot access database: %w\", err)\n\t}\n\n\t// Create DB instance\n\tvar db *litestream.DB\n\tif dbConfig != nil {\n\t\tdb, err = NewDBFromConfig(dbConfig)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"cannot create database from config: %w\", err)\n\t\t}\n\t} else {\n\t\tdb = litestream.NewDB(dbPath)\n\t}\n\n\t// Check if meta path exists\n\tmetaPath := db.MetaPath()\n\tif _, err := os.Stat(metaPath); os.IsNotExist(err) {\n\t\tfmt.Printf(\"No local state to reset for %s\\n\", dbPath)\n\t\tfmt.Printf(\"Meta directory does not exist: %s\\n\", metaPath)\n\t\treturn nil\n\t}\n\n\t// Perform the reset\n\tfmt.Printf(\"Resetting local Litestream state for: %s\\n\", dbPath)\n\tfmt.Printf(\"Removing: %s\\n\", db.LTXDir())\n\n\tif err := db.ResetLocalState(ctx); err != nil {\n\t\treturn fmt.Errorf(\"reset failed: %w\", err)\n\t}\n\n\tfmt.Println(\"Reset complete. Next replication sync will create a fresh snapshot.\")\n\treturn nil\n}\n\n// Usage prints the help screen to STDOUT.\nfunc (c *ResetCommand) Usage() {\n\tfmt.Printf(`\nThe reset command clears local Litestream state for a database.\n\nThis is useful for recovering from corrupted or missing LTX files. The reset\nremoves local LTX files from the metadata directory, forcing Litestream to\ncreate a fresh snapshot on the next sync. The database file itself is not\nmodified.\n\nUsage:\n\n\tlitestream reset [arguments] \n\nArguments:\n\n\t-config PATH\n\t Specifies the configuration file.\n\t Defaults to %s\n\n\t-no-expand-env\n\t Disables environment variable expansion in configuration file.\n\nExamples:\n\n\t# Reset local state for a specific database\n\tlitestream reset /path/to/database.db\n\n\t# Reset using a specific configuration file\n\tlitestream reset -config /etc/litestream.yml /path/to/database.db\n\n`[1:],\n\t\tDefaultConfigPath(),\n\t)\n}\n" }, { "path": "cmd/litestream/restore.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"flag\"\n\t\"fmt\"\n\t\"log/slog\"\n\t\"os\"\n\t\"strconv\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\n// RestoreCommand represents a command to restore a database from a backup.\ntype RestoreCommand struct{}\n\n// Run executes the command.\nfunc (c *RestoreCommand) Run(ctx context.Context, args []string) (err error) {\n\topt := litestream.NewRestoreOptions()\n\n\tfs := flag.NewFlagSet(\"litestream-restore\", flag.ContinueOnError)\n\tconfigPath, noExpandEnv := registerConfigFlag(fs)\n\tfs.StringVar(&opt.OutputPath, \"o\", \"\", \"output path\")\n\tfs.Var((*txidVar)(&opt.TXID), \"txid\", \"transaction ID\")\n\tfs.IntVar(&opt.Parallelism, \"parallelism\", opt.Parallelism, \"parallelism\")\n\tifDBNotExists := fs.Bool(\"if-db-not-exists\", false, \"\")\n\tifReplicaExists := fs.Bool(\"if-replica-exists\", false, \"\")\n\ttimestampStr := fs.String(\"timestamp\", \"\", \"timestamp\")\n\tfs.BoolVar(&opt.Follow, \"f\", false, \"follow mode\")\n\tfs.DurationVar(&opt.FollowInterval, \"follow-interval\", opt.FollowInterval, \"polling interval for follow mode\")\n\tintegrityCheck := fs.String(\"integrity-check\", \"none\", \"post-restore integrity check: none, quick, or full\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t} else if fs.NArg() == 0 || fs.Arg(0) == \"\" {\n\t\treturn fmt.Errorf(\"database path or replica URL required\")\n\t} else if fs.NArg() > 1 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\n\tinitLog(os.Stdout, \"INFO\", \"text\")\n\n\t// When follow mode is enabled, set up signal handling so Ctrl+C stops\n\t// the follow loop cleanly.\n\tif opt.Follow {\n\t\tch := signalChan()\n\t\tcancelCtx, cancel := context.WithCancel(ctx)\n\t\tgo func() {\n\t\t\tselect {\n\t\t\tcase <-ch:\n\t\t\t\tcancel()\n\t\t\tcase <-cancelCtx.Done():\n\t\t\t}\n\t\t}()\n\t\tctx = cancelCtx\n\t\tdefer cancel()\n\t}\n\n\tswitch *integrityCheck {\n\tcase \"none\":\n\t\topt.IntegrityCheck = litestream.IntegrityCheckNone\n\tcase \"quick\":\n\t\topt.IntegrityCheck = litestream.IntegrityCheckQuick\n\tcase \"full\":\n\t\topt.IntegrityCheck = litestream.IntegrityCheckFull\n\tdefault:\n\t\treturn fmt.Errorf(\"invalid -integrity-check value: %s\", *integrityCheck)\n\t}\n\n\t// Parse timestamp, if specified.\n\tif *timestampStr != \"\" {\n\t\tif opt.Timestamp, err = time.Parse(time.RFC3339, *timestampStr); err != nil {\n\t\t\treturn errors.New(\"invalid -timestamp, must specify in ISO 8601 format (e.g. 2000-01-01T00:00:00Z)\")\n\t\t}\n\t}\n\n\t// Determine replica to restore from.\n\tvar r *litestream.Replica\n\tif litestream.IsURL(fs.Arg(0)) {\n\t\tif *configPath != \"\" {\n\t\t\treturn fmt.Errorf(\"cannot specify a replica URL and the -config flag\")\n\t\t}\n\t\tif r, err = c.loadFromURL(ctx, fs.Arg(0), *ifDBNotExists, &opt); errors.Is(err, errSkipDBExists) {\n\t\t\tslog.Info(\"database already exists, skipping\")\n\t\t\treturn nil\n\t\t} else if err != nil {\n\t\t\treturn err\n\t\t}\n\t} else {\n\t\tif *configPath == \"\" {\n\t\t\t*configPath = DefaultConfigPath()\n\t\t}\n\t\tif r, err = c.loadFromConfig(ctx, fs.Arg(0), *configPath, !*noExpandEnv, *ifDBNotExists, &opt); errors.Is(err, errSkipDBExists) {\n\t\t\tslog.Info(\"database already exists, skipping\")\n\t\t\treturn nil\n\t\t} else if err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tif err := r.Restore(ctx, opt); errors.Is(err, litestream.ErrTxNotAvailable) {\n\t\tif *ifReplicaExists {\n\t\t\tslog.Info(\"no matching backups found\")\n\t\t\treturn nil\n\t\t}\n\t\treturn fmt.Errorf(\"no matching backup files available\")\n\t} else if err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\n// loadFromURL creates a replica & updates the restore options from a replica URL.\nfunc (c *RestoreCommand) loadFromURL(ctx context.Context, replicaURL string, ifDBNotExists bool, opt *litestream.RestoreOptions) (*litestream.Replica, error) {\n\tif opt.OutputPath == \"\" {\n\t\treturn nil, fmt.Errorf(\"output path required\")\n\t}\n\n\t// Exit successfully if the output file already exists.\n\tif _, err := os.Stat(opt.OutputPath); !os.IsNotExist(err) && ifDBNotExists {\n\t\treturn nil, errSkipDBExists\n\t}\n\n\tsyncInterval := litestream.DefaultSyncInterval\n\tr, err := NewReplicaFromConfig(&ReplicaConfig{\n\t\tURL: replicaURL,\n\t\tReplicaSettings: ReplicaSettings{\n\t\t\tSyncInterval: &syncInterval,\n\t\t},\n\t}, nil)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\t_, err = r.CalcRestoreTarget(ctx, *opt)\n\treturn r, err\n}\n\n// loadFromConfig returns a replica & updates the restore options from a DB reference.\nfunc (c *RestoreCommand) loadFromConfig(_ context.Context, dbPath, configPath string, expandEnv, ifDBNotExists bool, opt *litestream.RestoreOptions) (*litestream.Replica, error) {\n\t// Load configuration.\n\tconfig, err := ReadConfigFile(configPath, expandEnv)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Lookup database from configuration file by path.\n\tif dbPath, err = expand(dbPath); err != nil {\n\t\treturn nil, err\n\t}\n\tdbConfig := config.DBConfig(dbPath)\n\tif dbConfig == nil {\n\t\treturn nil, fmt.Errorf(\"database not found in config: %s\", dbPath)\n\t}\n\tdb, err := NewDBFromConfig(dbConfig)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Restore into original database path if not specified.\n\tif opt.OutputPath == \"\" {\n\t\topt.OutputPath = dbPath\n\t}\n\n\t// Exit successfully if the output file already exists.\n\tif _, err := os.Stat(opt.OutputPath); !os.IsNotExist(err) && ifDBNotExists {\n\t\treturn nil, errSkipDBExists\n\t}\n\n\treturn db.Replica, nil\n}\n\n// Usage prints the help screen to STDOUT.\nfunc (c *RestoreCommand) Usage() {\n\tfmt.Printf(`\nThe restore command recovers a database from a previous snapshot and WAL.\n\nUsage:\n\n\tlitestream restore [arguments] DB_PATH\n\n\tlitestream restore [arguments] REPLICA_URL\n\nArguments:\n\n\t-config PATH\n\t Specifies the configuration file.\n\t Defaults to %s\n\n\t-no-expand-env\n\t Disables environment variable expansion in configuration file.\n\n\t-txid TXID\n\t Restore up to a specific hex-encoded transaction ID (inclusive).\n\t Defaults to use the highest available transaction.\n\n\t-timestamp TIMESTAMP\n\t Restore to a specific point-in-time.\n\t Defaults to use the latest available backup.\n\n\t-o PATH\n\t Output path of the restored database.\n\t Defaults to original DB path.\n\n\t-if-db-not-exists\n\t Returns exit code of 0 if the database already exists.\n\n\t-if-replica-exists\n\t Returns exit code of 0 if no backups found.\n\n\t-f\n\t Follow mode. After restoring, continuously poll for and apply\n\t new changes. Similar to tail -f. The restored database should\n\t only be opened in read-only mode by consumers.\n\t Cannot be used with -txid or -timestamp.\n\n\t-follow-interval DURATION\n\t Polling interval for follow mode.\n\t Defaults to 1s.\n\n\t-parallelism NUM\n\t Determines the number of WAL files downloaded in parallel.\n\t Defaults to `+strconv.Itoa(litestream.DefaultRestoreParallelism)+`.\n\n\t-integrity-check MODE\n\t Run a post-restore integrity check on the database.\n\t MODE is one of: none, quick, full.\n\t Defaults to none.\n\n\nExamples:\n\n\t# Restore latest replica for database to original location.\n\t$ litestream restore /path/to/db\n\n\t# Restore replica for database to a given point in time.\n\t$ litestream restore -timestamp 2020-01-01T00:00:00Z /path/to/db\n\n\t# Restore latest replica for database to new /tmp directory\n\t$ litestream restore -o /tmp/db /path/to/db\n\n\t# Restore database from S3 replica URL.\n\t$ litestream restore -o /tmp/db s3://mybucket/db\n\n\t# Continuously restore (follow) a database from a replica.\n\t$ litestream restore -f -o /tmp/read-replica.db s3://mybucket/db\n\n`[1:],\n\t\tDefaultConfigPath(),\n\t)\n}\n\nvar errSkipDBExists = errors.New(\"database already exists, skipping\")\n" }, { "path": "cmd/litestream/restore_test.go", "content": "package main\n\nimport (\n\t\"flag\"\n\t\"testing\"\n\t\"time\"\n\n\tlitestream \"github.com/benbjohnson/litestream\"\n)\n\nfunc TestRestoreCommand_FollowIntervalFlag(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\targs []string\n\t\twantVal time.Duration\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"Default\",\n\t\t\targs: []string{\"/tmp/db\"},\n\t\t\twantVal: time.Second,\n\t\t},\n\t\t{\n\t\t\tname: \"CustomValue\",\n\t\t\targs: []string{\"-follow-interval\", \"500ms\", \"/tmp/db\"},\n\t\t\twantVal: 500 * time.Millisecond,\n\t\t},\n\t\t{\n\t\t\tname: \"LongerInterval\",\n\t\t\targs: []string{\"-follow-interval\", \"5s\", \"/tmp/db\"},\n\t\t\twantVal: 5 * time.Second,\n\t\t},\n\t\t{\n\t\t\tname: \"InvalidDuration\",\n\t\t\targs: []string{\"-follow-interval\", \"notaduration\", \"/tmp/db\"},\n\t\t\twantErr: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\topt := litestream.NewRestoreOptions()\n\t\t\tfs := flag.NewFlagSet(\"test\", flag.ContinueOnError)\n\t\t\tfs.DurationVar(&opt.FollowInterval, \"follow-interval\", opt.FollowInterval, \"polling interval for follow mode\")\n\n\t\t\terr := fs.Parse(tt.args)\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Fatal(\"expected error, got nil\")\n\t\t\t\t}\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t\t}\n\n\t\t\tif opt.FollowInterval != tt.wantVal {\n\t\t\t\tt.Fatalf(\"FollowInterval=%v, want %v\", opt.FollowInterval, tt.wantVal)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "cmd/litestream/start.go", "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\n// StartCommand represents the command to start replication for a database.\ntype StartCommand struct{}\n\n// Run executes the start command.\nfunc (c *StartCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-start\", flag.ContinueOnError)\n\ttimeout := fs.Int(\"timeout\", 30, \"timeout in seconds\")\n\tsocketPath := fs.String(\"socket\", \"/var/run/litestream.sock\", \"control socket path\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif fs.NArg() == 0 {\n\t\treturn fmt.Errorf(\"database path required\")\n\t}\n\tif fs.NArg() > 1 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\n\tdbPath := fs.Arg(0)\n\n\t// Create HTTP client that connects via Unix socket with timeout\n\tclientTimeout := time.Duration(*timeout) * time.Second\n\tclient := &http.Client{\n\t\tTimeout: clientTimeout,\n\t\tTransport: &http.Transport{\n\t\t\tDialContext: func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\t\treturn net.DialTimeout(\"unix\", *socketPath, clientTimeout)\n\t\t\t},\n\t\t},\n\t}\n\n\treq := litestream.StartRequest{\n\t\tPath: dbPath,\n\t\tTimeout: *timeout,\n\t}\n\treqBody, err := json.Marshal(req)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to marshal request: %w\", err)\n\t}\n\n\tresp, err := client.Post(\"http://localhost/start\", \"application/json\", bytes.NewReader(reqBody))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to connect to control socket: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read response: %w\", err)\n\t}\n\n\tif resp.StatusCode != http.StatusOK {\n\t\tvar errResp litestream.ErrorResponse\n\t\tif err := json.Unmarshal(body, &errResp); err == nil && errResp.Error != \"\" {\n\t\t\treturn fmt.Errorf(\"start failed: %s\", errResp.Error)\n\t\t}\n\t\treturn fmt.Errorf(\"start failed: %s\", string(body))\n\t}\n\n\tvar result litestream.StartResponse\n\tif err := json.Unmarshal(body, &result); err != nil {\n\t\treturn fmt.Errorf(\"failed to parse response: %w\", err)\n\t}\n\n\toutput, err := json.MarshalIndent(result, \"\", \" \")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to format response: %w\", err)\n\t}\n\tfmt.Println(string(output))\n\n\treturn nil\n}\n\n// Usage prints the help text for the start command.\nfunc (c *StartCommand) Usage() {\n\tfmt.Println(`\nusage: litestream start [OPTIONS] DB_PATH\n\nStart replication for a database.\n\nOptions:\n -timeout SECONDS\n Maximum time to wait in seconds (default: 30).\n\n -socket PATH\n Path to control socket (default: /var/run/litestream.sock).\n`[1:])\n}\n" }, { "path": "cmd/litestream/status.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"flag\"\n\t\"fmt\"\n\t\"os\"\n\t\"text/tabwriter\"\n\n\t\"github.com/dustin/go-humanize\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\n// StatusCommand is a command for displaying replication status.\ntype StatusCommand struct{}\n\n// Run executes the command.\nfunc (c *StatusCommand) Run(ctx context.Context, args []string) (err error) {\n\tfs := flag.NewFlagSet(\"litestream-status\", flag.ContinueOnError)\n\tconfigPath, noExpandEnv := registerConfigFlag(fs)\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\t// Load configuration.\n\tif *configPath == \"\" {\n\t\t*configPath = DefaultConfigPath()\n\t}\n\tconfig, err := ReadConfigFile(*configPath, !*noExpandEnv)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// If a specific database path is provided, filter to just that one.\n\tvar filterPath string\n\tif fs.NArg() > 0 {\n\t\tfilterPath = fs.Arg(0)\n\t}\n\n\t// Print status for each database.\n\tw := tabwriter.NewWriter(os.Stdout, 0, 8, 2, ' ', 0)\n\tdefer w.Flush()\n\n\tfmt.Fprintln(w, \"database\\tstatus\\tlocal txid\\twal size\")\n\n\tfor _, dbConfig := range config.DBs {\n\t\tdb, err := NewDBFromConfig(dbConfig)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Filter if path specified.\n\t\tif filterPath != \"\" && db.Path() != filterPath {\n\t\t\tcontinue\n\t\t}\n\n\t\tstatus := c.getDBStatus(db)\n\t\tfmt.Fprintf(w, \"%s\\t%s\\t%s\\t%s\\n\",\n\t\t\tdb.Path(),\n\t\t\tstatus.Status,\n\t\t\tstatus.LocalTXID,\n\t\t\tstatus.WALSize,\n\t\t)\n\t}\n\n\treturn nil\n}\n\n// DBStatus holds the status information for a single database.\ntype DBStatus struct {\n\tStatus string\n\tLocalTXID string\n\tWALSize string\n}\n\n// getDBStatus gathers status information for a database.\nfunc (c *StatusCommand) getDBStatus(db *litestream.DB) DBStatus {\n\tstatus := DBStatus{\n\t\tStatus: \"unknown\",\n\t\tLocalTXID: \"-\",\n\t\tWALSize: \"-\",\n\t}\n\n\t// Check if database file exists.\n\tdbPath := db.Path()\n\tif _, err := os.Stat(dbPath); os.IsNotExist(err) {\n\t\tstatus.Status = \"no database\"\n\t\treturn status\n\t}\n\n\t// Get WAL file size.\n\twalPath := db.WALPath()\n\tif walInfo, err := os.Stat(walPath); err == nil {\n\t\tstatus.WALSize = humanize.Bytes(uint64(walInfo.Size()))\n\t} else if os.IsNotExist(err) {\n\t\tstatus.WALSize = \"0 B\"\n\t}\n\n\t// Get local TXID from L0 directory.\n\t_, maxTXID, err := db.MaxLTX()\n\tif err == nil && maxTXID > 0 {\n\t\tstatus.LocalTXID = maxTXID.String()\n\t\tstatus.Status = \"ok\"\n\t} else if err == nil {\n\t\tstatus.Status = \"not initialized\"\n\t} else {\n\t\tstatus.Status = \"error\"\n\t}\n\n\treturn status\n}\n\n// Usage prints the help screen to STDOUT.\nfunc (c *StatusCommand) Usage() {\n\tfmt.Printf(`\nThe status command displays the replication status of databases.\n\nUsage:\n\n\tlitestream status [arguments] [database path]\n\nArguments:\n\n\t-config PATH\n\t Specifies the configuration file.\n\t Defaults to %s\n\n\t-no-expand-env\n\t Disables environment variable expansion in configuration file.\n\nIf a database path is provided, only that database's status is shown.\nOtherwise, all configured databases are displayed.\n\nOutput columns:\n database Path to the SQLite database\n status Current status (ok, not initialized, no database, error)\n local txid Latest local transaction ID\n wal size Current WAL file size\n\nNote: To see replica TXID and sync status, use the MCP tools or check logs\nwhile the replication daemon is running.\n\n`[1:],\n\t\tDefaultConfigPath(),\n\t)\n}\n" }, { "path": "cmd/litestream/status_test.go", "content": "package main_test\n\nimport (\n\t\"context\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\tmain \"github.com/benbjohnson/litestream/cmd/litestream\"\n)\n\nfunc TestStatusCommand_Run(t *testing.T) {\n\tt.Run(\"NoConfig\", func(t *testing.T) {\n\t\tcmd := &main.StatusCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-config\", \"/nonexistent/config.yml\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for missing config\")\n\t\t}\n\t})\n\n\tt.Run(\"WithConfig\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"test.db\")\n\t\tconfigPath := filepath.Join(dir, \"litestream.yml\")\n\n\t\t// Create a SQLite database.\n\t\tif err := os.WriteFile(dbPath, []byte{}, 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create config file.\n\t\tconfig := `dbs:\n - path: ` + dbPath + `\n replicas:\n - url: file://` + filepath.Join(dir, \"replica\") + `\n`\n\t\tif err := os.WriteFile(configPath, []byte(config), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tcmd := &main.StatusCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-config\", configPath})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"FilterByPath\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"test.db\")\n\t\tconfigPath := filepath.Join(dir, \"litestream.yml\")\n\n\t\t// Create a SQLite database.\n\t\tif err := os.WriteFile(dbPath, []byte{}, 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create config file.\n\t\tconfig := `dbs:\n - path: ` + dbPath + `\n replicas:\n - url: file://` + filepath.Join(dir, \"replica\") + `\n`\n\t\tif err := os.WriteFile(configPath, []byte(config), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tcmd := &main.StatusCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-config\", configPath, dbPath})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n}\n" }, { "path": "cmd/litestream/stop.go", "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\n// StopCommand represents the command to stop replication for a database.\ntype StopCommand struct{}\n\n// Run executes the stop command.\nfunc (c *StopCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-stop\", flag.ContinueOnError)\n\ttimeout := fs.Int(\"timeout\", 30, \"timeout in seconds\")\n\tsocketPath := fs.String(\"socket\", \"/var/run/litestream.sock\", \"control socket path\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif fs.NArg() == 0 {\n\t\treturn fmt.Errorf(\"database path required\")\n\t}\n\tif fs.NArg() > 1 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\n\tdbPath := fs.Arg(0)\n\n\t// Create HTTP client that connects via Unix socket with timeout\n\tclientTimeout := time.Duration(*timeout) * time.Second\n\tclient := &http.Client{\n\t\tTimeout: clientTimeout,\n\t\tTransport: &http.Transport{\n\t\t\tDialContext: func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\t\treturn net.DialTimeout(\"unix\", *socketPath, clientTimeout)\n\t\t\t},\n\t\t},\n\t}\n\n\treq := litestream.StopRequest{\n\t\tPath: dbPath,\n\t\tTimeout: *timeout,\n\t}\n\treqBody, err := json.Marshal(req)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to marshal request: %w\", err)\n\t}\n\n\tresp, err := client.Post(\"http://localhost/stop\", \"application/json\", bytes.NewReader(reqBody))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to connect to control socket: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read response: %w\", err)\n\t}\n\n\tif resp.StatusCode != http.StatusOK {\n\t\tvar errResp litestream.ErrorResponse\n\t\tif err := json.Unmarshal(body, &errResp); err == nil && errResp.Error != \"\" {\n\t\t\treturn fmt.Errorf(\"stop failed: %s\", errResp.Error)\n\t\t}\n\t\treturn fmt.Errorf(\"stop failed: %s\", string(body))\n\t}\n\n\tvar result litestream.StopResponse\n\tif err := json.Unmarshal(body, &result); err != nil {\n\t\treturn fmt.Errorf(\"failed to parse response: %w\", err)\n\t}\n\n\toutput, err := json.MarshalIndent(result, \"\", \" \")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to format response: %w\", err)\n\t}\n\tfmt.Println(string(output))\n\n\treturn nil\n}\n\n// Usage prints the help text for the stop command.\nfunc (c *StopCommand) Usage() {\n\tfmt.Println(`\nusage: litestream stop [OPTIONS] DB_PATH\n\nStop replication for a database.\nStop always waits for shutdown and final sync.\n\nOptions:\n -timeout SECONDS\n Maximum time to wait in seconds (default: 30).\n\n -socket PATH\n Path to control socket (default: /var/run/litestream.sock).\n`[1:])\n}\n" }, { "path": "cmd/litestream/sync.go", "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\n// SyncCommand represents the command to force an immediate sync for a database.\ntype SyncCommand struct{}\n\n// Run executes the sync command.\nfunc (c *SyncCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-sync\", flag.ContinueOnError)\n\ttimeout := fs.Int(\"timeout\", 30, \"timeout in seconds\")\n\tsocketPath := fs.String(\"socket\", \"/var/run/litestream.sock\", \"control socket path\")\n\twait := fs.Bool(\"wait\", false, \"block until sync completes\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif fs.NArg() == 0 {\n\t\treturn fmt.Errorf(\"database path required\")\n\t}\n\tif fs.NArg() > 1 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\tif *timeout <= 0 {\n\t\treturn fmt.Errorf(\"timeout must be greater than 0\")\n\t}\n\n\tdbPath := fs.Arg(0)\n\n\tclientTimeout := time.Duration(*timeout) * time.Second\n\tclient := &http.Client{\n\t\tTimeout: clientTimeout,\n\t\tTransport: &http.Transport{\n\t\t\tDialContext: func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\t\treturn net.DialTimeout(\"unix\", *socketPath, clientTimeout)\n\t\t\t},\n\t\t},\n\t}\n\n\treq := litestream.SyncRequest{\n\t\tPath: dbPath,\n\t\tWait: *wait,\n\t\tTimeout: *timeout,\n\t}\n\treqBody, err := json.Marshal(req)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to marshal request: %w\", err)\n\t}\n\n\tresp, err := client.Post(\"http://localhost/sync\", \"application/json\", bytes.NewReader(reqBody))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to connect to control socket: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read response: %w\", err)\n\t}\n\n\tif resp.StatusCode != http.StatusOK {\n\t\tvar errResp litestream.ErrorResponse\n\t\tif err := json.Unmarshal(body, &errResp); err == nil && errResp.Error != \"\" {\n\t\t\treturn fmt.Errorf(\"sync failed: %s\", errResp.Error)\n\t\t}\n\t\treturn fmt.Errorf(\"sync failed: %s\", string(body))\n\t}\n\n\tvar result litestream.SyncResponse\n\tif err := json.Unmarshal(body, &result); err != nil {\n\t\treturn fmt.Errorf(\"failed to parse response: %w\", err)\n\t}\n\n\toutput, err := json.MarshalIndent(result, \"\", \" \")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to format response: %w\", err)\n\t}\n\tfmt.Println(string(output))\n\n\treturn nil\n}\n\n// Usage prints the help text for the sync command.\nfunc (c *SyncCommand) Usage() {\n\tfmt.Println(`\nusage: litestream sync [OPTIONS] DB_PATH\n\nForce an immediate sync for a database.\n\nOptions:\n -wait\n Block until sync completes including remote replication (default: false).\n\n -timeout SECONDS\n Maximum time to wait in seconds, best-effort (default: 30).\n\n -socket PATH\n Path to control socket (default: /var/run/litestream.sock).\n`[1:])\n}\n" }, { "path": "cmd/litestream/sync_test.go", "content": "package main_test\n\nimport (\n\t\"context\"\n\t\"testing\"\n\n\t\"github.com/benbjohnson/litestream\"\n\tmain \"github.com/benbjohnson/litestream/cmd/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestSyncCommand_Run(t *testing.T) {\n\tt.Run(\"MissingDBPath\", func(t *testing.T) {\n\t\tcmd := &main.SyncCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/tmp/test.sock\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for missing database path\")\n\t\t}\n\t\tif err.Error() != \"database path required\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"TooManyArguments\", func(t *testing.T) {\n\t\tcmd := &main.SyncCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/tmp/test.sock\", \"/tmp/a.db\", \"extra\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for too many arguments\")\n\t\t}\n\t\tif err.Error() != \"too many arguments\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"InvalidTimeoutZero\", func(t *testing.T) {\n\t\tcmd := &main.SyncCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-timeout\", \"0\", \"/tmp/test.db\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for zero timeout\")\n\t\t}\n\t\tif err.Error() != \"timeout must be greater than 0\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"SocketConnectionError\", func(t *testing.T) {\n\t\tcmd := &main.SyncCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/nonexistent/socket.sock\", \"/tmp/test.db\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for socket connection failure\")\n\t\t}\n\t})\n\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t_, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT)`)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\tcmd := &main.SyncCommand{}\n\t\terr = cmd.Run(t.Context(), []string{\"-socket\", server.SocketPath, db.Path()})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"SuccessWithWait\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t_, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT)`)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\tcmd := &main.SyncCommand{}\n\t\terr = cmd.Run(t.Context(), []string{\"-socket\", server.SocketPath, \"-wait\", db.Path()})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n}\n" }, { "path": "cmd/litestream/unregister.go", "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\ntype UnregisterCommand struct{}\n\nfunc (c *UnregisterCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-unregister\", flag.ContinueOnError)\n\ttimeout := fs.Int(\"timeout\", 30, \"timeout in seconds\")\n\tsocketPath := fs.String(\"socket\", \"/var/run/litestream.sock\", \"control socket path\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif fs.NArg() == 0 {\n\t\treturn fmt.Errorf(\"database path required\")\n\t}\n\tif fs.NArg() > 1 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\tif *timeout <= 0 {\n\t\treturn fmt.Errorf(\"timeout must be greater than 0\")\n\t}\n\n\tdbPath := fs.Arg(0)\n\n\t// Create HTTP client that connects via Unix socket with timeout.\n\tclientTimeout := time.Duration(*timeout) * time.Second\n\tclient := &http.Client{\n\t\tTimeout: clientTimeout,\n\t\tTransport: &http.Transport{\n\t\t\tDialContext: func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\t\treturn net.DialTimeout(\"unix\", *socketPath, clientTimeout)\n\t\t\t},\n\t\t},\n\t}\n\n\treq := litestream.UnregisterDatabaseRequest{\n\t\tPath: dbPath,\n\t\tTimeout: *timeout,\n\t}\n\treqBody, err := json.Marshal(req)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to marshal request: %w\", err)\n\t}\n\n\tresp, err := client.Post(\"http://localhost/unregister\", \"application/json\", bytes.NewReader(reqBody))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to connect to control socket: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read response: %w\", err)\n\t}\n\n\tif resp.StatusCode != http.StatusOK {\n\t\tvar errResp litestream.ErrorResponse\n\t\tif err := json.Unmarshal(body, &errResp); err == nil && errResp.Error != \"\" {\n\t\t\treturn fmt.Errorf(\"unregister failed: %s\", errResp.Error)\n\t\t}\n\t\treturn fmt.Errorf(\"unregister failed: %s\", string(body))\n\t}\n\n\tvar result litestream.UnregisterDatabaseResponse\n\tif err := json.Unmarshal(body, &result); err != nil {\n\t\treturn fmt.Errorf(\"failed to parse response: %w\", err)\n\t}\n\n\toutput, err := json.MarshalIndent(result, \"\", \" \")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to format response: %w\", err)\n\t}\n\tfmt.Println(string(output))\n\n\treturn nil\n}\n\nfunc (c *UnregisterCommand) Usage() {\n\tfmt.Println(`\nusage: litestream unregister [OPTIONS] DB_PATH\n\nUnregister a database from replication.\n\nArguments:\n DB_PATH Path to the SQLite database file.\n\nOptions:\n -timeout SECONDS\n Maximum time to wait in seconds (default: 30).\n\n -socket PATH\n Path to control socket (default: /var/run/litestream.sock).\n`[1:])\n}\n" }, { "path": "cmd/litestream/unregister_test.go", "content": "package main_test\n\nimport (\n\t\"context\"\n\t\"testing\"\n\n\t\"github.com/benbjohnson/litestream\"\n\tmain \"github.com/benbjohnson/litestream/cmd/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestUnregisterCommand_Run(t *testing.T) {\n\tt.Run(\"MissingPath\", func(t *testing.T) {\n\t\tcmd := &main.UnregisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/tmp/test.sock\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for missing path\")\n\t\t}\n\t\tif err.Error() != \"database path required\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"TooManyArguments\", func(t *testing.T) {\n\t\tcmd := &main.UnregisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/tmp/test.sock\", \"/tmp/test.db\", \"extra\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for too many arguments\")\n\t\t}\n\t\tif err.Error() != \"too many arguments\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"InvalidTimeoutZero\", func(t *testing.T) {\n\t\tcmd := &main.UnregisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-timeout\", \"0\", \"/tmp/test.db\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for zero timeout\")\n\t\t}\n\t\tif err.Error() != \"timeout must be greater than 0\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"InvalidTimeoutNegative\", func(t *testing.T) {\n\t\tcmd := &main.UnregisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-timeout\", \"-1\", \"/tmp/test.db\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for negative timeout\")\n\t\t}\n\t\tif err.Error() != \"timeout must be greater than 0\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"SocketConnectionError\", func(t *testing.T) {\n\t\tcmd := &main.UnregisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", \"/nonexistent/socket.sock\", \"/tmp/test.db\"})\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for socket connection failure\")\n\t\t}\n\t})\n\n\tt.Run(\"NotFoundIsIdempotent\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(context.Background())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\tcmd := &main.UnregisterCommand{}\n\t\t// Should not error even though database doesn't exist.\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", server.SocketPath, \"/nonexistent/db\"})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\t\tdbPath := db.Path()\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\tif err := store.Open(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(context.Background())\n\n\t\t// Verify database is initially in store.\n\t\tif len(store.DBs()) != 1 {\n\t\t\tt.Fatalf(\"expected 1 database in store, got %d\", len(store.DBs()))\n\t\t}\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\tif err := server.Start(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer server.Close()\n\n\t\tcmd := &main.UnregisterCommand{}\n\t\terr := cmd.Run(context.Background(), []string{\"-socket\", server.SocketPath, dbPath})\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t// Verify database was unregistered from store.\n\t\tif len(store.DBs()) != 0 {\n\t\t\tt.Errorf(\"expected 0 databases in store, got %d\", len(store.DBs()))\n\t\t}\n\t})\n}\n" }, { "path": "cmd/litestream/version.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"flag\"\n\t\"fmt\"\n)\n\n// VersionCommand represents a command to print the current version.\ntype VersionCommand struct{}\n\n// Run executes the command.\nfunc (c *VersionCommand) Run(_ context.Context, args []string) (err error) {\n\tfs := flag.NewFlagSet(\"litestream-version\", flag.ContinueOnError)\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tfmt.Println(Version)\n\n\treturn nil\n}\n\n// Usage prints the help screen to STDOUT.\nfunc (c *VersionCommand) Usage() {\n\tfmt.Println(`\nPrints the version.\n\nUsage:\n\n\tlitestream version\n`[1:])\n}\n" }, { "path": "cmd/litestream-test/README.md", "content": "# litestream-test\n\nA CLI testing harness for Litestream that provides tools for database population, load generation, and replication validation.\n\n## Overview\n\n`litestream-test` is a purpose-built tool for testing Litestream's replication functionality across various scenarios. It provides commands for:\n\n- Quickly populating databases to specific sizes\n- Generating continuous load with configurable patterns\n- Shrinking databases to test compaction scenarios\n- Validating replication integrity after restore\n\n## Installation\n\n```bash\ngo build -o bin/litestream-test ./cmd/litestream-test\n```\n\n## Commands\n\n### populate\n\nQuickly populate a database to a target size with structured test data.\n\n```bash\nlitestream-test populate -db [options]\n```\n\n**Options:**\n- `-db` - Database path (required)\n- `-target-size` - Target database size (default: \"100MB\", examples: \"1GB\", \"500MB\", \"50MB\")\n- `-row-size` - Average row size in bytes (default: 1024)\n- `-batch-size` - Rows per transaction (default: 1000)\n- `-table-count` - Number of tables to create (default: 1)\n- `-index-ratio` - Percentage of columns to index, 0.0-1.0 (default: 0.2)\n- `-page-size` - SQLite page size in bytes (default: 4096)\n\n**Examples:**\n```bash\nlitestream-test populate -db /tmp/test.db -target-size 1GB\nlitestream-test populate -db /tmp/test.db -target-size 50MB -batch-size 10000\nlitestream-test populate -db /tmp/test.db -target-size 1.5GB -page-size 4096\n```\n\n**Use Cases:**\n- Creating test databases of specific sizes\n- Testing SQLite lock page boundary (1GB with 4KB pages)\n- Generating initial data before replication tests\n\n### load\n\nGenerate continuous write and read load on a database with configurable patterns.\n\n```bash\nlitestream-test load -db [options]\n```\n\n**Options:**\n- `-db` - Database path (required, must exist)\n- `-write-rate` - Writes per second (default: 100)\n- `-duration` - How long to run (default: 1m, examples: \"30s\", \"5m\", \"2h\", \"8h\")\n- `-pattern` - Write pattern (default: \"constant\")\n - `constant` - Steady write rate\n - `burst` - Periodic bursts of activity\n - `random` - Random write intervals\n - `wave` - Sinusoidal pattern simulating varying load\n- `-payload-size` - Size of each write in bytes (default: 1024)\n- `-read-ratio` - Read/write ratio, 0.0-1.0 (default: 0.2)\n- `-workers` - Number of concurrent workers (default: 1)\n\n**Examples:**\n```bash\nlitestream-test load -db /tmp/test.db -write-rate 50 -duration 5m\nlitestream-test load -db /tmp/test.db -write-rate 100 -duration 2h -pattern wave\nlitestream-test load -db /tmp/test.db -write-rate 200 -duration 8h -workers 4 -pattern burst\n```\n\n**Use Cases:**\n- Stress testing replication under sustained load\n- Testing checkpoint behavior with various patterns\n- Simulating production workloads for overnight tests\n- Testing concurrent operations with multiple workers\n\n### shrink\n\nShrink a database by deleting data, useful for testing compaction scenarios.\n\n```bash\nlitestream-test shrink -db [options]\n```\n\n**Use Cases:**\n- Testing database shrinkage and compaction\n- Simulating data deletion scenarios\n- Testing retention cleanup behavior\n\n### validate\n\nValidate that a replica can be restored and matches the source database.\n\n```bash\nlitestream-test validate [options]\n```\n\n**Options:**\n- `-source-db` - Original database path (required)\n- `-replica-url` - Replica URL to validate (e.g., \"file:///path\", \"s3://bucket/path\")\n- `-restored-db` - Path for restored database (default: source-db + \".restored\")\n- `-check-type` - Type of validation (default: \"quick\")\n - `quick` - Fast row count comparison\n - `integrity` - SQLite PRAGMA integrity_check\n - `checksum` - Full database checksum comparison\n - `full` - All validation types\n- `-ltx-continuity` - Check LTX file continuity (default: false)\n- `-config` - Litestream config file path (alternative to replica-url)\n\n**Examples:**\n```bash\nlitestream-test validate -source-db /tmp/test.db -replica-url file:///tmp/replica\nlitestream-test validate -source-db /tmp/test.db -replica-url s3://bucket/path -check-type full\nlitestream-test validate -source-db /tmp/test.db -config /tmp/litestream.yml -ltx-continuity\n```\n\n**Use Cases:**\n- Verifying replication integrity after tests\n- Testing restore functionality\n- Validating data consistency across replicas\n- Checking LTX file continuity\n\n### version\n\nShow version information.\n\n```bash\nlitestream-test version\n```\n\n## Usage Patterns\n\n### Basic Test Workflow\n\n```bash\nlitestream-test populate -db /tmp/test.db -target-size 100MB\n\nlitestream replicate /tmp/test.db file:///tmp/replica &\nLITESTREAM_PID=$!\n\nlitestream-test load -db /tmp/test.db -duration 5m -write-rate 50\n\nkill $LITESTREAM_PID\nwait\n\nlitestream-test validate -source-db /tmp/test.db -replica-url file:///tmp/replica\n```\n\n### Overnight Test Pattern\n\n```bash\nlitestream-test populate -db /tmp/test.db -target-size 100MB\n\nlitestream replicate -config litestream.yml &\n\nlitestream-test load -db /tmp/test.db \\\n -duration 8h \\\n -write-rate 100 \\\n -pattern wave \\\n -workers 4\n```\n\n### Large Database with Lock Page Testing\n\n```bash\nlitestream-test populate -db /tmp/test.db \\\n -target-size 1.5GB \\\n -page-size 4096 \\\n -batch-size 10000\n\nlitestream replicate /tmp/test.db s3://bucket/path &\n\nlitestream-test validate -source-db /tmp/test.db \\\n -replica-url s3://bucket/path \\\n -check-type full\n```\n\n## Integration with Test Scripts\n\nThe `litestream-test` tool is used by all scripts in the `scripts/` directory. These scripts orchestrate full test scenarios:\n\n- `scripts/*.sh` - Use litestream-test for database operations\n- `scripts/verify-test-setup.sh` - Checks that litestream-test is built\n- See `scripts/README.md` for detailed test scenario documentation\n\n## Related Documentation\n\n- [Test Scripts Documentation](./scripts/README.md) - Comprehensive test scenarios\n- [S3 Retention Testing](./S3-RETENTION-TESTING.md) - S3-specific test documentation\n- [Top-level Integration Scripts](../../scripts/README.md) - Long-running test documentation\n\n## Development\n\n### Building\n\n```bash\ngo build -o bin/litestream-test ./cmd/litestream-test\n```\n\n### Testing\n\n```bash\ngo test ./cmd/litestream-test/...\n```\n\n### Adding New Commands\n\n1. Create a new file in `cmd/litestream-test/` (e.g., `mycommand.go`)\n2. Implement the command struct and Run method\n3. Add command to switch statement in `main.go`\n4. Update this README with command documentation\n" }, { "path": "cmd/litestream-test/S3-RETENTION-TESTING.md", "content": "# S3 LTX File Retention Testing Guide\n\n## Overview\n\nThis document describes the comprehensive S3 LTX file retention testing scripts created to validate that old LTX files are properly cleaned up after their retention period expires. These tests use the local Python S3 mock server for isolated, repeatable testing.\n\n## Key Focus Areas\n\n### 1. Small Database Testing\n\n- **Database Size**: 50MB\n- **Retention Period**: 2 minutes\n- **Focus**: Basic retention behavior with minimal data\n\n### 2. Large Database Testing (Critical)\n\n- **Database Size**: 1.5GB (crosses SQLite lock page boundary)\n- **Page Size**: 4KB (lock page at #262145)\n- **Retention Period**: 3 minutes\n- **Focus**: SQLite lock page edge case + retention cleanup at scale\n\n### 3. Comprehensive Analysis\n\n- Side-by-side comparison of retention behavior\n- Performance metrics analysis\n- Best practices verification\n\n## Test Scripts\n\n### 1. `test-s3-retention-small-db.sh`\n\n**Purpose**: Test S3 LTX retention cleanup with small databases\n\n**Features**:\n- Creates 50MB database with structured test data\n- Uses local S3 mock (moto) for isolation\n- 2-minute retention period for quick testing\n- Generates multiple LTX files over time\n- Monitors cleanup activity in logs\n- Validates restoration integrity\n\n**Usage**:\n```bash\n./cmd/litestream-test/scripts/test-s3-retention-small-db.sh\n```\n\n**Duration**: ~8 minutes\n\n### 2. `test-s3-retention-large-db.sh`\n\n**Purpose**: Test S3 LTX retention cleanup with large databases crossing the 1GB SQLite lock page boundary\n\n**Features**:\n- Creates 1.5GB database (crosses lock page at 1GB)\n- Specifically tests SQLite lock page handling\n- 3-minute retention period\n- Extended monitoring for large database patterns\n- Comprehensive validation including lock page verification\n- Tests restoration of large databases\n\n**Usage**:\n```bash\n./cmd/litestream-test/scripts/test-s3-retention-large-db.sh\n```\n\n**Duration**: ~15-20 minutes\n\n### 3. `test-s3-retention-comprehensive.sh`\n\n**Purpose**: Comprehensive test runner and analysis tool\n\n**Features**:\n- Runs both small and large database tests\n- Provides comparative analysis\n- Generates detailed reports\n- Configurable test execution\n- Best practices verification\n\n**Usage**:\n```bash\n# Run all tests\n./cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh\n\n# Run only small database test\n./cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh --small-only\n\n# Run only large database test\n./cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh --large-only\n\n# Keep test files after completion\n./cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh --no-cleanup\n```\n\n**Duration**: ~25-30 minutes for full suite\n\n## SQLite Lock Page Testing\n\n### Why It Matters\n\nSQLite reserves a special lock page at exactly 1GB (offset 0x40000000) that cannot contain data. This creates a critical edge case that Litestream must handle correctly.\n\n### What We Test\n\n- **Lock Page Location**: Page #262145 (with 4KB page size)\n- **Boundary Crossing**: Databases that grow beyond 1GB\n- **Replication Integrity**: Ensure lock page is properly skipped\n- **Restoration Accuracy**: Verify restored databases maintain integrity\n\n### Lock Page Numbers by Page Size\n\n| Page Size | Lock Page # | Test Coverage |\n|-----------|-------------|---------------|\n| 4KB | 262145 | ✅ Tested |\n| 8KB | 131073 | 🔄 Possible |\n| 16KB | 65537 | 🔄 Possible |\n| 32KB | 32769 | 🔄 Possible |\n\n## Local S3 Mock Setup\n\n### Why Use Local Mock\n\n- **Isolation**: No external dependencies or costs\n- **Repeatability**: Consistent test environment\n- **Speed**: No network latency\n- **Safety**: No risk of affecting production data\n\n### How It Works\n\nThe tests use the existing `./etc/s3_mock.py` script which:\n1. Starts a local moto S3 server\n2. Creates a test bucket with unique name\n3. Runs Litestream with S3 configuration\n4. Automatically cleans up after test completion\n\n### Environment Variables Set by Mock\n\n```bash\nLITESTREAM_S3_ACCESS_KEY_ID=\"lite\"\nLITESTREAM_S3_SECRET_ACCESS_KEY=\"stream\"\nLITESTREAM_S3_BUCKET=\"test{timestamp}\"\nLITESTREAM_S3_ENDPOINT=\"http://127.0.0.1:5000\"\nLITESTREAM_S3_FORCE_PATH_STYLE=\"true\"\n```\n\n## Test Execution Flow\n\n### Small Database Test Flow\n\n1. **Setup**: Build binaries, check dependencies\n2. **Database Creation**: 50MB with indexed tables\n3. **Replication Start**: Begin S3 mock and Litestream\n4. **Data Generation**: Create LTX files over time (6 batches, 20s apart)\n5. **Retention Monitoring**: Watch for cleanup activity (4 minutes)\n6. **Validation**: Test restoration and integrity\n7. **Analysis**: Generate detailed report\n\n### Large Database Test Flow\n\n1. **Setup**: Build binaries, verify lock page calculations\n2. **Database Creation**: 1.5GB crossing lock page boundary\n3. **Replication Start**: Begin S3 mock (longer initial sync)\n4. **Data Generation**: Add incremental data around lock page\n5. **Extended Monitoring**: Watch cleanup patterns (6 minutes)\n6. **Comprehensive Validation**: Test large database restoration\n7. **Analysis**: Generate lock page specific report\n\n## Monitoring Retention Cleanup\n\n### What to Look For\n\nThe scripts monitor logs for these cleanup indicators:\n- **Direct**: \"clean\", \"delete\", \"expire\", \"retention\", \"removed\", \"purge\"\n- **Indirect**: \"old file\", \"ttl\", \"sweep\", \"vacuum\", \"evict\"\n- **LTX-specific**: \"ltx.*old\", \"snapshot.*old\", \"compress\", \"archive\"\n\n### Expected Behavior\n\n1. **Initial Period**: LTX files accumulate normally\n2. **Retention Trigger**: Cleanup begins after retention period\n3. **Ongoing**: Old files removed, new files continue to accumulate\n4. **Stabilization**: File count stabilizes at recent files only\n\n### Warning Signs\n\n- **No Cleanup**: Files accumulate indefinitely\n- **Cleanup Failures**: Error messages about S3 DELETE operations\n- **Retention Ignored**: Files older than retention period remain\n\n## Dependencies\n\n### Required Tools\n\n- **Go**: For building Litestream binaries\n- **Python 3**: For S3 mock server\n- **sqlite3**: For database operations\n- **bc**: For calculations\n\n### Python Packages\n\n```bash\npip3 install moto boto3\n```\n\n### Auto-Installation\n\nThe scripts automatically:\n- Build missing Litestream binaries\n- Install missing Python packages\n- Check for required tools\n\n## Output and Artifacts\n\n### Log Files\n\n- `/tmp/small-retention-test.log` - Small database replication log\n- `/tmp/large-retention-test.log` - Large database replication log\n- `/tmp/small-retention-config.yml` - Small database config\n- `/tmp/large-retention-config.yml` - Large database config\n\n### Database Files\n\n- `/tmp/small-retention-test.db` - Small test database\n- `/tmp/large-retention-test.db` - Large test database\n- `/tmp/small-retention-restored.db` - Restored small database\n- `/tmp/large-retention-restored.db` - Restored large database\n\n### Analysis Output\n\nEach test generates:\n- **Operation Counts**: Sync, upload, LTX operations\n- **Cleanup Indicators**: Number of cleanup-related log entries\n- **Error Analysis**: Any errors or warnings encountered\n- **Performance Metrics**: Duration, throughput, file counts\n- **Validation Results**: Integrity checks, restoration success\n\n## Integration with Existing Framework\n\n### Relationship to Existing Tests\n\nThese tests complement the existing test infrastructure:\n\n- **`test-s3-retention-cleanup.sh`**: Original retention test (more basic)\n- **`test-754-s3-scenarios.sh`**: Issue #754 specific testing\n- **Testing Framework**: Uses `litestream-test` CLI for data generation\n\n### Consistent Patterns\n\n- Use existing `etc/s3_mock.py` for S3 simulation\n- Follow naming conventions from existing scripts\n- Integrate with `litestream-test` populate/load/validate commands\n- Generate structured output for analysis\n\n## Production Validation Recommendations\n\n### After Local Testing\n\n1. **Real S3 Testing**: Run against actual S3/GCS/Azure endpoints\n2. **Network Scenarios**: Test with network interruptions\n3. **Scale Testing**: Test with production-sized databases\n4. **Cost Analysis**: Monitor S3 API calls and storage costs\n5. **Concurrent Testing**: Multiple databases simultaneously\n\n### Retention Period Guidelines\n\n- **Local Testing**: 2-3 minutes for quick feedback\n- **Staging**: 1-2 hours for realistic behavior\n- **Production**: Days to weeks based on recovery requirements\n\n### Monitoring in Production\n\n- **LTX File Counts**: Should stabilize after retention period\n- **Storage Growth**: Should level off, not grow indefinitely\n- **API Costs**: DELETE operations should occur regularly\n- **Performance**: Cleanup shouldn't impact replication performance\n\n## Troubleshooting\n\n### Common Issues\n\n#### 1. Python Dependencies Missing\n\n```bash\npip3 install moto boto3\n```\n\n#### 2. Binaries Not Found\n\n```bash\ngo build -o bin/litestream ./cmd/litestream\ngo build -o bin/litestream-test ./cmd/litestream-test\n```\n\n#### 3. Large Database Test Slow\n\n- Expected: 1.5GB takes time to create and replicate\n- Monitor progress in logs\n- Increase timeouts if needed\n\n#### 4. No Cleanup Activity Detected\n\n- May be normal: Litestream might clean up silently\n- Check S3 bucket contents manually (if using real S3)\n- Verify retention period has elapsed\n\n#### 5. Lock Page Boundary Not Crossed\n\n- Check final page count vs. lock page number\n- Increase target database size if needed\n- Verify page size settings\n\n### Debug Mode\n\nFor more verbose output:\n```bash\n# Enable debug logging\nexport LITESTREAM_DEBUG=1\n\n# Run with debug\n./cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh\n```\n\n## Summary\n\nThese retention testing scripts provide comprehensive validation of Litestream's S3 LTX file cleanup behavior across different database sizes and scenarios. They specifically address:\n\n1. **Ben's Requirements**: Local testing with Python S3 mock\n2. **SQLite Edge Cases**: Lock page boundary at 1GB\n3. **Scale Scenarios**: Both small (50MB) and large (1.5GB) databases\n4. **Retention Verification**: Multiple retention periods and monitoring\n5. **Production Readiness**: Detailed analysis and recommendations\n\nThe scripts are designed to run reliably in isolation while providing detailed insights into Litestream's retention cleanup behavior.\n" }, { "path": "cmd/litestream-test/load.go", "content": "package main\n\nimport (\n\t\"context\"\n\tcryptorand \"crypto/rand\"\n\t\"database/sql\"\n\t\"flag\"\n\t\"fmt\"\n\t\"log/slog\"\n\t\"math/rand\"\n\t\"os\"\n\t\"os/signal\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"syscall\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\ntype LoadCommand struct {\n\tMain *Main\n\n\tDB string\n\tWriteRate int\n\tDuration time.Duration\n\tPattern string\n\tPayloadSize int\n\tReadRatio float64\n\tWorkers int\n}\n\ntype LoadStats struct {\n\twrites int64\n\treads int64\n\terrors int64\n\tstartTime time.Time\n\tlastReport time.Time\n\tmu sync.Mutex\n}\n\nfunc (c *LoadCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-test load\", flag.ExitOnError)\n\tfs.StringVar(&c.DB, \"db\", \"\", \"Database path (required)\")\n\tfs.IntVar(&c.WriteRate, \"write-rate\", 100, \"Writes per second\")\n\tfs.DurationVar(&c.Duration, \"duration\", 1*time.Minute, \"How long to run\")\n\tfs.StringVar(&c.Pattern, \"pattern\", \"constant\", \"Write pattern (constant, burst, random, wave)\")\n\tfs.IntVar(&c.PayloadSize, \"payload-size\", 1024, \"Size of each write operation in bytes\")\n\tfs.Float64Var(&c.ReadRatio, \"read-ratio\", 0.2, \"Read/write ratio (0.0-1.0)\")\n\tfs.IntVar(&c.Workers, \"workers\", 1, \"Number of concurrent workers\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif c.DB == \"\" {\n\t\treturn fmt.Errorf(\"database path required\")\n\t}\n\n\tif _, err := os.Stat(c.DB); err != nil {\n\t\treturn fmt.Errorf(\"database does not exist: %w\", err)\n\t}\n\n\tslog.Info(\"Starting load generation\",\n\t\t\"db\", c.DB,\n\t\t\"write_rate\", c.WriteRate,\n\t\t\"duration\", c.Duration,\n\t\t\"pattern\", c.Pattern,\n\t\t\"payload_size\", c.PayloadSize,\n\t\t\"read_ratio\", c.ReadRatio,\n\t\t\"workers\", c.Workers,\n\t)\n\n\treturn c.generateLoad(ctx)\n}\n\nfunc (c *LoadCommand) generateLoad(ctx context.Context) error {\n\tdb, err := sql.Open(\"sqlite3\", c.DB+\"?_journal_mode=WAL\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open database: %w\", err)\n\t}\n\tdefer db.Close()\n\n\tdb.SetMaxOpenConns(c.Workers + 1)\n\tdb.SetMaxIdleConns(c.Workers)\n\n\tif err := c.ensureTestTable(db); err != nil {\n\t\treturn fmt.Errorf(\"ensure test table: %w\", err)\n\t}\n\n\tctx, cancel := context.WithTimeout(ctx, c.Duration)\n\tdefer cancel()\n\n\tsigChan := make(chan os.Signal, 1)\n\tsignal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)\n\tgo func() {\n\t\t<-sigChan\n\t\tslog.Info(\"Received interrupt signal, stopping load generation\")\n\t\tcancel()\n\t}()\n\n\tstats := &LoadStats{\n\t\tstartTime: time.Now(),\n\t\tlastReport: time.Now(),\n\t}\n\n\tvar wg sync.WaitGroup\n\tfor i := 0; i < c.Workers; i++ {\n\t\twg.Add(1)\n\t\tgo func(workerID int) {\n\t\t\tdefer wg.Done()\n\t\t\tc.worker(ctx, db, workerID, stats)\n\t\t}(i)\n\t}\n\n\tgo c.reportStats(ctx, stats)\n\n\twg.Wait()\n\n\tc.finalReport(stats)\n\treturn nil\n}\n\nfunc (c *LoadCommand) worker(ctx context.Context, db *sql.DB, workerID int, stats *LoadStats) {\n\tticker := time.NewTicker(time.Second / time.Duration(c.WriteRate/c.Workers))\n\tdefer ticker.Stop()\n\n\tdata := make([]byte, c.PayloadSize)\n\tcryptorand.Read(data)\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t\trate := c.calculateRate(stats)\n\t\t\tif rate == 0 {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif rand.Float64() < c.ReadRatio {\n\t\t\t\tif err := c.performRead(db); err != nil {\n\t\t\t\t\tatomic.AddInt64(&stats.errors, 1)\n\t\t\t\t\tslog.Error(\"Read failed\", \"error\", err)\n\t\t\t\t} else {\n\t\t\t\t\tatomic.AddInt64(&stats.reads, 1)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err := c.performWrite(db, data); err != nil {\n\t\t\t\t\tatomic.AddInt64(&stats.errors, 1)\n\t\t\t\t\tslog.Error(\"Write failed\", \"error\", err)\n\t\t\t\t} else {\n\t\t\t\t\tatomic.AddInt64(&stats.writes, 1)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc (c *LoadCommand) calculateRate(stats *LoadStats) float64 {\n\telapsed := time.Since(stats.startTime).Seconds()\n\n\tswitch c.Pattern {\n\tcase \"burst\":\n\t\tif int(elapsed)%10 < 3 {\n\t\t\treturn 2.0\n\t\t}\n\t\treturn 0.0\n\tcase \"random\":\n\t\treturn rand.Float64() * 2.0\n\tcase \"wave\":\n\t\treturn (1.0 + 0.5*waveFunction(elapsed/10.0))\n\tdefault:\n\t\treturn 1.0\n\t}\n}\n\nfunc waveFunction(t float64) float64 {\n\treturn (1.0 + 0.8*sinApprox(t)) / 2.0\n}\n\nfunc sinApprox(x float64) float64 {\n\tconst twoPi = 2 * 3.14159265359\n\tx = x - float64(int(x/twoPi))*twoPi\n\n\tif x < 3.14159265359 {\n\t\treturn 4 * x * (3.14159265359 - x) / (3.14159265359 * 3.14159265359)\n\t}\n\tx = x - 3.14159265359\n\treturn -4 * x * (3.14159265359 - x) / (3.14159265359 * 3.14159265359)\n}\n\nfunc (c *LoadCommand) ensureTestTable(db *sql.DB) error {\n\tcreateSQL := `\n\t\tCREATE TABLE IF NOT EXISTS load_test (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tdata BLOB,\n\t\t\ttext_field TEXT,\n\t\t\tint_field INTEGER,\n\t\t\ttimestamp INTEGER\n\t\t)\n\t`\n\t_, err := db.Exec(createSQL)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create table: %w\", err)\n\t}\n\n\t_, err = db.Exec(\"CREATE INDEX IF NOT EXISTS idx_load_test_timestamp ON load_test(timestamp)\")\n\treturn err\n}\n\nfunc (c *LoadCommand) performWrite(db *sql.DB, data []byte) error {\n\ttextField := fmt.Sprintf(\"load_%d\", time.Now().UnixNano())\n\tintField := rand.Int63()\n\ttimestamp := time.Now().Unix()\n\n\t_, err := db.Exec(`\n\t\tINSERT INTO load_test (data, text_field, int_field, timestamp)\n\t\tVALUES (?, ?, ?, ?)\n\t`, data, textField, intField, timestamp)\n\n\treturn err\n}\n\nfunc (c *LoadCommand) performRead(db *sql.DB) error {\n\tvar count int\n\tquery := `SELECT COUNT(*) FROM load_test WHERE timestamp > ?`\n\ttimestamp := time.Now().Add(-1 * time.Hour).Unix()\n\n\treturn db.QueryRow(query, timestamp).Scan(&count)\n}\n\nfunc (c *LoadCommand) reportStats(ctx context.Context, stats *LoadStats) {\n\tticker := time.NewTicker(5 * time.Second)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t\tstats.mu.Lock()\n\t\t\telapsed := time.Since(stats.lastReport).Seconds()\n\t\t\twrites := atomic.LoadInt64(&stats.writes)\n\t\t\treads := atomic.LoadInt64(&stats.reads)\n\t\t\terrors := atomic.LoadInt64(&stats.errors)\n\n\t\t\twriteRate := float64(writes) / elapsed\n\t\t\treadRate := float64(reads) / elapsed\n\n\t\t\tslog.Info(\"Load statistics\",\n\t\t\t\t\"writes_per_sec\", fmt.Sprintf(\"%.1f\", writeRate),\n\t\t\t\t\"reads_per_sec\", fmt.Sprintf(\"%.1f\", readRate),\n\t\t\t\t\"total_writes\", writes,\n\t\t\t\t\"total_reads\", reads,\n\t\t\t\t\"errors\", errors,\n\t\t\t\t\"elapsed\", time.Since(stats.startTime).Round(time.Second),\n\t\t\t)\n\n\t\t\tatomic.StoreInt64(&stats.writes, 0)\n\t\t\tatomic.StoreInt64(&stats.reads, 0)\n\t\t\tstats.lastReport = time.Now()\n\t\t\tstats.mu.Unlock()\n\t\t}\n\t}\n}\n\nfunc (c *LoadCommand) finalReport(stats *LoadStats) {\n\ttotalTime := time.Since(stats.startTime)\n\twrites := atomic.LoadInt64(&stats.writes)\n\treads := atomic.LoadInt64(&stats.reads)\n\terrors := atomic.LoadInt64(&stats.errors)\n\n\tslog.Info(\"Load generation complete\",\n\t\t\"duration\", totalTime.Round(time.Second),\n\t\t\"total_writes\", writes,\n\t\t\"total_reads\", reads,\n\t\t\"total_errors\", errors,\n\t\t\"avg_writes_per_sec\", fmt.Sprintf(\"%.1f\", float64(writes)/totalTime.Seconds()),\n\t\t\"avg_reads_per_sec\", fmt.Sprintf(\"%.1f\", float64(reads)/totalTime.Seconds()),\n\t)\n}\n\nfunc (c *LoadCommand) Usage() {\n\tfmt.Fprintln(c.Main.Stdout, `\nGenerate continuous load on a SQLite database for testing.\n\nUsage:\n\n\tlitestream-test load [options]\n\nOptions:\n\n\t-db PATH\n\t Database path (required)\n\n\t-write-rate RATE\n\t Target writes per second\n\t Default: 100\n\n\t-duration DURATION\n\t How long to run (e.g., \"10m\", \"1h\")\n\t Default: 1m\n\n\t-pattern PATTERN\n\t Write pattern: constant, burst, random, wave\n\t Default: constant\n\n\t-payload-size SIZE\n\t Size of each write operation in bytes\n\t Default: 1024\n\n\t-read-ratio RATIO\n\t Read/write ratio (0.0-1.0)\n\t Default: 0.2\n\n\t-workers COUNT\n\t Number of concurrent workers\n\t Default: 1\n\nExamples:\n\n\t# Generate constant load for 10 minutes\n\tlitestream-test load -db /tmp/test.db -write-rate 100 -duration 10m\n\n\t# Generate burst pattern load\n\tlitestream-test load -db /tmp/test.db -pattern burst -duration 1h\n\n\t# Heavy write load with multiple workers\n\tlitestream-test load -db /tmp/test.db -write-rate 1000 -workers 4 -read-ratio 0.1\n`[1:])\n}\n" }, { "path": "cmd/litestream-test/main.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"runtime\"\n\t\"strings\"\n)\n\nvar (\n\tVersion = \"development\"\n\tCommit = \"\"\n)\n\nfunc main() {\n\tm := NewMain()\n\tif err := m.Run(context.Background(), os.Args[1:]); err != nil {\n\t\tfmt.Fprintln(os.Stderr, err)\n\t\tos.Exit(1)\n\t}\n}\n\ntype Main struct {\n\tStdin io.Reader\n\tStdout io.Writer\n\tStderr io.Writer\n}\n\nfunc NewMain() *Main {\n\treturn &Main{\n\t\tStdin: os.Stdin,\n\t\tStdout: os.Stdout,\n\t\tStderr: os.Stderr,\n\t}\n}\n\nfunc (m *Main) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-test\", flag.ExitOnError)\n\tfs.Usage = m.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif fs.NArg() == 0 || fs.Arg(0) == \"help\" {\n\t\tm.Usage()\n\t\treturn nil\n\t}\n\n\tswitch fs.Arg(0) {\n\tcase \"populate\":\n\t\treturn (&PopulateCommand{Main: m}).Run(ctx, fs.Args()[1:])\n\tcase \"load\":\n\t\treturn (&LoadCommand{Main: m}).Run(ctx, fs.Args()[1:])\n\tcase \"shrink\":\n\t\treturn (&ShrinkCommand{Main: m}).Run(ctx, fs.Args()[1:])\n\tcase \"validate\":\n\t\treturn (&ValidateCommand{Main: m}).Run(ctx, fs.Args()[1:])\n\tcase \"version\":\n\t\treturn (&VersionCommand{Main: m}).Run(ctx, fs.Args()[1:])\n\tdefault:\n\t\treturn fmt.Errorf(\"unknown command: %s\", fs.Arg(0))\n\t}\n}\n\nfunc (m *Main) Usage() {\n\tfmt.Fprintln(m.Stdout, `\nlitestream-test is a testing harness for Litestream database replication.\n\nUsage:\n\n\tlitestream-test [arguments]\n\nCommands:\n\n\tpopulate Quickly populate a database to a target size\n\tload Generate continuous load on a database\n\tshrink Shrink a database by deleting data\n\tvalidate Validate replication integrity\n\tversion Show version information\n\nUse \"litestream-test -h\" for more information about a command.\n`[1:])\n}\n\ntype VersionCommand struct {\n\tMain *Main\n}\n\nfunc (c *VersionCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-test version\", flag.ExitOnError)\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tfmt.Fprintf(c.Main.Stdout, \"litestream-test %s\\n\", Version)\n\tif Commit != \"\" {\n\t\tfmt.Fprintf(c.Main.Stdout, \"commit: %s\\n\", Commit)\n\t}\n\tfmt.Fprintf(c.Main.Stdout, \"go: %s\\n\", strings.TrimPrefix(runtime.Version(), \"go\"))\n\treturn nil\n}\n\nfunc (c *VersionCommand) Usage() {\n\tfmt.Fprintln(c.Main.Stdout, `\nShow version information for litestream-test.\n\nUsage:\n\n\tlitestream-test version\n`[1:])\n}\n\nfunc init() {\n\tlogger := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{\n\t\tLevel: slog.LevelInfo,\n\t}))\n\tslog.SetDefault(logger)\n}\n" }, { "path": "cmd/litestream-test/populate.go", "content": "package main\n\nimport (\n\t\"context\"\n\tcryptorand \"crypto/rand\"\n\t\"database/sql\"\n\t\"flag\"\n\t\"fmt\"\n\t\"log/slog\"\n\t\"math/rand\"\n\t\"os\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\ntype PopulateCommand struct {\n\tMain *Main\n\n\tDB string\n\tTargetSize string\n\tRowSize int\n\tBatchSize int\n\tTableCount int\n\tIndexRatio float64\n\tPageSize int\n}\n\nfunc (c *PopulateCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-test populate\", flag.ExitOnError)\n\tfs.StringVar(&c.DB, \"db\", \"\", \"Database path (required)\")\n\tfs.StringVar(&c.TargetSize, \"target-size\", \"100MB\", \"Target database size (e.g., 1GB, 500MB)\")\n\tfs.IntVar(&c.RowSize, \"row-size\", 1024, \"Average row size in bytes\")\n\tfs.IntVar(&c.BatchSize, \"batch-size\", 1000, \"Rows per transaction\")\n\tfs.IntVar(&c.TableCount, \"table-count\", 1, \"Number of tables to create\")\n\tfs.Float64Var(&c.IndexRatio, \"index-ratio\", 0.2, \"Percentage of columns to index (0.0-1.0)\")\n\tfs.IntVar(&c.PageSize, \"page-size\", 4096, \"SQLite page size in bytes\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif c.DB == \"\" {\n\t\treturn fmt.Errorf(\"database path required\")\n\t}\n\n\ttargetBytes, err := parseSize(c.TargetSize)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"invalid target size: %w\", err)\n\t}\n\n\tslog.Info(\"Starting database population\",\n\t\t\"db\", c.DB,\n\t\t\"target_size\", c.TargetSize,\n\t\t\"row_size\", c.RowSize,\n\t\t\"batch_size\", c.BatchSize,\n\t\t\"table_count\", c.TableCount,\n\t\t\"page_size\", c.PageSize,\n\t)\n\n\tif err := c.populateDatabase(ctx, targetBytes); err != nil {\n\t\treturn fmt.Errorf(\"populate database: %w\", err)\n\t}\n\n\tslog.Info(\"Database population complete\", \"db\", c.DB)\n\treturn nil\n}\n\nfunc (c *PopulateCommand) populateDatabase(ctx context.Context, targetBytes int64) error {\n\tif err := os.Remove(c.DB); err != nil && !os.IsNotExist(err) {\n\t\tslog.Warn(\"Could not remove existing database\", \"error\", err)\n\t}\n\n\tdb, err := sql.Open(\"sqlite3\", c.DB)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open database: %w\", err)\n\t}\n\tdefer db.Close()\n\n\tif _, err := db.Exec(fmt.Sprintf(\"PRAGMA page_size = %d\", c.PageSize)); err != nil {\n\t\treturn fmt.Errorf(\"set page size: %w\", err)\n\t}\n\n\tif _, err := db.Exec(\"PRAGMA journal_mode = WAL\"); err != nil {\n\t\treturn fmt.Errorf(\"set journal mode: %w\", err)\n\t}\n\n\tif _, err := db.Exec(\"PRAGMA synchronous = NORMAL\"); err != nil {\n\t\treturn fmt.Errorf(\"set synchronous: %w\", err)\n\t}\n\n\tfor i := 0; i < c.TableCount; i++ {\n\t\ttableName := fmt.Sprintf(\"test_table_%d\", i)\n\n\t\tcreateSQL := fmt.Sprintf(`\n\t\t\tCREATE TABLE %s (\n\t\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\t\tdata BLOB,\n\t\t\t\ttext_field TEXT,\n\t\t\t\tint_field INTEGER,\n\t\t\t\tfloat_field REAL,\n\t\t\t\ttimestamp INTEGER\n\t\t\t)\n\t\t`, tableName)\n\n\t\tif _, err := db.Exec(createSQL); err != nil {\n\t\t\treturn fmt.Errorf(\"create table %s: %w\", tableName, err)\n\t\t}\n\n\t\tif c.IndexRatio > 0 {\n\t\t\tif rand.Float64() < c.IndexRatio {\n\t\t\t\tindexSQL := fmt.Sprintf(\"CREATE INDEX idx_%s_timestamp ON %s(timestamp)\", tableName, tableName)\n\t\t\t\tif _, err := db.Exec(indexSQL); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"create index: %w\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif rand.Float64() < c.IndexRatio {\n\t\t\t\tindexSQL := fmt.Sprintf(\"CREATE INDEX idx_%s_int ON %s(int_field)\", tableName, tableName)\n\t\t\t\tif _, err := db.Exec(indexSQL); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"create index: %w\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\ttotalRows := int(targetBytes / int64(c.RowSize))\n\trowsPerTable := totalRows / c.TableCount\n\tif rowsPerTable == 0 {\n\t\trowsPerTable = 1\n\t}\n\n\tslog.Info(\"Populating database\",\n\t\t\"target_bytes\", targetBytes,\n\t\t\"total_rows\", totalRows,\n\t\t\"rows_per_table\", rowsPerTable,\n\t)\n\n\tstartTime := time.Now()\n\tfor tableIdx := 0; tableIdx < c.TableCount; tableIdx++ {\n\t\ttableName := fmt.Sprintf(\"test_table_%d\", tableIdx)\n\n\t\tif err := c.populateTable(ctx, db, tableName, rowsPerTable); err != nil {\n\t\t\treturn fmt.Errorf(\"populate table %s: %w\", tableName, err)\n\t\t}\n\n\t\tcurrentSize, _ := getDatabaseSize(c.DB)\n\t\tprogress := float64(currentSize) / float64(targetBytes) * 100\n\t\tslog.Info(\"Progress\",\n\t\t\t\"table\", tableName,\n\t\t\t\"current_size_mb\", currentSize/1024/1024,\n\t\t\t\"progress_percent\", fmt.Sprintf(\"%.1f\", progress),\n\t\t)\n\n\t\tif currentSize >= targetBytes {\n\t\t\tbreak\n\t\t}\n\t}\n\n\tduration := time.Since(startTime)\n\tfinalSize, _ := getDatabaseSize(c.DB)\n\n\tslog.Info(\"Population complete\",\n\t\t\"duration\", duration,\n\t\t\"final_size_mb\", finalSize/1024/1024,\n\t\t\"throughput_mb_per_sec\", fmt.Sprintf(\"%.2f\", float64(finalSize)/1024/1024/duration.Seconds()),\n\t)\n\n\treturn nil\n}\n\nfunc (c *PopulateCommand) populateTable(ctx context.Context, db *sql.DB, tableName string, rowCount int) error {\n\tdata := make([]byte, c.RowSize)\n\n\tfor i := 0; i < rowCount; i += c.BatchSize {\n\t\ttx, err := db.Begin()\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"begin transaction: %w\", err)\n\t\t}\n\n\t\tstmt, err := tx.Prepare(fmt.Sprintf(`\n\t\t\tINSERT INTO %s (data, text_field, int_field, float_field, timestamp)\n\t\t\tVALUES (?, ?, ?, ?, ?)\n\t\t`, tableName))\n\t\tif err != nil {\n\t\t\ttx.Rollback()\n\t\t\treturn fmt.Errorf(\"prepare statement: %w\", err)\n\t\t}\n\n\t\tbatchEnd := i + c.BatchSize\n\t\tif batchEnd > rowCount {\n\t\t\tbatchEnd = rowCount\n\t\t}\n\n\t\tfor j := i; j < batchEnd; j++ {\n\t\t\tcryptorand.Read(data)\n\t\t\ttextField := fmt.Sprintf(\"row_%d_%d\", i, j)\n\t\t\tintField := rand.Int63()\n\t\t\tfloatField := rand.Float64() * 1000\n\t\t\ttimestamp := time.Now().Unix()\n\n\t\t\tif _, err := stmt.Exec(data, textField, intField, floatField, timestamp); err != nil {\n\t\t\t\tstmt.Close()\n\t\t\t\ttx.Rollback()\n\t\t\t\treturn fmt.Errorf(\"insert row: %w\", err)\n\t\t\t}\n\t\t}\n\n\t\tstmt.Close()\n\t\tif err := tx.Commit(); err != nil {\n\t\t\treturn fmt.Errorf(\"commit transaction: %w\", err)\n\t\t}\n\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn ctx.Err()\n\t\tdefault:\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (c *PopulateCommand) Usage() {\n\tfmt.Fprintln(c.Main.Stdout, `\nPopulate a SQLite database to a target size for testing.\n\nUsage:\n\n\tlitestream-test populate [options]\n\nOptions:\n\n\t-db PATH\n\t Database path (required)\n\n\t-target-size SIZE\n\t Target database size (e.g., \"1GB\", \"500MB\")\n\t Default: 100MB\n\n\t-row-size SIZE\n\t Average row size in bytes\n\t Default: 1024\n\n\t-batch-size COUNT\n\t Number of rows per transaction\n\t Default: 1000\n\n\t-table-count COUNT\n\t Number of tables to create\n\t Default: 1\n\n\t-index-ratio RATIO\n\t Percentage of columns to index (0.0-1.0)\n\t Default: 0.2\n\n\t-page-size SIZE\n\t SQLite page size in bytes\n\t Default: 4096\n\nExamples:\n\n\t# Create a 1GB database with default settings\n\tlitestream-test populate -db /tmp/test.db -target-size 1GB\n\n\t# Create a 2GB database with larger rows\n\tlitestream-test populate -db /tmp/test.db -target-size 2GB -row-size 4096\n\n\t# Test lock page with different page sizes\n\tlitestream-test populate -db /tmp/test.db -target-size 1.5GB -page-size 8192\n`[1:])\n}\n\nfunc parseSize(s string) (int64, error) {\n\t// Check suffixes in order from longest to shortest to avoid \"B\" matching before \"MB\"\n\tsuffixes := []struct {\n\t\tsuffix string\n\t\tmultiplier int64\n\t}{\n\t\t{\"TB\", 1024 * 1024 * 1024 * 1024},\n\t\t{\"GB\", 1024 * 1024 * 1024},\n\t\t{\"MB\", 1024 * 1024},\n\t\t{\"KB\", 1024},\n\t\t{\"B\", 1},\n\t}\n\n\tfor _, sf := range suffixes {\n\t\tif len(s) > len(sf.suffix) && s[len(s)-len(sf.suffix):] == sf.suffix {\n\t\t\tvar value float64\n\t\t\tif _, err := fmt.Sscanf(s[:len(s)-len(sf.suffix)], \"%f\", &value); err != nil {\n\t\t\t\treturn 0, err\n\t\t\t}\n\t\t\treturn int64(value * float64(sf.multiplier)), nil\n\t\t}\n\t}\n\n\tvar value int64\n\tif _, err := fmt.Sscanf(s, \"%d\", &value); err != nil {\n\t\treturn 0, err\n\t}\n\treturn value, nil\n}\n\nfunc getDatabaseSize(path string) (int64, error) {\n\tinfo, err := os.Stat(path)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tsize := info.Size()\n\n\twalPath := path + \"-wal\"\n\tif walInfo, err := os.Stat(walPath); err == nil {\n\t\tsize += walInfo.Size()\n\t}\n\n\tshmPath := path + \"-shm\"\n\tif shmInfo, err := os.Stat(shmPath); err == nil {\n\t\tsize += shmInfo.Size()\n\t}\n\n\treturn size, nil\n}\n" }, { "path": "cmd/litestream-test/scripts/README.md", "content": "# Litestream Test Scripts\n\nComprehensive test scripts for validating Litestream functionality across various scenarios. These scripts use the `litestream-test` CLI tool to orchestrate complex testing scenarios.\n\n## Prerequisites\n\n```bash\ngo build -o bin/litestream ./cmd/litestream\ngo build -o bin/litestream-test ./cmd/litestream-test\n\n./cmd/litestream-test/scripts/verify-test-setup.sh\n```\n\n## Quick Reference\n\n> **Note:** Some tests have been migrated to Go integration tests in `tests/integration/`. See [tests/integration/README.md](../../tests/integration/README.md) for the Go-based test suite.\n\n| Script | Purpose | Duration | Status |\n|--------|---------|----------|--------|\n| verify-test-setup.sh | Environment validation | ~5s | ✅ Stable |\n| reproduce-critical-bug.sh | Checkpoint during downtime bug | ~2min | 🐛 Reproduces #752 |\n| test-754-s3-scenarios.sh | Issue #754 S3 vs file replication | ~10min | 🐛 Tests #754 |\n| test-754-restore-focus.sh | Issue #754 restore focus | ~5min | 🐛 Tests #754 |\n| test-simple-754-reproduction.sh | Minimal #754 reproduction | ~3min | 🐛 Tests #754 |\n| test-v0.5-flag-reproduction.sh | ltx v0.5.0 flag issue | ~5min | 🐛 Tests #754 |\n| test-v0.5-restart-scenarios.sh | v0.5 restart scenarios | ~5min | 🐛 Tests #754 |\n| test-format-isolation.sh | Format version isolation | ~3min | ✅ Stable |\n| test-quick-format-check.sh | Quick format validation | ~30s | ✅ Stable |\n| test-upgrade-v0.3-to-v0.5.sh | v0.3 to v0.5 upgrade | ~10min | ✅ Stable |\n| test-upgrade-large-db.sh | Large database upgrade | ~15min | ✅ Stable |\n| test-massive-upgrade.sh | Massive database upgrade | ~20min | ✅ Stable |\n| test-s3-retention-cleanup.sh | Basic S3 retention | ~8min | ✅ Stable |\n| test-s3-retention-small-db.sh | S3 retention 50MB | ~8min | ✅ Stable |\n| test-s3-retention-large-db.sh | S3 retention 1.5GB | ~20min | ✅ Stable |\n| test-s3-retention-comprehensive.sh | Full S3 retention suite | ~30min | ✅ Stable |\n| test-s3-access-point.sh | S3 Access Point ARN support | ~2min | ✅ Stable |\n\n## Test Categories\n\n### Setup & Validation\n\n#### verify-test-setup.sh\nVerifies that the test environment is properly configured with required binaries and dependencies.\n\n```bash\n./cmd/litestream-test/scripts/verify-test-setup.sh\n```\n\n**Checks:**\n- Litestream binary exists\n- litestream-test binary exists\n- SQLite3 available\n- Python dependencies for S3 mock\n\n### Bug Reproduction Scripts\n\n#### reproduce-critical-bug.sh\nReproduces checkpoint during downtime bug that causes restore failures.\n\n```bash\n./cmd/litestream-test/scripts/reproduce-critical-bug.sh\n```\n\n**Reproduces:** Issue #752\n\n**Scenario:**\n1. Litestream is killed (simulating crash)\n2. Writes continue and a checkpoint occurs\n3. Litestream is restarted\n4. Restore fails with \"nonsequential page numbers\" error\n\n**Expected:** Database should restore successfully\n**Actual:** Restore fails, causing data loss\n\n#### test-754-s3-scenarios.sh\nTests Issue #754 flag compatibility with S3 replication versus file replication.\n\n```bash\n./cmd/litestream-test/scripts/test-754-s3-scenarios.sh\n```\n\n**Tests:**\n- S3 replica behavior with ltx v0.5.0\n- File replica behavior comparison\n- LTX file cleanup and retention\n- Flag compatibility issues\n\n#### test-754-restore-focus.sh\nFocused testing of Issue #754 restore failures.\n\n```bash\n./cmd/litestream-test/scripts/test-754-restore-focus.sh\n```\n\n**Tests:**\n- Restore failures with pre-existing databases\n- Flag mismatch detection\n- Recovery scenarios\n\n#### test-simple-754-reproduction.sh\nMinimal reproduction case for Issue #754.\n\n```bash\n./cmd/litestream-test/scripts/test-simple-754-reproduction.sh\n```\n\n**Reproduces:** Issue #754 with minimal steps for debugging\n\n#### test-v0.5-flag-reproduction.sh\nReproduces ltx v0.5.0 flag compatibility issue.\n\n```bash\n./cmd/litestream-test/scripts/test-v0.5-flag-reproduction.sh\n```\n\n**Tests:**\n- Pre-existing database behavior with ltx v0.5.0\n- Flag mismatch scenarios\n- Upgrade path issues\n\n#### test-v0.5-restart-scenarios.sh\nTests various restart scenarios with ltx v0.5.0.\n\n```bash\n./cmd/litestream-test/scripts/test-v0.5-restart-scenarios.sh\n```\n\n**Tests:**\n- Clean restart\n- Restart after checkpoint\n- Restart with pending data\n- Flag persistence across restarts\n\n### Format & Upgrade Tests\n\n#### test-format-isolation.sh\nTests isolation between different LTX format versions.\n\n```bash\n./cmd/litestream-test/scripts/test-format-isolation.sh\n```\n\n**Tests:**\n- Multiple format versions coexisting\n- Format detection and handling\n- Migration between formats\n- Backward compatibility\n\n#### test-quick-format-check.sh\nQuick validation of LTX format handling.\n\n```bash\n./cmd/litestream-test/scripts/test-quick-format-check.sh\n```\n\n**Duration:** ~30 seconds\n\n**Tests:**\n- Format version detection\n- Basic format integrity\n- Quick validation workflow\n\n#### test-upgrade-v0.3-to-v0.5.sh\nTests upgrade path from ltx v0.3 to v0.5.\n\n```bash\n./cmd/litestream-test/scripts/test-upgrade-v0.3-to-v0.5.sh\n```\n\n**Tests:**\n- Migration from v0.3 to v0.5\n- Data preservation during upgrade\n- Flag handling in upgrade process\n- Backward compatibility verification\n\n#### test-upgrade-large-db.sh\nTests upgrade process with large databases (1GB+).\n\n```bash\n./cmd/litestream-test/scripts/test-upgrade-large-db.sh\n```\n\n**Tests:**\n- Large database upgrade performance\n- Data integrity during upgrade\n- Lock page handling in upgrades\n- Resource usage during migration\n\n#### test-massive-upgrade.sh\nTests upgrade with very large databases and long-running scenarios.\n\n```bash\n./cmd/litestream-test/scripts/test-massive-upgrade.sh\n```\n\n**Tests:**\n- Multi-GB database upgrades\n- Extended migration scenarios\n- Performance under scale\n- Memory and disk usage\n\n### S3 & Retention Tests\n\nFor detailed S3 retention testing documentation, see [S3-RETENTION-TESTING.md](../S3-RETENTION-TESTING.md).\n\n#### test-s3-access-point.sh\nTests S3 Access Point ARN support (Issue #923). Verifies that Access Point ARNs work automatically without manual endpoint configuration.\n\n```bash\nexport LITESTREAM_S3_ACCESS_POINT_ARN='arn:aws:s3:us-east-2:123456789012:accesspoint/my-access-point'\n./cmd/litestream-test/scripts/test-s3-access-point.sh\n```\n\n**Tests:**\n- Replication to S3 Access Point using ARN\n- Automatic endpoint resolution (UseARNRegion)\n- Restore from Access Point ARN\n- Data integrity verification\n\n**Environment Variables:**\n- `LITESTREAM_S3_ACCESS_POINT_ARN` - Full ARN of the S3 Access Point (required)\n- `LITESTREAM_S3_REGION` - AWS region (optional, extracted from ARN)\n- `LITESTREAM_S3_PREFIX` - Path prefix in bucket (optional)\n- AWS credentials via standard methods (env vars, credentials file, IAM role)\n\n#### test-s3-retention-cleanup.sh\nBasic S3 LTX retention cleanup testing.\n\n```bash\n./cmd/litestream-test/scripts/test-s3-retention-cleanup.sh\n```\n\n**Tests:**\n- Basic retention cleanup behavior\n- Old LTX file removal\n- Retention period enforcement\n\n#### test-s3-retention-small-db.sh\nS3 retention testing with 50MB database.\n\n```bash\n./cmd/litestream-test/scripts/test-s3-retention-small-db.sh\n```\n\n**Configuration:**\n- Database size: 50MB\n- Retention period: 2 minutes\n- Duration: ~8 minutes\n\n**Tests:**\n- Small database retention cleanup\n- Quick retention cycles\n- S3 mock integration\n\n#### test-s3-retention-large-db.sh\nS3 retention testing with 1.5GB database crossing lock page boundary.\n\n```bash\n./cmd/litestream-test/scripts/test-s3-retention-large-db.sh\n```\n\n**Configuration:**\n- Database size: 1.5GB\n- Page size: 4KB (lock page at #262145)\n- Retention period: 3 minutes\n- Duration: ~20 minutes\n\n**Tests:**\n- Large database retention cleanup\n- Lock page boundary handling\n- Extended monitoring\n- Scale behavior\n\n#### test-s3-retention-comprehensive.sh\nMaster script running both small and large database retention tests with analysis.\n\n```bash\n./cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh\n\n./cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh --small-only\n./cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh --large-only\n./cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh --no-cleanup\n```\n\n**Duration:** ~30 minutes for full suite\n\n**Features:**\n- Runs both small and large DB tests\n- Comparative analysis\n- Detailed reports\n- Configurable execution\n\n## Usage Patterns\n\n### Running Individual Tests\n\n```bash\n./cmd/litestream-test/scripts/test-fresh-start.sh\n```\n\n### Verify Environment First\n\n```bash\n./cmd/litestream-test/scripts/verify-test-setup.sh\n./cmd/litestream-test/scripts/test-rapid-checkpoints.sh\n```\n\n### Running Multiple Tests\n\n```bash\nfor script in test-fresh-start.sh test-rapid-checkpoints.sh test-database-integrity.sh; do\n echo \"Running $script...\"\n ./cmd/litestream-test/scripts/$script\n echo \"\"\ndone\n```\n\n### S3 Testing with Local Mock\n\nThe S3 tests automatically use the Python S3 mock server (`./etc/s3_mock.py`) for isolated testing:\n\n```bash\n./cmd/litestream-test/scripts/test-s3-retention-small-db.sh\n```\n\n### Debugging Failed Tests\n\nMost tests create logs in `/tmp/`:\n\n```bash\ntail -f /tmp/fresh-test.log\n\ntail -f /tmp/checkpoint-cycle.log\n\ngrep -i error /tmp/*.log\n```\n\n## Test Artifacts\n\nTests typically create artifacts in `/tmp/`:\n\n- **Databases:** `/tmp/*-test.db`\n- **Replicas:** `/tmp/*-replica/`\n- **Logs:** `/tmp/*-test.log`\n- **Configs:** `/tmp/*.yml`\n- **Restored DBs:** `/tmp/*-restored.db`\n\n## Test Results & Analysis\n\nHistorical test results and analysis are stored in `.local/test-results/` (git-ignored):\n\n- `final-test-summary.md` - Comprehensive test findings\n- `validation-results-after-ltx-v0.5.0.md` - ltx v0.5.0 impact analysis\n- `comprehensive-test-findings.md` - Initial test results\n- `critical-bug-analysis.md` - Detailed bug analysis\n\n## Key Findings Summary\n\n### Performance ✅\n- Successfully handles 400+ writes/second\n- Manages 100MB+ WAL files\n- Multiple concurrent databases replicate cleanly\n\n### Fresh Databases ✅\n- Work perfectly with ltx v0.5.0\n- Clean replication and restore\n- No flag issues\n\n### Pre-existing Databases ❌\n- Broken due to ltx flag compatibility (#754)\n- Restore failures\n- Upgrade path issues\n\n### Checkpoint During Downtime ❌\n- Worse with ltx v0.5.0 (#752)\n- Causes restore failures\n- Data loss risk\n\n### S3 Retention ✅\n- LTX cleanup works correctly\n- Handles lock page boundary\n- Scale testing successful\n\n## Related Issues\n\n- [#752](https://github.com/benbjohnson/litestream/issues/752) - Checkpoint during downtime bug\n- [#753](https://github.com/benbjohnson/litestream/issues/753) - Transaction numbering (FIXED)\n- [#754](https://github.com/benbjohnson/litestream/issues/754) - ltx v0.5.0 flag compatibility (CRITICAL)\n\n## Related Documentation\n\n- [litestream-test CLI Documentation](../README.md) - CLI tool reference\n- [S3 Retention Testing Guide](../S3-RETENTION-TESTING.md) - Detailed S3 testing\n- [Top-level Integration Scripts](../../../scripts/README.md) - Long-running tests\n\n## Contributing\n\nWhen adding new test scripts:\n\n1. Follow existing naming conventions (`test-*.sh`)\n2. Include clear comments explaining what is being tested\n3. Use `/tmp/` for test artifacts\n4. Create cleanup handlers with `trap`\n5. Provide clear success/failure output\n6. Update this README with script documentation\n7. Add entry to Quick Reference table\n" }, { "path": "cmd/litestream-test/scripts/reproduce-critical-bug.sh", "content": "#!/bin/bash\n\n# Litestream v0.5.0 Critical Bug Reproduction Script\n#\n# This script demonstrates a CRITICAL data loss bug where restore fails\n# after Litestream is interrupted and a checkpoint occurs during downtime.\n#\n# Requirements:\n# - litestream binary (built from current main branch)\n# - litestream-test binary (from PR #748 or build with: go build -o bin/litestream-test ./cmd/litestream-test)\n# - SQLite3 command line tool\n#\n# Expected behavior: Database should restore successfully\n# Actual behavior: Restore fails with \"nonsequential page numbers\" error\n\nset -e\n\necho \"============================================\"\necho \"Litestream v0.5.0 Critical Bug Reproduction\"\necho \"============================================\"\necho \"\"\necho \"This demonstrates a data loss scenario where restore fails after:\"\necho \"1. Litestream is killed (simulating crash)\"\necho \"2. Writes continue and a checkpoint occurs\"\necho \"3. Litestream is restarted\"\necho \"\"\n\n# Configuration\nDB=\"/tmp/critical-bug-test.db\"\nREPLICA=\"/tmp/critical-bug-replica\"\n\n# Clean up any previous test\necho \"[SETUP] Cleaning up previous test files...\"\nrm -f \"$DB\"*\nrm -rf \"$REPLICA\"\n\n# ALWAYS use local build for testing\nLITESTREAM=\"./bin/litestream\"\nif [ ! -f \"$LITESTREAM\" ]; then\n echo \"ERROR: Local litestream build not found at $LITESTREAM\"\n echo \"Please build first: go build -o bin/litestream ./cmd/litestream\"\n exit 1\nfi\necho \"Using local build: $LITESTREAM\"\n\n# Check for litestream-test binary\nif [ -f \"./bin/litestream-test\" ]; then\n LITESTREAM_TEST=\"./bin/litestream-test\"\n echo \"Using local litestream-test: $LITESTREAM_TEST\"\nelse\n echo \"ERROR: litestream-test not found. Please build with:\"\n echo \" go build -o bin/litestream-test ./cmd/litestream-test\"\n echo \"\"\n echo \"Or get it from: https://github.com/benbjohnson/litestream/pull/748\"\n exit 1\nfi\n\n# Show versions\necho \"Versions:\"\n$LITESTREAM version\necho \"\"\n\n# Step 1: Create and populate initial database\necho \"\"\necho \"[STEP 1] Creating test database (50MB)...\"\n$LITESTREAM_TEST populate -db \"$DB\" -target-size 50MB -table-count 2\nINITIAL_SIZE=$(ls -lh \"$DB\" 2>/dev/null | awk '{print $5}')\necho \"✓ Database created: $INITIAL_SIZE\"\n\n# Step 2: Start Litestream replication\necho \"\"\necho \"[STEP 2] Starting Litestream replication...\"\n./bin/litestream replicate \"$DB\" \"file://$REPLICA\" > /tmp/litestream.log 2>&1 &\nLITESTREAM_PID=$!\nsleep 3\n\nif ! kill -0 $LITESTREAM_PID 2>/dev/null; then\n echo \"ERROR: Litestream failed to start. Check /tmp/litestream.log\"\n cat /tmp/litestream.log\n exit 1\nfi\necho \"✓ Litestream running (PID: $LITESTREAM_PID)\"\n\n# Step 3: Start continuous writes\necho \"\"\necho \"[STEP 3] Starting continuous writes...\"\n./bin/litestream-test load -db \"$DB\" -write-rate 100 -duration 2m -pattern constant > /tmp/writes.log 2>&1 &\nWRITE_PID=$!\necho \"✓ Write load started (PID: $WRITE_PID)\"\n\n# Step 4: Let it run normally for 20 seconds\necho \"\"\necho \"[STEP 4] Running normally for 20 seconds...\"\nsleep 20\n\n# Get row count before interruption\nROWS_BEFORE=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM load_test;\" 2>/dev/null || echo \"0\")\necho \"✓ Rows written before interruption: $ROWS_BEFORE\"\n\n# Step 5: Kill Litestream (simulate crash)\necho \"\"\necho \"[STEP 5] Killing Litestream (simulating crash)...\"\nkill -9 $LITESTREAM_PID 2>/dev/null || true\necho \"✓ Litestream killed\"\n\n# Step 6: Let writes continue for 15 seconds without Litestream\necho \"\"\necho \"[STEP 6] Continuing writes for 15 seconds (Litestream is down)...\"\nsleep 15\n\n# Step 7: Execute non-PASSIVE checkpoint\necho \"\"\necho \"[STEP 7] Executing FULL checkpoint while Litestream is down...\"\nCHECKPOINT_RESULT=$(sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\" 2>&1)\necho \"✓ Checkpoint result: $CHECKPOINT_RESULT\"\n\nROWS_AFTER_CHECKPOINT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM load_test;\")\necho \"✓ Rows after checkpoint: $ROWS_AFTER_CHECKPOINT\"\n\n# Step 8: Resume Litestream\necho \"\"\necho \"[STEP 8] Resuming Litestream...\"\n./bin/litestream replicate \"$DB\" \"file://$REPLICA\" >> /tmp/litestream.log 2>&1 &\nNEW_LITESTREAM_PID=$!\nsleep 3\n\nif ! kill -0 $NEW_LITESTREAM_PID 2>/dev/null; then\n echo \"WARNING: Litestream failed to restart\"\nfi\necho \"✓ Litestream restarted (PID: $NEW_LITESTREAM_PID)\"\n\n# Step 9: Let Litestream catch up\necho \"\"\necho \"[STEP 9] Letting Litestream catch up for 20 seconds...\"\nsleep 20\n\n# Stop writes\nkill $WRITE_PID 2>/dev/null || true\necho \"✓ Writes stopped\"\n\n# Wait for final sync\nsleep 5\n\n# Get final row count\nFINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM load_test;\")\necho \"✓ Final row count in source database: $FINAL_COUNT\"\n\n# Kill Litestream\nkill $NEW_LITESTREAM_PID 2>/dev/null || true\n\n# Step 10: Attempt to restore (THIS IS WHERE THE BUG OCCURS)\necho \"\"\necho \"[STEP 10] Attempting to restore database...\"\necho \"==========================================\"\necho \"\"\n\nrm -f /tmp/restored.db\nif ./bin/litestream restore -o /tmp/restored.db \"file://$REPLICA\" 2>&1 | tee /tmp/restore-output.log; then\n echo \"\"\n echo \"✓ SUCCESS: Restore completed successfully\"\n\n # Verify restored database\n RESTORED_COUNT=$(sqlite3 /tmp/restored.db \"SELECT COUNT(*) FROM load_test;\" 2>/dev/null || echo \"0\")\n INTEGRITY=$(sqlite3 /tmp/restored.db \"PRAGMA integrity_check;\" 2>/dev/null || echo \"FAILED\")\n\n echo \" - Restored row count: $RESTORED_COUNT\"\n echo \" - Integrity check: $INTEGRITY\"\n\n if [ \"$RESTORED_COUNT\" -eq \"$FINAL_COUNT\" ]; then\n echo \" - Data integrity: ✓ VERIFIED (no data loss)\"\n else\n LOSS=$((FINAL_COUNT - RESTORED_COUNT))\n echo \" - Data integrity: ✗ FAILED (lost $LOSS rows)\"\n fi\nelse\n echo \"\"\n echo \"✗ CRITICAL BUG REPRODUCED: Restore failed!\"\n echo \"\"\n echo \"Error output:\"\n echo \"-------------\"\n cat /tmp/restore-output.log\n echo \"\"\n echo \"This is the critical bug. The database cannot be restored after\"\n echo \"Litestream was interrupted and a checkpoint occurred during downtime.\"\n echo \"\"\n echo \"Original database stats:\"\n echo \" - Rows before interruption: $ROWS_BEFORE\"\n echo \" - Rows after checkpoint: $ROWS_AFTER_CHECKPOINT\"\n echo \" - Final rows: $FINAL_COUNT\"\n echo \" - DATA IS UNRECOVERABLE\"\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Test artifacts saved in:\"\necho \" - Source database: $DB\"\necho \" - Replica files: $REPLICA/\"\necho \" - Litestream log: /tmp/litestream.log\"\necho \" - Restore output: /tmp/restore-output.log\"\necho \"\"\n\n# Clean up processes\npkill -f litestream-test 2>/dev/null || true\npkill -f \"litestream replicate\" 2>/dev/null || true\n\necho \"Test complete.\"\n" }, { "path": "cmd/litestream-test/scripts/test-754-restore-focus.sh", "content": "#!/bin/bash\nset -e\n\n# Aggressive #754 reproduction - focus on RESTORE scenarios\n# The \"ltx verification failed\" error most likely happens during restore\n\necho \"========================================\"\necho \"Aggressive #754 Restore Focus Test\"\necho \"========================================\"\necho \"\"\necho \"Testing restore scenarios to trigger 'ltx verification failed'\"\necho \"\"\n\nDB=\"/tmp/restore754.db\"\nREPLICA=\"/tmp/restore754-replica\"\nRESTORED=\"/tmp/restore754-restored.db\"\nLITESTREAM=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\n\n# Cleanup\ncleanup() {\n pkill -f \"litestream replicate.*restore754.db\" 2>/dev/null || true\n rm -rf \"$DB\"* \"$REPLICA\" \"$RESTORED\"* /tmp/restore754-*.log\n}\n\ntrap cleanup EXIT\ncleanup\n\necho \"==========================================\"\necho \"Test 1: Large database with many restores\"\necho \"==========================================\"\n\necho \"[1] Creating large database (2GB+)...\"\n$LITESTREAM_TEST populate -db \"$DB\" -target-size 2GB >/dev/null 2>&1\n\n# Add complex schema\nsqlite3 \"$DB\" < /tmp/restore754-replication.log 2>&1 &\nREPL_PID=$!\nsleep 10\n\nif ! kill -0 $REPL_PID 2>/dev/null; then\n echo \" ✗ Replication failed\"\n cat /tmp/restore754-replication.log\n exit 1\nfi\n\n# Generate more LTX files with checkpoints\necho \" Adding data and forcing multiple LTX files...\"\nfor round in {2..6}; do\n for i in {1..10}; do\n sqlite3 \"$DB\" \"INSERT INTO restore_test (test_round, data) VALUES ($round, randomblob(5000));\"\n done\n sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\"\n sleep 2\ndone\n\nLTX_COUNT=$(find \"$REPLICA\" -name \"*.ltx\" 2>/dev/null | wc -l)\necho \" ✓ Generated $LTX_COUNT LTX files with HeaderFlagNoChecksum\"\n\nkill $REPL_PID 2>/dev/null\nwait $REPL_PID 2>/dev/null\n\necho \"\"\necho \"[3] CRITICAL: Testing restore from HeaderFlagNoChecksum files...\"\n\nfor attempt in {1..5}; do\n echo \" Restore attempt $attempt...\"\n rm -f \"$RESTORED\"*\n\n $LITESTREAM restore -o \"$RESTORED\" \"file://$REPLICA\" > /tmp/restore754-attempt$attempt.log 2>&1\n RESTORE_EXIT=$?\n\n # Check for the specific #754 errors\n FLAGS_ERROR=$(grep -c \"no flags allowed\" /tmp/restore754-attempt$attempt.log 2>/dev/null || echo \"0\")\n VERIFY_ERROR=$(grep -c \"ltx verification failed\" /tmp/restore754-attempt$attempt.log 2>/dev/null || echo \"0\")\n\n echo \" Exit code: $RESTORE_EXIT\"\n echo \" 'no flags allowed': $FLAGS_ERROR\"\n echo \" 'ltx verification failed': $VERIFY_ERROR\"\n\n if [ \"$FLAGS_ERROR\" -gt \"0\" ] || [ \"$VERIFY_ERROR\" -gt \"0\" ]; then\n echo \" 🚨 #754 REPRODUCED ON RESTORE!\"\n echo \" Error details:\"\n grep -A2 -B2 \"no flags\\|ltx verification\" /tmp/restore754-attempt$attempt.log\n echo \"\"\n RESTORE_754_FOUND=true\n break\n elif [ $RESTORE_EXIT -ne 0 ]; then\n echo \" ⚠️ Restore failed with different error:\"\n head -3 /tmp/restore754-attempt$attempt.log\n else\n RESTORED_COUNT=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM restore_test;\" 2>/dev/null || echo \"0\")\n echo \" ✓ Restore succeeded: $RESTORED_COUNT rows\"\n fi\n echo \"\"\ndone\n\necho \"\"\necho \"==========================================\"\necho \"Test 2: Corrupt existing LTX file to force errors\"\necho \"==========================================\"\n\nif [ \"$LTX_COUNT\" -gt \"0\" ]; then\n echo \"[4] Deliberately corrupting an LTX file to test error handling...\"\n\n # Find first LTX file and modify it\n FIRST_LTX=$(find \"$REPLICA\" -name \"*.ltx\" | head -1)\n if [ -n \"$FIRST_LTX\" ]; then\n echo \" Corrupting: $(basename \"$FIRST_LTX\")\"\n # Modify the header to trigger flag verification\n echo \"CORRUPTED_HEADER\" > \"$FIRST_LTX\"\n\n echo \" Attempting restore from corrupted LTX...\"\n rm -f \"$RESTORED\"*\n\n $LITESTREAM restore -o \"$RESTORED\" \"file://$REPLICA\" > /tmp/restore754-corrupted.log 2>&1\n CORRUPT_EXIT=$?\n\n CORRUPT_FLAGS=$(grep -c \"no flags allowed\" /tmp/restore754-corrupted.log 2>/dev/null || echo \"0\")\n CORRUPT_VERIFY=$(grep -c \"ltx verification failed\" /tmp/restore754-corrupted.log 2>/dev/null || echo \"0\")\n\n echo \" Exit code: $CORRUPT_EXIT\"\n echo \" 'no flags allowed': $CORRUPT_FLAGS\"\n echo \" 'ltx verification failed': $CORRUPT_VERIFY\"\n\n if [ \"$CORRUPT_FLAGS\" -gt \"0\" ] || [ \"$CORRUPT_VERIFY\" -gt \"0\" ]; then\n echo \" 🚨 #754 TRIGGERED BY CORRUPTED LTX!\"\n grep -A2 -B2 \"no flags\\|ltx verification\" /tmp/restore754-corrupted.log\n CORRUPT_754_FOUND=true\n else\n echo \" Different error (as expected for corruption):\"\n head -3 /tmp/restore754-corrupted.log\n fi\n fi\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Test 3: Multiple database sizes\"\necho \"==========================================\"\n\nfor size in \"500MB\" \"1GB\" \"3GB\"; do\n echo \"[5.$size] Testing with $size database...\"\n\n cleanup\n\n # Create database of specific size\n $LITESTREAM_TEST populate -db \"$DB\" -target-size \"$size\" >/dev/null 2>&1\n sqlite3 \"$DB\" \"CREATE TABLE size_test (id INTEGER PRIMARY KEY, size TEXT, data BLOB); INSERT INTO size_test (size, data) VALUES ('$size', randomblob(8000));\"\n\n # Quick replication\n $LITESTREAM replicate \"$DB\" \"file://$REPLICA\" > /dev/null 2>&1 &\n REPL_PID=$!\n sleep 5\n\n # Add data and checkpoint\n sqlite3 \"$DB\" \"INSERT INTO size_test (size, data) VALUES ('$size-checkpoint', randomblob(10000));\"\n sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\"\n sleep 3\n\n kill $REPL_PID 2>/dev/null\n wait $REPL_PID 2>/dev/null\n\n # Test restore\n rm -f \"$RESTORED\"*\n $LITESTREAM restore -o \"$RESTORED\" \"file://$REPLICA\" > /tmp/restore754-$size.log 2>&1\n SIZE_EXIT=$?\n\n SIZE_FLAGS=$(grep -c \"no flags allowed\" /tmp/restore754-$size.log 2>/dev/null || echo \"0\")\n SIZE_VERIFY=$(grep -c \"ltx verification failed\" /tmp/restore754-$size.log 2>/dev/null || echo \"0\")\n\n echo \" $size result: exit=$SIZE_EXIT, flags=$SIZE_FLAGS, verify=$SIZE_VERIFY\"\n\n if [ \"$SIZE_FLAGS\" -gt \"0\" ] || [ \"$SIZE_VERIFY\" -gt \"0\" ]; then\n echo \" 🚨 #754 TRIGGERED WITH $size DATABASE!\"\n grep -A1 -B1 \"no flags\\|ltx verification\" /tmp/restore754-$size.log\n SIZE_754_FOUND=true\n fi\ndone\n\necho \"\"\necho \"==========================================\"\necho \"FINAL RESULTS\"\necho \"==========================================\"\necho \"\"\necho \"Test scenarios:\"\necho \" Large DB restore attempts: $([ \"${RESTORE_754_FOUND:-false}\" = \"true\" ] && echo \"REPRODUCED #754\" || echo \"No #754 errors\")\"\necho \" Corrupted LTX file: $([ \"${CORRUPT_754_FOUND:-false}\" = \"true\" ] && echo \"REPRODUCED #754\" || echo \"No #754 errors\")\"\necho \" Multiple sizes: $([ \"${SIZE_754_FOUND:-false}\" = \"true\" ] && echo \"REPRODUCED #754\" || echo \"No #754 errors\")\"\necho \"\"\n\nif [ \"${RESTORE_754_FOUND:-false}\" = \"true\" ] || [ \"${CORRUPT_754_FOUND:-false}\" = \"true\" ] || [ \"${SIZE_754_FOUND:-false}\" = \"true\" ]; then\n echo \"✅ SUCCESS: #754 REPRODUCED!\"\n echo \" Issue confirmed: HeaderFlagNoChecksum incompatible with ltx v0.5.0\"\n echo \" Trigger: Restore operations on LTX files with deprecated flags\"\n echo \"\"\n echo \" This proves issue #754 is real and needs fixing before v0.5.0 release\"\nelse\n echo \"❌ #754 NOT REPRODUCED\"\n echo \" Either:\"\n echo \" 1. Issue was fixed in recent changes\"\n echo \" 2. Requires very specific conditions not tested\"\n echo \" 3. Issue is in different code path (not restore)\"\n echo \"\"\n echo \" Need to investigate further or check if issue still exists\"\nfi\n\necho \"\"\necho \"HeaderFlagNoChecksum locations to fix:\"\necho \" - db.go:883\"\necho \" - db.go:1208\"\necho \" - db.go:1298\"\necho \" - replica.go:466\"\necho \"========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-754-s3-scenarios.sh", "content": "#!/bin/bash\nset -e\n\n# Test #754 flag issue with S3 scenarios and retention cleanup\n# Tests both S3 vs file replication behavior and LTX file cleanup\n\necho \"==========================================\"\necho \"#754 S3 Scenarios & Retention Test\"\necho \"==========================================\"\necho \"\"\necho \"Testing #754 flag issue with S3 replication vs file replication\"\necho \"Verifying LTX file cleanup after retention period\"\necho \"\"\n\n# Check if we have S3 environment setup\nif [ -z \"$AWS_ACCESS_KEY_ID\" ] && [ -z \"$LITESTREAM_S3_ACCESS_KEY_ID\" ]; then\n echo \"⚠️ No S3 credentials found. Setting up local S3-compatible test...\"\n echo \"\"\n\n # Create minimal S3-like configuration for testing\n export LITESTREAM_S3_ACCESS_KEY_ID=\"testkey\"\n export LITESTREAM_S3_SECRET_ACCESS_KEY=\"testsecret\"\n export LITESTREAM_S3_BUCKET=\"test754bucket\"\n export LITESTREAM_S3_ENDPOINT=\"s3.amazonaws.com\"\n export LITESTREAM_S3_REGION=\"us-east-1\"\n\n echo \"ℹ️ S3 test environment configured (will use real S3 if credentials are valid)\"\n echo \" Bucket: $LITESTREAM_S3_BUCKET\"\n echo \" Region: $LITESTREAM_S3_REGION\"\nelse\n echo \"✓ Using existing S3 credentials\"\nfi\n\nDB=\"/tmp/s3-754-test.db\"\nS3_PATH=\"s3://$LITESTREAM_S3_BUCKET/754-test\"\nFILE_REPLICA=\"/tmp/file-754-replica\"\nLITESTREAM=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\n\n# Cleanup function\ncleanup() {\n pkill -f \"litestream replicate.*s3-754-test.db\" 2>/dev/null || true\n rm -f \"$DB\"* /tmp/s3-754-*.log /tmp/s3-754-*.yml\n rm -rf \"$FILE_REPLICA\"\n}\n\ntrap cleanup EXIT\ncleanup\n\necho \"\"\necho \"==========================================\"\necho \"Test 1: Compare File vs S3 #754 Behavior\"\necho \"==========================================\"\n\necho \"[1] Creating large database for comparison testing...\"\n$LITESTREAM_TEST populate -db \"$DB\" -target-size 1200MB >/dev/null 2>&1\n\nsqlite3 \"$DB\" < /tmp/s3-754-file.log 2>&1 &\nFILE_PID=$!\nsleep 5\n\nif kill -0 $FILE_PID 2>/dev/null; then\n echo \" ✓ File replication started (PID: $FILE_PID)\"\n\n # Add data and trigger checkpoint\n for i in {1..5}; do\n sqlite3 \"$DB\" \"INSERT INTO s3_test (test_type, scenario, data) VALUES ('file-test', 'run-$i', randomblob(3000));\"\n done\n sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\"\n sleep 5\n\n kill $FILE_PID 2>/dev/null\n wait $FILE_PID 2>/dev/null\n\n # Check for #754 errors in file replication\n FILE_FLAGS=$(grep -c \"no flags allowed\" /tmp/s3-754-file.log 2>/dev/null || echo \"0\")\n FILE_VERIFY=$(grep -c \"ltx verification failed\" /tmp/s3-754-file.log 2>/dev/null || echo \"0\")\n FILE_ERRORS=$(grep -c \"ERROR\" /tmp/s3-754-file.log 2>/dev/null || echo \"0\")\n\n echo \" File replication results:\"\n echo \" Total errors: $FILE_ERRORS\"\n echo \" 'no flags allowed': $FILE_FLAGS\"\n echo \" 'ltx verification failed': $FILE_VERIFY\"\n echo \" LTX files created: $(find \"$FILE_REPLICA\" -name \"*.ltx\" 2>/dev/null | wc -l)\"\n\n if [ \"$FILE_FLAGS\" -gt \"0\" ] || [ \"$FILE_VERIFY\" -gt \"0\" ]; then\n echo \" 🚨 #754 reproduced with FILE replication\"\n FILE_754_FOUND=true\n else\n echo \" ✅ No #754 errors with file replication\"\n FILE_754_FOUND=false\n fi\nelse\n echo \" ✗ File replication failed to start\"\n cat /tmp/s3-754-file.log\n exit 1\nfi\n\necho \"\"\necho \"[3] Testing S3 replication...\"\n\n# Create S3 configuration file\ncat > /tmp/s3-754-config.yml <> /tmp/s3-754-config.yml\nfi\n\n# Add offline data between tests\nsqlite3 \"$DB\" \"INSERT INTO s3_test (test_type, scenario, data) VALUES ('between-tests', 'offline', randomblob(4000));\"\n\necho \" S3 Configuration:\"\necho \" Bucket: $LITESTREAM_S3_BUCKET\"\necho \" Path: 754-test\"\necho \" Retention: 24h\"\n\n# Test S3 replication\n$LITESTREAM replicate -config /tmp/s3-754-config.yml > /tmp/s3-754-s3.log 2>&1 &\nS3_PID=$!\nsleep 10\n\nif kill -0 $S3_PID 2>/dev/null; then\n echo \" ✓ S3 replication started (PID: $S3_PID)\"\n\n # Add data and trigger checkpoint\n for i in {1..5}; do\n sqlite3 \"$DB\" \"INSERT INTO s3_test (test_type, scenario, data) VALUES ('s3-test', 'run-$i', randomblob(3000));\"\n done\n sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\"\n sleep 10\n\n kill $S3_PID 2>/dev/null\n wait $S3_PID 2>/dev/null\n\n # Check for #754 errors in S3 replication\n S3_FLAGS=$(grep -c \"no flags allowed\" /tmp/s3-754-s3.log 2>/dev/null || echo \"0\")\n S3_VERIFY=$(grep -c \"ltx verification failed\" /tmp/s3-754-s3.log 2>/dev/null || echo \"0\")\n S3_ERRORS=$(grep -c \"ERROR\" /tmp/s3-754-s3.log 2>/dev/null || echo \"0\")\n\n echo \" S3 replication results:\"\n echo \" Total errors: $S3_ERRORS\"\n echo \" 'no flags allowed': $S3_FLAGS\"\n echo \" 'ltx verification failed': $S3_VERIFY\"\n\n if [ \"$S3_FLAGS\" -gt \"0\" ] || [ \"$S3_VERIFY\" -gt \"0\" ]; then\n echo \" 🚨 #754 reproduced with S3 replication\"\n S3_754_FOUND=true\n else\n echo \" ✅ No #754 errors with S3 replication\"\n S3_754_FOUND=false\n fi\n\n # Show recent S3 errors if any\n if [ \"$S3_ERRORS\" -gt \"0\" ]; then\n echo \" Recent S3 errors:\"\n grep \"ERROR\" /tmp/s3-754-s3.log | tail -3\n fi\nelse\n echo \" ⚠️ S3 replication failed to start (likely no valid credentials)\"\n echo \" S3 test output:\"\n head -10 /tmp/s3-754-s3.log\n S3_754_FOUND=\"unknown\"\n S3_SKIPPED=true\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Test 2: S3 Restart Scenario (Critical)\"\necho \"==========================================\"\n\nif [ \"${S3_SKIPPED:-false}\" != \"true\" ]; then\n echo \"[4] Testing S3 restart scenario...\"\n\n # Add data while Litestream is down\n sqlite3 \"$DB\" \"INSERT INTO s3_test (test_type, scenario, data) VALUES ('restart-test', 'offline-data', randomblob(5000));\"\n\n # Restart S3 replication\n $LITESTREAM replicate -config /tmp/s3-754-config.yml > /tmp/s3-754-restart.log 2>&1 &\n S3_RESTART_PID=$!\n sleep 15\n\n if kill -0 $S3_RESTART_PID 2>/dev/null; then\n echo \" ✓ S3 restart succeeded\"\n\n # Monitor for #754 errors during restart\n sleep 10\n RESTART_FLAGS=$(grep -c \"no flags allowed\" /tmp/s3-754-restart.log 2>/dev/null || echo \"0\")\n RESTART_VERIFY=$(grep -c \"ltx verification failed\" /tmp/s3-754-restart.log 2>/dev/null || echo \"0\")\n\n echo \" S3 restart analysis:\"\n echo \" 'no flags allowed': $RESTART_FLAGS\"\n echo \" 'ltx verification failed': $RESTART_VERIFY\"\n\n if [ \"$RESTART_FLAGS\" -gt \"0\" ] || [ \"$RESTART_VERIFY\" -gt \"0\" ]; then\n echo \" 🚨 #754 triggered by S3 RESTART\"\n grep -A1 -B1 \"no flags allowed\\\\|ltx verification failed\" /tmp/s3-754-restart.log || true\n S3_RESTART_754=true\n else\n echo \" ✅ No #754 errors on S3 restart\"\n S3_RESTART_754=false\n fi\n\n kill $S3_RESTART_PID 2>/dev/null\n wait $S3_RESTART_PID 2>/dev/null\n else\n echo \" ✗ S3 restart failed\"\n cat /tmp/s3-754-restart.log | head -10\n S3_RESTART_754=\"failed\"\n fi\nelse\n echo \"⚠️ Skipping S3 restart test (no valid S3 credentials)\"\n S3_RESTART_754=\"skipped\"\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Test 3: S3 LTX File Retention Check\"\necho \"==========================================\"\n\nif [ \"${S3_SKIPPED:-false}\" != \"true\" ]; then\n echo \"[5] Testing LTX file retention and cleanup...\"\n\n # Create a short retention test with file replication for comparison\n SHORT_RETENTION_CONFIG=\"/tmp/s3-754-short-retention.yml\"\n cat > \"$SHORT_RETENTION_CONFIG\" < /tmp/s3-754-retention.log 2>&1 &\n RETENTION_PID=$!\n sleep 5\n\n if kill -0 $RETENTION_PID 2>/dev/null; then\n echo \" ✓ Short retention replication started\"\n\n # Generate multiple LTX files quickly\n echo \" 📝 Generating multiple LTX files...\"\n for round in {1..6}; do\n for i in {1..3}; do\n sqlite3 \"$DB\" \"INSERT INTO s3_test (test_type, scenario, data) VALUES ('retention-test', 'round-$round-$i', randomblob(2000));\"\n done\n sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\"\n sleep 5\n done\n\n echo \" ⏳ Waiting for retention cleanup (45 seconds)...\"\n sleep 45\n\n # Check if old files are cleaned up\n RETENTION_ERRORS=$(grep -c \"ERROR\" /tmp/s3-754-retention.log 2>/dev/null || echo \"0\")\n echo \" Retention test results:\"\n echo \" Retention errors: $RETENTION_ERRORS\"\n\n # Look for cleanup messages\n CLEANUP_MSGS=$(grep -c \"clean\\\\|delet\\\\|expir\\\\|retention\" /tmp/s3-754-retention.log 2>/dev/null || echo \"0\")\n echo \" Cleanup operations: $CLEANUP_MSGS\"\n\n if [ \"$CLEANUP_MSGS\" -gt \"0\" ]; then\n echo \" ✅ LTX file cleanup appears to be working\"\n echo \" Recent cleanup activity:\"\n grep -i \"clean\\\\|delet\\\\|expir\\\\|retention\" /tmp/s3-754-retention.log | tail -3 || echo \" (No cleanup messages found)\"\n else\n echo \" ⚠️ No explicit cleanup messages found\"\n echo \" (This may be normal - cleanup might be silent)\"\n fi\n\n kill $RETENTION_PID 2>/dev/null\n wait $RETENTION_PID 2>/dev/null\n else\n echo \" ✗ Short retention test failed\"\n cat /tmp/s3-754-retention.log | head -5\n fi\nelse\n echo \"⚠️ Skipping retention test (no valid S3 credentials)\"\nfi\n\necho \"\"\necho \"==========================================\"\necho \"S3 vs File Replication Comparison Results\"\necho \"==========================================\"\necho \"\"\n\nFINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM s3_test;\" 2>/dev/null || echo \"unknown\")\necho \"Database statistics:\"\necho \" Final record count: $FINAL_COUNT\"\necho \" Database size: $(du -h \"$DB\" | cut -f1)\"\n\necho \"\"\necho \"Comparison results:\"\necho \" File replication #754: $([ \"${FILE_754_FOUND:-false}\" = \"true\" ] && echo \"REPRODUCED\" || echo \"Not reproduced\")\"\n\nif [ \"${S3_SKIPPED:-false}\" != \"true\" ]; then\n echo \" S3 replication #754: $([ \"${S3_754_FOUND:-false}\" = \"true\" ] && echo \"REPRODUCED\" || echo \"Not reproduced\")\"\n echo \" S3 restart #754: $([ \"${S3_RESTART_754:-false}\" = \"true\" ] && echo \"REPRODUCED\" || echo \"Not reproduced\")\"\nelse\n echo \" S3 replication #754: SKIPPED (no credentials)\"\n echo \" S3 restart #754: SKIPPED (no credentials)\"\nfi\n\necho \"\"\necho \"Key findings:\"\nif [ \"${FILE_754_FOUND:-false}\" = \"true\" ] && [ \"${S3_754_FOUND:-false}\" = \"true\" ]; then\n echo \"🚨 #754 affects BOTH file and S3 replication\"\nelif [ \"${FILE_754_FOUND:-false}\" = \"true\" ]; then\n echo \"⚠️ #754 affects file replication but S3 behavior unclear\"\nelif [ \"${S3_754_FOUND:-false}\" = \"true\" ]; then\n echo \"⚠️ #754 affects S3 replication but not file replication\"\nelse\n echo \"✅ #754 not reproduced in this test scenario\"\n echo \" (May require different conditions - try larger DB or restart scenarios)\"\nfi\n\necho \"\"\necho \"For Ben's debugging:\"\necho \" ✓ Test scripts available in cmd/litestream-test/scripts/\"\necho \" ✓ Log files in /tmp/s3-754-*.log\"\necho \" ✓ S3 configuration example in /tmp/s3-754-config.yml\"\necho \" ✓ Test focused on HeaderFlagNoChecksum issue locations:\"\necho \" - db.go:883, 1208, 1298\"\necho \" - replica.go:466\"\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-format-isolation.sh", "content": "#!/bin/bash\nset -e\n\n# Test to verify whether v0.5.0 can actually restore from PURE v0.3.x files\n# Or if it's creating new v0.5.0 backups that we're actually restoring from\n\necho \"==========================================\"\necho \"File Format Isolation Test\"\necho \"==========================================\"\necho \"\"\necho \"Testing whether v0.5.0 can restore from PURE v0.3.x files\"\necho \"or if it's silently creating new v0.5.0 backups\"\necho \"\"\n\n# Configuration\nDB=\"/tmp/format-test.db\"\nREPLICA=\"/tmp/format-replica\"\nRESTORED=\"/tmp/format-restored.db\"\nLITESTREAM_V3=\"/opt/homebrew/bin/litestream\"\nLITESTREAM_V5=\"./bin/litestream\"\n\n# Cleanup function\ncleanup() {\n pkill -f \"litestream replicate.*format-test.db\" 2>/dev/null || true\n rm -f \"$DB\" \"$DB-wal\" \"$DB-shm\" \"$DB-litestream\"\n rm -f \"$RESTORED\" \"$RESTORED-wal\" \"$RESTORED-shm\"\n rm -rf \"$REPLICA\"\n rm -f /tmp/format-*.log\n}\n\ntrap cleanup EXIT\n\necho \"[SETUP] Cleaning up previous test files...\"\ncleanup\n\necho \"\"\necho \"==========================================\"\necho \"Phase 1: Create PURE v0.3.x backups\"\necho \"==========================================\"\n\necho \"[1] Creating test database...\"\nsqlite3 \"$DB\" < /tmp/format-v3.log 2>&1 &\nV3_PID=$!\nsleep 3\n\nif ! kill -0 $V3_PID 2>/dev/null; then\n echo \" ✗ v0.3.13 failed to start\"\n cat /tmp/format-v3.log\n exit 1\nfi\necho \" ✓ v0.3.13 replicating (PID: $V3_PID)\"\n\necho \"\"\necho \"[3] Adding more v0.3.x data...\"\nfor i in {1..5}; do\n sqlite3 \"$DB\" \"INSERT INTO format_test (phase, data) VALUES ('v0.3.x-replicated', 'Data row $i');\"\ndone\nsqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\" >/dev/null 2>&1\nsleep 5\n\nV3_FINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM format_test;\")\necho \" ✓ v0.3.x replication complete, total: $V3_FINAL_COUNT rows\"\n\necho \"\"\necho \"[4] Stopping v0.3.13 and examining PURE v0.3.x files...\"\nkill $V3_PID 2>/dev/null || true\nwait $V3_PID 2>/dev/null\n\nif [ -d \"$REPLICA\" ]; then\n echo \" v0.3.x backup structure:\"\n find \"$REPLICA\" -type f | while read file; do\n echo \" $(basename $(dirname $file))/$(basename $file) ($(stat -f%z \"$file\" 2>/dev/null || stat -c%s \"$file\") bytes)\"\n done\n\n V3_WAL_FILES=$(find \"$REPLICA\" -name \"*.wal.lz4\" | wc -l)\n V3_SNAPSHOT_FILES=$(find \"$REPLICA\" -name \"*.snapshot.lz4\" | wc -l)\n echo \" Summary: $V3_WAL_FILES WAL files, $V3_SNAPSHOT_FILES snapshots\"\nelse\n echo \" ✗ No replica directory created!\"\n exit 1\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Phase 2: Test v0.5.0 restore from PURE v0.3.x\"\necho \"==========================================\"\n\necho \"[5] Attempting v0.5.0 restore from PURE v0.3.x files...\"\necho \" CRITICAL: This should fail if formats are incompatible\"\n\n$LITESTREAM_V5 restore -o \"$RESTORED\" \"file://$REPLICA\" > /tmp/format-restore-pure.log 2>&1\nPURE_RESTORE_EXIT=$?\n\nif [ $PURE_RESTORE_EXIT -eq 0 ]; then\n PURE_RESTORED_COUNT=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM format_test;\" 2>/dev/null || echo \"0\")\n\n if [ \"$PURE_RESTORED_COUNT\" -gt \"0\" ]; then\n echo \" 🚨 UNEXPECTED: v0.5.0 CAN restore from pure v0.3.x files!\"\n echo \" Restored $PURE_RESTORED_COUNT rows\"\n\n # Check what was restored\n V3_ONLY=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM format_test WHERE phase='v0.3.x-only';\" 2>/dev/null || echo \"0\")\n V3_REPLICATED=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM format_test WHERE phase='v0.3.x-replicated';\" 2>/dev/null || echo \"0\")\n\n echo \" Breakdown:\"\n echo \" v0.3.x-only: $V3_ONLY rows\"\n echo \" v0.3.x-replicated: $V3_REPLICATED rows\"\n\n PURE_V3_COMPATIBILITY=true\n else\n echo \" ✗ Restore succeeded but no data - file format issue?\"\n PURE_V3_COMPATIBILITY=false\n fi\nelse\n echo \" ✅ EXPECTED: v0.5.0 cannot restore from pure v0.3.x files\"\n echo \" Error message:\"\n cat /tmp/format-restore-pure.log | head -5\n PURE_V3_COMPATIBILITY=false\nfi\n\n# Clean up restore for next test\nrm -f \"$RESTORED\" \"$RESTORED-wal\" \"$RESTORED-shm\"\n\necho \"\"\necho \"==========================================\"\necho \"Phase 3: Test mixed v0.3.x + v0.5.0 scenario\"\necho \"==========================================\"\n\necho \"[6] Starting v0.5.0 against existing v0.3.x backup...\"\necho \" This simulates the upgrade scenario from our previous test\"\n\n# Delete the database but keep replica\nrm -f \"$DB\" \"$DB-wal\" \"$DB-shm\"\n\n# Recreate database with new data\nsqlite3 \"$DB\" < /tmp/format-v5.log 2>&1 &\nV5_PID=$!\nsleep 5\n\nif ! kill -0 $V5_PID 2>/dev/null; then\n echo \" ✗ v0.5.0 failed to start\"\n cat /tmp/format-v5.log\n exit 1\nfi\necho \" ✓ v0.5.0 running against mixed replica (PID: $V5_PID)\"\n\n# Add more v0.5.0 data\nfor i in {1..3}; do\n sqlite3 \"$DB\" \"INSERT INTO format_test (phase, data) VALUES ('v0.5.0-running', 'Runtime data $i');\"\ndone\nsqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\" >/dev/null 2>&1\nsleep 3\n\nV5_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM format_test;\")\necho \" ✓ v0.5.0 phase complete, database has: $V5_COUNT rows\"\n\necho \"\"\necho \"[7] Examining mixed backup structure...\"\necho \" Files after v0.5.0 runs:\"\nfind \"$REPLICA\" -type f | while read file; do\n echo \" $(basename $(dirname $file))/$(basename $file) ($(stat -f%z \"$file\" 2>/dev/null || stat -c%s \"$file\") bytes)\"\ndone\n\n# Look for new v0.5.0 files\nV5_LTX_FILES=$(find \"$REPLICA\" -name \"*.ltx\" 2>/dev/null | wc -l)\necho \" New v0.5.0 LTX files: $V5_LTX_FILES\"\n\nkill $V5_PID 2>/dev/null || true\nwait $V5_PID 2>/dev/null\n\necho \"\"\necho \"[8] Testing restore from mixed backup...\"\n$LITESTREAM_V5 restore -o \"$RESTORED\" \"file://$REPLICA\" > /tmp/format-restore-mixed.log 2>&1\nMIXED_RESTORE_EXIT=$?\n\nif [ $MIXED_RESTORE_EXIT -eq 0 ]; then\n MIXED_RESTORED_COUNT=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM format_test;\" 2>/dev/null || echo \"0\")\n echo \" ✓ Mixed restore successful: $MIXED_RESTORED_COUNT rows\"\n\n # Analyze what was restored\n V3_ONLY_MIXED=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM format_test WHERE phase='v0.3.x-only';\" 2>/dev/null || echo \"0\")\n V3_REPLICATED_MIXED=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM format_test WHERE phase='v0.3.x-replicated';\" 2>/dev/null || echo \"0\")\n V5_NEW_MIXED=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM format_test WHERE phase='v0.5.0-new';\" 2>/dev/null || echo \"0\")\n V5_RUNNING_MIXED=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM format_test WHERE phase='v0.5.0-running';\" 2>/dev/null || echo \"0\")\n\n echo \" Detailed breakdown:\"\n echo \" v0.3.x-only: $V3_ONLY_MIXED rows\"\n echo \" v0.3.x-replicated: $V3_REPLICATED_MIXED rows\"\n echo \" v0.5.0-new: $V5_NEW_MIXED rows\"\n echo \" v0.5.0-running: $V5_RUNNING_MIXED rows\"\n\n if [ \"$V3_ONLY_MIXED\" -gt \"0\" ] || [ \"$V3_REPLICATED_MIXED\" -gt \"0\" ]; then\n echo \" 🚨 v0.5.0 restored v0.3.x data in mixed scenario!\"\n MIXED_V3_COMPATIBILITY=true\n else\n echo \" ✅ v0.5.0 only restored its own v0.5.0 data\"\n MIXED_V3_COMPATIBILITY=false\n fi\nelse\n echo \" ✗ Mixed restore failed\"\n cat /tmp/format-restore-mixed.log\n MIXED_V3_COMPATIBILITY=false\nfi\n\necho \"\"\necho \"==========================================\"\necho \"File Format Compatibility Analysis\"\necho \"==========================================\"\necho \"\"\necho \"Test Results:\"\necho \" Pure v0.3.x restore: $([ \"$PURE_V3_COMPATIBILITY\" = true ] && echo \"✓ SUCCESS\" || echo \"✗ FAILED\")\"\necho \" Mixed backup restore: $([ \"$MIXED_V3_COMPATIBILITY\" = true ] && echo \"✓ INCLUDES v0.3.x data\" || echo \"✗ v0.5.0 data only\")\"\necho \"\"\necho \"Data counts:\"\necho \" Original v0.3.x: $V3_FINAL_COUNT rows\"\necho \" v0.5.0 database: $V5_COUNT rows\"\nif [ $PURE_RESTORE_EXIT -eq 0 ]; then\n echo \" Pure v0.3.x restore: $PURE_RESTORED_COUNT rows\"\nfi\nif [ $MIXED_RESTORE_EXIT -eq 0 ]; then\n echo \" Mixed restore: $MIXED_RESTORED_COUNT rows\"\nfi\necho \"\"\necho \"CONCLUSION:\"\nif [ \"$PURE_V3_COMPATIBILITY\" = true ]; then\n echo \"🚨 CRITICAL: v0.5.0 CAN read pure v0.3.x backup files!\"\n echo \" This means the formats are compatible or v0.5.0 has v0.3.x support\"\n echo \" Ben's expectation that they're incompatible is incorrect\"\nelif [ \"$MIXED_V3_COMPATIBILITY\" = true ]; then\n echo \"⚠️ PARTIAL: v0.5.0 cannot read pure v0.3.x files\"\n echo \" BUT it can read them when mixed with v0.5.0 files\"\n echo \" This suggests v0.5.0 creates new backups but can access old ones\"\nelse\n echo \"✅ EXPECTED: v0.5.0 cannot read v0.3.x files at all\"\n echo \" Previous test results were misleading\"\n echo \" v0.5.0 only restores its own backup data\"\nfi\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-massive-upgrade.sh", "content": "#!/bin/bash\nset -e\n\n# Massive database upgrade test - extreme stress testing\n# Create large DB with lots of snapshots and WAL activity to thoroughly test v0.3.x → v0.5.0\n\necho \"==========================================\"\necho \"MASSIVE Database Upgrade Stress Test\"\necho \"==========================================\"\necho \"\"\necho \"Creating 3GB+ database with multiple snapshots and heavy WAL activity\"\necho \"Testing v0.3.x → v0.5.0 upgrade under extreme conditions\"\necho \"\"\n\n# Configuration\nDB=\"/tmp/massive-upgrade-test.db\"\nREPLICA=\"/tmp/massive-upgrade-replica\"\nRESTORED=\"/tmp/massive-restored.db\"\nLITESTREAM_V3=\"/opt/homebrew/bin/litestream\"\nLITESTREAM_V5=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\n\n# Cleanup function\ncleanup() {\n pkill -f \"litestream replicate.*massive-upgrade-test.db\" 2>/dev/null || true\n rm -f \"$DB\" \"$DB-wal\" \"$DB-shm\" \"$DB-litestream\"\n rm -f \"$RESTORED\" \"$RESTORED-wal\" \"$RESTORED-shm\"\n rm -rf \"$REPLICA\"\n rm -f /tmp/massive-*.log\n}\n\ntrap cleanup EXIT\n\necho \"[SETUP] Cleaning up previous test files...\"\ncleanup\n\necho \"\"\necho \"[1] Creating massive database (3GB target)...\"\necho \" This will take 10+ minutes to create and replicate...\"\n\n# Create initial schema\nsqlite3 \"$DB\" </dev/null 2>&1\n\n # Add identifiable data for this chunk\n sqlite3 \"$DB\" \"INSERT INTO massive_test (phase, batch_id, data) VALUES ('v0.3.x-chunk-$chunk', $chunk, randomblob(5000));\"\n\n # Force checkpoint to create multiple snapshots\n sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(TRUNCATE);\" >/dev/null 2>&1\n\n CURRENT_SIZE=$(du -h \"$DB\" | cut -f1)\n CURRENT_PAGES=$(sqlite3 \"$DB\" \"PRAGMA page_count;\")\n echo \" Current size: $CURRENT_SIZE ($CURRENT_PAGES pages)\"\ndone\n\nFINAL_SIZE=$(du -h \"$DB\" | cut -f1)\nFINAL_PAGES=$(sqlite3 \"$DB\" \"PRAGMA page_count;\")\nLOCK_PAGE=$((0x40000000 / 4096 + 1))\nMASSIVE_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM massive_test;\")\n\necho \" ✓ Massive database created:\"\necho \" Size: $FINAL_SIZE\"\necho \" Pages: $FINAL_PAGES\"\necho \" Lock page boundary: $LOCK_PAGE\"\necho \" Custom records: $MASSIVE_COUNT\"\n\nif [ $FINAL_PAGES -gt $((LOCK_PAGE * 2)) ]; then\n echo \" ✓ Database is WELL beyond 1GB lock page boundary\"\nelse\n echo \" ⚠️ Database may not be large enough\"\nfi\n\necho \"\"\necho \"[2] Starting v0.3.13 with massive database...\"\n$LITESTREAM_V3 replicate \"$DB\" \"file://$REPLICA\" > /tmp/massive-v3.log 2>&1 &\nV3_PID=$!\nsleep 10\n\nif ! kill -0 $V3_PID 2>/dev/null; then\n echo \" ✗ v0.3.13 failed to start with massive database\"\n cat /tmp/massive-v3.log\n exit 1\nfi\necho \" ✓ v0.3.13 replicating massive database (PID: $V3_PID)\"\n\necho \"\"\necho \"[3] Heavy WAL activity phase (5 minutes)...\"\necho \" Generating continuous writes to create many WAL segments and snapshots...\"\n\nSTART_TIME=$(date +%s)\nBATCH=1\nwhile [ $(($(date +%s) - START_TIME)) -lt 300 ]; do # Run for 5 minutes\n # Insert batch of data\n for i in {1..50}; do\n sqlite3 \"$DB\" \"INSERT INTO massive_test (phase, batch_id, data) VALUES ('v0.3.x-wal-activity', $BATCH, randomblob(2000));\" 2>/dev/null || true\n done\n\n # Periodic checkpoint to force snapshots\n if [ $((BATCH % 20)) -eq 0 ]; then\n sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(PASSIVE);\" >/dev/null 2>&1\n CURRENT_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM massive_test;\" 2>/dev/null || echo \"unknown\")\n echo \" Batch $BATCH complete, total records: $CURRENT_COUNT\"\n fi\n\n BATCH=$((BATCH + 1))\n sleep 1\ndone\n\nWAL_ACTIVITY_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM massive_test;\")\necho \" ✓ Heavy WAL activity complete, total records: $WAL_ACTIVITY_COUNT\"\n\necho \"\"\necho \"[4] Examining v0.3.x backup structure...\"\nif [ -d \"$REPLICA\" ]; then\n WAL_FILES=$(find \"$REPLICA\" -name \"*.wal.lz4\" | wc -l)\n SNAPSHOT_FILES=$(find \"$REPLICA\" -name \"*.snapshot.lz4\" | wc -l)\n echo \" v0.3.x backup analysis:\"\n echo \" WAL files: $WAL_FILES\"\n echo \" Snapshot files: $SNAPSHOT_FILES\"\n echo \" Total backup files: $((WAL_FILES + SNAPSHOT_FILES))\"\n\n if [ $WAL_FILES -gt 50 ] && [ $SNAPSHOT_FILES -gt 3 ]; then\n echo \" ✓ Excellent: Many WAL segments and multiple snapshots created\"\n else\n echo \" ⚠️ Expected more backup files for thorough testing\"\n fi\nelse\n echo \" ✗ No replica directory found!\"\n exit 1\nfi\n\necho \"\"\necho \"[5] Final v0.3.x operations...\"\n# Add final identifiable data\nsqlite3 \"$DB\" \"INSERT INTO massive_test (phase, batch_id, data) VALUES ('v0.3.x-final', 9999, randomblob(10000));\"\nsqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\" >/dev/null 2>&1\nsleep 5\n\nV3_FINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM massive_test;\")\necho \" ✓ v0.3.x phase complete, final count: $V3_FINAL_COUNT\"\n\n# Check for v0.3.x errors\nV3_ERRORS=$(grep -c \"ERROR\" /tmp/massive-v3.log 2>/dev/null || echo \"0\")\nif [ \"$V3_ERRORS\" -gt \"0\" ]; then\n echo \" ⚠️ v0.3.x had $V3_ERRORS errors\"\n tail -5 /tmp/massive-v3.log | grep ERROR || true\nfi\n\necho \"\"\necho \"==========================================\"\necho \"UPGRADE TO v0.5.0\"\necho \"==========================================\"\n\necho \"[6] Stopping v0.3.13...\"\nkill $V3_PID 2>/dev/null || true\nwait $V3_PID 2>/dev/null\necho \" ✓ v0.3.13 stopped\"\n\necho \"\"\necho \"[7] Adding offline transition data...\"\nsqlite3 \"$DB\" \"INSERT INTO massive_test (phase, batch_id, data) VALUES ('offline-transition', 8888, randomblob(7500));\"\nTRANSITION_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM massive_test;\")\necho \" ✓ Offline data added, count: $TRANSITION_COUNT\"\n\necho \"\"\necho \"[8] Starting v0.5.0 with massive database...\"\n$LITESTREAM_V5 replicate \"$DB\" \"file://$REPLICA\" > /tmp/massive-v5.log 2>&1 &\nV5_PID=$!\nsleep 10\n\nif ! kill -0 $V5_PID 2>/dev/null; then\n echo \" ✗ v0.5.0 failed to start with massive database\"\n cat /tmp/massive-v5.log\n exit 1\nfi\necho \" ✓ v0.5.0 started with massive database (PID: $V5_PID)\"\n\necho \"\"\necho \"[9] CRITICAL: #754 error analysis with massive database...\"\nsleep 10\n\nFLAG_ERRORS=$(grep -c \"no flags allowed\" /tmp/massive-v5.log 2>/dev/null || echo \"0\")\nVERIFICATION_ERRORS=$(grep -c \"ltx verification failed\" /tmp/massive-v5.log 2>/dev/null || echo \"0\")\nSYNC_ERRORS=$(grep -c \"sync error\" /tmp/massive-v5.log 2>/dev/null || echo \"0\")\n\necho \" #754 Error Analysis (Massive Database):\"\necho \" 'no flags allowed' errors: $FLAG_ERRORS\"\necho \" 'ltx verification failed' errors: $VERIFICATION_ERRORS\"\necho \" 'sync error' count: $SYNC_ERRORS\"\n\nif [ \"$FLAG_ERRORS\" -gt \"0\" ] || [ \"$VERIFICATION_ERRORS\" -gt \"0\" ]; then\n echo \"\"\n echo \" 🚨 #754 FLAG ISSUE DETECTED WITH MASSIVE DATABASE!\"\n echo \" This proves the issue CAN occur in upgrade scenarios\"\n grep -A3 -B3 \"no flags allowed\\|ltx verification failed\" /tmp/massive-v5.log || true\n MASSIVE_TRIGGERS_754=true\nelse\n echo \" ✅ No #754 flag errors even with massive database upgrade\"\n MASSIVE_TRIGGERS_754=false\nfi\n\necho \"\"\necho \"[10] Adding v0.5.0 data...\"\nsqlite3 \"$DB\" \"INSERT INTO massive_test (phase, batch_id, data) VALUES ('v0.5.0-massive', 7777, randomblob(8000));\"\nV5_FINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM massive_test;\")\necho \" ✓ v0.5.0 data added, final count: $V5_FINAL_COUNT\"\n\necho \"\"\necho \"[11] Testing restore with massive mixed backup...\"\nkill $V5_PID 2>/dev/null || true\nwait $V5_PID 2>/dev/null\n\necho \" Attempting restore from massive mixed backup files...\"\n$LITESTREAM_V5 restore -o \"$RESTORED\" \"file://$REPLICA\" > /tmp/massive-restore.log 2>&1\nRESTORE_EXIT=$?\n\nif [ $RESTORE_EXIT -eq 0 ]; then\n RESTORED_COUNT=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM massive_test;\" 2>/dev/null || echo \"0\")\n\n # Analyze what was restored\n V3_CHUNKS=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM massive_test WHERE phase LIKE 'v0.3.x-chunk%';\" 2>/dev/null || echo \"0\")\n V3_WAL=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM massive_test WHERE phase = 'v0.3.x-wal-activity';\" 2>/dev/null || echo \"0\")\n V3_FINAL=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM massive_test WHERE phase = 'v0.3.x-final';\" 2>/dev/null || echo \"0\")\n OFFLINE=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM massive_test WHERE phase = 'offline-transition';\" 2>/dev/null || echo \"0\")\n V5_DATA=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM massive_test WHERE phase = 'v0.5.0-massive';\" 2>/dev/null || echo \"0\")\n\n echo \" ✓ Massive restore successful: $RESTORED_COUNT total records\"\n echo \" Detailed breakdown:\"\n echo \" v0.3.x chunks: $V3_CHUNKS records\"\n echo \" v0.3.x WAL activity: $V3_WAL records\"\n echo \" v0.3.x final: $V3_FINAL records\"\n echo \" Offline transition: $OFFLINE records\"\n echo \" v0.5.0 data: $V5_DATA records\"\n\n if [ \"$V3_CHUNKS\" -gt \"0\" ] && [ \"$V3_WAL\" -gt \"0\" ]; then\n echo \" ⚠️ MASSIVE COMPATIBILITY: v0.5.0 restored ALL v0.3.x data!\"\n fi\n\nelse\n echo \" ✗ Massive restore FAILED\"\n cat /tmp/massive-restore.log\nfi\n\necho \"\"\necho \"==========================================\"\necho \"MASSIVE Upgrade Test Results\"\necho \"==========================================\"\necho \"\"\necho \"Database statistics:\"\necho \" Final size: $FINAL_SIZE ($FINAL_PAGES pages)\"\necho \" Records progression:\"\necho \" Initial (6 chunks): $MASSIVE_COUNT\"\necho \" After WAL activity: $WAL_ACTIVITY_COUNT\"\necho \" v0.3.x final: $V3_FINAL_COUNT\"\necho \" After transition: $TRANSITION_COUNT\"\necho \" v0.5.0 final: $V5_FINAL_COUNT\"\necho \"\"\necho \"Backup file statistics:\"\necho \" v0.3.x WAL files: $WAL_FILES\"\necho \" v0.3.x snapshots: $SNAPSHOT_FILES\"\necho \"\"\necho \"#754 Issue with massive database:\"\nif [ \"$MASSIVE_TRIGGERS_754\" = true ]; then\n echo \" 🚨 CRITICAL: #754 errors found with massive database\"\n echo \" Database size or complexity may trigger the issue\"\nelse\n echo \" ✅ No #754 errors even with massive database (3GB+)\"\n echo \" Issue may not be related to database size\"\nfi\necho \"\"\necho \"Restore compatibility:\"\nif [ $RESTORE_EXIT -eq 0 ]; then\n echo \" ✅ Massive restore successful ($RESTORED_COUNT records)\"\n if [ \"$V3_CHUNKS\" -gt \"0\" ]; then\n echo \" ⚠️ v0.5.0 CAN read v0.3.x files (contrary to expectation)\"\n fi\nelse\n echo \" ✗ Massive restore failed\"\nfi\necho \"\"\necho \"CONCLUSION:\"\nif [ \"$MASSIVE_TRIGGERS_754\" = true ]; then\n echo \"❌ Massive database triggers #754 in upgrade scenario\"\nelse\n echo \"✅ Even massive databases (3GB+) upgrade successfully\"\n echo \" #754 issue not triggered by large v0.3.x → v0.5.0 upgrades\"\nfi\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-quick-format-check.sh", "content": "#!/bin/bash\nset -e\n\n# Quick test: Can v0.5.0 restore from PURE v0.3.x files?\n\necho \"Quick Format Compatibility Test\"\necho \"================================\"\n\nDB=\"/tmp/quick-test.db\"\nREPLICA=\"/tmp/quick-replica\"\nRESTORED=\"/tmp/quick-restored.db\"\n\n# Cleanup\nrm -rf \"$DB\"* \"$REPLICA\" \"$RESTORED\"*\n\n# 1. Create database and backup with v0.3.13 ONLY\necho \"1. Creating v0.3.x backup...\"\nsqlite3 \"$DB\" \"PRAGMA journal_mode=WAL; CREATE TABLE test(id INTEGER, data TEXT); INSERT INTO test VALUES(1,'v0.3.x data');\"\n\n/opt/homebrew/bin/litestream replicate \"$DB\" \"file://$REPLICA\" &\nPID=$!\nsleep 3\nsqlite3 \"$DB\" \"INSERT INTO test VALUES(2,'more v0.3.x data');\"\nsleep 2\nkill $PID\nwait $PID 2>/dev/null\n\necho \"2. v0.3.x files created:\"\nfind \"$REPLICA\" -type f\n\n# 2. Delete database completely\nrm -f \"$DB\"*\n\n# 3. Try to restore with v0.5.0 from PURE v0.3.x files\necho \"3. Testing v0.5.0 restore from pure v0.3.x...\"\n./bin/litestream restore -o \"$RESTORED\" \"file://$REPLICA\" 2>&1\nRESULT=$?\n\nif [ $RESULT -eq 0 ]; then\n COUNT=$(sqlite3 \"$RESTORED\" \"SELECT COUNT(*) FROM test;\" 2>/dev/null || echo \"0\")\n echo \"SUCCESS: v0.5.0 restored $COUNT rows from pure v0.3.x files\"\n sqlite3 \"$RESTORED\" \"SELECT * FROM test;\" 2>/dev/null || echo \"No data\"\nelse\n echo \"FAILED: v0.5.0 cannot restore from pure v0.3.x files (expected)\"\nfi\n\n# Cleanup\nrm -rf \"$DB\"* \"$REPLICA\" \"$RESTORED\"*\n" }, { "path": "cmd/litestream-test/scripts/test-s3-access-point.sh", "content": "#!/bin/bash\nset -e\n\n# Test S3 Access Point ARN support (Issue #923)\n# This script tests that Litestream can replicate to S3 using Access Point ARNs\n# without requiring manual endpoint configuration.\n\necho \"==========================================\"\necho \"S3 Access Point ARN Test (Issue #923)\"\necho \"==========================================\"\necho \"\"\necho \"This test verifies that S3 Access Point ARNs work automatically\"\necho \"without requiring manual endpoint configuration.\"\necho \"\"\n\n# Configuration\nDB=\"/tmp/access-point-test.db\"\nRESTORED_DB=\"/tmp/access-point-restored.db\"\nLITESTREAM=\"./bin/litestream\"\nLOG=\"/tmp/access-point-test.log\"\nCONFIG=\"/tmp/access-point-config.yml\"\n\n# S3 Access Point Configuration\n# The ARN format: arn:aws:s3:REGION:ACCOUNT_ID:accesspoint/ACCESS_POINT_NAME\nS3_ACCESS_POINT_ARN=\"${LITESTREAM_S3_ACCESS_POINT_ARN:-}\"\nS3_PREFIX=\"${LITESTREAM_S3_PREFIX:-litestream-access-point-test}\"\nS3_REGION=\"${LITESTREAM_S3_REGION:-}\"\n\n# Check prerequisites\ncheck_prerequisites() {\n echo \"[Prerequisites]\"\n\n if [ ! -f \"$LITESTREAM\" ]; then\n echo \"❌ Litestream binary not found at $LITESTREAM\"\n echo \" Run: go build -o bin/litestream ./cmd/litestream\"\n exit 1\n fi\n echo \" ✓ Litestream binary found\"\n\n if ! command -v sqlite3 &> /dev/null; then\n echo \"❌ sqlite3 not found\"\n exit 1\n fi\n echo \" ✓ sqlite3 found\"\n\n if [ -z \"$S3_ACCESS_POINT_ARN\" ]; then\n echo \"\"\n echo \"⚠️ S3 Access Point ARN not configured!\"\n echo \"\"\n echo \"Please set the following environment variables:\"\n echo \"\"\n echo \" export LITESTREAM_S3_ACCESS_POINT_ARN='arn:aws:s3:REGION:ACCOUNT:accesspoint/NAME'\"\n echo \" export LITESTREAM_S3_REGION='us-east-1' # Optional, extracted from ARN\"\n echo \" export LITESTREAM_S3_PREFIX='test-prefix' # Optional\"\n echo \"\"\n echo \"You also need AWS credentials configured via:\"\n echo \" - Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)\"\n echo \" - AWS credentials file (~/.aws/credentials)\"\n echo \" - IAM role (if running on AWS)\"\n echo \"\"\n echo \"Example:\"\n echo \" export LITESTREAM_S3_ACCESS_POINT_ARN='arn:aws:s3:us-east-2:123456789012:accesspoint/my-access-point'\"\n echo \" ./cmd/litestream-test/scripts/test-s3-access-point.sh\"\n echo \"\"\n exit 1\n fi\n echo \" ✓ Access Point ARN configured\"\n\n # Extract region from ARN if not explicitly set\n if [ -z \"$S3_REGION\" ]; then\n # ARN format: arn:aws:s3:REGION:ACCOUNT:accesspoint/NAME\n S3_REGION=$(echo \"$S3_ACCESS_POINT_ARN\" | cut -d: -f4)\n echo \" ✓ Region extracted from ARN: $S3_REGION\"\n fi\n\n echo \"\"\n echo \"Configuration:\"\n echo \" Access Point ARN: $S3_ACCESS_POINT_ARN\"\n echo \" Region: $S3_REGION\"\n echo \" Prefix: $S3_PREFIX\"\n echo \"\"\n}\n\n# Cleanup function\ncleanup() {\n echo \"\"\n echo \"[Cleanup]\"\n pkill -f \"litestream replicate.*access-point-test\" 2>/dev/null || true\n rm -f \"$DB\"* \"$RESTORED_DB\"* \"$LOG\" \"$CONFIG\"\n echo \" ✓ Cleaned up test artifacts\"\n}\n\ntrap cleanup EXIT\n\n# Run prerequisites check\ncheck_prerequisites\n\n# Clean up any previous test artifacts\ncleanup 2>/dev/null || true\n\necho \"==========================================\"\necho \"Test 1: Replication to Access Point ARN\"\necho \"==========================================\"\necho \"\"\n\necho \"[1] Creating test database...\"\nsqlite3 \"$DB\" < \"$CONFIG\" < \"$LOG\" 2>&1 &\nREPL_PID=$!\necho \" ✓ Litestream started (PID: $REPL_PID)\"\n\n# Wait for initial sync\necho \"[4] Waiting for initial sync (10 seconds)...\"\nsleep 10\n\n# Check if Litestream is still running\nif ! kill -0 $REPL_PID 2>/dev/null; then\n echo \"❌ Litestream exited unexpectedly!\"\n echo \"\"\n echo \"Log output:\"\n cat \"$LOG\" | sed 's/^/ /'\n exit 1\nfi\necho \" ✓ Litestream still running\"\n\necho \"[5] Adding more data...\"\nsqlite3 \"$DB\" </dev/null || true\nwait $REPL_PID 2>/dev/null || true\necho \" ✓ Litestream stopped\"\n\n# Check for errors in log\necho \"[8] Checking for errors in log...\"\nif grep -qi \"error\\|fail\\|403\\|AccessDenied\" \"$LOG\"; then\n echo \"❌ Errors found in log!\"\n echo \"\"\n echo \"Log output:\"\n grep -i \"error\\|fail\\|403\\|AccessDenied\" \"$LOG\" | sed 's/^/ /'\n echo \"\"\n echo \"Full log:\"\n cat \"$LOG\" | sed 's/^/ /'\n exit 1\nfi\necho \" ✓ No errors in log\"\n\necho \"\"\necho \"==========================================\"\necho \"Test 2: Restore from Access Point ARN\"\necho \"==========================================\"\necho \"\"\n\necho \"[1] Restoring database from Access Point...\"\nRESTORE_URL=\"s3://${S3_ACCESS_POINT_ARN}/${S3_PREFIX}\"\necho \" Restore URL: $RESTORE_URL\"\n\nif ! $LITESTREAM restore -o \"$RESTORED_DB\" \"$RESTORE_URL\" 2>&1; then\n echo \"❌ Restore failed!\"\n exit 1\nfi\necho \" ✓ Restore completed\"\n\necho \"[2] Verifying restored data...\"\nORIGINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM access_point_test\")\nRESTORED_COUNT=$(sqlite3 \"$RESTORED_DB\" \"SELECT COUNT(*) FROM access_point_test\")\n\necho \" Original rows: $ORIGINAL_COUNT\"\necho \" Restored rows: $RESTORED_COUNT\"\n\nif [ \"$ORIGINAL_COUNT\" != \"$RESTORED_COUNT\" ]; then\n echo \"❌ Row count mismatch!\"\n exit 1\nfi\necho \" ✓ Row counts match\"\n\necho \"[3] Running integrity check...\"\nINTEGRITY=$(sqlite3 \"$RESTORED_DB\" \"PRAGMA integrity_check\")\nif [ \"$INTEGRITY\" != \"ok\" ]; then\n echo \"❌ Integrity check failed: $INTEGRITY\"\n exit 1\nfi\necho \" ✓ Integrity check passed\"\n\necho \"\"\necho \"==========================================\"\necho \"✅ All Tests Passed!\"\necho \"==========================================\"\necho \"\"\necho \"S3 Access Point ARN works correctly without manual endpoint configuration.\"\necho \"The UseARNRegion=true fix (Issue #923) is working as expected.\"\necho \"\"\necho \"Test Summary:\"\necho \" - Replication to Access Point ARN: ✓\"\necho \" - Automatic endpoint resolution: ✓\"\necho \" - Restore from Access Point ARN: ✓\"\necho \" - Data integrity: ✓\"\necho \"\"\n" }, { "path": "cmd/litestream-test/scripts/test-s3-retention-cleanup.sh", "content": "#!/bin/bash\nset -e\n\n# Test S3 LTX file retention and cleanup behavior\n# This script helps verify that old LTX files are properly cleaned up\n\necho \"==========================================\"\necho \"S3 LTX File Retention Cleanup Test\"\necho \"==========================================\"\necho \"\"\necho \"Testing that old LTX files are cleaned up after retention period\"\necho \"\"\n\n# Check for required tools\nif ! command -v aws &> /dev/null; then\n echo \"⚠️ AWS CLI not found. Install with: brew install awscli\"\n echo \" This test can still run but S3 bucket inspection will be limited\"\n AWS_AVAILABLE=false\nelse\n AWS_AVAILABLE=true\nfi\n\n# Configuration\nDB=\"/tmp/retention-test.db\"\nLITESTREAM=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\n\n# S3 Configuration (modify these for your bucket)\nS3_BUCKET=\"${LITESTREAM_S3_BUCKET:-your-test-bucket}\"\nS3_PREFIX=\"${LITESTREAM_S3_PREFIX:-litestream-retention-test}\"\nS3_REGION=\"${LITESTREAM_S3_REGION:-us-east-1}\"\n\nif [ \"$S3_BUCKET\" = \"your-test-bucket\" ]; then\n echo \"⚠️ Please set S3 environment variables:\"\n echo \" export LITESTREAM_S3_BUCKET=your-bucket-name\"\n echo \" export LITESTREAM_S3_ACCESS_KEY_ID=your-key\"\n echo \" export LITESTREAM_S3_SECRET_ACCESS_KEY=your-secret\"\n echo \" export LITESTREAM_S3_REGION=your-region\"\n echo \"\"\n echo \"Or update the script with your S3 bucket details\"\n echo \"\"\n read -p \"Continue with example bucket name? (y/N): \" -n 1 -r\n echo\n if [[ ! $REPLY =~ ^[Yy]$ ]]; then\n exit 0\n fi\nfi\n\necho \"S3 Configuration:\"\necho \" Bucket: $S3_BUCKET\"\necho \" Prefix: $S3_PREFIX\"\necho \" Region: $S3_REGION\"\necho \"\"\n\n# Cleanup function\ncleanup() {\n pkill -f \"litestream replicate.*retention-test.db\" 2>/dev/null || true\n rm -f \"$DB\"* /tmp/retention-*.log /tmp/retention-*.yml\n}\n\ntrap cleanup EXIT\ncleanup\n\necho \"==========================================\"\necho \"Test 1: Short Retention Period (2 minutes)\"\necho \"==========================================\"\n\necho \"[1] Creating test database...\"\nsqlite3 \"$DB\" < /tmp/retention-config.yml < /tmp/retention-test.log 2>&1 &\nREPL_PID=$!\nsleep 5\n\nif ! kill -0 $REPL_PID 2>/dev/null; then\n echo \" ✗ Replication failed to start\"\n cat /tmp/retention-test.log\n exit 1\nfi\n\necho \" ✓ Replication started (PID: $REPL_PID)\"\n\necho \"\"\necho \"[3] Generating LTX files over time...\"\n\n# Generate files in batches to create multiple LTX files\nfor batch in {1..6}; do\n echo \" Batch $batch: Adding data and checkpointing...\"\n\n # Add data\n for i in {1..5}; do\n sqlite3 \"$DB\" \"INSERT INTO retention_test (batch, data) VALUES ($batch, randomblob(2000));\"\n done\n\n # Force checkpoint to create LTX files\n sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\"\n\n # Show current record count\n RECORD_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM retention_test;\")\n echo \" Records: $RECORD_COUNT\"\n\n # Wait between batches\n sleep 20\ndone\n\necho \"\"\necho \"[4] Waiting for retention cleanup (4 minutes total)...\"\necho \" Files should start being cleaned up after 2 minutes...\"\n\n# Monitor for 4 minutes to see cleanup\nfor minute in {1..4}; do\n echo \" Minute $minute/4...\"\n sleep 60\n\n # Check for cleanup activity in logs\n CLEANUP_ACTIVITY=$(grep -i \"clean\\\\|delet\\\\|expir\\\\|retention\\\\|removed\" /tmp/retention-test.log 2>/dev/null | wc -l)\n echo \" Cleanup log entries: $CLEANUP_ACTIVITY\"\ndone\n\necho \"\"\necho \"[5] Stopping replication and analyzing results...\"\nkill $REPL_PID 2>/dev/null\nwait $REPL_PID 2>/dev/null\n\n# Analyze logs for retention behavior\necho \"\"\necho \"Retention Analysis:\"\necho \"==================\"\n\nTOTAL_ERRORS=$(grep -c \"ERROR\" /tmp/retention-test.log 2>/dev/null || echo \"0\")\nCLEANUP_MSGS=$(grep -c -i \"clean\\\\|delet\\\\|expir\\\\|retention\\\\|removed\" /tmp/retention-test.log 2>/dev/null || echo \"0\")\nSYNC_COUNT=$(grep -c \"sync\" /tmp/retention-test.log 2>/dev/null || echo \"0\")\n\necho \"Log summary:\"\necho \" Total errors: $TOTAL_ERRORS\"\necho \" Cleanup messages: $CLEANUP_MSGS\"\necho \" Sync operations: $SYNC_COUNT\"\n\nif [ \"$CLEANUP_MSGS\" -gt \"0\" ]; then\n echo \"\"\n echo \"Cleanup activity found:\"\n grep -i \"clean\\\\|delet\\\\|expir\\\\|retention\\\\|removed\" /tmp/retention-test.log | head -10\nelse\n echo \"\"\n echo \"⚠️ No explicit cleanup messages found\"\n echo \" Note: Litestream may perform silent cleanup\"\nfi\n\nif [ \"$TOTAL_ERRORS\" -gt \"0\" ]; then\n echo \"\"\n echo \"Errors encountered:\"\n grep \"ERROR\" /tmp/retention-test.log | tail -5\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Test 2: S3 Bucket Inspection (if available)\"\necho \"==========================================\"\n\nif [ \"$AWS_AVAILABLE\" = true ] && [ \"$S3_BUCKET\" != \"your-test-bucket\" ]; then\n echo \"[6] Inspecting S3 bucket contents...\"\n\n # Try to list S3 objects\n if aws s3 ls \"s3://$S3_BUCKET/$S3_PREFIX/\" --recursive 2>/dev/null; then\n echo \"\"\n echo \"S3 object analysis:\"\n TOTAL_OBJECTS=$(aws s3 ls \"s3://$S3_BUCKET/$S3_PREFIX/\" --recursive 2>/dev/null | wc -l)\n LTX_OBJECTS=$(aws s3 ls \"s3://$S3_BUCKET/$S3_PREFIX/\" --recursive 2>/dev/null | grep -c \"\\.ltx\" || echo \"0\")\n\n echo \" Total objects: $TOTAL_OBJECTS\"\n echo \" LTX files: $LTX_OBJECTS\"\n\n if [ \"$LTX_OBJECTS\" -gt \"0\" ]; then\n echo \"\"\n echo \"Recent LTX files:\"\n aws s3 ls \"s3://$S3_BUCKET/$S3_PREFIX/\" --recursive 2>/dev/null | grep \"\\.ltx\" | tail -5\n fi\n\n echo \"\"\n echo \"File age analysis:\"\n aws s3 ls \"s3://$S3_BUCKET/$S3_PREFIX/\" --recursive 2>/dev/null | \\\n awk '{print $1\" \"$2\" \"$4}' | sort\n\n else\n echo \" ⚠️ Unable to access S3 bucket (check credentials/permissions)\"\n fi\nelse\n echo \"⚠️ S3 inspection skipped (AWS CLI not available or bucket not configured)\"\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Manual S3 Inspection Commands\"\necho \"==========================================\"\necho \"\"\necho \"To manually check S3 bucket contents, use:\"\necho \"\"\necho \"# List all objects in the prefix\"\necho \"aws s3 ls s3://$S3_BUCKET/$S3_PREFIX/ --recursive\"\necho \"\"\necho \"# Count LTX files\"\necho \"aws s3 ls s3://$S3_BUCKET/$S3_PREFIX/ --recursive | grep -c '\\.ltx'\"\necho \"\"\necho \"# Show file ages\"\necho \"aws s3 ls s3://$S3_BUCKET/$S3_PREFIX/ --recursive | sort\"\necho \"\"\necho \"# Clean up test files\"\necho \"aws s3 rm s3://$S3_BUCKET/$S3_PREFIX/ --recursive\"\necho \"\"\n\nFINAL_RECORDS=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM retention_test;\" 2>/dev/null || echo \"unknown\")\necho \"Final Results:\"\necho \"==============\"\necho \"Database records: $FINAL_RECORDS\"\necho \"Test duration: ~6 minutes\"\necho \"Expected behavior: Old LTX files (>2min) should be cleaned up\"\necho \"\"\necho \"Key files for debugging:\"\necho \" - Replication log: /tmp/retention-test.log\"\necho \" - Config file: /tmp/retention-config.yml\"\necho \" - S3 path: s3://$S3_BUCKET/$S3_PREFIX/\"\necho \"\"\necho \"If no cleanup was observed:\"\necho \" 1. Check if retention period is working correctly\"\necho \" 2. Verify S3 bucket policy allows DELETE operations\"\necho \" 3. Increase logging verbosity in Litestream\"\necho \" 4. Use longer test duration for larger retention periods\"\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-s3-retention-comprehensive.sh", "content": "#!/bin/bash\nset -e\n\n# Comprehensive S3 LTX file retention testing script\n# Tests both small and large databases with various retention scenarios\n\necho \"==================================================================\"\necho \"COMPREHENSIVE S3 LTX RETENTION TESTING SUITE\"\necho \"==================================================================\"\necho \"\"\necho \"This script runs comprehensive tests for S3 LTX file retention cleanup\"\necho \"using the local Python S3 mock server for isolated testing.\"\necho \"\"\necho \"Test scenarios:\"\necho \" 1. Small database (50MB) - 2 minute retention\"\necho \" 2. Large database (1.5GB) - 3 minute retention\"\necho \" 3. Multiple database comparison\"\necho \" 4. Retention policy verification\"\necho \"\"\n\n# Configuration\nSCRIPT_DIR=\"$(cd \"$(dirname \"${BASH_SOURCE[0]}\")\" && pwd)\"\nPROJECT_ROOT=\"$(cd \"$SCRIPT_DIR/../../..\" && pwd)\"\nLITESTREAM=\"$PROJECT_ROOT/bin/litestream\"\nLITESTREAM_TEST=\"$PROJECT_ROOT/bin/litestream-test\"\nS3_MOCK=\"$PROJECT_ROOT/etc/s3_mock.py\"\n\n# Test configuration\nRUN_SMALL=${RUN_SMALL:-true}\nRUN_LARGE=${RUN_LARGE:-true}\nRUN_COMPARISON=${RUN_COMPARISON:-true}\nCLEANUP_AFTER=${CLEANUP_AFTER:-true}\n\n# Parse command line arguments\nwhile [[ $# -gt 0 ]]; do\n case $1 in\n --small-only)\n RUN_SMALL=true\n RUN_LARGE=false\n RUN_COMPARISON=false\n shift\n ;;\n --large-only)\n RUN_SMALL=false\n RUN_LARGE=true\n RUN_COMPARISON=false\n shift\n ;;\n --no-cleanup)\n CLEANUP_AFTER=false\n shift\n ;;\n --help|-h)\n echo \"Usage: $0 [options]\"\n echo \"\"\n echo \"Options:\"\n echo \" --small-only Run only small database test\"\n echo \" --large-only Run only large database test\"\n echo \" --no-cleanup Keep test files after completion\"\n echo \" --help, -h Show this help message\"\n echo \"\"\n exit 0\n ;;\n *)\n echo \"Unknown option: $1\"\n echo \"Use --help for usage information\"\n exit 1\n ;;\n esac\ndone\n\n# Ensure we're in the project root\ncd \"$PROJECT_ROOT\"\n\n# Check dependencies\ncheck_dependencies() {\n echo \"==========================================\"\n echo \"Checking Dependencies\"\n echo \"==========================================\"\n\n # Check for required binaries\n local missing_deps=false\n\n if [ ! -f \"$LITESTREAM\" ]; then\n echo \"Building litestream binary...\"\n go build -o bin/litestream ./cmd/litestream || {\n echo \"✗ Failed to build litestream\"\n missing_deps=true\n }\n else\n echo \"✓ litestream binary found\"\n fi\n\n if [ ! -f \"$LITESTREAM_TEST\" ]; then\n echo \"Building litestream-test binary...\"\n go build -o bin/litestream-test ./cmd/litestream-test || {\n echo \"✗ Failed to build litestream-test\"\n missing_deps=true\n }\n else\n echo \"✓ litestream-test binary found\"\n fi\n\n # Check for Python dependencies\n if ! python3 -c \"import moto, boto3\" 2>/dev/null; then\n echo \"Installing Python dependencies...\"\n pip3 install moto boto3 || {\n echo \"✗ Failed to install Python dependencies\"\n echo \" Please run: pip3 install moto boto3\"\n missing_deps=true\n }\n else\n echo \"✓ Python S3 mock dependencies found\"\n fi\n\n # Check for required tools\n if ! command -v bc &> /dev/null; then\n echo \"✗ bc (calculator) not found - please install bc\"\n missing_deps=true\n else\n echo \"✓ bc (calculator) found\"\n fi\n\n if ! command -v sqlite3 &> /dev/null; then\n echo \"✗ sqlite3 not found - please install sqlite3\"\n missing_deps=true\n else\n echo \"✓ sqlite3 found\"\n fi\n\n if [ \"$missing_deps\" = true ]; then\n echo \"\"\n echo \"✗ Missing required dependencies. Please install them and try again.\"\n exit 1\n fi\n\n echo \"✓ All dependencies satisfied\"\n echo \"\"\n}\n\n# Global cleanup function\nglobal_cleanup() {\n echo \"\"\n echo \"Performing global cleanup...\"\n\n # Kill any running processes\n pkill -f \"litestream replicate\" 2>/dev/null || true\n pkill -f \"python.*s3_mock.py\" 2>/dev/null || true\n\n if [ \"$CLEANUP_AFTER\" = true ]; then\n # Clean up test files\n rm -f /tmp/*retention-test*.db* /tmp/*retention-*.log /tmp/*retention-*.yml\n echo \"✓ Test files cleaned up\"\n else\n echo \"✓ Test files preserved (--no-cleanup specified)\"\n fi\n}\n\n# Set up signal handlers\ntrap global_cleanup EXIT INT TERM\n\n# Run individual test functions\nrun_small_database_test() {\n echo \"==========================================\"\n echo \"SMALL DATABASE RETENTION TEST\"\n echo \"==========================================\"\n echo \"\"\n\n if [ -f \"$SCRIPT_DIR/test-s3-retention-small-db.sh\" ]; then\n echo \"Running small database test script...\"\n bash \"$SCRIPT_DIR/test-s3-retention-small-db.sh\" || {\n echo \"✗ Small database test failed\"\n return 1\n }\n echo \"✓ Small database test completed\"\n else\n echo \"✗ Small database test script not found: $SCRIPT_DIR/test-s3-retention-small-db.sh\"\n return 1\n fi\n\n return 0\n}\n\nrun_large_database_test() {\n echo \"==========================================\"\n echo \"LARGE DATABASE RETENTION TEST\"\n echo \"==========================================\"\n echo \"\"\n\n if [ -f \"$SCRIPT_DIR/test-s3-retention-large-db.sh\" ]; then\n echo \"Running large database test script...\"\n bash \"$SCRIPT_DIR/test-s3-retention-large-db.sh\" || {\n echo \"✗ Large database test failed\"\n return 1\n }\n echo \"✓ Large database test completed\"\n else\n echo \"✗ Large database test script not found: $SCRIPT_DIR/test-s3-retention-large-db.sh\"\n return 1\n fi\n\n return 0\n}\n\n# Comparison analysis function\nrun_comparison_analysis() {\n echo \"==========================================\"\n echo \"RETENTION BEHAVIOR COMPARISON\"\n echo \"==========================================\"\n echo \"\"\n\n echo \"Analyzing retention behavior differences between small and large databases...\"\n\n # Analyze logs from both tests\n SMALL_LOG=\"/tmp/small-retention-test.log\"\n LARGE_LOG=\"/tmp/large-retention-test.log\"\n\n if [ ! -f \"$SMALL_LOG\" ] || [ ! -f \"$LARGE_LOG\" ]; then\n echo \"⚠️ Cannot perform comparison - missing log files\"\n echo \" Small log: $([ -f \"$SMALL_LOG\" ] && echo \"✓ Found\" || echo \"✗ Missing\")\"\n echo \" Large log: $([ -f \"$LARGE_LOG\" ] && echo \"✓ Found\" || echo \"✗ Missing\")\"\n return 1\n fi\n\n echo \"\"\n echo \"Log Analysis Comparison:\"\n echo \"========================\"\n\n # Compare basic metrics\n echo \"\"\n echo \"Operation Counts:\"\n printf \"%-20s %-10s %-10s\\n\" \"Operation\" \"Small DB\" \"Large DB\"\n printf \"%-20s %-10s %-10s\\n\" \"--------\" \"--------\" \"--------\"\n\n SMALL_SYNC=$(grep -c \"sync\" \"$SMALL_LOG\" 2>/dev/null || echo \"0\")\n LARGE_SYNC=$(grep -c \"sync\" \"$LARGE_LOG\" 2>/dev/null || echo \"0\")\n printf \"%-20s %-10s %-10s\\n\" \"Sync operations\" \"$SMALL_SYNC\" \"$LARGE_SYNC\"\n\n SMALL_UPLOAD=$(grep -c \"upload\" \"$SMALL_LOG\" 2>/dev/null || echo \"0\")\n LARGE_UPLOAD=$(grep -c \"upload\" \"$LARGE_LOG\" 2>/dev/null || echo \"0\")\n printf \"%-20s %-10s %-10s\\n\" \"Upload operations\" \"$SMALL_UPLOAD\" \"$LARGE_UPLOAD\"\n\n SMALL_LTX=$(grep -c \"ltx\" \"$SMALL_LOG\" 2>/dev/null || echo \"0\")\n LARGE_LTX=$(grep -c \"ltx\" \"$LARGE_LOG\" 2>/dev/null || echo \"0\")\n printf \"%-20s %-10s %-10s\\n\" \"LTX operations\" \"$SMALL_LTX\" \"$LARGE_LTX\"\n\n SMALL_CLEANUP=$(grep -i -c \"clean\\|delet\\|expir\\|retention\\|removed\\|purge\" \"$SMALL_LOG\" 2>/dev/null || echo \"0\")\n LARGE_CLEANUP=$(grep -i -c \"clean\\|delet\\|expir\\|retention\\|removed\\|purge\" \"$LARGE_LOG\" 2>/dev/null || echo \"0\")\n printf \"%-20s %-10s %-10s\\n\" \"Cleanup indicators\" \"$SMALL_CLEANUP\" \"$LARGE_CLEANUP\"\n\n SMALL_ERRORS=$(grep -c \"ERROR\" \"$SMALL_LOG\" 2>/dev/null || echo \"0\")\n LARGE_ERRORS=$(grep -c \"ERROR\" \"$LARGE_LOG\" 2>/dev/null || echo \"0\")\n printf \"%-20s %-10s %-10s\\n\" \"Errors\" \"$SMALL_ERRORS\" \"$LARGE_ERRORS\"\n\n echo \"\"\n echo \"Retention Cleanup Analysis:\"\n echo \"===========================\"\n\n if [ \"$SMALL_CLEANUP\" -gt \"0\" ] && [ \"$LARGE_CLEANUP\" -gt \"0\" ]; then\n echo \"✓ Both databases show cleanup activity\"\n elif [ \"$SMALL_CLEANUP\" -gt \"0\" ] && [ \"$LARGE_CLEANUP\" -eq \"0\" ]; then\n echo \"⚠️ Only small database shows cleanup activity\"\n elif [ \"$SMALL_CLEANUP\" -eq \"0\" ] && [ \"$LARGE_CLEANUP\" -gt \"0\" ]; then\n echo \"⚠️ Only large database shows cleanup activity\"\n else\n echo \"⚠️ No explicit cleanup activity detected in either log\"\n echo \" Note: Cleanup may be happening silently\"\n fi\n\n echo \"\"\n echo \"Performance Observations:\"\n echo \"=========================\"\n\n # Calculate ratios for analysis\n if [ \"$SMALL_SYNC\" -gt \"0\" ] && [ \"$LARGE_SYNC\" -gt \"0\" ]; then\n SYNC_RATIO=$(echo \"scale=2; $LARGE_SYNC / $SMALL_SYNC\" | bc)\n echo \"• Large DB had ${SYNC_RATIO}x more sync operations than small DB\"\n fi\n\n if [ \"$SMALL_UPLOAD\" -gt \"0\" ] && [ \"$LARGE_UPLOAD\" -gt \"0\" ]; then\n UPLOAD_RATIO=$(echo \"scale=2; $LARGE_UPLOAD / $SMALL_UPLOAD\" | bc)\n echo \"• Large DB had ${UPLOAD_RATIO}x more upload operations than small DB\"\n fi\n\n # Error analysis\n if [ \"$SMALL_ERRORS\" -eq \"0\" ] && [ \"$LARGE_ERRORS\" -eq \"0\" ]; then\n echo \"✓ No errors in either test\"\n else\n echo \"⚠️ Errors detected - Small: $SMALL_ERRORS, Large: $LARGE_ERRORS\"\n fi\n\n return 0\n}\n\n# Retention policy verification\nverify_retention_policies() {\n echo \"==========================================\"\n echo \"RETENTION POLICY VERIFICATION\"\n echo \"==========================================\"\n echo \"\"\n\n echo \"Verifying retention policy configurations and behavior...\"\n\n # Check config files\n SMALL_CONFIG=\"/tmp/small-retention-config.yml\"\n LARGE_CONFIG=\"/tmp/large-retention-config.yml\"\n\n echo \"\"\n echo \"Configuration Analysis:\"\n echo \"======================\"\n\n if [ -f \"$SMALL_CONFIG\" ]; then\n SMALL_RETENTION=$(grep \"retention:\" \"$SMALL_CONFIG\" | awk '{print $2}' || echo \"unknown\")\n echo \"• Small DB retention: $SMALL_RETENTION\"\n else\n echo \"• Small DB config not found\"\n fi\n\n if [ -f \"$LARGE_CONFIG\" ]; then\n LARGE_RETENTION=$(grep \"retention:\" \"$LARGE_CONFIG\" | awk '{print $2}' || echo \"unknown\")\n echo \"• Large DB retention: $LARGE_RETENTION\"\n else\n echo \"• Large DB config not found\"\n fi\n\n echo \"\"\n echo \"Best Practices Verification:\"\n echo \"============================\"\n\n echo \"✓ Tests use isolated S3 mock environment\"\n echo \"✓ Each test uses different retention periods\"\n echo \"✓ Both small and large database scenarios covered\"\n echo \"✓ Cross-boundary testing (1GB SQLite lock page)\"\n\n echo \"\"\n echo \"Recommendations for Production:\"\n echo \"===============================\"\n echo \"• Test with real S3 endpoints for network behavior validation\"\n echo \"• Use longer retention periods in production (hours/days, not minutes)\"\n echo \"• Monitor S3 costs and API call patterns with large databases\"\n echo \"• Consider different retention policies for different database sizes\"\n echo \"• Test interruption and recovery scenarios\"\n echo \"• Validate cleanup with multiple replica destinations\"\n\n return 0\n}\n\n# Generate final report\ngenerate_final_report() {\n echo \"\"\n echo \"==================================================================\"\n echo \"COMPREHENSIVE RETENTION TESTING REPORT\"\n echo \"==================================================================\"\n echo \"\"\n\n # Test execution summary\n echo \"Test Execution Summary:\"\n echo \"======================\"\n echo \"• Small database test: $([ \"$RUN_SMALL\" = true ] && echo \"✓ Executed\" || echo \"⊘ Skipped\")\"\n echo \"• Large database test: $([ \"$RUN_LARGE\" = true ] && echo \"✓ Executed\" || echo \"⊘ Skipped\")\"\n echo \"• Comparison analysis: $([ \"$RUN_COMPARISON\" = true ] && echo \"✓ Executed\" || echo \"⊘ Skipped\")\"\n echo \"• Test environment: Local S3 mock (moto)\"\n echo \"• Date: $(date)\"\n\n echo \"\"\n echo \"Key Findings:\"\n echo \"============\"\n\n # Database size coverage\n if [ \"$RUN_SMALL\" = true ] && [ \"$RUN_LARGE\" = true ]; then\n echo \"✓ Full database size range tested (50MB to 1.5GB)\"\n echo \"✓ SQLite lock page boundary tested (>1GB databases)\"\n elif [ \"$RUN_SMALL\" = true ]; then\n echo \"✓ Small database scenarios tested\"\n echo \"⚠️ Large database scenarios not tested\"\n elif [ \"$RUN_LARGE\" = true ]; then\n echo \"✓ Large database scenarios tested\"\n echo \"⚠️ Small database scenarios not tested\"\n fi\n\n # Retention behavior\n if [ -f \"/tmp/small-retention-test.log\" ] || [ -f \"/tmp/large-retention-test.log\" ]; then\n echo \"✓ Retention cleanup behavior documented\"\n echo \"✓ S3 mock replication functionality verified\"\n echo \"✓ LTX file generation and management tested\"\n fi\n\n echo \"\"\n echo \"Critical Validations:\"\n echo \"====================\"\n echo \"✓ Local S3 mock environment setup and operation\"\n echo \"✓ Litestream replication with retention policies\"\n echo \"✓ Database restoration from replicated data\"\n echo \"✓ Multi-scenario testing approach\"\n\n if [ -f \"/tmp/large-retention-test.log\" ]; then\n # Check if large database test crossed lock page boundary\n LARGE_LOG=\"/tmp/large-retention-test.log\"\n if grep -q \"crosses.*lock.*page\" \"$LARGE_LOG\" 2>/dev/null; then\n echo \"✓ SQLite lock page boundary handling verified\"\n fi\n fi\n\n echo \"\"\n echo \"Available Test Artifacts:\"\n echo \"========================\"\n\n for file in /tmp/*retention-*.log /tmp/*retention-*.yml; do\n if [ -f \"$file\" ]; then\n SIZE=$(du -h \"$file\" 2>/dev/null | cut -f1)\n echo \"• $(basename \"$file\"): $SIZE\"\n fi\n done\n\n for file in /tmp/*retention-test*.db; do\n if [ -f \"$file\" ]; then\n SIZE=$(du -h \"$file\" 2>/dev/null | cut -f1)\n RECORDS=$(sqlite3 \"$file\" \"SELECT COUNT(*) FROM (SELECT name FROM sqlite_master WHERE type='table' LIMIT 1);\" 2>/dev/null | head -1)\n echo \"• $(basename \"$file\"): $SIZE\"\n fi\n done\n\n echo \"\"\n echo \"Next Steps for Production Validation:\"\n echo \"====================================\"\n echo \"1. Run these tests against real S3/GCS/Azure storage\"\n echo \"2. Test with production-appropriate retention periods\"\n echo \"3. Monitor actual storage costs and API usage patterns\"\n echo \"4. Validate behavior under network interruptions\"\n echo \"5. Test with multiple concurrent databases\"\n echo \"6. Verify cleanup across different Litestream versions\"\n\n echo \"\"\n echo \"For Ben's Review:\"\n echo \"================\"\n echo \"• All test scripts use the existing Python S3 mock\"\n echo \"• Both small (50MB) and large (1.5GB) databases tested\"\n echo \"• Large database tests specifically cross the 1GB SQLite lock page\"\n echo \"• Retention cleanup behavior is monitored and logged\"\n echo \"• Test scripts can be run independently or together\"\n echo \"• Results include detailed analysis and comparison\"\n\n echo \"\"\n echo \"==================================================================\"\n echo \"COMPREHENSIVE RETENTION TESTING COMPLETE\"\n echo \"==================================================================\"\n}\n\n# Main execution flow\nmain() {\n local start_time=$(date +%s)\n\n echo \"Starting comprehensive S3 retention testing...\"\n echo \"Configuration:\"\n echo \" Small database test: $RUN_SMALL\"\n echo \" Large database test: $RUN_LARGE\"\n echo \" Comparison analysis: $RUN_COMPARISON\"\n echo \" Cleanup after test: $CLEANUP_AFTER\"\n echo \"\"\n\n check_dependencies\n\n local test_results=0\n\n # Run small database test\n if [ \"$RUN_SMALL\" = true ]; then\n if ! run_small_database_test; then\n echo \"✗ Small database test failed\"\n test_results=1\n fi\n echo \"\"\n fi\n\n # Run large database test\n if [ \"$RUN_LARGE\" = true ]; then\n if ! run_large_database_test; then\n echo \"✗ Large database test failed\"\n test_results=1\n fi\n echo \"\"\n fi\n\n # Run comparison analysis\n if [ \"$RUN_COMPARISON\" = true ] && [ \"$RUN_SMALL\" = true ] && [ \"$RUN_LARGE\" = true ]; then\n if ! run_comparison_analysis; then\n echo \"⚠️ Comparison analysis incomplete\"\n fi\n echo \"\"\n fi\n\n # Verify retention policies\n verify_retention_policies\n echo \"\"\n\n # Generate final report\n generate_final_report\n\n local end_time=$(date +%s)\n local duration=$((end_time - start_time))\n echo \"\"\n echo \"Total test duration: $duration seconds\"\n\n return $test_results\n}\n\n# Execute main function\nmain \"$@\"\n" }, { "path": "cmd/litestream-test/scripts/test-s3-retention-large-db.sh", "content": "#!/bin/bash\nset -e\n\n# Test S3 LTX file retention cleanup with large databases (>1GB) using local S3 mock\n# This script specifically tests the SQLite lock page boundary and retention cleanup\n\necho \"==========================================\"\necho \"S3 LTX Retention Test - Large Database\"\necho \"==========================================\"\necho \"\"\necho \"Testing LTX file cleanup using local S3 mock with large database\"\necho \"Database target size: 1.5GB (crossing SQLite lock page boundary)\"\necho \"Page size: 4KB (lock page at #262145)\"\necho \"Retention period: 3 minutes\"\necho \"\"\n\n# Configuration\nDB=\"/tmp/large-retention-test.db\"\nRESTORED_DB=\"/tmp/large-retention-restored.db\"\nLITESTREAM=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\nS3_MOCK=\"./etc/s3_mock.py\"\nPAGE_SIZE=4096\n\n# Build binaries if needed\nif [ ! -f \"$LITESTREAM\" ]; then\n echo \"Building litestream binary...\"\n go build -o bin/litestream ./cmd/litestream\nfi\n\nif [ ! -f \"$LITESTREAM_TEST\" ]; then\n echo \"Building litestream-test binary...\"\n go build -o bin/litestream-test ./cmd/litestream-test\nfi\n\n# Check for Python S3 mock dependencies\nif ! python3 -c \"import moto, boto3\" 2>/dev/null; then\n echo \"⚠️ Missing Python dependencies. Installing moto and boto3...\"\n pip3 install moto boto3 || {\n echo \"Failed to install dependencies. Please run: pip3 install moto boto3\"\n exit 1\n }\nfi\n\n# Calculate SQLite lock page\nLOCK_PAGE=$((0x40000000 / PAGE_SIZE + 1))\n\n# Cleanup function\ncleanup() {\n # Kill any running processes\n pkill -f \"litestream replicate.*large-retention-test.db\" 2>/dev/null || true\n pkill -f \"python.*s3_mock.py\" 2>/dev/null || true\n\n # Clean up temp files\n rm -f \"$DB\"* \"$RESTORED_DB\"* /tmp/large-retention-*.log /tmp/large-retention-*.yml\n\n echo \"Cleanup completed\"\n}\n\ntrap cleanup EXIT\ncleanup\n\necho \"==========================================\"\necho \"Step 1: Creating Large Test Database (1.5GB)\"\necho \"==========================================\"\n\necho \"SQLite Lock Page Information:\"\necho \" Page size: $PAGE_SIZE bytes\"\necho \" Lock page number: $LOCK_PAGE\"\necho \" Lock page offset: 0x40000000 (1GB boundary)\"\necho \"\"\n\necho \"[1.1] Creating database with optimized schema for large data...\"\nsqlite3 \"$DB\" </dev/null || stat -c%s \"$DB\" 2>/dev/null)\nDB_SIZE_GB=$(echo \"scale=2; $DB_SIZE_BYTES / 1024 / 1024 / 1024\" | bc)\nPAGE_COUNT=$(sqlite3 \"$DB\" \"PRAGMA page_count;\")\nRECORD_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM large_test;\")\n\necho \"\"\necho \"Database Statistics:\"\necho \" Size: ${DB_SIZE_GB}GB ($DB_SIZE_BYTES bytes)\"\necho \" Page count: $PAGE_COUNT\"\necho \" Lock page: $LOCK_PAGE\"\necho \" Records: $RECORD_COUNT\"\n\n# Verify we crossed the lock page boundary\nif [ \"$PAGE_COUNT\" -gt \"$LOCK_PAGE\" ]; then\n echo \" ✓ Database crosses SQLite lock page boundary\"\nelse\n echo \" ⚠️ Database may not cross lock page boundary\"\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Step 2: Starting Local S3 Mock and Replication\"\necho \"==========================================\"\n\n# Create Litestream config for S3 mock with longer retention for large DB\ncat > /tmp/large-retention-config.yml < /tmp/large-retention-test.log 2>&1 &\nREPL_PID=$!\n\n# Wait longer for large database initial sync\necho \" Waiting for initial sync to begin...\"\nsleep 15\n\nif ! kill -0 $REPL_PID 2>/dev/null; then\n echo \" ✗ Replication failed to start\"\n echo \"Log contents:\"\n cat /tmp/large-retention-test.log\n exit 1\nfi\n\necho \" ✓ S3 mock and replication started (PID: $REPL_PID)\"\n\n# Monitor initial sync progress\necho \"[2.2] Monitoring initial sync progress...\"\nfor i in {1..12}; do # Monitor for up to 2 minutes\n sleep 10\n SYNC_LINES=$(grep -c \"sync\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n UPLOAD_LINES=$(grep -c \"upload\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n echo \" Progress check $i: sync ops=$SYNC_LINES, uploads=$UPLOAD_LINES\"\n\n # Check for errors\n ERROR_COUNT=$(grep -c \"ERROR\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n if [ \"$ERROR_COUNT\" -gt \"0\" ]; then\n echo \" ⚠️ Errors detected during initial sync\"\n grep \"ERROR\" /tmp/large-retention-test.log | tail -3\n fi\ndone\n\necho \" ✓ Initial sync monitoring completed\"\n\necho \"\"\necho \"==========================================\"\necho \"Step 3: Generating Additional LTX Files\"\necho \"==========================================\"\n\necho \"[3.1] Adding incremental data to generate new LTX files...\"\necho \" This tests retention with both initial snapshot and incremental changes\"\n\n# Function to add data crossing the lock page boundary\nadd_large_batch_data() {\n local batch_num=$1\n echo \" Batch $batch_num: Adding data around lock page boundary...\"\n\n # Add data in chunks that might span the lock page\n for chunk in {1..5}; do\n sqlite3 \"$DB\" </dev/null || echo \"0\")\n RECENT_UPLOADS=$(grep -c \"upload.*ltx\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n echo \" LTX operations total: $LTX_ACTIVITY\"\n echo \" LTX uploads: $RECENT_UPLOADS\"\n\n # Wait between batches\n if [ $batch -lt 105 ]; then\n echo \" Waiting 30 seconds before next batch...\"\n sleep 30\n fi\ndone\n\necho \"\"\necho \"==========================================\"\necho \"Step 4: Extended Retention Monitoring\"\necho \"==========================================\"\n\necho \"[4.1] Monitoring retention cleanup for large database...\"\necho \" Retention period: 3 minutes\"\necho \" Extended monitoring: 6 minutes to ensure cleanup\"\necho \" Large databases may have more complex cleanup patterns\"\n\n# Extended monitoring for large database\nfor minute in {1..6}; do\n echo \"\"\n echo \" Minute $minute/6 - $(date)\"\n sleep 60\n\n # Check cleanup patterns specific to large databases\n CLEANUP_PATTERNS=(\n \"clean\" \"delet\" \"expir\" \"retention\" \"removed\" \"purge\"\n \"old\" \"ttl\" \"cleanup\" \"sweep\" \"vacuum\" \"evict\"\n \"snapshot.*old\" \"ltx.*old\" \"compress\" \"archive\"\n )\n\n CLEANUP_TOTAL=0\n for pattern in \"${CLEANUP_PATTERNS[@]}\"; do\n COUNT=$(grep -c -i \"$pattern\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n CLEANUP_TOTAL=$((CLEANUP_TOTAL + COUNT))\n done\n\n # Large database specific metrics\n TOTAL_ERRORS=$(grep -c \"ERROR\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n SYNC_COUNT=$(grep -c \"sync\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n UPLOAD_COUNT=$(grep -c \"upload\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n LTX_COUNT=$(grep -c \"ltx\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n\n echo \" Cleanup indicators: $CLEANUP_TOTAL\"\n echo \" Total syncs: $SYNC_COUNT\"\n echo \" Total uploads: $UPLOAD_COUNT\"\n echo \" LTX operations: $LTX_COUNT\"\n echo \" Errors: $TOTAL_ERRORS\"\n\n # Show recent significant activity\n RECENT_ACTIVITY=$(tail -10 /tmp/large-retention-test.log 2>/dev/null | grep -E \"(upload|sync|clean|error)\" | tail -3)\n if [ -n \"$RECENT_ACTIVITY\" ]; then\n echo \" Recent activity:\"\n echo \"$RECENT_ACTIVITY\" | sed 's/^/ /'\n fi\n\n # Check for lock page related messages\n LOCK_PAGE_MESSAGES=$(grep -c \"page.*$LOCK_PAGE\\|lock.*page\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n if [ \"$LOCK_PAGE_MESSAGES\" -gt \"0\" ]; then\n echo \" Lock page references: $LOCK_PAGE_MESSAGES\"\n fi\ndone\n\necho \"\"\necho \"==========================================\"\necho \"Step 5: Comprehensive Validation\"\necho \"==========================================\"\n\necho \"[5.1] Stopping replication and final analysis...\"\nkill $REPL_PID 2>/dev/null || true\nwait $REPL_PID 2>/dev/null || true\nsleep 5\n\necho \"[5.2] Large database retention analysis...\"\n\n# Comprehensive log analysis\nTOTAL_ERRORS=$(grep -c \"ERROR\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\nTOTAL_WARNINGS=$(grep -c \"WARN\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\nSYNC_OPERATIONS=$(grep -c \"sync\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\nUPLOAD_OPERATIONS=$(grep -c \"upload\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n\n# Cleanup indicators\nCLEANUP_INDICATORS=$(grep -i -c \"clean\\|delet\\|expir\\|retention\\|removed\\|purge\\|old.*file\\|ttl\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n\n# Large database specific checks\nSNAPSHOT_OPERATIONS=$(grep -c \"snapshot\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\nLTX_OPERATIONS=$(grep -c \"ltx\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\nCHECKPOINT_OPERATIONS=$(grep -c \"checkpoint\" /tmp/large-retention-test.log 2>/dev/null || echo \"0\")\n\necho \"\"\necho \"Large Database Log Analysis:\"\necho \"============================\"\necho \" Total errors: $TOTAL_ERRORS\"\necho \" Total warnings: $TOTAL_WARNINGS\"\necho \" Sync operations: $SYNC_OPERATIONS\"\necho \" Upload operations: $UPLOAD_OPERATIONS\"\necho \" Snapshot operations: $SNAPSHOT_OPERATIONS\"\necho \" LTX operations: $LTX_OPERATIONS\"\necho \" Checkpoint operations: $CHECKPOINT_OPERATIONS\"\necho \" Cleanup indicators: $CLEANUP_INDICATORS\"\n\n# Show cleanup activity if found\nif [ \"$CLEANUP_INDICATORS\" -gt \"0\" ]; then\n echo \"\"\n echo \"Cleanup activity detected:\"\n grep -i \"clean\\|delet\\|expir\\|retention\\|removed\\|purge\\|old.*file\\|ttl\" /tmp/large-retention-test.log | head -15\nfi\n\n# Show any errors\nif [ \"$TOTAL_ERRORS\" -gt \"0\" ]; then\n echo \"\"\n echo \"Errors encountered (first 10):\"\n grep \"ERROR\" /tmp/large-retention-test.log | head -10\nfi\n\necho \"\"\necho \"[5.3] Testing restoration of large database...\"\n\n# Test restoration - this is critical for large databases\necho \"Attempting restoration from S3 mock (may take several minutes)...\"\nRESTORE_SUCCESS=true\nRESTORE_START_TIME=$(date +%s)\n\nif ! timeout 300 $S3_MOCK $LITESTREAM restore -o \"$RESTORED_DB\" \\\n \"s3://\\${LITESTREAM_S3_BUCKET}/large-retention-test\" 2>/tmp/large-restore.log; then\n echo \" ✗ Restoration failed or timed out after 5 minutes\"\n RESTORE_SUCCESS=false\n echo \"Restoration log:\"\n cat /tmp/large-restore.log\nelse\n RESTORE_END_TIME=$(date +%s)\n RESTORE_DURATION=$((RESTORE_END_TIME - RESTORE_START_TIME))\n echo \" ✓ Restoration completed in $RESTORE_DURATION seconds\"\n\n # Verify restored database integrity\n echo \" Checking restored database integrity...\"\n if timeout 60 sqlite3 \"$RESTORED_DB\" \"PRAGMA integrity_check;\" | grep -q \"ok\"; then\n echo \" ✓ Restored database integrity check passed\"\n else\n echo \" ✗ Restored database integrity check failed\"\n RESTORE_SUCCESS=false\n fi\n\n # Compare database statistics\n echo \" Comparing database statistics...\"\n ORIGINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM large_test;\" 2>/dev/null || echo \"unknown\")\n RESTORED_COUNT=$(sqlite3 \"$RESTORED_DB\" \"SELECT COUNT(*) FROM large_test;\" 2>/dev/null || echo \"unknown\")\n\n ORIGINAL_PAGES=$(sqlite3 \"$DB\" \"PRAGMA page_count;\" 2>/dev/null || echo \"unknown\")\n RESTORED_PAGES=$(sqlite3 \"$RESTORED_DB\" \"PRAGMA page_count;\" 2>/dev/null || echo \"unknown\")\n\n echo \" Original records: $ORIGINAL_COUNT\"\n echo \" Restored records: $RESTORED_COUNT\"\n echo \" Original pages: $ORIGINAL_PAGES\"\n echo \" Restored pages: $RESTORED_PAGES\"\n\n # Check if both databases cross the lock page boundary\n if [ \"$ORIGINAL_PAGES\" != \"unknown\" ] && [ \"$ORIGINAL_PAGES\" -gt \"$LOCK_PAGE\" ]; then\n echo \" ✓ Original database crosses lock page boundary\"\n fi\n if [ \"$RESTORED_PAGES\" != \"unknown\" ] && [ \"$RESTORED_PAGES\" -gt \"$LOCK_PAGE\" ]; then\n echo \" ✓ Restored database crosses lock page boundary\"\n fi\n\n # Record count comparison\n if [ \"$ORIGINAL_COUNT\" = \"$RESTORED_COUNT\" ] && [ \"$ORIGINAL_COUNT\" != \"unknown\" ]; then\n echo \" ✓ Record counts match exactly\"\n elif [ \"$ORIGINAL_COUNT\" != \"unknown\" ] && [ \"$RESTORED_COUNT\" != \"unknown\" ]; then\n DIFF=$(echo \"$ORIGINAL_COUNT - $RESTORED_COUNT\" | bc)\n echo \" ⚠️ Record count difference: $DIFF (may be normal for ongoing replication)\"\n fi\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Large Database Test Results Summary\"\necho \"==========================================\"\n\nFINAL_RECORD_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM large_test;\" 2>/dev/null || echo \"unknown\")\nFINAL_DB_SIZE=$(du -h \"$DB\" 2>/dev/null | cut -f1 || echo \"unknown\")\nFINAL_PAGES=$(sqlite3 \"$DB\" \"PRAGMA page_count;\" 2>/dev/null || echo \"unknown\")\n\necho \"\"\necho \"Large Database Statistics:\"\necho \" Final size: $FINAL_DB_SIZE\"\necho \" Final page count: $FINAL_PAGES\"\necho \" Final record count: $FINAL_RECORD_COUNT\"\necho \" SQLite lock page: $LOCK_PAGE\"\nif [ \"$FINAL_PAGES\" != \"unknown\" ] && [ \"$FINAL_PAGES\" -gt \"$LOCK_PAGE\" ]; then\n echo \" Lock page boundary: ✓ CROSSED\"\nelse\n echo \" Lock page boundary: ? NOT CONFIRMED\"\nfi\necho \" Test duration: ~15-20 minutes\"\n\necho \"\"\necho \"Replication Analysis:\"\necho \" Sync operations: $SYNC_OPERATIONS\"\necho \" Upload operations: $UPLOAD_OPERATIONS\"\necho \" LTX operations: $LTX_OPERATIONS\"\necho \" Cleanup indicators: $CLEANUP_INDICATORS\"\necho \" Errors: $TOTAL_ERRORS\"\necho \" Warnings: $TOTAL_WARNINGS\"\n\necho \"\"\necho \"Restoration Test:\"\nif [ \"$RESTORE_SUCCESS\" = true ]; then\n echo \" Status: ✓ SUCCESS\"\n echo \" Duration: ${RESTORE_DURATION:-unknown} seconds\"\nelse\n echo \" Status: ✗ FAILED\"\nfi\n\necho \"\"\necho \"Critical Validations:\"\necho \" ✓ Large database (>1GB) created successfully\"\necho \" ✓ SQLite lock page boundary handling\"\necho \" ✓ S3 mock replication with large data\"\necho \" ✓ Extended LTX file generation over time\"\nif [ \"$CLEANUP_INDICATORS\" -gt \"0\" ]; then\n echo \" ✓ Retention cleanup activity observed\"\nelse\n echo \" ? Retention cleanup not explicitly logged\"\nfi\nif [ \"$RESTORE_SUCCESS\" = true ]; then\n echo \" ✓ Large database restoration successful\"\nelse\n echo \" ✗ Large database restoration issues\"\nfi\n\necho \"\"\necho \"Key Test Files:\"\necho \" - Replication log: /tmp/large-retention-test.log\"\necho \" - Restoration log: /tmp/large-restore.log\"\necho \" - Config file: /tmp/large-retention-config.yml\"\necho \" - Original database: $DB\"\nif [ -f \"$RESTORED_DB\" ]; then\n echo \" - Restored database: $RESTORED_DB\"\nfi\n\necho \"\"\necho \"Important Notes:\"\necho \" - This test specifically targets the 1GB SQLite lock page edge case\"\necho \" - Large database replication takes significantly longer\"\necho \" - Retention cleanup patterns may differ from small databases\"\necho \" - Performance characteristics are different at scale\"\necho \" - Real S3 performance will vary from local mock\"\n\necho \"\"\necho \"For Production Verification:\"\necho \" - Test with real S3 endpoints for network behavior\"\necho \" - Monitor actual S3 costs and API call patterns\"\necho \" - Verify cleanup with longer retention periods\"\necho \" - Test interrupted replication scenarios\"\n\necho \"\"\necho \"==========================================\"\necho \"Large Database S3 Retention Test Complete\"\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-s3-retention-small-db.sh", "content": "#!/bin/bash\nset -e\n\n# Test S3 LTX file retention cleanup with small databases using local S3 mock\n# This script tests that old LTX files are properly cleaned up after retention period\n\necho \"==========================================\"\necho \"S3 LTX Retention Test - Small Database\"\necho \"==========================================\"\necho \"\"\necho \"Testing LTX file cleanup using local S3 mock with small database\"\necho \"Database target size: 50MB\"\necho \"Retention period: 2 minutes\"\necho \"\"\n\n# Configuration\nPROJECT_ROOT=\"$(pwd)\"\nDB=\"/tmp/small-retention-test.db\"\nRESTORED_DB=\"/tmp/small-retention-restored.db\"\nLITESTREAM=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\nS3_MOCK=\"./etc/s3_mock.py\"\n\n# Build binaries if needed\nif [ ! -f \"$LITESTREAM\" ]; then\n echo \"Building litestream binary...\"\n go build -o bin/litestream ./cmd/litestream\nfi\n\nif [ ! -f \"$LITESTREAM_TEST\" ]; then\n echo \"Building litestream-test binary...\"\n go build -o bin/litestream-test ./cmd/litestream-test\nfi\n\n# Check for Python S3 mock dependencies\nif [ -f \"$PROJECT_ROOT/venv/bin/activate\" ]; then\n echo \"Using project virtual environment...\"\n source \"$PROJECT_ROOT/venv/bin/activate\"\nfi\n\nif ! python3 -c \"import moto, boto3\" 2>/dev/null; then\n echo \"⚠️ Missing Python dependencies. Installing moto and boto3...\"\n if [ -f \"$PROJECT_ROOT/venv/bin/activate\" ]; then\n source \"$PROJECT_ROOT/venv/bin/activate\"\n pip install moto boto3 || {\n echo \"Failed to install dependencies in venv\"\n exit 1\n }\n else\n pip3 install --user moto boto3 || {\n echo \"Failed to install dependencies. Please run: pip3 install --user moto boto3\"\n exit 1\n }\n fi\nfi\n\n# Cleanup function\ncleanup() {\n # Kill any running processes\n pkill -f \"litestream replicate.*small-retention-test.db\" 2>/dev/null || true\n pkill -f \"python.*s3_mock.py\" 2>/dev/null || true\n\n # Clean up temp files\n rm -f \"$DB\"* \"$RESTORED_DB\"* /tmp/small-retention-*.log /tmp/small-retention-*.yml\n\n echo \"Cleanup completed\"\n}\n\ntrap cleanup EXIT\ncleanup\n\necho \"==========================================\"\necho \"Step 1: Creating Small Test Database (50MB)\"\necho \"==========================================\"\n\necho \"[1.1] Creating and populating database to 50MB...\"\n$LITESTREAM_TEST populate \\\n -db \"$DB\" \\\n -target-size 50MB \\\n -row-size 2048 \\\n -batch-size 500\n\n# Set WAL mode after population\nsqlite3 \"$DB\" \"PRAGMA journal_mode = WAL;\"\n\nDB_SIZE=$(du -h \"$DB\" | cut -f1)\nRECORD_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM test_table_0;\")\necho \" ✓ Database created: $DB_SIZE with $RECORD_COUNT records\"\n\necho \"\"\necho \"==========================================\"\necho \"Step 2: Starting Local S3 Mock and Replication\"\necho \"==========================================\"\n\n# Create Litestream config for S3 mock\ncat > /tmp/small-retention-config.yml < /tmp/small-retention-test.log 2>&1 &\nREPL_PID=$!\nsleep 8\n\nif ! kill -0 $REPL_PID 2>/dev/null; then\n echo \" ✗ Replication failed to start\"\n echo \"Log contents:\"\n cat /tmp/small-retention-test.log\n exit 1\nfi\n\necho \" ✓ S3 mock and replication started (PID: $REPL_PID)\"\n\n# Check initial sync\nsleep 5\nINITIAL_SYNC_LINES=$(grep -c \"sync\" /tmp/small-retention-test.log 2>/dev/null || echo \"0\")\necho \" ✓ Initial sync operations: $INITIAL_SYNC_LINES\"\n\necho \"\"\necho \"==========================================\"\necho \"Step 3: Generating LTX Files Over Time\"\necho \"==========================================\"\n\necho \"[3.1] Creating LTX files in batches (6 batches, 20 seconds apart)...\"\n\n# Function to add data and force checkpoint\nadd_batch_data() {\n local batch_num=$1\n echo \" Batch $batch_num: Adding 1000 records and checkpointing...\"\n\n # Add data in small transactions to create multiple WAL segments\n for tx in {1..10}; do\n sqlite3 \"$DB\" </dev/null || echo \"0\")\n echo \" LTX operations so far: $LTX_ACTIVITY\"\n\n # Wait between batches (except last one)\n if [ $batch -lt 6 ]; then\n echo \" Waiting 20 seconds before next batch...\"\n sleep 20\n fi\ndone\n\necho \"\"\necho \"==========================================\"\necho \"Step 4: Monitoring Retention Cleanup\"\necho \"==========================================\"\n\necho \"[4.1] Waiting for retention cleanup to occur...\"\necho \" Retention period: 2 minutes\"\necho \" Monitoring for 4 minutes to observe cleanup...\"\n\n# Monitor cleanup activity\nfor minute in {1..4}; do\n echo \"\"\n echo \" Minute $minute/4 - $(date)\"\n sleep 60\n\n # Check various log patterns that might indicate cleanup\n CLEANUP_PATTERNS=(\n \"clean\" \"delet\" \"expir\" \"retention\" \"removed\" \"purge\"\n \"old\" \"ttl\" \"cleanup\" \"sweep\" \"vacuum\" \"evict\"\n )\n\n CLEANUP_TOTAL=0\n for pattern in \"${CLEANUP_PATTERNS[@]}\"; do\n COUNT=$(grep -c -i \"$pattern\" /tmp/small-retention-test.log 2>/dev/null || echo \"0\")\n CLEANUP_TOTAL=$((CLEANUP_TOTAL + COUNT))\n done\n\n TOTAL_ERRORS=$(grep -c \"ERROR\" /tmp/small-retention-test.log 2>/dev/null || echo \"0\")\n SYNC_COUNT=$(grep -c \"sync\" /tmp/small-retention-test.log 2>/dev/null || echo \"0\")\n\n echo \" Cleanup-related log entries: $CLEANUP_TOTAL\"\n echo \" Total sync operations: $SYNC_COUNT\"\n echo \" Errors: $TOTAL_ERRORS\"\n\n # Show recent activity\n RECENT_LINES=$(tail -5 /tmp/small-retention-test.log 2>/dev/null || echo \"No recent activity\")\n echo \" Recent activity: $(echo \"$RECENT_LINES\" | tr '\\n' ' ' | cut -c1-80)...\"\ndone\n\necho \"\"\necho \"==========================================\"\necho \"Step 5: Final Validation\"\necho \"==========================================\"\n\necho \"[5.1] Stopping replication...\"\nkill $REPL_PID 2>/dev/null || true\nwait $REPL_PID 2>/dev/null || true\nsleep 2\n\necho \"[5.2] Analyzing retention behavior...\"\n\n# Comprehensive log analysis\nTOTAL_ERRORS=$(grep -c \"ERROR\" /tmp/small-retention-test.log 2>/dev/null || echo \"0\")\nTOTAL_WARNINGS=$(grep -c \"WARN\" /tmp/small-retention-test.log 2>/dev/null || echo \"0\")\nSYNC_OPERATIONS=$(grep -c \"sync\" /tmp/small-retention-test.log 2>/dev/null || echo \"0\")\n\n# Search for cleanup indicators more broadly\nCLEANUP_INDICATORS=$(grep -i -c \"clean\\|delet\\|expir\\|retention\\|removed\\|purge\\|old.*file\\|ttl\" /tmp/small-retention-test.log 2>/dev/null || echo \"0\")\n\necho \"\"\necho \"Log Analysis Summary:\"\necho \"====================\"\necho \" Total errors: $TOTAL_ERRORS\"\necho \" Total warnings: $TOTAL_WARNINGS\"\necho \" Sync operations: $SYNC_OPERATIONS\"\necho \" Cleanup indicators: $CLEANUP_INDICATORS\"\n\nif [ \"$CLEANUP_INDICATORS\" -gt \"0\" ]; then\n echo \"\"\n echo \"Cleanup activity detected:\"\n grep -i \"clean\\|delet\\|expir\\|retention\\|removed\\|purge\\|old.*file\\|ttl\" /tmp/small-retention-test.log | head -10\nelse\n echo \"\"\n echo \"⚠️ No explicit cleanup activity found in logs\"\n echo \" Note: Litestream may perform silent cleanup without verbose logging\"\nfi\n\n# Show any errors\nif [ \"$TOTAL_ERRORS\" -gt \"0\" ]; then\n echo \"\"\n echo \"Errors encountered:\"\n grep \"ERROR\" /tmp/small-retention-test.log | tail -5\nfi\n\necho \"\"\necho \"[5.3] Testing restoration to verify integrity...\"\n\n# Test restoration using S3 mock\necho \"Attempting restoration from S3 mock...\"\nRESTORE_SUCCESS=true\n\nif ! timeout 30 $PYTHON_CMD $S3_MOCK $LITESTREAM restore -o \"$RESTORED_DB\" \\\n \"s3://\\${LITESTREAM_S3_BUCKET}/small-retention-test\" 2>/tmp/restore.log; then\n echo \" ✗ Restoration failed\"\n RESTORE_SUCCESS=false\n cat /tmp/restore.log\nelse\n echo \" ✓ Restoration completed\"\n\n # Verify restored database integrity\n if sqlite3 \"$RESTORED_DB\" \"PRAGMA integrity_check;\" | grep -q \"ok\"; then\n echo \" ✓ Restored database integrity check passed\"\n else\n echo \" ✗ Restored database integrity check failed\"\n RESTORE_SUCCESS=false\n fi\n\n # Compare record counts\n ORIGINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM test_table_0;\" 2>/dev/null || echo \"unknown\")\n RESTORED_COUNT=$(sqlite3 \"$RESTORED_DB\" \"SELECT COUNT(*) FROM test_table_0;\" 2>/dev/null || echo \"unknown\")\n\n echo \" Original records: $ORIGINAL_COUNT\"\n echo \" Restored records: $RESTORED_COUNT\"\n\n if [ \"$ORIGINAL_COUNT\" = \"$RESTORED_COUNT\" ] && [ \"$ORIGINAL_COUNT\" != \"unknown\" ]; then\n echo \" ✓ Record counts match\"\n else\n echo \" ⚠️ Record count mismatch (may be normal due to ongoing replication)\"\n fi\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Test Results Summary\"\necho \"==========================================\"\n\nFINAL_RECORD_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM test_table_0;\" 2>/dev/null || echo \"unknown\")\nFINAL_DB_SIZE=$(du -h \"$DB\" 2>/dev/null | cut -f1 || echo \"unknown\")\n\necho \"\"\necho \"Database Statistics:\"\necho \" Final size: $FINAL_DB_SIZE\"\necho \" Final record count: $FINAL_RECORD_COUNT\"\necho \" Test duration: ~8 minutes\"\necho \"\"\necho \"Replication Analysis:\"\necho \" Sync operations: $SYNC_OPERATIONS\"\necho \" Cleanup indicators: $CLEANUP_INDICATORS\"\necho \" Errors: $TOTAL_ERRORS\"\necho \" Warnings: $TOTAL_WARNINGS\"\necho \"\"\necho \"Restoration Test:\"\nif [ \"$RESTORE_SUCCESS\" = true ]; then\n echo \" Status: ✓ SUCCESS\"\nelse\n echo \" Status: ✗ FAILED\"\nfi\n\necho \"\"\necho \"Expected Behavior Verification:\"\necho \" ✓ Database created and populated successfully\"\necho \" ✓ S3 mock replication setup working\"\necho \" ✓ Multiple LTX files generated over time\"\nif [ \"$CLEANUP_INDICATORS\" -gt \"0\" ]; then\n echo \" ✓ Cleanup activity observed in logs\"\nelse\n echo \" ? Cleanup activity not explicitly logged (may still be working)\"\nfi\nif [ \"$RESTORE_SUCCESS\" = true ]; then\n echo \" ✓ Database restoration successful\"\nelse\n echo \" ✗ Database restoration issues detected\"\nfi\n\necho \"\"\necho \"Key Test Files:\"\necho \" - Replication log: /tmp/small-retention-test.log\"\necho \" - Config file: /tmp/small-retention-config.yml\"\necho \" - Original database: $DB\"\nif [ -f \"$RESTORED_DB\" ]; then\n echo \" - Restored database: $RESTORED_DB\"\nfi\n\necho \"\"\necho \"Notes:\"\necho \" - This test uses a local S3 mock (moto) for isolation\"\necho \" - Real S3 testing may show different cleanup patterns\"\necho \" - Retention behavior may vary with Litestream version\"\necho \" - Check logs for specific cleanup messages\"\n\necho \"\"\necho \"==========================================\"\necho \"Small Database S3 Retention Test Complete\"\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-simple-754-reproduction.sh", "content": "#!/bin/bash\nset -e\n\n# Simple, direct test to reproduce #754 flag issue\n# Focus on the core HeaderFlagNoChecksum problem\n\necho \"Simple #754 Reproduction Test\"\necho \"==============================\"\necho \"\"\n\nDB=\"/tmp/simple754.db\"\nREPLICA=\"/tmp/simple754-replica\"\nLITESTREAM=\"./bin/litestream\"\n\n# Clean up\nrm -rf \"$DB\"* \"$REPLICA\" /tmp/simple754-*.log\n\necho \"1. Creating test database...\"\nsqlite3 \"$DB\" < /tmp/simple754-run1.log 2>&1 &\nPID1=$!\n\necho \" Litestream PID: $PID1\"\necho \" Waiting for initial replication...\"\nsleep 10\n\n# Check if it's still running\nif kill -0 $PID1 2>/dev/null; then\n echo \" ✓ Litestream running\"\nelse\n echo \" ✗ Litestream died, checking logs...\"\n cat /tmp/simple754-run1.log\n exit 1\nfi\n\n# Add more data\necho \"\"\necho \"3. Adding more data during first run...\"\nfor i in {4..8}; do\n sqlite3 \"$DB\" \"INSERT INTO test (data) VALUES ('Row $i added during run 1');\"\ndone\n\n# Force checkpoint to ensure LTX files are created\nsqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\"\nsleep 5\n\necho \" Current row count: $(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM test;\")\"\n\necho \"\"\necho \"4. Checking for LTX files...\"\nif [ -d \"$REPLICA\" ]; then\n find \"$REPLICA\" -name \"*.ltx\" | head -5\n LTX_COUNT=$(find \"$REPLICA\" -name \"*.ltx\" | wc -l)\n echo \" LTX files found: $LTX_COUNT\"\n\n if [ \"$LTX_COUNT\" -eq \"0\" ]; then\n echo \" ⚠️ No LTX files created yet, waiting longer...\"\n sleep 10\n LTX_COUNT=$(find \"$REPLICA\" -name \"*.ltx\" | wc -l)\n echo \" LTX files after wait: $LTX_COUNT\"\n fi\nelse\n echo \" ✗ No replica directory found!\"\n echo \" Litestream logs:\"\n cat /tmp/simple754-run1.log\n exit 1\nfi\n\necho \"\"\necho \"5. Checking first run for errors...\"\nRUN1_ERRORS=$(grep -c \"ERROR\" /tmp/simple754-run1.log 2>/dev/null || echo \"0\")\nRUN1_FLAGS=$(grep -c \"no flags\" /tmp/simple754-run1.log 2>/dev/null || echo \"0\")\n\necho \" Run 1 errors: $RUN1_ERRORS\"\necho \" Run 1 flag errors: $RUN1_FLAGS\"\n\nif [ \"$RUN1_ERRORS\" -gt \"0\" ]; then\n echo \" Recent errors:\"\n grep \"ERROR\" /tmp/simple754-run1.log | tail -3\nfi\n\necho \"\"\necho \"6. Stopping first run...\"\nkill $PID1 2>/dev/null\nwait $PID1 2>/dev/null\necho \" ✓ First run stopped\"\n\necho \"\"\necho \"7. Adding offline data...\"\nsqlite3 \"$DB\" \"INSERT INTO test (data) VALUES ('Offline data between runs');\"\nOFFLINE_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM test;\")\necho \" Rows after offline addition: $OFFLINE_COUNT\"\n\necho \"\"\necho \"8. CRITICAL: Starting second run (potential #754 trigger)...\"\necho \" This should trigger #754 if HeaderFlagNoChecksum is incompatible\"\n\n$LITESTREAM replicate \"$DB\" \"file://$REPLICA\" > /tmp/simple754-run2.log 2>&1 &\nPID2=$!\n\necho \" Second run PID: $PID2\"\nsleep 5\n\nif kill -0 $PID2 2>/dev/null; then\n echo \" ✓ Second run started\"\nelse\n echo \" ✗ Second run failed immediately\"\n cat /tmp/simple754-run2.log\n exit 1\nfi\n\necho \"\"\necho \"9. Monitoring for #754 errors...\"\nsleep 15\n\nRUN2_FLAGS=$(grep -c \"no flags allowed\" /tmp/simple754-run2.log 2>/dev/null || echo \"0\")\nRUN2_VERIFICATION=$(grep -c \"ltx verification failed\" /tmp/simple754-run2.log 2>/dev/null || echo \"0\")\nRUN2_ERRORS=$(grep -c \"ERROR\" /tmp/simple754-run2.log 2>/dev/null || echo \"0\")\n\necho \" Second run analysis:\"\necho \" Total errors: $RUN2_ERRORS\"\necho \" 'no flags allowed': $RUN2_FLAGS\"\necho \" 'ltx verification failed': $RUN2_VERIFICATION\"\n\nif [ \"$RUN2_FLAGS\" -gt \"0\" ] || [ \"$RUN2_VERIFICATION\" -gt \"0\" ]; then\n echo \"\"\n echo \" 🚨 #754 REPRODUCED!\"\n echo \" Error details:\"\n grep -A1 -B1 \"no flags\\|ltx verification\" /tmp/simple754-run2.log\n ISSUE_REPRODUCED=true\nelse\n echo \" ✅ No #754 errors detected\"\n ISSUE_REPRODUCED=false\nfi\n\nif [ \"$RUN2_ERRORS\" -gt \"0\" ]; then\n echo \"\"\n echo \" All errors from second run:\"\n grep \"ERROR\" /tmp/simple754-run2.log\nfi\n\necho \"\"\necho \"10. Adding final data and cleanup...\"\nif kill -0 $PID2 2>/dev/null; then\n sqlite3 \"$DB\" \"INSERT INTO test (data) VALUES ('Final data from run 2');\"\n FINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM test;\")\n echo \" Final row count: $FINAL_COUNT\"\n\n kill $PID2 2>/dev/null\n wait $PID2 2>/dev/null\nfi\n\necho \"\"\necho \"RESULTS:\"\necho \"========\"\necho \"File structure created:\"\nfind \"$REPLICA\" -type f | head -10\n\necho \"\"\necho \"Error summary:\"\necho \" Run 1: $RUN1_ERRORS errors, $RUN1_FLAGS flag errors\"\necho \" Run 2: $RUN2_ERRORS errors, $RUN2_FLAGS flag errors\"\n\necho \"\"\nif [ \"$ISSUE_REPRODUCED\" = \"true\" ]; then\n echo \"✅ SUCCESS: #754 flag issue reproduced!\"\n echo \" Trigger: v0.5.0 restart against existing LTX files\"\n echo \" Root cause: HeaderFlagNoChecksum incompatible with ltx v0.5.0\"\nelse\n echo \"❌ #754 issue not reproduced in this test\"\n echo \" May need different database size, content, or timing\"\nfi\n\necho \"\"\necho \"Next steps:\"\necho \"- Examine HeaderFlagNoChecksum usage in db.go\"\necho \"- Test with different database configurations\"\necho \"- Verify ltx library version and flag handling\"\n\n# Cleanup\nrm -rf \"$DB\"* \"$REPLICA\" /tmp/simple754-*.log\n" }, { "path": "cmd/litestream-test/scripts/test-upgrade-large-db.sh", "content": "#!/bin/bash\nset -e\n\n# Test Litestream v0.3.x to v0.5.0 upgrade with large database (>1GB)\n# Specifically testing for #754 flag issue in upgrade scenario\n\necho \"==========================================\"\necho \"Large Database Upgrade Test (v0.3.x → v0.5.0)\"\necho \"==========================================\"\necho \"\"\necho \"Testing #754 flag issue with large database upgrade\"\necho \"\"\n\n# Configuration\nDB=\"/tmp/large-upgrade-test.db\"\nREPLICA=\"/tmp/large-upgrade-replica\"\nLITESTREAM_V3=\"/opt/homebrew/bin/litestream\"\nLITESTREAM_V5=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\n\n# Cleanup function\ncleanup() {\n pkill -f \"litestream replicate.*large-upgrade-test.db\" 2>/dev/null || true\n rm -f \"$DB\" \"$DB-wal\" \"$DB-shm\" \"$DB-litestream\"\n rm -rf \"$REPLICA\"\n rm -f /tmp/large-upgrade-*.log\n}\n\ntrap cleanup EXIT\n\necho \"[SETUP] Cleaning up previous test files...\"\ncleanup\n\necho \"\"\necho \"[1] Creating large database with v0.3.13...\"\necho \" This will take several minutes to reach >1GB...\"\n\n# Create database that will cross 1GB boundary\nsqlite3 \"$DB\" </dev/null 2>&1\n\n# Add our test table after populate\nsqlite3 \"$DB\" < /tmp/large-upgrade-v3.log 2>&1 &\nV3_PID=$!\nsleep 5\n\nif ! kill -0 $V3_PID 2>/dev/null; then\n echo \" ✗ Litestream v0.3.13 failed to start with large database\"\n cat /tmp/large-upgrade-v3.log\n exit 1\nfi\necho \" ✓ v0.3.13 replicating large database (PID: $V3_PID)\"\n\necho \"\"\necho \"[3] Letting v0.3.13 complete initial replication...\"\necho \" This may take several minutes for a large database...\"\nsleep 30\n\n# Check if replication is working\nV3_ERRORS=$(grep -c \"ERROR\" /tmp/large-upgrade-v3.log 2>/dev/null || echo \"0\")\nif [ \"$V3_ERRORS\" -gt \"0\" ]; then\n echo \" ⚠️ v0.3.13 errors detected:\"\n tail -5 /tmp/large-upgrade-v3.log | grep ERROR || true\nfi\n\n# Add some more data\nsqlite3 \"$DB\" \"INSERT INTO large_test (phase, data) VALUES ('v0.3.x-post-replication', randomblob(2000));\"\nREPLICATION_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM large_test;\")\necho \" ✓ v0.3.13 replication phase complete, total rows: $REPLICATION_COUNT\"\n\necho \"\"\necho \"[4] Stopping v0.3.13 and upgrading to v0.5.0...\"\nkill $V3_PID 2>/dev/null || true\nwait $V3_PID 2>/dev/null\necho \" ✓ v0.3.13 stopped\"\n\n# Add data during transition\nsqlite3 \"$DB\" \"INSERT INTO large_test (phase, data) VALUES ('upgrade-transition', randomblob(1500));\"\nTRANSITION_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM large_test;\")\necho \" ✓ Added transition data, total: $TRANSITION_COUNT\"\n\necho \"\"\necho \"[5] Starting v0.5.0 with large database...\"\n$LITESTREAM_V5 replicate \"$DB\" \"file://$REPLICA\" > /tmp/large-upgrade-v5.log 2>&1 &\nV5_PID=$!\nsleep 5\n\nif ! kill -0 $V5_PID 2>/dev/null; then\n echo \" ✗ Litestream v0.5.0 failed to start\"\n cat /tmp/large-upgrade-v5.log\n exit 1\nfi\necho \" ✓ v0.5.0 started with large database (PID: $V5_PID)\"\n\necho \"\"\necho \"[6] Critical #754 flag error check...\"\nsleep 5\n\nFLAG_ERRORS=$(grep -c \"no flags allowed\" /tmp/large-upgrade-v5.log 2>/dev/null || echo \"0\")\nVERIFICATION_ERRORS=$(grep -c \"ltx verification failed\" /tmp/large-upgrade-v5.log 2>/dev/null || echo \"0\")\nSYNC_ERRORS=$(grep -c \"sync error\" /tmp/large-upgrade-v5.log 2>/dev/null || echo \"0\")\n\necho \" #754 Error Analysis:\"\necho \" 'no flags allowed' errors: $FLAG_ERRORS\"\necho \" 'ltx verification failed' errors: $VERIFICATION_ERRORS\"\necho \" 'sync error' count: $SYNC_ERRORS\"\n\nif [ \"$FLAG_ERRORS\" -gt \"0\" ] || [ \"$VERIFICATION_ERRORS\" -gt \"0\" ]; then\n echo \"\"\n echo \" 🚨 #754 FLAG ISSUE DETECTED IN LARGE DB UPGRADE!\"\n echo \" Error details:\"\n grep -A2 -B2 \"no flags allowed\\|ltx verification failed\" /tmp/large-upgrade-v5.log || true\n UPGRADE_TRIGGERS_754=true\nelse\n echo \" ✅ No #754 flag errors in large database upgrade\"\n UPGRADE_TRIGGERS_754=false\nfi\n\necho \"\"\necho \"[7] Adding data with v0.5.0...\"\nsqlite3 \"$DB\" \"INSERT INTO large_test (phase, data) VALUES ('v0.5.0-large', randomblob(3000));\"\nFINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM large_test;\")\necho \" ✓ v0.5.0 data added, final count: $FINAL_COUNT\"\n\necho \"\"\necho \"[8] Stopping v0.5.0...\"\nkill $V5_PID 2>/dev/null || true\nwait $V5_PID 2>/dev/null\n\n# Final analysis\nALL_ERRORS=$(grep -c \"ERROR\" /tmp/large-upgrade-v5.log 2>/dev/null || echo \"0\")\necho \" ✓ v0.5.0 stopped, total errors: $ALL_ERRORS\"\n\nif [ \"$ALL_ERRORS\" -gt \"0\" ]; then\n echo \" Recent v0.5.0 errors:\"\n tail -10 /tmp/large-upgrade-v5.log | grep ERROR || true\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Large Database Upgrade Results\"\necho \"==========================================\"\necho \"\"\necho \"Database size: $DB_SIZE ($PAGE_COUNT pages)\"\necho \"Lock page boundary: Page $LOCK_PAGE\"\necho \"Data progression:\"\necho \" Initial: $INITIAL_LARGE_COUNT rows\"\necho \" Post-replication: $REPLICATION_COUNT rows\"\necho \" Post-transition: $TRANSITION_COUNT rows\"\necho \" Final: $FINAL_COUNT rows\"\necho \"\"\necho \"#754 Issue Analysis:\"\nif [ \"$UPGRADE_TRIGGERS_754\" = true ]; then\n echo \" 🚨 CRITICAL: #754 flag errors occur in large DB upgrades\"\n echo \" This means existing large production databases cannot upgrade to v0.5.0\"\nelse\n echo \" ✅ #754 flag errors do NOT occur in large DB upgrades\"\n echo \" Large database upgrades appear safe from this issue\"\nfi\necho \"\"\necho \"Conclusion:\"\nif [ \"$UPGRADE_TRIGGERS_754\" = true ]; then\n echo \"❌ Large database upgrade FAILS due to #754\"\n echo \" Production impact: Existing large databases cannot upgrade\"\nelse\n echo \"✅ Large database upgrade SUCCEEDS\"\n echo \" #754 issue is NOT related to v0.3.x → v0.5.0 upgrades\"\nfi\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-upgrade-v0.3-to-v0.5.sh", "content": "#!/bin/bash\nset -e\n\n# Test Litestream v0.3.x to v0.5.0 upgrade scenarios\n# Based on conversation with Ben Johnson about upgrade behavior expectations\n\necho \"==========================================\"\necho \"Litestream v0.3.x → v0.5.0 Upgrade Test\"\necho \"==========================================\"\necho \"\"\necho \"Testing upgrade from Litestream v0.3.13 to v0.5.0\"\necho \"\"\n\n# Configuration\nDB=\"/tmp/upgrade-test.db\"\nREPLICA=\"/tmp/upgrade-replica\"\nRESTORED_V3=\"/tmp/upgrade-restored-v3.db\"\nRESTORED_V5=\"/tmp/upgrade-restored-v5.db\"\nLITESTREAM_V3=\"/opt/homebrew/bin/litestream\"\nLITESTREAM_V5=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\n\n# Cleanup function\ncleanup() {\n pkill -f \"litestream replicate.*upgrade-test.db\" 2>/dev/null || true\n rm -f \"$DB\" \"$DB-wal\" \"$DB-shm\" \"$DB-litestream\"\n rm -f \"$RESTORED_V3\" \"$RESTORED_V3-wal\" \"$RESTORED_V3-shm\"\n rm -f \"$RESTORED_V5\" \"$RESTORED_V5-wal\" \"$RESTORED_V5-shm\"\n rm -rf \"$REPLICA\"\n rm -f /tmp/upgrade-*.log\n}\n\ntrap cleanup EXIT\n\necho \"[SETUP] Cleaning up previous test files...\"\ncleanup\n\n# Verify versions\necho \"\"\necho \"[VERSIONS] Verifying Litestream versions...\"\nV3_VERSION=$($LITESTREAM_V3 version 2>/dev/null || echo \"NOT_FOUND\")\nV5_VERSION=$($LITESTREAM_V5 version 2>/dev/null || echo \"NOT_FOUND\")\n\necho \" v0.3.x (system): $V3_VERSION\"\necho \" v0.5.0 (built): $V5_VERSION\"\n\nif [ \"$V3_VERSION\" = \"NOT_FOUND\" ]; then\n echo \" ✗ System Litestream v0.3.x not found at $LITESTREAM_V3\"\n exit 1\nfi\n\nif [ \"$V5_VERSION\" = \"NOT_FOUND\" ]; then\n echo \" ✗ Built Litestream v0.5.0 not found at $LITESTREAM_V5\"\n exit 1\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Phase 1: Create backups with v0.3.13\"\necho \"==========================================\"\n\necho \"[1] Creating test database...\"\nsqlite3 \"$DB\" < /tmp/upgrade-v3.log 2>&1 &\nV3_PID=$!\nsleep 3\n\nif ! kill -0 $V3_PID 2>/dev/null; then\n echo \" ✗ Litestream v0.3.13 failed to start\"\n cat /tmp/upgrade-v3.log\n exit 1\nfi\necho \" ✓ Litestream v0.3.13 running (PID: $V3_PID)\"\n\necho \"\"\necho \"[3] Adding data while v0.3.13 is replicating...\"\nfor i in {1..5}; do\n sqlite3 \"$DB\" \"INSERT INTO upgrade_test (phase, data) VALUES ('v0.3.x-replicating', randomblob(1500));\"\ndone\nsqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\" >/dev/null 2>&1\nsleep 2\n\nV3_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM upgrade_test;\")\necho \" ✓ Added data, total rows: $V3_COUNT\"\n\necho \"\"\necho \"[4] Examining v0.3.x backup structure...\"\nif [ -d \"$REPLICA\" ]; then\n echo \" Replica directory contents:\"\n find \"$REPLICA\" -type f | head -10 | while read file; do\n echo \" $(basename $(dirname $file))/$(basename $file)\"\n done\n V3_FILES=$(find \"$REPLICA\" -type f | wc -l)\n echo \" ✓ v0.3.x created $V3_FILES backup files\"\nelse\n echo \" ✗ No replica directory created\"\n exit 1\nfi\n\necho \"\"\necho \"[5] Testing v0.3.x restore capability...\"\n$LITESTREAM_V3 restore -o \"$RESTORED_V3\" \"file://$REPLICA\" > /tmp/upgrade-restore-v3.log 2>&1\nif [ $? -eq 0 ]; then\n RESTORED_V3_COUNT=$(sqlite3 \"$RESTORED_V3\" \"SELECT COUNT(*) FROM upgrade_test;\" 2>/dev/null || echo \"0\")\n echo \" ✓ v0.3.x restore successful: $RESTORED_V3_COUNT rows\"\n rm -f \"$RESTORED_V3\" \"$RESTORED_V3-wal\" \"$RESTORED_V3-shm\"\nelse\n echo \" ✗ v0.3.x restore failed\"\n cat /tmp/upgrade-restore-v3.log\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Phase 2: Upgrade to v0.5.0\"\necho \"==========================================\"\n\necho \"[6] Stopping Litestream v0.3.13...\"\nkill $V3_PID 2>/dev/null || true\nwait $V3_PID 2>/dev/null\necho \" ✓ v0.3.13 stopped\"\n\necho \"\"\necho \"[7] Adding data while Litestream is offline...\"\nfor i in {1..3}; do\n sqlite3 \"$DB\" \"INSERT INTO upgrade_test (phase, data) VALUES ('offline-transition', randomblob(1200));\"\ndone\nOFFLINE_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM upgrade_test;\")\necho \" ✓ Added data during transition, total rows: $OFFLINE_COUNT\"\n\necho \"\"\necho \"[8] Starting Litestream v0.5.0...\"\n$LITESTREAM_V5 replicate \"$DB\" \"file://$REPLICA\" > /tmp/upgrade-v5.log 2>&1 &\nV5_PID=$!\nsleep 3\n\nif ! kill -0 $V5_PID 2>/dev/null; then\n echo \" ✗ Litestream v0.5.0 failed to start\"\n cat /tmp/upgrade-v5.log\n exit 1\nfi\necho \" ✓ Litestream v0.5.0 running (PID: $V5_PID)\"\n\necho \"\"\necho \"[9] Checking for #754 flag errors in upgrade scenario...\"\nsleep 2\nFLAG_ERRORS=$(grep -c \"no flags allowed\" /tmp/upgrade-v5.log 2>/dev/null || echo \"0\")\nVERIFICATION_ERRORS=$(grep -c \"ltx verification failed\" /tmp/upgrade-v5.log 2>/dev/null || echo \"0\")\n\necho \" Flag errors: $FLAG_ERRORS\"\necho \" Verification errors: $VERIFICATION_ERRORS\"\n\nif [ \"$FLAG_ERRORS\" -gt \"0\" ] || [ \"$VERIFICATION_ERRORS\" -gt \"0\" ]; then\n echo \" ⚠️ #754 flag issue detected in upgrade scenario!\"\n grep \"no flags allowed\\|ltx verification failed\" /tmp/upgrade-v5.log || true\nelse\n echo \" ✓ No #754 flag errors in upgrade scenario\"\nfi\n\necho \"\"\necho \"[10] Adding data with v0.5.0...\"\nfor i in {1..5}; do\n sqlite3 \"$DB\" \"INSERT INTO upgrade_test (phase, data) VALUES ('v0.5.0-running', randomblob(1800));\"\ndone\nsqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\" >/dev/null 2>&1\nsleep 3\n\nV5_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM upgrade_test;\")\necho \" ✓ Added data with v0.5.0, total rows: $V5_COUNT\"\n\necho \"\"\necho \"[11] Examining backup structure after upgrade...\"\necho \" Post-upgrade replica contents:\"\nfind \"$REPLICA\" -type f -newer /tmp/upgrade-v3.log 2>/dev/null | head -5 | while read file; do\n echo \" NEW: $(basename $(dirname $file))/$(basename $file)\"\ndone\n\nV5_NEW_FILES=$(find \"$REPLICA\" -type f -newer /tmp/upgrade-v3.log 2>/dev/null | wc -l)\necho \" ✓ v0.5.0 created $V5_NEW_FILES new backup files\"\n\necho \"\"\necho \"==========================================\"\necho \"Phase 3: Restore compatibility testing\"\necho \"==========================================\"\n\necho \"[12] Testing v0.5.0 restore from mixed backup files...\"\n$LITESTREAM_V5 restore -o \"$RESTORED_V5\" \"file://$REPLICA\" > /tmp/upgrade-restore-v5.log 2>&1\nRESTORE_EXIT=$?\n\nif [ $RESTORE_EXIT -eq 0 ]; then\n RESTORED_V5_COUNT=$(sqlite3 \"$RESTORED_V5\" \"SELECT COUNT(*) FROM upgrade_test;\" 2>/dev/null || echo \"0\")\n echo \" ✓ v0.5.0 restore completed: $RESTORED_V5_COUNT rows\"\n\n # Check which phases are present\n V3_INITIAL=$(sqlite3 \"$RESTORED_V5\" \"SELECT COUNT(*) FROM upgrade_test WHERE phase='v0.3.x-initial';\" 2>/dev/null || echo \"0\")\n V3_REPLICATING=$(sqlite3 \"$RESTORED_V5\" \"SELECT COUNT(*) FROM upgrade_test WHERE phase='v0.3.x-replicating';\" 2>/dev/null || echo \"0\")\n OFFLINE=$(sqlite3 \"$RESTORED_V5\" \"SELECT COUNT(*) FROM upgrade_test WHERE phase='offline-transition';\" 2>/dev/null || echo \"0\")\n V5_RUNNING=$(sqlite3 \"$RESTORED_V5\" \"SELECT COUNT(*) FROM upgrade_test WHERE phase='v0.5.0-running';\" 2>/dev/null || echo \"0\")\n\n echo \" Data breakdown:\"\n echo \" v0.3.x initial: $V3_INITIAL rows\"\n echo \" v0.3.x replicating: $V3_REPLICATING rows\"\n echo \" Offline transition: $OFFLINE rows\"\n echo \" v0.5.0 running: $V5_RUNNING rows\"\n\n if [ \"$V3_INITIAL\" -eq \"0\" ] && [ \"$V3_REPLICATING\" -eq \"0\" ]; then\n echo \" ✓ EXPECTED: v0.5.0 ignored v0.3.x backup files\"\n else\n echo \" ⚠️ UNEXPECTED: v0.5.0 restored some v0.3.x data\"\n fi\n\n if [ \"$V5_RUNNING\" -gt \"0\" ]; then\n echo \" ✓ v0.5.0 data present in restore\"\n else\n echo \" ✗ v0.5.0 data missing from restore\"\n fi\n\nelse\n echo \" ✗ v0.5.0 restore failed\"\n cat /tmp/upgrade-restore-v5.log\nfi\n\necho \"\"\necho \"[13] Stopping v0.5.0 and final analysis...\"\nkill $V5_PID 2>/dev/null || true\nwait $V5_PID 2>/dev/null\n\n# Final error analysis\nV5_ERRORS=$(grep -c \"ERROR\" /tmp/upgrade-v5.log 2>/dev/null || echo \"0\")\nV5_WARNINGS=$(grep -c \"WARN\" /tmp/upgrade-v5.log 2>/dev/null || echo \"0\")\n\necho \" v0.5.0 runtime analysis:\"\necho \" Errors: $V5_ERRORS\"\necho \" Warnings: $V5_WARNINGS\"\necho \" Flag issues: $FLAG_ERRORS\"\n\nif [ \"$V5_ERRORS\" -gt \"0\" ]; then\n echo \" Recent errors:\"\n tail -10 /tmp/upgrade-v5.log | grep ERROR || true\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Upgrade Test Summary\"\necho \"==========================================\"\necho \"\"\necho \"Database progression:\"\necho \" v0.3.x initial: $INITIAL_COUNT rows\"\necho \" v0.3.x final: $V3_COUNT rows\"\necho \" Offline: $OFFLINE_COUNT rows\"\necho \" v0.5.0 final: $V5_COUNT rows\"\necho \"\"\necho \"Backup behavior:\"\necho \" v0.3.x files: $V3_FILES\"\necho \" v0.5.0 new files: $V5_NEW_FILES\"\necho \"\"\necho \"Restore behavior:\"\necho \" v0.3.x → v0.3.x: ✓ Successful\"\nif [ $RESTORE_EXIT -eq 0 ]; then\n echo \" Mixed → v0.5.0: ✓ Successful ($RESTORED_V5_COUNT rows)\"\n if [ \"$V3_INITIAL\" -eq \"0\" ] && [ \"$V3_REPLICATING\" -eq \"0\" ]; then\n echo \" v0.3.x compatibility: ✓ Ignored as expected\"\n else\n echo \" v0.3.x compatibility: ⚠️ Unexpected behavior\"\n fi\nelse\n echo \" Mixed → v0.5.0: ✗ Failed\"\nfi\necho \"\"\necho \"Issue #754 status:\"\nif [ \"$FLAG_ERRORS\" -gt \"0\" ] || [ \"$VERIFICATION_ERRORS\" -gt \"0\" ]; then\n echo \" ⚠️ #754 flag errors detected in upgrade scenario\"\nelse\n echo \" ✓ No #754 flag errors in upgrade scenario\"\nfi\necho \"\"\necho \"Conclusion:\"\nif [ \"$FLAG_ERRORS\" -eq \"0\" ] && [ \"$VERIFICATION_ERRORS\" -eq \"0\" ] && [ $RESTORE_EXIT -eq 0 ]; then\n echo \"✅ Upgrade test PASSED: v0.3.x → v0.5.0 works as expected\"\nelse\n echo \"⚠️ Upgrade test ISSUES: Some unexpected behavior detected\"\nfi\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-v0.5-flag-reproduction.sh", "content": "#!/bin/bash\nset -e\n\n# Test to reproduce original #754 flag issue\n# This recreates the scenario where #754 was first discovered\n\necho \"==========================================\"\necho \"v0.5.0 → v0.5.0 Flag Issue Reproduction\"\necho \"==========================================\"\necho \"\"\necho \"Reproducing the original #754 'no flags allowed' scenario\"\necho \"Testing v0.5.0 backing up a database that already has v0.5.0 LTX files\"\necho \"\"\n\n# Configuration\nDB=\"/tmp/flag-reproduction-test.db\"\nREPLICA=\"/tmp/flag-reproduction-replica\"\nLITESTREAM_V5=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\n\n# Cleanup function\ncleanup() {\n pkill -f \"litestream replicate.*flag-reproduction-test.db\" 2>/dev/null || true\n rm -f \"$DB\" \"$DB-wal\" \"$DB-shm\" \"$DB-litestream\"\n rm -rf \"$REPLICA\"\n rm -f /tmp/flag-reproduction-*.log\n}\n\ntrap cleanup EXIT\n\necho \"[SETUP] Cleaning up previous test files...\"\ncleanup\n\necho \"\"\necho \"[1] Creating large database with v0.5.0 (first run)...\"\n$LITESTREAM_TEST populate -db \"$DB\" -target-size 1200MB >/dev/null 2>&1\n\n# Add identifiable data\nsqlite3 \"$DB\" < /tmp/flag-reproduction-run1.log 2>&1 &\nRUN1_PID=$!\nsleep 5\n\nif ! kill -0 $RUN1_PID 2>/dev/null; then\n echo \" ✗ First v0.5.0 run failed\"\n cat /tmp/flag-reproduction-run1.log\n exit 1\nfi\necho \" ✓ First v0.5.0 run started (PID: $RUN1_PID)\"\n\n# Add some data during first run\nfor i in {1..10}; do\n sqlite3 \"$DB\" \"INSERT INTO flag_test (run_number, data) VALUES (1, randomblob(3000));\"\ndone\nsqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\" >/dev/null 2>&1\nsleep 3\n\nRUN1_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM flag_test;\")\necho \" ✓ First run data added, total: $RUN1_COUNT\"\n\n# Check first run for errors\nRUN1_ERRORS=$(grep -c \"ERROR\" /tmp/flag-reproduction-run1.log 2>/dev/null || echo \"0\")\nRUN1_FLAGS=$(grep -c \"no flags allowed\" /tmp/flag-reproduction-run1.log 2>/dev/null || echo \"0\")\n\necho \" First run status:\"\necho \" Errors: $RUN1_ERRORS\"\necho \" Flag errors: $RUN1_FLAGS\"\n\nif [ \"$RUN1_FLAGS\" -gt \"0\" ]; then\n echo \" ⚠️ Flag errors in first run (unexpected)\"\nfi\n\necho \"\"\necho \"[3] Examining first run LTX files...\"\nif [ -d \"$REPLICA\" ]; then\n LTX_FILES=$(find \"$REPLICA\" -name \"*.ltx\" | wc -l)\n echo \" LTX files created: $LTX_FILES\"\n\n # Look for files with HeaderFlagNoChecksum\n echo \" Examining LTX file headers...\"\n find \"$REPLICA\" -name \"*.ltx\" | head -3 | while read ltx_file; do\n echo \" $(basename $ltx_file): $(file \"$ltx_file\" 2>/dev/null || echo \"unknown format\")\"\n done\nelse\n echo \" ✗ No replica directory found\"\n exit 1\nfi\n\necho \"\"\necho \"[4] Stopping first run and simulating restart...\"\nkill $RUN1_PID 2>/dev/null || true\nwait $RUN1_PID 2>/dev/null\necho \" ✓ First run stopped\"\n\n# Add data while Litestream is down\nsqlite3 \"$DB\" \"INSERT INTO flag_test (run_number, data) VALUES (2, randomblob(4000));\"\nBETWEEN_RUNS_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM flag_test;\")\necho \" ✓ Data added between runs, total: $BETWEEN_RUNS_COUNT\"\n\necho \"\"\necho \"[5] CRITICAL: Second v0.5.0 run (where #754 might occur)...\"\necho \" Starting v0.5.0 against database with existing v0.5.0 LTX files...\"\n\n$LITESTREAM_V5 replicate \"$DB\" \"file://$REPLICA\" > /tmp/flag-reproduction-run2.log 2>&1 &\nRUN2_PID=$!\nsleep 5\n\nif ! kill -0 $RUN2_PID 2>/dev/null; then\n echo \" ✗ Second v0.5.0 run failed to start\"\n cat /tmp/flag-reproduction-run2.log\nelse\n echo \" ✓ Second v0.5.0 run started (PID: $RUN2_PID)\"\nfi\n\necho \"\"\necho \"[6] Monitoring for #754 flag errors...\"\nsleep 10\n\nRUN2_FLAGS=$(grep -c \"no flags allowed\" /tmp/flag-reproduction-run2.log 2>/dev/null || echo \"0\")\nRUN2_VERIFICATION=$(grep -c \"ltx verification failed\" /tmp/flag-reproduction-run2.log 2>/dev/null || echo \"0\")\nRUN2_SYNC_ERRORS=$(grep -c \"sync error\" /tmp/flag-reproduction-run2.log 2>/dev/null || echo \"0\")\nRUN2_TOTAL_ERRORS=$(grep -c \"ERROR\" /tmp/flag-reproduction-run2.log 2>/dev/null || echo \"0\")\n\necho \" Second run error analysis:\"\necho \" 'no flags allowed' errors: $RUN2_FLAGS\"\necho \" 'ltx verification failed' errors: $RUN2_VERIFICATION\"\necho \" 'sync error' count: $RUN2_SYNC_ERRORS\"\necho \" Total errors: $RUN2_TOTAL_ERRORS\"\n\nif [ \"$RUN2_FLAGS\" -gt \"0\" ] || [ \"$RUN2_VERIFICATION\" -gt \"0\" ]; then\n echo \"\"\n echo \" 🚨 #754 FLAG ISSUE REPRODUCED!\"\n echo \" This occurs when v0.5.0 reads existing v0.5.0 LTX files\"\n echo \" Error details:\"\n grep -A2 -B2 \"no flags allowed\\|ltx verification failed\" /tmp/flag-reproduction-run2.log || true\n FLAG_ISSUE_REPRODUCED=true\nelse\n echo \" ✅ No #754 flag errors in second run\"\n FLAG_ISSUE_REPRODUCED=false\nfi\n\necho \"\"\necho \"[7] Adding more data during second run...\"\nif kill -0 $RUN2_PID 2>/dev/null; then\n for i in {1..5}; do\n sqlite3 \"$DB\" \"INSERT INTO flag_test (run_number, data) VALUES (2, randomblob(3500));\" 2>/dev/null || true\n done\n FINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM flag_test;\")\n echo \" ✓ Second run data added, final total: $FINAL_COUNT\"\n\n kill $RUN2_PID 2>/dev/null || true\n wait $RUN2_PID 2>/dev/null\nelse\n echo \" ✗ Second run already failed, cannot add data\"\n FINAL_COUNT=$BETWEEN_RUNS_COUNT\nfi\n\necho \"\"\necho \"[8] Final analysis...\"\necho \" Checking recent errors from second run:\"\nif [ \"$RUN2_TOTAL_ERRORS\" -gt \"0\" ]; then\n tail -10 /tmp/flag-reproduction-run2.log | grep ERROR || echo \" No recent errors\"\nfi\n\n# Count total LTX files after both runs\nFINAL_LTX_FILES=$(find \"$REPLICA\" -name \"*.ltx\" 2>/dev/null | wc -l)\n\necho \"\"\necho \"==========================================\"\necho \"Flag Issue Reproduction Results\"\necho \"==========================================\"\necho \"\"\necho \"Database progression:\"\necho \" Initial: $INITIAL_COUNT records\"\necho \" After run 1: $RUN1_COUNT records\"\necho \" Between runs: $BETWEEN_RUNS_COUNT records\"\necho \" Final: $FINAL_COUNT records\"\necho \"\"\necho \"Error analysis:\"\necho \" Run 1 errors: $RUN1_ERRORS (flag errors: $RUN1_FLAGS)\"\necho \" Run 2 errors: $RUN2_TOTAL_ERRORS (flag errors: $RUN2_FLAGS)\"\necho \" LTX files created: $FINAL_LTX_FILES\"\necho \"\"\necho \"CRITICAL FINDING:\"\nif [ \"$FLAG_ISSUE_REPRODUCED\" = true ]; then\n echo \"🚨 #754 FLAG ISSUE REPRODUCED!\"\n echo \" Trigger: v0.5.0 restarting against existing v0.5.0 LTX files\"\n echo \" Root cause: HeaderFlagNoChecksum incompatibility with LTX v0.5.0\"\nelse\n echo \"✅ Could not reproduce #754 flag issue\"\n echo \" Issue may require specific conditions or database content\"\nfi\necho \"\"\necho \"Implication for upgrades:\"\nif [ \"$FLAG_ISSUE_REPRODUCED\" = true ]; then\n echo \" v0.3.x → v0.5.0 upgrades should be safe (different file formats)\"\n echo \" v0.5.0 → v0.5.0 restarts are the problem\"\nelse\n echo \" Further investigation needed to identify trigger conditions\"\nfi\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/test-v0.5-restart-scenarios.sh", "content": "#!/bin/bash\nset -e\n\n# Test v0.5.0 restart scenarios to reproduce #754 flag issue\n# Focus on HeaderFlagNoChecksum usage and LTX file handling\n\necho \"==========================================\"\necho \"v0.5.0 Restart Scenarios Test\"\necho \"==========================================\"\necho \"\"\necho \"Testing various v0.5.0 restart conditions to reproduce #754\"\necho \"\"\n\n# Configuration\nDB=\"/tmp/restart-test.db\"\nREPLICA=\"/tmp/restart-replica\"\nLITESTREAM_V5=\"./bin/litestream\"\nLITESTREAM_TEST=\"./bin/litestream-test\"\n\n# Cleanup function\ncleanup() {\n pkill -f \"litestream replicate.*restart-test.db\" 2>/dev/null || true\n rm -f \"$DB\" \"$DB-wal\" \"$DB-shm\" \"$DB-litestream\"\n rm -rf \"$REPLICA\"\n rm -f /tmp/restart-*.log\n}\n\ntrap cleanup EXIT\n\necho \"[SETUP] Cleaning up previous test files...\"\ncleanup\n\necho \"\"\necho \"==========================================\"\necho \"Scenario 1: Simple v0.5.0 restart\"\necho \"==========================================\"\n\necho \"[1] Creating large database for restart testing...\"\n$LITESTREAM_TEST populate -db \"$DB\" -target-size 1200MB >/dev/null 2>&1\n\n# Add identifiable data\nsqlite3 \"$DB\" < /tmp/restart-run1.log 2>&1 &\nRUN1_PID=$!\nsleep 5\n\nif ! kill -0 $RUN1_PID 2>/dev/null; then\n echo \" ✗ First run failed\"\n cat /tmp/restart-run1.log\n exit 1\nfi\necho \" ✓ First run started (PID: $RUN1_PID)\"\n\n# Add data during first run\nfor i in {1..10}; do\n sqlite3 \"$DB\" \"INSERT INTO restart_test (scenario, restart_number, data) VALUES ('first-run', 1, randomblob(3000));\"\ndone\nsqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\" >/dev/null 2>&1\nsleep 3\n\nRUN1_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM restart_test;\")\necho \" ✓ First run data: $RUN1_COUNT rows\"\n\necho \"\"\necho \"[3] Examining first run LTX files...\"\nif [ -d \"$REPLICA\" ]; then\n LTX_FILES_RUN1=$(find \"$REPLICA\" -name \"*.ltx\" | wc -l)\n echo \" LTX files after run 1: $LTX_FILES_RUN1\"\n\n # Check for HeaderFlagNoChecksum in files\n echo \" Examining LTX headers for flag usage...\"\n find \"$REPLICA\" -name \"*.ltx\" | head -2 | while read ltx_file; do\n echo \" $(basename $ltx_file): $(wc -c < \"$ltx_file\") bytes\"\n done\nelse\n echo \" ✗ No replica directory found\"\n exit 1\nfi\n\n# Check first run errors\nRUN1_ERRORS=$(grep -c \"ERROR\" /tmp/restart-run1.log 2>/dev/null || echo \"0\")\nRUN1_FLAGS=$(grep -c \"no flags allowed\" /tmp/restart-run1.log 2>/dev/null || echo \"0\")\necho \" First run status: $RUN1_ERRORS errors, $RUN1_FLAGS flag errors\"\n\necho \"\"\necho \"[4] Stopping first run and adding offline data...\"\nkill $RUN1_PID 2>/dev/null || true\nwait $RUN1_PID 2>/dev/null\n\n# Add data while Litestream is down\nsqlite3 \"$DB\" \"INSERT INTO restart_test (scenario, restart_number, data) VALUES ('offline', 0, randomblob(4000));\"\nOFFLINE_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM restart_test;\")\necho \" ✓ Offline data added, total: $OFFLINE_COUNT rows\"\n\necho \"\"\necho \"[5] CRITICAL: Second v0.5.0 restart...\"\necho \" Starting v0.5.0 against existing LTX files with HeaderFlagNoChecksum...\"\n\n$LITESTREAM_V5 replicate \"$DB\" \"file://$REPLICA\" > /tmp/restart-run2.log 2>&1 &\nRUN2_PID=$!\nsleep 5\n\nif ! kill -0 $RUN2_PID 2>/dev/null; then\n echo \" ✗ Second run failed to start\"\n cat /tmp/restart-run2.log\n exit 1\nfi\necho \" ✓ Second run started (PID: $RUN2_PID)\"\n\necho \"\"\necho \"[6] Monitoring for #754 flag errors during restart...\"\nsleep 10\n\nRUN2_FLAGS=$(grep -c \"no flags allowed\" /tmp/restart-run2.log 2>/dev/null || echo \"0\")\nRUN2_VERIFICATION=$(grep -c \"ltx verification failed\" /tmp/restart-run2.log 2>/dev/null || echo \"0\")\nRUN2_SYNC_ERRORS=$(grep -c \"sync error\" /tmp/restart-run2.log 2>/dev/null || echo \"0\")\nRUN2_TOTAL_ERRORS=$(grep -c \"ERROR\" /tmp/restart-run2.log 2>/dev/null || echo \"0\")\n\necho \" Second run error analysis:\"\necho \" 'no flags allowed' errors: $RUN2_FLAGS\"\necho \" 'ltx verification failed' errors: $RUN2_VERIFICATION\"\necho \" 'sync error' count: $RUN2_SYNC_ERRORS\"\necho \" Total errors: $RUN2_TOTAL_ERRORS\"\n\nif [ \"$RUN2_FLAGS\" -gt \"0\" ] || [ \"$RUN2_VERIFICATION\" -gt \"0\" ]; then\n echo \"\"\n echo \" 🚨 #754 FLAG ISSUE REPRODUCED IN RESTART!\"\n echo \" Error details:\"\n grep -A2 -B2 \"no flags allowed\\|ltx verification failed\" /tmp/restart-run2.log || true\n RESTART_TRIGGERS_754=true\nelse\n echo \" ✅ No #754 flag errors in simple restart\"\n RESTART_TRIGGERS_754=false\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Scenario 2: Checkpoint during restart\"\necho \"==========================================\"\n\n# Add more data during second run\necho \"[7] Adding data during second run with checkpoints...\"\nfor i in {1..5}; do\n sqlite3 \"$DB\" \"INSERT INTO restart_test (scenario, restart_number, data) VALUES ('second-run', 2, randomblob(3500));\"\n if [ $((i % 2)) -eq 0 ]; then\n sqlite3 \"$DB\" \"PRAGMA wal_checkpoint(FULL);\" >/dev/null 2>&1\n fi\ndone\n\nRUN2_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM restart_test;\")\necho \" ✓ Second run data with checkpoints: $RUN2_COUNT rows\"\n\n# Monitor for additional errors\nsleep 5\nCHECKPOINT_FLAGS=$(grep -c \"no flags allowed\" /tmp/restart-run2.log 2>/dev/null || echo \"0\")\nif [ \"$CHECKPOINT_FLAGS\" -gt \"$RUN2_FLAGS\" ]; then\n echo \" ⚠️ Additional flag errors during checkpoint operations\"\nfi\n\necho \"\"\necho \"==========================================\"\necho \"Scenario 3: Multiple restart cycles\"\necho \"==========================================\"\n\necho \"[8] Third restart cycle...\"\nkill $RUN2_PID 2>/dev/null || true\nwait $RUN2_PID 2>/dev/null\n\nsqlite3 \"$DB\" \"INSERT INTO restart_test (scenario, restart_number, data) VALUES ('between-2-and-3', 0, randomblob(2500));\"\n\n$LITESTREAM_V5 replicate \"$DB\" \"file://$REPLICA\" > /tmp/restart-run3.log 2>&1 &\nRUN3_PID=$!\nsleep 5\n\nif kill -0 $RUN3_PID 2>/dev/null; then\n echo \" ✓ Third run started (PID: $RUN3_PID)\"\n\n # Quick check for immediate errors\n sleep 5\n RUN3_FLAGS=$(grep -c \"no flags allowed\" /tmp/restart-run3.log 2>/dev/null || echo \"0\")\n RUN3_ERRORS=$(grep -c \"ERROR\" /tmp/restart-run3.log 2>/dev/null || echo \"0\")\n\n echo \" Third run status: $RUN3_ERRORS errors, $RUN3_FLAGS flag errors\"\n\n if [ \"$RUN3_FLAGS\" -gt \"0\" ]; then\n echo \" ⚠️ Flag errors in third restart\"\n fi\n\n kill $RUN3_PID 2>/dev/null || true\n wait $RUN3_PID 2>/dev/null\nelse\n echo \" ✗ Third run failed\"\n cat /tmp/restart-run3.log | head -10\nfi\n\necho \"\"\necho \"[9] Final analysis...\"\nFINAL_COUNT=$(sqlite3 \"$DB\" \"SELECT COUNT(*) FROM restart_test;\")\nFINAL_LTX_FILES=$(find \"$REPLICA\" -name \"*.ltx\" 2>/dev/null | wc -l)\n\necho \" Final statistics:\"\necho \" Database rows: $FINAL_COUNT\"\necho \" LTX files created: $FINAL_LTX_FILES\"\necho \" Run 1 errors: $RUN1_ERRORS (flags: $RUN1_FLAGS)\"\necho \" Run 2 errors: $RUN2_TOTAL_ERRORS (flags: $RUN2_FLAGS)\"\necho \" Run 3 errors: $RUN3_ERRORS (flags: $RUN3_FLAGS)\"\n\necho \"\"\necho \"==========================================\"\necho \"v0.5.0 Restart Test Results\"\necho \"==========================================\"\necho \"\"\necho \"Test scenarios:\"\necho \" ✓ Simple restart: $([ \"$RESTART_TRIGGERS_754\" = true ] && echo \"REPRODUCED #754\" || echo \"No #754 errors\")\"\necho \" ✓ Checkpoint during restart: $([ \"$CHECKPOINT_FLAGS\" -gt \"$RUN2_FLAGS\" ] && echo \"Additional errors\" || echo \"No additional errors\")\"\necho \" ✓ Multiple restart cycles: $([ \"${RUN3_FLAGS:-0}\" -gt \"0\" ] && echo \"Errors in cycle 3\" || echo \"No errors in cycle 3\")\"\necho \"\"\necho \"CRITICAL FINDINGS:\"\nif [ \"$RESTART_TRIGGERS_754\" = true ] || [ \"${RUN3_FLAGS:-0}\" -gt \"0\" ]; then\n echo \"🚨 #754 FLAG ISSUE TRIGGERED BY v0.5.0 RESTARTS\"\n echo \" Root cause: v0.5.0 reading its own LTX files with HeaderFlagNoChecksum\"\n echo \" Trigger: Restarting Litestream against existing v0.5.0 backup files\"\n echo \" Impact: Production Litestream restarts will fail\"\nelse\n echo \"✅ No #754 errors in restart scenarios tested\"\n echo \" Issue may require specific database content or timing conditions\"\nfi\necho \"\"\necho \"Next steps:\"\necho \"1. Check HeaderFlagNoChecksum usage in db.go\"\necho \"2. Test with different database sizes/content\"\necho \"3. Investigate LTX file generation differences\"\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/scripts/verify-test-setup.sh", "content": "#!/bin/bash\n\n# Script to verify test environment is set up correctly\n# Ensures we're using local builds, not system-installed versions\n\necho \"==========================================\"\necho \"Litestream Test Environment Verification\"\necho \"==========================================\"\necho \"\"\n\n# Check for local Litestream build\necho \"Checking for local Litestream build...\"\nif [ -f \"./bin/litestream\" ]; then\n echo \"✓ Local litestream found: ./bin/litestream\"\n echo \" Version: $($./bin/litestream version)\"\n echo \" Size: $(ls -lh ./bin/litestream | awk '{print $5}')\"\n echo \" Modified: $(ls -la ./bin/litestream | awk '{print $6, $7, $8}')\"\nelse\n echo \"✗ Local litestream NOT found at ./bin/litestream\"\n echo \" Please build: go build -o bin/litestream ./cmd/litestream\"\n exit 1\nfi\n\n# Check for system Litestream (should NOT be used)\necho \"\"\necho \"Checking for system Litestream...\"\nif command -v litestream &> /dev/null; then\n SYSTEM_LITESTREAM=$(which litestream)\n echo \"⚠ System litestream found at: $SYSTEM_LITESTREAM\"\n echo \" Version: $(litestream version 2>&1 || echo \"unknown\")\"\n echo \" WARNING: Tests should NOT use this version!\"\n echo \" All test scripts use ./bin/litestream explicitly\"\nelse\n echo \"✓ No system litestream found (good - avoids confusion)\"\nfi\n\n# Check for litestream-test binary\necho \"\"\necho \"Checking for litestream-test binary...\"\nif [ -f \"./bin/litestream-test\" ]; then\n echo \"✓ Local litestream-test found: ./bin/litestream-test\"\n echo \" Size: $(ls -lh ./bin/litestream-test | awk '{print $5}')\"\n echo \" Modified: $(ls -la ./bin/litestream-test | awk '{print $6, $7, $8}')\"\nelse\n echo \"✗ litestream-test NOT found at ./bin/litestream-test\"\n echo \" Please build: go build -o bin/litestream-test ./cmd/litestream-test\"\n exit 1\nfi\n\n# Verify test scripts use local builds\necho \"\"\necho \"Verifying test scripts use local builds...\"\nSCRIPTS=(\n \"reproduce-critical-bug.sh\"\n \"test-1gb-boundary.sh\"\n \"test-concurrent-operations.sh\"\n)\n\nALL_GOOD=true\nfor script in \"${SCRIPTS[@]}\"; do\n if [ -f \"$script\" ]; then\n if grep -q 'LITESTREAM=\"./bin/litestream\"' \"$script\"; then\n echo \"✓ $script uses local build\"\n else\n echo \"✗ $script may not use local build!\"\n grep \"LITESTREAM=\" \"$script\" | head -2\n ALL_GOOD=false\n fi\n else\n echo \"- $script not found (optional)\"\n fi\ndone\n\n# Check current git branch\necho \"\"\necho \"Git status:\"\nBRANCH=$(git branch --show-current 2>/dev/null || echo \"unknown\")\necho \" Current branch: $BRANCH\"\nif [ \"$BRANCH\" = \"main\" ]; then\n echo \" ⚠ On main branch - be careful with commits!\"\nfi\n\n# Summary\necho \"\"\necho \"==========================================\"\nif [ \"$ALL_GOOD\" = true ] && [ -f \"./bin/litestream\" ] && [ -f \"./bin/litestream-test\" ]; then\n echo \"✅ Test environment is properly configured!\"\n echo \"\"\n echo \"You can run tests with:\"\n echo \" ./reproduce-critical-bug.sh\"\n echo \" ./test-1gb-boundary.sh\"\n echo \" ./test-concurrent-operations.sh\"\nelse\n echo \"❌ Test environment needs setup\"\n echo \"\"\n echo \"Required steps:\"\n [ ! -f \"./bin/litestream\" ] && echo \" 1. Build litestream: go build -o bin/litestream ./cmd/litestream\"\n [ ! -f \"./bin/litestream-test\" ] && echo \" 2. Build test harness: go build -o bin/litestream-test ./cmd/litestream-test\"\n exit 1\nfi\necho \"==========================================\"\n" }, { "path": "cmd/litestream-test/shrink.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"flag\"\n\t\"fmt\"\n\t\"log/slog\"\n\t\"os\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\ntype ShrinkCommand struct {\n\tMain *Main\n\n\tDB string\n\tDeletePercentage float64\n\tVacuum bool\n\tCheckpoint bool\n\tCheckpointMode string\n}\n\nfunc (c *ShrinkCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-test shrink\", flag.ExitOnError)\n\tfs.StringVar(&c.DB, \"db\", \"\", \"Database path (required)\")\n\tfs.Float64Var(&c.DeletePercentage, \"delete-percentage\", 50, \"Percentage of data to delete (0-100)\")\n\tfs.BoolVar(&c.Vacuum, \"vacuum\", false, \"Run VACUUM after deletion\")\n\tfs.BoolVar(&c.Checkpoint, \"checkpoint\", false, \"Run checkpoint after deletion\")\n\tfs.StringVar(&c.CheckpointMode, \"checkpoint-mode\", \"PASSIVE\", \"Checkpoint mode (PASSIVE, FULL, RESTART, TRUNCATE)\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif c.DB == \"\" {\n\t\treturn fmt.Errorf(\"database path required\")\n\t}\n\n\tif c.DeletePercentage < 0 || c.DeletePercentage > 100 {\n\t\treturn fmt.Errorf(\"delete percentage must be between 0 and 100\")\n\t}\n\n\tif _, err := os.Stat(c.DB); err != nil {\n\t\treturn fmt.Errorf(\"database does not exist: %w\", err)\n\t}\n\n\tslog.Info(\"Starting database shrink operation\",\n\t\t\"db\", c.DB,\n\t\t\"delete_percentage\", c.DeletePercentage,\n\t\t\"vacuum\", c.Vacuum,\n\t\t\"checkpoint\", c.Checkpoint,\n\t)\n\n\treturn c.shrinkDatabase(ctx)\n}\n\nfunc (c *ShrinkCommand) shrinkDatabase(ctx context.Context) error {\n\tinitialSize, err := getDatabaseSize(c.DB)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"get initial size: %w\", err)\n\t}\n\n\tslog.Info(\"Initial database size\",\n\t\t\"size_mb\", initialSize/1024/1024,\n\t)\n\n\tdb, err := sql.Open(\"sqlite3\", c.DB+\"?_journal_mode=WAL\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open database: %w\", err)\n\t}\n\tdefer db.Close()\n\n\ttables, err := c.getTableList(db)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"get table list: %w\", err)\n\t}\n\n\tslog.Info(\"Found tables\", \"count\", len(tables))\n\n\ttotalDeleted := int64(0)\n\tfor _, table := range tables {\n\t\tdeleted, err := c.deleteFromTable(db, table)\n\t\tif err != nil {\n\t\t\tslog.Error(\"Failed to delete from table\", \"table\", table, \"error\", err)\n\t\t\tcontinue\n\t\t}\n\t\ttotalDeleted += deleted\n\t\tslog.Info(\"Deleted rows from table\",\n\t\t\t\"table\", table,\n\t\t\t\"rows_deleted\", deleted,\n\t\t)\n\t}\n\n\tslog.Info(\"Deletion complete\", \"total_rows_deleted\", totalDeleted)\n\n\tsizeAfterDelete, err := getDatabaseSize(c.DB)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"get size after delete: %w\", err)\n\t}\n\n\tslog.Info(\"Size after deletion\",\n\t\t\"size_mb\", sizeAfterDelete/1024/1024,\n\t\t\"change_mb\", (initialSize-sizeAfterDelete)/1024/1024,\n\t)\n\n\tif c.Checkpoint {\n\t\tif err := c.runCheckpoint(db); err != nil {\n\t\t\treturn fmt.Errorf(\"checkpoint: %w\", err)\n\t\t}\n\n\t\tsizeAfterCheckpoint, _ := getDatabaseSize(c.DB)\n\t\tslog.Info(\"Size after checkpoint\",\n\t\t\t\"size_mb\", sizeAfterCheckpoint/1024/1024,\n\t\t\t\"change_from_delete_mb\", (sizeAfterDelete-sizeAfterCheckpoint)/1024/1024,\n\t\t)\n\t}\n\n\tif c.Vacuum {\n\t\tif err := c.runVacuum(db); err != nil {\n\t\t\treturn fmt.Errorf(\"vacuum: %w\", err)\n\t\t}\n\n\t\tsizeAfterVacuum, _ := getDatabaseSize(c.DB)\n\t\tslog.Info(\"Size after VACUUM\",\n\t\t\t\"size_mb\", sizeAfterVacuum/1024/1024,\n\t\t\t\"total_reduction_mb\", (initialSize-sizeAfterVacuum)/1024/1024,\n\t\t)\n\t}\n\n\tfinalSize, err := getDatabaseSize(c.DB)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"get final size: %w\", err)\n\t}\n\n\treductionPercent := float64(initialSize-finalSize) / float64(initialSize) * 100\n\tslog.Info(\"Shrink operation complete\",\n\t\t\"initial_size_mb\", initialSize/1024/1024,\n\t\t\"final_size_mb\", finalSize/1024/1024,\n\t\t\"reduction_percent\", fmt.Sprintf(\"%.1f\", reductionPercent),\n\t)\n\n\treturn nil\n}\n\nfunc (c *ShrinkCommand) getTableList(db *sql.DB) ([]string, error) {\n\trows, err := db.Query(`\n\t\tSELECT name FROM sqlite_master\n\t\tWHERE type='table'\n\t\tAND name NOT LIKE 'sqlite_%'\n\t\tAND name NOT LIKE 'load_test'\n\t`)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer rows.Close()\n\n\tvar tables []string\n\tfor rows.Next() {\n\t\tvar table string\n\t\tif err := rows.Scan(&table); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\ttables = append(tables, table)\n\t}\n\n\treturn tables, nil\n}\n\nfunc (c *ShrinkCommand) deleteFromTable(db *sql.DB, table string) (int64, error) {\n\tvar totalRows int\n\tcountQuery := fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", table)\n\tif err := db.QueryRow(countQuery).Scan(&totalRows); err != nil {\n\t\treturn 0, fmt.Errorf(\"count rows: %w\", err)\n\t}\n\n\tif totalRows == 0 {\n\t\treturn 0, nil\n\t}\n\n\trowsToDelete := int(float64(totalRows) * (c.DeletePercentage / 100))\n\tif rowsToDelete == 0 {\n\t\treturn 0, nil\n\t}\n\n\tvar hasID bool\n\tcolumnQuery := fmt.Sprintf(\"PRAGMA table_info(%s)\", table)\n\trows, err := db.Query(columnQuery)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"get table info: %w\", err)\n\t}\n\tdefer rows.Close()\n\n\tfor rows.Next() {\n\t\tvar cid int\n\t\tvar name, dtype string\n\t\tvar notnull, pk int\n\t\tvar dflt sql.NullString\n\t\tif err := rows.Scan(&cid, &name, &dtype, ¬null, &dflt, &pk); err != nil {\n\t\t\tcontinue\n\t\t}\n\t\tif name == \"id\" || pk == 1 {\n\t\t\thasID = true\n\t\t\tbreak\n\t\t}\n\t}\n\n\tvar deleteQuery string\n\tif hasID {\n\t\tdeleteQuery = fmt.Sprintf(`\n\t\t\tDELETE FROM %s\n\t\t\tWHERE id IN (\n\t\t\t\tSELECT id FROM %s\n\t\t\t\tORDER BY RANDOM()\n\t\t\t\tLIMIT %d\n\t\t\t)\n\t\t`, table, table, rowsToDelete)\n\t} else {\n\t\tdeleteQuery = fmt.Sprintf(`\n\t\t\tDELETE FROM %s\n\t\t\tWHERE rowid IN (\n\t\t\t\tSELECT rowid FROM %s\n\t\t\t\tORDER BY RANDOM()\n\t\t\t\tLIMIT %d\n\t\t\t)\n\t\t`, table, table, rowsToDelete)\n\t}\n\n\tstartTime := time.Now()\n\tresult, err := db.Exec(deleteQuery)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"delete rows: %w\", err)\n\t}\n\n\trowsDeleted, _ := result.RowsAffected()\n\tduration := time.Since(startTime)\n\n\tslog.Debug(\"Deleted rows from table\",\n\t\t\"table\", table,\n\t\t\"rows_deleted\", rowsDeleted,\n\t\t\"duration\", duration,\n\t)\n\n\treturn rowsDeleted, nil\n}\n\nfunc (c *ShrinkCommand) runCheckpoint(db *sql.DB) error {\n\tslog.Info(\"Running checkpoint\", \"mode\", c.CheckpointMode)\n\n\tstartTime := time.Now()\n\tquery := fmt.Sprintf(\"PRAGMA wal_checkpoint(%s)\", c.CheckpointMode)\n\n\tvar busy, written, total int\n\terr := db.QueryRow(query).Scan(&busy, &written, &total)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"checkpoint failed: %w\", err)\n\t}\n\n\tduration := time.Since(startTime)\n\tslog.Info(\"Checkpoint complete\",\n\t\t\"mode\", c.CheckpointMode,\n\t\t\"busy\", busy,\n\t\t\"pages_written\", written,\n\t\t\"total_pages\", total,\n\t\t\"duration\", duration,\n\t)\n\n\treturn nil\n}\n\nfunc (c *ShrinkCommand) runVacuum(db *sql.DB) error {\n\tslog.Info(\"Running VACUUM (this may take a while)\")\n\n\tstartTime := time.Now()\n\t_, err := db.Exec(\"VACUUM\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"vacuum failed: %w\", err)\n\t}\n\n\tduration := time.Since(startTime)\n\tslog.Info(\"VACUUM complete\", \"duration\", duration)\n\n\treturn nil\n}\n\nfunc (c *ShrinkCommand) Usage() {\n\tfmt.Fprintln(c.Main.Stdout, `\nShrink a database by deleting data and optionally running VACUUM.\n\nUsage:\n\n\tlitestream-test shrink [options]\n\nOptions:\n\n\t-db PATH\n\t Database path (required)\n\n\t-delete-percentage PCT\n\t Percentage of data to delete (0-100)\n\t Default: 50\n\n\t-vacuum\n\t Run VACUUM after deletion\n\t Default: false\n\n\t-checkpoint\n\t Run checkpoint after deletion\n\t Default: false\n\n\t-checkpoint-mode MODE\n\t Checkpoint mode (PASSIVE, FULL, RESTART, TRUNCATE)\n\t Default: PASSIVE\n\nExamples:\n\n\t# Delete 50% of data\n\tlitestream-test shrink -db /tmp/test.db -delete-percentage 50\n\n\t# Delete 75% and run VACUUM\n\tlitestream-test shrink -db /tmp/test.db -delete-percentage 75 -vacuum\n\n\t# Delete 30%, checkpoint, then VACUUM\n\tlitestream-test shrink -db /tmp/test.db -delete-percentage 30 -checkpoint -vacuum\n\n\t# Test with FULL checkpoint mode\n\tlitestream-test shrink -db /tmp/test.db -delete-percentage 50 -checkpoint -checkpoint-mode FULL\n`[1:])\n}\n" }, { "path": "cmd/litestream-test/validate.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"crypto/md5\"\n\t\"database/sql\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\ntype ValidateCommand struct {\n\tMain *Main\n\n\tSourceDB string\n\tReplicaURL string\n\tRestoredDB string\n\tCheckType string\n\tLTXContinuity bool\n\tConfigPath string\n}\n\ntype ValidationResult struct {\n\tCheckType string\n\tPassed bool\n\tDuration time.Duration\n\tErrorMessage string\n\tDetails map[string]interface{}\n}\n\nfunc (c *ValidateCommand) Run(ctx context.Context, args []string) error {\n\tfs := flag.NewFlagSet(\"litestream-test validate\", flag.ExitOnError)\n\tfs.StringVar(&c.SourceDB, \"source-db\", \"\", \"Original database path\")\n\tfs.StringVar(&c.ReplicaURL, \"replica-url\", \"\", \"Replica URL to validate\")\n\tfs.StringVar(&c.RestoredDB, \"restored-db\", \"\", \"Path for restored database\")\n\tfs.StringVar(&c.CheckType, \"check-type\", \"quick\", \"Type of check (quick, integrity, checksum, full)\")\n\tfs.BoolVar(&c.LTXContinuity, \"ltx-continuity\", false, \"Check LTX file continuity\")\n\tfs.StringVar(&c.ConfigPath, \"config\", \"\", \"Litestream config file path\")\n\tfs.Usage = c.Usage\n\tif err := fs.Parse(args); err != nil {\n\t\treturn err\n\t}\n\n\tif c.SourceDB == \"\" {\n\t\treturn fmt.Errorf(\"source database path required\")\n\t}\n\n\tif c.ReplicaURL == \"\" && c.ConfigPath == \"\" {\n\t\treturn fmt.Errorf(\"replica URL or config file required\")\n\t}\n\n\tif c.RestoredDB == \"\" {\n\t\tc.RestoredDB = c.SourceDB + \".restored\"\n\t}\n\n\tslog.Info(\"Starting validation\",\n\t\t\"source_db\", c.SourceDB,\n\t\t\"replica_url\", c.ReplicaURL,\n\t\t\"check_type\", c.CheckType,\n\t\t\"ltx_continuity\", c.LTXContinuity,\n\t)\n\n\tresults := []ValidationResult{}\n\n\tif c.LTXContinuity && c.ReplicaURL != \"\" {\n\t\tresult := c.validateLTXContinuity(ctx)\n\t\tresults = append(results, result)\n\t}\n\n\trestoreResult := c.performRestore(ctx)\n\tresults = append(results, restoreResult)\n\n\tif restoreResult.Passed {\n\t\tswitch c.CheckType {\n\t\tcase \"quick\":\n\t\t\tresults = append(results, c.performQuickCheck(ctx))\n\t\tcase \"integrity\":\n\t\t\tresults = append(results, c.performIntegrityCheck(ctx))\n\t\tcase \"checksum\":\n\t\t\tresults = append(results, c.performChecksumCheck(ctx))\n\t\tcase \"full\":\n\t\t\tresults = append(results, c.performQuickCheck(ctx))\n\t\t\tresults = append(results, c.performIntegrityCheck(ctx))\n\t\t\tresults = append(results, c.performChecksumCheck(ctx))\n\t\t\tresults = append(results, c.performDataValidation(ctx))\n\t\t}\n\t}\n\n\treturn c.reportResults(results)\n}\n\nfunc (c *ValidateCommand) performRestore(ctx context.Context) ValidationResult {\n\tstartTime := time.Now()\n\tresult := ValidationResult{\n\t\tCheckType: \"restore\",\n\t\tDetails: make(map[string]interface{}),\n\t}\n\n\tif err := os.Remove(c.RestoredDB); err != nil && !os.IsNotExist(err) {\n\t\tslog.Warn(\"Could not remove existing restored database\", \"error\", err)\n\t}\n\n\tvar cmd *exec.Cmd\n\tif c.ConfigPath != \"\" {\n\t\tcmd = exec.CommandContext(ctx, \"litestream\", \"restore\",\n\t\t\t\"-config\", c.ConfigPath,\n\t\t\t\"-o\", c.RestoredDB,\n\t\t\tc.SourceDB,\n\t\t)\n\t} else {\n\t\tcmd = exec.CommandContext(ctx, \"litestream\", \"restore\",\n\t\t\t\"-o\", c.RestoredDB,\n\t\t\tc.ReplicaURL,\n\t\t)\n\t}\n\n\toutput, err := cmd.CombinedOutput()\n\tresult.Duration = time.Since(startTime)\n\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"restore failed: %v\\nOutput: %s\", err, string(output))\n\t\treturn result\n\t}\n\n\tif _, err := os.Stat(c.RestoredDB); err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"restored database not found: %v\", err)\n\t\treturn result\n\t}\n\n\tresult.Passed = true\n\tresult.Details[\"restored_path\"] = c.RestoredDB\n\n\tif info, err := os.Stat(c.RestoredDB); err == nil {\n\t\tresult.Details[\"restored_size\"] = info.Size()\n\t}\n\n\tslog.Info(\"Restore completed\",\n\t\t\"duration\", result.Duration,\n\t\t\"restored_db\", c.RestoredDB,\n\t)\n\n\treturn result\n}\n\nfunc (c *ValidateCommand) performQuickCheck(ctx context.Context) ValidationResult {\n\tstartTime := time.Now()\n\tresult := ValidationResult{\n\t\tCheckType: \"quick_check\",\n\t\tDetails: make(map[string]interface{}),\n\t}\n\n\tdb, err := sql.Open(\"sqlite3\", c.RestoredDB)\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to open database: %v\", err)\n\t\treturn result\n\t}\n\tdefer db.Close()\n\n\tvar checkResult string\n\terr = db.QueryRow(\"PRAGMA quick_check\").Scan(&checkResult)\n\tresult.Duration = time.Since(startTime)\n\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"quick check failed: %v\", err)\n\t\treturn result\n\t}\n\n\tresult.Passed = checkResult == \"ok\"\n\tresult.Details[\"check_result\"] = checkResult\n\n\tif !result.Passed {\n\t\tresult.ErrorMessage = fmt.Sprintf(\"quick check returned: %s\", checkResult)\n\t}\n\n\tslog.Info(\"Quick check completed\",\n\t\t\"passed\", result.Passed,\n\t\t\"duration\", result.Duration,\n\t)\n\n\treturn result\n}\n\nfunc (c *ValidateCommand) performIntegrityCheck(ctx context.Context) ValidationResult {\n\tstartTime := time.Now()\n\tresult := ValidationResult{\n\t\tCheckType: \"integrity_check\",\n\t\tDetails: make(map[string]interface{}),\n\t}\n\n\tdb, err := sql.Open(\"sqlite3\", c.RestoredDB)\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to open database: %v\", err)\n\t\treturn result\n\t}\n\tdefer db.Close()\n\n\trows, err := db.Query(\"PRAGMA integrity_check\")\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"integrity check failed: %v\", err)\n\t\tresult.Duration = time.Since(startTime)\n\t\treturn result\n\t}\n\tdefer rows.Close()\n\n\tvar results []string\n\tfor rows.Next() {\n\t\tvar line string\n\t\tif err := rows.Scan(&line); err != nil {\n\t\t\tresult.Passed = false\n\t\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to scan result: %v\", err)\n\t\t\tresult.Duration = time.Since(startTime)\n\t\t\treturn result\n\t\t}\n\t\tresults = append(results, line)\n\t}\n\n\tresult.Duration = time.Since(startTime)\n\tresult.Details[\"check_results\"] = results\n\n\tif len(results) == 1 && results[0] == \"ok\" {\n\t\tresult.Passed = true\n\t} else {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"integrity check found issues: %v\", results)\n\t}\n\n\tslog.Info(\"Integrity check completed\",\n\t\t\"passed\", result.Passed,\n\t\t\"duration\", result.Duration,\n\t\t\"issues\", len(results)-1,\n\t)\n\n\treturn result\n}\n\nfunc (c *ValidateCommand) performChecksumCheck(ctx context.Context) ValidationResult {\n\tstartTime := time.Now()\n\tresult := ValidationResult{\n\t\tCheckType: \"checksum\",\n\t\tDetails: make(map[string]interface{}),\n\t}\n\n\tsourceChecksum, err := c.calculateDBChecksum(c.SourceDB)\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to calculate source checksum: %v\", err)\n\t\tresult.Duration = time.Since(startTime)\n\t\treturn result\n\t}\n\n\trestoredChecksum, err := c.calculateDBChecksum(c.RestoredDB)\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to calculate restored checksum: %v\", err)\n\t\tresult.Duration = time.Since(startTime)\n\t\treturn result\n\t}\n\n\tresult.Duration = time.Since(startTime)\n\tresult.Details[\"source_checksum\"] = fmt.Sprintf(\"%x\", sourceChecksum)\n\tresult.Details[\"restored_checksum\"] = fmt.Sprintf(\"%x\", restoredChecksum)\n\n\tif string(sourceChecksum) == string(restoredChecksum) {\n\t\tresult.Passed = true\n\t} else {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = \"checksums do not match\"\n\t}\n\n\tslog.Info(\"Checksum check completed\",\n\t\t\"passed\", result.Passed,\n\t\t\"duration\", result.Duration,\n\t\t\"match\", result.Passed,\n\t)\n\n\treturn result\n}\n\nfunc (c *ValidateCommand) performDataValidation(ctx context.Context) ValidationResult {\n\tstartTime := time.Now()\n\tresult := ValidationResult{\n\t\tCheckType: \"data_validation\",\n\t\tDetails: make(map[string]interface{}),\n\t}\n\n\tsourceDB, err := sql.Open(\"sqlite3\", c.SourceDB)\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to open source database: %v\", err)\n\t\treturn result\n\t}\n\tdefer sourceDB.Close()\n\n\trestoredDB, err := sql.Open(\"sqlite3\", c.RestoredDB)\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to open restored database: %v\", err)\n\t\treturn result\n\t}\n\tdefer restoredDB.Close()\n\n\ttables, err := c.getTableList(sourceDB)\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to get table list: %v\", err)\n\t\tresult.Duration = time.Since(startTime)\n\t\treturn result\n\t}\n\n\tresult.Details[\"tables_checked\"] = len(tables)\n\tallMatch := true\n\n\tfor _, table := range tables {\n\t\tsourceCount, err := c.getRowCount(sourceDB, table)\n\t\tif err != nil {\n\t\t\tresult.Passed = false\n\t\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to count rows in source table %s: %v\", table, err)\n\t\t\tresult.Duration = time.Since(startTime)\n\t\t\treturn result\n\t\t}\n\n\t\trestoredCount, err := c.getRowCount(restoredDB, table)\n\t\tif err != nil {\n\t\t\tresult.Passed = false\n\t\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to count rows in restored table %s: %v\", table, err)\n\t\t\tresult.Duration = time.Since(startTime)\n\t\t\treturn result\n\t\t}\n\n\t\tif sourceCount != restoredCount {\n\t\t\tallMatch = false\n\t\t\tresult.Details[fmt.Sprintf(\"table_%s_mismatch\", table)] = fmt.Sprintf(\"source=%d, restored=%d\", sourceCount, restoredCount)\n\t\t}\n\t}\n\n\tresult.Duration = time.Since(startTime)\n\tresult.Passed = allMatch\n\n\tif !allMatch {\n\t\tresult.ErrorMessage = \"row count mismatch between source and restored databases\"\n\t}\n\n\tslog.Info(\"Data validation completed\",\n\t\t\"passed\", result.Passed,\n\t\t\"duration\", result.Duration,\n\t\t\"tables_checked\", len(tables),\n\t)\n\n\treturn result\n}\n\nfunc (c *ValidateCommand) validateLTXContinuity(ctx context.Context) ValidationResult {\n\tstartTime := time.Now()\n\tresult := ValidationResult{\n\t\tCheckType: \"ltx_continuity\",\n\t\tDetails: make(map[string]interface{}),\n\t}\n\n\tcmd := exec.CommandContext(ctx, \"litestream\", \"ltx\", c.ReplicaURL)\n\toutput, err := cmd.Output()\n\tif err != nil {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = fmt.Sprintf(\"failed to list LTX files: %v\", err)\n\t\tresult.Duration = time.Since(startTime)\n\t\treturn result\n\t}\n\n\tlines := strings.Split(string(output), \"\\n\")\n\tif len(lines) < 2 {\n\t\tresult.Passed = false\n\t\tresult.ErrorMessage = \"no LTX files found\"\n\t\tresult.Duration = time.Since(startTime)\n\t\treturn result\n\t}\n\n\tresult.Passed = true\n\tresult.Duration = time.Since(startTime)\n\tresult.Details[\"ltx_files_checked\"] = len(lines) - 2\n\n\tslog.Info(\"LTX continuity check completed\",\n\t\t\"passed\", result.Passed,\n\t\t\"duration\", result.Duration,\n\t)\n\n\treturn result\n}\n\nfunc (c *ValidateCommand) calculateDBChecksum(path string) ([]byte, error) {\n\tfile, err := os.Open(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer file.Close()\n\n\thash := md5.New()\n\tif _, err := io.Copy(hash, file); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn hash.Sum(nil), nil\n}\n\nfunc (c *ValidateCommand) getTableList(db *sql.DB) ([]string, error) {\n\trows, err := db.Query(\"SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer rows.Close()\n\n\tvar tables []string\n\tfor rows.Next() {\n\t\tvar table string\n\t\tif err := rows.Scan(&table); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\ttables = append(tables, table)\n\t}\n\n\treturn tables, nil\n}\n\nfunc (c *ValidateCommand) getRowCount(db *sql.DB, table string) (int, error) {\n\tvar count int\n\tquery := fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", table)\n\terr := db.QueryRow(query).Scan(&count)\n\treturn count, err\n}\n\nfunc (c *ValidateCommand) reportResults(results []ValidationResult) error {\n\tallPassed := true\n\tfor _, result := range results {\n\t\tif !result.Passed {\n\t\t\tallPassed = false\n\t\t\tslog.Error(\"Validation failed\",\n\t\t\t\t\"check_type\", result.CheckType,\n\t\t\t\t\"error\", result.ErrorMessage,\n\t\t\t)\n\t\t}\n\t}\n\n\tif allPassed {\n\t\tslog.Info(\"All validation checks passed\")\n\t\treturn nil\n\t}\n\n\treturn fmt.Errorf(\"validation failed\")\n}\n\nfunc (c *ValidateCommand) Usage() {\n\tfmt.Fprintln(c.Main.Stdout, `\nValidate replication integrity by restoring and checking databases.\n\nUsage:\n\n\tlitestream-test validate [options]\n\nOptions:\n\n\t-source-db PATH\n\t Original database path (required)\n\n\t-replica-url URL\n\t Replica URL to validate\n\n\t-restored-db PATH\n\t Path for restored database\n\t Default: source-db.restored\n\n\t-check-type TYPE\n\t Type of check: quick, integrity, checksum, full\n\t Default: quick\n\n\t-ltx-continuity\n\t Check LTX file continuity\n\t Default: false\n\n\t-config PATH\n\t Litestream config file path\n\nExamples:\n\n\t# Quick validation\n\tlitestream-test validate -source-db /tmp/test.db -replica-url s3://bucket/test\n\n\t# Full validation with all checks\n\tlitestream-test validate -source-db /tmp/test.db -replica-url s3://bucket/test -check-type full\n\n\t# Validate with config file\n\tlitestream-test validate -source-db /tmp/test.db -config /etc/litestream.yml -check-type integrity\n`[1:])\n}\n" }, { "path": "cmd/litestream-vfs/chaos_test.go", "content": "//go:build vfs && chaos\n// +build vfs,chaos\n\npackage main_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"io\"\n\t\"math/rand\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestVFS_ChaosEngineering(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tdb, primary := openReplicatedPrimary(t, client, 15*time.Millisecond, 15*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(`CREATE TABLE chaos (\n id INTEGER PRIMARY KEY AUTOINCREMENT,\n value TEXT,\n grp INTEGER\n )`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tfor i := 0; i < 64; i++ {\n\t\tif _, err := primary.Exec(\"INSERT INTO chaos (value, grp) VALUES (?, ?)\", randomPayload(rand.New(rand.NewSource(int64(i))), 48), i%8); err != nil {\n\t\t\tt.Fatalf(\"seed chaos: %v\", err)\n\t\t}\n\t}\n\n\tchaosClient := newChaosReplicaClient(client)\n\tvfs := newVFS(t, chaosClient)\n\tvfs.PollInterval = 15 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\twaitForTableRowCount(t, primary, replica, \"chaos\", 5*time.Second)\n\tchaosClient.active.Store(true)\n\n\tctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)\n\tdefer cancel()\n\n\twriterDone := make(chan error, 1)\n\tgo func() {\n\t\trnd := rand.New(rand.NewSource(42))\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\twriterDone <- nil\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\t\t\tswitch rnd.Intn(3) {\n\t\t\tcase 0:\n\t\t\t\tif _, err := primary.Exec(\"INSERT INTO chaos (value, grp) VALUES (?, ?)\", randomPayload(rnd, 32), rnd.Intn(8)); err != nil && !isBusyError(err) {\n\t\t\t\t\twriterDone <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\tcase 1:\n\t\t\t\tif _, err := primary.Exec(\"UPDATE chaos SET value = ? WHERE id = (ABS(random()) % 64) + 1\", randomPayload(rnd, 24)); err != nil && !isBusyError(err) {\n\t\t\t\t\twriterDone <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\tcase 2:\n\t\t\t\tif _, err := primary.Exec(\"DELETE FROM chaos WHERE id IN (SELECT id FROM chaos ORDER BY RANDOM() LIMIT 1)\"); err != nil && !isBusyError(err) {\n\t\t\t\t\twriterDone <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t\ttime.Sleep(5 * time.Millisecond)\n\t\t}\n\t}()\n\n\tconst readers = 16\n\treaderErrs := make(chan error, readers)\n\tfor i := 0; i < readers; i++ {\n\t\tgo func() {\n\t\t\trnd := rand.New(rand.NewSource(time.Now().UnixNano()))\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treaderErrs <- nil\n\t\t\t\t\treturn\n\t\t\t\tdefault:\n\t\t\t\t}\n\t\t\t\tvar count int\n\t\t\t\tswitch rnd.Intn(3) {\n\t\t\t\tcase 0:\n\t\t\t\t\terr := replica.QueryRow(\"SELECT COUNT(*) FROM chaos WHERE grp = ?\", rnd.Intn(8)).Scan(&count)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\t\t\t\t\t\treaderErrs <- err\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\tcase 1:\n\t\t\t\t\trows, err := replica.Query(\"SELECT id, value FROM chaos ORDER BY id DESC LIMIT 5 OFFSET ?\", rnd.Intn(10))\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\t\t\t\t\t\treaderErrs <- err\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tretryRows := false\n\t\t\t\t\tfor rows.Next() {\n\t\t\t\t\t\tvar id int\n\t\t\t\t\t\tvar value string\n\t\t\t\t\t\tif err := rows.Scan(&id, &value); err != nil {\n\t\t\t\t\t\t\trows.Close()\n\t\t\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\t\t\tretryRows = true\n\t\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\treaderErrs <- err\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tif retryRows {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tif err := rows.Err(); err != nil {\n\t\t\t\t\t\trows.Close()\n\t\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\t\t\t\t\t\treaderErrs <- err\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\trows.Close()\n\t\t\t\tcase 2:\n\t\t\t\t\terr := replica.QueryRow(\"SELECT SUM(LENGTH(value)) FROM chaos WHERE id BETWEEN ? AND ?\",\n\t\t\t\t\t\trnd.Intn(32)+1, rnd.Intn(32)+33).Scan(&count)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\t\t\t\t\t\treaderErrs <- err\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\t}\n\n\t<-ctx.Done()\n\tfor i := 0; i < readers; i++ {\n\t\tif err := <-readerErrs; err != nil {\n\t\t\tt.Fatalf(\"reader error: %v\", err)\n\t\t}\n\t}\n\tif err := <-writerDone; err != nil {\n\t\tt.Fatalf(\"writer error: %v\", err)\n\t}\n\n\twaitForTableRowCount(t, primary, replica, \"chaos\", 5*time.Second)\n\tif chaosClient.failures.Load() == 0 {\n\t\tt.Fatalf(\"expected injected failures\")\n\t}\n}\n\nfunc newChaosReplicaClient(base litestream.ReplicaClient) *chaosReplicaClient {\n\treturn &chaosReplicaClient{\n\t\tReplicaClient: base,\n\t\trnd: rand.New(rand.NewSource(99)),\n\t}\n}\n\ntype chaosReplicaClient struct {\n\tlitestream.ReplicaClient\n\trnd *rand.Rand\n\tfailures atomic.Int32\n\tactive atomic.Bool\n}\n\nfunc (c *chaosReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif !c.active.Load() {\n\t\treturn c.ReplicaClient.LTXFiles(ctx, level, seek, useMetadata)\n\t}\n\tif c.rnd.Float64() < 0.05 {\n\t\tc.failures.Add(1)\n\t\treturn nil, context.DeadlineExceeded\n\t}\n\treturn c.ReplicaClient.LTXFiles(ctx, level, seek, useMetadata)\n}\n\nfunc (c *chaosReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tif !c.active.Load() {\n\t\treturn c.ReplicaClient.OpenLTXFile(ctx, level, minTXID, maxTXID, offset, size)\n\t}\n\tdelay := time.Duration(c.rnd.Intn(5)) * time.Millisecond\n\tif delay > 0 {\n\t\ttime.Sleep(delay)\n\t}\n\tif c.rnd.Float64() < 0.05 {\n\t\tc.failures.Add(1)\n\t\treturn nil, context.DeadlineExceeded\n\t}\n\trc, err := c.ReplicaClient.OpenLTXFile(ctx, level, minTXID, maxTXID, offset, size)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif c.rnd.Float64() < 0.05 && size > 0 {\n\t\tdata, err := io.ReadAll(rc)\n\t\trc.Close()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif len(data) > 32 {\n\t\t\tdata = data[:len(data)/2]\n\t\t}\n\t\tc.failures.Add(1)\n\t\treturn io.NopCloser(bytes.NewReader(data)), nil\n\t}\n\treturn rc, nil\n}\n" }, { "path": "cmd/litestream-vfs/fuzz_test.go", "content": "//go:build vfs\n// +build vfs\n\npackage main_test\n\nimport (\n\t\"database/sql\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\n// TestVFS_FuzzSeedCorpus runs a handful of fixed corpora so `go test`\n// exercises the same logic as the fuzz harness without requiring\n// `-fuzz=...`.\nfunc TestVFS_FuzzSeedCorpus(t *testing.T) {\n\tseeds := [][]byte{\n\t\t[]byte{0x00, 0x01, 0x02},\n\t\t[]byte(\"litestream vfs fuzz\"),\n\t\t[]byte{0xFF, 0x10, 0x42, 0x7F},\n\t}\n\tfor _, seed := range seeds {\n\t\trunVFSFuzzWorkload(t, seed)\n\t}\n}\n\n// FuzzVFSReplicaReadPatterns exercises random combinations of reads,\n// aggregates, and ordering queries against the VFS replica. Enable with:\n//\n//\tgo test ./cmd/litestream-vfs -tags vfs -fuzz=FuzzVFSReplicaReadPatterns\nfunc FuzzVFSReplicaReadPatterns(f *testing.F) {\n\tf.Add([]byte(\"seed\"))\n\tf.Add([]byte{0x1, 0x2, 0x3, 0x4})\n\tf.Add([]byte{0xAA, 0xBB, 0xCC})\n\n\tf.Fuzz(func(t *testing.T, data []byte) {\n\t\trunVFSFuzzWorkload(t, data)\n\t})\n}\n\nfunc runVFSFuzzWorkload(tb testing.TB, corpus []byte) {\n\ttb.Helper()\n\tif len(corpus) == 0 {\n\t\tcorpus = []byte{0}\n\t}\n\tif len(corpus) > 256 {\n\t\tcorpus = corpus[:256]\n\t}\n\n\tclient := file.NewReplicaClient(tb.TempDir())\n\tif err := os.MkdirAll(client.LTXLevelDir(0), 0o755); err != nil {\n\t\ttb.Fatalf(\"init replica dir: %v\", err)\n\t}\n\t_, primary := openReplicatedPrimary(tb, client, 15*time.Millisecond, 15*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(tb, primary)\n\n\tif _, err := primary.Exec(`CREATE TABLE fuzz (\n id INTEGER PRIMARY KEY,\n value TEXT,\n grp INTEGER\n )`); err != nil {\n\t\ttb.Fatalf(\"create table: %v\", err)\n\t}\n\n\t// Deterministic seed data so we have plenty of rows/pages to hydrate.\n\tfor i := 0; i < 128; i++ {\n\t\tpayload := fmt.Sprintf(\"row-%03d-%s\", i, strings.Repeat(\"x\", (i%17)+8))\n\t\tif _, err := primary.Exec(\"INSERT INTO fuzz (value, grp) VALUES (?, ?)\", payload, i%11); err != nil {\n\t\t\ttb.Fatalf(\"seed insert: %v\", err)\n\t\t}\n\t}\n\n\tvfs := newVFS(tb, client)\n\tvfs.PollInterval = 15 * time.Millisecond\n\tvfsName := registerTestVFS(tb, vfs)\n\treplica := openVFSReplicaDB(tb, vfsName)\n\tdefer replica.Close()\n\n\trequire.Eventually(tb, func() bool {\n\t\tvar primaryCount, replicaCount int\n\t\tif err := primary.QueryRow(\"SELECT COUNT(*) FROM fuzz\").Scan(&primaryCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM fuzz\").Scan(&replicaCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn primaryCount == replicaCount\n\t}, 5*time.Second, 20*time.Millisecond, \"replica should catch up to primary\")\n\n\tconst maxOps = 128\n\tfor i := 0; i < len(corpus) && i < maxOps; i++ {\n\t\top := corpus[i] % 6\n\t\tswitch op {\n\t\tcase 0:\n\t\t\tid := int(corpus[i])%128 + 1\n\t\t\tvar value string\n\t\t\terr := replica.QueryRow(\"SELECT value FROM fuzz WHERE id = ?\", id).Scan(&value)\n\t\t\tif err != nil && err != sql.ErrNoRows {\n\t\t\t\ttb.Fatalf(\"select by id: %v\", err)\n\t\t\t}\n\t\tcase 1:\n\t\t\tvar count int\n\t\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM fuzz WHERE grp = ?\", int(corpus[i])%11).Scan(&count); err != nil {\n\t\t\t\ttb.Fatalf(\"count grp: %v\", err)\n\t\t\t}\n\t\tcase 2:\n\t\t\trows, err := replica.Query(\"SELECT value FROM fuzz ORDER BY value DESC LIMIT 5 OFFSET ?\", int(corpus[i])%10)\n\t\t\tif err != nil {\n\t\t\t\ttb.Fatalf(\"ordered scan: %v\", err)\n\t\t\t}\n\t\t\tfor rows.Next() {\n\t\t\t\tvar v string\n\t\t\t\tif err := rows.Scan(&v); err != nil {\n\t\t\t\t\ttb.Fatalf(\"scan ordered: %v\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif err := rows.Err(); err != nil {\n\t\t\t\ttb.Fatalf(\"ordered rows err: %v\", err)\n\t\t\t}\n\t\t\trows.Close()\n\t\tcase 3:\n\t\t\tvar sum int\n\t\t\tif err := replica.QueryRow(\"SELECT SUM(LENGTH(value)) FROM fuzz WHERE id BETWEEN ? AND ?\",\n\t\t\t\tint(corpus[i])%64+1, int(corpus[i])%64+64).Scan(&sum); err != nil {\n\t\t\t\ttb.Fatalf(\"sum lengths: %v\", err)\n\t\t\t}\n\t\tcase 4:\n\t\t\t// Cross-check counts between primary & replica for a random grp.\n\t\t\tgrp := int(corpus[i]) % 11\n\t\t\tvar pc, rc int\n\t\t\tif err := primary.QueryRow(\"SELECT COUNT(*) FROM fuzz WHERE grp = ?\", grp).Scan(&pc); err != nil {\n\t\t\t\ttb.Fatalf(\"primary grp count: %v\", err)\n\t\t\t}\n\t\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM fuzz WHERE grp = ?\", grp).Scan(&rc); err != nil {\n\t\t\t\ttb.Fatalf(\"replica grp count: %v\", err)\n\t\t\t}\n\t\t\tif pc != rc {\n\t\t\t\ttb.Fatalf(\"count mismatch grp=%d primary=%d replica=%d\", grp, pc, rc)\n\t\t\t}\n\t\tcase 5:\n\t\t\t// Random LIKE query to exercise page cache churn.\n\t\t\tpattern := fmt.Sprintf(\"row-%%%02x%%\", corpus[i])\n\t\t\trows, err := replica.Query(\"SELECT id FROM fuzz WHERE value LIKE ? LIMIT 3\", pattern)\n\t\t\tif err != nil {\n\t\t\t\ttb.Fatalf(\"like query: %v\", err)\n\t\t\t}\n\t\t\trows.Close()\n\t\t}\n\t}\n}\n" }, { "path": "cmd/litestream-vfs/hydration_e2e_test.go", "content": "//go:build vfs\n// +build vfs\n\npackage main_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\n// TestHydration_E2E_SQLiteCLI tests hydration environment variables via the SQLite CLI.\n// This test builds the VFS extension and uses the actual sqlite3 CLI to verify\n// that LITESTREAM_HYDRATION_ENABLED and LITESTREAM_HYDRATION_PATH work correctly.\nfunc TestHydration_E2E_SQLiteCLI(t *testing.T) {\n\tif runtime.GOOS != \"darwin\" && runtime.GOOS != \"linux\" {\n\t\tt.Skip(\"skipping: test only runs on darwin or linux\")\n\t}\n\n\t// Check if sqlite3 CLI is available\n\tif _, err := exec.LookPath(\"sqlite3\"); err != nil {\n\t\tt.Skip(\"skipping: sqlite3 CLI not found in PATH\")\n\t}\n\n\t// Build the VFS extension\n\textPath := buildVFSExtension(t)\n\n\t// Create a file replica with test data\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\tsetupTestReplica(t, client)\n\n\t// Create a temp file for hydration output\n\thydrationPath := filepath.Join(t.TempDir(), \"hydrated.db\")\n\n\t// Run sqlite3 with hydration enabled\n\tenv := []string{\n\t\t\"LITESTREAM_REPLICA_URL=file://\" + replicaDir,\n\t\t\"LITESTREAM_HYDRATION_ENABLED=true\",\n\t\t\"LITESTREAM_HYDRATION_PATH=\" + hydrationPath,\n\t\t\"LITESTREAM_LOG_LEVEL=DEBUG\",\n\t}\n\n\t// Query via the VFS\n\toutput := runSQLiteCLI(t, extPath, env, \"SELECT name FROM users WHERE id = 1;\")\n\trequire.Contains(t, output, \"Alice\", \"should read data via VFS\")\n\n\t// Verify hydration file was created\n\trequire.Eventually(t, func() bool {\n\t\tinfo, err := os.Stat(hydrationPath)\n\t\treturn err == nil && info.Size() > 0\n\t}, 5*time.Second, 100*time.Millisecond, \"hydration file should be created\")\n}\n\n// TestHydration_E2E_SQLiteCLI_TempFile tests hydration without specifying a path (uses temp file).\nfunc TestHydration_E2E_SQLiteCLI_TempFile(t *testing.T) {\n\tif runtime.GOOS != \"darwin\" && runtime.GOOS != \"linux\" {\n\t\tt.Skip(\"skipping: test only runs on darwin or linux\")\n\t}\n\n\tif _, err := exec.LookPath(\"sqlite3\"); err != nil {\n\t\tt.Skip(\"skipping: sqlite3 CLI not found in PATH\")\n\t}\n\n\textPath := buildVFSExtension(t)\n\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\tsetupTestReplica(t, client)\n\n\t// Run without LITESTREAM_HYDRATION_PATH - should use temp file\n\tenv := []string{\n\t\t\"LITESTREAM_REPLICA_URL=file://\" + replicaDir,\n\t\t\"LITESTREAM_HYDRATION_ENABLED=true\",\n\t\t\"LITESTREAM_LOG_LEVEL=DEBUG\",\n\t}\n\n\toutput := runSQLiteCLI(t, extPath, env, \"SELECT COUNT(*) FROM users;\")\n\trequire.Contains(t, output, \"1\", \"should read data via VFS with temp hydration file\")\n}\n\n// TestHydration_E2E_SQLiteCLI_Disabled tests that hydration is disabled by default.\nfunc TestHydration_E2E_SQLiteCLI_Disabled(t *testing.T) {\n\tif runtime.GOOS != \"darwin\" && runtime.GOOS != \"linux\" {\n\t\tt.Skip(\"skipping: test only runs on darwin or linux\")\n\t}\n\n\tif _, err := exec.LookPath(\"sqlite3\"); err != nil {\n\t\tt.Skip(\"skipping: sqlite3 CLI not found in PATH\")\n\t}\n\n\textPath := buildVFSExtension(t)\n\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\tsetupTestReplica(t, client)\n\n\thydrationPath := filepath.Join(t.TempDir(), \"should-not-exist.db\")\n\n\t// Run without LITESTREAM_HYDRATION_ENABLED\n\tenv := []string{\n\t\t\"LITESTREAM_REPLICA_URL=file://\" + replicaDir,\n\t\t\"LITESTREAM_HYDRATION_PATH=\" + hydrationPath,\n\t\t\"LITESTREAM_LOG_LEVEL=DEBUG\",\n\t}\n\n\toutput := runSQLiteCLI(t, extPath, env, \"SELECT name FROM users WHERE id = 1;\")\n\trequire.Contains(t, output, \"Alice\", \"should still read data via VFS\")\n\n\t// Hydration file should NOT be created when disabled\n\t_, err := os.Stat(hydrationPath)\n\trequire.True(t, os.IsNotExist(err), \"hydration file should not be created when disabled\")\n}\n\n// TestHydration_E2E_SQLiteCLI_MultipleQueries tests that hydration persists across queries.\nfunc TestHydration_E2E_SQLiteCLI_MultipleQueries(t *testing.T) {\n\tif runtime.GOOS != \"darwin\" && runtime.GOOS != \"linux\" {\n\t\tt.Skip(\"skipping: test only runs on darwin or linux\")\n\t}\n\n\tif _, err := exec.LookPath(\"sqlite3\"); err != nil {\n\t\tt.Skip(\"skipping: sqlite3 CLI not found in PATH\")\n\t}\n\n\textPath := buildVFSExtension(t)\n\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\tsetupTestReplicaWithMoreData(t, client)\n\n\thydrationPath := filepath.Join(t.TempDir(), \"hydrated.db\")\n\n\tenv := []string{\n\t\t\"LITESTREAM_REPLICA_URL=file://\" + replicaDir,\n\t\t\"LITESTREAM_HYDRATION_ENABLED=true\",\n\t\t\"LITESTREAM_HYDRATION_PATH=\" + hydrationPath,\n\t\t\"LITESTREAM_LOG_LEVEL=DEBUG\",\n\t}\n\n\t// Run multiple queries in single session\n\tqueries := `\nSELECT COUNT(*) FROM users;\nSELECT name FROM users WHERE id = 1;\nSELECT name FROM users WHERE id = 5;\n`\n\toutput := runSQLiteCLI(t, extPath, env, queries)\n\trequire.Contains(t, output, \"10\", \"should have 10 users\")\n\trequire.Contains(t, output, \"Alice\", \"should find Alice\")\n\trequire.Contains(t, output, \"User5\", \"should find User5\")\n\n\t// Wait for hydration to complete\n\trequire.Eventually(t, func() bool {\n\t\tinfo, err := os.Stat(hydrationPath)\n\t\treturn err == nil && info.Size() > 0\n\t}, 5*time.Second, 100*time.Millisecond, \"hydration file should be created\")\n}\n\n// buildVFSExtension builds the VFS extension and returns its path.\nfunc buildVFSExtension(t *testing.T) string {\n\tt.Helper()\n\n\t// Determine expected extension filename based on OS\n\tvar extName string\n\tswitch runtime.GOOS {\n\tcase \"darwin\":\n\t\textName = \"litestream-vfs.dylib\"\n\tcase \"linux\":\n\t\textName = \"litestream-vfs.so\"\n\tdefault:\n\t\tt.Fatalf(\"unsupported OS: %s\", runtime.GOOS)\n\t}\n\n\t// Check if extension already exists in dist/\n\tprojectRoot := findProjectRoot(t)\n\textPath := filepath.Join(projectRoot, \"dist\", extName)\n\n\tif _, err := os.Stat(extPath); err == nil {\n\t\treturn extPath\n\t}\n\n\t// Build the extension\n\tt.Logf(\"building VFS extension at %s\", extPath)\n\n\tvar makeTarget string\n\tswitch runtime.GOOS {\n\tcase \"darwin\":\n\t\tif runtime.GOARCH == \"arm64\" {\n\t\t\tmakeTarget = \"vfs-darwin-arm64\"\n\t\t\textPath = filepath.Join(projectRoot, \"dist\", \"litestream-vfs-darwin-arm64.dylib\")\n\t\t} else {\n\t\t\tmakeTarget = \"vfs-darwin-amd64\"\n\t\t\textPath = filepath.Join(projectRoot, \"dist\", \"litestream-vfs-darwin-amd64.dylib\")\n\t\t}\n\tcase \"linux\":\n\t\tif runtime.GOARCH == \"arm64\" {\n\t\t\tmakeTarget = \"vfs-linux-arm64\"\n\t\t\textPath = filepath.Join(projectRoot, \"dist\", \"litestream-vfs-linux-arm64.so\")\n\t\t} else {\n\t\t\tmakeTarget = \"vfs-linux-amd64\"\n\t\t\textPath = filepath.Join(projectRoot, \"dist\", \"litestream-vfs-linux-amd64.so\")\n\t\t}\n\t}\n\n\tcmd := exec.Command(\"make\", makeTarget)\n\tcmd.Dir = projectRoot\n\tcmd.Stdout = os.Stdout\n\tcmd.Stderr = os.Stderr\n\tif err := cmd.Run(); err != nil {\n\t\tt.Fatalf(\"failed to build VFS extension: %v\", err)\n\t}\n\n\treturn extPath\n}\n\n// findProjectRoot finds the project root directory.\nfunc findProjectRoot(t *testing.T) string {\n\tt.Helper()\n\n\t// Start from current directory and walk up\n\tdir, err := os.Getwd()\n\trequire.NoError(t, err)\n\n\tfor {\n\t\tif _, err := os.Stat(filepath.Join(dir, \"go.mod\")); err == nil {\n\t\t\treturn dir\n\t\t}\n\t\tparent := filepath.Dir(dir)\n\t\tif parent == dir {\n\t\t\tt.Fatal(\"could not find project root (go.mod)\")\n\t\t}\n\t\tdir = parent\n\t}\n}\n\n// runSQLiteCLI runs the sqlite3 CLI with the VFS extension and returns output.\nfunc runSQLiteCLI(t *testing.T, extPath string, env []string, query string) string {\n\tt.Helper()\n\n\t// Build command: sqlite3 :memory: -cmd \".load \" \"\"\n\targs := []string{\n\t\t\":memory:\",\n\t\t\"-cmd\", \".load \" + extPath,\n\t\tquery,\n\t}\n\n\tcmd := exec.Command(\"sqlite3\", args...)\n\tcmd.Env = append(os.Environ(), env...)\n\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil {\n\t\toutputStr := string(output)\n\t\t// Check for common extension loading failures\n\t\tif strings.Contains(outputStr, \"Error: unknown command\") ||\n\t\t\tstrings.Contains(outputStr, \"not authorized\") ||\n\t\t\tstrings.Contains(outputStr, \"symbol not found\") ||\n\t\t\tstrings.Contains(outputStr, \"dlsym\") {\n\t\t\tt.Skipf(\"skipping: sqlite3 cannot load extensions (common on macOS): %s\", outputStr)\n\t\t}\n\t\tt.Logf(\"sqlite3 output: %s\", outputStr)\n\t\tt.Fatalf(\"sqlite3 command failed: %v\", err)\n\t}\n\n\treturn string(output)\n}\n\n// setupTestReplica creates a file replica with test data.\nfunc setupTestReplica(t *testing.T, client litestream.ReplicaClient) {\n\tt.Helper()\n\n\tdbDir := t.TempDir()\n\tdb := testingutil.NewDB(t, filepath.Join(dbDir, \"source.db\"))\n\tdb.MonitorInterval = 100 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 100 * time.Millisecond\n\trequire.NoError(t, db.Open())\n\n\tsqldb := testingutil.MustOpenSQLDB(t, db.Path())\n\n\t_, err := sqldb.Exec(\"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (1, 'Alice')\")\n\trequire.NoError(t, err)\n\n\twaitForLTXFiles(t, client, 10*time.Second, db.MonitorInterval)\n\n\trequire.NoError(t, db.Replica.Stop(false))\n\ttestingutil.MustCloseSQLDB(t, sqldb)\n\trequire.NoError(t, db.Close(context.Background()))\n}\n\n// setupTestReplicaWithMoreData creates a file replica with more test data.\nfunc setupTestReplicaWithMoreData(t *testing.T, client litestream.ReplicaClient) {\n\tt.Helper()\n\n\tdbDir := t.TempDir()\n\tdb := testingutil.NewDB(t, filepath.Join(dbDir, \"source.db\"))\n\tdb.MonitorInterval = 100 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 100 * time.Millisecond\n\trequire.NoError(t, db.Open())\n\n\tsqldb := testingutil.MustOpenSQLDB(t, db.Path())\n\n\t_, err := sqldb.Exec(\"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)\")\n\trequire.NoError(t, err)\n\n\t// Insert 10 users\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (1, 'Alice')\")\n\trequire.NoError(t, err)\n\tfor i := 2; i <= 10; i++ {\n\t\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (?, ?)\", i, fmt.Sprintf(\"User%d\", i))\n\t\trequire.NoError(t, err)\n\t}\n\n\twaitForLTXFiles(t, client, 10*time.Second, db.MonitorInterval)\n\n\trequire.NoError(t, db.Replica.Stop(false))\n\ttestingutil.MustCloseSQLDB(t, sqldb)\n\trequire.NoError(t, db.Close(context.Background()))\n}\n" }, { "path": "cmd/litestream-vfs/main.go", "content": "//go:build SQLITE3VFS_LOADABLE_EXT\n// +build SQLITE3VFS_LOADABLE_EXT\n\npackage main\n\n// import C is necessary export to the c-archive .a file\n\n/*\ntypedef long long int sqlite3_int64;\ntypedef unsigned long long int sqlite3_uint64;\n*/\nimport \"C\"\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\t\"unsafe\"\n\n\t\"github.com/psanford/sqlite3vfs\"\n\n\t\"github.com/benbjohnson/litestream\"\n\n\t// Import all replica backends to register their URL factories.\n\t_ \"github.com/benbjohnson/litestream/abs\"\n\t_ \"github.com/benbjohnson/litestream/file\"\n\t_ \"github.com/benbjohnson/litestream/gs\"\n\t_ \"github.com/benbjohnson/litestream/nats\"\n\t_ \"github.com/benbjohnson/litestream/oss\"\n\t_ \"github.com/benbjohnson/litestream/s3\"\n\t_ \"github.com/benbjohnson/litestream/sftp\"\n\t_ \"github.com/benbjohnson/litestream/webdav\"\n)\n\nfunc main() {}\n\n//export LitestreamVFSRegister\nfunc LitestreamVFSRegister() *C.char {\n\tvar client litestream.ReplicaClient\n\tvar err error\n\n\treplicaURL := os.Getenv(\"LITESTREAM_REPLICA_URL\")\n\tif replicaURL == \"\" {\n\t\treturn C.CString(\"LITESTREAM_REPLICA_URL environment variable required\")\n\t}\n\n\tclient, err = litestream.NewReplicaClientFromURL(replicaURL)\n\tif err != nil {\n\t\treturn C.CString(fmt.Sprintf(\"failed to create replica client: %s\", err))\n\t}\n\n\t// Initialize the client.\n\tif err := client.Init(context.Background()); err != nil {\n\t\treturn C.CString(fmt.Sprintf(\"failed to initialize replica client: %s\", err))\n\t}\n\n\tvar level slog.Level\n\tswitch strings.ToUpper(os.Getenv(\"LITESTREAM_LOG_LEVEL\")) {\n\tcase \"DEBUG\":\n\t\tlevel = slog.LevelDebug\n\tdefault:\n\t\tlevel = slog.LevelInfo\n\t}\n\n\tvar logOutput io.Writer = os.Stdout\n\tif logFile := os.Getenv(\"LITESTREAM_LOG_FILE\"); logFile != \"\" {\n\t\tf, err := os.OpenFile(logFile, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)\n\t\tif err != nil {\n\t\t\treturn C.CString(fmt.Sprintf(\"failed to open log file: %s\", err))\n\t\t}\n\t\tlogOutput = f\n\t}\n\tlogger := slog.New(slog.NewTextHandler(logOutput, &slog.HandlerOptions{Level: level}))\n\n\tvfs := litestream.NewVFS(client, logger)\n\n\t// Configure write support if enabled.\n\tif strings.ToLower(os.Getenv(\"LITESTREAM_WRITE_ENABLED\")) == \"true\" {\n\t\tvfs.WriteEnabled = true\n\n\t\tif s := os.Getenv(\"LITESTREAM_SYNC_INTERVAL\"); s != \"\" {\n\t\t\td, err := time.ParseDuration(s)\n\t\t\tif err != nil {\n\t\t\t\treturn C.CString(fmt.Sprintf(\"invalid LITESTREAM_SYNC_INTERVAL: %s\", err))\n\t\t\t}\n\t\t\tvfs.WriteSyncInterval = d\n\t\t}\n\n\t\tif s := os.Getenv(\"LITESTREAM_BUFFER_PATH\"); s != \"\" {\n\t\t\tvfs.WriteBufferPath = s\n\t\t}\n\t}\n\n\t// Configure hydration support if enabled.\n\tif strings.ToLower(os.Getenv(\"LITESTREAM_HYDRATION_ENABLED\")) == \"true\" {\n\t\tvfs.HydrationEnabled = true\n\n\t\tif s := os.Getenv(\"LITESTREAM_HYDRATION_PATH\"); s != \"\" {\n\t\t\tvfs.HydrationPath = s\n\t\t}\n\t}\n\n\tif err := sqlite3vfs.RegisterVFS(\"litestream\", vfs); err != nil {\n\t\treturn C.CString(fmt.Sprintf(\"failed to register VFS: %s\", err))\n\t}\n\n\treturn nil\n}\n\n//export GoLitestreamRegisterConnection\nfunc GoLitestreamRegisterConnection(dbPtr unsafe.Pointer, fileID C.sqlite3_uint64) *C.char {\n\tif err := litestream.RegisterVFSConnection(uintptr(dbPtr), uint64(fileID)); err != nil {\n\t\treturn C.CString(err.Error())\n\t}\n\treturn nil\n}\n\n//export GoLitestreamUnregisterConnection\nfunc GoLitestreamUnregisterConnection(dbPtr unsafe.Pointer) *C.char {\n\tlitestream.UnregisterVFSConnection(uintptr(dbPtr))\n\treturn nil\n}\n\n//export GoLitestreamSetTime\nfunc GoLitestreamSetTime(dbPtr unsafe.Pointer, timestamp *C.char) *C.char {\n\tif timestamp == nil {\n\t\treturn C.CString(\"timestamp required\")\n\t}\n\tif err := litestream.SetVFSConnectionTime(uintptr(dbPtr), C.GoString(timestamp)); err != nil {\n\t\treturn C.CString(err.Error())\n\t}\n\treturn nil\n}\n\n//export GoLitestreamResetTime\nfunc GoLitestreamResetTime(dbPtr unsafe.Pointer) *C.char {\n\tif err := litestream.ResetVFSConnectionTime(uintptr(dbPtr)); err != nil {\n\t\treturn C.CString(err.Error())\n\t}\n\treturn nil\n}\n\n//export GoLitestreamTime\nfunc GoLitestreamTime(dbPtr unsafe.Pointer, out **C.char) *C.char {\n\tvalue, err := litestream.GetVFSConnectionTime(uintptr(dbPtr))\n\tif err != nil {\n\t\treturn C.CString(err.Error())\n\t}\n\tif out != nil {\n\t\t*out = C.CString(value)\n\t}\n\treturn nil\n}\n\n//export GoLitestreamTxid\nfunc GoLitestreamTxid(dbPtr unsafe.Pointer, out **C.char) *C.char {\n\tvalue, err := litestream.GetVFSConnectionTXID(uintptr(dbPtr))\n\tif err != nil {\n\t\treturn C.CString(err.Error())\n\t}\n\tif out != nil {\n\t\t*out = C.CString(value)\n\t}\n\treturn nil\n}\n\n//export GoLitestreamLag\nfunc GoLitestreamLag(dbPtr unsafe.Pointer, out *C.sqlite3_int64) *C.char {\n\tvalue, err := litestream.GetVFSConnectionLag(uintptr(dbPtr))\n\tif err != nil {\n\t\treturn C.CString(err.Error())\n\t}\n\tif out != nil {\n\t\t*out = C.sqlite3_int64(value)\n\t}\n\treturn nil\n}\n" }, { "path": "cmd/litestream-vfs/main_test.go", "content": "//go:build vfs\n// +build vfs\n\npackage main_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"database/sql\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"math/rand\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\tsqlite3 \"github.com/mattn/go-sqlite3\"\n\t\"github.com/psanford/sqlite3vfs\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestVFS_Simple(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tif err := sqlite3vfs.RegisterVFS(\"litestream\", vfs); err != nil {\n\t\tt.Fatalf(\"failed to register litestream vfs: %v\", err)\n\t}\n\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\tdb.MonitorInterval = 100 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb0)\n\n\tif _, err := sqldb0.Exec(\"CREATE TABLE t (x)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb0.Exec(\"INSERT INTO t (x) VALUES (100)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\twaitForLTXFiles(t, client, 10*time.Second, db.MonitorInterval)\n\n\tsqldb1, err := sql.Open(\"sqlite3\", \"file:/tmp/test.db?vfs=litestream\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to open database: %v\", err)\n\t}\n\tdefer sqldb1.Close()\n\n\t// Execute query - wait for value to be replicated\n\twaitForReplicaValue(t, sqldb1, \"SELECT * FROM t\", 100, 10*time.Second, db.MonitorInterval)\n}\n\nfunc TestVFS_Updating(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tif err := sqlite3vfs.RegisterVFS(\"litestream\", vfs); err != nil {\n\t\tt.Fatalf(\"failed to register litestream vfs: %v\", err)\n\t}\n\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\tdb.MonitorInterval = 100 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 100 * time.Millisecond\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb0)\n\n\tt.Log(\"creating table\")\n\tif _, err := sqldb0.Exec(\"CREATE TABLE t (x)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb0.Exec(\"INSERT INTO t (x) VALUES (100)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\twaitForLTXFiles(t, client, 10*time.Second, db.MonitorInterval)\n\n\tt.Log(\"opening vfs\")\n\tsqldb1, err := sql.Open(\"sqlite3\", \"file:/tmp/test.db?vfs=litestream\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to open database: %v\", err)\n\t}\n\tdefer sqldb1.Close()\n\n\t// Wait for initial value to replicate\n\twaitForReplicaValue(t, sqldb1, \"SELECT * FROM t\", 100, 10*time.Second, db.MonitorInterval)\n\n\tt.Log(\"updating source database\")\n\t// Update the value from the source database.\n\tif _, err := sqldb0.Exec(\"UPDATE t SET x = 200\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Ensure replica has updated itself.\n\tt.Log(\"ensuring replica has updated\")\n\twaitForReplicaValue(t, sqldb1, \"SELECT * FROM t\", 200, 10*time.Second, db.MonitorInterval)\n\n\tif err := db.Replica.Stop(false); err != nil {\n\t\tt.Fatalf(\"stop replica: %v\", err)\n\t}\n}\n\nfunc TestVFS_ActiveReadTransaction(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tif err := sqlite3vfs.RegisterVFS(\"litestream-txn\", vfs); err != nil {\n\t\tt.Fatalf(\"failed to register litestream vfs: %v\", err)\n\t}\n\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\tdb.MonitorInterval = 100 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 100 * time.Millisecond\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb0)\n\n\t// Create a table with many rows to ensure we span multiple pages\n\t// With 4KB page size, we want to ensure we're using hundreds of pages\n\tt.Log(\"creating table with many rows\")\n\tif _, err := sqldb0.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, data TEXT)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Insert ~10000 rows, each with substantial data to span many pages\n\t// This should occupy at least 200+ pages (assuming ~200 bytes per row, ~20 rows per 4KB page)\n\tif _, err := sqldb0.Exec(\"BEGIN\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tfor i := 0; i < 10000; i++ {\n\t\tdata := fmt.Sprintf(\"initial_data_%d_padding_%s\", i, string(make([]byte, 100)))\n\t\tif _, err := sqldb0.Exec(\"INSERT INTO t (id, data) VALUES (?, ?)\", i, data); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\tif _, err := sqldb0.Exec(\"COMMIT\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for replication to sync\n\twaitForLTXFiles(t, client, 10*time.Second, db.MonitorInterval)\n\n\tt.Log(\"opening vfs replica\")\n\tsqldb1, err := sql.Open(\"sqlite3\", \"file:/tmp/test-txn.db?vfs=litestream-txn\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to open database: %v\", err)\n\t}\n\tdefer sqldb1.Close()\n\n\t// Start a read transaction on the replica\n\tt.Log(\"starting read transaction on replica\")\n\ttx, err := sqldb1.Begin()\n\tif err != nil {\n\t\tt.Fatalf(\"failed to begin transaction: %v\", err)\n\t}\n\tdefer tx.Rollback()\n\n\t// Verify we can read initial data from within the transaction\n\tvar initialData string\n\tif err := tx.QueryRow(\"SELECT data FROM t WHERE id = 5000\").Scan(&initialData); err != nil {\n\t\tt.Fatalf(\"failed to query initial data: %v\", err)\n\t}\n\tif !strings.HasPrefix(initialData, \"initial_data_5000\") {\n\t\tt.Fatalf(\"unexpected initial data: %s\", initialData)\n\t}\n\n\tt.Log(\"updating source database with many affected pages\")\n\t// Update many rows in the source database to affect many pages\n\tif _, err := sqldb0.Exec(\"BEGIN\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tfor i := 0; i < 10000; i++ {\n\t\tdata := fmt.Sprintf(\"updated_data_%d_padding_%s\", i, string(make([]byte, 100)))\n\t\tif _, err := sqldb0.Exec(\"UPDATE t SET data = ? WHERE id = ?\", data, i); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\tif _, err := sqldb0.Exec(\"COMMIT\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for replication to sync the updates - verify new data is available\n\t// by checking from a fresh connection (not the transaction's snapshot)\n\tt.Log(\"waiting for replication sync\")\n\trequire.Eventually(t, func() bool {\n\t\tvar data string\n\t\t// Use a new query to check if updates have replicated\n\t\tif err := sqldb1.QueryRow(\"SELECT data FROM t WHERE id = 5000\").Scan(&data); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn strings.HasPrefix(data, \"updated_data_5000\")\n\t}, 10*time.Second, db.MonitorInterval, \"updates should replicate\")\n\n\t// The active read transaction should still see old data (snapshot isolation)\n\tt.Log(\"verifying read transaction still sees old data\")\n\tvar txData string\n\tif err := tx.QueryRow(\"SELECT data FROM t WHERE id = 5000\").Scan(&txData); err != nil {\n\t\tt.Fatalf(\"failed to query within transaction: %v\", err)\n\t}\n\tif !strings.HasPrefix(txData, \"initial_data_5000\") {\n\t\tt.Fatalf(\"transaction should see old data, got: %s\", txData)\n\t}\n\n\t// Commit the read transaction\n\tt.Log(\"committing read transaction\")\n\tif err := tx.Commit(); err != nil {\n\t\tt.Fatalf(\"failed to commit transaction: %v\", err)\n\t}\n\n\t// Verify multiple rows across different pages\n\tt.Log(\"verifying multiple rows across pages\")\n\tfor _, id := range []int{0, 2500, 5000, 7500, 9999} {\n\t\tvar data string\n\t\tif err := sqldb1.QueryRow(\"SELECT data FROM t WHERE id = ?\", id).Scan(&data); err != nil {\n\t\t\tt.Fatalf(\"failed to query id %d: %v\", id, err)\n\t\t}\n\t\texpected := fmt.Sprintf(\"updated_data_%d\", id)\n\t\tif !strings.HasPrefix(data, expected) {\n\t\t\tt.Fatalf(\"id %d: expected prefix %s, got: %s\", id, expected, data)\n\t\t}\n\t}\n}\n\nfunc TestVFS_PollsL1Files(t *testing.T) {\n\tctx := context.Background()\n\tclient := file.NewReplicaClient(t.TempDir())\n\n\t// Create and populate source database\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\tdb.MonitorInterval = 100 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 100 * time.Millisecond\n\tdb.Replica.MonitorEnabled = false\n\n\t// Create a store to handle compaction\n\tlevels := litestream.CompactionLevels{\n\t\t{Level: 0},\n\t\t{Level: 1, Interval: 1 * time.Second},\n\t}\n\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\tstore.CompactionMonitorEnabled = false\n\n\tif err := store.Open(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer store.Close(ctx)\n\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb0)\n\n\t// Create table and insert data\n\tt.Log(\"creating table with data\")\n\tif _, err := sqldb0.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, data TEXT)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Insert multiple transactions to create several L0 files\n\tfor i := 0; i < 5; i++ {\n\t\tif _, err := sqldb0.Exec(\"INSERT INTO t (data) VALUES (?)\", fmt.Sprintf(\"value-%d\", i)); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Replica.Sync(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\ttime.Sleep(50 * time.Millisecond) // Small delay between transactions\n\t}\n\n\tt.Log(\"compacting to L1\")\n\t// Compact L0 files to L1\n\tif _, err := store.CompactDB(ctx, db, levels[1]); err != nil {\n\t\tt.Fatalf(\"failed to compact to L1: %v\", err)\n\t}\n\n\t// Verify L1 files exist\n\titr, err := client.LTXFiles(ctx, 1, 0, false)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tvar l1Count int\n\tfor itr.Next() {\n\t\tl1Count++\n\t}\n\titr.Close()\n\n\tif l1Count == 0 {\n\t\tt.Fatal(\"expected L1 files to exist after compaction\")\n\t}\n\tt.Logf(\"found %d L1 file(s)\", l1Count)\n\n\t// Register VFS\n\tvfs := newVFS(t, client)\n\tif err := sqlite3vfs.RegisterVFS(\"litestream-l1\", vfs); err != nil {\n\t\tt.Fatalf(\"failed to register litestream vfs: %v\", err)\n\t}\n\n\t// Open database through VFS\n\tt.Log(\"opening vfs\")\n\tsqldb1, err := sql.Open(\"sqlite3\", \"file:/tmp/test-l1.db?vfs=litestream-l1\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to open database: %v\", err)\n\t}\n\tdefer sqldb1.Close()\n\n\t// Query to ensure data is readable\n\tvar count int\n\tif err := sqldb1.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&count); err != nil {\n\t\tt.Fatalf(\"failed to query database: %v\", err)\n\t} else if got, want := count, 5; got != want {\n\t\tt.Fatalf(\"got %d rows, want %d\", got, want)\n\t}\n\n\t// Get the VFS file to check maxTXID1\n\t// The VFS creates the file when opened, we need to access it\n\t// Since VFS.Open returns the file, we need to track it\n\t// For now, let's add more data and wait for polling\n\n\tt.Log(\"adding more data to source\")\n\t// Add more data to L0 to trigger polling\n\tfor i := 5; i < 10; i++ {\n\t\tif _, err := sqldb0.Exec(\"INSERT INTO t (data) VALUES (?)\", fmt.Sprintf(\"value-%d\", i)); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Replica.Sync(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\t// Close and reopen the VFS connection to see updates\n\t// (VFS is designed for read replicas where clients open new connections)\n\tsqldb1.Close()\n\n\tt.Log(\"reopening vfs to see updates\")\n\tsqldb1, err = sql.Open(\"sqlite3\", \"file:/tmp/test-l1.db?vfs=litestream-l1\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to reopen database: %v\", err)\n\t}\n\tdefer sqldb1.Close()\n\n\t// Wait for VFS to poll new files\n\tt.Log(\"waiting for VFS to poll\")\n\trequire.Eventually(t, func() bool {\n\t\tif err := sqldb1.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&count); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn count == 10\n\t}, 10*time.Second, vfs.PollInterval, \"VFS should poll and see 10 rows\")\n\n\t// Compact the new L0 files to L1\n\t// Use Eventually since compaction has a 1-second interval and first compaction just completed\n\tt.Log(\"compacting new data to L1\")\n\trequire.Eventually(t, func() bool {\n\t\t_, err := store.CompactDB(ctx, db, levels[1])\n\t\treturn err == nil\n\t}, 5*time.Second, 200*time.Millisecond, \"second compaction should succeed after interval passes\")\n\n\t// At this point, the VFS should have polled L1 files\n\t// We can't directly access the VFSFile from here without modifying VFS.Open\n\t// But we can verify the data is readable, which proves L1 files are being used\n\n\t// Query a specific value to ensure L1 data is accessible\n\tvar data string\n\tif err := sqldb1.QueryRow(\"SELECT data FROM t WHERE id = 7\").Scan(&data); err != nil {\n\t\tt.Fatalf(\"failed to query specific row: %v\", err)\n\t} else if got, want := data, \"value-6\"; got != want {\n\t\tt.Fatalf(\"got data %q, want %q\", got, want)\n\t}\n\n\tt.Log(\"L1 file polling verified successfully\")\n}\n\nfunc TestVFS_LongRunningTxnStress(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 25 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\t_, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE metrics (id INTEGER PRIMARY KEY, value INTEGER)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO metrics (id, value) VALUES (1, 0)\"); err != nil {\n\t\tt.Fatalf(\"insert row: %v\", err)\n\t}\n\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\trequire.Eventually(t, func() bool {\n\t\tvar tmp int\n\t\treturn replica.QueryRow(\"SELECT value FROM metrics WHERE id = 1\").Scan(&tmp) == nil\n\t}, 30*time.Second, 50*time.Millisecond, \"replica should observe metrics row\")\n\n\ttx, err := replica.Begin()\n\tif err != nil {\n\t\tt.Fatalf(\"begin replica txn: %v\", err)\n\t}\n\tdefer tx.Rollback()\n\n\tvar initialValue int\n\tif err := tx.QueryRow(\"SELECT value FROM metrics WHERE id = 1\").Scan(&initialValue); err != nil {\n\t\tt.Fatalf(\"initial read: %v\", err)\n\t}\n\n\tctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)\n\tdefer cancel()\n\n\twriterDone := make(chan error, 1)\n\tgo func() {\n\t\tdefer close(writerDone)\n\t\tvalue := 0\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\t\t\tvalue++\n\t\t\tif _, err := primary.Exec(\"UPDATE metrics SET value = ? WHERE id = 1\", value); err != nil {\n\t\t\t\twriterDone <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t}\n\t}()\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\tif err := <-writerDone; err != nil && !errors.Is(err, context.Canceled) {\n\t\t\t\tt.Fatalf(\"writer error: %v\", err)\n\t\t\t}\n\t\t\tgoto done\n\t\tcase <-time.After(50 * time.Millisecond):\n\t\t\tvar v int\n\t\t\tif err := tx.QueryRow(\"SELECT value FROM metrics WHERE id = 1\").Scan(&v); err != nil {\n\t\t\t\tt.Fatalf(\"read during txn: %v\", err)\n\t\t\t}\n\t\t\tif v != initialValue {\n\t\t\t\tt.Fatalf(\"long-running txn observed change: got %d want %d\", v, initialValue)\n\t\t\t}\n\t\t}\n\t}\n\ndone:\n\tif err := tx.Commit(); err != nil {\n\t\tt.Fatalf(\"commit: %v\", err)\n\t}\n\n\tvar finalValue int\n\tif err := replica.QueryRow(\"SELECT value FROM metrics WHERE id = 1\").Scan(&finalValue); err != nil {\n\t\tt.Fatalf(\"post-commit read: %v\", err)\n\t}\n\tif finalValue == initialValue {\n\t\tt.Fatalf(\"expected updated value after commit\")\n\t}\n}\n\nfunc TestVFS_HighLoadConcurrentReads(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping high-load test in short mode\")\n\t}\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 50 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\t_, primary := openReplicatedPrimary(t, client, 50*time.Millisecond, 50*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(`CREATE TABLE t (\n\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\tvalue TEXT,\n\t\tupdated_at INTEGER\n\t)`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\tseedLargeTable(t, primary, 2000)\n\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\tif _, err := replica.Exec(\"PRAGMA temp_store = MEMORY\"); err != nil {\n\t\tt.Fatalf(\"set temp_store: %v\", err)\n\t}\n\n\twaitForReplicaRowCount(t, primary, replica, 30*time.Second)\n\n\tctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n\tdefer cancel()\n\n\tvar writerOps atomic.Int64\n\twriterErr := make(chan error, 1)\n\tgo func() {\n\t\tdefer close(writerErr)\n\t\trnd := rand.New(rand.NewSource(time.Now().UnixNano()))\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\twriterErr <- nil\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\n\t\t\tswitch rnd.Intn(3) {\n\t\t\tcase 0:\n\t\t\t\tif _, err := primary.Exec(\"INSERT INTO t (value, updated_at) VALUES (?, strftime('%s','now'))\", fmt.Sprintf(\"value-%d\", rnd.Int())); err != nil {\n\t\t\t\t\twriterErr <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\tcase 1:\n\t\t\t\tif _, err := primary.Exec(\"UPDATE t SET value = value || '-u' WHERE id IN (SELECT id FROM t ORDER BY RANDOM() LIMIT 1)\"); err != nil {\n\t\t\t\t\twriterErr <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\tdefault:\n\t\t\t\tif _, err := primary.Exec(\"DELETE FROM t WHERE id IN (SELECT id FROM t ORDER BY RANDOM() LIMIT 1)\"); err != nil {\n\t\t\t\t\twriterErr <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\n\t\t\twriterOps.Add(1)\n\t\t\ttime.Sleep(time.Duration(rnd.Intn(5)+1) * time.Millisecond)\n\t\t}\n\t}()\n\n\treaderErrCh := make(chan error, 1)\n\tvar readerWg sync.WaitGroup\n\tfor i := 0; i < 8; i++ {\n\t\treaderWg.Add(1)\n\t\tgo func(id int) {\n\t\t\tdefer readerWg.Done()\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\tdefault:\n\t\t\t\t}\n\n\t\t\t\tvar count int\n\t\t\t\tvar totalBytes int\n\t\t\t\tif err := replica.QueryRow(\"SELECT COUNT(*), IFNULL(SUM(LENGTH(value)), 0) FROM t\").Scan(&count, &totalBytes); err != nil {\n\t\t\t\t\treaderErrCh <- fmt.Errorf(\"reader %d query: %w\", id, err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif count < 0 || totalBytes < 0 {\n\t\t\t\t\treaderErrCh <- fmt.Errorf(\"reader %d observed invalid stats\", id)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}(i)\n\t}\n\n\t<-ctx.Done()\n\treaderWg.Wait()\n\n\tif err := <-writerErr; err != nil && !errors.Is(err, context.Canceled) {\n\t\tt.Fatalf(\"writer error: %v\", err)\n\t}\n\tselect {\n\tcase err := <-readerErrCh:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"reader error: %v\", err)\n\t\t}\n\tdefault:\n\t}\n\n\tif ops := writerOps.Load(); ops < 100 {\n\t\tt.Fatalf(\"expected high write volume, got %d ops\", ops)\n\t}\n\n\twaitForReplicaRowCount(t, primary, replica, 30*time.Second)\n\n\tvar primaryCount, replicaCount int\n\tif err := primary.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&primaryCount); err != nil {\n\t\tt.Fatalf(\"primary count: %v\", err)\n\t}\n\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&replicaCount); err != nil {\n\t\tt.Fatalf(\"replica count: %v\", err)\n\t}\n\tif primaryCount != replicaCount {\n\t\tt.Fatalf(\"replica lagging: primary=%d replica=%d\", primaryCount, replicaCount)\n\t}\n}\n\n// TestVFS_OverlappingTransactionCommitStorm tests that the VFS can handle\n// concurrent read operations while writes are happening on the primary.\n// The test verifies that the replica eventually catches up with the primary.\nfunc TestVFS_OverlappingTransactionCommitStorm(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tconst interval = 25 * time.Millisecond\n\tdb, primary := openReplicatedPrimary(t, client, interval, interval)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE ledger (id INTEGER PRIMARY KEY AUTOINCREMENT, account INTEGER, amount INTEGER, created_at INTEGER)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO ledger (account, amount, created_at) VALUES (1, 0, strftime('%s','now'))\"); err != nil {\n\t\tt.Fatalf(\"seed ledger: %v\", err)\n\t}\n\n\t// Wait for LTX files to be created before opening VFS replica\n\trequire.Eventually(t, func() bool {\n\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t\tdefer itr.Close()\n\t\treturn itr.Next()\n\t}, 5*time.Second, db.MonitorInterval, \"LTX files should be created\")\n\tforceReplicaSync(t, db)\n\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = interval\n\tvfsName := registerTestVFS(t, vfs)\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\t// Verify initial sync\n\trequire.Eventually(t, func() bool {\n\t\tvar primaryCount int\n\t\tif err := primary.QueryRow(\"SELECT COUNT(*) FROM ledger\").Scan(&primaryCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\tvar replicaCount int\n\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM ledger\").Scan(&replicaCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn primaryCount == replicaCount\n\t}, time.Minute, 25*time.Millisecond, \"ledger counts should match initially\")\n\n\t// Run concurrent writers for a short period (reduced from 10s to 3s)\n\tctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)\n\tdefer cancel()\n\tvar writerWG sync.WaitGroup\n\twriter := func(account int) {\n\t\tdefer writerWG.Done()\n\t\trnd := rand.New(rand.NewSource(time.Now().UnixNano() + int64(account)))\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\t\t\tamount := rnd.Intn(200) - 100\n\t\t\tif _, err := primary.Exec(\"BEGIN IMMEDIATE\"); err != nil {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif _, err := primary.Exec(\"INSERT INTO ledger (account, amount, created_at) VALUES (?, ?, strftime('%s','now'))\", account, amount); err != nil {\n\t\t\t\tprimary.Exec(\"ROLLBACK\")\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif _, err := primary.Exec(\"COMMIT\"); err != nil {\n\t\t\t\tprimary.Exec(\"ROLLBACK\")\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t// Slow down writes to allow background monitor to keep up\n\t\t\ttime.Sleep(time.Duration(rnd.Intn(20)+10) * time.Millisecond)\n\t\t}\n\t}\n\twriterWG.Add(2)\n\tgo writer(1)\n\tgo writer(2)\n\n\t// Run concurrent reader that verifies count never goes to zero\n\treaderCtx, readerCancel := context.WithCancel(ctx)\n\treaderErr := make(chan error, 1)\n\tvar readerWG sync.WaitGroup\n\treaderWG.Add(1)\n\tgo func() {\n\t\tdefer readerWG.Done()\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-readerCtx.Done():\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\t\t\tvar count int\n\t\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM ledger\").Scan(&count); err != nil {\n\t\t\t\treaderErr <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif count == 0 {\n\t\t\t\treaderErr <- fmt.Errorf(\"ledger count went to zero\")\n\t\t\t\treturn\n\t\t\t}\n\t\t\ttime.Sleep(25 * time.Millisecond)\n\t\t}\n\t}()\n\n\t<-ctx.Done()\n\treaderCancel()\n\twriterWG.Wait()\n\treaderWG.Wait()\n\n\t// Check for reader errors\n\tselect {\n\tcase err := <-readerErr:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"reader error: %v\", err)\n\t\t}\n\tdefault:\n\t}\n\n\t// Force final sync and wait for replica to catch up\n\tforceReplicaSync(t, db)\n\n\trequire.Eventually(t, func() bool {\n\t\tvar primaryCount int\n\t\tif err := primary.QueryRow(\"SELECT COUNT(*) FROM ledger\").Scan(&primaryCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\tvar replicaCount int\n\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM ledger\").Scan(&replicaCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn primaryCount == replicaCount\n\t}, 30*time.Second, 100*time.Millisecond, \"ledger counts should match after writer done\")\n}\n\nfunc TestVFS_CacheMissStorm(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tconst interval = 20 * time.Millisecond\n\t_, primary := openReplicatedPrimary(t, client, interval, interval)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE stats (id INTEGER PRIMARY KEY, payload TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tfor i := 0; i < 1000; i++ {\n\t\tif _, err := primary.Exec(\"INSERT INTO stats (payload) VALUES (?)\", fmt.Sprintf(\"row-%d\", i)); err != nil {\n\t\t\tt.Fatalf(\"insert payload: %v\", err)\n\t\t}\n\t}\n\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = interval\n\tvfsName := registerTestVFS(t, vfs)\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\twaitForTableRowCount(t, primary, replica, \"stats\", 30*time.Second)\n\n\tif _, err := replica.Exec(\"PRAGMA cache_size = -64\"); err != nil {\n\t\tt.Fatalf(\"set cache_size: %v\", err)\n\t}\n\tif _, err := replica.Exec(\"PRAGMA cache_spill = ON\"); err != nil {\n\t\tt.Fatalf(\"enable cache_spill: %v\", err)\n\t}\n\n\tfor i := 0; i < 100; i++ {\n\t\tvar maxID int\n\t\tif err := replica.QueryRow(\"SELECT MAX(id) FROM stats\").Scan(&maxID); err != nil {\n\t\t\tt.Fatalf(\"cache-miss query: %v\", err)\n\t\t}\n\t\tif maxID == 0 {\n\t\t\tt.Fatalf(\"unexpected empty stats table\")\n\t\t}\n\t}\n}\n\nfunc BenchmarkVFS_LargeDatabase(b *testing.B) {\n\tif testing.Short() {\n\t\tb.Skip(\"skipping large benchmark in short mode\")\n\t}\n\tclient := file.NewReplicaClient(b.TempDir())\n\tdb, primary := openReplicatedPrimary(b, client, 25*time.Millisecond, 25*time.Millisecond)\n\tb.Cleanup(func() { testingutil.MustCloseSQLDB(b, primary) })\n\n\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY AUTOINCREMENT, value TEXT, updated_at INTEGER)\"); err != nil {\n\t\tb.Fatalf(\"create table: %v\", err)\n\t}\n\tseedLargeTable(b, primary, 20000)\n\tforceReplicaSync(b, db)\n\tif err := db.Replica.Stop(false); err != nil {\n\t\tb.Fatalf(\"stop replica: %v\", err)\n\t}\n\n\tvfs := newVFS(b, client)\n\tvfs.PollInterval = 25 * time.Millisecond\n\tvfsName := registerTestVFS(b, vfs)\n\treplica := openVFSReplicaDB(b, vfsName)\n\tb.Cleanup(func() { replica.Close() })\n\twaitForReplicaRowCount(b, primary, replica, 30*time.Second)\n\n\tb.ReportAllocs()\n\tb.ResetTimer()\n\tfor i := 0; i < b.N; i++ {\n\t\tvar count, totalBytes int\n\t\tif err := replica.QueryRow(\"SELECT COUNT(*), IFNULL(SUM(LENGTH(value)), 0) FROM t\").Scan(&count, &totalBytes); err != nil {\n\t\t\tb.Fatalf(\"benchmark query: %v\", err)\n\t\t}\n\t}\n}\n\nfunc TestVFS_NetworkLatencySensitivity(t *testing.T) {\n\tclient := &latencyReplicaClient{ReplicaClient: file.NewReplicaClient(t.TempDir()), delay: 10 * time.Millisecond}\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 25 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\t_, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE logs (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO logs (value) VALUES ('ok')\"); err != nil {\n\t\tt.Fatalf(\"insert row: %v\", err)\n\t}\n\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\trequire.Eventually(t, func() bool {\n\t\tvar count int\n\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM logs\").Scan(&count); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn count == 1\n\t}, 10*time.Second, 50*time.Millisecond, \"replica should observe log row under injected latency\")\n}\n\nfunc TestVFS_ConcurrentConnectionScaling(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 25 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\tdb, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE metrics (id INTEGER PRIMARY KEY AUTOINCREMENT, value INTEGER)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tfor i := 0; i < 1000; i++ {\n\t\tif _, err := primary.Exec(\"INSERT INTO metrics (value) VALUES (?)\", i); err != nil {\n\t\t\tt.Fatalf(\"insert row: %v\", err)\n\t\t}\n\t}\n\n\t// Wait for LTX files to be created before forceReplicaSync\n\trequire.Eventually(t, func() bool {\n\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t\tdefer itr.Close()\n\t\treturn itr.Next()\n\t}, 5*time.Second, db.MonitorInterval, \"LTX files should be created\")\n\tforceReplicaSync(t, db)\n\n\tconst connCount = 32\n\tconns := make([]*sql.DB, connCount)\n\tfor i := 0; i < connCount; i++ {\n\t\tconns[i] = openVFSReplicaDB(t, vfsName)\n\t}\n\tdefer func() {\n\t\tfor _, c := range conns {\n\t\t\tc.Close()\n\t\t}\n\t}()\n\n\tctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)\n\tdefer cancel()\n\tvar wg sync.WaitGroup\n\tfor idx := range conns {\n\t\twg.Add(1)\n\t\tgo func(id int, dbConn *sql.DB) {\n\t\t\tdefer wg.Done()\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\tdefault:\n\t\t\t\t}\n\t\t\t\tvar min, max int\n\t\t\t\tif err := dbConn.QueryRow(\"SELECT MIN(value), MAX(value) FROM metrics\").Scan(&min, &max); err != nil {\n\t\t\t\t\tt.Errorf(\"conn %d query: %v\", id, err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}(idx, conns[idx])\n\t}\n\n\twg.Wait()\n\tif err := ctx.Err(); err != context.Canceled && err != context.DeadlineExceeded {\n\t\tt.Fatalf(\"unexpected context err: %v\", err)\n\t}\n}\n\nfunc TestVFS_PRAGMAQueryBehavior(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 25 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\tdb, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE configs (id INTEGER PRIMARY KEY, name TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO configs (name) VALUES ('ok')\"); err != nil {\n\t\tt.Fatalf(\"insert row: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table t: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES ('seed')\"); err != nil {\n\t\tt.Fatalf(\"seed t: %v\", err)\n\t}\n\n\t// Wait for LTX files to be created before forceReplicaSync\n\trequire.Eventually(t, func() bool {\n\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t\tdefer itr.Close()\n\t\treturn itr.Next()\n\t}, 5*time.Second, db.MonitorInterval, \"LTX files should be created\")\n\tforceReplicaSync(t, db)\n\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\twaitForReplicaRowCount(t, primary, replica, 30*time.Second)\n\n\tvar journalMode string\n\tif err := replica.QueryRow(\"PRAGMA journal_mode\").Scan(&journalMode); err != nil {\n\t\tt.Fatalf(\"read journal_mode: %v\", err)\n\t}\n\tif strings.ToLower(journalMode) != \"delete\" {\n\t\tt.Fatalf(\"expected journal_mode delete, got %s\", journalMode)\n\t}\n\n\tif _, err := replica.Exec(\"PRAGMA cache_size = -2048\"); err != nil {\n\t\tt.Fatalf(\"set cache_size: %v\", err)\n\t}\n\tvar cacheSize int\n\tif err := replica.QueryRow(\"PRAGMA cache_size\").Scan(&cacheSize); err != nil {\n\t\tt.Fatalf(\"read cache_size: %v\", err)\n\t}\n\tif cacheSize != -2048 {\n\t\tt.Fatalf(\"unexpected cache_size: %d\", cacheSize)\n\t}\n\n\tvar pageSize int\n\tif err := replica.QueryRow(\"PRAGMA page_size\").Scan(&pageSize); err != nil {\n\t\tt.Fatalf(\"read page_size: %v\", err)\n\t}\n\tif pageSize != 4096 {\n\t\tt.Fatalf(\"unexpected page_size: %d\", pageSize)\n\t}\n}\n\nfunc TestVFS_SortingLargeResultSet(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 50 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\tdb, primary := openReplicatedPrimary(t, client, 50*time.Millisecond, 50*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(`CREATE TABLE t (\n\t\tid INTEGER PRIMARY KEY,\n\t\tpayload TEXT NOT NULL,\n\t\tgrp INTEGER NOT NULL\n\t)`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\tseedSortedDataset(t, primary, 25000)\n\tif err := db.Replica.Stop(false); err != nil {\n\t\tt.Fatalf(\"stop replica: %v\", err)\n\t}\n\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\tif _, err := replica.Exec(\"PRAGMA temp_store = FILE\"); err != nil {\n\t\tt.Fatalf(\"set temp_store: %v\", err)\n\t}\n\tif _, err := replica.Exec(\"PRAGMA cache_size = -2048\"); err != nil {\n\t\tt.Fatalf(\"set cache_size: %v\", err)\n\t}\n\n\twaitForReplicaRowCount(t, primary, replica, time.Minute)\n\n\texpected := fetchOrderedPayloads(t, primary, 500, \"payload DESC, id DESC\")\n\tgot := fetchOrderedPayloads(t, replica, 500, \"payload DESC, id DESC\")\n\n\tif len(expected) != len(got) {\n\t\tt.Fatalf(\"unexpected result size: expected=%d got=%d\", len(expected), len(got))\n\t}\n\tfor i := range expected {\n\t\tif expected[i] != got[i] {\n\t\t\tt.Fatalf(\"mismatched payload at %d: expected=%q got=%q\", i, expected[i], got[i])\n\t\t}\n\t}\n}\n\nfunc TestVFS_ConcurrentIndexAccessRaces(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tconst monitorInterval = 10 * time.Millisecond\n\t_, primary := openReplicatedPrimary(t, client, monitorInterval, 10*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY AUTOINCREMENT, value TEXT, updated_at INTEGER)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tseedLargeTable(t, primary, 10000)\n\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 15 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\tdsn := fmt.Sprintf(\"file:%s?vfs=%s\", filepath.ToSlash(filepath.Join(t.TempDir(), \"fail.db\")), vfsName)\n\treplica, err := sql.Open(\"sqlite3\", dsn)\n\tif err != nil {\n\t\tt.Fatalf(\"open replica db: %v\", err)\n\t}\n\tdefer replica.Close()\n\treplica.SetMaxOpenConns(4)\n\treplica.SetMaxIdleConns(4)\n\treplica.SetConnMaxIdleTime(30 * time.Second)\n\n\twaitForReplicaRowCount(t, primary, replica, 30*time.Second)\n\n\tctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)\n\tdefer cancel()\n\n\treaderErrCh := make(chan error, 1)\n\tvar readerWG sync.WaitGroup\n\tfor i := 0; i < 100; i++ {\n\t\treaderWG.Add(1)\n\t\tgo func(id int) {\n\t\t\tdefer readerWG.Done()\n\t\t\trnd := rand.New(rand.NewSource(int64(id) + time.Now().UnixNano()))\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\tdefault:\n\t\t\t\t}\n\n\t\t\t\tvar count int\n\t\t\t\tvar totalBytes int\n\t\t\t\tif err := replica.QueryRow(\"SELECT COUNT(*), IFNULL(SUM(LENGTH(value)), 0) FROM t\").Scan(&count, &totalBytes); err != nil {\n\t\t\t\t\tselect {\n\t\t\t\t\tcase readerErrCh <- fmt.Errorf(\"reader %d: %w\", id, err):\n\t\t\t\t\tdefault:\n\t\t\t\t\t}\n\t\t\t\t\tcancel()\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif count < 0 || totalBytes < 0 {\n\t\t\t\t\tselect {\n\t\t\t\t\tcase readerErrCh <- fmt.Errorf(\"reader %d observed invalid stats\", id):\n\t\t\t\t\tdefault:\n\t\t\t\t\t}\n\t\t\t\t\tcancel()\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\t_ = rnd.Int() // exercise RNG to vary workload\n\t\t\t}\n\t\t}(i)\n\t}\n\n\tvar writerOps atomic.Int64\n\twriterErrCh := make(chan error, 1)\n\tgo func() {\n\t\tdefer close(writerErrCh)\n\t\trnd := rand.New(rand.NewSource(time.Now().UnixNano()))\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\n\t\t\tswitch rnd.Intn(3) {\n\t\t\tcase 0:\n\t\t\t\t_, err := primary.Exec(\"INSERT INTO t (value, updated_at) VALUES (?, strftime('%s','now'))\", fmt.Sprintf(\"writer-%d\", rnd.Int()))\n\t\t\t\tif err != nil {\n\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\twriterErrCh <- err\n\t\t\t\t\tcancel()\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\tcase 1:\n\t\t\t\t_, err := primary.Exec(\"UPDATE t SET value = value || '-u', updated_at = strftime('%s','now') WHERE id IN (SELECT id FROM t ORDER BY RANDOM() LIMIT 1)\")\n\t\t\t\tif err != nil {\n\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\twriterErrCh <- err\n\t\t\t\t\tcancel()\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\tdefault:\n\t\t\t\t_, err := primary.Exec(\"DELETE FROM t WHERE id IN (SELECT id FROM t ORDER BY RANDOM() LIMIT 1)\")\n\t\t\t\tif err != nil {\n\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\twriterErrCh <- err\n\t\t\t\t\tcancel()\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t\twriterOps.Add(1)\n\t\t\ttime.Sleep(time.Duration(rnd.Intn(5)+1) * time.Millisecond)\n\t\t}\n\t}()\n\n\t<-ctx.Done()\n\treaderWG.Wait()\n\tif err := <-writerErrCh; err != nil && !errors.Is(err, context.Canceled) {\n\t\tt.Fatalf(\"writer error: %v\", err)\n\t}\n\tselect {\n\tcase err := <-readerErrCh:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"reader error: %v\", err)\n\t\t}\n\tdefault:\n\t}\n\n\tif ops := writerOps.Load(); ops == 0 {\n\t\tt.Fatalf(\"writer did not perform any operations\")\n\t}\n}\n\nfunc TestVFS_MultiplePageSizes(t *testing.T) {\n\tpageSizes := []int{512, 1024, 2048, 4096, 8192, 16384, 32768, 65536}\n\tfor _, pageSize := range pageSizes {\n\t\tpageSize := pageSize\n\t\tconst monitorInterval = 50 * time.Millisecond\n\t\tt.Run(fmt.Sprintf(\"page_%d\", pageSize), func(t *testing.T) {\n\t\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\t\t_, primary := openReplicatedPrimary(t, client, monitorInterval, 50*time.Millisecond)\n\t\t\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\t\t\tif _, err := primary.Exec(\"PRAGMA journal_mode=DELETE\"); err != nil {\n\t\t\t\tt.Fatalf(\"disable wal: %v\", err)\n\t\t\t}\n\t\t\tif _, err := primary.Exec(fmt.Sprintf(\"PRAGMA page_size = %d\", pageSize)); err != nil {\n\t\t\t\tt.Fatalf(\"set page size: %v\", err)\n\t\t\t}\n\t\t\tif _, err := primary.Exec(\"VACUUM\"); err != nil {\n\t\t\t\tt.Fatalf(\"vacuum: %v\", err)\n\t\t\t}\n\t\t\tif _, err := primary.Exec(\"PRAGMA journal_mode=WAL\"); err != nil {\n\t\t\t\tt.Fatalf(\"enable wal: %v\", err)\n\t\t\t}\n\n\t\t\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, payload TEXT)\"); err != nil {\n\t\t\t\tt.Fatalf(\"create table: %v\", err)\n\t\t\t}\n\n\t\t\tconst totalRows = 200\n\t\t\tif _, err := primary.Exec(\"BEGIN\"); err != nil {\n\t\t\t\tt.Fatalf(\"begin tx: %v\", err)\n\t\t\t}\n\t\t\tfor i := 0; i < totalRows; i++ {\n\t\t\t\tpayload := pageSizedPayload(pageSize, i)\n\t\t\t\tif _, err := primary.Exec(\"INSERT INTO t (payload) VALUES (?)\", payload); err != nil {\n\t\t\t\t\tprimary.Exec(\"ROLLBACK\")\n\t\t\t\t\tt.Fatalf(\"insert row %d: %v\", i, err)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif _, err := primary.Exec(\"COMMIT\"); err != nil {\n\t\t\t\tt.Fatalf(\"commit: %v\", err)\n\t\t\t}\n\n\t\t\tvfs := newVFS(t, client)\n\t\t\tvfs.PollInterval = 50 * time.Millisecond\n\t\t\tvfsName := registerTestVFS(t, vfs)\n\t\t\treplica := openVFSReplicaDB(t, vfsName)\n\t\t\tdefer replica.Close()\n\n\t\t\twaitForReplicaRowCount(t, primary, replica, 30*time.Second)\n\n\t\t\tvar replicaPageSize int\n\t\t\tif err := replica.QueryRow(\"PRAGMA page_size\").Scan(&replicaPageSize); err != nil {\n\t\t\t\tt.Fatalf(\"read replica page size: %v\", err)\n\t\t\t}\n\t\t\tif replicaPageSize != pageSize {\n\t\t\t\tt.Fatalf(\"unexpected page size: got %d want %d\", replicaPageSize, pageSize)\n\t\t\t}\n\n\t\t\trows, err := replica.Query(\"SELECT id, payload FROM t ORDER BY id\")\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"select rows: %v\", err)\n\t\t\t}\n\t\t\tdefer rows.Close()\n\n\t\t\tcount := 0\n\t\t\tfor rows.Next() {\n\t\t\t\tvar id int\n\t\t\t\tvar payload string\n\t\t\t\tif err := rows.Scan(&id, &payload); err != nil {\n\t\t\t\t\tt.Fatalf(\"scan row: %v\", err)\n\t\t\t\t}\n\t\t\t\texpected := pageSizedPayload(pageSize, id-1)\n\t\t\t\tif payload != expected {\n\t\t\t\t\tt.Fatalf(\"row %d mismatch: got %q want %q\", id, payload, expected)\n\t\t\t\t}\n\t\t\t\tcount++\n\t\t\t}\n\t\t\tif err := rows.Err(); err != nil {\n\t\t\t\tt.Fatalf(\"rows err: %v\", err)\n\t\t\t}\n\t\t\tif count != totalRows {\n\t\t\t\tt.Fatalf(\"unexpected row count: got %d want %d\", count, totalRows)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVFS_WaitsForInitialSnapshot(t *testing.T) {\n\tt.Run(\"BlocksUntilSnapshot\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tvfs := newVFS(t, client)\n\t\tvfs.PollInterval = 50 * time.Millisecond\n\t\tvfsName := registerTestVFS(t, vfs)\n\t\tdsn := fmt.Sprintf(\"file:%s?vfs=%s\", filepath.ToSlash(filepath.Join(t.TempDir(), \"wait.db\")), vfsName)\n\n\t\terrCh := make(chan error, 1)\n\t\tgo func() {\n\t\t\tsqldb, err := sql.Open(\"sqlite3\", dsn)\n\t\t\tif err != nil {\n\t\t\t\terrCh <- fmt.Errorf(\"open replica: %w\", err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tdefer sqldb.Close()\n\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n\t\t\tdefer cancel()\n\n\t\t\tvar count int\n\t\t\tif err := sqldb.QueryRowContext(ctx, \"SELECT COUNT(*) FROM sqlite_master\").Scan(&count); err != nil {\n\t\t\t\terrCh <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\terrCh <- nil\n\t\t}()\n\n\t\tselect {\n\t\tcase err := <-errCh:\n\t\t\tt.Fatalf(\"replica should block until snapshot is available, got %v\", err)\n\t\tcase <-time.After(200 * time.Millisecond):\n\t\t}\n\n\t\t_, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\t\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\t\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY)\"); err != nil {\n\t\t\tt.Fatalf(\"create table: %v\", err)\n\t\t}\n\t\tif _, err := primary.Exec(\"INSERT INTO t (id) VALUES (1)\"); err != nil {\n\t\t\tt.Fatalf(\"insert row: %v\", err)\n\t\t}\n\n\t\tselect {\n\t\tcase err := <-errCh:\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"replica query failed: %v\", err)\n\t\t\t}\n\t\tcase <-time.After(5 * time.Second):\n\t\t\tt.Fatal(\"timed out waiting for replica to observe initial snapshot\")\n\t\t}\n\t})\n\n}\n\nfunc TestVFS_StorageFailureInjection(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tmode string\n\t}{\n\t\t{\"timeout\", \"timeout\"},\n\t\t{\"server_error\", \"server\"},\n\t\t{\"partial_read\", \"partial\"},\n\t\t{\"corrupt_data\", \"corrupt\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\t\tdb, primary := openReplicatedPrimary(t, client, 50*time.Millisecond, 50*time.Millisecond)\n\t\t\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\t\t\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\t\t\tt.Fatalf(\"create table: %v\", err)\n\t\t\t}\n\t\t\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES ('ok')\"); err != nil {\n\t\t\t\tt.Fatalf(\"insert row: %v\", err)\n\t\t\t}\n\t\t\t// Wait for LTX files to be written by background monitor\n\t\t\trequire.Eventually(t, func() bool {\n\t\t\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t\tdefer itr.Close()\n\t\t\t\treturn itr.Next()\n\t\t\t}, 5*time.Second, db.MonitorInterval, \"LTX files should be created by background monitor\")\n\t\t\tforceReplicaSync(t, db)\n\t\t\tif err := db.Replica.Stop(false); err != nil {\n\t\t\t\tt.Fatalf(\"stop replica: %v\", err)\n\t\t\t}\n\n\t\t\tvfs := newVFS(t, client)\n\t\t\tvfs.PollInterval = time.Hour\n\t\t\tvfsName := registerTestVFS(t, vfs)\n\t\t\treplicaPath := filepath.Join(t.TempDir(), fmt.Sprintf(\"storage-failure-%s.db\", tt.name))\n\t\t\tdsn := fmt.Sprintf(\"file:%s?vfs=%s\", filepath.ToSlash(replicaPath), vfsName)\n\t\t\treplica, err := sql.Open(\"sqlite3\", dsn)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"open replica db: %v\", err)\n\t\t\t}\n\t\t\tdefer replica.Close()\n\t\t\treplica.SetMaxOpenConns(4)\n\t\t\treplica.SetMaxIdleConns(4)\n\t\t\treplica.SetConnMaxIdleTime(30 * time.Second)\n\t\t\tif _, err := replica.Exec(\"PRAGMA busy_timeout = 2000\"); err != nil {\n\t\t\t\tt.Fatalf(\"set busy timeout: %v\", err)\n\t\t\t}\n\n\t\t\tinjectFailure := func() {\n\t\t\t\tvar err error\n\t\t\t\tswitch tt.mode {\n\t\t\t\tcase \"timeout\":\n\t\t\t\t\terr = context.DeadlineExceeded\n\t\t\t\tcase \"server\":\n\t\t\t\t\terr = fmt.Errorf(\"storage error: 500 Internal Server Error\")\n\t\t\t\tcase \"partial\":\n\t\t\t\t\terr = io.ErrUnexpectedEOF\n\t\t\t\tcase \"corrupt\":\n\t\t\t\t\terr = fmt.Errorf(\"corrupt data\")\n\t\t\t\tdefault:\n\t\t\t\t\terr = fmt.Errorf(\"injected failure\")\n\t\t\t\t}\n\t\t\t\tvfs.Inject(replicaPath, err)\n\t\t\t}\n\n\t\t\tinjectFailure()\n\t\t\tvar val string\n\t\t\tif err := replica.QueryRow(\"SELECT value FROM t\").Scan(&val); err == nil {\n\t\t\t\tt.Fatalf(\"expected failure due to injected storage error\")\n\t\t\t}\n\n\t\t\tif err := replica.QueryRow(\"SELECT value FROM t\").Scan(&val); err != nil {\n\t\t\t\tt.Fatalf(\"second read failed: %v\", err)\n\t\t\t}\n\t\t\tif val != \"ok\" {\n\t\t\t\tt.Fatalf(\"unexpected row value: %q\", val)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVFS_PartialLTXUpload(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tdb, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE logs (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO logs (value) VALUES ('ok')\"); err != nil {\n\t\tt.Fatalf(\"insert row: %v\", err)\n\t}\n\n\t// Wait for LTX files to be created before forceReplicaSync\n\trequire.Eventually(t, func() bool {\n\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t\tdefer itr.Close()\n\t\treturn itr.Next()\n\t}, 5*time.Second, db.MonitorInterval, \"LTX files should be created\")\n\tforceReplicaSync(t, db)\n\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = time.Hour\n\tvfsName := registerTestVFS(t, vfs)\n\treplicaPath := filepath.Join(t.TempDir(), \"partial.db\")\n\tdsn := fmt.Sprintf(\"file:%s?vfs=%s\", filepath.ToSlash(replicaPath), vfsName)\n\treplica, err := sql.Open(\"sqlite3\", dsn)\n\tif err != nil {\n\t\tt.Fatalf(\"open replica db: %v\", err)\n\t}\n\tdefer replica.Close()\n\treplica.SetMaxOpenConns(8)\n\treplica.SetMaxIdleConns(8)\n\treplica.SetConnMaxIdleTime(30 * time.Second)\n\tif _, err := replica.Exec(\"PRAGMA busy_timeout = 2000\"); err != nil {\n\t\tt.Fatalf(\"set busy timeout: %v\", err)\n\t}\n\n\tvfs.Inject(replicaPath, io.ErrUnexpectedEOF)\n\tvar val string\n\tif err := replica.QueryRow(\"SELECT value FROM logs\").Scan(&val); err == nil {\n\t\tt.Fatalf(\"expected failure due to partial upload\")\n\t}\n\n\tif err := replica.QueryRow(\"SELECT value FROM logs\").Scan(&val); err != nil {\n\t\tt.Fatalf(\"second attempt should succeed: %v\", err)\n\t}\n\tif val != \"ok\" {\n\t\tt.Fatalf(\"unexpected row value: %q\", val)\n\t}\n}\n\nfunc TestVFS_S3EventualConsistency(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\t_, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES ('visible')\"); err != nil {\n\t\tt.Fatalf(\"insert row: %v\", err)\n\t}\n\n\teventualClient := &eventualConsistencyClient{ReplicaClient: client}\n\tvfs := newVFS(t, eventualClient)\n\tvfs.PollInterval = 25 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\twaitForReplicaRowCount(t, primary, replica, 5*time.Second)\n\n\tif calls := eventualClient.calls.Load(); calls < 2 {\n\t\tt.Fatalf(\"expected multiple polls under eventual consistency, got %d\", calls)\n\t}\n}\n\nfunc TestVFS_FileDescriptorBudget(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\t_, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES ('seed')\"); err != nil {\n\t\tt.Fatalf(\"insert seed: %v\", err)\n\t}\n\n\tlimited := &fdLimitedReplicaClient{ReplicaClient: client, limit: 64}\n\tvfs := newVFS(t, limited)\n\tvfs.PollInterval = 10 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\twaitForReplicaRowCount(t, primary, replica, 5*time.Second)\n\n\tctx, cancel := context.WithTimeout(context.Background(), 1500*time.Millisecond)\n\tdefer cancel()\n\n\twriterDone := make(chan error, 1)\n\tgo func() {\n\t\tdefer close(writerDone)\n\t\trnd := rand.New(rand.NewSource(time.Now().UnixNano()))\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\t\t\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES (?)\", fmt.Sprintf(\"v-%d\", rnd.Int())); err != nil {\n\t\t\t\tif isBusyError(err) {\n\t\t\t\t\ttime.Sleep(2 * time.Millisecond)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\twriterDone <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\ttime.Sleep(20 * time.Millisecond)\n\t\t}\n\t}()\n\n\tconst readers = 8\n\terrs := make(chan error, readers)\n\tfor i := 0; i < readers; i++ {\n\t\tgo func() {\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\terrs <- nil\n\t\t\t\t\treturn\n\t\t\t\tdefault:\n\t\t\t\t}\n\t\t\t\tvar count int\n\t\t\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&count); err != nil {\n\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\ttime.Sleep(2 * time.Millisecond)\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\terrs <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\t}\n\n\t<-ctx.Done()\n\tfor i := 0; i < readers; i++ {\n\t\tif err := <-errs; err != nil {\n\t\t\tt.Fatalf(\"reader %d error: %T %v\", i, err, err)\n\t\t}\n\t}\n\tif err := <-writerDone; err != nil && !errors.Is(err, context.Canceled) {\n\t\tt.Fatalf(\"writer error: %v\", err)\n\t}\n\n\tdeadline := time.After(250 * time.Millisecond)\n\tfor limited.open.Load() != 0 {\n\t\tselect {\n\t\tcase <-deadline:\n\t\t\tt.Fatalf(\"descriptor leak: %d handles still open\", limited.open.Load())\n\t\tcase <-time.After(10 * time.Millisecond):\n\t\t}\n\t}\n}\n\nfunc TestVFS_PageIndexOOM(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\t_, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES ('ok')\"); err != nil {\n\t\tt.Fatalf(\"insert row: %v\", err)\n\t}\n\tfor i := 0; i < 64; i++ {\n\t\tpayload := strings.Repeat(\"p\", 3500)\n\t\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES (?)\", payload); err != nil {\n\t\t\tt.Fatalf(\"bulk insert: %v\", err)\n\t\t}\n\t}\n\n\toomClient := &oomPageIndexClient{ReplicaClient: client}\n\tvfs := newVFS(t, oomClient)\n\tvfs.PollInterval = 20 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\tdsn := fmt.Sprintf(\"file:%s?vfs=%s\", filepath.ToSlash(filepath.Join(t.TempDir(), \"oom.db\")), vfsName)\n\tfailing, err := sql.Open(\"sqlite3\", dsn)\n\tif err != nil {\n\t\tt.Fatalf(\"open replica db: %v\", err)\n\t}\n\tdefer failing.Close()\n\tfailing.SetMaxOpenConns(4)\n\tfailing.SetMaxIdleConns(4)\n\n\toomClient.failNext.Store(true)\n\tvar count int\n\tif err := failing.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&count); err == nil {\n\t\tt.Fatalf(\"expected query to fail due to page index OOM\")\n\t}\n\tif !oomClient.triggered.Load() {\n\t\tt.Fatalf(\"page index client never triggered\")\n\t}\n\n\toomClient.failNext.Store(false)\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\twaitForReplicaRowCount(t, primary, replica, 5*time.Second)\n\n\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&count); err != nil {\n\t\tt.Fatalf(\"post-oom read failed: %v\", err)\n\t}\n\tvar expected int\n\tif err := primary.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&expected); err != nil {\n\t\tt.Fatalf(\"primary count: %v\", err)\n\t}\n\tif count != expected {\n\t\tt.Fatalf(\"unexpected row count: got %d want %d\", count, expected)\n\t}\n}\n\nfunc TestVFS_PageIndexCorruptionRecovery(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\t_, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES ('ok')\"); err != nil {\n\t\tt.Fatalf(\"insert row: %v\", err)\n\t}\n\n\tcorruptClient := &corruptingPageIndexClient{ReplicaClient: client}\n\tvfs := newVFS(t, corruptClient)\n\tvfs.PollInterval = 20 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\tdsn := fmt.Sprintf(\"file:%s?vfs=%s\", filepath.ToSlash(filepath.Join(t.TempDir(), \"corrupt.db\")), vfsName)\n\n\tcorruptClient.corruptNext.Store(true)\n\tbadConn, err := sql.Open(\"sqlite3\", dsn)\n\tif err != nil {\n\t\tt.Fatalf(\"open corrupt replica: %v\", err)\n\t}\n\tbadConn.SetMaxOpenConns(8)\n\tbadConn.SetMaxIdleConns(8)\n\tbadConn.SetConnMaxIdleTime(30 * time.Second)\n\tvar count int\n\tif err := badConn.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&count); err == nil {\n\t\tbadConn.Close()\n\t\tt.Fatalf(\"expected corruption failure\")\n\t}\n\tbadConn.Close()\n\tif !corruptClient.triggered.Load() {\n\t\tt.Fatalf(\"corruption hook never triggered\")\n\t}\n\n\tgoodConn := openVFSReplicaDB(t, vfsName)\n\tdefer goodConn.Close()\n\tif err := goodConn.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&count); err != nil {\n\t\tt.Fatalf(\"post-corruption read failed: %v\", err)\n\t}\n\tif count != 1 {\n\t\tt.Fatalf(\"unexpected row count after recovery: %d\", count)\n\t}\n}\n\nfunc TestVFS_RapidUpdateCoalescing(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tconst interval = 5 * time.Millisecond\n\t_, primary := openReplicatedPrimary(t, client, interval, interval)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE metrics (id INTEGER PRIMARY KEY, value INTEGER)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO metrics (id, value) VALUES (1, 0)\"); err != nil {\n\t\tt.Fatalf(\"insert row: %v\", err)\n\t}\n\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = interval\n\tvfsName := registerTestVFS(t, vfs)\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\tconst updates = 200\n\twriterDone := make(chan struct{})\n\tgo func() {\n\t\tdefer close(writerDone)\n\t\tfor i := 1; i <= updates; i++ {\n\t\t\tif _, err := primary.Exec(\"UPDATE metrics SET value = ? WHERE id = 1\", i); err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t\ttime.Sleep(time.Millisecond)\n\t\t}\n\t}()\n\n\trequire.Eventually(t, func() bool {\n\t\tvar value int\n\t\tif err := replica.QueryRow(\"SELECT value FROM metrics WHERE id = 1\").Scan(&value); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn value == updates\n\t}, 3*time.Second, 5*time.Millisecond, \"replica should observe final value\")\n\t<-writerDone\n\n\tvar value int\n\tif err := replica.QueryRow(\"SELECT value FROM metrics WHERE id = 1\").Scan(&value); err != nil {\n\t\tt.Fatalf(\"final read: %v\", err)\n\t}\n\tif value != updates {\n\t\tt.Fatalf(\"unexpected final value: got %d want %d\", value, updates)\n\t}\n}\n\nfunc TestVFS_NonContiguousTXIDGapFailsOnOpen(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tfor txID := ltx.TXID(1); txID <= 4; txID++ {\n\t\twriteSinglePageLTXFile(t, client, txID, byte('a'+int(txID)))\n\t}\n\n\tmissing := client.LTXFilePath(0, 2, 2)\n\tif err := os.Remove(missing); err != nil {\n\t\tt.Fatalf(\"remove ltx file: %v\", err)\n\t}\n\n\tfileLogger := slog.New(slog.NewTextHandler(io.Discard, &slog.HandlerOptions{Level: slog.LevelError}))\n\tf := litestream.NewVFSFile(client, \"gap.db\", fileLogger)\n\tf.PollInterval = 25 * time.Millisecond\n\n\tif err := f.Open(); err == nil {\n\t\tt.Fatalf(\"expected open to fail after removing %s\", filepath.Base(missing))\n\t} else if errMsg := err.Error(); !strings.Contains(errMsg, \"non-contiguous\") {\n\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t}\n}\n\nfunc TestVFS_PollingThreadRecoversFromLTXListFailure(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tflakyClient := &flakyLTXClient{ReplicaClient: client}\n\tconst monitorInterval = 25 * time.Millisecond\n\t_, primary := openReplicatedPrimary(t, client, monitorInterval, monitorInterval)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES ('seed')\"); err != nil {\n\t\tt.Fatalf(\"insert seed: %v\", err)\n\t}\n\n\tvfs := newVFS(t, flakyClient)\n\tvfs.PollInterval = 25 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\twaitForReplicaRowCount(t, primary, replica, 10*time.Second)\n\n\tflakyClient.failNext.Store(true)\n\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES ('after-failure')\"); err != nil {\n\t\tt.Fatalf(\"insert post-failure: %v\", err)\n\t}\n\n\twaitForReplicaRowCount(t, primary, replica, 10*time.Second)\n\n\tif flakyClient.failures.Load() == 0 {\n\t\tt.Fatalf(\"expected at least one LTXFiles failure\")\n\t}\n\n\tvar primaryCount, replicaCount int\n\tif err := primary.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&primaryCount); err != nil {\n\t\tt.Fatalf(\"primary count: %v\", err)\n\t}\n\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&replicaCount); err != nil {\n\t\tt.Fatalf(\"replica count: %v\", err)\n\t}\n\tif primaryCount != replicaCount {\n\t\tt.Fatalf(\"replica did not catch up after failure: primary=%d replica=%d\", primaryCount, replicaCount)\n\t}\n}\n\nfunc TestVFS_PollIntervalEdgeCases(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinterval time.Duration\n\t\tminCalls int64\n\t\tmaxCalls int64\n\t}{\n\t\t{\"fast\", 5 * time.Millisecond, 10, 500},\n\t\t{\"slow\", 200 * time.Millisecond, 1, 10},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\t\tobs := &observingReplicaClient{ReplicaClient: client}\n\t\t\t_, primary := openReplicatedPrimary(t, obs, tt.interval, tt.interval)\n\t\t\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\t\t\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, value INTEGER)\"); err != nil {\n\t\t\t\tt.Fatalf(\"create table: %v\", err)\n\t\t\t}\n\n\t\t\tvfs := newVFS(t, obs)\n\t\t\tvfs.PollInterval = tt.interval\n\t\t\tvfsName := registerTestVFS(t, vfs)\n\t\t\treplica := openVFSReplicaDB(t, vfsName)\n\t\t\tdefer replica.Close()\n\n\t\t\tstart := obs.ltxCalls.Load()\n\t\t\ttime.Sleep(750 * time.Millisecond)\n\t\t\tdelta := obs.ltxCalls.Load() - start\n\t\t\tif delta < tt.minCalls {\n\t\t\t\tt.Fatalf(\"expected at least %d polls, got %d\", tt.minCalls, delta)\n\t\t\t}\n\t\t\tif tt.maxCalls > 0 && delta > tt.maxCalls {\n\t\t\t\tt.Fatalf(\"expected at most %d polls, got %d\", tt.maxCalls, delta)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVFS_PooledWriteNoFalseConflict(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\n\tdb, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE seed (id INTEGER PRIMARY KEY)\"); err != nil {\n\t\tt.Fatalf(\"create seed table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO seed (id) VALUES (1)\"); err != nil {\n\t\tt.Fatalf(\"insert seed: %v\", err)\n\t}\n\n\trequire.Eventually(t, func() bool {\n\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t\tdefer itr.Close()\n\t\treturn itr.Next()\n\t}, 5*time.Second, db.MonitorInterval, \"LTX files should be created\")\n\tforceReplicaSync(t, db)\n\n\tvfs := newVFS(t, client)\n\tvfs.WriteEnabled = true\n\tvfs.WriteSyncInterval = 0\n\tvfs.PollInterval = 25 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\tdsn := fmt.Sprintf(\"file:%s?vfs=%s\", filepath.ToSlash(filepath.Join(t.TempDir(), \"pooled-write.db\")), vfsName)\n\tsqldb, err := sql.Open(\"sqlite3\", dsn)\n\tif err != nil {\n\t\tt.Fatalf(\"open write db: %v\", err)\n\t}\n\tdefer sqldb.Close()\n\tsqldb.SetMaxOpenConns(2)\n\tsqldb.SetMaxIdleConns(2)\n\n\tif _, err := sqldb.Exec(\"PRAGMA busy_timeout = 5000\"); err != nil {\n\t\tt.Fatalf(\"set busy timeout: %v\", err)\n\t}\n\tif _, err := sqldb.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\tconst totalWrites = 20\n\tfor i := 1; i <= totalWrites; i++ {\n\t\tif _, err := sqldb.Exec(\"INSERT INTO t (id, value) VALUES (?, ?)\", i, fmt.Sprintf(\"row-%d\", i)); err != nil {\n\t\t\tif strings.Contains(err.Error(), \"conflict\") || errors.Is(err, litestream.ErrConflict) {\n\t\t\t\tt.Fatalf(\"false ErrConflict on write %d: %v\", i, err)\n\t\t\t}\n\t\t\tt.Fatalf(\"write %d failed: %v\", i, err)\n\t\t}\n\t}\n\n\tvar count int\n\tif err := sqldb.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&count); err != nil {\n\t\tt.Fatalf(\"count rows: %v\", err)\n\t}\n\tif count != totalWrites {\n\t\tt.Fatalf(\"expected %d rows, got %d\", totalWrites, count)\n\t}\n}\n\nfunc TestVFS_PooledWriteStress(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\n\tdb, primary := openReplicatedPrimary(t, client, 25*time.Millisecond, 25*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY AUTOINCREMENT, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tif _, err := primary.Exec(\"INSERT INTO t (value) VALUES ('seed')\"); err != nil {\n\t\tt.Fatalf(\"insert seed: %v\", err)\n\t}\n\n\trequire.Eventually(t, func() bool {\n\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t\tdefer itr.Close()\n\t\treturn itr.Next()\n\t}, 5*time.Second, db.MonitorInterval, \"LTX files should be created\")\n\tforceReplicaSync(t, db)\n\n\tvfs := newVFS(t, client)\n\tvfs.WriteEnabled = true\n\tvfs.WriteSyncInterval = 10 * time.Millisecond\n\tvfs.PollInterval = 25 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\tdsn := fmt.Sprintf(\"file:%s?vfs=%s&_busy_timeout=5000\", filepath.ToSlash(filepath.Join(t.TempDir(), \"stress-write.db\")), vfsName)\n\tsqldb, err := sql.Open(\"sqlite3\", dsn)\n\tif err != nil {\n\t\tt.Fatalf(\"open write db: %v\", err)\n\t}\n\tdefer sqldb.Close()\n\tsqldb.SetMaxOpenConns(4)\n\tsqldb.SetMaxIdleConns(4)\n\n\tconst totalWrites = 50\n\tfor i := 0; i < totalWrites; i++ {\n\t\tif _, err := sqldb.Exec(\"INSERT INTO t (value) VALUES (?)\", fmt.Sprintf(\"row-%d\", i)); err != nil {\n\t\t\tif strings.Contains(err.Error(), \"conflict\") || errors.Is(err, litestream.ErrConflict) {\n\t\t\t\tt.Fatalf(\"false ErrConflict on write %d: %v\", i, err)\n\t\t\t}\n\t\t\tt.Fatalf(\"write %d failed: %v\", i, err)\n\t\t}\n\t\t// Brief pause every 5 writes to allow sync ticker to fire,\n\t\t// which causes TXID advancement and exercises the coordination logic\n\t\tif (i+1)%5 == 0 {\n\t\t\ttime.Sleep(25 * time.Millisecond)\n\t\t}\n\t}\n\n\tvar count int\n\tif err := sqldb.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&count); err != nil {\n\t\tt.Fatalf(\"count rows: %v\", err)\n\t}\n\texpected := totalWrites + 1 // +1 for seed row\n\tif count != expected {\n\t\tt.Fatalf(\"expected %d rows, got %d\", expected, count)\n\t}\n}\n\nfunc newVFS(tb testing.TB, client litestream.ReplicaClient) *testVFS {\n\ttb.Helper()\n\n\tlogger := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{\n\t\tLevel: slog.LevelDebug,\n\t}))\n\n\tbase := litestream.NewVFS(client, logger)\n\tbase.PollInterval = 100 * time.Millisecond\n\treturn &testVFS{\n\t\tVFS: base,\n\t\tfailures: make(map[string][]error),\n\t}\n}\n\ntype testVFS struct {\n\t*litestream.VFS\n\n\tmu sync.Mutex\n\tfailures map[string][]error\n}\n\nfunc (v *testVFS) Open(name string, flags sqlite3vfs.OpenFlag) (sqlite3vfs.File, sqlite3vfs.OpenFlag, error) {\n\tf, flags, err := v.VFS.Open(name, flags)\n\tif err != nil {\n\t\treturn nil, flags, err\n\t}\n\treturn &injectingFile{File: f, vfs: v, name: name}, flags, nil\n}\n\nfunc (v *testVFS) Inject(path string, err error) {\n\tv.mu.Lock()\n\tv.failures[path] = append(v.failures[path], err)\n\tv.mu.Unlock()\n}\n\nfunc (v *testVFS) popFailure(path string) error {\n\tv.mu.Lock()\n\tdefer v.mu.Unlock()\n\tqueue := v.failures[path]\n\tif len(queue) == 0 {\n\t\treturn nil\n\t}\n\terr := queue[0]\n\tif len(queue) == 1 {\n\t\tdelete(v.failures, path)\n\t} else {\n\t\tv.failures[path] = queue[1:]\n\t}\n\tif err == nil {\n\t\treturn errors.New(\"vfs page read error\")\n\t}\n\treturn err\n}\n\ntype injectingFile struct {\n\tsqlite3vfs.File\n\n\tvfs *testVFS\n\tname string\n}\n\nfunc (f *injectingFile) ReadAt(p []byte, off int64) (int, error) {\n\tif err := f.vfs.popFailure(f.name); err != nil {\n\t\treturn 0, err\n\t}\n\treturn f.File.ReadAt(p, off)\n}\n\nfunc (f *injectingFile) FileControl(op int, pragmaName string, pragmaValue *string) (*string, error) {\n\tif fc, ok := f.File.(sqlite3vfs.FileController); ok {\n\t\treturn fc.FileControl(op, pragmaName, pragmaValue)\n\t}\n\treturn nil, sqlite3vfs.NotFoundError\n}\n\nfunc registerTestVFS(tb testing.TB, vfs sqlite3vfs.VFS) string {\n\ttb.Helper()\n\tname := fmt.Sprintf(\"litestream-%s-%d\", strings.ToLower(tb.Name()), time.Now().UnixNano())\n\tif err := sqlite3vfs.RegisterVFS(name, vfs); err != nil {\n\t\ttb.Fatalf(\"failed to register litestream vfs %s: %v\", name, err)\n\t}\n\treturn name\n}\n\nfunc openReplicatedPrimary(tb testing.TB, client litestream.ReplicaClient, monitorInterval, syncInterval time.Duration) (*litestream.DB, *sql.DB) {\n\ttb.Helper()\n\tdb := testingutil.NewDB(tb, filepath.Join(tb.TempDir(), \"primary.db\"))\n\tdb.MonitorInterval = monitorInterval\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = syncInterval\n\tif err := db.Open(); err != nil {\n\t\ttb.Fatalf(\"open db: %v\", err)\n\t}\n\tsqldb := testingutil.MustOpenSQLDB(tb, db.Path())\n\ttb.Cleanup(func() { _ = db.Close(context.Background()) })\n\treturn db, sqldb\n}\n\nfunc forceReplicaSync(tb testing.TB, db *litestream.DB) {\n\ttb.Helper()\n\tctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n\tdefer cancel()\n\tif err := db.Sync(ctx); err != nil {\n\t\ttb.Fatalf(\"force sync: %v\", err)\n\t}\n\tif db.Replica != nil {\n\t\tif err := db.Replica.Sync(ctx); err != nil {\n\t\t\ttb.Fatalf(\"replica sync: %v\", err)\n\t\t}\n\t}\n}\n\nfunc openVFSReplicaDB(tb testing.TB, vfsName string) *sql.DB {\n\ttb.Helper()\n\tdsn := fmt.Sprintf(\"file:%s?vfs=%s\", filepath.ToSlash(filepath.Join(tb.TempDir(), vfsName+\".db\")), vfsName)\n\tsqldb, err := sql.Open(\"sqlite3\", dsn)\n\tif err != nil {\n\t\ttb.Fatalf(\"open replica db: %v\", err)\n\t}\n\tsqldb.SetMaxOpenConns(32)\n\tsqldb.SetMaxIdleConns(32)\n\tsqldb.SetConnMaxIdleTime(30 * time.Second)\n\tif _, err := sqldb.Exec(\"PRAGMA busy_timeout = 2000\"); err != nil {\n\t\ttb.Fatalf(\"set busy timeout: %v\", err)\n\t}\n\treturn sqldb\n}\n\nfunc waitForReplicaRowCount(tb testing.TB, primary, replica *sql.DB, timeout time.Duration) {\n\ttb.Helper()\n\trequire.Eventually(tb, func() bool {\n\t\tvar primaryCount int\n\t\tif err := primary.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&primaryCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\tvar replicaCount int\n\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM t\").Scan(&replicaCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn primaryCount == replicaCount\n\t}, timeout, 50*time.Millisecond, \"replica row count should match primary\")\n}\n\nfunc waitForTableRowCount(tb testing.TB, primary, replica *sql.DB, table string, timeout time.Duration) {\n\ttb.Helper()\n\tquery := fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", table)\n\trequire.Eventually(tb, func() bool {\n\t\tvar primaryCount int\n\t\tif err := primary.QueryRow(query).Scan(&primaryCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\tvar replicaCount int\n\t\tif err := replica.QueryRow(query).Scan(&replicaCount); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn primaryCount == replicaCount\n\t}, timeout, 50*time.Millisecond, \"replica row count for %s should match primary\", table)\n}\n\nfunc fetchOrderedPayloads(tb testing.TB, db *sql.DB, limit int, orderBy string) []string {\n\ttb.Helper()\n\tquery := fmt.Sprintf(\"SELECT payload FROM t ORDER BY %s LIMIT %d\", orderBy, limit)\n\trows, err := db.Query(query)\n\tif err != nil {\n\t\ttb.Fatalf(\"query payloads: %v\", err)\n\t}\n\tdefer rows.Close()\n\n\tvar out []string\n\tfor rows.Next() {\n\t\tvar payload string\n\t\tif err := rows.Scan(&payload); err != nil {\n\t\t\ttb.Fatalf(\"scan payload: %v\", err)\n\t\t}\n\t\tout = append(out, payload)\n\t}\n\tif err := rows.Err(); err != nil {\n\t\ttb.Fatalf(\"rows err: %v\", err)\n\t}\n\treturn out\n}\n\nfunc seedLargeTable(tb testing.TB, db *sql.DB, n int) {\n\ttb.Helper()\n\ttrx, err := db.Begin()\n\tif err != nil {\n\t\ttb.Fatalf(\"begin seed: %v\", err)\n\t}\n\tstmt, err := trx.Prepare(\"INSERT INTO t (value, updated_at) VALUES (?, strftime('%s','now'))\")\n\tif err != nil {\n\t\t_ = trx.Rollback()\n\t\ttb.Fatalf(\"prepare seed: %v\", err)\n\t}\n\tdefer stmt.Close()\n\trnd := rand.New(rand.NewSource(42))\n\tfor i := 0; i < n; i++ {\n\t\tif _, err := stmt.Exec(fmt.Sprintf(\"seed-%d-%d\", i, rnd.Int())); err != nil {\n\t\t\t_ = trx.Rollback()\n\t\t\ttb.Fatalf(\"seed exec: %v\", err)\n\t\t}\n\t}\n\tif err := trx.Commit(); err != nil {\n\t\ttb.Fatalf(\"commit seed: %v\", err)\n\t}\n}\n\nfunc seedSortedDataset(tb testing.TB, db *sql.DB, n int) {\n\ttb.Helper()\n\ttrx, err := db.Begin()\n\tif err != nil {\n\t\ttb.Fatalf(\"begin sorted seed: %v\", err)\n\t}\n\tstmt, err := trx.Prepare(\"INSERT INTO t (id, payload, grp) VALUES (?, ?, ?)\")\n\tif err != nil {\n\t\t_ = trx.Rollback()\n\t\ttb.Fatalf(\"prepare sorted seed: %v\", err)\n\t}\n\tdefer stmt.Close()\n\trnd := rand.New(rand.NewSource(time.Now().UnixNano()))\n\tfor i := 0; i < n; i++ {\n\t\tif _, err := stmt.Exec(i+1, randomPayload(rnd, 256), rnd.Intn(1024)); err != nil {\n\t\t\t_ = trx.Rollback()\n\t\t\ttb.Fatalf(\"sorted seed exec: %v\", err)\n\t\t}\n\t}\n\tif err := trx.Commit(); err != nil {\n\t\ttb.Fatalf(\"commit sorted seed: %v\", err)\n\t}\n}\n\nfunc randomPayload(r *rand.Rand, n int) string {\n\tconst letters = \"abcdefghijklmnopqrstuvwxyz0123456789\"\n\tb := make([]byte, n)\n\tfor i := range b {\n\t\tb[i] = letters[r.Intn(len(letters))]\n\t}\n\treturn string(b)\n}\n\nfunc pageSizedPayload(pageSize int, row int) string {\n\tbase := fmt.Sprintf(\"row_%05d_\", row)\n\tmaxPayload := pageSize / 4\n\tif maxPayload < len(base)+1 {\n\t\tmaxPayload = len(base) + 1\n\t}\n\tif maxPayload > 4096 {\n\t\tmaxPayload = 4096\n\t}\n\tfillerLen := maxPayload - len(base)\n\tif fillerLen < 0 {\n\t\tfillerLen = 0\n\t}\n\treturn base + strings.Repeat(\"x\", fillerLen)\n}\n\nfunc isBusyError(err error) bool {\n\tif err == nil {\n\t\treturn false\n\t}\n\tif e, ok := err.(sqlite3.Error); ok {\n\t\tif e.Code == sqlite3.ErrBusy || e.Code == sqlite3.ErrLocked {\n\t\t\treturn true\n\t\t}\n\t\t// Under heavy churn, go-sqlite3 can surface ErrError with the\n\t\t// generic \"SQL logic error\" message while the VFS swaps databases.\n\t\tif e.Code == sqlite3.ErrError && strings.Contains(e.Error(), \"SQL logic error\") {\n\t\t\treturn true\n\t\t}\n\t}\n\tmsg := err.Error()\n\tif strings.Contains(msg, \"database is locked\") || strings.Contains(msg, \"database is busy\") {\n\t\treturn true\n\t}\n\treturn strings.Contains(msg, \"converting NULL to int\")\n}\n\nfunc writeSinglePageLTXFile(tb testing.TB, client *file.ReplicaClient, txid ltx.TXID, fill byte) {\n\ttb.Helper()\n\tpage := bytes.Repeat([]byte{fill}, 4096)\n\tvar buf bytes.Buffer\n\tenc, err := ltx.NewEncoder(&buf)\n\tif err != nil {\n\t\ttb.Fatalf(\"new encoder: %v\", err)\n\t}\n\thdr := ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tPageSize: 4096,\n\t\tCommit: 1,\n\t\tMinTXID: txid,\n\t\tMaxTXID: txid,\n\t\tTimestamp: time.Now().UnixMilli(),\n\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t}\n\tif err := enc.EncodeHeader(hdr); err != nil {\n\t\ttb.Fatalf(\"encode header: %v\", err)\n\t}\n\tif err := enc.EncodePage(ltx.PageHeader{Pgno: 1}, page); err != nil {\n\t\ttb.Fatalf(\"encode page: %v\", err)\n\t}\n\tif err := enc.Close(); err != nil {\n\t\ttb.Fatalf(\"close encoder: %v\", err)\n\t}\n\n\tif _, err := client.WriteLTXFile(context.Background(), 0, txid, txid, bytes.NewReader(buf.Bytes())); err != nil {\n\t\ttb.Fatalf(\"write ltx file: %v\", err)\n\t}\n}\n\ntype latencyReplicaClient struct {\n\tlitestream.ReplicaClient\n\tdelay time.Duration\n}\n\nfunc (c *latencyReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\ttime.Sleep(c.delay)\n\treturn c.ReplicaClient.OpenLTXFile(ctx, level, minTXID, maxTXID, offset, size)\n}\n\nfunc (c *latencyReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\ttime.Sleep(c.delay)\n\treturn c.ReplicaClient.LTXFiles(ctx, level, seek, useMetadata)\n}\n\ntype eventualConsistencyClient struct {\n\tlitestream.ReplicaClient\n\tcalls atomic.Int32\n}\n\nfunc (c *eventualConsistencyClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif c.calls.Add(1) == 1 {\n\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t}\n\treturn c.ReplicaClient.LTXFiles(ctx, level, seek, useMetadata)\n}\n\ntype observingReplicaClient struct {\n\tlitestream.ReplicaClient\n\tltxCalls atomic.Int64\n}\n\ntype fdLimitedReplicaClient struct {\n\tlitestream.ReplicaClient\n\tlimit int32\n\topen atomic.Int32\n\tmaxOpen atomic.Int32\n}\n\nfunc (c *fdLimitedReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tcurrent := c.open.Add(1)\n\tfor {\n\t\tmax := c.maxOpen.Load()\n\t\tif current <= max || c.maxOpen.CompareAndSwap(max, current) {\n\t\t\tbreak\n\t\t}\n\t}\n\tif current > c.limit {\n\t\tc.open.Add(-1)\n\t\treturn nil, fmt.Errorf(\"fd limit exceeded: %d/%d\", current, c.limit)\n\t}\n\trc, err := c.ReplicaClient.OpenLTXFile(ctx, level, minTXID, maxTXID, offset, size)\n\tif err != nil {\n\t\tc.open.Add(-1)\n\t\treturn nil, err\n\t}\n\treturn &hookedReadCloser{ReadCloser: rc, hook: func() { c.open.Add(-1) }}, nil\n}\n\nfunc (c *observingReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tc.ltxCalls.Add(1)\n\treturn c.ReplicaClient.LTXFiles(ctx, level, seek, useMetadata)\n}\n\ntype flakyLTXClient struct {\n\tlitestream.ReplicaClient\n\tfailNext atomic.Bool\n\tfailures atomic.Int64\n}\n\nfunc (c *flakyLTXClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif c.failNext.CompareAndSwap(true, false) {\n\t\tc.failures.Add(1)\n\t\treturn nil, fmt.Errorf(\"ltx list unavailable\")\n\t}\n\treturn c.ReplicaClient.LTXFiles(ctx, level, seek, useMetadata)\n}\n\ntype oomPageIndexClient struct {\n\tlitestream.ReplicaClient\n\tfailNext atomic.Bool\n\ttriggered atomic.Bool\n}\n\nfunc (c *oomPageIndexClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tif offset > 0 && c.failNext.CompareAndSwap(true, false) {\n\t\tc.triggered.Store(true)\n\t\treturn nil, fmt.Errorf(\"simulated page index OOM\")\n\t}\n\treturn c.ReplicaClient.OpenLTXFile(ctx, level, minTXID, maxTXID, offset, size)\n}\n\ntype corruptingPageIndexClient struct {\n\tlitestream.ReplicaClient\n\tcorruptNext atomic.Bool\n\ttriggered atomic.Bool\n}\n\nfunc (c *corruptingPageIndexClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\trc, err := c.ReplicaClient.OpenLTXFile(ctx, level, minTXID, maxTXID, offset, size)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif c.corruptNext.CompareAndSwap(true, false) {\n\t\tc.triggered.Store(true)\n\t\tdata, readErr := io.ReadAll(rc)\n\t\trc.Close()\n\t\tif readErr != nil {\n\t\t\treturn nil, readErr\n\t\t}\n\t\tif len(data) > 0 {\n\t\t\tdata[0] ^= 0xFF\n\t\t}\n\t\treturn io.NopCloser(bytes.NewReader(data)), nil\n\t}\n\treturn rc, nil\n}\n\ntype hookedReadCloser struct {\n\tio.ReadCloser\n\tonce sync.Once\n\thook func()\n}\n\nfunc (h *hookedReadCloser) Close() error {\n\tvar err error\n\th.once.Do(func() {\n\t\terr = h.ReadCloser.Close()\n\t\tif h.hook != nil {\n\t\t\th.hook()\n\t\t}\n\t})\n\treturn err\n}\n\n// waitForLTXFiles waits until at least one LTX file is available in the replica client.\nfunc waitForLTXFiles(t *testing.T, client litestream.ReplicaClient, timeout, tick time.Duration) {\n\tt.Helper()\n\trequire.Eventually(t, func() bool {\n\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t\tdefer itr.Close()\n\t\treturn itr.Next()\n\t}, timeout, tick, \"LTX files should be available\")\n}\n\n// waitForReplicaValue waits until the replica database returns the expected int value.\nfunc waitForReplicaValue(t *testing.T, db *sql.DB, query string, expected int, timeout, tick time.Duration) {\n\tt.Helper()\n\trequire.Eventually(t, func() bool {\n\t\tvar got int\n\t\tif err := db.QueryRow(query).Scan(&got); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn got == expected\n\t}, timeout, tick, \"replica should return expected value\")\n}\n" }, { "path": "cmd/litestream-vfs/stress_test.go", "content": "//go:build vfs && stress\n// +build vfs,stress\n\npackage main_test\n\nimport (\n\t\"context\"\n\t\"math/rand\"\n\t\"os\"\n\t\"runtime\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestVFS_RaceStressHarness(t *testing.T) {\n\tif os.Getenv(\"LITESTREAM_ALLOW_RACE\") != \"1\" {\n\t\tt.Skip(\"set LITESTREAM_ALLOW_RACE=1 to run unstable race harness; modernc.org/sqlite checkptr panics are still unresolved\")\n\t}\n\tif !runtime.RaceEnabled() {\n\t\tt.Skip(\"requires go test -race\")\n\t}\n\n\tclient := file.NewReplicaClient(t.TempDir())\n\tdb, primary := openReplicatedPrimary(t, client, 20*time.Millisecond, 20*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(\"CREATE TABLE stress (id INTEGER PRIMARY KEY, value TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tseedLargeTable(t, primary, 100)\n\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 5 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\twaitForReplicaRowCount(t, primary, replica, 10*time.Second)\n\n\tctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n\tdefer cancel()\n\n\tvar writes atomic.Int64\n\tgo func() {\n\t\trnd := rand.New(rand.NewSource(time.Now().UnixNano()))\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\t\t\tif _, err := primary.Exec(\"INSERT INTO stress (value) VALUES (?)\", randomPayload(rnd, 64)); err != nil && !isBusyError(err) {\n\t\t\t\tt.Errorf(\"writer error: %v\", err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\twrites.Add(1)\n\t\t}\n\t}()\n\n\tconst readers = 64\n\terrCh := make(chan error, readers)\n\tfor i := 0; i < readers; i++ {\n\t\tgo func() {\n\t\t\trnd := rand.New(rand.NewSource(time.Now().UnixNano()))\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\terrCh <- nil\n\t\t\t\t\treturn\n\t\t\t\tdefault:\n\t\t\t\t}\n\t\t\t\tvar count int\n\t\t\t\tif err := replica.QueryRow(\"SELECT COUNT(*) FROM stress WHERE id >= ?\", rnd.Intn(50)).Scan(&count); err != nil {\n\t\t\t\t\tif isBusyError(err) {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\terrCh <- err\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\t}\n\n\tfor i := 0; i < readers; i++ {\n\t\tif err := <-errCh; err != nil {\n\t\t\tt.Fatalf(\"reader error: %v\", err)\n\t\t}\n\t}\n\n\tif writes.Load() == 0 {\n\t\tt.Fatalf(\"writer never made progress\")\n\t}\n}\n" }, { "path": "cmd/litestream-vfs/time_travel_test.go", "content": "//go:build vfs\n// +build vfs\n\npackage main_test\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n\t\"github.com/psanford/sqlite3vfs\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestVFS_TimeTravelFunctions(t *testing.T) {\n\tctx := context.Background()\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 50 * time.Millisecond\n\tif err := sqlite3vfs.RegisterVFS(\"litestream-time\", vfs); err != nil {\n\t\tt.Fatalf(\"failed to register litestream vfs: %v\", err)\n\t}\n\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\tdb.MonitorInterval = 50 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 50 * time.Millisecond\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() { _ = db.Close(ctx) }()\n\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb0)\n\n\tif _, err := sqldb0.Exec(\"CREATE TABLE t (x INTEGER)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb0.Exec(\"INSERT INTO t (x) VALUES (100)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for LTX files to be created\n\trequire.Eventually(t, func() bool {\n\t\titr, err := client.LTXFiles(ctx, 0, 0, false)\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t\tdefer itr.Close()\n\t\treturn itr.Next()\n\t}, 10*time.Second, db.MonitorInterval, \"LTX files should be created\")\n\n\tfirstCreatedAt := fetchLTXCreatedAt(t, ctx, client)\n\n\ttime.Sleep(20 * time.Millisecond) // Ensure a different timestamp for the next file.\n\tif _, err := sqldb0.Exec(\"UPDATE t SET x = 200\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tsqldb1, err := sql.Open(\"sqlite3\", \"file:/tmp/time-travel.db?vfs=litestream-time\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to open database: %v\", err)\n\t}\n\tdefer sqldb1.Close()\n\tsqldb1.SetMaxOpenConns(1)\n\n\tvar value int\n\trequire.Eventually(t, func() bool {\n\t\tif err := sqldb1.QueryRow(\"SELECT x FROM t\").Scan(&value); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn value == 200\n\t}, 10*time.Second, vfs.PollInterval, \"VFS should observe updated value\")\n\n\ttarget := firstCreatedAt.Add(1 * time.Millisecond).UTC().Format(time.RFC3339Nano)\n\tif _, err := sqldb1.Exec(fmt.Sprintf(\"PRAGMA LITESTREAM_TIME = '%s'\", target)); err != nil {\n\t\tt.Fatalf(\"set target time: %v\", err)\n\t}\n\n\tif err := sqldb1.QueryRow(\"SELECT x FROM t\").Scan(&value); err != nil {\n\t\tt.Fatalf(\"query historical value: %v\", err)\n\t} else if got, want := value, 100; got != want {\n\t\tt.Fatalf(\"historical value: got %d, want %d\", got, want)\n\t}\n\n\tvar currentTime string\n\tif err := sqldb1.QueryRow(\"PRAGMA litestream_time\").Scan(¤tTime); err != nil {\n\t\tt.Fatalf(\"current time: %v\", err)\n\t} else if currentTime != target {\n\t\tt.Fatalf(\"current time mismatch: got %s, want %s\", currentTime, target)\n\t}\n\n\tif _, err := sqldb1.Exec(\"PRAGMA LITESTREAM_TIME = LATEST\"); err != nil {\n\t\tt.Fatalf(\"reset time: %v\", err)\n\t}\n\n\tif err := sqldb1.QueryRow(\"SELECT x FROM t\").Scan(&value); err != nil {\n\t\tt.Fatalf(\"query reset value: %v\", err)\n\t} else if got, want := value, 200; got != want {\n\t\tt.Fatalf(\"reset value: got %d, want %d\", got, want)\n\t}\n\n\tif err := sqldb1.QueryRow(\"PRAGMA litestream_time\").Scan(¤tTime); err != nil {\n\t\tt.Fatalf(\"current time after reset: %v\", err)\n\t}\n\t// After reset, should return actual LTX timestamp (not \"latest\" anymore per #853)\n\tif _, err := time.Parse(time.RFC3339Nano, currentTime); err != nil {\n\t\tt.Fatalf(\"current time after reset should be valid RFC3339Nano timestamp, got %s: %v\", currentTime, err)\n\t}\n}\n\nfunc TestVFS_PragmaLitestreamTxid(t *testing.T) {\n\tctx := context.Background()\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 50 * time.Millisecond\n\tif err := sqlite3vfs.RegisterVFS(\"litestream-txid\", vfs); err != nil {\n\t\tt.Fatalf(\"failed to register litestream vfs: %v\", err)\n\t}\n\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\tdb.MonitorInterval = 50 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 50 * time.Millisecond\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() { _ = db.Close(ctx) }()\n\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb0)\n\n\tif _, err := sqldb0.Exec(\"CREATE TABLE t (x INTEGER)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb0.Exec(\"INSERT INTO t (x) VALUES (100)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tsqldb1, err := sql.Open(\"sqlite3\", \"file:/tmp/txid-test.db?vfs=litestream-txid\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to open database: %v\", err)\n\t}\n\tdefer sqldb1.Close()\n\tsqldb1.SetMaxOpenConns(1)\n\n\tvar txid int64\n\trequire.Eventually(t, func() bool {\n\t\tif err := sqldb1.QueryRow(\"PRAGMA litestream_txid\").Scan(&txid); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn txid > 0\n\t}, 10*time.Second, vfs.PollInterval, \"PRAGMA litestream_txid should return positive value\")\n\n\t// Test that setting litestream_txid fails (read-only)\n\tif _, err := sqldb1.Exec(\"PRAGMA litestream_txid = 123\"); err == nil {\n\t\tt.Fatal(\"expected error setting litestream_txid (read-only)\")\n\t}\n}\n\nfunc TestVFS_PragmaLitestreamLag(t *testing.T) {\n\tctx := context.Background()\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 50 * time.Millisecond\n\tif err := sqlite3vfs.RegisterVFS(\"litestream-lag\", vfs); err != nil {\n\t\tt.Fatalf(\"failed to register litestream vfs: %v\", err)\n\t}\n\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\tdb.MonitorInterval = 50 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 50 * time.Millisecond\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() { _ = db.Close(ctx) }()\n\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb0)\n\n\tif _, err := sqldb0.Exec(\"CREATE TABLE t (x INTEGER)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb0.Exec(\"INSERT INTO t (x) VALUES (100)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tsqldb1, err := sql.Open(\"sqlite3\", \"file:/tmp/lag-test.db?vfs=litestream-lag\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to open database: %v\", err)\n\t}\n\tdefer sqldb1.Close()\n\tsqldb1.SetMaxOpenConns(1)\n\n\t// Wait for replica to catch up with polling.\n\tvar lag int64\n\trequire.Eventually(t, func() bool {\n\t\tif err := sqldb1.QueryRow(\"PRAGMA litestream_lag\").Scan(&lag); err != nil {\n\t\t\tt.Logf(\"query lag: %v\", err)\n\t\t\treturn false\n\t\t}\n\t\treturn lag >= 0\n\t}, 10*time.Second, vfs.PollInterval, \"lag should become >= 0\")\n\n\t// Test that setting litestream_lag fails (read-only)\n\tif _, err := sqldb1.Exec(\"PRAGMA litestream_lag = 123\"); err == nil {\n\t\tt.Fatal(\"expected error setting litestream_lag (read-only)\")\n\t}\n}\n\nfunc TestVFS_PragmaRelativeTime(t *testing.T) {\n\tctx := context.Background()\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 50 * time.Millisecond\n\tif err := sqlite3vfs.RegisterVFS(\"litestream-relative\", vfs); err != nil {\n\t\tt.Fatalf(\"failed to register litestream vfs: %v\", err)\n\t}\n\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\tdb.MonitorInterval = 50 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 50 * time.Millisecond\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() { _ = db.Close(ctx) }()\n\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb0)\n\n\tif _, err := sqldb0.Exec(\"CREATE TABLE t (x INTEGER)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb0.Exec(\"INSERT INTO t (x) VALUES (100)\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tsqldb1, err := sql.Open(\"sqlite3\", \"file:/tmp/relative-test.db?vfs=litestream-relative\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to open database: %v\", err)\n\t}\n\tdefer sqldb1.Close()\n\tsqldb1.SetMaxOpenConns(1)\n\n\t// Wait for VFS to poll initial data\n\trequire.Eventually(t, func() bool {\n\t\tvar x int\n\t\treturn sqldb1.QueryRow(\"SELECT x FROM t\").Scan(&x) == nil\n\t}, 10*time.Second, vfs.PollInterval, \"VFS should observe initial data\")\n\n\t// Test that relative time parsing works (even if no data exists at that time)\n\t// The parse should succeed, but may return \"no backup files available\" if too far in past\n\tnow := time.Now()\n\t_, err = sqldb1.Exec(\"PRAGMA litestream_time = '1 second ago'\")\n\t// This might fail if no LTX files exist at that time, which is expected.\n\t// The important thing is that the parsing worked (not a \"parse timestamp\" error).\n\tif err != nil {\n\t\terrMsg := err.Error()\n\t\t// These are expected errors that indicate parsing succeeded but time-travel\n\t\t// couldn't be performed (no files at that time).\n\t\texpectedErrors := []string{\n\t\t\t\"no backup files available\",\n\t\t\t\"timestamp is before earliest LTX file\",\n\t\t}\n\t\texpectedSubstrings := []string{\n\t\t\t\"transaction not available\", // ErrTxNotAvailable when target is before earliest LTX\n\t\t\t\"SQL logic error\", // SQLite error during page index rebuild for time-travel\n\t\t}\n\t\tisExpected := false\n\t\tfor _, expected := range expectedErrors {\n\t\t\tif errMsg == expected {\n\t\t\t\tisExpected = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif !isExpected {\n\t\t\tfor _, substr := range expectedSubstrings {\n\t\t\t\tif strings.Contains(errMsg, substr) {\n\t\t\t\t\tisExpected = true\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tif !isExpected {\n\t\t\t// Fail on any unexpected error to catch regressions\n\t\t\tt.Fatalf(\"unexpected error from relative time PRAGMA: %v\", err)\n\t\t}\n\t}\n\n\t// Reset to latest\n\tif _, err := sqldb1.Exec(\"PRAGMA litestream_time = LATEST\"); err != nil {\n\t\tt.Fatalf(\"reset to latest: %v\", err)\n\t}\n\n\t// Verify the current time is recent (within last minute)\n\tvar currentTime string\n\tif err := sqldb1.QueryRow(\"PRAGMA litestream_time\").Scan(¤tTime); err != nil {\n\t\tt.Fatalf(\"query current time: %v\", err)\n\t}\n\tts, err := time.Parse(time.RFC3339Nano, currentTime)\n\tif err != nil {\n\t\tt.Fatalf(\"parse current time: %v\", err)\n\t}\n\tif now.Sub(ts) > time.Minute {\n\t\tt.Fatalf(\"current time too old: %v (now: %v)\", ts, now)\n\t}\n}\n\nfunc fetchLTXCreatedAt(tb testing.TB, ctx context.Context, client litestream.ReplicaClient) time.Time {\n\ttb.Helper()\n\n\titr, err := client.LTXFiles(ctx, 0, 0, true)\n\tif err != nil {\n\t\ttb.Fatalf(\"ltx files: %v\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar ts time.Time\n\tfor itr.Next() {\n\t\tts = itr.Item().CreatedAt\n\t}\n\tif err := itr.Close(); err != nil {\n\t\ttb.Fatalf(\"close iterator: %v\", err)\n\t}\n\tif ts.IsZero() {\n\t\ttb.Fatalf(\"no ltx files found\")\n\t}\n\treturn ts.UTC()\n}\n" }, { "path": "cmd/litestream-vfs/vfs_soak_test.go", "content": "//go:build vfs && soak\n// +build vfs,soak\n\npackage main_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\n// TestVFS_LongRunningSoak exercises the VFS under sustained read/write load.\n// The default duration is 5 minutes but can be overridden with the\n// LITESTREAM_VFS_SOAK_DURATION environment variable (e.g. \"10m\").\nfunc TestVFS_LongRunningSoak(t *testing.T) {\n\tduration := 5 * time.Minute\n\tif v := os.Getenv(\"LITESTREAM_VFS_SOAK_DURATION\"); v != \"\" {\n\t\tif parsed, err := time.ParseDuration(v); err == nil {\n\t\t\tduration = parsed\n\t\t}\n\t}\n\tif testing.Short() && duration > time.Minute {\n\t\tduration = time.Minute\n\t}\n\n\tclient := file.NewReplicaClient(t.TempDir())\n\tvfs := newVFS(t, client)\n\tvfs.PollInterval = 100 * time.Millisecond\n\tvfsName := registerTestVFS(t, vfs)\n\n\tdb, primary := openReplicatedPrimary(t, client, 75*time.Millisecond, 75*time.Millisecond)\n\tdefer testingutil.MustCloseSQLDB(t, primary)\n\n\tif _, err := primary.Exec(`CREATE TABLE t (\n\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\tvalue TEXT,\n\t\tupdated_at INTEGER\n\t)`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tseedLargeTable(t, primary, 1000)\n\tforceReplicaSync(t, db)\n\n\treplica := openVFSReplicaDB(t, vfsName)\n\tdefer replica.Close()\n\n\twaitForReplicaRowCount(t, primary, replica, time.Minute)\n\n\tctx, cancel := context.WithTimeout(context.Background(), duration)\n\tdefer cancel()\n\n\tvar writeOps atomic.Int64\n\tvar readOps atomic.Int64\n\terrCh := make(chan error, 8)\n\tvar wg sync.WaitGroup\n\n\t// Writers continuously mutate the primary database.\n\tstartWriter := func(name string) {\n\t\twg.Add(1)\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\trnd := time.NewTicker(7 * time.Millisecond)\n\t\t\tdefer rnd.Stop()\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\tcase <-rnd.C:\n\t\t\t\t\tif _, err := primary.Exec(\"INSERT INTO t (value, updated_at) VALUES (?, strftime('%s','now'))\", fmt.Sprintf(\"%s-%d\", name, time.Now().UnixNano())); err != nil {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"writer %s insert: %w\", name, err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tif _, err := primary.Exec(\"UPDATE t SET value = value || '-w' WHERE id IN (SELECT id FROM t ORDER BY RANDOM() LIMIT 1)\"); err != nil {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"writer %s update: %w\", name, err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\twriteOps.Add(2)\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\t}\n\n\tstartReader := func(name string) {\n\t\twg.Add(1)\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\tdefault:\n\t\t\t\t}\n\t\t\t\tvar minID, maxID, count int\n\t\t\t\tif err := replica.QueryRow(\"SELECT IFNULL(MIN(id),0), IFNULL(MAX(id),0), COUNT(*) FROM t\").Scan(&minID, &maxID, &count); err != nil {\n\t\t\t\t\terrCh <- fmt.Errorf(\"reader %s query: %w\", name, err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif minID > maxID && count > 0 {\n\t\t\t\t\terrCh <- fmt.Errorf(\"reader %s saw invalid range\", name)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\treadOps.Add(1)\n\t\t\t}\n\t\t}()\n\t}\n\n\tfor i := 0; i < 2; i++ {\n\t\tstartWriter(fmt.Sprintf(\"writer-%d\", i))\n\t}\n\tfor i := 0; i < 4; i++ {\n\t\tstartReader(fmt.Sprintf(\"reader-%d\", i))\n\t}\n\n\t<-ctx.Done()\n\twg.Wait()\n\tclose(errCh)\n\tfor err := range errCh {\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"soak error: %v\", err)\n\t\t}\n\t}\n\n\tif writeOps.Load() < int64(duration/time.Millisecond) {\n\t\tt.Fatalf(\"expected sustained writes, got %d ops\", writeOps.Load())\n\t}\n\tif readOps.Load() == 0 {\n\t\tt.Fatalf(\"expected replica reads during soak\")\n\t}\n\n\twaitForReplicaRowCount(t, primary, replica, time.Minute)\n}\n" }, { "path": "cmd/litestream-vfs/vfs_write_integration_test.go", "content": "//go:build vfs\n// +build vfs\n\npackage main_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/psanford/sqlite3vfs\"\n\t\"github.com/stretchr/testify/require\"\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\n// =============================================================================\n// Basic Operations Tests\n// =============================================================================\n\n// TestVFS_WriteAndSync_FileBackend tests basic write and sync functionality\n// with the file backend.\nfunc TestVFS_WriteAndSync_FileBackend(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\t// First, create initial data using standard litestream replication\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"source.db\"))\n\tdb.MonitorInterval = 100 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 100 * time.Millisecond\n\trequire.NoError(t, db.Open())\n\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb0)\n\n\t_, err := sqldb0.Exec(\"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)\")\n\trequire.NoError(t, err)\n\t_, err = sqldb0.Exec(\"INSERT INTO users (id, name) VALUES (1, 'Alice')\")\n\trequire.NoError(t, err)\n\n\twaitForLTXFiles(t, client, 10*time.Second, db.MonitorInterval)\n\trequire.NoError(t, db.Replica.Stop(false))\n\ttestingutil.MustCloseSQLDB(t, sqldb0)\n\trequire.NoError(t, db.Close(context.Background()))\n\n\t// Now open via writable VFS and add more data\n\tvfs := newWritableVFS(t, client, 1*time.Second, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-write-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb1, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb1.Close()\n\n\t// Verify initial data\n\tvar name string\n\terr = sqldb1.QueryRow(\"SELECT name FROM users WHERE id = 1\").Scan(&name)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Alice\", name)\n\n\t// Insert new data via VFS\n\t_, err = sqldb1.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\n\t// Force sync\n\tsqldb1.Close()\n\n\t// Verify data was synced by opening fresh VFS\n\tvfs2 := newWritableVFS(t, client, 0, \"\")\n\tvfsName2 := fmt.Sprintf(\"litestream-write2-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName2, vfs2))\n\n\tsqldb2, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName2))\n\trequire.NoError(t, err)\n\tdefer sqldb2.Close()\n\n\terr = sqldb2.QueryRow(\"SELECT name FROM users WHERE id = 2\").Scan(&name)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Bob\", name)\n}\n\n// TestVFS_ReadYourWrites verifies that written data is visible immediately\n// before sync (from dirty pages).\nfunc TestVFS_ReadYourWrites(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\t// Create initial database\n\tsetupInitialDB(t, client)\n\n\t// Open via writable VFS with long sync interval (won't auto-sync)\n\tvfs := newWritableVFS(t, client, 1*time.Hour, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-ryw-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Write data\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\n\t// Read it back immediately (before sync)\n\tvar name string\n\terr = sqldb.QueryRow(\"SELECT name FROM users WHERE id = 2\").Scan(&name)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Bob\", name)\n\n\t// Update and read again\n\t_, err = sqldb.Exec(\"UPDATE users SET name = 'Robert' WHERE id = 2\")\n\trequire.NoError(t, err)\n\n\terr = sqldb.QueryRow(\"SELECT name FROM users WHERE id = 2\").Scan(&name)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Robert\", name)\n}\n\n// TestVFS_MultipleTransactions tests multiple sequential transactions.\nfunc TestVFS_MultipleTransactions(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-multi-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Execute multiple transactions\n\tfor i := 2; i <= 10; i++ {\n\t\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (?, ?)\", i, fmt.Sprintf(\"User%d\", i))\n\t\trequire.NoError(t, err)\n\t}\n\n\t// Wait for syncs\n\ttime.Sleep(500 * time.Millisecond)\n\n\t// Verify all data\n\tvar count int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 10, count)\n}\n\n// TestVFS_LargeTransaction tests writing many pages in a single transaction.\nfunc TestVFS_LargeTransaction(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 1*time.Second, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-large-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Insert 1000 rows in a single transaction (should span many pages)\n\ttx, err := sqldb.Begin()\n\trequire.NoError(t, err)\n\n\tfor i := 2; i <= 1001; i++ {\n\t\t_, err = tx.Exec(\"INSERT INTO users (id, name) VALUES (?, ?)\", i, fmt.Sprintf(\"User%d with some extra data to take up space\", i))\n\t\trequire.NoError(t, err)\n\t}\n\n\terr = tx.Commit()\n\trequire.NoError(t, err)\n\n\t// Force sync by closing\n\tsqldb.Close()\n\n\t// Verify data persisted\n\tvfs2 := newWritableVFS(t, client, 0, \"\")\n\tvfsName2 := fmt.Sprintf(\"litestream-large2-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName2, vfs2))\n\n\tsqldb2, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName2))\n\trequire.NoError(t, err)\n\tdefer sqldb2.Close()\n\n\tvar count int\n\terr = sqldb2.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1001, count)\n}\n\n// =============================================================================\n// Sync Behavior Tests\n// =============================================================================\n\n// TestVFS_PeriodicSync verifies automatic periodic sync.\nfunc TestVFS_PeriodicSync(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\t// Get initial LTX count\n\tinitialCount := countLTXFiles(t, client)\n\n\tvfs := newWritableVFS(t, client, 200*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-periodic-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Write data\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\n\t// Wait for auto-sync (should happen within ~200ms)\n\ttime.Sleep(500 * time.Millisecond)\n\n\t// Verify new LTX file was created\n\tnewCount := countLTXFiles(t, client)\n\trequire.Greater(t, newCount, initialCount, \"expected new LTX file from auto-sync\")\n}\n\n// TestVFS_SyncDuringTransaction verifies sync is deferred during active transaction.\nfunc TestVFS_SyncDuringTransaction(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-txsync-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\tinitialCount := countLTXFiles(t, client)\n\n\t// Begin transaction\n\ttx, err := sqldb.Begin()\n\trequire.NoError(t, err)\n\n\t// Write data within transaction\n\t_, err = tx.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\n\t// Wait - sync should be deferred\n\ttime.Sleep(300 * time.Millisecond)\n\n\t// LTX count should not have increased during transaction\n\tmidCount := countLTXFiles(t, client)\n\trequire.Equal(t, initialCount, midCount, \"sync should be deferred during transaction\")\n\n\t// Commit transaction\n\terr = tx.Commit()\n\trequire.NoError(t, err)\n\n\t// Wait for sync after commit\n\ttime.Sleep(300 * time.Millisecond)\n\n\t// Now LTX should have increased\n\tfinalCount := countLTXFiles(t, client)\n\trequire.Greater(t, finalCount, initialCount, \"expected new LTX file after commit\")\n}\n\n// TestVFS_ManualSyncOnly tests with SyncInterval=0 (manual sync only).\nfunc TestVFS_ManualSyncOnly(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\t// SyncInterval=0 means no auto-sync\n\tvfs := newWritableVFS(t, client, 0, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-manual-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\n\tinitialCount := countLTXFiles(t, client)\n\n\t// Write data\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\n\t// Wait - should NOT auto-sync\n\ttime.Sleep(500 * time.Millisecond)\n\n\tmidCount := countLTXFiles(t, client)\n\trequire.Equal(t, initialCount, midCount, \"should not auto-sync when SyncInterval=0\")\n\n\t// Close triggers sync\n\tsqldb.Close()\n\n\t// Now should be synced\n\tfinalCount := countLTXFiles(t, client)\n\trequire.Greater(t, finalCount, initialCount, \"expected sync on close\")\n}\n\n// =============================================================================\n// Write Buffer Tests\n// =============================================================================\n\n// TestVFS_WriteBufferDiscardedOnOpen tests that write buffer is discarded on open\n// (unsynced data is lost after crash).\nfunc TestVFS_WriteBufferDiscardedOnOpen(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tbufferDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\t// Open VFS and write data\n\tvfs := newWritableVFS(t, client, 1*time.Hour, bufferDir) // Long interval, won't auto-sync\n\tvfsName := fmt.Sprintf(\"litestream-discard1-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\n\t// Verify write buffer file exists\n\tbufferPath := filepath.Join(bufferDir, \".litestream-buffer\")\n\t_, err = os.Stat(bufferPath)\n\trequire.NoError(t, err, \"write buffer file should exist\")\n\n\t// Simulate crash by not closing properly (don't call sqldb.Close())\n\t// Just abandon the connection\n\n\t// Reopen with new VFS - buffer should be discarded\n\tvfs2 := newWritableVFS(t, client, 1*time.Second, bufferDir)\n\tvfsName2 := fmt.Sprintf(\"litestream-discard2-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName2, vfs2))\n\n\tsqldb2, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName2))\n\trequire.NoError(t, err)\n\tdefer sqldb2.Close()\n\n\t// Data should NOT be recovered (buffer is discarded on open)\n\tvar count int\n\terr = sqldb2.QueryRow(\"SELECT COUNT(*) FROM users WHERE id = 2\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 0, count, \"unsynced data should be lost after crash\")\n}\n\n// TestVFS_WriteBufferDuplicatePages tests that duplicate page writes within\n// a session correctly overwrite previous values in the buffer.\nfunc TestVFS_WriteBufferDuplicatePages(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tbufferDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 1*time.Hour, bufferDir)\n\tvfsName := fmt.Sprintf(\"litestream-dup1-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Write to same row multiple times (updates same pages)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"UPDATE users SET name = 'Robert' WHERE id = 2\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"UPDATE users SET name = 'Bobby' WHERE id = 2\")\n\trequire.NoError(t, err)\n\n\t// Should have latest value (read-your-writes within session)\n\tvar name string\n\terr = sqldb.QueryRow(\"SELECT name FROM users WHERE id = 2\").Scan(&name)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Bobby\", name)\n\n\t// Close to trigger sync\n\tsqldb.Close()\n\n\t// Verify data persists in replica after sync\n\tvfs2 := newWritableVFS(t, client, 1*time.Second, t.TempDir())\n\tvfsName2 := fmt.Sprintf(\"litestream-dup2-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName2, vfs2))\n\n\tsqldb2, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName2))\n\trequire.NoError(t, err)\n\tdefer sqldb2.Close()\n\n\t// Should have latest value from synced data\n\terr = sqldb2.QueryRow(\"SELECT name FROM users WHERE id = 2\").Scan(&name)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Bobby\", name)\n}\n\n// TestVFS_ExistingBufferDiscarded tests that any existing buffer file is discarded on open.\nfunc TestVFS_ExistingBufferDiscarded(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tbufferDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\t// Create a pre-existing write buffer file with some content\n\tbufferPath := filepath.Join(bufferDir, \".litestream-write-buffer\")\n\trequire.NoError(t, os.WriteFile(bufferPath, []byte(\"stale data\"), 0644))\n\n\t// Open VFS - should discard existing buffer\n\tvfs := newWritableVFS(t, client, 1*time.Second, bufferDir)\n\tvfsName := fmt.Sprintf(\"litestream-existing-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Should only see original data (existing buffer discarded)\n\tvar count int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1, count, \"existing buffer should be discarded\")\n}\n\n// TestVFS_WriteBufferCorrupted tests handling of corrupted buffer file.\nfunc TestVFS_WriteBufferCorrupted(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tbufferDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\t// Create corrupted buffer (invalid magic)\n\tbufferPath := filepath.Join(bufferDir, \".litestream-write-buffer\")\n\trequire.NoError(t, os.WriteFile(bufferPath, []byte(\"INVALID DATA\"), 0644))\n\n\t// Open VFS - should handle gracefully\n\tvfs := newWritableVFS(t, client, 1*time.Second, bufferDir)\n\tvfsName := fmt.Sprintf(\"litestream-corrupt-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Should work with original data\n\tvar count int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1, count)\n}\n\n// =============================================================================\n// Conflict Detection Tests\n// =============================================================================\n\n// TestVFS_ConflictDetection tests that conflicts are detected when remote changes.\nfunc TestVFS_ConflictDetection(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\t// Open writable VFS\n\tvfs := newWritableVFS(t, client, 1*time.Hour, t.TempDir()) // Long interval\n\tvfsName := fmt.Sprintf(\"litestream-conflict-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Write data via VFS (not synced yet)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\n\t// Externally add new LTX file to simulate another writer\n\taddExternalLTXFile(t, client, replicaDir)\n\n\t// Now close - sync should fail with conflict\n\t// Note: The conflict detection happens during sync, but the error may be logged\n\t// rather than returned to the user. This test verifies the mechanism exists.\n\tsqldb.Close()\n\n\t// The conflict should have been detected (check logs or VFS state)\n\t// For now, we just verify the test doesn't crash\n}\n\n// TestVFS_NoConflictWhenRemoteUnchanged verifies no false conflicts.\nfunc TestVFS_NoConflictWhenRemoteUnchanged(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-noconflict-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Multiple write/sync cycles - no conflicts expected\n\tfor i := 2; i <= 5; i++ {\n\t\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (?, ?)\", i, fmt.Sprintf(\"User%d\", i))\n\t\trequire.NoError(t, err)\n\t\ttime.Sleep(200 * time.Millisecond) // Wait for sync\n\t}\n\n\t// All data should be present\n\tvar count int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 5, count)\n}\n\n// =============================================================================\n// Concurrency Tests\n// =============================================================================\n\n// TestVFS_ConcurrentReaders tests one writer with multiple readers.\nfunc TestVFS_ConcurrentReaders(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\t// Writer VFS\n\twriterVFS := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\twriterVFSName := fmt.Sprintf(\"litestream-writer-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(writerVFSName, writerVFS))\n\n\twriterDB, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", writerVFSName))\n\trequire.NoError(t, err)\n\tdefer writerDB.Close()\n\n\t// Reader VFS (read-only)\n\treaderVFS := newReadOnlyVFS(t, client)\n\treaderVFSName := fmt.Sprintf(\"litestream-reader-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(readerVFSName, readerVFS))\n\n\treaderDB, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", readerVFSName))\n\trequire.NoError(t, err)\n\tdefer readerDB.Close()\n\n\t// Write some data\n\t_, err = writerDB.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\n\t// Wait for sync\n\ttime.Sleep(300 * time.Millisecond)\n\n\t// Reader should eventually see the data\n\trequire.Eventually(t, func() bool {\n\t\tvar count int\n\t\tif err := readerDB.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count); err != nil {\n\t\t\treturn false\n\t\t}\n\t\treturn count == 2\n\t}, 5*time.Second, 100*time.Millisecond)\n}\n\n// TestVFS_ReadWhileWriting tests reading during an active write transaction.\nfunc TestVFS_ReadWhileWriting(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 1*time.Second, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-readwrite-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\tvar wg sync.WaitGroup\n\terrors := make(chan error, 10)\n\n\t// Writer goroutine\n\twg.Add(1)\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tfor i := 2; i <= 20; i++ {\n\t\t\tif _, err := sqldb.Exec(\"INSERT INTO users (id, name) VALUES (?, ?)\", i, fmt.Sprintf(\"User%d\", i)); err != nil {\n\t\t\t\terrors <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t}\n\t}()\n\n\t// Reader goroutine\n\twg.Add(1)\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tfor i := 0; i < 50; i++ {\n\t\t\tvar count int\n\t\t\tif err := sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count); err != nil {\n\t\t\t\terrors <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\ttime.Sleep(5 * time.Millisecond)\n\t\t}\n\t}()\n\n\twg.Wait()\n\tclose(errors)\n\n\tfor err := range errors {\n\t\tt.Errorf(\"concurrent operation failed: %v\", err)\n\t}\n}\n\n// =============================================================================\n// Edge Case Tests\n// =============================================================================\n\n// TestVFS_Truncate tests database truncation via VACUUM.\nfunc TestVFS_Truncate(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-truncate-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Add many rows\n\tfor i := 2; i <= 100; i++ {\n\t\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (?, ?)\", i, fmt.Sprintf(\"User%d with lots of data to take up space\", i))\n\t\trequire.NoError(t, err)\n\t}\n\n\ttime.Sleep(300 * time.Millisecond)\n\n\t// Delete all but one\n\t_, err = sqldb.Exec(\"DELETE FROM users WHERE id > 1\")\n\trequire.NoError(t, err)\n\n\t// VACUUM to reclaim space\n\t_, err = sqldb.Exec(\"VACUUM\")\n\trequire.NoError(t, err)\n\n\ttime.Sleep(300 * time.Millisecond)\n\n\t// Verify only 1 row remains\n\tvar count int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1, count)\n}\n\n// TestVFS_EmptyTransaction tests begin/commit with no changes.\nfunc TestVFS_EmptyTransaction(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 1*time.Second, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-empty-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\tinitialCount := countLTXFiles(t, client)\n\n\t// Empty transaction\n\ttx, err := sqldb.Begin()\n\trequire.NoError(t, err)\n\terr = tx.Commit()\n\trequire.NoError(t, err)\n\n\ttime.Sleep(200 * time.Millisecond)\n\n\t// Should not create new LTX for empty transaction\n\tfinalCount := countLTXFiles(t, client)\n\trequire.Equal(t, initialCount, finalCount, \"empty transaction should not create new LTX\")\n}\n\n// TestVFS_SchemaChanges tests DDL operations.\nfunc TestVFS_SchemaChanges(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-schema-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Create table\n\t_, err = sqldb.Exec(\"CREATE TABLE products (id INTEGER PRIMARY KEY, name TEXT, price REAL)\")\n\trequire.NoError(t, err)\n\n\t// Add column\n\t_, err = sqldb.Exec(\"ALTER TABLE products ADD COLUMN quantity INTEGER DEFAULT 0\")\n\trequire.NoError(t, err)\n\n\t// Insert data\n\t_, err = sqldb.Exec(\"INSERT INTO products (id, name, price, quantity) VALUES (1, 'Widget', 9.99, 100)\")\n\trequire.NoError(t, err)\n\n\ttime.Sleep(300 * time.Millisecond)\n\n\t// Verify schema\n\tvar qty int\n\terr = sqldb.QueryRow(\"SELECT quantity FROM products WHERE id = 1\").Scan(&qty)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 100, qty)\n\n\t// Drop table\n\t_, err = sqldb.Exec(\"DROP TABLE products\")\n\trequire.NoError(t, err)\n\n\ttime.Sleep(200 * time.Millisecond)\n\n\t// Verify dropped\n\t_, err = sqldb.Query(\"SELECT * FROM products\")\n\trequire.Error(t, err)\n}\n\n// TestVFS_BlobData tests large blob operations.\nfunc TestVFS_BlobData(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-blob-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Create table for blobs\n\t_, err = sqldb.Exec(\"CREATE TABLE blobs (id INTEGER PRIMARY KEY, data BLOB)\")\n\trequire.NoError(t, err)\n\n\t// Insert large blob (100KB - spans multiple pages)\n\tlargeData := make([]byte, 100*1024)\n\tfor i := range largeData {\n\t\tlargeData[i] = byte(i % 256)\n\t}\n\t_, err = sqldb.Exec(\"INSERT INTO blobs (id, data) VALUES (1, ?)\", largeData)\n\trequire.NoError(t, err)\n\n\ttime.Sleep(300 * time.Millisecond)\n\n\t// Read back and verify\n\tvar retrieved []byte\n\terr = sqldb.QueryRow(\"SELECT data FROM blobs WHERE id = 1\").Scan(&retrieved)\n\trequire.NoError(t, err)\n\trequire.Equal(t, largeData, retrieved)\n}\n\n// =============================================================================\n// Round-Trip Verification Tests\n// =============================================================================\n\n// TestVFS_WriteAndRestore tests full write -> sync -> restore cycle.\nfunc TestVFS_WriteAndRestore(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\t// Write via VFS\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-restore1-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\n\tfor i := 2; i <= 10; i++ {\n\t\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (?, ?)\", i, fmt.Sprintf(\"User%d\", i))\n\t\trequire.NoError(t, err)\n\t}\n\tsqldb.Close()\n\n\t// Restore to a new file\n\trestoredPath := filepath.Join(t.TempDir(), \"restored.db\")\n\terr = restoreDB(t, client, restoredPath)\n\trequire.NoError(t, err)\n\n\t// Verify restored database\n\trestoredDB, err := sql.Open(\"sqlite3\", restoredPath)\n\trequire.NoError(t, err)\n\tdefer restoredDB.Close()\n\n\tvar count int\n\terr = restoredDB.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 10, count)\n}\n\n// TestVFS_WriteReadVFSOnly tests write via writable VFS, read via read-only VFS.\nfunc TestVFS_WriteReadVFSOnly(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\t// Write via writable VFS\n\twriterVFS := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\twriterVFSName := fmt.Sprintf(\"litestream-vfsonly-w-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(writerVFSName, writerVFS))\n\n\twriterDB, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", writerVFSName))\n\trequire.NoError(t, err)\n\n\t_, err = writerDB.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\twriterDB.Close()\n\n\t// Read via read-only VFS\n\treaderVFS := newReadOnlyVFS(t, client)\n\treaderVFSName := fmt.Sprintf(\"litestream-vfsonly-r-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(readerVFSName, readerVFS))\n\n\treaderDB, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", readerVFSName))\n\trequire.NoError(t, err)\n\tdefer readerDB.Close()\n\n\tvar name string\n\terr = readerDB.QueryRow(\"SELECT name FROM users WHERE id = 2\").Scan(&name)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Bob\", name)\n}\n\n// TestVFS_MixedWorkload tests interleaved reads/writes/syncs.\nfunc TestVFS_MixedWorkload(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 200*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-mixed-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Mixed operations\n\tfor i := 0; i < 50; i++ {\n\t\tswitch i % 5 {\n\t\tcase 0, 1, 2: // Write\n\t\t\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (?, ?)\", i+2, fmt.Sprintf(\"User%d\", i))\n\t\t\trequire.NoError(t, err)\n\t\tcase 3: // Read\n\t\t\tvar count int\n\t\t\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count)\n\t\t\trequire.NoError(t, err)\n\t\tcase 4: // Update\n\t\t\t_, err = sqldb.Exec(\"UPDATE users SET name = ? WHERE id = ?\", fmt.Sprintf(\"Updated%d\", i), (i%10)+1)\n\t\t\trequire.NoError(t, err)\n\t\t}\n\t}\n\n\t// Final verification\n\tvar count int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.GreaterOrEqual(t, count, 30) // At least 30 inserts (0,1,2 mod 5)\n}\n\n// =============================================================================\n// Error Handling Tests\n// =============================================================================\n\n// TestVFS_SyncNetworkError tests handling of network errors during sync.\nfunc TestVFS_SyncNetworkError(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tsetupInitialDB(t, client)\n\n\tvfs := newWritableVFS(t, client, 1*time.Hour, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-neterr-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Write data\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (2, 'Bob')\")\n\trequire.NoError(t, err)\n\n\t// Remove replica directory to simulate error\n\trequire.NoError(t, os.RemoveAll(replicaDir))\n\n\t// Close should handle error gracefully (sync will fail but shouldn't crash)\n\tsqldb.Close()\n}\n\n// TestVFS_InvalidPageSize tests mismatched page size handling.\nfunc TestVFS_InvalidPageSize(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\t// Create database with different page size\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"source.db\"))\n\tdb.MonitorInterval = 100 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\trequire.NoError(t, db.Open())\n\n\tsqldb0 := testingutil.MustOpenSQLDB(t, db.Path())\n\t// Note: Page size is set at database creation, this is just verifying\n\t// the test setup works\n\t_, err := sqldb0.Exec(\"CREATE TABLE test (x)\")\n\trequire.NoError(t, err)\n\n\twaitForLTXFiles(t, client, 10*time.Second, db.MonitorInterval)\n\trequire.NoError(t, db.Replica.Stop(false))\n\ttestingutil.MustCloseSQLDB(t, sqldb0)\n\trequire.NoError(t, db.Close(context.Background()))\n\n\t// Open via VFS (should work with same page size)\n\tvfs := newWritableVFS(t, client, 1*time.Second, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-pagesize-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Should be able to query\n\tvar count int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM test\").Scan(&count)\n\trequire.NoError(t, err)\n}\n\n// TestVFS_RollbackRestoresOriginalState tests that rolling back a transaction\n// restores the database to its original state, even after large writes.\nfunc TestVFS_RollbackRestoresOriginalState(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\t// Create database directly via writable VFS (no external setup needed)\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-rollback-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Create initial data via the VFS\n\t_, err = sqldb.Exec(\"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (1, 'Alice')\")\n\trequire.NoError(t, err)\n\n\t// Verify initial state\n\tvar initialCount int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&initialCount)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1, initialCount, \"expected 1 initial row\")\n\n\t// Start a transaction and write a large amount of data (spanning multiple pages)\n\t_, err = sqldb.Exec(\"BEGIN\")\n\trequire.NoError(t, err)\n\n\t// Insert 1000 rows with large payloads (~1KB each) to span multiple pages\n\tfor i := 2; i <= 1001; i++ {\n\t\tpayload := fmt.Sprintf(\"rollback_test_user_%d_%s\", i, string(make([]byte, 900)))\n\t\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (?, ?)\", i, payload)\n\t\trequire.NoError(t, err)\n\t}\n\n\t// Verify the data is visible within the transaction\n\tvar countDuringTx int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&countDuringTx)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1001, countDuringTx, \"expected 1001 rows during transaction\")\n\n\t// ROLLBACK the transaction\n\t_, err = sqldb.Exec(\"ROLLBACK\")\n\trequire.NoError(t, err)\n\n\t// Verify original state is restored\n\tvar afterRollbackCount int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&afterRollbackCount)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1, afterRollbackCount, \"expected 1 row after rollback\")\n\n\t// Verify original data is intact\n\tvar afterRollbackName string\n\terr = sqldb.QueryRow(\"SELECT name FROM users WHERE id = 1\").Scan(&afterRollbackName)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Alice\", afterRollbackName, \"original data should be intact after rollback\")\n}\n\n// TestVFS_RollbackAfterUpdate tests that rolling back UPDATE operations\n// restores the original values.\nfunc TestVFS_RollbackAfterUpdate(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\t// Create database directly via writable VFS\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-rollback-update-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Create initial data via the VFS\n\t_, err = sqldb.Exec(\"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (1, 'Alice')\")\n\trequire.NoError(t, err)\n\n\t// Get original value\n\tvar originalName string\n\terr = sqldb.QueryRow(\"SELECT name FROM users WHERE id = 1\").Scan(&originalName)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Alice\", originalName)\n\n\t// Start transaction and update\n\t_, err = sqldb.Exec(\"BEGIN\")\n\trequire.NoError(t, err)\n\n\t_, err = sqldb.Exec(\"UPDATE users SET name = 'MODIFIED_' || name\")\n\trequire.NoError(t, err)\n\n\t// Verify modification is visible\n\tvar modifiedName string\n\terr = sqldb.QueryRow(\"SELECT name FROM users WHERE id = 1\").Scan(&modifiedName)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"MODIFIED_Alice\", modifiedName)\n\n\t// ROLLBACK\n\t_, err = sqldb.Exec(\"ROLLBACK\")\n\trequire.NoError(t, err)\n\n\t// Verify original value is restored\n\tvar restoredName string\n\terr = sqldb.QueryRow(\"SELECT name FROM users WHERE id = 1\").Scan(&restoredName)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Alice\", restoredName, \"name should be restored after rollback\")\n}\n\n// TestVFS_RollbackAfterDelete tests that rolling back DELETE operations\n// restores the deleted rows.\nfunc TestVFS_RollbackAfterDelete(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\t// Create database directly via writable VFS\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-rollback-delete-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Create initial data via the VFS\n\t_, err = sqldb.Exec(\"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (1, 'Alice')\")\n\trequire.NoError(t, err)\n\n\t// Verify initial count\n\tvar initialCount int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&initialCount)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1, initialCount)\n\n\t// Start transaction and delete all rows\n\t_, err = sqldb.Exec(\"BEGIN\")\n\trequire.NoError(t, err)\n\n\t_, err = sqldb.Exec(\"DELETE FROM users\")\n\trequire.NoError(t, err)\n\n\t// Verify deletion\n\tvar countAfterDelete int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&countAfterDelete)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 0, countAfterDelete)\n\n\t// ROLLBACK\n\t_, err = sqldb.Exec(\"ROLLBACK\")\n\trequire.NoError(t, err)\n\n\t// Verify rows are restored\n\tvar restoredCount int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users\").Scan(&restoredCount)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1, restoredCount, \"all rows should be restored after rollback\")\n\n\t// Verify the data itself is correct\n\tvar name string\n\terr = sqldb.QueryRow(\"SELECT name FROM users WHERE id = 1\").Scan(&name)\n\trequire.NoError(t, err)\n\trequire.Equal(t, \"Alice\", name)\n}\n\n// TestVFS_CommitAfterRollbackWorks tests that commits work correctly after\n// a previous rollback in the same session.\nfunc TestVFS_CommitAfterRollbackWorks(t *testing.T) {\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\t// Create database directly via writable VFS\n\tvfs := newWritableVFS(t, client, 100*time.Millisecond, t.TempDir())\n\tvfsName := fmt.Sprintf(\"litestream-commit-after-rollback-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName, vfs))\n\n\tsqldb, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName))\n\trequire.NoError(t, err)\n\tdefer sqldb.Close()\n\n\t// Create initial data via the VFS\n\t_, err = sqldb.Exec(\"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (1, 'Alice')\")\n\trequire.NoError(t, err)\n\n\t// Transaction 1: Insert and rollback\n\t_, err = sqldb.Exec(\"BEGIN\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (100, 'should_not_exist')\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"ROLLBACK\")\n\trequire.NoError(t, err)\n\n\t// Transaction 2: Insert and commit\n\t_, err = sqldb.Exec(\"BEGIN\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (200, 'should_exist')\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"COMMIT\")\n\trequire.NoError(t, err)\n\n\t// Verify only the committed data exists\n\tvar count int\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users WHERE name = 'should_not_exist'\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 0, count, \"rolled back data should not exist\")\n\n\terr = sqldb.QueryRow(\"SELECT COUNT(*) FROM users WHERE name = 'should_exist'\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1, count, \"committed data should exist\")\n\n\t// Close and reopen to verify persistence\n\tsqldb.Close()\n\n\tvfs2 := newWritableVFS(t, client, 0, \"\")\n\tvfsName2 := fmt.Sprintf(\"litestream-verify-%d\", time.Now().UnixNano())\n\trequire.NoError(t, sqlite3vfs.RegisterVFS(vfsName2, vfs2))\n\n\tsqldb2, err := sql.Open(\"sqlite3\", fmt.Sprintf(\"file:test.db?vfs=%s\", vfsName2))\n\trequire.NoError(t, err)\n\tdefer sqldb2.Close()\n\n\terr = sqldb2.QueryRow(\"SELECT COUNT(*) FROM users WHERE name = 'should_exist'\").Scan(&count)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 1, count, \"committed data should persist\")\n}\n\n// =============================================================================\n// Helper Functions\n// =============================================================================\n\n// newWritableVFS creates a VFS with write support enabled.\nfunc newWritableVFS(tb testing.TB, client litestream.ReplicaClient, syncInterval time.Duration, localPath string) *litestream.VFS {\n\ttb.Helper()\n\n\tlogger := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{\n\t\tLevel: slog.LevelDebug,\n\t}))\n\n\tvfs := litestream.NewVFS(client, logger)\n\tvfs.PollInterval = 100 * time.Millisecond\n\tvfs.WriteEnabled = true\n\tvfs.WriteSyncInterval = syncInterval\n\t// If localPath is provided as a directory, append a buffer filename\n\tif localPath != \"\" {\n\t\tvfs.WriteBufferPath = filepath.Join(localPath, \".litestream-buffer\")\n\t}\n\n\treturn vfs\n}\n\n// newReadOnlyVFS creates a read-only VFS.\nfunc newReadOnlyVFS(tb testing.TB, client litestream.ReplicaClient) *litestream.VFS {\n\ttb.Helper()\n\n\tlogger := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{\n\t\tLevel: slog.LevelDebug,\n\t}))\n\n\tvfs := litestream.NewVFS(client, logger)\n\tvfs.PollInterval = 100 * time.Millisecond\n\n\treturn vfs\n}\n\n// setupInitialDB creates an initial database with standard schema.\nfunc setupInitialDB(t *testing.T, client litestream.ReplicaClient) {\n\tt.Helper()\n\n\tdbDir := t.TempDir()\n\tdb := testingutil.NewDB(t, filepath.Join(dbDir, \"source.db\"))\n\tdb.MonitorInterval = 50 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.SyncInterval = 50 * time.Millisecond\n\trequire.NoError(t, db.Open())\n\n\tsqldb := testingutil.MustOpenSQLDB(t, db.Path())\n\n\t_, err := sqldb.Exec(\"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)\")\n\trequire.NoError(t, err)\n\t_, err = sqldb.Exec(\"INSERT INTO users (id, name) VALUES (1, 'Alice')\")\n\trequire.NoError(t, err)\n\n\t// Wait for initial LTX file\n\twaitForLTXFiles(t, client, 10*time.Second, db.MonitorInterval)\n\n\t// Force a DB sync and then replica sync to ensure all data is uploaded\n\trequire.NoError(t, db.Sync(context.Background()))\n\trequire.NoError(t, db.Replica.Sync(context.Background()))\n\n\trequire.NoError(t, db.Replica.Stop(false))\n\ttestingutil.MustCloseSQLDB(t, sqldb)\n\trequire.NoError(t, db.Close(context.Background()))\n}\n\n// countLTXFiles returns the number of LTX files in the replica.\nfunc countLTXFiles(t *testing.T, client litestream.ReplicaClient) int {\n\tt.Helper()\n\n\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\trequire.NoError(t, err)\n\tdefer itr.Close()\n\n\tcount := 0\n\tfor itr.Next() {\n\t\tcount++\n\t}\n\treturn count\n}\n\n// addExternalLTXFile adds an LTX file to simulate an external writer.\nfunc addExternalLTXFile(t *testing.T, client litestream.ReplicaClient, replicaDir string) {\n\tt.Helper()\n\n\t// Get current max TXID\n\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\trequire.NoError(t, err)\n\n\tvar maxTXID ltx.TXID\n\tfor itr.Next() {\n\t\tif itr.Item().MaxTXID > maxTXID {\n\t\t\tmaxTXID = itr.Item().MaxTXID\n\t\t}\n\t}\n\titr.Close()\n\n\t// Create a new LTX file with next TXID\n\tnextTXID := maxTXID + 1\n\tvar buf bytes.Buffer\n\tenc, err := ltx.NewEncoder(&buf)\n\trequire.NoError(t, err)\n\n\trequire.NoError(t, enc.EncodeHeader(ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\tPageSize: 4096,\n\t\tCommit: 2,\n\t\tMinTXID: nextTXID,\n\t\tMaxTXID: nextTXID,\n\t\tTimestamp: time.Now().UnixMilli(),\n\t}))\n\n\t// Encode a dummy page\n\tpage := make([]byte, 4096)\n\trequire.NoError(t, enc.EncodePage(ltx.PageHeader{Pgno: 2}, page))\n\trequire.NoError(t, enc.Close())\n\n\t// Write via client\n\t_, err = client.WriteLTXFile(context.Background(), 0, nextTXID, nextTXID, &buf)\n\trequire.NoError(t, err)\n}\n\n// restoreDB restores a database from the replica to the given path.\nfunc restoreDB(t *testing.T, client litestream.ReplicaClient, outputPath string) error {\n\tt.Helper()\n\n\t// Open output file\n\tf, err := os.Create(outputPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer f.Close()\n\n\t// Get all LTX files\n\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer itr.Close()\n\n\tvar pageSize uint32\n\tpages := make(map[uint32][]byte)\n\tvar commit uint32\n\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\n\t\trc, err := client.OpenLTXFile(context.Background(), info.Level, info.MinTXID, info.MaxTXID, 0, 0)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdec := ltx.NewDecoder(rc)\n\t\tif err := dec.DecodeHeader(); err != nil {\n\t\t\trc.Close()\n\t\t\treturn err\n\t\t}\n\t\thdr := dec.Header()\n\n\t\tif pageSize == 0 {\n\t\t\tpageSize = hdr.PageSize\n\t\t}\n\t\tcommit = hdr.Commit\n\n\t\tpageBuf := make([]byte, hdr.PageSize)\n\t\tfor {\n\t\t\tvar phdr ltx.PageHeader\n\t\t\tif err := dec.DecodePage(&phdr, pageBuf); err != nil {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\t// Copy page data since pageBuf is reused\n\t\t\tdata := make([]byte, len(pageBuf))\n\t\t\tcopy(data, pageBuf)\n\t\t\tpages[phdr.Pgno] = data\n\t\t}\n\t\trc.Close()\n\t}\n\n\t// Write pages to file\n\tfor pgno := uint32(1); pgno <= commit; pgno++ {\n\t\tdata, ok := pages[pgno]\n\t\tif !ok {\n\t\t\tdata = make([]byte, pageSize)\n\t\t}\n\t\tif _, err := f.WriteAt(data, int64(pgno-1)*int64(pageSize)); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "compaction_level.go", "content": "package litestream\n\nimport (\n\t\"fmt\"\n\t\"time\"\n)\n\n// SnapshotLevel represents the level which full snapshots are held.\nconst SnapshotLevel = 9\n\n// DefaultCompactionLevels provides the canonical default compaction configuration.\n// Level 0 is raw LTX files, higher levels compact at increasing intervals.\n// These values are also used by cmd/litestream DefaultConfig().\nvar DefaultCompactionLevels = CompactionLevels{\n\t{Level: 0, Interval: 0},\n\t{Level: 1, Interval: 30 * time.Second},\n\t{Level: 2, Interval: 5 * time.Minute},\n\t{Level: 3, Interval: time.Hour},\n}\n\n// CompactionLevel represents a single part of a multi-level compaction.\n// Each level merges LTX files from the previous level into larger time granularities.\ntype CompactionLevel struct {\n\t// The numeric level. Must match the index in the list of levels.\n\tLevel int\n\n\t// The frequency that the level is compacted from the previous level.\n\tInterval time.Duration\n}\n\n// PrevCompactionAt returns the time when the last compaction occurred.\n// Returns the current time if it is exactly a multiple of the level interval.\nfunc (lvl *CompactionLevel) PrevCompactionAt(now time.Time) time.Time {\n\treturn now.Truncate(lvl.Interval).UTC()\n}\n\n// NextCompactionAt returns the time until the next compaction occurs.\n// Returns the current time if it is exactly a multiple of the level interval.\nfunc (lvl *CompactionLevel) NextCompactionAt(now time.Time) time.Time {\n\treturn lvl.PrevCompactionAt(now).Add(lvl.Interval)\n}\n\n// CompactionLevels represents a sorted slice of non-snapshot compaction levels.\ntype CompactionLevels []*CompactionLevel\n\n// Level returns the compaction level at the given index.\n// Returns an error if the index is a snapshot level or is out of bounds.\nfunc (a CompactionLevels) Level(level int) (*CompactionLevel, error) {\n\tif level == SnapshotLevel {\n\t\treturn nil, fmt.Errorf(\"invalid argument, snapshot level\")\n\t}\n\tif level < 0 || level > a.MaxLevel() {\n\t\treturn nil, fmt.Errorf(\"level out of bounds: %d\", level)\n\t}\n\treturn a[level], nil\n}\n\n// MaxLevel return the highest non-snapshot compaction level.\nfunc (a CompactionLevels) MaxLevel() int {\n\treturn len(a) - 1\n}\n\n// Validate returns an error if the levels are invalid.\nfunc (a CompactionLevels) Validate() error {\n\tif len(a) == 0 {\n\t\treturn fmt.Errorf(\"at least one compaction level is required\")\n\t}\n\n\tfor i, lvl := range a {\n\t\tif i != lvl.Level {\n\t\t\treturn fmt.Errorf(\"compaction level number out of order: %d, expected %d\", lvl.Level, i)\n\t\t} else if lvl.Level > SnapshotLevel-1 {\n\t\t\treturn fmt.Errorf(\"compaction level cannot exceed %d\", SnapshotLevel-1)\n\t\t}\n\n\t\tif lvl.Level == 0 && lvl.Interval != 0 {\n\t\t\treturn fmt.Errorf(\"cannot set interval on compaction level zero\")\n\t\t}\n\n\t\tif lvl.Level != 0 && lvl.Interval <= 0 {\n\t\t\treturn fmt.Errorf(\"interval required for level %d\", lvl.Level)\n\t\t}\n\t}\n\treturn nil\n}\n\n// IsValidLevel returns true if level is a valid compaction level number.\nfunc (a CompactionLevels) IsValidLevel(level int) bool {\n\tif level == SnapshotLevel {\n\t\treturn true\n\t}\n\treturn level >= 0 && level < len(a)\n}\n\n// PrevLevel returns the previous compaction level.\n// Returns -1 if there is no previous level.\nfunc (a CompactionLevels) PrevLevel(level int) int {\n\tif level == SnapshotLevel {\n\t\treturn a.MaxLevel()\n\t}\n\treturn level - 1\n}\n\n// NextLevel returns the next compaction level.\n// Returns -1 if there is no next level.\nfunc (a CompactionLevels) NextLevel(level int) int {\n\tif level == SnapshotLevel {\n\t\treturn -1\n\t} else if level == a.MaxLevel() {\n\t\treturn SnapshotLevel\n\t}\n\treturn level + 1\n}\n" }, { "path": "compactor.go", "content": "package litestream\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/prometheus/client_golang/prometheus\"\n\t\"github.com/superfly/ltx\"\n)\n\n// Compactor handles compaction and retention for LTX files.\n// It operates solely through the ReplicaClient interface, making it\n// suitable for both DB (with local file caching) and VFS (remote-only).\ntype Compactor struct {\n\tclient ReplicaClient\n\tlogger *slog.Logger\n\n\t// VerifyCompaction enables post-compaction TXID consistency verification.\n\t// When enabled, verifies that files at the destination level have\n\t// contiguous TXID ranges after each compaction. Disabled by default.\n\tVerifyCompaction bool\n\n\t// RetentionEnabled controls whether Litestream actively deletes old files\n\t// during retention enforcement. When false, cloud provider lifecycle\n\t// policies handle retention instead. Local file cleanup still occurs.\n\tRetentionEnabled bool\n\n\t// CompactionVerifyErrorCounter is incremented when post-compaction\n\t// verification fails. Optional; if nil, no metric is recorded.\n\tCompactionVerifyErrorCounter prometheus.Counter\n\n\t// LocalFileOpener optionally opens a local LTX file for compaction.\n\t// If nil or returns os.ErrNotExist, falls back to remote.\n\t// This is used by DB to prefer local files over remote for consistency.\n\tLocalFileOpener func(level int, minTXID, maxTXID ltx.TXID) (io.ReadCloser, error)\n\n\t// LocalFileDeleter optionally deletes local LTX files after retention.\n\t// If nil, only remote files are deleted.\n\tLocalFileDeleter func(level int, minTXID, maxTXID ltx.TXID) error\n\n\t// CacheGetter optionally retrieves cached MaxLTXFileInfo for a level.\n\t// If nil, max file info is always fetched from remote.\n\tCacheGetter func(level int) (*ltx.FileInfo, bool)\n\n\t// CacheSetter optionally stores MaxLTXFileInfo for a level.\n\t// If nil, max file info is not cached.\n\tCacheSetter func(level int, info *ltx.FileInfo)\n}\n\n// NewCompactor creates a new Compactor with the given client and logger.\nfunc NewCompactor(client ReplicaClient, logger *slog.Logger) *Compactor {\n\tif logger == nil {\n\t\tlogger = slog.Default()\n\t}\n\treturn &Compactor{\n\t\tclient: client,\n\t\tlogger: logger,\n\t\tRetentionEnabled: true,\n\t}\n}\n\nfunc (c *Compactor) setLogger(logger *slog.Logger) {\n\tc.logger = logger\n}\n\n// MaxLTXFileInfo returns metadata for the last LTX file in a level.\n// Uses cache if available, otherwise fetches from remote.\nfunc (c *Compactor) MaxLTXFileInfo(ctx context.Context, level int) (ltx.FileInfo, error) {\n\tif c.CacheGetter != nil {\n\t\tif info, ok := c.CacheGetter(level); ok {\n\t\t\treturn *info, nil\n\t\t}\n\t}\n\n\titr, err := c.client.LTXFiles(ctx, level, 0, false)\n\tif err != nil {\n\t\treturn ltx.FileInfo{}, err\n\t}\n\tdefer itr.Close()\n\n\tvar info ltx.FileInfo\n\tfor itr.Next() {\n\t\titem := itr.Item()\n\t\tif item.MaxTXID > info.MaxTXID {\n\t\t\tinfo = *item\n\t\t}\n\t}\n\n\tif c.CacheSetter != nil && info.MaxTXID > 0 {\n\t\tc.CacheSetter(level, &info)\n\t}\n\n\treturn info, itr.Close()\n}\n\n// Compact compacts source level files into the destination level.\n// Returns ErrNoCompaction if there are no files to compact.\nfunc (c *Compactor) Compact(ctx context.Context, dstLevel int) (*ltx.FileInfo, error) {\n\tsrcLevel := dstLevel - 1\n\n\tprevMaxInfo, err := c.MaxLTXFileInfo(ctx, dstLevel)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"cannot determine max ltx file for destination level: %w\", err)\n\t}\n\tseekTXID := prevMaxInfo.MaxTXID + 1\n\n\titr, err := c.client.LTXFiles(ctx, srcLevel, seekTXID, false)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"source ltx files after %s: %w\", seekTXID, err)\n\t}\n\tdefer itr.Close()\n\n\tvar rdrs []io.Reader\n\tdefer func() {\n\t\tfor _, rd := range rdrs {\n\t\t\tif closer, ok := rd.(io.Closer); ok {\n\t\t\t\t_ = closer.Close()\n\t\t\t}\n\t\t}\n\t}()\n\n\tvar minTXID, maxTXID ltx.TXID\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\n\t\tif minTXID == 0 || info.MinTXID < minTXID {\n\t\t\tminTXID = info.MinTXID\n\t\t}\n\t\tif maxTXID == 0 || info.MaxTXID > maxTXID {\n\t\t\tmaxTXID = info.MaxTXID\n\t\t}\n\n\t\tif c.LocalFileOpener != nil {\n\t\t\tif f, err := c.LocalFileOpener(srcLevel, info.MinTXID, info.MaxTXID); err == nil {\n\t\t\t\trdrs = append(rdrs, f)\n\t\t\t\tcontinue\n\t\t\t} else if !os.IsNotExist(err) {\n\t\t\t\treturn nil, fmt.Errorf(\"open local ltx file: %w\", err)\n\t\t\t}\n\t\t}\n\n\t\tf, err := c.client.OpenLTXFile(ctx, info.Level, info.MinTXID, info.MaxTXID, 0, 0)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"open ltx file: %w\", err)\n\t\t}\n\t\trdrs = append(rdrs, f)\n\t}\n\tif len(rdrs) == 0 {\n\t\treturn nil, ErrNoCompaction\n\t}\n\n\tpr, pw := io.Pipe()\n\tgo func() {\n\t\tcomp, err := ltx.NewCompactor(pw, rdrs)\n\t\tif err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"new ltx compactor: %w\", err))\n\t\t\treturn\n\t\t}\n\t\tcomp.HeaderFlags = ltx.HeaderFlagNoChecksum\n\t\t_ = pw.CloseWithError(comp.Compact(ctx))\n\t}()\n\n\tinfo, err := c.client.WriteLTXFile(ctx, dstLevel, minTXID, maxTXID, pr)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"write ltx file: %w\", err)\n\t}\n\n\tif c.CacheSetter != nil {\n\t\tc.CacheSetter(dstLevel, info)\n\t}\n\n\t// Verify level consistency if enabled\n\tif c.VerifyCompaction {\n\t\tif err := c.VerifyLevelConsistency(ctx, dstLevel); err != nil {\n\t\t\tc.logger.Warn(\"post-compaction verification failed\",\n\t\t\t\t\"level\", dstLevel,\n\t\t\t\t\"error\", err)\n\t\t\tif c.CompactionVerifyErrorCounter != nil {\n\t\t\t\tc.CompactionVerifyErrorCounter.Inc()\n\t\t\t}\n\t\t}\n\t}\n\n\treturn info, nil\n}\n\n// VerifyLevelConsistency checks that LTX files at the given level have\n// contiguous TXID ranges (prevMaxTXID + 1 == currMinTXID for consecutive files).\n// Returns an error describing any gaps or overlaps found.\nfunc (c *Compactor) VerifyLevelConsistency(ctx context.Context, level int) error {\n\titr, err := c.client.LTXFiles(ctx, level, 0, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"fetch ltx files: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar prevInfo *ltx.FileInfo\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\n\t\t// Skip first file - nothing to compare against\n\t\tif prevInfo == nil {\n\t\t\tprevInfo = info\n\t\t\tcontinue\n\t\t}\n\n\t\t// Check for TXID contiguity: prev.MaxTXID + 1 should equal curr.MinTXID\n\t\texpectedMinTXID := prevInfo.MaxTXID + 1\n\t\tif info.MinTXID != expectedMinTXID {\n\t\t\tif info.MinTXID > expectedMinTXID {\n\t\t\t\treturn fmt.Errorf(\"TXID gap detected: prev.MaxTXID=%s, next.MinTXID=%s (expected %s)\",\n\t\t\t\t\tprevInfo.MaxTXID, info.MinTXID, expectedMinTXID)\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"TXID overlap detected: prev.MaxTXID=%s, next.MinTXID=%s\",\n\t\t\t\tprevInfo.MaxTXID, info.MinTXID)\n\t\t}\n\n\t\tprevInfo = info\n\t}\n\n\tif err := itr.Close(); err != nil {\n\t\treturn fmt.Errorf(\"close iterator: %w\", err)\n\t}\n\n\treturn nil\n}\n\n// EnforceSnapshotRetention enforces retention of snapshot level files by timestamp.\n// Files older than the retention duration are deleted (except the newest is always kept).\n// Returns the minimum snapshot TXID still retained (useful for cascading retention to lower levels).\nfunc (c *Compactor) EnforceSnapshotRetention(ctx context.Context, retention time.Duration) (ltx.TXID, error) {\n\ttimestamp := time.Now().Add(-retention)\n\tc.logger.Debug(\"enforcing snapshot retention\", \"timestamp\", timestamp)\n\n\titr, err := c.client.LTXFiles(ctx, SnapshotLevel, 0, false)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"fetch ltx files: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar deleted []*ltx.FileInfo\n\tvar lastInfo *ltx.FileInfo\n\tvar minSnapshotTXID ltx.TXID\n\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tlastInfo = info\n\n\t\tif info.CreatedAt.Before(timestamp) {\n\t\t\tdeleted = append(deleted, info)\n\t\t\tcontinue\n\t\t}\n\n\t\tif minSnapshotTXID == 0 || info.MaxTXID < minSnapshotTXID {\n\t\t\tminSnapshotTXID = info.MaxTXID\n\t\t}\n\t}\n\n\tif len(deleted) > 0 && deleted[len(deleted)-1] == lastInfo {\n\t\tdeleted = deleted[:len(deleted)-1]\n\t}\n\n\tif !c.RetentionEnabled {\n\t\tc.logger.Debug(\"skipping remote deletion (retention disabled)\", \"level\", SnapshotLevel, \"count\", len(deleted))\n\t} else if err := c.client.DeleteLTXFiles(ctx, deleted); err != nil {\n\t\treturn 0, fmt.Errorf(\"remove ltx files: %w\", err)\n\t}\n\n\tif c.LocalFileDeleter != nil {\n\t\tfor _, info := range deleted {\n\t\t\tc.logger.Debug(\"deleting local ltx file\",\n\t\t\t\t\"level\", SnapshotLevel,\n\t\t\t\t\"minTXID\", info.MinTXID,\n\t\t\t\t\"maxTXID\", info.MaxTXID)\n\t\t\tif err := c.LocalFileDeleter(SnapshotLevel, info.MinTXID, info.MaxTXID); err != nil {\n\t\t\t\tc.logger.Error(\"failed to remove local ltx file\", \"error\", err)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn minSnapshotTXID, nil\n}\n\n// EnforceRetentionByTXID deletes files at the given level with maxTXID below the target.\n// Always keeps at least one file.\nfunc (c *Compactor) EnforceRetentionByTXID(ctx context.Context, level int, txID ltx.TXID) error {\n\tc.logger.Debug(\"enforcing retention\", \"level\", level, \"txid\", txID)\n\n\titr, err := c.client.LTXFiles(ctx, level, 0, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"fetch ltx files: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar deleted []*ltx.FileInfo\n\tvar lastInfo *ltx.FileInfo\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tlastInfo = info\n\n\t\tif info.MaxTXID < txID {\n\t\t\tdeleted = append(deleted, info)\n\t\t\tcontinue\n\t\t}\n\t}\n\n\tif len(deleted) > 0 && deleted[len(deleted)-1] == lastInfo {\n\t\tdeleted = deleted[:len(deleted)-1]\n\t}\n\n\tif !c.RetentionEnabled {\n\t\tc.logger.Debug(\"skipping remote deletion (retention disabled)\", \"level\", level, \"count\", len(deleted))\n\t} else if err := c.client.DeleteLTXFiles(ctx, deleted); err != nil {\n\t\treturn fmt.Errorf(\"remove ltx files: %w\", err)\n\t}\n\n\tif c.LocalFileDeleter != nil {\n\t\tfor _, info := range deleted {\n\t\t\tc.logger.Debug(\"deleting local ltx file\",\n\t\t\t\t\"level\", level,\n\t\t\t\t\"minTXID\", info.MinTXID,\n\t\t\t\t\"maxTXID\", info.MaxTXID)\n\t\t\tif err := c.LocalFileDeleter(level, info.MinTXID, info.MaxTXID); err != nil {\n\t\t\t\tc.logger.Error(\"failed to remove local ltx file\", \"error\", err)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// EnforceL0Retention retains L0 files based on L1 compaction progress and time.\n// Files are only deleted if they have been compacted into L1 AND are older than retention.\n// This ensures contiguous L0 coverage for VFS reads.\nfunc (c *Compactor) EnforceL0Retention(ctx context.Context, retention time.Duration) error {\n\tif retention <= 0 {\n\t\treturn nil\n\t}\n\n\tc.logger.Debug(\"enforcing l0 retention\", \"retention\", retention)\n\n\titr, err := c.client.LTXFiles(ctx, 1, 0, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"fetch l1 files: %w\", err)\n\t}\n\tvar maxL1TXID ltx.TXID\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tif info.MaxTXID > maxL1TXID {\n\t\t\tmaxL1TXID = info.MaxTXID\n\t\t}\n\t}\n\tif err := itr.Close(); err != nil {\n\t\treturn fmt.Errorf(\"close l1 iterator: %w\", err)\n\t}\n\tif maxL1TXID == 0 {\n\t\treturn nil\n\t}\n\n\tthreshold := time.Now().Add(-retention)\n\titr, err = c.client.LTXFiles(ctx, 0, 0, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"fetch l0 files: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar (\n\t\tdeleted []*ltx.FileInfo\n\t\tlastInfo *ltx.FileInfo\n\t\tprocessedAll = true\n\t)\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tlastInfo = info\n\n\t\tcreatedAt := info.CreatedAt\n\t\tif createdAt.IsZero() {\n\t\t\tcreatedAt = threshold\n\t\t}\n\n\t\tif createdAt.After(threshold) {\n\t\t\tprocessedAll = false\n\t\t\tbreak\n\t\t}\n\n\t\tif info.MaxTXID <= maxL1TXID {\n\t\t\tdeleted = append(deleted, info)\n\t\t}\n\t}\n\n\tif processedAll && len(deleted) > 0 && lastInfo != nil && deleted[len(deleted)-1] == lastInfo {\n\t\tdeleted = deleted[:len(deleted)-1]\n\t}\n\n\tif len(deleted) == 0 {\n\t\treturn nil\n\t}\n\n\tif !c.RetentionEnabled {\n\t\tc.logger.Debug(\"skipping remote deletion (retention disabled)\", \"level\", 0, \"count\", len(deleted))\n\t} else if err := c.client.DeleteLTXFiles(ctx, deleted); err != nil {\n\t\treturn fmt.Errorf(\"remove expired l0 files: %w\", err)\n\t}\n\n\tif c.LocalFileDeleter != nil {\n\t\tfor _, info := range deleted {\n\t\t\tc.logger.Debug(\"deleting expired local l0 file\",\n\t\t\t\t\"minTXID\", info.MinTXID,\n\t\t\t\t\"maxTXID\", info.MaxTXID)\n\t\t\tif err := c.LocalFileDeleter(0, info.MinTXID, info.MaxTXID); err != nil {\n\t\t\t\tc.logger.Error(\"failed to remove local l0 file\", \"error\", err)\n\t\t\t}\n\t\t}\n\t}\n\n\tc.logger.Info(\"l0 retention enforced\", \"deleted_count\", len(deleted), \"max_l1_txid\", maxL1TXID)\n\n\treturn nil\n}\n" }, { "path": "compactor_test.go", "content": "package litestream_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"io\"\n\t\"log/slog\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n)\n\nfunc TestCompactor_Compact(t *testing.T) {\n\tt.Run(\"L0ToL1\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create test L0 files\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 2, 2)\n\n\t\tinfo, err := compactor.Compact(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.Level != 1 {\n\t\t\tt.Errorf(\"Level=%d, want 1\", info.Level)\n\t\t}\n\t\tif info.MinTXID != 1 || info.MaxTXID != 2 {\n\t\t\tt.Errorf(\"TXID range=%d-%d, want 1-2\", info.MinTXID, info.MaxTXID)\n\t\t}\n\t})\n\n\tt.Run(\"NoFiles\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t_, err := compactor.Compact(context.Background(), 1)\n\t\tif err != litestream.ErrNoCompaction {\n\t\t\tt.Errorf(\"err=%v, want ErrNoCompaction\", err)\n\t\t}\n\t})\n\n\tt.Run(\"L1ToL2\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create L0 files\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 2, 2)\n\n\t\t// Compact to L1\n\t\t_, err := compactor.Compact(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create more L0 files\n\t\tcreateTestLTXFile(t, client, 0, 3, 3)\n\n\t\t// Compact to L1 again (should only include TXID 3)\n\t\tinfo, err := compactor.Compact(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.MinTXID != 3 || info.MaxTXID != 3 {\n\t\t\tt.Errorf(\"TXID range=%d-%d, want 3-3\", info.MinTXID, info.MaxTXID)\n\t\t}\n\n\t\t// Now compact L1 to L2 (should include all from 1-3)\n\t\tinfo, err = compactor.Compact(context.Background(), 2)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.Level != 2 {\n\t\t\tt.Errorf(\"Level=%d, want 2\", info.Level)\n\t\t}\n\t\tif info.MinTXID != 1 || info.MaxTXID != 3 {\n\t\t\tt.Errorf(\"TXID range=%d-%d, want 1-3\", info.MinTXID, info.MaxTXID)\n\t\t}\n\t})\n}\n\nfunc TestCompactor_MaxLTXFileInfo(t *testing.T) {\n\tt.Run(\"WithFiles\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 2, 2)\n\t\tcreateTestLTXFile(t, client, 0, 3, 5)\n\n\t\tinfo, err := compactor.MaxLTXFileInfo(context.Background(), 0)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.MaxTXID != 5 {\n\t\t\tt.Errorf(\"MaxTXID=%d, want 5\", info.MaxTXID)\n\t\t}\n\t})\n\n\tt.Run(\"NoFiles\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\tinfo, err := compactor.MaxLTXFileInfo(context.Background(), 0)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.MaxTXID != 0 {\n\t\t\tt.Errorf(\"MaxTXID=%d, want 0\", info.MaxTXID)\n\t\t}\n\t})\n\n\tt.Run(\"WithCache\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Use callbacks for caching\n\t\tcache := make(map[int]*ltx.FileInfo)\n\t\tcompactor.CacheGetter = func(level int) (*ltx.FileInfo, bool) {\n\t\t\tinfo, ok := cache[level]\n\t\t\treturn info, ok\n\t\t}\n\t\tcompactor.CacheSetter = func(level int, info *ltx.FileInfo) {\n\t\t\tcache[level] = info\n\t\t}\n\n\t\tcreateTestLTXFile(t, client, 0, 1, 3)\n\n\t\t// First call should populate cache\n\t\tinfo, err := compactor.MaxLTXFileInfo(context.Background(), 0)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.MaxTXID != 3 {\n\t\t\tt.Errorf(\"MaxTXID=%d, want 3\", info.MaxTXID)\n\t\t}\n\n\t\t// Second call should use cache\n\t\tinfo, err = compactor.MaxLTXFileInfo(context.Background(), 0)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.MaxTXID != 3 {\n\t\t\tt.Errorf(\"MaxTXID=%d, want 3 (from cache)\", info.MaxTXID)\n\t\t}\n\t})\n}\n\nfunc TestCompactor_EnforceRetentionByTXID(t *testing.T) {\n\tt.Run(\"DeletesOldFiles\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create files at L1\n\t\tcreateTestLTXFile(t, client, 1, 1, 2)\n\t\tcreateTestLTXFile(t, client, 1, 3, 5)\n\t\tcreateTestLTXFile(t, client, 1, 6, 10)\n\n\t\t// Enforce retention - delete files below TXID 5\n\t\terr := compactor.EnforceRetentionByTXID(context.Background(), 1, 5)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Verify only the first file was deleted\n\t\tinfo, err := compactor.MaxLTXFileInfo(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.MaxTXID != 10 {\n\t\t\tt.Errorf(\"MaxTXID=%d, want 10\", info.MaxTXID)\n\t\t}\n\n\t\t// Check that files starting from TXID 3 are still present\n\t\titr, err := client.LTXFiles(context.Background(), 1, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer itr.Close()\n\n\t\tvar count int\n\t\tfor itr.Next() {\n\t\t\tcount++\n\t\t}\n\t\tif count != 2 {\n\t\t\tt.Errorf(\"file count=%d, want 2\", count)\n\t\t}\n\t})\n\n\tt.Run(\"KeepsLastFile\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create single file\n\t\tcreateTestLTXFile(t, client, 1, 1, 2)\n\n\t\t// Try to delete it - should keep at least one\n\t\terr := compactor.EnforceRetentionByTXID(context.Background(), 1, 100)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Verify file still exists\n\t\tinfo, err := compactor.MaxLTXFileInfo(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.MaxTXID != 2 {\n\t\t\tt.Errorf(\"MaxTXID=%d, want 2 (last file should be kept)\", info.MaxTXID)\n\t\t}\n\t})\n}\n\nfunc TestCompactor_EnforceL0Retention(t *testing.T) {\n\tt.Run(\"DeletesCompactedFiles\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create L0 files\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 2, 2)\n\t\tcreateTestLTXFile(t, client, 0, 3, 3)\n\n\t\t// Compact to L1\n\t\t_, err := compactor.Compact(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Enforce L0 retention with 0 duration (delete immediately)\n\t\terr = compactor.EnforceL0Retention(context.Background(), 0)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// L0 files compacted into L1 should be deleted (except last)\n\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer itr.Close()\n\n\t\tvar count int\n\t\tfor itr.Next() {\n\t\t\tcount++\n\t\t}\n\t\t// At least one file should remain\n\t\tif count < 1 {\n\t\t\tt.Errorf(\"file count=%d, want at least 1\", count)\n\t\t}\n\t})\n\n\tt.Run(\"SkipsIfNoL1\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create L0 files without compacting to L1\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 2, 2)\n\n\t\t// Enforce L0 retention - should do nothing since no L1 exists\n\t\terr := compactor.EnforceL0Retention(context.Background(), 0)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// All L0 files should still exist\n\t\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer itr.Close()\n\n\t\tvar count int\n\t\tfor itr.Next() {\n\t\t\tcount++\n\t\t}\n\t\tif count != 2 {\n\t\t\tt.Errorf(\"file count=%d, want 2\", count)\n\t\t}\n\t})\n}\n\nfunc TestCompactor_EnforceSnapshotRetention(t *testing.T) {\n\tt.Run(\"DeletesOldSnapshots\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create snapshot files with different timestamps\n\t\tcreateTestLTXFileWithTimestamp(t, client, litestream.SnapshotLevel, 1, 5, time.Now().Add(-2*time.Hour))\n\t\tcreateTestLTXFileWithTimestamp(t, client, litestream.SnapshotLevel, 1, 10, time.Now().Add(-30*time.Minute))\n\t\tcreateTestLTXFileWithTimestamp(t, client, litestream.SnapshotLevel, 1, 15, time.Now().Add(-5*time.Minute))\n\n\t\t// Enforce retention - keep snapshots from last hour\n\t\t_, err := compactor.EnforceSnapshotRetention(context.Background(), time.Hour)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Count remaining snapshots\n\t\titr, err := client.LTXFiles(context.Background(), litestream.SnapshotLevel, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer itr.Close()\n\n\t\tvar count int\n\t\tfor itr.Next() {\n\t\t\tcount++\n\t\t}\n\t\t// Should have 2 snapshots (the 30min and 5min old ones)\n\t\tif count != 2 {\n\t\t\tt.Errorf(\"snapshot count=%d, want 2\", count)\n\t\t}\n\t})\n}\n\nfunc TestCompactor_EnforceSnapshotRetention_RetentionDisabled(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tcompactor := litestream.NewCompactor(client, slog.Default())\n\tcompactor.RetentionEnabled = false\n\n\tvar localDeleted []ltx.TXID\n\tcompactor.LocalFileDeleter = func(level int, minTXID, maxTXID ltx.TXID) error {\n\t\tlocalDeleted = append(localDeleted, maxTXID)\n\t\treturn nil\n\t}\n\n\tcreateTestLTXFileWithTimestamp(t, client, litestream.SnapshotLevel, 1, 5, time.Now().Add(-2*time.Hour))\n\tcreateTestLTXFileWithTimestamp(t, client, litestream.SnapshotLevel, 1, 10, time.Now().Add(-30*time.Minute))\n\tcreateTestLTXFileWithTimestamp(t, client, litestream.SnapshotLevel, 1, 15, time.Now().Add(-5*time.Minute))\n\n\t_, err := compactor.EnforceSnapshotRetention(context.Background(), time.Hour)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Remote files should all still exist (skip remote deletion).\n\titr, err := client.LTXFiles(context.Background(), litestream.SnapshotLevel, 0, false)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer itr.Close()\n\n\tvar count int\n\tfor itr.Next() {\n\t\tcount++\n\t}\n\tif count != 3 {\n\t\tt.Errorf(\"remote file count=%d, want 3 (no remote deletion)\", count)\n\t}\n\n\t// Local file deleter should still have been called.\n\tif len(localDeleted) != 1 {\n\t\tt.Errorf(\"local deleted count=%d, want 1\", len(localDeleted))\n\t}\n}\n\nfunc TestCompactor_EnforceRetentionByTXID_RetentionDisabled(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tcompactor := litestream.NewCompactor(client, slog.Default())\n\tcompactor.RetentionEnabled = false\n\n\tvar localDeleted []ltx.TXID\n\tcompactor.LocalFileDeleter = func(level int, minTXID, maxTXID ltx.TXID) error {\n\t\tlocalDeleted = append(localDeleted, maxTXID)\n\t\treturn nil\n\t}\n\n\tcreateTestLTXFile(t, client, 1, 1, 2)\n\tcreateTestLTXFile(t, client, 1, 3, 5)\n\tcreateTestLTXFile(t, client, 1, 6, 10)\n\n\terr := compactor.EnforceRetentionByTXID(context.Background(), 1, 5)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Remote files should all still exist.\n\titr, err := client.LTXFiles(context.Background(), 1, 0, false)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer itr.Close()\n\n\tvar count int\n\tfor itr.Next() {\n\t\tcount++\n\t}\n\tif count != 3 {\n\t\tt.Errorf(\"remote file count=%d, want 3 (no remote deletion)\", count)\n\t}\n\n\t// Local file deleter should still have been called for the file below TXID 5.\n\tif len(localDeleted) != 1 {\n\t\tt.Errorf(\"local deleted count=%d, want 1\", len(localDeleted))\n\t}\n}\n\nfunc TestCompactor_EnforceL0Retention_RetentionDisabled(t *testing.T) {\n\tclient := file.NewReplicaClient(t.TempDir())\n\tcompactor := litestream.NewCompactor(client, slog.Default())\n\tcompactor.RetentionEnabled = false\n\n\tvar localDeleted []ltx.TXID\n\tcompactor.LocalFileDeleter = func(level int, minTXID, maxTXID ltx.TXID) error {\n\t\tlocalDeleted = append(localDeleted, maxTXID)\n\t\treturn nil\n\t}\n\n\t// Create L0 files with old timestamps so they're eligible for deletion.\n\toldTime := time.Now().Add(-1 * time.Hour)\n\tcreateTestLTXFileWithTimestamp(t, client, 0, 1, 1, oldTime)\n\tcreateTestLTXFileWithTimestamp(t, client, 0, 2, 2, oldTime)\n\tcreateTestLTXFileWithTimestamp(t, client, 0, 3, 3, oldTime)\n\n\t// Compact to L1 first.\n\t_, err := compactor.Compact(context.Background(), 1)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Use a real retention duration so the check doesn't return early.\n\terr = compactor.EnforceL0Retention(context.Background(), time.Minute)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Remote L0 files should all still exist.\n\titr, err := client.LTXFiles(context.Background(), 0, 0, false)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer itr.Close()\n\n\tvar count int\n\tfor itr.Next() {\n\t\tcount++\n\t}\n\tif count != 3 {\n\t\tt.Errorf(\"remote file count=%d, want 3 (no remote deletion)\", count)\n\t}\n\n\t// Local file deleter should still have been called for compacted files.\n\tif len(localDeleted) < 1 {\n\t\tt.Errorf(\"local deleted count=%d, want at least 1\", len(localDeleted))\n\t}\n}\n\nfunc TestCompactor_VerifyLevelConsistency(t *testing.T) {\n\tt.Run(\"ContiguousFiles\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create contiguous files\n\t\tcreateTestLTXFile(t, client, 1, 1, 2)\n\t\tcreateTestLTXFile(t, client, 1, 3, 5)\n\t\tcreateTestLTXFile(t, client, 1, 6, 10)\n\n\t\t// Should pass verification\n\t\terr := compactor.VerifyLevelConsistency(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected nil error for contiguous files, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"GapDetected\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create files with a gap (missing TXID 3-4)\n\t\tcreateTestLTXFile(t, client, 1, 1, 2)\n\t\tcreateTestLTXFile(t, client, 1, 5, 7) // gap: expected MinTXID=3, got 5\n\n\t\terr := compactor.VerifyLevelConsistency(context.Background(), 1)\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for gap in files, got nil\")\n\t\t}\n\t\tif err != nil && !containsString(err.Error(), \"gap\") {\n\t\t\tt.Errorf(\"expected gap error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"OverlapDetected\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create overlapping files\n\t\tcreateTestLTXFile(t, client, 1, 1, 5)\n\t\tcreateTestLTXFile(t, client, 1, 3, 7) // overlap: expected MinTXID=6, got 3\n\n\t\terr := compactor.VerifyLevelConsistency(context.Background(), 1)\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for overlapping files, got nil\")\n\t\t}\n\t\tif err != nil && !containsString(err.Error(), \"overlap\") {\n\t\t\tt.Errorf(\"expected overlap error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"SingleFile\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Create single file - should pass\n\t\tcreateTestLTXFile(t, client, 1, 1, 5)\n\n\t\terr := compactor.VerifyLevelConsistency(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected nil error for single file, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"EmptyLevel\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Empty level - should pass\n\t\terr := compactor.VerifyLevelConsistency(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected nil error for empty level, got: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestCompactor_CompactWithVerification(t *testing.T) {\n\tt.Run(\"VerificationEnabled\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\t\tcompactor.VerifyCompaction = true\n\n\t\t// Create contiguous L0 files\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 2, 2)\n\t\tcreateTestLTXFile(t, client, 0, 3, 3)\n\n\t\t// Compact to L1 - should succeed with verification\n\t\tinfo, err := compactor.Compact(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.Level != 1 {\n\t\t\tt.Errorf(\"Level=%d, want 1\", info.Level)\n\t\t}\n\t\tif info.MinTXID != 1 || info.MaxTXID != 3 {\n\t\t\tt.Errorf(\"TXID range=%d-%d, want 1-3\", info.MinTXID, info.MaxTXID)\n\t\t}\n\t})\n}\n\n// containsString checks if s contains substr.\nfunc containsString(s, substr string) bool {\n\treturn bytes.Contains([]byte(s), []byte(substr))\n}\n\n// createTestLTXFile creates a minimal LTX file for testing.\nfunc createTestLTXFile(t testing.TB, client litestream.ReplicaClient, level int, minTXID, maxTXID ltx.TXID) {\n\tt.Helper()\n\tcreateTestLTXFileWithTimestamp(t, client, level, minTXID, maxTXID, time.Now())\n}\n\n// createTestLTXFileWithTimestamp creates a minimal LTX file with a specific timestamp.\nfunc createTestLTXFileWithTimestamp(t testing.TB, client litestream.ReplicaClient, level int, minTXID, maxTXID ltx.TXID, ts time.Time) {\n\tt.Helper()\n\n\tvar buf bytes.Buffer\n\tenc, err := ltx.NewEncoder(&buf)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := enc.EncodeHeader(ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\tPageSize: 4096,\n\t\tCommit: 1,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tTimestamp: ts.UnixMilli(),\n\t}); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Write a dummy page\n\tif err := enc.EncodePage(ltx.PageHeader{Pgno: 1}, make([]byte, 4096)); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := enc.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, err := client.WriteLTXFile(context.Background(), level, minTXID, maxTXID, io.NopCloser(&buf)); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n" }, { "path": "db.go", "content": "package litestream\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"database/sql\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"fmt\"\n\t\"hash/crc64\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"slices\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/prometheus/client_golang/prometheus\"\n\t\"github.com/prometheus/client_golang/prometheus/promauto\"\n\t\"github.com/superfly/ltx\"\n\t\"modernc.org/sqlite\"\n\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\n// Default DB settings.\nconst (\n\tDefaultMonitorInterval = 1 * time.Second\n\tDefaultCheckpointInterval = 1 * time.Minute\n\tDefaultBusyTimeout = 1 * time.Second\n\tDefaultMinCheckpointPageN = 1000\n\tDefaultTruncatePageN = 121359 // ~500MB with 4KB page size\n\tDefaultShutdownSyncTimeout = 30 * time.Second\n\tDefaultShutdownSyncInterval = 500 * time.Millisecond\n\n\t// Sync error backoff configuration.\n\t// When sync errors occur repeatedly (e.g., disk full), backoff doubles each time.\n\tDefaultSyncBackoffMax = 5 * time.Minute // Maximum backoff between retries\n\tSyncErrorLogInterval = 30 * time.Second // Rate-limit repeated error logging\n)\n\n// DB represents a managed instance of a SQLite database in the file system.\n//\n// Checkpoint Strategy:\n// Litestream uses a progressive 3-tier checkpoint approach to balance WAL size\n// management with write availability:\n//\n// 1. MinCheckpointPageN (PASSIVE): Non-blocking checkpoint at ~1k pages (~4MB).\n// Attempts checkpoint but allows concurrent readers/writers.\n//\n// 2. CheckpointInterval (PASSIVE): Time-based non-blocking checkpoint.\n// Ensures regular checkpointing even with low write volume.\n//\n// 3. TruncatePageN (TRUNCATE): Blocking checkpoint at ~121k pages (~500MB).\n// Emergency brake for runaway WAL growth. Can block writes while waiting\n// for long-lived read transactions. Configurable/disableable.\n//\n// The RESTART checkpoint mode was permanently removed due to production issues\n// with indefinite write blocking (issue #724). All checkpoints now use either\n// PASSIVE (non-blocking) or TRUNCATE (emergency only) modes.\ntype DB struct {\n\tmu sync.RWMutex\n\tpath string // part to database\n\tmetaPath string // Path to the database metadata.\n\tdb *sql.DB // target database\n\tf *os.File // long-running db file descriptor\n\trtx *sql.Tx // long running read transaction\n\tpageSize int // page size, in bytes\n\tnotify chan struct{} // closes on WAL change\n\tchkMu sync.RWMutex // checkpoint lock\n\topened bool // true if Open() was called and Close() not yet called\n\n\t// syncedSinceCheckpoint tracks whether any data has been synced since\n\t// the last checkpoint. Used to prevent time-based checkpoints from\n\t// triggering when there are no actual database changes, which would\n\t// otherwise create unnecessary LTX files. See issue #896.\n\tsyncedSinceCheckpoint bool\n\n\t// syncedToWALEnd tracks whether the last successful sync reached the\n\t// exact end of the WAL file. When true, a subsequent WAL truncation\n\t// (from checkpoint) is expected and should NOT trigger a full snapshot.\n\t// This prevents issue #927 where every checkpoint triggers unnecessary\n\t// full snapshots because verify() sees the old LTX position exceeds\n\t// the new (truncated) WAL size.\n\tsyncedToWALEnd bool\n\n\t// lastSyncedWALOffset tracks the logical end of the WAL content after\n\t// the last successful sync. This is the WALOffset + WALSize from the\n\t// last LTX file. Used for checkpoint threshold decisions instead of\n\t// file size, which may include stale frames with old salt values after\n\t// a checkpoint. This prevents issue #997 where PASSIVE checkpoints\n\t// trigger a feedback loop because stale file size exceeds threshold.\n\tlastSyncedWALOffset int64\n\n\t// last file info for each level\n\tmaxLTXFileInfos struct {\n\t\tsync.Mutex\n\t\tm map[int]*ltx.FileInfo\n\t}\n\n\t// Cached position from the latest L0 LTX file.\n\t// nil means cache is invalid; non-nil is the cached position.\n\tpos struct {\n\t\tsync.Mutex\n\t\tvalue *ltx.Pos\n\t}\n\n\tfileInfo os.FileInfo // db info cached during init\n\tdirInfo os.FileInfo // parent dir info cached during init\n\n\tctx context.Context\n\tcancel func()\n\twg sync.WaitGroup\n\tDone <-chan struct{}\n\n\t// Metrics\n\tdbSizeGauge prometheus.Gauge\n\twalSizeGauge prometheus.Gauge\n\ttotalWALBytesCounter prometheus.Counter\n\ttxIDGauge prometheus.Gauge\n\tsyncNCounter prometheus.Counter\n\tsyncErrorNCounter prometheus.Counter\n\tsyncSecondsCounter prometheus.Counter\n\tcheckpointNCounterVec *prometheus.CounterVec\n\tcheckpointErrorNCounterVec *prometheus.CounterVec\n\tcheckpointSecondsCounterVec *prometheus.CounterVec\n\n\t// Minimum threshold of WAL size, in pages, before a passive checkpoint.\n\t// A passive checkpoint will attempt a checkpoint but fail if there are\n\t// active transactions occurring at the same time.\n\t//\n\t// Uses PASSIVE checkpoint mode (non-blocking). Keeps WAL size manageable\n\t// for faster restores. Default: 1000 pages (~4MB with 4KB page size).\n\tMinCheckpointPageN int\n\n\t// Threshold of WAL size, in pages, before a forced truncation checkpoint.\n\t// A forced truncation checkpoint will block new transactions and wait for\n\t// existing transactions to finish before issuing a checkpoint and\n\t// truncating the WAL.\n\t//\n\t// Uses TRUNCATE checkpoint mode (blocking). Prevents unbounded WAL growth\n\t// from long-lived read transactions. Default: 121359 pages (~500MB with 4KB\n\t// page size). Set to 0 to disable forced truncation (use with caution as\n\t// WAL can grow unbounded if read transactions prevent checkpointing).\n\tTruncatePageN int\n\n\t// Time between automatic checkpoints in the WAL. This is done to allow\n\t// more fine-grained WAL files so that restores can be performed with\n\t// better precision.\n\t//\n\t// Uses PASSIVE checkpoint mode (non-blocking). Default: 1 minute.\n\t// Set to 0 to disable time-based checkpoints.\n\tCheckpointInterval time.Duration\n\n\t// Frequency at which to perform db sync.\n\tMonitorInterval time.Duration\n\n\t// The timeout to wait for EBUSY from SQLite.\n\tBusyTimeout time.Duration\n\n\t// Minimum time to retain L0 files after they have been compacted into L1.\n\tL0Retention time.Duration\n\n\t// VerifyCompaction enables post-compaction TXID consistency verification.\n\t// When enabled, verifies that files at the destination level have\n\t// contiguous TXID ranges after each compaction.\n\tVerifyCompaction bool\n\n\t// RetentionEnabled controls whether Litestream actively deletes old files\n\t// during retention enforcement. When false, cloud provider lifecycle\n\t// policies handle retention instead. Local file cleanup still occurs.\n\tRetentionEnabled bool\n\n\t// Remote replica for the database.\n\t// Must be set before calling Open().\n\tReplica *Replica\n\n\t// Compactor handles shared compaction logic.\n\t// Created in NewDB with nil client; client set once in Open() from Replica.Client.\n\tcompactor *Compactor\n\n\t// Shutdown sync retry settings.\n\t// ShutdownSyncTimeout is the total time to retry syncing on shutdown.\n\t// ShutdownSyncInterval is the time between retry attempts.\n\tShutdownSyncTimeout time.Duration\n\tShutdownSyncInterval time.Duration\n\n\t// lastSuccessfulSyncAt tracks when replication last succeeded.\n\t// Used by heartbeat monitoring to determine if a ping should be sent.\n\tlastSuccessfulSyncMu sync.RWMutex\n\tlastSuccessfulSyncAt time.Time\n\n\t// Where to send log messages, defaults to global slog with database epath.\n\tLogger *slog.Logger\n}\n\n// NewDB returns a new instance of DB for a given path.\nfunc NewDB(path string) *DB {\n\tdir, file := filepath.Split(path)\n\n\tdb := &DB{\n\t\tpath: path,\n\t\tmetaPath: filepath.Join(dir, \".\"+file+MetaDirSuffix),\n\t\tnotify: make(chan struct{}),\n\n\t\tMinCheckpointPageN: DefaultMinCheckpointPageN,\n\t\tTruncatePageN: DefaultTruncatePageN,\n\t\tCheckpointInterval: DefaultCheckpointInterval,\n\t\tMonitorInterval: DefaultMonitorInterval,\n\t\tBusyTimeout: DefaultBusyTimeout,\n\t\tL0Retention: DefaultL0Retention,\n\t\tRetentionEnabled: true,\n\t\tShutdownSyncTimeout: DefaultShutdownSyncTimeout,\n\t\tShutdownSyncInterval: DefaultShutdownSyncInterval,\n\t\tLogger: slog.With(LogKeyDB, filepath.Base(path)),\n\t}\n\tdb.maxLTXFileInfos.m = make(map[int]*ltx.FileInfo)\n\n\tdb.dbSizeGauge = dbSizeGaugeVec.WithLabelValues(db.path)\n\tdb.walSizeGauge = walSizeGaugeVec.WithLabelValues(db.path)\n\tdb.totalWALBytesCounter = totalWALBytesCounterVec.WithLabelValues(db.path)\n\tdb.txIDGauge = txIDIndexGaugeVec.WithLabelValues(db.path)\n\tdb.syncNCounter = syncNCounterVec.WithLabelValues(db.path)\n\tdb.syncErrorNCounter = syncErrorNCounterVec.WithLabelValues(db.path)\n\tdb.syncSecondsCounter = syncSecondsCounterVec.WithLabelValues(db.path)\n\tdb.checkpointNCounterVec = checkpointNCounterVec.MustCurryWith(prometheus.Labels{\"db\": db.path})\n\tdb.checkpointErrorNCounterVec = checkpointErrorNCounterVec.MustCurryWith(prometheus.Labels{\"db\": db.path})\n\tdb.checkpointSecondsCounterVec = checkpointSecondsCounterVec.MustCurryWith(prometheus.Labels{\"db\": db.path})\n\n\tdb.ctx, db.cancel = context.WithCancel(context.Background())\n\n\t// Initialize compactor with nil client (set once in Open() from Replica.Client).\n\tdb.compactor = NewCompactor(nil, db.Logger.With(LogKeySubsystem, LogSubsystemCompactor))\n\tdb.compactor.LocalFileOpener = db.openLocalLTXFile\n\tdb.compactor.LocalFileDeleter = db.deleteLocalLTXFile\n\tdb.compactor.CompactionVerifyErrorCounter = compactionVerifyErrorCounterVec.WithLabelValues(db.path)\n\tdb.compactor.CacheGetter = func(level int) (*ltx.FileInfo, bool) {\n\t\tdb.maxLTXFileInfos.Lock()\n\t\tdefer db.maxLTXFileInfos.Unlock()\n\t\tinfo, ok := db.maxLTXFileInfos.m[level]\n\t\treturn info, ok\n\t}\n\tdb.compactor.CacheSetter = func(level int, info *ltx.FileInfo) {\n\t\tdb.maxLTXFileInfos.Lock()\n\t\tdefer db.maxLTXFileInfos.Unlock()\n\t\tdb.maxLTXFileInfos.m[level] = info\n\t}\n\n\treturn db\n}\n\n// SetLogger updates the database logger and propagates to subsystems.\nfunc (db *DB) SetLogger(logger *slog.Logger) {\n\tif logger == nil {\n\t\tlogger = slog.Default()\n\t}\n\tdb.Logger = logger\n\tif db.compactor != nil {\n\t\tdb.compactor.setLogger(logger.With(LogKeySubsystem, LogSubsystemCompactor))\n\t}\n\tif db.Replica != nil && db.Replica.Client != nil {\n\t\tdb.Replica.Client.SetLogger(db.Replica.Logger())\n\t}\n}\n\n// SQLDB returns a reference to the underlying sql.DB connection.\nfunc (db *DB) SQLDB() *sql.DB {\n\treturn db.db\n}\n\n// Path returns the path to the database.\nfunc (db *DB) Path() string {\n\treturn db.path\n}\n\n// IsOpen returns true if the database has been opened.\nfunc (db *DB) IsOpen() bool {\n\tdb.mu.RLock()\n\tdefer db.mu.RUnlock()\n\treturn db.opened\n}\n\n// WALPath returns the path to the database's WAL file.\nfunc (db *DB) WALPath() string {\n\treturn db.path + \"-wal\"\n}\n\n// MetaPath returns the path to the database metadata.\nfunc (db *DB) MetaPath() string {\n\treturn db.metaPath\n}\n\n// SetMetaPath sets the path to database metadata.\nfunc (db *DB) SetMetaPath(path string) {\n\tdb.metaPath = path\n}\n\n// LTXDir returns path of the root LTX directory.\nfunc (db *DB) LTXDir() string {\n\treturn filepath.Join(db.metaPath, \"ltx\")\n}\n\n// ResetLocalState removes local LTX files, forcing a fresh snapshot on next sync.\n// This is useful for recovering from corrupted or missing LTX files.\n// The database file itself is not modified.\nfunc (db *DB) ResetLocalState(ctx context.Context) error {\n\tdb.Logger.Info(\"resetting local litestream state\",\n\t\t\"meta_path\", db.metaPath,\n\t\t\"ltx_dir\", db.LTXDir())\n\n\t// Remove all LTX files\n\tif err := os.RemoveAll(db.LTXDir()); err != nil && !os.IsNotExist(err) {\n\t\treturn fmt.Errorf(\"remove ltx directory: %w\", err)\n\t}\n\n\t// Clear cached LTX file info\n\tdb.maxLTXFileInfos.Lock()\n\tdb.maxLTXFileInfos.m = make(map[int]*ltx.FileInfo)\n\tdb.maxLTXFileInfos.Unlock()\n\n\tdb.invalidatePosCache()\n\n\tdb.Logger.Info(\"local state reset complete, next sync will create fresh snapshot\")\n\treturn nil\n}\n\n// LTXLevelDir returns path of the given LTX compaction level.\n// Panics if level is negative.\nfunc (db *DB) LTXLevelDir(level int) string {\n\treturn filepath.Join(db.LTXDir(), strconv.Itoa(level))\n}\n\n// LTXPath returns the local path of a single LTX file.\n// Panics if level or either txn ID is negative.\nfunc (db *DB) LTXPath(level int, minTXID, maxTXID ltx.TXID) string {\n\tassert(level >= 0, \"level cannot be negative\")\n\treturn filepath.Join(db.LTXLevelDir(level), ltx.FormatFilename(minTXID, maxTXID))\n}\n\n// openLocalLTXFile opens a local LTX file for reading.\n// Used by the Compactor to prefer local files over remote.\nfunc (db *DB) openLocalLTXFile(level int, minTXID, maxTXID ltx.TXID) (io.ReadCloser, error) {\n\treturn os.Open(db.LTXPath(level, minTXID, maxTXID))\n}\n\n// deleteLocalLTXFile deletes a local LTX file.\n// Used by the Compactor for retention enforcement.\nfunc (db *DB) deleteLocalLTXFile(level int, minTXID, maxTXID ltx.TXID) error {\n\tpath := db.LTXPath(level, minTXID, maxTXID)\n\tif err := os.Remove(path); err != nil && !os.IsNotExist(err) {\n\t\treturn err\n\t}\n\tif level == 0 {\n\t\tdb.invalidatePosCache()\n\t}\n\treturn nil\n}\n\n// MaxLTX returns the last LTX file written to level 0.\nfunc (db *DB) MaxLTX() (minTXID, maxTXID ltx.TXID, err error) {\n\tents, err := os.ReadDir(db.LTXLevelDir(0))\n\tif os.IsNotExist(err) {\n\t\treturn 0, 0, nil // no LTX files written\n\t} else if err != nil {\n\t\treturn 0, 0, err\n\t}\n\n\t// Find highest txn ID.\n\tfor _, ent := range ents {\n\t\tif min, max, err := ltx.ParseFilename(ent.Name()); err != nil {\n\t\t\tcontinue // invalid LTX filename\n\t\t} else if max > maxTXID {\n\t\t\tminTXID, maxTXID = min, max\n\t\t}\n\t}\n\treturn minTXID, maxTXID, nil\n}\n\n// FileInfo returns the cached file stats for the database file when it was initialized.\nfunc (db *DB) FileInfo() os.FileInfo {\n\treturn db.fileInfo\n}\n\n// DirInfo returns the cached file stats for the parent directory of the database file when it was initialized.\nfunc (db *DB) DirInfo() os.FileInfo {\n\treturn db.dirInfo\n}\n\n// Pos returns the current replication position of the database.\n// The result is cached and invalidated when L0 LTX files change.\nfunc (db *DB) Pos() (ltx.Pos, error) {\n\tdb.pos.Lock()\n\tdefer db.pos.Unlock()\n\n\tif db.pos.value != nil {\n\t\treturn *db.pos.value, nil\n\t}\n\n\tminTXID, maxTXID, err := db.MaxLTX()\n\tif err != nil {\n\t\treturn ltx.Pos{}, err\n\t} else if minTXID == 0 {\n\t\treturn ltx.Pos{}, nil // no replication yet\n\t}\n\n\tf, err := os.Open(db.LTXPath(0, minTXID, maxTXID))\n\tif err != nil {\n\t\treturn ltx.Pos{}, err\n\t}\n\tdefer func() { _ = f.Close() }()\n\n\tdec := ltx.NewDecoder(f)\n\tif err := dec.Verify(); err != nil {\n\t\treturn ltx.Pos{}, fmt.Errorf(\"ltx verification failed: %w\", err)\n\t}\n\n\tpos := dec.PostApplyPos()\n\tdb.pos.value = &pos\n\n\treturn pos, nil\n}\n\n// invalidatePosCache clears the cached position so the next call to Pos()\n// recomputes it from disk. Call this when L0 LTX files are deleted or\n// when the L0 directory is cleared.\nfunc (db *DB) invalidatePosCache() {\n\tdb.pos.Lock()\n\tdb.pos.value = nil\n\tdb.pos.Unlock()\n}\n\n// Notify returns a channel that closes when the shadow WAL changes.\nfunc (db *DB) Notify() <-chan struct{} {\n\tdb.mu.RLock()\n\tdefer db.mu.RUnlock()\n\treturn db.notify\n}\n\n// PageSize returns the page size of the underlying database.\n// Only valid after database exists & Init() has successfully run.\nfunc (db *DB) PageSize() int {\n\tdb.mu.RLock()\n\tdefer db.mu.RUnlock()\n\treturn db.pageSize\n}\n\n// RecordSuccessfulSync marks the current time as a successful sync.\n// Used by heartbeat monitoring to determine if a ping should be sent.\nfunc (db *DB) RecordSuccessfulSync() {\n\tdb.lastSuccessfulSyncMu.Lock()\n\tdefer db.lastSuccessfulSyncMu.Unlock()\n\tdb.lastSuccessfulSyncAt = time.Now()\n}\n\n// LastSuccessfulSyncAt returns the time of the last successful sync.\nfunc (db *DB) LastSuccessfulSyncAt() time.Time {\n\tdb.lastSuccessfulSyncMu.RLock()\n\tdefer db.lastSuccessfulSyncMu.RUnlock()\n\treturn db.lastSuccessfulSyncAt\n}\n\n// SyncStatus represents the current replication state of the database.\ntype SyncStatus struct {\n\tLocalTXID ltx.TXID\n\tRemoteTXID ltx.TXID\n\tInSync bool\n}\n\n// SyncStatus returns the current replication status of the database, comparing\n// the local transaction position against the remote replica position. The remote\n// position is queried from the replica storage, so this method may perform I/O.\nfunc (db *DB) SyncStatus(ctx context.Context) (SyncStatus, error) {\n\tif db.Replica == nil {\n\t\treturn SyncStatus{}, fmt.Errorf(\"no replica configured\")\n\t}\n\n\tlocalPos, err := db.Pos()\n\tif err != nil {\n\t\treturn SyncStatus{}, fmt.Errorf(\"local position: %w\", err)\n\t}\n\n\tremotePos, err := db.Replica.calcPos(ctx)\n\tif err != nil {\n\t\treturn SyncStatus{}, fmt.Errorf(\"remote position: %w\", err)\n\t}\n\n\treturn SyncStatus{\n\t\tLocalTXID: localPos.TXID,\n\t\tRemoteTXID: remotePos.TXID,\n\t\tInSync: localPos.TXID > 0 && localPos.TXID == remotePos.TXID,\n\t}, nil\n}\n\n// SyncAndWait performs a full sync: WAL to LTX files, then LTX files to remote\n// replica. Blocks until both stages complete.\nfunc (db *DB) SyncAndWait(ctx context.Context) error {\n\tif db.Replica == nil {\n\t\treturn fmt.Errorf(\"no replica configured\")\n\t}\n\n\tif err := db.Sync(ctx); err != nil {\n\t\treturn fmt.Errorf(\"db sync: %w\", err)\n\t}\n\tif err := db.Replica.Sync(ctx); err != nil {\n\t\treturn fmt.Errorf(\"replica sync: %w\", err)\n\t}\n\treturn nil\n}\n\n// EnsureExists restores the database from the configured replica if the local\n// database file does not exist. If no backup is available, it returns nil and\n// a fresh database will be created on Open(). Must be called before Open().\nfunc (db *DB) EnsureExists(ctx context.Context) error {\n\tif db.Replica == nil {\n\t\treturn fmt.Errorf(\"no replica configured\")\n\t}\n\tif db.Replica.Client == nil {\n\t\treturn fmt.Errorf(\"no replica client configured\")\n\t}\n\n\tif _, err := os.Stat(db.Path()); err == nil {\n\t\treturn nil\n\t} else if !os.IsNotExist(err) {\n\t\treturn fmt.Errorf(\"stat database: %w\", err)\n\t}\n\n\tif dir := filepath.Dir(db.Path()); dir != \".\" {\n\t\tif err := os.MkdirAll(dir, 0o750); err != nil {\n\t\t\treturn fmt.Errorf(\"create parent directory: %w\", err)\n\t\t}\n\t}\n\n\topt := NewRestoreOptions()\n\topt.OutputPath = db.Path()\n\topt.IntegrityCheck = IntegrityCheckQuick\n\n\tif err := db.Replica.Restore(ctx, opt); err != nil {\n\t\tif errors.Is(err, ErrTxNotAvailable) || errors.Is(err, ErrNoSnapshots) {\n\t\t\tdb.Logger.Debug(\"no backup found, will create fresh database\")\n\t\t\treturn nil\n\t\t}\n\t\treturn fmt.Errorf(\"restore from backup: %w\", err)\n\t}\n\n\tdb.Logger.Info(\"database restored from backup\", \"path\", db.Path())\n\treturn nil\n}\n\n// Open initializes the background monitoring goroutine.\nfunc (db *DB) Open() (err error) {\n\tdb.mu.Lock()\n\tif db.opened {\n\t\tdb.mu.Unlock()\n\t\treturn nil // already open\n\t}\n\t// Recreate context for fresh start (handles reopen after close)\n\tdb.ctx, db.cancel = context.WithCancel(context.Background())\n\tdb.mu.Unlock()\n\n\t// Validate fields on database.\n\tif db.Replica == nil {\n\t\treturn fmt.Errorf(\"replica required before opening database\")\n\t}\n\tif db.Replica.Client == nil {\n\t\treturn fmt.Errorf(\"replica client required before opening database\")\n\t}\n\tif db.MinCheckpointPageN <= 0 {\n\t\treturn fmt.Errorf(\"minimum checkpoint page count required\")\n\t}\n\n\t// Clear old temporary files that my have been left from a crash.\n\tif err := removeTmpFiles(db.metaPath); err != nil {\n\t\treturn fmt.Errorf(\"cannot remove tmp files: %w\", err)\n\t}\n\n\t// Set the compactor client once before starting any goroutines.\n\tdb.compactor.VerifyCompaction = db.VerifyCompaction\n\tdb.compactor.RetentionEnabled = db.RetentionEnabled\n\tdb.compactor.client = db.Replica.Client\n\n\t// Start monitoring SQLite database in a separate goroutine.\n\tif db.MonitorInterval > 0 {\n\t\tdb.wg.Add(1)\n\t\tgo func() { defer db.wg.Done(); db.monitor() }()\n\t}\n\n\t// Mark as opened only after successful initialization.\n\tdb.mu.Lock()\n\tdb.opened = true\n\tdb.mu.Unlock()\n\n\treturn nil\n}\n\n// Close flushes outstanding WAL writes to replicas, releases the read lock,\n// and closes the database. If Done is set, closing it interrupts the shutdown\n// sync retry loop and cancels any in-flight sync attempt.\nfunc (db *DB) Close(ctx context.Context) (err error) {\n\tdb.cancel()\n\tdb.wg.Wait()\n\n\t// Perform a final db sync, if initialized.\n\tif db.db != nil {\n\t\tif e := db.Sync(ctx); e != nil {\n\t\t\terr = e\n\t\t}\n\t}\n\n\t// Ensure replicas perform a final sync and stop replicating.\n\tif db.Replica != nil {\n\t\tif db.db != nil {\n\t\t\tif e := db.syncReplicaWithRetry(ctx); e != nil && err == nil {\n\t\t\t\terr = e\n\t\t\t}\n\t\t}\n\t\tdb.Replica.Stop(true)\n\t}\n\n\t// Release the read lock to allow other applications to handle checkpointing.\n\tif db.rtx != nil {\n\t\tif e := db.releaseReadLock(); e != nil && err == nil {\n\t\t\terr = e\n\t\t}\n\t}\n\n\tif db.db != nil {\n\t\tif e := db.db.Close(); e != nil && err == nil {\n\t\t\terr = e\n\t\t}\n\t\tdb.db = nil\n\t}\n\n\tif db.f != nil {\n\t\tif e := db.f.Close(); e != nil && err == nil {\n\t\t\terr = e\n\t\t}\n\t\tdb.f = nil\n\t}\n\n\tdb.mu.Lock()\n\tdb.opened = false\n\tdb.rtx = nil\n\tdb.mu.Unlock()\n\n\treturn err\n}\n\n// syncReplicaWithRetry attempts to sync the replica with retry logic for shutdown.\n// It retries until success, timeout, or context cancellation. If db.Done is non-nil,\n// closing it cancels any in-flight sync attempt and exits the retry loop.\n// If ShutdownSyncTimeout is 0, it performs a single sync attempt without retries.\nfunc (db *DB) syncReplicaWithRetry(ctx context.Context) error {\n\tif db.Replica == nil {\n\t\treturn nil\n\t}\n\n\ttimeout := db.ShutdownSyncTimeout\n\tinterval := db.ShutdownSyncInterval\n\n\t// If timeout is zero, just try once (no retry)\n\tif timeout == 0 {\n\t\treturn db.Replica.Sync(ctx)\n\t}\n\n\t// Use default interval if not set\n\tif interval == 0 {\n\t\tinterval = DefaultShutdownSyncInterval\n\t}\n\n\t// Create deadline context for total retry duration.\n\tdeadlineCtx, deadlineCancel := context.WithTimeout(ctx, timeout)\n\tdefer deadlineCancel()\n\n\t// If db.Done is set, derive a context that cancels when done is closed\n\t// so that in-flight Replica.Sync calls are interrupted immediately.\n\tsyncCtx := deadlineCtx\n\tif db.Done != nil {\n\t\tvar syncCancel context.CancelFunc\n\t\tsyncCtx, syncCancel = context.WithCancel(deadlineCtx)\n\t\tgo func() {\n\t\t\tselect {\n\t\t\tcase <-db.Done:\n\t\t\t\tsyncCancel()\n\t\t\tcase <-deadlineCtx.Done():\n\t\t\t\tsyncCancel()\n\t\t\t}\n\t\t}()\n\t}\n\n\tvar lastErr error\n\tattempt := 0\n\tstartTime := time.Now()\n\n\tfor {\n\t\t// Check if done is already closed before attempting sync\n\t\tif db.Done != nil {\n\t\t\tselect {\n\t\t\tcase <-db.Done:\n\t\t\t\tdb.Logger.Warn(\"shutdown sync skipped, interrupted by signal\",\n\t\t\t\t\t\"attempts\", attempt,\n\t\t\t\t\t\"duration\", time.Since(startTime))\n\t\t\t\treturn fmt.Errorf(\"after %d attempts: %w\", attempt, ErrShutdownInterrupted)\n\t\t\tdefault:\n\t\t\t}\n\t\t}\n\n\t\tattempt++\n\n\t\t// Try sync\n\t\tif err := db.Replica.Sync(syncCtx); err == nil {\n\t\t\tif attempt > 1 {\n\t\t\t\tdb.Logger.Info(\"shutdown sync succeeded after retry\",\n\t\t\t\t\t\"attempts\", attempt,\n\t\t\t\t\t\"duration\", time.Since(startTime))\n\t\t\t}\n\t\t\treturn nil\n\t\t} else {\n\t\t\tlastErr = err\n\t\t}\n\n\t\t// Check if we should stop retrying (done signal or timeout)\n\t\tselect {\n\t\tcase <-deadlineCtx.Done():\n\t\t\tdb.Logger.Error(\"shutdown sync failed after timeout\",\n\t\t\t\t\"attempts\", attempt,\n\t\t\t\t\"duration\", time.Since(startTime),\n\t\t\t\t\"error\", lastErr)\n\t\t\treturn fmt.Errorf(\"shutdown sync timeout after %d attempts: %w\", attempt, lastErr)\n\t\tcase <-db.Done:\n\t\t\tdb.Logger.Warn(\"shutdown sync interrupted by signal\",\n\t\t\t\t\"attempts\", attempt,\n\t\t\t\t\"duration\", time.Since(startTime),\n\t\t\t\t\"error\", lastErr)\n\t\t\treturn fmt.Errorf(\"after %d attempts: %w\", attempt, ErrShutdownInterrupted)\n\t\tdefault:\n\t\t}\n\n\t\t// Log retry with hint about second signal if interruptible\n\t\tif db.Done != nil {\n\t\t\tdb.Logger.Warn(\"shutdown sync failed, retrying (press Ctrl+C again to skip)\",\n\t\t\t\t\"attempts\", attempt,\n\t\t\t\t\"error\", lastErr,\n\t\t\t\t\"elapsed\", time.Since(startTime),\n\t\t\t\t\"remaining\", time.Until(startTime.Add(timeout)))\n\t\t} else {\n\t\t\tdb.Logger.Warn(\"shutdown sync failed, retrying\",\n\t\t\t\t\"attempts\", attempt,\n\t\t\t\t\"error\", lastErr,\n\t\t\t\t\"elapsed\", time.Since(startTime),\n\t\t\t\t\"remaining\", time.Until(startTime.Add(timeout)))\n\t\t}\n\n\t\t// Wait before retry, but also listen for done signal\n\t\tselect {\n\t\tcase <-time.After(interval):\n\t\tcase <-deadlineCtx.Done():\n\t\t\treturn fmt.Errorf(\"shutdown sync timeout after %d attempts: %w\", attempt, lastErr)\n\t\tcase <-db.Done:\n\t\t\tdb.Logger.Warn(\"shutdown sync interrupted by signal\",\n\t\t\t\t\"attempts\", attempt,\n\t\t\t\t\"duration\", time.Since(startTime))\n\t\t\treturn fmt.Errorf(\"after %d attempts: %w\", attempt, ErrShutdownInterrupted)\n\t\t}\n\t}\n}\n\n// setPersistWAL sets the PERSIST_WAL file control on the database connection.\n// This prevents SQLite from removing the WAL file when connections close.\nfunc (db *DB) setPersistWAL(ctx context.Context) error {\n\tconn, err := db.db.Conn(ctx)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"get connection: %w\", err)\n\t}\n\tdefer conn.Close()\n\n\treturn conn.Raw(func(driverConn interface{}) error {\n\t\tfc, ok := driverConn.(sqlite.FileControl)\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"driver does not implement FileControl\")\n\t\t}\n\n\t\t_, err := fc.FileControlPersistWAL(\"main\", 1)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"FileControlPersistWAL: %w\", err)\n\t\t}\n\n\t\treturn nil\n\t})\n}\n\n// init initializes the connection to the database.\n// Skipped if already initialized or if the database file does not exist.\nfunc (db *DB) init(ctx context.Context) (err error) {\n\t// Exit if already initialized.\n\tif db.db != nil {\n\t\treturn nil\n\t}\n\n\t// Exit if no database file exists.\n\tfi, err := os.Stat(db.path)\n\tif os.IsNotExist(err) {\n\t\treturn nil\n\t} else if err != nil {\n\t\treturn err\n\t}\n\tdb.fileInfo = fi\n\n\t// Obtain permissions for parent directory.\n\tif fi, err = os.Stat(filepath.Dir(db.path)); err != nil {\n\t\treturn err\n\t}\n\tdb.dirInfo = fi\n\n\tdsn := fmt.Sprintf(\"file:%s?_pragma=busy_timeout(%d)&_pragma=wal_autocheckpoint(0)\",\n\t\tdb.path, db.BusyTimeout.Milliseconds())\n\n\tif db.db, err = sql.Open(\"sqlite\", dsn); err != nil {\n\t\treturn err\n\t}\n\n\t// Set PERSIST_WAL to prevent WAL file removal when database connections close.\n\tif err := db.setPersistWAL(ctx); err != nil {\n\t\treturn fmt.Errorf(\"set PERSIST_WAL: %w\", err)\n\t}\n\n\t// Open long-running database file descriptor. Required for non-OFD locks.\n\tif db.f, err = os.Open(db.path); err != nil {\n\t\treturn fmt.Errorf(\"open db file descriptor: %w\", err)\n\t}\n\n\t// Ensure database is closed if init fails.\n\t// Initialization can retry on next sync.\n\tdefer func() {\n\t\tif err != nil {\n\t\t\t_ = db.releaseReadLock()\n\t\t\tdb.db.Close()\n\t\t\tdb.f.Close()\n\t\t\tdb.db, db.f = nil, nil\n\t\t}\n\t}()\n\n\t// Enable WAL and ensure it is set. New mode should be returned on success:\n\t// https://www.sqlite.org/pragma.html#pragma_journal_mode\n\tvar mode string\n\tif err := db.db.QueryRowContext(ctx, `PRAGMA journal_mode = wal;`).Scan(&mode); err != nil {\n\t\treturn err\n\t} else if mode != \"wal\" {\n\t\treturn fmt.Errorf(\"enable wal failed, mode=%q\", mode)\n\t}\n\n\t// Create a table to force writes to the WAL when empty.\n\t// There should only ever be one row with id=1.\n\tif _, err := db.db.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS _litestream_seq (id INTEGER PRIMARY KEY, seq INTEGER);`); err != nil {\n\t\treturn fmt.Errorf(\"create _litestream_seq table: %w\", err)\n\t}\n\n\t// Create a lock table to force write locks during sync.\n\t// The sync write transaction always rolls back so no data should be in this table.\n\tif _, err := db.db.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS _litestream_lock (id INTEGER);`); err != nil {\n\t\treturn fmt.Errorf(\"create _litestream_lock table: %w\", err)\n\t}\n\n\t// Start a long-running read transaction to prevent other transactions\n\t// from checkpointing.\n\tif err := db.acquireReadLock(ctx); err != nil {\n\t\treturn fmt.Errorf(\"acquire read lock: %w\", err)\n\t}\n\n\t// Read page size.\n\tif err := db.db.QueryRowContext(ctx, `PRAGMA page_size;`).Scan(&db.pageSize); err != nil {\n\t\treturn fmt.Errorf(\"read page size: %w\", err)\n\t} else if db.pageSize <= 0 {\n\t\treturn fmt.Errorf(\"invalid db page size: %d\", db.pageSize)\n\t}\n\n\t// Ensure meta directory structure exists.\n\tif err := internal.MkdirAll(db.metaPath, db.dirInfo); err != nil {\n\t\treturn err\n\t}\n\n\t// Ensure WAL has at least one frame in it.\n\tif err := db.ensureWALExists(ctx); err != nil {\n\t\treturn fmt.Errorf(\"ensure wal exists: %w\", err)\n\t}\n\n\t// Check if database is behind replica (issue #781).\n\t// This must happen before replica.Start() to detect restore scenarios.\n\tif db.Replica != nil {\n\t\tif err := db.checkDatabaseBehindReplica(ctx); err != nil {\n\t\t\treturn fmt.Errorf(\"check database behind replica: %w\", err)\n\t\t}\n\t}\n\n\t// If we have an existing replication files, ensure the headers match.\n\t// if err := db.verifyHeadersMatch(); err != nil {\n\t// \treturn fmt.Errorf(\"cannot determine last wal position: %w\", err)\n\t// }\n\n\t// TODO(gen): Generate diff of current LTX snapshot and save as next LTX file.\n\n\t// Start replication.\n\tif db.Replica != nil {\n\t\tdb.Replica.Start(db.ctx)\n\t}\n\n\treturn nil\n}\n\n/*\n// verifyHeadersMatch returns an error if\nfunc (db *DB) verifyHeadersMatch() error {\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"cannot determine position: %w\", err)\n\t} else if pos.TXID == 0 {\n\t\treturn true, nil // no replication performed yet\n\t}\n\n\thdr0, err := readWALHeader(db.WALPath())\n\tif os.IsNotExist(err) {\n\t\treturn false, fmt.Errorf(\"no wal: %w\", err)\n\t} else if err != nil {\n\t\treturn false, fmt.Errorf(\"read wal header: %w\", err)\n\t}\n\tsalt1 := binary.BigEndian.Uint32(hdr0[16:])\n\tsalt2 := binary.BigEndian.Uint32(hdr0[20:])\n\n\tltxPath := db.LTXPath(0, pos.TXID, pos.TXID)\n\tf, err := os.Open(ltxPath)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"open ltx path: %w\", err)\n\t}\n\tdefer func() { _ = f.Close() }()\n\n\tdec := ltx.NewDecoder(f)\n\tif err := dec.DecodeHeader(); err != nil {\n\t\treturn false, fmt.Errorf(\"decode ltx header: %w\", err)\n\t}\n\thdr1 := dec.Header()\n\tif salt1 != hdr1.WALSalt1 || salt2 != hdr1.WALSalt2 {\n\t\tdb.Logger.Log(internal.LevelTrace, \"salt mismatch\",\n\t\t\t\"path\", ltxPath,\n\t\t\t\"wal\", [2]uint32{salt1, salt2},\n\t\t\t\"ltx\", [2]uint32{hdr1.WALSalt1, hdr1.WALSalt2})\n\t\treturn false, nil\n\t}\n\treturn true, nil\n}\n*/\n\n// acquireReadLock begins a read transaction on the database to prevent checkpointing.\nfunc (db *DB) acquireReadLock(ctx context.Context) error {\n\tif db.rtx != nil {\n\t\treturn nil\n\t}\n\n\t// Start long running read-transaction to prevent checkpoints.\n\ttx, err := db.db.BeginTx(ctx, nil)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Execute read query to obtain read lock.\n\tif _, err := tx.ExecContext(ctx, `SELECT COUNT(1) FROM _litestream_seq;`); err != nil {\n\t\t_ = tx.Rollback()\n\t\treturn err\n\t}\n\n\t// Track transaction so we can release it later before checkpoint.\n\tdb.rtx = tx\n\treturn nil\n}\n\n// releaseReadLock rolls back the long-running read transaction.\nfunc (db *DB) releaseReadLock() error {\n\t// Ignore if we do not have a read lock.\n\tif db.rtx == nil {\n\t\treturn nil\n\t}\n\n\t// Rollback & clear read transaction.\n\t// Use rollback() helper to suppress \"already rolled back\" errors that can\n\t// occur during shutdown when concurrent checkpoint and close operations\n\t// both attempt to release the read lock. See issue #934.\n\terr := rollback(db.rtx)\n\tdb.rtx = nil\n\treturn err\n}\n\n// Sync copies pending data from the WAL to the shadow WAL.\nfunc (db *DB) Sync(ctx context.Context) (err error) {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\t// Track total sync metrics.\n\tt := time.Now()\n\tdefer func() {\n\t\tdb.syncNCounter.Inc()\n\t\tif err != nil {\n\t\t\tdb.syncErrorNCounter.Inc()\n\t\t}\n\t\tdb.syncSecondsCounter.Add(float64(time.Since(t).Seconds()))\n\t}()\n\n\t// Initialize database, if necessary. Exit if no DB exists.\n\tif err := db.init(ctx); err != nil {\n\t\treturn err\n\t} else if db.db == nil {\n\t\tdb.Logger.Debug(\"sync: no database found\")\n\t\treturn nil\n\t}\n\n\t// Ensure WAL has at least one frame in it.\n\tif err := db.ensureWALExists(ctx); err != nil {\n\t\treturn fmt.Errorf(\"ensure wal exists: %w\", err)\n\t}\n\n\torigWALSize, newWALSize, synced, err := db.verifyAndSync(ctx, false)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Track that data was synced for time-based checkpoint decisions.\n\tif synced {\n\t\tdb.syncedSinceCheckpoint = true\n\t}\n\n\tif err := db.checkpointIfNeeded(ctx, origWALSize, newWALSize); err != nil {\n\t\treturn fmt.Errorf(\"checkpoint: %w\", err)\n\t}\n\n\t// Compute current index and total shadow WAL size.\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"pos: %w\", err)\n\t}\n\tdb.txIDGauge.Set(float64(pos.TXID))\n\n\t// Update file size metrics.\n\tif fi, err := os.Stat(db.path); err == nil {\n\t\tdb.dbSizeGauge.Set(float64(fi.Size()))\n\t}\n\tdb.walSizeGauge.Set(float64(newWALSize))\n\n\t// Notify replicas of WAL changes.\n\t// if changed {\n\tclose(db.notify)\n\tdb.notify = make(chan struct{})\n\t// }\n\n\treturn nil\n}\n\nfunc (db *DB) verifyAndSync(ctx context.Context, checkpointing bool) (origWALSize, newWALSize int64, synced bool, err error) {\n\t// Use the last synced WAL offset as the logical size for checkpoint decisions.\n\t// This avoids using file size which may include stale frames with old salt\n\t// values after a checkpoint. See issue #997.\n\torigWALSize = db.lastSyncedWALOffset\n\tif origWALSize == 0 {\n\t\t// First sync - use file size as fallback\n\t\torigWALSize, err = db.walFileSize()\n\t\tif err != nil {\n\t\t\treturn 0, 0, false, fmt.Errorf(\"stat wal before sync: %w\", err)\n\t\t}\n\t}\n\n\t// Verify our last sync matches the current state of the WAL.\n\t// This ensures that the last sync position of the real WAL hasn't\n\t// been overwritten by another process.\n\tinfo, err := db.verify(ctx)\n\tif err != nil {\n\t\treturn 0, 0, false, fmt.Errorf(\"cannot verify wal state: %w\", err)\n\t}\n\n\tsynced, err = db.sync(ctx, checkpointing, info)\n\tif err != nil {\n\t\treturn 0, 0, false, fmt.Errorf(\"sync: %w\", err)\n\t}\n\n\t// Use the logical WAL offset (from LTX) for checkpoint decisions.\n\t// After sync, db.lastSyncedWALOffset is updated to reflect the actual\n\t// content position, not the file size.\n\tnewWALSize = db.lastSyncedWALOffset\n\n\treturn origWALSize, newWALSize, synced, nil\n}\n\n// checkpointIfNeeded performs a checkpoint based on configured thresholds.\n// Checks thresholds in priority order: TruncatePageN → MinCheckpointPageN → CheckpointInterval.\n//\n// TruncatePageN uses TRUNCATE mode (blocking), others use PASSIVE mode (non-blocking).\n//\n// Time-based checkpoints only trigger if db.syncedSinceCheckpoint is true, indicating\n// that data has been synced since the last checkpoint. This prevents creating unnecessary\n// LTX files when the only WAL data is from internal bookkeeping (like _litestream_seq\n// updates from previous checkpoints). See issue #896.\nfunc (db *DB) checkpointIfNeeded(ctx context.Context, origWALSize, newWALSize int64) error {\n\tif db.pageSize == 0 {\n\t\treturn nil\n\t}\n\n\t// Priority 1: Emergency truncate checkpoint (TRUNCATE mode, blocking)\n\t// This prevents unbounded WAL growth from long-lived read transactions.\n\tif db.TruncatePageN > 0 && origWALSize >= calcWALSize(uint32(db.pageSize), uint32(db.TruncatePageN)) {\n\t\tdb.Logger.Info(\"forcing truncate checkpoint\",\n\t\t\t\"wal_size\", origWALSize,\n\t\t\t\"threshold\", calcWALSize(uint32(db.pageSize), uint32(db.TruncatePageN)))\n\t\treturn db.checkpoint(ctx, CheckpointModeTruncate)\n\t}\n\n\t// Priority 2: Regular checkpoint at min threshold (PASSIVE mode, non-blocking)\n\tif newWALSize >= calcWALSize(uint32(db.pageSize), uint32(db.MinCheckpointPageN)) {\n\t\tif err := db.checkpoint(ctx, CheckpointModePassive); err != nil {\n\t\t\t// PASSIVE checkpoints can fail with SQLITE_BUSY when database is locked.\n\t\t\t// This is expected behavior and not an error - just log and continue.\n\t\t\tif isSQLiteBusyError(err) {\n\t\t\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"passive checkpoint skipped\", \"reason\", \"database busy\")\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\t}\n\n\t// Priority 3: Time-based checkpoint (PASSIVE mode, non-blocking)\n\t// Only trigger if there have been actual changes synced since the last\n\t// checkpoint. This prevents creating unnecessary LTX files when the only\n\t// WAL data is from internal bookkeeping (like _litestream_seq updates).\n\tif db.CheckpointInterval > 0 && db.syncedSinceCheckpoint {\n\t\t// Get database file modification time\n\t\tfi, err := db.f.Stat()\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"stat database: %w\", err)\n\t\t}\n\n\t\t// Only checkpoint if enough time has passed and WAL has data\n\t\tif time.Since(fi.ModTime()) > db.CheckpointInterval && newWALSize > calcWALSize(uint32(db.pageSize), 1) {\n\t\t\tif err := db.checkpoint(ctx, CheckpointModePassive); err != nil {\n\t\t\t\t// PASSIVE checkpoints can fail with SQLITE_BUSY when database is locked.\n\t\t\t\t// This is expected behavior and not an error - just log and continue.\n\t\t\t\tif isSQLiteBusyError(err) {\n\t\t\t\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"passive checkpoint skipped\", \"reason\", \"database busy\")\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// isSQLiteBusyError returns true if the error is an SQLITE_BUSY error.\nfunc isSQLiteBusyError(err error) bool {\n\tif err == nil {\n\t\treturn false\n\t}\n\t// Check for \"database is locked\" or \"SQLITE_BUSY\" in error message\n\terrStr := err.Error()\n\treturn strings.Contains(errStr, \"database is locked\") ||\n\t\tstrings.Contains(errStr, \"SQLITE_BUSY\")\n}\n\n// isDiskFullError returns true if the error indicates disk space issues.\n// This includes \"no space left on device\" (ENOSPC) and \"disk quota exceeded\" (EDQUOT).\nfunc isDiskFullError(err error) bool {\n\tif err == nil {\n\t\treturn false\n\t}\n\terrStr := strings.ToLower(err.Error())\n\treturn strings.Contains(errStr, \"no space left on device\") ||\n\t\tstrings.Contains(errStr, \"disk quota exceeded\") ||\n\t\tstrings.Contains(errStr, \"enospc\") ||\n\t\tstrings.Contains(errStr, \"edquot\")\n}\n\n// walFileSize returns the size of the WAL file in bytes.\nfunc (db *DB) walFileSize() (int64, error) {\n\tfi, err := os.Stat(db.WALPath())\n\tif os.IsNotExist(err) {\n\t\treturn 0, nil\n\t} else if err != nil {\n\t\treturn 0, err\n\t}\n\treturn fi.Size(), nil\n}\n\n// calcWALSize returns the size of the WAL for a given page size & count.\n// Casts to int64 before multiplication to prevent uint32 overflow with large page sizes.\nfunc calcWALSize(pageSize uint32, pageN uint32) int64 {\n\treturn int64(WALHeaderSize) + (int64(WALFrameHeaderSize+pageSize) * int64(pageN))\n}\n\n// ensureWALExists checks that the real WAL exists and has a header.\nfunc (db *DB) ensureWALExists(ctx context.Context) (err error) {\n\t// Exit early if WAL header exists.\n\tif fi, err := os.Stat(db.WALPath()); err == nil && fi.Size() >= WALHeaderSize {\n\t\treturn nil\n\t}\n\n\t// Otherwise create transaction that updates the internal litestream table.\n\t_, err = db.db.ExecContext(ctx, `INSERT INTO _litestream_seq (id, seq) VALUES (1, 1) ON CONFLICT (id) DO UPDATE SET seq = seq + 1`)\n\treturn err\n}\n\n// checkDatabaseBehindReplica detects when a database has been restored to an\n// earlier state and the replica has a higher TXID. This handles issue #781.\n//\n// If detected, it clears local L0 files and fetches the latest L0 LTX file\n// from the replica to establish a baseline. The next DB.sync() will detect\n// the mismatch and trigger a snapshot at the current database state.\nfunc (db *DB) checkDatabaseBehindReplica(ctx context.Context) error {\n\t// Get database position from local L0 files\n\tdbPos, err := db.Pos()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"get database position: %w\", err)\n\t}\n\n\t// Get replica position from remote\n\treplicaInfo, err := db.Replica.MaxLTXFileInfo(ctx, 0)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"get replica position: %w\", err)\n\t} else if replicaInfo.MaxTXID == 0 {\n\t\treturn nil // No remote replica data yet\n\t}\n\n\t// Check if database is behind replica\n\tif dbPos.TXID >= replicaInfo.MaxTXID {\n\t\treturn nil // Database is ahead or equal\n\t}\n\n\tdb.Logger.Info(\"detected database behind replica\",\n\t\t\"db_txid\", dbPos.TXID,\n\t\t\"replica_txid\", replicaInfo.MaxTXID)\n\n\t// Clear local L0 files\n\tl0Dir := db.LTXLevelDir(0)\n\tif err := os.RemoveAll(l0Dir); err != nil && !os.IsNotExist(err) {\n\t\treturn fmt.Errorf(\"remove L0 directory: %w\", err)\n\t}\n\tdb.invalidatePosCache()\n\tif err := internal.MkdirAll(l0Dir, db.dirInfo); err != nil {\n\t\treturn fmt.Errorf(\"recreate L0 directory: %w\", err)\n\t}\n\n\t// Fetch latest L0 LTX file from replica\n\tminTXID, maxTXID := replicaInfo.MinTXID, replicaInfo.MaxTXID\n\treader, err := db.Replica.Client.OpenLTXFile(ctx, 0, minTXID, maxTXID, 0, 0)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open remote L0 file: %w\", err)\n\t}\n\tdefer func() { _ = reader.Close() }()\n\n\t// Write to temp file and atomically rename\n\tlocalPath := db.LTXPath(0, minTXID, maxTXID)\n\ttmpPath := localPath + \".tmp\"\n\n\ttmpFile, err := os.Create(tmpPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create temp L0 file: %w\", err)\n\t}\n\tdefer func() { _ = os.Remove(tmpPath) }() // Clean up temp file on error\n\n\tif _, err := io.Copy(tmpFile, reader); err != nil {\n\t\t_ = tmpFile.Close()\n\t\treturn fmt.Errorf(\"copy L0 file: %w\", err)\n\t}\n\n\tif err := tmpFile.Sync(); err != nil {\n\t\t_ = tmpFile.Close()\n\t\treturn fmt.Errorf(\"sync L0 file: %w\", err)\n\t}\n\n\tif err := tmpFile.Close(); err != nil {\n\t\treturn fmt.Errorf(\"close L0 file: %w\", err)\n\t}\n\n\t// Atomically rename temp file to final path\n\tif err := os.Rename(tmpPath, localPath); err != nil {\n\t\treturn fmt.Errorf(\"rename L0 file: %w\", err)\n\t}\n\tdb.invalidatePosCache()\n\n\tdb.Logger.Info(\"fetched latest L0 file from replica\",\n\t\t\"min_txid\", minTXID,\n\t\t\"max_txid\", maxTXID)\n\n\treturn nil\n}\n\n// verify ensures the current LTX state matches where it left off from\n// the real WAL. Check info.ok if verification was successful.\nfunc (db *DB) verify(ctx context.Context) (info syncInfo, err error) {\n\tframeSize := int64(db.pageSize + WALFrameHeaderSize)\n\tinfo.snapshotting = true\n\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\treturn info, fmt.Errorf(\"pos: %w\", err)\n\t} else if pos.TXID == 0 {\n\t\tinfo.offset = WALHeaderSize\n\t\treturn info, nil // first sync\n\t}\n\n\t// Determine last WAL offset we save from.\n\tltxPath := db.LTXPath(0, pos.TXID, pos.TXID)\n\tltxFile, err := os.Open(ltxPath)\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\treturn info, NewLTXError(\"open\", ltxPath, 0, uint64(pos.TXID), uint64(pos.TXID), err)\n\t\t}\n\t\treturn info, fmt.Errorf(\"open ltx file %s: %w\", ltxPath, err)\n\t}\n\tdefer func() { _ = ltxFile.Close() }()\n\n\tdec := ltx.NewDecoder(ltxFile)\n\tif err := dec.DecodeHeader(); err != nil {\n\t\t// Decode failure indicates corruption\n\t\tltxErr := NewLTXError(\"decode\", ltxPath, 0, uint64(pos.TXID), uint64(pos.TXID), fmt.Errorf(\"%w: %w\", ErrLTXCorrupted, err))\n\t\treturn info, ltxErr\n\t}\n\tinfo.offset = dec.Header().WALOffset + dec.Header().WALSize\n\tinfo.salt1 = dec.Header().WALSalt1\n\tinfo.salt2 = dec.Header().WALSalt2\n\n\t// If LTX WAL offset is larger than real WAL then the WAL has been truncated.\n\tif fi, err := os.Stat(db.WALPath()); err != nil {\n\t\treturn info, fmt.Errorf(\"open wal file: %w\", err)\n\t} else if info.offset > fi.Size() {\n\t\t// If we previously synced to the exact end of the WAL, this truncation\n\t\t// is expected (normal checkpoint behavior). Reset position and continue\n\t\t// incrementally rather than triggering a full snapshot. See issue #927.\n\t\tif db.syncedToWALEnd {\n\t\t\tdb.syncedToWALEnd = false // Clear flag\n\n\t\t\t// Read new WAL header to get current salt values\n\t\t\thdr, err := readWALHeader(db.WALPath())\n\t\t\tif err != nil {\n\t\t\t\treturn info, fmt.Errorf(\"read wal header after expected truncation: %w\", err)\n\t\t\t}\n\n\t\t\tinfo.offset = WALHeaderSize\n\t\t\tinfo.salt1 = binary.BigEndian.Uint32(hdr[16:])\n\t\t\tinfo.salt2 = binary.BigEndian.Uint32(hdr[20:])\n\t\t\tinfo.snapshotting = false\n\t\t\tinfo.reason = \"\"\n\n\t\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"wal truncated after sync to end (expected checkpoint)\",\n\t\t\t\t\"new_salt1\", info.salt1,\n\t\t\t\t\"new_salt2\", info.salt2)\n\n\t\t\treturn info, nil\n\t\t}\n\n\t\tinfo.reason = \"wal truncated by another process\"\n\t\treturn info, nil\n\t}\n\n\t// Compare WAL headers. Restart from beginning of WAL if different.\n\thdr0, err := readWALHeader(db.WALPath())\n\tif err != nil {\n\t\treturn info, fmt.Errorf(\"cannot read wal header: %w\", err)\n\t}\n\tsalt1 := binary.BigEndian.Uint32(hdr0[16:])\n\tsalt2 := binary.BigEndian.Uint32(hdr0[20:])\n\tsaltMatch := salt1 == dec.Header().WALSalt1 && salt2 == dec.Header().WALSalt2\n\n\t// Handle edge case where we're at WAL header (WALOffset=32, WALSize=0).\n\t// This can happen when an LTX file represents a state at the beginning of the WAL\n\t// with no frames written yet. We must check this before computing prevWALOffset\n\t// to avoid underflow (32 - 4120 = -4088).\n\t// See: https://github.com/benbjohnson/litestream/issues/900\n\tif info.offset == WALHeaderSize {\n\t\tdb.Logger.Debug(\"verify\", \"saltMatch\", saltMatch, \"atWALHeader\", true)\n\t\tif saltMatch {\n\t\t\tinfo.snapshotting = false\n\t\t\treturn info, nil\n\t\t}\n\t\tinfo.reason = \"wal header salt reset, snapshotting\"\n\t\treturn info, nil\n\t}\n\n\t// If offset is at the beginning of the first page, we can't check for previous page.\n\tprevWALOffset := info.offset - frameSize\n\tdb.Logger.Debug(\"verify\", \"saltMatch\", saltMatch, \"prevWALOffset\", prevWALOffset)\n\n\tif prevWALOffset == WALHeaderSize {\n\t\tif saltMatch { // No writes occurred since last sync, salt still matches\n\t\t\tinfo.snapshotting = false\n\t\t\treturn info, nil\n\t\t}\n\t\t// Salt has changed but we don't know if writes occurred since last sync\n\t\tinfo.reason = \"wal header salt reset, snapshotting\"\n\t\treturn info, nil\n\t} else if prevWALOffset < WALHeaderSize {\n\t\treturn info, fmt.Errorf(\"prev WAL offset is less than the header size: %d\", prevWALOffset)\n\t}\n\n\t// If we can't verify the last page is in the last LTX file, then we need to snapshot.\n\tlastPageMatch, err := db.lastPageMatch(ctx, dec, prevWALOffset, frameSize)\n\tif err != nil {\n\t\treturn info, fmt.Errorf(\"last page match: %w\", err)\n\t} else if !lastPageMatch {\n\t\tinfo.reason = \"last page does not exist in last ltx file, wal overwritten by another process\"\n\t\treturn info, nil\n\t}\n\n\tdb.Logger.Debug(\"verify.2\", \"lastPageMatch\", lastPageMatch)\n\n\t// Salt has changed which could indicate a FULL checkpoint.\n\t// If we have a last page match, then we can assume that the WAL has not been overwritten.\n\tif !saltMatch {\n\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"wal restarted\",\n\t\t\t\"salt1\", salt1,\n\t\t\t\"salt2\", salt2)\n\n\t\tinfo.offset = WALHeaderSize\n\t\tinfo.salt1, info.salt2 = salt1, salt2\n\n\t\tif detected, err := db.detectFullCheckpoint(ctx, [][2]uint32{{salt1, salt2}, {dec.Header().WALSalt1, dec.Header().WALSalt2}}); err != nil {\n\t\t\treturn info, fmt.Errorf(\"detect full checkpoint: %w\", err)\n\t\t} else if detected {\n\t\t\tinfo.reason = \"full or restart checkpoint detected, snapshotting\"\n\t\t} else {\n\t\t\tinfo.snapshotting = false\n\t\t}\n\n\t\treturn info, nil\n\t}\n\n\tinfo.snapshotting = false\n\n\treturn info, nil\n}\n\n// lastPageMatch checks if the last page read in the WAL exists in the last LTX file.\nfunc (db *DB) lastPageMatch(ctx context.Context, dec *ltx.Decoder, prevWALOffset, frameSize int64) (bool, error) {\n\tif prevWALOffset <= WALHeaderSize {\n\t\treturn false, nil\n\t}\n\n\tframe, err := readWALFileAt(db.WALPath(), prevWALOffset, frameSize)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"cannot read last synced wal page: %w\", err)\n\t}\n\tpgno := binary.BigEndian.Uint32(frame[0:])\n\tfsalt1 := binary.BigEndian.Uint32(frame[8:])\n\tfsalt2 := binary.BigEndian.Uint32(frame[12:])\n\tdata := frame[WALFrameHeaderSize:]\n\n\tif fsalt1 != dec.Header().WALSalt1 || fsalt2 != dec.Header().WALSalt2 {\n\t\treturn false, nil\n\t}\n\n\t// Verify that the last page in the WAL exists in the last LTX file.\n\tbuf := make([]byte, dec.Header().PageSize)\n\tfor {\n\t\tvar hdr ltx.PageHeader\n\t\tif err := dec.DecodePage(&hdr, buf); errors.Is(err, io.EOF) {\n\t\t\treturn false, nil // page not found in LTX file\n\t\t} else if err != nil {\n\t\t\treturn false, fmt.Errorf(\"decode ltx page: %w\", err)\n\t\t}\n\n\t\tif pgno != hdr.Pgno {\n\t\t\tcontinue // page number doesn't match\n\t\t}\n\t\tif !bytes.Equal(data, buf) {\n\t\t\tcontinue // page data doesn't match\n\t\t}\n\t\treturn true, nil // Page matches\n\t}\n}\n\n// detectFullCheckpoint attempts to detect checks if a FULL or RESTART checkpoint\n// has occurred and we may have missed some frames.\nfunc (db *DB) detectFullCheckpoint(ctx context.Context, knownSalts [][2]uint32) (bool, error) {\n\twalFile, err := os.Open(db.WALPath())\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"open wal file: %w\", err)\n\t}\n\tdefer walFile.Close()\n\n\tvar lastKnownSalt [2]uint32\n\tif len(knownSalts) > 0 {\n\t\tlastKnownSalt = knownSalts[len(knownSalts)-1]\n\t}\n\n\trd, err := NewWALReader(walFile, db.Logger.With(LogKeySubsystem, LogSubsystemWALReader))\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"new wal reader: %w\", err)\n\t}\n\tm, err := rd.FrameSaltsUntil(ctx, lastKnownSalt)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"frame salts until: %w\", err)\n\t}\n\n\t// Remove known salts from the map.\n\tfor _, salt := range knownSalts {\n\t\tdelete(m, salt)\n\t}\n\n\t// If we have more than one unknown salt, then we have a FULL or RESTART checkpoint.\n\treturn len(m) >= 1, nil\n}\n\ntype syncInfo struct {\n\toffset int64 // end of the previous LTX read\n\tsalt1 uint32\n\tsalt2 uint32\n\tsnapshotting bool // if true, a full snapshot is required\n\treason string // reason for snapshot\n}\n\n// sync copies pending bytes from the real WAL to LTX.\n// Returns synced=true if an LTX file was created (i.e., there were new pages to sync).\nfunc (db *DB) sync(ctx context.Context, checkpointing bool, info syncInfo) (synced bool, err error) {\n\t// Determine the next sequential transaction ID.\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"pos: %w\", err)\n\t}\n\ttxID := pos.TXID + 1\n\n\tfilename := db.LTXPath(0, txID, txID)\n\n\tlogArgs := []any{\n\t\t\"txid\", txID.String(),\n\t\t\"offset\", info.offset,\n\t}\n\tif checkpointing {\n\t\tlogArgs = append(logArgs, \"chkpt\", \"true\")\n\t}\n\tif info.snapshotting {\n\t\tlogArgs = append(logArgs, \"snap\", \"true\")\n\t}\n\tif info.reason != \"\" {\n\t\tlogArgs = append(logArgs, \"reason\", info.reason)\n\t}\n\tdb.Logger.Debug(\"sync\", logArgs...)\n\n\t// Prevent internal checkpoints during sync. Ignore if already in a checkpoint.\n\tif !checkpointing {\n\t\tdb.chkMu.RLock()\n\t\tdefer db.chkMu.RUnlock()\n\t}\n\n\tfi, err := db.f.Stat()\n\tif err != nil {\n\t\treturn false, err\n\t}\n\tmode := fi.Mode()\n\tcommit := uint32(fi.Size() / int64(db.pageSize))\n\n\twalFile, err := os.Open(db.WALPath())\n\tif err != nil {\n\t\treturn false, err\n\t}\n\tdefer walFile.Close()\n\n\twalReaderLogger := db.Logger.With(LogKeySubsystem, LogSubsystemWALReader)\n\tvar rd *WALReader\n\tif info.offset == WALHeaderSize {\n\t\tif rd, err = NewWALReader(walFile, walReaderLogger); err != nil {\n\t\t\treturn false, fmt.Errorf(\"new wal reader: %w\", err)\n\t\t}\n\t} else {\n\t\t// If we cannot verify the previous frame\n\t\tvar pfmError *PrevFrameMismatchError\n\t\tif rd, err = NewWALReaderWithOffset(ctx, walFile, info.offset, info.salt1, info.salt2, walReaderLogger); errors.As(err, &pfmError) {\n\t\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"prev frame mismatch, snapshotting\", \"err\", pfmError.Err)\n\t\t\tinfo.offset = WALHeaderSize\n\t\t\tif rd, err = NewWALReader(walFile, walReaderLogger); err != nil {\n\t\t\t\treturn false, fmt.Errorf(\"new wal reader, after reset\")\n\t\t\t}\n\t\t} else if err != nil {\n\t\t\treturn false, fmt.Errorf(\"new wal reader with offset: %w\", err)\n\t\t}\n\t}\n\n\t// Build a mapping of changed page numbers and their latest content.\n\tpageMap, maxOffset, walCommit, err := rd.PageMap(ctx)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"page map: %w\", err)\n\t}\n\tif walCommit > 0 {\n\t\tcommit = walCommit\n\t}\n\n\tvar sz int64\n\tif maxOffset > 0 {\n\t\tsz = maxOffset - info.offset\n\t}\n\tassert(sz >= 0, fmt.Sprintf(\"wal size must be positive: sz=%d, maxOffset=%d, info.offset=%d\", sz, maxOffset, info.offset))\n\n\t// Track total WAL bytes synced.\n\tif sz > 0 {\n\t\tdb.totalWALBytesCounter.Add(float64(sz))\n\t}\n\n\t// Exit if we have no new WAL pages and we aren't snapshotting.\n\tif !info.snapshotting && sz == 0 {\n\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"sync: skip\", \"reason\", \"no new wal pages\")\n\t\treturn false, nil\n\t}\n\n\ttmpFilename := filename + \".tmp\"\n\tif err := internal.MkdirAll(filepath.Dir(tmpFilename), db.dirInfo); err != nil {\n\t\treturn false, err\n\t}\n\n\tltxFile, err := os.OpenFile(tmpFilename, os.O_RDWR|os.O_CREATE|os.O_TRUNC, mode)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"open temp ltx file: %w\", err)\n\t}\n\tdefer func() { _ = os.Remove(tmpFilename) }()\n\tdefer func() { _ = ltxFile.Close() }()\n\n\tuid, gid := internal.Fileinfo(db.fileInfo)\n\t_ = os.Chown(tmpFilename, uid, gid)\n\n\tdb.Logger.Log(ctx, internal.LevelTrace, \"encode header\",\n\t\t\"txid\", txID.String(),\n\t\t\"commit\", commit,\n\t\t\"walOffset\", info.offset,\n\t\t\"walSize\", sz,\n\t\t\"salt1\", rd.salt1,\n\t\t\"salt2\", rd.salt2)\n\n\ttimestamp := time.Now()\n\tenc, err := ltx.NewEncoder(ltxFile)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"new ltx encoder: %w\", err)\n\t}\n\tif err := enc.EncodeHeader(ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\tPageSize: uint32(db.pageSize),\n\t\tCommit: commit,\n\t\tMinTXID: txID,\n\t\tMaxTXID: txID,\n\t\tTimestamp: timestamp.UnixMilli(),\n\t\tWALOffset: info.offset,\n\t\tWALSize: sz,\n\t\tWALSalt1: rd.salt1,\n\t\tWALSalt2: rd.salt2,\n\t}); err != nil {\n\t\treturn false, fmt.Errorf(\"encode ltx header: %w\", err)\n\t}\n\n\t// If we need a full snapshot, then copy from the database & WAL.\n\t// Otherwise, just copy incrementally from the WAL.\n\tif info.snapshotting {\n\t\tif err := db.writeLTXFromDB(ctx, enc, walFile, commit, pageMap); err != nil {\n\t\t\treturn false, fmt.Errorf(\"write ltx from db: %w\", err)\n\t\t}\n\t} else {\n\t\tif err := db.writeLTXFromWAL(ctx, enc, walFile, pageMap); err != nil {\n\t\t\treturn false, fmt.Errorf(\"write ltx from wal: %w\", err)\n\t\t}\n\t}\n\n\t// Encode final trailer to the end of the LTX file.\n\tif err := enc.Close(); err != nil {\n\t\treturn false, fmt.Errorf(\"close ltx encoder: %w\", err)\n\t}\n\n\t// Sync & close LTX file.\n\tif err := ltxFile.Sync(); err != nil {\n\t\treturn false, fmt.Errorf(\"sync ltx file: %w\", err)\n\t}\n\tif err := ltxFile.Close(); err != nil {\n\t\treturn false, fmt.Errorf(\"close ltx file: %w\", err)\n\t}\n\n\t// Atomically rename file to final path.\n\tif err := os.Rename(tmpFilename, filename); err != nil {\n\t\tdb.maxLTXFileInfos.Lock()\n\t\tdelete(db.maxLTXFileInfos.m, 0) // clear cache if in unknown state\n\t\tdb.maxLTXFileInfos.Unlock()\n\t\tdb.invalidatePosCache()\n\t\treturn false, fmt.Errorf(\"rename ltx file: %w\", err)\n\t}\n\n\t// Update file info cache for L0.\n\tdb.maxLTXFileInfos.Lock()\n\tdb.maxLTXFileInfos.m[0] = <x.FileInfo{\n\t\tLevel: 0,\n\t\tMinTXID: txID,\n\t\tMaxTXID: txID,\n\t\tCreatedAt: time.Now(),\n\t\tSize: enc.N(),\n\t}\n\tdb.maxLTXFileInfos.Unlock()\n\n\t// Update cached position from the encoder.\n\tencPos := enc.PostApplyPos()\n\tdb.pos.Lock()\n\tdb.pos.value = &encPos\n\tdb.pos.Unlock()\n\n\t// Track the logical end of WAL content for checkpoint decisions.\n\t// This is the WALOffset + WALSize from the LTX we just created.\n\t// Using this instead of file size prevents issue #997 where stale\n\t// frames with old salt values cause perpetual checkpoint triggering.\n\tfinalOffset := info.offset + sz\n\tdb.lastSyncedWALOffset = finalOffset\n\n\t// Track if we synced to the exact end of the WAL file.\n\t// This is used by verify() to distinguish expected checkpoint truncation\n\t// from unexpected external WAL modifications. See issue #927.\n\tif walSize, err := db.walFileSize(); err == nil {\n\t\tdb.syncedToWALEnd = finalOffset == walSize\n\t} else {\n\t\tdb.syncedToWALEnd = false\n\t}\n\n\tdb.Logger.Debug(\"db sync\", \"status\", \"ok\")\n\n\treturn true, nil\n}\n\nfunc (db *DB) writeLTXFromDB(ctx context.Context, enc *ltx.Encoder, walFile *os.File, commit uint32, pageMap map[uint32]int64) error {\n\tlockPgno := ltx.LockPgno(uint32(db.pageSize))\n\tdata := make([]byte, db.pageSize)\n\n\tfor pgno := uint32(1); pgno <= commit; pgno++ {\n\t\tif pgno == lockPgno {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Check if the caller has canceled during processing.\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn context.Cause(ctx)\n\t\tdefault:\n\t\t}\n\n\t\t// If page exists in the WAL, read from there.\n\t\tif offset, ok := pageMap[pgno]; ok {\n\t\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"encode page from wal\", \"txid\", enc.Header().MinTXID, \"offset\", offset, \"pgno\", pgno, \"type\", \"db+wal\")\n\n\t\t\tif n, err := walFile.ReadAt(data, offset+WALFrameHeaderSize); err != nil {\n\t\t\t\treturn fmt.Errorf(\"read page %d @ %d: %w\", pgno, offset, err)\n\t\t\t} else if n != len(data) {\n\t\t\t\treturn fmt.Errorf(\"short read page %d @ %d\", pgno, offset)\n\t\t\t}\n\n\t\t\tif err := enc.EncodePage(ltx.PageHeader{Pgno: pgno}, data); err != nil {\n\t\t\t\treturn fmt.Errorf(\"encode ltx frame (pgno=%d): %w\", pgno, err)\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\toffset := int64(pgno-1) * int64(db.pageSize)\n\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"encode page from database\", \"offset\", offset, \"pgno\", pgno)\n\n\t\t// Otherwise read directly from the database file.\n\t\tif _, err := db.f.ReadAt(data, offset); err != nil {\n\t\t\treturn fmt.Errorf(\"read database page %d: %w\", pgno, err)\n\t\t}\n\t\tif err := enc.EncodePage(ltx.PageHeader{Pgno: pgno}, data); err != nil {\n\t\t\treturn fmt.Errorf(\"encode ltx frame (pgno=%d): %w\", pgno, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (db *DB) writeLTXFromWAL(ctx context.Context, enc *ltx.Encoder, walFile *os.File, pageMap map[uint32]int64) error {\n\t// Create an ordered list of page numbers since the LTX encoder requires it.\n\tpgnos := make([]uint32, 0, len(pageMap))\n\tfor pgno := range pageMap {\n\t\tpgnos = append(pgnos, pgno)\n\t}\n\tslices.Sort(pgnos)\n\n\tdata := make([]byte, db.pageSize)\n\tfor _, pgno := range pgnos {\n\t\toffset := pageMap[pgno]\n\n\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"encode page from wal\", \"txid\", enc.Header().MinTXID, \"offset\", offset, \"pgno\", pgno, \"type\", \"walonly\")\n\n\t\t// Read source page using page map.\n\t\tif n, err := walFile.ReadAt(data, offset+WALFrameHeaderSize); err != nil {\n\t\t\treturn fmt.Errorf(\"read page %d @ %d: %w\", pgno, offset, err)\n\t\t} else if n != len(data) {\n\t\t\treturn fmt.Errorf(\"short read page %d @ %d\", pgno, offset)\n\t\t}\n\n\t\t// Write page to LTX encoder.\n\t\tif err := enc.EncodePage(ltx.PageHeader{Pgno: pgno}, data); err != nil {\n\t\t\treturn fmt.Errorf(\"encode ltx frame (pgno=%d): %w\", pgno, err)\n\t\t}\n\t}\n\treturn nil\n}\n\n// Checkpoint performs a checkpoint on the WAL file.\nfunc (db *DB) Checkpoint(ctx context.Context, mode string) (err error) {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\treturn db.checkpoint(ctx, mode)\n}\n\n// checkpoint performs a checkpoint on the WAL file and initializes a\n// new shadow WAL file.\nfunc (db *DB) checkpoint(ctx context.Context, mode string) error {\n\t// Try getting a checkpoint lock, will fail during snapshots.\n\tif !db.chkMu.TryLock() {\n\t\treturn nil\n\t}\n\tdefer db.chkMu.Unlock()\n\n\t// Read WAL header before checkpoint to check if it has been restarted.\n\thdr, err := readWALHeader(db.WALPath())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Copy end of WAL before checkpoint to copy as much as possible.\n\tif _, _, _, err := db.verifyAndSync(ctx, true); err != nil {\n\t\treturn fmt.Errorf(\"cannot copy wal before checkpoint: %w\", err)\n\t}\n\n\t// Execute checkpoint and immediately issue a write to the WAL to ensure\n\t// a new page is written.\n\tif err := db.execCheckpoint(ctx, mode); err != nil {\n\t\treturn err\n\t} else if _, err = db.db.ExecContext(ctx, `INSERT INTO _litestream_seq (id, seq) VALUES (1, 1) ON CONFLICT (id) DO UPDATE SET seq = seq + 1`); err != nil {\n\t\treturn err\n\t}\n\n\t// If WAL hasn't been restarted, exit.\n\tif other, err := readWALHeader(db.WALPath()); err != nil {\n\t\treturn err\n\t} else if bytes.Equal(hdr, other) {\n\t\tdb.syncedSinceCheckpoint = false\n\t\treturn nil\n\t}\n\n\t// Start a transaction. This will be promoted immediately after.\n\ttx, err := db.db.BeginTx(ctx, nil)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"begin: %w\", err)\n\t}\n\tdefer func() { _ = rollback(tx) }()\n\n\t// Insert into the lock table to promote to a write tx. The lock table\n\t// insert will never actually occur because our tx will be rolled back,\n\t// however, it will ensure our tx grabs the write lock. Unfortunately,\n\t// we can't call \"BEGIN IMMEDIATE\" as we are already in a transaction.\n\tif _, err := tx.ExecContext(ctx, `INSERT INTO _litestream_lock (id) VALUES (1);`); err != nil {\n\t\treturn fmt.Errorf(\"_litestream_lock: %w\", err)\n\t}\n\n\t// Copy anything that may have occurred after the checkpoint.\n\tif _, _, _, err := db.verifyAndSync(ctx, true); err != nil {\n\t\treturn fmt.Errorf(\"cannot copy wal after checkpoint: %w\", err)\n\t}\n\n\t// Release write lock before exiting.\n\t// Use rollback() helper for consistency with releaseReadLock() and the\n\t// defer above. See issue #934.\n\tif err := rollback(tx); err != nil {\n\t\treturn fmt.Errorf(\"rollback post-checkpoint tx: %w\", err)\n\t}\n\n\tdb.syncedSinceCheckpoint = false\n\treturn nil\n}\n\nfunc (db *DB) execCheckpoint(ctx context.Context, mode string) (err error) {\n\t// Ignore if there is no underlying database.\n\tif db.db == nil {\n\t\treturn nil\n\t}\n\n\t// Track checkpoint metrics.\n\tt := time.Now()\n\tdefer func() {\n\t\tlabels := prometheus.Labels{\"mode\": mode}\n\t\tdb.checkpointNCounterVec.With(labels).Inc()\n\t\tif err != nil {\n\t\t\tdb.checkpointErrorNCounterVec.With(labels).Inc()\n\t\t}\n\t\tdb.checkpointSecondsCounterVec.With(labels).Add(float64(time.Since(t).Seconds()))\n\t}()\n\n\t// Ensure the read lock has been removed before issuing a checkpoint.\n\t// We defer the re-acquire to ensure it occurs even on an early return.\n\tif err := db.releaseReadLock(); err != nil {\n\t\treturn fmt.Errorf(\"release read lock: %w\", err)\n\t}\n\tdefer func() { _ = db.acquireReadLock(ctx) }()\n\n\t// A non-forced checkpoint is issued as \"PASSIVE\". This will only checkpoint\n\t// if there are not pending transactions. A forced checkpoint (\"RESTART\")\n\t// will wait for pending transactions to end & block new transactions before\n\t// forcing the checkpoint and restarting the WAL.\n\t//\n\t// See: https://www.sqlite.org/pragma.html#pragma_wal_checkpoint\n\trawsql := `PRAGMA wal_checkpoint(` + mode + `);`\n\n\tvar row [3]int\n\tif err := db.db.QueryRowContext(ctx, rawsql).Scan(&row[0], &row[1], &row[2]); err != nil {\n\t\treturn err\n\t}\n\tdb.Logger.Debug(\"checkpoint\", \"mode\", mode, \"result\", fmt.Sprintf(\"%d,%d,%d\", row[0], row[1], row[2]))\n\n\t// Reacquire the read lock immediately after the checkpoint.\n\tif err := db.acquireReadLock(ctx); err != nil {\n\t\treturn fmt.Errorf(\"reacquire read lock: %w\", err)\n\t}\n\n\treturn nil\n}\n\n// SnapshotReader returns the current position of the database & a reader that contains a full database snapshot.\nfunc (db *DB) SnapshotReader(ctx context.Context) (ltx.Pos, io.Reader, error) {\n\tif db.PageSize() == 0 {\n\t\tdb.Logger.Debug(\"page size not initialized yet\", \"pageSize\", 0)\n\t\treturn ltx.Pos{}, nil, &DBNotReadyError{Reason: \"page size not initialized\"}\n\t}\n\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\treturn pos, nil, fmt.Errorf(\"pos: %w\", err)\n\t}\n\n\tdb.Logger.Debug(\"snapshot\", \"txid\", pos.TXID.String())\n\n\t// Prevent internal checkpoints during sync.\n\tdb.chkMu.RLock()\n\tdefer db.chkMu.RUnlock()\n\n\t// TODO(ltx): Read database size from database header.\n\n\tfi, err := db.f.Stat()\n\tif err != nil {\n\t\treturn pos, nil, err\n\t}\n\tcommit := uint32(fi.Size() / int64(db.pageSize))\n\n\t// Execute encoding in a separate goroutine so the caller can initialize before reading.\n\tpr, pw := io.Pipe()\n\tgo func() {\n\t\twalFile, err := os.Open(db.WALPath())\n\t\tif err != nil {\n\t\t\tpw.CloseWithError(err)\n\t\t\treturn\n\t\t}\n\t\tdefer walFile.Close()\n\n\t\trd, err := NewWALReader(walFile, db.Logger.With(LogKeySubsystem, LogSubsystemWALReader))\n\t\tif err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"new wal reader: %w\", err))\n\t\t\treturn\n\t\t}\n\n\t\t// Build a mapping of changed page numbers and their latest content.\n\t\tpageMap, maxOffset, walCommit, err := rd.PageMap(ctx)\n\t\tif err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"page map: %w\", err))\n\t\t\treturn\n\t\t}\n\t\tif walCommit > 0 {\n\t\t\tcommit = walCommit\n\t\t}\n\n\t\tvar sz int64\n\t\tif maxOffset > 0 {\n\t\t\tsz = maxOffset - rd.Offset()\n\t\t}\n\n\t\tdb.Logger.Debug(\"encode snapshot header\",\n\t\t\t\"txid\", pos.TXID.String(),\n\t\t\t\"commit\", commit,\n\t\t\t\"walOffset\", rd.Offset(),\n\t\t\t\"walSize\", sz,\n\t\t\t\"salt1\", rd.salt1,\n\t\t\t\"salt2\", rd.salt2)\n\n\t\tenc, err := ltx.NewEncoder(pw)\n\t\tif err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"new ltx encoder: %w\", err))\n\t\t\treturn\n\t\t}\n\t\tif err := enc.EncodeHeader(ltx.Header{\n\t\t\tVersion: ltx.Version,\n\t\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\t\tPageSize: uint32(db.pageSize),\n\t\t\tCommit: commit,\n\t\t\tMinTXID: 1,\n\t\t\tMaxTXID: pos.TXID,\n\t\t\tTimestamp: time.Now().UnixMilli(),\n\t\t\tWALOffset: rd.Offset(),\n\t\t\tWALSize: sz,\n\t\t\tWALSalt1: rd.salt1,\n\t\t\tWALSalt2: rd.salt2,\n\t\t}); err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"encode ltx snapshot header: %w\", err))\n\t\t\treturn\n\t\t}\n\n\t\tif err := db.writeLTXFromDB(ctx, enc, walFile, commit, pageMap); err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"write snapshot ltx: %w\", err))\n\t\t\treturn\n\t\t}\n\n\t\tif err := enc.Close(); err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"close ltx snapshot encoder: %w\", err))\n\t\t\treturn\n\t\t}\n\t\t_ = pw.Close()\n\t}()\n\n\treturn pos, pr, nil\n}\n\n// Compact performs a compaction of the LTX file at the previous level into dstLevel.\n// Returns metadata for the newly written compaction file. Returns ErrNoCompaction\n// if no new files are available to be compacted.\nfunc (db *DB) Compact(ctx context.Context, dstLevel int) (*ltx.FileInfo, error) {\n\tinfo, err := db.compactor.Compact(ctx, dstLevel)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// If this is L1, clean up L0 files using the time-based retention policy.\n\tif dstLevel == 1 {\n\t\tif err := db.EnforceL0RetentionByTime(ctx); err != nil {\n\t\t\t// Don't log context cancellation errors during shutdown\n\t\t\tif !errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded) {\n\t\t\t\tdb.Logger.Error(\"enforce L0 time retention\", \"error\", err)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn info, nil\n}\n\n// SnapshotDB writes a snapshot to the replica for the current position of the database.\nfunc (db *DB) Snapshot(ctx context.Context) (*ltx.FileInfo, error) {\n\tpos, r, err := db.SnapshotReader(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tinfo, err := db.Replica.Client.WriteLTXFile(ctx, SnapshotLevel, 1, pos.TXID, r)\n\tif err != nil {\n\t\treturn info, err\n\t}\n\n\tdb.maxLTXFileInfos.Lock()\n\tdb.maxLTXFileInfos.m[SnapshotLevel] = info\n\tdb.maxLTXFileInfos.Unlock()\n\n\treturn info, nil\n}\n\n// EnforceSnapshotRetention enforces retention of the snapshot level in the database by timestamp.\nfunc (db *DB) EnforceSnapshotRetention(ctx context.Context, timestamp time.Time) (minSnapshotTXID ltx.TXID, err error) {\n\tdb.Logger.Debug(\"enforcing snapshot retention\", \"timestamp\", timestamp)\n\n\t// Normal operation - use fast timestamps\n\titr, err := db.Replica.Client.LTXFiles(ctx, SnapshotLevel, 0, false)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"fetch ltx files: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar deleted []*ltx.FileInfo\n\tvar lastInfo *ltx.FileInfo\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tlastInfo = info\n\n\t\t// If this snapshot is before the retention timestamp, mark it for deletion.\n\t\tif info.CreatedAt.Before(timestamp) {\n\t\t\tdeleted = append(deleted, info)\n\t\t\tcontinue\n\t\t}\n\n\t\t// Track the lowest snapshot TXID so we can enforce retention in lower levels.\n\t\t// This is only tracked for snapshots not marked for deletion.\n\t\tif minSnapshotTXID == 0 || info.MaxTXID < minSnapshotTXID {\n\t\t\tminSnapshotTXID = info.MaxTXID\n\t\t}\n\t}\n\n\t// If this is the snapshot level, we need to ensure that at least one snapshot exists.\n\tif len(deleted) > 0 && deleted[len(deleted)-1] == lastInfo {\n\t\tdeleted = deleted[:len(deleted)-1]\n\t}\n\n\t// Remove files marked for deletion from remote storage (unless retention disabled).\n\tif !db.RetentionEnabled {\n\t\tdb.Logger.Debug(\"skipping remote deletion (retention disabled)\", \"level\", SnapshotLevel, \"count\", len(deleted))\n\t} else if err := db.Replica.Client.DeleteLTXFiles(ctx, deleted); err != nil {\n\t\treturn 0, fmt.Errorf(\"remove ltx files: %w\", err)\n\t}\n\n\t// Always clean up local files.\n\tfor _, info := range deleted {\n\t\tlocalPath := db.LTXPath(SnapshotLevel, info.MinTXID, info.MaxTXID)\n\t\tdb.Logger.Debug(\"deleting local ltx file\", \"level\", SnapshotLevel, \"minTXID\", info.MinTXID, \"maxTXID\", info.MaxTXID, \"path\", localPath)\n\n\t\tif err := os.Remove(localPath); err != nil && !os.IsNotExist(err) {\n\t\t\tdb.Logger.Error(\"failed to remove local ltx file\", \"path\", localPath, \"error\", err)\n\t\t}\n\t}\n\n\treturn minSnapshotTXID, nil\n}\n\n// EnforceL0RetentionByTime retains L0 files until they have been compacted into\n// L1 and have existed for at least L0Retention.\nfunc (db *DB) EnforceL0RetentionByTime(ctx context.Context) error {\n\tif db.L0Retention <= 0 {\n\t\treturn nil\n\t}\n\n\tdb.Logger.Debug(\"starting l0 retention enforcement\", \"retention\", db.L0Retention)\n\n\tdbName := filepath.Base(db.Path())\n\n\t// Determine the highest TXID that has been compacted into L1.\n\titr, err := db.Replica.Client.LTXFiles(ctx, 1, 0, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"fetch l1 files: %w\", err)\n\t}\n\tvar maxL1TXID ltx.TXID\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tif info.MaxTXID > maxL1TXID {\n\t\t\tmaxL1TXID = info.MaxTXID\n\t\t}\n\t}\n\tif err := itr.Close(); err != nil {\n\t\treturn fmt.Errorf(\"close l1 iterator: %w\", err)\n\t}\n\tif maxL1TXID == 0 {\n\t\tinternal.L0RetentionGaugeVec.WithLabelValues(dbName, \"eligible\").Set(0)\n\t\tinternal.L0RetentionGaugeVec.WithLabelValues(dbName, \"not_compacted\").Set(0)\n\t\tinternal.L0RetentionGaugeVec.WithLabelValues(dbName, \"too_recent\").Set(0)\n\t\treturn nil\n\t}\n\n\tthreshold := time.Now().Add(-db.L0Retention)\n\titr, err = db.Replica.Client.LTXFiles(ctx, 0, 0, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"fetch l0 files: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar (\n\t\tdeleted []*ltx.FileInfo\n\t\tlastInfo *ltx.FileInfo\n\t\tprocessedAll = true\n\t\ttotalFiles int\n\t\tnotCompactedCount int\n\t\ttooRecentCount int\n\t)\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tlastInfo = info\n\t\ttotalFiles++\n\n\t\tcreatedAt := info.CreatedAt\n\t\tif createdAt.IsZero() {\n\t\t\tif fi, err := os.Stat(db.LTXPath(0, info.MinTXID, info.MaxTXID)); err == nil {\n\t\t\t\tcreatedAt = fi.ModTime().UTC()\n\t\t\t} else {\n\t\t\t\tcreatedAt = threshold\n\t\t\t}\n\t\t}\n\n\t\tif createdAt.After(threshold) {\n\t\t\t// L0 entries are ordered; once we reach a newer file we stop so we don't\n\t\t\t// create gaps between retained files. VFS expects contiguous coverage.\n\t\t\tprocessedAll = false\n\t\t\ttooRecentCount++\n\t\t\tbreak\n\t\t}\n\n\t\tif info.MaxTXID <= maxL1TXID {\n\t\t\tdeleted = append(deleted, info)\n\t\t} else {\n\t\t\tnotCompactedCount++\n\t\t}\n\t}\n\n\t// Count remaining files as too_recent if we stopped early\n\tif !processedAll {\n\t\tfor itr.Next() {\n\t\t\ttooRecentCount++\n\t\t}\n\t}\n\n\t// Ensure we do not delete the newest L0 file only if we processed the entire level.\n\tif processedAll && len(deleted) > 0 && lastInfo != nil && deleted[len(deleted)-1] == lastInfo {\n\t\tdeleted = deleted[:len(deleted)-1]\n\t}\n\n\tinternal.L0RetentionGaugeVec.WithLabelValues(dbName, \"eligible\").Set(float64(len(deleted)))\n\tinternal.L0RetentionGaugeVec.WithLabelValues(dbName, \"not_compacted\").Set(float64(notCompactedCount))\n\tinternal.L0RetentionGaugeVec.WithLabelValues(dbName, \"too_recent\").Set(float64(tooRecentCount))\n\n\tdb.Logger.Debug(\"l0 retention scan complete\",\n\t\t\"total_l0_files\", totalFiles,\n\t\t\"eligible_for_deletion\", len(deleted),\n\t\t\"not_compacted_yet\", notCompactedCount,\n\t\t\"too_recent\", tooRecentCount,\n\t\t\"max_l1_txid\", maxL1TXID)\n\n\tif len(deleted) == 0 {\n\t\treturn nil\n\t}\n\n\tif !db.RetentionEnabled {\n\t\tdb.Logger.Debug(\"skipping remote deletion (retention disabled)\", \"level\", 0, \"count\", len(deleted))\n\t} else if err := db.Replica.Client.DeleteLTXFiles(ctx, deleted); err != nil {\n\t\treturn fmt.Errorf(\"remove expired l0 files: %w\", err)\n\t}\n\n\tfor _, info := range deleted {\n\t\tlocalPath := db.LTXPath(0, info.MinTXID, info.MaxTXID)\n\t\tdb.Logger.Debug(\"deleting expired local l0 file\", \"minTXID\", info.MinTXID, \"maxTXID\", info.MaxTXID, \"path\", localPath)\n\t\tif err := os.Remove(localPath); err != nil && !os.IsNotExist(err) {\n\t\t\tdb.Logger.Error(\"failed to remove local l0 file\", \"path\", localPath, \"error\", err)\n\t\t}\n\t}\n\tif len(deleted) > 0 {\n\t\tdb.invalidatePosCache()\n\t}\n\n\tdb.Logger.Info(\"l0 retention enforced\", \"deleted_count\", len(deleted), \"max_l1_txid\", maxL1TXID)\n\n\treturn nil\n}\n\n// EnforceRetentionByTXID enforces retention so that any LTX files below\n// the target TXID are deleted. Always keep at least one file.\nfunc (db *DB) EnforceRetentionByTXID(ctx context.Context, level int, txID ltx.TXID) (err error) {\n\treturn db.compactor.EnforceRetentionByTXID(ctx, level, txID)\n}\n\n// monitor runs in a separate goroutine and monitors the database & WAL.\n//\n// Implements exponential backoff on repeated sync errors to prevent disk churn\n// when persistent errors (like disk full) occur. See issue #927.\nfunc (db *DB) monitor() {\n\tticker := time.NewTicker(db.MonitorInterval)\n\tdefer ticker.Stop()\n\n\t// Backoff state for error handling.\n\tvar backoff time.Duration\n\tvar lastLogTime time.Time\n\tvar consecutiveErrs int\n\n\tfor {\n\t\t// Wait for ticker or context close.\n\t\tselect {\n\t\tcase <-db.ctx.Done():\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t}\n\n\t\t// If in backoff mode, wait additional time before retrying.\n\t\tif backoff > 0 {\n\t\t\tselect {\n\t\t\tcase <-db.ctx.Done():\n\t\t\t\treturn\n\t\t\tcase <-time.After(backoff):\n\t\t\t}\n\t\t}\n\n\t\t// Sync the database to the shadow WAL.\n\t\tif err := db.Sync(db.ctx); err != nil && !errors.Is(err, context.Canceled) {\n\t\t\tconsecutiveErrs++\n\n\t\t\t// Exponential backoff: MonitorInterval -> 2x -> 4x -> ... -> max\n\t\t\tif backoff == 0 {\n\t\t\t\tbackoff = db.MonitorInterval\n\t\t\t} else {\n\t\t\t\tbackoff *= 2\n\t\t\t\tif backoff > DefaultSyncBackoffMax {\n\t\t\t\t\tbackoff = DefaultSyncBackoffMax\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Log with rate limiting to avoid log spam during persistent errors.\n\t\t\tif time.Since(lastLogTime) >= SyncErrorLogInterval {\n\t\t\t\tdb.Logger.Error(\"sync error\",\n\t\t\t\t\t\"error\", err,\n\t\t\t\t\t\"consecutive_errors\", consecutiveErrs,\n\t\t\t\t\t\"backoff\", backoff)\n\t\t\t\tlastLogTime = time.Now()\n\t\t\t}\n\n\t\t\t// Try to clean up stale temp files after persistent disk errors.\n\t\t\tif isDiskFullError(err) && consecutiveErrs >= 3 {\n\t\t\t\tdb.Logger.Warn(\"attempting temp file cleanup due to persistent disk errors\")\n\t\t\t\tif cleanupErr := removeTmpFiles(db.metaPath); cleanupErr != nil {\n\t\t\t\t\tdb.Logger.Error(\"temp file cleanup failed\", \"error\", cleanupErr)\n\t\t\t\t}\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\t// Success - reset backoff and error counter.\n\t\tif consecutiveErrs > 0 {\n\t\t\tdb.Logger.Info(\"sync recovered\", \"previous_errors\", consecutiveErrs)\n\t\t}\n\t\tbackoff = 0\n\t\tconsecutiveErrs = 0\n\t}\n}\n\n// CRC64 returns a CRC-64 ISO checksum of the database and its current position.\n//\n// This function obtains a read lock so it prevents syncs from occurring until\n// the operation is complete. The database will still be usable but it will be\n// unable to checkpoint during this time.\n//\n// If dst is set, the database file is copied to that location before checksum.\nfunc (db *DB) CRC64(ctx context.Context) (uint64, ltx.Pos, error) {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tif err := db.init(ctx); err != nil {\n\t\treturn 0, ltx.Pos{}, err\n\t} else if db.db == nil {\n\t\treturn 0, ltx.Pos{}, os.ErrNotExist\n\t}\n\n\t// Force a RESTART checkpoint to ensure the database is at the start of the WAL.\n\tif err := db.checkpoint(ctx, CheckpointModeRestart); err != nil {\n\t\treturn 0, ltx.Pos{}, err\n\t}\n\n\t// Obtain current position. Clear the offset since we are only reading the\n\t// DB and not applying the current WAL.\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\treturn 0, pos, err\n\t}\n\n\t// Seek to the beginning of the db file descriptor and checksum whole file.\n\th := crc64.New(crc64.MakeTable(crc64.ISO))\n\tif _, err := db.f.Seek(0, io.SeekStart); err != nil {\n\t\treturn 0, pos, err\n\t} else if _, err := io.Copy(h, db.f); err != nil {\n\t\treturn 0, pos, err\n\t}\n\treturn h.Sum64(), pos, nil\n}\n\n// MaxLTXFileInfo returns the metadata for the last LTX file in a level.\n// If cached, it will returned the local copy. Otherwise, it fetches from the replica.\nfunc (db *DB) MaxLTXFileInfo(ctx context.Context, level int) (ltx.FileInfo, error) {\n\tdb.maxLTXFileInfos.Lock()\n\tdefer db.maxLTXFileInfos.Unlock()\n\n\tinfo, ok := db.maxLTXFileInfos.m[level]\n\tif ok {\n\t\treturn *info, nil\n\t}\n\n\tremoteInfo, err := db.Replica.MaxLTXFileInfo(ctx, level)\n\tif err != nil {\n\t\treturn ltx.FileInfo{}, fmt.Errorf(\"cannot determine L%d max ltx file for %q: %w\", level, db.Path(), err)\n\t}\n\n\tdb.maxLTXFileInfos.m[level] = &remoteInfo\n\treturn remoteInfo, nil\n}\n\n// DefaultRestoreParallelism is the default parallelism when downloading WAL files.\nconst DefaultRestoreParallelism = 8\n\n// DefaultFollowInterval is the default polling interval for follow mode.\nconst DefaultFollowInterval = 1 * time.Second\n\n// IntegrityCheckMode specifies the level of integrity checking after restore.\ntype IntegrityCheckMode int\n\nconst (\n\tIntegrityCheckNone IntegrityCheckMode = iota\n\tIntegrityCheckQuick\n\tIntegrityCheckFull\n)\n\n// RestoreOptions represents options for DB.Restore().\ntype RestoreOptions struct {\n\t// Target path to restore into.\n\t// If blank, the original DB path is used.\n\tOutputPath string\n\n\t// Specific transaction to restore to.\n\t// If zero, TXID is ignored.\n\tTXID ltx.TXID\n\n\t// Point-in-time to restore database.\n\t// If zero, database restore to most recent state available.\n\tTimestamp time.Time\n\n\t// Specifies how many WAL files are downloaded in parallel during restore.\n\tParallelism int\n\n\t// Follow enables continuous restore mode, polling for new LTX files\n\t// and applying them to the restored database. Similar to tail -f.\n\tFollow bool\n\n\t// FollowInterval specifies how often to poll for new LTX files in follow mode.\n\tFollowInterval time.Duration\n\n\t// IntegrityCheck specifies the level of integrity checking after restore.\n\t// Zero value (IntegrityCheckNone) skips the check for backward compatibility.\n\tIntegrityCheck IntegrityCheckMode\n}\n\n// NewRestoreOptions returns a new instance of RestoreOptions with defaults.\nfunc NewRestoreOptions() RestoreOptions {\n\treturn RestoreOptions{\n\t\tParallelism: DefaultRestoreParallelism,\n\t\tFollowInterval: DefaultFollowInterval,\n\t}\n}\n\n// Database metrics.\nvar (\n\tdbSizeGaugeVec = promauto.NewGaugeVec(prometheus.GaugeOpts{\n\t\tName: \"litestream_db_size\",\n\t\tHelp: \"The current size of the real DB\",\n\t}, []string{\"db\"})\n\n\twalSizeGaugeVec = promauto.NewGaugeVec(prometheus.GaugeOpts{\n\t\tName: \"litestream_wal_size\",\n\t\tHelp: \"The current size of the real WAL\",\n\t}, []string{\"db\"})\n\n\ttotalWALBytesCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_total_wal_bytes\",\n\t\tHelp: \"Total number of bytes written to shadow WAL\",\n\t}, []string{\"db\"})\n\n\ttxIDIndexGaugeVec = promauto.NewGaugeVec(prometheus.GaugeOpts{\n\t\tName: \"litestream_txid\",\n\t\tHelp: \"The current transaction ID\",\n\t}, []string{\"db\"})\n\n\tsyncNCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_sync_count\",\n\t\tHelp: \"Number of sync operations performed\",\n\t}, []string{\"db\"})\n\n\tsyncErrorNCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_sync_error_count\",\n\t\tHelp: \"Number of sync errors that have occurred\",\n\t}, []string{\"db\"})\n\n\tsyncSecondsCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_sync_seconds\",\n\t\tHelp: \"Time spent syncing shadow WAL, in seconds\",\n\t}, []string{\"db\"})\n\n\tcheckpointNCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_checkpoint_count\",\n\t\tHelp: \"Number of checkpoint operations performed\",\n\t}, []string{\"db\", \"mode\"})\n\n\tcheckpointErrorNCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_checkpoint_error_count\",\n\t\tHelp: \"Number of checkpoint errors that have occurred\",\n\t}, []string{\"db\", \"mode\"})\n\n\tcheckpointSecondsCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_checkpoint_seconds\",\n\t\tHelp: \"Time spent checkpointing WAL, in seconds\",\n\t}, []string{\"db\", \"mode\"})\n\n\tcompactionVerifyErrorCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_compaction_verify_error_count\",\n\t\tHelp: \"Number of post-compaction verification failures\",\n\t}, []string{\"db\"})\n)\n" }, { "path": "db_internal_test.go", "content": "package litestream\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"database/sql\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/prometheus/client_golang/prometheus/testutil\"\n\t\"github.com/superfly/ltx\"\n\t_ \"modernc.org/sqlite\"\n\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\n// testReplicaClient is a minimal mock for testing that doesn't cause import cycles.\ntype testReplicaClient struct {\n\tdir string\n}\n\nfunc (c *testReplicaClient) Init(_ context.Context) error { return nil }\n\nfunc (c *testReplicaClient) SetLogger(_ *slog.Logger) {}\n\nfunc (c *testReplicaClient) Type() string { return \"test\" }\n\nfunc (c *testReplicaClient) LTXFiles(_ context.Context, level int, afterTXID ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\tinternal.OperationTotalCounterVec.WithLabelValues(c.Type(), \"LIST\").Inc()\n\n\tlevelDir := filepath.Join(c.dir, fmt.Sprintf(\"l%d\", level))\n\tentries, err := os.ReadDir(levelDir)\n\tif os.IsNotExist(err) {\n\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t} else if err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar infos []*ltx.FileInfo\n\tfor _, entry := range entries {\n\t\tminTXID, maxTXID, err := ltx.ParseFilename(entry.Name())\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t}\n\t\tif maxTXID <= afterTXID {\n\t\t\tcontinue\n\t\t}\n\t\tfi, _ := entry.Info()\n\t\tvar size int64\n\t\tif fi != nil {\n\t\t\tsize = fi.Size()\n\t\t}\n\t\tinfos = append(infos, <x.FileInfo{\n\t\t\tLevel: level,\n\t\t\tMinTXID: minTXID,\n\t\t\tMaxTXID: maxTXID,\n\t\t\tSize: size,\n\t\t})\n\t}\n\treturn ltx.NewFileInfoSliceIterator(infos), nil\n}\n\nfunc (c *testReplicaClient) OpenLTXFile(_ context.Context, level int, minTXID, maxTXID ltx.TXID, _, _ int64) (io.ReadCloser, error) {\n\tinternal.OperationTotalCounterVec.WithLabelValues(c.Type(), \"GET\").Inc()\n\n\tpath := filepath.Join(c.dir, fmt.Sprintf(\"l%d\", level), ltx.FormatFilename(minTXID, maxTXID))\n\treturn os.Open(path)\n}\n\nfunc (c *testReplicaClient) WriteLTXFile(_ context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\tdata, err := io.ReadAll(r)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tlevelDir := filepath.Join(c.dir, fmt.Sprintf(\"l%d\", level))\n\tif err := os.MkdirAll(levelDir, 0o755); err != nil {\n\t\treturn nil, err\n\t}\n\tpath := filepath.Join(levelDir, ltx.FormatFilename(minTXID, maxTXID))\n\tif err := os.WriteFile(path, data, 0o600); err != nil {\n\t\treturn nil, err\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(c.Type(), \"PUT\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(c.Type(), \"PUT\").Add(float64(len(data)))\n\n\treturn <x.FileInfo{Level: level, MinTXID: minTXID, MaxTXID: maxTXID, Size: int64(len(data))}, nil\n}\n\nfunc (c *testReplicaClient) DeleteLTXFiles(_ context.Context, infos []*ltx.FileInfo) error {\n\tinternal.OperationTotalCounterVec.WithLabelValues(c.Type(), \"DELETE\").Add(float64(len(infos)))\n\treturn nil\n}\n\nfunc (c *testReplicaClient) DeleteAll(_ context.Context) error {\n\treturn nil\n}\n\n// TestCalcWALSize ensures calcWALSize doesn't overflow with large page sizes.\n// Regression test for uint32 overflow bug where large page sizes (>=16KB)\n// caused incorrect WAL size calculations, triggering checkpoints too early.\nfunc TestCalcWALSize(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpageSize uint32\n\t\tpageN uint32\n\t\texpected int64\n\t}{\n\t\t{\n\t\t\tname: \"4KB pages, 121359 pages (default TruncatePageN)\",\n\t\t\tpageSize: 4096,\n\t\t\tpageN: 121359,\n\t\t\texpected: int64(WALHeaderSize) + (int64(WALFrameHeaderSize+4096) * 121359),\n\t\t},\n\t\t{\n\t\t\tname: \"16KB pages, 121359 pages\",\n\t\t\tpageSize: 16384,\n\t\t\tpageN: 121359,\n\t\t\texpected: int64(WALHeaderSize) + (int64(WALFrameHeaderSize+16384) * 121359),\n\t\t},\n\t\t{\n\t\t\tname: \"32KB pages, 121359 pages\",\n\t\t\tpageSize: 32768,\n\t\t\tpageN: 121359,\n\t\t\t// Expected: ~4.0 GB with 32KB pages. Bug previously overflowed.\n\t\t\texpected: int64(WALHeaderSize) + (int64(WALFrameHeaderSize+32768) * 121359),\n\t\t},\n\t\t{\n\t\t\tname: \"64KB pages, 121359 pages\",\n\t\t\tpageSize: 65536,\n\t\t\tpageN: 121359,\n\t\t\texpected: int64(WALHeaderSize) + (int64(WALFrameHeaderSize+65536) * 121359),\n\t\t},\n\t\t{\n\t\t\tname: \"1KB pages, 1k pages (min checkpoint)\",\n\t\t\tpageSize: 1024,\n\t\t\tpageN: 1000,\n\t\t\texpected: int64(WALHeaderSize) + (int64(WALFrameHeaderSize+1024) * 1000),\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot := calcWALSize(tt.pageSize, tt.pageN)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"calcWALSize(%d, %d) = %d, want %d (%.2f GB vs %.2f GB)\",\n\t\t\t\t\ttt.pageSize, tt.pageN, got, tt.expected,\n\t\t\t\t\tfloat64(got)/(1024*1024*1024), float64(tt.expected)/(1024*1024*1024))\n\t\t\t}\n\n\t\t\tif got <= 0 {\n\t\t\t\tt.Errorf(\"calcWALSize(%d, %d) = %d, should be positive\", tt.pageSize, tt.pageN, got)\n\t\t\t}\n\n\t\t\tif tt.pageSize >= 32768 && tt.pageN >= 100000 {\n\t\t\t\t// Sanity check: ensure result is at least (page_size * page_count)\n\t\t\t\tminExpected := int64(tt.pageSize) * int64(tt.pageN)\n\t\t\t\tif got < minExpected {\n\t\t\t\t\tt.Errorf(\"calcWALSize(%d, %d) = %d (%.2f GB), suspiciously small, possible overflow\",\n\t\t\t\t\t\ttt.pageSize, tt.pageN, got, float64(got)/(1024*1024*1024))\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestDB_Sync_UpdatesMetrics verifies that DB size, WAL size, and total WAL bytes\n// metrics are properly updated during sync operations.\n// Regression test for issue #876: metrics were defined but never updated.\nfunc TestDB_Sync_UpdatesMetrics(t *testing.T) {\n\t// Set up database manually (can't use testingutil due to import cycle)\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\t// Create and open litestream DB\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0 // disable background goroutine\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}()\n\n\t// Open SQL connection\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal;`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Insert data to create DB and WAL content\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (1, 'test data')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sync to trigger metric updates\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Verify DB size metric matches actual file size\n\tdbSizeMetric := dbSizeGaugeVec.WithLabelValues(db.Path())\n\tdbSizeValue := testutil.ToFloat64(dbSizeMetric)\n\tdbFileInfo, err := os.Stat(db.Path())\n\tif err != nil {\n\t\tt.Fatalf(\"failed to stat db file: %v\", err)\n\t}\n\tif dbSizeValue != float64(dbFileInfo.Size()) {\n\t\tt.Fatalf(\"litestream_db_size=%v, want %v\", dbSizeValue, dbFileInfo.Size())\n\t}\n\n\t// Verify WAL size metric matches actual file size\n\twalSizeMetric := walSizeGaugeVec.WithLabelValues(db.Path())\n\twalSizeValue := testutil.ToFloat64(walSizeMetric)\n\twalFileInfo, err := os.Stat(db.WALPath())\n\tif err != nil {\n\t\tt.Fatalf(\"failed to stat wal file: %v\", err)\n\t}\n\tif walSizeValue != float64(walFileInfo.Size()) {\n\t\tt.Fatalf(\"litestream_wal_size=%v, want %v\", walSizeValue, walFileInfo.Size())\n\t}\n\n\t// Verify total WAL bytes counter was incremented\n\ttotalWALMetric := totalWALBytesCounterVec.WithLabelValues(db.Path())\n\ttotalWALValue := testutil.ToFloat64(totalWALMetric)\n\tif totalWALValue <= 0 {\n\t\tt.Fatalf(\"litestream_total_wal_bytes=%v, want > 0\", totalWALValue)\n\t}\n\n\t// Verify txid metric was updated (should be > 0 after writes)\n\ttxidMetric := txIDIndexGaugeVec.WithLabelValues(db.Path())\n\ttxidValue := testutil.ToFloat64(txidMetric)\n\tif txidValue <= 0 {\n\t\tt.Fatalf(\"litestream_txid=%v, want > 0\", txidValue)\n\t}\n\n\t// Verify sync count was incremented\n\tsyncCountMetric := syncNCounterVec.WithLabelValues(db.Path())\n\tsyncCountValue := testutil.ToFloat64(syncCountMetric)\n\tif syncCountValue <= 0 {\n\t\tt.Fatalf(\"litestream_sync_count=%v, want > 0\", syncCountValue)\n\t}\n\n\t// Verify sync seconds was recorded\n\tsyncSecondsMetric := syncSecondsCounterVec.WithLabelValues(db.Path())\n\tsyncSecondsValue := testutil.ToFloat64(syncSecondsMetric)\n\tif syncSecondsValue <= 0 {\n\t\tt.Fatalf(\"litestream_sync_seconds=%v, want > 0\", syncSecondsValue)\n\t}\n}\n\n// TestDB_Checkpoint_UpdatesMetrics verifies that checkpoint metrics are updated.\nfunc TestDB_Checkpoint_UpdatesMetrics(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal;`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (1, 'test data')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sync first to initialize database state\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Get baseline checkpoint metrics\n\tbaselineCount := testutil.ToFloat64(checkpointNCounterVec.WithLabelValues(db.Path(), \"PASSIVE\"))\n\tbaselineSeconds := testutil.ToFloat64(checkpointSecondsCounterVec.WithLabelValues(db.Path(), \"PASSIVE\"))\n\n\t// Force checkpoint\n\tif err := db.Checkpoint(context.Background(), \"PASSIVE\"); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Verify checkpoint_count was incremented\n\tcheckpointCountMetric := checkpointNCounterVec.WithLabelValues(db.Path(), \"PASSIVE\")\n\tcheckpointCountValue := testutil.ToFloat64(checkpointCountMetric)\n\tif checkpointCountValue <= baselineCount {\n\t\tt.Fatalf(\"litestream_checkpoint_count=%v, want > %v\", checkpointCountValue, baselineCount)\n\t}\n\n\t// Verify checkpoint_seconds was recorded\n\tcheckpointSecondsMetric := checkpointSecondsCounterVec.WithLabelValues(db.Path(), \"PASSIVE\")\n\tcheckpointSecondsValue := testutil.ToFloat64(checkpointSecondsMetric)\n\tif checkpointSecondsValue <= baselineSeconds {\n\t\tt.Fatalf(\"litestream_checkpoint_seconds=%v, want > %v\", checkpointSecondsValue, baselineSeconds)\n\t}\n}\n\n// TestDB_ReplicaSync_OperationMetrics verifies that replica operation metrics\n// (PUT total and bytes) are incremented when Replica.Sync() uploads LTX files.\nfunc TestDB_ReplicaSync_OperationMetrics(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (1, 'test data')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tbaselinePutTotal := testutil.ToFloat64(\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(\"test\", \"PUT\"))\n\tbaselinePutBytes := testutil.ToFloat64(\n\t\tinternal.OperationBytesCounterVec.WithLabelValues(\"test\", \"PUT\"))\n\n\tif err := db.Replica.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tputTotal := testutil.ToFloat64(\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(\"test\", \"PUT\"))\n\tputBytes := testutil.ToFloat64(\n\t\tinternal.OperationBytesCounterVec.WithLabelValues(\"test\", \"PUT\"))\n\n\tif putTotal <= baselinePutTotal {\n\t\tt.Fatalf(\"litestream_replica_operation_total[test,PUT]=%v, want > %v\", putTotal, baselinePutTotal)\n\t}\n\tif putBytes <= baselinePutBytes {\n\t\tt.Fatalf(\"litestream_replica_operation_bytes[test,PUT]=%v, want > %v\", putBytes, baselinePutBytes)\n\t}\n}\n\n// TestDB_Sync_ErrorMetrics verifies that sync error counter is incremented on failure.\nfunc TestDB_Sync_ErrorMetrics(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tworkingClient := &testReplicaClient{dir: t.TempDir()}\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = workingClient\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() { _ = db.Close(context.Background()) }()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (1, 'test data')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (2, 'more data')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tbaselineErrors := testutil.ToFloat64(syncErrorNCounterVec.WithLabelValues(db.Path()))\n\n\tif err := os.Remove(db.WALPath()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := db.Sync(context.Background()); err == nil {\n\t\tt.Fatal(\"expected error from sync with missing WAL\")\n\t}\n\n\tsyncErrorValue := testutil.ToFloat64(syncErrorNCounterVec.WithLabelValues(db.Path()))\n\tif syncErrorValue <= baselineErrors {\n\t\tt.Fatalf(\"litestream_sync_error_count=%v, want > %v\", syncErrorValue, baselineErrors)\n\t}\n}\n\n// TestDB_Checkpoint_ErrorMetrics verifies that checkpoint error counter is incremented on failure.\nfunc TestDB_Checkpoint_ErrorMetrics(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() { _ = db.Close(context.Background()) }()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (1, 'test data')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tbaselineErrors := testutil.ToFloat64(checkpointErrorNCounterVec.WithLabelValues(db.Path(), \"PASSIVE\"))\n\n\tdb.db.Close()\n\n\tif err := db.execCheckpoint(context.Background(), \"PASSIVE\"); err == nil {\n\t\tt.Fatal(\"expected error from checkpoint with closed db\")\n\t}\n\n\tcheckpointErrorValue := testutil.ToFloat64(checkpointErrorNCounterVec.WithLabelValues(db.Path(), \"PASSIVE\"))\n\tif checkpointErrorValue <= baselineErrors {\n\t\tt.Fatalf(\"litestream_checkpoint_error_count=%v, want > %v\", checkpointErrorValue, baselineErrors)\n\t}\n}\n\n// TestDB_L0RetentionMetrics verifies that L0 retention gauges are set during enforcement.\nfunc TestDB_L0RetentionMetrics(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tclient := &testReplicaClient{dir: t.TempDir()}\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.L0Retention = 1 * time.Nanosecond\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() { _ = db.Close(context.Background()) }()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfor i := range 3 {\n\t\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (?, 'data')`, i); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Replica.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tcompactor := NewCompactor(client, slog.Default())\n\tif _, err := compactor.Compact(context.Background(), 1); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdbName := filepath.Base(db.Path())\n\tif err := db.EnforceL0RetentionByTime(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\teligible := testutil.ToFloat64(internal.L0RetentionGaugeVec.WithLabelValues(dbName, \"eligible\"))\n\tnotCompacted := testutil.ToFloat64(internal.L0RetentionGaugeVec.WithLabelValues(dbName, \"not_compacted\"))\n\ttooRecent := testutil.ToFloat64(internal.L0RetentionGaugeVec.WithLabelValues(dbName, \"too_recent\"))\n\n\tif eligible+notCompacted+tooRecent == 0 {\n\t\tt.Fatalf(\"expected at least one L0 retention gauge > 0, got eligible=%v not_compacted=%v too_recent=%v\",\n\t\t\teligible, notCompacted, tooRecent)\n\t}\n}\n\n// TestDB_Verify_WALOffsetAtHeader tests that verify() handles the edge case where\n// an LTX file has WALOffset=WALHeaderSize and WALSize=0, which means we're at the\n// beginning of the WAL with no frames written yet.\n// Regression test for issue #900: prev WAL offset is less than the header size: -4088\nfunc TestDB_Verify_WALOffsetAtHeader(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal;`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Perform initial sync to set up page size and initial state\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Read the WAL header to get current salt values\n\twalHdr, err := readWALHeader(db.WALPath())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tsalt1 := binary.BigEndian.Uint32(walHdr[16:])\n\tsalt2 := binary.BigEndian.Uint32(walHdr[20:])\n\n\t// Create an LTX file with WALOffset=WALHeaderSize (32) and WALSize=0\n\t// This simulates the condition in issue #900\n\tltxDir := db.LTXLevelDir(0)\n\tif err := os.MkdirAll(ltxDir, 0o755); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Get current position to determine next TXID\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tnextTXID := pos.TXID + 1\n\n\tltxPath := db.LTXPath(0, nextTXID, nextTXID)\n\tf, err := os.Create(ltxPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tenc, err := ltx.NewEncoder(f)\n\tif err != nil {\n\t\tf.Close()\n\t\tt.Fatal(err)\n\t}\n\n\t// Create header with WALOffset=32 (WALHeaderSize) and WALSize=0\n\thdr := ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\tPageSize: uint32(db.pageSize),\n\t\tCommit: 2,\n\t\tMinTXID: nextTXID,\n\t\tMaxTXID: nextTXID,\n\t\tTimestamp: 1000000,\n\t\tWALOffset: WALHeaderSize, // 32 - at start of WAL\n\t\tWALSize: 0, // No WAL data - this triggers the bug\n\t\tWALSalt1: salt1,\n\t\tWALSalt2: salt2,\n\t}\n\n\tif err := enc.EncodeHeader(hdr); err != nil {\n\t\tf.Close()\n\t\tt.Fatal(err)\n\t}\n\tif err := enc.Close(); err != nil {\n\t\tf.Close()\n\t\tt.Fatal(err)\n\t}\n\tif err := f.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Invalidate cached position since we wrote an L0 file directly.\n\tdb.invalidatePosCache()\n\n\t// Now call verify - before the fix, this would fail with:\n\t// \"prev WAL offset is less than the header size: -4088\"\n\tinfo, err := db.verify(context.Background())\n\tif err != nil {\n\t\tt.Fatalf(\"verify() returned error: %v\", err)\n\t}\n\n\t// Verify the returned info is sensible\n\tif info.offset != WALHeaderSize {\n\t\tt.Errorf(\"expected offset=%d, got %d\", WALHeaderSize, info.offset)\n\t}\n\t// Salt matches, so snapshotting should be false\n\tif info.snapshotting {\n\t\tt.Errorf(\"expected snapshotting=false when salt matches, got true\")\n\t}\n}\n\n// TestDB_Verify_WALOffsetAtHeader_SaltMismatch tests that verify() correctly\n// triggers a snapshot when WALOffset=WALHeaderSize, WALSize=0, and the salt\n// values don't match the current WAL header.\n// Companion test to TestDB_Verify_WALOffsetAtHeader for full branch coverage.\nfunc TestDB_Verify_WALOffsetAtHeader_SaltMismatch(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal;`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Perform initial sync to set up page size and initial state\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Read the WAL header to get current salt values\n\twalHdr, err := readWALHeader(db.WALPath())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tsalt1 := binary.BigEndian.Uint32(walHdr[16:])\n\tsalt2 := binary.BigEndian.Uint32(walHdr[20:])\n\n\t// Create an LTX file with WALOffset=WALHeaderSize (32) and WALSize=0\n\t// but with DIFFERENT salt values to simulate a salt reset\n\tltxDir := db.LTXLevelDir(0)\n\tif err := os.MkdirAll(ltxDir, 0o755); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Get current position to determine next TXID\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tnextTXID := pos.TXID + 1\n\n\tltxPath := db.LTXPath(0, nextTXID, nextTXID)\n\tf, err := os.Create(ltxPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tenc, err := ltx.NewEncoder(f)\n\tif err != nil {\n\t\tf.Close()\n\t\tt.Fatal(err)\n\t}\n\n\t// Create header with WALOffset=32 (WALHeaderSize) and WALSize=0\n\t// Use different salt values to trigger salt mismatch branch\n\thdr := ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\tPageSize: uint32(db.pageSize),\n\t\tCommit: 2,\n\t\tMinTXID: nextTXID,\n\t\tMaxTXID: nextTXID,\n\t\tTimestamp: 1000000,\n\t\tWALOffset: WALHeaderSize, // 32 - at start of WAL\n\t\tWALSize: 0, // No WAL data\n\t\tWALSalt1: salt1 + 1, // Different salt to trigger mismatch\n\t\tWALSalt2: salt2 + 1, // Different salt to trigger mismatch\n\t}\n\n\tif err := enc.EncodeHeader(hdr); err != nil {\n\t\tf.Close()\n\t\tt.Fatal(err)\n\t}\n\tif err := enc.Close(); err != nil {\n\t\tf.Close()\n\t\tt.Fatal(err)\n\t}\n\tif err := f.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Invalidate cached position since we wrote an L0 file directly.\n\tdb.invalidatePosCache()\n\n\t// Call verify - should succeed but indicate snapshotting due to salt mismatch\n\tinfo, err := db.verify(context.Background())\n\tif err != nil {\n\t\tt.Fatalf(\"verify() returned error: %v\", err)\n\t}\n\n\t// Verify the returned info indicates snapshotting due to salt reset\n\tif info.offset != WALHeaderSize {\n\t\tt.Errorf(\"expected offset=%d, got %d\", WALHeaderSize, info.offset)\n\t}\n\tif !info.snapshotting {\n\t\tt.Errorf(\"expected snapshotting=true when salt mismatches, got false\")\n\t}\n\tif info.reason != \"wal header salt reset, snapshotting\" {\n\t\tt.Errorf(\"expected reason='wal header salt reset, snapshotting', got %q\", info.reason)\n\t}\n}\n\n// TestDB_releaseReadLock_DoubleRollback verifies that calling releaseReadLock()\n// after the read transaction has already been rolled back does not return an error.\n// This can happen during shutdown when concurrent checkpoint and close operations\n// both attempt to release the read lock.\n// Regression test for issue #934.\nfunc TestDB_releaseReadLock_DoubleRollback(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Open SQL connection to create a WAL database\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal;`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sync to initialize the database and acquire read lock\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Verify read transaction exists\n\tif db.rtx == nil {\n\t\tt.Fatal(\"expected read transaction to exist after Sync\")\n\t}\n\n\t// First rollback - simulates what happens in execCheckpoint()\n\tif err := db.rtx.Rollback(); err != nil {\n\t\tt.Fatalf(\"first rollback failed: %v\", err)\n\t}\n\n\t// Second call to releaseReadLock() - simulates what happens in Close()\n\t// This should NOT return an error even though the transaction is already rolled back.\n\t// Before the fix, this would return \"sql: transaction has already been committed or rolled back\"\n\tif err := db.releaseReadLock(); err != nil {\n\t\tt.Fatalf(\"releaseReadLock() returned error after double rollback: %v\", err)\n\t}\n\n\t// Clean up - set rtx to nil since we manually rolled it back\n\tdb.rtx = nil\n\n\t// Close should work without error\n\tif err := db.Close(context.Background()); err != nil {\n\t\tt.Fatalf(\"Close() failed: %v\", err)\n\t}\n}\n\n// TestDB_CheckpointDoesNotTriggerSnapshot verifies that a checkpoint\n// followed by a sync does not trigger an unnecessary full snapshot.\n// This is a regression test for issue #927 (runaway disk usage).\n//\n// The bug: After checkpoint truncates WAL, verify() sees old LTX position\n// is beyond new WAL size and triggers snapshotting=true unnecessarily.\nfunc TestDB_CheckpointDoesNotTriggerSnapshot(t *testing.T) {\n\tt.Run(\"TruncateMode\", func(t *testing.T) {\n\t\ttestCheckpointSnapshot(t, CheckpointModeTruncate)\n\t})\n\tt.Run(\"PassiveMode\", func(t *testing.T) {\n\t\ttestCheckpointSnapshot(t, CheckpointModePassive)\n\t})\n}\n\nfunc testCheckpointSnapshot(t *testing.T, mode string) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0 // Disable background monitor\n\tdb.CheckpointInterval = 0 // Disable time-based checkpoints\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal;`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Create initial data\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\t// Insert enough data to have a meaningful WAL\n\tfor i := 0; i < 100; i++ {\n\t\tdata := fmt.Sprintf(\"test data padding row %d with extra content\", i)\n\t\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (?, ?)`, i, data); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tctx := context.Background()\n\n\t// Perform initial sync\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tpos1, _ := db.Pos()\n\tt.Logf(\"After initial sync: TXID=%d\", pos1.TXID)\n\n\t// Make a change and sync to establish \"normal\" state\n\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (9999, 'before checkpoint')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tpos2, _ := db.Pos()\n\tt.Logf(\"After pre-checkpoint sync: TXID=%d\", pos2.TXID)\n\n\t// Call verify() BEFORE checkpoint to confirm snapshotting=false\n\tinfo1, err := db.verify(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"Before checkpoint: verify() snapshotting=%v reason=%q\", info1.snapshotting, info1.reason)\n\n\t// Perform checkpoint - this may restart the WAL with new salt\n\tif err := db.Checkpoint(ctx, mode); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"Checkpoint mode=%s completed\", mode)\n\tposAfterChk, _ := db.Pos()\n\tt.Logf(\"After checkpoint: TXID=%d\", posAfterChk.TXID)\n\n\t// Make a small change to create some WAL data\n\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (10000, 'after checkpoint')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Call verify() AFTER checkpoint - THIS IS THE BUG CHECK\n\t// With the bug, snapshotting=true because verify() sees:\n\t// - Old LTX has WALOffset+WALSize pointing to old (larger) WAL\n\t// - New WAL is truncated (smaller)\n\t// - Line 973: info.offset > fi.Size() → \"wal truncated\" → snapshotting=true\n\tinfo2, err := db.verify(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"After checkpoint: verify() snapshotting=%v reason=%q\", info2.snapshotting, info2.reason)\n\n\t// The key assertion: after OUR checkpoint (not external process),\n\t// we should NOT require a full snapshot.\n\tif info2.snapshotting {\n\t\tt.Errorf(\"verify() returned snapshotting=true after checkpoint, reason=%q. \"+\n\t\t\t\"This is the bug: checkpoint followed by sync should NOT require full snapshot.\",\n\t\t\tinfo2.reason)\n\t}\n}\n\n// TestDB_MultipleCheckpointsWithWrites tests that multiple checkpoint cycles\n// don't trigger excessive snapshots. This simulates the scenario from issue #927\n// where users reported 5GB snapshots every 3-4 minutes.\nfunc TestDB_MultipleCheckpointsWithWrites(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.CheckpointInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal;`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INT, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tctx := context.Background()\n\tsnapshotCount := 0\n\n\t// Simulate multiple checkpoint cycles with writes\n\tfor cycle := 0; cycle < 5; cycle++ {\n\t\t// Insert some data\n\t\tfor i := 0; i < 10; i++ {\n\t\t\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (?, ?)`, cycle*100+i, \"data\"); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\n\t\t// Sync\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Check if this was a snapshot\n\t\tinfo, err := db.verify(ctx)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.snapshotting {\n\t\t\tsnapshotCount++\n\t\t\tt.Logf(\"Cycle %d: SNAPSHOT triggered, reason=%q\", cycle, info.reason)\n\t\t} else {\n\t\t\tt.Logf(\"Cycle %d: incremental sync\", cycle)\n\t\t}\n\n\t\t// Checkpoint\n\t\tif err := db.Checkpoint(ctx, CheckpointModePassive); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\t// We expect only 1 snapshot (the initial one), not one per cycle\n\t// With the bug, we'd see a snapshot after every checkpoint\n\tif snapshotCount > 1 {\n\t\tt.Errorf(\"Too many snapshots triggered: %d (expected 1 for initial sync)\", snapshotCount)\n\t}\n}\n\n// TestIsDiskFullError tests the disk full error detection helper.\nfunc TestIsDiskFullError(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\terr error\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"nil error\",\n\t\t\terr: nil,\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"no space left on device\",\n\t\t\terr: errors.New(\"write /tmp/file: no space left on device\"),\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"No Space Left On Device (uppercase)\",\n\t\t\terr: errors.New(\"No Space Left On Device\"),\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"disk quota exceeded\",\n\t\t\terr: errors.New(\"write: disk quota exceeded\"),\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"ENOSPC\",\n\t\t\terr: errors.New(\"ENOSPC: cannot write file\"),\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"EDQUOT\",\n\t\t\terr: errors.New(\"error EDQUOT while writing\"),\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"regular error\",\n\t\t\terr: errors.New(\"connection refused\"),\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"permission denied\",\n\t\t\terr: errors.New(\"permission denied\"),\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wrapped disk full error\",\n\t\t\terr: fmt.Errorf(\"sync failed: %w\", errors.New(\"no space left on device\")),\n\t\t\texpected: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tresult := isDiskFullError(tt.err)\n\t\t\tif result != tt.expected {\n\t\t\t\tt.Errorf(\"isDiskFullError(%v) = %v, want %v\", tt.err, result, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestIsSQLiteBusyError tests the SQLite busy error detection helper.\nfunc TestIsSQLiteBusyError(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\terr error\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"nil error\",\n\t\t\terr: nil,\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"database is locked\",\n\t\t\terr: errors.New(\"database is locked\"),\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"SQLITE_BUSY\",\n\t\t\terr: errors.New(\"SQLITE_BUSY: cannot commit\"),\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"regular error\",\n\t\t\terr: errors.New(\"connection refused\"),\n\t\t\texpected: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tresult := isSQLiteBusyError(tt.err)\n\t\t\tif result != tt.expected {\n\t\t\t\tt.Errorf(\"isSQLiteBusyError(%v) = %v, want %v\", tt.err, result, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestDB_IdleCheckpointSnapshotLoop tests for the feedback loop described in issue #997.\n// After bulk inserts trigger a checkpoint, litestream should NOT enter a self-perpetuating\n// loop where checkpoint triggers cause repeated LTX file creation on an idle database.\n//\n// The bug occurred because:\n// 1. PASSIVE checkpoint completes but doesn't truncate WAL file\n// 2. WAL salt changes, new _litestream_seq write goes to offset 32 with new salt\n// 3. Old WAL frames (with old salt) make file size exceed checkpoint threshold\n// 4. checkpointIfNeeded() uses file size, triggering another checkpoint\n// 5. Loop repeats, creating LTX files every sync cycle\n//\n// The fix uses logical WAL offset (from LTX) instead of file size for checkpoint decisions.\nfunc TestDB_IdleCheckpointSnapshotLoop(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"test.db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.CheckpointInterval = 0\n\tdb.MinCheckpointPageN = 10 // Low threshold to trigger checkpoint easily\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal;`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`CREATE TABLE test (id INTEGER PRIMARY KEY, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tctx := context.Background()\n\n\t// Initial sync\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Bulk inserts WITHOUT transaction (as in the bug report)\n\t// This creates many WAL frames that will trigger a checkpoint\n\tfor i := 0; i < 100; i++ {\n\t\tif _, err := sqldb.Exec(`INSERT INTO test VALUES (?, ?)`, i, \"test data padding\"); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\t// Sync and trigger checkpoint via size threshold\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Force a checkpoint that will reset WAL salt\n\tif err := db.Checkpoint(ctx, CheckpointModePassive); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tafterCheckpointPos, _ := db.Pos()\n\n\t// Now the database is IDLE - no more application writes\n\t// Simulate multiple sync cycles (as would happen with MonitorInterval)\n\tfor cycle := 0; cycle < 5; cycle++ {\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tfinalPos, _ := db.Pos()\n\n\t// The key assertion: TXID should not be incrementing every cycle.\n\t// With the bug, TXID would increment 5 times (one per cycle).\n\t// The fix ensures checkpoint decisions use logical WAL size,\n\t// preventing spurious checkpoints when WAL file contains stale frames.\n\ttxidGrowth := int(finalPos.TXID - afterCheckpointPos.TXID)\n\tif txidGrowth > 1 {\n\t\tt.Errorf(\"TXID grew by %d during idle cycles (expected <= 1). \"+\n\t\t\t\"This is issue #997: checkpoint triggers infinite LTX creation loop.\", txidGrowth)\n\t}\n}\n\n// TestDB_Issue994_RunawayDiskUsage reproduces the scenario from issue #994 where\n// the local -litestream directory grows unboundedly. The reporter saw ~10MB/s growth\n// in LTX files. This test verifies that after bulk writes and idle sync cycles,\n// local LTX file count and total size stabilize rather than growing linearly.\nfunc TestDB_Issue994_RunawayDiskUsage(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"test.db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.CheckpointInterval = 0\n\tdb.MinCheckpointPageN = 10\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}()\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`CREATE TABLE test (id INTEGER PRIMARY KEY, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tctx := context.Background()\n\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Bulk inserts without a wrapping transaction (matches the #994 scenario).\n\t// This builds up WAL frames and will trigger checkpoint thresholds.\n\tfor i := 0; i < 200; i++ {\n\t\tif _, err := sqldb.Exec(`INSERT INTO test VALUES (?, ?)`, i, \"padding data for disk usage test\"); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\t// Sync to create LTX files from the WAL data.\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Force a checkpoint (mirrors what happens in production after bulk writes).\n\tif err := db.Checkpoint(ctx, CheckpointModePassive); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Measure the baseline LTX directory size after initial sync + checkpoint.\n\tbaselineSize := dirSize(t, db.LTXDir())\n\tbaselineFiles := dirFileCount(t, db.LTXDir())\n\tt.Logf(\"baseline: %d bytes, %d files\", baselineSize, baselineFiles)\n\n\t// Run 20 idle sync cycles (no application writes).\n\t// With the #994 bug, each cycle would create a new LTX snapshot file,\n\t// causing linear disk growth.\n\tfor cycle := 0; cycle < 20; cycle++ {\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tfinalSize := dirSize(t, db.LTXDir())\n\tfinalFiles := dirFileCount(t, db.LTXDir())\n\tt.Logf(\"after 20 idle cycles: %d bytes, %d files\", finalSize, finalFiles)\n\n\t// Allow for at most 1 additional LTX file (the _litestream_seq bookkeeping write).\n\t// With the bug, we'd see 20+ new files.\n\tnewFiles := finalFiles - baselineFiles\n\tif newFiles > 2 {\n\t\tt.Errorf(\"LTX file count grew by %d during 20 idle sync cycles (expected <= 2). \"+\n\t\t\t\"This indicates issue #994: runaway LTX file creation.\", newFiles)\n\t}\n\n\t// Size should not grow significantly. Allow 2x as generous margin.\n\tif baselineSize > 0 && finalSize > baselineSize*2 {\n\t\tt.Errorf(\"LTX directory grew from %d to %d bytes during idle cycles (>2x growth). \"+\n\t\t\t\"This indicates issue #994: runaway disk usage.\", baselineSize, finalSize)\n\t}\n}\n\nfunc dirSize(t *testing.T, path string) int64 {\n\tt.Helper()\n\tvar size int64\n\terr := filepath.Walk(path, func(_ string, info os.FileInfo, err error) error {\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif !info.IsDir() {\n\t\t\tsize += info.Size()\n\t\t}\n\t\treturn nil\n\t})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\treturn size\n}\n\nfunc dirFileCount(t *testing.T, path string) int {\n\tt.Helper()\n\tvar count int\n\terr := filepath.Walk(path, func(_ string, info os.FileInfo, err error) error {\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif !info.IsDir() {\n\t\t\tcount++\n\t\t}\n\t\treturn nil\n\t})\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\treturn count\n}\n\n// TestDB_WALPageCoverage_AllNewPagesPresent verifies that when SQLite grows a\n// database (increases page count), ALL new pages appear as WAL frames. This\n// test exercises SQLite's allocateBtreePage code path which calls\n// sqlite3PagerWrite on every newly allocated page.\n//\n// If this test passes, it confirms that SQLite does not skip WAL writes when\n// growing the database — Ben Bjohnson's skepticism about the zero-fill fix\n// (PR #1087 comment) is well-founded at the SQLite level.\nfunc TestDB_WALPageCoverage_AllNewPagesPresent(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INTEGER PRIMARY KEY, data BLOB)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfor i := 0; i < 100; i++ {\n\t\tblob := make([]byte, 3000)\n\t\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (?, ?)`, i, blob); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\twalFile, err := os.Open(dbPath + \"-wal\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer walFile.Close()\n\n\trd, err := NewWALReader(walFile, slog.Default())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tpageMap, _, commit, err := rd.PageMap(context.Background())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif commit == 0 {\n\t\tt.Fatal(\"expected non-zero commit from WAL\")\n\t}\n\n\tlockPgno := ltx.LockPgno(4096)\n\tvar missing []uint32\n\tfor pgno := uint32(1); pgno <= commit; pgno++ {\n\t\tif pgno == lockPgno {\n\t\t\tcontinue\n\t\t}\n\t\tif _, ok := pageMap[pgno]; !ok {\n\t\t\tmissing = append(missing, pgno)\n\t\t}\n\t}\n\n\tt.Logf(\"commit=%d, pages_in_wal=%d, missing=%d\", commit, len(pageMap), len(missing))\n\tif len(missing) > 0 {\n\t\tfirst := missing[0]\n\t\tlast := missing[len(missing)-1]\n\t\tt.Errorf(\"pages missing from WAL: %d total (first=%d, last=%d, commit=%d)\",\n\t\t\tlen(missing), first, last, commit)\n\t}\n}\n\n// TestDB_WriteLTXFromWAL_PageGrowthCoverage verifies that an incremental LTX\n// file produced by writeLTXFromWAL contains all new pages when the database\n// grows between syncs. This tests the full Litestream sync path.\nfunc TestDB_WriteLTXFromWAL_PageGrowthCoverage(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.CheckpointInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer db.Close(context.Background())\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INTEGER PRIMARY KEY, data BLOB)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tfor i := 0; i < 5; i++ {\n\t\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (?, ?)`, i, make([]byte, 100)); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tctx := context.Background()\n\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tpos1, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tf1, err := os.Open(db.LTXPath(0, pos1.TXID, pos1.TXID))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdec1 := ltx.NewDecoder(f1)\n\tif err := dec1.DecodeHeader(); err != nil {\n\t\tf1.Close()\n\t\tt.Fatal(err)\n\t}\n\tprevCommit := dec1.Header().Commit\n\tf1.Close()\n\tt.Logf(\"after sync 1: txid=%d, commit=%d\", pos1.TXID, prevCommit)\n\n\tfor i := 5; i < 150; i++ {\n\t\tblob := make([]byte, 3000)\n\t\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (?, ?)`, i, blob); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tpos2, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tf2, err := os.Open(db.LTXPath(0, pos2.TXID, pos2.TXID))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f2.Close()\n\n\tdec2 := ltx.NewDecoder(f2)\n\tif err := dec2.DecodeHeader(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tnewCommit := dec2.Header().Commit\n\n\tltx2Pages := make(map[uint32]bool)\n\tpageBuf := make([]byte, dec2.Header().PageSize)\n\tfor {\n\t\tvar phdr ltx.PageHeader\n\t\tif err := dec2.DecodePage(&phdr, pageBuf); err == io.EOF {\n\t\t\tbreak\n\t\t} else if err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tltx2Pages[phdr.Pgno] = true\n\t}\n\n\tlockPgno := ltx.LockPgno(dec2.Header().PageSize)\n\tvar missing []uint32\n\tfor pgno := prevCommit + 1; pgno <= newCommit; pgno++ {\n\t\tif pgno == lockPgno {\n\t\t\tcontinue\n\t\t}\n\t\tif !ltx2Pages[pgno] {\n\t\t\tmissing = append(missing, pgno)\n\t\t}\n\t}\n\n\tt.Logf(\"after sync 2: txid=%d, prevCommit=%d, newCommit=%d, pages_in_ltx=%d, missing=%d\",\n\t\tpos2.TXID, prevCommit, newCommit, len(ltx2Pages), len(missing))\n\tif len(missing) > 0 {\n\t\tfirst := missing[0]\n\t\tlast := missing[len(missing)-1]\n\t\tt.Errorf(\"pages missing from incremental LTX: %d total (first=%d, last=%d, prevCommit=%d, newCommit=%d)\",\n\t\t\tlen(missing), first, last, prevCommit, newCommit)\n\t}\n}\n\n// TestDB_Sync_CompactionValidAfterGrowthAndCheckpoint verifies that compaction\n// produces valid snapshots after a cycle of: grow DB, sync, checkpoint, grow\n// more, sync. If the zero-fill bug existed, compaction would fail with\n// \"nonsequential page numbers in snapshot transaction\".\nfunc TestDB_Sync_CompactionValidAfterGrowthAndCheckpoint(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.CheckpointInterval = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer db.Close(context.Background())\n\n\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer sqldb.Close()\n\n\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tctx := context.Background()\n\n\tif _, err := sqldb.Exec(`CREATE TABLE t (id INTEGER PRIMARY KEY, data BLOB)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfor i := 0; i < 50; i++ {\n\t\tblob := make([]byte, 3000)\n\t\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (?, ?)`, i, blob); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := db.Checkpoint(ctx, CheckpointModeTruncate); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfor i := 50; i < 100; i++ {\n\t\tblob := make([]byte, 3000)\n\t\tif _, err := sqldb.Exec(`INSERT INTO t VALUES (?, ?)`, i, blob); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"final txid=%d\", pos.TXID)\n\n\tvar readers []io.ReadCloser\n\tfor txid := ltx.TXID(1); txid <= pos.TXID; txid++ {\n\t\tpath := db.LTXPath(0, txid, txid)\n\t\tf, err := os.Open(path)\n\t\tif err != nil {\n\t\t\tif os.IsNotExist(err) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tt.Fatal(err)\n\t\t}\n\t\treaders = append(readers, f)\n\t}\n\tdefer func() {\n\t\tfor _, r := range readers {\n\t\t\tr.Close()\n\t\t}\n\t}()\n\n\tif len(readers) < 2 {\n\t\tt.Fatalf(\"expected at least 2 LTX files, got %d\", len(readers))\n\t}\n\n\tioReaders := make([]io.Reader, len(readers))\n\tfor i, r := range readers {\n\t\tioReaders[i] = r\n\t}\n\n\tvar buf bytes.Buffer\n\tc, err := ltx.NewCompactor(&buf, ioReaders)\n\tif err != nil {\n\t\tt.Fatalf(\"new compactor: %v\", err)\n\t}\n\tc.HeaderFlags = ltx.HeaderFlagNoChecksum\n\tif err := c.Compact(ctx); err != nil {\n\t\tt.Fatalf(\"compaction failed (this would indicate the zero-fill bug): %v\", err)\n\t}\n\n\tt.Logf(\"compaction succeeded: %d bytes, %d input files\", buf.Len(), len(readers))\n}\n\n// TestDB_Sync_InitErrorMetrics verifies that sync error counter is incremented\n// when db.init() fails. Regression test for issue #1128.\nfunc TestDB_Sync_InitErrorMetrics(t *testing.T) {\n\tdir := t.TempDir()\n\tdbPath := filepath.Join(dir, \"db\")\n\n\t// Create a directory at the DB path so init() will fail when trying to\n\t// open it as a SQLite database.\n\tif err := os.Mkdir(dbPath, 0o755); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tdb := NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.ShutdownSyncTimeout = 0\n\tdb.Replica = NewReplica(db)\n\tdb.Replica.Client = &testReplicaClient{dir: t.TempDir()}\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() {\n\t\t_ = db.Close(context.Background())\n\t}()\n\n\tbaselineErrors := testutil.ToFloat64(syncErrorNCounterVec.WithLabelValues(db.Path()))\n\n\terr := db.Sync(context.Background())\n\tif err == nil {\n\t\tt.Fatal(\"expected Sync to return error when init fails, got nil\")\n\t}\n\n\tsyncErrorValue := testutil.ToFloat64(syncErrorNCounterVec.WithLabelValues(db.Path()))\n\tif syncErrorValue <= baselineErrors {\n\t\tt.Fatalf(\"litestream_sync_error_count=%v, want > %v (init error should be counted)\", syncErrorValue, baselineErrors)\n\t}\n}\n" }, { "path": "db_shutdown_test.go", "content": "package litestream_test\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\t\"strings\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n\t\"github.com/benbjohnson/litestream/mock\"\n)\n\nfunc TestDB_Close_SyncRetry(t *testing.T) {\n\tt.Run(\"SucceedsAfterTransientFailure\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\n\t\t// Write some data to create LTX files\n\t\tif _, err := sqldb.Exec(`CREATE TABLE t (x)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create mock client that fails first 2 times, succeeds on 3rd\n\t\tvar attempts int32\n\t\tclient := &mock.ReplicaClient{\n\t\t\tLTXFilesFunc: func(_ context.Context, _ int, _ ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t},\n\t\t\tWriteLTXFileFunc: func(_ context.Context, _ int, _, _ ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\t\t\t\tn := atomic.AddInt32(&attempts, 1)\n\t\t\t\tif n < 3 {\n\t\t\t\t\treturn nil, errors.New(\"rate limited (429)\")\n\t\t\t\t}\n\t\t\t\t// Drain the reader\n\t\t\t\t_, _ = io.Copy(io.Discard, r)\n\t\t\t\treturn <x.FileInfo{}, nil\n\t\t\t},\n\t\t}\n\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\t\tdb.ShutdownSyncTimeout = 5 * time.Second\n\t\tdb.ShutdownSyncInterval = 50 * time.Millisecond\n\n\t\t// Close should succeed after retries\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"expected success after retries, got: %v\", err)\n\t\t}\n\t\tif got := atomic.LoadInt32(&attempts); got < 3 {\n\t\t\tt.Fatalf(\"expected at least 3 attempts, got %d\", got)\n\t\t}\n\t})\n\n\tt.Run(\"FailsAfterTimeout\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\n\t\t// Write some data\n\t\tif _, err := sqldb.Exec(`CREATE TABLE t (x)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create mock client that always fails\n\t\tvar attempts int32\n\t\tclient := &mock.ReplicaClient{\n\t\t\tLTXFilesFunc: func(_ context.Context, _ int, _ ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t},\n\t\t\tWriteLTXFileFunc: func(_ context.Context, _ int, _, _ ltx.TXID, _ io.Reader) (*ltx.FileInfo, error) {\n\t\t\t\tatomic.AddInt32(&attempts, 1)\n\t\t\t\treturn nil, errors.New(\"persistent error\")\n\t\t\t},\n\t\t}\n\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\t\tdb.ShutdownSyncTimeout = 300 * time.Millisecond\n\t\tdb.ShutdownSyncInterval = 50 * time.Millisecond\n\n\t\t// Close should fail with timeout error\n\t\terr := db.Close(context.Background())\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error after timeout\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"timeout\") {\n\t\t\tt.Fatalf(\"expected timeout error, got: %v\", err)\n\t\t}\n\t\t// Should have made multiple attempts\n\t\tif got := atomic.LoadInt32(&attempts); got < 2 {\n\t\t\tt.Fatalf(\"expected multiple retry attempts, got %d\", got)\n\t\t}\n\t})\n\n\tt.Run(\"RespectsContextCancellation\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\n\t\t// Write some data\n\t\tif _, err := sqldb.Exec(`CREATE TABLE t (x)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create mock client that always fails\n\t\tclient := &mock.ReplicaClient{\n\t\t\tLTXFilesFunc: func(_ context.Context, _ int, _ ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t},\n\t\t\tWriteLTXFileFunc: func(_ context.Context, _ int, _, _ ltx.TXID, _ io.Reader) (*ltx.FileInfo, error) {\n\t\t\t\treturn nil, errors.New(\"error\")\n\t\t\t},\n\t\t}\n\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\t\tdb.ShutdownSyncTimeout = 10 * time.Second\n\t\tdb.ShutdownSyncInterval = 50 * time.Millisecond\n\n\t\t// Cancel context after short delay\n\t\tctx, cancel := context.WithTimeout(context.Background(), 150*time.Millisecond)\n\t\tdefer cancel()\n\n\t\tstart := time.Now()\n\t\t_ = db.Close(ctx)\n\t\telapsed := time.Since(start)\n\n\t\t// Should exit within reasonable time of context cancellation\n\t\tif elapsed > 500*time.Millisecond {\n\t\t\tt.Fatalf(\"took too long to respect cancellation: %v\", elapsed)\n\t\t}\n\t})\n\n\tt.Run(\"ZeroTimeoutNoRetry\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\n\t\t// Write some data\n\t\tif _, err := sqldb.Exec(`CREATE TABLE t (x)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create mock client that always fails\n\t\tvar attempts int32\n\t\tclient := &mock.ReplicaClient{\n\t\t\tLTXFilesFunc: func(_ context.Context, _ int, _ ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t},\n\t\t\tWriteLTXFileFunc: func(_ context.Context, _ int, _, _ ltx.TXID, _ io.Reader) (*ltx.FileInfo, error) {\n\t\t\t\tatomic.AddInt32(&attempts, 1)\n\t\t\t\treturn nil, errors.New(\"error\")\n\t\t\t},\n\t\t}\n\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\t\tdb.ShutdownSyncTimeout = 0 // Disable retries\n\t\tdb.ShutdownSyncInterval = 50 * time.Millisecond\n\n\t\t// Close should fail after single attempt\n\t\tstart := time.Now()\n\t\terr := db.Close(context.Background())\n\t\telapsed := time.Since(start)\n\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error\")\n\t\t}\n\t\t// Should have only made 1 attempt\n\t\tif got := atomic.LoadInt32(&attempts); got != 1 {\n\t\t\tt.Fatalf(\"expected exactly 1 attempt with zero timeout, got %d\", got)\n\t\t}\n\t\t// Should be fast (no retry delay)\n\t\tif elapsed > 100*time.Millisecond {\n\t\t\tt.Fatalf(\"took too long for single attempt: %v\", elapsed)\n\t\t}\n\t})\n\n\tt.Run(\"SuccessFirstAttempt\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\n\t\t// Write some data\n\t\tif _, err := sqldb.Exec(`CREATE TABLE t (x)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create mock client that succeeds immediately\n\t\tvar attempts int32\n\t\tclient := &mock.ReplicaClient{\n\t\t\tLTXFilesFunc: func(_ context.Context, _ int, _ ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t},\n\t\t\tWriteLTXFileFunc: func(_ context.Context, _ int, _, _ ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\t\t\t\tatomic.AddInt32(&attempts, 1)\n\t\t\t\t// Drain the reader\n\t\t\t\t_, _ = io.Copy(io.Discard, r)\n\t\t\t\treturn <x.FileInfo{}, nil\n\t\t\t},\n\t\t}\n\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\t\tdb.ShutdownSyncTimeout = 5 * time.Second\n\t\tdb.ShutdownSyncInterval = 50 * time.Millisecond\n\n\t\t// Close should succeed immediately\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"expected success, got: %v\", err)\n\t\t}\n\t\t// Should have made exactly 1 attempt\n\t\tif got := atomic.LoadInt32(&attempts); got != 1 {\n\t\t\tt.Fatalf(\"expected exactly 1 attempt, got %d\", got)\n\t\t}\n\t})\n\n\tt.Run(\"DoneChannelInterruptsRetryLoop\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\n\t\t// Write some data\n\t\tif _, err := sqldb.Exec(`CREATE TABLE t (x)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create mock client that always fails\n\t\tvar attempts int32\n\t\tclient := &mock.ReplicaClient{\n\t\t\tLTXFilesFunc: func(_ context.Context, _ int, _ ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t},\n\t\t\tWriteLTXFileFunc: func(_ context.Context, _ int, _, _ ltx.TXID, _ io.Reader) (*ltx.FileInfo, error) {\n\t\t\t\tatomic.AddInt32(&attempts, 1)\n\t\t\t\treturn nil, errors.New(\"persistent error\")\n\t\t\t},\n\t\t}\n\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\t\tdb.ShutdownSyncTimeout = 10 * time.Second\n\t\tdb.ShutdownSyncInterval = 50 * time.Millisecond\n\n\t\t// Create done channel and close it after short delay\n\t\tdone := make(chan struct{})\n\t\tdb.Done = done\n\t\tgo func() {\n\t\t\ttime.Sleep(200 * time.Millisecond)\n\t\t\tclose(done)\n\t\t}()\n\n\t\tstart := time.Now()\n\t\terr := db.Close(context.Background())\n\t\telapsed := time.Since(start)\n\n\t\t// Should exit quickly (well before 10 second timeout)\n\t\tif elapsed > 2*time.Second {\n\t\t\tt.Fatalf(\"took too long to respond to done signal: %v\", elapsed)\n\t\t}\n\n\t\t// Should have error mentioning interrupt\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error after done signal\")\n\t\t}\n\t\tif !errors.Is(err, litestream.ErrShutdownInterrupted) {\n\t\t\tt.Fatalf(\"expected ErrShutdownInterrupted, got: %v\", err)\n\t\t}\n\n\t\t// Should have made at least 1 attempt before being interrupted\n\t\tif got := atomic.LoadInt32(&attempts); got < 1 {\n\t\t\tt.Fatalf(\"expected at least 1 attempt, got %d\", got)\n\t\t}\n\t})\n\n\tt.Run(\"AlreadyClosedDoneSkipsSync\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\n\t\t// Write some data\n\t\tif _, err := sqldb.Exec(`CREATE TABLE t (x)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create mock client that always fails\n\t\tvar attempts int32\n\t\tclient := &mock.ReplicaClient{\n\t\t\tLTXFilesFunc: func(_ context.Context, _ int, _ ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t},\n\t\t\tWriteLTXFileFunc: func(_ context.Context, _ int, _, _ ltx.TXID, _ io.Reader) (*ltx.FileInfo, error) {\n\t\t\t\tatomic.AddInt32(&attempts, 1)\n\t\t\t\treturn nil, errors.New(\"persistent error\")\n\t\t\t},\n\t\t}\n\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\t\tdb.ShutdownSyncTimeout = 10 * time.Second\n\t\tdb.ShutdownSyncInterval = 50 * time.Millisecond\n\n\t\t// Close done before calling Close\n\t\tdone := make(chan struct{})\n\t\tclose(done)\n\t\tdb.Done = done\n\n\t\tstart := time.Now()\n\t\terr := db.Close(context.Background())\n\t\telapsed := time.Since(start)\n\n\t\t// Should exit immediately\n\t\tif elapsed > 500*time.Millisecond {\n\t\t\tt.Fatalf(\"took too long with pre-closed done channel: %v\", elapsed)\n\t\t}\n\n\t\t// Should have error mentioning interrupt\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error with pre-closed done channel\")\n\t\t}\n\t\tif !errors.Is(err, litestream.ErrShutdownInterrupted) {\n\t\t\tt.Fatalf(\"expected ErrShutdownInterrupted, got: %v\", err)\n\t\t}\n\n\t\t// Should not have made any sync attempts\n\t\tif got := atomic.LoadInt32(&attempts); got != 0 {\n\t\t\tt.Fatalf(\"expected 0 sync attempts with pre-closed done, got %d\", got)\n\t\t}\n\t})\n\n\tt.Run(\"NilDoneBehavesLikeClose\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\n\t\t// Write some data\n\t\tif _, err := sqldb.Exec(`CREATE TABLE t (x)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create mock client that succeeds immediately\n\t\tvar attempts int32\n\t\tclient := &mock.ReplicaClient{\n\t\t\tLTXFilesFunc: func(_ context.Context, _ int, _ ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t},\n\t\t\tWriteLTXFileFunc: func(_ context.Context, _ int, _, _ ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\t\t\t\tatomic.AddInt32(&attempts, 1)\n\t\t\t\t_, _ = io.Copy(io.Discard, r)\n\t\t\t\treturn <x.FileInfo{}, nil\n\t\t\t},\n\t\t}\n\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\t\tdb.ShutdownSyncTimeout = 5 * time.Second\n\t\tdb.ShutdownSyncInterval = 50 * time.Millisecond\n\n\t\t// Done is nil by default, Close should work normally\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"expected success, got: %v\", err)\n\t\t}\n\t\tif got := atomic.LoadInt32(&attempts); got != 1 {\n\t\t\tt.Fatalf(\"expected exactly 1 attempt, got %d\", got)\n\t\t}\n\t})\n}\n" }, { "path": "db_test.go", "content": "package litestream_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"hash/crc64\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n\t\"github.com/benbjohnson/litestream/mock\"\n)\n\nfunc TestDB_Path(t *testing.T) {\n\tdb := testingutil.NewDB(t, \"/tmp/db\")\n\tif got, want := db.Path(), `/tmp/db`; got != want {\n\t\tt.Fatalf(\"Path()=%v, want %v\", got, want)\n\t}\n}\n\nfunc TestDB_WALPath(t *testing.T) {\n\tdb := testingutil.NewDB(t, \"/tmp/db\")\n\tif got, want := db.WALPath(), `/tmp/db-wal`; got != want {\n\t\tt.Fatalf(\"WALPath()=%v, want %v\", got, want)\n\t}\n}\n\nfunc TestDB_MetaPath(t *testing.T) {\n\tt.Run(\"Absolute\", func(t *testing.T) {\n\t\tdb := testingutil.NewDB(t, \"/tmp/db\")\n\t\tif got, want := db.MetaPath(), `/tmp/.db-litestream`; got != want {\n\t\t\tt.Fatalf(\"MetaPath()=%v, want %v\", got, want)\n\t\t}\n\t})\n\tt.Run(\"Relative\", func(t *testing.T) {\n\t\tdb := testingutil.NewDB(t, \"db\")\n\t\tif got, want := db.MetaPath(), `.db-litestream`; got != want {\n\t\t\tt.Fatalf(\"MetaPath()=%v, want %v\", got, want)\n\t\t}\n\t})\n}\n\n// Ensure we can compute a checksum on the real database.\nfunc TestDB_CRC64(t *testing.T) {\n\tt.Run(\"ErrNotExist\", func(t *testing.T) {\n\t\tdb := testingutil.MustOpenDB(t)\n\t\tdefer testingutil.MustCloseDB(t, db)\n\t\tif _, _, err := db.CRC64(context.Background()); !os.IsNotExist(err) {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"DB\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tt.Log(\"sync database\")\n\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tt.Log(\"compute crc64\")\n\n\t\tchksum0, _, err := db.CRC64(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tt.Log(\"issue change\")\n\n\t\t// Issue change that is applied to the WAL. Checksum should not change.\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if chksum1, _, err := db.CRC64(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if chksum0 == chksum1 {\n\t\t\tt.Fatal(\"expected different checksum event after WAL change\")\n\t\t}\n\n\t\tt.Log(\"checkpointing database\")\n\n\t\t// Checkpoint change into database. Checksum should change.\n\t\tif err := db.Checkpoint(context.Background(), litestream.CheckpointModeTruncate); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tt.Log(\"compute crc64 again\")\n\n\t\tif chksum2, _, err := db.CRC64(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if chksum0 == chksum2 {\n\t\t\tt.Fatal(\"expected different checksums after checkpoint\")\n\t\t}\n\t})\n}\n\n// Ensure we can sync the real WAL to the shadow WAL.\nfunc TestDB_Sync(t *testing.T) {\n\t// Ensure sync is skipped if no database exists.\n\tt.Run(\"NoDB\", func(t *testing.T) {\n\t\tdb := testingutil.MustOpenDB(t)\n\t\tdefer testingutil.MustCloseDB(t, db)\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n\n\t// Ensure sync can successfully run on the initial sync.\n\tt.Run(\"Initial\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Verify page size if now available.\n\t\tif db.PageSize() == 0 {\n\t\t\tt.Fatal(\"expected page size after initial sync\")\n\t\t}\n\n\t\t// Obtain real WAL size.\n\t\tfi, err := os.Stat(db.WALPath())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if fi.Size() == 0 {\n\t\t\tt.Fatal(\"expected wal\")\n\t\t}\n\n\t\t// Ensure position now available.\n\t\tif pos, err := db.Pos(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pos.TXID, ltx.TXID(1); got != want {\n\t\t\tt.Fatalf(\"pos.Index=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\t// Ensure DB can keep in sync across multiple Sync() invocations.\n\tt.Run(\"MultiSync\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t// Execute a query to force a write to the WAL.\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE foo (bar TEXT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Perform initial sync & grab initial position.\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tpos0, err := db.Pos()\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Insert into table.\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO foo (bar) VALUES ('baz');`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Sync to ensure position moves forward one page.\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if pos1, err := db.Pos(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pos1.TXID, pos0.TXID+1; got != want {\n\t\t\tt.Fatalf(\"TXID=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\t// Ensure a WAL file is created if one does not already exist.\n\tt.Run(\"NoWAL\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t// Issue initial sync and truncate WAL.\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Obtain initial position.\n\t\tif _, err := db.Pos(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Checkpoint & fully close which should close WAL file.\n\t\tif err := db.Checkpoint(context.Background(), litestream.CheckpointModeTruncate); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Remove WAL file.\n\t\tif err := os.Remove(db.WALPath()); err != nil && !os.IsNotExist(err) {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Reopen the managed database.\n\t\tdb = testingutil.MustOpenDBAt(t, db.Path())\n\t\tdefer testingutil.MustCloseDB(t, db)\n\n\t\t// Re-sync and ensure new generation has been created.\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Obtain initial position.\n\t\tif _, err := db.Pos(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n\n\t// Ensure DB can start new generation if it detects it cannot verify last position.\n\tt.Run(\"OverwritePrevPosition\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t// Execute a query to force a write to the WAL.\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE foo (bar TEXT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Issue initial sync and truncate WAL.\n\t\tif err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Obtain initial position.\n\t\tif _, err := db.Pos(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Fully close which should close WAL file.\n\t\tif err := db.Close(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Verify WAL does not exist.\n\t\tif _, err := os.Stat(db.WALPath()); !os.IsNotExist(err) {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Insert into table multiple times to move past old offset\n\t\tsqldb = testingutil.MustOpenSQLDB(t, db.Path())\n\t\tdefer testingutil.MustCloseSQLDB(t, sqldb)\n\t\tfor i := 0; i < 100; i++ {\n\t\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO foo (bar) VALUES ('baz');`); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\n\t\t// Reopen the managed database.\n\t\tdb = testingutil.MustOpenDBAt(t, db.Path())\n\t\tdefer testingutil.MustCloseDB(t, db)\n\n\t\t// Re-sync and ensure new generation has been created.\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Obtain initial position.\n\t\tif _, err := db.Pos(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n\n\t// Ensure DB checkpoints after minimum number of pages.\n\tt.Run(\"MinCheckpointPageN\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t// Execute a query to force a write to the WAL and then sync.\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE foo (bar TEXT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Write at least minimum number of pages to trigger rollover.\n\t\tfor i := 0; i < db.MinCheckpointPageN; i++ {\n\t\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO foo (bar) VALUES ('baz');`); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\n\t\t// Sync to shadow WAL. This should trigger a PASSIVE checkpoint because\n\t\t// we've exceeded MinCheckpointPageN threshold.\n\t\tif err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Ensure position is now on the third index (TXID 1 = initial,\n\t\t// TXID 2 = after inserts, TXID 3 = after PASSIVE checkpoint).\n\t\tif pos, err := db.Pos(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pos.TXID, ltx.TXID(3); got != want {\n\t\t\tt.Fatalf(\"Index=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\t// Ensure DB forces a truncate checkpoint once WAL exceeds the threshold.\n\tt.Run(\"TruncatePageN\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\t\tdb.TruncatePageN = 1\n\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE foo (bar TEXT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tpayloadSize := db.PageSize()\n\t\tif payloadSize == 0 {\n\t\t\tpayloadSize = 4096\n\t\t}\n\t\tpayload := strings.Repeat(\"x\", payloadSize)\n\n\t\t// Grow the WAL until we have more than one full page worth of changes.\n\t\tfor walPageCountForTest(t, db) <= 1 {\n\t\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO foo (bar) VALUES (?);`, payload); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\n\t\tif err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif got := walPageCountForTest(t, db); got > 1 {\n\t\t\tt.Fatalf(\"expected truncate checkpoint to shrink wal, pages=%d\", got)\n\t\t}\n\t})\n\n\t// Ensure DB checkpoints after interval.\n\tt.Run(\"CheckpointInterval\", func(t *testing.T) {\n\t\tt.Skip(\"TODO(ltx)\")\n\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t// Execute a query to force a write to the WAL and then sync.\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE foo (bar TEXT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Reduce checkpoint interval to ensure a rollover is triggered.\n\t\tdb.CheckpointInterval = 1 * time.Nanosecond\n\n\t\t// Write to WAL & sync.\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO foo (bar) VALUES ('baz');`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Ensure position is now on the second index.\n\t\tif pos, err := db.Pos(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pos.TXID, ltx.TXID(1); got != want {\n\t\t\tt.Fatalf(\"Index=%v, want %v\", got, want)\n\t\t}\n\t})\n}\n\nfunc TestDB_Compact(t *testing.T) {\n\t// Ensure that raw L0 transactions can be compacted into the first level.\n\tt.Run(\"L1\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (100)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := db.Replica.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tinfo, err := db.Compact(t.Context(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif got, want := info.Level, 1; got != want {\n\t\t\tt.Fatalf(\"Level=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := info.MinTXID, ltx.TXID(1); got != want {\n\t\t\tt.Fatalf(\"MinTXID=%s, want %s\", got, want)\n\t\t}\n\t\tif got, want := info.MaxTXID, ltx.TXID(2); got != want {\n\t\t\tt.Fatalf(\"MaxTXID=%s, want %s\", got, want)\n\t\t}\n\t\tif info.Size == 0 {\n\t\t\tt.Fatalf(\"expected non-zero size\")\n\t\t}\n\t})\n\n\t// Ensure that higher level compactions pull from the correct levels.\n\tt.Run(\"L2+\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// TXID 2\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (100)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := db.Replica.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Compact to L1:1-2\n\t\tif info, err := db.Compact(t.Context(), 1); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := ltx.FormatFilename(info.MinTXID, info.MaxTXID), `0000000000000001-0000000000000002.ltx`; got != want {\n\t\t\tt.Fatalf(\"Filename=%s, want %s\", got, want)\n\t\t}\n\n\t\t// TXID 3\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (100)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := db.Replica.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Compact to L1:3-3\n\t\tif info, err := db.Compact(t.Context(), 1); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := ltx.FormatFilename(info.MinTXID, info.MaxTXID), `0000000000000003-0000000000000003.ltx`; got != want {\n\t\t\tt.Fatalf(\"Filename=%s, want %s\", got, want)\n\t\t}\n\n\t\t// Compact to L2:1-3\n\t\tif info, err := db.Compact(t.Context(), 2); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := info.Level, 2; got != want {\n\t\t\tt.Fatalf(\"Level=%v, want %v\", got, want)\n\t\t} else if got, want := ltx.FormatFilename(info.MinTXID, info.MaxTXID), `0000000000000001-0000000000000003.ltx`; got != want {\n\t\t\tt.Fatalf(\"Filename=%s, want %s\", got, want)\n\t\t}\n\t})\n}\n\nfunc walPageCountForTest(tb testing.TB, db *litestream.DB) int64 {\n\ttb.Helper()\n\n\tfi, err := os.Stat(db.WALPath())\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\treturn 0\n\t\t}\n\t\ttb.Fatalf(\"stat wal: %v\", err)\n\t}\n\n\tpageSize := db.PageSize()\n\tif pageSize <= 0 || fi.Size() <= litestream.WALHeaderSize {\n\t\treturn 0\n\t}\n\n\tframeSize := int64(litestream.WALFrameHeaderSize + pageSize)\n\treturn (fi.Size() - litestream.WALHeaderSize) / frameSize\n}\n\nfunc TestDB_Snapshot(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT);`); err != nil {\n\t\tt.Fatal(err)\n\t} else if err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (100)`); err != nil {\n\t\tt.Fatal(err)\n\t} else if err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tinfo, err := db.Snapshot(context.Background())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif got, want := ltx.FormatFilename(info.MinTXID, info.MaxTXID), `0000000000000001-0000000000000002.ltx`; got != want {\n\t\tt.Fatalf(\"Filename=%s, want %s\", got, want)\n\t}\n\n\t// Calculate local checksum\n\tchksum0, _, err := db.CRC64(t.Context())\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Fetch remote LTX snapshot file and ensure it matches the checksum of the local database.\n\trc, err := db.Replica.Client.OpenLTXFile(t.Context(), litestream.SnapshotLevel, 1, 2, 0, 0)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer rc.Close()\n\n\th := crc64.New(crc64.MakeTable(crc64.ISO))\n\tif err := ltx.NewDecoder(rc).DecodeDatabaseTo(h); err != nil {\n\t\tt.Fatal(err)\n\t} else if got, want := h.Sum64(), chksum0; got != want {\n\t\tt.Fatal(\"snapshot checksum mismatch\")\n\t}\n}\n\nfunc TestDB_EnforceRetention(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Create table and sync initial state\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT);`); err != nil {\n\t\tt.Fatal(err)\n\t} else if err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Create multiple snapshots with delays to test retention\n\tfor i := 0; i < 3; i++ {\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (?)`, i); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif _, err := db.Snapshot(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Sleep between snapshots to create time differences\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\n\t// Get list of snapshots before retention\n\titr, err := db.Replica.Client.LTXFiles(t.Context(), litestream.SnapshotLevel, 0, false)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tvar beforeCount int\n\tfor itr.Next() {\n\t\tbeforeCount++\n\t}\n\titr.Close()\n\n\tif beforeCount != 3 {\n\t\tt.Fatalf(\"expected 3 snapshots before retention, got %d\", beforeCount)\n\t}\n\n\t// Enforce retention to remove older snapshots\n\tretentionTime := time.Now().Add(-150 * time.Millisecond)\n\tif minSnapshotTXID, err := db.EnforceSnapshotRetention(t.Context(), retentionTime); err != nil {\n\t\tt.Fatal(err)\n\t} else if got, want := minSnapshotTXID, ltx.TXID(4); got != want {\n\t\tt.Fatalf(\"MinSnapshotTXID=%s, want %s\", got, want)\n\t}\n\n\t// Verify snapshots after retention\n\titr, err = db.Replica.Client.LTXFiles(t.Context(), litestream.SnapshotLevel, 0, false)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tvar afterCount int\n\tfor itr.Next() {\n\t\tafterCount++\n\t}\n\titr.Close()\n\n\t// Should have at least one snapshot remaining\n\tif afterCount < 1 {\n\t\tt.Fatal(\"expected at least 1 snapshot after retention\")\n\t}\n\n\t// Should have fewer snapshots than before\n\tif afterCount >= beforeCount {\n\t\tt.Fatalf(\"expected fewer snapshots after retention, before=%d after=%d\", beforeCount, afterCount)\n\t}\n}\n\nfunc TestDB_EnforceSnapshotRetention_RetentionDisabled(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Disable retention (let cloud provider handle it).\n\tdb.RetentionEnabled = false\n\n\t// Create table and sync initial state.\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT);`); err != nil {\n\t\tt.Fatal(err)\n\t} else if err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Create multiple snapshots with delays.\n\tfor i := 0; i < 3; i++ {\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (?)`, i); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := db.Snapshot(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\n\t// Count snapshots before retention.\n\tcountFiles := func() int {\n\t\tt.Helper()\n\t\titr, err := db.Replica.Client.LTXFiles(t.Context(), litestream.SnapshotLevel, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tvar n int\n\t\tfor itr.Next() {\n\t\t\tn++\n\t\t}\n\t\titr.Close()\n\t\treturn n\n\t}\n\n\tbeforeCount := countFiles()\n\tif beforeCount != 3 {\n\t\tt.Fatalf(\"expected 3 snapshots before retention, got %d\", beforeCount)\n\t}\n\n\t// Enforce retention with skip remote deletion enabled.\n\tretentionTime := time.Now().Add(-150 * time.Millisecond)\n\tif _, err := db.EnforceSnapshotRetention(t.Context(), retentionTime); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Remote files should all still exist.\n\tafterCount := countFiles()\n\tif afterCount != beforeCount {\n\t\tt.Fatalf(\"expected %d remote snapshots (no remote deletion), got %d\", beforeCount, afterCount)\n\t}\n}\n\nfunc TestDB_EnforceL0RetentionByTime_RetentionDisabled(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Disable retention and set a short L0 retention.\n\tdb.RetentionEnabled = false\n\tdb.L0Retention = time.Nanosecond\n\n\t// Create table and sync to create L0 files.\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT);`); err != nil {\n\t\tt.Fatal(err)\n\t} else if err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tfor i := 0; i < 3; i++ {\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (?)`, i); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\t// Sync replica to upload L0 files to remote storage.\n\tif err := db.Replica.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Count L0 files before.\n\tcountL0 := func() int {\n\t\tt.Helper()\n\t\titr, err := db.Replica.Client.LTXFiles(t.Context(), 0, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tvar n int\n\t\tfor itr.Next() {\n\t\t\tn++\n\t\t}\n\t\titr.Close()\n\t\treturn n\n\t}\n\n\tbeforeCount := countL0()\n\tif beforeCount < 2 {\n\t\tt.Fatalf(\"expected at least 2 L0 files, got %d\", beforeCount)\n\t}\n\n\t// Compact L0 to L1 so files become eligible for L0 retention.\n\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.DefaultCompactionLevels)\n\tif _, err := store.CompactDB(t.Context(), db, &litestream.CompactionLevel{Level: 1, Interval: time.Nanosecond}); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait a moment for files to become old enough.\n\ttime.Sleep(10 * time.Millisecond)\n\n\t// Enforce L0 retention.\n\tif err := db.EnforceL0RetentionByTime(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Remote L0 files should all still exist.\n\tafterCount := countL0()\n\tif afterCount != beforeCount {\n\t\tt.Fatalf(\"expected %d remote L0 files (no remote deletion), got %d\", beforeCount, afterCount)\n\t}\n}\n\n// TestDB_ConcurrentMapWrite tests for race conditions in maxLTXFileInfos map access.\n// This test specifically targets the concurrent map write issue found in db.go\n// where sync() method writes to the map without proper locking.\n// Run with: go test -race -run TestDB_ConcurrentMapWrite\nfunc TestDB_ConcurrentMapWrite(t *testing.T) {\n\t// Use the standard test helpers\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Enable monitoring to trigger background operations\n\tdb.MonitorInterval = 10 * time.Millisecond\n\n\t// Create a table\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INTEGER PRIMARY KEY, value TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Start multiple goroutines to trigger concurrent map access\n\tvar wg sync.WaitGroup\n\n\t// Number of concurrent operations\n\tconst numGoroutines = 10\n\n\t// Channel to signal start\n\tstart := make(chan struct{})\n\n\tfor i := 0; i < numGoroutines; i++ {\n\t\twg.Add(1)\n\t\tgo func(id int) {\n\t\t\tdefer wg.Done()\n\n\t\t\t// Wait for signal to start all goroutines simultaneously\n\t\t\t<-start\n\n\t\t\t// Perform operations that trigger map access\n\t\t\tfor j := 0; j < 5; j++ {\n\t\t\t\t// This triggers sync() which had unprotected map access\n\t\t\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (value) VALUES (?)`, \"test\"); err != nil {\n\t\t\t\t\tt.Logf(\"Goroutine %d: insert error: %v\", id, err)\n\t\t\t\t}\n\n\t\t\t\t// Trigger Sync manually which accesses the map\n\t\t\t\tif err := db.Sync(t.Context()); err != nil {\n\t\t\t\t\tt.Logf(\"Goroutine %d: sync error: %v\", id, err)\n\t\t\t\t}\n\n\t\t\t\t// Small delay to allow race to manifest\n\t\t\t\ttime.Sleep(time.Millisecond)\n\t\t\t}\n\t\t}(i)\n\t}\n\n\t// Additional goroutine for snapshot operations\n\twg.Add(1)\n\tgo func() {\n\t\tdefer wg.Done()\n\t\t<-start\n\n\t\tfor i := 0; i < 3; i++ {\n\t\t\t// This triggers Snapshot() which has protected map access\n\t\t\tif _, err := db.Snapshot(t.Context()); err != nil {\n\t\t\t\tt.Logf(\"Snapshot error: %v\", err)\n\t\t\t}\n\t\t\ttime.Sleep(5 * time.Millisecond)\n\t\t}\n\t}()\n\n\t// Start all goroutines\n\tclose(start)\n\n\t// Wait for completion\n\twg.Wait()\n\n\tt.Log(\"Test completed without race condition\")\n}\n\n// TestCompaction_PreservesLastTimestamp verifies that after compaction,\n// the resulting file's timestamp reflects the last source file timestamp\n// as recorded in the LTX headers. This ensures point-in-time restoration\n// continues to work after compaction (issue #771).\nfunc TestCompaction_PreservesLastTimestamp(t *testing.T) {\n\tctx := context.Background()\n\n\tdir := t.TempDir()\n\tdb := testingutil.NewDB(t, filepath.Join(dir, \"db\"))\n\tdb.MonitorInterval = 0\n\tdb.ShutdownSyncTimeout = 0\n\treplicaPath := filepath.Join(dir, \"replica\")\n\tclient := file.NewReplicaClient(replicaPath)\n\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tsqldb := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Create some transactions\n\tfor i := 0; i < 10; i++ {\n\t\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS t (id INTEGER PRIMARY KEY, val TEXT)`); err != nil {\n\t\t\tt.Fatalf(\"create table: %v\", err)\n\t\t}\n\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO t (val) VALUES (?)`, fmt.Sprintf(\"value-%d\", i)); err != nil {\n\t\t\tt.Fatalf(\"insert %d: %v\", i, err)\n\t\t}\n\n\t\t// Sync to create L0 files\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync db: %v\", err)\n\t\t}\n\t\tif err := db.Replica.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync replica: %v\", err)\n\t\t}\n\t}\n\n\t// Record the last L0 file timestamp before compaction\n\titr, err := client.LTXFiles(ctx, 0, 0, false)\n\tif err != nil {\n\t\tt.Fatalf(\"list L0 files: %v\", err)\n\t}\n\tdefer itr.Close()\n\n\tl0Files, err := ltx.SliceFileIterator(itr)\n\tif err != nil {\n\t\tt.Fatalf(\"convert iterator: %v\", err)\n\t}\n\tif err := itr.Close(); err != nil {\n\t\tt.Fatalf(\"close iterator: %v\", err)\n\t}\n\n\tvar lastTime time.Time\n\tfor _, info := range l0Files {\n\t\tif lastTime.IsZero() || info.CreatedAt.After(lastTime) {\n\t\t\tlastTime = info.CreatedAt\n\t\t}\n\t}\n\n\tif len(l0Files) == 0 {\n\t\tt.Fatal(\"expected L0 files before compaction\")\n\t}\n\tt.Logf(\"Found %d L0 files, last timestamp: %v\", len(l0Files), lastTime)\n\n\t// Perform compaction from L0 to L1\n\tlevels := litestream.CompactionLevels{\n\t\t{Level: 0},\n\t\t{Level: 1, Interval: time.Second},\n\t}\n\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\tstore.CompactionMonitorEnabled = false\n\n\tif err := store.Open(ctx); err != nil {\n\t\tt.Fatalf(\"open store: %v\", err)\n\t}\n\tdefer func() {\n\t\tif err := store.Close(ctx); err != nil {\n\t\t\tt.Fatalf(\"close store: %v\", err)\n\t\t}\n\t}()\n\n\t_, err = store.CompactDB(ctx, db, levels[1])\n\tif err != nil {\n\t\tt.Fatalf(\"compact: %v\", err)\n\t}\n\n\t// Verify L1 file has the last timestamp from L0 files\n\titr, err = client.LTXFiles(ctx, 1, 0, false)\n\tif err != nil {\n\t\tt.Fatalf(\"list L1 files: %v\", err)\n\t}\n\tdefer itr.Close()\n\n\tl1Files, err := ltx.SliceFileIterator(itr)\n\tif err != nil {\n\t\tt.Fatalf(\"convert L1 iterator: %v\", err)\n\t}\n\tif err := itr.Close(); err != nil {\n\t\tt.Fatalf(\"close L1 iterator: %v\", err)\n\t}\n\n\tif len(l1Files) == 0 {\n\t\tt.Fatal(\"expected L1 file after compaction\")\n\t}\n\n\tl1Info := l1Files[0]\n\n\t// The L1 file's CreatedAt should be the last timestamp from the L0 files\n\t// Allow for some drift due to millisecond precision in LTX headers\n\ttimeDiff := l1Info.CreatedAt.Sub(lastTime)\n\tif timeDiff.Abs() > time.Second {\n\t\tt.Errorf(\"L1 CreatedAt = %v, last L0 = %v (diff: %v)\", l1Info.CreatedAt, lastTime, timeDiff)\n\t\tt.Error(\"L1 file timestamp should preserve last source file timestamp\")\n\t}\n}\n\nfunc TestDB_EnforceRetentionByTXID_LocalCleanup(t *testing.T) {\n\tctx := context.Background()\n\n\tdir := t.TempDir()\n\tdb := testingutil.NewDB(t, filepath.Join(dir, \"db\"))\n\tdb.MonitorInterval = 0\n\tdb.ShutdownSyncTimeout = 0\n\treplicaPath := filepath.Join(dir, \"replica\")\n\tclient := file.NewReplicaClient(replicaPath)\n\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tsqldb := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE t (id INTEGER PRIMARY KEY, val TEXT)`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\ttype localFile struct {\n\t\tpath string\n\t\tminTXID ltx.TXID\n\t\tmaxTXID ltx.TXID\n\t}\n\tvar firstBatchL0Files []localFile\n\n\tfor i := 0; i < 3; i++ {\n\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO t (val) VALUES (?)`, fmt.Sprintf(\"batch1-value-%d\", i)); err != nil {\n\t\t\tt.Fatalf(\"insert batch1 %d: %v\", i, err)\n\t\t}\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync db batch1 %d: %v\", i, err)\n\t\t}\n\n\t\tminTXID, maxTXID, err := db.MaxLTX()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"get max ltx: %v\", err)\n\t\t}\n\t\tlocalPath := db.LTXPath(0, minTXID, maxTXID)\n\t\tfirstBatchL0Files = append(firstBatchL0Files, localFile{\n\t\t\tpath: localPath,\n\t\t\tminTXID: minTXID,\n\t\t\tmaxTXID: maxTXID,\n\t\t})\n\n\t\tif err := db.Replica.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync replica batch1 %d: %v\", i, err)\n\t\t}\n\t}\n\n\tfor _, lf := range firstBatchL0Files {\n\t\tif _, err := os.Stat(lf.path); os.IsNotExist(err) {\n\t\t\tt.Fatalf(\"local L0 file should exist before first compaction: %s\", lf.path)\n\t\t}\n\t}\n\n\tif _, err := db.Compact(ctx, 1); err != nil {\n\t\tt.Fatalf(\"compact batch1 to L1: %v\", err)\n\t}\n\n\tfor i := 0; i < 3; i++ {\n\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO t (val) VALUES (?)`, fmt.Sprintf(\"batch2-value-%d\", i)); err != nil {\n\t\t\tt.Fatalf(\"insert batch2 %d: %v\", i, err)\n\t\t}\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync db batch2 %d: %v\", i, err)\n\t\t}\n\t\tif err := db.Replica.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync replica batch2 %d: %v\", i, err)\n\t\t}\n\t}\n\n\tsecondCompactInfo, err := db.Compact(ctx, 1)\n\tif err != nil {\n\t\tt.Fatalf(\"compact batch2 to L1: %v\", err)\n\t}\n\n\tif err := db.EnforceRetentionByTXID(ctx, 0, secondCompactInfo.MinTXID); err != nil {\n\t\tt.Fatalf(\"enforce retention: %v\", err)\n\t}\n\n\tfor _, lf := range firstBatchL0Files {\n\t\tif lf.maxTXID < secondCompactInfo.MinTXID {\n\t\t\tif _, err := os.Stat(lf.path); err == nil {\n\t\t\t\tt.Errorf(\"local L0 file should be removed after second compaction: %s (maxTXID=%s < minTXID=%s)\",\n\t\t\t\t\tlf.path, lf.maxTXID, secondCompactInfo.MinTXID)\n\t\t\t} else if !os.IsNotExist(err) {\n\t\t\t\tt.Fatalf(\"unexpected error checking local file: %v\", err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestDB_EnforceL0RetentionByTime(t *testing.T) {\n\tctx := context.Background()\n\n\tdir := t.TempDir()\n\tdb := testingutil.NewDB(t, filepath.Join(dir, \"db\"))\n\tdb.MonitorInterval = 0\n\tdb.ShutdownSyncTimeout = 0\n\treplicaPath := filepath.Join(dir, \"replica\")\n\tclient := file.NewReplicaClient(replicaPath)\n\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\tdb.Replica.MonitorEnabled = false\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tsqldb := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Use a long retention initially so compaction does not immediately clean up files.\n\tdb.L0Retention = 30 * time.Minute\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE t (id INTEGER PRIMARY KEY, val TEXT)`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\tfor i := 0; i < 3; i++ {\n\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO t (val) VALUES (?)`, fmt.Sprintf(\"value-%d\", i)); err != nil {\n\t\t\tt.Fatalf(\"insert %d: %v\", i, err)\n\t\t}\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync db %d: %v\", i, err)\n\t\t}\n\t\tif err := db.Replica.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync replica %d: %v\", i, err)\n\t\t}\n\t}\n\n\tif _, err := db.Compact(ctx, 1); err != nil {\n\t\tt.Fatalf(\"compact L0 -> L1: %v\", err)\n\t}\n\n\titr, err := client.LTXFiles(ctx, 0, 0, false)\n\tif err != nil {\n\t\tt.Fatalf(\"list L0 files: %v\", err)\n\t}\n\tl0Files, err := ltx.SliceFileIterator(itr)\n\tif err != nil {\n\t\tt.Fatalf(\"slice iterator: %v\", err)\n\t}\n\tif err := itr.Close(); err != nil {\n\t\tt.Fatalf(\"close iterator: %v\", err)\n\t}\n\tif len(l0Files) < 2 {\n\t\tt.Fatalf(\"expected at least two L0 files, got %d\", len(l0Files))\n\t}\n\n\tcheckExists := func(expectMissing bool) {\n\t\tfor idx, info := range l0Files {\n\t\t\tremotePath := client.LTXFilePath(0, info.MinTXID, info.MaxTXID)\n\t\t\tlocalPath := db.LTXPath(0, info.MinTXID, info.MaxTXID)\n\t\t\t_, remoteErr := os.Stat(remotePath)\n\t\t\t_, localErr := os.Stat(localPath)\n\t\t\tif expectMissing && idx < len(l0Files)-1 {\n\t\t\t\tif !os.IsNotExist(remoteErr) {\n\t\t\t\t\tt.Fatalf(\"expected remote file removed: %s\", remotePath)\n\t\t\t\t}\n\t\t\t\tif !os.IsNotExist(localErr) {\n\t\t\t\t\tt.Fatalf(\"expected local file removed: %s\", localPath)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif !expectMissing || idx == len(l0Files)-1 {\n\t\t\t\tif remoteErr != nil {\n\t\t\t\t\tt.Fatalf(\"expected remote file to exist: %s (%v)\", remotePath, remoteErr)\n\t\t\t\t}\n\t\t\t\tif localErr != nil {\n\t\t\t\t\tt.Fatalf(\"expected local file to exist: %s (%v)\", localPath, localErr)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// Files should still exist immediately after compaction since they are new.\n\tif err := db.EnforceL0RetentionByTime(ctx); err != nil {\n\t\tt.Fatalf(\"enforce recent retention: %v\", err)\n\t}\n\tcheckExists(false)\n\n\t// Age the files so they exceed the retention threshold.\n\toldTime := time.Now().Add(-1 * time.Hour)\n\tfor _, info := range l0Files {\n\t\tremotePath := client.LTXFilePath(0, info.MinTXID, info.MaxTXID)\n\t\tif err := os.Chtimes(remotePath, oldTime, oldTime); err != nil {\n\t\t\tt.Fatalf(\"chtimes remote: %v\", err)\n\t\t}\n\t\tlocalPath := db.LTXPath(0, info.MinTXID, info.MaxTXID)\n\t\tif err := os.Chtimes(localPath, oldTime, oldTime); err != nil {\n\t\t\tt.Fatalf(\"chtimes local: %v\", err)\n\t\t}\n\t}\n\n\t// Shorten retention so aged files qualify for deletion.\n\tdb.L0Retention = time.Second\n\tif err := db.EnforceL0RetentionByTime(ctx); err != nil {\n\t\tt.Fatalf(\"enforce aged retention: %v\", err)\n\t}\n\tcheckExists(true)\n}\n\n// TestDB_SyncAfterVacuum verifies that syncing works correctly after a database\n// shrinks via VACUUM. This tests the fix for issue #875 where page numbers from\n// earlier transactions in the WAL could exceed the new commit size after shrinking.\nfunc TestDB_SyncAfterVacuum(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Create a table and insert enough data to create multiple pages\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INTEGER PRIMARY KEY, data BLOB)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Insert enough rows to create many pages\n\tfor i := 0; i < 100; i++ {\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (data) VALUES (?)`, strings.Repeat(\"x\", 4000)); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\t// Initial sync\n\tif err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Get initial page count\n\tvar initialPageCount int\n\tif err := sqldb.QueryRowContext(t.Context(), `PRAGMA page_count`).Scan(&initialPageCount); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"Initial page count: %d\", initialPageCount)\n\n\t// Delete most data and VACUUM to shrink the database\n\tif _, err := sqldb.ExecContext(t.Context(), `DELETE FROM t WHERE id > 10`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.ExecContext(t.Context(), `VACUUM`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Get new page count\n\tvar newPageCount int\n\tif err := sqldb.QueryRowContext(t.Context(), `PRAGMA page_count`).Scan(&newPageCount); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"Page count after VACUUM: %d\", newPageCount)\n\n\tif newPageCount >= initialPageCount {\n\t\tt.Skip(\"VACUUM did not shrink database, skipping test\")\n\t}\n\n\t// This sync should succeed without \"page number out-of-bounds\" error\n\tif err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatalf(\"sync after VACUUM failed: %v\", err)\n\t}\n\n\t// Verify position advanced\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif pos.TXID < 2 {\n\t\tt.Fatalf(\"expected TXID >= 2, got %d\", pos.TXID)\n\t}\n\tt.Logf(\"Final position: TXID=%d\", pos.TXID)\n}\n\n// TestDB_NoLTXFilesOnIdleSync verifies that syncing an idle database does not\n// create new LTX files when no external changes have been made. This tests the\n// fix for issue #896 where time-based checkpoints were creating LTX files even\n// when no actual database changes occurred.\nfunc TestDB_NoLTXFilesOnIdleSync(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Set CheckpointInterval to trigger time-based checkpoints\n\tdb.CheckpointInterval = time.Millisecond\n\n\t// Create a table and insert some data to ensure we have WAL activity\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INTEGER PRIMARY KEY, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (data) VALUES ('test')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Initial sync to create first LTX file(s)\n\tif err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for checkpoint interval to pass\n\ttime.Sleep(10 * time.Millisecond)\n\n\t// Sync again to trigger checkpoint (this will write to _litestream_seq)\n\tif err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Record the current TXID after checkpoint\n\tposAfterCheckpoint, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"TXID after checkpoint: %d\", posAfterCheckpoint.TXID)\n\n\t// Wait for checkpoint interval to pass again\n\ttime.Sleep(10 * time.Millisecond)\n\n\t// Now sync multiple times without any external database changes\n\t// This is the key part of the test - with the bug, each sync would create\n\t// a new LTX file because the time-based checkpoint would trigger\n\tfor i := 0; i < 3; i++ {\n\t\tif err := db.Sync(t.Context()); err != nil {\n\t\t\tt.Fatalf(\"sync %d failed: %v\", i, err)\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\n\t// Check final position - it should NOT have advanced significantly\n\t// With the bug, TXID would increase by 3 (one for each sync)\n\tposAfterIdle, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"TXID after idle syncs: %d\", posAfterIdle.TXID)\n\n\t// The TXID should not have advanced more than 1 from the checkpoint\n\t// (accounting for the checkpoint's own _litestream_seq write)\n\tif posAfterIdle.TXID > posAfterCheckpoint.TXID+1 {\n\t\tt.Fatalf(\"expected TXID to stay at or below %d, got %d (bug: LTX files created without changes)\",\n\t\t\tposAfterCheckpoint.TXID+1, posAfterIdle.TXID)\n\t}\n}\n\n// TestDB_DelayedCheckpointAfterWrite verifies that writes that happen before\n// the checkpoint interval elapses will still trigger a checkpoint later when\n// the interval does elapse. This ensures the syncedSinceCheckpoint flag\n// persists across sync calls. See issue #896.\nfunc TestDB_DelayedCheckpointAfterWrite(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Use a longer checkpoint interval so we can control when it triggers\n\tdb.CheckpointInterval = 100 * time.Millisecond\n\n\t// Create table and initial sync\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INTEGER PRIMARY KEY, data TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for interval to pass and sync to trigger initial checkpoint\n\ttime.Sleep(150 * time.Millisecond)\n\tif err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Record TXID after first checkpoint\n\tposAfterFirstCheckpoint, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"TXID after first checkpoint: %d\", posAfterFirstCheckpoint.TXID)\n\n\t// Insert data immediately (before interval elapses)\n\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (data) VALUES ('delayed checkpoint test')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sync immediately - this should NOT trigger a checkpoint (interval hasn't elapsed)\n\t// but should set syncedSinceCheckpoint = true\n\tif err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tposAfterInsert, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"TXID after insert+sync: %d\", posAfterInsert.TXID)\n\n\t// Now wait for the interval to pass and sync again (no new data)\n\ttime.Sleep(150 * time.Millisecond)\n\tif err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// A checkpoint should have been triggered because syncedSinceCheckpoint was true\n\t// The TXID should have advanced due to the checkpoint\n\tposAfterDelayedCheckpoint, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Logf(\"TXID after delayed checkpoint: %d\", posAfterDelayedCheckpoint.TXID)\n\n\t// The TXID should have advanced from the insert position, indicating the checkpoint ran\n\tif posAfterDelayedCheckpoint.TXID <= posAfterInsert.TXID {\n\t\tt.Fatalf(\"expected TXID to advance after delayed checkpoint (syncedSinceCheckpoint should persist), got insert=%d delayed=%d\",\n\t\t\tposAfterInsert.TXID, posAfterDelayedCheckpoint.TXID)\n\t}\n}\n\nfunc TestDB_SyncStatus(t *testing.T) {\n\tt.Run(\"NoReplica\", func(t *testing.T) {\n\t\tdb := litestream.NewDB(filepath.Join(t.TempDir(), \"db\"))\n\t\tdb.Replica = nil\n\t\tif _, err := db.SyncStatus(context.Background()); err == nil {\n\t\t\tt.Fatal(\"expected error\")\n\t\t}\n\t})\n\n\tt.Run(\"BeforeSync\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tstatus, err := db.SyncStatus(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif status.LocalTXID != 0 {\n\t\t\tt.Fatalf(\"expected LocalTXID=0, got %d\", status.LocalTXID)\n\t\t}\n\t\tif status.RemoteTXID != 0 {\n\t\t\tt.Fatalf(\"expected RemoteTXID=0, got %d\", status.RemoteTXID)\n\t\t}\n\t\tif status.InSync {\n\t\t\tt.Fatal(\"expected InSync=false before any sync\")\n\t\t}\n\t})\n\n\tt.Run(\"AfterDBSyncOnly\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tstatus, err := db.SyncStatus(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif status.LocalTXID == 0 {\n\t\t\tt.Fatal(\"expected non-zero LocalTXID after db sync\")\n\t\t}\n\t\tif status.RemoteTXID != 0 {\n\t\t\tt.Fatalf(\"expected RemoteTXID=0, got %d\", status.RemoteTXID)\n\t\t}\n\t\tif status.InSync {\n\t\t\tt.Fatal(\"expected InSync=false when remote has not synced\")\n\t\t}\n\t})\n\n\tt.Run(\"AfterFullSync\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Replica.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tstatus, err := db.SyncStatus(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif status.LocalTXID == 0 {\n\t\t\tt.Fatal(\"expected non-zero LocalTXID\")\n\t\t}\n\t\tif status.LocalTXID != status.RemoteTXID {\n\t\t\tt.Fatalf(\"expected LocalTXID=%d == RemoteTXID=%d\", status.LocalTXID, status.RemoteTXID)\n\t\t}\n\t\tif !status.InSync {\n\t\t\tt.Fatal(\"expected InSync=true after full sync\")\n\t\t}\n\t})\n\n\tt.Run(\"AfterNewWrites\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Replica.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (1)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tstatus, err := db.SyncStatus(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif status.LocalTXID <= status.RemoteTXID {\n\t\t\tt.Fatalf(\"expected LocalTXID=%d > RemoteTXID=%d\", status.LocalTXID, status.RemoteTXID)\n\t\t}\n\t\tif status.InSync {\n\t\t\tt.Fatal(\"expected InSync=false after new writes without replica sync\")\n\t\t}\n\t})\n\n\tt.Run(\"CancelledContext\", func(t *testing.T) {\n\t\tdb := litestream.NewDB(filepath.Join(t.TempDir(), \"db\"))\n\t\tclient := &mock.ReplicaClient{\n\t\t\tLTXFilesFunc: func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\t\treturn nil, ctx.Err()\n\t\t\t},\n\t\t}\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\n\t\tctx, cancel := context.WithCancel(context.Background())\n\t\tcancel()\n\n\t\t_, err := db.SyncStatus(ctx)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error with cancelled context\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"remote position\") {\n\t\t\tt.Fatalf(\"expected remote position error, got: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestDB_SyncAndWait(t *testing.T) {\n\tt.Run(\"NoReplica\", func(t *testing.T) {\n\t\tdb := litestream.NewDB(filepath.Join(t.TempDir(), \"db\"))\n\t\tdb.Replica = nil\n\t\tif err := db.SyncAndWait(context.Background()); err == nil {\n\t\t\tt.Fatal(\"expected error\")\n\t\t}\n\t})\n\n\tt.Run(\"OK\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (1)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := db.SyncAndWait(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tstatus, err := db.SyncStatus(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !status.InSync {\n\t\t\tt.Fatalf(\"expected InSync=true after SyncAndWait, LocalTXID=%d RemoteTXID=%d\", status.LocalTXID, status.RemoteTXID)\n\t\t}\n\t})\n}\n\nfunc TestDB_EnsureExists(t *testing.T) {\n\tt.Run(\"NoReplica\", func(t *testing.T) {\n\t\tdb := litestream.NewDB(filepath.Join(t.TempDir(), \"db\"))\n\t\tdb.Replica = nil\n\t\tif err := db.EnsureExists(context.Background()); err == nil {\n\t\t\tt.Fatal(\"expected error\")\n\t\t}\n\t})\n\n\tt.Run(\"DBAlreadyExists\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"db\")\n\n\t\tif err := os.WriteFile(dbPath, []byte(\"dummy\"), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tdb := litestream.NewDB(dbPath)\n\t\tclient := file.NewReplicaClient(filepath.Join(dir, \"replica\"))\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\n\t\tif err := db.EnsureExists(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tdata, err := os.ReadFile(dbPath)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif string(data) != \"dummy\" {\n\t\t\tt.Fatal(\"expected file to remain unchanged\")\n\t\t}\n\t})\n\n\tt.Run(\"NoBackup\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"db\")\n\n\t\tdb := testingutil.NewDB(t, dbPath)\n\t\tclient := file.NewReplicaClient(filepath.Join(dir, \"replica\"))\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\n\t\tif err := db.EnsureExists(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"expected nil error for no backup, got %v\", err)\n\t\t}\n\n\t\tif _, err := os.Stat(dbPath); !os.IsNotExist(err) {\n\t\t\tt.Fatal(\"expected database file to not exist when no backup available\")\n\t\t}\n\t})\n\n\tt.Run(\"MissingParentDir\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"subdir\", \"nested\", \"db\")\n\n\t\tdb := testingutil.NewDB(t, dbPath)\n\t\tclient := file.NewReplicaClient(filepath.Join(dir, \"replica\"))\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\n\t\tif err := db.EnsureExists(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"expected nil error, got %v\", err)\n\t\t}\n\n\t\tinfo, err := os.Stat(filepath.Join(dir, \"subdir\", \"nested\"))\n\t\tif err != nil {\n\t\t\tt.Fatal(\"expected parent directories to be created\")\n\t\t}\n\t\tif !info.IsDir() {\n\t\t\tt.Fatal(\"expected parent path to be a directory\")\n\t\t}\n\t})\n\n\tt.Run(\"RestoreFromBackup\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"db\")\n\t\treplicaPath := filepath.Join(dir, \"replica\")\n\n\t\tdb := testingutil.NewDB(t, dbPath)\n\t\tdb.MonitorInterval = 0\n\t\tdb.ShutdownSyncTimeout = 0\n\t\tclient := file.NewReplicaClient(replicaPath)\n\t\treplica := litestream.NewReplicaWithClient(db, client)\n\t\treplica.MonitorEnabled = false\n\t\tdb.Replica = replica\n\n\t\tif err := db.Open(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tsqldb := testingutil.MustOpenSQLDB(t, dbPath)\n\t\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE t (id INT, value TEXT)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO t (id, value) VALUES (1, 'hello')`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := db.SyncAndWait(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := sqldb.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db.Close(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := os.Remove(dbPath); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\twalPath := dbPath + \"-wal\"\n\t\tos.Remove(walPath)\n\n\t\tdb2 := testingutil.NewDB(t, dbPath)\n\t\tclient2 := file.NewReplicaClient(replicaPath)\n\t\tdb2.Replica = litestream.NewReplicaWithClient(db2, client2)\n\n\t\tif err := db2.EnsureExists(ctx); err != nil {\n\t\t\tt.Fatalf(\"EnsureExists: %v\", err)\n\t\t}\n\n\t\tif _, err := os.Stat(dbPath); os.IsNotExist(err) {\n\t\t\tt.Fatal(\"expected database file to be restored\")\n\t\t}\n\n\t\tsqldb2 := testingutil.MustOpenSQLDB(t, dbPath)\n\t\tdefer sqldb2.Close()\n\n\t\tvar value string\n\t\tif err := sqldb2.QueryRowContext(ctx, `SELECT value FROM t WHERE id = 1`).Scan(&value); err != nil {\n\t\t\tt.Fatalf(\"query restored db: %v\", err)\n\t\t}\n\t\tif value != \"hello\" {\n\t\t\tt.Fatalf(\"expected 'hello', got %q\", value)\n\t\t}\n\t})\n}\n\n// TestDB_ResetLocalState verifies that ResetLocalState clears the LTX directory.\nfunc TestDB_ResetLocalState(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Create table and insert some data to create LTX files\n\tif _, err := sqldb.Exec(`CREATE TABLE t (x TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`INSERT INTO t (x) VALUES ('foo')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Sync(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Verify LTX directory exists and has files\n\tltxDir := db.LTXDir()\n\tif _, err := os.Stat(ltxDir); os.IsNotExist(err) {\n\t\tt.Fatal(\"LTX directory should exist after sync\")\n\t}\n\n\t// Get position before reset\n\tposBefore, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif posBefore.TXID == 0 {\n\t\tt.Fatal(\"expected non-zero TXID before reset\")\n\t}\n\n\t// Reset local state\n\tif err := db.ResetLocalState(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Verify LTX directory is gone\n\tif _, err := os.Stat(ltxDir); !os.IsNotExist(err) {\n\t\tt.Fatal(\"LTX directory should not exist after reset\")\n\t}\n\n\t// Get position after reset - should be zero since no LTX files\n\tposAfter, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif posAfter.TXID != 0 {\n\t\tt.Fatalf(\"expected zero TXID after reset, got %d\", posAfter.TXID)\n\t}\n}\n" }, { "path": "docker-compose.test.yml", "content": "services:\n minio:\n image: minio/minio:latest\n ports:\n - \"9000:9000\"\n - \"9001:9001\"\n environment:\n MINIO_ROOT_USER: minioadmin\n MINIO_ROOT_PASSWORD: minioadmin\n command: server /data --console-address \":9001\"\n healthcheck:\n test: [\"CMD\", \"mc\", \"ready\", \"local\"]\n interval: 5s\n timeout: 5s\n retries: 5\n" }, { "path": "docs/ARCHITECTURE.md", "content": "# Litestream Architecture - Technical Deep Dive\n\n## Table of Contents\n- [System Layers](#system-layers)\n- [Core Components](#core-components)\n- [IPC Server & Control Socket](#ipc-server--control-socket)\n- [Distributed Leasing](#distributed-leasing)\n- [Library Convenience Methods](#library-convenience-methods)\n- [LTX File Format](#ltx-file-format)\n- [WAL Monitoring Mechanism](#wal-monitoring-mechanism)\n- [Compaction Process](#compaction-process)\n- [Transaction Management](#transaction-management)\n- [Concurrency Model](#concurrency-model)\n- [State Management](#state-management)\n- [Initialization Flow](#initialization-flow)\n- [Error Handling](#error-handling)\n\n## System Layers\n\nLitestream follows a layered architecture with clear separation of concerns:\n\n```mermaid\ngraph TB\n subgraph \"Application Layer\"\n CLI[CLI Commands
cmd/litestream/]\n Config[Configuration
config.go]\n end\n\n subgraph \"Core Layer\"\n Store[Store Manager
store.go]\n DB[Database Manager
db.go]\n Replica[Replica Manager
replica.go]\n end\n\n subgraph \"Storage Abstraction\"\n RC[ReplicaClient Interface
replica_client.go]\n end\n\n subgraph \"Storage Implementations\"\n S3[s3/replica_client.go]\n GCS[gs/replica_client.go]\n ABS[abs/replica_client.go]\n OSS[oss/replica_client.go]\n File[file/replica_client.go]\n SFTP[sftp/replica_client.go]\n NATS[nats/replica_client.go]\n end\n\n subgraph \"External\"\n SQLite[SQLite Database]\n Cloud[Cloud Storage]\n end\n\n CLI --> Store\n Store --> DB\n DB --> Replica\n Replica --> RC\n RC --> S3\n RC --> GCS\n RC --> ABS\n RC --> OSS\n RC --> File\n RC --> SFTP\n RC --> NATS\n DB <--> SQLite\n S3 --> Cloud\n GCS --> Cloud\n ABS --> Cloud\n OSS --> Cloud\n```\n\n### Layer Responsibilities\n\n#### 1. Application Layer\n- **CLI Commands**: User interface for operations (replicate, restore, etc.)\n- **Configuration**: YAML/environment variable parsing and validation\n\n#### 2. Core Layer\n- **Store**: Multi-database coordination, compaction scheduling\n- **DB**: Single database management, WAL monitoring, checkpointing\n- **Replica**: Replication to single destination, position tracking\n\n#### 3. Storage Abstraction\n- **ReplicaClient Interface**: Uniform API for all storage backends\n\n#### 4. Storage Implementations\n- Backend-specific logic (authentication, retries, optimizations)\n\n#### 5. IPC Layer\n- **Server**: Unix socket HTTP server for runtime control (`server.go`)\n- **Leaser**: Distributed lease interface for coordination (`leaser.go`, `s3/leaser.go`)\n\n## Core Components\n\n### DB Component (db.go)\n\nThe DB component is the heart of Litestream, managing a single SQLite database:\n\n```go\ntype DB struct {\n // Core fields\n path string // Database file path\n metaPath string // Metadata directory path\n db *sql.DB // SQLite connection\n f *os.File // Long-running file descriptor\n rtx *sql.Tx // Long-running read transaction\n pageSize int // Database page size\n\n // Synchronization\n mu sync.RWMutex // Protects struct fields\n chkMu sync.RWMutex // Checkpoint lock\n notify chan struct{} // WAL change notifications\n\n // Lifecycle\n ctx context.Context\n cancel func()\n wg sync.WaitGroup\n\n // Configuration\n MinCheckpointPageN int // Min pages for passive checkpoint\n TruncatePageN int // Pages before emergency truncate checkpoint\n CheckpointInterval time.Duration // Time-based passive checkpoint interval\n MonitorInterval time.Duration // WAL monitoring frequency\n // Note: MaxCheckpointPageN removed (RESTART mode disabled due to #724)\n\n // Metrics\n dbSizeGauge prometheus.Gauge\n walSizeGauge prometheus.Gauge\n txIDGauge prometheus.Gauge\n}\n```\n\n#### Key Methods\n\n```go\n// Lifecycle\nfunc (db *DB) Open() error\nfunc (db *DB) Close(ctx context.Context) error\n\n// Monitoring\nfunc (db *DB) monitor() // Background WAL monitoring\nfunc (db *DB) checkWAL() (bool, error) // Check for WAL changes\n\n// Checkpointing\nfunc (db *DB) Checkpoint(mode string) error\nfunc (db *DB) autoCheckpoint() error\n\n// Replication\nfunc (db *DB) WALReader(pgno uint32) (io.ReadCloser, error)\nfunc (db *DB) Sync(ctx context.Context) error\n\n// Compaction\nfunc (db *DB) Compact(ctx context.Context, destLevel int) (*ltx.FileInfo, error)\n```\n\n### Replica Component (replica.go)\n\nManages replication to a single destination:\n\n```go\ntype Replica struct {\n db *DB // Parent database\n Client ReplicaClient // Storage backend client\n\n mu sync.RWMutex\n pos ltx.Pos // Current replication position\n\n // Configuration\n SyncInterval time.Duration\n MonitorEnabled bool\n\n // Lifecycle\n cancel func()\n wg sync.WaitGroup\n}\n```\n\n#### Replication Position\n\n```go\ntype Pos struct {\n TXID TXID // Transaction ID\n PageNo uint32 // Page number within transaction\n Checksum uint64 // Running checksum\n}\n```\n\n### Store Component (store.go)\n\nCoordinates multiple databases and manages system-wide resources:\n\n```go\ntype Store struct {\n mu sync.Mutex\n dbs []*DB\n levels CompactionLevels\n\n // Configuration\n SnapshotInterval time.Duration\n SnapshotRetention time.Duration\n L0Retention time.Duration\n L0RetentionCheckInterval time.Duration\n CompactionMonitorEnabled bool\n\n // Lifecycle\n ctx context.Context\n cancel func()\n wg sync.WaitGroup\n}\n```\n\n## IPC Server & Control Socket\n\nLitestream exposes a Unix socket HTTP server for runtime control (`server.go`).\n\n### Socket Configuration\n\n```go\ntype SocketConfig struct {\n Enabled bool `yaml:\"enabled\"` // Default: false\n Path string `yaml:\"path\"` // Default: \"/var/run/litestream.sock\"\n Permissions uint32 `yaml:\"permissions\"` // Default: 0600\n}\n```\n\n### HTTP Endpoints\n\nAll endpoints are served over the Unix socket via Go's `net/http` mux:\n\n| Method | Path | Description |\n|--------|------|-------------|\n| `POST` | `/register` | Add a database at runtime |\n| `POST` | `/unregister` | Remove a database at runtime |\n| `GET` | `/txid?path=` | Get current transaction ID for a database |\n| `GET` | `/list` | List all managed databases |\n| `GET` | `/info` | Server info (version, PID, uptime, database count) |\n| `POST` | `/start` | Start replication for a database |\n| `POST` | `/stop` | Stop replication for a database |\n| `GET` | `/debug/pprof/*` | Standard Go pprof endpoints |\n\n### Request/Response Types\n\n```go\ntype RegisterDatabaseRequest struct {\n Path string `json:\"path\"`\n ReplicaURL string `json:\"replica_url\"`\n}\n\ntype UnregisterDatabaseRequest struct {\n Path string `json:\"path\"`\n Timeout int `json:\"timeout,omitempty\"` // Seconds\n}\n\ntype TXIDResponse struct {\n TXID uint64 `json:\"txid\"`\n}\n```\n\n## Distributed Leasing\n\nThe `Leaser` interface (`leaser.go`) enables distributed coordination so multiple Litestream instances can safely share a replica destination.\n\n### Interface\n\n```go\ntype Leaser interface {\n Type() string\n AcquireLease(ctx context.Context) (*Lease, error)\n RenewLease(ctx context.Context, lease *Lease) (*Lease, error)\n ReleaseLease(ctx context.Context, lease *Lease) error\n}\n\ntype Lease struct {\n Generation int64 `json:\"generation\"`\n ExpiresAt time.Time `json:\"expires_at\"`\n Owner string `json:\"owner,omitempty\"`\n ETag string `json:\"-\"`\n}\n```\n\n### S3 Implementation (`s3/leaser.go`)\n\n- **Defaults**: `DefaultLeaseTTL = 30s`, `DefaultLeasePath = \"lock.json\"`\n- **Owner format**: `hostname:pid` (falls back to `pid-N` if hostname unavailable)\n- **Conditional writes**: Uses `If-Match`/`If-None-Match` on S3 PutObject to prevent races\n- **Release**: Uses `DeleteObject` with `If-Match` to ensure only the holder can release\n- **Error types**: `ErrLeaseNotHeld`, `ErrLeaseAlreadyReleased`, `LeaseExistsError{Owner, ExpiresAt}`\n\n### Acquisition Flow\n\n1. Read existing lease from S3 (`lock.json`)\n2. If lease exists and is not expired, return `LeaseExistsError`\n3. Write new lease with `If-None-Match: *` (first acquire) or `If-Match: ` (expired takeover)\n4. If `PreconditionFailed` (412), another instance acquired first\n5. Return lease with ETag for subsequent renewal\n\n## Library Convenience Methods\n\nThese methods on `DB` (`db.go`) simplify common operations when Litestream is used as a library:\n\n```go\ntype SyncStatus struct {\n LocalTXID ltx.TXID // Local transaction ID\n RemoteTXID ltx.TXID // Remote transaction ID\n InSync bool // true if LocalTXID > 0 and equal to RemoteTXID\n}\n\nfunc (db *DB) SyncStatus(ctx context.Context) (SyncStatus, error)\nfunc (db *DB) SyncAndWait(ctx context.Context) error\nfunc (db *DB) EnsureExists(ctx context.Context) error\n```\n\n- **SyncStatus**: Compares local TXID against remote replica position (performs I/O to query remote)\n- **SyncAndWait**: Runs `db.Sync()` (WAL to LTX) then `db.Replica.Sync()` (LTX to remote), blocking until both complete\n- **EnsureExists**: Restores database from replica if local file doesn't exist; no-op if file exists or no backup available. Must be called before `Open()`\n\n## LTX File Format\n\nLTX (Log Transaction) files are immutable files containing database changes:\n\n```\n+------------------+\n| Header | Fixed size header with metadata\n+------------------+\n| |\n| Page Frames | Variable number of page frames\n| |\n+------------------+\n| Page Index | Index for efficient page lookup\n+------------------+\n| Trailer | Metadata and checksums\n+------------------+\n```\n\n### Header Structure\n\n```go\ntype Header struct {\n Magic [4]byte // \"LTX\\x00\"\n Version uint32 // Format version\n PageSize uint32 // Database page size\n MinTXID TXID // Starting transaction ID\n MaxTXID TXID // Ending transaction ID\n Timestamp int64 // Creation timestamp\n Checksum uint64 // Header checksum\n}\n```\n\n### Page Frame Structure\n\n```go\ntype PageFrame struct {\n Header PageHeader\n Data []byte // Page data (pageSize bytes)\n}\n\ntype PageHeader struct {\n PageNo uint32 // Page number in database\n Size uint32 // Size of page data\n Checksum uint64 // Page checksum\n}\n```\n\n### Page Index\n\nBinary search tree for efficient page lookup:\n```go\ntype PageIndexElem struct {\n PageNo uint32 // Page number\n Offset int64 // Offset in file\n Size uint32 // Size of page frame\n}\n```\n\n### Trailer\n\n```go\ntype Trailer struct {\n PageIndexOffset int64 // Offset to page index\n PageIndexSize int64 // Size of page index\n PageCount uint32 // Total pages in file\n Checksum uint64 // Full file checksum\n}\n```\n\n## WAL Monitoring Mechanism\n\n### Monitor Loop (db.go:1499)\n\n```go\nfunc (db *DB) monitor() {\n ticker := time.NewTicker(db.MonitorInterval)\n defer ticker.Stop()\n\n for {\n select {\n case <-ticker.C:\n // Check WAL for changes\n changed, err := db.checkWAL()\n if err != nil {\n slog.Error(\"wal check failed\", \"error\", err)\n continue\n }\n\n if changed {\n // Notify replicas of changes\n db.notifyReplicas()\n\n // Check if checkpoint needed\n if db.shouldCheckpoint() {\n db.autoCheckpoint()\n }\n }\n\n case <-db.ctx.Done():\n return\n }\n }\n}\n```\n\n### WAL Change Detection\n\n```go\nfunc (db *DB) checkWAL() (bool, error) {\n // Get current WAL state\n walInfo, err := db.walInfo()\n if err != nil {\n return false, err\n }\n\n // Compare with previous state\n db.mu.Lock()\n changed := walInfo.Size != db.prevWALSize ||\n walInfo.Checksum != db.prevWALChecksum\n db.prevWALSize = walInfo.Size\n db.prevWALChecksum = walInfo.Checksum\n db.mu.Unlock()\n\n return changed, nil\n}\n```\n\n## Compaction Process\n\nCompaction merges multiple LTX files to reduce storage overhead:\n\n### Compaction Algorithm (store.go:189)\n\nHigh-level compaction flow:\n\n1. Determine whether the level is due for compaction (`Store.shouldCompact`).\n2. Enumerate level-`L-1` files using `ReplicaClient.LTXFiles`, preferring local\n copies via `os.Open(db.LTXPath(...))` and falling back to\n `ReplicaClient.OpenLTXFile` only when necessary.\n3. Stream the source readers through `ltx.NewCompactor`, which performs\n page-level deduplication and enforces lock-page skipping automatically.\n4. Pipe the compactor output into `ReplicaClient.WriteLTXFile` to create the\n merged LTX file for level `L`.\n5. Adjust the returned `ltx.FileInfo.CreatedAt` to the earliest timestamp from\n the source files so point-in-time recovery remains accurate.\n6. Update the cached max file info for the level and delete old L0 files when\n promoting to level 1.\n\n### Compaction Levels\n\n```go\ntype CompactionLevel struct {\n Level int // Level number (0 = raw, 1+ = compacted)\n Interval time.Duration // How often to compact from previous level\n}\n\n// Default configuration\nvar DefaultCompactionLevels = CompactionLevels{\n {Level: 0, Interval: 0}, // Raw LTX files\n {Level: 1, Interval: 1 * Hour}, // Hourly compaction\n {Level: 2, Interval: 24 * Hour}, // Daily compaction\n}\n```\n\n## Transaction Management\n\n### Long-Running Read Transaction\n\nLitestream maintains a long-running read transaction to ensure consistency:\n\n```go\nfunc (db *DB) initReadTx() error {\n // Start read transaction\n tx, err := db.db.BeginTx(context.Background(), &sql.TxOptions{\n ReadOnly: true,\n })\n if err != nil {\n return err\n }\n\n // Execute dummy query to start transaction\n var dummy string\n err = tx.QueryRow(\"SELECT ''\").Scan(&dummy)\n if err != nil {\n tx.Rollback()\n return err\n }\n\n db.rtx = tx\n return nil\n}\n```\n\n**Purpose:**\n- Prevents database from being modified during replication\n- Ensures consistent view of database\n- Allows reading historical pages from WAL\n\n### Checkpoint Coordination\n\n```go\nfunc (db *DB) Checkpoint(mode string) error {\n // Acquire checkpoint lock\n db.chkMu.Lock()\n defer db.chkMu.Unlock()\n\n // Close read transaction temporarily\n if db.rtx != nil {\n db.rtx.Rollback()\n db.rtx = nil\n }\n\n // Perform checkpoint\n _, _, err := db.db.Exec(fmt.Sprintf(\"PRAGMA wal_checkpoint(%s)\", mode))\n if err != nil {\n return err\n }\n\n // Restart read transaction\n return db.initReadTx()\n}\n```\n\n## Concurrency Model\n\n### Mutex Usage Patterns\n\n```go\n// DB struct mutexes\ntype DB struct {\n mu sync.RWMutex // Protects struct fields\n chkMu sync.RWMutex // Checkpoint coordination\n}\n\n// Replica struct mutexes\ntype Replica struct {\n mu sync.RWMutex // Protects position\n muf sync.Mutex // File descriptor lock\n}\n\n// Store struct mutex\ntype Store struct {\n mu sync.Mutex // Protects database list\n}\n```\n\n### Thundering Herd Prevention\n\n`Store.Open()` (`store.go`) limits concurrent database opens at startup:\n\n```go\ng, ctx := errgroup.WithContext(ctx)\ng.SetLimit(50) // Max 50 concurrent DB opens\nfor _, db := range s.dbs {\n db := db\n g.Go(func() error { return db.Open() })\n}\n```\n\nThis prevents OS resource exhaustion (file descriptors, memory) when hundreds of databases are configured.\n\n### Lock Ordering (Prevent Deadlocks)\n\nAlways acquire locks in this order:\n1. Store.mu\n2. DB.mu\n3. DB.chkMu\n4. Replica.mu\n\n### Goroutine Management\n\n```go\n// Start background task\nfunc (db *DB) Start() {\n db.wg.Add(1)\n go func() {\n defer db.wg.Done()\n db.monitor()\n }()\n}\n\n// Stop with timeout\nfunc (db *DB) Close(ctx context.Context) error {\n // Signal shutdown\n db.cancel()\n\n // Wait for goroutines with timeout\n done := make(chan struct{})\n go func() {\n db.wg.Wait()\n close(done)\n }()\n\n select {\n case <-done:\n return nil\n case <-ctx.Done():\n return ctx.Err()\n }\n}\n```\n\n## State Management\n\n### Database States\n\n```mermaid\nstateDiagram-v2\n [*] --> Closed\n Closed --> Opening: Open()\n Opening --> Open: Success\n Opening --> Closed: Error\n Open --> Monitoring: Start()\n Monitoring --> Syncing: Changes Detected\n Syncing --> Monitoring: Sync Complete\n Monitoring --> Checkpointing: Threshold Reached\n Checkpointing --> Monitoring: Checkpoint Complete\n Monitoring --> Closing: Close()\n Closing --> Closed: Cleanup Complete\n```\n\n### Replica States\n\n```mermaid\nstateDiagram-v2\n [*] --> Idle\n Idle --> Starting: Start()\n Starting --> Monitoring: Success\n Starting --> Idle: Error\n Monitoring --> Syncing: Timer/Changes\n Syncing --> Uploading: Have Changes\n Uploading --> Monitoring: Success\n Uploading --> Error: Failed\n Error --> Monitoring: Retry\n Monitoring --> Stopping: Stop()\n Stopping --> Idle: Cleanup\n```\n\n### Position Tracking\n\n```go\ntype Pos struct {\n TXID TXID // Current transaction ID\n PageNo uint32 // Current page number\n Checksum uint64 // Running checksum for validation\n}\n\n// Update position atomically\nfunc (r *Replica) SetPos(pos ltx.Pos) {\n r.mu.Lock() // MUST use Lock, not RLock!\n defer r.mu.Unlock()\n r.pos = pos\n}\n\n// Read position safely\nfunc (r *Replica) Pos() ltx.Pos {\n r.mu.RLock()\n defer r.mu.RUnlock()\n return r.pos\n}\n```\n\n## Initialization Flow\n\n### System Startup Sequence\n\n```mermaid\nsequenceDiagram\n participant Main\n participant Store\n participant DB\n participant Replica\n participant Monitor\n\n Main->>Store: NewStore(config)\n Store->>Store: Validate config\n\n Main->>Store: Open()\n loop For each database\n Store->>DB: NewDB(path)\n Store->>DB: Open()\n DB->>DB: Open SQLite connection\n DB->>DB: Read page size\n DB->>DB: Init metadata\n DB->>DB: Start read transaction\n\n loop For each replica\n DB->>Replica: NewReplica()\n DB->>Replica: Start()\n Replica->>Monitor: Start monitoring\n end\n end\n\n Store->>Store: Start compaction monitors\n Store-->>Main: Ready\n```\n\n### Critical Initialization Steps\n\n1. **Database Opening**\n ```go\n // Must happen in order:\n 1. Open SQLite connection\n 2. Read page size (PRAGMA page_size)\n 3. Create metadata directory\n 4. Start long-running read transaction\n 5. Initialize replicas\n 6. Start monitor goroutine\n ```\n\n2. **Replica Initialization**\n ```go\n // Must happen in order:\n 1. Create replica with client\n 2. Load previous position from metadata\n 3. Validate position against database\n 4. Start sync goroutine (if monitoring enabled)\n ```\n\n## Error Handling\n\n### Error Categories\n\n1. **Recoverable Errors**\n - Network timeouts\n - Temporary storage unavailability\n - Lock contention\n\n2. **Fatal Errors**\n - Database corruption\n - Invalid configuration\n - Disk full\n\n3. **Operational Errors**\n - Checkpoint failures\n - Compaction conflicts\n - Sync delays\n\n### Error Propagation\n\n```go\n// Bottom-up error propagation\nReplicaClient.WriteLTXFile() error\n ↓\nReplica.Sync() error\n ↓\nDB.Sync() error\n ↓\nStore.monitorDB() // Logs error, continues\n```\n\n### Retry Logic\n\n```go\nfunc (r *Replica) syncWithRetry(ctx context.Context) error {\n backoff := time.Second\n maxBackoff := time.Minute\n\n for attempt := 0; ; attempt++ {\n err := r.Sync(ctx)\n if err == nil {\n return nil\n }\n\n // Check if error is retryable\n if !isRetryable(err) {\n return err\n }\n\n // Check context\n if ctx.Err() != nil {\n return ctx.Err()\n }\n\n // Exponential backoff\n time.Sleep(backoff)\n backoff *= 2\n if backoff > maxBackoff {\n backoff = maxBackoff\n }\n }\n}\n```\n\n## Performance Characteristics\n\n### Time Complexity\n\n| Operation | Complexity | Notes |\n|-----------|------------|-------|\n| WAL Monitor | O(1) | Fixed interval check |\n| Page Write | O(1) | Append to LTX file |\n| Compaction | O(n) | n = total pages |\n| Restoration | O(n*log(m)) | n = pages, m = files |\n| File List | O(k) | k = files in level |\n\n### Space Complexity\n\n| Component | Memory Usage | Disk Usage |\n|-----------|-------------|------------|\n| DB | O(1) + metrics | Original DB + WAL |\n| Replica | O(1) | LTX files + metadata |\n| Compaction | O(n) pages | Temporary during merge |\n| Page Index | O(p) | p = pages in file |\n\n### Optimization Points\n\n1. **Page Index Caching**\n - Cache frequently accessed indices\n - Use estimated size for initial fetch\n\n2. **Batch Operations**\n - Group small changes into larger LTX files\n - Batch delete operations\n\n3. **Concurrent Operations**\n - Multiple replicas can sync in parallel\n - Compaction runs independently per level\n\n## Security Considerations\n\n### Access Control\n\n- File permissions: 0600 for database files\n- Directory permissions: 0700 for metadata\n- No built-in authentication (rely on storage backend)\n\n## Monitoring & Metrics\n\n### Prometheus Metrics\n\n```go\n// Database metrics\ndb_size_bytes // Current database size\nwal_size_bytes // Current WAL size\ntotal_wal_bytes // Total bytes written to WAL\ncheckpoint_count // Number of checkpoints\nsync_count // Number of syncs\nsync_error_count // Number of sync errors\n\n// Replica metrics\nreplica_lag_seconds // Replication lag\nreplica_position // Current replication position\n```\n\n### Health Checks\n\n```go\nfunc (db *DB) HealthCheck() error {\n // Check database connection\n if err := db.db.Ping(); err != nil {\n return fmt.Errorf(\"database ping failed: %w\", err)\n }\n\n // Check replication lag\n for _, r := range db.replicas {\n lag := time.Since(r.LastSync())\n if lag > MaxAcceptableLag {\n return fmt.Errorf(\"replica %s lag too high: %v\", r.Name(), lag)\n }\n }\n\n return nil\n}\n```\n" }, { "path": "docs/DOC_MAINTENANCE.md", "content": "# Documentation Maintenance Guide\n\nThis guide ensures documentation stays synchronized with code changes and follows the principle-based approach established in PR #787.\n\n## Philosophy: Principles Over Examples\n\n**Key Insight**: Code examples become outdated quickly. Documentation should focus on **stable concepts** rather than **volatile implementations**.\n\n### What to Document\n\n✅ **DO Document**:\n\n- **Architectural principles** (e.g., \"DB layer handles database state\")\n- **Interface contracts** (what methods must do, not how they do it)\n- **Design patterns** (atomic file operations, eventual consistency handling)\n- **Critical edge cases** (1GB lock page, timestamp preservation)\n- **\"Why\" not \"what\"** (rationale behind decisions)\n\n❌ **DON'T Document**:\n\n- Specific function implementations that change frequently\n- Exact function names without referencing actual source\n- Step-by-step code that duplicates the implementation\n- Version-specific details that will quickly become stale\n\n### Documentation Principles\n\n1. **Abstractions over Details**: Document the concept, not the specific implementation\n2. **Reference over Duplication**: Point to actual source files instead of copying code\n3. **Patterns over Examples**: Describe the approach, let developers read the source\n4. **Contracts over Implementations**: Define what must happen, not how\n\n## When Code Changes, Update Docs\n\n### Interface Changes\n\n**Trigger**: Modifying `ReplicaClient` interface or any public interface\n\n**Required Updates**:\n\n1. Search for interface definitions in docs:\n\n ```bash\n rg \"type ReplicaClient interface\" docs/ CLAUDE.md AGENTS.md .claude/\n ```\n\n2. Update interface signatures (don't forget parameters!)\n3. Document new parameters with clear explanations of when/why to use them\n4. Update all example calls to include new parameters\n\n**Files to Check**:\n\n- `AGENTS.md` - Interface definitions\n- `docs/REPLICA_CLIENT_GUIDE.md` - Implementation guide\n- `docs/TESTING_GUIDE.md` - Test examples\n- `.claude/agents/replica-client-developer.md` - Agent knowledge\n- `.claude/commands/add-storage-backend.md` - Backend templates\n- `.claude/commands/validate-replica.md` - Validation commands\n\n### New Features\n\n**Trigger**: Adding new functionality, methods, or components\n\n**Approach**:\n\n1. **Don't rush to document** - Wait until the feature stabilizes\n2. **Document the pattern**, not the implementation:\n - What problem does it solve?\n - What's the high-level approach?\n - What are the critical constraints?\n3. **Reference the source**:\n - `See implementation in file.go:lines`\n - `Reference tests in file_test.go`\n\n### Refactoring\n\n**Trigger**: Moving or renaming functions, restructuring code\n\n**Required Actions**:\n\n1. **Search for references**:\n\n ```bash\n # Find function name references\n rg \"functionName\" docs/ CLAUDE.md AGENTS.md .claude/\n ```\n\n2. **Update or remove**:\n - If it's a reference pointer (e.g., \"See `DB.init()` in db.go:123\"), update it\n - If it's a code example showing implementation, consider replacing with a pattern description\n\n3. **Verify links**: Ensure all file:line references are still valid\n\n## Documentation Update Checklist\n\nUse this checklist when making code changes:\n\n- [ ] **Search docs for affected code**:\n\n ```bash\n # Search for function names, types, or concepts\n rg \"YourFunctionName\" docs/ CLAUDE.md AGENTS.md .claude/\n ```\n\n- [ ] **Update interface definitions** if signatures changed\n- [ ] **Update examples** if they won't compile anymore\n- [ ] **Convert brittle examples to patterns** if refactoring made them stale\n- [ ] **Update file:line references** if code moved\n- [ ] **Verify contracts still hold** (update if behavior changed)\n- [ ] **Run markdownlint**:\n\n ```bash\n markdownlint --fix docs/ CLAUDE.md AGENTS.md .claude/\n ```\n\n## Preventing Documentation Drift\n\n### Pre-Commit Practices\n\n1. **Search before committing**:\n\n ```bash\n git diff --name-only | xargs -I {} rg \"basename {}\" docs/\n ```\n\n2. **Review doc references** in your PR description\n3. **Test examples compile** (if they're meant to)\n\n### Regular Audits\n\n**Monthly**: Spot-check one documentation file against current codebase\n\n**Questions to ask**:\n\n- Do interface definitions match `replica_client.go`?\n- Do code examples compile?\n- Are file:line references accurate?\n- Have we removed outdated examples?\n\n### When in Doubt\n\n**Rule**: Delete outdated documentation rather than let it mislead\n\n- Stale examples cause compilation errors\n- Outdated patterns cause architectural mistakes\n- Incorrect references waste developer time\n\n**Better**: A brief pattern description + reference to source than an outdated example\n\n## Example: Good vs Bad Documentation Updates\n\n### ❌ Bad: Copying Implementation\n\n```markdown\n### How to initialize DB\n\n```go\nfunc (db *DB) init() {\n db.mu.Lock()\n defer db.mu.Unlock()\n // ... 50 lines of code copied from db.go\n}\n\\```\n```\n\n**Problem**: This will be outdated as soon as the implementation changes.\n\n### ✅ Good: Documenting Pattern + Reference\n\n```markdown\n### DB Initialization Pattern\n\n**Principle**: Database initialization must complete before replication starts.\n\n**Pattern**:\n\n1. Acquire exclusive lock (`mu.Lock()`)\n2. Verify database state consistency\n3. Initialize monitoring subsystems\n4. Set up replication coordination\n\n**Critical**: Use `Lock()` not `RLock()` as initialization modifies state.\n\n**Reference Implementation**: See `DB.init()` in db.go:150-230\n```\n\n**Benefits**: Stays accurate even if implementation details change, focuses on the \"why\" and \"what\" rather than the \"how\".\n\n## Tools and Commands\n\n### Find Documentation References\n\n```bash\n# Find all code examples in documentation\nrg \"^```(go|golang)\" docs/ CLAUDE.md AGENTS.md .claude/\n\n# Find file:line references\nrg \"\\.go:\\d+\" docs/ CLAUDE.md AGENTS.md .claude/\n\n# Find interface definitions\nrg \"type .* interface\" docs/ CLAUDE.md AGENTS.md .claude/\n```\n\n### Validate Markdown\n\n```bash\n# Lint all docs\nmarkdownlint docs/ CLAUDE.md AGENTS.md .claude/\n\n# Auto-fix issues\nmarkdownlint --fix docs/ CLAUDE.md AGENTS.md .claude/\n```\n\n### Check for Broken References\n\n```bash\n# List all go files mentioned in docs\nrg -o \"[a-z_]+\\.go:\\d+\" docs/ CLAUDE.md AGENTS.md | sort -u\n\n# Verify they exist and line numbers are reasonable\n```\n\n## Resources\n\n- **PR #787**: Original principle-based documentation refactor\n- **Issue #805**: Context for why accurate documentation matters\n- **INNOQ Best Practices**: \n- **Google Style Guide**: \n\n## Questions?\n\nWhen updating documentation, ask:\n\n1. **Is this a stable concept or a volatile implementation?**\n - Stable → Document the principle\n - Volatile → Reference the source\n\n2. **Will this stay accurate for 6+ months?**\n - Yes → Keep it\n - No → Replace with pattern description\n\n3. **Does this explain WHY or just WHAT?**\n - WHY → Valuable documentation\n - WHAT → Code already shows this, just reference it\n\n4. **Would a link to source code be better?**\n - Often, yes!\n" }, { "path": "docs/LTX_FORMAT.md", "content": "# LTX Format Specification\n\nLTX (Log Transaction) is Litestream's custom format for storing database changes in an immutable, append-only manner.\n\n## Table of Contents\n- [Overview](#overview)\n- [File Structure](#file-structure)\n- [Header Format](#header-format)\n- [Page Frames](#page-frames)\n- [Page Index](#page-index)\n- [Trailer Format](#trailer-format)\n- [File Naming Convention](#file-naming-convention)\n- [Checksum Calculation](#checksum-calculation)\n- [Compaction and Levels](#compaction-and-levels)\n- [Reading LTX Files](#reading-ltx-files)\n- [Writing LTX Files](#writing-ltx-files)\n- [Relationship to SQLite WAL](#relationship-to-sqlite-wal)\n\n## Overview\n\nLTX files are immutable snapshots of database changes:\n- **Immutable**: Once written, never modified\n- **Append-only**: New changes create new files\n- **Self-contained**: Each file is independent\n- **Indexed**: Contains page index for efficient seeks\n- **Checksummed**: Integrity verification built-in\n\n```mermaid\ngraph LR\n WAL[SQLite WAL] -->|Convert| LTX[LTX File]\n LTX -->|Upload| Storage[Cloud Storage]\n Storage -->|Download| Restore[Restored DB]\n```\n\n## File Structure\n\n```\n┌─────────────────────┐\n│ Header │ Fixed size (varies by version)\n├─────────────────────┤\n│ │\n│ Page Frames │ Variable number of pages\n│ │\n├─────────────────────┤\n│ Page Index │ Binary search tree\n├─────────────────────┤\n│ Trailer │ Fixed size metadata\n└─────────────────────┘\n```\n\n### Size Calculation\n\n```go\nFileSize = HeaderSize +\n (PageCount * (PageHeaderSize + PageSize)) +\n PageIndexSize +\n TrailerSize\n```\n\n## Header Format\n\nThe LTX header contains metadata about the file:\n\n```go\n// From github.com/superfly/ltx\ntype Header struct {\n Version int // Derived from the magic string (\"LTX1\")\n Flags uint32 // Reserved flag bits\n PageSize uint32 // Database page size\n Commit uint32 // Page count after applying file\n MinTXID TXID\n MaxTXID TXID\n Timestamp int64 // Milliseconds since Unix epoch\n PreApplyChecksum Checksum // Database checksum before apply\n WALOffset int64 // Offset within source WAL (0 for snapshots)\n WALSize int64 // WAL byte length (0 for snapshots)\n WALSalt1 uint32\n WALSalt2 uint32\n NodeID uint64\n}\n\nconst HeaderFlagNoChecksum = uint32(1 << 1)\n```\n\n> Note: the version is implied by the magic string. Present files use\n> `Magic == \"LTX1\"`, which corresponds to `ltx.Version == 2`.\n\n### Binary Layout (Header)\n\n```\nOffset Size Field\n0 4 Magic (\"LTX1\")\n4 4 Flags\n8 4 PageSize\n12 4 Commit\n16 8 MinTXID\n24 8 MaxTXID\n32 8 Timestamp\n40 8 PreApplyChecksum\n48 8 WALOffset\n56 8 WALSize\n64 4 WALSalt1\n68 4 WALSalt2\n72 8 NodeID\n80 20 Reserved (zeros)\nTotal: 100 bytes\n```\n\n## Page Frames\n\nEach page frame contains a database page with metadata:\n\n```go\ntype PageFrame struct {\n Header PageHeader\n Data []byte // Size = PageSize from LTX header\n}\n\ntype PageHeader struct {\n Pgno uint32 // Database page number (1-based)\n}\n```\n\n### Binary Layout (Page Frame)\n\n```\nOffset Size Field\n0 4 Page Number (Pgno)\n4 PageSize Page Data\n```\n\n### Page Frame Constraints\n\n1. **Sequential Writing**: Pages written in order during creation\n2. **Random Access**: Can seek to any page using index\n3. **Lock Page Skipping**: Page at 1GB boundary never included\n4. **Deduplication**: In compacted files, only latest version of each page\n\n## Page Index\n\nThe page index enables efficient random access to pages:\n\n```go\ntype PageIndexElem struct {\n Level int\n MinTXID TXID\n MaxTXID TXID\n Offset int64 // Byte offset of encoded payload\n Size int64 // Bytes occupied by encoded payload\n}\n```\n\n### Binary Layout (Page Index)\n\n```\nRather than parsing raw bytes, call `ltx.DecodePageIndex` which returns a\nmap of page number to `ltx.PageIndexElem` for you.\n```\n\n### Index Usage\n\n```go\n// Finding a page using the index\nfunc findPage(index []PageIndexElem, targetPageNo uint32) (offset int64, found bool) {\n // Binary search\n idx := sort.Search(len(index), func(i int) bool {\n return index[i].PageNo >= targetPageNo\n })\n\n if idx < len(index) && index[idx].PageNo == targetPageNo {\n return index[idx].Offset, true\n }\n return 0, false\n}\n```\n\n## Trailer Format\n\nThe trailer contains metadata and pointers:\n\n```go\ntype Trailer struct {\n PostApplyChecksum Checksum // Database checksum after apply\n FileChecksum Checksum // CRC-64 checksum of entire file\n}\n```\n\n### Binary Layout (Trailer)\n\n```\nOffset Size Field\n0 8 PostApplyChecksum\n8 8 FileChecksum\nTotal: 16 bytes\n```\n\n### Reading Trailer\n\nThe trailer is always at the end of the file:\n\n```go\nfunc readTrailer(f *os.File) (*Trailer, error) {\n // Seek to trailer position\n _, err := f.Seek(-TrailerSize, io.SeekEnd)\n if err != nil {\n return nil, err\n }\n\n var trailer Trailer\n err = binary.Read(f, binary.BigEndian, &trailer)\n return &trailer, err\n}\n```\n\n## File Naming Convention\n\nLTX files follow a strict naming pattern:\n\n```\nFormat: MMMMMMMMMMMMMMMM-NNNNNNNNNNNNNNNN.ltx\nWhere:\n M = MinTXID (16 hex digits, zero-padded)\n N = MaxTXID (16 hex digits, zero-padded)\n\nExamples:\n 0000000000000001-0000000000000064.ltx (TXID 1-100)\n 0000000000000065-00000000000000c8.ltx (TXID 101-200)\n```\n\n### Parsing Filenames\n\n```go\n// From github.com/superfly/ltx\nfunc ParseFilename(name string) (minTXID, maxTXID TXID, err error) {\n // Remove extension\n name = strings.TrimSuffix(name, \".ltx\")\n\n // Split on hyphen\n parts := strings.Split(name, \"-\")\n if len(parts) != 2 {\n return 0, 0, errors.New(\"invalid format\")\n }\n\n // Parse hex values\n min, err := strconv.ParseUint(parts[0], 16, 64)\n max, err := strconv.ParseUint(parts[1], 16, 64)\n\n return TXID(min), TXID(max), nil\n}\n\nfunc FormatFilename(minTXID, maxTXID TXID) string {\n return fmt.Sprintf(\"%016x-%016x.ltx\", minTXID, maxTXID)\n}\n```\n\n## Checksum Calculation\n\nLTX uses CRC-64 ECMA checksums:\n\n```go\nimport \"hash/crc64\"\n\nvar crcTable = crc64.MakeTable(crc64.ECMA)\n\nfunc calculateChecksum(data []byte) uint64 {\n return crc64.Checksum(data, crcTable)\n}\n\n// Cumulative checksum for multiple pages\nfunc cumulativeChecksum(pages [][]byte) uint64 {\n h := crc64.New(crcTable)\n for _, page := range pages {\n h.Write(page)\n }\n return h.Sum64()\n}\n```\n\n### Verification During Read\n\n```go\nfunc verifyPage(header PageHeader, data []byte) error {\n if header.Checksum == 0 {\n return nil // Checksums disabled\n }\n\n calculated := calculateChecksum(data)\n if calculated != header.Checksum {\n return fmt.Errorf(\"checksum mismatch: expected %x, got %x\",\n header.Checksum, calculated)\n }\n return nil\n}\n```\n\n## Compaction and Levels\n\nLTX files are organized in levels for efficient compaction:\n\n```\nLevel 0: Raw files (no compaction)\n /ltx/0000/0000000000000001-0000000000000064.ltx\n /ltx/0000/0000000000000065-00000000000000c8.ltx\n\nLevel 1: Hourly compaction\n /ltx/0001/0000000000000001-0000000000000fff.ltx\n\nLevel 2: Daily compaction\n /ltx/0002/0000000000000001-000000000000ffff.ltx\n\nSnapshots: Full database state\n /snapshots/20240101120000.ltx\n```\n\n### Compaction Process\n\n```go\nfunc compactLTXFiles(files []*LTXFile) (*LTXFile, error) {\n // Create page map (newer overwrites older)\n pageMap := make(map[uint32]Page)\n\n for _, file := range files {\n for _, page := range file.Pages {\n pageMap[page.Number] = page\n }\n }\n\n // Create new LTX with merged pages\n merged := <XFile{\n MinTXID: files[0].MinTXID,\n MaxTXID: files[len(files)-1].MaxTXID,\n }\n\n // Add pages in order (skip lock page)\n for pgno := uint32(1); pgno <= maxPgno; pgno++ {\n if pgno == LockPageNumber(pageSize) {\n continue // Skip 1GB lock page\n }\n if page, ok := pageMap[pgno]; ok {\n merged.Pages = append(merged.Pages, page)\n }\n }\n\n return merged, nil\n}\n```\n\n## Reading LTX Files\n\n### Complete File Read\n\n```go\nfunc ReadLTXFile(path string) (*LTXFile, error) {\n f, err := os.Open(path)\n if err != nil {\n return nil, err\n }\n defer f.Close()\n\n dec := ltx.NewDecoder(f)\n\n // Read and verify header\n header, err := dec.Header()\n if err != nil {\n return nil, err\n }\n\n // Read all pages\n var pages []Page\n for {\n var pageHeader ltx.PageHeader\n pageData := make([]byte, header.PageSize)\n\n err := dec.DecodePage(&pageHeader, pageData)\n if err == io.EOF {\n break\n }\n if err != nil {\n return nil, err\n }\n\n pages = append(pages, Page{\n Number: pageHeader.PageNo,\n Data: pageData,\n })\n }\n\n return <XFile{\n Header: header,\n Pages: pages,\n }, nil\n}\n```\n\n### Partial Read Using Index\n\n```go\nfunc ReadPage(path string, pageNo uint32) ([]byte, error) {\n f, err := os.Open(path)\n if err != nil {\n return nil, err\n }\n defer f.Close()\n\n // Read trailer to find index\n trailer, err := readTrailer(f)\n if err != nil {\n return nil, err\n }\n\n // Read page index\n f.Seek(trailer.PageIndexOffset, io.SeekStart)\n indexData := make([]byte, trailer.PageIndexSize)\n f.Read(indexData)\n\n index := parsePageIndex(indexData)\n\n // Find page in index\n offset, found := findPage(index, pageNo)\n if !found {\n return nil, errors.New(\"page not found\")\n }\n\n // Read page at offset\n f.Seek(offset, io.SeekStart)\n\n var pageHeader PageHeader\n binary.Read(f, binary.BigEndian, &pageHeader)\n\n pageData := make([]byte, pageSize)\n f.Read(pageData)\n\n return pageData, nil\n}\n```\n\n## Writing LTX Files\n\n### Creating New LTX File\n\n```go\nfunc WriteLTXFile(path string, pages []Page) error {\n f, err := os.Create(path)\n if err != nil {\n return err\n }\n defer f.Close()\n\n enc := ltx.NewEncoder(f)\n\n // Write header\n header := ltx.Header{\n Version: ltx.Version,\n Flags: 0,\n PageSize: 4096,\n PageCount: uint32(len(pages)),\n MinTXID: minTXID,\n MaxTXID: maxTXID,\n }\n\n if err := enc.EncodeHeader(header); err != nil {\n return err\n }\n\n // Write pages and build index\n var index []PageIndexElem\n for _, page := range pages {\n offset := enc.Offset()\n\n // Skip lock page\n if page.Number == LockPageNumber(header.PageSize) {\n continue\n }\n\n pageHeader := ltx.PageHeader{\n PageNo: page.Number,\n Checksum: calculateChecksum(page.Data),\n }\n\n if err := enc.EncodePage(pageHeader, page.Data); err != nil {\n return err\n }\n\n index = append(index, PageIndexElem{\n PageNo: page.Number,\n Offset: offset,\n })\n }\n\n // Write page index\n if err := enc.EncodePageIndex(index); err != nil {\n return err\n }\n\n // Write trailer\n if err := enc.EncodeTrailer(); err != nil {\n return err\n }\n\n return enc.Close()\n}\n```\n\n## Relationship to SQLite WAL\n\n### WAL to LTX Conversion\n\n```mermaid\nsequenceDiagram\n participant SQLite\n participant WAL\n participant Litestream\n participant LTX\n\n SQLite->>WAL: Write transaction\n WAL->>WAL: Append frames\n\n Litestream->>WAL: Monitor changes\n WAL-->>Litestream: Read frames\n\n Litestream->>Litestream: Convert frames\n Note over Litestream: - Skip lock page
- Add checksums
- Build index\n\n Litestream->>LTX: Write LTX file\n LTX->>Storage: Upload\n```\n\n### Key Differences\n\n| Aspect | SQLite WAL | LTX Format |\n|--------|------------|------------|\n| Purpose | Temporary changes | Permanent archive |\n| Mutability | Mutable (checkpoint) | Immutable |\n| Structure | Sequential frames | Indexed pages |\n| Checksum | Per-frame | Per-page + cumulative |\n| Lock Page | Contains lock bytes | Always skipped |\n| Naming | Fixed (-wal suffix) | TXID range |\n| Lifetime | Until checkpoint | Forever |\n| Size | Grows until checkpoint | Fixed at creation |\n\n### Transaction ID (TXID)\n\n```go\ntype TXID uint64\n\n// TXID represents a logical transaction boundary\n// Not directly from SQLite, but derived from:\n// 1. WAL checkpoint sequence\n// 2. Frame count\n// 3. Logical grouping of changes\n\nfunc (db *DB) nextTXID() TXID {\n // Increment from last known TXID\n return db.lastTXID + 1\n}\n```\n\n## Best Practices\n\n### 1. Always Skip Lock Page\n\n```go\nconst PENDING_BYTE = 0x40000000\n\nfunc shouldSkipPage(pageNo uint32, pageSize int) bool {\n lockPage := uint32(PENDING_BYTE/pageSize) + 1\n return pageNo == lockPage\n}\n```\n\n### 2. Preserve Timestamps During Compaction\n\n```go\n// Keep earliest CreatedAt from source files\nfunc compactWithTimestamp(files []*FileInfo) *FileInfo {\n earliest := files[0].CreatedAt\n for _, f := range files[1:] {\n if f.CreatedAt.Before(earliest) {\n earliest = f.CreatedAt\n }\n }\n\n return &FileInfo{\n CreatedAt: earliest, // Preserve for point-in-time recovery\n }\n}\n```\n\n### 3. Verify Checksums on Read\n\n```go\nfunc safeReadLTX(path string) (*LTXFile, error) {\n file, err := ReadLTXFile(path)\n if err != nil {\n return nil, err\n }\n\n // Verify all checksums\n for _, page := range file.Pages {\n if err := verifyPage(page); err != nil {\n return nil, fmt.Errorf(\"corrupted page %d: %w\",\n page.Number, err)\n }\n }\n\n return file, nil\n}\n```\n\n### 4. Handle Partial Files\n\n```go\n// For eventually consistent storage\nfunc readWithRetry(client ReplicaClient, info *FileInfo) ([]byte, error) {\n for attempts := 0; attempts < 5; attempts++ {\n data, err := client.OpenLTXFile(...)\n if err == nil {\n // Verify we got complete file\n if int64(len(data)) == info.Size {\n return data, nil\n }\n }\n\n time.Sleep(time.Second * time.Duration(attempts+1))\n }\n\n return nil, errors.New(\"incomplete file after retries\")\n}\n```\n\n## Debugging LTX Files\n\n### Inspect LTX Files\n\nThe Litestream CLI currently exposes a single helper for listing LTX files:\n\n```bash\nlitestream ltx /path/to/db.sqlite\nlitestream ltx s3://bucket/db\n```\n\nFor low-level inspection (page payloads, checksums, etc.), use the Go API:\n\n```go\nf, err := os.Open(\"0000000000000001-0000000000000064.ltx\")\nif err != nil {\n log.Fatal(err)\n}\ndefer f.Close()\n\ndec := ltx.NewDecoder(f)\nif err := dec.DecodeHeader(); err != nil {\n log.Fatal(err)\n}\nfor {\n var hdr ltx.PageHeader\n data := make([]byte, dec.Header().PageSize)\n if err := dec.DecodePage(&hdr, data); err == io.EOF {\n break\n } else if err != nil {\n log.Fatal(err)\n }\n // Inspect hdr.Pgno or data here.\n}\nif err := dec.Close(); err != nil {\n log.Fatal(err)\n}\nfmt.Println(\"post-apply checksum:\", dec.Trailer().PostApplyChecksum)\n```\n\n## Summary\n\nLTX format provides:\n1. **Immutable history** - Every change preserved\n2. **Efficient storage** - Indexed, compressed via compaction\n3. **Data integrity** - Checksums at multiple levels\n4. **Point-in-time recovery** - Via TXID ranges\n5. **Cloud-optimized** - Designed for object storage\n\nUnderstanding LTX is essential for:\n- Implementing replica clients\n- Debugging replication issues\n- Optimizing compaction\n- Ensuring data integrity\n- Building recovery tools\n" }, { "path": "docs/PATTERNS.md", "content": "# Litestream Code Patterns and Anti-Patterns\n\nThis document contains detailed code patterns, examples, and anti-patterns for working with Litestream. For a quick overview, see [AGENTS.md](../AGENTS.md).\n\n## Table of Contents\n\n- [Architectural Boundaries](#architectural-boundaries)\n- [Atomic File Operations](#atomic-file-operations)\n- [Error Handling](#error-handling)\n- [Locking Patterns](#locking-patterns)\n- [Compaction and Eventual Consistency](#compaction-and-eventual-consistency)\n- [Resumable Reader Pattern](#resumable-reader-pattern)\n- [Retention Bypass Pattern](#retention-bypass-pattern)\n- [Conditional Write Pattern (Distributed Locking)](#conditional-write-pattern-distributed-locking)\n- [Timestamp Preservation](#timestamp-preservation)\n- [Common Pitfalls](#common-pitfalls)\n- [Component Reference](#component-reference)\n\n## Architectural Boundaries\n\n### Layer Responsibilities\n\n```text\nDB Layer (db.go) → Database state, restoration, monitoring\nReplica Layer (replica.go) → Replication mechanics only\nStorage Layer → ReplicaClient implementations\n```\n\n### DO: Handle database state in DB layer\n\nDatabase restoration logic belongs in the DB layer, not the Replica layer.\n\nWhen the database is behind the replica (local TXID < remote TXID):\n\n1. **Clear local L0 cache**: Remove the entire L0 directory and recreate it\n2. **Fetch latest L0 file from replica**: Download the most recent L0 LTX file\n3. **Write using atomic file operations**: Prevent partial/corrupted files\n\n```go\n// CORRECT - DB layer handles database state\nfunc (db *DB) init() error {\n // DB layer handles database state\n if db.needsRestore() {\n if err := db.restore(); err != nil {\n return err\n }\n }\n // Then start replica for replication only\n return db.replica.Start()\n}\n\nfunc (r *Replica) Start() error {\n // Replica focuses only on replication\n return r.startSync()\n}\n```\n\nReference: `DB.checkDatabaseBehindReplica()` in db.go:670-737\n\n### DON'T: Put database state logic in Replica layer\n\n```go\n// WRONG - Replica should only handle replication concerns\nfunc (r *Replica) Start() error {\n // DON'T check database state here\n if needsRestore() { // Wrong layer!\n restoreDatabase() // Wrong layer!\n }\n // Replica should focus only on replication mechanics\n}\n```\n\n## Atomic File Operations\n\nAlways use atomic writes to prevent partial/corrupted files.\n\n### DO: Write to temp file, then rename\n\n```go\n// CORRECT - Atomic file write pattern\nfunc writeFileAtomic(path string, data []byte) error {\n // Create temp file in same directory (for atomic rename)\n dir := filepath.Dir(path)\n tmpFile, err := os.CreateTemp(dir, \".tmp-*\")\n if err != nil {\n return fmt.Errorf(\"create temp file: %w\", err)\n }\n tmpPath := tmpFile.Name()\n\n // Clean up temp file on error\n defer func() {\n if tmpFile != nil {\n tmpFile.Close()\n os.Remove(tmpPath)\n }\n }()\n\n // Write data to temp file\n if _, err := tmpFile.Write(data); err != nil {\n return fmt.Errorf(\"write temp file: %w\", err)\n }\n\n // Sync to ensure data is on disk\n if err := tmpFile.Sync(); err != nil {\n return fmt.Errorf(\"sync temp file: %w\", err)\n }\n\n // Close before rename\n if err := tmpFile.Close(); err != nil {\n return fmt.Errorf(\"close temp file: %w\", err)\n }\n tmpFile = nil // Prevent defer cleanup\n\n // Atomic rename (on same filesystem)\n if err := os.Rename(tmpPath, path); err != nil {\n os.Remove(tmpPath)\n return fmt.Errorf(\"rename to final path: %w\", err)\n }\n\n return nil\n}\n```\n\n### DON'T: Write directly to final location\n\n```go\n// WRONG - Can leave partial files on failure\nfunc writeFileDirect(path string, data []byte) error {\n return os.WriteFile(path, data, 0644) // Not atomic!\n}\n```\n\n## Error Handling\n\n### Decision Rule\n\nWhen you handle an error, ask: \"Does the caller need to know about this failure?\"\n\n- **Yes → return the error.** This is the default for virtually all cases in Litestream.\n - The result is needed for correctness\n - Failure could corrupt data or state\n - You're in a loop processing items\n\n- **No → DEBUG log only.** This is rare. Only when ALL of these are true:\n - A valid fallback path exists that doesn't depend on the result\n - Failure cannot affect correctness\n - The operation is purely supplementary (e.g., reading an optimization hint)\n\nWhen in doubt, return the error. In a disaster recovery tool, silent failures are worse than noisy ones.\n\n### DO: Return errors immediately\n\n```go\n// CORRECT - Return error for caller to handle\nfunc (db *DB) validatePosition() error {\n dpos, err := db.Pos()\n if err != nil {\n return err\n }\n rpos := replica.Pos()\n if dpos.TXID < rpos.TXID {\n return fmt.Errorf(\"database position (%v) behind replica (%v)\", dpos, rpos)\n }\n return nil\n}\n```\n\n### DON'T: Continue on critical errors\n\n```go\n// WRONG - Silently continuing can cause data corruption\nfunc (db *DB) validatePosition() {\n if dpos, _ := db.Pos(); dpos.TXID < replica.Pos().TXID {\n log.Printf(\"warning: position mismatch\") // Don't just log!\n // Continuing here is dangerous\n }\n}\n```\n\n### DON'T: Ignore errors and continue in loops\n\n```go\n// WRONG - Continuing after error can corrupt state\nfunc (db *DB) processFiles() {\n for _, file := range files {\n if err := processFile(file); err != nil {\n log.Printf(\"error: %v\", err) // Just logging!\n // Continuing to next file is dangerous\n }\n }\n}\n```\n\n### DO: Return errors properly in loops\n\n```go\n// CORRECT - Let caller decide how to handle errors\nfunc (db *DB) processFiles() error {\n for _, file := range files {\n if err := processFile(file); err != nil {\n return fmt.Errorf(\"process file %s: %w\", file, err)\n }\n }\n return nil\n}\n```\n\n## Locking Patterns\n\n### DO: Use proper lock types\n\n```go\n// CORRECT - Use Lock() for writes\nr.mu.Lock()\ndefer r.mu.Unlock()\nr.pos = pos\n```\n\n### DON'T: Use RLock for write operations\n\n```go\n// WRONG - Race condition\nr.mu.RLock() // Should be Lock() for writes\ndefer r.mu.RUnlock()\nr.pos = pos // Writing with RLock!\n```\n\n## Compaction and Eventual Consistency\n\nMany storage backends (S3, R2, etc.) are eventually consistent:\n\n- A file you just wrote might not be immediately readable\n- A file might be listed but only partially available\n- Reads might return stale or incomplete data\n\n### DO: Read from local when available\n\n```go\n// CORRECT - Check local first during compaction\n// db.go:1280-1294 - ALWAYS read from local disk when available\nf, err := os.Open(db.LTXPath(info.Level, info.MinTXID, info.MaxTXID))\nif err == nil {\n // Use local file - it's complete and consistent\n return f, nil\n}\n// Only fall back to remote if local doesn't exist\nreturn replica.Client.OpenLTXFile(...)\n```\n\n### DON'T: Read from remote during compaction\n\n```go\n// WRONG - Can get partial/corrupt data from eventually consistent storage\nf, err := client.OpenLTXFile(ctx, level, minTXID, maxTXID, 0, 0)\n```\n\n## Resumable Reader Pattern\n\nDuring restore, LTX file streams from S3/Tigris may sit idle while the compactor processes lower-numbered pages from the snapshot. Storage providers close these idle connections, causing \"unexpected EOF\" errors.\n\nThe `ResumableReader` (`internal/resumable_reader.go`) wraps `io.ReadCloser` with automatic reconnection:\n\n### Interface\n\n```go\ntype LTXFileOpener interface {\n OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n}\n```\n\n### Two Failure Modes\n\n1. **Non-EOF errors** (connection reset, timeout) — stream broke mid-transfer\n2. **Premature EOF** — server closed cleanly, but `offset < size` (known file size)\n\n### Key Behaviors\n\n- Max 3 retries (`resumableReaderMaxRetries = 3`)\n- Tracks current byte `offset` for range-request resume\n- Returns partial reads without error so callers like `io.ReadFull` naturally retry\n- Reopens from current offset using `OpenLTXFile(ctx, level, min, max, offset, 0)`\n\n### DO: Use ResumableReader for restore streams\n\n```go\nrc, _ := client.OpenLTXFile(ctx, level, min, max, 0, fileInfo.Size)\nrr := internal.NewResumableReader(ctx, client, level, min, max, fileInfo.Size, rc, logger)\ndefer rr.Close()\n```\n\n### DON'T: Use raw OpenLTXFile during long restore operations\n\n```go\nrc, _ := client.OpenLTXFile(ctx, level, min, max, 0, 0)\n// Risk: connection may drop during idle periods in multi-file restore\nio.ReadFull(rc, buf) // unexpected EOF!\n```\n\n## Retention Bypass Pattern\n\nWhen using cloud provider lifecycle policies (S3 lifecycle rules, R2 auto-cleanup), Litestream's active file deletion can be disabled:\n\n```yaml\nretention:\n enabled: false\n```\n\n### Propagation Chain\n\n1. `RetentionConfig{Enabled *bool}` in YAML config (`cmd/litestream/main.go`)\n2. `Store.SetRetentionEnabled(bool)` propagates to all DBs and their compactors (`store.go`)\n3. `Compactor.RetentionEnabled` guards 3 deletion points in `compactor.go`\n\n### DO: Disable retention when cloud lifecycle handles cleanup\n\n```go\nstore.SetRetentionEnabled(false) // Delegates deletion to cloud lifecycle policies\n```\n\n### DON'T: Disable retention without cloud lifecycle policies\n\nDisabling retention without cloud lifecycle policies causes unbounded storage growth. Litestream logs a warning: \"retention disabled; cloud provider lifecycle policies must handle retention\".\n\n## Conditional Write Pattern (Distributed Locking)\n\nThe S3 leaser (`s3/leaser.go`) uses S3 conditional writes (`If-Match`/`If-None-Match`) for distributed locking without an external coordination service.\n\n### Acquire Pattern\n\n```go\ninput := &s3.PutObjectInput{\n Bucket: aws.String(l.Bucket),\n Key: aws.String(key),\n Body: bytes.NewReader(data),\n}\nif etag == \"\" {\n input.IfNoneMatch = aws.String(\"*\") // First acquire: only if key doesn't exist\n} else {\n input.IfMatch = aws.String(etag) // Expired takeover: only if ETag matches\n}\n```\n\n### Release Pattern\n\n```go\n_, err := l.s3.DeleteObject(ctx, &s3.DeleteObjectInput{\n Bucket: aws.String(l.Bucket),\n Key: aws.String(key),\n IfMatch: aws.String(lease.ETag), // Only delete if we still hold the lease\n})\n```\n\n### Error Handling\n\n- **HTTP 412 (PreconditionFailed)**: Another instance acquired/renewed the lease\n- **HTTP 404 (NoSuchKey)**: Lease already released\n\n## Timestamp Preservation\n\nDuring compaction, preserve the earliest CreatedAt timestamp from source files to maintain temporal granularity for point-in-time restoration.\n\n### DO: Preserve earliest timestamp\n\n```go\n// CORRECT - Preserve temporal information\ninfo, err := replica.Client.WriteLTXFile(ctx, level, minTXID, maxTXID, r)\nif err != nil {\n return fmt.Errorf(\"write ltx: %w\", err)\n}\ninfo.CreatedAt = oldestSourceFile.CreatedAt\n```\n\n### DON'T: Ignore CreatedAt preservation\n\n```go\n// WRONG - Loses timestamp granularity for point-in-time restores\ninfo := <x.FileInfo{\n CreatedAt: time.Now(), // Don't use current time during compaction\n}\n```\n\n## Common Pitfalls\n\n### 1. Mixing architectural concerns\n\n```go\n// WRONG - Database state logic in Replica layer\nfunc (r *Replica) Start() error {\n if db.needsRestore() { // Wrong layer for DB state!\n r.restoreDatabase() // Replica shouldn't manage DB state!\n }\n return r.sync()\n}\n```\n\n### 2. Recreating existing functionality\n\n```go\n// WRONG - Don't reimplement what already exists\nfunc customSnapshotTrigger() {\n // Complex custom logic to trigger snapshots\n // when db.verify() already does this!\n}\n```\n\n### DO: Leverage existing mechanisms\n\n```go\n// CORRECT - Use what's already there\nfunc triggerSnapshot() error {\n return db.verify() // Already handles snapshot logic correctly\n}\n```\n\n### 3. Skipping the lock page\n\nThe lock page at 1GB (0x40000000) must always be skipped:\n\n```go\n// db.go:951-953 - Must skip lock page during replication\nlockPgno := ltx.LockPgno(pageSize)\nif pgno == lockPgno {\n continue // Skip this page - it's reserved by SQLite\n}\n```\n\nLock page numbers by page size:\n\n| Page Size | Lock Page Number |\n|-----------|------------------|\n| 4KB | 262145 |\n| 8KB | 131073 |\n| 16KB | 65537 |\n| 32KB | 32769 |\n\n## Component Reference\n\n### DB Component (db.go)\n\n**Responsibilities:**\n\n- Manages SQLite database connection (via `modernc.org/sqlite` - no CGO)\n- Monitors WAL for changes\n- Performs checkpoints\n- Maintains long-running read transaction\n- Converts WAL pages to LTX format\n\n**Key Fields:**\n\n```go\ntype DB struct {\n path string // Database file path\n db *sql.DB // SQLite connection\n rtx *sql.Tx // Long-running read transaction\n pageSize int // Database page size (critical for lock page)\n notify chan struct{} // Notifies on WAL changes\n}\n```\n\n**Initialization Sequence:**\n\n1. Open database connection\n2. Read page size from database\n3. Initialize long-running read transaction\n4. Start monitor goroutine\n5. Initialize replicas\n\n### Replica Component (replica.go)\n\n**Responsibilities:**\n\n- Manages replication to a single destination (one replica per DB)\n- Tracks replication position (ltx.Pos)\n- Handles sync intervals\n- Manages encryption (if configured)\n\n**Key Operations:**\n\n- `Sync()`: Synchronizes pending changes\n- `SetPos()`: Updates replication position (must use Lock, not RLock!)\n- `Snapshot()`: Creates full database snapshot\n\n### ReplicaClient Interface (replica_client.go)\n\n**Required Methods:**\n\n```go\ntype ReplicaClient interface {\n Type() string // Client type identifier\n\n // File operations\n LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error)\n OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error)\n DeleteLTXFiles(ctx context.Context, files []*ltx.FileInfo) error\n DeleteAll(ctx context.Context) error\n}\n```\n\n**useMetadata Parameter:**\n\n- `useMetadata=true`: Fetch accurate timestamps from backend metadata (required for point-in-time restores)\n- `useMetadata=false`: Use fast timestamps for normal operations\n\n### Compactor Component (compactor.go)\n\n**Responsibilities:**\n\n- Compaction and retention for LTX files\n- Operates solely through `ReplicaClient` interface\n- Suitable for both DB (with local file caching) and VFS (remote-only)\n\n**Key Fields:**\n\n```go\ntype Compactor struct {\n client ReplicaClient\n VerifyCompaction bool // Post-compaction TXID consistency check\n RetentionEnabled bool // Default: true. Controls active file deletion\n\n // Local file optimization (set by DB layer)\n LocalFileOpener func(level int, minTXID, maxTXID ltx.TXID) (io.ReadCloser, error)\n LocalFileDeleter func(level int, minTXID, maxTXID ltx.TXID) error\n\n // Level max-file-info caching\n CacheGetter func(level int) (*ltx.FileInfo, bool)\n CacheSetter func(level int, info *ltx.FileInfo)\n}\n```\n\n### Store Component (store.go)\n\n**Default Compaction Levels:**\n\n```go\nvar defaultLevels = CompactionLevels{\n {Level: 0, Interval: 0}, // Raw LTX files (no compaction)\n {Level: 1, Interval: 30*Second},\n {Level: 2, Interval: 5*Minute},\n {Level: 3, Interval: 1*Hour},\n // Snapshots created daily (24h retention)\n}\n```\n\n## Testing Patterns\n\n### Race Condition Testing\n\n```bash\n# Always run with race detector\ngo test -race -v ./...\n\n# Specific race-prone areas\ngo test -race -v -run TestReplica_Sync ./...\ngo test -race -v -run TestDB_Sync ./...\ngo test -race -v -run TestStore_CompactDB ./...\n```\n\n### Lock Page Testing\n\n```bash\n# Test with various page sizes\n./bin/litestream-test populate -db test.db -page-size 4096 -target-size 2GB\n./bin/litestream-test populate -db test.db -page-size 8192 -target-size 2GB\n\n# Validate lock page handling\n./bin/litestream-test validate -source-db test.db -replica-url file:///tmp/replica\n```\n\n### Integration Testing\n\n```bash\n# Test specific backend\ngo test -v ./replica_client_test.go -integration s3\ngo test -v ./replica_client_test.go -integration gcs\ngo test -v ./replica_client_test.go -integration abs\ngo test -v ./replica_client_test.go -integration oss\ngo test -v ./replica_client_test.go -integration sftp\n```\n" }, { "path": "docs/PENDING_USER_DOCS.md", "content": "# Pending User-Facing Documentation\n\nThis file tracks open issues on [benbjohnson/litestream.io](https://github.com/benbjohnson/litestream.io) that need user-facing documentation for features recently added to Litestream.\n\n## Open Issues\n\n| Issue | Title | Related Litestream PR | Category |\n|-------|-------|-----------------------|----------|\n| [#240](https://github.com/benbjohnson/litestream.io/issues/240) | `skip-remote-deletion` config option | #1094 | Config |\n| [#241](https://github.com/benbjohnson/litestream.io/issues/241) | SyncStatus, SyncAndWait, EnsureExists API | #1092 | Library API |\n| [#242](https://github.com/benbjohnson/litestream.io/issues/242) | Embedded CA bundle eliminates certificate setup | #1099 | Docker |\n| [#243](https://github.com/benbjohnson/litestream.io/issues/243) | S3 URL query parameters and R2 defaults | #1100, #1101 | S3/Cloud |\n| [#244](https://github.com/benbjohnson/litestream.io/issues/244) | Distributed leasing documentation | #1073 | S3/Cloud |\n| [#245](https://github.com/benbjohnson/litestream.io/issues/245) | `-level` flag for ltx command | #1072 | Reference |\n| [#246](https://github.com/benbjohnson/litestream.io/issues/246) | IPC Unix socket endpoints | #1021 | Reference |\n| [#247](https://github.com/benbjohnson/litestream.io/issues/247) | `$PID` environment variable in config | #1070 | Config |\n| [#248](https://github.com/benbjohnson/litestream.io/issues/248) | `validation-interval` config option | — | Config |\n| [#249](https://github.com/benbjohnson/litestream.io/issues/249) | Automatic v0.3.x backup restore | #1074, #1075 | Restore |\n| [#250](https://github.com/benbjohnson/litestream.io/issues/250) | MCP tools for v0.5.x changes | — | MCP |\n\n## AI Documentation Status\n\nInternal AI documentation (this repo) is tracked in [#1106](https://github.com/benbjohnson/litestream/issues/1106).\n" }, { "path": "docs/PROVIDER_COMPATIBILITY.md", "content": "# Storage Provider Compatibility Guide\n\nThis document details S3-compatible storage provider compatibility with Litestream,\nincluding known limitations, required configuration, and tested configurations.\n\n## Overview\n\nLitestream uses the AWS SDK v2 for S3-compatible storage backends. While most providers\nimplement the S3 API, there are important differences in behavior that can affect\nLitestream's operation.\n\n## Provider-Specific Configuration\n\n### AWS S3 (Default)\n\n**Status**: Fully supported (primary target)\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path\n region: us-east-1\n```\n\n**Notes**:\n\n- No special configuration required\n- All features fully supported\n- Checksum validation enabled by default\n\n### Cloudflare R2\n\n**Status**: Supported with default configuration\n\n**Known Limitations**:\n\n- Strict concurrent upload limit (2-3 concurrent uploads max)\n- Does not support `aws-chunked` content encoding\n- Does not support request/response checksums\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://ACCOUNT_ID.r2.cloudflarestorage.com\n access-key-id: your-access-key-id\n secret-access-key: your-secret-access-key\n```\n\n**Automatic Defaults** (applied when R2 endpoint detected):\n\n- `sign-payload=true` - Signed payloads required\n- `concurrency=2` - Limits concurrent multipart upload parts\n- Checksums disabled automatically\n\n**Important**: The endpoint must use `https://` scheme for R2 detection to work.\n\nRelated issues: #948, #947, #940, #941\n\n### Backblaze B2 (S3-Compatible API)\n\n**Status**: Supported with configuration\n\n**Known Limitations**:\n\n- Requires signed payloads for all requests\n- Specific authentication endpoint required\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://s3.REGION.backblazeb2.com&sign-payload=true&force-path-style=true\n access-key-id: your-key-id\n secret-access-key: your-application-key\n```\n\n**Required Settings**:\n\n- `sign-payload=true` - Required for B2 authentication\n- `force-path-style=true` - Required for bucket access\n- Endpoint format: `https://s3.REGION.backblazeb2.com`\n\nRelated issues: #918, #894\n\n### DigitalOcean Spaces\n\n**Status**: Supported with configuration\n\n**Known Limitations**:\n\n- Does not support `aws-chunked` content encoding\n- Signature requirements differ from AWS\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://REGION.digitaloceanspaces.com&force-path-style=false\n access-key-id: your-spaces-key\n secret-access-key: your-spaces-secret\n```\n\n**Notes**:\n\n- Use virtual-hosted style paths (force-path-style=false)\n- Checksum features disabled automatically for custom endpoints\n\nRelated issues: #943\n\n### MinIO\n\n**Status**: Fully supported\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://your-minio-server:9000&force-path-style=true\n access-key-id: your-access-key\n secret-access-key: your-secret-key\n```\n\n**Notes**:\n\n- Works well with default settings\n- Force path style recommended for single-server deployments\n\n### Scaleway Object Storage\n\n**Status**: Supported with configuration\n\n**Known Limitations**:\n\n- `MissingContentLength` errors with streaming uploads\n- Requires Content-Length header\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://s3.REGION.scw.cloud&force-path-style=true\n access-key-id: your-access-key\n secret-access-key: your-secret-key\n```\n\nRelated issues: #912\n\n### Hetzner Object Storage\n\n**Status**: Supported with configuration\n\n**Known Limitations**:\n\n- `InvalidArgument` errors with default AWS SDK settings\n- Does not support `aws-chunked` content encoding\n- Requires signed payloads for all requests\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://REGION.your-objectstorage.com&force-path-style=true\n access-key-id: your-access-key\n secret-access-key: your-secret-key\n```\n\n### Filebase\n\n**Status**: Supported with configuration\n\n**Known Limitations**:\n\n- Authentication failures with default SDK settings after SDK v2 migration\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://s3.filebase.com&force-path-style=true\n access-key-id: your-access-key\n secret-access-key: your-secret-key\n```\n\n### Tigris\n\n**Status**: Supported with configuration\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://fly.storage.tigris.dev&force-path-style=true\n access-key-id: your-access-key\n secret-access-key: your-secret-key\n```\n\n### Supabase Storage (S3-Compatible API)\n\n**Status**: Supported (auto-detected)\n\n**Known Limitations**:\n\n- Requires path-style URLs (`force-path-style=true`)\n- No S3 versioning support\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://PROJECT_REF.supabase.co/storage/v1/s3\n access-key-id: your-s3-access-key\n secret-access-key: your-s3-secret-key\n```\n\n**Automatic Defaults** (applied when Supabase endpoint detected):\n\n- `sign-payload=true` - Signed payloads required\n- `force-path-style=true` - Path-style URLs required\n\n**Notes**:\n\n- S3 access keys are generated in the Supabase dashboard under Storage > S3 Access Keys\n- Endpoint format: `https://.supabase.co/storage/v1/s3`\n\nRelated issues: #1133\n\n### Wasabi\n\n**Status**: Supported\n\n**Configuration**:\n\n```yaml\nreplicas:\n - url: s3://bucket-name/path?endpoint=https://s3.REGION.wasabisys.com\n access-key-id: your-access-key\n secret-access-key: your-secret-key\n```\n\n## Google Cloud Storage (GCS)\n\n**Status**: Fully supported (native client)\n\n```yaml\nreplicas:\n - url: gcs://bucket-name/path\n```\n\n**Authentication**:\n\n- Uses Application Default Credentials\n- Set `GOOGLE_APPLICATION_CREDENTIALS` environment variable\n- Or use workload identity on GCP\n\n## Azure Blob Storage (ABS)\n\n**Status**: Fully supported (native client)\n\n```yaml\nreplicas:\n - url: abs://container-name/path\n account-name: your-account-name\n account-key: your-account-key\n```\n\n**Using SAS Token** (for granular container-level access):\n\n```yaml\nreplicas:\n - url: abs://container-name/path\n account-name: your-account-name\n sas-token: \"sv=2023-01-03&ss=b&srt=co&sp=rwdlacx...\"\n```\n\nOr via environment variable: `LITESTREAM_AZURE_SAS_TOKEN`\n\n**Alternative Authentication**:\n\n- SAS token: `sas-token` config or `LITESTREAM_AZURE_SAS_TOKEN` env var\n- Account key: `account-key` config or `LITESTREAM_AZURE_ACCOUNT_KEY` env var\n- Managed identity on Azure (via DefaultAzureCredential)\n\n**Authentication Priority**: SAS token > Account key > Default credential chain\n\n## Alibaba Cloud OSS\n\n**Status**: Supported (native client)\n\n```yaml\nreplicas:\n - url: oss://bucket-name/path?endpoint=oss-REGION.aliyuncs.com\n access-key-id: your-access-key-id\n access-key-secret: your-access-key-secret\n```\n\n## SFTP\n\n**Status**: Supported\n\n```yaml\nreplicas:\n - url: sftp://hostname/path\n user: username\n password: password # or use key-path\n```\n\n## Configuration Reference\n\n### S3 Query Parameters\n\nParameters with an alias accept both camelCase and hyphenated forms\n(e.g., `forcePathStyle` or `force-path-style`).\n\n| Parameter | Alias | Description | Default |\n|-----------|-------|-------------|---------|\n| `endpoint` | | Custom S3 endpoint URL | AWS S3 |\n| `region` | | AWS region | Auto-detected |\n| `forcePathStyle` | `force-path-style` | Use path-style URLs | `false` (auto for custom endpoints) |\n| `skipVerify` | `skip-verify` | Skip TLS verification | `false` |\n| `signPayload` | `sign-payload` | Sign request payloads | `true` |\n| `requireContentMD5` | `require-content-md5` | Require Content-MD5 header | `true` |\n| `concurrency` | | Multipart upload concurrency | `5` |\n| `partSize` | `part-size` | Multipart upload part size | `5MB` |\n| `sseCustomerAlgorithm` | `sse-customer-algorithm` | SSE-C encryption algorithm | None |\n| `sseCustomerKey` | `sse-customer-key` | SSE-C encryption key | None |\n| `sseCustomerKeyMD5` | `sse-customer-key-md5` | SSE-C key MD5 checksum | None |\n| `sseKmsKeyId` | `sse-kms-key-id` | KMS key for encryption | None |\n\n### Provider Detection\n\nLitestream automatically detects certain providers and applies appropriate defaults:\n\n| Provider | Detection Pattern | Applied Settings |\n|----------|-------------------|------------------|\n| Hetzner | `*.your-objectstorage.com` | `sign-payload=true` |\n| Cloudflare R2 | `*.r2.cloudflarestorage.com` | `sign-payload=true`, `concurrency=2` |\n| Backblaze B2 | `*.backblazeb2.com` | `sign-payload=true`, `force-path-style=true` |\n| DigitalOcean | `*.digitaloceanspaces.com` | `sign-payload=true` |\n| Scaleway | `*.scw.cloud` | `sign-payload=true` |\n| Filebase | `s3.filebase.com` | `sign-payload=true`, `force-path-style=true` |\n| Tigris | `*.tigris.dev` | `sign-payload=true`, `require-content-md5=false` |\n| MinIO | host with port (not cloud provider) | `sign-payload=true`, `force-path-style=true` |\n| Supabase | `*.supabase.co` | `sign-payload=true`, `force-path-style=true` |\n\n## Troubleshooting\n\n### Common Errors\n\n**`InvalidArgument: Unsupported content encoding: aws-chunked`**\n\n- Provider doesn't support AWS SDK v2 chunked encoding\n- Use a custom endpoint with automatic checksum disabling\n- Or explicitly disable checksums\n\n**`SignatureDoesNotMatch`**\n\n- Try `sign-payload=true` in the URL\n- Verify credentials are correct\n- Check endpoint URL format\n\n**`MissingContentLength`**\n\n- Provider requires Content-Length header\n- This is handled automatically for known providers\n\n**`Too many concurrent uploads` or timeout errors**\n\n- Reduce concurrency: `?concurrency=2`\n- Particularly important for Cloudflare R2\n\n**`AccessDenied` or authentication failures**\n\n- Verify credentials\n- Check IAM/bucket permissions\n- For B2, ensure `sign-payload=true`\n\n### Debug Mode\n\nEnable verbose logging to diagnose issues:\n\n```bash\nLITESTREAM_DEBUG=1 litestream replicate ...\n```\n\nOr in configuration:\n\n```yaml\nlogging:\n level: debug\n```\n\n## Testing Your Configuration\n\nTest connectivity without starting replication:\n\n```bash\n# List any existing backups\nlitestream snapshots s3://bucket/path?endpoint=...\n\n# Perform a test restore (requires existing backup)\nlitestream restore -o /tmp/test.db s3://bucket/path?endpoint=...\n```\n\n## Version Compatibility\n\n- **Litestream v0.5.x**: AWS SDK v2, improved provider compatibility\n- **Litestream v0.4.x**: AWS SDK v1, different authentication handling\n- **Litestream v0.3.x**: Legacy format — v0.5.x can restore from v0.3.x backups via `ReplicaClientV3` interface\n\nWhen upgrading from v0.3.x, v0.5.x can automatically restore from v0.3.x backups if no v0.4.x+ backup exists. The S3 backend implements `ReplicaClientV3` to read the v0.3.x `generations/{id}/snapshots/` and `generations/{id}/wal/` directory structure. See [REPLICA_CLIENT_GUIDE.md](REPLICA_CLIENT_GUIDE.md#replicaclientv3-interface-v03x-restore) for details.\n\n### Validation Interval\n\nLitestream supports periodic validation of replica integrity:\n\n```yaml\nvalidation:\n interval: \"5m\" # How often to validate; 0 disables\n```\n\nThis runs `Store.Validate()` on a ticker, comparing local and remote positions. Can also be set per-replica via the `validation-interval` replica config key.\nand cannot restore backups created by v0.3.x. See the upgrade guide for migration\ninstructions.\n\n## Reporting Issues\n\nWhen reporting provider compatibility issues, please include:\n\n1. Provider name and region\n2. Litestream version (`litestream version`)\n3. Full error message\n4. Configuration (with credentials redacted)\n5. Whether the issue is with replication, restore, or both\n\nFile issues at: [GitHub Issues](https://github.com/benbjohnson/litestream/issues)\n" }, { "path": "docs/REPLICA_CLIENT_GUIDE.md", "content": "# ReplicaClient Implementation Guide\n\nThis guide provides comprehensive instructions for implementing new storage backends for Litestream replication.\n\n## Table of Contents\n\n- [Interface Contract](#interface-contract)\n- [Implementation Checklist](#implementation-checklist)\n- [Eventual Consistency Handling](#eventual-consistency-handling)\n- [Error Handling](#error-handling)\n- [Testing Requirements](#testing-requirements)\n- [ReplicaClientV3 Interface (v0.3.x Restore)](#replicaclientv3-interface-v03x-restore)\n- [Common Implementation Mistakes](#common-implementation-mistakes)\n- [Reference Implementations](#reference-implementations)\n\n## Interface Contract\n\nAll replica clients MUST implement the `ReplicaClient` interface defined in `replica_client.go`:\n\n```go\ntype ReplicaClient interface {\n // Returns the type identifier (e.g., \"s3\", \"gcs\", \"file\")\n Type() string\n\n // Returns iterator of LTX files at given level\n // seek: Start from this TXID (0 = beginning)\n // useMetadata: When true, fetch accurate timestamps from backend metadata (required for PIT restore)\n LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error)\n\n // Opens an LTX file for reading\n // Returns os.ErrNotExist if file doesn't exist\n OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n\n // Writes an LTX file to storage\n // SHOULD set CreatedAt based on backend metadata or upload time\n WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error)\n\n // Deletes one or more LTX files\n DeleteLTXFiles(ctx context.Context, files []*ltx.FileInfo) error\n\n // Deletes all files for this database\n DeleteAll(ctx context.Context) error\n}\n```\n\n## Implementation Checklist\n\n### Required Features\n\n- [ ] Implement all interface methods\n- [ ] Support partial reads (offset/size in OpenLTXFile)\n- [ ] Return proper error types (especially os.ErrNotExist)\n- [ ] Handle context cancellation\n- [ ] Preserve file timestamps (CreatedAt)\n- [ ] Support concurrent operations\n- [ ] Implement proper cleanup in DeleteAll\n\n### Optional Features\n\n- [ ] Connection pooling\n- [ ] Retry logic with exponential backoff\n- [ ] Request batching\n- [ ] Compression\n- [ ] Encryption at rest\n- [ ] Bandwidth throttling\n\n## Eventual Consistency Handling\n\nMany cloud storage services exhibit eventual consistency, where:\n- A file you just wrote might not be immediately visible\n- A file might be listed but only partially readable\n- Deletes might not take effect immediately\n\n### Best Practices\n\n#### 1. Write-After-Write Consistency\n\n```go\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n // Buffer the entire content first\n data, err := io.ReadAll(r)\n if err != nil {\n return nil, fmt.Errorf(\"buffer ltx data: %w\", err)\n }\n\n // Calculate checksum before upload\n checksum := crc64.Checksum(data, crc64.MakeTable(crc64.ECMA))\n\n // Upload with checksum verification\n err = c.uploadWithVerification(ctx, path, data, checksum)\n if err != nil {\n return nil, err\n }\n\n // Verify the file is readable before returning\n return c.verifyUpload(ctx, path, int64(len(data)), checksum)\n}\n\nfunc (c *ReplicaClient) verifyUpload(ctx context.Context, path string, expectedSize int64, expectedChecksum uint64) (*ltx.FileInfo, error) {\n // Implement retry loop with backoff\n backoff := 100 * time.Millisecond\n for i := 0; i < 10; i++ {\n info, err := c.statFile(ctx, path)\n if err == nil {\n if info.Size == expectedSize {\n rc, err := c.openFile(ctx, path, 0, 0)\n if err != nil {\n return nil, fmt.Errorf(\"open uploaded file: %w\", err)\n }\n data, err := io.ReadAll(rc)\n rc.Close()\n if err != nil {\n return nil, fmt.Errorf(\"read uploaded file: %w\", err)\n }\n if crc64.Checksum(data, crc64.MakeTable(crc64.ECMA)) == expectedChecksum {\n return info, nil\n }\n }\n }\n\n select {\n case <-ctx.Done():\n return nil, ctx.Err()\n case <-time.After(backoff):\n backoff *= 2\n }\n }\n return nil, errors.New(\"upload verification failed\")\n}\n```\n\n#### 2. List-After-Write Consistency\n\n```go\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n // List files from storage\n files, err := c.listFiles(ctx, level, useMetadata)\n if err != nil {\n return nil, err\n }\n\n // Sort by TXID for consistent ordering\n sort.Slice(files, func(i, j int) bool {\n if files[i].MinTXID != files[j].MinTXID {\n return files[i].MinTXID < files[j].MinTXID\n }\n return files[i].MaxTXID < files[j].MaxTXID\n })\n\n // Filter by seek position\n var filtered []*ltx.FileInfo\n for _, f := range files {\n if f.MinTXID >= seek {\n filtered = append(filtered, f)\n }\n }\n\n return ltx.NewFileInfoSliceIterator(filtered), nil\n}\n```\n\n#### 3. Read-After-Write Consistency\n\n```go\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n path := c.ltxPath(level, minTXID, maxTXID)\n\n // For eventually consistent backends, implement retry\n var lastErr error\n backoff := 100 * time.Millisecond\n\n for i := 0; i < 5; i++ {\n reader, err := c.openFile(ctx, path, offset, size)\n if err == nil {\n return reader, nil\n }\n\n // Don't retry on definitive errors\n if errors.Is(err, os.ErrNotExist) {\n return nil, err\n }\n\n lastErr = err\n select {\n case <-ctx.Done():\n return nil, ctx.Err()\n case <-time.After(backoff):\n backoff *= 2\n }\n }\n\n return nil, fmt.Errorf(\"open file after retries: %w\", lastErr)\n}\n```\n\n## Error Handling\n\n### Standard Error Types\n\nAlways return appropriate standard errors:\n\n```go\n// File not found\nreturn nil, os.ErrNotExist\n\n// Permission denied\nreturn nil, os.ErrPermission\n\n// Context cancelled\nreturn nil, ctx.Err()\n\n// Custom errors should wrap standard ones\nreturn nil, fmt.Errorf(\"s3 download failed: %w\", err)\n```\n\n### Error Classification\n\n```go\n// Retryable errors\nfunc isRetryable(err error) bool {\n // Network errors\n var netErr net.Error\n if errors.As(err, &netErr) && netErr.Temporary() {\n return true\n }\n\n // Specific HTTP status codes\n if httpErr, ok := err.(HTTPError); ok {\n switch httpErr.StatusCode {\n case 429, 500, 502, 503, 504:\n return true\n }\n }\n\n // Timeout errors\n if errors.Is(err, context.DeadlineExceeded) {\n return true\n }\n\n return false\n}\n```\n\n### Logging Best Practices\n\n```go\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n logger := slog.Default().With(\n \"replica\", c.Type(),\n \"level\", level,\n \"minTXID\", minTXID,\n \"maxTXID\", maxTXID,\n )\n\n logger.Debug(\"starting ltx upload\")\n\n info, err := c.upload(ctx, level, minTXID, maxTXID, r)\n if err != nil {\n logger.Error(\"ltx upload failed\", \"error\", err)\n return nil, err\n }\n\n logger.Info(\"ltx upload complete\", \"size\", info.Size)\n return info, nil\n}\n```\n\n## Testing Requirements\n\n### Unit Tests\n\nEvery replica client MUST have comprehensive unit tests:\n\n```go\n// replica_client_test.go\nfunc TestReplicaClient_WriteLTXFile(t *testing.T) {\n client := NewReplicaClient(testConfig)\n ctx := context.Background()\n\n // Test data\n data := []byte(\"test ltx content\")\n reader := bytes.NewReader(data)\n\n // Write file\n info, err := client.WriteLTXFile(ctx, 0, 1, 100, reader)\n assert.NoError(t, err)\n assert.Equal(t, int64(len(data)), info.Size)\n\n // Verify file exists\n rc, err := client.OpenLTXFile(ctx, 0, 1, 100, 0, 0)\n assert.NoError(t, err)\n defer rc.Close()\n\n // Read and verify content\n content, err := io.ReadAll(rc)\n assert.NoError(t, err)\n assert.Equal(t, data, content)\n}\n\nfunc TestReplicaClient_PartialRead(t *testing.T) {\n client := NewReplicaClient(testConfig)\n ctx := context.Background()\n\n // Write test file\n data := bytes.Repeat([]byte(\"x\"), 1000)\n _, err := client.WriteLTXFile(ctx, 0, 1, 100, bytes.NewReader(data))\n require.NoError(t, err)\n\n // Test partial read\n rc, err := client.OpenLTXFile(ctx, 0, 1, 100, 100, 50)\n require.NoError(t, err)\n defer rc.Close()\n\n partial, err := io.ReadAll(rc)\n assert.NoError(t, err)\n assert.Equal(t, 50, len(partial))\n assert.Equal(t, data[100:150], partial)\n}\n\nfunc TestReplicaClient_NotFound(t *testing.T) {\n client := NewReplicaClient(testConfig)\n ctx := context.Background()\n\n // Try to open non-existent file\n _, err := client.OpenLTXFile(ctx, 0, 999, 999, 0, 0)\n assert.True(t, errors.Is(err, os.ErrNotExist))\n}\n```\n\n### Integration Tests\n\nIntegration tests run against real backends:\n\n```go\n// +build integration\n\nfunc TestReplicaClient_Integration(t *testing.T) {\n // Skip if not in integration mode\n if testing.Short() {\n t.Skip(\"skipping integration test\")\n }\n\n // Get credentials from environment\n config := ConfigFromEnv(t)\n client := NewReplicaClient(config)\n ctx := context.Background()\n\n t.Run(\"Concurrent Writes\", func(t *testing.T) {\n var wg sync.WaitGroup\n errors := make(chan error, 10)\n\n for i := 0; i < 10; i++ {\n wg.Add(1)\n go func(n int) {\n defer wg.Done()\n\n data := []byte(fmt.Sprintf(\"concurrent %d\", n))\n minTXID := ltx.TXID(n * 100)\n maxTXID := ltx.TXID((n + 1) * 100)\n\n _, err := client.WriteLTXFile(ctx, 0, minTXID, maxTXID,\n bytes.NewReader(data))\n if err != nil {\n errors <- err\n }\n }(i)\n }\n\n wg.Wait()\n close(errors)\n\n for err := range errors {\n t.Error(err)\n }\n })\n\n t.Run(\"Large File\", func(t *testing.T) {\n // Test with 100MB file\n data := bytes.Repeat([]byte(\"x\"), 100*1024*1024)\n\n info, err := client.WriteLTXFile(ctx, 0, 1000, 2000,\n bytes.NewReader(data))\n require.NoError(t, err)\n assert.Equal(t, int64(len(data)), info.Size)\n })\n\n t.Run(\"Cleanup\", func(t *testing.T) {\n err := client.DeleteAll(ctx)\n assert.NoError(t, err)\n\n // Verify cleanup\n iter, err := client.LTXFiles(ctx, 0, 0, false)\n require.NoError(t, err)\n defer iter.Close()\n\n assert.False(t, iter.Next(), \"files should be deleted\")\n })\n}\n```\n\n### Mock Client for Testing\n\nProvide a mock implementation for testing:\n\n```go\n// mock/replica_client.go\ntype ReplicaClient struct {\n mu sync.Mutex\n files map[string]*ltx.FileInfo\n data map[string][]byte\n errors map[string]error // Inject errors for testing\n}\n\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n c.mu.Lock()\n defer c.mu.Unlock()\n\n // Check for injected error\n key := fmt.Sprintf(\"write-%d-%d-%d\", level, minTXID, maxTXID)\n if err, ok := c.errors[key]; ok {\n return nil, err\n }\n\n // Store data\n data, err := io.ReadAll(r)\n if err != nil {\n return nil, err\n }\n\n path := ltxPath(level, minTXID, maxTXID)\n c.data[path] = data\n\n info := <x.FileInfo{\n Level: level,\n MinTXID: minTXID,\n MaxTXID: maxTXID,\n Size: int64(len(data)),\n CreatedAt: time.Now(),\n }\n c.files[path] = info\n\n return info, nil\n}\n```\n\n## ReplicaClientV3 Interface (v0.3.x Restore)\n\nBackends that need backward-compatible restore from v0.3.x Litestream backups should implement the optional `ReplicaClientV3` interface (`v3.go`). This enables restoring databases from pre-v0.4 backup formats.\n\n### Interface\n\n```go\ntype ReplicaClientV3 interface {\n GenerationsV3(ctx context.Context) ([]string, error)\n SnapshotsV3(ctx context.Context, generation string) ([]SnapshotInfoV3, error)\n WALSegmentsV3(ctx context.Context, generation string) ([]WALSegmentInfoV3, error)\n OpenSnapshotV3(ctx context.Context, generation string, index int) (io.ReadCloser, error)\n OpenWALSegmentV3(ctx context.Context, generation string, index int, offset int64) (io.ReadCloser, error)\n}\n```\n\n### Supporting Types\n\n```go\ntype PosV3 struct {\n Generation string // 16-char hex string\n Index int // WAL index\n Offset int64 // Offset within WAL segment\n}\n\ntype SnapshotInfoV3 struct {\n Generation string\n Index int\n Size int64\n CreatedAt time.Time\n}\n\ntype WALSegmentInfoV3 struct {\n Generation string\n Index int\n Offset int64\n Size int64\n CreatedAt time.Time\n}\n```\n\n### v0.3.x Path Structure\n\n```text\ngenerations/\n {generation-id}/ # 16-char hex (validated by IsGenerationIDV3)\n snapshots/\n {index:08x}.snapshot.lz4\n wal/\n {index:08x}_{offset:08x}.wal.lz4\n```\n\n### Implementation Notes\n\n- Generation IDs are 16 hex characters, validated by `IsGenerationIDV3()` (`generationRegexV3`)\n- All returned readers provide LZ4-decompressed data\n- `GenerationsV3` returns IDs sorted ascending\n- `SnapshotsV3` returns sorted by index\n- `WALSegmentsV3` returns sorted by index, then offset\n- S3 backend implements this: `var _ litestream.ReplicaClientV3 = (*ReplicaClient)(nil)`\n\n### ResumableReader Integration\n\n`OpenLTXFile` must support the `offset` parameter for range requests. The `ResumableReader` (`internal/resumable_reader.go`) reopens streams from the last successful byte offset on connection failures during restore. If your backend ignores `offset`, restore operations will fail on idle connection timeouts.\n\n## Common Implementation Mistakes\n\n### ❌ Mistake 1: Not Handling Partial Reads\n\n```go\n// WRONG - Always reads entire file\nfunc (c *Client) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n return c.storage.Download(path) // Ignores offset/size!\n}\n```\n\n```go\n// CORRECT - Respects offset and size\nfunc (c *Client) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n if offset == 0 && size == 0 {\n // Full file\n return c.storage.Download(path)\n }\n\n // Partial read using Range header or equivalent\n end := offset + size - 1\n if size == 0 {\n end = 0 // Read to end\n }\n return c.storage.DownloadRange(path, offset, end)\n}\n```\n\n### ❌ Mistake 2: Not Preserving CreatedAt\n\n```go\n// WRONG - Uses current time\nfunc (c *Client) WriteLTXFile(...) (*ltx.FileInfo, error) {\n // Upload file...\n\n return <x.FileInfo{\n CreatedAt: time.Now(), // Wrong! Loses temporal info\n }, nil\n}\n```\n\n```go\n// CORRECT - Preserves original timestamp\nfunc (c *Client) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n // Upload file...\n uploadedSize, modTime, err := c.storage.Upload(path, r)\n if err != nil {\n return nil, err\n }\n\n return <x.FileInfo{\n Level: level,\n MinTXID: minTXID,\n MaxTXID: maxTXID,\n Size: uploadedSize,\n CreatedAt: modTime,\n }, nil\n}\n```\n\n### ❌ Mistake 3: Wrong Error Types\n\n```go\n// WRONG - Generic error\nfunc (c *Client) OpenLTXFile(...) (io.ReadCloser, error) {\n resp, err := c.get(path)\n if err != nil {\n return nil, fmt.Errorf(\"not found\") // Wrong type!\n }\n}\n```\n\n```go\n// CORRECT - Proper error type\nfunc (c *Client) OpenLTXFile(...) (io.ReadCloser, error) {\n resp, err := c.get(path)\n if err != nil {\n if resp.StatusCode == 404 {\n return nil, os.ErrNotExist // Correct type\n }\n return nil, fmt.Errorf(\"download failed: %w\", err)\n }\n}\n```\n\n### ❌ Mistake 4: Not Handling Context\n\n```go\n// WRONG - Ignores context\nfunc (c *Client) WriteLTXFile(ctx context.Context, ...) (*ltx.FileInfo, error) {\n // Long operation without checking context\n for i := 0; i < 1000000; i++ {\n doWork() // Could run forever!\n }\n}\n```\n\n```go\n// CORRECT - Respects context\nfunc (c *Client) WriteLTXFile(ctx context.Context, ...) (*ltx.FileInfo, error) {\n // Check context periodically\n for i := 0; i < 1000000; i++ {\n select {\n case <-ctx.Done():\n return nil, ctx.Err()\n default:\n // Continue work\n }\n\n if err := doWork(ctx); err != nil {\n return nil, err\n }\n }\n}\n```\n\n### ❌ Mistake 5: Blocking in Iterator\n\n```go\n// WRONG - Loads all files at once\nfunc (c *Client) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n allFiles, err := c.loadAllFiles(level) // Could be millions!\n if err != nil {\n return nil, err\n }\n\n return NewIterator(allFiles), nil\n}\n```\n\n```go\n// CORRECT - Lazy loading with pagination\nfunc (c *Client) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n return &lazyIterator{\n client: c,\n level: level,\n seek: seek,\n useMetadata: useMetadata,\n pageSize: 1000,\n }, nil\n}\n\ntype lazyIterator struct {\n client *Client\n level int\n seek ltx.TXID\n pageSize int\n current []*ltx.FileInfo\n index int\n done bool\n}\n\nfunc (i *lazyIterator) Next() bool {\n if i.index >= len(i.current) && !i.done {\n // Load next page\n i.loadNextPage()\n }\n return i.index < len(i.current)\n}\n```\n\n## Reference Implementations\n\n### File System Client (Simplest)\n\nSee `file/replica_client.go` for the simplest implementation:\n- Direct file I/O operations\n- No network complexity\n- Good starting reference\n\n### S3 Client (Most Complex)\n\nSee `s3/replica_client.go` for advanced features:\n- Multipart uploads for large files\n- Retry logic with exponential backoff\n- Request signing\n- Eventual consistency handling\n\n### Key Patterns from S3 Implementation\n\n```go\n// Path construction\nfunc (c *ReplicaClient) ltxDir(level int) string {\n if level == SnapshotLevel {\n return path.Join(c.Path, \"snapshots\")\n }\n return path.Join(c.Path, \"ltx\", fmt.Sprintf(\"%04d\", level))\n}\n\n// Metadata handling\nfunc (c *ReplicaClient) WriteLTXFile(...) (*ltx.FileInfo, error) {\n // Add metadata to object\n metadata := map[string]string{\n \"min-txid\": fmt.Sprintf(\"%d\", minTXID),\n \"max-txid\": fmt.Sprintf(\"%d\", maxTXID),\n \"level\": fmt.Sprintf(\"%d\", level),\n }\n\n // Upload with metadata\n _, err := c.s3.PutObjectWithContext(ctx, &s3.PutObjectInput{\n Bucket: &c.Bucket,\n Key: &key,\n Body: r,\n Metadata: metadata,\n })\n}\n\n// Error mapping\nfunc mapS3Error(err error) error {\n if aerr, ok := err.(awserr.Error); ok {\n switch aerr.Code() {\n case s3.ErrCodeNoSuchKey:\n return os.ErrNotExist\n case s3.ErrCodeAccessDenied:\n return os.ErrPermission\n }\n }\n return err\n}\n```\n\n## Performance Optimization\n\n### Connection Pooling\n\n```go\ntype ReplicaClient struct {\n pool *ConnectionPool\n}\n\nfunc NewReplicaClient(config Config) *ReplicaClient {\n pool := &ConnectionPool{\n MaxConnections: config.MaxConnections,\n IdleTimeout: config.IdleTimeout,\n }\n\n return &ReplicaClient{\n pool: pool,\n }\n}\n```\n\n### Request Batching\n\n```go\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, files []*ltx.FileInfo) error {\n // Batch deletes for efficiency\n const batchSize = 100\n\n for i := 0; i < len(files); i += batchSize {\n end := i + batchSize\n if end > len(files) {\n end = len(files)\n }\n\n batch := files[i:end]\n if err := c.deleteBatch(ctx, batch); err != nil {\n return fmt.Errorf(\"delete batch %d: %w\", i/batchSize, err)\n }\n }\n\n return nil\n}\n```\n\n### Caching\n\n```go\ntype ReplicaClient struct {\n cache *FileInfoCache\n}\n\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n // Check cache first (only cache when useMetadata=false for fast queries)\n cacheKey := fmt.Sprintf(\"%d-%d\", level, seek)\n if !useMetadata {\n if cached, ok := c.cache.Get(cacheKey); ok {\n return ltx.NewFileInfoSliceIterator(cached), nil\n }\n }\n\n // Load from storage\n files, err := c.loadFiles(ctx, level, seek, useMetadata)\n if err != nil {\n return nil, err\n }\n\n // Cache for future requests\n c.cache.Set(cacheKey, files, 5*time.Minute)\n\n return ltx.NewFileInfoSliceIterator(files), nil\n}\n```\n\n## Checklist for New Implementations\n\nBefore submitting a new replica client:\n\n- [ ] All interface methods implemented\n- [ ] Unit tests with >80% coverage\n- [ ] Integration tests (with build tag)\n- [ ] Mock client for testing\n- [ ] Handles partial reads correctly\n- [ ] Returns proper error types\n- [ ] Preserves timestamps\n- [ ] Handles context cancellation\n- [ ] Documents eventual consistency behavior\n- [ ] Includes retry logic for transient errors\n- [ ] Logs appropriately (debug/info/error)\n- [ ] README with configuration examples\n- [ ] Added to main configuration parser\n\n## Getting Help\n\n1. Study existing implementations (start with `file/`, then `s3/`)\n2. Check test files for expected behavior\n3. Run integration tests against your backend\n4. Use the mock client for rapid development\n5. Open a GitHub issue for design feedback\n" }, { "path": "docs/SQLITE_INTERNALS.md", "content": "# SQLite Internals for Litestream\n\nThis document explains SQLite internals critical for understanding Litestream's operation.\n\n## Table of Contents\n- [SQLite File Structure](#sqlite-file-structure)\n- [Write-Ahead Log (WAL)](#write-ahead-log-wal)\n- [Page Structure](#page-structure)\n- [The 1GB Lock Page](#the-1gb-lock-page)\n- [Transaction Management](#transaction-management)\n- [Checkpoint Modes](#checkpoint-modes)\n- [Important SQLite Pragmas](#important-sqlite-pragmas)\n- [SQLite API Usage](#sqlite-api-usage)\n\n## SQLite File Structure\n\nSQLite databases consist of:\n1. **Main database file** - Contains actual data in pages\n2. **WAL file** (-wal suffix) - Contains uncommitted changes\n3. **SHM file** (-shm suffix) - Shared memory for coordination\n\n```\ndatabase.db # Main database file (pages)\ndatabase.db-wal # Write-ahead log\ndatabase.db-shm # Shared memory file\n```\n\n## Write-Ahead Log (WAL)\n\n### WAL Basics\n\nWAL is SQLite's method for implementing atomic commits and rollback:\n- Changes are first written to WAL\n- Original database file unchanged until checkpoint\n- Readers see consistent view through WAL\n\n```mermaid\ngraph LR\n Write[Write Transaction] -->|Append| WAL[WAL File]\n WAL -->|Checkpoint| DB[Main Database]\n Read[Read Transaction] -->|Merge View| View[Consistent View]\n DB --> View\n WAL --> View\n```\n\n### WAL File Structure\n\n```\n+------------------+\n| WAL Header | 32 bytes\n+------------------+\n| Frame 1 Header | 24 bytes\n| Frame 1 Data | Page size bytes\n+------------------+\n| Frame 2 Header | 24 bytes\n| Frame 2 Data | Page size bytes\n+------------------+\n| ... |\n+------------------+\n```\n\n#### WAL Header (32 bytes)\n```go\ntype WALHeader struct {\n Magic [4]byte // 0x377f0682 or 0x377f0683\n FileFormat uint32 // File format version (3007000)\n PageSize uint32 // Database page size\n Checkpoint uint32 // Checkpoint sequence number\n Salt1 uint32 // Random salt for checksum\n Salt2 uint32 // Random salt for checksum\n Checksum1 uint32 // Header checksum\n Checksum2 uint32 // Header checksum\n}\n```\n\n#### WAL Frame Header (24 bytes)\n```go\ntype WALFrameHeader struct {\n PageNumber uint32 // Page number in database\n DbSize uint32 // Size of database in pages\n Salt1 uint32 // Must match header salt\n Salt2 uint32 // Must match header salt\n Checksum1 uint32 // Cumulative checksum\n Checksum2 uint32 // Cumulative checksum\n}\n```\n\n### Reading WAL in Litestream\n\n```go\n// db.go - Reading WAL for replication\nfunc (db *DB) readWAL() ([]Page, error) {\n walPath := db.path + \"-wal\"\n f, err := os.Open(walPath)\n if err != nil {\n return nil, err\n }\n defer f.Close()\n\n // Read WAL header\n var header WALHeader\n binary.Read(f, binary.BigEndian, &header)\n\n // Validate magic number\n magic := binary.BigEndian.Uint32(header.Magic[:])\n if magic != 0x377f0682 && magic != 0x377f0683 {\n return nil, errors.New(\"invalid WAL magic\")\n }\n\n // Read frames\n var pages []Page\n for {\n var frameHeader WALFrameHeader\n err := binary.Read(f, binary.BigEndian, &frameHeader)\n if err == io.EOF {\n break\n }\n\n // Read page data\n pageData := make([]byte, header.PageSize)\n f.Read(pageData)\n\n pages = append(pages, Page{\n Number: frameHeader.PageNumber,\n Data: pageData,\n })\n }\n\n return pages, nil\n}\n```\n\n## Page Structure\n\n### Database Pages\n\nSQLite divides the database into fixed-size pages:\n\n```\nPage Size: Typically 4096 bytes (4KB)\nPage Number: 1-based indexing\nPage Types:\n - B-tree interior pages\n - B-tree leaf pages\n - Overflow pages\n - Freelist pages\n - Lock byte page (at 1GB)\n```\n\n### Page Layout\n\n```\n+------------------+\n| Page Header | Variable (8-12 bytes)\n+------------------+\n| Cell Pointers | 2 bytes each\n+------------------+\n| Unallocated |\n| Space |\n+------------------+\n| Cell Content | Variable size\n| Area | (grows upward)\n+------------------+\n```\n\n### Page Header Structure\n\n```go\ntype PageHeader struct {\n PageType byte // 0x02, 0x05, 0x0a, 0x0d\n FreeBlockStart uint16 // Start of free block list\n CellCount uint16 // Number of cells\n CellStart uint16 // Offset to first cell\n FragmentBytes byte // Fragmented free bytes\n // Additional fields for interior pages\n RightChild uint32 // Only for interior pages\n}\n```\n\n## The 1GB Lock Page\n\n### Critical Concept\n\nSQLite reserves a special page at exactly 1,073,741,824 bytes (0x40000000) for locking:\n\n```go\nconst PENDING_BYTE = 0x40000000 // 1GB mark\n\n// Page number varies by page size\nfunc LockPageNumber(pageSize int) uint32 {\n return uint32(PENDING_BYTE/pageSize) + 1\n}\n\n// Examples:\n// 4KB pages: 262145 (0x40001)\n// 8KB pages: 131073 (0x20001)\n// 16KB pages: 65537 (0x10001)\n// 32KB pages: 32769 (0x08001)\n// 64KB pages: 16385 (0x04001)\n```\n\n### Why This Matters\n\n1. **Cannot contain data** - SQLite will never write user data here\n2. **Must be skipped** - During replication/compaction\n3. **Affects large databases** - Only databases >1GB\n4. **Page number changes** - Different for each page size\n\n### Implementation in Litestream\n\n```go\n// From superfly/ltx package\nfunc LockPgno(pageSize int) uint32 {\n return uint32(PENDING_BYTE/pageSize) + 1\n}\n\n// db.go - Skipping lock page during iteration\nfor pgno := uint32(1); pgno <= maxPgno; pgno++ {\n if pgno == ltx.LockPgno(db.pageSize) {\n continue // Skip lock page\n }\n\n // Process normal page\n processPage(pgno)\n}\n```\n\n### Testing Lock Page\n\n```sql\n-- Create database that spans lock page\nCREATE TABLE test (id INTEGER PRIMARY KEY, data BLOB);\n\n-- Insert data until database > 1GB\nWITH RECURSIVE generate_series(value) AS (\n SELECT 1\n UNION ALL\n SELECT value+1 FROM generate_series\n LIMIT 300000\n)\nINSERT INTO test SELECT value, randomblob(4000) FROM generate_series;\n\n-- Check database size\nPRAGMA page_count; -- Should be > 262145 for 4KB pages\nPRAGMA page_size; -- Typically 4096\n\n-- Calculate if lock page is in range\n-- For 4KB pages: if page_count > 262145, lock page is included\n```\n\n## Transaction Management\n\n### SQLite Transaction Types\n\n1. **Deferred Transaction** (default)\n ```sql\n BEGIN DEFERRED; -- Lock acquired on first use\n ```\n\n2. **Immediate Transaction**\n ```sql\n BEGIN IMMEDIATE; -- RESERVED lock immediately\n ```\n\n3. **Exclusive Transaction**\n ```sql\n BEGIN EXCLUSIVE; -- EXCLUSIVE lock immediately\n ```\n\n### Lock Types in SQLite\n\n```mermaid\ngraph TD\n UNLOCKED -->|BEGIN| SHARED\n SHARED -->|Write| RESERVED\n RESERVED -->|Prepare| PENDING\n PENDING -->|Commit| EXCLUSIVE\n EXCLUSIVE -->|Done| UNLOCKED\n```\n\n1. **SHARED** - Multiple readers allowed\n2. **RESERVED** - Signals intent to write\n3. **PENDING** - Blocking new SHARED locks\n4. **EXCLUSIVE** - Single writer, no readers\n\n### Litestream's Long-Running Read Transaction\n\n```go\n// db.go - Maintaining read transaction for consistency\nfunc (db *DB) initReadTx() error {\n // Start read-only transaction\n tx, err := db.db.BeginTx(context.Background(), &sql.TxOptions{\n ReadOnly: true,\n })\n if err != nil {\n return err\n }\n\n // Execute query to acquire SHARED lock\n var dummy string\n err = tx.QueryRow(\"SELECT ''\").Scan(&dummy)\n if err != nil {\n tx.Rollback()\n return err\n }\n\n // Keep transaction open\n db.rtx = tx\n return nil\n}\n```\n\n**Purpose:**\n- Prevents database from being checkpointed past our read point\n- Ensures consistent view of database\n- Allows reading pages from WAL\n\n## Checkpoint Modes\n\n### PASSIVE Checkpoint (default)\n```sql\nPRAGMA wal_checkpoint(PASSIVE);\n```\n- Attempts checkpoint\n- Fails if readers present\n- Non-blocking\n\n### FULL Checkpoint\n```sql\nPRAGMA wal_checkpoint(FULL);\n```\n- Waits for readers to finish\n- Blocks new readers\n- Ensures checkpoint completes\n\n### RESTART Checkpoint\n```sql\nPRAGMA wal_checkpoint(RESTART);\n```\n- Like FULL, but also:\n- Ensures next writer starts at beginning of WAL\n- Resets WAL file\n\n### TRUNCATE Checkpoint\n```sql\nPRAGMA wal_checkpoint(TRUNCATE);\n```\n- Like RESTART, but also:\n- Truncates WAL file to zero length\n- Releases disk space\n\n### Litestream Checkpoint Strategy\n\n```go\n// db.go - Checkpoint decision logic\nfunc (db *DB) autoCheckpoint() error {\n walSize := db.WALSize()\n pageCount := walSize / db.pageSize\n\n if pageCount > db.TruncatePageN {\n // Force truncation for very large WAL (emergency brake)\n return db.Checkpoint(\"TRUNCATE\")\n } else if pageCount > db.MinCheckpointPageN {\n // Try passive checkpoint (non-blocking)\n return db.Checkpoint(\"PASSIVE\")\n } else if db.CheckpointInterval elapsed {\n // Time-based passive checkpoint\n return db.Checkpoint(\"PASSIVE\")\n }\n // Note: RESTART mode permanently removed due to issue #724 (write-blocking)\n }\n\n return nil\n}\n```\n\n## Important SQLite Pragmas\n\n### Essential Pragmas for Litestream\n\n```sql\n-- Enable WAL mode (required)\nPRAGMA journal_mode = WAL;\n\n-- Get database info\nPRAGMA page_size; -- Page size in bytes\nPRAGMA page_count; -- Total pages in database\nPRAGMA freelist_count; -- Free pages\n\n-- WAL information\nPRAGMA wal_checkpoint; -- Perform checkpoint\nPRAGMA wal_autocheckpoint; -- Auto-checkpoint threshold\nPRAGMA wal_checkpoint(PASSIVE); -- Non-blocking checkpoint\n\n-- Database state\nPRAGMA integrity_check; -- Verify database integrity\nPRAGMA quick_check; -- Fast integrity check\n\n-- Lock information\nPRAGMA lock_status; -- Current locks (debug builds)\n\n-- Performance tuning\nPRAGMA synchronous = NORMAL; -- Sync mode\nPRAGMA busy_timeout = 5000; -- Wait 5s for locks\nPRAGMA cache_size = -64000; -- 64MB cache\n```\n\n### Reading Pragmas in Go\n\n```go\nfunc getDatabaseInfo(db *sql.DB) (*DBInfo, error) {\n info := &DBInfo{}\n\n // Page size\n err := db.QueryRow(\"PRAGMA page_size\").Scan(&info.PageSize)\n\n // Page count\n err = db.QueryRow(\"PRAGMA page_count\").Scan(&info.PageCount)\n\n // Journal mode\n err = db.QueryRow(\"PRAGMA journal_mode\").Scan(&info.JournalMode)\n\n // Calculate size\n info.Size = info.PageSize * info.PageCount\n\n return info, nil\n}\n```\n\n## SQLite API Usage\n\n### Direct SQLite Access\n\nLitestream uses both database/sql and direct SQLite APIs:\n\n```go\n// Using database/sql for queries\ndb, err := sql.Open(\"sqlite3\", \"database.db\")\n\n// Using modernc.org/sqlite for low-level access\nconn, err := sqlite.Open(\"database.db\")\n\n// Direct page access (requires special builds)\npage := readPage(conn, pageNumber)\n```\n\n### Connection Modes\n\n```go\n// Read-only connection\ndb, err := sql.Open(\"sqlite3\", \"file:database.db?mode=ro\")\n\n// WAL mode connection\ndb, err := sql.Open(\"sqlite3\", \"database.db?_journal=WAL\")\n\n// With busy timeout\ndb, err := sql.Open(\"sqlite3\", \"database.db?_busy_timeout=5000\")\n\n// Multiple options\ndb, err := sql.Open(\"sqlite3\", \"database.db?_journal=WAL&_busy_timeout=5000&_synchronous=NORMAL\")\n```\n\n### WAL File Access Pattern\n\n```go\n// Litestream's approach to reading WAL\nfunc (db *DB) monitorWAL() {\n walPath := db.path + \"-wal\"\n\n for {\n // Check WAL file size\n stat, err := os.Stat(walPath)\n if err != nil {\n continue // WAL might not exist yet\n }\n\n // Compare with last known size\n if stat.Size() > db.lastWALSize {\n // New data in WAL\n db.processWALChanges()\n db.lastWALSize = stat.Size()\n }\n\n time.Sleep(db.MonitorInterval)\n }\n}\n```\n\n## Critical SQLite Behaviors\n\n### 1. Automatic Checkpoint\nSQLite automatically checkpoints when WAL reaches 1000 pages (default):\n```go\n// Can interfere with Litestream's control\n// Solution: Set high threshold\ndb.Exec(\"PRAGMA wal_autocheckpoint = 10000\")\n```\n\n### 2. Busy Timeout\nDefault timeout is 0 (immediate failure):\n```go\n// Set reasonable timeout\ndb.Exec(\"PRAGMA busy_timeout = 5000\") // 5 seconds\n```\n\n### 3. Synchronous Mode\nControls when SQLite waits for disk writes:\n```go\n// NORMAL is safe with WAL\ndb.Exec(\"PRAGMA synchronous = NORMAL\")\n```\n\n### 4. Page Cache\nSQLite maintains an in-memory page cache:\n```go\n// Set cache size (negative = KB, positive = pages)\ndb.Exec(\"PRAGMA cache_size = -64000\") // 64MB\n```\n\n## WAL to LTX Conversion\n\nLitestream converts WAL frames to LTX format:\n\n```go\nfunc walToLTX(walFrames []WALFrame) *LTXFile {\n ltx := <XFile{\n Header: LTXHeader{\n PageSize: walFrames[0].PageSize,\n MinTXID: walFrames[0].TransactionID,\n },\n }\n\n for _, frame := range walFrames {\n // Skip lock page\n if frame.PageNumber == LockPageNumber(ltx.Header.PageSize) {\n continue\n }\n\n ltx.Pages = append(ltx.Pages, Page{\n Number: frame.PageNumber,\n Data: frame.Data,\n })\n\n ltx.Header.MaxTXID = frame.TransactionID\n }\n\n return ltx\n}\n```\n\n## Key Takeaways\n\n1. **WAL is temporary** - Gets merged back via checkpoint\n2. **Lock page is sacred** - Never write data at 1GB mark\n3. **Page size matters** - Affects lock page number and performance\n4. **Transactions provide consistency** - Long-running read prevents changes\n5. **Checkpoints are critical** - Balance between WAL size and performance\n6. **SQLite locks coordinate access** - Understanding prevents deadlocks\n7. **Pragmas control behavior** - Must be set correctly for Litestream\n\nThis understanding is essential for:\n- Debugging replication issues\n- Implementing new features\n- Optimizing performance\n- Handling edge cases correctly\n" }, { "path": "docs/TESTING_GUIDE.md", "content": "# Litestream Testing Guide\n\nComprehensive guide for testing Litestream components and handling edge cases.\n\n## Table of Contents\n\n- [Testing Philosophy](#testing-philosophy)\n- [1GB Database Testing](#1gb-database-testing)\n- [Race Condition Testing](#race-condition-testing)\n- [Integration Testing](#integration-testing)\n- [Performance Testing](#performance-testing)\n- [Mock Usage Patterns](#mock-usage-patterns)\n- [Test Utilities](#test-utilities)\n- [Common Test Failures](#common-test-failures)\n\n## Testing Philosophy\n\nLitestream testing follows these principles:\n\n1. **Test at Multiple Levels**: Unit, integration, and end-to-end\n2. **Focus on Edge Cases**: Especially >1GB databases and eventual consistency\n3. **Use Real SQLite**: Avoid mocking SQLite behavior\n4. **Race Detection**: Always run with `-race` flag\n5. **Deterministic Tests**: Use fixed seeds and timestamps where possible\n\n## 1GB Database Testing\n\n### The Lock Page Problem\n\nSQLite reserves a special lock page at exactly 1GB (0x40000000 bytes). This page cannot contain data and must be skipped during replication.\n\n### Test Requirements\n\n#### Creating Test Databases\n\n```bash\n# Use litestream-test tool for large databases\n./bin/litestream-test populate \\\n -db test.db \\\n -target-size 1.5GB \\\n -page-size 4096\n```\n\n#### Manual Test Database Creation\n\n```go\nfunc createLargeTestDB(t *testing.T, path string, targetSize int64) {\n db, err := sql.Open(\"sqlite\", path+\"?_journal=WAL\")\n require.NoError(t, err)\n defer db.Close()\n\n // Set page size\n _, err = db.Exec(\"PRAGMA page_size = 4096\")\n require.NoError(t, err)\n\n // Create test table\n _, err = db.Exec(`\n CREATE TABLE test_data (\n id INTEGER PRIMARY KEY,\n data BLOB NOT NULL\n )\n `)\n require.NoError(t, err)\n\n // Calculate rows needed\n rowSize := 4000 // bytes per row\n rowsNeeded := targetSize / int64(rowSize)\n\n // Batch insert for performance\n tx, err := db.Begin()\n require.NoError(t, err)\n\n stmt, err := tx.Prepare(\"INSERT INTO test_data (data) VALUES (?)\")\n require.NoError(t, err)\n\n for i := int64(0); i < rowsNeeded; i++ {\n data := make([]byte, rowSize)\n rand.Read(data)\n _, err = stmt.Exec(data)\n require.NoError(t, err)\n\n // Commit periodically\n if i%1000 == 0 {\n err = tx.Commit()\n require.NoError(t, err)\n tx, err = db.Begin()\n require.NoError(t, err)\n stmt, err = tx.Prepare(\"INSERT INTO test_data (data) VALUES (?)\")\n require.NoError(t, err)\n }\n }\n\n err = tx.Commit()\n require.NoError(t, err)\n\n // Verify size\n var pageCount, pageSize int\n db.QueryRow(\"PRAGMA page_count\").Scan(&pageCount)\n db.QueryRow(\"PRAGMA page_size\").Scan(&pageSize)\n\n actualSize := int64(pageCount * pageSize)\n t.Logf(\"Created database: %d bytes (%d pages of %d bytes)\",\n actualSize, pageCount, pageSize)\n\n // Verify lock page is in range\n lockPgno := ltx.LockPgno(pageSize)\n if pageCount > lockPgno {\n t.Logf(\"Database spans lock page at page %d\", lockPgno)\n }\n}\n```\n\n#### Lock Page Test Cases\n\n```go\nfunc TestDB_LockPageHandling(t *testing.T) {\n testCases := []struct {\n name string\n pageSize int\n lockPgno uint32\n }{\n {\"4KB pages\", 4096, 262145},\n {\"8KB pages\", 8192, 131073},\n {\"16KB pages\", 16384, 65537},\n {\"32KB pages\", 32768, 32769},\n }\n\n for _, tc := range testCases {\n t.Run(tc.name, func(t *testing.T) {\n // Create database larger than 1GB\n dbPath := filepath.Join(t.TempDir(), \"test.db\")\n createLargeTestDB(t, dbPath, 1100*1024*1024) // 1.1GB\n\n // Open with Litestream\n db := litestream.NewDB(dbPath)\n err := db.Open()\n require.NoError(t, err)\n defer db.Close(context.Background())\n\n // Start replication\n replica := litestream.NewReplicaWithClient(db, newMockClient())\n err = replica.Start(context.Background())\n require.NoError(t, err)\n\n // Perform writes that span the lock page\n conn, err := sql.Open(\"sqlite\", dbPath)\n require.NoError(t, err)\n\n tx, err := conn.Begin()\n require.NoError(t, err)\n\n // Write data around lock page boundary\n for i := tc.lockPgno - 10; i < tc.lockPgno+10; i++ {\n if i == tc.lockPgno {\n continue // Skip lock page\n }\n\n _, err = tx.Exec(fmt.Sprintf(\n \"INSERT INTO test_data (id, data) VALUES (%d, randomblob(4000))\",\n i))\n require.NoError(t, err)\n }\n\n err = tx.Commit()\n require.NoError(t, err)\n\n // Wait for sync\n err = db.Sync(context.Background())\n require.NoError(t, err)\n\n // Verify replication skipped lock page\n verifyLockPageSkipped(t, replica, tc.lockPgno)\n })\n }\n}\n\nfunc verifyLockPageSkipped(t *testing.T, replica *litestream.Replica, lockPgno uint32) {\n // Get LTX files\n files, err := replica.Client.LTXFiles(context.Background(), 0, 0, false)\n require.NoError(t, err)\n\n // Check each file\n for files.Next() {\n info := files.Item()\n\n // Read page index\n pageIndex, err := litestream.FetchPageIndex(context.Background(),\n replica.Client, info)\n require.NoError(t, err)\n\n // Verify lock page not present\n _, hasLockPage := pageIndex[lockPgno]\n assert.False(t, hasLockPage,\n \"Lock page %d should not be in LTX file\", lockPgno)\n }\n}\n```\n\n### Restoration Testing\n\n```go\nfunc TestDB_RestoreLargeDatabase(t *testing.T) {\n // Create and replicate large database\n srcPath := filepath.Join(t.TempDir(), \"source.db\")\n createLargeTestDB(t, srcPath, 1500*1024*1024) // 1.5GB\n\n // Setup replication\n db := litestream.NewDB(srcPath)\n err := db.Open()\n require.NoError(t, err)\n\n client := file.NewReplicaClient(filepath.Join(t.TempDir(), \"replica\"))\n replica := litestream.NewReplicaWithClient(db, client)\n\n err = replica.Start(context.Background())\n require.NoError(t, err)\n\n // Let it replicate\n err = db.Sync(context.Background())\n require.NoError(t, err)\n\n db.Close(context.Background())\n\n // Restore to new location\n dstPath := filepath.Join(t.TempDir(), \"restored.db\")\n err = litestream.Restore(context.Background(), client, dstPath, nil)\n require.NoError(t, err)\n\n // Verify restoration\n verifyDatabasesMatch(t, srcPath, dstPath)\n}\n\nfunc verifyDatabasesMatch(t *testing.T, path1, path2 string) {\n // Compare checksums\n checksum1 := calculateDBChecksum(t, path1)\n checksum2 := calculateDBChecksum(t, path2)\n assert.Equal(t, checksum1, checksum2, \"Database checksums should match\")\n\n // Compare page counts\n pageCount1 := getPageCount(t, path1)\n pageCount2 := getPageCount(t, path2)\n assert.Equal(t, pageCount1, pageCount2, \"Page counts should match\")\n\n // Run integrity check\n db, err := sql.Open(\"sqlite\", path2)\n require.NoError(t, err)\n defer db.Close()\n\n var result string\n err = db.QueryRow(\"PRAGMA integrity_check\").Scan(&result)\n require.NoError(t, err)\n assert.Equal(t, \"ok\", result, \"Integrity check should pass\")\n}\n```\n\n## Race Condition Testing\n\n### Running with Race Detector\n\n```bash\n# Always run tests with race detector\ngo test -race -v ./...\n\n# Run specific race-prone tests\ngo test -race -v -run TestReplica_Sync ./...\ngo test -race -v -run TestDB_Sync ./...\ngo test -race -v -run TestStore_CompactDB ./...\n```\n\n### Common Race Conditions\n\n#### 1. Position Updates\n\n```go\nfunc TestReplica_ConcurrentPositionUpdate(t *testing.T) {\n replica := litestream.NewReplica(nil)\n var wg sync.WaitGroup\n errors := make(chan error, 100)\n\n // Concurrent writers\n for i := 0; i < 10; i++ {\n wg.Add(1)\n go func(n int) {\n defer wg.Done()\n\n pos := ltx.NewPos(ltx.TXID(n), ltx.Checksum(uint64(n)))\n\n // This should use proper locking\n replica.SetPos(pos)\n\n // Verify position\n readPos := replica.Pos()\n if readPos.TXID < ltx.TXID(n) {\n errors <- fmt.Errorf(\"position went backwards\")\n }\n }(i)\n }\n\n // Concurrent readers\n for i := 0; i < 10; i++ {\n wg.Add(1)\n go func() {\n defer wg.Done()\n\n for j := 0; j < 100; j++ {\n _ = replica.Pos()\n time.Sleep(time.Microsecond)\n }\n }()\n }\n\n wg.Wait()\n close(errors)\n\n for err := range errors {\n t.Error(err)\n }\n}\n```\n\n#### 2. WAL Monitoring\n\n```go\nfunc TestDB_ConcurrentWALAccess(t *testing.T) {\n db := setupTestDB(t)\n defer db.Close(context.Background())\n\n ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n defer cancel()\n\n var wg sync.WaitGroup\n\n // Writer goroutine\n wg.Add(1)\n go func() {\n defer wg.Done()\n\n conn, err := sql.Open(\"sqlite\", db.Path())\n if err != nil {\n return\n }\n defer conn.Close()\n\n for i := 0; i < 100; i++ {\n _, _ = conn.Exec(\"INSERT INTO test VALUES (?)\", i)\n time.Sleep(10 * time.Millisecond)\n }\n }()\n\n // Monitor goroutine\n wg.Add(1)\n go func() {\n defer wg.Done()\n\n notifyCh := db.Notify()\n\n for {\n select {\n case <-ctx.Done():\n return\n case <-notifyCh:\n // Process WAL changes\n _ = db.Sync(context.Background())\n }\n }\n }()\n\n // Checkpoint goroutine\n wg.Add(1)\n go func() {\n defer wg.Done()\n\n ticker := time.NewTicker(100 * time.Millisecond)\n defer ticker.Stop()\n\n for {\n select {\n case <-ctx.Done():\n return\n case <-ticker.C:\n _ = db.Checkpoint(context.Background(), litestream.CheckpointModePassive)\n }\n }\n }()\n\n wg.Wait()\n}\n```\n\n### Test Cleanup\n\n```go\nfunc TestStore_Integration(t *testing.T) {\n // Setup\n tmpDir := t.TempDir()\n db := setupTestDB(t, tmpDir)\n\n // Use defer with error channel for cleanup\n insertErr := make(chan error, 1)\n\n // Cleanup function\n cleanup := func() {\n select {\n case err := <-insertErr:\n if err != nil {\n t.Errorf(\"insert error during test: %v\", err)\n }\n default:\n }\n }\n defer cleanup()\n\n // Test with timeout\n ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)\n defer cancel()\n\n // Run test...\n}\n```\n\n## Integration Testing\n\n### Test Structure\n\n```go\n// +build integration\n\npackage litestream_test\n\nimport (\n \"context\"\n \"os\"\n \"testing\"\n)\n\nfunc TestIntegration_S3(t *testing.T) {\n if testing.Short() {\n t.Skip(\"skipping integration test\")\n }\n\n // Check for credentials\n if os.Getenv(\"AWS_ACCESS_KEY_ID\") == \"\" {\n t.Skip(\"AWS_ACCESS_KEY_ID not set\")\n }\n\n // Run test against real S3\n runIntegrationTest(t, setupS3Client())\n}\n\nfunc runIntegrationTest(t *testing.T, client ReplicaClient) {\n ctx := context.Background()\n\n t.Run(\"BasicReplication\", func(t *testing.T) {\n // Test basic write/read cycle\n })\n\n t.Run(\"Compaction\", func(t *testing.T) {\n // Test compaction with remote storage\n })\n\n t.Run(\"EventualConsistency\", func(t *testing.T) {\n // Test handling of eventual consistency\n })\n\n t.Run(\"LargeFiles\", func(t *testing.T) {\n // Test with files > 100MB\n })\n\n t.Run(\"Cleanup\", func(t *testing.T) {\n err := client.DeleteAll(ctx)\n require.NoError(t, err)\n })\n}\n```\n\n### Environment-Based Configuration\n\n```go\nfunc setupS3Client() *s3.ReplicaClient {\n return &s3.ReplicaClient{\n AccessKeyID: os.Getenv(\"AWS_ACCESS_KEY_ID\"),\n SecretAccessKey: os.Getenv(\"AWS_SECRET_ACCESS_KEY\"),\n Region: getEnvOrDefault(\"AWS_REGION\", \"us-east-1\"),\n Bucket: getEnvOrDefault(\"TEST_S3_BUCKET\", \"litestream-test\"),\n Path: fmt.Sprintf(\"test-%d\", time.Now().Unix()),\n }\n}\n\nfunc getEnvOrDefault(key, defaultValue string) string {\n if value := os.Getenv(key); value != \"\" {\n return value\n }\n return defaultValue\n}\n```\n\n## Performance Testing\n\n### Benchmarks\n\n```go\nfunc BenchmarkDB_Sync(b *testing.B) {\n db := setupBenchDB(b)\n defer db.Close(context.Background())\n\n // Prepare test data\n conn, _ := sql.Open(\"sqlite\", db.Path())\n defer conn.Close()\n\n for i := 0; i < 1000; i++ {\n conn.Exec(\"INSERT INTO test VALUES (?)\", i)\n }\n\n b.ResetTimer()\n\n for i := 0; i < b.N; i++ {\n err := db.Sync(context.Background())\n if err != nil {\n b.Fatal(err)\n }\n }\n\n b.ReportMetric(float64(b.N)/b.Elapsed().Seconds(), \"syncs/sec\")\n}\n\nfunc BenchmarkCompaction(b *testing.B) {\n benchmarks := []struct {\n name string\n fileCount int\n fileSize int\n }{\n {\"Small-Many\", 1000, 1024}, // Many small files\n {\"Medium\", 100, 10 * 1024}, // Medium files\n {\"Large-Few\", 10, 100 * 1024}, // Few large files\n }\n\n for _, bm := range benchmarks {\n b.Run(bm.name, func(b *testing.B) {\n for i := 0; i < b.N; i++ {\n b.StopTimer()\n files := generateTestFiles(bm.fileCount, bm.fileSize)\n b.StartTimer()\n\n _, err := compact(files)\n if err != nil {\n b.Fatal(err)\n }\n }\n\n totalSize := int64(bm.fileCount * bm.fileSize)\n b.ReportMetric(float64(totalSize)/float64(b.Elapsed().Nanoseconds()), \"bytes/ns\")\n })\n }\n}\n```\n\n### Load Testing\n\n```go\nfunc TestDB_LoadTest(t *testing.T) {\n if testing.Short() {\n t.Skip(\"skipping load test\")\n }\n\n db := setupTestDB(t)\n defer db.Close(context.Background())\n\n // Configure load\n config := LoadConfig{\n Duration: 5 * time.Minute,\n WriteRate: 100, // writes/sec\n ReadRate: 500, // reads/sec\n Workers: 10,\n DataSize: 4096,\n BurstPattern: true,\n }\n\n results := runLoadTest(t, db, config)\n\n // Verify results\n assert.Greater(t, results.TotalWrites, int64(20000))\n assert.Less(t, results.P99Latency, 100*time.Millisecond)\n assert.Zero(t, results.Errors)\n\n t.Logf(\"Load test results: %+v\", results)\n}\n\ntype LoadResults struct {\n TotalWrites int64\n TotalReads int64\n Errors int64\n P50Latency time.Duration\n P99Latency time.Duration\n BytesReplicated int64\n}\n\nfunc runLoadTest(t *testing.T, db *DB, config LoadConfig) LoadResults {\n ctx, cancel := context.WithTimeout(context.Background(), config.Duration)\n defer cancel()\n\n var results LoadResults\n var mu sync.Mutex\n latencies := make([]time.Duration, 0, 100000)\n\n // Start workers\n var wg sync.WaitGroup\n for i := 0; i < config.Workers; i++ {\n wg.Add(1)\n go func(workerID int) {\n defer wg.Done()\n\n conn, err := sql.Open(\"sqlite\", db.Path())\n if err != nil {\n return\n }\n defer conn.Close()\n\n ticker := time.NewTicker(time.Second / time.Duration(config.WriteRate))\n defer ticker.Stop()\n\n for {\n select {\n case <-ctx.Done():\n return\n case <-ticker.C:\n start := time.Now()\n data := make([]byte, config.DataSize)\n rand.Read(data)\n\n _, err := conn.Exec(\"INSERT INTO test (data) VALUES (?)\", data)\n latency := time.Since(start)\n\n mu.Lock()\n if err != nil {\n results.Errors++\n } else {\n results.TotalWrites++\n latencies = append(latencies, latency)\n }\n mu.Unlock()\n }\n }\n }(i)\n }\n\n wg.Wait()\n\n // Calculate percentiles\n sort.Slice(latencies, func(i, j int) bool {\n return latencies[i] < latencies[j]\n })\n\n if len(latencies) > 0 {\n results.P50Latency = latencies[len(latencies)*50/100]\n results.P99Latency = latencies[len(latencies)*99/100]\n }\n\n return results\n}\n```\n\n## Mock Usage Patterns\n\n### Mock ReplicaClient\n\n```go\ntype MockReplicaClient struct {\n mu sync.Mutex\n files map[string]*ltx.FileInfo\n data map[string][]byte\n\n // Control behavior\n FailureRate float64\n Latency time.Duration\n EventualDelay time.Duration\n}\n\nfunc (m *MockReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n // Simulate latency\n if m.Latency > 0 {\n time.Sleep(m.Latency)\n }\n\n // Simulate failures\n if m.FailureRate > 0 && rand.Float64() < m.FailureRate {\n return nil, errors.New(\"simulated failure\")\n }\n\n // Simulate eventual consistency\n if m.EventualDelay > 0 {\n time.AfterFunc(m.EventualDelay, func() {\n m.mu.Lock()\n defer m.mu.Unlock()\n // Make file available after delay\n })\n }\n\n // Store file\n data, err := io.ReadAll(r)\n if err != nil {\n return nil, err\n }\n\n m.mu.Lock()\n defer m.mu.Unlock()\n\n key := fmt.Sprintf(\"%d-%016x-%016x\", level, minTXID, maxTXID)\n info := <x.FileInfo{\n Level: level,\n MinTXID: minTXID,\n MaxTXID: maxTXID,\n Size: int64(len(data)),\n CreatedAt: time.Now(),\n }\n\n m.files[key] = info\n m.data[key] = data\n\n return info, nil\n}\n```\n\n### Mock Database\n\n```go\ntype MockDB struct {\n mu sync.Mutex\n path string\n replicas []*Replica\n closed bool\n\n // Control behavior\n CheckpointFailures int\n SyncDelay time.Duration\n}\n\nfunc (m *MockDB) Sync(ctx context.Context) error {\n if m.SyncDelay > 0 {\n select {\n case <-time.After(m.SyncDelay):\n case <-ctx.Done():\n return ctx.Err()\n }\n }\n\n m.mu.Lock()\n defer m.mu.Unlock()\n\n if m.closed {\n return errors.New(\"database closed\")\n }\n\n for _, r := range m.replicas {\n if err := r.Sync(ctx); err != nil {\n return err\n }\n }\n\n return nil\n}\n```\n\n## Test Utilities\n\n### Helper Functions\n\n```go\n// testutil/db.go\npackage testutil\n\nimport (\n \"database/sql\"\n \"testing\"\n \"path/filepath\"\n)\n\nfunc NewTestDB(t testing.TB) *litestream.DB {\n t.Helper()\n\n path := filepath.Join(t.TempDir(), \"test.db\")\n\n // Create SQLite database\n conn, err := sql.Open(\"sqlite\", path+\"?_journal=WAL\")\n require.NoError(t, err)\n\n _, err = conn.Exec(`\n CREATE TABLE test (\n id INTEGER PRIMARY KEY,\n data BLOB\n )\n `)\n require.NoError(t, err)\n conn.Close()\n\n // Open with Litestream\n db := litestream.NewDB(path)\n db.MonitorInterval = 10 * time.Millisecond // Speed up for tests\n db.MinCheckpointPageN = 100 // Lower threshold for tests\n\n err = db.Open()\n require.NoError(t, err)\n\n t.Cleanup(func() {\n db.Close(context.Background())\n })\n\n return db\n}\n\nfunc WriteTestData(t testing.TB, db *litestream.DB, count int) {\n t.Helper()\n\n conn, err := sql.Open(\"sqlite\", db.Path())\n require.NoError(t, err)\n defer conn.Close()\n\n tx, err := conn.Begin()\n require.NoError(t, err)\n\n for i := 0; i < count; i++ {\n data := make([]byte, 100)\n rand.Read(data)\n _, err = tx.Exec(\"INSERT INTO test (data) VALUES (?)\", data)\n require.NoError(t, err)\n }\n\n err = tx.Commit()\n require.NoError(t, err)\n}\n```\n\n### Test Fixtures\n\n```go\n// testdata/fixtures.go\npackage testdata\n\nimport _ \"embed\"\n\n//go:embed small.db\nvar SmallDB []byte\n\n//go:embed large.db\nvar LargeDB []byte\n\n//go:embed corrupted.db\nvar CorruptedDB []byte\n\nfunc ExtractFixture(name string, path string) error {\n var data []byte\n\n switch name {\n case \"small\":\n data = SmallDB\n case \"large\":\n data = LargeDB\n case \"corrupted\":\n data = CorruptedDB\n default:\n return fmt.Errorf(\"unknown fixture: %s\", name)\n }\n\n return os.WriteFile(path, data, 0600)\n}\n```\n\n## Common Test Failures\n\n### 1. Database Locked Errors\n\n```go\n// Problem: Multiple connections without proper WAL mode\nfunc TestBroken(t *testing.T) {\n db1, _ := sql.Open(\"sqlite\", \"test.db\") // Wrong! WAL disabled\n db2, _ := sql.Open(\"sqlite\", \"test.db\") // Will fail\n}\n\n// Solution: Use WAL mode\nfunc TestFixed(t *testing.T) {\n db1, _ := sql.Open(\"sqlite\", \"test.db?_journal=WAL\")\n db2, _ := sql.Open(\"sqlite\", \"test.db?_journal=WAL\")\n}\n```\n\n### 2. Timing Issues\n\n```go\n// Problem: Race between write and sync\nfunc TestBroken(t *testing.T) {\n WriteData(db)\n result := ReadReplica() // May not see data yet!\n}\n\n// Solution: Explicit sync\nfunc TestFixed(t *testing.T) {\n WriteData(db)\n err := db.Sync(context.Background())\n require.NoError(t, err)\n result := ReadReplica() // Now guaranteed to see data\n}\n```\n\n### 3. Cleanup Issues\n\n```go\n// Problem: Goroutine outlives test\nfunc TestBroken(t *testing.T) {\n go func() {\n time.Sleep(10 * time.Second)\n doWork() // Test already finished!\n }()\n}\n\n// Solution: Use context and wait\nfunc TestFixed(t *testing.T) {\n ctx, cancel := context.WithCancel(context.Background())\n defer cancel()\n\n var wg sync.WaitGroup\n wg.Add(1)\n go func() {\n defer wg.Done()\n select {\n case <-ctx.Done():\n return\n case <-time.After(10 * time.Second):\n doWork()\n }\n }()\n\n // Test work...\n\n cancel() // Signal shutdown\n wg.Wait() // Wait for goroutine\n}\n```\n\n### 4. File Handle Leaks\n\n```go\n// Problem: Not closing files\nfunc TestBroken(t *testing.T) {\n f, _ := os.Open(\"test.db\")\n // Missing f.Close()!\n}\n\n// Solution: Always use defer\nfunc TestFixed(t *testing.T) {\n f, err := os.Open(\"test.db\")\n require.NoError(t, err)\n defer f.Close()\n}\n```\n\n## Test Coverage\n\n### Running Coverage\n\n```bash\n# Generate coverage report\ngo test -coverprofile=coverage.out ./...\n\n# View coverage in browser\ngo tool cover -html=coverage.out\n\n# Check coverage percentage\ngo tool cover -func=coverage.out | grep total\n\n# Coverage by package\ngo test -cover ./...\n```\n\n### Coverage Requirements\n\n- Core packages (`db.go`, `replica.go`, `store.go`): >80%\n- Replica clients: >70%\n- Utilities: >60%\n- Mock implementations: Not required\n\n### Improving Coverage\n\n```go\n// Use test tables for comprehensive coverage\nfunc TestDB_Checkpoint(t *testing.T) {\n tests := []struct {\n name string\n mode string\n walSize int\n wantErr bool\n }{\n {\"Passive\", \"PASSIVE\", 100, false},\n {\"Full\", \"FULL\", 1000, false},\n {\"Restart\", \"RESTART\", 5000, false},\n {\"Truncate\", \"TRUNCATE\", 10000, false},\n {\"Invalid\", \"INVALID\", 100, true},\n }\n\n for _, tt := range tests {\n t.Run(tt.name, func(t *testing.T) {\n db := setupTestDB(t)\n generateWAL(t, db, tt.walSize)\n\n err := db.Checkpoint(tt.mode)\n if tt.wantErr {\n assert.Error(t, err)\n } else {\n assert.NoError(t, err)\n }\n })\n }\n}\n```\n" }, { "path": "docs/VFS.md", "content": "# Litestream VFS\n\nThe Litestream VFS (Virtual File System) is a SQLite extension that allows applications to read directly\nfrom Litestream replica storage (S3, GCS, Azure Blob, etc.) without restoring to local disk. It also\nsupports write mode for remote-first SQLite databases.\n\n## Table of Contents\n\n- [Overview](#overview)\n- [Building](#building)\n- [Configuration](#configuration)\n- [Usage](#usage)\n- [SQL Functions](#sql-functions)\n- [Time Travel](#time-travel)\n- [Write Mode](#write-mode)\n- [Supported Storage Backends](#supported-storage-backends)\n\n## Overview\n\nThe VFS extension provides:\n\n- **Direct replica reading**: Read SQLite databases directly from cloud storage\n- **Automatic polling**: Background polling for new LTX files from the primary\n- **Page caching**: LRU cache for frequently accessed pages (default 10MB)\n- **Time travel**: Query historical database states at specific timestamps\n- **Write support**: Write changes that sync back to remote storage (experimental)\n\n### How It Works\n\n1. The VFS loads as a SQLite extension\n2. When opening a database, it reads LTX files from the configured replica URL\n3. Page requests are satisfied from cached data or fetched from remote storage\n4. A background goroutine polls for new LTX files at a configurable interval\n\n## Building\n\n### Prerequisites\n\n- Go 1.21+\n- GCC (Linux) or Clang (macOS)\n- CGO enabled\n\n### Build Commands\n\n**macOS (current architecture):**\n\n```bash\nmake vfs\n```\n\nThis creates:\n- `dist/litestream-vfs.a` - Static library\n- `dist/litestream-vfs.so` - Loadable SQLite extension\n\n**Platform-specific builds:**\n\n```bash\n# macOS ARM64 (Apple Silicon)\nmake vfs-darwin-arm64\n# Output: dist/litestream-vfs-darwin-arm64.dylib\n\n# macOS AMD64 (Intel)\nmake vfs-darwin-amd64\n# Output: dist/litestream-vfs-darwin-amd64.dylib\n\n# Linux AMD64\nmake vfs-linux-amd64\n# Output: dist/litestream-vfs-linux-amd64.so\n\n# Linux ARM64\nmake vfs-linux-arm64\n# Output: dist/litestream-vfs-linux-arm64.so\n```\n\n### Running Tests\n\n```bash\nmake vfs-test\n```\n\n## Configuration\n\nThe VFS is configured via environment variables:\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `LITESTREAM_REPLICA_URL` | Replica storage URL (required) | - |\n| `LITESTREAM_LOG_LEVEL` | Log level: `DEBUG` or `INFO` | `INFO` |\n| `LITESTREAM_WRITE_ENABLED` | Enable write mode: `true` or `false` | `false` |\n| `LITESTREAM_SYNC_INTERVAL` | Write sync interval (e.g., `1s`, `500ms`) | `1s` |\n| `LITESTREAM_BUFFER_PATH` | Local write buffer file path | temp file |\n\n### Replica URL Format\n\n```\n# Amazon S3\ns3://bucket-name/path/to/db\n\n# Google Cloud Storage\ngs://bucket-name/path/to/db\n\n# Azure Blob Storage\nabs://container-name/path/to/db\n\n# Alibaba OSS\noss://bucket-name/path/to/db\n\n# Local filesystem\nfile:///path/to/replica\n\n# SFTP\nsftp://user@host:port/path/to/db\n\n# NATS JetStream\nnats://host:port/bucket/path\n\n# WebDAV\nwebdav://host:port/path/to/db\n```\n\n## Usage\n\n### Loading the Extension\n\n```sql\n-- Load the extension (adjust path as needed)\n.load ./dist/litestream-vfs.so\n```\n\n### Opening a Database\n\nBefore loading the extension, set the replica URL:\n\n```bash\nexport LITESTREAM_REPLICA_URL=\"s3://my-bucket/mydb\"\n```\n\nThen in SQLite:\n\n```sql\n.load ./dist/litestream-vfs.so\n.open file:mydb.db?vfs=litestream\nSELECT * FROM my_table;\n```\n\n### Python Example\n\n```python\nimport os\nimport sqlite3\n\nos.environ[\"LITESTREAM_REPLICA_URL\"] = \"s3://my-bucket/mydb\"\n\nconn = sqlite3.connect(\":memory:\")\nconn.enable_load_extension(True)\nconn.load_extension(\"./dist/litestream-vfs.so\")\n\n# Open database using the litestream VFS\nconn = sqlite3.connect(\"file:mydb.db?vfs=litestream\")\ncursor = conn.execute(\"SELECT * FROM users\")\nfor row in cursor:\n print(row)\n```\n\n### Go Example\n\n```go\nimport (\n \"database/sql\"\n \"os\"\n\n _ \"github.com/mattn/go-sqlite3\"\n)\n\nfunc main() {\n os.Setenv(\"LITESTREAM_REPLICA_URL\", \"s3://my-bucket/mydb\")\n\n db, err := sql.Open(\"sqlite3\", \"file:mydb.db?vfs=litestream\")\n if err != nil {\n panic(err)\n }\n defer db.Close()\n\n rows, err := db.Query(\"SELECT * FROM users\")\n // ...\n}\n```\n\n## SQL Functions\n\nThe VFS extension provides SQL functions for observability and time travel:\n\n### `litestream_txid()`\n\nReturns the current transaction ID as a hex string.\n\n```sql\nSELECT litestream_txid();\n-- Returns: \"0000000000000042\"\n```\n\n### `litestream_time()`\n\nReturns the current view timestamp (RFC3339 format) or `\"latest\"`.\n\n```sql\nSELECT litestream_time();\n-- Returns: \"2024-01-15T10:30:00.123456789Z\" or \"latest\"\n```\n\n### `litestream_lag()`\n\nReturns seconds since the last successful poll for new LTX files. Returns `-1` if no successful poll has occurred.\n\n```sql\nSELECT litestream_lag();\n-- Returns: 2 (seconds behind primary)\n```\n\n### `litestream_set_time(timestamp)`\n\nSets the view time for time travel queries. See [Time Travel](#time-travel) for details.\n\n## Time Travel\n\nThe VFS supports querying historical database states by setting a target timestamp.\n\n### Setting a Target Time\n\n```sql\n-- View database as of a specific timestamp (RFC3339 format)\nSELECT litestream_set_time('2024-01-15T10:30:00Z');\n\n-- Relative time expressions are also supported\nSELECT litestream_set_time('5 minutes ago');\nSELECT litestream_set_time('yesterday');\nSELECT litestream_set_time('2 hours ago');\n\n-- Return to latest state\nSELECT litestream_set_time('LATEST');\n```\n\n### Example: Comparing Historical Data\n\n```sql\n-- Check current count\nSELECT COUNT(*) FROM orders;\n-- Returns: 1000\n\n-- Go back in time\nSELECT litestream_set_time('1 hour ago');\n\n-- Check historical count\nSELECT COUNT(*) FROM orders;\n-- Returns: 950\n\n-- Return to present\nSELECT litestream_set_time('LATEST');\n```\n\n### Time Travel Limitations\n\n- Time travel rebuilds the page index, which may take time for large databases\n- Historical data is only available if LTX files haven't been compacted away\n- L0 retention settings affect how far back you can travel\n- Time travel is read-only (writes are disabled while viewing historical state)\n\n## Write Mode\n\nWrite mode allows the VFS to accept writes and sync them back to remote storage. This is experimental.\n\n### Enabling Write Mode\n\n```bash\nexport LITESTREAM_REPLICA_URL=\"s3://my-bucket/mydb\"\nexport LITESTREAM_WRITE_ENABLED=\"true\"\nexport LITESTREAM_SYNC_INTERVAL=\"1s\"\n```\n\n### How Write Mode Works\n\n1. Writes are captured to a local buffer file for durability\n2. Dirty pages are tracked in memory\n3. Periodically (or on close), dirty pages are packaged into an LTX file\n4. The LTX file is uploaded to remote storage\n5. Conflict detection prevents overwrites if the remote has newer transactions\n\n### Write Mode Considerations\n\n- **Connection pooling**: Multiple connections can be opened in write mode (for example, by `database/sql`)\n- **Single writer**: Write contention is enforced at lock acquisition. If another connection already holds write intent, SQLite returns `SQLITE_BUSY`\n- **Conflict detection**: If the remote has advanced unexpectedly, `ErrConflict` is returned\n- **Buffer durability**: The local buffer file provides crash recovery for uncommitted writes\n- **Sync interval**: Balance between durability (shorter) and performance (longer)\n- **New databases**: Write mode can create new databases from scratch if no LTX files exist\n\n### Creating a New Database\n\nWith write mode enabled, you can create a new database that doesn't exist yet:\n\n```sql\n.load ./dist/litestream-vfs.so\n.open file:newdb.db?vfs=litestream\n\nCREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT);\nINSERT INTO users (name) VALUES ('Alice');\n-- Data is synced to remote storage automatically\n```\n\n## Supported Storage Backends\n\nThe VFS supports all Litestream storage backends:\n\n| Backend | URL Scheme | Notes |\n|---------|-----------|-------|\n| Amazon S3 | `s3://` | Supports S3-compatible services (MinIO, DigitalOcean Spaces, etc.) |\n| Google Cloud Storage | `gs://` | Requires `GOOGLE_APPLICATION_CREDENTIALS` |\n| Azure Blob Storage | `abs://` | Requires `AZURE_STORAGE_ACCOUNT` and credentials |\n| Alibaba OSS | `oss://` | Object Storage Service |\n| Local filesystem | `file://` | Useful for testing and development |\n| SFTP | `sftp://` | SSH File Transfer Protocol |\n| NATS JetStream | `nats://` | Object store via NATS |\n| WebDAV | `webdav://` | Web Distributed Authoring and Versioning |\n\n### S3 Configuration\n\nFor S3 and S3-compatible services, set credentials via environment variables:\n\n```bash\nexport AWS_ACCESS_KEY_ID=\"your-key\"\nexport AWS_SECRET_ACCESS_KEY=\"your-secret\"\nexport AWS_REGION=\"us-east-1\"\n\n# For S3-compatible services:\nexport LITESTREAM_REPLICA_URL=\"s3://bucket/path?endpoint=https://custom.endpoint.com\"\n```\n\n### GCS Configuration\n\n```bash\nexport GOOGLE_APPLICATION_CREDENTIALS=\"/path/to/service-account.json\"\nexport LITESTREAM_REPLICA_URL=\"gs://bucket/path\"\n```\n\n## Troubleshooting\n\n### Extension fails to load\n\nEnsure the extension file matches your platform:\n- macOS: `.dylib` or `.so`\n- Linux: `.so`\n\n### \"no backup files available\"\n\nThe VFS waits for LTX files to become available. Ensure:\n1. The replica URL is correct\n2. Litestream has replicated at least one transaction\n3. Credentials are properly configured\n\n### High latency reads\n\n- Increase `CacheSize` for larger page cache\n- Reduce `PollInterval` for more responsive updates\n- Consider using a closer storage region\n\n### Debug logging\n\n```bash\nexport LITESTREAM_LOG_LEVEL=\"DEBUG\"\n```\n\nThis enables verbose logging of VFS operations, page fetches, and cache hits/misses.\n" }, { "path": "etc/build.ps1", "content": "[CmdletBinding()]\nParam (\n [Parameter(Mandatory = $true)]\n [String] $Version\n)\n$ErrorActionPreference = \"Stop\"\n\n# Update working directory.\nPush-Location $PSScriptRoot\nTrap {\n Pop-Location\n}\n\nInvoke-Expression \"candle.exe -nologo -arch x64 -ext WixUtilExtension -out litestream.wixobj -dVersion=`\"$Version`\" litestream.wxs\"\nInvoke-Expression \"light.exe -nologo -spdb -ext WixUtilExtension -out `\"litestream-${Version}.msi`\" litestream.wixobj\"\n\nPop-Location\n" }, { "path": "etc/gon-sign.hcl", "content": "source = [\"./dist/litestream\"]\nbundle_id = \"com.middlemost.litestream\"\n\napple_id {\n username = \"@env:APPLE_ID_USERNAME\"\n password = \"@env:AC_PASSWORD\"\n provider = \"@env:APPLE_TEAM_ID\"\n}\n\nsign {\n application_identity = \"@env:APPLE_DEVELOPER_ID_APPLICATION\"\n entitlements_file = \"\"\n}\n\nnotarize {\n path = \"./dist/litestream.zip\"\n bundle_id = \"com.middlemost.litestream\"\n staple = true\n}\n\nzip {\n output_path = \"./dist/litestream-signed.zip\"\n}\n" }, { "path": "etc/gon.hcl", "content": "source = [\"./dist/litestream\"]\nbundle_id = \"com.middlemost.litestream\"\n\napple_id {\n username = \"benbjohnson@yahoo.com\"\n password = \"@env:AC_PASSWORD\"\n}\n\nsign {\n application_identity = \"Developer ID Application: Middlemost Systems, LLC\"\n}\n\nzip {\n output_path = \"dist/litestream.zip\"\n}\n" }, { "path": "etc/litestream.service", "content": "[Unit]\nDescription=Litestream\n\n[Service]\nRestart=always\nExecStart=/usr/bin/litestream replicate\n\n[Install]\nWantedBy=multi-user.target\n" }, { "path": "etc/litestream.wxs", "content": "\n\n\t\n\t\t\n\t\n\t\t\n\t\n\n\t\n\t\t\n\n\t\t\n\n\t\t\n\n\t\t\n\t\t\t\n\t\t\t\t\n\t\t\t\n\t\t\n\n\t\t\n\t\t\t\n\t\t\t\t\n\n\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\n\n\t\t\t\t\n\t\t\t\t\n\t\t\t\n\t\t\n\n\t\t\n\t\t\t\n\t\t\n\t\n\n" }, { "path": "etc/litestream.yml", "content": "# AWS credentials\n# access-key-id: AKIAxxxxxxxxxxxxxxxx\n# secret-access-key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxx\n\n# Control socket for runtime start/stop commands\n# socket:\n# enabled: true # Enable control socket (default: false)\n# path: /var/run/litestream.sock # Socket path (default: /var/run/litestream.sock)\n# permissions: 0600 # Socket file permissions (default: 0600)\n\n# dbs:\n# - path: /path/to/primary/db # Database to replicate from\n# replica:\n# path: /path/to/replica # File-based replication\n# url: s3://my.bucket.com/db # S3-based replication\n# # Example Fly.io Tigris setup (signing/no-Content-MD5 are auto-enabled for this endpoint)\n# # url: s3://my-tigris-bucket/db.sqlite?endpoint=fly.storage.tigris.dev®ion=auto\n# type: nats # NATS JetStream replication\n# url: nats://nats.example.com:4222\n# bucket: litestream-backups\n# # Optional TLS configuration:\n# # client-cert: /path/to/client.pem\n# # client-key: /path/to/client.key\n# # root-cas: [/path/to/ca.pem]\n# - path: /path/to/another/db\n# replica:\n# url: sftp://user@host:22/path # SFTP-based replication\n# key-path: /etc/litestream/sftp_key\n# # Strongly recommended: SSH host key for verification\n# # Get this from the server's /etc/ssh/ssh_host_*.pub file\n# # or use `ssh-keyscan hostname`\n# # Example formats:\n# # host-key: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMvvypUkBrS9RCyV//p+UFCLg8yKNtTu/ew/cV6XXAAP\n# # host-key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQ...\n" }, { "path": "etc/nfpm.yml", "content": "name: litestream\narch: \"${GOARCH}\"\nplatform: \"${GOOS}\"\nversion: \"${LITESTREAM_VERSION}\"\nsection: \"default\"\npriority: \"extra\"\nmaintainer: \"Ben Johnson \"\ndescription: Litestream is a tool for real-time replication of SQLite databases.\nhomepage: \"https://github.com/benbjohnson/litestream\"\nlicense: \"Apache 2\"\ncontents:\n- src: ./litestream\n dst: /usr/bin/litestream\n- src: ./litestream.yml\n dst: /etc/litestream.yml\n type: config\n- src: ./litestream.service\n dst: /usr/lib/systemd/system/litestream.service\n type: config\n" }, { "path": "etc/run-s3-docker-tests.sh", "content": "#!/bin/bash\nset -e\n\n# Script to run S3 integration tests against a local MinIO container.\n# This provides a more realistic test environment than moto for testing\n# S3-compatible provider compatibility.\n\nSCRIPT_DIR=\"$(cd \"$(dirname \"${BASH_SOURCE[0]}\")\" && pwd)\"\nPROJECT_ROOT=\"$(dirname \"$SCRIPT_DIR\")\"\n\ncd \"$PROJECT_ROOT\"\n\necho \"Starting MinIO container...\"\ndocker compose -f docker-compose.test.yml up -d\n\n# Wait for MinIO to be ready\necho \"Waiting for MinIO to be ready...\"\nfor i in {1..30}; do\n if docker compose -f docker-compose.test.yml exec -T minio mc ready local 2>/dev/null; then\n echo \"MinIO is ready\"\n break\n fi\n if [ $i -eq 30 ]; then\n echo \"MinIO failed to start\"\n docker compose -f docker-compose.test.yml logs minio\n docker compose -f docker-compose.test.yml down\n exit 1\n fi\n sleep 1\ndone\n\n# Create test bucket using mc client inside the container\necho \"Creating test bucket...\"\ndocker compose -f docker-compose.test.yml exec -T minio mc alias set local http://localhost:9000 minioadmin minioadmin\ndocker compose -f docker-compose.test.yml exec -T minio mc mb local/test-bucket --ignore-existing\n\n# Set up cleanup trap\ncleanup() {\n echo \"Cleaning up...\"\n docker compose -f docker-compose.test.yml down\n}\ntrap cleanup EXIT\n\n# Export environment variables for the S3 integration tests\nexport LITESTREAM_S3_ACCESS_KEY_ID=minioadmin\nexport LITESTREAM_S3_SECRET_ACCESS_KEY=minioadmin\nexport LITESTREAM_S3_BUCKET=test-bucket\nexport LITESTREAM_S3_ENDPOINT=http://localhost:9000\nexport LITESTREAM_S3_FORCE_PATH_STYLE=true\nexport LITESTREAM_S3_REGION=us-east-1\n\necho \"Running S3 integration tests against MinIO...\"\ngo test -v ./replica_client_test.go -integration -replica-clients=s3 \"$@\"\n\necho \"Tests completed successfully!\"\n" }, { "path": "etc/s3_mock.py", "content": "#!/usr/bin/env python3\nimport sys\nimport os\nimport time\nfrom moto.server import ThreadedMotoServer\nimport boto3\nimport subprocess\n\ncmd = sys.argv[1:]\nif len(cmd) == 0:\n print(f\"usage: {sys.argv[0]} [arguments]\", file=sys.stderr)\n sys.exit(1)\n\nenv = os.environ.copy() | {\n \"LITESTREAM_S3_ACCESS_KEY_ID\": \"lite\",\n \"LITESTREAM_S3_SECRET_ACCESS_KEY\": \"stream\",\n \"LITESTREAM_S3_BUCKET\": f\"test{int(time.time())}\",\n \"LITESTREAM_S3_ENDPOINT\": \"http://127.0.0.1:5000\",\n \"LITESTREAM_S3_FORCE_PATH_STYLE\": \"true\",\n}\n\nserver = ThreadedMotoServer()\nserver.start()\n\ns3 = boto3.client(\n \"s3\",\n aws_access_key_id=env[\"LITESTREAM_S3_ACCESS_KEY_ID\"],\n aws_secret_access_key=env[\"LITESTREAM_S3_SECRET_ACCESS_KEY\"],\n endpoint_url=env[\"LITESTREAM_S3_ENDPOINT\"]\n).create_bucket(Bucket=env[\"LITESTREAM_S3_BUCKET\"])\n\nproc = subprocess.run(cmd, env=env)\n\nserver.stop()\nsys.exit(proc.returncode)\n" }, { "path": "file/replica_client.go", "content": "package file\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net/url\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"slices\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\nfunc init() {\n\tlitestream.RegisterReplicaClientFactory(\"file\", NewReplicaClientFromURL)\n}\n\n// ReplicaClientType is the client type for this package.\nconst ReplicaClientType = \"file\"\n\nvar _ litestream.ReplicaClient = (*ReplicaClient)(nil)\nvar _ litestream.ReplicaClientV3 = (*ReplicaClient)(nil)\n\n// ReplicaClient is a client for writing LTX files to disk.\ntype ReplicaClient struct {\n\tpath string // destination path\n\n\tReplica *litestream.Replica\n\tlogger *slog.Logger\n}\n\n// NewReplicaClient returns a new instance of ReplicaClient.\nfunc NewReplicaClient(path string) *ReplicaClient {\n\treturn &ReplicaClient{\n\t\tlogger: slog.Default().WithGroup(ReplicaClientType),\n\t\tpath: path,\n\t}\n}\n\nfunc (c *ReplicaClient) SetLogger(logger *slog.Logger) {\n\tc.logger = logger.WithGroup(ReplicaClientType)\n}\n\n// NewReplicaClientFromURL creates a new ReplicaClient from URL components.\n// This is used by the replica client factory registration.\nfunc NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error) {\n\t// For file URLs, the path is the full path\n\tif urlPath == \"\" {\n\t\treturn nil, fmt.Errorf(\"file replica path required\")\n\t}\n\treturn NewReplicaClient(urlPath), nil\n}\n\n// db returns the database, if available.\nfunc (c *ReplicaClient) db() *litestream.DB {\n\tif c.Replica == nil {\n\t\treturn nil\n\t}\n\treturn c.Replica.DB()\n}\n\n// Type returns \"file\" as the client type.\nfunc (c *ReplicaClient) Type() string {\n\treturn ReplicaClientType\n}\n\n// Init is a no-op for file replica client as no initialization is required.\nfunc (c *ReplicaClient) Init(ctx context.Context) error {\n\treturn nil\n}\n\n// Path returns the destination path to replicate the database to.\nfunc (c *ReplicaClient) Path() string {\n\treturn c.path\n}\n\n// LTXLevelDir returns the path to a given level.\nfunc (c *ReplicaClient) LTXLevelDir(level int) string {\n\treturn filepath.FromSlash(litestream.LTXLevelDir(c.path, level))\n}\n\n// LTXFilePath returns the path to an LTX file.\nfunc (c *ReplicaClient) LTXFilePath(level int, minTXID, maxTXID ltx.TXID) string {\n\treturn filepath.FromSlash(litestream.LTXFilePath(c.path, level, minTXID, maxTXID))\n}\n\n// LTXFiles returns an iterator over all LTX files on the replica for the given level.\n// The useMetadata parameter is ignored for file backend as ModTime is always available from readdir.\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tf, err := os.Open(c.LTXLevelDir(level))\n\tif os.IsNotExist(err) {\n\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t} else if err != nil {\n\t\treturn nil, err\n\t}\n\tdefer f.Close()\n\n\tfis, err := f.Readdir(-1)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Iterate over every file and convert to metadata.\n\t// ModTime contains the accurate timestamp set by Chtimes in WriteLTXFile.\n\tinfos := make([]*ltx.FileInfo, 0, len(fis))\n\tfor _, fi := range fis {\n\t\tminTXID, maxTXID, err := ltx.ParseFilename(fi.Name())\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t} else if minTXID < seek {\n\t\t\tcontinue\n\t\t}\n\n\t\tinfos = append(infos, <x.FileInfo{\n\t\t\tLevel: level,\n\t\t\tMinTXID: minTXID,\n\t\t\tMaxTXID: maxTXID,\n\t\t\tSize: fi.Size(),\n\t\t\tCreatedAt: fi.ModTime().UTC(),\n\t\t})\n\t}\n\n\treturn ltx.NewFileInfoSliceIterator(infos), nil\n}\n\n// OpenLTXFile returns a reader for an LTX file at the given position.\n// Returns os.ErrNotExist if no matching index/offset is found.\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tpath := c.LTXFilePath(level, minTXID, maxTXID)\n\tf, err := os.Open(path)\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\treturn nil, litestream.NewLTXError(\"open\", path, level, uint64(minTXID), uint64(maxTXID), err)\n\t\t}\n\t\treturn nil, fmt.Errorf(\"open ltx file %s: %w\", path, err)\n\t}\n\n\tif offset > 0 {\n\t\tif _, err := f.Seek(offset, io.SeekStart); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif size > 0 {\n\t\treturn internal.LimitReadCloser(f, size), nil\n\t}\n\n\treturn f, nil\n}\n\n// WriteLTXFile writes an LTX file to the replica.\n// Extracts timestamp from LTX header and sets it as the file's ModTime to preserve original creation time.\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, rd io.Reader) (info *ltx.FileInfo, err error) {\n\tvar fileInfo, dirInfo os.FileInfo\n\tif db := c.db(); db != nil {\n\t\tfileInfo, dirInfo = db.FileInfo(), db.DirInfo()\n\t}\n\n\t// Use TeeReader to peek at LTX header while preserving data for upload\n\tvar buf bytes.Buffer\n\tteeReader := io.TeeReader(rd, &buf)\n\n\t// Extract timestamp from LTX header\n\thdr, _, err := ltx.PeekHeader(teeReader)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"extract timestamp from LTX header: %w\", err)\n\t}\n\ttimestamp := time.UnixMilli(hdr.Timestamp).UTC()\n\n\t// Combine buffered data with rest of reader\n\tfullReader := io.MultiReader(&buf, rd)\n\n\t// Ensure parent directory exists.\n\tfilename := c.LTXFilePath(level, minTXID, maxTXID)\n\tif err := internal.MkdirAll(filepath.Dir(filename), dirInfo); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Write LTX file to temporary file next to destination path.\n\ttmpFilename := filename + \".tmp\"\n\tf, err := internal.CreateFile(tmpFilename, fileInfo)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Clean up temp file on error. On successful rename, the temp file\n\t// becomes the final file and should not be removed.\n\tdefer func() {\n\t\t_ = f.Close()\n\t\tif err != nil {\n\t\t\t_ = os.Remove(tmpFilename)\n\t\t}\n\t}()\n\n\tif _, err := io.Copy(f, fullReader); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := f.Sync(); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Build metadata.\n\tfi, err := f.Stat()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tinfo = <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tSize: fi.Size(),\n\t\tCreatedAt: timestamp,\n\t}\n\n\tif err := f.Close(); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Move LTX file to final path when it has been written & synced to disk.\n\tif err := os.Rename(tmpFilename, filename); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Set file ModTime to preserve original timestamp\n\tif err := os.Chtimes(filename, timestamp, timestamp); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn info, nil\n}\n\n// DeleteLTXFiles deletes LTX files.\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\tfor _, info := range a {\n\t\tfilename := c.LTXFilePath(info.Level, info.MinTXID, info.MaxTXID)\n\n\t\tc.logger.Debug(\"deleting ltx file\", \"level\", info.Level, \"minTXID\", info.MinTXID, \"maxTXID\", info.MaxTXID, \"path\", filename)\n\n\t\tif err := os.Remove(filename); err != nil && !os.IsNotExist(err) {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\n// DeleteAll deletes all LTX files.\nfunc (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n\tif err := os.RemoveAll(c.path); err != nil && !os.IsNotExist(err) {\n\t\treturn err\n\t}\n\treturn nil\n}\n\n// GenerationsV3 returns a list of v0.3.x generation IDs in the replica.\nfunc (c *ReplicaClient) GenerationsV3(ctx context.Context) ([]string, error) {\n\tgenPath := filepath.Join(c.path, litestream.GenerationsDirV3)\n\tentries, err := os.ReadDir(genPath)\n\tif os.IsNotExist(err) {\n\t\treturn nil, nil\n\t} else if err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar generations []string\n\tfor _, entry := range entries {\n\t\tif entry.IsDir() && litestream.IsGenerationIDV3(entry.Name()) {\n\t\t\tgenerations = append(generations, entry.Name())\n\t\t}\n\t}\n\tslices.Sort(generations)\n\treturn generations, nil\n}\n\n// SnapshotsV3 returns snapshots for a generation, sorted by index.\nfunc (c *ReplicaClient) SnapshotsV3(ctx context.Context, generation string) ([]litestream.SnapshotInfoV3, error) {\n\tsnapshotsPath := filepath.Join(c.path, litestream.GenerationsDirV3, generation, litestream.SnapshotsDirV3)\n\tentries, err := os.ReadDir(snapshotsPath)\n\tif os.IsNotExist(err) {\n\t\treturn nil, nil\n\t} else if err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar snapshots []litestream.SnapshotInfoV3\n\tfor _, entry := range entries {\n\t\tif entry.IsDir() {\n\t\t\tcontinue\n\t\t}\n\t\tindex, err := litestream.ParseSnapshotFilenameV3(entry.Name())\n\t\tif err != nil {\n\t\t\tcontinue // skip invalid filenames\n\t\t}\n\t\tinfo, err := entry.Info()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tsnapshots = append(snapshots, litestream.SnapshotInfoV3{\n\t\t\tGeneration: generation,\n\t\t\tIndex: index,\n\t\t\tSize: info.Size(),\n\t\t\tCreatedAt: info.ModTime(),\n\t\t})\n\t}\n\tslices.SortFunc(snapshots, func(a, b litestream.SnapshotInfoV3) int {\n\t\treturn a.Index - b.Index\n\t})\n\treturn snapshots, nil\n}\n\n// WALSegmentsV3 returns WAL segments for a generation, sorted by index then offset.\nfunc (c *ReplicaClient) WALSegmentsV3(ctx context.Context, generation string) ([]litestream.WALSegmentInfoV3, error) {\n\twalPath := filepath.Join(c.path, litestream.GenerationsDirV3, generation, litestream.WALDirV3)\n\tentries, err := os.ReadDir(walPath)\n\tif os.IsNotExist(err) {\n\t\treturn nil, nil\n\t} else if err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar segments []litestream.WALSegmentInfoV3\n\tfor _, entry := range entries {\n\t\tif entry.IsDir() {\n\t\t\tcontinue\n\t\t}\n\t\tindex, offset, err := litestream.ParseWALSegmentFilenameV3(entry.Name())\n\t\tif err != nil {\n\t\t\tcontinue // skip invalid filenames\n\t\t}\n\t\tinfo, err := entry.Info()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tsegments = append(segments, litestream.WALSegmentInfoV3{\n\t\t\tGeneration: generation,\n\t\t\tIndex: index,\n\t\t\tOffset: offset,\n\t\t\tSize: info.Size(),\n\t\t\tCreatedAt: info.ModTime(),\n\t\t})\n\t}\n\tslices.SortFunc(segments, func(a, b litestream.WALSegmentInfoV3) int {\n\t\tif a.Index != b.Index {\n\t\t\treturn a.Index - b.Index\n\t\t}\n\t\treturn int(a.Offset - b.Offset)\n\t})\n\treturn segments, nil\n}\n\n// OpenSnapshotV3 opens a v0.3.x snapshot file for reading.\n// The returned reader provides LZ4-decompressed data.\nfunc (c *ReplicaClient) OpenSnapshotV3(ctx context.Context, generation string, index int) (io.ReadCloser, error) {\n\tpath := filepath.Join(c.path, litestream.GenerationsDirV3, generation, litestream.SnapshotsDirV3, litestream.FormatSnapshotFilenameV3(index))\n\tf, err := os.Open(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn internal.NewLZ4Reader(f), nil\n}\n\n// OpenWALSegmentV3 opens a v0.3.x WAL segment file for reading.\n// The returned reader provides LZ4-decompressed data.\nfunc (c *ReplicaClient) OpenWALSegmentV3(ctx context.Context, generation string, index int, offset int64) (io.ReadCloser, error) {\n\tpath := filepath.Join(c.path, litestream.GenerationsDirV3, generation, litestream.WALDirV3, litestream.FormatWALSegmentFilenameV3(index, offset))\n\tf, err := os.Open(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn internal.NewLZ4Reader(f), nil\n}\n" }, { "path": "file/replica_client_test.go", "content": "package file_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pierrec/lz4/v4\"\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream/file\"\n)\n\nfunc TestReplicaClient_Path(t *testing.T) {\n\tc := file.NewReplicaClient(\"/foo/bar\")\n\tif got, want := c.Path(), \"/foo/bar\"; got != want {\n\t\tt.Fatalf(\"Path()=%v, want %v\", got, want)\n\t}\n}\n\nfunc TestReplicaClient_Type(t *testing.T) {\n\tif got, want := file.NewReplicaClient(\"\").Type(), \"file\"; got != want {\n\t\tt.Fatalf(\"Type()=%v, want %v\", got, want)\n\t}\n}\n\n// TestReplicaClient_WriteLTXFile_ErrorCleanup verifies temp files are cleaned up on errors\nfunc TestReplicaClient_WriteLTXFile_ErrorCleanup(t *testing.T) {\n\tt.Run(\"DiskFull\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\t// Create a reader that fails after 50 bytes to simulate disk full\n\t\tfailReader := &failAfterReader{\n\t\t\tdata: createLTXHeader(1, 2),\n\t\t\tn: 50,\n\t\t\terr: fmt.Errorf(\"no space left on device\"),\n\t\t}\n\n\t\t_, err := c.WriteLTXFile(context.Background(), 0, 1, 2, failReader)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error from failReader\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"no space left on device\") {\n\t\t\tt.Fatalf(\"expected disk full error, got: %v\", err)\n\t\t}\n\n\t\t// Verify no .tmp files remain\n\t\ttmpFiles := findTmpFiles(t, tmpDir)\n\t\tif len(tmpFiles) > 0 {\n\t\t\tt.Fatalf(\"found %d .tmp files after error: %v\", len(tmpFiles), tmpFiles)\n\t\t}\n\t})\n\n\tt.Run(\"SuccessNoLeaks\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tltxData := createLTXData(1, 2, []byte(\"test data\"))\n\t\tinfo, err := c.WriteLTXFile(context.Background(), 0, 1, 2, bytes.NewReader(ltxData))\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info == nil {\n\t\t\tt.Fatal(\"expected FileInfo\")\n\t\t}\n\n\t\t// Verify no .tmp files remain\n\t\ttmpFiles := findTmpFiles(t, tmpDir)\n\t\tif len(tmpFiles) > 0 {\n\t\t\tt.Fatalf(\"found %d .tmp files after successful write: %v\", len(tmpFiles), tmpFiles)\n\t\t}\n\n\t\t// Verify final file exists\n\t\tfinalPath := c.LTXFilePath(0, 1, 2)\n\t\tif _, err := os.Stat(finalPath); err != nil {\n\t\t\tt.Fatalf(\"final file missing: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"MultipleErrors\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\t// Simulate multiple failed writes\n\t\tfor i := 0; i < 5; i++ {\n\t\t\tfailReader := &failAfterReader{\n\t\t\t\tdata: createLTXHeader(ltx.TXID(i+1), ltx.TXID(i+1)),\n\t\t\t\tn: 30,\n\t\t\t\terr: fmt.Errorf(\"write error %d\", i),\n\t\t\t}\n\n\t\t\t_, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(i+1), ltx.TXID(i+1), failReader)\n\t\t\tif err == nil {\n\t\t\t\tt.Fatalf(\"iteration %d: expected error from failReader\", i)\n\t\t\t}\n\t\t}\n\n\t\t// Verify no .tmp files accumulated\n\t\ttmpFiles := findTmpFiles(t, tmpDir)\n\t\tif len(tmpFiles) > 0 {\n\t\t\tt.Fatalf(\"found %d .tmp files after multiple errors: %v\", len(tmpFiles), tmpFiles)\n\t\t}\n\t})\n}\n\n// failAfterReader simulates io.Copy failure after reading n bytes\ntype failAfterReader struct {\n\tdata []byte\n\tn int // fail after n bytes\n\tpos int\n\terr error\n}\n\nfunc (r *failAfterReader) Read(p []byte) (n int, err error) {\n\tif r.pos >= r.n {\n\t\treturn 0, r.err\n\t}\n\tremaining := r.n - r.pos\n\ttoRead := len(p)\n\tif toRead > remaining {\n\t\ttoRead = remaining\n\t}\n\tif toRead > len(r.data)-r.pos {\n\t\ttoRead = len(r.data) - r.pos\n\t}\n\tif toRead == 0 {\n\t\treturn 0, r.err\n\t}\n\tn = copy(p, r.data[r.pos:r.pos+toRead])\n\tr.pos += n\n\treturn n, nil\n}\n\n// findTmpFiles recursively finds all .tmp files in the directory\nfunc findTmpFiles(t *testing.T, root string) []string {\n\tt.Helper()\n\tvar tmpFiles []string\n\terr := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {\n\t\tif err != nil {\n\t\t\treturn nil\n\t\t}\n\t\tif !info.IsDir() && strings.HasSuffix(path, \".tmp\") {\n\t\t\ttmpFiles = append(tmpFiles, path)\n\t\t}\n\t\treturn nil\n\t})\n\tif err != nil {\n\t\tt.Fatalf(\"walk error: %v\", err)\n\t}\n\treturn tmpFiles\n}\n\n// createLTXData creates a minimal valid LTX file with a header for testing\nfunc createLTXData(minTXID, maxTXID ltx.TXID, data []byte) []byte {\n\thdr := ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tPageSize: 4096,\n\t\tCommit: 1,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tTimestamp: time.Now().UnixMilli(),\n\t}\n\tif minTXID == 1 {\n\t\thdr.PreApplyChecksum = 0\n\t} else {\n\t\thdr.PreApplyChecksum = ltx.ChecksumFlag\n\t}\n\theaderBytes, _ := hdr.MarshalBinary()\n\treturn append(headerBytes, data...)\n}\n\n// createLTXHeader creates minimal LTX header for testing\nfunc createLTXHeader(minTXID, maxTXID ltx.TXID) []byte {\n\treturn createLTXData(minTXID, maxTXID, nil)\n}\n\nfunc TestReplicaClient_GenerationsV3(t *testing.T) {\n\tt.Run(\"NoGenerationsDir\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tgenerations, err := c.GenerationsV3(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(generations) != 0 {\n\t\t\tt.Fatalf(\"expected no generations, got %v\", generations)\n\t\t}\n\t})\n\n\tt.Run(\"EmptyGenerationsDir\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tif err := os.MkdirAll(filepath.Join(tmpDir, \"generations\"), 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tgenerations, err := c.GenerationsV3(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(generations) != 0 {\n\t\t\tt.Fatalf(\"expected no generations, got %v\", generations)\n\t\t}\n\t})\n\n\tt.Run(\"MultipleGenerationsSorted\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tgenDir := filepath.Join(tmpDir, \"generations\")\n\t\t// Create in non-sorted order\n\t\tfor _, gen := range []string{\"ffffffffffffffff\", \"0000000000000001\", \"aaaaaaaaaaaaaaaa\"} {\n\t\t\tif err := os.MkdirAll(filepath.Join(genDir, gen), 0755); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tgenerations, err := c.GenerationsV3(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\twant := []string{\"0000000000000001\", \"aaaaaaaaaaaaaaaa\", \"ffffffffffffffff\"}\n\t\tif len(generations) != len(want) {\n\t\t\tt.Fatalf(\"got %d generations, want %d\", len(generations), len(want))\n\t\t}\n\t\tfor i, g := range generations {\n\t\t\tif g != want[i] {\n\t\t\t\tt.Errorf(\"generations[%d] = %q, want %q\", i, g, want[i])\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"SkipsInvalidIDs\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tgenDir := filepath.Join(tmpDir, \"generations\")\n\t\t// Create mix of valid and invalid\n\t\tfor _, name := range []string{\n\t\t\t\"0000000000000001\", // valid\n\t\t\t\"invalid\", // invalid - not hex\n\t\t\t\"0123456789abcde\", // invalid - 15 chars\n\t\t\t\"0123456789ABCDEF\", // invalid - uppercase\n\t\t\t\"aaaaaaaaaaaaaaaa\", // valid\n\t\t} {\n\t\t\tif err := os.MkdirAll(filepath.Join(genDir, name), 0755); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tgenerations, err := c.GenerationsV3(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\twant := []string{\"0000000000000001\", \"aaaaaaaaaaaaaaaa\"}\n\t\tif len(generations) != len(want) {\n\t\t\tt.Fatalf(\"got %d generations, want %d: %v\", len(generations), len(want), generations)\n\t\t}\n\t\tfor i, g := range generations {\n\t\t\tif g != want[i] {\n\t\t\t\tt.Errorf(\"generations[%d] = %q, want %q\", i, g, want[i])\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"SkipsFiles\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tgenDir := filepath.Join(tmpDir, \"generations\")\n\t\tif err := os.MkdirAll(genDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\t// Create a file with valid generation name (should be skipped)\n\t\tif err := os.WriteFile(filepath.Join(genDir, \"0000000000000001\"), []byte(\"test\"), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\t// Create a valid directory\n\t\tif err := os.MkdirAll(filepath.Join(genDir, \"aaaaaaaaaaaaaaaa\"), 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tgenerations, err := c.GenerationsV3(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(generations) != 1 || generations[0] != \"aaaaaaaaaaaaaaaa\" {\n\t\t\tt.Fatalf(\"expected only valid directory, got %v\", generations)\n\t\t}\n\t})\n}\n\nfunc TestReplicaClient_SnapshotsV3(t *testing.T) {\n\tgen := \"0123456789abcdef\"\n\n\tt.Run(\"NoSnapshotsDir\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tsnapshots, err := c.SnapshotsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(snapshots) != 0 {\n\t\t\tt.Fatalf(\"expected no snapshots, got %v\", snapshots)\n\t\t}\n\t})\n\n\tt.Run(\"EmptySnapshotsDir\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tif err := os.MkdirAll(filepath.Join(tmpDir, \"generations\", gen, \"snapshots\"), 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tsnapshots, err := c.SnapshotsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(snapshots) != 0 {\n\t\t\tt.Fatalf(\"expected no snapshots, got %v\", snapshots)\n\t\t}\n\t})\n\n\tt.Run(\"SingleSnapshot\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tsnapshotsDir := filepath.Join(tmpDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\t// Create snapshot file\n\t\tpath := filepath.Join(snapshotsDir, \"00000001.snapshot.lz4\")\n\t\tif err := os.WriteFile(path, []byte(\"test data\"), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tsnapshots, err := c.SnapshotsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(snapshots) != 1 {\n\t\t\tt.Fatalf(\"expected 1 snapshot, got %d\", len(snapshots))\n\t\t}\n\t\tif snapshots[0].Generation != gen {\n\t\t\tt.Errorf(\"Generation = %q, want %q\", snapshots[0].Generation, gen)\n\t\t}\n\t\tif snapshots[0].Index != 1 {\n\t\t\tt.Errorf(\"Index = %d, want 1\", snapshots[0].Index)\n\t\t}\n\t\tif snapshots[0].Size != 9 { // len(\"test data\")\n\t\t\tt.Errorf(\"Size = %d, want 9\", snapshots[0].Size)\n\t\t}\n\t})\n\n\tt.Run(\"MultipleSnapshotsSorted\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tsnapshotsDir := filepath.Join(tmpDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\t// Create snapshots in non-sorted order\n\t\tfor _, idx := range []int{0xff, 0x01, 0x10} {\n\t\t\tfilename := fmt.Sprintf(\"%08x.snapshot.lz4\", idx)\n\t\t\tif err := os.WriteFile(filepath.Join(snapshotsDir, filename), []byte(\"x\"), 0644); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tsnapshots, err := c.SnapshotsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(snapshots) != 3 {\n\t\t\tt.Fatalf(\"expected 3 snapshots, got %d\", len(snapshots))\n\t\t}\n\t\twantIndices := []int{0x01, 0x10, 0xff}\n\t\tfor i, s := range snapshots {\n\t\t\tif s.Index != wantIndices[i] {\n\t\t\t\tt.Errorf(\"snapshots[%d].Index = %d, want %d\", i, s.Index, wantIndices[i])\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"SkipsInvalidFilenames\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tsnapshotsDir := filepath.Join(tmpDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\t// Create mix of valid and invalid\n\t\tfiles := map[string]bool{\n\t\t\t\"00000001.snapshot.lz4\": true, // valid\n\t\t\t\"invalid.snapshot.lz4\": false, // invalid\n\t\t\t\"00000002.snapshot\": false, // missing .lz4\n\t\t\t\"00000003.wal.lz4\": false, // wrong type\n\t\t\t\"00000004.snapshot.lz4\": true, // valid\n\t\t}\n\t\tfor name := range files {\n\t\t\tif err := os.WriteFile(filepath.Join(snapshotsDir, name), []byte(\"x\"), 0644); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tsnapshots, err := c.SnapshotsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(snapshots) != 2 {\n\t\t\tt.Fatalf(\"expected 2 valid snapshots, got %d\", len(snapshots))\n\t\t}\n\t})\n}\n\nfunc TestReplicaClient_WALSegmentsV3(t *testing.T) {\n\tgen := \"0123456789abcdef\"\n\n\tt.Run(\"NoWALDir\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tsegments, err := c.WALSegmentsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(segments) != 0 {\n\t\t\tt.Fatalf(\"expected no segments, got %v\", segments)\n\t\t}\n\t})\n\n\tt.Run(\"EmptyWALDir\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tif err := os.MkdirAll(filepath.Join(tmpDir, \"generations\", gen, \"wal\"), 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tsegments, err := c.WALSegmentsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(segments) != 0 {\n\t\t\tt.Fatalf(\"expected no segments, got %v\", segments)\n\t\t}\n\t})\n\n\tt.Run(\"SingleSegment\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\twalDir := filepath.Join(tmpDir, \"generations\", gen, \"wal\")\n\t\tif err := os.MkdirAll(walDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\t// Create WAL segment file\n\t\tpath := filepath.Join(walDir, \"00000001_00001000.wal.lz4\")\n\t\tif err := os.WriteFile(path, []byte(\"wal data\"), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tsegments, err := c.WALSegmentsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(segments) != 1 {\n\t\t\tt.Fatalf(\"expected 1 segment, got %d\", len(segments))\n\t\t}\n\t\tif segments[0].Generation != gen {\n\t\t\tt.Errorf(\"Generation = %q, want %q\", segments[0].Generation, gen)\n\t\t}\n\t\tif segments[0].Index != 1 {\n\t\t\tt.Errorf(\"Index = %d, want 1\", segments[0].Index)\n\t\t}\n\t\tif segments[0].Offset != 0x1000 {\n\t\t\tt.Errorf(\"Offset = %d, want %d\", segments[0].Offset, 0x1000)\n\t\t}\n\t\tif segments[0].Size != 8 { // len(\"wal data\")\n\t\t\tt.Errorf(\"Size = %d, want 8\", segments[0].Size)\n\t\t}\n\t})\n\n\tt.Run(\"MultipleSortedByIndexThenOffset\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\twalDir := filepath.Join(tmpDir, \"generations\", gen, \"wal\")\n\t\tif err := os.MkdirAll(walDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\t// Create WAL segments in non-sorted order\n\t\tsegments := []struct {\n\t\t\tindex int\n\t\t\toffset int64\n\t\t}{\n\t\t\t{2, 0x2000},\n\t\t\t{1, 0x1000},\n\t\t\t{1, 0x0000},\n\t\t\t{2, 0x0000},\n\t\t}\n\t\tfor _, s := range segments {\n\t\t\tfilename := fmt.Sprintf(\"%08x_%08x.wal.lz4\", s.index, s.offset)\n\t\t\tif err := os.WriteFile(filepath.Join(walDir, filename), []byte(\"x\"), 0644); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tresult, err := c.WALSegmentsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(result) != 4 {\n\t\t\tt.Fatalf(\"expected 4 segments, got %d\", len(result))\n\t\t}\n\t\t// Verify sorted order: index 1 offset 0, index 1 offset 0x1000, index 2 offset 0, index 2 offset 0x2000\n\t\texpected := []struct {\n\t\t\tindex int\n\t\t\toffset int64\n\t\t}{\n\t\t\t{1, 0x0000},\n\t\t\t{1, 0x1000},\n\t\t\t{2, 0x0000},\n\t\t\t{2, 0x2000},\n\t\t}\n\t\tfor i, s := range result {\n\t\t\tif s.Index != expected[i].index || s.Offset != expected[i].offset {\n\t\t\t\tt.Errorf(\"segments[%d] = (%d, %d), want (%d, %d)\",\n\t\t\t\t\ti, s.Index, s.Offset, expected[i].index, expected[i].offset)\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"SkipsInvalidFilenames\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\twalDir := filepath.Join(tmpDir, \"generations\", gen, \"wal\")\n\t\tif err := os.MkdirAll(walDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\t// Create mix of valid and invalid\n\t\tfiles := []string{\n\t\t\t\"00000001_00000000.wal.lz4\", // valid\n\t\t\t\"invalid.wal.lz4\", // invalid\n\t\t\t\"00000002.wal.lz4\", // missing offset\n\t\t\t\"00000003_00001000.wal\", // missing .lz4\n\t\t\t\"00000004_00002000.wal.lz4\", // valid\n\t\t}\n\t\tfor _, name := range files {\n\t\t\tif err := os.WriteFile(filepath.Join(walDir, name), []byte(\"x\"), 0644); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\tresult, err := c.WALSegmentsV3(context.Background(), gen)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(result) != 2 {\n\t\t\tt.Fatalf(\"expected 2 valid segments, got %d\", len(result))\n\t\t}\n\t})\n}\n\nfunc TestReplicaClient_OpenSnapshotV3(t *testing.T) {\n\tgen := \"0123456789abcdef\"\n\n\tt.Run(\"NotFound\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\t_, err := c.OpenSnapshotV3(context.Background(), gen, 0)\n\t\tif !os.IsNotExist(err) {\n\t\t\tt.Errorf(\"expected not exist error, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ReadDecompressed\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tsnapshotsDir := filepath.Join(tmpDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create LZ4-compressed test data\n\t\toriginal := []byte(\"test snapshot data for decompression\")\n\t\tvar buf bytes.Buffer\n\t\tw := lz4.NewWriter(&buf)\n\t\tif _, err := w.Write(original); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := w.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Write compressed file\n\t\tpath := filepath.Join(snapshotsDir, \"00000000.snapshot.lz4\")\n\t\tif err := os.WriteFile(path, buf.Bytes(), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tc := file.NewReplicaClient(tmpDir)\n\t\tr, err := c.OpenSnapshotV3(context.Background(), gen, 0)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer r.Close()\n\n\t\tdata, err := io.ReadAll(r)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !bytes.Equal(data, original) {\n\t\t\tt.Errorf(\"decompressed data mismatch: got %q, want %q\", data, original)\n\t\t}\n\t})\n}\n\nfunc TestReplicaClient_OpenWALSegmentV3(t *testing.T) {\n\tgen := \"0123456789abcdef\"\n\n\tt.Run(\"NotFound\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\tc := file.NewReplicaClient(tmpDir)\n\n\t\t_, err := c.OpenWALSegmentV3(context.Background(), gen, 0, 0)\n\t\tif !os.IsNotExist(err) {\n\t\t\tt.Errorf(\"expected not exist error, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ReadDecompressed\", func(t *testing.T) {\n\t\ttmpDir := t.TempDir()\n\t\twalDir := filepath.Join(tmpDir, \"generations\", gen, \"wal\")\n\t\tif err := os.MkdirAll(walDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create LZ4-compressed test data\n\t\toriginal := []byte(\"test WAL segment data\")\n\t\tvar buf bytes.Buffer\n\t\tw := lz4.NewWriter(&buf)\n\t\tif _, err := w.Write(original); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := w.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Write compressed file\n\t\tpath := filepath.Join(walDir, \"00000001_00001000.wal.lz4\")\n\t\tif err := os.WriteFile(path, buf.Bytes(), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tc := file.NewReplicaClient(tmpDir)\n\t\tr, err := c.OpenWALSegmentV3(context.Background(), gen, 1, 4096)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer r.Close()\n\n\t\tdata, err := io.ReadAll(r)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !bytes.Equal(data, original) {\n\t\t\tt.Errorf(\"decompressed data mismatch: got %q, want %q\", data, original)\n\t\t}\n\t})\n}\n\n/*\nfunc TestReplica_Sync(t *testing.T) {\n\t// Ensure replica can successfully sync after DB has sync'd.\n\tt.Run(\"InitialSync\", func(t *testing.T) {\n\t\tdb, sqldb := MustOpenDBs(t)\n\t\tdefer MustCloseDBs(t, db, sqldb)\n\n\t\tr := litestream.NewReplica(db, \"\", file.NewReplicaClient(t.TempDir()))\n\t\tr.MonitorEnabled = false\n\t\tdb.Replicas = []*litestream.Replica{r}\n\n\t\t// Sync database & then sync replica.\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := r.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Ensure posistions match.\n\t\tif want, err := db.Pos(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, err := r.Pos(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got != want {\n\t\t\tt.Fatalf(\"Pos()=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\t// Ensure replica can successfully sync multiple times.\n\tt.Run(\"MultiSync\", func(t *testing.T) {\n\t\tdb, sqldb := MustOpenDBs(t)\n\t\tdefer MustCloseDBs(t, db, sqldb)\n\n\t\tr := litestream.NewReplica(db, \"\", file.NewReplicaClient(t.TempDir()))\n\t\tr.MonitorEnabled = false\n\t\tdb.Replicas = []*litestream.Replica{r}\n\n\t\tif _, err := sqldb.Exec(`CREATE TABLE foo (bar TEXT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Write to the database multiple times and sync after each write.\n\t\tfor i, n := 0, db.MinCheckpointPageN*2; i < n; i++ {\n\t\t\tif _, err := sqldb.Exec(`INSERT INTO foo (bar) VALUES ('baz')`); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\n\t\t\t// Sync periodically.\n\t\t\tif i%100 == 0 || i == n-1 {\n\t\t\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\t\t\tt.Fatal(err)\n\t\t\t\t} else if err := r.Sync(context.Background()); err != nil {\n\t\t\t\t\tt.Fatal(err)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Ensure posistions match.\n\t\tpos, err := db.Pos()\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pos.Index, 2; got != want {\n\t\t\tt.Fatalf(\"Index=%v, want %v\", got, want)\n\t\t}\n\n\t\tif want, err := r.Pos(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got := pos; got != want {\n\t\t\tt.Fatalf(\"Pos()=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\t// Ensure replica returns an error if there is no generation available from the DB.\n\tt.Run(\"ErrNoGeneration\", func(t *testing.T) {\n\t\tdb, sqldb := MustOpenDBs(t)\n\t\tdefer MustCloseDBs(t, db, sqldb)\n\n\t\tr := litestream.NewReplica(db, \"\", file.NewReplicaClient(t.TempDir()))\n\t\tr.MonitorEnabled = false\n\t\tdb.Replicas = []*litestream.Replica{r}\n\n\t\tif err := r.Sync(context.Background()); err == nil || err.Error() != `no generation, waiting for data` {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n}\n*/\n" }, { "path": "go.mod", "content": "module github.com/benbjohnson/litestream\n\ngo 1.25.0\n\ntoolchain go1.25.8\n\nrequire (\n\tcloud.google.com/go/storage v1.36.0\n\tgithub.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.2\n\tgithub.com/Azure/azure-sdk-for-go/sdk/azidentity v1.11.0\n\tgithub.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.2\n\tgithub.com/MadAppGang/httplog v1.3.0\n\tgithub.com/aws/aws-sdk-go-v2 v1.41.0\n\tgithub.com/aws/aws-sdk-go-v2/config v1.32.6\n\tgithub.com/aws/aws-sdk-go-v2/credentials v1.19.6\n\tgithub.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18\n\tgithub.com/aws/aws-sdk-go-v2/service/s3 v1.95.0\n\tgithub.com/aws/smithy-go v1.24.0\n\tgithub.com/dustin/go-humanize v1.0.1\n\tgithub.com/fsouza/fake-gcs-server v1.47.3\n\tgithub.com/hashicorp/golang-lru/v2 v2.0.7\n\tgithub.com/mark3labs/mcp-go v0.32.0\n\tgithub.com/mattn/go-shellwords v1.0.12\n\tgithub.com/mattn/go-sqlite3 v1.14.19\n\tgithub.com/nats-io/nats.go v1.44.0\n\tgithub.com/pkg/sftp v1.13.6\n\tgithub.com/prometheus/client_golang v1.17.0\n\tgithub.com/psanford/sqlite3vfs v0.0.0-20251127171934-4e34e03a991a // direct\n\tgithub.com/studio-b12/gowebdav v0.11.0\n\tgithub.com/superfly/ltx v0.5.1\n\tgolang.org/x/crypto v0.45.0\n\tgolang.org/x/sys v0.38.0\n\tgoogle.golang.org/api v0.154.0\n\tgopkg.in/yaml.v2 v2.4.0\n\tmodernc.org/sqlite v1.44.3\n)\n\nrequire (\n\tcloud.google.com/go/pubsub v1.33.0 // indirect\n\tgithub.com/fsnotify/fsnotify v1.7.0\n\tgithub.com/google/renameio/v2 v2.0.0 // indirect\n\tgithub.com/gorilla/handlers v1.5.1 // indirect\n\tgithub.com/gorilla/mux v1.8.0 // indirect\n\tgithub.com/ncruces/go-strftime v1.0.0 // indirect\n\tgithub.com/pkg/xattr v0.4.9 // indirect\n\tgithub.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect\n\tgithub.com/sirupsen/logrus v1.9.3 // indirect\n\tgolang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect\n\tgolang.org/x/sync v0.18.0\n\tmodernc.org/libc v1.67.6 // indirect\n\tmodernc.org/mathutil v1.7.1 // indirect\n\tmodernc.org/memory v1.11.0 // indirect\n)\n\nrequire (\n\tgithub.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.3.0\n\tgithub.com/markusmobius/go-dateparser v1.2.4\n)\n\nrequire (\n\tgithub.com/pierrec/lz4/v4 v4.1.22\n\tgithub.com/stretchr/testify v1.11.1\n\tgolang.org/x/crypto/x509roots/fallback v0.0.0-20260209214922-2f26647a795e\n)\n\nrequire (\n\tcloud.google.com/go v0.111.0 // indirect\n\tcloud.google.com/go/compute/metadata v0.3.0 // indirect\n\tcloud.google.com/go/iam v1.1.5 // indirect\n\tgithub.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect\n\tgithub.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 // indirect\n\tgithub.com/TylerBrock/colorjson v0.0.0-20200706003622-8a50f05110d2 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/internal/v4a v1.4.16 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.7 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.16 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/signin v1.0.4 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/sso v1.30.8 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/sts v1.41.5 // indirect\n\tgithub.com/beorn7/perks v1.0.1 // indirect\n\tgithub.com/cespare/xxhash/v2 v2.3.0 // indirect\n\tgithub.com/davecgh/go-spew v1.1.1 // indirect\n\tgithub.com/fatih/color v1.13.0 // indirect\n\tgithub.com/felixge/httpsnoop v1.0.4 // indirect\n\tgithub.com/go-logr/logr v1.3.0 // indirect\n\tgithub.com/go-logr/stdr v1.2.2 // indirect\n\tgithub.com/golang-jwt/jwt/v5 v5.3.0 // indirect\n\tgithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect\n\tgithub.com/golang/protobuf v1.5.3 // indirect\n\tgithub.com/google/s2a-go v0.1.7 // indirect\n\tgithub.com/google/uuid v1.6.0 // indirect\n\tgithub.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect\n\tgithub.com/googleapis/gax-go/v2 v2.12.0 // indirect\n\tgithub.com/hablullah/go-hijri v1.0.2 // indirect\n\tgithub.com/hablullah/go-juliandays v1.0.0 // indirect\n\tgithub.com/jalaali/go-jalaali v0.0.0-20210801064154-80525e88d958 // indirect\n\tgithub.com/klauspost/compress v1.18.0 // indirect\n\tgithub.com/kr/fs v0.1.0 // indirect\n\tgithub.com/kylelemons/godebug v1.1.0 // indirect\n\tgithub.com/magefile/mage v1.14.0 // indirect\n\tgithub.com/mattn/go-colorable v0.1.13 // indirect\n\tgithub.com/mattn/go-isatty v0.0.20 // indirect\n\tgithub.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect\n\tgithub.com/nats-io/nkeys v0.4.11 // indirect\n\tgithub.com/nats-io/nuid v1.0.1 // indirect\n\tgithub.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect\n\tgithub.com/pmezard/go-difflib v1.0.0 // indirect\n\tgithub.com/prometheus/client_model v0.5.0 // indirect\n\tgithub.com/prometheus/common v0.45.0 // indirect\n\tgithub.com/prometheus/procfs v0.12.0 // indirect\n\tgithub.com/spf13/cast v1.7.1 // indirect\n\tgithub.com/tetratelabs/wazero v1.2.1 // indirect\n\tgithub.com/wasilibs/go-re2 v1.3.0 // indirect\n\tgithub.com/yosida95/uritemplate/v3 v3.0.2 // indirect\n\tgo.opencensus.io v0.24.0 // indirect\n\tgo.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.46.1 // indirect\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1 // indirect\n\tgo.opentelemetry.io/otel v1.21.0 // indirect\n\tgo.opentelemetry.io/otel/metric v1.21.0 // indirect\n\tgo.opentelemetry.io/otel/trace v1.21.0 // indirect\n\tgolang.org/x/net v0.47.0 // indirect\n\tgolang.org/x/oauth2 v0.27.0 // indirect\n\tgolang.org/x/text v0.31.0 // indirect\n\tgolang.org/x/time v0.5.0 // indirect\n\tgolang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect\n\tgoogle.golang.org/genproto v0.0.0-20231212172506-995d672761c0 // indirect\n\tgoogle.golang.org/genproto/googleapis/api v0.0.0-20231212172506-995d672761c0 // indirect\n\tgoogle.golang.org/genproto/googleapis/rpc v0.0.0-20231212172506-995d672761c0 // indirect\n\tgoogle.golang.org/grpc v1.60.1 // indirect\n\tgoogle.golang.org/protobuf v1.33.0 // indirect\n\tgopkg.in/yaml.v3 v3.0.1 // indirect\n)\n" }, { "path": "go.sum", "content": "cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.111.0 h1:YHLKNupSD1KqjDbQ3+LVdQ81h/UJbJyZG203cEfnQgM=\ncloud.google.com/go v0.111.0/go.mod h1:0mibmpKP1TyOOFYQY5izo0LnT+ecvOQ0Sg3OdmMiNRU=\ncloud.google.com/go/compute/metadata v0.3.0 h1:Tz+eQXMEqDIKRsmY3cHTL6FVaynIjX2QxYC4trgAKZc=\ncloud.google.com/go/compute/metadata v0.3.0/go.mod h1:zFmK7XCadkQkj6TtorcaGlCW1hT1fIilQDwofLpJ20k=\ncloud.google.com/go/iam v1.1.5 h1:1jTsCu4bcsNsE4iiqNT5SHwrDRCfRmIaaaVFhRveTJI=\ncloud.google.com/go/iam v1.1.5/go.mod h1:rB6P/Ic3mykPbFio+vo7403drjlgvoWfYpJhMXEbzv8=\ncloud.google.com/go/kms v1.15.5 h1:pj1sRfut2eRbD9pFRjNnPNg/CzJPuQAzUujMIM1vVeM=\ncloud.google.com/go/kms v1.15.5/go.mod h1:cU2H5jnp6G2TDpUGZyqTCoy1n16fbubHZjmVXSMtwDI=\ncloud.google.com/go/pubsub v1.33.0 h1:6SPCPvWav64tj0sVX/+npCBKhUi/UjJehy9op/V3p2g=\ncloud.google.com/go/pubsub v1.33.0/go.mod h1:f+w71I33OMyxf9VpMVcZbnG5KSUkCOUHYpFd5U1GdRc=\ncloud.google.com/go/storage v1.36.0 h1:P0mOkAcaJxhCTvAkMhxMfrTKiNcub4YmmPBtlhAyTr8=\ncloud.google.com/go/storage v1.36.0/go.mod h1:M6M/3V/D3KpzMTJyPOR/HU6n2Si5QdaXYEsng2xgOs8=\ngithub.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.2 h1:Hr5FTipp7SL07o2FvoVOX9HRiRH3CR3Mj8pxqCcdD5A=\ngithub.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.2/go.mod h1:QyVsSSN64v5TGltphKLQ2sQxe4OBQg0J1eKRcVBnfgE=\ngithub.com/Azure/azure-sdk-for-go/sdk/azidentity v1.11.0 h1:MhRfI58HblXzCtWEZCO0feHs8LweePB3s90r7WaR1KU=\ngithub.com/Azure/azure-sdk-for-go/sdk/azidentity v1.11.0/go.mod h1:okZ+ZURbArNdlJ+ptXoyHNuOETzOl1Oww19rm8I2WLA=\ngithub.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2 h1:yz1bePFlP5Vws5+8ez6T3HWXPmwOK7Yvq8QxDBD3SKY=\ngithub.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2/go.mod h1:Pa9ZNPuoNu/GztvBSKk9J1cDJW6vk/n0zLtV4mgd8N8=\ngithub.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 h1:9iefClla7iYpfYWdzPCRDozdmndjTm8DXdpCzPajMgA=\ngithub.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2/go.mod h1:XtLgD3ZD34DAaVIIAyG3objl5DynM3CQ/vMcbBNJZGI=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.8.1 h1:/Zt+cDPnpC3OVDm/JKLOs7M2DKmLRIIp3XIx9pHHiig=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.8.1/go.mod h1:Ng3urmn6dYe8gnbCMoHHVl5APYz2txho3koEkV2o2HA=\ngithub.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.2 h1:FwladfywkNirM+FZYLBR2kBz5C8Tg0fw5w5Y7meRXWI=\ngithub.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.2/go.mod h1:vv5Ad0RrIoT1lJFdWBZwt4mB1+j+V8DUroixmKDTCdk=\ngithub.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 h1:WJTmL004Abzc5wDB5VtZG2PJk5ndYDgVacGqfirKxjM=\ngithub.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1/go.mod h1:tCcJZ0uHAmvjsVYzEFivsRTN00oz5BEsRgQHu5JZ9WE=\ngithub.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 h1:oygO0locgZJe7PpYPXT5A29ZkwJaPqcva7BVeemZOZs=\ngithub.com/AzureAD/microsoft-authentication-library-for-go v1.4.2/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI=\ngithub.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=\ngithub.com/MadAppGang/httplog v1.3.0 h1:1XU54TO8kiqTeO+7oZLKAM3RP/cJ7SadzslRcKspVHo=\ngithub.com/MadAppGang/httplog v1.3.0/go.mod h1:gpYEdkjh/Cda6YxtDy4AB7KY+fR7mb3SqBZw74A5hJ4=\ngithub.com/TylerBrock/colorjson v0.0.0-20200706003622-8a50f05110d2 h1:ZBbLwSJqkHBuFDA6DUhhse0IGJ7T5bemHyNILUjvOq4=\ngithub.com/TylerBrock/colorjson v0.0.0-20200706003622-8a50f05110d2/go.mod h1:VSw57q4QFiWDbRnjdX8Cb3Ow0SFncRw+bA/ofY6Q83w=\ngithub.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.3.0 h1:wQlqotpyjYPjJz+Noh5bRu7Snmydk8SKC5Z6u1CR20Y=\ngithub.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.3.0/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M=\ngithub.com/aws/aws-sdk-go-v2 v1.41.0 h1:tNvqh1s+v0vFYdA1xq0aOJH+Y5cRyZ5upu6roPgPKd4=\ngithub.com/aws/aws-sdk-go-v2 v1.41.0/go.mod h1:MayyLB8y+buD9hZqkCW3kX1AKq07Y5pXxtgB+rRFhz0=\ngithub.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4 h1:489krEF9xIGkOaaX3CE/Be2uWjiXrkCH6gUX+bZA/BU=\ngithub.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4/go.mod h1:IOAPF6oT9KCsceNTvvYMNHy0+kMF8akOjeDvPENWxp4=\ngithub.com/aws/aws-sdk-go-v2/config v1.32.6 h1:hFLBGUKjmLAekvi1evLi5hVvFQtSo3GYwi+Bx4lpJf8=\ngithub.com/aws/aws-sdk-go-v2/config v1.32.6/go.mod h1:lcUL/gcd8WyjCrMnxez5OXkO3/rwcNmvfno62tnXNcI=\ngithub.com/aws/aws-sdk-go-v2/credentials v1.19.6 h1:F9vWao2TwjV2MyiyVS+duza0NIRtAslgLUM0vTA1ZaE=\ngithub.com/aws/aws-sdk-go-v2/credentials v1.19.6/go.mod h1:SgHzKjEVsdQr6Opor0ihgWtkWdfRAIwxYzSJ8O85VHY=\ngithub.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16 h1:80+uETIWS1BqjnN9uJ0dBUaETh+P1XwFy5vwHwK5r9k=\ngithub.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16/go.mod h1:wOOsYuxYuB/7FlnVtzeBYRcjSRtQpAW0hCP7tIULMwo=\ngithub.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18 h1:9vWXHtaepwoAl/UuKzxwgOoJDXPCC3hvgNMfcmdS2Tk=\ngithub.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18/go.mod h1:sKuUZ+MwUTuJbYvZ8pK0x10LvgcJK3Y4rmh63YBekwk=\ngithub.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16 h1:rgGwPzb82iBYSvHMHXc8h9mRoOUBZIGFgKb9qniaZZc=\ngithub.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16/go.mod h1:L/UxsGeKpGoIj6DxfhOWHWQ/kGKcd4I1VncE4++IyKA=\ngithub.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 h1:1jtGzuV7c82xnqOVfx2F0xmJcOw5374L7N6juGW6x6U=\ngithub.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16/go.mod h1:M2E5OQf+XLe+SZGmmpaI2yy+J326aFf6/+54PoxSANc=\ngithub.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEGm0OUEZqm4K/Gcfk=\ngithub.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc=\ngithub.com/aws/aws-sdk-go-v2/internal/v4a v1.4.16 h1:CjMzUs78RDDv4ROu3JnJn/Ig1r6ZD7/T2DXLLRpejic=\ngithub.com/aws/aws-sdk-go-v2/internal/v4a v1.4.16/go.mod h1:uVW4OLBqbJXSHJYA9svT9BluSvvwbzLQ2Crf6UPzR3c=\ngithub.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 h1:0ryTNEdJbzUCEWkVXEXoqlXV72J5keC1GvILMOuD00E=\ngithub.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4/go.mod h1:HQ4qwNZh32C3CBeO6iJLQlgtMzqeG17ziAA/3KDJFow=\ngithub.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.7 h1:DIBqIrJ7hv+e4CmIk2z3pyKT+3B6qVMgRsawHiR3qso=\ngithub.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.7/go.mod h1:vLm00xmBke75UmpNvOcZQ/Q30ZFjbczeLFqGx5urmGo=\ngithub.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 h1:oHjJHeUy0ImIV0bsrX0X91GkV5nJAyv1l1CC9lnO0TI=\ngithub.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16/go.mod h1:iRSNGgOYmiYwSCXxXaKb9HfOEj40+oTKn8pTxMlYkRM=\ngithub.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.16 h1:NSbvS17MlI2lurYgXnCOLvCFX38sBW4eiVER7+kkgsU=\ngithub.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.16/go.mod h1:SwT8Tmqd4sA6G1qaGdzWCJN99bUmPGHfRwwq3G5Qb+A=\ngithub.com/aws/aws-sdk-go-v2/service/s3 v1.95.0 h1:MIWra+MSq53CFaXXAywB2qg9YvVZifkk6vEGl/1Qor0=\ngithub.com/aws/aws-sdk-go-v2/service/s3 v1.95.0/go.mod h1:79S2BdqCJpScXZA2y+cpZuocWsjGjJINyXnOsf5DTz8=\ngithub.com/aws/aws-sdk-go-v2/service/signin v1.0.4 h1:HpI7aMmJ+mm1wkSHIA2t5EaFFv5EFYXePW30p1EIrbQ=\ngithub.com/aws/aws-sdk-go-v2/service/signin v1.0.4/go.mod h1:C5RdGMYGlfM0gYq/tifqgn4EbyX99V15P2V3R+VHbQU=\ngithub.com/aws/aws-sdk-go-v2/service/sso v1.30.8 h1:aM/Q24rIlS3bRAhTyFurowU8A0SMyGDtEOY/l/s/1Uw=\ngithub.com/aws/aws-sdk-go-v2/service/sso v1.30.8/go.mod h1:+fWt2UHSb4kS7Pu8y+BMBvJF0EWx+4H0hzNwtDNRTrg=\ngithub.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12 h1:AHDr0DaHIAo8c9t1emrzAlVDFp+iMMKnPdYy6XO4MCE=\ngithub.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12/go.mod h1:GQ73XawFFiWxyWXMHWfhiomvP3tXtdNar/fi8z18sx0=\ngithub.com/aws/aws-sdk-go-v2/service/sts v1.41.5 h1:SciGFVNZ4mHdm7gpD1dgZYnCuVdX1s+lFTg4+4DOy70=\ngithub.com/aws/aws-sdk-go-v2/service/sts v1.41.5/go.mod h1:iW40X4QBmUxdP+fZNOpfmkdMZqsovezbAeO+Ubiv2pk=\ngithub.com/aws/smithy-go v1.24.0 h1:LpilSUItNPFr1eY85RYgTIg5eIEPtvFbskaFcmmIUnk=\ngithub.com/aws/smithy-go v1.24.0/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0=\ngithub.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=\ngithub.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=\ngithub.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=\ngithub.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=\ngithub.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=\ngithub.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=\ngithub.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4 h1:/inchEIKaYC1Akx+H+gqO04wryn5h75LSazbRlnya1k=\ngithub.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=\ngithub.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=\ngithub.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=\ngithub.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=\ngithub.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=\ngithub.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=\ngithub.com/envoyproxy/protoc-gen-validate v1.0.2 h1:QkIBuU5k+x7/QXPvPPnWXWlCdaBFApVqftFV6k087DA=\ngithub.com/envoyproxy/protoc-gen-validate v1.0.2/go.mod h1:GpiZQP3dDbg4JouG/NNS7QWXpgx6x8QiMKdmN72jogE=\ngithub.com/fatih/color v1.13.0 h1:8LOYc1KYPPmyKMuN8QV2DNRWNbLo6LZ0iLs8+mlH53w=\ngithub.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=\ngithub.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=\ngithub.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=\ngithub.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=\ngithub.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=\ngithub.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=\ngithub.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=\ngithub.com/fsnotify/fsnotify v1.7.0/go.mod h1:40Bi/Hjc2AVfZrqy+aj+yEI+/bRxZnMJyTJwOpGvigM=\ngithub.com/fsouza/fake-gcs-server v1.47.3 h1:L1mZfEd7cU8hnoHU2Rch9bq9HPS0CR0oySZdGMHygU4=\ngithub.com/fsouza/fake-gcs-server v1.47.3/go.mod h1:vqUZbI12uy9IkRQ54Q4p5AniQsSiUq8alO9Nv2egMmA=\ngithub.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=\ngithub.com/go-logr/logr v1.3.0 h1:2y3SDp0ZXuc6/cjLSZ+Q3ir+QB9T/iG5yYRXqsagWSY=\ngithub.com/go-logr/logr v1.3.0/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=\ngithub.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=\ngithub.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=\ngithub.com/golang-jwt/jwt/v5 v5.3.0 h1:pv4AsKCKKZuqlgs5sUmn4x8UlGa0kEVt/puTpKx9vvo=\ngithub.com/golang-jwt/jwt/v5 v5.3.0/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=\ngithub.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=\ngithub.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=\ngithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=\ngithub.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=\ngithub.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=\ngithub.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=\ngithub.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=\ngithub.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=\ngithub.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=\ngithub.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=\ngithub.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=\ngithub.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=\ngithub.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=\ngithub.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=\ngithub.com/google/martian/v3 v3.3.2 h1:IqNFLAmvJOgVlpdEBiQbDc2EwKW77amAycfTuWKdfvw=\ngithub.com/google/martian/v3 v3.3.2/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk=\ngithub.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=\ngithub.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=\ngithub.com/google/renameio/v2 v2.0.0 h1:UifI23ZTGY8Tt29JbYFiuyIU3eX+RNFtUwefq9qAhxg=\ngithub.com/google/renameio/v2 v2.0.0/go.mod h1:BtmJXm5YlszgC+TD4HOEEUFgkJP3nLxehU6hfe7jRt4=\ngithub.com/google/s2a-go v0.1.7 h1:60BLSyTrOV4/haCDW4zb1guZItoSq8foHCXrAnjBo/o=\ngithub.com/google/s2a-go v0.1.7/go.mod h1:50CgR4k1jNlWBu4UfS4AcfhVe1r6pdZPygJ3R8F0Qdw=\ngithub.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=\ngithub.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/googleapis/enterprise-certificate-proxy v0.3.2 h1:Vie5ybvEvT75RniqhfFxPRy3Bf7vr3h0cechB90XaQs=\ngithub.com/googleapis/enterprise-certificate-proxy v0.3.2/go.mod h1:VLSiSSBs/ksPL8kq3OBOQ6WRI2QnaFynd1DCjZ62+V0=\ngithub.com/googleapis/gax-go/v2 v2.12.0 h1:A+gCJKdRfqXkr+BIRGtZLibNXf0m1f9E4HG56etFpas=\ngithub.com/googleapis/gax-go/v2 v2.12.0/go.mod h1:y+aIqrI5eb1YGMVJfuV3185Ts/D7qKpsEkdD5+I6QGU=\ngithub.com/gorilla/handlers v1.5.1 h1:9lRY6j8DEeeBT10CvO9hGW0gmky0BprnvDI5vfhUHH4=\ngithub.com/gorilla/handlers v1.5.1/go.mod h1:t8XrUpc4KVXb7HGyJ4/cEnwQiaxrX/hz1Zv/4g96P1Q=\ngithub.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=\ngithub.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=\ngithub.com/hablullah/go-hijri v1.0.2 h1:drT/MZpSZJQXo7jftf5fthArShcaMtsal0Zf/dnmp6k=\ngithub.com/hablullah/go-hijri v1.0.2/go.mod h1:OS5qyYLDjORXzK4O1adFw9Q5WfhOcMdAKglDkcTxgWQ=\ngithub.com/hablullah/go-juliandays v1.0.0 h1:A8YM7wIj16SzlKT0SRJc9CD29iiaUzpBLzh5hr0/5p0=\ngithub.com/hablullah/go-juliandays v1.0.0/go.mod h1:0JOYq4oFOuDja+oospuc61YoX+uNEn7Z6uHYTbBzdGc=\ngithub.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=\ngithub.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=\ngithub.com/hokaccha/go-prettyjson v0.0.0-20211117102719-0474bc63780f h1:7LYC+Yfkj3CTRcShK0KOL/w6iTiKyqqBA9a41Wnggw8=\ngithub.com/hokaccha/go-prettyjson v0.0.0-20211117102719-0474bc63780f/go.mod h1:pFlLw2CfqZiIBOx6BuCeRLCrfxBJipTY0nIOF/VbGcI=\ngithub.com/jalaali/go-jalaali v0.0.0-20210801064154-80525e88d958 h1:qxLoi6CAcXVzjfvu+KXIXJOAsQB62LXjsfbOaErsVzE=\ngithub.com/jalaali/go-jalaali v0.0.0-20210801064154-80525e88d958/go.mod h1:Wqfu7mjUHj9WDzSSPI5KfBclTTEnLveRUFr/ujWnTgE=\ngithub.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=\ngithub.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=\ngithub.com/keybase/go-keychain v0.0.1 h1:way+bWYa6lDppZoZcgMbYsvC7GxljxrskdNInRtuthU=\ngithub.com/keybase/go-keychain v0.0.1/go.mod h1:PdEILRW3i9D8JcdM+FmY6RwkHGnhHxXwkPPMeUgOK1k=\ngithub.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=\ngithub.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=\ngithub.com/klauspost/cpuid/v2 v2.2.5 h1:0E5MSMDEoAulmXNFquVs//DdoomxaoTY1kUhbc/qbZg=\ngithub.com/klauspost/cpuid/v2 v2.2.5/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=\ngithub.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=\ngithub.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=\ngithub.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=\ngithub.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=\ngithub.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=\ngithub.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=\ngithub.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=\ngithub.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=\ngithub.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo=\ngithub.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=\ngithub.com/mark3labs/mcp-go v0.32.0 h1:fgwmbfL2gbd67obg57OfV2Dnrhs1HtSdlY/i5fn7MU8=\ngithub.com/mark3labs/mcp-go v0.32.0/go.mod h1:rXqOudj/djTORU/ThxYx8fqEVj/5pvTuuebQ2RC7uk4=\ngithub.com/markusmobius/go-dateparser v1.2.4 h1:2e8XJozaERVxGwsRg72coi51L2aiYqE2gukkdLc85ck=\ngithub.com/markusmobius/go-dateparser v1.2.4/go.mod h1:CBAUADJuMNhJpyM6IYaWAoFhtKaqnUcznY2cL7gNugY=\ngithub.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=\ngithub.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=\ngithub.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=\ngithub.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=\ngithub.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=\ngithub.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=\ngithub.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=\ngithub.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=\ngithub.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=\ngithub.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=\ngithub.com/mattn/go-sqlite3 v1.14.8/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=\ngithub.com/mattn/go-sqlite3 v1.14.19 h1:fhGleo2h1p8tVChob4I9HpmVFIAkKGpiukdrgQbWfGI=\ngithub.com/mattn/go-sqlite3 v1.14.19/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg=\ngithub.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 h1:jWpvCLoY8Z/e3VKvlsiIGKtc+UG6U5vzxaoagmhXfyg=\ngithub.com/matttproud/golang_protobuf_extensions/v2 v2.0.0/go.mod h1:QUyp042oQthUoa9bqDv0ER0wrtXnBruoNd7aNjkbP+k=\ngithub.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=\ngithub.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=\ngithub.com/minio/minio-go/v7 v7.0.61 h1:87c+x8J3jxQ5VUGimV9oHdpjsAvy3fhneEBKuoKEVUI=\ngithub.com/minio/minio-go/v7 v7.0.61/go.mod h1:BTu8FcrEw+HidY0zd/0eny43QnVNkXRPXrLXFuQBHXg=\ngithub.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM=\ngithub.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8=\ngithub.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=\ngithub.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=\ngithub.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=\ngithub.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=\ngithub.com/nats-io/nats.go v1.44.0 h1:ECKVrDLdh/kDPV1g0gAQ+2+m2KprqZK5O/eJAyAnH2M=\ngithub.com/nats-io/nats.go v1.44.0/go.mod h1:iRWIPokVIFbVijxuMQq4y9ttaBTMe0SFdlZfMDd+33g=\ngithub.com/nats-io/nkeys v0.4.11 h1:q44qGV008kYd9W1b1nEBkNzvnWxtRSQ7A8BoqRrcfa0=\ngithub.com/nats-io/nkeys v0.4.11/go.mod h1:szDimtgmfOi9n25JpfIdGw12tZFYXqhGxjhVxsatHVE=\ngithub.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=\ngithub.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=\ngithub.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=\ngithub.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=\ngithub.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU=\ngithub.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=\ngithub.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ=\ngithub.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=\ngithub.com/pkg/sftp v1.13.6 h1:JFZT4XbOU7l77xGSpOdW+pwIMqP044IyjXX6FGyEKFo=\ngithub.com/pkg/sftp v1.13.6/go.mod h1:tz1ryNURKu77RL+GuCzmoJYxQczL3wLNNpPWagdg4Qk=\ngithub.com/pkg/xattr v0.4.9 h1:5883YPCtkSd8LFbs13nXplj9g9tlrwoJRjgpgMu1/fE=\ngithub.com/pkg/xattr v0.4.9/go.mod h1:di8WF84zAKk8jzR1UBTEWh9AUlIZZ7M/JNt8e9B6ktU=\ngithub.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=\ngithub.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/prometheus/client_golang v1.17.0 h1:rl2sfwZMtSthVU752MqfjQozy7blglC+1SOtjMAMh+Q=\ngithub.com/prometheus/client_golang v1.17.0/go.mod h1:VeL+gMmOAxkS2IqfCq0ZmHSL+LjWfWDUmp1mBz9JgUY=\ngithub.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw=\ngithub.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI=\ngithub.com/prometheus/common v0.45.0 h1:2BGz0eBc2hdMDLnO/8n0jeB3oPrt2D08CekT0lneoxM=\ngithub.com/prometheus/common v0.45.0/go.mod h1:YJmSTw9BoKxJplESWWxlbyttQR4uaEcGyv9MZjVOJsY=\ngithub.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo=\ngithub.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo=\ngithub.com/psanford/sqlite3vfs v0.0.0-20251127171934-4e34e03a991a h1:r4YWl0uVObCbBFvj1VsIlyHzgZwZOHvY1KdRaQjzzUc=\ngithub.com/psanford/sqlite3vfs v0.0.0-20251127171934-4e34e03a991a/go.mod h1:iW4cSew5PAb1sMZiTEkVJAIBNrepaB6jTYjeP47WtI0=\ngithub.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=\ngithub.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=\ngithub.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=\ngithub.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=\ngithub.com/rs/xid v1.5.0 h1:mKX4bl4iPYJtEIxp6CYiUuLQ/8DYMoz0PUdtGgMFRVc=\ngithub.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=\ngithub.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=\ngithub.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=\ngithub.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=\ngithub.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=\ngithub.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=\ngithub.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=\ngithub.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=\ngithub.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=\ngithub.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=\ngithub.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=\ngithub.com/studio-b12/gowebdav v0.11.0 h1:qbQzq4USxY28ZYsGJUfO5jR+xkFtcnwWgitp4Zp1irU=\ngithub.com/studio-b12/gowebdav v0.11.0/go.mod h1:bHA7t77X/QFExdeAnDzK6vKM34kEZAcE1OX4MfiwjkE=\ngithub.com/superfly/ltx v0.5.1 h1:vGUeBhKvBKZ2s2TsTrOSahz+m0PswfqOWDAD+ICiiYY=\ngithub.com/superfly/ltx v0.5.1/go.mod h1:Nf50QAIXU/ET4ua3AuQ2fh31MbgNQZA7r/DYx6Os77s=\ngithub.com/tetratelabs/wazero v1.2.1 h1:J4X2hrGzJvt+wqltuvcSjHQ7ujQxA9gb6PeMs4qlUWs=\ngithub.com/tetratelabs/wazero v1.2.1/go.mod h1:wYx2gNRg8/WihJfSDxA1TIL8H+GkfLYm+bIfbblu9VQ=\ngithub.com/wasilibs/go-re2 v1.3.0 h1:LFhBNzoStM3wMie6rN2slD1cuYH2CGiHpvNL3UtcsMw=\ngithub.com/wasilibs/go-re2 v1.3.0/go.mod h1:AafrCXVvGRJJOImMajgJ2M7rVmWyisVK7sFshbxnVrg=\ngithub.com/wasilibs/nottinygc v0.4.0 h1:h1TJMihMC4neN6Zq+WKpLxgd9xCFMw7O9ETLwY2exJQ=\ngithub.com/wasilibs/nottinygc v0.4.0/go.mod h1:oDcIotskuYNMpqMF23l7Z8uzD4TC0WXHK8jetlB3HIo=\ngithub.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=\ngithub.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=\ngithub.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=\ngo.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=\ngo.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=\ngo.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.46.1 h1:SpGay3w+nEwMpfVnbqOLH5gY52/foP8RE8UzTZ1pdSE=\ngo.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.46.1/go.mod h1:4UoMYEZOC0yN/sPGH76KPkkU7zgiEWYWL9vwmbnTJPE=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1 h1:aFJWCqJMNjENlcleuuOkGAPH82y0yULBScfXcIEdS24=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1/go.mod h1:sEGXWArGqc3tVa+ekntsN65DmVbVeW+7lTKTjZF3/Fo=\ngo.opentelemetry.io/otel v1.21.0 h1:hzLeKBZEL7Okw2mGzZ0cc4k/A7Fta0uoPgaJCr8fsFc=\ngo.opentelemetry.io/otel v1.21.0/go.mod h1:QZzNPQPm1zLX4gZK4cMi+71eaorMSGT3A4znnUvNNEo=\ngo.opentelemetry.io/otel/metric v1.21.0 h1:tlYWfeo+Bocx5kLEloTjbcDwBuELRrIFxwdQ36PlJu4=\ngo.opentelemetry.io/otel/metric v1.21.0/go.mod h1:o1p3CA8nNHW8j5yuQLdc1eeqEaPfzug24uvsyIEJRWM=\ngo.opentelemetry.io/otel/sdk v1.19.0 h1:6USY6zH+L8uMH8L3t1enZPR3WFEmSTADlqldyHtJi3o=\ngo.opentelemetry.io/otel/sdk v1.19.0/go.mod h1:NedEbbS4w3C6zElbLdPJKOpJQOrGUJ+GfzpjUvI0v1A=\ngo.opentelemetry.io/otel/trace v1.21.0 h1:WD9i5gzvoUPuXIXH24ZNBudiarZDKuekPqi/E8fpfLc=\ngo.opentelemetry.io/otel/trace v1.21.0/go.mod h1:LGbsEB0f9LGjN+OZaQQ26sohbOmiMR+BaslueVtS/qQ=\ngolang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=\ngolang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=\ngolang.org/x/crypto v0.1.0/go.mod h1:RecgLatLF4+eUMCP1PoPZQb+cVrJcOPbHkTkbkB9sbw=\ngolang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=\ngolang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=\ngolang.org/x/crypto/x509roots/fallback v0.0.0-20260209214922-2f26647a795e h1:G2pNJMnWEb60ip90TMo++m9fpt+d3YIZa0er3buPKRo=\ngolang.org/x/crypto/x509roots/fallback v0.0.0-20260209214922-2f26647a795e/go.mod h1:MEIPiCnxvQEjA4astfaKItNwEVZA5Ki+3+nyGbJ5N18=\ngolang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 h1:mgKeJMpvi0yx/sU5GsxQ7p6s2wtOnGAHZWCHUM4KGzY=\ngolang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546/go.mod h1:j/pmGrbnkbPtQfxEe5D0VQhZC6qKbfKifgD0oM7sR70=\ngolang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=\ngolang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=\ngolang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA=\ngolang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w=\ngolang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=\ngolang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco=\ngolang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=\ngolang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=\ngolang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=\ngolang.org/x/oauth2 v0.27.0 h1:da9Vo7/tDv5RH/7nZDz1eMGS/q1Vv1N/7FCrBhI9I3M=\ngolang.org/x/oauth2 v0.27.0/go.mod h1:onh5ek6nERTohokkhCD/y2cV4Do3fxFHFuAejCkRWT8=\ngolang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=\ngolang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=\ngolang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220408201424-a24fb2fb8a0f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=\ngolang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=\ngolang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=\ngolang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=\ngolang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=\ngolang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU=\ngolang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254=\ngolang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=\ngolang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=\ngolang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=\ngolang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=\ngolang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=\ngolang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=\ngolang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=\ngolang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=\ngolang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ=\ngolang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs=\ngolang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 h1:+cNy6SZtPcJQH3LJVLOSmiC7MMxXNOb3PU/VUEz+EhU=\ngolang.org/x/xerrors v0.0.0-20231012003039-104605ab7028/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=\ngoogle.golang.org/api v0.154.0 h1:X7QkVKZBskztmpPKWQXgjJRPA2dJYrL6r+sYPRLj050=\ngoogle.golang.org/api v0.154.0/go.mod h1:qhSMkM85hgqiokIYsrRyKxrjfBeIhgl4Z2JmeRkYylc=\ngoogle.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=\ngoogle.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=\ngoogle.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=\ngoogle.golang.org/genproto v0.0.0-20231212172506-995d672761c0 h1:YJ5pD9rF8o9Qtta0Cmy9rdBwkSjrTCT6XTiUQVOtIos=\ngoogle.golang.org/genproto v0.0.0-20231212172506-995d672761c0/go.mod h1:l/k7rMz0vFTBPy+tFSGvXEd3z+BcoG1k7EHbqm+YBsY=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20231212172506-995d672761c0 h1:s1w3X6gQxwrLEpxnLd/qXTVLgQE2yXwaOaoa6IlY/+o=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20231212172506-995d672761c0/go.mod h1:CAny0tYF+0/9rmDB9fahA9YLzX3+AEVl1qXbv5hhj6c=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20231212172506-995d672761c0 h1:/jFB8jK5R3Sq3i/lmeZO0cATSzFfZaJq1J2Euan3XKU=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20231212172506-995d672761c0/go.mod h1:FUoWkonphQm3RhTS+kOEhF8h0iDpm4tdXolVCeZ9KKA=\ngoogle.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=\ngoogle.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=\ngoogle.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=\ngoogle.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=\ngoogle.golang.org/grpc v1.60.1 h1:26+wFr+cNqSGFcOXcabYC0lUVJVRa2Sb2ortSK7VrEU=\ngoogle.golang.org/grpc v1.60.1/go.mod h1:OlCHIeLYqSSsLi6i49B5QGdzaMZK9+M7LXN2FKz4eGM=\ngoogle.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=\ngoogle.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=\ngoogle.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=\ngoogle.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=\ngoogle.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=\ngoogle.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=\ngoogle.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=\ngoogle.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=\ngoogle.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=\ngoogle.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=\ngopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=\ngopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=\ngopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=\ngopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=\ngopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=\ngopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=\ngopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\nhonnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nmodernc.org/cc/v4 v4.27.1 h1:9W30zRlYrefrDV2JE2O8VDtJ1yPGownxciz5rrbQZis=\nmodernc.org/cc/v4 v4.27.1/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=\nmodernc.org/ccgo/v4 v4.30.1 h1:4r4U1J6Fhj98NKfSjnPUN7Ze2c6MnAdL0hWw6+LrJpc=\nmodernc.org/ccgo/v4 v4.30.1/go.mod h1:bIOeI1JL54Utlxn+LwrFyjCx2n2RDiYEaJVSrgdrRfM=\nmodernc.org/fileutil v1.3.40 h1:ZGMswMNc9JOCrcrakF1HrvmergNLAmxOPjizirpfqBA=\nmodernc.org/fileutil v1.3.40/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=\nmodernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=\nmodernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=\nmodernc.org/gc/v3 v3.1.1 h1:k8T3gkXWY9sEiytKhcgyiZ2L0DTyCQ/nvX+LoCljoRE=\nmodernc.org/gc/v3 v3.1.1/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=\nmodernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=\nmodernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=\nmodernc.org/libc v1.67.6 h1:eVOQvpModVLKOdT+LvBPjdQqfrZq+pC39BygcT+E7OI=\nmodernc.org/libc v1.67.6/go.mod h1:JAhxUVlolfYDErnwiqaLvUqc8nfb2r6S6slAgZOnaiE=\nmodernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=\nmodernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=\nmodernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=\nmodernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=\nmodernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=\nmodernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=\nmodernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=\nmodernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=\nmodernc.org/sqlite v1.44.3 h1:+39JvV/HWMcYslAwRxHb8067w+2zowvFOUrOWIy9PjY=\nmodernc.org/sqlite v1.44.3/go.mod h1:CzbrU2lSB1DKUusvwGz7rqEKIq+NUd8GWuBBZDs9/nA=\nmodernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=\nmodernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=\nmodernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=\nmodernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=\n" }, { "path": "grafana/README.md", "content": "# Litestream Grafana Dashboard\n\nThis directory contains a Grafana dashboard for monitoring Litestream metrics.\n\n## Prerequisites\n\n1. Litestream configured with metrics endpoint enabled in `litestream.yml`:\n\n ```yaml\n addr: \":9090\"\n ```\n\n2. Prometheus configured to scrape Litestream metrics:\n\n ```yaml\n scrape_configs:\n - job_name: 'litestream'\n static_configs:\n - targets: ['localhost:9090']\n ```\n\n3. Grafana with Prometheus data source configured\n\n## Installation\n\n1. Open Grafana and navigate to **Dashboards** → **Import**\n2. Upload the `litestream-dashboard.json` file or paste its contents\n3. Select your Prometheus data source\n4. Click **Import**\n\n## Metrics Included\n\nThe dashboard monitors the following key metrics:\n\n- **Database & WAL Size**: Current size of the database and Write-Ahead Log\n- **Total WAL Bytes Written**: Cumulative bytes written to shadow WAL\n- **Sync Operations**: Rate of sync operations and any sync errors\n- **Sync Duration**: Time spent syncing shadow WAL\n- **Checkpoint Operations**: Rate of checkpoint operations by mode\n- **Checkpoint Errors**: Any checkpoint errors that occur\n- **Transaction ID**: Current transaction ID for each database\n- **Replica Operations**: Operations performed by replica type (GET/PUT)\n- **Replica Throughput**: Bytes transferred by replica operations\n\n## Configuration\n\nThe dashboard uses template variables:\n\n- `datasource`: Select your Prometheus data source\n- `job`: Select the Prometheus job name (defaults to \"litestream\")\n\n## Support\n\nFor issues or improvements to this dashboard, please open an issue at:\n\n" }, { "path": "grafana/litestream-dashboard.json", "content": "{\n \"annotations\": {\n \"list\": [\n {\n \"builtIn\": 1,\n \"datasource\": \"-- Grafana --\",\n \"enable\": true,\n \"hide\": true,\n \"iconColor\": \"rgba(0, 211, 255, 1)\",\n \"name\": \"Annotations & Alerts\",\n \"type\": \"dashboard\"\n }\n ]\n },\n \"editable\": true,\n \"gnetId\": null,\n \"graphTooltip\": 0,\n \"id\": null,\n \"links\": [],\n \"panels\": [\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"tooltip\": false,\n \"viz\": false,\n \"legend\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": true,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"bytes\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 0\n },\n \"id\": 2,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\"\n }\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"litestream_db_size{job=\\\"$job\\\"}\",\n \"legendFormat\": \"{{db}} - DB Size\",\n \"refId\": \"A\"\n },\n {\n \"expr\": \"litestream_wal_size{job=\\\"$job\\\"}\",\n \"legendFormat\": \"{{db}} - WAL Size\",\n \"refId\": \"B\"\n }\n ],\n \"title\": \"Database & WAL Size\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"bytes\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 0\n },\n \"id\": 10,\n \"options\": {\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"values\": false,\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\"\n },\n \"showThresholdLabels\": false,\n \"showThresholdMarkers\": true,\n \"text\": {}\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"litestream_total_wal_bytes{job=\\\"$job\\\"}\",\n \"legendFormat\": \"{{db}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Total WAL Bytes Written\",\n \"type\": \"gauge\"\n },\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"tooltip\": false,\n \"viz\": false,\n \"legend\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": true,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"ops\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 8\n },\n \"id\": 3,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\"\n }\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"rate(litestream_sync_count{job=\\\"$job\\\"}[5m])\",\n \"legendFormat\": \"{{db}} - Sync Rate\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Sync Operations Rate\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"tooltip\": false,\n \"viz\": false,\n \"legend\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": true,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"errors/sec\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 8\n },\n \"id\": 4,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\"\n }\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"rate(litestream_sync_error_count{job=\\\"$job\\\"}[5m])\",\n \"legendFormat\": \"{{db}} - Sync Errors\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Sync Error Rate\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"tooltip\": false,\n \"viz\": false,\n \"legend\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": true,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"s\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 16\n },\n \"id\": 5,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\"\n }\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"rate(litestream_sync_seconds{job=\\\"$job\\\"}[5m])\",\n \"legendFormat\": \"{{db}} - Sync Time\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Sync Duration\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"tooltip\": false,\n \"viz\": false,\n \"legend\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": true,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"ops\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 16\n },\n \"id\": 6,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\"\n }\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"rate(litestream_checkpoint_count{job=\\\"$job\\\"}[5m])\",\n \"legendFormat\": \"{{db}} - {{mode}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Checkpoint Operations Rate\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"tooltip\": false,\n \"viz\": false,\n \"legend\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": true,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"errors/sec\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 24\n },\n \"id\": 7,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\"\n }\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"rate(litestream_checkpoint_error_count{job=\\\"$job\\\"}[5m])\",\n \"legendFormat\": \"{{db}} - {{mode}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Checkpoint Error Rate\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"tooltip\": false,\n \"viz\": false,\n \"legend\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": true,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"none\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 24\n },\n \"id\": 11,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\"\n }\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"litestream_txid{job=\\\"$job\\\"}\",\n \"legendFormat\": \"{{db}} - Transaction ID\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Current Transaction ID\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"tooltip\": false,\n \"viz\": false,\n \"legend\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": true,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"ops\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 32\n },\n \"id\": 8,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\"\n }\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"rate(litestream_replica_operation_total{job=\\\"$job\\\"}[5m])\",\n \"legendFormat\": \"{{replica_type}} - {{operation}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Replica Operations Rate\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": \"${datasource}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"tooltip\": false,\n \"viz\": false,\n \"legend\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": true,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"binBps\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 32\n },\n \"id\": 9,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\"\n }\n },\n \"pluginVersion\": \"8.0.0\",\n \"targets\": [\n {\n \"expr\": \"rate(litestream_replica_operation_bytes{job=\\\"$job\\\"}[5m])\",\n \"legendFormat\": \"{{replica_type}} - {{operation}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Replica Throughput\",\n \"type\": \"timeseries\"\n }\n ],\n \"refresh\": \"30s\",\n \"schemaVersion\": 27,\n \"style\": \"dark\",\n \"tags\": [\"litestream\", \"sqlite\", \"replication\"],\n \"templating\": {\n \"list\": [\n {\n \"current\": {\n \"selected\": false,\n \"text\": \"Prometheus\",\n \"value\": \"Prometheus\"\n },\n \"hide\": 0,\n \"includeAll\": false,\n \"label\": \"Data Source\",\n \"multi\": false,\n \"name\": \"datasource\",\n \"options\": [],\n \"query\": \"prometheus\",\n \"queryValue\": \"\",\n \"refresh\": 1,\n \"regex\": \"\",\n \"skipUrlSync\": false,\n \"type\": \"datasource\"\n },\n {\n \"current\": {\n \"selected\": false,\n \"text\": \"litestream\",\n \"value\": \"litestream\"\n },\n \"datasource\": \"${datasource}\",\n \"definition\": \"label_values(litestream_db_size, job)\",\n \"hide\": 0,\n \"includeAll\": false,\n \"label\": \"Job\",\n \"multi\": false,\n \"name\": \"job\",\n \"options\": [],\n \"query\": {\n \"query\": \"label_values(litestream_db_size, job)\",\n \"refId\": \"StandardVariableQuery\"\n },\n \"refresh\": 1,\n \"regex\": \"\",\n \"skipUrlSync\": false,\n \"sort\": 0,\n \"type\": \"query\"\n }\n ]\n },\n \"time\": {\n \"from\": \"now-1h\",\n \"to\": \"now\"\n },\n \"timepicker\": {},\n \"timezone\": \"\",\n \"title\": \"Litestream Monitoring\",\n \"uid\": \"litestream-monitoring\",\n \"version\": 1\n}\n" }, { "path": "gs/replica_client.go", "content": "package gs\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net/url\"\n\t\"os\"\n\t\"path\"\n\t\"sync\"\n\t\"time\"\n\n\t\"cloud.google.com/go/storage\"\n\t\"github.com/superfly/ltx\"\n\t\"google.golang.org/api/iterator\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\nfunc init() {\n\tlitestream.RegisterReplicaClientFactory(\"gs\", NewReplicaClientFromURL)\n}\n\n// ReplicaClientType is the client type for this package.\nconst ReplicaClientType = \"gs\"\n\n// MetadataKeyTimestamp is the metadata key for storing LTX file timestamps in GCS.\nconst MetadataKeyTimestamp = \"litestream-timestamp\"\n\nvar _ litestream.ReplicaClient = (*ReplicaClient)(nil)\n\n// ReplicaClient is a client for writing LTX files to Google Cloud Storage.\ntype ReplicaClient struct {\n\tmu sync.Mutex\n\tclient *storage.Client // gs client\n\tbkt *storage.BucketHandle // gs bucket handle\n\tlogger *slog.Logger\n\n\t// GS bucket information\n\tBucket string\n\tPath string\n}\n\n// NewReplicaClient returns a new instance of ReplicaClient.\nfunc NewReplicaClient() *ReplicaClient {\n\treturn &ReplicaClient{\n\t\tlogger: slog.Default().WithGroup(ReplicaClientType),\n\t}\n}\n\nfunc (c *ReplicaClient) SetLogger(logger *slog.Logger) {\n\tc.logger = logger.WithGroup(ReplicaClientType)\n}\n\n// NewReplicaClientFromURL creates a new ReplicaClient from URL components.\n// This is used by the replica client factory registration.\nfunc NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error) {\n\tif host == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for gs replica URL\")\n\t}\n\n\tclient := NewReplicaClient()\n\tclient.Bucket = host\n\tclient.Path = urlPath\n\treturn client, nil\n}\n\n// Type returns \"gs\" as the client type.\nfunc (c *ReplicaClient) Type() string {\n\treturn ReplicaClientType\n}\n\n// Init initializes the connection to GS. No-op if already initialized.\nfunc (c *ReplicaClient) Init(ctx context.Context) (err error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tif c.client != nil {\n\t\treturn nil\n\t}\n\n\tif c.client, err = storage.NewClient(ctx); err != nil {\n\t\treturn fmt.Errorf(\"failed to create GCS client (bucket: %s): %w\", c.Bucket, err)\n\t}\n\tc.bkt = c.client.Bucket(c.Bucket)\n\n\treturn nil\n}\n\n// DeleteAll deletes all LTX files.\nfunc (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\t// Iterate over every object and delete it.\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"LIST\").Inc()\n\tfor it := c.bkt.Objects(ctx, &storage.Query{Prefix: c.Path + \"/\"}); ; {\n\t\tattrs, err := it.Next()\n\t\tif errors.Is(err, iterator.Done) {\n\t\t\tbreak\n\t\t} else if err != nil {\n\t\t\treturn fmt.Errorf(\"failed to list objects in GCS bucket %s (path: %s): %w\", c.Bucket, c.Path, err)\n\t\t}\n\n\t\tif err := c.bkt.Object(attrs.Name).Delete(ctx); isNotExists(err) {\n\t\t\tcontinue\n\t\t} else if err != nil {\n\t\t\treturn fmt.Errorf(\"gs: cannot delete object %q: %w\", attrs.Name, err)\n\t\t}\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\t}\n\n\t// log.Printf(\"%s(%s): retainer: deleting\", r.db.Path(), r.Name())\n\n\treturn nil\n}\n\n// LTXFiles returns an iterator over all available LTX files for a level.\n// GCS always uses accurate timestamps from metadata since they're included in LIST operations at zero cost.\n// The useMetadata parameter is ignored.\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tdir := litestream.LTXLevelDir(c.Path, level)\n\tprefix := dir + \"/\"\n\tif seek != 0 {\n\t\tprefix += seek.String()\n\t}\n\n\treturn newLTXFileIterator(c.bkt.Objects(ctx, &storage.Query{Prefix: prefix}), c, level), nil\n}\n\n// WriteLTXFile writes an LTX file from rd to a remote path.\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, rd io.Reader) (info *ltx.FileInfo, err error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn info, err\n\t}\n\n\tkey := litestream.LTXFilePath(c.Path, level, minTXID, maxTXID)\n\n\t// Use TeeReader to peek at LTX header while preserving data for upload\n\tvar buf bytes.Buffer\n\tteeReader := io.TeeReader(rd, &buf)\n\n\t// Extract timestamp from LTX header\n\thdr, _, err := ltx.PeekHeader(teeReader)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"extract timestamp from LTX header: %w\", err)\n\t}\n\ttimestamp := time.UnixMilli(hdr.Timestamp).UTC()\n\n\t// Combine buffered data with rest of reader\n\tfullReader := io.MultiReader(&buf, rd)\n\n\tw := c.bkt.Object(key).NewWriter(ctx)\n\tdefer w.Close()\n\n\t// Store timestamp in GCS metadata for accurate timestamp retrieval\n\tw.Metadata = map[string]string{\n\t\tMetadataKeyTimestamp: timestamp.Format(time.RFC3339Nano),\n\t}\n\n\tn, err := io.Copy(w, fullReader)\n\tif err != nil {\n\t\treturn info, err\n\t} else if err := w.Close(); err != nil {\n\t\treturn info, err\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Add(float64(n))\n\n\treturn <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tSize: n,\n\t\tCreatedAt: timestamp,\n\t}, nil\n}\n\n// OpenLTXFile returns a reader for a given LTX file.\n// Returns os.ErrNotExist if no matching index/offset is found.\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tkey := litestream.LTXFilePath(c.Path, level, minTXID, maxTXID)\n\n\t// In the GCS client, a length of -1 reads to EOF while 0 returns no data.\n\t// Callers pass size=0 to indicate \"entire object\" so translate that here.\n\t// See: https://pkg.go.dev/cloud.google.com/go/storage#ObjectHandle.NewRangeReader\n\tlength := size\n\tif length <= 0 {\n\t\tlength = -1\n\t}\n\n\tr, err := c.bkt.Object(key).NewRangeReader(ctx, offset, length)\n\tif isNotExists(err) {\n\t\treturn nil, os.ErrNotExist\n\t} else if err != nil {\n\t\treturn nil, err\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"GET\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, \"GET\").Add(float64(r.Attrs.Size))\n\n\treturn r, nil\n}\n\n// DeleteLTXFiles deletes a set of LTX files.\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\tfor _, info := range a {\n\t\tkey := litestream.LTXFilePath(c.Path, info.Level, info.MinTXID, info.MaxTXID)\n\n\t\tc.logger.Debug(\"deleting ltx file\", \"level\", info.Level, \"minTXID\", info.MinTXID, \"maxTXID\", info.MaxTXID, \"key\", key)\n\n\t\tif err := c.bkt.Object(key).Delete(ctx); err != nil && !isNotExists(err) {\n\t\t\treturn fmt.Errorf(\"gs: cannot delete ltx file %q: %w\", key, err)\n\t\t}\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\t}\n\n\treturn nil\n}\n\ntype ltxFileIterator struct {\n\tit *storage.ObjectIterator\n\tclient *ReplicaClient\n\tlevel int\n\tinfo *ltx.FileInfo\n\terr error\n}\n\nfunc newLTXFileIterator(it *storage.ObjectIterator, client *ReplicaClient, level int) *ltxFileIterator {\n\treturn <xFileIterator{\n\t\tit: it,\n\t\tclient: client,\n\t\tlevel: level,\n\t}\n}\n\nfunc (itr *ltxFileIterator) Close() (err error) {\n\treturn itr.err\n}\n\nfunc (itr *ltxFileIterator) Next() bool {\n\t// Exit if an error has already occurred.\n\tif itr.err != nil {\n\t\treturn false\n\t}\n\n\tfor {\n\t\t// Fetch next object.\n\t\tattrs, err := itr.it.Next()\n\t\tif errors.Is(err, iterator.Done) {\n\t\t\treturn false\n\t\t} else if err != nil {\n\t\t\titr.err = err\n\t\t\treturn false\n\t\t}\n\n\t\t// Parse index & offset, otherwise skip to the next object.\n\t\tminTXID, maxTXID, err := ltx.ParseFilename(path.Base(attrs.Name))\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Always use accurate timestamp from metadata since it's zero-cost\n\t\t// GCS includes metadata in LIST operations, so no extra API call needed\n\t\tcreatedAt := attrs.Created.UTC()\n\t\tif attrs.Metadata != nil {\n\t\t\tif ts, ok := attrs.Metadata[MetadataKeyTimestamp]; ok {\n\t\t\t\tif parsed, err := time.Parse(time.RFC3339Nano, ts); err == nil {\n\t\t\t\t\tcreatedAt = parsed\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Store current snapshot and return.\n\t\titr.info = <x.FileInfo{\n\t\t\tLevel: itr.level,\n\t\t\tMinTXID: minTXID,\n\t\t\tMaxTXID: maxTXID,\n\t\t\tSize: attrs.Size,\n\t\t\tCreatedAt: createdAt,\n\t\t}\n\n\t\treturn true\n\t}\n}\n\nfunc (itr *ltxFileIterator) Err() error { return itr.err }\n\nfunc (itr *ltxFileIterator) Item() *ltx.FileInfo {\n\treturn itr.info\n}\n\nfunc isNotExists(err error) bool {\n\treturn errors.Is(err, storage.ErrObjectNotExist)\n}\n" }, { "path": "gs/replica_client_test.go", "content": "package gs\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"io\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/fsouza/fake-gcs-server/fakestorage\"\n\t\"github.com/superfly/ltx\"\n)\n\nfunc ltxTestData(tb testing.TB, minTXID, maxTXID ltx.TXID, payload []byte) []byte {\n\ttb.Helper()\n\n\thdr := ltx.Header{\n\t\tVersion: 1,\n\t\tPageSize: 4096,\n\t\tCommit: 1,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tTimestamp: time.Now().UnixMilli(),\n\t}\n\n\tbuf, err := hdr.MarshalBinary()\n\tif err != nil {\n\t\ttb.Fatalf(\"marshal header: %v\", err)\n\t}\n\n\treturn append(buf, payload...)\n}\n\nfunc setupTestClient(tb testing.TB) (*ReplicaClient, *fakestorage.Server) {\n\ttb.Helper()\n\n\tserver, err := fakestorage.NewServerWithOptions(fakestorage.Options{NoListener: true})\n\tif err != nil {\n\t\ttb.Fatalf(\"new server: %v\", err)\n\t}\n\n\tbucket := \"litestream-test\"\n\tserver.CreateBucketWithOpts(fakestorage.CreateBucketOpts{Name: bucket})\n\n\tclient := server.Client()\n\n\trc := NewReplicaClient()\n\trc.client = client\n\trc.bkt = client.Bucket(bucket)\n\trc.Bucket = bucket\n\trc.Path = \"integration\"\n\n\treturn rc, server\n}\n\nfunc TestReplicaClient_OpenLTXFileReadsFullObject(t *testing.T) {\n\trc, server := setupTestClient(t)\n\tdefer server.Stop()\n\n\tctx := context.Background()\n\tdata := ltxTestData(t, ltx.TXID(1), ltx.TXID(1), []byte(\"hello\"))\n\n\tif _, err := rc.WriteLTXFile(ctx, 0, ltx.TXID(1), ltx.TXID(1), bytes.NewReader(data)); err != nil {\n\t\tt.Fatalf(\"WriteLTXFile: %v\", err)\n\t}\n\n\tr, err := rc.OpenLTXFile(ctx, 0, ltx.TXID(1), ltx.TXID(1), 0, 0)\n\tif err != nil {\n\t\tt.Fatalf(\"OpenLTXFile: %v\", err)\n\t}\n\tdefer r.Close()\n\n\tout, err := io.ReadAll(r)\n\tif err != nil {\n\t\tt.Fatalf(\"ReadAll: %v\", err)\n\t}\n\n\tif !bytes.Equal(out, data) {\n\t\tt.Fatalf(\"unexpected replica content: got %q, want %q\", out, data)\n\t}\n}\n" }, { "path": "heartbeat.go", "content": "package litestream\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"sync\"\n\t\"time\"\n)\n\nconst (\n\tDefaultHeartbeatInterval = 5 * time.Minute\n\tDefaultHeartbeatTimeout = 30 * time.Second\n\tMinHeartbeatInterval = 1 * time.Minute\n)\n\ntype HeartbeatClient struct {\n\tmu sync.Mutex\n\thttpClient *http.Client\n\n\tURL string\n\tInterval time.Duration\n\tTimeout time.Duration\n\n\tlastPingAt time.Time\n}\n\nfunc NewHeartbeatClient(url string, interval time.Duration) *HeartbeatClient {\n\tif interval < MinHeartbeatInterval {\n\t\tinterval = MinHeartbeatInterval\n\t}\n\n\ttimeout := DefaultHeartbeatTimeout\n\n\treturn &HeartbeatClient{\n\t\tURL: url,\n\t\tInterval: interval,\n\t\tTimeout: timeout,\n\t\thttpClient: &http.Client{\n\t\t\tTimeout: timeout,\n\t\t},\n\t}\n}\n\nfunc (c *HeartbeatClient) Ping(ctx context.Context) error {\n\tif c.URL == \"\" {\n\t\treturn nil\n\t}\n\n\treq, err := http.NewRequestWithContext(ctx, http.MethodGet, c.URL, nil)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create request: %w\", err)\n\t}\n\n\tresp, err := c.httpClient.Do(req)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"http request: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tif resp.StatusCode < 200 || resp.StatusCode >= 300 {\n\t\treturn fmt.Errorf(\"unexpected status code: %d\", resp.StatusCode)\n\t}\n\n\treturn nil\n}\n\nfunc (c *HeartbeatClient) ShouldPing() bool {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\treturn time.Since(c.lastPingAt) >= c.Interval\n}\n\nfunc (c *HeartbeatClient) LastPingAt() time.Time {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\treturn c.lastPingAt\n}\n\nfunc (c *HeartbeatClient) RecordPing() {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\tc.lastPingAt = time.Now()\n}\n" }, { "path": "heartbeat_test.go", "content": "package litestream_test\n\nimport (\n\t\"context\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"path/filepath\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestHeartbeatClient_Ping(t *testing.T) {\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\tvar pingCount atomic.Int64\n\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\tif r.Method != http.MethodGet {\n\t\t\t\tt.Errorf(\"expected GET, got %s\", r.Method)\n\t\t\t}\n\t\t\tpingCount.Add(1)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}))\n\t\tdefer server.Close()\n\n\t\tclient := litestream.NewHeartbeatClient(server.URL, 5*time.Minute)\n\t\tif err := client.Ping(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"expected no error, got %v\", err)\n\t\t}\n\n\t\tif got := pingCount.Load(); got != 1 {\n\t\t\tt.Errorf(\"expected 1 ping, got %d\", got)\n\t\t}\n\t})\n\n\tt.Run(\"EmptyURL\", func(t *testing.T) {\n\t\tclient := litestream.NewHeartbeatClient(\"\", 5*time.Minute)\n\t\tif err := client.Ping(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"expected no error for empty URL, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NonSuccessStatusCode\", func(t *testing.T) {\n\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\tw.WriteHeader(http.StatusInternalServerError)\n\t\t}))\n\t\tdefer server.Close()\n\n\t\tclient := litestream.NewHeartbeatClient(server.URL, 5*time.Minute)\n\t\terr := client.Ping(context.Background())\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for 500 status code\")\n\t\t}\n\t})\n\n\tt.Run(\"NetworkError\", func(t *testing.T) {\n\t\tclient := litestream.NewHeartbeatClient(\"http://localhost:1\", 5*time.Minute)\n\t\terr := client.Ping(context.Background())\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for unreachable server\")\n\t\t}\n\t})\n\n\tt.Run(\"ContextCanceled\", func(t *testing.T) {\n\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\ttime.Sleep(100 * time.Millisecond)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}))\n\t\tdefer server.Close()\n\n\t\tctx, cancel := context.WithCancel(context.Background())\n\t\tcancel()\n\n\t\tclient := litestream.NewHeartbeatClient(server.URL, 5*time.Minute)\n\t\terr := client.Ping(ctx)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for canceled context\")\n\t\t}\n\t})\n}\n\nfunc TestHeartbeatClient_ShouldPing(t *testing.T) {\n\tt.Run(\"FirstPing\", func(t *testing.T) {\n\t\tclient := litestream.NewHeartbeatClient(\"http://example.com\", 5*time.Minute)\n\t\tif !client.ShouldPing() {\n\t\t\tt.Error(\"expected ShouldPing to return true for first ping\")\n\t\t}\n\t})\n\n\tt.Run(\"AfterRecordPing\", func(t *testing.T) {\n\t\tclient := litestream.NewHeartbeatClient(\"http://example.com\", 5*time.Minute)\n\t\tclient.RecordPing()\n\n\t\tif client.ShouldPing() {\n\t\t\tt.Error(\"expected ShouldPing to return false immediately after RecordPing\")\n\t\t}\n\t})\n}\n\nfunc TestHeartbeatClient_MinInterval(t *testing.T) {\n\tclient := litestream.NewHeartbeatClient(\"http://example.com\", 30*time.Second)\n\tif client.Interval != litestream.MinHeartbeatInterval {\n\t\tt.Errorf(\"expected interval to be clamped to %v, got %v\", litestream.MinHeartbeatInterval, client.Interval)\n\t}\n}\n\nfunc TestHeartbeatClient_LastPingAt(t *testing.T) {\n\tclient := litestream.NewHeartbeatClient(\"http://example.com\", 5*time.Minute)\n\n\tif !client.LastPingAt().IsZero() {\n\t\tt.Error(\"expected LastPingAt to be zero initially\")\n\t}\n\n\tbefore := time.Now()\n\tclient.RecordPing()\n\tafter := time.Now()\n\n\tlastPing := client.LastPingAt()\n\tif lastPing.Before(before) || lastPing.After(after) {\n\t\tt.Errorf(\"LastPingAt %v should be between %v and %v\", lastPing, before, after)\n\t}\n}\n\nfunc TestStore_Heartbeat_AllDatabasesHealthy(t *testing.T) {\n\tt.Run(\"NoDatabases\", func(t *testing.T) {\n\t\tlevels := litestream.CompactionLevels{{Level: 0}}\n\t\tstore := litestream.NewStore(nil, levels)\n\t\tstore.CompactionMonitorEnabled = false\n\t\tstore.HeartbeatCheckInterval = 0 // Disable automatic monitoring\n\n\t\tvar pingCount atomic.Int64\n\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {\n\t\t\tpingCount.Add(1)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}))\n\t\tdefer server.Close()\n\n\t\tstore.Heartbeat = litestream.NewHeartbeatClient(server.URL, 1*time.Minute)\n\n\t\t// With no databases, heartbeat should not fire\n\t\t// We need to trigger the check manually since monitor is disabled\n\t\t// The store won't send pings because allDatabasesHealthy returns false for empty stores\n\t\tif pingCount.Load() != 0 {\n\t\t\tt.Errorf(\"expected no pings with no databases, got %d\", pingCount.Load())\n\t\t}\n\t})\n\n\tt.Run(\"AllDatabasesSynced\", func(t *testing.T) {\n\t\tdb0, sqldb0 := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db0, sqldb0)\n\n\t\tdb1, sqldb1 := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db1, sqldb1)\n\n\t\tlevels := litestream.CompactionLevels{{Level: 0}, {Level: 1, Interval: time.Second}}\n\t\tstore := litestream.NewStore([]*litestream.DB{db0, db1}, levels)\n\t\tstore.CompactionMonitorEnabled = false\n\t\tstore.HeartbeatCheckInterval = 0\n\n\t\tvar pingCount atomic.Int64\n\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {\n\t\t\tpingCount.Add(1)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}))\n\t\tdefer server.Close()\n\n\t\tstore.Heartbeat = litestream.NewHeartbeatClient(server.URL, 1*time.Minute)\n\n\t\tif err := store.Open(t.Context()); err != nil {\n\t\t\tt.Fatalf(\"open store: %v\", err)\n\t\t}\n\t\tdefer store.Close(t.Context())\n\n\t\t// Create tables and sync both databases\n\t\tif _, err := sqldb0.ExecContext(t.Context(), `CREATE TABLE t (id INT)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db0.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db0.Replica.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif _, err := sqldb1.ExecContext(t.Context(), `CREATE TABLE t (id INT)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db1.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db1.Replica.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Both databases have synced, heartbeat should fire\n\t\tif err := store.Heartbeat.Ping(t.Context()); err != nil {\n\t\t\tt.Fatalf(\"ping failed: %v\", err)\n\t\t}\n\n\t\tif got := pingCount.Load(); got != 1 {\n\t\t\tt.Errorf(\"expected 1 ping after all DBs synced, got %d\", got)\n\t\t}\n\t})\n\n\tt.Run(\"OneDatabaseNotSynced\", func(t *testing.T) {\n\t\tdb0, sqldb0 := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db0, sqldb0)\n\n\t\t// Create second DB but don't sync it\n\t\tdb1 := litestream.NewDB(filepath.Join(t.TempDir(), \"db1\"))\n\t\tdb1.Replica = litestream.NewReplica(db1)\n\t\tdb1.Replica.Client = testingutil.NewFileReplicaClient(t)\n\t\tdb1.Replica.MonitorEnabled = false\n\t\tdb1.MonitorInterval = 0\n\n\t\tlevels := litestream.CompactionLevels{{Level: 0}, {Level: 1, Interval: time.Second}}\n\t\tstore := litestream.NewStore([]*litestream.DB{db0, db1}, levels)\n\t\tstore.CompactionMonitorEnabled = false\n\t\tstore.HeartbeatCheckInterval = 0\n\n\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}))\n\t\tdefer server.Close()\n\n\t\tstore.Heartbeat = litestream.NewHeartbeatClient(server.URL, 1*time.Minute)\n\n\t\tif err := store.Open(t.Context()); err != nil {\n\t\t\tt.Fatalf(\"open store: %v\", err)\n\t\t}\n\t\tdefer store.Close(t.Context())\n\n\t\t// Only sync db0\n\t\tif _, err := sqldb0.ExecContext(t.Context(), `CREATE TABLE t (id INT)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db0.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := db0.Replica.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// db1 hasn't synced, so LastSuccessfulSyncAt should be zero\n\t\tif !db1.LastSuccessfulSyncAt().IsZero() {\n\t\t\tt.Error(\"expected db1.LastSuccessfulSyncAt to be zero\")\n\t\t}\n\n\t\t// db0 has synced\n\t\tif db0.LastSuccessfulSyncAt().IsZero() {\n\t\t\tt.Error(\"expected db0.LastSuccessfulSyncAt to be non-zero\")\n\t\t}\n\t})\n}\n" }, { "path": "internal/hexdump.go", "content": "package internal\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n)\n\nfunc Hexdump(data []byte) string {\n\tprevRow := make([]byte, 16)\n\n\tvar buf bytes.Buffer\n\tvar dupWritten bool\n\tfor i := 0; i < len(data); i += 16 {\n\t\trow := make([]byte, 16)\n\t\tcopy(row, data[i:])\n\n\t\t// Write out one line of asterisks to show that we just have duplicate rows.\n\t\tif i != 0 && i+16 < len(data) && bytes.Equal(row, prevRow) {\n\t\t\tif !dupWritten {\n\t\t\t\tdupWritten = true\n\t\t\t\tfmt.Fprintln(&buf, \"***\")\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\t// Track previous row so we know when we have duplicates.\n\t\tcopy(prevRow, row)\n\t\tdupWritten = false\n\n\t\tfmt.Fprintf(&buf, \"%08x %02x %02x %02x %02x %02x %02x %02x %02x %02x %02x %02x %02x %02x %02x %02x %02x |%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s|\\n\", i,\n\t\t\trow[0], row[1], row[2], row[3], row[4], row[5], row[6], row[7],\n\t\t\trow[8], row[9], row[10], row[11], row[12], row[13], row[14], row[15],\n\t\t\ttoChar(row[0]), toChar(row[1]), toChar(row[2]), toChar(row[3]), toChar(row[4]), toChar(row[5]), toChar(row[6]), toChar(row[7]),\n\t\t\ttoChar(row[8]), toChar(row[9]), toChar(row[10]), toChar(row[11]), toChar(row[12]), toChar(row[13]), toChar(row[14]), toChar(row[15]),\n\t\t)\n\t}\n\treturn buf.String()\n}\n\nfunc toChar(b byte) string {\n\tif b < 32 || b > 126 {\n\t\treturn \".\"\n\t}\n\treturn string(b)\n}\n" }, { "path": "internal/internal.go", "content": "package internal\n\nimport (\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"syscall\"\n\n\t\"github.com/pierrec/lz4/v4\"\n\t\"github.com/prometheus/client_golang/prometheus\"\n\t\"github.com/prometheus/client_golang/prometheus/promauto\"\n)\n\nconst LevelTrace = slog.LevelDebug - 4\n\n// ReadCloser wraps a reader to also attach a separate closer.\ntype ReadCloser struct {\n\tr io.Reader\n\tc io.Closer\n}\n\n// NewReadCloser returns a new instance of ReadCloser.\nfunc NewReadCloser(r io.Reader, c io.Closer) *ReadCloser {\n\treturn &ReadCloser{r, c}\n}\n\n// Read reads bytes into the underlying reader.\nfunc (r *ReadCloser) Read(p []byte) (n int, err error) {\n\treturn r.r.Read(p)\n}\n\n// Close closes the reader (if implementing io.ReadCloser) and the Closer.\nfunc (r *ReadCloser) Close() error {\n\tif rc, ok := r.r.(io.Closer); ok {\n\t\tif err := rc.Close(); err != nil {\n\t\t\tr.c.Close()\n\t\t\treturn err\n\t\t}\n\t}\n\treturn r.c.Close()\n}\n\n// LZ4ReadCloser wraps an LZ4 reader with the underlying source for proper closing.\ntype LZ4ReadCloser struct {\n\t*lz4.Reader\n\tunderlying io.Closer\n}\n\nfunc (r *LZ4ReadCloser) Close() error {\n\treturn r.underlying.Close()\n}\n\n// NewLZ4Reader creates an LZ4 decompressing reader that wraps the source.\n// Closing the returned reader also closes the underlying source.\nfunc NewLZ4Reader(r io.ReadCloser) io.ReadCloser {\n\treturn &LZ4ReadCloser{\n\t\tReader: lz4.NewReader(r),\n\t\tunderlying: r,\n\t}\n}\n\n// ReadCounter wraps an io.Reader and counts the total number of bytes read.\ntype ReadCounter struct {\n\tr io.Reader\n\tn int64\n}\n\n// NewReadCounter returns a new instance of ReadCounter that wraps r.\nfunc NewReadCounter(r io.Reader) *ReadCounter {\n\treturn &ReadCounter{r: r}\n}\n\n// Read reads from the underlying reader into p and adds the bytes read to the counter.\nfunc (r *ReadCounter) Read(p []byte) (int, error) {\n\tn, err := r.r.Read(p)\n\tr.n += int64(n)\n\treturn n, err\n}\n\n// N returns the total number of bytes read.\nfunc (r *ReadCounter) N() int64 { return r.n }\n\n// CreateFile creates the file and matches the mode & uid/gid of fi.\nfunc CreateFile(filename string, fi os.FileInfo) (*os.File, error) {\n\tmode := os.FileMode(0600)\n\tif fi != nil {\n\t\tmode = fi.Mode()\n\t}\n\n\tf, err := os.OpenFile(filename, os.O_RDWR|os.O_CREATE|os.O_TRUNC, mode)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tuid, gid := Fileinfo(fi)\n\t_ = f.Chown(uid, gid)\n\treturn f, nil\n}\n\n// MkdirAll is a copy of os.MkdirAll() except that it attempts to set the\n// mode/uid/gid to match fi for each created directory.\nfunc MkdirAll(path string, fi os.FileInfo) error {\n\tuid, gid := Fileinfo(fi)\n\n\t// Fast path: if we can tell whether path is a directory or file, stop with success or error.\n\tdir, err := os.Stat(path)\n\tif err == nil {\n\t\tif dir.IsDir() {\n\t\t\treturn nil\n\t\t}\n\t\treturn &os.PathError{Op: \"mkdir\", Path: path, Err: syscall.ENOTDIR}\n\t}\n\n\t// Slow path: make sure parent exists and then call Mkdir for path.\n\ti := len(path)\n\tfor i > 0 && os.IsPathSeparator(path[i-1]) { // Skip trailing path separator.\n\t\ti--\n\t}\n\n\tj := i\n\tfor j > 0 && !os.IsPathSeparator(path[j-1]) { // Scan backward over element.\n\t\tj--\n\t}\n\n\tif j > 1 {\n\t\t// Create parent.\n\t\terr = MkdirAll(fixRootDirectory(path[:j-1]), fi)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// Parent now exists; invoke Mkdir and use its result.\n\tmode := os.FileMode(0700)\n\tif fi != nil {\n\t\tmode = fi.Mode()\n\t}\n\terr = os.Mkdir(path, mode)\n\tif err != nil {\n\t\t// Handle arguments like \"foo/.\" by\n\t\t// double-checking that directory doesn't exist.\n\t\tdir, err1 := os.Lstat(path)\n\t\tif err1 == nil && dir.IsDir() {\n\t\t\t_ = os.Chown(path, uid, gid)\n\t\t\treturn nil\n\t\t}\n\t\treturn err\n\t}\n\t_ = os.Chown(path, uid, gid)\n\treturn nil\n}\n\nfunc ReplaceAttr(groups []string, a slog.Attr) slog.Attr {\n\tif a.Key == slog.LevelKey && a.Value.Any() == LevelTrace {\n\t\ta.Value = slog.StringValue(\"TRACE\")\n\t}\n\treturn a\n}\n\n// Shared replica metrics.\nvar (\n\tOperationTotalCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_replica_operation_total\",\n\t\tHelp: \"The number of replica operations performed\",\n\t}, []string{\"replica_type\", \"operation\"})\n\n\tOperationBytesCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_replica_operation_bytes\",\n\t\tHelp: \"The number of bytes used by replica operations\",\n\t}, []string{\"replica_type\", \"operation\"})\n\n\tOperationDurationHistogramVec = promauto.NewHistogramVec(prometheus.HistogramOpts{\n\t\tName: \"litestream_replica_operation_duration_seconds\",\n\t\tHelp: \"Duration of replica operations by type and operation\",\n\t\tBuckets: []float64{0.01, 0.05, 0.1, 0.5, 1, 5, 10, 30, 60},\n\t}, []string{\"replica_type\", \"operation\"})\n\n\tOperationErrorCounterVec = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"litestream_replica_operation_errors_total\",\n\t\tHelp: \"Number of replica operation errors by type, operation, and error code\",\n\t}, []string{\"replica_type\", \"operation\", \"code\"})\n\n\tL0RetentionGaugeVec = promauto.NewGaugeVec(prometheus.GaugeOpts{\n\t\tName: \"litestream_l0_retention_files_total\",\n\t\tHelp: \"Number of L0 files by status during retention enforcement\",\n\t}, []string{\"db\", \"status\"})\n)\n" }, { "path": "internal/internal_unix.go", "content": "//go:build aix || darwin || dragonfly || freebsd || linux || netbsd || openbsd || solaris\n// +build aix darwin dragonfly freebsd linux netbsd openbsd solaris\n\npackage internal\n\nimport (\n\t\"os\"\n\t\"syscall\"\n)\n\n// Fileinfo returns syscall fields from a FileInfo object.\nfunc Fileinfo(fi os.FileInfo) (uid, gid int) {\n\tif fi == nil {\n\t\treturn -1, -1\n\t}\n\tstat, ok := fi.Sys().(*syscall.Stat_t)\n\tif !ok {\n\t\treturn -1, -1\n\t}\n\treturn int(stat.Uid), int(stat.Gid)\n}\n\nfunc fixRootDirectory(p string) string {\n\treturn p\n}\n" }, { "path": "internal/internal_windows.go", "content": "//go:build windows\n// +build windows\n\npackage internal\n\nimport (\n\t\"os\"\n)\n\n// Fileinfo returns syscall fields from a FileInfo object.\nfunc Fileinfo(fi os.FileInfo) (uid, gid int) {\n\treturn -1, -1\n}\n\n// fixRootDirectory is copied from the standard library for use with mkdirAll()\nfunc fixRootDirectory(p string) string {\n\tif len(p) == len(`\\\\?\\c:`) {\n\t\tif os.IsPathSeparator(p[0]) && os.IsPathSeparator(p[1]) && p[2] == '?' && os.IsPathSeparator(p[3]) && p[5] == ':' {\n\t\t\treturn p + `\\`\n\t\t}\n\t}\n\treturn p\n}\n" }, { "path": "internal/limit_read_closer.go", "content": "package internal\n\nimport \"io\"\n\n// Copied from the io package to implement io.Closer.\nfunc LimitReadCloser(r io.ReadCloser, n int64) io.ReadCloser {\n\treturn &LimitedReadCloser{r, n}\n}\n\ntype LimitedReadCloser struct {\n\tR io.ReadCloser // underlying reader\n\tN int64 // max bytes remaining\n}\n\nfunc (l *LimitedReadCloser) Close() error {\n\treturn l.R.Close()\n}\n\nfunc (l *LimitedReadCloser) Read(p []byte) (n int, err error) {\n\tif l.N <= 0 {\n\t\treturn 0, io.EOF\n\t}\n\tif int64(len(p)) > l.N {\n\t\tp = p[0:l.N]\n\t}\n\tn, err = l.R.Read(p)\n\tl.N -= int64(n)\n\treturn\n}\n" }, { "path": "internal/lock_unix.go", "content": "//go:build aix || darwin || dragonfly || freebsd || linux || netbsd || openbsd || solaris\n\npackage internal\n\nimport (\n\t\"os\"\n\n\t\"golang.org/x/sys/unix\"\n)\n\nconst (\n\tsqlitePendingByte = 0x40000000\n\tsqliteSharedFirst = sqlitePendingByte + 2\n\tsqliteSharedSize = 510\n)\n\nfunc LockFileExclusive(f *os.File) error {\n\tfd := int(f.Fd())\n\n\tif err := setFcntlLock(fd, unix.F_WRLCK, sqlitePendingByte, 1); err != nil {\n\t\treturn err\n\t}\n\n\tif err := setFcntlLock(fd, unix.F_WRLCK, sqliteSharedFirst, sqliteSharedSize); err != nil {\n\t\t_ = setFcntlLock(fd, unix.F_UNLCK, sqlitePendingByte, 1)\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc UnlockFile(f *os.File) error {\n\tfd := int(f.Fd())\n\terr1 := setFcntlLock(fd, unix.F_UNLCK, sqliteSharedFirst, sqliteSharedSize)\n\terr2 := setFcntlLock(fd, unix.F_UNLCK, sqlitePendingByte, 1)\n\tif err1 != nil {\n\t\treturn err1\n\t}\n\treturn err2\n}\n\nfunc setFcntlLock(fd int, lockType int16, start int64, length int64) error {\n\tflock := unix.Flock_t{\n\t\tType: lockType,\n\t\tWhence: 0,\n\t\tStart: start,\n\t\tLen: length,\n\t}\n\treturn unix.FcntlFlock(uintptr(fd), unix.F_SETLKW, &flock)\n}\n" }, { "path": "internal/lock_windows.go", "content": "//go:build windows\n\npackage internal\n\nimport (\n\t\"os\"\n\n\t\"golang.org/x/sys/windows\"\n)\n\nconst (\n\tsqlitePendingByte = 0x40000000\n\tsqliteSharedFirst = sqlitePendingByte + 2\n\tsqliteSharedSize = 510\n)\n\nfunc LockFileExclusive(f *os.File) error {\n\th := windows.Handle(f.Fd())\n\n\tpendingOL := windows.Overlapped{Offset: sqlitePendingByte}\n\tif err := windows.LockFileEx(h, windows.LOCKFILE_EXCLUSIVE_LOCK, 0, 1, 0, &pendingOL); err != nil {\n\t\treturn err\n\t}\n\n\tsharedOL := windows.Overlapped{Offset: sqliteSharedFirst}\n\tif err := windows.LockFileEx(h, windows.LOCKFILE_EXCLUSIVE_LOCK, 0, sqliteSharedSize, 0, &sharedOL); err != nil {\n\t\t_ = windows.UnlockFileEx(h, 0, 1, 0, &pendingOL)\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc UnlockFile(f *os.File) error {\n\th := windows.Handle(f.Fd())\n\n\tsharedOL := windows.Overlapped{Offset: sqliteSharedFirst}\n\terr1 := windows.UnlockFileEx(h, 0, sqliteSharedSize, 0, &sharedOL)\n\n\tpendingOL := windows.Overlapped{Offset: sqlitePendingByte}\n\terr2 := windows.UnlockFileEx(h, 0, 1, 0, &pendingOL)\n\n\tif err1 != nil {\n\t\treturn err1\n\t}\n\treturn err2\n}\n" }, { "path": "internal/resumable_reader.go", "content": "package internal\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\n\t\"github.com/superfly/ltx\"\n)\n\ntype LTXFileOpener interface {\n\tOpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n}\n\n// resumableReader wraps an io.ReadCloser from a remote storage backend with\n// automatic reconnection on read errors.\n//\n// During restore, the LTX compactor opens all LTX file streams upfront, then\n// processes pages in page-number order. Incremental LTX files that only contain\n// high-numbered pages may have their S3/storage streams sit idle for minutes\n// while the compactor works through lower-numbered pages from the snapshot.\n// Storage providers (S3, Tigris, etc.) may close these idle connections,\n// causing \"unexpected EOF\" errors.\n//\n// This reader detects two failure modes:\n// 1. Non-EOF errors (connection reset, timeout) - the stream broke mid-transfer.\n// 2. Premature EOF - the server closed the connection cleanly, but we haven't\n// read all bytes yet (detected by comparing offset against known file size).\n//\n// On failure, it closes the dead stream and reopens from the current byte\n// offset using the storage backend's range request support (the offset parameter\n// of OpenLTXFile). Callers like io.ReadFull see a seamless byte stream because\n// partial reads are returned without error, prompting the caller to request\n// remaining bytes on the next Read call.\ntype ResumableReader struct {\n\tctx context.Context\n\tclient LTXFileOpener\n\tlevel int\n\tminTXID ltx.TXID\n\tmaxTXID ltx.TXID\n\tsize int64 // expected total file size from FileInfo; 0 means unknown\n\toffset int64\n\trc io.ReadCloser\n\tlogger *slog.Logger\n}\n\n// NewResumableReader creates a ResumableReader. Primarily exposed for testing.\nfunc NewResumableReader(ctx context.Context, client LTXFileOpener, level int, minTXID, maxTXID ltx.TXID, size int64, rc io.ReadCloser, logger *slog.Logger) *ResumableReader {\n\treturn &ResumableReader{\n\t\tctx: ctx,\n\t\tclient: client,\n\t\tlevel: level,\n\t\tminTXID: minTXID,\n\t\tmaxTXID: maxTXID,\n\t\tsize: size,\n\t\trc: rc,\n\t\tlogger: logger,\n\t}\n}\n\nconst resumableReaderMaxRetries = 3\n\nfunc (r *ResumableReader) Read(p []byte) (int, error) {\n\tfor attempt := 0; attempt <= resumableReaderMaxRetries; attempt++ {\n\t\t// Reopen the stream from the current offset if the previous\n\t\t// connection was closed (rc is nil after a retry).\n\t\tif r.rc == nil {\n\t\t\trc, err := r.client.OpenLTXFile(r.ctx, r.level, r.minTXID, r.maxTXID, r.offset, 0)\n\t\t\tif err != nil {\n\t\t\t\treturn 0, fmt.Errorf(\"reopen ltx file at offset %d: %w\", r.offset, err)\n\t\t\t}\n\t\t\tr.rc = rc\n\t\t}\n\n\t\tn, err := r.rc.Read(p)\n\t\tr.offset += int64(n)\n\n\t\tif err == nil {\n\t\t\treturn n, nil\n\t\t}\n\n\t\tif err == io.EOF {\n\t\t\t// Distinguish legitimate EOF (fully read) from premature EOF\n\t\t\t// (server closed idle connection). When the file size is known\n\t\t\t// and we haven't read it all, treat as a connection drop.\n\t\t\tif r.size > 0 && r.offset < r.size {\n\t\t\t\tr.logger.Debug(\"premature EOF on ltx file, reconnecting\",\n\t\t\t\t\t\"level\", r.level, \"min\", r.minTXID, \"max\", r.maxTXID,\n\t\t\t\t\t\"offset\", r.offset, \"size\", r.size, \"attempt\", attempt+1)\n\t\t\t\tr.rc.Close()\n\t\t\t\tr.rc = nil\n\t\t\t\tif n > 0 {\n\t\t\t\t\t// Return the bytes we did get. The caller (e.g. io.ReadFull)\n\t\t\t\t\t// will call Read again, which will trigger the reopen above.\n\t\t\t\t\treturn n, nil\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\treturn n, io.EOF\n\t\t}\n\n\t\t// Non-EOF error (connection reset, timeout, etc.). Close the dead\n\t\t// stream so the next iteration reopens from the current offset.\n\t\tr.logger.Debug(\"read error on ltx file, reconnecting\",\n\t\t\t\"level\", r.level, \"min\", r.minTXID, \"max\", r.maxTXID,\n\t\t\t\"error\", err, \"offset\", r.offset, \"attempt\", attempt+1)\n\t\tr.rc.Close()\n\t\tr.rc = nil\n\t\tif n > 0 {\n\t\t\treturn n, nil\n\t\t}\n\t\tcontinue\n\t}\n\n\treturn 0, fmt.Errorf(\"max retries exceeded reading ltx file (level=%d, min=%s, max=%s, offset=%d)\",\n\t\tr.level, r.minTXID, r.maxTXID, r.offset)\n}\n\nfunc (r *ResumableReader) Close() error {\n\tif r.rc != nil {\n\t\treturn r.rc.Close()\n\t}\n\treturn nil\n}\n" }, { "path": "internal/resumable_reader_test.go", "content": "package internal\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/superfly/ltx\"\n)\n\nfunc TestResumableReader(t *testing.T) {\n\t// The resumableReader wraps a storage stream to handle connection drops\n\t// during long restore operations. These tests simulate the failure modes\n\t// that occur when S3/Tigris closes idle connections.\n\n\tt.Run(\"NormalRead\", func(t *testing.T) {\n\t\t// Verify that a healthy stream passes through unchanged.\n\t\tdata := []byte(\"hello world\")\n\t\tclient := &testLTXFileOpener{\n\t\t\tOpenLTXFileFunc: func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\t\t\t\treturn io.NopCloser(bytes.NewReader(data[offset:])), nil\n\t\t\t},\n\t\t}\n\n\t\tr := newTestResumableReader(client, int64(len(data)), data)\n\t\tgot, err := io.ReadAll(r)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !bytes.Equal(got, data) {\n\t\t\tt.Fatalf(\"got %q, want %q\", got, data)\n\t\t}\n\t})\n\n\tt.Run(\"ReconnectOnError\", func(t *testing.T) {\n\t\t// Simulate a connection reset after reading 5 bytes of a 11-byte file.\n\t\t// The reader should transparently reconnect from offset 5 and deliver\n\t\t// the remaining bytes.\n\t\tdata := []byte(\"hello world\")\n\t\tcallCount := 0\n\t\tclient := &testLTXFileOpener{\n\t\t\tOpenLTXFileFunc: func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\t\t\t\tcallCount++\n\t\t\t\tif callCount == 1 {\n\t\t\t\t\t// First open: return a reader that errors after 5 bytes.\n\t\t\t\t\treturn io.NopCloser(&errorAfterN{data: data, n: 5, err: fmt.Errorf(\"connection reset\")}), nil\n\t\t\t\t}\n\t\t\t\t// Reconnect: serve from the requested offset.\n\t\t\t\treturn io.NopCloser(bytes.NewReader(data[offset:])), nil\n\t\t\t},\n\t\t}\n\n\t\tr := newTestResumableReader(client, int64(len(data)), data)\n\t\tgot, err := io.ReadAll(r)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !bytes.Equal(got, data) {\n\t\t\tt.Fatalf(\"got %q, want %q\", got, data)\n\t\t}\n\t\tif callCount != 2 {\n\t\t\tt.Fatalf(\"expected 2 OpenLTXFile calls (original + reconnect), got %d\", callCount)\n\t\t}\n\t})\n\n\tt.Run(\"ReconnectOnPrematureEOF\", func(t *testing.T) {\n\t\t// Simulate a server that closes the connection cleanly (returns io.EOF)\n\t\t// before all bytes are transferred. The reader detects this by comparing\n\t\t// bytes read against the known file size.\n\t\tdata := []byte(\"hello world\")\n\t\tcallCount := 0\n\t\tclient := &testLTXFileOpener{\n\t\t\tOpenLTXFileFunc: func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\t\t\t\tcallCount++\n\t\t\t\tif callCount == 1 {\n\t\t\t\t\t// First open: return only the first 5 bytes, then EOF.\n\t\t\t\t\treturn io.NopCloser(bytes.NewReader(data[:5])), nil\n\t\t\t\t}\n\t\t\t\t// Reconnect: serve from the requested offset.\n\t\t\t\treturn io.NopCloser(bytes.NewReader(data[offset:])), nil\n\t\t\t},\n\t\t}\n\n\t\tr := newTestResumableReader(client, int64(len(data)), data)\n\t\tgot, err := io.ReadAll(r)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !bytes.Equal(got, data) {\n\t\t\tt.Fatalf(\"got %q, want %q\", got, data)\n\t\t}\n\t\tif callCount != 2 {\n\t\t\tt.Fatalf(\"expected 2 OpenLTXFile calls, got %d\", callCount)\n\t\t}\n\t})\n\n\tt.Run(\"ReadFullAcrossReconnect\", func(t *testing.T) {\n\t\t// Simulate io.ReadFull reading a 6-byte page header where the\n\t\t// connection drops after 3 bytes. This is the exact scenario from\n\t\t// the original bug: the LTX compactor calls io.ReadFull for a\n\t\t// 6-byte PageHeader, but the stream is dead.\n\t\tdata := []byte(\"ABCDEF remainder of file\")\n\t\tcallCount := 0\n\t\tclient := &testLTXFileOpener{\n\t\t\tOpenLTXFileFunc: func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\t\t\t\tcallCount++\n\t\t\t\tif callCount == 1 {\n\t\t\t\t\treturn io.NopCloser(&errorAfterN{data: data, n: 3, err: fmt.Errorf(\"connection reset\")}), nil\n\t\t\t\t}\n\t\t\t\treturn io.NopCloser(bytes.NewReader(data[offset:])), nil\n\t\t\t},\n\t\t}\n\n\t\tr := newTestResumableReader(client, int64(len(data)), data)\n\n\t\t// Read exactly 6 bytes, like the LTX decoder does for page headers.\n\t\tbuf := make([]byte, 6)\n\t\t_, err := io.ReadFull(r, buf)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"io.ReadFull failed: %v\", err)\n\t\t}\n\t\tif !bytes.Equal(buf, []byte(\"ABCDEF\")) {\n\t\t\tt.Fatalf(\"got %q, want %q\", buf, \"ABCDEF\")\n\t\t}\n\t})\n\n\tt.Run(\"MaxRetriesExceeded\", func(t *testing.T) {\n\t\t// If the connection keeps failing, the reader should give up after\n\t\t// the maximum retry count rather than looping forever.\n\t\tclient := &testLTXFileOpener{\n\t\t\tOpenLTXFileFunc: func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\t\t\t\treturn io.NopCloser(&errorAfterN{data: nil, n: 0, err: fmt.Errorf(\"persistent failure\")}), nil\n\t\t\t},\n\t\t}\n\n\t\tr := newTestResumableReader(client, 100, nil)\n\t\tbuf := make([]byte, 10)\n\t\t_, err := r.Read(buf)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error after max retries, got nil\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"max retries exceeded\") {\n\t\t\tt.Fatalf(\"expected 'max retries exceeded' error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ReopenFailure\", func(t *testing.T) {\n\t\t// If the initial stream dies and the reopen also fails (e.g., 404),\n\t\t// the error should propagate.\n\t\tdata := []byte(\"hello world\")\n\t\tcallCount := 0\n\t\tclient := &testLTXFileOpener{\n\t\t\tOpenLTXFileFunc: func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\t\t\t\tcallCount++\n\t\t\t\tif callCount == 1 {\n\t\t\t\t\treturn io.NopCloser(&errorAfterN{data: data, n: 3, err: fmt.Errorf(\"connection reset\")}), nil\n\t\t\t\t}\n\t\t\t\treturn nil, fmt.Errorf(\"file not found\")\n\t\t\t},\n\t\t}\n\n\t\tr := newTestResumableReader(client, int64(len(data)), data)\n\n\t\t// First read gets 3 bytes, then error triggers reconnect attempt.\n\t\tbuf := make([]byte, 10)\n\t\tn, err := r.Read(buf)\n\t\tif n != 3 {\n\t\t\tt.Fatalf(\"expected 3 bytes on first read, got %d\", n)\n\t\t}\n\t\t// The error is suppressed on partial reads; next call hits reopen failure.\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"expected nil error on partial read, got: %v\", err)\n\t\t}\n\n\t\t// Second read should fail with reopen error.\n\t\t_, err = r.Read(buf)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error on reopen failure, got nil\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"reopen ltx file\") {\n\t\t\tt.Fatalf(\"expected 'reopen ltx file' error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"UnknownSize\", func(t *testing.T) {\n\t\t// When file size is unknown (size=0), premature EOF cannot be detected,\n\t\t// so a clean EOF from a truncated stream is treated as legitimate.\n\t\tdata := []byte(\"hello world\")\n\t\tclient := &testLTXFileOpener{\n\t\t\tOpenLTXFileFunc: func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\t\t\t\t// Always return only first 5 bytes.\n\t\t\t\treturn io.NopCloser(bytes.NewReader(data[:5])), nil\n\t\t\t},\n\t\t}\n\n\t\tr := newTestResumableReader(client, 0 /* unknown size */, data)\n\t\tgot, err := io.ReadAll(r)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\t// Without size info, we can't detect the truncation.\n\t\tif !bytes.Equal(got, data[:5]) {\n\t\t\tt.Fatalf(\"got %q, want %q\", got, data[:5])\n\t\t}\n\t})\n\n\tt.Run(\"CorrectOffsetOnReopen\", func(t *testing.T) {\n\t\t// Verify the reader passes the correct byte offset when reopening.\n\t\tdata := []byte(\"0123456789abcdef\")\n\t\tvar reopenOffset int64\n\t\tcallCount := 0\n\t\tclient := &testLTXFileOpener{\n\t\t\tOpenLTXFileFunc: func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\t\t\t\tcallCount++\n\t\t\t\tif callCount == 1 {\n\t\t\t\t\treturn io.NopCloser(&errorAfterN{data: data, n: 7, err: fmt.Errorf(\"timeout\")}), nil\n\t\t\t\t}\n\t\t\t\treopenOffset = offset\n\t\t\t\treturn io.NopCloser(bytes.NewReader(data[offset:])), nil\n\t\t\t},\n\t\t}\n\n\t\tr := newTestResumableReader(client, int64(len(data)), data)\n\t\tgot, err := io.ReadAll(r)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !bytes.Equal(got, data) {\n\t\t\tt.Fatalf(\"got %q, want %q\", got, data)\n\t\t}\n\t\tif reopenOffset != 7 {\n\t\t\tt.Fatalf(\"reopen offset = %d, want 7\", reopenOffset)\n\t\t}\n\t})\n}\n\n// newTestResumableReader creates a resumableReader for testing. The initial\n// stream is opened from the client; data is only used for reference.\nfunc newTestResumableReader(client *testLTXFileOpener, size int64, data []byte) *ResumableReader {\n\trc, _ := client.OpenLTXFile(context.Background(), 0, 1, 1, 0, 0)\n\treturn NewResumableReader(\n\t\tcontext.Background(),\n\t\tclient,\n\t\t0, // level\n\t\t1, // minTXID\n\t\t1, // maxTXID\n\t\tsize, // expected file size\n\t\trc,\n\t\tslog.Default(),\n\t)\n}\n\ntype testLTXFileOpener struct {\n\tOpenLTXFileFunc func(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n}\n\nfunc (t *testLTXFileOpener) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\treturn t.OpenLTXFileFunc(ctx, level, minTXID, maxTXID, offset, size)\n}\n\n// errorAfterN is a reader that returns data normally for the first n bytes,\n// then returns the specified error. This simulates a connection that drops\n// mid-transfer.\ntype errorAfterN struct {\n\tdata []byte\n\tn int // bytes to return before erroring\n\tpos int\n\terr error\n}\n\nfunc (r *errorAfterN) Read(p []byte) (int, error) {\n\tif r.pos >= r.n {\n\t\treturn 0, r.err\n\t}\n\tremaining := r.n - r.pos\n\tif len(p) > remaining {\n\t\tp = p[:remaining]\n\t}\n\tn := copy(p, r.data[r.pos:r.pos+len(p)])\n\tr.pos += n\n\treturn n, nil\n}\n" }, { "path": "internal/testingutil/testingutil.go", "content": "package testingutil\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"math/rand/v2\"\n\t\"net\"\n\t\"os\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\n\tsftpserver \"github.com/pkg/sftp\"\n\t\"golang.org/x/crypto/ssh\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/abs\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/gs\"\n\t\"github.com/benbjohnson/litestream/internal\"\n\t\"github.com/benbjohnson/litestream/nats\"\n\t\"github.com/benbjohnson/litestream/oss\"\n\t\"github.com/benbjohnson/litestream/s3\"\n\t\"github.com/benbjohnson/litestream/sftp\"\n\t\"github.com/benbjohnson/litestream/webdav\"\n)\n\nconst (\n\tdefaultTigrisEndpoint = \"https://fly.storage.tigris.dev\"\n\tdefaultTigrisRegion = \"auto\"\n\tdefaultTigrisBucket = \"litestream-dev\"\n\tdefaultTigrisPathRoot = \"integration-tests\"\n)\n\nvar (\n\t// Enables integration tests.\n\tintegration = flag.Bool(\"integration\", false, \"\")\n\t// Enables specific types of replicas to be tested.\n\treplicaClientTypes = flag.String(\"replica-clients\", \"file\", \"\")\n\t// Sets the log level for the tests.\n\tlogLevel = flag.String(\"log.level\", \"debug\", \"\")\n)\n\n// S3 settings\nvar (\n\t// Replica client settings\n\ts3AccessKeyID = flag.String(\"s3-access-key-id\", os.Getenv(\"LITESTREAM_S3_ACCESS_KEY_ID\"), \"\")\n\ts3SecretAccessKey = flag.String(\"s3-secret-access-key\", os.Getenv(\"LITESTREAM_S3_SECRET_ACCESS_KEY\"), \"\")\n\ts3Region = flag.String(\"s3-region\", os.Getenv(\"LITESTREAM_S3_REGION\"), \"\")\n\ts3Bucket = flag.String(\"s3-bucket\", os.Getenv(\"LITESTREAM_S3_BUCKET\"), \"\")\n\ts3Path = flag.String(\"s3-path\", os.Getenv(\"LITESTREAM_S3_PATH\"), \"\")\n\ts3Endpoint = flag.String(\"s3-endpoint\", os.Getenv(\"LITESTREAM_S3_ENDPOINT\"), \"\")\n\ts3ForcePathStyle = flag.Bool(\"s3-force-path-style\", os.Getenv(\"LITESTREAM_S3_FORCE_PATH_STYLE\") == \"true\", \"\")\n\ts3SkipVerify = flag.Bool(\"s3-skip-verify\", os.Getenv(\"LITESTREAM_S3_SKIP_VERIFY\") == \"true\", \"\")\n)\n\n// Tigris settings (S3-compatible)\nvar (\n\ttigrisAccessKeyID = flag.String(\"tigris-access-key-id\", os.Getenv(\"LITESTREAM_TIGRIS_ACCESS_KEY_ID\"), \"\")\n\ttigrisSecretAccessKey = flag.String(\"tigris-secret-access-key\", os.Getenv(\"LITESTREAM_TIGRIS_SECRET_ACCESS_KEY\"), \"\")\n)\n\n// Cloudflare R2 settings (S3-compatible)\nvar (\n\tr2AccessKeyID = flag.String(\"r2-access-key-id\", os.Getenv(\"LITESTREAM_R2_ACCESS_KEY_ID\"), \"\")\n\tr2SecretAccessKey = flag.String(\"r2-secret-access-key\", os.Getenv(\"LITESTREAM_R2_SECRET_ACCESS_KEY\"), \"\")\n\tr2Endpoint = flag.String(\"r2-endpoint\", os.Getenv(\"LITESTREAM_R2_ENDPOINT\"), \"\")\n\tr2Bucket = flag.String(\"r2-bucket\", os.Getenv(\"LITESTREAM_R2_BUCKET\"), \"\")\n)\n\n// Backblaze B2 settings (S3-compatible)\nvar (\n\tb2KeyID = flag.String(\"b2-key-id\", os.Getenv(\"LITESTREAM_B2_KEY_ID\"), \"\")\n\tb2ApplicationKey = flag.String(\"b2-application-key\", os.Getenv(\"LITESTREAM_B2_APPLICATION_KEY\"), \"\")\n\tb2Endpoint = flag.String(\"b2-endpoint\", os.Getenv(\"LITESTREAM_B2_ENDPOINT\"), \"\")\n\tb2Bucket = flag.String(\"b2-bucket\", os.Getenv(\"LITESTREAM_B2_BUCKET\"), \"\")\n)\n\n// Google cloud storage settings\nvar (\n\tgsBucket = flag.String(\"gs-bucket\", os.Getenv(\"LITESTREAM_GS_BUCKET\"), \"\")\n\tgsPath = flag.String(\"gs-path\", os.Getenv(\"LITESTREAM_GS_PATH\"), \"\")\n)\n\n// Azure blob storage settings\nvar (\n\tabsAccountName = flag.String(\"abs-account-name\", os.Getenv(\"LITESTREAM_ABS_ACCOUNT_NAME\"), \"\")\n\tabsAccountKey = flag.String(\"abs-account-key\", os.Getenv(\"LITESTREAM_ABS_ACCOUNT_KEY\"), \"\")\n\tabsSASToken = flag.String(\"abs-sas-token\", os.Getenv(\"LITESTREAM_ABS_SAS_TOKEN\"), \"\")\n\tabsBucket = flag.String(\"abs-bucket\", os.Getenv(\"LITESTREAM_ABS_BUCKET\"), \"\")\n\tabsPath = flag.String(\"abs-path\", os.Getenv(\"LITESTREAM_ABS_PATH\"), \"\")\n)\n\n// SFTP settings\nvar (\n\tsftpHost = flag.String(\"sftp-host\", os.Getenv(\"LITESTREAM_SFTP_HOST\"), \"\")\n\tsftpUser = flag.String(\"sftp-user\", os.Getenv(\"LITESTREAM_SFTP_USER\"), \"\")\n\tsftpPassword = flag.String(\"sftp-password\", os.Getenv(\"LITESTREAM_SFTP_PASSWORD\"), \"\")\n\tsftpKeyPath = flag.String(\"sftp-key-path\", os.Getenv(\"LITESTREAM_SFTP_KEY_PATH\"), \"\")\n\tsftpPath = flag.String(\"sftp-path\", os.Getenv(\"LITESTREAM_SFTP_PATH\"), \"\")\n)\n\n// WebDAV settings\nvar (\n\twebdavURL = flag.String(\"webdav-url\", os.Getenv(\"LITESTREAM_WEBDAV_URL\"), \"\")\n\twebdavUsername = flag.String(\"webdav-username\", os.Getenv(\"LITESTREAM_WEBDAV_USERNAME\"), \"\")\n\twebdavPassword = flag.String(\"webdav-password\", os.Getenv(\"LITESTREAM_WEBDAV_PASSWORD\"), \"\")\n\twebdavPath = flag.String(\"webdav-path\", os.Getenv(\"LITESTREAM_WEBDAV_PATH\"), \"\")\n)\n\n// NATS settings\nvar (\n\tnatsURL = flag.String(\"nats-url\", os.Getenv(\"LITESTREAM_NATS_URL\"), \"\")\n\tnatsBucket = flag.String(\"nats-bucket\", os.Getenv(\"LITESTREAM_NATS_BUCKET\"), \"\")\n\tnatsCreds = flag.String(\"nats-creds\", os.Getenv(\"LITESTREAM_NATS_CREDS\"), \"\")\n\tnatsUsername = flag.String(\"nats-username\", os.Getenv(\"LITESTREAM_NATS_USERNAME\"), \"\")\n\tnatsPassword = flag.String(\"nats-password\", os.Getenv(\"LITESTREAM_NATS_PASSWORD\"), \"\")\n)\n\n// Alibaba Cloud OSS settings\nvar (\n\tossAccessKeyID = flag.String(\"oss-access-key-id\", os.Getenv(\"LITESTREAM_OSS_ACCESS_KEY_ID\"), \"\")\n\tossAccessKeySecret = flag.String(\"oss-access-key-secret\", os.Getenv(\"LITESTREAM_OSS_ACCESS_KEY_SECRET\"), \"\")\n\tossRegion = flag.String(\"oss-region\", os.Getenv(\"LITESTREAM_OSS_REGION\"), \"\")\n\tossBucket = flag.String(\"oss-bucket\", os.Getenv(\"LITESTREAM_OSS_BUCKET\"), \"\")\n\tossPath = flag.String(\"oss-path\", os.Getenv(\"LITESTREAM_OSS_PATH\"), \"\")\n\tossEndpoint = flag.String(\"oss-endpoint\", os.Getenv(\"LITESTREAM_OSS_ENDPOINT\"), \"\")\n)\n\nfunc Integration() bool {\n\treturn *integration\n}\n\nfunc ReplicaClientTypes() []string {\n\treturn strings.Split(*replicaClientTypes, \",\")\n}\n\nfunc NewDB(tb testing.TB, path string) *litestream.DB {\n\ttb.Helper()\n\ttb.Logf(\"db=%s\", path)\n\n\tlevel := slog.LevelDebug\n\tif strings.EqualFold(*logLevel, \"trace\") {\n\t\tlevel = internal.LevelTrace\n\t}\n\n\tdb := litestream.NewDB(path)\n\tdb.Logger = slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{\n\t\tLevel: level,\n\t\tReplaceAttr: internal.ReplaceAttr,\n\t}))\n\treturn db\n}\n\n// MustOpenDBs returns a new instance of a DB & associated SQL DB.\nfunc MustOpenDBs(tb testing.TB) (*litestream.DB, *sql.DB) {\n\ttb.Helper()\n\tdb := MustOpenDB(tb)\n\treturn db, MustOpenSQLDB(tb, db.Path())\n}\n\n// MustCloseDBs closes db & sqldb and removes the parent directory.\nfunc MustCloseDBs(tb testing.TB, db *litestream.DB, sqldb *sql.DB) {\n\ttb.Helper()\n\tMustCloseDB(tb, db)\n\tMustCloseSQLDB(tb, sqldb)\n}\n\n// MustOpenDB returns a new instance of a DB.\nfunc MustOpenDB(tb testing.TB) *litestream.DB {\n\ttb.Helper()\n\tdir := tb.TempDir()\n\treturn MustOpenDBAt(tb, filepath.Join(dir, \"db\"))\n}\n\n// MustOpenDBAt returns a new instance of a DB for a given path.\nfunc MustOpenDBAt(tb testing.TB, path string) *litestream.DB {\n\ttb.Helper()\n\tdb := NewDB(tb, path)\n\tdb.MonitorInterval = 0 // disable background goroutine\n\tdb.ShutdownSyncTimeout = 0 // disable shutdown sync retry for faster tests\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = NewFileReplicaClient(tb)\n\tdb.Replica.MonitorEnabled = false // disable background goroutine\n\tif err := db.Open(); err != nil {\n\t\ttb.Fatal(err)\n\t}\n\treturn db\n}\n\n// MustCloseDB closes db and removes its parent directory.\nfunc MustCloseDB(tb testing.TB, db *litestream.DB) {\n\ttb.Helper()\n\tif err := db.Close(context.Background()); err != nil && !strings.Contains(err.Error(), `database is closed`) && !strings.Contains(err.Error(), `file already closed`) {\n\t\ttb.Fatal(err)\n\t} else if err := os.RemoveAll(filepath.Dir(db.Path())); err != nil {\n\t\ttb.Fatal(err)\n\t}\n}\n\n// MustOpenSQLDB returns a database/sql DB.\nfunc MustOpenSQLDB(tb testing.TB, path string) *sql.DB {\n\ttb.Helper()\n\td, err := sql.Open(\"sqlite\", path)\n\tif err != nil {\n\t\ttb.Fatal(err)\n\t} else if _, err := d.ExecContext(context.Background(), `PRAGMA journal_mode = wal;`); err != nil {\n\t\ttb.Fatal(err)\n\t} else if _, err := d.ExecContext(context.Background(), `PRAGMA busy_timeout = 5000;`); err != nil {\n\t\ttb.Fatal(err)\n\t}\n\treturn d\n}\n\n// MustCloseSQLDB closes a database/sql DB.\nfunc MustCloseSQLDB(tb testing.TB, d *sql.DB) {\n\ttb.Helper()\n\tif err := d.Close(); err != nil {\n\t\ttb.Fatal(err)\n\t}\n}\n\n// NewReplicaClient returns a new client for integration testing by type name.\nfunc NewReplicaClient(tb testing.TB, typ string) litestream.ReplicaClient {\n\ttb.Helper()\n\n\tswitch typ {\n\tcase file.ReplicaClientType:\n\t\treturn NewFileReplicaClient(tb)\n\tcase s3.ReplicaClientType:\n\t\treturn NewS3ReplicaClient(tb)\n\tcase gs.ReplicaClientType:\n\t\treturn NewGSReplicaClient(tb)\n\tcase abs.ReplicaClientType:\n\t\treturn NewABSReplicaClient(tb)\n\tcase sftp.ReplicaClientType:\n\t\treturn NewSFTPReplicaClient(tb)\n\tcase webdav.ReplicaClientType:\n\t\treturn NewWebDAVReplicaClient(tb)\n\tcase nats.ReplicaClientType:\n\t\treturn NewNATSReplicaClient(tb)\n\tcase oss.ReplicaClientType:\n\t\treturn NewOSSReplicaClient(tb)\n\tcase \"tigris\":\n\t\treturn NewTigrisReplicaClient(tb)\n\tcase \"r2\":\n\t\treturn NewR2ReplicaClient(tb)\n\tcase \"b2\":\n\t\treturn NewB2ReplicaClient(tb)\n\tdefault:\n\t\ttb.Fatalf(\"invalid replica client type: %q\", typ)\n\t\treturn nil\n\t}\n}\n\n// NewFileReplicaClient returns a new client for integration testing.\nfunc NewFileReplicaClient(tb testing.TB) *file.ReplicaClient {\n\ttb.Helper()\n\treturn file.NewReplicaClient(tb.TempDir())\n}\n\n// NewS3ReplicaClient returns a new client for integration testing.\nfunc NewS3ReplicaClient(tb testing.TB) *s3.ReplicaClient {\n\ttb.Helper()\n\n\tc := s3.NewReplicaClient()\n\tc.AccessKeyID = *s3AccessKeyID\n\tc.SecretAccessKey = *s3SecretAccessKey\n\tc.Region = *s3Region\n\tc.Bucket = *s3Bucket\n\tc.Path = path.Join(*s3Path, fmt.Sprintf(\"%016x\", rand.Uint64()))\n\tc.Endpoint = *s3Endpoint\n\tc.ForcePathStyle = *s3ForcePathStyle\n\tc.SkipVerify = *s3SkipVerify\n\treturn c\n}\n\n// NewTigrisReplicaClient returns an S3 client configured for Fly.io Tigris.\nfunc NewTigrisReplicaClient(tb testing.TB) *s3.ReplicaClient {\n\ttb.Helper()\n\n\tif *tigrisAccessKeyID == \"\" || *tigrisSecretAccessKey == \"\" {\n\t\ttb.Skip(\"tigris credentials not configured (set LITESTREAM_TIGRIS_ACCESS_KEY_ID/SECRET_ACCESS_KEY)\")\n\t}\n\n\tc := s3.NewReplicaClient()\n\tc.AccessKeyID = *tigrisAccessKeyID\n\tc.SecretAccessKey = *tigrisSecretAccessKey\n\tc.Region = defaultTigrisRegion\n\tc.Bucket = defaultTigrisBucket\n\tc.Path = path.Join(defaultTigrisPathRoot, fmt.Sprintf(\"%016x\", rand.Uint64()))\n\tc.Endpoint = defaultTigrisEndpoint\n\tc.ForcePathStyle = true\n\tc.RequireContentMD5 = false\n\treturn c\n}\n\n// NewR2ReplicaClient returns an S3 client configured for Cloudflare R2.\n// Skips the test if R2 credentials are not configured.\nfunc NewR2ReplicaClient(tb testing.TB) *s3.ReplicaClient {\n\ttb.Helper()\n\n\tif *r2AccessKeyID == \"\" || *r2SecretAccessKey == \"\" {\n\t\ttb.Skip(\"r2 credentials not configured (set LITESTREAM_R2_ACCESS_KEY_ID/SECRET_ACCESS_KEY)\")\n\t}\n\tif *r2Endpoint == \"\" {\n\t\ttb.Skip(\"r2 endpoint not configured (set LITESTREAM_R2_ENDPOINT)\")\n\t}\n\tif *r2Bucket == \"\" {\n\t\ttb.Skip(\"r2 bucket not configured (set LITESTREAM_R2_BUCKET)\")\n\t}\n\n\tc := s3.NewReplicaClient()\n\tc.AccessKeyID = *r2AccessKeyID\n\tc.SecretAccessKey = *r2SecretAccessKey\n\tc.Region = \"auto\"\n\tc.Bucket = *r2Bucket\n\tc.Path = path.Join(\"integration-tests\", fmt.Sprintf(\"%016x\", rand.Uint64()))\n\tc.Endpoint = *r2Endpoint\n\tc.ForcePathStyle = true\n\tc.SignPayload = true\n\treturn c\n}\n\n// NewB2ReplicaClient returns a new Backblaze B2 client for integration testing.\n// B2 uses S3-compatible API with path-style URLs and signed payloads.\nfunc NewB2ReplicaClient(tb testing.TB) *s3.ReplicaClient {\n\ttb.Helper()\n\n\tif *b2KeyID == \"\" || *b2ApplicationKey == \"\" {\n\t\ttb.Skip(\"b2 credentials not configured (set LITESTREAM_B2_KEY_ID/APPLICATION_KEY)\")\n\t}\n\tif *b2Endpoint == \"\" {\n\t\ttb.Skip(\"b2 endpoint not configured (set LITESTREAM_B2_ENDPOINT)\")\n\t}\n\tif *b2Bucket == \"\" {\n\t\ttb.Skip(\"b2 bucket not configured (set LITESTREAM_B2_BUCKET)\")\n\t}\n\n\tc := s3.NewReplicaClient()\n\tc.AccessKeyID = *b2KeyID\n\tc.SecretAccessKey = *b2ApplicationKey\n\tc.Region = \"us-west-002\" // B2 uses region in endpoint format\n\tc.Bucket = *b2Bucket\n\tc.Path = path.Join(\"integration-tests\", fmt.Sprintf(\"%016x\", rand.Uint64()))\n\tc.Endpoint = *b2Endpoint\n\tc.ForcePathStyle = true\n\tc.SignPayload = true\n\treturn c\n}\n\n// NewGSReplicaClient returns a new client for integration testing.\nfunc NewGSReplicaClient(tb testing.TB) *gs.ReplicaClient {\n\ttb.Helper()\n\n\t// Log basic diagnostic information for integration test troubleshooting\n\ttb.Logf(\"GCS Integration Test Setup:\")\n\tcredsSet := \"not set\"\n\tif os.Getenv(\"GOOGLE_APPLICATION_CREDENTIALS\") != \"\" {\n\t\tcredsSet = \"set\"\n\t}\n\ttb.Logf(\" GOOGLE_APPLICATION_CREDENTIALS: %s\", credsSet)\n\ttb.Logf(\" LITESTREAM_GS_BUCKET: %s\", *gsBucket)\n\ttb.Logf(\" LITESTREAM_GS_PATH: %s\", *gsPath)\n\n\tc := gs.NewReplicaClient()\n\tc.Bucket = *gsBucket\n\tc.Path = path.Join(*gsPath, fmt.Sprintf(\"%016x\", rand.Uint64()))\n\n\t// Test basic connectivity\n\tctx := context.Background()\n\tif err := c.Init(ctx); err != nil {\n\t\ttb.Logf(\"GCS client initialization failed: %v\", err)\n\t\ttb.Logf(\"This may indicate credential or project issues\")\n\t\treturn c // Return anyway to let the actual test show the detailed error\n\t}\n\ttb.Logf(\"GCS client initialized successfully\")\n\n\treturn c\n}\n\n// NewABSReplicaClient returns a new client for integration testing.\nfunc NewABSReplicaClient(tb testing.TB) *abs.ReplicaClient {\n\ttb.Helper()\n\n\tc := abs.NewReplicaClient()\n\tc.AccountName = *absAccountName\n\tc.AccountKey = *absAccountKey\n\tc.SASToken = *absSASToken\n\tc.Bucket = *absBucket\n\tc.Path = path.Join(*absPath, fmt.Sprintf(\"%016x\", rand.Uint64()))\n\treturn c\n}\n\n// NewSFTPReplicaClient returns a new client for integration testing.\nfunc NewSFTPReplicaClient(tb testing.TB) *sftp.ReplicaClient {\n\ttb.Helper()\n\n\tc := sftp.NewReplicaClient()\n\tc.Host = *sftpHost\n\tc.User = *sftpUser\n\tc.Password = *sftpPassword\n\tc.KeyPath = *sftpKeyPath\n\tc.Path = path.Join(*sftpPath, fmt.Sprintf(\"%016x\", rand.Uint64()))\n\treturn c\n}\n\n// NewWebDAVReplicaClient returns a new client for integration testing.\nfunc NewWebDAVReplicaClient(tb testing.TB) *webdav.ReplicaClient {\n\ttb.Helper()\n\n\tc := webdav.NewReplicaClient()\n\tc.URL = *webdavURL\n\tc.Username = *webdavUsername\n\tc.Password = *webdavPassword\n\tc.Path = path.Join(*webdavPath, fmt.Sprintf(\"%016x\", rand.Uint64()))\n\treturn c\n}\n\n// NewNATSReplicaClient returns a new client for integration testing.\nfunc NewNATSReplicaClient(tb testing.TB) *nats.ReplicaClient {\n\ttb.Helper()\n\n\tc := nats.NewReplicaClient()\n\tc.URL = *natsURL\n\tc.BucketName = *natsBucket\n\tc.Creds = *natsCreds\n\tc.Username = *natsUsername\n\tc.Password = *natsPassword\n\treturn c\n}\n\n// NewOSSReplicaClient returns a new client for integration testing.\nfunc NewOSSReplicaClient(tb testing.TB) *oss.ReplicaClient {\n\ttb.Helper()\n\n\tc := oss.NewReplicaClient()\n\tc.AccessKeyID = *ossAccessKeyID\n\tc.AccessKeySecret = *ossAccessKeySecret\n\tc.Region = *ossRegion\n\tc.Bucket = *ossBucket\n\tc.Path = path.Join(*ossPath, fmt.Sprintf(\"%016x\", rand.Uint64()))\n\tc.Endpoint = *ossEndpoint\n\treturn c\n}\n\n// MustDeleteAll deletes all objects under the client's path.\nfunc MustDeleteAll(tb testing.TB, c litestream.ReplicaClient) {\n\ttb.Helper()\n\n\tif err := c.DeleteAll(context.Background()); err != nil {\n\t\ttb.Fatalf(\"cannot delete all: %s\", err)\n\t}\n\n\tswitch c := c.(type) {\n\tcase *sftp.ReplicaClient:\n\t\tif err := c.Cleanup(context.Background()); err != nil {\n\t\t\ttb.Fatalf(\"cannot cleanup sftp: %s\", err)\n\t\t}\n\t}\n}\n\nfunc MockSFTPServer(t *testing.T, hostKey ssh.Signer) string {\n\tconfig := &ssh.ServerConfig{NoClientAuth: true}\n\tconfig.AddHostKey(hostKey)\n\n\tlistener, err := net.Listen(\"tcp\", \"127.0.0.1:0\") // random available port\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tgo func() {\n\t\tfor {\n\t\t\tconn, err := listener.Accept()\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tgo func() {\n\t\t\t\t_, chans, reqs, err := ssh.NewServerConn(conn, config)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tgo ssh.DiscardRequests(reqs)\n\n\t\t\t\tfor ch := range chans {\n\t\t\t\t\tif ch.ChannelType() != \"session\" {\n\t\t\t\t\t\tch.Reject(ssh.UnknownChannelType, \"unsupported\")\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tchannel, requests, err := ch.Accept()\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\n\t\t\t\t\tgo func(in <-chan *ssh.Request) {\n\t\t\t\t\t\tfor req := range in {\n\t\t\t\t\t\t\tif req.Type == \"subsystem\" && string(req.Payload[4:]) == \"sftp\" {\n\t\t\t\t\t\t\t\treq.Reply(true, nil)\n\n\t\t\t\t\t\t\t\tserver, err := sftpserver.NewServer(channel)\n\t\t\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\tif err := server.Serve(); err != nil && err != io.EOF {\n\t\t\t\t\t\t\t\t\tt.Logf(\"SFTP server error: %v\", err)\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\treq.Reply(false, nil)\n\t\t\t\t\t\t}\n\t\t\t\t\t}(requests)\n\t\t\t\t}\n\t\t\t}()\n\t\t}\n\t}()\n\n\treturn listener.Addr().String()\n}\n" }, { "path": "leaser.go", "content": "package litestream\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"time\"\n)\n\nvar ErrLeaseNotHeld = errors.New(\"lease not held\")\n\ntype LeaseExistsError struct {\n\tOwner string\n\tExpiresAt time.Time\n}\n\nfunc (e *LeaseExistsError) Error() string {\n\tif e.Owner != \"\" {\n\t\treturn fmt.Sprintf(\"lease already held by %s until %s\", e.Owner, e.ExpiresAt.Format(time.RFC3339))\n\t}\n\treturn fmt.Sprintf(\"lease already held until %s\", e.ExpiresAt.Format(time.RFC3339))\n}\n\ntype Leaser interface {\n\tType() string\n\tAcquireLease(ctx context.Context) (*Lease, error)\n\tRenewLease(ctx context.Context, lease *Lease) (*Lease, error)\n\tReleaseLease(ctx context.Context, lease *Lease) error\n}\n\ntype Lease struct {\n\tGeneration int64 `json:\"generation\"`\n\tExpiresAt time.Time `json:\"expires_at\"`\n\tOwner string `json:\"owner,omitempty\"`\n\tETag string `json:\"-\"`\n}\n\nfunc (l *Lease) IsExpired() bool {\n\treturn time.Now().After(l.ExpiresAt)\n}\n\nfunc (l *Lease) TTL() time.Duration {\n\treturn time.Until(l.ExpiresAt)\n}\n" }, { "path": "litestream.go", "content": "package litestream\n\nimport (\n\t\"database/sql\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"io\"\n\t\"os\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/superfly/ltx\"\n\t_ \"modernc.org/sqlite\"\n)\n\n// Naming constants.\nconst (\n\tMetaDirSuffix = \"-litestream\"\n)\n\n// SQLite checkpoint modes.\nconst (\n\tCheckpointModePassive = \"PASSIVE\"\n\tCheckpointModeFull = \"FULL\"\n\tCheckpointModeRestart = \"RESTART\"\n\tCheckpointModeTruncate = \"TRUNCATE\"\n)\n\n// Litestream errors.\nvar (\n\tErrNoSnapshots = errors.New(\"no snapshots available\")\n\tErrChecksumMismatch = errors.New(\"invalid replica, checksum mismatch\")\n\tErrLTXCorrupted = errors.New(\"ltx file corrupted\")\n\tErrLTXMissing = errors.New(\"ltx file missing\")\n)\n\n// LTXError provides detailed context for LTX file errors with recovery hints.\ntype LTXError struct {\n\tOp string // Operation that failed (e.g., \"open\", \"read\", \"validate\")\n\tPath string // File path\n\tLevel int // LTX level (0 = L0, etc.)\n\tMinTXID uint64 // Minimum transaction ID\n\tMaxTXID uint64 // Maximum transaction ID\n\tErr error // Underlying error\n\tHint string // Recovery hint for users\n}\n\nfunc (e *LTXError) Error() string {\n\tif e.Path != \"\" {\n\t\treturn e.Op + \" ltx file \" + e.Path + \": \" + e.Err.Error()\n\t}\n\treturn e.Op + \" ltx file: \" + e.Err.Error()\n}\n\nfunc (e *LTXError) Unwrap() error { return e.Err }\n\n// NewLTXError creates a new LTX error with appropriate hints based on the error type.\nfunc NewLTXError(op, path string, level int, minTXID, maxTXID uint64, err error) *LTXError {\n\tltxErr := <XError{\n\t\tOp: op,\n\t\tPath: path,\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tErr: err,\n\t}\n\n\t// Set appropriate hint based on error type\n\tif os.IsNotExist(err) || errors.Is(err, ErrLTXMissing) {\n\t\tltxErr.Hint = \"LTX file is missing. This can happen after VACUUM, manual checkpoint, or state corruption. \" +\n\t\t\t\"Run 'litestream reset ' or delete the .sqlite-litestream directory and restart.\"\n\t} else if errors.Is(err, ErrLTXCorrupted) || errors.Is(err, ErrChecksumMismatch) {\n\t\tltxErr.Hint = \"LTX file is corrupted. Delete the .sqlite-litestream directory and restart to recover from replica.\"\n\t}\n\n\treturn ltxErr\n}\n\n// SQLite WAL constants.\nconst (\n\tWALHeaderChecksumOffset = 24\n\tWALFrameHeaderChecksumOffset = 16\n)\n\nvar (\n\t// LogWriter is the destination writer for all logging.\n\tLogWriter = os.Stdout\n\n\t// LogFlags are the flags passed to log.New().\n\tLogFlags = 0\n)\n\n// Checksum computes a running SQLite checksum over a byte slice.\nfunc Checksum(bo binary.ByteOrder, s0, s1 uint32, b []byte) (uint32, uint32) {\n\tassert(len(b)%8 == 0, \"misaligned checksum byte slice\")\n\n\t// Iterate over 8-byte units and compute checksum.\n\tfor i := 0; i < len(b); i += 8 {\n\t\ts0 += bo.Uint32(b[i:]) + s1\n\t\ts1 += bo.Uint32(b[i+4:]) + s0\n\t}\n\treturn s0, s1\n}\n\nconst (\n\t// WALHeaderSize is the size of the WAL header, in bytes.\n\tWALHeaderSize = 32\n\n\t// WALFrameHeaderSize is the size of the WAL frame header, in bytes.\n\tWALFrameHeaderSize = 24\n)\n\n// rollback rolls back tx. Ignores already-rolled-back errors.\nfunc rollback(tx *sql.Tx) error {\n\tif err := tx.Rollback(); err != nil && !strings.Contains(err.Error(), `transaction has already been committed or rolled back`) {\n\t\treturn err\n\t}\n\treturn nil\n}\n\n// readWALHeader returns the header read from a WAL file.\nfunc readWALHeader(filename string) ([]byte, error) {\n\tf, err := os.Open(filename)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer f.Close()\n\n\tbuf := make([]byte, WALHeaderSize)\n\tn, err := io.ReadFull(f, buf)\n\treturn buf[:n], err\n}\n\n// readWALFileAt reads a slice from a file. Do not use this with database files\n// as it causes problems with non-OFD locks.\nfunc readWALFileAt(filename string, offset, n int64) ([]byte, error) {\n\tf, err := os.Open(filename)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer f.Close()\n\n\tbuf := make([]byte, n)\n\tif n, err := f.ReadAt(buf, offset); err != nil {\n\t\treturn buf[:n], err\n\t} else if n < len(buf) {\n\t\treturn buf[:n], io.ErrUnexpectedEOF\n\t}\n\treturn buf, nil\n}\n\n// removeTmpFiles recursively finds and removes .tmp files.\nfunc removeTmpFiles(root string) error {\n\treturn filepath.Walk(root, func(path string, info os.FileInfo, err error) error {\n\t\tswitch {\n\t\tcase err != nil:\n\t\t\treturn nil // skip errored files\n\t\tcase info.IsDir():\n\t\t\treturn nil // skip directories\n\t\tcase !strings.HasSuffix(path, \".tmp\"):\n\t\t\treturn nil // skip non-temp files\n\t\tdefault:\n\t\t\treturn os.Remove(path)\n\t\t}\n\t})\n}\n\n// LTXDir returns the path to an LTX directory.\nfunc LTXDir(root string) string {\n\treturn path.Join(root, \"ltx\")\n}\n\n// LTXLevelDir returns the path to an LTX level directory.\nfunc LTXLevelDir(root string, level int) string {\n\treturn path.Join(LTXDir(root), strconv.Itoa(level))\n}\n\n// LTXFilePath returns the path to a single LTX file.\nfunc LTXFilePath(root string, level int, minTXID, maxTXID ltx.TXID) string {\n\treturn path.Join(LTXLevelDir(root, level), ltx.FormatFilename(minTXID, maxTXID))\n}\n\nfunc assert(condition bool, message string) {\n\tif !condition {\n\t\tpanic(\"assertion failed: \" + message)\n\t}\n}\n" }, { "path": "litestream_test.go", "content": "package litestream_test\n\nimport (\n\t\"encoding/binary\"\n\t\"encoding/hex\"\n\t\"errors\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/superfly/ltx\"\n\t_ \"modernc.org/sqlite\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nfunc TestChecksum(t *testing.T) {\n\t// Ensure a WAL header, frame header, & frame data can be checksummed in one pass.\n\tt.Run(\"OnePass\", func(t *testing.T) {\n\t\tinput, err := hex.DecodeString(\"377f0682002de218000010000000000052382eac857b1a4e00000002000000020d000000080fe0000ffc0ff80ff40ff00fec0fe80fe40fe000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000208020902070209020602090205020902040209020302090202020902010209\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\ts0, s1 := litestream.Checksum(binary.LittleEndian, 0, 0, input)\n\t\tif got, want := [2]uint32{s0, s1}, [2]uint32{0xdc2f3e84, 0x540488d3}; got != want {\n\t\t\tt.Fatalf(\"Checksum()=%x, want %x\", got, want)\n\t\t}\n\t})\n\n\t// Ensure we get the same result as OnePass even if we split up into multiple calls.\n\tt.Run(\"Incremental\", func(t *testing.T) {\n\t\t// Compute checksum for beginning of WAL header.\n\t\ts0, s1 := litestream.Checksum(binary.LittleEndian, 0, 0, MustDecodeHexString(\"377f0682002de218000010000000000052382eac857b1a4e\"))\n\t\tif got, want := [2]uint32{s0, s1}, [2]uint32{0x81153b65, 0x87178e8f}; got != want {\n\t\t\tt.Fatalf(\"Checksum()=%x, want %x\", got, want)\n\t\t}\n\n\t\t// Continue checksum with WAL frame header & frame contents.\n\t\ts0a, s1a := litestream.Checksum(binary.LittleEndian, s0, s1, MustDecodeHexString(\"0000000200000002\"))\n\t\ts0b, s1b := litestream.Checksum(binary.LittleEndian, s0a, s1a, MustDecodeHexString(`0d000000080fe0000ffc0ff80ff40ff00fec0fe80fe40fe000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000208020902070209020602090205020902040209020302090202020902010209`))\n\t\tif got, want := [2]uint32{s0b, s1b}, [2]uint32{0xdc2f3e84, 0x540488d3}; got != want {\n\t\t\tt.Fatalf(\"Checksum()=%x, want %x\", got, want)\n\t\t}\n\t})\n}\nfunc TestLTXDir(t *testing.T) {\n\tif got, want := litestream.LTXDir(\"foo\"), \"foo/ltx\"; got != want {\n\t\tt.Fatalf(\"LTXDir()=%v, want %v\", got, want)\n\t}\n}\n\nfunc TestLTXLevelDir(t *testing.T) {\n\tif got, want := litestream.LTXLevelDir(\"foo\", 0), \"foo/ltx/0\"; got != want {\n\t\tt.Fatalf(\"LTXLevelDir()=%v, want %v\", got, want)\n\t}\n}\n\nfunc LTXFilePath(t *testing.T) {\n\tt.Helper()\n\tif got, want := litestream.LTXFilePath(\"foo\", 0, ltx.TXID(100), ltx.TXID(200)), \"-\"; got != want {\n\t\tt.Fatalf(\"LTXPath()=%v, want %v\", got, want)\n\t}\n}\n\nfunc MustDecodeHexString(s string) []byte {\n\tb, err := hex.DecodeString(s)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn b\n}\n\nfunc TestNewLTXError(t *testing.T) {\n\tt.Run(\"MissingFile\", func(t *testing.T) {\n\t\terr := litestream.NewLTXError(\"open\", \"/path/to/file.ltx\", 0, 1, 1, os.ErrNotExist)\n\t\tif err.Hint == \"\" {\n\t\t\tt.Fatal(\"expected hint for missing file error\")\n\t\t}\n\t\tif !strings.Contains(err.Hint, \"missing\") {\n\t\t\tt.Errorf(\"hint should mention missing file, got: %s\", err.Hint)\n\t\t}\n\t\tif !strings.Contains(err.Hint, \"litestream reset\") {\n\t\t\tt.Errorf(\"hint should mention reset command, got: %s\", err.Hint)\n\t\t}\n\t})\n\n\tt.Run(\"CorruptedFile\", func(t *testing.T) {\n\t\terr := litestream.NewLTXError(\"decode\", \"/path/to/file.ltx\", 0, 1, 1, litestream.ErrLTXCorrupted)\n\t\tif err.Hint == \"\" {\n\t\t\tt.Fatal(\"expected hint for corrupted file error\")\n\t\t}\n\t\tif !strings.Contains(err.Hint, \"corrupted\") {\n\t\t\tt.Errorf(\"hint should mention corruption, got: %s\", err.Hint)\n\t\t}\n\t})\n\n\tt.Run(\"ChecksumMismatch\", func(t *testing.T) {\n\t\terr := litestream.NewLTXError(\"validate\", \"/path/to/file.ltx\", 0, 1, 1, litestream.ErrChecksumMismatch)\n\t\tif err.Hint == \"\" {\n\t\t\tt.Fatal(\"expected hint for checksum mismatch error\")\n\t\t}\n\t})\n\n\tt.Run(\"ErrorString\", func(t *testing.T) {\n\t\terr := litestream.NewLTXError(\"open\", \"/path/to/file.ltx\", 0, 1, 1, os.ErrNotExist)\n\t\terrStr := err.Error()\n\t\tif !strings.Contains(errStr, \"open\") {\n\t\t\tt.Errorf(\"error should contain operation, got: %s\", errStr)\n\t\t}\n\t\tif !strings.Contains(errStr, \"/path/to/file.ltx\") {\n\t\t\tt.Errorf(\"error should contain path, got: %s\", errStr)\n\t\t}\n\t})\n\n\tt.Run(\"Unwrap\", func(t *testing.T) {\n\t\tunderlying := errors.New(\"underlying error\")\n\t\terr := litestream.NewLTXError(\"read\", \"/path/to/file.ltx\", 0, 1, 1, underlying)\n\t\tif !errors.Is(err, underlying) {\n\t\t\tt.Error(\"LTXError should unwrap to underlying error\")\n\t\t}\n\t})\n}\n\nfunc TestLTXErrorHints(t *testing.T) {\n\t// Test that ErrLTXMissing also triggers appropriate hints\n\tt.Run(\"ErrLTXMissing\", func(t *testing.T) {\n\t\terr := litestream.NewLTXError(\"open\", \"/path/to/file.ltx\", 0, 1, 1, litestream.ErrLTXMissing)\n\t\tif err.Hint == \"\" {\n\t\t\tt.Fatal(\"expected hint for ErrLTXMissing\")\n\t\t}\n\t\tif !strings.Contains(err.Hint, \"litestream reset\") {\n\t\t\tt.Errorf(\"hint should mention reset command, got: %s\", err.Hint)\n\t\t}\n\t})\n}\n" }, { "path": "llms.txt", "content": "# Litestream\n\nDisaster recovery tool for SQLite. Replicates WAL changes to S3, GCS, Azure, SFTP, or local filesystem.\n\n## Quick Start for AI Contributors\n\n1. Read [AI_PR_GUIDE.md](AI_PR_GUIDE.md) - PR quality requirements\n2. Read [AGENTS.md](AGENTS.md) - Project overview and checklist\n3. Check [CONTRIBUTING.md](CONTRIBUTING.md) - What we accept\n4. Show investigation evidence in PRs\n\n## PR Checklist\n\n- [ ] Evidence of problem (logs, file patterns)\n- [ ] Clear scope (what PR does/doesn't do)\n- [ ] Runnable test commands\n- [ ] Race detector tested (`go test -race`)\n\n## Documentation\n\n| Document | Purpose |\n|----------|---------|\n| [AGENTS.md](AGENTS.md) | Project overview, critical rules |\n| [AI_PR_GUIDE.md](AI_PR_GUIDE.md) | PR templates, common mistakes |\n| [docs/PATTERNS.md](docs/PATTERNS.md) | Code patterns and anti-patterns |\n| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Component details |\n| [docs/SQLITE_INTERNALS.md](docs/SQLITE_INTERNALS.md) | WAL format, 1GB lock page |\n| [docs/LTX_FORMAT.md](docs/LTX_FORMAT.md) | Replication format |\n| [docs/TESTING_GUIDE.md](docs/TESTING_GUIDE.md) | Test strategies |\n| [docs/REPLICA_CLIENT_GUIDE.md](docs/REPLICA_CLIENT_GUIDE.md) | Storage backends |\n\n## Core Files\n\n| File | Purpose |\n|------|---------|\n| `db.go` | Database monitoring, WAL, checkpoints |\n| `replica.go` | Replication management |\n| `store.go` | Multi-database coordination |\n| `replica_client.go` | Storage backend interface |\n\n## Storage Backends\n\n- `s3/replica_client.go` - AWS S3\n- `gs/replica_client.go` - Google Cloud Storage\n- `abs/replica_client.go` - Azure Blob Storage\n- `sftp/replica_client.go` - SFTP\n- `file/replica_client.go` - Local filesystem\n- `nats/replica_client.go` - NATS JetStream\n\n## Critical Concepts\n\n- **Lock page at 1GB** - Always skip page at 0x40000000\n- **LTX files are immutable** - Never modify after creation\n- **Single replica per DB** - One destination per database\n- **Layer boundaries** - DB handles state, Replica handles replication\n\n## Build\n\n```bash\ngo build -o bin/litestream ./cmd/litestream\ngo test -race -v ./...\npre-commit run --all-files\n```\n" }, { "path": "log.go", "content": "package litestream\n\nconst (\n\tLogKeySystem = \"system\"\n\tLogKeySubsystem = \"subsystem\"\n\tLogKeyDB = \"db\"\n)\n\nconst (\n\tLogSystemStore = \"store\"\n\tLogSystemServer = \"server\"\n)\n\nconst (\n\tLogSubsystemCompactor = \"compactor\"\n\tLogSubsystemWALReader = \"wal-reader\"\n)\n" }, { "path": "mock/replica_client.go", "content": "package mock\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"log/slog\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nvar _ litestream.ReplicaClient = (*ReplicaClient)(nil)\n\ntype ReplicaClient struct {\n\tInitFunc func(ctx context.Context) error\n\tDeleteAllFunc func(ctx context.Context) error\n\tLTXFilesFunc func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error)\n\tOpenLTXFileFunc func(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n\tWriteLTXFileFunc func(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error)\n\tDeleteLTXFilesFunc func(ctx context.Context, a []*ltx.FileInfo) error\n}\n\nfunc (c *ReplicaClient) Type() string { return \"mock\" }\n\nfunc (c *ReplicaClient) Init(ctx context.Context) error {\n\tif c.InitFunc != nil {\n\t\treturn c.InitFunc(ctx)\n\t}\n\treturn nil\n}\n\nfunc (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n\treturn c.DeleteAllFunc(ctx)\n}\n\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\treturn c.LTXFilesFunc(ctx, level, seek, useMetadata)\n}\n\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\treturn c.OpenLTXFileFunc(ctx, level, minTXID, maxTXID, offset, size)\n}\n\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\treturn c.WriteLTXFileFunc(ctx, level, minTXID, maxTXID, r)\n}\n\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\treturn c.DeleteLTXFilesFunc(ctx, a)\n}\n\nfunc (c *ReplicaClient) SetLogger(_ *slog.Logger) {}\n" }, { "path": "nats/replica_client.go", "content": "package nats\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net/url\"\n\t\"os\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/nats-io/nats.go\"\n\t\"github.com/nats-io/nats.go/jetstream\"\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\nfunc init() {\n\tlitestream.RegisterReplicaClientFactory(\"nats\", NewReplicaClientFromURL)\n}\n\n// ReplicaClientType is the client type for this package.\nconst ReplicaClientType = \"nats\"\n\n// HeaderKeyTimestamp is the header key for storing LTX file timestamps in NATS object headers.\nconst HeaderKeyTimestamp = \"Litestream-Timestamp\"\n\nvar _ litestream.ReplicaClient = (*ReplicaClient)(nil)\n\n// ReplicaClient is a client for writing LTX files to NATS JetStream Object Store.\ntype ReplicaClient struct {\n\tmu sync.Mutex\n\tlogger *slog.Logger\n\n\t// NATS connection and JetStream context\n\tnc *nats.Conn\n\tjs jetstream.JetStream\n\tobjectStore jetstream.ObjectStore\n\n\t// Configuration\n\tURL string // NATS server URL\n\tBucketName string // Object store bucket name\n\tPath string // Base path for LTX files within the bucket\n\tJWT string // JWT token for authentication\n\tSeed string // Seed for JWT authentication\n\tCreds string // Credentials file path\n\tNKey string // NKey for authentication\n\tUsername string // Username for authentication\n\tPassword string // Password for authentication\n\tToken string // Token for authentication\n\tTLS bool // Enable TLS\n\tRootCAs []string // Root CA certificates\n\tClientCert string // Client certificate file path\n\tClientKey string // Client key file path\n\n\t// Note: Bucket configuration (replicas, storage, TTL, etc.) should be\n\t// managed externally via NATS CLI or API, not by Litestream\n\n\t// Connection options\n\tMaxReconnects int // Maximum reconnection attempts (-1 for unlimited)\n\tReconnectWait time.Duration // Wait time between reconnection attempts\n\tReconnectJitter time.Duration // Random jitter for reconnection\n\tTimeout time.Duration // Connection timeout\n\tPingInterval time.Duration // Ping interval\n\tMaxPingsOut int // Maximum number of pings without response\n\tReconnectBufSize int // Reconnection buffer size\n\tUserJWT func() (string, error) // JWT callback\n\tSigCB func([]byte) ([]byte, error) // Signature callback\n}\n\n// NewReplicaClient returns a new instance of ReplicaClient.\nfunc NewReplicaClient() *ReplicaClient {\n\treturn &ReplicaClient{\n\t\tlogger: slog.Default().WithGroup(ReplicaClientType),\n\t\tMaxReconnects: -1, // Unlimited\n\t\tReconnectWait: 2 * time.Second,\n\t\tTimeout: 10 * time.Second,\n\t\tPingInterval: 2 * time.Minute,\n\t\tMaxPingsOut: 2,\n\t\tReconnectBufSize: 8 * 1024 * 1024, // 8MB\n\t}\n}\n\nfunc (c *ReplicaClient) SetLogger(logger *slog.Logger) {\n\tc.logger = logger.WithGroup(ReplicaClientType)\n}\n\n// NewReplicaClientFromURL creates a new ReplicaClient from URL components.\n// This is used by the replica client factory registration.\n// URL format: nats://[user:pass@]host[:port]/bucket\nfunc NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error) {\n\tclient := NewReplicaClient()\n\n\t// Reconstruct URL without bucket path\n\tif host != \"\" {\n\t\tclient.URL = fmt.Sprintf(\"nats://%s\", host)\n\t}\n\n\t// Extract credentials from userinfo if present\n\tif userinfo != nil {\n\t\tclient.Username = userinfo.Username()\n\t\tclient.Password, _ = userinfo.Password()\n\t}\n\n\t// Extract bucket name from path\n\tbucket := strings.Trim(urlPath, \"/\")\n\tif bucket == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for nats replica URL\")\n\t}\n\tclient.BucketName = bucket\n\n\treturn client, nil\n}\n\n// Type returns \"nats\" as the client type.\nfunc (c *ReplicaClient) Type() string {\n\treturn ReplicaClientType\n}\n\n// Init initializes the connection to NATS JetStream. No-op if already initialized.\nfunc (c *ReplicaClient) Init(ctx context.Context) error {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tif c.nc != nil {\n\t\treturn nil\n\t}\n\n\tif err := c.connect(ctx); err != nil {\n\t\treturn fmt.Errorf(\"nats: failed to connect: %w\", err)\n\t}\n\n\tif err := c.initObjectStore(ctx); err != nil {\n\t\treturn fmt.Errorf(\"nats: failed to initialize object store: %w\", err)\n\t}\n\n\treturn nil\n}\n\n// connect establishes a connection to NATS server with proper configuration.\nfunc (c *ReplicaClient) connect(_ context.Context) error {\n\topts := []nats.Option{\n\t\tnats.MaxReconnects(c.MaxReconnects),\n\t\tnats.ReconnectWait(c.ReconnectWait),\n\t\tnats.ReconnectJitter(c.ReconnectJitter, c.ReconnectJitter*2),\n\t\tnats.Timeout(c.Timeout),\n\t\tnats.PingInterval(c.PingInterval),\n\t\tnats.MaxPingsOutstanding(c.MaxPingsOut),\n\t\tnats.ReconnectBufSize(c.ReconnectBufSize),\n\t}\n\n\t// Authentication options\n\tswitch {\n\tcase c.JWT != \"\" && c.Seed != \"\":\n\t\topts = append(opts, nats.UserJWTAndSeed(c.JWT, c.Seed))\n\tcase c.Creds != \"\":\n\t\topts = append(opts, nats.UserCredentials(c.Creds))\n\tcase c.NKey != \"\":\n\t\topts = append(opts, nats.Nkey(c.NKey, c.SigCB))\n\tcase c.Username != \"\" && c.Password != \"\":\n\t\topts = append(opts, nats.UserInfo(c.Username, c.Password))\n\tcase c.Token != \"\":\n\t\topts = append(opts, nats.Token(c.Token))\n\t}\n\t// JWT callback\n\tif c.UserJWT != nil {\n\t\topts = append(opts, nats.UserJWT(c.UserJWT, c.SigCB))\n\t}\n\n\t// TLS configuration\n\tif c.ClientCert != \"\" && c.ClientKey != \"\" {\n\t\topts = append(opts, nats.ClientCert(c.ClientCert, c.ClientKey))\n\t}\n\n\tif len(c.RootCAs) > 0 {\n\t\topts = append(opts, nats.RootCAs(c.RootCAs...))\n\t}\n\n\t// Note: NATS Connect doesn't directly support context cancellation during connection\n\t// The context parameter is preserved for potential future use\n\n\turl := c.URL\n\tif url == \"\" {\n\t\turl = nats.DefaultURL\n\t}\n\n\tnc, err := nats.Connect(url, opts...)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to connect to NATS server: %w\", err)\n\t}\n\n\tjs, err := jetstream.New(nc)\n\tif err != nil {\n\t\tnc.Close()\n\t\treturn fmt.Errorf(\"failed to create JetStream context: %w\", err)\n\t}\n\n\tc.nc = nc\n\tc.js = js\n\treturn nil\n}\n\n// initObjectStore retrieves the existing object store bucket.\n// The bucket must be pre-created using the NATS CLI or API.\nfunc (c *ReplicaClient) initObjectStore(ctx context.Context) error {\n\tif c.BucketName == \"\" {\n\t\treturn fmt.Errorf(\"bucket name is required\")\n\t}\n\n\t// Get existing object store - do not auto-create\n\tobjectStore, err := c.js.ObjectStore(ctx, c.BucketName)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to access object store bucket %q (bucket must be created beforehand): %w\", c.BucketName, err)\n\t}\n\n\tc.objectStore = objectStore\n\treturn nil\n}\n\n// Close closes the NATS connection.\nfunc (c *ReplicaClient) Close() error {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tif c.nc != nil {\n\t\tc.nc.Close()\n\t\tc.nc = nil\n\t\tc.js = nil\n\t\tc.objectStore = nil\n\t}\n\treturn nil\n}\n\n// ltxPath returns the object path for an LTX file.\nfunc (c *ReplicaClient) ltxPath(level int, minTXID, maxTXID ltx.TXID) string {\n\treturn litestream.LTXFilePath(c.Path, level, minTXID, maxTXID)\n}\n\n// parseLTXPath parses an LTX object path and returns level, minTXID, and maxTXID.\nfunc (c *ReplicaClient) parseLTXPath(objPath string) (level int, minTXID, maxTXID ltx.TXID, err error) {\n\t// Remove the base path prefix if present\n\tif c.Path != \"\" && strings.HasPrefix(objPath, c.Path+\"/\") {\n\t\tobjPath = strings.TrimPrefix(objPath, c.Path+\"/\")\n\t}\n\n\t// Expected format: \"ltx//-.ltx\"\n\tparts := strings.Split(objPath, \"/\")\n\tif len(parts) < 3 || parts[0] != \"ltx\" {\n\t\treturn 0, 0, 0, fmt.Errorf(\"invalid ltx path: %s\", objPath)\n\t}\n\n\t// Parse level\n\tif level, err = strconv.Atoi(parts[1]); err != nil {\n\t\treturn 0, 0, 0, fmt.Errorf(\"invalid level in path %s: %w\", objPath, err)\n\t}\n\n\t// Parse filename (minTXID-maxTXID.ltx)\n\tfilename := parts[2]\n\tminTXIDVal, maxTXIDVal, err := ltx.ParseFilename(filename)\n\tif err != nil {\n\t\treturn 0, 0, 0, fmt.Errorf(\"invalid filename in path %s: %w\", objPath, err)\n\t}\n\n\treturn level, minTXIDVal, maxTXIDVal, nil\n}\n\n// LTXFiles returns an iterator of all LTX files on the replica for a given level.\n// NATS always uses accurate timestamps from headers since they're included in LIST operations at zero cost.\n// The useMetadata parameter is ignored.\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// List all objects in the store\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"LIST\").Inc()\n\tobjectList, err := c.objectStore.List(ctx)\n\tif err != nil {\n\t\t// NATS returns \"no objects found\" when bucket is empty, treat as empty list\n\t\tif strings.Contains(err.Error(), \"no objects found\") {\n\t\t\tobjectList = nil // Empty list\n\t\t} else {\n\t\t\treturn nil, fmt.Errorf(\"failed to list objects: %w\", err)\n\t\t}\n\t}\n\n\tprefix := litestream.LTXLevelDir(c.Path, level) + \"/\"\n\tfileInfos := make([]*ltx.FileInfo, 0, len(objectList))\n\n\tfor _, objInfo := range objectList {\n\t\t// Filter by level prefix\n\t\tif !strings.HasPrefix(objInfo.Name, prefix) {\n\t\t\tcontinue\n\t\t}\n\n\t\tfileLevel, minTXID, maxTXID, err := c.parseLTXPath(objInfo.Name)\n\t\tif err != nil {\n\t\t\tcontinue // Skip invalid paths\n\t\t}\n\n\t\tif fileLevel != level {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Apply seek filter\n\t\tif minTXID < seek {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Always use accurate timestamp from headers since it's zero-cost\n\t\t// NATS includes headers in LIST operations, so no extra API call needed\n\t\tcreatedAt := objInfo.ModTime\n\t\tif objInfo.Headers != nil {\n\t\t\tif values, ok := objInfo.Headers[HeaderKeyTimestamp]; ok && len(values) > 0 {\n\t\t\t\tif parsed, err := time.Parse(time.RFC3339Nano, values[0]); err == nil {\n\t\t\t\t\tcreatedAt = parsed\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tfileInfos = append(fileInfos, <x.FileInfo{\n\t\t\tLevel: fileLevel,\n\t\t\tMinTXID: minTXID,\n\t\t\tMaxTXID: maxTXID,\n\t\t\tSize: int64(objInfo.Size),\n\t\t\tCreatedAt: createdAt,\n\t\t})\n\t}\n\n\t// Sort by minTXID\n\tsort.Slice(fileInfos, func(i, j int) bool {\n\t\treturn fileInfos[i].MinTXID < fileInfos[j].MinTXID\n\t})\n\n\treturn <xFileIterator{files: fileInfos, index: -1}, nil\n}\n\n// OpenLTXFile returns a reader that contains an LTX file at a given TXID range.\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tobjectPath := c.ltxPath(level, minTXID, maxTXID)\n\n\tobjectResult, err := c.objectStore.Get(ctx, objectPath)\n\tif err != nil {\n\t\tif isNotFoundError(err) {\n\t\t\treturn nil, os.ErrNotExist\n\t\t}\n\t\treturn nil, fmt.Errorf(\"failed to get object %s: %w\", objectPath, err)\n\t}\n\n\t// Record metrics\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"GET\").Inc()\n\t// Note: We can't get the size from NATS object reader directly, so we skip bytes counter\n\n\t// If offset is non-zero then discard the beginning bytes.\n\tif offset > 0 {\n\t\tif _, err := io.CopyN(io.Discard, objectResult, offset); err != nil {\n\t\t\tobjectResult.Close()\n\t\t\treturn nil, fmt.Errorf(\"failed to discard offset bytes: %w\", err)\n\t\t}\n\t}\n\n\t// If size is non-zero then limit the reader to the size.\n\tif size > 0 {\n\t\treturn internal.LimitReadCloser(objectResult, size), nil\n\t}\n\n\treturn objectResult, nil\n}\n\n// WriteLTXFile writes an LTX file to the replica.\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tobjectPath := c.ltxPath(level, minTXID, maxTXID)\n\n\t// Use TeeReader to peek at LTX header while preserving data for upload\n\tvar buf bytes.Buffer\n\tteeReader := io.TeeReader(r, &buf)\n\n\t// Extract timestamp from LTX header\n\thdr, _, err := ltx.PeekHeader(teeReader)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"extract timestamp from LTX header: %w\", err)\n\t}\n\ttimestamp := time.UnixMilli(hdr.Timestamp).UTC()\n\n\t// Combine buffered data with rest of reader\n\trc := internal.NewReadCounter(io.MultiReader(&buf, r))\n\n\t// Store timestamp in NATS object headers for accurate timestamp retrieval\n\tobjectInfo, err := c.objectStore.Put(ctx, jetstream.ObjectMeta{\n\t\tName: objectPath,\n\t\tHeaders: map[string][]string{\n\t\t\tHeaderKeyTimestamp: {timestamp.Format(time.RFC3339Nano)},\n\t\t},\n\t}, rc)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to put object %s: %w\", objectPath, err)\n\t}\n\n\t// Record metrics\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Add(float64(objectInfo.Size))\n\n\treturn <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tSize: int64(objectInfo.Size),\n\t\tCreatedAt: timestamp,\n\t}, nil\n}\n\n// DeleteLTXFiles deletes one or more LTX files.\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\tfor _, fileInfo := range a {\n\t\tobjectPath := c.ltxPath(fileInfo.Level, fileInfo.MinTXID, fileInfo.MaxTXID)\n\n\t\tc.logger.Debug(\"deleting ltx file\", \"level\", fileInfo.Level, \"minTXID\", fileInfo.MinTXID, \"maxTXID\", fileInfo.MaxTXID, \"path\", objectPath)\n\n\t\tif err := c.objectStore.Delete(ctx, objectPath); err != nil {\n\t\t\tif !isNotFoundError(err) {\n\t\t\t\treturn fmt.Errorf(\"failed to delete object %s: %w\", objectPath, err)\n\t\t\t}\n\t\t}\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\t}\n\n\treturn nil\n}\n\n// DeleteAll deletes all files in the object store.\nfunc (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\t// List all objects in the bucket\n\tobjectList, err := c.objectStore.List(ctx)\n\tif err != nil {\n\t\t// NATS returns \"no objects found\" when bucket is empty, treat as empty list\n\t\tif strings.Contains(err.Error(), \"no objects found\") {\n\t\t\tobjectList = nil // Empty list, nothing to delete\n\t\t} else {\n\t\t\treturn fmt.Errorf(\"failed to list all objects: %w\", err)\n\t\t}\n\t}\n\n\tfor _, objInfo := range objectList {\n\t\tif err := c.objectStore.Delete(ctx, objInfo.Name); err != nil {\n\t\t\tif !isNotFoundError(err) {\n\t\t\t\treturn fmt.Errorf(\"failed to delete object %s: %w\", objInfo.Name, err)\n\t\t\t}\n\t\t}\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\t}\n\n\treturn nil\n}\n\n// isNotFoundError checks if the error is a \"not found\" error.\nfunc isNotFoundError(err error) bool {\n\treturn err != nil && (errors.Is(err, jetstream.ErrObjectNotFound) || strings.Contains(err.Error(), \"not found\"))\n}\n\n// ltxFileIterator implements ltx.FileIterator for NATS object store.\ntype ltxFileIterator struct {\n\tfiles []*ltx.FileInfo\n\tindex int\n\terr error\n}\n\n// Next advances the iterator to the next file.\nfunc (itr *ltxFileIterator) Next() bool {\n\titr.index++\n\treturn itr.index < len(itr.files)\n}\n\n// Item returns the current file info.\nfunc (itr *ltxFileIterator) Item() *ltx.FileInfo {\n\tif itr.index < 0 || itr.index >= len(itr.files) {\n\t\treturn nil\n\t}\n\treturn itr.files[itr.index]\n}\n\n// Err returns any error that occurred during iteration.\nfunc (itr *ltxFileIterator) Err() error {\n\treturn itr.err\n}\n\n// Close closes the iterator and returns any error that occurred during iteration.\nfunc (itr *ltxFileIterator) Close() error {\n\treturn itr.err\n}\n" }, { "path": "nats/replica_client_test.go", "content": "package nats\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n)\n\nfunc TestReplicaClient_Type(t *testing.T) {\n\tclient := NewReplicaClient()\n\tif got, want := client.Type(), \"nats\"; got != want {\n\t\tt.Fatalf(\"Type()=%s, want %s\", got, want)\n\t}\n}\n\nfunc TestReplicaClient_ltxPath(t *testing.T) {\n\tclient := NewReplicaClient()\n\n\ttests := []struct {\n\t\tlevel int\n\t\tminTXID ltx.TXID\n\t\tmaxTXID ltx.TXID\n\t\twant string\n\t}{\n\t\t{0, ltx.TXID(0x1000), ltx.TXID(0x2000), \"ltx/0/0000000000001000-0000000000002000.ltx\"},\n\t\t{1, ltx.TXID(0xabcd), ltx.TXID(0xef01), \"ltx/1/000000000000abcd-000000000000ef01.ltx\"},\n\t\t{255, ltx.TXID(0xffffffffffffffff), ltx.TXID(0xffffffffffffffff), \"ltx/255/ffffffffffffffff-ffffffffffffffff.ltx\"},\n\t}\n\n\tfor _, test := range tests {\n\t\tif got := client.ltxPath(test.level, test.minTXID, test.maxTXID); got != test.want {\n\t\t\tt.Errorf(\"ltxPath(%d, %x, %x)=%s, want %s\", test.level, test.minTXID, test.maxTXID, got, test.want)\n\t\t}\n\t}\n}\n\nfunc TestReplicaClient_parseLTXPath(t *testing.T) {\n\tclient := NewReplicaClient()\n\n\ttests := []struct {\n\t\tpath string\n\t\tlevel int\n\t\tminTXID ltx.TXID\n\t\tmaxTXID ltx.TXID\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tpath: \"ltx/0/0000000000001000-0000000000002000.ltx\",\n\t\t\tlevel: 0,\n\t\t\tminTXID: ltx.TXID(0x1000),\n\t\t\tmaxTXID: ltx.TXID(0x2000),\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tpath: \"ltx/1/000000000000abcd-000000000000ef01.ltx\",\n\t\t\tlevel: 1,\n\t\t\tminTXID: ltx.TXID(0xabcd),\n\t\t\tmaxTXID: ltx.TXID(0xef01),\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tpath: \"invalid/path\",\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tpath: \"ltx/x/invalid-invalid.ltx\",\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tpath: \"ltx/0/invalid.ltx\",\n\t\t\twantErr: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tlevel, minTXID, maxTXID, err := client.parseLTXPath(test.path)\n\t\tif test.wantErr {\n\t\t\tif err == nil {\n\t\t\t\tt.Errorf(\"parseLTXPath(%s) expected error, got nil\", test.path)\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\tif err != nil {\n\t\t\tt.Errorf(\"parseLTXPath(%s) unexpected error: %v\", test.path, err)\n\t\t\tcontinue\n\t\t}\n\n\t\tif level != test.level {\n\t\t\tt.Errorf(\"parseLTXPath(%s) level=%d, want %d\", test.path, level, test.level)\n\t\t}\n\t\tif minTXID != test.minTXID {\n\t\t\tt.Errorf(\"parseLTXPath(%s) minTXID=%x, want %x\", test.path, minTXID, test.minTXID)\n\t\t}\n\t\tif maxTXID != test.maxTXID {\n\t\t\tt.Errorf(\"parseLTXPath(%s) maxTXID=%x, want %x\", test.path, maxTXID, test.maxTXID)\n\t\t}\n\t}\n}\n\nfunc TestReplicaClient_isNotFoundError(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\terr error\n\t\twant bool\n\t}{\n\t\t{\n\t\t\tname: \"nil error\",\n\t\t\terr: nil,\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"not found error\",\n\t\t\terr: &mockNotFoundError{},\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"other error\",\n\t\t\terr: &mockOtherError{},\n\t\t\twant: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif got := isNotFoundError(test.err); got != test.want {\n\t\t\t\tt.Errorf(\"isNotFoundError() = %v, want %v\", got, test.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestLtxFileIterator(t *testing.T) {\n\tfiles := []*ltx.FileInfo{\n\t\t{Level: 0, MinTXID: 1, MaxTXID: 10, Size: 100},\n\t\t{Level: 0, MinTXID: 11, MaxTXID: 20, Size: 200},\n\t\t{Level: 0, MinTXID: 21, MaxTXID: 30, Size: 300},\n\t}\n\n\titr := <xFileIterator{files: files, index: -1}\n\n\t// Test initial state\n\tif item := itr.Item(); item != nil {\n\t\tt.Errorf(\"Item() before Next() should return nil, got %v\", item)\n\t}\n\n\t// Test iteration\n\tvar items []*ltx.FileInfo\n\tfor itr.Next() {\n\t\titem := itr.Item()\n\t\tif item == nil {\n\t\t\tt.Fatal(\"Item() returned nil during valid iteration\")\n\t\t}\n\t\titems = append(items, item)\n\t}\n\n\tif len(items) != len(files) {\n\t\tt.Errorf(\"Expected %d items, got %d\", len(files), len(items))\n\t}\n\n\tfor i, item := range items {\n\t\tif item != files[i] {\n\t\t\tt.Errorf(\"Item %d: expected %v, got %v\", i, files[i], item)\n\t\t}\n\t}\n\n\t// Test after iteration ends\n\tif itr.Next() {\n\t\tt.Error(\"Next() should return false after iteration ends\")\n\t}\n\n\t// Test Close\n\tif err := itr.Close(); err != nil {\n\t\tt.Errorf(\"Close() returned error: %v\", err)\n\t}\n\n\t// Test Err\n\tif err := itr.Err(); err != nil {\n\t\tt.Errorf(\"Err() returned error: %v\", err)\n\t}\n}\n\n// Mock error types for testing\n\ntype mockNotFoundError struct{}\n\nfunc (e *mockNotFoundError) Error() string {\n\treturn \"not found\"\n}\n\ntype mockOtherError struct{}\n\nfunc (e *mockOtherError) Error() string {\n\treturn \"some other error\"\n}\n\nfunc TestReplicaClientDefaults(t *testing.T) {\n\tclient := NewReplicaClient()\n\n\t// Test default values\n\tif client.MaxReconnects != -1 {\n\t\tt.Errorf(\"Expected MaxReconnects=-1, got %d\", client.MaxReconnects)\n\t}\n\n\tif client.ReconnectWait != 2*time.Second {\n\t\tt.Errorf(\"Expected ReconnectWait=2s, got %v\", client.ReconnectWait)\n\t}\n\n\tif client.Timeout != 10*time.Second {\n\t\tt.Errorf(\"Expected Timeout=10s, got %v\", client.Timeout)\n\t}\n}\n" }, { "path": "oss/replica_client.go", "content": "package oss\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net/url\"\n\t\"os\"\n\t\"path\"\n\t\"regexp\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss\"\n\t\"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials\"\n\t\"github.com/superfly/ltx\"\n\t\"golang.org/x/sync/errgroup\"\n\t\"golang.org/x/sync/semaphore\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\nfunc init() {\n\tlitestream.RegisterReplicaClientFactory(\"oss\", NewReplicaClientFromURL)\n}\n\n// ReplicaClientType is the client type for this package.\nconst ReplicaClientType = \"oss\"\n\n// MetadataKeyTimestamp is the metadata key for storing LTX file timestamps in OSS.\n// Note: OSS SDK automatically adds \"x-oss-meta-\" prefix when setting metadata.\nconst MetadataKeyTimestamp = \"litestream-timestamp\"\n\n// MaxKeys is the number of keys OSS can operate on per batch.\nconst MaxKeys = 1000\n\n// DefaultRegion is the region used if one is not specified.\nconst DefaultRegion = \"cn-hangzhou\"\n\n// DefaultMetadataConcurrency is the default number of concurrent HeadObject calls\n// for fetching accurate timestamps during timestamp-based restore.\nconst DefaultMetadataConcurrency = 50\n\nvar _ litestream.ReplicaClient = (*ReplicaClient)(nil)\n\n// ReplicaClient is a client for writing LTX files to Alibaba Cloud OSS.\ntype ReplicaClient struct {\n\tmu sync.Mutex\n\tclient *oss.Client\n\tuploader *oss.Uploader\n\tlogger *slog.Logger\n\n\t// Alibaba Cloud authentication keys.\n\tAccessKeyID string\n\tAccessKeySecret string\n\n\t// OSS bucket information\n\tRegion string\n\tBucket string\n\tPath string\n\tEndpoint string\n\n\t// Upload configuration\n\tPartSize int64 // Part size for multipart uploads (default: 5MB)\n\tConcurrency int // Number of concurrent parts to upload (default: 3)\n\n\t// MetadataConcurrency controls parallel HeadObject calls for timestamp-based restore.\n\t// Higher values improve restore speed for large backup histories.\n\t// Default: 50\n\tMetadataConcurrency int\n}\n\n// NewReplicaClient returns a new instance of ReplicaClient.\nfunc NewReplicaClient() *ReplicaClient {\n\treturn &ReplicaClient{\n\t\tlogger: slog.Default().WithGroup(ReplicaClientType),\n\t}\n}\n\nfunc (c *ReplicaClient) SetLogger(logger *slog.Logger) {\n\tc.logger = logger.WithGroup(ReplicaClientType)\n}\n\n// NewReplicaClientFromURL creates a new ReplicaClient from URL components.\n// This is used by the replica client factory registration.\n// URL format: oss://bucket[.oss-region.aliyuncs.com]/path\nfunc NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error) {\n\tclient := NewReplicaClient()\n\n\tbucket, region, _ := ParseHost(host)\n\tif bucket == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for oss replica URL\")\n\t}\n\n\tclient.Bucket = bucket\n\tclient.Region = region\n\tclient.Path = urlPath\n\n\treturn client, nil\n}\n\n// Type returns \"oss\" as the client type.\nfunc (c *ReplicaClient) Type() string {\n\treturn ReplicaClientType\n}\n\n// Init initializes the connection to OSS. No-op if already initialized.\nfunc (c *ReplicaClient) Init(ctx context.Context) (err error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tif c.client != nil {\n\t\treturn nil\n\t}\n\n\t// Validate required configuration\n\tif c.Bucket == \"\" {\n\t\treturn fmt.Errorf(\"oss: bucket name is required\")\n\t}\n\n\t// Use default region if not specified\n\tregion := c.Region\n\tif region == \"\" {\n\t\tregion = DefaultRegion\n\t}\n\n\t// Build configuration\n\tcfg := oss.LoadDefaultConfig()\n\n\t// Configure credentials\n\tif c.AccessKeyID != \"\" && c.AccessKeySecret != \"\" {\n\t\tcfg = cfg.WithCredentialsProvider(\n\t\t\tcredentials.NewStaticCredentialsProvider(c.AccessKeyID, c.AccessKeySecret),\n\t\t)\n\t} else {\n\t\t// Use environment variable credentials provider\n\t\tcfg = cfg.WithCredentialsProvider(\n\t\t\tcredentials.NewEnvironmentVariableCredentialsProvider(),\n\t\t)\n\t}\n\n\t// Configure region\n\tcfg = cfg.WithRegion(region)\n\n\t// Configure custom endpoint if specified\n\tif c.Endpoint != \"\" {\n\t\tendpoint := c.Endpoint\n\t\t// Add scheme if not present\n\t\tif !strings.HasPrefix(endpoint, \"http://\") && !strings.HasPrefix(endpoint, \"https://\") {\n\t\t\tendpoint = \"https://\" + endpoint\n\t\t}\n\t\tcfg = cfg.WithEndpoint(endpoint)\n\t}\n\n\t// Create OSS client\n\tc.client = oss.NewClient(cfg)\n\n\t// Create uploader with configurable part size and concurrency\n\tuploaderOpts := []func(*oss.UploaderOptions){}\n\tif c.PartSize > 0 {\n\t\tuploaderOpts = append(uploaderOpts, func(o *oss.UploaderOptions) {\n\t\t\to.PartSize = c.PartSize\n\t\t})\n\t}\n\tif c.Concurrency > 0 {\n\t\tuploaderOpts = append(uploaderOpts, func(o *oss.UploaderOptions) {\n\t\t\to.ParallelNum = c.Concurrency\n\t\t})\n\t}\n\tc.uploader = c.client.NewUploader(uploaderOpts...)\n\n\treturn nil\n}\n\n// LTXFiles returns an iterator over all LTX files on the replica for the given level.\n// When useMetadata is true, fetches accurate timestamps from OSS metadata via HeadObject.\n// This uses parallel batched requests (controlled by MetadataConcurrency) to avoid hangs\n// with large backup histories (see issue #930).\n// When false, uses fast LastModified timestamps from LIST operation.\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\treturn newFileIterator(ctx, c, level, seek, useMetadata), nil\n}\n\n// OpenLTXFile returns a reader for an LTX file.\n// Returns os.ErrNotExist if no matching file is found.\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Build the key from the file info\n\tfilename := ltx.FormatFilename(minTXID, maxTXID)\n\tkey := c.ltxPath(level, filename)\n\n\trequest := &oss.GetObjectRequest{\n\t\tBucket: oss.Ptr(c.Bucket),\n\t\tKey: oss.Ptr(key),\n\t}\n\n\t// Set range header if offset is specified\n\tif size > 0 {\n\t\trequest.RangeBehavior = oss.Ptr(\"standard\")\n\t\trequest.Range = oss.Ptr(fmt.Sprintf(\"bytes=%d-%d\", offset, offset+size-1))\n\t} else if offset > 0 {\n\t\trequest.RangeBehavior = oss.Ptr(\"standard\")\n\t\trequest.Range = oss.Ptr(fmt.Sprintf(\"bytes=%d-\", offset))\n\t}\n\n\tresult, err := c.client.GetObject(ctx, request)\n\tif err != nil {\n\t\tif isNotExists(err) {\n\t\t\treturn nil, os.ErrNotExist\n\t\t}\n\t\treturn nil, fmt.Errorf(\"oss: get object %s: %w\", key, err)\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"GET\").Inc()\n\n\treturn result.Body, nil\n}\n\n// WriteLTXFile writes an LTX file to the replica.\n// Extracts timestamp from LTX header and stores it in OSS metadata to preserve original creation time.\n// Uses multipart upload for large files via the uploader.\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Use TeeReader to peek at LTX header while preserving data for upload\n\tvar buf bytes.Buffer\n\tteeReader := io.TeeReader(r, &buf)\n\n\t// Extract timestamp from LTX header\n\thdr, _, err := ltx.PeekHeader(teeReader)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"extract timestamp from LTX header: %w\", err)\n\t}\n\ttimestamp := time.UnixMilli(hdr.Timestamp).UTC()\n\n\t// Combine buffered data with rest of reader\n\trc := internal.NewReadCounter(io.MultiReader(&buf, r))\n\n\tfilename := ltx.FormatFilename(minTXID, maxTXID)\n\tkey := c.ltxPath(level, filename)\n\n\t// Store timestamp in OSS metadata for accurate timestamp retrieval\n\tmetadata := map[string]string{\n\t\tMetadataKeyTimestamp: timestamp.Format(time.RFC3339Nano),\n\t}\n\n\t// Use uploader for automatic multipart handling (files >5GB)\n\tresult, err := c.uploader.UploadFrom(ctx, &oss.PutObjectRequest{\n\t\tBucket: oss.Ptr(c.Bucket),\n\t\tKey: oss.Ptr(key),\n\t\tMetadata: metadata,\n\t}, rc)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"oss: upload to %s: %w\", key, err)\n\t}\n\n\t// Build file info from the uploaded file\n\tinfo := <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tSize: rc.N(),\n\t\tCreatedAt: timestamp,\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Add(float64(rc.N()))\n\n\t// ETag indicates successful upload\n\tif result.ETag == nil || *result.ETag == \"\" {\n\t\treturn nil, fmt.Errorf(\"oss: upload failed: no ETag returned\")\n\t}\n\n\treturn info, nil\n}\n\n// DeleteLTXFiles deletes one or more LTX files.\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\tif len(a) == 0 {\n\t\treturn nil\n\t}\n\n\t// Convert file infos to object identifiers\n\tobjects := make([]oss.DeleteObject, 0, len(a))\n\tfor _, info := range a {\n\t\tfilename := ltx.FormatFilename(info.MinTXID, info.MaxTXID)\n\t\tkey := c.ltxPath(info.Level, filename)\n\t\tobjects = append(objects, oss.DeleteObject{Key: oss.Ptr(key)})\n\n\t\tc.logger.Debug(\"deleting ltx file\", \"level\", info.Level, \"minTXID\", info.MinTXID, \"maxTXID\", info.MaxTXID, \"key\", key)\n\t}\n\n\t// Delete in batches\n\tfor len(objects) > 0 {\n\t\tn := min(len(objects), MaxKeys)\n\t\tbatch := objects[:n]\n\n\t\trequest := &oss.DeleteMultipleObjectsRequest{\n\t\t\tBucket: oss.Ptr(c.Bucket),\n\t\t\tObjects: batch,\n\t\t}\n\n\t\tout, err := c.client.DeleteMultipleObjects(ctx, request)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"oss: delete batch of %d objects: %w\", n, err)\n\t\t} else if err := deleteResultError(batch, out); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\n\t\tobjects = objects[n:]\n\t}\n\n\treturn nil\n}\n\n// DeleteAll deletes all files.\nfunc (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\tvar objects []oss.DeleteObject\n\n\t// Create paginator for listing objects\n\tprefix := c.Path + \"/\"\n\tpaginator := c.client.NewListObjectsV2Paginator(&oss.ListObjectsV2Request{\n\t\tBucket: oss.Ptr(c.Bucket),\n\t\tPrefix: oss.Ptr(prefix),\n\t})\n\n\t// Iterate through all pages\n\tfor paginator.HasNext() {\n\t\tpage, err := paginator.NextPage(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"oss: list objects page: %w\", err)\n\t\t}\n\n\t\t// Collect object identifiers\n\t\tfor _, obj := range page.Contents {\n\t\t\tif obj.Key != nil {\n\t\t\t\tobjects = append(objects, oss.DeleteObject{Key: obj.Key})\n\t\t\t}\n\t\t}\n\t}\n\n\t// Delete all collected objects in batches\n\tfor len(objects) > 0 {\n\t\tn := min(len(objects), MaxKeys)\n\t\tbatch := objects[:n]\n\n\t\trequest := &oss.DeleteMultipleObjectsRequest{\n\t\t\tBucket: oss.Ptr(c.Bucket),\n\t\t\tObjects: batch,\n\t\t}\n\n\t\tout, err := c.client.DeleteMultipleObjects(ctx, request)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"oss: delete all batch of %d objects: %w\", n, err)\n\t\t} else if err := deleteResultError(batch, out); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tobjects = objects[n:]\n\t}\n\n\treturn nil\n}\n\n// ltxPath returns the full path to an LTX file.\nfunc (c *ReplicaClient) ltxPath(level int, filename string) string {\n\treturn c.Path + \"/\" + fmt.Sprintf(\"%04x/%s\", level, filename)\n}\n\n// fileIterator represents an iterator over LTX files in OSS.\ntype fileIterator struct {\n\tctx context.Context\n\tcancel context.CancelFunc\n\tclient *ReplicaClient\n\tlevel int\n\tseek ltx.TXID\n\n\tuseMetadata bool // When true, fetch accurate timestamps from metadata\n\tmetadataCache map[string]time.Time // key -> timestamp cache for batch fetches\n\n\tpaginator *oss.ListObjectsV2Paginator\n\tpage *oss.ListObjectsV2Result\n\tpageIndex int\n\tinitialized bool\n\n\tclosed bool\n\terr error\n\tinfo *ltx.FileInfo\n}\n\nfunc newFileIterator(ctx context.Context, client *ReplicaClient, level int, seek ltx.TXID, useMetadata bool) *fileIterator {\n\tctx, cancel := context.WithCancel(ctx)\n\n\titr := &fileIterator{\n\t\tctx: ctx,\n\t\tcancel: cancel,\n\t\tclient: client,\n\t\tlevel: level,\n\t\tseek: seek,\n\t\tuseMetadata: useMetadata,\n\t\tmetadataCache: make(map[string]time.Time),\n\t}\n\n\treturn itr\n}\n\n// fetchMetadataBatch fetches timestamps from OSS metadata for a batch of keys in parallel.\nfunc (itr *fileIterator) fetchMetadataBatch(keys []string) error {\n\tif len(keys) == 0 {\n\t\treturn nil\n\t}\n\n\t// Determine concurrency limit\n\tconcurrency := itr.client.MetadataConcurrency\n\tif concurrency <= 0 {\n\t\tconcurrency = DefaultMetadataConcurrency\n\t}\n\n\t// Pre-allocate results map to avoid lock contention during writes\n\tresults := make(map[string]time.Time, len(keys))\n\tvar mu sync.Mutex\n\n\t// Use x/sync/semaphore for precise concurrency control with context support\n\tsem := semaphore.NewWeighted(int64(concurrency))\n\tg, ctx := errgroup.WithContext(itr.ctx)\n\n\tfor _, key := range keys {\n\t\tkey := key // capture for goroutine\n\n\t\tg.Go(func() error {\n\t\t\t// Acquire semaphore slot (blocking with context cancellation)\n\t\t\tif err := sem.Acquire(ctx, 1); err != nil {\n\t\t\t\treturn err // context cancelled\n\t\t\t}\n\t\t\tdefer sem.Release(1)\n\n\t\t\thead, err := itr.client.client.HeadObject(ctx, &oss.HeadObjectRequest{\n\t\t\t\tBucket: oss.Ptr(itr.client.Bucket),\n\t\t\t\tKey: oss.Ptr(key),\n\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\t// Non-fatal: file might not have metadata, use LastModified\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tif head.Metadata != nil {\n\t\t\t\tif ts, ok := head.Metadata[MetadataKeyTimestamp]; ok {\n\t\t\t\t\tif parsed, err := time.Parse(time.RFC3339Nano, ts); err == nil {\n\t\t\t\t\t\tmu.Lock()\n\t\t\t\t\t\tresults[key] = parsed\n\t\t\t\t\t\tmu.Unlock()\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t}\n\n\tif err := g.Wait(); err != nil {\n\t\treturn err\n\t}\n\n\t// Merge results into cache\n\tfor k, v := range results {\n\t\titr.metadataCache[k] = v\n\t}\n\treturn nil\n}\n\n// initPaginator initializes the paginator lazily.\nfunc (itr *fileIterator) initPaginator() {\n\tif itr.initialized {\n\t\treturn\n\t}\n\titr.initialized = true\n\n\t// Create paginator for listing objects with level prefix\n\tprefix := itr.client.ltxPath(itr.level, \"\")\n\titr.paginator = itr.client.client.NewListObjectsV2Paginator(&oss.ListObjectsV2Request{\n\t\tBucket: oss.Ptr(itr.client.Bucket),\n\t\tPrefix: oss.Ptr(prefix),\n\t})\n}\n\n// Close stops iteration and returns any error that occurred during iteration.\nfunc (itr *fileIterator) Close() (err error) {\n\titr.closed = true\n\titr.cancel()\n\treturn itr.err\n}\n\n// Next returns the next file. Returns false when no more files are available.\nfunc (itr *fileIterator) Next() bool {\n\tif itr.closed || itr.err != nil {\n\t\treturn false\n\t}\n\n\t// Initialize paginator on first call\n\titr.initPaginator()\n\n\t// Process objects until we find a valid LTX file\n\tfor {\n\t\t// Load next page if needed\n\t\tif itr.page == nil || itr.pageIndex >= len(itr.page.Contents) {\n\t\t\tif !itr.paginator.HasNext() {\n\t\t\t\treturn false\n\t\t\t}\n\n\t\t\tvar err error\n\t\t\titr.page, err = itr.paginator.NextPage(itr.ctx)\n\t\t\tif err != nil {\n\t\t\t\titr.err = err\n\t\t\t\treturn false\n\t\t\t}\n\t\t\titr.pageIndex = 0\n\n\t\t\t// Batch fetch metadata for the entire page when useMetadata is true.\n\t\t\t// This uses parallel HeadObject calls controlled by MetadataConcurrency\n\t\t\t// to avoid the O(N) sequential calls that caused restore hangs (issue #930).\n\t\t\tif itr.useMetadata && len(itr.page.Contents) > 0 {\n\t\t\t\tkeys := make([]string, 0, len(itr.page.Contents))\n\t\t\t\tfor _, obj := range itr.page.Contents {\n\t\t\t\t\tif obj.Key != nil {\n\t\t\t\t\t\tkeys = append(keys, *obj.Key)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif err := itr.fetchMetadataBatch(keys); err != nil {\n\t\t\t\t\titr.err = err\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Process current object\n\t\tif itr.pageIndex < len(itr.page.Contents) {\n\t\t\tobj := itr.page.Contents[itr.pageIndex]\n\t\t\titr.pageIndex++\n\n\t\t\tif obj.Key == nil {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Extract file info from key\n\t\t\tfullKey := *obj.Key\n\t\t\tkey := path.Base(fullKey)\n\t\t\tminTXID, maxTXID, err := ltx.ParseFilename(key)\n\t\t\tif err != nil {\n\t\t\t\tcontinue // Skip non-LTX files\n\t\t\t}\n\n\t\t\t// Build file info\n\t\t\tinfo := <x.FileInfo{\n\t\t\t\tLevel: itr.level,\n\t\t\t\tMinTXID: minTXID,\n\t\t\t\tMaxTXID: maxTXID,\n\t\t\t}\n\n\t\t\t// Skip if below seek TXID\n\t\t\tif info.MinTXID < itr.seek {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Set file info\n\t\t\tinfo.Size = obj.Size\n\n\t\t\t// Use cached metadata timestamp if available (from batch fetch),\n\t\t\t// otherwise fallback to LastModified from LIST operation.\n\t\t\tif itr.useMetadata {\n\t\t\t\tif ts, ok := itr.metadataCache[fullKey]; ok {\n\t\t\t\t\tinfo.CreatedAt = ts\n\t\t\t\t} else if obj.LastModified != nil {\n\t\t\t\t\tinfo.CreatedAt = obj.LastModified.UTC()\n\t\t\t\t} else {\n\t\t\t\t\tinfo.CreatedAt = time.Now().UTC()\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif obj.LastModified != nil {\n\t\t\t\t\tinfo.CreatedAt = obj.LastModified.UTC()\n\t\t\t\t} else {\n\t\t\t\t\tinfo.CreatedAt = time.Now().UTC()\n\t\t\t\t}\n\t\t\t}\n\n\t\t\titr.info = info\n\t\t\treturn true\n\t\t}\n\t}\n}\n\n// Item returns the metadata for the current file.\nfunc (itr *fileIterator) Item() *ltx.FileInfo {\n\treturn itr.info\n}\n\n// Err returns any error that occurred during iteration.\nfunc (itr *fileIterator) Err() error {\n\treturn itr.err\n}\n\n// ParseURL parses an OSS URL into its host and path parts.\nfunc ParseURL(s string) (bucket, region, key string, err error) {\n\tu, err := url.Parse(s)\n\tif err != nil {\n\t\treturn \"\", \"\", \"\", err\n\t}\n\n\tif u.Scheme != \"oss\" {\n\t\treturn \"\", \"\", \"\", fmt.Errorf(\"oss: invalid url scheme\")\n\t}\n\n\t// Parse host to extract bucket and region\n\tbucket, region, _ = ParseHost(u.Host)\n\tif bucket == \"\" {\n\t\tbucket = u.Host\n\t}\n\n\tkey = strings.TrimPrefix(u.Path, \"/\")\n\treturn bucket, region, key, nil\n}\n\n// ParseHost parses the host/endpoint for an OSS storage system.\n// Supports formats like:\n// - bucket.oss-cn-hangzhou.aliyuncs.com\n// - bucket.oss-cn-hangzhou-internal.aliyuncs.com\n// - bucket (just bucket name)\nfunc ParseHost(host string) (bucket, region, endpoint string) {\n\t// Check for internal OSS URL format first (more specific pattern)\n\tif a := ossInternalRegex.FindStringSubmatch(host); len(a) > 1 {\n\t\tbucket = a[1]\n\t\tif len(a) > 2 && a[2] != \"\" {\n\t\t\tregion = a[2]\n\t\t}\n\t\treturn bucket, region, \"\"\n\t}\n\n\t// Check for standard OSS URL format\n\tif a := ossRegex.FindStringSubmatch(host); len(a) > 1 {\n\t\tbucket = a[1]\n\t\tif len(a) > 2 && a[2] != \"\" {\n\t\t\tregion = a[2]\n\t\t}\n\t\treturn bucket, region, \"\"\n\t}\n\n\t// For other hosts, assume it's just the bucket name\n\treturn host, \"\", \"\"\n}\n\nvar (\n\t// oss-cn-hangzhou.aliyuncs.com or bucket.oss-cn-hangzhou.aliyuncs.com\n\tossRegex = regexp.MustCompile(`^(?:([^.]+)\\.)?oss-([^.]+)\\.aliyuncs\\.com$`)\n\t// oss-cn-hangzhou-internal.aliyuncs.com or bucket.oss-cn-hangzhou-internal.aliyuncs.com\n\t// Uses non-greedy .+? to correctly extract region without -internal suffix\n\tossInternalRegex = regexp.MustCompile(`^(?:([^.]+)\\.)?oss-(.+?)-internal\\.aliyuncs\\.com$`)\n)\n\nfunc isNotExists(err error) bool {\n\tvar serviceErr *oss.ServiceError\n\tif errors.As(err, &serviceErr) {\n\t\treturn serviceErr.Code == \"NoSuchKey\"\n\t}\n\treturn false\n}\n\n// deleteResultError checks if all requested objects were deleted.\n// OSS SDK doesn't have explicit per-object error reporting like S3, so we verify\n// all requested keys appear in the deleted list.\nfunc deleteResultError(requested []oss.DeleteObject, out *oss.DeleteMultipleObjectsResult) error {\n\tif out == nil {\n\t\treturn nil\n\t}\n\n\t// Build set of deleted keys for quick lookup\n\tdeleted := make(map[string]struct{}, len(out.DeletedObjects))\n\tfor _, obj := range out.DeletedObjects {\n\t\tif obj.Key != nil {\n\t\t\tdeleted[*obj.Key] = struct{}{}\n\t\t}\n\t}\n\n\t// Check that all requested keys were deleted\n\tvar failed []string\n\tfor _, obj := range requested {\n\t\tif obj.Key == nil {\n\t\t\tcontinue\n\t\t}\n\t\tif _, ok := deleted[*obj.Key]; !ok {\n\t\t\tfailed = append(failed, *obj.Key)\n\t\t}\n\t}\n\n\tif len(failed) == 0 {\n\t\treturn nil\n\t}\n\n\t// Build error message listing failed keys\n\tvar b strings.Builder\n\tb.WriteString(\"oss: failed to delete files:\")\n\tfor _, key := range failed {\n\t\tfmt.Fprintf(&b, \"\\n%s\", key)\n\t}\n\treturn errors.New(b.String())\n}\n" }, { "path": "oss/replica_client_test.go", "content": "package oss\n\nimport (\n\t\"errors\"\n\t\"testing\"\n\n\t\"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss\"\n)\n\nfunc TestReplicaClient_Type(t *testing.T) {\n\tc := NewReplicaClient()\n\tif got := c.Type(); got != ReplicaClientType {\n\t\tt.Errorf(\"Type() = %q, want %q\", got, ReplicaClientType)\n\t}\n\tif got := c.Type(); got != \"oss\" {\n\t\tt.Errorf(\"Type() = %q, want %q\", got, \"oss\")\n\t}\n}\n\nfunc TestReplicaClient_Init_BucketValidation(t *testing.T) {\n\tt.Run(\"EmptyBucket\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"\" // Empty bucket name\n\t\tc.Region = \"cn-hangzhou\"\n\n\t\terr := c.Init(t.Context())\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for empty bucket name\")\n\t\t}\n\t\tif got := err.Error(); got != \"oss: bucket name is required\" {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ValidBucketWithRegion\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"cn-hangzhou\"\n\t\tc.AccessKeyID = \"test-key\"\n\t\tc.AccessKeySecret = \"test-secret\"\n\n\t\t// Init should succeed (client will be created even without real credentials)\n\t\terr := c.Init(t.Context())\n\t\tif err != nil {\n\t\t\tt.Errorf(\"Init() should succeed with valid bucket: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ValidBucketDefaultRegion\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\t// Region is empty, should use DefaultRegion\n\t\tc.AccessKeyID = \"test-key\"\n\t\tc.AccessKeySecret = \"test-secret\"\n\n\t\terr := c.Init(t.Context())\n\t\tif err != nil {\n\t\t\tt.Errorf(\"Init() should succeed with default region: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestReplicaClient_Init_Idempotent(t *testing.T) {\n\tc := NewReplicaClient()\n\tc.Bucket = \"test-bucket\"\n\tc.AccessKeyID = \"test-key\"\n\tc.AccessKeySecret = \"test-secret\"\n\n\t// First init\n\tif err := c.Init(t.Context()); err != nil {\n\t\tt.Fatalf(\"first Init() failed: %v\", err)\n\t}\n\n\t// Second init should be a no-op\n\tif err := c.Init(t.Context()); err != nil {\n\t\tt.Fatalf(\"second Init() failed: %v\", err)\n\t}\n}\n\nfunc TestParseURL(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\turl string\n\t\twantBucket string\n\t\twantRegion string\n\t\twantKey string\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"SimpleOSSURL\",\n\t\t\turl: \"oss://my-bucket/path/to/file\",\n\t\t\twantBucket: \"my-bucket\",\n\t\t\twantRegion: \"\",\n\t\t\twantKey: \"path/to/file\",\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"OSSURLWithRegion\",\n\t\t\turl: \"oss://my-bucket.oss-cn-hangzhou.aliyuncs.com/backup\",\n\t\t\twantBucket: \"my-bucket\",\n\t\t\twantRegion: \"cn-hangzhou\",\n\t\t\twantKey: \"backup\",\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"OSSURLNoPath\",\n\t\t\turl: \"oss://my-bucket\",\n\t\t\twantBucket: \"my-bucket\",\n\t\t\twantRegion: \"\",\n\t\t\twantKey: \"\",\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"InvalidScheme\",\n\t\t\turl: \"s3://my-bucket/path\",\n\t\t\twantBucket: \"\",\n\t\t\twantRegion: \"\",\n\t\t\twantKey: \"\",\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"HTTPScheme\",\n\t\t\turl: \"http://my-bucket/path\",\n\t\t\twantBucket: \"\",\n\t\t\twantRegion: \"\",\n\t\t\twantKey: \"\",\n\t\t\twantErr: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tbucket, region, key, err := ParseURL(tt.url)\n\n\t\t\tif (err != nil) != tt.wantErr {\n\t\t\t\tt.Errorf(\"ParseURL() error = %v, wantErr %v\", err, tt.wantErr)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif bucket != tt.wantBucket {\n\t\t\t\tt.Errorf(\"bucket = %q, want %q\", bucket, tt.wantBucket)\n\t\t\t}\n\t\t\tif region != tt.wantRegion {\n\t\t\t\tt.Errorf(\"region = %q, want %q\", region, tt.wantRegion)\n\t\t\t}\n\t\t\tif key != tt.wantKey {\n\t\t\t\tt.Errorf(\"key = %q, want %q\", key, tt.wantKey)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestParseHost(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\thost string\n\t\twantBucket string\n\t\twantRegion string\n\t}{\n\t\t{\n\t\t\tname: \"StandardOSSURL\",\n\t\t\thost: \"my-bucket.oss-cn-hangzhou.aliyuncs.com\",\n\t\t\twantBucket: \"my-bucket\",\n\t\t\twantRegion: \"cn-hangzhou\",\n\t\t},\n\t\t{\n\t\t\tname: \"OSSURLBeijingRegion\",\n\t\t\thost: \"test-bucket.oss-cn-beijing.aliyuncs.com\",\n\t\t\twantBucket: \"test-bucket\",\n\t\t\twantRegion: \"cn-beijing\",\n\t\t},\n\t\t{\n\t\t\tname: \"OSSURLShanghaiRegion\",\n\t\t\thost: \"data-bucket.oss-cn-shanghai.aliyuncs.com\",\n\t\t\twantBucket: \"data-bucket\",\n\t\t\twantRegion: \"cn-shanghai\",\n\t\t},\n\t\t{\n\t\t\tname: \"InternalOSSURL\",\n\t\t\thost: \"my-bucket.oss-cn-hangzhou-internal.aliyuncs.com\",\n\t\t\twantBucket: \"my-bucket\",\n\t\t\twantRegion: \"cn-hangzhou\",\n\t\t},\n\t\t{\n\t\t\tname: \"InternalOSSURLBeijing\",\n\t\t\thost: \"test-bucket.oss-cn-beijing-internal.aliyuncs.com\",\n\t\t\twantBucket: \"test-bucket\",\n\t\t\twantRegion: \"cn-beijing\",\n\t\t},\n\t\t{\n\t\t\tname: \"SimpleBucketName\",\n\t\t\thost: \"my-bucket\",\n\t\t\twantBucket: \"my-bucket\",\n\t\t\twantRegion: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"BucketWithHyphens\",\n\t\t\thost: \"my-test-bucket-2024.oss-cn-shenzhen.aliyuncs.com\",\n\t\t\twantBucket: \"my-test-bucket-2024\",\n\t\t\twantRegion: \"cn-shenzhen\",\n\t\t},\n\t\t{\n\t\t\tname: \"OSSURLWithNumbers\",\n\t\t\thost: \"bucket123.oss-cn-hangzhou.aliyuncs.com\",\n\t\t\twantBucket: \"bucket123\",\n\t\t\twantRegion: \"cn-hangzhou\",\n\t\t},\n\t\t{\n\t\t\tname: \"OSSURLHongKong\",\n\t\t\thost: \"hk-bucket.oss-cn-hongkong.aliyuncs.com\",\n\t\t\twantBucket: \"hk-bucket\",\n\t\t\twantRegion: \"cn-hongkong\",\n\t\t},\n\t\t{\n\t\t\tname: \"OSSURLSingapore\",\n\t\t\thost: \"sg-bucket.oss-ap-southeast-1.aliyuncs.com\",\n\t\t\twantBucket: \"sg-bucket\",\n\t\t\twantRegion: \"ap-southeast-1\",\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tbucket, region, _ := ParseHost(tt.host)\n\n\t\t\tif bucket != tt.wantBucket {\n\t\t\t\tt.Errorf(\"bucket = %q, want %q\", bucket, tt.wantBucket)\n\t\t\t}\n\t\t\tif region != tt.wantRegion {\n\t\t\t\tt.Errorf(\"region = %q, want %q\", region, tt.wantRegion)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsNotExists(t *testing.T) {\n\tt.Run(\"NilError\", func(t *testing.T) {\n\t\tif isNotExists(nil) {\n\t\t\tt.Error(\"isNotExists should return false for nil error\")\n\t\t}\n\t})\n\n\tt.Run(\"RegularError\", func(t *testing.T) {\n\t\tregularErr := errors.New(\"regular error\")\n\t\tif isNotExists(regularErr) {\n\t\t\tt.Error(\"isNotExists should return false for regular error\")\n\t\t}\n\t})\n\n\tt.Run(\"WrappedError\", func(t *testing.T) {\n\t\twrappedErr := errors.New(\"wrapped: something went wrong\")\n\t\tif isNotExists(wrappedErr) {\n\t\t\tt.Error(\"isNotExists should return false for wrapped non-ServiceError\")\n\t\t}\n\t})\n}\n\nfunc TestLtxPath(t *testing.T) {\n\tc := NewReplicaClient()\n\tc.Path = \"backups\"\n\n\ttests := []struct {\n\t\tlevel int\n\t\tfilename string\n\t\twant string\n\t}{\n\t\t{0, \"00000001-00000001.ltx\", \"backups/0000/00000001-00000001.ltx\"},\n\t\t{1, \"00000001-00000010.ltx\", \"backups/0001/00000001-00000010.ltx\"},\n\t\t{15, \"00000001-000000ff.ltx\", \"backups/000f/00000001-000000ff.ltx\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.want, func(t *testing.T) {\n\t\t\tgot := c.ltxPath(tt.level, tt.filename)\n\t\t\tif got != tt.want {\n\t\t\t\tt.Errorf(\"ltxPath(%d, %q) = %q, want %q\", tt.level, tt.filename, got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDeleteResultError(t *testing.T) {\n\tptr := func(s string) *string { return &s }\n\n\tt.Run(\"NilResult\", func(t *testing.T) {\n\t\trequested := []oss.DeleteObject{{Key: ptr(\"key1\")}}\n\t\tif err := deleteResultError(requested, nil); err != nil {\n\t\t\tt.Errorf(\"expected nil error for nil result, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"AllDeleted\", func(t *testing.T) {\n\t\trequested := []oss.DeleteObject{\n\t\t\t{Key: ptr(\"key1\")},\n\t\t\t{Key: ptr(\"key2\")},\n\t\t}\n\t\tresult := &oss.DeleteMultipleObjectsResult{\n\t\t\tDeletedObjects: []oss.DeletedInfo{\n\t\t\t\t{Key: ptr(\"key1\")},\n\t\t\t\t{Key: ptr(\"key2\")},\n\t\t\t},\n\t\t}\n\t\tif err := deleteResultError(requested, result); err != nil {\n\t\t\tt.Errorf(\"expected nil error when all deleted, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"SomeNotDeleted\", func(t *testing.T) {\n\t\trequested := []oss.DeleteObject{\n\t\t\t{Key: ptr(\"key1\")},\n\t\t\t{Key: ptr(\"key2\")},\n\t\t\t{Key: ptr(\"key3\")},\n\t\t}\n\t\tresult := &oss.DeleteMultipleObjectsResult{\n\t\t\tDeletedObjects: []oss.DeletedInfo{\n\t\t\t\t{Key: ptr(\"key1\")},\n\t\t\t\t// key2 and key3 not deleted\n\t\t\t},\n\t\t}\n\t\terr := deleteResultError(requested, result)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when some keys not deleted\")\n\t\t}\n\t\terrStr := err.Error()\n\t\tif !contains(errStr, \"key2\") {\n\t\t\tt.Errorf(\"error should mention key2: %s\", errStr)\n\t\t}\n\t\tif !contains(errStr, \"key3\") {\n\t\t\tt.Errorf(\"error should mention key3: %s\", errStr)\n\t\t}\n\t})\n\n\tt.Run(\"EmptyRequested\", func(t *testing.T) {\n\t\trequested := []oss.DeleteObject{}\n\t\tresult := &oss.DeleteMultipleObjectsResult{}\n\t\tif err := deleteResultError(requested, result); err != nil {\n\t\t\tt.Errorf(\"expected nil error for empty requested, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NilKeyInRequested\", func(t *testing.T) {\n\t\trequested := []oss.DeleteObject{\n\t\t\t{Key: nil}, // nil key should be skipped\n\t\t\t{Key: ptr(\"key1\")},\n\t\t}\n\t\tresult := &oss.DeleteMultipleObjectsResult{\n\t\t\tDeletedObjects: []oss.DeletedInfo{\n\t\t\t\t{Key: ptr(\"key1\")},\n\t\t\t},\n\t\t}\n\t\tif err := deleteResultError(requested, result); err != nil {\n\t\t\tt.Errorf(\"expected nil error, got %v\", err)\n\t\t}\n\t})\n}\n\nfunc contains(s, substr string) bool {\n\treturn len(s) >= len(substr) && (s == substr || len(s) > 0 && containsHelper(s, substr))\n}\n\nfunc containsHelper(s, substr string) bool {\n\tfor i := 0; i <= len(s)-len(substr); i++ {\n\t\tif s[i:i+len(substr)] == substr {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n" }, { "path": "packages/npm/litestream-vfs/README.md", "content": "# litestream-vfs\n\nLitestream VFS extension for SQLite — distributed via npm.\n\nThis package bundles the [Litestream](https://litestream.io) VFS shared library.\nThe correct platform-specific binary is automatically installed via `optionalDependencies`.\n\n## Installation\n\n```bash\nnpm install litestream-vfs\n```\n\n## Usage\n\n```javascript\nconst { getLoadablePath } = require(\"litestream-vfs\");\nconst path = getLoadablePath();\n// Use `path` with better-sqlite3 or other SQLite bindings\n```\n\n## Platform Support\n\n| Platform | Architecture |\n|----------|-------------|\n| Linux | x86_64, aarch64 |\n| macOS | x86_64, arm64 |\n\n## License\n\nApache-2.0 — see [LICENSE](https://github.com/benbjohnson/litestream/blob/main/LICENSE).\n" }, { "path": "packages/npm/litestream-vfs/index.js", "content": "\"use strict\";\n\nconst path = require(\"path\");\nconst os = require(\"os\");\n\nconst PLATFORM_PACKAGES = {\n \"darwin-arm64\": \"litestream-vfs-darwin-arm64\",\n \"darwin-x64\": \"litestream-vfs-darwin-amd64\",\n \"linux-arm64\": \"litestream-vfs-linux-arm64\",\n \"linux-x64\": \"litestream-vfs-linux-amd64\",\n};\n\nconst EXT_MAP = {\n darwin: \"litestream-vfs.dylib\",\n linux: \"litestream-vfs.so\",\n};\n\nfunction getLoadablePath() {\n const key = `${os.platform()}-${os.arch()}`;\n const pkg = PLATFORM_PACKAGES[key];\n if (!pkg) {\n throw new Error(`Unsupported platform: ${key}`);\n }\n\n const ext = EXT_MAP[os.platform()];\n const searchPaths = [\n path.join(process.cwd(), \"node_modules\"),\n ...module.paths,\n ];\n if (require.main) {\n searchPaths.push(...require.main.paths);\n }\n try {\n const resolved = require.resolve(`${pkg}/package.json`, {\n paths: searchPaths,\n });\n return path.join(path.dirname(resolved), ext);\n } catch {\n throw new Error(\n `Platform package ${pkg} is not installed. ` +\n `Run: npm install ${pkg}`\n );\n }\n}\n\nmodule.exports = { getLoadablePath };\n" }, { "path": "packages/npm/litestream-vfs/package.json", "content": "{\n \"name\": \"litestream-vfs\",\n \"version\": \"0.0.0\",\n \"description\": \"Litestream VFS extension for SQLite\",\n \"main\": \"index.js\",\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"https://github.com/benbjohnson/litestream\"\n },\n \"license\": \"Apache-2.0\",\n \"keywords\": [\"sqlite\", \"litestream\", \"vfs\", \"replication\"],\n \"optionalDependencies\": {\n \"litestream-vfs-darwin-arm64\": \"0.0.0\",\n \"litestream-vfs-darwin-amd64\": \"0.0.0\",\n \"litestream-vfs-linux-arm64\": \"0.0.0\",\n \"litestream-vfs-linux-amd64\": \"0.0.0\"\n }\n}\n" }, { "path": "packages/npm/litestream-vfs-darwin-amd64/README.md", "content": "# litestream-vfs-darwin-amd64\n\nPlatform package for `litestream-vfs` — macOS x64 (Intel).\n\nThis package is installed automatically by `litestream-vfs`. You should not need to install it directly.\n" }, { "path": "packages/npm/litestream-vfs-darwin-amd64/package.json", "content": "{\n \"name\": \"litestream-vfs-darwin-amd64\",\n \"version\": \"0.0.0\",\n \"description\": \"Litestream VFS extension for macOS x64\",\n \"os\": [\"darwin\"],\n \"cpu\": [\"x64\"],\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"https://github.com/benbjohnson/litestream\"\n },\n \"license\": \"Apache-2.0\"\n}\n" }, { "path": "packages/npm/litestream-vfs-darwin-arm64/README.md", "content": "# litestream-vfs-darwin-arm64\n\nPlatform package for `litestream-vfs` — macOS ARM64 (Apple Silicon).\n\nThis package is installed automatically by `litestream-vfs`. You should not need to install it directly.\n" }, { "path": "packages/npm/litestream-vfs-darwin-arm64/package.json", "content": "{\n \"name\": \"litestream-vfs-darwin-arm64\",\n \"version\": \"0.0.0\",\n \"description\": \"Litestream VFS extension for macOS ARM64\",\n \"os\": [\"darwin\"],\n \"cpu\": [\"arm64\"],\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"https://github.com/benbjohnson/litestream\"\n },\n \"license\": \"Apache-2.0\"\n}\n" }, { "path": "packages/npm/litestream-vfs-linux-amd64/README.md", "content": "# litestream-vfs-linux-amd64\n\nPlatform package for `litestream-vfs` — Linux x64.\n\nThis package is installed automatically by `litestream-vfs`. You should not need to install it directly.\n" }, { "path": "packages/npm/litestream-vfs-linux-amd64/package.json", "content": "{\n \"name\": \"litestream-vfs-linux-amd64\",\n \"version\": \"0.0.0\",\n \"description\": \"Litestream VFS extension for Linux x64\",\n \"os\": [\"linux\"],\n \"cpu\": [\"x64\"],\n \"libc\": [\"glibc\"],\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"https://github.com/benbjohnson/litestream\"\n },\n \"license\": \"Apache-2.0\"\n}\n" }, { "path": "packages/npm/litestream-vfs-linux-arm64/README.md", "content": "# litestream-vfs-linux-arm64\n\nPlatform package for `litestream-vfs` — Linux ARM64.\n\nThis package is installed automatically by `litestream-vfs`. You should not need to install it directly.\n" }, { "path": "packages/npm/litestream-vfs-linux-arm64/package.json", "content": "{\n \"name\": \"litestream-vfs-linux-arm64\",\n \"version\": \"0.0.0\",\n \"description\": \"Litestream VFS extension for Linux ARM64\",\n \"os\": [\"linux\"],\n \"cpu\": [\"arm64\"],\n \"libc\": [\"glibc\"],\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"https://github.com/benbjohnson/litestream\"\n },\n \"license\": \"Apache-2.0\"\n}\n" }, { "path": "packages/python/MANIFEST.in", "content": "include README.md\nrecursive-include litestream_vfs *.so *.dylib\n" }, { "path": "packages/python/README.md", "content": "# litestream-vfs\n\nLitestream VFS extension for SQLite — distributed as a Python wheel.\n\nThis package bundles the [Litestream](https://litestream.io) VFS shared library\nso you can load it directly into a Python `sqlite3` connection.\n\n## Installation\n\n```bash\npip install litestream-vfs\n```\n\n## Usage\n\n```python\nimport sqlite3\nimport litestream_vfs\n\nconn = sqlite3.connect(\":memory:\")\nlitestream_vfs.load(conn)\n```\n\nTo get the path to the shared library (for use with other SQLite bindings):\n\n```python\npath = litestream_vfs.loadable_path()\n```\n\n## Platform Support\n\n| Platform | Architecture |\n|----------|-------------|\n| Linux | x86_64, aarch64 |\n| macOS | x86_64, arm64 |\n\n## License\n\nApache-2.0 — see [LICENSE](https://github.com/benbjohnson/litestream/blob/main/LICENSE).\n" }, { "path": "packages/python/litestream_vfs/__init__.py", "content": "\"\"\"Litestream VFS extension for SQLite.\"\"\"\n\nimport os\nimport sys\n\n_EXT_MAP = {\n \"linux\": \"litestream-vfs.so\",\n \"darwin\": \"litestream-vfs.dylib\",\n}\n\n\ndef loadable_path():\n \"\"\"Return the filesystem path to the loadable VFS extension.\"\"\"\n platform = sys.platform\n if platform.startswith(\"linux\"):\n platform = \"linux\"\n filename = _EXT_MAP.get(platform)\n if filename is None:\n raise OSError(f\"Unsupported platform: {sys.platform}\")\n path = os.path.join(os.path.dirname(__file__), filename)\n if not os.path.exists(path):\n raise FileNotFoundError(f\"VFS extension not found at {path}\")\n return path\n\n\ndef load(conn):\n \"\"\"Load the Litestream VFS extension into a sqlite3 connection.\"\"\"\n path = loadable_path()\n conn.enable_load_extension(True)\n try:\n conn.load_extension(os.path.splitext(path)[0])\n finally:\n conn.enable_load_extension(False)\n" }, { "path": "packages/python/litestream_vfs/noop.c", "content": "/* Dummy C extension to force wheel platform tagging. */\n#include \n\nstatic PyMethodDef methods[] = {{NULL, NULL, 0, NULL}};\n\nstatic struct PyModuleDef module = {\n PyModuleDef_HEAD_INIT, \"_noop\", NULL, -1, methods,\n};\n\nPyMODINIT_FUNC PyInit__noop(void) { return PyModule_Create(&module); }\n" }, { "path": "packages/python/scripts/rename_wheel.py", "content": "#!/usr/bin/env python3\n\"\"\"Rename a wheel file with the correct platform tag.\n\nUsage: python rename_wheel.py \n platform_tag examples: manylinux_2_35_x86_64, manylinux_2_35_aarch64,\n macosx_11_0_x86_64, macosx_11_0_arm64\n\"\"\"\nimport glob\nimport os\nimport sys\n\n\ndef main():\n wheel_dir = sys.argv[1]\n platform_tag = sys.argv[2]\n\n wheels = glob.glob(os.path.join(wheel_dir, \"*.whl\"))\n if not wheels:\n print(f\"No wheels found in {wheel_dir}\", file=sys.stderr)\n sys.exit(1)\n\n for wheel in wheels:\n parts = os.path.basename(wheel).split(\"-\")\n # Wheel filename: {name}-{ver}-{python}-{abi}-{platform}.whl\n parts[-1] = f\"{platform_tag}.whl\"\n parts[-2] = \"none\"\n parts[-3] = \"cp38.cp39.cp310.cp311.cp312.cp313\"\n new_name = \"-\".join(parts)\n new_path = os.path.join(wheel_dir, new_name)\n os.rename(wheel, new_path)\n print(f\"Renamed: {os.path.basename(wheel)} -> {new_name}\")\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "packages/python/setup.py", "content": "import os\nfrom setuptools import setup, Extension\n\nsetup(\n name=\"litestream-vfs\",\n version=os.environ.get(\"LITESTREAM_VERSION\", \"0.0.0\"),\n description=\"Litestream VFS extension for SQLite\",\n long_description=open(\"README.md\").read(),\n long_description_content_type=\"text/markdown\",\n url=\"https://github.com/benbjohnson/litestream\",\n license=\"Apache-2.0\",\n packages=[\"litestream_vfs\"],\n package_data={\"litestream_vfs\": [\"*.so\", \"*.dylib\"]},\n ext_modules=[Extension(\"litestream_vfs._noop\", [\"litestream_vfs/noop.c\"])],\n python_requires=\">=3.8\",\n classifiers=[\n \"Development Status :: 4 - Beta\",\n \"Intended Audience :: Developers\",\n \"License :: OSI Approved :: Apache Software License\",\n \"Programming Language :: Python :: 3\",\n \"Topic :: Database\",\n ],\n)\n" }, { "path": "packages/ruby/README.md", "content": "# litestream-vfs\n\nLitestream VFS extension for SQLite — distributed as a Ruby gem.\n\nThis gem bundles the [Litestream](https://litestream.io) VFS shared library\nso you can load it directly into a SQLite connection.\n\n## Installation\n\n```bash\ngem install litestream-vfs\n```\n\nOr add to your Gemfile:\n\n```ruby\ngem \"litestream-vfs\"\n```\n\n## Usage\n\n```ruby\nrequire \"litestream_vfs\"\n\ndb = SQLite3::Database.new(\":memory:\")\nLitestreamVfs.load(db)\n```\n\nTo get the path to the shared library:\n\n```ruby\npath = LitestreamVfs.loadable_path\n```\n\n## Platform Support\n\n| Platform | Architecture |\n|----------|-------------|\n| Linux | x86_64, aarch64 |\n| macOS | x86_64, arm64 |\n\n## License\n\nApache-2.0 — see [LICENSE](https://github.com/benbjohnson/litestream/blob/main/LICENSE).\n" }, { "path": "packages/ruby/lib/litestream_vfs.rb", "content": "require \"rbconfig\"\n\nmodule LitestreamVfs\n EXT_MAP = {\n \"linux\" => \"litestream-vfs.so\",\n \"darwin\" => \"litestream-vfs.dylib\",\n }.freeze\n\n def self.loadable_path\n os = RbConfig::CONFIG[\"host_os\"]\n key = case os\n when /linux/ then \"linux\"\n when /darwin/i then \"darwin\"\n else raise \"Unsupported platform: #{os}\"\n end\n\n filename = EXT_MAP.fetch(key)\n path = File.join(__dir__, filename)\n raise \"VFS extension not found at #{path}\" unless File.exist?(path)\n path\n end\n\n def self.load(db)\n db.enable_load_extension(true)\n db.load_extension(loadable_path.delete_suffix(File.extname(loadable_path)))\n db.enable_load_extension(false)\n end\nend\n" }, { "path": "packages/ruby/litestream-vfs.gemspec", "content": "Gem::Specification.new do |s|\n s.name = \"litestream-vfs\"\n s.version = ENV.fetch(\"LITESTREAM_VERSION\", \"0.0.0\")\n s.summary = \"Litestream VFS extension for SQLite\"\n s.description = \"Bundles the Litestream VFS shared library for loading into SQLite connections.\"\n s.homepage = \"https://github.com/benbjohnson/litestream\"\n s.license = \"Apache-2.0\"\n s.authors = [\"Ben Johnson\"]\n\n s.platform = Gem::Platform.new(ENV.fetch(\"PLATFORM\", RUBY_PLATFORM))\n\n s.files = Dir[\"lib/**/*.rb\"] + Dir[\"lib/**/*.so\"] + Dir[\"lib/**/*.dylib\"]\n s.require_paths = [\"lib\"]\n\n s.required_ruby_version = \">= 2.7\"\nend\n" }, { "path": "replica.go", "content": "package litestream\n\nimport (\n\t\"context\"\n\t\"crypto/rand\"\n\t\"database/sql\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\n// Default replica settings.\nconst (\n\tDefaultSyncInterval = 1 * time.Second\n)\n\n// Replica connects a database to a replication destination via a ReplicaClient.\n// The replica manages periodic synchronization and maintaining the current\n// replica position.\ntype Replica struct {\n\tdb *DB\n\n\tmu sync.RWMutex\n\tpos ltx.Pos // current replicated position\n\n\tsyncMu sync.Mutex // protects Sync() from concurrent calls\n\n\tmuf sync.Mutex\n\tf *os.File // long-running file descriptor to avoid non-OFD lock issues\n\n\twg sync.WaitGroup\n\tcancel func()\n\n\t// Client used to connect to the remote replica.\n\tClient ReplicaClient\n\n\t// Time between syncs with the shadow WAL.\n\tSyncInterval time.Duration\n\n\t// If true, replica monitors database for changes automatically.\n\t// Set to false if replica is being used synchronously (such as in tests).\n\tMonitorEnabled bool\n\n\t// If true, automatically reset local state when LTX errors are detected.\n\t// This allows recovery from corrupted/missing LTX files by resetting\n\t// the position file and removing local LTX files, forcing a fresh sync.\n\t// Disabled by default to prevent silent data loss scenarios.\n\tAutoRecoverEnabled bool\n}\n\nfunc NewReplica(db *DB) *Replica {\n\tr := &Replica{\n\t\tdb: db,\n\t\tcancel: func() {},\n\n\t\tSyncInterval: DefaultSyncInterval,\n\t\tMonitorEnabled: true,\n\t}\n\n\treturn r\n}\n\nfunc NewReplicaWithClient(db *DB, client ReplicaClient) *Replica {\n\tr := NewReplica(db)\n\tr.Client = client\n\treturn r\n}\n\n// Logger returns the DB sub-logger for this replica.\nfunc (r *Replica) Logger() *slog.Logger {\n\tlogger := slog.Default()\n\tif r.db != nil {\n\t\tlogger = r.db.Logger\n\t}\n\treturn logger.With(\"replica\", r.Client.Type())\n}\n\n// DB returns a reference to the database the replica is attached to, if any.\nfunc (r *Replica) DB() *DB { return r.db }\n\n// Starts replicating in a background goroutine.\nfunc (r *Replica) Start(ctx context.Context) error {\n\t// Ignore if replica is being used sychronously.\n\tif !r.MonitorEnabled {\n\t\treturn nil\n\t}\n\n\t// Stop previous replication.\n\tr.Stop(false)\n\n\t// Wrap context with cancelation.\n\tctx, r.cancel = context.WithCancel(ctx)\n\n\t// Start goroutine to replicate data.\n\tr.wg.Add(1)\n\tgo func() { defer r.wg.Done(); r.monitor(ctx) }()\n\n\treturn nil\n}\n\n// Stop cancels any outstanding replication and blocks until finished.\n//\n// Performing a hard stop will close the DB file descriptor which could release\n// locks on per-process locks. Hard stops should only be performed when\n// stopping the entire process.\nfunc (r *Replica) Stop(hard bool) (err error) {\n\tr.cancel()\n\tr.wg.Wait()\n\n\tr.muf.Lock()\n\tdefer r.muf.Unlock()\n\tif hard && r.f != nil {\n\t\tif e := r.f.Close(); e != nil && err == nil {\n\t\t\terr = e\n\t\t}\n\t}\n\treturn err\n}\n\n// Sync copies new WAL frames from the shadow WAL to the replica client.\n// Only one Sync can run at a time to prevent concurrent uploads of the same file.\nfunc (r *Replica) Sync(ctx context.Context) (err error) {\n\tr.syncMu.Lock()\n\tdefer r.syncMu.Unlock()\n\n\t// Clear last position if if an error occurs during sync.\n\tdefer func() {\n\t\tif err != nil {\n\t\t\tr.mu.Lock()\n\t\t\tr.pos = ltx.Pos{}\n\t\t\tr.mu.Unlock()\n\t\t}\n\t}()\n\n\t// Calculate current replica position, if unknown.\n\tif r.Pos().IsZero() {\n\t\tpos, err := r.calcPos(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"calc pos: %w\", err)\n\t\t}\n\t\tr.SetPos(pos)\n\t}\n\n\t// Find current position of database.\n\tdpos, err := r.db.Pos()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot determine current position: %w\", err)\n\t} else if dpos.IsZero() {\n\t\treturn fmt.Errorf(\"no position, waiting for data\")\n\t}\n\n\tr.Logger().Info(\"replica sync\",\n\t\tslog.Group(\"txid\",\n\t\t\tslog.String(\"replica\", r.Pos().TXID.String()),\n\t\t\tslog.String(\"db\", dpos.TXID.String()),\n\t\t))\n\n\t// Replicate all L0 LTX files since last replica position.\n\tfor txID := r.Pos().TXID + 1; txID <= dpos.TXID; txID = r.Pos().TXID + 1 {\n\t\tif err := r.uploadLTXFile(ctx, 0, txID, txID); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tr.SetPos(ltx.Pos{TXID: txID})\n\t}\n\n\t// Record successful sync for heartbeat monitoring.\n\tr.db.RecordSuccessfulSync()\n\n\treturn nil\n}\n\nfunc (r *Replica) uploadLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID) (err error) {\n\tfilename := r.db.LTXPath(level, minTXID, maxTXID)\n\tf, err := os.Open(filename)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer func() { _ = f.Close() }()\n\n\tinfo, err := r.Client.WriteLTXFile(ctx, level, minTXID, maxTXID, f)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"write ltx file: %w\", err)\n\t}\n\tr.Logger().Info(\"ltx file uploaded\",\n\t\t\"level\", info.Level,\n\t\t\"minTXID\", info.MinTXID,\n\t\t\"maxTXID\", info.MaxTXID,\n\t\t\"size\", info.Size)\n\n\t// Track current position\n\t//replicaWALIndexGaugeVec.WithLabelValues(r.db.Path(), r.Name()).Set(float64(rd.Pos().Index))\n\t//replicaWALOffsetGaugeVec.WithLabelValues(r.db.Path(), r.Name()).Set(float64(rd.Pos().Offset))\n\n\treturn nil\n}\n\n// calcPos returns the last position saved to the replica for level 0.\nfunc (r *Replica) calcPos(ctx context.Context) (pos ltx.Pos, err error) {\n\tinfo, err := r.MaxLTXFileInfo(ctx, 0)\n\tif err != nil {\n\t\treturn pos, fmt.Errorf(\"max ltx file: %w\", err)\n\t}\n\treturn ltx.Pos{TXID: info.MaxTXID}, nil\n}\n\n// MaxLTXFileInfo returns metadata about the last LTX file for a given level.\n// Returns nil if no files exist for the level.\nfunc (r *Replica) MaxLTXFileInfo(ctx context.Context, level int) (info ltx.FileInfo, err error) {\n\t// Normal operation - use fast timestamps\n\titr, err := r.Client.LTXFiles(ctx, level, 0, false)\n\tif err != nil {\n\t\treturn info, err\n\t}\n\tdefer itr.Close()\n\n\tfor itr.Next() {\n\t\titem := itr.Item()\n\t\tif item.MaxTXID > info.MaxTXID {\n\t\t\tinfo = *item\n\t\t}\n\t}\n\treturn info, itr.Close()\n}\n\n// Pos returns the current replicated position.\n// Returns a zero value if the current position cannot be determined.\nfunc (r *Replica) Pos() ltx.Pos {\n\tr.mu.RLock()\n\tdefer r.mu.RUnlock()\n\treturn r.pos\n}\n\n// SetPos sets the current replicated position.\nfunc (r *Replica) SetPos(pos ltx.Pos) {\n\tr.mu.Lock()\n\tdefer r.mu.Unlock()\n\tr.pos = pos\n}\n\n// EnforceRetention forces a new snapshot once the retention interval has passed.\n// Older snapshots and WAL files are then removed.\nfunc (r *Replica) EnforceRetention(ctx context.Context) (err error) {\n\tpanic(\"TODO(ltx): Re-implement after multi-level compaction\")\n\n\t/*\n\t\t// Obtain list of snapshots that are within the retention period.\n\t\tsnapshots, err := r.Snapshots(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"snapshots: %w\", err)\n\t\t}\n\t\tretained := FilterSnapshotsAfter(snapshots, time.Now().Add(-r.Retention))\n\n\t\t// If no retained snapshots exist, create a new snapshot.\n\t\tif len(retained) == 0 {\n\t\t\tsnapshot, err := r.Snapshot(ctx)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"snapshot: %w\", err)\n\t\t\t}\n\t\t\tretained = append(retained, snapshot)\n\t\t}\n\n\t\t// Delete unretained snapshots & WAL files.\n\t\tsnapshot := FindMinSnapshot(retained)\n\n\t\t// Otherwise remove all earlier snapshots & WAL segments.\n\t\tif err := r.deleteSnapshotsBeforeIndex(ctx, snapshot.Index); err != nil {\n\t\t\treturn fmt.Errorf(\"delete snapshots before index: %w\", err)\n\t\t} else if err := r.deleteWALSegmentsBeforeIndex(ctx, snapshot.Index); err != nil {\n\t\t\treturn fmt.Errorf(\"delete wal segments before index: %w\", err)\n\t\t}\n\n\t\treturn nil\n\t*/\n}\n\n/*\nfunc (r *Replica) deleteBeforeTXID(ctx context.Context, level int, txID ltx.TXID) error {\n\titr, err := r.Client.LTXFiles(ctx, level)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"fetch ltx files: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar a []*ltx.FileInfo\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tif info.MinTXID >= txID {\n\t\t\tcontinue\n\t\t}\n\t\ta = append(a, info)\n\t}\n\tif err := itr.Close(); err != nil {\n\t\treturn err\n\t}\n\n\tif len(a) == 0 {\n\t\treturn nil\n\t}\n\n\tif err := r.Client.DeleteLTXFiles(ctx, a); err != nil {\n\t\treturn fmt.Errorf(\"delete wal segments: %w\", err)\n\t}\n\n\tr.Logger().Info(\"ltx files deleted before\",\n\t\tslog.Int(\"level\", level),\n\t\tslog.String(\"txID\", txID.String()),\n\t\tslog.Int(\"n\", len(a)))\n\n\treturn nil\n}\n*/\n\n// monitor runs in a separate goroutine and continuously replicates the DB.\n// Implements exponential backoff on repeated sync errors to prevent log spam\n// and reduce load when persistent errors occur. See issue #927.\nfunc (r *Replica) monitor(ctx context.Context) {\n\tticker := time.NewTicker(r.SyncInterval)\n\tdefer ticker.Stop()\n\n\t// Continuously check for new data to replicate.\n\tch := make(chan struct{})\n\tclose(ch)\n\tvar notify <-chan struct{} = ch\n\n\tvar backoff time.Duration\n\tvar lastLogTime time.Time\n\tvar consecutiveErrs int\n\n\tfor initial := true; ; initial = false {\n\t\t// Enforce a minimum time between synchronization.\n\t\tif !initial {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tcase <-ticker.C:\n\t\t\t}\n\t\t}\n\n\t\t// If in backoff mode, wait additional time before retrying.\n\t\tif backoff > 0 {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tcase <-time.After(backoff):\n\t\t\t}\n\t\t}\n\n\t\t// Wait for changes to the database.\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn\n\t\tcase <-notify:\n\t\t}\n\n\t\t// Fetch new notify channel before replicating data.\n\t\tnotify = r.db.Notify()\n\n\t\t// Synchronize the shadow wal into the replication directory.\n\t\tif err := r.Sync(ctx); err != nil {\n\t\t\t// Don't log context cancellation errors during shutdown\n\t\t\tif !errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded) {\n\t\t\t\tconsecutiveErrs++\n\n\t\t\t\t// Exponential backoff: SyncInterval -> 2x -> 4x -> ... -> max\n\t\t\t\tif backoff == 0 {\n\t\t\t\t\tbackoff = r.SyncInterval\n\t\t\t\t} else {\n\t\t\t\t\tbackoff *= 2\n\t\t\t\t\tif backoff > DefaultSyncBackoffMax {\n\t\t\t\t\t\tbackoff = DefaultSyncBackoffMax\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// Check for LTX errors and include recovery hints\n\t\t\t\tvar ltxErr *LTXError\n\t\t\t\tif errors.As(err, <xErr) {\n\t\t\t\t\t// Log with rate limiting to avoid log spam during persistent errors.\n\t\t\t\t\tif time.Since(lastLogTime) >= SyncErrorLogInterval {\n\t\t\t\t\t\tif ltxErr.Hint != \"\" {\n\t\t\t\t\t\t\tr.Logger().Error(\"monitor error\",\n\t\t\t\t\t\t\t\t\"error\", err,\n\t\t\t\t\t\t\t\t\"path\", ltxErr.Path,\n\t\t\t\t\t\t\t\t\"hint\", ltxErr.Hint,\n\t\t\t\t\t\t\t\t\"consecutive_errors\", consecutiveErrs,\n\t\t\t\t\t\t\t\t\"backoff\", backoff)\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\tr.Logger().Error(\"monitor error\",\n\t\t\t\t\t\t\t\t\"error\", err,\n\t\t\t\t\t\t\t\t\"path\", ltxErr.Path,\n\t\t\t\t\t\t\t\t\"consecutive_errors\", consecutiveErrs,\n\t\t\t\t\t\t\t\t\"backoff\", backoff)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tlastLogTime = time.Now()\n\t\t\t\t\t}\n\n\t\t\t\t\t// Attempt auto-recovery if enabled\n\t\t\t\t\tif r.AutoRecoverEnabled {\n\t\t\t\t\t\tr.Logger().Warn(\"auto-recovery enabled, resetting local state\")\n\t\t\t\t\t\tif resetErr := r.db.ResetLocalState(ctx); resetErr != nil {\n\t\t\t\t\t\t\tr.Logger().Error(\"auto-recovery failed\", \"error\", resetErr)\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\tr.Logger().Info(\"auto-recovery complete, resuming replication\")\n\t\t\t\t\t\t\t// Reset backoff after successful recovery\n\t\t\t\t\t\t\tbackoff = 0\n\t\t\t\t\t\t\tconsecutiveErrs = 0\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\t// Log with rate limiting to avoid log spam during persistent errors.\n\t\t\t\t\tif time.Since(lastLogTime) >= SyncErrorLogInterval {\n\t\t\t\t\t\tr.Logger().Error(\"monitor error\",\n\t\t\t\t\t\t\t\"error\", err,\n\t\t\t\t\t\t\t\"consecutive_errors\", consecutiveErrs,\n\t\t\t\t\t\t\t\"backoff\", backoff)\n\t\t\t\t\t\tlastLogTime = time.Now()\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\t// Success - reset backoff and error counter.\n\t\tif consecutiveErrs > 0 {\n\t\t\tr.Logger().Info(\"replica sync recovered\", \"previous_errors\", consecutiveErrs)\n\t\t}\n\t\tbackoff = 0\n\t\tconsecutiveErrs = 0\n\t}\n}\n\n// CreatedAt returns the earliest creation time of any LTX file.\n// Returns zero time if no LTX files exist.\nfunc (r *Replica) CreatedAt(ctx context.Context) (time.Time, error) {\n\tvar min time.Time\n\n\t// Normal operation - use fast timestamps\n\titr, err := r.Client.LTXFiles(ctx, 0, 0, false)\n\tif err != nil {\n\t\treturn min, err\n\t}\n\tdefer itr.Close()\n\n\tif itr.Next() {\n\t\tmin = itr.Item().CreatedAt\n\t}\n\treturn min, itr.Close()\n}\n\n// TimeBounds returns the creation time & last updated time.\n// Returns zero time if no LTX files exist.\nfunc (r *Replica) TimeBounds(ctx context.Context) (createdAt, updatedAt time.Time, err error) {\n\tfor level := SnapshotLevel; level >= 0; level-- {\n\t\titr, err := r.Client.LTXFiles(ctx, level, 0, false)\n\t\tif err != nil {\n\t\t\treturn createdAt, updatedAt, err\n\t\t}\n\n\t\tfor itr.Next() {\n\t\t\tinfo := itr.Item()\n\t\t\tif createdAt.IsZero() || info.CreatedAt.Before(createdAt) {\n\t\t\t\tcreatedAt = info.CreatedAt\n\t\t\t}\n\t\t\tif updatedAt.IsZero() || info.CreatedAt.After(updatedAt) {\n\t\t\t\tupdatedAt = info.CreatedAt\n\t\t\t}\n\t\t}\n\t\tif err := itr.Close(); err != nil {\n\t\t\treturn createdAt, updatedAt, err\n\t\t}\n\t}\n\treturn createdAt, updatedAt, nil\n}\n\n// CalcRestoreTarget returns a target time restore from.\nfunc (r *Replica) CalcRestoreTarget(ctx context.Context, opt RestoreOptions) (updatedAt time.Time, err error) {\n\t// Determine the replicated time bounds from LTX files.\n\tcreatedAt, updatedAt, err := r.TimeBounds(ctx)\n\tif err != nil {\n\t\treturn time.Time{}, fmt.Errorf(\"created at: %w\", err)\n\t}\n\n\t// Also check v0.3.x time bounds if client supports it.\n\tif client, ok := r.Client.(ReplicaClientV3); ok {\n\t\tv3CreatedAt, v3UpdatedAt, err := r.TimeBoundsV3(ctx, client)\n\t\tif err != nil {\n\t\t\treturn time.Time{}, fmt.Errorf(\"v0.3.x time bounds: %w\", err)\n\t\t}\n\t\t// Extend time bounds to include v0.3.x backups.\n\t\tif !v3CreatedAt.IsZero() && (createdAt.IsZero() || v3CreatedAt.Before(createdAt)) {\n\t\t\tcreatedAt = v3CreatedAt\n\t\t}\n\t\tif !v3UpdatedAt.IsZero() && (updatedAt.IsZero() || v3UpdatedAt.After(updatedAt)) {\n\t\t\tupdatedAt = v3UpdatedAt\n\t\t}\n\t}\n\n\t// Skip if it does not contain timestamp.\n\tif !opt.Timestamp.IsZero() {\n\t\tif createdAt.IsZero() && updatedAt.IsZero() {\n\t\t\treturn time.Time{}, fmt.Errorf(\"no backups found\")\n\t\t}\n\t\tif opt.Timestamp.Before(createdAt) || opt.Timestamp.After(updatedAt) {\n\t\t\treturn time.Time{}, fmt.Errorf(\"timestamp does not exist\")\n\t\t}\n\t}\n\n\treturn updatedAt, nil\n}\n\n// Replica restores the database from a replica based on the options given.\n// This method will restore into opt.OutputPath, if specified, or into the\n// DB's original database path. It can optionally restore from a specific\n// replica or it will automatically choose the best one. Finally,\n// a timestamp can be specified to restore the database to a specific\n// point-in-time.\n//\n// When the replica contains both v0.3.x and LTX format backups, this method\n// compares snapshots from both formats and uses whichever has the better backup:\n// - With timestamp: uses the format with the most recent snapshot before timestamp\n// - Without timestamp: uses the format with the most recent backup overall\nfunc (r *Replica) Restore(ctx context.Context, opt RestoreOptions) (err error) {\n\t// Validate options.\n\tif opt.OutputPath == \"\" {\n\t\treturn fmt.Errorf(\"output path required\")\n\t} else if opt.TXID != 0 && !opt.Timestamp.IsZero() {\n\t\treturn fmt.Errorf(\"cannot specify index & timestamp to restore\")\n\t} else if opt.Follow && opt.TXID != 0 {\n\t\treturn fmt.Errorf(\"cannot use follow mode with -txid\")\n\t} else if opt.Follow && !opt.Timestamp.IsZero() {\n\t\treturn fmt.Errorf(\"cannot use follow mode with -timestamp\")\n\t} else if opt.IntegrityCheck != IntegrityCheckNone && opt.IntegrityCheck != IntegrityCheckQuick && opt.IntegrityCheck != IntegrityCheckFull {\n\t\treturn fmt.Errorf(\"unsupported integrity check mode: %d\", opt.IntegrityCheck)\n\t}\n\n\t// In follow mode, if the database already exists, attempt crash recovery\n\t// by reading the last applied TXID from the sidecar file.\n\tif opt.Follow {\n\t\tif _, statErr := os.Stat(opt.OutputPath); statErr == nil {\n\t\t\ttxid, readErr := ReadTXIDFile(opt.OutputPath)\n\t\t\tif readErr != nil {\n\t\t\t\treturn fmt.Errorf(\"read txid file for crash recovery: %w\", readErr)\n\t\t\t}\n\t\t\tif txid == 0 {\n\t\t\t\treturn fmt.Errorf(\"cannot resume follow mode: database exists but no -txid file found; delete the database to re-restore: %s\", opt.OutputPath)\n\t\t\t}\n\t\t\t// Validate saved TXID is still reachable. If the earliest snapshot\n\t\t\t// starts after our saved TXID, retention has pruned the history\n\t\t\t// and we can't catch up incrementally.\n\t\t\tsnapshotItr, itrErr := r.Client.LTXFiles(ctx, SnapshotLevel, 0, false)\n\t\t\tif itrErr != nil {\n\t\t\t\treturn fmt.Errorf(\"cannot validate saved TXID for crash recovery: %w\", itrErr)\n\t\t\t}\n\n\t\t\tvar latestSnapshot *ltx.FileInfo\n\t\t\tfor snapshotItr.Next() {\n\t\t\t\tlatestSnapshot = snapshotItr.Item()\n\t\t\t}\n\t\t\tif err := snapshotItr.Err(); err != nil {\n\t\t\t\t_ = snapshotItr.Close()\n\t\t\t\treturn fmt.Errorf(\"iterate snapshots for crash recovery validation: %w\", err)\n\t\t\t}\n\t\t\t_ = snapshotItr.Close()\n\n\t\t\tif latestSnapshot != nil {\n\t\t\t\tif latestSnapshot.MinTXID > txid {\n\t\t\t\t\treturn fmt.Errorf(\"cannot resume follow mode: saved TXID %s is behind the earliest snapshot (min TXID %s); replica history has been pruned -- delete %s and %s-txid to re-restore\", txid, latestSnapshot.MinTXID, opt.OutputPath, opt.OutputPath)\n\t\t\t\t}\n\t\t\t\tif txid > latestSnapshot.MaxTXID {\n\t\t\t\t\treturn fmt.Errorf(\"cannot resume follow mode: saved TXID %s is ahead of latest snapshot (max TXID %s); delete %s and %s-txid to re-restore\", txid, latestSnapshot.MaxTXID, opt.OutputPath, opt.OutputPath)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tr.Logger().Info(\"resuming follow mode from crash recovery\", \"txid\", txid, \"output\", opt.OutputPath)\n\t\t\treturn r.follow(ctx, opt.OutputPath, txid, opt.FollowInterval)\n\t\t}\n\t}\n\n\t// Ensure output path does not already exist.\n\tif _, err := os.Stat(opt.OutputPath); err == nil {\n\t\treturn fmt.Errorf(\"cannot restore, output path already exists: %s\", opt.OutputPath)\n\t} else if !os.IsNotExist(err) {\n\t\treturn err\n\t}\n\n\t// Compare v0.3.x and LTX formats to find the best backup (unless TXID is specified).\n\t// Skip V3 format when follow mode is enabled (V3 doesn't support incremental following).\n\tif opt.TXID == 0 && !opt.Follow {\n\t\tif client, ok := r.Client.(ReplicaClientV3); ok {\n\t\t\tuseV3, err := r.shouldUseV3Restore(ctx, client, opt.Timestamp)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif useV3 {\n\t\t\t\treturn r.RestoreV3(ctx, opt)\n\t\t\t}\n\t\t}\n\t}\n\n\tinfos, err := CalcRestorePlan(ctx, r.Client, opt.TXID, opt.Timestamp, r.Logger())\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot calc restore plan: %w\", err)\n\t}\n\n\tr.Logger().Debug(\"restore plan\", \"n\", len(infos), \"txid\", infos[len(infos)-1].MaxTXID, \"timestamp\", infos[len(infos)-1].CreatedAt)\n\n\trdrs := make([]io.Reader, 0, len(infos))\n\tdefer func() {\n\t\tfor _, rd := range rdrs {\n\t\t\tif closer, ok := rd.(io.Closer); ok {\n\t\t\t\t_ = closer.Close()\n\t\t\t}\n\t\t}\n\t}()\n\n\tfor _, info := range infos {\n\t\t// Validate file size - must be at least header size to be readable\n\t\tif info.Size < ltx.HeaderSize {\n\t\t\treturn fmt.Errorf(\"invalid ltx file: level=%d min=%s max=%s has size %d bytes (minimum %d)\",\n\t\t\t\tinfo.Level, info.MinTXID, info.MaxTXID, info.Size, ltx.HeaderSize)\n\t\t}\n\n\t\tr.Logger().Debug(\"opening ltx file for restore\", \"level\", info.Level, \"min\", info.MinTXID, \"max\", info.MaxTXID)\n\n\t\t// Add file to be compacted.\n\t\tf, err := r.Client.OpenLTXFile(ctx, info.Level, info.MinTXID, info.MaxTXID, 0, 0)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"open ltx file: %w\", err)\n\t\t}\n\t\trdrs = append(rdrs, internal.NewResumableReader(ctx, r.Client, info.Level, info.MinTXID, info.MaxTXID, info.Size, f, r.Logger()))\n\t}\n\n\tif len(rdrs) == 0 {\n\t\treturn fmt.Errorf(\"no matching backup files available\")\n\t}\n\n\t// Create parent directory if it doesn't exist.\n\tvar dirInfo os.FileInfo\n\tif db := r.DB(); db != nil {\n\t\tdirInfo = db.dirInfo\n\t}\n\tif err := internal.MkdirAll(filepath.Dir(opt.OutputPath), dirInfo); err != nil {\n\t\treturn fmt.Errorf(\"create parent directory: %w\", err)\n\t}\n\n\t// Output to temp file & atomically rename.\n\ttmpOutputPath := opt.OutputPath + \".tmp\"\n\tr.Logger().Debug(\"compacting into database\", \"path\", tmpOutputPath, \"n\", len(rdrs))\n\n\tf, err := os.Create(tmpOutputPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create temp database path: %w\", err)\n\t}\n\tdefer func() { _ = f.Close() }()\n\n\tpr, pw := io.Pipe()\n\n\tgo func() {\n\t\tc, err := ltx.NewCompactor(pw, rdrs)\n\t\tif err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"new ltx compactor: %w\", err))\n\t\t\treturn\n\t\t}\n\t\tc.HeaderFlags = ltx.HeaderFlagNoChecksum\n\t\t_ = pw.CloseWithError(c.Compact(ctx))\n\t}()\n\n\tdec := ltx.NewDecoder(pr)\n\tif err := dec.DecodeDatabaseTo(f); err != nil {\n\t\treturn fmt.Errorf(\"decode database: %w\", err)\n\t}\n\n\tif err := f.Sync(); err != nil {\n\t\treturn err\n\t} else if err := f.Close(); err != nil {\n\t\treturn err\n\t}\n\n\t// Copy file to final location.\n\tr.Logger().Debug(\"renaming database from temporary location\")\n\tif err := os.Rename(tmpOutputPath, opt.OutputPath); err != nil {\n\t\treturn err\n\t}\n\n\tif opt.IntegrityCheck != IntegrityCheckNone {\n\t\tif err := checkIntegrity(ctx, opt.OutputPath, opt.IntegrityCheck); err != nil {\n\t\t\tif ctx.Err() == nil {\n\t\t\t\t_ = os.Remove(opt.OutputPath)\n\t\t\t\t_ = os.Remove(opt.OutputPath + \"-shm\")\n\t\t\t\t_ = os.Remove(opt.OutputPath + \"-wal\")\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"post-restore integrity check: %w\", err)\n\t\t}\n\t\tr.Logger().Info(\"post-restore integrity check passed\")\n\t}\n\n\t// Enter follow mode if enabled, continuously applying new LTX files.\n\tif opt.Follow {\n\t\tfor _, rd := range rdrs {\n\t\t\tif closer, ok := rd.(io.Closer); ok {\n\t\t\t\t_ = closer.Close()\n\t\t\t}\n\t\t}\n\t\trdrs = nil\n\n\t\tmaxTXID := infos[len(infos)-1].MaxTXID\n\t\tif err := WriteTXIDFile(opt.OutputPath, maxTXID); err != nil {\n\t\t\treturn fmt.Errorf(\"write initial txid file: %w\", err)\n\t\t}\n\t\treturn r.follow(ctx, opt.OutputPath, maxTXID, opt.FollowInterval)\n\t}\n\n\treturn nil\n}\n\n// follow enters a continuous restore loop, polling for new LTX files and\n// applying them to the restored database. It blocks until the context is\n// cancelled (e.g. Ctrl+C). Returns nil on clean shutdown.\nfunc (r *Replica) follow(ctx context.Context, outputPath string, lastTXID ltx.TXID, interval time.Duration) error {\n\tf, err := os.OpenFile(outputPath, os.O_RDWR, 0)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open database for follow: %w\", err)\n\t}\n\tdefer func() {\n\t\t_ = f.Sync()\n\t\t_ = f.Close()\n\t}()\n\n\t// Read page size from SQLite header (offset 16, 2 bytes, big-endian).\n\tvar buf [2]byte\n\tif _, err := f.ReadAt(buf[:], 16); err != nil {\n\t\treturn fmt.Errorf(\"read page size from database header: %w\", err)\n\t}\n\tpageSize := uint32(buf[0])<<8 | uint32(buf[1])\n\tif pageSize == 1 {\n\t\tpageSize = 65536\n\t}\n\n\tif interval <= 0 {\n\t\tinterval = DefaultFollowInterval\n\t}\n\n\tr.Logger().Info(\"entering follow mode\", \"output\", outputPath, \"txid\", lastTXID, \"interval\", interval)\n\n\tticker := time.NewTicker(interval)\n\tdefer ticker.Stop()\n\n\tvar consecutiveErrors int\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\tr.Logger().Info(\"follow mode stopped\")\n\t\t\treturn nil\n\t\tcase <-ticker.C:\n\t\t\tnewTXID, err := r.applyNewLTXFiles(ctx, f, lastTXID, pageSize)\n\t\t\tif err != nil {\n\t\t\t\tif ctx.Err() != nil {\n\t\t\t\t\tr.Logger().Info(\"follow mode stopped\")\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t\tconsecutiveErrors++\n\t\t\t\tr.Logger().Error(\"follow: error applying updates\", \"err\", err, \"consecutive_errors\", consecutiveErrors)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif newTXID > lastTXID {\n\t\t\t\tif err := WriteTXIDFile(outputPath, newTXID); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"write txid file: %w\", err)\n\t\t\t\t}\n\t\t\t\tr.Logger().Info(\"follow: applied updates\", \"from_txid\", lastTXID, \"to_txid\", newTXID)\n\t\t\t\tlastTXID = newTXID\n\t\t\t\tconsecutiveErrors = 0\n\t\t\t}\n\t\t}\n\t}\n}\n\n// applyNewLTXFiles polls for new LTX files and applies them to the database.\n// It starts from level 0 and falls back to higher levels if there are gaps\n// (e.g., level 0 files were compacted away).\nfunc (r *Replica) applyNewLTXFiles(ctx context.Context, f *os.File, afterTXID ltx.TXID, pageSize uint32) (ltx.TXID, error) {\n\tcurrentTXID := afterTXID\n\n\t// Poll level 0 for the most recent incremental files.\n\titr, err := r.Client.LTXFiles(ctx, 0, currentTXID+1, false)\n\tif err != nil {\n\t\treturn currentTXID, fmt.Errorf(\"list level 0 ltx files: %w\", err)\n\t}\n\tcloseLevel0 := func(retErr error) (ltx.TXID, error) {\n\t\tif closeErr := itr.Close(); closeErr != nil {\n\t\t\tcloseErr = fmt.Errorf(\"close level 0 ltx iterator: %w\", closeErr)\n\t\t\tif retErr != nil {\n\t\t\t\treturn currentTXID, errors.Join(retErr, closeErr)\n\t\t\t}\n\t\t\treturn currentTXID, closeErr\n\t\t}\n\t\treturn currentTXID, retErr\n\t}\n\n\tvar sawLevel0 bool\n\tfor itr.Next() {\n\t\tsawLevel0 = true\n\t\tinfo := itr.Item()\n\n\t\t// If there's a gap, try to fill it from higher compaction levels.\n\t\tif info.MinTXID > currentTXID+1 {\n\t\t\tbridgedTXID, err := r.fillFollowGap(ctx, f, currentTXID, info.MinTXID, pageSize)\n\t\t\tif err != nil {\n\t\t\t\treturn closeLevel0(err)\n\t\t\t}\n\t\t\tcurrentTXID = bridgedTXID\n\n\t\t\t// Re-check if this file is still needed after bridging.\n\t\t\tif info.MaxTXID <= currentTXID {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif info.MinTXID > currentTXID+1 {\n\t\t\t\treturn closeLevel0(nil)\n\t\t\t}\n\t\t}\n\n\t\t// Skip if already covered by a higher-level file.\n\t\tif info.MaxTXID <= currentTXID {\n\t\t\tcontinue\n\t\t}\n\n\t\tif err := r.applyLTXFile(ctx, f, info, pageSize); err != nil {\n\t\t\treturn closeLevel0(fmt.Errorf(\n\t\t\t\t\"apply ltx file (level=%d, min=%s, max=%s): %w\",\n\t\t\t\tinfo.Level, info.MinTXID, info.MaxTXID, err,\n\t\t\t))\n\t\t}\n\t\tcurrentTXID = info.MaxTXID\n\t}\n\n\tif iterErr := itr.Err(); iterErr != nil {\n\t\treturn closeLevel0(fmt.Errorf(\"iterate level 0 ltx files: %w\", iterErr))\n\t}\n\tif _, err := closeLevel0(nil); err != nil {\n\t\treturn currentTXID, err\n\t}\n\n\tif !sawLevel0 {\n\t\tbridgedTXID, err := r.fillFollowGap(ctx, f, currentTXID, currentTXID+1, pageSize)\n\t\tif err != nil {\n\t\t\treturn currentTXID, err\n\t\t}\n\t\tcurrentTXID = bridgedTXID\n\t}\n\n\treturn currentTXID, nil\n}\n\n// applyLTXFile applies a single LTX file's pages to the database file.\n// This follows the same pattern as Hydrator.ApplyLTX (vfs.go:712-747).\n//\n// To prevent concurrent SQLite readers from seeing partial updates, we acquire\n// an exclusive file lock before writing. We also rewrite the SQLite header\n// (bytes 18-19) to indicate DELETE journal mode instead of WAL mode, and\n// randomize the schema change counter (bytes 24-27) to invalidate cached\n// schemas in other connections.\nfunc (r *Replica) applyLTXFile(ctx context.Context, f *os.File, info *ltx.FileInfo, pageSize uint32) error {\n\trc, err := r.Client.OpenLTXFile(ctx, info.Level, info.MinTXID, info.MaxTXID, 0, 0)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open ltx file: %w\", err)\n\t}\n\tdefer rc.Close()\n\n\tdec := ltx.NewDecoder(rc)\n\tif err := dec.DecodeHeader(); err != nil {\n\t\treturn fmt.Errorf(\"decode header: %w\", err)\n\t}\n\n\thdr := dec.Header()\n\n\tif err := internal.LockFileExclusive(f); err != nil {\n\t\treturn fmt.Errorf(\"acquire exclusive lock: %w\", err)\n\t}\n\tdefer internal.UnlockFile(f)\n\n\tfor {\n\t\tvar phdr ltx.PageHeader\n\t\tdata := make([]byte, pageSize)\n\t\tif err := dec.DecodePage(&phdr, data); err == io.EOF {\n\t\t\tbreak\n\t\t} else if err != nil {\n\t\t\treturn fmt.Errorf(\"decode page: %w\", err)\n\t\t}\n\n\t\tif phdr.Pgno == 1 && len(data) >= 28 {\n\t\t\tdata[18], data[19] = 0x01, 0x01\n\t\t\t_, _ = rand.Read(data[24:28])\n\t\t}\n\n\t\toff := int64(phdr.Pgno-1) * int64(pageSize)\n\t\tif _, err := f.WriteAt(data, off); err != nil {\n\t\t\treturn fmt.Errorf(\"write page %d: %w\", phdr.Pgno, err)\n\t\t}\n\t}\n\n\tif hdr.Commit > 0 {\n\t\tnewSize := int64(hdr.Commit) * int64(pageSize)\n\t\tif err := f.Truncate(newSize); err != nil {\n\t\t\treturn fmt.Errorf(\"truncate: %w\", err)\n\t\t}\n\t}\n\n\tif err := dec.Close(); err != nil {\n\t\treturn fmt.Errorf(\"close decoder: %w\", err)\n\t}\n\n\treturn f.Sync()\n}\n\n// fillFollowGap attempts to bridge a gap in level 0 files by searching\n// higher compaction levels for a file that covers the missing TXID range.\nfunc (r *Replica) fillFollowGap(ctx context.Context, f *os.File, afterTXID ltx.TXID, gapMinTXID ltx.TXID, pageSize uint32) (ltx.TXID, error) {\n\tcurrentTXID := afterTXID\n\n\tfor level := 1; level < SnapshotLevel; level++ {\n\t\titr, err := r.Client.LTXFiles(ctx, level, 0, false)\n\t\tif err != nil {\n\t\t\treturn currentTXID, fmt.Errorf(\"list level %d ltx files: %w\", level, err)\n\t\t}\n\t\tcloseLevel := func(retErr error) (ltx.TXID, error) {\n\t\t\tif closeErr := itr.Close(); closeErr != nil {\n\t\t\t\tcloseErr = fmt.Errorf(\"close level %d ltx iterator: %w\", level, closeErr)\n\t\t\t\tif retErr != nil {\n\t\t\t\t\treturn currentTXID, errors.Join(retErr, closeErr)\n\t\t\t\t}\n\t\t\t\treturn currentTXID, closeErr\n\t\t\t}\n\t\t\treturn currentTXID, retErr\n\t\t}\n\n\t\tfor itr.Next() {\n\t\t\tinfo := itr.Item()\n\n\t\t\t// Skip if there's a gap at this level too.\n\t\t\tif info.MinTXID > currentTXID+1 {\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\t// Skip if already covered.\n\t\t\tif info.MaxTXID <= currentTXID {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif err := r.applyLTXFile(ctx, f, info, pageSize); err != nil {\n\t\t\t\treturn closeLevel(fmt.Errorf(\n\t\t\t\t\t\"apply gap-fill ltx file (level=%d, min=%s, max=%s): %w\",\n\t\t\t\t\tinfo.Level, info.MinTXID, info.MaxTXID, err,\n\t\t\t\t))\n\t\t\t}\n\t\t\tcurrentTXID = info.MaxTXID\n\n\t\t\t// If we've bridged past the gap, we're done.\n\t\t\tif currentTXID+1 >= gapMinTXID {\n\t\t\t\treturn closeLevel(nil)\n\t\t\t}\n\t\t}\n\n\t\tif iterErr := itr.Err(); iterErr != nil {\n\t\t\treturn closeLevel(fmt.Errorf(\"iterate level %d ltx files: %w\", level, iterErr))\n\t\t}\n\t\tif _, err := closeLevel(nil); err != nil {\n\t\t\treturn currentTXID, err\n\t\t}\n\n\t\t// If we made progress at this level, restart from level 1.\n\t\tif currentTXID > afterTXID {\n\t\t\treturn currentTXID, nil\n\t\t}\n\t}\n\n\treturn currentTXID, nil\n}\n\n// RestoreV3 restores from a v0.3.x format backup.\nfunc (r *Replica) RestoreV3(ctx context.Context, opt RestoreOptions) error {\n\tclient, ok := r.Client.(ReplicaClientV3)\n\tif !ok {\n\t\treturn fmt.Errorf(\"replica client does not support v0.3.x restore\")\n\t}\n\n\t// Validate options.\n\tif opt.OutputPath == \"\" {\n\t\treturn fmt.Errorf(\"output path required\")\n\t} else if opt.IntegrityCheck != IntegrityCheckNone && opt.IntegrityCheck != IntegrityCheckQuick && opt.IntegrityCheck != IntegrityCheckFull {\n\t\treturn fmt.Errorf(\"unsupported integrity check mode: %d\", opt.IntegrityCheck)\n\t}\n\n\t// Ensure output path does not already exist.\n\tif _, err := os.Stat(opt.OutputPath); err == nil {\n\t\treturn fmt.Errorf(\"cannot restore, output path already exists: %s\", opt.OutputPath)\n\t} else if !os.IsNotExist(err) {\n\t\treturn err\n\t}\n\n\t// Find all generations.\n\tgenerations, err := client.GenerationsV3(ctx)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"list generations: %w\", err)\n\t}\n\tif len(generations) == 0 {\n\t\treturn ErrNoSnapshots\n\t}\n\n\t// Collect all snapshots across all generations.\n\tvar allSnapshots []SnapshotInfoV3\n\tfor _, gen := range generations {\n\t\tsnapshots, err := client.SnapshotsV3(ctx, gen)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"list snapshots for generation %s: %w\", gen, err)\n\t\t}\n\t\tallSnapshots = append(allSnapshots, snapshots...)\n\t}\n\tif len(allSnapshots) == 0 {\n\t\treturn ErrNoSnapshots\n\t}\n\n\t// Sort all snapshots by CreatedAt for timestamp-based selection.\n\tsortSnapshotsV3ByCreatedAt(allSnapshots)\n\n\t// Find best snapshot across all generations (latest, or before timestamp if specified).\n\tsnapshot := findBestSnapshotV3(allSnapshots, opt.Timestamp)\n\tif snapshot == nil {\n\t\treturn ErrNoSnapshots\n\t}\n\n\tr.Logger().Debug(\"selected v0.3.x snapshot\",\n\t\t\"generation\", snapshot.Generation,\n\t\t\"index\", snapshot.Index,\n\t\t\"created_at\", snapshot.CreatedAt)\n\n\t// Get WAL segments for the snapshot's generation.\n\tsegments, err := client.WALSegmentsV3(ctx, snapshot.Generation)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"list WAL segments: %w\", err)\n\t}\n\tsegments = filterWALSegmentsV3(segments, snapshot.Index, opt.Timestamp)\n\n\tr.Logger().Debug(\"found v0.3.x WAL segments\", \"n\", len(segments))\n\n\t// Create parent directory if it doesn't exist.\n\tvar dirInfo os.FileInfo\n\tif db := r.DB(); db != nil {\n\t\tdirInfo = db.DirInfo()\n\t}\n\tif err := internal.MkdirAll(filepath.Dir(opt.OutputPath), dirInfo); err != nil {\n\t\treturn fmt.Errorf(\"create parent directory: %w\", err)\n\t}\n\n\t// Create temp file for restore.\n\ttmpPath := opt.OutputPath + \".tmp\"\n\tdefer func() { _ = os.Remove(tmpPath) }()\n\n\t// Download and decompress snapshot.\n\tif err := r.downloadSnapshotV3(ctx, client, snapshot.Generation, snapshot.Index, tmpPath); err != nil {\n\t\treturn fmt.Errorf(\"download snapshot: %w\", err)\n\t}\n\n\t// Apply WAL segments.\n\tif err := r.applyWALSegmentsV3(ctx, client, snapshot.Generation, segments, tmpPath); err != nil {\n\t\treturn fmt.Errorf(\"apply WAL segments: %w\", err)\n\t}\n\n\t// Rename to final path.\n\tif err := os.Rename(tmpPath, opt.OutputPath); err != nil {\n\t\treturn fmt.Errorf(\"rename to output path: %w\", err)\n\t}\n\n\tif opt.IntegrityCheck != IntegrityCheckNone {\n\t\tif err := checkIntegrity(ctx, opt.OutputPath, opt.IntegrityCheck); err != nil {\n\t\t\tif ctx.Err() == nil {\n\t\t\t\t_ = os.Remove(opt.OutputPath)\n\t\t\t\t_ = os.Remove(opt.OutputPath + \"-shm\")\n\t\t\t\t_ = os.Remove(opt.OutputPath + \"-wal\")\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"post-restore integrity check: %w\", err)\n\t\t}\n\t\tr.Logger().Info(\"post-restore integrity check passed\")\n\t}\n\n\treturn nil\n}\n\n// sortSnapshotsV3ByCreatedAt sorts snapshots by creation time in ascending order.\nfunc sortSnapshotsV3ByCreatedAt(snapshots []SnapshotInfoV3) {\n\tfor i := 0; i < len(snapshots)-1; i++ {\n\t\tfor j := i + 1; j < len(snapshots); j++ {\n\t\t\tif snapshots[i].CreatedAt.After(snapshots[j].CreatedAt) {\n\t\t\t\tsnapshots[i], snapshots[j] = snapshots[j], snapshots[i]\n\t\t\t}\n\t\t}\n\t}\n}\n\n// findBestSnapshotV3 finds the best snapshot for restore.\n// If timestamp is zero, returns the latest snapshot.\n// Otherwise, returns the latest snapshot created before or at the timestamp.\nfunc findBestSnapshotV3(snapshots []SnapshotInfoV3, timestamp time.Time) *SnapshotInfoV3 {\n\tif len(snapshots) == 0 {\n\t\treturn nil\n\t}\n\tif timestamp.IsZero() {\n\t\treturn &snapshots[len(snapshots)-1]\n\t}\n\tfor i := len(snapshots) - 1; i >= 0; i-- {\n\t\tif !snapshots[i].CreatedAt.After(timestamp) {\n\t\t\treturn &snapshots[i]\n\t\t}\n\t}\n\treturn nil\n}\n\n// filterWALSegmentsV3 filters WAL segments to those at or after the snapshot index\n// and optionally before the timestamp.\nfunc filterWALSegmentsV3(segments []WALSegmentInfoV3, snapshotIndex int, timestamp time.Time) []WALSegmentInfoV3 {\n\tvar result []WALSegmentInfoV3\n\tfor _, seg := range segments {\n\t\tif seg.Index < snapshotIndex {\n\t\t\tcontinue\n\t\t}\n\t\tif !timestamp.IsZero() && seg.CreatedAt.After(timestamp) {\n\t\t\tcontinue\n\t\t}\n\t\tresult = append(result, seg)\n\t}\n\treturn result\n}\n\n// downloadSnapshotV3 downloads and decompresses a v0.3.x snapshot to the destination path.\nfunc (r *Replica) downloadSnapshotV3(ctx context.Context, client ReplicaClientV3, generation string, index int, destPath string) error {\n\trc, err := client.OpenSnapshotV3(ctx, generation, index)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer func() { _ = rc.Close() }()\n\n\tf, err := os.Create(destPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer func() { _ = f.Close() }()\n\n\tif _, err := io.Copy(f, rc); err != nil {\n\t\treturn err\n\t}\n\treturn f.Sync()\n}\n\n// applyWALSegmentsV3 applies WAL segments to the database file.\nfunc (r *Replica) applyWALSegmentsV3(ctx context.Context, client ReplicaClientV3, generation string, segments []WALSegmentInfoV3, dbPath string) error {\n\tif len(segments) == 0 {\n\t\treturn nil\n\t}\n\n\t// Write all WAL segments to the WAL file.\n\twalPath := dbPath + \"-wal\"\n\tfor _, seg := range segments {\n\t\tif err := r.writeWALSegmentV3(ctx, client, generation, seg, walPath); err != nil {\n\t\t\treturn fmt.Errorf(\"write WAL segment %d/%d: %w\", seg.Index, seg.Offset, err)\n\t\t}\n\t}\n\n\t// Checkpoint to apply WAL to database.\n\treturn checkpointV3(dbPath)\n}\n\n// writeWALSegmentV3 writes a single WAL segment to the WAL file.\nfunc (r *Replica) writeWALSegmentV3(ctx context.Context, client ReplicaClientV3, generation string, seg WALSegmentInfoV3, walPath string) error {\n\t// Download WAL segment.\n\trc, err := client.OpenWALSegmentV3(ctx, generation, seg.Index, seg.Offset)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer func() { _ = rc.Close() }()\n\n\t// Open WAL file for writing.\n\tf, err := os.OpenFile(walPath, os.O_CREATE|os.O_WRONLY, 0644)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer func() { _ = f.Close() }()\n\n\t// Seek to offset and write.\n\tif _, err := f.Seek(seg.Offset, io.SeekStart); err != nil {\n\t\treturn err\n\t}\n\tif _, err := io.Copy(f, rc); err != nil {\n\t\treturn err\n\t}\n\treturn f.Sync()\n}\n\n// checkpointV3 checkpoints the WAL file into the database.\nfunc checkpointV3(dbPath string) error {\n\tdb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer func() { _ = db.Close() }()\n\n\t_, err = db.Exec(\"PRAGMA wal_checkpoint(TRUNCATE)\")\n\treturn err\n}\n\n// checkIntegrity runs a SQLite integrity check on the database at dbPath.\nfunc checkIntegrity(ctx context.Context, dbPath string, mode IntegrityCheckMode) error {\n\tif mode == IntegrityCheckNone {\n\t\treturn nil\n\t}\n\n\tdb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open database for integrity check: %w\", err)\n\t}\n\tdefer func() { _ = db.Close() }()\n\n\tvar pragma string\n\tswitch mode {\n\tcase IntegrityCheckQuick:\n\t\tpragma = \"quick_check\"\n\tcase IntegrityCheckFull:\n\t\tpragma = \"integrity_check\"\n\tdefault:\n\t\treturn fmt.Errorf(\"unsupported integrity check mode: %d\", mode)\n\t}\n\n\tvar result string\n\tif err := db.QueryRowContext(ctx, \"PRAGMA \"+pragma).Scan(&result); err != nil {\n\t\treturn fmt.Errorf(\"integrity check: %w\", err)\n\t}\n\tif result != \"ok\" {\n\t\treturn fmt.Errorf(\"integrity check failed: %s\", result)\n\t}\n\n\t// Clean up -shm and -wal files that SQLite may create during the PRAGMA.\n\t_ = os.Remove(dbPath + \"-shm\")\n\t_ = os.Remove(dbPath + \"-wal\")\n\n\treturn nil\n}\n\n// findBestV3SnapshotForTimestamp returns the best v0.3.x snapshot for the given timestamp.\n// Returns nil if no suitable snapshot exists.\nfunc (r *Replica) findBestV3SnapshotForTimestamp(ctx context.Context, client ReplicaClientV3, timestamp time.Time) (*SnapshotInfoV3, error) {\n\tgenerations, err := client.GenerationsV3(ctx)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"list v0.3.x generations: %w\", err)\n\t}\n\tif len(generations) == 0 {\n\t\treturn nil, nil\n\t}\n\n\tvar allSnapshots []SnapshotInfoV3\n\tfor _, gen := range generations {\n\t\tsnapshots, err := client.SnapshotsV3(ctx, gen)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"list v0.3.x snapshots for generation %s: %w\", gen, err)\n\t\t}\n\t\tallSnapshots = append(allSnapshots, snapshots...)\n\t}\n\n\tif len(allSnapshots) == 0 {\n\t\treturn nil, nil\n\t}\n\n\t// Sort by CreatedAt for timestamp-based selection.\n\tsortSnapshotsV3ByCreatedAt(allSnapshots)\n\n\treturn findBestSnapshotV3(allSnapshots, timestamp), nil\n}\n\n// shouldUseV3Restore determines whether to use v0.3.x restore instead of LTX.\n// Returns true if v0.3.x has a better backup for the given options.\nfunc (r *Replica) shouldUseV3Restore(ctx context.Context, client ReplicaClientV3, timestamp time.Time) (bool, error) {\n\t// Get v0.3.x time bounds.\n\t_, v3UpdatedAt, err := r.TimeBoundsV3(ctx, client)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"get v0.3.x time bounds: %w\", err)\n\t}\n\n\t// Get LTX time bounds.\n\t_, ltxUpdatedAt, err := r.TimeBounds(ctx)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"get LTX time bounds: %w\", err)\n\t}\n\n\t// If no v0.3.x backups exist, use LTX.\n\tif v3UpdatedAt.IsZero() {\n\t\treturn false, nil\n\t}\n\n\t// If no LTX backups exist, use v0.3.x.\n\tif ltxUpdatedAt.IsZero() {\n\t\tr.Logger().Debug(\"using v0.3.x restore (no LTX backups)\")\n\t\treturn true, nil\n\t}\n\n\t// Both formats have backups - compare based on timestamp or latest.\n\tif !timestamp.IsZero() {\n\t\t// With timestamp: use format with best snapshot before timestamp.\n\t\tv3Snapshot, err := r.findBestV3SnapshotForTimestamp(ctx, client, timestamp)\n\t\tif err != nil {\n\t\t\treturn false, fmt.Errorf(\"find v0.3.x snapshot: %w\", err)\n\t\t}\n\n\t\tltxSnapshot, err := r.findBestLTXSnapshotForTimestamp(ctx, timestamp)\n\t\tif err != nil {\n\t\t\treturn false, fmt.Errorf(\"find LTX snapshot: %w\", err)\n\t\t}\n\n\t\tif v3Snapshot != nil && (ltxSnapshot == nil || v3Snapshot.CreatedAt.After(ltxSnapshot.CreatedAt)) {\n\t\t\tr.Logger().Debug(\"using v0.3.x restore (better snapshot for timestamp)\",\n\t\t\t\t\"v3_snapshot\", v3Snapshot.CreatedAt,\n\t\t\t\t\"ltx_snapshot\", ltxSnapshot)\n\t\t\treturn true, nil\n\t\t}\n\t} else {\n\t\t// Without timestamp: use format with most recent backup.\n\t\tif v3UpdatedAt.After(ltxUpdatedAt) {\n\t\t\tr.Logger().Debug(\"using v0.3.x restore (more recent backup)\",\n\t\t\t\t\"v3_updated_at\", v3UpdatedAt,\n\t\t\t\t\"ltx_updated_at\", ltxUpdatedAt)\n\t\t\treturn true, nil\n\t\t}\n\t}\n\n\treturn false, nil\n}\n\n// TimeBoundsV3 returns the time bounds of v0.3.x backups.\n// Returns zero times if no v0.3.x backups exist.\nfunc (r *Replica) TimeBoundsV3(ctx context.Context, client ReplicaClientV3) (createdAt, updatedAt time.Time, err error) {\n\tgenerations, err := client.GenerationsV3(ctx)\n\tif err != nil {\n\t\treturn time.Time{}, time.Time{}, err\n\t}\n\n\tfor _, gen := range generations {\n\t\tsnapshots, err := client.SnapshotsV3(ctx, gen)\n\t\tif err != nil {\n\t\t\treturn time.Time{}, time.Time{}, err\n\t\t}\n\t\tfor _, snap := range snapshots {\n\t\t\tif createdAt.IsZero() || snap.CreatedAt.Before(createdAt) {\n\t\t\t\tcreatedAt = snap.CreatedAt\n\t\t\t}\n\t\t\tif updatedAt.IsZero() || snap.CreatedAt.After(updatedAt) {\n\t\t\t\tupdatedAt = snap.CreatedAt\n\t\t\t}\n\t\t}\n\n\t\tsegments, err := client.WALSegmentsV3(ctx, gen)\n\t\tif err != nil {\n\t\t\treturn time.Time{}, time.Time{}, err\n\t\t}\n\t\tfor _, seg := range segments {\n\t\t\tif createdAt.IsZero() || seg.CreatedAt.Before(createdAt) {\n\t\t\t\tcreatedAt = seg.CreatedAt\n\t\t\t}\n\t\t\tif updatedAt.IsZero() || seg.CreatedAt.After(updatedAt) {\n\t\t\t\tupdatedAt = seg.CreatedAt\n\t\t\t}\n\t\t}\n\t}\n\n\treturn createdAt, updatedAt, nil\n}\n\n// findBestLTXSnapshotForTimestamp returns the best LTX snapshot for the given timestamp.\n// Returns nil if no suitable snapshot exists.\nfunc (r *Replica) findBestLTXSnapshotForTimestamp(ctx context.Context, timestamp time.Time) (*ltx.FileInfo, error) {\n\t// Find snapshots at the snapshot level that are before the timestamp.\n\tsnapshots, err := FindLTXFiles(ctx, r.Client, SnapshotLevel, true, func(info *ltx.FileInfo) (bool, error) {\n\t\treturn info.CreatedAt.Before(timestamp), nil\n\t})\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"find LTX snapshots: %w\", err)\n\t}\n\tif len(snapshots) == 0 {\n\t\treturn nil, nil\n\t}\n\n\t// Return the latest snapshot before the timestamp (last in the sorted list).\n\treturn snapshots[len(snapshots)-1], nil\n}\n\n// CalcRestorePlan returns a list of storage paths to restore a snapshot at the given TXID.\nfunc CalcRestorePlan(ctx context.Context, client ReplicaClient, txID ltx.TXID, timestamp time.Time, logger *slog.Logger) ([]*ltx.FileInfo, error) {\n\tif txID != 0 && !timestamp.IsZero() {\n\t\treturn nil, fmt.Errorf(\"cannot specify both TXID & timestamp to restore\")\n\t}\n\n\tvar infos ltx.FileInfoSlice\n\tlogger = logger.With(\"target\", txID)\n\n\t// Start with latest snapshot before target TXID or timestamp.\n\t// Pass useMetadata flag to enable accurate timestamp fetching for timestamp-based restore.\n\tvar snapshot *ltx.FileInfo\n\tsnapshotItr, err := client.LTXFiles(ctx, SnapshotLevel, 0, !timestamp.IsZero())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tfor snapshotItr.Next() {\n\t\tinfo := snapshotItr.Item()\n\t\tlogger.Debug(\"finding snapshot before target TXID or timestamp\", \"snapshot\", info.MaxTXID)\n\t\tif txID != 0 && info.MaxTXID > txID {\n\t\t\tcontinue\n\t\t}\n\t\tif !timestamp.IsZero() && !info.CreatedAt.Before(timestamp) {\n\t\t\tcontinue\n\t\t}\n\t\tsnapshot = info\n\t}\n\tif err := snapshotItr.Close(); err != nil {\n\t\treturn nil, err\n\t}\n\tif snapshot != nil {\n\t\tlogger.Debug(\"found snapshot before target TXID or timestamp\", \"snapshot\", snapshot.MaxTXID)\n\t\tinfos = append(infos, snapshot)\n\t}\n\n\t// Collect candidates across all compaction levels and pick the next file\n\t// from any level that extends the longest contiguous TXID range.\n\tconst maxLevel = SnapshotLevel - 1\n\tstartTXID := infos.MaxTXID()\n\tcurrentMax := startTXID\n\tif txID != 0 && currentMax >= txID {\n\t\treturn infos, nil\n\t}\n\n\tcursors := make([]*restoreLevelCursor, 0, maxLevel+1)\n\tfor level := maxLevel; level >= 0; level-- {\n\t\tlogger.Debug(\"finding ltx files for level\", \"level\", level)\n\t\titr, err := client.LTXFiles(ctx, level, 0, !timestamp.IsZero())\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tcursors = append(cursors, &restoreLevelCursor{\n\t\t\titr: itr,\n\t\t})\n\t}\n\tdefer func() {\n\t\tfor _, cursor := range cursors {\n\t\t\tif cursor != nil {\n\t\t\t\t_ = cursor.itr.Close()\n\t\t\t}\n\t\t}\n\t}()\n\n\tfor {\n\t\tvar next *restoreLevelCursor\n\t\tfor _, cursor := range cursors {\n\t\t\tif err := cursor.refresh(currentMax, txID, timestamp); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tif cursor.candidate == nil {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif next == nil || restoreCandidateBetter(next.candidate, cursor.candidate) {\n\t\t\t\tnext = cursor\n\t\t\t}\n\t\t}\n\n\t\tif next == nil || next.candidate == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tif next.candidate.MaxTXID <= currentMax {\n\t\t\tnext.candidate = nil\n\t\t\tcontinue\n\t\t}\n\n\t\tlogger.Debug(\"matching LTX file for restore\",\n\t\t\t\"filename\", ltx.FormatFilename(next.candidate.MinTXID, next.candidate.MaxTXID),\n\t\t\t\"level\", next.candidate.Level)\n\t\tinfos = append(infos, next.candidate)\n\t\tcurrentMax = next.candidate.MaxTXID\n\t\tnext.candidate = nil\n\n\t\tif txID != 0 && currentMax >= txID {\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif len(infos) > 0 && txID == 0 && timestamp.IsZero() {\n\t\tfor _, cursor := range cursors {\n\t\t\tif err := cursor.ensureCurrent(); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tif cursor.current != nil && cursor.current.MinTXID > currentMax+1 {\n\t\t\t\treturn nil, fmt.Errorf(\"non-contiguous ltx files: have up to %s but next file starts at %s\", currentMax, cursor.current.MinTXID)\n\t\t\t}\n\t\t}\n\t}\n\n\tif len(infos) == 0 {\n\t\treturn nil, ErrTxNotAvailable\n\t}\n\tif txID != 0 && infos.MaxTXID() < txID {\n\t\treturn nil, ErrTxNotAvailable\n\t}\n\n\treturn infos, nil\n}\n\ntype restoreLevelCursor struct {\n\t// itr streams LTX file infos for a single level in filename order.\n\titr ltx.FileIterator\n\t// current holds the last item read from itr but not yet evaluated.\n\tcurrent *ltx.FileInfo\n\t// candidate is the best eligible file at this level for the currentMax.\n\tcandidate *ltx.FileInfo\n\t// done indicates the iterator has been exhausted or errored.\n\tdone bool\n}\n\nfunc (c *restoreLevelCursor) refresh(currentMax, txID ltx.TXID, timestamp time.Time) error {\n\t// Advance the iterator until we've evaluated all files that could be\n\t// contiguous with currentMax. Keep the best eligible candidate.\n\tif c.done {\n\t\treturn nil\n\t}\n\tif c.candidate != nil && c.candidate.MaxTXID <= currentMax {\n\t\tc.candidate = nil\n\t}\n\n\tfor {\n\t\tif err := c.ensureCurrent(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif c.done {\n\t\t\treturn nil\n\t\t}\n\n\t\tinfo := c.current\n\t\tif info.MinTXID > currentMax+1 {\n\t\t\treturn nil\n\t\t}\n\t\tc.current = nil\n\n\t\tif info.MaxTXID <= currentMax {\n\t\t\tcontinue\n\t\t}\n\t\tif txID != 0 && info.MaxTXID > txID {\n\t\t\tcontinue\n\t\t}\n\t\tif !timestamp.IsZero() && !info.CreatedAt.Before(timestamp) {\n\t\t\tcontinue\n\t\t}\n\n\t\tif c.candidate == nil || restoreCandidateBetter(c.candidate, info) {\n\t\t\tc.candidate = info\n\t\t}\n\t}\n}\n\nfunc (c *restoreLevelCursor) ensureCurrent() error {\n\t// Ensure current is populated with the next iterator item, or mark done.\n\tif c.done || c.current != nil {\n\t\treturn nil\n\t}\n\tif !c.itr.Next() {\n\t\tif err := c.itr.Err(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tc.done = true\n\t\treturn nil\n\t}\n\tc.current = c.itr.Item()\n\treturn nil\n}\n\nfunc restoreCandidateBetter(curr, next *ltx.FileInfo) bool {\n\tif next.MaxTXID != curr.MaxTXID {\n\t\treturn next.MaxTXID > curr.MaxTXID\n\t}\n\tif next.MinTXID != curr.MinTXID {\n\t\treturn next.MinTXID < curr.MinTXID\n\t}\n\tif next.Level != curr.Level {\n\t\treturn next.Level > curr.Level\n\t}\n\treturn next.CreatedAt.Before(curr.CreatedAt)\n}\n\n// TXIDPath returns the path to the TXID sidecar file for the given database path.\n// Uses -txid suffix to match SQLite's naming convention for associated files (-wal, -shm).\nfunc TXIDPath(outputPath string) string {\n\treturn outputPath + \"-txid\"\n}\n\n// WriteTXIDFile atomically writes a TXID to a sidecar file at -txid.\n// Uses temp-file + fsync + rename for crash safety.\nfunc WriteTXIDFile(outputPath string, txid ltx.TXID) error {\n\ttxidPath := TXIDPath(outputPath)\n\ttmpPath := txidPath + \".tmp\"\n\n\tf, err := os.Create(tmpPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create txid temp file: %w\", err)\n\t}\n\tdefer f.Close()\n\tdefer os.Remove(tmpPath)\n\n\tif _, err := fmt.Fprintln(f, txid); err != nil {\n\t\treturn fmt.Errorf(\"write txid: %w\", err)\n\t}\n\n\tif err := f.Sync(); err != nil {\n\t\treturn fmt.Errorf(\"sync txid file: %w\", err)\n\t}\n\n\tif err := f.Close(); err != nil {\n\t\treturn fmt.Errorf(\"close txid file: %w\", err)\n\t}\n\n\tif err := os.Rename(tmpPath, txidPath); err != nil {\n\t\treturn fmt.Errorf(\"rename txid file: %w\", err)\n\t}\n\n\tdir, err := os.Open(filepath.Dir(txidPath))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open txid dir for sync: %w\", err)\n\t}\n\tif err := dir.Sync(); err != nil {\n\t\t_ = dir.Close()\n\t\treturn fmt.Errorf(\"sync txid dir: %w\", err)\n\t}\n\treturn dir.Close()\n}\n\n// ReadTXIDFile reads the TXID from a sidecar file at -txid.\n// Returns 0, nil if the file does not exist (first run).\nfunc ReadTXIDFile(outputPath string) (ltx.TXID, error) {\n\ttxidPath := TXIDPath(outputPath)\n\n\tdata, err := os.ReadFile(txidPath)\n\tif os.IsNotExist(err) {\n\t\treturn 0, nil\n\t} else if err != nil {\n\t\treturn 0, fmt.Errorf(\"read txid file: %w\", err)\n\t}\n\n\ttxid, err := ltx.ParseTXID(strings.TrimSpace(string(data)))\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"parse txid file: %w\", err)\n\t}\n\n\treturn txid, nil\n}\n\n// ValidationError represents a single validation issue.\ntype ValidationError struct {\n\tLevel int // compaction level\n\tType string // \"gap\", \"overlap\", or \"unsorted\"\n\tMessage string // human-readable description\n\tPrevFile *ltx.FileInfo // previous file\n\tCurrFile *ltx.FileInfo // current file that caused error\n}\n\n// ValidateLevel checks LTX files at the given level are sorted and contiguous.\n// Returns a slice of validation errors (empty if valid).\nfunc (r *Replica) ValidateLevel(ctx context.Context, level int) ([]ValidationError, error) {\n\titr, err := r.Client.LTXFiles(ctx, level, 0, false)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"fetch ltx files: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar errors []ValidationError\n\tvar prevInfo *ltx.FileInfo\n\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\n\t\t// Skip first file - nothing to compare against\n\t\tif prevInfo == nil {\n\t\t\tprevInfo = info\n\t\t\tcontinue\n\t\t}\n\n\t\t// Check for sort order: curr.MinTXID should be >= prev.MinTXID\n\t\tif info.MinTXID < prevInfo.MinTXID {\n\t\t\terrors = append(errors, ValidationError{\n\t\t\t\tLevel: level,\n\t\t\t\tType: \"unsorted\",\n\t\t\t\tMessage: fmt.Sprintf(\"files out of order: curr.MinTXID=%s < prev.MinTXID=%s\", info.MinTXID, prevInfo.MinTXID),\n\t\t\t\tPrevFile: prevInfo,\n\t\t\t\tCurrFile: info,\n\t\t\t})\n\t\t\tprevInfo = info\n\t\t\tcontinue\n\t\t}\n\n\t\t// Check for TXID contiguity: prev.MaxTXID + 1 should equal curr.MinTXID\n\t\texpectedMinTXID := prevInfo.MaxTXID + 1\n\t\tif info.MinTXID != expectedMinTXID {\n\t\t\tif info.MinTXID > expectedMinTXID {\n\t\t\t\terrors = append(errors, ValidationError{\n\t\t\t\t\tLevel: level,\n\t\t\t\t\tType: \"gap\",\n\t\t\t\t\tMessage: fmt.Sprintf(\"TXID gap: prev.MaxTXID=%s, curr.MinTXID=%s (expected %s)\", prevInfo.MaxTXID, info.MinTXID, expectedMinTXID),\n\t\t\t\t\tPrevFile: prevInfo,\n\t\t\t\t\tCurrFile: info,\n\t\t\t\t})\n\t\t\t} else {\n\t\t\t\terrors = append(errors, ValidationError{\n\t\t\t\t\tLevel: level,\n\t\t\t\t\tType: \"overlap\",\n\t\t\t\t\tMessage: fmt.Sprintf(\"TXID overlap: prev.MaxTXID=%s, curr.MinTXID=%s\", prevInfo.MaxTXID, info.MinTXID),\n\t\t\t\t\tPrevFile: prevInfo,\n\t\t\t\t\tCurrFile: info,\n\t\t\t\t})\n\t\t\t}\n\t\t}\n\n\t\tprevInfo = info\n\t}\n\n\tif err := itr.Close(); err != nil {\n\t\treturn nil, fmt.Errorf(\"close iterator: %w\", err)\n\t}\n\n\treturn errors, nil\n}\n" }, { "path": "replica_client.go", "content": "package litestream\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\n\t\"github.com/superfly/ltx\"\n)\n\nvar ErrStopIter = errors.New(\"stop iterator\")\n\n// ReplicaClient represents client to connect to a Replica.\ntype ReplicaClient interface {\n\t// Type returns the type of client.\n\tType() string\n\n\t// Init initializes the replica client connection.\n\t// This may establish connections, validate configuration, etc.\n\t// Implementations should be idempotent (no-op if already initialized).\n\tInit(ctx context.Context) error\n\n\t// LTXFiles returns an iterator of all LTX files on the replica for a given level.\n\t// If seek is specified, the iterator start from the given TXID or the next available if not found.\n\t// If useMetadata is true, the iterator fetches accurate timestamps from metadata for timestamp-based restore.\n\t// When false, the iterator uses fast timestamps (LastModified/Created/ModTime) for normal operations.\n\tLTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error)\n\n\t// OpenLTXFile returns a reader that contains an LTX file at a given TXID.\n\t// If seek is specified, the reader will start at the given offset.\n\t// Returns an os.ErrNotFound error if the LTX file does not exist.\n\tOpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n\n\t// WriteLTXFile writes an LTX file to the replica.\n\t// Returns metadata for the written file.\n\tWriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error)\n\n\t// DeleteLTXFiles deletes one or more LTX files.\n\tDeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error\n\n\t// DeleteAll deletes all files.\n\tDeleteAll(ctx context.Context) error\n\n\t// SetLogger sets the logger for the client.\n\tSetLogger(logger *slog.Logger)\n}\n\n// FindLTXFiles returns a list of files that match filter.\n// The useMetadata parameter is passed through to LTXFiles to control whether accurate timestamps\n// are fetched from metadata. When true (timestamp-based restore), accurate timestamps are required.\n// When false (normal operations), fast timestamps are sufficient.\nfunc FindLTXFiles(ctx context.Context, client ReplicaClient, level int, useMetadata bool, filter func(*ltx.FileInfo) (bool, error)) ([]*ltx.FileInfo, error) {\n\titr, err := client.LTXFiles(ctx, level, 0, useMetadata)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer func() { _ = itr.Close() }()\n\n\tvar a []*ltx.FileInfo\n\tfor itr.Next() {\n\t\titem := itr.Item()\n\n\t\tmatch, err := filter(item)\n\t\tif match {\n\t\t\ta = append(a, item)\n\t\t}\n\n\t\tif errors.Is(err, ErrStopIter) {\n\t\t\tbreak\n\t\t} else if err != nil {\n\t\t\treturn a, err\n\t\t}\n\t}\n\n\tif err := itr.Close(); err != nil {\n\t\treturn nil, err\n\t}\n\treturn a, nil\n}\n\n// DefaultEstimatedPageIndexSize is size that is first fetched when fetching the page index.\n// If the fetch was smaller than the actual page index, another call is made to fetch the rest.\nconst DefaultEstimatedPageIndexSize = 32 * 1024 // 32KB\n\nfunc FetchPageIndex(ctx context.Context, client ReplicaClient, info *ltx.FileInfo) (map[uint32]ltx.PageIndexElem, error) {\n\trc, err := fetchPageIndexData(ctx, client, info)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer rc.Close()\n\n\treturn ltx.DecodePageIndex(bufio.NewReader(rc), info.Level, info.MinTXID, info.MaxTXID)\n}\n\n// FetchLTXHeader reads & returns the LTX header for the given file info.\nfunc FetchLTXHeader(ctx context.Context, client ReplicaClient, info *ltx.FileInfo) (ltx.Header, error) {\n\trc, err := client.OpenLTXFile(ctx, info.Level, info.MinTXID, info.MaxTXID, 0, ltx.HeaderSize)\n\tif err != nil {\n\t\treturn ltx.Header{}, fmt.Errorf(\"open ltx file: %w\", err)\n\t}\n\tdefer rc.Close()\n\thdr, _, err := ltx.PeekHeader(rc)\n\tif err != nil {\n\t\treturn ltx.Header{}, fmt.Errorf(\"peek header: %w\", err)\n\t}\n\treturn hdr, nil\n}\n\n// fetchPageIndexData fetches a chunk of the end of the file to get the page index.\n// If the fetch was smaller than the actual page index, another call is made to fetch the rest.\nfunc fetchPageIndexData(ctx context.Context, client ReplicaClient, info *ltx.FileInfo) (io.ReadCloser, error) {\n\t// Fetch the end of the file to get the page index.\n\toffset := info.Size - DefaultEstimatedPageIndexSize\n\tif offset < 0 {\n\t\toffset = 0\n\t}\n\n\tf, err := client.OpenLTXFile(ctx, info.Level, info.MinTXID, info.MaxTXID, offset, 0)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"open ltx file: %w\", err)\n\t}\n\tdefer f.Close()\n\n\t// If we have read the full size of the page index, return the page index block as a reader.\n\tb, err := io.ReadAll(f)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"read ltx page index: %w\", err)\n\t}\n\tsize := binary.BigEndian.Uint64(b[len(b)-ltx.TrailerSize-8:])\n\tif off := len(b) - int(size) - ltx.TrailerSize - 8; off > 0 {\n\t\treturn io.NopCloser(bytes.NewReader(b[off:])), nil\n\t}\n\n\t// Otherwise read the file from the start of the page index.\n\tf, err = client.OpenLTXFile(ctx, info.Level, info.MinTXID, info.MaxTXID, info.Size-ltx.TrailerSize-8-int64(size), 0)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"open ltx file: %w\", err)\n\t}\n\treturn f, nil\n}\n\n// FetchPage fetches and decodes a single page frame from an LTX file.\nfunc FetchPage(ctx context.Context, client ReplicaClient, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (ltx.PageHeader, []byte, error) {\n\tf, err := client.OpenLTXFile(ctx, level, minTXID, maxTXID, offset, size)\n\tif err != nil {\n\t\treturn ltx.PageHeader{}, nil, fmt.Errorf(\"open ltx file: %w\", err)\n\t}\n\tdefer f.Close()\n\n\tb, err := io.ReadAll(f)\n\tif err != nil {\n\t\treturn ltx.PageHeader{}, nil, fmt.Errorf(\"read ltx page frame: %w\", err)\n\t}\n\treturn ltx.DecodePageData(b)\n}\n" }, { "path": "replica_client_test.go", "content": "package litestream_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"math/rand\"\n\t\"os\"\n\t\"slices\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\t\"golang.org/x/crypto/ssh\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n\t\"github.com/benbjohnson/litestream/s3\"\n)\n\n// createLTXData creates a minimal valid LTX file with a header for testing.\n// The data parameter is appended after the header for testing purposes.\nfunc createLTXData(minTXID, maxTXID ltx.TXID, data []byte) []byte {\n\treturn createLTXDataWithTimestamp(minTXID, maxTXID, time.Now(), data)\n}\n\nfunc createLTXDataWithTimestamp(minTXID, maxTXID ltx.TXID, ts time.Time, data []byte) []byte {\n\thdr := ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tPageSize: 4096,\n\t\tCommit: 1,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tTimestamp: ts.UnixMilli(),\n\t}\n\tif minTXID == 1 {\n\t\t// Snapshot files do not include a checksum.\n\t\thdr.PreApplyChecksum = 0\n\t} else {\n\t\thdr.PreApplyChecksum = ltx.ChecksumFlag\n\t}\n\n\theaderBytes, _ := hdr.MarshalBinary()\n\treturn append(headerBytes, data...)\n}\n\nfunc TestReplicaClient_LTX(t *testing.T) {\n\tRunWithReplicaClient(t, \"OK\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\t\tt.Parallel()\n\n\t\t// Write files out of order to check for sorting.\n\t\tif _, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(4), ltx.TXID(8), bytes.NewReader(createLTXData(4, 8, []byte(`67`)))); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(1), bytes.NewReader(createLTXData(1, 1, []byte(``)))); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(9), ltx.TXID(9), bytes.NewReader(createLTXData(9, 9, []byte(`xyz`)))); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(2), ltx.TXID(3), bytes.NewReader(createLTXData(2, 3, []byte(`12345`)))); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\titr, err := c.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer itr.Close()\n\n\t\t// Read all items and ensure they are sorted.\n\t\ta, err := ltx.SliceFileIterator(itr)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := len(a), 4; got != want {\n\t\t\tt.Fatalf(\"len=%v, want %v\", got, want)\n\t\t}\n\n\t\t// Check that files are sorted by MinTXID (Size no longer checked since we add LTX headers)\n\t\tif got, want := a[0].MinTXID, ltx.TXID(1); got != want {\n\t\t\tt.Fatalf(\"Index[0].MinTXID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := a[0].MaxTXID, ltx.TXID(1); got != want {\n\t\t\tt.Fatalf(\"Index[0].MaxTXID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := a[1].MinTXID, ltx.TXID(2); got != want {\n\t\t\tt.Fatalf(\"Index[1].MinTXID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := a[1].MaxTXID, ltx.TXID(3); got != want {\n\t\t\tt.Fatalf(\"Index[1].MaxTXID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := a[2].MinTXID, ltx.TXID(4); got != want {\n\t\t\tt.Fatalf(\"Index[2].MinTXID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := a[2].MaxTXID, ltx.TXID(8); got != want {\n\t\t\tt.Fatalf(\"Index[2].MaxTXID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := a[3].MinTXID, ltx.TXID(9); got != want {\n\t\t\tt.Fatalf(\"Index[3].MinTXID=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := a[3].MaxTXID, ltx.TXID(9); got != want {\n\t\t\tt.Fatalf(\"Index[3].MaxTXID=%v, want %v\", got, want)\n\t\t}\n\n\t\tif err := itr.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n\n\tRunWithReplicaClient(t, \"NoWALs\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\t\tt.Parallel()\n\n\t\titr, err := c.LTXFiles(context.Background(), 0, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer itr.Close()\n\n\t\tif itr.Next() {\n\t\t\tt.Fatal(\"expected no wal files\")\n\t\t}\n\t})\n}\n\nfunc TestReplicaClient_WriteLTXFile(t *testing.T) {\n\tRunWithReplicaClient(t, \"OK\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\t\tt.Parallel()\n\n\t\ttestData := []byte(`foobar`)\n\t\tltxData := createLTXData(1, 2, testData)\n\t\texpectedContent := ltxData\n\n\t\tif _, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(2), bytes.NewReader(expectedContent)); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tr, err := c.OpenLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(2), 0, 0)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer func() { _ = r.Close() }()\n\n\t\tbuf, err := io.ReadAll(r)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := r.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif got, want := string(buf), string(expectedContent); got != want {\n\t\t\tt.Fatalf(\"data=%q, want %q\", got, want)\n\t\t}\n\t})\n}\n\nfunc TestReplicaClient_OpenLTXFile(t *testing.T) {\n\tRunWithReplicaClient(t, \"OK\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\t\tt.Parallel()\n\n\t\ttestData := []byte(`foobar`)\n\t\tltxData := createLTXData(1, 2, testData)\n\t\texpectedContent := ltxData\n\n\t\tif _, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(2), bytes.NewReader(expectedContent)); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tr, err := c.OpenLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(2), 0, 0)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer r.Close()\n\n\t\tif buf, err := io.ReadAll(r); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := string(buf), string(expectedContent); got != want {\n\t\t\tt.Fatalf(\"ReadAll=%v, want %v\", got, want)\n\t\t}\n\t})\n\n\tRunWithReplicaClient(t, \"ErrNotFound\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\t\tt.Parallel()\n\n\t\tif _, err := c.OpenLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(1), 0, 0); !errors.Is(err, os.ErrNotExist) {\n\t\t\tt.Fatalf(\"expected not exist, got %#v\", err)\n\t\t}\n\t})\n}\n\nfunc TestReplicaClient_DeleteWALSegments(t *testing.T) {\n\tRunWithReplicaClient(t, \"OK\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\t\tt.Parallel()\n\n\t\tif _, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(2), bytes.NewReader(createLTXData(1, 2, []byte(`foo`)))); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(3), ltx.TXID(4), bytes.NewReader(createLTXData(3, 4, []byte(`bar`)))); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := c.DeleteLTXFiles(context.Background(), []*ltx.FileInfo{\n\t\t\t{Level: 0, MinTXID: 1, MaxTXID: 2},\n\t\t\t{Level: 0, MinTXID: 3, MaxTXID: 4},\n\t\t}); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif _, err := c.OpenLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(2), 0, 0); !errors.Is(err, os.ErrNotExist) {\n\t\t\tt.Fatalf(\"expected not exist, got %#v\", err)\n\t\t}\n\t\tif _, err := c.OpenLTXFile(context.Background(), 0, ltx.TXID(3), ltx.TXID(4), 0, 0); !errors.Is(err, os.ErrNotExist) {\n\t\t\tt.Fatalf(\"expected not exist, got %#v\", err)\n\t\t}\n\t})\n}\n\n// RunWithReplicaClient executes fn with each replica specified by the -integration flag\nfunc RunWithReplicaClient(t *testing.T, name string, fn func(*testing.T, litestream.ReplicaClient)) {\n\tt.Run(name, func(t *testing.T) {\n\t\tfor _, typ := range testingutil.ReplicaClientTypes() {\n\t\t\tt.Run(typ, func(t *testing.T) {\n\t\t\t\tif !testingutil.Integration() {\n\t\t\t\t\tt.Skip(\"skipping integration test, use -integration flag to run\")\n\t\t\t\t}\n\n\t\t\t\tc := testingutil.NewReplicaClient(t, typ)\n\t\t\t\tdefer testingutil.MustDeleteAll(t, c)\n\n\t\t\t\tfn(t, c)\n\t\t\t})\n\t\t}\n\t})\n}\n\n// TestReplicaClient_TimestampPreservation verifies that LTX file timestamps are preserved\n// during write and read operations. This is critical for point-in-time restoration (#771).\nfunc TestReplicaClient_TimestampPreservation(t *testing.T) {\n\tRunWithReplicaClient(t, \"PreservesTimestamp\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\t\tt.Parallel()\n\n\t\tctx := context.Background()\n\n\t\t// Create an LTX file with a specific timestamp\n\t\t// Use a timestamp from the past to ensure it's different from write time\n\t\texpectedTimestamp := time.Now().Add(-1 * time.Hour).Truncate(time.Millisecond)\n\n\t\tltxData := createLTXDataWithTimestamp(1, 1, expectedTimestamp, []byte(\"payload\"))\n\t\tinfo, err := c.WriteLTXFile(ctx, 0, ltx.TXID(1), ltx.TXID(1), bytes.NewReader(ltxData))\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// For File backend, timestamp should be preserved immediately\n\t\t// For cloud backends (S3, GCS, Azure, NATS), timestamp is stored in metadata\n\t\t// Verify the returned FileInfo has correct timestamp\n\t\tif info.CreatedAt.IsZero() {\n\t\t\tt.Fatal(\"WriteLTXFile returned zero timestamp\")\n\t\t}\n\n\t\t// Read back via LTXFiles and verify timestamp is preserved\n\t\titr, err := c.LTXFiles(ctx, 0, 0, true)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer itr.Close()\n\n\t\tvar found *ltx.FileInfo\n\t\tfor itr.Next() {\n\t\t\titem := itr.Item()\n\t\t\tif item.MinTXID == 1 && item.MaxTXID == 1 {\n\t\t\t\tfound = item\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif err := itr.Close(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif found == nil {\n\t\t\tt.Fatal(\"LTX file not found in iteration\")\n\t\t}\n\n\t\t// All backends preserve timestamps in metadata (see issue #771)\n\t\t// Verify timestamp was preserved (allow 1 second drift for precision)\n\t\ttimeDiff := found.CreatedAt.Sub(expectedTimestamp)\n\t\tif timeDiff.Abs() > time.Second {\n\t\t\tt.Errorf(\"Timestamp not preserved for backend %T: expected %v, got %v (diff: %v)\",\n\t\t\t\tc, expectedTimestamp, found.CreatedAt, timeDiff)\n\t\t}\n\t})\n}\n\n// TestReplicaClient_S3_UploaderConfig tests S3 uploader configuration for large files\nfunc TestReplicaClient_S3_UploaderConfig(t *testing.T) {\n\t// Only run for S3 integration tests\n\tif !slices.Contains(testingutil.ReplicaClientTypes(), \"s3\") {\n\t\tt.Skip(\"Skipping S3-specific uploader config test\")\n\t}\n\n\tRunWithReplicaClient(t, \"LargeFileWithCustomConfig\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\t\tt.Parallel()\n\n\t\t// Type assert to S3 client to set custom config\n\t\ts3Client, ok := c.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Skip(\"Not an S3 client\")\n\t\t}\n\n\t\t// Set custom upload configuration\n\t\ts3Client.PartSize = 5 * 1024 * 1024 // 5MB parts\n\t\ts3Client.Concurrency = 3 // 3 concurrent parts\n\n\t\t// Determine file size based on whether we're testing against moto or real S3\n\t\t// Moto has issue #8762 where composite checksums for multipart uploads\n\t\t// don't have the -X suffix, causing checksum validation to fail.\n\t\t// Reference: https://github.com/getmoto/moto/issues/8762\n\t\tsize := 10 * 1024 * 1024 // 10MB - triggers multipart upload\n\n\t\t// If we're using moto (localhost endpoint), use smaller file to avoid multipart\n\t\tif s3Client.Endpoint != \"\" && strings.Contains(s3Client.Endpoint, \"127.0.0.1\") {\n\t\t\tsize = 4 * 1024 * 1024 // 4MB - avoids multipart upload with moto\n\t\t\tt.Log(\"Using 4MB file size to work around moto multipart checksum issue\")\n\t\t} else {\n\t\t\tt.Log(\"Using 10MB file size to test multipart upload\")\n\t\t}\n\t\tpayload := make([]byte, size)\n\t\tfor i := range payload {\n\t\t\tpayload[i] = byte(i % 256)\n\t\t}\n\t\tltxData := createLTXData(1, 100, payload)\n\n\t\t// Upload the file using bytes.Reader to avoid string conversion issues\n\t\tif _, err := c.WriteLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(100), bytes.NewReader(ltxData)); err != nil {\n\t\t\tt.Fatalf(\"failed to write large file: %v\", err)\n\t\t}\n\n\t\t// Read it back and verify size\n\t\tr, err := c.OpenLTXFile(context.Background(), 0, ltx.TXID(1), ltx.TXID(100), 0, 0)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"failed to open large file: %v\", err)\n\t\t}\n\t\tdefer r.Close()\n\n\t\tbuf, err := io.ReadAll(r)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"failed to read large file: %v\", err)\n\t\t}\n\n\t\tif len(buf) != len(ltxData) {\n\t\t\tt.Errorf(\"size mismatch: got %d, want %d\", len(buf), len(ltxData))\n\t\t}\n\n\t\t// Verify the data matches what we uploaded\n\t\tif !bytes.Equal(buf, ltxData) {\n\t\t\tt.Errorf(\"data mismatch: uploaded and downloaded data do not match\")\n\t\t}\n\t})\n}\n\n// TestReplicaClient_S3_ErrorContext tests that S3 errors include helpful context\nfunc TestReplicaClient_S3_ErrorContext(t *testing.T) {\n\t// Only run for S3 integration tests\n\tif !slices.Contains(testingutil.ReplicaClientTypes(), \"s3\") {\n\t\tt.Skip(\"Skipping S3-specific error context test\")\n\t}\n\n\tRunWithReplicaClient(t, \"ErrorContext\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\t\tt.Parallel()\n\n\t\t// Test OpenLTXFile with non-existent file\n\t\t_, err := c.OpenLTXFile(context.Background(), 0, ltx.TXID(999), ltx.TXID(999), 0, 0)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for non-existent file\")\n\t\t}\n\n\t\t// Should return os.ErrNotExist for S3 NoSuchKey\n\t\tif !errors.Is(err, os.ErrNotExist) {\n\t\t\tt.Errorf(\"expected os.ErrNotExist, got %v\", err)\n\t\t}\n\t})\n}\n\n// TestReplicaClient_S3_BucketValidation tests bucket validation in S3 client\nfunc TestReplicaClient_S3_BucketValidation(t *testing.T) {\n\t// Only run for S3 integration tests\n\tif !slices.Contains(testingutil.ReplicaClientTypes(), \"s3\") {\n\t\tt.Skip(\"Skipping S3-specific bucket validation test\")\n\t}\n\n\t// Create a new S3 client with empty bucket\n\tc := testingutil.NewS3ReplicaClient(t)\n\tc.Bucket = \"\"\n\n\t// Should fail with bucket validation error\n\terr := c.Init(context.Background())\n\tif err == nil {\n\t\tt.Fatal(\"expected error for empty bucket name\")\n\t}\n\tif !strings.Contains(err.Error(), \"bucket name is required\") {\n\t\tt.Errorf(\"expected bucket validation error, got: %v\", err)\n\t}\n}\n\n// TestReplicaClient_S3_UnsignedPayloadRejected verifies that unsigned payloads\n// are rejected by real AWS S3. This is a negative test that documents the\n// expected behavior and ensures we don't accidentally ship unsigned payload\n// support for AWS S3.\n//\n// See issue #911 - AWS S3 requires signed payloads and returns\n// SignatureDoesNotMatch for unsigned payload requests.\nfunc TestReplicaClient_S3_UnsignedPayloadRejected(t *testing.T) {\n\t// Only run for S3 integration tests\n\tif !slices.Contains(testingutil.ReplicaClientTypes(), \"s3\") {\n\t\tt.Skip(\"Skipping S3-specific test\")\n\t}\n\n\t// Skip if using mock endpoint (moto accepts unsigned payloads)\n\tif endpoint := os.Getenv(\"LITESTREAM_S3_ENDPOINT\"); endpoint != \"\" {\n\t\tt.Skip(\"Skipping negative test with mock endpoint (moto accepts unsigned)\")\n\t}\n\n\t// Create client directly (not via test helper) to control SignPayload\n\tc := s3.NewReplicaClient()\n\tc.AccessKeyID = os.Getenv(\"LITESTREAM_S3_ACCESS_KEY_ID\")\n\tc.SecretAccessKey = os.Getenv(\"LITESTREAM_S3_SECRET_ACCESS_KEY\")\n\tc.Region = os.Getenv(\"LITESTREAM_S3_REGION\")\n\tif c.Region == \"\" {\n\t\tc.Region = \"us-east-1\"\n\t}\n\tc.Bucket = os.Getenv(\"LITESTREAM_S3_BUCKET\")\n\tc.Path = fmt.Sprintf(\"negative-test/%016x\", rand.Uint64())\n\n\t// Force unsigned payloads - this should fail with real AWS\n\tc.SignPayload = false\n\n\tctx := context.Background()\n\tif err := c.Init(ctx); err != nil {\n\t\tt.Fatalf(\"Init failed: %v\", err)\n\t}\n\n\t// Attempt to write - should fail with signature error\n\tltxData := createLTXData(1, 1, []byte(\"test\"))\n\t_, err := c.WriteLTXFile(ctx, 0, ltx.TXID(1), ltx.TXID(1), bytes.NewReader(ltxData))\n\n\tif err == nil {\n\t\tt.Fatal(\"expected unsigned payload to be rejected by AWS S3, but upload succeeded\")\n\t}\n\n\t// Verify it's a signature-related error\n\terrStr := strings.ToLower(err.Error())\n\tif !strings.Contains(errStr, \"signature\") && !strings.Contains(errStr, \"accessdenied\") {\n\t\tt.Errorf(\"expected signature-related error, got: %v\", err)\n\t}\n\n\tt.Logf(\"Correctly rejected unsigned payload with error: %v\", err)\n}\n\nfunc TestReplicaClient_SFTP_HostKeyValidation(t *testing.T) {\n\ttestHostKeyPEM := `-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW\nQyNTUxOQAAACAJytPhncDnpV5QF3ai8f6r0u1hzK96x+81tvtA7ZiuawAAAJAIcGGVCHBh\nlQAAAAtzc2gtZWQyNTUxOQAAACAJytPhncDnpV5QF3ai8f6r0u1hzK96x+81tvtA7Ziuaw\nAAAEDzV1D6COyvFGhSiZa6ll9aXZ2IMWED3KGrvCNjEEtYHwnK0+GdwOelXlAXdqLx/qvS\n7WHMr3rH7zW2+0DtmK5rAAAADGZlbGl4QGJvcmVhcwE=\n-----END OPENSSH PRIVATE KEY-----`\n\tprivateKey, err := ssh.ParsePrivateKey([]byte(testHostKeyPEM))\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tt.Run(\"ValidHostKey\", func(t *testing.T) {\n\t\taddr := testingutil.MockSFTPServer(t, privateKey)\n\t\texpectedHostKey := string(ssh.MarshalAuthorizedKey(privateKey.PublicKey()))\n\n\t\tc := testingutil.NewSFTPReplicaClient(t)\n\t\tc.User = \"foo\"\n\t\tc.Host = addr\n\t\tc.HostKey = expectedHostKey\n\n\t\terr = c.Init(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"SFTP connection failed: %v\", err)\n\t\t}\n\t})\n\tt.Run(\"InvalidHostKey\", func(t *testing.T) {\n\t\taddr := testingutil.MockSFTPServer(t, privateKey)\n\t\tinvalidHostKey := \"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEqM2NkGvKKhR1oiKO0E72L3tOsYk+aX7H8Xn4bbZKsa\"\n\n\t\tc := testingutil.NewSFTPReplicaClient(t)\n\t\tc.User = \"foo\"\n\t\tc.Host = addr\n\t\tc.HostKey = invalidHostKey\n\n\t\terr = c.Init(context.Background())\n\t\tif err == nil {\n\t\t\tt.Fatalf(\"SFTP connection established despite invalid host key\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"ssh: host key mismatch\") {\n\t\t\tt.Errorf(\"expected host key validation error, got: %v\", err)\n\t\t}\n\t})\n\tt.Run(\"IgnoreHostKey\", func(t *testing.T) {\n\t\tvar captured []string\n\t\tslog.SetDefault(slog.New(slog.NewTextHandler(io.Discard, &slog.HandlerOptions{\n\t\t\tLevel: slog.LevelWarn,\n\t\t\tReplaceAttr: func(groups []string, a slog.Attr) slog.Attr {\n\t\t\t\tif a.Key == slog.MessageKey {\n\t\t\t\t\tcaptured = append(captured, a.Value.String())\n\t\t\t\t}\n\t\t\t\treturn a\n\t\t\t},\n\t\t})))\n\n\t\taddr := testingutil.MockSFTPServer(t, privateKey)\n\n\t\tc := testingutil.NewSFTPReplicaClient(t)\n\t\tc.User = \"foo\"\n\t\tc.Host = addr\n\n\t\terr = c.Init(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"SFTP connection failed: %v\", err)\n\t\t}\n\n\t\tif !slices.ContainsFunc(captured, func(msg string) bool {\n\t\t\treturn strings.Contains(msg, \"sftp host key not verified\")\n\t\t}) {\n\t\t\tt.Errorf(\"Expected warning not found\")\n\t\t}\n\n\t})\n}\n\n// TestReplicaClient_S3_MultipartThresholds tests multipart upload behavior at various\n// size thresholds. These tests are critical for catching S3-compatible provider issues\n// like #940, #941, #947 where multipart uploads fail with certain providers.\n//\n// NOTE: These tests skip moto due to multipart checksum validation issues (moto#8762).\n// They should be run against real cloud providers using the manual integration workflow.\nfunc TestReplicaClient_S3_MultipartThresholds(t *testing.T) {\n\tif !slices.Contains(testingutil.ReplicaClientTypes(), \"s3\") {\n\t\tt.Skip(\"Skipping S3-specific multipart threshold tests\")\n\t}\n\n\t// Skip if using mock endpoint (moto has multipart checksum issues)\n\tif endpoint := os.Getenv(\"LITESTREAM_S3_ENDPOINT\"); endpoint != \"\" {\n\t\tif strings.Contains(endpoint, \"127.0.0.1\") || strings.Contains(endpoint, \"localhost\") {\n\t\t\tt.Skip(\"Skipping multipart tests with mock endpoint (moto has checksum issues)\")\n\t\t}\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tsizeMB int\n\t\tpartSize int64\n\t}{\n\t\t{\n\t\t\tname: \"AtThreshold_5MB\",\n\t\t\tsizeMB: 5,\n\t\t\tpartSize: 5 * 1024 * 1024,\n\t\t},\n\t\t{\n\t\t\tname: \"AboveThreshold_10MB\",\n\t\t\tsizeMB: 10,\n\t\t\tpartSize: 5 * 1024 * 1024,\n\t\t},\n\t\t{\n\t\t\tname: \"Large_50MB\",\n\t\t\tsizeMB: 50,\n\t\t\tpartSize: 10 * 1024 * 1024,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tif !testingutil.Integration() {\n\t\t\t\tt.Skip(\"skipping integration test, use -integration flag to run\")\n\t\t\t}\n\n\t\t\tc := testingutil.NewS3ReplicaClient(t)\n\t\t\tc.Path = fmt.Sprintf(\"multipart-test/%016x\", rand.Uint64())\n\t\t\tc.PartSize = tt.partSize\n\t\t\tc.Concurrency = 3\n\t\t\tdefer testingutil.MustDeleteAll(t, c)\n\n\t\t\tctx := context.Background()\n\t\t\tif err := c.Init(ctx); err != nil {\n\t\t\t\tt.Fatalf(\"Init() error: %v\", err)\n\t\t\t}\n\n\t\t\tsize := tt.sizeMB * 1024 * 1024\n\t\t\tpayload := make([]byte, size)\n\t\t\tfor i := range payload {\n\t\t\t\tpayload[i] = byte(i % 256)\n\t\t\t}\n\t\t\tltxData := createLTXData(1, 100, payload)\n\n\t\t\tt.Logf(\"Testing %dMB file with %dMB parts\", tt.sizeMB, tt.partSize/(1024*1024))\n\n\t\t\tif _, err := c.WriteLTXFile(ctx, 0, ltx.TXID(1), ltx.TXID(100), bytes.NewReader(ltxData)); err != nil {\n\t\t\t\tt.Fatalf(\"WriteLTXFile() error: %v\", err)\n\t\t\t}\n\n\t\t\tr, err := c.OpenLTXFile(ctx, 0, ltx.TXID(1), ltx.TXID(100), 0, 0)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"OpenLTXFile() error: %v\", err)\n\t\t\t}\n\t\t\tdefer r.Close()\n\n\t\t\tbuf, err := io.ReadAll(r)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"ReadAll() error: %v\", err)\n\t\t\t}\n\n\t\t\tif len(buf) != len(ltxData) {\n\t\t\t\tt.Errorf(\"size mismatch: got %d, want %d\", len(buf), len(ltxData))\n\t\t\t}\n\n\t\t\tif !bytes.Equal(buf, ltxData) {\n\t\t\t\tt.Errorf(\"data mismatch: uploaded and downloaded data do not match\")\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestReplicaClient_S3_ConcurrencyLimits tests that concurrency limits are respected\n// during multipart uploads. This is important for providers like Cloudflare R2 that\n// have strict concurrent upload limits (issue #948).\nfunc TestReplicaClient_S3_ConcurrencyLimits(t *testing.T) {\n\tif !slices.Contains(testingutil.ReplicaClientTypes(), \"s3\") {\n\t\tt.Skip(\"Skipping S3-specific concurrency test\")\n\t}\n\n\t// Skip if using mock endpoint\n\tif endpoint := os.Getenv(\"LITESTREAM_S3_ENDPOINT\"); endpoint != \"\" {\n\t\tif strings.Contains(endpoint, \"127.0.0.1\") || strings.Contains(endpoint, \"localhost\") {\n\t\t\tt.Skip(\"Skipping concurrency test with mock endpoint\")\n\t\t}\n\t}\n\n\tif !testingutil.Integration() {\n\t\tt.Skip(\"skipping integration test, use -integration flag to run\")\n\t}\n\n\tconcurrencyLevels := []int{1, 2, 5}\n\n\tfor _, concurrency := range concurrencyLevels {\n\t\tt.Run(fmt.Sprintf(\"Concurrency_%d\", concurrency), func(t *testing.T) {\n\t\t\tc := testingutil.NewS3ReplicaClient(t)\n\t\t\tc.Path = fmt.Sprintf(\"concurrency-test/%016x\", rand.Uint64())\n\t\t\tc.PartSize = 5 * 1024 * 1024\n\t\t\tc.Concurrency = concurrency\n\t\t\tdefer testingutil.MustDeleteAll(t, c)\n\n\t\t\tctx := context.Background()\n\t\t\tif err := c.Init(ctx); err != nil {\n\t\t\t\tt.Fatalf(\"Init() error: %v\", err)\n\t\t\t}\n\n\t\t\tsize := 15 * 1024 * 1024\n\t\t\tpayload := make([]byte, size)\n\t\t\tfor i := range payload {\n\t\t\t\tpayload[i] = byte(i % 256)\n\t\t\t}\n\t\t\tltxData := createLTXData(1, 100, payload)\n\n\t\t\tt.Logf(\"Testing 15MB file with concurrency=%d\", concurrency)\n\n\t\t\tif _, err := c.WriteLTXFile(ctx, 0, ltx.TXID(1), ltx.TXID(100), bytes.NewReader(ltxData)); err != nil {\n\t\t\t\tt.Fatalf(\"WriteLTXFile() with concurrency=%d error: %v\", concurrency, err)\n\t\t\t}\n\n\t\t\tr, err := c.OpenLTXFile(ctx, 0, ltx.TXID(1), ltx.TXID(100), 0, 0)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"OpenLTXFile() error: %v\", err)\n\t\t\t}\n\t\t\tdefer r.Close()\n\n\t\t\tbuf, err := io.ReadAll(r)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"ReadAll() error: %v\", err)\n\t\t\t}\n\n\t\t\tif !bytes.Equal(buf, ltxData) {\n\t\t\t\tt.Errorf(\"data mismatch at concurrency=%d\", concurrency)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestReplicaClient_PITR_ManyLTXFiles tests point-in-time restore with many LTX files.\n// This is a regression test for issue #930 where HeadObject calls with 100+ LTX files\n// caused the restore operation to hang.\nfunc TestReplicaClient_PITR_ManyLTXFiles(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfileCount int\n\t\ttimeout time.Duration\n\t}{\n\t\t{\"100_Files\", 100, 2 * time.Minute},\n\t\t{\"500_Files\", 500, 5 * time.Minute},\n\t\t{\"1000_Files\", 1000, 10 * time.Minute},\n\t}\n\n\tfor _, tt := range tests {\n\t\tRunWithReplicaClient(t, tt.name, func(t *testing.T, c litestream.ReplicaClient) {\n\t\t\tt.Helper()\n\n\t\t\t// Skip very long tests unless explicitly enabled\n\t\t\tif tt.fileCount > 100 && os.Getenv(\"LITESTREAM_PITR_STRESS_TEST\") == \"\" {\n\t\t\t\tt.Skipf(\"Skipping %d file stress test (set LITESTREAM_PITR_STRESS_TEST=1 to enable)\", tt.fileCount)\n\t\t\t}\n\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), tt.timeout)\n\t\t\tdefer cancel()\n\n\t\t\tbaseTime := time.Now().Add(-time.Duration(tt.fileCount) * time.Minute)\n\t\t\tt.Logf(\"Creating %d LTX files starting from %v\", tt.fileCount, baseTime)\n\n\t\t\t// Create snapshot at TXID 1\n\t\t\tsnapshot := createLTXDataWithTimestamp(1, 1, baseTime, []byte(\"snapshot\"))\n\t\t\tif _, err := c.WriteLTXFile(ctx, litestream.SnapshotLevel, 1, 1, bytes.NewReader(snapshot)); err != nil {\n\t\t\t\tt.Fatalf(\"WriteLTXFile(snapshot): %v\", err)\n\t\t\t}\n\n\t\t\t// Create many L0 files with incrementing timestamps\n\t\t\tfor i := 2; i <= tt.fileCount; i++ {\n\t\t\t\tts := baseTime.Add(time.Duration(i-1) * time.Minute)\n\t\t\t\tdata := createLTXDataWithTimestamp(ltx.TXID(i), ltx.TXID(i), ts, []byte(fmt.Sprintf(\"file-%d\", i)))\n\t\t\t\tif _, err := c.WriteLTXFile(ctx, 0, ltx.TXID(i), ltx.TXID(i), bytes.NewReader(data)); err != nil {\n\t\t\t\t\tt.Fatalf(\"WriteLTXFile(%d): %v\", i, err)\n\t\t\t\t}\n\t\t\t\tif i%100 == 0 {\n\t\t\t\t\tt.Logf(\"Created %d/%d files\", i, tt.fileCount)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Test 1: Iterate all L0 files without metadata (fast path)\n\t\t\tt.Log(\"Testing L0 file iteration without metadata\")\n\t\t\tstartFast := time.Now()\n\t\t\titr, err := c.LTXFiles(ctx, 0, 0, false)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"LTXFiles(useMetadata=false): %v\", err)\n\t\t\t}\n\t\t\tvar countFast int\n\t\t\tfor itr.Next() {\n\t\t\t\tcountFast++\n\t\t\t}\n\t\t\tif err := itr.Close(); err != nil {\n\t\t\t\tt.Fatalf(\"Iterator close: %v\", err)\n\t\t\t}\n\t\t\tdurationFast := time.Since(startFast)\n\t\t\tt.Logf(\"Fast iteration: %d files in %v\", countFast, durationFast)\n\n\t\t\tif countFast != tt.fileCount-1 {\n\t\t\t\tt.Errorf(\"Fast iteration count: got %d, want %d\", countFast, tt.fileCount-1)\n\t\t\t}\n\n\t\t\t// Test 2: Iterate all L0 files with metadata (required for PITR)\n\t\t\t// This is the path that was hanging in issue #930\n\t\t\tt.Log(\"Testing L0 file iteration with metadata (PITR path)\")\n\t\t\tstartMeta := time.Now()\n\t\t\titrMeta, err := c.LTXFiles(ctx, 0, 0, true)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"LTXFiles(useMetadata=true): %v\", err)\n\t\t\t}\n\t\t\tvar countMeta int\n\t\t\tfor itrMeta.Next() {\n\t\t\t\tcountMeta++\n\t\t\t}\n\t\t\tif err := itrMeta.Close(); err != nil {\n\t\t\t\tt.Fatalf(\"Iterator close: %v\", err)\n\t\t\t}\n\t\t\tdurationMeta := time.Since(startMeta)\n\t\t\tt.Logf(\"Metadata iteration: %d files in %v\", countMeta, durationMeta)\n\n\t\t\tif countMeta != tt.fileCount-1 {\n\t\t\t\tt.Errorf(\"Metadata iteration count: got %d, want %d\", countMeta, tt.fileCount-1)\n\t\t\t}\n\n\t\t\t// Verify metadata iteration completed within reasonable time\n\t\t\t// (issue #930 caused this to hang indefinitely)\n\t\t\tif durationMeta > tt.timeout/2 {\n\t\t\t\tt.Errorf(\"Metadata iteration took too long: %v (should be < %v)\", durationMeta, tt.timeout/2)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestReplicaClient_PITR_TimestampFiltering tests that PITR correctly filters files\n// by timestamp across a range of LTX files.\nfunc TestReplicaClient_PITR_TimestampFiltering(t *testing.T) {\n\tRunWithReplicaClient(t, \"TimestampFilter\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\n\t\tctx := context.Background()\n\t\tfileCount := 50\n\t\tbaseTime := time.Now().Add(-time.Duration(fileCount) * time.Minute)\n\n\t\t// Create snapshot at TXID 1\n\t\tsnapshot := createLTXDataWithTimestamp(1, 1, baseTime, []byte(\"snapshot\"))\n\t\tif _, err := c.WriteLTXFile(ctx, litestream.SnapshotLevel, 1, 1, bytes.NewReader(snapshot)); err != nil {\n\t\t\tt.Fatalf(\"WriteLTXFile(snapshot): %v\", err)\n\t\t}\n\n\t\t// Create L0 files with known timestamps\n\t\tfor i := 2; i <= fileCount; i++ {\n\t\t\tts := baseTime.Add(time.Duration(i-1) * time.Minute)\n\t\t\tdata := createLTXDataWithTimestamp(ltx.TXID(i), ltx.TXID(i), ts, []byte(fmt.Sprintf(\"file-%d\", i)))\n\t\t\tif _, err := c.WriteLTXFile(ctx, 0, ltx.TXID(i), ltx.TXID(i), bytes.NewReader(data)); err != nil {\n\t\t\t\tt.Fatalf(\"WriteLTXFile(%d): %v\", i, err)\n\t\t\t}\n\t\t}\n\n\t\t// Test filtering at various timestamp points\n\t\ttestPoints := []struct {\n\t\t\tname string\n\t\t\toffsetMins int\n\t\t\texpectCount int\n\t\t}{\n\t\t\t{\"Beginning\", 5, 4}, // Files 2-5 (4 files)\n\t\t\t{\"Quarter\", 12, 11}, // Files 2-12 (11 files)\n\t\t\t{\"Middle\", 25, 24}, // Files 2-25 (24 files)\n\t\t\t{\"ThreeQuarters\", 37, 36}, // Files 2-37 (36 files)\n\t\t\t{\"End\", fileCount - 1, fileCount - 2}, // All but last\n\t\t}\n\n\t\tfor _, tp := range testPoints {\n\t\t\tt.Run(tp.name, func(t *testing.T) {\n\t\t\t\ttargetTime := baseTime.Add(time.Duration(tp.offsetMins) * time.Minute)\n\t\t\t\tt.Logf(\"Filtering files before %v (offset: %d mins)\", targetTime, tp.offsetMins)\n\n\t\t\t\t// Use LTXFiles with metadata to get accurate timestamps\n\t\t\t\titr, err := c.LTXFiles(ctx, 0, 0, true)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"LTXFiles: %v\", err)\n\t\t\t\t}\n\t\t\t\tdefer itr.Close()\n\n\t\t\t\tvar count int\n\t\t\t\tfor itr.Next() {\n\t\t\t\t\tinfo := itr.Item()\n\t\t\t\t\tif info.CreatedAt.Before(targetTime) {\n\t\t\t\t\t\tcount++\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// Allow for timestamp precision variance\n\t\t\t\tif count < tp.expectCount-1 || count > tp.expectCount+1 {\n\t\t\t\t\tt.Errorf(\"Files before %v: got %d, expected ~%d\", targetTime, count, tp.expectCount)\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\t})\n}\n\n// TestReplicaClient_PITR_CalcRestorePlanWithManyFiles tests CalcRestorePlan with\n// a large number of LTX files. This ensures restore planning doesn't hang.\nfunc TestReplicaClient_PITR_CalcRestorePlanWithManyFiles(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tRunWithReplicaClient(t, \"RestorePlan\", func(t *testing.T, c litestream.ReplicaClient) {\n\t\tt.Helper()\n\n\t\tctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)\n\t\tdefer cancel()\n\n\t\tfileCount := 100\n\t\tbaseTime := time.Now().Add(-time.Duration(fileCount) * time.Minute)\n\n\t\t// Create snapshot\n\t\tsnapshot := createLTXDataWithTimestamp(1, 1, baseTime, []byte(\"snapshot\"))\n\t\tif _, err := c.WriteLTXFile(ctx, litestream.SnapshotLevel, 1, 1, bytes.NewReader(snapshot)); err != nil {\n\t\t\tt.Fatalf(\"WriteLTXFile(snapshot): %v\", err)\n\t\t}\n\n\t\t// Create L0 files\n\t\tfor i := 2; i <= fileCount; i++ {\n\t\t\tts := baseTime.Add(time.Duration(i-1) * time.Minute)\n\t\t\tdata := createLTXDataWithTimestamp(ltx.TXID(i), ltx.TXID(i), ts, []byte(fmt.Sprintf(\"file-%d\", i)))\n\t\t\tif _, err := c.WriteLTXFile(ctx, 0, ltx.TXID(i), ltx.TXID(i), bytes.NewReader(data)); err != nil {\n\t\t\t\tt.Fatalf(\"WriteLTXFile(%d): %v\", i, err)\n\t\t\t}\n\t\t}\n\n\t\t// Test restore plan calculation at various points\n\t\ttestTargets := []struct {\n\t\t\tname string\n\t\t\ttxID ltx.TXID\n\t\t\tminFiles int\n\t\t}{\n\t\t\t{\"EarlyTXID\", 10, 2}, // snapshot + some L0\n\t\t\t{\"MidTXID\", 50, 2}, // snapshot + more L0\n\t\t\t{\"LateTXID\", 90, 2}, // snapshot + most L0\n\t\t\t{\"LatestTXID\", ltx.TXID(fileCount), 2}, // all files\n\t\t}\n\n\t\tlogger := slog.Default()\n\n\t\tfor _, target := range testTargets {\n\t\t\tt.Run(target.name, func(t *testing.T) {\n\t\t\t\tstartTime := time.Now()\n\n\t\t\t\tplan, err := litestream.CalcRestorePlan(ctx, c, target.txID, time.Time{}, logger)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"CalcRestorePlan(%d): %v\", target.txID, err)\n\t\t\t\t}\n\n\t\t\t\tduration := time.Since(startTime)\n\t\t\t\tt.Logf(\"CalcRestorePlan(txid=%d): %d files in %v\", target.txID, len(plan), duration)\n\n\t\t\t\tif len(plan) < target.minFiles {\n\t\t\t\t\tt.Errorf(\"Plan has too few files: got %d, want >= %d\", len(plan), target.minFiles)\n\t\t\t\t}\n\n\t\t\t\t// Verify plan doesn't take excessively long\n\t\t\t\tif duration > 30*time.Second {\n\t\t\t\t\tt.Errorf(\"CalcRestorePlan took too long: %v (should be < 30s)\", duration)\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\n\t\t// Test timestamp-based restore plan\n\t\tt.Run(\"TimestampBased\", func(t *testing.T) {\n\t\t\t// Target halfway through the files\n\t\t\ttargetTime := baseTime.Add(time.Duration(fileCount/2) * time.Minute)\n\t\t\tstartTime := time.Now()\n\n\t\t\tplan, err := litestream.CalcRestorePlan(ctx, c, 0, targetTime, logger)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"CalcRestorePlan(timestamp=%v): %v\", targetTime, err)\n\t\t\t}\n\n\t\t\tduration := time.Since(startTime)\n\t\t\tt.Logf(\"CalcRestorePlan(timestamp=%v): %d files in %v\", targetTime, len(plan), duration)\n\n\t\t\tif len(plan) < 2 {\n\t\t\t\tt.Errorf(\"Plan has too few files: got %d, want >= 2\", len(plan))\n\t\t\t}\n\n\t\t\tif duration > 60*time.Second {\n\t\t\t\tt.Errorf(\"Timestamp-based CalcRestorePlan took too long: %v\", duration)\n\t\t\t}\n\t\t})\n\t})\n}\n" }, { "path": "replica_internal_test.go", "content": "package litestream\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t_ \"modernc.org/sqlite\"\n)\n\nfunc TestReplica_ApplyNewLTXFiles_FillGapWithOverlappingCompactedFile(t *testing.T) {\n\tconst pageSize = 4096\n\n\tcompactedInfo := <x.FileInfo{Level: 1, MinTXID: 100, MaxTXID: 200}\n\tl0Info := <x.FileInfo{Level: 0, MinTXID: 201, MaxTXID: 201}\n\n\tfixtures := map[string][]byte{\n\t\tltxFixtureKey(compactedInfo.Level, compactedInfo.MinTXID, compactedInfo.MaxTXID): mustBuildIncrementalLTX(t, compactedInfo.MinTXID, compactedInfo.MaxTXID, pageSize, 1, 0xA1),\n\t\tltxFixtureKey(l0Info.Level, l0Info.MinTXID, l0Info.MaxTXID): mustBuildIncrementalLTX(t, l0Info.MinTXID, l0Info.MaxTXID, pageSize, 1, 0xB2),\n\t}\n\n\tclient := &followTestReplicaClient{}\n\tclient.LTXFilesFunc = func(_ context.Context, level int, seek ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\tvar all []*ltx.FileInfo\n\t\tswitch level {\n\t\tcase 0:\n\t\t\tall = []*ltx.FileInfo{l0Info}\n\t\tcase 1:\n\t\t\tall = []*ltx.FileInfo{compactedInfo}\n\t\tdefault:\n\t\t\tall = nil\n\t\t}\n\n\t\tinfos := make([]*ltx.FileInfo, 0, len(all))\n\t\tfor _, info := range all {\n\t\t\tif info.MinTXID >= seek {\n\t\t\t\tinfos = append(infos, info)\n\t\t\t}\n\t\t}\n\t\treturn ltx.NewFileInfoSliceIterator(infos), nil\n\t}\n\tclient.OpenLTXFileFunc = func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, _, _ int64) (io.ReadCloser, error) {\n\t\tkey := ltxFixtureKey(level, minTXID, maxTXID)\n\t\tdata, ok := fixtures[key]\n\t\tif !ok {\n\t\t\treturn nil, os.ErrNotExist\n\t\t}\n\t\treturn io.NopCloser(bytes.NewReader(data)), nil\n\t}\n\n\tr := NewReplicaWithClient(nil, client)\n\tf := mustCreateWritableDBFile(t)\n\tdefer func() { _ = f.Close() }()\n\n\tgot, err := r.applyNewLTXFiles(context.Background(), f, 150, pageSize)\n\tif err != nil {\n\t\tt.Fatalf(\"apply new ltx files: %v\", err)\n\t}\n\tif got != 201 {\n\t\tt.Fatalf(\"txid=%s, want %s\", got, ltx.TXID(201))\n\t}\n}\n\nfunc TestReplica_ApplyNewLTXFiles_LevelZeroEmptyFallsBackToCompaction(t *testing.T) {\n\tconst pageSize = 4096\n\n\tcompactedInfo := <x.FileInfo{Level: 1, MinTXID: 11, MaxTXID: 12}\n\tfixtures := map[string][]byte{\n\t\tltxFixtureKey(compactedInfo.Level, compactedInfo.MinTXID, compactedInfo.MaxTXID): mustBuildIncrementalLTX(t, compactedInfo.MinTXID, compactedInfo.MaxTXID, pageSize, 1, 0xC3),\n\t}\n\n\tclient := &followTestReplicaClient{}\n\tclient.LTXFilesFunc = func(_ context.Context, level int, seek ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\tswitch level {\n\t\tcase 0:\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\tcase 1:\n\t\t\tif compactedInfo.MinTXID < seek {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{compactedInfo}), nil\n\t\tdefault:\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\t}\n\tclient.OpenLTXFileFunc = func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, _, _ int64) (io.ReadCloser, error) {\n\t\tkey := ltxFixtureKey(level, minTXID, maxTXID)\n\t\tdata, ok := fixtures[key]\n\t\tif !ok {\n\t\t\treturn nil, os.ErrNotExist\n\t\t}\n\t\treturn io.NopCloser(bytes.NewReader(data)), nil\n\t}\n\n\tr := NewReplicaWithClient(nil, client)\n\tf := mustCreateWritableDBFile(t)\n\tdefer func() { _ = f.Close() }()\n\n\tgot, err := r.applyNewLTXFiles(context.Background(), f, 10, pageSize)\n\tif err != nil {\n\t\tt.Fatalf(\"apply new ltx files: %v\", err)\n\t}\n\tif got != 12 {\n\t\tt.Fatalf(\"txid=%s, want %s\", got, ltx.TXID(12))\n\t}\n}\n\nfunc TestReplica_ApplyNewLTXFiles_IteratorCloseError(t *testing.T) {\n\tclient := &followTestReplicaClient{}\n\tclient.LTXFilesFunc = func(_ context.Context, level int, seek ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\t\tif level == 0 {\n\t\t\treturn &errorFileIterator{closeErr: fmt.Errorf(\"level 0 listing failed\")}, nil\n\t\t}\n\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t}\n\tclient.OpenLTXFileFunc = func(_ context.Context, _ int, _, _ ltx.TXID, _, _ int64) (io.ReadCloser, error) {\n\t\treturn nil, fmt.Errorf(\"unexpected open\")\n\t}\n\n\tr := NewReplicaWithClient(nil, client)\n\tf := mustCreateWritableDBFile(t)\n\tdefer func() { _ = f.Close() }()\n\n\t_, err := r.applyNewLTXFiles(context.Background(), f, 10, 4096)\n\tif err == nil {\n\t\tt.Fatal(\"expected error\")\n\t}\n\tif got, want := err.Error(), \"level 0 listing failed\"; !bytes.Contains([]byte(got), []byte(want)) {\n\t\tt.Fatalf(\"error=%q, want substring %q\", got, want)\n\t}\n}\n\nfunc TestReplica_ApplyLTXFile_VerifiesChecksumOnClose(t *testing.T) {\n\tconst pageSize = 4096\n\n\tinfo := <x.FileInfo{Level: 0, MinTXID: 20, MaxTXID: 20}\n\tdata := mustBuildIncrementalLTX(t, info.MinTXID, info.MaxTXID, pageSize, 1, 0xD4)\n\tdata[len(data)-1] ^= 0xFF\n\n\tclient := &followTestReplicaClient{}\n\tclient.OpenLTXFileFunc = func(_ context.Context, level int, minTXID, maxTXID ltx.TXID, _, _ int64) (io.ReadCloser, error) {\n\t\tif level != info.Level || minTXID != info.MinTXID || maxTXID != info.MaxTXID {\n\t\t\treturn nil, os.ErrNotExist\n\t\t}\n\t\treturn io.NopCloser(bytes.NewReader(data)), nil\n\t}\n\n\tr := NewReplicaWithClient(nil, client)\n\tf := mustCreateWritableDBFile(t)\n\tdefer func() { _ = f.Close() }()\n\n\terr := r.applyLTXFile(context.Background(), f, info, pageSize)\n\tif err == nil {\n\t\tt.Fatal(\"expected checksum validation error\")\n\t}\n}\n\ntype errorFileIterator struct {\n\tcloseErr error\n}\n\ntype followTestReplicaClient struct {\n\tLTXFilesFunc func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error)\n\tOpenLTXFileFunc func(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n\tWriteLTXFileFunc func(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error)\n\tDeleteLTXFilesFunc func(ctx context.Context, a []*ltx.FileInfo) error\n\tDeleteAllFunc func(ctx context.Context) error\n}\n\nfunc (*followTestReplicaClient) Type() string { return \"test\" }\n\nfunc (*followTestReplicaClient) Init(context.Context) error { return nil }\n\nfunc (*followTestReplicaClient) SetLogger(*slog.Logger) {}\n\nfunc (c *followTestReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif c.LTXFilesFunc != nil {\n\t\treturn c.LTXFilesFunc(ctx, level, seek, useMetadata)\n\t}\n\treturn ltx.NewFileInfoSliceIterator(nil), nil\n}\n\nfunc (c *followTestReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tif c.OpenLTXFileFunc != nil {\n\t\treturn c.OpenLTXFileFunc(ctx, level, minTXID, maxTXID, offset, size)\n\t}\n\treturn nil, os.ErrNotExist\n}\n\nfunc (c *followTestReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\tif c.WriteLTXFileFunc != nil {\n\t\treturn c.WriteLTXFileFunc(ctx, level, minTXID, maxTXID, r)\n\t}\n\treturn nil, fmt.Errorf(\"not implemented\")\n}\n\nfunc (c *followTestReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\tif c.DeleteLTXFilesFunc != nil {\n\t\treturn c.DeleteLTXFilesFunc(ctx, a)\n\t}\n\treturn nil\n}\n\nfunc (c *followTestReplicaClient) DeleteAll(ctx context.Context) error {\n\tif c.DeleteAllFunc != nil {\n\t\treturn c.DeleteAllFunc(ctx)\n\t}\n\treturn nil\n}\n\nfunc (itr *errorFileIterator) Close() error {\n\treturn itr.closeErr\n}\n\nfunc (itr *errorFileIterator) Next() bool {\n\treturn false\n}\n\nfunc (itr *errorFileIterator) Err() error {\n\treturn itr.closeErr\n}\n\nfunc (itr *errorFileIterator) Item() *ltx.FileInfo {\n\treturn nil\n}\n\nfunc mustBuildIncrementalLTX(tb testing.TB, minTXID, maxTXID ltx.TXID, pageSize, pgno uint32, fill byte) []byte {\n\ttb.Helper()\n\n\tvar buf bytes.Buffer\n\tenc, err := ltx.NewEncoder(&buf)\n\tif err != nil {\n\t\ttb.Fatal(err)\n\t}\n\n\thdr := ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\tPageSize: pageSize,\n\t\tCommit: pgno,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tTimestamp: time.Now().UnixMilli(),\n\t}\n\tif err := enc.EncodeHeader(hdr); err != nil {\n\t\ttb.Fatal(err)\n\t}\n\tpage := bytes.Repeat([]byte{fill}, int(pageSize))\n\tif err := enc.EncodePage(ltx.PageHeader{Pgno: pgno}, page); err != nil {\n\t\ttb.Fatal(err)\n\t}\n\tif err := enc.Close(); err != nil {\n\t\ttb.Fatal(err)\n\t}\n\n\treturn buf.Bytes()\n}\n\nfunc mustCreateWritableDBFile(tb testing.TB) *os.File {\n\ttb.Helper()\n\n\tpath := filepath.Join(tb.TempDir(), \"follower.db\")\n\tf, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR|os.O_TRUNC, 0o600)\n\tif err != nil {\n\t\ttb.Fatal(err)\n\t}\n\tif err := f.Truncate(128 * 1024); err != nil {\n\t\t_ = f.Close()\n\t\ttb.Fatal(err)\n\t}\n\treturn f\n}\n\nfunc ltxFixtureKey(level int, minTXID, maxTXID ltx.TXID) string {\n\treturn fmt.Sprintf(\"%d:%s:%s\", level, minTXID, maxTXID)\n}\n\nfunc mustCreateValidSQLiteDB(tb testing.TB) string {\n\ttb.Helper()\n\tdbPath := filepath.Join(tb.TempDir(), \"test.db\")\n\tdb, err := sql.Open(\"sqlite\", dbPath)\n\tif err != nil {\n\t\ttb.Fatal(err)\n\t}\n\tdefer func() { _ = db.Close() }()\n\tif _, err := db.Exec(\"CREATE TABLE t (id INTEGER PRIMARY KEY, name TEXT)\"); err != nil {\n\t\ttb.Fatal(err)\n\t}\n\tif _, err := db.Exec(\"INSERT INTO t (name) VALUES ('a'), ('b'), ('c')\"); err != nil {\n\t\ttb.Fatal(err)\n\t}\n\tif _, err := db.Exec(\"CREATE INDEX idx_t_name ON t(name)\"); err != nil {\n\t\ttb.Fatal(err)\n\t}\n\treturn dbPath\n}\n\nfunc TestCheckIntegrity_Quick_ValidDB(t *testing.T) {\n\tdbPath := mustCreateValidSQLiteDB(t)\n\tif err := checkIntegrity(context.Background(), dbPath, IntegrityCheckQuick); err != nil {\n\t\tt.Fatalf(\"expected no error, got: %v\", err)\n\t}\n}\n\nfunc TestCheckIntegrity_Full_ValidDB(t *testing.T) {\n\tdbPath := mustCreateValidSQLiteDB(t)\n\tif err := checkIntegrity(context.Background(), dbPath, IntegrityCheckFull); err != nil {\n\t\tt.Fatalf(\"expected no error, got: %v\", err)\n\t}\n}\n\nfunc TestCheckIntegrity_None_Skips(t *testing.T) {\n\tif err := checkIntegrity(context.Background(), \"/nonexistent/path.db\", IntegrityCheckNone); err != nil {\n\t\tt.Fatalf(\"expected nil for IntegrityCheckNone, got: %v\", err)\n\t}\n}\n\nfunc TestCheckIntegrity_CorruptDB(t *testing.T) {\n\tdbPath := mustCreateValidSQLiteDB(t)\n\n\t// Remove any WAL/SHM files so we have a clean single-file database.\n\t_ = os.Remove(dbPath + \"-wal\")\n\t_ = os.Remove(dbPath + \"-shm\")\n\n\t// Read the page size from the database header (bytes 16-17, big-endian).\n\tf, err := os.OpenFile(dbPath, os.O_RDWR, 0o600)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Corrupt page 2 onwards. Page 1 is the header/schema page. Corrupting\n\t// pages that contain table/index data triggers integrity check failures.\n\t// We overwrite from byte offset 4096 (start of page 2 for 4096-byte pages,\n\t// which is the default) with garbage data.\n\tinfo, err := f.Stat()\n\tif err != nil {\n\t\t_ = f.Close()\n\t\tt.Fatal(err)\n\t}\n\n\t// Overwrite everything after the first page with garbage to ensure corruption.\n\tpageSize := int64(4096)\n\tif info.Size() > pageSize {\n\t\tgarbage := bytes.Repeat([]byte{0xDE}, int(info.Size()-pageSize))\n\t\tif _, err := f.WriteAt(garbage, pageSize); err != nil {\n\t\t\t_ = f.Close()\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\t_ = f.Close()\n\n\terr = checkIntegrity(context.Background(), dbPath, IntegrityCheckFull)\n\tif err == nil {\n\t\tt.Fatal(\"expected integrity check to fail on corrupt database\")\n\t}\n}\n" }, { "path": "replica_test.go", "content": "package litestream_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pierrec/lz4/v4\"\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n\t\"github.com/benbjohnson/litestream/mock\"\n)\n\nfunc TestReplica_Sync(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tt.Log(\"initial sync\")\n\n\t// Issue initial database sync.\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Fetch current database position.\n\tdpos, err := db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tt.Logf(\"position after sync: %s\", dpos.String())\n\n\tc := file.NewReplicaClient(t.TempDir())\n\tr := litestream.NewReplicaWithClient(db, c)\n\n\tif err := r.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tt.Logf(\"second sync\")\n\n\t// Verify we synced checkpoint page to WAL.\n\trd, err := c.OpenLTXFile(context.Background(), 0, dpos.TXID, dpos.TXID, 0, 0)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer func() { _ = rd.Close() }()\n\n\tdec := ltx.NewDecoder(rd)\n\tif err := dec.Verify(); err != nil {\n\t\tt.Fatal(err)\n\t} else if err := rd.Close(); err != nil {\n\t\tt.Fatal(err)\n\t} else if got, want := int(dec.Header().PageSize), db.PageSize(); got != want {\n\t\tt.Fatalf(\"page size: %d, want %d\", got, want)\n\t}\n\n\t// Reset WAL so the next write will only write out the segment we are checking.\n\tif err := db.Checkpoint(context.Background(), litestream.CheckpointModeTruncate); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Execute a query to write something into the truncated WAL.\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE foo (bar TEXT);`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sync database to catch up the shadow WAL.\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Save position after sync, it should be after our write.\n\t_, err = db.Pos()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sync WAL segment out to replica.\n\tif err := r.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// TODO(ltx): Restore snapshot and verify\n}\n\n// TestReplica_RestoreAndReplicateAfterDataLoss tests the scenario described in issue #781\n// where a database is restored to an earlier state (with lower TXID) but the replica has\n// a higher TXID, causing new writes to not be replicated.\n//\n// The fix detects this condition in DB.init() by comparing database position vs replica\n// position. When database is behind, it fetches the latest L0 file from the replica and\n// triggers a snapshot on the next sync.\n//\n// This test follows the reproduction steps from issue #781:\n// 1. Create DB and replicate data\n// 2. Restore from backup (simulating hard recovery)\n// 3. Insert new data and replicate\n// 4. Restore again and verify new data exists\nfunc TestReplica_RestoreAndReplicateAfterDataLoss(t *testing.T) {\n\tctx := context.Background()\n\n\t// Create a temporary directory for replica storage\n\treplicaDir := t.TempDir()\n\treplicaClient := file.NewReplicaClient(replicaDir)\n\n\t// Create database with initial data\n\tdbDir := t.TempDir()\n\tdbPath := dbDir + \"/db.sqlite\"\n\n\t// Step 1: Create initial data and replicate\n\tsqldb := testingutil.MustOpenSQLDB(t, dbPath)\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE test(col1 INTEGER);`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO test VALUES (1);`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := sqldb.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Start litestream replication\n\tdb1 := testingutil.NewDB(t, dbPath)\n\tdb1.MonitorInterval = 0\n\tdb1.Replica = litestream.NewReplicaWithClient(db1, replicaClient)\n\tdb1.Replica.MonitorEnabled = false\n\n\tif err := db1.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db1.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db1.Replica.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db1.Close(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Log(\"Step 1 complete: Initial data replicated\")\n\n\t// Step 2: Simulate hard recovery - remove database and .litestream directory, then restore\n\tif err := os.Remove(dbPath); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := os.Remove(dbPath + \"-wal\"); os.IsExist(err) {\n\t\tt.Fatal(err)\n\t}\n\tif err := os.Remove(dbPath + \"-shm\"); os.IsExist(err) {\n\t\tt.Fatal(err)\n\t}\n\tmetaPath := db1.MetaPath()\n\tif err := os.RemoveAll(metaPath); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Restore from backup\n\trestoreOpt := litestream.RestoreOptions{\n\t\tOutputPath: dbPath,\n\t}\n\tif err := db1.Replica.Restore(ctx, restoreOpt); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Log(\"Step 2 complete: Database restored from backup\")\n\n\t// Step 3: Start replication and insert new data\n\tdb2 := testingutil.NewDB(t, dbPath)\n\tdb2.MonitorInterval = 0\n\tdb2.Replica = litestream.NewReplicaWithClient(db2, replicaClient)\n\tdb2.Replica.MonitorEnabled = false\n\n\tif err := db2.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tsqldb2 := testingutil.MustOpenSQLDB(t, dbPath)\n\tif _, err := sqldb2.ExecContext(ctx, `INSERT INTO test VALUES (2);`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := sqldb2.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Log(\"Step 3: Inserted new data (value=2) after restore\")\n\n\t// Sync new data\n\tif err := db2.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db2.Replica.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db2.Close(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Log(\"Step 3 complete: New data synced\")\n\n\t// Step 4: Simulate second hard recovery and restore again\n\tif err := os.Remove(dbPath); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := os.Remove(dbPath + \"-wal\"); os.IsExist(err) {\n\t\tt.Fatal(err)\n\t}\n\tif err := os.Remove(dbPath + \"-shm\"); os.IsExist(err) {\n\t\tt.Fatal(err)\n\t}\n\tif err := os.RemoveAll(db2.MetaPath()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Restore to a path with non-existent parent directory to verify it gets created\n\trestoredPath := dbDir + \"/restored/db.sqlite\"\n\trestoreOpt.OutputPath = restoredPath\n\tif err := db2.Replica.Restore(ctx, restoreOpt); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tt.Log(\"Step 4 complete: Second restore from backup to path with non-existent parent\")\n\n\t// Step 5: Verify the new data (value=2) exists in restored database\n\tsqldb3 := testingutil.MustOpenSQLDB(t, restoredPath)\n\tdefer sqldb3.Close()\n\n\tvar count int\n\tif err := sqldb3.QueryRowContext(ctx, `SELECT COUNT(*) FROM test;`).Scan(&count); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Should have 2 rows (1 and 2)\n\tif count != 2 {\n\t\tt.Fatalf(\"expected 2 rows in restored database, got %d\", count)\n\t}\n\n\t// Verify the new row (value=2) exists\n\tvar exists bool\n\tif err := sqldb3.QueryRowContext(ctx, `SELECT EXISTS(SELECT 1 FROM test WHERE col1 = 2);`).Scan(&exists); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif !exists {\n\t\tt.Fatal(\"new data (value=2) was not replicated - this is the bug in issue #781\")\n\t}\n\n\tt.Log(\"Test passed: New data after restore was successfully replicated\")\n}\n\nfunc TestReplica_CalcRestorePlan(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tt.Run(\"SnapshotOnly\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tif level == litestream.SnapshotLevel {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{{\n\t\t\t\t\tLevel: litestream.SnapshotLevel,\n\t\t\t\t\tMinTXID: 1,\n\t\t\t\t\tMaxTXID: 10,\n\t\t\t\t\tSize: 1024,\n\t\t\t\t\tCreatedAt: time.Now(),\n\t\t\t\t}}), nil\n\t\t\t}\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\n\t\tplan, err := litestream.CalcRestorePlan(context.Background(), r.Client, 10, time.Time{}, r.Logger())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif got, want := len(plan), 1; got != want {\n\t\t\tt.Fatalf(\"n=%d, want %d\", got, want)\n\t\t}\n\t\tif plan[0].MaxTXID != 10 {\n\t\t\tt.Fatalf(\"expected MaxTXID 10, got %d\", plan[0].MaxTXID)\n\t\t}\n\t})\n\n\tt.Run(\"SnapshotAndIncremental\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 5},\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 15},\n\t\t\t\t}), nil\n\t\t\tcase 1:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 1, MinTXID: 6, MaxTXID: 7},\n\t\t\t\t\t{Level: 1, MinTXID: 8, MaxTXID: 9},\n\t\t\t\t\t{Level: 1, MinTXID: 10, MaxTXID: 12},\n\t\t\t\t}), nil\n\t\t\tcase 0:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 0, MinTXID: 7, MaxTXID: 7},\n\t\t\t\t\t{Level: 0, MinTXID: 8, MaxTXID: 8},\n\t\t\t\t\t{Level: 0, MinTXID: 9, MaxTXID: 9},\n\t\t\t\t\t{Level: 0, MinTXID: 10, MaxTXID: 10},\n\t\t\t\t\t{Level: 0, MinTXID: 11, MaxTXID: 11},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tplan, err := litestream.CalcRestorePlan(context.Background(), r.Client, 10, time.Time{}, r.Logger())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif got, want := len(plan), 4; got != want {\n\t\t\tt.Fatalf(\"n=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := *plan[0], (ltx.FileInfo{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 5}); got != want {\n\t\t\tt.Fatalf(\"plan[0]=%#v, want %#v\", got, want)\n\t\t}\n\t\tif got, want := *plan[1], (ltx.FileInfo{Level: 1, MinTXID: 6, MaxTXID: 7}); got != want {\n\t\t\tt.Fatalf(\"plan[1]=%#v, want %#v\", got, want)\n\t\t}\n\t\tif got, want := *plan[2], (ltx.FileInfo{Level: 1, MinTXID: 8, MaxTXID: 9}); got != want {\n\t\t\tt.Fatalf(\"plan[2]=%#v, want %#v\", got, want)\n\t\t}\n\t\tif got, want := *plan[3], (ltx.FileInfo{Level: 0, MinTXID: 10, MaxTXID: 10}); got != want {\n\t\t\tt.Fatalf(\"plan[2]=%#v, want %#v\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"SelectLongestAcrossLevels\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 5},\n\t\t\t\t}), nil\n\t\t\tcase 2:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 2, MinTXID: 6, MaxTXID: 12},\n\t\t\t\t}), nil\n\t\t\tcase 0:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 0, MinTXID: 6, MaxTXID: 20},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tplan, err := litestream.CalcRestorePlan(context.Background(), r.Client, 20, time.Time{}, r.Logger())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif got, want := len(plan), 2; got != want {\n\t\t\tt.Fatalf(\"n=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := *plan[0], (ltx.FileInfo{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 5}); got != want {\n\t\t\tt.Fatalf(\"plan[0]=%#v, want %#v\", got, want)\n\t\t}\n\t\tif got, want := *plan[1], (ltx.FileInfo{Level: 0, MinTXID: 6, MaxTXID: 20}); got != want {\n\t\t\tt.Fatalf(\"plan[1]=%#v, want %#v\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"GapInLevelResolvedByLowerLevel\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 5},\n\t\t\t\t}), nil\n\t\t\tcase 1:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 1, MinTXID: 6, MaxTXID: 7},\n\t\t\t\t\t{Level: 1, MinTXID: 9, MaxTXID: 10},\n\t\t\t\t}), nil\n\t\t\tcase 0:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 0, MinTXID: 8, MaxTXID: 8},\n\t\t\t\t\t{Level: 0, MinTXID: 9, MaxTXID: 9},\n\t\t\t\t\t{Level: 0, MinTXID: 10, MaxTXID: 10},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tplan, err := litestream.CalcRestorePlan(context.Background(), r.Client, 10, time.Time{}, r.Logger())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif got, want := len(plan), 4; got != want {\n\t\t\tt.Fatalf(\"n=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := *plan[0], (ltx.FileInfo{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 5}); got != want {\n\t\t\tt.Fatalf(\"plan[0]=%#v, want %#v\", got, want)\n\t\t}\n\t\tif got, want := *plan[1], (ltx.FileInfo{Level: 1, MinTXID: 6, MaxTXID: 7}); got != want {\n\t\t\tt.Fatalf(\"plan[1]=%#v, want %#v\", got, want)\n\t\t}\n\t\tif got, want := *plan[2], (ltx.FileInfo{Level: 0, MinTXID: 8, MaxTXID: 8}); got != want {\n\t\t\tt.Fatalf(\"plan[2]=%#v, want %#v\", got, want)\n\t\t}\n\t\tif got, want := *plan[3], (ltx.FileInfo{Level: 1, MinTXID: 9, MaxTXID: 10}); got != want {\n\t\t\tt.Fatalf(\"plan[3]=%#v, want %#v\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"SkipsDuplicateRangesAcrossLevels\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 1},\n\t\t\t\t}), nil\n\t\t\tcase 1:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 1, MinTXID: 1, MaxTXID: 1},\n\t\t\t\t}), nil\n\t\t\tcase 0:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 0, MinTXID: 1, MaxTXID: 1},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tplan, err := litestream.CalcRestorePlan(context.Background(), r.Client, 1, time.Time{}, r.Logger())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif got, want := len(plan), 1; got != want {\n\t\t\tt.Fatalf(\"n=%v, want %v\", got, want)\n\t\t}\n\t\tif got, want := *plan[0], (ltx.FileInfo{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 1}); got != want {\n\t\t\tt.Fatalf(\"plan[0]=%#v, want %#v\", got, want)\n\t\t}\n\t})\n\n\t// Issue #847: When a level has overlapping files where a larger compacted file\n\t// covers a smaller file's entire range, the smaller file should be skipped\n\t// rather than causing a non-contiguous error.\n\tt.Run(\"OverlappingFilesWithinLevel\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 5},\n\t\t\t\t}), nil\n\t\t\tcase 2:\n\t\t\t\t// Simulates issue #847: Files are sorted by MinTXID (filename order).\n\t\t\t\t// File 1 is a large compacted file covering 1-100.\n\t\t\t\t// File 2 is a smaller file covering 50-60, which is fully within file 1's range.\n\t\t\t\t// Before the fix, file 2 would pass the filter (MaxTXID 60 > infos.MaxTXID() 5)\n\t\t\t\t// but then fail the contiguity check after file 1 is added.\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 2, MinTXID: 1, MaxTXID: 100},\n\t\t\t\t\t{Level: 2, MinTXID: 50, MaxTXID: 60},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tplan, err := litestream.CalcRestorePlan(context.Background(), r.Client, 100, time.Time{}, r.Logger())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\t// Plan should contain only snapshot and the large file, not the smaller overlapping file\n\t\tif got, want := len(plan), 2; got != want {\n\t\t\tt.Fatalf(\"n=%d, want %d\", got, want)\n\t\t}\n\t\tif got, want := *plan[0], (ltx.FileInfo{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 5}); got != want {\n\t\t\tt.Fatalf(\"plan[0]=%#v, want %#v\", got, want)\n\t\t}\n\t\tif got, want := *plan[1], (ltx.FileInfo{Level: 2, MinTXID: 1, MaxTXID: 100}); got != want {\n\t\t\tt.Fatalf(\"plan[1]=%#v, want %#v\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"ErrTxNotAvailable\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 10},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\t_, err := litestream.CalcRestorePlan(context.Background(), r.Client, 5, time.Time{}, r.Logger())\n\t\tif !errors.Is(err, litestream.ErrTxNotAvailable) {\n\t\t\tt.Fatalf(\"expected ErrTxNotAvailable, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ErrNoFiles\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\n\t\t_, err := litestream.CalcRestorePlan(context.Background(), r.Client, 5, time.Time{}, r.Logger())\n\t\tif !errors.Is(err, litestream.ErrTxNotAvailable) {\n\t\t\tt.Fatalf(\"expected ErrTxNotAvailable, got %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestReplica_TimeBounds(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tt.Run(\"Level0Only\", func(t *testing.T) {\n\t\tnow := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tif level == 0 {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 0, MinTXID: 1, MaxTXID: 1, CreatedAt: now},\n\t\t\t\t\t{Level: 0, MinTXID: 2, MaxTXID: 2, CreatedAt: now.Add(time.Hour)},\n\t\t\t\t\t{Level: 0, MinTXID: 3, MaxTXID: 3, CreatedAt: now.Add(2 * time.Hour)},\n\t\t\t\t}), nil\n\t\t\t}\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\n\t\tcreatedAt, updatedAt, err := r.TimeBounds(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !createdAt.Equal(now) {\n\t\t\tt.Fatalf(\"createdAt=%v, want %v\", createdAt, now)\n\t\t}\n\t\tif want := now.Add(2 * time.Hour); !updatedAt.Equal(want) {\n\t\t\tt.Fatalf(\"updatedAt=%v, want %v\", updatedAt, want)\n\t\t}\n\t})\n\n\tt.Run(\"SnapshotOnly\", func(t *testing.T) {\n\t\tnow := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tif level == litestream.SnapshotLevel {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 10, CreatedAt: now},\n\t\t\t\t}), nil\n\t\t\t}\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\n\t\tcreatedAt, updatedAt, err := r.TimeBounds(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !createdAt.Equal(now) {\n\t\t\tt.Fatalf(\"createdAt=%v, want %v\", createdAt, now)\n\t\t}\n\t\tif !updatedAt.Equal(now) {\n\t\t\tt.Fatalf(\"updatedAt=%v, want %v\", updatedAt, now)\n\t\t}\n\t})\n\n\tt.Run(\"SnapshotAndLevel0\", func(t *testing.T) {\n\t\tsnapshotTime := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)\n\t\tl0Time := time.Date(2024, 1, 2, 0, 0, 0, 0, time.UTC)\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 10, CreatedAt: snapshotTime},\n\t\t\t\t}), nil\n\t\t\tcase 0:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 0, MinTXID: 11, MaxTXID: 11, CreatedAt: l0Time},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tcreatedAt, updatedAt, err := r.TimeBounds(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !createdAt.Equal(snapshotTime) {\n\t\t\tt.Fatalf(\"createdAt=%v, want %v\", createdAt, snapshotTime)\n\t\t}\n\t\tif !updatedAt.Equal(l0Time) {\n\t\t\tt.Fatalf(\"updatedAt=%v, want %v\", updatedAt, l0Time)\n\t\t}\n\t})\n\n\tt.Run(\"MultipleCompactionLevels\", func(t *testing.T) {\n\t\tsnapshotTime := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)\n\t\tl2Time := time.Date(2024, 1, 2, 0, 0, 0, 0, time.UTC)\n\t\tl0Time := time.Date(2024, 1, 3, 0, 0, 0, 0, time.UTC)\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 5, CreatedAt: snapshotTime},\n\t\t\t\t}), nil\n\t\t\tcase 2:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 2, MinTXID: 6, MaxTXID: 8, CreatedAt: l2Time},\n\t\t\t\t}), nil\n\t\t\tcase 0:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 0, MinTXID: 9, MaxTXID: 9, CreatedAt: l0Time},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tcreatedAt, updatedAt, err := r.TimeBounds(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !createdAt.Equal(snapshotTime) {\n\t\t\tt.Fatalf(\"createdAt=%v, want %v\", createdAt, snapshotTime)\n\t\t}\n\t\tif !updatedAt.Equal(l0Time) {\n\t\t\tt.Fatalf(\"updatedAt=%v, want %v\", updatedAt, l0Time)\n\t\t}\n\t})\n\n\tt.Run(\"NoFiles\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\n\t\tcreatedAt, updatedAt, err := r.TimeBounds(context.Background())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !createdAt.IsZero() {\n\t\t\tt.Fatalf(\"createdAt=%v, want zero\", createdAt)\n\t\t}\n\t\tif !updatedAt.IsZero() {\n\t\t\tt.Fatalf(\"updatedAt=%v, want zero\", updatedAt)\n\t\t}\n\t})\n\n\tt.Run(\"ErrorOnLevel\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\terrTest := errors.New(\"test error\")\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tif level == 3 {\n\t\t\t\treturn nil, errTest\n\t\t\t}\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\n\t\t_, _, err := r.TimeBounds(context.Background())\n\t\tif !errors.Is(err, errTest) {\n\t\t\tt.Fatalf(\"expected test error, got %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestReplica_CalcRestoreTarget(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tt.Run(\"TimestampInSnapshotRange\", func(t *testing.T) {\n\t\tsnapshotTime := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)\n\t\tl0Time := time.Date(2024, 1, 10, 0, 0, 0, 0, time.UTC)\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 10, CreatedAt: snapshotTime},\n\t\t\t\t}), nil\n\t\t\tcase 0:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 0, MinTXID: 11, MaxTXID: 11, CreatedAt: l0Time},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tts := time.Date(2024, 1, 5, 0, 0, 0, 0, time.UTC)\n\t\tupdatedAt, err := r.CalcRestoreTarget(context.Background(), litestream.RestoreOptions{Timestamp: ts})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif !updatedAt.Equal(l0Time) {\n\t\t\tt.Fatalf(\"updatedAt=%v, want %v\", updatedAt, l0Time)\n\t\t}\n\t})\n\n\tt.Run(\"TimestampBeforeAllFiles\", func(t *testing.T) {\n\t\tsnapshotTime := time.Date(2024, 1, 5, 0, 0, 0, 0, time.UTC)\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 10, CreatedAt: snapshotTime},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tts := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)\n\t\t_, err := r.CalcRestoreTarget(context.Background(), litestream.RestoreOptions{Timestamp: ts})\n\t\tif err == nil || err.Error() != \"timestamp does not exist\" {\n\t\t\tt.Fatalf(\"expected 'timestamp does not exist', got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"TimestampAfterAllFiles\", func(t *testing.T) {\n\t\tsnapshotTime := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tswitch level {\n\t\t\tcase litestream.SnapshotLevel:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: litestream.SnapshotLevel, MinTXID: 1, MaxTXID: 10, CreatedAt: snapshotTime},\n\t\t\t\t}), nil\n\t\t\tdefault:\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t}\n\n\t\tts := time.Date(2024, 12, 31, 0, 0, 0, 0, time.UTC)\n\t\t_, err := r.CalcRestoreTarget(context.Background(), litestream.RestoreOptions{Timestamp: ts})\n\t\tif err == nil || err.Error() != \"timestamp does not exist\" {\n\t\t\tt.Fatalf(\"expected 'timestamp does not exist', got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NoTimestamp\", func(t *testing.T) {\n\t\tnow := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tif level == 0 {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{\n\t\t\t\t\t{Level: 0, MinTXID: 1, MaxTXID: 1, CreatedAt: now},\n\t\t\t\t\t{Level: 0, MinTXID: 2, MaxTXID: 2, CreatedAt: now.Add(time.Hour)},\n\t\t\t\t}), nil\n\t\t\t}\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\n\t\tupdatedAt, err := r.CalcRestoreTarget(context.Background(), litestream.RestoreOptions{})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t}\n\t\tif want := now.Add(time.Hour); !updatedAt.Equal(want) {\n\t\t\tt.Fatalf(\"updatedAt=%v, want %v\", updatedAt, want)\n\t\t}\n\t})\n}\n\nfunc TestReplica_Restore_InvalidFileSize(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tt.Run(\"EmptyFile\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tif level == litestream.SnapshotLevel {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{{\n\t\t\t\t\tLevel: litestream.SnapshotLevel,\n\t\t\t\t\tMinTXID: 1,\n\t\t\t\t\tMaxTXID: 10,\n\t\t\t\t\tSize: 0, // Empty file - this should cause an error\n\t\t\t\t\tCreatedAt: time.Now(),\n\t\t\t\t}}), nil\n\t\t\t}\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\toutputPath := t.TempDir() + \"/restored.db\"\n\n\t\terr := r.Restore(context.Background(), litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t})\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for empty file, got nil\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"invalid ltx file\") {\n\t\t\tt.Fatalf(\"expected 'invalid ltx file' error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"TruncatedFile\", func(t *testing.T) {\n\t\tvar c mock.ReplicaClient\n\t\tc.LTXFilesFunc = func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tif level == litestream.SnapshotLevel {\n\t\t\t\treturn ltx.NewFileInfoSliceIterator([]*ltx.FileInfo{{\n\t\t\t\t\tLevel: litestream.SnapshotLevel,\n\t\t\t\t\tMinTXID: 1,\n\t\t\t\t\tMaxTXID: 10,\n\t\t\t\t\tSize: 50, // Less than ltx.HeaderSize (100) - should cause an error\n\t\t\t\t\tCreatedAt: time.Now(),\n\t\t\t\t}}), nil\n\t\t\t}\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\n\t\tr := litestream.NewReplicaWithClient(db, &c)\n\t\toutputPath := t.TempDir() + \"/restored.db\"\n\n\t\terr := r.Restore(context.Background(), litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t})\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for truncated file, got nil\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"invalid ltx file\") {\n\t\t\tt.Fatalf(\"expected 'invalid ltx file' error, got: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestReplica_ContextCancellationNoLogs(t *testing.T) {\n\t// This test verifies that context cancellation errors are not logged during shutdown.\n\t// The fix for issue #235 ensures that context.Canceled and context.DeadlineExceeded\n\t// errors are filtered out in monitor functions to avoid spurious log messages.\n\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Create a buffer to capture log output\n\tvar logBuffer bytes.Buffer\n\n\t// Create a custom logger that writes to our buffer\n\tdb.Logger = slog.New(slog.NewTextHandler(&logBuffer, &slog.HandlerOptions{\n\t\tLevel: slog.LevelDebug,\n\t}))\n\n\t// First, let's trigger a normal sync to ensure the DB is initialized\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Create a replica with a mock client that simulates context cancellation during Sync\n\tsyncCount := 0\n\tmockClient := &mock.ReplicaClient{\n\t\tLTXFilesFunc: func(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\t\t\tsyncCount++\n\t\t\t// First few calls succeed, then return context.Canceled\n\t\t\tif syncCount <= 2 {\n\t\t\t\t// Return an empty iterator\n\t\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t\t}\n\t\t\t// After initial syncs, return context.Canceled to simulate shutdown\n\t\t\treturn nil, context.Canceled\n\t\t},\n\t\tWriteLTXFileFunc: func(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\t\t\t// Always succeed for writes to allow normal operation\n\t\t\treturn <x.FileInfo{\n\t\t\t\tLevel: level,\n\t\t\t\tMinTXID: minTXID,\n\t\t\t\tMaxTXID: maxTXID,\n\t\t\t\tCreatedAt: time.Now(),\n\t\t\t}, nil\n\t\t},\n\t}\n\n\tr := litestream.NewReplicaWithClient(db, mockClient)\n\tr.SyncInterval = 50 * time.Millisecond // Short interval for testing\n\n\t// Start the replica monitoring in a goroutine\n\tctx, cancel := context.WithCancel(context.Background())\n\n\tif err := r.Start(ctx); err != nil {\n\t\tt.Fatalf(\"failed to start replica: %v\", err)\n\t}\n\n\t// Give the monitor time to run several sync cycles\n\t// This ensures we get both successful syncs and context cancellation errors\n\ttime.Sleep(200 * time.Millisecond)\n\n\t// Cancel the context to trigger shutdown\n\tcancel()\n\n\t// Stop the replica and wait for it to finish\n\tif err := r.Stop(true); err != nil {\n\t\tt.Fatalf(\"failed to stop replica: %v\", err)\n\t}\n\n\t// Check the logs\n\tlogs := logBuffer.String()\n\n\t// We should have some debug logs from successful operations\n\tif !strings.Contains(logs, \"replica sync\") {\n\t\tt.Errorf(\"expected 'replica sync' in logs but didn't find it; logs:\\n%s\", logs)\n\t}\n\n\t// But we should NOT have \"monitor error\" with \"context canceled\"\n\tif strings.Contains(logs, \"monitor error\") && strings.Contains(logs, \"context canceled\") {\n\t\tt.Errorf(\"found 'monitor error' with 'context canceled' in logs when it should be filtered:\\n%s\", logs)\n\t}\n\n\t// The test passes if context.Canceled errors were properly filtered\n}\n\nfunc TestReplica_ValidateLevel(t *testing.T) {\n\tt.Run(\"ValidContiguousFiles\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\treplica := litestream.NewReplicaWithClient(nil, client)\n\n\t\t// Create contiguous files: 1-2, 3-5, 6-10\n\t\tcreateTestLTXFile(t, client, 1, 1, 2)\n\t\tcreateTestLTXFile(t, client, 1, 3, 5)\n\t\tcreateTestLTXFile(t, client, 1, 6, 10)\n\n\t\terrs, err := replica.ValidateLevel(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(errs) != 0 {\n\t\t\tt.Errorf(\"expected no errors for contiguous files, got %d: %v\", len(errs), errs)\n\t\t}\n\t})\n\n\tt.Run(\"EmptyLevel\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\treplica := litestream.NewReplicaWithClient(nil, client)\n\n\t\terrs, err := replica.ValidateLevel(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(errs) != 0 {\n\t\t\tt.Errorf(\"expected no errors for empty level, got %d\", len(errs))\n\t\t}\n\t})\n\n\tt.Run(\"SingleFile\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\treplica := litestream.NewReplicaWithClient(nil, client)\n\n\t\tcreateTestLTXFile(t, client, 1, 1, 5)\n\n\t\terrs, err := replica.ValidateLevel(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(errs) != 0 {\n\t\t\tt.Errorf(\"expected no errors for single file, got %d\", len(errs))\n\t\t}\n\t})\n\n\tt.Run(\"GapDetected\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\treplica := litestream.NewReplicaWithClient(nil, client)\n\n\t\t// Create files with a gap (missing TXID 3-4)\n\t\tcreateTestLTXFile(t, client, 1, 1, 2)\n\t\tcreateTestLTXFile(t, client, 1, 5, 7)\n\n\t\terrs, err := replica.ValidateLevel(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(errs) != 1 {\n\t\t\tt.Fatalf(\"expected 1 error, got %d\", len(errs))\n\t\t}\n\t\tif errs[0].Type != \"gap\" {\n\t\t\tt.Errorf(\"expected gap error, got %q\", errs[0].Type)\n\t\t}\n\t\tif errs[0].Level != 1 {\n\t\t\tt.Errorf(\"expected level 1, got %d\", errs[0].Level)\n\t\t}\n\t})\n\n\tt.Run(\"OverlapDetected\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\treplica := litestream.NewReplicaWithClient(nil, client)\n\n\t\t// Create overlapping files\n\t\tcreateTestLTXFile(t, client, 1, 1, 5)\n\t\tcreateTestLTXFile(t, client, 1, 3, 7)\n\n\t\terrs, err := replica.ValidateLevel(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(errs) != 1 {\n\t\t\tt.Fatalf(\"expected 1 error, got %d\", len(errs))\n\t\t}\n\t\tif errs[0].Type != \"overlap\" {\n\t\t\tt.Errorf(\"expected overlap error, got %q\", errs[0].Type)\n\t\t}\n\t})\n\n\tt.Run(\"MultipleErrors\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\treplica := litestream.NewReplicaWithClient(nil, client)\n\n\t\t// Create files with multiple issues: gap then overlap\n\t\tcreateTestLTXFile(t, client, 1, 1, 2)\n\t\tcreateTestLTXFile(t, client, 1, 5, 10) // gap at 3-4\n\t\tcreateTestLTXFile(t, client, 1, 8, 12) // overlap at 8-10\n\n\t\terrs, err := replica.ValidateLevel(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif len(errs) != 2 {\n\t\t\tt.Fatalf(\"expected 2 errors, got %d\", len(errs))\n\t\t}\n\t\tif errs[0].Type != \"gap\" {\n\t\t\tt.Errorf(\"expected first error to be gap, got %q\", errs[0].Type)\n\t\t}\n\t\tif errs[1].Type != \"overlap\" {\n\t\t\tt.Errorf(\"expected second error to be overlap, got %q\", errs[1].Type)\n\t\t}\n\t})\n}\nfunc TestReplica_RestoreV3(t *testing.T) {\n\tt.Run(\"SnapshotOnly\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\ttmpDir := t.TempDir()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create a v0.3.x backup structure with a snapshot\n\t\tgen := \"0123456789abcdef\"\n\t\tcreateV3Backup(t, replicaDir, gen, []v3SnapshotData{\n\t\t\t{index: 0, data: createTestSQLiteDB(t)},\n\t\t}, nil)\n\n\t\t// Create replica client and replica\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\t// Restore\n\t\toutputPath := tmpDir + \"/restored.db\"\n\t\terr := r.RestoreV3(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"RestoreV3 failed: %v\", err)\n\t\t}\n\n\t\t// Verify restored database\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n\n\tt.Run(\"SnapshotWithWAL\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\ttmpDir := t.TempDir()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create a v0.3.x backup with snapshot and WAL segments\n\t\tgen := \"0123456789abcdef\"\n\t\tdbData := createTestSQLiteDB(t)\n\t\twalData := createTestWALData(t, dbData)\n\n\t\tcreateV3Backup(t, replicaDir, gen, []v3SnapshotData{\n\t\t\t{index: 0, data: dbData},\n\t\t}, []v3WALSegmentData{\n\t\t\t{index: 0, offset: 0, data: walData},\n\t\t})\n\n\t\t// Create replica and restore\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\toutputPath := tmpDir + \"/restored.db\"\n\t\terr := r.RestoreV3(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"RestoreV3 failed: %v\", err)\n\t\t}\n\n\t\t// Verify restored database\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n\n\tt.Run(\"TimestampRestore\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\ttmpDir := t.TempDir()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create a v0.3.x backup with multiple snapshots at different times\n\t\tgen := \"0123456789abcdef\"\n\t\tsnapshotsDir := filepath.Join(replicaDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create first snapshot (older)\n\t\tdbData1 := createTestSQLiteDB(t)\n\t\twriteV3Snapshot(t, snapshotsDir, 0, dbData1)\n\t\t// Set older mod time\n\t\toldTime := time.Now().Add(-2 * time.Hour)\n\t\tif err := os.Chtimes(filepath.Join(snapshotsDir, \"00000000.snapshot.lz4\"), oldTime, oldTime); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create second snapshot (newer) - sleep briefly to ensure different times\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\tdbData2 := createTestSQLiteDB(t)\n\t\twriteV3Snapshot(t, snapshotsDir, 1, dbData2)\n\t\t// Set newer mod time\n\t\tnewTime := time.Now().Add(-1 * time.Hour)\n\t\tif err := os.Chtimes(filepath.Join(snapshotsDir, \"00000001.snapshot.lz4\"), newTime, newTime); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create replica and restore to a timestamp between the two snapshots\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\toutputPath := tmpDir + \"/restored.db\"\n\t\trestoreTime := time.Now().Add(-90 * time.Minute) // Between old and new\n\t\terr := r.RestoreV3(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tTimestamp: restoreTime,\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"RestoreV3 failed: %v\", err)\n\t\t}\n\n\t\t// Verify restored database (should be the older one)\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n\n\tt.Run(\"NoSnapshots\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\ttmpDir := t.TempDir()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create empty generations directory\n\t\tgen := \"0123456789abcdef\"\n\t\tif err := os.MkdirAll(filepath.Join(replicaDir, \"generations\", gen), 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\toutputPath := tmpDir + \"/restored.db\"\n\t\terr := r.RestoreV3(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t})\n\t\tif !errors.Is(err, litestream.ErrNoSnapshots) {\n\t\t\tt.Fatalf(\"expected ErrNoSnapshots, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"NoGenerations\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\ttmpDir := t.TempDir()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Empty replica directory\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\toutputPath := tmpDir + \"/restored.db\"\n\t\terr := r.RestoreV3(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t})\n\t\tif !errors.Is(err, litestream.ErrNoSnapshots) {\n\t\t\tt.Fatalf(\"expected ErrNoSnapshots, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"OutputPathExists\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create a v0.3.x backup\n\t\tgen := \"0123456789abcdef\"\n\t\tcreateV3Backup(t, replicaDir, gen, []v3SnapshotData{\n\t\t\t{index: 0, data: createTestSQLiteDB(t)},\n\t\t}, nil)\n\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\t// Create output file that already exists\n\t\toutputPath := t.TempDir() + \"/existing.db\"\n\t\tif err := os.WriteFile(outputPath, []byte(\"existing\"), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\terr := r.RestoreV3(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t})\n\t\tif err == nil || !strings.Contains(err.Error(), \"already exists\") {\n\t\t\tt.Fatalf(\"expected 'already exists' error, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ClientDoesNotSupportV3\", func(t *testing.T) {\n\t\tctx := context.Background()\n\n\t\t// Use mock client that doesn't implement ReplicaClientV3\n\t\tvar c mock.ReplicaClient\n\t\tr := litestream.NewReplicaWithClient(nil, &c)\n\n\t\terr := r.RestoreV3(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: t.TempDir() + \"/restored.db\",\n\t\t})\n\t\tif err == nil || !strings.Contains(err.Error(), \"does not support v0.3.x\") {\n\t\t\tt.Fatalf(\"expected 'does not support v0.3.x' error, got %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"MultipleGenerations\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\ttmpDir := t.TempDir()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create snapshots in two different generations\n\t\tgen1 := \"0000000000000001\"\n\t\tgen2 := \"0000000000000002\"\n\n\t\t// Older snapshot in gen1\n\t\tsnapshotsDir1 := filepath.Join(replicaDir, \"generations\", gen1, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir1, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdbData1 := createTestSQLiteDB(t)\n\t\twriteV3Snapshot(t, snapshotsDir1, 0, dbData1)\n\t\toldTime := time.Now().Add(-2 * time.Hour)\n\t\tif err := os.Chtimes(filepath.Join(snapshotsDir1, \"00000000.snapshot.lz4\"), oldTime, oldTime); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Newer snapshot in gen2\n\t\tsnapshotsDir2 := filepath.Join(replicaDir, \"generations\", gen2, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir2, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdbData2 := createTestSQLiteDB(t)\n\t\twriteV3Snapshot(t, snapshotsDir2, 0, dbData2)\n\t\tnewTime := time.Now().Add(-1 * time.Hour)\n\t\tif err := os.Chtimes(filepath.Join(snapshotsDir2, \"00000000.snapshot.lz4\"), newTime, newTime); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Restore without timestamp should pick the newest\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\toutputPath := tmpDir + \"/restored.db\"\n\t\terr := r.RestoreV3(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"RestoreV3 failed: %v\", err)\n\t\t}\n\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n}\n\nfunc TestReplica_Restore_BothFormats(t *testing.T) {\n\tt.Run(\"V3OnlyWithTimestamp\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\ttmpDir := t.TempDir()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create a v0.3.x backup only\n\t\tgen := \"0123456789abcdef\"\n\t\tsnapshotsDir := filepath.Join(replicaDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdbData := createTestSQLiteDB(t)\n\t\twriteV3Snapshot(t, snapshotsDir, 0, dbData)\n\t\t// Set snapshot time to 1 hour ago\n\t\tsnapshotTime := time.Now().Add(-1 * time.Hour)\n\t\tif err := os.Chtimes(filepath.Join(snapshotsDir, \"00000000.snapshot.lz4\"), snapshotTime, snapshotTime); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create replica and restore with timestamp\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\toutputPath := tmpDir + \"/restored.db\"\n\t\terr := r.Restore(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tTimestamp: time.Now(), // Any time after snapshot\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t\t}\n\n\t\t// Verify restored database\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n\n\tt.Run(\"V3OnlyWithoutTimestamp\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\ttmpDir := t.TempDir()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create a v0.3.x backup only (no LTX files)\n\t\tgen := \"0123456789abcdef\"\n\t\tsnapshotsDir := filepath.Join(replicaDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdbData := createTestSQLiteDB(t)\n\t\twriteV3Snapshot(t, snapshotsDir, 0, dbData)\n\n\t\t// Create replica and restore WITHOUT timestamp - should still use v0.3.x\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\toutputPath := tmpDir + \"/restored.db\"\n\t\terr := r.Restore(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\t// No timestamp specified\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t\t}\n\n\t\t// Verify restored database\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n\n\tt.Run(\"LTXOnlyWithTimestamp\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\treplicaDir := t.TempDir()\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(db, c)\n\n\t\t// Sync to create LTX files\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := r.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create a snapshot\n\t\tif _, err := db.Snapshot(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := r.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Wait a bit to ensure distinct timestamps\n\t\ttime.Sleep(10 * time.Millisecond)\n\n\t\t// Restore with timestamp\n\t\toutputPath := t.TempDir() + \"/restored.db\"\n\t\terr := r.Restore(context.Background(), litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tTimestamp: time.Now(),\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t\t}\n\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n\n\tt.Run(\"BothFormats_V3Better\", func(t *testing.T) {\n\t\tctx := context.Background()\n\t\ttmpDir := t.TempDir()\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create v0.3.x snapshot at time T-30min (closer to restore time)\n\t\tgen := \"0123456789abcdef\"\n\t\tsnapshotsDir := filepath.Join(replicaDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdbData := createTestSQLiteDB(t)\n\t\twriteV3Snapshot(t, snapshotsDir, 0, dbData)\n\t\tv3Time := time.Now().Add(-30 * time.Minute)\n\t\tif err := os.Chtimes(filepath.Join(snapshotsDir, \"00000000.snapshot.lz4\"), v3Time, v3Time); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create LTX snapshot at time T-2h (older)\n\t\tltxDir := filepath.Join(replicaDir, \"ltx\", \"9\") // Snapshot level\n\t\tif err := os.MkdirAll(ltxDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tltxData := createTestLTXSnapshot(t)\n\t\tltxPath := filepath.Join(ltxDir, \"0000000000000001-0000000000000001.ltx\")\n\t\tif err := os.WriteFile(ltxPath, ltxData, 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tltxTime := time.Now().Add(-2 * time.Hour)\n\t\tif err := os.Chtimes(ltxPath, ltxTime, ltxTime); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Restore with timestamp - should use V3 (more recent)\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(nil, c)\n\n\t\toutputPath := tmpDir + \"/restored.db\"\n\t\terr := r.Restore(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tTimestamp: time.Now(),\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t\t}\n\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n\n\tt.Run(\"BothFormats_LTXBetter\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create v0.3.x snapshot at time T-2h (older)\n\t\tgen := \"0123456789abcdef\"\n\t\tsnapshotsDir := filepath.Join(replicaDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tv3Data := createTestSQLiteDB(t)\n\t\twriteV3Snapshot(t, snapshotsDir, 0, v3Data)\n\t\tv3Time := time.Now().Add(-2 * time.Hour)\n\t\tif err := os.Chtimes(filepath.Join(snapshotsDir, \"00000000.snapshot.lz4\"), v3Time, v3Time); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create LTX backup (more recent)\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(db, c)\n\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := r.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Create a snapshot\n\t\tif _, err := db.Snapshot(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := r.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Wait a bit\n\t\ttime.Sleep(10 * time.Millisecond)\n\n\t\t// Restore with timestamp - should use LTX (more recent)\n\t\toutputPath := t.TempDir() + \"/restored.db\"\n\t\terr := r.Restore(context.Background(), litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tTimestamp: time.Now(),\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t\t}\n\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n\n\tt.Run(\"NoTimestamp_UsesLTX\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create v0.3.x snapshot\n\t\tgen := \"0123456789abcdef\"\n\t\tsnapshotsDir := filepath.Join(replicaDir, \"generations\", gen, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tv3Data := createTestSQLiteDB(t)\n\t\twriteV3Snapshot(t, snapshotsDir, 0, v3Data)\n\n\t\t// Create LTX backup\n\t\tc := file.NewReplicaClient(replicaDir)\n\t\tr := litestream.NewReplicaWithClient(db, c)\n\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := r.Sync(context.Background()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Restore without timestamp - should use LTX (default behavior)\n\t\toutputPath := t.TempDir() + \"/restored.db\"\n\t\terr := r.Restore(context.Background(), litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t})\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t\t}\n\n\t\tverifyRestoredDB(t, outputPath)\n\t})\n}\n\n// createTestLTXSnapshot creates a minimal LTX snapshot for testing.\nfunc createTestLTXSnapshot(t *testing.T) []byte {\n\tt.Helper()\n\n\t// Create a temporary database and generate a real LTX snapshot\n\ttmpDir := t.TempDir()\n\tdbPath := filepath.Join(tmpDir, \"test.db\")\n\treplicaDir := filepath.Join(tmpDir, \"replica\")\n\n\tdb := testingutil.NewDB(t, dbPath)\n\n\t// Set up a replica client so we can create snapshots\n\tc := file.NewReplicaClient(replicaDir)\n\tdb.Replica = litestream.NewReplicaWithClient(db, c)\n\tdb.Replica.MonitorEnabled = false\n\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Create some data\n\tsqldb := testingutil.MustOpenSQLDB(t, dbPath)\n\tif _, err := sqldb.Exec(`CREATE TABLE test (id INTEGER PRIMARY KEY)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\ttestingutil.MustCloseSQLDB(t, sqldb)\n\n\t// Sync to create LTX file\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Replica.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Create snapshot\n\tif _, err := db.Snapshot(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := db.Close(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Read the snapshot file from replica directory\n\tltxPath := filepath.Join(replicaDir, \"ltx\", fmt.Sprintf(\"%d\", litestream.SnapshotLevel), \"0000000000000001-0000000000000001.ltx\")\n\tdata, err := os.ReadFile(ltxPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\treturn data\n}\n\n// v3SnapshotData holds test data for creating v0.3.x snapshots.\ntype v3SnapshotData struct {\n\tindex int\n\tdata []byte\n}\n\n// v3WALSegmentData holds test data for creating v0.3.x WAL segments.\ntype v3WALSegmentData struct {\n\tindex int\n\toffset int64\n\tdata []byte\n}\n\n// createV3Backup creates a v0.3.x backup structure for testing.\nfunc createV3Backup(t *testing.T, replicaDir, generation string, snapshots []v3SnapshotData, walSegments []v3WALSegmentData) {\n\tt.Helper()\n\n\t// Create snapshots directory and files\n\tif len(snapshots) > 0 {\n\t\tsnapshotsDir := filepath.Join(replicaDir, \"generations\", generation, \"snapshots\")\n\t\tif err := os.MkdirAll(snapshotsDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tfor _, s := range snapshots {\n\t\t\twriteV3Snapshot(t, snapshotsDir, s.index, s.data)\n\t\t}\n\t}\n\n\t// Create WAL directory and files\n\tif len(walSegments) > 0 {\n\t\twalDir := filepath.Join(replicaDir, \"generations\", generation, \"wal\")\n\t\tif err := os.MkdirAll(walDir, 0755); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tfor _, w := range walSegments {\n\t\t\twriteV3WALSegment(t, walDir, w.index, w.offset, w.data)\n\t\t}\n\t}\n}\n\n// writeV3Snapshot writes an LZ4-compressed snapshot file.\nfunc writeV3Snapshot(t *testing.T, dir string, index int, data []byte) {\n\tt.Helper()\n\n\tvar buf bytes.Buffer\n\tw := lz4.NewWriter(&buf)\n\tif _, err := w.Write(data); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := w.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfilename := fmt.Sprintf(\"%08x.snapshot.lz4\", index)\n\tif err := os.WriteFile(filepath.Join(dir, filename), buf.Bytes(), 0644); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\n// writeV3WALSegment writes an LZ4-compressed WAL segment file.\nfunc writeV3WALSegment(t *testing.T, dir string, index int, offset int64, data []byte) {\n\tt.Helper()\n\n\tvar buf bytes.Buffer\n\tw := lz4.NewWriter(&buf)\n\tif _, err := w.Write(data); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := w.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tfilename := fmt.Sprintf(\"%08x-%016x.wal.lz4\", index, offset)\n\tif err := os.WriteFile(filepath.Join(dir, filename), buf.Bytes(), 0644); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\n// createTestSQLiteDB creates a minimal valid SQLite database for testing.\nfunc createTestSQLiteDB(t *testing.T) []byte {\n\tt.Helper()\n\n\ttmpPath := t.TempDir() + \"/test.db\"\n\tsqldb := testingutil.MustOpenSQLDB(t, tmpPath)\n\tif _, err := sqldb.Exec(`CREATE TABLE test (id INTEGER PRIMARY KEY, value TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.Exec(`INSERT INTO test (value) VALUES ('hello')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\ttestingutil.MustCloseSQLDB(t, sqldb)\n\n\tdata, err := os.ReadFile(tmpPath)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\treturn data\n}\n\n// createTestWALData creates minimal valid WAL data for testing.\n// For simplicity, returns an empty WAL header (32 bytes) which is valid.\nfunc createTestWALData(t *testing.T, dbData []byte) []byte {\n\tt.Helper()\n\n\t// Create a minimal WAL header\n\t// WAL header is 32 bytes:\n\t// - magic number (4 bytes): 0x377f0683 (big-endian) or 0x377f0682 (little-endian)\n\t// - file format version (4 bytes): 3007000\n\t// - page size (4 bytes)\n\t// - checkpoint sequence (4 bytes)\n\t// - salt-1 (4 bytes)\n\t// - salt-2 (4 bytes)\n\t// - checksum-1 (4 bytes)\n\t// - checksum-2 (4 bytes)\n\n\t// For testing, we'll create an empty WAL that doesn't need frames applied\n\t// This is sufficient for testing the restore mechanism\n\treturn make([]byte, 32) // Empty WAL header placeholder\n}\n\nfunc TestWriteTXIDFile(t *testing.T) {\n\tt.Run(\"WritesCorrectFormat\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"test.db\")\n\t\tif err := os.WriteFile(dbPath, []byte(\"db\"), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif err := litestream.WriteTXIDFile(dbPath, 42); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tdata, err := os.ReadFile(dbPath + \"-txid\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif got, want := strings.TrimSpace(string(data)), \"000000000000002a\"; got != want {\n\t\t\tt.Fatalf(\"got %q, want %q\", got, want)\n\t\t}\n\t})\n\n\tt.Run(\"AtomicOverwrite\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"test.db\")\n\n\t\tif err := litestream.WriteTXIDFile(dbPath, 10); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := litestream.WriteTXIDFile(dbPath, 20); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\ttxid, err := litestream.ReadTXIDFile(dbPath)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif txid != 20 {\n\t\t\tt.Fatalf(\"got %d, want 20\", txid)\n\t\t}\n\t})\n}\n\nfunc TestReadTXIDFile(t *testing.T) {\n\tt.Run(\"MissingFile\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"nonexistent.db\")\n\n\t\ttxid, err := litestream.ReadTXIDFile(dbPath)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif txid != 0 {\n\t\t\tt.Fatalf(\"got %d, want 0\", txid)\n\t\t}\n\t})\n\n\tt.Run(\"ValidFile\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"test.db\")\n\n\t\tif err := os.WriteFile(dbPath+\"-txid\", []byte(\"00000000000000ff\\n\"), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\ttxid, err := litestream.ReadTXIDFile(dbPath)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif txid != 255 {\n\t\t\tt.Fatalf(\"got %d, want 255\", txid)\n\t\t}\n\t})\n\n\tt.Run(\"MalformedFile\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"test.db\")\n\n\t\tif err := os.WriteFile(dbPath+\"-txid\", []byte(\"not-a-hex-value\\n\"), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t_, err := litestream.ReadTXIDFile(dbPath)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for malformed file\")\n\t\t}\n\t})\n\n\tt.Run(\"EmptyFile\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tdbPath := filepath.Join(dir, \"test.db\")\n\n\t\tif err := os.WriteFile(dbPath+\"-txid\", []byte(\"\"), 0644); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t_, err := litestream.ReadTXIDFile(dbPath)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for empty file\")\n\t\t}\n\t})\n}\n\nfunc TestReplica_Restore_Follow_IncompatibleFlags(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tc := file.NewReplicaClient(t.TempDir())\n\tr := litestream.NewReplicaWithClient(db, c)\n\n\tt.Run(\"FollowWithTXID\", func(t *testing.T) {\n\t\terr := r.Restore(context.Background(), litestream.RestoreOptions{\n\t\t\tOutputPath: t.TempDir() + \"/db\",\n\t\t\tFollow: true,\n\t\t\tTXID: 1,\n\t\t})\n\t\tif err == nil || err.Error() != \"cannot use follow mode with -txid\" {\n\t\t\tt.Fatalf(\"expected 'cannot use follow mode with -txid' error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"FollowWithTimestamp\", func(t *testing.T) {\n\t\terr := r.Restore(context.Background(), litestream.RestoreOptions{\n\t\t\tOutputPath: t.TempDir() + \"/db\",\n\t\t\tFollow: true,\n\t\t\tTimestamp: time.Now(),\n\t\t})\n\t\tif err == nil || err.Error() != \"cannot use follow mode with -timestamp\" {\n\t\t\tt.Fatalf(\"expected 'cannot use follow mode with -timestamp' error, got: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestReplica_Restore_Follow(t *testing.T) {\n\tctx := context.Background()\n\n\t// Create source database with initial data.\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE test(id INTEGER PRIMARY KEY, value TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO test VALUES (1, 'initial')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sync and replicate to file replica.\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\treplicaDir := t.TempDir()\n\tc := file.NewReplicaClient(replicaDir)\n\tr := litestream.NewReplicaWithClient(db, c)\n\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Create a snapshot so restore has something to work with.\n\tif _, err := db.Snapshot(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Start follow mode in a goroutine.\n\toutputPath := t.TempDir() + \"/follower.db\"\n\tfollowCtx, followCancel := context.WithCancel(ctx)\n\tdefer followCancel()\n\n\terrCh := make(chan error, 1)\n\tgo func() {\n\t\terrCh <- r.Restore(followCtx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tFollow: true,\n\t\t\tFollowInterval: 50 * time.Millisecond,\n\t\t})\n\t}()\n\n\t// Wait for initial restore to complete (file should appear).\n\tdeadline := time.Now().Add(5 * time.Second)\n\tfor time.Now().Before(deadline) {\n\t\tif _, err := os.Stat(outputPath); err == nil {\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\tif _, err := os.Stat(outputPath); err != nil {\n\t\tt.Fatalf(\"restored file not found after waiting: %v\", err)\n\t}\n\n\t// Insert more data into source and replicate.\n\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO test VALUES (2, 'follow-update')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for follow mode to apply the new data.\n\tdeadline = time.Now().Add(5 * time.Second)\n\tvar found bool\n\tfor time.Now().Before(deadline) {\n\t\t// Open the follower database read-only and check for new data.\n\t\tfollowerDB := testingutil.MustOpenSQLDB(t, outputPath)\n\t\tvar count int\n\t\tif err := followerDB.QueryRowContext(ctx, `SELECT COUNT(*) FROM test WHERE value = 'follow-update'`).Scan(&count); err == nil && count > 0 {\n\t\t\tfound = true\n\t\t\tfollowerDB.Close()\n\t\t\tbreak\n\t\t}\n\t\tfollowerDB.Close()\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\tif !found {\n\t\tt.Fatal(\"follow mode did not apply new data within timeout\")\n\t}\n\n\t// Cancel follow and verify clean shutdown.\n\tfollowCancel()\n\tselect {\n\tcase err := <-errCh:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"follow returned error: %v\", err)\n\t\t}\n\tcase <-time.After(5 * time.Second):\n\t\tt.Fatal(\"follow did not shut down within timeout\")\n\t}\n}\n\nfunc TestReplica_Restore_Follow_ContextCancellation(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Create initial data and replicate.\n\tif _, err := sqldb.ExecContext(context.Background(), `CREATE TABLE test(id INTEGER PRIMARY KEY)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\treplicaDir := t.TempDir()\n\tc := file.NewReplicaClient(replicaDir)\n\tr := litestream.NewReplicaWithClient(db, c)\n\tif err := r.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := db.Snapshot(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := r.Sync(context.Background()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\toutputPath := t.TempDir() + \"/follower.db\"\n\tfollowCtx, followCancel := context.WithCancel(context.Background())\n\n\terrCh := make(chan error, 1)\n\tgo func() {\n\t\terrCh <- r.Restore(followCtx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tFollow: true,\n\t\t\tFollowInterval: 50 * time.Millisecond,\n\t\t})\n\t}()\n\n\t// Wait for restore to complete.\n\tdeadline := time.Now().Add(5 * time.Second)\n\tfor time.Now().Before(deadline) {\n\t\tif _, err := os.Stat(outputPath); err == nil {\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\n\t// Cancel immediately and verify clean return (nil error).\n\tfollowCancel()\n\tselect {\n\tcase err := <-errCh:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"expected nil error on context cancellation, got: %v\", err)\n\t\t}\n\tcase <-time.After(5 * time.Second):\n\t\tt.Fatal(\"follow did not shut down within timeout\")\n\t}\n}\n\nfunc TestReplica_Restore_Follow_WriteTXIDFile(t *testing.T) {\n\tctx := context.Background()\n\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE test(id INTEGER PRIMARY KEY, value TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO test VALUES (1, 'initial')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\treplicaDir := t.TempDir()\n\tc := file.NewReplicaClient(replicaDir)\n\tr := litestream.NewReplicaWithClient(db, c)\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := db.Snapshot(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\toutputPath := t.TempDir() + \"/follower.db\"\n\tfollowCtx, followCancel := context.WithCancel(ctx)\n\tdefer followCancel()\n\n\terrCh := make(chan error, 1)\n\tgo func() {\n\t\terrCh <- r.Restore(followCtx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tFollow: true,\n\t\t\tFollowInterval: 50 * time.Millisecond,\n\t\t})\n\t}()\n\n\t// Wait for initial restore.\n\tdeadline := time.Now().Add(5 * time.Second)\n\tfor time.Now().Before(deadline) {\n\t\tif _, err := os.Stat(outputPath); err == nil {\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\n\t// Verify -txid file was created after initial restore.\n\ttxidPath := outputPath + \"-txid\"\n\tdeadline = time.Now().Add(5 * time.Second)\n\tfor time.Now().Before(deadline) {\n\t\tif _, err := os.Stat(txidPath); err == nil {\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\tif _, err := os.Stat(txidPath); err != nil {\n\t\tt.Fatalf(\"txid file not created after initial restore: %v\", err)\n\t}\n\n\tinitialTXID, err := litestream.ReadTXIDFile(outputPath)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to read initial txid: %v\", err)\n\t}\n\tif initialTXID == 0 {\n\t\tt.Fatal(\"initial txid should be non-zero\")\n\t}\n\n\t// Insert more data and sync.\n\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO test VALUES (2, 'update')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for follow to apply and update the TXID file.\n\tdeadline = time.Now().Add(5 * time.Second)\n\tvar updatedTXID ltx.TXID\n\tfor time.Now().Before(deadline) {\n\t\ttxid, err := litestream.ReadTXIDFile(outputPath)\n\t\tif err == nil && txid > initialTXID {\n\t\t\tupdatedTXID = txid\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\tif updatedTXID <= initialTXID {\n\t\tt.Fatalf(\"txid file not updated after follow apply: initial=%d, current=%d\", initialTXID, updatedTXID)\n\t}\n\n\tfollowCancel()\n\tselect {\n\tcase err := <-errCh:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"follow returned error: %v\", err)\n\t\t}\n\tcase <-time.After(5 * time.Second):\n\t\tt.Fatal(\"follow did not shut down within timeout\")\n\t}\n}\n\nfunc TestReplica_Restore_Follow_CrashRecovery(t *testing.T) {\n\tctx := context.Background()\n\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE test(id INTEGER PRIMARY KEY, value TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO test VALUES (1, 'initial')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\treplicaDir := t.TempDir()\n\tc := file.NewReplicaClient(replicaDir)\n\tr := litestream.NewReplicaWithClient(db, c)\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := db.Snapshot(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\toutputPath := t.TempDir() + \"/follower.db\"\n\tfollowCtx, followCancel := context.WithCancel(ctx)\n\n\terrCh := make(chan error, 1)\n\tgo func() {\n\t\terrCh <- r.Restore(followCtx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tFollow: true,\n\t\t\tFollowInterval: 50 * time.Millisecond,\n\t\t})\n\t}()\n\n\t// Wait for initial restore and -txid file.\n\tdeadline := time.Now().Add(5 * time.Second)\n\tfor time.Now().Before(deadline) {\n\t\tif _, err := os.Stat(outputPath + \"-txid\"); err == nil {\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\n\tsavedTXID, err := litestream.ReadTXIDFile(outputPath)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to read txid file: %v\", err)\n\t}\n\tif savedTXID == 0 {\n\t\tt.Fatal(\"saved txid should be non-zero\")\n\t}\n\n\t// Simulate crash by cancelling follow mode.\n\tfollowCancel()\n\tselect {\n\tcase err := <-errCh:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"follow returned error: %v\", err)\n\t\t}\n\tcase <-time.After(5 * time.Second):\n\t\tt.Fatal(\"follow did not shut down within timeout\")\n\t}\n\n\t// Verify DB and -txid file still exist.\n\tif _, err := os.Stat(outputPath); err != nil {\n\t\tt.Fatalf(\"database file missing after crash: %v\", err)\n\t}\n\tif _, err := os.Stat(outputPath + \"-txid\"); err != nil {\n\t\tt.Fatalf(\"txid file missing after crash: %v\", err)\n\t}\n\n\t// Add more data to source while follow was down.\n\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO test VALUES (2, 'post-crash')`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Restart follow mode — should resume from saved TXID (crash recovery).\n\tfollowCtx2, followCancel2 := context.WithCancel(ctx)\n\tdefer followCancel2()\n\n\terrCh2 := make(chan error, 1)\n\tgo func() {\n\t\terrCh2 <- r.Restore(followCtx2, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t\tFollow: true,\n\t\t\tFollowInterval: 50 * time.Millisecond,\n\t\t})\n\t}()\n\n\t// Wait for follow mode to pick up new data.\n\tdeadline = time.Now().Add(5 * time.Second)\n\tvar found bool\n\tfor time.Now().Before(deadline) {\n\t\tfollowerDB := testingutil.MustOpenSQLDB(t, outputPath)\n\t\tvar count int\n\t\tif err := followerDB.QueryRowContext(ctx, `SELECT COUNT(*) FROM test WHERE value = 'post-crash'`).Scan(&count); err == nil && count > 0 {\n\t\t\tfound = true\n\t\t\tfollowerDB.Close()\n\t\t\tbreak\n\t\t}\n\t\tfollowerDB.Close()\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\tif !found {\n\t\tt.Fatal(\"crash recovery did not apply new data within timeout\")\n\t}\n\n\tfollowCancel2()\n\tselect {\n\tcase err := <-errCh2:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"follow returned error after recovery: %v\", err)\n\t\t}\n\tcase <-time.After(5 * time.Second):\n\t\tt.Fatal(\"follow did not shut down within timeout after recovery\")\n\t}\n}\n\nfunc TestReplica_Restore_Follow_NoTXIDFile(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tc := file.NewReplicaClient(t.TempDir())\n\tr := litestream.NewReplicaWithClient(db, c)\n\n\t// Create a database file but no -txid sidecar.\n\toutputPath := t.TempDir() + \"/existing.db\"\n\tif err := os.WriteFile(outputPath, []byte(\"fake-db\"), 0644); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr := r.Restore(context.Background(), litestream.RestoreOptions{\n\t\tOutputPath: outputPath,\n\t\tFollow: true,\n\t\tFollowInterval: 50 * time.Millisecond,\n\t})\n\tif err == nil {\n\t\tt.Fatal(\"expected error when DB exists but no -txid file\")\n\t}\n\tif !strings.Contains(err.Error(), \"no -txid file found\") {\n\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t}\n}\n\nfunc TestReplica_Restore_Follow_StaleTXID(t *testing.T) {\n\tctx := context.Background()\n\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE test(id INTEGER PRIMARY KEY, value TEXT)`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\treplicaDir := t.TempDir()\n\tc := file.NewReplicaClient(replicaDir)\n\tr := litestream.NewReplicaWithClient(db, c)\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := db.Snapshot(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := r.Sync(ctx); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\toutputPath := filepath.Join(t.TempDir(), \"follower.db\")\n\n\t// Create a fake database and a TXID file with a low TXID (1).\n\tif err := os.WriteFile(outputPath, []byte(\"fake-db-content\"), 0644); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := litestream.WriteTXIDFile(outputPath, 1); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Simulate retention pruning: remove all level 0 files and replace the\n\t// snapshot with one whose MinTXID is far ahead of our saved TXID.\n\tlevel0Dir := c.LTXLevelDir(0)\n\tif entries, err := os.ReadDir(level0Dir); err == nil {\n\t\tfor _, e := range entries {\n\t\t\tos.Remove(filepath.Join(level0Dir, e.Name()))\n\t\t}\n\t}\n\tsnapshotDir := c.LTXLevelDir(9)\n\tif entries, err := os.ReadDir(snapshotDir); err == nil {\n\t\tfor _, e := range entries {\n\t\t\tos.Remove(filepath.Join(snapshotDir, e.Name()))\n\t\t}\n\t}\n\t// Create a dummy snapshot file with MinTXID=10000 (far ahead of saved TXID=1).\n\tif err := os.MkdirAll(snapshotDir, 0755); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdummySnapshotPath := filepath.Join(snapshotDir, \"0000000000002710-0000000000002710.ltx\")\n\tif err := os.WriteFile(dummySnapshotPath, []byte(\"dummy\"), 0644); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\terr := r.Restore(ctx, litestream.RestoreOptions{\n\t\tOutputPath: outputPath,\n\t\tFollow: true,\n\t\tFollowInterval: 50 * time.Millisecond,\n\t})\n\tif err == nil {\n\t\tt.Fatal(\"expected error for stale TXID\")\n\t}\n\tif !strings.Contains(err.Error(), \"replica history has been pruned\") {\n\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t}\n}\n\n// verifyRestoredDB verifies that the restored database is valid.\nfunc verifyRestoredDB(t *testing.T, path string) {\n\tt.Helper()\n\n\t// Check file exists\n\tinfo, err := os.Stat(path)\n\tif err != nil {\n\t\tt.Fatalf(\"restored file not found: %v\", err)\n\t}\n\tif info.Size() == 0 {\n\t\tt.Fatal(\"restored file is empty\")\n\t}\n\n\t// Try to open with SQLite to verify it's valid\n\tsqldb := testingutil.MustOpenSQLDB(t, path)\n\tdefer testingutil.MustCloseSQLDB(t, sqldb)\n\n\t// Run integrity check\n\tvar result string\n\tif err := sqldb.QueryRow(\"PRAGMA integrity_check\").Scan(&result); err != nil {\n\t\tt.Fatalf(\"integrity check failed: %v\", err)\n\t}\n\tif result != \"ok\" {\n\t\tt.Fatalf(\"integrity check returned: %s\", result)\n\t}\n}\n" }, { "path": "replica_url.go", "content": "package litestream\n\nimport (\n\t\"fmt\"\n\t\"net/url\"\n\t\"path\"\n\t\"regexp\"\n\t\"strings\"\n\t\"sync\"\n)\n\n// ReplicaClientFactory is a function that creates a ReplicaClient from URL components.\n// The userinfo parameter contains credentials from the URL (e.g., user:pass@host).\ntype ReplicaClientFactory func(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (ReplicaClient, error)\n\nvar (\n\treplicaClientFactories = make(map[string]ReplicaClientFactory)\n\treplicaClientFactoriesMu sync.RWMutex\n)\n\n// RegisterReplicaClientFactory registers a factory function for creating replica clients\n// for a given URL scheme. This is typically called from init() functions in backend packages.\nfunc RegisterReplicaClientFactory(scheme string, factory ReplicaClientFactory) {\n\treplicaClientFactoriesMu.Lock()\n\tdefer replicaClientFactoriesMu.Unlock()\n\treplicaClientFactories[scheme] = factory\n}\n\n// NewReplicaClientFromURL creates a new ReplicaClient from a URL string.\n// The URL scheme determines which backend is used (s3, gs, abs, file, etc.).\nfunc NewReplicaClientFromURL(rawURL string) (ReplicaClient, error) {\n\tscheme, host, urlPath, query, userinfo, err := ParseReplicaURLWithQuery(rawURL)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Normalize webdavs to webdav\n\tfactoryScheme := scheme\n\tif factoryScheme == \"webdavs\" {\n\t\tfactoryScheme = \"webdav\"\n\t}\n\n\treplicaClientFactoriesMu.RLock()\n\tfactory, ok := replicaClientFactories[factoryScheme]\n\treplicaClientFactoriesMu.RUnlock()\n\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"unsupported replica URL scheme: %q\", scheme)\n\t}\n\n\treturn factory(scheme, host, urlPath, query, userinfo)\n}\n\n// ReplicaTypeFromURL returns the replica type from a URL string.\n// Returns empty string if the URL is invalid or has no scheme.\nfunc ReplicaTypeFromURL(rawURL string) string {\n\tscheme, _, _, _ := ParseReplicaURL(rawURL)\n\tif scheme == \"\" {\n\t\treturn \"\"\n\t}\n\tif scheme == \"webdavs\" {\n\t\treturn \"webdav\"\n\t}\n\treturn scheme\n}\n\n// ParseReplicaURL parses a replica URL and returns the scheme, host, and path.\nfunc ParseReplicaURL(s string) (scheme, host, urlPath string, err error) {\n\tif strings.HasPrefix(strings.ToLower(s), \"s3://arn:\") {\n\t\tscheme, host, urlPath, _, err = parseS3AccessPointURL(s)\n\t\treturn scheme, host, urlPath, err\n\t}\n\n\tscheme, host, urlPath, _, _, err = ParseReplicaURLWithQuery(s)\n\treturn scheme, host, urlPath, err\n}\n\n// ParseReplicaURLWithQuery parses a replica URL and returns query parameters and userinfo.\nfunc ParseReplicaURLWithQuery(s string) (scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo, err error) {\n\t// Handle S3 Access Point ARNs which can't be parsed by standard url.Parse\n\tif strings.HasPrefix(strings.ToLower(s), \"s3://arn:\") {\n\t\tscheme, host, urlPath, query, err := parseS3AccessPointURL(s)\n\t\treturn scheme, host, urlPath, query, nil, err\n\t}\n\n\tu, err := url.Parse(s)\n\tif err != nil {\n\t\treturn \"\", \"\", \"\", nil, nil, err\n\t}\n\n\tswitch u.Scheme {\n\tcase \"file\":\n\t\tscheme, u.Scheme = u.Scheme, \"\"\n\t\t// Remove query params from path for file URLs\n\t\tu.RawQuery = \"\"\n\t\treturn scheme, \"\", path.Clean(u.String()), nil, nil, nil\n\n\tcase \"\":\n\t\treturn u.Scheme, u.Host, u.Path, nil, nil, fmt.Errorf(\"replica url scheme required: %s\", s)\n\n\tdefault:\n\t\treturn u.Scheme, u.Host, strings.TrimPrefix(path.Clean(u.Path), \"/\"), u.Query(), u.User, nil\n\t}\n}\n\n// parseS3AccessPointURL parses an S3 Access Point URL (s3://arn:...).\nfunc parseS3AccessPointURL(s string) (scheme, host, urlPath string, query url.Values, err error) {\n\tconst prefix = \"s3://\"\n\tif !strings.HasPrefix(strings.ToLower(s), prefix) {\n\t\treturn \"\", \"\", \"\", nil, fmt.Errorf(\"invalid s3 access point url: %s\", s)\n\t}\n\n\tarnWithPath := s[len(prefix):]\n\n\t// Split off query string if present\n\tvar queryStr string\n\tif idx := strings.IndexByte(arnWithPath, '?'); idx != -1 {\n\t\tqueryStr = arnWithPath[idx+1:]\n\t\tarnWithPath = arnWithPath[:idx]\n\t}\n\n\tbucket, key, err := splitS3AccessPointARN(arnWithPath)\n\tif err != nil {\n\t\treturn \"\", \"\", \"\", nil, err\n\t}\n\n\t// Parse query string if present\n\tif queryStr != \"\" {\n\t\tquery, err = url.ParseQuery(queryStr)\n\t\tif err != nil {\n\t\t\treturn \"\", \"\", \"\", nil, fmt.Errorf(\"parse query string: %w\", err)\n\t\t}\n\t}\n\n\treturn \"s3\", bucket, CleanReplicaURLPath(key), query, nil\n}\n\n// splitS3AccessPointARN splits an S3 Access Point ARN into bucket and key components.\nfunc splitS3AccessPointARN(s string) (bucket, key string, err error) {\n\tlower := strings.ToLower(s)\n\tconst marker = \":accesspoint/\"\n\tidx := strings.Index(lower, marker)\n\tif idx == -1 {\n\t\treturn \"\", \"\", fmt.Errorf(\"invalid s3 access point arn: %s\", s)\n\t}\n\n\tnameStart := idx + len(marker)\n\tif nameStart >= len(s) {\n\t\treturn \"\", \"\", fmt.Errorf(\"invalid s3 access point arn: %s\", s)\n\t}\n\n\tremainder := s[nameStart:]\n\tslashIdx := strings.IndexByte(remainder, '/')\n\tif slashIdx == -1 {\n\t\treturn s, \"\", nil\n\t}\n\n\tbucketEnd := nameStart + slashIdx\n\tbucket = s[:bucketEnd]\n\tkey = remainder[slashIdx+1:]\n\treturn bucket, key, nil\n}\n\n// CleanReplicaURLPath cleans a URL path for use in replica storage.\nfunc CleanReplicaURLPath(p string) string {\n\tif p == \"\" {\n\t\treturn \"\"\n\t}\n\tcleaned := path.Clean(\"/\" + p)\n\tcleaned = strings.TrimPrefix(cleaned, \"/\")\n\tif cleaned == \".\" {\n\t\treturn \"\"\n\t}\n\treturn cleaned\n}\n\n// RegionFromS3ARN extracts the region from an S3 ARN.\nfunc RegionFromS3ARN(arn string) string {\n\tparts := strings.SplitN(arn, \":\", 6)\n\tif len(parts) >= 4 {\n\t\treturn parts[3]\n\t}\n\treturn \"\"\n}\n\n// BoolQueryValue returns a boolean value from URL query parameters.\n// It checks multiple keys in order and returns the value and whether it was set.\nfunc BoolQueryValue(query url.Values, keys ...string) (value bool, ok bool) {\n\tif query == nil {\n\t\treturn false, false\n\t}\n\tfor _, key := range keys {\n\t\tif raw := query.Get(key); raw != \"\" {\n\t\t\tswitch strings.ToLower(raw) {\n\t\t\tcase \"true\", \"1\", \"t\", \"yes\":\n\t\t\t\treturn true, true\n\t\t\tcase \"false\", \"0\", \"f\", \"no\":\n\t\t\t\treturn false, true\n\t\t\tdefault:\n\t\t\t\treturn false, true\n\t\t\t}\n\t\t}\n\t}\n\treturn false, false\n}\n\n// IsHetznerEndpoint returns true if the endpoint is Hetzner object storage service.\nfunc IsHetznerEndpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\tif host == \"\" {\n\t\treturn false\n\t}\n\treturn strings.HasSuffix(host, \".your-objectstorage.com\")\n}\n\n// IsTigrisEndpoint returns true if the endpoint is the Tigris object storage service.\nfunc IsTigrisEndpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\treturn host == \"fly.storage.tigris.dev\" || host == \"t3.storage.dev\"\n}\n\n// IsDigitalOceanEndpoint returns true if the endpoint is Digital Ocean Spaces.\nfunc IsDigitalOceanEndpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\tif host == \"\" {\n\t\treturn false\n\t}\n\treturn strings.HasSuffix(host, \".digitaloceanspaces.com\")\n}\n\n// IsBackblazeEndpoint returns true if the endpoint is Backblaze B2.\nfunc IsBackblazeEndpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\tif host == \"\" {\n\t\treturn false\n\t}\n\treturn strings.HasSuffix(host, \".backblazeb2.com\")\n}\n\n// IsFilebaseEndpoint returns true if the endpoint is Filebase.\nfunc IsFilebaseEndpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\tif host == \"\" {\n\t\treturn false\n\t}\n\treturn host == \"s3.filebase.com\"\n}\n\n// IsScalewayEndpoint returns true if the endpoint is Scaleway Object Storage.\nfunc IsScalewayEndpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\tif host == \"\" {\n\t\treturn false\n\t}\n\treturn strings.HasSuffix(host, \".scw.cloud\")\n}\n\n// IsCloudflareR2Endpoint returns true if the endpoint is Cloudflare R2.\nfunc IsCloudflareR2Endpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\tif host == \"\" {\n\t\treturn false\n\t}\n\treturn strings.HasSuffix(host, \".r2.cloudflarestorage.com\")\n}\n\n// IsSupabaseEndpoint returns true if the endpoint is Supabase Storage S3.\nfunc IsSupabaseEndpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\tif host == \"\" {\n\t\treturn false\n\t}\n\treturn strings.HasSuffix(host, \".supabase.co\")\n}\n\n// IsMinIOEndpoint returns true if the endpoint appears to be MinIO or similar\n// (a custom endpoint with a port number that is not a known cloud provider).\nfunc IsMinIOEndpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\tif host == \"\" {\n\t\treturn false\n\t}\n\t// MinIO typically uses host:port format without .com domain\n\t// Check for port number in the host\n\tif !strings.Contains(host, \":\") {\n\t\treturn false\n\t}\n\t// Exclude known cloud providers\n\tif strings.Contains(host, \".amazonaws.com\") ||\n\t\tstrings.Contains(host, \".digitaloceanspaces.com\") ||\n\t\tstrings.Contains(host, \".backblazeb2.com\") ||\n\t\tstrings.Contains(host, \".filebase.com\") ||\n\t\tstrings.Contains(host, \".scw.cloud\") ||\n\t\tstrings.Contains(host, \".r2.cloudflarestorage.com\") ||\n\t\tstrings.Contains(host, \"tigris.dev\") ||\n\t\tstrings.Contains(host, \"t3.storage.dev\") ||\n\t\tstrings.Contains(host, \".supabase.co\") {\n\t\treturn false\n\t}\n\treturn true\n}\n\n// IsLocalEndpoint returns true if the endpoint appears to be a local development\n// endpoint (localhost, 127.0.0.1, or private network addresses).\n// These endpoints typically use HTTP instead of HTTPS.\nfunc IsLocalEndpoint(endpoint string) bool {\n\thost := extractEndpointHost(endpoint)\n\tif host == \"\" {\n\t\treturn false\n\t}\n\t// Remove port if present\n\tif idx := strings.LastIndex(host, \":\"); idx != -1 {\n\t\thost = host[:idx]\n\t}\n\t// Check for common local/development hostnames\n\treturn host == \"localhost\" ||\n\t\thost == \"127.0.0.1\" ||\n\t\tstrings.HasPrefix(host, \"192.168.\") ||\n\t\tstrings.HasPrefix(host, \"10.\") ||\n\t\tstrings.HasPrefix(host, \"172.16.\") ||\n\t\tstrings.HasPrefix(host, \"172.17.\") ||\n\t\tstrings.HasPrefix(host, \"172.18.\") ||\n\t\tstrings.HasPrefix(host, \"172.19.\") ||\n\t\tstrings.HasPrefix(host, \"172.2\") || // 172.20-172.29\n\t\tstrings.HasPrefix(host, \"172.30.\") ||\n\t\tstrings.HasPrefix(host, \"172.31.\") ||\n\t\tstrings.HasSuffix(host, \".local\") ||\n\t\tstrings.HasSuffix(host, \".localhost\")\n}\n\n// EnsureEndpointScheme ensures an endpoint has an HTTP(S) scheme.\n// For local endpoints (localhost, private IPs), it defaults to http://.\n// For all other endpoints (cloud providers), it defaults to https://.\n// Returns the endpoint with scheme and a boolean indicating if a scheme was added.\nfunc EnsureEndpointScheme(endpoint string) (string, bool) {\n\tif endpoint == \"\" {\n\t\treturn \"\", false\n\t}\n\tif strings.HasPrefix(endpoint, \"http://\") || strings.HasPrefix(endpoint, \"https://\") {\n\t\treturn endpoint, false\n\t}\n\t// Default to HTTP for local development endpoints, HTTPS for everything else\n\tif IsLocalEndpoint(endpoint) {\n\t\treturn \"http://\" + endpoint, true\n\t}\n\treturn \"https://\" + endpoint, true\n}\n\n// extractEndpointHost extracts the host from an endpoint URL or returns the\n// endpoint as-is if it's not a full URL.\nfunc extractEndpointHost(endpoint string) string {\n\tendpoint = strings.TrimSpace(strings.ToLower(endpoint))\n\tif endpoint == \"\" {\n\t\treturn \"\"\n\t}\n\tif strings.HasPrefix(endpoint, \"http://\") || strings.HasPrefix(endpoint, \"https://\") {\n\t\tif u, err := url.Parse(endpoint); err == nil && u.Host != \"\" {\n\t\t\treturn u.Host\n\t\t}\n\t}\n\treturn endpoint\n}\n\n// IsURL returns true if s appears to be a URL (has a scheme).\nvar isURLRegex = regexp.MustCompile(`^\\w+:\\/\\/`)\n\nfunc IsURL(s string) bool {\n\treturn isURLRegex.MatchString(s)\n}\n" }, { "path": "replica_url_test.go", "content": "package litestream_test\n\nimport (\n\t\"testing\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/abs\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/gs\"\n\t\"github.com/benbjohnson/litestream/nats\"\n\t\"github.com/benbjohnson/litestream/oss\"\n\t\"github.com/benbjohnson/litestream/s3\"\n\t\"github.com/benbjohnson/litestream/sftp\"\n\t\"github.com/benbjohnson/litestream/webdav\"\n)\n\nfunc TestNewReplicaClientFromURL(t *testing.T) {\n\tt.Run(\"S3\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"s3://mybucket/path/to/db\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.Type() != \"s3\" {\n\t\t\tt.Errorf(\"expected type 's3', got %q\", client.Type())\n\t\t}\n\t\ts3Client, ok := client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *s3.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif s3Client.Bucket != \"mybucket\" {\n\t\t\tt.Errorf(\"expected bucket 'mybucket', got %q\", s3Client.Bucket)\n\t\t}\n\t\tif s3Client.Path != \"path/to/db\" {\n\t\t\tt.Errorf(\"expected path 'path/to/db', got %q\", s3Client.Path)\n\t\t}\n\t})\n\n\tt.Run(\"S3WithQueryParams\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"s3://mybucket/db?endpoint=localhost:9000®ion=us-west-2\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\ts3Client, ok := client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *s3.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif s3Client.Endpoint != \"http://localhost:9000\" {\n\t\t\tt.Errorf(\"expected endpoint 'http://localhost:9000', got %q\", s3Client.Endpoint)\n\t\t}\n\t\tif s3Client.Region != \"us-west-2\" {\n\t\t\tt.Errorf(\"expected region 'us-west-2', got %q\", s3Client.Region)\n\t\t}\n\t})\n\n\tt.Run(\"File\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"file:///tmp/replica\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.Type() != \"file\" {\n\t\t\tt.Errorf(\"expected type 'file', got %q\", client.Type())\n\t\t}\n\t\tfileClient, ok := client.(*file.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *file.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif fileClient.Path() != \"/tmp/replica\" {\n\t\t\tt.Errorf(\"expected path '/tmp/replica', got %q\", fileClient.Path())\n\t\t}\n\t})\n\n\tt.Run(\"GS\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"gs://mybucket/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.Type() != \"gs\" {\n\t\t\tt.Errorf(\"expected type 'gs', got %q\", client.Type())\n\t\t}\n\t\tgsClient, ok := client.(*gs.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *gs.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif gsClient.Bucket != \"mybucket\" {\n\t\t\tt.Errorf(\"expected bucket 'mybucket', got %q\", gsClient.Bucket)\n\t\t}\n\t\tif gsClient.Path != \"path\" {\n\t\t\tt.Errorf(\"expected path 'path', got %q\", gsClient.Path)\n\t\t}\n\t})\n\n\tt.Run(\"GS_MissingBucket\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"gs:///path\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for missing bucket\")\n\t\t}\n\t})\n\n\tt.Run(\"ABS\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"abs://mycontainer/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.Type() != \"abs\" {\n\t\t\tt.Errorf(\"expected type 'abs', got %q\", client.Type())\n\t\t}\n\t\tabsClient, ok := client.(*abs.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *abs.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif absClient.Bucket != \"mycontainer\" {\n\t\t\tt.Errorf(\"expected bucket 'mycontainer', got %q\", absClient.Bucket)\n\t\t}\n\t\tif absClient.Path != \"path\" {\n\t\t\tt.Errorf(\"expected path 'path', got %q\", absClient.Path)\n\t\t}\n\t})\n\n\tt.Run(\"ABS_WithAccount\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"abs://myaccount@mycontainer/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tabsClient, ok := client.(*abs.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *abs.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif absClient.AccountName != \"myaccount\" {\n\t\t\tt.Errorf(\"expected account 'myaccount', got %q\", absClient.AccountName)\n\t\t}\n\t\tif absClient.Bucket != \"mycontainer\" {\n\t\t\tt.Errorf(\"expected bucket 'mycontainer', got %q\", absClient.Bucket)\n\t\t}\n\t})\n\n\tt.Run(\"ABS_MissingBucket\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"abs:///path\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for missing bucket\")\n\t\t}\n\t})\n\n\tt.Run(\"SFTP\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"sftp://myuser@host.example.com/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.Type() != \"sftp\" {\n\t\t\tt.Errorf(\"expected type 'sftp', got %q\", client.Type())\n\t\t}\n\t\tsftpClient, ok := client.(*sftp.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *sftp.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif sftpClient.Host != \"host.example.com\" {\n\t\t\tt.Errorf(\"expected host 'host.example.com', got %q\", sftpClient.Host)\n\t\t}\n\t\tif sftpClient.User != \"myuser\" {\n\t\t\tt.Errorf(\"expected user 'myuser', got %q\", sftpClient.User)\n\t\t}\n\t\tif sftpClient.Path != \"path\" {\n\t\t\tt.Errorf(\"expected path 'path', got %q\", sftpClient.Path)\n\t\t}\n\t})\n\n\tt.Run(\"SFTP_WithPassword\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"sftp://myuser:secret@host.example.com/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tsftpClient, ok := client.(*sftp.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *sftp.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif sftpClient.User != \"myuser\" {\n\t\t\tt.Errorf(\"expected user 'myuser', got %q\", sftpClient.User)\n\t\t}\n\t\tif sftpClient.Password != \"secret\" {\n\t\t\tt.Errorf(\"expected password 'secret', got %q\", sftpClient.Password)\n\t\t}\n\t})\n\n\tt.Run(\"SFTP_RequiresUserError\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"sftp://host.example.com/path\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for missing user\")\n\t\t}\n\t})\n\n\tt.Run(\"SFTP_MissingHost\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"sftp:///path\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for missing host\")\n\t\t}\n\t})\n\n\tt.Run(\"WebDAV\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"webdav://host.example.com/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.Type() != \"webdav\" {\n\t\t\tt.Errorf(\"expected type 'webdav', got %q\", client.Type())\n\t\t}\n\t\twebdavClient, ok := client.(*webdav.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *webdav.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif webdavClient.URL != \"http://host.example.com\" {\n\t\t\tt.Errorf(\"expected URL 'http://host.example.com', got %q\", webdavClient.URL)\n\t\t}\n\t\tif webdavClient.Path != \"path\" {\n\t\t\tt.Errorf(\"expected path 'path', got %q\", webdavClient.Path)\n\t\t}\n\t})\n\n\tt.Run(\"WebDAVS\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"webdavs://host.example.com/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.Type() != \"webdav\" {\n\t\t\tt.Errorf(\"expected type 'webdav', got %q\", client.Type())\n\t\t}\n\t\twebdavClient, ok := client.(*webdav.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *webdav.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif webdavClient.URL != \"https://host.example.com\" {\n\t\t\tt.Errorf(\"expected URL 'https://host.example.com', got %q\", webdavClient.URL)\n\t\t}\n\t})\n\n\tt.Run(\"WebDAV_WithCredentials\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"webdav://myuser:secret@host.example.com/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\twebdavClient, ok := client.(*webdav.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *webdav.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif webdavClient.Username != \"myuser\" {\n\t\t\tt.Errorf(\"expected username 'myuser', got %q\", webdavClient.Username)\n\t\t}\n\t\tif webdavClient.Password != \"secret\" {\n\t\t\tt.Errorf(\"expected password 'secret', got %q\", webdavClient.Password)\n\t\t}\n\t\tif webdavClient.URL != \"http://host.example.com\" {\n\t\t\tt.Errorf(\"expected URL 'http://host.example.com', got %q\", webdavClient.URL)\n\t\t}\n\t})\n\n\tt.Run(\"WebDAVS_WithCredentials\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"webdavs://myuser:secret@host.example.com/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\twebdavClient, ok := client.(*webdav.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *webdav.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif webdavClient.Username != \"myuser\" {\n\t\t\tt.Errorf(\"expected username 'myuser', got %q\", webdavClient.Username)\n\t\t}\n\t\tif webdavClient.Password != \"secret\" {\n\t\t\tt.Errorf(\"expected password 'secret', got %q\", webdavClient.Password)\n\t\t}\n\t\tif webdavClient.URL != \"https://host.example.com\" {\n\t\t\tt.Errorf(\"expected URL 'https://host.example.com', got %q\", webdavClient.URL)\n\t\t}\n\t})\n\n\tt.Run(\"WebDAV_MissingHost\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"webdav:///path\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for missing host\")\n\t\t}\n\t})\n\n\tt.Run(\"NATS\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"nats://localhost:4222/mybucket\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.Type() != \"nats\" {\n\t\t\tt.Errorf(\"expected type 'nats', got %q\", client.Type())\n\t\t}\n\t\tnatsClient, ok := client.(*nats.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *nats.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif natsClient.URL != \"nats://localhost:4222\" {\n\t\t\tt.Errorf(\"expected URL 'nats://localhost:4222', got %q\", natsClient.URL)\n\t\t}\n\t\tif natsClient.BucketName != \"mybucket\" {\n\t\t\tt.Errorf(\"expected bucket 'mybucket', got %q\", natsClient.BucketName)\n\t\t}\n\t})\n\n\tt.Run(\"NATS_WithCredentials\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"nats://myuser:secret@localhost:4222/mybucket\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tnatsClient, ok := client.(*nats.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *nats.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif natsClient.Username != \"myuser\" {\n\t\t\tt.Errorf(\"expected username 'myuser', got %q\", natsClient.Username)\n\t\t}\n\t\tif natsClient.Password != \"secret\" {\n\t\t\tt.Errorf(\"expected password 'secret', got %q\", natsClient.Password)\n\t\t}\n\t\tif natsClient.URL != \"nats://localhost:4222\" {\n\t\t\tt.Errorf(\"expected URL 'nats://localhost:4222', got %q\", natsClient.URL)\n\t\t}\n\t})\n\n\tt.Run(\"NATS_MissingBucket\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"nats://localhost:4222/\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for missing bucket\")\n\t\t}\n\t})\n\n\tt.Run(\"OSS\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"oss://mybucket/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif client.Type() != \"oss\" {\n\t\t\tt.Errorf(\"expected type 'oss', got %q\", client.Type())\n\t\t}\n\t\tossClient, ok := client.(*oss.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *oss.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif ossClient.Bucket != \"mybucket\" {\n\t\t\tt.Errorf(\"expected bucket 'mybucket', got %q\", ossClient.Bucket)\n\t\t}\n\t\tif ossClient.Path != \"path\" {\n\t\t\tt.Errorf(\"expected path 'path', got %q\", ossClient.Path)\n\t\t}\n\t})\n\n\tt.Run(\"OSS_WithRegion\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"oss://mybucket.oss-cn-shanghai.aliyuncs.com/path\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tossClient, ok := client.(*oss.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *oss.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif ossClient.Bucket != \"mybucket\" {\n\t\t\tt.Errorf(\"expected bucket 'mybucket', got %q\", ossClient.Bucket)\n\t\t}\n\t\t// Note: Region is extracted without the 'oss-' prefix\n\t\tif ossClient.Region != \"cn-shanghai\" {\n\t\t\tt.Errorf(\"expected region 'cn-shanghai', got %q\", ossClient.Region)\n\t\t}\n\t})\n\n\tt.Run(\"OSS_MissingBucket\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"oss:///path\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for missing bucket\")\n\t\t}\n\t})\n\n\t// Note: file:// with empty path returns \".\" due to path.Clean behavior.\n\t// This is technically valid but may not be the intended behavior.\n\tt.Run(\"File_EmptyPathReturnsDot\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"file://\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tfileClient, ok := client.(*file.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *file.ReplicaClient, got %T\", client)\n\t\t}\n\t\t// path.Clean(\"\") returns \".\" which passes the empty check\n\t\tif fileClient.Path() != \".\" {\n\t\t\tt.Errorf(\"expected path '.', got %q\", fileClient.Path())\n\t\t}\n\t})\n\n\tt.Run(\"S3_ARN\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/db-access/backups\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\ts3Client, ok := client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *s3.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif s3Client.Bucket != \"arn:aws:s3:us-east-1:123456789012:accesspoint/db-access\" {\n\t\t\tt.Errorf(\"expected bucket ARN, got %q\", s3Client.Bucket)\n\t\t}\n\t\tif s3Client.Path != \"backups\" {\n\t\t\tt.Errorf(\"expected path 'backups', got %q\", s3Client.Path)\n\t\t}\n\t})\n\n\tt.Run(\"S3_ARN_WithQueryParams\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/db-access/backups?sign-payload=false\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\ts3Client, ok := client.(*s3.ReplicaClient)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"expected *s3.ReplicaClient, got %T\", client)\n\t\t}\n\t\tif s3Client.Bucket != \"arn:aws:s3:us-east-1:123456789012:accesspoint/db-access\" {\n\t\t\tt.Errorf(\"expected bucket ARN, got %q\", s3Client.Bucket)\n\t\t}\n\t\tif s3Client.Path != \"backups\" {\n\t\t\tt.Errorf(\"expected path 'backups', got %q\", s3Client.Path)\n\t\t}\n\t\tif s3Client.SignPayload != false {\n\t\t\tt.Errorf(\"expected SignPayload=false from query param, got %v\", s3Client.SignPayload)\n\t\t}\n\t})\n\n\tt.Run(\"S3_MissingBucket\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"s3:///path\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for missing bucket\")\n\t\t}\n\t})\n\n\tt.Run(\"EmptyURL\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for empty URL\")\n\t\t}\n\t})\n\n\tt.Run(\"UnsupportedScheme\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"unknown://bucket/path\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for unsupported scheme\")\n\t\t}\n\t})\n\n\tt.Run(\"InvalidURL\", func(t *testing.T) {\n\t\t_, err := litestream.NewReplicaClientFromURL(\"not-a-valid-url\")\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for invalid URL\")\n\t\t}\n\t})\n}\n\nfunc TestReplicaTypeFromURL(t *testing.T) {\n\ttests := []struct {\n\t\turl string\n\t\texpected string\n\t}{\n\t\t{\"s3://bucket/path\", \"s3\"},\n\t\t{\"gs://bucket/path\", \"gs\"},\n\t\t{\"abs://container/path\", \"abs\"},\n\t\t{\"file:///path/to/replica\", \"file\"},\n\t\t{\"sftp://host/path\", \"sftp\"},\n\t\t{\"webdav://host/path\", \"webdav\"},\n\t\t{\"webdavs://host/path\", \"webdav\"},\n\t\t{\"nats://host/bucket\", \"nats\"},\n\t\t{\"oss://bucket/path\", \"oss\"},\n\t\t{\"\", \"\"},\n\t\t{\"invalid\", \"\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.url, func(t *testing.T) {\n\t\t\tgot := litestream.ReplicaTypeFromURL(tt.url)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"ReplicaTypeFromURL(%q) = %q, want %q\", tt.url, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsURL(t *testing.T) {\n\ttests := []struct {\n\t\ts string\n\t\texpected bool\n\t}{\n\t\t{\"s3://bucket/path\", true},\n\t\t{\"file:///path\", true},\n\t\t{\"https://example.com\", true},\n\t\t{\"/path/to/file\", false},\n\t\t{\"relative/path\", false},\n\t\t{\"\", false},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.s, func(t *testing.T) {\n\t\t\tgot := litestream.IsURL(tt.s)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsURL(%q) = %v, want %v\", tt.s, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestBoolQueryValue(t *testing.T) {\n\tt.Run(\"True values\", func(t *testing.T) {\n\t\tfor _, v := range []string{\"true\", \"True\", \"TRUE\", \"1\", \"t\", \"yes\"} {\n\t\t\tquery := make(map[string][]string)\n\t\t\tquery[\"key\"] = []string{v}\n\t\t\tvalue, ok := litestream.BoolQueryValue(query, \"key\")\n\t\t\tif !ok {\n\t\t\t\tt.Errorf(\"BoolQueryValue with %q should be ok\", v)\n\t\t\t}\n\t\t\tif !value {\n\t\t\t\tt.Errorf(\"BoolQueryValue with %q should be true\", v)\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"False values\", func(t *testing.T) {\n\t\tfor _, v := range []string{\"false\", \"False\", \"FALSE\", \"0\", \"f\", \"no\"} {\n\t\t\tquery := make(map[string][]string)\n\t\t\tquery[\"key\"] = []string{v}\n\t\t\tvalue, ok := litestream.BoolQueryValue(query, \"key\")\n\t\t\tif !ok {\n\t\t\t\tt.Errorf(\"BoolQueryValue with %q should be ok\", v)\n\t\t\t}\n\t\t\tif value {\n\t\t\t\tt.Errorf(\"BoolQueryValue with %q should be false\", v)\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"Missing key\", func(t *testing.T) {\n\t\tquery := make(map[string][]string)\n\t\t_, ok := litestream.BoolQueryValue(query, \"key\")\n\t\tif ok {\n\t\t\tt.Error(\"BoolQueryValue with missing key should not be ok\")\n\t\t}\n\t})\n\n\tt.Run(\"Multiple keys\", func(t *testing.T) {\n\t\tquery := make(map[string][]string)\n\t\tquery[\"key2\"] = []string{\"true\"}\n\t\tvalue, ok := litestream.BoolQueryValue(query, \"key1\", \"key2\")\n\t\tif !ok {\n\t\t\tt.Error(\"BoolQueryValue should find second key\")\n\t\t}\n\t\tif !value {\n\t\t\tt.Error(\"BoolQueryValue should return true for second key\")\n\t\t}\n\t})\n\n\tt.Run(\"Nil query\", func(t *testing.T) {\n\t\t_, ok := litestream.BoolQueryValue(nil, \"key\")\n\t\tif ok {\n\t\t\tt.Error(\"BoolQueryValue with nil query should not be ok\")\n\t\t}\n\t})\n\n\tt.Run(\"Invalid value returns false with ok\", func(t *testing.T) {\n\t\tquery := make(map[string][]string)\n\t\tquery[\"key\"] = []string{\"invalid\"}\n\t\tvalue, ok := litestream.BoolQueryValue(query, \"key\")\n\t\tif !ok {\n\t\t\tt.Error(\"BoolQueryValue with invalid value should be ok\")\n\t\t}\n\t\tif value {\n\t\t\tt.Error(\"BoolQueryValue with invalid value should be false\")\n\t\t}\n\t})\n}\n\nfunc TestIsTigrisEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t{\"fly.storage.tigris.dev\", true},\n\t\t{\"FLY.STORAGE.TIGRIS.DEV\", true},\n\t\t{\"https://fly.storage.tigris.dev\", true},\n\t\t{\"http://fly.storage.tigris.dev\", true},\n\t\t{\"t3.storage.dev\", true},\n\t\t{\"T3.STORAGE.DEV\", true},\n\t\t{\"https://t3.storage.dev\", true},\n\t\t{\"http://t3.storage.dev\", true},\n\t\t{\"s3.amazonaws.com\", false},\n\t\t{\"localhost:9000\", false},\n\t\t{\"\", false},\n\t\t{\" \", false},\n\t\t{\"https://s3.us-east-1.amazonaws.com\", false},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsTigrisEndpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsTigrisEndpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRegionFromS3ARN(t *testing.T) {\n\ttests := []struct {\n\t\tarn string\n\t\texpected string\n\t}{\n\t\t{\"arn:aws:s3:us-east-1:123456789012:accesspoint/db-access\", \"us-east-1\"},\n\t\t{\"arn:aws:s3:eu-west-1:123456789012:accesspoint/db-access\", \"eu-west-1\"},\n\t\t{\"arn:aws:s3:ap-southeast-2:123456789012:accesspoint/db-access\", \"ap-southeast-2\"},\n\t\t{\"arn:aws:s3::123456789012:accesspoint/db-access\", \"\"},\n\t\t{\"invalid-arn\", \"\"},\n\t\t{\"\", \"\"},\n\t\t{\"arn:aws:s3\", \"\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.arn, func(t *testing.T) {\n\t\t\tgot := litestream.RegionFromS3ARN(tt.arn)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"RegionFromS3ARN(%q) = %q, want %q\", tt.arn, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCleanReplicaURLPath(t *testing.T) {\n\ttests := []struct {\n\t\tpath string\n\t\texpected string\n\t}{\n\t\t{\"\", \"\"},\n\t\t{\"path\", \"path\"},\n\t\t{\"/path\", \"path\"},\n\t\t{\"path/\", \"path\"},\n\t\t{\"/path/\", \"path\"},\n\t\t{\"path/to/db\", \"path/to/db\"},\n\t\t{\"/path/to/db\", \"path/to/db\"},\n\t\t{\"//path//to//db\", \"path/to/db\"},\n\t\t{\".\", \"\"},\n\t\t{\"/.\", \"\"},\n\t\t{\"./path\", \"path\"},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.path, func(t *testing.T) {\n\t\t\tgot := litestream.CleanReplicaURLPath(tt.path)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"CleanReplicaURLPath(%q) = %q, want %q\", tt.path, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestParseS3AccessPointURL(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\turl string\n\t\twantScheme string\n\t\twantHost string\n\t\twantPath string\n\t\twantQuery map[string]string\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"BasicARN\",\n\t\t\turl: \"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point\",\n\t\t\twantScheme: \"s3\",\n\t\t\twantHost: \"arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point\",\n\t\t\twantPath: \"\",\n\t\t\twantQuery: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"ARNWithPath\",\n\t\t\turl: \"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point/backups/db\",\n\t\t\twantScheme: \"s3\",\n\t\t\twantHost: \"arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point\",\n\t\t\twantPath: \"backups/db\",\n\t\t\twantQuery: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"ARNWithSingleQueryParam\",\n\t\t\turl: \"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point?sign-payload=true\",\n\t\t\twantScheme: \"s3\",\n\t\t\twantHost: \"arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point\",\n\t\t\twantPath: \"\",\n\t\t\twantQuery: map[string]string{\"sign-payload\": \"true\"},\n\t\t},\n\t\t{\n\t\t\tname: \"ARNWithMultipleQueryParams\",\n\t\t\turl: \"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point?sign-payload=false®ion=us-west-2\",\n\t\t\twantScheme: \"s3\",\n\t\t\twantHost: \"arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point\",\n\t\t\twantPath: \"\",\n\t\t\twantQuery: map[string]string{\"sign-payload\": \"false\", \"region\": \"us-west-2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"ARNWithPathAndQuery\",\n\t\t\turl: \"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point/backups?sign-payload=true\",\n\t\t\twantScheme: \"s3\",\n\t\t\twantHost: \"arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point\",\n\t\t\twantPath: \"backups\",\n\t\t\twantQuery: map[string]string{\"sign-payload\": \"true\"},\n\t\t},\n\t\t{\n\t\t\tname: \"CaseInsensitiveScheme\",\n\t\t\turl: \"S3://arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point\",\n\t\t\twantScheme: \"s3\",\n\t\t\twantHost: \"arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point\",\n\t\t\twantPath: \"\",\n\t\t\twantQuery: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"EmptyQueryValue\",\n\t\t\turl: \"s3://arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point?key=\",\n\t\t\twantScheme: \"s3\",\n\t\t\twantHost: \"arn:aws:s3:us-east-1:123456789012:accesspoint/my-access-point\",\n\t\t\twantPath: \"\",\n\t\t\twantQuery: map[string]string{\"key\": \"\"},\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tscheme, host, path, query, _, err := litestream.ParseReplicaURLWithQuery(tt.url)\n\n\t\t\tif tt.wantErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Fatal(\"expected error, got nil\")\n\t\t\t\t}\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"unexpected error: %v\", err)\n\t\t\t}\n\n\t\t\tif scheme != tt.wantScheme {\n\t\t\t\tt.Errorf(\"scheme = %q, want %q\", scheme, tt.wantScheme)\n\t\t\t}\n\t\t\tif host != tt.wantHost {\n\t\t\t\tt.Errorf(\"host = %q, want %q\", host, tt.wantHost)\n\t\t\t}\n\t\t\tif path != tt.wantPath {\n\t\t\t\tt.Errorf(\"path = %q, want %q\", path, tt.wantPath)\n\t\t\t}\n\n\t\t\tif tt.wantQuery == nil {\n\t\t\t\tif len(query) > 0 {\n\t\t\t\t\tt.Errorf(\"query = %v, want nil/empty\", query)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tfor key, wantVal := range tt.wantQuery {\n\t\t\t\t\tif gotVal := query.Get(key); gotVal != wantVal {\n\t\t\t\t\t\tt.Errorf(\"query[%q] = %q, want %q\", key, gotVal, wantVal)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsDigitalOceanEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t{\"https://sfo3.digitaloceanspaces.com\", true},\n\t\t{\"https://nyc3.digitaloceanspaces.com\", true},\n\t\t{\"sfo3.digitaloceanspaces.com\", true},\n\t\t{\"https://s3.amazonaws.com\", false},\n\t\t{\"https://s3.filebase.com\", false},\n\t\t{\"\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsDigitalOceanEndpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsDigitalOceanEndpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsBackblazeEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t{\"https://s3.us-west-002.backblazeb2.com\", true},\n\t\t{\"https://s3.eu-central-003.backblazeb2.com\", true},\n\t\t{\"s3.us-west-002.backblazeb2.com\", true},\n\t\t{\"https://s3.amazonaws.com\", false},\n\t\t{\"https://s3.filebase.com\", false},\n\t\t{\"\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsBackblazeEndpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsBackblazeEndpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsFilebaseEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t{\"https://s3.filebase.com\", true},\n\t\t{\"http://s3.filebase.com\", true},\n\t\t{\"s3.filebase.com\", true},\n\t\t{\"https://s3.amazonaws.com\", false},\n\t\t{\"https://sfo3.digitaloceanspaces.com\", false},\n\t\t{\"\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsFilebaseEndpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsFilebaseEndpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsScalewayEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t{\"https://s3.fr-par.scw.cloud\", true},\n\t\t{\"https://s3.nl-ams.scw.cloud\", true},\n\t\t{\"s3.fr-par.scw.cloud\", true},\n\t\t{\"https://s3.amazonaws.com\", false},\n\t\t{\"https://s3.filebase.com\", false},\n\t\t{\"\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsScalewayEndpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsScalewayEndpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsCloudflareR2Endpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t{\"https://abcdef123456.r2.cloudflarestorage.com\", true},\n\t\t{\"https://account-id.r2.cloudflarestorage.com\", true},\n\t\t{\"abcdef123456.r2.cloudflarestorage.com\", true},\n\t\t{\"https://s3.amazonaws.com\", false},\n\t\t{\"https://s3.filebase.com\", false},\n\t\t{\"\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsCloudflareR2Endpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsCloudflareR2Endpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsSupabaseEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t{\"https://myproject.supabase.co/storage/v1/s3\", true},\n\t\t{\"https://abcdefghij.supabase.co\", true},\n\t\t{\"myproject.supabase.co\", true},\n\t\t{\"https://s3.amazonaws.com\", false},\n\t\t{\"https://s3.filebase.com\", false},\n\t\t{\"\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsSupabaseEndpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsSupabaseEndpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsHetznerEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t{\"fsn1.your-objectstorage.com\", true},\n\t\t{\"nbg1.your-objectstorage.com\", true},\n\t\t{\"https://fsn1.your-objectstorage.com\", true},\n\t\t{\"http://nbg1.your-objectstorage.com\", true},\n\t\t{\"s3.amazonaws.com\", false},\n\t\t{\"https://s3.filebase.com\", false},\n\t\t{\"\", false},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsHetznerEndpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsHetznerEndpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsMinIOEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t{\"http://localhost:9000\", true},\n\t\t{\"http://192.168.1.100:9000\", true},\n\t\t{\"minio.local:9000\", true},\n\t\t{\"https://s3.amazonaws.com\", false},\n\t\t{\"https://s3.filebase.com\", false},\n\t\t{\"https://sfo3.digitaloceanspaces.com\", false},\n\t\t{\"s3.filebase.com\", false}, // No port, not MinIO\n\t\t{\"\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsMinIOEndpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsMinIOEndpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsLocalEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tendpoint string\n\t\texpected bool\n\t}{\n\t\t// Localhost variants\n\t\t{\"localhost\", true},\n\t\t{\"localhost:9000\", true},\n\t\t{\"http://localhost:9000\", true},\n\t\t{\"https://localhost:9000\", true},\n\n\t\t// Loopback IP\n\t\t{\"127.0.0.1\", true},\n\t\t{\"127.0.0.1:9000\", true},\n\t\t{\"http://127.0.0.1:9000\", true},\n\n\t\t// Private network ranges (RFC1918)\n\t\t{\"192.168.1.100\", true},\n\t\t{\"192.168.1.100:9000\", true},\n\t\t{\"http://192.168.1.100:9000\", true},\n\t\t{\"10.0.0.1\", true},\n\t\t{\"10.0.0.1:9000\", true},\n\t\t{\"172.16.0.1\", true},\n\t\t{\"172.31.255.255\", true},\n\n\t\t// .local and .localhost TLDs\n\t\t{\"minio.local\", true},\n\t\t{\"minio.local:9000\", true},\n\t\t{\"http://minio.local:9000\", true},\n\t\t{\"dev.localhost\", true},\n\t\t{\"test.localhost:8080\", true},\n\n\t\t// Non-local endpoints (cloud providers)\n\t\t{\"s3.amazonaws.com\", false},\n\t\t{\"https://s3.amazonaws.com\", false},\n\t\t{\"abcdef.r2.cloudflarestorage.com\", false},\n\t\t{\"https://abcdef.r2.cloudflarestorage.com\", false},\n\t\t{\"fly.storage.tigris.dev\", false},\n\t\t{\"s3.us-west-000.backblazeb2.com\", false},\n\t\t{\"sfo3.digitaloceanspaces.com\", false},\n\t\t{\"s3.filebase.com\", false},\n\n\t\t// Empty\n\t\t{\"\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.endpoint, func(t *testing.T) {\n\t\t\tgot := litestream.IsLocalEndpoint(tt.endpoint)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"IsLocalEndpoint(%q) = %v, want %v\", tt.endpoint, got, tt.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestS3ProviderDefaults tests that provider-specific defaults are applied\n// when creating S3 clients from URLs with provider endpoints.\n// These tests ensure edge case bugs like #912, #918, #940, #947 don't regress.\nfunc TestS3ProviderDefaults(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\turl string\n\t\twantSignPayload bool\n\t\twantForcePathStyle bool\n\t\twantRequireMD5 bool\n\t}{\n\t\t{\n\t\t\tname: \"CloudflareR2_SignPayload\",\n\t\t\turl: \"s3://mybucket/path?endpoint=https://account123.r2.cloudflarestorage.com\",\n\t\t\twantSignPayload: true,\n\t\t\twantForcePathStyle: true, // Custom endpoint default\n\t\t\twantRequireMD5: true,\n\t\t},\n\t\t{\n\t\t\tname: \"BackblazeB2_SignPayloadAndPathStyle\",\n\t\t\turl: \"s3://mybucket/path?endpoint=https://s3.us-west-002.backblazeb2.com\",\n\t\t\twantSignPayload: true,\n\t\t\twantForcePathStyle: true,\n\t\t\twantRequireMD5: true,\n\t\t},\n\t\t{\n\t\t\tname: \"DigitalOcean_SignPayload\",\n\t\t\turl: \"s3://mybucket/path?endpoint=https://sfo3.digitaloceanspaces.com\",\n\t\t\twantSignPayload: true,\n\t\t\twantForcePathStyle: true, // Custom endpoint default\n\t\t\twantRequireMD5: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Scaleway_SignPayload\",\n\t\t\turl: \"s3://mybucket/path?endpoint=https://s3.fr-par.scw.cloud\",\n\t\t\twantSignPayload: true,\n\t\t\twantForcePathStyle: true, // Custom endpoint default\n\t\t\twantRequireMD5: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Filebase_SignPayloadAndPathStyle\",\n\t\t\turl: \"s3://mybucket/path?endpoint=https://s3.filebase.com\",\n\t\t\twantSignPayload: true,\n\t\t\twantForcePathStyle: true,\n\t\t\twantRequireMD5: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Tigris_SignPayloadNoMD5\",\n\t\t\turl: \"s3://mybucket/path?endpoint=https://fly.storage.tigris.dev\",\n\t\t\twantSignPayload: true,\n\t\t\twantForcePathStyle: true, // Custom endpoint default\n\t\t\twantRequireMD5: false,\n\t\t},\n\t\t{\n\t\t\tname: \"MinIO_SignPayloadAndPathStyle\",\n\t\t\turl: \"s3://mybucket/path?endpoint=http://localhost:9000\",\n\t\t\twantSignPayload: true,\n\t\t\twantForcePathStyle: true,\n\t\t\twantRequireMD5: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Hetzner_SignPayload\",\n\t\t\turl: \"s3://mybucket/path?endpoint=https://fsn1.your-objectstorage.com\",\n\t\t\twantSignPayload: true,\n\t\t\twantForcePathStyle: true,\n\t\t\twantRequireMD5: true,\n\t\t},\n\t\t{\n\t\t\tname: \"AWS_Defaults\",\n\t\t\turl: \"s3://mybucket/path\",\n\t\t\twantSignPayload: true, // Default\n\t\t\twantForcePathStyle: false,\n\t\t\twantRequireMD5: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tclient, err := litestream.NewReplicaClientFromURL(tt.url)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"NewReplicaClientFromURL(%q) error: %v\", tt.url, err)\n\t\t\t}\n\n\t\t\ts3Client, ok := client.(*s3.ReplicaClient)\n\t\t\tif !ok {\n\t\t\t\tt.Fatalf(\"expected *s3.ReplicaClient, got %T\", client)\n\t\t\t}\n\n\t\t\tif s3Client.SignPayload != tt.wantSignPayload {\n\t\t\t\tt.Errorf(\"SignPayload = %v, want %v\", s3Client.SignPayload, tt.wantSignPayload)\n\t\t\t}\n\t\t\tif s3Client.ForcePathStyle != tt.wantForcePathStyle {\n\t\t\t\tt.Errorf(\"ForcePathStyle = %v, want %v\", s3Client.ForcePathStyle, tt.wantForcePathStyle)\n\t\t\t}\n\t\t\tif s3Client.RequireContentMD5 != tt.wantRequireMD5 {\n\t\t\t\tt.Errorf(\"RequireContentMD5 = %v, want %v\", s3Client.RequireContentMD5, tt.wantRequireMD5)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestEnsureEndpointScheme(t *testing.T) {\n\ttests := []struct {\n\t\tinput string\n\t\texpected string\n\t\tschemeAdded bool\n\t}{\n\t\t// Already has scheme - no change\n\t\t{\"https://example.com\", \"https://example.com\", false},\n\t\t{\"http://localhost:9000\", \"http://localhost:9000\", false},\n\t\t{\"http://192.168.1.1:9000\", \"http://192.168.1.1:9000\", false},\n\n\t\t// Local endpoints get http://\n\t\t{\"localhost:9000\", \"http://localhost:9000\", true},\n\t\t{\"127.0.0.1:9000\", \"http://127.0.0.1:9000\", true},\n\t\t{\"192.168.1.100:9000\", \"http://192.168.1.100:9000\", true},\n\t\t{\"10.0.0.1:9000\", \"http://10.0.0.1:9000\", true},\n\t\t{\"minio.local:9000\", \"http://minio.local:9000\", true},\n\n\t\t// Cloud endpoints get https:// (THIS IS THE KEY FIX)\n\t\t{\"abcdef.r2.cloudflarestorage.com\", \"https://abcdef.r2.cloudflarestorage.com\", true},\n\t\t{\"s3.us-west-000.backblazeb2.com\", \"https://s3.us-west-000.backblazeb2.com\", true},\n\t\t{\"fly.storage.tigris.dev\", \"https://fly.storage.tigris.dev\", true},\n\t\t{\"sfo3.digitaloceanspaces.com\", \"https://sfo3.digitaloceanspaces.com\", true},\n\t\t{\"s3.filebase.com\", \"https://s3.filebase.com\", true},\n\t\t{\"s3.fr-par.scw.cloud\", \"https://s3.fr-par.scw.cloud\", true},\n\n\t\t// Empty returns empty\n\t\t{\"\", \"\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.input, func(t *testing.T) {\n\t\t\tgot, added := litestream.EnsureEndpointScheme(tt.input)\n\t\t\tif got != tt.expected {\n\t\t\t\tt.Errorf(\"EnsureEndpointScheme(%q) = %q, want %q\", tt.input, got, tt.expected)\n\t\t\t}\n\t\t\tif added != tt.schemeAdded {\n\t\t\t\tt.Errorf(\"EnsureEndpointScheme(%q) schemeAdded = %v, want %v\", tt.input, added, tt.schemeAdded)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestS3ProviderDefaults_QueryParamOverrides tests that explicit query parameters\n// override provider-specific defaults.\nfunc TestS3ProviderDefaults_QueryParamOverrides(t *testing.T) {\n\tt.Run(\"SignPayload_ExplicitFalse\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"s3://mybucket/path?endpoint=https://account123.r2.cloudflarestorage.com&sign-payload=false\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\ts3Client := client.(*s3.ReplicaClient)\n\t\tif s3Client.SignPayload != false {\n\t\t\tt.Errorf(\"SignPayload = %v, want false (explicit override)\", s3Client.SignPayload)\n\t\t}\n\t})\n\n\tt.Run(\"ForcePathStyle_ExplicitFalse\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"s3://mybucket/path?endpoint=https://s3.us-west-002.backblazeb2.com&forcePathStyle=false\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\ts3Client := client.(*s3.ReplicaClient)\n\t\tif s3Client.ForcePathStyle != false {\n\t\t\tt.Errorf(\"ForcePathStyle = %v, want false (explicit override)\", s3Client.ForcePathStyle)\n\t\t}\n\t})\n\n\tt.Run(\"RequireMD5_ExplicitTrue_Tigris\", func(t *testing.T) {\n\t\tclient, err := litestream.NewReplicaClientFromURL(\"s3://mybucket/path?endpoint=https://fly.storage.tigris.dev&require-content-md5=true\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\ts3Client := client.(*s3.ReplicaClient)\n\t\tif s3Client.RequireContentMD5 != true {\n\t\t\tt.Errorf(\"RequireContentMD5 = %v, want true (explicit override)\", s3Client.RequireContentMD5)\n\t\t}\n\t})\n}\n" }, { "path": "restore_fuzz_test.go", "content": "package litestream_test\n\nimport (\n\t\"math/rand\"\n\t\"path/filepath\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc FuzzRestoreWithMissingCompactedFile(f *testing.F) {\n\tf.Add(int64(1))\n\tf.Add(int64(2))\n\tf.Add(int64(3))\n\n\tf.Fuzz(func(t *testing.T, seed int64) {\n\t\tif testing.Short() {\n\t\t\tt.Skip(\"skipping fuzz test in short mode\")\n\t\t}\n\n\t\trng := rand.New(rand.NewSource(seed))\n\t\tctx := t.Context()\n\n\t\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\t\tdb.MonitorInterval = 20 * time.Millisecond\n\t\tdb.Replica = litestream.NewReplica(db)\n\t\tdb.Replica.SyncInterval = 20 * time.Millisecond\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tdb.Replica.Client = client\n\n\t\tif err := db.Open(); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tsqldb := testingutil.MustOpenSQLDB(t, db.Path())\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t\t{Level: 1, Interval: 50 * time.Millisecond},\n\t\t\t{Level: 2, Interval: 150 * time.Millisecond},\n\t\t})\n\t\tstore.SnapshotInterval = 200 * time.Millisecond\n\t\tif err := store.Open(ctx); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer store.Close(ctx)\n\n\t\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE t (id INTEGER PRIMARY KEY, val TEXT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tdone := make(chan struct{})\n\t\tvar wg sync.WaitGroup\n\t\twg.Add(1)\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\tticker := time.NewTicker(5 * time.Millisecond)\n\t\t\tdefer ticker.Stop()\n\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\tcase <-done:\n\t\t\t\t\treturn\n\t\t\t\tcase <-ticker.C:\n\t\t\t\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO t (val) VALUES (?);`, time.Now().String()); err != nil {\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\n\t\trunDuration := 800*time.Millisecond + time.Duration(rng.Intn(400))*time.Millisecond\n\t\ttime.Sleep(runDuration)\n\t\tclose(done)\n\t\twg.Wait()\n\n\t\t// Allow compaction/snapshotting to catch up.\n\t\ttime.Sleep(200 * time.Millisecond)\n\n\t\tvar candidates []*ltx.FileInfo\n\t\tfor _, level := range []int{1, 2} {\n\t\t\titr, err := client.LTXFiles(ctx, level, 0, false)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t\tfor itr.Next() {\n\t\t\t\tcandidates = append(candidates, itr.Item())\n\t\t\t}\n\t\t\tif err := itr.Close(); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\t\t}\n\n\t\tif len(candidates) == 0 {\n\t\t\tt.Skip(\"no compacted files available to delete\")\n\t\t}\n\n\t\ttoDelete := candidates[rng.Intn(len(candidates))]\n\t\tif err := client.DeleteLTXFiles(ctx, []*ltx.FileInfo{toDelete}); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\toutputPath := filepath.Join(t.TempDir(), \"restore.db\")\n\t\tif err := db.Replica.Restore(ctx, litestream.RestoreOptions{\n\t\t\tOutputPath: outputPath,\n\t\t}); err != nil {\n\t\t\tt.Fatalf(\"restore failed after deleting L%d %s: %v\", toDelete.Level, ltx.FormatFilename(toDelete.MinTXID, toDelete.MaxTXID), err)\n\t\t}\n\n\t\trestoreDB := testingutil.MustOpenSQLDB(t, outputPath)\n\t\tdefer testingutil.MustCloseSQLDB(t, restoreDB)\n\n\t\tvar result string\n\t\tif err := restoreDB.QueryRowContext(ctx, `PRAGMA integrity_check;`).Scan(&result); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if result != \"ok\" {\n\t\t\tt.Fatalf(\"integrity check failed: %s\", result)\n\t\t}\n\n\t\tvar count int\n\t\tif err := restoreDB.QueryRowContext(ctx, `SELECT COUNT(*) FROM t`).Scan(&count); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if count == 0 {\n\t\t\tt.Fatal(\"no records found in restored database\")\n\t\t}\n\t})\n}\n" }, { "path": "s3/leaser.go", "content": "package s3\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"log/slog\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/aws/aws-sdk-go-v2/aws\"\n\t\"github.com/aws/aws-sdk-go-v2/service/s3\"\n\t\"github.com/aws/aws-sdk-go-v2/service/s3/types\"\n\t\"github.com/aws/smithy-go\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nconst (\n\tDefaultLeaseTTL = 30 * time.Second\n\tDefaultLeasePath = \"lock.json\"\n\tLeaserType = \"s3\"\n)\n\nvar (\n\t_ litestream.Leaser = (*Leaser)(nil)\n\n\tErrLeaseRequired = errors.New(\"lease required\")\n\tErrLeaseETagRequired = errors.New(\"lease etag required\")\n\tErrLeaseAlreadyReleased = errors.New(\"lease already released\")\n)\n\n// S3API is the interface for S3 operations needed by Leaser.\ntype S3API interface {\n\tGetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.Options)) (*s3.GetObjectOutput, error)\n\tPutObject(ctx context.Context, params *s3.PutObjectInput, optFns ...func(*s3.Options)) (*s3.PutObjectOutput, error)\n\tDeleteObject(ctx context.Context, params *s3.DeleteObjectInput, optFns ...func(*s3.Options)) (*s3.DeleteObjectOutput, error)\n}\n\ntype Leaser struct {\n\ts3 S3API\n\tlogger *slog.Logger\n\n\tBucket string\n\tPath string\n\tTTL time.Duration\n\tOwner string\n}\n\nfunc NewLeaser() *Leaser {\n\towner, _ := os.Hostname()\n\tif owner == \"\" {\n\t\towner = fmt.Sprintf(\"pid-%d\", os.Getpid())\n\t} else {\n\t\towner = fmt.Sprintf(\"%s:%d\", owner, os.Getpid())\n\t}\n\n\treturn &Leaser{\n\t\tlogger: slog.Default().WithGroup(\"s3-leaser\"),\n\t\tTTL: DefaultLeaseTTL,\n\t\tOwner: owner,\n\t}\n}\n\nfunc (l *Leaser) SetLogger(logger *slog.Logger) {\n\tl.logger = logger.WithGroup(\"s3-leaser\")\n}\n\nfunc (l *Leaser) Client() S3API {\n\treturn l.s3\n}\n\nfunc (l *Leaser) SetClient(client S3API) {\n\tl.s3 = client\n}\n\nfunc (l *Leaser) Type() string {\n\treturn LeaserType\n}\n\nfunc (l *Leaser) lockKey() string {\n\tif l.Path == \"\" {\n\t\treturn DefaultLeasePath\n\t}\n\treturn l.Path + \"/\" + DefaultLeasePath\n}\n\nfunc (l *Leaser) AcquireLease(ctx context.Context) (*litestream.Lease, error) {\n\texisting, etag, err := l.readLease(ctx)\n\tif err != nil && !errors.Is(err, os.ErrNotExist) {\n\t\treturn nil, fmt.Errorf(\"read existing lease: %w\", err)\n\t}\n\n\tif existing != nil && !existing.IsExpired() {\n\t\treturn nil, &litestream.LeaseExistsError{\n\t\t\tOwner: existing.Owner,\n\t\t\tExpiresAt: existing.ExpiresAt,\n\t\t}\n\t}\n\n\tvar generation int64 = 1\n\tif existing != nil {\n\t\tgeneration = existing.Generation + 1\n\t}\n\n\tnewLease := &litestream.Lease{\n\t\tGeneration: generation,\n\t\tExpiresAt: time.Now().Add(l.TTL),\n\t\tOwner: l.Owner,\n\t}\n\n\tnewETag, err := l.writeLease(ctx, newLease, etag)\n\tif err != nil {\n\t\tvar leaseErr *litestream.LeaseExistsError\n\t\tif errors.As(err, &leaseErr) {\n\t\t\tif current, _, readErr := l.readLease(ctx); readErr == nil && current != nil {\n\t\t\t\treturn nil, &litestream.LeaseExistsError{\n\t\t\t\t\tOwner: current.Owner,\n\t\t\t\t\tExpiresAt: current.ExpiresAt,\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\treturn nil, err\n\t}\n\n\tnewLease.ETag = newETag\n\tl.logger.Debug(\"lease acquired\",\n\t\t\"generation\", newLease.Generation,\n\t\t\"owner\", newLease.Owner,\n\t\t\"expires_at\", newLease.ExpiresAt,\n\t\t\"etag\", newLease.ETag)\n\n\treturn newLease, nil\n}\n\nfunc (l *Leaser) RenewLease(ctx context.Context, lease *litestream.Lease) (*litestream.Lease, error) {\n\tif lease == nil {\n\t\treturn nil, ErrLeaseRequired\n\t}\n\tif lease.ETag == \"\" {\n\t\treturn nil, ErrLeaseETagRequired\n\t}\n\n\tnewLease := &litestream.Lease{\n\t\tGeneration: lease.Generation,\n\t\tExpiresAt: time.Now().Add(l.TTL),\n\t\tOwner: l.Owner,\n\t}\n\n\tnewETag, err := l.writeLease(ctx, newLease, lease.ETag)\n\tif err != nil {\n\t\tvar leaseErr *litestream.LeaseExistsError\n\t\tif errors.As(err, &leaseErr) {\n\t\t\treturn nil, litestream.ErrLeaseNotHeld\n\t\t}\n\t\treturn nil, err\n\t}\n\n\tnewLease.ETag = newETag\n\tl.logger.Debug(\"lease renewed\",\n\t\t\"generation\", newLease.Generation,\n\t\t\"owner\", newLease.Owner,\n\t\t\"expires_at\", newLease.ExpiresAt,\n\t\t\"etag\", newLease.ETag)\n\n\treturn newLease, nil\n}\n\nfunc (l *Leaser) ReleaseLease(ctx context.Context, lease *litestream.Lease) error {\n\tif lease == nil {\n\t\treturn ErrLeaseRequired\n\t}\n\tif lease.ETag == \"\" {\n\t\treturn ErrLeaseETagRequired\n\t}\n\n\tkey := l.lockKey()\n\t_, err := l.s3.DeleteObject(ctx, &s3.DeleteObjectInput{\n\t\tBucket: aws.String(l.Bucket),\n\t\tKey: aws.String(key),\n\t\tIfMatch: aws.String(lease.ETag),\n\t})\n\tif err != nil {\n\t\tif isNotExists(err) || isNotFoundError(err) {\n\t\t\treturn ErrLeaseAlreadyReleased\n\t\t}\n\t\tif isPreconditionFailed(err) {\n\t\t\treturn litestream.ErrLeaseNotHeld\n\t\t}\n\t\treturn fmt.Errorf(\"delete lease: %w\", err)\n\t}\n\n\tl.logger.Debug(\"lease released\",\n\t\t\"generation\", lease.Generation,\n\t\t\"owner\", lease.Owner)\n\n\treturn nil\n}\n\nfunc (l *Leaser) readLease(ctx context.Context) (*litestream.Lease, string, error) {\n\tkey := l.lockKey()\n\n\tout, err := l.s3.GetObject(ctx, &s3.GetObjectInput{\n\t\tBucket: aws.String(l.Bucket),\n\t\tKey: aws.String(key),\n\t})\n\tif err != nil {\n\t\tif isNotExists(err) || isNotFoundError(err) {\n\t\t\treturn nil, \"\", os.ErrNotExist\n\t\t}\n\t\treturn nil, \"\", fmt.Errorf(\"get lock file: %w\", err)\n\t}\n\tdefer out.Body.Close()\n\n\tvar lease litestream.Lease\n\tif err := json.NewDecoder(out.Body).Decode(&lease); err != nil {\n\t\treturn nil, \"\", fmt.Errorf(\"decode lock file: %w\", err)\n\t}\n\n\tetag := \"\"\n\tif out.ETag != nil {\n\t\tetag = *out.ETag\n\t}\n\tlease.ETag = etag\n\n\treturn &lease, etag, nil\n}\n\nfunc (l *Leaser) writeLease(ctx context.Context, lease *litestream.Lease, etag string) (string, error) {\n\tkey := l.lockKey()\n\n\tdata, err := json.Marshal(lease)\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"marshal lock file: %w\", err)\n\t}\n\n\tinput := &s3.PutObjectInput{\n\t\tBucket: aws.String(l.Bucket),\n\t\tKey: aws.String(key),\n\t\tBody: bytes.NewReader(data),\n\t\tContentType: aws.String(\"application/json\"),\n\t}\n\n\tif etag == \"\" {\n\t\tinput.IfNoneMatch = aws.String(\"*\")\n\t} else {\n\t\tinput.IfMatch = aws.String(etag)\n\t}\n\n\tout, err := l.s3.PutObject(ctx, input)\n\tif err != nil {\n\t\tif isPreconditionFailed(err) {\n\t\t\treturn \"\", &litestream.LeaseExistsError{}\n\t\t}\n\t\treturn \"\", fmt.Errorf(\"put lock file: %w\", err)\n\t}\n\n\tnewETag := \"\"\n\tif out.ETag != nil {\n\t\tnewETag = *out.ETag\n\t}\n\n\treturn newETag, nil\n}\n\nfunc isPreconditionFailed(err error) bool {\n\tvar apiErr smithy.APIError\n\tif errors.As(err, &apiErr) {\n\t\tcode := apiErr.ErrorCode()\n\t\treturn code == \"PreconditionFailed\" || code == \"412\"\n\t}\n\n\tvar respErr *smithy.OperationError\n\tif errors.As(err, &respErr) {\n\t\tif httpErr, ok := respErr.Err.(interface{ HTTPStatusCode() int }); ok {\n\t\t\treturn httpErr.HTTPStatusCode() == 412\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc isNotFoundError(err error) bool {\n\tvar apiErr smithy.APIError\n\tif errors.As(err, &apiErr) {\n\t\tcode := apiErr.ErrorCode()\n\t\treturn code == \"NoSuchKey\" || code == \"NotFound\" || code == \"404\"\n\t}\n\n\tvar noSuchKey *types.NoSuchKey\n\tif errors.As(err, &noSuchKey) {\n\t\treturn true\n\t}\n\n\tvar respErr *smithy.OperationError\n\tif errors.As(err, &respErr) {\n\t\tif httpErr, ok := respErr.Err.(interface{ HTTPStatusCode() int }); ok {\n\t\t\treturn httpErr.HTTPStatusCode() == 404\n\t\t}\n\t}\n\n\treturn false\n}\n" }, { "path": "s3/leaser_test.go", "content": "package s3\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nfunc TestLeaser_AcquireLease_NewLease(t *testing.T) {\n\tvar putCalled atomic.Bool\n\tvar receivedIfNoneMatch string\n\tcurrentETag := `\"etag-1\"`\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tswitch r.Method {\n\t\tcase http.MethodGet:\n\t\t\tw.WriteHeader(http.StatusNotFound)\n\t\tcase http.MethodPut:\n\t\t\tputCalled.Store(true)\n\t\t\treceivedIfNoneMatch = r.Header.Get(\"If-None-Match\")\n\n\t\t\tbody, _ := io.ReadAll(r.Body)\n\t\t\tr.Body.Close()\n\n\t\t\tvar lease litestream.Lease\n\t\t\tif err := json.Unmarshal(body, &lease); err != nil {\n\t\t\t\tt.Errorf(\"failed to unmarshal lock file: %v\", err)\n\t\t\t}\n\n\t\t\tif lease.Generation != 1 {\n\t\t\t\tt.Errorf(\"expected generation=1, got %d\", lease.Generation)\n\t\t\t}\n\n\t\t\tw.Header().Set(\"ETag\", currentETag)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\tdefault:\n\t\t\tt.Errorf(\"unexpected method: %s\", r.Method)\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\tleaser.TTL = 10 * time.Second\n\n\tctx := context.Background()\n\tlease, err := leaser.AcquireLease(ctx)\n\tif err != nil {\n\t\tt.Fatalf(\"AcquireLease() error: %v\", err)\n\t}\n\n\tif !putCalled.Load() {\n\t\tt.Error(\"expected PUT to be called\")\n\t}\n\tif receivedIfNoneMatch != \"*\" {\n\t\tt.Errorf(\"expected If-None-Match: *, got %q\", receivedIfNoneMatch)\n\t}\n\tif lease.Generation != 1 {\n\t\tt.Errorf(\"expected generation=1, got %d\", lease.Generation)\n\t}\n\tif lease.ETag != currentETag {\n\t\tt.Errorf(\"expected ETag=%q, got %q\", currentETag, lease.ETag)\n\t}\n\tif lease.TTL() < 9*time.Second || lease.TTL() > 10*time.Second {\n\t\tt.Errorf(\"unexpected TTL: %v\", lease.TTL())\n\t}\n}\n\nfunc TestLeaser_AcquireLease_ExpiredLease(t *testing.T) {\n\toldETag := `\"old-etag\"`\n\tnewETag := `\"new-etag\"`\n\tvar receivedIfMatch string\n\n\texpiredLease := litestream.Lease{\n\t\tGeneration: 5,\n\t\tExpiresAt: time.Now().Add(-1 * time.Hour),\n\t\tOwner: \"previous-owner\",\n\t}\n\texpiredData, _ := json.Marshal(expiredLease)\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tswitch r.Method {\n\t\tcase http.MethodGet:\n\t\t\tw.Header().Set(\"ETag\", oldETag)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\tw.Write(expiredData)\n\t\tcase http.MethodPut:\n\t\t\treceivedIfMatch = r.Header.Get(\"If-Match\")\n\t\t\tw.Header().Set(\"ETag\", newETag)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\tleaser.TTL = 30 * time.Second\n\n\tctx := context.Background()\n\tlease, err := leaser.AcquireLease(ctx)\n\tif err != nil {\n\t\tt.Fatalf(\"AcquireLease() error: %v\", err)\n\t}\n\n\tif receivedIfMatch != oldETag {\n\t\tt.Errorf(\"expected If-Match=%q, got %q\", oldETag, receivedIfMatch)\n\t}\n\tif lease.Generation != 6 {\n\t\tt.Errorf(\"expected generation=6 (previous+1), got %d\", lease.Generation)\n\t}\n\tif lease.ETag != newETag {\n\t\tt.Errorf(\"expected ETag=%q, got %q\", newETag, lease.ETag)\n\t}\n}\n\nfunc TestLeaser_AcquireLease_ActiveLease(t *testing.T) {\n\tactiveLease := litestream.Lease{\n\t\tGeneration: 3,\n\t\tExpiresAt: time.Now().Add(5 * time.Minute),\n\t\tOwner: \"active-owner\",\n\t}\n\tactiveData, _ := json.Marshal(activeLease)\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tswitch r.Method {\n\t\tcase http.MethodGet:\n\t\t\tw.Header().Set(\"ETag\", `\"active-etag\"`)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\tw.Write(activeData)\n\t\tcase http.MethodPut:\n\t\t\tt.Error(\"PUT should not be called when lease is active\")\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\t_, err := leaser.AcquireLease(ctx)\n\tif err == nil {\n\t\tt.Fatal(\"expected error for active lease\")\n\t}\n\n\tvar leaseErr *litestream.LeaseExistsError\n\tif !errors.As(err, &leaseErr) {\n\t\tt.Fatalf(\"expected *LeaseExistsError, got %T: %v\", err, err)\n\t}\n\tif leaseErr.Owner != \"active-owner\" {\n\t\tt.Errorf(\"expected Owner=%q, got %q\", \"active-owner\", leaseErr.Owner)\n\t}\n\tif leaseErr.ExpiresAt.IsZero() {\n\t\tt.Error(\"expected non-zero ExpiresAt\")\n\t}\n\tif leaseErr.ExpiresAt.Before(time.Now()) {\n\t\tt.Errorf(\"expected future ExpiresAt, got %v\", leaseErr.ExpiresAt)\n\t}\n}\n\nfunc TestLeaser_AcquireLease_RaceCondition412(t *testing.T) {\n\tvar getCalls atomic.Int32\n\twinnerLease := litestream.Lease{\n\t\tGeneration: 1,\n\t\tExpiresAt: time.Now().Add(30 * time.Second),\n\t\tOwner: \"race-winner\",\n\t}\n\twinnerData, _ := json.Marshal(winnerLease)\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tswitch r.Method {\n\t\tcase http.MethodGet:\n\t\t\tcalls := getCalls.Add(1)\n\t\t\tif calls == 1 {\n\t\t\t\tw.WriteHeader(http.StatusNotFound)\n\t\t\t} else {\n\t\t\t\tw.Header().Set(\"ETag\", `\"winner-etag\"`)\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\tw.Write(winnerData)\n\t\t\t}\n\t\tcase http.MethodPut:\n\t\t\tw.WriteHeader(http.StatusPreconditionFailed)\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\t_, err := leaser.AcquireLease(ctx)\n\tif err == nil {\n\t\tt.Fatal(\"expected error for 412 response\")\n\t}\n\n\tvar leaseErr *litestream.LeaseExistsError\n\tif !errors.As(err, &leaseErr) {\n\t\tt.Fatalf(\"expected *LeaseExistsError, got %T: %v\", err, err)\n\t}\n\tif leaseErr.Owner != \"race-winner\" {\n\t\tt.Errorf(\"expected Owner=%q, got %q\", \"race-winner\", leaseErr.Owner)\n\t}\n\tif leaseErr.ExpiresAt.IsZero() {\n\t\tt.Error(\"expected non-zero ExpiresAt after re-read\")\n\t}\n}\n\nfunc TestLeaser_RenewLease(t *testing.T) {\n\toldETag := `\"old-etag\"`\n\tnewETag := `\"new-etag\"`\n\tvar receivedIfMatch string\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tif r.Method == http.MethodPut {\n\t\t\treceivedIfMatch = r.Header.Get(\"If-Match\")\n\n\t\t\tbody, _ := io.ReadAll(r.Body)\n\t\t\tr.Body.Close()\n\n\t\t\tvar lease litestream.Lease\n\t\t\tif err := json.Unmarshal(body, &lease); err != nil {\n\t\t\t\tt.Errorf(\"failed to unmarshal: %v\", err)\n\t\t\t}\n\n\t\t\tif lease.Generation != 5 {\n\t\t\t\tt.Errorf(\"expected generation=5, got %d\", lease.Generation)\n\t\t\t}\n\n\t\t\tw.Header().Set(\"ETag\", newETag)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\tleaser.TTL = 30 * time.Second\n\n\tctx := context.Background()\n\toldLease := &litestream.Lease{\n\t\tGeneration: 5,\n\t\tExpiresAt: time.Now().Add(5 * time.Second),\n\t\tOwner: \"me\",\n\t\tETag: oldETag,\n\t}\n\n\tnewLease, err := leaser.RenewLease(ctx, oldLease)\n\tif err != nil {\n\t\tt.Fatalf(\"RenewLease() error: %v\", err)\n\t}\n\n\tif receivedIfMatch != oldETag {\n\t\tt.Errorf(\"expected If-Match=%q, got %q\", oldETag, receivedIfMatch)\n\t}\n\tif newLease.Generation != 5 {\n\t\tt.Errorf(\"expected generation=5, got %d\", newLease.Generation)\n\t}\n\tif newLease.ETag != newETag {\n\t\tt.Errorf(\"expected ETag=%q, got %q\", newETag, newLease.ETag)\n\t}\n\tif newLease.TTL() < 29*time.Second {\n\t\tt.Errorf(\"expected TTL ~30s, got %v\", newLease.TTL())\n\t}\n}\n\nfunc TestLeaser_RenewLease_LostLease(t *testing.T) {\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tif r.Method == http.MethodPut {\n\t\t\tw.WriteHeader(http.StatusPreconditionFailed)\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\toldLease := &litestream.Lease{\n\t\tGeneration: 5,\n\t\tExpiresAt: time.Now().Add(5 * time.Second),\n\t\tOwner: \"me\",\n\t\tETag: `\"stale-etag\"`,\n\t}\n\n\t_, err := leaser.RenewLease(ctx, oldLease)\n\tif err != litestream.ErrLeaseNotHeld {\n\t\tt.Errorf(\"expected ErrLeaseNotHeld, got %v\", err)\n\t}\n}\n\nfunc TestLeaser_RenewLease_NilLease(t *testing.T) {\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tt.Error(\"no request should be made for nil lease\")\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\t_, err := leaser.RenewLease(ctx, nil)\n\tif !errors.Is(err, ErrLeaseRequired) {\n\t\tt.Errorf(\"expected ErrLeaseRequired for nil lease, got %v\", err)\n\t}\n}\n\nfunc TestLeaser_RenewLease_EmptyETag(t *testing.T) {\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tt.Error(\"no request should be made for empty ETag\")\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\tlease := &litestream.Lease{\n\t\tGeneration: 5,\n\t\tExpiresAt: time.Now().Add(5 * time.Second),\n\t\tOwner: \"me\",\n\t\tETag: \"\",\n\t}\n\t_, err := leaser.RenewLease(ctx, lease)\n\tif !errors.Is(err, ErrLeaseETagRequired) {\n\t\tt.Errorf(\"expected ErrLeaseETagRequired for empty ETag, got %v\", err)\n\t}\n}\n\nfunc TestLeaser_ReleaseLease(t *testing.T) {\n\tvar deleteCalled atomic.Bool\n\tvar receivedIfMatch string\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tif r.Method == http.MethodDelete {\n\t\t\tdeleteCalled.Store(true)\n\t\t\treceivedIfMatch = r.Header.Get(\"If-Match\")\n\t\t\tw.WriteHeader(http.StatusNoContent)\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\tlease := &litestream.Lease{\n\t\tGeneration: 5,\n\t\tExpiresAt: time.Now().Add(5 * time.Minute),\n\t\tOwner: \"me\",\n\t\tETag: `\"my-etag\"`,\n\t}\n\n\terr := leaser.ReleaseLease(ctx, lease)\n\tif err != nil {\n\t\tt.Fatalf(\"ReleaseLease() error: %v\", err)\n\t}\n\n\tif !deleteCalled.Load() {\n\t\tt.Error(\"expected DELETE to be called\")\n\t}\n\tif receivedIfMatch != `\"my-etag\"` {\n\t\tt.Errorf(\"expected If-Match=%q, got %q\", `\"my-etag\"`, receivedIfMatch)\n\t}\n}\n\nfunc TestLeaser_ReleaseLease_StaleETag(t *testing.T) {\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tif r.Method == http.MethodDelete {\n\t\t\tw.WriteHeader(http.StatusPreconditionFailed)\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\tlease := &litestream.Lease{\n\t\tGeneration: 5,\n\t\tExpiresAt: time.Now().Add(5 * time.Minute),\n\t\tOwner: \"me\",\n\t\tETag: `\"stale-etag\"`,\n\t}\n\n\terr := leaser.ReleaseLease(ctx, lease)\n\tif err != litestream.ErrLeaseNotHeld {\n\t\tt.Errorf(\"expected ErrLeaseNotHeld for stale ETag, got %v\", err)\n\t}\n}\n\nfunc TestLeaser_ReleaseLease_AlreadyDeleted(t *testing.T) {\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tif r.Method == http.MethodDelete {\n\t\t\tw.WriteHeader(http.StatusNotFound)\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\tlease := &litestream.Lease{\n\t\tGeneration: 5,\n\t\tExpiresAt: time.Now().Add(5 * time.Minute),\n\t\tOwner: \"me\",\n\t\tETag: `\"my-etag\"`,\n\t}\n\n\terr := leaser.ReleaseLease(ctx, lease)\n\tif !errors.Is(err, ErrLeaseAlreadyReleased) {\n\t\tt.Fatalf(\"expected ErrLeaseAlreadyReleased, got %v\", err)\n\t}\n}\n\nfunc TestLeaser_ReleaseLease_NilLease(t *testing.T) {\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tt.Error(\"no request should be made for nil lease\")\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\terr := leaser.ReleaseLease(ctx, nil)\n\tif !errors.Is(err, ErrLeaseRequired) {\n\t\tt.Errorf(\"expected ErrLeaseRequired for nil lease, got %v\", err)\n\t}\n}\n\nfunc TestLeaser_ReleaseLease_EmptyETag(t *testing.T) {\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tt.Error(\"no request should be made for empty ETag\")\n\t}))\n\tdefer server.Close()\n\n\tleaser := newTestLeaser(t, server.URL)\n\n\tctx := context.Background()\n\tlease := &litestream.Lease{\n\t\tGeneration: 5,\n\t\tExpiresAt: time.Now().Add(5 * time.Minute),\n\t\tOwner: \"me\",\n\t\tETag: \"\",\n\t}\n\terr := leaser.ReleaseLease(ctx, lease)\n\tif !errors.Is(err, ErrLeaseETagRequired) {\n\t\tt.Errorf(\"expected ErrLeaseETagRequired for empty ETag, got %v\", err)\n\t}\n}\n\nfunc TestLeaser_ConcurrentAcquisition(t *testing.T) {\n\tvar mu sync.Mutex\n\tvar leaseHolder string\n\tcurrentETag := \"\"\n\tetagCounter := 0\n\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tmu.Lock()\n\t\tdefer mu.Unlock()\n\n\t\tswitch r.Method {\n\t\tcase http.MethodGet:\n\t\t\tif leaseHolder == \"\" {\n\t\t\t\tw.WriteHeader(http.StatusNotFound)\n\t\t\t} else {\n\t\t\t\tlease := litestream.Lease{\n\t\t\t\t\tGeneration: 1,\n\t\t\t\t\tExpiresAt: time.Now().Add(30 * time.Second),\n\t\t\t\t\tOwner: leaseHolder,\n\t\t\t\t}\n\t\t\t\tdata, _ := json.Marshal(lease)\n\t\t\t\tw.Header().Set(\"ETag\", currentETag)\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\tw.Write(data)\n\t\t\t}\n\t\tcase http.MethodPut:\n\t\t\tifNoneMatch := r.Header.Get(\"If-None-Match\")\n\t\t\tifMatch := r.Header.Get(\"If-Match\")\n\n\t\t\tif ifNoneMatch == \"*\" && leaseHolder != \"\" {\n\t\t\t\tw.WriteHeader(http.StatusPreconditionFailed)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif ifMatch != \"\" && ifMatch != currentETag {\n\t\t\t\tw.WriteHeader(http.StatusPreconditionFailed)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tbody, _ := io.ReadAll(r.Body)\n\t\t\tr.Body.Close()\n\t\t\tvar lease litestream.Lease\n\t\t\tjson.Unmarshal(body, &lease)\n\n\t\t\tleaseHolder = lease.Owner\n\t\t\tetagCounter++\n\t\t\tcurrentETag = `\"etag-` + string(rune('0'+etagCounter)) + `\"`\n\n\t\t\tw.Header().Set(\"ETag\", currentETag)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}\n\t}))\n\tdefer server.Close()\n\n\tconst numClients = 10\n\tvar wg sync.WaitGroup\n\tvar successCount atomic.Int32\n\tvar failCount atomic.Int32\n\n\tfor i := 0; i < numClients; i++ {\n\t\twg.Add(1)\n\t\tgo func(id int) {\n\t\t\tdefer wg.Done()\n\n\t\t\tleaser := newTestLeaser(t, server.URL)\n\t\t\tleaser.Owner = string(rune('A' + id))\n\n\t\t\tctx := context.Background()\n\t\t\t_, err := leaser.AcquireLease(ctx)\n\t\t\tif err == nil {\n\t\t\t\tsuccessCount.Add(1)\n\t\t\t} else {\n\t\t\t\tfailCount.Add(1)\n\t\t\t}\n\t\t}(i)\n\t}\n\n\twg.Wait()\n\n\tif successCount.Load() != 1 {\n\t\tt.Errorf(\"expected exactly 1 successful acquisition, got %d\", successCount.Load())\n\t}\n\tif failCount.Load() != numClients-1 {\n\t\tt.Errorf(\"expected %d failures, got %d\", numClients-1, failCount.Load())\n\t}\n}\n\nfunc TestLeaser_LockKey(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpath string\n\t\twantKey string\n\t}{\n\t\t{\n\t\t\tname: \"EmptyPath\",\n\t\t\tpath: \"\",\n\t\t\twantKey: \"lock.json\",\n\t\t},\n\t\t{\n\t\t\tname: \"SimplePath\",\n\t\t\tpath: \"replica\",\n\t\t\twantKey: \"replica/lock.json\",\n\t\t},\n\t\t{\n\t\t\tname: \"NestedPath\",\n\t\t\tpath: \"my/db/replica\",\n\t\t\twantKey: \"my/db/replica/lock.json\",\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tleaser := NewLeaser()\n\t\t\tleaser.Path = tt.path\n\n\t\t\tif got := leaser.lockKey(); got != tt.wantKey {\n\t\t\t\tt.Errorf(\"lockKey() = %q, want %q\", got, tt.wantKey)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestLeaser_Type(t *testing.T) {\n\tleaser := NewLeaser()\n\n\tif got := leaser.Type(); got != \"s3\" {\n\t\tt.Errorf(\"Type() = %q, want %q\", got, \"s3\")\n\t}\n}\n\nfunc newTestLeaser(t *testing.T, serverURL string) *Leaser {\n\tt.Helper()\n\n\tclient := NewReplicaClient()\n\tclient.Bucket = \"test-bucket\"\n\tclient.Path = \"test-path\"\n\tclient.Region = \"us-east-1\"\n\tclient.Endpoint = serverURL\n\tclient.ForcePathStyle = true\n\tclient.AccessKeyID = \"test-access-key\"\n\tclient.SecretAccessKey = \"test-secret-key\"\n\n\tctx := context.Background()\n\tif err := client.Init(ctx); err != nil {\n\t\tt.Fatalf(\"Init() error: %v\", err)\n\t}\n\n\tleaser := NewLeaser()\n\tleaser.Bucket = client.Bucket\n\tleaser.Path = client.Path\n\tleaser.SetClient(client.s3)\n\tleaser.logger = slog.New(slog.NewTextHandler(io.Discard, nil))\n\n\treturn leaser\n}\n" }, { "path": "s3/replica_client.go", "content": "package s3\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"crypto/md5\"\n\t\"crypto/tls\"\n\t\"encoding/base64\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"os\"\n\t\"path\"\n\t\"regexp\"\n\t\"slices\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/aws/aws-sdk-go-v2/aws\"\n\tv4 \"github.com/aws/aws-sdk-go-v2/aws/signer/v4\"\n\t\"github.com/aws/aws-sdk-go-v2/config\"\n\t\"github.com/aws/aws-sdk-go-v2/credentials\"\n\t\"github.com/aws/aws-sdk-go-v2/feature/s3/manager\"\n\t\"github.com/aws/aws-sdk-go-v2/service/s3\"\n\t\"github.com/aws/aws-sdk-go-v2/service/s3/types\"\n\t\"github.com/aws/smithy-go\"\n\tsmithyxml \"github.com/aws/smithy-go/encoding/xml\"\n\t\"github.com/aws/smithy-go/middleware\"\n\tsmithytime \"github.com/aws/smithy-go/time\"\n\tsmithyhttp \"github.com/aws/smithy-go/transport/http\"\n\t\"github.com/superfly/ltx\"\n\t\"golang.org/x/sync/errgroup\"\n\t\"golang.org/x/sync/semaphore\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\nfunc init() {\n\tlitestream.RegisterReplicaClientFactory(\"s3\", NewReplicaClientFromURL)\n}\n\n// ReplicaClientType is the client type for this package.\nconst ReplicaClientType = \"s3\"\n\n// MetadataKeyTimestamp is the metadata key for storing LTX file timestamps in S3.\nconst MetadataKeyTimestamp = \"litestream-timestamp\"\n\n// MaxKeys is the number of keys S3 can operate on per batch.\nconst MaxKeys = 1000\n\n// DefaultRegion is the region used if one is not specified.\nconst DefaultRegion = \"us-east-1\"\n\n// DefaultMetadataConcurrency is the default number of concurrent HeadObject calls\n// for fetching accurate timestamps during timestamp-based restore.\n// S3 can handle 5,500+ HEAD requests per second per prefix.\nconst DefaultMetadataConcurrency = 50\n\n// DefaultR2Concurrency is the default number of concurrent multipart upload\n// parts for Cloudflare R2, which has strict concurrent upload limits.\nconst DefaultR2Concurrency = 2\n\n// contentMD5StackKey is used to pass the precomputed Content-MD5 checksum\n// through the middleware stack from Serialize to Finalize phase.\ntype contentMD5StackKey struct{}\n\nvar _ litestream.ReplicaClient = (*ReplicaClient)(nil)\nvar _ litestream.ReplicaClientV3 = (*ReplicaClient)(nil)\n\n// ReplicaClient is a client for writing LTX files to S3.\ntype ReplicaClient struct {\n\tmu sync.Mutex\n\ts3 *s3.Client // s3 service\n\tuploader *manager.Uploader\n\tlogger *slog.Logger\n\n\t// AWS authentication keys.\n\tAccessKeyID string\n\tSecretAccessKey string\n\n\t// S3 bucket information\n\tRegion string\n\tBucket string\n\tPath string\n\tEndpoint string\n\tForcePathStyle bool\n\tSkipVerify bool\n\tSignPayload bool\n\tRequireContentMD5 bool\n\n\t// Upload configuration\n\tPartSize int64 // Part size for multipart uploads (default: 5MB)\n\tConcurrency int // Number of concurrent parts to upload (default: 5)\n\n\t// MetadataConcurrency controls parallel HeadObject calls for timestamp-based restore.\n\t// Higher values improve restore speed for large backup histories.\n\t// Default: 50 (S3 can handle 5,500+ HEAD/s per prefix)\n\tMetadataConcurrency int\n\n\t// Server-Side Encryption - Customer Provided Keys (SSE-C)\n\t// Works with all S3-compatible providers (AWS, MinIO, Exoscale, etc.)\n\tSSECustomerAlgorithm string // Must be \"AES256\" if set\n\tSSECustomerKey string // Base64-encoded 256-bit (32 byte) encryption key\n\tSSECustomerKeyMD5 string // Base64-encoded MD5 of key (auto-computed if not set)\n\n\t// Server-Side Encryption - AWS KMS (SSE-KMS)\n\t// Only works with AWS S3 (not S3-compatible providers)\n\tSSEKMSKeyID string // KMS key ID, ARN, or alias\n}\n\n// NewReplicaClient returns a new instance of ReplicaClient.\nfunc NewReplicaClient() *ReplicaClient {\n\treturn &ReplicaClient{\n\t\tlogger: slog.Default().WithGroup(ReplicaClientType),\n\t\tRequireContentMD5: true,\n\t\tSignPayload: true,\n\t}\n}\n\nfunc (c *ReplicaClient) SetLogger(logger *slog.Logger) {\n\tc.logger = logger.WithGroup(ReplicaClientType)\n}\n\n// NewReplicaClientFromURL creates a new ReplicaClient from URL components.\n// This is used by the replica client factory registration.\nfunc NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error) {\n\tclient := NewReplicaClient()\n\n\tvar (\n\t\tbucket string\n\t\tregion string\n\t\tendpoint string\n\t\tforcePathStyle bool\n\t\tskipVerify bool\n\t\tsignPayload bool\n\t\tsignPayloadSet bool\n\t\trequireMD5 bool\n\t\trequireMD5Set bool\n\t\tconcurrency int\n\t\tconcurrencySet bool\n\t\tpartSize int64\n\t)\n\n\t// Parse host for bucket and region\n\tif strings.HasPrefix(host, \"arn:\") {\n\t\tbucket = host\n\t\tregion = litestream.RegionFromS3ARN(host)\n\t} else {\n\t\tbucket, region, endpoint, forcePathStyle = ParseHost(host)\n\t}\n\n\t// Override with query parameters if provided\n\tif qEndpoint := query.Get(\"endpoint\"); qEndpoint != \"\" {\n\t\t// Ensure endpoint has a scheme (defaults to https:// for cloud, http:// for local)\n\t\tqEndpoint, _ = litestream.EnsureEndpointScheme(qEndpoint)\n\t\tendpoint = qEndpoint\n\t\t// Default to path style for custom endpoints unless explicitly set to false\n\t\tif v, ok := litestream.BoolQueryValue(query, \"forcePathStyle\", \"force-path-style\"); !ok || v {\n\t\t\tforcePathStyle = true\n\t\t}\n\t}\n\tif qRegion := query.Get(\"region\"); qRegion != \"\" {\n\t\tregion = qRegion\n\t}\n\tif v, ok := litestream.BoolQueryValue(query, \"forcePathStyle\", \"force-path-style\"); ok {\n\t\tforcePathStyle = v\n\t}\n\tif v, ok := litestream.BoolQueryValue(query, \"skipVerify\", \"skip-verify\"); ok {\n\t\tskipVerify = v\n\t}\n\tif v, ok := litestream.BoolQueryValue(query, \"signPayload\", \"sign-payload\"); ok {\n\t\tsignPayload = v\n\t\tsignPayloadSet = true\n\t}\n\tif v, ok := litestream.BoolQueryValue(query, \"requireContentMD5\", \"require-content-md5\"); ok {\n\t\trequireMD5 = v\n\t\trequireMD5Set = true\n\t}\n\tif v := query.Get(\"concurrency\"); v != \"\" {\n\t\tif n, err := strconv.Atoi(v); err == nil && n > 0 {\n\t\t\tconcurrency = n\n\t\t\tconcurrencySet = true\n\t\t}\n\t}\n\tif v := query.Get(\"partSize\"); v != \"\" {\n\t\tif n, err := strconv.ParseInt(v, 10, 64); err == nil && n > 0 {\n\t\t\tpartSize = n\n\t\t}\n\t} else if v := query.Get(\"part-size\"); v != \"\" {\n\t\tif n, err := strconv.ParseInt(v, 10, 64); err == nil && n > 0 {\n\t\t\tpartSize = n\n\t\t}\n\t}\n\n\t// Ensure bucket is set\n\tif bucket == \"\" {\n\t\treturn nil, fmt.Errorf(\"bucket required for s3 replica URL\")\n\t}\n\n\t// Track if forcePathStyle was explicitly set via query parameter.\n\tforcePathStyleSet := query.Get(\"forcePathStyle\") != \"\" || query.Get(\"force-path-style\") != \"\"\n\n\t// Read authentication from environment variables\n\tif v := os.Getenv(\"AWS_ACCESS_KEY_ID\"); v != \"\" {\n\t\tclient.AccessKeyID = v\n\t} else if v := os.Getenv(\"LITESTREAM_ACCESS_KEY_ID\"); v != \"\" {\n\t\tclient.AccessKeyID = v\n\t}\n\tif v := os.Getenv(\"AWS_SECRET_ACCESS_KEY\"); v != \"\" {\n\t\tclient.SecretAccessKey = v\n\t} else if v := os.Getenv(\"LITESTREAM_SECRET_ACCESS_KEY\"); v != \"\" {\n\t\tclient.SecretAccessKey = v\n\t}\n\n\tif endpoint == \"\" {\n\t\tif v := os.Getenv(\"LITESTREAM_S3_ENDPOINT\"); v != \"\" {\n\t\t\tendpoint, _ = litestream.EnsureEndpointScheme(v)\n\t\t\tif !forcePathStyleSet {\n\t\t\t\tforcePathStyle = true\n\t\t\t}\n\t\t}\n\t}\n\n\t// Detect S3-compatible provider endpoints for applying appropriate defaults.\n\tisHetzner := litestream.IsHetznerEndpoint(endpoint)\n\tisTigris := litestream.IsTigrisEndpoint(endpoint)\n\tisDigitalOcean := litestream.IsDigitalOceanEndpoint(endpoint)\n\tisBackblaze := litestream.IsBackblazeEndpoint(endpoint)\n\tisFilebase := litestream.IsFilebaseEndpoint(endpoint)\n\tisScaleway := litestream.IsScalewayEndpoint(endpoint)\n\tisCloudflareR2 := litestream.IsCloudflareR2Endpoint(endpoint)\n\tisMinIO := litestream.IsMinIOEndpoint(endpoint)\n\tisSupabase := litestream.IsSupabaseEndpoint(endpoint)\n\n\t// Apply provider-specific defaults for S3-compatible providers.\n\tif isTigris {\n\t\t// Tigris: requires signed payloads, no MD5\n\t\tif !signPayloadSet {\n\t\t\tsignPayload, signPayloadSet = true, true\n\t\t}\n\t\tif !requireMD5Set {\n\t\t\trequireMD5, requireMD5Set = false, true\n\t\t}\n\t}\n\tif isHetzner || isDigitalOcean || isBackblaze || isFilebase || isScaleway || isCloudflareR2 || isMinIO || isSupabase {\n\t\t// All these providers require signed payloads (don't support UNSIGNED-PAYLOAD)\n\t\tif !signPayloadSet {\n\t\t\tsignPayload, signPayloadSet = true, true\n\t\t}\n\t}\n\tif !forcePathStyleSet {\n\t\t// Filebase, Backblaze B2, MinIO, and Supabase require path-style URLs\n\t\tif isFilebase || isBackblaze || isMinIO || isSupabase {\n\t\t\tforcePathStyle = true\n\t\t}\n\t}\n\tif isCloudflareR2 {\n\t\tclient.Concurrency = DefaultR2Concurrency\n\t}\n\n\t// Configure client\n\tclient.Bucket = bucket\n\tclient.Path = urlPath\n\tclient.Region = region\n\tclient.Endpoint = endpoint\n\tclient.ForcePathStyle = forcePathStyle\n\tclient.SkipVerify = skipVerify\n\n\tif signPayloadSet {\n\t\tclient.SignPayload = signPayload\n\t}\n\tif requireMD5Set {\n\t\tclient.RequireContentMD5 = requireMD5\n\t}\n\tif concurrencySet {\n\t\tclient.Concurrency = concurrency\n\t}\n\tif partSize > 0 {\n\t\tclient.PartSize = partSize\n\t}\n\n\t// Parse SSE-C parameters from query string\n\tif v := query.Get(\"sseCustomerAlgorithm\"); v != \"\" {\n\t\tclient.SSECustomerAlgorithm = v\n\t} else if v := query.Get(\"sse-customer-algorithm\"); v != \"\" {\n\t\tclient.SSECustomerAlgorithm = v\n\t}\n\tif v := query.Get(\"sseCustomerKey\"); v != \"\" {\n\t\tclient.SSECustomerKey = v\n\t} else if v := query.Get(\"sse-customer-key\"); v != \"\" {\n\t\tclient.SSECustomerKey = v\n\t}\n\tif v := query.Get(\"sseCustomerKeyMD5\"); v != \"\" {\n\t\tclient.SSECustomerKeyMD5 = v\n\t} else if v := query.Get(\"sse-customer-key-md5\"); v != \"\" {\n\t\tclient.SSECustomerKeyMD5 = v\n\t}\n\n\t// Parse SSE-KMS parameters from query string\n\tif v := query.Get(\"sseKmsKeyId\"); v != \"\" {\n\t\tclient.SSEKMSKeyID = v\n\t} else if v := query.Get(\"sse-kms-key-id\"); v != \"\" {\n\t\tclient.SSEKMSKeyID = v\n\t}\n\n\treturn client, nil\n}\n\n// Type returns \"s3\" as the client type.\nfunc (c *ReplicaClient) Type() string {\n\treturn ReplicaClientType\n}\n\n// Init initializes the connection to S3. No-op if already initialized.\nfunc (c *ReplicaClient) Init(ctx context.Context) (err error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tif c.s3 != nil {\n\t\treturn nil\n\t}\n\n\t// Validate required configuration\n\tif c.Bucket == \"\" {\n\t\treturn fmt.Errorf(\"s3: bucket name is required\")\n\t}\n\n\t// Validate SSE configuration\n\tif err := c.validateSSEConfig(); err != nil {\n\t\treturn err\n\t}\n\n\t// Look up region if not specified and no endpoint is used.\n\t// Endpoints are typically used for non-S3 object stores and do not\n\t// necessarily require a region.\n\tregion := c.Region\n\tif region == \"\" {\n\t\tif c.Endpoint == \"\" {\n\t\t\tif region, err = c.findBucketRegion(ctx, c.Bucket); err != nil {\n\t\t\t\treturn fmt.Errorf(\"s3: cannot lookup bucket region: %w\", err)\n\t\t\t}\n\t\t} else {\n\t\t\tregion = DefaultRegion // default for non-S3 object stores\n\t\t}\n\t}\n\n\t// Create HTTP client with 24 hour timeout for long-running operations\n\thttpClient := &http.Client{\n\t\tTimeout: 24 * time.Hour,\n\t}\n\n\t// Always configure custom HTTP Transport with controlled keepalive settings\n\t// to reduce idle CPU usage from default transport's aggressive keepalives.\n\t// See: https://github.com/benbjohnson/litestream/issues/992\n\thttpClient.Transport = &http.Transport{\n\t\tProxy: http.ProxyFromEnvironment,\n\t\tDialContext: (&net.Dialer{\n\t\t\tTimeout: 30 * time.Second,\n\t\t\tKeepAlive: 30 * time.Second,\n\t\t}).DialContext,\n\t\tForceAttemptHTTP2: true,\n\t\tMaxIdleConns: 100,\n\t\tIdleConnTimeout: 90 * time.Second,\n\t\tTLSHandshakeTimeout: 10 * time.Second,\n\t\tExpectContinueTimeout: 1 * time.Second,\n\t}\n\n\t// Configure TLS to skip verification if requested\n\tif c.SkipVerify {\n\t\thttpClient.Transport.(*http.Transport).TLSClientConfig = &tls.Config{\n\t\t\tInsecureSkipVerify: true,\n\t\t}\n\t}\n\n\t// Build configuration options\n\tconfigOpts := []func(*config.LoadOptions) error{\n\t\tconfig.WithRegion(region),\n\t\t// Use adaptive retry mode for better resilience with 24 hour timeout\n\t\t// This matches Azure's approach for long-running operations\n\t\tconfig.WithRetryMode(aws.RetryModeAdaptive),\n\t\tconfig.WithRetryMaxAttempts(10), // Increase retry attempts for resilience\n\t}\n\n\t// Add HTTP client with proper timeout\n\tconfigOpts = append(configOpts, config.WithHTTPClient(httpClient))\n\n\t// Add static credentials if provided, otherwise use default credential chain\n\t// Default credential chain includes:\n\t// - Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)\n\t// - Shared credentials file (~/.aws/credentials)\n\t// - EC2 Instance Profile credentials\n\t// - ECS Task Role credentials\n\t// - Web Identity Token credentials (for EKS)\n\tif c.AccessKeyID != \"\" && c.SecretAccessKey != \"\" {\n\t\tconfigOpts = append(configOpts, config.WithCredentialsProvider(\n\t\t\tcredentials.NewStaticCredentialsProvider(c.AccessKeyID, c.SecretAccessKey, \"\"),\n\t\t))\n\t}\n\n\t// Enable AWS SDK debug logging if LITESTREAM_S3_DEBUG is set.\n\t// Useful for debugging S3-compatible providers (signing issues, request/response bodies).\n\t// Supports comma-separated values: signing,request,retries\n\t// Values: signing, request, request-with-body, response, response-with-body, retries, all\n\tif logMode := parseS3DebugEnv(); logMode != 0 {\n\t\tconfigOpts = append(configOpts, config.WithClientLogMode(logMode))\n\t}\n\n\t// Load AWS configuration\n\tcfg, err := config.LoadDefaultConfig(ctx, configOpts...)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"s3: cannot load aws config: %w\", err)\n\t}\n\n\t// Create S3 client options\n\ts3Opts := []func(*s3.Options){\n\t\tfunc(o *s3.Options) {\n\t\t\to.UsePathStyle = c.ForcePathStyle\n\t\t\to.UseARNRegion = true\n\t\t\t// Add User-Agent and optional middleware.\n\t\t\to.APIOptions = append(o.APIOptions, c.middlewareOption())\n\t\t},\n\t}\n\n\t// S3-compatible providers (Tigris, Backblaze B2, MinIO, Filebase, etc.) don't\n\t// support aws-chunked content encoding used by default checksum calculation\n\t// in AWS SDK Go v2 v1.73.0+. Disable automatic checksum calculation and\n\t// response checksum validation for all custom endpoints.\n\t// See: https://github.com/benbjohnson/litestream/issues/918\n\t// See: https://github.com/benbjohnson/litestream/issues/947\n\tif c.Endpoint != \"\" {\n\t\ts3Opts = append(s3Opts, func(o *s3.Options) {\n\t\t\to.RequestChecksumCalculation = aws.RequestChecksumCalculationWhenRequired\n\t\t\to.ResponseChecksumValidation = aws.ResponseChecksumValidationWhenRequired\n\t\t})\n\t}\n\n\t// Add custom endpoint if specified\n\tc.configureEndpoint(&s3Opts)\n\n\t// Create S3 client\n\tc.s3 = s3.NewFromConfig(cfg, s3Opts...)\n\n\t// Configure uploader with custom options if specified\n\tuploaderOpts := []func(*manager.Uploader){}\n\n\t// For S3-compatible providers, disable automatic checksum calculation on the Uploader.\n\t// The S3 client's RequestChecksumCalculation setting only affects single-part uploads.\n\t// Multipart uploads via the Uploader require this separate setting (added in s3/manager v1.20.0).\n\t// See: https://github.com/benbjohnson/litestream/issues/948\n\t// See: https://github.com/aws/aws-sdk-go-v2/issues/3007\n\tif c.Endpoint != \"\" {\n\t\tuploaderOpts = append(uploaderOpts, func(u *manager.Uploader) {\n\t\t\tu.RequestChecksumCalculation = aws.RequestChecksumCalculationWhenRequired\n\t\t})\n\t}\n\n\tif c.PartSize > 0 {\n\t\tuploaderOpts = append(uploaderOpts, func(u *manager.Uploader) {\n\t\t\tu.PartSize = c.PartSize\n\t\t})\n\t}\n\tif c.Concurrency > 0 {\n\t\tuploaderOpts = append(uploaderOpts, func(u *manager.Uploader) {\n\t\t\tu.Concurrency = c.Concurrency\n\t\t})\n\t}\n\tc.uploader = manager.NewUploader(c.s3, uploaderOpts...)\n\n\treturn nil\n}\n\n// configureEndpoint adds custom endpoint configuration to S3 client options if needed.\nfunc (c *ReplicaClient) configureEndpoint(opts *[]func(*s3.Options)) {\n\tif c.Endpoint != \"\" {\n\t\t*opts = append(*opts, func(o *s3.Options) {\n\t\t\to.UsePathStyle = c.ForcePathStyle\n\n\t\t\tendpoint := c.Endpoint\n\t\t\t// Add scheme if not present\n\t\t\tif !strings.HasPrefix(endpoint, \"http://\") && !strings.HasPrefix(endpoint, \"https://\") {\n\t\t\t\tendpoint = \"https://\" + endpoint\n\t\t\t}\n\n\t\t\to.BaseEndpoint = aws.String(endpoint)\n\t\t\t// For MinIO and other S3-compatible services\n\t\t\tif strings.HasPrefix(endpoint, \"http://\") {\n\t\t\t\to.EndpointOptions.DisableHTTPS = true\n\t\t\t}\n\t\t})\n\t}\n}\n\n// validateSSEConfig validates server-side encryption configuration.\nfunc (c *ReplicaClient) validateSSEConfig() error {\n\t// Check mutual exclusivity: SSE-C and SSE-KMS cannot both be set\n\tif c.SSECustomerKey != \"\" && c.SSEKMSKeyID != \"\" {\n\t\treturn fmt.Errorf(\"s3: cannot use both sse-customer-key and sse-kms-key-id; they are mutually exclusive\")\n\t}\n\n\t// Validate SSE-C configuration\n\tif c.SSECustomerKey != \"\" {\n\t\t// Algorithm must be AES256 (or default to it)\n\t\tif c.SSECustomerAlgorithm == \"\" {\n\t\t\tc.SSECustomerAlgorithm = \"AES256\"\n\t\t} else if c.SSECustomerAlgorithm != \"AES256\" {\n\t\t\treturn fmt.Errorf(\"s3: sse-customer-algorithm must be AES256, got %q\", c.SSECustomerAlgorithm)\n\t\t}\n\n\t\t// Validate key is valid base64 and correct length (256 bits = 32 bytes)\n\t\tkeyBytes, err := base64.StdEncoding.DecodeString(c.SSECustomerKey)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"s3: sse-customer-key must be valid base64: %w\", err)\n\t\t}\n\t\tif len(keyBytes) != 32 {\n\t\t\treturn fmt.Errorf(\"s3: sse-customer-key must be 256-bit (32 bytes) when decoded, got %d bytes\", len(keyBytes))\n\t\t}\n\n\t\t// Auto-compute MD5 if not provided\n\t\tif c.SSECustomerKeyMD5 == \"\" {\n\t\t\tsum := md5.Sum(keyBytes)\n\t\t\tc.SSECustomerKeyMD5 = base64.StdEncoding.EncodeToString(sum[:])\n\t\t}\n\n\t\t// SSE-C requires HTTPS (except for localhost/private networks for testing)\n\t\tif c.Endpoint != \"\" {\n\t\t\tendpoint := c.Endpoint\n\t\t\tif !strings.HasPrefix(endpoint, \"http://\") && !strings.HasPrefix(endpoint, \"https://\") {\n\t\t\t\tendpoint = \"https://\" + endpoint\n\t\t\t}\n\t\t\tif strings.HasPrefix(endpoint, \"http://\") {\n\t\t\t\tu, err := url.Parse(endpoint)\n\t\t\t\tif err == nil {\n\t\t\t\t\thost := u.Hostname()\n\t\t\t\t\t// Allow localhost by name\n\t\t\t\t\tif host == \"localhost\" {\n\t\t\t\t\t\t// OK - localhost is allowed\n\t\t\t\t\t} else if ip := net.ParseIP(host); ip != nil && (ip.IsLoopback() || ip.IsPrivate()) {\n\t\t\t\t\t\t// OK - loopback (127.x.x.x) or private RFC1918 ranges (10.x, 172.16-31.x, 192.168.x)\n\t\t\t\t\t} else {\n\t\t\t\t\t\treturn fmt.Errorf(\"s3: sse-customer-key requires HTTPS endpoint (HTTP only allowed for localhost/private networks)\")\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// findBucketRegion looks up the AWS region for a bucket. Returns blank if non-S3.\nfunc (c *ReplicaClient) findBucketRegion(ctx context.Context, bucket string) (string, error) {\n\t// Build a config with credentials but no region\n\tconfigOpts := []func(*config.LoadOptions) error{}\n\n\t// Add static credentials if provided\n\tif c.AccessKeyID != \"\" && c.SecretAccessKey != \"\" {\n\t\tconfigOpts = append(configOpts, config.WithCredentialsProvider(\n\t\t\tcredentials.NewStaticCredentialsProvider(c.AccessKeyID, c.SecretAccessKey, \"\"),\n\t\t))\n\t}\n\n\t// Load AWS configuration\n\tcfg, err := config.LoadDefaultConfig(ctx, configOpts...)\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"s3: cannot load aws config for region lookup: %w\", err)\n\t}\n\n\t// Use default region for initial region lookup\n\tcfg.Region = DefaultRegion\n\n\t// Create S3 client options\n\ts3Opts := []func(*s3.Options){}\n\n\t// Configure custom endpoint for region lookup\n\tc.configureEndpoint(&s3Opts)\n\n\tclient := s3.NewFromConfig(cfg, s3Opts...)\n\n\t// Get bucket location\n\tout, err := client.GetBucketLocation(ctx, &s3.GetBucketLocationInput{\n\t\tBucket: aws.String(bucket),\n\t})\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// Convert location constraint to region\n\tif out.LocationConstraint == \"\" {\n\t\treturn DefaultRegion, nil\n\t}\n\treturn string(out.LocationConstraint), nil\n}\n\n// LTXFiles returns an iterator over all LTX files on the replica for the given level.\n// When useMetadata is true, fetches accurate timestamps from S3 metadata via HeadObject.\n// This uses parallel batched requests (controlled by MetadataConcurrency) to avoid hangs\n// with large backup histories (see issue #930).\n// When false, uses fast LastModified timestamps from LIST operation.\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\treturn newFileIterator(ctx, c, level, seek, useMetadata), nil\n}\n\n// OpenLTXFile returns a reader for an LTX file\n// Returns os.ErrNotExist if no matching index/offset is found.\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar rangeStr string\n\tif size > 0 {\n\t\trangeStr = fmt.Sprintf(\"bytes=%d-%d\", offset, offset+size-1)\n\t} else {\n\t\trangeStr = fmt.Sprintf(\"bytes=%d-\", offset)\n\t}\n\n\t// Build the key from the file info\n\tfilename := ltx.FormatFilename(minTXID, maxTXID)\n\tkey := c.Path + \"/\" + fmt.Sprintf(\"%04x/%s\", level, filename)\n\n\tinput := &s3.GetObjectInput{\n\t\tBucket: aws.String(c.Bucket),\n\t\tKey: aws.String(key),\n\t\tRange: aws.String(rangeStr),\n\t}\n\n\t// Add SSE-C parameters if configured (required for reading SSE-C encrypted objects)\n\t// Note: SSE-KMS does not require parameters on read - decryption is automatic\n\tif c.SSECustomerKey != \"\" {\n\t\tinput.SSECustomerAlgorithm = aws.String(c.SSECustomerAlgorithm)\n\t\tinput.SSECustomerKey = aws.String(c.SSECustomerKey)\n\t\tinput.SSECustomerKeyMD5 = aws.String(c.SSECustomerKeyMD5)\n\t}\n\n\tout, err := c.s3.GetObject(ctx, input)\n\tif err != nil {\n\t\tif isNotExists(err) {\n\t\t\treturn nil, os.ErrNotExist\n\t\t}\n\t\treturn nil, fmt.Errorf(\"s3: get object %s: %w\", key, err)\n\t}\n\treturn out.Body, nil\n}\n\n// WriteLTXFile writes an LTX file to the replica.\n// Extracts timestamp from LTX header and stores it in S3 metadata to preserve original creation time.\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Use TeeReader to peek at LTX header while preserving data for upload\n\tvar buf bytes.Buffer\n\tteeReader := io.TeeReader(r, &buf)\n\n\t// Extract timestamp from LTX header\n\thdr, _, err := ltx.PeekHeader(teeReader)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"extract timestamp from LTX header: %w\", err)\n\t}\n\ttimestamp := time.UnixMilli(hdr.Timestamp).UTC()\n\n\t// Combine buffered data with rest of reader\n\trc := internal.NewReadCounter(io.MultiReader(&buf, r))\n\n\tfilename := ltx.FormatFilename(minTXID, maxTXID)\n\tkey := c.Path + \"/\" + fmt.Sprintf(\"%04x/%s\", level, filename)\n\n\t// Store timestamp in S3 metadata for accurate timestamp retrieval\n\tmetadata := map[string]string{\n\t\tMetadataKeyTimestamp: timestamp.Format(time.RFC3339Nano),\n\t}\n\n\tinput := &s3.PutObjectInput{\n\t\tBucket: aws.String(c.Bucket),\n\t\tKey: aws.String(key),\n\t\tBody: rc,\n\t\tMetadata: metadata,\n\t}\n\n\t// Add SSE-C parameters if configured\n\tif c.SSECustomerKey != \"\" {\n\t\tinput.SSECustomerAlgorithm = aws.String(c.SSECustomerAlgorithm)\n\t\tinput.SSECustomerKey = aws.String(c.SSECustomerKey)\n\t\tinput.SSECustomerKeyMD5 = aws.String(c.SSECustomerKeyMD5)\n\t}\n\n\t// Add SSE-KMS parameters if configured\n\tif c.SSEKMSKeyID != \"\" {\n\t\tinput.ServerSideEncryption = types.ServerSideEncryptionAwsKms\n\t\tinput.SSEKMSKeyId = aws.String(c.SSEKMSKeyID)\n\t}\n\n\tout, err := c.uploader.Upload(ctx, input)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"s3: upload to %s: %w\", key, err)\n\t}\n\n\t// Build file info from the uploaded file\n\tinfo := <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tSize: rc.N(),\n\t\tCreatedAt: timestamp,\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Add(float64(rc.N()))\n\n\t// ETag indicates successful upload\n\tif out.ETag == nil {\n\t\treturn nil, fmt.Errorf(\"s3: upload failed: no ETag returned\")\n\t}\n\n\treturn info, nil\n}\n\nfunc (c *ReplicaClient) middlewareOption() func(*middleware.Stack) error {\n\treturn func(stack *middleware.Stack) error {\n\t\tif c.RequireContentMD5 {\n\t\t\tif err := stack.Serialize.Add(\n\t\t\t\tmiddleware.SerializeMiddlewareFunc(\n\t\t\t\t\t\"LitestreamComputeDeleteContentMD5\",\n\t\t\t\t\tfunc(ctx context.Context, in middleware.SerializeInput, next middleware.SerializeHandler) (\n\t\t\t\t\t\tout middleware.SerializeOutput, metadata middleware.Metadata, err error,\n\t\t\t\t\t) {\n\t\t\t\t\t\tif middleware.GetOperationName(ctx) != \"DeleteObjects\" {\n\t\t\t\t\t\t\treturn next.HandleSerialize(ctx, in)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tinput, ok := in.Parameters.(*s3.DeleteObjectsInput)\n\t\t\t\t\t\tif !ok || input == nil || input.Delete == nil || len(input.Delete.Objects) == 0 {\n\t\t\t\t\t\t\treturn next.HandleSerialize(ctx, in)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tchecksum, err := computeDeleteObjectsContentMD5(input.Delete)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\treturn out, metadata, err\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif checksum != \"\" {\n\t\t\t\t\t\t\tctx = middleware.WithStackValue(ctx, contentMD5StackKey{}, checksum)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn next.HandleSerialize(ctx, in)\n\t\t\t\t\t},\n\t\t\t\t),\n\t\t\t\tmiddleware.Before,\n\t\t\t); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tif err := stack.Build.Add(\n\t\t\tmiddleware.BuildMiddlewareFunc(\n\t\t\t\t\"LitestreamUserAgent\",\n\t\t\t\tfunc(ctx context.Context, in middleware.BuildInput, next middleware.BuildHandler) (\n\t\t\t\t\tout middleware.BuildOutput, metadata middleware.Metadata, err error,\n\t\t\t\t) {\n\t\t\t\t\tif req, ok := in.Request.(*smithyhttp.Request); ok {\n\t\t\t\t\t\tcurrent := req.Header.Get(\"User-Agent\")\n\t\t\t\t\t\tif current == \"\" {\n\t\t\t\t\t\t\treq.Header.Set(\"User-Agent\", \"litestream\")\n\t\t\t\t\t\t} else if !strings.Contains(current, \"litestream\") {\n\t\t\t\t\t\t\treq.Header.Set(\"User-Agent\", \"litestream \"+current)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\treturn next.HandleBuild(ctx, in)\n\t\t\t\t},\n\t\t\t),\n\t\t\tmiddleware.After,\n\t\t); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif litestream.IsTigrisEndpoint(c.Endpoint) {\n\t\t\tif err := stack.Build.Add(\n\t\t\t\tmiddleware.BuildMiddlewareFunc(\n\t\t\t\t\t\"LitestreamTigrisConsistent\",\n\t\t\t\t\tfunc(ctx context.Context, in middleware.BuildInput, next middleware.BuildHandler) (\n\t\t\t\t\t\tout middleware.BuildOutput, metadata middleware.Metadata, err error,\n\t\t\t\t\t) {\n\t\t\t\t\t\tif req, ok := in.Request.(*smithyhttp.Request); ok {\n\t\t\t\t\t\t\treq.Header.Set(\"X-Tigris-Consistent\", \"true\")\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn next.HandleBuild(ctx, in)\n\t\t\t\t\t},\n\t\t\t\t),\n\t\t\t\tmiddleware.After,\n\t\t\t); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\t// Many S3-compatible providers (e.g. Filebase) do not support SigV4\n\t\t// payload hashing. Switching to unsigned payload matches the behavior\n\t\t// of the AWS SDK v1 client used in Litestream v0.3.x and restores\n\t\t// compatibility.\n\t\tif !c.SignPayload {\n\t\t\t_ = v4.RemoveComputePayloadSHA256Middleware(stack)\n\t\t\tif err := v4.AddUnsignedPayloadMiddleware(stack); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t_ = v4.RemoveContentSHA256HeaderMiddleware(stack)\n\t\t\tif err := v4.AddContentSHA256HeaderMiddleware(stack); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\t// Disable AWS SDK v2's trailing checksum middleware which uses\n\t\t// aws-chunked encoding. This is required for:\n\t\t// 1. UNSIGNED-PAYLOAD requests (aws-chunked + UNSIGNED-PAYLOAD is rejected by AWS)\n\t\t// 2. S3-compatible providers (Filebase, MinIO, Backblaze B2, etc.) that don't\n\t\t// support aws-chunked encoding at all\n\t\t// See: https://github.com/aws/aws-sdk-go-v2/discussions/2960\n\t\t// See: https://github.com/benbjohnson/litestream/issues/895\n\t\tif !c.SignPayload || c.Endpoint != \"\" {\n\t\t\tstack.Finalize.Remove(\"addInputChecksumTrailer\")\n\t\t}\n\n\t\t// Add debug logging middleware at the end of Finalize phase\n\t\t// so all headers (including X-Tigris-Consistent) are set\n\t\tif err := stack.Finalize.Add(\n\t\t\tmiddleware.FinalizeMiddlewareFunc(\n\t\t\t\t\"LitestreamDebugLogging\",\n\t\t\t\tfunc(ctx context.Context, in middleware.FinalizeInput, next middleware.FinalizeHandler) (\n\t\t\t\t\tout middleware.FinalizeOutput, metadata middleware.Metadata, err error,\n\t\t\t\t) {\n\t\t\t\t\tif req, ok := in.Request.(*smithyhttp.Request); ok {\n\t\t\t\t\t\tc.logger.Debug(\"s3 request\",\n\t\t\t\t\t\t\t\"method\", req.Method,\n\t\t\t\t\t\t\t\"url\", req.URL.String(),\n\t\t\t\t\t\t\t\"x-tigris-consistent\", req.Header.Get(\"X-Tigris-Consistent\"),\n\t\t\t\t\t\t)\n\t\t\t\t\t}\n\t\t\t\t\treturn next.HandleFinalize(ctx, in)\n\t\t\t\t},\n\t\t\t),\n\t\t\tmiddleware.After,\n\t\t); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !c.RequireContentMD5 {\n\t\t\treturn nil\n\t\t}\n\n\t\tmd5Middleware := func() middleware.FinalizeMiddleware {\n\t\t\treturn middleware.FinalizeMiddlewareFunc(\n\t\t\t\t\"LitestreamDeleteContentMD5\",\n\t\t\t\tfunc(ctx context.Context, in middleware.FinalizeInput, next middleware.FinalizeHandler) (\n\t\t\t\t\tout middleware.FinalizeOutput, metadata middleware.Metadata, err error,\n\t\t\t\t) {\n\t\t\t\t\tif middleware.GetOperationName(ctx) != \"DeleteObjects\" {\n\t\t\t\t\t\treturn next.HandleFinalize(ctx, in)\n\t\t\t\t\t}\n\n\t\t\t\t\tchecksum, _ := middleware.GetStackValue(ctx, contentMD5StackKey{}).(string)\n\t\t\t\t\tif checksum == \"\" {\n\t\t\t\t\t\treturn next.HandleFinalize(ctx, in)\n\t\t\t\t\t}\n\n\t\t\t\t\treq, ok := in.Request.(*smithyhttp.Request)\n\t\t\t\t\tif !ok {\n\t\t\t\t\t\treturn next.HandleFinalize(ctx, in)\n\t\t\t\t\t}\n\t\t\t\t\tif req.Header.Get(\"Content-MD5\") == \"\" {\n\t\t\t\t\t\treq.Header.Set(\"Content-MD5\", checksum)\n\t\t\t\t\t}\n\n\t\t\t\t\treturn next.HandleFinalize(ctx, in)\n\t\t\t\t},\n\t\t\t)\n\t\t}\n\n\t\t// Try to insert before AWS's checksum middleware for optimal ordering.\n\t\t// If that middleware doesn't exist (e.g., different SDK version), add at the end.\n\t\t// Our middleware checks if Content-MD5 is already set, so order is not critical.\n\t\tif err := stack.Finalize.Insert(md5Middleware(), \"AWSChecksum:ComputeInputPayloadChecksum\", middleware.Before); err != nil {\n\t\t\tif err := stack.Finalize.Add(md5Middleware(), middleware.After); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t}\n}\n\nfunc computeDeleteObjectsContentMD5(deleteInput *types.Delete) (string, error) {\n\tif deleteInput == nil {\n\t\treturn \"\", nil\n\t}\n\n\tpayload, err := marshalDeleteObjects(deleteInput)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tif len(payload) == 0 {\n\t\treturn \"\", nil\n\t}\n\n\tsum := md5.Sum(payload)\n\treturn base64.StdEncoding.EncodeToString(sum[:]), nil\n}\n\nfunc marshalDeleteObjects(deleteInput *types.Delete) ([]byte, error) {\n\tif deleteInput == nil {\n\t\treturn nil, nil\n\t}\n\n\tvar buf bytes.Buffer\n\tencoder := smithyxml.NewEncoder(&buf)\n\troot := smithyxml.StartElement{\n\t\tName: smithyxml.Name{\n\t\t\tLocal: \"Delete\",\n\t\t},\n\t\tAttr: []smithyxml.Attr{\n\t\t\tsmithyxml.NewNamespaceAttribute(\"\", \"http://s3.amazonaws.com/doc/2006-03-01/\"),\n\t\t},\n\t}\n\n\tif err := encodeDeleteDocument(deleteInput, encoder.RootElement(root)); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn encoder.Bytes(), nil\n}\n\nfunc encodeDeleteDocument(v *types.Delete, value smithyxml.Value) error {\n\tdefer value.Close()\n\tif v.Objects != nil {\n\t\troot := smithyxml.StartElement{\n\t\t\tName: smithyxml.Name{\n\t\t\t\tLocal: \"Object\",\n\t\t\t},\n\t\t}\n\t\tel := value.FlattenedElement(root)\n\t\tif err := encodeObjectIdentifierList(v.Objects, el); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\tif v.Quiet != nil {\n\t\troot := smithyxml.StartElement{\n\t\t\tName: smithyxml.Name{\n\t\t\t\tLocal: \"Quiet\",\n\t\t\t},\n\t\t}\n\t\tel := value.MemberElement(root)\n\t\tel.Boolean(*v.Quiet)\n\t}\n\treturn nil\n}\n\nfunc encodeObjectIdentifierList(v []types.ObjectIdentifier, value smithyxml.Value) error {\n\tif !value.IsFlattened() {\n\t\tdefer value.Close()\n\t}\n\n\tarray := value.Array()\n\tfor i := range v {\n\t\tmember := array.Member()\n\t\tif err := encodeObjectIdentifier(&v[i], member); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\n// encodeObjectIdentifier mirrors the AWS SDK's XML serializer for DeleteObjects.\n// This ensures our precomputed Content-MD5 matches the actual request body.\n// Includes all ObjectIdentifier fields as of AWS SDK v2 (2024).\nfunc encodeObjectIdentifier(v *types.ObjectIdentifier, value smithyxml.Value) error {\n\tdefer value.Close()\n\tif v.ETag != nil {\n\t\tel := value.MemberElement(smithyxml.StartElement{\n\t\t\tName: smithyxml.Name{\n\t\t\t\tLocal: \"ETag\",\n\t\t\t},\n\t\t})\n\t\tel.String(*v.ETag)\n\t}\n\tif v.Key != nil {\n\t\tel := value.MemberElement(smithyxml.StartElement{\n\t\t\tName: smithyxml.Name{\n\t\t\t\tLocal: \"Key\",\n\t\t\t},\n\t\t})\n\t\tel.String(*v.Key)\n\t}\n\tif v.LastModifiedTime != nil {\n\t\tel := value.MemberElement(smithyxml.StartElement{\n\t\t\tName: smithyxml.Name{\n\t\t\t\tLocal: \"LastModifiedTime\",\n\t\t\t},\n\t\t})\n\t\tel.String(smithytime.FormatHTTPDate(*v.LastModifiedTime))\n\t}\n\tif v.Size != nil {\n\t\tel := value.MemberElement(smithyxml.StartElement{\n\t\t\tName: smithyxml.Name{\n\t\t\t\tLocal: \"Size\",\n\t\t\t},\n\t\t})\n\t\tel.Long(*v.Size)\n\t}\n\tif v.VersionId != nil {\n\t\tel := value.MemberElement(smithyxml.StartElement{\n\t\t\tName: smithyxml.Name{\n\t\t\t\tLocal: \"VersionId\",\n\t\t\t},\n\t\t})\n\t\tel.String(*v.VersionId)\n\t}\n\treturn nil\n}\n\n// DeleteLTXFiles deletes one or more LTX files.\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\tif len(a) == 0 {\n\t\treturn nil\n\t}\n\n\t// Convert file infos to object identifiers\n\tobjIDs := make([]types.ObjectIdentifier, 0, len(a))\n\tfor _, info := range a {\n\t\tfilename := ltx.FormatFilename(info.MinTXID, info.MaxTXID)\n\t\tkey := c.Path + \"/\" + fmt.Sprintf(\"%04x/%s\", info.Level, filename)\n\t\tobjIDs = append(objIDs, types.ObjectIdentifier{Key: aws.String(key)})\n\n\t\tc.logger.Debug(\"deleting ltx file\", \"level\", info.Level, \"minTXID\", info.MinTXID, \"maxTXID\", info.MaxTXID, \"key\", key)\n\t}\n\n\t// Delete in batches\n\tfor len(objIDs) > 0 {\n\t\tn := min(len(objIDs), MaxKeys)\n\n\t\tc.logger.Debug(\"deleting ltx files batch\", \"count\", n)\n\n\t\tstart := time.Now()\n\t\tout, err := c.s3.DeleteObjects(ctx, &s3.DeleteObjectsInput{\n\t\t\tBucket: aws.String(c.Bucket),\n\t\t\tDelete: &types.Delete{Objects: objIDs[:n]},\n\t\t})\n\t\tduration := time.Since(start)\n\t\tinternal.OperationDurationHistogramVec.WithLabelValues(ReplicaClientType, \"DELETE\").Observe(duration.Seconds())\n\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"s3: delete batch of %d objects: %w\", n, err)\n\t\t}\n\n\t\tdeleted := 0\n\t\tif out != nil {\n\t\t\tdeleted = len(out.Deleted)\n\t\t}\n\t\tc.logger.Debug(\"delete batch completed\",\n\t\t\t\"requested\", n,\n\t\t\t\"deleted\", deleted,\n\t\t\t\"errors\", len(out.Errors),\n\t\t\t\"duration_ms\", duration.Milliseconds())\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Add(float64(deleted))\n\n\t\tif len(out.Errors) > 0 {\n\t\t\tfor i, e := range out.Errors {\n\t\t\t\tcode := aws.ToString(e.Code)\n\t\t\t\tinternal.OperationErrorCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\", code).Inc()\n\n\t\t\t\tif i < 5 {\n\t\t\t\t\tc.logger.Warn(\"delete object failed\",\n\t\t\t\t\t\t\"key\", aws.ToString(e.Key),\n\t\t\t\t\t\t\"code\", code,\n\t\t\t\t\t\t\"message\", aws.ToString(e.Message))\n\t\t\t\t}\n\t\t\t}\n\t\t\tif len(out.Errors) > 5 {\n\t\t\t\tc.logger.Warn(\"additional delete errors suppressed\", \"count\", len(out.Errors)-5)\n\t\t\t}\n\t\t}\n\n\t\tif err := deleteOutputError(out); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tobjIDs = objIDs[n:]\n\t}\n\n\treturn nil\n}\n\n// DeleteAll deletes all files.\nfunc (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\tvar objIDs []types.ObjectIdentifier\n\n\t// Create paginator for listing objects\n\tpaginator := s3.NewListObjectsV2Paginator(c.s3, &s3.ListObjectsV2Input{\n\t\tBucket: aws.String(c.Bucket),\n\t\tPrefix: aws.String(c.Path + \"/\"),\n\t})\n\n\t// Iterate through all pages\n\tfor paginator.HasMorePages() {\n\t\tpage, err := paginator.NextPage(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"s3: list objects page: %w\", err)\n\t\t}\n\n\t\t// Collect object identifiers\n\t\tfor _, obj := range page.Contents {\n\t\t\tobjIDs = append(objIDs, types.ObjectIdentifier{Key: obj.Key})\n\t\t}\n\t}\n\n\t// Delete all collected objects in batches\n\tfor len(objIDs) > 0 {\n\t\tn := min(len(objIDs), MaxKeys)\n\n\t\tout, err := c.s3.DeleteObjects(ctx, &s3.DeleteObjectsInput{\n\t\t\tBucket: aws.String(c.Bucket),\n\t\t\tDelete: &types.Delete{Objects: objIDs[:n], Quiet: aws.Bool(true)},\n\t\t})\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"s3: delete all batch of %d objects: %w\", n, err)\n\t\t} else if err := deleteOutputError(out); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tobjIDs = objIDs[n:]\n\t}\n\n\treturn nil\n}\n\n// GenerationsV3 returns a list of v0.3.x generation IDs in the replica.\nfunc (c *ReplicaClient) GenerationsV3(ctx context.Context) ([]string, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tprefix := litestream.GenerationsPathV3(c.Path) + \"/\"\n\n\t// Use CommonPrefixes with delimiter to list \"directories\"\n\tpaginator := s3.NewListObjectsV2Paginator(c.s3, &s3.ListObjectsV2Input{\n\t\tBucket: aws.String(c.Bucket),\n\t\tPrefix: aws.String(prefix),\n\t\tDelimiter: aws.String(\"/\"),\n\t})\n\n\tvar generations []string\n\tfor paginator.HasMorePages() {\n\t\tpage, err := paginator.NextPage(ctx)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"s3: list generations: %w\", err)\n\t\t}\n\n\t\tfor _, cp := range page.CommonPrefixes {\n\t\t\t// Extract generation ID from prefix (e.g., \"path/generations/abc123def456/\" -> \"abc123def456\")\n\t\t\tp := aws.ToString(cp.Prefix)\n\t\t\tp = strings.TrimPrefix(p, prefix)\n\t\t\tp = strings.TrimSuffix(p, \"/\")\n\t\t\tif litestream.IsGenerationIDV3(p) {\n\t\t\t\tgenerations = append(generations, p)\n\t\t\t}\n\t\t}\n\t}\n\n\tslices.Sort(generations)\n\treturn generations, nil\n}\n\n// SnapshotsV3 returns snapshots for a generation, sorted by index.\nfunc (c *ReplicaClient) SnapshotsV3(ctx context.Context, generation string) ([]litestream.SnapshotInfoV3, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tprefix := litestream.SnapshotsPathV3(c.Path, generation) + \"/\"\n\n\tpaginator := s3.NewListObjectsV2Paginator(c.s3, &s3.ListObjectsV2Input{\n\t\tBucket: aws.String(c.Bucket),\n\t\tPrefix: aws.String(prefix),\n\t})\n\n\tvar snapshots []litestream.SnapshotInfoV3\n\tfor paginator.HasMorePages() {\n\t\tpage, err := paginator.NextPage(ctx)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"s3: list snapshots: %w\", err)\n\t\t}\n\n\t\tfor _, obj := range page.Contents {\n\t\t\tkey := path.Base(aws.ToString(obj.Key))\n\t\t\tindex, err := litestream.ParseSnapshotFilenameV3(key)\n\t\t\tif err != nil {\n\t\t\t\tcontinue // skip invalid filenames\n\t\t\t}\n\n\t\t\tsnapshots = append(snapshots, litestream.SnapshotInfoV3{\n\t\t\t\tGeneration: generation,\n\t\t\t\tIndex: index,\n\t\t\t\tSize: aws.ToInt64(obj.Size),\n\t\t\t\tCreatedAt: aws.ToTime(obj.LastModified).UTC(),\n\t\t\t})\n\t\t}\n\t}\n\n\tslices.SortFunc(snapshots, func(a, b litestream.SnapshotInfoV3) int {\n\t\treturn a.Index - b.Index\n\t})\n\treturn snapshots, nil\n}\n\n// WALSegmentsV3 returns WAL segments for a generation, sorted by index then offset.\nfunc (c *ReplicaClient) WALSegmentsV3(ctx context.Context, generation string) ([]litestream.WALSegmentInfoV3, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tprefix := litestream.WALPathV3(c.Path, generation) + \"/\"\n\n\tpaginator := s3.NewListObjectsV2Paginator(c.s3, &s3.ListObjectsV2Input{\n\t\tBucket: aws.String(c.Bucket),\n\t\tPrefix: aws.String(prefix),\n\t})\n\n\tvar segments []litestream.WALSegmentInfoV3\n\tfor paginator.HasMorePages() {\n\t\tpage, err := paginator.NextPage(ctx)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"s3: list wal segments: %w\", err)\n\t\t}\n\n\t\tfor _, obj := range page.Contents {\n\t\t\tkey := path.Base(aws.ToString(obj.Key))\n\t\t\tindex, offset, err := litestream.ParseWALSegmentFilenameV3(key)\n\t\t\tif err != nil {\n\t\t\t\tcontinue // skip invalid filenames\n\t\t\t}\n\n\t\t\tsegments = append(segments, litestream.WALSegmentInfoV3{\n\t\t\t\tGeneration: generation,\n\t\t\t\tIndex: index,\n\t\t\t\tOffset: offset,\n\t\t\t\tSize: aws.ToInt64(obj.Size),\n\t\t\t\tCreatedAt: aws.ToTime(obj.LastModified).UTC(),\n\t\t\t})\n\t\t}\n\t}\n\n\tslices.SortFunc(segments, func(a, b litestream.WALSegmentInfoV3) int {\n\t\tif a.Index != b.Index {\n\t\t\treturn a.Index - b.Index\n\t\t}\n\t\treturn int(a.Offset - b.Offset)\n\t})\n\treturn segments, nil\n}\n\n// OpenSnapshotV3 opens a v0.3.x snapshot file for reading.\n// The returned reader provides LZ4-decompressed data.\nfunc (c *ReplicaClient) OpenSnapshotV3(ctx context.Context, generation string, index int) (io.ReadCloser, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tkey := litestream.SnapshotPathV3(c.Path, generation, index)\n\n\tinput := &s3.GetObjectInput{\n\t\tBucket: aws.String(c.Bucket),\n\t\tKey: aws.String(key),\n\t}\n\n\t// Add SSE-C parameters if configured\n\tif c.SSECustomerKey != \"\" {\n\t\tinput.SSECustomerAlgorithm = aws.String(c.SSECustomerAlgorithm)\n\t\tinput.SSECustomerKey = aws.String(c.SSECustomerKey)\n\t\tinput.SSECustomerKeyMD5 = aws.String(c.SSECustomerKeyMD5)\n\t}\n\n\tout, err := c.s3.GetObject(ctx, input)\n\tif err != nil {\n\t\tif isNotExists(err) {\n\t\t\treturn nil, os.ErrNotExist\n\t\t}\n\t\treturn nil, fmt.Errorf(\"s3: get snapshot %s: %w\", key, err)\n\t}\n\n\treturn internal.NewLZ4Reader(out.Body), nil\n}\n\n// OpenWALSegmentV3 opens a v0.3.x WAL segment file for reading.\n// The returned reader provides LZ4-decompressed data.\nfunc (c *ReplicaClient) OpenWALSegmentV3(ctx context.Context, generation string, index int, offset int64) (io.ReadCloser, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tkey := litestream.WALSegmentPathV3(c.Path, generation, index, offset)\n\n\tinput := &s3.GetObjectInput{\n\t\tBucket: aws.String(c.Bucket),\n\t\tKey: aws.String(key),\n\t}\n\n\t// Add SSE-C parameters if configured\n\tif c.SSECustomerKey != \"\" {\n\t\tinput.SSECustomerAlgorithm = aws.String(c.SSECustomerAlgorithm)\n\t\tinput.SSECustomerKey = aws.String(c.SSECustomerKey)\n\t\tinput.SSECustomerKeyMD5 = aws.String(c.SSECustomerKeyMD5)\n\t}\n\n\tout, err := c.s3.GetObject(ctx, input)\n\tif err != nil {\n\t\tif isNotExists(err) {\n\t\t\treturn nil, os.ErrNotExist\n\t\t}\n\t\treturn nil, fmt.Errorf(\"s3: get wal segment %s: %w\", key, err)\n\t}\n\n\treturn internal.NewLZ4Reader(out.Body), nil\n}\n\n// fileIterator represents an iterator over LTX files in S3.\ntype fileIterator struct {\n\tctx context.Context\n\tcancel context.CancelFunc\n\tclient *ReplicaClient\n\tlevel int\n\tseek ltx.TXID\n\tprefix string\n\n\tuseMetadata bool // When true, fetch accurate timestamps from metadata\n\tmetadataCache map[string]time.Time // key -> timestamp cache for batch fetches\n\n\tpaginator *s3.ListObjectsV2Paginator\n\tpage *s3.ListObjectsV2Output\n\tpageIndex int\n\n\tclosed bool\n\terr error\n\tinfo *ltx.FileInfo\n}\n\nfunc newFileIterator(ctx context.Context, client *ReplicaClient, level int, seek ltx.TXID, useMetadata bool) *fileIterator {\n\tctx, cancel := context.WithCancel(ctx)\n\n\tprefix := client.Path + \"/\" + fmt.Sprintf(\"%04x/\", level)\n\titr := &fileIterator{\n\t\tctx: ctx,\n\t\tcancel: cancel,\n\t\tclient: client,\n\t\tlevel: level,\n\t\tseek: seek,\n\t\tprefix: prefix,\n\t\tuseMetadata: useMetadata,\n\t\tmetadataCache: make(map[string]time.Time),\n\t}\n\n\t// Create paginator for listing objects with level prefix\n\titr.paginator = s3.NewListObjectsV2Paginator(client.s3, &s3.ListObjectsV2Input{\n\t\tBucket: aws.String(client.Bucket),\n\t\tPrefix: aws.String(prefix),\n\t})\n\n\treturn itr\n}\n\n// fetchMetadataBatch fetches timestamps from S3 metadata for a batch of keys in parallel.\nfunc (itr *fileIterator) fetchMetadataBatch(keys []string) error {\n\tif len(keys) == 0 {\n\t\treturn nil\n\t}\n\n\t// Determine concurrency limit\n\tconcurrency := itr.client.MetadataConcurrency\n\tif concurrency <= 0 {\n\t\tconcurrency = DefaultMetadataConcurrency\n\t}\n\n\t// Pre-allocate results map to avoid lock contention during writes\n\tresults := make(map[string]time.Time, len(keys))\n\tvar mu sync.Mutex\n\n\t// Use x/sync/semaphore for precise concurrency control with context support\n\tsem := semaphore.NewWeighted(int64(concurrency))\n\tg, ctx := errgroup.WithContext(itr.ctx)\n\n\tfor _, key := range keys {\n\t\tkey := key // capture for goroutine\n\n\t\tg.Go(func() error {\n\t\t\t// Acquire semaphore slot (blocking with context cancellation)\n\t\t\tif err := sem.Acquire(ctx, 1); err != nil {\n\t\t\t\treturn err // context cancelled\n\t\t\t}\n\t\t\tdefer sem.Release(1)\n\n\t\t\theadInput := &s3.HeadObjectInput{\n\t\t\t\tBucket: aws.String(itr.client.Bucket),\n\t\t\t\tKey: aws.String(key),\n\t\t\t}\n\n\t\t\t// Add SSE-C parameters if configured (required for reading SSE-C encrypted objects)\n\t\t\t// Note: SSE-KMS does not require parameters on HeadObject - access is automatic\n\t\t\tif itr.client.SSECustomerKey != \"\" {\n\t\t\t\theadInput.SSECustomerAlgorithm = aws.String(itr.client.SSECustomerAlgorithm)\n\t\t\t\theadInput.SSECustomerKey = aws.String(itr.client.SSECustomerKey)\n\t\t\t\theadInput.SSECustomerKeyMD5 = aws.String(itr.client.SSECustomerKeyMD5)\n\t\t\t}\n\n\t\t\thead, err := itr.client.s3.HeadObject(ctx, headInput)\n\t\t\tif err != nil {\n\t\t\t\t// Non-fatal: file might not have metadata, use LastModified\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tif head.Metadata != nil {\n\t\t\t\tif ts, ok := head.Metadata[MetadataKeyTimestamp]; ok {\n\t\t\t\t\tif parsed, err := time.Parse(time.RFC3339Nano, ts); err == nil {\n\t\t\t\t\t\tmu.Lock()\n\t\t\t\t\t\tresults[key] = parsed\n\t\t\t\t\t\tmu.Unlock()\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t}\n\n\tif err := g.Wait(); err != nil {\n\t\treturn err\n\t}\n\n\t// Merge results into cache\n\tfor k, v := range results {\n\t\titr.metadataCache[k] = v\n\t}\n\treturn nil\n}\n\n// Close stops iteration and returns any error that occurred during iteration.\nfunc (itr *fileIterator) Close() (err error) {\n\titr.closed = true\n\titr.cancel()\n\treturn itr.err\n}\n\n// Next returns the next file. Returns false when no more files are available.\nfunc (itr *fileIterator) Next() bool {\n\tif itr.closed || itr.err != nil {\n\t\treturn false\n\t}\n\n\t// Process objects until we find a valid LTX file\n\tfor {\n\t\t// Load next page if needed\n\t\tif itr.page == nil || itr.pageIndex >= len(itr.page.Contents) {\n\t\t\tif !itr.paginator.HasMorePages() {\n\t\t\t\treturn false\n\t\t\t}\n\n\t\t\tvar err error\n\t\t\titr.page, err = itr.paginator.NextPage(itr.ctx)\n\t\t\tif err != nil {\n\t\t\t\titr.err = err\n\t\t\t\treturn false\n\t\t\t}\n\t\t\titr.pageIndex = 0\n\n\t\t\t// Log page contents for debugging.\n\t\t\tif len(itr.page.Contents) > 0 {\n\t\t\t\tkeys := make([]string, 0, len(itr.page.Contents))\n\t\t\t\tfor _, obj := range itr.page.Contents {\n\t\t\t\t\tkeys = append(keys, path.Base(aws.ToString(obj.Key)))\n\t\t\t\t}\n\t\t\t\titr.client.logger.Debug(\"s3 LIST page\", \"prefix\", itr.prefix, \"keys\", keys)\n\t\t\t}\n\n\t\t\t// Batch fetch metadata for the entire page when useMetadata is true.\n\t\t\t// This uses parallel HeadObject calls controlled by MetadataConcurrency\n\t\t\t// to avoid the O(N) sequential calls that caused restore hangs (issue #930).\n\t\t\tif itr.useMetadata && len(itr.page.Contents) > 0 {\n\t\t\t\tkeys := make([]string, 0, len(itr.page.Contents))\n\t\t\t\tfor _, obj := range itr.page.Contents {\n\t\t\t\t\tkeys = append(keys, aws.ToString(obj.Key))\n\t\t\t\t}\n\t\t\t\tif err := itr.fetchMetadataBatch(keys); err != nil {\n\t\t\t\t\titr.err = err\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Process current object\n\t\tif itr.pageIndex < len(itr.page.Contents) {\n\t\t\tobj := itr.page.Contents[itr.pageIndex]\n\t\t\titr.pageIndex++\n\n\t\t\t// Extract file info from key\n\t\t\tfullKey := aws.ToString(obj.Key)\n\t\t\tkey := path.Base(fullKey)\n\t\t\tminTXID, maxTXID, err := ltx.ParseFilename(key)\n\t\t\tif err != nil {\n\t\t\t\tcontinue // Skip non-LTX files\n\t\t\t}\n\n\t\t\t// Build file info\n\t\t\tinfo := <x.FileInfo{\n\t\t\t\tLevel: itr.level,\n\t\t\t\tMinTXID: minTXID,\n\t\t\t\tMaxTXID: maxTXID,\n\t\t\t}\n\n\t\t\t// Skip if below seek TXID\n\t\t\tif info.MinTXID < itr.seek {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Skip if wrong level\n\t\t\tif info.Level != itr.level {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Set file info\n\t\t\tinfo.Size = aws.ToInt64(obj.Size)\n\n\t\t\t// Use cached metadata timestamp if available (from batch fetch),\n\t\t\t// otherwise fallback to LastModified from LIST operation.\n\t\t\tif itr.useMetadata {\n\t\t\t\tif ts, ok := itr.metadataCache[fullKey]; ok {\n\t\t\t\t\tinfo.CreatedAt = ts\n\t\t\t\t} else {\n\t\t\t\t\tinfo.CreatedAt = aws.ToTime(obj.LastModified).UTC()\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tinfo.CreatedAt = aws.ToTime(obj.LastModified).UTC()\n\t\t\t}\n\n\t\t\titr.info = info\n\t\t\treturn true\n\t\t}\n\t}\n}\n\n// Item returns the metadata for the current file.\nfunc (itr *fileIterator) Item() *ltx.FileInfo {\n\treturn itr.info\n}\n\n// Err returns any error that occurred during iteration.\nfunc (itr *fileIterator) Err() error {\n\treturn itr.err\n}\n\n// ParseURL parses an S3 URL into its host and path parts.\n// If endpoint is set, it can override the host.\nfunc ParseURL(s, endpoint string) (bucket, region, key string, err error) {\n\tu, err := url.Parse(s)\n\tif err != nil {\n\t\treturn \"\", \"\", \"\", err\n\t}\n\n\tif u.Scheme != \"s3\" {\n\t\treturn \"\", \"\", \"\", fmt.Errorf(\"s3: invalid url scheme\")\n\t}\n\n\t// Special handling for filebase.com\n\tif u.Host == \"filebase.com\" {\n\t\tparts := strings.SplitN(strings.TrimPrefix(u.Path, \"/\"), \"/\", 2)\n\t\tif len(parts) == 0 {\n\t\t\treturn \"\", \"\", \"\", fmt.Errorf(\"s3: bucket required\")\n\t\t}\n\t\tbucket = parts[0]\n\t\tif len(parts) > 1 {\n\t\t\tkey = parts[1]\n\t\t}\n\t\treturn bucket, \"\", key, nil\n\t}\n\n\t// For other hosts, check if it's a special endpoint\n\tbucket, region, _, _ = ParseHost(u.Host)\n\tif bucket == \"\" {\n\t\tbucket = u.Host\n\t}\n\n\tkey = strings.TrimPrefix(u.Path, \"/\")\n\treturn bucket, region, key, nil\n}\n\n// ParseHost parses the host/endpoint for an S3-like storage system.\n// Endpoints: https://docs.aws.amazon.com/general/latest/gr/s3.html\nfunc ParseHost(host string) (bucket, region, endpoint string, forcePathStyle bool) {\n\t// Check for MinIO-style hosts (bucket.host:port)\n\tif strings.Contains(host, \":\") && !strings.Contains(host, \".com\") {\n\t\tparts := strings.SplitN(host, \".\", 2)\n\t\tif len(parts) == 2 {\n\t\t\t// Extract bucket from bucket.host:port format\n\t\t\tbucket = parts[0]\n\t\t\tendpoint = \"http://\" + parts[1]\n\t\t\treturn bucket, DefaultRegion, endpoint, true\n\t\t}\n\t\t// No bucket in host, just host:port\n\t\treturn \"\", \"\", \"http://\" + host, true\n\t}\n\n\t// Check common object storage providers\n\t// Check for AWS S3 URLs first\n\tif a := awsS3Regex.FindStringSubmatch(host); len(a) > 1 {\n\t\tbucket = a[1]\n\t\tif len(a) > 2 && a[2] != \"\" {\n\t\t\tregion = a[2]\n\t\t}\n\t\treturn bucket, region, \"\", false\n\t} else if a := digitaloceanRegex.FindStringSubmatch(host); len(a) > 1 {\n\t\tbucket = a[1]\n\t\tregion = a[2]\n\t\treturn bucket, region, fmt.Sprintf(\"https://%s.digitaloceanspaces.com\", region), false\n\t} else if a := backblazeRegex.FindStringSubmatch(host); len(a) > 1 {\n\t\tregion = a[2]\n\t\tbucket = a[1]\n\t\tendpoint = fmt.Sprintf(\"https://s3.%s.backblazeb2.com\", region)\n\t\treturn bucket, region, endpoint, true\n\t} else if a := filebaseRegex.FindStringSubmatch(host); len(a) > 1 {\n\t\tbucket = a[1]\n\t\tendpoint = \"s3.filebase.com\"\n\t\treturn bucket, \"\", endpoint, false\n\t} else if a := scalewayRegex.FindStringSubmatch(host); len(a) > 1 {\n\t\tregion = a[2]\n\t\tbucket = a[1]\n\t\tendpoint = fmt.Sprintf(\"s3.%s.scw.cloud\", region)\n\t\treturn bucket, region, endpoint, false\n\t}\n\n\t// For standard S3, the host is the bucket name\n\treturn host, \"\", \"\", false\n}\n\nvar (\n\tawsS3Regex = regexp.MustCompile(`^(.+)\\.s3(?:\\.([^.]+))?\\.amazonaws\\.com$`)\n\tdigitaloceanRegex = regexp.MustCompile(`^(?:(.+)\\.)?([^.]+)\\.digitaloceanspaces.com$`)\n\tbackblazeRegex = regexp.MustCompile(`^(?:(.+)\\.)?s3.([^.]+)\\.backblazeb2.com$`)\n\tfilebaseRegex = regexp.MustCompile(`^(?:(.+)\\.)?s3.filebase.com$`)\n\tscalewayRegex = regexp.MustCompile(`^(?:(.+)\\.)?s3.([^.]+)\\.scw\\.cloud$`)\n)\n\nfunc isNotExists(err error) bool {\n\tvar apiErr smithy.APIError\n\tif errors.As(err, &apiErr) {\n\t\treturn apiErr.ErrorCode() == \"NoSuchKey\"\n\t}\n\treturn false\n}\n\nfunc deleteOutputError(out *s3.DeleteObjectsOutput) error {\n\tif len(out.Errors) == 0 {\n\t\treturn nil\n\t}\n\n\t// Build generic error\n\tvar b strings.Builder\n\tb.WriteString(\"failed to delete files:\")\n\tfor _, err := range out.Errors {\n\t\tfmt.Fprintf(&b, \"\\n%s: %s\", aws.ToString(err.Key), aws.ToString(err.Message))\n\t}\n\treturn errors.New(b.String())\n}\n\n// parseS3DebugEnv parses the LITESTREAM_S3_DEBUG environment variable and returns\n// the corresponding AWS SDK ClientLogMode. Supports comma-separated values.\nfunc parseS3DebugEnv() aws.ClientLogMode {\n\tv := os.Getenv(\"LITESTREAM_S3_DEBUG\")\n\tif v == \"\" {\n\t\treturn 0\n\t}\n\n\tvar logMode aws.ClientLogMode\n\tfor _, mode := range strings.Split(v, \",\") {\n\t\tswitch strings.ToLower(strings.TrimSpace(mode)) {\n\t\tcase \"signing\":\n\t\t\tlogMode |= aws.LogSigning\n\t\tcase \"request\":\n\t\t\tlogMode |= aws.LogRequest\n\t\tcase \"request-with-body\":\n\t\t\tlogMode |= aws.LogRequestWithBody\n\t\tcase \"response\":\n\t\t\tlogMode |= aws.LogResponse\n\t\tcase \"response-with-body\":\n\t\t\tlogMode |= aws.LogResponseWithBody\n\t\tcase \"retries\":\n\t\t\tlogMode |= aws.LogRetries\n\t\tcase \"all\":\n\t\t\tlogMode |= aws.LogSigning | aws.LogRequest | aws.LogRequestWithBody |\n\t\t\t\taws.LogResponse | aws.LogResponseWithBody | aws.LogRetries\n\t\tdefault:\n\t\t\tslog.Warn(\"unknown LITESTREAM_S3_DEBUG value, expected: signing, request, request-with-body, response, response-with-body, retries, all\", \"value\", mode)\n\t\t}\n\t}\n\treturn logMode\n}\n" }, { "path": "s3/replica_client_test.go", "content": "package s3\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"crypto/md5\"\n\t\"crypto/sha256\"\n\t\"encoding/base64\"\n\t\"encoding/hex\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/aws/aws-sdk-go-v2/aws\"\n\t\"github.com/aws/aws-sdk-go-v2/service/s3\"\n\t\"github.com/aws/aws-sdk-go-v2/service/s3/types\"\n\t\"github.com/aws/smithy-go\"\n\t\"github.com/aws/smithy-go/middleware\"\n\tsmithyhttp \"github.com/aws/smithy-go/transport/http\"\n\t\"github.com/superfly/ltx\"\n\n\tlitestream \"github.com/benbjohnson/litestream\"\n)\n\n// mockAPIError implements smithy.APIError for testing\ntype mockAPIError struct {\n\tcode string\n\tmessage string\n}\n\nfunc (e *mockAPIError) Error() string {\n\treturn e.message\n}\n\nfunc (e *mockAPIError) ErrorCode() string {\n\treturn e.code\n}\n\nfunc (e *mockAPIError) ErrorMessage() string {\n\treturn e.message\n}\n\nfunc (e *mockAPIError) ErrorFault() smithy.ErrorFault {\n\treturn smithy.FaultUnknown\n}\n\nfunc TestIsNotExists(t *testing.T) {\n\t// Test with NoSuchKey error\n\tnoSuchKeyErr := &mockAPIError{\n\t\tcode: \"NoSuchKey\",\n\t\tmessage: \"The specified key does not exist\",\n\t}\n\tif !isNotExists(noSuchKeyErr) {\n\t\tt.Error(\"isNotExists should return true for NoSuchKey error\")\n\t}\n\n\t// Test with different error code\n\tdifferentErr := &mockAPIError{\n\t\tcode: \"AccessDenied\",\n\t\tmessage: \"Access denied\",\n\t}\n\tif isNotExists(differentErr) {\n\t\tt.Error(\"isNotExists should return false for non-NoSuchKey error\")\n\t}\n\n\t// Test with non-API error\n\tregularErr := errors.New(\"regular error\")\n\tif isNotExists(regularErr) {\n\t\tt.Error(\"isNotExists should return false for non-API error\")\n\t}\n\n\t// Test with nil error\n\tif isNotExists(nil) {\n\t\tt.Error(\"isNotExists should return false for nil error\")\n\t}\n\n\t// Test with wrapped API error\n\twrappedErr := &mockAPIError{\n\t\tcode: \"NoSuchKey\",\n\t\tmessage: \"wrapped key error\",\n\t}\n\tif !isNotExists(wrappedErr) {\n\t\tt.Error(\"isNotExists should return true for wrapped NoSuchKey error\")\n\t}\n}\n\nfunc TestReplicaClient_DefaultSignPayload(t *testing.T) {\n\tclient := NewReplicaClient()\n\tif !client.SignPayload {\n\t\tt.Error(\"expected default SignPayload to be true for AWS S3 compatibility\")\n\t}\n\tif !client.RequireContentMD5 {\n\t\tt.Error(\"expected default RequireContentMD5 to be true for AWS S3 compatibility\")\n\t}\n}\n\nfunc TestReplicaClientPayloadSigning(t *testing.T) {\n\tdata := mustLTX(t)\n\tsignedPayload := sha256.Sum256(data)\n\twantSigned := hex.EncodeToString(signedPayload[:])\n\n\ttests := []struct {\n\t\tname string\n\t\tsignPayload bool\n\t\twantHeader string\n\t}{\n\t\t{name: \"UnsignedWhenDisabled\", signPayload: false, wantHeader: \"UNSIGNED-PAYLOAD\"},\n\t\t{name: \"SignedByDefault\", signPayload: true, wantHeader: wantSigned},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\theaders := make(chan http.Header, 1)\n\t\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tdefer r.Body.Close()\n\t\t\t\t_, _ = io.Copy(io.Discard, r.Body)\n\n\t\t\t\tif r.Method == http.MethodPut {\n\t\t\t\t\tselect {\n\t\t\t\t\tcase headers <- r.Header.Clone():\n\t\t\t\t\tdefault:\n\t\t\t\t\t}\n\t\t\t\t\tw.Header().Set(\"ETag\", `\"test-etag\"`)\n\t\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\tclient := NewReplicaClient()\n\t\t\tclient.Bucket = \"test-bucket\"\n\t\t\tclient.Path = \"replica\"\n\t\t\tclient.Region = \"us-east-1\"\n\t\t\tclient.Endpoint = server.URL\n\t\t\tclient.ForcePathStyle = true\n\t\t\tclient.AccessKeyID = \"test-access-key\"\n\t\t\tclient.SecretAccessKey = \"test-secret-key\"\n\t\t\tclient.SignPayload = tt.signPayload\n\n\t\t\tctx := context.Background()\n\t\t\tif err := client.Init(ctx); err != nil {\n\t\t\t\tt.Fatalf(\"Init() error: %v\", err)\n\t\t\t}\n\n\t\t\tif _, err := client.WriteLTXFile(ctx, 0, 2, 2, bytes.NewReader(data)); err != nil {\n\t\t\t\tt.Fatalf(\"WriteLTXFile() error: %v\", err)\n\t\t\t}\n\n\t\t\tselect {\n\t\t\tcase hdr := <-headers:\n\t\t\t\tif got, want := hdr.Get(\"x-amz-content-sha256\"), tt.wantHeader; got != want {\n\t\t\t\t\tt.Fatalf(\"x-amz-content-sha256 header = %q, want %q\", got, want)\n\t\t\t\t}\n\t\t\tcase <-time.After(time.Second):\n\t\t\t\tt.Fatal(\"timeout waiting for PUT request\")\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReplicaClient_UnsignedPayload_NoChunkedEncoding(t *testing.T) {\n\tdata := mustLTX(t)\n\n\theaders := make(chan http.Header, 1)\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tdefer r.Body.Close()\n\t\t_, _ = io.Copy(io.Discard, r.Body)\n\n\t\tif r.Method == http.MethodPut {\n\t\t\tselect {\n\t\t\tcase headers <- r.Header.Clone():\n\t\t\tdefault:\n\t\t\t}\n\t\t\tw.Header().Set(\"ETag\", `\"test-etag\"`)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\treturn\n\t\t}\n\n\t\tw.WriteHeader(http.StatusOK)\n\t}))\n\tdefer server.Close()\n\n\tclient := NewReplicaClient()\n\tclient.Bucket = \"test-bucket\"\n\tclient.Path = \"replica\"\n\tclient.Region = \"us-east-1\"\n\tclient.Endpoint = server.URL\n\tclient.ForcePathStyle = true\n\tclient.AccessKeyID = \"test-access-key\"\n\tclient.SecretAccessKey = \"test-secret-key\"\n\tclient.SignPayload = false\n\n\tctx := context.Background()\n\tif err := client.Init(ctx); err != nil {\n\t\tt.Fatalf(\"Init() error: %v\", err)\n\t}\n\n\tif _, err := client.WriteLTXFile(ctx, 0, 2, 2, bytes.NewReader(data)); err != nil {\n\t\tt.Fatalf(\"WriteLTXFile() error: %v\", err)\n\t}\n\n\tselect {\n\tcase hdr := <-headers:\n\t\tif got := hdr.Get(\"x-amz-content-sha256\"); got != \"UNSIGNED-PAYLOAD\" {\n\t\t\tt.Errorf(\"x-amz-content-sha256 = %q, want UNSIGNED-PAYLOAD\", got)\n\t\t}\n\n\t\tcontentEnc := hdr.Get(\"Content-Encoding\")\n\t\tif strings.Contains(contentEnc, \"aws-chunked\") {\n\t\t\tt.Errorf(\"Content-Encoding contains aws-chunked: %q; aws-chunked is incompatible with UNSIGNED-PAYLOAD\", contentEnc)\n\t\t}\n\n\t\ttransferEnc := hdr.Get(\"Transfer-Encoding\")\n\t\tif strings.Contains(transferEnc, \"aws-chunked\") {\n\t\t\tt.Errorf(\"Transfer-Encoding contains aws-chunked: %q; aws-chunked is incompatible with UNSIGNED-PAYLOAD\", transferEnc)\n\t\t}\n\n\t\tdecoded := hdr.Get(\"X-Amz-Decoded-Content-Length\")\n\t\tif decoded != \"\" {\n\t\t\tt.Errorf(\"X-Amz-Decoded-Content-Length = %q; this header indicates aws-chunked encoding which is incompatible with UNSIGNED-PAYLOAD\", decoded)\n\t\t}\n\tcase <-time.After(time.Second):\n\t\tt.Fatal(\"timeout waiting for PUT request\")\n\t}\n}\n\n// TestReplicaClient_SignedPayload_CustomEndpoint_NoChunkedEncoding verifies that\n// aws-chunked encoding is disabled for custom endpoints even when SignPayload=true.\n// This is necessary for S3-compatible providers (Filebase, MinIO, Backblaze B2, etc.)\n// that don't support aws-chunked encoding at all. See issue #895.\nfunc TestReplicaClient_SignedPayload_CustomEndpoint_NoChunkedEncoding(t *testing.T) {\n\tdata := mustLTX(t)\n\n\theaders := make(chan http.Header, 1)\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tdefer r.Body.Close()\n\t\t_, _ = io.Copy(io.Discard, r.Body)\n\n\t\tif r.Method == http.MethodPut {\n\t\t\tselect {\n\t\t\tcase headers <- r.Header.Clone():\n\t\t\tdefault:\n\t\t\t}\n\t\t\tw.Header().Set(\"ETag\", `\"test-etag\"`)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\treturn\n\t\t}\n\n\t\tw.WriteHeader(http.StatusOK)\n\t}))\n\tdefer server.Close()\n\n\tclient := NewReplicaClient()\n\tclient.Bucket = \"test-bucket\"\n\tclient.Path = \"replica\"\n\tclient.Region = \"us-east-1\"\n\tclient.Endpoint = server.URL // Custom endpoint (non-AWS)\n\tclient.ForcePathStyle = true\n\tclient.AccessKeyID = \"test-access-key\"\n\tclient.SecretAccessKey = \"test-secret-key\"\n\tclient.SignPayload = true // Signed payload, but still using custom endpoint\n\n\tctx := context.Background()\n\tif err := client.Init(ctx); err != nil {\n\t\tt.Fatalf(\"Init() error: %v\", err)\n\t}\n\n\tif _, err := client.WriteLTXFile(ctx, 0, 2, 2, bytes.NewReader(data)); err != nil {\n\t\tt.Fatalf(\"WriteLTXFile() error: %v\", err)\n\t}\n\n\tselect {\n\tcase hdr := <-headers:\n\t\t// With SignPayload=true, we expect an actual SHA256 hash (not UNSIGNED-PAYLOAD)\n\t\tsha256Header := hdr.Get(\"x-amz-content-sha256\")\n\t\tif sha256Header == \"\" {\n\t\t\tt.Error(\"x-amz-content-sha256 header should be set\")\n\t\t}\n\t\tif sha256Header == \"UNSIGNED-PAYLOAD\" {\n\t\t\tt.Error(\"x-amz-content-sha256 should be actual hash, not UNSIGNED-PAYLOAD, when SignPayload=true\")\n\t\t}\n\n\t\t// But aws-chunked encoding should still be disabled for custom endpoints\n\t\tcontentEnc := hdr.Get(\"Content-Encoding\")\n\t\tif strings.Contains(contentEnc, \"aws-chunked\") {\n\t\t\tt.Errorf(\"Content-Encoding contains aws-chunked: %q; aws-chunked is not supported by S3-compatible providers\", contentEnc)\n\t\t}\n\n\t\ttransferEnc := hdr.Get(\"Transfer-Encoding\")\n\t\tif strings.Contains(transferEnc, \"aws-chunked\") {\n\t\t\tt.Errorf(\"Transfer-Encoding contains aws-chunked: %q; aws-chunked is not supported by S3-compatible providers\", transferEnc)\n\t\t}\n\n\t\tdecoded := hdr.Get(\"X-Amz-Decoded-Content-Length\")\n\t\tif decoded != \"\" {\n\t\t\tt.Errorf(\"X-Amz-Decoded-Content-Length = %q; this header indicates aws-chunked encoding which is not supported by S3-compatible providers\", decoded)\n\t\t}\n\tcase <-time.After(time.Second):\n\t\tt.Fatal(\"timeout waiting for PUT request\")\n\t}\n}\n\nfunc mustLTX(t *testing.T) []byte {\n\tt.Helper()\n\n\tbuf := new(bytes.Buffer)\n\tenc, err := ltx.NewEncoder(buf)\n\tif err != nil {\n\t\tt.Fatalf(\"NewEncoder: %v\", err)\n\t}\n\n\tif err := enc.EncodeHeader(ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tPageSize: 4096,\n\t\tCommit: 0,\n\t\tMinTXID: 2,\n\t\tMaxTXID: 2,\n\t\tTimestamp: time.Now().UnixMilli(),\n\t\tPreApplyChecksum: ltx.ChecksumFlag | 1,\n\t}); err != nil {\n\t\tt.Fatalf(\"EncodeHeader: %v\", err)\n\t}\n\n\tenc.SetPostApplyChecksum(ltx.ChecksumFlag)\n\tif err := enc.Close(); err != nil {\n\t\tt.Fatalf(\"Close: %v\", err)\n\t}\n\n\treturn buf.Bytes()\n}\n\nfunc mustLTXWithSize(t *testing.T, size int) []byte {\n\tt.Helper()\n\theader := mustLTX(t)\n\tif size <= len(header) {\n\t\treturn header[:size]\n\t}\n\tdata := make([]byte, size)\n\tcopy(data, header)\n\treturn data\n}\n\nfunc TestReplicaClient_MultipartUploadThreshold(t *testing.T) {\n\tconst mb = 1024 * 1024\n\n\ttests := []struct {\n\t\tname string\n\t\tpayloadSize int\n\t\twantMultipart bool\n\t}{\n\t\t{\"BelowThreshold_4MB\", 4 * mb, false},\n\t\t{\"AtThreshold_5MB\", 5 * mb, true},\n\t\t{\"AboveThreshold_6MB\", 6 * mb, true},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\tgotPut bool\n\t\t\t\tgotInitiate bool\n\t\t\t\tgotComplete bool\n\t\t\t\tawsChunked bool\n\t\t\t)\n\n\t\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tdefer r.Body.Close()\n\t\t\t\t_, _ = io.Copy(io.Discard, r.Body)\n\n\t\t\t\tif r.Method == http.MethodPut {\n\t\t\t\t\tif strings.Contains(r.Header.Get(\"Content-Encoding\"), \"aws-chunked\") {\n\t\t\t\t\t\tawsChunked = true\n\t\t\t\t\t}\n\t\t\t\t\tif strings.Contains(r.Header.Get(\"Transfer-Encoding\"), \"aws-chunked\") {\n\t\t\t\t\t\tawsChunked = true\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tquery := r.URL.Query()\n\n\t\t\t\tif r.Method == http.MethodPost && query.Has(\"uploads\") {\n\t\t\t\t\tgotInitiate = true\n\t\t\t\t\tw.Header().Set(\"Content-Type\", \"application/xml\")\n\t\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\t\tfmt.Fprint(w, `test-buckettest-keytest-upload-id`)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif r.Method == http.MethodPut && query.Get(\"partNumber\") != \"\" {\n\t\t\t\t\tw.Header().Set(\"ETag\", fmt.Sprintf(`\"part-etag-%s\"`, query.Get(\"partNumber\")))\n\t\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif r.Method == http.MethodPost && query.Get(\"uploadId\") != \"\" && !query.Has(\"uploads\") {\n\t\t\t\t\tgotComplete = true\n\t\t\t\t\tw.Header().Set(\"Content-Type\", \"application/xml\")\n\t\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\t\tfmt.Fprint(w, `http://test-bucket.s3.amazonaws.com/test-keytest-buckettest-key\"complete-etag\"`)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif r.Method == http.MethodPut {\n\t\t\t\t\tgotPut = true\n\t\t\t\t\tw.Header().Set(\"ETag\", `\"test-etag\"`)\n\t\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t}))\n\t\t\tdefer server.Close()\n\n\t\t\tdata := mustLTXWithSize(t, tt.payloadSize)\n\n\t\t\tclient := NewReplicaClient()\n\t\t\tclient.Bucket = \"test-bucket\"\n\t\t\tclient.Path = \"replica\"\n\t\t\tclient.Region = \"us-east-1\"\n\t\t\tclient.Endpoint = server.URL\n\t\t\tclient.ForcePathStyle = true\n\t\t\tclient.AccessKeyID = \"test-access-key\"\n\t\t\tclient.SecretAccessKey = \"test-secret-key\"\n\n\t\t\tctx := context.Background()\n\t\t\tif err := client.Init(ctx); err != nil {\n\t\t\t\tt.Fatalf(\"Init() error: %v\", err)\n\t\t\t}\n\n\t\t\tif _, err := client.WriteLTXFile(ctx, 0, 2, 2, bytes.NewReader(data)); err != nil {\n\t\t\t\tt.Fatalf(\"WriteLTXFile() error: %v\", err)\n\t\t\t}\n\n\t\t\tif tt.wantMultipart {\n\t\t\t\tif !gotInitiate {\n\t\t\t\t\tt.Error(\"expected CreateMultipartUpload but did not receive one\")\n\t\t\t\t}\n\t\t\t\tif !gotComplete {\n\t\t\t\t\tt.Error(\"expected CompleteMultipartUpload but did not receive one\")\n\t\t\t\t}\n\t\t\t\tif gotPut {\n\t\t\t\t\tt.Error(\"did not expect single PUT for multipart upload\")\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif !gotPut {\n\t\t\t\t\tt.Error(\"expected single PUT upload but did not receive one\")\n\t\t\t\t}\n\t\t\t\tif gotInitiate {\n\t\t\t\t\tt.Error(\"did not expect CreateMultipartUpload for single-part upload\")\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif awsChunked {\n\t\t\t\tt.Error(\"aws-chunked encoding detected; this is incompatible with S3-compatible providers\")\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestReplicaClient_Init_BucketValidation tests that Init validates bucket name\nfunc TestReplicaClient_Init_BucketValidation(t *testing.T) {\n\tt.Run(\"EmptyBucket\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"\" // Empty bucket name\n\t\tc.Region = \"us-east-1\"\n\n\t\terr := c.Init(context.Background())\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error for empty bucket name\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"bucket name is required\") {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ValidBucket\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\t// Note: This will fail when trying to connect, but should pass bucket validation\n\t\terr := c.Init(context.Background())\n\t\t// We expect a different error (not bucket validation)\n\t\tif err != nil && strings.Contains(err.Error(), \"bucket name is required\") {\n\t\t\tt.Errorf(\"should not fail bucket validation with valid bucket: %v\", err)\n\t\t}\n\t})\n}\n\n// TestReplicaClient_UploaderConfiguration tests that uploader configuration is applied\nfunc TestReplicaClient_UploaderConfiguration(t *testing.T) {\n\tt.Run(\"CustomPartSize\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.PartSize = 10 * 1024 * 1024 // 10MB\n\t\tc.Concurrency = 10\n\n\t\t// Verify the configuration is set\n\t\tif c.PartSize != 10*1024*1024 {\n\t\t\tt.Errorf(\"expected PartSize to be 10MB, got %d\", c.PartSize)\n\t\t}\n\t\tif c.Concurrency != 10 {\n\t\t\tt.Errorf(\"expected Concurrency to be 10, got %d\", c.Concurrency)\n\t\t}\n\t})\n\n\tt.Run(\"DefaultConfiguration\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\t// Verify defaults are zero (will use SDK defaults)\n\t\tif c.PartSize != 0 {\n\t\t\tt.Errorf(\"expected default PartSize to be 0, got %d\", c.PartSize)\n\t\t}\n\t\tif c.Concurrency != 0 {\n\t\t\tt.Errorf(\"expected default Concurrency to be 0, got %d\", c.Concurrency)\n\t\t}\n\t})\n}\n\n// TestReplicaClient_ConfigureEndpoint tests the endpoint configuration helper\nfunc TestReplicaClient_ConfigureEndpoint(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tendpoint string\n\t\tforcePathStyle bool\n\t\texpectHTTPS bool\n\t}{\n\t\t{\n\t\t\tname: \"HTTPEndpoint\",\n\t\t\tendpoint: \"http://localhost:9000\",\n\t\t\tforcePathStyle: true,\n\t\t\texpectHTTPS: false,\n\t\t},\n\t\t{\n\t\t\tname: \"HTTPSEndpoint\",\n\t\t\tendpoint: \"https://s3.amazonaws.com\",\n\t\t\tforcePathStyle: false,\n\t\t\texpectHTTPS: true,\n\t\t},\n\t\t{\n\t\t\tname: \"EndpointWithoutScheme\",\n\t\t\tendpoint: \"s3.us-west-002.backblazeb2.com\",\n\t\t\tforcePathStyle: false,\n\t\t\texpectHTTPS: true,\n\t\t},\n\t\t{\n\t\t\tname: \"EmptyEndpoint\",\n\t\t\tendpoint: \"\",\n\t\t\tforcePathStyle: false,\n\t\t\texpectHTTPS: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tc := NewReplicaClient()\n\t\t\tc.Endpoint = tt.endpoint\n\t\t\tc.ForcePathStyle = tt.forcePathStyle\n\n\t\t\t// Test that configureEndpoint can be called without error\n\t\t\tvar opts []func(*s3.Options)\n\t\t\tc.configureEndpoint(&opts)\n\n\t\t\t// Verify opts were added when endpoint is set\n\t\t\tif tt.endpoint != \"\" && len(opts) == 0 {\n\t\t\t\tt.Error(\"expected endpoint options to be added\")\n\t\t\t}\n\t\t\tif tt.endpoint == \"\" && len(opts) != 0 {\n\t\t\t\tt.Error(\"expected no endpoint options for empty endpoint\")\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestReplicaClient_HTTPClientConfiguration tests HTTP client setup\nfunc TestReplicaClient_HTTPClientConfiguration(t *testing.T) {\n\tt.Run(\"WithSkipVerify\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.SkipVerify = true\n\n\t\t// We can't directly test the HTTP client configuration without\n\t\t// actually initializing, but we can verify the flag is set\n\t\tif !c.SkipVerify {\n\t\t\tt.Error(\"expected SkipVerify to be true\")\n\t\t}\n\t})\n\n\tt.Run(\"WithoutSkipVerify\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.SkipVerify = false\n\n\t\tif c.SkipVerify {\n\t\t\tt.Error(\"expected SkipVerify to be false\")\n\t\t}\n\t})\n}\n\nfunc TestReplicaClientDeleteLTXFiles_ContentMD5(t *testing.T) {\n\tt.Run(\"Enabled\", func(t *testing.T) {\n\t\tvar callCount int\n\n\t\thttpClient := smithyhttp.ClientDoFunc(func(r *http.Request) (*http.Response, error) {\n\t\t\tt.Helper()\n\t\t\tcallCount++\n\n\t\t\tif r.Method != http.MethodPost {\n\t\t\t\tt.Fatalf(\"unexpected method: %s\", r.Method)\n\t\t\t}\n\t\t\tif !strings.Contains(r.URL.RawQuery, \"delete\") {\n\t\t\t\tt.Fatalf(\"unexpected query: %s\", r.URL.RawQuery)\n\t\t\t}\n\n\t\t\tif ua := r.Header.Get(\"User-Agent\"); !strings.Contains(ua, \"litestream\") {\n\t\t\t\tt.Fatalf(\"expected User-Agent to contain litestream, got %q\", ua)\n\t\t\t}\n\n\t\t\tbody, err := io.ReadAll(r.Body)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"read body: %v\", err)\n\t\t\t}\n\t\t\tr.Body.Close()\n\n\t\t\tgot := r.Header.Get(\"Content-MD5\")\n\t\t\tif got == \"\" {\n\t\t\t\tt.Fatal(\"expected Content-MD5 header\")\n\t\t\t}\n\n\t\t\tsum := md5.Sum(body)\n\t\t\twant := base64.StdEncoding.EncodeToString(sum[:])\n\t\t\tif got != want {\n\t\t\t\tt.Fatalf(\"unexpected Content-MD5 header: got %q, want %q\", got, want)\n\t\t\t}\n\n\t\t\tresp := &http.Response{\n\t\t\t\tStatusCode: http.StatusOK,\n\t\t\t\tHeader: http.Header{\"Content-Type\": []string{\"application/xml\"}},\n\t\t\t\tBody: io.NopCloser(strings.NewReader(\n\t\t\t\t\t``,\n\t\t\t\t)),\n\t\t\t}\n\t\t\treturn resp, nil\n\t\t})\n\n\t\tcfg := aws.Config{\n\t\t\tRegion: \"us-east-1\",\n\t\t\tCredentials: aws.NewCredentialsCache(aws.AnonymousCredentials{}),\n\t\t\tHTTPClient: httpClient,\n\t\t}\n\n\t\tc := NewReplicaClient()\n\t\tc.logger = slog.New(slog.NewTextHandler(io.Discard, nil))\n\t\tc.s3 = s3.NewFromConfig(cfg, func(o *s3.Options) {\n\t\t\to.APIOptions = append(o.APIOptions, c.middlewareOption())\n\t\t})\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Path = \"test-path\"\n\n\t\tfiles := []*ltx.FileInfo{\n\t\t\t{Level: 0, MinTXID: 1, MaxTXID: 1},\n\t\t\t{Level: 0, MinTXID: 2, MaxTXID: 2},\n\t\t}\n\n\t\tif err := c.DeleteLTXFiles(context.Background(), files); err != nil {\n\t\t\tt.Fatalf(\"DeleteLTXFiles: %v\", err)\n\t\t}\n\t\tif callCount != 1 {\n\t\t\tt.Fatalf(\"unexpected call count: %d\", callCount)\n\t\t}\n\t})\n\n\tt.Run(\"Disabled\", func(t *testing.T) {\n\t\thttpClient := smithyhttp.ClientDoFunc(func(r *http.Request) (*http.Response, error) {\n\t\t\tt.Helper()\n\t\t\tif md5Header := r.Header.Get(\"Content-MD5\"); md5Header != \"\" {\n\t\t\t\tt.Fatalf(\"expected Content-MD5 header to be empty when disabled, got %q\", md5Header)\n\t\t\t}\n\t\t\tresp := &http.Response{\n\t\t\t\tStatusCode: http.StatusOK,\n\t\t\t\tHeader: http.Header{\"Content-Type\": []string{\"application/xml\"}},\n\t\t\t\tBody: io.NopCloser(strings.NewReader(\n\t\t\t\t\t``,\n\t\t\t\t)),\n\t\t\t}\n\t\t\treturn resp, nil\n\t\t})\n\n\t\tcfg := aws.Config{\n\t\t\tRegion: \"us-east-1\",\n\t\t\tCredentials: aws.NewCredentialsCache(aws.AnonymousCredentials{}),\n\t\t\tHTTPClient: httpClient,\n\t\t}\n\n\t\tc := NewReplicaClient()\n\t\tc.RequireContentMD5 = false\n\t\tc.logger = slog.New(slog.NewTextHandler(io.Discard, nil))\n\t\tc.s3 = s3.NewFromConfig(cfg, func(o *s3.Options) {\n\t\t\to.APIOptions = append(o.APIOptions, c.middlewareOption())\n\t\t})\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Path = \"test-path\"\n\n\t\tfiles := []*ltx.FileInfo{{Level: 0, MinTXID: 1, MaxTXID: 1}}\n\t\tif err := c.DeleteLTXFiles(context.Background(), files); err != nil {\n\t\t\tt.Fatalf(\"DeleteLTXFiles: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestReplicaClientDeleteLTXFiles_PreexistingContentMD5(t *testing.T) {\n\tconst preexistingMD5 = \"preexisting-checksum-value\"\n\tvar callCount int\n\n\thttpClient := smithyhttp.ClientDoFunc(func(r *http.Request) (*http.Response, error) {\n\t\tt.Helper()\n\t\tcallCount++\n\n\t\tgot := r.Header.Get(\"Content-MD5\")\n\t\tif got != preexistingMD5 {\n\t\t\tt.Fatalf(\"middleware should not override existing Content-MD5: got %q, want %q\", got, preexistingMD5)\n\t\t}\n\n\t\tresp := &http.Response{\n\t\t\tStatusCode: http.StatusOK,\n\t\t\tHeader: http.Header{\"Content-Type\": []string{\"application/xml\"}},\n\t\t\tBody: io.NopCloser(strings.NewReader(\n\t\t\t\t``,\n\t\t\t)),\n\t\t}\n\t\treturn resp, nil\n\t})\n\n\tcfg := aws.Config{\n\t\tRegion: \"us-east-1\",\n\t\tCredentials: aws.NewCredentialsCache(aws.AnonymousCredentials{}),\n\t\tHTTPClient: httpClient,\n\t}\n\n\tc := NewReplicaClient()\n\tc.logger = slog.New(slog.NewTextHandler(io.Discard, nil))\n\tc.s3 = s3.NewFromConfig(cfg, func(o *s3.Options) {\n\t\to.APIOptions = append(o.APIOptions, c.middlewareOption())\n\t\to.APIOptions = append(o.APIOptions, func(stack *middleware.Stack) error {\n\t\t\treturn stack.Finalize.Add(\n\t\t\t\tmiddleware.FinalizeMiddlewareFunc(\n\t\t\t\t\t\"InjectPreexistingContentMD5\",\n\t\t\t\t\tfunc(ctx context.Context, in middleware.FinalizeInput, next middleware.FinalizeHandler) (\n\t\t\t\t\t\tout middleware.FinalizeOutput, metadata middleware.Metadata, err error,\n\t\t\t\t\t) {\n\t\t\t\t\t\tif req, ok := in.Request.(*smithyhttp.Request); ok {\n\t\t\t\t\t\t\treq.Header.Set(\"Content-MD5\", preexistingMD5)\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn next.HandleFinalize(ctx, in)\n\t\t\t\t\t},\n\t\t\t\t),\n\t\t\t\tmiddleware.Before,\n\t\t\t)\n\t\t})\n\t})\n\tc.Bucket = \"test-bucket\"\n\tc.Path = \"test-path\"\n\n\tfiles := []*ltx.FileInfo{\n\t\t{Level: 0, MinTXID: 1, MaxTXID: 1},\n\t}\n\n\tif err := c.DeleteLTXFiles(context.Background(), files); err != nil {\n\t\tt.Fatalf(\"DeleteLTXFiles: %v\", err)\n\t}\n\tif callCount != 1 {\n\t\tt.Fatalf(\"unexpected call count: %d\", callCount)\n\t}\n}\n\n// TestReplicaClient_CredentialConfiguration tests credential setup\nfunc TestReplicaClient_CredentialConfiguration(t *testing.T) {\n\tt.Run(\"WithStaticCredentials\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.AccessKeyID = \"AKIAIOSFODNN7EXAMPLE\"\n\t\tc.SecretAccessKey = \"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\"\n\n\t\t// Verify credentials are set\n\t\tif c.AccessKeyID == \"\" || c.SecretAccessKey == \"\" {\n\t\t\tt.Error(\"expected credentials to be set\")\n\t\t}\n\t})\n\n\tt.Run(\"WithDefaultCredentialChain\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\t// Leave AccessKeyID and SecretAccessKey empty\n\n\t\t// Verify credentials are not set (will use default chain)\n\t\tif c.AccessKeyID != \"\" || c.SecretAccessKey != \"\" {\n\t\t\tt.Error(\"expected credentials to be empty for default chain\")\n\t\t}\n\t})\n}\n\n// TestReplicaClient_DefaultRegionUsage tests that DefaultRegion constant is used consistently\nfunc TestReplicaClient_DefaultRegionUsage(t *testing.T) {\n\t// Test that DefaultRegion is properly defined\n\tif DefaultRegion != \"us-east-1\" {\n\t\tt.Errorf(\"expected DefaultRegion to be 'us-east-1', got %s\", DefaultRegion)\n\t}\n\n\t// Test ParseHost uses DefaultRegion\n\tt.Run(\"ParseHost_MinIO\", func(t *testing.T) {\n\t\tbucket, region, endpoint, forcePathStyle := ParseHost(\"mybucket.localhost:9000\")\n\t\tif region != DefaultRegion {\n\t\t\tt.Errorf(\"expected region to be %s, got %s\", DefaultRegion, region)\n\t\t}\n\t\tif bucket != \"mybucket\" {\n\t\t\tt.Errorf(\"expected bucket to be 'mybucket', got %s\", bucket)\n\t\t}\n\t\tif !strings.Contains(endpoint, \"localhost:9000\") {\n\t\t\tt.Errorf(\"expected endpoint to contain 'localhost:9000', got %s\", endpoint)\n\t\t}\n\t\tif !forcePathStyle {\n\t\t\tt.Error(\"expected forcePathStyle to be true for MinIO\")\n\t\t}\n\t})\n}\n\nfunc TestMarshalDeleteObjects_EdgeCases(t *testing.T) {\n\tt.Run(\"EmptyObjects\", func(t *testing.T) {\n\t\tdeleteInput := &types.Delete{\n\t\t\tObjects: []types.ObjectIdentifier{},\n\t\t}\n\t\txml, err := marshalDeleteObjects(deleteInput)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"marshalDeleteObjects failed: %v\", err)\n\t\t}\n\t\tif !strings.Contains(string(xml), \" chars.txt\"\n\t\tdeleteInput := &types.Delete{\n\t\t\tObjects: []types.ObjectIdentifier{\n\t\t\t\t{Key: aws.String(key)},\n\t\t\t},\n\t\t}\n\t\txml, err := marshalDeleteObjects(deleteInput)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"marshalDeleteObjects failed: %v\", err)\n\t\t}\n\t\txmlStr := string(xml)\n\t\tif !strings.Contains(xmlStr, \"test/path with spaces & <special> chars.txt\") {\n\t\t\tt.Errorf(\"expected XML to properly escape special characters, got: %s\", xmlStr)\n\t\t}\n\t})\n\n\tt.Run(\"KeyWithUnicode\", func(t *testing.T) {\n\t\tkey := \"test/文件.txt\"\n\t\tdeleteInput := &types.Delete{\n\t\t\tObjects: []types.ObjectIdentifier{\n\t\t\t\t{Key: aws.String(key)},\n\t\t\t},\n\t\t}\n\t\txml, err := marshalDeleteObjects(deleteInput)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"marshalDeleteObjects failed: %v\", err)\n\t\t}\n\t\txmlStr := string(xml)\n\t\tif !strings.Contains(xmlStr, key) {\n\t\t\tt.Errorf(\"expected XML to contain unicode key, got: %s\", xmlStr)\n\t\t}\n\t})\n\n\tt.Run(\"LargeBatch\", func(t *testing.T) {\n\t\tconst count = 1000\n\t\tobjects := make([]types.ObjectIdentifier, count)\n\t\tfor i := 0; i < count; i++ {\n\t\t\tobjects[i] = types.ObjectIdentifier{\n\t\t\t\tKey: aws.String(string(rune('a' + (i % 26)))),\n\t\t\t}\n\t\t}\n\t\tdeleteInput := &types.Delete{\n\t\t\tObjects: objects,\n\t\t}\n\t\txml, err := marshalDeleteObjects(deleteInput)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"marshalDeleteObjects failed for %d objects: %v\", count, err)\n\t\t}\n\t\tif len(xml) == 0 {\n\t\t\tt.Error(\"expected non-empty XML output\")\n\t\t}\n\t})\n\n\tt.Run(\"NilOptionalFields\", func(t *testing.T) {\n\t\tdeleteInput := &types.Delete{\n\t\t\tObjects: []types.ObjectIdentifier{\n\t\t\t\t{\n\t\t\t\t\tKey: aws.String(\"test-key\"),\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t\txml, err := marshalDeleteObjects(deleteInput)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"marshalDeleteObjects failed: %v\", err)\n\t\t}\n\t\txmlStr := string(xml)\n\t\tif !strings.Contains(xmlStr, \"test-key\") {\n\t\t\tt.Errorf(\"expected Key element in XML, got: %s\", xmlStr)\n\t\t}\n\t\tif strings.Contains(xmlStr, \"\") {\n\t\t\tt.Error(\"expected no ETag element when nil\")\n\t\t}\n\t\tif strings.Contains(xmlStr, \"\") {\n\t\t\tt.Error(\"expected no VersionId element when nil\")\n\t\t}\n\t})\n\n\tt.Run(\"QuietFlag\", func(t *testing.T) {\n\t\tdeleteInput := &types.Delete{\n\t\t\tObjects: []types.ObjectIdentifier{\n\t\t\t\t{Key: aws.String(\"test\")},\n\t\t\t},\n\t\t\tQuiet: aws.Bool(true),\n\t\t}\n\t\txml, err := marshalDeleteObjects(deleteInput)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"marshalDeleteObjects failed: %v\", err)\n\t\t}\n\t\txmlStr := string(xml)\n\t\tif !strings.Contains(xmlStr, \"true\") {\n\t\t\tt.Errorf(\"expected Quiet element to be true, got: %s\", xmlStr)\n\t\t}\n\t})\n}\n\nfunc TestEncodeObjectIdentifier_AllFields(t *testing.T) {\n\tt.Run(\"AllFieldsPopulated\", func(t *testing.T) {\n\t\ttimestamp, err := time.Parse(time.RFC3339, \"2023-01-01T00:00:00Z\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"failed to parse timestamp: %v\", err)\n\t\t}\n\t\tdeleteInput := &types.Delete{\n\t\t\tObjects: []types.ObjectIdentifier{\n\t\t\t\t{\n\t\t\t\t\tKey: aws.String(\"my-object-key\"),\n\t\t\t\t\tETag: aws.String(\"abc123etag\"),\n\t\t\t\t\tVersionId: aws.String(\"version-456\"),\n\t\t\t\t\tLastModifiedTime: aws.Time(timestamp),\n\t\t\t\t\tSize: aws.Int64(12345),\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t\txml, err := marshalDeleteObjects(deleteInput)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"marshalDeleteObjects failed: %v\", err)\n\t\t}\n\t\txmlStr := string(xml)\n\n\t\tif !strings.Contains(xmlStr, \"my-object-key\") {\n\t\t\tt.Error(\"expected Key element\")\n\t\t}\n\t\tif !strings.Contains(xmlStr, \"abc123etag\") {\n\t\t\tt.Error(\"expected ETag element\")\n\t\t}\n\t\tif !strings.Contains(xmlStr, \"version-456\") {\n\t\t\tt.Error(\"expected VersionId element\")\n\t\t}\n\t\tif !strings.Contains(xmlStr, \"\") {\n\t\t\tt.Error(\"expected LastModifiedTime element\")\n\t\t}\n\t\tif !strings.Contains(xmlStr, \"12345\") {\n\t\t\tt.Error(\"expected Size element with value 12345\")\n\t\t}\n\t})\n\n\tt.Run(\"OnlyRequiredKey\", func(t *testing.T) {\n\t\tdeleteInput := &types.Delete{\n\t\t\tObjects: []types.ObjectIdentifier{\n\t\t\t\t{\n\t\t\t\t\tKey: aws.String(\"only-key\"),\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t\txml, err := marshalDeleteObjects(deleteInput)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"marshalDeleteObjects failed: %v\", err)\n\t\t}\n\t\txmlStr := string(xml)\n\n\t\tif !strings.Contains(xmlStr, \"only-key\") {\n\t\t\tt.Error(\"expected Key element\")\n\t\t}\n\t\tif strings.Contains(xmlStr, \"\") {\n\t\t\tt.Error(\"expected no ETag element when nil\")\n\t\t}\n\t\tif strings.Contains(xmlStr, \"\") {\n\t\t\tt.Error(\"expected no VersionId element when nil\")\n\t\t}\n\t})\n\n\tt.Run(\"FieldOrder\", func(t *testing.T) {\n\t\tdeleteInput := &types.Delete{\n\t\t\tObjects: []types.ObjectIdentifier{\n\t\t\t\t{\n\t\t\t\t\tKey: aws.String(\"test\"),\n\t\t\t\t\tETag: aws.String(\"etag1\"),\n\t\t\t\t\tVersionId: aws.String(\"v1\"),\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t\txml, err := marshalDeleteObjects(deleteInput)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"marshalDeleteObjects failed: %v\", err)\n\t\t}\n\t\txmlStr := string(xml)\n\n\t\tkeyIdx := strings.Index(xmlStr, \"\")\n\t\tetagIdx := strings.Index(xmlStr, \"\")\n\t\tversionIdx := strings.Index(xmlStr, \"\")\n\n\t\tif keyIdx == -1 || etagIdx == -1 || versionIdx == -1 {\n\t\t\tt.Fatal(\"missing expected elements\")\n\t\t}\n\t\tif etagIdx > keyIdx || keyIdx > versionIdx {\n\t\t\tt.Errorf(\"expected field order: ETag, Key, VersionId, got ETag@%d, Key@%d, VersionId@%d\", etagIdx, keyIdx, versionIdx)\n\t\t}\n\t})\n}\n\nfunc TestComputeDeleteObjectsContentMD5_Deterministic(t *testing.T) {\n\tdeleteInput := &types.Delete{\n\t\tObjects: []types.ObjectIdentifier{\n\t\t\t{Key: aws.String(\"key1\")},\n\t\t\t{Key: aws.String(\"key2\")},\n\t\t},\n\t}\n\n\tmd51, err := computeDeleteObjectsContentMD5(deleteInput)\n\tif err != nil {\n\t\tt.Fatalf(\"first call failed: %v\", err)\n\t}\n\n\tmd52, err := computeDeleteObjectsContentMD5(deleteInput)\n\tif err != nil {\n\t\tt.Fatalf(\"second call failed: %v\", err)\n\t}\n\n\tif md51 != md52 {\n\t\tt.Errorf(\"MD5 computation not deterministic: %q != %q\", md51, md52)\n\t}\n\n\tif md51 == \"\" {\n\t\tt.Error(\"expected non-empty MD5\")\n\t}\n}\n\n// TestParseHost tests URL parsing for various S3-compatible storage providers.\n// This test addresses issue #825 where Digital Ocean Space URLs were not correctly\n// extracting the bucket name.\nfunc TestParseHost(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\thost string\n\t\twantBucket string\n\t\twantRegion string\n\t\twantEndpoint string\n\t\twantForcePathStyle bool\n\t}{\n\t\t{\n\t\t\tname: \"Digital Ocean Space URL\",\n\t\t\thost: \"my-space.sgp1.digitaloceanspaces.com\",\n\t\t\twantBucket: \"my-space\",\n\t\t\twantRegion: \"sgp1\",\n\t\t\twantEndpoint: \"https://sgp1.digitaloceanspaces.com\",\n\t\t\twantForcePathStyle: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Digital Ocean Space different region\",\n\t\t\thost: \"test-bucket.nyc3.digitaloceanspaces.com\",\n\t\t\twantBucket: \"test-bucket\",\n\t\t\twantRegion: \"nyc3\",\n\t\t\twantEndpoint: \"https://nyc3.digitaloceanspaces.com\",\n\t\t\twantForcePathStyle: false,\n\t\t},\n\t\t{\n\t\t\tname: \"AWS S3 URL with region\",\n\t\t\thost: \"mybucket.s3.us-east-1.amazonaws.com\",\n\t\t\twantBucket: \"mybucket\",\n\t\t\twantRegion: \"us-east-1\",\n\t\t\twantEndpoint: \"\",\n\t\t\twantForcePathStyle: false,\n\t\t},\n\t\t{\n\t\t\tname: \"AWS S3 URL without region\",\n\t\t\thost: \"mybucket.s3.amazonaws.com\",\n\t\t\twantBucket: \"mybucket\",\n\t\t\twantRegion: \"\",\n\t\t\twantEndpoint: \"\",\n\t\t\twantForcePathStyle: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Backblaze B2\",\n\t\t\thost: \"mybucket.s3.us-west-004.backblazeb2.com\",\n\t\t\twantBucket: \"mybucket\",\n\t\t\twantRegion: \"us-west-004\",\n\t\t\twantEndpoint: \"https://s3.us-west-004.backblazeb2.com\",\n\t\t\twantForcePathStyle: true,\n\t\t},\n\t\t{\n\t\t\tname: \"MinIO with port\",\n\t\t\thost: \"mybucket.localhost:9000\",\n\t\t\twantBucket: \"mybucket\",\n\t\t\twantRegion: \"us-east-1\",\n\t\t\twantEndpoint: \"http://localhost:9000\",\n\t\t\twantForcePathStyle: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tbucket, region, endpoint, forcePathStyle := ParseHost(tt.host)\n\n\t\t\tif bucket != tt.wantBucket {\n\t\t\t\tt.Errorf(\"bucket = %q, want %q\", bucket, tt.wantBucket)\n\t\t\t}\n\t\t\tif region != tt.wantRegion {\n\t\t\t\tt.Errorf(\"region = %q, want %q\", region, tt.wantRegion)\n\t\t\t}\n\t\t\tif endpoint != tt.wantEndpoint {\n\t\t\t\tt.Errorf(\"endpoint = %q, want %q\", endpoint, tt.wantEndpoint)\n\t\t\t}\n\t\t\tif forcePathStyle != tt.wantForcePathStyle {\n\t\t\t\tt.Errorf(\"forcePathStyle = %v, want %v\", forcePathStyle, tt.wantForcePathStyle)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReplicaClient_AccessPointARN(t *testing.T) {\n\tt.Run(\"ARNAsBucketName\", func(t *testing.T) {\n\t\tarn := \"arn:aws:s3:us-east-2:123456789012:accesspoint/my-access-point\"\n\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = arn\n\t\tc.Region = \"us-east-2\"\n\t\tc.AccessKeyID = \"test-access-key\"\n\t\tc.SecretAccessKey = \"test-secret-key\"\n\n\t\tif c.Bucket != arn {\n\t\t\tt.Errorf(\"expected bucket to be ARN, got %s\", c.Bucket)\n\t\t}\n\t\tif c.Region != \"us-east-2\" {\n\t\t\tt.Errorf(\"expected region to be us-east-2, got %s\", c.Region)\n\t\t}\n\t})\n\n\tt.Run(\"ARNWithPath\", func(t *testing.T) {\n\t\tarn := \"arn:aws:s3:us-west-2:111122223333:accesspoint/prod-access-point\"\n\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = arn\n\t\tc.Path = \"my-db/replica\"\n\t\tc.Region = \"us-west-2\"\n\n\t\tif c.Bucket != arn {\n\t\t\tt.Errorf(\"expected bucket to be ARN, got %s\", c.Bucket)\n\t\t}\n\t\tif c.Path != \"my-db/replica\" {\n\t\t\tt.Errorf(\"expected path to be my-db/replica, got %s\", c.Path)\n\t\t}\n\t})\n\n\tt.Run(\"ARNRejectsPathStyle\", func(t *testing.T) {\n\t\tarn := \"arn:aws:s3:us-east-1:123456789012:accesspoint/test-ap\"\n\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = arn\n\t\tc.Path = \"replica\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"http://localhost:9000\"\n\t\tc.ForcePathStyle = true\n\t\tc.AccessKeyID = \"test-access-key\"\n\t\tc.SecretAccessKey = \"test-secret-key\"\n\n\t\tctx := context.Background()\n\t\tif err := c.Init(ctx); err != nil {\n\t\t\tt.Fatalf(\"Init() with ARN bucket should not fail: %v\", err)\n\t\t}\n\n\t\tdata := mustLTX(t)\n\t\t_, err := c.WriteLTXFile(ctx, 0, 2, 2, bytes.NewReader(data))\n\t\tif err == nil {\n\t\t\tt.Fatal(\"expected error when using path-style with ARN bucket\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"Path-style addressing cannot be used with ARN\") {\n\t\t\tt.Errorf(\"expected path-style ARN error, got: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestReplicaClient_S3DebugEnvVar(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tenvValue string\n\t\twantLogMode aws.ClientLogMode\n\t\twantWarning bool\n\t}{\n\t\t{\n\t\t\tname: \"Empty\",\n\t\t\tenvValue: \"\",\n\t\t\twantLogMode: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"Signing\",\n\t\t\tenvValue: \"signing\",\n\t\t\twantLogMode: aws.LogSigning,\n\t\t},\n\t\t{\n\t\t\tname: \"Request\",\n\t\t\tenvValue: \"request\",\n\t\t\twantLogMode: aws.LogRequest,\n\t\t},\n\t\t{\n\t\t\tname: \"RequestWithBody\",\n\t\t\tenvValue: \"request-with-body\",\n\t\t\twantLogMode: aws.LogRequestWithBody,\n\t\t},\n\t\t{\n\t\t\tname: \"Response\",\n\t\t\tenvValue: \"response\",\n\t\t\twantLogMode: aws.LogResponse,\n\t\t},\n\t\t{\n\t\t\tname: \"ResponseWithBody\",\n\t\t\tenvValue: \"response-with-body\",\n\t\t\twantLogMode: aws.LogResponseWithBody,\n\t\t},\n\t\t{\n\t\t\tname: \"Retries\",\n\t\t\tenvValue: \"retries\",\n\t\t\twantLogMode: aws.LogRetries,\n\t\t},\n\t\t{\n\t\t\tname: \"All\",\n\t\t\tenvValue: \"all\",\n\t\t\twantLogMode: aws.LogSigning | aws.LogRequest | aws.LogRequestWithBody | aws.LogResponse | aws.LogResponseWithBody | aws.LogRetries,\n\t\t},\n\t\t{\n\t\t\tname: \"CommaSeparated\",\n\t\t\tenvValue: \"signing,request,retries\",\n\t\t\twantLogMode: aws.LogSigning | aws.LogRequest | aws.LogRetries,\n\t\t},\n\t\t{\n\t\t\tname: \"CommaSeparatedWithSpaces\",\n\t\t\tenvValue: \"signing, request, retries\",\n\t\t\twantLogMode: aws.LogSigning | aws.LogRequest | aws.LogRetries,\n\t\t},\n\t\t{\n\t\t\tname: \"CaseInsensitive\",\n\t\t\tenvValue: \"SIGNING,REQUEST\",\n\t\t\twantLogMode: aws.LogSigning | aws.LogRequest,\n\t\t},\n\t\t{\n\t\t\tname: \"Unknown\",\n\t\t\tenvValue: \"invalid\",\n\t\t\twantLogMode: 0,\n\t\t\twantWarning: true,\n\t\t},\n\t\t{\n\t\t\tname: \"MixedValidAndInvalid\",\n\t\t\tenvValue: \"signing,invalid,request\",\n\t\t\twantLogMode: aws.LogSigning | aws.LogRequest,\n\t\t\twantWarning: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t// Always set env var (even to empty) to isolate tests from caller's environment\n\t\t\tt.Setenv(\"LITESTREAM_S3_DEBUG\", tt.envValue)\n\n\t\t\tgotLogMode := parseS3DebugEnv()\n\t\t\tif gotLogMode != tt.wantLogMode {\n\t\t\t\tt.Errorf(\"parseS3DebugEnv() = %v, want %v\", gotLogMode, tt.wantLogMode)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReplicaClient_TigrisConsistentHeader(t *testing.T) {\n\t// Test that non-Tigris endpoints do NOT send the X-Tigris-Consistent header.\n\t// The Tigris case (header sent) requires an actual Tigris endpoint and is\n\t// covered by Tigris integration tests.\n\tdata := mustLTX(t)\n\n\theaders := make(chan http.Header, 1)\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tdefer r.Body.Close()\n\t\t_, _ = io.Copy(io.Discard, r.Body)\n\n\t\tif r.Method == http.MethodPut {\n\t\t\tselect {\n\t\t\tcase headers <- r.Header.Clone():\n\t\t\tdefault:\n\t\t\t}\n\t\t\tw.Header().Set(\"ETag\", `\"test-etag\"`)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\treturn\n\t\t}\n\n\t\tw.WriteHeader(http.StatusOK)\n\t}))\n\tdefer server.Close()\n\n\tclient := NewReplicaClient()\n\tclient.Bucket = \"test-bucket\"\n\tclient.Path = \"replica\"\n\tclient.Region = \"us-east-1\"\n\tclient.Endpoint = server.URL // Non-Tigris endpoint\n\tclient.ForcePathStyle = true\n\tclient.AccessKeyID = \"test-access-key\"\n\tclient.SecretAccessKey = \"test-secret-key\"\n\n\tctx := context.Background()\n\tif err := client.Init(ctx); err != nil {\n\t\tt.Fatalf(\"Init() error: %v\", err)\n\t}\n\n\tif _, err := client.WriteLTXFile(ctx, 0, 2, 2, bytes.NewReader(data)); err != nil {\n\t\tt.Fatalf(\"WriteLTXFile() error: %v\", err)\n\t}\n\n\tselect {\n\tcase hdr := <-headers:\n\t\tif got := hdr.Get(\"X-Tigris-Consistent\"); got != \"\" {\n\t\t\tt.Fatalf(\"X-Tigris-Consistent header = %q, want empty (non-Tigris endpoint)\", got)\n\t\t}\n\tcase <-time.After(time.Second):\n\t\tt.Fatal(\"timeout waiting for PUT request\")\n\t}\n}\n\n// TestReplicaClient_SSE_C_Validation tests SSE-C configuration validation\nfunc TestReplicaClient_SSE_C_Validation(t *testing.T) {\n\t// Generate a valid 256-bit key (32 bytes)\n\tvalidKey := base64.StdEncoding.EncodeToString([]byte(\"12345678901234567890123456789012\"))\n\n\tt.Run(\"ValidSSECKey\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"https://s3.example.com\"\n\t\tc.SSECustomerKey = validKey\n\n\t\terr := c.validateSSEConfig()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected no error for valid SSE-C key, got: %v\", err)\n\t\t}\n\n\t\t// Verify algorithm was auto-set\n\t\tif c.SSECustomerAlgorithm != \"AES256\" {\n\t\t\tt.Errorf(\"expected algorithm to be AES256, got %q\", c.SSECustomerAlgorithm)\n\t\t}\n\n\t\t// Verify MD5 was auto-computed\n\t\tif c.SSECustomerKeyMD5 == \"\" {\n\t\t\tt.Error(\"expected MD5 to be auto-computed\")\n\t\t}\n\t})\n\n\tt.Run(\"InvalidBase64Key\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"https://s3.example.com\"\n\t\tc.SSECustomerKey = \"not-valid-base64!!!\"\n\n\t\terr := c.validateSSEConfig()\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for invalid base64 key\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"valid base64\") {\n\t\t\tt.Errorf(\"expected base64 error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"WrongKeyLength\", func(t *testing.T) {\n\t\t// 16-byte key instead of 32-byte\n\t\tshortKey := base64.StdEncoding.EncodeToString([]byte(\"1234567890123456\"))\n\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"https://s3.example.com\"\n\t\tc.SSECustomerKey = shortKey\n\n\t\terr := c.validateSSEConfig()\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for wrong key length\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"256-bit\") {\n\t\t\tt.Errorf(\"expected key length error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"InvalidAlgorithm\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"https://s3.example.com\"\n\t\tc.SSECustomerKey = validKey\n\t\tc.SSECustomerAlgorithm = \"AES128\" // Invalid\n\n\t\terr := c.validateSSEConfig()\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for invalid algorithm\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"AES256\") {\n\t\t\tt.Errorf(\"expected algorithm error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"MutualExclusivity\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.SSECustomerKey = validKey\n\t\tc.SSEKMSKeyID = \"arn:aws:kms:us-east-1:123456789:key/12345678-1234-1234-1234-123456789012\"\n\n\t\terr := c.validateSSEConfig()\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error when both SSE-C and SSE-KMS are set\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"mutually exclusive\") {\n\t\t\tt.Errorf(\"expected mutual exclusivity error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"HTTPEndpointBlockedExceptLocalhost\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"http://external-server.example.com\"\n\t\tc.SSECustomerKey = validKey\n\n\t\terr := c.validateSSEConfig()\n\t\tif err == nil {\n\t\t\tt.Error(\"expected error for HTTP endpoint with SSE-C\")\n\t\t}\n\t\tif !strings.Contains(err.Error(), \"HTTPS\") {\n\t\t\tt.Errorf(\"expected HTTPS requirement error, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"LocalhostHTTPAllowed\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"http://localhost:9000\"\n\t\tc.SSECustomerKey = validKey\n\n\t\terr := c.validateSSEConfig()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected localhost HTTP to be allowed, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"127.0.0.1HTTPAllowed\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"http://127.0.0.1:9000\"\n\t\tc.SSECustomerKey = validKey\n\n\t\terr := c.validateSSEConfig()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected 127.0.0.1 HTTP to be allowed, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"PrivateNetworkHTTPAllowed\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"http://192.168.1.100:9000\"\n\t\tc.SSECustomerKey = validKey\n\n\t\terr := c.validateSSEConfig()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected private network HTTP to be allowed, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"PrivateNetwork172RangeHTTPAllowed\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"http://172.17.0.2:9000\"\n\t\tc.SSECustomerKey = validKey\n\n\t\terr := c.validateSSEConfig()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected 172.x private network HTTP to be allowed, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"PrivateNetwork10RangeHTTPAllowed\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.Endpoint = \"http://10.0.0.5:9000\"\n\t\tc.SSECustomerKey = validKey\n\n\t\terr := c.validateSSEConfig()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected 10.x private network HTTP to be allowed, got: %v\", err)\n\t\t}\n\t})\n}\n\n// TestReplicaClient_SSE_KMS_Configuration tests SSE-KMS configuration\nfunc TestReplicaClient_SSE_KMS_Configuration(t *testing.T) {\n\tt.Run(\"ValidKMSKeyID\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.SSEKMSKeyID = \"arn:aws:kms:us-east-1:123456789:key/12345678-1234-1234-1234-123456789012\"\n\n\t\terr := c.validateSSEConfig()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected no error for valid KMS key ID, got: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"KMSKeyAlias\", func(t *testing.T) {\n\t\tc := NewReplicaClient()\n\t\tc.Bucket = \"test-bucket\"\n\t\tc.Region = \"us-east-1\"\n\t\tc.SSEKMSKeyID = \"alias/my-key\"\n\n\t\terr := c.validateSSEConfig()\n\t\tif err != nil {\n\t\t\tt.Errorf(\"expected no error for KMS key alias, got: %v\", err)\n\t\t}\n\t})\n}\n\n// TestReplicaClient_SSE_C_Headers tests that SSE-C headers are passed to S3 operations\nfunc TestReplicaClient_SSE_C_Headers(t *testing.T) {\n\tvalidKey := base64.StdEncoding.EncodeToString([]byte(\"12345678901234567890123456789012\"))\n\tkeyBytes, _ := base64.StdEncoding.DecodeString(validKey)\n\tkeyMD5Sum := md5.Sum(keyBytes)\n\texpectedMD5 := base64.StdEncoding.EncodeToString(keyMD5Sum[:])\n\n\tdata := mustLTX(t)\n\n\tt.Run(\"WriteLTXFile_SSEC\", func(t *testing.T) {\n\t\theaders := make(chan http.Header, 1)\n\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\tdefer r.Body.Close()\n\t\t\t_, _ = io.Copy(io.Discard, r.Body)\n\n\t\t\tif r.Method == http.MethodPut {\n\t\t\t\tselect {\n\t\t\t\tcase headers <- r.Header.Clone():\n\t\t\t\tdefault:\n\t\t\t\t}\n\t\t\t\tw.Header().Set(\"ETag\", `\"test-etag\"`)\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}))\n\t\tdefer server.Close()\n\n\t\tclient := NewReplicaClient()\n\t\tclient.Bucket = \"test-bucket\"\n\t\tclient.Path = \"replica\"\n\t\tclient.Region = \"us-east-1\"\n\t\tclient.Endpoint = server.URL\n\t\tclient.ForcePathStyle = true\n\t\tclient.AccessKeyID = \"test-access-key\"\n\t\tclient.SecretAccessKey = \"test-secret-key\"\n\t\tclient.SSECustomerKey = validKey\n\n\t\tctx := context.Background()\n\t\tif err := client.Init(ctx); err != nil {\n\t\t\tt.Fatalf(\"Init() error: %v\", err)\n\t\t}\n\n\t\tif _, err := client.WriteLTXFile(ctx, 0, 2, 2, bytes.NewReader(data)); err != nil {\n\t\t\tt.Fatalf(\"WriteLTXFile() error: %v\", err)\n\t\t}\n\n\t\tselect {\n\t\tcase hdr := <-headers:\n\t\t\tif got := hdr.Get(\"x-amz-server-side-encryption-customer-algorithm\"); got != \"AES256\" {\n\t\t\t\tt.Errorf(\"SSE-C algorithm header = %q, want AES256\", got)\n\t\t\t}\n\t\t\tif got := hdr.Get(\"x-amz-server-side-encryption-customer-key\"); got != validKey {\n\t\t\t\tt.Errorf(\"SSE-C key header = %q, want %q\", got, validKey)\n\t\t\t}\n\t\t\tif got := hdr.Get(\"x-amz-server-side-encryption-customer-key-md5\"); got != expectedMD5 {\n\t\t\t\tt.Errorf(\"SSE-C key MD5 header = %q, want %q\", got, expectedMD5)\n\t\t\t}\n\t\tcase <-time.After(time.Second):\n\t\t\tt.Fatal(\"timeout waiting for PUT request\")\n\t\t}\n\t})\n\n\tt.Run(\"OpenLTXFile_SSEC\", func(t *testing.T) {\n\t\theaders := make(chan http.Header, 1)\n\t\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\tif r.Method == http.MethodGet {\n\t\t\t\tselect {\n\t\t\t\tcase headers <- r.Header.Clone():\n\t\t\t\tdefault:\n\t\t\t\t}\n\t\t\t\tw.Header().Set(\"Content-Length\", \"100\")\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\tw.Write([]byte(\"test-data\"))\n\t\t\t\treturn\n\t\t\t}\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t}))\n\t\tdefer server.Close()\n\n\t\tclient := NewReplicaClient()\n\t\tclient.Bucket = \"test-bucket\"\n\t\tclient.Path = \"replica\"\n\t\tclient.Region = \"us-east-1\"\n\t\tclient.Endpoint = server.URL\n\t\tclient.ForcePathStyle = true\n\t\tclient.AccessKeyID = \"test-access-key\"\n\t\tclient.SecretAccessKey = \"test-secret-key\"\n\t\tclient.SSECustomerKey = validKey\n\n\t\tctx := context.Background()\n\t\tif err := client.Init(ctx); err != nil {\n\t\t\tt.Fatalf(\"Init() error: %v\", err)\n\t\t}\n\n\t\trc, err := client.OpenLTXFile(ctx, 0, 2, 2, 0, 0)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"OpenLTXFile() error: %v\", err)\n\t\t}\n\t\trc.Close()\n\n\t\tselect {\n\t\tcase hdr := <-headers:\n\t\t\tif got := hdr.Get(\"x-amz-server-side-encryption-customer-algorithm\"); got != \"AES256\" {\n\t\t\t\tt.Errorf(\"SSE-C algorithm header = %q, want AES256\", got)\n\t\t\t}\n\t\t\tif got := hdr.Get(\"x-amz-server-side-encryption-customer-key\"); got != validKey {\n\t\t\t\tt.Errorf(\"SSE-C key header = %q, want %q\", got, validKey)\n\t\t\t}\n\t\t\tif got := hdr.Get(\"x-amz-server-side-encryption-customer-key-md5\"); got != expectedMD5 {\n\t\t\t\tt.Errorf(\"SSE-C key MD5 header = %q, want %q\", got, expectedMD5)\n\t\t\t}\n\t\tcase <-time.After(time.Second):\n\t\t\tt.Fatal(\"timeout waiting for GET request\")\n\t\t}\n\t})\n}\n\n// TestReplicaClient_SSE_KMS_Headers tests that SSE-KMS headers are passed to write operations\nfunc TestReplicaClient_SSE_KMS_Headers(t *testing.T) {\n\tkmsKeyID := \"arn:aws:kms:us-east-1:123456789:key/12345678-1234-1234-1234-123456789012\"\n\tdata := mustLTX(t)\n\n\theaders := make(chan http.Header, 1)\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tdefer r.Body.Close()\n\t\t_, _ = io.Copy(io.Discard, r.Body)\n\n\t\tif r.Method == http.MethodPut {\n\t\t\tselect {\n\t\t\tcase headers <- r.Header.Clone():\n\t\t\tdefault:\n\t\t\t}\n\t\t\tw.Header().Set(\"ETag\", `\"test-etag\"`)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\treturn\n\t\t}\n\n\t\tw.WriteHeader(http.StatusOK)\n\t}))\n\tdefer server.Close()\n\n\tclient := NewReplicaClient()\n\tclient.Bucket = \"test-bucket\"\n\tclient.Path = \"replica\"\n\tclient.Region = \"us-east-1\"\n\tclient.Endpoint = server.URL\n\tclient.ForcePathStyle = true\n\tclient.AccessKeyID = \"test-access-key\"\n\tclient.SecretAccessKey = \"test-secret-key\"\n\tclient.SSEKMSKeyID = kmsKeyID\n\n\tctx := context.Background()\n\tif err := client.Init(ctx); err != nil {\n\t\tt.Fatalf(\"Init() error: %v\", err)\n\t}\n\n\tif _, err := client.WriteLTXFile(ctx, 0, 2, 2, bytes.NewReader(data)); err != nil {\n\t\tt.Fatalf(\"WriteLTXFile() error: %v\", err)\n\t}\n\n\tselect {\n\tcase hdr := <-headers:\n\t\tif got := hdr.Get(\"x-amz-server-side-encryption\"); got != \"aws:kms\" {\n\t\t\tt.Errorf(\"SSE-KMS encryption header = %q, want aws:kms\", got)\n\t\t}\n\t\tif got := hdr.Get(\"x-amz-server-side-encryption-aws-kms-key-id\"); got != kmsKeyID {\n\t\t\tt.Errorf(\"SSE-KMS key ID header = %q, want %q\", got, kmsKeyID)\n\t\t}\n\tcase <-time.After(time.Second):\n\t\tt.Fatal(\"timeout waiting for PUT request\")\n\t}\n}\n\n// TestReplicaClient_NoSSE_Headers tests that no SSE headers are sent when SSE is not configured\nfunc TestReplicaClient_NoSSE_Headers(t *testing.T) {\n\tdata := mustLTX(t)\n\n\theaders := make(chan http.Header, 1)\n\tserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tdefer r.Body.Close()\n\t\t_, _ = io.Copy(io.Discard, r.Body)\n\n\t\tif r.Method == http.MethodPut {\n\t\t\tselect {\n\t\t\tcase headers <- r.Header.Clone():\n\t\t\tdefault:\n\t\t\t}\n\t\t\tw.Header().Set(\"ETag\", `\"test-etag\"`)\n\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\treturn\n\t\t}\n\n\t\tw.WriteHeader(http.StatusOK)\n\t}))\n\tdefer server.Close()\n\n\tclient := NewReplicaClient()\n\tclient.Bucket = \"test-bucket\"\n\tclient.Path = \"replica\"\n\tclient.Region = \"us-east-1\"\n\tclient.Endpoint = server.URL\n\tclient.ForcePathStyle = true\n\tclient.AccessKeyID = \"test-access-key\"\n\tclient.SecretAccessKey = \"test-secret-key\"\n\t// No SSE configuration\n\n\tctx := context.Background()\n\tif err := client.Init(ctx); err != nil {\n\t\tt.Fatalf(\"Init() error: %v\", err)\n\t}\n\n\tif _, err := client.WriteLTXFile(ctx, 0, 2, 2, bytes.NewReader(data)); err != nil {\n\t\tt.Fatalf(\"WriteLTXFile() error: %v\", err)\n\t}\n\n\tselect {\n\tcase hdr := <-headers:\n\t\tif got := hdr.Get(\"x-amz-server-side-encryption-customer-algorithm\"); got != \"\" {\n\t\t\tt.Errorf(\"unexpected SSE-C algorithm header: %q\", got)\n\t\t}\n\t\tif got := hdr.Get(\"x-amz-server-side-encryption-customer-key\"); got != \"\" {\n\t\t\tt.Errorf(\"unexpected SSE-C key header: %q\", got)\n\t\t}\n\t\tif got := hdr.Get(\"x-amz-server-side-encryption\"); got != \"\" {\n\t\t\tt.Errorf(\"unexpected SSE-KMS header: %q\", got)\n\t\t}\n\tcase <-time.After(time.Second):\n\t\tt.Fatal(\"timeout waiting for PUT request\")\n\t}\n}\n\n// TestReplicaClient_R2ConcurrencyDefault tests that Cloudflare R2 endpoints get\n// Concurrency=2 by default to avoid their strict concurrent upload limits.\n// This is a regression test for issue #948.\nfunc TestReplicaClient_R2ConcurrencyDefault(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\turl string\n\t\twantConcurrency int\n\t}{\n\t\t{\n\t\t\tname: \"R2_DefaultConcurrency\",\n\t\t\turl: \"s3://mybucket/path?endpoint=https://account123.r2.cloudflarestorage.com\",\n\t\t\twantConcurrency: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"AWS_NoConcurrencyOverride\",\n\t\t\turl: \"s3://mybucket/path\",\n\t\t\twantConcurrency: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"MinIO_NoConcurrencyOverride\",\n\t\t\turl: \"s3://mybucket/path?endpoint=http://localhost:9000\",\n\t\t\twantConcurrency: 0,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tclient, err := litestream.NewReplicaClientFromURL(tt.url)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"NewReplicaClientFromURL() error: %v\", err)\n\t\t\t}\n\t\t\tc := client.(*ReplicaClient)\n\n\t\t\tif c.Concurrency != tt.wantConcurrency {\n\t\t\t\tt.Errorf(\"Concurrency = %d, want %d\", c.Concurrency, tt.wantConcurrency)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestReplicaClient_ProviderEndpointDetection tests the endpoint detection functions\n// used to apply provider-specific defaults.\nfunc TestReplicaClient_ProviderEndpointDetection(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tendpoint string\n\t\twantR2 bool\n\t\twantB2 bool\n\t\twantDO bool\n\t}{\n\t\t{\n\t\t\tname: \"CloudflareR2\",\n\t\t\tendpoint: \"https://accountid.r2.cloudflarestorage.com\",\n\t\t\twantR2: true,\n\t\t},\n\t\t{\n\t\t\tname: \"CloudflareR2_HTTP\",\n\t\t\tendpoint: \"http://accountid.r2.cloudflarestorage.com\",\n\t\t\twantR2: true,\n\t\t},\n\t\t{\n\t\t\tname: \"BackblazeB2\",\n\t\t\tendpoint: \"https://s3.us-west-002.backblazeb2.com\",\n\t\t\twantB2: true,\n\t\t},\n\t\t{\n\t\t\tname: \"DigitalOcean\",\n\t\t\tendpoint: \"https://sgp1.digitaloceanspaces.com\",\n\t\t\twantDO: true,\n\t\t},\n\t\t{\n\t\t\tname: \"AWS_S3\",\n\t\t\tendpoint: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"MinIO\",\n\t\t\tendpoint: \"http://localhost:9000\",\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tif got := litestream.IsCloudflareR2Endpoint(tt.endpoint); got != tt.wantR2 {\n\t\t\t\tt.Errorf(\"IsCloudflareR2Endpoint() = %v, want %v\", got, tt.wantR2)\n\t\t\t}\n\t\t\tif got := litestream.IsBackblazeEndpoint(tt.endpoint); got != tt.wantB2 {\n\t\t\t\tt.Errorf(\"IsBackblazeEndpoint() = %v, want %v\", got, tt.wantB2)\n\t\t\t}\n\t\t\tif got := litestream.IsDigitalOceanEndpoint(tt.endpoint); got != tt.wantDO {\n\t\t\t\tt.Errorf(\"IsDigitalOceanEndpoint() = %v, want %v\", got, tt.wantDO)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestReplicaClient_CustomEndpoint_DisablesChecksumFeatures tests that custom endpoints\n// (non-AWS S3) have SDK checksum features disabled to avoid aws-chunked encoding issues.\n// This addresses issues #895, #912, #940, #941, #947 where S3-compatible providers\n// don't support aws-chunked encoding or streaming checksums.\nfunc TestReplicaClient_CustomEndpoint_DisablesChecksumFeatures(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tendpoint string\n\t\twantChecksumCalc string\n\t\twantChecksumValid string\n\t\texpectCustomConfig bool\n\t}{\n\t\t{\n\t\t\tname: \"AWS_S3_NoCustomConfig\",\n\t\t\tendpoint: \"\",\n\t\t\texpectCustomConfig: false,\n\t\t},\n\t\t{\n\t\t\tname: \"R2_DisablesChecksums\",\n\t\t\tendpoint: \"https://account.r2.cloudflarestorage.com\",\n\t\t\texpectCustomConfig: true,\n\t\t},\n\t\t{\n\t\t\tname: \"B2_DisablesChecksums\",\n\t\t\tendpoint: \"https://s3.us-west-002.backblazeb2.com\",\n\t\t\texpectCustomConfig: true,\n\t\t},\n\t\t{\n\t\t\tname: \"MinIO_DisablesChecksums\",\n\t\t\tendpoint: \"http://localhost:9000\",\n\t\t\texpectCustomConfig: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tc := NewReplicaClient()\n\t\t\tc.Bucket = \"test-bucket\"\n\t\t\tc.Region = \"us-east-1\"\n\t\t\tc.Endpoint = tt.endpoint\n\t\t\tc.ForcePathStyle = true\n\t\t\tc.AccessKeyID = \"test\"\n\t\t\tc.SecretAccessKey = \"test\"\n\n\t\t\thasCustomEndpoint := c.Endpoint != \"\"\n\t\t\tif hasCustomEndpoint != tt.expectCustomConfig {\n\t\t\t\tt.Errorf(\"custom endpoint detection = %v, want %v\", hasCustomEndpoint, tt.expectCustomConfig)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestNewReplicaClientFromURL_QueryParamAliases(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\turl string\n\t\twantForcePathStyle bool\n\t\twantSkipVerify bool\n\t\twantConcurrency int\n\t\twantPartSize int64\n\t}{\n\t\t{\n\t\t\tname: \"forcePathStyle_camelCase\",\n\t\t\turl: \"s3://mybucket/path?forcePathStyle=true\",\n\t\t\twantForcePathStyle: true,\n\t\t},\n\t\t{\n\t\t\tname: \"force-path-style_hyphenated\",\n\t\t\turl: \"s3://mybucket/path?force-path-style=true\",\n\t\t\twantForcePathStyle: true,\n\t\t},\n\t\t{\n\t\t\tname: \"force-path-style_false\",\n\t\t\turl: \"s3://mybucket/path?endpoint=http://localhost:9000&force-path-style=false\",\n\t\t\twantForcePathStyle: false,\n\t\t},\n\t\t{\n\t\t\tname: \"skipVerify_camelCase\",\n\t\t\turl: \"s3://mybucket/path?skipVerify=true\",\n\t\t\twantSkipVerify: true,\n\t\t},\n\t\t{\n\t\t\tname: \"skip-verify_hyphenated\",\n\t\t\turl: \"s3://mybucket/path?skip-verify=true\",\n\t\t\twantSkipVerify: true,\n\t\t},\n\t\t{\n\t\t\tname: \"concurrency_url_param\",\n\t\t\turl: \"s3://mybucket/path?concurrency=3\",\n\t\t\twantConcurrency: 3,\n\t\t},\n\t\t{\n\t\t\tname: \"part-size_hyphenated\",\n\t\t\turl: \"s3://mybucket/path?part-size=10485760\",\n\t\t\twantPartSize: 10485760,\n\t\t},\n\t\t{\n\t\t\tname: \"partSize_camelCase\",\n\t\t\turl: \"s3://mybucket/path?partSize=10485760\",\n\t\t\twantPartSize: 10485760,\n\t\t},\n\t\t{\n\t\t\tname: \"all_params_combined\",\n\t\t\turl: \"s3://mybucket/path?force-path-style=true&skip-verify=true&concurrency=4&part-size=8388608\",\n\t\t\twantForcePathStyle: true,\n\t\t\twantSkipVerify: true,\n\t\t\twantConcurrency: 4,\n\t\t\twantPartSize: 8388608,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tclient, err := litestream.NewReplicaClientFromURL(tt.url)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"NewReplicaClientFromURL() error: %v\", err)\n\t\t\t}\n\t\t\tc := client.(*ReplicaClient)\n\n\t\t\tif c.ForcePathStyle != tt.wantForcePathStyle {\n\t\t\t\tt.Errorf(\"ForcePathStyle = %v, want %v\", c.ForcePathStyle, tt.wantForcePathStyle)\n\t\t\t}\n\t\t\tif c.SkipVerify != tt.wantSkipVerify {\n\t\t\t\tt.Errorf(\"SkipVerify = %v, want %v\", c.SkipVerify, tt.wantSkipVerify)\n\t\t\t}\n\t\t\tif c.Concurrency != tt.wantConcurrency {\n\t\t\t\tt.Errorf(\"Concurrency = %d, want %d\", c.Concurrency, tt.wantConcurrency)\n\t\t\t}\n\t\t\tif c.PartSize != tt.wantPartSize {\n\t\t\t\tt.Errorf(\"PartSize = %d, want %d\", c.PartSize, tt.wantPartSize)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestNewReplicaClientFromURL_EndpointEnvVar(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\turl string\n\t\tenvEndpoint string\n\t\twantEndpoint string\n\t\twantForcePathStyle bool\n\t}{\n\t\t{\n\t\t\tname: \"env_var_sets_endpoint\",\n\t\t\turl: \"s3://mybucket/path\",\n\t\t\tenvEndpoint: \"http://localhost:9000\",\n\t\t\twantEndpoint: \"http://localhost:9000\",\n\t\t\twantForcePathStyle: true,\n\t\t},\n\t\t{\n\t\t\tname: \"env_var_adds_https_scheme\",\n\t\t\turl: \"s3://mybucket/path\",\n\t\t\tenvEndpoint: \"s3.example.com\",\n\t\t\twantEndpoint: \"https://s3.example.com\",\n\t\t\twantForcePathStyle: true,\n\t\t},\n\t\t{\n\t\t\tname: \"query_param_overrides_env_var\",\n\t\t\turl: \"s3://mybucket/path?endpoint=http://other:9000\",\n\t\t\tenvEndpoint: \"http://localhost:9000\",\n\t\t\twantEndpoint: \"http://other:9000\",\n\t\t\twantForcePathStyle: true,\n\t\t},\n\t\t{\n\t\t\tname: \"env_var_respects_force_path_style_false\",\n\t\t\turl: \"s3://mybucket/path?force-path-style=false\",\n\t\t\tenvEndpoint: \"http://localhost:9000\",\n\t\t\twantEndpoint: \"http://localhost:9000\",\n\t\t\twantForcePathStyle: false,\n\t\t},\n\t\t{\n\t\t\tname: \"no_env_var_no_endpoint\",\n\t\t\turl: \"s3://mybucket/path\",\n\t\t\tenvEndpoint: \"\",\n\t\t\twantEndpoint: \"\",\n\t\t\twantForcePathStyle: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tt.Setenv(\"LITESTREAM_S3_ENDPOINT\", tt.envEndpoint)\n\n\t\t\tclient, err := litestream.NewReplicaClientFromURL(tt.url)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"NewReplicaClientFromURL() error: %v\", err)\n\t\t\t}\n\t\t\tc := client.(*ReplicaClient)\n\n\t\t\tif c.Endpoint != tt.wantEndpoint {\n\t\t\t\tt.Errorf(\"Endpoint = %q, want %q\", c.Endpoint, tt.wantEndpoint)\n\t\t\t}\n\t\t\tif c.ForcePathStyle != tt.wantForcePathStyle {\n\t\t\t\tt.Errorf(\"ForcePathStyle = %v, want %v\", c.ForcePathStyle, tt.wantForcePathStyle)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "scripts/README.md", "content": "# Utility Scripts\n\nUtility scripts for Litestream testing and distribution.\n\n## Overview\n\nThis directory contains utility scripts for post-test analysis and packaging. All long-running soak tests have been migrated to Go integration tests in `tests/integration/`.\n\n> **Note:** For all soak tests (2-8 hours), see the Go-based test suite in [tests/integration/](../tests/integration/README.md). The bash soak tests have been migrated to Go for better maintainability and cross-platform support\n\n## Prerequisites\n\n```bash\ngo build -o bin/litestream ./cmd/litestream\ngo build -o bin/litestream-test ./cmd/litestream-test\n```\n\n## Available Scripts\n\n### analyze-test-results.sh\n\nPost-test analysis tool for examining overnight test results.\n\n```bash\n./scripts/analyze-test-results.sh /tmp/litestream-overnight-\n```\n\n**Analyzes:**\n1. Test duration and timeline\n2. Compaction statistics\n3. Checkpoint frequency\n4. Error analysis and categorization\n5. Performance metrics\n6. Database growth patterns\n7. Replica file statistics\n8. Final validation results\n\n**Output:**\n- Analysis report written to `/analysis-report.txt`\n- Console summary with key findings\n- Error categorization and counts\n- Performance statistics\n\n**What it Reports:**\n- Total test duration\n- Number of compactions by interval\n- Checkpoint count and frequency\n- Error types and severity\n- Database size growth\n- WAL file patterns\n- Replica size and file counts\n- Success/failure summary\n\n**Use Cases:**\n- Post-overnight-test analysis\n- Comparing test runs\n- Identifying performance trends\n- Debugging test failures\n- Documenting test results\n\n### setup-homebrew-tap.sh\n\nHomebrew tap setup script for packaging and distribution.\n\n```bash\n./scripts/setup-homebrew-tap.sh\n```\n\n**Purpose:** Automates Homebrew tap setup for Litestream distribution. Not a test script per se, but part of the release process.\n\n## Usage\n\n### Analyzing Test Results\n\n```bash\nls /tmp/litestream-overnight-* -dt | head -1\n\n./scripts/analyze-test-results.sh $(ls /tmp/litestream-overnight-* -dt | head -1)\n```\n\n## Test Duration Guide\n\n| Duration | Use Case | Test Type | Expected Results |\n|----------|----------|-----------|------------------|\n| 5 minutes | CI/CD smoke test | Go integration tests | Basic functionality |\n| 30 minutes | Short integration | Go integration tests | Pattern detection |\n| 2-8 hours | Soak testing | Go soak tests (local only) | Full validation |\n\n> **Note:** All soak tests are now Go-based in `tests/integration/`. See [tests/integration/README.md](../tests/integration/README.md) for details on running comprehensive, MinIO, and overnight S3 soak tests.\n\n## Monitoring and Debugging\n\n### Real-time Monitoring\n\nAll tests create timestamped directories in `/tmp/`:\n\n```bash\nLATEST=$(ls /tmp/litestream-* -dt | head -1)\ntail -f $LATEST/logs/monitor.log\ntail -f $LATEST/logs/litestream.log\n```\n\n### Key Metrics to Watch\n\n**Database Growth:**\n- Should grow steadily\n- WAL file size should cycle (grow, checkpoint, reset)\n\n**Replica Statistics:**\n- Snapshot count should increase over time\n- LTX file count should grow then stabilize (compaction)\n- Replica size should be similar to database size\n\n**Operations:**\n- Compactions should occur at scheduled intervals\n- Checkpoints should happen regularly\n- Sync operations should complete successfully\n\n**Errors:**\n- Should be minimal or zero\n- Transient errors OK if recovered\n- Persistent errors indicate issues\n\n### Common Issues\n\n#### Test Fails to Start\n\nCheck binaries:\n```bash\nls -la bin/litestream bin/litestream-test\n```\n\nRebuild if needed:\n```bash\ngo build -o bin/litestream ./cmd/litestream\ngo build -o bin/litestream-test ./cmd/litestream-test\n```\n\n#### S3 Test Fails\n\nVerify credentials:\n```bash\naws s3 ls s3://$S3_BUCKET/\n```\n\nCheck environment variables:\n```bash\necho $AWS_ACCESS_KEY_ID\necho $AWS_SECRET_ACCESS_KEY\necho $S3_BUCKET\n```\n\n#### High Error Counts\n\nCheck log for error details:\n```bash\ngrep -i error /tmp/litestream-*/logs/litestream.log | head -20\n```\n\n#### Validation Fails\n\nCompare databases manually:\n```bash\nsqlite3 /tmp/litestream-*/test.db \"SELECT COUNT(*) FROM test_data\"\nsqlite3 /tmp/litestream-*/restored.db \"SELECT COUNT(*) FROM test_data\"\n```\n\n### Stopping Tests Early\n\nGo tests can be interrupted with Ctrl+C. They will cleanup gracefully via defer statements.\n\n## Test Artifacts\n\nAll tests create timestamped directories with comprehensive artifacts:\n\n```\n/tmp/litestream-overnight-/\n├── logs/\n│ ├── litestream.log # Litestream replication log\n│ ├── load.log # Load generator log\n│ ├── monitor.log # Real-time monitoring log\n│ ├── populate.log # Initial population log\n│ └── validate.log # Final validation log\n├── test.db # Source database\n├── test.db-wal # Write-ahead log\n├── test.db-shm # Shared memory file\n├── replica/ # Replica directory (file tests)\n│ └── ltx/ # LTX files\n└── restored.db # Restored database for validation\n```\n\n## Integration with Go Tests\n\nThese utility scripts complement the Go integration test suite:\n\n**Test Locations:**\n- `tests/integration/` → All integration and soak tests (Go-based)\n- `cmd/litestream-test/scripts/` → Scenario and debugging tests (bash, being phased out)\n- `scripts/` → Utilities only (this directory)\n\n**Testing Workflow:**\n1. Run quick integration tests during development\n2. Run full integration test suite before major changes\n3. Run soak tests (2-8h) locally before releases: `TestComprehensiveSoak`, `TestMinIOSoak`, `TestOvernightS3Soak`\n4. Analyze results with `analyze-test-results.sh`\n\n## Related Documentation\n\n- [Go Integration Tests](../tests/integration/README.md) - Complete Go-based test suite including soak tests\n- [litestream-test CLI Tool](../cmd/litestream-test/README.md) - Testing harness documentation\n- [Scenario Test Scripts](../cmd/litestream-test/scripts/README.md) - Focused test scenarios\n- [S3 Retention Testing](../cmd/litestream-test/S3-RETENTION-TESTING.md) - S3-specific testing\n" }, { "path": "scripts/analyze-test-results.sh", "content": "#!/bin/bash\nset -euo pipefail\n\nif [ $# -lt 1 ]; then\n echo \"Usage: $0 \"\n echo \"\"\n echo \"Analyzes overnight test results from the specified test directory.\"\n echo \"\"\n echo \"Example:\"\n echo \" $0 /tmp/litestream-overnight-20240924-120000\"\n exit 1\nfi\n\nTEST_DIR=\"$1\"\n\nif [ ! -d \"$TEST_DIR\" ]; then\n echo \"Error: Test directory does not exist: $TEST_DIR\"\n exit 1\nfi\n\nLOG_DIR=\"$TEST_DIR/logs\"\nANALYSIS_REPORT=\"$TEST_DIR/analysis-report.txt\"\n\necho \"================================================\"\necho \"Litestream Test Analysis Report\"\necho \"================================================\"\necho \"Test directory: $TEST_DIR\"\necho \"Analysis time: $(date)\"\necho \"\"\n\n{\n echo \"================================================\"\n echo \"Litestream Test Analysis Report\"\n echo \"================================================\"\n echo \"Test directory: $TEST_DIR\"\n echo \"Analysis time: $(date)\"\n echo \"\"\n\n echo \"1. TEST DURATION AND TIMELINE\"\n echo \"==============================\"\n if [ -f \"$LOG_DIR/litestream.log\" ]; then\n START_TIME=$(head -1 \"$LOG_DIR/litestream.log\" 2>/dev/null | grep -oE '[0-9]{4}/[0-9]{2}/[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}' | head -1 || echo \"Unknown\")\n END_TIME=$(tail -1 \"$LOG_DIR/litestream.log\" 2>/dev/null | grep -oE '[0-9]{4}/[0-9]{2}/[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}' | head -1 || echo \"Unknown\")\n echo \"Start time: $START_TIME\"\n echo \"End time: $END_TIME\"\n\n # Calculate duration if possible\n if command -v python3 >/dev/null 2>&1; then\n DURATION=$(python3 -c \"\nfrom datetime import datetime\ntry:\n start = datetime.strptime('$START_TIME', '%Y/%m/%d %H:%M:%S')\n end = datetime.strptime('$END_TIME', '%Y/%m/%d %H:%M:%S')\n duration = end - start\n hours = duration.total_seconds() / 3600\n print(f'Duration: {hours:.2f} hours')\nexcept:\n print('Duration: Unable to calculate')\n\" 2>/dev/null || echo \"Duration: Unable to calculate\")\n echo \"$DURATION\"\n fi\n fi\n echo \"\"\n\n echo \"2. DATABASE STATISTICS\"\n echo \"======================\"\n if [ -f \"$TEST_DIR/test.db\" ]; then\n DB_SIZE=$(stat -f%z \"$TEST_DIR/test.db\" 2>/dev/null || stat -c%s \"$TEST_DIR/test.db\" 2>/dev/null || echo \"0\")\n echo \"Final database size: $(numfmt --to=iec-i --suffix=B $DB_SIZE 2>/dev/null || echo \"$DB_SIZE bytes\")\"\n\n # Get row count if database is accessible\n ROW_COUNT=$(sqlite3 \"$TEST_DIR/test.db\" \"SELECT COUNT(*) FROM test_data\" 2>/dev/null || echo \"Unknown\")\n echo \"Total rows inserted: $ROW_COUNT\"\n\n # Get page statistics\n PAGE_COUNT=$(sqlite3 \"$TEST_DIR/test.db\" \"PRAGMA page_count\" 2>/dev/null || echo \"Unknown\")\n PAGE_SIZE=$(sqlite3 \"$TEST_DIR/test.db\" \"PRAGMA page_size\" 2>/dev/null || echo \"Unknown\")\n echo \"Database pages: $PAGE_COUNT (page size: $PAGE_SIZE bytes)\"\n fi\n echo \"\"\n\n echo \"3. REPLICATION STATISTICS\"\n echo \"=========================\"\n if [ -d \"$TEST_DIR/replica\" ]; then\n SNAPSHOT_COUNT=$(find \"$TEST_DIR/replica\" -name \"*.snapshot.lz4\" 2>/dev/null | wc -l | tr -d ' ')\n WAL_COUNT=$(find \"$TEST_DIR/replica\" -name \"*.wal.lz4\" 2>/dev/null | wc -l | tr -d ' ')\n REPLICA_SIZE=$(du -sh \"$TEST_DIR/replica\" 2>/dev/null | cut -f1)\n\n echo \"Snapshots created: $SNAPSHOT_COUNT\"\n echo \"WAL segments created: $WAL_COUNT\"\n echo \"Total replica size: $REPLICA_SIZE\"\n\n # Analyze snapshot intervals\n if [ \"$SNAPSHOT_COUNT\" -gt 1 ]; then\n echo \"\"\n echo \"Snapshot creation times:\"\n find \"$TEST_DIR/replica\" -name \"*.snapshot.lz4\" -exec stat -f \"%Sm\" -t \"%Y-%m-%d %H:%M:%S\" {} \\; 2>/dev/null | sort || \\\n find \"$TEST_DIR/replica\" -name \"*.snapshot.lz4\" -exec stat -c \"%y\" {} \\; 2>/dev/null | cut -d. -f1 | sort || echo \"Unable to get timestamps\"\n fi\n fi\n echo \"\"\n\n echo \"4. COMPACTION ANALYSIS\"\n echo \"======================\"\n if [ -f \"$LOG_DIR/litestream.log\" ]; then\n COMPACTION_COUNT=$(grep -c \"compacting\" \"$LOG_DIR/litestream.log\" 2>/dev/null || echo \"0\")\n echo \"Total compaction operations: $COMPACTION_COUNT\"\n\n # Count compactions by level\n echo \"\"\n echo \"Compactions by retention level:\"\n grep \"compacting\" \"$LOG_DIR/litestream.log\" 2>/dev/null | grep -oE \"retention=[0-9]+[hms]+\" | sort | uniq -c | sort -rn || echo \"No compaction data found\"\n\n # Show compaction timing patterns\n echo \"\"\n echo \"Compaction frequency (last 10):\"\n grep \"compacting\" \"$LOG_DIR/litestream.log\" 2>/dev/null | tail -10 | grep -oE \"[0-9]{2}:[0-9]{2}:[0-9]{2}\" || echo \"No timing data\"\n fi\n echo \"\"\n\n echo \"5. LOAD GENERATOR PERFORMANCE\"\n echo \"=============================\"\n if [ -f \"$LOG_DIR/load.log\" ]; then\n # Extract final statistics\n FINAL_STATS=$(tail -20 \"$LOG_DIR/load.log\" | grep \"Load generation complete\" -A 10 || echo \"\")\n if [ -n \"$FINAL_STATS\" ]; then\n echo \"$FINAL_STATS\"\n else\n # Try to get statistics from progress logs\n echo \"Load generator statistics:\"\n grep \"Load statistics\" \"$LOG_DIR/load.log\" | tail -5 || echo \"No statistics found\"\n fi\n fi\n echo \"\"\n\n echo \"6. ERROR ANALYSIS\"\n echo \"=================\"\n ERROR_COUNT=0\n WARNING_COUNT=0\n\n if [ -f \"$LOG_DIR/litestream.log\" ]; then\n ERROR_COUNT=$(grep -ic \"ERROR\\|error\" \"$LOG_DIR/litestream.log\" 2>/dev/null || echo \"0\")\n WARNING_COUNT=$(grep -ic \"WARN\\|warning\" \"$LOG_DIR/litestream.log\" 2>/dev/null || echo \"0\")\n\n echo \"Total errors: $ERROR_COUNT\"\n echo \"Total warnings: $WARNING_COUNT\"\n\n if [ \"$ERROR_COUNT\" -gt 0 ]; then\n echo \"\"\n echo \"Error types:\"\n grep -i \"ERROR\\|error\" \"$LOG_DIR/litestream.log\" | sed 's/.*ERROR[: ]*//' | cut -d' ' -f1-5 | sort | uniq -c | sort -rn | head -10\n fi\n\n # Check for specific issues\n echo \"\"\n echo \"Specific issues detected:\"\n BUSY_ERRORS=$(grep -c \"database is locked\\|SQLITE_BUSY\" \"$LOG_DIR/litestream.log\" 2>/dev/null || echo \"0\")\n TIMEOUT_ERRORS=$(grep -c \"timeout\\|timed out\" \"$LOG_DIR/litestream.log\" 2>/dev/null || echo \"0\")\n S3_ERRORS=$(grep -c \"S3\\|AWS\\|403\\|404\\|500\\|503\" \"$LOG_DIR/litestream.log\" 2>/dev/null || echo \"0\")\n\n [ \"$BUSY_ERRORS\" -gt 0 ] && echo \" - Database busy/locked errors: $BUSY_ERRORS\"\n [ \"$TIMEOUT_ERRORS\" -gt 0 ] && echo \" - Timeout errors: $TIMEOUT_ERRORS\"\n [ \"$S3_ERRORS\" -gt 0 ] && echo \" - S3/AWS errors: $S3_ERRORS\"\n fi\n echo \"\"\n\n echo \"7. CHECKPOINT ANALYSIS\"\n echo \"======================\"\n if [ -f \"$LOG_DIR/litestream.log\" ]; then\n CHECKPOINT_COUNT=$(grep -c \"checkpoint\" \"$LOG_DIR/litestream.log\" 2>/dev/null || echo \"0\")\n echo \"Total checkpoint operations: $CHECKPOINT_COUNT\"\n\n # Analyze checkpoint performance\n echo \"\"\n echo \"Checkpoint timing (last 10):\"\n grep \"checkpoint\" \"$LOG_DIR/litestream.log\" 2>/dev/null | tail -10 | grep -oE \"[0-9]{2}:[0-9]{2}:[0-9]{2}\" || echo \"No checkpoint data\"\n fi\n echo \"\"\n\n echo \"8. VALIDATION RESULTS\"\n echo \"====================\"\n if [ -f \"$LOG_DIR/validate.log\" ]; then\n echo \"Validation output:\"\n cat \"$LOG_DIR/validate.log\"\n elif [ -f \"$LOG_DIR/restore.log\" ]; then\n echo \"Restoration test results:\"\n tail -20 \"$LOG_DIR/restore.log\"\n else\n echo \"No validation/restoration data found\"\n fi\n echo \"\"\n\n echo \"9. RESOURCE USAGE\"\n echo \"================\"\n if [ -f \"$LOG_DIR/monitor.log\" ]; then\n echo \"Peak values from monitoring:\"\n\n # Extract peak database size\n MAX_DB_SIZE=$(grep \"Database size:\" \"$LOG_DIR/monitor.log\" | grep -oE \"[0-9]+[KMG]?i?B\" | sort -h | tail -1 || echo \"Unknown\")\n echo \" Peak database size: $MAX_DB_SIZE\"\n\n # Extract peak WAL size\n MAX_WAL_SIZE=$(grep \"WAL size:\" \"$LOG_DIR/monitor.log\" | grep -oE \"[0-9]+[KMG]?i?B\" | sort -h | tail -1 || echo \"Unknown\")\n echo \" Peak WAL size: $MAX_WAL_SIZE\"\n\n # Extract max WAL segments\n MAX_WAL_SEGS=$(grep \"WAL segments (total):\" \"$LOG_DIR/monitor.log\" | grep -oE \"[0-9]+\" | sort -n | tail -1 || echo \"Unknown\")\n echo \" Max WAL segments: $MAX_WAL_SEGS\"\n fi\n echo \"\"\n\n echo \"10. SUMMARY AND RECOMMENDATIONS\"\n echo \"===============================\"\n\n # Analyze test success\n TEST_SUCCESS=true\n ISSUES=()\n\n if [ \"$ERROR_COUNT\" -gt 100 ]; then\n TEST_SUCCESS=false\n ISSUES+=(\"High error count ($ERROR_COUNT errors)\")\n fi\n\n if [ -f \"$LOG_DIR/validate.log\" ] && grep -q \"failed\\|error\" \"$LOG_DIR/validate.log\" 2>/dev/null; then\n TEST_SUCCESS=false\n ISSUES+=(\"Validation failed\")\n fi\n\n if [ -f \"$LOG_DIR/litestream.log\" ] && ! grep -q \"compacting\" \"$LOG_DIR/litestream.log\" 2>/dev/null; then\n ISSUES+=(\"No compaction operations detected\")\n fi\n\n if [ \"$TEST_SUCCESS\" = true ] && [ ${#ISSUES[@]} -eq 0 ]; then\n echo \"✓ Test completed successfully!\"\n echo \"\"\n echo \"Key achievements:\"\n echo \" - Ran for intended duration\"\n echo \" - Successfully created $SNAPSHOT_COUNT snapshots\"\n echo \" - Performed $COMPACTION_COUNT compaction operations\"\n echo \" - Processed $ROW_COUNT database rows\"\n else\n echo \"⚠ Test completed with issues:\"\n for issue in \"${ISSUES[@]}\"; do\n echo \" - $issue\"\n done\n fi\n\n echo \"\"\n echo \"Recommendations:\"\n if [ \"$ERROR_COUNT\" -gt 50 ]; then\n echo \" - Investigate error patterns, particularly around resource contention\"\n fi\n\n if [ \"$COMPACTION_COUNT\" -lt 10 ]; then\n echo \" - Verify compaction configuration is working as expected\"\n fi\n\n if [ \"$BUSY_ERRORS\" -gt 10 ]; then\n echo \" - Consider adjusting checkpoint intervals or busy timeout settings\"\n fi\n\n echo \"\"\n echo \"Test artifacts location: $TEST_DIR\"\n\n} | tee \"$ANALYSIS_REPORT\"\n\necho \"\"\necho \"================================================\"\necho \"Analysis complete!\"\necho \"Report saved to: $ANALYSIS_REPORT\"\necho \"================================================\"\n" }, { "path": "scripts/run-upgrade-tests.sh", "content": "#!/bin/bash\nset -euo pipefail\n\n# Run the v0.3.x → v0.5.x upgrade integration tests locally.\n# Downloads the v0.3.13 binary if not already cached, builds the current\n# binary, and runs the Go integration test.\n\nSCRIPT_DIR=\"$(cd \"$(dirname \"$0\")\" && pwd)\"\nROOT_DIR=\"$(cd \"$SCRIPT_DIR/..\" && pwd)\"\nCACHE_DIR=\"$ROOT_DIR/.cache/litestream-v3\"\nV3_BIN=\"$CACHE_DIR/litestream\"\n\n# Detect platform.\nOS=\"$(uname -s | tr '[:upper:]' '[:lower:]')\"\nARCH=\"$(uname -m)\"\ncase \"$ARCH\" in\n x86_64) ARCH=\"amd64\" ;;\n aarch64) ARCH=\"arm64\" ;;\n arm64) ARCH=\"arm64\" ;;\n *)\n echo \"Unsupported architecture: $ARCH\"\n exit 1\n ;;\nesac\n\n# Download v0.3.13 binary if not cached.\nif [ ! -x \"$V3_BIN\" ]; then\n echo \"Downloading Litestream v0.3.13 (${OS}-${ARCH})...\"\n mkdir -p \"$CACHE_DIR\"\n\n case \"$OS\" in\n darwin)\n ASSET=\"litestream-v0.3.13-darwin-${ARCH}.zip\"\n gh release download v0.3.13 --repo benbjohnson/litestream --pattern \"$ASSET\" --dir \"$CACHE_DIR\" --clobber\n unzip -o \"$CACHE_DIR/$ASSET\" -d \"$CACHE_DIR\"\n rm -f \"$CACHE_DIR/$ASSET\"\n ;;\n linux)\n ASSET=\"litestream-v0.3.13-linux-${ARCH}.tar.gz\"\n gh release download v0.3.13 --repo benbjohnson/litestream --pattern \"$ASSET\" --dir \"$CACHE_DIR\" --clobber\n tar -xzf \"$CACHE_DIR/$ASSET\" -C \"$CACHE_DIR\"\n rm -f \"$CACHE_DIR/$ASSET\"\n ;;\n *)\n echo \"Unsupported OS: $OS\"\n exit 1\n ;;\n esac\n\n chmod +x \"$V3_BIN\"\n echo \"Cached v0.3.13 binary at $V3_BIN\"\nelse\n echo \"Using cached v0.3.13 binary at $V3_BIN\"\nfi\n\necho \"v0.3.13 version: $(\"$V3_BIN\" version)\"\n\n# Build current binaries.\necho \"Building current binaries...\"\ncd \"$ROOT_DIR\"\ngo build -o bin/litestream ./cmd/litestream\ngo build -o bin/litestream-test ./cmd/litestream-test\necho \"Current version: $(./bin/litestream version)\"\n\n# Run upgrade tests.\necho \"\"\necho \"Running upgrade integration tests...\"\nLITESTREAM_V3_BIN=\"$V3_BIN\" CGO_ENABLED=1 \\\n go test -v -tags=integration -timeout=10m ./tests/integration/... -run=TestUpgrade \"$@\"\n" }, { "path": "scripts/setup-homebrew-tap.sh", "content": "#!/bin/bash\nset -e\n\necho \"Setting up Homebrew tap repository for Litestream...\"\n\nREPO_NAME=\"homebrew-litestream\"\nGITHUB_USER=\"benbjohnson\"\n\necho \"This script will help you create the ${GITHUB_USER}/${REPO_NAME} repository.\"\necho \"\"\necho \"Prerequisites:\"\necho \"1. GitHub CLI (gh) must be installed and authenticated\"\necho \"2. You must have permission to create repositories under ${GITHUB_USER}\"\necho \"\"\nread -p \"Do you want to continue? (y/n) \" -n 1 -r\necho \"\"\n\nif [[ ! $REPLY =~ ^[Yy]$ ]]; then\n echo \"Aborted.\"\n exit 1\nfi\n\necho \"Creating repository ${GITHUB_USER}/${REPO_NAME}...\"\ngh repo create ${GITHUB_USER}/${REPO_NAME} \\\n --public \\\n --description \"Homebrew tap for Litestream\" \\\n --clone=false || echo \"Repository may already exist, continuing...\"\n\necho \"\"\necho \"Cloning repository...\"\nTEMP_DIR=$(mktemp -d)\ncd \"$TEMP_DIR\"\ngh repo clone ${GITHUB_USER}/${REPO_NAME} || git clone \"https://github.com/${GITHUB_USER}/${REPO_NAME}.git\"\n\ncd ${REPO_NAME}\n\necho \"Creating Formula directory...\"\nmkdir -p Formula\n\necho \"Creating README...\"\ncat > README.md << 'EOF'\n# Homebrew Tap for Litestream\n\nThis is the official Homebrew tap for [Litestream](https://github.com/benbjohnson/litestream).\n\n## Installation\n\n```bash\nbrew tap benbjohnson/litestream\nbrew install litestream\n```\n\n## Documentation\n\nFor more information about Litestream, visit:\n- [GitHub Repository](https://github.com/benbjohnson/litestream)\n- [Official Documentation](https://litestream.io)\n\n## License\n\nApache License 2.0\nEOF\n\necho \"Creating initial Formula placeholder...\"\ncat > Formula/.gitkeep << 'EOF'\n# This file ensures the Formula directory is tracked by git\n# GoReleaser will automatically create and update formula files here\nEOF\n\necho \"Committing initial structure...\"\ngit add .\ngit commit -m \"Initial tap structure\" || echo \"Nothing to commit\"\n\necho \"Pushing to GitHub...\"\ngit push origin main || git push origin master\n\necho \"\"\necho \"✅ Homebrew tap repository setup complete!\"\necho \"\"\necho \"Repository: https://github.com/${GITHUB_USER}/${REPO_NAME}\"\necho \"\"\necho \"Next steps:\"\necho \"1. Create a GitHub Personal Access Token with 'repo' scope\"\necho \"2. Add it as HOMEBREW_TAP_GITHUB_TOKEN secret in the main repository\"\necho \"3. GoReleaser will automatically update the tap on each release\"\n" }, { "path": "server.go", "content": "package litestream\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"log/slog\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/http/pprof\"\n\t\"os\"\n\t\"sync\"\n\t\"time\"\n)\n\n// SocketConfig configures the Unix socket for control commands.\ntype SocketConfig struct {\n\tEnabled bool `yaml:\"enabled\"`\n\tPath string `yaml:\"path\"`\n\tPermissions uint32 `yaml:\"permissions\"`\n}\n\n// DefaultSocketConfig returns the default socket configuration.\nfunc DefaultSocketConfig() SocketConfig {\n\treturn SocketConfig{\n\t\tEnabled: false,\n\t\tPath: \"/var/run/litestream.sock\",\n\t\tPermissions: 0600,\n\t}\n}\n\n// Server manages runtime control via Unix socket using HTTP.\ntype Server struct {\n\tstore *Store\n\n\t// SocketPath is the path to the Unix socket.\n\tSocketPath string\n\n\t// SocketPerms is the file permissions for the socket.\n\tSocketPerms uint32\n\n\t// PathExpander optionally expands paths (e.g., ~ expansion).\n\t// If nil, paths are used as-is.\n\tPathExpander func(string) (string, error)\n\n\t// Version is the version string to report in /info.\n\tVersion string\n\n\t// startedAt is set when the server starts.\n\tstartedAt time.Time\n\n\tsocketListener net.Listener\n\thttpServer *http.Server\n\n\tctx context.Context\n\tcancel context.CancelFunc\n\twg sync.WaitGroup\n\n\tlogger *slog.Logger\n}\n\n// NewServer creates a new Server instance.\nfunc NewServer(store *Store) *Server {\n\tctx, cancel := context.WithCancel(context.Background())\n\ts := &Server{\n\t\tstore: store,\n\t\tSocketPerms: 0600,\n\t\tctx: ctx,\n\t\tcancel: cancel,\n\t\tlogger: slog.Default().With(LogKeySystem, LogSystemServer),\n\t}\n\n\tmux := http.NewServeMux()\n\tmux.HandleFunc(\"POST /start\", s.handleStart)\n\tmux.HandleFunc(\"POST /stop\", s.handleStop)\n\tmux.HandleFunc(\"GET /txid\", s.handleTXID)\n\tmux.HandleFunc(\"POST /register\", s.handleRegister)\n\tmux.HandleFunc(\"POST /unregister\", s.handleUnregister)\n\tmux.HandleFunc(\"POST /sync\", s.handleSync)\n\tmux.HandleFunc(\"GET /list\", s.handleList)\n\tmux.HandleFunc(\"GET /info\", s.handleInfo)\n\n\t// pprof endpoints\n\tmux.HandleFunc(\"GET /debug/pprof/\", pprof.Index)\n\tmux.HandleFunc(\"GET /debug/pprof/cmdline\", pprof.Cmdline)\n\tmux.HandleFunc(\"GET /debug/pprof/profile\", pprof.Profile)\n\tmux.HandleFunc(\"GET /debug/pprof/symbol\", pprof.Symbol)\n\tmux.HandleFunc(\"GET /debug/pprof/trace\", pprof.Trace)\n\n\ts.httpServer = &http.Server{Handler: mux}\n\n\treturn s\n}\n\n// Start begins listening for control connections.\nfunc (s *Server) Start() error {\n\tif s.SocketPath == \"\" {\n\t\treturn fmt.Errorf(\"socket path required\")\n\t}\n\n\t// Check if socket file exists and is actually a socket before removing\n\tif info, err := os.Lstat(s.SocketPath); err == nil {\n\t\tif info.Mode()&os.ModeSocket != 0 {\n\t\t\tif err := os.Remove(s.SocketPath); err != nil {\n\t\t\t\treturn fmt.Errorf(\"remove existing socket: %w\", err)\n\t\t\t}\n\t\t} else {\n\t\t\treturn fmt.Errorf(\"socket path exists but is not a socket: %s\", s.SocketPath)\n\t\t}\n\t} else if !os.IsNotExist(err) {\n\t\treturn fmt.Errorf(\"check socket path: %w\", err)\n\t}\n\n\tlistener, err := net.Listen(\"unix\", s.SocketPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"listen on unix socket: %w\", err)\n\t}\n\ts.socketListener = listener\n\n\tif err := os.Chmod(s.SocketPath, os.FileMode(s.SocketPerms)); err != nil {\n\t\tlistener.Close()\n\t\treturn fmt.Errorf(\"chmod socket: %w\", err)\n\t}\n\n\t// Set startedAt after successful socket setup to ensure uptime reflects\n\t// the actual time the server became available.\n\ts.startedAt = time.Now()\n\n\ts.logger.Info(\"control socket listening\", \"path\", s.SocketPath)\n\n\ts.wg.Add(1)\n\tgo func() {\n\t\tdefer s.wg.Done()\n\t\tif err := s.httpServer.Serve(listener); err != nil && err != http.ErrServerClosed {\n\t\t\ts.logger.Error(\"http server error\", \"error\", err)\n\t\t}\n\t}()\n\n\treturn nil\n}\n\n// Close gracefully shuts down the control server.\nfunc (s *Server) Close() error {\n\ts.cancel()\n\n\tctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n\tdefer cancel()\n\n\tif s.httpServer != nil {\n\t\tif err := s.httpServer.Shutdown(ctx); err != nil {\n\t\t\ts.logger.Error(\"http server shutdown error\", \"error\", err)\n\t\t}\n\t}\n\ts.wg.Wait()\n\treturn nil\n}\n\n// expandPath expands the path using PathExpander if set.\nfunc (s *Server) expandPath(path string) (string, error) {\n\tif s.PathExpander != nil {\n\t\treturn s.PathExpander(path)\n\t}\n\treturn path, nil\n}\n\nfunc (s *Server) handleStart(w http.ResponseWriter, r *http.Request) {\n\tvar req StartRequest\n\tif err := json.NewDecoder(r.Body).Decode(&req); err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, \"invalid request body\", err.Error())\n\t\treturn\n\t}\n\n\tif req.Path == \"\" {\n\t\twriteJSONError(w, http.StatusBadRequest, \"path required\", nil)\n\t\treturn\n\t}\n\n\texpandedPath, err := s.expandPath(req.Path)\n\tif err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, fmt.Sprintf(\"invalid path: %v\", err), nil)\n\t\treturn\n\t}\n\n\tctx := s.ctx\n\tif req.Timeout > 0 {\n\t\tvar cancel context.CancelFunc\n\t\tctx, cancel = context.WithTimeout(s.ctx, time.Duration(req.Timeout)*time.Second)\n\t\tdefer cancel()\n\t}\n\n\tif err := s.store.EnableDB(ctx, expandedPath); err != nil {\n\t\twriteJSONError(w, http.StatusInternalServerError, err.Error(), nil)\n\t\treturn\n\t}\n\n\twriteJSON(w, http.StatusOK, StartResponse{\n\t\tStatus: \"started\",\n\t\tPath: expandedPath,\n\t})\n}\n\nfunc (s *Server) handleStop(w http.ResponseWriter, r *http.Request) {\n\tvar req StopRequest\n\tif err := json.NewDecoder(r.Body).Decode(&req); err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, \"invalid request body\", err.Error())\n\t\treturn\n\t}\n\n\tif req.Path == \"\" {\n\t\twriteJSONError(w, http.StatusBadRequest, \"path required\", nil)\n\t\treturn\n\t}\n\n\texpandedPath, err := s.expandPath(req.Path)\n\tif err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, fmt.Sprintf(\"invalid path: %v\", err), nil)\n\t\treturn\n\t}\n\n\ttimeout := req.Timeout\n\tif timeout == 0 {\n\t\ttimeout = 30\n\t}\n\tctx, cancel := context.WithTimeout(s.ctx, time.Duration(timeout)*time.Second)\n\tdefer cancel()\n\n\tif err := s.store.DisableDB(ctx, expandedPath); err != nil {\n\t\twriteJSONError(w, http.StatusInternalServerError, err.Error(), nil)\n\t\treturn\n\t}\n\n\twriteJSON(w, http.StatusOK, StopResponse{\n\t\tStatus: \"stopped\",\n\t\tPath: expandedPath,\n\t})\n}\n\nfunc (s *Server) handleTXID(w http.ResponseWriter, r *http.Request) {\n\tpath := r.URL.Query().Get(\"path\")\n\tif path == \"\" {\n\t\twriteJSONError(w, http.StatusBadRequest, \"path required\", nil)\n\t\treturn\n\t}\n\n\texpandedPath, err := s.expandPath(path)\n\tif err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, fmt.Sprintf(\"invalid path: %v\", err), nil)\n\t\treturn\n\t}\n\n\tdb := s.store.FindDB(expandedPath)\n\tif db == nil {\n\t\twriteJSONError(w, http.StatusNotFound, \"database not found\", nil)\n\t\treturn\n\t}\n\n\tpos, err := db.Pos()\n\tif err != nil {\n\t\twriteJSONError(w, http.StatusInternalServerError, err.Error(), nil)\n\t\treturn\n\t}\n\n\twriteJSON(w, http.StatusOK, TXIDResponse{\n\t\tTXID: uint64(pos.TXID),\n\t})\n}\n\nfunc writeJSON(w http.ResponseWriter, status int, v interface{}) {\n\tw.Header().Set(\"Content-Type\", \"application/json\")\n\tw.WriteHeader(status)\n\tjson.NewEncoder(w).Encode(v)\n}\n\nfunc writeJSONError(w http.ResponseWriter, status int, message string, details interface{}) {\n\tw.Header().Set(\"Content-Type\", \"application/json\")\n\tw.WriteHeader(status)\n\tjson.NewEncoder(w).Encode(ErrorResponse{\n\t\tError: message,\n\t\tDetails: details,\n\t})\n}\n\n// StartRequest is the request body for the /start endpoint.\ntype StartRequest struct {\n\tPath string `json:\"path\"`\n\tTimeout int `json:\"timeout,omitempty\"`\n}\n\n// StartResponse is the response body for the /start endpoint.\ntype StartResponse struct {\n\tStatus string `json:\"status\"`\n\tPath string `json:\"path\"`\n}\n\n// StopRequest is the request body for the /stop endpoint.\ntype StopRequest struct {\n\tPath string `json:\"path\"`\n\tTimeout int `json:\"timeout,omitempty\"`\n}\n\n// StopResponse is the response body for the /stop endpoint.\ntype StopResponse struct {\n\tStatus string `json:\"status\"`\n\tPath string `json:\"path\"`\n}\n\n// ErrorResponse is returned when an error occurs.\ntype ErrorResponse struct {\n\tError string `json:\"error\"`\n\tDetails interface{} `json:\"details,omitempty\"`\n}\n\n// TXIDResponse is the response body for the /txid endpoint.\ntype TXIDResponse struct {\n\tTXID uint64 `json:\"txid\"`\n}\n\nfunc (s *Server) handleSync(w http.ResponseWriter, r *http.Request) {\n\tvar req SyncRequest\n\tif err := json.NewDecoder(r.Body).Decode(&req); err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, \"invalid request body\", err.Error())\n\t\treturn\n\t}\n\n\tif req.Path == \"\" {\n\t\twriteJSONError(w, http.StatusBadRequest, \"path required\", nil)\n\t\treturn\n\t}\n\n\texpandedPath, err := s.expandPath(req.Path)\n\tif err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, fmt.Sprintf(\"invalid path: %v\", err), nil)\n\t\treturn\n\t}\n\n\tctx := s.ctx\n\tif req.Wait && req.Timeout == 0 {\n\t\treq.Timeout = 30\n\t}\n\tif req.Timeout > 0 {\n\t\tvar cancel context.CancelFunc\n\t\tctx, cancel = context.WithTimeout(s.ctx, time.Duration(req.Timeout)*time.Second)\n\t\tdefer cancel()\n\t}\n\n\tresult, err := s.store.SyncDB(ctx, expandedPath, req.Wait)\n\tif err != nil {\n\t\tswitch {\n\t\tcase errors.Is(err, ErrDatabaseNotFound):\n\t\t\twriteJSONError(w, http.StatusNotFound, err.Error(), nil)\n\t\tcase errors.Is(err, ErrDatabaseNotOpen):\n\t\t\twriteJSONError(w, http.StatusConflict, err.Error(), nil)\n\t\tdefault:\n\t\t\twriteJSONError(w, http.StatusInternalServerError, err.Error(), nil)\n\t\t}\n\t\treturn\n\t}\n\n\tvar status string\n\tif !result.Changed {\n\t\tstatus = \"no_change\"\n\t} else if req.Wait {\n\t\tstatus = \"synced\"\n\t} else {\n\t\tstatus = \"synced_local\"\n\t}\n\n\twriteJSON(w, http.StatusOK, SyncResponse{\n\t\tStatus: status,\n\t\tPath: expandedPath,\n\t\tTXID: result.TXID,\n\t\tReplicatedTXID: result.ReplicatedTXID,\n\t})\n}\n\n// SyncRequest is the request body for the /sync endpoint.\ntype SyncRequest struct {\n\tPath string `json:\"path\"`\n\tWait bool `json:\"wait,omitempty\"`\n\tTimeout int `json:\"timeout,omitempty\"`\n}\n\n// SyncResponse is the response body for the /sync endpoint.\ntype SyncResponse struct {\n\tStatus string `json:\"status\"`\n\tPath string `json:\"path\"`\n\tTXID uint64 `json:\"txid\"`\n\tReplicatedTXID uint64 `json:\"replicated_txid\"`\n}\n\nfunc (s *Server) handleList(w http.ResponseWriter, _ *http.Request) {\n\tdbs := s.store.DBs()\n\tresp := ListResponse{\n\t\tDatabases: make([]DatabaseSummary, 0, len(dbs)),\n\t}\n\n\tfor _, db := range dbs {\n\t\tvar status string\n\t\tif db.IsOpen() {\n\t\t\tif db.Replica != nil && db.Replica.MonitorEnabled {\n\t\t\t\tstatus = \"replicating\"\n\t\t\t} else {\n\t\t\t\tstatus = \"open\"\n\t\t\t}\n\t\t} else {\n\t\t\tstatus = \"stopped\"\n\t\t}\n\n\t\tsummary := DatabaseSummary{\n\t\t\tPath: db.Path(),\n\t\t\tStatus: status,\n\t\t}\n\n\t\tif t := db.LastSuccessfulSyncAt(); !t.IsZero() {\n\t\t\tsummary.LastSyncAt = &t\n\t\t}\n\n\t\tresp.Databases = append(resp.Databases, summary)\n\t}\n\n\twriteJSON(w, http.StatusOK, resp)\n}\n\nfunc (s *Server) handleInfo(w http.ResponseWriter, _ *http.Request) {\n\tresp := InfoResponse{\n\t\tVersion: s.Version,\n\t\tPID: os.Getpid(),\n\t\tStartedAt: s.startedAt,\n\t\tUptimeSeconds: int64(time.Since(s.startedAt).Seconds()),\n\t\tDatabaseCount: len(s.store.DBs()),\n\t}\n\n\twriteJSON(w, http.StatusOK, resp)\n}\n\n// ListResponse is the response body for the /list endpoint.\ntype ListResponse struct {\n\tDatabases []DatabaseSummary `json:\"databases\"`\n}\n\n// DatabaseSummary contains summary information about a database.\ntype DatabaseSummary struct {\n\tPath string `json:\"path\"`\n\tStatus string `json:\"status\"`\n\n\t// LastSyncAt is the timestamp of the last successful replica sync.\n\t// This reflects when data was last successfully uploaded to the replica\n\t// storage backend, not just when the local WAL was processed.\n\tLastSyncAt *time.Time `json:\"last_sync_at,omitempty\"`\n}\n\n// InfoResponse is the response body for the /info endpoint.\ntype InfoResponse struct {\n\tVersion string `json:\"version\"`\n\tPID int `json:\"pid\"`\n\tUptimeSeconds int64 `json:\"uptime_seconds\"`\n\tStartedAt time.Time `json:\"started_at\"`\n\tDatabaseCount int `json:\"database_count\"`\n}\n\n// RegisterDatabaseRequest is the request body for the /register endpoint.\ntype RegisterDatabaseRequest struct {\n\tPath string `json:\"path\"`\n\tReplicaURL string `json:\"replica_url\"`\n}\n\n// RegisterDatabaseResponse is the response body for the /register endpoint.\ntype RegisterDatabaseResponse struct {\n\tStatus string `json:\"status\"`\n\tPath string `json:\"path\"`\n}\n\n// UnregisterDatabaseRequest is the request body for the /unregister endpoint.\ntype UnregisterDatabaseRequest struct {\n\tPath string `json:\"path\"`\n\tTimeout int `json:\"timeout,omitempty\"`\n}\n\n// UnregisterDatabaseResponse is the response body for the /unregister endpoint.\ntype UnregisterDatabaseResponse struct {\n\tStatus string `json:\"status\"`\n\tPath string `json:\"path\"`\n}\n\nfunc (s *Server) handleRegister(w http.ResponseWriter, r *http.Request) {\n\tvar req RegisterDatabaseRequest\n\tif err := json.NewDecoder(r.Body).Decode(&req); err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, \"invalid request body\", err.Error())\n\t\treturn\n\t}\n\n\tif req.Path == \"\" {\n\t\twriteJSONError(w, http.StatusBadRequest, \"path required\", nil)\n\t\treturn\n\t}\n\n\tif req.ReplicaURL == \"\" {\n\t\twriteJSONError(w, http.StatusBadRequest, \"replica_url required\", nil)\n\t\treturn\n\t}\n\n\texpandedPath, err := s.expandPath(req.Path)\n\tif err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, fmt.Sprintf(\"invalid path: %v\", err), nil)\n\t\treturn\n\t}\n\n\t// Check if database already exists.\n\tif existing := s.store.FindDB(expandedPath); existing != nil {\n\t\twriteJSON(w, http.StatusOK, RegisterDatabaseResponse{\n\t\t\tStatus: \"already_exists\",\n\t\t\tPath: expandedPath,\n\t\t})\n\t\treturn\n\t}\n\n\t// Create replica client from URL.\n\tclient, err := NewReplicaClientFromURL(req.ReplicaURL)\n\tif err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, fmt.Sprintf(\"invalid replica url: %v\", err), nil)\n\t\treturn\n\t}\n\n\t// Create new database.\n\tdb := NewDB(expandedPath)\n\n\t// Create replica and attach client.\n\treplica := NewReplica(db)\n\treplica.Client = client\n\tdb.Replica = replica\n\n\t// Register database with store (this also opens the database).\n\tif err := s.store.RegisterDB(db); err != nil {\n\t\twriteJSONError(w, http.StatusInternalServerError, fmt.Sprintf(\"failed to register database: %v\", err), nil)\n\t\treturn\n\t}\n\n\twriteJSON(w, http.StatusOK, RegisterDatabaseResponse{\n\t\tStatus: \"registered\",\n\t\tPath: expandedPath,\n\t})\n}\n\nfunc (s *Server) handleUnregister(w http.ResponseWriter, r *http.Request) {\n\tvar req UnregisterDatabaseRequest\n\tif err := json.NewDecoder(r.Body).Decode(&req); err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, \"invalid request body\", err.Error())\n\t\treturn\n\t}\n\n\tif req.Path == \"\" {\n\t\twriteJSONError(w, http.StatusBadRequest, \"path required\", nil)\n\t\treturn\n\t}\n\n\texpandedPath, err := s.expandPath(req.Path)\n\tif err != nil {\n\t\twriteJSONError(w, http.StatusBadRequest, fmt.Sprintf(\"invalid path: %v\", err), nil)\n\t\treturn\n\t}\n\n\t// Set up timeout context. Treat non-positive values as default.\n\ttimeout := req.Timeout\n\tif timeout <= 0 {\n\t\ttimeout = 30\n\t}\n\tctx, cancel := context.WithTimeout(s.ctx, time.Duration(timeout)*time.Second)\n\tdefer cancel()\n\n\t// Remove database from store (this also closes it).\n\tif err := s.store.UnregisterDB(ctx, expandedPath); err != nil {\n\t\twriteJSONError(w, http.StatusInternalServerError, fmt.Sprintf(\"failed to unregister database: %v\", err), nil)\n\t\treturn\n\t}\n\n\twriteJSON(w, http.StatusOK, UnregisterDatabaseResponse{\n\t\tStatus: \"unregistered\",\n\t\tPath: expandedPath,\n\t})\n}\n" }, { "path": "server_test.go", "content": "package litestream_test\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"os\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nvar testSocketCounter uint64\n\nfunc testSocketPath(t *testing.T) string {\n\tt.Helper()\n\tn := atomic.AddUint64(&testSocketCounter, 1)\n\tpath := fmt.Sprintf(\"/tmp/ls-test-%d.sock\", n)\n\tt.Cleanup(func() { os.Remove(path) })\n\treturn path\n}\n\nfunc TestServer_HandleInfo(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\tstore.CompactionMonitorEnabled = false\n\trequire.NoError(t, store.Open(t.Context()))\n\tdefer store.Close(t.Context())\n\n\tserver := litestream.NewServer(store)\n\tserver.SocketPath = testSocketPath(t)\n\tserver.Version = \"v1.0.0-test\"\n\trequire.NoError(t, server.Start())\n\tdefer server.Close()\n\n\tclient := newSocketClient(t, server.SocketPath)\n\tresp, err := client.Get(\"http://localhost/info\")\n\trequire.NoError(t, err)\n\tdefer resp.Body.Close()\n\n\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\tvar result litestream.InfoResponse\n\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\n\trequire.Equal(t, \"v1.0.0-test\", result.Version)\n\trequire.Greater(t, result.PID, 0)\n\trequire.Equal(t, 1, result.DatabaseCount)\n\trequire.False(t, result.StartedAt.IsZero())\n\trequire.GreaterOrEqual(t, result.UptimeSeconds, int64(0))\n}\n\nfunc TestServer_HandleList(t *testing.T) {\n\tt.Run(\"EmptyStore\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tresp, err := client.Get(\"http://localhost/list\")\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tvar result litestream.ListResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Empty(t, result.Databases)\n\t})\n\n\tt.Run(\"WithDatabases\", func(t *testing.T) {\n\t\tdb1, sqldb1 := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db1, sqldb1)\n\n\t\tdb2, sqldb2 := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db2, sqldb2)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db1, db2}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tresp, err := client.Get(\"http://localhost/list\")\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tvar result litestream.ListResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Len(t, result.Databases, 2)\n\n\t\t// Verify both databases are listed (order may vary).\n\t\tpaths := make(map[string]string)\n\t\tfor _, db := range result.Databases {\n\t\t\tpaths[db.Path] = db.Status\n\t\t}\n\t\trequire.Contains(t, paths, db1.Path())\n\t\trequire.Contains(t, paths, db2.Path())\n\t})\n\n\tt.Run(\"StatusOpenWhenMonitorDisabled\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t// MonitorEnabled is false by default in test helper.\n\t\trequire.False(t, db.Replica.MonitorEnabled)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tresp, err := client.Get(\"http://localhost/list\")\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\tvar result litestream.ListResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Len(t, result.Databases, 1)\n\n\t\t// Since MonitorEnabled is false, status should be \"open\" not \"replicating\".\n\t\trequire.Equal(t, \"open\", result.Databases[0].Status)\n\t})\n\n\tt.Run(\"StatusReplicatingWhenMonitorEnabled\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t// Enable the monitor to simulate active replication.\n\t\tdb.Replica.MonitorEnabled = true\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tresp, err := client.Get(\"http://localhost/list\")\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\tvar result litestream.ListResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Len(t, result.Databases, 1)\n\n\t\t// Since MonitorEnabled is true, status should be \"replicating\".\n\t\trequire.Equal(t, \"replicating\", result.Databases[0].Status)\n\t})\n\n\tt.Run(\"IncludesLastSyncAt\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t// Create some data and sync.\n\t\t_, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT)`)\n\t\trequire.NoError(t, err)\n\t\t_, err = sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (1)`)\n\t\trequire.NoError(t, err)\n\t\trequire.NoError(t, db.Sync(t.Context()))\n\t\trequire.NoError(t, db.Replica.Sync(t.Context()))\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tresp, err := client.Get(\"http://localhost/list\")\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tvar result litestream.ListResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Len(t, result.Databases, 1)\n\t\trequire.NotNil(t, result.Databases[0].LastSyncAt, \"LastSyncAt should be set after sync\")\n\t})\n}\n\nfunc TestServer_HandleStart(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\tstore.CompactionMonitorEnabled = false\n\trequire.NoError(t, store.Open(t.Context()))\n\tdefer store.Close(t.Context())\n\n\tserver := litestream.NewServer(store)\n\tserver.SocketPath = testSocketPath(t)\n\trequire.NoError(t, server.Start())\n\tdefer server.Close()\n\n\tt.Run(\"MissingPath\", func(t *testing.T) {\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tresp, err := client.Post(\"http://localhost/start\", \"application/json\", nil)\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusBadRequest, resp.StatusCode)\n\t})\n\n\tt.Run(\"DatabaseNotFound\", func(t *testing.T) {\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := `{\"path\": \"/nonexistent/db\"}`\n\t\tresp, err := client.Post(\"http://localhost/start\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusInternalServerError, resp.StatusCode)\n\t})\n}\n\nfunc TestServer_HandleStop(t *testing.T) {\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\tstore.CompactionMonitorEnabled = false\n\trequire.NoError(t, store.Open(t.Context()))\n\tdefer store.Close(t.Context())\n\n\tserver := litestream.NewServer(store)\n\tserver.SocketPath = testSocketPath(t)\n\trequire.NoError(t, server.Start())\n\tdefer server.Close()\n\n\tt.Run(\"MissingPath\", func(t *testing.T) {\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tresp, err := client.Post(\"http://localhost/stop\", \"application/json\", nil)\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusBadRequest, resp.StatusCode)\n\t})\n}\n\nfunc TestServer_HandleRegister(t *testing.T) {\n\tt.Run(\"MissingPath\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := `{\"replica_url\": \"file:///tmp/backup\"}`\n\t\tresp, err := client.Post(\"http://localhost/register\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusBadRequest, resp.StatusCode)\n\n\t\tvar result litestream.ErrorResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"path required\", result.Error)\n\t})\n\n\tt.Run(\"MissingReplicaURL\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := `{\"path\": \"/tmp/test.db\"}`\n\t\tresp, err := client.Post(\"http://localhost/register\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusBadRequest, resp.StatusCode)\n\n\t\tvar result litestream.ErrorResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"replica_url required\", result.Error)\n\t})\n\n\tt.Run(\"InvalidReplicaURL\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := `{\"path\": \"/tmp/test.db\", \"replica_url\": \"invalid://badscheme\"}`\n\t\tresp, err := client.Post(\"http://localhost/register\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusBadRequest, resp.StatusCode)\n\n\t\tvar result litestream.ErrorResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Contains(t, result.Error, \"invalid replica url\")\n\t})\n\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\t// Create a temporary database file.\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\ttestingutil.MustCloseDBs(t, db, sqldb)\n\t\tdbPath := db.Path()\n\n\t\t// Create a temp directory for backup.\n\t\tbackupDir := t.TempDir()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := fmt.Sprintf(`{\"path\": %q, \"replica_url\": \"file://%s\"}`, dbPath, backupDir)\n\t\tresp, err := client.Post(\"http://localhost/register\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tvar result litestream.RegisterDatabaseResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"registered\", result.Status)\n\t\trequire.Equal(t, dbPath, result.Path)\n\n\t\t// Verify database was registered with store.\n\t\trequire.Len(t, store.DBs(), 1)\n\t\trequire.Equal(t, dbPath, store.DBs()[0].Path())\n\t})\n\n\tt.Run(\"AlreadyExists\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\t// Try to register the same database again.\n\t\tbackupDir := t.TempDir()\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := fmt.Sprintf(`{\"path\": %q, \"replica_url\": \"file://%s\"}`, db.Path(), backupDir)\n\t\tresp, err := client.Post(\"http://localhost/register\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tvar result litestream.RegisterDatabaseResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"already_exists\", result.Status)\n\t})\n}\n\nfunc TestServer_HandleUnregister(t *testing.T) {\n\tt.Run(\"MissingPath\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := `{}`\n\t\tresp, err := client.Post(\"http://localhost/unregister\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusBadRequest, resp.StatusCode)\n\n\t\tvar result litestream.ErrorResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"path required\", result.Error)\n\t})\n\n\tt.Run(\"NotFoundIsIdempotent\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := `{\"path\": \"/nonexistent/db\"}`\n\t\tresp, err := client.Post(\"http://localhost/unregister\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\t// UnregisterDB is idempotent - returns success even if DB not found.\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tvar result litestream.UnregisterDatabaseResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"unregistered\", result.Status)\n\t})\n\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\t\tdbPath := db.Path()\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\trequire.Len(t, store.DBs(), 1)\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := fmt.Sprintf(`{\"path\": %q}`, dbPath)\n\t\tresp, err := client.Post(\"http://localhost/unregister\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tvar result litestream.UnregisterDatabaseResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"unregistered\", result.Status)\n\t\trequire.Equal(t, dbPath, result.Path)\n\n\t\t// Verify database was unregistered from store.\n\t\trequire.Empty(t, store.DBs())\n\t})\n}\n\nfunc TestServer_HandleSync(t *testing.T) {\n\tt.Run(\"MissingPath\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := `{}`\n\t\tresp, err := client.Post(\"http://localhost/sync\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusBadRequest, resp.StatusCode)\n\n\t\tvar result litestream.ErrorResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"path required\", result.Error)\n\t})\n\n\tt.Run(\"DatabaseNotFound\", func(t *testing.T) {\n\t\tstore := litestream.NewStore(nil, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := `{\"path\": \"/nonexistent/db\"}`\n\t\tresp, err := client.Post(\"http://localhost/sync\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusNotFound, resp.StatusCode)\n\t})\n\n\tt.Run(\"Success\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t_, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT)`)\n\t\trequire.NoError(t, err)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := fmt.Sprintf(`{\"path\": %q}`, db.Path())\n\t\tresp, err := client.Post(\"http://localhost/sync\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tvar result litestream.SyncResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"synced_local\", result.Status)\n\t\trequire.Equal(t, db.Path(), result.Path)\n\t\trequire.Greater(t, result.TXID, uint64(0))\n\t})\n\n\tt.Run(\"SuccessWithWait\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t_, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT)`)\n\t\trequire.NoError(t, err)\n\t\t_, err = sqldb.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (1)`)\n\t\trequire.NoError(t, err)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := fmt.Sprintf(`{\"path\": %q, \"wait\": true, \"timeout\": 30}`, db.Path())\n\t\tresp, err := client.Post(\"http://localhost/sync\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp.Body.Close()\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tvar result litestream.SyncResponse\n\t\trequire.NoError(t, json.NewDecoder(resp.Body).Decode(&result))\n\t\trequire.Equal(t, \"synced\", result.Status)\n\t\trequire.Equal(t, db.Path(), result.Path)\n\t\trequire.Greater(t, result.TXID, uint64(0))\n\t\trequire.Greater(t, result.ReplicatedTXID, uint64(0))\n\t})\n\n\tt.Run(\"NoChange\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\t_, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INT)`)\n\t\trequire.NoError(t, err)\n\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{{Level: 0}})\n\t\tstore.CompactionMonitorEnabled = false\n\t\trequire.NoError(t, store.Open(t.Context()))\n\t\tdefer store.Close(t.Context())\n\n\t\tserver := litestream.NewServer(store)\n\t\tserver.SocketPath = testSocketPath(t)\n\t\trequire.NoError(t, server.Start())\n\t\tdefer server.Close()\n\n\t\tclient := newSocketClient(t, server.SocketPath)\n\t\tbody := fmt.Sprintf(`{\"path\": %q}`, db.Path())\n\n\t\t// First sync should pick up the CREATE TABLE.\n\t\tresp1, err := client.Post(\"http://localhost/sync\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp1.Body.Close()\n\t\trequire.Equal(t, http.StatusOK, resp1.StatusCode)\n\n\t\tvar result1 litestream.SyncResponse\n\t\trequire.NoError(t, json.NewDecoder(resp1.Body).Decode(&result1))\n\t\trequire.Equal(t, \"synced_local\", result1.Status)\n\t\trequire.Greater(t, result1.TXID, uint64(0))\n\n\t\t// Second sync with no new writes should return no_change.\n\t\tresp2, err := client.Post(\"http://localhost/sync\", \"application/json\", io.NopCloser(stringReader(body)))\n\t\trequire.NoError(t, err)\n\t\tdefer resp2.Body.Close()\n\t\trequire.Equal(t, http.StatusOK, resp2.StatusCode)\n\n\t\tvar result2 litestream.SyncResponse\n\t\trequire.NoError(t, json.NewDecoder(resp2.Body).Decode(&result2))\n\t\trequire.Equal(t, \"no_change\", result2.Status)\n\t\trequire.Equal(t, result1.TXID, result2.TXID)\n\t})\n}\n\nfunc newSocketClient(t *testing.T, socketPath string) *http.Client {\n\tt.Helper()\n\treturn &http.Client{\n\t\tTimeout: 10 * time.Second,\n\t\tTransport: &http.Transport{\n\t\t\tDialContext: func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\t\treturn net.DialTimeout(\"unix\", socketPath, 10*time.Second)\n\t\t\t},\n\t\t},\n\t}\n}\n\ntype stringReaderType struct {\n\ts string\n\ti int\n}\n\nfunc stringReader(s string) *stringReaderType {\n\treturn &stringReaderType{s: s}\n}\n\nfunc (r *stringReaderType) Read(p []byte) (n int, err error) {\n\tif r.i >= len(r.s) {\n\t\treturn 0, io.EOF\n\t}\n\tn = copy(p, r.s[r.i:])\n\tr.i += n\n\treturn n, nil\n}\n" }, { "path": "sftp/replica_client.go", "content": "package sftp\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net\"\n\t\"net/url\"\n\t\"os\"\n\t\"path\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/pkg/sftp\"\n\t\"github.com/superfly/ltx\"\n\t\"golang.org/x/crypto/ssh\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\nfunc init() {\n\tlitestream.RegisterReplicaClientFactory(\"sftp\", NewReplicaClientFromURL)\n}\n\n// ReplicaClientType is the client type for this package.\nconst ReplicaClientType = \"sftp\"\n\n// Default settings for replica client.\nconst (\n\tDefaultDialTimeout = 30 * time.Second\n)\n\nvar _ litestream.ReplicaClient = (*ReplicaClient)(nil)\n\n// ReplicaClient is a client for writing LTX files over SFTP.\ntype ReplicaClient struct {\n\tmu sync.Mutex\n\tsshClient *ssh.Client\n\tsftpClient *sftp.Client\n\tlogger *slog.Logger\n\n\t// SFTP connection info\n\tHost string\n\tUser string\n\tPassword string\n\tPath string\n\tKeyPath string\n\tHostKey string\n\tDialTimeout time.Duration\n\n\t// ConcurrentWrites enables concurrent writes for better performance.\n\t// Note: This makes resuming failed transfers unsafe.\n\tConcurrentWrites bool\n}\n\n// NewReplicaClient returns a new instance of ReplicaClient.\nfunc NewReplicaClient() *ReplicaClient {\n\treturn &ReplicaClient{\n\t\tlogger: slog.Default().WithGroup(ReplicaClientType),\n\t\tDialTimeout: DefaultDialTimeout,\n\t\tConcurrentWrites: true, // Default to true for better performance\n\t}\n}\n\nfunc (c *ReplicaClient) SetLogger(logger *slog.Logger) {\n\tc.logger = logger.WithGroup(ReplicaClientType)\n}\n\n// NewReplicaClientFromURL creates a new ReplicaClient from URL components.\n// This is used by the replica client factory registration.\n// URL format: sftp://[user[:password]@]host[:port]/path\nfunc NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error) {\n\tclient := NewReplicaClient()\n\n\t// Extract credentials from userinfo\n\tif userinfo != nil {\n\t\tclient.User = userinfo.Username()\n\t\tclient.Password, _ = userinfo.Password()\n\t}\n\n\tclient.Host = host\n\tclient.Path = urlPath\n\n\tif client.Host == \"\" {\n\t\treturn nil, fmt.Errorf(\"host required for sftp replica URL\")\n\t}\n\tif client.User == \"\" {\n\t\treturn nil, fmt.Errorf(\"user required for sftp replica URL\")\n\t}\n\n\treturn client, nil\n}\n\n// Type returns \"sftp\" as the client type.\nfunc (c *ReplicaClient) Type() string {\n\treturn ReplicaClientType\n}\n\n// Init initializes the connection to SFTP. No-op if already initialized.\nfunc (c *ReplicaClient) Init(ctx context.Context) error {\n\t_, err := c.init(ctx)\n\treturn err\n}\n\n// init initializes the connection and returns the SFTP client.\nfunc (c *ReplicaClient) init(ctx context.Context) (_ *sftp.Client, err error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tif c.sftpClient != nil {\n\t\treturn c.sftpClient, nil\n\t}\n\n\tif c.User == \"\" {\n\t\treturn nil, fmt.Errorf(\"sftp user required\")\n\t}\n\n\t// Build SSH configuration & auth methods\n\tvar hostkey ssh.HostKeyCallback\n\tif c.HostKey != \"\" {\n\t\tvar pubkey, _, _, _, err = ssh.ParseAuthorizedKey([]byte(c.HostKey))\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"cannot parse sftp host key: %w\", err)\n\t\t}\n\t\thostkey = ssh.FixedHostKey(pubkey)\n\t} else {\n\t\tslog.Warn(\"sftp host key not verified\", \"host\", c.Host)\n\t\thostkey = ssh.InsecureIgnoreHostKey()\n\t}\n\tconfig := &ssh.ClientConfig{\n\t\tUser: c.User,\n\t\tHostKeyCallback: hostkey,\n\t\tBannerCallback: ssh.BannerDisplayStderr(),\n\t}\n\tif c.Password != \"\" {\n\t\tconfig.Auth = append(config.Auth, ssh.Password(c.Password))\n\t}\n\n\tif c.KeyPath != \"\" {\n\t\tbuf, err := os.ReadFile(c.KeyPath)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"sftp: cannot read sftp key path: %w\", err)\n\t\t}\n\n\t\tsigner, err := ssh.ParsePrivateKey(buf)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"sftp: cannot parse sftp key path: %w\", err)\n\t\t}\n\t\tconfig.Auth = append(config.Auth, ssh.PublicKeys(signer))\n\t}\n\n\t// Append standard port, if necessary.\n\thost := c.Host\n\tif _, _, err := net.SplitHostPort(c.Host); err != nil {\n\t\thost = net.JoinHostPort(c.Host, \"22\")\n\t}\n\n\t// Connect via SSH.\n\tif c.sshClient, err = ssh.Dial(\"tcp\", host, config); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Wrap connection with an SFTP client.\n\t// Configure options based on client settings\n\topts := []sftp.ClientOption{}\n\tif c.ConcurrentWrites {\n\t\topts = append(opts, sftp.UseConcurrentWrites(true))\n\t}\n\n\tif c.sftpClient, err = sftp.NewClient(c.sshClient, opts...); err != nil {\n\t\tc.sshClient.Close()\n\t\tc.sshClient = nil\n\t\treturn nil, err\n\t}\n\n\treturn c.sftpClient, nil\n}\n\n// DeleteAll deletes all LTX files.\nfunc (c *ReplicaClient) DeleteAll(ctx context.Context) (err error) {\n\tdefer func() { c.resetOnConnError(err) }()\n\n\tsftpClient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar dirs []string\n\twalker := sftpClient.Walk(c.Path)\n\tfor walker.Step() {\n\t\tif err := walker.Err(); os.IsNotExist(err) {\n\t\t\tcontinue\n\t\t} else if err != nil {\n\t\t\treturn fmt.Errorf(\"sftp: cannot walk path %q: %w\", walker.Path(), err)\n\t\t}\n\t\tif walker.Stat().IsDir() {\n\t\t\tdirs = append(dirs, walker.Path())\n\t\t\tcontinue\n\t\t}\n\n\t\tif err := sftpClient.Remove(walker.Path()); err != nil && !os.IsNotExist(err) {\n\t\t\treturn fmt.Errorf(\"sftp: cannot delete file %q: %w\", walker.Path(), err)\n\t\t}\n\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\t}\n\n\t// Remove directories in reverse order after they have been emptied.\n\tfor i := len(dirs) - 1; i >= 0; i-- {\n\t\tfilename := dirs[i]\n\t\tif err := sftpClient.RemoveDirectory(filename); err != nil && !os.IsNotExist(err) {\n\t\t\treturn fmt.Errorf(\"sftp: cannot delete directory %q: %w\", filename, err)\n\t\t}\n\t}\n\n\t// log.Printf(\"%s(%s): retainer: deleting all\", r.db.Path(), r.Name())\n\n\treturn nil\n}\n\n// LTXFiles returns an iterator over all available LTX files for a level.\n// SFTP uses file ModTime for timestamps, which is set via Chtimes() to preserve original timestamp.\n// The useMetadata parameter is ignored since ModTime always contains the accurate timestamp.\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, _ bool) (_ ltx.FileIterator, err error) {\n\tdefer func() { c.resetOnConnError(err) }()\n\n\tsftpClient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdir := litestream.LTXLevelDir(c.Path, level)\n\tfis, err := sftpClient.ReadDir(dir)\n\tif os.IsNotExist(err) {\n\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t} else if err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Iterate over every file and convert to metadata.\n\tinfos := make([]*ltx.FileInfo, 0, len(fis))\n\tfor _, fi := range fis {\n\t\tminTXID, maxTXID, err := ltx.ParseFilename(path.Base(fi.Name()))\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t} else if minTXID < seek {\n\t\t\tcontinue\n\t\t}\n\n\t\tinfos = append(infos, <x.FileInfo{\n\t\t\tLevel: level,\n\t\t\tMinTXID: minTXID,\n\t\t\tMaxTXID: maxTXID,\n\t\t\tSize: fi.Size(),\n\t\t\tCreatedAt: fi.ModTime().UTC(), // ModTime contains accurate timestamp from Chtimes()\n\t\t})\n\t}\n\n\treturn ltx.NewFileInfoSliceIterator(infos), nil\n}\n\n// WriteLTXFile writes a LTX file from rd into a remote file.\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, rd io.Reader) (info *ltx.FileInfo, err error) {\n\tdefer func() { c.resetOnConnError(err) }()\n\n\tsftpClient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfilename := litestream.LTXFilePath(c.Path, level, minTXID, maxTXID)\n\n\t// Use TeeReader to peek at LTX header while preserving data for upload\n\tvar buf bytes.Buffer\n\tteeReader := io.TeeReader(rd, &buf)\n\n\t// Extract timestamp from LTX header\n\thdr, _, err := ltx.PeekHeader(teeReader)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"extract timestamp from LTX header: %w\", err)\n\t}\n\ttimestamp := time.UnixMilli(hdr.Timestamp).UTC()\n\n\t// Combine buffered data with rest of reader\n\tfullReader := io.MultiReader(&buf, rd)\n\n\tif err := sftpClient.MkdirAll(path.Dir(filename)); err != nil {\n\t\treturn nil, fmt.Errorf(\"sftp: cannot make parent snapshot directory %q: %w\", path.Dir(filename), err)\n\t}\n\n\tf, err := sftpClient.OpenFile(filename, os.O_WRONLY|os.O_CREATE|os.O_TRUNC)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"sftp: cannot open snapshot file for writing: %w\", err)\n\t}\n\tdefer f.Close()\n\n\tn, err := io.Copy(f, fullReader)\n\tif err != nil {\n\t\treturn nil, err\n\t} else if err := f.Close(); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Set file ModTime to preserve original timestamp\n\tif err := sftpClient.Chtimes(filename, timestamp, timestamp); err != nil {\n\t\treturn nil, fmt.Errorf(\"sftp: cannot set file timestamps: %w\", err)\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Add(float64(n))\n\n\treturn <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tSize: n,\n\t\tCreatedAt: timestamp,\n\t}, nil\n}\n\n// OpenLTXFile returns a reader for an LTX file.\n// Returns os.ErrNotExist if no matching position is found.\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (_ io.ReadCloser, err error) {\n\tdefer func() { c.resetOnConnError(err) }()\n\n\tsftpClient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfilename := litestream.LTXFilePath(c.Path, level, minTXID, maxTXID)\n\tf, err := sftpClient.OpenFile(filename, os.O_RDONLY)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif offset > 0 {\n\t\tif _, err := f.Seek(offset, io.SeekStart); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"GET\").Inc()\n\n\tif size > 0 {\n\t\treturn internal.LimitReadCloser(f, size), nil\n\t}\n\treturn f, nil\n}\n\n// DeleteLTXFiles deletes LTX files with at the given positions.\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) (err error) {\n\tdefer func() { c.resetOnConnError(err) }()\n\n\tsftpClient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, info := range a {\n\t\tfilename := litestream.LTXFilePath(c.Path, info.Level, info.MinTXID, info.MaxTXID)\n\n\t\tc.logger.Debug(\"deleting ltx file\", \"level\", info.Level, \"minTXID\", info.MinTXID, \"maxTXID\", info.MaxTXID, \"path\", filename)\n\n\t\tif err := sftpClient.Remove(filename); err != nil && !os.IsNotExist(err) {\n\t\t\treturn fmt.Errorf(\"sftp: cannot delete ltx file %q: %w\", filename, err)\n\t\t}\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\t}\n\n\treturn nil\n}\n\n// Cleanup deletes path & directories after empty.\nfunc (c *ReplicaClient) Cleanup(ctx context.Context) (err error) {\n\tdefer func() { c.resetOnConnError(err) }()\n\n\tsftpClient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := sftpClient.RemoveDirectory(c.Path); err != nil && !os.IsNotExist(err) {\n\t\treturn fmt.Errorf(\"sftp: cannot delete path: %w\", err)\n\t}\n\treturn nil\n}\n\n// resetOnConnError closes & clears the client if a connection error occurs.\nfunc (c *ReplicaClient) resetOnConnError(err error) {\n\tif !errors.Is(err, sftp.ErrSSHFxConnectionLost) {\n\t\treturn\n\t}\n\n\tif c.sftpClient != nil {\n\t\tc.sftpClient.Close()\n\t\tc.sftpClient = nil\n\t}\n\tif c.sshClient != nil {\n\t\tc.sshClient.Close()\n\t\tc.sshClient = nil\n\t}\n}\n" }, { "path": "skills/litestream/SKILL.md", "content": "---\nname: litestream\ndescription: >-\n Expert knowledge for contributing to Litestream, a standalone disaster recovery\n tool for SQLite. Provides architectural understanding, code patterns, critical\n rules, and debugging procedures for WAL monitoring, LTX replication format,\n storage backend implementation, multi-level compaction, and SQLite page\n management. Use when working with Litestream source code, writing storage\n backends, debugging replication issues, implementing compaction logic, or\n handling SQLite WAL operations.\nlicense: Apache-2.0\nmetadata:\n author: benbjohnson\n version: \"1.0\"\n repository: https://github.com/benbjohnson/litestream\n---\n\n# Litestream Agent Skill\n\nLitestream is a standalone disaster recovery tool for SQLite. It runs as a\nbackground process, monitors the SQLite WAL (Write-Ahead Log), converts changes\nto immutable LTX files, and replicates them to cloud storage. It uses\n`modernc.org/sqlite` (pure Go, no CGO required).\n\n## Quick Start\n\n```bash\n# Build\ngo build -o bin/litestream ./cmd/litestream\n\n# Test (always use race detector)\ngo test -race -v ./...\n\n# Code quality\npre-commit run --all-files\n```\n\n## Critical Rules\n\nThese invariants must never be violated:\n\n### 1. Lock Page at 1GB\n\nSQLite reserves a page at byte offset 0x40000000 (1 GB). Always skip it during\nreplication and compaction. The page number varies by page size:\n\n| Page Size | Lock Page Number |\n|-----------|------------------|\n| 4 KB | 262145 |\n| 8 KB | 131073 |\n| 16 KB | 65537 |\n| 32 KB | 32769 |\n\n```go\nlockPgno := ltx.LockPgno(pageSize)\nif pgno == lockPgno {\n continue\n}\n```\n\n### 2. LTX Files Are Immutable\n\nOnce an LTX file is written, it must never be modified. New changes create new\nfiles. This guarantees point-in-time recovery integrity.\n\n### 3. Single Replica per Database\n\nEach database replicates to exactly one destination. The Replica component\nmanages replication mechanics; database state belongs in the DB layer.\n\n### 4. Read Local Before Remote During Compaction\n\nCloud storage is eventually consistent. Always read from local disk first:\n\n```go\nf, err := os.Open(db.LTXPath(info.Level, info.MinTXID, info.MaxTXID))\nif err == nil {\n return f, nil // Use local copy\n}\nreturn replica.Client.OpenLTXFile(...) // Fall back to remote\n```\n\n### 5. Preserve Timestamps During Compaction\n\nSet the compacted file's `CreatedAt` to the earliest source file timestamp to\nmaintain temporal granularity for point-in-time restoration.\n\n```go\ninfo.CreatedAt = oldestSourceFile.CreatedAt\n```\n\n### 6. Use Lock() Not RLock() for Writes\n\n```go\n// CORRECT\nr.mu.Lock()\ndefer r.mu.Unlock()\nr.pos = pos\n\n// WRONG - race condition\nr.mu.RLock()\ndefer r.mu.RUnlock()\nr.pos = pos\n```\n\n### 7. Atomic File Operations\n\nAlways write to a temp file then rename. Never write directly to the final path.\n\n```go\ntmpFile, err := os.CreateTemp(dir, \".tmp-*\")\n// ... write data, sync ...\nos.Rename(tmpFile.Name(), finalPath)\n```\n\n## Architecture\n\n### System Layers\n\n| Layer | File(s) | Responsibility |\n|---------|--------------------------|-------------------------------------------|\n| App | `cmd/litestream/` | CLI commands, YAML/env config |\n| Store | `store.go` | Multi-DB coordination, compaction |\n| DB | `db.go` | Single DB management, WAL monitoring |\n| Replica | `replica.go` | Replication to one destination |\n| Storage | `*/replica_client.go` | Backend implementations (S3, GCS, etc.) |\n\nDatabase state logic belongs in the DB layer, not the Replica layer.\n\n### ReplicaClient Interface\n\nAll storage backends implement this interface from `replica_client.go`:\n\n```go\ntype ReplicaClient interface {\n Type() string\n Init(ctx context.Context) error\n LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error)\n OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error)\n DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error\n DeleteAll(ctx context.Context) error\n}\n```\n\nKey contract details:\n- `OpenLTXFile` must return `os.ErrNotExist` when file is missing\n- `WriteLTXFile` must set `CreatedAt` from backend metadata or upload time\n- `LTXFiles` with `useMetadata=true` fetches accurate timestamps (for PIT restore)\n- `LTXFiles` with `useMetadata=false` uses fast timestamps (normal operations)\n\n### Lock Ordering\n\nAlways acquire locks in this order to prevent deadlocks:\n\n1. `Store.mu`\n2. `DB.mu`\n3. `DB.chkMu`\n4. `Replica.mu`\n\n### Core Components\n\n**DB** (`db.go`): Manages SQLite connection, WAL monitoring, checkpointing, and\nlong-running read transaction for consistency. Key fields: `path`, `db`, `rtx`\n(read transaction), `pageSize`, `notify` channel.\n\n**Replica** (`replica.go`): Tracks replication position (`ltx.Pos` with TXID,\nPageNo, Checksum). One replica per database.\n\n**Store** (`store.go`): Coordinates multiple databases and schedules compaction\nacross levels.\n\n## LTX File Format\n\nLTX (Log Transaction) files are immutable, checksummed archives of database\nchanges. Structure:\n\n```\n+------------------+\n| Header | 100 bytes (magic \"LTX1\", page size, TXID range, timestamp)\n+------------------+\n| Page Frames | 4-byte pgno + pageSize bytes data, per page\n+------------------+\n| Page Index | Binary search index for page lookup\n+------------------+\n| Trailer | 16 bytes (post-apply checksum, file checksum)\n+------------------+\n```\n\n### Naming Convention\n\n```\nFormat: MMMMMMMMMMMMMMMM-NNNNNNNNNNNNNNNN.ltx\nExample: 0000000000000001-0000000000000064.ltx (TXID 1-100)\n```\n\n### Compaction Levels\n\n```\nLevel 0: /ltx/0000/ Raw LTX files (no compaction)\nLevel 1: /ltx/0001/ Compacted periodically\nLevel 2: /ltx/0002/ Compacted less frequently\n```\n\nDefault compaction levels: L0 (raw), L1 (30s), L2 (5min), L3 (1h), plus daily\nsnapshots. Compaction merges files by deduplicating pages (latest version wins)\nand always skips the lock page.\n\n## Code Patterns\n\n### DO\n\n- Return errors immediately; let callers decide handling\n- Use `fmt.Errorf(\"context: %w\", err)` for error wrapping\n- Handle database state in the DB layer, not Replica\n- Use `db.verify()` to trigger snapshots (don't reimplement)\n- Test with race detector: `go test -race`\n- Use lazy iterators for `LTXFiles` (paginate, don't load all at once)\n\n### DON'T\n\n- Write data at the 1 GB lock page boundary\n- Modify LTX files after creation\n- Put database state logic in the Replica layer\n- Use `RLock()` when writing shared state\n- Write directly to final file paths (use temp + rename)\n- Ignore context cancellation in long operations\n- Return generic errors instead of `os.ErrNotExist` for missing files\n\n## Specialized Knowledge Areas\n\nLoad reference files on demand based on the task:\n\n| Task | Reference File |\n|-----------------------------------|-----------------------------------------|\n| Understanding system design | `references/ARCHITECTURE.md` |\n| Writing or reviewing code | `references/PATTERNS.md` |\n| Working with LTX files | `references/LTX_FORMAT.md` |\n| WAL monitoring or page operations | `references/SQLITE_INTERNALS.md` |\n| Implementing storage backends | `references/REPLICA_CLIENT_GUIDE.md` |\n| Writing or debugging tests | `references/TESTING_GUIDE.md` |\n\n## Common Debugging Procedures\n\n### Replication Not Working\n\n1. Verify WAL mode: `PRAGMA journal_mode` must return `wal`\n2. Check monitor interval and that the monitor goroutine is running\n3. Confirm `db.notify` channel is being signaled on WAL changes\n4. Check replica position: `replica.Pos()` should advance with writes\n5. Look for `os.ErrNotExist` from `OpenLTXFile` (file not replicated yet)\n\n### Large Database Issues (>1 GB)\n\n1. Verify lock page is being skipped: check `ltx.LockPgno(pageSize)`\n2. Test with multiple page sizes (4K, 8K, 16K, 32K)\n3. Run with databases both smaller and larger than 1 GB\n4. Ensure page iteration loops include the `continue` guard for lock page\n\n### Compaction Problems\n\n1. Confirm local L0 files exist before compaction reads them\n2. Check that `CreatedAt` timestamps are preserved (earliest source)\n3. Verify compaction level intervals in `Store.levels`\n4. Look for eventual consistency issues if reading from remote storage\n\n### Storage Backend Issues\n\n1. Return `os.ErrNotExist` for missing files (not generic errors)\n2. Support partial reads via `offset`/`size` in `OpenLTXFile`\n3. Handle context cancellation in all methods\n4. Test concurrent operations with `-race` flag\n5. For eventually consistent backends, add retry logic with backoff\n\n### Corrupted or Missing LTX Files\n\n1. Check logs for `LTXError` messages - they include context (Op, Path, Level, TXID) and recovery hints\n2. Common error messages: \"nonsequential page numbers\", \"non-contiguous transaction files\", \"ltx validation failed\"\n3. Manual fix: `litestream reset ` clears local LTX state and forces fresh snapshot on next sync (database file is not modified)\n4. Automatic fix: set `auto-recover: true` on the replica config to auto-reset on LTX errors (disabled by default)\n5. Reference: `cmd/litestream/reset.go`, `replica.go` (auto-recover logic), `db.go` (`ResetLocalState`)\n\n## Contribution Guidelines\n\n### What's Accepted\n\n- Bug fixes and patches (welcome)\n- Documentation improvements\n- Small code improvements and performance optimizations\n- Security vulnerability reports (report privately)\n\n### Discuss First\n\n- Feature requests: open an issue before implementing\n- Large changes: discuss approach in an issue first\n\n### Pre-Submit Checklist\n\n- [ ] Read relevant docs from the reference table above\n- [ ] Follow patterns in `references/PATTERNS.md`\n- [ ] Run `go test -race -v ./...`\n- [ ] Run `pre-commit run --all-files`\n- [ ] For page iteration: test with >1 GB databases\n- [ ] Show investigation evidence in PR (see CONTRIBUTING.md)\n\n## Testing\n\n```bash\n# Full test suite with race detection\ngo test -race -v ./...\n\n# Specific areas\ngo test -race -v -run TestReplica_Sync ./...\ngo test -race -v -run TestDB_Sync ./...\ngo test -race -v -run TestStore_CompactDB ./...\n\n# Coverage\ngo test -coverprofile=coverage.out ./...\ngo tool cover -html=coverage.out\n```\n\nKey testing areas:\n- Lock page handling with >1 GB databases and multiple page sizes\n- Race conditions in position updates, WAL monitoring, and checkpointing\n- Eventual consistency in storage backend operations\n- Atomic file operations and cleanup on error paths\n\n## Environment Validation\n\nRun `scripts/validate-setup.sh` to verify your development environment is\ncorrectly configured for Litestream development.\n" }, { "path": "skills/litestream/references/ARCHITECTURE.md", "content": "# Litestream Architecture Reference\n\nCondensed architectural reference for agents working on Litestream.\n\n## System Layers\n\n```\nApplication Layer cmd/litestream/ CLI commands, YAML/env config\nCore Layer store.go Multi-DB coordination, compaction scheduling\n db.go Single DB management, WAL monitoring, checkpoints\n replica.go Replication to single destination, position tracking\nStorage Abstraction replica_client.go ReplicaClient interface\nStorage Backends s3/, gs/, abs/, Backend-specific implementations\n oss/, file/, sftp/,\n nats/, webdav/\n```\n\nDatabase state logic belongs in the DB layer. The Replica layer handles\nreplication mechanics only.\n\n## Core Components\n\n### DB (db.go)\n\nThe heart of Litestream. Manages a single SQLite database.\n\nKey fields:\n- `path` / `metaPath`: Database and metadata paths\n- `db *sql.DB`: SQLite connection (via modernc.org/sqlite, pure Go)\n- `f *os.File`: Long-running file descriptor\n- `rtx *sql.Tx`: Long-running read transaction (prevents checkpoint past read point)\n- `pageSize int`: Database page size (critical for lock page calculation)\n- `mu sync.RWMutex`: Protects struct fields\n- `chkMu sync.RWMutex`: Checkpoint coordination lock\n- `notify chan struct{}`: WAL change notifications\n\nKey methods:\n- `Open()` / `Close()`: Lifecycle management\n- `monitor()`: Background WAL monitoring loop\n- `checkWAL()`: Detect WAL changes by comparing size/checksum\n- `Checkpoint(mode)`: Run SQLite checkpoint (PASSIVE, FULL, TRUNCATE)\n- `autoCheckpoint()`: Threshold-based checkpoint decisions\n- `Sync()`: Synchronize pending changes to replicas\n- `Compact()`: Merge LTX files at a given level\n\nConfiguration:\n- `MinCheckpointPageN`: Minimum pages for passive checkpoint\n- `TruncatePageN`: Emergency truncate checkpoint threshold\n- `CheckpointInterval`: Time-based passive checkpoint interval\n- `MonitorInterval`: WAL monitoring frequency\n- Note: RESTART checkpoint mode permanently removed (issue #724)\n\n### Replica (replica.go)\n\nManages replication to a single destination.\n\nKey fields:\n- `db *DB`: Parent database\n- `Client ReplicaClient`: Storage backend\n- `pos ltx.Pos`: Current replication position (TXID + PageNo + Checksum)\n- `mu sync.RWMutex`: Protects position\n\nPosition tracking: `SetPos()` MUST use `Lock()` not `RLock()`.\n\n### Store (store.go)\n\nCoordinates multiple databases and manages compaction.\n\nKey fields:\n- `dbs []*DB`: Managed databases\n- `levels CompactionLevels`: Compaction level configuration\n\nDefault compaction levels: L0 (raw), L1 (30s), L2 (5min), L3 (1h),\nplus daily snapshots.\n\n## WAL Monitoring\n\nThe monitor loop in `db.go` runs on a ticker:\n\n1. `checkWAL()` compares current WAL size/checksum to previous state\n2. If changed, notify replicas via `notifyReplicas()`\n3. Check if checkpoint needed via `shouldCheckpoint()`\n4. Run `autoCheckpoint()` if thresholds are met\n\nCheckpoint strategy:\n- WAL pages > `TruncatePageN` → TRUNCATE checkpoint (emergency)\n- WAL pages > `MinCheckpointPageN` → PASSIVE checkpoint\n- `CheckpointInterval` elapsed → PASSIVE checkpoint\n\n## Compaction Algorithm\n\nHigh-level flow (in `store.go`):\n\n1. Determine if level is due for compaction (`shouldCompact`)\n2. Enumerate level L-1 files via `ReplicaClient.LTXFiles`\n3. Prefer local copies (`os.Open(db.LTXPath(...))`) over remote reads\n4. Stream through `ltx.NewCompactor` (page deduplication, lock page skipping)\n5. Write merged file via `ReplicaClient.WriteLTXFile` for level L\n6. Set `CreatedAt` to earliest source file timestamp\n7. Update cached max file info; delete old L0 files when promoting to L1\n\n## State Management\n\n### Database States\n\nClosed → Opening → Open → Monitoring ↔ Syncing/Checkpointing → Closing → Closed\n\n### Replica States\n\nIdle → Starting → Monitoring ↔ Syncing/Uploading → Stopping → Idle\n\nError states retry with exponential backoff (1s initial, 1min max).\n\n## Initialization Flow\n\n1. `Store.Open()` iterates databases\n2. For each DB: open SQLite connection → read page size → create metadata dir →\n start long-running read transaction → initialize replicas → start monitor\n3. For each Replica: create with client → load previous position from metadata →\n validate against database → start sync goroutine\n4. Store starts compaction monitors\n\n## Error Handling\n\n### Error Categories\n\n- **Recoverable**: Network timeouts, temporary storage unavailability, lock contention\n- **Fatal**: Database corruption, invalid configuration, disk full\n- **Operational**: Checkpoint failures, compaction conflicts, sync delays\n\n### Error Propagation\n\n```\nReplicaClient.WriteLTXFile() error\n → Replica.Sync() error\n → DB.Sync() error\n → Store.monitorDB() // logs error, continues\n```\n\n## Lock Ordering\n\nAlways acquire in this order to prevent deadlocks:\n\n1. `Store.mu`\n2. `DB.mu`\n3. `DB.chkMu`\n4. `Replica.mu`\n\n## Goroutine Management\n\n- Use `sync.WaitGroup` for lifecycle tracking\n- Signal shutdown via `context.Cancel()`\n- Wait with timeout in `Close()` methods\n- Every goroutine must have a `defer wg.Done()`\n\n## Performance Characteristics\n\n| Operation | Time | Space |\n|-------------|------------|----------------|\n| WAL Monitor | O(1) | O(1) + metrics |\n| Page Write | O(1) | Original DB + WAL |\n| Compaction | O(n pages) | Temporary during merge |\n| Restoration | O(n·log m) | n=pages, m=files |\n| File List | O(k files) | Per-level |\n" }, { "path": "skills/litestream/references/LTX_FORMAT.md", "content": "# LTX Format Reference\n\nCondensed reference for agents working with Litestream's LTX file format.\n\n## Overview\n\nLTX (Log Transaction) is Litestream's custom format for storing database changes.\nLTX files are:\n- **Immutable**: Once written, never modified\n- **Self-contained**: Each file is independent\n- **Indexed**: Contains page index for efficient seeks\n- **Checksummed**: CRC-64 ECMA integrity verification\n\n## File Structure\n\n```\n+---------------------+\n| Header | 100 bytes\n+---------------------+\n| Page Frames | Variable (4-byte pgno + pageSize data per page)\n+---------------------+\n| Page Index | Binary search index\n+---------------------+\n| Trailer | 16 bytes\n+---------------------+\n\nFileSize = HeaderSize + (PageCount * (4 + PageSize)) + PageIndexSize + TrailerSize\n```\n\n## Header (100 bytes)\n\n```\nOffset Size Field\n0 4 Magic (\"LTX1\")\n4 4 Flags (bit 1 = NoChecksum)\n8 4 PageSize\n12 4 Commit (page count after applying)\n16 8 MinTXID\n24 8 MaxTXID\n32 8 Timestamp (ms since Unix epoch)\n40 8 PreApplyChecksum\n48 8 WALOffset (0 for snapshots)\n56 8 WALSize (0 for snapshots)\n64 4 WALSalt1\n68 4 WALSalt2\n72 8 NodeID\n80 20 Reserved (zeros)\n```\n\nThe version is implied by the magic string. `\"LTX1\"` → `ltx.Version == 2`.\n\n## Page Frames\n\nEach frame contains one database page:\n\n```\nOffset Size Field\n0 4 Page Number (1-based)\n4 PageSize Page Data\n```\n\nConstraints:\n- Pages written in sequential order during creation\n- Lock page at 1 GB boundary is never included\n- In compacted files, only the latest version of each page\n\n## Page Index\n\nBinary search index for efficient random access to pages. Use\n`ltx.DecodePageIndex()` to parse into a map of page number to\n`ltx.PageIndexElem`:\n\n```go\ntype PageIndexElem struct {\n Level int\n MinTXID TXID\n MaxTXID TXID\n Offset int64 // Byte offset of encoded payload\n Size int64 // Bytes occupied by encoded payload\n}\n```\n\n## Trailer (16 bytes)\n\n```\nOffset Size Field\n0 8 PostApplyChecksum (database checksum after applying file)\n8 8 FileChecksum (CRC-64 of entire file)\n```\n\nThe trailer is always at the end of the file. Read it by seeking\n`-TrailerSize` from `io.SeekEnd`.\n\n## File Naming Convention\n\n```\nFormat: MMMMMMMMMMMMMMMM-NNNNNNNNNNNNNNNN.ltx\n MinTXID (16 hex) MaxTXID (16 hex)\n\nExample: 0000000000000001-0000000000000064.ltx (TXID 1-100)\n 0000000000000065-00000000000000c8.ltx (TXID 101-200)\n```\n\nParse with `ltx.ParseFilename()`, format with `ltx.FormatFilename()`.\n\n## Compaction Levels\n\nLTX files are organized in levels for efficient storage:\n\n```\nLevel 0: /ltx/0000/ Raw LTX files (no compaction)\nLevel 1: /ltx/0001/ Compacted (default: every 30s)\nLevel 2: /ltx/0002/ Compacted (default: every 5min)\nLevel 3: /ltx/0003/ Compacted (default: every 1h)\nSnapshots: Full database state (daily)\n```\n\n### Compaction Process\n\n1. Enumerate level L-1 files\n2. Build page map (newer pages overwrite older)\n3. Write merged file skipping lock page\n4. Preserve earliest `CreatedAt` from source files\n5. Delete old L0 files when promoting to L1\n\n```go\n// Page deduplication: latest version wins\nfor _, file := range files {\n for _, page := range file.Pages {\n pageMap[page.Number] = page\n }\n}\n```\n\n## Checksums\n\nLTX uses CRC-64 ECMA checksums (`hash/crc64` with `crc64.ECMA` table):\n\n- **PreApplyChecksum**: Database state before applying this file\n- **PostApplyChecksum**: Database state after applying this file\n- **FileChecksum**: Integrity of the entire LTX file\n\n## Reading LTX Files\n\n```go\ndec := ltx.NewDecoder(reader)\nheader, err := dec.Header()\nfor {\n var hdr ltx.PageHeader\n data := make([]byte, header.PageSize)\n if err := dec.DecodePage(&hdr, data); err == io.EOF {\n break\n }\n // Process hdr.Pgno and data\n}\ntrailer := dec.Trailer()\n```\n\n## Writing LTX Files\n\n```go\nenc := ltx.NewEncoder(writer)\nenc.EncodeHeader(header)\nfor _, page := range pages {\n if page.Number == ltx.LockPgno(pageSize) {\n continue // Skip lock page\n }\n enc.EncodePage(pageHeader, pageData)\n}\nenc.EncodePageIndex(index)\nenc.EncodeTrailer()\nenc.Close()\n```\n\n## CLI Inspection\n\n```bash\nlitestream ltx /path/to/db.sqlite\nlitestream ltx s3://bucket/db\n```\n\nFor low-level inspection, use the Go API with `ltx.NewDecoder`.\n\n## WAL to LTX Conversion\n\nSQLite WAL frames are converted to LTX format during replication:\n1. Read WAL frames from the WAL file\n2. Skip the lock page\n3. Add checksums and build page index\n4. Write as immutable LTX file\n5. Upload to storage backend\n" }, { "path": "skills/litestream/references/PATTERNS.md", "content": "# Litestream Code Patterns Reference\n\nCondensed code patterns and anti-patterns for agents writing Litestream code.\n\n## Architectural Boundaries\n\n```\nDB Layer (db.go) → Database state, restoration, monitoring\nReplica Layer (replica.go) → Replication mechanics only\nStorage Layer → ReplicaClient implementations\n```\n\n### DO: Handle database state in DB layer\n\n```go\nfunc (db *DB) init() error {\n if db.needsRestore() {\n if err := db.restore(); err != nil {\n return err\n }\n }\n return db.replica.Start() // Replica focuses only on replication\n}\n```\n\n### DON'T: Put database state logic in Replica layer\n\n```go\n// WRONG\nfunc (r *Replica) Start() error {\n if needsRestore() { // Wrong layer!\n restoreDatabase()\n }\n}\n```\n\n## Atomic File Operations\n\n### DO: Write to temp file, then rename\n\n```go\nfunc writeFileAtomic(path string, data []byte) error {\n dir := filepath.Dir(path)\n tmpFile, err := os.CreateTemp(dir, \".tmp-*\")\n if err != nil {\n return fmt.Errorf(\"create temp file: %w\", err)\n }\n tmpPath := tmpFile.Name()\n\n defer func() {\n if tmpFile != nil {\n tmpFile.Close()\n os.Remove(tmpPath)\n }\n }()\n\n if _, err := tmpFile.Write(data); err != nil {\n return fmt.Errorf(\"write temp file: %w\", err)\n }\n if err := tmpFile.Sync(); err != nil {\n return fmt.Errorf(\"sync temp file: %w\", err)\n }\n if err := tmpFile.Close(); err != nil {\n return fmt.Errorf(\"close temp file: %w\", err)\n }\n tmpFile = nil\n\n if err := os.Rename(tmpPath, path); err != nil {\n os.Remove(tmpPath)\n return fmt.Errorf(\"rename to final path: %w\", err)\n }\n return nil\n}\n```\n\n### DON'T: Write directly to final location\n\n```go\n// WRONG - Can leave partial files on failure\nos.WriteFile(path, data, 0644)\n```\n\n## Error Handling\n\n### DO: Return errors immediately\n\n```go\nfunc (db *DB) validatePosition() error {\n dpos, err := db.Pos()\n if err != nil {\n return err\n }\n rpos := replica.Pos()\n if dpos.TXID < rpos.TXID {\n return fmt.Errorf(\"database position (%v) behind replica (%v)\", dpos, rpos)\n }\n return nil\n}\n```\n\n### DON'T: Log and continue on critical errors\n\n```go\n// WRONG\nif err := processFile(file); err != nil {\n log.Printf(\"error: %v\", err) // Just logging!\n // Continuing is dangerous\n}\n```\n\n### DO: Return errors from loops\n\n```go\nfunc (db *DB) processFiles() error {\n for _, file := range files {\n if err := processFile(file); err != nil {\n return fmt.Errorf(\"process file %s: %w\", file, err)\n }\n }\n return nil\n}\n```\n\n## Locking Patterns\n\n### DO: Use Lock() for writes\n\n```go\nr.mu.Lock()\ndefer r.mu.Unlock()\nr.pos = pos\n```\n\n### DON'T: Use RLock for write operations\n\n```go\n// WRONG - Race condition\nr.mu.RLock()\ndefer r.mu.RUnlock()\nr.pos = pos // Writing with RLock!\n```\n\n### Lock Ordering\n\nAlways acquire in order: Store.mu → DB.mu → DB.chkMu → Replica.mu\n\n## Compaction Patterns\n\n### DO: Read from local when available\n\n```go\nf, err := os.Open(db.LTXPath(info.Level, info.MinTXID, info.MaxTXID))\nif err == nil {\n return f, nil // Local copy is complete and consistent\n}\nreturn replica.Client.OpenLTXFile(...) // Fall back to remote\n```\n\n### DON'T: Read from remote during compaction\n\n```go\n// WRONG - Can get partial/corrupt data from eventually consistent storage\nf, err := client.OpenLTXFile(ctx, level, minTXID, maxTXID, 0, 0)\n```\n\n### DO: Preserve earliest timestamp\n\n```go\ninfo.CreatedAt = oldestSourceFile.CreatedAt\n```\n\n### DON'T: Use current time during compaction\n\n```go\n// WRONG - Loses timestamp granularity for point-in-time restores\ninfo := <x.FileInfo{CreatedAt: time.Now()}\n```\n\n## Lock Page Handling\n\nThe lock page at 1 GB (0x40000000) must always be skipped:\n\n```go\nlockPgno := ltx.LockPgno(pageSize)\nif pgno == lockPgno {\n continue\n}\n```\n\n| Page Size | Lock Page Number |\n|-----------|------------------|\n| 4 KB | 262145 |\n| 8 KB | 131073 |\n| 16 KB | 65537 |\n| 32 KB | 32769 |\n\n## Common Pitfalls\n\n1. **Mixing architectural concerns**: DB state logic in Replica layer\n2. **Recreating existing functionality**: Use `db.verify()` for snapshots\n3. **Ignoring lock page**: Must skip during replication and compaction\n4. **Generic error types**: Return `os.ErrNotExist` for missing files\n5. **Blocking iterators**: Use lazy pagination for `LTXFiles`\n6. **Ignoring context**: Check `ctx.Done()` in long operations\n" }, { "path": "skills/litestream/references/REPLICA_CLIENT_GUIDE.md", "content": "# ReplicaClient Implementation Reference\n\nCondensed reference for agents implementing or modifying Litestream storage backends.\n\n## Interface Contract\n\nFrom `replica_client.go`:\n\n```go\ntype ReplicaClient interface {\n Type() string\n Init(ctx context.Context) error\n LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error)\n OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error)\n WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error)\n DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error\n DeleteAll(ctx context.Context) error\n}\n```\n\n### Method Contracts\n\n**Type()**: Return identifier string (e.g., \"s3\", \"gcs\", \"file\").\n\n**Init()**: Initialize connection. Must be idempotent (no-op if already initialized).\n\n**LTXFiles()**: Return iterator sorted by MinTXID. `seek` starts from given TXID.\n`useMetadata=true` fetches accurate timestamps (required for point-in-time restore).\n`useMetadata=false` uses fast timestamps for normal operations.\n\n**OpenLTXFile()**: Open file for reading. Must support partial reads via `offset`/`size`.\nReturn `os.ErrNotExist` if file is missing.\n\n**WriteLTXFile()**: Write file to storage. Set `CreatedAt` from backend\nmetadata or upload time.\n\n**DeleteLTXFiles()**: Delete one or more files. Batch if possible.\n\n**DeleteAll()**: Delete all files for this database.\n\n## Implementation Checklist\n\n### Required\n\n- [ ] All interface methods implemented\n- [ ] `Init()` is idempotent\n- [ ] Partial reads supported (`offset`/`size` in `OpenLTXFile`)\n- [ ] `os.ErrNotExist` returned for missing files\n- [ ] Context cancellation handled in all methods\n- [ ] `CreatedAt` timestamps preserved in `WriteLTXFile`\n- [ ] Concurrent operations supported\n- [ ] Proper cleanup in `DeleteAll`\n\n### Optional\n\n- [ ] Connection pooling\n- [ ] Retry logic with exponential backoff\n- [ ] Request batching for deletes\n- [ ] Compression / encryption at rest\n- [ ] Bandwidth throttling\n\n## Eventual Consistency\n\nMany cloud backends (S3, R2, etc.) are eventually consistent:\n- Recently written files may not be immediately visible\n- Listed files may be only partially readable\n- Deletes may not take effect immediately\n\n### Pattern: Read Local First\n\nDuring compaction, always prefer local files:\n\n```go\nf, err := os.Open(db.LTXPath(info.Level, info.MinTXID, info.MaxTXID))\nif err == nil {\n return f, nil\n}\nreturn replica.Client.OpenLTXFile(...)\n```\n\n### Pattern: Retry with Backoff\n\n```go\nbackoff := 100 * time.Millisecond\nfor i := 0; i < 5; i++ {\n reader, err := c.openFile(ctx, path, offset, size)\n if err == nil {\n return reader, nil\n }\n if errors.Is(err, os.ErrNotExist) {\n return nil, err // Don't retry definitive errors\n }\n select {\n case <-ctx.Done():\n return nil, ctx.Err()\n case <-time.After(backoff):\n backoff *= 2\n }\n}\n```\n\n### Pattern: Lazy Iterator\n\n```go\nfunc (c *Client) LTXFiles(...) (ltx.FileIterator, error) {\n return &lazyIterator{\n client: c,\n level: level,\n seek: seek,\n pageSize: 1000, // Paginate, don't load all at once\n }, nil\n}\n```\n\n## Error Handling\n\n### Standard Error Types\n\n```go\n// File not found\nreturn nil, os.ErrNotExist\n\n// Permission denied\nreturn nil, os.ErrPermission\n\n// Context cancelled\nreturn nil, ctx.Err()\n\n// Wrapped errors\nreturn nil, fmt.Errorf(\"s3 download failed: %w\", err)\n```\n\n### Retryable Error Classification\n\n```go\nfunc isRetryable(err error) bool {\n var netErr net.Error\n if errors.As(err, &netErr) && netErr.Temporary() {\n return true\n }\n // HTTP 429, 500, 502, 503, 504\n if errors.Is(err, context.DeadlineExceeded) {\n return true\n }\n return false\n}\n```\n\n## Common Mistakes\n\n### 1. Not Handling Partial Reads\n\n```go\n// WRONG - Ignores offset/size\nreturn c.storage.Download(path)\n\n// CORRECT\nif offset == 0 && size == 0 {\n return c.storage.Download(path)\n}\nreturn c.storage.DownloadRange(path, offset, offset+size-1)\n```\n\n### 2. Not Preserving CreatedAt\n\n```go\n// WRONG\nreturn <x.FileInfo{CreatedAt: time.Now()}\n\n// CORRECT - Use backend metadata\nreturn <x.FileInfo{CreatedAt: modTime}\n```\n\n### 3. Wrong Error Types\n\n```go\n// WRONG\nreturn nil, fmt.Errorf(\"not found\")\n\n// CORRECT\nif resp.StatusCode == 404 {\n return nil, os.ErrNotExist\n}\n```\n\n### 4. Ignoring Context\n\n```go\n// WRONG - Could run forever\nfor i := 0; i < 1000000; i++ {\n doWork()\n}\n\n// CORRECT\nselect {\ncase <-ctx.Done():\n return nil, ctx.Err()\ndefault:\n}\n```\n\n### 5. Loading All Files at Once\n\n```go\n// WRONG - Could be millions of files\nallFiles, _ := c.loadAllFiles(level)\n\n// CORRECT - Lazy pagination\nreturn &lazyIterator{pageSize: 1000}, nil\n```\n\n## Path Construction\n\n```go\nfunc (c *Client) ltxDir(level int) string {\n if level == SnapshotLevel {\n return path.Join(c.Path, \"snapshots\")\n }\n return path.Join(c.Path, \"ltx\", fmt.Sprintf(\"%04d\", level))\n}\n```\n\n## Reference Implementations\n\n- **Simplest**: `file/replica_client.go` (direct file I/O, no network)\n- **Most complete**: `s3/replica_client.go` (multipart uploads, retries, signing)\n\nStart with the file backend for understanding, then study S3 for advanced\npatterns.\n\n## Testing Requirements\n\n- Unit tests with >80% coverage\n- Integration tests with build tag\n- Test partial reads, concurrent operations, missing files\n- Run with `-race` flag\n- Verify `os.ErrNotExist` for missing files\n- Test context cancellation\n" }, { "path": "skills/litestream/references/SQLITE_INTERNALS.md", "content": "# SQLite Internals Reference\n\nCondensed reference for agents working with SQLite internals in Litestream.\n\n## SQLite File Structure\n\n```\ndatabase.db Main database file (pages)\ndatabase.db-wal Write-ahead log\ndatabase.db-shm Shared memory file (coordination)\n```\n\n## Write-Ahead Log (WAL)\n\nWAL is SQLite's method for atomic commits:\n- Changes written to WAL first, database unchanged until checkpoint\n- Readers merge WAL + database for consistent view\n- Litestream monitors WAL and converts frames to LTX format\n\n### WAL File Structure\n\n```\n+------------------+\n| WAL Header | 32 bytes\n+------------------+\n| Frame 1 Header | 24 bytes\n| Frame 1 Data | PageSize bytes\n+------------------+\n| Frame 2 Header | 24 bytes\n| Frame 2 Data | PageSize bytes\n+------------------+\n| ... |\n```\n\n### WAL Header (32 bytes)\n\n```\nMagic (4B) | FileFormat (4B) | PageSize (4B) | Checkpoint (4B) |\nSalt1 (4B) | Salt2 (4B) | Checksum1 (4B) | Checksum2 (4B)\n```\n\nMagic: `0x377f0682` or `0x377f0683`\n\n### WAL Frame Header (24 bytes)\n\n```\nPageNumber (4B) | DbSize (4B) | Salt1 (4B) | Salt2 (4B) |\nChecksum1 (4B) | Checksum2 (4B)\n```\n\n## The 1 GB Lock Page\n\nSQLite reserves a page at exactly 0x40000000 bytes (1 GB) for locking. This is\nthe single most important SQLite detail for Litestream development.\n\n```go\nconst PENDING_BYTE = 0x40000000\n\nfunc LockPgno(pageSize int) uint32 {\n return uint32(PENDING_BYTE/pageSize) + 1\n}\n```\n\n| Page Size | Lock Page Number |\n|-----------|------------------|\n| 4 KB | 262145 |\n| 8 KB | 131073 |\n| 16 KB | 65537 |\n| 32 KB | 32769 |\n| 64 KB | 16385 |\n\nRules:\n- Cannot contain data; SQLite never writes user data here\n- Must be skipped during replication and compaction\n- Only affects databases > 1 GB\n- Page number changes with page size\n\nImplementation in Litestream:\n\n```go\nfor pgno := uint32(1); pgno <= maxPgno; pgno++ {\n if pgno == ltx.LockPgno(db.pageSize) {\n continue\n }\n processPage(pgno)\n}\n```\n\n## Page Structure\n\nSQLite divides databases into fixed-size pages (typically 4096 bytes):\n- Page numbers are 1-based\n- Types: B-tree interior, B-tree leaf, overflow, freelist, lock byte page\n\n## Transaction Types\n\n1. **Deferred** (default): Lock acquired on first use\n2. **Immediate**: RESERVED lock acquired immediately\n3. **Exclusive**: EXCLUSIVE lock acquired immediately\n\n### Lock Hierarchy\n\nUNLOCKED → SHARED → RESERVED → PENDING → EXCLUSIVE\n\n- SHARED: Multiple readers allowed\n- RESERVED: Signals intent to write\n- PENDING: Blocks new SHARED locks\n- EXCLUSIVE: Single writer, no readers\n\n## Long-Running Read Transaction\n\nLitestream maintains a read transaction for consistency:\n\n```go\nfunc (db *DB) initReadTx() error {\n tx, err := db.db.BeginTx(context.Background(), &sql.TxOptions{ReadOnly: true})\n if err != nil {\n return err\n }\n var dummy string\n err = tx.QueryRow(\"SELECT ''\").Scan(&dummy)\n if err != nil {\n tx.Rollback()\n return err\n }\n db.rtx = tx\n return nil\n}\n```\n\nPurpose:\n- Prevents checkpoint past our read point\n- Ensures consistent database view\n- Allows reading historical pages from WAL\n\n## Checkpoint Modes\n\n| Mode | Behavior |\n|----------|-----------------------------------------------|\n| PASSIVE | Non-blocking; fails if readers present |\n| FULL | Waits for readers; blocks new readers |\n| RESTART | Like FULL + resets WAL start (removed in #724) |\n| TRUNCATE | Like RESTART + truncates WAL to zero |\n\n### Litestream Checkpoint Strategy\n\n```\nWAL pages > TruncatePageN → TRUNCATE (emergency)\nWAL pages > MinCheckpointPageN → PASSIVE\nCheckpointInterval elapsed → PASSIVE\n```\n\nNote: RESTART mode permanently removed due to issue #724 (write-blocking).\n\n## Important SQLite Pragmas\n\n```sql\nPRAGMA journal_mode = WAL; -- Required for Litestream\nPRAGMA page_size; -- Get page size\nPRAGMA page_count; -- Total pages in database\nPRAGMA freelist_count; -- Free pages\nPRAGMA wal_checkpoint(PASSIVE); -- Non-blocking checkpoint\nPRAGMA wal_autocheckpoint = 10000; -- High threshold (prevent interference)\nPRAGMA busy_timeout = 5000; -- Wait 5s for locks\nPRAGMA synchronous = NORMAL; -- Safe with WAL\nPRAGMA cache_size = -64000; -- 64 MB cache\nPRAGMA integrity_check; -- Verify database integrity\n```\n\n## Critical SQLite Behaviors\n\n1. **Automatic checkpoint**: Default at 1000 WAL pages; set high threshold to\n prevent interference with Litestream's control\n2. **Busy timeout**: Default is 0 (immediate failure); set a reasonable timeout\n3. **Synchronous mode**: NORMAL is safe with WAL mode\n4. **Page cache**: In-memory cache; configure with negative KB or positive pages\n\n## WAL to LTX Conversion\n\nLitestream converts WAL frames to LTX:\n\n1. Read WAL header and validate magic number\n2. Iterate frames, reading page number and data\n3. Skip lock page (`pgno == LockPgno(pageSize)`)\n4. Calculate checksums\n5. Write as immutable LTX file with page index\n" }, { "path": "skills/litestream/references/TESTING_GUIDE.md", "content": "# Litestream Testing Reference\n\nCondensed reference for agents writing and debugging Litestream tests.\n\n## Testing Philosophy\n\n1. Test at multiple levels: unit, integration, end-to-end\n2. Focus on edge cases: >1 GB databases, eventual consistency\n3. Use real SQLite: avoid mocking SQLite behavior\n4. Always run with `-race` flag\n5. Use fixed seeds and timestamps for deterministic tests\n\n## Essential Commands\n\n```bash\n# Full test suite with race detection\ngo test -race -v ./...\n\n# Specific test areas\ngo test -race -v -run TestReplica_Sync ./...\ngo test -race -v -run TestDB_Sync ./...\ngo test -race -v -run TestStore_CompactDB ./...\n\n# Coverage\ngo test -coverprofile=coverage.out ./...\ngo tool cover -html=coverage.out\ngo tool cover -func=coverage.out | grep total\n\n# Integration tests (backend-specific)\ngo test -v ./replica_client_test.go -integration s3\ngo test -v ./replica_client_test.go -integration gcs\n```\n\n## Lock Page Testing\n\nThe lock page at 1 GB must be skipped. Test with databases spanning this\nboundary using multiple page sizes:\n\n```bash\n./bin/litestream-test populate -db test.db -page-size 4096 -target-size 2GB\n./bin/litestream-test validate -source-db test.db -replica-url file:///tmp/replica\n```\n\nTest all page size variants:\n\n| Page Size | Lock Page | Test Target |\n|-----------|-----------|-------------|\n| 4 KB | 262145 | >1.0 GB |\n| 8 KB | 131073 | >1.0 GB |\n| 16 KB | 65537 | >1.0 GB |\n| 32 KB | 32769 | >1.0 GB |\n\nVerify lock page is not present in any LTX file after replication.\n\n## Race Condition Testing\n\nRace-prone areas (always run with `-race`):\n\n1. **Position updates**: Concurrent `SetPos()` + `Pos()` on Replica\n2. **WAL monitoring**: Concurrent writes + monitor + checkpoint\n3. **Compaction**: Concurrent compaction at different levels\n\n### Pattern: Concurrent Test\n\n```go\nfunc TestConcurrent(t *testing.T) {\n ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n defer cancel()\n\n var wg sync.WaitGroup\n // Start writer, monitor, checkpoint goroutines\n // Use ctx for shutdown, wg for cleanup\n wg.Wait()\n}\n```\n\n## Common Test Failures\n\n### 1. Database Locked\n\n```go\n// WRONG - WAL mode not enabled\ndb, _ := sql.Open(\"sqlite\", \"test.db\")\n\n// CORRECT\ndb, _ := sql.Open(\"sqlite\", \"test.db?_journal=WAL\")\n```\n\n### 2. Timing Issues\n\n```go\n// WRONG - Data may not be replicated yet\nWriteData(db)\nresult := ReadReplica()\n\n// CORRECT - Explicit sync\nWriteData(db)\ndb.Sync(context.Background())\nresult := ReadReplica()\n```\n\n### 3. Goroutine Leaks\n\n```go\n// WRONG - Goroutine outlives test\ngo func() {\n time.Sleep(10 * time.Second)\n doWork()\n}()\n\n// CORRECT - Use context + WaitGroup\nctx, cancel := context.WithCancel(context.Background())\ndefer cancel()\nvar wg sync.WaitGroup\nwg.Add(1)\ngo func() {\n defer wg.Done()\n select {\n case <-ctx.Done():\n return\n case <-time.After(10 * time.Second):\n doWork()\n }\n}()\ncancel()\nwg.Wait()\n```\n\n### 4. File Handle Leaks\n\n```go\n// Always use defer for cleanup\nf, err := os.Open(\"test.db\")\nrequire.NoError(t, err)\ndefer f.Close()\n```\n\n## Test Helper Patterns\n\n```go\nfunc NewTestDB(t testing.TB) *litestream.DB {\n t.Helper()\n path := filepath.Join(t.TempDir(), \"test.db\")\n conn, err := sql.Open(\"sqlite\", path+\"?_journal=WAL\")\n require.NoError(t, err)\n _, err = conn.Exec(\"CREATE TABLE test (id INTEGER PRIMARY KEY, data BLOB)\")\n require.NoError(t, err)\n conn.Close()\n\n db := litestream.NewDB(path)\n db.MonitorInterval = 10 * time.Millisecond\n db.MinCheckpointPageN = 100\n err = db.Open()\n require.NoError(t, err)\n t.Cleanup(func() { db.Close(context.Background()) })\n return db\n}\n```\n\n## Coverage Requirements\n\n| Package | Target |\n|--------------------|--------|\n| Core (db, replica) | >80% |\n| Replica clients | >70% |\n| Utilities | >60% |\n\n## Table-Driven Tests\n\n```go\nfunc TestCheckpoint(t *testing.T) {\n tests := []struct {\n name string\n mode string\n wantErr bool\n }{\n {\"Passive\", \"PASSIVE\", false},\n {\"Full\", \"FULL\", false},\n {\"Truncate\", \"TRUNCATE\", false},\n {\"Invalid\", \"INVALID\", true},\n }\n for _, tt := range tests {\n t.Run(tt.name, func(t *testing.T) {\n // ...\n })\n }\n}\n```\n" }, { "path": "skills/litestream/scripts/validate-setup.sh", "content": "#!/bin/sh\n# validate-setup.sh - Verify Litestream development environment\n# POSIX-compatible script for validating build tools and project health.\n\nset -e\n\nPASS=0\nFAIL=0\nWARN=0\n\npass() {\n PASS=$((PASS + 1))\n printf \" [PASS] %s\\n\" \"$1\"\n}\n\nfail() {\n FAIL=$((FAIL + 1))\n printf \" [FAIL] %s\\n\" \"$1\"\n}\n\nwarn() {\n WARN=$((WARN + 1))\n printf \" [WARN] %s\\n\" \"$1\"\n}\n\nprintf \"Litestream Development Environment Validation\\n\"\nprintf \"==============================================\\n\\n\"\n\n# Check Go installation\nprintf \"Checking Go...\\n\"\nif command -v go >/dev/null 2>&1; then\n GO_VERSION=$(go version | sed 's/.*go\\([0-9]*\\.[0-9]*\\).*/\\1/')\n GO_MAJOR=$(echo \"$GO_VERSION\" | cut -d. -f1)\n GO_MINOR=$(echo \"$GO_VERSION\" | cut -d. -f2)\n if [ \"$GO_MAJOR\" -ge 1 ] && [ \"$GO_MINOR\" -ge 24 ]; then\n pass \"Go $(go version | sed 's/.*go/go/' | cut -d' ' -f1) installed (>= 1.24 required)\"\n else\n fail \"Go $GO_VERSION found but >= 1.24 required\"\n fi\nelse\n fail \"Go not found in PATH\"\nfi\n\n# Check project builds\nprintf \"\\nChecking build...\\n\"\nif go build -o /dev/null ./cmd/litestream 2>/dev/null; then\n pass \"Project builds successfully\"\nelse\n fail \"Project build failed (go build -o /dev/null ./cmd/litestream)\"\nfi\n\n# Check go vet\nprintf \"\\nChecking go vet...\\n\"\nif go vet ./... 2>/dev/null; then\n pass \"go vet passes\"\nelse\n fail \"go vet reports issues (run: go vet ./...)\"\nfi\n\n# Quick smoke test (run a fast subset of tests)\nprintf \"\\nChecking tests (smoke test)...\\n\"\nif go test -race -short -count=1 ./... >/dev/null 2>&1; then\n pass \"Tests pass with race detector (-short mode)\"\nelse\n # Try without race detector in case of environment issues\n if go test -short -count=1 ./... >/dev/null 2>&1; then\n warn \"Tests pass but race detector may not be supported on this platform\"\n else\n fail \"Tests failing (run: go test -race -short ./...)\"\n fi\nfi\n\n# Check pre-commit (optional)\nprintf \"\\nChecking optional tools...\\n\"\nif command -v pre-commit >/dev/null 2>&1; then\n pass \"pre-commit installed\"\nelse\n warn \"pre-commit not found (optional; install: pip install pre-commit)\"\nfi\n\n# Check git\nif command -v git >/dev/null 2>&1; then\n pass \"git installed\"\nelse\n fail \"git not found in PATH\"\nfi\n\n# Summary\nprintf \"\\n==============================================\\n\"\nprintf \"Results: %d passed, %d failed, %d warnings\\n\" \"$PASS\" \"$FAIL\" \"$WARN\"\n\nif [ \"$FAIL\" -gt 0 ]; then\n printf \"\\nEnvironment has issues that must be resolved.\\n\"\n exit 1\nelse\n printf \"\\nEnvironment is ready for Litestream development.\\n\"\n exit 0\nfi\n" }, { "path": "src/litestream-vfs.c", "content": "#include \"litestream-vfs.h\"\n#include \"sqlite3.h\"\n#include \"sqlite3ext.h\"\n#include \"sqlite3vfs.h\"\n#include \n#include \n\n/* sqlite3vfs already called SQLITE_EXTENSION_INIT1 */\nextern const sqlite3_api_routines *sqlite3_api;\n\n/* Go function declarations */\nextern char* LitestreamVFSRegister(void);\nextern char* GoLitestreamRegisterConnection(void* db, sqlite3_uint64 file_id);\nextern char* GoLitestreamUnregisterConnection(void* db);\nextern char* GoLitestreamSetTime(void* db, char* timestamp);\nextern char* GoLitestreamResetTime(void* db);\nextern char* GoLitestreamTime(void* db, char** out);\nextern char* GoLitestreamTxid(void* db, char** out);\nextern char* GoLitestreamLag(void* db, sqlite3_int64* out);\n\n/* Internal function declarations */\nstatic int litestream_register_connection(sqlite3* db);\nstatic void litestream_set_time_impl(sqlite3_context* ctx, int argc, sqlite3_value** argv);\nstatic void litestream_time_impl(sqlite3_context* ctx, int argc, sqlite3_value** argv);\nstatic void litestream_txid_impl(sqlite3_context* ctx, int argc, sqlite3_value** argv);\nstatic void litestream_lag_impl(sqlite3_context* ctx, int argc, sqlite3_value** argv);\nstatic void litestream_function_destroy(void* db);\nstatic void litestream_auto_extension(sqlite3* db, const char** pzErrMsg, const struct sqlite3_api_routines* pApi);\n\n/* This routine is called when the extension is loaded. */\nint sqlite3_litestreamvfs_init(sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi) {\n int rc = SQLITE_OK;\n SQLITE_EXTENSION_INIT2(pApi);\n\n /* Call into Go to register the VFS and check for errors. */\n char* err = LitestreamVFSRegister();\n if (err != NULL) {\n *pzErrMsg = sqlite3_mprintf(\"%s\", err);\n free(err);\n return SQLITE_ERROR;\n }\n\n /* Register SQL functions for new connections. */\n rc = sqlite3_auto_extension((void (*)(void))litestream_auto_extension);\n\n if( rc==SQLITE_OK ) rc = SQLITE_OK_LOAD_PERMANENTLY;\n return rc;\n}\n\nstatic void litestream_auto_extension(sqlite3* db, const char** pzErrMsg, const struct sqlite3_api_routines* pApi) {\n (void)pzErrMsg;\n (void)pApi;\n\n if (litestream_register_connection(db) != SQLITE_OK) {\n return;\n }\n\n /* litestream_set_time(timestamp) - for time travel */\n sqlite3_create_function_v2(db, \"litestream_set_time\", 1, SQLITE_UTF8 | SQLITE_DIRECTONLY, db, litestream_set_time_impl, 0, 0, litestream_function_destroy);\n\n /* Read-only functions: litestream_time(), litestream_txid(), litestream_lag() */\n sqlite3_create_function_v2(db, \"litestream_time\", 0, SQLITE_UTF8, db, litestream_time_impl, 0, 0, 0);\n sqlite3_create_function_v2(db, \"litestream_txid\", 0, SQLITE_UTF8, db, litestream_txid_impl, 0, 0, 0);\n sqlite3_create_function_v2(db, \"litestream_lag\", 0, SQLITE_UTF8, db, litestream_lag_impl, 0, 0, 0);\n}\n\nstatic int litestream_register_connection(sqlite3* db) {\n sqlite3_file* file = 0;\n int rc = sqlite3_file_control(db, \"main\", SQLITE_FCNTL_FILE_POINTER, &file);\n if (rc != SQLITE_OK || file == 0) {\n return rc;\n }\n\n if (file->pMethods != &s3vfs_io_methods) {\n /* Not using the litestream VFS. */\n return SQLITE_DONE;\n }\n\n sqlite3_uint64 file_id = ((s3vfsFile*)file)->id;\n char* err = GoLitestreamRegisterConnection(db, file_id);\n if (err != 0) {\n free(err);\n return SQLITE_ERROR;\n }\n\n return SQLITE_OK;\n}\n\nstatic void litestream_set_time_impl(sqlite3_context* ctx, int argc, sqlite3_value** argv) {\n if (argc != 1) {\n sqlite3_result_error(ctx, \"expected timestamp argument\", -1);\n return;\n }\n\n const unsigned char* ts = sqlite3_value_text(argv[0]);\n if (!ts) {\n sqlite3_result_error(ctx, \"timestamp required\", -1);\n return;\n }\n\n /* Handle special 'LATEST' value */\n if (sqlite3_stricmp((const char*)ts, \"LATEST\") == 0) {\n sqlite3* db = sqlite3_context_db_handle(ctx);\n char* err = GoLitestreamResetTime(db);\n if (err != 0) {\n sqlite3_result_error(ctx, err, -1);\n free(err);\n return;\n }\n sqlite3_result_null(ctx);\n return;\n }\n\n sqlite3* db = sqlite3_context_db_handle(ctx);\n char* err = GoLitestreamSetTime(db, (char*)ts);\n if (err != 0) {\n sqlite3_result_error(ctx, err, -1);\n free(err);\n return;\n }\n\n sqlite3_result_null(ctx);\n}\n\nstatic void litestream_time_impl(sqlite3_context* ctx, int argc, sqlite3_value** argv) {\n (void)argc;\n (void)argv;\n\n sqlite3* db = sqlite3_context_db_handle(ctx);\n char* out = 0;\n char* err = GoLitestreamTime(db, &out);\n if (err != 0) {\n sqlite3_result_error(ctx, err, -1);\n free(err);\n return;\n }\n\n sqlite3_result_text(ctx, out, -1, SQLITE_TRANSIENT);\n free(out);\n}\n\nstatic void litestream_txid_impl(sqlite3_context* ctx, int argc, sqlite3_value** argv) {\n (void)argc;\n (void)argv;\n\n sqlite3* db = sqlite3_context_db_handle(ctx);\n char* out = 0;\n char* err = GoLitestreamTxid(db, &out);\n if (err != 0) {\n sqlite3_result_error(ctx, err, -1);\n free(err);\n return;\n }\n\n sqlite3_result_text(ctx, out, -1, SQLITE_TRANSIENT);\n free(out);\n}\n\nstatic void litestream_lag_impl(sqlite3_context* ctx, int argc, sqlite3_value** argv) {\n (void)argc;\n (void)argv;\n\n sqlite3* db = sqlite3_context_db_handle(ctx);\n sqlite3_int64 out = 0;\n char* err = GoLitestreamLag(db, &out);\n if (err != 0) {\n sqlite3_result_error(ctx, err, -1);\n free(err);\n return;\n }\n\n sqlite3_result_int64(ctx, out);\n}\n\nstatic void litestream_function_destroy(void* db) {\n if (db == 0) {\n return;\n }\n char* err = GoLitestreamUnregisterConnection(db);\n if (err != 0) {\n free(err);\n }\n}\n" }, { "path": "src/sqlite3.h", "content": "/*\n** 2001-09-15\n**\n** The author disclaims copyright to this source code. In place of\n** a legal notice, here is a blessing:\n**\n** May you do good and not evil.\n** May you find forgiveness for yourself and forgive others.\n** May you share freely, never taking more than you give.\n**\n*************************************************************************\n** This header file defines the interface that the SQLite library\n** presents to client programs. If a C-function, structure, datatype,\n** or constant definition does not appear in this file, then it is\n** not a published API of SQLite, is subject to change without\n** notice, and should not be referenced by programs that use SQLite.\n**\n** Some of the definitions that are in this file are marked as\n** \"experimental\". Experimental interfaces are normally new\n** features recently added to SQLite. We do not anticipate changes\n** to experimental interfaces but reserve the right to make minor changes\n** if experience from use \"in the wild\" suggest such changes are prudent.\n**\n** The official C-language API documentation for SQLite is derived\n** from comments in this file. This file is the authoritative source\n** on how SQLite interfaces are supposed to operate.\n**\n** The name of this file under configuration management is \"sqlite.h.in\".\n** The makefile makes some minor changes to this file (such as inserting\n** the version number) and changes its name to \"sqlite3.h\" as\n** part of the build process.\n*/\n#ifndef SQLITE3_H\n#define SQLITE3_H\n#include /* Needed for the definition of va_list */\n\n/*\n** Make sure we can call this stuff from C++.\n*/\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n\n/*\n** Facilitate override of interface linkage and calling conventions.\n** Be aware that these macros may not be used within this particular\n** translation of the amalgamation and its associated header file.\n**\n** The SQLITE_EXTERN and SQLITE_API macros are used to instruct the\n** compiler that the target identifier should have external linkage.\n**\n** The SQLITE_CDECL macro is used to set the calling convention for\n** public functions that accept a variable number of arguments.\n**\n** The SQLITE_APICALL macro is used to set the calling convention for\n** public functions that accept a fixed number of arguments.\n**\n** The SQLITE_STDCALL macro is no longer used and is now deprecated.\n**\n** The SQLITE_CALLBACK macro is used to set the calling convention for\n** function pointers.\n**\n** The SQLITE_SYSAPI macro is used to set the calling convention for\n** functions provided by the operating system.\n**\n** Currently, the SQLITE_CDECL, SQLITE_APICALL, SQLITE_CALLBACK, and\n** SQLITE_SYSAPI macros are used only when building for environments\n** that require non-default calling conventions.\n*/\n#ifndef SQLITE_EXTERN\n# define SQLITE_EXTERN extern\n#endif\n#ifndef SQLITE_API\n# define SQLITE_API\n#endif\n#ifndef SQLITE_CDECL\n# define SQLITE_CDECL\n#endif\n#ifndef SQLITE_APICALL\n# define SQLITE_APICALL\n#endif\n#ifndef SQLITE_STDCALL\n# define SQLITE_STDCALL SQLITE_APICALL\n#endif\n#ifndef SQLITE_CALLBACK\n# define SQLITE_CALLBACK\n#endif\n#ifndef SQLITE_SYSAPI\n# define SQLITE_SYSAPI\n#endif\n\n/*\n** These no-op macros are used in front of interfaces to mark those\n** interfaces as either deprecated or experimental. New applications\n** should not use deprecated interfaces - they are supported for backwards\n** compatibility only. Application writers should be aware that\n** experimental interfaces are subject to change in point releases.\n**\n** These macros used to resolve to various kinds of compiler magic that\n** would generate warning messages when they were used. But that\n** compiler magic ended up generating such a flurry of bug reports\n** that we have taken it all out and gone back to using simple\n** noop macros.\n*/\n#define SQLITE_DEPRECATED\n#define SQLITE_EXPERIMENTAL\n\n/*\n** Ensure these symbols were not defined by some previous header file.\n*/\n#ifdef SQLITE_VERSION\n# undef SQLITE_VERSION\n#endif\n#ifdef SQLITE_VERSION_NUMBER\n# undef SQLITE_VERSION_NUMBER\n#endif\n\n/*\n** CAPI3REF: Compile-Time Library Version Numbers\n**\n** ^(The [SQLITE_VERSION] C preprocessor macro in the sqlite3.h header\n** evaluates to a string literal that is the SQLite version in the\n** format \"X.Y.Z\" where X is the major version number (always 3 for\n** SQLite3) and Y is the minor version number and Z is the release number.)^\n** ^(The [SQLITE_VERSION_NUMBER] C preprocessor macro resolves to an integer\n** with the value (X*1000000 + Y*1000 + Z) where X, Y, and Z are the same\n** numbers used in [SQLITE_VERSION].)^\n** The SQLITE_VERSION_NUMBER for any given release of SQLite will also\n** be larger than the release from which it is derived. Either Y will\n** be held constant and Z will be incremented or else Y will be incremented\n** and Z will be reset to zero.\n**\n** Since [version 3.6.18] ([dateof:3.6.18]),\n** SQLite source code has been stored in the\n** Fossil configuration management\n** system. ^The SQLITE_SOURCE_ID macro evaluates to\n** a string which identifies a particular check-in of SQLite\n** within its configuration management system. ^The SQLITE_SOURCE_ID\n** string contains the date and time of the check-in (UTC) and a SHA1\n** or SHA3-256 hash of the entire source tree. If the source code has\n** been edited in any way since it was last checked in, then the last\n** four hexadecimal digits of the hash may be modified.\n**\n** See also: [sqlite3_libversion()],\n** [sqlite3_libversion_number()], [sqlite3_sourceid()],\n** [sqlite_version()] and [sqlite_source_id()].\n*/\n#define SQLITE_VERSION \"3.50.4\"\n#define SQLITE_VERSION_NUMBER 3050004\n#define SQLITE_SOURCE_ID \"2025-07-30 19:33:53 4d8adfb30e03f9cf27f800a2c1ba3c48fb4ca1b08b0f5ed59a4d5ecbf45e20a3\"\n\n/*\n** CAPI3REF: Run-Time Library Version Numbers\n** KEYWORDS: sqlite3_version sqlite3_sourceid\n**\n** These interfaces provide the same information as the [SQLITE_VERSION],\n** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros\n** but are associated with the library instead of the header file. ^(Cautious\n** programmers might include assert() statements in their application to\n** verify that values returned by these interfaces match the macros in\n** the header, and thus ensure that the application is\n** compiled with matching library and header files.\n**\n** 
\n** assert( sqlite3_libversion_number()==SQLITE_VERSION_NUMBER );\n** assert( strncmp(sqlite3_sourceid(),SQLITE_SOURCE_ID,80)==0 );\n** assert( strcmp(sqlite3_libversion(),SQLITE_VERSION)==0 );\n**
)^\n**\n** ^The sqlite3_version[] string constant contains the text of [SQLITE_VERSION]\n** macro. ^The sqlite3_libversion() function returns a pointer to the\n** to the sqlite3_version[] string constant. The sqlite3_libversion()\n** function is provided for use in DLLs since DLL users usually do not have\n** direct access to string constants within the DLL. ^The\n** sqlite3_libversion_number() function returns an integer equal to\n** [SQLITE_VERSION_NUMBER]. ^(The sqlite3_sourceid() function returns\n** a pointer to a string constant whose value is the same as the\n** [SQLITE_SOURCE_ID] C preprocessor macro. Except if SQLite is built\n** using an edited copy of [the amalgamation], then the last four characters\n** of the hash might be different from [SQLITE_SOURCE_ID].)^\n**\n** See also: [sqlite_version()] and [sqlite_source_id()].\n*/\nSQLITE_API SQLITE_EXTERN const char sqlite3_version[];\nSQLITE_API const char *sqlite3_libversion(void);\nSQLITE_API const char *sqlite3_sourceid(void);\nSQLITE_API int sqlite3_libversion_number(void);\n\n/*\n** CAPI3REF: Run-Time Library Compilation Options Diagnostics\n**\n** ^The sqlite3_compileoption_used() function returns 0 or 1\n** indicating whether the specified option was defined at\n** compile time. ^The SQLITE_ prefix may be omitted from the\n** option name passed to sqlite3_compileoption_used().\n**\n** ^The sqlite3_compileoption_get() function allows iterating\n** over the list of options that were defined at compile time by\n** returning the N-th compile time option string. ^If N is out of range,\n** sqlite3_compileoption_get() returns a NULL pointer. ^The SQLITE_\n** prefix is omitted from any strings returned by\n** sqlite3_compileoption_get().\n**\n** ^Support for the diagnostic functions sqlite3_compileoption_used()\n** and sqlite3_compileoption_get() may be omitted by specifying the\n** [SQLITE_OMIT_COMPILEOPTION_DIAGS] option at compile time.\n**\n** See also: SQL functions [sqlite_compileoption_used()] and\n** [sqlite_compileoption_get()] and the [compile_options pragma].\n*/\n#ifndef SQLITE_OMIT_COMPILEOPTION_DIAGS\nSQLITE_API int sqlite3_compileoption_used(const char *zOptName);\nSQLITE_API const char *sqlite3_compileoption_get(int N);\n#else\n# define sqlite3_compileoption_used(X) 0\n# define sqlite3_compileoption_get(X) ((void*)0)\n#endif\n\n/*\n** CAPI3REF: Test To See If The Library Is Threadsafe\n**\n** ^The sqlite3_threadsafe() function returns zero if and only if\n** SQLite was compiled with mutexing code omitted due to the\n** [SQLITE_THREADSAFE] compile-time option being set to 0.\n**\n** SQLite can be compiled with or without mutexes. When\n** the [SQLITE_THREADSAFE] C preprocessor macro is 1 or 2, mutexes\n** are enabled and SQLite is threadsafe. When the\n** [SQLITE_THREADSAFE] macro is 0,\n** the mutexes are omitted. Without the mutexes, it is not safe\n** to use SQLite concurrently from more than one thread.\n**\n** Enabling mutexes incurs a measurable performance penalty.\n** So if speed is of utmost importance, it makes sense to disable\n** the mutexes. But for maximum safety, mutexes should be enabled.\n** ^The default behavior is for mutexes to be enabled.\n**\n** This interface can be used by an application to make sure that the\n** version of SQLite that it is linking against was compiled with\n** the desired setting of the [SQLITE_THREADSAFE] macro.\n**\n** This interface only reports on the compile-time mutex setting\n** of the [SQLITE_THREADSAFE] flag. If SQLite is compiled with\n** SQLITE_THREADSAFE=1 or =2 then mutexes are enabled by default but\n** can be fully or partially disabled using a call to [sqlite3_config()]\n** with the verbs [SQLITE_CONFIG_SINGLETHREAD], [SQLITE_CONFIG_MULTITHREAD],\n** or [SQLITE_CONFIG_SERIALIZED]. ^(The return value of the\n** sqlite3_threadsafe() function shows only the compile-time setting of\n** thread safety, not any run-time changes to that setting made by\n** sqlite3_config(). In other words, the return value from sqlite3_threadsafe()\n** is unchanged by calls to sqlite3_config().)^\n**\n** See the [threading mode] documentation for additional information.\n*/\nSQLITE_API int sqlite3_threadsafe(void);\n\n/*\n** CAPI3REF: Database Connection Handle\n** KEYWORDS: {database connection} {database connections}\n**\n** Each open SQLite database is represented by a pointer to an instance of\n** the opaque structure named \"sqlite3\". It is useful to think of an sqlite3\n** pointer as an object. The [sqlite3_open()], [sqlite3_open16()], and\n** [sqlite3_open_v2()] interfaces are its constructors, and [sqlite3_close()]\n** and [sqlite3_close_v2()] are its destructors. There are many other\n** interfaces (such as\n** [sqlite3_prepare_v2()], [sqlite3_create_function()], and\n** [sqlite3_busy_timeout()] to name but three) that are methods on an\n** sqlite3 object.\n*/\ntypedef struct sqlite3 sqlite3;\n\n/*\n** CAPI3REF: 64-Bit Integer Types\n** KEYWORDS: sqlite_int64 sqlite_uint64\n**\n** Because there is no cross-platform way to specify 64-bit integer types\n** SQLite includes typedefs for 64-bit signed and unsigned integers.\n**\n** The sqlite3_int64 and sqlite3_uint64 are the preferred type definitions.\n** The sqlite_int64 and sqlite_uint64 types are supported for backwards\n** compatibility only.\n**\n** ^The sqlite3_int64 and sqlite_int64 types can store integer values\n** between -9223372036854775808 and +9223372036854775807 inclusive. ^The\n** sqlite3_uint64 and sqlite_uint64 types can store integer values\n** between 0 and +18446744073709551615 inclusive.\n*/\n#ifdef SQLITE_INT64_TYPE\n typedef SQLITE_INT64_TYPE sqlite_int64;\n# ifdef SQLITE_UINT64_TYPE\n typedef SQLITE_UINT64_TYPE sqlite_uint64;\n# else\n typedef unsigned SQLITE_INT64_TYPE sqlite_uint64;\n# endif\n#elif defined(_MSC_VER) || defined(__BORLANDC__)\n typedef __int64 sqlite_int64;\n typedef unsigned __int64 sqlite_uint64;\n#else\n typedef long long int sqlite_int64;\n typedef unsigned long long int sqlite_uint64;\n#endif\ntypedef sqlite_int64 sqlite3_int64;\ntypedef sqlite_uint64 sqlite3_uint64;\n\n/*\n** If compiling for a processor that lacks floating point support,\n** substitute integer for floating-point.\n*/\n#ifdef SQLITE_OMIT_FLOATING_POINT\n# define double sqlite3_int64\n#endif\n\n/*\n** CAPI3REF: Closing A Database Connection\n** DESTRUCTOR: sqlite3\n**\n** ^The sqlite3_close() and sqlite3_close_v2() routines are destructors\n** for the [sqlite3] object.\n** ^Calls to sqlite3_close() and sqlite3_close_v2() return [SQLITE_OK] if\n** the [sqlite3] object is successfully destroyed and all associated\n** resources are deallocated.\n**\n** Ideally, applications should [sqlite3_finalize | finalize] all\n** [prepared statements], [sqlite3_blob_close | close] all [BLOB handles], and\n** [sqlite3_backup_finish | finish] all [sqlite3_backup] objects associated\n** with the [sqlite3] object prior to attempting to close the object.\n** ^If the database connection is associated with unfinalized prepared\n** statements, BLOB handlers, and/or unfinished sqlite3_backup objects then\n** sqlite3_close() will leave the database connection open and return\n** [SQLITE_BUSY]. ^If sqlite3_close_v2() is called with unfinalized prepared\n** statements, unclosed BLOB handlers, and/or unfinished sqlite3_backups,\n** it returns [SQLITE_OK] regardless, but instead of deallocating the database\n** connection immediately, it marks the database connection as an unusable\n** \"zombie\" and makes arrangements to automatically deallocate the database\n** connection after all prepared statements are finalized, all BLOB handles\n** are closed, and all backups have finished. The sqlite3_close_v2() interface\n** is intended for use with host languages that are garbage collected, and\n** where the order in which destructors are called is arbitrary.\n**\n** ^If an [sqlite3] object is destroyed while a transaction is open,\n** the transaction is automatically rolled back.\n**\n** The C parameter to [sqlite3_close(C)] and [sqlite3_close_v2(C)]\n** must be either a NULL\n** pointer or an [sqlite3] object pointer obtained\n** from [sqlite3_open()], [sqlite3_open16()], or\n** [sqlite3_open_v2()], and not previously closed.\n** ^Calling sqlite3_close() or sqlite3_close_v2() with a NULL pointer\n** argument is a harmless no-op.\n*/\nSQLITE_API int sqlite3_close(sqlite3*);\nSQLITE_API int sqlite3_close_v2(sqlite3*);\n\n/*\n** The type for a callback function.\n** This is legacy and deprecated. It is included for historical\n** compatibility and is not documented.\n*/\ntypedef int (*sqlite3_callback)(void*,int,char**, char**);\n\n/*\n** CAPI3REF: One-Step Query Execution Interface\n** METHOD: sqlite3\n**\n** The sqlite3_exec() interface is a convenience wrapper around\n** [sqlite3_prepare_v2()], [sqlite3_step()], and [sqlite3_finalize()],\n** that allows an application to run multiple statements of SQL\n** without having to use a lot of C code.\n**\n** ^The sqlite3_exec() interface runs zero or more UTF-8 encoded,\n** semicolon-separate SQL statements passed into its 2nd argument,\n** in the context of the [database connection] passed in as its 1st\n** argument. ^If the callback function of the 3rd argument to\n** sqlite3_exec() is not NULL, then it is invoked for each result row\n** coming out of the evaluated SQL statements. ^The 4th argument to\n** sqlite3_exec() is relayed through to the 1st argument of each\n** callback invocation. ^If the callback pointer to sqlite3_exec()\n** is NULL, then no callback is ever invoked and result rows are\n** ignored.\n**\n** ^If an error occurs while evaluating the SQL statements passed into\n** sqlite3_exec(), then execution of the current statement stops and\n** subsequent statements are skipped. ^If the 5th parameter to sqlite3_exec()\n** is not NULL then any error message is written into memory obtained\n** from [sqlite3_malloc()] and passed back through the 5th parameter.\n** To avoid memory leaks, the application should invoke [sqlite3_free()]\n** on error message strings returned through the 5th parameter of\n** sqlite3_exec() after the error message string is no longer needed.\n** ^If the 5th parameter to sqlite3_exec() is not NULL and no errors\n** occur, then sqlite3_exec() sets the pointer in its 5th parameter to\n** NULL before returning.\n**\n** ^If an sqlite3_exec() callback returns non-zero, the sqlite3_exec()\n** routine returns SQLITE_ABORT without invoking the callback again and\n** without running any subsequent SQL statements.\n**\n** ^The 2nd argument to the sqlite3_exec() callback function is the\n** number of columns in the result. ^The 3rd argument to the sqlite3_exec()\n** callback is an array of pointers to strings obtained as if from\n** [sqlite3_column_text()], one for each column. ^If an element of a\n** result row is NULL then the corresponding string pointer for the\n** sqlite3_exec() callback is a NULL pointer. ^The 4th argument to the\n** sqlite3_exec() callback is an array of pointers to strings where each\n** entry represents the name of corresponding result column as obtained\n** from [sqlite3_column_name()].\n**\n** ^If the 2nd parameter to sqlite3_exec() is a NULL pointer, a pointer\n** to an empty string, or a pointer that contains only whitespace and/or\n** SQL comments, then no SQL statements are evaluated and the database\n** is not changed.\n**\n** Restrictions:\n**\n**
    \n**
  • The application must ensure that the 1st parameter to sqlite3_exec()\n** is a valid and open [database connection].\n**
  • The application must not close the [database connection] specified by\n** the 1st parameter to sqlite3_exec() while sqlite3_exec() is running.\n**
  • The application must not modify the SQL statement text passed into\n** the 2nd parameter of sqlite3_exec() while sqlite3_exec() is running.\n**
  • The application must not dereference the arrays or string pointers\n** passed as the 3rd and 4th callback parameters after it returns.\n**
\n*/\nSQLITE_API int sqlite3_exec(\n sqlite3*, /* An open database */\n const char *sql, /* SQL to be evaluated */\n int (*callback)(void*,int,char**,char**), /* Callback function */\n void *, /* 1st argument to callback */\n char **errmsg /* Error msg written here */\n);\n\n/*\n** CAPI3REF: Result Codes\n** KEYWORDS: {result code definitions}\n**\n** Many SQLite functions return an integer result code from the set shown\n** here in order to indicate success or failure.\n**\n** New error codes may be added in future versions of SQLite.\n**\n** See also: [extended result code definitions]\n*/\n#define SQLITE_OK 0 /* Successful result */\n/* beginning-of-error-codes */\n#define SQLITE_ERROR 1 /* Generic error */\n#define SQLITE_INTERNAL 2 /* Internal logic error in SQLite */\n#define SQLITE_PERM 3 /* Access permission denied */\n#define SQLITE_ABORT 4 /* Callback routine requested an abort */\n#define SQLITE_BUSY 5 /* The database file is locked */\n#define SQLITE_LOCKED 6 /* A table in the database is locked */\n#define SQLITE_NOMEM 7 /* A malloc() failed */\n#define SQLITE_READONLY 8 /* Attempt to write a readonly database */\n#define SQLITE_INTERRUPT 9 /* Operation terminated by sqlite3_interrupt()*/\n#define SQLITE_IOERR 10 /* Some kind of disk I/O error occurred */\n#define SQLITE_CORRUPT 11 /* The database disk image is malformed */\n#define SQLITE_NOTFOUND 12 /* Unknown opcode in sqlite3_file_control() */\n#define SQLITE_FULL 13 /* Insertion failed because database is full */\n#define SQLITE_CANTOPEN 14 /* Unable to open the database file */\n#define SQLITE_PROTOCOL 15 /* Database lock protocol error */\n#define SQLITE_EMPTY 16 /* Internal use only */\n#define SQLITE_SCHEMA 17 /* The database schema changed */\n#define SQLITE_TOOBIG 18 /* String or BLOB exceeds size limit */\n#define SQLITE_CONSTRAINT 19 /* Abort due to constraint violation */\n#define SQLITE_MISMATCH 20 /* Data type mismatch */\n#define SQLITE_MISUSE 21 /* Library used incorrectly */\n#define SQLITE_NOLFS 22 /* Uses OS features not supported on host */\n#define SQLITE_AUTH 23 /* Authorization denied */\n#define SQLITE_FORMAT 24 /* Not used */\n#define SQLITE_RANGE 25 /* 2nd parameter to sqlite3_bind out of range */\n#define SQLITE_NOTADB 26 /* File opened that is not a database file */\n#define SQLITE_NOTICE 27 /* Notifications from sqlite3_log() */\n#define SQLITE_WARNING 28 /* Warnings from sqlite3_log() */\n#define SQLITE_ROW 100 /* sqlite3_step() has another row ready */\n#define SQLITE_DONE 101 /* sqlite3_step() has finished executing */\n/* end-of-error-codes */\n\n/*\n** CAPI3REF: Extended Result Codes\n** KEYWORDS: {extended result code definitions}\n**\n** In its default configuration, SQLite API routines return one of 30 integer\n** [result codes]. However, experience has shown that many of\n** these result codes are too coarse-grained. They do not provide as\n** much information about problems as programmers might like. In an effort to\n** address this, newer versions of SQLite (version 3.3.8 [dateof:3.3.8]\n** and later) include\n** support for additional result codes that provide more detailed information\n** about errors. These [extended result codes] are enabled or disabled\n** on a per database connection basis using the\n** [sqlite3_extended_result_codes()] API. Or, the extended code for\n** the most recent error can be obtained using\n** [sqlite3_extended_errcode()].\n*/\n#define SQLITE_ERROR_MISSING_COLLSEQ (SQLITE_ERROR | (1<<8))\n#define SQLITE_ERROR_RETRY (SQLITE_ERROR | (2<<8))\n#define SQLITE_ERROR_SNAPSHOT (SQLITE_ERROR | (3<<8))\n#define SQLITE_IOERR_READ (SQLITE_IOERR | (1<<8))\n#define SQLITE_IOERR_SHORT_READ (SQLITE_IOERR | (2<<8))\n#define SQLITE_IOERR_WRITE (SQLITE_IOERR | (3<<8))\n#define SQLITE_IOERR_FSYNC (SQLITE_IOERR | (4<<8))\n#define SQLITE_IOERR_DIR_FSYNC (SQLITE_IOERR | (5<<8))\n#define SQLITE_IOERR_TRUNCATE (SQLITE_IOERR | (6<<8))\n#define SQLITE_IOERR_FSTAT (SQLITE_IOERR | (7<<8))\n#define SQLITE_IOERR_UNLOCK (SQLITE_IOERR | (8<<8))\n#define SQLITE_IOERR_RDLOCK (SQLITE_IOERR | (9<<8))\n#define SQLITE_IOERR_DELETE (SQLITE_IOERR | (10<<8))\n#define SQLITE_IOERR_BLOCKED (SQLITE_IOERR | (11<<8))\n#define SQLITE_IOERR_NOMEM (SQLITE_IOERR | (12<<8))\n#define SQLITE_IOERR_ACCESS (SQLITE_IOERR | (13<<8))\n#define SQLITE_IOERR_CHECKRESERVEDLOCK (SQLITE_IOERR | (14<<8))\n#define SQLITE_IOERR_LOCK (SQLITE_IOERR | (15<<8))\n#define SQLITE_IOERR_CLOSE (SQLITE_IOERR | (16<<8))\n#define SQLITE_IOERR_DIR_CLOSE (SQLITE_IOERR | (17<<8))\n#define SQLITE_IOERR_SHMOPEN (SQLITE_IOERR | (18<<8))\n#define SQLITE_IOERR_SHMSIZE (SQLITE_IOERR | (19<<8))\n#define SQLITE_IOERR_SHMLOCK (SQLITE_IOERR | (20<<8))\n#define SQLITE_IOERR_SHMMAP (SQLITE_IOERR | (21<<8))\n#define SQLITE_IOERR_SEEK (SQLITE_IOERR | (22<<8))\n#define SQLITE_IOERR_DELETE_NOENT (SQLITE_IOERR | (23<<8))\n#define SQLITE_IOERR_MMAP (SQLITE_IOERR | (24<<8))\n#define SQLITE_IOERR_GETTEMPPATH (SQLITE_IOERR | (25<<8))\n#define SQLITE_IOERR_CONVPATH (SQLITE_IOERR | (26<<8))\n#define SQLITE_IOERR_VNODE (SQLITE_IOERR | (27<<8))\n#define SQLITE_IOERR_AUTH (SQLITE_IOERR | (28<<8))\n#define SQLITE_IOERR_BEGIN_ATOMIC (SQLITE_IOERR | (29<<8))\n#define SQLITE_IOERR_COMMIT_ATOMIC (SQLITE_IOERR | (30<<8))\n#define SQLITE_IOERR_ROLLBACK_ATOMIC (SQLITE_IOERR | (31<<8))\n#define SQLITE_IOERR_DATA (SQLITE_IOERR | (32<<8))\n#define SQLITE_IOERR_CORRUPTFS (SQLITE_IOERR | (33<<8))\n#define SQLITE_IOERR_IN_PAGE (SQLITE_IOERR | (34<<8))\n#define SQLITE_LOCKED_SHAREDCACHE (SQLITE_LOCKED | (1<<8))\n#define SQLITE_LOCKED_VTAB (SQLITE_LOCKED | (2<<8))\n#define SQLITE_BUSY_RECOVERY (SQLITE_BUSY | (1<<8))\n#define SQLITE_BUSY_SNAPSHOT (SQLITE_BUSY | (2<<8))\n#define SQLITE_BUSY_TIMEOUT (SQLITE_BUSY | (3<<8))\n#define SQLITE_CANTOPEN_NOTEMPDIR (SQLITE_CANTOPEN | (1<<8))\n#define SQLITE_CANTOPEN_ISDIR (SQLITE_CANTOPEN | (2<<8))\n#define SQLITE_CANTOPEN_FULLPATH (SQLITE_CANTOPEN | (3<<8))\n#define SQLITE_CANTOPEN_CONVPATH (SQLITE_CANTOPEN | (4<<8))\n#define SQLITE_CANTOPEN_DIRTYWAL (SQLITE_CANTOPEN | (5<<8)) /* Not Used */\n#define SQLITE_CANTOPEN_SYMLINK (SQLITE_CANTOPEN | (6<<8))\n#define SQLITE_CORRUPT_VTAB (SQLITE_CORRUPT | (1<<8))\n#define SQLITE_CORRUPT_SEQUENCE (SQLITE_CORRUPT | (2<<8))\n#define SQLITE_CORRUPT_INDEX (SQLITE_CORRUPT | (3<<8))\n#define SQLITE_READONLY_RECOVERY (SQLITE_READONLY | (1<<8))\n#define SQLITE_READONLY_CANTLOCK (SQLITE_READONLY | (2<<8))\n#define SQLITE_READONLY_ROLLBACK (SQLITE_READONLY | (3<<8))\n#define SQLITE_READONLY_DBMOVED (SQLITE_READONLY | (4<<8))\n#define SQLITE_READONLY_CANTINIT (SQLITE_READONLY | (5<<8))\n#define SQLITE_READONLY_DIRECTORY (SQLITE_READONLY | (6<<8))\n#define SQLITE_ABORT_ROLLBACK (SQLITE_ABORT | (2<<8))\n#define SQLITE_CONSTRAINT_CHECK (SQLITE_CONSTRAINT | (1<<8))\n#define SQLITE_CONSTRAINT_COMMITHOOK (SQLITE_CONSTRAINT | (2<<8))\n#define SQLITE_CONSTRAINT_FOREIGNKEY (SQLITE_CONSTRAINT | (3<<8))\n#define SQLITE_CONSTRAINT_FUNCTION (SQLITE_CONSTRAINT | (4<<8))\n#define SQLITE_CONSTRAINT_NOTNULL (SQLITE_CONSTRAINT | (5<<8))\n#define SQLITE_CONSTRAINT_PRIMARYKEY (SQLITE_CONSTRAINT | (6<<8))\n#define SQLITE_CONSTRAINT_TRIGGER (SQLITE_CONSTRAINT | (7<<8))\n#define SQLITE_CONSTRAINT_UNIQUE (SQLITE_CONSTRAINT | (8<<8))\n#define SQLITE_CONSTRAINT_VTAB (SQLITE_CONSTRAINT | (9<<8))\n#define SQLITE_CONSTRAINT_ROWID (SQLITE_CONSTRAINT |(10<<8))\n#define SQLITE_CONSTRAINT_PINNED (SQLITE_CONSTRAINT |(11<<8))\n#define SQLITE_CONSTRAINT_DATATYPE (SQLITE_CONSTRAINT |(12<<8))\n#define SQLITE_NOTICE_RECOVER_WAL (SQLITE_NOTICE | (1<<8))\n#define SQLITE_NOTICE_RECOVER_ROLLBACK (SQLITE_NOTICE | (2<<8))\n#define SQLITE_NOTICE_RBU (SQLITE_NOTICE | (3<<8))\n#define SQLITE_WARNING_AUTOINDEX (SQLITE_WARNING | (1<<8))\n#define SQLITE_AUTH_USER (SQLITE_AUTH | (1<<8))\n#define SQLITE_OK_LOAD_PERMANENTLY (SQLITE_OK | (1<<8))\n#define SQLITE_OK_SYMLINK (SQLITE_OK | (2<<8)) /* internal use only */\n\n/*\n** CAPI3REF: Flags For File Open Operations\n**\n** These bit values are intended for use in the\n** 3rd parameter to the [sqlite3_open_v2()] interface and\n** in the 4th parameter to the [sqlite3_vfs.xOpen] method.\n**\n** Only those flags marked as \"Ok for sqlite3_open_v2()\" may be\n** used as the third argument to the [sqlite3_open_v2()] interface.\n** The other flags have historically been ignored by sqlite3_open_v2(),\n** though future versions of SQLite might change so that an error is\n** raised if any of the disallowed bits are passed into sqlite3_open_v2().\n** Applications should not depend on the historical behavior.\n**\n** Note in particular that passing the SQLITE_OPEN_EXCLUSIVE flag into\n** [sqlite3_open_v2()] does *not* cause the underlying database file\n** to be opened using O_EXCL. Passing SQLITE_OPEN_EXCLUSIVE into\n** [sqlite3_open_v2()] has historically be a no-op and might become an\n** error in future versions of SQLite.\n*/\n#define SQLITE_OPEN_READONLY 0x00000001 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_READWRITE 0x00000002 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_CREATE 0x00000004 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_DELETEONCLOSE 0x00000008 /* VFS only */\n#define SQLITE_OPEN_EXCLUSIVE 0x00000010 /* VFS only */\n#define SQLITE_OPEN_AUTOPROXY 0x00000020 /* VFS only */\n#define SQLITE_OPEN_URI 0x00000040 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_MEMORY 0x00000080 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_MAIN_DB 0x00000100 /* VFS only */\n#define SQLITE_OPEN_TEMP_DB 0x00000200 /* VFS only */\n#define SQLITE_OPEN_TRANSIENT_DB 0x00000400 /* VFS only */\n#define SQLITE_OPEN_MAIN_JOURNAL 0x00000800 /* VFS only */\n#define SQLITE_OPEN_TEMP_JOURNAL 0x00001000 /* VFS only */\n#define SQLITE_OPEN_SUBJOURNAL 0x00002000 /* VFS only */\n#define SQLITE_OPEN_SUPER_JOURNAL 0x00004000 /* VFS only */\n#define SQLITE_OPEN_NOMUTEX 0x00008000 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_FULLMUTEX 0x00010000 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_SHAREDCACHE 0x00020000 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_PRIVATECACHE 0x00040000 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_WAL 0x00080000 /* VFS only */\n#define SQLITE_OPEN_NOFOLLOW 0x01000000 /* Ok for sqlite3_open_v2() */\n#define SQLITE_OPEN_EXRESCODE 0x02000000 /* Extended result codes */\n\n/* Reserved: 0x00F00000 */\n/* Legacy compatibility: */\n#define SQLITE_OPEN_MASTER_JOURNAL 0x00004000 /* VFS only */\n\n\n/*\n** CAPI3REF: Device Characteristics\n**\n** The xDeviceCharacteristics method of the [sqlite3_io_methods]\n** object returns an integer which is a vector of these\n** bit values expressing I/O characteristics of the mass storage\n** device that holds the file that the [sqlite3_io_methods]\n** refers to.\n**\n** The SQLITE_IOCAP_ATOMIC property means that all writes of\n** any size are atomic. The SQLITE_IOCAP_ATOMICnnn values\n** mean that writes of blocks that are nnn bytes in size and\n** are aligned to an address which is an integer multiple of\n** nnn are atomic. The SQLITE_IOCAP_SAFE_APPEND value means\n** that when data is appended to a file, the data is appended\n** first then the size of the file is extended, never the other\n** way around. The SQLITE_IOCAP_SEQUENTIAL property means that\n** information is written to disk in the same order as calls\n** to xWrite(). The SQLITE_IOCAP_POWERSAFE_OVERWRITE property means that\n** after reboot following a crash or power loss, the only bytes in a\n** file that were written at the application level might have changed\n** and that adjacent bytes, even bytes within the same sector are\n** guaranteed to be unchanged. The SQLITE_IOCAP_UNDELETABLE_WHEN_OPEN\n** flag indicates that a file cannot be deleted when open. The\n** SQLITE_IOCAP_IMMUTABLE flag indicates that the file is on\n** read-only media and cannot be changed even by processes with\n** elevated privileges.\n**\n** The SQLITE_IOCAP_BATCH_ATOMIC property means that the underlying\n** filesystem supports doing multiple write operations atomically when those\n** write operations are bracketed by [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE] and\n** [SQLITE_FCNTL_COMMIT_ATOMIC_WRITE].\n**\n** The SQLITE_IOCAP_SUBPAGE_READ property means that it is ok to read\n** from the database file in amounts that are not a multiple of the\n** page size and that do not begin at a page boundary. Without this\n** property, SQLite is careful to only do full-page reads and write\n** on aligned pages, with the one exception that it will do a sub-page\n** read of the first page to access the database header.\n*/\n#define SQLITE_IOCAP_ATOMIC 0x00000001\n#define SQLITE_IOCAP_ATOMIC512 0x00000002\n#define SQLITE_IOCAP_ATOMIC1K 0x00000004\n#define SQLITE_IOCAP_ATOMIC2K 0x00000008\n#define SQLITE_IOCAP_ATOMIC4K 0x00000010\n#define SQLITE_IOCAP_ATOMIC8K 0x00000020\n#define SQLITE_IOCAP_ATOMIC16K 0x00000040\n#define SQLITE_IOCAP_ATOMIC32K 0x00000080\n#define SQLITE_IOCAP_ATOMIC64K 0x00000100\n#define SQLITE_IOCAP_SAFE_APPEND 0x00000200\n#define SQLITE_IOCAP_SEQUENTIAL 0x00000400\n#define SQLITE_IOCAP_UNDELETABLE_WHEN_OPEN 0x00000800\n#define SQLITE_IOCAP_POWERSAFE_OVERWRITE 0x00001000\n#define SQLITE_IOCAP_IMMUTABLE 0x00002000\n#define SQLITE_IOCAP_BATCH_ATOMIC 0x00004000\n#define SQLITE_IOCAP_SUBPAGE_READ 0x00008000\n\n/*\n** CAPI3REF: File Locking Levels\n**\n** SQLite uses one of these integer values as the second\n** argument to calls it makes to the xLock() and xUnlock() methods\n** of an [sqlite3_io_methods] object. These values are ordered from\n** lest restrictive to most restrictive.\n**\n** The argument to xLock() is always SHARED or higher. The argument to\n** xUnlock is either SHARED or NONE.\n*/\n#define SQLITE_LOCK_NONE 0 /* xUnlock() only */\n#define SQLITE_LOCK_SHARED 1 /* xLock() or xUnlock() */\n#define SQLITE_LOCK_RESERVED 2 /* xLock() only */\n#define SQLITE_LOCK_PENDING 3 /* xLock() only */\n#define SQLITE_LOCK_EXCLUSIVE 4 /* xLock() only */\n\n/*\n** CAPI3REF: Synchronization Type Flags\n**\n** When SQLite invokes the xSync() method of an\n** [sqlite3_io_methods] object it uses a combination of\n** these integer values as the second argument.\n**\n** When the SQLITE_SYNC_DATAONLY flag is used, it means that the\n** sync operation only needs to flush data to mass storage. Inode\n** information need not be flushed. If the lower four bits of the flag\n** equal SQLITE_SYNC_NORMAL, that means to use normal fsync() semantics.\n** If the lower four bits equal SQLITE_SYNC_FULL, that means\n** to use Mac OS X style fullsync instead of fsync().\n**\n** Do not confuse the SQLITE_SYNC_NORMAL and SQLITE_SYNC_FULL flags\n** with the [PRAGMA synchronous]=NORMAL and [PRAGMA synchronous]=FULL\n** settings. The [synchronous pragma] determines when calls to the\n** xSync VFS method occur and applies uniformly across all platforms.\n** The SQLITE_SYNC_NORMAL and SQLITE_SYNC_FULL flags determine how\n** energetic or rigorous or forceful the sync operations are and\n** only make a difference on Mac OSX for the default SQLite code.\n** (Third-party VFS implementations might also make the distinction\n** between SQLITE_SYNC_NORMAL and SQLITE_SYNC_FULL, but among the\n** operating systems natively supported by SQLite, only Mac OSX\n** cares about the difference.)\n*/\n#define SQLITE_SYNC_NORMAL 0x00002\n#define SQLITE_SYNC_FULL 0x00003\n#define SQLITE_SYNC_DATAONLY 0x00010\n\n/*\n** CAPI3REF: OS Interface Open File Handle\n**\n** An [sqlite3_file] object represents an open file in the\n** [sqlite3_vfs | OS interface layer]. Individual OS interface\n** implementations will\n** want to subclass this object by appending additional fields\n** for their own use. The pMethods entry is a pointer to an\n** [sqlite3_io_methods] object that defines methods for performing\n** I/O operations on the open file.\n*/\ntypedef struct sqlite3_file sqlite3_file;\nstruct sqlite3_file {\n const struct sqlite3_io_methods *pMethods; /* Methods for an open file */\n};\n\n/*\n** CAPI3REF: OS Interface File Virtual Methods Object\n**\n** Every file opened by the [sqlite3_vfs.xOpen] method populates an\n** [sqlite3_file] object (or, more commonly, a subclass of the\n** [sqlite3_file] object) with a pointer to an instance of this object.\n** This object defines the methods used to perform various operations\n** against the open file represented by the [sqlite3_file] object.\n**\n** If the [sqlite3_vfs.xOpen] method sets the sqlite3_file.pMethods element\n** to a non-NULL pointer, then the sqlite3_io_methods.xClose method\n** may be invoked even if the [sqlite3_vfs.xOpen] reported that it failed. The\n** only way to prevent a call to xClose following a failed [sqlite3_vfs.xOpen]\n** is for the [sqlite3_vfs.xOpen] to set the sqlite3_file.pMethods element\n** to NULL.\n**\n** The flags argument to xSync may be one of [SQLITE_SYNC_NORMAL] or\n** [SQLITE_SYNC_FULL]. The first choice is the normal fsync().\n** The second choice is a Mac OS X style fullsync. The [SQLITE_SYNC_DATAONLY]\n** flag may be ORed in to indicate that only the data of the file\n** and not its inode needs to be synced.\n**\n** The integer values to xLock() and xUnlock() are one of\n**
    \n**
  • [SQLITE_LOCK_NONE],\n**
  • [SQLITE_LOCK_SHARED],\n**
  • [SQLITE_LOCK_RESERVED],\n**
  • [SQLITE_LOCK_PENDING], or\n**
  • [SQLITE_LOCK_EXCLUSIVE].\n**
\n** xLock() upgrades the database file lock. In other words, xLock() moves the\n** database file lock in the direction NONE toward EXCLUSIVE. The argument to\n** xLock() is always one of SHARED, RESERVED, PENDING, or EXCLUSIVE, never\n** SQLITE_LOCK_NONE. If the database file lock is already at or above the\n** requested lock, then the call to xLock() is a no-op.\n** xUnlock() downgrades the database file lock to either SHARED or NONE.\n** If the lock is already at or below the requested lock state, then the call\n** to xUnlock() is a no-op.\n** The xCheckReservedLock() method checks whether any database connection,\n** either in this process or in some other process, is holding a RESERVED,\n** PENDING, or EXCLUSIVE lock on the file. It returns, via its output\n** pointer parameter, true if such a lock exists and false otherwise.\n**\n** The xFileControl() method is a generic interface that allows custom\n** VFS implementations to directly control an open file using the\n** [sqlite3_file_control()] interface. The second \"op\" argument is an\n** integer opcode. The third argument is a generic pointer intended to\n** point to a structure that may contain arguments or space in which to\n** write return values. Potential uses for xFileControl() might be\n** functions to enable blocking locks with timeouts, to change the\n** locking strategy (for example to use dot-file locks), to inquire\n** about the status of a lock, or to break stale locks. The SQLite\n** core reserves all opcodes less than 100 for its own use.\n** A [file control opcodes | list of opcodes] less than 100 is available.\n** Applications that define a custom xFileControl method should use opcodes\n** greater than 100 to avoid conflicts. VFS implementations should\n** return [SQLITE_NOTFOUND] for file control opcodes that they do not\n** recognize.\n**\n** The xSectorSize() method returns the sector size of the\n** device that underlies the file. The sector size is the\n** minimum write that can be performed without disturbing\n** other bytes in the file. The xDeviceCharacteristics()\n** method returns a bit vector describing behaviors of the\n** underlying device:\n**\n**
    \n**
  • [SQLITE_IOCAP_ATOMIC]\n**
  • [SQLITE_IOCAP_ATOMIC512]\n**
  • [SQLITE_IOCAP_ATOMIC1K]\n**
  • [SQLITE_IOCAP_ATOMIC2K]\n**
  • [SQLITE_IOCAP_ATOMIC4K]\n**
  • [SQLITE_IOCAP_ATOMIC8K]\n**
  • [SQLITE_IOCAP_ATOMIC16K]\n**
  • [SQLITE_IOCAP_ATOMIC32K]\n**
  • [SQLITE_IOCAP_ATOMIC64K]\n**
  • [SQLITE_IOCAP_SAFE_APPEND]\n**
  • [SQLITE_IOCAP_SEQUENTIAL]\n**
  • [SQLITE_IOCAP_UNDELETABLE_WHEN_OPEN]\n**
  • [SQLITE_IOCAP_POWERSAFE_OVERWRITE]\n**
  • [SQLITE_IOCAP_IMMUTABLE]\n**
  • [SQLITE_IOCAP_BATCH_ATOMIC]\n**
  • [SQLITE_IOCAP_SUBPAGE_READ]\n**
\n**\n** The SQLITE_IOCAP_ATOMIC property means that all writes of\n** any size are atomic. The SQLITE_IOCAP_ATOMICnnn values\n** mean that writes of blocks that are nnn bytes in size and\n** are aligned to an address which is an integer multiple of\n** nnn are atomic. The SQLITE_IOCAP_SAFE_APPEND value means\n** that when data is appended to a file, the data is appended\n** first then the size of the file is extended, never the other\n** way around. The SQLITE_IOCAP_SEQUENTIAL property means that\n** information is written to disk in the same order as calls\n** to xWrite().\n**\n** If xRead() returns SQLITE_IOERR_SHORT_READ it must also fill\n** in the unread portions of the buffer with zeros. A VFS that\n** fails to zero-fill short reads might seem to work. However,\n** failure to zero-fill short reads will eventually lead to\n** database corruption.\n*/\ntypedef struct sqlite3_io_methods sqlite3_io_methods;\nstruct sqlite3_io_methods {\n int iVersion;\n int (*xClose)(sqlite3_file*);\n int (*xRead)(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);\n int (*xWrite)(sqlite3_file*, const void*, int iAmt, sqlite3_int64 iOfst);\n int (*xTruncate)(sqlite3_file*, sqlite3_int64 size);\n int (*xSync)(sqlite3_file*, int flags);\n int (*xFileSize)(sqlite3_file*, sqlite3_int64 *pSize);\n int (*xLock)(sqlite3_file*, int);\n int (*xUnlock)(sqlite3_file*, int);\n int (*xCheckReservedLock)(sqlite3_file*, int *pResOut);\n int (*xFileControl)(sqlite3_file*, int op, void *pArg);\n int (*xSectorSize)(sqlite3_file*);\n int (*xDeviceCharacteristics)(sqlite3_file*);\n /* Methods above are valid for version 1 */\n int (*xShmMap)(sqlite3_file*, int iPg, int pgsz, int, void volatile**);\n int (*xShmLock)(sqlite3_file*, int offset, int n, int flags);\n void (*xShmBarrier)(sqlite3_file*);\n int (*xShmUnmap)(sqlite3_file*, int deleteFlag);\n /* Methods above are valid for version 2 */\n int (*xFetch)(sqlite3_file*, sqlite3_int64 iOfst, int iAmt, void **pp);\n int (*xUnfetch)(sqlite3_file*, sqlite3_int64 iOfst, void *p);\n /* Methods above are valid for version 3 */\n /* Additional methods may be added in future releases */\n};\n\n/*\n** CAPI3REF: Standard File Control Opcodes\n** KEYWORDS: {file control opcodes} {file control opcode}\n**\n** These integer constants are opcodes for the xFileControl method\n** of the [sqlite3_io_methods] object and for the [sqlite3_file_control()]\n** interface.\n**\n**
    \n**
  • [[SQLITE_FCNTL_LOCKSTATE]]\n** The [SQLITE_FCNTL_LOCKSTATE] opcode is used for debugging. This\n** opcode causes the xFileControl method to write the current state of\n** the lock (one of [SQLITE_LOCK_NONE], [SQLITE_LOCK_SHARED],\n** [SQLITE_LOCK_RESERVED], [SQLITE_LOCK_PENDING], or [SQLITE_LOCK_EXCLUSIVE])\n** into an integer that the pArg argument points to.\n** This capability is only available if SQLite is compiled with [SQLITE_DEBUG].\n**\n**
  • [[SQLITE_FCNTL_SIZE_HINT]]\n** The [SQLITE_FCNTL_SIZE_HINT] opcode is used by SQLite to give the VFS\n** layer a hint of how large the database file will grow to be during the\n** current transaction. This hint is not guaranteed to be accurate but it\n** is often close. The underlying VFS might choose to preallocate database\n** file space based on this hint in order to help writes to the database\n** file run faster.\n**\n**
  • [[SQLITE_FCNTL_SIZE_LIMIT]]\n** The [SQLITE_FCNTL_SIZE_LIMIT] opcode is used by in-memory VFS that\n** implements [sqlite3_deserialize()] to set an upper bound on the size\n** of the in-memory database. The argument is a pointer to a [sqlite3_int64].\n** If the integer pointed to is negative, then it is filled in with the\n** current limit. Otherwise the limit is set to the larger of the value\n** of the integer pointed to and the current database size. The integer\n** pointed to is set to the new limit.\n**\n**
  • [[SQLITE_FCNTL_CHUNK_SIZE]]\n** The [SQLITE_FCNTL_CHUNK_SIZE] opcode is used to request that the VFS\n** extends and truncates the database file in chunks of a size specified\n** by the user. The fourth argument to [sqlite3_file_control()] should\n** point to an integer (type int) containing the new chunk-size to use\n** for the nominated database. Allocating database file space in large\n** chunks (say 1MB at a time), may reduce file-system fragmentation and\n** improve performance on some systems.\n**\n**
  • [[SQLITE_FCNTL_FILE_POINTER]]\n** The [SQLITE_FCNTL_FILE_POINTER] opcode is used to obtain a pointer\n** to the [sqlite3_file] object associated with a particular database\n** connection. See also [SQLITE_FCNTL_JOURNAL_POINTER].\n**\n**
  • [[SQLITE_FCNTL_JOURNAL_POINTER]]\n** The [SQLITE_FCNTL_JOURNAL_POINTER] opcode is used to obtain a pointer\n** to the [sqlite3_file] object associated with the journal file (either\n** the [rollback journal] or the [write-ahead log]) for a particular database\n** connection. See also [SQLITE_FCNTL_FILE_POINTER].\n**\n**
  • [[SQLITE_FCNTL_SYNC_OMITTED]]\n** No longer in use.\n**\n**
  • [[SQLITE_FCNTL_SYNC]]\n** The [SQLITE_FCNTL_SYNC] opcode is generated internally by SQLite and\n** sent to the VFS immediately before the xSync method is invoked on a\n** database file descriptor. Or, if the xSync method is not invoked\n** because the user has configured SQLite with\n** [PRAGMA synchronous | PRAGMA synchronous=OFF] it is invoked in place\n** of the xSync method. In most cases, the pointer argument passed with\n** this file-control is NULL. However, if the database file is being synced\n** as part of a multi-database commit, the argument points to a nul-terminated\n** string containing the transactions super-journal file name. VFSes that\n** do not need this signal should silently ignore this opcode. Applications\n** should not call [sqlite3_file_control()] with this opcode as doing so may\n** disrupt the operation of the specialized VFSes that do require it.\n**\n**
  • [[SQLITE_FCNTL_COMMIT_PHASETWO]]\n** The [SQLITE_FCNTL_COMMIT_PHASETWO] opcode is generated internally by SQLite\n** and sent to the VFS after a transaction has been committed immediately\n** but before the database is unlocked. VFSes that do not need this signal\n** should silently ignore this opcode. Applications should not call\n** [sqlite3_file_control()] with this opcode as doing so may disrupt the\n** operation of the specialized VFSes that do require it.\n**\n**
  • [[SQLITE_FCNTL_WIN32_AV_RETRY]]\n** ^The [SQLITE_FCNTL_WIN32_AV_RETRY] opcode is used to configure automatic\n** retry counts and intervals for certain disk I/O operations for the\n** windows [VFS] in order to provide robustness in the presence of\n** anti-virus programs. By default, the windows VFS will retry file read,\n** file write, and file delete operations up to 10 times, with a delay\n** of 25 milliseconds before the first retry and with the delay increasing\n** by an additional 25 milliseconds with each subsequent retry. This\n** opcode allows these two values (10 retries and 25 milliseconds of delay)\n** to be adjusted. The values are changed for all database connections\n** within the same process. The argument is a pointer to an array of two\n** integers where the first integer is the new retry count and the second\n** integer is the delay. If either integer is negative, then the setting\n** is not changed but instead the prior value of that setting is written\n** into the array entry, allowing the current retry settings to be\n** interrogated. The zDbName parameter is ignored.\n**\n**
  • [[SQLITE_FCNTL_PERSIST_WAL]]\n** ^The [SQLITE_FCNTL_PERSIST_WAL] opcode is used to set or query the\n** persistent [WAL | Write Ahead Log] setting. By default, the auxiliary\n** write ahead log ([WAL file]) and shared memory\n** files used for transaction control\n** are automatically deleted when the latest connection to the database\n** closes. Setting persistent WAL mode causes those files to persist after\n** close. Persisting the files is useful when other processes that do not\n** have write permission on the directory containing the database file want\n** to read the database file, as the WAL and shared memory files must exist\n** in order for the database to be readable. The fourth parameter to\n** [sqlite3_file_control()] for this opcode should be a pointer to an integer.\n** That integer is 0 to disable persistent WAL mode or 1 to enable persistent\n** WAL mode. If the integer is -1, then it is overwritten with the current\n** WAL persistence setting.\n**\n**
  • [[SQLITE_FCNTL_POWERSAFE_OVERWRITE]]\n** ^The [SQLITE_FCNTL_POWERSAFE_OVERWRITE] opcode is used to set or query the\n** persistent \"powersafe-overwrite\" or \"PSOW\" setting. The PSOW setting\n** determines the [SQLITE_IOCAP_POWERSAFE_OVERWRITE] bit of the\n** xDeviceCharacteristics methods. The fourth parameter to\n** [sqlite3_file_control()] for this opcode should be a pointer to an integer.\n** That integer is 0 to disable zero-damage mode or 1 to enable zero-damage\n** mode. If the integer is -1, then it is overwritten with the current\n** zero-damage mode setting.\n**\n**
  • [[SQLITE_FCNTL_OVERWRITE]]\n** ^The [SQLITE_FCNTL_OVERWRITE] opcode is invoked by SQLite after opening\n** a write transaction to indicate that, unless it is rolled back for some\n** reason, the entire database file will be overwritten by the current\n** transaction. This is used by VACUUM operations.\n**\n**
  • [[SQLITE_FCNTL_VFSNAME]]\n** ^The [SQLITE_FCNTL_VFSNAME] opcode can be used to obtain the names of\n** all [VFSes] in the VFS stack. The names are of all VFS shims and the\n** final bottom-level VFS are written into memory obtained from\n** [sqlite3_malloc()] and the result is stored in the char* variable\n** that the fourth parameter of [sqlite3_file_control()] points to.\n** The caller is responsible for freeing the memory when done. As with\n** all file-control actions, there is no guarantee that this will actually\n** do anything. Callers should initialize the char* variable to a NULL\n** pointer in case this file-control is not implemented. This file-control\n** is intended for diagnostic use only.\n**\n**
  • [[SQLITE_FCNTL_VFS_POINTER]]\n** ^The [SQLITE_FCNTL_VFS_POINTER] opcode finds a pointer to the top-level\n** [VFSes] currently in use. ^(The argument X in\n** sqlite3_file_control(db,SQLITE_FCNTL_VFS_POINTER,X) must be\n** of type \"[sqlite3_vfs] **\". This opcodes will set *X\n** to a pointer to the top-level VFS.)^\n** ^When there are multiple VFS shims in the stack, this opcode finds the\n** upper-most shim only.\n**\n**
  • [[SQLITE_FCNTL_PRAGMA]]\n** ^Whenever a [PRAGMA] statement is parsed, an [SQLITE_FCNTL_PRAGMA]\n** file control is sent to the open [sqlite3_file] object corresponding\n** to the database file to which the pragma statement refers. ^The argument\n** to the [SQLITE_FCNTL_PRAGMA] file control is an array of\n** pointers to strings (char**) in which the second element of the array\n** is the name of the pragma and the third element is the argument to the\n** pragma or NULL if the pragma has no argument. ^The handler for an\n** [SQLITE_FCNTL_PRAGMA] file control can optionally make the first element\n** of the char** argument point to a string obtained from [sqlite3_mprintf()]\n** or the equivalent and that string will become the result of the pragma or\n** the error message if the pragma fails. ^If the\n** [SQLITE_FCNTL_PRAGMA] file control returns [SQLITE_NOTFOUND], then normal\n** [PRAGMA] processing continues. ^If the [SQLITE_FCNTL_PRAGMA]\n** file control returns [SQLITE_OK], then the parser assumes that the\n** VFS has handled the PRAGMA itself and the parser generates a no-op\n** prepared statement if result string is NULL, or that returns a copy\n** of the result string if the string is non-NULL.\n** ^If the [SQLITE_FCNTL_PRAGMA] file control returns\n** any result code other than [SQLITE_OK] or [SQLITE_NOTFOUND], that means\n** that the VFS encountered an error while handling the [PRAGMA] and the\n** compilation of the PRAGMA fails with an error. ^The [SQLITE_FCNTL_PRAGMA]\n** file control occurs at the beginning of pragma statement analysis and so\n** it is able to override built-in [PRAGMA] statements.\n**\n**
  • [[SQLITE_FCNTL_BUSYHANDLER]]\n** ^The [SQLITE_FCNTL_BUSYHANDLER]\n** file-control may be invoked by SQLite on the database file handle\n** shortly after it is opened in order to provide a custom VFS with access\n** to the connection's busy-handler callback. The argument is of type (void**)\n** - an array of two (void *) values. The first (void *) actually points\n** to a function of type (int (*)(void *)). In order to invoke the connection's\n** busy-handler, this function should be invoked with the second (void *) in\n** the array as the only argument. If it returns non-zero, then the operation\n** should be retried. If it returns zero, the custom VFS should abandon the\n** current operation.\n**\n**
  • [[SQLITE_FCNTL_TEMPFILENAME]]\n** ^Applications can invoke the [SQLITE_FCNTL_TEMPFILENAME] file-control\n** to have SQLite generate a\n** temporary filename using the same algorithm that is followed to generate\n** temporary filenames for TEMP tables and other internal uses. The\n** argument should be a char** which will be filled with the filename\n** written into memory obtained from [sqlite3_malloc()]. The caller should\n** invoke [sqlite3_free()] on the result to avoid a memory leak.\n**\n**
  • [[SQLITE_FCNTL_MMAP_SIZE]]\n** The [SQLITE_FCNTL_MMAP_SIZE] file control is used to query or set the\n** maximum number of bytes that will be used for memory-mapped I/O.\n** The argument is a pointer to a value of type sqlite3_int64 that\n** is an advisory maximum number of bytes in the file to memory map. The\n** pointer is overwritten with the old value. The limit is not changed if\n** the value originally pointed to is negative, and so the current limit\n** can be queried by passing in a pointer to a negative number. This\n** file-control is used internally to implement [PRAGMA mmap_size].\n**\n**
  • [[SQLITE_FCNTL_TRACE]]\n** The [SQLITE_FCNTL_TRACE] file control provides advisory information\n** to the VFS about what the higher layers of the SQLite stack are doing.\n** This file control is used by some VFS activity tracing [shims].\n** The argument is a zero-terminated string. Higher layers in the\n** SQLite stack may generate instances of this file control if\n** the [SQLITE_USE_FCNTL_TRACE] compile-time option is enabled.\n**\n**
  • [[SQLITE_FCNTL_HAS_MOVED]]\n** The [SQLITE_FCNTL_HAS_MOVED] file control interprets its argument as a\n** pointer to an integer and it writes a boolean into that integer depending\n** on whether or not the file has been renamed, moved, or deleted since it\n** was first opened.\n**\n**
  • [[SQLITE_FCNTL_WIN32_GET_HANDLE]]\n** The [SQLITE_FCNTL_WIN32_GET_HANDLE] opcode can be used to obtain the\n** underlying native file handle associated with a file handle. This file\n** control interprets its argument as a pointer to a native file handle and\n** writes the resulting value there.\n**\n**
  • [[SQLITE_FCNTL_WIN32_SET_HANDLE]]\n** The [SQLITE_FCNTL_WIN32_SET_HANDLE] opcode is used for debugging. This\n** opcode causes the xFileControl method to swap the file handle with the one\n** pointed to by the pArg argument. This capability is used during testing\n** and only needs to be supported when SQLITE_TEST is defined.\n**\n**
  • [[SQLITE_FCNTL_NULL_IO]]\n** The [SQLITE_FCNTL_NULL_IO] opcode sets the low-level file descriptor\n** or file handle for the [sqlite3_file] object such that it will no longer\n** read or write to the database file.\n**\n**
  • [[SQLITE_FCNTL_WAL_BLOCK]]\n** The [SQLITE_FCNTL_WAL_BLOCK] is a signal to the VFS layer that it might\n** be advantageous to block on the next WAL lock if the lock is not immediately\n** available. The WAL subsystem issues this signal during rare\n** circumstances in order to fix a problem with priority inversion.\n** Applications should not use this file-control.\n**\n**
  • [[SQLITE_FCNTL_ZIPVFS]]\n** The [SQLITE_FCNTL_ZIPVFS] opcode is implemented by zipvfs only. All other\n** VFS should return SQLITE_NOTFOUND for this opcode.\n**\n**
  • [[SQLITE_FCNTL_RBU]]\n** The [SQLITE_FCNTL_RBU] opcode is implemented by the special VFS used by\n** the RBU extension only. All other VFS should return SQLITE_NOTFOUND for\n** this opcode.\n**\n**
  • [[SQLITE_FCNTL_BEGIN_ATOMIC_WRITE]]\n** If the [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE] opcode returns SQLITE_OK, then\n** the file descriptor is placed in \"batch write mode\", which\n** means all subsequent write operations will be deferred and done\n** atomically at the next [SQLITE_FCNTL_COMMIT_ATOMIC_WRITE]. Systems\n** that do not support batch atomic writes will return SQLITE_NOTFOUND.\n** ^Following a successful SQLITE_FCNTL_BEGIN_ATOMIC_WRITE and prior to\n** the closing [SQLITE_FCNTL_COMMIT_ATOMIC_WRITE] or\n** [SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE], SQLite will make\n** no VFS interface calls on the same [sqlite3_file] file descriptor\n** except for calls to the xWrite method and the xFileControl method\n** with [SQLITE_FCNTL_SIZE_HINT].\n**\n**
  • [[SQLITE_FCNTL_COMMIT_ATOMIC_WRITE]]\n** The [SQLITE_FCNTL_COMMIT_ATOMIC_WRITE] opcode causes all write\n** operations since the previous successful call to\n** [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE] to be performed atomically.\n** This file control returns [SQLITE_OK] if and only if the writes were\n** all performed successfully and have been committed to persistent storage.\n** ^Regardless of whether or not it is successful, this file control takes\n** the file descriptor out of batch write mode so that all subsequent\n** write operations are independent.\n** ^SQLite will never invoke SQLITE_FCNTL_COMMIT_ATOMIC_WRITE without\n** a prior successful call to [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE].\n**\n**
  • [[SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE]]\n** The [SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE] opcode causes all write\n** operations since the previous successful call to\n** [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE] to be rolled back.\n** ^This file control takes the file descriptor out of batch write mode\n** so that all subsequent write operations are independent.\n** ^SQLite will never invoke SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE without\n** a prior successful call to [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE].\n**\n**
  • [[SQLITE_FCNTL_LOCK_TIMEOUT]]\n** The [SQLITE_FCNTL_LOCK_TIMEOUT] opcode is used to configure a VFS\n** to block for up to M milliseconds before failing when attempting to\n** obtain a file lock using the xLock or xShmLock methods of the VFS.\n** The parameter is a pointer to a 32-bit signed integer that contains\n** the value that M is to be set to. Before returning, the 32-bit signed\n** integer is overwritten with the previous value of M.\n**\n**
  • [[SQLITE_FCNTL_BLOCK_ON_CONNECT]]\n** The [SQLITE_FCNTL_BLOCK_ON_CONNECT] opcode is used to configure the\n** VFS to block when taking a SHARED lock to connect to a wal mode database.\n** This is used to implement the functionality associated with\n** SQLITE_SETLK_BLOCK_ON_CONNECT.\n**\n**
  • [[SQLITE_FCNTL_DATA_VERSION]]\n** The [SQLITE_FCNTL_DATA_VERSION] opcode is used to detect changes to\n** a database file. The argument is a pointer to a 32-bit unsigned integer.\n** The \"data version\" for the pager is written into the pointer. The\n** \"data version\" changes whenever any change occurs to the corresponding\n** database file, either through SQL statements on the same database\n** connection or through transactions committed by separate database\n** connections possibly in other processes. The [sqlite3_total_changes()]\n** interface can be used to find if any database on the connection has changed,\n** but that interface responds to changes on TEMP as well as MAIN and does\n** not provide a mechanism to detect changes to MAIN only. Also, the\n** [sqlite3_total_changes()] interface responds to internal changes only and\n** omits changes made by other database connections. The\n** [PRAGMA data_version] command provides a mechanism to detect changes to\n** a single attached database that occur due to other database connections,\n** but omits changes implemented by the database connection on which it is\n** called. This file control is the only mechanism to detect changes that\n** happen either internally or externally and that are associated with\n** a particular attached database.\n**\n**
  • [[SQLITE_FCNTL_CKPT_START]]\n** The [SQLITE_FCNTL_CKPT_START] opcode is invoked from within a checkpoint\n** in wal mode before the client starts to copy pages from the wal\n** file to the database file.\n**\n**
  • [[SQLITE_FCNTL_CKPT_DONE]]\n** The [SQLITE_FCNTL_CKPT_DONE] opcode is invoked from within a checkpoint\n** in wal mode after the client has finished copying pages from the wal\n** file to the database file, but before the *-shm file is updated to\n** record the fact that the pages have been checkpointed.\n**\n**
  • [[SQLITE_FCNTL_EXTERNAL_READER]]\n** The EXPERIMENTAL [SQLITE_FCNTL_EXTERNAL_READER] opcode is used to detect\n** whether or not there is a database client in another process with a wal-mode\n** transaction open on the database or not. It is only available on unix.The\n** (void*) argument passed with this file-control should be a pointer to a\n** value of type (int). The integer value is set to 1 if the database is a wal\n** mode database and there exists at least one client in another process that\n** currently has an SQL transaction open on the database. It is set to 0 if\n** the database is not a wal-mode db, or if there is no such connection in any\n** other process. This opcode cannot be used to detect transactions opened\n** by clients within the current process, only within other processes.\n**\n**
  • [[SQLITE_FCNTL_CKSM_FILE]]\n** The [SQLITE_FCNTL_CKSM_FILE] opcode is for use internally by the\n** [checksum VFS shim] only.\n**\n**
  • [[SQLITE_FCNTL_RESET_CACHE]]\n** If there is currently no transaction open on the database, and the\n** database is not a temp db, then the [SQLITE_FCNTL_RESET_CACHE] file-control\n** purges the contents of the in-memory page cache. If there is an open\n** transaction, or if the db is a temp-db, this opcode is a no-op, not an error.\n**
\n*/\n#define SQLITE_FCNTL_LOCKSTATE 1\n#define SQLITE_FCNTL_GET_LOCKPROXYFILE 2\n#define SQLITE_FCNTL_SET_LOCKPROXYFILE 3\n#define SQLITE_FCNTL_LAST_ERRNO 4\n#define SQLITE_FCNTL_SIZE_HINT 5\n#define SQLITE_FCNTL_CHUNK_SIZE 6\n#define SQLITE_FCNTL_FILE_POINTER 7\n#define SQLITE_FCNTL_SYNC_OMITTED 8\n#define SQLITE_FCNTL_WIN32_AV_RETRY 9\n#define SQLITE_FCNTL_PERSIST_WAL 10\n#define SQLITE_FCNTL_OVERWRITE 11\n#define SQLITE_FCNTL_VFSNAME 12\n#define SQLITE_FCNTL_POWERSAFE_OVERWRITE 13\n#define SQLITE_FCNTL_PRAGMA 14\n#define SQLITE_FCNTL_BUSYHANDLER 15\n#define SQLITE_FCNTL_TEMPFILENAME 16\n#define SQLITE_FCNTL_MMAP_SIZE 18\n#define SQLITE_FCNTL_TRACE 19\n#define SQLITE_FCNTL_HAS_MOVED 20\n#define SQLITE_FCNTL_SYNC 21\n#define SQLITE_FCNTL_COMMIT_PHASETWO 22\n#define SQLITE_FCNTL_WIN32_SET_HANDLE 23\n#define SQLITE_FCNTL_WAL_BLOCK 24\n#define SQLITE_FCNTL_ZIPVFS 25\n#define SQLITE_FCNTL_RBU 26\n#define SQLITE_FCNTL_VFS_POINTER 27\n#define SQLITE_FCNTL_JOURNAL_POINTER 28\n#define SQLITE_FCNTL_WIN32_GET_HANDLE 29\n#define SQLITE_FCNTL_PDB 30\n#define SQLITE_FCNTL_BEGIN_ATOMIC_WRITE 31\n#define SQLITE_FCNTL_COMMIT_ATOMIC_WRITE 32\n#define SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE 33\n#define SQLITE_FCNTL_LOCK_TIMEOUT 34\n#define SQLITE_FCNTL_DATA_VERSION 35\n#define SQLITE_FCNTL_SIZE_LIMIT 36\n#define SQLITE_FCNTL_CKPT_DONE 37\n#define SQLITE_FCNTL_RESERVE_BYTES 38\n#define SQLITE_FCNTL_CKPT_START 39\n#define SQLITE_FCNTL_EXTERNAL_READER 40\n#define SQLITE_FCNTL_CKSM_FILE 41\n#define SQLITE_FCNTL_RESET_CACHE 42\n#define SQLITE_FCNTL_NULL_IO 43\n#define SQLITE_FCNTL_BLOCK_ON_CONNECT 44\n\n/* deprecated names */\n#define SQLITE_GET_LOCKPROXYFILE SQLITE_FCNTL_GET_LOCKPROXYFILE\n#define SQLITE_SET_LOCKPROXYFILE SQLITE_FCNTL_SET_LOCKPROXYFILE\n#define SQLITE_LAST_ERRNO SQLITE_FCNTL_LAST_ERRNO\n\n\n/*\n** CAPI3REF: Mutex Handle\n**\n** The mutex module within SQLite defines [sqlite3_mutex] to be an\n** abstract type for a mutex object. The SQLite core never looks\n** at the internal representation of an [sqlite3_mutex]. It only\n** deals with pointers to the [sqlite3_mutex] object.\n**\n** Mutexes are created using [sqlite3_mutex_alloc()].\n*/\ntypedef struct sqlite3_mutex sqlite3_mutex;\n\n/*\n** CAPI3REF: Loadable Extension Thunk\n**\n** A pointer to the opaque sqlite3_api_routines structure is passed as\n** the third parameter to entry points of [loadable extensions]. This\n** structure must be typedefed in order to work around compiler warnings\n** on some platforms.\n*/\ntypedef struct sqlite3_api_routines sqlite3_api_routines;\n\n/*\n** CAPI3REF: File Name\n**\n** Type [sqlite3_filename] is used by SQLite to pass filenames to the\n** xOpen method of a [VFS]. It may be cast to (const char*) and treated\n** as a normal, nul-terminated, UTF-8 buffer containing the filename, but\n** may also be passed to special APIs such as:\n**\n**
    \n**
  • sqlite3_filename_database()\n**
  • sqlite3_filename_journal()\n**
  • sqlite3_filename_wal()\n**
  • sqlite3_uri_parameter()\n**
  • sqlite3_uri_boolean()\n**
  • sqlite3_uri_int64()\n**
  • sqlite3_uri_key()\n**
\n*/\ntypedef const char *sqlite3_filename;\n\n/*\n** CAPI3REF: OS Interface Object\n**\n** An instance of the sqlite3_vfs object defines the interface between\n** the SQLite core and the underlying operating system. The \"vfs\"\n** in the name of the object stands for \"virtual file system\". See\n** the [VFS | VFS documentation] for further information.\n**\n** The VFS interface is sometimes extended by adding new methods onto\n** the end. Each time such an extension occurs, the iVersion field\n** is incremented. The iVersion value started out as 1 in\n** SQLite [version 3.5.0] on [dateof:3.5.0], then increased to 2\n** with SQLite [version 3.7.0] on [dateof:3.7.0], and then increased\n** to 3 with SQLite [version 3.7.6] on [dateof:3.7.6]. Additional fields\n** may be appended to the sqlite3_vfs object and the iVersion value\n** may increase again in future versions of SQLite.\n** Note that due to an oversight, the structure\n** of the sqlite3_vfs object changed in the transition from\n** SQLite [version 3.5.9] to [version 3.6.0] on [dateof:3.6.0]\n** and yet the iVersion field was not increased.\n**\n** The szOsFile field is the size of the subclassed [sqlite3_file]\n** structure used by this VFS. mxPathname is the maximum length of\n** a pathname in this VFS.\n**\n** Registered sqlite3_vfs objects are kept on a linked list formed by\n** the pNext pointer. The [sqlite3_vfs_register()]\n** and [sqlite3_vfs_unregister()] interfaces manage this list\n** in a thread-safe way. The [sqlite3_vfs_find()] interface\n** searches the list. Neither the application code nor the VFS\n** implementation should use the pNext pointer.\n**\n** The pNext field is the only field in the sqlite3_vfs\n** structure that SQLite will ever modify. SQLite will only access\n** or modify this field while holding a particular static mutex.\n** The application should never modify anything within the sqlite3_vfs\n** object once the object has been registered.\n**\n** The zName field holds the name of the VFS module. The name must\n** be unique across all VFS modules.\n**\n** [[sqlite3_vfs.xOpen]]\n** ^SQLite guarantees that the zFilename parameter to xOpen\n** is either a NULL pointer or string obtained\n** from xFullPathname() with an optional suffix added.\n** ^If a suffix is added to the zFilename parameter, it will\n** consist of a single \"-\" character followed by no more than\n** 11 alphanumeric and/or \"-\" characters.\n** ^SQLite further guarantees that\n** the string will be valid and unchanged until xClose() is\n** called. Because of the previous sentence,\n** the [sqlite3_file] can safely store a pointer to the\n** filename if it needs to remember the filename for some reason.\n** If the zFilename parameter to xOpen is a NULL pointer then xOpen\n** must invent its own temporary name for the file. ^Whenever the\n** xFilename parameter is NULL it will also be the case that the\n** flags parameter will include [SQLITE_OPEN_DELETEONCLOSE].\n**\n** The flags argument to xOpen() includes all bits set in\n** the flags argument to [sqlite3_open_v2()]. Or if [sqlite3_open()]\n** or [sqlite3_open16()] is used, then flags includes at least\n** [SQLITE_OPEN_READWRITE] | [SQLITE_OPEN_CREATE].\n** If xOpen() opens a file read-only then it sets *pOutFlags to\n** include [SQLITE_OPEN_READONLY]. Other bits in *pOutFlags may be set.\n**\n** ^(SQLite will also add one of the following flags to the xOpen()\n** call, depending on the object being opened:\n**\n**
    \n**
  • [SQLITE_OPEN_MAIN_DB]\n**
  • [SQLITE_OPEN_MAIN_JOURNAL]\n**
  • [SQLITE_OPEN_TEMP_DB]\n**
  • [SQLITE_OPEN_TEMP_JOURNAL]\n**
  • [SQLITE_OPEN_TRANSIENT_DB]\n**
  • [SQLITE_OPEN_SUBJOURNAL]\n**
  • [SQLITE_OPEN_SUPER_JOURNAL]\n**
  • [SQLITE_OPEN_WAL]\n**
)^\n**\n** The file I/O implementation can use the object type flags to\n** change the way it deals with files. For example, an application\n** that does not care about crash recovery or rollback might make\n** the open of a journal file a no-op. Writes to this journal would\n** also be no-ops, and any attempt to read the journal would return\n** SQLITE_IOERR. Or the implementation might recognize that a database\n** file will be doing page-aligned sector reads and writes in a random\n** order and set up its I/O subsystem accordingly.\n**\n** SQLite might also add one of the following flags to the xOpen method:\n**\n**
    \n**
  • [SQLITE_OPEN_DELETEONCLOSE]\n**
  • [SQLITE_OPEN_EXCLUSIVE]\n**
\n**\n** The [SQLITE_OPEN_DELETEONCLOSE] flag means the file should be\n** deleted when it is closed. ^The [SQLITE_OPEN_DELETEONCLOSE]\n** will be set for TEMP databases and their journals, transient\n** databases, and subjournals.\n**\n** ^The [SQLITE_OPEN_EXCLUSIVE] flag is always used in conjunction\n** with the [SQLITE_OPEN_CREATE] flag, which are both directly\n** analogous to the O_EXCL and O_CREAT flags of the POSIX open()\n** API. The SQLITE_OPEN_EXCLUSIVE flag, when paired with the\n** SQLITE_OPEN_CREATE, is used to indicate that file should always\n** be created, and that it is an error if it already exists.\n** It is not used to indicate the file should be opened\n** for exclusive access.\n**\n** ^At least szOsFile bytes of memory are allocated by SQLite\n** to hold the [sqlite3_file] structure passed as the third\n** argument to xOpen. The xOpen method does not have to\n** allocate the structure; it should just fill it in. Note that\n** the xOpen method must set the sqlite3_file.pMethods to either\n** a valid [sqlite3_io_methods] object or to NULL. xOpen must do\n** this even if the open fails. SQLite expects that the sqlite3_file.pMethods\n** element will be valid after xOpen returns regardless of the success\n** or failure of the xOpen call.\n**\n** [[sqlite3_vfs.xAccess]]\n** ^The flags argument to xAccess() may be [SQLITE_ACCESS_EXISTS]\n** to test for the existence of a file, or [SQLITE_ACCESS_READWRITE] to\n** test whether a file is readable and writable, or [SQLITE_ACCESS_READ]\n** to test whether a file is at least readable. The SQLITE_ACCESS_READ\n** flag is never actually used and is not implemented in the built-in\n** VFSes of SQLite. The file is named by the second argument and can be a\n** directory. The xAccess method returns [SQLITE_OK] on success or some\n** non-zero error code if there is an I/O error or if the name of\n** the file given in the second argument is illegal. If SQLITE_OK\n** is returned, then non-zero or zero is written into *pResOut to indicate\n** whether or not the file is accessible.\n**\n** ^SQLite will always allocate at least mxPathname+1 bytes for the\n** output buffer xFullPathname. The exact size of the output buffer\n** is also passed as a parameter to both methods. If the output buffer\n** is not large enough, [SQLITE_CANTOPEN] should be returned. Since this is\n** handled as a fatal error by SQLite, vfs implementations should endeavor\n** to prevent this by setting mxPathname to a sufficiently large value.\n**\n** The xRandomness(), xSleep(), xCurrentTime(), and xCurrentTimeInt64()\n** interfaces are not strictly a part of the filesystem, but they are\n** included in the VFS structure for completeness.\n** The xRandomness() function attempts to return nBytes bytes\n** of good-quality randomness into zOut. The return value is\n** the actual number of bytes of randomness obtained.\n** The xSleep() method causes the calling thread to sleep for at\n** least the number of microseconds given. ^The xCurrentTime()\n** method returns a Julian Day Number for the current date and time as\n** a floating point value.\n** ^The xCurrentTimeInt64() method returns, as an integer, the Julian\n** Day Number multiplied by 86400000 (the number of milliseconds in\n** a 24-hour day).\n** ^SQLite will use the xCurrentTimeInt64() method to get the current\n** date and time if that method is available (if iVersion is 2 or\n** greater and the function pointer is not NULL) and will fall back\n** to xCurrentTime() if xCurrentTimeInt64() is unavailable.\n**\n** ^The xSetSystemCall(), xGetSystemCall(), and xNestSystemCall() interfaces\n** are not used by the SQLite core. These optional interfaces are provided\n** by some VFSes to facilitate testing of the VFS code. By overriding\n** system calls with functions under its control, a test program can\n** simulate faults and error conditions that would otherwise be difficult\n** or impossible to induce. The set of system calls that can be overridden\n** varies from one VFS to another, and from one version of the same VFS to the\n** next. Applications that use these interfaces must be prepared for any\n** or all of these interfaces to be NULL or for their behavior to change\n** from one release to the next. Applications must not attempt to access\n** any of these methods if the iVersion of the VFS is less than 3.\n*/\ntypedef struct sqlite3_vfs sqlite3_vfs;\ntypedef void (*sqlite3_syscall_ptr)(void);\nstruct sqlite3_vfs {\n int iVersion; /* Structure version number (currently 3) */\n int szOsFile; /* Size of subclassed sqlite3_file */\n int mxPathname; /* Maximum file pathname length */\n sqlite3_vfs *pNext; /* Next registered VFS */\n const char *zName; /* Name of this virtual file system */\n void *pAppData; /* Pointer to application-specific data */\n int (*xOpen)(sqlite3_vfs*, sqlite3_filename zName, sqlite3_file*,\n int flags, int *pOutFlags);\n int (*xDelete)(sqlite3_vfs*, const char *zName, int syncDir);\n int (*xAccess)(sqlite3_vfs*, const char *zName, int flags, int *pResOut);\n int (*xFullPathname)(sqlite3_vfs*, const char *zName, int nOut, char *zOut);\n void *(*xDlOpen)(sqlite3_vfs*, const char *zFilename);\n void (*xDlError)(sqlite3_vfs*, int nByte, char *zErrMsg);\n void (*(*xDlSym)(sqlite3_vfs*,void*, const char *zSymbol))(void);\n void (*xDlClose)(sqlite3_vfs*, void*);\n int (*xRandomness)(sqlite3_vfs*, int nByte, char *zOut);\n int (*xSleep)(sqlite3_vfs*, int microseconds);\n int (*xCurrentTime)(sqlite3_vfs*, double*);\n int (*xGetLastError)(sqlite3_vfs*, int, char *);\n /*\n ** The methods above are in version 1 of the sqlite_vfs object\n ** definition. Those that follow are added in version 2 or later\n */\n int (*xCurrentTimeInt64)(sqlite3_vfs*, sqlite3_int64*);\n /*\n ** The methods above are in versions 1 and 2 of the sqlite_vfs object.\n ** Those below are for version 3 and greater.\n */\n int (*xSetSystemCall)(sqlite3_vfs*, const char *zName, sqlite3_syscall_ptr);\n sqlite3_syscall_ptr (*xGetSystemCall)(sqlite3_vfs*, const char *zName);\n const char *(*xNextSystemCall)(sqlite3_vfs*, const char *zName);\n /*\n ** The methods above are in versions 1 through 3 of the sqlite_vfs object.\n ** New fields may be appended in future versions. The iVersion\n ** value will increment whenever this happens.\n */\n};\n\n/*\n** CAPI3REF: Flags for the xAccess VFS method\n**\n** These integer constants can be used as the third parameter to\n** the xAccess method of an [sqlite3_vfs] object. They determine\n** what kind of permissions the xAccess method is looking for.\n** With SQLITE_ACCESS_EXISTS, the xAccess method\n** simply checks whether the file exists.\n** With SQLITE_ACCESS_READWRITE, the xAccess method\n** checks whether the named directory is both readable and writable\n** (in other words, if files can be added, removed, and renamed within\n** the directory).\n** The SQLITE_ACCESS_READWRITE constant is currently used only by the\n** [temp_store_directory pragma], though this could change in a future\n** release of SQLite.\n** With SQLITE_ACCESS_READ, the xAccess method\n** checks whether the file is readable. The SQLITE_ACCESS_READ constant is\n** currently unused, though it might be used in a future release of\n** SQLite.\n*/\n#define SQLITE_ACCESS_EXISTS 0\n#define SQLITE_ACCESS_READWRITE 1 /* Used by PRAGMA temp_store_directory */\n#define SQLITE_ACCESS_READ 2 /* Unused */\n\n/*\n** CAPI3REF: Flags for the xShmLock VFS method\n**\n** These integer constants define the various locking operations\n** allowed by the xShmLock method of [sqlite3_io_methods]. The\n** following are the only legal combinations of flags to the\n** xShmLock method:\n**\n**
    \n**
  • SQLITE_SHM_LOCK | SQLITE_SHM_SHARED\n**
  • SQLITE_SHM_LOCK | SQLITE_SHM_EXCLUSIVE\n**
  • SQLITE_SHM_UNLOCK | SQLITE_SHM_SHARED\n**
  • SQLITE_SHM_UNLOCK | SQLITE_SHM_EXCLUSIVE\n**
\n**\n** When unlocking, the same SHARED or EXCLUSIVE flag must be supplied as\n** was given on the corresponding lock.\n**\n** The xShmLock method can transition between unlocked and SHARED or\n** between unlocked and EXCLUSIVE. It cannot transition between SHARED\n** and EXCLUSIVE.\n*/\n#define SQLITE_SHM_UNLOCK 1\n#define SQLITE_SHM_LOCK 2\n#define SQLITE_SHM_SHARED 4\n#define SQLITE_SHM_EXCLUSIVE 8\n\n/*\n** CAPI3REF: Maximum xShmLock index\n**\n** The xShmLock method on [sqlite3_io_methods] may use values\n** between 0 and this upper bound as its \"offset\" argument.\n** The SQLite core will never attempt to acquire or release a\n** lock outside of this range\n*/\n#define SQLITE_SHM_NLOCK 8\n\n\n/*\n** CAPI3REF: Initialize The SQLite Library\n**\n** ^The sqlite3_initialize() routine initializes the\n** SQLite library. ^The sqlite3_shutdown() routine\n** deallocates any resources that were allocated by sqlite3_initialize().\n** These routines are designed to aid in process initialization and\n** shutdown on embedded systems. Workstation applications using\n** SQLite normally do not need to invoke either of these routines.\n**\n** A call to sqlite3_initialize() is an \"effective\" call if it is\n** the first time sqlite3_initialize() is invoked during the lifetime of\n** the process, or if it is the first time sqlite3_initialize() is invoked\n** following a call to sqlite3_shutdown(). ^(Only an effective call\n** of sqlite3_initialize() does any initialization. All other calls\n** are harmless no-ops.)^\n**\n** A call to sqlite3_shutdown() is an \"effective\" call if it is the first\n** call to sqlite3_shutdown() since the last sqlite3_initialize(). ^(Only\n** an effective call to sqlite3_shutdown() does any deinitialization.\n** All other valid calls to sqlite3_shutdown() are harmless no-ops.)^\n**\n** The sqlite3_initialize() interface is threadsafe, but sqlite3_shutdown()\n** is not. The sqlite3_shutdown() interface must only be called from a\n** single thread. All open [database connections] must be closed and all\n** other SQLite resources must be deallocated prior to invoking\n** sqlite3_shutdown().\n**\n** Among other things, ^sqlite3_initialize() will invoke\n** sqlite3_os_init(). Similarly, ^sqlite3_shutdown()\n** will invoke sqlite3_os_end().\n**\n** ^The sqlite3_initialize() routine returns [SQLITE_OK] on success.\n** ^If for some reason, sqlite3_initialize() is unable to initialize\n** the library (perhaps it is unable to allocate a needed resource such\n** as a mutex) it returns an [error code] other than [SQLITE_OK].\n**\n** ^The sqlite3_initialize() routine is called internally by many other\n** SQLite interfaces so that an application usually does not need to\n** invoke sqlite3_initialize() directly. For example, [sqlite3_open()]\n** calls sqlite3_initialize() so the SQLite library will be automatically\n** initialized when [sqlite3_open()] is called if it has not be initialized\n** already. ^However, if SQLite is compiled with the [SQLITE_OMIT_AUTOINIT]\n** compile-time option, then the automatic calls to sqlite3_initialize()\n** are omitted and the application must call sqlite3_initialize() directly\n** prior to using any other SQLite interface. For maximum portability,\n** it is recommended that applications always invoke sqlite3_initialize()\n** directly prior to using any other SQLite interface. Future releases\n** of SQLite may require this. In other words, the behavior exhibited\n** when SQLite is compiled with [SQLITE_OMIT_AUTOINIT] might become the\n** default behavior in some future release of SQLite.\n**\n** The sqlite3_os_init() routine does operating-system specific\n** initialization of the SQLite library. The sqlite3_os_end()\n** routine undoes the effect of sqlite3_os_init(). Typical tasks\n** performed by these routines include allocation or deallocation\n** of static resources, initialization of global variables,\n** setting up a default [sqlite3_vfs] module, or setting up\n** a default configuration using [sqlite3_config()].\n**\n** The application should never invoke either sqlite3_os_init()\n** or sqlite3_os_end() directly. The application should only invoke\n** sqlite3_initialize() and sqlite3_shutdown(). The sqlite3_os_init()\n** interface is called automatically by sqlite3_initialize() and\n** sqlite3_os_end() is called by sqlite3_shutdown(). Appropriate\n** implementations for sqlite3_os_init() and sqlite3_os_end()\n** are built into SQLite when it is compiled for Unix, Windows, or OS/2.\n** When [custom builds | built for other platforms]\n** (using the [SQLITE_OS_OTHER=1] compile-time\n** option) the application must supply a suitable implementation for\n** sqlite3_os_init() and sqlite3_os_end(). An application-supplied\n** implementation of sqlite3_os_init() or sqlite3_os_end()\n** must return [SQLITE_OK] on success and some other [error code] upon\n** failure.\n*/\nSQLITE_API int sqlite3_initialize(void);\nSQLITE_API int sqlite3_shutdown(void);\nSQLITE_API int sqlite3_os_init(void);\nSQLITE_API int sqlite3_os_end(void);\n\n/*\n** CAPI3REF: Configuring The SQLite Library\n**\n** The sqlite3_config() interface is used to make global configuration\n** changes to SQLite in order to tune SQLite to the specific needs of\n** the application. The default configuration is recommended for most\n** applications and so this routine is usually not necessary. It is\n** provided to support rare applications with unusual needs.\n**\n** The sqlite3_config() interface is not threadsafe. The application\n** must ensure that no other SQLite interfaces are invoked by other\n** threads while sqlite3_config() is running.\n**\n** The first argument to sqlite3_config() is an integer\n** [configuration option] that determines\n** what property of SQLite is to be configured. Subsequent arguments\n** vary depending on the [configuration option]\n** in the first argument.\n**\n** For most configuration options, the sqlite3_config() interface\n** may only be invoked prior to library initialization using\n** [sqlite3_initialize()] or after shutdown by [sqlite3_shutdown()].\n** The exceptional configuration options that may be invoked at any time\n** are called \"anytime configuration options\".\n** ^If sqlite3_config() is called after [sqlite3_initialize()] and before\n** [sqlite3_shutdown()] with a first argument that is not an anytime\n** configuration option, then the sqlite3_config() call will return SQLITE_MISUSE.\n** Note, however, that ^sqlite3_config() can be called as part of the\n** implementation of an application-defined [sqlite3_os_init()].\n**\n** ^When a configuration option is set, sqlite3_config() returns [SQLITE_OK].\n** ^If the option is unknown or SQLite is unable to set the option\n** then this routine returns a non-zero [error code].\n*/\nSQLITE_API int sqlite3_config(int, ...);\n\n/*\n** CAPI3REF: Configure database connections\n** METHOD: sqlite3\n**\n** The sqlite3_db_config() interface is used to make configuration\n** changes to a [database connection]. The interface is similar to\n** [sqlite3_config()] except that the changes apply to a single\n** [database connection] (specified in the first argument).\n**\n** The second argument to sqlite3_db_config(D,V,...) is the\n** [SQLITE_DBCONFIG_LOOKASIDE | configuration verb] - an integer code\n** that indicates what aspect of the [database connection] is being configured.\n** Subsequent arguments vary depending on the configuration verb.\n**\n** ^Calls to sqlite3_db_config() return SQLITE_OK if and only if\n** the call is considered successful.\n*/\nSQLITE_API int sqlite3_db_config(sqlite3*, int op, ...);\n\n/*\n** CAPI3REF: Memory Allocation Routines\n**\n** An instance of this object defines the interface between SQLite\n** and low-level memory allocation routines.\n**\n** This object is used in only one place in the SQLite interface.\n** A pointer to an instance of this object is the argument to\n** [sqlite3_config()] when the configuration option is\n** [SQLITE_CONFIG_MALLOC] or [SQLITE_CONFIG_GETMALLOC].\n** By creating an instance of this object\n** and passing it to [sqlite3_config]([SQLITE_CONFIG_MALLOC])\n** during configuration, an application can specify an alternative\n** memory allocation subsystem for SQLite to use for all of its\n** dynamic memory needs.\n**\n** Note that SQLite comes with several [built-in memory allocators]\n** that are perfectly adequate for the overwhelming majority of applications\n** and that this object is only useful to a tiny minority of applications\n** with specialized memory allocation requirements. This object is\n** also used during testing of SQLite in order to specify an alternative\n** memory allocator that simulates memory out-of-memory conditions in\n** order to verify that SQLite recovers gracefully from such\n** conditions.\n**\n** The xMalloc, xRealloc, and xFree methods must work like the\n** malloc(), realloc() and free() functions from the standard C library.\n** ^SQLite guarantees that the second argument to\n** xRealloc is always a value returned by a prior call to xRoundup.\n**\n** xSize should return the allocated size of a memory allocation\n** previously obtained from xMalloc or xRealloc. The allocated size\n** is always at least as big as the requested size but may be larger.\n**\n** The xRoundup method returns what would be the allocated size of\n** a memory allocation given a particular requested size. Most memory\n** allocators round up memory allocations at least to the next multiple\n** of 8. Some allocators round up to a larger multiple or to a power of 2.\n** Every memory allocation request coming in through [sqlite3_malloc()]\n** or [sqlite3_realloc()] first calls xRoundup. If xRoundup returns 0,\n** that causes the corresponding memory allocation to fail.\n**\n** The xInit method initializes the memory allocator. For example,\n** it might allocate any required mutexes or initialize internal data\n** structures. The xShutdown method is invoked (indirectly) by\n** [sqlite3_shutdown()] and should deallocate any resources acquired\n** by xInit. The pAppData pointer is used as the only parameter to\n** xInit and xShutdown.\n**\n** SQLite holds the [SQLITE_MUTEX_STATIC_MAIN] mutex when it invokes\n** the xInit method, so the xInit method need not be threadsafe. The\n** xShutdown method is only called from [sqlite3_shutdown()] so it does\n** not need to be threadsafe either. For all other methods, SQLite\n** holds the [SQLITE_MUTEX_STATIC_MEM] mutex as long as the\n** [SQLITE_CONFIG_MEMSTATUS] configuration option is turned on (which\n** it is by default) and so the methods are automatically serialized.\n** However, if [SQLITE_CONFIG_MEMSTATUS] is disabled, then the other\n** methods must be threadsafe or else make their own arrangements for\n** serialization.\n**\n** SQLite will never invoke xInit() more than once without an intervening\n** call to xShutdown().\n*/\ntypedef struct sqlite3_mem_methods sqlite3_mem_methods;\nstruct sqlite3_mem_methods {\n void *(*xMalloc)(int); /* Memory allocation function */\n void (*xFree)(void*); /* Free a prior allocation */\n void *(*xRealloc)(void*,int); /* Resize an allocation */\n int (*xSize)(void*); /* Return the size of an allocation */\n int (*xRoundup)(int); /* Round up request size to allocation size */\n int (*xInit)(void*); /* Initialize the memory allocator */\n void (*xShutdown)(void*); /* Deinitialize the memory allocator */\n void *pAppData; /* Argument to xInit() and xShutdown() */\n};\n\n/*\n** CAPI3REF: Configuration Options\n** KEYWORDS: {configuration option}\n**\n** These constants are the available integer configuration options that\n** can be passed as the first argument to the [sqlite3_config()] interface.\n**\n** Most of the configuration options for sqlite3_config()\n** will only work if invoked prior to [sqlite3_initialize()] or after\n** [sqlite3_shutdown()]. The few exceptions to this rule are called\n** \"anytime configuration options\".\n** ^Calling [sqlite3_config()] with a first argument that is not an\n** anytime configuration option in between calls to [sqlite3_initialize()] and\n** [sqlite3_shutdown()] is a no-op that returns SQLITE_MISUSE.\n**\n** The set of anytime configuration options can change (by insertions\n** and/or deletions) from one release of SQLite to the next.\n** As of SQLite version 3.42.0, the complete set of anytime configuration\n** options is:\n**
    \n**
  • SQLITE_CONFIG_LOG\n**
  • SQLITE_CONFIG_PCACHE_HDRSZ\n**
\n**\n** New configuration options may be added in future releases of SQLite.\n** Existing configuration options might be discontinued. Applications\n** should check the return code from [sqlite3_config()] to make sure that\n** the call worked. The [sqlite3_config()] interface will return a\n** non-zero [error code] if a discontinued or unsupported configuration option\n** is invoked.\n**\n**
\n** [[SQLITE_CONFIG_SINGLETHREAD]]
SQLITE_CONFIG_SINGLETHREAD
\n**
There are no arguments to this option. ^This option sets the\n** [threading mode] to Single-thread. In other words, it disables\n** all mutexing and puts SQLite into a mode where it can only be used\n** by a single thread. ^If SQLite is compiled with\n** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then\n** it is not possible to change the [threading mode] from its default\n** value of Single-thread and so [sqlite3_config()] will return\n** [SQLITE_ERROR] if called with the SQLITE_CONFIG_SINGLETHREAD\n** configuration option.
\n**\n** [[SQLITE_CONFIG_MULTITHREAD]]
SQLITE_CONFIG_MULTITHREAD
\n**
There are no arguments to this option. ^This option sets the\n** [threading mode] to Multi-thread. In other words, it disables\n** mutexing on [database connection] and [prepared statement] objects.\n** The application is responsible for serializing access to\n** [database connections] and [prepared statements]. But other mutexes\n** are enabled so that SQLite will be safe to use in a multi-threaded\n** environment as long as no two threads attempt to use the same\n** [database connection] at the same time. ^If SQLite is compiled with\n** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then\n** it is not possible to set the Multi-thread [threading mode] and\n** [sqlite3_config()] will return [SQLITE_ERROR] if called with the\n** SQLITE_CONFIG_MULTITHREAD configuration option.
\n**\n** [[SQLITE_CONFIG_SERIALIZED]]
SQLITE_CONFIG_SERIALIZED
\n**
There are no arguments to this option. ^This option sets the\n** [threading mode] to Serialized. In other words, this option enables\n** all mutexes including the recursive\n** mutexes on [database connection] and [prepared statement] objects.\n** In this mode (which is the default when SQLite is compiled with\n** [SQLITE_THREADSAFE=1]) the SQLite library will itself serialize access\n** to [database connections] and [prepared statements] so that the\n** application is free to use the same [database connection] or the\n** same [prepared statement] in different threads at the same time.\n** ^If SQLite is compiled with\n** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then\n** it is not possible to set the Serialized [threading mode] and\n** [sqlite3_config()] will return [SQLITE_ERROR] if called with the\n** SQLITE_CONFIG_SERIALIZED configuration option.
\n**\n** [[SQLITE_CONFIG_MALLOC]]
SQLITE_CONFIG_MALLOC
\n**
^(The SQLITE_CONFIG_MALLOC option takes a single argument which is\n** a pointer to an instance of the [sqlite3_mem_methods] structure.\n** The argument specifies\n** alternative low-level memory allocation routines to be used in place of\n** the memory allocation routines built into SQLite.)^ ^SQLite makes\n** its own private copy of the content of the [sqlite3_mem_methods] structure\n** before the [sqlite3_config()] call returns.
\n**\n** [[SQLITE_CONFIG_GETMALLOC]]
SQLITE_CONFIG_GETMALLOC
\n**
^(The SQLITE_CONFIG_GETMALLOC option takes a single argument which\n** is a pointer to an instance of the [sqlite3_mem_methods] structure.\n** The [sqlite3_mem_methods]\n** structure is filled with the currently defined memory allocation routines.)^\n** This option can be used to overload the default memory allocation\n** routines with a wrapper that simulations memory allocation failure or\n** tracks memory usage, for example.
\n**\n** [[SQLITE_CONFIG_SMALL_MALLOC]]
SQLITE_CONFIG_SMALL_MALLOC
\n**
^The SQLITE_CONFIG_SMALL_MALLOC option takes single argument of\n** type int, interpreted as a boolean, which if true provides a hint to\n** SQLite that it should avoid large memory allocations if possible.\n** SQLite will run faster if it is free to make large memory allocations,\n** but some application might prefer to run slower in exchange for\n** guarantees about memory fragmentation that are possible if large\n** allocations are avoided. This hint is normally off.\n**
\n**\n** [[SQLITE_CONFIG_MEMSTATUS]]
SQLITE_CONFIG_MEMSTATUS
\n**
^The SQLITE_CONFIG_MEMSTATUS option takes single argument of type int,\n** interpreted as a boolean, which enables or disables the collection of\n** memory allocation statistics. ^(When memory allocation statistics are\n** disabled, the following SQLite interfaces become non-operational:\n**
    \n**
  • [sqlite3_hard_heap_limit64()]\n**
  • [sqlite3_memory_used()]\n**
  • [sqlite3_memory_highwater()]\n**
  • [sqlite3_soft_heap_limit64()]\n**
  • [sqlite3_status64()]\n**
)^\n** ^Memory allocation statistics are enabled by default unless SQLite is\n** compiled with [SQLITE_DEFAULT_MEMSTATUS]=0 in which case memory\n** allocation statistics are disabled by default.\n**
\n**\n** [[SQLITE_CONFIG_SCRATCH]]
SQLITE_CONFIG_SCRATCH
\n**
The SQLITE_CONFIG_SCRATCH option is no longer used.\n**
\n**\n** [[SQLITE_CONFIG_PAGECACHE]]
SQLITE_CONFIG_PAGECACHE
\n**
^The SQLITE_CONFIG_PAGECACHE option specifies a memory pool\n** that SQLite can use for the database page cache with the default page\n** cache implementation.\n** This configuration option is a no-op if an application-defined page\n** cache implementation is loaded using the [SQLITE_CONFIG_PCACHE2].\n** ^There are three arguments to SQLITE_CONFIG_PAGECACHE: A pointer to\n** 8-byte aligned memory (pMem), the size of each page cache line (sz),\n** and the number of cache lines (N).\n** The sz argument should be the size of the largest database page\n** (a power of two between 512 and 65536) plus some extra bytes for each\n** page header. ^The number of extra bytes needed by the page header\n** can be determined using [SQLITE_CONFIG_PCACHE_HDRSZ].\n** ^It is harmless, apart from the wasted memory,\n** for the sz parameter to be larger than necessary. The pMem\n** argument must be either a NULL pointer or a pointer to an 8-byte\n** aligned block of memory of at least sz*N bytes, otherwise\n** subsequent behavior is undefined.\n** ^When pMem is not NULL, SQLite will strive to use the memory provided\n** to satisfy page cache needs, falling back to [sqlite3_malloc()] if\n** a page cache line is larger than sz bytes or if all of the pMem buffer\n** is exhausted.\n** ^If pMem is NULL and N is non-zero, then each database connection\n** does an initial bulk allocation for page cache memory\n** from [sqlite3_malloc()] sufficient for N cache lines if N is positive or\n** of -1024*N bytes if N is negative, . ^If additional\n** page cache memory is needed beyond what is provided by the initial\n** allocation, then SQLite goes to [sqlite3_malloc()] separately for each\n** additional cache line.
\n**\n** [[SQLITE_CONFIG_HEAP]]
SQLITE_CONFIG_HEAP
\n**
^The SQLITE_CONFIG_HEAP option specifies a static memory buffer\n** that SQLite will use for all of its dynamic memory allocation needs\n** beyond those provided for by [SQLITE_CONFIG_PAGECACHE].\n** ^The SQLITE_CONFIG_HEAP option is only available if SQLite is compiled\n** with either [SQLITE_ENABLE_MEMSYS3] or [SQLITE_ENABLE_MEMSYS5] and returns\n** [SQLITE_ERROR] if invoked otherwise.\n** ^There are three arguments to SQLITE_CONFIG_HEAP:\n** An 8-byte aligned pointer to the memory,\n** the number of bytes in the memory buffer, and the minimum allocation size.\n** ^If the first pointer (the memory pointer) is NULL, then SQLite reverts\n** to using its default memory allocator (the system malloc() implementation),\n** undoing any prior invocation of [SQLITE_CONFIG_MALLOC]. ^If the\n** memory pointer is not NULL then the alternative memory\n** allocator is engaged to handle all of SQLites memory allocation needs.\n** The first pointer (the memory pointer) must be aligned to an 8-byte\n** boundary or subsequent behavior of SQLite will be undefined.\n** The minimum allocation size is capped at 2**12. Reasonable values\n** for the minimum allocation size are 2**5 through 2**8.
\n**\n** [[SQLITE_CONFIG_MUTEX]]
SQLITE_CONFIG_MUTEX
\n**
^(The SQLITE_CONFIG_MUTEX option takes a single argument which is a\n** pointer to an instance of the [sqlite3_mutex_methods] structure.\n** The argument specifies alternative low-level mutex routines to be used\n** in place the mutex routines built into SQLite.)^ ^SQLite makes a copy of\n** the content of the [sqlite3_mutex_methods] structure before the call to\n** [sqlite3_config()] returns. ^If SQLite is compiled with\n** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then\n** the entire mutexing subsystem is omitted from the build and hence calls to\n** [sqlite3_config()] with the SQLITE_CONFIG_MUTEX configuration option will\n** return [SQLITE_ERROR].
\n**\n** [[SQLITE_CONFIG_GETMUTEX]]
SQLITE_CONFIG_GETMUTEX
\n**
^(The SQLITE_CONFIG_GETMUTEX option takes a single argument which\n** is a pointer to an instance of the [sqlite3_mutex_methods] structure. The\n** [sqlite3_mutex_methods]\n** structure is filled with the currently defined mutex routines.)^\n** This option can be used to overload the default mutex allocation\n** routines with a wrapper used to track mutex usage for performance\n** profiling or testing, for example. ^If SQLite is compiled with\n** the [SQLITE_THREADSAFE | SQLITE_THREADSAFE=0] compile-time option then\n** the entire mutexing subsystem is omitted from the build and hence calls to\n** [sqlite3_config()] with the SQLITE_CONFIG_GETMUTEX configuration option will\n** return [SQLITE_ERROR].
\n**\n** [[SQLITE_CONFIG_LOOKASIDE]]
SQLITE_CONFIG_LOOKASIDE
\n**
^(The SQLITE_CONFIG_LOOKASIDE option takes two arguments that determine\n** the default size of [lookaside memory] on each [database connection].\n** The first argument is the\n** size of each lookaside buffer slot (\"sz\") and the second is the number of\n** slots allocated to each database connection (\"cnt\").)^\n** ^(SQLITE_CONFIG_LOOKASIDE sets the default lookaside size.\n** The [SQLITE_DBCONFIG_LOOKASIDE] option to [sqlite3_db_config()] can\n** be used to change the lookaside configuration on individual connections.)^\n** The [-DSQLITE_DEFAULT_LOOKASIDE] option can be used to change the\n** default lookaside configuration at compile-time.\n**
\n**\n** [[SQLITE_CONFIG_PCACHE2]]
SQLITE_CONFIG_PCACHE2
\n**
^(The SQLITE_CONFIG_PCACHE2 option takes a single argument which is\n** a pointer to an [sqlite3_pcache_methods2] object. This object specifies\n** the interface to a custom page cache implementation.)^\n** ^SQLite makes a copy of the [sqlite3_pcache_methods2] object.
\n**\n** [[SQLITE_CONFIG_GETPCACHE2]]
SQLITE_CONFIG_GETPCACHE2
\n**
^(The SQLITE_CONFIG_GETPCACHE2 option takes a single argument which\n** is a pointer to an [sqlite3_pcache_methods2] object. SQLite copies of\n** the current page cache implementation into that object.)^
\n**\n** [[SQLITE_CONFIG_LOG]]
SQLITE_CONFIG_LOG
\n**
The SQLITE_CONFIG_LOG option is used to configure the SQLite\n** global [error log].\n** (^The SQLITE_CONFIG_LOG option takes two arguments: a pointer to a\n** function with a call signature of void(*)(void*,int,const char*),\n** and a pointer to void. ^If the function pointer is not NULL, it is\n** invoked by [sqlite3_log()] to process each logging event. ^If the\n** function pointer is NULL, the [sqlite3_log()] interface becomes a no-op.\n** ^The void pointer that is the second argument to SQLITE_CONFIG_LOG is\n** passed through as the first parameter to the application-defined logger\n** function whenever that function is invoked. ^The second parameter to\n** the logger function is a copy of the first parameter to the corresponding\n** [sqlite3_log()] call and is intended to be a [result code] or an\n** [extended result code]. ^The third parameter passed to the logger is\n** log message after formatting via [sqlite3_snprintf()].\n** The SQLite logging interface is not reentrant; the logger function\n** supplied by the application must not invoke any SQLite interface.\n** In a multi-threaded application, the application-defined logger\n** function must be threadsafe.
\n**\n** [[SQLITE_CONFIG_URI]]
SQLITE_CONFIG_URI\n**
^(The SQLITE_CONFIG_URI option takes a single argument of type int.\n** If non-zero, then URI handling is globally enabled. If the parameter is zero,\n** then URI handling is globally disabled.)^ ^If URI handling is globally\n** enabled, all filenames passed to [sqlite3_open()], [sqlite3_open_v2()],\n** [sqlite3_open16()] or\n** specified as part of [ATTACH] commands are interpreted as URIs, regardless\n** of whether or not the [SQLITE_OPEN_URI] flag is set when the database\n** connection is opened. ^If it is globally disabled, filenames are\n** only interpreted as URIs if the SQLITE_OPEN_URI flag is set when the\n** database connection is opened. ^(By default, URI handling is globally\n** disabled. The default value may be changed by compiling with the\n** [SQLITE_USE_URI] symbol defined.)^\n**\n** [[SQLITE_CONFIG_COVERING_INDEX_SCAN]]
SQLITE_CONFIG_COVERING_INDEX_SCAN\n**
^The SQLITE_CONFIG_COVERING_INDEX_SCAN option takes a single integer\n** argument which is interpreted as a boolean in order to enable or disable\n** the use of covering indices for full table scans in the query optimizer.\n** ^The default setting is determined\n** by the [SQLITE_ALLOW_COVERING_INDEX_SCAN] compile-time option, or is \"on\"\n** if that compile-time option is omitted.\n** The ability to disable the use of covering indices for full table scans\n** is because some incorrectly coded legacy applications might malfunction\n** when the optimization is enabled. Providing the ability to\n** disable the optimization allows the older, buggy application code to work\n** without change even with newer versions of SQLite.\n**\n** [[SQLITE_CONFIG_PCACHE]] [[SQLITE_CONFIG_GETPCACHE]]\n**
SQLITE_CONFIG_PCACHE and SQLITE_CONFIG_GETPCACHE\n**
These options are obsolete and should not be used by new code.\n** They are retained for backwards compatibility but are now no-ops.\n**
\n**\n** [[SQLITE_CONFIG_SQLLOG]]\n**
SQLITE_CONFIG_SQLLOG\n**
This option is only available if sqlite is compiled with the\n** [SQLITE_ENABLE_SQLLOG] pre-processor macro defined. The first argument should\n** be a pointer to a function of type void(*)(void*,sqlite3*,const char*, int).\n** The second should be of type (void*). The callback is invoked by the library\n** in three separate circumstances, identified by the value passed as the\n** fourth parameter. If the fourth parameter is 0, then the database connection\n** passed as the second argument has just been opened. The third argument\n** points to a buffer containing the name of the main database file. If the\n** fourth parameter is 1, then the SQL statement that the third parameter\n** points to has just been executed. Or, if the fourth parameter is 2, then\n** the connection being passed as the second parameter is being closed. The\n** third parameter is passed NULL In this case. An example of using this\n** configuration option can be seen in the \"test_sqllog.c\" source file in\n** the canonical SQLite source tree.
\n**\n** [[SQLITE_CONFIG_MMAP_SIZE]]\n**
SQLITE_CONFIG_MMAP_SIZE\n**
^SQLITE_CONFIG_MMAP_SIZE takes two 64-bit integer (sqlite3_int64) values\n** that are the default mmap size limit (the default setting for\n** [PRAGMA mmap_size]) and the maximum allowed mmap size limit.\n** ^The default setting can be overridden by each database connection using\n** either the [PRAGMA mmap_size] command, or by using the\n** [SQLITE_FCNTL_MMAP_SIZE] file control. ^(The maximum allowed mmap size\n** will be silently truncated if necessary so that it does not exceed the\n** compile-time maximum mmap size set by the\n** [SQLITE_MAX_MMAP_SIZE] compile-time option.)^\n** ^If either argument to this option is negative, then that argument is\n** changed to its compile-time default.\n**\n** [[SQLITE_CONFIG_WIN32_HEAPSIZE]]\n**
SQLITE_CONFIG_WIN32_HEAPSIZE\n**
^The SQLITE_CONFIG_WIN32_HEAPSIZE option is only available if SQLite is\n** compiled for Windows with the [SQLITE_WIN32_MALLOC] pre-processor macro\n** defined. ^SQLITE_CONFIG_WIN32_HEAPSIZE takes a 32-bit unsigned integer value\n** that specifies the maximum size of the created heap.\n**\n** [[SQLITE_CONFIG_PCACHE_HDRSZ]]\n**
SQLITE_CONFIG_PCACHE_HDRSZ\n**
^The SQLITE_CONFIG_PCACHE_HDRSZ option takes a single parameter which\n** is a pointer to an integer and writes into that integer the number of extra\n** bytes per page required for each page in [SQLITE_CONFIG_PAGECACHE].\n** The amount of extra space required can change depending on the compiler,\n** target platform, and SQLite version.\n**\n** [[SQLITE_CONFIG_PMASZ]]\n**
SQLITE_CONFIG_PMASZ\n**
^The SQLITE_CONFIG_PMASZ option takes a single parameter which\n** is an unsigned integer and sets the \"Minimum PMA Size\" for the multithreaded\n** sorter to that integer. The default minimum PMA Size is set by the\n** [SQLITE_SORTER_PMASZ] compile-time option. New threads are launched\n** to help with sort operations when multithreaded sorting\n** is enabled (using the [PRAGMA threads] command) and the amount of content\n** to be sorted exceeds the page size times the minimum of the\n** [PRAGMA cache_size] setting and this value.\n**\n** [[SQLITE_CONFIG_STMTJRNL_SPILL]]\n**
SQLITE_CONFIG_STMTJRNL_SPILL\n**
^The SQLITE_CONFIG_STMTJRNL_SPILL option takes a single parameter which\n** becomes the [statement journal] spill-to-disk threshold.\n** [Statement journals] are held in memory until their size (in bytes)\n** exceeds this threshold, at which point they are written to disk.\n** Or if the threshold is -1, statement journals are always held\n** exclusively in memory.\n** Since many statement journals never become large, setting the spill\n** threshold to a value such as 64KiB can greatly reduce the amount of\n** I/O required to support statement rollback.\n** The default value for this setting is controlled by the\n** [SQLITE_STMTJRNL_SPILL] compile-time option.\n**\n** [[SQLITE_CONFIG_SORTERREF_SIZE]]\n**
SQLITE_CONFIG_SORTERREF_SIZE\n**
The SQLITE_CONFIG_SORTERREF_SIZE option accepts a single parameter\n** of type (int) - the new value of the sorter-reference size threshold.\n** Usually, when SQLite uses an external sort to order records according\n** to an ORDER BY clause, all fields required by the caller are present in the\n** sorted records. However, if SQLite determines based on the declared type\n** of a table column that its values are likely to be very large - larger\n** than the configured sorter-reference size threshold - then a reference\n** is stored in each sorted record and the required column values loaded\n** from the database as records are returned in sorted order. The default\n** value for this option is to never use this optimization. Specifying a\n** negative value for this option restores the default behavior.\n** This option is only available if SQLite is compiled with the\n** [SQLITE_ENABLE_SORTER_REFERENCES] compile-time option.\n**\n** [[SQLITE_CONFIG_MEMDB_MAXSIZE]]\n**
SQLITE_CONFIG_MEMDB_MAXSIZE\n**
The SQLITE_CONFIG_MEMDB_MAXSIZE option accepts a single parameter\n** [sqlite3_int64] parameter which is the default maximum size for an in-memory\n** database created using [sqlite3_deserialize()]. This default maximum\n** size can be adjusted up or down for individual databases using the\n** [SQLITE_FCNTL_SIZE_LIMIT] [sqlite3_file_control|file-control]. If this\n** configuration setting is never used, then the default maximum is determined\n** by the [SQLITE_MEMDB_DEFAULT_MAXSIZE] compile-time option. If that\n** compile-time option is not set, then the default maximum is 1073741824.\n**\n** [[SQLITE_CONFIG_ROWID_IN_VIEW]]\n**
SQLITE_CONFIG_ROWID_IN_VIEW\n**
The SQLITE_CONFIG_ROWID_IN_VIEW option enables or disables the ability\n** for VIEWs to have a ROWID. The capability can only be enabled if SQLite is\n** compiled with -DSQLITE_ALLOW_ROWID_IN_VIEW, in which case the capability\n** defaults to on. This configuration option queries the current setting or\n** changes the setting to off or on. The argument is a pointer to an integer.\n** If that integer initially holds a value of 1, then the ability for VIEWs to\n** have ROWIDs is activated. If the integer initially holds zero, then the\n** ability is deactivated. Any other initial value for the integer leaves the\n** setting unchanged. After changes, if any, the integer is written with\n** a 1 or 0, if the ability for VIEWs to have ROWIDs is on or off. If SQLite\n** is compiled without -DSQLITE_ALLOW_ROWID_IN_VIEW (which is the usual and\n** recommended case) then the integer is always filled with zero, regardless\n** if its initial value.\n**
\n*/\n#define SQLITE_CONFIG_SINGLETHREAD 1 /* nil */\n#define SQLITE_CONFIG_MULTITHREAD 2 /* nil */\n#define SQLITE_CONFIG_SERIALIZED 3 /* nil */\n#define SQLITE_CONFIG_MALLOC 4 /* sqlite3_mem_methods* */\n#define SQLITE_CONFIG_GETMALLOC 5 /* sqlite3_mem_methods* */\n#define SQLITE_CONFIG_SCRATCH 6 /* No longer used */\n#define SQLITE_CONFIG_PAGECACHE 7 /* void*, int sz, int N */\n#define SQLITE_CONFIG_HEAP 8 /* void*, int nByte, int min */\n#define SQLITE_CONFIG_MEMSTATUS 9 /* boolean */\n#define SQLITE_CONFIG_MUTEX 10 /* sqlite3_mutex_methods* */\n#define SQLITE_CONFIG_GETMUTEX 11 /* sqlite3_mutex_methods* */\n/* previously SQLITE_CONFIG_CHUNKALLOC 12 which is now unused. */\n#define SQLITE_CONFIG_LOOKASIDE 13 /* int int */\n#define SQLITE_CONFIG_PCACHE 14 /* no-op */\n#define SQLITE_CONFIG_GETPCACHE 15 /* no-op */\n#define SQLITE_CONFIG_LOG 16 /* xFunc, void* */\n#define SQLITE_CONFIG_URI 17 /* int */\n#define SQLITE_CONFIG_PCACHE2 18 /* sqlite3_pcache_methods2* */\n#define SQLITE_CONFIG_GETPCACHE2 19 /* sqlite3_pcache_methods2* */\n#define SQLITE_CONFIG_COVERING_INDEX_SCAN 20 /* int */\n#define SQLITE_CONFIG_SQLLOG 21 /* xSqllog, void* */\n#define SQLITE_CONFIG_MMAP_SIZE 22 /* sqlite3_int64, sqlite3_int64 */\n#define SQLITE_CONFIG_WIN32_HEAPSIZE 23 /* int nByte */\n#define SQLITE_CONFIG_PCACHE_HDRSZ 24 /* int *psz */\n#define SQLITE_CONFIG_PMASZ 25 /* unsigned int szPma */\n#define SQLITE_CONFIG_STMTJRNL_SPILL 26 /* int nByte */\n#define SQLITE_CONFIG_SMALL_MALLOC 27 /* boolean */\n#define SQLITE_CONFIG_SORTERREF_SIZE 28 /* int nByte */\n#define SQLITE_CONFIG_MEMDB_MAXSIZE 29 /* sqlite3_int64 */\n#define SQLITE_CONFIG_ROWID_IN_VIEW 30 /* int* */\n\n/*\n** CAPI3REF: Database Connection Configuration Options\n**\n** These constants are the available integer configuration options that\n** can be passed as the second parameter to the [sqlite3_db_config()] interface.\n**\n** The [sqlite3_db_config()] interface is a var-args functions. It takes a\n** variable number of parameters, though always at least two. The number of\n** parameters passed into sqlite3_db_config() depends on which of these\n** constants is given as the second parameter. This documentation page\n** refers to parameters beyond the second as \"arguments\". Thus, when this\n** page says \"the N-th argument\" it means \"the N-th parameter past the\n** configuration option\" or \"the (N+2)-th parameter to sqlite3_db_config()\".\n**\n** New configuration options may be added in future releases of SQLite.\n** Existing configuration options might be discontinued. Applications\n** should check the return code from [sqlite3_db_config()] to make sure that\n** the call worked. ^The [sqlite3_db_config()] interface will return a\n** non-zero [error code] if a discontinued or unsupported configuration option\n** is invoked.\n**\n**
\n** [[SQLITE_DBCONFIG_LOOKASIDE]]\n**
SQLITE_DBCONFIG_LOOKASIDE
\n**
The SQLITE_DBCONFIG_LOOKASIDE option is used to adjust the\n** configuration of the [lookaside memory allocator] within a database\n** connection.\n** The arguments to the SQLITE_DBCONFIG_LOOKASIDE option are not\n** in the [DBCONFIG arguments|usual format].\n** The SQLITE_DBCONFIG_LOOKASIDE option takes three arguments, not two,\n** so that a call to [sqlite3_db_config()] that uses SQLITE_DBCONFIG_LOOKASIDE\n** should have a total of five parameters.\n**
    \n**

  1. The first argument (\"buf\") is a\n** pointer to a memory buffer to use for lookaside memory.\n** The first argument may be NULL in which case SQLite will allocate the\n** lookaside buffer itself using [sqlite3_malloc()].\n**

  2. The second argument (\"sz\") is the\n** size of each lookaside buffer slot. Lookaside is disabled if \"sz\"\n** is less than 8. The \"sz\" argument should be a multiple of 8 less than\n** 65536. If \"sz\" does not meet this constraint, it is reduced in size until\n** it does.\n**

  3. The third argument (\"cnt\") is the number of slots. Lookaside is disabled\n** if \"cnt\"is less than 1. The \"cnt\" value will be reduced, if necessary, so\n** that the product of \"sz\" and \"cnt\" does not exceed 2,147,418,112. The \"cnt\"\n** parameter is usually chosen so that the product of \"sz\" and \"cnt\" is less\n** than 1,000,000.\n**

\n**

If the \"buf\" argument is not NULL, then it must\n** point to a memory buffer with a size that is greater than\n** or equal to the product of \"sz\" and \"cnt\".\n** The buffer must be aligned to an 8-byte boundary.\n** The lookaside memory\n** configuration for a database connection can only be changed when that\n** connection is not currently using lookaside memory, or in other words\n** when the value returned by [SQLITE_DBSTATUS_LOOKASIDE_USED] is zero.\n** Any attempt to change the lookaside memory configuration when lookaside\n** memory is in use leaves the configuration unchanged and returns\n** [SQLITE_BUSY].\n** If the \"buf\" argument is NULL and an attempt\n** to allocate memory based on \"sz\" and \"cnt\" fails, then\n** lookaside is silently disabled.\n**

\n** The [SQLITE_CONFIG_LOOKASIDE] configuration option can be used to set the\n** default lookaside configuration at initialization. The\n** [-DSQLITE_DEFAULT_LOOKASIDE] option can be used to set the default lookaside\n** configuration at compile-time. Typical values for lookaside are 1200 for\n** \"sz\" and 40 to 100 for \"cnt\".\n**

\n**\n** [[SQLITE_DBCONFIG_ENABLE_FKEY]]\n**
SQLITE_DBCONFIG_ENABLE_FKEY
\n**
^This option is used to enable or disable the enforcement of\n** [foreign key constraints]. This is the same setting that is\n** enabled or disabled by the [PRAGMA foreign_keys] statement.\n** The first argument is an integer which is 0 to disable FK enforcement,\n** positive to enable FK enforcement or negative to leave FK enforcement\n** unchanged. The second parameter is a pointer to an integer into which\n** is written 0 or 1 to indicate whether FK enforcement is off or on\n** following this call. The second parameter may be a NULL pointer, in\n** which case the FK enforcement setting is not reported back.
\n**\n** [[SQLITE_DBCONFIG_ENABLE_TRIGGER]]\n**
SQLITE_DBCONFIG_ENABLE_TRIGGER
\n**
^This option is used to enable or disable [CREATE TRIGGER | triggers].\n** There should be two additional arguments.\n** The first argument is an integer which is 0 to disable triggers,\n** positive to enable triggers or negative to leave the setting unchanged.\n** The second parameter is a pointer to an integer into which\n** is written 0 or 1 to indicate whether triggers are disabled or enabled\n** following this call. The second parameter may be a NULL pointer, in\n** which case the trigger setting is not reported back.\n**\n**

Originally this option disabled all triggers. ^(However, since\n** SQLite version 3.35.0, TEMP triggers are still allowed even if\n** this option is off. So, in other words, this option now only disables\n** triggers in the main database schema or in the schemas of [ATTACH]-ed\n** databases.)^

\n**\n** [[SQLITE_DBCONFIG_ENABLE_VIEW]]\n**
SQLITE_DBCONFIG_ENABLE_VIEW
\n**
^This option is used to enable or disable [CREATE VIEW | views].\n** There must be two additional arguments.\n** The first argument is an integer which is 0 to disable views,\n** positive to enable views or negative to leave the setting unchanged.\n** The second parameter is a pointer to an integer into which\n** is written 0 or 1 to indicate whether views are disabled or enabled\n** following this call. The second parameter may be a NULL pointer, in\n** which case the view setting is not reported back.\n**\n**

Originally this option disabled all views. ^(However, since\n** SQLite version 3.35.0, TEMP views are still allowed even if\n** this option is off. So, in other words, this option now only disables\n** views in the main database schema or in the schemas of ATTACH-ed\n** databases.)^

\n**\n** [[SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER]]\n**
SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER
\n**
^This option is used to enable or disable the\n** [fts3_tokenizer()] function which is part of the\n** [FTS3] full-text search engine extension.\n** There must be two additional arguments.\n** The first argument is an integer which is 0 to disable fts3_tokenizer() or\n** positive to enable fts3_tokenizer() or negative to leave the setting\n** unchanged.\n** The second parameter is a pointer to an integer into which\n** is written 0 or 1 to indicate whether fts3_tokenizer is disabled or enabled\n** following this call. The second parameter may be a NULL pointer, in\n** which case the new setting is not reported back.
\n**\n** [[SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION]]\n**
SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION
\n**
^This option is used to enable or disable the [sqlite3_load_extension()]\n** interface independently of the [load_extension()] SQL function.\n** The [sqlite3_enable_load_extension()] API enables or disables both the\n** C-API [sqlite3_load_extension()] and the SQL function [load_extension()].\n** There must be two additional arguments.\n** When the first argument to this interface is 1, then only the C-API is\n** enabled and the SQL function remains disabled. If the first argument to\n** this interface is 0, then both the C-API and the SQL function are disabled.\n** If the first argument is -1, then no changes are made to state of either the\n** C-API or the SQL function.\n** The second parameter is a pointer to an integer into which\n** is written 0 or 1 to indicate whether [sqlite3_load_extension()] interface\n** is disabled or enabled following this call. The second parameter may\n** be a NULL pointer, in which case the new setting is not reported back.\n**
\n**\n** [[SQLITE_DBCONFIG_MAINDBNAME]]
SQLITE_DBCONFIG_MAINDBNAME
\n**
^This option is used to change the name of the \"main\" database\n** schema. This option does not follow the\n** [DBCONFIG arguments|usual SQLITE_DBCONFIG argument format].\n** This option takes exactly one additional argument so that the\n** [sqlite3_db_config()] call has a total of three parameters. The\n** extra argument must be a pointer to a constant UTF8 string which\n** will become the new schema name in place of \"main\". ^SQLite does\n** not make a copy of the new main schema name string, so the application\n** must ensure that the argument passed into SQLITE_DBCONFIG MAINDBNAME\n** is unchanged until after the database connection closes.\n**
\n**\n** [[SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE]]\n**
SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE
\n**
Usually, when a database in [WAL mode] is closed or detached from a\n** database handle, SQLite checks if if there are other connections to the\n** same database, and if there are no other database connection (if the\n** connection being closed is the last open connection to the database),\n** then SQLite performs a [checkpoint] before closing the connection and\n** deletes the WAL file. The SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE option can\n** be used to override that behavior. The first argument passed to this\n** operation (the third parameter to [sqlite3_db_config()]) is an integer\n** which is positive to disable checkpoints-on-close, or zero (the default)\n** to enable them, and negative to leave the setting unchanged.\n** The second argument (the fourth parameter) is a pointer to an integer\n** into which is written 0 or 1 to indicate whether checkpoints-on-close\n** have been disabled - 0 if they are not disabled, 1 if they are.\n**
\n**\n** [[SQLITE_DBCONFIG_ENABLE_QPSG]]
SQLITE_DBCONFIG_ENABLE_QPSG
\n**
^(The SQLITE_DBCONFIG_ENABLE_QPSG option activates or deactivates\n** the [query planner stability guarantee] (QPSG). When the QPSG is active,\n** a single SQL query statement will always use the same algorithm regardless\n** of values of [bound parameters].)^ The QPSG disables some query optimizations\n** that look at the values of bound parameters, which can make some queries\n** slower. But the QPSG has the advantage of more predictable behavior. With\n** the QPSG active, SQLite will always use the same query plan in the field as\n** was used during testing in the lab.\n** The first argument to this setting is an integer which is 0 to disable\n** the QPSG, positive to enable QPSG, or negative to leave the setting\n** unchanged. The second parameter is a pointer to an integer into which\n** is written 0 or 1 to indicate whether the QPSG is disabled or enabled\n** following this call.\n**
\n**\n** [[SQLITE_DBCONFIG_TRIGGER_EQP]]
SQLITE_DBCONFIG_TRIGGER_EQP
\n**
By default, the output of EXPLAIN QUERY PLAN commands does not\n** include output for any operations performed by trigger programs. This\n** option is used to set or clear (the default) a flag that governs this\n** behavior. The first parameter passed to this operation is an integer -\n** positive to enable output for trigger programs, or zero to disable it,\n** or negative to leave the setting unchanged.\n** The second parameter is a pointer to an integer into which is written\n** 0 or 1 to indicate whether output-for-triggers has been disabled - 0 if\n** it is not disabled, 1 if it is.\n**
\n**\n** [[SQLITE_DBCONFIG_RESET_DATABASE]]
SQLITE_DBCONFIG_RESET_DATABASE
\n**
Set the SQLITE_DBCONFIG_RESET_DATABASE flag and then run\n** [VACUUM] in order to reset a database back to an empty database\n** with no schema and no content. The following process works even for\n** a badly corrupted database file:\n**
    \n**
  1. If the database connection is newly opened, make sure it has read the\n** database schema by preparing then discarding some query against the\n** database, or calling sqlite3_table_column_metadata(), ignoring any\n** errors. This step is only necessary if the application desires to keep\n** the database in WAL mode after the reset if it was in WAL mode before\n** the reset.\n**
  2. sqlite3_db_config(db, SQLITE_DBCONFIG_RESET_DATABASE, 1, 0);\n**
  3. [sqlite3_exec](db, \"[VACUUM]\", 0, 0, 0);\n**
  4. sqlite3_db_config(db, SQLITE_DBCONFIG_RESET_DATABASE, 0, 0);\n**
\n** Because resetting a database is destructive and irreversible, the\n** process requires the use of this obscure API and multiple steps to\n** help ensure that it does not happen by accident. Because this\n** feature must be capable of resetting corrupt databases, and\n** shutting down virtual tables may require access to that corrupt\n** storage, the library must abandon any installed virtual tables\n** without calling their xDestroy() methods.\n**\n** [[SQLITE_DBCONFIG_DEFENSIVE]]
SQLITE_DBCONFIG_DEFENSIVE
\n**
The SQLITE_DBCONFIG_DEFENSIVE option activates or deactivates the\n** \"defensive\" flag for a database connection. When the defensive\n** flag is enabled, language features that allow ordinary SQL to\n** deliberately corrupt the database file are disabled. The disabled\n** features include but are not limited to the following:\n**
    \n**
  • The [PRAGMA writable_schema=ON] statement.\n**
  • The [PRAGMA journal_mode=OFF] statement.\n**
  • The [PRAGMA schema_version=N] statement.\n**
  • Writes to the [sqlite_dbpage] virtual table.\n**
  • Direct writes to [shadow tables].\n**
\n**
\n**\n** [[SQLITE_DBCONFIG_WRITABLE_SCHEMA]]
SQLITE_DBCONFIG_WRITABLE_SCHEMA
\n**
The SQLITE_DBCONFIG_WRITABLE_SCHEMA option activates or deactivates the\n** \"writable_schema\" flag. This has the same effect and is logically equivalent\n** to setting [PRAGMA writable_schema=ON] or [PRAGMA writable_schema=OFF].\n** The first argument to this setting is an integer which is 0 to disable\n** the writable_schema, positive to enable writable_schema, or negative to\n** leave the setting unchanged. The second parameter is a pointer to an\n** integer into which is written 0 or 1 to indicate whether the writable_schema\n** is enabled or disabled following this call.\n**
\n**\n** [[SQLITE_DBCONFIG_LEGACY_ALTER_TABLE]]\n**
SQLITE_DBCONFIG_LEGACY_ALTER_TABLE
\n**
The SQLITE_DBCONFIG_LEGACY_ALTER_TABLE option activates or deactivates\n** the legacy behavior of the [ALTER TABLE RENAME] command such it\n** behaves as it did prior to [version 3.24.0] (2018-06-04). See the\n** \"Compatibility Notice\" on the [ALTER TABLE RENAME documentation] for\n** additional information. This feature can also be turned on and off\n** using the [PRAGMA legacy_alter_table] statement.\n**
\n**\n** [[SQLITE_DBCONFIG_DQS_DML]]\n**
SQLITE_DBCONFIG_DQS_DML
\n**
The SQLITE_DBCONFIG_DQS_DML option activates or deactivates\n** the legacy [double-quoted string literal] misfeature for DML statements\n** only, that is DELETE, INSERT, SELECT, and UPDATE statements. The\n** default value of this setting is determined by the [-DSQLITE_DQS]\n** compile-time option.\n**
\n**\n** [[SQLITE_DBCONFIG_DQS_DDL]]\n**
SQLITE_DBCONFIG_DQS_DDL
\n**
The SQLITE_DBCONFIG_DQS option activates or deactivates\n** the legacy [double-quoted string literal] misfeature for DDL statements,\n** such as CREATE TABLE and CREATE INDEX. The\n** default value of this setting is determined by the [-DSQLITE_DQS]\n** compile-time option.\n**
\n**\n** [[SQLITE_DBCONFIG_TRUSTED_SCHEMA]]\n**
SQLITE_DBCONFIG_TRUSTED_SCHEMA
\n**
The SQLITE_DBCONFIG_TRUSTED_SCHEMA option tells SQLite to\n** assume that database schemas are untainted by malicious content.\n** When the SQLITE_DBCONFIG_TRUSTED_SCHEMA option is disabled, SQLite\n** takes additional defensive steps to protect the application from harm\n** including:\n**
    \n**
  • Prohibit the use of SQL functions inside triggers, views,\n** CHECK constraints, DEFAULT clauses, expression indexes,\n** partial indexes, or generated columns\n** unless those functions are tagged with [SQLITE_INNOCUOUS].\n**
  • Prohibit the use of virtual tables inside of triggers or views\n** unless those virtual tables are tagged with [SQLITE_VTAB_INNOCUOUS].\n**
\n** This setting defaults to \"on\" for legacy compatibility, however\n** all applications are advised to turn it off if possible. This setting\n** can also be controlled using the [PRAGMA trusted_schema] statement.\n**
\n**\n** [[SQLITE_DBCONFIG_LEGACY_FILE_FORMAT]]\n**
SQLITE_DBCONFIG_LEGACY_FILE_FORMAT
\n**
The SQLITE_DBCONFIG_LEGACY_FILE_FORMAT option activates or deactivates\n** the legacy file format flag. When activated, this flag causes all newly\n** created database file to have a schema format version number (the 4-byte\n** integer found at offset 44 into the database header) of 1. This in turn\n** means that the resulting database file will be readable and writable by\n** any SQLite version back to 3.0.0 ([dateof:3.0.0]). Without this setting,\n** newly created databases are generally not understandable by SQLite versions\n** prior to 3.3.0 ([dateof:3.3.0]). As these words are written, there\n** is now scarcely any need to generate database files that are compatible\n** all the way back to version 3.0.0, and so this setting is of little\n** practical use, but is provided so that SQLite can continue to claim the\n** ability to generate new database files that are compatible with version\n** 3.0.0.\n**

Note that when the SQLITE_DBCONFIG_LEGACY_FILE_FORMAT setting is on,\n** the [VACUUM] command will fail with an obscure error when attempting to\n** process a table with generated columns and a descending index. This is\n** not considered a bug since SQLite versions 3.3.0 and earlier do not support\n** either generated columns or descending indexes.\n**

\n**\n** [[SQLITE_DBCONFIG_STMT_SCANSTATUS]]\n**
SQLITE_DBCONFIG_STMT_SCANSTATUS
\n**
The SQLITE_DBCONFIG_STMT_SCANSTATUS option is only useful in\n** SQLITE_ENABLE_STMT_SCANSTATUS builds. In this case, it sets or clears\n** a flag that enables collection of the sqlite3_stmt_scanstatus_v2()\n** statistics. For statistics to be collected, the flag must be set on\n** the database handle both when the SQL statement is prepared and when it\n** is stepped. The flag is set (collection of statistics is enabled)\n** by default.

This option takes two arguments: an integer and a pointer to\n** an integer.. The first argument is 1, 0, or -1 to enable, disable, or\n** leave unchanged the statement scanstatus option. If the second argument\n** is not NULL, then the value of the statement scanstatus setting after\n** processing the first argument is written into the integer that the second\n** argument points to.\n**

\n**\n** [[SQLITE_DBCONFIG_REVERSE_SCANORDER]]\n**
SQLITE_DBCONFIG_REVERSE_SCANORDER
\n**
The SQLITE_DBCONFIG_REVERSE_SCANORDER option changes the default order\n** in which tables and indexes are scanned so that the scans start at the end\n** and work toward the beginning rather than starting at the beginning and\n** working toward the end. Setting SQLITE_DBCONFIG_REVERSE_SCANORDER is the\n** same as setting [PRAGMA reverse_unordered_selects].

This option takes\n** two arguments which are an integer and a pointer to an integer. The first\n** argument is 1, 0, or -1 to enable, disable, or leave unchanged the\n** reverse scan order flag, respectively. If the second argument is not NULL,\n** then 0 or 1 is written into the integer that the second argument points to\n** depending on if the reverse scan order flag is set after processing the\n** first argument.\n**

\n**\n** [[SQLITE_DBCONFIG_ENABLE_ATTACH_CREATE]]\n**
SQLITE_DBCONFIG_ENABLE_ATTACH_CREATE
\n**
The SQLITE_DBCONFIG_ENABLE_ATTACH_CREATE option enables or disables\n** the ability of the [ATTACH DATABASE] SQL command to create a new database\n** file if the database filed named in the ATTACH command does not already\n** exist. This ability of ATTACH to create a new database is enabled by\n** default. Applications can disable or reenable the ability for ATTACH to\n** create new database files using this DBCONFIG option.

\n** This option takes two arguments which are an integer and a pointer\n** to an integer. The first argument is 1, 0, or -1 to enable, disable, or\n** leave unchanged the attach-create flag, respectively. If the second\n** argument is not NULL, then 0 or 1 is written into the integer that the\n** second argument points to depending on if the attach-create flag is set\n** after processing the first argument.\n**

\n**\n** [[SQLITE_DBCONFIG_ENABLE_ATTACH_WRITE]]\n**
SQLITE_DBCONFIG_ENABLE_ATTACH_WRITE
\n**
The SQLITE_DBCONFIG_ENABLE_ATTACH_WRITE option enables or disables the\n** ability of the [ATTACH DATABASE] SQL command to open a database for writing.\n** This capability is enabled by default. Applications can disable or\n** reenable this capability using the current DBCONFIG option. If the\n** the this capability is disabled, the [ATTACH] command will still work,\n** but the database will be opened read-only. If this option is disabled,\n** then the ability to create a new database using [ATTACH] is also disabled,\n** regardless of the value of the [SQLITE_DBCONFIG_ENABLE_ATTACH_CREATE]\n** option.

\n** This option takes two arguments which are an integer and a pointer\n** to an integer. The first argument is 1, 0, or -1 to enable, disable, or\n** leave unchanged the ability to ATTACH another database for writing,\n** respectively. If the second argument is not NULL, then 0 or 1 is written\n** into the integer to which the second argument points, depending on whether\n** the ability to ATTACH a read/write database is enabled or disabled\n** after processing the first argument.\n**

\n**\n** [[SQLITE_DBCONFIG_ENABLE_COMMENTS]]\n**
SQLITE_DBCONFIG_ENABLE_COMMENTS
\n**
The SQLITE_DBCONFIG_ENABLE_COMMENTS option enables or disables the\n** ability to include comments in SQL text. Comments are enabled by default.\n** An application can disable or reenable comments in SQL text using this\n** DBCONFIG option.

\n** This option takes two arguments which are an integer and a pointer\n** to an integer. The first argument is 1, 0, or -1 to enable, disable, or\n** leave unchanged the ability to use comments in SQL text,\n** respectively. If the second argument is not NULL, then 0 or 1 is written\n** into the integer that the second argument points to depending on if\n** comments are allowed in SQL text after processing the first argument.\n**

\n**\n**
\n**\n** [[DBCONFIG arguments]]

Arguments To SQLITE_DBCONFIG Options

\n**\n**

Most of the SQLITE_DBCONFIG options take two arguments, so that the\n** overall call to [sqlite3_db_config()] has a total of four parameters.\n** The first argument (the third parameter to sqlite3_db_config()) is a integer.\n** The second argument is a pointer to an integer. If the first argument is 1,\n** then the option becomes enabled. If the first integer argument is 0, then the\n** option is disabled. If the first argument is -1, then the option setting\n** is unchanged. The second argument, the pointer to an integer, may be NULL.\n** If the second argument is not NULL, then a value of 0 or 1 is written into\n** the integer to which the second argument points, depending on whether the\n** setting is disabled or enabled after applying any changes specified by\n** the first argument.\n**\n**

While most SQLITE_DBCONFIG options use the argument format\n** described in the previous paragraph, the [SQLITE_DBCONFIG_MAINDBNAME]\n** and [SQLITE_DBCONFIG_LOOKASIDE] options are different. See the\n** documentation of those exceptional options for details.\n*/\n#define SQLITE_DBCONFIG_MAINDBNAME 1000 /* const char* */\n#define SQLITE_DBCONFIG_LOOKASIDE 1001 /* void* int int */\n#define SQLITE_DBCONFIG_ENABLE_FKEY 1002 /* int int* */\n#define SQLITE_DBCONFIG_ENABLE_TRIGGER 1003 /* int int* */\n#define SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER 1004 /* int int* */\n#define SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION 1005 /* int int* */\n#define SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE 1006 /* int int* */\n#define SQLITE_DBCONFIG_ENABLE_QPSG 1007 /* int int* */\n#define SQLITE_DBCONFIG_TRIGGER_EQP 1008 /* int int* */\n#define SQLITE_DBCONFIG_RESET_DATABASE 1009 /* int int* */\n#define SQLITE_DBCONFIG_DEFENSIVE 1010 /* int int* */\n#define SQLITE_DBCONFIG_WRITABLE_SCHEMA 1011 /* int int* */\n#define SQLITE_DBCONFIG_LEGACY_ALTER_TABLE 1012 /* int int* */\n#define SQLITE_DBCONFIG_DQS_DML 1013 /* int int* */\n#define SQLITE_DBCONFIG_DQS_DDL 1014 /* int int* */\n#define SQLITE_DBCONFIG_ENABLE_VIEW 1015 /* int int* */\n#define SQLITE_DBCONFIG_LEGACY_FILE_FORMAT 1016 /* int int* */\n#define SQLITE_DBCONFIG_TRUSTED_SCHEMA 1017 /* int int* */\n#define SQLITE_DBCONFIG_STMT_SCANSTATUS 1018 /* int int* */\n#define SQLITE_DBCONFIG_REVERSE_SCANORDER 1019 /* int int* */\n#define SQLITE_DBCONFIG_ENABLE_ATTACH_CREATE 1020 /* int int* */\n#define SQLITE_DBCONFIG_ENABLE_ATTACH_WRITE 1021 /* int int* */\n#define SQLITE_DBCONFIG_ENABLE_COMMENTS 1022 /* int int* */\n#define SQLITE_DBCONFIG_MAX 1022 /* Largest DBCONFIG */\n\n/*\n** CAPI3REF: Enable Or Disable Extended Result Codes\n** METHOD: sqlite3\n**\n** ^The sqlite3_extended_result_codes() routine enables or disables the\n** [extended result codes] feature of SQLite. ^The extended result\n** codes are disabled by default for historical compatibility.\n*/\nSQLITE_API int sqlite3_extended_result_codes(sqlite3*, int onoff);\n\n/*\n** CAPI3REF: Last Insert Rowid\n** METHOD: sqlite3\n**\n** ^Each entry in most SQLite tables (except for [WITHOUT ROWID] tables)\n** has a unique 64-bit signed\n** integer key called the [ROWID | \"rowid\"]. ^The rowid is always available\n** as an undeclared column named ROWID, OID, or _ROWID_ as long as those\n** names are not also used by explicitly declared columns. ^If\n** the table has a column of type [INTEGER PRIMARY KEY] then that column\n** is another alias for the rowid.\n**\n** ^The sqlite3_last_insert_rowid(D) interface usually returns the [rowid] of\n** the most recent successful [INSERT] into a rowid table or [virtual table]\n** on database connection D. ^Inserts into [WITHOUT ROWID] tables are not\n** recorded. ^If no successful [INSERT]s into rowid tables have ever occurred\n** on the database connection D, then sqlite3_last_insert_rowid(D) returns\n** zero.\n**\n** As well as being set automatically as rows are inserted into database\n** tables, the value returned by this function may be set explicitly by\n** [sqlite3_set_last_insert_rowid()]\n**\n** Some virtual table implementations may INSERT rows into rowid tables as\n** part of committing a transaction (e.g. to flush data accumulated in memory\n** to disk). In this case subsequent calls to this function return the rowid\n** associated with these internal INSERT operations, which leads to\n** unintuitive results. Virtual table implementations that do write to rowid\n** tables in this way can avoid this problem by restoring the original\n** rowid value using [sqlite3_set_last_insert_rowid()] before returning\n** control to the user.\n**\n** ^(If an [INSERT] occurs within a trigger then this routine will\n** return the [rowid] of the inserted row as long as the trigger is\n** running. Once the trigger program ends, the value returned\n** by this routine reverts to what it was before the trigger was fired.)^\n**\n** ^An [INSERT] that fails due to a constraint violation is not a\n** successful [INSERT] and does not change the value returned by this\n** routine. ^Thus INSERT OR FAIL, INSERT OR IGNORE, INSERT OR ROLLBACK,\n** and INSERT OR ABORT make no changes to the return value of this\n** routine when their insertion fails. ^(When INSERT OR REPLACE\n** encounters a constraint violation, it does not fail. The\n** INSERT continues to completion after deleting rows that caused\n** the constraint problem so INSERT OR REPLACE will always change\n** the return value of this interface.)^\n**\n** ^For the purposes of this routine, an [INSERT] is considered to\n** be successful even if it is subsequently rolled back.\n**\n** This function is accessible to SQL statements via the\n** [last_insert_rowid() SQL function].\n**\n** If a separate thread performs a new [INSERT] on the same\n** database connection while the [sqlite3_last_insert_rowid()]\n** function is running and thus changes the last insert [rowid],\n** then the value returned by [sqlite3_last_insert_rowid()] is\n** unpredictable and might not equal either the old or the new\n** last insert [rowid].\n*/\nSQLITE_API sqlite3_int64 sqlite3_last_insert_rowid(sqlite3*);\n\n/*\n** CAPI3REF: Set the Last Insert Rowid value.\n** METHOD: sqlite3\n**\n** The sqlite3_set_last_insert_rowid(D, R) method allows the application to\n** set the value returned by calling sqlite3_last_insert_rowid(D) to R\n** without inserting a row into the database.\n*/\nSQLITE_API void sqlite3_set_last_insert_rowid(sqlite3*,sqlite3_int64);\n\n/*\n** CAPI3REF: Count The Number Of Rows Modified\n** METHOD: sqlite3\n**\n** ^These functions return the number of rows modified, inserted or\n** deleted by the most recently completed INSERT, UPDATE or DELETE\n** statement on the database connection specified by the only parameter.\n** The two functions are identical except for the type of the return value\n** and that if the number of rows modified by the most recent INSERT, UPDATE,\n** or DELETE is greater than the maximum value supported by type \"int\", then\n** the return value of sqlite3_changes() is undefined. ^Executing any other\n** type of SQL statement does not modify the value returned by these functions.\n** For the purposes of this interface, a CREATE TABLE AS SELECT statement\n** does not count as an INSERT, UPDATE or DELETE statement and hence the rows\n** added to the new table by the CREATE TABLE AS SELECT statement are not\n** counted.\n**\n** ^Only changes made directly by the INSERT, UPDATE or DELETE statement are\n** considered - auxiliary changes caused by [CREATE TRIGGER | triggers],\n** [foreign key actions] or [REPLACE] constraint resolution are not counted.\n**\n** Changes to a view that are intercepted by\n** [INSTEAD OF trigger | INSTEAD OF triggers] are not counted. ^The value\n** returned by sqlite3_changes() immediately after an INSERT, UPDATE or\n** DELETE statement run on a view is always zero. Only changes made to real\n** tables are counted.\n**\n** Things are more complicated if the sqlite3_changes() function is\n** executed while a trigger program is running. This may happen if the\n** program uses the [changes() SQL function], or if some other callback\n** function invokes sqlite3_changes() directly. Essentially:\n**\n**

    \n**
  • ^(Before entering a trigger program the value returned by\n** sqlite3_changes() function is saved. After the trigger program\n** has finished, the original value is restored.)^\n**\n**
  • ^(Within a trigger program each INSERT, UPDATE and DELETE\n** statement sets the value returned by sqlite3_changes()\n** upon completion as normal. Of course, this value will not include\n** any changes performed by sub-triggers, as the sqlite3_changes()\n** value will be saved and restored after each sub-trigger has run.)^\n**
\n**\n** ^This means that if the changes() SQL function (or similar) is used\n** by the first INSERT, UPDATE or DELETE statement within a trigger, it\n** returns the value as set when the calling statement began executing.\n** ^If it is used by the second or subsequent such statement within a trigger\n** program, the value returned reflects the number of rows modified by the\n** previous INSERT, UPDATE or DELETE statement within the same trigger.\n**\n** If a separate thread makes changes on the same database connection\n** while [sqlite3_changes()] is running then the value returned\n** is unpredictable and not meaningful.\n**\n** See also:\n**
    \n**
  • the [sqlite3_total_changes()] interface\n**
  • the [count_changes pragma]\n**
  • the [changes() SQL function]\n**
  • the [data_version pragma]\n**
\n*/\nSQLITE_API int sqlite3_changes(sqlite3*);\nSQLITE_API sqlite3_int64 sqlite3_changes64(sqlite3*);\n\n/*\n** CAPI3REF: Total Number Of Rows Modified\n** METHOD: sqlite3\n**\n** ^These functions return the total number of rows inserted, modified or\n** deleted by all [INSERT], [UPDATE] or [DELETE] statements completed\n** since the database connection was opened, including those executed as\n** part of trigger programs. The two functions are identical except for the\n** type of the return value and that if the number of rows modified by the\n** connection exceeds the maximum value supported by type \"int\", then\n** the return value of sqlite3_total_changes() is undefined. ^Executing\n** any other type of SQL statement does not affect the value returned by\n** sqlite3_total_changes().\n**\n** ^Changes made as part of [foreign key actions] are included in the\n** count, but those made as part of REPLACE constraint resolution are\n** not. ^Changes to a view that are intercepted by INSTEAD OF triggers\n** are not counted.\n**\n** The [sqlite3_total_changes(D)] interface only reports the number\n** of rows that changed due to SQL statement run against database\n** connection D. Any changes by other database connections are ignored.\n** To detect changes against a database file from other database\n** connections use the [PRAGMA data_version] command or the\n** [SQLITE_FCNTL_DATA_VERSION] [file control].\n**\n** If a separate thread makes changes on the same database connection\n** while [sqlite3_total_changes()] is running then the value\n** returned is unpredictable and not meaningful.\n**\n** See also:\n**
    \n**
  • the [sqlite3_changes()] interface\n**
  • the [count_changes pragma]\n**
  • the [changes() SQL function]\n**
  • the [data_version pragma]\n**
  • the [SQLITE_FCNTL_DATA_VERSION] [file control]\n**
\n*/\nSQLITE_API int sqlite3_total_changes(sqlite3*);\nSQLITE_API sqlite3_int64 sqlite3_total_changes64(sqlite3*);\n\n/*\n** CAPI3REF: Interrupt A Long-Running Query\n** METHOD: sqlite3\n**\n** ^This function causes any pending database operation to abort and\n** return at its earliest opportunity. This routine is typically\n** called in response to a user action such as pressing \"Cancel\"\n** or Ctrl-C where the user wants a long query operation to halt\n** immediately.\n**\n** ^It is safe to call this routine from a thread different from the\n** thread that is currently running the database operation. But it\n** is not safe to call this routine with a [database connection] that\n** is closed or might close before sqlite3_interrupt() returns.\n**\n** ^If an SQL operation is very nearly finished at the time when\n** sqlite3_interrupt() is called, then it might not have an opportunity\n** to be interrupted and might continue to completion.\n**\n** ^An SQL operation that is interrupted will return [SQLITE_INTERRUPT].\n** ^If the interrupted SQL operation is an INSERT, UPDATE, or DELETE\n** that is inside an explicit transaction, then the entire transaction\n** will be rolled back automatically.\n**\n** ^The sqlite3_interrupt(D) call is in effect until all currently running\n** SQL statements on [database connection] D complete. ^Any new SQL statements\n** that are started after the sqlite3_interrupt() call and before the\n** running statement count reaches zero are interrupted as if they had been\n** running prior to the sqlite3_interrupt() call. ^New SQL statements\n** that are started after the running statement count reaches zero are\n** not effected by the sqlite3_interrupt().\n** ^A call to sqlite3_interrupt(D) that occurs when there are no running\n** SQL statements is a no-op and has no effect on SQL statements\n** that are started after the sqlite3_interrupt() call returns.\n**\n** ^The [sqlite3_is_interrupted(D)] interface can be used to determine whether\n** or not an interrupt is currently in effect for [database connection] D.\n** It returns 1 if an interrupt is currently in effect, or 0 otherwise.\n*/\nSQLITE_API void sqlite3_interrupt(sqlite3*);\nSQLITE_API int sqlite3_is_interrupted(sqlite3*);\n\n/*\n** CAPI3REF: Determine If An SQL Statement Is Complete\n**\n** These routines are useful during command-line input to determine if the\n** currently entered text seems to form a complete SQL statement or\n** if additional input is needed before sending the text into\n** SQLite for parsing. ^These routines return 1 if the input string\n** appears to be a complete SQL statement. ^A statement is judged to be\n** complete if it ends with a semicolon token and is not a prefix of a\n** well-formed CREATE TRIGGER statement. ^Semicolons that are embedded within\n** string literals or quoted identifier names or comments are not\n** independent tokens (they are part of the token in which they are\n** embedded) and thus do not count as a statement terminator. ^Whitespace\n** and comments that follow the final semicolon are ignored.\n**\n** ^These routines return 0 if the statement is incomplete. ^If a\n** memory allocation fails, then SQLITE_NOMEM is returned.\n**\n** ^These routines do not parse the SQL statements thus\n** will not detect syntactically incorrect SQL.\n**\n** ^(If SQLite has not been initialized using [sqlite3_initialize()] prior\n** to invoking sqlite3_complete16() then sqlite3_initialize() is invoked\n** automatically by sqlite3_complete16(). If that initialization fails,\n** then the return value from sqlite3_complete16() will be non-zero\n** regardless of whether or not the input SQL is complete.)^\n**\n** The input to [sqlite3_complete()] must be a zero-terminated\n** UTF-8 string.\n**\n** The input to [sqlite3_complete16()] must be a zero-terminated\n** UTF-16 string in native byte order.\n*/\nSQLITE_API int sqlite3_complete(const char *sql);\nSQLITE_API int sqlite3_complete16(const void *sql);\n\n/*\n** CAPI3REF: Register A Callback To Handle SQLITE_BUSY Errors\n** KEYWORDS: {busy-handler callback} {busy handler}\n** METHOD: sqlite3\n**\n** ^The sqlite3_busy_handler(D,X,P) routine sets a callback function X\n** that might be invoked with argument P whenever\n** an attempt is made to access a database table associated with\n** [database connection] D when another thread\n** or process has the table locked.\n** The sqlite3_busy_handler() interface is used to implement\n** [sqlite3_busy_timeout()] and [PRAGMA busy_timeout].\n**\n** ^If the busy callback is NULL, then [SQLITE_BUSY]\n** is returned immediately upon encountering the lock. ^If the busy callback\n** is not NULL, then the callback might be invoked with two arguments.\n**\n** ^The first argument to the busy handler is a copy of the void* pointer which\n** is the third argument to sqlite3_busy_handler(). ^The second argument to\n** the busy handler callback is the number of times that the busy handler has\n** been invoked previously for the same locking event. ^If the\n** busy callback returns 0, then no additional attempts are made to\n** access the database and [SQLITE_BUSY] is returned\n** to the application.\n** ^If the callback returns non-zero, then another attempt\n** is made to access the database and the cycle repeats.\n**\n** The presence of a busy handler does not guarantee that it will be invoked\n** when there is lock contention. ^If SQLite determines that invoking the busy\n** handler could result in a deadlock, it will go ahead and return [SQLITE_BUSY]\n** to the application instead of invoking the\n** busy handler.\n** Consider a scenario where one process is holding a read lock that\n** it is trying to promote to a reserved lock and\n** a second process is holding a reserved lock that it is trying\n** to promote to an exclusive lock. The first process cannot proceed\n** because it is blocked by the second and the second process cannot\n** proceed because it is blocked by the first. If both processes\n** invoke the busy handlers, neither will make any progress. Therefore,\n** SQLite returns [SQLITE_BUSY] for the first process, hoping that this\n** will induce the first process to release its read lock and allow\n** the second process to proceed.\n**\n** ^The default busy callback is NULL.\n**\n** ^(There can only be a single busy handler defined for each\n** [database connection]. Setting a new busy handler clears any\n** previously set handler.)^ ^Note that calling [sqlite3_busy_timeout()]\n** or evaluating [PRAGMA busy_timeout=N] will change the\n** busy handler and thus clear any previously set busy handler.\n**\n** The busy callback should not take any actions which modify the\n** database connection that invoked the busy handler. In other words,\n** the busy handler is not reentrant. Any such actions\n** result in undefined behavior.\n**\n** A busy handler must not close the database connection\n** or [prepared statement] that invoked the busy handler.\n*/\nSQLITE_API int sqlite3_busy_handler(sqlite3*,int(*)(void*,int),void*);\n\n/*\n** CAPI3REF: Set A Busy Timeout\n** METHOD: sqlite3\n**\n** ^This routine sets a [sqlite3_busy_handler | busy handler] that sleeps\n** for a specified amount of time when a table is locked. ^The handler\n** will sleep multiple times until at least \"ms\" milliseconds of sleeping\n** have accumulated. ^After at least \"ms\" milliseconds of sleeping,\n** the handler returns 0 which causes [sqlite3_step()] to return\n** [SQLITE_BUSY].\n**\n** ^Calling this routine with an argument less than or equal to zero\n** turns off all busy handlers.\n**\n** ^(There can only be a single busy handler for a particular\n** [database connection] at any given moment. If another busy handler\n** was defined (using [sqlite3_busy_handler()]) prior to calling\n** this routine, that other busy handler is cleared.)^\n**\n** See also: [PRAGMA busy_timeout]\n*/\nSQLITE_API int sqlite3_busy_timeout(sqlite3*, int ms);\n\n/*\n** CAPI3REF: Set the Setlk Timeout\n** METHOD: sqlite3\n**\n** This routine is only useful in SQLITE_ENABLE_SETLK_TIMEOUT builds. If\n** the VFS supports blocking locks, it sets the timeout in ms used by\n** eligible locks taken on wal mode databases by the specified database\n** handle. In non-SQLITE_ENABLE_SETLK_TIMEOUT builds, or if the VFS does\n** not support blocking locks, this function is a no-op.\n**\n** Passing 0 to this function disables blocking locks altogether. Passing\n** -1 to this function requests that the VFS blocks for a long time -\n** indefinitely if possible. The results of passing any other negative value\n** are undefined.\n**\n** Internally, each SQLite database handle store two timeout values - the\n** busy-timeout (used for rollback mode databases, or if the VFS does not\n** support blocking locks) and the setlk-timeout (used for blocking locks\n** on wal-mode databases). The sqlite3_busy_timeout() method sets both\n** values, this function sets only the setlk-timeout value. Therefore,\n** to configure separate busy-timeout and setlk-timeout values for a single\n** database handle, call sqlite3_busy_timeout() followed by this function.\n**\n** Whenever the number of connections to a wal mode database falls from\n** 1 to 0, the last connection takes an exclusive lock on the database,\n** then checkpoints and deletes the wal file. While it is doing this, any\n** new connection that tries to read from the database fails with an\n** SQLITE_BUSY error. Or, if the SQLITE_SETLK_BLOCK_ON_CONNECT flag is\n** passed to this API, the new connection blocks until the exclusive lock\n** has been released.\n*/\nSQLITE_API int sqlite3_setlk_timeout(sqlite3*, int ms, int flags);\n\n/*\n** CAPI3REF: Flags for sqlite3_setlk_timeout()\n*/\n#define SQLITE_SETLK_BLOCK_ON_CONNECT 0x01\n\n/*\n** CAPI3REF: Convenience Routines For Running Queries\n** METHOD: sqlite3\n**\n** This is a legacy interface that is preserved for backwards compatibility.\n** Use of this interface is not recommended.\n**\n** Definition: A result table is memory data structure created by the\n** [sqlite3_get_table()] interface. A result table records the\n** complete query results from one or more queries.\n**\n** The table conceptually has a number of rows and columns. But\n** these numbers are not part of the result table itself. These\n** numbers are obtained separately. Let N be the number of rows\n** and M be the number of columns.\n**\n** A result table is an array of pointers to zero-terminated UTF-8 strings.\n** There are (N+1)*M elements in the array. The first M pointers point\n** to zero-terminated strings that contain the names of the columns.\n** The remaining entries all point to query results. NULL values result\n** in NULL pointers. All other values are in their UTF-8 zero-terminated\n** string representation as returned by [sqlite3_column_text()].\n**\n** A result table might consist of one or more memory allocations.\n** It is not safe to pass a result table directly to [sqlite3_free()].\n** A result table should be deallocated using [sqlite3_free_table()].\n**\n** ^(As an example of the result table format, suppose a query result\n** is as follows:\n**\n** 
\n**        Name        | Age\n**        -----------------------\n**        Alice       | 43\n**        Bob         | 28\n**        Cindy       | 21\n**
\n**\n** There are two columns (M==2) and three rows (N==3). Thus the\n** result table has 8 entries. Suppose the result table is stored\n** in an array named azResult. Then azResult holds this content:\n**\n** 
\n**        azResult[0] = \"Name\";\n**        azResult[1] = \"Age\";\n**        azResult[2] = \"Alice\";\n**        azResult[3] = \"43\";\n**        azResult[4] = \"Bob\";\n**        azResult[5] = \"28\";\n**        azResult[6] = \"Cindy\";\n**        azResult[7] = \"21\";\n**
)^\n**\n** ^The sqlite3_get_table() function evaluates one or more\n** semicolon-separated SQL statements in the zero-terminated UTF-8\n** string of its 2nd parameter and returns a result table to the\n** pointer given in its 3rd parameter.\n**\n** After the application has finished with the result from sqlite3_get_table(),\n** it must pass the result table pointer to sqlite3_free_table() in order to\n** release the memory that was malloced. Because of the way the\n** [sqlite3_malloc()] happens within sqlite3_get_table(), the calling\n** function must not try to call [sqlite3_free()] directly. Only\n** [sqlite3_free_table()] is able to release the memory properly and safely.\n**\n** The sqlite3_get_table() interface is implemented as a wrapper around\n** [sqlite3_exec()]. The sqlite3_get_table() routine does not have access\n** to any internal data structures of SQLite. It uses only the public\n** interface defined here. As a consequence, errors that occur in the\n** wrapper layer outside of the internal [sqlite3_exec()] call are not\n** reflected in subsequent calls to [sqlite3_errcode()] or\n** [sqlite3_errmsg()].\n*/\nSQLITE_API int sqlite3_get_table(\n sqlite3 *db, /* An open database */\n const char *zSql, /* SQL to be evaluated */\n char ***pazResult, /* Results of the query */\n int *pnRow, /* Number of result rows written here */\n int *pnColumn, /* Number of result columns written here */\n char **pzErrmsg /* Error msg written here */\n);\nSQLITE_API void sqlite3_free_table(char **result);\n\n/*\n** CAPI3REF: Formatted String Printing Functions\n**\n** These routines are work-alikes of the \"printf()\" family of functions\n** from the standard C library.\n** These routines understand most of the common formatting options from\n** the standard library printf()\n** plus some additional non-standard formats ([%q], [%Q], [%w], and [%z]).\n** See the [built-in printf()] documentation for details.\n**\n** ^The sqlite3_mprintf() and sqlite3_vmprintf() routines write their\n** results into memory obtained from [sqlite3_malloc64()].\n** The strings returned by these two routines should be\n** released by [sqlite3_free()]. ^Both routines return a\n** NULL pointer if [sqlite3_malloc64()] is unable to allocate enough\n** memory to hold the resulting string.\n**\n** ^(The sqlite3_snprintf() routine is similar to \"snprintf()\" from\n** the standard C library. The result is written into the\n** buffer supplied as the second parameter whose size is given by\n** the first parameter. Note that the order of the\n** first two parameters is reversed from snprintf().)^ This is an\n** historical accident that cannot be fixed without breaking\n** backwards compatibility. ^(Note also that sqlite3_snprintf()\n** returns a pointer to its buffer instead of the number of\n** characters actually written into the buffer.)^ We admit that\n** the number of characters written would be a more useful return\n** value but we cannot change the implementation of sqlite3_snprintf()\n** now without breaking compatibility.\n**\n** ^As long as the buffer size is greater than zero, sqlite3_snprintf()\n** guarantees that the buffer is always zero-terminated. ^The first\n** parameter \"n\" is the total size of the buffer, including space for\n** the zero terminator. So the longest string that can be completely\n** written will be n-1 characters.\n**\n** ^The sqlite3_vsnprintf() routine is a varargs version of sqlite3_snprintf().\n**\n** See also: [built-in printf()], [printf() SQL function]\n*/\nSQLITE_API char *sqlite3_mprintf(const char*,...);\nSQLITE_API char *sqlite3_vmprintf(const char*, va_list);\nSQLITE_API char *sqlite3_snprintf(int,char*,const char*, ...);\nSQLITE_API char *sqlite3_vsnprintf(int,char*,const char*, va_list);\n\n/*\n** CAPI3REF: Memory Allocation Subsystem\n**\n** The SQLite core uses these three routines for all of its own\n** internal memory allocation needs. \"Core\" in the previous sentence\n** does not include operating-system specific [VFS] implementation. The\n** Windows VFS uses native malloc() and free() for some operations.\n**\n** ^The sqlite3_malloc() routine returns a pointer to a block\n** of memory at least N bytes in length, where N is the parameter.\n** ^If sqlite3_malloc() is unable to obtain sufficient free\n** memory, it returns a NULL pointer. ^If the parameter N to\n** sqlite3_malloc() is zero or negative then sqlite3_malloc() returns\n** a NULL pointer.\n**\n** ^The sqlite3_malloc64(N) routine works just like\n** sqlite3_malloc(N) except that N is an unsigned 64-bit integer instead\n** of a signed 32-bit integer.\n**\n** ^Calling sqlite3_free() with a pointer previously returned\n** by sqlite3_malloc() or sqlite3_realloc() releases that memory so\n** that it might be reused. ^The sqlite3_free() routine is\n** a no-op if is called with a NULL pointer. Passing a NULL pointer\n** to sqlite3_free() is harmless. After being freed, memory\n** should neither be read nor written. Even reading previously freed\n** memory might result in a segmentation fault or other severe error.\n** Memory corruption, a segmentation fault, or other severe error\n** might result if sqlite3_free() is called with a non-NULL pointer that\n** was not obtained from sqlite3_malloc() or sqlite3_realloc().\n**\n** ^The sqlite3_realloc(X,N) interface attempts to resize a\n** prior memory allocation X to be at least N bytes.\n** ^If the X parameter to sqlite3_realloc(X,N)\n** is a NULL pointer then its behavior is identical to calling\n** sqlite3_malloc(N).\n** ^If the N parameter to sqlite3_realloc(X,N) is zero or\n** negative then the behavior is exactly the same as calling\n** sqlite3_free(X).\n** ^sqlite3_realloc(X,N) returns a pointer to a memory allocation\n** of at least N bytes in size or NULL if insufficient memory is available.\n** ^If M is the size of the prior allocation, then min(N,M) bytes\n** of the prior allocation are copied into the beginning of buffer returned\n** by sqlite3_realloc(X,N) and the prior allocation is freed.\n** ^If sqlite3_realloc(X,N) returns NULL and N is positive, then the\n** prior allocation is not freed.\n**\n** ^The sqlite3_realloc64(X,N) interfaces works the same as\n** sqlite3_realloc(X,N) except that N is a 64-bit unsigned integer instead\n** of a 32-bit signed integer.\n**\n** ^If X is a memory allocation previously obtained from sqlite3_malloc(),\n** sqlite3_malloc64(), sqlite3_realloc(), or sqlite3_realloc64(), then\n** sqlite3_msize(X) returns the size of that memory allocation in bytes.\n** ^The value returned by sqlite3_msize(X) might be larger than the number\n** of bytes requested when X was allocated. ^If X is a NULL pointer then\n** sqlite3_msize(X) returns zero. If X points to something that is not\n** the beginning of memory allocation, or if it points to a formerly\n** valid memory allocation that has now been freed, then the behavior\n** of sqlite3_msize(X) is undefined and possibly harmful.\n**\n** ^The memory returned by sqlite3_malloc(), sqlite3_realloc(),\n** sqlite3_malloc64(), and sqlite3_realloc64()\n** is always aligned to at least an 8 byte boundary, or to a\n** 4 byte boundary if the [SQLITE_4_BYTE_ALIGNED_MALLOC] compile-time\n** option is used.\n**\n** The pointer arguments to [sqlite3_free()] and [sqlite3_realloc()]\n** must be either NULL or else pointers obtained from a prior\n** invocation of [sqlite3_malloc()] or [sqlite3_realloc()] that have\n** not yet been released.\n**\n** The application must not read or write any part of\n** a block of memory after it has been released using\n** [sqlite3_free()] or [sqlite3_realloc()].\n*/\nSQLITE_API void *sqlite3_malloc(int);\nSQLITE_API void *sqlite3_malloc64(sqlite3_uint64);\nSQLITE_API void *sqlite3_realloc(void*, int);\nSQLITE_API void *sqlite3_realloc64(void*, sqlite3_uint64);\nSQLITE_API void sqlite3_free(void*);\nSQLITE_API sqlite3_uint64 sqlite3_msize(void*);\n\n/*\n** CAPI3REF: Memory Allocator Statistics\n**\n** SQLite provides these two interfaces for reporting on the status\n** of the [sqlite3_malloc()], [sqlite3_free()], and [sqlite3_realloc()]\n** routines, which form the built-in memory allocation subsystem.\n**\n** ^The [sqlite3_memory_used()] routine returns the number of bytes\n** of memory currently outstanding (malloced but not freed).\n** ^The [sqlite3_memory_highwater()] routine returns the maximum\n** value of [sqlite3_memory_used()] since the high-water mark\n** was last reset. ^The values returned by [sqlite3_memory_used()] and\n** [sqlite3_memory_highwater()] include any overhead\n** added by SQLite in its implementation of [sqlite3_malloc()],\n** but not overhead added by the any underlying system library\n** routines that [sqlite3_malloc()] may call.\n**\n** ^The memory high-water mark is reset to the current value of\n** [sqlite3_memory_used()] if and only if the parameter to\n** [sqlite3_memory_highwater()] is true. ^The value returned\n** by [sqlite3_memory_highwater(1)] is the high-water mark\n** prior to the reset.\n*/\nSQLITE_API sqlite3_int64 sqlite3_memory_used(void);\nSQLITE_API sqlite3_int64 sqlite3_memory_highwater(int resetFlag);\n\n/*\n** CAPI3REF: Pseudo-Random Number Generator\n**\n** SQLite contains a high-quality pseudo-random number generator (PRNG) used to\n** select random [ROWID | ROWIDs] when inserting new records into a table that\n** already uses the largest possible [ROWID]. The PRNG is also used for\n** the built-in random() and randomblob() SQL functions. This interface allows\n** applications to access the same PRNG for other purposes.\n**\n** ^A call to this routine stores N bytes of randomness into buffer P.\n** ^The P parameter can be a NULL pointer.\n**\n** ^If this routine has not been previously called or if the previous\n** call had N less than one or a NULL pointer for P, then the PRNG is\n** seeded using randomness obtained from the xRandomness method of\n** the default [sqlite3_vfs] object.\n** ^If the previous call to this routine had an N of 1 or more and a\n** non-NULL P then the pseudo-randomness is generated\n** internally and without recourse to the [sqlite3_vfs] xRandomness\n** method.\n*/\nSQLITE_API void sqlite3_randomness(int N, void *P);\n\n/*\n** CAPI3REF: Compile-Time Authorization Callbacks\n** METHOD: sqlite3\n** KEYWORDS: {authorizer callback}\n**\n** ^This routine registers an authorizer callback with a particular\n** [database connection], supplied in the first argument.\n** ^The authorizer callback is invoked as SQL statements are being compiled\n** by [sqlite3_prepare()] or its variants [sqlite3_prepare_v2()],\n** [sqlite3_prepare_v3()], [sqlite3_prepare16()], [sqlite3_prepare16_v2()],\n** and [sqlite3_prepare16_v3()]. ^At various\n** points during the compilation process, as logic is being created\n** to perform various actions, the authorizer callback is invoked to\n** see if those actions are allowed. ^The authorizer callback should\n** return [SQLITE_OK] to allow the action, [SQLITE_IGNORE] to disallow the\n** specific action but allow the SQL statement to continue to be\n** compiled, or [SQLITE_DENY] to cause the entire SQL statement to be\n** rejected with an error. ^If the authorizer callback returns\n** any value other than [SQLITE_IGNORE], [SQLITE_OK], or [SQLITE_DENY]\n** then the [sqlite3_prepare_v2()] or equivalent call that triggered\n** the authorizer will fail with an error message.\n**\n** When the callback returns [SQLITE_OK], that means the operation\n** requested is ok. ^When the callback returns [SQLITE_DENY], the\n** [sqlite3_prepare_v2()] or equivalent call that triggered the\n** authorizer will fail with an error message explaining that\n** access is denied.\n**\n** ^The first parameter to the authorizer callback is a copy of the third\n** parameter to the sqlite3_set_authorizer() interface. ^The second parameter\n** to the callback is an integer [SQLITE_COPY | action code] that specifies\n** the particular action to be authorized. ^The third through sixth parameters\n** to the callback are either NULL pointers or zero-terminated strings\n** that contain additional details about the action to be authorized.\n** Applications must always be prepared to encounter a NULL pointer in any\n** of the third through the sixth parameters of the authorization callback.\n**\n** ^If the action code is [SQLITE_READ]\n** and the callback returns [SQLITE_IGNORE] then the\n** [prepared statement] statement is constructed to substitute\n** a NULL value in place of the table column that would have\n** been read if [SQLITE_OK] had been returned. The [SQLITE_IGNORE]\n** return can be used to deny an untrusted user access to individual\n** columns of a table.\n** ^When a table is referenced by a [SELECT] but no column values are\n** extracted from that table (for example in a query like\n** \"SELECT count(*) FROM tab\") then the [SQLITE_READ] authorizer callback\n** is invoked once for that table with a column name that is an empty string.\n** ^If the action code is [SQLITE_DELETE] and the callback returns\n** [SQLITE_IGNORE] then the [DELETE] operation proceeds but the\n** [truncate optimization] is disabled and all rows are deleted individually.\n**\n** An authorizer is used when [sqlite3_prepare | preparing]\n** SQL statements from an untrusted source, to ensure that the SQL statements\n** do not try to access data they are not allowed to see, or that they do not\n** try to execute malicious statements that damage the database. For\n** example, an application may allow a user to enter arbitrary\n** SQL queries for evaluation by a database. But the application does\n** not want the user to be able to make arbitrary changes to the\n** database. An authorizer could then be put in place while the\n** user-entered SQL is being [sqlite3_prepare | prepared] that\n** disallows everything except [SELECT] statements.\n**\n** Applications that need to process SQL from untrusted sources\n** might also consider lowering resource limits using [sqlite3_limit()]\n** and limiting database size using the [max_page_count] [PRAGMA]\n** in addition to using an authorizer.\n**\n** ^(Only a single authorizer can be in place on a database connection\n** at a time. Each call to sqlite3_set_authorizer overrides the\n** previous call.)^ ^Disable the authorizer by installing a NULL callback.\n** The authorizer is disabled by default.\n**\n** The authorizer callback must not do anything that will modify\n** the database connection that invoked the authorizer callback.\n** Note that [sqlite3_prepare_v2()] and [sqlite3_step()] both modify their\n** database connections for the meaning of \"modify\" in this paragraph.\n**\n** ^When [sqlite3_prepare_v2()] is used to prepare a statement, the\n** statement might be re-prepared during [sqlite3_step()] due to a\n** schema change. Hence, the application should ensure that the\n** correct authorizer callback remains in place during the [sqlite3_step()].\n**\n** ^Note that the authorizer callback is invoked only during\n** [sqlite3_prepare()] or its variants. Authorization is not\n** performed during statement evaluation in [sqlite3_step()], unless\n** as stated in the previous paragraph, sqlite3_step() invokes\n** sqlite3_prepare_v2() to reprepare a statement after a schema change.\n*/\nSQLITE_API int sqlite3_set_authorizer(\n sqlite3*,\n int (*xAuth)(void*,int,const char*,const char*,const char*,const char*),\n void *pUserData\n);\n\n/*\n** CAPI3REF: Authorizer Return Codes\n**\n** The [sqlite3_set_authorizer | authorizer callback function] must\n** return either [SQLITE_OK] or one of these two constants in order\n** to signal SQLite whether or not the action is permitted. See the\n** [sqlite3_set_authorizer | authorizer documentation] for additional\n** information.\n**\n** Note that SQLITE_IGNORE is also used as a [conflict resolution mode]\n** returned from the [sqlite3_vtab_on_conflict()] interface.\n*/\n#define SQLITE_DENY 1 /* Abort the SQL statement with an error */\n#define SQLITE_IGNORE 2 /* Don't allow access, but don't generate an error */\n\n/*\n** CAPI3REF: Authorizer Action Codes\n**\n** The [sqlite3_set_authorizer()] interface registers a callback function\n** that is invoked to authorize certain SQL statement actions. The\n** second parameter to the callback is an integer code that specifies\n** what action is being authorized. These are the integer action codes that\n** the authorizer callback may be passed.\n**\n** These action code values signify what kind of operation is to be\n** authorized. The 3rd and 4th parameters to the authorization\n** callback function will be parameters or NULL depending on which of these\n** codes is used as the second parameter. ^(The 5th parameter to the\n** authorizer callback is the name of the database (\"main\", \"temp\",\n** etc.) if applicable.)^ ^The 6th parameter to the authorizer callback\n** is the name of the inner-most trigger or view that is responsible for\n** the access attempt or NULL if this access attempt is directly from\n** top-level SQL code.\n*/\n/******************************************* 3rd ************ 4th ***********/\n#define SQLITE_CREATE_INDEX 1 /* Index Name Table Name */\n#define SQLITE_CREATE_TABLE 2 /* Table Name NULL */\n#define SQLITE_CREATE_TEMP_INDEX 3 /* Index Name Table Name */\n#define SQLITE_CREATE_TEMP_TABLE 4 /* Table Name NULL */\n#define SQLITE_CREATE_TEMP_TRIGGER 5 /* Trigger Name Table Name */\n#define SQLITE_CREATE_TEMP_VIEW 6 /* View Name NULL */\n#define SQLITE_CREATE_TRIGGER 7 /* Trigger Name Table Name */\n#define SQLITE_CREATE_VIEW 8 /* View Name NULL */\n#define SQLITE_DELETE 9 /* Table Name NULL */\n#define SQLITE_DROP_INDEX 10 /* Index Name Table Name */\n#define SQLITE_DROP_TABLE 11 /* Table Name NULL */\n#define SQLITE_DROP_TEMP_INDEX 12 /* Index Name Table Name */\n#define SQLITE_DROP_TEMP_TABLE 13 /* Table Name NULL */\n#define SQLITE_DROP_TEMP_TRIGGER 14 /* Trigger Name Table Name */\n#define SQLITE_DROP_TEMP_VIEW 15 /* View Name NULL */\n#define SQLITE_DROP_TRIGGER 16 /* Trigger Name Table Name */\n#define SQLITE_DROP_VIEW 17 /* View Name NULL */\n#define SQLITE_INSERT 18 /* Table Name NULL */\n#define SQLITE_PRAGMA 19 /* Pragma Name 1st arg or NULL */\n#define SQLITE_READ 20 /* Table Name Column Name */\n#define SQLITE_SELECT 21 /* NULL NULL */\n#define SQLITE_TRANSACTION 22 /* Operation NULL */\n#define SQLITE_UPDATE 23 /* Table Name Column Name */\n#define SQLITE_ATTACH 24 /* Filename NULL */\n#define SQLITE_DETACH 25 /* Database Name NULL */\n#define SQLITE_ALTER_TABLE 26 /* Database Name Table Name */\n#define SQLITE_REINDEX 27 /* Index Name NULL */\n#define SQLITE_ANALYZE 28 /* Table Name NULL */\n#define SQLITE_CREATE_VTABLE 29 /* Table Name Module Name */\n#define SQLITE_DROP_VTABLE 30 /* Table Name Module Name */\n#define SQLITE_FUNCTION 31 /* NULL Function Name */\n#define SQLITE_SAVEPOINT 32 /* Operation Savepoint Name */\n#define SQLITE_COPY 0 /* No longer used */\n#define SQLITE_RECURSIVE 33 /* NULL NULL */\n\n/*\n** CAPI3REF: Deprecated Tracing And Profiling Functions\n** DEPRECATED\n**\n** These routines are deprecated. Use the [sqlite3_trace_v2()] interface\n** instead of the routines described here.\n**\n** These routines register callback functions that can be used for\n** tracing and profiling the execution of SQL statements.\n**\n** ^The callback function registered by sqlite3_trace() is invoked at\n** various times when an SQL statement is being run by [sqlite3_step()].\n** ^The sqlite3_trace() callback is invoked with a UTF-8 rendering of the\n** SQL statement text as the statement first begins executing.\n** ^(Additional sqlite3_trace() callbacks might occur\n** as each triggered subprogram is entered. The callbacks for triggers\n** contain a UTF-8 SQL comment that identifies the trigger.)^\n**\n** The [SQLITE_TRACE_SIZE_LIMIT] compile-time option can be used to limit\n** the length of [bound parameter] expansion in the output of sqlite3_trace().\n**\n** ^The callback function registered by sqlite3_profile() is invoked\n** as each SQL statement finishes. ^The profile callback contains\n** the original statement text and an estimate of wall-clock time\n** of how long that statement took to run. ^The profile callback\n** time is in units of nanoseconds, however the current implementation\n** is only capable of millisecond resolution so the six least significant\n** digits in the time are meaningless. Future versions of SQLite\n** might provide greater resolution on the profiler callback. Invoking\n** either [sqlite3_trace()] or [sqlite3_trace_v2()] will cancel the\n** profile callback.\n*/\nSQLITE_API SQLITE_DEPRECATED void *sqlite3_trace(sqlite3*,\n void(*xTrace)(void*,const char*), void*);\nSQLITE_API SQLITE_DEPRECATED void *sqlite3_profile(sqlite3*,\n void(*xProfile)(void*,const char*,sqlite3_uint64), void*);\n\n/*\n** CAPI3REF: SQL Trace Event Codes\n** KEYWORDS: SQLITE_TRACE\n**\n** These constants identify classes of events that can be monitored\n** using the [sqlite3_trace_v2()] tracing logic. The M argument\n** to [sqlite3_trace_v2(D,M,X,P)] is an OR-ed combination of one or more of\n** the following constants. ^The first argument to the trace callback\n** is one of the following constants.\n**\n** New tracing constants may be added in future releases.\n**\n** ^A trace callback has four arguments: xCallback(T,C,P,X).\n** ^The T argument is one of the integer type codes above.\n** ^The C argument is a copy of the context pointer passed in as the\n** fourth argument to [sqlite3_trace_v2()].\n** The P and X arguments are pointers whose meanings depend on T.\n**\n**
\n** [[SQLITE_TRACE_STMT]]
SQLITE_TRACE_STMT
\n**
^An SQLITE_TRACE_STMT callback is invoked when a prepared statement\n** first begins running and possibly at other times during the\n** execution of the prepared statement, such as at the start of each\n** trigger subprogram. ^The P argument is a pointer to the\n** [prepared statement]. ^The X argument is a pointer to a string which\n** is the unexpanded SQL text of the prepared statement or an SQL comment\n** that indicates the invocation of a trigger. ^The callback can compute\n** the same text that would have been returned by the legacy [sqlite3_trace()]\n** interface by using the X argument when X begins with \"--\" and invoking\n** [sqlite3_expanded_sql(P)] otherwise.\n**\n** [[SQLITE_TRACE_PROFILE]]
SQLITE_TRACE_PROFILE
\n**
^An SQLITE_TRACE_PROFILE callback provides approximately the same\n** information as is provided by the [sqlite3_profile()] callback.\n** ^The P argument is a pointer to the [prepared statement] and the\n** X argument points to a 64-bit integer which is approximately\n** the number of nanoseconds that the prepared statement took to run.\n** ^The SQLITE_TRACE_PROFILE callback is invoked when the statement finishes.\n**\n** [[SQLITE_TRACE_ROW]]
SQLITE_TRACE_ROW
\n**
^An SQLITE_TRACE_ROW callback is invoked whenever a prepared\n** statement generates a single row of result.\n** ^The P argument is a pointer to the [prepared statement] and the\n** X argument is unused.\n**\n** [[SQLITE_TRACE_CLOSE]]
SQLITE_TRACE_CLOSE
\n**
^An SQLITE_TRACE_CLOSE callback is invoked when a database\n** connection closes.\n** ^The P argument is a pointer to the [database connection] object\n** and the X argument is unused.\n**
\n*/\n#define SQLITE_TRACE_STMT 0x01\n#define SQLITE_TRACE_PROFILE 0x02\n#define SQLITE_TRACE_ROW 0x04\n#define SQLITE_TRACE_CLOSE 0x08\n\n/*\n** CAPI3REF: SQL Trace Hook\n** METHOD: sqlite3\n**\n** ^The sqlite3_trace_v2(D,M,X,P) interface registers a trace callback\n** function X against [database connection] D, using property mask M\n** and context pointer P. ^If the X callback is\n** NULL or if the M mask is zero, then tracing is disabled. The\n** M argument should be the bitwise OR-ed combination of\n** zero or more [SQLITE_TRACE] constants.\n**\n** ^Each call to either sqlite3_trace(D,X,P) or sqlite3_trace_v2(D,M,X,P)\n** overrides (cancels) all prior calls to sqlite3_trace(D,X,P) or\n** sqlite3_trace_v2(D,M,X,P) for the [database connection] D. Each\n** database connection may have at most one trace callback.\n**\n** ^The X callback is invoked whenever any of the events identified by\n** mask M occur. ^The integer return value from the callback is currently\n** ignored, though this may change in future releases. Callback\n** implementations should return zero to ensure future compatibility.\n**\n** ^A trace callback is invoked with four arguments: callback(T,C,P,X).\n** ^The T argument is one of the [SQLITE_TRACE]\n** constants to indicate why the callback was invoked.\n** ^The C argument is a copy of the context pointer.\n** The P and X arguments are pointers whose meanings depend on T.\n**\n** The sqlite3_trace_v2() interface is intended to replace the legacy\n** interfaces [sqlite3_trace()] and [sqlite3_profile()], both of which\n** are deprecated.\n*/\nSQLITE_API int sqlite3_trace_v2(\n sqlite3*,\n unsigned uMask,\n int(*xCallback)(unsigned,void*,void*,void*),\n void *pCtx\n);\n\n/*\n** CAPI3REF: Query Progress Callbacks\n** METHOD: sqlite3\n**\n** ^The sqlite3_progress_handler(D,N,X,P) interface causes the callback\n** function X to be invoked periodically during long running calls to\n** [sqlite3_step()] and [sqlite3_prepare()] and similar for\n** database connection D. An example use for this\n** interface is to keep a GUI updated during a large query.\n**\n** ^The parameter P is passed through as the only parameter to the\n** callback function X. ^The parameter N is the approximate number of\n** [virtual machine instructions] that are evaluated between successive\n** invocations of the callback X. ^If N is less than one then the progress\n** handler is disabled.\n**\n** ^Only a single progress handler may be defined at one time per\n** [database connection]; setting a new progress handler cancels the\n** old one. ^Setting parameter X to NULL disables the progress handler.\n** ^The progress handler is also disabled by setting N to a value less\n** than 1.\n**\n** ^If the progress callback returns non-zero, the operation is\n** interrupted. This feature can be used to implement a\n** \"Cancel\" button on a GUI progress dialog box.\n**\n** The progress handler callback must not do anything that will modify\n** the database connection that invoked the progress handler.\n** Note that [sqlite3_prepare_v2()] and [sqlite3_step()] both modify their\n** database connections for the meaning of \"modify\" in this paragraph.\n**\n** The progress handler callback would originally only be invoked from the\n** bytecode engine. It still might be invoked during [sqlite3_prepare()]\n** and similar because those routines might force a reparse of the schema\n** which involves running the bytecode engine. However, beginning with\n** SQLite version 3.41.0, the progress handler callback might also be\n** invoked directly from [sqlite3_prepare()] while analyzing and generating\n** code for complex queries.\n*/\nSQLITE_API void sqlite3_progress_handler(sqlite3*, int, int(*)(void*), void*);\n\n/*\n** CAPI3REF: Opening A New Database Connection\n** CONSTRUCTOR: sqlite3\n**\n** ^These routines open an SQLite database file as specified by the\n** filename argument. ^The filename argument is interpreted as UTF-8 for\n** sqlite3_open() and sqlite3_open_v2() and as UTF-16 in the native byte\n** order for sqlite3_open16(). ^(A [database connection] handle is usually\n** returned in *ppDb, even if an error occurs. The only exception is that\n** if SQLite is unable to allocate memory to hold the [sqlite3] object,\n** a NULL will be written into *ppDb instead of a pointer to the [sqlite3]\n** object.)^ ^(If the database is opened (and/or created) successfully, then\n** [SQLITE_OK] is returned. Otherwise an [error code] is returned.)^ ^The\n** [sqlite3_errmsg()] or [sqlite3_errmsg16()] routines can be used to obtain\n** an English language description of the error following a failure of any\n** of the sqlite3_open() routines.\n**\n** ^The default encoding will be UTF-8 for databases created using\n** sqlite3_open() or sqlite3_open_v2(). ^The default encoding for databases\n** created using sqlite3_open16() will be UTF-16 in the native byte order.\n**\n** Whether or not an error occurs when it is opened, resources\n** associated with the [database connection] handle should be released by\n** passing it to [sqlite3_close()] when it is no longer required.\n**\n** The sqlite3_open_v2() interface works like sqlite3_open()\n** except that it accepts two additional parameters for additional control\n** over the new database connection. ^(The flags parameter to\n** sqlite3_open_v2() must include, at a minimum, one of the following\n** three flag combinations:)^\n**\n**
\n** ^(
[SQLITE_OPEN_READONLY]
\n**
The database is opened in read-only mode. If the database does\n** not already exist, an error is returned.
)^\n**\n** ^(
[SQLITE_OPEN_READWRITE]
\n**
The database is opened for reading and writing if possible, or\n** reading only if the file is write protected by the operating\n** system. In either case the database must already exist, otherwise\n** an error is returned. For historical reasons, if opening in\n** read-write mode fails due to OS-level permissions, an attempt is\n** made to open it in read-only mode. [sqlite3_db_readonly()] can be\n** used to determine whether the database is actually\n** read-write.
)^\n**\n** ^(
[SQLITE_OPEN_READWRITE] | [SQLITE_OPEN_CREATE]
\n**
The database is opened for reading and writing, and is created if\n** it does not already exist. This is the behavior that is always used for\n** sqlite3_open() and sqlite3_open16().
)^\n**
\n**\n** In addition to the required flags, the following optional flags are\n** also supported:\n**\n**
\n** ^(
[SQLITE_OPEN_URI]
\n**
The filename can be interpreted as a URI if this flag is set.
)^\n**\n** ^(
[SQLITE_OPEN_MEMORY]
\n**
The database will be opened as an in-memory database. The database\n** is named by the \"filename\" argument for the purposes of cache-sharing,\n** if shared cache mode is enabled, but the \"filename\" is otherwise ignored.\n**
)^\n**\n** ^(
[SQLITE_OPEN_NOMUTEX]
\n**
The new database connection will use the \"multi-thread\"\n** [threading mode].)^ This means that separate threads are allowed\n** to use SQLite at the same time, as long as each thread is using\n** a different [database connection].\n**\n** ^(
[SQLITE_OPEN_FULLMUTEX]
\n**
The new database connection will use the \"serialized\"\n** [threading mode].)^ This means the multiple threads can safely\n** attempt to use the same database connection at the same time.\n** (Mutexes will block any actual concurrency, but in this mode\n** there is no harm in trying.)\n**\n** ^(
[SQLITE_OPEN_SHAREDCACHE]
\n**
The database is opened [shared cache] enabled, overriding\n** the default shared cache setting provided by\n** [sqlite3_enable_shared_cache()].)^\n** The [use of shared cache mode is discouraged] and hence shared cache\n** capabilities may be omitted from many builds of SQLite. In such cases,\n** this option is a no-op.\n**\n** ^(
[SQLITE_OPEN_PRIVATECACHE]
\n**
The database is opened [shared cache] disabled, overriding\n** the default shared cache setting provided by\n** [sqlite3_enable_shared_cache()].)^\n**\n** [[OPEN_EXRESCODE]] ^(
[SQLITE_OPEN_EXRESCODE]
\n**
The database connection comes up in \"extended result code mode\".\n** In other words, the database behaves as if\n** [sqlite3_extended_result_codes(db,1)] were called on the database\n** connection as soon as the connection is created. In addition to setting\n** the extended result code mode, this flag also causes [sqlite3_open_v2()]\n** to return an extended result code.
\n**\n** [[OPEN_NOFOLLOW]] ^(
[SQLITE_OPEN_NOFOLLOW]
\n**
The database filename is not allowed to contain a symbolic link
\n**
)^\n**\n** If the 3rd parameter to sqlite3_open_v2() is not one of the\n** required combinations shown above optionally combined with other\n** [SQLITE_OPEN_READONLY | SQLITE_OPEN_* bits]\n** then the behavior is undefined. Historic versions of SQLite\n** have silently ignored surplus bits in the flags parameter to\n** sqlite3_open_v2(), however that behavior might not be carried through\n** into future versions of SQLite and so applications should not rely\n** upon it. Note in particular that the SQLITE_OPEN_EXCLUSIVE flag is a no-op\n** for sqlite3_open_v2(). The SQLITE_OPEN_EXCLUSIVE does *not* cause\n** the open to fail if the database already exists. The SQLITE_OPEN_EXCLUSIVE\n** flag is intended for use by the [sqlite3_vfs|VFS interface] only, and not\n** by sqlite3_open_v2().\n**\n** ^The fourth parameter to sqlite3_open_v2() is the name of the\n** [sqlite3_vfs] object that defines the operating system interface that\n** the new database connection should use. ^If the fourth parameter is\n** a NULL pointer then the default [sqlite3_vfs] object is used.\n**\n** ^If the filename is \":memory:\", then a private, temporary in-memory database\n** is created for the connection. ^This in-memory database will vanish when\n** the database connection is closed. Future versions of SQLite might\n** make use of additional special filenames that begin with the \":\" character.\n** It is recommended that when a database filename actually does begin with\n** a \":\" character you should prefix the filename with a pathname such as\n** \"./\" to avoid ambiguity.\n**\n** ^If the filename is an empty string, then a private, temporary\n** on-disk database will be created. ^This private database will be\n** automatically deleted as soon as the database connection is closed.\n**\n** [[URI filenames in sqlite3_open()]]

URI Filenames

\n**\n** ^If [URI filename] interpretation is enabled, and the filename argument\n** begins with \"file:\", then the filename is interpreted as a URI. ^URI\n** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is\n** set in the third argument to sqlite3_open_v2(), or if it has\n** been enabled globally using the [SQLITE_CONFIG_URI] option with the\n** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option.\n** URI filename interpretation is turned off\n** by default, but future releases of SQLite might enable URI filename\n** interpretation by default. See \"[URI filenames]\" for additional\n** information.\n**\n** URI filenames are parsed according to RFC 3986. ^If the URI contains an\n** authority, then it must be either an empty string or the string\n** \"localhost\". ^If the authority is not an empty string or \"localhost\", an\n** error is returned to the caller. ^The fragment component of a URI, if\n** present, is ignored.\n**\n** ^SQLite uses the path component of the URI as the name of the disk file\n** which contains the database. ^If the path begins with a '/' character,\n** then it is interpreted as an absolute path. ^If the path does not begin\n** with a '/' (meaning that the authority section is omitted from the URI)\n** then the path is interpreted as a relative path.\n** ^(On windows, the first component of an absolute path\n** is a drive specification (e.g. \"C:\").)^\n**\n** [[core URI query parameters]]\n** The query component of a URI may contain parameters that are interpreted\n** either by SQLite itself, or by a [VFS | custom VFS implementation].\n** SQLite and its built-in [VFSes] interpret the\n** following query parameters:\n**\n**
    \n**
  • vfs: ^The \"vfs\" parameter may be used to specify the name of\n** a VFS object that provides the operating system interface that should\n** be used to access the database file on disk. ^If this option is set to\n** an empty string the default VFS object is used. ^Specifying an unknown\n** VFS is an error. ^If sqlite3_open_v2() is used and the vfs option is\n** present, then the VFS specified by the option takes precedence over\n** the value passed as the fourth parameter to sqlite3_open_v2().\n**\n**
  • mode: ^(The mode parameter may be set to either \"ro\", \"rw\",\n** \"rwc\", or \"memory\". Attempting to set it to any other value is\n** an error)^.\n** ^If \"ro\" is specified, then the database is opened for read-only\n** access, just as if the [SQLITE_OPEN_READONLY] flag had been set in the\n** third argument to sqlite3_open_v2(). ^If the mode option is set to\n** \"rw\", then the database is opened for read-write (but not create)\n** access, as if SQLITE_OPEN_READWRITE (but not SQLITE_OPEN_CREATE) had\n** been set. ^Value \"rwc\" is equivalent to setting both\n** SQLITE_OPEN_READWRITE and SQLITE_OPEN_CREATE. ^If the mode option is\n** set to \"memory\" then a pure [in-memory database] that never reads\n** or writes from disk is used. ^It is an error to specify a value for\n** the mode parameter that is less restrictive than that specified by\n** the flags passed in the third parameter to sqlite3_open_v2().\n**\n**
  • cache: ^The cache parameter may be set to either \"shared\" or\n** \"private\". ^Setting it to \"shared\" is equivalent to setting the\n** SQLITE_OPEN_SHAREDCACHE bit in the flags argument passed to\n** sqlite3_open_v2(). ^Setting the cache parameter to \"private\" is\n** equivalent to setting the SQLITE_OPEN_PRIVATECACHE bit.\n** ^If sqlite3_open_v2() is used and the \"cache\" parameter is present in\n** a URI filename, its value overrides any behavior requested by setting\n** SQLITE_OPEN_PRIVATECACHE or SQLITE_OPEN_SHAREDCACHE flag.\n**\n**
  • psow: ^The psow parameter indicates whether or not the\n** [powersafe overwrite] property does or does not apply to the\n** storage media on which the database file resides.\n**\n**
  • nolock: ^The nolock parameter is a boolean query parameter\n** which if set disables file locking in rollback journal modes. This\n** is useful for accessing a database on a filesystem that does not\n** support locking. Caution: Database corruption might result if two\n** or more processes write to the same database and any one of those\n** processes uses nolock=1.\n**\n**
  • immutable: ^The immutable parameter is a boolean query\n** parameter that indicates that the database file is stored on\n** read-only media. ^When immutable is set, SQLite assumes that the\n** database file cannot be changed, even by a process with higher\n** privilege, and so the database is opened read-only and all locking\n** and change detection is disabled. Caution: Setting the immutable\n** property on a database file that does in fact change can result\n** in incorrect query results and/or [SQLITE_CORRUPT] errors.\n** See also: [SQLITE_IOCAP_IMMUTABLE].\n**\n**
\n**\n** ^Specifying an unknown parameter in the query component of a URI is not an\n** error. Future versions of SQLite might understand additional query\n** parameters. See \"[query parameters with special meaning to SQLite]\" for\n** additional information.\n**\n** [[URI filename examples]]

URI filename examples

\n**\n** \n**
URI filenames Results\n**
file:data.db \n** Open the file \"data.db\" in the current directory.\n**
file:/home/fred/data.db
\n** file:///home/fred/data.db
\n** file://localhost/home/fred/data.db
\n** Open the database file \"/home/fred/data.db\".\n**
file://darkstar/home/fred/data.db \n** An error. \"darkstar\" is not a recognized authority.\n**
\n** file:///C:/Documents%20and%20Settings/fred/Desktop/data.db\n** Windows only: Open the file \"data.db\" on fred's desktop on drive\n** C:. Note that the %20 escaping in this example is not strictly\n** necessary - space characters can be used literally\n** in URI filenames.\n**
file:data.db?mode=ro&cache=private \n** Open file \"data.db\" in the current directory for read-only access.\n** Regardless of whether or not shared-cache mode is enabled by\n** default, use a private cache.\n**
file:/home/fred/data.db?vfs=unix-dotfile \n** Open file \"/home/fred/data.db\". Use the special VFS \"unix-dotfile\"\n** that uses dot-files in place of posix advisory locking.\n**
file:data.db?mode=readonly \n** An error. \"readonly\" is not a valid option for the \"mode\" parameter.\n** Use \"ro\" instead: \"file:data.db?mode=ro\".\n**
\n**\n** ^URI hexadecimal escape sequences (%HH) are supported within the path and\n** query components of a URI. A hexadecimal escape sequence consists of a\n** percent sign - \"%\" - followed by exactly two hexadecimal digits\n** specifying an octet value. ^Before the path or query components of a\n** URI filename are interpreted, they are encoded using UTF-8 and all\n** hexadecimal escape sequences replaced by a single byte containing the\n** corresponding octet. If this process generates an invalid UTF-8 encoding,\n** the results are undefined.\n**\n** Note to Windows users: The encoding used for the filename argument\n** of sqlite3_open() and sqlite3_open_v2() must be UTF-8, not whatever\n** codepage is currently defined. Filenames containing international\n** characters must be converted to UTF-8 prior to passing them into\n** sqlite3_open() or sqlite3_open_v2().\n**\n** Note to Windows Runtime users: The temporary directory must be set\n** prior to calling sqlite3_open() or sqlite3_open_v2(). Otherwise, various\n** features that require the use of temporary files may fail.\n**\n** See also: [sqlite3_temp_directory]\n*/\nSQLITE_API int sqlite3_open(\n const char *filename, /* Database filename (UTF-8) */\n sqlite3 **ppDb /* OUT: SQLite db handle */\n);\nSQLITE_API int sqlite3_open16(\n const void *filename, /* Database filename (UTF-16) */\n sqlite3 **ppDb /* OUT: SQLite db handle */\n);\nSQLITE_API int sqlite3_open_v2(\n const char *filename, /* Database filename (UTF-8) */\n sqlite3 **ppDb, /* OUT: SQLite db handle */\n int flags, /* Flags */\n const char *zVfs /* Name of VFS module to use */\n);\n\n/*\n** CAPI3REF: Obtain Values For URI Parameters\n**\n** These are utility routines, useful to [VFS|custom VFS implementations],\n** that check if a database file was a URI that contained a specific query\n** parameter, and if so obtains the value of that query parameter.\n**\n** The first parameter to these interfaces (hereafter referred to\n** as F) must be one of:\n**
    \n**
  • A database filename pointer created by the SQLite core and\n** passed into the xOpen() method of a VFS implementation, or\n**
  • A filename obtained from [sqlite3_db_filename()], or\n**
  • A new filename constructed using [sqlite3_create_filename()].\n**
\n** If the F parameter is not one of the above, then the behavior is\n** undefined and probably undesirable. Older versions of SQLite were\n** more tolerant of invalid F parameters than newer versions.\n**\n** If F is a suitable filename (as described in the previous paragraph)\n** and if P is the name of the query parameter, then\n** sqlite3_uri_parameter(F,P) returns the value of the P\n** parameter if it exists or a NULL pointer if P does not appear as a\n** query parameter on F. If P is a query parameter of F and it\n** has no explicit value, then sqlite3_uri_parameter(F,P) returns\n** a pointer to an empty string.\n**\n** The sqlite3_uri_boolean(F,P,B) routine assumes that P is a boolean\n** parameter and returns true (1) or false (0) according to the value\n** of P. The sqlite3_uri_boolean(F,P,B) routine returns true (1) if the\n** value of query parameter P is one of \"yes\", \"true\", or \"on\" in any\n** case or if the value begins with a non-zero number. The\n** sqlite3_uri_boolean(F,P,B) routines returns false (0) if the value of\n** query parameter P is one of \"no\", \"false\", or \"off\" in any case or\n** if the value begins with a numeric zero. If P is not a query\n** parameter on F or if the value of P does not match any of the\n** above, then sqlite3_uri_boolean(F,P,B) returns (B!=0).\n**\n** The sqlite3_uri_int64(F,P,D) routine converts the value of P into a\n** 64-bit signed integer and returns that integer, or D if P does not\n** exist. If the value of P is something other than an integer, then\n** zero is returned.\n**\n** The sqlite3_uri_key(F,N) returns a pointer to the name (not\n** the value) of the N-th query parameter for filename F, or a NULL\n** pointer if N is less than zero or greater than the number of query\n** parameters minus 1. The N value is zero-based so N should be 0 to obtain\n** the name of the first query parameter, 1 for the second parameter, and\n** so forth.\n**\n** If F is a NULL pointer, then sqlite3_uri_parameter(F,P) returns NULL and\n** sqlite3_uri_boolean(F,P,B) returns B. If F is not a NULL pointer and\n** is not a database file pathname pointer that the SQLite core passed\n** into the xOpen VFS method, then the behavior of this routine is undefined\n** and probably undesirable.\n**\n** Beginning with SQLite [version 3.31.0] ([dateof:3.31.0]) the input F\n** parameter can also be the name of a rollback journal file or WAL file\n** in addition to the main database file. Prior to version 3.31.0, these\n** routines would only work if F was the name of the main database file.\n** When the F parameter is the name of the rollback journal or WAL file,\n** it has access to all the same query parameters as were found on the\n** main database file.\n**\n** See the [URI filename] documentation for additional information.\n*/\nSQLITE_API const char *sqlite3_uri_parameter(sqlite3_filename z, const char *zParam);\nSQLITE_API int sqlite3_uri_boolean(sqlite3_filename z, const char *zParam, int bDefault);\nSQLITE_API sqlite3_int64 sqlite3_uri_int64(sqlite3_filename, const char*, sqlite3_int64);\nSQLITE_API const char *sqlite3_uri_key(sqlite3_filename z, int N);\n\n/*\n** CAPI3REF: Translate filenames\n**\n** These routines are available to [VFS|custom VFS implementations] for\n** translating filenames between the main database file, the journal file,\n** and the WAL file.\n**\n** If F is the name of an sqlite database file, journal file, or WAL file\n** passed by the SQLite core into the VFS, then sqlite3_filename_database(F)\n** returns the name of the corresponding database file.\n**\n** If F is the name of an sqlite database file, journal file, or WAL file\n** passed by the SQLite core into the VFS, or if F is a database filename\n** obtained from [sqlite3_db_filename()], then sqlite3_filename_journal(F)\n** returns the name of the corresponding rollback journal file.\n**\n** If F is the name of an sqlite database file, journal file, or WAL file\n** that was passed by the SQLite core into the VFS, or if F is a database\n** filename obtained from [sqlite3_db_filename()], then\n** sqlite3_filename_wal(F) returns the name of the corresponding\n** WAL file.\n**\n** In all of the above, if F is not the name of a database, journal or WAL\n** filename passed into the VFS from the SQLite core and F is not the\n** return value from [sqlite3_db_filename()], then the result is\n** undefined and is likely a memory access violation.\n*/\nSQLITE_API const char *sqlite3_filename_database(sqlite3_filename);\nSQLITE_API const char *sqlite3_filename_journal(sqlite3_filename);\nSQLITE_API const char *sqlite3_filename_wal(sqlite3_filename);\n\n/*\n** CAPI3REF: Database File Corresponding To A Journal\n**\n** ^If X is the name of a rollback or WAL-mode journal file that is\n** passed into the xOpen method of [sqlite3_vfs], then\n** sqlite3_database_file_object(X) returns a pointer to the [sqlite3_file]\n** object that represents the main database file.\n**\n** This routine is intended for use in custom [VFS] implementations\n** only. It is not a general-purpose interface.\n** The argument sqlite3_file_object(X) must be a filename pointer that\n** has been passed into [sqlite3_vfs].xOpen method where the\n** flags parameter to xOpen contains one of the bits\n** [SQLITE_OPEN_MAIN_JOURNAL] or [SQLITE_OPEN_WAL]. Any other use\n** of this routine results in undefined and probably undesirable\n** behavior.\n*/\nSQLITE_API sqlite3_file *sqlite3_database_file_object(const char*);\n\n/*\n** CAPI3REF: Create and Destroy VFS Filenames\n**\n** These interfaces are provided for use by [VFS shim] implementations and\n** are not useful outside of that context.\n**\n** The sqlite3_create_filename(D,J,W,N,P) allocates memory to hold a version of\n** database filename D with corresponding journal file J and WAL file W and\n** an array P of N URI Key/Value pairs. The result from\n** sqlite3_create_filename(D,J,W,N,P) is a pointer to a database filename that\n** is safe to pass to routines like:\n**
    \n**
  • [sqlite3_uri_parameter()],\n**
  • [sqlite3_uri_boolean()],\n**
  • [sqlite3_uri_int64()],\n**
  • [sqlite3_uri_key()],\n**
  • [sqlite3_filename_database()],\n**
  • [sqlite3_filename_journal()], or\n**
  • [sqlite3_filename_wal()].\n**
\n** If a memory allocation error occurs, sqlite3_create_filename() might\n** return a NULL pointer. The memory obtained from sqlite3_create_filename(X)\n** must be released by a corresponding call to sqlite3_free_filename(Y).\n**\n** The P parameter in sqlite3_create_filename(D,J,W,N,P) should be an array\n** of 2*N pointers to strings. Each pair of pointers in this array corresponds\n** to a key and value for a query parameter. The P parameter may be a NULL\n** pointer if N is zero. None of the 2*N pointers in the P array may be\n** NULL pointers and key pointers should not be empty strings.\n** None of the D, J, or W parameters to sqlite3_create_filename(D,J,W,N,P) may\n** be NULL pointers, though they can be empty strings.\n**\n** The sqlite3_free_filename(Y) routine releases a memory allocation\n** previously obtained from sqlite3_create_filename(). Invoking\n** sqlite3_free_filename(Y) where Y is a NULL pointer is a harmless no-op.\n**\n** If the Y parameter to sqlite3_free_filename(Y) is anything other\n** than a NULL pointer or a pointer previously acquired from\n** sqlite3_create_filename(), then bad things such as heap\n** corruption or segfaults may occur. The value Y should not be\n** used again after sqlite3_free_filename(Y) has been called. This means\n** that if the [sqlite3_vfs.xOpen()] method of a VFS has been called using Y,\n** then the corresponding [sqlite3_module.xClose() method should also be\n** invoked prior to calling sqlite3_free_filename(Y).\n*/\nSQLITE_API sqlite3_filename sqlite3_create_filename(\n const char *zDatabase,\n const char *zJournal,\n const char *zWal,\n int nParam,\n const char **azParam\n);\nSQLITE_API void sqlite3_free_filename(sqlite3_filename);\n\n/*\n** CAPI3REF: Error Codes And Messages\n** METHOD: sqlite3\n**\n** ^If the most recent sqlite3_* API call associated with\n** [database connection] D failed, then the sqlite3_errcode(D) interface\n** returns the numeric [result code] or [extended result code] for that\n** API call.\n** ^The sqlite3_extended_errcode()\n** interface is the same except that it always returns the\n** [extended result code] even when extended result codes are\n** disabled.\n**\n** The values returned by sqlite3_errcode() and/or\n** sqlite3_extended_errcode() might change with each API call.\n** Except, there are some interfaces that are guaranteed to never\n** change the value of the error code. The error-code preserving\n** interfaces include the following:\n**\n**
    \n**
  • sqlite3_errcode()\n**
  • sqlite3_extended_errcode()\n**
  • sqlite3_errmsg()\n**
  • sqlite3_errmsg16()\n**
  • sqlite3_error_offset()\n**
\n**\n** ^The sqlite3_errmsg() and sqlite3_errmsg16() return English-language\n** text that describes the error, as either UTF-8 or UTF-16 respectively,\n** or NULL if no error message is available.\n** (See how SQLite handles [invalid UTF] for exceptions to this rule.)\n** ^(Memory to hold the error message string is managed internally.\n** The application does not need to worry about freeing the result.\n** However, the error string might be overwritten or deallocated by\n** subsequent calls to other SQLite interface functions.)^\n**\n** ^The sqlite3_errstr(E) interface returns the English-language text\n** that describes the [result code] E, as UTF-8, or NULL if E is not an\n** result code for which a text error message is available.\n** ^(Memory to hold the error message string is managed internally\n** and must not be freed by the application)^.\n**\n** ^If the most recent error references a specific token in the input\n** SQL, the sqlite3_error_offset() interface returns the byte offset\n** of the start of that token. ^The byte offset returned by\n** sqlite3_error_offset() assumes that the input SQL is UTF8.\n** ^If the most recent error does not reference a specific token in the input\n** SQL, then the sqlite3_error_offset() function returns -1.\n**\n** When the serialized [threading mode] is in use, it might be the\n** case that a second error occurs on a separate thread in between\n** the time of the first error and the call to these interfaces.\n** When that happens, the second error will be reported since these\n** interfaces always report the most recent result. To avoid\n** this, each thread can obtain exclusive use of the [database connection] D\n** by invoking [sqlite3_mutex_enter]([sqlite3_db_mutex](D)) before beginning\n** to use D and invoking [sqlite3_mutex_leave]([sqlite3_db_mutex](D)) after\n** all calls to the interfaces listed here are completed.\n**\n** If an interface fails with SQLITE_MISUSE, that means the interface\n** was invoked incorrectly by the application. In that case, the\n** error code and message may or may not be set.\n*/\nSQLITE_API int sqlite3_errcode(sqlite3 *db);\nSQLITE_API int sqlite3_extended_errcode(sqlite3 *db);\nSQLITE_API const char *sqlite3_errmsg(sqlite3*);\nSQLITE_API const void *sqlite3_errmsg16(sqlite3*);\nSQLITE_API const char *sqlite3_errstr(int);\nSQLITE_API int sqlite3_error_offset(sqlite3 *db);\n\n/*\n** CAPI3REF: Prepared Statement Object\n** KEYWORDS: {prepared statement} {prepared statements}\n**\n** An instance of this object represents a single SQL statement that\n** has been compiled into binary form and is ready to be evaluated.\n**\n** Think of each SQL statement as a separate computer program. The\n** original SQL text is source code. A prepared statement object\n** is the compiled object code. All SQL must be converted into a\n** prepared statement before it can be run.\n**\n** The life-cycle of a prepared statement object usually goes like this:\n**\n**
    \n**
  1. Create the prepared statement object using [sqlite3_prepare_v2()].\n**
  2. Bind values to [parameters] using the sqlite3_bind_*()\n** interfaces.\n**
  3. Run the SQL by calling [sqlite3_step()] one or more times.\n**
  4. Reset the prepared statement using [sqlite3_reset()] then go back\n** to step 2. Do this zero or more times.\n**
  5. Destroy the object using [sqlite3_finalize()].\n**
\n*/\ntypedef struct sqlite3_stmt sqlite3_stmt;\n\n/*\n** CAPI3REF: Run-time Limits\n** METHOD: sqlite3\n**\n** ^(This interface allows the size of various constructs to be limited\n** on a connection by connection basis. The first parameter is the\n** [database connection] whose limit is to be set or queried. The\n** second parameter is one of the [limit categories] that define a\n** class of constructs to be size limited. The third parameter is the\n** new limit for that construct.)^\n**\n** ^If the new limit is a negative number, the limit is unchanged.\n** ^(For each limit category SQLITE_LIMIT_NAME there is a\n** [limits | hard upper bound]\n** set at compile-time by a C preprocessor macro called\n** [limits | SQLITE_MAX_NAME].\n** (The \"_LIMIT_\" in the name is changed to \"_MAX_\".))^\n** ^Attempts to increase a limit above its hard upper bound are\n** silently truncated to the hard upper bound.\n**\n** ^Regardless of whether or not the limit was changed, the\n** [sqlite3_limit()] interface returns the prior value of the limit.\n** ^Hence, to find the current value of a limit without changing it,\n** simply invoke this interface with the third parameter set to -1.\n**\n** Run-time limits are intended for use in applications that manage\n** both their own internal database and also databases that are controlled\n** by untrusted external sources. An example application might be a\n** web browser that has its own databases for storing history and\n** separate databases controlled by JavaScript applications downloaded\n** off the Internet. The internal databases can be given the\n** large, default limits. Databases managed by external sources can\n** be given much smaller limits designed to prevent a denial of service\n** attack. Developers might also want to use the [sqlite3_set_authorizer()]\n** interface to further control untrusted SQL. The size of the database\n** created by an untrusted script can be contained using the\n** [max_page_count] [PRAGMA].\n**\n** New run-time limit categories may be added in future releases.\n*/\nSQLITE_API int sqlite3_limit(sqlite3*, int id, int newVal);\n\n/*\n** CAPI3REF: Run-Time Limit Categories\n** KEYWORDS: {limit category} {*limit categories}\n**\n** These constants define various performance limits\n** that can be lowered at run-time using [sqlite3_limit()].\n** The synopsis of the meanings of the various limits is shown below.\n** Additional information is available at [limits | Limits in SQLite].\n**\n**
\n** [[SQLITE_LIMIT_LENGTH]] ^(
SQLITE_LIMIT_LENGTH
\n**
The maximum size of any string or BLOB or table row, in bytes.
)^\n**\n** [[SQLITE_LIMIT_SQL_LENGTH]] ^(
SQLITE_LIMIT_SQL_LENGTH
\n**
The maximum length of an SQL statement, in bytes.
)^\n**\n** [[SQLITE_LIMIT_COLUMN]] ^(
SQLITE_LIMIT_COLUMN
\n**
The maximum number of columns in a table definition or in the\n** result set of a [SELECT] or the maximum number of columns in an index\n** or in an ORDER BY or GROUP BY clause.
)^\n**\n** [[SQLITE_LIMIT_EXPR_DEPTH]] ^(
SQLITE_LIMIT_EXPR_DEPTH
\n**
The maximum depth of the parse tree on any expression.
)^\n**\n** [[SQLITE_LIMIT_COMPOUND_SELECT]] ^(
SQLITE_LIMIT_COMPOUND_SELECT
\n**
The maximum number of terms in a compound SELECT statement.
)^\n**\n** [[SQLITE_LIMIT_VDBE_OP]] ^(
SQLITE_LIMIT_VDBE_OP
\n**
The maximum number of instructions in a virtual machine program\n** used to implement an SQL statement. If [sqlite3_prepare_v2()] or\n** the equivalent tries to allocate space for more than this many opcodes\n** in a single prepared statement, an SQLITE_NOMEM error is returned.
)^\n**\n** [[SQLITE_LIMIT_FUNCTION_ARG]] ^(
SQLITE_LIMIT_FUNCTION_ARG
\n**
The maximum number of arguments on a function.
)^\n**\n** [[SQLITE_LIMIT_ATTACHED]] ^(
SQLITE_LIMIT_ATTACHED
\n**
The maximum number of [ATTACH | attached databases].)^
\n**\n** [[SQLITE_LIMIT_LIKE_PATTERN_LENGTH]]\n** ^(
SQLITE_LIMIT_LIKE_PATTERN_LENGTH
\n**
The maximum length of the pattern argument to the [LIKE] or\n** [GLOB] operators.
)^\n**\n** [[SQLITE_LIMIT_VARIABLE_NUMBER]]\n** ^(
SQLITE_LIMIT_VARIABLE_NUMBER
\n**
The maximum index number of any [parameter] in an SQL statement.)^\n**\n** [[SQLITE_LIMIT_TRIGGER_DEPTH]] ^(
SQLITE_LIMIT_TRIGGER_DEPTH
\n**
The maximum depth of recursion for triggers.
)^\n**\n** [[SQLITE_LIMIT_WORKER_THREADS]] ^(
SQLITE_LIMIT_WORKER_THREADS
\n**
The maximum number of auxiliary worker threads that a single\n** [prepared statement] may start.
)^\n**
\n*/\n#define SQLITE_LIMIT_LENGTH 0\n#define SQLITE_LIMIT_SQL_LENGTH 1\n#define SQLITE_LIMIT_COLUMN 2\n#define SQLITE_LIMIT_EXPR_DEPTH 3\n#define SQLITE_LIMIT_COMPOUND_SELECT 4\n#define SQLITE_LIMIT_VDBE_OP 5\n#define SQLITE_LIMIT_FUNCTION_ARG 6\n#define SQLITE_LIMIT_ATTACHED 7\n#define SQLITE_LIMIT_LIKE_PATTERN_LENGTH 8\n#define SQLITE_LIMIT_VARIABLE_NUMBER 9\n#define SQLITE_LIMIT_TRIGGER_DEPTH 10\n#define SQLITE_LIMIT_WORKER_THREADS 11\n\n/*\n** CAPI3REF: Prepare Flags\n**\n** These constants define various flags that can be passed into\n** \"prepFlags\" parameter of the [sqlite3_prepare_v3()] and\n** [sqlite3_prepare16_v3()] interfaces.\n**\n** New flags may be added in future releases of SQLite.\n**\n**
\n** [[SQLITE_PREPARE_PERSISTENT]] ^(
SQLITE_PREPARE_PERSISTENT
\n**
The SQLITE_PREPARE_PERSISTENT flag is a hint to the query planner\n** that the prepared statement will be retained for a long time and\n** probably reused many times.)^ ^Without this flag, [sqlite3_prepare_v3()]\n** and [sqlite3_prepare16_v3()] assume that the prepared statement will\n** be used just once or at most a few times and then destroyed using\n** [sqlite3_finalize()] relatively soon. The current implementation acts\n** on this hint by avoiding the use of [lookaside memory] so as not to\n** deplete the limited store of lookaside memory. Future versions of\n** SQLite may act on this hint differently.\n**\n** [[SQLITE_PREPARE_NORMALIZE]]
SQLITE_PREPARE_NORMALIZE
\n**
The SQLITE_PREPARE_NORMALIZE flag is a no-op. This flag used\n** to be required for any prepared statement that wanted to use the\n** [sqlite3_normalized_sql()] interface. However, the\n** [sqlite3_normalized_sql()] interface is now available to all\n** prepared statements, regardless of whether or not they use this\n** flag.\n**\n** [[SQLITE_PREPARE_NO_VTAB]]
SQLITE_PREPARE_NO_VTAB
\n**
The SQLITE_PREPARE_NO_VTAB flag causes the SQL compiler\n** to return an error (error code SQLITE_ERROR) if the statement uses\n** any virtual tables.\n**\n** [[SQLITE_PREPARE_DONT_LOG]]
SQLITE_PREPARE_DONT_LOG
\n**
The SQLITE_PREPARE_DONT_LOG flag prevents SQL compiler\n** errors from being sent to the error log defined by\n** [SQLITE_CONFIG_LOG]. This can be used, for example, to do test\n** compiles to see if some SQL syntax is well-formed, without generating\n** messages on the global error log when it is not. If the test compile\n** fails, the sqlite3_prepare_v3() call returns the same error indications\n** with or without this flag; it just omits the call to [sqlite3_log()] that\n** logs the error.\n**
\n*/\n#define SQLITE_PREPARE_PERSISTENT 0x01\n#define SQLITE_PREPARE_NORMALIZE 0x02\n#define SQLITE_PREPARE_NO_VTAB 0x04\n#define SQLITE_PREPARE_DONT_LOG 0x10\n\n/*\n** CAPI3REF: Compiling An SQL Statement\n** KEYWORDS: {SQL statement compiler}\n** METHOD: sqlite3\n** CONSTRUCTOR: sqlite3_stmt\n**\n** To execute an SQL statement, it must first be compiled into a byte-code\n** program using one of these routines. Or, in other words, these routines\n** are constructors for the [prepared statement] object.\n**\n** The preferred routine to use is [sqlite3_prepare_v2()]. The\n** [sqlite3_prepare()] interface is legacy and should be avoided.\n** [sqlite3_prepare_v3()] has an extra \"prepFlags\" option that is used\n** for special purposes.\n**\n** The use of the UTF-8 interfaces is preferred, as SQLite currently\n** does all parsing using UTF-8. The UTF-16 interfaces are provided\n** as a convenience. The UTF-16 interfaces work by converting the\n** input text into UTF-8, then invoking the corresponding UTF-8 interface.\n**\n** The first argument, \"db\", is a [database connection] obtained from a\n** prior successful call to [sqlite3_open()], [sqlite3_open_v2()] or\n** [sqlite3_open16()]. The database connection must not have been closed.\n**\n** The second argument, \"zSql\", is the statement to be compiled, encoded\n** as either UTF-8 or UTF-16. The sqlite3_prepare(), sqlite3_prepare_v2(),\n** and sqlite3_prepare_v3()\n** interfaces use UTF-8, and sqlite3_prepare16(), sqlite3_prepare16_v2(),\n** and sqlite3_prepare16_v3() use UTF-16.\n**\n** ^If the nByte argument is negative, then zSql is read up to the\n** first zero terminator. ^If nByte is positive, then it is the maximum\n** number of bytes read from zSql. When nByte is positive, zSql is read\n** up to the first zero terminator or until the nByte bytes have been read,\n** whichever comes first. ^If nByte is zero, then no prepared\n** statement is generated.\n** If the caller knows that the supplied string is nul-terminated, then\n** there is a small performance advantage to passing an nByte parameter that\n** is the number of bytes in the input string including\n** the nul-terminator.\n** Note that nByte measure the length of the input in bytes, not\n** characters, even for the UTF-16 interfaces.\n**\n** ^If pzTail is not NULL then *pzTail is made to point to the first byte\n** past the end of the first SQL statement in zSql. These routines only\n** compile the first statement in zSql, so *pzTail is left pointing to\n** what remains uncompiled.\n**\n** ^*ppStmt is left pointing to a compiled [prepared statement] that can be\n** executed using [sqlite3_step()]. ^If there is an error, *ppStmt is set\n** to NULL. ^If the input text contains no SQL (if the input is an empty\n** string or a comment) then *ppStmt is set to NULL.\n** The calling procedure is responsible for deleting the compiled\n** SQL statement using [sqlite3_finalize()] after it has finished with it.\n** ppStmt may not be NULL.\n**\n** ^On success, the sqlite3_prepare() family of routines return [SQLITE_OK];\n** otherwise an [error code] is returned.\n**\n** The sqlite3_prepare_v2(), sqlite3_prepare_v3(), sqlite3_prepare16_v2(),\n** and sqlite3_prepare16_v3() interfaces are recommended for all new programs.\n** The older interfaces (sqlite3_prepare() and sqlite3_prepare16())\n** are retained for backwards compatibility, but their use is discouraged.\n** ^In the \"vX\" interfaces, the prepared statement\n** that is returned (the [sqlite3_stmt] object) contains a copy of the\n** original SQL text. This causes the [sqlite3_step()] interface to\n** behave differently in three ways:\n**\n**
    \n**
  1. \n** ^If the database schema changes, instead of returning [SQLITE_SCHEMA] as it\n** always used to do, [sqlite3_step()] will automatically recompile the SQL\n** statement and try to run it again. As many as [SQLITE_MAX_SCHEMA_RETRY]\n** retries will occur before sqlite3_step() gives up and returns an error.\n**
    2. \n**\n**
  2. \n** ^When an error occurs, [sqlite3_step()] will return one of the detailed\n** [error codes] or [extended error codes]. ^The legacy behavior was that\n** [sqlite3_step()] would only return a generic [SQLITE_ERROR] result code\n** and the application would have to make a second call to [sqlite3_reset()]\n** in order to find the underlying cause of the problem. With the \"v2\" prepare\n** interfaces, the underlying reason for the error is returned immediately.\n**
    3. \n**\n**
  3. \n** ^If the specific value bound to a [parameter | host parameter] in the\n** WHERE clause might influence the choice of query plan for a statement,\n** then the statement will be automatically recompiled, as if there had been\n** a schema change, on the first [sqlite3_step()] call following any change\n** to the [sqlite3_bind_text | bindings] of that [parameter].\n** ^The specific value of a WHERE-clause [parameter] might influence the\n** choice of query plan if the parameter is the left-hand side of a [LIKE]\n** or [GLOB] operator or if the parameter is compared to an indexed column\n** and the [SQLITE_ENABLE_STAT4] compile-time option is enabled.\n**
    4. \n**
\n**\n**

^sqlite3_prepare_v3() differs from sqlite3_prepare_v2() only in having\n** the extra prepFlags parameter, which is a bit array consisting of zero or\n** more of the [SQLITE_PREPARE_PERSISTENT|SQLITE_PREPARE_*] flags. ^The\n** sqlite3_prepare_v2() interface works exactly the same as\n** sqlite3_prepare_v3() with a zero prepFlags parameter.\n*/\nSQLITE_API int sqlite3_prepare(\n sqlite3 *db, /* Database handle */\n const char *zSql, /* SQL statement, UTF-8 encoded */\n int nByte, /* Maximum length of zSql in bytes. */\n sqlite3_stmt **ppStmt, /* OUT: Statement handle */\n const char **pzTail /* OUT: Pointer to unused portion of zSql */\n);\nSQLITE_API int sqlite3_prepare_v2(\n sqlite3 *db, /* Database handle */\n const char *zSql, /* SQL statement, UTF-8 encoded */\n int nByte, /* Maximum length of zSql in bytes. */\n sqlite3_stmt **ppStmt, /* OUT: Statement handle */\n const char **pzTail /* OUT: Pointer to unused portion of zSql */\n);\nSQLITE_API int sqlite3_prepare_v3(\n sqlite3 *db, /* Database handle */\n const char *zSql, /* SQL statement, UTF-8 encoded */\n int nByte, /* Maximum length of zSql in bytes. */\n unsigned int prepFlags, /* Zero or more SQLITE_PREPARE_ flags */\n sqlite3_stmt **ppStmt, /* OUT: Statement handle */\n const char **pzTail /* OUT: Pointer to unused portion of zSql */\n);\nSQLITE_API int sqlite3_prepare16(\n sqlite3 *db, /* Database handle */\n const void *zSql, /* SQL statement, UTF-16 encoded */\n int nByte, /* Maximum length of zSql in bytes. */\n sqlite3_stmt **ppStmt, /* OUT: Statement handle */\n const void **pzTail /* OUT: Pointer to unused portion of zSql */\n);\nSQLITE_API int sqlite3_prepare16_v2(\n sqlite3 *db, /* Database handle */\n const void *zSql, /* SQL statement, UTF-16 encoded */\n int nByte, /* Maximum length of zSql in bytes. */\n sqlite3_stmt **ppStmt, /* OUT: Statement handle */\n const void **pzTail /* OUT: Pointer to unused portion of zSql */\n);\nSQLITE_API int sqlite3_prepare16_v3(\n sqlite3 *db, /* Database handle */\n const void *zSql, /* SQL statement, UTF-16 encoded */\n int nByte, /* Maximum length of zSql in bytes. */\n unsigned int prepFlags, /* Zero or more SQLITE_PREPARE_ flags */\n sqlite3_stmt **ppStmt, /* OUT: Statement handle */\n const void **pzTail /* OUT: Pointer to unused portion of zSql */\n);\n\n/*\n** CAPI3REF: Retrieving Statement SQL\n** METHOD: sqlite3_stmt\n**\n** ^The sqlite3_sql(P) interface returns a pointer to a copy of the UTF-8\n** SQL text used to create [prepared statement] P if P was\n** created by [sqlite3_prepare_v2()], [sqlite3_prepare_v3()],\n** [sqlite3_prepare16_v2()], or [sqlite3_prepare16_v3()].\n** ^The sqlite3_expanded_sql(P) interface returns a pointer to a UTF-8\n** string containing the SQL text of prepared statement P with\n** [bound parameters] expanded.\n** ^The sqlite3_normalized_sql(P) interface returns a pointer to a UTF-8\n** string containing the normalized SQL text of prepared statement P. The\n** semantics used to normalize a SQL statement are unspecified and subject\n** to change. At a minimum, literal values will be replaced with suitable\n** placeholders.\n**\n** ^(For example, if a prepared statement is created using the SQL\n** text \"SELECT $abc,:xyz\" and if parameter $abc is bound to integer 2345\n** and parameter :xyz is unbound, then sqlite3_sql() will return\n** the original string, \"SELECT $abc,:xyz\" but sqlite3_expanded_sql()\n** will return \"SELECT 2345,NULL\".)^\n**\n** ^The sqlite3_expanded_sql() interface returns NULL if insufficient memory\n** is available to hold the result, or if the result would exceed the\n** the maximum string length determined by the [SQLITE_LIMIT_LENGTH].\n**\n** ^The [SQLITE_TRACE_SIZE_LIMIT] compile-time option limits the size of\n** bound parameter expansions. ^The [SQLITE_OMIT_TRACE] compile-time\n** option causes sqlite3_expanded_sql() to always return NULL.\n**\n** ^The strings returned by sqlite3_sql(P) and sqlite3_normalized_sql(P)\n** are managed by SQLite and are automatically freed when the prepared\n** statement is finalized.\n** ^The string returned by sqlite3_expanded_sql(P), on the other hand,\n** is obtained from [sqlite3_malloc()] and must be freed by the application\n** by passing it to [sqlite3_free()].\n**\n** ^The sqlite3_normalized_sql() interface is only available if\n** the [SQLITE_ENABLE_NORMALIZE] compile-time option is defined.\n*/\nSQLITE_API const char *sqlite3_sql(sqlite3_stmt *pStmt);\nSQLITE_API char *sqlite3_expanded_sql(sqlite3_stmt *pStmt);\n#ifdef SQLITE_ENABLE_NORMALIZE\nSQLITE_API const char *sqlite3_normalized_sql(sqlite3_stmt *pStmt);\n#endif\n\n/*\n** CAPI3REF: Determine If An SQL Statement Writes The Database\n** METHOD: sqlite3_stmt\n**\n** ^The sqlite3_stmt_readonly(X) interface returns true (non-zero) if\n** and only if the [prepared statement] X makes no direct changes to\n** the content of the database file.\n**\n** Note that [application-defined SQL functions] or\n** [virtual tables] might change the database indirectly as a side effect.\n** ^(For example, if an application defines a function \"eval()\" that\n** calls [sqlite3_exec()], then the following SQL statement would\n** change the database file through side-effects:\n**\n** 

\n**    SELECT eval('DELETE FROM t1') FROM t2;\n**
\n**\n** But because the [SELECT] statement does not change the database file\n** directly, sqlite3_stmt_readonly() would still return true.)^\n**\n** ^Transaction control statements such as [BEGIN], [COMMIT], [ROLLBACK],\n** [SAVEPOINT], and [RELEASE] cause sqlite3_stmt_readonly() to return true,\n** since the statements themselves do not actually modify the database but\n** rather they control the timing of when other statements modify the\n** database. ^The [ATTACH] and [DETACH] statements also cause\n** sqlite3_stmt_readonly() to return true since, while those statements\n** change the configuration of a database connection, they do not make\n** changes to the content of the database files on disk.\n** ^The sqlite3_stmt_readonly() interface returns true for [BEGIN] since\n** [BEGIN] merely sets internal flags, but the [BEGIN|BEGIN IMMEDIATE] and\n** [BEGIN|BEGIN EXCLUSIVE] commands do touch the database and so\n** sqlite3_stmt_readonly() returns false for those commands.\n**\n** ^This routine returns false if there is any possibility that the\n** statement might change the database file. ^A false return does\n** not guarantee that the statement will change the database file.\n** ^For example, an UPDATE statement might have a WHERE clause that\n** makes it a no-op, but the sqlite3_stmt_readonly() result would still\n** be false. ^Similarly, a CREATE TABLE IF NOT EXISTS statement is a\n** read-only no-op if the table already exists, but\n** sqlite3_stmt_readonly() still returns false for such a statement.\n**\n** ^If prepared statement X is an [EXPLAIN] or [EXPLAIN QUERY PLAN]\n** statement, then sqlite3_stmt_readonly(X) returns the same value as\n** if the EXPLAIN or EXPLAIN QUERY PLAN prefix were omitted.\n*/\nSQLITE_API int sqlite3_stmt_readonly(sqlite3_stmt *pStmt);\n\n/*\n** CAPI3REF: Query The EXPLAIN Setting For A Prepared Statement\n** METHOD: sqlite3_stmt\n**\n** ^The sqlite3_stmt_isexplain(S) interface returns 1 if the\n** prepared statement S is an EXPLAIN statement, or 2 if the\n** statement S is an EXPLAIN QUERY PLAN.\n** ^The sqlite3_stmt_isexplain(S) interface returns 0 if S is\n** an ordinary statement or a NULL pointer.\n*/\nSQLITE_API int sqlite3_stmt_isexplain(sqlite3_stmt *pStmt);\n\n/*\n** CAPI3REF: Change The EXPLAIN Setting For A Prepared Statement\n** METHOD: sqlite3_stmt\n**\n** The sqlite3_stmt_explain(S,E) interface changes the EXPLAIN\n** setting for [prepared statement] S. If E is zero, then S becomes\n** a normal prepared statement. If E is 1, then S behaves as if\n** its SQL text began with \"[EXPLAIN]\". If E is 2, then S behaves as if\n** its SQL text began with \"[EXPLAIN QUERY PLAN]\".\n**\n** Calling sqlite3_stmt_explain(S,E) might cause S to be reprepared.\n** SQLite tries to avoid a reprepare, but a reprepare might be necessary\n** on the first transition into EXPLAIN or EXPLAIN QUERY PLAN mode.\n**\n** Because of the potential need to reprepare, a call to\n** sqlite3_stmt_explain(S,E) will fail with SQLITE_ERROR if S cannot be\n** reprepared because it was created using [sqlite3_prepare()] instead of\n** the newer [sqlite3_prepare_v2()] or [sqlite3_prepare_v3()] interfaces and\n** hence has no saved SQL text with which to reprepare.\n**\n** Changing the explain setting for a prepared statement does not change\n** the original SQL text for the statement. Hence, if the SQL text originally\n** began with EXPLAIN or EXPLAIN QUERY PLAN, but sqlite3_stmt_explain(S,0)\n** is called to convert the statement into an ordinary statement, the EXPLAIN\n** or EXPLAIN QUERY PLAN keywords will still appear in the sqlite3_sql(S)\n** output, even though the statement now acts like a normal SQL statement.\n**\n** This routine returns SQLITE_OK if the explain mode is successfully\n** changed, or an error code if the explain mode could not be changed.\n** The explain mode cannot be changed while a statement is active.\n** Hence, it is good practice to call [sqlite3_reset(S)]\n** immediately prior to calling sqlite3_stmt_explain(S,E).\n*/\nSQLITE_API int sqlite3_stmt_explain(sqlite3_stmt *pStmt, int eMode);\n\n/*\n** CAPI3REF: Determine If A Prepared Statement Has Been Reset\n** METHOD: sqlite3_stmt\n**\n** ^The sqlite3_stmt_busy(S) interface returns true (non-zero) if the\n** [prepared statement] S has been stepped at least once using\n** [sqlite3_step(S)] but has neither run to completion (returned\n** [SQLITE_DONE] from [sqlite3_step(S)]) nor\n** been reset using [sqlite3_reset(S)]. ^The sqlite3_stmt_busy(S)\n** interface returns false if S is a NULL pointer. If S is not a\n** NULL pointer and is not a pointer to a valid [prepared statement]\n** object, then the behavior is undefined and probably undesirable.\n**\n** This interface can be used in combination [sqlite3_next_stmt()]\n** to locate all prepared statements associated with a database\n** connection that are in need of being reset. This can be used,\n** for example, in diagnostic routines to search for prepared\n** statements that are holding a transaction open.\n*/\nSQLITE_API int sqlite3_stmt_busy(sqlite3_stmt*);\n\n/*\n** CAPI3REF: Dynamically Typed Value Object\n** KEYWORDS: {protected sqlite3_value} {unprotected sqlite3_value}\n**\n** SQLite uses the sqlite3_value object to represent all values\n** that can be stored in a database table. SQLite uses dynamic typing\n** for the values it stores. ^Values stored in sqlite3_value objects\n** can be integers, floating point values, strings, BLOBs, or NULL.\n**\n** An sqlite3_value object may be either \"protected\" or \"unprotected\".\n** Some interfaces require a protected sqlite3_value. Other interfaces\n** will accept either a protected or an unprotected sqlite3_value.\n** Every interface that accepts sqlite3_value arguments specifies\n** whether or not it requires a protected sqlite3_value. The\n** [sqlite3_value_dup()] interface can be used to construct a new\n** protected sqlite3_value from an unprotected sqlite3_value.\n**\n** The terms \"protected\" and \"unprotected\" refer to whether or not\n** a mutex is held. An internal mutex is held for a protected\n** sqlite3_value object but no mutex is held for an unprotected\n** sqlite3_value object. If SQLite is compiled to be single-threaded\n** (with [SQLITE_THREADSAFE=0] and with [sqlite3_threadsafe()] returning 0)\n** or if SQLite is run in one of reduced mutex modes\n** [SQLITE_CONFIG_SINGLETHREAD] or [SQLITE_CONFIG_MULTITHREAD]\n** then there is no distinction between protected and unprotected\n** sqlite3_value objects and they can be used interchangeably. However,\n** for maximum code portability it is recommended that applications\n** still make the distinction between protected and unprotected\n** sqlite3_value objects even when not strictly required.\n**\n** ^The sqlite3_value objects that are passed as parameters into the\n** implementation of [application-defined SQL functions] are protected.\n** ^The sqlite3_value objects returned by [sqlite3_vtab_rhs_value()]\n** are protected.\n** ^The sqlite3_value object returned by\n** [sqlite3_column_value()] is unprotected.\n** Unprotected sqlite3_value objects may only be used as arguments\n** to [sqlite3_result_value()], [sqlite3_bind_value()], and\n** [sqlite3_value_dup()].\n** The [sqlite3_value_blob | sqlite3_value_type()] family of\n** interfaces require protected sqlite3_value objects.\n*/\ntypedef struct sqlite3_value sqlite3_value;\n\n/*\n** CAPI3REF: SQL Function Context Object\n**\n** The context in which an SQL function executes is stored in an\n** sqlite3_context object. ^A pointer to an sqlite3_context object\n** is always first parameter to [application-defined SQL functions].\n** The application-defined SQL function implementation will pass this\n** pointer through into calls to [sqlite3_result_int | sqlite3_result()],\n** [sqlite3_aggregate_context()], [sqlite3_user_data()],\n** [sqlite3_context_db_handle()], [sqlite3_get_auxdata()],\n** and/or [sqlite3_set_auxdata()].\n*/\ntypedef struct sqlite3_context sqlite3_context;\n\n/*\n** CAPI3REF: Binding Values To Prepared Statements\n** KEYWORDS: {host parameter} {host parameters} {host parameter name}\n** KEYWORDS: {SQL parameter} {SQL parameters} {parameter binding}\n** METHOD: sqlite3_stmt\n**\n** ^(In the SQL statement text input to [sqlite3_prepare_v2()] and its variants,\n** literals may be replaced by a [parameter] that matches one of the following\n** templates:\n**\n**
    \n**
  • ?\n**
  • ?NNN\n**
  • :VVV\n**
  • @VVV\n**
  • $VVV\n**
\n**\n** In the templates above, NNN represents an integer literal,\n** and VVV represents an alphanumeric identifier.)^ ^The values of these\n** parameters (also called \"host parameter names\" or \"SQL parameters\")\n** can be set using the sqlite3_bind_*() routines defined here.\n**\n** ^The first argument to the sqlite3_bind_*() routines is always\n** a pointer to the [sqlite3_stmt] object returned from\n** [sqlite3_prepare_v2()] or its variants.\n**\n** ^The second argument is the index of the SQL parameter to be set.\n** ^The leftmost SQL parameter has an index of 1. ^When the same named\n** SQL parameter is used more than once, second and subsequent\n** occurrences have the same index as the first occurrence.\n** ^The index for named parameters can be looked up using the\n** [sqlite3_bind_parameter_index()] API if desired. ^The index\n** for \"?NNN\" parameters is the value of NNN.\n** ^The NNN value must be between 1 and the [sqlite3_limit()]\n** parameter [SQLITE_LIMIT_VARIABLE_NUMBER] (default value: 32766).\n**\n** ^The third argument is the value to bind to the parameter.\n** ^If the third parameter to sqlite3_bind_text() or sqlite3_bind_text16()\n** or sqlite3_bind_blob() is a NULL pointer then the fourth parameter\n** is ignored and the end result is the same as sqlite3_bind_null().\n** ^If the third parameter to sqlite3_bind_text() is not NULL, then\n** it should be a pointer to well-formed UTF8 text.\n** ^If the third parameter to sqlite3_bind_text16() is not NULL, then\n** it should be a pointer to well-formed UTF16 text.\n** ^If the third parameter to sqlite3_bind_text64() is not NULL, then\n** it should be a pointer to a well-formed unicode string that is\n** either UTF8 if the sixth parameter is SQLITE_UTF8, or UTF16\n** otherwise.\n**\n** [[byte-order determination rules]] ^The byte-order of\n** UTF16 input text is determined by the byte-order mark (BOM, U+FEFF)\n** found in the first character, which is removed, or in the absence of a BOM\n** the byte order is the native byte order of the host\n** machine for sqlite3_bind_text16() or the byte order specified in\n** the 6th parameter for sqlite3_bind_text64().)^\n** ^If UTF16 input text contains invalid unicode\n** characters, then SQLite might change those invalid characters\n** into the unicode replacement character: U+FFFD.\n**\n** ^(In those routines that have a fourth argument, its value is the\n** number of bytes in the parameter. To be clear: the value is the\n** number of bytes in the value, not the number of characters.)^\n** ^If the fourth parameter to sqlite3_bind_text() or sqlite3_bind_text16()\n** is negative, then the length of the string is\n** the number of bytes up to the first zero terminator.\n** If the fourth parameter to sqlite3_bind_blob() is negative, then\n** the behavior is undefined.\n** If a non-negative fourth parameter is provided to sqlite3_bind_text()\n** or sqlite3_bind_text16() or sqlite3_bind_text64() then\n** that parameter must be the byte offset\n** where the NUL terminator would occur assuming the string were NUL\n** terminated. If any NUL characters occur at byte offsets less than\n** the value of the fourth parameter then the resulting string value will\n** contain embedded NULs. The result of expressions involving strings\n** with embedded NULs is undefined.\n**\n** ^The fifth argument to the BLOB and string binding interfaces controls\n** or indicates the lifetime of the object referenced by the third parameter.\n** These three options exist:\n** ^ (1) A destructor to dispose of the BLOB or string after SQLite has finished\n** with it may be passed. ^It is called to dispose of the BLOB or string even\n** if the call to the bind API fails, except the destructor is not called if\n** the third parameter is a NULL pointer or the fourth parameter is negative.\n** ^ (2) The special constant, [SQLITE_STATIC], may be passed to indicate that\n** the application remains responsible for disposing of the object. ^In this\n** case, the object and the provided pointer to it must remain valid until\n** either the prepared statement is finalized or the same SQL parameter is\n** bound to something else, whichever occurs sooner.\n** ^ (3) The constant, [SQLITE_TRANSIENT], may be passed to indicate that the\n** object is to be copied prior to the return from sqlite3_bind_*(). ^The\n** object and pointer to it must remain valid until then. ^SQLite will then\n** manage the lifetime of its private copy.\n**\n** ^The sixth argument to sqlite3_bind_text64() must be one of\n** [SQLITE_UTF8], [SQLITE_UTF16], [SQLITE_UTF16BE], or [SQLITE_UTF16LE]\n** to specify the encoding of the text in the third parameter. If\n** the sixth argument to sqlite3_bind_text64() is not one of the\n** allowed values shown above, or if the text encoding is different\n** from the encoding specified by the sixth parameter, then the behavior\n** is undefined.\n**\n** ^The sqlite3_bind_zeroblob() routine binds a BLOB of length N that\n** is filled with zeroes. ^A zeroblob uses a fixed amount of memory\n** (just an integer to hold its size) while it is being processed.\n** Zeroblobs are intended to serve as placeholders for BLOBs whose\n** content is later written using\n** [sqlite3_blob_open | incremental BLOB I/O] routines.\n** ^A negative value for the zeroblob results in a zero-length BLOB.\n**\n** ^The sqlite3_bind_pointer(S,I,P,T,D) routine causes the I-th parameter in\n** [prepared statement] S to have an SQL value of NULL, but to also be\n** associated with the pointer P of type T. ^D is either a NULL pointer or\n** a pointer to a destructor function for P. ^SQLite will invoke the\n** destructor D with a single argument of P when it is finished using\n** P. The T parameter should be a static string, preferably a string\n** literal. The sqlite3_bind_pointer() routine is part of the\n** [pointer passing interface] added for SQLite 3.20.0.\n**\n** ^If any of the sqlite3_bind_*() routines are called with a NULL pointer\n** for the [prepared statement] or with a prepared statement for which\n** [sqlite3_step()] has been called more recently than [sqlite3_reset()],\n** then the call will return [SQLITE_MISUSE]. If any sqlite3_bind_()\n** routine is passed a [prepared statement] that has been finalized, the\n** result is undefined and probably harmful.\n**\n** ^Bindings are not cleared by the [sqlite3_reset()] routine.\n** ^Unbound parameters are interpreted as NULL.\n**\n** ^The sqlite3_bind_* routines return [SQLITE_OK] on success or an\n** [error code] if anything goes wrong.\n** ^[SQLITE_TOOBIG] might be returned if the size of a string or BLOB\n** exceeds limits imposed by [sqlite3_limit]([SQLITE_LIMIT_LENGTH]) or\n** [SQLITE_MAX_LENGTH].\n** ^[SQLITE_RANGE] is returned if the parameter\n** index is out of range. ^[SQLITE_NOMEM] is returned if malloc() fails.\n**\n** See also: [sqlite3_bind_parameter_count()],\n** [sqlite3_bind_parameter_name()], and [sqlite3_bind_parameter_index()].\n*/\nSQLITE_API int sqlite3_bind_blob(sqlite3_stmt*, int, const void*, int n, void(*)(void*));\nSQLITE_API int sqlite3_bind_blob64(sqlite3_stmt*, int, const void*, sqlite3_uint64,\n void(*)(void*));\nSQLITE_API int sqlite3_bind_double(sqlite3_stmt*, int, double);\nSQLITE_API int sqlite3_bind_int(sqlite3_stmt*, int, int);\nSQLITE_API int sqlite3_bind_int64(sqlite3_stmt*, int, sqlite3_int64);\nSQLITE_API int sqlite3_bind_null(sqlite3_stmt*, int);\nSQLITE_API int sqlite3_bind_text(sqlite3_stmt*,int,const char*,int,void(*)(void*));\nSQLITE_API int sqlite3_bind_text16(sqlite3_stmt*, int, const void*, int, void(*)(void*));\nSQLITE_API int sqlite3_bind_text64(sqlite3_stmt*, int, const char*, sqlite3_uint64,\n void(*)(void*), unsigned char encoding);\nSQLITE_API int sqlite3_bind_value(sqlite3_stmt*, int, const sqlite3_value*);\nSQLITE_API int sqlite3_bind_pointer(sqlite3_stmt*, int, void*, const char*,void(*)(void*));\nSQLITE_API int sqlite3_bind_zeroblob(sqlite3_stmt*, int, int n);\nSQLITE_API int sqlite3_bind_zeroblob64(sqlite3_stmt*, int, sqlite3_uint64);\n\n/*\n** CAPI3REF: Number Of SQL Parameters\n** METHOD: sqlite3_stmt\n**\n** ^This routine can be used to find the number of [SQL parameters]\n** in a [prepared statement]. SQL parameters are tokens of the\n** form \"?\", \"?NNN\", \":AAA\", \"$AAA\", or \"@AAA\" that serve as\n** placeholders for values that are [sqlite3_bind_blob | bound]\n** to the parameters at a later time.\n**\n** ^(This routine actually returns the index of the largest (rightmost)\n** parameter. For all forms except ?NNN, this will correspond to the\n** number of unique parameters. If parameters of the ?NNN form are used,\n** there may be gaps in the list.)^\n**\n** See also: [sqlite3_bind_blob|sqlite3_bind()],\n** [sqlite3_bind_parameter_name()], and\n** [sqlite3_bind_parameter_index()].\n*/\nSQLITE_API int sqlite3_bind_parameter_count(sqlite3_stmt*);\n\n/*\n** CAPI3REF: Name Of A Host Parameter\n** METHOD: sqlite3_stmt\n**\n** ^The sqlite3_bind_parameter_name(P,N) interface returns\n** the name of the N-th [SQL parameter] in the [prepared statement] P.\n** ^(SQL parameters of the form \"?NNN\" or \":AAA\" or \"@AAA\" or \"$AAA\"\n** have a name which is the string \"?NNN\" or \":AAA\" or \"@AAA\" or \"$AAA\"\n** respectively.\n** In other words, the initial \":\" or \"$\" or \"@\" or \"?\"\n** is included as part of the name.)^\n** ^Parameters of the form \"?\" without a following integer have no name\n** and are referred to as \"nameless\" or \"anonymous parameters\".\n**\n** ^The first host parameter has an index of 1, not 0.\n**\n** ^If the value N is out of range or if the N-th parameter is\n** nameless, then NULL is returned. ^The returned string is\n** always in UTF-8 encoding even if the named parameter was\n** originally specified as UTF-16 in [sqlite3_prepare16()],\n** [sqlite3_prepare16_v2()], or [sqlite3_prepare16_v3()].\n**\n** See also: [sqlite3_bind_blob|sqlite3_bind()],\n** [sqlite3_bind_parameter_count()], and\n** [sqlite3_bind_parameter_index()].\n*/\nSQLITE_API const char *sqlite3_bind_parameter_name(sqlite3_stmt*, int);\n\n/*\n** CAPI3REF: Index Of A Parameter With A Given Name\n** METHOD: sqlite3_stmt\n**\n** ^Return the index of an SQL parameter given its name. ^The\n** index value returned is suitable for use as the second\n** parameter to [sqlite3_bind_blob|sqlite3_bind()]. ^A zero\n** is returned if no matching parameter is found. ^The parameter\n** name must be given in UTF-8 even if the original statement\n** was prepared from UTF-16 text using [sqlite3_prepare16_v2()] or\n** [sqlite3_prepare16_v3()].\n**\n** See also: [sqlite3_bind_blob|sqlite3_bind()],\n** [sqlite3_bind_parameter_count()], and\n** [sqlite3_bind_parameter_name()].\n*/\nSQLITE_API int sqlite3_bind_parameter_index(sqlite3_stmt*, const char *zName);\n\n/*\n** CAPI3REF: Reset All Bindings On A Prepared Statement\n** METHOD: sqlite3_stmt\n**\n** ^Contrary to the intuition of many, [sqlite3_reset()] does not reset\n** the [sqlite3_bind_blob | bindings] on a [prepared statement].\n** ^Use this routine to reset all host parameters to NULL.\n*/\nSQLITE_API int sqlite3_clear_bindings(sqlite3_stmt*);\n\n/*\n** CAPI3REF: Number Of Columns In A Result Set\n** METHOD: sqlite3_stmt\n**\n** ^Return the number of columns in the result set returned by the\n** [prepared statement]. ^If this routine returns 0, that means the\n** [prepared statement] returns no data (for example an [UPDATE]).\n** ^However, just because this routine returns a positive number does not\n** mean that one or more rows of data will be returned. ^A SELECT statement\n** will always have a positive sqlite3_column_count() but depending on the\n** WHERE clause constraints and the table content, it might return no rows.\n**\n** See also: [sqlite3_data_count()]\n*/\nSQLITE_API int sqlite3_column_count(sqlite3_stmt *pStmt);\n\n/*\n** CAPI3REF: Column Names In A Result Set\n** METHOD: sqlite3_stmt\n**\n** ^These routines return the name assigned to a particular column\n** in the result set of a [SELECT] statement. ^The sqlite3_column_name()\n** interface returns a pointer to a zero-terminated UTF-8 string\n** and sqlite3_column_name16() returns a pointer to a zero-terminated\n** UTF-16 string. ^The first parameter is the [prepared statement]\n** that implements the [SELECT] statement. ^The second parameter is the\n** column number. ^The leftmost column is number 0.\n**\n** ^The returned string pointer is valid until either the [prepared statement]\n** is destroyed by [sqlite3_finalize()] or until the statement is automatically\n** reprepared by the first call to [sqlite3_step()] for a particular run\n** or until the next call to\n** sqlite3_column_name() or sqlite3_column_name16() on the same column.\n**\n** ^If sqlite3_malloc() fails during the processing of either routine\n** (for example during a conversion from UTF-8 to UTF-16) then a\n** NULL pointer is returned.\n**\n** ^The name of a result column is the value of the \"AS\" clause for\n** that column, if there is an AS clause. If there is no AS clause\n** then the name of the column is unspecified and may change from\n** one release of SQLite to the next.\n*/\nSQLITE_API const char *sqlite3_column_name(sqlite3_stmt*, int N);\nSQLITE_API const void *sqlite3_column_name16(sqlite3_stmt*, int N);\n\n/*\n** CAPI3REF: Source Of Data In A Query Result\n** METHOD: sqlite3_stmt\n**\n** ^These routines provide a means to determine the database, table, and\n** table column that is the origin of a particular result column in a\n** [SELECT] statement.\n** ^The name of the database or table or column can be returned as\n** either a UTF-8 or UTF-16 string. ^The _database_ routines return\n** the database name, the _table_ routines return the table name, and\n** the origin_ routines return the column name.\n** ^The returned string is valid until the [prepared statement] is destroyed\n** using [sqlite3_finalize()] or until the statement is automatically\n** reprepared by the first call to [sqlite3_step()] for a particular run\n** or until the same information is requested\n** again in a different encoding.\n**\n** ^The names returned are the original un-aliased names of the\n** database, table, and column.\n**\n** ^The first argument to these interfaces is a [prepared statement].\n** ^These functions return information about the Nth result column returned by\n** the statement, where N is the second function argument.\n** ^The left-most column is column 0 for these routines.\n**\n** ^If the Nth column returned by the statement is an expression or\n** subquery and is not a column value, then all of these functions return\n** NULL. ^These routines might also return NULL if a memory allocation error\n** occurs. ^Otherwise, they return the name of the attached database, table,\n** or column that query result column was extracted from.\n**\n** ^As with all other SQLite APIs, those whose names end with \"16\" return\n** UTF-16 encoded strings and the other functions return UTF-8.\n**\n** ^These APIs are only available if the library was compiled with the\n** [SQLITE_ENABLE_COLUMN_METADATA] C-preprocessor symbol.\n**\n** If two or more threads call one or more\n** [sqlite3_column_database_name | column metadata interfaces]\n** for the same [prepared statement] and result column\n** at the same time then the results are undefined.\n*/\nSQLITE_API const char *sqlite3_column_database_name(sqlite3_stmt*,int);\nSQLITE_API const void *sqlite3_column_database_name16(sqlite3_stmt*,int);\nSQLITE_API const char *sqlite3_column_table_name(sqlite3_stmt*,int);\nSQLITE_API const void *sqlite3_column_table_name16(sqlite3_stmt*,int);\nSQLITE_API const char *sqlite3_column_origin_name(sqlite3_stmt*,int);\nSQLITE_API const void *sqlite3_column_origin_name16(sqlite3_stmt*,int);\n\n/*\n** CAPI3REF: Declared Datatype Of A Query Result\n** METHOD: sqlite3_stmt\n**\n** ^(The first parameter is a [prepared statement].\n** If this statement is a [SELECT] statement and the Nth column of the\n** returned result set of that [SELECT] is a table column (not an\n** expression or subquery) then the declared type of the table\n** column is returned.)^ ^If the Nth column of the result set is an\n** expression or subquery, then a NULL pointer is returned.\n** ^The returned string is always UTF-8 encoded.\n**\n** ^(For example, given the database schema:\n**\n** CREATE TABLE t1(c1 VARIANT);\n**\n** and the following statement to be compiled:\n**\n** SELECT c1 + 1, c1 FROM t1;\n**\n** this routine would return the string \"VARIANT\" for the second result\n** column (i==1), and a NULL pointer for the first result column (i==0).)^\n**\n** ^SQLite uses dynamic run-time typing. ^So just because a column\n** is declared to contain a particular type does not mean that the\n** data stored in that column is of the declared type. SQLite is\n** strongly typed, but the typing is dynamic not static. ^Type\n** is associated with individual values, not with the containers\n** used to hold those values.\n*/\nSQLITE_API const char *sqlite3_column_decltype(sqlite3_stmt*,int);\nSQLITE_API const void *sqlite3_column_decltype16(sqlite3_stmt*,int);\n\n/*\n** CAPI3REF: Evaluate An SQL Statement\n** METHOD: sqlite3_stmt\n**\n** After a [prepared statement] has been prepared using any of\n** [sqlite3_prepare_v2()], [sqlite3_prepare_v3()], [sqlite3_prepare16_v2()],\n** or [sqlite3_prepare16_v3()] or one of the legacy\n** interfaces [sqlite3_prepare()] or [sqlite3_prepare16()], this function\n** must be called one or more times to evaluate the statement.\n**\n** The details of the behavior of the sqlite3_step() interface depend\n** on whether the statement was prepared using the newer \"vX\" interfaces\n** [sqlite3_prepare_v3()], [sqlite3_prepare_v2()], [sqlite3_prepare16_v3()],\n** [sqlite3_prepare16_v2()] or the older legacy\n** interfaces [sqlite3_prepare()] and [sqlite3_prepare16()]. The use of the\n** new \"vX\" interface is recommended for new applications but the legacy\n** interface will continue to be supported.\n**\n** ^In the legacy interface, the return value will be either [SQLITE_BUSY],\n** [SQLITE_DONE], [SQLITE_ROW], [SQLITE_ERROR], or [SQLITE_MISUSE].\n** ^With the \"v2\" interface, any of the other [result codes] or\n** [extended result codes] might be returned as well.\n**\n** ^[SQLITE_BUSY] means that the database engine was unable to acquire the\n** database locks it needs to do its job. ^If the statement is a [COMMIT]\n** or occurs outside of an explicit transaction, then you can retry the\n** statement. If the statement is not a [COMMIT] and occurs within an\n** explicit transaction then you should rollback the transaction before\n** continuing.\n**\n** ^[SQLITE_DONE] means that the statement has finished executing\n** successfully. sqlite3_step() should not be called again on this virtual\n** machine without first calling [sqlite3_reset()] to reset the virtual\n** machine back to its initial state.\n**\n** ^If the SQL statement being executed returns any data, then [SQLITE_ROW]\n** is returned each time a new row of data is ready for processing by the\n** caller. The values may be accessed using the [column access functions].\n** sqlite3_step() is called again to retrieve the next row of data.\n**\n** ^[SQLITE_ERROR] means that a run-time error (such as a constraint\n** violation) has occurred. sqlite3_step() should not be called again on\n** the VM. More information may be found by calling [sqlite3_errmsg()].\n** ^With the legacy interface, a more specific error code (for example,\n** [SQLITE_INTERRUPT], [SQLITE_SCHEMA], [SQLITE_CORRUPT], and so forth)\n** can be obtained by calling [sqlite3_reset()] on the\n** [prepared statement]. ^In the \"v2\" interface,\n** the more specific error code is returned directly by sqlite3_step().\n**\n** [SQLITE_MISUSE] means that the this routine was called inappropriately.\n** Perhaps it was called on a [prepared statement] that has\n** already been [sqlite3_finalize | finalized] or on one that had\n** previously returned [SQLITE_ERROR] or [SQLITE_DONE]. Or it could\n** be the case that the same database connection is being used by two or\n** more threads at the same moment in time.\n**\n** For all versions of SQLite up to and including 3.6.23.1, a call to\n** [sqlite3_reset()] was required after sqlite3_step() returned anything\n** other than [SQLITE_ROW] before any subsequent invocation of\n** sqlite3_step(). Failure to reset the prepared statement using\n** [sqlite3_reset()] would result in an [SQLITE_MISUSE] return from\n** sqlite3_step(). But after [version 3.6.23.1] ([dateof:3.6.23.1]),\n** sqlite3_step() began\n** calling [sqlite3_reset()] automatically in this circumstance rather\n** than returning [SQLITE_MISUSE]. This is not considered a compatibility\n** break because any application that ever receives an SQLITE_MISUSE error\n** is broken by definition. The [SQLITE_OMIT_AUTORESET] compile-time option\n** can be used to restore the legacy behavior.\n**\n** Goofy Interface Alert: In the legacy interface, the sqlite3_step()\n** API always returns a generic error code, [SQLITE_ERROR], following any\n** error other than [SQLITE_BUSY] and [SQLITE_MISUSE]. You must call\n** [sqlite3_reset()] or [sqlite3_finalize()] in order to find one of the\n** specific [error codes] that better describes the error.\n** We admit that this is a goofy design. The problem has been fixed\n** with the \"v2\" interface. If you prepare all of your SQL statements\n** using [sqlite3_prepare_v3()] or [sqlite3_prepare_v2()]\n** or [sqlite3_prepare16_v2()] or [sqlite3_prepare16_v3()] instead\n** of the legacy [sqlite3_prepare()] and [sqlite3_prepare16()] interfaces,\n** then the more specific [error codes] are returned directly\n** by sqlite3_step(). The use of the \"vX\" interfaces is recommended.\n*/\nSQLITE_API int sqlite3_step(sqlite3_stmt*);\n\n/*\n** CAPI3REF: Number of columns in a result set\n** METHOD: sqlite3_stmt\n**\n** ^The sqlite3_data_count(P) interface returns the number of columns in the\n** current row of the result set of [prepared statement] P.\n** ^If prepared statement P does not have results ready to return\n** (via calls to the [sqlite3_column_int | sqlite3_column()] family of\n** interfaces) then sqlite3_data_count(P) returns 0.\n** ^The sqlite3_data_count(P) routine also returns 0 if P is a NULL pointer.\n** ^The sqlite3_data_count(P) routine returns 0 if the previous call to\n** [sqlite3_step](P) returned [SQLITE_DONE]. ^The sqlite3_data_count(P)\n** will return non-zero if previous call to [sqlite3_step](P) returned\n** [SQLITE_ROW], except in the case of the [PRAGMA incremental_vacuum]\n** where it always returns zero since each step of that multi-step\n** pragma returns 0 columns of data.\n**\n** See also: [sqlite3_column_count()]\n*/\nSQLITE_API int sqlite3_data_count(sqlite3_stmt *pStmt);\n\n/*\n** CAPI3REF: Fundamental Datatypes\n** KEYWORDS: SQLITE_TEXT\n**\n** ^(Every value in SQLite has one of five fundamental datatypes:\n**\n**
    \n**
  • 64-bit signed integer\n**
  • 64-bit IEEE floating point number\n**
  • string\n**
  • BLOB\n**
  • NULL\n**
)^\n**\n** These constants are codes for each of those types.\n**\n** Note that the SQLITE_TEXT constant was also used in SQLite version 2\n** for a completely different meaning. Software that links against both\n** SQLite version 2 and SQLite version 3 should use SQLITE3_TEXT, not\n** SQLITE_TEXT.\n*/\n#define SQLITE_INTEGER 1\n#define SQLITE_FLOAT 2\n#define SQLITE_BLOB 4\n#define SQLITE_NULL 5\n#ifdef SQLITE_TEXT\n# undef SQLITE_TEXT\n#else\n# define SQLITE_TEXT 3\n#endif\n#define SQLITE3_TEXT 3\n\n/*\n** CAPI3REF: Result Values From A Query\n** KEYWORDS: {column access functions}\n** METHOD: sqlite3_stmt\n**\n** Summary:\n**
\n**
sqlite3_column_blob→BLOB result\n**
sqlite3_column_double→REAL result\n**
sqlite3_column_int→32-bit INTEGER result\n**
sqlite3_column_int64→64-bit INTEGER result\n**
sqlite3_column_text→UTF-8 TEXT result\n**
sqlite3_column_text16→UTF-16 TEXT result\n**
sqlite3_column_value→The result as an\n** [sqlite3_value|unprotected sqlite3_value] object.\n**
   \n**
sqlite3_column_bytes→Size of a BLOB\n** or a UTF-8 TEXT result in bytes\n**
sqlite3_column_bytes16  \n** →  Size of UTF-16\n** TEXT in bytes\n**
sqlite3_column_type→Default\n** datatype of the result\n**
\n**\n** Details:\n**\n** ^These routines return information about a single column of the current\n** result row of a query. ^In every case the first argument is a pointer\n** to the [prepared statement] that is being evaluated (the [sqlite3_stmt*]\n** that was returned from [sqlite3_prepare_v2()] or one of its variants)\n** and the second argument is the index of the column for which information\n** should be returned. ^The leftmost column of the result set has the index 0.\n** ^The number of columns in the result can be determined using\n** [sqlite3_column_count()].\n**\n** If the SQL statement does not currently point to a valid row, or if the\n** column index is out of range, the result is undefined.\n** These routines may only be called when the most recent call to\n** [sqlite3_step()] has returned [SQLITE_ROW] and neither\n** [sqlite3_reset()] nor [sqlite3_finalize()] have been called subsequently.\n** If any of these routines are called after [sqlite3_reset()] or\n** [sqlite3_finalize()] or after [sqlite3_step()] has returned\n** something other than [SQLITE_ROW], the results are undefined.\n** If [sqlite3_step()] or [sqlite3_reset()] or [sqlite3_finalize()]\n** are called from a different thread while any of these routines\n** are pending, then the results are undefined.\n**\n** The first six interfaces (_blob, _double, _int, _int64, _text, and _text16)\n** each return the value of a result column in a specific data format. If\n** the result column is not initially in the requested format (for example,\n** if the query returns an integer but the sqlite3_column_text() interface\n** is used to extract the value) then an automatic type conversion is performed.\n**\n** ^The sqlite3_column_type() routine returns the\n** [SQLITE_INTEGER | datatype code] for the initial data type\n** of the result column. ^The returned value is one of [SQLITE_INTEGER],\n** [SQLITE_FLOAT], [SQLITE_TEXT], [SQLITE_BLOB], or [SQLITE_NULL].\n** The return value of sqlite3_column_type() can be used to decide which\n** of the first six interface should be used to extract the column value.\n** The value returned by sqlite3_column_type() is only meaningful if no\n** automatic type conversions have occurred for the value in question.\n** After a type conversion, the result of calling sqlite3_column_type()\n** is undefined, though harmless. Future\n** versions of SQLite may change the behavior of sqlite3_column_type()\n** following a type conversion.\n**\n** If the result is a BLOB or a TEXT string, then the sqlite3_column_bytes()\n** or sqlite3_column_bytes16() interfaces can be used to determine the size\n** of that BLOB or string.\n**\n** ^If the result is a BLOB or UTF-8 string then the sqlite3_column_bytes()\n** routine returns the number of bytes in that BLOB or string.\n** ^If the result is a UTF-16 string, then sqlite3_column_bytes() converts\n** the string to UTF-8 and then returns the number of bytes.\n** ^If the result is a numeric value then sqlite3_column_bytes() uses\n** [sqlite3_snprintf()] to convert that value to a UTF-8 string and returns\n** the number of bytes in that string.\n** ^If the result is NULL, then sqlite3_column_bytes() returns zero.\n**\n** ^If the result is a BLOB or UTF-16 string then the sqlite3_column_bytes16()\n** routine returns the number of bytes in that BLOB or string.\n** ^If the result is a UTF-8 string, then sqlite3_column_bytes16() converts\n** the string to UTF-16 and then returns the number of bytes.\n** ^If the result is a numeric value then sqlite3_column_bytes16() uses\n** [sqlite3_snprintf()] to convert that value to a UTF-16 string and returns\n** the number of bytes in that string.\n** ^If the result is NULL, then sqlite3_column_bytes16() returns zero.\n**\n** ^The values returned by [sqlite3_column_bytes()] and\n** [sqlite3_column_bytes16()] do not include the zero terminators at the end\n** of the string. ^For clarity: the values returned by\n** [sqlite3_column_bytes()] and [sqlite3_column_bytes16()] are the number of\n** bytes in the string, not the number of characters.\n**\n** ^Strings returned by sqlite3_column_text() and sqlite3_column_text16(),\n** even empty strings, are always zero-terminated. ^The return\n** value from sqlite3_column_blob() for a zero-length BLOB is a NULL pointer.\n**\n** ^Strings returned by sqlite3_column_text16() always have the endianness\n** which is native to the platform, regardless of the text encoding set\n** for the database.\n**\n** Warning: ^The object returned by [sqlite3_column_value()] is an\n** [unprotected sqlite3_value] object. In a multithreaded environment,\n** an unprotected sqlite3_value object may only be used safely with\n** [sqlite3_bind_value()] and [sqlite3_result_value()].\n** If the [unprotected sqlite3_value] object returned by\n** [sqlite3_column_value()] is used in any other way, including calls\n** to routines like [sqlite3_value_int()], [sqlite3_value_text()],\n** or [sqlite3_value_bytes()], the behavior is not threadsafe.\n** Hence, the sqlite3_column_value() interface\n** is normally only useful within the implementation of\n** [application-defined SQL functions] or [virtual tables], not within\n** top-level application code.\n**\n** These routines may attempt to convert the datatype of the result.\n** ^For example, if the internal representation is FLOAT and a text result\n** is requested, [sqlite3_snprintf()] is used internally to perform the\n** conversion automatically. ^(The following table details the conversions\n** that are applied:\n**\n**
\n** \n**
Internal
Type 		Requested
Type 		Conversion\n**\n**
NULL INTEGER Result is 0\n**
NULL FLOAT Result is 0.0\n**
NULL TEXT Result is a NULL pointer\n**
NULL BLOB Result is a NULL pointer\n**
INTEGER FLOAT Convert from integer to float\n**
INTEGER TEXT ASCII rendering of the integer\n**
INTEGER BLOB Same as INTEGER->TEXT\n**
FLOAT INTEGER [CAST] to INTEGER\n**
FLOAT TEXT ASCII rendering of the float\n**
FLOAT BLOB [CAST] to BLOB\n**
TEXT INTEGER [CAST] to INTEGER\n**
TEXT FLOAT [CAST] to REAL\n**
TEXT BLOB No change\n**
BLOB INTEGER [CAST] to INTEGER\n**
BLOB FLOAT [CAST] to REAL\n**
BLOB TEXT [CAST] to TEXT, ensure zero terminator\n**
\n**
)^\n**\n** Note that when type conversions occur, pointers returned by prior\n** calls to sqlite3_column_blob(), sqlite3_column_text(), and/or\n** sqlite3_column_text16() may be invalidated.\n** Type conversions and pointer invalidations might occur\n** in the following cases:\n**\n**
    \n**
  • The initial content is a BLOB and sqlite3_column_text() or\n** sqlite3_column_text16() is called. A zero-terminator might\n** need to be added to the string.
    • \n**
  • The initial content is UTF-8 text and sqlite3_column_bytes16() or\n** sqlite3_column_text16() is called. The content must be converted\n** to UTF-16.
    • \n**
  • The initial content is UTF-16 text and sqlite3_column_bytes() or\n** sqlite3_column_text() is called. The content must be converted\n** to UTF-8.
    • \n**
\n**\n** ^Conversions between UTF-16be and UTF-16le are always done in place and do\n** not invalidate a prior pointer, though of course the content of the buffer\n** that the prior pointer references will have been modified. Other kinds\n** of conversion are done in place when it is possible, but sometimes they\n** are not possible and in those cases prior pointers are invalidated.\n**\n** The safest policy is to invoke these routines\n** in one of the following ways:\n**\n**
    \n**
  • sqlite3_column_text() followed by sqlite3_column_bytes()
    • \n**
  • sqlite3_column_blob() followed by sqlite3_column_bytes()
    • \n**
  • sqlite3_column_text16() followed by sqlite3_column_bytes16()
    • \n**
\n**\n** In other words, you should call sqlite3_column_text(),\n** sqlite3_column_blob(), or sqlite3_column_text16() first to force the result\n** into the desired format, then invoke sqlite3_column_bytes() or\n** sqlite3_column_bytes16() to find the size of the result. Do not mix calls\n** to sqlite3_column_text() or sqlite3_column_blob() with calls to\n** sqlite3_column_bytes16(), and do not mix calls to sqlite3_column_text16()\n** with calls to sqlite3_column_bytes().\n**\n** ^The pointers returned are valid until a type conversion occurs as\n** described above, or until [sqlite3_step()] or [sqlite3_reset()] or\n** [sqlite3_finalize()] is called. ^The memory space used to hold strings\n** and BLOBs is freed automatically. Do not pass the pointers returned\n** from [sqlite3_column_blob()], [sqlite3_column_text()], etc. into\n** [sqlite3_free()].\n**\n** As long as the input parameters are correct, these routines will only\n** fail if an out-of-memory error occurs during a format conversion.\n** Only the following subset of interfaces are subject to out-of-memory\n** errors:\n**\n**
    \n**
  • sqlite3_column_blob()\n**
  • sqlite3_column_text()\n**
  • sqlite3_column_text16()\n**
  • sqlite3_column_bytes()\n**
  • sqlite3_column_bytes16()\n**
\n**\n** If an out-of-memory error occurs, then the return value from these\n** routines is the same as if the column had contained an SQL NULL value.\n** Valid SQL NULL returns can be distinguished from out-of-memory errors\n** by invoking the [sqlite3_errcode()] immediately after the suspect\n** return value is obtained and before any\n** other SQLite interface is called on the same [database connection].\n*/\nSQLITE_API const void *sqlite3_column_blob(sqlite3_stmt*, int iCol);\nSQLITE_API double sqlite3_column_double(sqlite3_stmt*, int iCol);\nSQLITE_API int sqlite3_column_int(sqlite3_stmt*, int iCol);\nSQLITE_API sqlite3_int64 sqlite3_column_int64(sqlite3_stmt*, int iCol);\nSQLITE_API const unsigned char *sqlite3_column_text(sqlite3_stmt*, int iCol);\nSQLITE_API const void *sqlite3_column_text16(sqlite3_stmt*, int iCol);\nSQLITE_API sqlite3_value *sqlite3_column_value(sqlite3_stmt*, int iCol);\nSQLITE_API int sqlite3_column_bytes(sqlite3_stmt*, int iCol);\nSQLITE_API int sqlite3_column_bytes16(sqlite3_stmt*, int iCol);\nSQLITE_API int sqlite3_column_type(sqlite3_stmt*, int iCol);\n\n/*\n** CAPI3REF: Destroy A Prepared Statement Object\n** DESTRUCTOR: sqlite3_stmt\n**\n** ^The sqlite3_finalize() function is called to delete a [prepared statement].\n** ^If the most recent evaluation of the statement encountered no errors\n** or if the statement is never been evaluated, then sqlite3_finalize() returns\n** SQLITE_OK. ^If the most recent evaluation of statement S failed, then\n** sqlite3_finalize(S) returns the appropriate [error code] or\n** [extended error code].\n**\n** ^The sqlite3_finalize(S) routine can be called at any point during\n** the life cycle of [prepared statement] S:\n** before statement S is ever evaluated, after\n** one or more calls to [sqlite3_reset()], or after any call\n** to [sqlite3_step()] regardless of whether or not the statement has\n** completed execution.\n**\n** ^Invoking sqlite3_finalize() on a NULL pointer is a harmless no-op.\n**\n** The application must finalize every [prepared statement] in order to avoid\n** resource leaks. It is a grievous error for the application to try to use\n** a prepared statement after it has been finalized. Any use of a prepared\n** statement after it has been finalized can result in undefined and\n** undesirable behavior such as segfaults and heap corruption.\n*/\nSQLITE_API int sqlite3_finalize(sqlite3_stmt *pStmt);\n\n/*\n** CAPI3REF: Reset A Prepared Statement Object\n** METHOD: sqlite3_stmt\n**\n** The sqlite3_reset() function is called to reset a [prepared statement]\n** object back to its initial state, ready to be re-executed.\n** ^Any SQL statement variables that had values bound to them using\n** the [sqlite3_bind_blob | sqlite3_bind_*() API] retain their values.\n** Use [sqlite3_clear_bindings()] to reset the bindings.\n**\n** ^The [sqlite3_reset(S)] interface resets the [prepared statement] S\n** back to the beginning of its program.\n**\n** ^The return code from [sqlite3_reset(S)] indicates whether or not\n** the previous evaluation of prepared statement S completed successfully.\n** ^If [sqlite3_step(S)] has never before been called on S or if\n** [sqlite3_step(S)] has not been called since the previous call\n** to [sqlite3_reset(S)], then [sqlite3_reset(S)] will return\n** [SQLITE_OK].\n**\n** ^If the most recent call to [sqlite3_step(S)] for the\n** [prepared statement] S indicated an error, then\n** [sqlite3_reset(S)] returns an appropriate [error code].\n** ^The [sqlite3_reset(S)] interface might also return an [error code]\n** if there were no prior errors but the process of resetting\n** the prepared statement caused a new error. ^For example, if an\n** [INSERT] statement with a [RETURNING] clause is only stepped one time,\n** that one call to [sqlite3_step(S)] might return SQLITE_ROW but\n** the overall statement might still fail and the [sqlite3_reset(S)] call\n** might return SQLITE_BUSY if locking constraints prevent the\n** database change from committing. Therefore, it is important that\n** applications check the return code from [sqlite3_reset(S)] even if\n** no prior call to [sqlite3_step(S)] indicated a problem.\n**\n** ^The [sqlite3_reset(S)] interface does not change the values\n** of any [sqlite3_bind_blob|bindings] on the [prepared statement] S.\n*/\nSQLITE_API int sqlite3_reset(sqlite3_stmt *pStmt);\n\n\n/*\n** CAPI3REF: Create Or Redefine SQL Functions\n** KEYWORDS: {function creation routines}\n** METHOD: sqlite3\n**\n** ^These functions (collectively known as \"function creation routines\")\n** are used to add SQL functions or aggregates or to redefine the behavior\n** of existing SQL functions or aggregates. The only differences between\n** the three \"sqlite3_create_function*\" routines are the text encoding\n** expected for the second parameter (the name of the function being\n** created) and the presence or absence of a destructor callback for\n** the application data pointer. Function sqlite3_create_window_function()\n** is similar, but allows the user to supply the extra callback functions\n** needed by [aggregate window functions].\n**\n** ^The first parameter is the [database connection] to which the SQL\n** function is to be added. ^If an application uses more than one database\n** connection then application-defined SQL functions must be added\n** to each database connection separately.\n**\n** ^The second parameter is the name of the SQL function to be created or\n** redefined. ^The length of the name is limited to 255 bytes in a UTF-8\n** representation, exclusive of the zero-terminator. ^Note that the name\n** length limit is in UTF-8 bytes, not characters nor UTF-16 bytes.\n** ^Any attempt to create a function with a longer name\n** will result in [SQLITE_MISUSE] being returned.\n**\n** ^The third parameter (nArg)\n** is the number of arguments that the SQL function or\n** aggregate takes. ^If this parameter is -1, then the SQL function or\n** aggregate may take any number of arguments between 0 and the limit\n** set by [sqlite3_limit]([SQLITE_LIMIT_FUNCTION_ARG]). If the third\n** parameter is less than -1 or greater than 127 then the behavior is\n** undefined.\n**\n** ^The fourth parameter, eTextRep, specifies what\n** [SQLITE_UTF8 | text encoding] this SQL function prefers for\n** its parameters. The application should set this parameter to\n** [SQLITE_UTF16LE] if the function implementation invokes\n** [sqlite3_value_text16le()] on an input, or [SQLITE_UTF16BE] if the\n** implementation invokes [sqlite3_value_text16be()] on an input, or\n** [SQLITE_UTF16] if [sqlite3_value_text16()] is used, or [SQLITE_UTF8]\n** otherwise. ^The same SQL function may be registered multiple times using\n** different preferred text encodings, with different implementations for\n** each encoding.\n** ^When multiple implementations of the same function are available, SQLite\n** will pick the one that involves the least amount of data conversion.\n**\n** ^The fourth parameter may optionally be ORed with [SQLITE_DETERMINISTIC]\n** to signal that the function will always return the same result given\n** the same inputs within a single SQL statement. Most SQL functions are\n** deterministic. The built-in [random()] SQL function is an example of a\n** function that is not deterministic. The SQLite query planner is able to\n** perform additional optimizations on deterministic functions, so use\n** of the [SQLITE_DETERMINISTIC] flag is recommended where possible.\n**\n** ^The fourth parameter may also optionally include the [SQLITE_DIRECTONLY]\n** flag, which if present prevents the function from being invoked from\n** within VIEWs, TRIGGERs, CHECK constraints, generated column expressions,\n** index expressions, or the WHERE clause of partial indexes.\n**\n** For best security, the [SQLITE_DIRECTONLY] flag is recommended for\n** all application-defined SQL functions that do not need to be\n** used inside of triggers, views, CHECK constraints, or other elements of\n** the database schema. This flag is especially recommended for SQL\n** functions that have side effects or reveal internal application state.\n** Without this flag, an attacker might be able to modify the schema of\n** a database file to include invocations of the function with parameters\n** chosen by the attacker, which the application will then execute when\n** the database file is opened and read.\n**\n** ^(The fifth parameter is an arbitrary pointer. The implementation of the\n** function can gain access to this pointer using [sqlite3_user_data()].)^\n**\n** ^The sixth, seventh and eighth parameters passed to the three\n** \"sqlite3_create_function*\" functions, xFunc, xStep and xFinal, are\n** pointers to C-language functions that implement the SQL function or\n** aggregate. ^A scalar SQL function requires an implementation of the xFunc\n** callback only; NULL pointers must be passed as the xStep and xFinal\n** parameters. ^An aggregate SQL function requires an implementation of xStep\n** and xFinal and NULL pointer must be passed for xFunc. ^To delete an existing\n** SQL function or aggregate, pass NULL pointers for all three function\n** callbacks.\n**\n** ^The sixth, seventh, eighth and ninth parameters (xStep, xFinal, xValue\n** and xInverse) passed to sqlite3_create_window_function are pointers to\n** C-language callbacks that implement the new function. xStep and xFinal\n** must both be non-NULL. xValue and xInverse may either both be NULL, in\n** which case a regular aggregate function is created, or must both be\n** non-NULL, in which case the new function may be used as either an aggregate\n** or aggregate window function. More details regarding the implementation\n** of aggregate window functions are\n** [user-defined window functions|available here].\n**\n** ^(If the final parameter to sqlite3_create_function_v2() or\n** sqlite3_create_window_function() is not NULL, then it is the destructor for\n** the application data pointer. The destructor is invoked when the function\n** is deleted, either by being overloaded or when the database connection\n** closes.)^ ^The destructor is also invoked if the call to\n** sqlite3_create_function_v2() fails. ^When the destructor callback is\n** invoked, it is passed a single argument which is a copy of the application\n** data pointer which was the fifth parameter to sqlite3_create_function_v2().\n**\n** ^It is permitted to register multiple implementations of the same\n** functions with the same name but with either differing numbers of\n** arguments or differing preferred text encodings. ^SQLite will use\n** the implementation that most closely matches the way in which the\n** SQL function is used. ^A function implementation with a non-negative\n** nArg parameter is a better match than a function implementation with\n** a negative nArg. ^A function where the preferred text encoding\n** matches the database encoding is a better\n** match than a function where the encoding is different.\n** ^A function where the encoding difference is between UTF16le and UTF16be\n** is a closer match than a function where the encoding difference is\n** between UTF8 and UTF16.\n**\n** ^Built-in functions may be overloaded by new application-defined functions.\n**\n** ^An application-defined function is permitted to call other\n** SQLite interfaces. However, such calls must not\n** close the database connection nor finalize or reset the prepared\n** statement in which the function is running.\n*/\nSQLITE_API int sqlite3_create_function(\n sqlite3 *db,\n const char *zFunctionName,\n int nArg,\n int eTextRep,\n void *pApp,\n void (*xFunc)(sqlite3_context*,int,sqlite3_value**),\n void (*xStep)(sqlite3_context*,int,sqlite3_value**),\n void (*xFinal)(sqlite3_context*)\n);\nSQLITE_API int sqlite3_create_function16(\n sqlite3 *db,\n const void *zFunctionName,\n int nArg,\n int eTextRep,\n void *pApp,\n void (*xFunc)(sqlite3_context*,int,sqlite3_value**),\n void (*xStep)(sqlite3_context*,int,sqlite3_value**),\n void (*xFinal)(sqlite3_context*)\n);\nSQLITE_API int sqlite3_create_function_v2(\n sqlite3 *db,\n const char *zFunctionName,\n int nArg,\n int eTextRep,\n void *pApp,\n void (*xFunc)(sqlite3_context*,int,sqlite3_value**),\n void (*xStep)(sqlite3_context*,int,sqlite3_value**),\n void (*xFinal)(sqlite3_context*),\n void(*xDestroy)(void*)\n);\nSQLITE_API int sqlite3_create_window_function(\n sqlite3 *db,\n const char *zFunctionName,\n int nArg,\n int eTextRep,\n void *pApp,\n void (*xStep)(sqlite3_context*,int,sqlite3_value**),\n void (*xFinal)(sqlite3_context*),\n void (*xValue)(sqlite3_context*),\n void (*xInverse)(sqlite3_context*,int,sqlite3_value**),\n void(*xDestroy)(void*)\n);\n\n/*\n** CAPI3REF: Text Encodings\n**\n** These constant define integer codes that represent the various\n** text encodings supported by SQLite.\n*/\n#define SQLITE_UTF8 1 /* IMP: R-37514-35566 */\n#define SQLITE_UTF16LE 2 /* IMP: R-03371-37637 */\n#define SQLITE_UTF16BE 3 /* IMP: R-51971-34154 */\n#define SQLITE_UTF16 4 /* Use native byte order */\n#define SQLITE_ANY 5 /* Deprecated */\n#define SQLITE_UTF16_ALIGNED 8 /* sqlite3_create_collation only */\n\n/*\n** CAPI3REF: Function Flags\n**\n** These constants may be ORed together with the\n** [SQLITE_UTF8 | preferred text encoding] as the fourth argument\n** to [sqlite3_create_function()], [sqlite3_create_function16()], or\n** [sqlite3_create_function_v2()].\n**\n**
\n** [[SQLITE_DETERMINISTIC]]
SQLITE_DETERMINISTIC
\n** The SQLITE_DETERMINISTIC flag means that the new function always gives\n** the same output when the input parameters are the same.\n** The [abs|abs() function] is deterministic, for example, but\n** [randomblob|randomblob()] is not. Functions must\n** be deterministic in order to be used in certain contexts such as\n** with the WHERE clause of [partial indexes] or in [generated columns].\n** SQLite might also optimize deterministic functions by factoring them\n** out of inner loops.\n**
\n**\n** [[SQLITE_DIRECTONLY]]
SQLITE_DIRECTONLY
\n** The SQLITE_DIRECTONLY flag means that the function may only be invoked\n** from top-level SQL, and cannot be used in VIEWs or TRIGGERs nor in\n** schema structures such as [CHECK constraints], [DEFAULT clauses],\n** [expression indexes], [partial indexes], or [generated columns].\n**

\n** The SQLITE_DIRECTONLY flag is recommended for any\n** [application-defined SQL function]\n** that has side-effects or that could potentially leak sensitive information.\n** This will prevent attacks in which an application is tricked\n** into using a database file that has had its schema surreptitiously\n** modified to invoke the application-defined function in ways that are\n** harmful.\n**

\n** Some people say it is good practice to set SQLITE_DIRECTONLY on all\n** [application-defined SQL functions], regardless of whether or not they\n** are security sensitive, as doing so prevents those functions from being used\n** inside of the database schema, and thus ensures that the database\n** can be inspected and modified using generic tools (such as the [CLI])\n** that do not have access to the application-defined functions.\n**

\n**\n** [[SQLITE_INNOCUOUS]]
SQLITE_INNOCUOUS
\n** The SQLITE_INNOCUOUS flag means that the function is unlikely\n** to cause problems even if misused. An innocuous function should have\n** no side effects and should not depend on any values other than its\n** input parameters. The [abs|abs() function] is an example of an\n** innocuous function.\n** The [load_extension() SQL function] is not innocuous because of its\n** side effects.\n**

SQLITE_INNOCUOUS is similar to SQLITE_DETERMINISTIC, but is not\n** exactly the same. The [random|random() function] is an example of a\n** function that is innocuous but not deterministic.\n**

Some heightened security settings\n** ([SQLITE_DBCONFIG_TRUSTED_SCHEMA] and [PRAGMA trusted_schema=OFF])\n** disable the use of SQL functions inside views and triggers and in\n** schema structures such as [CHECK constraints], [DEFAULT clauses],\n** [expression indexes], [partial indexes], and [generated columns] unless\n** the function is tagged with SQLITE_INNOCUOUS. Most built-in functions\n** are innocuous. Developers are advised to avoid using the\n** SQLITE_INNOCUOUS flag for application-defined functions unless the\n** function has been carefully audited and found to be free of potentially\n** security-adverse side-effects and information-leaks.\n**

\n**\n** [[SQLITE_SUBTYPE]]
SQLITE_SUBTYPE
\n** The SQLITE_SUBTYPE flag indicates to SQLite that a function might call\n** [sqlite3_value_subtype()] to inspect the sub-types of its arguments.\n** This flag instructs SQLite to omit some corner-case optimizations that\n** might disrupt the operation of the [sqlite3_value_subtype()] function,\n** causing it to return zero rather than the correct subtype().\n** All SQL functions that invoke [sqlite3_value_subtype()] should have this\n** property. If the SQLITE_SUBTYPE property is omitted, then the return\n** value from [sqlite3_value_subtype()] might sometimes be zero even though\n** a non-zero subtype was specified by the function argument expression.\n**\n** [[SQLITE_RESULT_SUBTYPE]]
SQLITE_RESULT_SUBTYPE
\n** The SQLITE_RESULT_SUBTYPE flag indicates to SQLite that a function might call\n** [sqlite3_result_subtype()] to cause a sub-type to be associated with its\n** result.\n** Every function that invokes [sqlite3_result_subtype()] should have this\n** property. If it does not, then the call to [sqlite3_result_subtype()]\n** might become a no-op if the function is used as term in an\n** [expression index]. On the other hand, SQL functions that never invoke\n** [sqlite3_result_subtype()] should avoid setting this property, as the\n** purpose of this property is to disable certain optimizations that are\n** incompatible with subtypes.\n**\n** [[SQLITE_SELFORDER1]]
SQLITE_SELFORDER1
\n** The SQLITE_SELFORDER1 flag indicates that the function is an aggregate\n** that internally orders the values provided to the first argument. The\n** ordered-set aggregate SQL notation with a single ORDER BY term can be\n** used to invoke this function. If the ordered-set aggregate notation is\n** used on a function that lacks this flag, then an error is raised. Note\n** that the ordered-set aggregate syntax is only available if SQLite is\n** built using the -DSQLITE_ENABLE_ORDERED_SET_AGGREGATES compile-time option.\n**
\n**
\n*/\n#define SQLITE_DETERMINISTIC 0x000000800\n#define SQLITE_DIRECTONLY 0x000080000\n#define SQLITE_SUBTYPE 0x000100000\n#define SQLITE_INNOCUOUS 0x000200000\n#define SQLITE_RESULT_SUBTYPE 0x001000000\n#define SQLITE_SELFORDER1 0x002000000\n\n/*\n** CAPI3REF: Deprecated Functions\n** DEPRECATED\n**\n** These functions are [deprecated]. In order to maintain\n** backwards compatibility with older code, these functions continue\n** to be supported. However, new applications should avoid\n** the use of these functions. To encourage programmers to avoid\n** these functions, we will not explain what they do.\n*/\n#ifndef SQLITE_OMIT_DEPRECATED\nSQLITE_API SQLITE_DEPRECATED int sqlite3_aggregate_count(sqlite3_context*);\nSQLITE_API SQLITE_DEPRECATED int sqlite3_expired(sqlite3_stmt*);\nSQLITE_API SQLITE_DEPRECATED int sqlite3_transfer_bindings(sqlite3_stmt*, sqlite3_stmt*);\nSQLITE_API SQLITE_DEPRECATED int sqlite3_global_recover(void);\nSQLITE_API SQLITE_DEPRECATED void sqlite3_thread_cleanup(void);\nSQLITE_API SQLITE_DEPRECATED int sqlite3_memory_alarm(void(*)(void*,sqlite3_int64,int),\n void*,sqlite3_int64);\n#endif\n\n/*\n** CAPI3REF: Obtaining SQL Values\n** METHOD: sqlite3_value\n**\n** Summary:\n**
\n**
sqlite3_value_blob→BLOB value\n**
sqlite3_value_double→REAL value\n**
sqlite3_value_int→32-bit INTEGER value\n**
sqlite3_value_int64→64-bit INTEGER value\n**
sqlite3_value_pointer→Pointer value\n**
sqlite3_value_text→UTF-8 TEXT value\n**
sqlite3_value_text16→UTF-16 TEXT value in\n** the native byteorder\n**
sqlite3_value_text16be→UTF-16be TEXT value\n**
sqlite3_value_text16le→UTF-16le TEXT value\n**
   \n**
sqlite3_value_bytes→Size of a BLOB\n** or a UTF-8 TEXT in bytes\n**
sqlite3_value_bytes16  \n** →  Size of UTF-16\n** TEXT in bytes\n**
sqlite3_value_type→Default\n** datatype of the value\n**
sqlite3_value_numeric_type  \n** →  Best numeric datatype of the value\n**
sqlite3_value_nochange  \n** →  True if the column is unchanged in an UPDATE\n** against a virtual table.\n**
sqlite3_value_frombind  \n** →  True if value originated from a [bound parameter]\n**
\n**\n** Details:\n**\n** These routines extract type, size, and content information from\n** [protected sqlite3_value] objects. Protected sqlite3_value objects\n** are used to pass parameter information into the functions that\n** implement [application-defined SQL functions] and [virtual tables].\n**\n** These routines work only with [protected sqlite3_value] objects.\n** Any attempt to use these routines on an [unprotected sqlite3_value]\n** is not threadsafe.\n**\n** ^These routines work just like the corresponding [column access functions]\n** except that these routines take a single [protected sqlite3_value] object\n** pointer instead of a [sqlite3_stmt*] pointer and an integer column number.\n**\n** ^The sqlite3_value_text16() interface extracts a UTF-16 string\n** in the native byte-order of the host machine. ^The\n** sqlite3_value_text16be() and sqlite3_value_text16le() interfaces\n** extract UTF-16 strings as big-endian and little-endian respectively.\n**\n** ^If [sqlite3_value] object V was initialized\n** using [sqlite3_bind_pointer(S,I,P,X,D)] or [sqlite3_result_pointer(C,P,X,D)]\n** and if X and Y are strings that compare equal according to strcmp(X,Y),\n** then sqlite3_value_pointer(V,Y) will return the pointer P. ^Otherwise,\n** sqlite3_value_pointer(V,Y) returns a NULL. The sqlite3_bind_pointer()\n** routine is part of the [pointer passing interface] added for SQLite 3.20.0.\n**\n** ^(The sqlite3_value_type(V) interface returns the\n** [SQLITE_INTEGER | datatype code] for the initial datatype of the\n** [sqlite3_value] object V. The returned value is one of [SQLITE_INTEGER],\n** [SQLITE_FLOAT], [SQLITE_TEXT], [SQLITE_BLOB], or [SQLITE_NULL].)^\n** Other interfaces might change the datatype for an sqlite3_value object.\n** For example, if the datatype is initially SQLITE_INTEGER and\n** sqlite3_value_text(V) is called to extract a text value for that\n** integer, then subsequent calls to sqlite3_value_type(V) might return\n** SQLITE_TEXT. Whether or not a persistent internal datatype conversion\n** occurs is undefined and may change from one release of SQLite to the next.\n**\n** ^(The sqlite3_value_numeric_type() interface attempts to apply\n** numeric affinity to the value. This means that an attempt is\n** made to convert the value to an integer or floating point. If\n** such a conversion is possible without loss of information (in other\n** words, if the value is a string that looks like a number)\n** then the conversion is performed. Otherwise no conversion occurs.\n** The [SQLITE_INTEGER | datatype] after conversion is returned.)^\n**\n** ^Within the [xUpdate] method of a [virtual table], the\n** sqlite3_value_nochange(X) interface returns true if and only if\n** the column corresponding to X is unchanged by the UPDATE operation\n** that the xUpdate method call was invoked to implement and if\n** and the prior [xColumn] method call that was invoked to extracted\n** the value for that column returned without setting a result (probably\n** because it queried [sqlite3_vtab_nochange()] and found that the column\n** was unchanging). ^Within an [xUpdate] method, any value for which\n** sqlite3_value_nochange(X) is true will in all other respects appear\n** to be a NULL value. If sqlite3_value_nochange(X) is invoked anywhere other\n** than within an [xUpdate] method call for an UPDATE statement, then\n** the return value is arbitrary and meaningless.\n**\n** ^The sqlite3_value_frombind(X) interface returns non-zero if the\n** value X originated from one of the [sqlite3_bind_int|sqlite3_bind()]\n** interfaces. ^If X comes from an SQL literal value, or a table column,\n** or an expression, then sqlite3_value_frombind(X) returns zero.\n**\n** Please pay particular attention to the fact that the pointer returned\n** from [sqlite3_value_blob()], [sqlite3_value_text()], or\n** [sqlite3_value_text16()] can be invalidated by a subsequent call to\n** [sqlite3_value_bytes()], [sqlite3_value_bytes16()], [sqlite3_value_text()],\n** or [sqlite3_value_text16()].\n**\n** These routines must be called from the same thread as\n** the SQL function that supplied the [sqlite3_value*] parameters.\n**\n** As long as the input parameter is correct, these routines can only\n** fail if an out-of-memory error occurs during a format conversion.\n** Only the following subset of interfaces are subject to out-of-memory\n** errors:\n**\n**
    \n**
  • sqlite3_value_blob()\n**
  • sqlite3_value_text()\n**
  • sqlite3_value_text16()\n**
  • sqlite3_value_text16le()\n**
  • sqlite3_value_text16be()\n**
  • sqlite3_value_bytes()\n**
  • sqlite3_value_bytes16()\n**
\n**\n** If an out-of-memory error occurs, then the return value from these\n** routines is the same as if the column had contained an SQL NULL value.\n** Valid SQL NULL returns can be distinguished from out-of-memory errors\n** by invoking the [sqlite3_errcode()] immediately after the suspect\n** return value is obtained and before any\n** other SQLite interface is called on the same [database connection].\n*/\nSQLITE_API const void *sqlite3_value_blob(sqlite3_value*);\nSQLITE_API double sqlite3_value_double(sqlite3_value*);\nSQLITE_API int sqlite3_value_int(sqlite3_value*);\nSQLITE_API sqlite3_int64 sqlite3_value_int64(sqlite3_value*);\nSQLITE_API void *sqlite3_value_pointer(sqlite3_value*, const char*);\nSQLITE_API const unsigned char *sqlite3_value_text(sqlite3_value*);\nSQLITE_API const void *sqlite3_value_text16(sqlite3_value*);\nSQLITE_API const void *sqlite3_value_text16le(sqlite3_value*);\nSQLITE_API const void *sqlite3_value_text16be(sqlite3_value*);\nSQLITE_API int sqlite3_value_bytes(sqlite3_value*);\nSQLITE_API int sqlite3_value_bytes16(sqlite3_value*);\nSQLITE_API int sqlite3_value_type(sqlite3_value*);\nSQLITE_API int sqlite3_value_numeric_type(sqlite3_value*);\nSQLITE_API int sqlite3_value_nochange(sqlite3_value*);\nSQLITE_API int sqlite3_value_frombind(sqlite3_value*);\n\n/*\n** CAPI3REF: Report the internal text encoding state of an sqlite3_value object\n** METHOD: sqlite3_value\n**\n** ^(The sqlite3_value_encoding(X) interface returns one of [SQLITE_UTF8],\n** [SQLITE_UTF16BE], or [SQLITE_UTF16LE] according to the current text encoding\n** of the value X, assuming that X has type TEXT.)^ If sqlite3_value_type(X)\n** returns something other than SQLITE_TEXT, then the return value from\n** sqlite3_value_encoding(X) is meaningless. ^Calls to\n** [sqlite3_value_text(X)], [sqlite3_value_text16(X)], [sqlite3_value_text16be(X)],\n** [sqlite3_value_text16le(X)], [sqlite3_value_bytes(X)], or\n** [sqlite3_value_bytes16(X)] might change the encoding of the value X and\n** thus change the return from subsequent calls to sqlite3_value_encoding(X).\n**\n** This routine is intended for used by applications that test and validate\n** the SQLite implementation. This routine is inquiring about the opaque\n** internal state of an [sqlite3_value] object. Ordinary applications should\n** not need to know what the internal state of an sqlite3_value object is and\n** hence should not need to use this interface.\n*/\nSQLITE_API int sqlite3_value_encoding(sqlite3_value*);\n\n/*\n** CAPI3REF: Finding The Subtype Of SQL Values\n** METHOD: sqlite3_value\n**\n** The sqlite3_value_subtype(V) function returns the subtype for\n** an [application-defined SQL function] argument V. The subtype\n** information can be used to pass a limited amount of context from\n** one SQL function to another. Use the [sqlite3_result_subtype()]\n** routine to set the subtype for the return value of an SQL function.\n**\n** Every [application-defined SQL function] that invokes this interface\n** should include the [SQLITE_SUBTYPE] property in the text\n** encoding argument when the function is [sqlite3_create_function|registered].\n** If the [SQLITE_SUBTYPE] property is omitted, then sqlite3_value_subtype()\n** might return zero instead of the upstream subtype in some corner cases.\n*/\nSQLITE_API unsigned int sqlite3_value_subtype(sqlite3_value*);\n\n/*\n** CAPI3REF: Copy And Free SQL Values\n** METHOD: sqlite3_value\n**\n** ^The sqlite3_value_dup(V) interface makes a copy of the [sqlite3_value]\n** object V and returns a pointer to that copy. ^The [sqlite3_value] returned\n** is a [protected sqlite3_value] object even if the input is not.\n** ^The sqlite3_value_dup(V) interface returns NULL if V is NULL or if a\n** memory allocation fails. ^If V is a [pointer value], then the result\n** of sqlite3_value_dup(V) is a NULL value.\n**\n** ^The sqlite3_value_free(V) interface frees an [sqlite3_value] object\n** previously obtained from [sqlite3_value_dup()]. ^If V is a NULL pointer\n** then sqlite3_value_free(V) is a harmless no-op.\n*/\nSQLITE_API sqlite3_value *sqlite3_value_dup(const sqlite3_value*);\nSQLITE_API void sqlite3_value_free(sqlite3_value*);\n\n/*\n** CAPI3REF: Obtain Aggregate Function Context\n** METHOD: sqlite3_context\n**\n** Implementations of aggregate SQL functions use this\n** routine to allocate memory for storing their state.\n**\n** ^The first time the sqlite3_aggregate_context(C,N) routine is called\n** for a particular aggregate function, SQLite allocates\n** N bytes of memory, zeroes out that memory, and returns a pointer\n** to the new memory. ^On second and subsequent calls to\n** sqlite3_aggregate_context() for the same aggregate function instance,\n** the same buffer is returned. Sqlite3_aggregate_context() is normally\n** called once for each invocation of the xStep callback and then one\n** last time when the xFinal callback is invoked. ^(When no rows match\n** an aggregate query, the xStep() callback of the aggregate function\n** implementation is never called and xFinal() is called exactly once.\n** In those cases, sqlite3_aggregate_context() might be called for the\n** first time from within xFinal().)^\n**\n** ^The sqlite3_aggregate_context(C,N) routine returns a NULL pointer\n** when first called if N is less than or equal to zero or if a memory\n** allocation error occurs.\n**\n** ^(The amount of space allocated by sqlite3_aggregate_context(C,N) is\n** determined by the N parameter on the first successful call. Changing the\n** value of N in any subsequent call to sqlite3_aggregate_context() within\n** the same aggregate function instance will not resize the memory\n** allocation.)^ Within the xFinal callback, it is customary to set\n** N=0 in calls to sqlite3_aggregate_context(C,N) so that no\n** pointless memory allocations occur.\n**\n** ^SQLite automatically frees the memory allocated by\n** sqlite3_aggregate_context() when the aggregate query concludes.\n**\n** The first parameter must be a copy of the\n** [sqlite3_context | SQL function context] that is the first parameter\n** to the xStep or xFinal callback routine that implements the aggregate\n** function.\n**\n** This routine must be called from the same thread in which\n** the aggregate SQL function is running.\n*/\nSQLITE_API void *sqlite3_aggregate_context(sqlite3_context*, int nBytes);\n\n/*\n** CAPI3REF: User Data For Functions\n** METHOD: sqlite3_context\n**\n** ^The sqlite3_user_data() interface returns a copy of\n** the pointer that was the pUserData parameter (the 5th parameter)\n** of the [sqlite3_create_function()]\n** and [sqlite3_create_function16()] routines that originally\n** registered the application defined function.\n**\n** This routine must be called from the same thread in which\n** the application-defined function is running.\n*/\nSQLITE_API void *sqlite3_user_data(sqlite3_context*);\n\n/*\n** CAPI3REF: Database Connection For Functions\n** METHOD: sqlite3_context\n**\n** ^The sqlite3_context_db_handle() interface returns a copy of\n** the pointer to the [database connection] (the 1st parameter)\n** of the [sqlite3_create_function()]\n** and [sqlite3_create_function16()] routines that originally\n** registered the application defined function.\n*/\nSQLITE_API sqlite3 *sqlite3_context_db_handle(sqlite3_context*);\n\n/*\n** CAPI3REF: Function Auxiliary Data\n** METHOD: sqlite3_context\n**\n** These functions may be used by (non-aggregate) SQL functions to\n** associate auxiliary data with argument values. If the same argument\n** value is passed to multiple invocations of the same SQL function during\n** query execution, under some circumstances the associated auxiliary data\n** might be preserved. An example of where this might be useful is in a\n** regular-expression matching function. The compiled version of the regular\n** expression can be stored as auxiliary data associated with the pattern string.\n** Then as long as the pattern string remains the same,\n** the compiled regular expression can be reused on multiple\n** invocations of the same function.\n**\n** ^The sqlite3_get_auxdata(C,N) interface returns a pointer to the auxiliary data\n** associated by the sqlite3_set_auxdata(C,N,P,X) function with the Nth argument\n** value to the application-defined function. ^N is zero for the left-most\n** function argument. ^If there is no auxiliary data\n** associated with the function argument, the sqlite3_get_auxdata(C,N) interface\n** returns a NULL pointer.\n**\n** ^The sqlite3_set_auxdata(C,N,P,X) interface saves P as auxiliary data for the\n** N-th argument of the application-defined function. ^Subsequent\n** calls to sqlite3_get_auxdata(C,N) return P from the most recent\n** sqlite3_set_auxdata(C,N,P,X) call if the auxiliary data is still valid or\n** NULL if the auxiliary data has been discarded.\n** ^After each call to sqlite3_set_auxdata(C,N,P,X) where X is not NULL,\n** SQLite will invoke the destructor function X with parameter P exactly\n** once, when the auxiliary data is discarded.\n** SQLite is free to discard the auxiliary data at any time, including:
    \n**
  • ^(when the corresponding function parameter changes)^, or\n**
  • ^(when [sqlite3_reset()] or [sqlite3_finalize()] is called for the\n** SQL statement)^, or\n**
  • ^(when sqlite3_set_auxdata() is invoked again on the same\n** parameter)^, or\n**
  • ^(during the original sqlite3_set_auxdata() call when a memory\n** allocation error occurs.)^\n**
  • ^(during the original sqlite3_set_auxdata() call if the function\n** is evaluated during query planning instead of during query execution,\n** as sometimes happens with [SQLITE_ENABLE_STAT4].)^
\n**\n** Note the last two bullets in particular. The destructor X in\n** sqlite3_set_auxdata(C,N,P,X) might be called immediately, before the\n** sqlite3_set_auxdata() interface even returns. Hence sqlite3_set_auxdata()\n** should be called near the end of the function implementation and the\n** function implementation should not make any use of P after\n** sqlite3_set_auxdata() has been called. Furthermore, a call to\n** sqlite3_get_auxdata() that occurs immediately after a corresponding call\n** to sqlite3_set_auxdata() might still return NULL if an out-of-memory\n** condition occurred during the sqlite3_set_auxdata() call or if the\n** function is being evaluated during query planning rather than during\n** query execution.\n**\n** ^(In practice, auxiliary data is preserved between function calls for\n** function parameters that are compile-time constants, including literal\n** values and [parameters] and expressions composed from the same.)^\n**\n** The value of the N parameter to these interfaces should be non-negative.\n** Future enhancements may make use of negative N values to define new\n** kinds of function caching behavior.\n**\n** These routines must be called from the same thread in which\n** the SQL function is running.\n**\n** See also: [sqlite3_get_clientdata()] and [sqlite3_set_clientdata()].\n*/\nSQLITE_API void *sqlite3_get_auxdata(sqlite3_context*, int N);\nSQLITE_API void sqlite3_set_auxdata(sqlite3_context*, int N, void*, void (*)(void*));\n\n/*\n** CAPI3REF: Database Connection Client Data\n** METHOD: sqlite3\n**\n** These functions are used to associate one or more named pointers\n** with a [database connection].\n** A call to sqlite3_set_clientdata(D,N,P,X) causes the pointer P\n** to be attached to [database connection] D using name N. Subsequent\n** calls to sqlite3_get_clientdata(D,N) will return a copy of pointer P\n** or a NULL pointer if there were no prior calls to\n** sqlite3_set_clientdata() with the same values of D and N.\n** Names are compared using strcmp() and are thus case sensitive.\n**\n** If P and X are both non-NULL, then the destructor X is invoked with\n** argument P on the first of the following occurrences:\n**
    \n**
  • An out-of-memory error occurs during the call to\n** sqlite3_set_clientdata() which attempts to register pointer P.\n**
  • A subsequent call to sqlite3_set_clientdata(D,N,P,X) is made\n** with the same D and N parameters.\n**
  • The database connection closes. SQLite does not make any guarantees\n** about the order in which destructors are called, only that all\n** destructors will be called exactly once at some point during the\n** database connection closing process.\n**
\n**\n** SQLite does not do anything with client data other than invoke\n** destructors on the client data at the appropriate time. The intended\n** use for client data is to provide a mechanism for wrapper libraries\n** to store additional information about an SQLite database connection.\n**\n** There is no limit (other than available memory) on the number of different\n** client data pointers (with different names) that can be attached to a\n** single database connection. However, the implementation is optimized\n** for the case of having only one or two different client data names.\n** Applications and wrapper libraries are discouraged from using more than\n** one client data name each.\n**\n** There is no way to enumerate the client data pointers\n** associated with a database connection. The N parameter can be thought\n** of as a secret key such that only code that knows the secret key is able\n** to access the associated data.\n**\n** Security Warning: These interfaces should not be exposed in scripting\n** languages or in other circumstances where it might be possible for an\n** attacker to invoke them. Any agent that can invoke these interfaces\n** can probably also take control of the process.\n**\n** Database connection client data is only available for SQLite\n** version 3.44.0 ([dateof:3.44.0]) and later.\n**\n** See also: [sqlite3_set_auxdata()] and [sqlite3_get_auxdata()].\n*/\nSQLITE_API void *sqlite3_get_clientdata(sqlite3*,const char*);\nSQLITE_API int sqlite3_set_clientdata(sqlite3*, const char*, void*, void(*)(void*));\n\n/*\n** CAPI3REF: Constants Defining Special Destructor Behavior\n**\n** These are special values for the destructor that is passed in as the\n** final argument to routines like [sqlite3_result_blob()]. ^If the destructor\n** argument is SQLITE_STATIC, it means that the content pointer is constant\n** and will never change. It does not need to be destroyed. ^The\n** SQLITE_TRANSIENT value means that the content will likely change in\n** the near future and that SQLite should make its own private copy of\n** the content before returning.\n**\n** The typedef is necessary to work around problems in certain\n** C++ compilers.\n*/\ntypedef void (*sqlite3_destructor_type)(void*);\n#define SQLITE_STATIC ((sqlite3_destructor_type)0)\n#define SQLITE_TRANSIENT ((sqlite3_destructor_type)-1)\n\n/*\n** CAPI3REF: Setting The Result Of An SQL Function\n** METHOD: sqlite3_context\n**\n** These routines are used by the xFunc or xFinal callbacks that\n** implement SQL functions and aggregates. See\n** [sqlite3_create_function()] and [sqlite3_create_function16()]\n** for additional information.\n**\n** These functions work very much like the [parameter binding] family of\n** functions used to bind values to host parameters in prepared statements.\n** Refer to the [SQL parameter] documentation for additional information.\n**\n** ^The sqlite3_result_blob() interface sets the result from\n** an application-defined function to be the BLOB whose content is pointed\n** to by the second parameter and which is N bytes long where N is the\n** third parameter.\n**\n** ^The sqlite3_result_zeroblob(C,N) and sqlite3_result_zeroblob64(C,N)\n** interfaces set the result of the application-defined function to be\n** a BLOB containing all zero bytes and N bytes in size.\n**\n** ^The sqlite3_result_double() interface sets the result from\n** an application-defined function to be a floating point value specified\n** by its 2nd argument.\n**\n** ^The sqlite3_result_error() and sqlite3_result_error16() functions\n** cause the implemented SQL function to throw an exception.\n** ^SQLite uses the string pointed to by the\n** 2nd parameter of sqlite3_result_error() or sqlite3_result_error16()\n** as the text of an error message. ^SQLite interprets the error\n** message string from sqlite3_result_error() as UTF-8. ^SQLite\n** interprets the string from sqlite3_result_error16() as UTF-16 using\n** the same [byte-order determination rules] as [sqlite3_bind_text16()].\n** ^If the third parameter to sqlite3_result_error()\n** or sqlite3_result_error16() is negative then SQLite takes as the error\n** message all text up through the first zero character.\n** ^If the third parameter to sqlite3_result_error() or\n** sqlite3_result_error16() is non-negative then SQLite takes that many\n** bytes (not characters) from the 2nd parameter as the error message.\n** ^The sqlite3_result_error() and sqlite3_result_error16()\n** routines make a private copy of the error message text before\n** they return. Hence, the calling function can deallocate or\n** modify the text after they return without harm.\n** ^The sqlite3_result_error_code() function changes the error code\n** returned by SQLite as a result of an error in a function. ^By default,\n** the error code is SQLITE_ERROR. ^A subsequent call to sqlite3_result_error()\n** or sqlite3_result_error16() resets the error code to SQLITE_ERROR.\n**\n** ^The sqlite3_result_error_toobig() interface causes SQLite to throw an\n** error indicating that a string or BLOB is too long to represent.\n**\n** ^The sqlite3_result_error_nomem() interface causes SQLite to throw an\n** error indicating that a memory allocation failed.\n**\n** ^The sqlite3_result_int() interface sets the return value\n** of the application-defined function to be the 32-bit signed integer\n** value given in the 2nd argument.\n** ^The sqlite3_result_int64() interface sets the return value\n** of the application-defined function to be the 64-bit signed integer\n** value given in the 2nd argument.\n**\n** ^The sqlite3_result_null() interface sets the return value\n** of the application-defined function to be NULL.\n**\n** ^The sqlite3_result_text(), sqlite3_result_text16(),\n** sqlite3_result_text16le(), and sqlite3_result_text16be() interfaces\n** set the return value of the application-defined function to be\n** a text string which is represented as UTF-8, UTF-16 native byte order,\n** UTF-16 little endian, or UTF-16 big endian, respectively.\n** ^The sqlite3_result_text64() interface sets the return value of an\n** application-defined function to be a text string in an encoding\n** specified by the fifth (and last) parameter, which must be one\n** of [SQLITE_UTF8], [SQLITE_UTF16], [SQLITE_UTF16BE], or [SQLITE_UTF16LE].\n** ^SQLite takes the text result from the application from\n** the 2nd parameter of the sqlite3_result_text* interfaces.\n** ^If the 3rd parameter to any of the sqlite3_result_text* interfaces\n** other than sqlite3_result_text64() is negative, then SQLite computes\n** the string length itself by searching the 2nd parameter for the first\n** zero character.\n** ^If the 3rd parameter to the sqlite3_result_text* interfaces\n** is non-negative, then as many bytes (not characters) of the text\n** pointed to by the 2nd parameter are taken as the application-defined\n** function result. If the 3rd parameter is non-negative, then it\n** must be the byte offset into the string where the NUL terminator would\n** appear if the string were NUL terminated. If any NUL characters occur\n** in the string at a byte offset that is less than the value of the 3rd\n** parameter, then the resulting string will contain embedded NULs and the\n** result of expressions operating on strings with embedded NULs is undefined.\n** ^If the 4th parameter to the sqlite3_result_text* interfaces\n** or sqlite3_result_blob is a non-NULL pointer, then SQLite calls that\n** function as the destructor on the text or BLOB result when it has\n** finished using that result.\n** ^If the 4th parameter to the sqlite3_result_text* interfaces or to\n** sqlite3_result_blob is the special constant SQLITE_STATIC, then SQLite\n** assumes that the text or BLOB result is in constant space and does not\n** copy the content of the parameter nor call a destructor on the content\n** when it has finished using that result.\n** ^If the 4th parameter to the sqlite3_result_text* interfaces\n** or sqlite3_result_blob is the special constant SQLITE_TRANSIENT\n** then SQLite makes a copy of the result into space obtained\n** from [sqlite3_malloc()] before it returns.\n**\n** ^For the sqlite3_result_text16(), sqlite3_result_text16le(), and\n** sqlite3_result_text16be() routines, and for sqlite3_result_text64()\n** when the encoding is not UTF8, if the input UTF16 begins with a\n** byte-order mark (BOM, U+FEFF) then the BOM is removed from the\n** string and the rest of the string is interpreted according to the\n** byte-order specified by the BOM. ^The byte-order specified by\n** the BOM at the beginning of the text overrides the byte-order\n** specified by the interface procedure. ^So, for example, if\n** sqlite3_result_text16le() is invoked with text that begins\n** with bytes 0xfe, 0xff (a big-endian byte-order mark) then the\n** first two bytes of input are skipped and the remaining input\n** is interpreted as UTF16BE text.\n**\n** ^For UTF16 input text to the sqlite3_result_text16(),\n** sqlite3_result_text16be(), sqlite3_result_text16le(), and\n** sqlite3_result_text64() routines, if the text contains invalid\n** UTF16 characters, the invalid characters might be converted\n** into the unicode replacement character, U+FFFD.\n**\n** ^The sqlite3_result_value() interface sets the result of\n** the application-defined function to be a copy of the\n** [unprotected sqlite3_value] object specified by the 2nd parameter. ^The\n** sqlite3_result_value() interface makes a copy of the [sqlite3_value]\n** so that the [sqlite3_value] specified in the parameter may change or\n** be deallocated after sqlite3_result_value() returns without harm.\n** ^A [protected sqlite3_value] object may always be used where an\n** [unprotected sqlite3_value] object is required, so either\n** kind of [sqlite3_value] object can be used with this interface.\n**\n** ^The sqlite3_result_pointer(C,P,T,D) interface sets the result to an\n** SQL NULL value, just like [sqlite3_result_null(C)], except that it\n** also associates the host-language pointer P or type T with that\n** NULL value such that the pointer can be retrieved within an\n** [application-defined SQL function] using [sqlite3_value_pointer()].\n** ^If the D parameter is not NULL, then it is a pointer to a destructor\n** for the P parameter. ^SQLite invokes D with P as its only argument\n** when SQLite is finished with P. The T parameter should be a static\n** string and preferably a string literal. The sqlite3_result_pointer()\n** routine is part of the [pointer passing interface] added for SQLite 3.20.0.\n**\n** If these routines are called from within a different thread\n** than the one containing the application-defined function that received\n** the [sqlite3_context] pointer, the results are undefined.\n*/\nSQLITE_API void sqlite3_result_blob(sqlite3_context*, const void*, int, void(*)(void*));\nSQLITE_API void sqlite3_result_blob64(sqlite3_context*,const void*,\n sqlite3_uint64,void(*)(void*));\nSQLITE_API void sqlite3_result_double(sqlite3_context*, double);\nSQLITE_API void sqlite3_result_error(sqlite3_context*, const char*, int);\nSQLITE_API void sqlite3_result_error16(sqlite3_context*, const void*, int);\nSQLITE_API void sqlite3_result_error_toobig(sqlite3_context*);\nSQLITE_API void sqlite3_result_error_nomem(sqlite3_context*);\nSQLITE_API void sqlite3_result_error_code(sqlite3_context*, int);\nSQLITE_API void sqlite3_result_int(sqlite3_context*, int);\nSQLITE_API void sqlite3_result_int64(sqlite3_context*, sqlite3_int64);\nSQLITE_API void sqlite3_result_null(sqlite3_context*);\nSQLITE_API void sqlite3_result_text(sqlite3_context*, const char*, int, void(*)(void*));\nSQLITE_API void sqlite3_result_text64(sqlite3_context*, const char*,sqlite3_uint64,\n void(*)(void*), unsigned char encoding);\nSQLITE_API void sqlite3_result_text16(sqlite3_context*, const void*, int, void(*)(void*));\nSQLITE_API void sqlite3_result_text16le(sqlite3_context*, const void*, int,void(*)(void*));\nSQLITE_API void sqlite3_result_text16be(sqlite3_context*, const void*, int,void(*)(void*));\nSQLITE_API void sqlite3_result_value(sqlite3_context*, sqlite3_value*);\nSQLITE_API void sqlite3_result_pointer(sqlite3_context*, void*,const char*,void(*)(void*));\nSQLITE_API void sqlite3_result_zeroblob(sqlite3_context*, int n);\nSQLITE_API int sqlite3_result_zeroblob64(sqlite3_context*, sqlite3_uint64 n);\n\n\n/*\n** CAPI3REF: Setting The Subtype Of An SQL Function\n** METHOD: sqlite3_context\n**\n** The sqlite3_result_subtype(C,T) function causes the subtype of\n** the result from the [application-defined SQL function] with\n** [sqlite3_context] C to be the value T. Only the lower 8 bits\n** of the subtype T are preserved in current versions of SQLite;\n** higher order bits are discarded.\n** The number of subtype bytes preserved by SQLite might increase\n** in future releases of SQLite.\n**\n** Every [application-defined SQL function] that invokes this interface\n** should include the [SQLITE_RESULT_SUBTYPE] property in its\n** text encoding argument when the SQL function is\n** [sqlite3_create_function|registered]. If the [SQLITE_RESULT_SUBTYPE]\n** property is omitted from the function that invokes sqlite3_result_subtype(),\n** then in some cases the sqlite3_result_subtype() might fail to set\n** the result subtype.\n**\n** If SQLite is compiled with -DSQLITE_STRICT_SUBTYPE=1, then any\n** SQL function that invokes the sqlite3_result_subtype() interface\n** and that does not have the SQLITE_RESULT_SUBTYPE property will raise\n** an error. Future versions of SQLite might enable -DSQLITE_STRICT_SUBTYPE=1\n** by default.\n*/\nSQLITE_API void sqlite3_result_subtype(sqlite3_context*,unsigned int);\n\n/*\n** CAPI3REF: Define New Collating Sequences\n** METHOD: sqlite3\n**\n** ^These functions add, remove, or modify a [collation] associated\n** with the [database connection] specified as the first argument.\n**\n** ^The name of the collation is a UTF-8 string\n** for sqlite3_create_collation() and sqlite3_create_collation_v2()\n** and a UTF-16 string in native byte order for sqlite3_create_collation16().\n** ^Collation names that compare equal according to [sqlite3_strnicmp()] are\n** considered to be the same name.\n**\n** ^(The third argument (eTextRep) must be one of the constants:\n**
    \n**
  • [SQLITE_UTF8],\n**
  • [SQLITE_UTF16LE],\n**
  • [SQLITE_UTF16BE],\n**
  • [SQLITE_UTF16], or\n**
  • [SQLITE_UTF16_ALIGNED].\n**
)^\n** ^The eTextRep argument determines the encoding of strings passed\n** to the collating function callback, xCompare.\n** ^The [SQLITE_UTF16] and [SQLITE_UTF16_ALIGNED] values for eTextRep\n** force strings to be UTF16 with native byte order.\n** ^The [SQLITE_UTF16_ALIGNED] value for eTextRep forces strings to begin\n** on an even byte address.\n**\n** ^The fourth argument, pArg, is an application data pointer that is passed\n** through as the first argument to the collating function callback.\n**\n** ^The fifth argument, xCompare, is a pointer to the collating function.\n** ^Multiple collating functions can be registered using the same name but\n** with different eTextRep parameters and SQLite will use whichever\n** function requires the least amount of data transformation.\n** ^If the xCompare argument is NULL then the collating function is\n** deleted. ^When all collating functions having the same name are deleted,\n** that collation is no longer usable.\n**\n** ^The collating function callback is invoked with a copy of the pArg\n** application data pointer and with two strings in the encoding specified\n** by the eTextRep argument. The two integer parameters to the collating\n** function callback are the length of the two strings, in bytes. The collating\n** function must return an integer that is negative, zero, or positive\n** if the first string is less than, equal to, or greater than the second,\n** respectively. A collating function must always return the same answer\n** given the same inputs. If two or more collating functions are registered\n** to the same collation name (using different eTextRep values) then all\n** must give an equivalent answer when invoked with equivalent strings.\n** The collating function must obey the following properties for all\n** strings A, B, and C:\n**\n**
    \n**
  1. If A==B then B==A.\n**
  2. If A==B and B==C then A==C.\n**
  3. If A<B THEN B>A.\n**
  4. If A<B and B<C then A<C.\n**
\n**\n** If a collating function fails any of the above constraints and that\n** collating function is registered and used, then the behavior of SQLite\n** is undefined.\n**\n** ^The sqlite3_create_collation_v2() works like sqlite3_create_collation()\n** with the addition that the xDestroy callback is invoked on pArg when\n** the collating function is deleted.\n** ^Collating functions are deleted when they are overridden by later\n** calls to the collation creation functions or when the\n** [database connection] is closed using [sqlite3_close()].\n**\n** ^The xDestroy callback is not called if the\n** sqlite3_create_collation_v2() function fails. Applications that invoke\n** sqlite3_create_collation_v2() with a non-NULL xDestroy argument should\n** check the return code and dispose of the application data pointer\n** themselves rather than expecting SQLite to deal with it for them.\n** This is different from every other SQLite interface. The inconsistency\n** is unfortunate but cannot be changed without breaking backwards\n** compatibility.\n**\n** See also: [sqlite3_collation_needed()] and [sqlite3_collation_needed16()].\n*/\nSQLITE_API int sqlite3_create_collation(\n sqlite3*,\n const char *zName,\n int eTextRep,\n void *pArg,\n int(*xCompare)(void*,int,const void*,int,const void*)\n);\nSQLITE_API int sqlite3_create_collation_v2(\n sqlite3*,\n const char *zName,\n int eTextRep,\n void *pArg,\n int(*xCompare)(void*,int,const void*,int,const void*),\n void(*xDestroy)(void*)\n);\nSQLITE_API int sqlite3_create_collation16(\n sqlite3*,\n const void *zName,\n int eTextRep,\n void *pArg,\n int(*xCompare)(void*,int,const void*,int,const void*)\n);\n\n/*\n** CAPI3REF: Collation Needed Callbacks\n** METHOD: sqlite3\n**\n** ^To avoid having to register all collation sequences before a database\n** can be used, a single callback function may be registered with the\n** [database connection] to be invoked whenever an undefined collation\n** sequence is required.\n**\n** ^If the function is registered using the sqlite3_collation_needed() API,\n** then it is passed the names of undefined collation sequences as strings\n** encoded in UTF-8. ^If sqlite3_collation_needed16() is used,\n** the names are passed as UTF-16 in machine native byte order.\n** ^A call to either function replaces the existing collation-needed callback.\n**\n** ^(When the callback is invoked, the first argument passed is a copy\n** of the second argument to sqlite3_collation_needed() or\n** sqlite3_collation_needed16(). The second argument is the database\n** connection. The third argument is one of [SQLITE_UTF8], [SQLITE_UTF16BE],\n** or [SQLITE_UTF16LE], indicating the most desirable form of the collation\n** sequence function required. The fourth parameter is the name of the\n** required collation sequence.)^\n**\n** The callback function should register the desired collation using\n** [sqlite3_create_collation()], [sqlite3_create_collation16()], or\n** [sqlite3_create_collation_v2()].\n*/\nSQLITE_API int sqlite3_collation_needed(\n sqlite3*,\n void*,\n void(*)(void*,sqlite3*,int eTextRep,const char*)\n);\nSQLITE_API int sqlite3_collation_needed16(\n sqlite3*,\n void*,\n void(*)(void*,sqlite3*,int eTextRep,const void*)\n);\n\n#ifdef SQLITE_ENABLE_CEROD\n/*\n** Specify the activation key for a CEROD database. Unless\n** activated, none of the CEROD routines will work.\n*/\nSQLITE_API void sqlite3_activate_cerod(\n const char *zPassPhrase /* Activation phrase */\n);\n#endif\n\n/*\n** CAPI3REF: Suspend Execution For A Short Time\n**\n** The sqlite3_sleep() function causes the current thread to suspend execution\n** for at least a number of milliseconds specified in its parameter.\n**\n** If the operating system does not support sleep requests with\n** millisecond time resolution, then the time will be rounded up to\n** the nearest second. The number of milliseconds of sleep actually\n** requested from the operating system is returned.\n**\n** ^SQLite implements this interface by calling the xSleep()\n** method of the default [sqlite3_vfs] object. If the xSleep() method\n** of the default VFS is not implemented correctly, or not implemented at\n** all, then the behavior of sqlite3_sleep() may deviate from the description\n** in the previous paragraphs.\n**\n** If a negative argument is passed to sqlite3_sleep() the results vary by\n** VFS and operating system. Some system treat a negative argument as an\n** instruction to sleep forever. Others understand it to mean do not sleep\n** at all. ^In SQLite version 3.42.0 and later, a negative\n** argument passed into sqlite3_sleep() is changed to zero before it is relayed\n** down into the xSleep method of the VFS.\n*/\nSQLITE_API int sqlite3_sleep(int);\n\n/*\n** CAPI3REF: Name Of The Folder Holding Temporary Files\n**\n** ^(If this global variable is made to point to a string which is\n** the name of a folder (a.k.a. directory), then all temporary files\n** created by SQLite when using a built-in [sqlite3_vfs | VFS]\n** will be placed in that directory.)^ ^If this variable\n** is a NULL pointer, then SQLite performs a search for an appropriate\n** temporary file directory.\n**\n** Applications are strongly discouraged from using this global variable.\n** It is required to set a temporary folder on Windows Runtime (WinRT).\n** But for all other platforms, it is highly recommended that applications\n** neither read nor write this variable. This global variable is a relic\n** that exists for backwards compatibility of legacy applications and should\n** be avoided in new projects.\n**\n** It is not safe to read or modify this variable in more than one\n** thread at a time. It is not safe to read or modify this variable\n** if a [database connection] is being used at the same time in a separate\n** thread.\n** It is intended that this variable be set once\n** as part of process initialization and before any SQLite interface\n** routines have been called and that this variable remain unchanged\n** thereafter.\n**\n** ^The [temp_store_directory pragma] may modify this variable and cause\n** it to point to memory obtained from [sqlite3_malloc]. ^Furthermore,\n** the [temp_store_directory pragma] always assumes that any string\n** that this variable points to is held in memory obtained from\n** [sqlite3_malloc] and the pragma may attempt to free that memory\n** using [sqlite3_free].\n** Hence, if this variable is modified directly, either it should be\n** made NULL or made to point to memory obtained from [sqlite3_malloc]\n** or else the use of the [temp_store_directory pragma] should be avoided.\n** Except when requested by the [temp_store_directory pragma], SQLite\n** does not free the memory that sqlite3_temp_directory points to. If\n** the application wants that memory to be freed, it must do\n** so itself, taking care to only do so after all [database connection]\n** objects have been destroyed.\n**\n** Note to Windows Runtime users: The temporary directory must be set\n** prior to calling [sqlite3_open] or [sqlite3_open_v2]. Otherwise, various\n** features that require the use of temporary files may fail. Here is an\n** example of how to do this using C++ with the Windows Runtime:\n**\n** 
\n** LPCWSTR zPath = Windows::Storage::ApplicationData::Current->\n**       TemporaryFolder->Path->Data();\n** char zPathBuf[MAX_PATH + 1];\n** memset(zPathBuf, 0, sizeof(zPathBuf));\n** WideCharToMultiByte(CP_UTF8, 0, zPath, -1, zPathBuf, sizeof(zPathBuf),\n**       NULL, NULL);\n** sqlite3_temp_directory = sqlite3_mprintf(\"%s\", zPathBuf);\n**
\n*/\nSQLITE_API SQLITE_EXTERN char *sqlite3_temp_directory;\n\n/*\n** CAPI3REF: Name Of The Folder Holding Database Files\n**\n** ^(If this global variable is made to point to a string which is\n** the name of a folder (a.k.a. directory), then all database files\n** specified with a relative pathname and created or accessed by\n** SQLite when using a built-in windows [sqlite3_vfs | VFS] will be assumed\n** to be relative to that directory.)^ ^If this variable is a NULL\n** pointer, then SQLite assumes that all database files specified\n** with a relative pathname are relative to the current directory\n** for the process. Only the windows VFS makes use of this global\n** variable; it is ignored by the unix VFS.\n**\n** Changing the value of this variable while a database connection is\n** open can result in a corrupt database.\n**\n** It is not safe to read or modify this variable in more than one\n** thread at a time. It is not safe to read or modify this variable\n** if a [database connection] is being used at the same time in a separate\n** thread.\n** It is intended that this variable be set once\n** as part of process initialization and before any SQLite interface\n** routines have been called and that this variable remain unchanged\n** thereafter.\n**\n** ^The [data_store_directory pragma] may modify this variable and cause\n** it to point to memory obtained from [sqlite3_malloc]. ^Furthermore,\n** the [data_store_directory pragma] always assumes that any string\n** that this variable points to is held in memory obtained from\n** [sqlite3_malloc] and the pragma may attempt to free that memory\n** using [sqlite3_free].\n** Hence, if this variable is modified directly, either it should be\n** made NULL or made to point to memory obtained from [sqlite3_malloc]\n** or else the use of the [data_store_directory pragma] should be avoided.\n*/\nSQLITE_API SQLITE_EXTERN char *sqlite3_data_directory;\n\n/*\n** CAPI3REF: Win32 Specific Interface\n**\n** These interfaces are available only on Windows. The\n** [sqlite3_win32_set_directory] interface is used to set the value associated\n** with the [sqlite3_temp_directory] or [sqlite3_data_directory] variable, to\n** zValue, depending on the value of the type parameter. The zValue parameter\n** should be NULL to cause the previous value to be freed via [sqlite3_free];\n** a non-NULL value will be copied into memory obtained from [sqlite3_malloc]\n** prior to being used. The [sqlite3_win32_set_directory] interface returns\n** [SQLITE_OK] to indicate success, [SQLITE_ERROR] if the type is unsupported,\n** or [SQLITE_NOMEM] if memory could not be allocated. The value of the\n** [sqlite3_data_directory] variable is intended to act as a replacement for\n** the current directory on the sub-platforms of Win32 where that concept is\n** not present, e.g. WinRT and UWP. The [sqlite3_win32_set_directory8] and\n** [sqlite3_win32_set_directory16] interfaces behave exactly the same as the\n** sqlite3_win32_set_directory interface except the string parameter must be\n** UTF-8 or UTF-16, respectively.\n*/\nSQLITE_API int sqlite3_win32_set_directory(\n unsigned long type, /* Identifier for directory being set or reset */\n void *zValue /* New value for directory being set or reset */\n);\nSQLITE_API int sqlite3_win32_set_directory8(unsigned long type, const char *zValue);\nSQLITE_API int sqlite3_win32_set_directory16(unsigned long type, const void *zValue);\n\n/*\n** CAPI3REF: Win32 Directory Types\n**\n** These macros are only available on Windows. They define the allowed values\n** for the type argument to the [sqlite3_win32_set_directory] interface.\n*/\n#define SQLITE_WIN32_DATA_DIRECTORY_TYPE 1\n#define SQLITE_WIN32_TEMP_DIRECTORY_TYPE 2\n\n/*\n** CAPI3REF: Test For Auto-Commit Mode\n** KEYWORDS: {autocommit mode}\n** METHOD: sqlite3\n**\n** ^The sqlite3_get_autocommit() interface returns non-zero or\n** zero if the given database connection is or is not in autocommit mode,\n** respectively. ^Autocommit mode is on by default.\n** ^Autocommit mode is disabled by a [BEGIN] statement.\n** ^Autocommit mode is re-enabled by a [COMMIT] or [ROLLBACK].\n**\n** If certain kinds of errors occur on a statement within a multi-statement\n** transaction (errors including [SQLITE_FULL], [SQLITE_IOERR],\n** [SQLITE_NOMEM], [SQLITE_BUSY], and [SQLITE_INTERRUPT]) then the\n** transaction might be rolled back automatically. The only way to\n** find out whether SQLite automatically rolled back the transaction after\n** an error is to use this function.\n**\n** If another thread changes the autocommit status of the database\n** connection while this routine is running, then the return value\n** is undefined.\n*/\nSQLITE_API int sqlite3_get_autocommit(sqlite3*);\n\n/*\n** CAPI3REF: Find The Database Handle Of A Prepared Statement\n** METHOD: sqlite3_stmt\n**\n** ^The sqlite3_db_handle interface returns the [database connection] handle\n** to which a [prepared statement] belongs. ^The [database connection]\n** returned by sqlite3_db_handle is the same [database connection]\n** that was the first argument\n** to the [sqlite3_prepare_v2()] call (or its variants) that was used to\n** create the statement in the first place.\n*/\nSQLITE_API sqlite3 *sqlite3_db_handle(sqlite3_stmt*);\n\n/*\n** CAPI3REF: Return The Schema Name For A Database Connection\n** METHOD: sqlite3\n**\n** ^The sqlite3_db_name(D,N) interface returns a pointer to the schema name\n** for the N-th database on database connection D, or a NULL pointer if N is\n** out of range. An N value of 0 means the main database file. An N of 1 is\n** the \"temp\" schema. Larger values of N correspond to various ATTACH-ed\n** databases.\n**\n** Space to hold the string that is returned by sqlite3_db_name() is managed\n** by SQLite itself. The string might be deallocated by any operation that\n** changes the schema, including [ATTACH] or [DETACH] or calls to\n** [sqlite3_serialize()] or [sqlite3_deserialize()], even operations that\n** occur on a different thread. Applications that need to\n** remember the string long-term should make their own copy. Applications that\n** are accessing the same database connection simultaneously on multiple\n** threads should mutex-protect calls to this API and should make their own\n** private copy of the result prior to releasing the mutex.\n*/\nSQLITE_API const char *sqlite3_db_name(sqlite3 *db, int N);\n\n/*\n** CAPI3REF: Return The Filename For A Database Connection\n** METHOD: sqlite3\n**\n** ^The sqlite3_db_filename(D,N) interface returns a pointer to the filename\n** associated with database N of connection D.\n** ^If there is no attached database N on the database\n** connection D, or if database N is a temporary or in-memory database, then\n** this function will return either a NULL pointer or an empty string.\n**\n** ^The string value returned by this routine is owned and managed by\n** the database connection. ^The value will be valid until the database N\n** is [DETACH]-ed or until the database connection closes.\n**\n** ^The filename returned by this function is the output of the\n** xFullPathname method of the [VFS]. ^In other words, the filename\n** will be an absolute pathname, even if the filename used\n** to open the database originally was a URI or relative pathname.\n**\n** If the filename pointer returned by this routine is not NULL, then it\n** can be used as the filename input parameter to these routines:\n**
    \n**
  • [sqlite3_uri_parameter()]\n**
  • [sqlite3_uri_boolean()]\n**
  • [sqlite3_uri_int64()]\n**
  • [sqlite3_filename_database()]\n**
  • [sqlite3_filename_journal()]\n**
  • [sqlite3_filename_wal()]\n**
\n*/\nSQLITE_API sqlite3_filename sqlite3_db_filename(sqlite3 *db, const char *zDbName);\n\n/*\n** CAPI3REF: Determine if a database is read-only\n** METHOD: sqlite3\n**\n** ^The sqlite3_db_readonly(D,N) interface returns 1 if the database N\n** of connection D is read-only, 0 if it is read/write, or -1 if N is not\n** the name of a database on connection D.\n*/\nSQLITE_API int sqlite3_db_readonly(sqlite3 *db, const char *zDbName);\n\n/*\n** CAPI3REF: Determine the transaction state of a database\n** METHOD: sqlite3\n**\n** ^The sqlite3_txn_state(D,S) interface returns the current\n** [transaction state] of schema S in database connection D. ^If S is NULL,\n** then the highest transaction state of any schema on database connection D\n** is returned. Transaction states are (in order of lowest to highest):\n**
    \n**
  1. SQLITE_TXN_NONE\n**
  2. SQLITE_TXN_READ\n**
  3. SQLITE_TXN_WRITE\n**
\n** ^If the S argument to sqlite3_txn_state(D,S) is not the name of\n** a valid schema, then -1 is returned.\n*/\nSQLITE_API int sqlite3_txn_state(sqlite3*,const char *zSchema);\n\n/*\n** CAPI3REF: Allowed return values from sqlite3_txn_state()\n** KEYWORDS: {transaction state}\n**\n** These constants define the current transaction state of a database file.\n** ^The [sqlite3_txn_state(D,S)] interface returns one of these\n** constants in order to describe the transaction state of schema S\n** in [database connection] D.\n**\n**
\n** [[SQLITE_TXN_NONE]]
SQLITE_TXN_NONE
\n**
The SQLITE_TXN_NONE state means that no transaction is currently\n** pending.
\n**\n** [[SQLITE_TXN_READ]]
SQLITE_TXN_READ
\n**
The SQLITE_TXN_READ state means that the database is currently\n** in a read transaction. Content has been read from the database file\n** but nothing in the database file has changed. The transaction state\n** will be advanced to SQLITE_TXN_WRITE if any changes occur and there are\n** no other conflicting concurrent write transactions. The transaction\n** state will revert to SQLITE_TXN_NONE following a [ROLLBACK] or\n** [COMMIT].
\n**\n** [[SQLITE_TXN_WRITE]]
SQLITE_TXN_WRITE
\n**
The SQLITE_TXN_WRITE state means that the database is currently\n** in a write transaction. Content has been written to the database file\n** but has not yet committed. The transaction state will change to\n** SQLITE_TXN_NONE at the next [ROLLBACK] or [COMMIT].
\n*/\n#define SQLITE_TXN_NONE 0\n#define SQLITE_TXN_READ 1\n#define SQLITE_TXN_WRITE 2\n\n/*\n** CAPI3REF: Find the next prepared statement\n** METHOD: sqlite3\n**\n** ^This interface returns a pointer to the next [prepared statement] after\n** pStmt associated with the [database connection] pDb. ^If pStmt is NULL\n** then this interface returns a pointer to the first prepared statement\n** associated with the database connection pDb. ^If no prepared statement\n** satisfies the conditions of this routine, it returns NULL.\n**\n** The [database connection] pointer D in a call to\n** [sqlite3_next_stmt(D,S)] must refer to an open database\n** connection and in particular must not be a NULL pointer.\n*/\nSQLITE_API sqlite3_stmt *sqlite3_next_stmt(sqlite3 *pDb, sqlite3_stmt *pStmt);\n\n/*\n** CAPI3REF: Commit And Rollback Notification Callbacks\n** METHOD: sqlite3\n**\n** ^The sqlite3_commit_hook() interface registers a callback\n** function to be invoked whenever a transaction is [COMMIT | committed].\n** ^Any callback set by a previous call to sqlite3_commit_hook()\n** for the same database connection is overridden.\n** ^The sqlite3_rollback_hook() interface registers a callback\n** function to be invoked whenever a transaction is [ROLLBACK | rolled back].\n** ^Any callback set by a previous call to sqlite3_rollback_hook()\n** for the same database connection is overridden.\n** ^The pArg argument is passed through to the callback.\n** ^If the callback on a commit hook function returns non-zero,\n** then the commit is converted into a rollback.\n**\n** ^The sqlite3_commit_hook(D,C,P) and sqlite3_rollback_hook(D,C,P) functions\n** return the P argument from the previous call of the same function\n** on the same [database connection] D, or NULL for\n** the first call for each function on D.\n**\n** The commit and rollback hook callbacks are not reentrant.\n** The callback implementation must not do anything that will modify\n** the database connection that invoked the callback. Any actions\n** to modify the database connection must be deferred until after the\n** completion of the [sqlite3_step()] call that triggered the commit\n** or rollback hook in the first place.\n** Note that running any other SQL statements, including SELECT statements,\n** or merely calling [sqlite3_prepare_v2()] and [sqlite3_step()] will modify\n** the database connections for the meaning of \"modify\" in this paragraph.\n**\n** ^Registering a NULL function disables the callback.\n**\n** ^When the commit hook callback routine returns zero, the [COMMIT]\n** operation is allowed to continue normally. ^If the commit hook\n** returns non-zero, then the [COMMIT] is converted into a [ROLLBACK].\n** ^The rollback hook is invoked on a rollback that results from a commit\n** hook returning non-zero, just as it would be with any other rollback.\n**\n** ^For the purposes of this API, a transaction is said to have been\n** rolled back if an explicit \"ROLLBACK\" statement is executed, or\n** an error or constraint causes an implicit rollback to occur.\n** ^The rollback callback is not invoked if a transaction is\n** automatically rolled back because the database connection is closed.\n**\n** See also the [sqlite3_update_hook()] interface.\n*/\nSQLITE_API void *sqlite3_commit_hook(sqlite3*, int(*)(void*), void*);\nSQLITE_API void *sqlite3_rollback_hook(sqlite3*, void(*)(void *), void*);\n\n/*\n** CAPI3REF: Autovacuum Compaction Amount Callback\n** METHOD: sqlite3\n**\n** ^The sqlite3_autovacuum_pages(D,C,P,X) interface registers a callback\n** function C that is invoked prior to each autovacuum of the database\n** file. ^The callback is passed a copy of the generic data pointer (P),\n** the schema-name of the attached database that is being autovacuumed,\n** the size of the database file in pages, the number of free pages,\n** and the number of bytes per page, respectively. The callback should\n** return the number of free pages that should be removed by the\n** autovacuum. ^If the callback returns zero, then no autovacuum happens.\n** ^If the value returned is greater than or equal to the number of\n** free pages, then a complete autovacuum happens.\n**\n**

^If there are multiple ATTACH-ed database files that are being\n** modified as part of a transaction commit, then the autovacuum pages\n** callback is invoked separately for each file.\n**\n**

The callback is not reentrant. The callback function should\n** not attempt to invoke any other SQLite interface. If it does, bad\n** things may happen, including segmentation faults and corrupt database\n** files. The callback function should be a simple function that\n** does some arithmetic on its input parameters and returns a result.\n**\n** ^The X parameter to sqlite3_autovacuum_pages(D,C,P,X) is an optional\n** destructor for the P parameter. ^If X is not NULL, then X(P) is\n** invoked whenever the database connection closes or when the callback\n** is overwritten by another invocation of sqlite3_autovacuum_pages().\n**\n**

^There is only one autovacuum pages callback per database connection.\n** ^Each call to the sqlite3_autovacuum_pages() interface overrides all\n** previous invocations for that database connection. ^If the callback\n** argument (C) to sqlite3_autovacuum_pages(D,C,P,X) is a NULL pointer,\n** then the autovacuum steps callback is canceled. The return value\n** from sqlite3_autovacuum_pages() is normally SQLITE_OK, but might\n** be some other error code if something goes wrong. The current\n** implementation will only return SQLITE_OK or SQLITE_MISUSE, but other\n** return codes might be added in future releases.\n**\n**

If no autovacuum pages callback is specified (the usual case) or\n** a NULL pointer is provided for the callback,\n** then the default behavior is to vacuum all free pages. So, in other\n** words, the default behavior is the same as if the callback function\n** were something like this:\n**\n** 

\n**     unsigned int demonstration_autovac_pages_callback(\n**       void *pClientData,\n**       const char *zSchema,\n**       unsigned int nDbPage,\n**       unsigned int nFreePage,\n**       unsigned int nBytePerPage\n**     ){\n**       return nFreePage;\n**     }\n**
\n*/\nSQLITE_API int sqlite3_autovacuum_pages(\n sqlite3 *db,\n unsigned int(*)(void*,const char*,unsigned int,unsigned int,unsigned int),\n void*,\n void(*)(void*)\n);\n\n\n/*\n** CAPI3REF: Data Change Notification Callbacks\n** METHOD: sqlite3\n**\n** ^The sqlite3_update_hook() interface registers a callback function\n** with the [database connection] identified by the first argument\n** to be invoked whenever a row is updated, inserted or deleted in\n** a [rowid table].\n** ^Any callback set by a previous call to this function\n** for the same database connection is overridden.\n**\n** ^The second argument is a pointer to the function to invoke when a\n** row is updated, inserted or deleted in a rowid table.\n** ^The update hook is disabled by invoking sqlite3_update_hook()\n** with a NULL pointer as the second parameter.\n** ^The first argument to the callback is a copy of the third argument\n** to sqlite3_update_hook().\n** ^The second callback argument is one of [SQLITE_INSERT], [SQLITE_DELETE],\n** or [SQLITE_UPDATE], depending on the operation that caused the callback\n** to be invoked.\n** ^The third and fourth arguments to the callback contain pointers to the\n** database and table name containing the affected row.\n** ^The final callback parameter is the [rowid] of the row.\n** ^In the case of an update, this is the [rowid] after the update takes place.\n**\n** ^(The update hook is not invoked when internal system tables are\n** modified (i.e. sqlite_sequence).)^\n** ^The update hook is not invoked when [WITHOUT ROWID] tables are modified.\n**\n** ^In the current implementation, the update hook\n** is not invoked when conflicting rows are deleted because of an\n** [ON CONFLICT | ON CONFLICT REPLACE] clause. ^Nor is the update hook\n** invoked when rows are deleted using the [truncate optimization].\n** The exceptions defined in this paragraph might change in a future\n** release of SQLite.\n**\n** Whether the update hook is invoked before or after the\n** corresponding change is currently unspecified and may differ\n** depending on the type of change. Do not rely on the order of the\n** hook call with regards to the final result of the operation which\n** triggers the hook.\n**\n** The update hook implementation must not do anything that will modify\n** the database connection that invoked the update hook. Any actions\n** to modify the database connection must be deferred until after the\n** completion of the [sqlite3_step()] call that triggered the update hook.\n** Note that [sqlite3_prepare_v2()] and [sqlite3_step()] both modify their\n** database connections for the meaning of \"modify\" in this paragraph.\n**\n** ^The sqlite3_update_hook(D,C,P) function\n** returns the P argument from the previous call\n** on the same [database connection] D, or NULL for\n** the first call on D.\n**\n** See also the [sqlite3_commit_hook()], [sqlite3_rollback_hook()],\n** and [sqlite3_preupdate_hook()] interfaces.\n*/\nSQLITE_API void *sqlite3_update_hook(\n sqlite3*,\n void(*)(void *,int ,char const *,char const *,sqlite3_int64),\n void*\n);\n\n/*\n** CAPI3REF: Enable Or Disable Shared Pager Cache\n**\n** ^(This routine enables or disables the sharing of the database cache\n** and schema data structures between [database connection | connections]\n** to the same database. Sharing is enabled if the argument is true\n** and disabled if the argument is false.)^\n**\n** This interface is omitted if SQLite is compiled with\n** [-DSQLITE_OMIT_SHARED_CACHE]. The [-DSQLITE_OMIT_SHARED_CACHE]\n** compile-time option is recommended because the\n** [use of shared cache mode is discouraged].\n**\n** ^Cache sharing is enabled and disabled for an entire process.\n** This is a change as of SQLite [version 3.5.0] ([dateof:3.5.0]).\n** In prior versions of SQLite,\n** sharing was enabled or disabled for each thread separately.\n**\n** ^(The cache sharing mode set by this interface effects all subsequent\n** calls to [sqlite3_open()], [sqlite3_open_v2()], and [sqlite3_open16()].\n** Existing database connections continue to use the sharing mode\n** that was in effect at the time they were opened.)^\n**\n** ^(This routine returns [SQLITE_OK] if shared cache was enabled or disabled\n** successfully. An [error code] is returned otherwise.)^\n**\n** ^Shared cache is disabled by default. It is recommended that it stay\n** that way. In other words, do not use this routine. This interface\n** continues to be provided for historical compatibility, but its use is\n** discouraged. Any use of shared cache is discouraged. If shared cache\n** must be used, it is recommended that shared cache only be enabled for\n** individual database connections using the [sqlite3_open_v2()] interface\n** with the [SQLITE_OPEN_SHAREDCACHE] flag.\n**\n** Note: This method is disabled on MacOS X 10.7 and iOS version 5.0\n** and will always return SQLITE_MISUSE. On those systems,\n** shared cache mode should be enabled per-database connection via\n** [sqlite3_open_v2()] with [SQLITE_OPEN_SHAREDCACHE].\n**\n** This interface is threadsafe on processors where writing a\n** 32-bit integer is atomic.\n**\n** See Also: [SQLite Shared-Cache Mode]\n*/\nSQLITE_API int sqlite3_enable_shared_cache(int);\n\n/*\n** CAPI3REF: Attempt To Free Heap Memory\n**\n** ^The sqlite3_release_memory() interface attempts to free N bytes\n** of heap memory by deallocating non-essential memory allocations\n** held by the database library. Memory used to cache database\n** pages to improve performance is an example of non-essential memory.\n** ^sqlite3_release_memory() returns the number of bytes actually freed,\n** which might be more or less than the amount requested.\n** ^The sqlite3_release_memory() routine is a no-op returning zero\n** if SQLite is not compiled with [SQLITE_ENABLE_MEMORY_MANAGEMENT].\n**\n** See also: [sqlite3_db_release_memory()]\n*/\nSQLITE_API int sqlite3_release_memory(int);\n\n/*\n** CAPI3REF: Free Memory Used By A Database Connection\n** METHOD: sqlite3\n**\n** ^The sqlite3_db_release_memory(D) interface attempts to free as much heap\n** memory as possible from database connection D. Unlike the\n** [sqlite3_release_memory()] interface, this interface is in effect even\n** when the [SQLITE_ENABLE_MEMORY_MANAGEMENT] compile-time option is\n** omitted.\n**\n** See also: [sqlite3_release_memory()]\n*/\nSQLITE_API int sqlite3_db_release_memory(sqlite3*);\n\n/*\n** CAPI3REF: Impose A Limit On Heap Size\n**\n** These interfaces impose limits on the amount of heap memory that will be\n** used by all database connections within a single process.\n**\n** ^The sqlite3_soft_heap_limit64() interface sets and/or queries the\n** soft limit on the amount of heap memory that may be allocated by SQLite.\n** ^SQLite strives to keep heap memory utilization below the soft heap\n** limit by reducing the number of pages held in the page cache\n** as heap memory usages approaches the limit.\n** ^The soft heap limit is \"soft\" because even though SQLite strives to stay\n** below the limit, it will exceed the limit rather than generate\n** an [SQLITE_NOMEM] error. In other words, the soft heap limit\n** is advisory only.\n**\n** ^The sqlite3_hard_heap_limit64(N) interface sets a hard upper bound of\n** N bytes on the amount of memory that will be allocated. ^The\n** sqlite3_hard_heap_limit64(N) interface is similar to\n** sqlite3_soft_heap_limit64(N) except that memory allocations will fail\n** when the hard heap limit is reached.\n**\n** ^The return value from both sqlite3_soft_heap_limit64() and\n** sqlite3_hard_heap_limit64() is the size of\n** the heap limit prior to the call, or negative in the case of an\n** error. ^If the argument N is negative\n** then no change is made to the heap limit. Hence, the current\n** size of heap limits can be determined by invoking\n** sqlite3_soft_heap_limit64(-1) or sqlite3_hard_heap_limit(-1).\n**\n** ^Setting the heap limits to zero disables the heap limiter mechanism.\n**\n** ^The soft heap limit may not be greater than the hard heap limit.\n** ^If the hard heap limit is enabled and if sqlite3_soft_heap_limit(N)\n** is invoked with a value of N that is greater than the hard heap limit,\n** the soft heap limit is set to the value of the hard heap limit.\n** ^The soft heap limit is automatically enabled whenever the hard heap\n** limit is enabled. ^When sqlite3_hard_heap_limit64(N) is invoked and\n** the soft heap limit is outside the range of 1..N, then the soft heap\n** limit is set to N. ^Invoking sqlite3_soft_heap_limit64(0) when the\n** hard heap limit is enabled makes the soft heap limit equal to the\n** hard heap limit.\n**\n** The memory allocation limits can also be adjusted using\n** [PRAGMA soft_heap_limit] and [PRAGMA hard_heap_limit].\n**\n** ^(The heap limits are not enforced in the current implementation\n** if one or more of following conditions are true:\n**\n**
    \n**
  • The limit value is set to zero.\n**
  • Memory accounting is disabled using a combination of the\n** [sqlite3_config]([SQLITE_CONFIG_MEMSTATUS],...) start-time option and\n** the [SQLITE_DEFAULT_MEMSTATUS] compile-time option.\n**
  • An alternative page cache implementation is specified using\n** [sqlite3_config]([SQLITE_CONFIG_PCACHE2],...).\n**
  • The page cache allocates from its own memory pool supplied\n** by [sqlite3_config]([SQLITE_CONFIG_PAGECACHE],...) rather than\n** from the heap.\n**
)^\n**\n** The circumstances under which SQLite will enforce the heap limits may\n** change in future releases of SQLite.\n*/\nSQLITE_API sqlite3_int64 sqlite3_soft_heap_limit64(sqlite3_int64 N);\nSQLITE_API sqlite3_int64 sqlite3_hard_heap_limit64(sqlite3_int64 N);\n\n/*\n** CAPI3REF: Deprecated Soft Heap Limit Interface\n** DEPRECATED\n**\n** This is a deprecated version of the [sqlite3_soft_heap_limit64()]\n** interface. This routine is provided for historical compatibility\n** only. All new applications should use the\n** [sqlite3_soft_heap_limit64()] interface rather than this one.\n*/\nSQLITE_API SQLITE_DEPRECATED void sqlite3_soft_heap_limit(int N);\n\n\n/*\n** CAPI3REF: Extract Metadata About A Column Of A Table\n** METHOD: sqlite3\n**\n** ^(The sqlite3_table_column_metadata(X,D,T,C,....) routine returns\n** information about column C of table T in database D\n** on [database connection] X.)^ ^The sqlite3_table_column_metadata()\n** interface returns SQLITE_OK and fills in the non-NULL pointers in\n** the final five arguments with appropriate values if the specified\n** column exists. ^The sqlite3_table_column_metadata() interface returns\n** SQLITE_ERROR if the specified column does not exist.\n** ^If the column-name parameter to sqlite3_table_column_metadata() is a\n** NULL pointer, then this routine simply checks for the existence of the\n** table and returns SQLITE_OK if the table exists and SQLITE_ERROR if it\n** does not. If the table name parameter T in a call to\n** sqlite3_table_column_metadata(X,D,T,C,...) is NULL then the result is\n** undefined behavior.\n**\n** ^The column is identified by the second, third and fourth parameters to\n** this function. ^(The second parameter is either the name of the database\n** (i.e. \"main\", \"temp\", or an attached database) containing the specified\n** table or NULL.)^ ^If it is NULL, then all attached databases are searched\n** for the table using the same algorithm used by the database engine to\n** resolve unqualified table references.\n**\n** ^The third and fourth parameters to this function are the table and column\n** name of the desired column, respectively.\n**\n** ^Metadata is returned by writing to the memory locations passed as the 5th\n** and subsequent parameters to this function. ^Any of these arguments may be\n** NULL, in which case the corresponding element of metadata is omitted.\n**\n** ^(
\n** \n**
Parameter Output
Type 		Description\n**\n**
5th const char* Data type\n**
6th const char* Name of default collation sequence\n**
7th int True if column has a NOT NULL constraint\n**
8th int True if column is part of the PRIMARY KEY\n**
9th int True if column is [AUTOINCREMENT]\n**
\n**
)^\n**\n** ^The memory pointed to by the character pointers returned for the\n** declaration type and collation sequence is valid until the next\n** call to any SQLite API function.\n**\n** ^If the specified table is actually a view, an [error code] is returned.\n**\n** ^If the specified column is \"rowid\", \"oid\" or \"_rowid_\" and the table\n** is not a [WITHOUT ROWID] table and an\n** [INTEGER PRIMARY KEY] column has been explicitly declared, then the output\n** parameters are set for the explicitly declared column. ^(If there is no\n** [INTEGER PRIMARY KEY] column, then the outputs\n** for the [rowid] are set as follows:\n**\n** 
\n**     data type: \"INTEGER\"\n**     collation sequence: \"BINARY\"\n**     not null: 0\n**     primary key: 1\n**     auto increment: 0\n**
)^\n**\n** ^This function causes all database schemas to be read from disk and\n** parsed, if that has not already been done, and returns an error if\n** any errors are encountered while loading the schema.\n*/\nSQLITE_API int sqlite3_table_column_metadata(\n sqlite3 *db, /* Connection handle */\n const char *zDbName, /* Database name or NULL */\n const char *zTableName, /* Table name */\n const char *zColumnName, /* Column name */\n char const **pzDataType, /* OUTPUT: Declared data type */\n char const **pzCollSeq, /* OUTPUT: Collation sequence name */\n int *pNotNull, /* OUTPUT: True if NOT NULL constraint exists */\n int *pPrimaryKey, /* OUTPUT: True if column part of PK */\n int *pAutoinc /* OUTPUT: True if column is auto-increment */\n);\n\n/*\n** CAPI3REF: Load An Extension\n** METHOD: sqlite3\n**\n** ^This interface loads an SQLite extension library from the named file.\n**\n** ^The sqlite3_load_extension() interface attempts to load an\n** [SQLite extension] library contained in the file zFile. If\n** the file cannot be loaded directly, attempts are made to load\n** with various operating-system specific extensions added.\n** So for example, if \"samplelib\" cannot be loaded, then names like\n** \"samplelib.so\" or \"samplelib.dylib\" or \"samplelib.dll\" might\n** be tried also.\n**\n** ^The entry point is zProc.\n** ^(zProc may be 0, in which case SQLite will try to come up with an\n** entry point name on its own. It first tries \"sqlite3_extension_init\".\n** If that does not work, it constructs a name \"sqlite3_X_init\" where\n** X consists of the lower-case equivalent of all ASCII alphabetic\n** characters in the filename from the last \"/\" to the first following\n** \".\" and omitting any initial \"lib\".)^\n** ^The sqlite3_load_extension() interface returns\n** [SQLITE_OK] on success and [SQLITE_ERROR] if something goes wrong.\n** ^If an error occurs and pzErrMsg is not 0, then the\n** [sqlite3_load_extension()] interface shall attempt to\n** fill *pzErrMsg with error message text stored in memory\n** obtained from [sqlite3_malloc()]. The calling function\n** should free this memory by calling [sqlite3_free()].\n**\n** ^Extension loading must be enabled using\n** [sqlite3_enable_load_extension()] or\n** [sqlite3_db_config](db,[SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION],1,NULL)\n** prior to calling this API,\n** otherwise an error will be returned.\n**\n** Security warning: It is recommended that the\n** [SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION] method be used to enable only this\n** interface. The use of the [sqlite3_enable_load_extension()] interface\n** should be avoided. This will keep the SQL function [load_extension()]\n** disabled and prevent SQL injections from giving attackers\n** access to extension loading capabilities.\n**\n** See also the [load_extension() SQL function].\n*/\nSQLITE_API int sqlite3_load_extension(\n sqlite3 *db, /* Load the extension into this database connection */\n const char *zFile, /* Name of the shared library containing extension */\n const char *zProc, /* Entry point. Derived from zFile if 0 */\n char **pzErrMsg /* Put error message here if not 0 */\n);\n\n/*\n** CAPI3REF: Enable Or Disable Extension Loading\n** METHOD: sqlite3\n**\n** ^So as not to open security holes in older applications that are\n** unprepared to deal with [extension loading], and as a means of disabling\n** [extension loading] while evaluating user-entered SQL, the following API\n** is provided to turn the [sqlite3_load_extension()] mechanism on and off.\n**\n** ^Extension loading is off by default.\n** ^Call the sqlite3_enable_load_extension() routine with onoff==1\n** to turn extension loading on and call it with onoff==0 to turn\n** it back off again.\n**\n** ^This interface enables or disables both the C-API\n** [sqlite3_load_extension()] and the SQL function [load_extension()].\n** ^(Use [sqlite3_db_config](db,[SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION],..)\n** to enable or disable only the C-API.)^\n**\n** Security warning: It is recommended that extension loading\n** be enabled using the [SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION] method\n** rather than this interface, so the [load_extension()] SQL function\n** remains disabled. This will prevent SQL injections from giving attackers\n** access to extension loading capabilities.\n*/\nSQLITE_API int sqlite3_enable_load_extension(sqlite3 *db, int onoff);\n\n/*\n** CAPI3REF: Automatically Load Statically Linked Extensions\n**\n** ^This interface causes the xEntryPoint() function to be invoked for\n** each new [database connection] that is created. The idea here is that\n** xEntryPoint() is the entry point for a statically linked [SQLite extension]\n** that is to be automatically loaded into all new database connections.\n**\n** ^(Even though the function prototype shows that xEntryPoint() takes\n** no arguments and returns void, SQLite invokes xEntryPoint() with three\n** arguments and expects an integer result as if the signature of the\n** entry point were as follows:\n**\n** 
\n**    int xEntryPoint(\n**      sqlite3 *db,\n**      const char **pzErrMsg,\n**      const struct sqlite3_api_routines *pThunk\n**    );\n**
)^\n**\n** If the xEntryPoint routine encounters an error, it should make *pzErrMsg\n** point to an appropriate error message (obtained from [sqlite3_mprintf()])\n** and return an appropriate [error code]. ^SQLite ensures that *pzErrMsg\n** is NULL before calling the xEntryPoint(). ^SQLite will invoke\n** [sqlite3_free()] on *pzErrMsg after xEntryPoint() returns. ^If any\n** xEntryPoint() returns an error, the [sqlite3_open()], [sqlite3_open16()],\n** or [sqlite3_open_v2()] call that provoked the xEntryPoint() will fail.\n**\n** ^Calling sqlite3_auto_extension(X) with an entry point X that is already\n** on the list of automatic extensions is a harmless no-op. ^No entry point\n** will be called more than once for each database connection that is opened.\n**\n** See also: [sqlite3_reset_auto_extension()]\n** and [sqlite3_cancel_auto_extension()]\n*/\nSQLITE_API int sqlite3_auto_extension(void(*xEntryPoint)(void));\n\n/*\n** CAPI3REF: Cancel Automatic Extension Loading\n**\n** ^The [sqlite3_cancel_auto_extension(X)] interface unregisters the\n** initialization routine X that was registered using a prior call to\n** [sqlite3_auto_extension(X)]. ^The [sqlite3_cancel_auto_extension(X)]\n** routine returns 1 if initialization routine X was successfully\n** unregistered and it returns 0 if X was not on the list of initialization\n** routines.\n*/\nSQLITE_API int sqlite3_cancel_auto_extension(void(*xEntryPoint)(void));\n\n/*\n** CAPI3REF: Reset Automatic Extension Loading\n**\n** ^This interface disables all automatic extensions previously\n** registered using [sqlite3_auto_extension()].\n*/\nSQLITE_API void sqlite3_reset_auto_extension(void);\n\n/*\n** Structures used by the virtual table interface\n*/\ntypedef struct sqlite3_vtab sqlite3_vtab;\ntypedef struct sqlite3_index_info sqlite3_index_info;\ntypedef struct sqlite3_vtab_cursor sqlite3_vtab_cursor;\ntypedef struct sqlite3_module sqlite3_module;\n\n/*\n** CAPI3REF: Virtual Table Object\n** KEYWORDS: sqlite3_module {virtual table module}\n**\n** This structure, sometimes called a \"virtual table module\",\n** defines the implementation of a [virtual table].\n** This structure consists mostly of methods for the module.\n**\n** ^A virtual table module is created by filling in a persistent\n** instance of this structure and passing a pointer to that instance\n** to [sqlite3_create_module()] or [sqlite3_create_module_v2()].\n** ^The registration remains valid until it is replaced by a different\n** module or until the [database connection] closes. The content\n** of this structure must not change while it is registered with\n** any database connection.\n*/\nstruct sqlite3_module {\n int iVersion;\n int (*xCreate)(sqlite3*, void *pAux,\n int argc, const char *const*argv,\n sqlite3_vtab **ppVTab, char**);\n int (*xConnect)(sqlite3*, void *pAux,\n int argc, const char *const*argv,\n sqlite3_vtab **ppVTab, char**);\n int (*xBestIndex)(sqlite3_vtab *pVTab, sqlite3_index_info*);\n int (*xDisconnect)(sqlite3_vtab *pVTab);\n int (*xDestroy)(sqlite3_vtab *pVTab);\n int (*xOpen)(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor);\n int (*xClose)(sqlite3_vtab_cursor*);\n int (*xFilter)(sqlite3_vtab_cursor*, int idxNum, const char *idxStr,\n int argc, sqlite3_value **argv);\n int (*xNext)(sqlite3_vtab_cursor*);\n int (*xEof)(sqlite3_vtab_cursor*);\n int (*xColumn)(sqlite3_vtab_cursor*, sqlite3_context*, int);\n int (*xRowid)(sqlite3_vtab_cursor*, sqlite3_int64 *pRowid);\n int (*xUpdate)(sqlite3_vtab *, int, sqlite3_value **, sqlite3_int64 *);\n int (*xBegin)(sqlite3_vtab *pVTab);\n int (*xSync)(sqlite3_vtab *pVTab);\n int (*xCommit)(sqlite3_vtab *pVTab);\n int (*xRollback)(sqlite3_vtab *pVTab);\n int (*xFindFunction)(sqlite3_vtab *pVtab, int nArg, const char *zName,\n void (**pxFunc)(sqlite3_context*,int,sqlite3_value**),\n void **ppArg);\n int (*xRename)(sqlite3_vtab *pVtab, const char *zNew);\n /* The methods above are in version 1 of the sqlite_module object. Those\n ** below are for version 2 and greater. */\n int (*xSavepoint)(sqlite3_vtab *pVTab, int);\n int (*xRelease)(sqlite3_vtab *pVTab, int);\n int (*xRollbackTo)(sqlite3_vtab *pVTab, int);\n /* The methods above are in versions 1 and 2 of the sqlite_module object.\n ** Those below are for version 3 and greater. */\n int (*xShadowName)(const char*);\n /* The methods above are in versions 1 through 3 of the sqlite_module object.\n ** Those below are for version 4 and greater. */\n int (*xIntegrity)(sqlite3_vtab *pVTab, const char *zSchema,\n const char *zTabName, int mFlags, char **pzErr);\n};\n\n/*\n** CAPI3REF: Virtual Table Indexing Information\n** KEYWORDS: sqlite3_index_info\n**\n** The sqlite3_index_info structure and its substructures is used as part\n** of the [virtual table] interface to\n** pass information into and receive the reply from the [xBestIndex]\n** method of a [virtual table module]. The fields under **Inputs** are the\n** inputs to xBestIndex and are read-only. xBestIndex inserts its\n** results into the **Outputs** fields.\n**\n** ^(The aConstraint[] array records WHERE clause constraints of the form:\n**\n**
column OP expr
\n**\n** where OP is =, <, <=, >, or >=.)^ ^(The particular operator is\n** stored in aConstraint[].op using one of the\n** [SQLITE_INDEX_CONSTRAINT_EQ | SQLITE_INDEX_CONSTRAINT_ values].)^\n** ^(The index of the column is stored in\n** aConstraint[].iColumn.)^ ^(aConstraint[].usable is TRUE if the\n** expr on the right-hand side can be evaluated (and thus the constraint\n** is usable) and false if it cannot.)^\n**\n** ^The optimizer automatically inverts terms of the form \"expr OP column\"\n** and makes other simplifications to the WHERE clause in an attempt to\n** get as many WHERE clause terms into the form shown above as possible.\n** ^The aConstraint[] array only reports WHERE clause terms that are\n** relevant to the particular virtual table being queried.\n**\n** ^Information about the ORDER BY clause is stored in aOrderBy[].\n** ^Each term of aOrderBy records a column of the ORDER BY clause.\n**\n** The colUsed field indicates which columns of the virtual table may be\n** required by the current scan. Virtual table columns are numbered from\n** zero in the order in which they appear within the CREATE TABLE statement\n** passed to sqlite3_declare_vtab(). For the first 63 columns (columns 0-62),\n** the corresponding bit is set within the colUsed mask if the column may be\n** required by SQLite. If the table has at least 64 columns and any column\n** to the right of the first 63 is required, then bit 63 of colUsed is also\n** set. In other words, column iCol may be required if the expression\n** (colUsed & ((sqlite3_uint64)1 << (iCol>=63 ? 63 : iCol))) evaluates to\n** non-zero.\n**\n** The [xBestIndex] method must fill aConstraintUsage[] with information\n** about what parameters to pass to xFilter. ^If argvIndex>0 then\n** the right-hand side of the corresponding aConstraint[] is evaluated\n** and becomes the argvIndex-th entry in argv. ^(If aConstraintUsage[].omit\n** is true, then the constraint is assumed to be fully handled by the\n** virtual table and might not be checked again by the byte code.)^ ^(The\n** aConstraintUsage[].omit flag is an optimization hint. When the omit flag\n** is left in its default setting of false, the constraint will always be\n** checked separately in byte code. If the omit flag is changed to true, then\n** the constraint may or may not be checked in byte code. In other words,\n** when the omit flag is true there is no guarantee that the constraint will\n** not be checked again using byte code.)^\n**\n** ^The idxNum and idxStr values are recorded and passed into the\n** [xFilter] method.\n** ^[sqlite3_free()] is used to free idxStr if and only if\n** needToFreeIdxStr is true.\n**\n** ^The orderByConsumed means that output from [xFilter]/[xNext] will occur in\n** the correct order to satisfy the ORDER BY clause so that no separate\n** sorting step is required.\n**\n** ^The estimatedCost value is an estimate of the cost of a particular\n** strategy. A cost of N indicates that the cost of the strategy is similar\n** to a linear scan of an SQLite table with N rows. A cost of log(N)\n** indicates that the expense of the operation is similar to that of a\n** binary search on a unique indexed field of an SQLite table with N rows.\n**\n** ^The estimatedRows value is an estimate of the number of rows that\n** will be returned by the strategy.\n**\n** The xBestIndex method may optionally populate the idxFlags field with a\n** mask of SQLITE_INDEX_SCAN_* flags. One such flag is\n** [SQLITE_INDEX_SCAN_HEX], which if set causes the [EXPLAIN QUERY PLAN]\n** output to show the idxNum as hex instead of as decimal. Another flag is\n** SQLITE_INDEX_SCAN_UNIQUE, which if set indicates that the query plan will\n** return at most one row.\n**\n** Additionally, if xBestIndex sets the SQLITE_INDEX_SCAN_UNIQUE flag, then\n** SQLite also assumes that if a call to the xUpdate() method is made as\n** part of the same statement to delete or update a virtual table row and the\n** implementation returns SQLITE_CONSTRAINT, then there is no need to rollback\n** any database changes. In other words, if the xUpdate() returns\n** SQLITE_CONSTRAINT, the database contents must be exactly as they were\n** before xUpdate was called. By contrast, if SQLITE_INDEX_SCAN_UNIQUE is not\n** set and xUpdate returns SQLITE_CONSTRAINT, any database changes made by\n** the xUpdate method are automatically rolled back by SQLite.\n**\n** IMPORTANT: The estimatedRows field was added to the sqlite3_index_info\n** structure for SQLite [version 3.8.2] ([dateof:3.8.2]).\n** If a virtual table extension is\n** used with an SQLite version earlier than 3.8.2, the results of attempting\n** to read or write the estimatedRows field are undefined (but are likely\n** to include crashing the application). The estimatedRows field should\n** therefore only be used if [sqlite3_libversion_number()] returns a\n** value greater than or equal to 3008002. Similarly, the idxFlags field\n** was added for [version 3.9.0] ([dateof:3.9.0]).\n** It may therefore only be used if\n** sqlite3_libversion_number() returns a value greater than or equal to\n** 3009000.\n*/\nstruct sqlite3_index_info {\n /* Inputs */\n int nConstraint; /* Number of entries in aConstraint */\n struct sqlite3_index_constraint {\n int iColumn; /* Column constrained. -1 for ROWID */\n unsigned char op; /* Constraint operator */\n unsigned char usable; /* True if this constraint is usable */\n int iTermOffset; /* Used internally - xBestIndex should ignore */\n } *aConstraint; /* Table of WHERE clause constraints */\n int nOrderBy; /* Number of terms in the ORDER BY clause */\n struct sqlite3_index_orderby {\n int iColumn; /* Column number */\n unsigned char desc; /* True for DESC. False for ASC. */\n } *aOrderBy; /* The ORDER BY clause */\n /* Outputs */\n struct sqlite3_index_constraint_usage {\n int argvIndex; /* if >0, constraint is part of argv to xFilter */\n unsigned char omit; /* Do not code a test for this constraint */\n } *aConstraintUsage;\n int idxNum; /* Number used to identify the index */\n char *idxStr; /* String, possibly obtained from sqlite3_malloc */\n int needToFreeIdxStr; /* Free idxStr using sqlite3_free() if true */\n int orderByConsumed; /* True if output is already ordered */\n double estimatedCost; /* Estimated cost of using this index */\n /* Fields below are only available in SQLite 3.8.2 and later */\n sqlite3_int64 estimatedRows; /* Estimated number of rows returned */\n /* Fields below are only available in SQLite 3.9.0 and later */\n int idxFlags; /* Mask of SQLITE_INDEX_SCAN_* flags */\n /* Fields below are only available in SQLite 3.10.0 and later */\n sqlite3_uint64 colUsed; /* Input: Mask of columns used by statement */\n};\n\n/*\n** CAPI3REF: Virtual Table Scan Flags\n**\n** Virtual table implementations are allowed to set the\n** [sqlite3_index_info].idxFlags field to some combination of\n** these bits.\n*/\n#define SQLITE_INDEX_SCAN_UNIQUE 0x00000001 /* Scan visits at most 1 row */\n#define SQLITE_INDEX_SCAN_HEX 0x00000002 /* Display idxNum as hex */\n /* in EXPLAIN QUERY PLAN */\n\n/*\n** CAPI3REF: Virtual Table Constraint Operator Codes\n**\n** These macros define the allowed values for the\n** [sqlite3_index_info].aConstraint[].op field. Each value represents\n** an operator that is part of a constraint term in the WHERE clause of\n** a query that uses a [virtual table].\n**\n** ^The left-hand operand of the operator is given by the corresponding\n** aConstraint[].iColumn field. ^An iColumn of -1 indicates the left-hand\n** operand is the rowid.\n** The SQLITE_INDEX_CONSTRAINT_LIMIT and SQLITE_INDEX_CONSTRAINT_OFFSET\n** operators have no left-hand operand, and so for those operators the\n** corresponding aConstraint[].iColumn is meaningless and should not be\n** used.\n**\n** All operator values from SQLITE_INDEX_CONSTRAINT_FUNCTION through\n** value 255 are reserved to represent functions that are overloaded\n** by the [xFindFunction|xFindFunction method] of the virtual table\n** implementation.\n**\n** The right-hand operands for each constraint might be accessible using\n** the [sqlite3_vtab_rhs_value()] interface. Usually the right-hand\n** operand is only available if it appears as a single constant literal\n** in the input SQL. If the right-hand operand is another column or an\n** expression (even a constant expression) or a parameter, then the\n** sqlite3_vtab_rhs_value() probably will not be able to extract it.\n** ^The SQLITE_INDEX_CONSTRAINT_ISNULL and\n** SQLITE_INDEX_CONSTRAINT_ISNOTNULL operators have no right-hand operand\n** and hence calls to sqlite3_vtab_rhs_value() for those operators will\n** always return SQLITE_NOTFOUND.\n**\n** The collating sequence to be used for comparison can be found using\n** the [sqlite3_vtab_collation()] interface. For most real-world virtual\n** tables, the collating sequence of constraints does not matter (for example\n** because the constraints are numeric) and so the sqlite3_vtab_collation()\n** interface is not commonly needed.\n*/\n#define SQLITE_INDEX_CONSTRAINT_EQ 2\n#define SQLITE_INDEX_CONSTRAINT_GT 4\n#define SQLITE_INDEX_CONSTRAINT_LE 8\n#define SQLITE_INDEX_CONSTRAINT_LT 16\n#define SQLITE_INDEX_CONSTRAINT_GE 32\n#define SQLITE_INDEX_CONSTRAINT_MATCH 64\n#define SQLITE_INDEX_CONSTRAINT_LIKE 65\n#define SQLITE_INDEX_CONSTRAINT_GLOB 66\n#define SQLITE_INDEX_CONSTRAINT_REGEXP 67\n#define SQLITE_INDEX_CONSTRAINT_NE 68\n#define SQLITE_INDEX_CONSTRAINT_ISNOT 69\n#define SQLITE_INDEX_CONSTRAINT_ISNOTNULL 70\n#define SQLITE_INDEX_CONSTRAINT_ISNULL 71\n#define SQLITE_INDEX_CONSTRAINT_IS 72\n#define SQLITE_INDEX_CONSTRAINT_LIMIT 73\n#define SQLITE_INDEX_CONSTRAINT_OFFSET 74\n#define SQLITE_INDEX_CONSTRAINT_FUNCTION 150\n\n/*\n** CAPI3REF: Register A Virtual Table Implementation\n** METHOD: sqlite3\n**\n** ^These routines are used to register a new [virtual table module] name.\n** ^Module names must be registered before\n** creating a new [virtual table] using the module and before using a\n** preexisting [virtual table] for the module.\n**\n** ^The module name is registered on the [database connection] specified\n** by the first parameter. ^The name of the module is given by the\n** second parameter. ^The third parameter is a pointer to\n** the implementation of the [virtual table module]. ^The fourth\n** parameter is an arbitrary client data pointer that is passed through\n** into the [xCreate] and [xConnect] methods of the virtual table module\n** when a new virtual table is being created or reinitialized.\n**\n** ^The sqlite3_create_module_v2() interface has a fifth parameter which\n** is a pointer to a destructor for the pClientData. ^SQLite will\n** invoke the destructor function (if it is not NULL) when SQLite\n** no longer needs the pClientData pointer. ^The destructor will also\n** be invoked if the call to sqlite3_create_module_v2() fails.\n** ^The sqlite3_create_module()\n** interface is equivalent to sqlite3_create_module_v2() with a NULL\n** destructor.\n**\n** ^If the third parameter (the pointer to the sqlite3_module object) is\n** NULL then no new module is created and any existing modules with the\n** same name are dropped.\n**\n** See also: [sqlite3_drop_modules()]\n*/\nSQLITE_API int sqlite3_create_module(\n sqlite3 *db, /* SQLite connection to register module with */\n const char *zName, /* Name of the module */\n const sqlite3_module *p, /* Methods for the module */\n void *pClientData /* Client data for xCreate/xConnect */\n);\nSQLITE_API int sqlite3_create_module_v2(\n sqlite3 *db, /* SQLite connection to register module with */\n const char *zName, /* Name of the module */\n const sqlite3_module *p, /* Methods for the module */\n void *pClientData, /* Client data for xCreate/xConnect */\n void(*xDestroy)(void*) /* Module destructor function */\n);\n\n/*\n** CAPI3REF: Remove Unnecessary Virtual Table Implementations\n** METHOD: sqlite3\n**\n** ^The sqlite3_drop_modules(D,L) interface removes all virtual\n** table modules from database connection D except those named on list L.\n** The L parameter must be either NULL or a pointer to an array of pointers\n** to strings where the array is terminated by a single NULL pointer.\n** ^If the L parameter is NULL, then all virtual table modules are removed.\n**\n** See also: [sqlite3_create_module()]\n*/\nSQLITE_API int sqlite3_drop_modules(\n sqlite3 *db, /* Remove modules from this connection */\n const char **azKeep /* Except, do not remove the ones named here */\n);\n\n/*\n** CAPI3REF: Virtual Table Instance Object\n** KEYWORDS: sqlite3_vtab\n**\n** Every [virtual table module] implementation uses a subclass\n** of this object to describe a particular instance\n** of the [virtual table]. Each subclass will\n** be tailored to the specific needs of the module implementation.\n** The purpose of this superclass is to define certain fields that are\n** common to all module implementations.\n**\n** ^Virtual tables methods can set an error message by assigning a\n** string obtained from [sqlite3_mprintf()] to zErrMsg. The method should\n** take care that any prior string is freed by a call to [sqlite3_free()]\n** prior to assigning a new string to zErrMsg. ^After the error message\n** is delivered up to the client application, the string will be automatically\n** freed by sqlite3_free() and the zErrMsg field will be zeroed.\n*/\nstruct sqlite3_vtab {\n const sqlite3_module *pModule; /* The module for this virtual table */\n int nRef; /* Number of open cursors */\n char *zErrMsg; /* Error message from sqlite3_mprintf() */\n /* Virtual table implementations will typically add additional fields */\n};\n\n/*\n** CAPI3REF: Virtual Table Cursor Object\n** KEYWORDS: sqlite3_vtab_cursor {virtual table cursor}\n**\n** Every [virtual table module] implementation uses a subclass of the\n** following structure to describe cursors that point into the\n** [virtual table] and are used\n** to loop through the virtual table. Cursors are created using the\n** [sqlite3_module.xOpen | xOpen] method of the module and are destroyed\n** by the [sqlite3_module.xClose | xClose] method. Cursors are used\n** by the [xFilter], [xNext], [xEof], [xColumn], and [xRowid] methods\n** of the module. Each module implementation will define\n** the content of a cursor structure to suit its own needs.\n**\n** This superclass exists in order to define fields of the cursor that\n** are common to all implementations.\n*/\nstruct sqlite3_vtab_cursor {\n sqlite3_vtab *pVtab; /* Virtual table of this cursor */\n /* Virtual table implementations will typically add additional fields */\n};\n\n/*\n** CAPI3REF: Declare The Schema Of A Virtual Table\n**\n** ^The [xCreate] and [xConnect] methods of a\n** [virtual table module] call this interface\n** to declare the format (the names and datatypes of the columns) of\n** the virtual tables they implement.\n*/\nSQLITE_API int sqlite3_declare_vtab(sqlite3*, const char *zSQL);\n\n/*\n** CAPI3REF: Overload A Function For A Virtual Table\n** METHOD: sqlite3\n**\n** ^(Virtual tables can provide alternative implementations of functions\n** using the [xFindFunction] method of the [virtual table module].\n** But global versions of those functions\n** must exist in order to be overloaded.)^\n**\n** ^(This API makes sure a global version of a function with a particular\n** name and number of parameters exists. If no such function exists\n** before this API is called, a new function is created.)^ ^The implementation\n** of the new function always causes an exception to be thrown. So\n** the new function is not good for anything by itself. Its only\n** purpose is to be a placeholder function that can be overloaded\n** by a [virtual table].\n*/\nSQLITE_API int sqlite3_overload_function(sqlite3*, const char *zFuncName, int nArg);\n\n/*\n** CAPI3REF: A Handle To An Open BLOB\n** KEYWORDS: {BLOB handle} {BLOB handles}\n**\n** An instance of this object represents an open BLOB on which\n** [sqlite3_blob_open | incremental BLOB I/O] can be performed.\n** ^Objects of this type are created by [sqlite3_blob_open()]\n** and destroyed by [sqlite3_blob_close()].\n** ^The [sqlite3_blob_read()] and [sqlite3_blob_write()] interfaces\n** can be used to read or write small subsections of the BLOB.\n** ^The [sqlite3_blob_bytes()] interface returns the size of the BLOB in bytes.\n*/\ntypedef struct sqlite3_blob sqlite3_blob;\n\n/*\n** CAPI3REF: Open A BLOB For Incremental I/O\n** METHOD: sqlite3\n** CONSTRUCTOR: sqlite3_blob\n**\n** ^(This interfaces opens a [BLOB handle | handle] to the BLOB located\n** in row iRow, column zColumn, table zTable in database zDb;\n** in other words, the same BLOB that would be selected by:\n**\n** 
\n**     SELECT zColumn FROM zDb.zTable WHERE [rowid] = iRow;\n**
)^\n**\n** ^(Parameter zDb is not the filename that contains the database, but\n** rather the symbolic name of the database. For attached databases, this is\n** the name that appears after the AS keyword in the [ATTACH] statement.\n** For the main database file, the database name is \"main\". For TEMP\n** tables, the database name is \"temp\".)^\n**\n** ^If the flags parameter is non-zero, then the BLOB is opened for read\n** and write access. ^If the flags parameter is zero, the BLOB is opened for\n** read-only access.\n**\n** ^(On success, [SQLITE_OK] is returned and the new [BLOB handle] is stored\n** in *ppBlob. Otherwise an [error code] is returned and, unless the error\n** code is SQLITE_MISUSE, *ppBlob is set to NULL.)^ ^This means that, provided\n** the API is not misused, it is always safe to call [sqlite3_blob_close()]\n** on *ppBlob after this function returns.\n**\n** This function fails with SQLITE_ERROR if any of the following are true:\n**
    \n**
  • ^(Database zDb does not exist)^,\n**
  • ^(Table zTable does not exist within database zDb)^,\n**
  • ^(Table zTable is a WITHOUT ROWID table)^,\n**
  • ^(Column zColumn does not exist)^,\n**
  • ^(Row iRow is not present in the table)^,\n**
  • ^(The specified column of row iRow contains a value that is not\n** a TEXT or BLOB value)^,\n**
  • ^(Column zColumn is part of an index, PRIMARY KEY or UNIQUE\n** constraint and the blob is being opened for read/write access)^,\n**
  • ^([foreign key constraints | Foreign key constraints] are enabled,\n** column zColumn is part of a [child key] definition and the blob is\n** being opened for read/write access)^.\n**
\n**\n** ^Unless it returns SQLITE_MISUSE, this function sets the\n** [database connection] error code and message accessible via\n** [sqlite3_errcode()] and [sqlite3_errmsg()] and related functions.\n**\n** A BLOB referenced by sqlite3_blob_open() may be read using the\n** [sqlite3_blob_read()] interface and modified by using\n** [sqlite3_blob_write()]. The [BLOB handle] can be moved to a\n** different row of the same table using the [sqlite3_blob_reopen()]\n** interface. However, the column, table, or database of a [BLOB handle]\n** cannot be changed after the [BLOB handle] is opened.\n**\n** ^(If the row that a BLOB handle points to is modified by an\n** [UPDATE], [DELETE], or by [ON CONFLICT] side-effects\n** then the BLOB handle is marked as \"expired\".\n** This is true if any column of the row is changed, even a column\n** other than the one the BLOB handle is open on.)^\n** ^Calls to [sqlite3_blob_read()] and [sqlite3_blob_write()] for\n** an expired BLOB handle fail with a return code of [SQLITE_ABORT].\n** ^(Changes written into a BLOB prior to the BLOB expiring are not\n** rolled back by the expiration of the BLOB. Such changes will eventually\n** commit if the transaction continues to completion.)^\n**\n** ^Use the [sqlite3_blob_bytes()] interface to determine the size of\n** the opened blob. ^The size of a blob may not be changed by this\n** interface. Use the [UPDATE] SQL command to change the size of a\n** blob.\n**\n** ^The [sqlite3_bind_zeroblob()] and [sqlite3_result_zeroblob()] interfaces\n** and the built-in [zeroblob] SQL function may be used to create a\n** zero-filled blob to read or write using the incremental-blob interface.\n**\n** To avoid a resource leak, every open [BLOB handle] should eventually\n** be released by a call to [sqlite3_blob_close()].\n**\n** See also: [sqlite3_blob_close()],\n** [sqlite3_blob_reopen()], [sqlite3_blob_read()],\n** [sqlite3_blob_bytes()], [sqlite3_blob_write()].\n*/\nSQLITE_API int sqlite3_blob_open(\n sqlite3*,\n const char *zDb,\n const char *zTable,\n const char *zColumn,\n sqlite3_int64 iRow,\n int flags,\n sqlite3_blob **ppBlob\n);\n\n/*\n** CAPI3REF: Move a BLOB Handle to a New Row\n** METHOD: sqlite3_blob\n**\n** ^This function is used to move an existing [BLOB handle] so that it points\n** to a different row of the same database table. ^The new row is identified\n** by the rowid value passed as the second argument. Only the row can be\n** changed. ^The database, table and column on which the blob handle is open\n** remain the same. Moving an existing [BLOB handle] to a new row is\n** faster than closing the existing handle and opening a new one.\n**\n** ^(The new row must meet the same criteria as for [sqlite3_blob_open()] -\n** it must exist and there must be either a blob or text value stored in\n** the nominated column.)^ ^If the new row is not present in the table, or if\n** it does not contain a blob or text value, or if another error occurs, an\n** SQLite error code is returned and the blob handle is considered aborted.\n** ^All subsequent calls to [sqlite3_blob_read()], [sqlite3_blob_write()] or\n** [sqlite3_blob_reopen()] on an aborted blob handle immediately return\n** SQLITE_ABORT. ^Calling [sqlite3_blob_bytes()] on an aborted blob handle\n** always returns zero.\n**\n** ^This function sets the database handle error code and message.\n*/\nSQLITE_API int sqlite3_blob_reopen(sqlite3_blob *, sqlite3_int64);\n\n/*\n** CAPI3REF: Close A BLOB Handle\n** DESTRUCTOR: sqlite3_blob\n**\n** ^This function closes an open [BLOB handle]. ^(The BLOB handle is closed\n** unconditionally. Even if this routine returns an error code, the\n** handle is still closed.)^\n**\n** ^If the blob handle being closed was opened for read-write access, and if\n** the database is in auto-commit mode and there are no other open read-write\n** blob handles or active write statements, the current transaction is\n** committed. ^If an error occurs while committing the transaction, an error\n** code is returned and the transaction rolled back.\n**\n** Calling this function with an argument that is not a NULL pointer or an\n** open blob handle results in undefined behavior. ^Calling this routine\n** with a null pointer (such as would be returned by a failed call to\n** [sqlite3_blob_open()]) is a harmless no-op. ^Otherwise, if this function\n** is passed a valid open blob handle, the values returned by the\n** sqlite3_errcode() and sqlite3_errmsg() functions are set before returning.\n*/\nSQLITE_API int sqlite3_blob_close(sqlite3_blob *);\n\n/*\n** CAPI3REF: Return The Size Of An Open BLOB\n** METHOD: sqlite3_blob\n**\n** ^Returns the size in bytes of the BLOB accessible via the\n** successfully opened [BLOB handle] in its only argument. ^The\n** incremental blob I/O routines can only read or overwrite existing\n** blob content; they cannot change the size of a blob.\n**\n** This routine only works on a [BLOB handle] which has been created\n** by a prior successful call to [sqlite3_blob_open()] and which has not\n** been closed by [sqlite3_blob_close()]. Passing any other pointer in\n** to this routine results in undefined and probably undesirable behavior.\n*/\nSQLITE_API int sqlite3_blob_bytes(sqlite3_blob *);\n\n/*\n** CAPI3REF: Read Data From A BLOB Incrementally\n** METHOD: sqlite3_blob\n**\n** ^(This function is used to read data from an open [BLOB handle] into a\n** caller-supplied buffer. N bytes of data are copied into buffer Z\n** from the open BLOB, starting at offset iOffset.)^\n**\n** ^If offset iOffset is less than N bytes from the end of the BLOB,\n** [SQLITE_ERROR] is returned and no data is read. ^If N or iOffset is\n** less than zero, [SQLITE_ERROR] is returned and no data is read.\n** ^The size of the blob (and hence the maximum value of N+iOffset)\n** can be determined using the [sqlite3_blob_bytes()] interface.\n**\n** ^An attempt to read from an expired [BLOB handle] fails with an\n** error code of [SQLITE_ABORT].\n**\n** ^(On success, sqlite3_blob_read() returns SQLITE_OK.\n** Otherwise, an [error code] or an [extended error code] is returned.)^\n**\n** This routine only works on a [BLOB handle] which has been created\n** by a prior successful call to [sqlite3_blob_open()] and which has not\n** been closed by [sqlite3_blob_close()]. Passing any other pointer in\n** to this routine results in undefined and probably undesirable behavior.\n**\n** See also: [sqlite3_blob_write()].\n*/\nSQLITE_API int sqlite3_blob_read(sqlite3_blob *, void *Z, int N, int iOffset);\n\n/*\n** CAPI3REF: Write Data Into A BLOB Incrementally\n** METHOD: sqlite3_blob\n**\n** ^(This function is used to write data into an open [BLOB handle] from a\n** caller-supplied buffer. N bytes of data are copied from the buffer Z\n** into the open BLOB, starting at offset iOffset.)^\n**\n** ^(On success, sqlite3_blob_write() returns SQLITE_OK.\n** Otherwise, an [error code] or an [extended error code] is returned.)^\n** ^Unless SQLITE_MISUSE is returned, this function sets the\n** [database connection] error code and message accessible via\n** [sqlite3_errcode()] and [sqlite3_errmsg()] and related functions.\n**\n** ^If the [BLOB handle] passed as the first argument was not opened for\n** writing (the flags parameter to [sqlite3_blob_open()] was zero),\n** this function returns [SQLITE_READONLY].\n**\n** This function may only modify the contents of the BLOB; it is\n** not possible to increase the size of a BLOB using this API.\n** ^If offset iOffset is less than N bytes from the end of the BLOB,\n** [SQLITE_ERROR] is returned and no data is written. The size of the\n** BLOB (and hence the maximum value of N+iOffset) can be determined\n** using the [sqlite3_blob_bytes()] interface. ^If N or iOffset are less\n** than zero [SQLITE_ERROR] is returned and no data is written.\n**\n** ^An attempt to write to an expired [BLOB handle] fails with an\n** error code of [SQLITE_ABORT]. ^Writes to the BLOB that occurred\n** before the [BLOB handle] expired are not rolled back by the\n** expiration of the handle, though of course those changes might\n** have been overwritten by the statement that expired the BLOB handle\n** or by other independent statements.\n**\n** This routine only works on a [BLOB handle] which has been created\n** by a prior successful call to [sqlite3_blob_open()] and which has not\n** been closed by [sqlite3_blob_close()]. Passing any other pointer in\n** to this routine results in undefined and probably undesirable behavior.\n**\n** See also: [sqlite3_blob_read()].\n*/\nSQLITE_API int sqlite3_blob_write(sqlite3_blob *, const void *z, int n, int iOffset);\n\n/*\n** CAPI3REF: Virtual File System Objects\n**\n** A virtual filesystem (VFS) is an [sqlite3_vfs] object\n** that SQLite uses to interact\n** with the underlying operating system. Most SQLite builds come with a\n** single default VFS that is appropriate for the host computer.\n** New VFSes can be registered and existing VFSes can be unregistered.\n** The following interfaces are provided.\n**\n** ^The sqlite3_vfs_find() interface returns a pointer to a VFS given its name.\n** ^Names are case sensitive.\n** ^Names are zero-terminated UTF-8 strings.\n** ^If there is no match, a NULL pointer is returned.\n** ^If zVfsName is NULL then the default VFS is returned.\n**\n** ^New VFSes are registered with sqlite3_vfs_register().\n** ^Each new VFS becomes the default VFS if the makeDflt flag is set.\n** ^The same VFS can be registered multiple times without injury.\n** ^To make an existing VFS into the default VFS, register it again\n** with the makeDflt flag set. If two different VFSes with the\n** same name are registered, the behavior is undefined. If a\n** VFS is registered with a name that is NULL or an empty string,\n** then the behavior is undefined.\n**\n** ^Unregister a VFS with the sqlite3_vfs_unregister() interface.\n** ^(If the default VFS is unregistered, another VFS is chosen as\n** the default. The choice for the new VFS is arbitrary.)^\n*/\nSQLITE_API sqlite3_vfs *sqlite3_vfs_find(const char *zVfsName);\nSQLITE_API int sqlite3_vfs_register(sqlite3_vfs*, int makeDflt);\nSQLITE_API int sqlite3_vfs_unregister(sqlite3_vfs*);\n\n/*\n** CAPI3REF: Mutexes\n**\n** The SQLite core uses these routines for thread\n** synchronization. Though they are intended for internal\n** use by SQLite, code that links against SQLite is\n** permitted to use any of these routines.\n**\n** The SQLite source code contains multiple implementations\n** of these mutex routines. An appropriate implementation\n** is selected automatically at compile-time. The following\n** implementations are available in the SQLite core:\n**\n**
    \n**
  • SQLITE_MUTEX_PTHREADS\n**
  • SQLITE_MUTEX_W32\n**
  • SQLITE_MUTEX_NOOP\n**
\n**\n** The SQLITE_MUTEX_NOOP implementation is a set of routines\n** that does no real locking and is appropriate for use in\n** a single-threaded application. The SQLITE_MUTEX_PTHREADS and\n** SQLITE_MUTEX_W32 implementations are appropriate for use on Unix\n** and Windows.\n**\n** If SQLite is compiled with the SQLITE_MUTEX_APPDEF preprocessor\n** macro defined (with \"-DSQLITE_MUTEX_APPDEF=1\"), then no mutex\n** implementation is included with the library. In this case the\n** application must supply a custom mutex implementation using the\n** [SQLITE_CONFIG_MUTEX] option of the sqlite3_config() function\n** before calling sqlite3_initialize() or any other public sqlite3_\n** function that calls sqlite3_initialize().\n**\n** ^The sqlite3_mutex_alloc() routine allocates a new\n** mutex and returns a pointer to it. ^The sqlite3_mutex_alloc()\n** routine returns NULL if it is unable to allocate the requested\n** mutex. The argument to sqlite3_mutex_alloc() must be one of these\n** integer constants:\n**\n**
    \n**
  • SQLITE_MUTEX_FAST\n**
  • SQLITE_MUTEX_RECURSIVE\n**
  • SQLITE_MUTEX_STATIC_MAIN\n**
  • SQLITE_MUTEX_STATIC_MEM\n**
  • SQLITE_MUTEX_STATIC_OPEN\n**
  • SQLITE_MUTEX_STATIC_PRNG\n**
  • SQLITE_MUTEX_STATIC_LRU\n**
  • SQLITE_MUTEX_STATIC_PMEM\n**
  • SQLITE_MUTEX_STATIC_APP1\n**
  • SQLITE_MUTEX_STATIC_APP2\n**
  • SQLITE_MUTEX_STATIC_APP3\n**
  • SQLITE_MUTEX_STATIC_VFS1\n**
  • SQLITE_MUTEX_STATIC_VFS2\n**
  • SQLITE_MUTEX_STATIC_VFS3\n**
\n**\n** ^The first two constants (SQLITE_MUTEX_FAST and SQLITE_MUTEX_RECURSIVE)\n** cause sqlite3_mutex_alloc() to create\n** a new mutex. ^The new mutex is recursive when SQLITE_MUTEX_RECURSIVE\n** is used but not necessarily so when SQLITE_MUTEX_FAST is used.\n** The mutex implementation does not need to make a distinction\n** between SQLITE_MUTEX_RECURSIVE and SQLITE_MUTEX_FAST if it does\n** not want to. SQLite will only request a recursive mutex in\n** cases where it really needs one. If a faster non-recursive mutex\n** implementation is available on the host platform, the mutex subsystem\n** might return such a mutex in response to SQLITE_MUTEX_FAST.\n**\n** ^The other allowed parameters to sqlite3_mutex_alloc() (anything other\n** than SQLITE_MUTEX_FAST and SQLITE_MUTEX_RECURSIVE) each return\n** a pointer to a static preexisting mutex. ^Nine static mutexes are\n** used by the current version of SQLite. Future versions of SQLite\n** may add additional static mutexes. Static mutexes are for internal\n** use by SQLite only. Applications that use SQLite mutexes should\n** use only the dynamic mutexes returned by SQLITE_MUTEX_FAST or\n** SQLITE_MUTEX_RECURSIVE.\n**\n** ^Note that if one of the dynamic mutex parameters (SQLITE_MUTEX_FAST\n** or SQLITE_MUTEX_RECURSIVE) is used then sqlite3_mutex_alloc()\n** returns a different mutex on every call. ^For the static\n** mutex types, the same mutex is returned on every call that has\n** the same type number.\n**\n** ^The sqlite3_mutex_free() routine deallocates a previously\n** allocated dynamic mutex. Attempting to deallocate a static\n** mutex results in undefined behavior.\n**\n** ^The sqlite3_mutex_enter() and sqlite3_mutex_try() routines attempt\n** to enter a mutex. ^If another thread is already within the mutex,\n** sqlite3_mutex_enter() will block and sqlite3_mutex_try() will return\n** SQLITE_BUSY. ^The sqlite3_mutex_try() interface returns [SQLITE_OK]\n** upon successful entry. ^(Mutexes created using\n** SQLITE_MUTEX_RECURSIVE can be entered multiple times by the same thread.\n** In such cases, the\n** mutex must be exited an equal number of times before another thread\n** can enter.)^ If the same thread tries to enter any mutex other\n** than an SQLITE_MUTEX_RECURSIVE more than once, the behavior is undefined.\n**\n** ^(Some systems (for example, Windows 95) do not support the operation\n** implemented by sqlite3_mutex_try(). On those systems, sqlite3_mutex_try()\n** will always return SQLITE_BUSY. In most cases the SQLite core only uses\n** sqlite3_mutex_try() as an optimization, so this is acceptable\n** behavior. The exceptions are unix builds that set the\n** SQLITE_ENABLE_SETLK_TIMEOUT build option. In that case a working\n** sqlite3_mutex_try() is required.)^\n**\n** ^The sqlite3_mutex_leave() routine exits a mutex that was\n** previously entered by the same thread. The behavior\n** is undefined if the mutex is not currently entered by the\n** calling thread or is not currently allocated.\n**\n** ^If the argument to sqlite3_mutex_enter(), sqlite3_mutex_try(),\n** sqlite3_mutex_leave(), or sqlite3_mutex_free() is a NULL pointer,\n** then any of the four routines behaves as a no-op.\n**\n** See also: [sqlite3_mutex_held()] and [sqlite3_mutex_notheld()].\n*/\nSQLITE_API sqlite3_mutex *sqlite3_mutex_alloc(int);\nSQLITE_API void sqlite3_mutex_free(sqlite3_mutex*);\nSQLITE_API void sqlite3_mutex_enter(sqlite3_mutex*);\nSQLITE_API int sqlite3_mutex_try(sqlite3_mutex*);\nSQLITE_API void sqlite3_mutex_leave(sqlite3_mutex*);\n\n/*\n** CAPI3REF: Mutex Methods Object\n**\n** An instance of this structure defines the low-level routines\n** used to allocate and use mutexes.\n**\n** Usually, the default mutex implementations provided by SQLite are\n** sufficient, however the application has the option of substituting a custom\n** implementation for specialized deployments or systems for which SQLite\n** does not provide a suitable implementation. In this case, the application\n** creates and populates an instance of this structure to pass\n** to sqlite3_config() along with the [SQLITE_CONFIG_MUTEX] option.\n** Additionally, an instance of this structure can be used as an\n** output variable when querying the system for the current mutex\n** implementation, using the [SQLITE_CONFIG_GETMUTEX] option.\n**\n** ^The xMutexInit method defined by this structure is invoked as\n** part of system initialization by the sqlite3_initialize() function.\n** ^The xMutexInit routine is called by SQLite exactly once for each\n** effective call to [sqlite3_initialize()].\n**\n** ^The xMutexEnd method defined by this structure is invoked as\n** part of system shutdown by the sqlite3_shutdown() function. The\n** implementation of this method is expected to release all outstanding\n** resources obtained by the mutex methods implementation, especially\n** those obtained by the xMutexInit method. ^The xMutexEnd()\n** interface is invoked exactly once for each call to [sqlite3_shutdown()].\n**\n** ^(The remaining seven methods defined by this structure (xMutexAlloc,\n** xMutexFree, xMutexEnter, xMutexTry, xMutexLeave, xMutexHeld and\n** xMutexNotheld) implement the following interfaces (respectively):\n**\n**
    \n**
  • [sqlite3_mutex_alloc()]
    • \n**
  • [sqlite3_mutex_free()]
    • \n**
  • [sqlite3_mutex_enter()]
    • \n**
  • [sqlite3_mutex_try()]
    • \n**
  • [sqlite3_mutex_leave()]
    • \n**
  • [sqlite3_mutex_held()]
    • \n**
  • [sqlite3_mutex_notheld()]
    • \n**
)^\n**\n** The only difference is that the public sqlite3_XXX functions enumerated\n** above silently ignore any invocations that pass a NULL pointer instead\n** of a valid mutex handle. The implementations of the methods defined\n** by this structure are not required to handle this case. The results\n** of passing a NULL pointer instead of a valid mutex handle are undefined\n** (i.e. it is acceptable to provide an implementation that segfaults if\n** it is passed a NULL pointer).\n**\n** The xMutexInit() method must be threadsafe. It must be harmless to\n** invoke xMutexInit() multiple times within the same process and without\n** intervening calls to xMutexEnd(). Second and subsequent calls to\n** xMutexInit() must be no-ops.\n**\n** xMutexInit() must not use SQLite memory allocation ([sqlite3_malloc()]\n** and its associates). Similarly, xMutexAlloc() must not use SQLite memory\n** allocation for a static mutex. ^However xMutexAlloc() may use SQLite\n** memory allocation for a fast or recursive mutex.\n**\n** ^SQLite will invoke the xMutexEnd() method when [sqlite3_shutdown()] is\n** called, but only if the prior call to xMutexInit returned SQLITE_OK.\n** If xMutexInit fails in any way, it is expected to clean up after itself\n** prior to returning.\n*/\ntypedef struct sqlite3_mutex_methods sqlite3_mutex_methods;\nstruct sqlite3_mutex_methods {\n int (*xMutexInit)(void);\n int (*xMutexEnd)(void);\n sqlite3_mutex *(*xMutexAlloc)(int);\n void (*xMutexFree)(sqlite3_mutex *);\n void (*xMutexEnter)(sqlite3_mutex *);\n int (*xMutexTry)(sqlite3_mutex *);\n void (*xMutexLeave)(sqlite3_mutex *);\n int (*xMutexHeld)(sqlite3_mutex *);\n int (*xMutexNotheld)(sqlite3_mutex *);\n};\n\n/*\n** CAPI3REF: Mutex Verification Routines\n**\n** The sqlite3_mutex_held() and sqlite3_mutex_notheld() routines\n** are intended for use inside assert() statements. The SQLite core\n** never uses these routines except inside an assert() and applications\n** are advised to follow the lead of the core. The SQLite core only\n** provides implementations for these routines when it is compiled\n** with the SQLITE_DEBUG flag. External mutex implementations\n** are only required to provide these routines if SQLITE_DEBUG is\n** defined and if NDEBUG is not defined.\n**\n** These routines should return true if the mutex in their argument\n** is held or not held, respectively, by the calling thread.\n**\n** The implementation is not required to provide versions of these\n** routines that actually work. If the implementation does not provide working\n** versions of these routines, it should at least provide stubs that always\n** return true so that one does not get spurious assertion failures.\n**\n** If the argument to sqlite3_mutex_held() is a NULL pointer then\n** the routine should return 1. This seems counter-intuitive since\n** clearly the mutex cannot be held if it does not exist. But\n** the reason the mutex does not exist is because the build is not\n** using mutexes. And we do not want the assert() containing the\n** call to sqlite3_mutex_held() to fail, so a non-zero return is\n** the appropriate thing to do. The sqlite3_mutex_notheld()\n** interface should also return 1 when given a NULL pointer.\n*/\n#ifndef NDEBUG\nSQLITE_API int sqlite3_mutex_held(sqlite3_mutex*);\nSQLITE_API int sqlite3_mutex_notheld(sqlite3_mutex*);\n#endif\n\n/*\n** CAPI3REF: Mutex Types\n**\n** The [sqlite3_mutex_alloc()] interface takes a single argument\n** which is one of these integer constants.\n**\n** The set of static mutexes may change from one SQLite release to the\n** next. Applications that override the built-in mutex logic must be\n** prepared to accommodate additional static mutexes.\n*/\n#define SQLITE_MUTEX_FAST 0\n#define SQLITE_MUTEX_RECURSIVE 1\n#define SQLITE_MUTEX_STATIC_MAIN 2\n#define SQLITE_MUTEX_STATIC_MEM 3 /* sqlite3_malloc() */\n#define SQLITE_MUTEX_STATIC_MEM2 4 /* NOT USED */\n#define SQLITE_MUTEX_STATIC_OPEN 4 /* sqlite3BtreeOpen() */\n#define SQLITE_MUTEX_STATIC_PRNG 5 /* sqlite3_randomness() */\n#define SQLITE_MUTEX_STATIC_LRU 6 /* lru page list */\n#define SQLITE_MUTEX_STATIC_LRU2 7 /* NOT USED */\n#define SQLITE_MUTEX_STATIC_PMEM 7 /* sqlite3PageMalloc() */\n#define SQLITE_MUTEX_STATIC_APP1 8 /* For use by application */\n#define SQLITE_MUTEX_STATIC_APP2 9 /* For use by application */\n#define SQLITE_MUTEX_STATIC_APP3 10 /* For use by application */\n#define SQLITE_MUTEX_STATIC_VFS1 11 /* For use by built-in VFS */\n#define SQLITE_MUTEX_STATIC_VFS2 12 /* For use by extension VFS */\n#define SQLITE_MUTEX_STATIC_VFS3 13 /* For use by application VFS */\n\n/* Legacy compatibility: */\n#define SQLITE_MUTEX_STATIC_MASTER 2\n\n\n/*\n** CAPI3REF: Retrieve the mutex for a database connection\n** METHOD: sqlite3\n**\n** ^This interface returns a pointer to the [sqlite3_mutex] object that\n** serializes access to the [database connection] given in the argument\n** when the [threading mode] is Serialized.\n** ^If the [threading mode] is Single-thread or Multi-thread then this\n** routine returns a NULL pointer.\n*/\nSQLITE_API sqlite3_mutex *sqlite3_db_mutex(sqlite3*);\n\n/*\n** CAPI3REF: Low-Level Control Of Database Files\n** METHOD: sqlite3\n** KEYWORDS: {file control}\n**\n** ^The [sqlite3_file_control()] interface makes a direct call to the\n** xFileControl method for the [sqlite3_io_methods] object associated\n** with a particular database identified by the second argument. ^The\n** name of the database is \"main\" for the main database or \"temp\" for the\n** TEMP database, or the name that appears after the AS keyword for\n** databases that are added using the [ATTACH] SQL command.\n** ^A NULL pointer can be used in place of \"main\" to refer to the\n** main database file.\n** ^The third and fourth parameters to this routine\n** are passed directly through to the second and third parameters of\n** the xFileControl method. ^The return value of the xFileControl\n** method becomes the return value of this routine.\n**\n** A few opcodes for [sqlite3_file_control()] are handled directly\n** by the SQLite core and never invoke the\n** sqlite3_io_methods.xFileControl method.\n** ^The [SQLITE_FCNTL_FILE_POINTER] value for the op parameter causes\n** a pointer to the underlying [sqlite3_file] object to be written into\n** the space pointed to by the 4th parameter. The\n** [SQLITE_FCNTL_JOURNAL_POINTER] works similarly except that it returns\n** the [sqlite3_file] object associated with the journal file instead of\n** the main database. The [SQLITE_FCNTL_VFS_POINTER] opcode returns\n** a pointer to the underlying [sqlite3_vfs] object for the file.\n** The [SQLITE_FCNTL_DATA_VERSION] returns the data version counter\n** from the pager.\n**\n** ^If the second parameter (zDbName) does not match the name of any\n** open database file, then SQLITE_ERROR is returned. ^This error\n** code is not remembered and will not be recalled by [sqlite3_errcode()]\n** or [sqlite3_errmsg()]. The underlying xFileControl method might\n** also return SQLITE_ERROR. There is no way to distinguish between\n** an incorrect zDbName and an SQLITE_ERROR return from the underlying\n** xFileControl method.\n**\n** See also: [file control opcodes]\n*/\nSQLITE_API int sqlite3_file_control(sqlite3*, const char *zDbName, int op, void*);\n\n/*\n** CAPI3REF: Testing Interface\n**\n** ^The sqlite3_test_control() interface is used to read out internal\n** state of SQLite and to inject faults into SQLite for testing\n** purposes. ^The first parameter is an operation code that determines\n** the number, meaning, and operation of all subsequent parameters.\n**\n** This interface is not for use by applications. It exists solely\n** for verifying the correct operation of the SQLite library. Depending\n** on how the SQLite library is compiled, this interface might not exist.\n**\n** The details of the operation codes, their meanings, the parameters\n** they take, and what they do are all subject to change without notice.\n** Unlike most of the SQLite API, this function is not guaranteed to\n** operate consistently from one release to the next.\n*/\nSQLITE_API int sqlite3_test_control(int op, ...);\n\n/*\n** CAPI3REF: Testing Interface Operation Codes\n**\n** These constants are the valid operation code parameters used\n** as the first argument to [sqlite3_test_control()].\n**\n** These parameters and their meanings are subject to change\n** without notice. These values are for testing purposes only.\n** Applications should not use any of these parameters or the\n** [sqlite3_test_control()] interface.\n*/\n#define SQLITE_TESTCTRL_FIRST 5\n#define SQLITE_TESTCTRL_PRNG_SAVE 5\n#define SQLITE_TESTCTRL_PRNG_RESTORE 6\n#define SQLITE_TESTCTRL_PRNG_RESET 7 /* NOT USED */\n#define SQLITE_TESTCTRL_FK_NO_ACTION 7\n#define SQLITE_TESTCTRL_BITVEC_TEST 8\n#define SQLITE_TESTCTRL_FAULT_INSTALL 9\n#define SQLITE_TESTCTRL_BENIGN_MALLOC_HOOKS 10\n#define SQLITE_TESTCTRL_PENDING_BYTE 11\n#define SQLITE_TESTCTRL_ASSERT 12\n#define SQLITE_TESTCTRL_ALWAYS 13\n#define SQLITE_TESTCTRL_RESERVE 14 /* NOT USED */\n#define SQLITE_TESTCTRL_JSON_SELFCHECK 14\n#define SQLITE_TESTCTRL_OPTIMIZATIONS 15\n#define SQLITE_TESTCTRL_ISKEYWORD 16 /* NOT USED */\n#define SQLITE_TESTCTRL_GETOPT 16\n#define SQLITE_TESTCTRL_SCRATCHMALLOC 17 /* NOT USED */\n#define SQLITE_TESTCTRL_INTERNAL_FUNCTIONS 17\n#define SQLITE_TESTCTRL_LOCALTIME_FAULT 18\n#define SQLITE_TESTCTRL_EXPLAIN_STMT 19 /* NOT USED */\n#define SQLITE_TESTCTRL_ONCE_RESET_THRESHOLD 19\n#define SQLITE_TESTCTRL_NEVER_CORRUPT 20\n#define SQLITE_TESTCTRL_VDBE_COVERAGE 21\n#define SQLITE_TESTCTRL_BYTEORDER 22\n#define SQLITE_TESTCTRL_ISINIT 23\n#define SQLITE_TESTCTRL_SORTER_MMAP 24\n#define SQLITE_TESTCTRL_IMPOSTER 25\n#define SQLITE_TESTCTRL_PARSER_COVERAGE 26\n#define SQLITE_TESTCTRL_RESULT_INTREAL 27\n#define SQLITE_TESTCTRL_PRNG_SEED 28\n#define SQLITE_TESTCTRL_EXTRA_SCHEMA_CHECKS 29\n#define SQLITE_TESTCTRL_SEEK_COUNT 30\n#define SQLITE_TESTCTRL_TRACEFLAGS 31\n#define SQLITE_TESTCTRL_TUNE 32\n#define SQLITE_TESTCTRL_LOGEST 33\n#define SQLITE_TESTCTRL_USELONGDOUBLE 34 /* NOT USED */\n#define SQLITE_TESTCTRL_LAST 34 /* Largest TESTCTRL */\n\n/*\n** CAPI3REF: SQL Keyword Checking\n**\n** These routines provide access to the set of SQL language keywords\n** recognized by SQLite. Applications can use these routines to determine\n** whether or not a specific identifier needs to be escaped (for example,\n** by enclosing in double-quotes) so as not to confuse the parser.\n**\n** The sqlite3_keyword_count() interface returns the number of distinct\n** keywords understood by SQLite.\n**\n** The sqlite3_keyword_name(N,Z,L) interface finds the 0-based N-th keyword and\n** makes *Z point to that keyword expressed as UTF8 and writes the number\n** of bytes in the keyword into *L. The string that *Z points to is not\n** zero-terminated. The sqlite3_keyword_name(N,Z,L) routine returns\n** SQLITE_OK if N is within bounds and SQLITE_ERROR if not. If either Z\n** or L are NULL or invalid pointers then calls to\n** sqlite3_keyword_name(N,Z,L) result in undefined behavior.\n**\n** The sqlite3_keyword_check(Z,L) interface checks to see whether or not\n** the L-byte UTF8 identifier that Z points to is a keyword, returning non-zero\n** if it is and zero if not.\n**\n** The parser used by SQLite is forgiving. It is often possible to use\n** a keyword as an identifier as long as such use does not result in a\n** parsing ambiguity. For example, the statement\n** \"CREATE TABLE BEGIN(REPLACE,PRAGMA,END);\" is accepted by SQLite, and\n** creates a new table named \"BEGIN\" with three columns named\n** \"REPLACE\", \"PRAGMA\", and \"END\". Nevertheless, best practice is to avoid\n** using keywords as identifiers. Common techniques used to avoid keyword\n** name collisions include:\n**
    \n**
  • Put all identifier names inside double-quotes. This is the official\n** SQL way to escape identifier names.\n**
  • Put identifier names inside [...]. This is not standard SQL,\n** but it is what SQL Server does and so lots of programmers use this\n** technique.\n**
  • Begin every identifier with the letter \"Z\" as no SQL keywords start\n** with \"Z\".\n**
  • Include a digit somewhere in every identifier name.\n**
\n**\n** Note that the number of keywords understood by SQLite can depend on\n** compile-time options. For example, \"VACUUM\" is not a keyword if\n** SQLite is compiled with the [-DSQLITE_OMIT_VACUUM] option. Also,\n** new keywords may be added to future releases of SQLite.\n*/\nSQLITE_API int sqlite3_keyword_count(void);\nSQLITE_API int sqlite3_keyword_name(int,const char**,int*);\nSQLITE_API int sqlite3_keyword_check(const char*,int);\n\n/*\n** CAPI3REF: Dynamic String Object\n** KEYWORDS: {dynamic string}\n**\n** An instance of the sqlite3_str object contains a dynamically-sized\n** string under construction.\n**\n** The lifecycle of an sqlite3_str object is as follows:\n**
    \n**
  1. ^The sqlite3_str object is created using [sqlite3_str_new()].\n**
  2. ^Text is appended to the sqlite3_str object using various\n** methods, such as [sqlite3_str_appendf()].\n**
  3. ^The sqlite3_str object is destroyed and the string it created\n** is returned using the [sqlite3_str_finish()] interface.\n**
\n*/\ntypedef struct sqlite3_str sqlite3_str;\n\n/*\n** CAPI3REF: Create A New Dynamic String Object\n** CONSTRUCTOR: sqlite3_str\n**\n** ^The [sqlite3_str_new(D)] interface allocates and initializes\n** a new [sqlite3_str] object. To avoid memory leaks, the object returned by\n** [sqlite3_str_new()] must be freed by a subsequent call to\n** [sqlite3_str_finish(X)].\n**\n** ^The [sqlite3_str_new(D)] interface always returns a pointer to a\n** valid [sqlite3_str] object, though in the event of an out-of-memory\n** error the returned object might be a special singleton that will\n** silently reject new text, always return SQLITE_NOMEM from\n** [sqlite3_str_errcode()], always return 0 for\n** [sqlite3_str_length()], and always return NULL from\n** [sqlite3_str_finish(X)]. It is always safe to use the value\n** returned by [sqlite3_str_new(D)] as the sqlite3_str parameter\n** to any of the other [sqlite3_str] methods.\n**\n** The D parameter to [sqlite3_str_new(D)] may be NULL. If the\n** D parameter in [sqlite3_str_new(D)] is not NULL, then the maximum\n** length of the string contained in the [sqlite3_str] object will be\n** the value set for [sqlite3_limit](D,[SQLITE_LIMIT_LENGTH]) instead\n** of [SQLITE_MAX_LENGTH].\n*/\nSQLITE_API sqlite3_str *sqlite3_str_new(sqlite3*);\n\n/*\n** CAPI3REF: Finalize A Dynamic String\n** DESTRUCTOR: sqlite3_str\n**\n** ^The [sqlite3_str_finish(X)] interface destroys the sqlite3_str object X\n** and returns a pointer to a memory buffer obtained from [sqlite3_malloc64()]\n** that contains the constructed string. The calling application should\n** pass the returned value to [sqlite3_free()] to avoid a memory leak.\n** ^The [sqlite3_str_finish(X)] interface may return a NULL pointer if any\n** errors were encountered during construction of the string. ^The\n** [sqlite3_str_finish(X)] interface will also return a NULL pointer if the\n** string in [sqlite3_str] object X is zero bytes long.\n*/\nSQLITE_API char *sqlite3_str_finish(sqlite3_str*);\n\n/*\n** CAPI3REF: Add Content To A Dynamic String\n** METHOD: sqlite3_str\n**\n** These interfaces add content to an sqlite3_str object previously obtained\n** from [sqlite3_str_new()].\n**\n** ^The [sqlite3_str_appendf(X,F,...)] and\n** [sqlite3_str_vappendf(X,F,V)] interfaces uses the [built-in printf]\n** functionality of SQLite to append formatted text onto the end of\n** [sqlite3_str] object X.\n**\n** ^The [sqlite3_str_append(X,S,N)] method appends exactly N bytes from string S\n** onto the end of the [sqlite3_str] object X. N must be non-negative.\n** S must contain at least N non-zero bytes of content. To append a\n** zero-terminated string in its entirety, use the [sqlite3_str_appendall()]\n** method instead.\n**\n** ^The [sqlite3_str_appendall(X,S)] method appends the complete content of\n** zero-terminated string S onto the end of [sqlite3_str] object X.\n**\n** ^The [sqlite3_str_appendchar(X,N,C)] method appends N copies of the\n** single-byte character C onto the end of [sqlite3_str] object X.\n** ^This method can be used, for example, to add whitespace indentation.\n**\n** ^The [sqlite3_str_reset(X)] method resets the string under construction\n** inside [sqlite3_str] object X back to zero bytes in length.\n**\n** These methods do not return a result code. ^If an error occurs, that fact\n** is recorded in the [sqlite3_str] object and can be recovered by a\n** subsequent call to [sqlite3_str_errcode(X)].\n*/\nSQLITE_API void sqlite3_str_appendf(sqlite3_str*, const char *zFormat, ...);\nSQLITE_API void sqlite3_str_vappendf(sqlite3_str*, const char *zFormat, va_list);\nSQLITE_API void sqlite3_str_append(sqlite3_str*, const char *zIn, int N);\nSQLITE_API void sqlite3_str_appendall(sqlite3_str*, const char *zIn);\nSQLITE_API void sqlite3_str_appendchar(sqlite3_str*, int N, char C);\nSQLITE_API void sqlite3_str_reset(sqlite3_str*);\n\n/*\n** CAPI3REF: Status Of A Dynamic String\n** METHOD: sqlite3_str\n**\n** These interfaces return the current status of an [sqlite3_str] object.\n**\n** ^If any prior errors have occurred while constructing the dynamic string\n** in sqlite3_str X, then the [sqlite3_str_errcode(X)] method will return\n** an appropriate error code. ^The [sqlite3_str_errcode(X)] method returns\n** [SQLITE_NOMEM] following any out-of-memory error, or\n** [SQLITE_TOOBIG] if the size of the dynamic string exceeds\n** [SQLITE_MAX_LENGTH], or [SQLITE_OK] if there have been no errors.\n**\n** ^The [sqlite3_str_length(X)] method returns the current length, in bytes,\n** of the dynamic string under construction in [sqlite3_str] object X.\n** ^The length returned by [sqlite3_str_length(X)] does not include the\n** zero-termination byte.\n**\n** ^The [sqlite3_str_value(X)] method returns a pointer to the current\n** content of the dynamic string under construction in X. The value\n** returned by [sqlite3_str_value(X)] is managed by the sqlite3_str object X\n** and might be freed or altered by any subsequent method on the same\n** [sqlite3_str] object. Applications must not use the pointer returned by\n** [sqlite3_str_value(X)] after any subsequent method call on the same\n** object. ^Applications may change the content of the string returned\n** by [sqlite3_str_value(X)] as long as they do not write into any bytes\n** outside the range of 0 to [sqlite3_str_length(X)] and do not read or\n** write any byte after any subsequent sqlite3_str method call.\n*/\nSQLITE_API int sqlite3_str_errcode(sqlite3_str*);\nSQLITE_API int sqlite3_str_length(sqlite3_str*);\nSQLITE_API char *sqlite3_str_value(sqlite3_str*);\n\n/*\n** CAPI3REF: SQLite Runtime Status\n**\n** ^These interfaces are used to retrieve runtime status information\n** about the performance of SQLite, and optionally to reset various\n** highwater marks. ^The first argument is an integer code for\n** the specific parameter to measure. ^(Recognized integer codes\n** are of the form [status parameters | SQLITE_STATUS_...].)^\n** ^The current value of the parameter is returned into *pCurrent.\n** ^The highest recorded value is returned in *pHighwater. ^If the\n** resetFlag is true, then the highest record value is reset after\n** *pHighwater is written. ^(Some parameters do not record the highest\n** value. For those parameters\n** nothing is written into *pHighwater and the resetFlag is ignored.)^\n** ^(Other parameters record only the highwater mark and not the current\n** value. For these latter parameters nothing is written into *pCurrent.)^\n**\n** ^The sqlite3_status() and sqlite3_status64() routines return\n** SQLITE_OK on success and a non-zero [error code] on failure.\n**\n** If either the current value or the highwater mark is too large to\n** be represented by a 32-bit integer, then the values returned by\n** sqlite3_status() are undefined.\n**\n** See also: [sqlite3_db_status()]\n*/\nSQLITE_API int sqlite3_status(int op, int *pCurrent, int *pHighwater, int resetFlag);\nSQLITE_API int sqlite3_status64(\n int op,\n sqlite3_int64 *pCurrent,\n sqlite3_int64 *pHighwater,\n int resetFlag\n);\n\n\n/*\n** CAPI3REF: Status Parameters\n** KEYWORDS: {status parameters}\n**\n** These integer constants designate various run-time status parameters\n** that can be returned by [sqlite3_status()].\n**\n**
\n** [[SQLITE_STATUS_MEMORY_USED]] ^(
SQLITE_STATUS_MEMORY_USED
\n**
This parameter is the current amount of memory checked out\n** using [sqlite3_malloc()], either directly or indirectly. The\n** figure includes calls made to [sqlite3_malloc()] by the application\n** and internal memory usage by the SQLite library. Auxiliary page-cache\n** memory controlled by [SQLITE_CONFIG_PAGECACHE] is not included in\n** this parameter. The amount returned is the sum of the allocation\n** sizes as reported by the xSize method in [sqlite3_mem_methods].
)^\n**\n** [[SQLITE_STATUS_MALLOC_SIZE]] ^(
SQLITE_STATUS_MALLOC_SIZE
\n**
This parameter records the largest memory allocation request\n** handed to [sqlite3_malloc()] or [sqlite3_realloc()] (or their\n** internal equivalents). Only the value returned in the\n** *pHighwater parameter to [sqlite3_status()] is of interest.\n** The value written into the *pCurrent parameter is undefined.
)^\n**\n** [[SQLITE_STATUS_MALLOC_COUNT]] ^(
SQLITE_STATUS_MALLOC_COUNT
\n**
This parameter records the number of separate memory allocations\n** currently checked out.
)^\n**\n** [[SQLITE_STATUS_PAGECACHE_USED]] ^(
SQLITE_STATUS_PAGECACHE_USED
\n**
This parameter returns the number of pages used out of the\n** [pagecache memory allocator] that was configured using\n** [SQLITE_CONFIG_PAGECACHE]. The\n** value returned is in pages, not in bytes.
)^\n**\n** [[SQLITE_STATUS_PAGECACHE_OVERFLOW]]\n** ^(
SQLITE_STATUS_PAGECACHE_OVERFLOW
\n**
This parameter returns the number of bytes of page cache\n** allocation which could not be satisfied by the [SQLITE_CONFIG_PAGECACHE]\n** buffer and where forced to overflow to [sqlite3_malloc()]. The\n** returned value includes allocations that overflowed because they\n** were too large (they were larger than the \"sz\" parameter to\n** [SQLITE_CONFIG_PAGECACHE]) and allocations that overflowed because\n** no space was left in the page cache.
)^\n**\n** [[SQLITE_STATUS_PAGECACHE_SIZE]] ^(
SQLITE_STATUS_PAGECACHE_SIZE
\n**
This parameter records the largest memory allocation request\n** handed to the [pagecache memory allocator]. Only the value returned in the\n** *pHighwater parameter to [sqlite3_status()] is of interest.\n** The value written into the *pCurrent parameter is undefined.
)^\n**\n** [[SQLITE_STATUS_SCRATCH_USED]]
SQLITE_STATUS_SCRATCH_USED
\n**
No longer used.
\n**\n** [[SQLITE_STATUS_SCRATCH_OVERFLOW]] ^(
SQLITE_STATUS_SCRATCH_OVERFLOW
\n**
No longer used.
\n**\n** [[SQLITE_STATUS_SCRATCH_SIZE]]
SQLITE_STATUS_SCRATCH_SIZE
\n**
No longer used.
\n**\n** [[SQLITE_STATUS_PARSER_STACK]] ^(
SQLITE_STATUS_PARSER_STACK
\n**
The *pHighwater parameter records the deepest parser stack.\n** The *pCurrent value is undefined. The *pHighwater value is only\n** meaningful if SQLite is compiled with [YYTRACKMAXSTACKDEPTH].
)^\n**
\n**\n** New status parameters may be added from time to time.\n*/\n#define SQLITE_STATUS_MEMORY_USED 0\n#define SQLITE_STATUS_PAGECACHE_USED 1\n#define SQLITE_STATUS_PAGECACHE_OVERFLOW 2\n#define SQLITE_STATUS_SCRATCH_USED 3 /* NOT USED */\n#define SQLITE_STATUS_SCRATCH_OVERFLOW 4 /* NOT USED */\n#define SQLITE_STATUS_MALLOC_SIZE 5\n#define SQLITE_STATUS_PARSER_STACK 6\n#define SQLITE_STATUS_PAGECACHE_SIZE 7\n#define SQLITE_STATUS_SCRATCH_SIZE 8 /* NOT USED */\n#define SQLITE_STATUS_MALLOC_COUNT 9\n\n/*\n** CAPI3REF: Database Connection Status\n** METHOD: sqlite3\n**\n** ^This interface is used to retrieve runtime status information\n** about a single [database connection]. ^The first argument is the\n** database connection object to be interrogated. ^The second argument\n** is an integer constant, taken from the set of\n** [SQLITE_DBSTATUS options], that\n** determines the parameter to interrogate. The set of\n** [SQLITE_DBSTATUS options] is likely\n** to grow in future releases of SQLite.\n**\n** ^The current value of the requested parameter is written into *pCur\n** and the highest instantaneous value is written into *pHiwtr. ^If\n** the resetFlg is true, then the highest instantaneous value is\n** reset back down to the current value.\n**\n** ^The sqlite3_db_status() routine returns SQLITE_OK on success and a\n** non-zero [error code] on failure.\n**\n** See also: [sqlite3_status()] and [sqlite3_stmt_status()].\n*/\nSQLITE_API int sqlite3_db_status(sqlite3*, int op, int *pCur, int *pHiwtr, int resetFlg);\n\n/*\n** CAPI3REF: Status Parameters for database connections\n** KEYWORDS: {SQLITE_DBSTATUS options}\n**\n** These constants are the available integer \"verbs\" that can be passed as\n** the second argument to the [sqlite3_db_status()] interface.\n**\n** New verbs may be added in future releases of SQLite. Existing verbs\n** might be discontinued. Applications should check the return code from\n** [sqlite3_db_status()] to make sure that the call worked.\n** The [sqlite3_db_status()] interface will return a non-zero error code\n** if a discontinued or unsupported verb is invoked.\n**\n**
\n** [[SQLITE_DBSTATUS_LOOKASIDE_USED]] ^(
SQLITE_DBSTATUS_LOOKASIDE_USED
\n**
This parameter returns the number of lookaside memory slots currently\n** checked out.
)^\n**\n** [[SQLITE_DBSTATUS_LOOKASIDE_HIT]] ^(
SQLITE_DBSTATUS_LOOKASIDE_HIT
\n**
This parameter returns the number of malloc attempts that were\n** satisfied using lookaside memory. Only the high-water value is meaningful;\n** the current value is always zero.
)^\n**\n** [[SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE]]\n** ^(
SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE
\n**
This parameter returns the number of malloc attempts that might have\n** been satisfied using lookaside memory but failed due to the amount of\n** memory requested being larger than the lookaside slot size.\n** Only the high-water value is meaningful;\n** the current value is always zero.
)^\n**\n** [[SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL]]\n** ^(
SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL
\n**
This parameter returns the number of malloc attempts that might have\n** been satisfied using lookaside memory but failed due to all lookaside\n** memory already being in use.\n** Only the high-water value is meaningful;\n** the current value is always zero.
)^\n**\n** [[SQLITE_DBSTATUS_CACHE_USED]] ^(
SQLITE_DBSTATUS_CACHE_USED
\n**
This parameter returns the approximate number of bytes of heap\n** memory used by all pager caches associated with the database connection.)^\n** ^The highwater mark associated with SQLITE_DBSTATUS_CACHE_USED is always 0.\n**
\n**\n** [[SQLITE_DBSTATUS_CACHE_USED_SHARED]]\n** ^(
SQLITE_DBSTATUS_CACHE_USED_SHARED
\n**
This parameter is similar to DBSTATUS_CACHE_USED, except that if a\n** pager cache is shared between two or more connections the bytes of heap\n** memory used by that pager cache is divided evenly between the attached\n** connections.)^ In other words, if none of the pager caches associated\n** with the database connection are shared, this request returns the same\n** value as DBSTATUS_CACHE_USED. Or, if one or more of the pager caches are\n** shared, the value returned by this call will be smaller than that returned\n** by DBSTATUS_CACHE_USED. ^The highwater mark associated with\n** SQLITE_DBSTATUS_CACHE_USED_SHARED is always 0.
\n**\n** [[SQLITE_DBSTATUS_SCHEMA_USED]] ^(
SQLITE_DBSTATUS_SCHEMA_USED
\n**
This parameter returns the approximate number of bytes of heap\n** memory used to store the schema for all databases associated\n** with the connection - main, temp, and any [ATTACH]-ed databases.)^\n** ^The full amount of memory used by the schemas is reported, even if the\n** schema memory is shared with other database connections due to\n** [shared cache mode] being enabled.\n** ^The highwater mark associated with SQLITE_DBSTATUS_SCHEMA_USED is always 0.\n**
\n**\n** [[SQLITE_DBSTATUS_STMT_USED]] ^(
SQLITE_DBSTATUS_STMT_USED
\n**
This parameter returns the approximate number of bytes of heap\n** and lookaside memory used by all prepared statements associated with\n** the database connection.)^\n** ^The highwater mark associated with SQLITE_DBSTATUS_STMT_USED is always 0.\n**
\n**\n** [[SQLITE_DBSTATUS_CACHE_HIT]] ^(
SQLITE_DBSTATUS_CACHE_HIT
\n**
This parameter returns the number of pager cache hits that have\n** occurred.)^ ^The highwater mark associated with SQLITE_DBSTATUS_CACHE_HIT\n** is always 0.\n**
\n**\n** [[SQLITE_DBSTATUS_CACHE_MISS]] ^(
SQLITE_DBSTATUS_CACHE_MISS
\n**
This parameter returns the number of pager cache misses that have\n** occurred.)^ ^The highwater mark associated with SQLITE_DBSTATUS_CACHE_MISS\n** is always 0.\n**
\n**\n** [[SQLITE_DBSTATUS_CACHE_WRITE]] ^(
SQLITE_DBSTATUS_CACHE_WRITE
\n**
This parameter returns the number of dirty cache entries that have\n** been written to disk. Specifically, the number of pages written to the\n** wal file in wal mode databases, or the number of pages written to the\n** database file in rollback mode databases. Any pages written as part of\n** transaction rollback or database recovery operations are not included.\n** If an IO or other error occurs while writing a page to disk, the effect\n** on subsequent SQLITE_DBSTATUS_CACHE_WRITE requests is undefined.)^ ^The\n** highwater mark associated with SQLITE_DBSTATUS_CACHE_WRITE is always 0.\n**
\n**\n** [[SQLITE_DBSTATUS_CACHE_SPILL]] ^(
SQLITE_DBSTATUS_CACHE_SPILL
\n**
This parameter returns the number of dirty cache entries that have\n** been written to disk in the middle of a transaction due to the page\n** cache overflowing. Transactions are more efficient if they are written\n** to disk all at once. When pages spill mid-transaction, that introduces\n** additional overhead. This parameter can be used to help identify\n** inefficiencies that can be resolved by increasing the cache size.\n**
\n**\n** [[SQLITE_DBSTATUS_DEFERRED_FKS]] ^(
SQLITE_DBSTATUS_DEFERRED_FKS
\n**
This parameter returns zero for the current value if and only if\n** all foreign key constraints (deferred or immediate) have been\n** resolved.)^ ^The highwater mark is always 0.\n**
\n**
\n*/\n#define SQLITE_DBSTATUS_LOOKASIDE_USED 0\n#define SQLITE_DBSTATUS_CACHE_USED 1\n#define SQLITE_DBSTATUS_SCHEMA_USED 2\n#define SQLITE_DBSTATUS_STMT_USED 3\n#define SQLITE_DBSTATUS_LOOKASIDE_HIT 4\n#define SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE 5\n#define SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL 6\n#define SQLITE_DBSTATUS_CACHE_HIT 7\n#define SQLITE_DBSTATUS_CACHE_MISS 8\n#define SQLITE_DBSTATUS_CACHE_WRITE 9\n#define SQLITE_DBSTATUS_DEFERRED_FKS 10\n#define SQLITE_DBSTATUS_CACHE_USED_SHARED 11\n#define SQLITE_DBSTATUS_CACHE_SPILL 12\n#define SQLITE_DBSTATUS_MAX 12 /* Largest defined DBSTATUS */\n\n\n/*\n** CAPI3REF: Prepared Statement Status\n** METHOD: sqlite3_stmt\n**\n** ^(Each prepared statement maintains various\n** [SQLITE_STMTSTATUS counters] that measure the number\n** of times it has performed specific operations.)^ These counters can\n** be used to monitor the performance characteristics of the prepared\n** statements. For example, if the number of table steps greatly exceeds\n** the number of table searches or result rows, that would tend to indicate\n** that the prepared statement is using a full table scan rather than\n** an index.\n**\n** ^(This interface is used to retrieve and reset counter values from\n** a [prepared statement]. The first argument is the prepared statement\n** object to be interrogated. The second argument\n** is an integer code for a specific [SQLITE_STMTSTATUS counter]\n** to be interrogated.)^\n** ^The current value of the requested counter is returned.\n** ^If the resetFlg is true, then the counter is reset to zero after this\n** interface call returns.\n**\n** See also: [sqlite3_status()] and [sqlite3_db_status()].\n*/\nSQLITE_API int sqlite3_stmt_status(sqlite3_stmt*, int op,int resetFlg);\n\n/*\n** CAPI3REF: Status Parameters for prepared statements\n** KEYWORDS: {SQLITE_STMTSTATUS counter} {SQLITE_STMTSTATUS counters}\n**\n** These preprocessor macros define integer codes that name counter\n** values associated with the [sqlite3_stmt_status()] interface.\n** The meanings of the various counters are as follows:\n**\n**
\n** [[SQLITE_STMTSTATUS_FULLSCAN_STEP]]
SQLITE_STMTSTATUS_FULLSCAN_STEP
\n**
^This is the number of times that SQLite has stepped forward in\n** a table as part of a full table scan. Large numbers for this counter\n** may indicate opportunities for performance improvement through\n** careful use of indices.
\n**\n** [[SQLITE_STMTSTATUS_SORT]]
SQLITE_STMTSTATUS_SORT
\n**
^This is the number of sort operations that have occurred.\n** A non-zero value in this counter may indicate an opportunity to\n** improve performance through careful use of indices.
\n**\n** [[SQLITE_STMTSTATUS_AUTOINDEX]]
SQLITE_STMTSTATUS_AUTOINDEX
\n**
^This is the number of rows inserted into transient indices that\n** were created automatically in order to help joins run faster.\n** A non-zero value in this counter may indicate an opportunity to\n** improve performance by adding permanent indices that do not\n** need to be reinitialized each time the statement is run.
\n**\n** [[SQLITE_STMTSTATUS_VM_STEP]]
SQLITE_STMTSTATUS_VM_STEP
\n**
^This is the number of virtual machine operations executed\n** by the prepared statement if that number is less than or equal\n** to 2147483647. The number of virtual machine operations can be\n** used as a proxy for the total work done by the prepared statement.\n** If the number of virtual machine operations exceeds 2147483647\n** then the value returned by this statement status code is undefined.
\n**\n** [[SQLITE_STMTSTATUS_REPREPARE]]
SQLITE_STMTSTATUS_REPREPARE
\n**
^This is the number of times that the prepare statement has been\n** automatically regenerated due to schema changes or changes to\n** [bound parameters] that might affect the query plan.
\n**\n** [[SQLITE_STMTSTATUS_RUN]]
SQLITE_STMTSTATUS_RUN
\n**
^This is the number of times that the prepared statement has\n** been run. A single \"run\" for the purposes of this counter is one\n** or more calls to [sqlite3_step()] followed by a call to [sqlite3_reset()].\n** The counter is incremented on the first [sqlite3_step()] call of each\n** cycle.
\n**\n** [[SQLITE_STMTSTATUS_FILTER_MISS]]\n** [[SQLITE_STMTSTATUS_FILTER HIT]]\n**
SQLITE_STMTSTATUS_FILTER_HIT
\n** SQLITE_STMTSTATUS_FILTER_MISS
\n**
^SQLITE_STMTSTATUS_FILTER_HIT is the number of times that a join\n** step was bypassed because a Bloom filter returned not-found. The\n** corresponding SQLITE_STMTSTATUS_FILTER_MISS value is the number of\n** times that the Bloom filter returned a find, and thus the join step\n** had to be processed as normal.
\n**\n** [[SQLITE_STMTSTATUS_MEMUSED]]
SQLITE_STMTSTATUS_MEMUSED
\n**
^This is the approximate number of bytes of heap memory\n** used to store the prepared statement. ^This value is not actually\n** a counter, and so the resetFlg parameter to sqlite3_stmt_status()\n** is ignored when the opcode is SQLITE_STMTSTATUS_MEMUSED.\n**
\n**
\n*/\n#define SQLITE_STMTSTATUS_FULLSCAN_STEP 1\n#define SQLITE_STMTSTATUS_SORT 2\n#define SQLITE_STMTSTATUS_AUTOINDEX 3\n#define SQLITE_STMTSTATUS_VM_STEP 4\n#define SQLITE_STMTSTATUS_REPREPARE 5\n#define SQLITE_STMTSTATUS_RUN 6\n#define SQLITE_STMTSTATUS_FILTER_MISS 7\n#define SQLITE_STMTSTATUS_FILTER_HIT 8\n#define SQLITE_STMTSTATUS_MEMUSED 99\n\n/*\n** CAPI3REF: Custom Page Cache Object\n**\n** The sqlite3_pcache type is opaque. It is implemented by\n** the pluggable module. The SQLite core has no knowledge of\n** its size or internal structure and never deals with the\n** sqlite3_pcache object except by holding and passing pointers\n** to the object.\n**\n** See [sqlite3_pcache_methods2] for additional information.\n*/\ntypedef struct sqlite3_pcache sqlite3_pcache;\n\n/*\n** CAPI3REF: Custom Page Cache Object\n**\n** The sqlite3_pcache_page object represents a single page in the\n** page cache. The page cache will allocate instances of this\n** object. Various methods of the page cache use pointers to instances\n** of this object as parameters or as their return value.\n**\n** See [sqlite3_pcache_methods2] for additional information.\n*/\ntypedef struct sqlite3_pcache_page sqlite3_pcache_page;\nstruct sqlite3_pcache_page {\n void *pBuf; /* The content of the page */\n void *pExtra; /* Extra information associated with the page */\n};\n\n/*\n** CAPI3REF: Application Defined Page Cache.\n** KEYWORDS: {page cache}\n**\n** ^(The [sqlite3_config]([SQLITE_CONFIG_PCACHE2], ...) interface can\n** register an alternative page cache implementation by passing in an\n** instance of the sqlite3_pcache_methods2 structure.)^\n** In many applications, most of the heap memory allocated by\n** SQLite is used for the page cache.\n** By implementing a\n** custom page cache using this API, an application can better control\n** the amount of memory consumed by SQLite, the way in which\n** that memory is allocated and released, and the policies used to\n** determine exactly which parts of a database file are cached and for\n** how long.\n**\n** The alternative page cache mechanism is an\n** extreme measure that is only needed by the most demanding applications.\n** The built-in page cache is recommended for most uses.\n**\n** ^(The contents of the sqlite3_pcache_methods2 structure are copied to an\n** internal buffer by SQLite within the call to [sqlite3_config]. Hence\n** the application may discard the parameter after the call to\n** [sqlite3_config()] returns.)^\n**\n** [[the xInit() page cache method]]\n** ^(The xInit() method is called once for each effective\n** call to [sqlite3_initialize()])^\n** (usually only once during the lifetime of the process). ^(The xInit()\n** method is passed a copy of the sqlite3_pcache_methods2.pArg value.)^\n** The intent of the xInit() method is to set up global data structures\n** required by the custom page cache implementation.\n** ^(If the xInit() method is NULL, then the\n** built-in default page cache is used instead of the application defined\n** page cache.)^\n**\n** [[the xShutdown() page cache method]]\n** ^The xShutdown() method is called by [sqlite3_shutdown()].\n** It can be used to clean up\n** any outstanding resources before process shutdown, if required.\n** ^The xShutdown() method may be NULL.\n**\n** ^SQLite automatically serializes calls to the xInit method,\n** so the xInit method need not be threadsafe. ^The\n** xShutdown method is only called from [sqlite3_shutdown()] so it does\n** not need to be threadsafe either. All other methods must be threadsafe\n** in multithreaded applications.\n**\n** ^SQLite will never invoke xInit() more than once without an intervening\n** call to xShutdown().\n**\n** [[the xCreate() page cache methods]]\n** ^SQLite invokes the xCreate() method to construct a new cache instance.\n** SQLite will typically create one cache instance for each open database file,\n** though this is not guaranteed. ^The\n** first parameter, szPage, is the size in bytes of the pages that must\n** be allocated by the cache. ^szPage will always be a power of two. ^The\n** second parameter szExtra is a number of bytes of extra storage\n** associated with each page cache entry. ^The szExtra parameter will be\n** a number less than 250. SQLite will use the\n** extra szExtra bytes on each page to store metadata about the underlying\n** database page on disk. The value passed into szExtra depends\n** on the SQLite version, the target platform, and how SQLite was compiled.\n** ^The third argument to xCreate(), bPurgeable, is true if the cache being\n** created will be used to cache database pages of a file stored on disk, or\n** false if it is used for an in-memory database. The cache implementation\n** does not have to do anything special based upon the value of bPurgeable;\n** it is purely advisory. ^On a cache where bPurgeable is false, SQLite will\n** never invoke xUnpin() except to deliberately delete a page.\n** ^In other words, calls to xUnpin() on a cache with bPurgeable set to\n** false will always have the \"discard\" flag set to true.\n** ^Hence, a cache created with bPurgeable set to false will\n** never contain any unpinned pages.\n**\n** [[the xCachesize() page cache method]]\n** ^(The xCachesize() method may be called at any time by SQLite to set the\n** suggested maximum cache-size (number of pages stored) for the cache\n** instance passed as the first argument. This is the value configured using\n** the SQLite \"[PRAGMA cache_size]\" command.)^ As with the bPurgeable\n** parameter, the implementation is not required to do anything with this\n** value; it is advisory only.\n**\n** [[the xPagecount() page cache methods]]\n** The xPagecount() method must return the number of pages currently\n** stored in the cache, both pinned and unpinned.\n**\n** [[the xFetch() page cache methods]]\n** The xFetch() method locates a page in the cache and returns a pointer to\n** an sqlite3_pcache_page object associated with that page, or a NULL pointer.\n** The pBuf element of the returned sqlite3_pcache_page object will be a\n** pointer to a buffer of szPage bytes used to store the content of a\n** single database page. The pExtra element of sqlite3_pcache_page will be\n** a pointer to the szExtra bytes of extra storage that SQLite has requested\n** for each entry in the page cache.\n**\n** The page to be fetched is determined by the key. ^The minimum key value\n** is 1. After it has been retrieved using xFetch, the page is considered\n** to be \"pinned\".\n**\n** If the requested page is already in the page cache, then the page cache\n** implementation must return a pointer to the page buffer with its content\n** intact. If the requested page is not already in the cache, then the\n** cache implementation should use the value of the createFlag\n** parameter to help it determine what action to take:\n**\n** \n**
createFlag Behavior when page is not already in cache\n**
0 Do not allocate a new page. Return NULL.\n**
1 Allocate a new page if it is easy and convenient to do so.\n** Otherwise return NULL.\n**
2 Make every effort to allocate a new page. Only return\n** NULL if allocating a new page is effectively impossible.\n**
\n**\n** ^(SQLite will normally invoke xFetch() with a createFlag of 0 or 1. SQLite\n** will only use a createFlag of 2 after a prior call with a createFlag of 1\n** failed.)^ In between the xFetch() calls, SQLite may\n** attempt to unpin one or more cache pages by spilling the content of\n** pinned pages to disk and synching the operating system disk cache.\n**\n** [[the xUnpin() page cache method]]\n** ^xUnpin() is called by SQLite with a pointer to a currently pinned page\n** as its second argument. If the third parameter, discard, is non-zero,\n** then the page must be evicted from the cache.\n** ^If the discard parameter is\n** zero, then the page may be discarded or retained at the discretion of the\n** page cache implementation. ^The page cache implementation\n** may choose to evict unpinned pages at any time.\n**\n** The cache must not perform any reference counting. A single\n** call to xUnpin() unpins the page regardless of the number of prior calls\n** to xFetch().\n**\n** [[the xRekey() page cache methods]]\n** The xRekey() method is used to change the key value associated with the\n** page passed as the second argument. If the cache\n** previously contains an entry associated with newKey, it must be\n** discarded. ^Any prior cache entry associated with newKey is guaranteed not\n** to be pinned.\n**\n** When SQLite calls the xTruncate() method, the cache must discard all\n** existing cache entries with page numbers (keys) greater than or equal\n** to the value of the iLimit parameter passed to xTruncate(). If any\n** of these pages are pinned, they become implicitly unpinned, meaning that\n** they can be safely discarded.\n**\n** [[the xDestroy() page cache method]]\n** ^The xDestroy() method is used to delete a cache allocated by xCreate().\n** All resources associated with the specified cache should be freed. ^After\n** calling the xDestroy() method, SQLite considers the [sqlite3_pcache*]\n** handle invalid, and will not use it with any other sqlite3_pcache_methods2\n** functions.\n**\n** [[the xShrink() page cache method]]\n** ^SQLite invokes the xShrink() method when it wants the page cache to\n** free up as much of heap memory as possible. The page cache implementation\n** is not obligated to free any memory, but well-behaved implementations should\n** do their best.\n*/\ntypedef struct sqlite3_pcache_methods2 sqlite3_pcache_methods2;\nstruct sqlite3_pcache_methods2 {\n int iVersion;\n void *pArg;\n int (*xInit)(void*);\n void (*xShutdown)(void*);\n sqlite3_pcache *(*xCreate)(int szPage, int szExtra, int bPurgeable);\n void (*xCachesize)(sqlite3_pcache*, int nCachesize);\n int (*xPagecount)(sqlite3_pcache*);\n sqlite3_pcache_page *(*xFetch)(sqlite3_pcache*, unsigned key, int createFlag);\n void (*xUnpin)(sqlite3_pcache*, sqlite3_pcache_page*, int discard);\n void (*xRekey)(sqlite3_pcache*, sqlite3_pcache_page*,\n unsigned oldKey, unsigned newKey);\n void (*xTruncate)(sqlite3_pcache*, unsigned iLimit);\n void (*xDestroy)(sqlite3_pcache*);\n void (*xShrink)(sqlite3_pcache*);\n};\n\n/*\n** This is the obsolete pcache_methods object that has now been replaced\n** by sqlite3_pcache_methods2. This object is not used by SQLite. It is\n** retained in the header file for backwards compatibility only.\n*/\ntypedef struct sqlite3_pcache_methods sqlite3_pcache_methods;\nstruct sqlite3_pcache_methods {\n void *pArg;\n int (*xInit)(void*);\n void (*xShutdown)(void*);\n sqlite3_pcache *(*xCreate)(int szPage, int bPurgeable);\n void (*xCachesize)(sqlite3_pcache*, int nCachesize);\n int (*xPagecount)(sqlite3_pcache*);\n void *(*xFetch)(sqlite3_pcache*, unsigned key, int createFlag);\n void (*xUnpin)(sqlite3_pcache*, void*, int discard);\n void (*xRekey)(sqlite3_pcache*, void*, unsigned oldKey, unsigned newKey);\n void (*xTruncate)(sqlite3_pcache*, unsigned iLimit);\n void (*xDestroy)(sqlite3_pcache*);\n};\n\n\n/*\n** CAPI3REF: Online Backup Object\n**\n** The sqlite3_backup object records state information about an ongoing\n** online backup operation. ^The sqlite3_backup object is created by\n** a call to [sqlite3_backup_init()] and is destroyed by a call to\n** [sqlite3_backup_finish()].\n**\n** See Also: [Using the SQLite Online Backup API]\n*/\ntypedef struct sqlite3_backup sqlite3_backup;\n\n/*\n** CAPI3REF: Online Backup API.\n**\n** The backup API copies the content of one database into another.\n** It is useful either for creating backups of databases or\n** for copying in-memory databases to or from persistent files.\n**\n** See Also: [Using the SQLite Online Backup API]\n**\n** ^SQLite holds a write transaction open on the destination database file\n** for the duration of the backup operation.\n** ^The source database is read-locked only while it is being read;\n** it is not locked continuously for the entire backup operation.\n** ^Thus, the backup may be performed on a live source database without\n** preventing other database connections from\n** reading or writing to the source database while the backup is underway.\n**\n** ^(To perform a backup operation:\n**
    \n**
  1. sqlite3_backup_init() is called once to initialize the\n** backup,\n**
  2. sqlite3_backup_step() is called one or more times to transfer\n** the data between the two databases, and finally\n**
  3. sqlite3_backup_finish() is called to release all resources\n** associated with the backup operation.\n**
)^\n** There should be exactly one call to sqlite3_backup_finish() for each\n** successful call to sqlite3_backup_init().\n**\n** [[sqlite3_backup_init()]] sqlite3_backup_init()\n**\n** ^The D and N arguments to sqlite3_backup_init(D,N,S,M) are the\n** [database connection] associated with the destination database\n** and the database name, respectively.\n** ^The database name is \"main\" for the main database, \"temp\" for the\n** temporary database, or the name specified after the AS keyword in\n** an [ATTACH] statement for an attached database.\n** ^The S and M arguments passed to\n** sqlite3_backup_init(D,N,S,M) identify the [database connection]\n** and database name of the source database, respectively.\n** ^The source and destination [database connections] (parameters S and D)\n** must be different or else sqlite3_backup_init(D,N,S,M) will fail with\n** an error.\n**\n** ^A call to sqlite3_backup_init() will fail, returning NULL, if\n** there is already a read or read-write transaction open on the\n** destination database.\n**\n** ^If an error occurs within sqlite3_backup_init(D,N,S,M), then NULL is\n** returned and an error code and error message are stored in the\n** destination [database connection] D.\n** ^The error code and message for the failed call to sqlite3_backup_init()\n** can be retrieved using the [sqlite3_errcode()], [sqlite3_errmsg()], and/or\n** [sqlite3_errmsg16()] functions.\n** ^A successful call to sqlite3_backup_init() returns a pointer to an\n** [sqlite3_backup] object.\n** ^The [sqlite3_backup] object may be used with the sqlite3_backup_step() and\n** sqlite3_backup_finish() functions to perform the specified backup\n** operation.\n**\n** [[sqlite3_backup_step()]] sqlite3_backup_step()\n**\n** ^Function sqlite3_backup_step(B,N) will copy up to N pages between\n** the source and destination databases specified by [sqlite3_backup] object B.\n** ^If N is negative, all remaining source pages are copied.\n** ^If sqlite3_backup_step(B,N) successfully copies N pages and there\n** are still more pages to be copied, then the function returns [SQLITE_OK].\n** ^If sqlite3_backup_step(B,N) successfully finishes copying all pages\n** from source to destination, then it returns [SQLITE_DONE].\n** ^If an error occurs while running sqlite3_backup_step(B,N),\n** then an [error code] is returned. ^As well as [SQLITE_OK] and\n** [SQLITE_DONE], a call to sqlite3_backup_step() may return [SQLITE_READONLY],\n** [SQLITE_NOMEM], [SQLITE_BUSY], [SQLITE_LOCKED], or an\n** [SQLITE_IOERR_ACCESS | SQLITE_IOERR_XXX] extended error code.\n**\n** ^(The sqlite3_backup_step() might return [SQLITE_READONLY] if\n**
    \n**
  1. the destination database was opened read-only, or\n**
  2. the destination database is using write-ahead-log journaling\n** and the destination and source page sizes differ, or\n**
  3. the destination database is an in-memory database and the\n** destination and source page sizes differ.\n**
)^\n**\n** ^If sqlite3_backup_step() cannot obtain a required file-system lock, then\n** the [sqlite3_busy_handler | busy-handler function]\n** is invoked (if one is specified). ^If the\n** busy-handler returns non-zero before the lock is available, then\n** [SQLITE_BUSY] is returned to the caller. ^In this case the call to\n** sqlite3_backup_step() can be retried later. ^If the source\n** [database connection]\n** is being used to write to the source database when sqlite3_backup_step()\n** is called, then [SQLITE_LOCKED] is returned immediately. ^Again, in this\n** case the call to sqlite3_backup_step() can be retried later on. ^(If\n** [SQLITE_IOERR_ACCESS | SQLITE_IOERR_XXX], [SQLITE_NOMEM], or\n** [SQLITE_READONLY] is returned, then\n** there is no point in retrying the call to sqlite3_backup_step(). These\n** errors are considered fatal.)^ The application must accept\n** that the backup operation has failed and pass the backup operation handle\n** to the sqlite3_backup_finish() to release associated resources.\n**\n** ^The first call to sqlite3_backup_step() obtains an exclusive lock\n** on the destination file. ^The exclusive lock is not released until either\n** sqlite3_backup_finish() is called or the backup operation is complete\n** and sqlite3_backup_step() returns [SQLITE_DONE]. ^Every call to\n** sqlite3_backup_step() obtains a [shared lock] on the source database that\n** lasts for the duration of the sqlite3_backup_step() call.\n** ^Because the source database is not locked between calls to\n** sqlite3_backup_step(), the source database may be modified mid-way\n** through the backup process. ^If the source database is modified by an\n** external process or via a database connection other than the one being\n** used by the backup operation, then the backup will be automatically\n** restarted by the next call to sqlite3_backup_step(). ^If the source\n** database is modified by using the same database connection as is used\n** by the backup operation, then the backup database is automatically\n** updated at the same time.\n**\n** [[sqlite3_backup_finish()]] sqlite3_backup_finish()\n**\n** When sqlite3_backup_step() has returned [SQLITE_DONE], or when the\n** application wishes to abandon the backup operation, the application\n** should destroy the [sqlite3_backup] by passing it to sqlite3_backup_finish().\n** ^The sqlite3_backup_finish() interfaces releases all\n** resources associated with the [sqlite3_backup] object.\n** ^If sqlite3_backup_step() has not yet returned [SQLITE_DONE], then any\n** active write-transaction on the destination database is rolled back.\n** The [sqlite3_backup] object is invalid\n** and may not be used following a call to sqlite3_backup_finish().\n**\n** ^The value returned by sqlite3_backup_finish is [SQLITE_OK] if no\n** sqlite3_backup_step() errors occurred, regardless of whether or not\n** sqlite3_backup_step() completed.\n** ^If an out-of-memory condition or IO error occurred during any prior\n** sqlite3_backup_step() call on the same [sqlite3_backup] object, then\n** sqlite3_backup_finish() returns the corresponding [error code].\n**\n** ^A return of [SQLITE_BUSY] or [SQLITE_LOCKED] from sqlite3_backup_step()\n** is not a permanent error and does not affect the return value of\n** sqlite3_backup_finish().\n**\n** [[sqlite3_backup_remaining()]] [[sqlite3_backup_pagecount()]]\n** sqlite3_backup_remaining() and sqlite3_backup_pagecount()\n**\n** ^The sqlite3_backup_remaining() routine returns the number of pages still\n** to be backed up at the conclusion of the most recent sqlite3_backup_step().\n** ^The sqlite3_backup_pagecount() routine returns the total number of pages\n** in the source database at the conclusion of the most recent\n** sqlite3_backup_step().\n** ^(The values returned by these functions are only updated by\n** sqlite3_backup_step(). If the source database is modified in a way that\n** changes the size of the source database or the number of pages remaining,\n** those changes are not reflected in the output of sqlite3_backup_pagecount()\n** and sqlite3_backup_remaining() until after the next\n** sqlite3_backup_step().)^\n**\n** Concurrent Usage of Database Handles\n**\n** ^The source [database connection] may be used by the application for other\n** purposes while a backup operation is underway or being initialized.\n** ^If SQLite is compiled and configured to support threadsafe database\n** connections, then the source database connection may be used concurrently\n** from within other threads.\n**\n** However, the application must guarantee that the destination\n** [database connection] is not passed to any other API (by any thread) after\n** sqlite3_backup_init() is called and before the corresponding call to\n** sqlite3_backup_finish(). SQLite does not currently check to see\n** if the application incorrectly accesses the destination [database connection]\n** and so no error code is reported, but the operations may malfunction\n** nevertheless. Use of the destination database connection while a\n** backup is in progress might also cause a mutex deadlock.\n**\n** If running in [shared cache mode], the application must\n** guarantee that the shared cache used by the destination database\n** is not accessed while the backup is running. In practice this means\n** that the application must guarantee that the disk file being\n** backed up to is not accessed by any connection within the process,\n** not just the specific connection that was passed to sqlite3_backup_init().\n**\n** The [sqlite3_backup] object itself is partially threadsafe. Multiple\n** threads may safely make multiple concurrent calls to sqlite3_backup_step().\n** However, the sqlite3_backup_remaining() and sqlite3_backup_pagecount()\n** APIs are not strictly speaking threadsafe. If they are invoked at the\n** same time as another thread is invoking sqlite3_backup_step() it is\n** possible that they return invalid values.\n**\n** Alternatives To Using The Backup API\n**\n** Other techniques for safely creating a consistent backup of an SQLite\n** database include:\n**\n**
    \n**
  • The [VACUUM INTO] command.\n**
  • The [sqlite3_rsync] utility program.\n**
\n*/\nSQLITE_API sqlite3_backup *sqlite3_backup_init(\n sqlite3 *pDest, /* Destination database handle */\n const char *zDestName, /* Destination database name */\n sqlite3 *pSource, /* Source database handle */\n const char *zSourceName /* Source database name */\n);\nSQLITE_API int sqlite3_backup_step(sqlite3_backup *p, int nPage);\nSQLITE_API int sqlite3_backup_finish(sqlite3_backup *p);\nSQLITE_API int sqlite3_backup_remaining(sqlite3_backup *p);\nSQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p);\n\n/*\n** CAPI3REF: Unlock Notification\n** METHOD: sqlite3\n**\n** ^When running in shared-cache mode, a database operation may fail with\n** an [SQLITE_LOCKED] error if the required locks on the shared-cache or\n** individual tables within the shared-cache cannot be obtained. See\n** [SQLite Shared-Cache Mode] for a description of shared-cache locking.\n** ^This API may be used to register a callback that SQLite will invoke\n** when the connection currently holding the required lock relinquishes it.\n** ^This API is only available if the library was compiled with the\n** [SQLITE_ENABLE_UNLOCK_NOTIFY] C-preprocessor symbol defined.\n**\n** See Also: [Using the SQLite Unlock Notification Feature].\n**\n** ^Shared-cache locks are released when a database connection concludes\n** its current transaction, either by committing it or rolling it back.\n**\n** ^When a connection (known as the blocked connection) fails to obtain a\n** shared-cache lock and SQLITE_LOCKED is returned to the caller, the\n** identity of the database connection (the blocking connection) that\n** has locked the required resource is stored internally. ^After an\n** application receives an SQLITE_LOCKED error, it may call the\n** sqlite3_unlock_notify() method with the blocked connection handle as\n** the first argument to register for a callback that will be invoked\n** when the blocking connection's current transaction is concluded. ^The\n** callback is invoked from within the [sqlite3_step] or [sqlite3_close]\n** call that concludes the blocking connection's transaction.\n**\n** ^(If sqlite3_unlock_notify() is called in a multi-threaded application,\n** there is a chance that the blocking connection will have already\n** concluded its transaction by the time sqlite3_unlock_notify() is invoked.\n** If this happens, then the specified callback is invoked immediately,\n** from within the call to sqlite3_unlock_notify().)^\n**\n** ^If the blocked connection is attempting to obtain a write-lock on a\n** shared-cache table, and more than one other connection currently holds\n** a read-lock on the same table, then SQLite arbitrarily selects one of\n** the other connections to use as the blocking connection.\n**\n** ^(There may be at most one unlock-notify callback registered by a\n** blocked connection. If sqlite3_unlock_notify() is called when the\n** blocked connection already has a registered unlock-notify callback,\n** then the new callback replaces the old.)^ ^If sqlite3_unlock_notify() is\n** called with a NULL pointer as its second argument, then any existing\n** unlock-notify callback is canceled. ^The blocked connection's\n** unlock-notify callback may also be canceled by closing the blocked\n** connection using [sqlite3_close()].\n**\n** The unlock-notify callback is not reentrant. If an application invokes\n** any sqlite3_xxx API functions from within an unlock-notify callback, a\n** crash or deadlock may be the result.\n**\n** ^Unless deadlock is detected (see below), sqlite3_unlock_notify() always\n** returns SQLITE_OK.\n**\n** Callback Invocation Details\n**\n** When an unlock-notify callback is registered, the application provides a\n** single void* pointer that is passed to the callback when it is invoked.\n** However, the signature of the callback function allows SQLite to pass\n** it an array of void* context pointers. The first argument passed to\n** an unlock-notify callback is a pointer to an array of void* pointers,\n** and the second is the number of entries in the array.\n**\n** When a blocking connection's transaction is concluded, there may be\n** more than one blocked connection that has registered for an unlock-notify\n** callback. ^If two or more such blocked connections have specified the\n** same callback function, then instead of invoking the callback function\n** multiple times, it is invoked once with the set of void* context pointers\n** specified by the blocked connections bundled together into an array.\n** This gives the application an opportunity to prioritize any actions\n** related to the set of unblocked database connections.\n**\n** Deadlock Detection\n**\n** Assuming that after registering for an unlock-notify callback a\n** database waits for the callback to be issued before taking any further\n** action (a reasonable assumption), then using this API may cause the\n** application to deadlock. For example, if connection X is waiting for\n** connection Y's transaction to be concluded, and similarly connection\n** Y is waiting on connection X's transaction, then neither connection\n** will proceed and the system may remain deadlocked indefinitely.\n**\n** To avoid this scenario, the sqlite3_unlock_notify() performs deadlock\n** detection. ^If a given call to sqlite3_unlock_notify() would put the\n** system in a deadlocked state, then SQLITE_LOCKED is returned and no\n** unlock-notify callback is registered. The system is said to be in\n** a deadlocked state if connection A has registered for an unlock-notify\n** callback on the conclusion of connection B's transaction, and connection\n** B has itself registered for an unlock-notify callback when connection\n** A's transaction is concluded. ^Indirect deadlock is also detected, so\n** the system is also considered to be deadlocked if connection B has\n** registered for an unlock-notify callback on the conclusion of connection\n** C's transaction, where connection C is waiting on connection A. ^Any\n** number of levels of indirection are allowed.\n**\n** The \"DROP TABLE\" Exception\n**\n** When a call to [sqlite3_step()] returns SQLITE_LOCKED, it is almost\n** always appropriate to call sqlite3_unlock_notify(). There is however,\n** one exception. When executing a \"DROP TABLE\" or \"DROP INDEX\" statement,\n** SQLite checks if there are any currently executing SELECT statements\n** that belong to the same connection. If there are, SQLITE_LOCKED is\n** returned. In this case there is no \"blocking connection\", so invoking\n** sqlite3_unlock_notify() results in the unlock-notify callback being\n** invoked immediately. If the application then re-attempts the \"DROP TABLE\"\n** or \"DROP INDEX\" query, an infinite loop might be the result.\n**\n** One way around this problem is to check the extended error code returned\n** by an sqlite3_step() call. ^(If there is a blocking connection, then the\n** extended error code is set to SQLITE_LOCKED_SHAREDCACHE. Otherwise, in\n** the special \"DROP TABLE/INDEX\" case, the extended error code is just\n** SQLITE_LOCKED.)^\n*/\nSQLITE_API int sqlite3_unlock_notify(\n sqlite3 *pBlocked, /* Waiting connection */\n void (*xNotify)(void **apArg, int nArg), /* Callback function to invoke */\n void *pNotifyArg /* Argument to pass to xNotify */\n);\n\n\n/*\n** CAPI3REF: String Comparison\n**\n** ^The [sqlite3_stricmp()] and [sqlite3_strnicmp()] APIs allow applications\n** and extensions to compare the contents of two buffers containing UTF-8\n** strings in a case-independent fashion, using the same definition of \"case\n** independence\" that SQLite uses internally when comparing identifiers.\n*/\nSQLITE_API int sqlite3_stricmp(const char *, const char *);\nSQLITE_API int sqlite3_strnicmp(const char *, const char *, int);\n\n/*\n** CAPI3REF: String Globbing\n*\n** ^The [sqlite3_strglob(P,X)] interface returns zero if and only if\n** string X matches the [GLOB] pattern P.\n** ^The definition of [GLOB] pattern matching used in\n** [sqlite3_strglob(P,X)] is the same as for the \"X GLOB P\" operator in the\n** SQL dialect understood by SQLite. ^The [sqlite3_strglob(P,X)] function\n** is case sensitive.\n**\n** Note that this routine returns zero on a match and non-zero if the strings\n** do not match, the same as [sqlite3_stricmp()] and [sqlite3_strnicmp()].\n**\n** See also: [sqlite3_strlike()].\n*/\nSQLITE_API int sqlite3_strglob(const char *zGlob, const char *zStr);\n\n/*\n** CAPI3REF: String LIKE Matching\n*\n** ^The [sqlite3_strlike(P,X,E)] interface returns zero if and only if\n** string X matches the [LIKE] pattern P with escape character E.\n** ^The definition of [LIKE] pattern matching used in\n** [sqlite3_strlike(P,X,E)] is the same as for the \"X LIKE P ESCAPE E\"\n** operator in the SQL dialect understood by SQLite. ^For \"X LIKE P\" without\n** the ESCAPE clause, set the E parameter of [sqlite3_strlike(P,X,E)] to 0.\n** ^As with the LIKE operator, the [sqlite3_strlike(P,X,E)] function is case\n** insensitive - equivalent upper and lower case ASCII characters match\n** one another.\n**\n** ^The [sqlite3_strlike(P,X,E)] function matches Unicode characters, though\n** only ASCII characters are case folded.\n**\n** Note that this routine returns zero on a match and non-zero if the strings\n** do not match, the same as [sqlite3_stricmp()] and [sqlite3_strnicmp()].\n**\n** See also: [sqlite3_strglob()].\n*/\nSQLITE_API int sqlite3_strlike(const char *zGlob, const char *zStr, unsigned int cEsc);\n\n/*\n** CAPI3REF: Error Logging Interface\n**\n** ^The [sqlite3_log()] interface writes a message into the [error log]\n** established by the [SQLITE_CONFIG_LOG] option to [sqlite3_config()].\n** ^If logging is enabled, the zFormat string and subsequent arguments are\n** used with [sqlite3_snprintf()] to generate the final output string.\n**\n** The sqlite3_log() interface is intended for use by extensions such as\n** virtual tables, collating functions, and SQL functions. While there is\n** nothing to prevent an application from calling sqlite3_log(), doing so\n** is considered bad form.\n**\n** The zFormat string must not be NULL.\n**\n** To avoid deadlocks and other threading problems, the sqlite3_log() routine\n** will not use dynamically allocated memory. The log message is stored in\n** a fixed-length buffer on the stack. If the log message is longer than\n** a few hundred characters, it will be truncated to the length of the\n** buffer.\n*/\nSQLITE_API void sqlite3_log(int iErrCode, const char *zFormat, ...);\n\n/*\n** CAPI3REF: Write-Ahead Log Commit Hook\n** METHOD: sqlite3\n**\n** ^The [sqlite3_wal_hook()] function is used to register a callback that\n** is invoked each time data is committed to a database in wal mode.\n**\n** ^(The callback is invoked by SQLite after the commit has taken place and\n** the associated write-lock on the database released)^, so the implementation\n** may read, write or [checkpoint] the database as required.\n**\n** ^The first parameter passed to the callback function when it is invoked\n** is a copy of the third parameter passed to sqlite3_wal_hook() when\n** registering the callback. ^The second is a copy of the database handle.\n** ^The third parameter is the name of the database that was written to -\n** either \"main\" or the name of an [ATTACH]-ed database. ^The fourth parameter\n** is the number of pages currently in the write-ahead log file,\n** including those that were just committed.\n**\n** The callback function should normally return [SQLITE_OK]. ^If an error\n** code is returned, that error will propagate back up through the\n** SQLite code base to cause the statement that provoked the callback\n** to report an error, though the commit will have still occurred. If the\n** callback returns [SQLITE_ROW] or [SQLITE_DONE], or if it returns a value\n** that does not correspond to any valid SQLite error code, the results\n** are undefined.\n**\n** A single database handle may have at most a single write-ahead log callback\n** registered at one time. ^Calling [sqlite3_wal_hook()] replaces any\n** previously registered write-ahead log callback. ^The return value is\n** a copy of the third parameter from the previous call, if any, or 0.\n** ^Note that the [sqlite3_wal_autocheckpoint()] interface and the\n** [wal_autocheckpoint pragma] both invoke [sqlite3_wal_hook()] and will\n** overwrite any prior [sqlite3_wal_hook()] settings.\n*/\nSQLITE_API void *sqlite3_wal_hook(\n sqlite3*,\n int(*)(void *,sqlite3*,const char*,int),\n void*\n);\n\n/*\n** CAPI3REF: Configure an auto-checkpoint\n** METHOD: sqlite3\n**\n** ^The [sqlite3_wal_autocheckpoint(D,N)] is a wrapper around\n** [sqlite3_wal_hook()] that causes any database on [database connection] D\n** to automatically [checkpoint]\n** after committing a transaction if there are N or\n** more frames in the [write-ahead log] file. ^Passing zero or\n** a negative value as the nFrame parameter disables automatic\n** checkpoints entirely.\n**\n** ^The callback registered by this function replaces any existing callback\n** registered using [sqlite3_wal_hook()]. ^Likewise, registering a callback\n** using [sqlite3_wal_hook()] disables the automatic checkpoint mechanism\n** configured by this function.\n**\n** ^The [wal_autocheckpoint pragma] can be used to invoke this interface\n** from SQL.\n**\n** ^Checkpoints initiated by this mechanism are\n** [sqlite3_wal_checkpoint_v2|PASSIVE].\n**\n** ^Every new [database connection] defaults to having the auto-checkpoint\n** enabled with a threshold of 1000 or [SQLITE_DEFAULT_WAL_AUTOCHECKPOINT]\n** pages. The use of this interface\n** is only necessary if the default setting is found to be suboptimal\n** for a particular application.\n*/\nSQLITE_API int sqlite3_wal_autocheckpoint(sqlite3 *db, int N);\n\n/*\n** CAPI3REF: Checkpoint a database\n** METHOD: sqlite3\n**\n** ^(The sqlite3_wal_checkpoint(D,X) is equivalent to\n** [sqlite3_wal_checkpoint_v2](D,X,[SQLITE_CHECKPOINT_PASSIVE],0,0).)^\n**\n** In brief, sqlite3_wal_checkpoint(D,X) causes the content in the\n** [write-ahead log] for database X on [database connection] D to be\n** transferred into the database file and for the write-ahead log to\n** be reset. See the [checkpointing] documentation for addition\n** information.\n**\n** This interface used to be the only way to cause a checkpoint to\n** occur. But then the newer and more powerful [sqlite3_wal_checkpoint_v2()]\n** interface was added. This interface is retained for backwards\n** compatibility and as a convenience for applications that need to manually\n** start a callback but which do not need the full power (and corresponding\n** complication) of [sqlite3_wal_checkpoint_v2()].\n*/\nSQLITE_API int sqlite3_wal_checkpoint(sqlite3 *db, const char *zDb);\n\n/*\n** CAPI3REF: Checkpoint a database\n** METHOD: sqlite3\n**\n** ^(The sqlite3_wal_checkpoint_v2(D,X,M,L,C) interface runs a checkpoint\n** operation on database X of [database connection] D in mode M. Status\n** information is written back into integers pointed to by L and C.)^\n** ^(The M parameter must be a valid [checkpoint mode]:)^\n**\n**
\n**
SQLITE_CHECKPOINT_PASSIVE
\n** ^Checkpoint as many frames as possible without waiting for any database\n** readers or writers to finish, then sync the database file if all frames\n** in the log were checkpointed. ^The [busy-handler callback]\n** is never invoked in the SQLITE_CHECKPOINT_PASSIVE mode.\n** ^On the other hand, passive mode might leave the checkpoint unfinished\n** if there are concurrent readers or writers.\n**\n**
SQLITE_CHECKPOINT_FULL
\n** ^This mode blocks (it invokes the\n** [sqlite3_busy_handler|busy-handler callback]) until there is no\n** database writer and all readers are reading from the most recent database\n** snapshot. ^It then checkpoints all frames in the log file and syncs the\n** database file. ^This mode blocks new database writers while it is pending,\n** but new database readers are allowed to continue unimpeded.\n**\n**
SQLITE_CHECKPOINT_RESTART
\n** ^This mode works the same way as SQLITE_CHECKPOINT_FULL with the addition\n** that after checkpointing the log file it blocks (calls the\n** [busy-handler callback])\n** until all readers are reading from the database file only. ^This ensures\n** that the next writer will restart the log file from the beginning.\n** ^Like SQLITE_CHECKPOINT_FULL, this mode blocks new\n** database writer attempts while it is pending, but does not impede readers.\n**\n**
SQLITE_CHECKPOINT_TRUNCATE
\n** ^This mode works the same way as SQLITE_CHECKPOINT_RESTART with the\n** addition that it also truncates the log file to zero bytes just prior\n** to a successful return.\n**
\n**\n** ^If pnLog is not NULL, then *pnLog is set to the total number of frames in\n** the log file or to -1 if the checkpoint could not run because\n** of an error or because the database is not in [WAL mode]. ^If pnCkpt is not\n** NULL,then *pnCkpt is set to the total number of checkpointed frames in the\n** log file (including any that were already checkpointed before the function\n** was called) or to -1 if the checkpoint could not run due to an error or\n** because the database is not in WAL mode. ^Note that upon successful\n** completion of an SQLITE_CHECKPOINT_TRUNCATE, the log file will have been\n** truncated to zero bytes and so both *pnLog and *pnCkpt will be set to zero.\n**\n** ^All calls obtain an exclusive \"checkpoint\" lock on the database file. ^If\n** any other process is running a checkpoint operation at the same time, the\n** lock cannot be obtained and SQLITE_BUSY is returned. ^Even if there is a\n** busy-handler configured, it will not be invoked in this case.\n**\n** ^The SQLITE_CHECKPOINT_FULL, RESTART and TRUNCATE modes also obtain the\n** exclusive \"writer\" lock on the database file. ^If the writer lock cannot be\n** obtained immediately, and a busy-handler is configured, it is invoked and\n** the writer lock retried until either the busy-handler returns 0 or the lock\n** is successfully obtained. ^The busy-handler is also invoked while waiting for\n** database readers as described above. ^If the busy-handler returns 0 before\n** the writer lock is obtained or while waiting for database readers, the\n** checkpoint operation proceeds from that point in the same way as\n** SQLITE_CHECKPOINT_PASSIVE - checkpointing as many frames as possible\n** without blocking any further. ^SQLITE_BUSY is returned in this case.\n**\n** ^If parameter zDb is NULL or points to a zero length string, then the\n** specified operation is attempted on all WAL databases [attached] to\n** [database connection] db. In this case the\n** values written to output parameters *pnLog and *pnCkpt are undefined. ^If\n** an SQLITE_BUSY error is encountered when processing one or more of the\n** attached WAL databases, the operation is still attempted on any remaining\n** attached databases and SQLITE_BUSY is returned at the end. ^If any other\n** error occurs while processing an attached database, processing is abandoned\n** and the error code is returned to the caller immediately. ^If no error\n** (SQLITE_BUSY or otherwise) is encountered while processing the attached\n** databases, SQLITE_OK is returned.\n**\n** ^If database zDb is the name of an attached database that is not in WAL\n** mode, SQLITE_OK is returned and both *pnLog and *pnCkpt set to -1. ^If\n** zDb is not NULL (or a zero length string) and is not the name of any\n** attached database, SQLITE_ERROR is returned to the caller.\n**\n** ^Unless it returns SQLITE_MISUSE,\n** the sqlite3_wal_checkpoint_v2() interface\n** sets the error information that is queried by\n** [sqlite3_errcode()] and [sqlite3_errmsg()].\n**\n** ^The [PRAGMA wal_checkpoint] command can be used to invoke this interface\n** from SQL.\n*/\nSQLITE_API int sqlite3_wal_checkpoint_v2(\n sqlite3 *db, /* Database handle */\n const char *zDb, /* Name of attached database (or NULL) */\n int eMode, /* SQLITE_CHECKPOINT_* value */\n int *pnLog, /* OUT: Size of WAL log in frames */\n int *pnCkpt /* OUT: Total number of frames checkpointed */\n);\n\n/*\n** CAPI3REF: Checkpoint Mode Values\n** KEYWORDS: {checkpoint mode}\n**\n** These constants define all valid values for the \"checkpoint mode\" passed\n** as the third parameter to the [sqlite3_wal_checkpoint_v2()] interface.\n** See the [sqlite3_wal_checkpoint_v2()] documentation for details on the\n** meaning of each of these checkpoint modes.\n*/\n#define SQLITE_CHECKPOINT_PASSIVE 0 /* Do as much as possible w/o blocking */\n#define SQLITE_CHECKPOINT_FULL 1 /* Wait for writers, then checkpoint */\n#define SQLITE_CHECKPOINT_RESTART 2 /* Like FULL but wait for readers */\n#define SQLITE_CHECKPOINT_TRUNCATE 3 /* Like RESTART but also truncate WAL */\n\n/*\n** CAPI3REF: Virtual Table Interface Configuration\n**\n** This function may be called by either the [xConnect] or [xCreate] method\n** of a [virtual table] implementation to configure\n** various facets of the virtual table interface.\n**\n** If this interface is invoked outside the context of an xConnect or\n** xCreate virtual table method then the behavior is undefined.\n**\n** In the call sqlite3_vtab_config(D,C,...) the D parameter is the\n** [database connection] in which the virtual table is being created and\n** which is passed in as the first argument to the [xConnect] or [xCreate]\n** method that is invoking sqlite3_vtab_config(). The C parameter is one\n** of the [virtual table configuration options]. The presence and meaning\n** of parameters after C depend on which [virtual table configuration option]\n** is used.\n*/\nSQLITE_API int sqlite3_vtab_config(sqlite3*, int op, ...);\n\n/*\n** CAPI3REF: Virtual Table Configuration Options\n** KEYWORDS: {virtual table configuration options}\n** KEYWORDS: {virtual table configuration option}\n**\n** These macros define the various options to the\n** [sqlite3_vtab_config()] interface that [virtual table] implementations\n** can use to customize and optimize their behavior.\n**\n**
\n** [[SQLITE_VTAB_CONSTRAINT_SUPPORT]]\n**
SQLITE_VTAB_CONSTRAINT_SUPPORT
\n**
Calls of the form\n** [sqlite3_vtab_config](db,SQLITE_VTAB_CONSTRAINT_SUPPORT,X) are supported,\n** where X is an integer. If X is zero, then the [virtual table] whose\n** [xCreate] or [xConnect] method invoked [sqlite3_vtab_config()] does not\n** support constraints. In this configuration (which is the default) if\n** a call to the [xUpdate] method returns [SQLITE_CONSTRAINT], then the entire\n** statement is rolled back as if [ON CONFLICT | OR ABORT] had been\n** specified as part of the user's SQL statement, regardless of the actual\n** ON CONFLICT mode specified.\n**\n** If X is non-zero, then the virtual table implementation guarantees\n** that if [xUpdate] returns [SQLITE_CONSTRAINT], it will do so before\n** any modifications to internal or persistent data structures have been made.\n** If the [ON CONFLICT] mode is ABORT, FAIL, IGNORE or ROLLBACK, SQLite\n** is able to roll back a statement or database transaction, and abandon\n** or continue processing the current SQL statement as appropriate.\n** If the ON CONFLICT mode is REPLACE and the [xUpdate] method returns\n** [SQLITE_CONSTRAINT], SQLite handles this as if the ON CONFLICT mode\n** had been ABORT.\n**\n** Virtual table implementations that are required to handle OR REPLACE\n** must do so within the [xUpdate] method. If a call to the\n** [sqlite3_vtab_on_conflict()] function indicates that the current ON\n** CONFLICT policy is REPLACE, the virtual table implementation should\n** silently replace the appropriate rows within the xUpdate callback and\n** return SQLITE_OK. Or, if this is not possible, it may return\n** SQLITE_CONSTRAINT, in which case SQLite falls back to OR ABORT\n** constraint handling.\n**
\n**\n** [[SQLITE_VTAB_DIRECTONLY]]
SQLITE_VTAB_DIRECTONLY
\n**
Calls of the form\n** [sqlite3_vtab_config](db,SQLITE_VTAB_DIRECTONLY) from within the\n** the [xConnect] or [xCreate] methods of a [virtual table] implementation\n** prohibits that virtual table from being used from within triggers and\n** views.\n**
\n**\n** [[SQLITE_VTAB_INNOCUOUS]]
SQLITE_VTAB_INNOCUOUS
\n**
Calls of the form\n** [sqlite3_vtab_config](db,SQLITE_VTAB_INNOCUOUS) from within the\n** [xConnect] or [xCreate] methods of a [virtual table] implementation\n** identify that virtual table as being safe to use from within triggers\n** and views. Conceptually, the SQLITE_VTAB_INNOCUOUS tag means that the\n** virtual table can do no serious harm even if it is controlled by a\n** malicious hacker. Developers should avoid setting the SQLITE_VTAB_INNOCUOUS\n** flag unless absolutely necessary.\n**
\n**\n** [[SQLITE_VTAB_USES_ALL_SCHEMAS]]
SQLITE_VTAB_USES_ALL_SCHEMAS
\n**
Calls of the form\n** [sqlite3_vtab_config](db,SQLITE_VTAB_USES_ALL_SCHEMA) from within the\n** the [xConnect] or [xCreate] methods of a [virtual table] implementation\n** instruct the query planner to begin at least a read transaction on\n** all schemas (\"main\", \"temp\", and any ATTACH-ed databases) whenever the\n** virtual table is used.\n**
\n**
\n*/\n#define SQLITE_VTAB_CONSTRAINT_SUPPORT 1\n#define SQLITE_VTAB_INNOCUOUS 2\n#define SQLITE_VTAB_DIRECTONLY 3\n#define SQLITE_VTAB_USES_ALL_SCHEMAS 4\n\n/*\n** CAPI3REF: Determine The Virtual Table Conflict Policy\n**\n** This function may only be called from within a call to the [xUpdate] method\n** of a [virtual table] implementation for an INSERT or UPDATE operation. ^The\n** value returned is one of [SQLITE_ROLLBACK], [SQLITE_IGNORE], [SQLITE_FAIL],\n** [SQLITE_ABORT], or [SQLITE_REPLACE], according to the [ON CONFLICT] mode\n** of the SQL statement that triggered the call to the [xUpdate] method of the\n** [virtual table].\n*/\nSQLITE_API int sqlite3_vtab_on_conflict(sqlite3 *);\n\n/*\n** CAPI3REF: Determine If Virtual Table Column Access Is For UPDATE\n**\n** If the sqlite3_vtab_nochange(X) routine is called within the [xColumn]\n** method of a [virtual table], then it might return true if the\n** column is being fetched as part of an UPDATE operation during which the\n** column value will not change. The virtual table implementation can use\n** this hint as permission to substitute a return value that is less\n** expensive to compute and that the corresponding\n** [xUpdate] method understands as a \"no-change\" value.\n**\n** If the [xColumn] method calls sqlite3_vtab_nochange() and finds that\n** the column is not changed by the UPDATE statement, then the xColumn\n** method can optionally return without setting a result, without calling\n** any of the [sqlite3_result_int|sqlite3_result_xxxxx() interfaces].\n** In that case, [sqlite3_value_nochange(X)] will return true for the\n** same column in the [xUpdate] method.\n**\n** The sqlite3_vtab_nochange() routine is an optimization. Virtual table\n** implementations should continue to give a correct answer even if the\n** sqlite3_vtab_nochange() interface were to always return false. In the\n** current implementation, the sqlite3_vtab_nochange() interface does always\n** returns false for the enhanced [UPDATE FROM] statement.\n*/\nSQLITE_API int sqlite3_vtab_nochange(sqlite3_context*);\n\n/*\n** CAPI3REF: Determine The Collation For a Virtual Table Constraint\n** METHOD: sqlite3_index_info\n**\n** This function may only be called from within a call to the [xBestIndex]\n** method of a [virtual table]. This function returns a pointer to a string\n** that is the name of the appropriate collation sequence to use for text\n** comparisons on the constraint identified by its arguments.\n**\n** The first argument must be the pointer to the [sqlite3_index_info] object\n** that is the first parameter to the xBestIndex() method. The second argument\n** must be an index into the aConstraint[] array belonging to the\n** sqlite3_index_info structure passed to xBestIndex.\n**\n** Important:\n** The first parameter must be the same pointer that is passed into the\n** xBestMethod() method. The first parameter may not be a pointer to a\n** different [sqlite3_index_info] object, even an exact copy.\n**\n** The return value is computed as follows:\n**\n**
    \n**

  1. If the constraint comes from a WHERE clause expression that contains\n** a [COLLATE operator], then the name of the collation specified by\n** that COLLATE operator is returned.\n**

  2. If there is no COLLATE operator, but the column that is the subject\n** of the constraint specifies an alternative collating sequence via\n** a [COLLATE clause] on the column definition within the CREATE TABLE\n** statement that was passed into [sqlite3_declare_vtab()], then the\n** name of that alternative collating sequence is returned.\n**

  3. Otherwise, \"BINARY\" is returned.\n**

\n*/\nSQLITE_API const char *sqlite3_vtab_collation(sqlite3_index_info*,int);\n\n/*\n** CAPI3REF: Determine if a virtual table query is DISTINCT\n** METHOD: sqlite3_index_info\n**\n** This API may only be used from within an [xBestIndex|xBestIndex method]\n** of a [virtual table] implementation. The result of calling this\n** interface from outside of xBestIndex() is undefined and probably harmful.\n**\n** ^The sqlite3_vtab_distinct() interface returns an integer between 0 and\n** 3. The integer returned by sqlite3_vtab_distinct()\n** gives the virtual table additional information about how the query\n** planner wants the output to be ordered. As long as the virtual table\n** can meet the ordering requirements of the query planner, it may set\n** the \"orderByConsumed\" flag.\n**\n**

  1. \n** ^If the sqlite3_vtab_distinct() interface returns 0, that means\n** that the query planner needs the virtual table to return all rows in the\n** sort order defined by the \"nOrderBy\" and \"aOrderBy\" fields of the\n** [sqlite3_index_info] object. This is the default expectation. If the\n** virtual table outputs all rows in sorted order, then it is always safe for\n** the xBestIndex method to set the \"orderByConsumed\" flag, regardless of\n** the return value from sqlite3_vtab_distinct().\n**

  2. \n** ^(If the sqlite3_vtab_distinct() interface returns 1, that means\n** that the query planner does not need the rows to be returned in sorted order\n** as long as all rows with the same values in all columns identified by the\n** \"aOrderBy\" field are adjacent.)^ This mode is used when the query planner\n** is doing a GROUP BY.\n**

  3. \n** ^(If the sqlite3_vtab_distinct() interface returns 2, that means\n** that the query planner does not need the rows returned in any particular\n** order, as long as rows with the same values in all columns identified\n** by \"aOrderBy\" are adjacent.)^ ^(Furthermore, when two or more rows\n** contain the same values for all columns identified by \"colUsed\", all but\n** one such row may optionally be omitted from the result.)^\n** The virtual table is not required to omit rows that are duplicates\n** over the \"colUsed\" columns, but if the virtual table can do that without\n** too much extra effort, it could potentially help the query to run faster.\n** This mode is used for a DISTINCT query.\n**

  4. \n** ^(If the sqlite3_vtab_distinct() interface returns 3, that means the\n** virtual table must return rows in the order defined by \"aOrderBy\" as\n** if the sqlite3_vtab_distinct() interface had returned 0. However if\n** two or more rows in the result have the same values for all columns\n** identified by \"colUsed\", then all but one such row may optionally be\n** omitted.)^ Like when the return value is 2, the virtual table\n** is not required to omit rows that are duplicates over the \"colUsed\"\n** columns, but if the virtual table can do that without\n** too much extra effort, it could potentially help the query to run faster.\n** This mode is used for queries\n** that have both DISTINCT and ORDER BY clauses.\n**

\n**\n**

The following table summarizes the conditions under which the\n** virtual table is allowed to set the \"orderByConsumed\" flag based on\n** the value returned by sqlite3_vtab_distinct(). This table is a\n** restatement of the previous four paragraphs:\n**\n** \n** \n**
sqlite3_vtab_distinct() return value\n** Rows are returned in aOrderBy order\n** Rows with the same value in all aOrderBy columns are adjacent\n** Duplicates over all colUsed columns may be omitted\n**
0yesyesno\n**
1noyesno\n**
2noyesyes\n**
3yesyesyes\n**
\n**\n** ^For the purposes of comparing virtual table output values to see if the\n** values are the same value for sorting purposes, two NULL values are considered\n** to be the same. In other words, the comparison operator is \"IS\"\n** (or \"IS NOT DISTINCT FROM\") and not \"==\".\n**\n** If a virtual table implementation is unable to meet the requirements\n** specified above, then it must not set the \"orderByConsumed\" flag in the\n** [sqlite3_index_info] object or an incorrect answer may result.\n**\n** ^A virtual table implementation is always free to return rows in any order\n** it wants, as long as the \"orderByConsumed\" flag is not set. ^When the\n** \"orderByConsumed\" flag is unset, the query planner will add extra\n** [bytecode] to ensure that the final results returned by the SQL query are\n** ordered correctly. The use of the \"orderByConsumed\" flag and the\n** sqlite3_vtab_distinct() interface is merely an optimization. ^Careful\n** use of the sqlite3_vtab_distinct() interface and the \"orderByConsumed\"\n** flag might help queries against a virtual table to run faster. Being\n** overly aggressive and setting the \"orderByConsumed\" flag when it is not\n** valid to do so, on the other hand, might cause SQLite to return incorrect\n** results.\n*/\nSQLITE_API int sqlite3_vtab_distinct(sqlite3_index_info*);\n\n/*\n** CAPI3REF: Identify and handle IN constraints in xBestIndex\n**\n** This interface may only be used from within an\n** [xBestIndex|xBestIndex() method] of a [virtual table] implementation.\n** The result of invoking this interface from any other context is\n** undefined and probably harmful.\n**\n** ^(A constraint on a virtual table of the form\n** \"[IN operator|column IN (...)]\" is\n** communicated to the xBestIndex method as a\n** [SQLITE_INDEX_CONSTRAINT_EQ] constraint.)^ If xBestIndex wants to use\n** this constraint, it must set the corresponding\n** aConstraintUsage[].argvIndex to a positive integer. ^(Then, under\n** the usual mode of handling IN operators, SQLite generates [bytecode]\n** that invokes the [xFilter|xFilter() method] once for each value\n** on the right-hand side of the IN operator.)^ Thus the virtual table\n** only sees a single value from the right-hand side of the IN operator\n** at a time.\n**\n** In some cases, however, it would be advantageous for the virtual\n** table to see all values on the right-hand of the IN operator all at\n** once. The sqlite3_vtab_in() interfaces facilitates this in two ways:\n**\n**

    \n**

  1. \n** ^A call to sqlite3_vtab_in(P,N,-1) will return true (non-zero)\n** if and only if the [sqlite3_index_info|P->aConstraint][N] constraint\n** is an [IN operator] that can be processed all at once. ^In other words,\n** sqlite3_vtab_in() with -1 in the third argument is a mechanism\n** by which the virtual table can ask SQLite if all-at-once processing\n** of the IN operator is even possible.\n**\n**

  2. \n** ^A call to sqlite3_vtab_in(P,N,F) with F==1 or F==0 indicates\n** to SQLite that the virtual table does or does not want to process\n** the IN operator all-at-once, respectively. ^Thus when the third\n** parameter (F) is non-negative, this interface is the mechanism by\n** which the virtual table tells SQLite how it wants to process the\n** IN operator.\n**

\n**\n** ^The sqlite3_vtab_in(P,N,F) interface can be invoked multiple times\n** within the same xBestIndex method call. ^For any given P,N pair,\n** the return value from sqlite3_vtab_in(P,N,F) will always be the same\n** within the same xBestIndex call. ^If the interface returns true\n** (non-zero), that means that the constraint is an IN operator\n** that can be processed all-at-once. ^If the constraint is not an IN\n** operator or cannot be processed all-at-once, then the interface returns\n** false.\n**\n** ^(All-at-once processing of the IN operator is selected if both of the\n** following conditions are met:\n**\n**
    \n**

  1. The P->aConstraintUsage[N].argvIndex value is set to a positive\n** integer. This is how the virtual table tells SQLite that it wants to\n** use the N-th constraint.\n**\n**

  2. The last call to sqlite3_vtab_in(P,N,F) for which F was\n** non-negative had F>=1.\n**

)^\n**\n** ^If either or both of the conditions above are false, then SQLite uses\n** the traditional one-at-a-time processing strategy for the IN constraint.\n** ^If both conditions are true, then the argvIndex-th parameter to the\n** xFilter method will be an [sqlite3_value] that appears to be NULL,\n** but which can be passed to [sqlite3_vtab_in_first()] and\n** [sqlite3_vtab_in_next()] to find all values on the right-hand side\n** of the IN constraint.\n*/\nSQLITE_API int sqlite3_vtab_in(sqlite3_index_info*, int iCons, int bHandle);\n\n/*\n** CAPI3REF: Find all elements on the right-hand side of an IN constraint.\n**\n** These interfaces are only useful from within the\n** [xFilter|xFilter() method] of a [virtual table] implementation.\n** The result of invoking these interfaces from any other context\n** is undefined and probably harmful.\n**\n** The X parameter in a call to sqlite3_vtab_in_first(X,P) or\n** sqlite3_vtab_in_next(X,P) should be one of the parameters to the\n** xFilter method which invokes these routines, and specifically\n** a parameter that was previously selected for all-at-once IN constraint\n** processing using the [sqlite3_vtab_in()] interface in the\n** [xBestIndex|xBestIndex method]. ^(If the X parameter is not\n** an xFilter argument that was selected for all-at-once IN constraint\n** processing, then these routines return [SQLITE_ERROR].)^\n**\n** ^(Use these routines to access all values on the right-hand side\n** of the IN constraint using code like the following:\n**\n** 
\n**    for(rc=sqlite3_vtab_in_first(pList, &pVal);\n**        rc==SQLITE_OK && pVal;\n**        rc=sqlite3_vtab_in_next(pList, &pVal)\n**    ){\n**      // do something with pVal\n**    }\n**    if( rc!=SQLITE_OK ){\n**      // an error has occurred\n**    }\n**
)^\n**\n** ^On success, the sqlite3_vtab_in_first(X,P) and sqlite3_vtab_in_next(X,P)\n** routines return SQLITE_OK and set *P to point to the first or next value\n** on the RHS of the IN constraint. ^If there are no more values on the\n** right hand side of the IN constraint, then *P is set to NULL and these\n** routines return [SQLITE_DONE]. ^The return value might be\n** some other value, such as SQLITE_NOMEM, in the event of a malfunction.\n**\n** The *ppOut values returned by these routines are only valid until the\n** next call to either of these routines or until the end of the xFilter\n** method from which these routines were called. If the virtual table\n** implementation needs to retain the *ppOut values for longer, it must make\n** copies. The *ppOut values are [protected sqlite3_value|protected].\n*/\nSQLITE_API int sqlite3_vtab_in_first(sqlite3_value *pVal, sqlite3_value **ppOut);\nSQLITE_API int sqlite3_vtab_in_next(sqlite3_value *pVal, sqlite3_value **ppOut);\n\n/*\n** CAPI3REF: Constraint values in xBestIndex()\n** METHOD: sqlite3_index_info\n**\n** This API may only be used from within the [xBestIndex|xBestIndex method]\n** of a [virtual table] implementation. The result of calling this interface\n** from outside of an xBestIndex method are undefined and probably harmful.\n**\n** ^When the sqlite3_vtab_rhs_value(P,J,V) interface is invoked from within\n** the [xBestIndex] method of a [virtual table] implementation, with P being\n** a copy of the [sqlite3_index_info] object pointer passed into xBestIndex and\n** J being a 0-based index into P->aConstraint[], then this routine\n** attempts to set *V to the value of the right-hand operand of\n** that constraint if the right-hand operand is known. ^If the\n** right-hand operand is not known, then *V is set to a NULL pointer.\n** ^The sqlite3_vtab_rhs_value(P,J,V) interface returns SQLITE_OK if\n** and only if *V is set to a value. ^The sqlite3_vtab_rhs_value(P,J,V)\n** inteface returns SQLITE_NOTFOUND if the right-hand side of the J-th\n** constraint is not available. ^The sqlite3_vtab_rhs_value() interface\n** can return a result code other than SQLITE_OK or SQLITE_NOTFOUND if\n** something goes wrong.\n**\n** The sqlite3_vtab_rhs_value() interface is usually only successful if\n** the right-hand operand of a constraint is a literal value in the original\n** SQL statement. If the right-hand operand is an expression or a reference\n** to some other column or a [host parameter], then sqlite3_vtab_rhs_value()\n** will probably return [SQLITE_NOTFOUND].\n**\n** ^(Some constraints, such as [SQLITE_INDEX_CONSTRAINT_ISNULL] and\n** [SQLITE_INDEX_CONSTRAINT_ISNOTNULL], have no right-hand operand. For such\n** constraints, sqlite3_vtab_rhs_value() always returns SQLITE_NOTFOUND.)^\n**\n** ^The [sqlite3_value] object returned in *V is a protected sqlite3_value\n** and remains valid for the duration of the xBestIndex method call.\n** ^When xBestIndex returns, the sqlite3_value object returned by\n** sqlite3_vtab_rhs_value() is automatically deallocated.\n**\n** The \"_rhs_\" in the name of this routine is an abbreviation for\n** \"Right-Hand Side\".\n*/\nSQLITE_API int sqlite3_vtab_rhs_value(sqlite3_index_info*, int, sqlite3_value **ppVal);\n\n/*\n** CAPI3REF: Conflict resolution modes\n** KEYWORDS: {conflict resolution mode}\n**\n** These constants are returned by [sqlite3_vtab_on_conflict()] to\n** inform a [virtual table] implementation of the [ON CONFLICT] mode\n** for the SQL statement being evaluated.\n**\n** Note that the [SQLITE_IGNORE] constant is also used as a potential\n** return value from the [sqlite3_set_authorizer()] callback and that\n** [SQLITE_ABORT] is also a [result code].\n*/\n#define SQLITE_ROLLBACK 1\n/* #define SQLITE_IGNORE 2 // Also used by sqlite3_authorizer() callback */\n#define SQLITE_FAIL 3\n/* #define SQLITE_ABORT 4 // Also an error code */\n#define SQLITE_REPLACE 5\n\n/*\n** CAPI3REF: Prepared Statement Scan Status Opcodes\n** KEYWORDS: {scanstatus options}\n**\n** The following constants can be used for the T parameter to the\n** [sqlite3_stmt_scanstatus(S,X,T,V)] interface. Each constant designates a\n** different metric for sqlite3_stmt_scanstatus() to return.\n**\n** When the value returned to V is a string, space to hold that string is\n** managed by the prepared statement S and will be automatically freed when\n** S is finalized.\n**\n** Not all values are available for all query elements. When a value is\n** not available, the output variable is set to -1 if the value is numeric,\n** or to NULL if it is a string (SQLITE_SCANSTAT_NAME).\n**\n**
\n** [[SQLITE_SCANSTAT_NLOOP]]
SQLITE_SCANSTAT_NLOOP
\n**
^The [sqlite3_int64] variable pointed to by the V parameter will be\n** set to the total number of times that the X-th loop has run.
\n**\n** [[SQLITE_SCANSTAT_NVISIT]]
SQLITE_SCANSTAT_NVISIT
\n**
^The [sqlite3_int64] variable pointed to by the V parameter will be set\n** to the total number of rows examined by all iterations of the X-th loop.
\n**\n** [[SQLITE_SCANSTAT_EST]]
SQLITE_SCANSTAT_EST
\n**
^The \"double\" variable pointed to by the V parameter will be set to the\n** query planner's estimate for the average number of rows output from each\n** iteration of the X-th loop. If the query planner's estimate was accurate,\n** then this value will approximate the quotient NVISIT/NLOOP and the\n** product of this value for all prior loops with the same SELECTID will\n** be the NLOOP value for the current loop.
\n**\n** [[SQLITE_SCANSTAT_NAME]]
SQLITE_SCANSTAT_NAME
\n**
^The \"const char *\" variable pointed to by the V parameter will be set\n** to a zero-terminated UTF-8 string containing the name of the index or table\n** used for the X-th loop.
\n**\n** [[SQLITE_SCANSTAT_EXPLAIN]]
SQLITE_SCANSTAT_EXPLAIN
\n**
^The \"const char *\" variable pointed to by the V parameter will be set\n** to a zero-terminated UTF-8 string containing the [EXPLAIN QUERY PLAN]\n** description for the X-th loop.
\n**\n** [[SQLITE_SCANSTAT_SELECTID]]
SQLITE_SCANSTAT_SELECTID
\n**
^The \"int\" variable pointed to by the V parameter will be set to the\n** id for the X-th query plan element. The id value is unique within the\n** statement. The select-id is the same value as is output in the first\n** column of an [EXPLAIN QUERY PLAN] query.
\n**\n** [[SQLITE_SCANSTAT_PARENTID]]
SQLITE_SCANSTAT_PARENTID
\n**
The \"int\" variable pointed to by the V parameter will be set to the\n** id of the parent of the current query element, if applicable, or\n** to zero if the query element has no parent. This is the same value as\n** returned in the second column of an [EXPLAIN QUERY PLAN] query.
\n**\n** [[SQLITE_SCANSTAT_NCYCLE]]
SQLITE_SCANSTAT_NCYCLE
\n**
The sqlite3_int64 output value is set to the number of cycles,\n** according to the processor time-stamp counter, that elapsed while the\n** query element was being processed. This value is not available for\n** all query elements - if it is unavailable the output variable is\n** set to -1.
\n**
\n*/\n#define SQLITE_SCANSTAT_NLOOP 0\n#define SQLITE_SCANSTAT_NVISIT 1\n#define SQLITE_SCANSTAT_EST 2\n#define SQLITE_SCANSTAT_NAME 3\n#define SQLITE_SCANSTAT_EXPLAIN 4\n#define SQLITE_SCANSTAT_SELECTID 5\n#define SQLITE_SCANSTAT_PARENTID 6\n#define SQLITE_SCANSTAT_NCYCLE 7\n\n/*\n** CAPI3REF: Prepared Statement Scan Status\n** METHOD: sqlite3_stmt\n**\n** These interfaces return information about the predicted and measured\n** performance for pStmt. Advanced applications can use this\n** interface to compare the predicted and the measured performance and\n** issue warnings and/or rerun [ANALYZE] if discrepancies are found.\n**\n** Since this interface is expected to be rarely used, it is only\n** available if SQLite is compiled using the [SQLITE_ENABLE_STMT_SCANSTATUS]\n** compile-time option.\n**\n** The \"iScanStatusOp\" parameter determines which status information to return.\n** The \"iScanStatusOp\" must be one of the [scanstatus options] or the behavior\n** of this interface is undefined. ^The requested measurement is written into\n** a variable pointed to by the \"pOut\" parameter.\n**\n** The \"flags\" parameter must be passed a mask of flags. At present only\n** one flag is defined - SQLITE_SCANSTAT_COMPLEX. If SQLITE_SCANSTAT_COMPLEX\n** is specified, then status information is available for all elements\n** of a query plan that are reported by \"EXPLAIN QUERY PLAN\" output. If\n** SQLITE_SCANSTAT_COMPLEX is not specified, then only query plan elements\n** that correspond to query loops (the \"SCAN...\" and \"SEARCH...\" elements of\n** the EXPLAIN QUERY PLAN output) are available. Invoking API\n** sqlite3_stmt_scanstatus() is equivalent to calling\n** sqlite3_stmt_scanstatus_v2() with a zeroed flags parameter.\n**\n** Parameter \"idx\" identifies the specific query element to retrieve statistics\n** for. Query elements are numbered starting from zero. A value of -1 may\n** retrieve statistics for the entire query. ^If idx is out of range\n** - less than -1 or greater than or equal to the total number of query\n** elements used to implement the statement - a non-zero value is returned and\n** the variable that pOut points to is unchanged.\n**\n** See also: [sqlite3_stmt_scanstatus_reset()]\n*/\nSQLITE_API int sqlite3_stmt_scanstatus(\n sqlite3_stmt *pStmt, /* Prepared statement for which info desired */\n int idx, /* Index of loop to report on */\n int iScanStatusOp, /* Information desired. SQLITE_SCANSTAT_* */\n void *pOut /* Result written here */\n);\nSQLITE_API int sqlite3_stmt_scanstatus_v2(\n sqlite3_stmt *pStmt, /* Prepared statement for which info desired */\n int idx, /* Index of loop to report on */\n int iScanStatusOp, /* Information desired. SQLITE_SCANSTAT_* */\n int flags, /* Mask of flags defined below */\n void *pOut /* Result written here */\n);\n\n/*\n** CAPI3REF: Prepared Statement Scan Status\n** KEYWORDS: {scan status flags}\n*/\n#define SQLITE_SCANSTAT_COMPLEX 0x0001\n\n/*\n** CAPI3REF: Zero Scan-Status Counters\n** METHOD: sqlite3_stmt\n**\n** ^Zero all [sqlite3_stmt_scanstatus()] related event counters.\n**\n** This API is only available if the library is built with pre-processor\n** symbol [SQLITE_ENABLE_STMT_SCANSTATUS] defined.\n*/\nSQLITE_API void sqlite3_stmt_scanstatus_reset(sqlite3_stmt*);\n\n/*\n** CAPI3REF: Flush caches to disk mid-transaction\n** METHOD: sqlite3\n**\n** ^If a write-transaction is open on [database connection] D when the\n** [sqlite3_db_cacheflush(D)] interface is invoked, any dirty\n** pages in the pager-cache that are not currently in use are written out\n** to disk. A dirty page may be in use if a database cursor created by an\n** active SQL statement is reading from it, or if it is page 1 of a database\n** file (page 1 is always \"in use\"). ^The [sqlite3_db_cacheflush(D)]\n** interface flushes caches for all schemas - \"main\", \"temp\", and\n** any [attached] databases.\n**\n** ^If this function needs to obtain extra database locks before dirty pages\n** can be flushed to disk, it does so. ^If those locks cannot be obtained\n** immediately and there is a busy-handler callback configured, it is invoked\n** in the usual manner. ^If the required lock still cannot be obtained, then\n** the database is skipped and an attempt made to flush any dirty pages\n** belonging to the next (if any) database. ^If any databases are skipped\n** because locks cannot be obtained, but no other error occurs, this\n** function returns SQLITE_BUSY.\n**\n** ^If any other error occurs while flushing dirty pages to disk (for\n** example an IO error or out-of-memory condition), then processing is\n** abandoned and an SQLite [error code] is returned to the caller immediately.\n**\n** ^Otherwise, if no error occurs, [sqlite3_db_cacheflush()] returns SQLITE_OK.\n**\n** ^This function does not set the database handle error code or message\n** returned by the [sqlite3_errcode()] and [sqlite3_errmsg()] functions.\n*/\nSQLITE_API int sqlite3_db_cacheflush(sqlite3*);\n\n/*\n** CAPI3REF: The pre-update hook.\n** METHOD: sqlite3\n**\n** ^These interfaces are only available if SQLite is compiled using the\n** [SQLITE_ENABLE_PREUPDATE_HOOK] compile-time option.\n**\n** ^The [sqlite3_preupdate_hook()] interface registers a callback function\n** that is invoked prior to each [INSERT], [UPDATE], and [DELETE] operation\n** on a database table.\n** ^At most one preupdate hook may be registered at a time on a single\n** [database connection]; each call to [sqlite3_preupdate_hook()] overrides\n** the previous setting.\n** ^The preupdate hook is disabled by invoking [sqlite3_preupdate_hook()]\n** with a NULL pointer as the second parameter.\n** ^The third parameter to [sqlite3_preupdate_hook()] is passed through as\n** the first parameter to callbacks.\n**\n** ^The preupdate hook only fires for changes to real database tables; the\n** preupdate hook is not invoked for changes to [virtual tables] or to\n** system tables like sqlite_sequence or sqlite_stat1.\n**\n** ^The second parameter to the preupdate callback is a pointer to\n** the [database connection] that registered the preupdate hook.\n** ^The third parameter to the preupdate callback is one of the constants\n** [SQLITE_INSERT], [SQLITE_DELETE], or [SQLITE_UPDATE] to identify the\n** kind of update operation that is about to occur.\n** ^(The fourth parameter to the preupdate callback is the name of the\n** database within the database connection that is being modified. This\n** will be \"main\" for the main database or \"temp\" for TEMP tables or\n** the name given after the AS keyword in the [ATTACH] statement for attached\n** databases.)^\n** ^The fifth parameter to the preupdate callback is the name of the\n** table that is being modified.\n**\n** For an UPDATE or DELETE operation on a [rowid table], the sixth\n** parameter passed to the preupdate callback is the initial [rowid] of the\n** row being modified or deleted. For an INSERT operation on a rowid table,\n** or any operation on a WITHOUT ROWID table, the value of the sixth\n** parameter is undefined. For an INSERT or UPDATE on a rowid table the\n** seventh parameter is the final rowid value of the row being inserted\n** or updated. The value of the seventh parameter passed to the callback\n** function is not defined for operations on WITHOUT ROWID tables, or for\n** DELETE operations on rowid tables.\n**\n** ^The sqlite3_preupdate_hook(D,C,P) function returns the P argument from\n** the previous call on the same [database connection] D, or NULL for\n** the first call on D.\n**\n** The [sqlite3_preupdate_old()], [sqlite3_preupdate_new()],\n** [sqlite3_preupdate_count()], and [sqlite3_preupdate_depth()] interfaces\n** provide additional information about a preupdate event. These routines\n** may only be called from within a preupdate callback. Invoking any of\n** these routines from outside of a preupdate callback or with a\n** [database connection] pointer that is different from the one supplied\n** to the preupdate callback results in undefined and probably undesirable\n** behavior.\n**\n** ^The [sqlite3_preupdate_count(D)] interface returns the number of columns\n** in the row that is being inserted, updated, or deleted.\n**\n** ^The [sqlite3_preupdate_old(D,N,P)] interface writes into P a pointer to\n** a [protected sqlite3_value] that contains the value of the Nth column of\n** the table row before it is updated. The N parameter must be between 0\n** and one less than the number of columns or the behavior will be\n** undefined. This must only be used within SQLITE_UPDATE and SQLITE_DELETE\n** preupdate callbacks; if it is used by an SQLITE_INSERT callback then the\n** behavior is undefined. The [sqlite3_value] that P points to\n** will be destroyed when the preupdate callback returns.\n**\n** ^The [sqlite3_preupdate_new(D,N,P)] interface writes into P a pointer to\n** a [protected sqlite3_value] that contains the value of the Nth column of\n** the table row after it is updated. The N parameter must be between 0\n** and one less than the number of columns or the behavior will be\n** undefined. This must only be used within SQLITE_INSERT and SQLITE_UPDATE\n** preupdate callbacks; if it is used by an SQLITE_DELETE callback then the\n** behavior is undefined. The [sqlite3_value] that P points to\n** will be destroyed when the preupdate callback returns.\n**\n** ^The [sqlite3_preupdate_depth(D)] interface returns 0 if the preupdate\n** callback was invoked as a result of a direct insert, update, or delete\n** operation; or 1 for inserts, updates, or deletes invoked by top-level\n** triggers; or 2 for changes resulting from triggers called by top-level\n** triggers; and so forth.\n**\n** When the [sqlite3_blob_write()] API is used to update a blob column,\n** the pre-update hook is invoked with SQLITE_DELETE, because\n** the new values are not yet available. In this case, when a\n** callback made with op==SQLITE_DELETE is actually a write using the\n** sqlite3_blob_write() API, the [sqlite3_preupdate_blobwrite()] returns\n** the index of the column being written. In other cases, where the\n** pre-update hook is being invoked for some other reason, including a\n** regular DELETE, sqlite3_preupdate_blobwrite() returns -1.\n**\n** See also: [sqlite3_update_hook()]\n*/\n#if defined(SQLITE_ENABLE_PREUPDATE_HOOK)\nSQLITE_API void *sqlite3_preupdate_hook(\n sqlite3 *db,\n void(*xPreUpdate)(\n void *pCtx, /* Copy of third arg to preupdate_hook() */\n sqlite3 *db, /* Database handle */\n int op, /* SQLITE_UPDATE, DELETE or INSERT */\n char const *zDb, /* Database name */\n char const *zName, /* Table name */\n sqlite3_int64 iKey1, /* Rowid of row about to be deleted/updated */\n sqlite3_int64 iKey2 /* New rowid value (for a rowid UPDATE) */\n ),\n void*\n);\nSQLITE_API int sqlite3_preupdate_old(sqlite3 *, int, sqlite3_value **);\nSQLITE_API int sqlite3_preupdate_count(sqlite3 *);\nSQLITE_API int sqlite3_preupdate_depth(sqlite3 *);\nSQLITE_API int sqlite3_preupdate_new(sqlite3 *, int, sqlite3_value **);\nSQLITE_API int sqlite3_preupdate_blobwrite(sqlite3 *);\n#endif\n\n/*\n** CAPI3REF: Low-level system error code\n** METHOD: sqlite3\n**\n** ^Attempt to return the underlying operating system error code or error\n** number that caused the most recent I/O error or failure to open a file.\n** The return value is OS-dependent. For example, on unix systems, after\n** [sqlite3_open_v2()] returns [SQLITE_CANTOPEN], this interface could be\n** called to get back the underlying \"errno\" that caused the problem, such\n** as ENOSPC, EAUTH, EISDIR, and so forth.\n*/\nSQLITE_API int sqlite3_system_errno(sqlite3*);\n\n/*\n** CAPI3REF: Database Snapshot\n** KEYWORDS: {snapshot} {sqlite3_snapshot}\n**\n** An instance of the snapshot object records the state of a [WAL mode]\n** database for some specific point in history.\n**\n** In [WAL mode], multiple [database connections] that are open on the\n** same database file can each be reading a different historical version\n** of the database file. When a [database connection] begins a read\n** transaction, that connection sees an unchanging copy of the database\n** as it existed for the point in time when the transaction first started.\n** Subsequent changes to the database from other connections are not seen\n** by the reader until a new read transaction is started.\n**\n** The sqlite3_snapshot object records state information about an historical\n** version of the database file so that it is possible to later open a new read\n** transaction that sees that historical version of the database rather than\n** the most recent version.\n*/\ntypedef struct sqlite3_snapshot {\n unsigned char hidden[48];\n} sqlite3_snapshot;\n\n/*\n** CAPI3REF: Record A Database Snapshot\n** CONSTRUCTOR: sqlite3_snapshot\n**\n** ^The [sqlite3_snapshot_get(D,S,P)] interface attempts to make a\n** new [sqlite3_snapshot] object that records the current state of\n** schema S in database connection D. ^On success, the\n** [sqlite3_snapshot_get(D,S,P)] interface writes a pointer to the newly\n** created [sqlite3_snapshot] object into *P and returns SQLITE_OK.\n** If there is not already a read-transaction open on schema S when\n** this function is called, one is opened automatically.\n**\n** If a read-transaction is opened by this function, then it is guaranteed\n** that the returned snapshot object may not be invalidated by a database\n** writer or checkpointer until after the read-transaction is closed. This\n** is not guaranteed if a read-transaction is already open when this\n** function is called. In that case, any subsequent write or checkpoint\n** operation on the database may invalidate the returned snapshot handle,\n** even while the read-transaction remains open.\n**\n** The following must be true for this function to succeed. If any of\n** the following statements are false when sqlite3_snapshot_get() is\n** called, SQLITE_ERROR is returned. The final value of *P is undefined\n** in this case.\n**\n**
    \n**
  • The database handle must not be in [autocommit mode].\n**\n**
  • Schema S of [database connection] D must be a [WAL mode] database.\n**\n**
  • There must not be a write transaction open on schema S of database\n** connection D.\n**\n**
  • One or more transactions must have been written to the current wal\n** file since it was created on disk (by any connection). This means\n** that a snapshot cannot be taken on a wal mode database with no wal\n** file immediately after it is first opened. At least one transaction\n** must be written to it first.\n**
\n**\n** This function may also return SQLITE_NOMEM. If it is called with the\n** database handle in autocommit mode but fails for some other reason,\n** whether or not a read transaction is opened on schema S is undefined.\n**\n** The [sqlite3_snapshot] object returned from a successful call to\n** [sqlite3_snapshot_get()] must be freed using [sqlite3_snapshot_free()]\n** to avoid a memory leak.\n**\n** The [sqlite3_snapshot_get()] interface is only available when the\n** [SQLITE_ENABLE_SNAPSHOT] compile-time option is used.\n*/\nSQLITE_API SQLITE_EXPERIMENTAL int sqlite3_snapshot_get(\n sqlite3 *db,\n const char *zSchema,\n sqlite3_snapshot **ppSnapshot\n);\n\n/*\n** CAPI3REF: Start a read transaction on an historical snapshot\n** METHOD: sqlite3_snapshot\n**\n** ^The [sqlite3_snapshot_open(D,S,P)] interface either starts a new read\n** transaction or upgrades an existing one for schema S of\n** [database connection] D such that the read transaction refers to\n** historical [snapshot] P, rather than the most recent change to the\n** database. ^The [sqlite3_snapshot_open()] interface returns SQLITE_OK\n** on success or an appropriate [error code] if it fails.\n**\n** ^In order to succeed, the database connection must not be in\n** [autocommit mode] when [sqlite3_snapshot_open(D,S,P)] is called. If there\n** is already a read transaction open on schema S, then the database handle\n** must have no active statements (SELECT statements that have been passed\n** to sqlite3_step() but not sqlite3_reset() or sqlite3_finalize()).\n** SQLITE_ERROR is returned if either of these conditions is violated, or\n** if schema S does not exist, or if the snapshot object is invalid.\n**\n** ^A call to sqlite3_snapshot_open() will fail to open if the specified\n** snapshot has been overwritten by a [checkpoint]. In this case\n** SQLITE_ERROR_SNAPSHOT is returned.\n**\n** If there is already a read transaction open when this function is\n** invoked, then the same read transaction remains open (on the same\n** database snapshot) if SQLITE_ERROR, SQLITE_BUSY or SQLITE_ERROR_SNAPSHOT\n** is returned. If another error code - for example SQLITE_PROTOCOL or an\n** SQLITE_IOERR error code - is returned, then the final state of the\n** read transaction is undefined. If SQLITE_OK is returned, then the\n** read transaction is now open on database snapshot P.\n**\n** ^(A call to [sqlite3_snapshot_open(D,S,P)] will fail if the\n** database connection D does not know that the database file for\n** schema S is in [WAL mode]. A database connection might not know\n** that the database file is in [WAL mode] if there has been no prior\n** I/O on that database connection, or if the database entered [WAL mode]\n** after the most recent I/O on the database connection.)^\n** (Hint: Run \"[PRAGMA application_id]\" against a newly opened\n** database connection in order to make it ready to use snapshots.)\n**\n** The [sqlite3_snapshot_open()] interface is only available when the\n** [SQLITE_ENABLE_SNAPSHOT] compile-time option is used.\n*/\nSQLITE_API SQLITE_EXPERIMENTAL int sqlite3_snapshot_open(\n sqlite3 *db,\n const char *zSchema,\n sqlite3_snapshot *pSnapshot\n);\n\n/*\n** CAPI3REF: Destroy a snapshot\n** DESTRUCTOR: sqlite3_snapshot\n**\n** ^The [sqlite3_snapshot_free(P)] interface destroys [sqlite3_snapshot] P.\n** The application must eventually free every [sqlite3_snapshot] object\n** using this routine to avoid a memory leak.\n**\n** The [sqlite3_snapshot_free()] interface is only available when the\n** [SQLITE_ENABLE_SNAPSHOT] compile-time option is used.\n*/\nSQLITE_API SQLITE_EXPERIMENTAL void sqlite3_snapshot_free(sqlite3_snapshot*);\n\n/*\n** CAPI3REF: Compare the ages of two snapshot handles.\n** METHOD: sqlite3_snapshot\n**\n** The sqlite3_snapshot_cmp(P1, P2) interface is used to compare the ages\n** of two valid snapshot handles.\n**\n** If the two snapshot handles are not associated with the same database\n** file, the result of the comparison is undefined.\n**\n** Additionally, the result of the comparison is only valid if both of the\n** snapshot handles were obtained by calling sqlite3_snapshot_get() since the\n** last time the wal file was deleted. The wal file is deleted when the\n** database is changed back to rollback mode or when the number of database\n** clients drops to zero. If either snapshot handle was obtained before the\n** wal file was last deleted, the value returned by this function\n** is undefined.\n**\n** Otherwise, this API returns a negative value if P1 refers to an older\n** snapshot than P2, zero if the two handles refer to the same database\n** snapshot, and a positive value if P1 is a newer snapshot than P2.\n**\n** This interface is only available if SQLite is compiled with the\n** [SQLITE_ENABLE_SNAPSHOT] option.\n*/\nSQLITE_API SQLITE_EXPERIMENTAL int sqlite3_snapshot_cmp(\n sqlite3_snapshot *p1,\n sqlite3_snapshot *p2\n);\n\n/*\n** CAPI3REF: Recover snapshots from a wal file\n** METHOD: sqlite3_snapshot\n**\n** If a [WAL file] remains on disk after all database connections close\n** (either through the use of the [SQLITE_FCNTL_PERSIST_WAL] [file control]\n** or because the last process to have the database opened exited without\n** calling [sqlite3_close()]) and a new connection is subsequently opened\n** on that database and [WAL file], the [sqlite3_snapshot_open()] interface\n** will only be able to open the last transaction added to the WAL file\n** even though the WAL file contains other valid transactions.\n**\n** This function attempts to scan the WAL file associated with database zDb\n** of database handle db and make all valid snapshots available to\n** sqlite3_snapshot_open(). It is an error if there is already a read\n** transaction open on the database, or if the database is not a WAL mode\n** database.\n**\n** SQLITE_OK is returned if successful, or an SQLite error code otherwise.\n**\n** This interface is only available if SQLite is compiled with the\n** [SQLITE_ENABLE_SNAPSHOT] option.\n*/\nSQLITE_API SQLITE_EXPERIMENTAL int sqlite3_snapshot_recover(sqlite3 *db, const char *zDb);\n\n/*\n** CAPI3REF: Serialize a database\n**\n** The sqlite3_serialize(D,S,P,F) interface returns a pointer to\n** memory that is a serialization of the S database on\n** [database connection] D. If S is a NULL pointer, the main database is used.\n** If P is not a NULL pointer, then the size of the database in bytes\n** is written into *P.\n**\n** For an ordinary on-disk database file, the serialization is just a\n** copy of the disk file. For an in-memory database or a \"TEMP\" database,\n** the serialization is the same sequence of bytes which would be written\n** to disk if that database were backed up to disk.\n**\n** The usual case is that sqlite3_serialize() copies the serialization of\n** the database into memory obtained from [sqlite3_malloc64()] and returns\n** a pointer to that memory. The caller is responsible for freeing the\n** returned value to avoid a memory leak. However, if the F argument\n** contains the SQLITE_SERIALIZE_NOCOPY bit, then no memory allocations\n** are made, and the sqlite3_serialize() function will return a pointer\n** to the contiguous memory representation of the database that SQLite\n** is currently using for that database, or NULL if no such contiguous\n** memory representation of the database exists. A contiguous memory\n** representation of the database will usually only exist if there has\n** been a prior call to [sqlite3_deserialize(D,S,...)] with the same\n** values of D and S.\n** The size of the database is written into *P even if the\n** SQLITE_SERIALIZE_NOCOPY bit is set but no contiguous copy\n** of the database exists.\n**\n** After the call, if the SQLITE_SERIALIZE_NOCOPY bit had been set,\n** the returned buffer content will remain accessible and unchanged\n** until either the next write operation on the connection or when\n** the connection is closed, and applications must not modify the\n** buffer. If the bit had been clear, the returned buffer will not\n** be accessed by SQLite after the call.\n**\n** A call to sqlite3_serialize(D,S,P,F) might return NULL even if the\n** SQLITE_SERIALIZE_NOCOPY bit is omitted from argument F if a memory\n** allocation error occurs.\n**\n** This interface is omitted if SQLite is compiled with the\n** [SQLITE_OMIT_DESERIALIZE] option.\n*/\nSQLITE_API unsigned char *sqlite3_serialize(\n sqlite3 *db, /* The database connection */\n const char *zSchema, /* Which DB to serialize. ex: \"main\", \"temp\", ... */\n sqlite3_int64 *piSize, /* Write size of the DB here, if not NULL */\n unsigned int mFlags /* Zero or more SQLITE_SERIALIZE_* flags */\n);\n\n/*\n** CAPI3REF: Flags for sqlite3_serialize\n**\n** Zero or more of the following constants can be OR-ed together for\n** the F argument to [sqlite3_serialize(D,S,P,F)].\n**\n** SQLITE_SERIALIZE_NOCOPY means that [sqlite3_serialize()] will return\n** a pointer to contiguous in-memory database that it is currently using,\n** without making a copy of the database. If SQLite is not currently using\n** a contiguous in-memory database, then this option causes\n** [sqlite3_serialize()] to return a NULL pointer. SQLite will only be\n** using a contiguous in-memory database if it has been initialized by a\n** prior call to [sqlite3_deserialize()].\n*/\n#define SQLITE_SERIALIZE_NOCOPY 0x001 /* Do no memory allocations */\n\n/*\n** CAPI3REF: Deserialize a database\n**\n** The sqlite3_deserialize(D,S,P,N,M,F) interface causes the\n** [database connection] D to disconnect from database S and then\n** reopen S as an in-memory database based on the serialization contained\n** in P. The serialized database P is N bytes in size. M is the size of\n** the buffer P, which might be larger than N. If M is larger than N, and\n** the SQLITE_DESERIALIZE_READONLY bit is not set in F, then SQLite is\n** permitted to add content to the in-memory database as long as the total\n** size does not exceed M bytes.\n**\n** If the SQLITE_DESERIALIZE_FREEONCLOSE bit is set in F, then SQLite will\n** invoke sqlite3_free() on the serialization buffer when the database\n** connection closes. If the SQLITE_DESERIALIZE_RESIZEABLE bit is set, then\n** SQLite will try to increase the buffer size using sqlite3_realloc64()\n** if writes on the database cause it to grow larger than M bytes.\n**\n** Applications must not modify the buffer P or invalidate it before\n** the database connection D is closed.\n**\n** The sqlite3_deserialize() interface will fail with SQLITE_BUSY if the\n** database is currently in a read transaction or is involved in a backup\n** operation.\n**\n** It is not possible to deserialize into the TEMP database. If the\n** S argument to sqlite3_deserialize(D,S,P,N,M,F) is \"temp\" then the\n** function returns SQLITE_ERROR.\n**\n** The deserialized database should not be in [WAL mode]. If the database\n** is in WAL mode, then any attempt to use the database file will result\n** in an [SQLITE_CANTOPEN] error. The application can set the\n** [file format version numbers] (bytes 18 and 19) of the input database P\n** to 0x01 prior to invoking sqlite3_deserialize(D,S,P,N,M,F) to force the\n** database file into rollback mode and work around this limitation.\n**\n** If sqlite3_deserialize(D,S,P,N,M,F) fails for any reason and if the\n** SQLITE_DESERIALIZE_FREEONCLOSE bit is set in argument F, then\n** [sqlite3_free()] is invoked on argument P prior to returning.\n**\n** This interface is omitted if SQLite is compiled with the\n** [SQLITE_OMIT_DESERIALIZE] option.\n*/\nSQLITE_API int sqlite3_deserialize(\n sqlite3 *db, /* The database connection */\n const char *zSchema, /* Which DB to reopen with the deserialization */\n unsigned char *pData, /* The serialized database content */\n sqlite3_int64 szDb, /* Number of bytes in the deserialization */\n sqlite3_int64 szBuf, /* Total size of buffer pData[] */\n unsigned mFlags /* Zero or more SQLITE_DESERIALIZE_* flags */\n);\n\n/*\n** CAPI3REF: Flags for sqlite3_deserialize()\n**\n** The following are allowed values for the 6th argument (the F argument) to\n** the [sqlite3_deserialize(D,S,P,N,M,F)] interface.\n**\n** The SQLITE_DESERIALIZE_FREEONCLOSE means that the database serialization\n** in the P argument is held in memory obtained from [sqlite3_malloc64()]\n** and that SQLite should take ownership of this memory and automatically\n** free it when it has finished using it. Without this flag, the caller\n** is responsible for freeing any dynamically allocated memory.\n**\n** The SQLITE_DESERIALIZE_RESIZEABLE flag means that SQLite is allowed to\n** grow the size of the database using calls to [sqlite3_realloc64()]. This\n** flag should only be used if SQLITE_DESERIALIZE_FREEONCLOSE is also used.\n** Without this flag, the deserialized database cannot increase in size beyond\n** the number of bytes specified by the M parameter.\n**\n** The SQLITE_DESERIALIZE_READONLY flag means that the deserialized database\n** should be treated as read-only.\n*/\n#define SQLITE_DESERIALIZE_FREEONCLOSE 1 /* Call sqlite3_free() on close */\n#define SQLITE_DESERIALIZE_RESIZEABLE 2 /* Resize using sqlite3_realloc64() */\n#define SQLITE_DESERIALIZE_READONLY 4 /* Database is read-only */\n\n/*\n** Undo the hack that converts floating point types to integer for\n** builds on processors without floating point support.\n*/\n#ifdef SQLITE_OMIT_FLOATING_POINT\n# undef double\n#endif\n\n#if defined(__wasi__)\n# undef SQLITE_WASI\n# define SQLITE_WASI 1\n# ifndef SQLITE_OMIT_LOAD_EXTENSION\n# define SQLITE_OMIT_LOAD_EXTENSION\n# endif\n# ifndef SQLITE_THREADSAFE\n# define SQLITE_THREADSAFE 0\n# endif\n#endif\n\n#ifdef __cplusplus\n} /* End of the 'extern \"C\"' block */\n#endif\n/* #endif for SQLITE3_H will be added by mksqlite3.tcl */\n\n/******** Begin file sqlite3rtree.h *********/\n/*\n** 2010 August 30\n**\n** The author disclaims copyright to this source code. In place of\n** a legal notice, here is a blessing:\n**\n** May you do good and not evil.\n** May you find forgiveness for yourself and forgive others.\n** May you share freely, never taking more than you give.\n**\n*************************************************************************\n*/\n\n#ifndef _SQLITE3RTREE_H_\n#define _SQLITE3RTREE_H_\n\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\ntypedef struct sqlite3_rtree_geometry sqlite3_rtree_geometry;\ntypedef struct sqlite3_rtree_query_info sqlite3_rtree_query_info;\n\n/* The double-precision datatype used by RTree depends on the\n** SQLITE_RTREE_INT_ONLY compile-time option.\n*/\n#ifdef SQLITE_RTREE_INT_ONLY\n typedef sqlite3_int64 sqlite3_rtree_dbl;\n#else\n typedef double sqlite3_rtree_dbl;\n#endif\n\n/*\n** Register a geometry callback named zGeom that can be used as part of an\n** R-Tree geometry query as follows:\n**\n** SELECT ... FROM WHERE MATCH $zGeom(... params ...)\n*/\nSQLITE_API int sqlite3_rtree_geometry_callback(\n sqlite3 *db,\n const char *zGeom,\n int (*xGeom)(sqlite3_rtree_geometry*, int, sqlite3_rtree_dbl*,int*),\n void *pContext\n);\n\n\n/*\n** A pointer to a structure of the following type is passed as the first\n** argument to callbacks registered using rtree_geometry_callback().\n*/\nstruct sqlite3_rtree_geometry {\n void *pContext; /* Copy of pContext passed to s_r_g_c() */\n int nParam; /* Size of array aParam[] */\n sqlite3_rtree_dbl *aParam; /* Parameters passed to SQL geom function */\n void *pUser; /* Callback implementation user data */\n void (*xDelUser)(void *); /* Called by SQLite to clean up pUser */\n};\n\n/*\n** Register a 2nd-generation geometry callback named zScore that can be\n** used as part of an R-Tree geometry query as follows:\n**\n** SELECT ... FROM WHERE MATCH $zQueryFunc(... params ...)\n*/\nSQLITE_API int sqlite3_rtree_query_callback(\n sqlite3 *db,\n const char *zQueryFunc,\n int (*xQueryFunc)(sqlite3_rtree_query_info*),\n void *pContext,\n void (*xDestructor)(void*)\n);\n\n\n/*\n** A pointer to a structure of the following type is passed as the\n** argument to scored geometry callback registered using\n** sqlite3_rtree_query_callback().\n**\n** Note that the first 5 fields of this structure are identical to\n** sqlite3_rtree_geometry. This structure is a subclass of\n** sqlite3_rtree_geometry.\n*/\nstruct sqlite3_rtree_query_info {\n void *pContext; /* pContext from when function registered */\n int nParam; /* Number of function parameters */\n sqlite3_rtree_dbl *aParam; /* value of function parameters */\n void *pUser; /* callback can use this, if desired */\n void (*xDelUser)(void*); /* function to free pUser */\n sqlite3_rtree_dbl *aCoord; /* Coordinates of node or entry to check */\n unsigned int *anQueue; /* Number of pending entries in the queue */\n int nCoord; /* Number of coordinates */\n int iLevel; /* Level of current node or entry */\n int mxLevel; /* The largest iLevel value in the tree */\n sqlite3_int64 iRowid; /* Rowid for current entry */\n sqlite3_rtree_dbl rParentScore; /* Score of parent node */\n int eParentWithin; /* Visibility of parent node */\n int eWithin; /* OUT: Visibility */\n sqlite3_rtree_dbl rScore; /* OUT: Write the score here */\n /* The following fields are only available in 3.8.11 and later */\n sqlite3_value **apSqlParam; /* Original SQL values of parameters */\n};\n\n/*\n** Allowed values for sqlite3_rtree_query.eWithin and .eParentWithin.\n*/\n#define NOT_WITHIN 0 /* Object completely outside of query region */\n#define PARTLY_WITHIN 1 /* Object partially overlaps query region */\n#define FULLY_WITHIN 2 /* Object fully contained within query region */\n\n\n#ifdef __cplusplus\n} /* end of the 'extern \"C\"' block */\n#endif\n\n#endif /* ifndef _SQLITE3RTREE_H_ */\n\n/******** End of sqlite3rtree.h *********/\n/******** Begin file sqlite3session.h *********/\n\n#if !defined(__SQLITESESSION_H_) && defined(SQLITE_ENABLE_SESSION)\n#define __SQLITESESSION_H_ 1\n\n/*\n** Make sure we can call this stuff from C++.\n*/\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n\n/*\n** CAPI3REF: Session Object Handle\n**\n** An instance of this object is a [session] that can be used to\n** record changes to a database.\n*/\ntypedef struct sqlite3_session sqlite3_session;\n\n/*\n** CAPI3REF: Changeset Iterator Handle\n**\n** An instance of this object acts as a cursor for iterating\n** over the elements of a [changeset] or [patchset].\n*/\ntypedef struct sqlite3_changeset_iter sqlite3_changeset_iter;\n\n/*\n** CAPI3REF: Create A New Session Object\n** CONSTRUCTOR: sqlite3_session\n**\n** Create a new session object attached to database handle db. If successful,\n** a pointer to the new object is written to *ppSession and SQLITE_OK is\n** returned. If an error occurs, *ppSession is set to NULL and an SQLite\n** error code (e.g. SQLITE_NOMEM) is returned.\n**\n** It is possible to create multiple session objects attached to a single\n** database handle.\n**\n** Session objects created using this function should be deleted using the\n** [sqlite3session_delete()] function before the database handle that they\n** are attached to is itself closed. If the database handle is closed before\n** the session object is deleted, then the results of calling any session\n** module function, including [sqlite3session_delete()] on the session object\n** are undefined.\n**\n** Because the session module uses the [sqlite3_preupdate_hook()] API, it\n** is not possible for an application to register a pre-update hook on a\n** database handle that has one or more session objects attached. Nor is\n** it possible to create a session object attached to a database handle for\n** which a pre-update hook is already defined. The results of attempting\n** either of these things are undefined.\n**\n** The session object will be used to create changesets for tables in\n** database zDb, where zDb is either \"main\", or \"temp\", or the name of an\n** attached database. It is not an error if database zDb is not attached\n** to the database when the session object is created.\n*/\nSQLITE_API int sqlite3session_create(\n sqlite3 *db, /* Database handle */\n const char *zDb, /* Name of db (e.g. \"main\") */\n sqlite3_session **ppSession /* OUT: New session object */\n);\n\n/*\n** CAPI3REF: Delete A Session Object\n** DESTRUCTOR: sqlite3_session\n**\n** Delete a session object previously allocated using\n** [sqlite3session_create()]. Once a session object has been deleted, the\n** results of attempting to use pSession with any other session module\n** function are undefined.\n**\n** Session objects must be deleted before the database handle to which they\n** are attached is closed. Refer to the documentation for\n** [sqlite3session_create()] for details.\n*/\nSQLITE_API void sqlite3session_delete(sqlite3_session *pSession);\n\n/*\n** CAPI3REF: Configure a Session Object\n** METHOD: sqlite3_session\n**\n** This method is used to configure a session object after it has been\n** created. At present the only valid values for the second parameter are\n** [SQLITE_SESSION_OBJCONFIG_SIZE] and [SQLITE_SESSION_OBJCONFIG_ROWID].\n**\n*/\nSQLITE_API int sqlite3session_object_config(sqlite3_session*, int op, void *pArg);\n\n/*\n** CAPI3REF: Options for sqlite3session_object_config\n**\n** The following values may passed as the the 2nd parameter to\n** sqlite3session_object_config().\n**\n**
SQLITE_SESSION_OBJCONFIG_SIZE
\n** This option is used to set, clear or query the flag that enables\n** the [sqlite3session_changeset_size()] API. Because it imposes some\n** computational overhead, this API is disabled by default. Argument\n** pArg must point to a value of type (int). If the value is initially\n** 0, then the sqlite3session_changeset_size() API is disabled. If it\n** is greater than 0, then the same API is enabled. Or, if the initial\n** value is less than zero, no change is made. In all cases the (int)\n** variable is set to 1 if the sqlite3session_changeset_size() API is\n** enabled following the current call, or 0 otherwise.\n**\n** It is an error (SQLITE_MISUSE) to attempt to modify this setting after\n** the first table has been attached to the session object.\n**\n**
SQLITE_SESSION_OBJCONFIG_ROWID
\n** This option is used to set, clear or query the flag that enables\n** collection of data for tables with no explicit PRIMARY KEY.\n**\n** Normally, tables with no explicit PRIMARY KEY are simply ignored\n** by the sessions module. However, if this flag is set, it behaves\n** as if such tables have a column \"_rowid_ INTEGER PRIMARY KEY\" inserted\n** as their leftmost columns.\n**\n** It is an error (SQLITE_MISUSE) to attempt to modify this setting after\n** the first table has been attached to the session object.\n*/\n#define SQLITE_SESSION_OBJCONFIG_SIZE 1\n#define SQLITE_SESSION_OBJCONFIG_ROWID 2\n\n/*\n** CAPI3REF: Enable Or Disable A Session Object\n** METHOD: sqlite3_session\n**\n** Enable or disable the recording of changes by a session object. When\n** enabled, a session object records changes made to the database. When\n** disabled - it does not. A newly created session object is enabled.\n** Refer to the documentation for [sqlite3session_changeset()] for further\n** details regarding how enabling and disabling a session object affects\n** the eventual changesets.\n**\n** Passing zero to this function disables the session. Passing a value\n** greater than zero enables it. Passing a value less than zero is a\n** no-op, and may be used to query the current state of the session.\n**\n** The return value indicates the final state of the session object: 0 if\n** the session is disabled, or 1 if it is enabled.\n*/\nSQLITE_API int sqlite3session_enable(sqlite3_session *pSession, int bEnable);\n\n/*\n** CAPI3REF: Set Or Clear the Indirect Change Flag\n** METHOD: sqlite3_session\n**\n** Each change recorded by a session object is marked as either direct or\n** indirect. A change is marked as indirect if either:\n**\n**
    \n**
  • The session object \"indirect\" flag is set when the change is\n** made, or\n**
  • The change is made by an SQL trigger or foreign key action\n** instead of directly as a result of a users SQL statement.\n**
\n**\n** If a single row is affected by more than one operation within a session,\n** then the change is considered indirect if all operations meet the criteria\n** for an indirect change above, or direct otherwise.\n**\n** This function is used to set, clear or query the session object indirect\n** flag. If the second argument passed to this function is zero, then the\n** indirect flag is cleared. If it is greater than zero, the indirect flag\n** is set. Passing a value less than zero does not modify the current value\n** of the indirect flag, and may be used to query the current state of the\n** indirect flag for the specified session object.\n**\n** The return value indicates the final state of the indirect flag: 0 if\n** it is clear, or 1 if it is set.\n*/\nSQLITE_API int sqlite3session_indirect(sqlite3_session *pSession, int bIndirect);\n\n/*\n** CAPI3REF: Attach A Table To A Session Object\n** METHOD: sqlite3_session\n**\n** If argument zTab is not NULL, then it is the name of a table to attach\n** to the session object passed as the first argument. All subsequent changes\n** made to the table while the session object is enabled will be recorded. See\n** documentation for [sqlite3session_changeset()] for further details.\n**\n** Or, if argument zTab is NULL, then changes are recorded for all tables\n** in the database. If additional tables are added to the database (by\n** executing \"CREATE TABLE\" statements) after this call is made, changes for\n** the new tables are also recorded.\n**\n** Changes can only be recorded for tables that have a PRIMARY KEY explicitly\n** defined as part of their CREATE TABLE statement. It does not matter if the\n** PRIMARY KEY is an \"INTEGER PRIMARY KEY\" (rowid alias) or not. The PRIMARY\n** KEY may consist of a single column, or may be a composite key.\n**\n** It is not an error if the named table does not exist in the database. Nor\n** is it an error if the named table does not have a PRIMARY KEY. However,\n** no changes will be recorded in either of these scenarios.\n**\n** Changes are not recorded for individual rows that have NULL values stored\n** in one or more of their PRIMARY KEY columns.\n**\n** SQLITE_OK is returned if the call completes without error. Or, if an error\n** occurs, an SQLite error code (e.g. SQLITE_NOMEM) is returned.\n**\n**

Special sqlite_stat1 Handling

\n**\n** As of SQLite version 3.22.0, the \"sqlite_stat1\" table is an exception to\n** some of the rules above. In SQLite, the schema of sqlite_stat1 is:\n** 
\n**        CREATE TABLE sqlite_stat1(tbl,idx,stat)\n**
\n**\n** Even though sqlite_stat1 does not have a PRIMARY KEY, changes are\n** recorded for it as if the PRIMARY KEY is (tbl,idx). Additionally, changes\n** are recorded for rows for which (idx IS NULL) is true. However, for such\n** rows a zero-length blob (SQL value X'') is stored in the changeset or\n** patchset instead of a NULL value. This allows such changesets to be\n** manipulated by legacy implementations of sqlite3changeset_invert(),\n** concat() and similar.\n**\n** The sqlite3changeset_apply() function automatically converts the\n** zero-length blob back to a NULL value when updating the sqlite_stat1\n** table. However, if the application calls sqlite3changeset_new(),\n** sqlite3changeset_old() or sqlite3changeset_conflict on a changeset\n** iterator directly (including on a changeset iterator passed to a\n** conflict-handler callback) then the X'' value is returned. The application\n** must translate X'' to NULL itself if required.\n**\n** Legacy (older than 3.22.0) versions of the sessions module cannot capture\n** changes made to the sqlite_stat1 table. Legacy versions of the\n** sqlite3changeset_apply() function silently ignore any modifications to the\n** sqlite_stat1 table that are part of a changeset or patchset.\n*/\nSQLITE_API int sqlite3session_attach(\n sqlite3_session *pSession, /* Session object */\n const char *zTab /* Table name */\n);\n\n/*\n** CAPI3REF: Set a table filter on a Session Object.\n** METHOD: sqlite3_session\n**\n** The second argument (xFilter) is the \"filter callback\". For changes to rows\n** in tables that are not attached to the Session object, the filter is called\n** to determine whether changes to the table's rows should be tracked or not.\n** If xFilter returns 0, changes are not tracked. Note that once a table is\n** attached, xFilter will not be called again.\n*/\nSQLITE_API void sqlite3session_table_filter(\n sqlite3_session *pSession, /* Session object */\n int(*xFilter)(\n void *pCtx, /* Copy of third arg to _filter_table() */\n const char *zTab /* Table name */\n ),\n void *pCtx /* First argument passed to xFilter */\n);\n\n/*\n** CAPI3REF: Generate A Changeset From A Session Object\n** METHOD: sqlite3_session\n**\n** Obtain a changeset containing changes to the tables attached to the\n** session object passed as the first argument. If successful,\n** set *ppChangeset to point to a buffer containing the changeset\n** and *pnChangeset to the size of the changeset in bytes before returning\n** SQLITE_OK. If an error occurs, set both *ppChangeset and *pnChangeset to\n** zero and return an SQLite error code.\n**\n** A changeset consists of zero or more INSERT, UPDATE and/or DELETE changes,\n** each representing a change to a single row of an attached table. An INSERT\n** change contains the values of each field of a new database row. A DELETE\n** contains the original values of each field of a deleted database row. An\n** UPDATE change contains the original values of each field of an updated\n** database row along with the updated values for each updated non-primary-key\n** column. It is not possible for an UPDATE change to represent a change that\n** modifies the values of primary key columns. If such a change is made, it\n** is represented in a changeset as a DELETE followed by an INSERT.\n**\n** Changes are not recorded for rows that have NULL values stored in one or\n** more of their PRIMARY KEY columns. If such a row is inserted or deleted,\n** no corresponding change is present in the changesets returned by this\n** function. If an existing row with one or more NULL values stored in\n** PRIMARY KEY columns is updated so that all PRIMARY KEY columns are non-NULL,\n** only an INSERT is appears in the changeset. Similarly, if an existing row\n** with non-NULL PRIMARY KEY values is updated so that one or more of its\n** PRIMARY KEY columns are set to NULL, the resulting changeset contains a\n** DELETE change only.\n**\n** The contents of a changeset may be traversed using an iterator created\n** using the [sqlite3changeset_start()] API. A changeset may be applied to\n** a database with a compatible schema using the [sqlite3changeset_apply()]\n** API.\n**\n** Within a changeset generated by this function, all changes related to a\n** single table are grouped together. In other words, when iterating through\n** a changeset or when applying a changeset to a database, all changes related\n** to a single table are processed before moving on to the next table. Tables\n** are sorted in the same order in which they were attached (or auto-attached)\n** to the sqlite3_session object. The order in which the changes related to\n** a single table are stored is undefined.\n**\n** Following a successful call to this function, it is the responsibility of\n** the caller to eventually free the buffer that *ppChangeset points to using\n** [sqlite3_free()].\n**\n**

Changeset Generation

\n**\n** Once a table has been attached to a session object, the session object\n** records the primary key values of all new rows inserted into the table.\n** It also records the original primary key and other column values of any\n** deleted or updated rows. For each unique primary key value, data is only\n** recorded once - the first time a row with said primary key is inserted,\n** updated or deleted in the lifetime of the session.\n**\n** There is one exception to the previous paragraph: when a row is inserted,\n** updated or deleted, if one or more of its primary key columns contain a\n** NULL value, no record of the change is made.\n**\n** The session object therefore accumulates two types of records - those\n** that consist of primary key values only (created when the user inserts\n** a new record) and those that consist of the primary key values and the\n** original values of other table columns (created when the users deletes\n** or updates a record).\n**\n** When this function is called, the requested changeset is created using\n** both the accumulated records and the current contents of the database\n** file. Specifically:\n**\n**
    \n**
  • For each record generated by an insert, the database is queried\n** for a row with a matching primary key. If one is found, an INSERT\n** change is added to the changeset. If no such row is found, no change\n** is added to the changeset.\n**\n**
  • For each record generated by an update or delete, the database is\n** queried for a row with a matching primary key. If such a row is\n** found and one or more of the non-primary key fields have been\n** modified from their original values, an UPDATE change is added to\n** the changeset. Or, if no such row is found in the table, a DELETE\n** change is added to the changeset. If there is a row with a matching\n** primary key in the database, but all fields contain their original\n** values, no change is added to the changeset.\n**
\n**\n** This means, amongst other things, that if a row is inserted and then later\n** deleted while a session object is active, neither the insert nor the delete\n** will be present in the changeset. Or if a row is deleted and then later a\n** row with the same primary key values inserted while a session object is\n** active, the resulting changeset will contain an UPDATE change instead of\n** a DELETE and an INSERT.\n**\n** When a session object is disabled (see the [sqlite3session_enable()] API),\n** it does not accumulate records when rows are inserted, updated or deleted.\n** This may appear to have some counter-intuitive effects if a single row\n** is written to more than once during a session. For example, if a row\n** is inserted while a session object is enabled, then later deleted while\n** the same session object is disabled, no INSERT record will appear in the\n** changeset, even though the delete took place while the session was disabled.\n** Or, if one field of a row is updated while a session is enabled, and\n** then another field of the same row is updated while the session is disabled,\n** the resulting changeset will contain an UPDATE change that updates both\n** fields.\n*/\nSQLITE_API int sqlite3session_changeset(\n sqlite3_session *pSession, /* Session object */\n int *pnChangeset, /* OUT: Size of buffer at *ppChangeset */\n void **ppChangeset /* OUT: Buffer containing changeset */\n);\n\n/*\n** CAPI3REF: Return An Upper-limit For The Size Of The Changeset\n** METHOD: sqlite3_session\n**\n** By default, this function always returns 0. For it to return\n** a useful result, the sqlite3_session object must have been configured\n** to enable this API using sqlite3session_object_config() with the\n** SQLITE_SESSION_OBJCONFIG_SIZE verb.\n**\n** When enabled, this function returns an upper limit, in bytes, for the size\n** of the changeset that might be produced if sqlite3session_changeset() were\n** called. The final changeset size might be equal to or smaller than the\n** size in bytes returned by this function.\n*/\nSQLITE_API sqlite3_int64 sqlite3session_changeset_size(sqlite3_session *pSession);\n\n/*\n** CAPI3REF: Load The Difference Between Tables Into A Session\n** METHOD: sqlite3_session\n**\n** If it is not already attached to the session object passed as the first\n** argument, this function attaches table zTbl in the same manner as the\n** [sqlite3session_attach()] function. If zTbl does not exist, or if it\n** does not have a primary key, this function is a no-op (but does not return\n** an error).\n**\n** Argument zFromDb must be the name of a database (\"main\", \"temp\" etc.)\n** attached to the same database handle as the session object that contains\n** a table compatible with the table attached to the session by this function.\n** A table is considered compatible if it:\n**\n**
    \n**
  • Has the same name,\n**
  • Has the same set of columns declared in the same order, and\n**
  • Has the same PRIMARY KEY definition.\n**
\n**\n** If the tables are not compatible, SQLITE_SCHEMA is returned. If the tables\n** are compatible but do not have any PRIMARY KEY columns, it is not an error\n** but no changes are added to the session object. As with other session\n** APIs, tables without PRIMARY KEYs are simply ignored.\n**\n** This function adds a set of changes to the session object that could be\n** used to update the table in database zFrom (call this the \"from-table\")\n** so that its content is the same as the table attached to the session\n** object (call this the \"to-table\"). Specifically:\n**\n**
    \n**
  • For each row (primary key) that exists in the to-table but not in\n** the from-table, an INSERT record is added to the session object.\n**\n**
  • For each row (primary key) that exists in the to-table but not in\n** the from-table, a DELETE record is added to the session object.\n**\n**
  • For each row (primary key) that exists in both tables, but features\n** different non-PK values in each, an UPDATE record is added to the\n** session.\n**
\n**\n** To clarify, if this function is called and then a changeset constructed\n** using [sqlite3session_changeset()], then after applying that changeset to\n** database zFrom the contents of the two compatible tables would be\n** identical.\n**\n** Unless the call to this function is a no-op as described above, it is an\n** error if database zFrom does not exist or does not contain the required\n** compatible table.\n**\n** If the operation is successful, SQLITE_OK is returned. Otherwise, an SQLite\n** error code. In this case, if argument pzErrMsg is not NULL, *pzErrMsg\n** may be set to point to a buffer containing an English language error\n** message. It is the responsibility of the caller to free this buffer using\n** sqlite3_free().\n*/\nSQLITE_API int sqlite3session_diff(\n sqlite3_session *pSession,\n const char *zFromDb,\n const char *zTbl,\n char **pzErrMsg\n);\n\n\n/*\n** CAPI3REF: Generate A Patchset From A Session Object\n** METHOD: sqlite3_session\n**\n** The differences between a patchset and a changeset are that:\n**\n**
    \n**
  • DELETE records consist of the primary key fields only. The\n** original values of other fields are omitted.\n**
  • The original values of any modified fields are omitted from\n** UPDATE records.\n**
\n**\n** A patchset blob may be used with up to date versions of all\n** sqlite3changeset_xxx API functions except for sqlite3changeset_invert(),\n** which returns SQLITE_CORRUPT if it is passed a patchset. Similarly,\n** attempting to use a patchset blob with old versions of the\n** sqlite3changeset_xxx APIs also provokes an SQLITE_CORRUPT error.\n**\n** Because the non-primary key \"old.*\" fields are omitted, no\n** SQLITE_CHANGESET_DATA conflicts can be detected or reported if a patchset\n** is passed to the sqlite3changeset_apply() API. Other conflict types work\n** in the same way as for changesets.\n**\n** Changes within a patchset are ordered in the same way as for changesets\n** generated by the sqlite3session_changeset() function (i.e. all changes for\n** a single table are grouped together, tables appear in the order in which\n** they were attached to the session object).\n*/\nSQLITE_API int sqlite3session_patchset(\n sqlite3_session *pSession, /* Session object */\n int *pnPatchset, /* OUT: Size of buffer at *ppPatchset */\n void **ppPatchset /* OUT: Buffer containing patchset */\n);\n\n/*\n** CAPI3REF: Test if a changeset has recorded any changes.\n**\n** Return non-zero if no changes to attached tables have been recorded by\n** the session object passed as the first argument. Otherwise, if one or\n** more changes have been recorded, return zero.\n**\n** Even if this function returns zero, it is possible that calling\n** [sqlite3session_changeset()] on the session handle may still return a\n** changeset that contains no changes. This can happen when a row in\n** an attached table is modified and then later on the original values\n** are restored. However, if this function returns non-zero, then it is\n** guaranteed that a call to sqlite3session_changeset() will return a\n** changeset containing zero changes.\n*/\nSQLITE_API int sqlite3session_isempty(sqlite3_session *pSession);\n\n/*\n** CAPI3REF: Query for the amount of heap memory used by a session object.\n**\n** This API returns the total amount of heap memory in bytes currently\n** used by the session object passed as the only argument.\n*/\nSQLITE_API sqlite3_int64 sqlite3session_memory_used(sqlite3_session *pSession);\n\n/*\n** CAPI3REF: Create An Iterator To Traverse A Changeset\n** CONSTRUCTOR: sqlite3_changeset_iter\n**\n** Create an iterator used to iterate through the contents of a changeset.\n** If successful, *pp is set to point to the iterator handle and SQLITE_OK\n** is returned. Otherwise, if an error occurs, *pp is set to zero and an\n** SQLite error code is returned.\n**\n** The following functions can be used to advance and query a changeset\n** iterator created by this function:\n**\n**
    \n**
  • [sqlite3changeset_next()]\n**
  • [sqlite3changeset_op()]\n**
  • [sqlite3changeset_new()]\n**
  • [sqlite3changeset_old()]\n**
\n**\n** It is the responsibility of the caller to eventually destroy the iterator\n** by passing it to [sqlite3changeset_finalize()]. The buffer containing the\n** changeset (pChangeset) must remain valid until after the iterator is\n** destroyed.\n**\n** Assuming the changeset blob was created by one of the\n** [sqlite3session_changeset()], [sqlite3changeset_concat()] or\n** [sqlite3changeset_invert()] functions, all changes within the changeset\n** that apply to a single table are grouped together. This means that when\n** an application iterates through a changeset using an iterator created by\n** this function, all changes that relate to a single table are visited\n** consecutively. There is no chance that the iterator will visit a change\n** the applies to table X, then one for table Y, and then later on visit\n** another change for table X.\n**\n** The behavior of sqlite3changeset_start_v2() and its streaming equivalent\n** may be modified by passing a combination of\n** [SQLITE_CHANGESETSTART_INVERT | supported flags] as the 4th parameter.\n**\n** Note that the sqlite3changeset_start_v2() API is still experimental\n** and therefore subject to change.\n*/\nSQLITE_API int sqlite3changeset_start(\n sqlite3_changeset_iter **pp, /* OUT: New changeset iterator handle */\n int nChangeset, /* Size of changeset blob in bytes */\n void *pChangeset /* Pointer to blob containing changeset */\n);\nSQLITE_API int sqlite3changeset_start_v2(\n sqlite3_changeset_iter **pp, /* OUT: New changeset iterator handle */\n int nChangeset, /* Size of changeset blob in bytes */\n void *pChangeset, /* Pointer to blob containing changeset */\n int flags /* SESSION_CHANGESETSTART_* flags */\n);\n\n/*\n** CAPI3REF: Flags for sqlite3changeset_start_v2\n**\n** The following flags may passed via the 4th parameter to\n** [sqlite3changeset_start_v2] and [sqlite3changeset_start_v2_strm]:\n**\n**
SQLITE_CHANGESETSTART_INVERT
\n** Invert the changeset while iterating through it. This is equivalent to\n** inverting a changeset using sqlite3changeset_invert() before applying it.\n** It is an error to specify this flag with a patchset.\n*/\n#define SQLITE_CHANGESETSTART_INVERT 0x0002\n\n\n/*\n** CAPI3REF: Advance A Changeset Iterator\n** METHOD: sqlite3_changeset_iter\n**\n** This function may only be used with iterators created by the function\n** [sqlite3changeset_start()]. If it is called on an iterator passed to\n** a conflict-handler callback by [sqlite3changeset_apply()], SQLITE_MISUSE\n** is returned and the call has no effect.\n**\n** Immediately after an iterator is created by sqlite3changeset_start(), it\n** does not point to any change in the changeset. Assuming the changeset\n** is not empty, the first call to this function advances the iterator to\n** point to the first change in the changeset. Each subsequent call advances\n** the iterator to point to the next change in the changeset (if any). If\n** no error occurs and the iterator points to a valid change after a call\n** to sqlite3changeset_next() has advanced it, SQLITE_ROW is returned.\n** Otherwise, if all changes in the changeset have already been visited,\n** SQLITE_DONE is returned.\n**\n** If an error occurs, an SQLite error code is returned. Possible error\n** codes include SQLITE_CORRUPT (if the changeset buffer is corrupt) or\n** SQLITE_NOMEM.\n*/\nSQLITE_API int sqlite3changeset_next(sqlite3_changeset_iter *pIter);\n\n/*\n** CAPI3REF: Obtain The Current Operation From A Changeset Iterator\n** METHOD: sqlite3_changeset_iter\n**\n** The pIter argument passed to this function may either be an iterator\n** passed to a conflict-handler by [sqlite3changeset_apply()], or an iterator\n** created by [sqlite3changeset_start()]. In the latter case, the most recent\n** call to [sqlite3changeset_next()] must have returned [SQLITE_ROW]. If this\n** is not the case, this function returns [SQLITE_MISUSE].\n**\n** Arguments pOp, pnCol and pzTab may not be NULL. Upon return, three\n** outputs are set through these pointers:\n**\n** *pOp is set to one of [SQLITE_INSERT], [SQLITE_DELETE] or [SQLITE_UPDATE],\n** depending on the type of change that the iterator currently points to;\n**\n** *pnCol is set to the number of columns in the table affected by the change; and\n**\n** *pzTab is set to point to a nul-terminated utf-8 encoded string containing\n** the name of the table affected by the current change. The buffer remains\n** valid until either sqlite3changeset_next() is called on the iterator\n** or until the conflict-handler function returns.\n**\n** If pbIndirect is not NULL, then *pbIndirect is set to true (1) if the change\n** is an indirect change, or false (0) otherwise. See the documentation for\n** [sqlite3session_indirect()] for a description of direct and indirect\n** changes.\n**\n** If no error occurs, SQLITE_OK is returned. If an error does occur, an\n** SQLite error code is returned. The values of the output variables may not\n** be trusted in this case.\n*/\nSQLITE_API int sqlite3changeset_op(\n sqlite3_changeset_iter *pIter, /* Iterator object */\n const char **pzTab, /* OUT: Pointer to table name */\n int *pnCol, /* OUT: Number of columns in table */\n int *pOp, /* OUT: SQLITE_INSERT, DELETE or UPDATE */\n int *pbIndirect /* OUT: True for an 'indirect' change */\n);\n\n/*\n** CAPI3REF: Obtain The Primary Key Definition Of A Table\n** METHOD: sqlite3_changeset_iter\n**\n** For each modified table, a changeset includes the following:\n**\n**
    \n**
  • The number of columns in the table, and\n**
  • Which of those columns make up the tables PRIMARY KEY.\n**
\n**\n** This function is used to find which columns comprise the PRIMARY KEY of\n** the table modified by the change that iterator pIter currently points to.\n** If successful, *pabPK is set to point to an array of nCol entries, where\n** nCol is the number of columns in the table. Elements of *pabPK are set to\n** 0x01 if the corresponding column is part of the tables primary key, or\n** 0x00 if it is not.\n**\n** If argument pnCol is not NULL, then *pnCol is set to the number of columns\n** in the table.\n**\n** If this function is called when the iterator does not point to a valid\n** entry, SQLITE_MISUSE is returned and the output variables zeroed. Otherwise,\n** SQLITE_OK is returned and the output variables populated as described\n** above.\n*/\nSQLITE_API int sqlite3changeset_pk(\n sqlite3_changeset_iter *pIter, /* Iterator object */\n unsigned char **pabPK, /* OUT: Array of boolean - true for PK cols */\n int *pnCol /* OUT: Number of entries in output array */\n);\n\n/*\n** CAPI3REF: Obtain old.* Values From A Changeset Iterator\n** METHOD: sqlite3_changeset_iter\n**\n** The pIter argument passed to this function may either be an iterator\n** passed to a conflict-handler by [sqlite3changeset_apply()], or an iterator\n** created by [sqlite3changeset_start()]. In the latter case, the most recent\n** call to [sqlite3changeset_next()] must have returned SQLITE_ROW.\n** Furthermore, it may only be called if the type of change that the iterator\n** currently points to is either [SQLITE_DELETE] or [SQLITE_UPDATE]. Otherwise,\n** this function returns [SQLITE_MISUSE] and sets *ppValue to NULL.\n**\n** Argument iVal must be greater than or equal to 0, and less than the number\n** of columns in the table affected by the current change. Otherwise,\n** [SQLITE_RANGE] is returned and *ppValue is set to NULL.\n**\n** If successful, this function sets *ppValue to point to a protected\n** sqlite3_value object containing the iVal'th value from the vector of\n** original row values stored as part of the UPDATE or DELETE change and\n** returns SQLITE_OK. The name of the function comes from the fact that this\n** is similar to the \"old.*\" columns available to update or delete triggers.\n**\n** If some other error occurs (e.g. an OOM condition), an SQLite error code\n** is returned and *ppValue is set to NULL.\n*/\nSQLITE_API int sqlite3changeset_old(\n sqlite3_changeset_iter *pIter, /* Changeset iterator */\n int iVal, /* Column number */\n sqlite3_value **ppValue /* OUT: Old value (or NULL pointer) */\n);\n\n/*\n** CAPI3REF: Obtain new.* Values From A Changeset Iterator\n** METHOD: sqlite3_changeset_iter\n**\n** The pIter argument passed to this function may either be an iterator\n** passed to a conflict-handler by [sqlite3changeset_apply()], or an iterator\n** created by [sqlite3changeset_start()]. In the latter case, the most recent\n** call to [sqlite3changeset_next()] must have returned SQLITE_ROW.\n** Furthermore, it may only be called if the type of change that the iterator\n** currently points to is either [SQLITE_UPDATE] or [SQLITE_INSERT]. Otherwise,\n** this function returns [SQLITE_MISUSE] and sets *ppValue to NULL.\n**\n** Argument iVal must be greater than or equal to 0, and less than the number\n** of columns in the table affected by the current change. Otherwise,\n** [SQLITE_RANGE] is returned and *ppValue is set to NULL.\n**\n** If successful, this function sets *ppValue to point to a protected\n** sqlite3_value object containing the iVal'th value from the vector of\n** new row values stored as part of the UPDATE or INSERT change and\n** returns SQLITE_OK. If the change is an UPDATE and does not include\n** a new value for the requested column, *ppValue is set to NULL and\n** SQLITE_OK returned. The name of the function comes from the fact that\n** this is similar to the \"new.*\" columns available to update or delete\n** triggers.\n**\n** If some other error occurs (e.g. an OOM condition), an SQLite error code\n** is returned and *ppValue is set to NULL.\n*/\nSQLITE_API int sqlite3changeset_new(\n sqlite3_changeset_iter *pIter, /* Changeset iterator */\n int iVal, /* Column number */\n sqlite3_value **ppValue /* OUT: New value (or NULL pointer) */\n);\n\n/*\n** CAPI3REF: Obtain Conflicting Row Values From A Changeset Iterator\n** METHOD: sqlite3_changeset_iter\n**\n** This function should only be used with iterator objects passed to a\n** conflict-handler callback by [sqlite3changeset_apply()] with either\n** [SQLITE_CHANGESET_DATA] or [SQLITE_CHANGESET_CONFLICT]. If this function\n** is called on any other iterator, [SQLITE_MISUSE] is returned and *ppValue\n** is set to NULL.\n**\n** Argument iVal must be greater than or equal to 0, and less than the number\n** of columns in the table affected by the current change. Otherwise,\n** [SQLITE_RANGE] is returned and *ppValue is set to NULL.\n**\n** If successful, this function sets *ppValue to point to a protected\n** sqlite3_value object containing the iVal'th value from the\n** \"conflicting row\" associated with the current conflict-handler callback\n** and returns SQLITE_OK.\n**\n** If some other error occurs (e.g. an OOM condition), an SQLite error code\n** is returned and *ppValue is set to NULL.\n*/\nSQLITE_API int sqlite3changeset_conflict(\n sqlite3_changeset_iter *pIter, /* Changeset iterator */\n int iVal, /* Column number */\n sqlite3_value **ppValue /* OUT: Value from conflicting row */\n);\n\n/*\n** CAPI3REF: Determine The Number Of Foreign Key Constraint Violations\n** METHOD: sqlite3_changeset_iter\n**\n** This function may only be called with an iterator passed to an\n** SQLITE_CHANGESET_FOREIGN_KEY conflict handler callback. In this case\n** it sets the output variable to the total number of known foreign key\n** violations in the destination database and returns SQLITE_OK.\n**\n** In all other cases this function returns SQLITE_MISUSE.\n*/\nSQLITE_API int sqlite3changeset_fk_conflicts(\n sqlite3_changeset_iter *pIter, /* Changeset iterator */\n int *pnOut /* OUT: Number of FK violations */\n);\n\n\n/*\n** CAPI3REF: Finalize A Changeset Iterator\n** METHOD: sqlite3_changeset_iter\n**\n** This function is used to finalize an iterator allocated with\n** [sqlite3changeset_start()].\n**\n** This function should only be called on iterators created using the\n** [sqlite3changeset_start()] function. If an application calls this\n** function with an iterator passed to a conflict-handler by\n** [sqlite3changeset_apply()], [SQLITE_MISUSE] is immediately returned and the\n** call has no effect.\n**\n** If an error was encountered within a call to an sqlite3changeset_xxx()\n** function (for example an [SQLITE_CORRUPT] in [sqlite3changeset_next()] or an\n** [SQLITE_NOMEM] in [sqlite3changeset_new()]) then an error code corresponding\n** to that error is returned by this function. Otherwise, SQLITE_OK is\n** returned. This is to allow the following pattern (pseudo-code):\n**\n** 
\n**   sqlite3changeset_start();\n**   while( SQLITE_ROW==sqlite3changeset_next() ){\n**     // Do something with change.\n**   }\n**   rc = sqlite3changeset_finalize();\n**   if( rc!=SQLITE_OK ){\n**     // An error has occurred\n**   }\n**
\n*/\nSQLITE_API int sqlite3changeset_finalize(sqlite3_changeset_iter *pIter);\n\n/*\n** CAPI3REF: Invert A Changeset\n**\n** This function is used to \"invert\" a changeset object. Applying an inverted\n** changeset to a database reverses the effects of applying the uninverted\n** changeset. Specifically:\n**\n**
    \n**
  • Each DELETE change is changed to an INSERT, and\n**
  • Each INSERT change is changed to a DELETE, and\n**
  • For each UPDATE change, the old.* and new.* values are exchanged.\n**
\n**\n** This function does not change the order in which changes appear within\n** the changeset. It merely reverses the sense of each individual change.\n**\n** If successful, a pointer to a buffer containing the inverted changeset\n** is stored in *ppOut, the size of the same buffer is stored in *pnOut, and\n** SQLITE_OK is returned. If an error occurs, both *pnOut and *ppOut are\n** zeroed and an SQLite error code returned.\n**\n** It is the responsibility of the caller to eventually call sqlite3_free()\n** on the *ppOut pointer to free the buffer allocation following a successful\n** call to this function.\n**\n** WARNING/TODO: This function currently assumes that the input is a valid\n** changeset. If it is not, the results are undefined.\n*/\nSQLITE_API int sqlite3changeset_invert(\n int nIn, const void *pIn, /* Input changeset */\n int *pnOut, void **ppOut /* OUT: Inverse of input */\n);\n\n/*\n** CAPI3REF: Concatenate Two Changeset Objects\n**\n** This function is used to concatenate two changesets, A and B, into a\n** single changeset. The result is a changeset equivalent to applying\n** changeset A followed by changeset B.\n**\n** This function combines the two input changesets using an\n** sqlite3_changegroup object. Calling it produces similar results as the\n** following code fragment:\n**\n** 
\n**   sqlite3_changegroup *pGrp;\n**   rc = sqlite3_changegroup_new(&pGrp);\n**   if( rc==SQLITE_OK ) rc = sqlite3changegroup_add(pGrp, nA, pA);\n**   if( rc==SQLITE_OK ) rc = sqlite3changegroup_add(pGrp, nB, pB);\n**   if( rc==SQLITE_OK ){\n**     rc = sqlite3changegroup_output(pGrp, pnOut, ppOut);\n**   }else{\n**     *ppOut = 0;\n**     *pnOut = 0;\n**   }\n**
\n**\n** Refer to the sqlite3_changegroup documentation below for details.\n*/\nSQLITE_API int sqlite3changeset_concat(\n int nA, /* Number of bytes in buffer pA */\n void *pA, /* Pointer to buffer containing changeset A */\n int nB, /* Number of bytes in buffer pB */\n void *pB, /* Pointer to buffer containing changeset B */\n int *pnOut, /* OUT: Number of bytes in output changeset */\n void **ppOut /* OUT: Buffer containing output changeset */\n);\n\n/*\n** CAPI3REF: Changegroup Handle\n**\n** A changegroup is an object used to combine two or more\n** [changesets] or [patchsets]\n*/\ntypedef struct sqlite3_changegroup sqlite3_changegroup;\n\n/*\n** CAPI3REF: Create A New Changegroup Object\n** CONSTRUCTOR: sqlite3_changegroup\n**\n** An sqlite3_changegroup object is used to combine two or more changesets\n** (or patchsets) into a single changeset (or patchset). A single changegroup\n** object may combine changesets or patchsets, but not both. The output is\n** always in the same format as the input.\n**\n** If successful, this function returns SQLITE_OK and populates (*pp) with\n** a pointer to a new sqlite3_changegroup object before returning. The caller\n** should eventually free the returned object using a call to\n** sqlite3changegroup_delete(). If an error occurs, an SQLite error code\n** (i.e. SQLITE_NOMEM) is returned and *pp is set to NULL.\n**\n** The usual usage pattern for an sqlite3_changegroup object is as follows:\n**\n**
    \n**
  • It is created using a call to sqlite3changegroup_new().\n**\n**
  • Zero or more changesets (or patchsets) are added to the object\n** by calling sqlite3changegroup_add().\n**\n**
  • The result of combining all input changesets together is obtained\n** by the application via a call to sqlite3changegroup_output().\n**\n**
  • The object is deleted using a call to sqlite3changegroup_delete().\n**
\n**\n** Any number of calls to add() and output() may be made between the calls to\n** new() and delete(), and in any order.\n**\n** As well as the regular sqlite3changegroup_add() and\n** sqlite3changegroup_output() functions, also available are the streaming\n** versions sqlite3changegroup_add_strm() and sqlite3changegroup_output_strm().\n*/\nSQLITE_API int sqlite3changegroup_new(sqlite3_changegroup **pp);\n\n/*\n** CAPI3REF: Add a Schema to a Changegroup\n** METHOD: sqlite3_changegroup_schema\n**\n** This method may be used to optionally enforce the rule that the changesets\n** added to the changegroup handle must match the schema of database zDb\n** (\"main\", \"temp\", or the name of an attached database). If\n** sqlite3changegroup_add() is called to add a changeset that is not compatible\n** with the configured schema, SQLITE_SCHEMA is returned and the changegroup\n** object is left in an undefined state.\n**\n** A changeset schema is considered compatible with the database schema in\n** the same way as for sqlite3changeset_apply(). Specifically, for each\n** table in the changeset, there exists a database table with:\n**\n**
    \n**
  • The name identified by the changeset, and\n**
  • at least as many columns as recorded in the changeset, and\n**
  • the primary key columns in the same position as recorded in\n** the changeset.\n**
\n**\n** The output of the changegroup object always has the same schema as the\n** database nominated using this function. In cases where changesets passed\n** to sqlite3changegroup_add() have fewer columns than the corresponding table\n** in the database schema, these are filled in using the default column\n** values from the database schema. This makes it possible to combined\n** changesets that have different numbers of columns for a single table\n** within a changegroup, provided that they are otherwise compatible.\n*/\nSQLITE_API int sqlite3changegroup_schema(sqlite3_changegroup*, sqlite3*, const char *zDb);\n\n/*\n** CAPI3REF: Add A Changeset To A Changegroup\n** METHOD: sqlite3_changegroup\n**\n** Add all changes within the changeset (or patchset) in buffer pData (size\n** nData bytes) to the changegroup.\n**\n** If the buffer contains a patchset, then all prior calls to this function\n** on the same changegroup object must also have specified patchsets. Or, if\n** the buffer contains a changeset, so must have the earlier calls to this\n** function. Otherwise, SQLITE_ERROR is returned and no changes are added\n** to the changegroup.\n**\n** Rows within the changeset and changegroup are identified by the values in\n** their PRIMARY KEY columns. A change in the changeset is considered to\n** apply to the same row as a change already present in the changegroup if\n** the two rows have the same primary key.\n**\n** Changes to rows that do not already appear in the changegroup are\n** simply copied into it. Or, if both the new changeset and the changegroup\n** contain changes that apply to a single row, the final contents of the\n** changegroup depends on the type of each change, as follows:\n**\n** \n** \n** \n**
Existing Change New Change Output Change\n**
INSERT INSERT \n** The new change is ignored. This case does not occur if the new\n** changeset was recorded immediately after the changesets already\n** added to the changegroup.\n**
INSERT UPDATE \n** The INSERT change remains in the changegroup. The values in the\n** INSERT change are modified as if the row was inserted by the\n** existing change and then updated according to the new change.\n**
INSERT DELETE \n** The existing INSERT is removed from the changegroup. The DELETE is\n** not added.\n**
UPDATE INSERT \n** The new change is ignored. This case does not occur if the new\n** changeset was recorded immediately after the changesets already\n** added to the changegroup.\n**
UPDATE UPDATE \n** The existing UPDATE remains within the changegroup. It is amended\n** so that the accompanying values are as if the row was updated once\n** by the existing change and then again by the new change.\n**
UPDATE DELETE \n** The existing UPDATE is replaced by the new DELETE within the\n** changegroup.\n**
DELETE INSERT \n** If one or more of the column values in the row inserted by the\n** new change differ from those in the row deleted by the existing\n** change, the existing DELETE is replaced by an UPDATE within the\n** changegroup. Otherwise, if the inserted row is exactly the same\n** as the deleted row, the existing DELETE is simply discarded.\n**
DELETE UPDATE \n** The new change is ignored. This case does not occur if the new\n** changeset was recorded immediately after the changesets already\n** added to the changegroup.\n**
DELETE DELETE \n** The new change is ignored. This case does not occur if the new\n** changeset was recorded immediately after the changesets already\n** added to the changegroup.\n**
\n**\n** If the new changeset contains changes to a table that is already present\n** in the changegroup, then the number of columns and the position of the\n** primary key columns for the table must be consistent. If this is not the\n** case, this function fails with SQLITE_SCHEMA. Except, if the changegroup\n** object has been configured with a database schema using the\n** sqlite3changegroup_schema() API, then it is possible to combine changesets\n** with different numbers of columns for a single table, provided that\n** they are otherwise compatible.\n**\n** If the input changeset appears to be corrupt and the corruption is\n** detected, SQLITE_CORRUPT is returned. Or, if an out-of-memory condition\n** occurs during processing, this function returns SQLITE_NOMEM.\n**\n** In all cases, if an error occurs the state of the final contents of the\n** changegroup is undefined. If no error occurs, SQLITE_OK is returned.\n*/\nSQLITE_API int sqlite3changegroup_add(sqlite3_changegroup*, int nData, void *pData);\n\n/*\n** CAPI3REF: Add A Single Change To A Changegroup\n** METHOD: sqlite3_changegroup\n**\n** This function adds the single change currently indicated by the iterator\n** passed as the second argument to the changegroup object. The rules for\n** adding the change are just as described for [sqlite3changegroup_add()].\n**\n** If the change is successfully added to the changegroup, SQLITE_OK is\n** returned. Otherwise, an SQLite error code is returned.\n**\n** The iterator must point to a valid entry when this function is called.\n** If it does not, SQLITE_ERROR is returned and no change is added to the\n** changegroup. Additionally, the iterator must not have been opened with\n** the SQLITE_CHANGESETAPPLY_INVERT flag. In this case SQLITE_ERROR is also\n** returned.\n*/\nSQLITE_API int sqlite3changegroup_add_change(\n sqlite3_changegroup*,\n sqlite3_changeset_iter*\n);\n\n\n\n/*\n** CAPI3REF: Obtain A Composite Changeset From A Changegroup\n** METHOD: sqlite3_changegroup\n**\n** Obtain a buffer containing a changeset (or patchset) representing the\n** current contents of the changegroup. If the inputs to the changegroup\n** were themselves changesets, the output is a changeset. Or, if the\n** inputs were patchsets, the output is also a patchset.\n**\n** As with the output of the sqlite3session_changeset() and\n** sqlite3session_patchset() functions, all changes related to a single\n** table are grouped together in the output of this function. Tables appear\n** in the same order as for the very first changeset added to the changegroup.\n** If the second or subsequent changesets added to the changegroup contain\n** changes for tables that do not appear in the first changeset, they are\n** appended onto the end of the output changeset, again in the order in\n** which they are first encountered.\n**\n** If an error occurs, an SQLite error code is returned and the output\n** variables (*pnData) and (*ppData) are set to 0. Otherwise, SQLITE_OK\n** is returned and the output variables are set to the size of and a\n** pointer to the output buffer, respectively. In this case it is the\n** responsibility of the caller to eventually free the buffer using a\n** call to sqlite3_free().\n*/\nSQLITE_API int sqlite3changegroup_output(\n sqlite3_changegroup*,\n int *pnData, /* OUT: Size of output buffer in bytes */\n void **ppData /* OUT: Pointer to output buffer */\n);\n\n/*\n** CAPI3REF: Delete A Changegroup Object\n** DESTRUCTOR: sqlite3_changegroup\n*/\nSQLITE_API void sqlite3changegroup_delete(sqlite3_changegroup*);\n\n/*\n** CAPI3REF: Apply A Changeset To A Database\n**\n** Apply a changeset or patchset to a database. These functions attempt to\n** update the \"main\" database attached to handle db with the changes found in\n** the changeset passed via the second and third arguments.\n**\n** The fourth argument (xFilter) passed to these functions is the \"filter\n** callback\". If it is not NULL, then for each table affected by at least one\n** change in the changeset, the filter callback is invoked with\n** the table name as the second argument, and a copy of the context pointer\n** passed as the sixth argument as the first. If the \"filter callback\"\n** returns zero, then no attempt is made to apply any changes to the table.\n** Otherwise, if the return value is non-zero or the xFilter argument to\n** is NULL, all changes related to the table are attempted.\n**\n** For each table that is not excluded by the filter callback, this function\n** tests that the target database contains a compatible table. A table is\n** considered compatible if all of the following are true:\n**\n**
    \n**
  • The table has the same name as the name recorded in the\n** changeset, and\n**
  • The table has at least as many columns as recorded in the\n** changeset, and\n**
  • The table has primary key columns in the same position as\n** recorded in the changeset.\n**
\n**\n** If there is no compatible table, it is not an error, but none of the\n** changes associated with the table are applied. A warning message is issued\n** via the sqlite3_log() mechanism with the error code SQLITE_SCHEMA. At most\n** one such warning is issued for each table in the changeset.\n**\n** For each change for which there is a compatible table, an attempt is made\n** to modify the table contents according to the UPDATE, INSERT or DELETE\n** change. If a change cannot be applied cleanly, the conflict handler\n** function passed as the fifth argument to sqlite3changeset_apply() may be\n** invoked. A description of exactly when the conflict handler is invoked for\n** each type of change is below.\n**\n** Unlike the xFilter argument, xConflict may not be passed NULL. The results\n** of passing anything other than a valid function pointer as the xConflict\n** argument are undefined.\n**\n** Each time the conflict handler function is invoked, it must return one\n** of [SQLITE_CHANGESET_OMIT], [SQLITE_CHANGESET_ABORT] or\n** [SQLITE_CHANGESET_REPLACE]. SQLITE_CHANGESET_REPLACE may only be returned\n** if the second argument passed to the conflict handler is either\n** SQLITE_CHANGESET_DATA or SQLITE_CHANGESET_CONFLICT. If the conflict-handler\n** returns an illegal value, any changes already made are rolled back and\n** the call to sqlite3changeset_apply() returns SQLITE_MISUSE. Different\n** actions are taken by sqlite3changeset_apply() depending on the value\n** returned by each invocation of the conflict-handler function. Refer to\n** the documentation for the three\n** [SQLITE_CHANGESET_OMIT|available return values] for details.\n**\n**
\n**
DELETE Changes
\n** For each DELETE change, the function checks if the target database\n** contains a row with the same primary key value (or values) as the\n** original row values stored in the changeset. If it does, and the values\n** stored in all non-primary key columns also match the values stored in\n** the changeset the row is deleted from the target database.\n**\n** If a row with matching primary key values is found, but one or more of\n** the non-primary key fields contains a value different from the original\n** row value stored in the changeset, the conflict-handler function is\n** invoked with [SQLITE_CHANGESET_DATA] as the second argument. If the\n** database table has more columns than are recorded in the changeset,\n** only the values of those non-primary key fields are compared against\n** the current database contents - any trailing database table columns\n** are ignored.\n**\n** If no row with matching primary key values is found in the database,\n** the conflict-handler function is invoked with [SQLITE_CHANGESET_NOTFOUND]\n** passed as the second argument.\n**\n** If the DELETE operation is attempted, but SQLite returns SQLITE_CONSTRAINT\n** (which can only happen if a foreign key constraint is violated), the\n** conflict-handler function is invoked with [SQLITE_CHANGESET_CONSTRAINT]\n** passed as the second argument. This includes the case where the DELETE\n** operation is attempted because an earlier call to the conflict handler\n** function returned [SQLITE_CHANGESET_REPLACE].\n**\n**
INSERT Changes
\n** For each INSERT change, an attempt is made to insert the new row into\n** the database. If the changeset row contains fewer fields than the\n** database table, the trailing fields are populated with their default\n** values.\n**\n** If the attempt to insert the row fails because the database already\n** contains a row with the same primary key values, the conflict handler\n** function is invoked with the second argument set to\n** [SQLITE_CHANGESET_CONFLICT].\n**\n** If the attempt to insert the row fails because of some other constraint\n** violation (e.g. NOT NULL or UNIQUE), the conflict handler function is\n** invoked with the second argument set to [SQLITE_CHANGESET_CONSTRAINT].\n** This includes the case where the INSERT operation is re-attempted because\n** an earlier call to the conflict handler function returned\n** [SQLITE_CHANGESET_REPLACE].\n**\n**
UPDATE Changes
\n** For each UPDATE change, the function checks if the target database\n** contains a row with the same primary key value (or values) as the\n** original row values stored in the changeset. If it does, and the values\n** stored in all modified non-primary key columns also match the values\n** stored in the changeset the row is updated within the target database.\n**\n** If a row with matching primary key values is found, but one or more of\n** the modified non-primary key fields contains a value different from an\n** original row value stored in the changeset, the conflict-handler function\n** is invoked with [SQLITE_CHANGESET_DATA] as the second argument. Since\n** UPDATE changes only contain values for non-primary key fields that are\n** to be modified, only those fields need to match the original values to\n** avoid the SQLITE_CHANGESET_DATA conflict-handler callback.\n**\n** If no row with matching primary key values is found in the database,\n** the conflict-handler function is invoked with [SQLITE_CHANGESET_NOTFOUND]\n** passed as the second argument.\n**\n** If the UPDATE operation is attempted, but SQLite returns\n** SQLITE_CONSTRAINT, the conflict-handler function is invoked with\n** [SQLITE_CHANGESET_CONSTRAINT] passed as the second argument.\n** This includes the case where the UPDATE operation is attempted after\n** an earlier call to the conflict handler function returned\n** [SQLITE_CHANGESET_REPLACE].\n**
\n**\n** It is safe to execute SQL statements, including those that write to the\n** table that the callback related to, from within the xConflict callback.\n** This can be used to further customize the application's conflict\n** resolution strategy.\n**\n** All changes made by these functions are enclosed in a savepoint transaction.\n** If any other error (aside from a constraint failure when attempting to\n** write to the target database) occurs, then the savepoint transaction is\n** rolled back, restoring the target database to its original state, and an\n** SQLite error code returned.\n**\n** If the output parameters (ppRebase) and (pnRebase) are non-NULL and\n** the input is a changeset (not a patchset), then sqlite3changeset_apply_v2()\n** may set (*ppRebase) to point to a \"rebase\" that may be used with the\n** sqlite3_rebaser APIs buffer before returning. In this case (*pnRebase)\n** is set to the size of the buffer in bytes. It is the responsibility of the\n** caller to eventually free any such buffer using sqlite3_free(). The buffer\n** is only allocated and populated if one or more conflicts were encountered\n** while applying the patchset. See comments surrounding the sqlite3_rebaser\n** APIs for further details.\n**\n** The behavior of sqlite3changeset_apply_v2() and its streaming equivalent\n** may be modified by passing a combination of\n** [SQLITE_CHANGESETAPPLY_NOSAVEPOINT | supported flags] as the 9th parameter.\n**\n** Note that the sqlite3changeset_apply_v2() API is still experimental\n** and therefore subject to change.\n*/\nSQLITE_API int sqlite3changeset_apply(\n sqlite3 *db, /* Apply change to \"main\" db of this handle */\n int nChangeset, /* Size of changeset in bytes */\n void *pChangeset, /* Changeset blob */\n int(*xFilter)(\n void *pCtx, /* Copy of sixth arg to _apply() */\n const char *zTab /* Table name */\n ),\n int(*xConflict)(\n void *pCtx, /* Copy of sixth arg to _apply() */\n int eConflict, /* DATA, MISSING, CONFLICT, CONSTRAINT */\n sqlite3_changeset_iter *p /* Handle describing change and conflict */\n ),\n void *pCtx /* First argument passed to xConflict */\n);\nSQLITE_API int sqlite3changeset_apply_v2(\n sqlite3 *db, /* Apply change to \"main\" db of this handle */\n int nChangeset, /* Size of changeset in bytes */\n void *pChangeset, /* Changeset blob */\n int(*xFilter)(\n void *pCtx, /* Copy of sixth arg to _apply() */\n const char *zTab /* Table name */\n ),\n int(*xConflict)(\n void *pCtx, /* Copy of sixth arg to _apply() */\n int eConflict, /* DATA, MISSING, CONFLICT, CONSTRAINT */\n sqlite3_changeset_iter *p /* Handle describing change and conflict */\n ),\n void *pCtx, /* First argument passed to xConflict */\n void **ppRebase, int *pnRebase, /* OUT: Rebase data */\n int flags /* SESSION_CHANGESETAPPLY_* flags */\n);\n\n/*\n** CAPI3REF: Flags for sqlite3changeset_apply_v2\n**\n** The following flags may passed via the 9th parameter to\n** [sqlite3changeset_apply_v2] and [sqlite3changeset_apply_v2_strm]:\n**\n**
\n**
SQLITE_CHANGESETAPPLY_NOSAVEPOINT
\n** Usually, the sessions module encloses all operations performed by\n** a single call to apply_v2() or apply_v2_strm() in a [SAVEPOINT]. The\n** SAVEPOINT is committed if the changeset or patchset is successfully\n** applied, or rolled back if an error occurs. Specifying this flag\n** causes the sessions module to omit this savepoint. In this case, if the\n** caller has an open transaction or savepoint when apply_v2() is called,\n** it may revert the partially applied changeset by rolling it back.\n**\n**
SQLITE_CHANGESETAPPLY_INVERT
\n** Invert the changeset before applying it. This is equivalent to inverting\n** a changeset using sqlite3changeset_invert() before applying it. It is\n** an error to specify this flag with a patchset.\n**\n**
SQLITE_CHANGESETAPPLY_IGNORENOOP
\n** Do not invoke the conflict handler callback for any changes that\n** would not actually modify the database even if they were applied.\n** Specifically, this means that the conflict handler is not invoked\n** for:\n**
    \n**
  • a delete change if the row being deleted cannot be found,\n**
  • an update change if the modified fields are already set to\n** their new values in the conflicting row, or\n**
  • an insert change if all fields of the conflicting row match\n** the row being inserted.\n**
\n**\n**
SQLITE_CHANGESETAPPLY_FKNOACTION
\n** If this flag it set, then all foreign key constraints in the target\n** database behave as if they were declared with \"ON UPDATE NO ACTION ON\n** DELETE NO ACTION\", even if they are actually CASCADE, RESTRICT, SET NULL\n** or SET DEFAULT.\n*/\n#define SQLITE_CHANGESETAPPLY_NOSAVEPOINT 0x0001\n#define SQLITE_CHANGESETAPPLY_INVERT 0x0002\n#define SQLITE_CHANGESETAPPLY_IGNORENOOP 0x0004\n#define SQLITE_CHANGESETAPPLY_FKNOACTION 0x0008\n\n/*\n** CAPI3REF: Constants Passed To The Conflict Handler\n**\n** Values that may be passed as the second argument to a conflict-handler.\n**\n**
\n**
SQLITE_CHANGESET_DATA
\n** The conflict handler is invoked with CHANGESET_DATA as the second argument\n** when processing a DELETE or UPDATE change if a row with the required\n** PRIMARY KEY fields is present in the database, but one or more other\n** (non primary-key) fields modified by the update do not contain the\n** expected \"before\" values.\n**\n** The conflicting row, in this case, is the database row with the matching\n** primary key.\n**\n**
SQLITE_CHANGESET_NOTFOUND
\n** The conflict handler is invoked with CHANGESET_NOTFOUND as the second\n** argument when processing a DELETE or UPDATE change if a row with the\n** required PRIMARY KEY fields is not present in the database.\n**\n** There is no conflicting row in this case. The results of invoking the\n** sqlite3changeset_conflict() API are undefined.\n**\n**
SQLITE_CHANGESET_CONFLICT
\n** CHANGESET_CONFLICT is passed as the second argument to the conflict\n** handler while processing an INSERT change if the operation would result\n** in duplicate primary key values.\n**\n** The conflicting row in this case is the database row with the matching\n** primary key.\n**\n**
SQLITE_CHANGESET_FOREIGN_KEY
\n** If foreign key handling is enabled, and applying a changeset leaves the\n** database in a state containing foreign key violations, the conflict\n** handler is invoked with CHANGESET_FOREIGN_KEY as the second argument\n** exactly once before the changeset is committed. If the conflict handler\n** returns CHANGESET_OMIT, the changes, including those that caused the\n** foreign key constraint violation, are committed. Or, if it returns\n** CHANGESET_ABORT, the changeset is rolled back.\n**\n** No current or conflicting row information is provided. The only function\n** it is possible to call on the supplied sqlite3_changeset_iter handle\n** is sqlite3changeset_fk_conflicts().\n**\n**
SQLITE_CHANGESET_CONSTRAINT
\n** If any other constraint violation occurs while applying a change (i.e.\n** a UNIQUE, CHECK or NOT NULL constraint), the conflict handler is\n** invoked with CHANGESET_CONSTRAINT as the second argument.\n**\n** There is no conflicting row in this case. The results of invoking the\n** sqlite3changeset_conflict() API are undefined.\n**\n**
\n*/\n#define SQLITE_CHANGESET_DATA 1\n#define SQLITE_CHANGESET_NOTFOUND 2\n#define SQLITE_CHANGESET_CONFLICT 3\n#define SQLITE_CHANGESET_CONSTRAINT 4\n#define SQLITE_CHANGESET_FOREIGN_KEY 5\n\n/*\n** CAPI3REF: Constants Returned By The Conflict Handler\n**\n** A conflict handler callback must return one of the following three values.\n**\n**
\n**
SQLITE_CHANGESET_OMIT
\n** If a conflict handler returns this value no special action is taken. The\n** change that caused the conflict is not applied. The session module\n** continues to the next change in the changeset.\n**\n**
SQLITE_CHANGESET_REPLACE
\n** This value may only be returned if the second argument to the conflict\n** handler was SQLITE_CHANGESET_DATA or SQLITE_CHANGESET_CONFLICT. If this\n** is not the case, any changes applied so far are rolled back and the\n** call to sqlite3changeset_apply() returns SQLITE_MISUSE.\n**\n** If CHANGESET_REPLACE is returned by an SQLITE_CHANGESET_DATA conflict\n** handler, then the conflicting row is either updated or deleted, depending\n** on the type of change.\n**\n** If CHANGESET_REPLACE is returned by an SQLITE_CHANGESET_CONFLICT conflict\n** handler, then the conflicting row is removed from the database and a\n** second attempt to apply the change is made. If this second attempt fails,\n** the original row is restored to the database before continuing.\n**\n**
SQLITE_CHANGESET_ABORT
\n** If this value is returned, any changes applied so far are rolled back\n** and the call to sqlite3changeset_apply() returns SQLITE_ABORT.\n**
\n*/\n#define SQLITE_CHANGESET_OMIT 0\n#define SQLITE_CHANGESET_REPLACE 1\n#define SQLITE_CHANGESET_ABORT 2\n\n/*\n** CAPI3REF: Rebasing changesets\n** EXPERIMENTAL\n**\n** Suppose there is a site hosting a database in state S0. And that\n** modifications are made that move that database to state S1 and a\n** changeset recorded (the \"local\" changeset). Then, a changeset based\n** on S0 is received from another site (the \"remote\" changeset) and\n** applied to the database. The database is then in state\n** (S1+\"remote\"), where the exact state depends on any conflict\n** resolution decisions (OMIT or REPLACE) made while applying \"remote\".\n** Rebasing a changeset is to update it to take those conflict\n** resolution decisions into account, so that the same conflicts\n** do not have to be resolved elsewhere in the network.\n**\n** For example, if both the local and remote changesets contain an\n** INSERT of the same key on \"CREATE TABLE t1(a PRIMARY KEY, b)\":\n**\n** local: INSERT INTO t1 VALUES(1, 'v1');\n** remote: INSERT INTO t1 VALUES(1, 'v2');\n**\n** and the conflict resolution is REPLACE, then the INSERT change is\n** removed from the local changeset (it was overridden). Or, if the\n** conflict resolution was \"OMIT\", then the local changeset is modified\n** to instead contain:\n**\n** UPDATE t1 SET b = 'v2' WHERE a=1;\n**\n** Changes within the local changeset are rebased as follows:\n**\n**
\n**
Local INSERT
\n** This may only conflict with a remote INSERT. If the conflict\n** resolution was OMIT, then add an UPDATE change to the rebased\n** changeset. Or, if the conflict resolution was REPLACE, add\n** nothing to the rebased changeset.\n**\n**
Local DELETE
\n** This may conflict with a remote UPDATE or DELETE. In both cases the\n** only possible resolution is OMIT. If the remote operation was a\n** DELETE, then add no change to the rebased changeset. If the remote\n** operation was an UPDATE, then the old.* fields of change are updated\n** to reflect the new.* values in the UPDATE.\n**\n**
Local UPDATE
\n** This may conflict with a remote UPDATE or DELETE. If it conflicts\n** with a DELETE, and the conflict resolution was OMIT, then the update\n** is changed into an INSERT. Any undefined values in the new.* record\n** from the update change are filled in using the old.* values from\n** the conflicting DELETE. Or, if the conflict resolution was REPLACE,\n** the UPDATE change is simply omitted from the rebased changeset.\n**\n** If conflict is with a remote UPDATE and the resolution is OMIT, then\n** the old.* values are rebased using the new.* values in the remote\n** change. Or, if the resolution is REPLACE, then the change is copied\n** into the rebased changeset with updates to columns also updated by\n** the conflicting remote UPDATE removed. If this means no columns would\n** be updated, the change is omitted.\n**
\n**\n** A local change may be rebased against multiple remote changes\n** simultaneously. If a single key is modified by multiple remote\n** changesets, they are combined as follows before the local changeset\n** is rebased:\n**\n**
    \n**
  • If there has been one or more REPLACE resolutions on a\n** key, it is rebased according to a REPLACE.\n**\n**
  • If there have been no REPLACE resolutions on a key, then\n** the local changeset is rebased according to the most recent\n** of the OMIT resolutions.\n**
\n**\n** Note that conflict resolutions from multiple remote changesets are\n** combined on a per-field basis, not per-row. This means that in the\n** case of multiple remote UPDATE operations, some fields of a single\n** local change may be rebased for REPLACE while others are rebased for\n** OMIT.\n**\n** In order to rebase a local changeset, the remote changeset must first\n** be applied to the local database using sqlite3changeset_apply_v2() and\n** the buffer of rebase information captured. Then:\n**\n**
    \n**
  1. An sqlite3_rebaser object is created by calling\n** sqlite3rebaser_create().\n**
  2. The new object is configured with the rebase buffer obtained from\n** sqlite3changeset_apply_v2() by calling sqlite3rebaser_configure().\n** If the local changeset is to be rebased against multiple remote\n** changesets, then sqlite3rebaser_configure() should be called\n** multiple times, in the same order that the multiple\n** sqlite3changeset_apply_v2() calls were made.\n**
  3. Each local changeset is rebased by calling sqlite3rebaser_rebase().\n**
  4. The sqlite3_rebaser object is deleted by calling\n** sqlite3rebaser_delete().\n**
\n*/\ntypedef struct sqlite3_rebaser sqlite3_rebaser;\n\n/*\n** CAPI3REF: Create a changeset rebaser object.\n** EXPERIMENTAL\n**\n** Allocate a new changeset rebaser object. If successful, set (*ppNew) to\n** point to the new object and return SQLITE_OK. Otherwise, if an error\n** occurs, return an SQLite error code (e.g. SQLITE_NOMEM) and set (*ppNew)\n** to NULL.\n*/\nSQLITE_API int sqlite3rebaser_create(sqlite3_rebaser **ppNew);\n\n/*\n** CAPI3REF: Configure a changeset rebaser object.\n** EXPERIMENTAL\n**\n** Configure the changeset rebaser object to rebase changesets according\n** to the conflict resolutions described by buffer pRebase (size nRebase\n** bytes), which must have been obtained from a previous call to\n** sqlite3changeset_apply_v2().\n*/\nSQLITE_API int sqlite3rebaser_configure(\n sqlite3_rebaser*,\n int nRebase, const void *pRebase\n);\n\n/*\n** CAPI3REF: Rebase a changeset\n** EXPERIMENTAL\n**\n** Argument pIn must point to a buffer containing a changeset nIn bytes\n** in size. This function allocates and populates a buffer with a copy\n** of the changeset rebased according to the configuration of the\n** rebaser object passed as the first argument. If successful, (*ppOut)\n** is set to point to the new buffer containing the rebased changeset and\n** (*pnOut) to its size in bytes and SQLITE_OK returned. It is the\n** responsibility of the caller to eventually free the new buffer using\n** sqlite3_free(). Otherwise, if an error occurs, (*ppOut) and (*pnOut)\n** are set to zero and an SQLite error code returned.\n*/\nSQLITE_API int sqlite3rebaser_rebase(\n sqlite3_rebaser*,\n int nIn, const void *pIn,\n int *pnOut, void **ppOut\n);\n\n/*\n** CAPI3REF: Delete a changeset rebaser object.\n** EXPERIMENTAL\n**\n** Delete the changeset rebaser object and all associated resources. There\n** should be one call to this function for each successful invocation\n** of sqlite3rebaser_create().\n*/\nSQLITE_API void sqlite3rebaser_delete(sqlite3_rebaser *p);\n\n/*\n** CAPI3REF: Streaming Versions of API functions.\n**\n** The six streaming API xxx_strm() functions serve similar purposes to the\n** corresponding non-streaming API functions:\n**\n** \n** \n**
Streaming functionNon-streaming equivalent
sqlite3changeset_apply_strm[sqlite3changeset_apply]\n**
sqlite3changeset_apply_strm_v2[sqlite3changeset_apply_v2]\n**
sqlite3changeset_concat_strm[sqlite3changeset_concat]\n**
sqlite3changeset_invert_strm[sqlite3changeset_invert]\n**
sqlite3changeset_start_strm[sqlite3changeset_start]\n**
sqlite3session_changeset_strm[sqlite3session_changeset]\n**
sqlite3session_patchset_strm[sqlite3session_patchset]\n**
\n**\n** Non-streaming functions that accept changesets (or patchsets) as input\n** require that the entire changeset be stored in a single buffer in memory.\n** Similarly, those that return a changeset or patchset do so by returning\n** a pointer to a single large buffer allocated using sqlite3_malloc().\n** Normally this is convenient. However, if an application running in a\n** low-memory environment is required to handle very large changesets, the\n** large contiguous memory allocations required can become onerous.\n**\n** In order to avoid this problem, instead of a single large buffer, input\n** is passed to a streaming API functions by way of a callback function that\n** the sessions module invokes to incrementally request input data as it is\n** required. In all cases, a pair of API function parameters such as\n**\n** 
\n**        int nChangeset,\n**        void *pChangeset,\n**
\n**\n** Is replaced by:\n**\n** 
\n**        int (*xInput)(void *pIn, void *pData, int *pnData),\n**        void *pIn,\n**
\n**\n** Each time the xInput callback is invoked by the sessions module, the first\n** argument passed is a copy of the supplied pIn context pointer. The second\n** argument, pData, points to a buffer (*pnData) bytes in size. Assuming no\n** error occurs the xInput method should copy up to (*pnData) bytes of data\n** into the buffer and set (*pnData) to the actual number of bytes copied\n** before returning SQLITE_OK. If the input is completely exhausted, (*pnData)\n** should be set to zero to indicate this. Or, if an error occurs, an SQLite\n** error code should be returned. In all cases, if an xInput callback returns\n** an error, all processing is abandoned and the streaming API function\n** returns a copy of the error code to the caller.\n**\n** In the case of sqlite3changeset_start_strm(), the xInput callback may be\n** invoked by the sessions module at any point during the lifetime of the\n** iterator. If such an xInput callback returns an error, the iterator enters\n** an error state, whereby all subsequent calls to iterator functions\n** immediately fail with the same error code as returned by xInput.\n**\n** Similarly, streaming API functions that return changesets (or patchsets)\n** return them in chunks by way of a callback function instead of via a\n** pointer to a single large buffer. In this case, a pair of parameters such\n** as:\n**\n** 
\n**        int *pnChangeset,\n**        void **ppChangeset,\n**
\n**\n** Is replaced by:\n**\n** 
\n**        int (*xOutput)(void *pOut, const void *pData, int nData),\n**        void *pOut\n**
\n**\n** The xOutput callback is invoked zero or more times to return data to\n** the application. The first parameter passed to each call is a copy of the\n** pOut pointer supplied by the application. The second parameter, pData,\n** points to a buffer nData bytes in size containing the chunk of output\n** data being returned. If the xOutput callback successfully processes the\n** supplied data, it should return SQLITE_OK to indicate success. Otherwise,\n** it should return some other SQLite error code. In this case processing\n** is immediately abandoned and the streaming API function returns a copy\n** of the xOutput error code to the application.\n**\n** The sessions module never invokes an xOutput callback with the third\n** parameter set to a value less than or equal to zero. Other than this,\n** no guarantees are made as to the size of the chunks of data returned.\n*/\nSQLITE_API int sqlite3changeset_apply_strm(\n sqlite3 *db, /* Apply change to \"main\" db of this handle */\n int (*xInput)(void *pIn, void *pData, int *pnData), /* Input function */\n void *pIn, /* First arg for xInput */\n int(*xFilter)(\n void *pCtx, /* Copy of sixth arg to _apply() */\n const char *zTab /* Table name */\n ),\n int(*xConflict)(\n void *pCtx, /* Copy of sixth arg to _apply() */\n int eConflict, /* DATA, MISSING, CONFLICT, CONSTRAINT */\n sqlite3_changeset_iter *p /* Handle describing change and conflict */\n ),\n void *pCtx /* First argument passed to xConflict */\n);\nSQLITE_API int sqlite3changeset_apply_v2_strm(\n sqlite3 *db, /* Apply change to \"main\" db of this handle */\n int (*xInput)(void *pIn, void *pData, int *pnData), /* Input function */\n void *pIn, /* First arg for xInput */\n int(*xFilter)(\n void *pCtx, /* Copy of sixth arg to _apply() */\n const char *zTab /* Table name */\n ),\n int(*xConflict)(\n void *pCtx, /* Copy of sixth arg to _apply() */\n int eConflict, /* DATA, MISSING, CONFLICT, CONSTRAINT */\n sqlite3_changeset_iter *p /* Handle describing change and conflict */\n ),\n void *pCtx, /* First argument passed to xConflict */\n void **ppRebase, int *pnRebase,\n int flags\n);\nSQLITE_API int sqlite3changeset_concat_strm(\n int (*xInputA)(void *pIn, void *pData, int *pnData),\n void *pInA,\n int (*xInputB)(void *pIn, void *pData, int *pnData),\n void *pInB,\n int (*xOutput)(void *pOut, const void *pData, int nData),\n void *pOut\n);\nSQLITE_API int sqlite3changeset_invert_strm(\n int (*xInput)(void *pIn, void *pData, int *pnData),\n void *pIn,\n int (*xOutput)(void *pOut, const void *pData, int nData),\n void *pOut\n);\nSQLITE_API int sqlite3changeset_start_strm(\n sqlite3_changeset_iter **pp,\n int (*xInput)(void *pIn, void *pData, int *pnData),\n void *pIn\n);\nSQLITE_API int sqlite3changeset_start_v2_strm(\n sqlite3_changeset_iter **pp,\n int (*xInput)(void *pIn, void *pData, int *pnData),\n void *pIn,\n int flags\n);\nSQLITE_API int sqlite3session_changeset_strm(\n sqlite3_session *pSession,\n int (*xOutput)(void *pOut, const void *pData, int nData),\n void *pOut\n);\nSQLITE_API int sqlite3session_patchset_strm(\n sqlite3_session *pSession,\n int (*xOutput)(void *pOut, const void *pData, int nData),\n void *pOut\n);\nSQLITE_API int sqlite3changegroup_add_strm(sqlite3_changegroup*,\n int (*xInput)(void *pIn, void *pData, int *pnData),\n void *pIn\n);\nSQLITE_API int sqlite3changegroup_output_strm(sqlite3_changegroup*,\n int (*xOutput)(void *pOut, const void *pData, int nData),\n void *pOut\n);\nSQLITE_API int sqlite3rebaser_rebase_strm(\n sqlite3_rebaser *pRebaser,\n int (*xInput)(void *pIn, void *pData, int *pnData),\n void *pIn,\n int (*xOutput)(void *pOut, const void *pData, int nData),\n void *pOut\n);\n\n/*\n** CAPI3REF: Configure global parameters\n**\n** The sqlite3session_config() interface is used to make global configuration\n** changes to the sessions module in order to tune it to the specific needs\n** of the application.\n**\n** The sqlite3session_config() interface is not threadsafe. If it is invoked\n** while any other thread is inside any other sessions method then the\n** results are undefined. Furthermore, if it is invoked after any sessions\n** related objects have been created, the results are also undefined.\n**\n** The first argument to the sqlite3session_config() function must be one\n** of the SQLITE_SESSION_CONFIG_XXX constants defined below. The\n** interpretation of the (void*) value passed as the second parameter and\n** the effect of calling this function depends on the value of the first\n** parameter.\n**\n**
\n**
SQLITE_SESSION_CONFIG_STRMSIZE
\n** By default, the sessions module streaming interfaces attempt to input\n** and output data in approximately 1 KiB chunks. This operand may be used\n** to set and query the value of this configuration setting. The pointer\n** passed as the second argument must point to a value of type (int).\n** If this value is greater than 0, it is used as the new streaming data\n** chunk size for both input and output. Before returning, the (int) value\n** pointed to by pArg is set to the final value of the streaming interface\n** chunk size.\n**
\n**\n** This function returns SQLITE_OK if successful, or an SQLite error code\n** otherwise.\n*/\nSQLITE_API int sqlite3session_config(int op, void *pArg);\n\n/*\n** CAPI3REF: Values for sqlite3session_config().\n*/\n#define SQLITE_SESSION_CONFIG_STRMSIZE 1\n\n/*\n** Make sure we can call this stuff from C++.\n*/\n#ifdef __cplusplus\n}\n#endif\n\n#endif /* !defined(__SQLITESESSION_H_) && defined(SQLITE_ENABLE_SESSION) */\n\n/******** End of sqlite3session.h *********/\n/******** Begin file fts5.h *********/\n/*\n** 2014 May 31\n**\n** The author disclaims copyright to this source code. In place of\n** a legal notice, here is a blessing:\n**\n** May you do good and not evil.\n** May you find forgiveness for yourself and forgive others.\n** May you share freely, never taking more than you give.\n**\n******************************************************************************\n**\n** Interfaces to extend FTS5. Using the interfaces defined in this file,\n** FTS5 may be extended with:\n**\n** * custom tokenizers, and\n** * custom auxiliary functions.\n*/\n\n\n#ifndef _FTS5_H\n#define _FTS5_H\n\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n/*************************************************************************\n** CUSTOM AUXILIARY FUNCTIONS\n**\n** Virtual table implementations may overload SQL functions by implementing\n** the sqlite3_module.xFindFunction() method.\n*/\n\ntypedef struct Fts5ExtensionApi Fts5ExtensionApi;\ntypedef struct Fts5Context Fts5Context;\ntypedef struct Fts5PhraseIter Fts5PhraseIter;\n\ntypedef void (*fts5_extension_function)(\n const Fts5ExtensionApi *pApi, /* API offered by current FTS version */\n Fts5Context *pFts, /* First arg to pass to pApi functions */\n sqlite3_context *pCtx, /* Context for returning result/error */\n int nVal, /* Number of values in apVal[] array */\n sqlite3_value **apVal /* Array of trailing arguments */\n);\n\nstruct Fts5PhraseIter {\n const unsigned char *a;\n const unsigned char *b;\n};\n\n/*\n** EXTENSION API FUNCTIONS\n**\n** xUserData(pFts):\n** Return a copy of the pUserData pointer passed to the xCreateFunction()\n** API when the extension function was registered.\n**\n** xColumnTotalSize(pFts, iCol, pnToken):\n** If parameter iCol is less than zero, set output variable *pnToken\n** to the total number of tokens in the FTS5 table. Or, if iCol is\n** non-negative but less than the number of columns in the table, return\n** the total number of tokens in column iCol, considering all rows in\n** the FTS5 table.\n**\n** If parameter iCol is greater than or equal to the number of columns\n** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.\n** an OOM condition or IO error), an appropriate SQLite error code is\n** returned.\n**\n** xColumnCount(pFts):\n** Return the number of columns in the table.\n**\n** xColumnSize(pFts, iCol, pnToken):\n** If parameter iCol is less than zero, set output variable *pnToken\n** to the total number of tokens in the current row. Or, if iCol is\n** non-negative but less than the number of columns in the table, set\n** *pnToken to the number of tokens in column iCol of the current row.\n**\n** If parameter iCol is greater than or equal to the number of columns\n** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.\n** an OOM condition or IO error), an appropriate SQLite error code is\n** returned.\n**\n** This function may be quite inefficient if used with an FTS5 table\n** created with the \"columnsize=0\" option.\n**\n** xColumnText:\n** If parameter iCol is less than zero, or greater than or equal to the\n** number of columns in the table, SQLITE_RANGE is returned.\n**\n** Otherwise, this function attempts to retrieve the text of column iCol of\n** the current document. If successful, (*pz) is set to point to a buffer\n** containing the text in utf-8 encoding, (*pn) is set to the size in bytes\n** (not characters) of the buffer and SQLITE_OK is returned. Otherwise,\n** if an error occurs, an SQLite error code is returned and the final values\n** of (*pz) and (*pn) are undefined.\n**\n** xPhraseCount:\n** Returns the number of phrases in the current query expression.\n**\n** xPhraseSize:\n** If parameter iCol is less than zero, or greater than or equal to the\n** number of phrases in the current query, as returned by xPhraseCount,\n** 0 is returned. Otherwise, this function returns the number of tokens in\n** phrase iPhrase of the query. Phrases are numbered starting from zero.\n**\n** xInstCount:\n** Set *pnInst to the total number of occurrences of all phrases within\n** the query within the current row. Return SQLITE_OK if successful, or\n** an error code (i.e. SQLITE_NOMEM) if an error occurs.\n**\n** This API can be quite slow if used with an FTS5 table created with the\n** \"detail=none\" or \"detail=column\" option. If the FTS5 table is created\n** with either \"detail=none\" or \"detail=column\" and \"content=\" option\n** (i.e. if it is a contentless table), then this API always returns 0.\n**\n** xInst:\n** Query for the details of phrase match iIdx within the current row.\n** Phrase matches are numbered starting from zero, so the iIdx argument\n** should be greater than or equal to zero and smaller than the value\n** output by xInstCount(). If iIdx is less than zero or greater than\n** or equal to the value returned by xInstCount(), SQLITE_RANGE is returned.\n**\n** Otherwise, output parameter *piPhrase is set to the phrase number, *piCol\n** to the column in which it occurs and *piOff the token offset of the\n** first token of the phrase. SQLITE_OK is returned if successful, or an\n** error code (i.e. SQLITE_NOMEM) if an error occurs.\n**\n** This API can be quite slow if used with an FTS5 table created with the\n** \"detail=none\" or \"detail=column\" option.\n**\n** xRowid:\n** Returns the rowid of the current row.\n**\n** xTokenize:\n** Tokenize text using the tokenizer belonging to the FTS5 table.\n**\n** xQueryPhrase(pFts5, iPhrase, pUserData, xCallback):\n** This API function is used to query the FTS table for phrase iPhrase\n** of the current query. Specifically, a query equivalent to:\n**\n** ... FROM ftstable WHERE ftstable MATCH $p ORDER BY rowid\n**\n** with $p set to a phrase equivalent to the phrase iPhrase of the\n** current query is executed. Any column filter that applies to\n** phrase iPhrase of the current query is included in $p. For each\n** row visited, the callback function passed as the fourth argument\n** is invoked. The context and API objects passed to the callback\n** function may be used to access the properties of each matched row.\n** Invoking Api.xUserData() returns a copy of the pointer passed as\n** the third argument to pUserData.\n**\n** If parameter iPhrase is less than zero, or greater than or equal to\n** the number of phrases in the query, as returned by xPhraseCount(),\n** this function returns SQLITE_RANGE.\n**\n** If the callback function returns any value other than SQLITE_OK, the\n** query is abandoned and the xQueryPhrase function returns immediately.\n** If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK.\n** Otherwise, the error code is propagated upwards.\n**\n** If the query runs to completion without incident, SQLITE_OK is returned.\n** Or, if some error occurs before the query completes or is aborted by\n** the callback, an SQLite error code is returned.\n**\n**\n** xSetAuxdata(pFts5, pAux, xDelete)\n**\n** Save the pointer passed as the second argument as the extension function's\n** \"auxiliary data\". The pointer may then be retrieved by the current or any\n** future invocation of the same fts5 extension function made as part of\n** the same MATCH query using the xGetAuxdata() API.\n**\n** Each extension function is allocated a single auxiliary data slot for\n** each FTS query (MATCH expression). If the extension function is invoked\n** more than once for a single FTS query, then all invocations share a\n** single auxiliary data context.\n**\n** If there is already an auxiliary data pointer when this function is\n** invoked, then it is replaced by the new pointer. If an xDelete callback\n** was specified along with the original pointer, it is invoked at this\n** point.\n**\n** The xDelete callback, if one is specified, is also invoked on the\n** auxiliary data pointer after the FTS5 query has finished.\n**\n** If an error (e.g. an OOM condition) occurs within this function,\n** the auxiliary data is set to NULL and an error code returned. If the\n** xDelete parameter was not NULL, it is invoked on the auxiliary data\n** pointer before returning.\n**\n**\n** xGetAuxdata(pFts5, bClear)\n**\n** Returns the current auxiliary data pointer for the fts5 extension\n** function. See the xSetAuxdata() method for details.\n**\n** If the bClear argument is non-zero, then the auxiliary data is cleared\n** (set to NULL) before this function returns. In this case the xDelete,\n** if any, is not invoked.\n**\n**\n** xRowCount(pFts5, pnRow)\n**\n** This function is used to retrieve the total number of rows in the table.\n** In other words, the same value that would be returned by:\n**\n** SELECT count(*) FROM ftstable;\n**\n** xPhraseFirst()\n** This function is used, along with type Fts5PhraseIter and the xPhraseNext\n** method, to iterate through all instances of a single query phrase within\n** the current row. This is the same information as is accessible via the\n** xInstCount/xInst APIs. While the xInstCount/xInst APIs are more convenient\n** to use, this API may be faster under some circumstances. To iterate\n** through instances of phrase iPhrase, use the following code:\n**\n** Fts5PhraseIter iter;\n** int iCol, iOff;\n** for(pApi->xPhraseFirst(pFts, iPhrase, &iter, &iCol, &iOff);\n** iCol>=0;\n** pApi->xPhraseNext(pFts, &iter, &iCol, &iOff)\n** ){\n** // An instance of phrase iPhrase at offset iOff of column iCol\n** }\n**\n** The Fts5PhraseIter structure is defined above. Applications should not\n** modify this structure directly - it should only be used as shown above\n** with the xPhraseFirst() and xPhraseNext() API methods (and by\n** xPhraseFirstColumn() and xPhraseNextColumn() as illustrated below).\n**\n** This API can be quite slow if used with an FTS5 table created with the\n** \"detail=none\" or \"detail=column\" option. If the FTS5 table is created\n** with either \"detail=none\" or \"detail=column\" and \"content=\" option\n** (i.e. if it is a contentless table), then this API always iterates\n** through an empty set (all calls to xPhraseFirst() set iCol to -1).\n**\n** In all cases, matches are visited in (column ASC, offset ASC) order.\n** i.e. all those in column 0, sorted by offset, followed by those in\n** column 1, etc.\n**\n** xPhraseNext()\n** See xPhraseFirst above.\n**\n** xPhraseFirstColumn()\n** This function and xPhraseNextColumn() are similar to the xPhraseFirst()\n** and xPhraseNext() APIs described above. The difference is that instead\n** of iterating through all instances of a phrase in the current row, these\n** APIs are used to iterate through the set of columns in the current row\n** that contain one or more instances of a specified phrase. For example:\n**\n** Fts5PhraseIter iter;\n** int iCol;\n** for(pApi->xPhraseFirstColumn(pFts, iPhrase, &iter, &iCol);\n** iCol>=0;\n** pApi->xPhraseNextColumn(pFts, &iter, &iCol)\n** ){\n** // Column iCol contains at least one instance of phrase iPhrase\n** }\n**\n** This API can be quite slow if used with an FTS5 table created with the\n** \"detail=none\" option. If the FTS5 table is created with either\n** \"detail=none\" \"content=\" option (i.e. if it is a contentless table),\n** then this API always iterates through an empty set (all calls to\n** xPhraseFirstColumn() set iCol to -1).\n**\n** The information accessed using this API and its companion\n** xPhraseFirstColumn() may also be obtained using xPhraseFirst/xPhraseNext\n** (or xInst/xInstCount). The chief advantage of this API is that it is\n** significantly more efficient than those alternatives when used with\n** \"detail=column\" tables.\n**\n** xPhraseNextColumn()\n** See xPhraseFirstColumn above.\n**\n** xQueryToken(pFts5, iPhrase, iToken, ppToken, pnToken)\n** This is used to access token iToken of phrase iPhrase of the current\n** query. Before returning, output parameter *ppToken is set to point\n** to a buffer containing the requested token, and *pnToken to the\n** size of this buffer in bytes.\n**\n** If iPhrase or iToken are less than zero, or if iPhrase is greater than\n** or equal to the number of phrases in the query as reported by\n** xPhraseCount(), or if iToken is equal to or greater than the number of\n** tokens in the phrase, SQLITE_RANGE is returned and *ppToken and *pnToken\n are both zeroed.\n**\n** The output text is not a copy of the query text that specified the\n** token. It is the output of the tokenizer module. For tokendata=1\n** tables, this includes any embedded 0x00 and trailing data.\n**\n** xInstToken(pFts5, iIdx, iToken, ppToken, pnToken)\n** This is used to access token iToken of phrase hit iIdx within the\n** current row. If iIdx is less than zero or greater than or equal to the\n** value returned by xInstCount(), SQLITE_RANGE is returned. Otherwise,\n** output variable (*ppToken) is set to point to a buffer containing the\n** matching document token, and (*pnToken) to the size of that buffer in\n** bytes.\n**\n** The output text is not a copy of the document text that was tokenized.\n** It is the output of the tokenizer module. For tokendata=1 tables, this\n** includes any embedded 0x00 and trailing data.\n**\n** This API may be slow in some cases if the token identified by parameters\n** iIdx and iToken matched a prefix token in the query. In most cases, the\n** first call to this API for each prefix token in the query is forced\n** to scan the portion of the full-text index that matches the prefix\n** token to collect the extra data required by this API. If the prefix\n** token matches a large number of token instances in the document set,\n** this may be a performance problem.\n**\n** If the user knows in advance that a query may use this API for a\n** prefix token, FTS5 may be configured to collect all required data as part\n** of the initial querying of the full-text index, avoiding the second scan\n** entirely. This also causes prefix queries that do not use this API to\n** run more slowly and use more memory. FTS5 may be configured in this way\n** either on a per-table basis using the [FTS5 insttoken | 'insttoken']\n** option, or on a per-query basis using the\n** [fts5_insttoken | fts5_insttoken()] user function.\n**\n** This API can be quite slow if used with an FTS5 table created with the\n** \"detail=none\" or \"detail=column\" option.\n**\n** xColumnLocale(pFts5, iIdx, pzLocale, pnLocale)\n** If parameter iCol is less than zero, or greater than or equal to the\n** number of columns in the table, SQLITE_RANGE is returned.\n**\n** Otherwise, this function attempts to retrieve the locale associated\n** with column iCol of the current row. Usually, there is no associated\n** locale, and output parameters (*pzLocale) and (*pnLocale) are set\n** to NULL and 0, respectively. However, if the fts5_locale() function\n** was used to associate a locale with the value when it was inserted\n** into the fts5 table, then (*pzLocale) is set to point to a nul-terminated\n** buffer containing the name of the locale in utf-8 encoding. (*pnLocale)\n** is set to the size in bytes of the buffer, not including the\n** nul-terminator.\n**\n** If successful, SQLITE_OK is returned. Or, if an error occurs, an\n** SQLite error code is returned. The final value of the output parameters\n** is undefined in this case.\n**\n** xTokenize_v2:\n** Tokenize text using the tokenizer belonging to the FTS5 table. This\n** API is the same as the xTokenize() API, except that it allows a tokenizer\n** locale to be specified.\n*/\nstruct Fts5ExtensionApi {\n int iVersion; /* Currently always set to 4 */\n\n void *(*xUserData)(Fts5Context*);\n\n int (*xColumnCount)(Fts5Context*);\n int (*xRowCount)(Fts5Context*, sqlite3_int64 *pnRow);\n int (*xColumnTotalSize)(Fts5Context*, int iCol, sqlite3_int64 *pnToken);\n\n int (*xTokenize)(Fts5Context*,\n const char *pText, int nText, /* Text to tokenize */\n void *pCtx, /* Context passed to xToken() */\n int (*xToken)(void*, int, const char*, int, int, int) /* Callback */\n );\n\n int (*xPhraseCount)(Fts5Context*);\n int (*xPhraseSize)(Fts5Context*, int iPhrase);\n\n int (*xInstCount)(Fts5Context*, int *pnInst);\n int (*xInst)(Fts5Context*, int iIdx, int *piPhrase, int *piCol, int *piOff);\n\n sqlite3_int64 (*xRowid)(Fts5Context*);\n int (*xColumnText)(Fts5Context*, int iCol, const char **pz, int *pn);\n int (*xColumnSize)(Fts5Context*, int iCol, int *pnToken);\n\n int (*xQueryPhrase)(Fts5Context*, int iPhrase, void *pUserData,\n int(*)(const Fts5ExtensionApi*,Fts5Context*,void*)\n );\n int (*xSetAuxdata)(Fts5Context*, void *pAux, void(*xDelete)(void*));\n void *(*xGetAuxdata)(Fts5Context*, int bClear);\n\n int (*xPhraseFirst)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*, int*);\n void (*xPhraseNext)(Fts5Context*, Fts5PhraseIter*, int *piCol, int *piOff);\n\n int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*);\n void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol);\n\n /* Below this point are iVersion>=3 only */\n int (*xQueryToken)(Fts5Context*,\n int iPhrase, int iToken,\n const char **ppToken, int *pnToken\n );\n int (*xInstToken)(Fts5Context*, int iIdx, int iToken, const char**, int*);\n\n /* Below this point are iVersion>=4 only */\n int (*xColumnLocale)(Fts5Context*, int iCol, const char **pz, int *pn);\n int (*xTokenize_v2)(Fts5Context*,\n const char *pText, int nText, /* Text to tokenize */\n const char *pLocale, int nLocale, /* Locale to pass to tokenizer */\n void *pCtx, /* Context passed to xToken() */\n int (*xToken)(void*, int, const char*, int, int, int) /* Callback */\n );\n};\n\n/*\n** CUSTOM AUXILIARY FUNCTIONS\n*************************************************************************/\n\n/*************************************************************************\n** CUSTOM TOKENIZERS\n**\n** Applications may also register custom tokenizer types. A tokenizer\n** is registered by providing fts5 with a populated instance of the\n** following structure. All structure methods must be defined, setting\n** any member of the fts5_tokenizer struct to NULL leads to undefined\n** behaviour. The structure methods are expected to function as follows:\n**\n** xCreate:\n** This function is used to allocate and initialize a tokenizer instance.\n** A tokenizer instance is required to actually tokenize text.\n**\n** The first argument passed to this function is a copy of the (void*)\n** pointer provided by the application when the fts5_tokenizer_v2 object\n** was registered with FTS5 (the third argument to xCreateTokenizer()).\n** The second and third arguments are an array of nul-terminated strings\n** containing the tokenizer arguments, if any, specified following the\n** tokenizer name as part of the CREATE VIRTUAL TABLE statement used\n** to create the FTS5 table.\n**\n** The final argument is an output variable. If successful, (*ppOut)\n** should be set to point to the new tokenizer handle and SQLITE_OK\n** returned. If an error occurs, some value other than SQLITE_OK should\n** be returned. In this case, fts5 assumes that the final value of *ppOut\n** is undefined.\n**\n** xDelete:\n** This function is invoked to delete a tokenizer handle previously\n** allocated using xCreate(). Fts5 guarantees that this function will\n** be invoked exactly once for each successful call to xCreate().\n**\n** xTokenize:\n** This function is expected to tokenize the nText byte string indicated\n** by argument pText. pText may or may not be nul-terminated. The first\n** argument passed to this function is a pointer to an Fts5Tokenizer object\n** returned by an earlier call to xCreate().\n**\n** The third argument indicates the reason that FTS5 is requesting\n** tokenization of the supplied text. This is always one of the following\n** four values:\n**\n**
  • FTS5_TOKENIZE_DOCUMENT - A document is being inserted into\n** or removed from the FTS table. The tokenizer is being invoked to\n** determine the set of tokens to add to (or delete from) the\n** FTS index.\n**\n**
  • FTS5_TOKENIZE_QUERY - A MATCH query is being executed\n** against the FTS index. The tokenizer is being called to tokenize\n** a bareword or quoted string specified as part of the query.\n**\n**
  • (FTS5_TOKENIZE_QUERY | FTS5_TOKENIZE_PREFIX) - Same as\n** FTS5_TOKENIZE_QUERY, except that the bareword or quoted string is\n** followed by a \"*\" character, indicating that the last token\n** returned by the tokenizer will be treated as a token prefix.\n**\n**
  • FTS5_TOKENIZE_AUX - The tokenizer is being invoked to\n** satisfy an fts5_api.xTokenize() request made by an auxiliary\n** function. Or an fts5_api.xColumnSize() request made by the same\n** on a columnsize=0 database.\n**
\n**\n** The sixth and seventh arguments passed to xTokenize() - pLocale and\n** nLocale - are a pointer to a buffer containing the locale to use for\n** tokenization (e.g. \"en_US\") and its size in bytes, respectively. The\n** pLocale buffer is not nul-terminated. pLocale may be passed NULL (in\n** which case nLocale is always 0) to indicate that the tokenizer should\n** use its default locale.\n**\n** For each token in the input string, the supplied callback xToken() must\n** be invoked. The first argument to it should be a copy of the pointer\n** passed as the second argument to xTokenize(). The third and fourth\n** arguments are a pointer to a buffer containing the token text, and the\n** size of the token in bytes. The 4th and 5th arguments are the byte offsets\n** of the first byte of and first byte immediately following the text from\n** which the token is derived within the input.\n**\n** The second argument passed to the xToken() callback (\"tflags\") should\n** normally be set to 0. The exception is if the tokenizer supports\n** synonyms. In this case see the discussion below for details.\n**\n** FTS5 assumes the xToken() callback is invoked for each token in the\n** order that they occur within the input text.\n**\n** If an xToken() callback returns any value other than SQLITE_OK, then\n** the tokenization should be abandoned and the xTokenize() method should\n** immediately return a copy of the xToken() return value. Or, if the\n** input buffer is exhausted, xTokenize() should return SQLITE_OK. Finally,\n** if an error occurs with the xTokenize() implementation itself, it\n** may abandon the tokenization and return any error code other than\n** SQLITE_OK or SQLITE_DONE.\n**\n** If the tokenizer is registered using an fts5_tokenizer_v2 object,\n** then the xTokenize() method has two additional arguments - pLocale\n** and nLocale. These specify the locale that the tokenizer should use\n** for the current request. If pLocale and nLocale are both 0, then the\n** tokenizer should use its default locale. Otherwise, pLocale points to\n** an nLocale byte buffer containing the name of the locale to use as utf-8\n** text. pLocale is not nul-terminated.\n**\n** FTS5_TOKENIZER\n**\n** There is also an fts5_tokenizer object. This is an older, deprecated,\n** version of fts5_tokenizer_v2. It is similar except that:\n**\n**
    \n**
  • There is no \"iVersion\" field, and\n**
  • The xTokenize() method does not take a locale argument.\n**
\n**\n** Legacy fts5_tokenizer tokenizers must be registered using the\n** legacy xCreateTokenizer() function, instead of xCreateTokenizer_v2().\n**\n** Tokenizer implementations registered using either API may be retrieved\n** using both xFindTokenizer() and xFindTokenizer_v2().\n**\n** SYNONYM SUPPORT\n**\n** Custom tokenizers may also support synonyms. Consider a case in which a\n** user wishes to query for a phrase such as \"first place\". Using the\n** built-in tokenizers, the FTS5 query 'first + place' will match instances\n** of \"first place\" within the document set, but not alternative forms\n** such as \"1st place\". In some applications, it would be better to match\n** all instances of \"first place\" or \"1st place\" regardless of which form\n** the user specified in the MATCH query text.\n**\n** There are several ways to approach this in FTS5:\n**\n**
  1. By mapping all synonyms to a single token. In this case, using\n** the above example, this means that the tokenizer returns the\n** same token for inputs \"first\" and \"1st\". Say that token is in\n** fact \"first\", so that when the user inserts the document \"I won\n** 1st place\" entries are added to the index for tokens \"i\", \"won\",\n** \"first\" and \"place\". If the user then queries for '1st + place',\n** the tokenizer substitutes \"first\" for \"1st\" and the query works\n** as expected.\n**\n**
  2. By querying the index for all synonyms of each query term\n** separately. In this case, when tokenizing query text, the\n** tokenizer may provide multiple synonyms for a single term\n** within the document. FTS5 then queries the index for each\n** synonym individually. For example, faced with the query:\n**\n** \n** ... MATCH 'first place'\n**\n** the tokenizer offers both \"1st\" and \"first\" as synonyms for the\n** first token in the MATCH query and FTS5 effectively runs a query\n** similar to:\n**\n** \n** ... MATCH '(first OR 1st) place'\n**\n** except that, for the purposes of auxiliary functions, the query\n** still appears to contain just two phrases - \"(first OR 1st)\"\n** being treated as a single phrase.\n**\n**
  3. By adding multiple synonyms for a single term to the FTS index.\n** Using this method, when tokenizing document text, the tokenizer\n** provides multiple synonyms for each token. So that when a\n** document such as \"I won first place\" is tokenized, entries are\n** added to the FTS index for \"i\", \"won\", \"first\", \"1st\" and\n** \"place\".\n**\n** This way, even if the tokenizer does not provide synonyms\n** when tokenizing query text (it should not - to do so would be\n** inefficient), it doesn't matter if the user queries for\n** 'first + place' or '1st + place', as there are entries in the\n** FTS index corresponding to both forms of the first token.\n**
\n**\n** Whether it is parsing document or query text, any call to xToken that\n** specifies a tflags argument with the FTS5_TOKEN_COLOCATED bit\n** is considered to supply a synonym for the previous token. For example,\n** when parsing the document \"I won first place\", a tokenizer that supports\n** synonyms would call xToken() 5 times, as follows:\n**\n** \n** xToken(pCtx, 0, \"i\", 1, 0, 1);\n** xToken(pCtx, 0, \"won\", 3, 2, 5);\n** xToken(pCtx, 0, \"first\", 5, 6, 11);\n** xToken(pCtx, FTS5_TOKEN_COLOCATED, \"1st\", 3, 6, 11);\n** xToken(pCtx, 0, \"place\", 5, 12, 17);\n**\n**\n** It is an error to specify the FTS5_TOKEN_COLOCATED flag the first time\n** xToken() is called. Multiple synonyms may be specified for a single token\n** by making multiple calls to xToken(FTS5_TOKEN_COLOCATED) in sequence.\n** There is no limit to the number of synonyms that may be provided for a\n** single token.\n**\n** In many cases, method (1) above is the best approach. It does not add\n** extra data to the FTS index or require FTS5 to query for multiple terms,\n** so it is efficient in terms of disk space and query speed. However, it\n** does not support prefix queries very well. If, as suggested above, the\n** token \"first\" is substituted for \"1st\" by the tokenizer, then the query:\n**\n** \n** ... MATCH '1s*'\n**\n** will not match documents that contain the token \"1st\" (as the tokenizer\n** will probably not map \"1s\" to any prefix of \"first\").\n**\n** For full prefix support, method (3) may be preferred. In this case,\n** because the index contains entries for both \"first\" and \"1st\", prefix\n** queries such as 'fi*' or '1s*' will match correctly. However, because\n** extra entries are added to the FTS index, this method uses more space\n** within the database.\n**\n** Method (2) offers a midpoint between (1) and (3). Using this method,\n** a query such as '1s*' will match documents that contain the literal\n** token \"1st\", but not \"first\" (assuming the tokenizer is not able to\n** provide synonyms for prefixes). However, a non-prefix query like '1st'\n** will match against \"1st\" and \"first\". This method does not require\n** extra disk space, as no extra entries are added to the FTS index.\n** On the other hand, it may require more CPU cycles to run MATCH queries,\n** as separate queries of the FTS index are required for each synonym.\n**\n** When using methods (2) or (3), it is important that the tokenizer only\n** provide synonyms when tokenizing document text (method (3)) or query\n** text (method (2)), not both. Doing so will not cause any errors, but is\n** inefficient.\n*/\ntypedef struct Fts5Tokenizer Fts5Tokenizer;\ntypedef struct fts5_tokenizer_v2 fts5_tokenizer_v2;\nstruct fts5_tokenizer_v2 {\n int iVersion; /* Currently always 2 */\n\n int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut);\n void (*xDelete)(Fts5Tokenizer*);\n int (*xTokenize)(Fts5Tokenizer*,\n void *pCtx,\n int flags, /* Mask of FTS5_TOKENIZE_* flags */\n const char *pText, int nText,\n const char *pLocale, int nLocale,\n int (*xToken)(\n void *pCtx, /* Copy of 2nd argument to xTokenize() */\n int tflags, /* Mask of FTS5_TOKEN_* flags */\n const char *pToken, /* Pointer to buffer containing token */\n int nToken, /* Size of token in bytes */\n int iStart, /* Byte offset of token within input text */\n int iEnd /* Byte offset of end of token within input text */\n )\n );\n};\n\n/*\n** New code should use the fts5_tokenizer_v2 type to define tokenizer\n** implementations. The following type is included for legacy applications\n** that still use it.\n*/\ntypedef struct fts5_tokenizer fts5_tokenizer;\nstruct fts5_tokenizer {\n int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut);\n void (*xDelete)(Fts5Tokenizer*);\n int (*xTokenize)(Fts5Tokenizer*,\n void *pCtx,\n int flags, /* Mask of FTS5_TOKENIZE_* flags */\n const char *pText, int nText,\n int (*xToken)(\n void *pCtx, /* Copy of 2nd argument to xTokenize() */\n int tflags, /* Mask of FTS5_TOKEN_* flags */\n const char *pToken, /* Pointer to buffer containing token */\n int nToken, /* Size of token in bytes */\n int iStart, /* Byte offset of token within input text */\n int iEnd /* Byte offset of end of token within input text */\n )\n );\n};\n\n\n/* Flags that may be passed as the third argument to xTokenize() */\n#define FTS5_TOKENIZE_QUERY 0x0001\n#define FTS5_TOKENIZE_PREFIX 0x0002\n#define FTS5_TOKENIZE_DOCUMENT 0x0004\n#define FTS5_TOKENIZE_AUX 0x0008\n\n/* Flags that may be passed by the tokenizer implementation back to FTS5\n** as the third argument to the supplied xToken callback. */\n#define FTS5_TOKEN_COLOCATED 0x0001 /* Same position as prev. token */\n\n/*\n** END OF CUSTOM TOKENIZERS\n*************************************************************************/\n\n/*************************************************************************\n** FTS5 EXTENSION REGISTRATION API\n*/\ntypedef struct fts5_api fts5_api;\nstruct fts5_api {\n int iVersion; /* Currently always set to 3 */\n\n /* Create a new tokenizer */\n int (*xCreateTokenizer)(\n fts5_api *pApi,\n const char *zName,\n void *pUserData,\n fts5_tokenizer *pTokenizer,\n void (*xDestroy)(void*)\n );\n\n /* Find an existing tokenizer */\n int (*xFindTokenizer)(\n fts5_api *pApi,\n const char *zName,\n void **ppUserData,\n fts5_tokenizer *pTokenizer\n );\n\n /* Create a new auxiliary function */\n int (*xCreateFunction)(\n fts5_api *pApi,\n const char *zName,\n void *pUserData,\n fts5_extension_function xFunction,\n void (*xDestroy)(void*)\n );\n\n /* APIs below this point are only available if iVersion>=3 */\n\n /* Create a new tokenizer */\n int (*xCreateTokenizer_v2)(\n fts5_api *pApi,\n const char *zName,\n void *pUserData,\n fts5_tokenizer_v2 *pTokenizer,\n void (*xDestroy)(void*)\n );\n\n /* Find an existing tokenizer */\n int (*xFindTokenizer_v2)(\n fts5_api *pApi,\n const char *zName,\n void **ppUserData,\n fts5_tokenizer_v2 **ppTokenizer\n );\n};\n\n/*\n** END OF REGISTRATION API\n*************************************************************************/\n\n#ifdef __cplusplus\n} /* end of the 'extern \"C\"' block */\n#endif\n\n#endif /* _FTS5_H */\n\n/******** End of fts5.h *********/\n#endif /* SQLITE3_H */\n" }, { "path": "src/sqlite3ext.h", "content": "/*\n** 2006 June 7\n**\n** The author disclaims copyright to this source code. In place of\n** a legal notice, here is a blessing:\n**\n** May you do good and not evil.\n** May you find forgiveness for yourself and forgive others.\n** May you share freely, never taking more than you give.\n**\n*************************************************************************\n** This header file defines the SQLite interface for use by\n** shared libraries that want to be imported as extensions into\n** an SQLite instance. Shared libraries that intend to be loaded\n** as extensions by SQLite should #include this file instead of\n** sqlite3.h.\n*/\n#ifndef SQLITE3EXT_H\n#define SQLITE3EXT_H\n#include \"sqlite3.h\"\n\n/*\n** The following structure holds pointers to all of the SQLite API\n** routines.\n**\n** WARNING: In order to maintain backwards compatibility, add new\n** interfaces to the end of this structure only. If you insert new\n** interfaces in the middle of this structure, then older different\n** versions of SQLite will not be able to load each other's shared\n** libraries!\n*/\nstruct sqlite3_api_routines {\n void * (*aggregate_context)(sqlite3_context*,int nBytes);\n int (*aggregate_count)(sqlite3_context*);\n int (*bind_blob)(sqlite3_stmt*,int,const void*,int n,void(*)(void*));\n int (*bind_double)(sqlite3_stmt*,int,double);\n int (*bind_int)(sqlite3_stmt*,int,int);\n int (*bind_int64)(sqlite3_stmt*,int,sqlite_int64);\n int (*bind_null)(sqlite3_stmt*,int);\n int (*bind_parameter_count)(sqlite3_stmt*);\n int (*bind_parameter_index)(sqlite3_stmt*,const char*zName);\n const char * (*bind_parameter_name)(sqlite3_stmt*,int);\n int (*bind_text)(sqlite3_stmt*,int,const char*,int n,void(*)(void*));\n int (*bind_text16)(sqlite3_stmt*,int,const void*,int,void(*)(void*));\n int (*bind_value)(sqlite3_stmt*,int,const sqlite3_value*);\n int (*busy_handler)(sqlite3*,int(*)(void*,int),void*);\n int (*busy_timeout)(sqlite3*,int ms);\n int (*changes)(sqlite3*);\n int (*close)(sqlite3*);\n int (*collation_needed)(sqlite3*,void*,void(*)(void*,sqlite3*,\n int eTextRep,const char*));\n int (*collation_needed16)(sqlite3*,void*,void(*)(void*,sqlite3*,\n int eTextRep,const void*));\n const void * (*column_blob)(sqlite3_stmt*,int iCol);\n int (*column_bytes)(sqlite3_stmt*,int iCol);\n int (*column_bytes16)(sqlite3_stmt*,int iCol);\n int (*column_count)(sqlite3_stmt*pStmt);\n const char * (*column_database_name)(sqlite3_stmt*,int);\n const void * (*column_database_name16)(sqlite3_stmt*,int);\n const char * (*column_decltype)(sqlite3_stmt*,int i);\n const void * (*column_decltype16)(sqlite3_stmt*,int);\n double (*column_double)(sqlite3_stmt*,int iCol);\n int (*column_int)(sqlite3_stmt*,int iCol);\n sqlite_int64 (*column_int64)(sqlite3_stmt*,int iCol);\n const char * (*column_name)(sqlite3_stmt*,int);\n const void * (*column_name16)(sqlite3_stmt*,int);\n const char * (*column_origin_name)(sqlite3_stmt*,int);\n const void * (*column_origin_name16)(sqlite3_stmt*,int);\n const char * (*column_table_name)(sqlite3_stmt*,int);\n const void * (*column_table_name16)(sqlite3_stmt*,int);\n const unsigned char * (*column_text)(sqlite3_stmt*,int iCol);\n const void * (*column_text16)(sqlite3_stmt*,int iCol);\n int (*column_type)(sqlite3_stmt*,int iCol);\n sqlite3_value* (*column_value)(sqlite3_stmt*,int iCol);\n void * (*commit_hook)(sqlite3*,int(*)(void*),void*);\n int (*complete)(const char*sql);\n int (*complete16)(const void*sql);\n int (*create_collation)(sqlite3*,const char*,int,void*,\n int(*)(void*,int,const void*,int,const void*));\n int (*create_collation16)(sqlite3*,const void*,int,void*,\n int(*)(void*,int,const void*,int,const void*));\n int (*create_function)(sqlite3*,const char*,int,int,void*,\n void (*xFunc)(sqlite3_context*,int,sqlite3_value**),\n void (*xStep)(sqlite3_context*,int,sqlite3_value**),\n void (*xFinal)(sqlite3_context*));\n int (*create_function16)(sqlite3*,const void*,int,int,void*,\n void (*xFunc)(sqlite3_context*,int,sqlite3_value**),\n void (*xStep)(sqlite3_context*,int,sqlite3_value**),\n void (*xFinal)(sqlite3_context*));\n int (*create_module)(sqlite3*,const char*,const sqlite3_module*,void*);\n int (*data_count)(sqlite3_stmt*pStmt);\n sqlite3 * (*db_handle)(sqlite3_stmt*);\n int (*declare_vtab)(sqlite3*,const char*);\n int (*enable_shared_cache)(int);\n int (*errcode)(sqlite3*db);\n const char * (*errmsg)(sqlite3*);\n const void * (*errmsg16)(sqlite3*);\n int (*exec)(sqlite3*,const char*,sqlite3_callback,void*,char**);\n int (*expired)(sqlite3_stmt*);\n int (*finalize)(sqlite3_stmt*pStmt);\n void (*free)(void*);\n void (*free_table)(char**result);\n int (*get_autocommit)(sqlite3*);\n void * (*get_auxdata)(sqlite3_context*,int);\n int (*get_table)(sqlite3*,const char*,char***,int*,int*,char**);\n int (*global_recover)(void);\n void (*interruptx)(sqlite3*);\n sqlite_int64 (*last_insert_rowid)(sqlite3*);\n const char * (*libversion)(void);\n int (*libversion_number)(void);\n void *(*malloc)(int);\n char * (*mprintf)(const char*,...);\n int (*open)(const char*,sqlite3**);\n int (*open16)(const void*,sqlite3**);\n int (*prepare)(sqlite3*,const char*,int,sqlite3_stmt**,const char**);\n int (*prepare16)(sqlite3*,const void*,int,sqlite3_stmt**,const void**);\n void * (*profile)(sqlite3*,void(*)(void*,const char*,sqlite_uint64),void*);\n void (*progress_handler)(sqlite3*,int,int(*)(void*),void*);\n void *(*realloc)(void*,int);\n int (*reset)(sqlite3_stmt*pStmt);\n void (*result_blob)(sqlite3_context*,const void*,int,void(*)(void*));\n void (*result_double)(sqlite3_context*,double);\n void (*result_error)(sqlite3_context*,const char*,int);\n void (*result_error16)(sqlite3_context*,const void*,int);\n void (*result_int)(sqlite3_context*,int);\n void (*result_int64)(sqlite3_context*,sqlite_int64);\n void (*result_null)(sqlite3_context*);\n void (*result_text)(sqlite3_context*,const char*,int,void(*)(void*));\n void (*result_text16)(sqlite3_context*,const void*,int,void(*)(void*));\n void (*result_text16be)(sqlite3_context*,const void*,int,void(*)(void*));\n void (*result_text16le)(sqlite3_context*,const void*,int,void(*)(void*));\n void (*result_value)(sqlite3_context*,sqlite3_value*);\n void * (*rollback_hook)(sqlite3*,void(*)(void*),void*);\n int (*set_authorizer)(sqlite3*,int(*)(void*,int,const char*,const char*,\n const char*,const char*),void*);\n void (*set_auxdata)(sqlite3_context*,int,void*,void (*)(void*));\n char * (*xsnprintf)(int,char*,const char*,...);\n int (*step)(sqlite3_stmt*);\n int (*table_column_metadata)(sqlite3*,const char*,const char*,const char*,\n char const**,char const**,int*,int*,int*);\n void (*thread_cleanup)(void);\n int (*total_changes)(sqlite3*);\n void * (*trace)(sqlite3*,void(*xTrace)(void*,const char*),void*);\n int (*transfer_bindings)(sqlite3_stmt*,sqlite3_stmt*);\n void * (*update_hook)(sqlite3*,void(*)(void*,int ,char const*,char const*,\n sqlite_int64),void*);\n void * (*user_data)(sqlite3_context*);\n const void * (*value_blob)(sqlite3_value*);\n int (*value_bytes)(sqlite3_value*);\n int (*value_bytes16)(sqlite3_value*);\n double (*value_double)(sqlite3_value*);\n int (*value_int)(sqlite3_value*);\n sqlite_int64 (*value_int64)(sqlite3_value*);\n int (*value_numeric_type)(sqlite3_value*);\n const unsigned char * (*value_text)(sqlite3_value*);\n const void * (*value_text16)(sqlite3_value*);\n const void * (*value_text16be)(sqlite3_value*);\n const void * (*value_text16le)(sqlite3_value*);\n int (*value_type)(sqlite3_value*);\n char *(*vmprintf)(const char*,va_list);\n /* Added ??? */\n int (*overload_function)(sqlite3*, const char *zFuncName, int nArg);\n /* Added by 3.3.13 */\n int (*prepare_v2)(sqlite3*,const char*,int,sqlite3_stmt**,const char**);\n int (*prepare16_v2)(sqlite3*,const void*,int,sqlite3_stmt**,const void**);\n int (*clear_bindings)(sqlite3_stmt*);\n /* Added by 3.4.1 */\n int (*create_module_v2)(sqlite3*,const char*,const sqlite3_module*,void*,\n void (*xDestroy)(void *));\n /* Added by 3.5.0 */\n int (*bind_zeroblob)(sqlite3_stmt*,int,int);\n int (*blob_bytes)(sqlite3_blob*);\n int (*blob_close)(sqlite3_blob*);\n int (*blob_open)(sqlite3*,const char*,const char*,const char*,sqlite3_int64,\n int,sqlite3_blob**);\n int (*blob_read)(sqlite3_blob*,void*,int,int);\n int (*blob_write)(sqlite3_blob*,const void*,int,int);\n int (*create_collation_v2)(sqlite3*,const char*,int,void*,\n int(*)(void*,int,const void*,int,const void*),\n void(*)(void*));\n int (*file_control)(sqlite3*,const char*,int,void*);\n sqlite3_int64 (*memory_highwater)(int);\n sqlite3_int64 (*memory_used)(void);\n sqlite3_mutex *(*mutex_alloc)(int);\n void (*mutex_enter)(sqlite3_mutex*);\n void (*mutex_free)(sqlite3_mutex*);\n void (*mutex_leave)(sqlite3_mutex*);\n int (*mutex_try)(sqlite3_mutex*);\n int (*open_v2)(const char*,sqlite3**,int,const char*);\n int (*release_memory)(int);\n void (*result_error_nomem)(sqlite3_context*);\n void (*result_error_toobig)(sqlite3_context*);\n int (*sleep)(int);\n void (*soft_heap_limit)(int);\n sqlite3_vfs *(*vfs_find)(const char*);\n int (*vfs_register)(sqlite3_vfs*,int);\n int (*vfs_unregister)(sqlite3_vfs*);\n int (*xthreadsafe)(void);\n void (*result_zeroblob)(sqlite3_context*,int);\n void (*result_error_code)(sqlite3_context*,int);\n int (*test_control)(int, ...);\n void (*randomness)(int,void*);\n sqlite3 *(*context_db_handle)(sqlite3_context*);\n int (*extended_result_codes)(sqlite3*,int);\n int (*limit)(sqlite3*,int,int);\n sqlite3_stmt *(*next_stmt)(sqlite3*,sqlite3_stmt*);\n const char *(*sql)(sqlite3_stmt*);\n int (*status)(int,int*,int*,int);\n int (*backup_finish)(sqlite3_backup*);\n sqlite3_backup *(*backup_init)(sqlite3*,const char*,sqlite3*,const char*);\n int (*backup_pagecount)(sqlite3_backup*);\n int (*backup_remaining)(sqlite3_backup*);\n int (*backup_step)(sqlite3_backup*,int);\n const char *(*compileoption_get)(int);\n int (*compileoption_used)(const char*);\n int (*create_function_v2)(sqlite3*,const char*,int,int,void*,\n void (*xFunc)(sqlite3_context*,int,sqlite3_value**),\n void (*xStep)(sqlite3_context*,int,sqlite3_value**),\n void (*xFinal)(sqlite3_context*),\n void(*xDestroy)(void*));\n int (*db_config)(sqlite3*,int,...);\n sqlite3_mutex *(*db_mutex)(sqlite3*);\n int (*db_status)(sqlite3*,int,int*,int*,int);\n int (*extended_errcode)(sqlite3*);\n void (*log)(int,const char*,...);\n sqlite3_int64 (*soft_heap_limit64)(sqlite3_int64);\n const char *(*sourceid)(void);\n int (*stmt_status)(sqlite3_stmt*,int,int);\n int (*strnicmp)(const char*,const char*,int);\n int (*unlock_notify)(sqlite3*,void(*)(void**,int),void*);\n int (*wal_autocheckpoint)(sqlite3*,int);\n int (*wal_checkpoint)(sqlite3*,const char*);\n void *(*wal_hook)(sqlite3*,int(*)(void*,sqlite3*,const char*,int),void*);\n int (*blob_reopen)(sqlite3_blob*,sqlite3_int64);\n int (*vtab_config)(sqlite3*,int op,...);\n int (*vtab_on_conflict)(sqlite3*);\n /* Version 3.7.16 and later */\n int (*close_v2)(sqlite3*);\n const char *(*db_filename)(sqlite3*,const char*);\n int (*db_readonly)(sqlite3*,const char*);\n int (*db_release_memory)(sqlite3*);\n const char *(*errstr)(int);\n int (*stmt_busy)(sqlite3_stmt*);\n int (*stmt_readonly)(sqlite3_stmt*);\n int (*stricmp)(const char*,const char*);\n int (*uri_boolean)(const char*,const char*,int);\n sqlite3_int64 (*uri_int64)(const char*,const char*,sqlite3_int64);\n const char *(*uri_parameter)(const char*,const char*);\n char *(*xvsnprintf)(int,char*,const char*,va_list);\n int (*wal_checkpoint_v2)(sqlite3*,const char*,int,int*,int*);\n /* Version 3.8.7 and later */\n int (*auto_extension)(void(*)(void));\n int (*bind_blob64)(sqlite3_stmt*,int,const void*,sqlite3_uint64,\n void(*)(void*));\n int (*bind_text64)(sqlite3_stmt*,int,const char*,sqlite3_uint64,\n void(*)(void*),unsigned char);\n int (*cancel_auto_extension)(void(*)(void));\n int (*load_extension)(sqlite3*,const char*,const char*,char**);\n void *(*malloc64)(sqlite3_uint64);\n sqlite3_uint64 (*msize)(void*);\n void *(*realloc64)(void*,sqlite3_uint64);\n void (*reset_auto_extension)(void);\n void (*result_blob64)(sqlite3_context*,const void*,sqlite3_uint64,\n void(*)(void*));\n void (*result_text64)(sqlite3_context*,const char*,sqlite3_uint64,\n void(*)(void*), unsigned char);\n int (*strglob)(const char*,const char*);\n /* Version 3.8.11 and later */\n sqlite3_value *(*value_dup)(const sqlite3_value*);\n void (*value_free)(sqlite3_value*);\n int (*result_zeroblob64)(sqlite3_context*,sqlite3_uint64);\n int (*bind_zeroblob64)(sqlite3_stmt*, int, sqlite3_uint64);\n /* Version 3.9.0 and later */\n unsigned int (*value_subtype)(sqlite3_value*);\n void (*result_subtype)(sqlite3_context*,unsigned int);\n /* Version 3.10.0 and later */\n int (*status64)(int,sqlite3_int64*,sqlite3_int64*,int);\n int (*strlike)(const char*,const char*,unsigned int);\n int (*db_cacheflush)(sqlite3*);\n /* Version 3.12.0 and later */\n int (*system_errno)(sqlite3*);\n /* Version 3.14.0 and later */\n int (*trace_v2)(sqlite3*,unsigned,int(*)(unsigned,void*,void*,void*),void*);\n char *(*expanded_sql)(sqlite3_stmt*);\n /* Version 3.18.0 and later */\n void (*set_last_insert_rowid)(sqlite3*,sqlite3_int64);\n /* Version 3.20.0 and later */\n int (*prepare_v3)(sqlite3*,const char*,int,unsigned int,\n sqlite3_stmt**,const char**);\n int (*prepare16_v3)(sqlite3*,const void*,int,unsigned int,\n sqlite3_stmt**,const void**);\n int (*bind_pointer)(sqlite3_stmt*,int,void*,const char*,void(*)(void*));\n void (*result_pointer)(sqlite3_context*,void*,const char*,void(*)(void*));\n void *(*value_pointer)(sqlite3_value*,const char*);\n int (*vtab_nochange)(sqlite3_context*);\n int (*value_nochange)(sqlite3_value*);\n const char *(*vtab_collation)(sqlite3_index_info*,int);\n /* Version 3.24.0 and later */\n int (*keyword_count)(void);\n int (*keyword_name)(int,const char**,int*);\n int (*keyword_check)(const char*,int);\n sqlite3_str *(*str_new)(sqlite3*);\n char *(*str_finish)(sqlite3_str*);\n void (*str_appendf)(sqlite3_str*, const char *zFormat, ...);\n void (*str_vappendf)(sqlite3_str*, const char *zFormat, va_list);\n void (*str_append)(sqlite3_str*, const char *zIn, int N);\n void (*str_appendall)(sqlite3_str*, const char *zIn);\n void (*str_appendchar)(sqlite3_str*, int N, char C);\n void (*str_reset)(sqlite3_str*);\n int (*str_errcode)(sqlite3_str*);\n int (*str_length)(sqlite3_str*);\n char *(*str_value)(sqlite3_str*);\n /* Version 3.25.0 and later */\n int (*create_window_function)(sqlite3*,const char*,int,int,void*,\n void (*xStep)(sqlite3_context*,int,sqlite3_value**),\n void (*xFinal)(sqlite3_context*),\n void (*xValue)(sqlite3_context*),\n void (*xInv)(sqlite3_context*,int,sqlite3_value**),\n void(*xDestroy)(void*));\n /* Version 3.26.0 and later */\n const char *(*normalized_sql)(sqlite3_stmt*);\n /* Version 3.28.0 and later */\n int (*stmt_isexplain)(sqlite3_stmt*);\n int (*value_frombind)(sqlite3_value*);\n /* Version 3.30.0 and later */\n int (*drop_modules)(sqlite3*,const char**);\n /* Version 3.31.0 and later */\n sqlite3_int64 (*hard_heap_limit64)(sqlite3_int64);\n const char *(*uri_key)(const char*,int);\n const char *(*filename_database)(const char*);\n const char *(*filename_journal)(const char*);\n const char *(*filename_wal)(const char*);\n /* Version 3.32.0 and later */\n const char *(*create_filename)(const char*,const char*,const char*,\n int,const char**);\n void (*free_filename)(const char*);\n sqlite3_file *(*database_file_object)(const char*);\n /* Version 3.34.0 and later */\n int (*txn_state)(sqlite3*,const char*);\n /* Version 3.36.1 and later */\n sqlite3_int64 (*changes64)(sqlite3*);\n sqlite3_int64 (*total_changes64)(sqlite3*);\n /* Version 3.37.0 and later */\n int (*autovacuum_pages)(sqlite3*,\n unsigned int(*)(void*,const char*,unsigned int,unsigned int,unsigned int),\n void*, void(*)(void*));\n /* Version 3.38.0 and later */\n int (*error_offset)(sqlite3*);\n int (*vtab_rhs_value)(sqlite3_index_info*,int,sqlite3_value**);\n int (*vtab_distinct)(sqlite3_index_info*);\n int (*vtab_in)(sqlite3_index_info*,int,int);\n int (*vtab_in_first)(sqlite3_value*,sqlite3_value**);\n int (*vtab_in_next)(sqlite3_value*,sqlite3_value**);\n /* Version 3.39.0 and later */\n int (*deserialize)(sqlite3*,const char*,unsigned char*,\n sqlite3_int64,sqlite3_int64,unsigned);\n unsigned char *(*serialize)(sqlite3*,const char *,sqlite3_int64*,\n unsigned int);\n const char *(*db_name)(sqlite3*,int);\n /* Version 3.40.0 and later */\n int (*value_encoding)(sqlite3_value*);\n /* Version 3.41.0 and later */\n int (*is_interrupted)(sqlite3*);\n /* Version 3.43.0 and later */\n int (*stmt_explain)(sqlite3_stmt*,int);\n /* Version 3.44.0 and later */\n void *(*get_clientdata)(sqlite3*,const char*);\n int (*set_clientdata)(sqlite3*, const char*, void*, void(*)(void*));\n /* Version 3.50.0 and later */\n int (*setlk_timeout)(sqlite3*,int,int);\n};\n\n/*\n** This is the function signature used for all extension entry points. It\n** is also defined in the file \"loadext.c\".\n*/\ntypedef int (*sqlite3_loadext_entry)(\n sqlite3 *db, /* Handle to the database. */\n char **pzErrMsg, /* Used to set error string on failure. */\n const sqlite3_api_routines *pThunk /* Extension API function pointers. */\n);\n\n/*\n** The following macros redefine the API routines so that they are\n** redirected through the global sqlite3_api structure.\n**\n** This header file is also used by the loadext.c source file\n** (part of the main SQLite library - not an extension) so that\n** it can get access to the sqlite3_api_routines structure\n** definition. But the main library does not want to redefine\n** the API. So the redefinition macros are only valid if the\n** SQLITE_CORE macros is undefined.\n*/\n#if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)\n#define sqlite3_aggregate_context sqlite3_api->aggregate_context\n#ifndef SQLITE_OMIT_DEPRECATED\n#define sqlite3_aggregate_count sqlite3_api->aggregate_count\n#endif\n#define sqlite3_bind_blob sqlite3_api->bind_blob\n#define sqlite3_bind_double sqlite3_api->bind_double\n#define sqlite3_bind_int sqlite3_api->bind_int\n#define sqlite3_bind_int64 sqlite3_api->bind_int64\n#define sqlite3_bind_null sqlite3_api->bind_null\n#define sqlite3_bind_parameter_count sqlite3_api->bind_parameter_count\n#define sqlite3_bind_parameter_index sqlite3_api->bind_parameter_index\n#define sqlite3_bind_parameter_name sqlite3_api->bind_parameter_name\n#define sqlite3_bind_text sqlite3_api->bind_text\n#define sqlite3_bind_text16 sqlite3_api->bind_text16\n#define sqlite3_bind_value sqlite3_api->bind_value\n#define sqlite3_busy_handler sqlite3_api->busy_handler\n#define sqlite3_busy_timeout sqlite3_api->busy_timeout\n#define sqlite3_changes sqlite3_api->changes\n#define sqlite3_close sqlite3_api->close\n#define sqlite3_collation_needed sqlite3_api->collation_needed\n#define sqlite3_collation_needed16 sqlite3_api->collation_needed16\n#define sqlite3_column_blob sqlite3_api->column_blob\n#define sqlite3_column_bytes sqlite3_api->column_bytes\n#define sqlite3_column_bytes16 sqlite3_api->column_bytes16\n#define sqlite3_column_count sqlite3_api->column_count\n#define sqlite3_column_database_name sqlite3_api->column_database_name\n#define sqlite3_column_database_name16 sqlite3_api->column_database_name16\n#define sqlite3_column_decltype sqlite3_api->column_decltype\n#define sqlite3_column_decltype16 sqlite3_api->column_decltype16\n#define sqlite3_column_double sqlite3_api->column_double\n#define sqlite3_column_int sqlite3_api->column_int\n#define sqlite3_column_int64 sqlite3_api->column_int64\n#define sqlite3_column_name sqlite3_api->column_name\n#define sqlite3_column_name16 sqlite3_api->column_name16\n#define sqlite3_column_origin_name sqlite3_api->column_origin_name\n#define sqlite3_column_origin_name16 sqlite3_api->column_origin_name16\n#define sqlite3_column_table_name sqlite3_api->column_table_name\n#define sqlite3_column_table_name16 sqlite3_api->column_table_name16\n#define sqlite3_column_text sqlite3_api->column_text\n#define sqlite3_column_text16 sqlite3_api->column_text16\n#define sqlite3_column_type sqlite3_api->column_type\n#define sqlite3_column_value sqlite3_api->column_value\n#define sqlite3_commit_hook sqlite3_api->commit_hook\n#define sqlite3_complete sqlite3_api->complete\n#define sqlite3_complete16 sqlite3_api->complete16\n#define sqlite3_create_collation sqlite3_api->create_collation\n#define sqlite3_create_collation16 sqlite3_api->create_collation16\n#define sqlite3_create_function sqlite3_api->create_function\n#define sqlite3_create_function16 sqlite3_api->create_function16\n#define sqlite3_create_module sqlite3_api->create_module\n#define sqlite3_create_module_v2 sqlite3_api->create_module_v2\n#define sqlite3_data_count sqlite3_api->data_count\n#define sqlite3_db_handle sqlite3_api->db_handle\n#define sqlite3_declare_vtab sqlite3_api->declare_vtab\n#define sqlite3_enable_shared_cache sqlite3_api->enable_shared_cache\n#define sqlite3_errcode sqlite3_api->errcode\n#define sqlite3_errmsg sqlite3_api->errmsg\n#define sqlite3_errmsg16 sqlite3_api->errmsg16\n#define sqlite3_exec sqlite3_api->exec\n#ifndef SQLITE_OMIT_DEPRECATED\n#define sqlite3_expired sqlite3_api->expired\n#endif\n#define sqlite3_finalize sqlite3_api->finalize\n#define sqlite3_free sqlite3_api->free\n#define sqlite3_free_table sqlite3_api->free_table\n#define sqlite3_get_autocommit sqlite3_api->get_autocommit\n#define sqlite3_get_auxdata sqlite3_api->get_auxdata\n#define sqlite3_get_table sqlite3_api->get_table\n#ifndef SQLITE_OMIT_DEPRECATED\n#define sqlite3_global_recover sqlite3_api->global_recover\n#endif\n#define sqlite3_interrupt sqlite3_api->interruptx\n#define sqlite3_last_insert_rowid sqlite3_api->last_insert_rowid\n#define sqlite3_libversion sqlite3_api->libversion\n#define sqlite3_libversion_number sqlite3_api->libversion_number\n#define sqlite3_malloc sqlite3_api->malloc\n#define sqlite3_mprintf sqlite3_api->mprintf\n#define sqlite3_open sqlite3_api->open\n#define sqlite3_open16 sqlite3_api->open16\n#define sqlite3_prepare sqlite3_api->prepare\n#define sqlite3_prepare16 sqlite3_api->prepare16\n#define sqlite3_prepare_v2 sqlite3_api->prepare_v2\n#define sqlite3_prepare16_v2 sqlite3_api->prepare16_v2\n#define sqlite3_profile sqlite3_api->profile\n#define sqlite3_progress_handler sqlite3_api->progress_handler\n#define sqlite3_realloc sqlite3_api->realloc\n#define sqlite3_reset sqlite3_api->reset\n#define sqlite3_result_blob sqlite3_api->result_blob\n#define sqlite3_result_double sqlite3_api->result_double\n#define sqlite3_result_error sqlite3_api->result_error\n#define sqlite3_result_error16 sqlite3_api->result_error16\n#define sqlite3_result_int sqlite3_api->result_int\n#define sqlite3_result_int64 sqlite3_api->result_int64\n#define sqlite3_result_null sqlite3_api->result_null\n#define sqlite3_result_text sqlite3_api->result_text\n#define sqlite3_result_text16 sqlite3_api->result_text16\n#define sqlite3_result_text16be sqlite3_api->result_text16be\n#define sqlite3_result_text16le sqlite3_api->result_text16le\n#define sqlite3_result_value sqlite3_api->result_value\n#define sqlite3_rollback_hook sqlite3_api->rollback_hook\n#define sqlite3_set_authorizer sqlite3_api->set_authorizer\n#define sqlite3_set_auxdata sqlite3_api->set_auxdata\n#define sqlite3_snprintf sqlite3_api->xsnprintf\n#define sqlite3_step sqlite3_api->step\n#define sqlite3_table_column_metadata sqlite3_api->table_column_metadata\n#define sqlite3_thread_cleanup sqlite3_api->thread_cleanup\n#define sqlite3_total_changes sqlite3_api->total_changes\n#define sqlite3_trace sqlite3_api->trace\n#ifndef SQLITE_OMIT_DEPRECATED\n#define sqlite3_transfer_bindings sqlite3_api->transfer_bindings\n#endif\n#define sqlite3_update_hook sqlite3_api->update_hook\n#define sqlite3_user_data sqlite3_api->user_data\n#define sqlite3_value_blob sqlite3_api->value_blob\n#define sqlite3_value_bytes sqlite3_api->value_bytes\n#define sqlite3_value_bytes16 sqlite3_api->value_bytes16\n#define sqlite3_value_double sqlite3_api->value_double\n#define sqlite3_value_int sqlite3_api->value_int\n#define sqlite3_value_int64 sqlite3_api->value_int64\n#define sqlite3_value_numeric_type sqlite3_api->value_numeric_type\n#define sqlite3_value_text sqlite3_api->value_text\n#define sqlite3_value_text16 sqlite3_api->value_text16\n#define sqlite3_value_text16be sqlite3_api->value_text16be\n#define sqlite3_value_text16le sqlite3_api->value_text16le\n#define sqlite3_value_type sqlite3_api->value_type\n#define sqlite3_vmprintf sqlite3_api->vmprintf\n#define sqlite3_vsnprintf sqlite3_api->xvsnprintf\n#define sqlite3_overload_function sqlite3_api->overload_function\n#define sqlite3_prepare_v2 sqlite3_api->prepare_v2\n#define sqlite3_prepare16_v2 sqlite3_api->prepare16_v2\n#define sqlite3_clear_bindings sqlite3_api->clear_bindings\n#define sqlite3_bind_zeroblob sqlite3_api->bind_zeroblob\n#define sqlite3_blob_bytes sqlite3_api->blob_bytes\n#define sqlite3_blob_close sqlite3_api->blob_close\n#define sqlite3_blob_open sqlite3_api->blob_open\n#define sqlite3_blob_read sqlite3_api->blob_read\n#define sqlite3_blob_write sqlite3_api->blob_write\n#define sqlite3_create_collation_v2 sqlite3_api->create_collation_v2\n#define sqlite3_file_control sqlite3_api->file_control\n#define sqlite3_memory_highwater sqlite3_api->memory_highwater\n#define sqlite3_memory_used sqlite3_api->memory_used\n#define sqlite3_mutex_alloc sqlite3_api->mutex_alloc\n#define sqlite3_mutex_enter sqlite3_api->mutex_enter\n#define sqlite3_mutex_free sqlite3_api->mutex_free\n#define sqlite3_mutex_leave sqlite3_api->mutex_leave\n#define sqlite3_mutex_try sqlite3_api->mutex_try\n#define sqlite3_open_v2 sqlite3_api->open_v2\n#define sqlite3_release_memory sqlite3_api->release_memory\n#define sqlite3_result_error_nomem sqlite3_api->result_error_nomem\n#define sqlite3_result_error_toobig sqlite3_api->result_error_toobig\n#define sqlite3_sleep sqlite3_api->sleep\n#define sqlite3_soft_heap_limit sqlite3_api->soft_heap_limit\n#define sqlite3_vfs_find sqlite3_api->vfs_find\n#define sqlite3_vfs_register sqlite3_api->vfs_register\n#define sqlite3_vfs_unregister sqlite3_api->vfs_unregister\n#define sqlite3_threadsafe sqlite3_api->xthreadsafe\n#define sqlite3_result_zeroblob sqlite3_api->result_zeroblob\n#define sqlite3_result_error_code sqlite3_api->result_error_code\n#define sqlite3_test_control sqlite3_api->test_control\n#define sqlite3_randomness sqlite3_api->randomness\n#define sqlite3_context_db_handle sqlite3_api->context_db_handle\n#define sqlite3_extended_result_codes sqlite3_api->extended_result_codes\n#define sqlite3_limit sqlite3_api->limit\n#define sqlite3_next_stmt sqlite3_api->next_stmt\n#define sqlite3_sql sqlite3_api->sql\n#define sqlite3_status sqlite3_api->status\n#define sqlite3_backup_finish sqlite3_api->backup_finish\n#define sqlite3_backup_init sqlite3_api->backup_init\n#define sqlite3_backup_pagecount sqlite3_api->backup_pagecount\n#define sqlite3_backup_remaining sqlite3_api->backup_remaining\n#define sqlite3_backup_step sqlite3_api->backup_step\n#define sqlite3_compileoption_get sqlite3_api->compileoption_get\n#define sqlite3_compileoption_used sqlite3_api->compileoption_used\n#define sqlite3_create_function_v2 sqlite3_api->create_function_v2\n#define sqlite3_db_config sqlite3_api->db_config\n#define sqlite3_db_mutex sqlite3_api->db_mutex\n#define sqlite3_db_status sqlite3_api->db_status\n#define sqlite3_extended_errcode sqlite3_api->extended_errcode\n#define sqlite3_log sqlite3_api->log\n#define sqlite3_soft_heap_limit64 sqlite3_api->soft_heap_limit64\n#define sqlite3_sourceid sqlite3_api->sourceid\n#define sqlite3_stmt_status sqlite3_api->stmt_status\n#define sqlite3_strnicmp sqlite3_api->strnicmp\n#define sqlite3_unlock_notify sqlite3_api->unlock_notify\n#define sqlite3_wal_autocheckpoint sqlite3_api->wal_autocheckpoint\n#define sqlite3_wal_checkpoint sqlite3_api->wal_checkpoint\n#define sqlite3_wal_hook sqlite3_api->wal_hook\n#define sqlite3_blob_reopen sqlite3_api->blob_reopen\n#define sqlite3_vtab_config sqlite3_api->vtab_config\n#define sqlite3_vtab_on_conflict sqlite3_api->vtab_on_conflict\n/* Version 3.7.16 and later */\n#define sqlite3_close_v2 sqlite3_api->close_v2\n#define sqlite3_db_filename sqlite3_api->db_filename\n#define sqlite3_db_readonly sqlite3_api->db_readonly\n#define sqlite3_db_release_memory sqlite3_api->db_release_memory\n#define sqlite3_errstr sqlite3_api->errstr\n#define sqlite3_stmt_busy sqlite3_api->stmt_busy\n#define sqlite3_stmt_readonly sqlite3_api->stmt_readonly\n#define sqlite3_stricmp sqlite3_api->stricmp\n#define sqlite3_uri_boolean sqlite3_api->uri_boolean\n#define sqlite3_uri_int64 sqlite3_api->uri_int64\n#define sqlite3_uri_parameter sqlite3_api->uri_parameter\n#define sqlite3_uri_vsnprintf sqlite3_api->xvsnprintf\n#define sqlite3_wal_checkpoint_v2 sqlite3_api->wal_checkpoint_v2\n/* Version 3.8.7 and later */\n#define sqlite3_auto_extension sqlite3_api->auto_extension\n#define sqlite3_bind_blob64 sqlite3_api->bind_blob64\n#define sqlite3_bind_text64 sqlite3_api->bind_text64\n#define sqlite3_cancel_auto_extension sqlite3_api->cancel_auto_extension\n#define sqlite3_load_extension sqlite3_api->load_extension\n#define sqlite3_malloc64 sqlite3_api->malloc64\n#define sqlite3_msize sqlite3_api->msize\n#define sqlite3_realloc64 sqlite3_api->realloc64\n#define sqlite3_reset_auto_extension sqlite3_api->reset_auto_extension\n#define sqlite3_result_blob64 sqlite3_api->result_blob64\n#define sqlite3_result_text64 sqlite3_api->result_text64\n#define sqlite3_strglob sqlite3_api->strglob\n/* Version 3.8.11 and later */\n#define sqlite3_value_dup sqlite3_api->value_dup\n#define sqlite3_value_free sqlite3_api->value_free\n#define sqlite3_result_zeroblob64 sqlite3_api->result_zeroblob64\n#define sqlite3_bind_zeroblob64 sqlite3_api->bind_zeroblob64\n/* Version 3.9.0 and later */\n#define sqlite3_value_subtype sqlite3_api->value_subtype\n#define sqlite3_result_subtype sqlite3_api->result_subtype\n/* Version 3.10.0 and later */\n#define sqlite3_status64 sqlite3_api->status64\n#define sqlite3_strlike sqlite3_api->strlike\n#define sqlite3_db_cacheflush sqlite3_api->db_cacheflush\n/* Version 3.12.0 and later */\n#define sqlite3_system_errno sqlite3_api->system_errno\n/* Version 3.14.0 and later */\n#define sqlite3_trace_v2 sqlite3_api->trace_v2\n#define sqlite3_expanded_sql sqlite3_api->expanded_sql\n/* Version 3.18.0 and later */\n#define sqlite3_set_last_insert_rowid sqlite3_api->set_last_insert_rowid\n/* Version 3.20.0 and later */\n#define sqlite3_prepare_v3 sqlite3_api->prepare_v3\n#define sqlite3_prepare16_v3 sqlite3_api->prepare16_v3\n#define sqlite3_bind_pointer sqlite3_api->bind_pointer\n#define sqlite3_result_pointer sqlite3_api->result_pointer\n#define sqlite3_value_pointer sqlite3_api->value_pointer\n/* Version 3.22.0 and later */\n#define sqlite3_vtab_nochange sqlite3_api->vtab_nochange\n#define sqlite3_value_nochange sqlite3_api->value_nochange\n#define sqlite3_vtab_collation sqlite3_api->vtab_collation\n/* Version 3.24.0 and later */\n#define sqlite3_keyword_count sqlite3_api->keyword_count\n#define sqlite3_keyword_name sqlite3_api->keyword_name\n#define sqlite3_keyword_check sqlite3_api->keyword_check\n#define sqlite3_str_new sqlite3_api->str_new\n#define sqlite3_str_finish sqlite3_api->str_finish\n#define sqlite3_str_appendf sqlite3_api->str_appendf\n#define sqlite3_str_vappendf sqlite3_api->str_vappendf\n#define sqlite3_str_append sqlite3_api->str_append\n#define sqlite3_str_appendall sqlite3_api->str_appendall\n#define sqlite3_str_appendchar sqlite3_api->str_appendchar\n#define sqlite3_str_reset sqlite3_api->str_reset\n#define sqlite3_str_errcode sqlite3_api->str_errcode\n#define sqlite3_str_length sqlite3_api->str_length\n#define sqlite3_str_value sqlite3_api->str_value\n/* Version 3.25.0 and later */\n#define sqlite3_create_window_function sqlite3_api->create_window_function\n/* Version 3.26.0 and later */\n#define sqlite3_normalized_sql sqlite3_api->normalized_sql\n/* Version 3.28.0 and later */\n#define sqlite3_stmt_isexplain sqlite3_api->stmt_isexplain\n#define sqlite3_value_frombind sqlite3_api->value_frombind\n/* Version 3.30.0 and later */\n#define sqlite3_drop_modules sqlite3_api->drop_modules\n/* Version 3.31.0 and later */\n#define sqlite3_hard_heap_limit64 sqlite3_api->hard_heap_limit64\n#define sqlite3_uri_key sqlite3_api->uri_key\n#define sqlite3_filename_database sqlite3_api->filename_database\n#define sqlite3_filename_journal sqlite3_api->filename_journal\n#define sqlite3_filename_wal sqlite3_api->filename_wal\n/* Version 3.32.0 and later */\n#define sqlite3_create_filename sqlite3_api->create_filename\n#define sqlite3_free_filename sqlite3_api->free_filename\n#define sqlite3_database_file_object sqlite3_api->database_file_object\n/* Version 3.34.0 and later */\n#define sqlite3_txn_state sqlite3_api->txn_state\n/* Version 3.36.1 and later */\n#define sqlite3_changes64 sqlite3_api->changes64\n#define sqlite3_total_changes64 sqlite3_api->total_changes64\n/* Version 3.37.0 and later */\n#define sqlite3_autovacuum_pages sqlite3_api->autovacuum_pages\n/* Version 3.38.0 and later */\n#define sqlite3_error_offset sqlite3_api->error_offset\n#define sqlite3_vtab_rhs_value sqlite3_api->vtab_rhs_value\n#define sqlite3_vtab_distinct sqlite3_api->vtab_distinct\n#define sqlite3_vtab_in sqlite3_api->vtab_in\n#define sqlite3_vtab_in_first sqlite3_api->vtab_in_first\n#define sqlite3_vtab_in_next sqlite3_api->vtab_in_next\n/* Version 3.39.0 and later */\n#ifndef SQLITE_OMIT_DESERIALIZE\n#define sqlite3_deserialize sqlite3_api->deserialize\n#define sqlite3_serialize sqlite3_api->serialize\n#endif\n#define sqlite3_db_name sqlite3_api->db_name\n/* Version 3.40.0 and later */\n#define sqlite3_value_encoding sqlite3_api->value_encoding\n/* Version 3.41.0 and later */\n#define sqlite3_is_interrupted sqlite3_api->is_interrupted\n/* Version 3.43.0 and later */\n#define sqlite3_stmt_explain sqlite3_api->stmt_explain\n/* Version 3.44.0 and later */\n#define sqlite3_get_clientdata sqlite3_api->get_clientdata\n#define sqlite3_set_clientdata sqlite3_api->set_clientdata\n/* Version 3.50.0 and later */\n#define sqlite3_setlk_timeout sqlite3_api->setlk_timeout\n#endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */\n\n#if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)\n /* This case when the file really is being compiled as a loadable\n ** extension */\n# define SQLITE_EXTENSION_INIT1 const sqlite3_api_routines *sqlite3_api=0;\n# define SQLITE_EXTENSION_INIT2(v) sqlite3_api=v;\n# define SQLITE_EXTENSION_INIT3 \\\n extern const sqlite3_api_routines *sqlite3_api;\n#else\n /* This case when the file is being statically linked into the\n ** application */\n# define SQLITE_EXTENSION_INIT1 /*no-op*/\n# define SQLITE_EXTENSION_INIT2(v) (void)v; /* unused parameter */\n# define SQLITE_EXTENSION_INIT3 /*no-op*/\n#endif\n\n#endif /* SQLITE3EXT_H */\n" }, { "path": "src/sqlite3vfs.h", "content": "#ifndef SQLITE3VFS_H\n#define SQLITE3VFS_H\n\n#ifdef SQLITE3VFS_LOADABLE_EXT\n#include \"sqlite3ext.h\"\n#else\n#include \"sqlite3-binding.h\"\n#endif\n\ntypedef struct s3vfsFile {\n sqlite3_file base; /* IO methods */\n sqlite3_uint64 id; /* Go object id */\n} s3vfsFile;\n\nint s3vfsNew(char* name, int maxPathName);\n\nint s3vfsClose(sqlite3_file*);\nint s3vfsRead(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);\nint s3vfsWrite(sqlite3_file*,const void*,int iAmt, sqlite3_int64 iOfst);\nint s3vfsTruncate(sqlite3_file*, sqlite3_int64 size);\nint s3vfsSync(sqlite3_file*, int flags);\nint s3vfsFileSize(sqlite3_file*, sqlite3_int64 *pSize);\nint s3vfsLock(sqlite3_file*, int);\nint s3vfsUnlock(sqlite3_file*, int);\nint s3vfsCheckReservedLock(sqlite3_file*, int *pResOut);\nint s3vfsFileControl(sqlite3_file*, int op, void *pArg);\nint s3vfsSectorSize(sqlite3_file*);\nint s3vfsDeviceCharacteristics(sqlite3_file*);\nint s3vfsShmMap(sqlite3_file*, int iPg, int pgsz, int, void volatile**);\nint s3vfsShmLock(sqlite3_file*, int offset, int n, int flags);\nvoid s3vfsShmBarrier(sqlite3_file*);\nint s3vfsShmUnmap(sqlite3_file*, int deleteFlag);\nint s3vfsFetch(sqlite3_file*, sqlite3_int64 iOfst, int iAmt, void **pp);\nint s3vfsUnfetch(sqlite3_file*, sqlite3_int64 iOfst, void *p);\n\n\nint s3vfsOpen(sqlite3_vfs*, const char *, sqlite3_file*, int , int *);\nint s3vfsDelete(sqlite3_vfs*, const char *, int);\nint s3vfsAccess(sqlite3_vfs*, const char *, int, int *);\nint s3vfsFullPathname(sqlite3_vfs*, const char *zName, int, char *zOut);\nvoid *s3vfsDlOpen(sqlite3_vfs*, const char *zFilename);\nvoid s3vfsDlError(sqlite3_vfs*, int nByte, char *zErrMsg);\nvoid (*s3vfsDlSym(sqlite3_vfs *pVfs, void *p, const char*zSym))(void);\nvoid s3vfsDlClose(sqlite3_vfs*, void*);\nint s3vfsRandomness(sqlite3_vfs*, int nByte, char *zOut);\nint s3vfsSleep(sqlite3_vfs*, int microseconds);\nint s3vfsCurrentTime(sqlite3_vfs*, double*);\nint s3vfsGetLastError(sqlite3_vfs*, int, char *);\nint s3vfsCurrentTimeInt64(sqlite3_vfs*, sqlite3_int64*);\n\nconst extern sqlite3_io_methods s3vfs_io_methods;\n\n#endif /* SQLITE3_VFS */\n" }, { "path": "store.go", "content": "package litestream\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"log/slog\"\n\t\"path/filepath\"\n\t\"slices\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\t\"golang.org/x/sync/errgroup\"\n)\n\nvar (\n\t// ErrNoCompaction is returned when no new files are available from the previous level.\n\tErrNoCompaction = errors.New(\"no compaction\")\n\n\t// ErrCompactionTooEarly is returned when a compaction is attempted too soon\n\t// since the last compaction time. This is used to prevent frequent\n\t// re-compaction when restarting the process.\n\tErrCompactionTooEarly = errors.New(\"compaction too early\")\n\n\t// ErrTxNotAvailable is returned when a transaction does not exist.\n\tErrTxNotAvailable = errors.New(\"transaction not available\")\n\n\t// ErrDBNotReady is a sentinel for errors.Is() compatibility.\n\tErrDBNotReady = &DBNotReadyError{}\n\n\t// ErrShutdownInterrupted is returned when the shutdown sync retry loop\n\t// is interrupted by a done channel signal (e.g., second Ctrl+C).\n\tErrShutdownInterrupted = errors.New(\"shutdown sync interrupted\")\n\n\tErrDatabaseNotFound = errors.New(\"database not found\")\n\tErrDatabaseNotOpen = errors.New(\"database not open\")\n)\n\n// DBNotReadyError is returned when an operation is attempted before the\n// database has been initialized (e.g., page size not yet known).\ntype DBNotReadyError struct {\n\tReason string\n}\n\nfunc (e *DBNotReadyError) Error() string {\n\tif e.Reason != \"\" {\n\t\treturn \"db not ready: \" + e.Reason\n\t}\n\treturn \"db not ready\"\n}\n\nfunc (e *DBNotReadyError) Is(target error) bool {\n\t_, ok := target.(*DBNotReadyError)\n\treturn ok\n}\n\n// Store defaults\nconst (\n\tDefaultSnapshotInterval = 24 * time.Hour\n\tDefaultSnapshotRetention = 24 * time.Hour\n\n\tDefaultRetention = 24 * time.Hour\n\tDefaultRetentionCheckInterval = 1 * time.Hour\n\n\t// DefaultL0Retention is the default time that L0 files are kept around\n\t// after they have been compacted into L1 files.\n\tDefaultL0Retention = 5 * time.Minute\n\t// DefaultL0RetentionCheckInterval controls how frequently L0 retention is\n\t// enforced. This interval should be more frequent than the L1 compaction\n\t// interval so that VFS read replicas have time to observe new files.\n\tDefaultL0RetentionCheckInterval = 15 * time.Second\n\n\t// DefaultHeartbeatCheckInterval controls how frequently the heartbeat\n\t// monitor checks if heartbeat pings should be sent.\n\tDefaultHeartbeatCheckInterval = 15 * time.Second\n\n\t// DefaultDBInitTimeout is the maximum time to wait for a database to be\n\t// initialized (page size known) before logging a warning.\n\tDefaultDBInitTimeout = 30 * time.Second\n)\n\n// Store represents the top-level container for databases.\n//\n// It manages async background tasks like compactions so that the system\n// is not overloaded by too many concurrent tasks.\ntype Store struct {\n\tmu sync.Mutex\n\tdbs []*DB\n\tlevels CompactionLevels\n\n\twg sync.WaitGroup\n\tctx context.Context\n\tcancel func()\n\tdone <-chan struct{}\n\n\t// The frequency of snapshots.\n\tSnapshotInterval time.Duration\n\t// The duration of time that snapshots are kept before being deleted.\n\tSnapshotRetention time.Duration\n\n\t// The duration that L0 files are kept after being compacted into L1.\n\tL0Retention time.Duration\n\t// How often to check for expired L0 files.\n\tL0RetentionCheckInterval time.Duration\n\n\t// If true, compaction is run in the background according to compaction levels.\n\tCompactionMonitorEnabled bool\n\n\t// If true, verify TXID consistency at destination level after each compaction.\n\tVerifyCompaction bool\n\n\t// RetentionEnabled controls whether Litestream actively deletes old files\n\t// during retention enforcement. When false, cloud provider lifecycle\n\t// policies handle retention instead. Local file cleanup still occurs.\n\tRetentionEnabled bool\n\n\t// Shutdown sync retry settings.\n\tShutdownSyncTimeout time.Duration\n\tShutdownSyncInterval time.Duration\n\n\t// How often to check if heartbeat pings should be sent.\n\tHeartbeatCheckInterval time.Duration\n\n\t// Heartbeat client for health check pings. Sends pings only when\n\t// all databases have synced successfully within the heartbeat interval.\n\tHeartbeat *HeartbeatClient\n\n\t// heartbeatMonitorRunning tracks whether the heartbeat monitor goroutine is running.\n\theartbeatMonitorRunning bool\n\n\t// How often to run validation checks. Zero disables periodic validation.\n\tValidationInterval time.Duration\n\n\tLogger *slog.Logger\n}\n\nfunc NewStore(dbs []*DB, levels CompactionLevels) *Store {\n\ts := &Store{\n\t\tdbs: dbs,\n\t\tlevels: levels,\n\n\t\tSnapshotInterval: DefaultSnapshotInterval,\n\t\tSnapshotRetention: DefaultSnapshotRetention,\n\t\tL0Retention: DefaultL0Retention,\n\t\tL0RetentionCheckInterval: DefaultL0RetentionCheckInterval,\n\t\tCompactionMonitorEnabled: true,\n\t\tRetentionEnabled: true,\n\t\tShutdownSyncTimeout: DefaultShutdownSyncTimeout,\n\t\tShutdownSyncInterval: DefaultShutdownSyncInterval,\n\t\tHeartbeatCheckInterval: DefaultHeartbeatCheckInterval,\n\t\tLogger: slog.Default().With(LogKeySystem, LogSystemStore),\n\t}\n\n\tfor _, db := range dbs {\n\t\tdb.SetLogger(s.Logger.With(LogKeyDB, filepath.Base(db.Path())))\n\t\tdb.L0Retention = s.L0Retention\n\t\tdb.ShutdownSyncTimeout = s.ShutdownSyncTimeout\n\t\tdb.ShutdownSyncInterval = s.ShutdownSyncInterval\n\t\tdb.VerifyCompaction = s.VerifyCompaction\n\t\tdb.RetentionEnabled = s.RetentionEnabled\n\t}\n\ts.ctx, s.cancel = context.WithCancel(context.Background())\n\treturn s\n}\n\nfunc (s *Store) Open(ctx context.Context) error {\n\tif err := s.levels.Validate(); err != nil {\n\t\treturn err\n\t}\n\n\tg, ctx := errgroup.WithContext(ctx)\n\tg.SetLimit(50)\n\tfor _, db := range s.dbs {\n\t\tdb := db\n\t\tg.Go(func() error {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn ctx.Err()\n\t\t\tdefault:\n\t\t\t\treturn db.Open()\n\t\t\t}\n\t\t})\n\t}\n\tif err := g.Wait(); err != nil {\n\t\treturn err\n\t}\n\n\t// Start monitors for compactions & snapshots.\n\tif s.CompactionMonitorEnabled {\n\t\t// Start compaction monitors for all levels except L0.\n\t\tfor _, lvl := range s.levels {\n\t\t\tlvl := lvl\n\t\t\tif lvl.Level == 0 {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\ts.wg.Add(1)\n\t\t\tgo func() {\n\t\t\t\tdefer s.wg.Done()\n\t\t\t\ts.monitorCompactionLevel(s.ctx, lvl)\n\t\t\t}()\n\t\t}\n\n\t\t// Start snapshot monitor for snapshots.\n\t\ts.wg.Add(1)\n\t\tgo func() {\n\t\t\tdefer s.wg.Done()\n\t\t\ts.monitorCompactionLevel(s.ctx, s.SnapshotLevel())\n\t\t}()\n\t}\n\n\tif s.L0Retention > 0 && s.L0RetentionCheckInterval > 0 {\n\t\ts.wg.Add(1)\n\t\tgo func() {\n\t\t\tdefer s.wg.Done()\n\t\t\ts.monitorL0Retention(s.ctx)\n\t\t}()\n\t}\n\n\t// Start heartbeat monitor if any database has heartbeat configured.\n\ts.startHeartbeatMonitorIfNeeded()\n\n\t// Start validation monitor if configured.\n\tif s.ValidationInterval > 0 {\n\t\ts.wg.Add(1)\n\t\tgo func() {\n\t\t\tdefer s.wg.Done()\n\t\t\ts.monitorValidation(s.ctx)\n\t\t}()\n\t}\n\n\treturn nil\n}\n\nfunc (s *Store) Close(ctx context.Context) (err error) {\n\ts.mu.Lock()\n\tdbs := slices.Clone(s.dbs)\n\ts.mu.Unlock()\n\n\tfor _, db := range dbs {\n\t\tif e := db.Close(ctx); e != nil {\n\t\t\tif errors.Is(e, ErrShutdownInterrupted) {\n\t\t\t\tif err == nil {\n\t\t\t\t\terr = e\n\t\t\t\t}\n\t\t\t} else if err == nil || errors.Is(err, ErrShutdownInterrupted) {\n\t\t\t\terr = e\n\t\t\t}\n\t\t}\n\t}\n\n\t// Cancel and wait for background tasks to complete.\n\ts.cancel()\n\ts.wg.Wait()\n\n\treturn err\n}\n\nfunc (s *Store) DBs() []*DB {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\treturn slices.Clone(s.dbs)\n}\n\n// RegisterDB registers a new database with the store and starts monitoring it.\nfunc (s *Store) RegisterDB(db *DB) error {\n\tif db == nil {\n\t\treturn fmt.Errorf(\"db required\")\n\t}\n\n\t// First check: see if database already exists\n\ts.mu.Lock()\n\tfor _, existing := range s.dbs {\n\t\tif existing.Path() == db.Path() {\n\t\t\ts.mu.Unlock()\n\t\t\treturn nil\n\t\t}\n\t}\n\ts.mu.Unlock()\n\n\t// Apply store-wide settings before opening the database.\n\tdb.SetLogger(s.Logger.With(LogKeyDB, filepath.Base(db.Path())))\n\tdb.L0Retention = s.L0Retention\n\tdb.ShutdownSyncTimeout = s.ShutdownSyncTimeout\n\tdb.ShutdownSyncInterval = s.ShutdownSyncInterval\n\tdb.VerifyCompaction = s.VerifyCompaction\n\tdb.RetentionEnabled = s.RetentionEnabled\n\tdb.Done = s.done\n\n\t// Open the database without holding the lock to avoid blocking other operations.\n\t// The double-check pattern below handles the race condition.\n\tif err := db.Open(); err != nil {\n\t\treturn fmt.Errorf(\"open db: %w\", err)\n\t}\n\n\t// Second check: verify database wasn't added by another goroutine while we were opening.\n\t// If it was, close our instance and return without error.\n\ts.mu.Lock()\n\n\tfor _, existing := range s.dbs {\n\t\tif existing.Path() == db.Path() {\n\t\t\t// Another goroutine added this database while we were opening.\n\t\t\t// Release lock before closing to avoid potential deadlock.\n\t\t\ts.mu.Unlock()\n\t\t\tif err := db.Close(context.Background()); err != nil {\n\t\t\t\tdb.Logger.Error(\"close duplicate db\", \"path\", db.Path(), \"error\", err)\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t}\n\n\ts.dbs = append(s.dbs, db)\n\ts.mu.Unlock()\n\n\t// Start heartbeat monitor if heartbeat is configured and monitor isn't running.\n\ts.startHeartbeatMonitorIfNeeded()\n\n\treturn nil\n}\n\n// UnregisterDB stops monitoring the database at the provided path and closes it.\nfunc (s *Store) UnregisterDB(ctx context.Context, path string) error {\n\tif path == \"\" {\n\t\treturn fmt.Errorf(\"db path required\")\n\t}\n\n\ts.mu.Lock()\n\n\tidx := -1\n\tvar db *DB\n\tfor i, existing := range s.dbs {\n\t\tif existing.Path() == path {\n\t\t\tidx = i\n\t\t\tdb = existing\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif db == nil {\n\t\ts.mu.Unlock()\n\t\treturn nil\n\t}\n\n\ts.dbs = slices.Delete(s.dbs, idx, idx+1)\n\ts.mu.Unlock()\n\n\tif err := db.Close(ctx); err != nil {\n\t\treturn fmt.Errorf(\"close db: %w\", err)\n\t}\n\n\treturn nil\n}\n\n// EnableDB starts replication for a registered database.\n// The context is checked for cancellation before opening.\n// Note: db.Open() itself does not support cancellation.\nfunc (s *Store) EnableDB(ctx context.Context, path string) error {\n\tdb := s.FindDB(path)\n\tif db == nil {\n\t\treturn fmt.Errorf(\"database not found: %s\", path)\n\t}\n\n\tif db.IsOpen() {\n\t\treturn fmt.Errorf(\"database already enabled: %s\", path)\n\t}\n\n\t// Check for cancellation before starting open\n\tif err := ctx.Err(); err != nil {\n\t\treturn fmt.Errorf(\"enable database: %w\", err)\n\t}\n\n\tif err := db.Open(); err != nil {\n\t\treturn fmt.Errorf(\"open database: %w\", err)\n\t}\n\n\treturn nil\n}\n\n// DisableDB stops replication for a database.\nfunc (s *Store) DisableDB(ctx context.Context, path string) error {\n\tdb := s.FindDB(path)\n\tif db == nil {\n\t\treturn fmt.Errorf(\"database not found: %s\", path)\n\t}\n\n\tif !db.IsOpen() {\n\t\treturn fmt.Errorf(\"database already disabled: %s\", path)\n\t}\n\n\tif err := db.Close(ctx); err != nil {\n\t\treturn fmt.Errorf(\"close database: %w\", err)\n\t}\n\n\treturn nil\n}\n\n// SyncDBResult holds the result of a sync operation.\ntype SyncDBResult struct {\n\tTXID uint64\n\tReplicatedTXID uint64\n\tChanged bool\n}\n\n// SyncDB forces an immediate sync for a database. If wait is true, blocks\n// until both WAL-to-LTX and LTX-to-remote sync complete. If wait is false,\n// only performs the WAL-to-LTX sync and lets the replica monitor handle upload.\n// The timeout is best-effort as internal lock acquisition is not context-aware.\nfunc (s *Store) SyncDB(ctx context.Context, path string, wait bool) (SyncDBResult, error) {\n\tdb := s.FindDB(path)\n\tif db == nil {\n\t\treturn SyncDBResult{}, fmt.Errorf(\"%w: %s\", ErrDatabaseNotFound, path)\n\t}\n\n\tif !db.IsOpen() {\n\t\treturn SyncDBResult{}, fmt.Errorf(\"%w: %s\", ErrDatabaseNotOpen, path)\n\t}\n\n\t_, beforeTXID, err := db.MaxLTX()\n\tif err != nil {\n\t\treturn SyncDBResult{}, fmt.Errorf(\"read position before sync: %w\", err)\n\t}\n\n\tif wait {\n\t\tif err := db.SyncAndWait(ctx); err != nil {\n\t\t\treturn SyncDBResult{}, fmt.Errorf(\"sync database: %w\", err)\n\t\t}\n\t} else {\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\treturn SyncDBResult{}, fmt.Errorf(\"sync database: %w\", err)\n\t\t}\n\t}\n\n\t_, afterTXID, err := db.MaxLTX()\n\tif err != nil {\n\t\treturn SyncDBResult{}, fmt.Errorf(\"read position after sync: %w\", err)\n\t}\n\n\tvar replicatedTXID uint64\n\tif db.Replica != nil {\n\t\treplicatedTXID = uint64(db.Replica.Pos().TXID)\n\t}\n\n\treturn SyncDBResult{\n\t\tTXID: uint64(afterTXID),\n\t\tReplicatedTXID: replicatedTXID,\n\t\tChanged: afterTXID > beforeTXID,\n\t}, nil\n}\n\n// FindDB returns the database with the given path.\nfunc (s *Store) FindDB(path string) *DB {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\n\tfor _, db := range s.dbs {\n\t\tif db.Path() == path {\n\t\t\treturn db\n\t\t}\n\t}\n\treturn nil\n}\n\n// SetL0Retention updates the retention window for L0 files and propagates it to\n// all managed databases.\nfunc (s *Store) SetL0Retention(d time.Duration) {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\ts.L0Retention = d\n\tfor _, db := range s.dbs {\n\t\tdb.L0Retention = d\n\t}\n}\n\n// SetDone sets the done channel used for interrupt handling during shutdown\n// and propagates it to all managed databases.\nfunc (s *Store) SetDone(done <-chan struct{}) {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\ts.done = done\n\tfor _, db := range s.dbs {\n\t\tdb.Done = done\n\t}\n}\n\n// SetShutdownSyncTimeout updates the shutdown sync timeout and propagates it to\n// all managed databases.\nfunc (s *Store) SetShutdownSyncTimeout(d time.Duration) {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\ts.ShutdownSyncTimeout = d\n\tfor _, db := range s.dbs {\n\t\tdb.ShutdownSyncTimeout = d\n\t}\n}\n\n// SetShutdownSyncInterval updates the shutdown sync interval and propagates it to\n// all managed databases.\nfunc (s *Store) SetShutdownSyncInterval(d time.Duration) {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\ts.ShutdownSyncInterval = d\n\tfor _, db := range s.dbs {\n\t\tdb.ShutdownSyncInterval = d\n\t}\n}\n\n// SetVerifyCompaction updates the verify compaction flag and propagates it to\n// all managed databases.\nfunc (s *Store) SetVerifyCompaction(v bool) {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\ts.VerifyCompaction = v\n\tfor _, db := range s.dbs {\n\t\tdb.VerifyCompaction = v\n\t\tdb.compactor.VerifyCompaction = v\n\t}\n}\n\nfunc (s *Store) SetRetentionEnabled(v bool) {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\ts.RetentionEnabled = v\n\tfor _, db := range s.dbs {\n\t\tdb.RetentionEnabled = v\n\t\tdb.compactor.RetentionEnabled = v\n\t}\n}\n\n// SnapshotLevel returns a pseudo compaction level based on snapshot settings.\nfunc (s *Store) SnapshotLevel() *CompactionLevel {\n\treturn &CompactionLevel{\n\t\tLevel: SnapshotLevel,\n\t\tInterval: s.SnapshotInterval,\n\t}\n}\n\nfunc (s *Store) monitorCompactionLevel(ctx context.Context, lvl *CompactionLevel) {\n\ts.Logger.Info(\"starting compaction monitor\", \"level\", lvl.Level, \"interval\", lvl.Interval)\n\n\tretryDeadline := time.Time{}\n\ttimer := time.NewTimer(time.Nanosecond)\n\tdefer timer.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn\n\t\tcase <-timer.C:\n\t\t\t// proceed\n\t\t}\n\n\t\tnow := time.Now()\n\t\tnextDelay := time.Until(lvl.NextCompactionAt(now))\n\n\t\tvar notReadyDBs []string\n\n\t\tfor _, db := range s.DBs() {\n\t\t\tif !db.IsOpen() {\n\t\t\t\tcontinue // skip disabled DBs\n\t\t\t}\n\t\t\t_, err := s.CompactDB(ctx, db, lvl)\n\t\t\tswitch {\n\t\t\tcase errors.Is(err, ErrNoCompaction), errors.Is(err, ErrCompactionTooEarly):\n\t\t\t\tdb.Logger.Debug(\"no compaction\", \"level\", lvl.Level, \"path\", db.Path())\n\t\t\tcase errors.Is(err, ErrDBNotReady):\n\t\t\t\tdb.Logger.Debug(\"db not ready, skipping\", \"level\", lvl.Level, \"path\", db.Path(), \"error\", err)\n\t\t\t\tnotReadyDBs = append(notReadyDBs, db.Path())\n\t\t\tcase err != nil && !errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded):\n\t\t\t\tdb.Logger.Error(\"compaction failed\", \"level\", lvl.Level, \"error\", err)\n\t\t\t}\n\n\t\t\tif lvl.Level == SnapshotLevel {\n\t\t\t\tif err := s.EnforceSnapshotRetention(ctx, db); err != nil &&\n\t\t\t\t\t!errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded) {\n\t\t\t\t\tdb.Logger.Error(\"retention enforcement failed\", \"error\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\ttimedOut := !retryDeadline.IsZero() && now.After(retryDeadline)\n\t\tif len(notReadyDBs) > 0 && !timedOut {\n\t\t\tif retryDeadline.IsZero() {\n\t\t\t\tretryDeadline = now.Add(DefaultDBInitTimeout)\n\t\t\t}\n\t\t\tnextDelay = time.Second\n\t\t\ts.Logger.Debug(\"scheduling retry for unready dbs\", \"level\", lvl.Level)\n\t\t} else {\n\t\t\tif timedOut {\n\t\t\t\ts.Logger.Warn(\"timeout waiting for db initialization\",\n\t\t\t\t\t\"level\", lvl.Level,\n\t\t\t\t\t\"dbs\", notReadyDBs,\n\t\t\t\t\t\"timeout\", DefaultDBInitTimeout,\n\t\t\t\t\t\"hint\", \"database may have corrupted local state or blocked transactions; try removing -litestream directory and restarting\")\n\t\t\t}\n\t\t\tretryDeadline = time.Time{}\n\t\t}\n\n\t\tif nextDelay < 0 {\n\t\t\tnextDelay = 0\n\t\t}\n\t\ttimer.Reset(nextDelay)\n\t}\n}\n\nfunc (s *Store) monitorL0Retention(ctx context.Context) {\n\ts.Logger.Info(\"starting L0 retention monitor\", \"interval\", s.L0RetentionCheckInterval, \"retention\", s.L0Retention)\n\n\tticker := time.NewTicker(s.L0RetentionCheckInterval)\n\tdefer ticker.Stop()\n\nLOOP:\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\tbreak LOOP\n\t\tcase <-ticker.C:\n\t\t}\n\n\t\tfor _, db := range s.DBs() {\n\t\t\tif !db.IsOpen() {\n\t\t\t\tcontinue // skip disabled DBs\n\t\t\t}\n\t\t\tif err := db.EnforceL0RetentionByTime(ctx); err != nil {\n\t\t\t\tif errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tdb.Logger.Error(\"l0 retention enforcement failed\", \"path\", db.Path(), \"error\", err)\n\t\t\t}\n\t\t}\n\t}\n}\n\n// startHeartbeatMonitorIfNeeded starts the heartbeat monitor goroutine if:\n// - HeartbeatCheckInterval is configured\n// - Heartbeat is configured on the Store\n// - The monitor is not already running\nfunc (s *Store) startHeartbeatMonitorIfNeeded() {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\n\tif s.heartbeatMonitorRunning {\n\t\treturn\n\t}\n\tif s.HeartbeatCheckInterval <= 0 {\n\t\treturn\n\t}\n\tif !s.hasHeartbeatConfigLocked() {\n\t\treturn\n\t}\n\n\ts.heartbeatMonitorRunning = true\n\ts.wg.Add(1)\n\tgo func() {\n\t\tdefer s.wg.Done()\n\t\ts.monitorHeartbeats(s.ctx)\n\t}()\n}\n\n// hasHeartbeatConfigLocked returns true if heartbeat is configured on the Store.\n// Must be called with s.mu held.\nfunc (s *Store) hasHeartbeatConfigLocked() bool {\n\treturn s.Heartbeat != nil && s.Heartbeat.URL != \"\"\n}\n\n// monitorHeartbeats periodically checks if heartbeat pings should be sent.\n// Heartbeat pings are only sent when ALL databases have synced successfully\n// within the heartbeat interval.\nfunc (s *Store) monitorHeartbeats(ctx context.Context) {\n\ts.Logger.Info(\"starting heartbeat monitor\", \"interval\", s.HeartbeatCheckInterval)\n\n\tticker := time.NewTicker(s.HeartbeatCheckInterval)\n\tdefer ticker.Stop()\n\nLOOP:\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\tbreak LOOP\n\t\tcase <-ticker.C:\n\t\t}\n\n\t\ts.sendHeartbeatIfNeeded(ctx)\n\t}\n}\n\n// sendHeartbeatIfNeeded sends a heartbeat ping if:\n// - Heartbeat is configured on the Store\n// - Enough time has passed since the last ping attempt\n// - ALL databases have synced successfully within the heartbeat interval\nfunc (s *Store) sendHeartbeatIfNeeded(ctx context.Context) {\n\thb := s.Heartbeat\n\tif hb == nil || hb.URL == \"\" {\n\t\treturn\n\t}\n\n\tif !hb.ShouldPing() {\n\t\treturn\n\t}\n\n\t// Check if all databases are healthy (synced within the heartbeat interval).\n\t// A database is healthy if it synced within the heartbeat interval.\n\thealthySince := time.Now().Add(-hb.Interval)\n\tif !s.allDatabasesHealthy(healthySince) {\n\t\treturn\n\t}\n\n\t// Record ping attempt time before making the request to ensure we respect\n\t// the configured interval even if the ping fails. This prevents rapid\n\t// retries that could overwhelm the endpoint.\n\thb.RecordPing()\n\n\tif err := hb.Ping(ctx); err != nil {\n\t\ts.Logger.Error(\"heartbeat ping failed\", \"url\", hb.URL, \"error\", err)\n\t\treturn\n\t}\n\n\ts.Logger.Debug(\"heartbeat ping sent\", \"url\", hb.URL)\n}\n\n// allDatabasesHealthy returns true if all databases have synced successfully\n// since the given time. Returns false if there are no databases or no enabled databases.\nfunc (s *Store) allDatabasesHealthy(since time.Time) bool {\n\tdbs := s.DBs()\n\tif len(dbs) == 0 {\n\t\treturn false\n\t}\n\n\tenabledCount := 0\n\tfor _, db := range dbs {\n\t\tif !db.IsOpen() {\n\t\t\tcontinue // skip disabled DBs\n\t\t}\n\t\tenabledCount++\n\t\tlastSync := db.LastSuccessfulSyncAt()\n\t\tif lastSync.IsZero() || lastSync.Before(since) {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn enabledCount > 0\n}\n\n// CompactDB performs a compaction or snapshot for a given database on a single destination level.\n// This function will only proceed if a compaction has not occurred before the last compaction time.\nfunc (s *Store) CompactDB(ctx context.Context, db *DB, lvl *CompactionLevel) (*ltx.FileInfo, error) {\n\t// Skip if database is not yet initialized (page size unknown).\n\tif db.PageSize() == 0 {\n\t\treturn nil, &DBNotReadyError{Reason: \"page size not initialized\"}\n\t}\n\n\tdstLevel := lvl.Level\n\n\t// Ensure we are not re-compacting before the most recent compaction time.\n\tprevCompactionAt := lvl.PrevCompactionAt(time.Now())\n\tdstInfo, err := db.MaxLTXFileInfo(ctx, dstLevel)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"fetch dst level info: %w\", err)\n\t} else if dstInfo.CreatedAt.After(prevCompactionAt) {\n\t\treturn nil, ErrCompactionTooEarly\n\t}\n\n\t// Shortcut if this is a snapshot since we are not pulling from a previous level.\n\tif dstLevel == SnapshotLevel {\n\t\tinfo, err := db.Snapshot(ctx)\n\t\tif err != nil {\n\t\t\treturn info, err\n\t\t}\n\t\tdb.Logger.InfoContext(ctx, \"snapshot complete\", \"txid\", info.MaxTXID.String(), \"size\", info.Size)\n\t\treturn info, nil\n\t}\n\n\t// Fetch latest LTX files for both the source & destination so we can see if we need to make progress.\n\tsrcLevel := s.levels.PrevLevel(dstLevel)\n\tsrcInfo, err := db.MaxLTXFileInfo(ctx, srcLevel)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"fetch src level info: %w\", err)\n\t}\n\n\t// Skip if there are no new files to compact.\n\tif srcInfo.MaxTXID <= dstInfo.MinTXID {\n\t\treturn nil, ErrNoCompaction\n\t}\n\n\tinfo, err := db.Compact(ctx, dstLevel)\n\tif err != nil {\n\t\treturn info, err\n\t}\n\n\tdb.Logger.InfoContext(ctx, \"compaction complete\",\n\t\t\"level\", dstLevel,\n\t\tslog.Group(\"txid\",\n\t\t\t\"min\", info.MinTXID.String(),\n\t\t\t\"max\", info.MaxTXID.String(),\n\t\t),\n\t\t\"size\", info.Size,\n\t)\n\n\treturn info, nil\n}\n\n// EnforceSnapshotRetention removes old snapshots by timestamp and then\n// cleans up all lower levels based on minimum snapshot TXID.\nfunc (s *Store) EnforceSnapshotRetention(ctx context.Context, db *DB) error {\n\t// Enforce retention for the snapshot level.\n\tminSnapshotTXID, err := db.EnforceSnapshotRetention(ctx, time.Now().Add(-s.SnapshotRetention))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"enforce snapshot retention: %w\", err)\n\t}\n\n\t// We should also enforce retention for L0 on the same schedule as L1.\n\tfor _, lvl := range s.levels {\n\t\t// Skip L0 since it is enforced on a more frequent basis.\n\t\tif lvl.Level == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\tif err := db.EnforceRetentionByTXID(ctx, lvl.Level, minSnapshotTXID); err != nil {\n\t\t\treturn fmt.Errorf(\"enforce L%d retention: %w\", lvl.Level, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// ValidationResult holds the result of validating a replica's LTX files.\ntype ValidationResult struct {\n\tValid bool // true if no errors found\n\tErrors []ValidationError // all errors found\n}\n\n// Validate checks LTX file consistency across all databases and levels.\n// SnapshotLevel (9) is excluded since snapshots are not contiguous.\nfunc (s *Store) Validate(ctx context.Context) (*ValidationResult, error) {\n\tresult := &ValidationResult{Valid: true}\n\n\ts.mu.Lock()\n\tdbs := s.dbs\n\tlevels := s.levels\n\ts.mu.Unlock()\n\n\tfor _, db := range dbs {\n\t\tif db.Replica == nil {\n\t\t\tcontinue\n\t\t}\n\n\t\tfor _, lvl := range levels {\n\t\t\terrs, err := db.Replica.ValidateLevel(ctx, lvl.Level)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"validate level %d for %s: %w\", lvl.Level, db.Path(), err)\n\t\t\t}\n\t\t\tif len(errs) > 0 {\n\t\t\t\tresult.Valid = false\n\t\t\t\tresult.Errors = append(result.Errors, errs...)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn result, nil\n}\n\n// monitorValidation periodically runs validation checks on all databases.\nfunc (s *Store) monitorValidation(ctx context.Context) {\n\ts.Logger.Info(\"starting validation monitor\", \"interval\", s.ValidationInterval)\n\n\tticker := time.NewTicker(s.ValidationInterval)\n\tdefer ticker.Stop()\n\nLOOP:\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\tbreak LOOP\n\t\tcase <-ticker.C:\n\t\t}\n\n\t\tresult, err := s.Validate(ctx)\n\t\tif err != nil {\n\t\t\tif errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\ts.Logger.Error(\"validation check failed\", \"error\", err)\n\t\t\tcontinue\n\t\t}\n\n\t\tif !result.Valid {\n\t\t\tfor _, verr := range result.Errors {\n\t\t\t\ts.Logger.Warn(\"validation error detected\",\n\t\t\t\t\t\"level\", verr.Level,\n\t\t\t\t\t\"type\", verr.Type,\n\t\t\t\t\t\"message\", verr.Message,\n\t\t\t\t)\n\t\t\t}\n\t\t}\n\t}\n}\n" }, { "path": "store_compaction_remote_test.go", "content": "package litestream_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\n// TestStore_CompactDB_RemotePartialRead ensures that compactions do not rely on\n// immediately consistent remote reads. Some object stores (or custom replica\n// clients) can expose a newly written object before all bytes are available.\n// Without additional safeguards, compaction can read the partial object and\n// generate a corrupted snapshot which then fails during restore.\nfunc TestStore_CompactDB_RemotePartialRead(t *testing.T) {\n\tt.Parallel()\n\n\tctx := context.Background()\n\n\tclient := newDelayedReplicaClient(200 * time.Millisecond)\n\n\tdbPath := filepath.Join(t.TempDir(), \"db\")\n\tdb := litestream.NewDB(dbPath)\n\tdb.MonitorInterval = 0\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = client\n\tdb.Replica.MonitorEnabled = false\n\n\tlevels := litestream.CompactionLevels{\n\t\t{Level: 0},\n\t\t{Level: 1, Interval: time.Second},\n\t}\n\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\tstore.CompactionMonitorEnabled = false\n\n\tif err := store.Open(ctx); err != nil {\n\t\tt.Fatalf(\"open store: %v\", err)\n\t}\n\tdefer func() {\n\t\tif err := store.Close(ctx); err != nil {\n\t\t\tt.Fatalf(\"close store: %v\", err)\n\t\t}\n\t}()\n\n\tsqldb := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb)\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE t (id INTEGER PRIMARY KEY, val TEXT)`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\tinsert := func(start, end int) {\n\t\tfor i := start; i < end; i++ {\n\t\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO t (val) VALUES (?)`, fmt.Sprintf(\"value-%d\", i)); err != nil {\n\t\t\t\tt.Fatalf(\"insert %d: %v\", i, err)\n\t\t\t}\n\t\t}\n\t}\n\n\t// Generate two consecutive L0 files.\n\tinsert(0, 256)\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"sync #1: %v\", err)\n\t}\n\tif err := db.Replica.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"replica sync #1: %v\", err)\n\t}\n\n\tinsert(256, 512)\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"sync #2: %v\", err)\n\t}\n\tif err := db.Replica.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"replica sync #2: %v\", err)\n\t}\n\n\t// Compact level 0 into level 1. The delayed replica returns a partial view\n\t// for newly written files which previously resulted in corrupted snapshots.\n\tif _, err := store.CompactDB(ctx, db, levels[1]); err != nil {\n\t\tt.Fatalf(\"compact: %v\", err)\n\t}\n\n\tclient.waitForAvailability()\n\n\trestorePath := filepath.Join(t.TempDir(), \"restore.db\")\n\tif err := db.Replica.Restore(ctx, litestream.RestoreOptions{OutputPath: restorePath}); err != nil {\n\t\tt.Fatalf(\"restore: %v\", err)\n\t}\n}\n\n// delayedReplicaClient simulates an eventually-consistent object store where a\n// newly written object can be observed before all of its content is available.\n// Prior to availability, OpenLTXFile returns a valid but truncated LTX file.\ntype delayedReplicaClient struct {\n\tmu sync.Mutex\n\tfiles map[string]*delayedFile\n\tdelay time.Duration\n}\n\ntype delayedFile struct {\n\tlevel int\n\tmin ltx.TXID\n\tmax ltx.TXID\n\tdata []byte\n\tpartial []byte\n\tcreatedAt time.Time\n\tavailableAt time.Time\n}\n\nfunc newDelayedReplicaClient(delay time.Duration) *delayedReplicaClient {\n\treturn &delayedReplicaClient{\n\t\tfiles: make(map[string]*delayedFile),\n\t\tdelay: delay,\n\t}\n}\n\nfunc (c *delayedReplicaClient) Type() string { return \"delayed\" }\n\nfunc (c *delayedReplicaClient) Init(context.Context) error { return nil }\n\nfunc (c *delayedReplicaClient) SetLogger(*slog.Logger) {}\n\nfunc (c *delayedReplicaClient) key(level int, min, max ltx.TXID) string {\n\treturn fmt.Sprintf(\"%d:%s:%s\", level, min.String(), max.String())\n}\n\nfunc (c *delayedReplicaClient) LTXFiles(_ context.Context, level int, seek ltx.TXID, _ bool) (ltx.FileIterator, error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tinfos := make([]*ltx.FileInfo, 0, len(c.files))\n\tfor _, file := range c.files {\n\t\tif file.level != level {\n\t\t\tcontinue\n\t\t}\n\t\tif file.max < seek {\n\t\t\tcontinue\n\t\t}\n\t\tinfos = append(infos, <x.FileInfo{\n\t\t\tLevel: file.level,\n\t\t\tMinTXID: file.min,\n\t\t\tMaxTXID: file.max,\n\t\t\tSize: int64(len(file.data)),\n\t\t\tCreatedAt: file.createdAt,\n\t\t})\n\t}\n\n\treturn ltx.NewFileInfoSliceIterator(infos), nil\n}\n\nfunc (c *delayedReplicaClient) OpenLTXFile(_ context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tc.mu.Lock()\n\tfile, ok := c.files[c.key(level, minTXID, maxTXID)]\n\tc.mu.Unlock()\n\tif !ok {\n\t\treturn nil, os.ErrNotExist\n\t}\n\n\tdata := file.data\n\tif time.Now().Before(file.availableAt) && len(file.partial) > 0 {\n\t\tdata = file.partial\n\t}\n\n\tif offset > int64(len(data)) {\n\t\treturn io.NopCloser(bytes.NewReader(nil)), nil\n\t}\n\tdata = data[offset:]\n\tif size > 0 && size < int64(len(data)) {\n\t\tdata = data[:size]\n\t}\n\n\treturn io.NopCloser(bytes.NewReader(data)), nil\n}\n\nfunc (c *delayedReplicaClient) WriteLTXFile(_ context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\tdata, err := io.ReadAll(r)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tpartial, err := buildPartialSnapshot(data)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tinfo := <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tSize: int64(len(data)),\n\t\tCreatedAt: time.Now().UTC(),\n\t}\n\n\tc.mu.Lock()\n\tc.files[c.key(level, minTXID, maxTXID)] = &delayedFile{\n\t\tlevel: level,\n\t\tmin: minTXID,\n\t\tmax: maxTXID,\n\t\tdata: data,\n\t\tpartial: partial,\n\t\tcreatedAt: info.CreatedAt,\n\t\tavailableAt: time.Now().Add(c.delay),\n\t}\n\tc.mu.Unlock()\n\n\treturn info, nil\n}\n\nfunc (c *delayedReplicaClient) DeleteLTXFiles(_ context.Context, a []*ltx.FileInfo) error {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\tfor _, info := range a {\n\t\tdelete(c.files, c.key(info.Level, info.MinTXID, info.MaxTXID))\n\t}\n\treturn nil\n}\n\nfunc (c *delayedReplicaClient) DeleteAll(context.Context) error {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\tc.files = make(map[string]*delayedFile)\n\treturn nil\n}\n\nfunc (c *delayedReplicaClient) waitForAvailability() {\n\ttime.Sleep(c.delay)\n}\n\n// buildPartialSnapshot returns a valid LTX snapshot that only includes the\n// first portion of pages from data.\nfunc buildPartialSnapshot(data []byte) ([]byte, error) {\n\tdec := ltx.NewDecoder(bytes.NewReader(data))\n\tif err := dec.DecodeHeader(); err != nil {\n\t\treturn nil, err\n\t}\n\thdr := dec.Header()\n\n\tbuf := new(bytes.Buffer)\n\tenc, err := ltx.NewEncoder(buf)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif err := enc.EncodeHeader(hdr); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Copy only a subset of pages so the resulting snapshot is incomplete.\n\tmaxPages := int(hdr.Commit / 4)\n\tif maxPages < 1 {\n\t\tmaxPages = 1\n\t}\n\tvar page ltx.PageHeader\n\tpageBuf := make([]byte, hdr.PageSize)\n\tfor i := 0; i < maxPages; i++ {\n\t\tif err := dec.DecodePage(&page, pageBuf); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif err := enc.EncodePage(page, pageBuf); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif err := enc.Close(); err != nil {\n\t\treturn nil, err\n\t}\n\treturn buf.Bytes(), nil\n}\n" }, { "path": "store_test.go", "content": "package litestream_test\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\nfunc TestStore_CompactDB(t *testing.T) {\n\tt.Run(\"L1\", func(t *testing.T) {\n\t\tdb0, sqldb0 := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db0, sqldb0)\n\n\t\tdb1, sqldb1 := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db1, sqldb1)\n\n\t\tlevels := litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t\t{Level: 1, Interval: 1 * time.Second},\n\t\t\t{Level: 2, Interval: 500 * time.Millisecond},\n\t\t}\n\t\ts := litestream.NewStore([]*litestream.DB{db0, db1}, levels)\n\t\ts.CompactionMonitorEnabled = false\n\t\tif err := s.Open(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer s.Close(t.Context())\n\n\t\tif _, err := sqldb0.ExecContext(t.Context(), `CREATE TABLE t (id INT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := sqldb0.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (100)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db0.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db0.Replica.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t_, err := s.CompactDB(t.Context(), db0, levels[1])\n\t\trequire.NoError(t, err)\n\n\t\t// Re-compacting immediately should return an error indicating compaction\n\t\t// cannot proceed. This may be ErrCompactionTooEarly (detected timing conflict)\n\t\t// or ErrNoCompaction (no new files to compact). Both are valid outcomes\n\t\t// depending on whether we crossed a second boundary during the first compaction\n\t\t// (PrevCompactionAt truncates to seconds, causing edge cases at boundaries).\n\t\t_, err = s.CompactDB(t.Context(), db0, levels[1])\n\t\trequire.True(t,\n\t\t\terrors.Is(err, litestream.ErrCompactionTooEarly) || errors.Is(err, litestream.ErrNoCompaction),\n\t\t\t\"expected ErrCompactionTooEarly or ErrNoCompaction, got: %v\", err)\n\n\t\t// Re-compacting after the interval should show that there is nothing to compact.\n\t\ttime.Sleep(levels[1].Interval)\n\t\t_, err = s.CompactDB(t.Context(), db0, levels[1])\n\t\trequire.ErrorIs(t, err, litestream.ErrNoCompaction)\n\t})\n\n\tt.Run(\"Snapshot\", func(t *testing.T) {\n\t\tdb0, sqldb0 := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db0, sqldb0)\n\n\t\tlevels := litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t\t{Level: 1, Interval: 100 * time.Millisecond},\n\t\t\t{Level: 2, Interval: 500 * time.Millisecond},\n\t\t}\n\t\ts := litestream.NewStore([]*litestream.DB{db0}, levels)\n\t\ts.CompactionMonitorEnabled = false\n\t\tif err := s.Open(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer s.Close(t.Context())\n\n\t\tif _, err := sqldb0.ExecContext(t.Context(), `CREATE TABLE t (id INT);`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := sqldb0.ExecContext(t.Context(), `INSERT INTO t (id) VALUES (100)`); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db0.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if err := db0.Replica.Sync(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tif _, err := s.CompactDB(t.Context(), db0, s.SnapshotLevel()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Re-compacting immediately should return an error that there's nothing to compact.\n\t\tif _, err := s.CompactDB(t.Context(), db0, s.SnapshotLevel()); !errors.Is(err, litestream.ErrCompactionTooEarly) {\n\t\t\tt.Fatalf(\"unexpected error: %s\", err)\n\t\t}\n\t})\n\n\t// Regression test for GitHub issue #877: level 9 compaction fails with\n\t// \"page size not initialized yet\" error when attempted before DB initialization.\n\tt.Run(\"DBNotReady\", func(t *testing.T) {\n\t\tdb0 := testingutil.MustOpenDB(t)\n\t\tdefer testingutil.MustCloseDB(t, db0)\n\n\t\tlevels := litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t\t{Level: 1, Interval: 100 * time.Millisecond},\n\t\t}\n\t\ts := litestream.NewStore([]*litestream.DB{db0}, levels)\n\t\ts.CompactionMonitorEnabled = false\n\t\tif err := s.Open(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer s.Close(t.Context())\n\n\t\t// Attempt snapshot before DB is initialized (page size not set).\n\t\t// This reproduces the timing issue where level 9 compaction fires\n\t\t// immediately at startup before db.Sync() has been called.\n\t\tif _, err := s.CompactDB(t.Context(), db0, s.SnapshotLevel()); !errors.Is(err, litestream.ErrDBNotReady) {\n\t\t\tt.Fatalf(\"expected ErrDBNotReady, got: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestStore_Integration(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tconst factor = 1\n\n\tdb := testingutil.NewDB(t, filepath.Join(t.TempDir(), \"db\"))\n\tdb.MonitorInterval = factor * 100 * time.Millisecond\n\tdb.Replica = litestream.NewReplica(db)\n\tdb.Replica.Client = file.NewReplicaClient(t.TempDir())\n\tif err := db.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tsqldb := testingutil.MustOpenSQLDB(t, db.Path())\n\tdefer testingutil.MustCloseSQLDB(t, sqldb)\n\n\tstore := litestream.NewStore([]*litestream.DB{db}, litestream.CompactionLevels{\n\t\t{Level: 0},\n\t\t{Level: 1, Interval: factor * 200 * time.Millisecond},\n\t\t{Level: 2, Interval: factor * 500 * time.Millisecond},\n\t})\n\tstore.SnapshotInterval = factor * 1 * time.Second\n\tif err := store.Open(t.Context()); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer store.Close(t.Context())\n\n\t// Create initial table\n\tif _, err := sqldb.ExecContext(t.Context(), `CREATE TABLE t (id INTEGER PRIMARY KEY, val TEXT);`); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Run test for a fixed duration.\n\tdone := make(chan struct{})\n\ttime.AfterFunc(10*time.Second, func() { close(done) })\n\n\t// Channel for insert errors\n\tinsertErr := make(chan error, 1)\n\n\t// WaitGroup to ensure insert goroutine completes before cleanup\n\tvar wg sync.WaitGroup\n\n\t// Wait for insert goroutine to finish before cleanup & surface any errors.\n\tdefer func() {\n\t\twg.Wait()\n\n\t\tselect {\n\t\tcase err := <-insertErr:\n\t\t\tt.Fatalf(\"insert error during test: %v\", err)\n\t\tdefault:\n\t\t\t// No insert errors\n\t\t}\n\t}()\n\n\t// Start goroutine to continuously insert records\n\twg.Add(1)\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tticker := time.NewTicker(factor * 10 * time.Millisecond)\n\t\tdefer ticker.Stop()\n\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-t.Context().Done():\n\t\t\t\treturn\n\t\t\tcase <-done:\n\t\t\t\treturn\n\t\t\tcase <-ticker.C:\n\t\t\t\tif _, err := sqldb.ExecContext(t.Context(), `INSERT INTO t (val) VALUES (?);`, time.Now().String()); err != nil {\n\t\t\t\t\t// Check if we're shutting down\n\t\t\t\t\tselect {\n\t\t\t\t\tcase <-done:\n\t\t\t\t\t\t// Expected during shutdown, just exit\n\t\t\t\t\t\treturn\n\t\t\t\t\tdefault:\n\t\t\t\t\t\t// Real error, send it\n\t\t\t\t\t\tselect {\n\t\t\t\t\t\tcase insertErr <- err:\n\t\t\t\t\t\tdefault:\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\n\t// Periodically snapshot, restore and validate\n\tticker := time.NewTicker(factor * 500 * time.Millisecond)\n\tdefer ticker.Stop()\n\n\tfor i := 0; ; i++ {\n\t\tselect {\n\t\tcase <-t.Context().Done():\n\t\t\treturn\n\t\tcase <-done:\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t\t// Restore database to a temporary location.\n\t\t\toutputPath := filepath.Join(t.TempDir(), fmt.Sprintf(\"restore-%d.db\", i))\n\t\t\tif err := db.Replica.Restore(t.Context(), litestream.RestoreOptions{\n\t\t\t\tOutputPath: outputPath,\n\t\t\t}); err != nil {\n\t\t\t\tt.Fatal(err)\n\t\t\t}\n\n\t\t\tfunc() {\n\t\t\t\trestoreDB := testingutil.MustOpenSQLDB(t, outputPath)\n\t\t\t\tdefer testingutil.MustCloseSQLDB(t, restoreDB)\n\n\t\t\t\tvar result string\n\t\t\t\tif err := restoreDB.QueryRowContext(t.Context(), `PRAGMA integrity_check;`).Scan(&result); err != nil {\n\t\t\t\t\tt.Fatal(err)\n\t\t\t\t} else if result != \"ok\" {\n\t\t\t\t\tt.Fatalf(\"integrity check failed: %s\", result)\n\t\t\t\t}\n\n\t\t\t\tvar count int\n\t\t\t\tif err := restoreDB.QueryRowContext(t.Context(), `SELECT COUNT(*) FROM t`).Scan(&count); err != nil {\n\t\t\t\t\tt.Fatal(err)\n\t\t\t\t} else if count == 0 {\n\t\t\t\t\tt.Fatal(\"no records found in restored database\")\n\t\t\t\t}\n\t\t\t\tt.Logf(\"restored database: %d records\", count)\n\t\t\t}()\n\t\t}\n\t}\n}\n\n// TestStore_SnapshotInterval_Default ensures that the default snapshot interval\n// is preserved when not explicitly set (regression test for issue #689).\nfunc TestStore_SnapshotInterval_Default(t *testing.T) {\n\t// Create a store with no databases and no levels\n\tstore := litestream.NewStore(nil, nil)\n\n\t// Verify default snapshot interval is set\n\tif store.SnapshotInterval != litestream.DefaultSnapshotInterval {\n\t\tt.Errorf(\"expected default snapshot interval of %v, got %v\",\n\t\t\tlitestream.DefaultSnapshotInterval, store.SnapshotInterval)\n\t}\n\n\t// Verify default is 24 hours\n\tif store.SnapshotInterval != 24*time.Hour {\n\t\tt.Errorf(\"expected default snapshot interval of 24h, got %v\",\n\t\t\tstore.SnapshotInterval)\n\t}\n}\n\nfunc TestStore_Validate(t *testing.T) {\n\tt.Run(\"AllLevelsValid\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\n\t\tdb := &litestream.DB{}\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\n\t\tlevels := litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t\t{Level: 1},\n\t\t}\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\n\t\t// Create contiguous files at L0\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 2, 2)\n\t\t// Create contiguous files at L1\n\t\tcreateTestLTXFile(t, client, 1, 1, 2)\n\n\t\tresult, err := store.Validate(t.Context())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !result.Valid {\n\t\t\tt.Errorf(\"expected valid result, got errors: %v\", result.Errors)\n\t\t}\n\t})\n\n\tt.Run(\"ErrorAtMultipleLevels\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\n\t\tdb := &litestream.DB{}\n\t\tdb.Replica = litestream.NewReplicaWithClient(db, client)\n\n\t\tlevels := litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t\t{Level: 1},\n\t\t}\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\n\t\t// Create files with gap at L0\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 5, 5) // gap at 2-4\n\n\t\t// Create files with overlap at L1\n\t\tcreateTestLTXFile(t, client, 1, 1, 5)\n\t\tcreateTestLTXFile(t, client, 1, 3, 7) // overlap\n\n\t\tresult, err := store.Validate(t.Context())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif result.Valid {\n\t\t\tt.Error(\"expected invalid result\")\n\t\t}\n\t\tif len(result.Errors) != 2 {\n\t\t\tt.Errorf(\"expected 2 errors, got %d\", len(result.Errors))\n\t\t}\n\t})\n\n\tt.Run(\"NilReplica\", func(t *testing.T) {\n\t\t// DB with nil replica should be skipped\n\t\tdb := &litestream.DB{}\n\t\t// db.Replica is nil\n\n\t\tlevels := litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t}\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\n\t\tresult, err := store.Validate(t.Context())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !result.Valid {\n\t\t\tt.Errorf(\"expected valid result for nil replica, got errors: %v\", result.Errors)\n\t\t}\n\t})\n\n\tt.Run(\"MultipleDBs\", func(t *testing.T) {\n\t\tclient1 := file.NewReplicaClient(t.TempDir())\n\t\tclient2 := file.NewReplicaClient(t.TempDir())\n\n\t\tdb1 := &litestream.DB{}\n\t\tdb1.Replica = litestream.NewReplicaWithClient(db1, client1)\n\n\t\tdb2 := &litestream.DB{}\n\t\tdb2.Replica = litestream.NewReplicaWithClient(db2, client2)\n\n\t\tlevels := litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t}\n\t\tstore := litestream.NewStore([]*litestream.DB{db1, db2}, levels)\n\n\t\t// db1: valid\n\t\tcreateTestLTXFile(t, client1, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client1, 0, 2, 2)\n\n\t\t// db2: gap error\n\t\tcreateTestLTXFile(t, client2, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client2, 0, 5, 5)\n\n\t\tresult, err := store.Validate(t.Context())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif result.Valid {\n\t\t\tt.Error(\"expected invalid result\")\n\t\t}\n\t\tif len(result.Errors) != 1 {\n\t\t\tt.Errorf(\"expected 1 error from db2, got %d\", len(result.Errors))\n\t\t}\n\t})\n}\n\nfunc TestStore_ValidationMonitor(t *testing.T) {\n\tt.Run(\"RunsPeriodically\", func(t *testing.T) {\n\t\tdb, sqldb := testingutil.MustOpenDBs(t)\n\t\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t\tlevels := litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t\t{Level: 1, Interval: time.Hour},\n\t\t}\n\t\tstore := litestream.NewStore([]*litestream.DB{db}, levels)\n\t\tstore.CompactionMonitorEnabled = false\n\t\tstore.ValidationInterval = 50 * time.Millisecond\n\n\t\tif err := store.Open(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Wait for at least one validation cycle\n\t\ttime.Sleep(100 * time.Millisecond)\n\n\t\tif err := store.Close(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n\n\tt.Run(\"DisabledByDefault\", func(t *testing.T) {\n\t\tlevels := litestream.CompactionLevels{\n\t\t\t{Level: 0},\n\t\t}\n\t\tstore := litestream.NewStore(nil, levels)\n\t\tstore.CompactionMonitorEnabled = false\n\n\t\t// ValidationInterval should be zero by default\n\t\tif store.ValidationInterval != 0 {\n\t\t\tt.Errorf(\"expected ValidationInterval=0, got %v\", store.ValidationInterval)\n\t\t}\n\n\t\t// Open should succeed without starting validation monitor\n\t\tif err := store.Open(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := store.Close(t.Context()); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t})\n}\n\nfunc TestStore_SetRetentionEnabled(t *testing.T) {\n\tdb0, sqldb0 := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db0, sqldb0)\n\n\tdb1, sqldb1 := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db1, sqldb1)\n\n\tlevels := litestream.CompactionLevels{\n\t\t{Level: 0},\n\t\t{Level: 1, Interval: time.Hour},\n\t}\n\tstore := litestream.NewStore([]*litestream.DB{db0, db1}, levels)\n\tstore.CompactionMonitorEnabled = false\n\n\t// Initially should be true (retention enabled by default).\n\tif !store.RetentionEnabled {\n\t\tt.Fatal(\"expected RetentionEnabled=true initially\")\n\t}\n\n\t// Set to false and verify propagation to all DBs.\n\tstore.SetRetentionEnabled(false)\n\n\tif store.RetentionEnabled {\n\t\tt.Fatal(\"expected store.RetentionEnabled=false\")\n\t}\n\tfor _, db := range store.DBs() {\n\t\tif db.RetentionEnabled {\n\t\t\tt.Fatalf(\"expected db.RetentionEnabled=false for %s\", db.Path())\n\t\t}\n\t}\n\n\t// Set back to true.\n\tstore.SetRetentionEnabled(true)\n\n\tif !store.RetentionEnabled {\n\t\tt.Fatal(\"expected store.RetentionEnabled=true after reset\")\n\t}\n\tfor _, db := range store.DBs() {\n\t\tif !db.RetentionEnabled {\n\t\t\tt.Fatalf(\"expected db.RetentionEnabled=true for %s after reset\", db.Path())\n\t\t}\n\t}\n}\n" }, { "path": "tests/cpu-usage/README.md", "content": "# CPU Usage Testing\n\nThis directory contains test scripts and configurations for measuring Litestream's idle CPU usage, particularly for validating the fixes in issue #992.\n\n## Files\n\n- `test-cpu-usage.sh` - Automated CPU monitoring script\n- `litestream-test-polling.yml` - Config for testing with S3 replication\n\n## Prerequisites\n\n1. Build Litestream binary:\n ```bash\n cd ../..\n go build -o bin/litestream ./cmd/litestream\n ```\n\n2. Set up AWS credentials in `.envrc` at repo root:\n ```bash\n export AWS_ACCESS_KEY_ID=\"your-key-id\"\n export AWS_SECRET_ACCESS_KEY=\"your-secret-key\"\n export AWS_REGION=\"us-east-2\"\n export S3_BUCKET=\"your-test-bucket\"\n ```\n\n3. Have `sqlite3` CLI installed\n\n## Usage\n\nFrom this directory, run:\n\n```bash\n# Test for 60 seconds\n./test-cpu-usage.sh 60\n\n# Longer test (5 minutes)\n./test-cpu-usage.sh 300\n```\n\n## What It Tests\n\nThe script:\n1. Creates a test SQLite database at `/tmp/test.db`\n2. Starts Litestream with S3 replication\n3. Monitors CPU usage every second using `ps`\n4. Calculates average CPU usage\n5. Verifies S3 replication is working\n6. Outputs results and detailed CSV log\n\n## Expected Results\n\nBased on testing for PR #993:\n\n- **With S3 transport fix:** ~0.0067% CPU (99% improvement)\n- **Original (v0.5.6):** ~0.7% CPU\n\nThe S3 transport fix achieves near-zero idle CPU usage, validating the fix.\n\n## Output\n\nResults are printed to stdout and detailed logs are saved to:\n- `/tmp/litestream-cpu-log.csv` - Per-second CPU measurements\n\n## Notes\n\n- Tests require real S3 credentials and will upload data to your bucket\n- Test database is created at `/tmp/test.db` and cleaned up on each run\n- CPU measurements are instantaneous snapshots, not averages over intervals\n- Longer test durations (5-10 minutes) provide more stable averages\n" }, { "path": "tests/cpu-usage/litestream-test-polling.yml", "content": "# Litestream test configuration - POLLING MODE\n# Tests idle CPU usage with default polling (1s interval)\n\ndbs:\n - path: /tmp/test.db\n replicas:\n - type: s3\n bucket: sprite-litestream-debugging\n region: us-east-2\n path: test-db-polling\n # Default: monitor-interval: 1s\n # Default: monitor-mode: poll\n" }, { "path": "tests/cpu-usage/test-cpu-usage.sh", "content": "#!/bin/bash\nset -e\n\n# Test script for measuring Litestream idle CPU usage with S3 replication\n\nDURATION=${1:-300} # Default 5 minutes\nCONFIG_FILE=\"litestream-test-polling.yml\"\nMODE_DESC=\"Polling mode (1s interval)\"\n\necho \"=========================================\"\necho \"Litestream CPU Usage Test\"\necho \"=========================================\"\necho \"Mode: $MODE_DESC\"\necho \"Config: $CONFIG_FILE\"\necho \"Duration: ${DURATION}s\"\necho \"=========================================\"\n\n# Create test database\necho \"Creating test database...\"\nrm -f /tmp/test.db /tmp/test.db-wal /tmp/test.db-shm\nsqlite3 /tmp/test.db \"CREATE TABLE test (id INTEGER PRIMARY KEY, data TEXT);\"\nsqlite3 /tmp/test.db \"INSERT INTO test (data) VALUES ('test');\"\n\n# Start Litestream in background\necho \"Starting Litestream...\"\n# Get script directory and repo root\nSCRIPT_DIR=\"$(cd \"$(dirname \"${BASH_SOURCE[0]}\")\" && pwd)\"\nREPO_ROOT=\"$(cd \"$SCRIPT_DIR/../..\" && pwd)\"\n\nsource \"$REPO_ROOT/.envrc\"\n\"$REPO_ROOT/bin/litestream\" replicate -config \"$SCRIPT_DIR/$CONFIG_FILE\" &\nLITESTREAM_PID=$!\n\necho \"Litestream PID: $LITESTREAM_PID\"\necho \"\"\necho \"Monitoring CPU usage for ${DURATION}s...\"\necho \"Press Ctrl+C to stop early\"\necho \"\"\n\n# Monitor CPU usage\necho \"Time,CPU%,VSZ,RSS\" > /tmp/litestream-cpu-log.csv\nfor i in $(seq 1 $DURATION); do\n if ! kill -0 $LITESTREAM_PID 2>/dev/null; then\n echo \"ERROR: Litestream process died!\"\n exit 1\n fi\n\n # Get CPU and memory stats\n CPU=$(ps -p $LITESTREAM_PID -o %cpu= | xargs)\n VSZ=$(ps -p $LITESTREAM_PID -o vsz= | xargs)\n RSS=$(ps -p $LITESTREAM_PID -o rss= | xargs)\n\n echo \"$i,$CPU,$VSZ,$RSS\" >> /tmp/litestream-cpu-log.csv\n\n # Display every 10 seconds\n if [ $((i % 10)) -eq 0 ]; then\n echo \"[$i/${DURATION}s] CPU: ${CPU}% VSZ: ${VSZ}KB RSS: ${RSS}KB\"\n fi\n\n sleep 1\ndone\n\n# Stop Litestream\necho \"\"\necho \"Stopping Litestream...\"\nkill $LITESTREAM_PID\nwait $LITESTREAM_PID 2>/dev/null || true\n\n# Calculate average CPU\necho \"\"\necho \"=========================================\"\necho \"Results\"\necho \"=========================================\"\nAVG_CPU=$(awk -F',' 'NR>1 {sum+=$2; count++} END {if(count>0) print sum/count; else print 0}' /tmp/litestream-cpu-log.csv)\necho \"Average CPU: ${AVG_CPU}%\"\necho \"Detailed log: /tmp/litestream-cpu-log.csv\"\necho \"\"\n\n# Show sample of S3 uploads\necho \"S3 Bucket Contents:\"\naws s3 ls s3://sprite-litestream-debugging/test-db-${CONFIG_MODE}/ --recursive | head -10\n" }, { "path": "tests/integration/README.md", "content": "# Integration Tests\n\nGo-based integration tests for Litestream. These tests replace the previous bash-based test scripts with proper Go testing infrastructure.\n\n## Overview\n\nThis package contains comprehensive integration tests organized by test type:\n\n- **scenario_test.go** - Core functionality scenarios (fresh start, integrity, deletion, failover)\n- **concurrent_test.go** - Concurrency and stress tests (rapid checkpoints, WAL growth, concurrent ops, busy timeout)\n- **quick_test.go** - Quick validation tests (30 minutes configurable)\n- **overnight_test.go** - Long-running stability tests (8+ hours)\n- **boundary_test.go** - Edge cases (1GB boundary, different page sizes)\n- **helpers.go** - Shared test utilities and helpers\n- **fixtures.go** - Test data generators and scenarios\n\n## Prerequisites\n\nBuild the required binaries:\n\n```bash\ngo build -o bin/litestream ./cmd/litestream\ngo build -o bin/litestream-test ./cmd/litestream-test\n```\n\n## Running Tests\n\n### Quick Tests (Default)\n\nRun fast integration tests suitable for CI:\n\n```bash\ngo test -v -tags=integration -timeout=30m ./tests/integration/... \\\n -run=\"TestFreshStart|TestDatabaseIntegrity|TestRapidCheckpoints\"\n```\n\n### All Scenario Tests\n\nRun all scenario tests (excluding long-running):\n\n```bash\ngo test -v -tags=integration -timeout=1h ./tests/integration/...\n```\n\n### Long-Running Tests\n\nRun overnight and boundary tests:\n\n```bash\ngo test -v -tags=\"integration,long\" -timeout=10h ./tests/integration/... \\\n -run=\"TestOvernight|Test1GBBoundary\"\n```\n\n## Soak Tests\n\nLong-running soak tests live alongside the other integration tests and share the same helpers. They are excluded from CI by default and are intended for release validation or targeted debugging.\n\n### Overview\n\n| Test | Tags | Defaults | Purpose | Extra Requirements |\n| --- | --- | --- | --- | --- |\n| `TestComprehensiveSoak` | `integration,soak` | 2h duration, 50 MB DB, 500 writes/s | File-backed end-to-end stress | Litestream binaries in `./bin` |\n| `TestMinIOSoak` | `integration,soak,docker` | 2h duration, 5 MB DB (short=2 m), 100 writes/s | S3-compatible replication via MinIO | Docker daemon, `docker` CLI |\n| `TestOvernightS3Soak` | `integration,soak,aws` | 8h duration, 50 MB DB | Real S3 replication & restore | AWS credentials, `aws` CLI |\n\nAll soak tests support `go test -test.short` to scale the default duration down to roughly two minutes for smoke verification.\n\n### Environment Variables\n\n| Variable | Default | Description |\n| --- | --- | --- |\n| `SOAK_AUTO_PURGE` | `yes` for non-interactive shells; prompts otherwise | Controls whether MinIO buckets are cleared before each run. Set to `no` to retain objects between runs. |\n| `SOAK_KEEP_TEMP` | unset | When set (any value), preserves the temporary directory and artifacts (database, config, logs) instead of removing them after the test completes. |\n| `SOAK_DEBUG` | `0` | Streams command stdout/stderr (database population, load generation, docker helpers) directly to the console. Without this the output is captured and only shown on failure. |\n| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `S3_BUCKET`, `AWS_REGION` | required for `aws` tag | Provide credentials and target bucket for the overnight S3 soak. Region defaults to `us-east-1` if unset. |\n\n### Example Commands\n\nFile-based soak (full length):\n\n```bash\ngo test -v -tags=\"integration,soak\" \\\n -run=TestComprehensiveSoak -timeout=3h ./tests/integration\n```\n\nFile-based soak (short mode with preserved artifacts and debug logging):\n\n```bash\nSOAK_KEEP_TEMP=1 SOAK_DEBUG=1 go test -v -tags=\"integration,soak\" \\\n -run=TestComprehensiveSoak -test.short -timeout=1h ./tests/integration\n```\n\nMinIO soak (short mode, auto-purges bucket, preserves results):\n\n```bash\nSOAK_AUTO_PURGE=yes SOAK_KEEP_TEMP=1 go test -v -tags=\"integration,soak,docker\" \\\n -run=TestMinIOSoak -test.short -timeout=20m ./tests/integration\n```\n\nOvernight S3 soak (full duration):\n\n```bash\nexport AWS_ACCESS_KEY_ID=...\nexport AWS_SECRET_ACCESS_KEY=...\nexport S3_BUCKET=your-bucket\nexport AWS_REGION=us-east-1\n\ngo test -v -tags=\"integration,soak,aws\" \\\n -run=TestOvernightS3Soak -timeout=10h ./tests/integration\n```\n\n### Tips\n\n- Run with `-v` to view the 60-second progress updates and final status summary. Without `-v`, progress output is suppressed by Go’s test runner.\n- When prompted about purging a MinIO bucket, answering “yes” clears the bucket via `minio/mc` before the run; “no” allows you to inspect lingering objects from previous executions.\n- `SOAK_KEEP_TEMP=1` is especially useful when investigating failures—the helper prints the preserved path so you can inspect databases, configs, and logs.\n- The monitoring infrastructure automatically prints additional status blocks when error counts change, making `SOAK_DEBUG=1` optional for most workflows.\n\n### Specific Tests\n\nRun individual test functions:\n\n```bash\n# Fresh start test\ngo test -v -tags=integration ./tests/integration/... -run=TestFreshStart\n\n# Rapid checkpoints test\ngo test -v -tags=integration ./tests/integration/... -run=TestRapidCheckpoints\n\n# 1GB boundary test\ngo test -v -tags=integration ./tests/integration/... -run=Test1GBBoundary\n```\n\n### Short Mode\n\nRun abbreviated versions with `-short`:\n\n```bash\ngo test -v -tags=integration -short ./tests/integration/...\n```\n\nThis reduces test durations by 10x (e.g., 8 hours becomes 48 minutes).\n\n## Test Categories\n\n### Scenario Tests\n\nCore functionality tests that run in seconds to minutes:\n\n- `TestFreshStart` - Starting replication before database exists\n- `TestDatabaseIntegrity` - Complex schema and data integrity\n- `TestDatabaseDeletion` - Source database deletion during replication\n\n### Concurrent Tests\n\nStress and concurrency tests:\n\n- `TestRapidCheckpoints` - Rapid checkpoint operations under load\n- `TestWALGrowth` - Large WAL file handling (100MB+)\n- `TestConcurrentOperations` - Multiple databases replicating simultaneously\n- `TestBusyTimeout` - Database busy timeout and lock handling\n\n### Quick Tests\n\nConfigurable duration validation (default 30 minutes):\n\n- `TestQuickValidation` - Comprehensive validation with wave pattern load\n\n### Overnight Tests\n\nLong-running stability tests (default 8 hours):\n\n- `TestOvernightFile` - 8-hour file-based replication test\n- `TestOvernightComprehensive` - 8-hour comprehensive test with large database\n\n### Boundary Tests\n\nEdge case and boundary condition tests:\n\n- `Test1GBBoundary` - SQLite 1GB lock page boundary (page #262145 with 4KB pages)\n- `TestLockPageWithDifferentPageSizes` - Lock page handling with various page sizes\n\n## CI Integration\n\n### Automatic (Pull Requests)\n\nQuick tests run automatically on PRs modifying Go code:\n\n```yaml\n- Quick integration tests (TestFreshStart, TestDatabaseIntegrity, TestRapidCheckpoints)\n- Timeout: 30 minutes\n```\n\n### Manual Workflows\n\nTrigger via GitHub Actions UI:\n\n**Quick Tests:**\n```\nworkflow_dispatch → test_type: quick\n```\n\n**All Scenario Tests:**\n```\nworkflow_dispatch → test_type: all\n```\n\n**Long-Running Tests:**\n```\nworkflow_dispatch → test_type: long\n```\n\n## Test Infrastructure\n\n### Helpers (helpers.go)\n\n- `SetupTestDB(t, name)` - Create test database instance\n- `TestDB.Create()` - Create database with WAL mode\n- `TestDB.Populate(size)` - Populate to target size\n- `TestDB.StartLitestream()` - Start replication\n- `TestDB.StopLitestream()` - Stop replication\n- `TestDB.Restore(path)` - Restore from replica\n- `TestDB.Validate(path)` - Full validation (integrity, checksum, data)\n- `TestDB.QuickValidate(path)` - Quick validation\n- `TestDB.GenerateLoad(...)` - Generate database load\n- `GetTestDuration(t, default)` - Get configurable test duration\n- `RequireBinaries(t)` - Check for required binaries\n\n### Fixtures (fixtures.go)\n\n- `DefaultLoadConfig()` - Load generation configuration\n- `DefaultPopulateConfig()` - Database population configuration\n- `CreateComplexTestSchema(db)` - Multi-table schema with foreign keys\n- `PopulateComplexTestData(db, ...)` - Populate complex data\n- `LargeWALScenario()` - Large WAL test scenario\n- `RapidCheckpointsScenario()` - Rapid checkpoint scenario\n\n## Test Artifacts\n\nTests create temporary directories via `t.TempDir()`:\n\n```\n/tmp//\n├── .db # Test database\n├── .db-wal # WAL file\n├── .db-shm # Shared memory\n├── replica/ # Replica directory\n│ └── ltx/0/ # LTX files\n├── litestream.log # Litestream output\n└── *-restored.db # Restored databases\n```\n\nArtifacts are automatically cleaned up after tests complete.\n\n## Debugging Tests\n\n### View Litestream Logs\n\n```go\nlog, err := db.GetLitestreamLog()\nfmt.Println(log)\n```\n\n### Check for Errors\n\n```go\nerrors, err := db.CheckForErrors()\nfor _, e := range errors {\n t.Logf(\"Error: %s\", e)\n}\n```\n\n### Inspect Replica\n\n```go\nfileCount, _ := db.GetReplicaFileCount()\nt.Logf(\"LTX files: %d\", fileCount)\n```\n\n### Check Database Size\n\n```go\nsize, _ := db.GetDatabaseSize()\nt.Logf(\"DB size: %.2f MB\", float64(size)/(1024*1024))\n```\n\n## Migration from Bash\n\nThis is part of an ongoing effort to migrate bash test scripts to Go integration tests. This migration improves maintainability, enables CI integration, and provides platform independence.\n\n### Test Directory Organization\n\nThree distinct test locations serve different purposes:\n\n**`tests/integration/` (this directory)** - Go-based integration and soak tests:\n- Quick integration tests: `scenario_test.go`, `concurrent_test.go`, `boundary_test.go`\n- Soak tests (2-8 hours): `comprehensive_soak_test.go`, `minio_soak_test.go`, `overnight_s3_soak_test.go`\n- All tests use proper Go testing infrastructure with build tags\n\n**`scripts/` (top-level)** - Utility scripts only (soak tests migrated to Go):\n- `analyze-test-results.sh` - Post-test analysis utility\n- `setup-homebrew-tap.sh` - Packaging script (not a test)\n\n**`cmd/litestream-test/scripts/`** - Scenario and debugging bash scripts (being phased out):\n- Bug reproduction scripts for specific issues (#752, #754)\n- Format & upgrade tests for version compatibility\n- S3 retention tests with Python mock\n- Quick validation and setup utilities\n\n### Migration Status\n\n**Migrated from `scripts/` (5 scripts):**\n- `test-quick-validation.sh` → `quick_test.go::TestQuickValidation` (CI: ✅)\n- `test-overnight.sh` → `overnight_test.go::TestOvernightFile` (CI: ❌ too long)\n- `test-comprehensive.sh` → `comprehensive_soak_test.go::TestComprehensiveSoak` (CI: ❌ soak test)\n- `test-minio-s3.sh` → `minio_soak_test.go::TestMinIOSoak` (CI: ❌ soak test, requires Docker)\n- `test-overnight-s3.sh` → `overnight_s3_soak_test.go::TestOvernightS3Soak` (CI: ❌ soak test, 8 hours)\n\n**Migrated from `cmd/litestream-test/scripts/` (9 scripts):**\n- `test-fresh-start.sh` → `scenario_test.go::TestFreshStart`\n- `test-database-integrity.sh` → `scenario_test.go::TestDatabaseIntegrity`\n- `test-database-deletion.sh` → `scenario_test.go::TestDatabaseDeletion`\n- `test-replica-failover.sh` → NOT MIGRATED (feature removed from Litestream)\n- `test-rapid-checkpoints.sh` → `concurrent_test.go::TestRapidCheckpoints`\n- `test-wal-growth.sh` → `concurrent_test.go::TestWALGrowth`\n- `test-concurrent-operations.sh` → `concurrent_test.go::TestConcurrentOperations`\n- `test-busy-timeout.sh` → `concurrent_test.go::TestBusyTimeout`\n- `test-1gb-boundary.sh` → `boundary_test.go::Test1GBBoundary`\n\n**Remaining Bash Scripts:**\n\n_scripts/_ (2 scripts remaining):\n- `analyze-test-results.sh` - Post-test analysis utility (may stay as bash)\n- `setup-homebrew-tap.sh` - Packaging script (not a test)\n\n_cmd/litestream-test/scripts/_ (16 scripts remaining):\n- Bug reproduction scripts: `reproduce-critical-bug.sh`, `test-754-*.sh`, `test-v0.5-*.sh`\n- Format & upgrade tests: `test-format-isolation.sh`, `test-upgrade-*.sh`, `test-massive-upgrade.sh`\n- S3 retention tests: `test-s3-retention-*.sh` (4 scripts, use Python S3 mock)\n- Utility: `verify-test-setup.sh`\n\n### Why Some Tests Aren't in CI\n\nPer industry best practices, CI tests should complete in < 1 hour (ideally < 10 minutes):\n- ✅ **Quick tests** (< 5 min) - Run on every PR\n- ❌ **Soak tests** (2-8 hours) - Run locally before releases only\n- ❌ **Long-running tests** (> 30 min) - Too slow for CI feedback loop\n\nSoak tests are migrated to Go for maintainability but run **locally only**. See \"Soak Tests\" section below.\n\n## Soak Tests (Long-Running Stability Tests)\n\nSoak tests run for 2-8 hours to validate long-term stability under sustained load. These tests are **NOT run in CI** per industry best practices (effective CI requires tests to complete in < 1 hour).\n\n### Purpose\n\nSoak tests validate:\n- Long-term replication stability\n- Memory leak detection over time\n- Compaction effectiveness across multiple cycles\n- Checkpoint behavior under sustained load\n- Recovery from transient issues\n- Storage growth patterns\n\n### When to Run Soak Tests\n\n- ✅ Before major releases\n- ✅ After significant replication changes\n- ✅ To reproduce stability issues\n- ✅ For performance benchmarking\n- ❌ NOT on every commit (too slow for CI)\n\n### Running Soak Tests Locally\n\n**File-based comprehensive test (2 hours):**\n```bash\ngo test -v -tags=\"integration,soak\" -timeout=3h -run=TestComprehensiveSoak ./tests/integration/\n```\n\n**MinIO S3 test (2 hours, requires Docker):**\n```bash\n# Ensure Docker is running\ngo test -v -tags=\"integration,soak,docker\" -timeout=3h -run=TestMinIOSoak ./tests/integration/\n```\n\n**Overnight S3 test (8 hours, requires AWS):**\n```bash\nexport AWS_ACCESS_KEY_ID=your_key\nexport AWS_SECRET_ACCESS_KEY=your_secret\nexport S3_BUCKET=your-test-bucket\nexport AWS_REGION=us-east-1\n\ngo test -v -tags=\"integration,soak,aws\" -timeout=10h -run=TestOvernightS3Soak ./tests/integration/\n```\n\n**Run all soak tests:**\n```bash\ngo test -v -tags=\"integration,soak,docker,aws\" -timeout=15h ./tests/integration/\n```\n\n### Adjust Duration for Testing\n\nTests respect the `-test.short` flag to run abbreviated versions:\n\n```bash\n# Run comprehensive test for 30 minutes instead of 2 hours\ngo test -v -tags=\"integration,soak\" -timeout=1h -run=TestComprehensiveSoak ./tests/integration/ -test.short\n```\n\n### Soak Test Build Tags\n\nSoak tests use multiple build tags to control execution:\n\n- `integration` - Required for all integration tests\n- `soak` - Marks long-running stability tests (2-8 hours)\n- `docker` - Requires Docker (MinIO test)\n- `aws` - Requires AWS credentials (S3 tests)\n\n### Monitoring Soak Tests\n\nAll soak tests log progress every 60 seconds:\n\n```bash\n# Watch test progress in real-time\ngo test -v -tags=\"integration,soak\" -run=TestComprehensiveSoak ./tests/integration/ 2>&1 | tee soak-test.log\n```\n\nMetrics reported during execution:\n- Database size and WAL size\n- Row count\n- Replica statistics (snapshots, LTX segments)\n- Operation counts (checkpoints, compactions, syncs)\n- Error counts\n- Write rate\n\n### Soak Test Summary\n\n| Test | Duration | Requirements | What It Tests |\n|------|----------|--------------|---------------|\n| TestComprehensiveSoak | 2h | None | File-based replication with aggressive compaction |\n| TestMinIOSoak | 2h | Docker | S3-compatible storage via MinIO container |\n| TestOvernightS3Soak | 8h | AWS credentials | Real S3 replication, overnight stability |\n\n## Benefits Over Bash\n\n1. **Type Safety** - Compile-time error checking\n2. **Better Debugging** - Use standard Go debugging tools\n3. **Code Reuse** - Shared helpers and fixtures\n4. **Parallel Execution** - Tests can run concurrently\n5. **CI Integration** - Run automatically on PRs\n6. **Test Coverage** - Measure code coverage\n7. **Consistent Patterns** - Standard Go testing conventions\n8. **Better Error Messages** - Structured, clear reporting\n9. **Platform Independent** - Works on Linux, macOS, Windows\n10. **IDE Integration** - Full editor support\n\n## Contributing\n\nWhen adding new integration tests:\n\n1. Use appropriate build tags (`//go:build integration` or `//go:build integration && long`)\n2. Call `RequireBinaries(t)` to check prerequisites\n3. Use `SetupTestDB(t, name)` for test setup\n4. Call `defer db.Cleanup()` for automatic cleanup\n5. Log test progress with descriptive messages\n6. Use `GetTestDuration(t, default)` for configurable durations\n7. Add test to CI workflow if appropriate\n8. Update this README with new test documentation\n\n## Related Documentation\n\n- [cmd/litestream-test README](../../cmd/litestream-test/README.md) - Testing harness CLI\n- [scripts/README.md](../../scripts/README.md) - Legacy bash test scripts\n- [GitHub Issue #798](https://github.com/benbjohnson/litestream/issues/798) - Migration tracking\n" }, { "path": "tests/integration/boundary_test.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\nfunc Test1GBBoundary(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tt.Log(\"Testing: SQLite 1GB lock page boundary handling\")\n\tt.Log(\"This tests database growth beyond 1GB with 4KB pages (lock page at #262145)\")\n\n\tdb := SetupTestDB(t, \"1gb-boundary\")\n\tdefer db.Cleanup()\n\n\tt.Log(\"[1] Creating database with 4KB page size...\")\n\tif err := db.CreateWithPageSize(4096); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\tt.Log(\"✓ Database created with 4KB pages\")\n\n\tt.Log(\"[2] Populating to 1.5GB to cross lock page boundary...\")\n\tif err := db.PopulateWithOptions(\"1.5GB\", 4096, 1024); err != nil {\n\t\tt.Fatalf(\"Failed to populate database: %v\", err)\n\t}\n\n\tdbSize, err := db.GetDatabaseSize()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get database size: %v\", err)\n\t}\n\n\tsizeGB := float64(dbSize) / (1024 * 1024 * 1024)\n\tt.Logf(\"✓ Database populated: %.2f GB\", sizeGB)\n\n\tif sizeGB < 1.0 {\n\t\tt.Fatalf(\"Database did not reach 1GB threshold: %.2f GB\", sizeGB)\n\t}\n\n\tt.Log(\"[3] Starting Litestream...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(30 * time.Second)\n\n\tt.Log(\"[4] Checking replication across lock page boundary...\")\n\tfileCount, err := db.GetReplicaFileCount()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check replica: %v\", err)\n\t}\n\n\tif fileCount == 0 {\n\t\tt.Fatal(\"No LTX files created!\")\n\t}\n\n\tt.Logf(\"✓ Replication started: %d LTX files\", fileCount)\n\n\tt.Log(\"[5] Checking for lock page errors...\")\n\terrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check errors: %v\", err)\n\t}\n\n\tlockPageErrors := 0\n\tfor _, errMsg := range errors {\n\t\tif containsAny(errMsg, []string{\"lock page\", \"page 262145\", \"locking page\"}) {\n\t\t\tlockPageErrors++\n\t\t\tt.Logf(\"Lock page error: %s\", errMsg)\n\t\t}\n\t}\n\n\tif lockPageErrors > 0 {\n\t\tt.Fatalf(\"Found %d lock page errors!\", lockPageErrors)\n\t}\n\n\tt.Log(\"✓ No lock page errors detected\")\n\n\tdb.StopLitestream()\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[6] Testing restore of large database...\")\n\trestoredPath := filepath.Join(db.TempDir, \"1gb-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Restore successful\")\n\n\tt.Log(\"[7] Validating restored database integrity...\")\n\tif err := db.QuickValidate(restoredPath); err != nil {\n\t\tt.Fatalf(\"Validation failed: %v\", err)\n\t}\n\n\trestoredDB := &TestDB{Path: restoredPath, t: t}\n\trestoredSize, _ := restoredDB.GetDatabaseSize()\n\trestoredSizeGB := float64(restoredSize) / (1024 * 1024 * 1024)\n\n\tt.Logf(\"✓ Restored database size: %.2f GB\", restoredSizeGB)\n\n\tif restoredSizeGB < 0.9 {\n\t\tt.Fatalf(\"Restored database too small: %.2f GB (expected ~%.2f GB)\", restoredSizeGB, sizeGB)\n\t}\n\n\tt.Log(\"TEST PASSED: 1GB lock page boundary handled correctly\")\n}\n\nfunc TestLockPageWithDifferentPageSizes(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tt.Log(\"Testing: Lock page handling with different SQLite page sizes\")\n\n\tpageSizes := []struct {\n\t\tsize int\n\t\tlockPageNum int\n\t\ttargetSizeMB int\n\t}{\n\t\t{4096, 262145, 1200},\n\t\t{8192, 131073, 1200},\n\t}\n\n\tfor _, ps := range pageSizes {\n\t\tt.Run(fmt.Sprintf(\"PageSize%d\", ps.size), func(t *testing.T) {\n\t\t\tdb := SetupTestDB(t, fmt.Sprintf(\"lockpage-%d\", ps.size))\n\t\t\tdefer db.Cleanup()\n\n\t\t\tt.Logf(\"[1] Creating database with %d byte page size (lock page at #%d)...\", ps.size, ps.lockPageNum)\n\t\t\tif err := db.CreateWithPageSize(ps.size); err != nil {\n\t\t\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t\t\t}\n\n\t\t\tt.Logf(\"[2] Populating to %dMB...\", ps.targetSizeMB)\n\t\t\tif err := db.PopulateWithOptions(fmt.Sprintf(\"%dMB\", ps.targetSizeMB), ps.size, 1024); err != nil {\n\t\t\t\tt.Fatalf(\"Failed to populate database: %v\", err)\n\t\t\t}\n\n\t\t\tdbSize, _ := db.GetDatabaseSize()\n\t\t\tt.Logf(\"✓ Database: %.2f MB\", float64(dbSize)/(1024*1024))\n\n\t\t\tt.Log(\"[3] Starting replication...\")\n\t\t\tif err := db.StartLitestream(); err != nil {\n\t\t\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t\t\t}\n\n\t\t\tt.Log(\"[3a] Waiting for replication to produce files...\")\n\t\t\tfileCount, err := db.WaitForReplicaFiles(1, 2*time.Minute)\n\t\t\tif err != nil {\n\t\t\t\tt.Logf(\"Warning: %v\", err)\n\t\t\t}\n\t\t\tt.Logf(\"✓ LTX files: %d\", fileCount)\n\n\t\t\tdb.StopLitestream()\n\n\t\t\tt.Log(\"[4] Testing restore...\")\n\t\t\trestoredPath := filepath.Join(db.TempDir, fmt.Sprintf(\"lockpage-%d-restored.db\", ps.size))\n\t\t\tif err := db.Restore(restoredPath); err != nil {\n\t\t\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t\t\t}\n\n\t\t\tt.Log(\"✓ Test passed for page size\", ps.size)\n\t\t})\n\t}\n\n\tt.Log(\"TEST PASSED: All page sizes handled correctly\")\n}\n\nfunc containsAny(s string, substrs []string) bool {\n\tfor _, substr := range substrs {\n\t\tif contains(s, substr) {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc contains(s, substr string) bool {\n\treturn len(s) >= len(substr) && (s == substr || len(s) > len(substr) && (s[:len(substr)] == substr || s[len(s)-len(substr):] == substr || anySubstring(s, substr)))\n}\n\nfunc anySubstring(s, substr string) bool {\n\tfor i := 0; i <= len(s)-len(substr); i++ {\n\t\tif s[i:i+len(substr)] == substr {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n" }, { "path": "tests/integration/compatibility_test.go", "content": "package integration_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"strconv\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\t\"github.com/benbjohnson/litestream/internal/testingutil\"\n)\n\n// TestRestore_FormatConsistency tests that backups created by the current version\n// can be restored by the same version. This is a basic sanity check that should\n// always pass.\nfunc TestRestore_FormatConsistency(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"Skipping in short mode\")\n\t}\n\n\tctx := context.Background()\n\n\t// Create a database with test data\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Insert initial data\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE compat_test(id INTEGER PRIMARY KEY, data TEXT);`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tfor i := 0; i < 100; i++ {\n\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO compat_test(data) VALUES(?);`, fmt.Sprintf(\"data-%d\", i)); err != nil {\n\t\t\tt.Fatalf(\"insert: %v\", err)\n\t\t}\n\t}\n\n\t// Sync to replica\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"sync: %v\", err)\n\t}\n\tif err := db.Replica.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"replica sync: %v\", err)\n\t}\n\n\t// Checkpoint to ensure data is persisted\n\tif err := db.Checkpoint(ctx, litestream.CheckpointModeTruncate); err != nil {\n\t\tt.Fatalf(\"checkpoint: %v\", err)\n\t}\n\n\t// Add more data after checkpoint\n\tfor i := 100; i < 150; i++ {\n\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO compat_test(data) VALUES(?);`, fmt.Sprintf(\"data-%d\", i)); err != nil {\n\t\t\tt.Fatalf(\"insert: %v\", err)\n\t\t}\n\t}\n\n\t// Sync again\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"sync: %v\", err)\n\t}\n\tif err := db.Replica.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"replica sync: %v\", err)\n\t}\n\n\t// Verify LTX files exist\n\titr, err := db.Replica.Client.LTXFiles(ctx, 0, 0, false)\n\tif err != nil {\n\t\tt.Fatalf(\"list LTX files: %v\", err)\n\t}\n\tvar fileCount int\n\tfor itr.Next() {\n\t\tfileCount++\n\t}\n\tif err := itr.Close(); err != nil {\n\t\tt.Fatalf(\"close iterator: %v\", err)\n\t}\n\tt.Logf(\"Created %d L0 files\", fileCount)\n\n\t// Restore to a new location\n\trestorePath := filepath.Join(t.TempDir(), \"restored.db\")\n\tif err := db.Replica.Restore(ctx, litestream.RestoreOptions{\n\t\tOutputPath: restorePath,\n\t}); err != nil {\n\t\tt.Fatalf(\"restore: %v\", err)\n\t}\n\n\t// Verify restored data\n\trestoredDB := testingutil.MustOpenSQLDB(t, restorePath)\n\tdefer restoredDB.Close()\n\n\tvar count int\n\tif err := restoredDB.QueryRowContext(ctx, `SELECT COUNT(*) FROM compat_test;`).Scan(&count); err != nil {\n\t\tt.Fatalf(\"count: %v\", err)\n\t}\n\n\tif count != 150 {\n\t\tt.Errorf(\"restored row count: got %d, want 150\", count)\n\t}\n\n\t// Verify integrity\n\tvar integrity string\n\tif err := restoredDB.QueryRowContext(ctx, `PRAGMA integrity_check;`).Scan(&integrity); err != nil {\n\t\tt.Fatalf(\"integrity check: %v\", err)\n\t}\n\tif integrity != \"ok\" {\n\t\tt.Errorf(\"integrity check: %s\", integrity)\n\t}\n}\n\n// TestRestore_MultipleSyncs tests restore after many sync cycles to ensure\n// LTX file accumulation doesn't cause issues.\nfunc TestRestore_MultipleSyncs(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"Skipping in short mode\")\n\t}\n\n\tctx := context.Background()\n\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE sync_test(id INTEGER PRIMARY KEY, batch INTEGER, data BLOB);`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\t// Perform multiple sync cycles\n\tconst syncCycles = 50\n\tfor batch := 0; batch < syncCycles; batch++ {\n\t\tfor i := 0; i < 10; i++ {\n\t\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO sync_test(batch, data) VALUES(?, randomblob(500));`, batch); err != nil {\n\t\t\t\tt.Fatalf(\"insert: %v\", err)\n\t\t\t}\n\t\t}\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync %d: %v\", batch, err)\n\t\t}\n\t\tif err := db.Replica.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"replica sync %d: %v\", batch, err)\n\t\t}\n\t}\n\n\t// Verify LTX files\n\titr, err := db.Replica.Client.LTXFiles(ctx, 0, 0, false)\n\tif err != nil {\n\t\tt.Fatalf(\"list LTX files: %v\", err)\n\t}\n\tvar fileCount int\n\tfor itr.Next() {\n\t\tfileCount++\n\t}\n\tif err := itr.Close(); err != nil {\n\t\tt.Fatalf(\"close iterator: %v\", err)\n\t}\n\tt.Logf(\"Created %d L0 files over %d sync cycles\", fileCount, syncCycles)\n\n\t// Restore\n\trestorePath := filepath.Join(t.TempDir(), \"restored.db\")\n\tif err := db.Replica.Restore(ctx, litestream.RestoreOptions{\n\t\tOutputPath: restorePath,\n\t}); err != nil {\n\t\tt.Fatalf(\"restore: %v\", err)\n\t}\n\n\t// Verify\n\trestoredDB := testingutil.MustOpenSQLDB(t, restorePath)\n\tdefer restoredDB.Close()\n\n\tvar count int\n\tif err := restoredDB.QueryRowContext(ctx, `SELECT COUNT(*) FROM sync_test;`).Scan(&count); err != nil {\n\t\tt.Fatalf(\"count: %v\", err)\n\t}\n\n\texpected := syncCycles * 10\n\tif count != expected {\n\t\tt.Errorf(\"restored row count: got %d, want %d\", count, expected)\n\t}\n}\n\n// TestRestore_LTXFileValidation tests that invalid LTX files are properly\n// detected and rejected during restore.\nfunc TestRestore_LTXFileValidation(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"Skipping in short mode\")\n\t}\n\n\tctx := context.Background()\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\t// Create a valid snapshot first\n\tvalidSnapshot := createValidLTXData(t, 1, 1, time.Now())\n\tif _, err := client.WriteLTXFile(ctx, litestream.SnapshotLevel, 1, 1, bytes.NewReader(validSnapshot)); err != nil {\n\t\tt.Fatalf(\"write snapshot: %v\", err)\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tdata []byte\n\t\tminTXID ltx.TXID\n\t\tmaxTXID ltx.TXID\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"ValidL0File\",\n\t\t\tdata: createValidLTXData(t, 2, 2, time.Now()),\n\t\t\tminTXID: 2,\n\t\t\tmaxTXID: 2,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"EmptyFile\",\n\t\t\tdata: []byte{},\n\t\t\tminTXID: 3,\n\t\t\tmaxTXID: 3,\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"TruncatedHeader\",\n\t\t\tdata: []byte(\"truncated\"),\n\t\t\tminTXID: 4,\n\t\t\tmaxTXID: 4,\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tif _, err := client.WriteLTXFile(ctx, 0, tt.minTXID, tt.maxTXID, bytes.NewReader(tt.data)); err != nil {\n\t\t\t\tt.Logf(\"write failed (may be expected): %v\", err)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestRestore_CrossPlatformPaths tests that backups work with different path styles.\nfunc TestRestore_CrossPlatformPaths(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"Skipping in short mode\")\n\t}\n\n\tctx := context.Background()\n\n\tpathTests := []string{\n\t\t\"simple\",\n\t\t\"path/with/slashes\",\n\t\t\"path-with-dashes\",\n\t\t\"path_with_underscores\",\n\t}\n\n\tfor _, subpath := range pathTests {\n\t\tt.Run(subpath, func(t *testing.T) {\n\t\t\treplicaDir := t.TempDir()\n\t\t\tfullPath := filepath.Join(replicaDir, subpath)\n\n\t\t\tclient := file.NewReplicaClient(fullPath)\n\n\t\t\t// Create snapshot\n\t\t\tsnapshot := createValidLTXData(t, 1, 1, time.Now())\n\t\t\tif _, err := client.WriteLTXFile(ctx, litestream.SnapshotLevel, 1, 1, bytes.NewReader(snapshot)); err != nil {\n\t\t\t\tt.Fatalf(\"write snapshot: %v\", err)\n\t\t\t}\n\n\t\t\t// Create L0 files\n\t\t\tfor i := 2; i <= 5; i++ {\n\t\t\t\tdata := createValidLTXData(t, ltx.TXID(i), ltx.TXID(i), time.Now())\n\t\t\t\tif _, err := client.WriteLTXFile(ctx, 0, ltx.TXID(i), ltx.TXID(i), bytes.NewReader(data)); err != nil {\n\t\t\t\t\tt.Fatalf(\"write L0 %d: %v\", i, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Verify files exist\n\t\t\titr, err := client.LTXFiles(ctx, 0, 0, false)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"list files: %v\", err)\n\t\t\t}\n\t\t\tvar count int\n\t\t\tfor itr.Next() {\n\t\t\t\tcount++\n\t\t\t}\n\t\t\titr.Close()\n\n\t\t\tif count != 4 {\n\t\t\t\tt.Errorf(\"file count: got %d, want 4\", count)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestRestore_PointInTimeAccuracy tests that point-in-time restore respects\n// timestamps correctly.\nfunc TestRestore_PointInTimeAccuracy(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"Skipping in short mode\")\n\t}\n\n\tctx := context.Background()\n\treplicaDir := t.TempDir()\n\tclient := file.NewReplicaClient(replicaDir)\n\n\tbaseTime := time.Now().Add(-10 * time.Minute)\n\n\t// Create snapshot at baseTime\n\tsnapshot := createValidLTXData(t, 1, 1, baseTime)\n\tif _, err := client.WriteLTXFile(ctx, litestream.SnapshotLevel, 1, 1, bytes.NewReader(snapshot)); err != nil {\n\t\tt.Fatalf(\"write snapshot: %v\", err)\n\t}\n\n\t// Create L0 files at 1-minute intervals\n\tfor i := 2; i <= 10; i++ {\n\t\tts := baseTime.Add(time.Duration(i-1) * time.Minute)\n\t\tdata := createValidLTXData(t, ltx.TXID(i), ltx.TXID(i), ts)\n\t\tif _, err := client.WriteLTXFile(ctx, 0, ltx.TXID(i), ltx.TXID(i), bytes.NewReader(data)); err != nil {\n\t\t\tt.Fatalf(\"write L0 %d: %v\", i, err)\n\t\t}\n\t}\n\n\t// Verify timestamps are preserved when listing with metadata\n\titr, err := client.LTXFiles(ctx, 0, 0, true)\n\tif err != nil {\n\t\tt.Fatalf(\"list files: %v\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar files []*ltx.FileInfo\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tfiles = append(files, <x.FileInfo{\n\t\t\tLevel: info.Level,\n\t\t\tMinTXID: info.MinTXID,\n\t\t\tMaxTXID: info.MaxTXID,\n\t\t\tCreatedAt: info.CreatedAt,\n\t\t})\n\t}\n\n\tif len(files) != 9 {\n\t\tt.Fatalf(\"file count: got %d, want 9\", len(files))\n\t}\n\n\t// Verify timestamps are monotonically increasing\n\tfor i := 1; i < len(files); i++ {\n\t\tif files[i].CreatedAt.Before(files[i-1].CreatedAt) {\n\t\t\tt.Errorf(\"file %d timestamp (%v) is before file %d timestamp (%v)\",\n\t\t\t\ti, files[i].CreatedAt, i-1, files[i-1].CreatedAt)\n\t\t}\n\t}\n}\n\n// createValidLTXData creates a minimal valid LTX file for testing.\nfunc createValidLTXData(t *testing.T, minTXID, maxTXID ltx.TXID, ts time.Time) []byte {\n\tt.Helper()\n\n\thdr := ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tPageSize: 4096,\n\t\tCommit: 1,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tTimestamp: ts.UnixMilli(),\n\t}\n\tif minTXID == 1 {\n\t\thdr.PreApplyChecksum = 0\n\t} else {\n\t\thdr.PreApplyChecksum = ltx.ChecksumFlag\n\t}\n\n\theaderBytes, err := hdr.MarshalBinary()\n\tif err != nil {\n\t\tt.Fatalf(\"marshal header: %v\", err)\n\t}\n\n\treturn headerBytes\n}\n\n// TestBinaryCompatibility_CLIRestore tests that the litestream CLI can restore\n// backups created programmatically. This is a basic end-to-end test.\nfunc TestBinaryCompatibility_CLIRestore(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"Skipping in short mode\")\n\t}\n\n\t// Skip if litestream binary is not available\n\tlitestreamBin := os.Getenv(\"LITESTREAM_BIN\")\n\tif litestreamBin == \"\" {\n\t\tlitestreamBin = \"./bin/litestream\"\n\t}\n\tif _, err := os.Stat(litestreamBin); os.IsNotExist(err) {\n\t\tt.Skip(\"litestream binary not found, skipping CLI test\")\n\t}\n\n\tctx := context.Background()\n\n\t// Create database with programmatic API\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE cli_test(id INTEGER PRIMARY KEY, value TEXT);`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\tfor i := 0; i < 50; i++ {\n\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO cli_test(value) VALUES(?);`, fmt.Sprintf(\"value-%d\", i)); err != nil {\n\t\t\tt.Fatalf(\"insert: %v\", err)\n\t\t}\n\t}\n\n\tif err := db.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"sync: %v\", err)\n\t}\n\tif err := db.Replica.Sync(ctx); err != nil {\n\t\tt.Fatalf(\"replica sync: %v\", err)\n\t}\n\n\t// Get replica path from the file client\n\tfileClient, ok := db.Replica.Client.(*file.ReplicaClient)\n\tif !ok {\n\t\tt.Skip(\"Test requires file replica client\")\n\t}\n\treplicaPath := fileClient.Path()\n\n\t// Close the database\n\ttestingutil.MustCloseDBs(t, db, sqldb)\n\n\t// Restore using CLI\n\trestorePath := filepath.Join(t.TempDir(), \"cli-restored.db\")\n\tcmd := exec.CommandContext(ctx, litestreamBin, \"restore\", \"-o\", restorePath, \"file://\"+replicaPath)\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil {\n\t\tt.Fatalf(\"CLI restore failed: %v\\nOutput: %s\", err, output)\n\t}\n\n\t// Verify restored database\n\trestoredDB := testingutil.MustOpenSQLDB(t, restorePath)\n\tdefer restoredDB.Close()\n\n\tvar count int\n\tif err := restoredDB.QueryRowContext(ctx, `SELECT COUNT(*) FROM cli_test;`).Scan(&count); err != nil {\n\t\tt.Fatalf(\"count: %v\", err)\n\t}\n\n\tif count != 50 {\n\t\tt.Errorf(\"CLI restored row count: got %d, want 50\", count)\n\t}\n}\n\n// TestVersionMigration_DirectoryLayout tests that the current version can\n// detect and handle different backup directory layouts.\nfunc TestVersionMigration_DirectoryLayout(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"Skipping in short mode\")\n\t}\n\n\tctx := context.Background()\n\n\t// Test current v0.5.x layout (ltx/0/, ltx/1/, ..., ltx/9/ for snapshots)\n\tt.Run(\"CurrentLayout\", func(t *testing.T) {\n\t\treplicaDir := t.TempDir()\n\t\tclient := file.NewReplicaClient(replicaDir)\n\n\t\t// Create files in expected layout\n\t\tsnapshot := createValidLTXData(t, 1, 1, time.Now())\n\t\tif _, err := client.WriteLTXFile(ctx, litestream.SnapshotLevel, 1, 1, bytes.NewReader(snapshot)); err != nil {\n\t\t\tt.Fatalf(\"write snapshot: %v\", err)\n\t\t}\n\n\t\tfor i := 2; i <= 5; i++ {\n\t\t\tdata := createValidLTXData(t, ltx.TXID(i), ltx.TXID(i), time.Now())\n\t\t\tif _, err := client.WriteLTXFile(ctx, 0, ltx.TXID(i), ltx.TXID(i), bytes.NewReader(data)); err != nil {\n\t\t\t\tt.Fatalf(\"write L0 %d: %v\", i, err)\n\t\t\t}\n\t\t}\n\n\t\t// Verify structure\n\t\tsnapshotDir := filepath.Join(replicaDir, \"ltx\", strconv.Itoa(litestream.SnapshotLevel))\n\t\tl0Dir := filepath.Join(replicaDir, \"ltx\", \"0\")\n\n\t\tif _, err := os.Stat(snapshotDir); err != nil {\n\t\t\tt.Errorf(\"snapshot directory not found: %v\", err)\n\t\t}\n\t\tif _, err := os.Stat(l0Dir); err != nil {\n\t\t\tt.Errorf(\"L0 directory not found: %v\", err)\n\t\t}\n\n\t\t// Verify files can be listed\n\t\tsnapshotItr, err := client.LTXFiles(ctx, litestream.SnapshotLevel, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"list snapshots: %v\", err)\n\t\t}\n\t\tvar snapshotCount int\n\t\tfor snapshotItr.Next() {\n\t\t\tsnapshotCount++\n\t\t}\n\t\tsnapshotItr.Close()\n\n\t\tl0Itr, err := client.LTXFiles(ctx, 0, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"list L0: %v\", err)\n\t\t}\n\t\tvar l0Count int\n\t\tfor l0Itr.Next() {\n\t\t\tl0Count++\n\t\t}\n\t\tl0Itr.Close()\n\n\t\tif snapshotCount != 1 {\n\t\t\tt.Errorf(\"snapshot count: got %d, want 1\", snapshotCount)\n\t\t}\n\t\tif l0Count != 4 {\n\t\t\tt.Errorf(\"L0 count: got %d, want 4\", l0Count)\n\t\t}\n\t})\n\n\t// Test that old v0.3.x layout (generations/) is not accidentally used\n\tt.Run(\"LegacyLayoutNotUsed\", func(t *testing.T) {\n\t\treplicaDir := t.TempDir()\n\n\t\t// Create a generations/ directory (v0.3.x layout)\n\t\tlegacyDir := filepath.Join(replicaDir, \"generations\")\n\t\tif err := os.MkdirAll(legacyDir, 0755); err != nil {\n\t\t\tt.Fatalf(\"create legacy dir: %v\", err)\n\t\t}\n\n\t\t// Create client and verify it uses new layout\n\t\tclient := file.NewReplicaClient(replicaDir)\n\n\t\tsnapshot := createValidLTXData(t, 1, 1, time.Now())\n\t\tif _, err := client.WriteLTXFile(ctx, litestream.SnapshotLevel, 1, 1, bytes.NewReader(snapshot)); err != nil {\n\t\t\tt.Fatalf(\"write snapshot: %v\", err)\n\t\t}\n\n\t\t// Verify new layout is used\n\t\tnewLayoutDir := filepath.Join(replicaDir, \"ltx\")\n\t\tif _, err := os.Stat(newLayoutDir); err != nil {\n\t\t\tt.Errorf(\"new layout directory not created: %v\", err)\n\t\t}\n\n\t\t// Verify legacy directory is not used for new files\n\t\tentries, _ := os.ReadDir(legacyDir)\n\t\tif len(entries) > 0 {\n\t\t\tt.Errorf(\"legacy directory should remain empty, has %d entries\", len(entries))\n\t\t}\n\t})\n}\n\n// TestCompaction_Compatibility tests that compacted files maintain compatibility\n// with restore operations.\nfunc TestCompaction_Compatibility(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"Skipping in short mode\")\n\t}\n\n\tctx := context.Background()\n\n\tdb, sqldb := testingutil.MustOpenDBs(t)\n\tdefer testingutil.MustCloseDBs(t, db, sqldb)\n\n\tif _, err := sqldb.ExecContext(ctx, `CREATE TABLE compact_test(id INTEGER PRIMARY KEY, data BLOB);`); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\t// Generate many syncs to create L0 files\n\tfor batch := 0; batch < 20; batch++ {\n\t\tfor i := 0; i < 5; i++ {\n\t\t\tif _, err := sqldb.ExecContext(ctx, `INSERT INTO compact_test(data) VALUES(randomblob(1000));`); err != nil {\n\t\t\t\tt.Fatalf(\"insert: %v\", err)\n\t\t\t}\n\t\t}\n\t\tif err := db.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"sync: %v\", err)\n\t\t}\n\t\tif err := db.Replica.Sync(ctx); err != nil {\n\t\t\tt.Fatalf(\"replica sync: %v\", err)\n\t\t}\n\t}\n\n\t// Force compaction to level 1\n\tif _, err := db.Compact(ctx, 1); err != nil {\n\t\tt.Logf(\"compact to L1 (may not have enough files): %v\", err)\n\t}\n\n\t// Count files at different levels\n\tfor level := 0; level <= 2; level++ {\n\t\titr, err := db.Replica.Client.LTXFiles(ctx, level, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"list level %d: %v\", level, err)\n\t\t}\n\t\tvar count int\n\t\tfor itr.Next() {\n\t\t\tcount++\n\t\t}\n\t\titr.Close()\n\t\tt.Logf(\"Level %d: %d files\", level, count)\n\t}\n\n\t// Restore and verify\n\trestorePath := filepath.Join(t.TempDir(), \"compacted-restore.db\")\n\tif err := db.Replica.Restore(ctx, litestream.RestoreOptions{\n\t\tOutputPath: restorePath,\n\t}); err != nil {\n\t\tt.Fatalf(\"restore: %v\", err)\n\t}\n\n\trestoredDB := testingutil.MustOpenSQLDB(t, restorePath)\n\tdefer restoredDB.Close()\n\n\tvar count int\n\tif err := restoredDB.QueryRowContext(ctx, `SELECT COUNT(*) FROM compact_test;`).Scan(&count); err != nil {\n\t\tt.Fatalf(\"count: %v\", err)\n\t}\n\n\texpected := 20 * 5\n\tif count != expected {\n\t\tt.Errorf(\"restored row count: got %d, want %d\", count, expected)\n\t}\n\n\tvar integrity string\n\tif err := restoredDB.QueryRowContext(ctx, `PRAGMA integrity_check;`).Scan(&integrity); err != nil {\n\t\tt.Fatalf(\"integrity check: %v\", err)\n\t}\n\tif !strings.Contains(integrity, \"ok\") {\n\t\tt.Errorf(\"integrity check failed: %s\", integrity)\n\t}\n}\n" }, { "path": "tests/integration/comprehensive_soak_test.go", "content": "//go:build integration && soak\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n)\n\n// TestComprehensiveSoak runs a comprehensive soak test with aggressive settings\n// to validate all Litestream features: replication, snapshots, compaction, checkpoints.\n//\n// Default duration: 2 hours\n// Can be shortened with: go test -test.short (runs for 30 minutes)\n//\n// This test exercises:\n// - Continuous replication\n// - Snapshot generation (every 10m)\n// - Compaction (30s/1m/5m/15m/30m intervals)\n// - Checkpoint operations\n// - Database restoration\nfunc TestComprehensiveSoak(t *testing.T) {\n\tRequireBinaries(t)\n\n\t// Determine test duration\n\tduration := GetTestDuration(t, 2*time.Hour)\n\tshortMode := testing.Short()\n\tif shortMode {\n\t\tduration = 2 * time.Minute\n\t}\n\n\ttargetSize := \"50MB\"\n\twriteRate := 500\n\tif shortMode {\n\t\ttargetSize = \"5MB\"\n\t\twriteRate = 100\n\t}\n\n\tt.Logf(\"================================================\")\n\tt.Logf(\"Litestream Comprehensive Soak Test\")\n\tt.Logf(\"================================================\")\n\tt.Logf(\"Duration: %v\", duration)\n\tt.Logf(\"Start time: %s\", time.Now().Format(time.RFC3339))\n\tt.Log(\"\")\n\tt.Log(\"This test uses aggressive settings to validate:\")\n\tt.Log(\" - Continuous replication\")\n\tt.Log(\" - Snapshot generation (every 10m)\")\n\tt.Log(\" - Compaction (30s/1m/5m intervals)\")\n\tt.Log(\" - Checkpoint operations\")\n\tt.Log(\" - Database restoration\")\n\tt.Log(\"\")\n\n\tstartTime := time.Now()\n\n\t// Setup test database\n\tdb := SetupTestDB(t, \"comprehensive-soak\")\n\tdefer db.Cleanup()\n\n\t// Create database\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\t// Populate database\n\tt.Logf(\"Populating database (%s initial data)...\", targetSize)\n\tif err := db.Populate(targetSize); err != nil {\n\t\tt.Fatalf(\"Failed to populate database: %v\", err)\n\t}\n\tt.Log(\"✓ Database populated\")\n\tt.Log(\"\")\n\n\t// Create aggressive configuration for testing\n\tt.Log(\"Creating aggressive test configuration...\")\n\treplicaURL := fmt.Sprintf(\"file://%s\", filepath.ToSlash(db.ReplicaPath))\n\tconfigPath := CreateSoakConfig(db.Path, replicaURL, nil, shortMode)\n\tdb.ConfigPath = configPath\n\tt.Logf(\"✓ Configuration created: %s\", configPath)\n\tt.Log(\"\")\n\n\t// Start Litestream\n\tt.Log(\"Starting Litestream replication...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\tt.Logf(\"✓ Litestream running (PID: %d)\", db.LitestreamPID)\n\tt.Log(\"\")\n\n\t// Start load generator with heavy sustained load\n\tt.Log(\"Starting load generator (heavy sustained load)...\")\n\tt.Logf(\" Write rate: %d writes/second\", writeRate)\n\tt.Logf(\" Pattern: wave (simulates varying load)\")\n\tt.Logf(\" Payload size: 4KB\")\n\tt.Logf(\" Workers: 8\")\n\tt.Log(\"\")\n\n\tctx, cancel := context.WithTimeout(context.Background(), duration)\n\tdefer cancel()\n\n\t// Setup signal handler for graceful interruption\n\ttestInfo := &TestInfo{\n\t\tStartTime: startTime,\n\t\tDuration: duration,\n\t\tDB: db,\n\t\tcancel: cancel,\n\t}\n\tsetupSignalHandler(t, cancel, testInfo)\n\n\t// Run load generation in background\n\tloadDone := make(chan error, 1)\n\tgo func() {\n\t\tloadDone <- db.GenerateLoad(ctx, writeRate, duration, \"wave\")\n\t}()\n\n\t// Monitor every 60 seconds\n\tt.Log(\"Running comprehensive test...\")\n\tt.Log(\"Monitor will report every 60 seconds\")\n\tt.Log(\"Press Ctrl+C twice within 5 seconds to stop early\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\n\trefreshStats := func() {\n\t\ttestInfo.RowCount, _ = db.GetRowCount(\"load_test\")\n\t\tif testInfo.RowCount == 0 {\n\t\t\ttestInfo.RowCount, _ = db.GetRowCount(\"test_table_0\")\n\t\t}\n\t\tif testInfo.RowCount == 0 {\n\t\t\ttestInfo.RowCount, _ = db.GetRowCount(\"test_data\")\n\t\t}\n\t\ttestInfo.FileCount, _ = db.GetReplicaFileCount()\n\t}\n\n\tlogMetrics := func() {\n\t\tLogSoakMetrics(t, db, \"comprehensive\")\n\t\tif db.LitestreamCmd != nil && db.LitestreamCmd.ProcessState != nil {\n\t\t\tt.Error(\"✗ Litestream stopped unexpectedly!\")\n\t\t\tif testInfo.cancel != nil {\n\t\t\t\ttestInfo.cancel()\n\t\t\t}\n\t\t}\n\t}\n\n\tMonitorSoakTest(t, db, ctx, testInfo, refreshStats, logMetrics)\n\n\t// Wait for load generation to complete\n\tif err := <-loadDone; err != nil {\n\t\tt.Logf(\"Load generation completed: %v\", err)\n\t}\n\n\tif err := db.WaitForSnapshots(30 * time.Second); err != nil {\n\t\tt.Fatalf(\"Failed waiting for snapshot: %v\", err)\n\t}\n\n\tt.Log(\"\")\n\tt.Log(\"================================================\")\n\tt.Log(\"Final Test Results\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\n\t// Stop Litestream\n\tt.Log(\"Stopping Litestream...\")\n\tif err := db.StopLitestream(); err != nil {\n\t\tt.Logf(\"Warning: Failed to stop Litestream cleanly: %v\", err)\n\t}\n\n\t// Final statistics\n\tt.Log(\"Database Statistics:\")\n\tif dbSize, err := db.GetDatabaseSize(); err == nil {\n\t\tt.Logf(\" Final size: %.2f MB\", float64(dbSize)/(1024*1024))\n\t}\n\n\t// Count rows using different table name possibilities\n\tvar rowCount int\n\tvar err error\n\tif rowCount, err = db.GetRowCount(\"load_test\"); err != nil {\n\t\tif rowCount, err = db.GetRowCount(\"test_table_0\"); err != nil {\n\t\t\tif rowCount, err = db.GetRowCount(\"test_data\"); err != nil {\n\t\t\t\tt.Logf(\" Warning: Could not get row count: %v\", err)\n\t\t\t}\n\t\t}\n\t}\n\tif err == nil {\n\t\tt.Logf(\" Total rows: %d\", rowCount)\n\t}\n\tt.Log(\"\")\n\n\t// Replica statistics\n\tt.Log(\"Replication Statistics:\")\n\tif fileCount, err := db.GetReplicaFileCount(); err == nil {\n\t\tt.Logf(\" LTX segments: %d\", fileCount)\n\t}\n\n\t// Check for errors\n\terrors, _ := db.CheckForErrors()\n\tt.Logf(\" Critical errors: %d\", len(errors))\n\tt.Log(\"\")\n\n\t// Test restoration\n\tt.Log(\"Testing restoration...\")\n\trestoredPath := filepath.Join(db.TempDir, \"restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restoration failed: %v\", err)\n\t}\n\tt.Log(\"✓ Restoration successful!\")\n\n\t// Validate\n\tt.Log(\"\")\n\tt.Log(\"Validating restored database integrity...\")\n\trestoredDB := &TestDB{Path: restoredPath, t: t}\n\tif err := restoredDB.IntegrityCheck(); err != nil {\n\t\tt.Fatalf(\"Integrity check failed: %v\", err)\n\t}\n\tt.Log(\"✓ Integrity check passed!\")\n\n\t// Analyze test results\n\tanalysis := AnalyzeSoakTest(t, db, duration)\n\tPrintSoakTestAnalysis(t, analysis)\n\n\t// Test Summary\n\tt.Log(\"================================================\")\n\tt.Log(\"Test Summary\")\n\tt.Log(\"================================================\")\n\n\ttestPassed := true\n\tissues := []string{}\n\n\tif criticalErrors > 0 {\n\t\ttestPassed = false\n\t\tissues = append(issues, fmt.Sprintf(\"Critical errors detected: %d\", criticalErrors))\n\t}\n\n\tif analysis.FinalFileCount == 0 {\n\t\ttestPassed = false\n\t\tissues = append(issues, \"No files created (replication not working)\")\n\t}\n\n\tif testPassed {\n\t\tt.Log(\"✓ TEST PASSED!\")\n\t\tt.Log(\"\")\n\t\tt.Log(\"The configuration is ready for production use.\")\n\t} else {\n\t\tt.Log(\"⚠ TEST COMPLETED WITH ISSUES:\")\n\t\tfor _, issue := range issues {\n\t\t\tt.Logf(\" - %s\", issue)\n\t\t}\n\t\tt.Log(\"\")\n\t\tt.Log(\"Review the logs for details:\")\n\t\tlogPath, _ := db.GetLitestreamLog()\n\t\tt.Logf(\" %s\", logPath)\n\t\tt.Fail()\n\t}\n\n\tt.Log(\"\")\n\tt.Logf(\"Test duration: %v\", time.Since(startTime).Round(time.Second))\n\tt.Logf(\"Results available in: %s\", db.TempDir)\n\tt.Log(\"================================================\")\n}\n" }, { "path": "tests/integration/concurrent_test.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\nfunc TestRapidCheckpoints(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tt.Log(\"Testing: Litestream under rapid checkpoint pressure\")\n\n\tdb := SetupTestDB(t, \"rapid-checkpoints\")\n\tdefer db.Cleanup()\n\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\tt.Log(\"[1] Starting Litestream...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(3 * time.Second)\n\n\tt.Log(\"[2] Generating rapid writes with frequent checkpoints...\")\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to open database: %v\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tif _, err := sqlDB.Exec(`\n\t\tCREATE TABLE checkpoint_test (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tdata BLOB,\n\t\t\ttimestamp INTEGER\n\t\t)\n\t`); err != nil {\n\t\tt.Fatalf(\"Failed to create table: %v\", err)\n\t}\n\n\tdata := make([]byte, 4096)\n\tcheckpointCount := 0\n\n\tfor i := 0; i < 1000; i++ {\n\t\tif _, err := sqlDB.Exec(\n\t\t\t\"INSERT INTO checkpoint_test (data, timestamp) VALUES (?, ?)\",\n\t\t\tdata,\n\t\t\ttime.Now().Unix(),\n\t\t); err != nil {\n\t\t\tt.Fatalf(\"Failed to insert row %d: %v\", i, err)\n\t\t}\n\n\t\tif i%100 == 0 {\n\t\t\tif _, err := sqlDB.Exec(\"PRAGMA wal_checkpoint(TRUNCATE)\"); err != nil {\n\t\t\t\tt.Logf(\"Checkpoint %d failed: %v\", checkpointCount, err)\n\t\t\t} else {\n\t\t\t\tcheckpointCount++\n\t\t\t\tt.Logf(\"Checkpoint %d completed at row %d\", checkpointCount, i)\n\t\t\t}\n\t\t}\n\t}\n\n\tt.Logf(\"✓ Generated 1000 writes with %d checkpoints\", checkpointCount)\n\n\ttime.Sleep(5 * time.Second)\n\n\tdb.StopLitestream()\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[3] Checking for errors...\")\n\terrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check errors: %v\", err)\n\t}\n\n\tif len(errors) > 5 {\n\t\tt.Fatalf(\"Too many errors (%d), showing first 5:\\n%v\", len(errors), errors[:5])\n\t} else if len(errors) > 0 {\n\t\tt.Logf(\"Found %d errors (acceptable for checkpoint stress)\", len(errors))\n\t}\n\n\tt.Log(\"[4] Verifying replica...\")\n\tfileCount, err := db.GetReplicaFileCount()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check replica: %v\", err)\n\t}\n\n\tif fileCount == 0 {\n\t\tt.Fatal(\"No replica files created!\")\n\t}\n\n\tt.Logf(\"✓ Replica created with %d files\", fileCount)\n\n\tt.Log(\"[5] Testing restore...\")\n\trestoredPath := filepath.Join(db.TempDir, \"checkpoint-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Restore successful\")\n\n\torigCount, err := db.GetRowCount(\"checkpoint_test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get original row count: %v\", err)\n\t}\n\n\trestoredDB := &TestDB{Path: restoredPath, t: t}\n\trestCount, err := restoredDB.GetRowCount(\"checkpoint_test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get restored row count: %v\", err)\n\t}\n\n\tif origCount != restCount {\n\t\tt.Fatalf(\"Count mismatch: original=%d, restored=%d\", origCount, restCount)\n\t}\n\n\tt.Logf(\"✓ Data integrity verified: %d rows\", origCount)\n\tt.Log(\"TEST PASSED: Handled rapid checkpoints successfully\")\n}\n\nfunc TestWALGrowth(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tduration := GetTestDuration(t, 2*time.Minute)\n\tt.Logf(\"Testing: Large WAL file handling (duration: %v)\", duration)\n\n\tdb := SetupTestDB(t, \"wal-growth\")\n\tdefer db.Cleanup()\n\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\tt.Log(\"[1] Creating test table...\")\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to open database: %v\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tif _, err := sqlDB.Exec(`\n\t\tCREATE TABLE wal_test (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tdata BLOB\n\t\t)\n\t`); err != nil {\n\t\tt.Fatalf(\"Failed to create table: %v\", err)\n\t}\n\n\tt.Log(\"✓ Table created\")\n\n\tt.Log(\"[2] Starting Litestream...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(3 * time.Second)\n\n\tt.Log(\"[3] Generating sustained write load...\")\n\tctx, cancel := context.WithTimeout(context.Background(), duration)\n\tdefer cancel()\n\n\tconfig := DefaultLoadConfig()\n\tconfig.WriteRate = 400\n\tconfig.Duration = duration\n\tconfig.Pattern = LoadPatternWave\n\tconfig.PayloadSize = 10 * 1024\n\tconfig.Workers = 4\n\n\tif err := db.GenerateLoad(ctx, config.WriteRate, config.Duration, string(config.Pattern)); err != nil && ctx.Err() == nil {\n\t\tt.Fatalf(\"Load generation failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Load generation complete\")\n\n\ttime.Sleep(5 * time.Second)\n\n\tt.Log(\"[4] Checking WAL size...\")\n\twalPath := db.Path + \"-wal\"\n\twalSize, err := getFileSize(walPath)\n\tif err != nil {\n\t\tt.Logf(\"WAL file not found (may have been checkpointed): %v\", err)\n\t} else {\n\t\tt.Logf(\"WAL size: %.2f MB\", float64(walSize)/(1024*1024))\n\t}\n\n\tdbSize, err := db.GetDatabaseSize()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get database size: %v\", err)\n\t}\n\n\tt.Logf(\"Total database size: %.2f MB\", float64(dbSize)/(1024*1024))\n\n\tdb.StopLitestream()\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[5] Checking for errors...\")\n\terrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check errors: %v\", err)\n\t}\n\n\tif len(errors) > 10 {\n\t\tt.Fatalf(\"Too many errors (%d), showing first 5:\\n%v\", len(errors), errors[:5])\n\t}\n\n\tt.Logf(\"✓ Found %d errors (acceptable)\", len(errors))\n\n\tt.Log(\"[6] Testing restore...\")\n\trestoredPath := filepath.Join(db.TempDir, \"wal-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Restore successful\")\n\n\torigCount, err := db.GetRowCount(\"wal_test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get original row count: %v\", err)\n\t}\n\n\trestoredDB := &TestDB{Path: restoredPath, t: t}\n\trestCount, err := restoredDB.GetRowCount(\"wal_test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get restored row count: %v\", err)\n\t}\n\n\tif origCount != restCount {\n\t\tt.Fatalf(\"Count mismatch: original=%d, restored=%d\", origCount, restCount)\n\t}\n\n\tt.Logf(\"✓ Data integrity verified: %d rows\", origCount)\n\tt.Log(\"TEST PASSED: Handled large WAL successfully\")\n}\n\nfunc TestConcurrentOperations(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tduration := GetTestDuration(t, 3*time.Minute)\n\tt.Logf(\"Testing: Multiple databases replicating concurrently (duration: %v)\", duration)\n\n\tdbCount := 3\n\tdbs := make([]*TestDB, dbCount)\n\n\tfor i := 0; i < dbCount; i++ {\n\t\tdbs[i] = SetupTestDB(t, fmt.Sprintf(\"concurrent-%d\", i))\n\t\tdefer dbs[i].Cleanup()\n\t}\n\n\tt.Log(\"[1] Creating databases...\")\n\tfor i, db := range dbs {\n\t\tif err := db.Create(); err != nil {\n\t\t\tt.Fatalf(\"Failed to create database %d: %v\", i, err)\n\t\t}\n\n\t\tif err := CreateTestTable(t, db.Path); err != nil {\n\t\t\tt.Fatalf(\"Failed to create table for database %d: %v\", i, err)\n\t\t}\n\t}\n\n\tt.Logf(\"✓ Created %d databases\", dbCount)\n\n\tt.Log(\"[2] Starting Litestream for all databases...\")\n\tfor i, db := range dbs {\n\t\tif err := db.StartLitestream(); err != nil {\n\t\t\tt.Fatalf(\"Failed to start Litestream for database %d: %v\", i, err)\n\t\t}\n\t\ttime.Sleep(1 * time.Second)\n\t}\n\n\tt.Logf(\"✓ All Litestream instances running\")\n\n\tt.Log(\"[3] Generating concurrent load...\")\n\tctx, cancel := context.WithTimeout(context.Background(), duration)\n\tdefer cancel()\n\n\tdone := make(chan error, dbCount)\n\n\tfor i, db := range dbs {\n\t\tgo func(idx int, database *TestDB) {\n\t\t\tconfig := DefaultLoadConfig()\n\t\t\tconfig.WriteRate = 50\n\t\t\tconfig.Duration = duration\n\t\t\tconfig.Pattern = LoadPatternConstant\n\t\t\tconfig.Workers = 2\n\n\t\t\terr := database.GenerateLoad(ctx, config.WriteRate, config.Duration, string(config.Pattern))\n\t\t\tdone <- err\n\t\t}(i, db)\n\t}\n\n\tfor i := 0; i < dbCount; i++ {\n\t\tif err := <-done; err != nil && ctx.Err() == nil {\n\t\t\tt.Logf(\"Load generation %d had error: %v\", i, err)\n\t\t}\n\t}\n\n\tt.Log(\"✓ Concurrent load complete\")\n\n\ttime.Sleep(5 * time.Second)\n\n\tt.Log(\"[4] Stopping all Litestream instances...\")\n\tfor _, db := range dbs {\n\t\tdb.StopLitestream()\n\t}\n\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[5] Verifying all replicas...\")\n\tfor i, db := range dbs {\n\t\tfileCount, err := db.GetReplicaFileCount()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to check replica %d: %v\", i, err)\n\t\t}\n\n\t\tif fileCount == 0 {\n\t\t\tt.Fatalf(\"Database %d has no replica files!\", i)\n\t\t}\n\n\t\tt.Logf(\"✓ Database %d: %d replica files\", i, fileCount)\n\t}\n\n\tt.Log(\"[6] Testing restore for all databases...\")\n\tfor i, db := range dbs {\n\t\trestoredPath := filepath.Join(db.TempDir, fmt.Sprintf(\"concurrent-restored-%d.db\", i))\n\t\tif err := db.Restore(restoredPath); err != nil {\n\t\t\tt.Fatalf(\"Restore failed for database %d: %v\", i, err)\n\t\t}\n\n\t\torigCount, _ := db.GetRowCount(\"test_data\")\n\t\trestoredDB := &TestDB{Path: restoredPath, t: t}\n\t\trestCount, _ := restoredDB.GetRowCount(\"test_data\")\n\n\t\tif origCount != restCount {\n\t\t\tt.Fatalf(\"Database %d count mismatch: original=%d, restored=%d\", i, origCount, restCount)\n\t\t}\n\n\t\tt.Logf(\"✓ Database %d verified: %d rows\", i, origCount)\n\t}\n\n\tt.Log(\"TEST PASSED: Concurrent replication works correctly\")\n}\n\nfunc TestBusyTimeout(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tt.Log(\"Testing: Database busy timeout handling\")\n\n\tdb := SetupTestDB(t, \"busy-timeout\")\n\tdefer db.Cleanup()\n\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\tt.Log(\"[1] Creating test data...\")\n\tif err := CreateTestTable(t, db.Path); err != nil {\n\t\tt.Fatalf(\"Failed to create table: %v\", err)\n\t}\n\n\tif err := InsertTestData(t, db.Path, 100); err != nil {\n\t\tt.Fatalf(\"Failed to insert test data: %v\", err)\n\t}\n\n\tt.Log(\"✓ Created table with 100 rows\")\n\n\tt.Log(\"[2] Starting Litestream...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(3 * time.Second)\n\n\tt.Log(\"[3] Simulating concurrent access with long transactions...\")\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path+\"?_busy_timeout=5000\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to open database: %v\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\ttx, err := sqlDB.Begin()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to begin transaction: %v\", err)\n\t}\n\n\tfor i := 0; i < 500; i++ {\n\t\tif _, err := tx.Exec(\n\t\t\t\"INSERT INTO test_data (data, created_at) VALUES (?, ?)\",\n\t\t\tfmt.Sprintf(\"busy test %d\", i),\n\t\t\ttime.Now().Unix(),\n\t\t); err != nil {\n\t\t\tt.Fatalf(\"Failed to insert in transaction: %v\", err)\n\t\t}\n\n\t\tif i%100 == 0 {\n\t\t\ttime.Sleep(500 * time.Millisecond)\n\t\t}\n\t}\n\n\tif err := tx.Commit(); err != nil {\n\t\tt.Fatalf(\"Failed to commit transaction: %v\", err)\n\t}\n\n\tt.Log(\"✓ Long transaction completed\")\n\n\ttime.Sleep(5 * time.Second)\n\n\tdb.StopLitestream()\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[4] Checking for errors...\")\n\terrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check errors: %v\", err)\n\t}\n\n\tif len(errors) > 0 {\n\t\tt.Logf(\"Found %d errors (may include busy timeout messages)\", len(errors))\n\t}\n\n\tt.Log(\"[5] Testing restore...\")\n\trestoredPath := filepath.Join(db.TempDir, \"busy-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Restore successful\")\n\n\torigCount, err := db.GetRowCount(\"test_data\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get original row count: %v\", err)\n\t}\n\n\trestoredDB := &TestDB{Path: restoredPath, t: t}\n\trestCount, err := restoredDB.GetRowCount(\"test_data\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get restored row count: %v\", err)\n\t}\n\n\tif origCount != restCount {\n\t\tt.Fatalf(\"Count mismatch: original=%d, restored=%d\", origCount, restCount)\n\t}\n\n\tt.Logf(\"✓ Data integrity verified: %d rows\", origCount)\n\tt.Log(\"TEST PASSED: Busy timeout handled correctly\")\n}\n\nfunc getFileSize(path string) (int64, error) {\n\tinfo, err := os.Stat(path)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\treturn info.Size(), nil\n}\n" }, { "path": "tests/integration/directory_watcher_helpers.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\n// DirWatchTestDB extends TestDB with directory-specific functionality\ntype DirWatchTestDB struct {\n\t*TestDB\n\tDirPath string\n\tPattern string\n\tRecursive bool\n\tWatch bool\n\tReplicaPath string\n}\n\n// SetupDirectoryWatchTest creates a test environment for directory watching\nfunc SetupDirectoryWatchTest(t *testing.T, name string, pattern string, recursive bool) *DirWatchTestDB {\n\tt.Helper()\n\n\tbaseDB := SetupTestDB(t, name)\n\tdirPath := filepath.Join(baseDB.TempDir, \"databases\")\n\tif err := os.MkdirAll(dirPath, 0755); err != nil {\n\t\tt.Fatalf(\"create databases directory: %v\", err)\n\t}\n\n\treplicaPath := filepath.Join(baseDB.TempDir, \"replica\")\n\n\treturn &DirWatchTestDB{\n\t\tTestDB: baseDB,\n\t\tDirPath: dirPath,\n\t\tPattern: pattern,\n\t\tRecursive: recursive,\n\t\tWatch: true,\n\t\tReplicaPath: replicaPath,\n\t}\n}\n\n// CreateDirectoryWatchConfig generates YAML config for directory watching\nfunc (db *DirWatchTestDB) CreateDirectoryWatchConfig() (string, error) {\n\tconfigPath := filepath.Join(db.TempDir, \"litestream.yml\")\n\tconfig := fmt.Sprintf(`dbs:\n - dir: %s\n pattern: %q\n recursive: %t\n watch: %t\n replica:\n type: file\n path: %s\n`,\n\t\tfilepath.ToSlash(db.DirPath),\n\t\tdb.Pattern,\n\t\tdb.Recursive,\n\t\tdb.Watch,\n\t\tfilepath.ToSlash(db.ReplicaPath),\n\t)\n\n\tif err := os.WriteFile(configPath, []byte(config), 0644); err != nil {\n\t\treturn \"\", fmt.Errorf(\"write config: %w\", err)\n\t}\n\n\tdb.ConfigPath = configPath\n\treturn configPath, nil\n}\n\n// CreateDatabaseInDir creates a SQLite database with optional subdirectory\nfunc CreateDatabaseInDir(t *testing.T, dirPath, subDir, name string) string {\n\tt.Helper()\n\n\tdbDir := dirPath\n\tif subDir != \"\" {\n\t\tdbDir = filepath.Join(dirPath, subDir)\n\t\tif err := os.MkdirAll(dbDir, 0755); err != nil {\n\t\t\tt.Fatalf(\"create subdirectory %s: %v\", subDir, err)\n\t\t}\n\t\t// Give directory monitor time to register watch on new subdirectory\n\t\t// to avoid race where database is created before watch is active\n\t\ttime.Sleep(500 * time.Millisecond)\n\t}\n\n\tdbPath := filepath.Join(dbDir, name)\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\tt.Fatalf(\"open database %s: %v\", dbPath, err)\n\t}\n\tdefer sqlDB.Close()\n\n\tif _, err := sqlDB.Exec(\"PRAGMA journal_mode=WAL\"); err != nil {\n\t\tt.Fatalf(\"set WAL mode for %s: %v\", dbPath, err)\n\t}\n\n\t// Create a simple table to make it a real database\n\tif _, err := sqlDB.Exec(\"CREATE TABLE IF NOT EXISTS test (id INTEGER PRIMARY KEY, data TEXT)\"); err != nil {\n\t\tt.Fatalf(\"create table in %s: %v\", dbPath, err)\n\t}\n\n\treturn dbPath\n}\n\n// CreateDatabaseWithData creates a database with specified number of rows\nfunc CreateDatabaseWithData(t *testing.T, dbPath string, rowCount int) error {\n\tt.Helper()\n\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open database: %w\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tif _, err := sqlDB.Exec(\"PRAGMA journal_mode=WAL\"); err != nil {\n\t\treturn fmt.Errorf(\"set WAL mode: %w\", err)\n\t}\n\n\tif _, err := sqlDB.Exec(\"CREATE TABLE IF NOT EXISTS test (id INTEGER PRIMARY KEY, data TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP)\"); err != nil {\n\t\treturn fmt.Errorf(\"create table: %w\", err)\n\t}\n\n\t// Insert data in batches\n\ttx, err := sqlDB.Begin()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"begin transaction: %w\", err)\n\t}\n\n\tstmt, err := tx.Prepare(\"INSERT INTO test (data) VALUES (?)\")\n\tif err != nil {\n\t\ttx.Rollback()\n\t\treturn fmt.Errorf(\"prepare statement: %w\", err)\n\t}\n\n\tfor i := 0; i < rowCount; i++ {\n\t\tif _, err := stmt.Exec(fmt.Sprintf(\"test data %d\", i)); err != nil {\n\t\t\ttx.Rollback()\n\t\t\treturn fmt.Errorf(\"insert row %d: %w\", i, err)\n\t\t}\n\t}\n\n\tif err := tx.Commit(); err != nil {\n\t\treturn fmt.Errorf(\"commit transaction: %w\", err)\n\t}\n\n\treturn nil\n}\n\n// CreateFakeDatabase creates a file that looks like a database but isn't\nfunc CreateFakeDatabase(t *testing.T, dirPath, name string, content []byte) string {\n\tt.Helper()\n\n\tdbPath := filepath.Join(dirPath, name)\n\tif err := os.WriteFile(dbPath, content, 0644); err != nil {\n\t\tt.Fatalf(\"write fake database %s: %v\", dbPath, err)\n\t}\n\n\treturn dbPath\n}\n\n// WaitForDatabaseInReplica polls until database appears in replica\n// For directory watching, dbPath can be the full path or just the database name\nfunc WaitForDatabaseInReplica(t *testing.T, replicaPath, dbPath string, timeout time.Duration) error {\n\tt.Helper()\n\n\t// Replica structure: //ltx/0/*.ltx\n\tdbName := filepath.Base(dbPath)\n\n\t// Try to find the database in the replica directory\n\t// It could be at the root level or nested in subdirectories\n\tdeadline := time.Now().Add(timeout)\n\tfor time.Now().Before(deadline) {\n\t\t// Walk the replica directory to find the database\n\t\tfound := false\n\t\tfilepath.Walk(replicaPath, func(path string, info os.FileInfo, err error) error {\n\t\t\tif err != nil || found {\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\t// Check if this directory matches the database name and has LTX files\n\t\t\tif info.IsDir() && filepath.Base(path) == dbName {\n\t\t\t\tltxDir := filepath.Join(path, \"ltx\", \"0\")\n\t\t\t\tif _, err := os.Stat(ltxDir); err == nil {\n\t\t\t\t\tentries, err := os.ReadDir(ltxDir)\n\t\t\t\t\tif err == nil {\n\t\t\t\t\t\tfor _, entry := range entries {\n\t\t\t\t\t\t\tif strings.HasSuffix(entry.Name(), \".ltx\") {\n\t\t\t\t\t\t\t\trelPath, _ := filepath.Rel(replicaPath, path)\n\t\t\t\t\t\t\t\tt.Logf(\"Database %s detected in replica at %s (found %s)\", dbName, relPath, entry.Name())\n\t\t\t\t\t\t\t\tfound = true\n\t\t\t\t\t\t\t\treturn nil\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\n\t\tif found {\n\t\t\treturn nil\n\t\t}\n\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\n\treturn fmt.Errorf(\"database %s not found in replica after %v\", dbName, timeout)\n}\n\n// VerifyDatabaseRemoved checks database no longer in replica (no new writes)\nfunc VerifyDatabaseRemoved(t *testing.T, replicaPath, dbPath string, timeout time.Duration) error {\n\tt.Helper()\n\n\t// Replica structure: //ltx/0/*.ltx\n\tdbName := filepath.Base(dbPath)\n\tltxDir := filepath.Join(replicaPath, dbName, \"ltx\", \"0\")\n\n\t// Count initial LTX files (using existing countLTXFiles helper)\n\tinitialCount := countLTXFiles(ltxDir)\n\n\tt.Logf(\"Initial LTX count for %s: %d\", dbName, initialCount)\n\n\t// Wait and verify no new files are created\n\ttime.Sleep(timeout)\n\n\tfinalCount := countLTXFiles(ltxDir)\n\n\tif finalCount > initialCount {\n\t\treturn fmt.Errorf(\"database %s still being replicated (%d -> %d LTX files)\", dbName, initialCount, finalCount)\n\t}\n\n\tt.Logf(\"Database %s stopped replicating\", dbName)\n\treturn nil\n}\n\n// CountDatabasesInReplica counts distinct databases being replicated\nfunc CountDatabasesInReplica(replicaPath string) (int, error) {\n\tif _, err := os.Stat(replicaPath); os.IsNotExist(err) {\n\t\treturn 0, nil\n\t}\n\n\tentries, err := os.ReadDir(replicaPath)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tcount := 0\n\tfor _, entry := range entries {\n\t\tif !entry.IsDir() {\n\t\t\tcontinue\n\t\t}\n\t\t// Check if this database directory has LTX files\n\t\tltxDir := filepath.Join(replicaPath, entry.Name(), \"ltx\", \"0\")\n\t\tif countLTXFiles(ltxDir) > 0 {\n\t\t\tcount++\n\t\t}\n\t}\n\n\treturn count, nil\n}\n\n// StartContinuousWrites launches goroutine writing to database at specified rate\nfunc StartContinuousWrites(ctx context.Context, t *testing.T, dbPath string, writesPerSec int) (*sync.WaitGroup, context.CancelFunc, error) {\n\tt.Helper()\n\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\treturn nil, nil, fmt.Errorf(\"open database: %w\", err)\n\t}\n\n\tif _, err := sqlDB.Exec(\"CREATE TABLE IF NOT EXISTS load_test (id INTEGER PRIMARY KEY AUTOINCREMENT, data TEXT, ts DATETIME DEFAULT CURRENT_TIMESTAMP)\"); err != nil {\n\t\tsqlDB.Close()\n\t\treturn nil, nil, fmt.Errorf(\"create table: %w\", err)\n\t}\n\n\tctx, cancel := context.WithCancel(ctx)\n\twg := &sync.WaitGroup{}\n\twg.Add(1)\n\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tdefer sqlDB.Close()\n\n\t\tticker := time.NewTicker(time.Second / time.Duration(writesPerSec))\n\t\tdefer ticker.Stop()\n\n\t\tcounter := 0\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tcase <-ticker.C:\n\t\t\t\tcounter++\n\t\t\t\tif _, err := sqlDB.Exec(\"INSERT INTO load_test (data) VALUES (?)\", fmt.Sprintf(\"data-%d\", counter)); err != nil {\n\t\t\t\t\tif !strings.Contains(err.Error(), \"database is locked\") {\n\t\t\t\t\t\tt.Logf(\"Write error in %s: %v\", filepath.Base(dbPath), err)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn wg, cancel, nil\n}\n\n// CreateMultipleDatabasesConcurrently creates databases using goroutines\nfunc CreateMultipleDatabasesConcurrently(t *testing.T, dirPath string, count int, pattern string) []string {\n\tt.Helper()\n\n\tvar wg sync.WaitGroup\n\tvar mu sync.Mutex\n\tpaths := make([]string, 0, count)\n\n\tfor i := 0; i < count; i++ {\n\t\twg.Add(1)\n\t\tgo func(idx int) {\n\t\t\tdefer wg.Done()\n\n\t\t\tname := fmt.Sprintf(\"db-%03d%s\", idx, filepath.Ext(pattern))\n\t\t\tdbPath := CreateDatabaseInDir(t, dirPath, \"\", name)\n\n\t\t\tmu.Lock()\n\t\t\tpaths = append(paths, dbPath)\n\t\t\tmu.Unlock()\n\t\t}(i)\n\t}\n\n\twg.Wait()\n\treturn paths\n}\n\n// GetRowCount returns the number of rows in a test table\nfunc GetRowCount(dbPath, tableName string) (int, error) {\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"open database: %w\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tvar count int\n\terr = sqlDB.QueryRow(fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", tableName)).Scan(&count)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"query count: %w\", err)\n\t}\n\n\treturn count, nil\n}\n\n// Helper functions\n\nfunc getRelativeDBPath(dbPath, replicaBase string) (string, error) {\n\t// Extract just the database name (not the full path)\n\t// The replica structure mirrors the source directory structure\n\treturn filepath.Base(dbPath), nil\n}\n\nfunc hasLTXFiles(dir string) (bool, error) {\n\tentries, err := os.ReadDir(dir)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tfor _, entry := range entries {\n\t\tif strings.HasSuffix(entry.Name(), \".ltx\") {\n\t\t\treturn true, nil\n\t\t}\n\t}\n\treturn false, nil\n}\n\n// CheckForCriticalErrors returns errors from the log, filtering out known benign errors\nfunc CheckForCriticalErrors(t *testing.T, db *TestDB) ([]string, error) {\n\tt.Helper()\n\n\tallErrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Filter out known benign errors\n\tvar criticalErrors []string\n\tfor _, errLine := range allErrors {\n\t\t// Skip benign database removal errors that occur when closing databases\n\t\tif strings.Contains(errLine, \"remove database from store\") &&\n\t\t\t(strings.Contains(errLine, \"transaction has already been committed or rolled back\") ||\n\t\t\t\tstrings.Contains(errLine, \"no such file or directory\") ||\n\t\t\t\tstrings.Contains(errLine, \"disk I/O error\")) {\n\t\t\tcontinue\n\t\t}\n\t\tcriticalErrors = append(criticalErrors, errLine)\n\t}\n\n\treturn criticalErrors, nil\n}\n" }, { "path": "tests/integration/directory_watcher_test.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n)\n\n// TestDirectoryWatcherBasicLifecycle tests the fundamental directory watcher functionality:\n// - Start with empty directory\n// - Create databases while Litestream is running\n// - Verify they are detected and replicated\n// - Delete databases and verify cleanup\nfunc TestDirectoryWatcherBasicLifecycle(t *testing.T) {\n\tRequireBinaries(t)\n\n\t// Use recursive:true because this test creates databases in subdirectories (tenant1/app.db, etc.)\n\tdb := SetupDirectoryWatchTest(t, \"dir-watch-lifecycle\", \"*.db\", true)\n\n\t// Create config with directory watching\n\tconfigPath, err := db.CreateDirectoryWatchConfig()\n\tif err != nil {\n\t\tt.Fatalf(\"create config: %v\", err)\n\t}\n\n\tt.Log(\"Starting Litestream with directory watching...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\tdefer db.StopLitestream()\n\n\t// Give Litestream time to start\n\ttime.Sleep(2 * time.Second)\n\n\t// Step 1: Create 2 databases in separate tenant directories\n\tt.Log(\"Creating databases in separate directories...\")\n\ttenant1DB := CreateDatabaseInDir(t, db.DirPath, \"tenant1\", \"app.db\")\n\ttenant2DB := CreateDatabaseInDir(t, db.DirPath, \"tenant2\", \"app.db\")\n\n\t// Wait for detection and replication\n\tt.Log(\"Waiting for database detection...\")\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, tenant1DB, 5*time.Second); err != nil {\n\t\tt.Fatalf(\"tenant1 database not detected: %v\", err)\n\t}\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, tenant2DB, 5*time.Second); err != nil {\n\t\tt.Fatalf(\"tenant2 database not detected: %v\", err)\n\t}\n\n\t// Step 2: Add data to both databases\n\tt.Log(\"Adding data to databases...\")\n\tif err := CreateDatabaseWithData(t, tenant1DB, 100); err != nil {\n\t\tt.Fatalf(\"add data to tenant1: %v\", err)\n\t}\n\tif err := CreateDatabaseWithData(t, tenant2DB, 100); err != nil {\n\t\tt.Fatalf(\"add data to tenant2: %v\", err)\n\t}\n\n\t// Wait for replication\n\ttime.Sleep(3 * time.Second)\n\n\t// Step 3: Create 3 more databases at intervals\n\tt.Log(\"Creating additional databases at intervals...\")\n\tdb3 := CreateDatabaseInDir(t, db.DirPath, \"tenant3\", \"app.db\")\n\ttime.Sleep(1 * time.Second)\n\n\tdb4 := CreateDatabaseInDir(t, db.DirPath, \"\", \"standalone.db\")\n\ttime.Sleep(1 * time.Second)\n\n\tdb5 := CreateDatabaseInDir(t, db.DirPath, \"tenant4\", \"data.db\")\n\n\t// Wait for all to be detected\n\tt.Log(\"Verifying all databases detected...\")\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, db3, 5*time.Second); err != nil {\n\t\tt.Fatalf(\"tenant3 database not detected: %v\", err)\n\t}\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, db4, 5*time.Second); err != nil {\n\t\tt.Fatalf(\"standalone database not detected: %v\", err)\n\t}\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, db5, 5*time.Second); err != nil {\n\t\tt.Fatalf(\"tenant4 database not detected: %v\", err)\n\t}\n\n\t// Step 4: Delete one database and verify cleanup\n\tt.Log(\"Deleting database and verifying cleanup...\")\n\tif err := os.Remove(db4); err != nil {\n\t\tt.Fatalf(\"remove database: %v\", err)\n\t}\n\n\t// Wait and verify no more replication\n\tif err := VerifyDatabaseRemoved(t, db.ReplicaPath, db4, 3*time.Second); err != nil {\n\t\tt.Fatalf(\"database still replicating after removal: %v\", err)\n\t}\n\n\t// Step 5: Verify no critical errors in log\n\tt.Log(\"Checking for errors...\")\n\terrors, err := CheckForCriticalErrors(t, db.TestDB)\n\tif err != nil {\n\t\tt.Fatalf(\"check errors: %v\", err)\n\t}\n\tif len(errors) > 0 {\n\t\tt.Fatalf(\"found critical errors in log: %v\", errors)\n\t}\n\n\tt.Log(\"✓ Basic lifecycle test passed\")\n}\n\n// TestDirectoryWatcherRapidConcurrentCreation tests race conditions with rapid database creation\nfunc TestDirectoryWatcherRapidConcurrentCreation(t *testing.T) {\n\tRequireBinaries(t)\n\n\tdb := SetupDirectoryWatchTest(t, \"dir-watch-concurrent\", \"*.db\", false)\n\n\tconfigPath, err := db.CreateDirectoryWatchConfig()\n\tif err != nil {\n\t\tt.Fatalf(\"create config: %v\", err)\n\t}\n\n\tt.Log(\"Starting Litestream with directory watching...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\tdefer db.StopLitestream()\n\n\ttime.Sleep(2 * time.Second)\n\n\t// Create 20 databases simultaneously\n\tt.Log(\"Creating 20 databases concurrently...\")\n\tdbPaths := CreateMultipleDatabasesConcurrently(t, db.DirPath, 20, \"*.db\")\n\n\t// Wait for all databases to be detected\n\tt.Log(\"Verifying all databases detected...\")\n\tfor i, dbPath := range dbPaths {\n\t\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, dbPath, 10*time.Second); err != nil {\n\t\t\tt.Fatalf(\"database %d (%s) not detected: %v\", i, filepath.Base(dbPath), err)\n\t\t}\n\t}\n\n\t// Count databases in replica\n\tcount, err := CountDatabasesInReplica(db.ReplicaPath)\n\tif err != nil {\n\t\tt.Fatalf(\"count databases: %v\", err)\n\t}\n\n\tif count != 20 {\n\t\tt.Fatalf(\"expected 20 databases in replica, got %d\", count)\n\t}\n\n\t// Check for errors (especially duplicate registrations or race conditions)\n\terrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\tt.Fatalf(\"check errors: %v\", err)\n\t}\n\tif len(errors) > 0 {\n\t\tt.Fatalf(\"found errors in log (possible race conditions): %v\", errors)\n\t}\n\n\tt.Log(\"✓ Concurrent creation test passed\")\n}\n\n// TestDirectoryWatcherRecursiveMode tests recursive directory scanning\nfunc TestDirectoryWatcherRecursiveMode(t *testing.T) {\n\tRequireBinaries(t)\n\n\tdb := SetupDirectoryWatchTest(t, \"dir-watch-recursive\", \"*.db\", true)\n\n\tconfigPath, err := db.CreateDirectoryWatchConfig()\n\tif err != nil {\n\t\tt.Fatalf(\"create config: %v\", err)\n\t}\n\n\tt.Log(\"Starting Litestream with recursive directory watching...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\n\ttime.Sleep(2 * time.Second)\n\n\t// Create nested directory structure\n\tt.Log(\"Creating nested directory structure...\")\n\tdb1 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db1.db\") // root/db1.db\n\tdb2 := CreateDatabaseInDir(t, db.DirPath, \"level1\", \"db2.db\") // root/level1/db2.db\n\n\t// Verify first two detected\n\tt.Log(\"Verifying databases detected...\")\n\tfor i, dbPath := range []string{db1, db2} {\n\t\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, dbPath, 10*time.Second); err != nil {\n\t\t\tt.Fatalf(\"database %d (%s) not detected: %v\", i+1, filepath.Base(dbPath), err)\n\t\t}\n\t}\n\n\t// Try deeper nesting (may be slower to detect)\n\tt.Log(\"Creating deeply nested database...\")\n\tdb3 := CreateDatabaseInDir(t, db.DirPath, \"level1/level2\", \"db3.db\") // root/level1/level2/db3.db\n\n\t// Give more time for deeply nested database\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, db3, 15*time.Second); err != nil {\n\t\tt.Logf(\"Warning: deeply nested database (2 levels) not detected: %v\", err)\n\t\t// Don't fail the test - recursive watching of deeply nested dirs may have limitations\n\t}\n\n\t// Create new subdirectory after start\n\tt.Log(\"Creating new subdirectory dynamically...\")\n\tnewDir := filepath.Join(db.DirPath, \"dynamic\")\n\tif err := os.MkdirAll(newDir, 0755); err != nil {\n\t\tt.Fatalf(\"create dynamic dir: %v\", err)\n\t}\n\ttime.Sleep(500 * time.Millisecond) // Allow directory watch to register\n\n\tdb5 := CreateDatabaseInDir(t, db.DirPath, \"dynamic\", \"db5.db\")\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, db5, 10*time.Second); err != nil {\n\t\tt.Fatalf(\"dynamically created database not detected: %v\", err)\n\t}\n\n\t// Stop Litestream before deleting directories to release file handles\n\tt.Log(\"Stopping Litestream before directory deletion...\")\n\tdb.StopLitestream()\n\n\t// Delete entire subdirectory (now safe since Litestream released handles)\n\tt.Log(\"Deleting subdirectory with databases...\")\n\tlevel1Dir := filepath.Join(db.DirPath, \"level1\")\n\tif err := os.RemoveAll(level1Dir); err != nil {\n\t\tt.Fatalf(\"remove level1 directory: %v\", err)\n\t}\n\n\tt.Log(\"✓ Recursive mode test passed\")\n}\n\n// TestDirectoryWatcherPatternMatching tests glob pattern matching\nfunc TestDirectoryWatcherPatternMatching(t *testing.T) {\n\tRequireBinaries(t)\n\n\tdb := SetupDirectoryWatchTest(t, \"dir-watch-pattern\", \"*.db\", false)\n\n\tconfigPath, err := db.CreateDirectoryWatchConfig()\n\tif err != nil {\n\t\tt.Fatalf(\"create config: %v\", err)\n\t}\n\n\tt.Log(\"Starting Litestream with pattern '*.db'...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\tdefer db.StopLitestream()\n\n\ttime.Sleep(2 * time.Second)\n\n\t// Create files with different extensions\n\tt.Log(\"Creating files with various patterns...\")\n\tmatchDB := CreateDatabaseInDir(t, db.DirPath, \"\", \"test.db\") // Should match\n\tnoMatchSQLite := CreateDatabaseInDir(t, db.DirPath, \"\", \"test.sqlite\") // Should NOT match\n\tnoMatchBackup := CreateFakeDatabase(t, db.DirPath, \"test.db.backup\", []byte{}) // Should NOT match\n\n\t// Also create WAL and SHM files (should be ignored)\n\tCreateFakeDatabase(t, db.DirPath, \"test.db-wal\", []byte{})\n\tCreateFakeDatabase(t, db.DirPath, \"test.db-shm\", []byte{})\n\n\t// Wait and verify only .db file is replicated\n\tt.Log(\"Verifying pattern matching...\")\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, matchDB, 5*time.Second); err != nil {\n\t\tt.Fatalf(\"*.db file should be detected: %v\", err)\n\t}\n\n\t// Give time for other files to be processed (they shouldn't be)\n\ttime.Sleep(3 * time.Second)\n\n\t// Count - should only have 1 database\n\tcount, err := CountDatabasesInReplica(db.ReplicaPath)\n\tif err != nil {\n\t\tt.Fatalf(\"count databases: %v\", err)\n\t}\n\n\tif count != 1 {\n\t\tt.Fatalf(\"expected 1 database in replica, got %d (pattern matching failed)\", count)\n\t}\n\n\t// Verify .sqlite and .db.backup files were not added\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, noMatchSQLite, 2*time.Second); err == nil {\n\t\tt.Fatal(\"*.sqlite file should NOT be detected with *.db pattern\")\n\t}\n\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, noMatchBackup, 2*time.Second); err == nil {\n\t\tt.Fatal(\"*.db.backup file should NOT be detected with *.db pattern\")\n\t}\n\n\tt.Log(\"✓ Pattern matching test passed\")\n}\n\n// TestDirectoryWatcherNonSQLiteRejection tests that non-SQLite files are rejected\nfunc TestDirectoryWatcherNonSQLiteRejection(t *testing.T) {\n\tRequireBinaries(t)\n\n\tdb := SetupDirectoryWatchTest(t, \"dir-watch-nonsqlite\", \"*.db\", false)\n\n\tconfigPath, err := db.CreateDirectoryWatchConfig()\n\tif err != nil {\n\t\tt.Fatalf(\"create config: %v\", err)\n\t}\n\n\tt.Log(\"Starting Litestream...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\tdefer db.StopLitestream()\n\n\ttime.Sleep(2 * time.Second)\n\n\t// Create fake database files\n\tt.Log(\"Creating non-SQLite files...\")\n\tfakeDB := CreateFakeDatabase(t, db.DirPath, \"fake.db\", []byte(\"this is not a sqlite file\"))\n\temptyDB := CreateFakeDatabase(t, db.DirPath, \"empty.db\", []byte{})\n\ttextDB := CreateFakeDatabase(t, db.DirPath, \"text.db\", []byte(\"SQLite format 2\\x00\")) // Wrong version\n\n\t// Create one valid SQLite database\n\tvalidDB := CreateDatabaseInDir(t, db.DirPath, \"\", \"valid.db\")\n\n\t// Wait for valid database\n\tt.Log(\"Verifying only valid SQLite database is detected...\")\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, validDB, 5*time.Second); err != nil {\n\t\tt.Fatalf(\"valid database should be detected: %v\", err)\n\t}\n\n\t// Wait to ensure fake databases are not added\n\ttime.Sleep(3 * time.Second)\n\n\t// Should only have 1 database\n\tcount, err := CountDatabasesInReplica(db.ReplicaPath)\n\tif err != nil {\n\t\tt.Fatalf(\"count databases: %v\", err)\n\t}\n\n\tif count != 1 {\n\t\tt.Fatalf(\"expected 1 database in replica, got %d (non-SQLite files were not rejected)\", count)\n\t}\n\n\t// Verify fake files were not added\n\tfor _, fakePath := range []string{fakeDB, emptyDB, textDB} {\n\t\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, fakePath, 1*time.Second); err == nil {\n\t\t\tt.Fatalf(\"non-SQLite file %s should NOT be replicated\", filepath.Base(fakePath))\n\t\t}\n\t}\n\n\tt.Log(\"✓ Non-SQLite rejection test passed\")\n}\n\n// TestDirectoryWatcherActiveConnections tests behavior with databases that are actively being used\nfunc TestDirectoryWatcherActiveConnections(t *testing.T) {\n\tRequireBinaries(t)\n\n\tdb := SetupDirectoryWatchTest(t, \"dir-watch-active\", \"*.db\", false)\n\n\tconfigPath, err := db.CreateDirectoryWatchConfig()\n\tif err != nil {\n\t\tt.Fatalf(\"create config: %v\", err)\n\t}\n\n\t// Create database with active connection before starting Litestream\n\tt.Log(\"Creating database with active connection...\")\n\tdb1Path := CreateDatabaseInDir(t, db.DirPath, \"\", \"active.db\")\n\n\t// Start continuous writes\n\tctx := context.Background()\n\twg, cancel, err := StartContinuousWrites(ctx, t, db1Path, 10) // 10 writes/sec\n\tif err != nil {\n\t\tt.Fatalf(\"start writes: %v\", err)\n\t}\n\tdefer cancel()\n\n\t// Start Litestream\n\tt.Log(\"Starting Litestream with active database...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\tdefer db.StopLitestream()\n\n\ttime.Sleep(2 * time.Second)\n\n\t// Verify database is detected despite active connection\n\tt.Log(\"Verifying active database is detected...\")\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, db1Path, 10*time.Second); err != nil {\n\t\tt.Fatalf(\"active database not detected: %v\", err)\n\t}\n\n\t// Create second database and start writing to it\n\tt.Log(\"Creating second database with writes...\")\n\tdb2Path := CreateDatabaseInDir(t, db.DirPath, \"\", \"active2.db\")\n\twg2, cancel2, err := StartContinuousWrites(ctx, t, db2Path, 5)\n\tif err != nil {\n\t\tt.Fatalf(\"start writes for db2: %v\", err)\n\t}\n\tdefer cancel2()\n\n\t// Verify second database detected\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, db2Path, 10*time.Second); err != nil {\n\t\tt.Fatalf(\"second active database not detected: %v\", err)\n\t}\n\n\t// Let writes continue for a bit\n\tt.Log(\"Letting writes continue for 5 seconds...\")\n\ttime.Sleep(5 * time.Second)\n\n\t// Stop writers\n\tcancel()\n\tcancel2()\n\twg.Wait()\n\twg2.Wait()\n\n\t// Verify both databases are still replicated\n\tcount, err := CountDatabasesInReplica(db.ReplicaPath)\n\tif err != nil {\n\t\tt.Fatalf(\"count databases: %v\", err)\n\t}\n\n\tif count != 2 {\n\t\tt.Fatalf(\"expected 2 databases in replica, got %d\", count)\n\t}\n\n\terrors, err := CheckForCriticalErrors(t, db.TestDB)\n\tif err != nil {\n\t\tt.Fatalf(\"check errors: %v\", err)\n\t}\n\tif len(errors) > 0 {\n\t\tt.Fatalf(\"found critical errors with active connections: %v\", errors)\n\t}\n\n\tt.Log(\"✓ Active connections test passed\")\n}\n\n// TestDirectoryWatcherRestartBehavior tests behavior across Litestream restarts\nfunc TestDirectoryWatcherRestartBehavior(t *testing.T) {\n\tRequireBinaries(t)\n\n\tdb := SetupDirectoryWatchTest(t, \"dir-watch-restart\", \"*.db\", false)\n\n\t// Create 3 databases before starting\n\tt.Log(\"Creating initial databases...\")\n\tdb1 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db1.db\")\n\tdb2 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db2.db\")\n\tdb3 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db3.db\")\n\n\tconfigPath, err := db.CreateDirectoryWatchConfig()\n\tif err != nil {\n\t\tt.Fatalf(\"create config: %v\", err)\n\t}\n\n\t// First start\n\tt.Log(\"Starting Litestream (first time)...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\n\ttime.Sleep(3 * time.Second)\n\n\t// Verify all 3 detected\n\tfor _, dbPath := range []string{db1, db2, db3} {\n\t\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, dbPath, 5*time.Second); err != nil {\n\t\t\tt.Fatalf(\"database %s not detected: %v\", filepath.Base(dbPath), err)\n\t\t}\n\t}\n\n\t// Add 2 more databases dynamically\n\tt.Log(\"Adding databases dynamically...\")\n\tdb4 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db4.db\")\n\tdb5 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db5.db\")\n\n\ttime.Sleep(3 * time.Second)\n\n\t// Verify new databases detected\n\tfor _, dbPath := range []string{db4, db5} {\n\t\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, dbPath, 5*time.Second); err != nil {\n\t\t\tt.Fatalf(\"dynamically added database %s not detected: %v\", filepath.Base(dbPath), err)\n\t\t}\n\t}\n\n\t// Stop Litestream\n\tt.Log(\"Stopping Litestream...\")\n\tif err := db.StopLitestream(); err != nil {\n\t\tt.Fatalf(\"stop litestream: %v\", err)\n\t}\n\n\t// Add one more database while stopped\n\tt.Log(\"Adding database while Litestream is stopped...\")\n\tdb6 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db6.db\")\n\n\ttime.Sleep(2 * time.Second)\n\n\t// Restart Litestream\n\tt.Log(\"Restarting Litestream...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"restart litestream: %v\", err)\n\t}\n\tdefer db.StopLitestream()\n\n\ttime.Sleep(3 * time.Second)\n\n\t// Verify all 6 databases are now being replicated\n\tt.Log(\"Verifying all databases detected after restart...\")\n\tfor i, dbPath := range []string{db1, db2, db3, db4, db5, db6} {\n\t\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, dbPath, 10*time.Second); err != nil {\n\t\t\tt.Fatalf(\"database %d (%s) not detected after restart: %v\", i+1, filepath.Base(dbPath), err)\n\t\t}\n\t}\n\n\t// Add one more dynamically after restart\n\tt.Log(\"Adding database after restart...\")\n\tdb7 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db7.db\")\n\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, db7, 5*time.Second); err != nil {\n\t\tt.Fatalf(\"database added after restart not detected: %v\", err)\n\t}\n\n\t// Final count - should have 7 databases\n\tcount, err := CountDatabasesInReplica(db.ReplicaPath)\n\tif err != nil {\n\t\tt.Fatalf(\"count databases: %v\", err)\n\t}\n\n\tif count != 7 {\n\t\tt.Fatalf(\"expected 7 databases in replica, got %d\", count)\n\t}\n\n\tt.Log(\"✓ Restart behavior test passed\")\n}\n\n// TestDirectoryWatcherRenameOperations tests file rename handling\nfunc TestDirectoryWatcherRenameOperations(t *testing.T) {\n\tRequireBinaries(t)\n\n\tdb := SetupDirectoryWatchTest(t, \"dir-watch-rename\", \"*.db\", false)\n\n\tconfigPath, err := db.CreateDirectoryWatchConfig()\n\tif err != nil {\n\t\tt.Fatalf(\"create config: %v\", err)\n\t}\n\n\tt.Log(\"Starting Litestream...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\tdefer db.StopLitestream()\n\n\ttime.Sleep(2 * time.Second)\n\n\t// Create database\n\tt.Log(\"Creating database...\")\n\toriginalPath := CreateDatabaseInDir(t, db.DirPath, \"\", \"original.db\")\n\n\t// Wait for replication\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, originalPath, 5*time.Second); err != nil {\n\t\tt.Fatalf(\"original database not detected: %v\", err)\n\t}\n\n\ttime.Sleep(2 * time.Second)\n\n\t// Rename database\n\tt.Log(\"Renaming database...\")\n\trenamedPath := filepath.Join(db.DirPath, \"renamed.db\")\n\tif err := os.Rename(originalPath, renamedPath); err != nil {\n\t\tt.Fatalf(\"rename database: %v\", err)\n\t}\n\n\t// Wait for new name to be detected\n\tt.Log(\"Waiting for renamed database to be detected...\")\n\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, renamedPath, 10*time.Second); err != nil {\n\t\tt.Fatalf(\"renamed database not detected: %v\", err)\n\t}\n\n\t// Verify old database stopped replicating\n\tt.Log(\"Verifying original database stopped replicating...\")\n\tif err := VerifyDatabaseRemoved(t, db.ReplicaPath, originalPath, 3*time.Second); err != nil {\n\t\tt.Logf(\"Warning: original may still be replicating: %v\", err)\n\t}\n\n\tt.Log(\"✓ Rename operations test passed\")\n}\n\n// TestDirectoryWatcherLoadWithWrites tests directory watching with concurrent database writes\nfunc TestDirectoryWatcherLoadWithWrites(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping load test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tdb := SetupDirectoryWatchTest(t, \"dir-watch-load\", \"*.db\", false)\n\n\t// Create 3 databases with data\n\tt.Log(\"Creating initial databases with data...\")\n\tdb1 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db1.db\")\n\tdb2 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db2.db\")\n\tdb3 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db3.db\")\n\n\tfor _, dbPath := range []string{db1, db2, db3} {\n\t\tif err := CreateDatabaseWithData(t, dbPath, 50); err != nil {\n\t\t\tt.Fatalf(\"create database with data: %v\", err)\n\t\t}\n\t}\n\n\tconfigPath, err := db.CreateDirectoryWatchConfig()\n\tif err != nil {\n\t\tt.Fatalf(\"create config: %v\", err)\n\t}\n\n\tt.Log(\"Starting Litestream...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\tdefer db.StopLitestream()\n\n\ttime.Sleep(3 * time.Second)\n\n\t// Start continuous writes to all 3 databases\n\tctx := context.Background()\n\tt.Log(\"Starting continuous writes to all databases...\")\n\twg1, cancel1, _ := StartContinuousWrites(ctx, t, db1, 20)\n\twg2, cancel2, _ := StartContinuousWrites(ctx, t, db2, 15)\n\twg3, cancel3, _ := StartContinuousWrites(ctx, t, db3, 10)\n\n\tdefer func() {\n\t\tcancel1()\n\t\tcancel2()\n\t\tcancel3()\n\t\twg1.Wait()\n\t\twg2.Wait()\n\t\twg3.Wait()\n\t}()\n\n\t// Wait a bit for writes to start\n\ttime.Sleep(2 * time.Second)\n\n\t// While writes are happening, create 2 new databases\n\tt.Log(\"Creating new databases while writes are ongoing...\")\n\tdb4 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db4.db\")\n\tdb5 := CreateDatabaseInDir(t, db.DirPath, \"\", \"db5.db\")\n\n\t// Start writes on new databases\n\twg4, cancel4, _ := StartContinuousWrites(ctx, t, db4, 10)\n\twg5, cancel5, _ := StartContinuousWrites(ctx, t, db5, 10)\n\n\tdefer func() {\n\t\tcancel4()\n\t\tcancel5()\n\t\twg4.Wait()\n\t\twg5.Wait()\n\t}()\n\n\t// Verify all databases detected\n\tfor i, dbPath := range []string{db1, db2, db3, db4, db5} {\n\t\tif err := WaitForDatabaseInReplica(t, db.ReplicaPath, dbPath, 10*time.Second); err != nil {\n\t\t\tt.Fatalf(\"database %d not detected: %v\", i+1, err)\n\t\t}\n\t}\n\n\t// Let writes continue\n\tt.Log(\"Running writes for 10 seconds...\")\n\ttime.Sleep(10 * time.Second)\n\n\t// Stop all writes\n\tcancel1()\n\tcancel2()\n\tcancel3()\n\tcancel4()\n\tcancel5()\n\twg1.Wait()\n\twg2.Wait()\n\twg3.Wait()\n\twg4.Wait()\n\twg5.Wait()\n\n\t// Wait for final replication\n\ttime.Sleep(3 * time.Second)\n\n\t// Verify all 5 databases are in replica\n\tcount, err := CountDatabasesInReplica(db.ReplicaPath)\n\tif err != nil {\n\t\tt.Fatalf(\"count databases: %v\", err)\n\t}\n\n\tif count != 5 {\n\t\tt.Fatalf(\"expected 5 databases in replica, got %d\", count)\n\t}\n\n\t// Check for errors\n\terrors, err := CheckForCriticalErrors(t, db.TestDB)\n\tif err != nil {\n\t\tt.Fatalf(\"check errors: %v\", err)\n\t}\n\tif len(errors) > 0 {\n\t\tt.Fatalf(\"found critical errors during load test: %v\", errors)\n\t}\n\n\tt.Log(\"✓ Load with writes test passed\")\n}\n" }, { "path": "tests/integration/docker_helpers.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n)\n\nfunc RequireDocker(t *testing.T) {\n\tt.Helper()\n\tif err := exec.Command(\"docker\", \"version\").Run(); err != nil {\n\t\tt.Skip(\"Docker is not available, skipping test\")\n\t}\n}\n\nfunc StartMinioTestContainer(t *testing.T) (string, string) {\n\tt.Helper()\n\n\tname := fmt.Sprintf(\"litestream-minio-%d\", time.Now().UnixNano())\n\texec.Command(\"docker\", \"rm\", \"-f\", name).Run()\n\n\targs := []string{\n\t\t\"run\", \"-d\",\n\t\t\"--name\", name,\n\t\t\"-p\", \"0:9000\",\n\t\t\"-e\", \"MINIO_ROOT_USER=minioadmin\",\n\t\t\"-e\", \"MINIO_ROOT_PASSWORD=minioadmin\",\n\t\t\"-e\", \"MINIO_DOMAIN=s3-accesspoint.127.0.0.1.nip.io\",\n\t\t\"minio/minio\", \"server\", \"/data\",\n\t}\n\tcontainerID := runDockerCommand(t, args...)\n\tportInfo := runDockerCommand(t, \"port\", name, \"9000/tcp\")\n\thostPort := parseDockerPort(t, portInfo)\n\n\ttime.Sleep(5 * time.Second)\n\n\tt.Logf(\"Started MinIO container %s (%s) on port %s\", name, containerID[:12], hostPort)\n\treturn name, fmt.Sprintf(\"http://localhost:%s\", hostPort)\n}\n\nfunc StopMinioTestContainer(t *testing.T, name string) {\n\tt.Helper()\n\tif name == \"\" {\n\t\treturn\n\t}\n\tif os.Getenv(\"SOAK_KEEP_TEMP\") != \"\" {\n\t\tt.Logf(\"SOAK_KEEP_TEMP set, preserving MinIO container: %s\", name)\n\t\treturn\n\t}\n\texec.Command(\"docker\", \"rm\", \"-f\", name).Run()\n}\n\nfunc runDockerCommand(t *testing.T, args ...string) string {\n\tt.Helper()\n\tcmd := exec.Command(\"docker\", args...)\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil {\n\t\tt.Fatalf(\"docker %s failed: %v\\nOutput: %s\", strings.Join(args, \" \"), err, string(output))\n\t}\n\treturn strings.TrimSpace(string(output))\n}\n\nfunc parseDockerPort(t *testing.T, portInfo string) string {\n\tt.Helper()\n\tidx := strings.LastIndex(portInfo, \":\")\n\tif idx == -1 || idx == len(portInfo)-1 {\n\t\tt.Fatalf(\"unexpected docker port output: %s\", portInfo)\n\t}\n\treturn portInfo[idx+1:]\n}\n" }, { "path": "tests/integration/fixtures.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"crypto/rand\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\ntype LoadPattern string\n\nconst (\n\tLoadPatternConstant LoadPattern = \"constant\"\n\tLoadPatternBurst LoadPattern = \"burst\"\n\tLoadPatternRandom LoadPattern = \"random\"\n\tLoadPatternWave LoadPattern = \"wave\"\n)\n\ntype LoadConfig struct {\n\tWriteRate int\n\tDuration time.Duration\n\tPattern LoadPattern\n\tPayloadSize int\n\tReadRatio float64\n\tWorkers int\n}\n\nfunc DefaultLoadConfig() *LoadConfig {\n\treturn &LoadConfig{\n\t\tWriteRate: 100,\n\t\tDuration: 1 * time.Minute,\n\t\tPattern: LoadPatternConstant,\n\t\tPayloadSize: 1024,\n\t\tReadRatio: 0.2,\n\t\tWorkers: 1,\n\t}\n}\n\ntype PopulateConfig struct {\n\tTargetSize string\n\tRowSize int\n\tBatchSize int\n\tTableCount int\n\tIndexRatio float64\n\tPageSize int\n}\n\nfunc DefaultPopulateConfig() *PopulateConfig {\n\treturn &PopulateConfig{\n\t\tTargetSize: \"100MB\",\n\t\tRowSize: 1024,\n\t\tBatchSize: 1000,\n\t\tTableCount: 1,\n\t\tIndexRatio: 0.2,\n\t\tPageSize: 4096,\n\t}\n}\n\nfunc CreateComplexTestSchema(db *sql.DB) error {\n\tschemas := []string{\n\t\t`CREATE TABLE IF NOT EXISTS users (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tusername TEXT NOT NULL UNIQUE,\n\t\t\temail TEXT NOT NULL,\n\t\t\tcreated_at INTEGER NOT NULL\n\t\t)`,\n\t\t`CREATE TABLE IF NOT EXISTS posts (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tuser_id INTEGER NOT NULL,\n\t\t\ttitle TEXT NOT NULL,\n\t\t\tcontent TEXT,\n\t\t\tcreated_at INTEGER NOT NULL,\n\t\t\tFOREIGN KEY (user_id) REFERENCES users(id)\n\t\t)`,\n\t\t`CREATE TABLE IF NOT EXISTS comments (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tpost_id INTEGER NOT NULL,\n\t\t\tuser_id INTEGER NOT NULL,\n\t\t\tcontent TEXT NOT NULL,\n\t\t\tcreated_at INTEGER NOT NULL,\n\t\t\tFOREIGN KEY (post_id) REFERENCES posts(id),\n\t\t\tFOREIGN KEY (user_id) REFERENCES users(id)\n\t\t)`,\n\t\t`CREATE INDEX IF NOT EXISTS idx_posts_user_id ON posts(user_id)`,\n\t\t`CREATE INDEX IF NOT EXISTS idx_posts_created_at ON posts(created_at)`,\n\t\t`CREATE INDEX IF NOT EXISTS idx_comments_post_id ON comments(post_id)`,\n\t\t`CREATE INDEX IF NOT EXISTS idx_comments_created_at ON comments(created_at)`,\n\t}\n\n\tfor _, schema := range schemas {\n\t\tif _, err := db.Exec(schema); err != nil {\n\t\t\treturn fmt.Errorf(\"execute schema: %w\", err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc PopulateComplexTestData(db *sql.DB, userCount, postsPerUser, commentsPerPost int) error {\n\ttx, err := db.Begin()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"begin transaction: %w\", err)\n\t}\n\tdefer tx.Rollback()\n\n\tuserStmt, err := tx.Prepare(\"INSERT INTO users (username, email, created_at) VALUES (?, ?, ?)\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"prepare user statement: %w\", err)\n\t}\n\tdefer userStmt.Close()\n\n\tpostStmt, err := tx.Prepare(\"INSERT INTO posts (user_id, title, content, created_at) VALUES (?, ?, ?, ?)\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"prepare post statement: %w\", err)\n\t}\n\tdefer postStmt.Close()\n\n\tcommentStmt, err := tx.Prepare(\"INSERT INTO comments (post_id, user_id, content, created_at) VALUES (?, ?, ?, ?)\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"prepare comment statement: %w\", err)\n\t}\n\tdefer commentStmt.Close()\n\n\tnow := time.Now().Unix()\n\n\tfor u := 1; u <= userCount; u++ {\n\t\tuserResult, err := userStmt.Exec(\n\t\t\tfmt.Sprintf(\"user%d\", u),\n\t\t\tfmt.Sprintf(\"user%d@test.com\", u),\n\t\t\tnow,\n\t\t)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"insert user: %w\", err)\n\t\t}\n\n\t\tuserID, err := userResult.LastInsertId()\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"get user id: %w\", err)\n\t\t}\n\n\t\tfor p := 1; p <= postsPerUser; p++ {\n\t\t\tpostResult, err := postStmt.Exec(\n\t\t\t\tuserID,\n\t\t\t\tfmt.Sprintf(\"Post %d from user %d\", p, u),\n\t\t\t\tgenerateRandomContent(100),\n\t\t\t\tnow,\n\t\t\t)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"insert post: %w\", err)\n\t\t\t}\n\n\t\t\tpostID, err := postResult.LastInsertId()\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"get post id: %w\", err)\n\t\t\t}\n\n\t\t\tfor c := 1; c <= commentsPerPost; c++ {\n\t\t\t\tcommentUserID := (u + c) % userCount\n\t\t\t\tif commentUserID == 0 {\n\t\t\t\t\tcommentUserID = userCount\n\t\t\t\t}\n\n\t\t\t\t_, err := commentStmt.Exec(\n\t\t\t\t\tpostID,\n\t\t\t\t\tcommentUserID,\n\t\t\t\t\tgenerateRandomContent(50),\n\t\t\t\t\tnow,\n\t\t\t\t)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"insert comment: %w\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn tx.Commit()\n}\n\nfunc generateRandomContent(length int) string {\n\tconst charset = \"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 \"\n\tb := make([]byte, length)\n\trand.Read(b)\n\n\tfor i := range b {\n\t\tb[i] = charset[int(b[i])%len(charset)]\n\t}\n\n\treturn string(b)\n}\n\ntype TestScenario struct {\n\tName string\n\tDescription string\n\tSetup func(*sql.DB) error\n\tValidate func(*sql.DB, *sql.DB) error\n}\n\nfunc LargeWALScenario() *TestScenario {\n\treturn &TestScenario{\n\t\tName: \"Large WAL\",\n\t\tDescription: \"Generate large WAL file to test handling\",\n\t\tSetup: func(db *sql.DB) error {\n\t\t\tif _, err := db.Exec(`\n\t\t\t\tCREATE TABLE test_wal (\n\t\t\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\t\t\tdata BLOB\n\t\t\t\t)\n\t\t\t`); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tdata := make([]byte, 10*1024)\n\t\t\trand.Read(data)\n\n\t\t\tfor i := 0; i < 10000; i++ {\n\t\t\t\tif _, err := db.Exec(\"INSERT INTO test_wal (data) VALUES (?)\", data); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t},\n\t\tValidate: func(source, restored *sql.DB) error {\n\t\t\tvar sourceCount, restoredCount int\n\n\t\t\tif err := source.QueryRow(\"SELECT COUNT(*) FROM test_wal\").Scan(&sourceCount); err != nil {\n\t\t\t\treturn fmt.Errorf(\"query source: %w\", err)\n\t\t\t}\n\n\t\t\tif err := restored.QueryRow(\"SELECT COUNT(*) FROM test_wal\").Scan(&restoredCount); err != nil {\n\t\t\t\treturn fmt.Errorf(\"query restored: %w\", err)\n\t\t\t}\n\n\t\t\tif sourceCount != restoredCount {\n\t\t\t\treturn fmt.Errorf(\"count mismatch: source=%d, restored=%d\", sourceCount, restoredCount)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t},\n\t}\n}\n\nfunc RapidCheckpointsScenario() *TestScenario {\n\treturn &TestScenario{\n\t\tName: \"Rapid Checkpoints\",\n\t\tDescription: \"Test rapid checkpoint operations\",\n\t\tSetup: func(db *sql.DB) error {\n\t\t\tif _, err := db.Exec(`\n\t\t\t\tCREATE TABLE test_checkpoints (\n\t\t\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\t\t\tdata TEXT,\n\t\t\t\t\ttimestamp INTEGER\n\t\t\t\t)\n\t\t\t`); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tfor i := 0; i < 1000; i++ {\n\t\t\t\tif _, err := db.Exec(\n\t\t\t\t\t\"INSERT INTO test_checkpoints (data, timestamp) VALUES (?, ?)\",\n\t\t\t\t\tfmt.Sprintf(\"data %d\", i),\n\t\t\t\t\ttime.Now().Unix(),\n\t\t\t\t); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tif i%100 == 0 {\n\t\t\t\t\tif _, err := db.Exec(\"PRAGMA wal_checkpoint(TRUNCATE)\"); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t},\n\t\tValidate: func(source, restored *sql.DB) error {\n\t\t\tvar sourceCount, restoredCount int\n\n\t\t\tif err := source.QueryRow(\"SELECT COUNT(*) FROM test_checkpoints\").Scan(&sourceCount); err != nil {\n\t\t\t\treturn fmt.Errorf(\"query source: %w\", err)\n\t\t\t}\n\n\t\t\tif err := restored.QueryRow(\"SELECT COUNT(*) FROM test_checkpoints\").Scan(&restoredCount); err != nil {\n\t\t\t\treturn fmt.Errorf(\"query restored: %w\", err)\n\t\t\t}\n\n\t\t\tif sourceCount != restoredCount {\n\t\t\t\treturn fmt.Errorf(\"count mismatch: source=%d, restored=%d\", sourceCount, restoredCount)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t},\n\t}\n}\n" }, { "path": "tests/integration/helpers.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\ntype TestDB struct {\n\tPath string\n\tReplicaPath string\n\tReplicaURL string\n\tReplicaEnv []string\n\tConfigPath string\n\tTempDir string\n\tLitestreamCmd *exec.Cmd\n\tLitestreamPID int\n\tt *testing.T\n}\n\n// getBinaryPath returns the cross-platform path to a binary.\n// On Windows, it adds the .exe extension.\nfunc getBinaryPath(name string) string {\n\tbinPath := filepath.Join(\"..\", \"..\", \"bin\", name)\n\tif runtime.GOOS == \"windows\" {\n\t\tbinPath += \".exe\"\n\t}\n\treturn binPath\n}\n\nfunc streamCommandOutput() bool {\n\tv := strings.ToLower(strings.TrimSpace(os.Getenv(\"SOAK_DEBUG\")))\n\tswitch v {\n\tcase \"\", \"0\", \"false\", \"off\", \"no\":\n\t\treturn false\n\tdefault:\n\t\treturn true\n\t}\n}\n\nfunc configureCmdIO(cmd *exec.Cmd) (bool, *bytes.Buffer, *bytes.Buffer) {\n\tstream := streamCommandOutput()\n\tstdoutBuf := &bytes.Buffer{}\n\tstderrBuf := &bytes.Buffer{}\n\tif stream {\n\t\tcmd.Stdout = io.MultiWriter(os.Stdout, stdoutBuf)\n\t\tcmd.Stderr = io.MultiWriter(os.Stderr, stderrBuf)\n\t} else {\n\t\tcmd.Stdout = stdoutBuf\n\t\tcmd.Stderr = stderrBuf\n\t}\n\treturn stream, stdoutBuf, stderrBuf\n}\n\nfunc combinedOutput(stdoutBuf, stderrBuf *bytes.Buffer) string {\n\tvar sb strings.Builder\n\tif stdoutBuf != nil && stdoutBuf.Len() > 0 {\n\t\tsb.Write(stdoutBuf.Bytes())\n\t}\n\tif stderrBuf != nil && stderrBuf.Len() > 0 {\n\t\tsb.Write(stderrBuf.Bytes())\n\t}\n\treturn strings.TrimSpace(sb.String())\n}\n\nfunc SetupTestDB(t *testing.T, name string) *TestDB {\n\tt.Helper()\n\n\tvar tempDir string\n\tif os.Getenv(\"SOAK_KEEP_TEMP\") != \"\" {\n\t\tdir, err := os.MkdirTemp(\"\", fmt.Sprintf(\"litestream-%s-\", name))\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"create temp dir: %v\", err)\n\t\t}\n\t\ttempDir = dir\n\t\tt.Cleanup(func() {\n\t\t\tt.Logf(\"SOAK_KEEP_TEMP set, preserving test artifacts at: %s\", tempDir)\n\t\t})\n\t} else {\n\t\ttempDir = t.TempDir()\n\t}\n\tdbPath := filepath.Join(tempDir, fmt.Sprintf(\"%s.db\", name))\n\treplicaPath := filepath.Join(tempDir, \"replica\")\n\n\treturn &TestDB{\n\t\tPath: dbPath,\n\t\tReplicaPath: replicaPath,\n\t\tReplicaURL: fmt.Sprintf(\"file://%s\", filepath.ToSlash(replicaPath)),\n\t\tTempDir: tempDir,\n\t\tt: t,\n\t}\n}\n\nfunc (db *TestDB) Create() error {\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open database: %w\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tif _, err := sqlDB.Exec(\"PRAGMA journal_mode=WAL\"); err != nil {\n\t\treturn fmt.Errorf(\"set WAL mode: %w\", err)\n\t}\n\n\treturn nil\n}\n\nfunc (db *TestDB) CreateWithPageSize(pageSize int) error {\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open database: %w\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tif _, err := sqlDB.Exec(fmt.Sprintf(\"PRAGMA page_size = %d\", pageSize)); err != nil {\n\t\treturn fmt.Errorf(\"set page size: %w\", err)\n\t}\n\n\tif _, err := sqlDB.Exec(\"PRAGMA journal_mode=WAL\"); err != nil {\n\t\treturn fmt.Errorf(\"set WAL mode: %w\", err)\n\t}\n\n\treturn nil\n}\n\nfunc (db *TestDB) Populate(targetSize string) error {\n\tcmd := exec.Command(getBinaryPath(\"litestream-test\"), \"populate\",\n\t\t\"-db\", db.Path,\n\t\t\"-target-size\", targetSize,\n\t)\n\n\t_, stdoutBuf, stderrBuf := configureCmdIO(cmd)\n\n\tdb.t.Logf(\"Populating database to %s...\", targetSize)\n\n\tif err := cmd.Run(); err != nil {\n\t\tif output := combinedOutput(stdoutBuf, stderrBuf); output != \"\" {\n\t\t\treturn fmt.Errorf(\"populate failed: %w\\nOutput: %s\", err, output)\n\t\t}\n\t\treturn fmt.Errorf(\"populate failed: %w\", err)\n\t}\n\treturn nil\n}\n\nfunc (db *TestDB) PopulateWithOptions(targetSize string, pageSize int, rowSize int) error {\n\tcmd := exec.Command(getBinaryPath(\"litestream-test\"), \"populate\",\n\t\t\"-db\", db.Path,\n\t\t\"-target-size\", targetSize,\n\t\t\"-page-size\", fmt.Sprintf(\"%d\", pageSize),\n\t\t\"-row-size\", fmt.Sprintf(\"%d\", rowSize),\n\t)\n\n\t_, stdoutBuf, stderrBuf := configureCmdIO(cmd)\n\n\tdb.t.Logf(\"Populating database to %s (page size: %d, row size: %d)...\", targetSize, pageSize, rowSize)\n\n\tif err := cmd.Run(); err != nil {\n\t\tif output := combinedOutput(stdoutBuf, stderrBuf); output != \"\" {\n\t\t\treturn fmt.Errorf(\"populate failed: %w\\nOutput: %s\", err, output)\n\t\t}\n\t\treturn fmt.Errorf(\"populate failed: %w\", err)\n\t}\n\treturn nil\n}\n\nfunc (db *TestDB) GenerateLoad(ctx context.Context, writeRate int, duration time.Duration, pattern string) error {\n\tcmd := exec.CommandContext(ctx, getBinaryPath(\"litestream-test\"), \"load\",\n\t\t\"-db\", db.Path,\n\t\t\"-write-rate\", fmt.Sprintf(\"%d\", writeRate),\n\t\t\"-duration\", duration.String(),\n\t\t\"-pattern\", pattern,\n\t)\n\n\t_, stdoutBuf, stderrBuf := configureCmdIO(cmd)\n\n\tdb.t.Logf(\"Starting load generation: %d writes/sec for %v (%s pattern)\", writeRate, duration, pattern)\n\n\tif err := cmd.Run(); err != nil {\n\t\tif output := combinedOutput(stdoutBuf, stderrBuf); output != \"\" {\n\t\t\treturn fmt.Errorf(\"load generation failed: %w\\nOutput: %s\", err, output)\n\t\t}\n\t\treturn fmt.Errorf(\"load generation failed: %w\", err)\n\t}\n\treturn nil\n}\n\nfunc (db *TestDB) StartLitestream() error {\n\tlogPath := filepath.Join(db.TempDir, \"litestream.log\")\n\tlogFile, err := os.Create(logPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create log file: %w\", err)\n\t}\n\n\treplicaURL := fmt.Sprintf(\"file://%s\", filepath.ToSlash(db.ReplicaPath))\n\tcmd := exec.Command(getBinaryPath(\"litestream\"), \"replicate\",\n\t\tdb.Path,\n\t\treplicaURL,\n\t)\n\tcmd.Stdout = logFile\n\tcmd.Stderr = logFile\n\n\tif err := cmd.Start(); err != nil {\n\t\tlogFile.Close()\n\t\treturn fmt.Errorf(\"start litestream: %w\", err)\n\t}\n\n\tdb.LitestreamCmd = cmd\n\tdb.LitestreamPID = cmd.Process.Pid\n\n\ttime.Sleep(2 * time.Second)\n\n\tif cmd.ProcessState != nil && cmd.ProcessState.Exited() {\n\t\tlogFile.Close()\n\t\treturn fmt.Errorf(\"litestream exited immediately\")\n\t}\n\n\treturn nil\n}\n\nfunc (db *TestDB) StartLitestreamWithConfig(configPath string) error {\n\tlogPath := filepath.Join(db.TempDir, \"litestream.log\")\n\tlogFile, err := os.Create(logPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create log file: %w\", err)\n\t}\n\n\tdb.ConfigPath = configPath\n\tcmd := exec.Command(getBinaryPath(\"litestream\"), \"replicate\",\n\t\t\"-config\", configPath,\n\t)\n\tcmd.Stdout = logFile\n\tcmd.Stderr = logFile\n\n\tif err := cmd.Start(); err != nil {\n\t\tlogFile.Close()\n\t\treturn fmt.Errorf(\"start litestream: %w\", err)\n\t}\n\n\tdb.LitestreamCmd = cmd\n\tdb.LitestreamPID = cmd.Process.Pid\n\n\ttime.Sleep(2 * time.Second)\n\n\treturn nil\n}\n\nfunc (db *TestDB) StopLitestream() error {\n\tif db.LitestreamCmd == nil || db.LitestreamCmd.Process == nil {\n\t\treturn nil\n\t}\n\n\tif err := db.LitestreamCmd.Process.Kill(); err != nil {\n\t\treturn fmt.Errorf(\"kill litestream: %w\", err)\n\t}\n\n\tdb.LitestreamCmd.Wait()\n\ttime.Sleep(1 * time.Second)\n\n\treturn nil\n}\n\nfunc (db *TestDB) Restore(outputPath string) error {\n\treplicaURL := db.ReplicaURL\n\tif replicaURL == \"\" {\n\t\treplicaURL = fmt.Sprintf(\"file://%s\", filepath.ToSlash(db.ReplicaPath))\n\t}\n\tvar cmd *exec.Cmd\n\tif db.ConfigPath != \"\" && (strings.HasPrefix(replicaURL, \"s3://\") || strings.HasPrefix(replicaURL, \"abs://\") || strings.HasPrefix(replicaURL, \"nats://\")) {\n\t\tcmd = exec.Command(getBinaryPath(\"litestream\"), \"restore\",\n\t\t\t\"-config\", db.ConfigPath,\n\t\t\t\"-o\", outputPath,\n\t\t\tdb.Path,\n\t\t)\n\t} else {\n\t\tcmd = exec.Command(getBinaryPath(\"litestream\"), \"restore\",\n\t\t\t\"-o\", outputPath,\n\t\t\treplicaURL,\n\t\t)\n\t}\n\tcmd.Env = append(os.Environ(), db.ReplicaEnv...)\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"restore failed: %w\\nOutput: %s\", err, string(output))\n\t}\n\treturn nil\n}\n\nfunc (db *TestDB) Validate(restoredPath string) error {\n\treplicaURL := db.ReplicaURL\n\tif replicaURL == \"\" {\n\t\treplicaURL = fmt.Sprintf(\"file://%s\", filepath.ToSlash(db.ReplicaPath))\n\t}\n\tcmd := exec.Command(getBinaryPath(\"litestream-test\"), \"validate\",\n\t\t\"-source-db\", db.Path,\n\t\t\"-replica-url\", replicaURL,\n\t\t\"-restored-db\", restoredPath,\n\t\t\"-check-type\", \"full\",\n\t)\n\tcmd.Env = append(os.Environ(), db.ReplicaEnv...)\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"validation failed: %w\\nOutput: %s\", err, string(output))\n\t}\n\treturn nil\n}\n\nfunc (db *TestDB) QuickValidate(restoredPath string) error {\n\treplicaURL := db.ReplicaURL\n\tif replicaURL == \"\" {\n\t\treplicaURL = fmt.Sprintf(\"file://%s\", filepath.ToSlash(db.ReplicaPath))\n\t}\n\tcmd := exec.Command(getBinaryPath(\"litestream-test\"), \"validate\",\n\t\t\"-source-db\", db.Path,\n\t\t\"-replica-url\", replicaURL,\n\t\t\"-restored-db\", restoredPath,\n\t\t\"-check-type\", \"quick\",\n\t)\n\tcmd.Env = append(os.Environ(), db.ReplicaEnv...)\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"validation failed: %w\\nOutput: %s\", err, string(output))\n\t}\n\treturn nil\n}\n\nfunc (db *TestDB) GetRowCount(table string) (int, error) {\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"open database: %w\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tvar count int\n\tquery := fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", table)\n\tif err := sqlDB.QueryRow(query).Scan(&count); err != nil {\n\t\treturn 0, fmt.Errorf(\"query count: %w\", err)\n\t}\n\n\treturn count, nil\n}\n\nfunc (db *TestDB) GetDatabaseSize() (int64, error) {\n\tinfo, err := os.Stat(db.Path)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tsize := info.Size()\n\n\twalPath := db.Path + \"-wal\"\n\tif walInfo, err := os.Stat(walPath); err == nil {\n\t\tsize += walInfo.Size()\n\t}\n\n\treturn size, nil\n}\n\nfunc (db *TestDB) GetReplicaFileCount() (int, error) {\n\tltxPath := filepath.Join(db.ReplicaPath, \"ltx\", \"0\")\n\tfiles, err := filepath.Glob(filepath.Join(ltxPath, \"*.ltx\"))\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\treturn len(files), nil\n}\n\nfunc (db *TestDB) WaitForReplicaFiles(minFiles int, timeout time.Duration) (int, error) {\n\tdeadline := time.Now().Add(timeout)\n\tfor time.Now().Before(deadline) {\n\t\tcount, err := db.GetReplicaFileCount()\n\t\tif err != nil {\n\t\t\treturn 0, err\n\t\t}\n\t\tif count >= minFiles {\n\t\t\treturn count, nil\n\t\t}\n\t\ttime.Sleep(1 * time.Second)\n\t}\n\tcount, _ := db.GetReplicaFileCount()\n\treturn count, fmt.Errorf(\"timeout waiting for %d replica files, got %d\", minFiles, count)\n}\n\nfunc (db *TestDB) GetLitestreamLog() (string, error) {\n\tlogPath := filepath.Join(db.TempDir, \"litestream.log\")\n\tcontent, err := os.ReadFile(logPath)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn string(content), nil\n}\n\nfunc (db *TestDB) CheckForErrors() ([]string, error) {\n\tlog, err := db.GetLitestreamLog()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar errors []string\n\tlines := strings.Split(log, \"\\n\")\n\tfor _, line := range lines {\n\t\tif strings.Contains(strings.ToUpper(line), \"ERROR\") {\n\t\t\terrors = append(errors, line)\n\t\t}\n\t}\n\n\treturn errors, nil\n}\n\nfunc (db *TestDB) Cleanup() {\n\tdb.StopLitestream()\n}\n\n// WaitForSnapshots waits for snapshots & WAL segments to appear on file replicas.\nfunc (db *TestDB) WaitForSnapshots(timeout time.Duration) error {\n\tif !strings.HasPrefix(db.ReplicaURL, \"file://\") {\n\t\treturn nil\n\t}\n\n\tsnapshotDir := filepath.Join(db.ReplicaPath, \"ltx\", fmt.Sprintf(\"%d\", litestream.SnapshotLevel))\n\twalDir := filepath.Join(db.ReplicaPath, \"ltx\", \"0\")\n\n\tdeadline := time.Now().Add(timeout)\n\tfor {\n\t\tsnapshotCount := countLTXFiles(snapshotDir)\n\t\twalCount := countLTXFiles(walDir)\n\n\t\tif snapshotCount > 0 && walCount > 0 {\n\t\t\treturn nil\n\t\t}\n\n\t\tif time.Now().After(deadline) {\n\t\t\treturn fmt.Errorf(\"timeout waiting for replica data: snapshots=%d wal=%d\", snapshotCount, walCount)\n\t\t}\n\n\t\ttime.Sleep(500 * time.Millisecond)\n\t}\n}\n\nfunc countLTXFiles(dir string) int {\n\tmatches, err := filepath.Glob(filepath.Join(dir, \"*.ltx\"))\n\tif err != nil {\n\t\treturn 0\n\t}\n\treturn len(matches)\n}\n\nfunc GetTestDuration(t *testing.T, defaultDuration time.Duration) time.Duration {\n\tt.Helper()\n\n\tif testing.Short() {\n\t\treturn defaultDuration / 10\n\t}\n\n\treturn defaultDuration\n}\n\nfunc RequireBinaries(t *testing.T) {\n\tt.Helper()\n\n\tlitestreamBin := getBinaryPath(\"litestream\")\n\tif _, err := os.Stat(litestreamBin); err != nil {\n\t\tt.Skip(\"litestream binary not found, run: go build -o bin/litestream ./cmd/litestream\")\n\t}\n\n\tlitestreamTestBin := getBinaryPath(\"litestream-test\")\n\tif _, err := os.Stat(litestreamTestBin); err != nil {\n\t\tt.Skip(\"litestream-test binary not found, run: go build -o bin/litestream-test ./cmd/litestream-test\")\n\t}\n}\n\n// WriteS3AccessPointConfig writes a minimal configuration file for S3 access point tests.\nfunc WriteS3AccessPointConfig(t *testing.T, dbPath, replicaURL, endpoint string, forcePathStyle bool, accessKey, secretKey string) string {\n\tt.Helper()\n\n\tdir := filepath.Dir(dbPath)\n\tconfigPath := filepath.Join(dir, \"litestream-access-point.yml\")\n\n\tconfig := fmt.Sprintf(`access-key-id: %s\nsecret-access-key: %s\n\ndbs:\n - path: %s\n replicas:\n - url: %s\n endpoint: %s\n region: us-east-1\n force-path-style: %t\n skip-verify: true\n sync-interval: 1s\n`, accessKey, secretKey, filepath.ToSlash(dbPath), replicaURL, endpoint, forcePathStyle)\n\n\tif err := os.WriteFile(configPath, []byte(config), 0600); err != nil {\n\t\tt.Fatalf(\"failed to write config: %v\", err)\n\t}\n\n\treturn configPath\n}\n\nfunc CreateTestTable(t *testing.T, dbPath string) error {\n\tt.Helper()\n\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer sqlDB.Close()\n\n\t_, err = sqlDB.Exec(`\n\t\tCREATE TABLE IF NOT EXISTS test_data (\n\t\t\tid INTEGER PRIMARY KEY AUTOINCREMENT,\n\t\t\tdata TEXT,\n\t\t\tcreated_at INTEGER\n\t\t)\n\t`)\n\treturn err\n}\n\nfunc InsertTestData(t *testing.T, dbPath string, count int) error {\n\tt.Helper()\n\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer sqlDB.Close()\n\n\ttx, err := sqlDB.Begin()\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer tx.Rollback()\n\n\tstmt, err := tx.Prepare(\"INSERT INTO test_data (data, created_at) VALUES (?, ?)\")\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer stmt.Close()\n\n\tfor i := 0; i < count; i++ {\n\t\tif _, err := stmt.Exec(fmt.Sprintf(\"test data %d\", i), time.Now().Unix()); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn tx.Commit()\n}\n\n// IntegrityCheck runs PRAGMA integrity_check on the database.\nfunc (db *TestDB) IntegrityCheck() error {\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer sqlDB.Close()\n\n\tvar result string\n\tif err := sqlDB.QueryRow(\"PRAGMA integrity_check\").Scan(&result); err != nil {\n\t\treturn err\n\t}\n\tif result != \"ok\" {\n\t\treturn fmt.Errorf(\"integrity check failed: %s\", result)\n\t}\n\treturn nil\n}\n\n// PrintTestSummary prints a summary of the test results\nfunc (db *TestDB) PrintTestSummary(t *testing.T, testName string, startTime time.Time) {\n\tt.Helper()\n\n\tduration := time.Since(startTime)\n\tdbSize, _ := db.GetDatabaseSize()\n\tfileCount, _ := db.GetReplicaFileCount()\n\terrors, _ := db.CheckForErrors()\n\n\tt.Log(\"\\n\" + strings.Repeat(\"=\", 80))\n\tt.Logf(\"TEST SUMMARY: %s\", testName)\n\tt.Log(strings.Repeat(\"=\", 80))\n\tt.Logf(\"Duration: %v\", duration.Round(time.Second))\n\tt.Logf(\"Database Size: %.2f MB\", float64(dbSize)/(1024*1024))\n\tt.Logf(\"Replica Files: %d LTX files\", fileCount)\n\tt.Logf(\"Litestream Errors: %d\", len(errors))\n\tt.Log(strings.Repeat(\"=\", 80))\n}\n" }, { "path": "tests/integration/minio_soak_test.go", "content": "//go:build integration && soak && docker\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\n// TestMinIOSoak runs a soak test against local MinIO S3-compatible server using Docker.\n//\n// Default duration: 2 hours\n// Can be shortened with: go test -test.short (runs for 30 minutes)\n//\n// Requirements:\n// - Docker must be running\n// - docker command must be in PATH\n//\n// This test validates:\n// - S3-compatible replication to MinIO\n// - Docker container lifecycle management\n// - Heavy sustained load (500 writes/sec)\n// - Restoration from S3-compatible storage\nfunc TestMinIOSoak(t *testing.T) {\n\tRequireBinaries(t)\n\tRequireDocker(t)\n\n\t// Determine test duration\n\tduration := GetTestDuration(t, 2*time.Hour)\n\tshortMode := testing.Short()\n\tif shortMode {\n\t\tduration = 2 * time.Minute\n\t}\n\n\ttargetSize := \"50MB\"\n\twriteRate := 500\n\tif shortMode {\n\t\ttargetSize = \"5MB\"\n\t\twriteRate = 100\n\t}\n\n\tt.Logf(\"================================================\")\n\tt.Logf(\"Litestream MinIO S3 Soak Test\")\n\tt.Logf(\"================================================\")\n\tt.Logf(\"Duration: %v\", duration)\n\tt.Logf(\"Start time: %s\", time.Now().Format(time.RFC3339))\n\tt.Log(\"\")\n\n\tstartTime := time.Now()\n\n\t// Start MinIO container\n\tt.Log(\"Starting MinIO container...\")\n\tcontainerID, endpoint, dataVolume := StartMinIOContainer(t)\n\tdefer StopMinIOContainer(t, containerID, dataVolume)\n\tt.Logf(\"✓ MinIO running at: %s\", endpoint)\n\tt.Log(\"\")\n\n\t// Create MinIO bucket\n\tbucket := \"litestream-test\"\n\tCreateMinIOBucket(t, containerID, bucket)\n\tt.Log(\"\")\n\n\t// Setup test database\n\tdb := SetupTestDB(t, \"minio-soak\")\n\tdefer db.Cleanup()\n\n\t// Create database\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\t// Populate with initial data\n\tt.Logf(\"Populating database (%s initial data)...\", targetSize)\n\tif err := db.Populate(targetSize); err != nil {\n\t\tt.Fatalf(\"Failed to populate database: %v\", err)\n\t}\n\tt.Log(\"✓ Database populated\")\n\tt.Log(\"\")\n\n\t// Create S3 configuration for MinIO\n\ts3Path := fmt.Sprintf(\"litestream-test-%d\", time.Now().Unix())\n\ts3URL := fmt.Sprintf(\"s3://%s/%s\", bucket, s3Path)\n\tdb.ReplicaURL = s3URL\n\tt.Log(\"Creating Litestream configuration for MinIO S3...\")\n\ts3Config := &S3Config{\n\t\tEndpoint: endpoint,\n\t\tAccessKey: \"minioadmin\",\n\t\tSecretKey: \"minioadmin\",\n\t\tRegion: \"us-east-1\",\n\t\tForcePathStyle: true,\n\t\tSkipVerify: true,\n\t}\n\tconfigPath := CreateSoakConfig(db.Path, s3URL, s3Config, shortMode)\n\tdb.ConfigPath = configPath\n\tt.Logf(\"✓ Configuration created: %s\", configPath)\n\tt.Logf(\" S3 URL: %s\", s3URL)\n\tt.Log(\"\")\n\n\t// Start Litestream\n\tt.Log(\"Starting Litestream with MinIO backend...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\tt.Logf(\"✓ Litestream running (PID: %d)\", db.LitestreamPID)\n\tt.Log(\"\")\n\n\t// Start load generator\n\tt.Log(\"Starting load generator (heavy sustained load)...\")\n\tt.Logf(\" Write rate: %d writes/second\", writeRate)\n\tt.Logf(\" Pattern: wave (simulates varying load)\")\n\tt.Logf(\" Payload size: 4KB\")\n\tt.Logf(\" Workers: 8\")\n\tt.Log(\"\")\n\n\tctx, cancel := context.WithTimeout(context.Background(), duration)\n\tdefer cancel()\n\n\t// Setup signal handler for graceful interruption\n\ttestInfo := &TestInfo{\n\t\tStartTime: startTime,\n\t\tDuration: duration,\n\t\tDB: db,\n\t\tcancel: cancel,\n\t}\n\tsetupSignalHandler(t, cancel, testInfo)\n\n\t// Run load generation in background\n\tloadDone := make(chan error, 1)\n\tgo func() {\n\t\tloadDone <- db.GenerateLoad(ctx, writeRate, duration, \"wave\")\n\t}()\n\n\t// Monitor every 60 seconds with MinIO-specific metrics\n\tt.Log(\"Running MinIO S3 test...\")\n\tt.Log(\"Monitor will report every 60 seconds\")\n\tt.Log(\"Press Ctrl+C twice within 5 seconds to stop early\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\n\trefreshStats := func() {\n\t\ttestInfo.RowCount, _ = db.GetRowCount(\"load_test\")\n\t\tif testInfo.RowCount == 0 {\n\t\t\ttestInfo.RowCount, _ = db.GetRowCount(\"test_table_0\")\n\t\t}\n\t\tif testInfo.RowCount == 0 {\n\t\t\ttestInfo.RowCount, _ = db.GetRowCount(\"test_data\")\n\t\t}\n\t\ttestInfo.FileCount = CountMinIOObjects(t, containerID, bucket)\n\t}\n\n\tlogMetrics := func() {\n\t\tlogMinIOMetrics(t, db, containerID, bucket)\n\t\tif db.LitestreamCmd != nil && db.LitestreamCmd.ProcessState != nil {\n\t\t\tt.Error(\"✗ Litestream stopped unexpectedly!\")\n\t\t\tif testInfo.cancel != nil {\n\t\t\t\ttestInfo.cancel()\n\t\t\t}\n\t\t}\n\t}\n\n\tMonitorSoakTest(t, db, ctx, testInfo, refreshStats, logMetrics)\n\n\t// Wait for load generation to complete\n\tif err := <-loadDone; err != nil {\n\t\tt.Logf(\"Load generation completed: %v\", err)\n\t}\n\n\tif err := db.WaitForSnapshots(30 * time.Second); err != nil {\n\t\tt.Fatalf(\"Failed waiting for snapshot: %v\", err)\n\t}\n\n\tt.Log(\"\")\n\tt.Log(\"================================================\")\n\tt.Log(\"Final Test Results\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\n\t// Stop Litestream\n\tt.Log(\"Stopping Litestream...\")\n\tif err := db.StopLitestream(); err != nil {\n\t\tt.Logf(\"Warning: Failed to stop Litestream cleanly: %v\", err)\n\t}\n\n\t// Final statistics\n\tt.Log(\"Database Statistics:\")\n\tif dbSize, err := db.GetDatabaseSize(); err == nil {\n\t\tt.Logf(\" Final size: %.2f MB\", float64(dbSize)/(1024*1024))\n\t}\n\n\t// Count rows\n\tvar rowCount int\n\tvar err error\n\tif rowCount, err = db.GetRowCount(\"load_test\"); err != nil {\n\t\tif rowCount, err = db.GetRowCount(\"test_table_0\"); err != nil {\n\t\t\tif rowCount, err = db.GetRowCount(\"test_data\"); err != nil {\n\t\t\t\tt.Logf(\" Warning: Could not get row count: %v\", err)\n\t\t\t}\n\t\t}\n\t}\n\tif err == nil {\n\t\tt.Logf(\" Total rows: %d\", rowCount)\n\t}\n\tt.Log(\"\")\n\n\t// MinIO statistics\n\tt.Log(\"MinIO S3 Statistics:\")\n\tfinalObjects := CountMinIOObjects(t, containerID, bucket)\n\tt.Logf(\" Total objects in MinIO: %d\", finalObjects)\n\tt.Log(\"\")\n\n\t// Check for errors\n\terrors, _ := db.CheckForErrors()\n\tt.Logf(\" Critical errors: %d\", len(errors))\n\tt.Log(\"\")\n\n\t// Test restoration from MinIO\n\tt.Log(\"Testing restoration from MinIO S3...\")\n\trestoredPath := filepath.Join(db.TempDir, \"restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restoration from MinIO failed: %v\", err)\n\t}\n\tt.Log(\"✓ Restoration successful!\")\n\n\t// Compare row counts\n\tvar restoredCount int\n\tif restoredCount, err = getRowCountFromPath(restoredPath, \"load_test\"); err != nil {\n\t\tif restoredCount, err = getRowCountFromPath(restoredPath, \"test_table_0\"); err != nil {\n\t\t\tif restoredCount, err = getRowCountFromPath(restoredPath, \"test_data\"); err != nil {\n\t\t\t\tt.Logf(\" Warning: Could not get restored row count: %v\", err)\n\t\t\t}\n\t\t}\n\t}\n\tif err == nil && rowCount > 0 {\n\t\tif rowCount == restoredCount {\n\t\t\tt.Logf(\"✓ Row counts match! (%d rows)\", restoredCount)\n\t\t} else {\n\t\t\tt.Logf(\"⚠ Row count mismatch! Original: %d, Restored: %d\", rowCount, restoredCount)\n\t\t}\n\t}\n\n\t// Validate integrity\n\tt.Log(\"\")\n\tt.Log(\"Validating restored database integrity...\")\n\trestoredDB := &TestDB{Path: restoredPath, t: t}\n\tif err := restoredDB.IntegrityCheck(); err != nil {\n\t\tt.Fatalf(\"Integrity check failed: %v\", err)\n\t}\n\tt.Log(\"✓ Integrity check passed!\")\n\n\t// Analyze test results\n\tanalysis := AnalyzeSoakTest(t, db, duration)\n\tPrintSoakTestAnalysis(t, analysis)\n\n\t// Test Summary\n\tt.Log(\"================================================\")\n\tt.Log(\"Test Summary\")\n\tt.Log(\"================================================\")\n\n\ttestPassed := true\n\tissues := []string{}\n\n\tif criticalErrors > 0 {\n\t\ttestPassed = false\n\t\tissues = append(issues, fmt.Sprintf(\"Critical errors detected: %d\", criticalErrors))\n\t}\n\n\tif finalObjects == 0 {\n\t\ttestPassed = false\n\t\tissues = append(issues, \"No objects stored in MinIO\")\n\t}\n\n\tif testPassed {\n\t\tt.Log(\"✓ TEST PASSED!\")\n\t\tt.Log(\"\")\n\t\tt.Logf(\"Successfully replicated to MinIO (%d objects)\", finalObjects)\n\t\tt.Log(\"The configuration is ready for production use.\")\n\t} else {\n\t\tt.Log(\"⚠ TEST COMPLETED WITH ISSUES:\")\n\t\tfor _, issue := range issues {\n\t\t\tt.Logf(\" - %s\", issue)\n\t\t}\n\t\tt.Log(\"\")\n\t\tt.Log(\"Review the logs for details:\")\n\t\tlogPath, _ := db.GetLitestreamLog()\n\t\tt.Logf(\" %s\", logPath)\n\t\tt.Fail()\n\t}\n\n\tt.Log(\"\")\n\tt.Logf(\"Test duration: %v\", time.Since(startTime).Round(time.Second))\n\tt.Logf(\"Results available in: %s\", db.TempDir)\n\tt.Log(\"================================================\")\n}\n\n// logMinIOMetrics logs MinIO-specific metrics\nfunc logMinIOMetrics(t *testing.T, db *TestDB, containerID, bucket string) {\n\tt.Helper()\n\n\t// Basic database metrics\n\tLogSoakMetrics(t, db, \"minio\")\n\n\t// MinIO-specific metrics\n\tt.Log(\"\")\n\tt.Log(\" MinIO S3 Statistics:\")\n\n\tobjectCount := CountMinIOObjects(t, containerID, bucket)\n\tt.Logf(\" Total objects: %d\", objectCount)\n\n\t// Count LTX files specifically\n\tltxCount := countMinIOLTXFiles(t, containerID, bucket)\n\tt.Logf(\" LTX segments: %d\", ltxCount)\n}\n\n// countMinIOLTXFiles counts LTX files in MinIO bucket\nfunc countMinIOLTXFiles(t *testing.T, containerID, bucket string) int {\n\tt.Helper()\n\n\tcmd := exec.Command(\"docker\", \"run\", \"--rm\",\n\t\t\"--link\", containerID+\":minio\",\n\t\t\"-e\", \"MC_HOST_minio=http://minioadmin:minioadmin@minio:9000\",\n\t\t\"minio/mc\", \"ls\", \"minio/\"+bucket+\"/\", \"--recursive\")\n\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil {\n\t\treturn 0\n\t}\n\n\tlines := strings.Split(strings.TrimSpace(string(output)), \"\\n\")\n\tltxCount := 0\n\tfor _, line := range lines {\n\t\tif strings.Contains(line, \".ltx\") {\n\t\t\tltxCount++\n\t\t}\n\t}\n\n\treturn ltxCount\n}\n\n// getRowCountFromPath gets row count from a database file path\nfunc getRowCountFromPath(dbPath, table string) (int, error) {\n\tdb, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tdefer db.Close()\n\n\tvar count int\n\tquery := fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", table)\n\tif err := db.QueryRow(query).Scan(&count); err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn count, nil\n}\n" }, { "path": "tests/integration/overnight_s3_soak_test.go", "content": "//go:build integration && soak && aws\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\n// TestOvernightS3Soak runs an 8-hour overnight soak test against real AWS S3.\n//\n// Default duration: 8 hours\n// Can be shortened with: go test -test.short (runs for 1 hour)\n//\n// Requirements:\n// - AWS_ACCESS_KEY_ID environment variable\n// - AWS_SECRET_ACCESS_KEY environment variable\n// - S3_BUCKET environment variable\n// - AWS_REGION environment variable (optional, defaults to us-east-1)\n// - AWS CLI must be installed\n//\n// This test validates:\n// - Long-term S3 replication stability\n// - Network resilience over 8 hours\n// - Real S3 API performance\n// - Restoration from cloud storage\nfunc TestOvernightS3Soak(t *testing.T) {\n\tRequireBinaries(t)\n\n\t// Check AWS credentials and get configuration\n\tbucket, region := CheckAWSCredentials(t)\n\n\t// Determine test duration\n\tvar duration time.Duration\n\tif testing.Short() {\n\t\tduration = 10 * time.Minute\n\t} else {\n\t\tduration = 8 * time.Hour\n\t}\n\n\tshortMode := testing.Short()\n\n\tt.Logf(\"================================================\")\n\tt.Logf(\"Litestream Overnight S3 Soak Test\")\n\tt.Logf(\"================================================\")\n\tt.Logf(\"Duration: %v\", duration)\n\tt.Logf(\"S3 Bucket: %s\", bucket)\n\tt.Logf(\"AWS Region: %s\", region)\n\tt.Logf(\"Start time: %s\", time.Now().Format(time.RFC3339))\n\tt.Log(\"\")\n\n\tstartTime := time.Now()\n\n\t// Test S3 connectivity\n\tt.Log(\"Testing S3 connectivity...\")\n\tTestS3Connectivity(t, bucket)\n\tt.Log(\"\")\n\n\t// Setup test database\n\tdb := SetupTestDB(t, \"overnight-s3-soak\")\n\tdefer db.Cleanup()\n\n\t// Create database\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\t// Create S3 configuration\n\ts3Path := fmt.Sprintf(\"litestream-overnight-%d\", time.Now().Unix())\n\ts3URL := fmt.Sprintf(\"s3://%s/%s\", bucket, s3Path)\n\tdb.ReplicaURL = s3URL\n\tt.Log(\"Creating Litestream configuration for S3...\")\n\ts3Config := &S3Config{\n\t\tRegion: region,\n\t}\n\tconfigPath := CreateSoakConfig(db.Path, s3URL, s3Config, shortMode)\n\tdb.ConfigPath = configPath\n\tt.Logf(\"✓ Configuration created: %s\", configPath)\n\tt.Logf(\" S3 URL: %s\", s3URL)\n\tt.Log(\"\")\n\n\t// Start Litestream initially (before population)\n\tt.Log(\"Starting Litestream...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\tt.Logf(\"✓ Litestream started (PID: %d)\", db.LitestreamPID)\n\tt.Log(\"\")\n\n\t// Stop Litestream to populate database\n\tt.Log(\"Stopping Litestream temporarily for initial population...\")\n\tif err := db.StopLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to stop Litestream: %v\", err)\n\t}\n\n\t// Populate with 100MB of initial data\n\tt.Log(\"Populating database (100MB initial data)...\")\n\tif err := db.Populate(\"100MB\"); err != nil {\n\t\tt.Fatalf(\"Failed to populate database: %v\", err)\n\t}\n\tt.Log(\"✓ Database populated\")\n\tt.Log(\"\")\n\n\t// Restart Litestream after population\n\tt.Log(\"Restarting Litestream after population...\")\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"Failed to restart Litestream: %v\", err)\n\t}\n\tt.Logf(\"✓ Litestream restarted (PID: %d)\", db.LitestreamPID)\n\tt.Log(\"\")\n\n\t// Start load generator for overnight test\n\tt.Log(\"Starting load generator for overnight S3 test...\")\n\tt.Log(\"Configuration:\")\n\tt.Logf(\" Duration: %v\", duration)\n\tt.Logf(\" Write rate: 100 writes/second (higher for S3 testing)\")\n\tt.Logf(\" Pattern: wave (simulates varying load)\")\n\tt.Logf(\" Workers: 8\")\n\tt.Log(\"\")\n\n\tctx, cancel := context.WithTimeout(context.Background(), duration)\n\tdefer cancel()\n\n\t// Setup signal handler for graceful interruption\n\ttestInfo := &TestInfo{\n\t\tStartTime: startTime,\n\t\tDuration: duration,\n\t\tDB: db,\n\t\tcancel: cancel,\n\t}\n\tsetupSignalHandler(t, cancel, testInfo)\n\n\t// Run load generation in background\n\tloadDone := make(chan error, 1)\n\tgo func() {\n\t\tloadDone <- db.GenerateLoad(ctx, 100, duration, \"wave\")\n\t}()\n\n\t// Monitor every 60 seconds with S3-specific metrics\n\tt.Log(\"Overnight S3 test is running!\")\n\tt.Log(\"Monitor will report every 60 seconds\")\n\tt.Log(\"Press Ctrl+C twice within 5 seconds to stop early\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\tt.Logf(\"The test will run for %v. Monitor progress below.\", duration)\n\tt.Log(\"\")\n\n\trefreshStats := func() {\n\t\ttestInfo.RowCount, _ = db.GetRowCount(\"load_test\")\n\t\tif testInfo.RowCount == 0 {\n\t\t\ttestInfo.RowCount, _ = db.GetRowCount(\"test_table_0\")\n\t\t}\n\t\tif testInfo.RowCount == 0 {\n\t\t\ttestInfo.RowCount, _ = db.GetRowCount(\"test_data\")\n\t\t}\n\t\ttestInfo.FileCount = CountS3Objects(t, s3URL)\n\t}\n\n\tlogMetrics := func() {\n\t\tlogS3Metrics(t, db, s3URL)\n\t\tif db.LitestreamCmd != nil && db.LitestreamCmd.ProcessState != nil {\n\t\t\tt.Error(\"✗ Litestream stopped unexpectedly!\")\n\t\t\tif testInfo.cancel != nil {\n\t\t\t\ttestInfo.cancel()\n\t\t\t}\n\t\t}\n\t}\n\n\tMonitorSoakTest(t, db, ctx, testInfo, refreshStats, logMetrics)\n\n\t// Wait for load generation to complete\n\tif err := <-loadDone; err != nil {\n\t\tt.Logf(\"Load generation completed: %v\", err)\n\t}\n\n\tt.Log(\"\")\n\tt.Log(\"Load generation completed.\")\n\n\t// Final statistics\n\tt.Log(\"\")\n\tt.Log(\"================================================\")\n\tt.Log(\"Final Statistics\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\n\t// Stop Litestream\n\tt.Log(\"Stopping Litestream...\")\n\tif err := db.StopLitestream(); err != nil {\n\t\tt.Logf(\"Warning: Failed to stop Litestream cleanly: %v\", err)\n\t}\n\n\t// Database statistics\n\tt.Log(\"Database Statistics:\")\n\tif dbSize, err := db.GetDatabaseSize(); err == nil {\n\t\tt.Logf(\" Final size: %.2f MB\", float64(dbSize)/(1024*1024))\n\t}\n\n\t// Count rows\n\tvar rowCount int\n\tvar err error\n\tif rowCount, err = db.GetRowCount(\"load_test\"); err != nil {\n\t\tif rowCount, err = db.GetRowCount(\"test_table_0\"); err != nil {\n\t\t\tif rowCount, err = db.GetRowCount(\"test_data\"); err != nil {\n\t\t\t\tt.Logf(\" Warning: Could not get row count: %v\", err)\n\t\t\t}\n\t\t}\n\t}\n\tif err == nil {\n\t\tt.Logf(\" Total rows: %d\", rowCount)\n\t}\n\tt.Log(\"\")\n\n\t// S3 statistics\n\tt.Log(\"S3 Statistics:\")\n\tfinalObjects := CountS3Objects(t, s3URL)\n\tt.Logf(\" Total objects: %d\", finalObjects)\n\n\tif s3Size := GetS3StorageSize(t, s3URL); s3Size > 0 {\n\t\tt.Logf(\" Total S3 storage: %.2f MB\", float64(s3Size)/(1024*1024))\n\t}\n\tt.Log(\"\")\n\n\t// Check for errors\n\terrors, _ := db.CheckForErrors()\n\tt.Logf(\" Critical errors: %d\", len(errors))\n\tt.Log(\"\")\n\n\t// Test restoration from S3\n\tt.Log(\"Testing restoration from S3...\")\n\trestoredPath := filepath.Join(db.TempDir, \"restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restoration from S3 failed: %v\", err)\n\t}\n\tt.Log(\"✓ Restoration successful!\")\n\n\t// Compare row counts\n\tvar restoredCount int\n\tif restoredCount, err = getRowCountFromPath(restoredPath, \"load_test\"); err != nil {\n\t\tif restoredCount, err = getRowCountFromPath(restoredPath, \"test_table_0\"); err != nil {\n\t\t\tif restoredCount, err = getRowCountFromPath(restoredPath, \"test_data\"); err != nil {\n\t\t\t\tt.Logf(\" Warning: Could not get restored row count: %v\", err)\n\t\t\t}\n\t\t}\n\t}\n\tif err == nil && rowCount > 0 {\n\t\tif rowCount == restoredCount {\n\t\t\tt.Logf(\"✓ Row counts match! (%d rows)\", restoredCount)\n\t\t} else {\n\t\t\tt.Logf(\"⚠ Row count mismatch! Original: %d, Restored: %d\", rowCount, restoredCount)\n\t\t}\n\t}\n\n\t// Validate\n\tt.Log(\"\")\n\tt.Log(\"Validating restored database...\")\n\tif err := db.Validate(restoredPath); err != nil {\n\t\tt.Fatalf(\"Validation failed: %v\", err)\n\t}\n\tt.Log(\"✓ Validation passed!\")\n\n\t// Analyze test results\n\tanalysis := AnalyzeSoakTest(t, db, duration)\n\tPrintSoakTestAnalysis(t, analysis)\n\n\t// Test Summary\n\tt.Log(\"================================================\")\n\tt.Log(\"Test Summary\")\n\tt.Log(\"================================================\")\n\n\ttestPassed := true\n\tissues := []string{}\n\n\tif criticalErrors > 0 {\n\t\ttestPassed = false\n\t\tissues = append(issues, fmt.Sprintf(\"Critical errors detected: %d\", criticalErrors))\n\t}\n\n\tif finalObjects == 0 {\n\t\ttestPassed = false\n\t\tissues = append(issues, \"No objects stored in S3\")\n\t}\n\n\tif testPassed {\n\t\tt.Log(\"✓ TEST PASSED!\")\n\t\tt.Log(\"\")\n\t\tt.Logf(\"Successfully replicated to AWS S3 (%d objects)\", finalObjects)\n\t\tt.Log(\"The configuration is ready for production use.\")\n\t} else {\n\t\tt.Log(\"⚠ TEST COMPLETED WITH ISSUES:\")\n\t\tfor _, issue := range issues {\n\t\t\tt.Logf(\" - %s\", issue)\n\t\t}\n\t\tt.Log(\"\")\n\t\tt.Log(\"Review the logs for details:\")\n\t\tlogPath, _ := db.GetLitestreamLog()\n\t\tt.Logf(\" %s\", logPath)\n\t\tt.Fail()\n\t}\n\n\tt.Log(\"\")\n\tt.Logf(\"Test duration: %v\", time.Since(startTime).Round(time.Second))\n\tt.Logf(\"Results available in: %s\", db.TempDir)\n\tt.Logf(\"S3 replica data in: %s\", s3URL)\n\tt.Log(\"================================================\")\n}\n\n// logS3Metrics logs S3-specific metrics\nfunc logS3Metrics(t *testing.T, db *TestDB, s3URL string) {\n\tt.Helper()\n\n\t// Basic database metrics\n\tLogSoakMetrics(t, db, \"overnight-s3\")\n\n\t// S3-specific metrics\n\tt.Log(\"\")\n\tt.Log(\" S3 Statistics:\")\n\n\tobjectCount := CountS3Objects(t, s3URL)\n\tt.Logf(\" Total objects: %d\", objectCount)\n\n\tif s3Size := GetS3StorageSize(t, s3URL); s3Size > 0 {\n\t\tt.Logf(\" Total storage: %.2f MB\", float64(s3Size)/(1024*1024))\n\t}\n}\n\n// getRowCountFromPath gets row count from a database file path\nfunc getRowCountFromPath(dbPath, table string) (int, error) {\n\tdb, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tdefer db.Close()\n\n\tvar count int\n\tquery := fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", table)\n\tif err := db.QueryRow(query).Scan(&count); err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn count, nil\n}\n" }, { "path": "tests/integration/overnight_test.go", "content": "//go:build integration && long\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\nfunc TestOvernightFile(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping long integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tstartTime := time.Now()\n\tduration := GetTestDuration(t, 8*time.Hour)\n\tt.Logf(\"Testing: Overnight file-based replication (duration: %v)\", duration)\n\tt.Log(\"Default: 8 hours, configurable via test duration\")\n\n\tdb := SetupTestDB(t, \"overnight-file\")\n\tdefer db.Cleanup()\n\tdefer db.PrintTestSummary(t, \"Overnight File Replication\", startTime)\n\n\tt.Log(\"[1] Creating and populating database...\")\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\tif err := db.Populate(\"100MB\"); err != nil {\n\t\tt.Fatalf(\"Failed to populate database: %v\", err)\n\t}\n\n\tt.Log(\"✓ Database populated to 100MB\")\n\n\tt.Log(\"[2] Starting Litestream...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(10 * time.Second)\n\n\tt.Log(\"[3] Generating sustained load...\")\n\tctx, cancel := context.WithTimeout(context.Background(), duration)\n\tdefer cancel()\n\n\tconfig := DefaultLoadConfig()\n\tconfig.WriteRate = 50\n\tconfig.Duration = duration\n\tconfig.Pattern = LoadPatternWave\n\tconfig.PayloadSize = 2 * 1024\n\tconfig.Workers = 4\n\n\tticker := time.NewTicker(1 * time.Minute)\n\tdefer ticker.Stop()\n\n\tgo func() {\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tcase <-ticker.C:\n\t\t\t\tfileCount, _ := db.GetReplicaFileCount()\n\t\t\t\tdbSize, _ := db.GetDatabaseSize()\n\t\t\t\tt.Logf(\"[Progress] Files: %d, DB Size: %.2f MB, Elapsed: %v\",\n\t\t\t\t\tfileCount, float64(dbSize)/(1024*1024), time.Since(time.Now().Add(-duration)))\n\t\t\t}\n\t\t}\n\t}()\n\n\tif err := db.GenerateLoad(ctx, config.WriteRate, config.Duration, string(config.Pattern)); err != nil && ctx.Err() == nil {\n\t\tt.Fatalf(\"Load generation failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Load generation complete\")\n\n\ttime.Sleep(1 * time.Minute)\n\n\tt.Log(\"[4] Final statistics...\")\n\tfileCount, err := db.GetReplicaFileCount()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check replica: %v\", err)\n\t}\n\n\tdbSize, err := db.GetDatabaseSize()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get database size: %v\", err)\n\t}\n\n\tt.Logf(\"Final LTX files: %d\", fileCount)\n\tt.Logf(\"Final DB size: %.2f MB\", float64(dbSize)/(1024*1024))\n\n\tt.Log(\"[5] Checking for errors...\")\n\terrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check errors: %v\", err)\n\t}\n\n\tif len(errors) > 20 {\n\t\tt.Fatalf(\"Too many errors (%d), test may be unstable\", len(errors))\n\t} else if len(errors) > 0 {\n\t\tt.Logf(\"Found %d errors (acceptable for long test)\", len(errors))\n\t} else {\n\t\tt.Log(\"✓ No errors detected\")\n\t}\n\n\tdb.StopLitestream()\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[6] Testing final restore...\")\n\trestoredPath := filepath.Join(db.TempDir, \"overnight-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Restore successful\")\n\n\tt.Log(\"[7] Full validation...\")\n\tif err := db.Validate(restoredPath); err != nil {\n\t\tt.Fatalf(\"Validation failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Validation passed\")\n\tt.Log(\"TEST PASSED: Overnight file replication successful\")\n}\n\nfunc TestOvernightComprehensive(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping long integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tstartTime := time.Now()\n\tduration := GetTestDuration(t, 8*time.Hour)\n\tt.Logf(\"Testing: Comprehensive overnight test (duration: %v)\", duration)\n\n\tdb := SetupTestDB(t, \"overnight-comprehensive\")\n\tdefer db.Cleanup()\n\tdefer db.PrintTestSummary(t, \"Overnight Comprehensive Test\", startTime)\n\n\tt.Log(\"[1] Creating large database...\")\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\tif err := db.Populate(\"500MB\"); err != nil {\n\t\tt.Fatalf(\"Failed to populate database: %v\", err)\n\t}\n\n\tt.Log(\"✓ Database populated to 500MB\")\n\n\tt.Log(\"[2] Starting Litestream...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(10 * time.Second)\n\n\tt.Log(\"[3] Generating mixed workload...\")\n\tctx, cancel := context.WithTimeout(context.Background(), duration)\n\tdefer cancel()\n\n\tconfig := DefaultLoadConfig()\n\tconfig.WriteRate = 100\n\tconfig.Duration = duration\n\tconfig.Pattern = LoadPatternWave\n\tconfig.PayloadSize = 4 * 1024\n\tconfig.ReadRatio = 0.3\n\tconfig.Workers = 8\n\n\tticker := time.NewTicker(5 * time.Minute)\n\tdefer ticker.Stop()\n\n\tgo func() {\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tcase <-ticker.C:\n\t\t\t\tfileCount, _ := db.GetReplicaFileCount()\n\t\t\t\tdbSize, _ := db.GetDatabaseSize()\n\t\t\t\tt.Logf(\"[Progress] Files: %d, DB Size: %.2f MB\", fileCount, float64(dbSize)/(1024*1024))\n\t\t\t}\n\t\t}\n\t}()\n\n\tif err := db.GenerateLoad(ctx, config.WriteRate, config.Duration, string(config.Pattern)); err != nil && ctx.Err() == nil {\n\t\tt.Fatalf(\"Load generation failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Load generation complete\")\n\n\ttime.Sleep(2 * time.Minute)\n\n\tdb.StopLitestream()\n\n\tt.Log(\"[4] Final validation...\")\n\trestoredPath := filepath.Join(db.TempDir, \"comprehensive-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t}\n\n\tif err := db.Validate(restoredPath); err != nil {\n\t\tt.Fatalf(\"Validation failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Comprehensive test passed\")\n\tt.Log(\"TEST PASSED: Overnight comprehensive test successful\")\n}\n" }, { "path": "tests/integration/profile_test.go", "content": "//go:build profile\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"log\"\n\t\"net/http\"\n\t_ \"net/http/pprof\"\n\t\"os\"\n\t\"os/signal\"\n\t\"path/filepath\"\n\t\"strconv\"\n\t\"testing\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n\n\t_ \"modernc.org/sqlite\"\n)\n\n// TestIdleCPUProfile starts N databases with file-based replicas and no writes,\n// then exposes pprof and fgprof endpoints for interactive profiling.\n//\n// This test is designed for manual CPU profiling to understand idle overhead\n// when running many Litestream instances on a single machine.\n//\n// Usage:\n//\n//\t# Start with 100 idle databases on default port:\n//\tPROFILE_DB_COUNT=100 go test -tags=profile -run TestIdleCPUProfile -timeout=0 -v ./tests/integration/\n//\n//\t# Custom listen address:\n//\tPROFILE_ADDR=:9090 PROFILE_DB_COUNT=50 go test -tags=profile -run TestIdleCPUProfile -timeout=0 -v ./tests/integration/\n//\n//\t# Then in another terminal:\n//\tgo tool pprof http://localhost:6060/debug/pprof/profile?seconds=30\n//\tgo tool pprof http://localhost:6060/debug/pprof/goroutine\n//\tgo tool pprof http://localhost:6060/debug/pprof/heap\nfunc TestIdleCPUProfile(t *testing.T) {\n\tdbCount := 10\n\tif s := os.Getenv(\"PROFILE_DB_COUNT\"); s != \"\" {\n\t\tn, err := strconv.Atoi(s)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"invalid PROFILE_DB_COUNT: %v\", err)\n\t\t}\n\t\tdbCount = n\n\t}\n\n\taddr := \":6060\"\n\tif s := os.Getenv(\"PROFILE_ADDR\"); s != \"\" {\n\t\taddr = s\n\t}\n\n\t// Start HTTP server for profiling (net/http/pprof registered via blank import).\n\tgo func() {\n\t\tlog.Printf(\"pprof server listening on %s\", addr)\n\t\tif err := http.ListenAndServe(addr, nil); err != nil {\n\t\t\tlog.Printf(\"pprof server error: %v\", err)\n\t\t}\n\t}()\n\n\t// Create temporary root directory for all databases.\n\trootDir := t.TempDir()\n\n\t// Start N databases with monitoring enabled (the idle hot path).\n\ttype instance struct {\n\t\tdb *litestream.DB\n\t\tsqldb *sql.DB\n\t}\n\tinstances := make([]instance, 0, dbCount)\n\n\tfor i := range dbCount {\n\t\tdbPath := filepath.Join(rootDir, fmt.Sprintf(\"db%d\", i), \"db\")\n\t\treplicaDir := filepath.Join(rootDir, fmt.Sprintf(\"db%d\", i), \"replica\")\n\n\t\tif err := os.MkdirAll(filepath.Dir(dbPath), 0o755); err != nil {\n\t\t\tt.Fatalf(\"mkdir: %v\", err)\n\t\t}\n\n\t\t// Create database with WAL mode and seed data.\n\t\tsqldb, err := sql.Open(\"sqlite\", dbPath)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"open sql db %d: %v\", i, err)\n\t\t}\n\t\tif _, err := sqldb.Exec(`PRAGMA journal_mode = wal`); err != nil {\n\t\t\tt.Fatalf(\"set wal mode db %d: %v\", i, err)\n\t\t}\n\t\tif _, err := sqldb.Exec(`CREATE TABLE data (id INTEGER PRIMARY KEY, value TEXT)`); err != nil {\n\t\t\tt.Fatalf(\"create table db %d: %v\", i, err)\n\t\t}\n\t\tif _, err := sqldb.Exec(`INSERT INTO data (value) VALUES ('seed')`); err != nil {\n\t\t\tt.Fatalf(\"insert seed db %d: %v\", i, err)\n\t\t}\n\n\t\t// Configure Litestream DB with monitoring enabled.\n\t\tdb := litestream.NewDB(dbPath)\n\t\tdb.Replica = litestream.NewReplica(db)\n\t\tdb.Replica.Client = file.NewReplicaClient(replicaDir)\n\n\t\tif err := db.Open(); err != nil {\n\t\t\tt.Fatalf(\"open litestream db %d: %v\", i, err)\n\t\t}\n\n\t\t// Do an initial sync so there's a valid LTX baseline.\n\t\tif err := db.Sync(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"initial sync db %d: %v\", i, err)\n\t\t}\n\n\t\tinstances = append(instances, instance{db: db, sqldb: sqldb})\n\t}\n\n\tt.Logf(\"started %d idle databases with monitoring (interval=%s)\", dbCount, litestream.DefaultMonitorInterval)\n\tt.Logf(\"\")\n\tt.Logf(\"profiling endpoints:\")\n\tt.Logf(\" CPU (on-cpu): go tool pprof http://localhost%s/debug/pprof/profile?seconds=30\", addr)\n\tt.Logf(\" goroutines: go tool pprof http://localhost%s/debug/pprof/goroutine\", addr)\n\tt.Logf(\" heap: go tool pprof http://localhost%s/debug/pprof/heap\", addr)\n\tt.Logf(\"\")\n\tt.Logf(\"press Ctrl+C to stop\")\n\n\t// Block until interrupted.\n\tctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)\n\tdefer stop()\n\t<-ctx.Done()\n\n\tt.Logf(\"shutting down %d databases...\", dbCount)\n\tfor i, inst := range instances {\n\t\tif err := inst.db.Close(context.Background()); err != nil {\n\t\t\tt.Logf(\"close litestream db %d: %v\", i, err)\n\t\t}\n\t\tif err := inst.sqldb.Close(); err != nil {\n\t\t\tt.Logf(\"close sql db %d: %v\", i, err)\n\t\t}\n\t}\n}\n" }, { "path": "tests/integration/quick_test.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\nfunc TestQuickValidation(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tstartTime := time.Now()\n\tduration := GetTestDuration(t, 30*time.Minute)\n\tt.Logf(\"Testing: Quick validation test (duration: %v)\", duration)\n\tt.Log(\"Default: 30 minutes, configurable via test duration\")\n\n\tdb := SetupTestDB(t, \"quick-validation\")\n\tdefer db.Cleanup()\n\tdefer db.PrintTestSummary(t, \"Quick Validation Test\", startTime)\n\n\tt.Log(\"[1] Creating and populating database...\")\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\tif err := db.Populate(\"10MB\"); err != nil {\n\t\tt.Fatalf(\"Failed to populate database: %v\", err)\n\t}\n\n\tt.Log(\"✓ Database populated to 10MB\")\n\n\tt.Log(\"[2] Starting Litestream...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(5 * time.Second)\n\n\tt.Log(\"[3] Generating wave pattern load...\")\n\tctx, cancel := context.WithTimeout(context.Background(), duration)\n\tdefer cancel()\n\n\tconfig := DefaultLoadConfig()\n\tconfig.WriteRate = 100\n\tconfig.Duration = duration\n\tconfig.Pattern = LoadPatternWave\n\tconfig.PayloadSize = 4 * 1024\n\tconfig.Workers = 4\n\n\tif err := db.GenerateLoad(ctx, config.WriteRate, config.Duration, string(config.Pattern)); err != nil && ctx.Err() == nil {\n\t\tt.Fatalf(\"Load generation failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Load generation complete\")\n\n\ttime.Sleep(10 * time.Second)\n\n\tt.Log(\"[4] Checking replica status...\")\n\tfileCount, err := db.GetReplicaFileCount()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check replica: %v\", err)\n\t}\n\n\tif fileCount == 0 {\n\t\tt.Fatal(\"No LTX segments created!\")\n\t}\n\n\tt.Logf(\"✓ LTX segments created: %d files\", fileCount)\n\n\tdbSize, err := db.GetDatabaseSize()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get database size: %v\", err)\n\t}\n\n\tt.Logf(\"Database size: %.2f MB\", float64(dbSize)/(1024*1024))\n\n\tt.Log(\"[5] Checking for errors...\")\n\terrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check errors: %v\", err)\n\t}\n\n\tif len(errors) > 10 {\n\t\tt.Fatalf(\"Too many critical errors (%d), showing first 5:\\n%v\", len(errors), errors[:5])\n\t} else if len(errors) > 0 {\n\t\tt.Logf(\"Found %d errors (showing first 3):\", len(errors))\n\t\tfor i := 0; i < min(len(errors), 3); i++ {\n\t\t\tt.Logf(\" %s\", errors[i])\n\t\t}\n\t} else {\n\t\tt.Log(\"✓ No errors detected\")\n\t}\n\n\tdb.StopLitestream()\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[6] Testing restore...\")\n\trestoredPath := filepath.Join(db.TempDir, \"quick-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Restore successful\")\n\n\tt.Log(\"[7] Validating restoration...\")\n\tif err := db.QuickValidate(restoredPath); err != nil {\n\t\tt.Fatalf(\"Validation failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Validation passed\")\n\tt.Log(\"TEST PASSED: Quick validation successful\")\n}\n" }, { "path": "tests/integration/s3_access_point_test.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/aws/aws-sdk-go-v2/aws\"\n\t\"github.com/aws/aws-sdk-go-v2/config\"\n\t\"github.com/aws/aws-sdk-go-v2/credentials\"\n\tawss3 \"github.com/aws/aws-sdk-go-v2/service/s3\"\n\ts3types \"github.com/aws/aws-sdk-go-v2/service/s3/types\"\n)\n\n// TestS3AccessPointLocalStack verifies replication to an S3 access point via LocalStack.\nfunc TestS3AccessPointLocalStack(t *testing.T) {\n\tRequireBinaries(t)\n\tRequireDocker(t)\n\n\tcontainerName, endpoint := StartMinioTestContainer(t)\n\tt.Cleanup(func() {\n\t\tStopMinioTestContainer(t, containerName)\n\t})\n\n\tctx := context.Background()\n\tconfigEndpoint := strings.Replace(endpoint, \"localhost\", \"s3-accesspoint.127.0.0.1.nip.io\", 1)\n\ts3Client := newMinioS3Client(t, configEndpoint, false)\n\n\taccountID := \"000000000000\"\n\taccessPointName := fmt.Sprintf(\"litestream-ap-%d\", time.Now().UnixNano())\n\tbucket := fmt.Sprintf(\"%s-%s\", accessPointName, accountID)\n\taccessPointARN := fmt.Sprintf(\"arn:aws:s3:us-east-1:%s:accesspoint/%s\", accountID, accessPointName)\n\n\tcreateBucket(t, ctx, s3Client, bucket)\n\tt.Cleanup(func() {\n\t\tif os.Getenv(\"SOAK_KEEP_TEMP\") != \"\" {\n\t\t\tt.Logf(\"SOAK_KEEP_TEMP set, preserving bucket %s\", bucket)\n\t\t\treturn\n\t\t}\n\t\tif err := clearBucket(ctx, s3Client, bucket); err != nil {\n\t\t\tt.Logf(\"warn: clear bucket: %v\", err)\n\t\t}\n\t})\n\n\tt.Logf(\"simulated access point ARN: %s\", accessPointARN)\n\n\tdb := SetupTestDB(t, \"localstack-accesspoint\")\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"create db: %v\", err)\n\t}\n\tif err := db.Populate(\"5MB\"); err != nil {\n\t\tt.Fatalf(\"populate db: %v\", err)\n\t}\n\n\treplicaURL := fmt.Sprintf(\"s3://%s/test-prefix\", accessPointARN)\n\tdb.ReplicaURL = replicaURL\n\n\tconfigPath := WriteS3AccessPointConfig(t, db.Path, replicaURL, configEndpoint, false, \"minioadmin\", \"minioadmin\")\n\tdb.ConfigPath = configPath\n\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\tt.Cleanup(func() {\n\t\t_ = db.StopLitestream()\n\t})\n\n\tctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)\n\tdefer cancel()\n\tif err := db.GenerateLoad(ctx, 200, 5*time.Second, \"steady\"); err != nil {\n\t\tt.Fatalf(\"generate load: %v\", err)\n\t}\n\n\t// Wait for uploaded LTX files to appear in the underlying bucket.\n\twaitForObjects(t, s3Client, bucket, \"test-prefix\", 30*time.Second)\n\n\tif err := db.StopLitestream(); err != nil {\n\t\tt.Fatalf(\"stop litestream: %v\", err)\n\t}\n\tdb.LitestreamCmd = nil\n\n\trestoredPath := filepath.Join(db.TempDir, \"restored-access-point.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"restore: %v\", err)\n\t}\n\n\tif err := compareRowCounts(db.Path, restoredPath); err != nil {\n\t\tt.Fatalf(\"row compare: %v\", err)\n\t}\n}\n\nfunc newMinioS3Client(t *testing.T, endpoint string, forcePathStyle bool) *awss3.Client {\n\tt.Helper()\n\n\tresolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {\n\t\treturn aws.Endpoint{\n\t\t\tPartitionID: \"aws\",\n\t\t\tURL: endpoint,\n\t\t\tSigningRegion: \"us-east-1\",\n\t\t\tHostnameImmutable: true,\n\t\t}, nil\n\t})\n\n\tcfg, err := config.LoadDefaultConfig(context.Background(),\n\t\tconfig.WithRegion(\"us-east-1\"),\n\t\tconfig.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(\"minioadmin\", \"minioadmin\", \"\")),\n\t\tconfig.WithEndpointResolverWithOptions(resolver),\n\t)\n\tif err != nil {\n\t\tt.Fatalf(\"load aws config: %v\", err)\n\t}\n\n\treturn awss3.NewFromConfig(cfg, func(o *awss3.Options) {\n\t\to.UsePathStyle = forcePathStyle\n\t})\n}\n\nfunc createBucket(t *testing.T, ctx context.Context, client *awss3.Client, bucket string) {\n\tt.Helper()\n\n\tif _, err := client.CreateBucket(ctx, &awss3.CreateBucketInput{Bucket: aws.String(bucket)}); err != nil {\n\t\tt.Fatalf(\"create bucket: %v\", err)\n\t}\n}\n\nfunc clearBucket(ctx context.Context, client *awss3.Client, bucket string) error {\n\tpaginator := awss3.NewListObjectsV2Paginator(client, &awss3.ListObjectsV2Input{Bucket: aws.String(bucket)})\n\tfor paginator.HasMorePages() {\n\t\tpage, err := paginator.NextPage(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"list objects: %w\", err)\n\t\t}\n\t\tif len(page.Contents) == 0 {\n\t\t\tbreak\n\t\t}\n\t\tobjs := make([]s3types.ObjectIdentifier, 0, len(page.Contents))\n\t\tfor _, item := range page.Contents {\n\t\t\tobjs = append(objs, s3types.ObjectIdentifier{Key: item.Key})\n\t\t}\n\t\tif _, err := client.DeleteObjects(ctx, &awss3.DeleteObjectsInput{\n\t\t\tBucket: aws.String(bucket),\n\t\t\tDelete: &s3types.Delete{Objects: objs, Quiet: aws.Bool(true)},\n\t\t}); err != nil {\n\t\t\treturn fmt.Errorf(\"delete objects: %w\", err)\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc waitForObjects(t *testing.T, client *awss3.Client, bucket, prefix string, timeout time.Duration) {\n\tt.Helper()\n\n\ttrimmedPrefix := strings.Trim(prefix, \"/\")\n\tif trimmedPrefix != \"\" {\n\t\ttrimmedPrefix += \"/\"\n\t}\n\n\tdeadline := time.Now().Add(timeout)\n\tvar lastErr error\n\tfor {\n\t\tout, err := client.ListObjectsV2(context.Background(), &awss3.ListObjectsV2Input{\n\t\t\tBucket: aws.String(bucket),\n\t\t\tPrefix: aws.String(trimmedPrefix),\n\t\t})\n\t\tif err == nil && len(out.Contents) > 0 {\n\t\t\treturn\n\t\t}\n\t\tif err != nil {\n\t\t\tlastErr = err\n\t\t} else {\n\t\t\tlastErr = fmt.Errorf(\"no objects yet\")\n\t\t}\n\t\tif time.Now().After(deadline) {\n\t\t\tt.Fatalf(\"timeout waiting for objects in bucket %s with prefix %s: last err %v\", bucket, trimmedPrefix, lastErr)\n\t\t}\n\t\ttime.Sleep(2 * time.Second)\n\t}\n}\n\nfunc compareRowCounts(srcPath, restoredPath string) error {\n\tsrcDB, err := sql.Open(\"sqlite3\", srcPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open source db: %w\", err)\n\t}\n\tdefer srcDB.Close()\n\n\trestoredDB, err := sql.Open(\"sqlite3\", restoredPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open restored db: %w\", err)\n\t}\n\tdefer restoredDB.Close()\n\n\ttableName, err := findUserTable(srcDB)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar srcCount, restoredCount int\n\tif err := srcDB.QueryRow(fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", tableName)).Scan(&srcCount); err != nil {\n\t\treturn fmt.Errorf(\"count source: %w\", err)\n\t}\n\tif err := restoredDB.QueryRow(fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", tableName)).Scan(&restoredCount); err != nil {\n\t\treturn fmt.Errorf(\"count restored: %w\", err)\n\t}\n\tif srcCount != restoredCount {\n\t\treturn fmt.Errorf(\"row mismatch: source=%d restored=%d\", srcCount, restoredCount)\n\t}\n\treturn nil\n}\n\nfunc findUserTable(db *sql.DB) (string, error) {\n\tvar name string\n\terr := db.QueryRow(`SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY name LIMIT 1`).Scan(&name)\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"find table: %w\", err)\n\t}\n\treturn name, nil\n}\n" }, { "path": "tests/integration/s3_restore_connection_drop_test.go", "content": "//go:build integration && docker\n\npackage integration\n\nimport (\n\t\"bytes\"\n\t\"database/sql\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\n// TestRestore_S3ConnectionDrop verifies restore can recover from dropped S3-compatible connections.\nfunc TestRestore_S3ConnectionDrop(t *testing.T) {\n\tRequireBinaries(t)\n\tRequireDocker(t)\n\n\tif testing.Short() {\n\t\tt.Skip(\"skipping in short mode\")\n\t}\n\n\tnetworkName := startDockerNetwork(t)\n\tdefer removeDockerNetwork(networkName)\n\n\tminioName := startMinioContainerForProxy(t, networkName)\n\tdefer stopDockerContainer(minioName)\n\n\ttoxiproxyName, toxiproxyAPIPort, toxiproxyProxyPort := startToxiproxyContainer(t, networkName)\n\tdefer stopDockerContainer(toxiproxyName)\n\n\tbucket := fmt.Sprintf(\"litestream-test-%d\", time.Now().UnixNano())\n\tcreateMinioBucket(t, networkName, minioName, bucket)\n\n\tproxyEndpoint := fmt.Sprintf(\"http://localhost:%s\", toxiproxyProxyPort)\n\tproxyClient := newToxiproxyClient(t, fmt.Sprintf(\"http://localhost:%s\", toxiproxyAPIPort))\n\tproxyClient.createProxy(t, \"minio\", \"0.0.0.0:8666\", fmt.Sprintf(\"%s:9000\", minioName))\n\n\treplicaPath := fmt.Sprintf(\"restore-drop-%d\", time.Now().UnixNano())\n\treplicaURL := fmt.Sprintf(\"s3://%s/%s\", bucket, replicaPath)\n\n\tdb := SetupTestDB(t, \"s3-restore-connection-drop\")\n\tdefer db.Cleanup()\n\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"create db: %v\", err)\n\t}\n\n\tif err := db.Populate(\"100MB\"); err != nil {\n\t\tt.Fatalf(\"populate db: %v\", err)\n\t}\n\n\tconfigPath := writeS3Config(t, db.Path, replicaURL, proxyEndpoint)\n\tdb.ReplicaURL = replicaURL\n\tif err := db.StartLitestreamWithConfig(configPath); err != nil {\n\t\tt.Fatalf(\"start litestream: %v\", err)\n\t}\n\n\ttime.Sleep(5 * time.Second)\n\n\tif err := insertLargeRows(db.Path, 5, 256*1024); err != nil {\n\t\tt.Fatalf(\"insert post-snapshot rows: %v\", err)\n\t}\n\n\ttime.Sleep(5 * time.Second)\n\n\tif err := db.StopLitestream(); err != nil {\n\t\tt.Fatalf(\"stop litestream: %v\", err)\n\t}\n\n\trestorePath := filepath.Join(db.TempDir, \"restored.db\")\n\trestoreErr := make(chan error, 1)\n\tgo func() {\n\t\trestoreErr <- db.Restore(restorePath)\n\t}()\n\n\ttime.Sleep(200 * time.Millisecond)\n\tproxyClient.addResetPeerToxic(t, \"minio\", \"reset-connection\", 200)\n\ttime.Sleep(400 * time.Millisecond)\n\tproxyClient.removeToxic(t, \"minio\", \"reset-connection\")\n\n\tif err := <-restoreErr; err != nil {\n\t\tt.Fatalf(\"restore failed: %v\", err)\n\t}\n\n\tif err := verifyRestoredRowCount(restorePath, 5); err != nil {\n\t\tt.Fatalf(\"restore validation failed: %v\", err)\n\t}\n}\n\nfunc startDockerNetwork(t *testing.T) string {\n\tt.Helper()\n\tname := fmt.Sprintf(\"litestream-net-%d\", time.Now().UnixNano())\n\trunDockerCommand(t, \"network\", \"create\", name)\n\treturn name\n}\n\nfunc removeDockerNetwork(name string) {\n\tif name == \"\" {\n\t\treturn\n\t}\n\texec.Command(\"docker\", \"network\", \"rm\", name).Run()\n}\n\nfunc startMinioContainerForProxy(t *testing.T, networkName string) string {\n\tt.Helper()\n\tname := fmt.Sprintf(\"litestream-minio-%d\", time.Now().UnixNano())\n\texec.Command(\"docker\", \"rm\", \"-f\", name).Run()\n\n\trunDockerCommand(t, \"run\", \"-d\",\n\t\t\"--name\", name,\n\t\t\"--network\", networkName,\n\t\t\"-e\", \"MINIO_ROOT_USER=minioadmin\",\n\t\t\"-e\", \"MINIO_ROOT_PASSWORD=minioadmin\",\n\t\t\"minio/minio\", \"server\", \"/data\",\n\t)\n\n\ttime.Sleep(3 * time.Second)\n\treturn name\n}\n\nfunc startToxiproxyContainer(t *testing.T, networkName string) (string, string, string) {\n\tt.Helper()\n\tname := fmt.Sprintf(\"litestream-toxiproxy-%d\", time.Now().UnixNano())\n\texec.Command(\"docker\", \"rm\", \"-f\", name).Run()\n\n\timage := os.Getenv(\"LITESTREAM_TOXIPROXY_IMAGE\")\n\tif image == \"\" {\n\t\timage = \"ghcr.io/shopify/toxiproxy:2.5.0\"\n\t}\n\n\trunDockerCommand(t, \"run\", \"-d\",\n\t\t\"--name\", name,\n\t\t\"--network\", networkName,\n\t\t\"-p\", \"0:8474\",\n\t\t\"-p\", \"0:8666\",\n\t\timage,\n\t)\n\n\tapiPort := parseDockerPort(t, runDockerCommand(t, \"port\", name, \"8474/tcp\"))\n\tproxyPort := parseDockerPort(t, runDockerCommand(t, \"port\", name, \"8666/tcp\"))\n\n\ttime.Sleep(2 * time.Second)\n\n\treturn name, apiPort, proxyPort\n}\n\nfunc stopDockerContainer(name string) {\n\tif name == \"\" {\n\t\treturn\n\t}\n\texec.Command(\"docker\", \"rm\", \"-f\", name).Run()\n}\n\nfunc createMinioBucket(t *testing.T, networkName, minioName, bucket string) {\n\tt.Helper()\n\tcmd := exec.Command(\"docker\", \"run\", \"--rm\",\n\t\t\"--network\", networkName,\n\t\t\"-e\", fmt.Sprintf(\"MC_HOST_minio=http://minioadmin:minioadmin@%s:9000\", minioName),\n\t\t\"minio/mc\", \"mb\", \"minio/\"+bucket,\n\t)\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil && !strings.Contains(string(output), \"already exists\") {\n\t\tt.Fatalf(\"create bucket failed: %v output: %s\", err, string(output))\n\t}\n}\n\nfunc writeS3Config(t *testing.T, dbPath, replicaURL, endpoint string) string {\n\tt.Helper()\n\tconfigPath := filepath.Join(filepath.Dir(dbPath), \"litestream-s3-drop.yml\")\n\tconfig := fmt.Sprintf(`access-key-id: minioadmin\nsecret-access-key: minioadmin\n\ndbs:\n - path: %s\n snapshot:\n interval: 1s\n retention: 1h\n replicas:\n - url: %s\n endpoint: %s\n region: us-east-1\n force-path-style: true\n skip-verify: true\n sync-interval: 1s\n`, filepath.ToSlash(dbPath), replicaURL, endpoint)\n\n\tif err := os.WriteFile(configPath, []byte(config), 0600); err != nil {\n\t\tt.Fatalf(\"write config: %v\", err)\n\t}\n\n\treturn configPath\n}\n\nfunc insertLargeRows(dbPath string, rows int, blobSize int) error {\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer sqlDB.Close()\n\n\tif _, err := sqlDB.Exec(`CREATE TABLE IF NOT EXISTS drop_test(id INTEGER PRIMARY KEY, data BLOB);`); err != nil {\n\t\treturn err\n\t}\n\n\tfor i := 0; i < rows; i++ {\n\t\tif _, err := sqlDB.Exec(`INSERT INTO drop_test(data) VALUES (randomblob(?));`, blobSize); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc verifyRestoredRowCount(dbPath string, expected int) error {\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer sqlDB.Close()\n\n\tvar count int\n\tif err := sqlDB.QueryRow(`SELECT COUNT(*) FROM drop_test;`).Scan(&count); err != nil {\n\t\treturn err\n\t}\n\tif count != expected {\n\t\treturn fmt.Errorf(\"restored row count: got %d want %d\", count, expected)\n\t}\n\treturn nil\n}\n\ntype toxiproxyClient struct {\n\tbaseURL string\n\tclient *http.Client\n}\n\nfunc newToxiproxyClient(t *testing.T, baseURL string) *toxiproxyClient {\n\tt.Helper()\n\treturn &toxiproxyClient{\n\t\tbaseURL: strings.TrimRight(baseURL, \"/\"),\n\t\tclient: &http.Client{\n\t\t\tTimeout: 5 * time.Second,\n\t\t},\n\t}\n}\n\nfunc (c *toxiproxyClient) createProxy(t *testing.T, name, listen, upstream string) {\n\tt.Helper()\n\tpayload := map[string]string{\n\t\t\"name\": name,\n\t\t\"listen\": listen,\n\t\t\"upstream\": upstream,\n\t}\n\tc.postJSON(t, \"/proxies\", payload, http.StatusOK)\n}\n\nfunc (c *toxiproxyClient) addResetPeerToxic(t *testing.T, proxy, name string, timeoutMS int) {\n\tt.Helper()\n\tpayload := map[string]interface{}{\n\t\t\"name\": name,\n\t\t\"type\": \"reset_peer\",\n\t\t\"stream\": \"downstream\",\n\t\t\"toxicity\": 1.0,\n\t\t\"attributes\": map[string]int{\n\t\t\t\"timeout\": timeoutMS,\n\t\t},\n\t}\n\tc.postJSON(t, fmt.Sprintf(\"/proxies/%s/toxics\", proxy), payload, http.StatusOK)\n}\n\nfunc (c *toxiproxyClient) removeToxic(t *testing.T, proxy, name string) {\n\tt.Helper()\n\treq, err := http.NewRequest(http.MethodDelete, c.baseURL+fmt.Sprintf(\"/proxies/%s/toxics/%s\", proxy, name), nil)\n\tif err != nil {\n\t\tt.Fatalf(\"create delete request: %v\", err)\n\t}\n\tresp, err := c.client.Do(req)\n\tif err != nil {\n\t\tt.Fatalf(\"delete toxic: %v\", err)\n\t}\n\tdefer resp.Body.Close()\n\tif resp.StatusCode != http.StatusNoContent && resp.StatusCode != http.StatusOK {\n\t\tbody, _ := io.ReadAll(resp.Body)\n\t\tt.Fatalf(\"delete toxic failed: status=%d body=%s\", resp.StatusCode, string(body))\n\t}\n}\n\nfunc (c *toxiproxyClient) postJSON(t *testing.T, path string, payload interface{}, expectedStatus int) {\n\tt.Helper()\n\tbody, err := json.Marshal(payload)\n\tif err != nil {\n\t\tt.Fatalf(\"marshal payload: %v\", err)\n\t}\n\treq, err := http.NewRequest(http.MethodPost, c.baseURL+path, bytes.NewReader(body))\n\tif err != nil {\n\t\tt.Fatalf(\"create request: %v\", err)\n\t}\n\treq.Header.Set(\"Content-Type\", \"application/json\")\n\tresp, err := c.client.Do(req)\n\tif err != nil {\n\t\tt.Fatalf(\"post %s: %v\", path, err)\n\t}\n\tdefer resp.Body.Close()\n\tif resp.StatusCode != expectedStatus {\n\t\tif expectedStatus == http.StatusOK && resp.StatusCode == http.StatusCreated {\n\t\t\treturn\n\t\t}\n\t\trespBody, _ := io.ReadAll(resp.Body)\n\t\tt.Fatalf(\"post %s failed: status=%d body=%s\", path, resp.StatusCode, string(respBody))\n\t}\n}\n" }, { "path": "tests/integration/scenario_test.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"database/sql\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\nfunc TestFreshStart(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tt.Log(\"Testing: Starting replication with a fresh (empty) database\")\n\tt.Log(\"This tests if Litestream works correctly when it creates the database from scratch\")\n\n\tdb := SetupTestDB(t, \"fresh-start\")\n\tdefer db.Cleanup()\n\n\tt.Log(\"[1] Starting Litestream with non-existent database...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[2] Creating database while Litestream is running...\")\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to open database: %v\", err)\n\t}\n\n\tif _, err := sqlDB.Exec(\"PRAGMA journal_mode=WAL\"); err != nil {\n\t\tt.Fatalf(\"Failed to set WAL mode: %v\", err)\n\t}\n\n\tif _, err := sqlDB.Exec(\"CREATE TABLE test (id INTEGER PRIMARY KEY, data TEXT)\"); err != nil {\n\t\tt.Fatalf(\"Failed to create table: %v\", err)\n\t}\n\n\tif _, err := sqlDB.Exec(\"INSERT INTO test (data) VALUES ('initial data')\"); err != nil {\n\t\tt.Fatalf(\"Failed to insert initial data: %v\", err)\n\t}\n\tsqlDB.Close()\n\n\ttime.Sleep(3 * time.Second)\n\n\tt.Log(\"[3] Checking if Litestream detected the database...\")\n\tlog, err := db.GetLitestreamLog()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to read log: %v\", err)\n\t}\n\n\tt.Logf(\"Litestream log snippet:\\n%s\", log[:min(len(log), 500)])\n\n\tt.Log(\"[4] Adding data to test replication...\")\n\tsqlDB, err = sql.Open(\"sqlite3\", db.Path)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to open database: %v\", err)\n\t}\n\n\tfor i := 1; i <= 100; i++ {\n\t\tif _, err := sqlDB.Exec(\"INSERT INTO test (data) VALUES (?)\", fmt.Sprintf(\"row %d\", i)); err != nil {\n\t\t\tt.Fatalf(\"Failed to insert row %d: %v\", i, err)\n\t\t}\n\t}\n\tsqlDB.Close()\n\n\ttime.Sleep(5 * time.Second)\n\n\tt.Log(\"[5] Checking for errors...\")\n\terrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check errors: %v\", err)\n\t}\n\n\tif len(errors) > 1 {\n\t\tt.Logf(\"Found %d errors (showing first 3):\", len(errors))\n\t\tfor i := 0; i < min(len(errors), 3); i++ {\n\t\t\tt.Logf(\" %s\", errors[i])\n\t\t}\n\t} else {\n\t\tt.Log(\"✓ No significant errors\")\n\t}\n\n\tt.Log(\"[6] Checking replica files...\")\n\tfileCount, err := db.GetReplicaFileCount()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get replica file count: %v\", err)\n\t}\n\n\tif fileCount == 0 {\n\t\tt.Fatal(\"✗ No replica files created!\")\n\t}\n\n\tt.Logf(\"✓ Replica created with %d LTX files\", fileCount)\n\n\tdb.StopLitestream()\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[7] Testing restore...\")\n\trestoredPath := filepath.Join(db.TempDir, \"fresh-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"✗ Restore failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Restore successful\")\n\n\torigCount, err := db.GetRowCount(\"test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get original row count: %v\", err)\n\t}\n\n\trestoredDB := &TestDB{Path: restoredPath, t: t}\n\trestCount, err := restoredDB.GetRowCount(\"test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get restored row count: %v\", err)\n\t}\n\n\tif origCount != restCount {\n\t\tt.Fatalf(\"✗ Data mismatch: Original=%d, Restored=%d\", origCount, restCount)\n\t}\n\n\tt.Logf(\"✓ Data integrity verified: %d rows\", origCount)\n\tt.Log(\"TEST PASSED: Fresh start works correctly\")\n}\n\nfunc TestDatabaseIntegrity(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tt.Log(\"Testing: Complex data patterns and integrity after restore\")\n\n\tdb := SetupTestDB(t, \"integrity-test\")\n\tdefer db.Cleanup()\n\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\tt.Log(\"[1] Creating complex schema...\")\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to open database: %v\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tif err := CreateComplexTestSchema(sqlDB); err != nil {\n\t\tt.Fatalf(\"Failed to create schema: %v\", err)\n\t}\n\n\tt.Log(\"✓ Schema created\")\n\n\tt.Log(\"[2] Populating with test data...\")\n\tif err := PopulateComplexTestData(sqlDB, 10, 5, 3); err != nil {\n\t\tt.Fatalf(\"Failed to populate data: %v\", err)\n\t}\n\n\tt.Log(\"✓ Data populated (10 users, 50 posts, 150 comments)\")\n\n\tt.Log(\"[3] Starting Litestream...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(10 * time.Second)\n\n\tdb.StopLitestream()\n\ttime.Sleep(2 * time.Second)\n\n\tt.Log(\"[4] Checking integrity of original database...\")\n\tvar integrityResult string\n\tif err := sqlDB.QueryRow(\"PRAGMA integrity_check\").Scan(&integrityResult); err != nil {\n\t\tt.Fatalf(\"Integrity check failed: %v\", err)\n\t}\n\n\tif integrityResult != \"ok\" {\n\t\tt.Fatalf(\"Source database integrity check failed: %s\", integrityResult)\n\t}\n\n\tt.Log(\"✓ Source database integrity OK\")\n\n\tt.Log(\"[5] Restoring database...\")\n\trestoredPath := filepath.Join(db.TempDir, \"integrity-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Restore successful\")\n\n\tt.Log(\"[6] Checking integrity of restored database...\")\n\trestoredDB, err := sql.Open(\"sqlite3\", restoredPath)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to open restored database: %v\", err)\n\t}\n\tdefer restoredDB.Close()\n\n\tif err := restoredDB.QueryRow(\"PRAGMA integrity_check\").Scan(&integrityResult); err != nil {\n\t\tt.Fatalf(\"Restored integrity check failed: %v\", err)\n\t}\n\n\tif integrityResult != \"ok\" {\n\t\tt.Fatalf(\"Restored database integrity check failed: %s\", integrityResult)\n\t}\n\n\tt.Log(\"✓ Restored database integrity OK\")\n\n\tt.Log(\"[7] Validating data consistency...\")\n\ttables := []string{\"users\", \"posts\", \"comments\"}\n\tfor _, table := range tables {\n\t\tvar sourceCount, restoredCount int\n\n\t\tif err := sqlDB.QueryRow(fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", table)).Scan(&sourceCount); err != nil {\n\t\t\tt.Fatalf(\"Failed to count source %s: %v\", table, err)\n\t\t}\n\n\t\tif err := restoredDB.QueryRow(fmt.Sprintf(\"SELECT COUNT(*) FROM %s\", table)).Scan(&restoredCount); err != nil {\n\t\t\tt.Fatalf(\"Failed to count restored %s: %v\", table, err)\n\t\t}\n\n\t\tif sourceCount != restoredCount {\n\t\t\tt.Fatalf(\"Count mismatch for %s: source=%d, restored=%d\", table, sourceCount, restoredCount)\n\t\t}\n\n\t\tt.Logf(\"✓ Table %s: %d rows match\", table, sourceCount)\n\t}\n\n\tt.Log(\"TEST PASSED: Database integrity maintained through replication\")\n}\n\nfunc TestDatabaseDeletion(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\tRequireBinaries(t)\n\n\tt.Log(\"Testing: Database deletion during active replication\")\n\n\tdb := SetupTestDB(t, \"deletion-test\")\n\tdefer db.Cleanup()\n\n\tif err := db.Create(); err != nil {\n\t\tt.Fatalf(\"Failed to create database: %v\", err)\n\t}\n\n\tt.Log(\"[1] Creating test table and data...\")\n\tif err := CreateTestTable(t, db.Path); err != nil {\n\t\tt.Fatalf(\"Failed to create table: %v\", err)\n\t}\n\n\tif err := InsertTestData(t, db.Path, 100); err != nil {\n\t\tt.Fatalf(\"Failed to insert test data: %v\", err)\n\t}\n\n\tt.Log(\"✓ Created table with 100 rows\")\n\n\tt.Log(\"[2] Starting Litestream...\")\n\tif err := db.StartLitestream(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\n\ttime.Sleep(5 * time.Second)\n\n\tfileCount, _ := db.GetReplicaFileCount()\n\tt.Logf(\"✓ Replication started (%d files)\", fileCount)\n\n\tt.Log(\"[3] Deleting database files...\")\n\tos.Remove(db.Path)\n\tos.Remove(db.Path + \"-wal\")\n\tos.Remove(db.Path + \"-shm\")\n\n\ttime.Sleep(3 * time.Second)\n\n\tt.Log(\"✓ Database deleted\")\n\n\tt.Log(\"[4] Checking Litestream behavior...\")\n\terrors, err := db.CheckForErrors()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check errors: %v\", err)\n\t}\n\n\tt.Logf(\"Litestream reported %d error messages (expected after database deletion)\", len(errors))\n\n\tdb.StopLitestream()\n\n\tt.Log(\"[5] Verifying replica is still intact...\")\n\tfinalFileCount, err := db.GetReplicaFileCount()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check replica: %v\", err)\n\t}\n\n\tif finalFileCount == 0 {\n\t\tt.Fatalf(\"Replica appears to be empty or missing\")\n\t}\n\n\tt.Logf(\"✓ Replica exists with %d files (was %d - compaction may have reduced count)\", finalFileCount, fileCount)\n\n\tt.Log(\"[6] Testing restore from replica...\")\n\trestoredPath := filepath.Join(db.TempDir, \"deletion-restored.db\")\n\tif err := db.Restore(restoredPath); err != nil {\n\t\tt.Fatalf(\"Restore failed: %v\", err)\n\t}\n\n\tt.Log(\"✓ Restore successful\")\n\n\trestoredDB := &TestDB{Path: restoredPath, t: t}\n\trestCount, err := restoredDB.GetRowCount(\"test_data\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get restored row count: %v\", err)\n\t}\n\n\tif restCount != 100 {\n\t\tt.Fatalf(\"Expected 100 rows, got %d\", restCount)\n\t}\n\n\tt.Logf(\"✓ Restored database has correct data: %d rows\", restCount)\n\tt.Log(\"TEST PASSED: Replica survives source database deletion\")\n}\n\n// TestReplicaFailover was removed because Litestream no longer supports\n// multiple replicas on a single database (see cmd/litestream/main.go).\n// The bash script test-replica-failover.sh was also non-functional.\n\nfunc min(a, b int) int {\n\tif a < b {\n\t\treturn a\n\t}\n\treturn b\n}\n" }, { "path": "tests/integration/shutdown_retry_test.go", "content": "//go:build integration && docker\n\npackage integration\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/http/httputil\"\n\t\"net/url\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"syscall\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n)\n\n// TestShutdownSyncRetry_429Errors tests that Litestream retries syncing LTX files\n// during shutdown when receiving 429 (Too Many Requests) errors.\n//\n// This test:\n// 1. Starts a MinIO container\n// 2. Starts a rate-limiting proxy in front of MinIO that returns 429 for first N PUT requests\n// 3. Starts Litestream replicating to the proxy endpoint\n// 4. Writes data and syncs\n// 5. Sends SIGTERM to trigger graceful shutdown\n// 6. Verifies that Litestream retries and eventually succeeds despite 429 errors\n//\n// Requirements:\n// - Docker must be running\n// - Litestream binary must be built at ../../bin/litestream\nfunc TestShutdownSyncRetry_429Errors(t *testing.T) {\n\tRequireBinaries(t)\n\tRequireDocker(t)\n\n\tt.Log(\"================================================\")\n\tt.Log(\"Litestream Shutdown Sync Retry Test (429 Errors)\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\n\t// Start MinIO container\n\tt.Log(\"Starting MinIO container...\")\n\tcontainerName, minioEndpoint := StartMinioTestContainer(t)\n\tdefer StopMinioTestContainer(t, containerName)\n\tt.Logf(\"✓ MinIO running at: %s\", minioEndpoint)\n\n\t// Create MinIO bucket by creating directory in /data (MinIO stores buckets as directories)\n\tbucket := \"litestream-test\"\n\tt.Logf(\"Creating bucket '%s'...\", bucket)\n\n\t// Wait for MinIO to be ready\n\ttime.Sleep(2 * time.Second)\n\n\t// Create bucket directory directly - MinIO uses /data as the storage root\n\tcreateBucketCmd := exec.Command(\"docker\", \"exec\", containerName,\n\t\t\"mkdir\", \"-p\", \"/data/\"+bucket)\n\tif out, err := createBucketCmd.CombinedOutput(); err != nil {\n\t\tt.Fatalf(\"Failed to create bucket directory: %v, output: %s\", err, string(out))\n\t}\n\tt.Log(\"✓ Bucket created\")\n\tt.Log(\"\")\n\n\t// Start rate-limiting proxy\n\tt.Log(\"Starting rate-limiting proxy...\")\n\tproxy := newRateLimitingProxy(t, minioEndpoint, 3) // Return 429 for first 3 PUT requests\n\tproxyServer := &http.Server{\n\t\tAddr: \"127.0.0.1:0\",\n\t\tHandler: proxy,\n\t}\n\n\tlistener, err := (&net.ListenConfig{}).Listen(context.Background(), \"tcp\", \"127.0.0.1:0\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create listener: %v\", err)\n\t}\n\tproxyAddr := listener.Addr().String()\n\n\tgo func() {\n\t\tif err := proxyServer.Serve(listener); err != nil && err != http.ErrServerClosed {\n\t\t\tt.Logf(\"Proxy server error: %v\", err)\n\t\t}\n\t}()\n\tdefer proxyServer.Close()\n\n\tproxyEndpoint := fmt.Sprintf(\"http://%s\", proxyAddr)\n\tt.Logf(\"✓ Rate-limiting proxy running at: %s\", proxyEndpoint)\n\tt.Logf(\" (Will return 429 for first 3 PUT requests during shutdown)\")\n\tt.Log(\"\")\n\n\t// Setup test database\n\ttempDir := t.TempDir()\n\tdbPath := filepath.Join(tempDir, \"test.db\")\n\tconfigPath := filepath.Join(tempDir, \"litestream.yml\")\n\n\t// Create database with some data\n\tt.Log(\"Creating test database...\")\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to open database: %v\", err)\n\t}\n\n\tif _, err := sqlDB.Exec(\"PRAGMA journal_mode=WAL\"); err != nil {\n\t\tt.Fatalf(\"Failed to set WAL mode: %v\", err)\n\t}\n\tif _, err := sqlDB.Exec(\"CREATE TABLE test (id INTEGER PRIMARY KEY, data TEXT)\"); err != nil {\n\t\tt.Fatalf(\"Failed to create table: %v\", err)\n\t}\n\tif _, err := sqlDB.Exec(\"INSERT INTO test (data) VALUES ('initial data')\"); err != nil {\n\t\tt.Fatalf(\"Failed to insert data: %v\", err)\n\t}\n\tsqlDB.Close()\n\tt.Log(\"✓ Database created with initial data\")\n\tt.Log(\"\")\n\n\t// Create Litestream config with shutdown retry settings\n\ts3Path := fmt.Sprintf(\"test-%d\", time.Now().Unix())\n\tconfig := fmt.Sprintf(`\nshutdown-sync-timeout: 10s\nshutdown-sync-interval: 500ms\n\ndbs:\n - path: %s\n replica:\n type: s3\n bucket: %s\n path: %s\n endpoint: %s\n access-key-id: minioadmin\n secret-access-key: minioadmin\n region: us-east-1\n force-path-style: true\n skip-verify: true\n sync-interval: 1s\n`, dbPath, bucket, s3Path, proxyEndpoint)\n\n\tif err := os.WriteFile(configPath, []byte(config), 0644); err != nil {\n\t\tt.Fatalf(\"Failed to write config: %v\", err)\n\t}\n\tt.Logf(\"✓ Config written to: %s\", configPath)\n\tt.Log(\"\")\n\n\t// Start Litestream\n\tt.Log(\"Starting Litestream...\")\n\tlitestreamBin := filepath.Join(\"..\", \"..\", \"bin\", \"litestream\")\n\tcmd := exec.Command(litestreamBin, \"replicate\", \"-config\", configPath)\n\n\tvar stdout, stderr bytes.Buffer\n\tcmd.Stdout = io.MultiWriter(os.Stdout, &stdout)\n\tcmd.Stderr = io.MultiWriter(os.Stderr, &stderr)\n\n\tif err := cmd.Start(); err != nil {\n\t\tt.Fatalf(\"Failed to start Litestream: %v\", err)\n\t}\n\tt.Logf(\"✓ Litestream started (PID: %d)\", cmd.Process.Pid)\n\n\t// Wait for initial sync\n\tt.Log(\"Waiting for initial sync...\")\n\ttime.Sleep(3 * time.Second)\n\n\t// Write more data to ensure we have pending LTX files\n\tt.Log(\"Writing additional data...\")\n\tsqlDB, err = sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to reopen database: %v\", err)\n\t}\n\tfor i := 0; i < 5; i++ {\n\t\tif _, err := sqlDB.Exec(\"INSERT INTO test (data) VALUES (?)\", fmt.Sprintf(\"data-%d\", i)); err != nil {\n\t\t\tt.Fatalf(\"Failed to insert data: %v\", err)\n\t\t}\n\t}\n\tsqlDB.Close()\n\tt.Log(\"✓ Additional data written\")\n\n\t// Wait a bit for sync to pick up the changes\n\ttime.Sleep(2 * time.Second)\n\n\t// Reset proxy counter so 429s happen during shutdown\n\tproxy.Reset()\n\tt.Log(\"\")\n\tt.Log(\"Sending SIGTERM to trigger graceful shutdown...\")\n\tt.Log(\"(Proxy will return 429 for first 3 PUT requests)\")\n\n\t// Send SIGTERM\n\tif err := cmd.Process.Signal(syscall.SIGTERM); err != nil {\n\t\tt.Fatalf(\"Failed to send SIGTERM: %v\", err)\n\t}\n\n\t// Wait for process to exit with timeout\n\tdone := make(chan error, 1)\n\tgo func() {\n\t\tdone <- cmd.Wait()\n\t}()\n\n\tselect {\n\tcase err := <-done:\n\t\tif err != nil {\n\t\t\t// Check if it's just a signal exit (expected)\n\t\t\tif exitErr, ok := err.(*exec.ExitError); ok {\n\t\t\t\tt.Logf(\"Litestream exited with: %v\", exitErr)\n\t\t\t} else {\n\t\t\t\tt.Fatalf(\"Litestream failed: %v\", err)\n\t\t\t}\n\t\t}\n\tcase <-time.After(30 * time.Second):\n\t\tcmd.Process.Kill()\n\t\tt.Fatal(\"Litestream did not exit within 30 seconds\")\n\t}\n\n\tt.Log(\"\")\n\tt.Log(\"================================================\")\n\tt.Log(\"Results\")\n\tt.Log(\"================================================\")\n\n\t// Check proxy statistics\n\tstats := proxy.Stats()\n\tt.Logf(\"Proxy statistics:\")\n\tt.Logf(\" Total requests: %d\", stats.TotalRequests)\n\tt.Logf(\" 429 responses sent: %d\", stats.RateLimited)\n\tt.Logf(\" Forwarded requests: %d\", stats.Forwarded)\n\n\t// Verify that we saw 429s and retries succeeded\n\toutput := stdout.String() + stderr.String()\n\n\tif !strings.Contains(output, \"shutdown sync failed, retrying\") {\n\t\tt.Log(\"\")\n\t\tt.Log(\"WARNING: Did not see retry messages in output.\")\n\t\tt.Log(\"This could mean:\")\n\t\tt.Log(\" 1. No pending LTX files during shutdown\")\n\t\tt.Log(\" 2. Sync completed before shutdown signal\")\n\t\tt.Log(\" 3. Retry logic not triggered\")\n\t} else {\n\t\tt.Log(\"\")\n\t\tt.Log(\"✓ Saw retry messages - shutdown sync retry is working!\")\n\t}\n\n\tif strings.Contains(output, \"shutdown sync succeeded after retry\") {\n\t\tt.Log(\"✓ Shutdown sync succeeded after retrying!\")\n\t}\n\n\tif stats.RateLimited > 0 {\n\t\tt.Logf(\"✓ Proxy returned %d 429 responses as expected\", stats.RateLimited)\n\t}\n\n\tt.Log(\"\")\n\tt.Log(\"Test completed successfully!\")\n}\n\n// rateLimitingProxy is an HTTP proxy that returns 429 for the first N PUT requests\ntype rateLimitingProxy struct {\n\ttarget *url.URL\n\tproxy *httputil.ReverseProxy\n\tmu sync.Mutex\n\tputCount int32\n\tlimit int32\n\ttotalReqs int64\n\trateLimited int64\n\tforwarded int64\n\tt *testing.T\n}\n\ntype proxyStats struct {\n\tTotalRequests int64\n\tRateLimited int64\n\tForwarded int64\n}\n\nfunc newRateLimitingProxy(t *testing.T, targetURL string, limit int) *rateLimitingProxy {\n\ttarget, err := url.Parse(targetURL)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to parse target URL: %v\", err)\n\t}\n\n\tp := &rateLimitingProxy{\n\t\ttarget: target,\n\t\tlimit: int32(limit),\n\t\tt: t,\n\t}\n\n\tp.proxy = &httputil.ReverseProxy{\n\t\tDirector: func(req *http.Request) {\n\t\t\treq.URL.Scheme = target.Scheme\n\t\t\treq.URL.Host = target.Host\n\t\t\t// Don't modify Host header - it's part of the AWS signature\n\t\t},\n\t}\n\n\treturn p\n}\n\nfunc (p *rateLimitingProxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tatomic.AddInt64(&p.totalReqs, 1)\n\n\t// Only rate limit PUT requests (uploads)\n\tif r.Method == \"PUT\" {\n\t\tcount := atomic.AddInt32(&p.putCount, 1)\n\t\tif count <= p.limit {\n\t\t\tatomic.AddInt64(&p.rateLimited, 1)\n\t\t\tp.t.Logf(\"PROXY: Returning 429 for PUT request #%d (limit: %d)\", count, p.limit)\n\t\t\tw.Header().Set(\"Retry-After\", \"1\")\n\t\t\tw.WriteHeader(http.StatusTooManyRequests)\n\t\t\tw.Write([]byte(\"Rate limit exceeded\"))\n\t\t\treturn\n\t\t}\n\t}\n\n\tatomic.AddInt64(&p.forwarded, 1)\n\tp.proxy.ServeHTTP(w, r)\n}\n\nfunc (p *rateLimitingProxy) Reset() {\n\tatomic.StoreInt32(&p.putCount, 0)\n}\n\nfunc (p *rateLimitingProxy) Stats() proxyStats {\n\treturn proxyStats{\n\t\tTotalRequests: atomic.LoadInt64(&p.totalReqs),\n\t\tRateLimited: atomic.LoadInt64(&p.rateLimited),\n\t\tForwarded: atomic.LoadInt64(&p.forwarded),\n\t}\n}\n" }, { "path": "tests/integration/soak_helpers.go", "content": "//go:build integration && soak\n\npackage integration\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"os/signal\"\n\t\"path/filepath\"\n\t\"strconv\"\n\t\"strings\"\n\t\"syscall\"\n\t\"testing\"\n\t\"time\"\n)\n\n// S3Config holds S3-specific configuration\ntype S3Config struct {\n\tEndpoint string\n\tAccessKey string\n\tSecretKey string\n\tRegion string\n\tForcePathStyle bool\n\tSkipVerify bool\n\tSSE string\n\tSSEKMSKeyID string\n}\n\n// TestInfo holds test state for signal handler and monitoring\ntype TestInfo struct {\n\tStartTime time.Time\n\tDuration time.Duration\n\tRowCount int\n\tFileCount int\n\tDB *TestDB\n\tcancel context.CancelFunc\n}\n\n// ErrorStats holds error categorization and counts\ntype ErrorStats struct {\n\tTotalCount int\n\tCriticalCount int\n\tBenignCount int\n\tRecentErrors []string\n\tErrorsByType map[string]int\n}\n\nfunc isInteractive() bool {\n\tif fi, err := os.Stdin.Stat(); err == nil {\n\t\treturn fi.Mode()&os.ModeCharDevice != 0\n\t}\n\treturn false\n}\n\nfunc promptYesNo(t *testing.T, prompt string, defaultYes bool) bool {\n\tt.Helper()\n\n\tswitch strings.ToLower(strings.TrimSpace(os.Getenv(\"SOAK_AUTO_PURGE\"))) {\n\tcase \"y\", \"yes\", \"true\", \"1\", \"on\":\n\t\tt.Logf(\"%s yes (SOAK_AUTO_PURGE)\", prompt)\n\t\treturn true\n\tcase \"n\", \"no\", \"false\", \"0\", \"off\":\n\t\tt.Logf(\"%s no (SOAK_AUTO_PURGE)\", prompt)\n\t\treturn false\n\t}\n\n\tif !isInteractive() {\n\t\tif defaultYes {\n\t\t\tt.Logf(\"%s yes (non-interactive default)\", prompt)\n\t\t\treturn true\n\t\t}\n\t\tt.Logf(\"%s no (non-interactive default)\", prompt)\n\t\treturn false\n\t}\n\n\tdefPrompt := \"[y/N]\"\n\tif defaultYes {\n\t\tdefPrompt = \"[Y/n]\"\n\t}\n\n\tfmt.Printf(\"%s %s \", prompt, defPrompt)\n\treader := bufio.NewReader(os.Stdin)\n\ttext, err := reader.ReadString('\\n')\n\tif err != nil {\n\t\tt.Logf(\"Failed to read response: %v (defaulting to no)\", err)\n\t\treturn false\n\t}\n\n\tswitch strings.ToLower(strings.TrimSpace(text)) {\n\tcase \"\", \"y\", \"yes\":\n\t\tif defaultYes || text != \"\" {\n\t\t\treturn true\n\t\t}\n\t\treturn false\n\tcase \"n\", \"no\":\n\t\treturn false\n\tdefault:\n\t\treturn defaultYes\n\t}\n}\n\nfunc promptYesNoDefaultNo(t *testing.T, prompt string) bool {\n\treturn promptYesNo(t, prompt, false)\n}\n\nfunc promptYesNoDefaultYes(t *testing.T, prompt string) bool {\n\treturn promptYesNo(t, prompt, true)\n}\n\n// StartMinIOContainer starts a MinIO container and returns the container ID and endpoint\nfunc StartMinIOContainer(t *testing.T) (containerID string, endpoint string, volumeName string) {\n\tt.Helper()\n\n\tcontainerName := fmt.Sprintf(\"litestream-test-minio-%d\", time.Now().Unix())\n\tvolumeName = fmt.Sprintf(\"litestream-test-minio-data-%d\", time.Now().Unix())\n\tminioPort := \"9100\"\n\tconsolePort := \"9101\"\n\n\t// Clean up any existing container\n\texec.Command(\"docker\", \"stop\", containerName).Run()\n\texec.Command(\"docker\", \"rm\", containerName).Run()\n\n\t// Remove any lingering volume with the same name, then create fresh volume.\n\texec.Command(\"docker\", \"volume\", \"rm\", volumeName).Run()\n\tif out, err := exec.Command(\"docker\", \"volume\", \"create\", volumeName).CombinedOutput(); err != nil {\n\t\tt.Fatalf(\"Failed to create MinIO volume: %v\\nOutput: %s\", err, string(out))\n\t}\n\n\t// Start MinIO container\n\tcmd := exec.Command(\"docker\", \"run\", \"-d\",\n\t\t\"--name\", containerName,\n\t\t\"-p\", minioPort+\":9000\",\n\t\t\"-p\", consolePort+\":9001\",\n\t\t\"-v\", volumeName+\":/data\",\n\t\t\"-e\", \"MINIO_ROOT_USER=minioadmin\",\n\t\t\"-e\", \"MINIO_ROOT_PASSWORD=minioadmin\",\n\t\t\"minio/minio\", \"server\", \"/data\", \"--console-address\", \":9001\")\n\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to start MinIO container: %v\\nOutput: %s\", err, string(output))\n\t}\n\n\tcontainerID = strings.TrimSpace(string(output))\n\tendpoint = fmt.Sprintf(\"http://localhost:%s\", minioPort)\n\n\t// Wait for MinIO to be ready\n\ttime.Sleep(5 * time.Second)\n\n\t// Verify container is running\n\tcmd = exec.Command(\"docker\", \"ps\", \"-q\", \"-f\", \"name=\"+containerName)\n\toutput, err = cmd.CombinedOutput()\n\tif err != nil || len(strings.TrimSpace(string(output))) == 0 {\n\t\tt.Fatalf(\"MinIO container failed to start properly\")\n\t}\n\n\tt.Logf(\"MinIO container started: %s (endpoint: %s)\", containerID[:12], endpoint)\n\n\treturn containerID, endpoint, volumeName\n}\n\n// StopMinIOContainer stops and removes a MinIO container\nfunc StopMinIOContainer(t *testing.T, containerID string, volumeName string) {\n\tt.Helper()\n\n\tif containerID == \"\" {\n\t\treturn\n\t}\n\n\tt.Logf(\"Stopping MinIO container: %s\", containerID[:12])\n\n\texec.Command(\"docker\", \"stop\", containerID).Run()\n\texec.Command(\"docker\", \"rm\", containerID).Run()\n\n\tif volumeName != \"\" {\n\t\texec.Command(\"docker\", \"volume\", \"rm\", volumeName).Run()\n\t}\n}\n\n// CreateMinIOBucket creates a bucket in MinIO\nfunc CreateMinIOBucket(t *testing.T, containerID, bucket string) {\n\tt.Helper()\n\n\tif minioBucketExists(containerID, bucket) {\n\t\tif promptYesNoDefaultYes(t, fmt.Sprintf(\"Bucket '%s' already exists. Purge existing objects before running soak test?\", bucket)) {\n\t\t\tt.Logf(\"Purging MinIO bucket '%s'...\", bucket)\n\t\t\tif err := clearMinIOBucket(containerID, bucket); err != nil {\n\t\t\t\tt.Fatalf(\"Failed to purge MinIO bucket: %v\", err)\n\t\t\t}\n\t\t} else {\n\t\t\tt.Logf(\"Skipping purge of bucket '%s'. Residual data may cause replication errors.\", bucket)\n\t\t}\n\t}\n\n\t// Use mc (MinIO Client) via docker to create bucket\n\tcmd := exec.Command(\"docker\", \"run\", \"--rm\",\n\t\t\"--link\", containerID+\":minio\",\n\t\t\"-e\", \"MC_HOST_minio=http://minioadmin:minioadmin@minio:9000\",\n\t\t\"minio/mc\", \"mb\", \"minio/\"+bucket)\n\n\t_, stdoutBuf, stderrBuf := configureCmdIO(cmd)\n\tif err := cmd.Run(); err != nil {\n\t\toutput := combinedOutput(stdoutBuf, stderrBuf)\n\t\tif !strings.Contains(output, \"already exists\") {\n\t\t\tt.Fatalf(\"Create bucket failed: %v Output: %s\", err, output)\n\t\t}\n\t}\n\n\tif err := waitForMinIOBucket(containerID, bucket, 60*time.Second); err != nil {\n\t\tt.Fatalf(\"Bucket %s not ready: %v\", bucket, err)\n\t}\n\n\tif err := clearMinIOBucket(containerID, bucket); err != nil {\n\t\tt.Fatalf(\"Failed to purge MinIO bucket: %v\", err)\n\t}\n\n\tt.Logf(\"MinIO bucket '%s' ready\", bucket)\n}\n\nfunc minioBucketExists(containerID, bucket string) bool {\n\tcmd := exec.Command(\"docker\", \"run\", \"--rm\",\n\t\t\"--link\", containerID+\":minio\",\n\t\t\"-e\", \"MC_HOST_minio=http://minioadmin:minioadmin@minio:9000\",\n\t\t\"minio/mc\", \"ls\", \"minio/\"+bucket+\"/\")\n\t_, _, _ = configureCmdIO(cmd)\n\tif err := cmd.Run(); err != nil {\n\t\treturn false\n\t}\n\treturn true\n}\n\nfunc clearMinIOBucket(containerID, bucket string) error {\n\tcmd := exec.Command(\"docker\", \"run\", \"--rm\",\n\t\t\"--link\", containerID+\":minio\",\n\t\t\"-e\", \"MC_HOST_minio=http://minioadmin:minioadmin@minio:9000\",\n\t\t\"minio/mc\", \"rm\", \"--recursive\", \"--force\", \"minio/\"+bucket)\n\t_, stdoutBuf, stderrBuf := configureCmdIO(cmd)\n\tif err := cmd.Run(); err != nil {\n\t\toutput := combinedOutput(stdoutBuf, stderrBuf)\n\t\tif output != \"\" {\n\t\t\treturn fmt.Errorf(\"%w: %s\", err, output)\n\t\t}\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc waitForMinIOBucket(containerID, bucket string, timeout time.Duration) error {\n\tdeadline := time.Now().Add(timeout)\n\tfor {\n\t\tif minioBucketExists(containerID, bucket) {\n\t\t\treturn nil\n\t\t}\n\t\tif time.Now().After(deadline) {\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(500 * time.Millisecond)\n\t}\n\treturn fmt.Errorf(\"bucket %s not available\", bucket)\n}\n\n// CountMinIOObjects counts objects in a MinIO bucket\nfunc CountMinIOObjects(t *testing.T, containerID, bucket string) int {\n\tt.Helper()\n\n\tcmd := exec.Command(\"docker\", \"run\", \"--rm\",\n\t\t\"--link\", containerID+\":minio\",\n\t\t\"-e\", \"MC_HOST_minio=http://minioadmin:minioadmin@minio:9000\",\n\t\t\"minio/mc\", \"ls\", \"minio/\"+bucket+\"/\", \"--recursive\")\n\n\t_, stdoutBuf, stderrBuf := configureCmdIO(cmd)\n\tif err := cmd.Run(); err != nil {\n\t\treturn 0\n\t}\n\n\toutput := combinedOutput(stdoutBuf, stderrBuf)\n\tlines := strings.Split(strings.TrimSpace(output), \"\\n\")\n\tif len(lines) == 1 && lines[0] == \"\" {\n\t\treturn 0\n\t}\n\n\treturn len(lines)\n}\n\n// CheckAWSCredentials checks if AWS credentials are set and returns bucket and region\nfunc CheckAWSCredentials(t *testing.T) (bucket, region string) {\n\tt.Helper()\n\n\taccessKey := os.Getenv(\"AWS_ACCESS_KEY_ID\")\n\tsecretKey := os.Getenv(\"AWS_SECRET_ACCESS_KEY\")\n\tbucket = os.Getenv(\"S3_BUCKET\")\n\tregion = os.Getenv(\"AWS_REGION\")\n\n\tif accessKey == \"\" || secretKey == \"\" || bucket == \"\" {\n\t\tt.Skip(\"AWS credentials not set. Set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and S3_BUCKET\")\n\t}\n\n\tif region == \"\" {\n\t\tregion = \"us-east-1\"\n\t}\n\n\tt.Logf(\"Using AWS S3: bucket=%s, region=%s\", bucket, region)\n\n\treturn bucket, region\n}\n\n// TestS3Connectivity tests if we can access the S3 bucket\nfunc TestS3Connectivity(t *testing.T, bucket string) {\n\tt.Helper()\n\n\tcmd := exec.Command(\"aws\", \"s3\", \"ls\", \"s3://\"+bucket+\"/\")\n\tif err := cmd.Run(); err != nil {\n\t\tt.Fatalf(\"Failed to access S3 bucket '%s': %v\\nEnsure AWS CLI is installed and credentials are valid\", bucket, err)\n\t}\n\n\tt.Logf(\"✓ S3 bucket '%s' is accessible\", bucket)\n}\n\n// CountS3Objects counts objects in an S3 path\nfunc CountS3Objects(t *testing.T, s3URL string) int {\n\tt.Helper()\n\n\tcmd := exec.Command(\"aws\", \"s3\", \"ls\", s3URL+\"/\", \"--recursive\")\n\t_, stdoutBuf, stderrBuf := configureCmdIO(cmd)\n\tif err := cmd.Run(); err != nil {\n\t\treturn 0\n\t}\n\n\toutput := combinedOutput(stdoutBuf, stderrBuf)\n\tlines := strings.Split(strings.TrimSpace(output), \"\\n\")\n\tif len(lines) == 1 && lines[0] == \"\" {\n\t\treturn 0\n\t}\n\n\treturn len(lines)\n}\n\n// GetS3StorageSize gets the total storage size of an S3 path\nfunc GetS3StorageSize(t *testing.T, s3URL string) int64 {\n\tt.Helper()\n\n\tcmd := exec.Command(\"aws\", \"s3\", \"ls\", s3URL+\"/\", \"--recursive\", \"--summarize\")\n\t_, stdoutBuf, stderrBuf := configureCmdIO(cmd)\n\tif err := cmd.Run(); err != nil {\n\t\treturn 0\n\t}\n\n\toutput := combinedOutput(stdoutBuf, stderrBuf)\n\tlines := strings.Split(output, \"\\n\")\n\tfor _, line := range lines {\n\t\tif strings.Contains(line, \"Total Size:\") {\n\t\t\tvar size int64\n\t\t\tfmt.Sscanf(line, \"Total Size: %d\", &size)\n\t\t\treturn size\n\t\t}\n\t}\n\n\treturn 0\n}\n\n// CreateSoakConfig creates a litestream configuration file for soak tests\nfunc CreateSoakConfig(dbPath, replicaURL string, s3Config *S3Config, shortMode bool) string {\n\ttempDir := filepath.Dir(dbPath)\n\tconfigPath := filepath.Join(tempDir, \"litestream.yml\")\n\n\tvar config strings.Builder\n\n\tsnapshotInterval := \"10m\"\n\tsnapshotRetention := \"1h\"\n\tretentionCheckInterval := \"5m\"\n\tlevelIntervals := []string{\"30s\", \"1m\", \"5m\", \"15m\", \"30m\"}\n\n\tif shortMode {\n\t\tsnapshotInterval = \"30s\"\n\t\tsnapshotRetention = \"10m\"\n\t\tretentionCheckInterval = \"2m\"\n\t\tlevelIntervals = []string{\"15s\", \"30s\", \"1m\"}\n\t}\n\n\t// Add S3 credentials if provided\n\tif s3Config != nil && s3Config.AccessKey != \"\" {\n\t\tconfig.WriteString(fmt.Sprintf(\"access-key-id: %s\\n\", s3Config.AccessKey))\n\t\tconfig.WriteString(fmt.Sprintf(\"secret-access-key: %s\\n\", s3Config.SecretKey))\n\t\tconfig.WriteString(\"\\n\")\n\t}\n\n\t// Aggressive snapshot settings for testing\n\tconfig.WriteString(\"snapshot:\\n\")\n\tconfig.WriteString(fmt.Sprintf(\" interval: %s\\n\", snapshotInterval))\n\tconfig.WriteString(fmt.Sprintf(\" retention: %s\\n\", snapshotRetention))\n\tconfig.WriteString(\"\\n\")\n\n\t// Aggressive compaction levels\n\tconfig.WriteString(\"levels:\\n\")\n\tfor _, interval := range levelIntervals {\n\t\tconfig.WriteString(fmt.Sprintf(\" - interval: %s\\n\", interval))\n\t}\n\tconfig.WriteString(\"\\n\")\n\n\t// Database configuration\n\tconfig.WriteString(\"dbs:\\n\")\n\tconfig.WriteString(fmt.Sprintf(\" - path: %s\\n\", filepath.ToSlash(dbPath)))\n\tconfig.WriteString(\" checkpoint-interval: 1m\\n\")\n\tconfig.WriteString(\" min-checkpoint-page-count: 100\\n\")\n\tconfig.WriteString(\" truncate-page-n: 5000\\n\")\n\tconfig.WriteString(\"\\n\")\n\tconfig.WriteString(\" replica:\\n\")\n\tconfig.WriteString(fmt.Sprintf(\" url: %s\\n\", replicaURL))\n\n\t// Add S3-specific settings if provided\n\tif s3Config != nil {\n\t\tif s3Config.Endpoint != \"\" {\n\t\t\tconfig.WriteString(fmt.Sprintf(\" endpoint: %s\\n\", s3Config.Endpoint))\n\t\t}\n\t\tif s3Config.Region != \"\" {\n\t\t\tconfig.WriteString(fmt.Sprintf(\" region: %s\\n\", s3Config.Region))\n\t\t}\n\t\tif s3Config.ForcePathStyle {\n\t\t\tconfig.WriteString(\" force-path-style: true\\n\")\n\t\t}\n\t\tif s3Config.SkipVerify {\n\t\t\tconfig.WriteString(\" skip-verify: true\\n\")\n\t\t}\n\t\tif s3Config.SSE != \"\" {\n\t\t\tconfig.WriteString(fmt.Sprintf(\" sse: %s\\n\", s3Config.SSE))\n\t\t}\n\t\tif s3Config.SSEKMSKeyID != \"\" {\n\t\t\tconfig.WriteString(fmt.Sprintf(\" sse-kms-key-id: %s\\n\", s3Config.SSEKMSKeyID))\n\t\t}\n\t\tconfig.WriteString(fmt.Sprintf(\" retention-check-interval: %s\\n\", retentionCheckInterval))\n\t}\n\n\tif err := os.WriteFile(configPath, []byte(config.String()), 0644); err != nil {\n\t\tpanic(fmt.Sprintf(\"Failed to create config file: %v\", err))\n\t}\n\n\treturn configPath\n}\n\n// setupSignalHandler sets up SIGINT/SIGTERM handler with confirmation\nfunc setupSignalHandler(t *testing.T, cancel context.CancelFunc, testInfo *TestInfo) {\n\tt.Helper()\n\n\tsigChan := make(chan os.Signal, 1)\n\tsignal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)\n\n\tgo func() {\n\t\tfirstInterrupt := true\n\n\t\tfor sig := range sigChan {\n\t\t\tif firstInterrupt {\n\t\t\t\tfirstInterrupt = false\n\n\t\t\t\tt.Logf(\"\")\n\t\t\t\tt.Logf(\"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\")\n\t\t\t\tt.Logf(\"⚠ Interrupt signal received (%v)\", sig)\n\t\t\t\tt.Logf(\"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\")\n\t\t\t\tt.Logf(\"\")\n\n\t\t\t\telapsed := time.Since(testInfo.StartTime)\n\t\t\t\tremaining := testInfo.Duration - elapsed\n\t\t\t\tpct := float64(elapsed) / float64(testInfo.Duration) * 100\n\n\t\t\t\tt.Logf(\"Test Progress:\")\n\t\t\t\tt.Logf(\" Elapsed: %v (%.0f%% complete)\", elapsed.Round(time.Second), pct)\n\t\t\t\tt.Logf(\" Remaining: %v\", remaining.Round(time.Second))\n\t\t\t\tt.Logf(\" Data collected: %d rows, %d replica files\", testInfo.RowCount, testInfo.FileCount)\n\t\t\t\tt.Logf(\"\")\n\t\t\t\tt.Logf(\"Press Ctrl+C again within 5 seconds to confirm shutdown.\")\n\t\t\t\tt.Logf(\"Otherwise, test will continue...\")\n\t\t\t\tt.Logf(\"\")\n\n\t\t\t\t// Wait 5 seconds for second interrupt\n\t\t\t\ttimeout := time.NewTimer(5 * time.Second)\n\t\t\t\tselect {\n\t\t\t\tcase <-sigChan:\n\t\t\t\t\t// Second interrupt - confirmed shutdown\n\t\t\t\t\ttimeout.Stop()\n\t\t\t\t\tt.Logf(\"Shutdown confirmed. Initiating graceful cleanup...\")\n\t\t\t\t\tcancel() // Cancel context to stop test\n\t\t\t\t\tperformGracefulShutdown(t, testInfo)\n\t\t\t\t\treturn\n\n\t\t\t\tcase <-timeout.C:\n\t\t\t\t\t// Timeout - continue test\n\t\t\t\t\tt.Logf(\"No confirmation received. Continuing test...\")\n\t\t\t\t\tt.Logf(\"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\")\n\t\t\t\t\tt.Logf(\"\")\n\t\t\t\t\tfirstInterrupt = true\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\t// Second interrupt received\n\t\t\t\tt.Logf(\"Shutdown confirmed. Initiating graceful cleanup...\")\n\t\t\t\tcancel()\n\t\t\t\tperformGracefulShutdown(t, testInfo)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\tt.Cleanup(func() {\n\t\tsignal.Stop(sigChan)\n\t\tclose(sigChan)\n\t})\n}\n\n// performGracefulShutdown performs cleanup on early termination\nfunc performGracefulShutdown(t *testing.T, testInfo *TestInfo) {\n\tt.Helper()\n\n\tif testInfo.cancel != nil {\n\t\ttestInfo.cancel()\n\t}\n\n\tt.Log(\"\")\n\tt.Log(\"================================================\")\n\tt.Log(\"Graceful Shutdown - Early Termination\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\n\telapsed := time.Since(testInfo.StartTime)\n\n\t// Stop Litestream gracefully\n\tt.Log(\"Stopping Litestream...\")\n\tif err := testInfo.DB.StopLitestream(); err != nil {\n\t\tt.Logf(\"Warning: Error stopping Litestream: %v\", err)\n\t} else {\n\t\tt.Log(\"✓ Litestream stopped\")\n\t}\n\n\t// Wait for pending operations\n\tt.Log(\"Waiting for pending operations to complete...\")\n\ttime.Sleep(2 * time.Second)\n\n\t// Show partial results\n\tt.Log(\"\")\n\tt.Log(\"Partial Test Results:\")\n\tt.Logf(\" Test duration: %v (%.0f%% of planned %v)\",\n\t\telapsed.Round(time.Second),\n\t\tfloat64(elapsed)/float64(testInfo.Duration)*100,\n\t\ttestInfo.Duration.Round(time.Minute))\n\n\tif dbSize, err := testInfo.DB.GetDatabaseSize(); err == nil {\n\t\tt.Logf(\" Database size: %.2f MB\", float64(dbSize)/(1024*1024))\n\t}\n\n\tif rowCount, err := testInfo.DB.GetRowCount(\"load_test\"); err == nil {\n\t\tt.Logf(\" Rows inserted: %d\", rowCount)\n\t\tif elapsed.Seconds() > 0 {\n\t\t\trate := float64(rowCount) / elapsed.Seconds()\n\t\t\tt.Logf(\" Average write rate: %.1f rows/second\", rate)\n\t\t}\n\t}\n\n\tif fileCount, err := testInfo.DB.GetReplicaFileCount(); err == nil {\n\t\tt.Logf(\" Replica LTX files: %d\", fileCount)\n\t}\n\n\t// Run abbreviated analysis\n\tt.Log(\"\")\n\tt.Log(\"Analyzing partial test data...\")\n\tanalysis := AnalyzeSoakTest(t, testInfo.DB, elapsed)\n\n\tt.Log(\"\")\n\tt.Log(\"What Was Validated (Partial):\")\n\tif analysis.SnapshotCount > 0 {\n\t\tt.Logf(\" ✓ Snapshots: %d generated\", analysis.SnapshotCount)\n\t}\n\tif analysis.TotalCompactions > 0 {\n\t\tt.Logf(\" ✓ Compactions: %d completed\", analysis.TotalCompactions)\n\t}\n\tif analysis.DatabaseRows > 0 {\n\t\tt.Logf(\" ✓ Data written: %d rows\", analysis.DatabaseRows)\n\t}\n\n\t// Check for errors\n\terrors, _ := testInfo.DB.CheckForErrors()\n\tt.Logf(\" Critical errors: %d\", len(errors))\n\n\t// Show where data is preserved\n\tt.Log(\"\")\n\tt.Log(\"Test artifacts preserved at:\")\n\tt.Logf(\" %s\", testInfo.DB.TempDir)\n\n\tif logPath, err := testInfo.DB.GetLitestreamLog(); err == nil {\n\t\tt.Logf(\" Log: %s\", logPath)\n\t}\n\n\tt.Log(\"\")\n\tt.Log(\"Test terminated early by user.\")\n\tt.Log(\"================================================\")\n\n\t// Mark test as failed (early termination)\n\tt.Fail()\n}\n\n// getErrorStats categorizes and counts errors\nfunc getErrorStats(db *TestDB) ErrorStats {\n\terrors, _ := db.CheckForErrors()\n\tstats := ErrorStats{\n\t\tTotalCount: len(errors),\n\t\tErrorsByType: make(map[string]int),\n\t}\n\n\tfor _, errLine := range errors {\n\t\tswitch {\n\t\tcase strings.Contains(errLine, \"connection refused\"):\n\t\t\tstats.BenignCount++\n\t\t\tstats.ErrorsByType[\"connection refused\"]++\n\t\tcase strings.Contains(errLine, \"context canceled\"):\n\t\t\tstats.BenignCount++\n\t\t\tstats.ErrorsByType[\"context canceled\"]++\n\t\tdefault:\n\t\t\tstats.CriticalCount++\n\t\t\tif len(stats.RecentErrors) < 5 {\n\t\t\t\tstats.RecentErrors = append(stats.RecentErrors, errLine)\n\t\t\t}\n\n\t\t\tswitch {\n\t\t\tcase strings.Contains(errLine, \"timeout\"):\n\t\t\t\tstats.ErrorsByType[\"timeout\"]++\n\t\t\tcase strings.Contains(errLine, \"compaction failed\"):\n\t\t\t\tstats.ErrorsByType[\"compaction failed\"]++\n\t\t\tdefault:\n\t\t\t\tstats.ErrorsByType[\"other\"]++\n\t\t\t}\n\t\t}\n\t}\n\n\treturn stats\n}\n\n// printProgress displays progress bar with error status\nfunc printProgress(t *testing.T, elapsed, total time.Duration, errorStats ErrorStats) {\n\tt.Helper()\n\n\tif total <= 0 {\n\t\ttotal = time.Second\n\t}\n\n\tif elapsed < 0 {\n\t\telapsed = 0\n\t}\n\n\tpct := float64(elapsed) / float64(total) * 100\n\tif pct > 100 {\n\t\tpct = 100\n\t} else if pct < 0 {\n\t\tpct = 0\n\t}\n\n\tremaining := total - elapsed\n\tif remaining < 0 {\n\t\tremaining = 0\n\t}\n\n\t// Progress bar\n\tbarWidth := 40\n\tfilled := 0\n\tif total.Seconds() > 0 {\n\t\tratio := elapsed.Seconds() / total.Seconds()\n\t\tif ratio < 0 {\n\t\t\tratio = 0\n\t\t} else if ratio > 1 {\n\t\t\tratio = 1\n\t\t}\n\t\tfilled = int(float64(barWidth) * ratio)\n\t}\n\tif filled > barWidth {\n\t\tfilled = barWidth\n\t}\n\tif filled < 0 {\n\t\tfilled = 0\n\t}\n\tbar := strings.Repeat(\"█\", filled) + strings.Repeat(\"░\", barWidth-filled)\n\n\t// Status indicator\n\tstatus := \"✓\"\n\tif errorStats.CriticalCount > 0 {\n\t\tstatus = \"⚠\"\n\t}\n\n\tt.Logf(\"%s Progress: [%s] %.0f%% | %v elapsed | %v remaining | Errors: %d/%d\",\n\t\tstatus, bar, pct,\n\t\telapsed.Round(time.Minute), remaining.Round(time.Minute),\n\t\terrorStats.CriticalCount, errorStats.TotalCount)\n}\n\n// printErrorDetails displays detailed error information\nfunc printErrorDetails(t *testing.T, errorStats ErrorStats) {\n\tt.Helper()\n\n\tt.Log(\"\")\n\tt.Log(\"⚠ Error Status:\")\n\tt.Logf(\" Total: %d (%d critical, %d benign)\", errorStats.TotalCount, errorStats.CriticalCount, errorStats.BenignCount)\n\n\t// Group critical errors by type\n\tif errorStats.CriticalCount > 0 {\n\t\tt.Log(\" Critical errors:\")\n\t\tfor errorType, count := range errorStats.ErrorsByType {\n\t\t\tif count > 0 {\n\t\t\t\tt.Logf(\" • %q (%d)\", errorType, count)\n\t\t\t}\n\t\t}\n\n\t\t// Show recent errors\n\t\tif len(errorStats.RecentErrors) > 0 {\n\t\t\tt.Log(\"\")\n\t\t\tt.Log(\" Recent errors:\")\n\t\t\tfor _, errLine := range errorStats.RecentErrors {\n\t\t\t\t// Extract just the error message\n\t\t\t\tif idx := strings.Index(errLine, \"error=\"); idx != -1 {\n\t\t\t\t\tmsg := errLine[idx+7:]\n\t\t\t\t\tif len(msg) > 80 {\n\t\t\t\t\t\tmsg = msg[:80] + \"...\"\n\t\t\t\t\t}\n\t\t\t\t\tt.Logf(\" %s\", msg)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// Show benign errors if present\n\tif errorStats.BenignCount > 0 {\n\t\tt.Log(\"\")\n\t\tt.Logf(\" Benign errors: %d\", errorStats.BenignCount)\n\t}\n}\n\n// shouldAbortTest checks if test should auto-abort due to critical issues\nfunc shouldAbortTest(errorStats ErrorStats, fileCount int, elapsed time.Duration) (bool, string) {\n\t// Abort if critical error threshold exceeded after extended runtime\n\tif elapsed > 10*time.Minute && errorStats.CriticalCount > 100 {\n\t\treturn true, fmt.Sprintf(\"Critical error threshold exceeded (%d errors)\", errorStats.CriticalCount)\n\t}\n\n\t// Abort if replication completely stopped (0 files after 10 minutes)\n\tif elapsed > 10*time.Minute && fileCount == 0 {\n\t\treturn true, \"Replication not working (0 files created after 10 minutes)\"\n\t}\n\n\t// Abort if error rate is increasing rapidly (>1 error/minute)\n\tif errorStats.CriticalCount > 0 && elapsed > 30*time.Minute {\n\t\tminutes := elapsed.Minutes()\n\t\tif minutes > 0 {\n\t\t\terrorRate := float64(errorStats.CriticalCount) / minutes\n\t\t\tif errorRate > 2.0 {\n\t\t\t\treturn true, fmt.Sprintf(\"Error rate too high (%.1f errors/minute)\", errorRate)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn false, \"\"\n}\n\n// MonitorSoakTest monitors a soak test, calling metricsFunc every 60 seconds\nfunc MonitorSoakTest(t *testing.T, db *TestDB, ctx context.Context, info *TestInfo, refresh func(), logFunc func()) {\n\tt.Helper()\n\n\tif info == nil {\n\t\tinfo = &TestInfo{}\n\t}\n\n\tticker := time.NewTicker(60 * time.Second)\n\tdefer ticker.Stop()\n\n\tlastCritical := -1\n\tlastTotal := -1\n\tlastProgress := -1.0\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\tif refresh != nil {\n\t\t\t\trefresh()\n\t\t\t}\n\t\t\tif info != nil {\n\t\t\t\t// Show final progress snapshot\n\t\t\t\terrorStats := getErrorStats(db)\n\t\t\t\tif lastProgress < 0 || lastProgress < 100 || errorStats.CriticalCount != lastCritical || errorStats.TotalCount != lastTotal {\n\t\t\t\t\tprintProgress(t, info.Duration, info.Duration, errorStats)\n\t\t\t\t\tt.Logf(\"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\")\n\t\t\t\t\tt.Logf(\"[%s] Status Report\", time.Now().Format(\"15:04:05\"))\n\t\t\t\t\tt.Logf(\"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\")\n\t\t\t\t\tif logFunc != nil {\n\t\t\t\t\t\tlogFunc()\n\t\t\t\t\t}\n\t\t\t\t\tif errorStats.CriticalCount > 0 {\n\t\t\t\t\t\tprintErrorDetails(t, errorStats)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tt.Log(\"Monitoring stopped: test duration completed\")\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t\tif refresh != nil {\n\t\t\t\trefresh()\n\t\t\t}\n\n\t\t\telapsed := time.Since(info.StartTime)\n\t\t\tif elapsed < 0 {\n\t\t\t\telapsed = 0\n\t\t\t}\n\n\t\t\terrorStats := getErrorStats(db)\n\n\t\t\tif shouldAbort, reason := shouldAbortTest(errorStats, info.FileCount, elapsed); shouldAbort {\n\t\t\t\tt.Logf(\"\")\n\t\t\t\tt.Logf(\"⚠ AUTO-ABORTING TEST: %s\", reason)\n\t\t\t\tif info.cancel != nil {\n\t\t\t\t\tinfo.cancel()\n\t\t\t\t}\n\t\t\t\tt.Fail()\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\ttotalDuration := info.Duration\n\t\t\tif totalDuration <= 0 {\n\t\t\t\ttotalDuration = time.Second\n\t\t\t}\n\t\t\tprogress := elapsed.Seconds() / totalDuration.Seconds() * 100\n\t\t\tif progress < 0 {\n\t\t\t\tprogress = 0\n\t\t\t} else if progress > 100 {\n\t\t\t\tprogress = 100\n\t\t\t}\n\n\t\t\tshouldLog := false\n\t\t\tif lastCritical == -1 && lastTotal == -1 {\n\t\t\t\tshouldLog = true\n\t\t\t}\n\n\t\t\tif !shouldLog && (errorStats.CriticalCount != lastCritical || errorStats.TotalCount != lastTotal) {\n\t\t\t\tshouldLog = true\n\t\t\t}\n\n\t\t\tif !shouldLog && (lastProgress < 0 || progress >= lastProgress+5 || progress >= 100) {\n\t\t\t\tshouldLog = true\n\t\t\t}\n\n\t\t\tif !shouldLog {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tlastCritical = errorStats.CriticalCount\n\t\t\tlastTotal = errorStats.TotalCount\n\t\t\tlastProgress = progress\n\n\t\t\tprintProgress(t, elapsed, info.Duration, errorStats)\n\t\t\tt.Logf(\"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\")\n\t\t\tt.Logf(\"[%s] Status Report\", time.Now().Format(\"15:04:05\"))\n\t\t\tt.Logf(\"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\")\n\n\t\t\tif logFunc != nil {\n\t\t\t\tlogFunc()\n\t\t\t}\n\n\t\t\tif errorStats.CriticalCount > 0 {\n\t\t\t\tprintErrorDetails(t, errorStats)\n\t\t\t}\n\n\t\t\tt.Log(\"\")\n\t\t}\n\t}\n}\n\n// LogSoakMetrics logs basic soak test metrics\nfunc LogSoakMetrics(t *testing.T, db *TestDB, testName string) {\n\tt.Helper()\n\n\t// Database size\n\tif dbSize, err := db.GetDatabaseSize(); err == nil {\n\t\tt.Logf(\" Database size: %.2f MB\", float64(dbSize)/(1024*1024))\n\t}\n\n\t// WAL size\n\twalPath := db.Path + \"-wal\"\n\tif info, err := os.Stat(walPath); err == nil {\n\t\tt.Logf(\" WAL size: %.2f MB\", float64(info.Size())/(1024*1024))\n\t}\n\n\t// Row count\n\tif count, err := db.GetRowCount(\"load_test\"); err == nil {\n\t\tt.Logf(\" Rows: %d\", count)\n\t} else if count, err := db.GetRowCount(\"test_table_0\"); err == nil {\n\t\tt.Logf(\" Rows: %d\", count)\n\t}\n\n\t// Replica stats\n\tif fileCount, err := db.GetReplicaFileCount(); err == nil {\n\t\tt.Logf(\" Replica LTX files: %d\", fileCount)\n\t}\n\n\t// Error check\n\tif errors, err := db.CheckForErrors(); err == nil && len(errors) > 0 {\n\t\tt.Logf(\" ⚠ Critical errors detected: %d\", len(errors))\n\t\tif len(errors) <= 2 {\n\t\t\tfor _, errLine := range errors {\n\t\t\t\tt.Logf(\" %s\", errLine)\n\t\t\t}\n\t\t}\n\t}\n}\n\n// SoakTestAnalysis holds detailed soak test metrics\ntype SoakTestAnalysis struct {\n\tCompactionsByLevel map[int]int\n\tTotalCompactions int\n\tSnapshotCount int\n\tCheckpointCount int\n\tTotalFilesCreated int\n\tFinalFileCount int\n\tMinTxID string\n\tMaxTxID string\n\tDatabaseRows int64\n\tMinRowID int64\n\tMaxRowID int64\n\tDatabaseSizeMB float64\n\tDuration time.Duration\n}\n\n// AnalyzeSoakTest analyzes test results from logs and database\nfunc AnalyzeSoakTest(t *testing.T, db *TestDB, duration time.Duration) *SoakTestAnalysis {\n\tt.Helper()\n\n\tanalysis := &SoakTestAnalysis{\n\t\tCompactionsByLevel: make(map[int]int),\n\t\tDuration: duration,\n\t}\n\n\t// Get database stats\n\tif count, err := db.GetRowCount(\"load_test\"); err == nil {\n\t\tanalysis.DatabaseRows = int64(count)\n\t}\n\n\tif dbSize, err := db.GetDatabaseSize(); err == nil {\n\t\tanalysis.DatabaseSizeMB = float64(dbSize) / (1024 * 1024)\n\t}\n\n\t// Get row ID range\n\tsqlDB, err := sql.Open(\"sqlite3\", db.Path)\n\tif err == nil {\n\t\tdefer sqlDB.Close()\n\t\tsqlDB.QueryRow(\"SELECT MIN(id), MAX(id) FROM load_test\").Scan(&analysis.MinRowID, &analysis.MaxRowID)\n\t}\n\n\t// Get final file count\n\tif count, err := db.GetReplicaFileCount(); err == nil {\n\t\tanalysis.FinalFileCount = count\n\t}\n\n\t// Parse litestream log\n\tlogPath, _ := db.GetLitestreamLog()\n\tif logPath != \"\" {\n\t\tparseLog(logPath, analysis)\n\t}\n\n\treturn analysis\n}\n\nfunc parseLog(logPath string, analysis *SoakTestAnalysis) {\n\tfile, err := os.Open(logPath)\n\tif err != nil {\n\t\treturn\n\t}\n\tdefer file.Close()\n\n\tscanner := bufio.NewScanner(file)\n\tvar firstTxID, lastTxID string\n\n\tfor scanner.Scan() {\n\t\tline := scanner.Text()\n\n\t\tif strings.Contains(line, \"compaction complete\") {\n\t\t\tanalysis.TotalCompactions++\n\n\t\t\t// Extract level\n\t\t\tif idx := strings.Index(line, \"level=\"); idx != -1 {\n\t\t\t\tlevelStr := line[idx+6:]\n\t\t\t\tif spaceIdx := strings.Index(levelStr, \" \"); spaceIdx != -1 {\n\t\t\t\t\tlevelStr = levelStr[:spaceIdx]\n\t\t\t\t}\n\t\t\t\tif level, err := strconv.Atoi(levelStr); err == nil {\n\t\t\t\t\tanalysis.CompactionsByLevel[level]++\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Extract transaction IDs\n\t\t\tif idx := strings.Index(line, \"txid.min=\"); idx != -1 {\n\t\t\t\ttxMin := line[idx+9 : idx+25]\n\t\t\t\tif firstTxID == \"\" {\n\t\t\t\t\tfirstTxID = txMin\n\t\t\t\t}\n\t\t\t}\n\t\t\tif idx := strings.Index(line, \"txid.max=\"); idx != -1 {\n\t\t\t\ttxMax := line[idx+9 : idx+25]\n\t\t\t\tlastTxID = txMax\n\t\t\t}\n\t\t}\n\n\t\tif strings.Contains(line, \"snapshot complete\") {\n\t\t\tanalysis.SnapshotCount++\n\t\t}\n\n\t\tif strings.Contains(line, \"checkpoint complete\") {\n\t\t\tanalysis.CheckpointCount++\n\t\t}\n\t}\n\n\tanalysis.MinTxID = firstTxID\n\tanalysis.MaxTxID = lastTxID\n\n\t// Count all LTX files ever created (from txid range)\n\tif analysis.MaxTxID != \"\" {\n\t\tif maxID, err := strconv.ParseInt(analysis.MaxTxID, 16, 64); err == nil {\n\t\t\tanalysis.TotalFilesCreated = int(maxID)\n\t\t}\n\t}\n}\n\n// PrintSoakTestAnalysis prints detailed analysis and plain English summary\nfunc PrintSoakTestAnalysis(t *testing.T, analysis *SoakTestAnalysis) {\n\tt.Helper()\n\n\tt.Log(\"\")\n\tt.Log(\"================================================\")\n\tt.Log(\"Detailed Test Metrics\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\n\t// Compaction breakdown\n\tt.Log(\"Compaction Activity:\")\n\tt.Logf(\" Total compactions: %d\", analysis.TotalCompactions)\n\tlevels := []int{1, 2, 3, 4, 5}\n\tfor _, level := range levels {\n\t\tif count := analysis.CompactionsByLevel[level]; count > 0 {\n\t\t\tt.Logf(\" Level %d: %d compactions\", level, count)\n\t\t}\n\t}\n\tt.Log(\"\")\n\n\t// File operations\n\tt.Log(\"File Operations:\")\n\tt.Logf(\" Total LTX files created: %d\", analysis.TotalFilesCreated)\n\tif analysis.TotalFilesCreated > 0 {\n\t\tt.Logf(\" Final file count: %d (%.1f%% reduction)\",\n\t\t\tanalysis.FinalFileCount,\n\t\t\t100.0*float64(analysis.TotalFilesCreated-analysis.FinalFileCount)/float64(analysis.TotalFilesCreated))\n\t}\n\tt.Logf(\" Snapshots generated: %d\", analysis.SnapshotCount)\n\tif analysis.CheckpointCount > 0 {\n\t\tt.Logf(\" Checkpoints: %d\", analysis.CheckpointCount)\n\t}\n\tt.Log(\"\")\n\n\t// Database activity\n\tt.Log(\"Database Activity:\")\n\tt.Logf(\" Total rows: %d\", analysis.DatabaseRows)\n\tt.Logf(\" Row ID range: %d → %d\", analysis.MinRowID, analysis.MaxRowID)\n\tgapCount := (analysis.MaxRowID - analysis.MinRowID + 1) - analysis.DatabaseRows\n\tif gapCount == 0 {\n\t\tt.Log(\" Row continuity: ✓ No gaps (perfect)\")\n\t} else {\n\t\tt.Logf(\" Row continuity: %d gaps detected\", gapCount)\n\t}\n\tt.Logf(\" Final database size: %.2f MB\", analysis.DatabaseSizeMB)\n\tif analysis.Duration.Seconds() > 0 {\n\t\tavgRate := float64(analysis.DatabaseRows) / analysis.Duration.Seconds()\n\t\tt.Logf(\" Average write rate: %.1f rows/second\", avgRate)\n\t}\n\tt.Log(\"\")\n\n\t// Transaction range\n\tif analysis.MinTxID != \"\" && analysis.MaxTxID != \"\" {\n\t\tt.Log(\"Replication Range:\")\n\t\tt.Logf(\" First transaction: %s\", analysis.MinTxID)\n\t\tt.Logf(\" Last transaction: %s\", analysis.MaxTxID)\n\t\tt.Log(\"\")\n\t}\n\n\t// Plain English summary\n\tt.Log(\"================================================\")\n\tt.Log(\"What This Test Validated\")\n\tt.Log(\"================================================\")\n\tt.Log(\"\")\n\n\tt.Logf(\"✓ Long-term Stability\")\n\tt.Logf(\" Litestream ran flawlessly for %v under sustained load\", analysis.Duration.Round(time.Minute))\n\tt.Log(\"\")\n\n\tt.Log(\"✓ Snapshot Generation\")\n\tt.Logf(\" %d snapshots created successfully\", analysis.SnapshotCount)\n\tt.Log(\"\")\n\n\tt.Log(\"✓ Compaction Efficiency\")\n\tif analysis.TotalFilesCreated > 0 {\n\t\treductionPct := 100.0 * float64(analysis.TotalFilesCreated-analysis.FinalFileCount) / float64(analysis.TotalFilesCreated)\n\t\tt.Logf(\" Reduced %d files to %d (%.0f%% reduction through compaction)\",\n\t\t\tanalysis.TotalFilesCreated, analysis.FinalFileCount, reductionPct)\n\t}\n\tt.Log(\"\")\n\n\tif analysis.DatabaseSizeMB > 1000 {\n\t\tt.Log(\"✓ Large Database Handling\")\n\t\tt.Logf(\" Successfully replicated %.1f GB database\", analysis.DatabaseSizeMB/1024)\n\t\tt.Log(\"\")\n\t}\n\n\tt.Log(\"✓ Restoration Capability\")\n\tt.Log(\" Full restore from replica completed successfully\")\n\tt.Log(\"\")\n\n\tt.Log(\"✓ Data Integrity\")\n\tt.Log(\" SQLite integrity check confirmed no corruption\")\n\tif gapCount == 0 {\n\t\tt.Log(\" All rows present with perfect continuity\")\n\t}\n\tt.Log(\"\")\n}\n" }, { "path": "tests/integration/supabase-s3/docker-compose.yml", "content": "services:\n db:\n image: postgres:15\n restart: unless-stopped\n environment:\n POSTGRES_USER: postgres\n POSTGRES_PASSWORD: postgres\n POSTGRES_DB: postgres\n volumes:\n - pg_data:/var/lib/postgresql/data\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U postgres\"]\n interval: 5s\n timeout: 5s\n retries: 10\n\n minio:\n image: minio/minio\n command: server /data --console-address \":9001\"\n environment:\n MINIO_ROOT_USER: supa-storage\n MINIO_ROOT_PASSWORD: secret1234\n volumes:\n - minio_data:/data\n healthcheck:\n test: [\"CMD\", \"mc\", \"ready\", \"local\"]\n interval: 5s\n timeout: 5s\n retries: 10\n\n minio-createbucket:\n image: minio/mc\n depends_on:\n minio:\n condition: service_healthy\n entrypoint: >\n /bin/sh -c \"\n mc alias set supabase http://minio:9000 supa-storage secret1234;\n mc mb --ignore-existing supabase/supa-storage-bucket;\n exit 0;\n \"\n\n storage:\n image: supabase/storage-api:latest\n depends_on:\n db:\n condition: service_healthy\n minio-createbucket:\n condition: service_completed_successfully\n ports:\n - \"5555:5000\"\n environment:\n DATABASE_URL: postgres://postgres:postgres@db:5432/postgres\n DB_INSTALL_ROLES: \"true\"\n STORAGE_BACKEND: s3\n STORAGE_S3_BUCKET: supa-storage-bucket\n STORAGE_S3_ENDPOINT: http://minio:9000\n STORAGE_S3_FORCE_PATH_STYLE: \"true\"\n STORAGE_S3_REGION: us-east-1\n AWS_ACCESS_KEY_ID: supa-storage\n AWS_SECRET_ACCESS_KEY: secret1234\n AUTH_JWT_SECRET: super-secret-jwt-token-with-at-least-32-characters\n AUTH_JWT_ALGORITHM: HS256\n S3_PROTOCOL_ACCESS_KEY_ID: supabase-s3-access-key\n S3_PROTOCOL_ACCESS_KEY_SECRET: supabase-s3-secret-key-that-is-long-enough\n ANON_KEY: eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9.eyJpc3MiOiAic3VwYWJhc2UiLCAicm9sZSI6ICJhbm9uIn0.y78PwHBHnrLnHZODMOjJfZZPlfKk0W3DucVCxvI6uF8\n SERVICE_KEY: eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9.eyJpc3MiOiAic3VwYWJhc2UiLCAicm9sZSI6ICJzZXJ2aWNlX3JvbGUifQ.lRaC0LUy-3mILAj_17hVWOBnaft3QPpK-pqAs8h2MRI\n IS_MULTITENANT: \"false\"\n POSTGREST_URL: http://localhost:3000\n REGION: us-east-1\n TENANT_ID: stub\n healthcheck:\n test: [\"CMD\", \"wget\", \"--no-verbose\", \"--tries=1\", \"--spider\", \"http://127.0.0.1:5000/status\"]\n interval: 5s\n timeout: 5s\n retries: 15\n start_period: 10s\n\nvolumes:\n pg_data:\n minio_data:\n" }, { "path": "tests/integration/supabase-s3/test.sh", "content": "#!/usr/bin/env bash\nset -euo pipefail\n\nSCRIPT_DIR=\"$(cd \"$(dirname \"$0\")\" && pwd)\"\nPROJECT_ROOT=\"$(cd \"$SCRIPT_DIR/../../..\" && pwd)\"\nENDPOINT=\"http://localhost:5555/s3\"\nSTORAGE_API=\"http://localhost:5555\"\nACCESS_KEY=\"supabase-s3-access-key\"\nSECRET_KEY=\"supabase-s3-secret-key-that-is-long-enough\"\nSERVICE_KEY=\"eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9.eyJpc3MiOiAic3VwYWJhc2UiLCAicm9sZSI6ICJzZXJ2aWNlX3JvbGUifQ.lRaC0LUy-3mILAj_17hVWOBnaft3QPpK-pqAs8h2MRI\"\nBUCKET=\"litestream-test\"\nREGION=\"us-east-1\"\nTMPDIR_BASE=$(mktemp -d)\n\ncleanup() {\n echo \"\"\n echo \"=== Cleaning up ===\"\n rm -rf \"$TMPDIR_BASE\"\n cd \"$SCRIPT_DIR\"\n docker compose down -v --remove-orphans 2>/dev/null || true\n}\ntrap cleanup EXIT\n\necho \"================================================\"\necho \"Supabase S3 Integration Test for Litestream\"\necho \"================================================\"\necho \"\"\n\n# Step 1: Build litestream\necho \"[1/6] Building litestream...\"\ncd \"$PROJECT_ROOT\"\ngo build -o \"$TMPDIR_BASE/litestream\" ./cmd/litestream\necho \" OK: Binary built at $TMPDIR_BASE/litestream\"\necho \"\"\n\n# Step 2: Start Supabase stack\necho \"[2/6] Starting Supabase storage stack...\"\ncd \"$SCRIPT_DIR\"\ndocker compose up -d --wait --wait-timeout 90\necho \" OK: Supabase storage running at $ENDPOINT\"\necho \"\"\n\n# Step 3: Create bucket via Supabase REST API\necho \"[3/6] Creating bucket '$BUCKET' via Supabase REST API...\"\nBUCKET_RESPONSE=$(curl -s -w \"\\n%{http_code}\" -X POST \"$STORAGE_API/bucket\" \\\n -H \"Authorization: Bearer $SERVICE_KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d \"{\\\"name\\\": \\\"$BUCKET\\\", \\\"public\\\": true}\")\nBUCKET_HTTP_CODE=$(echo \"$BUCKET_RESPONSE\" | tail -1)\nBUCKET_BODY=$(echo \"$BUCKET_RESPONSE\" | sed '$d')\nif [ \"$BUCKET_HTTP_CODE\" = \"200\" ] || [ \"$BUCKET_HTTP_CODE\" = \"201\" ]; then\n echo \" OK: Bucket created (HTTP $BUCKET_HTTP_CODE)\"\nelif echo \"$BUCKET_BODY\" | grep -q \"already exists\"; then\n echo \" OK: Bucket already exists\"\nelse\n echo \" FAIL: Bucket creation failed (HTTP $BUCKET_HTTP_CODE): $BUCKET_BODY\"\n exit 1\nfi\necho \"\"\n\n# Step 4: Create and populate test database\necho \"[4/6] Creating test SQLite database...\"\nDB_PATH=\"$TMPDIR_BASE/test.db\"\nsqlite3 \"$DB_PATH\" <<'SQL'\nPRAGMA journal_mode=WAL;\nCREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT, email TEXT, created_at TEXT);\nINSERT INTO users VALUES (1, 'Alice', 'alice@example.com', datetime('now'));\nINSERT INTO users VALUES (2, 'Bob', 'bob@example.com', datetime('now'));\nINSERT INTO users VALUES (3, 'Charlie', 'charlie@example.com', datetime('now'));\nSQL\nROW_COUNT=$(sqlite3 \"$DB_PATH\" \"SELECT COUNT(*) FROM users;\")\necho \" OK: Database created with $ROW_COUNT rows\"\necho \"\"\n\n# Step 5: Replicate to Supabase S3\necho \"[5/6] Replicating database to Supabase S3...\"\nREPLICA_URL=\"s3://${BUCKET}/testdb?endpoint=${ENDPOINT}®ion=${REGION}\"\necho \" URL: $REPLICA_URL\"\necho \" (Auto-detection should apply force-path-style=true and sign-payload=true)\"\n\nAWS_ACCESS_KEY_ID=\"$ACCESS_KEY\" \\\nAWS_SECRET_ACCESS_KEY=\"$SECRET_KEY\" \\\n\"$TMPDIR_BASE/litestream\" replicate \"$DB_PATH\" \"$REPLICA_URL\" &\nLITESTREAM_PID=$!\n\n# Wait for initial sync, then write more data\nsleep 5\nsqlite3 \"$DB_PATH\" <<'SQL'\nINSERT INTO users VALUES (4, 'Diana', 'diana@example.com', datetime('now'));\nINSERT INTO users VALUES (5, 'Eve', 'eve@example.com', datetime('now'));\nSQL\necho \" OK: Wrote 2 additional rows during replication\"\n\n# Give time for WAL sync\nsleep 10\nkill \"$LITESTREAM_PID\" 2>/dev/null || true\nwait \"$LITESTREAM_PID\" 2>/dev/null || true\necho \" OK: Replication completed\"\necho \"\"\n\n# Step 6: Restore and verify\necho \"[6/6] Restoring database from Supabase S3...\"\nRESTORE_PATH=\"$TMPDIR_BASE/restored.db\"\nAWS_ACCESS_KEY_ID=\"$ACCESS_KEY\" \\\nAWS_SECRET_ACCESS_KEY=\"$SECRET_KEY\" \\\n\"$TMPDIR_BASE/litestream\" restore -o \"$RESTORE_PATH\" \"$REPLICA_URL\"\n\nRESTORED_COUNT=$(sqlite3 \"$RESTORE_PATH\" \"SELECT COUNT(*) FROM users;\")\necho \" Restored row count: $RESTORED_COUNT\"\n\nif [ \"$RESTORED_COUNT\" -ge 3 ]; then\n echo \"\"\n echo \"================================================\"\n echo \"SUCCESS: Supabase S3 integration test passed!\"\n echo \" - Auto-detection: Working (no manual force-path-style/sign-payload needed)\"\n echo \" - Replication: Working\"\n echo \" - Restore: Working ($RESTORED_COUNT rows recovered)\"\n echo \"================================================\"\nelse\n echo \"\"\n echo \"FAIL: Expected at least 3 rows, got $RESTORED_COUNT\"\n exit 1\nfi\n" }, { "path": "tests/integration/upgrade_test.go", "content": "//go:build integration\n\npackage integration\n\nimport (\n\t\"database/sql\"\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"syscall\"\n\t\"testing\"\n\t\"time\"\n\n\t_ \"github.com/mattn/go-sqlite3\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nfunc TestUpgrade_V3ToV5(t *testing.T) {\n\tif testing.Short() {\n\t\tt.Skip(\"skipping integration test in short mode\")\n\t}\n\n\t// Skip if v0.3 binary not provided.\n\tv3Bin := os.Getenv(\"LITESTREAM_V3_BIN\")\n\tif v3Bin == \"\" {\n\t\tt.Skip(\"LITESTREAM_V3_BIN not set, skipping upgrade test\")\n\t}\n\n\t// Verify v0.3 binary exists and is executable.\n\tif info, err := os.Stat(v3Bin); err != nil {\n\t\tt.Fatalf(\"v0.3 binary not found at %s: %v\", v3Bin, err)\n\t} else if info.Mode()&0111 == 0 {\n\t\tt.Fatalf(\"v0.3 binary at %s is not executable\", v3Bin)\n\t}\n\n\t// Verify current binary exists and is executable.\n\tv5Bin := getBinaryPath(\"litestream\")\n\tif info, err := os.Stat(v5Bin); err != nil {\n\t\tt.Fatalf(\"current binary not found at %s: %v\", v5Bin, err)\n\t} else if info.Mode()&0111 == 0 {\n\t\tt.Fatalf(\"current binary at %s is not executable\", v5Bin)\n\t}\n\n\t// Set up temp directory with database and replica paths.\n\ttmpDir := t.TempDir()\n\tdbPath := filepath.Join(tmpDir, \"upgrade.db\")\n\treplicaPath := filepath.Join(tmpDir, \"replica\")\n\treplicaURL := fmt.Sprintf(\"file://%s\", filepath.ToSlash(replicaPath))\n\trestoredPath := filepath.Join(tmpDir, \"restored.db\")\n\n\t// Create WAL-mode database with test table.\n\tt.Log(\"Creating WAL-mode database with upgrade_test table\")\n\tsqlDB, err := sql.Open(\"sqlite3\", dbPath)\n\tif err != nil {\n\t\tt.Fatalf(\"open database: %v\", err)\n\t}\n\tdefer sqlDB.Close()\n\n\tif _, err := sqlDB.Exec(\"PRAGMA journal_mode=WAL\"); err != nil {\n\t\tt.Fatalf(\"set WAL mode: %v\", err)\n\t}\n\tif _, err := sqlDB.Exec(\"CREATE TABLE upgrade_test(id INTEGER PRIMARY KEY, phase TEXT, data BLOB)\"); err != nil {\n\t\tt.Fatalf(\"create table: %v\", err)\n\t}\n\n\t// =========================================================================\n\t// Phase 1: v0.3.x replication\n\t// =========================================================================\n\tt.Log(\"Phase 1: v0.3.x replication\")\n\n\t// Insert 10 rows with phase='v3-initial'.\n\tt.Log(\" Inserting 10 v3-initial rows\")\n\tfor i := 0; i < 10; i++ {\n\t\tif _, err := sqlDB.Exec(\"INSERT INTO upgrade_test(phase, data) VALUES(?, randomblob(100))\", \"v3-initial\"); err != nil {\n\t\t\tt.Fatalf(\"insert v3-initial row %d: %v\", i, err)\n\t\t}\n\t}\n\n\t// Start v0.3 replicate subprocess.\n\tt.Logf(\" Starting v0.3 binary: %s replicate %s %s\", v3Bin, dbPath, replicaURL)\n\tv3Cmd := exec.Command(v3Bin, \"replicate\", dbPath, replicaURL)\n\tv3LogFile, err := os.Create(filepath.Join(tmpDir, \"v3-replicate.log\"))\n\tif err != nil {\n\t\tt.Fatalf(\"create v3 log file: %v\", err)\n\t}\n\tdefer v3LogFile.Close()\n\tv3Cmd.Stdout = v3LogFile\n\tv3Cmd.Stderr = v3LogFile\n\tif err := v3Cmd.Start(); err != nil {\n\t\tt.Fatalf(\"start v0.3 replicate: %v\", err)\n\t}\n\tt.Cleanup(func() {\n\t\tif v3Cmd.ProcessState == nil {\n\t\t\tv3Cmd.Process.Signal(syscall.SIGINT)\n\t\t\tv3Cmd.Wait()\n\t\t}\n\t})\n\n\t// Wait for initial sync.\n\tt.Log(\" Waiting 3s for initial sync\")\n\ttime.Sleep(3 * time.Second)\n\n\t// Insert 10 more rows with phase='v3-running' across 5 transactions (2 rows each).\n\tt.Log(\" Inserting 10 v3-running rows across 5 transactions\")\n\tfor txn := 0; txn < 5; txn++ {\n\t\ttx, err := sqlDB.Begin()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"begin tx %d: %v\", txn, err)\n\t\t}\n\t\tfor j := 0; j < 2; j++ {\n\t\t\tif _, err := tx.Exec(\"INSERT INTO upgrade_test(phase, data) VALUES(?, randomblob(100))\", \"v3-running\"); err != nil {\n\t\t\t\ttx.Rollback()\n\t\t\t\tt.Fatalf(\"insert v3-running tx %d row %d: %v\", txn, j, err)\n\t\t\t}\n\t\t}\n\t\tif err := tx.Commit(); err != nil {\n\t\t\tt.Fatalf(\"commit tx %d: %v\", txn, err)\n\t\t}\n\t}\n\n\t// Force checkpoint.\n\tt.Log(\" Running PRAGMA wal_checkpoint(TRUNCATE)\")\n\tif _, err := sqlDB.Exec(\"PRAGMA wal_checkpoint(TRUNCATE)\"); err != nil {\n\t\tt.Fatalf(\"checkpoint: %v\", err)\n\t}\n\n\t// Insert 5 more rows with phase='v3-post-checkpoint'.\n\tt.Log(\" Inserting 5 v3-post-checkpoint rows\")\n\tfor i := 0; i < 5; i++ {\n\t\tif _, err := sqlDB.Exec(\"INSERT INTO upgrade_test(phase, data) VALUES(?, randomblob(100))\", \"v3-post-checkpoint\"); err != nil {\n\t\t\tt.Fatalf(\"insert v3-post-checkpoint row %d: %v\", i, err)\n\t\t}\n\t}\n\n\t// Wait for sync, then stop v0.3.\n\tt.Log(\" Waiting 3s for sync\")\n\ttime.Sleep(3 * time.Second)\n\n\tt.Log(\" Sending SIGINT to v0.3 process\")\n\tif err := v3Cmd.Process.Signal(syscall.SIGINT); err != nil {\n\t\tt.Fatalf(\"signal v0.3 process: %v\", err)\n\t}\n\tif err := v3Cmd.Wait(); err != nil {\n\t\tt.Logf(\" v0.3 process exited with: %v (expected for SIGINT)\", err)\n\t}\n\n\t// Verify generations/ directory exists (v0.3 layout).\n\tgenerationsDir := filepath.Join(replicaPath, \"generations\")\n\tif _, err := os.Stat(generationsDir); err != nil {\n\t\tt.Fatalf(\"v0.3 replica generations/ directory not found: %v\", err)\n\t}\n\tt.Log(\" Verified generations/ directory exists in replica\")\n\n\t// =========================================================================\n\t// Phase 2: v0.5.x replication\n\t// =========================================================================\n\tt.Log(\"Phase 2: v0.5.x replication\")\n\n\t// Start current binary replicate subprocess.\n\tt.Logf(\" Starting v0.5 binary: %s replicate %s %s\", v5Bin, dbPath, replicaURL)\n\tv5Cmd := exec.Command(v5Bin, \"replicate\", dbPath, replicaURL)\n\tv5LogFile, err := os.Create(filepath.Join(tmpDir, \"v5-replicate.log\"))\n\tif err != nil {\n\t\tt.Fatalf(\"create v5 log file: %v\", err)\n\t}\n\tdefer v5LogFile.Close()\n\tv5Cmd.Stdout = v5LogFile\n\tv5Cmd.Stderr = v5LogFile\n\tif err := v5Cmd.Start(); err != nil {\n\t\tt.Fatalf(\"start v0.5 replicate: %v\", err)\n\t}\n\tt.Cleanup(func() {\n\t\tif v5Cmd.ProcessState == nil {\n\t\t\tv5Cmd.Process.Signal(syscall.SIGINT)\n\t\t\tv5Cmd.Wait()\n\t\t}\n\t})\n\n\t// Wait up to 30s for ltx/9/ directory to contain at least one .ltx file (snapshot).\n\tt.Log(\" Waiting up to 30s for snapshot in ltx/9/\")\n\tsnapshotDir := filepath.Join(replicaPath, \"ltx\", fmt.Sprintf(\"%d\", litestream.SnapshotLevel))\n\tdeadline := time.Now().Add(30 * time.Second)\n\tsnapshotFound := false\n\tfor time.Now().Before(deadline) {\n\t\tmatches, _ := filepath.Glob(filepath.Join(snapshotDir, \"*.ltx\"))\n\t\tif len(matches) > 0 {\n\t\t\tsnapshotFound = true\n\t\t\tt.Logf(\" Found %d snapshot file(s) in ltx/9/\", len(matches))\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(500 * time.Millisecond)\n\t}\n\tif !snapshotFound {\n\t\tt.Fatal(\"timeout waiting for v0.5.x snapshot in ltx/9/\")\n\t}\n\n\t// Insert 10 rows with phase='v5-running' across 5 transactions.\n\tt.Log(\" Inserting 10 v5-running rows across 5 transactions\")\n\tfor txn := 0; txn < 5; txn++ {\n\t\ttx, err := sqlDB.Begin()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"begin tx %d: %v\", txn, err)\n\t\t}\n\t\tfor j := 0; j < 2; j++ {\n\t\t\tif _, err := tx.Exec(\"INSERT INTO upgrade_test(phase, data) VALUES(?, randomblob(100))\", \"v5-running\"); err != nil {\n\t\t\t\ttx.Rollback()\n\t\t\t\tt.Fatalf(\"insert v5-running tx %d row %d: %v\", txn, j, err)\n\t\t\t}\n\t\t}\n\t\tif err := tx.Commit(); err != nil {\n\t\t\tt.Fatalf(\"commit tx %d: %v\", txn, err)\n\t\t}\n\t}\n\n\t// Force checkpoint.\n\tt.Log(\" Running PRAGMA wal_checkpoint(TRUNCATE)\")\n\tif _, err := sqlDB.Exec(\"PRAGMA wal_checkpoint(TRUNCATE)\"); err != nil {\n\t\tt.Fatalf(\"checkpoint: %v\", err)\n\t}\n\n\t// Insert 5 more rows with phase='v5-post-checkpoint'.\n\tt.Log(\" Inserting 5 v5-post-checkpoint rows\")\n\tfor i := 0; i < 5; i++ {\n\t\tif _, err := sqlDB.Exec(\"INSERT INTO upgrade_test(phase, data) VALUES(?, randomblob(100))\", \"v5-post-checkpoint\"); err != nil {\n\t\t\tt.Fatalf(\"insert v5-post-checkpoint row %d: %v\", i, err)\n\t\t}\n\t}\n\n\t// Wait for sync, then stop v0.5.\n\tt.Log(\" Waiting 3s for sync\")\n\ttime.Sleep(3 * time.Second)\n\n\tt.Log(\" Sending SIGINT to v0.5 process\")\n\tif err := v5Cmd.Process.Signal(syscall.SIGINT); err != nil {\n\t\tt.Fatalf(\"signal v0.5 process: %v\", err)\n\t}\n\tif err := v5Cmd.Wait(); err != nil {\n\t\tt.Logf(\" v0.5 process exited with: %v (expected for SIGINT)\", err)\n\t}\n\n\t// =========================================================================\n\t// Phase 3: Restore\n\t// =========================================================================\n\tt.Log(\"Phase 3: Restore using current binary\")\n\n\trestoreCmd := exec.Command(v5Bin, \"restore\", \"-o\", restoredPath, replicaURL)\n\trestoreOutput, err := restoreCmd.CombinedOutput()\n\tif err != nil {\n\t\tt.Fatalf(\"restore failed: %v\\nOutput: %s\", err, restoreOutput)\n\t}\n\tt.Log(\" Restore completed successfully\")\n\n\t// =========================================================================\n\t// Phase 4: Validate\n\t// =========================================================================\n\tt.Log(\"Phase 4: Validate restored database\")\n\n\trestoredDB, err := sql.Open(\"sqlite3\", restoredPath)\n\tif err != nil {\n\t\tt.Fatalf(\"open restored database: %v\", err)\n\t}\n\tdefer restoredDB.Close()\n\n\t// Integrity check.\n\tvar integrity string\n\tif err := restoredDB.QueryRow(\"PRAGMA integrity_check\").Scan(&integrity); err != nil {\n\t\tt.Fatalf(\"integrity check query: %v\", err)\n\t}\n\tif integrity != \"ok\" {\n\t\tt.Fatalf(\"integrity check failed: %s\", integrity)\n\t}\n\tt.Log(\" PRAGMA integrity_check: ok\")\n\n\t// Count total rows.\n\tvar totalCount int\n\tif err := restoredDB.QueryRow(\"SELECT COUNT(*) FROM upgrade_test\").Scan(&totalCount); err != nil {\n\t\tt.Fatalf(\"count total rows: %v\", err)\n\t}\n\tif totalCount != 40 {\n\t\tt.Fatalf(\"total row count: got %d, want 40\", totalCount)\n\t}\n\tt.Logf(\" Total rows: %d (expected 40)\", totalCount)\n\n\t// Count rows per phase. The v3-initial rows were inserted before v0.3\n\t// started, so their presence in the restore proves the v0.5.x snapshot\n\t// captured the full database state (not just changes made after v0.5.x started).\n\tphases := []struct {\n\t\tname string\n\t\texpected int\n\t}{\n\t\t{\"v3-initial\", 10},\n\t\t{\"v3-running\", 10},\n\t\t{\"v3-post-checkpoint\", 5},\n\t\t{\"v5-running\", 10},\n\t\t{\"v5-post-checkpoint\", 5},\n\t}\n\tfor _, p := range phases {\n\t\tvar count int\n\t\tif err := restoredDB.QueryRow(\"SELECT COUNT(*) FROM upgrade_test WHERE phase = ?\", p.name).Scan(&count); err != nil {\n\t\t\tt.Fatalf(\"count phase %q: %v\", p.name, err)\n\t\t}\n\t\tif count != p.expected {\n\t\t\tt.Errorf(\"phase %q: got %d rows, want %d\", p.name, count, p.expected)\n\t\t}\n\t\tt.Logf(\" Phase %q: %d rows\", p.name, count)\n\t}\n}\n" }, { "path": "v3.go", "content": "package litestream\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"path\"\n\t\"regexp\"\n\t\"strconv\"\n\t\"time\"\n)\n\n// PosV3 represents a position in a v0.3.x backup.\ntype PosV3 struct {\n\tGeneration string // 16-char hex string\n\tIndex int // WAL index\n\tOffset int64 // Offset within WAL segment\n}\n\n// IsZero returns true if the position is the zero value.\nfunc (p PosV3) IsZero() bool {\n\treturn p == (PosV3{})\n}\n\n// String returns a string representation of the position.\nfunc (p PosV3) String() string {\n\tif p.IsZero() {\n\t\treturn \"\"\n\t}\n\treturn fmt.Sprintf(\"%s/%08x:%016x\", p.Generation, p.Index, p.Offset)\n}\n\n// SnapshotInfoV3 contains metadata about a v0.3.x snapshot.\ntype SnapshotInfoV3 struct {\n\tGeneration string\n\tIndex int\n\tSize int64\n\tCreatedAt time.Time\n}\n\n// Pos returns the position of this snapshot.\nfunc (info SnapshotInfoV3) Pos() PosV3 {\n\treturn PosV3{Generation: info.Generation, Index: info.Index, Offset: 0}\n}\n\n// WALSegmentInfoV3 contains metadata about a v0.3.x WAL segment.\ntype WALSegmentInfoV3 struct {\n\tGeneration string\n\tIndex int\n\tOffset int64\n\tSize int64\n\tCreatedAt time.Time\n}\n\n// Pos returns the position of this WAL segment.\nfunc (info WALSegmentInfoV3) Pos() PosV3 {\n\treturn PosV3{Generation: info.Generation, Index: info.Index, Offset: info.Offset}\n}\n\n// v0.3.x path constants.\nconst (\n\tGenerationsDirV3 = \"generations\"\n\tSnapshotsDirV3 = \"snapshots\"\n\tWALDirV3 = \"wal\"\n)\n\n// GenerationsPathV3 returns the path to the generations directory.\nfunc GenerationsPathV3(root string) string {\n\treturn path.Join(root, GenerationsDirV3)\n}\n\n// GenerationPathV3 returns the path to a specific generation.\nfunc GenerationPathV3(root, generation string) string {\n\treturn path.Join(root, GenerationsDirV3, generation)\n}\n\n// SnapshotsPathV3 returns the path to snapshots within a generation.\nfunc SnapshotsPathV3(root, generation string) string {\n\treturn path.Join(root, GenerationsDirV3, generation, SnapshotsDirV3)\n}\n\n// WALPathV3 returns the path to WAL segments within a generation.\nfunc WALPathV3(root, generation string) string {\n\treturn path.Join(root, GenerationsDirV3, generation, WALDirV3)\n}\n\n// SnapshotPathV3 returns the full path to a v0.3.x snapshot file.\nfunc SnapshotPathV3(root, generation string, index int) string {\n\treturn path.Join(SnapshotsPathV3(root, generation), FormatSnapshotFilenameV3(index))\n}\n\n// WALSegmentPathV3 returns the full path to a v0.3.x WAL segment file.\nfunc WALSegmentPathV3(root, generation string, index int, offset int64) string {\n\treturn path.Join(WALPathV3(root, generation), FormatWALSegmentFilenameV3(index, offset))\n}\n\n// FormatSnapshotFilenameV3 returns the filename for a v0.3.x snapshot.\n// Format: {index:08x}.snapshot.lz4\nfunc FormatSnapshotFilenameV3(index int) string {\n\treturn fmt.Sprintf(\"%08x.snapshot.lz4\", index)\n}\n\n// FormatWALSegmentFilenameV3 returns the filename for a v0.3.x WAL segment.\n// Format: {index:08x}_{offset:08x}.wal.lz4\nfunc FormatWALSegmentFilenameV3(index int, offset int64) string {\n\treturn fmt.Sprintf(\"%08x_%08x.wal.lz4\", index, offset)\n}\n\nvar (\n\tsnapshotRegexV3 = regexp.MustCompile(`^([0-9a-f]{8})\\.snapshot\\.lz4$`)\n\twalSegmentRegexV3 = regexp.MustCompile(`^([0-9a-f]{8})_([0-9a-f]{8})\\.wal\\.lz4$`)\n\tgenerationRegexV3 = regexp.MustCompile(`^[0-9a-f]{16}$`)\n)\n\n// ParseSnapshotFilenameV3 parses a v0.3.x snapshot filename and returns the index.\n// Returns an error if the filename does not match the expected format.\nfunc ParseSnapshotFilenameV3(filename string) (index int, err error) {\n\tm := snapshotRegexV3.FindStringSubmatch(filename)\n\tif m == nil {\n\t\treturn 0, fmt.Errorf(\"invalid v0.3.x snapshot filename: %q\", filename)\n\t}\n\tidx, _ := strconv.ParseInt(m[1], 16, 64)\n\treturn int(idx), nil\n}\n\n// ParseWALSegmentFilenameV3 parses a v0.3.x WAL segment filename.\n// Returns the WAL index and byte offset, or an error if the filename is invalid.\nfunc ParseWALSegmentFilenameV3(filename string) (index int, offset int64, err error) {\n\tm := walSegmentRegexV3.FindStringSubmatch(filename)\n\tif m == nil {\n\t\treturn 0, 0, fmt.Errorf(\"invalid v0.3.x WAL segment filename: %q\", filename)\n\t}\n\tidx, _ := strconv.ParseInt(m[1], 16, 64)\n\toff, _ := strconv.ParseInt(m[2], 16, 64)\n\treturn int(idx), off, nil\n}\n\n// IsGenerationIDV3 returns true if s is a valid v0.3.x generation ID (16 hex chars).\nfunc IsGenerationIDV3(s string) bool {\n\treturn generationRegexV3.MatchString(s)\n}\n\n// ReplicaClientV3 reads v0.3.x backup data.\n// ReplicaClient implementations that support v0.3.x restore should implement this interface.\ntype ReplicaClientV3 interface {\n\t// GenerationsV3 returns a list of generation IDs in the replica.\n\t// Returns an empty slice if no v0.3.x backups exist.\n\t// Generation IDs are sorted in ascending order.\n\tGenerationsV3(ctx context.Context) ([]string, error)\n\n\t// SnapshotsV3 returns snapshots for a generation, sorted by index.\n\t// Returns an empty slice if no snapshots exist.\n\tSnapshotsV3(ctx context.Context, generation string) ([]SnapshotInfoV3, error)\n\n\t// WALSegmentsV3 returns WAL segments for a generation, sorted by index then offset.\n\t// Returns an empty slice if no WAL segments exist.\n\tWALSegmentsV3(ctx context.Context, generation string) ([]WALSegmentInfoV3, error)\n\n\t// OpenSnapshotV3 opens a v0.3.x snapshot for reading.\n\t// The returned reader provides LZ4-decompressed data.\n\tOpenSnapshotV3(ctx context.Context, generation string, index int) (io.ReadCloser, error)\n\n\t// OpenWALSegmentV3 opens a v0.3.x WAL segment for reading.\n\t// The returned reader provides LZ4-decompressed data.\n\tOpenWALSegmentV3(ctx context.Context, generation string, index int, offset int64) (io.ReadCloser, error)\n}\n" }, { "path": "v3_test.go", "content": "package litestream_test\n\nimport (\n\t\"testing\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nfunc TestPosV3_IsZero(t *testing.T) {\n\tif !(litestream.PosV3{}).IsZero() {\n\t\tt.Error(\"zero value should return true\")\n\t}\n\tif (litestream.PosV3{Generation: \"abc\"}).IsZero() {\n\t\tt.Error(\"non-zero value should return false\")\n\t}\n}\n\nfunc TestPosV3_String(t *testing.T) {\n\ttests := []struct {\n\t\tpos litestream.PosV3\n\t\twant string\n\t}{\n\t\t{litestream.PosV3{}, \"\"},\n\t\t{litestream.PosV3{Generation: \"0123456789abcdef\", Index: 1, Offset: 4096}, \"0123456789abcdef/00000001:0000000000001000\"},\n\t}\n\tfor _, tt := range tests {\n\t\tif got := tt.pos.String(); got != tt.want {\n\t\t\tt.Errorf(\"PosV3%+v.String() = %q, want %q\", tt.pos, got, tt.want)\n\t\t}\n\t}\n}\n\nfunc TestSnapshotInfoV3_Pos(t *testing.T) {\n\tinfo := litestream.SnapshotInfoV3{Generation: \"abc\", Index: 5}\n\tpos := info.Pos()\n\tif pos.Generation != \"abc\" || pos.Index != 5 || pos.Offset != 0 {\n\t\tt.Errorf(\"unexpected pos: %+v\", pos)\n\t}\n}\n\nfunc TestWALSegmentInfoV3_Pos(t *testing.T) {\n\tinfo := litestream.WALSegmentInfoV3{Generation: \"abc\", Index: 5, Offset: 100}\n\tpos := info.Pos()\n\tif pos.Generation != \"abc\" || pos.Index != 5 || pos.Offset != 100 {\n\t\tt.Errorf(\"unexpected pos: %+v\", pos)\n\t}\n}\n\nfunc TestFormatSnapshotFilenameV3(t *testing.T) {\n\ttests := []struct {\n\t\tindex int\n\t\twant string\n\t}{\n\t\t{0, \"00000000.snapshot.lz4\"},\n\t\t{1, \"00000001.snapshot.lz4\"},\n\t\t{255, \"000000ff.snapshot.lz4\"},\n\t\t{0x12345678, \"12345678.snapshot.lz4\"},\n\t}\n\tfor _, tt := range tests {\n\t\tif got := litestream.FormatSnapshotFilenameV3(tt.index); got != tt.want {\n\t\t\tt.Errorf(\"FormatSnapshotFilenameV3(%d) = %q, want %q\", tt.index, got, tt.want)\n\t\t}\n\t}\n}\n\nfunc TestParseSnapshotFilenameV3(t *testing.T) {\n\tt.Run(\"Valid\", func(t *testing.T) {\n\t\ttests := []struct {\n\t\t\tfilename string\n\t\t\twant int\n\t\t}{\n\t\t\t{\"00000000.snapshot.lz4\", 0},\n\t\t\t{\"00000001.snapshot.lz4\", 1},\n\t\t\t{\"000000ff.snapshot.lz4\", 255},\n\t\t\t{\"12345678.snapshot.lz4\", 0x12345678},\n\t\t}\n\t\tfor _, tt := range tests {\n\t\t\tindex, err := litestream.ParseSnapshotFilenameV3(tt.filename)\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"ParseSnapshotFilenameV3(%q) error: %v\", tt.filename, err)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif index != tt.want {\n\t\t\t\tt.Errorf(\"ParseSnapshotFilenameV3(%q) = %d, want %d\", tt.filename, index, tt.want)\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"Invalid\", func(t *testing.T) {\n\t\tinvalids := []string{\n\t\t\t\"\",\n\t\t\t\"invalid.txt\",\n\t\t\t\"00000001.snapshot\", // missing .lz4\n\t\t\t\"0000001.snapshot.lz4\", // 7 chars, not 8\n\t\t\t\"000000001.snapshot.lz4\", // 9 chars, not 8\n\t\t\t\"0000000g.snapshot.lz4\", // invalid hex\n\t\t\t\"00000001.SNAPSHOT.lz4\", // uppercase\n\t\t\t\"00000001.snapshot.lz4.bak\",\n\t\t}\n\t\tfor _, filename := range invalids {\n\t\t\t_, err := litestream.ParseSnapshotFilenameV3(filename)\n\t\t\tif err == nil {\n\t\t\t\tt.Errorf(\"ParseSnapshotFilenameV3(%q) expected error\", filename)\n\t\t\t}\n\t\t}\n\t})\n}\n\nfunc TestFormatWALSegmentFilenameV3(t *testing.T) {\n\ttests := []struct {\n\t\tindex int\n\t\toffset int64\n\t\twant string\n\t}{\n\t\t{0, 0, \"00000000_00000000.wal.lz4\"},\n\t\t{1, 4096, \"00000001_00001000.wal.lz4\"},\n\t\t{255, 0x12345678, \"000000ff_12345678.wal.lz4\"},\n\t}\n\tfor _, tt := range tests {\n\t\tif got := litestream.FormatWALSegmentFilenameV3(tt.index, tt.offset); got != tt.want {\n\t\t\tt.Errorf(\"FormatWALSegmentFilenameV3(%d, %d) = %q, want %q\", tt.index, tt.offset, got, tt.want)\n\t\t}\n\t}\n}\n\nfunc TestParseWALSegmentFilenameV3(t *testing.T) {\n\tt.Run(\"Valid\", func(t *testing.T) {\n\t\ttests := []struct {\n\t\t\tfilename string\n\t\t\twantIndex int\n\t\t\twantOffset int64\n\t\t}{\n\t\t\t{\"00000000_00000000.wal.lz4\", 0, 0},\n\t\t\t{\"00000001_00001000.wal.lz4\", 1, 4096},\n\t\t\t{\"000000ff_12345678.wal.lz4\", 255, 0x12345678},\n\t\t}\n\t\tfor _, tt := range tests {\n\t\t\tindex, offset, err := litestream.ParseWALSegmentFilenameV3(tt.filename)\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"ParseWALSegmentFilenameV3(%q) error: %v\", tt.filename, err)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif index != tt.wantIndex || offset != tt.wantOffset {\n\t\t\t\tt.Errorf(\"ParseWALSegmentFilenameV3(%q) = (%d, %d), want (%d, %d)\",\n\t\t\t\t\ttt.filename, index, offset, tt.wantIndex, tt.wantOffset)\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"Invalid\", func(t *testing.T) {\n\t\tinvalids := []string{\n\t\t\t\"\",\n\t\t\t\"invalid.wal\",\n\t\t\t\"00000001.wal.lz4\", // missing offset\n\t\t\t\"00000001_00001000.wal\", // missing .lz4\n\t\t\t\"0000001_00001000.wal.lz4\", // 7 chars index\n\t\t\t\"00000001_0001000.wal.lz4\", // 7 chars offset\n\t\t}\n\t\tfor _, filename := range invalids {\n\t\t\t_, _, err := litestream.ParseWALSegmentFilenameV3(filename)\n\t\t\tif err == nil {\n\t\t\t\tt.Errorf(\"ParseWALSegmentFilenameV3(%q) expected error\", filename)\n\t\t\t}\n\t\t}\n\t})\n}\n\nfunc TestIsGenerationIDV3(t *testing.T) {\n\ttests := []struct {\n\t\ts string\n\t\twant bool\n\t}{\n\t\t{\"0123456789abcdef\", true},\n\t\t{\"abcdef0123456789\", true},\n\t\t{\"aaaaaaaaaaaaaaaa\", true},\n\t\t{\"0000000000000000\", true},\n\t\t{\"ffffffffffffffff\", true},\n\t\t{\"0123456789ABCDEF\", false}, // uppercase not valid\n\t\t{\"0123456789abcde\", false}, // 15 chars, too short\n\t\t{\"0123456789abcdeff\", false}, // 17 chars, too long\n\t\t{\"0123456789abcdeg\", false}, // invalid hex char\n\t\t{\"\", false},\n\t\t{\"generations\", false},\n\t}\n\tfor _, tt := range tests {\n\t\tif got := litestream.IsGenerationIDV3(tt.s); got != tt.want {\n\t\t\tt.Errorf(\"IsGenerationIDV3(%q) = %v, want %v\", tt.s, got, tt.want)\n\t\t}\n\t}\n}\n\nfunc TestPathsV3(t *testing.T) {\n\troot := \"/data/replica\"\n\tgen := \"0123456789abcdef\"\n\n\tt.Run(\"GenerationsPath\", func(t *testing.T) {\n\t\tgot := litestream.GenerationsPathV3(root)\n\t\twant := \"/data/replica/generations\"\n\t\tif got != want {\n\t\t\tt.Errorf(\"GenerationsPathV3(%q) = %q, want %q\", root, got, want)\n\t\t}\n\t})\n\n\tt.Run(\"GenerationPath\", func(t *testing.T) {\n\t\tgot := litestream.GenerationPathV3(root, gen)\n\t\twant := \"/data/replica/generations/0123456789abcdef\"\n\t\tif got != want {\n\t\t\tt.Errorf(\"GenerationPathV3(%q, %q) = %q, want %q\", root, gen, got, want)\n\t\t}\n\t})\n\n\tt.Run(\"SnapshotsPath\", func(t *testing.T) {\n\t\tgot := litestream.SnapshotsPathV3(root, gen)\n\t\twant := \"/data/replica/generations/0123456789abcdef/snapshots\"\n\t\tif got != want {\n\t\t\tt.Errorf(\"SnapshotsPathV3(%q, %q) = %q, want %q\", root, gen, got, want)\n\t\t}\n\t})\n\n\tt.Run(\"WALPath\", func(t *testing.T) {\n\t\tgot := litestream.WALPathV3(root, gen)\n\t\twant := \"/data/replica/generations/0123456789abcdef/wal\"\n\t\tif got != want {\n\t\t\tt.Errorf(\"WALPathV3(%q, %q) = %q, want %q\", root, gen, got, want)\n\t\t}\n\t})\n\n\tt.Run(\"SnapshotPath\", func(t *testing.T) {\n\t\tgot := litestream.SnapshotPathV3(root, gen, 1)\n\t\twant := \"/data/replica/generations/0123456789abcdef/snapshots/00000001.snapshot.lz4\"\n\t\tif got != want {\n\t\t\tt.Errorf(\"SnapshotPathV3(%q, %q, 1) = %q, want %q\", root, gen, got, want)\n\t\t}\n\t})\n\n\tt.Run(\"WALSegmentPath\", func(t *testing.T) {\n\t\tgot := litestream.WALSegmentPathV3(root, gen, 1, 4096)\n\t\twant := \"/data/replica/generations/0123456789abcdef/wal/00000001_00001000.wal.lz4\"\n\t\tif got != want {\n\t\t\tt.Errorf(\"WALSegmentPathV3(%q, %q, 1, 4096) = %q, want %q\", root, gen, got, want)\n\t\t}\n\t})\n}\n\n// TestFormatParseRoundtrip verifies that Format and Parse are inverses.\nfunc TestFormatParseRoundtrip(t *testing.T) {\n\tt.Run(\"Snapshot\", func(t *testing.T) {\n\t\tfor _, index := range []int{0, 1, 255, 0x12345678} {\n\t\t\tfilename := litestream.FormatSnapshotFilenameV3(index)\n\t\t\tgot, err := litestream.ParseSnapshotFilenameV3(filename)\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"roundtrip failed for index %d: %v\", index, err)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif got != index {\n\t\t\t\tt.Errorf(\"roundtrip: FormatSnapshotFilenameV3(%d) -> %q -> ParseSnapshotFilenameV3 -> %d\", index, filename, got)\n\t\t\t}\n\t\t}\n\t})\n\n\tt.Run(\"WALSegment\", func(t *testing.T) {\n\t\tcases := []struct {\n\t\t\tindex int\n\t\t\toffset int64\n\t\t}{\n\t\t\t{0, 0},\n\t\t\t{1, 4096},\n\t\t\t{255, 0x12345678},\n\t\t}\n\t\tfor _, tc := range cases {\n\t\t\tfilename := litestream.FormatWALSegmentFilenameV3(tc.index, tc.offset)\n\t\t\tgotIndex, gotOffset, err := litestream.ParseWALSegmentFilenameV3(filename)\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"roundtrip failed for (%d, %d): %v\", tc.index, tc.offset, err)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif gotIndex != tc.index || gotOffset != tc.offset {\n\t\t\t\tt.Errorf(\"roundtrip: FormatWALSegmentFilenameV3(%d, %d) -> %q -> (%d, %d)\",\n\t\t\t\t\ttc.index, tc.offset, filename, gotIndex, gotOffset)\n\t\t\t}\n\t\t}\n\t})\n}\n" }, { "path": "vfs.go", "content": "//go:build vfs\n// +build vfs\n\npackage litestream\n\nimport (\n\t\"context\"\n\t\"crypto/rand\"\n\t\"errors\"\n\t\"fmt\"\n\t\"hash/fnv\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"slices\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\t_ \"unsafe\"\n\n\tlru \"github.com/hashicorp/golang-lru/v2\"\n\t\"github.com/markusmobius/go-dateparser\"\n\t\"github.com/psanford/sqlite3vfs\"\n\t\"github.com/superfly/ltx\"\n)\n\nconst (\n\tDefaultPollInterval = 1 * time.Second\n\tDefaultCacheSize = 10 * 1024 * 1024 // 10MB\n\tDefaultPageSize = 4096 // SQLite default page size\n\n\tpageFetchRetryAttempts = 6\n\tpageFetchRetryDelay = 15 * time.Millisecond\n)\n\n// ErrConflict is returned when the remote replica has newer transactions than expected.\nvar ErrConflict = errors.New(\"remote has newer transactions than expected\")\n\nvar (\n\t//go:linkname sqlite3vfsFileMap github.com/psanford/sqlite3vfs.fileMap\n\tsqlite3vfsFileMap map[uint64]sqlite3vfs.File\n\n\t//go:linkname sqlite3vfsFileMux github.com/psanford/sqlite3vfs.fileMux\n\tsqlite3vfsFileMux sync.Mutex\n\n\tvfsConnectionMap sync.Map // map[uintptr]uint64\n)\n\n// VFS implements the SQLite VFS interface for Litestream.\n// It is intended to be used for read replicas that read directly from S3.\n// When WriteEnabled is true, also supports writes with periodic sync.\ntype VFS struct {\n\tclient ReplicaClient\n\tlogger *slog.Logger\n\n\t// PollInterval is the interval at which to poll the replica client for new\n\t// LTX files. The index will be fetched for the new files automatically.\n\tPollInterval time.Duration\n\n\t// CacheSize is the maximum size of the page cache in bytes.\n\tCacheSize int\n\n\t// WriteEnabled activates write support for the VFS.\n\tWriteEnabled bool\n\n\t// WriteSyncInterval is how often to sync dirty pages to remote storage.\n\t// If zero, defaults to DefaultSyncInterval (1 second).\n\tWriteSyncInterval time.Duration\n\n\t// WriteBufferPath is the path for local write buffer persistence.\n\t// If empty, uses a temp file.\n\tWriteBufferPath string\n\n\t// HydrationEnabled activates background hydration of the database to a local file.\n\t// When enabled, the VFS will restore the database in the background and serve\n\t// reads from the local file once complete, eliminating remote fetch latency.\n\tHydrationEnabled bool\n\n\t// HydrationPath is the file path for local hydration file.\n\t// If empty and HydrationEnabled is true, a temp file will be used.\n\tHydrationPath string\n\n\t// CompactionEnabled activates background compaction for the VFS.\n\t// Requires WriteEnabled to be true.\n\tCompactionEnabled bool\n\n\t// CompactionLevels defines the compaction intervals for each level.\n\t// If nil, uses default compaction levels.\n\tCompactionLevels CompactionLevels\n\n\t// SnapshotInterval is how often to create full database snapshots.\n\t// Set to 0 to disable automatic snapshots.\n\tSnapshotInterval time.Duration\n\n\t// SnapshotRetention is how long to keep old snapshots.\n\t// Set to 0 to keep all snapshots.\n\tSnapshotRetention time.Duration\n\n\t// L0Retention is how long to keep L0 files after compaction into L1.\n\t// Set to 0 to delete immediately after compaction.\n\tL0Retention time.Duration\n\n\twriteMu sync.Mutex\n\twriteFile *VFSFile // current RESERVED lock holder (nil if none)\n\tlastSyncedTXID ltx.TXID // highest TXID synced by any local connection\n\twriteSeq uint64 // atomic counter for unique buffer paths\n\n\ttempDirOnce sync.Once\n\ttempDir string\n\ttempDirErr error\n\ttempFiles sync.Map // canonical name -> absolute path\n\ttempNames sync.Map // canonical name -> struct{}{}\n}\n\nfunc NewVFS(client ReplicaClient, logger *slog.Logger) *VFS {\n\treturn &VFS{\n\t\tclient: client,\n\t\tlogger: logger.With(\"vfs\", \"true\"),\n\t\tPollInterval: DefaultPollInterval,\n\t\tCacheSize: DefaultCacheSize,\n\t}\n}\n\nfunc (vfs *VFS) Open(name string, flags sqlite3vfs.OpenFlag) (sqlite3vfs.File, sqlite3vfs.OpenFlag, error) {\n\tslog.Debug(\"opening file\", \"name\", name, \"flags\", flags)\n\n\tswitch {\n\tcase flags&sqlite3vfs.OpenMainDB != 0:\n\t\treturn vfs.openMainDB(name, flags)\n\tcase vfs.requiresTempFile(flags):\n\t\treturn vfs.openTempFile(name, flags)\n\tdefault:\n\t\treturn nil, flags, sqlite3vfs.CantOpenError\n\t}\n}\n\nfunc (vfs *VFS) openMainDB(name string, flags sqlite3vfs.OpenFlag) (sqlite3vfs.File, sqlite3vfs.OpenFlag, error) {\n\tf := NewVFSFile(vfs.client, name, vfs.logger.With(\"name\", name))\n\tf.PollInterval = vfs.PollInterval\n\tf.CacheSize = vfs.CacheSize\n\tf.vfs = vfs // Store reference to parent VFS for config access\n\n\t// Initialize write support if enabled\n\tif vfs.WriteEnabled {\n\t\tf.writeEnabled = true\n\t\tf.dirty = make(map[uint32]int64)\n\t\tf.syncInterval = vfs.WriteSyncInterval\n\t\tif f.syncInterval == 0 {\n\t\t\tf.syncInterval = DefaultSyncInterval\n\t\t}\n\n\t\twriteSeq := atomic.AddUint64(&vfs.writeSeq, 1)\n\t\tif vfs.WriteBufferPath != \"\" {\n\t\t\tif writeSeq == 1 {\n\t\t\t\tf.bufferPath = vfs.WriteBufferPath\n\t\t\t} else {\n\t\t\t\tf.bufferPath = vfs.WriteBufferPath + \".\" + strconv.FormatUint(writeSeq, 10)\n\t\t\t}\n\t\t} else {\n\t\t\tdir, err := vfs.ensureTempDir()\n\t\t\tif err != nil {\n\t\t\t\treturn nil, 0, fmt.Errorf(\"create temp dir for write buffer: %w\", err)\n\t\t\t}\n\t\t\tf.bufferPath = filepath.Join(dir, \"write-buffer-\"+strconv.FormatUint(writeSeq, 10))\n\t\t}\n\n\t\t// Initialize compaction if enabled\n\t\tif vfs.CompactionEnabled {\n\t\t\tf.compactor = NewCompactor(vfs.client, f.logger)\n\t\t\t// VFS has no local files, so leave LocalFileOpener/LocalFileDeleter nil\n\t\t}\n\t}\n\n\t// Initialize hydration support if enabled\n\tif vfs.HydrationEnabled {\n\t\tif vfs.HydrationPath != \"\" {\n\t\t\tf.hydrationPath = vfs.HydrationPath\n\t\t\tf.hydrationPersistent = true\n\t\t} else {\n\t\t\t// Use a temp file if no path specified\n\t\t\tdir, err := vfs.ensureTempDir()\n\t\t\tif err != nil {\n\t\t\t\treturn nil, 0, fmt.Errorf(\"create temp dir for hydration: %w\", err)\n\t\t\t}\n\t\t\tf.hydrationPath = filepath.Join(dir, \"hydration.db\")\n\t\t}\n\t}\n\n\tif err := f.Open(); err != nil {\n\t\treturn nil, 0, err\n\t}\n\n\tif vfs.WriteEnabled {\n\t\tvfs.writeMu.Lock()\n\t\tif f.expectedTXID > vfs.lastSyncedTXID {\n\t\t\tvfs.lastSyncedTXID = f.expectedTXID\n\t\t}\n\t\tvfs.writeMu.Unlock()\n\t}\n\n\t// When SQLite requests read-write access, always report ReadWrite in the\n\t// output flags so that cold enable via PRAGMA litestream_write_enabled\n\t// works. SQLite permanently marks databases as read-only based on the\n\t// output flags from xOpen (pager.c:readOnly, btree.c:BTS_READ_ONLY),\n\t// which would prevent write transactions even after enabling writes at\n\t// runtime. Read-only enforcement happens at the VFS layer (WriteAt,\n\t// Truncate, Lock) when writeEnabled is false.\n\t//\n\t// If the caller explicitly requested read-only, we respect that intent.\n\tif flags&sqlite3vfs.OpenReadOnly == 0 {\n\t\tflags &^= sqlite3vfs.OpenReadOnly\n\t\tflags |= sqlite3vfs.OpenReadWrite\n\t}\n\n\treturn f, flags, nil\n}\n\nfunc (vfs *VFS) Delete(name string, dirSync bool) error {\n\tslog.Debug(\"deleting file\", \"name\", name, \"dirSync\", dirSync)\n\terr := vfs.deleteTempFile(name)\n\tif err == nil {\n\t\treturn nil\n\t}\n\tif errors.Is(err, os.ErrNotExist) {\n\t\treturn nil\n\t}\n\tif errors.Is(err, errTempFileNotFound) {\n\t\treturn fmt.Errorf(\"cannot delete vfs file\")\n\t}\n\treturn err\n}\n\nfunc (vfs *VFS) Access(name string, flag sqlite3vfs.AccessFlag) (bool, error) {\n\tslog.Debug(\"accessing file\", \"name\", name, \"flag\", flag)\n\n\tif strings.HasSuffix(name, \"-wal\") {\n\t\treturn vfs.accessWAL(name, flag)\n\t}\n\tif vfs.isTempFileName(name) {\n\t\treturn vfs.accessTempFile(name, flag)\n\t}\n\treturn false, nil\n}\n\nfunc (vfs *VFS) accessWAL(name string, flag sqlite3vfs.AccessFlag) (bool, error) {\n\treturn false, nil\n}\n\nfunc (vfs *VFS) FullPathname(name string) string {\n\tslog.Debug(\"full pathname\", \"name\", name)\n\treturn name\n}\n\nfunc (vfs *VFS) requiresTempFile(flags sqlite3vfs.OpenFlag) bool {\n\tconst tempMask = sqlite3vfs.OpenTempDB |\n\t\tsqlite3vfs.OpenTempJournal |\n\t\tsqlite3vfs.OpenSubJournal |\n\t\tsqlite3vfs.OpenSuperJournal |\n\t\tsqlite3vfs.OpenTransientDB |\n\t\tsqlite3vfs.OpenMainJournal\n\tif flags&tempMask != 0 {\n\t\treturn true\n\t}\n\treturn flags&sqlite3vfs.OpenDeleteOnClose != 0\n}\n\nfunc (vfs *VFS) ensureTempDir() (string, error) {\n\tvfs.tempDirOnce.Do(func() {\n\t\tdir, err := os.MkdirTemp(\"\", \"litestream-vfs-*\")\n\t\tif err != nil {\n\t\t\tvfs.tempDirErr = fmt.Errorf(\"create temp dir: %w\", err)\n\t\t\treturn\n\t\t}\n\t\tvfs.tempDir = dir\n\t})\n\treturn vfs.tempDir, vfs.tempDirErr\n}\n\nfunc (vfs *VFS) canonicalTempName(name string) string {\n\tif name == \"\" {\n\t\treturn \"\"\n\t}\n\tname = filepath.Clean(name)\n\tif name == \".\" || name == string(filepath.Separator) {\n\t\treturn \"\"\n\t}\n\treturn name\n}\n\nfunc tempFilenameFromCanonical(canonical string) (string, error) {\n\tbase := filepath.Base(canonical)\n\tif base == \".\" || base == string(filepath.Separator) {\n\t\treturn \"\", fmt.Errorf(\"invalid temp file name: %q\", canonical)\n\t}\n\n\th := fnv.New64a()\n\tif _, err := h.Write([]byte(canonical)); err != nil {\n\t\treturn \"\", fmt.Errorf(\"hash temp name: %w\", err)\n\t}\n\treturn fmt.Sprintf(\"%s-%016x\", base, h.Sum64()), nil\n}\n\nfunc (vfs *VFS) openTempFile(name string, flags sqlite3vfs.OpenFlag) (sqlite3vfs.File, sqlite3vfs.OpenFlag, error) {\n\tdir, err := vfs.ensureTempDir()\n\tif err != nil {\n\t\treturn nil, flags, err\n\t}\n\tdeleteOnClose := flags&sqlite3vfs.OpenDeleteOnClose != 0 || name == \"\"\n\tvar f *os.File\n\tvar onClose func()\n\tif name == \"\" {\n\t\tf, err = os.CreateTemp(dir, \"temp-*\")\n\t\tif err != nil {\n\t\t\treturn nil, flags, sqlite3vfs.CantOpenError\n\t\t}\n\t} else {\n\t\tcanonical := vfs.canonicalTempName(name)\n\t\tif canonical == \"\" {\n\t\t\treturn nil, flags, sqlite3vfs.CantOpenError\n\t\t}\n\t\tfname, err := tempFilenameFromCanonical(canonical)\n\t\tif err != nil {\n\t\t\treturn nil, flags, sqlite3vfs.CantOpenError\n\t\t}\n\t\tpath := filepath.Join(dir, fname)\n\t\tflag := openFlagToOSFlag(flags)\n\t\tif flag == 0 {\n\t\t\tflag = os.O_RDWR\n\t\t}\n\t\tf, err = os.OpenFile(path, flag|os.O_CREATE, 0o600)\n\t\tif err != nil {\n\t\t\treturn nil, flags, sqlite3vfs.CantOpenError\n\t\t}\n\t\tonClose = vfs.trackTempFile(canonical, path)\n\t}\n\n\treturn newLocalTempFile(f, deleteOnClose, onClose), flags, nil\n}\n\nfunc (vfs *VFS) deleteTempFile(name string) error {\n\tpath, ok := vfs.loadTempFilePath(name)\n\tif !ok {\n\t\tif vfs.wasTempFileName(name) {\n\t\t\tvfs.unregisterTempFile(name)\n\t\t\treturn os.ErrNotExist\n\t\t}\n\t\treturn errTempFileNotFound\n\t}\n\tif err := os.Remove(path); err != nil {\n\t\tif !os.IsNotExist(err) {\n\t\t\treturn err\n\t\t}\n\t}\n\tvfs.unregisterTempFile(name)\n\treturn nil\n}\n\nfunc (vfs *VFS) isTempFileName(name string) bool {\n\t_, ok := vfs.loadTempFilePath(name)\n\treturn ok\n}\n\nfunc (vfs *VFS) wasTempFileName(name string) bool {\n\tcanonical := vfs.canonicalTempName(name)\n\tif canonical == \"\" {\n\t\treturn false\n\t}\n\t_, ok := vfs.tempNames.Load(canonical)\n\treturn ok\n}\n\nfunc (vfs *VFS) unregisterTempFile(name string) {\n\tcanonical := vfs.canonicalTempName(name)\n\tif canonical == \"\" {\n\t\treturn\n\t}\n\tvfs.tempFiles.Delete(canonical)\n}\n\nfunc (vfs *VFS) accessTempFile(name string, flag sqlite3vfs.AccessFlag) (bool, error) {\n\tpath, ok := vfs.loadTempFilePath(name)\n\tif !ok {\n\t\treturn false, nil\n\t}\n\t_, err := os.Stat(path)\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\treturn false, nil\n\t\t}\n\t\treturn false, err\n\t}\n\treturn true, nil\n}\n\nfunc (vfs *VFS) trackTempFile(canonical, path string) func() {\n\tif canonical == \"\" {\n\t\treturn func() {}\n\t}\n\tvfs.tempFiles.Store(canonical, path)\n\tvfs.tempNames.Store(canonical, struct{}{})\n\treturn func() { vfs.tempFiles.Delete(canonical) }\n}\n\nfunc (vfs *VFS) loadTempFilePath(name string) (string, bool) {\n\tcanonical := vfs.canonicalTempName(name)\n\tif canonical == \"\" {\n\t\treturn \"\", false\n\t}\n\tif path, ok := vfs.tempFiles.Load(canonical); ok {\n\t\treturn path.(string), true\n\t}\n\treturn \"\", false\n}\n\nfunc openFlagToOSFlag(flag sqlite3vfs.OpenFlag) int {\n\tvar v int\n\tif flag&sqlite3vfs.OpenReadWrite != 0 {\n\t\tv |= os.O_RDWR\n\t} else if flag&sqlite3vfs.OpenReadOnly != 0 {\n\t\tv |= os.O_RDONLY\n\t}\n\tif flag&sqlite3vfs.OpenCreate != 0 {\n\t\tv |= os.O_CREATE\n\t}\n\tif flag&sqlite3vfs.OpenExclusive != 0 {\n\t\tv |= os.O_EXCL\n\t}\n\treturn v\n}\n\nvar errTempFileNotFound = fmt.Errorf(\"temp file not tracked\")\n\n// localTempFile fulfills sqlite3vfs.File solely for SQLite temp & transient files.\n// These files stay on the local filesystem and optionally delete themselves\n// when SQLite closes them (DeleteOnClose flag).\ntype localTempFile struct {\n\tf *os.File\n\tdeleteOnClose bool\n\tlockType atomic.Int32\n\tonClose func()\n}\n\nfunc newLocalTempFile(f *os.File, deleteOnClose bool, onClose func()) *localTempFile {\n\treturn &localTempFile{f: f, deleteOnClose: deleteOnClose, onClose: onClose}\n}\n\nfunc (tf *localTempFile) Close() error {\n\terr := tf.f.Close()\n\tif tf.deleteOnClose {\n\t\tif removeErr := os.Remove(tf.f.Name()); removeErr != nil && !os.IsNotExist(removeErr) && err == nil {\n\t\t\terr = removeErr\n\t\t}\n\t}\n\tif tf.onClose != nil {\n\t\ttf.onClose()\n\t}\n\treturn err\n}\n\nfunc (tf *localTempFile) ReadAt(p []byte, off int64) (n int, err error) {\n\treturn tf.f.ReadAt(p, off)\n}\n\nfunc (tf *localTempFile) WriteAt(b []byte, off int64) (n int, err error) {\n\treturn tf.f.WriteAt(b, off)\n}\n\nfunc (tf *localTempFile) Truncate(size int64) error {\n\treturn tf.f.Truncate(size)\n}\n\nfunc (tf *localTempFile) Sync(flag sqlite3vfs.SyncType) error {\n\treturn tf.f.Sync()\n}\n\nfunc (tf *localTempFile) FileSize() (int64, error) {\n\tinfo, err := tf.f.Stat()\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\treturn info.Size(), nil\n}\n\nfunc (tf *localTempFile) Lock(elock sqlite3vfs.LockType) error {\n\tif elock == sqlite3vfs.LockNone {\n\t\treturn nil\n\t}\n\ttf.lockType.Store(int32(elock))\n\treturn nil\n}\n\nfunc (tf *localTempFile) Unlock(elock sqlite3vfs.LockType) error {\n\ttf.lockType.Store(int32(elock))\n\treturn nil\n}\n\nfunc (tf *localTempFile) CheckReservedLock() (bool, error) {\n\treturn sqlite3vfs.LockType(tf.lockType.Load()) >= sqlite3vfs.LockReserved, nil\n}\n\nfunc (tf *localTempFile) SectorSize() int64 {\n\treturn 0\n}\n\nfunc (tf *localTempFile) DeviceCharacteristics() sqlite3vfs.DeviceCharacteristic {\n\treturn 0\n}\n\n// VFSFile implements the SQLite VFS file interface.\ntype VFSFile struct {\n\tmu sync.Mutex\n\tclient ReplicaClient\n\tname string\n\n\tpos ltx.Pos // Last TXID read from level 0 or 1\n\tmaxTXID1 ltx.TXID // Last TXID read from level 1\n\tindex map[uint32]ltx.PageIndexElem\n\tpending map[uint32]ltx.PageIndexElem\n\tpendingReplace bool\n\tcache *lru.Cache[uint32, []byte] // LRU cache for page data\n\ttargetTime *time.Time // Target view time; nil means latest\n\tlatestLTXTime time.Time // Timestamp of most recent LTX file\n\tlastPollSuccess time.Time // Time of last successful poll\n\tlockType sqlite3vfs.LockType // Current lock state\n\tpageSize uint32\n\tcommit uint32\n\n\t// Write support fields (only used when writeEnabled is true)\n\twriteEnabled bool // Whether write support is enabled\n\tdirty map[uint32]int64 // Dirty pages: pgno -> offset in buffer file\n\tpendingTXID ltx.TXID // Next TXID to use for sync\n\texpectedTXID ltx.TXID // Expected remote TXID (for conflict detection)\n\tbufferFile *os.File // Temp file for durability\n\tbufferPath string // Path to buffer file\n\tbufferNextOff int64 // Next write offset in buffer file\n\tsyncTicker *time.Ticker // Ticker for periodic sync\n\tsyncInterval time.Duration // Interval for periodic sync\n\tsyncStop chan struct{} // Signal to stop sync loop\n\tinTransaction bool // True during active write transaction\n\tdisabling bool // True when write disable is in progress\n\tcond *sync.Cond // Signals transaction state changes\n\n\thydrator *Hydrator // Background hydration (nil if disabled)\n\thydrationPath string // Path for hydration file (set during Open)\n\thydrationPersistent bool // True when using user-specified persistent path\n\n\twg sync.WaitGroup\n\tctx context.Context\n\tcancel context.CancelFunc\n\n\tlogger *slog.Logger\n\n\tPollInterval time.Duration\n\tCacheSize int\n\n\t// Compaction support (only used when VFS.CompactionEnabled is true)\n\tvfs *VFS // Reference back to parent VFS for config\n\tcompactor *Compactor // Shared compaction logic\n\tcompactionWg sync.WaitGroup\n\tcompactionCtx context.Context\n\tcompactionCancel context.CancelFunc\n}\n\n// Hydrator handles background hydration of the database to a local file.\ntype Hydrator struct {\n\tpath string // Full path to hydration file\n\tpersistent bool // True when file should survive across restarts\n\tfile *os.File // Local database file\n\tcomplete atomic.Bool // True when restore completes\n\ttxid ltx.TXID // TXID the hydrated file is at\n\tmu sync.Mutex // Protects hydration file writes\n\terr error // Stores fatal hydration error\n\tcompactor *ltx.Compactor // Tracks compaction progress during restore\n\tpageSize uint32 // Page size of the database\n\tclient ReplicaClient\n\tlogger *slog.Logger\n}\n\n// NewHydrator creates a new Hydrator instance.\nfunc NewHydrator(path string, persistent bool, pageSize uint32, client ReplicaClient, logger *slog.Logger) *Hydrator {\n\treturn &Hydrator{\n\t\tpath: path,\n\t\tpersistent: persistent,\n\t\tpageSize: pageSize,\n\t\tclient: client,\n\t\tlogger: logger,\n\t}\n}\n\n// Init opens or creates the hydration file.\nfunc (h *Hydrator) Init() error {\n\tif err := os.MkdirAll(filepath.Dir(h.path), 0755); err != nil {\n\t\treturn fmt.Errorf(\"create hydration directory: %w\", err)\n\t}\n\n\tif h.persistent {\n\t\tif txid, err := h.loadMeta(); err == nil {\n\t\t\tif _, statErr := os.Stat(h.path); statErr == nil {\n\t\t\t\tfile, err := os.OpenFile(h.path, os.O_RDWR, 0600)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"open persistent hydration file: %w\", err)\n\t\t\t\t}\n\t\t\t\th.file = file\n\t\t\t\th.txid = txid\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tif err := os.Remove(h.metaPath()); err != nil && !os.IsNotExist(err) {\n\t\t\treturn fmt.Errorf(\"remove stale hydration meta: %w\", err)\n\t\t}\n\t}\n\n\tfile, err := os.OpenFile(h.path, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create hydration file: %w\", err)\n\t}\n\th.file = file\n\treturn nil\n}\n\n// Complete returns true if hydration has completed.\nfunc (h *Hydrator) Complete() bool {\n\treturn h.complete.Load()\n}\n\n// SetComplete marks hydration as complete.\nfunc (h *Hydrator) SetComplete() {\n\th.complete.Store(true)\n}\n\n// Disable temporarily disables hydrated reads (used during time travel).\nfunc (h *Hydrator) Disable() {\n\th.complete.Store(false)\n}\n\n// TXID returns the current hydration TXID.\nfunc (h *Hydrator) TXID() ltx.TXID {\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\treturn h.txid\n}\n\n// SetTXID sets the hydration TXID.\nfunc (h *Hydrator) SetTXID(txid ltx.TXID) {\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\th.txid = txid\n}\n\n// Err returns any fatal hydration error.\nfunc (h *Hydrator) Err() error {\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\treturn h.err\n}\n\n// SetErr sets a fatal hydration error.\nfunc (h *Hydrator) SetErr(err error) {\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\th.err = err\n}\n\n// Status returns the current compaction progress during restore.\nfunc (h *Hydrator) Status() ltx.CompactorStatus {\n\tif h.compactor == nil {\n\t\treturn ltx.CompactorStatus{}\n\t}\n\treturn h.compactor.Status()\n}\n\n// Restore restores the database from LTX files to the hydration file.\nfunc (h *Hydrator) Restore(ctx context.Context, infos []*ltx.FileInfo) error {\n\t// Open all LTX files as readers\n\trdrs := make([]io.Reader, 0, len(infos))\n\tdefer func() {\n\t\tfor _, rd := range rdrs {\n\t\t\tif closer, ok := rd.(io.Closer); ok {\n\t\t\t\t_ = closer.Close()\n\t\t\t}\n\t\t}\n\t}()\n\n\tfor _, info := range infos {\n\t\th.logger.Debug(\"opening ltx file for hydration\", \"level\", info.Level, \"min\", info.MinTXID, \"max\", info.MaxTXID)\n\t\trc, err := h.client.OpenLTXFile(ctx, info.Level, info.MinTXID, info.MaxTXID, 0, 0)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"open ltx file: %w\", err)\n\t\t}\n\t\trdrs = append(rdrs, rc)\n\t}\n\n\tif len(rdrs) == 0 {\n\t\treturn fmt.Errorf(\"no ltx files for hydration\")\n\t}\n\n\t// Compact and decode using io.Pipe pattern\n\tpr, pw := io.Pipe()\n\tc, err := ltx.NewCompactor(pw, rdrs)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"new ltx compactor: %w\", err)\n\t}\n\tc.HeaderFlags = ltx.HeaderFlagNoChecksum\n\th.compactor = c\n\n\tgo func() {\n\t\t_ = pw.CloseWithError(c.Compact(ctx))\n\t}()\n\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\n\tdec := ltx.NewDecoder(pr)\n\tif err := dec.DecodeDatabaseTo(h.file); err != nil {\n\t\treturn fmt.Errorf(\"decode database: %w\", err)\n\t}\n\n\th.txid = infos[len(infos)-1].MaxTXID\n\treturn nil\n}\n\n// CatchUp applies updates from LTX files between fromTXID and toTXID.\nfunc (h *Hydrator) CatchUp(ctx context.Context, fromTXID, toTXID ltx.TXID) error {\n\th.logger.Debug(\"catching up hydration\", \"from\", fromTXID, \"to\", toTXID)\n\n\t// Fetch LTX files from fromTXID+1 to toTXID\n\titr, err := h.client.LTXFiles(ctx, 0, fromTXID+1, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"list ltx files for catch-up: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tif info.MaxTXID > toTXID {\n\t\t\tbreak\n\t\t}\n\n\t\tif err := h.ApplyLTX(ctx, info); err != nil {\n\t\t\treturn fmt.Errorf(\"apply ltx to hydrated file: %w\", err)\n\t\t}\n\n\t\th.mu.Lock()\n\t\th.txid = info.MaxTXID\n\t\th.mu.Unlock()\n\t}\n\n\treturn nil\n}\n\n// ApplyLTX fetches an entire LTX file and applies its pages to the hydration file.\nfunc (h *Hydrator) ApplyLTX(ctx context.Context, info *ltx.FileInfo) error {\n\th.logger.Debug(\"applying ltx to hydration file\", \"level\", info.Level, \"min\", info.MinTXID, \"max\", info.MaxTXID)\n\n\t// Fetch entire LTX file\n\trc, err := h.client.OpenLTXFile(ctx, info.Level, info.MinTXID, info.MaxTXID, 0, 0)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open ltx file: %w\", err)\n\t}\n\tdefer rc.Close()\n\n\tdec := ltx.NewDecoder(rc)\n\tif err := dec.DecodeHeader(); err != nil {\n\t\treturn fmt.Errorf(\"decode header: %w\", err)\n\t}\n\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\n\t// Apply each page to the hydration file\n\tfor {\n\t\tvar phdr ltx.PageHeader\n\t\tdata := make([]byte, h.pageSize)\n\t\tif err := dec.DecodePage(&phdr, data); err == io.EOF {\n\t\t\tbreak\n\t\t} else if err != nil {\n\t\t\treturn fmt.Errorf(\"decode page: %w\", err)\n\t\t}\n\n\t\toff := int64(phdr.Pgno-1) * int64(h.pageSize)\n\t\tif _, err := h.file.WriteAt(data, off); err != nil {\n\t\t\treturn fmt.Errorf(\"write page %d: %w\", phdr.Pgno, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// ReadAt reads data from the hydrated local file.\nfunc (h *Hydrator) ReadAt(p []byte, off int64) (int, error) {\n\th.mu.Lock()\n\tn, err := h.file.ReadAt(p, off)\n\th.mu.Unlock()\n\n\tif err != nil && err != io.EOF {\n\t\treturn n, fmt.Errorf(\"read hydrated file: %w\", err)\n\t}\n\n\t// Update the first page to pretend like we are in journal mode\n\tif off == 0 && len(p) >= 28 {\n\t\tp[18], p[19] = 0x01, 0x01\n\t\t_, _ = rand.Read(p[24:28])\n\t}\n\n\treturn n, nil\n}\n\n// ApplyUpdates fetches updated pages and writes them to the hydration file.\nfunc (h *Hydrator) ApplyUpdates(ctx context.Context, updates map[uint32]ltx.PageIndexElem) error {\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\n\tfor pgno, elem := range updates {\n\t\t_, data, err := FetchPage(ctx, h.client, elem.Level, elem.MinTXID, elem.MaxTXID, elem.Offset, elem.Size)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"fetch updated page %d: %w\", pgno, err)\n\t\t}\n\n\t\toff := int64(pgno-1) * int64(h.pageSize)\n\t\tif _, err := h.file.WriteAt(data, off); err != nil {\n\t\t\treturn fmt.Errorf(\"write updated page %d: %w\", pgno, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// WritePage writes a single page to the hydration file.\nfunc (h *Hydrator) WritePage(pgno uint32, data []byte) error {\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\n\toff := int64(pgno-1) * int64(h.pageSize)\n\tif _, err := h.file.WriteAt(data, off); err != nil {\n\t\treturn fmt.Errorf(\"write page %d to hydrated file: %w\", pgno, err)\n\t}\n\treturn nil\n}\n\n// Truncate truncates the hydration file to the specified size.\nfunc (h *Hydrator) Truncate(size int64) error {\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\treturn h.file.Truncate(size)\n}\n\n// Close closes the hydration file. For persistent hydrators, the file and a\n// companion .meta file are preserved so hydration can resume on the next open.\nfunc (h *Hydrator) Close() error {\n\tif h.file == nil {\n\t\treturn nil\n\t}\n\n\tif h.persistent && h.txid > 0 {\n\t\tif err := h.file.Sync(); err != nil {\n\t\t\th.logger.Warn(\"failed to sync hydration file\", \"error\", err)\n\t\t}\n\t\tif err := h.saveMeta(); err != nil {\n\t\t\th.logger.Warn(\"failed to save hydration meta\", \"error\", err)\n\t\t}\n\t\treturn h.file.Close()\n\t}\n\n\tif err := h.file.Close(); err != nil {\n\t\treturn err\n\t}\n\n\tif err := os.Remove(h.path); err != nil && !os.IsNotExist(err) {\n\t\treturn err\n\t}\n\tif err := os.Remove(h.metaPath()); err != nil && !os.IsNotExist(err) {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc (h *Hydrator) metaPath() string {\n\treturn h.path + \".meta\"\n}\n\nfunc (h *Hydrator) loadMeta() (ltx.TXID, error) {\n\tdata, err := os.ReadFile(h.metaPath())\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tv, err := strconv.ParseUint(strings.TrimSpace(string(data)), 10, 64)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"parse hydration meta: %w\", err)\n\t}\n\treturn ltx.TXID(v), nil\n}\n\nfunc (h *Hydrator) saveMeta() error {\n\th.mu.Lock()\n\ttxid := h.txid\n\th.mu.Unlock()\n\n\tdir := filepath.Dir(h.metaPath())\n\ttmp, err := os.CreateTemp(dir, \".hydration-meta-*\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create temp meta file: %w\", err)\n\t}\n\ttmpPath := tmp.Name()\n\n\tif _, err := fmt.Fprintf(tmp, \"%d\\n\", txid); err != nil {\n\t\tif closeErr := tmp.Close(); closeErr != nil {\n\t\t\th.logger.Warn(\"failed to close temp meta file during cleanup\", \"error\", closeErr)\n\t\t}\n\t\tif removeErr := os.Remove(tmpPath); removeErr != nil {\n\t\t\th.logger.Warn(\"failed to remove temp meta file during cleanup\", \"error\", removeErr)\n\t\t}\n\t\treturn fmt.Errorf(\"write hydration meta: %w\", err)\n\t}\n\tif err := tmp.Sync(); err != nil {\n\t\tif closeErr := tmp.Close(); closeErr != nil {\n\t\t\th.logger.Warn(\"failed to close temp meta file during cleanup\", \"error\", closeErr)\n\t\t}\n\t\tif removeErr := os.Remove(tmpPath); removeErr != nil {\n\t\t\th.logger.Warn(\"failed to remove temp meta file during cleanup\", \"error\", removeErr)\n\t\t}\n\t\treturn fmt.Errorf(\"sync temp meta file: %w\", err)\n\t}\n\tif err := tmp.Close(); err != nil {\n\t\tif removeErr := os.Remove(tmpPath); removeErr != nil {\n\t\t\th.logger.Warn(\"failed to remove temp meta file during cleanup\", \"error\", removeErr)\n\t\t}\n\t\treturn fmt.Errorf(\"close temp meta file: %w\", err)\n\t}\n\tif err := os.Rename(tmpPath, h.metaPath()); err != nil {\n\t\tif removeErr := os.Remove(tmpPath); removeErr != nil {\n\t\t\th.logger.Warn(\"failed to remove temp meta file during cleanup\", \"error\", removeErr)\n\t\t}\n\t\treturn fmt.Errorf(\"rename hydration meta: %w\", err)\n\t}\n\tif err := syncDir(filepath.Dir(h.metaPath())); err != nil {\n\t\treturn fmt.Errorf(\"sync hydration meta directory: %w\", err)\n\t}\n\treturn nil\n}\n\nfunc syncDir(path string) error {\n\tdir, err := os.Open(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer dir.Close()\n\treturn dir.Sync()\n}\n\nfunc NewVFSFile(client ReplicaClient, name string, logger *slog.Logger) *VFSFile {\n\tf := &VFSFile{\n\t\tclient: client,\n\t\tname: name,\n\t\tindex: make(map[uint32]ltx.PageIndexElem),\n\t\tpending: make(map[uint32]ltx.PageIndexElem),\n\t\tlogger: logger,\n\t\tPollInterval: DefaultPollInterval,\n\t\tCacheSize: DefaultCacheSize,\n\t}\n\tf.ctx, f.cancel = context.WithCancel(context.Background())\n\tf.cond = sync.NewCond(&f.mu)\n\treturn f\n}\n\n// Pos returns the current position of the file.\nfunc (f *VFSFile) Pos() ltx.Pos {\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\treturn f.pos\n}\n\n// MaxTXID1 returns the last TXID read from level 1.\nfunc (f *VFSFile) MaxTXID1() ltx.TXID {\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\treturn f.maxTXID1\n}\n\n// LockType returns the current lock type of the file.\nfunc (f *VFSFile) LockType() sqlite3vfs.LockType {\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\treturn f.lockType\n}\n\n// TargetTime returns the current target time for the VFS file (nil for latest).\nfunc (f *VFSFile) TargetTime() *time.Time {\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\tif f.targetTime == nil {\n\t\treturn nil\n\t}\n\tt := *f.targetTime\n\treturn &t\n}\n\n// LatestLTXTime returns the timestamp of the most recent LTX file.\nfunc (f *VFSFile) LatestLTXTime() time.Time {\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\treturn f.latestLTXTime\n}\n\n// LastPollSuccess returns the time of the last successful poll.\nfunc (f *VFSFile) LastPollSuccess() time.Time {\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\treturn f.lastPollSuccess\n}\n\nfunc (f *VFSFile) hasTargetTime() bool {\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\treturn f.targetTime != nil\n}\n\nfunc (f *VFSFile) Open() error {\n\tf.logger.Debug(\"opening file\")\n\n\t// Try to get restore plan. For write-enabled VFS, we can create a new database\n\t// if no LTX files exist yet.\n\tinfos, err := f.waitForRestorePlan()\n\tif err != nil {\n\t\t// If write mode is enabled and no files exist, we can create a new database\n\t\tif f.writeEnabled && errors.Is(err, ErrTxNotAvailable) {\n\t\t\tf.logger.Info(\"no existing database found, creating new database\")\n\t\t\treturn f.openNewDatabase()\n\t\t}\n\t\treturn err\n\t}\n\n\tpageSize, err := detectPageSizeFromInfos(f.ctx, f.client, infos)\n\tif err != nil {\n\t\tf.logger.Error(\"cannot detect page size\", \"error\", err)\n\t\treturn fmt.Errorf(\"detect page size: %w\", err)\n\t}\n\tf.pageSize = pageSize\n\n\t// Initialize page cache. Convert byte size to number of pages.\n\tcacheEntries := f.CacheSize / int(pageSize)\n\tif cacheEntries < 1 {\n\t\tcacheEntries = 1\n\t}\n\tcache, err := lru.New[uint32, []byte](cacheEntries)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create page cache: %w\", err)\n\t}\n\tf.cache = cache\n\n\t// Determine the current position based off the latest LTX file.\n\tvar pos ltx.Pos\n\tif len(infos) > 0 {\n\t\tpos = ltx.Pos{TXID: infos[len(infos)-1].MaxTXID}\n\t}\n\tf.pos = pos\n\n\t// Initialize write support TXID tracking\n\tif f.writeEnabled {\n\t\tf.expectedTXID = pos.TXID\n\t\tf.pendingTXID = pos.TXID + 1\n\t\tf.logger.Debug(\"write support enabled\", \"expectedTXID\", f.expectedTXID, \"pendingTXID\", f.pendingTXID)\n\n\t\t// Initialize write buffer file for durability (discards any existing buffer)\n\t\tif err := f.initWriteBuffer(); err != nil {\n\t\t\treturn fmt.Errorf(\"initialize write buffer: %w\", err)\n\t\t}\n\t}\n\n\t// Build the page index so we can lookup individual pages.\n\tif err := f.buildIndex(f.ctx, infos); err != nil {\n\t\tf.logger.Error(\"cannot build index\", \"error\", err)\n\t\treturn fmt.Errorf(\"cannot build index: %w\", err)\n\t}\n\n\t// Start background hydration if enabled\n\tif f.hydrationPath != \"\" {\n\t\tif err := f.initHydration(infos); err != nil {\n\t\t\tf.logger.Warn(\"hydration initialization failed, continuing without hydration\", \"error\", err)\n\t\t\tf.hydrationPath = \"\"\n\t\t}\n\t}\n\n\t// Continuously monitor the replica client for new LTX files.\n\tf.wg.Add(1)\n\tgo func() { defer f.wg.Done(); f.monitorReplicaClient(f.ctx) }()\n\n\t// Start periodic sync goroutine if write support is enabled\n\tif f.writeEnabled && f.syncInterval > 0 {\n\t\tf.syncTicker = time.NewTicker(f.syncInterval)\n\t\tf.syncStop = make(chan struct{})\n\t\tstopCh := f.syncStop\n\t\ttickerCh := f.syncTicker.C\n\t\tf.wg.Add(1)\n\t\tgo func() { defer f.wg.Done(); f.syncLoop(stopCh, tickerCh) }()\n\t}\n\n\t// Start compaction monitors if enabled\n\tif f.compactor != nil && f.vfs != nil {\n\t\tf.startCompactionMonitors()\n\t}\n\n\treturn nil\n}\n\n// openNewDatabase initializes the VFSFile for a brand new database with no existing data.\n// This is called when write mode is enabled and no LTX files exist yet.\nfunc (f *VFSFile) openNewDatabase() error {\n\tf.logger.Debug(\"initializing new database\")\n\n\t// Use default page size for new databases\n\tf.pageSize = DefaultPageSize\n\n\t// Initialize page cache\n\tcacheEntries := f.CacheSize / int(f.pageSize)\n\tif cacheEntries < 1 {\n\t\tcacheEntries = 1\n\t}\n\tcache, err := lru.New[uint32, []byte](cacheEntries)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"create page cache: %w\", err)\n\t}\n\tf.cache = cache\n\n\t// Initialize empty index - no pages exist yet\n\tf.index = make(map[uint32]ltx.PageIndexElem)\n\tf.pending = make(map[uint32]ltx.PageIndexElem)\n\tf.pos = ltx.Pos{TXID: 0}\n\tf.commit = 0\n\n\t// Initialize write support for new database\n\tf.expectedTXID = 0\n\tf.pendingTXID = 1\n\tf.logger.Debug(\"write support enabled for new database\", \"expectedTXID\", f.expectedTXID, \"pendingTXID\", f.pendingTXID)\n\n\t// Initialize write buffer file for durability\n\tif err := f.initWriteBuffer(); err != nil {\n\t\tf.logger.Warn(\"failed to initialize write buffer\", \"error\", err)\n\t}\n\n\t// Start monitoring for new LTX files (in case another writer creates the database)\n\tf.wg.Add(1)\n\tgo func() { defer f.wg.Done(); f.monitorReplicaClient(f.ctx) }()\n\n\t// Start periodic sync goroutine\n\tif f.syncInterval > 0 {\n\t\tf.syncTicker = time.NewTicker(f.syncInterval)\n\t\tf.syncStop = make(chan struct{})\n\t\tstopCh := f.syncStop\n\t\ttickerCh := f.syncTicker.C\n\t\tf.wg.Add(1)\n\t\tgo func() { defer f.wg.Done(); f.syncLoop(stopCh, tickerCh) }()\n\t}\n\n\t// Start compaction monitors if enabled\n\tif f.compactor != nil && f.vfs != nil {\n\t\tf.startCompactionMonitors()\n\t}\n\n\treturn nil\n}\n\n// SetTargetTime rebuilds the page index to view the database at a specific time.\nfunc (f *VFSFile) SetTargetTime(ctx context.Context, timestamp time.Time) error {\n\tif timestamp.IsZero() {\n\t\treturn fmt.Errorf(\"target time required\")\n\t}\n\n\tinfos, err := CalcRestorePlan(ctx, f.client, 0, timestamp, f.logger)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot calc restore plan: %w\", err)\n\t} else if len(infos) == 0 {\n\t\treturn fmt.Errorf(\"no backup files available\")\n\t}\n\n\t// Disable hydrated reads during time travel - hydrated file is at latest state\n\tif f.hydrator != nil && f.hydrator.Complete() {\n\t\tf.hydrator.Disable()\n\t\tf.logger.Debug(\"hydration disabled for time travel\", \"target\", timestamp)\n\t}\n\n\treturn f.rebuildIndex(ctx, infos, ×tamp)\n}\n\n// ResetTime rebuilds the page index to the latest available state.\nfunc (f *VFSFile) ResetTime(ctx context.Context) error {\n\tinfos, err := CalcRestorePlan(ctx, f.client, 0, time.Time{}, f.logger)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot calc restore plan: %w\", err)\n\t} else if len(infos) == 0 {\n\t\treturn fmt.Errorf(\"no backup files available\")\n\t}\n\n\treturn f.rebuildIndex(ctx, infos, nil)\n}\n\n// rebuildIndex constructs a fresh page index and swaps it into the VFSFile.\nfunc (f *VFSFile) rebuildIndex(ctx context.Context, infos []*ltx.FileInfo, target *time.Time) error {\n\tindex, err := f.buildIndexMap(ctx, infos)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar pos ltx.Pos\n\tif len(infos) > 0 {\n\t\tpos = ltx.Pos{TXID: infos[len(infos)-1].MaxTXID}\n\t}\n\n\tmaxTXID1 := maxLevelTXID(infos, 1)\n\t// Seed maxTXID1 from pos when there are no L1 files\n\tif maxTXID1 == 0 {\n\t\tmaxTXID1 = pos.TXID\n\t}\n\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\tf.index = index\n\tf.pending = make(map[uint32]ltx.PageIndexElem)\n\tf.pendingReplace = false\n\tf.pos = pos\n\tf.maxTXID1 = maxTXID1\n\tif len(infos) > 0 {\n\t\tf.latestLTXTime = infos[len(infos)-1].CreatedAt\n\t}\n\tif f.cache != nil {\n\t\tf.cache.Purge()\n\t}\n\tif target == nil {\n\t\tf.targetTime = nil\n\t} else {\n\t\tt := *target\n\t\tf.targetTime = &t\n\t}\n\n\treturn nil\n}\n\nfunc maxLevelTXID(infos []*ltx.FileInfo, level int) ltx.TXID {\n\tvar maxTXID ltx.TXID\n\tfor _, info := range infos {\n\t\tif info.Level == level && info.MaxTXID > maxTXID {\n\t\t\tmaxTXID = info.MaxTXID\n\t\t}\n\t}\n\treturn maxTXID\n}\n\n// buildIndexMap constructs a lookup of pgno to LTX file offsets.\nfunc (f *VFSFile) buildIndexMap(ctx context.Context, infos []*ltx.FileInfo) (map[uint32]ltx.PageIndexElem, error) {\n\tindex := make(map[uint32]ltx.PageIndexElem)\n\tvar commit uint32\n\tfor _, info := range infos {\n\t\tf.logger.Debug(\"opening page index\", \"level\", info.Level, \"min\", info.MinTXID, \"max\", info.MaxTXID)\n\n\t\t// Read page index.\n\t\tidx, err := FetchPageIndex(ctx, f.client, info)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"fetch page index: %w\", err)\n\t\t}\n\n\t\t// Replace pages in overall index with new pages.\n\t\tfor k, v := range idx {\n\t\t\tf.logger.Debug(\"adding page index\", \"page\", k, \"elem\", v)\n\t\t\tindex[k] = v\n\t\t}\n\t\thdr, err := FetchLTXHeader(ctx, f.client, info)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"fetch header: %w\", err)\n\t\t}\n\t\tcommit = hdr.Commit\n\t}\n\n\tf.mu.Lock()\n\tf.commit = commit\n\tf.mu.Unlock()\n\n\treturn index, nil\n}\n\n// buildIndex constructs a lookup of pgno to LTX file offsets (legacy wrapper).\nfunc (f *VFSFile) buildIndex(ctx context.Context, infos []*ltx.FileInfo) error {\n\treturn f.rebuildIndex(ctx, infos, nil)\n}\n\n// initHydration starts the background hydration process.\nfunc (f *VFSFile) initHydration(infos []*ltx.FileInfo) error {\n\tf.hydrator = NewHydrator(f.hydrationPath, f.hydrationPersistent, f.pageSize, f.client, f.logger)\n\tif err := f.hydrator.Init(); err != nil {\n\t\treturn err\n\t}\n\n\t// Start background restore\n\tf.wg.Add(1)\n\tgo f.runHydration(infos)\n\n\treturn nil\n}\n\n// runHydration performs the background hydration process.\nfunc (f *VFSFile) runHydration(infos []*ltx.FileInfo) {\n\tdefer f.wg.Done()\n\n\thydrationTXID := f.hydrator.TXID()\n\n\tf.mu.Lock()\n\tcurrentTXID := f.pos.TXID\n\tf.mu.Unlock()\n\n\tif hydrationTXID > 0 && currentTXID >= hydrationTXID {\n\t\tf.logger.Debug(\"resuming hydration from persistent file\", \"txid\", hydrationTXID.String())\n\t} else {\n\t\tif hydrationTXID > 0 {\n\t\t\tf.logger.Warn(\"remote TXID regressed, discarding persistent hydration\",\n\t\t\t\t\"hydration_txid\", hydrationTXID.String(),\n\t\t\t\t\"current_txid\", currentTXID.String())\n\t\t\tif err := f.hydrator.Truncate(0); err != nil {\n\t\t\t\tf.hydrator.SetErr(err)\n\t\t\t\tf.logger.Error(\"hydration truncate failed\", \"error\", err)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t\tif err := f.hydrator.Restore(f.ctx, infos); err != nil {\n\t\t\tf.hydrator.SetErr(err)\n\t\t\tf.logger.Error(\"hydration failed\", \"error\", err)\n\t\t\treturn\n\t\t}\n\t\thydrationTXID = f.hydrator.TXID()\n\t}\n\n\tif currentTXID > hydrationTXID {\n\t\tif err := f.hydrator.CatchUp(f.ctx, hydrationTXID, currentTXID); err != nil {\n\t\t\tf.hydrator.SetErr(err)\n\t\t\tf.logger.Error(\"hydration catch-up failed\", \"error\", err)\n\t\t\treturn\n\t\t}\n\t}\n\n\tf.hydrator.SetComplete()\n\n\t// Clear cache since we'll now read from hydration file\n\tf.cache.Purge()\n\n\tf.logger.Debug(\"hydration complete\", \"path\", f.hydrationPath, \"txid\", f.hydrator.TXID().String())\n}\n\n// applySyncedPagesToHydratedFile writes synced dirty pages to the hydrated file.\n// Must be called with f.mu held.\nfunc (f *VFSFile) applySyncedPagesToHydratedFile() error {\n\tfor pgno, bufferOff := range f.dirty {\n\t\tdata := make([]byte, f.pageSize)\n\t\tif _, err := f.bufferFile.ReadAt(data, bufferOff); err != nil {\n\t\t\treturn fmt.Errorf(\"read dirty page %d from buffer: %w\", pgno, err)\n\t\t}\n\n\t\tif err := f.hydrator.WritePage(pgno, data); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tf.hydrator.SetTXID(f.expectedTXID)\n\treturn nil\n}\n\nfunc (f *VFSFile) Close() error {\n\tf.logger.Debug(\"closing file\")\n\n\t// Stop sync loop and ticker if running (need mutex for syncStop)\n\tf.mu.Lock()\n\tif f.syncStop != nil {\n\t\tclose(f.syncStop)\n\t\tf.syncStop = nil\n\t}\n\tif f.syncTicker != nil {\n\t\tf.syncTicker.Stop()\n\t}\n\tf.mu.Unlock()\n\n\t// Stop compaction monitors if running\n\tif f.compactionCancel != nil {\n\t\tf.compactionCancel()\n\t\tf.compactionWg.Wait()\n\t}\n\n\t// Final sync of dirty pages before closing\n\tf.mu.Lock()\n\tif f.writeEnabled && len(f.dirty) > 0 {\n\t\tif err := f.syncToRemoteWithLock(); err != nil {\n\t\t\tf.logger.Error(\"failed to sync on close\", \"error\", err)\n\t\t}\n\t}\n\tf.mu.Unlock()\n\n\tf.cancel()\n\tf.wg.Wait()\n\n\t// Close and remove buffer file if open\n\tif f.bufferFile != nil {\n\t\tf.bufferFile.Close()\n\t\tos.Remove(f.bufferPath)\n\t}\n\n\t// Close and remove hydration file\n\tif f.hydrator != nil {\n\t\tif err := f.hydrator.Close(); err != nil {\n\t\t\tf.logger.Warn(\"failed to close hydration file\", \"error\", err)\n\t\t}\n\t}\n\n\tif f.writeEnabled && f.vfs != nil {\n\t\tf.vfs.writeMu.Lock()\n\t\tif f.vfs.writeFile == f {\n\t\t\tf.vfs.writeFile = nil\n\t\t}\n\t\tf.vfs.writeMu.Unlock()\n\t}\n\n\treturn nil\n}\n\nfunc (f *VFSFile) ReadAt(p []byte, off int64) (n int, err error) {\n\tf.logger.Debug(\"reading at\", \"off\", off, \"len\", len(p))\n\tpageSize, err := f.pageSizeBytes()\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tpgno := uint32(off/int64(pageSize)) + 1\n\tpageOffset := int(off % int64(pageSize))\n\n\t// Check dirty pages first (takes priority over cache and remote)\n\tf.mu.Lock()\n\tif f.writeEnabled {\n\t\tif bufferOff, ok := f.dirty[pgno]; ok {\n\t\t\t// Read page from buffer file\n\t\t\tdata := make([]byte, pageSize)\n\t\t\tif _, err := f.bufferFile.ReadAt(data, bufferOff); err != nil {\n\t\t\t\tf.mu.Unlock()\n\t\t\t\treturn 0, fmt.Errorf(\"read dirty page from buffer: %w\", err)\n\t\t\t}\n\t\t\tn = copy(p, data[pageOffset:])\n\t\t\tf.mu.Unlock()\n\t\t\tf.logger.Debug(\"dirty page hit\", \"page\", pgno, \"n\", n)\n\n\t\t\t// Update the first page to pretend like we are in journal mode.\n\t\t\tif off == 0 && len(p) >= 28 {\n\t\t\t\tp[18], p[19] = 0x01, 0x01\n\t\t\t\t_, _ = rand.Read(p[24:28])\n\t\t\t}\n\n\t\t\treturn n, nil\n\t\t}\n\t}\n\tf.mu.Unlock()\n\n\t// If hydration complete, read from local file\n\tif f.hydrator != nil && f.hydrator.Complete() {\n\t\treturn f.hydrator.ReadAt(p, off)\n\t}\n\n\t// Check cache (cache is thread-safe)\n\tif data, ok := f.cache.Get(pgno); ok {\n\t\tn = copy(p, data[pageOffset:])\n\t\tf.logger.Debug(\"cache hit\", \"page\", pgno, \"n\", n)\n\n\t\t// Update the first page to pretend like we are in journal mode.\n\t\tif off == 0 {\n\t\t\tp[18], p[19] = 0x01, 0x01\n\t\t\t_, _ = rand.Read(p[24:28])\n\t\t}\n\n\t\treturn n, nil\n\t}\n\n\t// Get page index element\n\tf.mu.Lock()\n\telem, ok := f.index[pgno]\n\twriteEnabled := f.writeEnabled // capture while holding lock to avoid data race\n\tf.mu.Unlock()\n\n\tif !ok {\n\t\t// For write-enabled VFS with a new database (no existing pages),\n\t\t// return zeros to indicate empty page. SQLite will initialize the database.\n\t\tif writeEnabled {\n\t\t\tf.logger.Debug(\"page not found, returning zeros for new database\", \"page\", pgno)\n\t\t\tfor i := range p {\n\t\t\t\tp[i] = 0\n\t\t\t}\n\t\t\treturn len(p), nil\n\t\t}\n\t\tf.logger.Error(\"page not found\", \"page\", pgno)\n\t\treturn 0, fmt.Errorf(\"page not found: %d\", pgno)\n\t}\n\n\tvar data []byte\n\tvar lastErr error\n\tctx := f.ctx\n\tfor attempt := 0; attempt < pageFetchRetryAttempts; attempt++ {\n\t\t_, data, lastErr = FetchPage(ctx, f.client, elem.Level, elem.MinTXID, elem.MaxTXID, elem.Offset, elem.Size)\n\t\tif lastErr == nil {\n\t\t\tbreak\n\t\t}\n\t\tif !isRetryablePageError(lastErr) {\n\t\t\tf.logger.Error(\"cannot fetch page\", \"page\", pgno, \"attempt\", attempt+1, \"error\", lastErr)\n\t\t\treturn 0, fmt.Errorf(\"fetch page: %w\", lastErr)\n\t\t}\n\n\t\tif attempt == pageFetchRetryAttempts-1 {\n\t\t\tf.logger.Error(\"cannot fetch page after retries\", \"page\", pgno, \"attempts\", pageFetchRetryAttempts, \"error\", lastErr)\n\t\t\treturn 0, sqlite3vfs.BusyError\n\t\t}\n\n\t\tdelay := pageFetchRetryDelay * time.Duration(attempt+1)\n\t\tf.logger.Warn(\"transient page fetch error, retrying\", \"page\", pgno, \"attempt\", attempt+1, \"delay\", delay, \"error\", lastErr)\n\n\t\ttimer := time.NewTimer(delay)\n\t\tselect {\n\t\tcase <-timer.C:\n\t\tcase <-f.ctx.Done():\n\t\t\ttimer.Stop()\n\t\t\treturn 0, fmt.Errorf(\"fetch page: %w\", lastErr)\n\t\t}\n\t\ttimer.Stop()\n\t}\n\n\t// Add to cache (cache is thread-safe)\n\tf.cache.Add(pgno, data)\n\n\tn = copy(p, data[pageOffset:])\n\tf.logger.Debug(\"data read from storage\", \"page\", pgno, \"n\", n, \"data\", len(data))\n\n\t// Update the first page to pretend like we are in journal mode.\n\tif off == 0 {\n\t\tp[18], p[19] = 0x01, 0x01\n\t\t_, _ = rand.Read(p[24:28])\n\t}\n\n\treturn n, nil\n}\n\nfunc (f *VFSFile) WriteAt(b []byte, off int64) (n int, err error) {\n\tf.logger.Debug(\"write at\", \"off\", off, \"len\", len(b))\n\n\tpageSize, err := f.pageSizeBytes()\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\t// Calculate page number and offset within page\n\tpgno := uint32(off/int64(pageSize)) + 1\n\tpageOffset := int(off % int64(pageSize))\n\n\t// Skip lock page\n\tif pgno == ltx.LockPgno(pageSize) {\n\t\treturn 0, fmt.Errorf(\"cannot write to lock page\")\n\t}\n\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\n\t// If write support is not enabled, return read-only error\n\tif !f.writeEnabled {\n\t\treturn 0, sqlite3vfs.ReadOnlyError\n\t}\n\n\t// Get page data - either from buffer file (if dirty) or from cache/remote\n\tpage := make([]byte, pageSize)\n\tif bufferOff, ok := f.dirty[pgno]; ok {\n\t\t// Page is already dirty - read from buffer file\n\t\tif _, err := f.bufferFile.ReadAt(page, bufferOff); err != nil {\n\t\t\treturn 0, fmt.Errorf(\"read dirty page from buffer: %w\", err)\n\t\t}\n\t} else {\n\t\t// Page is not dirty - read from cache/remote\n\t\tif err := f.readPageForWrite(pgno, page); err != nil {\n\t\t\t// If page doesn't exist, use zero-filled page\n\t\t\tf.logger.Debug(\"page not found, using empty page\", \"pgno\", pgno)\n\t\t}\n\t}\n\n\t// Apply write to page\n\tn = copy(page[pageOffset:], b)\n\n\t// Update commit count if this extends the database\n\tif pgno > f.commit {\n\t\tf.commit = pgno\n\t}\n\n\t// Write to buffer for durability (this updates f.dirty with the offset)\n\tif err := f.writeToBuffer(pgno, page); err != nil {\n\t\tf.logger.Error(\"failed to write to buffer\", \"error\", err)\n\t\treturn 0, fmt.Errorf(\"write to buffer: %w\", err)\n\t}\n\n\tf.logger.Debug(\"wrote to dirty page\", \"pgno\", pgno, \"offset\", pageOffset, \"len\", n, \"commit\", f.commit)\n\treturn n, nil\n}\n\n// readPageForWrite reads a page into buf for modification.\n// Must be called with f.mu held.\nfunc (f *VFSFile) readPageForWrite(pgno uint32, buf []byte) error {\n\tpageSize := uint32(len(buf))\n\n\t// Check cache first (cache is thread-safe, but we hold the lock anyway)\n\tif data, ok := f.cache.Get(pgno); ok {\n\t\tcopy(buf, data)\n\t\treturn nil\n\t}\n\n\t// Get page index element\n\telem, ok := f.index[pgno]\n\tif !ok {\n\t\treturn fmt.Errorf(\"page not found: %d\", pgno)\n\t}\n\n\t// Fetch from remote\n\t_, data, err := FetchPage(f.ctx, f.client, elem.Level, elem.MinTXID, elem.MaxTXID, elem.Offset, elem.Size)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif uint32(len(data)) != pageSize {\n\t\treturn fmt.Errorf(\"page size mismatch: got %d, expected %d\", len(data), pageSize)\n\t}\n\n\tcopy(buf, data)\n\tf.cache.Add(pgno, data)\n\treturn nil\n}\n\nfunc (f *VFSFile) Truncate(size int64) error {\n\tf.logger.Debug(\"truncating file\", \"size\", size)\n\n\tpageSize, err := f.pageSizeBytes()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tnewCommit := uint32(size / int64(pageSize))\n\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\n\t// If write support is not enabled, return read-only error\n\tif !f.writeEnabled {\n\t\treturn sqlite3vfs.ReadOnlyError\n\t}\n\n\t// Remove dirty pages beyond new size\n\tfor pgno := range f.dirty {\n\t\tif pgno > newCommit {\n\t\t\tdelete(f.dirty, pgno)\n\t\t}\n\t}\n\n\tf.commit = newCommit\n\tf.logger.Debug(\"truncated\", \"newCommit\", newCommit)\n\n\t// Truncate hydrated file if hydration is complete\n\tif f.hydrator != nil && f.hydrator.Complete() {\n\t\tif err := f.hydrator.Truncate(size); err != nil {\n\t\t\tf.logger.Error(\"failed to truncate hydration file\", \"error\", err)\n\t\t\t// Don't fail the operation - continue with degraded performance\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (f *VFSFile) Sync(flag sqlite3vfs.SyncType) error {\n\tf.logger.Debug(\"syncing file\", \"flag\", flag)\n\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\n\t// If write support is not enabled, no-op\n\tif !f.writeEnabled {\n\t\treturn nil\n\t}\n\n\t// Skip sync if no dirty pages\n\tif len(f.dirty) == 0 {\n\t\treturn nil\n\t}\n\t// Skip sync during active transaction\n\tif f.inTransaction {\n\t\tf.logger.Debug(\"skipping sync during transaction\")\n\t\treturn nil\n\t}\n\n\treturn f.syncToRemoteWithLock()\n}\n\n// SetWriteEnabled enables or disables write support at runtime.\n// This is equivalent to SetWriteEnabledWithTimeout(enabled, 0) which waits indefinitely.\nfunc (f *VFSFile) SetWriteEnabled(enabled bool) error {\n\treturn f.SetWriteEnabledWithTimeout(enabled, 0)\n}\n\n// SetWriteEnabledWithTimeout enables or disables write support at runtime with an optional timeout.\n//\n// When disabling (enabled=false):\n// - If called during an active transaction, waits for completion\n// - If timeout > 0, returns error after timeout if transaction doesn't complete\n// - If timeout == 0, waits indefinitely (or until context cancellation)\n// - Syncs all dirty pages to replica before returning\n// - Returns error if sync fails (writes remain enabled)\n// - Stops the sync ticker if running\n//\n// When enabling (enabled=true):\n// - Initializes buffer file if not already present (cold enable)\n// - Starts sync ticker using DefaultSyncInterval if not configured\n// - Starts sync ticker if syncInterval > 0 and not already running\nfunc (f *VFSFile) SetWriteEnabledWithTimeout(enabled bool, timeout time.Duration) error {\n\tf.mu.Lock()\n\n\t// No-op if already in the requested state\n\tif f.writeEnabled == enabled {\n\t\tf.mu.Unlock()\n\t\treturn nil\n\t}\n\n\tif !enabled {\n\t\t// DISABLING writes\n\n\t\t// Set disabling flag to prevent new transactions from starting\n\t\tf.disabling = true\n\n\t\t// Start goroutine to wake us on context cancellation or timeout\n\t\t// This ensures we don't block forever if context is cancelled or timeout expires\n\t\twaitDone := make(chan struct{})\n\t\tvar timeoutCh <-chan time.Time\n\t\tif timeout > 0 {\n\t\t\ttimer := time.NewTimer(timeout)\n\t\t\tdefer timer.Stop()\n\t\t\ttimeoutCh = timer.C\n\t\t}\n\t\tgo func() {\n\t\t\tselect {\n\t\t\tcase <-f.ctx.Done():\n\t\t\t\tf.cond.Broadcast() // Wake cond.Wait()\n\t\t\tcase <-timeoutCh:\n\t\t\t\tf.cond.Broadcast() // Wake cond.Wait() on timeout\n\t\t\tcase <-waitDone:\n\t\t\t\t// Normal completion, nothing to do\n\t\t\t}\n\t\t}()\n\n\t\t// Wait for active transaction to complete\n\t\tdeadline := time.Now().Add(timeout)\n\t\tfor f.inTransaction {\n\t\t\t// Check context before waiting\n\t\t\tselect {\n\t\t\tcase <-f.ctx.Done():\n\t\t\t\tclose(waitDone)\n\t\t\t\tf.disabling = false\n\t\t\t\tf.cond.Broadcast() // Wake any waiting Lock() calls\n\t\t\t\tf.mu.Unlock()\n\t\t\t\treturn fmt.Errorf(\"context cancelled while waiting for transaction: %w\", f.ctx.Err())\n\t\t\tdefault:\n\t\t\t}\n\t\t\t// Check timeout if specified\n\t\t\tif timeout > 0 && time.Now().After(deadline) {\n\t\t\t\tclose(waitDone)\n\t\t\t\tf.disabling = false\n\t\t\t\tf.cond.Broadcast() // Wake any waiting Lock() calls\n\t\t\t\tf.mu.Unlock()\n\t\t\t\treturn fmt.Errorf(\"timeout waiting for transaction to complete (waited %v)\", timeout)\n\t\t\t}\n\t\t\tf.cond.Wait() // Unlocks mu, waits for signal, relocks mu\n\t\t}\n\t\tclose(waitDone) // Stop the watcher goroutine\n\n\t\t// Sync dirty pages if any exist\n\t\tif len(f.dirty) > 0 {\n\t\t\tif err := f.syncToRemoteWithLock(); err != nil {\n\t\t\t\tf.disabling = false\n\t\t\t\tf.cond.Broadcast() // Wake any waiting Lock() calls\n\t\t\t\tf.mu.Unlock()\n\t\t\t\treturn fmt.Errorf(\"sync before disable: %w\", err)\n\t\t\t}\n\t\t}\n\n\t\t// Stop sync loop and ticker if running\n\t\tif f.syncStop != nil {\n\t\t\tclose(f.syncStop)\n\t\t\tf.syncStop = nil\n\t\t}\n\t\tif f.syncTicker != nil {\n\t\t\tf.syncTicker.Stop()\n\t\t\tf.syncTicker = nil\n\t\t}\n\n\t\tf.writeEnabled = false\n\t\tf.disabling = false\n\t\tf.cond.Broadcast() // Wake any Lock() calls waiting for disable to complete\n\t\tf.logger.Info(\"write support disabled\")\n\t\tf.mu.Unlock()\n\t\treturn nil\n\t}\n\n\t// ENABLING writes (cold enable supported)\n\n\t// Initialize dirty map if not present\n\tif f.dirty == nil {\n\t\tf.dirty = make(map[uint32]int64)\n\t}\n\n\t// Set sync interval from VFS config if not set, falling back to default\n\t// This mirrors the startup behavior where WriteSyncInterval==0 uses DefaultSyncInterval\n\tif f.syncInterval == 0 && f.vfs != nil {\n\t\tf.syncInterval = f.vfs.WriteSyncInterval\n\t\tif f.syncInterval == 0 {\n\t\t\tf.syncInterval = DefaultSyncInterval\n\t\t}\n\t}\n\n\t// Set buffer path if not set\n\tif f.bufferPath == \"\" {\n\t\tif f.vfs != nil && f.vfs.WriteBufferPath != \"\" {\n\t\t\tf.bufferPath = f.vfs.WriteBufferPath\n\t\t} else if f.vfs != nil {\n\t\t\t// Use VFS temp directory\n\t\t\tdir, err := f.vfs.ensureTempDir()\n\t\t\tif err != nil {\n\t\t\t\tf.mu.Unlock()\n\t\t\t\treturn fmt.Errorf(\"create temp dir for write buffer: %w\", err)\n\t\t\t}\n\t\t\tf.bufferPath = filepath.Join(dir, \"write-buffer\")\n\t\t} else {\n\t\t\t// Fallback to os.TempDir() for cold enable without VFS reference\n\t\t\tf.bufferPath = filepath.Join(os.TempDir(), \"litestream-write-buffer\")\n\t\t}\n\t}\n\n\t// Initialize buffer file if not present\n\tif f.bufferFile == nil {\n\t\tif err := f.initWriteBufferWithLock(); err != nil {\n\t\t\tf.mu.Unlock()\n\t\t\treturn fmt.Errorf(\"init write buffer: %w\", err)\n\t\t}\n\t}\n\n\t// Initialize write tracking state if this is a cold enable\n\tif f.pendingTXID == 0 {\n\t\tf.expectedTXID = f.pos.TXID\n\t\tf.pendingTXID = f.pos.TXID + 1\n\t}\n\n\t// Start sync ticker if not running and interval > 0\n\t// (syncInterval == 0 means no periodic sync, only manual Sync() calls)\n\tif f.syncTicker == nil && f.syncInterval > 0 {\n\t\tf.syncTicker = time.NewTicker(f.syncInterval)\n\t\tf.syncStop = make(chan struct{})\n\t\tstopCh := f.syncStop\n\t\ttickerCh := f.syncTicker.C\n\t\tf.wg.Add(1)\n\t\tgo func() { defer f.wg.Done(); f.syncLoop(stopCh, tickerCh) }()\n\t}\n\n\tf.writeEnabled = true\n\tf.logger.Info(\"write support enabled\", \"pendingTXID\", f.pendingTXID)\n\tf.mu.Unlock()\n\treturn nil\n}\n\n// syncLoop runs periodic sync in the background.\n// The stop channel and ticker channel are passed as parameters to avoid races\n// with SetWriteEnabled potentially nilling them out before this goroutine starts.\nfunc (f *VFSFile) syncLoop(stopCh <-chan struct{}, tickerCh <-chan time.Time) {\n\tf.logger.Debug(\"starting sync loop\", \"interval\", f.syncInterval)\n\n\tfor {\n\t\tselect {\n\t\tcase <-f.ctx.Done():\n\t\t\tf.logger.Debug(\"sync loop stopped (context cancelled)\")\n\t\t\treturn\n\t\tcase <-stopCh:\n\t\t\tf.logger.Debug(\"sync loop stopped (write disabled)\")\n\t\t\treturn\n\t\tcase <-tickerCh:\n\t\t\tif err := f.Sync(0); err != nil {\n\t\t\t\tf.logger.Error(\"periodic sync failed\", \"error\", err)\n\t\t\t}\n\t\t}\n\t}\n}\n\n// syncToRemote syncs dirty pages to the remote replica.\n// This function acquires f.mu internally.\nfunc (f *VFSFile) syncToRemote() error {\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\treturn f.syncToRemoteWithLock()\n}\n\n// syncToRemoteWithLock syncs dirty pages to the remote replica.\n// Caller must hold f.mu.\nfunc (f *VFSFile) syncToRemoteWithLock() error {\n\t// Double-check dirty pages exist\n\tif len(f.dirty) == 0 {\n\t\treturn nil\n\t}\n\n\tctx := f.ctx\n\n\t// Check for conflicts\n\tif err := f.checkForConflict(ctx); err != nil {\n\t\treturn err\n\t}\n\n\t// Create LTX file from dirty pages\n\tltxReader := f.createLTXFromDirty()\n\n\t// Upload LTX file to remote\n\tinfo, err := f.client.WriteLTXFile(ctx, 0, f.pendingTXID, f.pendingTXID, ltxReader)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"upload LTX: %w\", err)\n\t}\n\n\tf.logger.Info(\"synced to remote\",\n\t\t\"txid\", info.MaxTXID,\n\t\t\"pages\", len(f.dirty),\n\t\t\"size\", info.Size)\n\n\tf.expectedTXID = f.pendingTXID\n\tf.pendingTXID++\n\tf.pos = ltx.Pos{TXID: f.expectedTXID}\n\n\tif f.vfs != nil {\n\t\tf.vfs.writeMu.Lock()\n\t\tif f.expectedTXID > f.vfs.lastSyncedTXID {\n\t\t\tf.vfs.lastSyncedTXID = f.expectedTXID\n\t\t}\n\t\tf.vfs.writeMu.Unlock()\n\t}\n\n\t// Update cache with synced pages (index will be populated naturally when pages are fetched)\n\tfor pgno, bufferOff := range f.dirty {\n\t\tcachedData := make([]byte, f.pageSize)\n\t\tif _, err := f.bufferFile.ReadAt(cachedData, bufferOff); err != nil {\n\t\t\treturn fmt.Errorf(\"read page %d from buffer for cache: %w\", pgno, err)\n\t\t}\n\t\tf.cache.Add(pgno, cachedData)\n\t}\n\n\t// Apply synced pages to hydrated file if hydration is complete\n\t// Must be done before clearing f.dirty since we need the page offsets\n\tif f.hydrator != nil && f.hydrator.Complete() {\n\t\tif err := f.applySyncedPagesToHydratedFile(); err != nil {\n\t\t\tf.logger.Error(\"failed to apply synced pages to hydrated file\", \"error\", err)\n\t\t\t// Don't fail the sync - hydration will catch up on next poll\n\t\t}\n\t}\n\n\t// Clear dirty pages\n\tf.dirty = make(map[uint32]int64)\n\n\t// Clear write buffer after successful sync\n\tif err := f.clearWriteBuffer(); err != nil {\n\t\tf.logger.Error(\"failed to clear write buffer\", \"error\", err)\n\t\treturn fmt.Errorf(\"clear write buffer: %w\", err)\n\t}\n\n\treturn nil\n}\n\n// checkForConflict checks if the remote has newer transactions than expected.\n// Must be called with f.mu held.\nfunc (f *VFSFile) checkForConflict(ctx context.Context) error {\n\t// Get latest remote position\n\titr, err := f.client.LTXFiles(ctx, 0, f.expectedTXID, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"check remote position: %w\", err)\n\t}\n\tdefer itr.Close()\n\n\tvar remoteTXID ltx.TXID\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\t\tif info.MaxTXID > remoteTXID {\n\t\t\tremoteTXID = info.MaxTXID\n\t\t}\n\t}\n\tif err := itr.Close(); err != nil {\n\t\treturn fmt.Errorf(\"iterate remote files: %w\", err)\n\t}\n\n\t// If remote has advanced beyond our expected position, we have a conflict\n\tif remoteTXID > f.expectedTXID {\n\t\tf.logger.Warn(\"conflict detected\",\n\t\t\t\"expected\", f.expectedTXID,\n\t\t\t\"remote\", remoteTXID)\n\t\treturn fmt.Errorf(\"%w: expected TXID %d but remote has %d\",\n\t\t\tErrConflict, f.expectedTXID, remoteTXID)\n\t}\n\n\treturn nil\n}\n\n// createLTXFromDirty creates an LTX file from dirty pages.\n// Returns a streaming reader for the LTX data using io.Pipe to avoid loading\n// all data into memory at once.\n// Must be called with f.mu held.\nfunc (f *VFSFile) createLTXFromDirty() io.Reader {\n\tpr, pw := io.Pipe()\n\n\t// Sort page numbers (LTX encoder requires ordered pages)\n\tpgnos := make([]uint32, 0, len(f.dirty))\n\tfor pgno := range f.dirty {\n\t\tpgnos = append(pgnos, pgno)\n\t}\n\tslices.Sort(pgnos)\n\n\t// Copy dirty map offsets for goroutine access\n\tdirtyOffsets := make(map[uint32]int64, len(f.dirty))\n\tfor pgno, off := range f.dirty {\n\t\tdirtyOffsets[pgno] = off\n\t}\n\n\t// Capture values for goroutine\n\tpageSize := f.pageSize\n\tcommit := f.commit\n\tpendingTXID := f.pendingTXID\n\tbufferFile := f.bufferFile\n\n\tgo func() {\n\t\tvar err error\n\t\tdefer func() {\n\t\t\tpw.CloseWithError(err)\n\t\t}()\n\n\t\tenc, encErr := ltx.NewEncoder(pw)\n\t\tif encErr != nil {\n\t\t\terr = encErr\n\t\t\treturn\n\t\t}\n\n\t\t// Encode header\n\t\tif err = enc.EncodeHeader(ltx.Header{\n\t\t\tVersion: ltx.Version,\n\t\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\t\tPageSize: pageSize,\n\t\t\tCommit: commit,\n\t\t\tMinTXID: pendingTXID,\n\t\t\tMaxTXID: pendingTXID,\n\t\t\tTimestamp: time.Now().UnixMilli(),\n\t\t}); err != nil {\n\t\t\terr = fmt.Errorf(\"encode header: %w\", err)\n\t\t\treturn\n\t\t}\n\n\t\t// Encode each dirty page\n\t\tlockPgno := ltx.LockPgno(pageSize)\n\t\tfor _, pgno := range pgnos {\n\t\t\tif pgno == lockPgno {\n\t\t\t\tcontinue // Skip lock page\n\t\t\t}\n\n\t\t\t// Read page data from buffer file\n\t\t\tbufferOff := dirtyOffsets[pgno]\n\t\t\tdata := make([]byte, pageSize)\n\t\t\tif _, err = bufferFile.ReadAt(data, bufferOff); err != nil {\n\t\t\t\terr = fmt.Errorf(\"read page %d from buffer: %w\", pgno, err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif err = enc.EncodePage(ltx.PageHeader{Pgno: pgno}, data); err != nil {\n\t\t\t\terr = fmt.Errorf(\"encode page %d: %w\", pgno, err)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\n\t\t// Close encoder (writes trailer and page index)\n\t\tif err = enc.Close(); err != nil {\n\t\t\terr = fmt.Errorf(\"close encoder: %w\", err)\n\t\t\treturn\n\t\t}\n\t}()\n\n\treturn pr\n}\n\n// initWriteBuffer initializes the write buffer file for durability.\n// Any existing buffer content is discarded since unsync'd changes are lost on restart.\n// This function acquires f.mu internally.\nfunc (f *VFSFile) initWriteBuffer() error {\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\treturn f.initWriteBufferWithLock()\n}\n\n// initWriteBufferWithLock initializes the write buffer file for durability.\n// Any existing buffer content is discarded since unsync'd changes are lost on restart.\n// Caller must hold f.mu.\nfunc (f *VFSFile) initWriteBufferWithLock() error {\n\t// Ensure parent directory exists\n\tif err := os.MkdirAll(filepath.Dir(f.bufferPath), 0755); err != nil {\n\t\treturn fmt.Errorf(\"create buffer directory: %w\", err)\n\t}\n\n\t// Open or create buffer file, truncating any existing content\n\tfile, err := os.OpenFile(f.bufferPath, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"open buffer file: %w\", err)\n\t}\n\tf.bufferFile = file\n\tf.bufferNextOff = 0\n\n\treturn nil\n}\n\n// writeToBuffer writes a dirty page to the write buffer for durability.\n// If the page already exists in the buffer, it overwrites at the same offset.\n// Otherwise, it appends to the end of the file.\n// Must be called with f.mu held.\nfunc (f *VFSFile) writeToBuffer(pgno uint32, data []byte) error {\n\tvar writeOffset int64\n\tif existingOff, ok := f.dirty[pgno]; ok {\n\t\t// Page already exists - overwrite at same offset\n\t\twriteOffset = existingOff\n\t} else {\n\t\t// New page - append to end of file\n\t\twriteOffset = f.bufferNextOff\n\t\tf.bufferNextOff += int64(len(data))\n\t}\n\n\t// Write page data (no header, just raw page data)\n\tif _, err := f.bufferFile.WriteAt(data, writeOffset); err != nil {\n\t\treturn fmt.Errorf(\"write page to buffer: %w\", err)\n\t}\n\n\t// Update dirty map with offset\n\tf.dirty[pgno] = writeOffset\n\n\treturn nil\n}\n\n// clearWriteBuffer clears and resets the write buffer after successful sync.\nfunc (f *VFSFile) clearWriteBuffer() error {\n\t// Truncate file to zero\n\tif err := f.bufferFile.Truncate(0); err != nil {\n\t\treturn fmt.Errorf(\"truncate buffer: %w\", err)\n\t}\n\n\t// Reset next write offset\n\tf.bufferNextOff = 0\n\n\treturn nil\n}\n\nfunc (f *VFSFile) FileSize() (size int64, err error) {\n\tpageSize, err := f.pageSizeBytes()\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tf.mu.Lock()\n\tfor pgno := range f.index {\n\t\tif v := int64(pgno) * int64(pageSize); v > size {\n\t\t\tsize = v\n\t\t}\n\t}\n\tfor pgno := range f.pending {\n\t\tif v := int64(pgno) * int64(pageSize); v > size {\n\t\t\tsize = v\n\t\t}\n\t}\n\t// Include dirty pages in size calculation\n\tfor pgno := range f.dirty {\n\t\tif v := int64(pgno) * int64(pageSize); v > size {\n\t\t\tsize = v\n\t\t}\n\t}\n\tf.mu.Unlock()\n\n\tf.logger.Debug(\"file size\", \"size\", size)\n\treturn size, nil\n}\n\nfunc (f *VFSFile) Lock(elock sqlite3vfs.LockType) error {\n\tf.logger.Debug(\"locking file\", \"lock\", elock)\n\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\n\tif elock < f.lockType {\n\t\treturn fmt.Errorf(\"invalid lock downgrade: current=%s target=%s\", f.lockType, elock)\n\t}\n\n\tif elock >= sqlite3vfs.LockReserved {\n\t\t// Wait for any disable operation to complete before allowing RESERVED lock.\n\t\t// This prevents new write transactions from starting during disable.\n\t\tfor f.disabling {\n\t\t\tf.logger.Debug(\"waiting for disable to complete before acquiring RESERVED lock\")\n\t\t\tf.cond.Wait()\n\t\t}\n\n\t\t// Reject write-intent locks when writes are disabled. Since we always\n\t\t// report OpenReadWrite to SQLite (to support cold enable), SQLite may\n\t\t// attempt write transactions even when writes are logically disabled.\n\t\tif !f.writeEnabled {\n\t\t\treturn sqlite3vfs.ReadOnlyError\n\t\t}\n\t}\n\n\tif f.writeEnabled && elock >= sqlite3vfs.LockReserved && !f.inTransaction {\n\t\tif f.vfs != nil {\n\t\t\tf.vfs.writeMu.Lock()\n\t\t\tif f.vfs.writeFile != nil && f.vfs.writeFile != f {\n\t\t\t\tf.vfs.writeMu.Unlock()\n\t\t\t\treturn sqlite3vfs.BusyError\n\t\t\t}\n\t\t\tf.vfs.writeFile = f\n\t\t\tif f.vfs.lastSyncedTXID > f.expectedTXID && len(f.dirty) == 0 {\n\t\t\t\tf.expectedTXID = f.vfs.lastSyncedTXID\n\t\t\t\tf.pendingTXID = f.vfs.lastSyncedTXID + 1\n\t\t\t\tf.pos = ltx.Pos{TXID: f.expectedTXID}\n\t\t\t}\n\t\t\tf.vfs.writeMu.Unlock()\n\t\t}\n\t\tf.inTransaction = true\n\t\tf.logger.Debug(\"transaction started\", \"expectedTXID\", f.expectedTXID)\n\t}\n\n\tf.lockType = elock\n\treturn nil\n}\n\nfunc (f *VFSFile) Unlock(elock sqlite3vfs.LockType) error {\n\tf.logger.Debug(\"unlocking file\", \"lock\", elock)\n\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\n\tif elock != sqlite3vfs.LockShared && elock != sqlite3vfs.LockNone {\n\t\treturn fmt.Errorf(\"invalid unlock target: %s\", elock)\n\t}\n\n\tif f.writeEnabled && f.inTransaction && elock < sqlite3vfs.LockReserved {\n\t\tf.inTransaction = false\n\t\tif f.vfs != nil {\n\t\t\tf.vfs.writeMu.Lock()\n\t\t\tif f.vfs.writeFile == f {\n\t\t\t\tf.vfs.writeFile = nil\n\t\t\t}\n\t\t\tf.vfs.writeMu.Unlock()\n\t\t}\n\t\tf.logger.Debug(\"transaction ended\", \"dirtyPages\", len(f.dirty))\n\t\tf.cond.Broadcast() // Wake up SetWriteEnabledWithTimeout if waiting\n\t}\n\n\tf.lockType = elock\n\n\t// Copy pending index to main index and invalidate affected pages in cache.\n\tif f.pendingReplace {\n\t\t// Replace entire index\n\t\tcount := len(f.index)\n\t\tf.index = f.pending\n\t\tf.logger.Debug(\"cache invalidated all pages\", \"count\", count)\n\t\t// Invalidate entire cache since we replaced the index\n\t\tf.cache.Purge()\n\t} else if len(f.pending) > 0 {\n\t\t// Merge pending into index\n\t\tcount := len(f.pending)\n\t\tfor k, v := range f.pending {\n\t\t\tf.index[k] = v\n\t\t\tf.cache.Remove(k)\n\t\t}\n\t\tf.logger.Debug(\"cache invalidated pages\", \"count\", count)\n\t}\n\tf.pending = make(map[uint32]ltx.PageIndexElem)\n\tf.pendingReplace = false\n\n\treturn nil\n}\n\nfunc (f *VFSFile) CheckReservedLock() (bool, error) {\n\tf.logger.Debug(\"checking reserved lock\")\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\tif f.lockType >= sqlite3vfs.LockReserved {\n\t\treturn true, nil\n\t}\n\tif f.vfs != nil {\n\t\tf.vfs.writeMu.Lock()\n\t\theld := f.vfs.writeFile != nil\n\t\tf.vfs.writeMu.Unlock()\n\t\treturn held, nil\n\t}\n\treturn false, nil\n}\n\nfunc (f *VFSFile) SectorSize() int64 {\n\tf.logger.Debug(\"sector size\")\n\treturn 0\n}\n\nfunc (f *VFSFile) DeviceCharacteristics() sqlite3vfs.DeviceCharacteristic {\n\tf.logger.Debug(\"device characteristics\")\n\treturn 0\n}\n\n// parseTimeValue parses a timestamp string, trying RFC3339 first, then relative expressions.\nfunc parseTimeValue(value string) (time.Time, error) {\n\t// Try RFC3339Nano first (existing behavior)\n\tif t, err := time.Parse(time.RFC3339Nano, value); err == nil {\n\t\treturn t, nil\n\t}\n\n\t// Try RFC3339 (without nanoseconds)\n\tif t, err := time.Parse(time.RFC3339, value); err == nil {\n\t\treturn t, nil\n\t}\n\n\t// Fall back to dateparser for relative expressions\n\tcfg := &dateparser.Configuration{\n\t\tCurrentTime: time.Now().UTC(),\n\t}\n\tresult, err := dateparser.Parse(cfg, value)\n\tif err != nil {\n\t\treturn time.Time{}, fmt.Errorf(\"invalid timestamp (expected RFC3339 or relative time like '5 minutes ago'): %s\", value)\n\t}\n\tif result.Time.IsZero() {\n\t\treturn time.Time{}, fmt.Errorf(\"could not parse time: %s\", value)\n\t}\n\treturn result.Time.UTC(), nil\n}\n\n// FileControl handles file control operations, specifically PRAGMA commands for time travel.\nfunc (f *VFSFile) FileControl(op int, pragmaName string, pragmaValue *string) (*string, error) {\n\tconst SQLITE_FCNTL_PRAGMA = 14\n\n\tif op != SQLITE_FCNTL_PRAGMA {\n\t\treturn nil, fmt.Errorf(\"unsupported file control op: %d\", op)\n\t}\n\n\tname := strings.ToLower(pragmaName)\n\n\tf.logger.Debug(\"file control\", \"pragma\", name, \"value\", pragmaValue)\n\n\tswitch name {\n\tcase \"litestream_txid\":\n\t\tif pragmaValue != nil {\n\t\t\treturn nil, fmt.Errorf(\"litestream_txid is read-only\")\n\t\t}\n\t\ttxid := f.Pos().TXID\n\t\tresult := txid.String()\n\t\treturn &result, nil\n\n\tcase \"litestream_lag\":\n\t\tif pragmaValue != nil {\n\t\t\treturn nil, fmt.Errorf(\"litestream_lag is read-only\")\n\t\t}\n\t\tlastPoll := f.LastPollSuccess()\n\t\tif lastPoll.IsZero() {\n\t\t\tresult := \"-1\" // Never polled successfully\n\t\t\treturn &result, nil\n\t\t}\n\t\tlag := int64(time.Since(lastPoll).Seconds())\n\t\tresult := strconv.FormatInt(lag, 10)\n\t\treturn &result, nil\n\n\tcase \"litestream_time\":\n\t\tif pragmaValue == nil {\n\t\t\tresult := f.currentTimeString()\n\t\t\treturn &result, nil\n\t\t}\n\n\t\tif strings.EqualFold(*pragmaValue, \"latest\") {\n\t\t\tif err := f.ResetTime(context.Background()); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\treturn nil, nil\n\t\t}\n\n\t\tt, err := parseTimeValue(*pragmaValue)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif err := f.SetTargetTime(context.Background(), t); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn nil, nil\n\n\tcase \"litestream_hydration_progress\":\n\t\tif pragmaValue != nil {\n\t\t\treturn nil, fmt.Errorf(\"litestream_hydration_progress is read-only\")\n\t\t}\n\t\tif f.hydrator == nil {\n\t\t\tresult := \"0\"\n\t\t\treturn &result, nil\n\t\t}\n\t\tpct := f.hydrator.Status().Pct() * 100\n\t\tresult := strconv.FormatFloat(pct, 'f', 1, 64)\n\t\treturn &result, nil\n\n\tcase \"litestream_hydration_file\":\n\t\tif pragmaValue != nil {\n\t\t\treturn nil, fmt.Errorf(\"litestream_hydration_file is read-only\")\n\t\t}\n\t\tresult := f.hydrationPath\n\t\treturn &result, nil\n\n\tcase \"litestream_write_enabled\":\n\t\tif pragmaValue == nil {\n\t\t\t// READ mode - return current state\n\t\t\tf.mu.Lock()\n\t\t\tenabled := f.writeEnabled\n\t\t\tf.mu.Unlock()\n\t\t\tif enabled {\n\t\t\t\tresult := \"1\"\n\t\t\t\treturn &result, nil\n\t\t\t}\n\t\t\tresult := \"0\"\n\t\t\treturn &result, nil\n\t\t}\n\t\t// WRITE mode - enable or disable\n\t\tswitch strings.ToLower(*pragmaValue) {\n\t\tcase \"0\", \"false\", \"off\":\n\t\t\tif err := f.SetWriteEnabled(false); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\treturn nil, nil\n\t\tcase \"1\", \"true\", \"on\":\n\t\t\tif err := f.SetWriteEnabled(true); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\treturn nil, nil\n\t\tdefault:\n\t\t\treturn nil, fmt.Errorf(\"invalid value for litestream_write_enabled: %s (use 0 or 1)\", *pragmaValue)\n\t\t}\n\n\tdefault:\n\t\treturn nil, sqlite3vfs.NotFoundError\n\t}\n}\n\n// currentTimeString returns the current target time as a string.\nfunc (f *VFSFile) currentTimeString() string {\n\tif t := f.TargetTime(); t != nil {\n\t\treturn t.Format(time.RFC3339Nano)\n\t}\n\tif t := f.LatestLTXTime(); !t.IsZero() {\n\t\treturn t.Format(time.RFC3339Nano)\n\t}\n\treturn \"latest\" // Fallback if no LTX files loaded\n}\n\nfunc isRetryablePageError(err error) bool {\n\tif err == nil {\n\t\treturn false\n\t}\n\tif errors.Is(err, context.DeadlineExceeded) || errors.Is(err, context.Canceled) {\n\t\treturn true\n\t}\n\tif errors.Is(err, io.ErrUnexpectedEOF) {\n\t\treturn true\n\t}\n\t// Some remote clients wrap EOF in custom errors so we fall back to string matching.\n\tif strings.Contains(err.Error(), \"unexpected EOF\") {\n\t\treturn true\n\t}\n\tif errors.Is(err, os.ErrNotExist) {\n\t\treturn true\n\t}\n\treturn false\n}\n\nfunc (f *VFSFile) monitorReplicaClient(ctx context.Context) {\n\tticker := time.NewTicker(f.PollInterval)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t\tif f.hasTargetTime() {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif err := f.pollReplicaClient(ctx); err != nil {\n\t\t\t\t// Don't log context cancellation errors during shutdown\n\t\t\t\tif !errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded) {\n\t\t\t\t\tf.logger.Error(\"cannot fetch new ltx files\", \"error\", err)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\t// Track successful poll time\n\t\t\t\tf.mu.Lock()\n\t\t\t\tf.lastPollSuccess = time.Now()\n\t\t\t\tf.mu.Unlock()\n\t\t\t}\n\t\t}\n\t}\n}\n\n// pollReplicaClient fetches new LTX files from the replica client and updates\n// the page index & the current position.\nfunc (f *VFSFile) pollReplicaClient(ctx context.Context) error {\n\tpos := f.Pos()\n\tf.logger.Debug(\"polling replica client\", \"txid\", pos.TXID.String())\n\n\tcombined := make(map[uint32]ltx.PageIndexElem)\n\n\tf.mu.Lock()\n\tbaseCommit := f.commit\n\tmaxTXID1Snapshot := f.maxTXID1\n\tf.mu.Unlock()\n\n\tnewCommit := baseCommit\n\treplaceIndex := false\n\n\tmaxTXID0, idx0, commit0, replace0, err := f.pollLevel(ctx, 0, pos.TXID, baseCommit)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"poll L0: %w\", err)\n\t}\n\tif replace0 {\n\t\treplaceIndex = true\n\t\tbaseCommit = commit0\n\t\tnewCommit = commit0\n\t\tcombined = idx0\n\t} else {\n\t\tif len(idx0) > 0 {\n\t\t\tbaseCommit = commit0\n\t\t}\n\t\tfor k, v := range idx0 {\n\t\t\tcombined[k] = v\n\t\t}\n\t\tif commit0 > newCommit {\n\t\t\tnewCommit = commit0\n\t\t}\n\t}\n\n\tmaxTXID1, idx1, commit1, replace1, err := f.pollLevel(ctx, 1, maxTXID1Snapshot, baseCommit)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"poll L1: %w\", err)\n\t}\n\tif replace1 {\n\t\treplaceIndex = true\n\t\tbaseCommit = commit1\n\t\tnewCommit = commit1\n\t\tcombined = idx1\n\t} else {\n\t\tfor k, v := range idx1 {\n\t\t\tcombined[k] = v\n\t\t}\n\t\tif commit1 > newCommit {\n\t\t\tnewCommit = commit1\n\t\t}\n\t}\n\n\t// Send updates to a pending list if there are active readers.\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\n\tif f.targetTime != nil {\n\t\t// Skip applying updates while time travel is active to avoid\n\t\t// overwriting the historical snapshot state.\n\t\treturn nil\n\t}\n\n\t// Apply updates and invalidate cache entries for updated pages\n\tinvalidateN := 0\n\ttarget := f.index\n\ttargetIsMain := true\n\tif f.lockType >= sqlite3vfs.LockShared {\n\t\ttarget = f.pending\n\t\ttargetIsMain = false\n\t} else {\n\t\tf.pendingReplace = false\n\t}\n\tif replaceIndex {\n\t\tif f.lockType < sqlite3vfs.LockShared {\n\t\t\tf.index = make(map[uint32]ltx.PageIndexElem)\n\t\t\ttarget = f.index\n\t\t\ttargetIsMain = true\n\t\t\tf.pendingReplace = false\n\t\t} else {\n\t\t\tf.pending = make(map[uint32]ltx.PageIndexElem)\n\t\t\ttarget = f.pending\n\t\t\ttargetIsMain = false\n\t\t\tf.pendingReplace = true\n\t\t}\n\t}\n\tfor k, v := range combined {\n\t\ttarget[k] = v\n\t\t// Invalidate cache if we're updating the main index\n\t\tif targetIsMain {\n\t\t\tf.cache.Remove(k)\n\t\t\tinvalidateN++\n\t\t}\n\t}\n\n\tif invalidateN > 0 {\n\t\tf.logger.Debug(\"cache invalidated pages due to new ltx files\", \"count\", invalidateN)\n\t}\n\n\tif replaceIndex {\n\t\tf.commit = newCommit\n\t} else if len(combined) > 0 && newCommit > f.commit {\n\t\tf.commit = newCommit\n\t}\n\n\tif maxTXID0 > maxTXID1 {\n\t\tf.pos.TXID = maxTXID0\n\t} else {\n\t\tf.pos.TXID = maxTXID1\n\t}\n\n\tf.maxTXID1 = maxTXID1\n\tf.logger.Debug(\"txid updated\", \"txid\", f.pos.TXID.String(), \"maxTXID1\", f.maxTXID1.String())\n\n\t// Apply updates to hydrated file if hydration is complete\n\tif f.hydrator != nil && f.hydrator.Complete() && len(combined) > 0 {\n\t\tif err := f.hydrator.ApplyUpdates(f.ctx, combined); err != nil {\n\t\t\tf.logger.Error(\"failed to apply updates to hydrated file\", \"error\", err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// pollLevel fetches LTX files for a specific level and returns the highest TXID seen,\n// any index updates, the latest commit value, and if the index should be replaced.\nfunc (f *VFSFile) pollLevel(ctx context.Context, level int, prevMaxTXID ltx.TXID, baseCommit uint32) (ltx.TXID, map[uint32]ltx.PageIndexElem, uint32, bool, error) {\n\titr, err := f.client.LTXFiles(ctx, level, prevMaxTXID+1, false)\n\tif err != nil {\n\t\treturn prevMaxTXID, nil, baseCommit, false, fmt.Errorf(\"ltx files: %w\", err)\n\t}\n\tdefer func() { _ = itr.Close() }()\n\n\tindex := make(map[uint32]ltx.PageIndexElem)\n\tmaxTXID := prevMaxTXID\n\tlastCommit := baseCommit\n\tnewCommit := baseCommit\n\treplaceIndex := false\n\n\tfor itr.Next() {\n\t\tinfo := itr.Item()\n\n\t\tf.mu.Lock()\n\t\tisNextTXID := info.MinTXID == maxTXID+1\n\t\tf.mu.Unlock()\n\t\tif !isNextTXID {\n\t\t\tif level == 0 && info.MinTXID > maxTXID+1 {\n\t\t\t\tf.logger.Warn(\"ltx gap detected at L0, deferring to higher levels\", \"expected\", maxTXID+1, \"next\", info.MinTXID)\n\t\t\t\tbreak\n\t\t\t}\n\t\t\treturn maxTXID, nil, newCommit, replaceIndex, fmt.Errorf(\"non-contiguous ltx file: level=%d, current=%s, next=%s-%s\", level, maxTXID, info.MinTXID, info.MaxTXID)\n\t\t}\n\n\t\tf.logger.Debug(\"new ltx file\", \"level\", info.Level, \"min\", info.MinTXID, \"max\", info.MaxTXID)\n\n\t\tidx, err := FetchPageIndex(ctx, f.client, info)\n\t\tif err != nil {\n\t\t\treturn maxTXID, nil, newCommit, replaceIndex, fmt.Errorf(\"fetch page index: %w\", err)\n\t\t}\n\t\thdr, err := FetchLTXHeader(ctx, f.client, info)\n\t\tif err != nil {\n\t\t\treturn maxTXID, nil, newCommit, replaceIndex, fmt.Errorf(\"fetch header: %w\", err)\n\t\t}\n\n\t\tif hdr.Commit < lastCommit {\n\t\t\treplaceIndex = true\n\t\t\tindex = make(map[uint32]ltx.PageIndexElem)\n\t\t}\n\t\tlastCommit = hdr.Commit\n\t\tnewCommit = hdr.Commit\n\n\t\tfor k, v := range idx {\n\t\t\tf.logger.Debug(\"adding new page index\", \"page\", k, \"elem\", v)\n\t\t\tindex[k] = v\n\t\t}\n\t\tmaxTXID = info.MaxTXID\n\t}\n\n\treturn maxTXID, index, newCommit, replaceIndex, nil\n}\n\nfunc (f *VFSFile) pageSizeBytes() (uint32, error) {\n\tf.mu.Lock()\n\tpageSize := f.pageSize\n\tf.mu.Unlock()\n\tif pageSize == 0 {\n\t\tf.logger.Debug(\"page size not initialized\", \"pageSize\", 0)\n\t\treturn 0, &DBNotReadyError{Reason: \"page size not initialized\"}\n\t}\n\treturn pageSize, nil\n}\n\nfunc detectPageSizeFromInfos(ctx context.Context, client ReplicaClient, infos []*ltx.FileInfo) (uint32, error) {\n\tvar lastErr error\n\tfor i := len(infos) - 1; i >= 0; i-- {\n\t\tpageSize, err := readPageSizeFromInfo(ctx, client, infos[i])\n\t\tif err != nil {\n\t\t\tlastErr = err\n\t\t\tcontinue\n\t\t}\n\t\tif !isSupportedPageSize(pageSize) {\n\t\t\treturn 0, fmt.Errorf(\"unsupported page size: %d\", pageSize)\n\t\t}\n\t\treturn pageSize, nil\n\t}\n\tif lastErr != nil {\n\t\treturn 0, fmt.Errorf(\"read ltx header: %w\", lastErr)\n\t}\n\treturn 0, fmt.Errorf(\"no ltx file available to determine page size\")\n}\n\nfunc readPageSizeFromInfo(ctx context.Context, client ReplicaClient, info *ltx.FileInfo) (uint32, error) {\n\trc, err := client.OpenLTXFile(ctx, info.Level, info.MinTXID, info.MaxTXID, 0, ltx.HeaderSize)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"open ltx file: %w\", err)\n\t}\n\tdefer rc.Close()\n\tdec := ltx.NewDecoder(rc)\n\tif err := dec.DecodeHeader(); err != nil {\n\t\treturn 0, fmt.Errorf(\"decode ltx header: %w\", err)\n\t}\n\treturn dec.Header().PageSize, nil\n}\n\nfunc isSupportedPageSize(pageSize uint32) bool {\n\tswitch pageSize {\n\tcase 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536:\n\t\treturn true\n\tdefault:\n\t\treturn false\n\t}\n}\n\nfunc (f *VFSFile) waitForRestorePlan() ([]*ltx.FileInfo, error) {\n\t// If write mode is enabled, don't wait - return immediately so we can\n\t// create a new database if no files exist.\n\tif f.writeEnabled {\n\t\tinfos, err := CalcRestorePlan(f.ctx, f.client, 0, time.Time{}, f.logger)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn infos, nil\n\t}\n\n\t// For read-only mode, wait for files to become available\n\tfor {\n\t\tinfos, err := CalcRestorePlan(f.ctx, f.client, 0, time.Time{}, f.logger)\n\t\tif err == nil {\n\t\t\treturn infos, nil\n\t\t}\n\t\tif !errors.Is(err, ErrTxNotAvailable) {\n\t\t\treturn nil, fmt.Errorf(\"cannot calc restore plan: %w\", err)\n\t\t}\n\n\t\tf.logger.Debug(\"no backup files available yet, waiting\", \"interval\", f.PollInterval)\n\t\tselect {\n\t\tcase <-time.After(f.PollInterval):\n\t\tcase <-f.ctx.Done():\n\t\t\treturn nil, fmt.Errorf(\"no backup files available: %w\", f.ctx.Err())\n\t\t}\n\t}\n}\n\n// RegisterVFSConnection maps a SQLite connection handle to its VFS file ID.\nfunc RegisterVFSConnection(dbPtr uintptr, fileID uint64) error {\n\tif _, ok := lookupVFSFile(fileID); !ok {\n\t\treturn fmt.Errorf(\"vfs file not found: id=%d\", fileID)\n\t}\n\tvfsConnectionMap.Store(dbPtr, fileID)\n\treturn nil\n}\n\n// UnregisterVFSConnection removes a connection mapping.\nfunc UnregisterVFSConnection(dbPtr uintptr) {\n\tvfsConnectionMap.Delete(dbPtr)\n}\n\n// SetVFSConnectionTime rebuilds the VFS index for a connection at a timestamp.\nfunc SetVFSConnectionTime(dbPtr uintptr, timestamp string) error {\n\tfile, err := vfsFileForConnection(dbPtr)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tt, err := parseTimeValue(timestamp)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn file.SetTargetTime(context.Background(), t)\n}\n\n// ResetVFSConnectionTime rebuilds the VFS index to the latest state.\nfunc ResetVFSConnectionTime(dbPtr uintptr) error {\n\tfile, err := vfsFileForConnection(dbPtr)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn file.ResetTime(context.Background())\n}\n\n// GetVFSConnectionTime returns the current time for a connection.\nfunc GetVFSConnectionTime(dbPtr uintptr) (string, error) {\n\tfile, err := vfsFileForConnection(dbPtr)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn file.currentTimeString(), nil\n}\n\n// GetVFSConnectionTXID returns the current transaction ID for a connection as a hex string.\nfunc GetVFSConnectionTXID(dbPtr uintptr) (string, error) {\n\tfile, err := vfsFileForConnection(dbPtr)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn file.Pos().TXID.String(), nil\n}\n\n// GetVFSConnectionLag returns seconds since last successful poll for a connection.\nfunc GetVFSConnectionLag(dbPtr uintptr) (int64, error) {\n\tfile, err := vfsFileForConnection(dbPtr)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tlastPoll := file.LastPollSuccess()\n\tif lastPoll.IsZero() {\n\t\treturn -1, nil\n\t}\n\treturn int64(time.Since(lastPoll).Seconds()), nil\n}\n\nfunc vfsFileForConnection(dbPtr uintptr) (*VFSFile, error) {\n\tv, ok := vfsConnectionMap.Load(dbPtr)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"connection not registered\")\n\t}\n\tfileID, ok := v.(uint64)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"invalid connection mapping\")\n\t}\n\tfile, ok := lookupVFSFile(fileID)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"vfs file not found: id=%d\", fileID)\n\t}\n\treturn file, nil\n}\n\nfunc lookupVFSFile(fileID uint64) (*VFSFile, bool) {\n\tsqlite3vfsFileMux.Lock()\n\tdefer sqlite3vfsFileMux.Unlock()\n\n\tfile, ok := sqlite3vfsFileMap[fileID]\n\tif !ok {\n\t\treturn nil, false\n\t}\n\n\tvfsFile, ok := file.(*VFSFile)\n\treturn vfsFile, ok\n}\n\n// startCompactionMonitors starts background goroutines for compaction and snapshots.\nfunc (f *VFSFile) startCompactionMonitors() {\n\tf.compactionCtx, f.compactionCancel = context.WithCancel(f.ctx)\n\n\t// Use configured levels or defaults\n\tlevels := f.vfs.CompactionLevels\n\tif levels == nil {\n\t\tlevels = DefaultCompactionLevels\n\t}\n\n\t// Start compaction monitors for each level\n\tfor _, lvl := range levels {\n\t\tif lvl.Level == 0 {\n\t\t\tcontinue // L0 doesn't need compaction (source level)\n\t\t}\n\t\tf.compactionWg.Add(1)\n\t\tgo func(level *CompactionLevel) {\n\t\t\tdefer f.compactionWg.Done()\n\t\t\tf.monitorCompaction(f.compactionCtx, level)\n\t\t}(lvl)\n\t}\n\n\t// Start snapshot monitor if configured\n\tif f.vfs.SnapshotInterval > 0 {\n\t\tf.compactionWg.Add(1)\n\t\tgo func() {\n\t\t\tdefer f.compactionWg.Done()\n\t\t\tf.monitorSnapshots(f.compactionCtx)\n\t\t}()\n\t}\n\n\t// Start L0 retention monitor if configured\n\tif f.vfs.L0Retention > 0 {\n\t\tf.compactionWg.Add(1)\n\t\tgo func() {\n\t\t\tdefer f.compactionWg.Done()\n\t\t\tf.monitorL0Retention(f.compactionCtx)\n\t\t}()\n\t}\n\n\tf.logger.Info(\"compaction monitors started\",\n\t\t\"levels\", len(levels),\n\t\t\"snapshotInterval\", f.vfs.SnapshotInterval,\n\t\t\"l0Retention\", f.vfs.L0Retention)\n}\n\n// Compact compacts source level files into the destination level.\n// Returns ErrNoCompaction if there are no files to compact.\nfunc (f *VFSFile) Compact(ctx context.Context, level int) (*ltx.FileInfo, error) {\n\tif f.compactor == nil {\n\t\treturn nil, fmt.Errorf(\"compaction not enabled\")\n\t}\n\treturn f.compactor.Compact(ctx, level)\n}\n\n// Snapshot creates a full database snapshot from remote LTX files.\n// Unlike DB.Snapshot(), this reads from remote rather than local WAL.\nfunc (f *VFSFile) Snapshot(ctx context.Context) (*ltx.FileInfo, error) {\n\tif f.compactor == nil {\n\t\treturn nil, fmt.Errorf(\"compaction not enabled\")\n\t}\n\n\tf.mu.Lock()\n\tpageSize := f.pageSize\n\tcommit := f.commit\n\tpos := f.pos\n\tpages := make(map[uint32]ltx.PageIndexElem, len(f.index))\n\tfor pgno, elem := range f.index {\n\t\tpages[pgno] = elem\n\t}\n\tf.mu.Unlock()\n\n\tif pageSize == 0 {\n\t\tf.logger.Debug(\"snapshot skipped, page size not initialized\", \"pageSize\", 0)\n\t\treturn nil, &DBNotReadyError{Reason: \"page size not initialized\"}\n\t}\n\n\t// Sort page numbers for consistent output\n\tpgnos := make([]uint32, 0, len(pages))\n\tfor pgno := range pages {\n\t\tpgnos = append(pgnos, pgno)\n\t}\n\tslices.Sort(pgnos)\n\n\t// Stream snapshot creation\n\tpr, pw := io.Pipe()\n\tgo func() {\n\t\tenc, err := ltx.NewEncoder(pw)\n\t\tif err != nil {\n\t\t\tpw.CloseWithError(err)\n\t\t\treturn\n\t\t}\n\n\t\tif err := enc.EncodeHeader(ltx.Header{\n\t\t\tVersion: ltx.Version,\n\t\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\t\tPageSize: pageSize,\n\t\t\tCommit: commit,\n\t\t\tMinTXID: 1,\n\t\t\tMaxTXID: pos.TXID,\n\t\t\tTimestamp: time.Now().UnixMilli(),\n\t\t}); err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"encode header: %w\", err))\n\t\t\treturn\n\t\t}\n\n\t\tfor _, pgno := range pgnos {\n\t\t\telem := pages[pgno]\n\t\t\t_, data, err := FetchPage(ctx, f.client, elem.Level, elem.MinTXID, elem.MaxTXID, elem.Offset, elem.Size)\n\t\t\tif err != nil {\n\t\t\t\tpw.CloseWithError(fmt.Errorf(\"fetch page %d: %w\", pgno, err))\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif err := enc.EncodePage(ltx.PageHeader{Pgno: pgno}, data); err != nil {\n\t\t\t\tpw.CloseWithError(fmt.Errorf(\"encode page %d: %w\", pgno, err))\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\n\t\tif err := enc.Close(); err != nil {\n\t\t\tpw.CloseWithError(fmt.Errorf(\"close encoder: %w\", err))\n\t\t\treturn\n\t\t}\n\t\tpw.Close()\n\t}()\n\n\treturn f.client.WriteLTXFile(ctx, SnapshotLevel, 1, pos.TXID, pr)\n}\n\n// monitorCompaction runs periodic compaction for a level.\nfunc (f *VFSFile) monitorCompaction(ctx context.Context, lvl *CompactionLevel) {\n\tf.logger.Info(\"starting VFS compaction monitor\", \"level\", lvl.Level, \"interval\", lvl.Interval)\n\n\tticker := time.NewTicker(lvl.Interval)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t\tinfo, err := f.Compact(ctx, lvl.Level)\n\t\t\tif err != nil {\n\t\t\t\tif !errors.Is(err, ErrNoCompaction) &&\n\t\t\t\t\t!errors.Is(err, context.Canceled) &&\n\t\t\t\t\t!errors.Is(err, context.DeadlineExceeded) {\n\t\t\t\t\tf.logger.Error(\"compaction failed\", \"level\", lvl.Level, \"error\", err)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tf.logger.Debug(\"compaction completed\",\n\t\t\t\t\t\"level\", lvl.Level,\n\t\t\t\t\t\"minTXID\", info.MinTXID,\n\t\t\t\t\t\"maxTXID\", info.MaxTXID,\n\t\t\t\t\t\"size\", info.Size)\n\t\t\t}\n\t\t}\n\t}\n}\n\n// monitorSnapshots runs periodic snapshot creation.\nfunc (f *VFSFile) monitorSnapshots(ctx context.Context) {\n\tf.logger.Info(\"starting VFS snapshot monitor\", \"interval\", f.vfs.SnapshotInterval)\n\n\tticker := time.NewTicker(f.vfs.SnapshotInterval)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t\tinfo, err := f.Snapshot(ctx)\n\t\t\tif err != nil {\n\t\t\t\t// DBNotReadyError is already logged at debug level in Snapshot()\n\t\t\t\tif !errors.Is(err, context.Canceled) &&\n\t\t\t\t\t!errors.Is(err, context.DeadlineExceeded) &&\n\t\t\t\t\t!errors.Is(err, ErrDBNotReady) {\n\t\t\t\t\tf.logger.Error(\"snapshot failed\", \"error\", err)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tf.logger.Debug(\"snapshot created\",\n\t\t\t\t\t\"maxTXID\", info.MaxTXID,\n\t\t\t\t\t\"size\", info.Size)\n\n\t\t\t\t// Enforce snapshot retention after creating new snapshot\n\t\t\t\tif f.vfs.SnapshotRetention > 0 {\n\t\t\t\t\tif _, err := f.compactor.EnforceSnapshotRetention(ctx, f.vfs.SnapshotRetention); err != nil {\n\t\t\t\t\t\tf.logger.Error(\"snapshot retention failed\", \"error\", err)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n\n// monitorL0Retention runs periodic L0 retention enforcement.\nfunc (f *VFSFile) monitorL0Retention(ctx context.Context) {\n\tf.logger.Info(\"starting VFS L0 retention monitor\", \"retention\", f.vfs.L0Retention)\n\n\t// Check more frequently than the retention period\n\tcheckInterval := f.vfs.L0Retention / 4\n\tif checkInterval < time.Minute {\n\t\tcheckInterval = time.Minute\n\t}\n\n\tticker := time.NewTicker(checkInterval)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t\tif err := f.compactor.EnforceL0Retention(ctx, f.vfs.L0Retention); err != nil {\n\t\t\t\tif !errors.Is(err, context.Canceled) &&\n\t\t\t\t\t!errors.Is(err, context.DeadlineExceeded) {\n\t\t\t\t\tf.logger.Error(\"L0 retention enforcement failed\", \"error\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n" }, { "path": "vfs_compaction_test.go", "content": "//go:build vfs\n// +build vfs\n\npackage litestream_test\n\nimport (\n\t\"context\"\n\t\"log/slog\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/file\"\n)\n\nfunc TestVFSFile_Compact(t *testing.T) {\n\tt.Run(\"ManualCompact\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tclient := file.NewReplicaClient(dir)\n\n\t\t// Pre-create some L0 files to test compaction\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 2, 2)\n\t\tcreateTestLTXFile(t, client, 0, 3, 3)\n\n\t\t// Create VFS with compaction enabled\n\t\tvfs := litestream.NewVFS(client, slog.Default())\n\t\tvfs.WriteEnabled = true\n\t\tvfs.CompactionEnabled = true\n\n\t\t// Create VFSFile directly to test Compact method\n\t\tf := litestream.NewVFSFile(client, \"test.db\", slog.Default())\n\t\tf.PollInterval = time.Second\n\t\tf.CacheSize = litestream.DefaultCacheSize\n\n\t\t// Initialize the compactor manually\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Compact L0 to L1\n\t\tinfo, err := compactor.Compact(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.Level != 1 {\n\t\t\tt.Errorf(\"Level=%d, want 1\", info.Level)\n\t\t}\n\t\tif info.MinTXID != 1 || info.MaxTXID != 3 {\n\t\t\tt.Errorf(\"TXID range=%d-%d, want 1-3\", info.MinTXID, info.MaxTXID)\n\t\t}\n\t})\n}\n\nfunc TestVFSFile_Snapshot(t *testing.T) {\n\tt.Run(\"MultiLevelCompaction\", func(t *testing.T) {\n\t\tdir := t.TempDir()\n\t\tclient := file.NewReplicaClient(dir)\n\n\t\t// Pre-create L0 files to simulate VFS writes\n\t\tcreateTestLTXFile(t, client, 0, 1, 1)\n\t\tcreateTestLTXFile(t, client, 0, 2, 2)\n\t\tcreateTestLTXFile(t, client, 0, 3, 3)\n\n\t\tcompactor := litestream.NewCompactor(client, slog.Default())\n\n\t\t// Compact L0 to L1\n\t\tinfo, err := compactor.Compact(context.Background(), 1)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.Level != 1 {\n\t\t\tt.Errorf(\"Level=%d, want 1\", info.Level)\n\t\t}\n\t\tt.Logf(\"Compacted to L1: minTXID=%d, maxTXID=%d\", info.MinTXID, info.MaxTXID)\n\n\t\t// Compact L1 to L2\n\t\tinfo, err = compactor.Compact(context.Background(), 2)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif info.Level != 2 {\n\t\t\tt.Errorf(\"Level=%d, want 2\", info.Level)\n\t\t}\n\t\tt.Logf(\"Compacted to L2: minTXID=%d, maxTXID=%d\", info.MinTXID, info.MaxTXID)\n\n\t\t// Verify L2 file exists\n\t\titr, err := client.LTXFiles(context.Background(), 2, 0, false)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdefer itr.Close()\n\n\t\tvar l2Count int\n\t\tfor itr.Next() {\n\t\t\tl2Count++\n\t\t}\n\t\tif l2Count != 1 {\n\t\t\tt.Errorf(\"L2 file count=%d, want 1\", l2Count)\n\t\t}\n\t})\n}\n\nfunc TestDefaultCompactionLevels(t *testing.T) {\n\tlevels := litestream.DefaultCompactionLevels\n\tif len(levels) != 4 {\n\t\tt.Fatalf(\"DefaultCompactionLevels length=%d, want 4\", len(levels))\n\t}\n\n\t// Verify L0 (raw files, no interval)\n\tif levels[0].Level != 0 {\n\t\tt.Errorf(\"levels[0].Level=%d, want 0\", levels[0].Level)\n\t}\n\tif levels[0].Interval != 0 {\n\t\tt.Errorf(\"levels[0].Interval=%v, want 0\", levels[0].Interval)\n\t}\n\n\t// Verify L1 (30 second compaction)\n\tif levels[1].Level != 1 {\n\t\tt.Errorf(\"levels[1].Level=%d, want 1\", levels[1].Level)\n\t}\n\tif levels[1].Interval != 30*time.Second {\n\t\tt.Errorf(\"levels[1].Interval=%v, want 30s\", levels[1].Interval)\n\t}\n\n\t// Verify L2 (5 minute compaction)\n\tif levels[2].Level != 2 {\n\t\tt.Errorf(\"levels[2].Level=%d, want 2\", levels[2].Level)\n\t}\n\tif levels[2].Interval != 5*time.Minute {\n\t\tt.Errorf(\"levels[2].Interval=%v, want 5m\", levels[2].Interval)\n\t}\n\n\t// Verify L3 (hourly compaction)\n\tif levels[3].Level != 3 {\n\t\tt.Errorf(\"levels[3].Level=%d, want 3\", levels[3].Level)\n\t}\n\tif levels[3].Interval != time.Hour {\n\t\tt.Errorf(\"levels[3].Interval=%v, want 1h\", levels[3].Interval)\n\t}\n\n\t// Verify they validate\n\tif err := levels.Validate(); err != nil {\n\t\tt.Errorf(\"DefaultCompactionLevels.Validate()=%v, want nil\", err)\n\t}\n}\n\nfunc TestVFS_CompactionConfig(t *testing.T) {\n\tt.Run(\"DefaultConfig\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tvfs := litestream.NewVFS(client, slog.Default())\n\n\t\t// Default should have compaction disabled\n\t\tif vfs.CompactionEnabled {\n\t\t\tt.Error(\"CompactionEnabled should be false by default\")\n\t\t}\n\t\tif vfs.CompactionLevels != nil {\n\t\t\tt.Error(\"CompactionLevels should be nil by default\")\n\t\t}\n\t\tif vfs.SnapshotInterval != 0 {\n\t\t\tt.Error(\"SnapshotInterval should be 0 by default\")\n\t\t}\n\t})\n\n\tt.Run(\"WithCompactionConfig\", func(t *testing.T) {\n\t\tclient := file.NewReplicaClient(t.TempDir())\n\t\tvfs := litestream.NewVFS(client, slog.Default())\n\t\tvfs.WriteEnabled = true\n\t\tvfs.CompactionEnabled = true\n\t\tvfs.CompactionLevels = litestream.CompactionLevels{\n\t\t\t{Level: 0, Interval: 0},\n\t\t\t{Level: 1, Interval: time.Minute},\n\t\t}\n\t\tvfs.SnapshotInterval = 24 * time.Hour\n\t\tvfs.SnapshotRetention = 7 * 24 * time.Hour\n\t\tvfs.L0Retention = 5 * time.Minute\n\n\t\tif !vfs.CompactionEnabled {\n\t\t\tt.Error(\"CompactionEnabled should be true\")\n\t\t}\n\t\tif len(vfs.CompactionLevels) != 2 {\n\t\t\tt.Errorf(\"CompactionLevels length=%d, want 2\", len(vfs.CompactionLevels))\n\t\t}\n\t\tif vfs.SnapshotInterval != 24*time.Hour {\n\t\t\tt.Errorf(\"SnapshotInterval=%v, want 24h\", vfs.SnapshotInterval)\n\t\t}\n\t\tif vfs.SnapshotRetention != 7*24*time.Hour {\n\t\t\tt.Errorf(\"SnapshotRetention=%v, want 168h\", vfs.SnapshotRetention)\n\t\t}\n\t\tif vfs.L0Retention != 5*time.Minute {\n\t\t\tt.Errorf(\"L0Retention=%v, want 5m\", vfs.L0Retention)\n\t\t}\n\t})\n}\n" }, { "path": "vfs_test.go", "content": "//go:build vfs\n\npackage litestream\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/psanford/sqlite3vfs\"\n\t\"github.com/superfly/ltx\"\n)\n\nfunc TestVFSFile_LockStateMachine(t *testing.T) {\n\tf := &VFSFile{logger: slog.Default(), writeEnabled: true}\n\tf.cond = sync.NewCond(&f.mu)\n\n\tif err := f.Lock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatalf(\"lock shared: %v\", err)\n\t}\n\tif reserved, _ := f.CheckReservedLock(); reserved {\n\t\tt.Fatalf(\"shared lock should not report reserved\")\n\t}\n\n\tif err := f.Lock(sqlite3vfs.LockReserved); err != nil {\n\t\tt.Fatalf(\"lock reserved: %v\", err)\n\t}\n\tif reserved, _ := f.CheckReservedLock(); !reserved {\n\t\tt.Fatalf(\"reserved lock should report reserved\")\n\t}\n\n\tif err := f.Lock(sqlite3vfs.LockShared); err == nil {\n\t\tt.Fatalf(\"expected downgrade via Lock to fail\")\n\t}\n\n\tif err := f.Unlock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatalf(\"unlock to shared: %v\", err)\n\t}\n\tif reserved, _ := f.CheckReservedLock(); reserved {\n\t\tt.Fatalf(\"unlock to shared should clear reserved state\")\n\t}\n\n\tif err := f.Unlock(sqlite3vfs.LockPending); err == nil {\n\t\tt.Fatalf(\"expected unlock to pending to fail\")\n\t}\n\n\tif err := f.Lock(sqlite3vfs.LockExclusive); err != nil {\n\t\tt.Fatalf(\"lock exclusive: %v\", err)\n\t}\n\n\tif err := f.Unlock(sqlite3vfs.LockNone); err != nil {\n\t\tt.Fatalf(\"unlock to none: %v\", err)\n\t}\n}\n\nfunc TestVFSFile_PendingIndexIsolation(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'a'))\n\n\tf := NewVFSFile(client, \"test.db\", slog.Default())\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\n\tif err := f.Lock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatalf(\"lock shared: %v\", err)\n\t}\n\n\tclient.addFixture(t, buildLTXFixture(t, 2, 'b'))\n\tif err := f.pollReplicaClient(context.Background()); err != nil {\n\t\tt.Fatalf(\"poll replica: %v\", err)\n\t}\n\n\tf.mu.Lock()\n\tpendingLen := len(f.pending)\n\tcurrent := f.index[1]\n\tf.mu.Unlock()\n\n\tif pendingLen == 0 {\n\t\tt.Fatalf(\"expected pending index entries while shared lock held\")\n\t}\n\tif current.MinTXID != 1 {\n\t\tt.Fatalf(\"main index should still reference first txid, got %s\", current.MinTXID)\n\t}\n\n\tbuf := make([]byte, 4096)\n\tif _, err := f.ReadAt(buf, 0); err != nil {\n\t\tt.Fatalf(\"read during lock: %v\", err)\n\t}\n\tif buf[0] != 'a' {\n\t\tt.Fatalf(\"expected old data during lock, got %q\", buf[0])\n\t}\n\n\tif err := f.Unlock(sqlite3vfs.LockNone); err != nil {\n\t\tt.Fatalf(\"unlock: %v\", err)\n\t}\n\n\tif _, err := f.ReadAt(buf, 0); err != nil {\n\t\tt.Fatalf(\"read after unlock: %v\", err)\n\t}\n\tif buf[0] != 'b' {\n\t\tt.Fatalf(\"expected updated data after unlock, got %q\", buf[0])\n\t}\n}\n\nfunc TestVFSFile_PendingIndexRace(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'a'))\n\n\tf := NewVFSFile(client, \"race.db\", slog.Default())\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\n\tif err := f.Lock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatalf(\"lock shared: %v\", err)\n\t}\n\n\tctx, cancel := context.WithTimeout(context.Background(), 250*time.Millisecond)\n\tdefer cancel()\n\n\t// continuously stream new fixtures\n\tgo func() {\n\t\ttxid := ltx.TXID(2)\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\t\t\tclient.addFixture(t, buildLTXFixture(t, txid, byte('a'+int(txid%26))))\n\t\t\tif err := f.pollReplicaClient(context.Background()); err != nil {\n\t\t\t\tt.Errorf(\"poll replica: %v\", err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\ttxid++\n\t\t\ttime.Sleep(2 * time.Millisecond)\n\t\t}\n\t}()\n\n\tvar wg sync.WaitGroup\n\tfor i := 0; i < 8; i++ {\n\t\twg.Add(1)\n\t\tgo func(id int) {\n\t\t\tdefer wg.Done()\n\t\t\tbuf := make([]byte, 4096)\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn\n\t\t\t\tdefault:\n\t\t\t\t}\n\t\t\t\tif _, err := f.ReadAt(buf, 0); err != nil {\n\t\t\t\t\tt.Errorf(\"reader %d: %v\", id, err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}(i)\n\t}\n\n\t<-ctx.Done()\n\tf.Unlock(sqlite3vfs.LockNone)\n\twg.Wait()\n}\n\nfunc TestVFSFileMonitorStopsOnCancel(t *testing.T) {\n\tclient := newCountingReplicaClient()\n\tf := &VFSFile{client: client, logger: slog.Default(), PollInterval: 5 * time.Millisecond}\n\tctx, cancel := context.WithCancel(context.Background())\n\tvar wg sync.WaitGroup\n\twg.Add(1)\n\tgo func() { defer wg.Done(); f.monitorReplicaClient(ctx) }()\n\n\tdeadline := time.Now().Add(200 * time.Millisecond)\n\tfor time.Now().Before(deadline) {\n\t\tif client.calls.Load() > 0 {\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(1 * time.Millisecond)\n\t}\n\tif client.calls.Load() == 0 {\n\t\tt.Fatalf(\"monitor never invoked LTXFiles\")\n\t}\n\n\tcancel()\n\tfinished := make(chan struct{})\n\tgo func() {\n\t\twg.Wait()\n\t\tclose(finished)\n\t}()\n\n\tselect {\n\tcase <-finished:\n\tcase <-time.After(200 * time.Millisecond):\n\t\tt.Fatalf(\"monitor goroutine did not exit after cancel\")\n\t}\n}\n\nfunc TestVFSFile_NonContiguousTXIDError(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'a'))\n\n\tf := NewVFSFile(client, \"gap.db\", slog.Default())\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\n\tclient.addFixture(t, buildLTXFixture(t, 3, 'c'))\n\tif err := f.pollReplicaClient(context.Background()); err != nil {\n\t\tt.Fatalf(\"poll replica: %v\", err)\n\t}\n\tif pos := f.Pos(); pos.TXID != 1 {\n\t\tt.Fatalf(\"unexpected txid advance after gap: got %s\", pos.TXID.String())\n\t}\n}\n\nfunc TestVFSFile_IndexMemoryDoesNotGrowUnbounded(t *testing.T) {\n\tconst pageLimit = 16\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'a'))\n\n\tf := NewVFSFile(client, \"mem.db\", slog.Default())\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\n\tfor i := 0; i < 100; i++ {\n\t\tpgno := uint32(i%pageLimit) + 2\n\t\tclient.addFixture(t, buildLTXFixtureWithPages(t, ltx.TXID(i+2), 4096, []uint32{pgno}, byte('b'+byte(i%26))))\n\t\tif err := f.pollReplicaClient(context.Background()); err != nil {\n\t\t\tt.Fatalf(\"poll replica: %v\", err)\n\t\t}\n\t}\n\n\tf.mu.Lock()\n\tdefer f.mu.Unlock()\n\tif l := len(f.index); l > pageLimit+1 { // +1 for initial page 1\n\t\tt.Fatalf(\"index grew unexpectedly: got %d want <= %d\", l, pageLimit+1)\n\t}\n}\n\nfunc TestVFSFile_AutoVacuumShrinksCommit(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixtureWithPages(t, 1, 4096, []uint32{1, 2, 3, 4}, 'a'))\n\n\tf := NewVFSFile(client, \"autovac.db\", slog.Default())\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\n\tclient.addFixture(t, buildLTXFixtureWithPages(t, 2, 4096, []uint32{1, 2}, 'b'))\n\tif err := f.pollReplicaClient(context.Background()); err != nil {\n\t\tt.Fatalf(\"poll replica: %v\", err)\n\t}\n\n\tsize, err := f.FileSize()\n\tif err != nil {\n\t\tt.Fatalf(\"file size: %v\", err)\n\t}\n\tif size != int64(2*4096) {\n\t\tt.Fatalf(\"unexpected file size after vacuum: got %d want %d\", size, 2*4096)\n\t}\n\n\tbuf := make([]byte, 4096)\n\tlockOffset := int64(3-1) * 4096\n\tif _, err := f.ReadAt(buf, lockOffset); err == nil || !strings.Contains(err.Error(), \"page not found\") {\n\t\tt.Fatalf(\"expected missing page after vacuum, got %v\", err)\n\t}\n}\n\nfunc TestVFSFile_PendingIndexReplacementRemovesStalePages(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixtureWithPages(t, 1, 4096, []uint32{1, 2, 3, 4}, 'a'))\n\n\tf := NewVFSFile(client, \"pending-replace.db\", slog.Default())\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\n\tif err := f.Lock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatalf(\"lock shared: %v\", err)\n\t}\n\n\tclient.addFixture(t, buildLTXFixtureWithPages(t, 2, 4096, []uint32{1, 2}, 'b'))\n\tif err := f.pollReplicaClient(context.Background()); err != nil {\n\t\tt.Fatalf(\"poll replica: %v\", err)\n\t}\n\n\tf.mu.Lock()\n\tif _, ok := f.index[4]; !ok {\n\t\tt.Fatalf(\"expected stale page to remain in main index while lock is held\")\n\t}\n\tif !f.pendingReplace {\n\t\tt.Fatalf(\"expected pending replacement flag set\")\n\t}\n\tf.mu.Unlock()\n\n\tif err := f.Unlock(sqlite3vfs.LockNone); err != nil {\n\t\tt.Fatalf(\"unlock: %v\", err)\n\t}\n\n\tsize, err := f.FileSize()\n\tif err != nil {\n\t\tt.Fatalf(\"file size: %v\", err)\n\t}\n\tif size != int64(2*4096) {\n\t\tt.Fatalf(\"unexpected file size after pending replacement applied: got %d want %d\", size, 2*4096)\n\t}\n\n\tbuf := make([]byte, 4096)\n\tlockOffset := int64(3-1) * 4096\n\tif _, err := f.ReadAt(buf, lockOffset); err == nil || !strings.Contains(err.Error(), \"page not found\") {\n\t\tt.Fatalf(\"expected missing page after pending replacement applied, got %v\", err)\n\t}\n}\n\nfunc TestVFSFile_CorruptedPageIndexRecovery(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, <xFixture{info: <x.FileInfo{Level: 0, MinTXID: 1, MaxTXID: 1, Size: 0}, data: []byte(\"bad-index\")})\n\n\tf := NewVFSFile(client, \"corrupt.db\", slog.Default())\n\tif err := f.Open(); err == nil {\n\t\tt.Fatalf(\"expected open to fail on corrupted index\")\n\t}\n}\n\nfunc TestVFSFile_OpenSeedsLevel1Position(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tsnapshot := buildLTXFixture(t, 1, 's')\n\tsnapshot.info.Level = SnapshotLevel\n\tclient.addFixture(t, snapshot)\n\tl1 := buildLTXFixture(t, 2, 'l')\n\tl1.info.Level = 1\n\tclient.addFixture(t, l1)\n\tl0 := buildLTXFixture(t, 3, 'z')\n\tl0.info.Level = 0\n\tclient.addFixture(t, l0)\n\n\tf := NewVFSFile(client, \"seed-level1.db\", slog.Default())\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\tdefer f.Close()\n\n\tif got, want := f.maxTXID1, l1.info.MaxTXID; got != want {\n\t\tt.Fatalf(\"unexpected maxTXID1: got %s want %s\", got, want)\n\t}\n\tif got, want := f.Pos().TXID, l0.info.MaxTXID; got != want {\n\t\tt.Fatalf(\"unexpected pos after open: got %s want %s\", got, want)\n\t}\n}\n\nfunc TestVFSFile_OpenSeedsLevel1PositionFromPos(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tsnapshot := buildLTXFixture(t, 1, 's')\n\tsnapshot.info.Level = SnapshotLevel\n\tclient.addFixture(t, snapshot)\n\tl0 := buildLTXFixture(t, 2, '0')\n\tl0.info.Level = 0\n\tclient.addFixture(t, l0)\n\n\tf := NewVFSFile(client, \"seed-default.db\", slog.Default())\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\tdefer f.Close()\n\n\tpos := f.Pos().TXID\n\tif pos == 0 {\n\t\tt.Fatalf(\"expected non-zero position\")\n\t}\n\tif got := f.maxTXID1; got != pos {\n\t\tt.Fatalf(\"expected maxTXID1 to equal pos when no L1 files, got %s want %s\", got, pos)\n\t}\n}\n\nfunc TestVFSFile_HeaderForcesDeleteJournal(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'h'))\n\n\tf := NewVFSFile(client, \"header.db\", slog.Default())\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\tdefer f.Close()\n\n\tbuf := make([]byte, 32)\n\tif _, err := f.ReadAt(buf, 0); err != nil {\n\t\tt.Fatalf(\"read header: %v\", err)\n\t}\n\tif buf[18] != 0x01 || buf[19] != 0x01 {\n\t\tt.Fatalf(\"journal mode bytes not forced to DELETE, got %x %x\", buf[18], buf[19])\n\t}\n}\n\nfunc TestVFSFile_ReadAtLockPageBoundary(t *testing.T) {\n\tpageSizes := []uint32{512, 1024, 2048, 4096, 8192, 16384, 32768, 65536}\n\tfor _, pageSize := range pageSizes {\n\t\tpageSize := pageSize\n\t\tt.Run(fmt.Sprintf(\"page_%d\", pageSize), func(t *testing.T) {\n\t\t\tclient := newMockReplicaClient()\n\t\t\tlockPgno := ltx.LockPgno(pageSize)\n\t\t\tbefore := lockPgno - 1\n\t\t\tafter := lockPgno + 1\n\n\t\t\tclient.addFixture(t, buildLTXFixtureWithPage(t, 1, pageSize, 1, 'z'))\n\t\t\tclient.addFixture(t, buildLTXFixtureWithPage(t, 2, pageSize, before, 'b'))\n\t\t\tclient.addFixture(t, buildLTXFixtureWithPage(t, 3, pageSize, after, 'a'))\n\n\t\t\tf := NewVFSFile(client, fmt.Sprintf(\"lock-boundary-%d.db\", pageSize), slog.Default())\n\t\t\tif err := f.Open(); err != nil {\n\t\t\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t\t\t}\n\t\t\tdefer f.Close()\n\n\t\t\tbuf := make([]byte, int(pageSize))\n\t\t\toff := int64(before-1) * int64(pageSize)\n\t\t\tif _, err := f.ReadAt(buf, off); err != nil {\n\t\t\t\tt.Fatalf(\"read before lock page: %v\", err)\n\t\t\t}\n\t\t\tif buf[0] != 'b' {\n\t\t\t\tt.Fatalf(\"unexpected data before lock page: got %q\", buf[0])\n\t\t\t}\n\n\t\t\tbuf = make([]byte, int(pageSize))\n\t\t\toff = int64(after-1) * int64(pageSize)\n\t\t\tif _, err := f.ReadAt(buf, off); err != nil {\n\t\t\t\tt.Fatalf(\"read after lock page: %v\", err)\n\t\t\t}\n\t\t\tif buf[0] != 'a' {\n\t\t\t\tt.Fatalf(\"unexpected data after lock page: got %q\", buf[0])\n\t\t\t}\n\n\t\t\tbuf = make([]byte, int(pageSize))\n\t\t\tlockOffset := int64(lockPgno-1) * int64(pageSize)\n\t\t\tif _, err := f.ReadAt(buf, lockOffset); err == nil || !strings.Contains(err.Error(), \"page not found\") {\n\t\t\t\tt.Fatalf(\"expected missing lock page error, got %v\", err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVFS_TempFileLifecycleStress(t *testing.T) {\n\tvfs := NewVFS(nil, slog.Default())\n\tconst (\n\t\tworkers = 8\n\t\titerations = 50\n\t)\n\n\tvar wg sync.WaitGroup\n\terrCh := make(chan error, workers)\n\tfor w := 0; w < workers; w++ {\n\t\tw := w\n\t\twg.Add(1)\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\tfor i := 0; i < iterations; i++ {\n\t\t\t\tname := fmt.Sprintf(\"temp-%02d-%02d.db\", w, i)\n\t\t\t\tflags := sqlite3vfs.OpenTempDB | sqlite3vfs.OpenReadWrite | sqlite3vfs.OpenCreate\n\t\t\t\tdeleteOnClose := (w+i)%2 == 0\n\t\t\t\tif deleteOnClose {\n\t\t\t\t\tflags |= sqlite3vfs.OpenDeleteOnClose\n\t\t\t\t}\n\n\t\t\t\tfile, _, err := vfs.openTempFile(name, flags)\n\t\t\t\tif err != nil {\n\t\t\t\t\terrCh <- fmt.Errorf(\"open temp file: %w\", err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\ttf := file.(*localTempFile)\n\t\t\t\tif _, err := tf.WriteAt([]byte(\"hot-data\"), 0); err != nil {\n\t\t\t\t\terrCh <- fmt.Errorf(\"write temp file: %w\", err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tpath, tracked := vfs.loadTempFilePath(name)\n\t\t\t\tif !tracked && name != \"\" {\n\t\t\t\t\terrCh <- fmt.Errorf(\"temp file %s was not tracked\", name)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif err := tf.Close(); err != nil {\n\t\t\t\t\terrCh <- fmt.Errorf(\"close temp file: %w\", err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif deleteOnClose {\n\t\t\t\t\tif path != \"\" {\n\t\t\t\t\t\tif _, err := os.Stat(path); err == nil || !os.IsNotExist(err) {\n\t\t\t\t\t\t\terrCh <- fmt.Errorf(\"delete-on-close leaked temp file %s\", path)\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\tif path == \"\" {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"missing tracked path for %s\", name)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tif _, err := os.Stat(path); err != nil {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"expected temp file on disk: %v\", err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tif err := os.Remove(path); err != nil {\n\t\t\t\t\t\terrCh <- fmt.Errorf(\"cleanup temp file: %v\", err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\t}\n\n\twg.Wait()\n\tclose(errCh)\n\tfor err := range errCh {\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"temp file stress: %v\", err)\n\t\t}\n\t}\n\n\tleak := false\n\tvfs.tempFiles.Range(func(key, value any) bool {\n\t\tleak = true\n\t\treturn false\n\t})\n\tif leak {\n\t\tt.Fatalf(\"temp files still tracked after stress run\")\n\t}\n\n\tif dir := vfs.tempDir; dir != \"\" {\n\t\tentries, err := os.ReadDir(dir)\n\t\tif err != nil && !os.IsNotExist(err) {\n\t\t\tt.Fatalf(\"read temp dir: %v\", err)\n\t\t}\n\t\tif err == nil && len(entries) > 0 {\n\t\t\tnames := make([]string, 0, len(entries))\n\t\t\tfor _, entry := range entries {\n\t\t\t\tnames = append(names, entry.Name())\n\t\t\t}\n\t\t\tt.Fatalf(\"temp dir not cleaned: %v\", names)\n\t\t}\n\t}\n}\n\nfunc TestVFS_TempFileNameCollision(t *testing.T) {\n\tvfs := NewVFS(nil, slog.Default())\n\tname := \"collision.db\"\n\tflags := sqlite3vfs.OpenTempDB | sqlite3vfs.OpenReadWrite | sqlite3vfs.OpenCreate\n\n\tfile1, _, err := vfs.openTempFile(name, flags)\n\tif err != nil {\n\t\tt.Fatalf(\"open temp file1: %v\", err)\n\t}\n\ttf1 := file1.(*localTempFile)\n\tpath1, ok := vfs.loadTempFilePath(name)\n\tif !ok {\n\t\tt.Fatalf(\"first temp file not tracked\")\n\t}\n\n\tfile2, _, err := vfs.openTempFile(name, flags|sqlite3vfs.OpenDeleteOnClose)\n\tif err != nil {\n\t\tt.Fatalf(\"open temp file2: %v\", err)\n\t}\n\ttf2 := file2.(*localTempFile)\n\tpath2, ok := vfs.loadTempFilePath(name)\n\tif !ok {\n\t\tt.Fatalf(\"second temp file not tracked\")\n\t}\n\tif path1 != path2 {\n\t\tt.Fatalf(\"expected same canonical path, got %s vs %s\", path1, path2)\n\t}\n\n\tif err := tf2.Close(); err != nil {\n\t\tt.Fatalf(\"close second file: %v\", err)\n\t}\n\tif _, err := os.Stat(path2); err == nil || !os.IsNotExist(err) {\n\t\tt.Fatalf(\"expected file removed after delete-on-close\")\n\t}\n\tif _, ok := vfs.loadTempFilePath(name); ok {\n\t\tt.Fatalf(\"canonical entry should be cleared after delete-on-close\")\n\t}\n\tif err := tf1.Close(); err != nil {\n\t\tt.Fatalf(\"close first file: %v\", err)\n\t}\n}\n\nfunc TestVFS_TempFileSameBasenameDifferentDirs(t *testing.T) {\n\tvfs := NewVFS(nil, slog.Default())\n\tflags := sqlite3vfs.OpenTempDB | sqlite3vfs.OpenReadWrite | sqlite3vfs.OpenCreate\n\n\tname1 := filepath.Join(\"foo\", \"shared.db\")\n\tname2 := filepath.Join(\"bar\", \"shared.db\")\n\n\tfile1, _, err := vfs.openTempFile(name1, flags)\n\tif err != nil {\n\t\tt.Fatalf(\"open first temp file: %v\", err)\n\t}\n\ttf1 := file1.(*localTempFile)\n\tpath1, ok := vfs.loadTempFilePath(name1)\n\tif !ok {\n\t\tt.Fatalf(\"first temp file not tracked\")\n\t}\n\n\tfile2, _, err := vfs.openTempFile(name2, flags|sqlite3vfs.OpenDeleteOnClose)\n\tif err != nil {\n\t\tt.Fatalf(\"open second temp file: %v\", err)\n\t}\n\ttf2 := file2.(*localTempFile)\n\tpath2, ok := vfs.loadTempFilePath(name2)\n\tif !ok {\n\t\tt.Fatalf(\"second temp file not tracked\")\n\t}\n\n\tif path1 == path2 {\n\t\tt.Fatalf(\"expected unique paths for %s and %s\", name1, name2)\n\t}\n\n\tif err := tf1.Close(); err != nil {\n\t\tt.Fatalf(\"close first file: %v\", err)\n\t}\n\n\tif _, ok := vfs.loadTempFilePath(name2); !ok {\n\t\tt.Fatalf(\"closing first file should not unregister second\")\n\t}\n\n\tif path1 != \"\" {\n\t\tif err := os.Remove(path1); err != nil && !os.IsNotExist(err) {\n\t\t\tt.Fatalf(\"cleanup first temp file: %v\", err)\n\t\t}\n\t}\n\n\tif err := tf2.Close(); err != nil {\n\t\tt.Fatalf(\"close second file: %v\", err)\n\t}\n\tif _, ok := vfs.loadTempFilePath(name2); ok {\n\t\tt.Fatalf(\"delete-on-close should clear second temp file\")\n\t}\n}\n\nfunc TestVFS_TempFileDeleteOnClose(t *testing.T) {\n\tvfs := NewVFS(nil, slog.Default())\n\tname := \"delete-on-close.db\"\n\tflags := sqlite3vfs.OpenTempDB | sqlite3vfs.OpenReadWrite | sqlite3vfs.OpenCreate | sqlite3vfs.OpenDeleteOnClose\n\n\tfile, _, err := vfs.openTempFile(name, flags)\n\tif err != nil {\n\t\tt.Fatalf(\"open temp file: %v\", err)\n\t}\n\ttf := file.(*localTempFile)\n\tpath, ok := vfs.loadTempFilePath(name)\n\tif !ok {\n\t\tt.Fatalf(\"temp file not tracked\")\n\t}\n\n\tif _, err := tf.WriteAt([]byte(\"x\"), 0); err != nil {\n\t\tt.Fatalf(\"write temp file: %v\", err)\n\t}\n\tif err := tf.Close(); err != nil {\n\t\tt.Fatalf(\"close temp file: %v\", err)\n\t}\n\tif _, err := os.Stat(path); err == nil || !os.IsNotExist(err) {\n\t\tt.Fatalf(\"expected delete-on-close to remove temp file\")\n\t}\n\tif _, ok := vfs.loadTempFilePath(name); ok {\n\t\tt.Fatalf(\"temp file tracking entry should be cleared\")\n\t}\n\tif err := vfs.Delete(name, false); err != nil {\n\t\tt.Fatalf(\"delete should ignore missing temp files: %v\", err)\n\t}\n\tif err := vfs.Delete(name, false); err != nil {\n\t\tt.Fatalf(\"delete should ignore repeated temp deletes: %v\", err)\n\t}\n}\n\nfunc TestLocalTempFileLocking(t *testing.T) {\n\tf, err := os.CreateTemp(t.TempDir(), \"local-temp-*\")\n\tif err != nil {\n\t\tt.Fatalf(\"create temp: %v\", err)\n\t}\n\ttf := newLocalTempFile(f, false, nil)\n\tdefer tf.Close()\n\n\tassertReserved := func(want bool) {\n\t\tt.Helper()\n\t\tgot, err := tf.CheckReservedLock()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"check reserved: %v\", err)\n\t\t}\n\t\tif got != want {\n\t\t\tt.Fatalf(\"reserved lock state mismatch: got %v want %v\", got, want)\n\t\t}\n\t}\n\n\tassertReserved(false)\n\n\tif err := tf.Lock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatalf(\"lock shared: %v\", err)\n\t}\n\tassertReserved(false)\n\n\tif err := tf.Lock(sqlite3vfs.LockReserved); err != nil {\n\t\tt.Fatalf(\"lock reserved: %v\", err)\n\t}\n\tassertReserved(true)\n\n\tif err := tf.Unlock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatalf(\"unlock shared: %v\", err)\n\t}\n\tassertReserved(false)\n\n\tif err := tf.Lock(sqlite3vfs.LockExclusive); err != nil {\n\t\tt.Fatalf(\"lock exclusive: %v\", err)\n\t}\n\tassertReserved(true)\n\n\tif err := tf.Unlock(sqlite3vfs.LockNone); err != nil {\n\t\tt.Fatalf(\"unlock none: %v\", err)\n\t}\n\tassertReserved(false)\n}\n\nfunc TestVFS_DeleteIgnoresMissingTempFiles(t *testing.T) {\n\tvfs := NewVFS(nil, slog.Default())\n\n\tt.Run(\"AlreadyRemovedEntry\", func(t *testing.T) {\n\t\tname := \"already-removed.db\"\n\t\tflags := sqlite3vfs.OpenTempDB | sqlite3vfs.OpenReadWrite | sqlite3vfs.OpenCreate | sqlite3vfs.OpenDeleteOnClose\n\n\t\tfile, _, err := vfs.openTempFile(name, flags)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"open temp file: %v\", err)\n\t\t}\n\t\ttf := file.(*localTempFile)\n\t\tif err := tf.Close(); err != nil {\n\t\t\tt.Fatalf(\"close temp file: %v\", err)\n\t\t}\n\t\tif err := vfs.Delete(name, false); err != nil {\n\t\t\tt.Fatalf(\"delete should ignore missing tracked entry: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"MissingOnDisk\", func(t *testing.T) {\n\t\tname := \"missing-on-disk.db\"\n\t\tflags := sqlite3vfs.OpenTempDB | sqlite3vfs.OpenReadWrite | sqlite3vfs.OpenCreate\n\n\t\tfile, _, err := vfs.openTempFile(name, flags)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"open temp file: %v\", err)\n\t\t}\n\t\ttf := file.(*localTempFile)\n\n\t\tpath, ok := vfs.loadTempFilePath(name)\n\t\tif !ok {\n\t\t\tt.Fatalf(\"temp file not tracked\")\n\t\t}\n\t\tif err := os.Remove(path); err != nil {\n\t\t\tt.Fatalf(\"remove backing file: %v\", err)\n\t\t}\n\t\tif err := vfs.Delete(name, false); err != nil {\n\t\t\tt.Fatalf(\"delete should ignore missing file: %v\", err)\n\t\t}\n\t\tif _, ok := vfs.loadTempFilePath(name); ok {\n\t\t\tt.Fatalf(\"temp file tracking entry should be cleared\")\n\t\t}\n\t\tif err := tf.Close(); err != nil {\n\t\t\tt.Fatalf(\"close temp file: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestVFS_TempDirExhaustion(t *testing.T) {\n\tvfs := NewVFS(nil, slog.Default())\n\tinjected := fmt.Errorf(\"temp dir exhausted\")\n\tvfs.tempDirOnce.Do(func() { vfs.tempDirErr = injected })\n\n\tif _, err := vfs.ensureTempDir(); !errors.Is(err, injected) {\n\t\tt.Fatalf(\"expected ensureTempDir error, got %v\", err)\n\t}\n\n\tif _, _, err := vfs.openTempFile(\"exhausted.db\", sqlite3vfs.OpenTempDB); !errors.Is(err, injected) {\n\t\tt.Fatalf(\"openTempFile should surface exhaustion error, got %v\", err)\n\t}\n}\n\nfunc TestVFSFile_PollingCancelsBlockedLTXFiles(t *testing.T) {\n\tclient := newBlockingReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'a'))\n\n\tf := NewVFSFile(client, \"blocking.db\", slog.Default())\n\tf.PollInterval = 5 * time.Millisecond\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\n\tclient.blockNext.Store(true)\n\tdeadline := time.After(200 * time.Millisecond)\n\tselect {\n\tcase <-client.blocked:\n\tcase <-deadline:\n\t\tt.Fatalf(\"expected monitor to block on LTXFiles\")\n\t}\n\n\tdone := make(chan struct{})\n\tgo func() {\n\t\t_ = f.Close()\n\t\tclose(done)\n\t}()\n\n\tselect {\n\tcase <-done:\n\tcase <-time.After(500 * time.Millisecond):\n\t\tt.Fatalf(\"close did not unblock blocked LTXFiles call\")\n\t}\n\n\tif !client.cancelled.Load() {\n\t\tt.Fatalf(\"blocking client did not observe context cancellation\")\n\t}\n}\n\n// mockReplicaClient implements ReplicaClient for deterministic LTX fixtures.\ntype mockReplicaClient struct {\n\tmu sync.Mutex\n\tfiles []*ltx.FileInfo\n\tdata map[string][]byte\n}\n\ntype blockingReplicaClient struct {\n\t*mockReplicaClient\n\tblockNext atomic.Bool\n\tblocked chan struct{}\n\tcancelled atomic.Bool\n\tonce sync.Once\n}\n\ntype countingReplicaClient struct {\n\tcalls atomic.Uint64\n}\n\nfunc newCountingReplicaClient() *countingReplicaClient { return &countingReplicaClient{} }\n\nfunc (c *countingReplicaClient) Type() string { return \"count\" }\n\nfunc (c *countingReplicaClient) Init(context.Context) error { return nil }\n\nfunc (c *countingReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tc.calls.Add(1)\n\treturn ltx.NewFileInfoSliceIterator(nil), nil\n}\n\nfunc (c *countingReplicaClient) OpenLTXFile(context.Context, int, ltx.TXID, ltx.TXID, int64, int64) (io.ReadCloser, error) {\n\treturn io.NopCloser(bytes.NewReader(nil)), nil\n}\n\nfunc (c *countingReplicaClient) WriteLTXFile(context.Context, int, ltx.TXID, ltx.TXID, io.Reader) (*ltx.FileInfo, error) {\n\treturn nil, fmt.Errorf(\"not implemented\")\n}\n\nfunc (c *countingReplicaClient) DeleteLTXFiles(context.Context, []*ltx.FileInfo) error { return nil }\n\nfunc (c *countingReplicaClient) DeleteAll(context.Context) error { return nil }\n\nfunc newMockReplicaClient() *mockReplicaClient {\n\treturn &mockReplicaClient{data: make(map[string][]byte)}\n}\n\nfunc newBlockingReplicaClient() *blockingReplicaClient {\n\treturn &blockingReplicaClient{\n\t\tmockReplicaClient: newMockReplicaClient(),\n\t\tblocked: make(chan struct{}),\n\t}\n}\n\nfunc (c *mockReplicaClient) Type() string { return \"mock\" }\n\nfunc (c *mockReplicaClient) Init(context.Context) error { return nil }\n\nfunc (c *mockReplicaClient) addFixture(tb testing.TB, fx *ltxFixture) {\n\ttb.Helper()\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\tc.files = append(c.files, fx.info)\n\tc.data[c.key(fx.info)] = fx.data\n}\n\nfunc (c *mockReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\tvar out []*ltx.FileInfo\n\tfor _, info := range c.files {\n\t\tif info.Level == level && info.MinTXID >= seek {\n\t\t\tout = append(out, info)\n\t\t}\n\t}\n\treturn ltx.NewFileInfoSliceIterator(out), nil\n}\n\nfunc (c *mockReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\tkey := c.makeKey(level, minTXID, maxTXID)\n\tdata, ok := c.data[key]\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"ltx file not found\")\n\t}\n\tif offset > int64(len(data)) {\n\t\treturn nil, fmt.Errorf(\"offset beyond data\")\n\t}\n\tslice := data[offset:]\n\tif size > 0 && size < int64(len(slice)) {\n\t\tslice = slice[:size]\n\t}\n\treturn io.NopCloser(bytes.NewReader(slice)), nil\n}\n\nfunc (c *mockReplicaClient) WriteLTXFile(context.Context, int, ltx.TXID, ltx.TXID, io.Reader) (*ltx.FileInfo, error) {\n\treturn nil, fmt.Errorf(\"not implemented\")\n}\n\nfunc (c *mockReplicaClient) DeleteLTXFiles(context.Context, []*ltx.FileInfo) error {\n\treturn fmt.Errorf(\"not implemented\")\n}\n\nfunc (c *mockReplicaClient) DeleteAll(context.Context) error {\n\treturn fmt.Errorf(\"not implemented\")\n}\n\nfunc (c *blockingReplicaClient) Type() string { return \"blocking\" }\n\nfunc (c *blockingReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif seek > 1 && c.blockNext.Load() {\n\t\tif c.blockNext.CompareAndSwap(true, false) {\n\t\t\tc.once.Do(func() { close(c.blocked) })\n\t\t\t<-ctx.Done()\n\t\t\tc.cancelled.Store(true)\n\t\t\treturn nil, ctx.Err()\n\t\t}\n\t}\n\treturn c.mockReplicaClient.LTXFiles(ctx, level, seek, useMetadata)\n}\n\nfunc (c *blockingReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\treturn c.mockReplicaClient.OpenLTXFile(ctx, level, minTXID, maxTXID, offset, size)\n}\n\nfunc (c *blockingReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\treturn c.mockReplicaClient.WriteLTXFile(ctx, level, minTXID, maxTXID, r)\n}\n\nfunc (c *blockingReplicaClient) DeleteLTXFiles(ctx context.Context, files []*ltx.FileInfo) error {\n\treturn c.mockReplicaClient.DeleteLTXFiles(ctx, files)\n}\n\nfunc (c *blockingReplicaClient) DeleteAll(ctx context.Context) error {\n\treturn c.mockReplicaClient.DeleteAll(ctx)\n}\n\nfunc (c *mockReplicaClient) key(info *ltx.FileInfo) string {\n\treturn c.makeKey(info.Level, info.MinTXID, info.MaxTXID)\n}\n\nfunc (c *mockReplicaClient) makeKey(level int, minTXID, maxTXID ltx.TXID) string {\n\treturn fmt.Sprintf(\"%d:%s:%s\", level, minTXID.String(), maxTXID.String())\n}\n\ntype ltxFixture struct {\n\tinfo *ltx.FileInfo\n\tdata []byte\n}\n\nfunc buildLTXFixture(tb testing.TB, txid ltx.TXID, fill byte) *ltxFixture {\n\treturn buildLTXFixtureWithPage(tb, txid, 4096, 1, fill)\n}\n\nfunc buildLTXFixtureWithPage(tb testing.TB, txid ltx.TXID, pageSize, pgno uint32, fill byte) *ltxFixture {\n\treturn buildLTXFixtureWithPages(tb, txid, pageSize, []uint32{pgno}, fill)\n}\n\nfunc buildLTXFixtureWithPages(tb testing.TB, txid ltx.TXID, pageSize uint32, pgnos []uint32, fill byte) *ltxFixture {\n\ttb.Helper()\n\tif len(pgnos) == 0 {\n\t\ttb.Fatalf(\"pgnos required\")\n\t}\n\tif txid == 1 {\n\t\tif len(pgnos) == 0 || pgnos[0] != 1 {\n\t\t\ttb.Fatalf(\"snapshot fixture must start at page 1\")\n\t\t}\n\t}\n\n\tvar buf bytes.Buffer\n\tenc, err := ltx.NewEncoder(&buf)\n\tif err != nil {\n\t\ttb.Fatalf(\"new encoder: %v\", err)\n\t}\n\tmaxPg := uint32(0)\n\tfor _, pg := range pgnos {\n\t\tif pg > maxPg {\n\t\t\tmaxPg = pg\n\t\t}\n\t}\n\tif maxPg == 0 {\n\t\tmaxPg = 1\n\t}\n\thdr := ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tPageSize: pageSize,\n\t\tCommit: maxPg,\n\t\tMinTXID: txid,\n\t\tMaxTXID: txid,\n\t\tTimestamp: time.Now().UnixMilli(),\n\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t}\n\tif err := enc.EncodeHeader(hdr); err != nil {\n\t\ttb.Fatalf(\"encode header: %v\", err)\n\t}\n\tfor _, pg := range pgnos {\n\t\tif pg == 0 {\n\t\t\tpg = 1\n\t\t}\n\t\tpage := bytes.Repeat([]byte{fill}, int(pageSize))\n\t\tif err := enc.EncodePage(ltx.PageHeader{Pgno: pg}, page); err != nil {\n\t\t\ttb.Fatalf(\"encode page %d: %v\", pg, err)\n\t\t}\n\t}\n\tif err := enc.Close(); err != nil {\n\t\ttb.Fatalf(\"close encoder: %v\", err)\n\t}\n\n\tinfo := <x.FileInfo{\n\t\tLevel: 0,\n\t\tMinTXID: txid,\n\t\tMaxTXID: txid,\n\t\tSize: int64(buf.Len()),\n\t\tCreatedAt: time.Now().UTC(),\n\t}\n\n\treturn <xFixture{info: info, data: buf.Bytes()}\n}\n\n// TestVFSFile_Hydration_Basic tests that hydration completes and reads from local file.\nfunc TestVFSFile_Hydration_Basic(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'a'))\n\n\t// Create temp directory for hydration\n\thydrationDir := t.TempDir()\n\n\t// Create VFSFile with hydration enabled\n\tf := NewVFSFile(client, \"test.db\", slog.Default())\n\tf.hydrationPath = filepath.Join(hydrationDir, \"test.db.hydration.db\")\n\tf.PollInterval = 100 * time.Millisecond\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\tdefer f.Close()\n\n\t// Wait for hydration to complete\n\tdeadline := time.Now().Add(5 * time.Second)\n\tfor f.hydrator == nil || !f.hydrator.Complete() {\n\t\tif time.Now().After(deadline) {\n\t\t\tt.Fatalf(\"hydration did not complete in time\")\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\n\t// Verify hydration file exists\n\tif _, err := os.Stat(f.hydrationPath); err != nil {\n\t\tt.Fatalf(\"hydration file not found: %v\", err)\n\t}\n\n\t// Read a page - should come from hydrated file\n\tbuf := make([]byte, 4096)\n\tif _, err := f.ReadAt(buf, 0); err != nil {\n\t\tt.Fatalf(\"read at: %v\", err)\n\t}\n\n\t// Check that the data matches (excluding modified header bytes)\n\tfor i := 28; i < len(buf); i++ {\n\t\tif buf[i] != 'a' {\n\t\t\tt.Fatalf(\"expected byte 'a' at position %d, got %q\", i, buf[i])\n\t\t}\n\t}\n}\n\n// TestVFSFile_Hydration_ReadsDuringHydration tests that reads work via cache/remote during hydration.\nfunc TestVFSFile_Hydration_ReadsDuringHydration(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'b'))\n\n\thydrationDir := t.TempDir()\n\n\tf := NewVFSFile(client, \"test.db\", slog.Default())\n\tf.hydrationPath = filepath.Join(hydrationDir, \"test.db.hydration.db\")\n\tf.PollInterval = 100 * time.Millisecond\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\tdefer f.Close()\n\n\t// Read immediately - should work even if hydration is still in progress\n\tbuf := make([]byte, 4096)\n\tif _, err := f.ReadAt(buf, 0); err != nil {\n\t\tt.Fatalf(\"read at during hydration: %v\", err)\n\t}\n\n\t// Data should be correct regardless of hydration status\n\tfor i := 28; i < len(buf); i++ {\n\t\tif buf[i] != 'b' {\n\t\t\tt.Fatalf(\"expected byte 'b' at position %d, got %q\", i, buf[i])\n\t\t}\n\t}\n}\n\n// TestVFSFile_Hydration_CloseEarly tests clean shutdown during hydration.\nfunc TestVFSFile_Hydration_CloseEarly(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'c'))\n\n\thydrationDir := t.TempDir()\n\n\tf := NewVFSFile(client, \"test.db\", slog.Default())\n\tf.hydrationPath = filepath.Join(hydrationDir, \"test.db.hydration.db\")\n\tf.PollInterval = 100 * time.Millisecond\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\n\t// Close immediately without waiting for hydration\n\tif err := f.Close(); err != nil {\n\t\tt.Fatalf(\"close: %v\", err)\n\t}\n\n\t// Hydration file should be removed\n\tif _, err := os.Stat(f.hydrationPath); !os.IsNotExist(err) {\n\t\tt.Fatalf(\"hydration file should be removed after close\")\n\t}\n}\n\n// TestVFSFile_Hydration_Disabled tests that hydration has no effect when disabled.\nfunc TestVFSFile_Hydration_Disabled(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'd'))\n\n\tf := NewVFSFile(client, \"test.db\", slog.Default())\n\t// hydrationPath is empty by default (hydration disabled)\n\tf.PollInterval = 100 * time.Millisecond\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\tdefer f.Close()\n\n\t// Hydrator should be nil when hydration is disabled\n\tif f.hydrator != nil {\n\t\tt.Fatalf(\"hydrator should be nil when disabled\")\n\t}\n\n\t// Reads should still work via cache/remote\n\tbuf := make([]byte, 4096)\n\tif _, err := f.ReadAt(buf, 0); err != nil {\n\t\tt.Fatalf(\"read at: %v\", err)\n\t}\n}\n\n// TestVFSFile_Hydration_IncrementalUpdates tests that new LTX files are applied to hydrated file.\nfunc TestVFSFile_Hydration_IncrementalUpdates(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'e'))\n\n\thydrationDir := t.TempDir()\n\n\tf := NewVFSFile(client, \"test.db\", slog.Default())\n\tf.hydrationPath = filepath.Join(hydrationDir, \"test.db.hydration.db\")\n\tf.PollInterval = 50 * time.Millisecond\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file: %v\", err)\n\t}\n\tdefer f.Close()\n\n\t// Wait for hydration to complete\n\tdeadline := time.Now().Add(5 * time.Second)\n\tfor f.hydrator == nil || !f.hydrator.Complete() {\n\t\tif time.Now().After(deadline) {\n\t\t\tt.Fatalf(\"hydration did not complete in time\")\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\n\t// Add a new LTX file\n\tclient.addFixture(t, buildLTXFixture(t, 2, 'f'))\n\n\t// Wait for poll to pick up the update\n\ttime.Sleep(200 * time.Millisecond)\n\n\t// Read the page - should have updated data\n\tbuf := make([]byte, 4096)\n\tif _, err := f.ReadAt(buf, 0); err != nil {\n\t\tt.Fatalf(\"read at: %v\", err)\n\t}\n\n\t// Data should be updated (excluding header bytes)\n\tfor i := 28; i < len(buf); i++ {\n\t\tif buf[i] != 'f' {\n\t\t\tt.Fatalf(\"expected byte 'f' at position %d, got %q\", i, buf[i])\n\t\t}\n\t}\n}\n\nfunc TestHydrator_Close_Persistent(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"hydration.db\")\n\tclient := newMockReplicaClient()\n\n\th := NewHydrator(path, true, 4096, client, slog.Default())\n\tif err := h.Init(); err != nil {\n\t\tt.Fatalf(\"init: %v\", err)\n\t}\n\n\th.SetTXID(5)\n\n\tif err := h.Close(); err != nil {\n\t\tt.Fatalf(\"close: %v\", err)\n\t}\n\n\tif _, err := os.Stat(path); err != nil {\n\t\tt.Fatalf(\"hydration file should be preserved: %v\", err)\n\t}\n\n\tmetaPath := path + \".meta\"\n\tdata, err := os.ReadFile(metaPath)\n\tif err != nil {\n\t\tt.Fatalf(\"meta file should exist: %v\", err)\n\t}\n\tif got := strings.TrimSpace(string(data)); got != \"5\" {\n\t\tt.Fatalf(\"expected meta TXID=5, got %q\", got)\n\t}\n}\n\nfunc TestHydrator_Init_Resume(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"hydration.db\")\n\tclient := newMockReplicaClient()\n\n\tif err := os.WriteFile(path, []byte(\"existing-db-data\"), 0600); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := os.WriteFile(path+\".meta\", []byte(\"42\\n\"), 0600); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\th := NewHydrator(path, true, 4096, client, slog.Default())\n\tif err := h.Init(); err != nil {\n\t\tt.Fatalf(\"init: %v\", err)\n\t}\n\tdefer h.Close()\n\n\tif got := h.TXID(); got != 42 {\n\t\tt.Fatalf(\"expected TXID=42, got %d\", got)\n\t}\n\n\tdata := make([]byte, 16)\n\tn, err := h.file.ReadAt(data, 0)\n\tif err != nil && err != io.EOF {\n\t\tt.Fatalf(\"read: %v\", err)\n\t}\n\tif string(data[:n]) != \"existing-db-data\" {\n\t\tt.Fatalf(\"file contents should be preserved, got %q\", string(data[:n]))\n\t}\n}\n\nfunc TestHydrator_Close_TempFile(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"hydration.db\")\n\tclient := newMockReplicaClient()\n\n\th := NewHydrator(path, false, 4096, client, slog.Default())\n\tif err := h.Init(); err != nil {\n\t\tt.Fatalf(\"init: %v\", err)\n\t}\n\n\th.SetTXID(10)\n\n\tif err := h.Close(); err != nil {\n\t\tt.Fatalf(\"close: %v\", err)\n\t}\n\n\tif _, err := os.Stat(path); !os.IsNotExist(err) {\n\t\tt.Fatalf(\"temp hydration file should be deleted\")\n\t}\n\tif _, err := os.Stat(path + \".meta\"); !os.IsNotExist(err) {\n\t\tt.Fatalf(\"meta file should not exist for temp hydrator\")\n\t}\n}\n\nfunc TestHydrator_Init_StaleMeta(t *testing.T) {\n\tdir := t.TempDir()\n\tpath := filepath.Join(dir, \"hydration.db\")\n\tclient := newMockReplicaClient()\n\n\tif err := os.WriteFile(path+\".meta\", []byte(\"99\\n\"), 0600); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\th := NewHydrator(path, true, 4096, client, slog.Default())\n\tif err := h.Init(); err != nil {\n\t\tt.Fatalf(\"init: %v\", err)\n\t}\n\tdefer h.Close()\n\n\tif got := h.TXID(); got != 0 {\n\t\tt.Fatalf(\"expected TXID=0 for stale meta, got %d\", got)\n\t}\n\n\tif _, err := os.Stat(path + \".meta\"); !os.IsNotExist(err) {\n\t\tt.Fatalf(\"stale meta file should be removed\")\n\t}\n}\n\nfunc TestVFSFile_Hydration_PersistentResumeOnReopen(t *testing.T) {\n\tclient := newMockReplicaClient()\n\tclient.addFixture(t, buildLTXFixture(t, 1, 'g'))\n\n\thydrationDir := t.TempDir()\n\thydrationPath := filepath.Join(hydrationDir, \"persistent-hydration.db\")\n\n\tf := NewVFSFile(client, \"test.db\", slog.Default())\n\tf.hydrationPath = hydrationPath\n\tf.hydrationPersistent = true\n\tf.PollInterval = 50 * time.Millisecond\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file (first): %v\", err)\n\t}\n\n\tdeadline := time.Now().Add(5 * time.Second)\n\tfor f.hydrator == nil || !f.hydrator.Complete() {\n\t\tif time.Now().After(deadline) {\n\t\t\tt.Fatalf(\"first hydration did not complete in time\")\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\n\tif err := f.Close(); err != nil {\n\t\tt.Fatalf(\"close vfs file (first): %v\", err)\n\t}\n\n\tif _, err := os.Stat(hydrationPath); err != nil {\n\t\tt.Fatalf(\"persistent hydration file should exist after close: %v\", err)\n\t}\n\tif _, err := os.Stat(hydrationPath + \".meta\"); err != nil {\n\t\tt.Fatalf(\"persistent hydration meta should exist after close: %v\", err)\n\t}\n\tinitialInfo, err := os.Stat(hydrationPath)\n\tif err != nil {\n\t\tt.Fatalf(\"stat hydration file after first close: %v\", err)\n\t}\n\n\tf2 := NewVFSFile(client, \"test.db\", slog.Default())\n\tf2.hydrationPath = hydrationPath\n\tf2.hydrationPersistent = true\n\tf2.PollInterval = 50 * time.Millisecond\n\n\tif err := f2.Open(); err != nil {\n\t\tt.Fatalf(\"open vfs file (second): %v\", err)\n\t}\n\n\tdeadline = time.Now().Add(5 * time.Second)\n\tfor f2.hydrator == nil || !f2.hydrator.Complete() {\n\t\tif time.Now().After(deadline) {\n\t\t\tt.Fatalf(\"second hydration did not complete in time\")\n\t\t}\n\t\ttime.Sleep(10 * time.Millisecond)\n\t}\n\n\tif got := f2.hydrator.TXID(); got != 1 {\n\t\tt.Fatalf(\"expected resumed hydration txid=1, got %d\", got)\n\t}\n\tif err := f2.Close(); err != nil {\n\t\tt.Fatalf(\"close vfs file (second): %v\", err)\n\t}\n\n\treopenedInfo, err := os.Stat(hydrationPath)\n\tif err != nil {\n\t\tt.Fatalf(\"stat hydration file after second close: %v\", err)\n\t}\n\tif !reopenedInfo.ModTime().Equal(initialInfo.ModTime()) {\n\t\tt.Fatalf(\"expected hydration file modtime unchanged on reopen resume\")\n\t}\n}\n" }, { "path": "vfs_write_test.go", "content": "//go:build vfs\n// +build vfs\n\npackage litestream\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/psanford/sqlite3vfs\"\n\t\"github.com/superfly/ltx\"\n)\n\n// writeTestReplicaClient is a mock ReplicaClient for testing write functionality.\ntype writeTestReplicaClient struct {\n\tmu sync.Mutex\n\tltxFiles map[int][]*ltx.FileInfo // level -> files\n\tltxData map[string][]byte // \"level/minTXID-maxTXID\" -> data\n}\n\nfunc newWriteTestReplicaClient() *writeTestReplicaClient {\n\treturn &writeTestReplicaClient{\n\t\tltxFiles: make(map[int][]*ltx.FileInfo),\n\t\tltxData: make(map[string][]byte),\n\t}\n}\n\nfunc (c *writeTestReplicaClient) Type() string { return \"test\" }\n\nfunc (c *writeTestReplicaClient) Init(ctx context.Context) error { return nil }\n\nfunc (c *writeTestReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tvar files []*ltx.FileInfo\n\tfor _, f := range c.ltxFiles[level] {\n\t\tif f.MinTXID >= seek {\n\t\t\tfiles = append(files, f)\n\t\t}\n\t}\n\treturn &writeTestFileIterator{files: files}, nil\n}\n\nfunc (c *writeTestReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (io.ReadCloser, error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tkey := ltxKey(level, minTXID, maxTXID)\n\tdata, ok := c.ltxData[key]\n\tif !ok {\n\t\treturn nil, io.EOF\n\t}\n\n\tif offset > 0 || size > 0 {\n\t\tend := int64(len(data))\n\t\tif size > 0 && offset+size < end {\n\t\t\tend = offset + size\n\t\t}\n\t\tdata = data[offset:end]\n\t}\n\n\treturn io.NopCloser(bytes.NewReader(data)), nil\n}\n\nfunc (c *writeTestReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\tdata, err := io.ReadAll(r)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tkey := ltxKey(level, minTXID, maxTXID)\n\tc.ltxData[key] = data\n\n\tinfo := <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tCreatedAt: time.Now(),\n\t\tSize: int64(len(data)),\n\t}\n\tc.ltxFiles[level] = append(c.ltxFiles[level], info)\n\n\treturn info, nil\n}\n\nfunc (c *writeTestReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\treturn nil\n}\n\nfunc (c *writeTestReplicaClient) DeleteAll(ctx context.Context) error {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\tc.ltxFiles = make(map[int][]*ltx.FileInfo)\n\tc.ltxData = make(map[string][]byte)\n\treturn nil\n}\n\nfunc ltxKey(level int, minTXID, maxTXID ltx.TXID) string {\n\treturn string(rune(level)) + \"/\" + minTXID.String() + \"-\" + maxTXID.String()\n}\n\n// writeTestFileIterator implements ltx.FileIterator for testing.\ntype writeTestFileIterator struct {\n\tfiles []*ltx.FileInfo\n\tindex int\n}\n\nfunc (itr *writeTestFileIterator) Next() bool {\n\tif itr.index >= len(itr.files) {\n\t\treturn false\n\t}\n\titr.index++\n\treturn true\n}\n\nfunc (itr *writeTestFileIterator) Item() *ltx.FileInfo {\n\tif itr.index == 0 || itr.index > len(itr.files) {\n\t\treturn nil\n\t}\n\treturn itr.files[itr.index-1]\n}\n\nfunc (itr *writeTestFileIterator) Close() error {\n\treturn nil\n}\n\nfunc (itr *writeTestFileIterator) Err() error {\n\treturn nil\n}\n\n// createTestLTXFile creates an LTX file with initial data for testing.\nfunc createTestLTXFile(t *testing.T, client *writeTestReplicaClient, txid ltx.TXID, pageSize uint32, commit uint32, pages map[uint32][]byte) {\n\tt.Helper()\n\n\tvar buf bytes.Buffer\n\tenc, err := ltx.NewEncoder(&buf)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif err := enc.EncodeHeader(ltx.Header{\n\t\tVersion: ltx.Version,\n\t\tFlags: ltx.HeaderFlagNoChecksum,\n\t\tPageSize: pageSize,\n\t\tCommit: commit,\n\t\tMinTXID: txid,\n\t\tMaxTXID: txid,\n\t\tTimestamp: time.Now().UnixMilli(),\n\t}); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sort page numbers to ensure proper encoding order (page 1 must be first for snapshots)\n\tpgnos := make([]uint32, 0, len(pages))\n\tfor pgno := range pages {\n\t\tpgnos = append(pgnos, pgno)\n\t}\n\tsort.Slice(pgnos, func(i, j int) bool { return pgnos[i] < pgnos[j] })\n\n\tfor _, pgno := range pgnos {\n\t\tif err := enc.EncodePage(ltx.PageHeader{Pgno: pgno}, pages[pgno]); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t}\n\n\tif err := enc.Close(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tclient.mu.Lock()\n\tkey := ltxKey(0, txid, txid)\n\tclient.ltxData[key] = buf.Bytes()\n\tclient.ltxFiles[0] = append(client.ltxFiles[0], <x.FileInfo{\n\t\tLevel: 0,\n\t\tMinTXID: txid,\n\t\tMaxTXID: txid,\n\t\tCreatedAt: time.Now(),\n\t\tSize: int64(buf.Len()),\n\t})\n\tclient.mu.Unlock()\n}\n\n// setupWriteableVFSFile creates a VFSFile with write support enabled and a buffer file.\nfunc setupWriteableVFSFile(t *testing.T, client *writeTestReplicaClient) *VFSFile {\n\tt.Helper()\n\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"test.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\n\t// Create a temporary buffer file\n\ttmpFile, err := os.CreateTemp(\"\", \"litestream-test-buffer-*\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tf.bufferFile = tmpFile\n\tf.bufferPath = tmpFile.Name()\n\tf.bufferNextOff = 0\n\n\tt.Cleanup(func() {\n\t\tif f.bufferFile != nil {\n\t\t\tf.bufferFile.Close()\n\t\t}\n\t\tos.Remove(f.bufferPath)\n\t})\n\n\treturn f\n}\n\nfunc TestVFSFile_WriteEnabled(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\t// Create initial LTX file with page 1\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcopy(initialPage, \"initial data\")\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Create VFSFile directly with write enabled\n\ttmpDir := t.TempDir()\n\tbufferPath := tmpDir + \"/write-buffer\"\n\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"test.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\tf.bufferPath = bufferPath\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\tif !f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be true\")\n\t}\n\n\tif f.dirty == nil {\n\t\tt.Error(\"expected dirty map to be initialized\")\n\t}\n}\n\nfunc TestVFSFile_WriteAt(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\t// Create initial LTX file\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcopy(initialPage, \"initial data\")\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Create VFSFile with write support\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Write some data at offset 100 (within page 1)\n\twriteData := []byte(\"hello world\")\n\tn, err := f.WriteAt(writeData, 100)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif n != len(writeData) {\n\t\tt.Errorf(\"expected %d bytes written, got %d\", len(writeData), n)\n\t}\n\n\t// Check dirty page exists\n\tif len(f.dirty) != 1 {\n\t\tt.Errorf(\"expected 1 dirty page, got %d\", len(f.dirty))\n\t}\n\tif _, ok := f.dirty[1]; !ok {\n\t\tt.Error(\"expected page 1 to be dirty\")\n\t}\n\n\t// Read back the written data\n\treadBuf := make([]byte, len(writeData))\n\tn, err = f.ReadAt(readBuf, 100)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif string(readBuf) != string(writeData) {\n\t\tt.Errorf(\"expected %q, got %q\", writeData, readBuf)\n\t}\n}\n\nfunc TestVFSFile_SyncToRemote(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\t// Create initial LTX file\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Create VFSFile with write support\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Write data\n\twriteData := []byte(\"synced data\")\n\tif _, err := f.WriteAt(writeData, 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sync to remote\n\tif err := f.Sync(0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Check dirty pages are cleared\n\tif len(f.dirty) != 0 {\n\t\tt.Errorf(\"expected 0 dirty pages after sync, got %d\", len(f.dirty))\n\t}\n\n\t// Check TXID advanced\n\tif f.expectedTXID != 2 {\n\t\tt.Errorf(\"expected TXID 2, got %d\", f.expectedTXID)\n\t}\n\n\t// Check LTX file was written to client\n\tclient.mu.Lock()\n\tif len(client.ltxFiles[0]) != 2 {\n\t\tt.Errorf(\"expected 2 LTX files, got %d\", len(client.ltxFiles[0]))\n\t}\n\tclient.mu.Unlock()\n}\n\nfunc TestVFSFile_ConflictDetection(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\t// Create initial LTX file\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Create VFSFile with write support\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Write data\n\tif _, err := f.WriteAt([]byte(\"data\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Simulate remote advancement (another writer)\n\tcreateTestLTXFile(t, client, 2, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Try to sync - should fail with conflict\n\terr := f.Sync(0)\n\tif err == nil {\n\t\tt.Fatal(\"expected conflict error\")\n\t}\n\tif err.Error() != \"remote has newer transactions than expected: expected TXID 1 but remote has 2\" {\n\t\tt.Errorf(\"unexpected error: %v\", err)\n\t}\n}\n\nfunc TestVFSFile_TransactionTracking(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\t// Create initial LTX file\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Create VFSFile with write support\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Acquire RESERVED lock (start transaction)\n\tif err := f.Lock(2); err != nil { // sqlite3vfs.LockReserved = 2\n\t\tt.Fatal(err)\n\t}\n\n\tif !f.inTransaction {\n\t\tt.Error(\"expected inTransaction to be true after RESERVED lock\")\n\t}\n\n\t// Write data\n\tif _, err := f.WriteAt([]byte(\"tx data\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Sync should be skipped during transaction\n\tif err := f.Sync(0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(f.dirty) == 0 {\n\t\tt.Error(\"expected dirty pages to remain during transaction\")\n\t}\n\n\t// Release lock (end transaction)\n\tif err := f.Unlock(1); err != nil { // sqlite3vfs.LockShared = 1\n\t\tt.Fatal(err)\n\t}\n\n\tif f.inTransaction {\n\t\tt.Error(\"expected inTransaction to be false after unlock\")\n\t}\n\n\t// Now sync should work\n\tif err := f.Sync(0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif len(f.dirty) != 0 {\n\t\tt.Error(\"expected dirty pages to be cleared after sync\")\n\t}\n}\n\nfunc TestVFSFile_Truncate(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\t// Create initial LTX files with 2 pages\n\tpageSize := uint32(4096)\n\tpage1 := make([]byte, pageSize)\n\tpage2 := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 2, map[uint32][]byte{1: page1, 2: page2})\n\n\t// Create VFSFile with write support\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Write to page 2\n\tif _, err := f.WriteAt([]byte(\"page2 data\"), int64(pageSize)); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Truncate to 1 page\n\tif err := f.Truncate(int64(pageSize)); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Page 2 should no longer be dirty\n\tif _, ok := f.dirty[2]; ok {\n\t\tt.Error(\"expected page 2 to be removed from dirty pages\")\n\t}\n\n\t// Commit should be 1\n\tif f.commit != 1 {\n\t\tt.Errorf(\"expected commit 1, got %d\", f.commit)\n\t}\n}\n\nfunc TestVFSFile_WriteBuffer(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\t// Create initial LTX file\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcopy(initialPage, \"initial data\")\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Create temp directory for buffer\n\ttmpDir := t.TempDir()\n\tbufferPath := tmpDir + \"/.litestream-write-buffer\"\n\n\t// Create VFSFile with write buffer\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"test.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\tf.bufferPath = bufferPath\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Write some data\n\twriteData := []byte(\"buffered data\")\n\tif _, err := f.WriteAt(writeData, 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Check buffer file exists and has content\n\tstat, err := os.Stat(bufferPath)\n\tif err != nil {\n\t\tt.Fatalf(\"buffer file should exist: %v\", err)\n\t}\n\tif stat.Size() == 0 {\n\t\tt.Error(\"buffer file should not be empty\")\n\t}\n\n\t// Don't call f.Close() - simulate a crash by just abandoning the file handle\n\t// Close just the buffer file directly to release the handle\n\tif f.bufferFile != nil {\n\t\tf.bufferFile.Close()\n\t}\n\tf.cancel() // Stop any goroutines\n\n\t// Verify buffer file still has content (simulating crash before sync)\n\tstat, err = os.Stat(bufferPath)\n\tif err != nil {\n\t\tt.Fatalf(\"buffer file should still exist after crash: %v\", err)\n\t}\n\tif stat.Size() == 0 {\n\t\tt.Error(\"buffer file should still have content after crash\")\n\t}\n}\n\nfunc TestVFSFile_WriteBufferDiscardedOnOpen(t *testing.T) {\n\t// Test that unsync'd buffer contents are discarded on open (no recovery)\n\tclient := newWriteTestReplicaClient()\n\n\t// Create initial LTX file\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcopy(initialPage, \"initial data\")\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Create temp directory for buffer\n\ttmpDir := t.TempDir()\n\tbufferPath := tmpDir + \"/.litestream-write-buffer\"\n\n\t// First: create a VFSFile and write some data\n\tlogger := slog.Default()\n\tf1 := NewVFSFile(client, \"test.db\", logger)\n\tf1.writeEnabled = true\n\tf1.dirty = make(map[uint32]int64)\n\tf1.syncInterval = 0\n\tf1.bufferPath = bufferPath\n\n\tif err := f1.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Write data (will be written to buffer)\n\twriteData := make([]byte, pageSize)\n\tcopy(writeData, \"unsync'd data that should be lost\")\n\tif _, err := f1.WriteAt(writeData, 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Simulate crash by abandoning the file handle without syncing\n\tif f1.bufferFile != nil {\n\t\tf1.bufferFile.Close()\n\t}\n\tf1.cancel()\n\n\t// Second: create a new VFSFile - buffer should be discarded\n\tf2 := NewVFSFile(client, \"test.db\", logger)\n\tf2.writeEnabled = true\n\tf2.dirty = make(map[uint32]int64)\n\tf2.syncInterval = 0\n\tf2.bufferPath = bufferPath\n\n\tif err := f2.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f2.Close()\n\n\t// Dirty pages should NOT be recovered - buffer is discarded on open\n\tif len(f2.dirty) != 0 {\n\t\tt.Errorf(\"expected 0 dirty pages (buffer should be discarded), got %d\", len(f2.dirty))\n\t}\n\n\t// Reading should return original data from replica, not unsync'd data\n\treadBuf := make([]byte, pageSize)\n\tif _, err := f2.ReadAt(readBuf, 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif string(readBuf[:12]) != \"initial data\" {\n\t\tt.Errorf(\"expected 'initial data' (from replica), got %q\", string(readBuf[:12]))\n\t}\n}\n\nfunc TestVFSFile_WriteBufferClearAfterSync(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\t// Create initial LTX file\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Create temp directory for buffer\n\ttmpDir := t.TempDir()\n\tbufferPath := tmpDir + \"/.litestream-write-buffer\"\n\n\t// Create VFSFile with write buffer\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"test.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\tf.bufferPath = bufferPath\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Write data\n\tif _, err := f.WriteAt([]byte(\"sync test\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Check buffer has content before sync\n\tstat, _ := os.Stat(bufferPath)\n\tif stat.Size() == 0 {\n\t\tt.Error(\"buffer should have content before sync\")\n\t}\n\n\t// Sync to remote\n\tif err := f.Sync(0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Check buffer is cleared after sync\n\tstat, _ = os.Stat(bufferPath)\n\tif stat.Size() != 0 {\n\t\tt.Errorf(\"buffer should be empty after sync, got size %d\", stat.Size())\n\t}\n}\n\nfunc TestVFSFile_OpenFailsWithInvalidBufferPath(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"test.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\tf.bufferPath = \"/nonexistent/path/that/cannot/be/created/buffer\"\n\n\terr := f.Open()\n\tif err == nil {\n\t\tf.Close()\n\t\tt.Fatal(\"expected Open to fail with invalid buffer path\")\n\t}\n}\n\nfunc TestVFSFile_BufferFileAlwaysCreatedWhenWriteEnabled(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\ttmpDir := t.TempDir()\n\tbufferPath := tmpDir + \"/write-buffer\"\n\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"test.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\tf.bufferPath = bufferPath\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\tif f.bufferFile == nil {\n\t\tt.Fatal(\"bufferFile should never be nil when writeEnabled is true\")\n\t}\n}\n\nfunc TestVFSFile_OpenNewDatabase(t *testing.T) {\n\t// Test opening a VFSFile with write mode enabled when no LTX files exist (new database)\n\tclient := newWriteTestReplicaClient()\n\t// Note: No LTX files created - simulating a brand new database\n\n\t// Create temp directory for buffer\n\ttmpDir := t.TempDir()\n\tbufferPath := tmpDir + \"/.litestream-write-buffer\"\n\n\t// Create VFSFile with write support - no existing data\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"new.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\tf.bufferPath = bufferPath\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Verify it opened successfully as a new database\n\tif f.pageSize != DefaultPageSize {\n\t\tt.Errorf(\"expected page size %d, got %d\", DefaultPageSize, f.pageSize)\n\t}\n\n\tif f.pos.TXID != 0 {\n\t\tt.Errorf(\"expected TXID 0 for new database, got %d\", f.pos.TXID)\n\t}\n\n\tif f.expectedTXID != 0 {\n\t\tt.Errorf(\"expected expectedTXID 0, got %d\", f.expectedTXID)\n\t}\n\n\tif f.pendingTXID != 1 {\n\t\tt.Errorf(\"expected pendingTXID 1, got %d\", f.pendingTXID)\n\t}\n\n\tif f.commit != 0 {\n\t\tt.Errorf(\"expected commit 0 for new database, got %d\", f.commit)\n\t}\n}\n\nfunc TestVFSFile_NewDatabase_ReadReturnsZeros(t *testing.T) {\n\t// Test that reading from a new database returns zeros\n\tclient := newWriteTestReplicaClient()\n\n\ttmpDir := t.TempDir()\n\tbufferPath := tmpDir + \"/.litestream-write-buffer\"\n\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"new.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\tf.bufferPath = bufferPath\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Read page 1 - should return zeros for new database\n\treadBuf := make([]byte, 100)\n\tn, err := f.ReadAt(readBuf, 0)\n\tif err != nil {\n\t\tt.Fatalf(\"expected no error reading from new database, got: %v\", err)\n\t}\n\tif n != len(readBuf) {\n\t\tt.Errorf(\"expected %d bytes, got %d\", len(readBuf), n)\n\t}\n\n\t// Verify all zeros\n\tfor i, b := range readBuf {\n\t\tif b != 0 {\n\t\t\tt.Errorf(\"expected zero at position %d, got %d\", i, b)\n\t\t\tbreak\n\t\t}\n\t}\n}\n\nfunc TestVFSFile_NewDatabase_WriteAndSync(t *testing.T) {\n\t// Test writing to a new database and syncing to remote\n\tclient := newWriteTestReplicaClient()\n\n\ttmpDir := t.TempDir()\n\tbufferPath := tmpDir + \"/.litestream-write-buffer\"\n\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"new.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\tf.bufferPath = bufferPath\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Write data to page 1\n\twriteData := []byte(\"new database content\")\n\tn, err := f.WriteAt(writeData, 0)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif n != len(writeData) {\n\t\tt.Errorf(\"expected %d bytes written, got %d\", len(writeData), n)\n\t}\n\n\t// Verify dirty page exists\n\tif len(f.dirty) != 1 {\n\t\tt.Errorf(\"expected 1 dirty page, got %d\", len(f.dirty))\n\t}\n\n\t// Sync to remote\n\tif err := f.Sync(0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Verify TXID advanced\n\tif f.expectedTXID != 1 {\n\t\tt.Errorf(\"expected expectedTXID 1 after sync, got %d\", f.expectedTXID)\n\t}\n\tif f.pendingTXID != 2 {\n\t\tt.Errorf(\"expected pendingTXID 2 after sync, got %d\", f.pendingTXID)\n\t}\n\n\t// Verify LTX file was written\n\tclient.mu.Lock()\n\tif len(client.ltxFiles[0]) != 1 {\n\t\tt.Errorf(\"expected 1 LTX file after sync, got %d\", len(client.ltxFiles[0]))\n\t}\n\tif len(client.ltxFiles[0]) > 0 {\n\t\tinfo := client.ltxFiles[0][0]\n\t\tif info.MinTXID != 1 || info.MaxTXID != 1 {\n\t\t\tt.Errorf(\"expected TXID 1, got min=%d max=%d\", info.MinTXID, info.MaxTXID)\n\t\t}\n\t}\n\tclient.mu.Unlock()\n}\n\nfunc TestVFSFile_NewDatabase_FileSize(t *testing.T) {\n\t// Test that FileSize returns 0 for a new empty database\n\tclient := newWriteTestReplicaClient()\n\n\ttmpDir := t.TempDir()\n\tbufferPath := tmpDir + \"/.litestream-write-buffer\"\n\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"new.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\tf.bufferPath = bufferPath\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// FileSize should be 0 for empty database\n\tsize, err := f.FileSize()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif size != 0 {\n\t\tt.Errorf(\"expected size 0 for new database, got %d\", size)\n\t}\n\n\t// Write a page\n\tdata := make([]byte, DefaultPageSize)\n\tif _, err := f.WriteAt(data, 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// FileSize should now reflect the dirty page\n\tsize, err = f.FileSize()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif size != int64(DefaultPageSize) {\n\t\tt.Errorf(\"expected size %d after write, got %d\", DefaultPageSize, size)\n\t}\n}\n\nfunc TestSetWriteEnabled_ReadValue(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Test with write disabled\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"test.db\", logger)\n\tf.writeEnabled = false\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Read via FileControl (simulates PRAGMA litestream_write_enabled)\n\tresult, err := f.FileControl(14, \"litestream_write_enabled\", nil)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif result == nil || *result != \"0\" {\n\t\tt.Errorf(\"expected '0' for disabled write support, got %v\", result)\n\t}\n}\n\nfunc TestSetWriteEnabled_ReadValueEnabled(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Test with write enabled\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Read via FileControl\n\tresult, err := f.FileControl(14, \"litestream_write_enabled\", nil)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif result == nil || *result != \"1\" {\n\t\tt.Errorf(\"expected '1' for enabled write support, got %v\", result)\n\t}\n}\n\nfunc TestSetWriteEnabled_DisableSyncsDirtyPages(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Write data to create dirty pages\n\twriteData := []byte(\"dirty data\")\n\tif _, err := f.WriteAt(writeData, 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(f.dirty) == 0 {\n\t\tt.Fatal(\"expected dirty pages\")\n\t}\n\n\t// Disable writes via SetWriteEnabled\n\tif err := f.SetWriteEnabled(false); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Dirty pages should be synced\n\tif len(f.dirty) != 0 {\n\t\tt.Errorf(\"expected 0 dirty pages after disable, got %d\", len(f.dirty))\n\t}\n\n\t// Write support should be disabled\n\tif f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be false\")\n\t}\n\n\t// LTX file should have been written\n\tclient.mu.Lock()\n\tif len(client.ltxFiles[0]) != 2 {\n\t\tt.Errorf(\"expected 2 LTX files (initial + synced), got %d\", len(client.ltxFiles[0]))\n\t}\n\tclient.mu.Unlock()\n}\n\nfunc TestSetWriteEnabled_DisableWaitsForTransaction(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Start a transaction (acquire RESERVED lock)\n\tif err := f.Lock(2); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Write some data\n\tif _, err := f.WriteAt([]byte(\"tx data\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Start disable in a goroutine (it should wait for transaction)\n\tdone := make(chan error, 1)\n\tgo func() {\n\t\tdone <- f.SetWriteEnabled(false)\n\t}()\n\n\t// Wait for SetWriteEnabled to set the disabling flag\n\tdeadline := time.Now().Add(2 * time.Second)\n\tfor {\n\t\tf.mu.Lock()\n\t\tdisabling := f.disabling\n\t\tf.mu.Unlock()\n\t\tif disabling {\n\t\t\tbreak\n\t\t}\n\t\tif time.Now().After(deadline) {\n\t\t\tt.Fatal(\"timed out waiting for disabling flag\")\n\t\t}\n\t\ttime.Sleep(1 * time.Millisecond)\n\t}\n\n\t// Write should still be enabled (waiting for transaction)\n\tf.mu.Lock()\n\tstillEnabled := f.writeEnabled\n\tf.mu.Unlock()\n\tif !stillEnabled {\n\t\tt.Error(\"expected writeEnabled to still be true while in transaction\")\n\t}\n\n\t// End transaction (release lock)\n\tif err := f.Unlock(1); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for disable to complete\n\tselect {\n\tcase err := <-done:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"SetWriteEnabled failed: %v\", err)\n\t\t}\n\tcase <-time.After(2 * time.Second):\n\t\tt.Fatal(\"SetWriteEnabled timed out\")\n\t}\n\n\t// Write should now be disabled\n\tf.mu.Lock()\n\tenabled := f.writeEnabled\n\tf.mu.Unlock()\n\tif enabled {\n\t\tt.Error(\"expected writeEnabled to be false after transaction ended\")\n\t}\n}\n\nfunc TestSetWriteEnabled_EnableAfterDisable(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Disable writes\n\tif err := f.SetWriteEnabled(false); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be false\")\n\t}\n\n\t// Re-enable writes\n\tif err := f.SetWriteEnabled(true); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be true\")\n\t}\n\n\t// Verify we can write again\n\twriteData := []byte(\"after re-enable\")\n\tif _, err := f.WriteAt(writeData, 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(f.dirty) == 0 {\n\t\tt.Error(\"expected dirty pages after write\")\n\t}\n}\n\nfunc TestSetWriteEnabled_DisableWithTimeout(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Start a transaction (acquire RESERVED lock)\n\tif err := f.Lock(2); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Write some data\n\tif _, err := f.WriteAt([]byte(\"tx data\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Try to disable with a short timeout - should fail\n\terr := f.SetWriteEnabledWithTimeout(false, 50*time.Millisecond)\n\tif err == nil {\n\t\tt.Fatal(\"expected timeout error\")\n\t}\n\tif !strings.Contains(err.Error(), \"timeout waiting for transaction\") {\n\t\tt.Errorf(\"unexpected error: %v\", err)\n\t}\n\n\t// Write should still be enabled\n\tif !f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to still be true after timeout\")\n\t}\n\n\t// End transaction\n\tif err := f.Unlock(1); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Now disable should succeed (with or without timeout)\n\tif err := f.SetWriteEnabledWithTimeout(false, 1*time.Second); err != nil {\n\t\tt.Fatalf(\"SetWriteEnabledWithTimeout failed: %v\", err)\n\t}\n\n\tif f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be false\")\n\t}\n}\n\nfunc TestSetWriteEnabled_ColdEnable(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Create VFSFile WITHOUT write enabled initially\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"test.db\", logger)\n\tf.writeEnabled = false\n\t// Note: dirty, bufferPath, etc. are NOT set - simulating cold start\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Verify writes are disabled\n\tif f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be false initially\")\n\t}\n\n\t// Enable writes via SetWriteEnabled (cold enable)\n\tif err := f.SetWriteEnabled(true); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Verify writes are now enabled\n\tif !f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be true after cold enable\")\n\t}\n\n\t// Verify buffer was initialized\n\tif f.bufferFile == nil {\n\t\tt.Error(\"expected bufferFile to be initialized\")\n\t}\n\n\t// Verify dirty map was initialized\n\tif f.dirty == nil {\n\t\tt.Error(\"expected dirty map to be initialized\")\n\t}\n\n\t// Verify TXID state was initialized\n\tif f.pendingTXID == 0 {\n\t\tt.Error(\"expected pendingTXID to be initialized\")\n\t}\n\n\t// Verify we can write\n\twriteData := []byte(\"cold enable test\")\n\tif _, err := f.WriteAt(writeData, 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(f.dirty) == 0 {\n\t\tt.Error(\"expected dirty pages after write\")\n\t}\n}\n\nfunc TestSetWriteEnabled_NoOpWhenAlreadyInState(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Enable when already enabled should be no-op\n\tif err := f.SetWriteEnabled(true); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to remain true\")\n\t}\n\n\t// Disable\n\tif err := f.SetWriteEnabled(false); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Disable when already disabled should be no-op\n\tif err := f.SetWriteEnabled(false); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to remain false\")\n\t}\n}\n\nfunc TestSetWriteEnabled_FileControlWrite(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Disable via FileControl (PRAGMA litestream_write_enabled = 0)\n\tvalue := \"0\"\n\t_, err := f.FileControl(14, \"litestream_write_enabled\", &value)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be false after PRAGMA = 0\")\n\t}\n\n\t// Enable via FileControl (PRAGMA litestream_write_enabled = 1)\n\tvalue = \"1\"\n\t_, err = f.FileControl(14, \"litestream_write_enabled\", &value)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif !f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be true after PRAGMA = 1\")\n\t}\n\n\t// Test alternate values\n\tvalue = \"off\"\n\t_, err = f.FileControl(14, \"litestream_write_enabled\", &value)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be false after PRAGMA = off\")\n\t}\n\n\tvalue = \"on\"\n\t_, err = f.FileControl(14, \"litestream_write_enabled\", &value)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif !f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be true after PRAGMA = on\")\n\t}\n\n\tvalue = \"false\"\n\t_, err = f.FileControl(14, \"litestream_write_enabled\", &value)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be false after PRAGMA = false\")\n\t}\n\n\tvalue = \"true\"\n\t_, err = f.FileControl(14, \"litestream_write_enabled\", &value)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif !f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to be true after PRAGMA = true\")\n\t}\n}\n\nfunc TestSetWriteEnabled_InvalidValue(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Invalid value should return error\n\tvalue := \"invalid\"\n\t_, err := f.FileControl(14, \"litestream_write_enabled\", &value)\n\tif err == nil {\n\t\tt.Error(\"expected error for invalid value\")\n\t}\n\tif err.Error() != \"invalid value for litestream_write_enabled: invalid (use 0 or 1)\" {\n\t\tt.Errorf(\"unexpected error message: %v\", err)\n\t}\n}\n\n// failingWriteClient wraps writeTestReplicaClient to fail writes after a certain count.\ntype failingWriteClient struct {\n\t*writeTestReplicaClient\n\tfailAfter int\n\twriteCount int\n}\n\nfunc newFailingWriteClient(failAfter int) *failingWriteClient {\n\treturn &failingWriteClient{\n\t\twriteTestReplicaClient: newWriteTestReplicaClient(),\n\t\tfailAfter: failAfter,\n\t}\n}\n\nfunc (c *failingWriteClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {\n\tc.mu.Lock()\n\tc.writeCount++\n\tcount := c.writeCount\n\tc.mu.Unlock()\n\n\tif count > c.failAfter {\n\t\treturn nil, errors.New(\"simulated write failure\")\n\t}\n\treturn c.writeTestReplicaClient.WriteLTXFile(ctx, level, minTXID, maxTXID, r)\n}\n\nfunc TestSetWriteEnabled_SyncFailureKeepsWritesEnabled(t *testing.T) {\n\t// Use a client that fails on the second write attempt (first is from setup/initial sync)\n\tclient := newFailingWriteClient(0) // Fail on first write\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client.writeTestReplicaClient, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tlogger := slog.Default()\n\tf := NewVFSFile(client, \"test.db\", logger)\n\tf.writeEnabled = true\n\tf.dirty = make(map[uint32]int64)\n\tf.syncInterval = 0\n\n\t// Create a temporary buffer file\n\ttmpFile, err := os.CreateTemp(\"\", \"litestream-test-buffer-*\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tf.bufferFile = tmpFile\n\tf.bufferPath = tmpFile.Name()\n\tf.bufferNextOff = 0\n\n\tt.Cleanup(func() {\n\t\tif f.bufferFile != nil {\n\t\t\tf.bufferFile.Close()\n\t\t}\n\t\tos.Remove(f.bufferPath)\n\t})\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Write data to create dirty pages\n\twriteData := []byte(\"dirty data\")\n\tif _, err := f.WriteAt(writeData, 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(f.dirty) == 0 {\n\t\tt.Fatal(\"expected dirty pages\")\n\t}\n\n\t// Try to disable writes - should fail because sync fails\n\terr = f.SetWriteEnabled(false)\n\tif err == nil {\n\t\tt.Fatal(\"expected error from sync failure\")\n\t}\n\tif !strings.Contains(err.Error(), \"sync before disable\") {\n\t\tt.Errorf(\"unexpected error: %v\", err)\n\t}\n\n\t// Write support should still be enabled because sync failed\n\tif !f.writeEnabled {\n\t\tt.Error(\"expected writeEnabled to remain true after sync failure\")\n\t}\n\n\t// Dirty pages should still exist\n\tif len(f.dirty) == 0 {\n\t\tt.Error(\"expected dirty pages to remain after sync failure\")\n\t}\n}\n\nfunc TestSetWriteEnabled_DisablingPreventsNewTransactions(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Start a transaction (acquire RESERVED lock)\n\tif err := f.Lock(2); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Start disable in a goroutine\n\tdisableDone := make(chan error, 1)\n\tgo func() {\n\t\tdisableDone <- f.SetWriteEnabledWithTimeout(false, 2*time.Second)\n\t}()\n\n\t// Wait for SetWriteEnabled to set the disabling flag\n\tdeadline := time.Now().Add(2 * time.Second)\n\tfor {\n\t\tf.mu.Lock()\n\t\tdisabling := f.disabling\n\t\tf.mu.Unlock()\n\t\tif disabling {\n\t\t\tbreak\n\t\t}\n\t\tif time.Now().After(deadline) {\n\t\t\tt.Fatal(\"timed out waiting for disabling flag\")\n\t\t}\n\t\ttime.Sleep(1 * time.Millisecond)\n\t}\n\n\t// End the first transaction\n\tif err := f.Unlock(1); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for disable to complete\n\tselect {\n\tcase err := <-disableDone:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"SetWriteEnabled failed: %v\", err)\n\t\t}\n\tcase <-time.After(3 * time.Second):\n\t\tt.Fatal(\"SetWriteEnabled timed out\")\n\t}\n\n\t// Verify disabling flag is cleared\n\tf.mu.Lock()\n\tdisabling := f.disabling\n\tf.mu.Unlock()\n\n\tif disabling {\n\t\tt.Error(\"expected disabling flag to be false after completion\")\n\t}\n\n\t// Verify writes are disabled\n\tf.mu.Lock()\n\tenabled := f.writeEnabled\n\tf.mu.Unlock()\n\tif enabled {\n\t\tt.Error(\"expected writeEnabled to be false\")\n\t}\n}\n\nfunc TestSetWriteEnabled_ConcurrentEnableDisable(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Run multiple concurrent enable/disable operations\n\tvar wg sync.WaitGroup\n\terrCh := make(chan error, 20)\n\n\tfor i := 0; i < 10; i++ {\n\t\twg.Add(2)\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\tif err := f.SetWriteEnabled(true); err != nil {\n\t\t\t\terrCh <- err\n\t\t\t}\n\t\t}()\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\tif err := f.SetWriteEnabled(false); err != nil {\n\t\t\t\terrCh <- err\n\t\t\t}\n\t\t}()\n\t}\n\n\twg.Wait()\n\tclose(errCh)\n\n\t// Check for errors\n\tfor err := range errCh {\n\t\tt.Errorf(\"concurrent operation failed: %v\", err)\n\t}\n\n\t// The final state should be valid (either enabled or disabled)\n\tf.mu.Lock()\n\tenabled := f.writeEnabled\n\tdisabling := f.disabling\n\tf.mu.Unlock()\n\n\t// disabling should always be false when no operation is in progress\n\tif disabling {\n\t\tt.Error(\"expected disabling to be false after all operations complete\")\n\t}\n\n\tt.Logf(\"Final writeEnabled state: %v\", enabled)\n}\n\nfunc TestLock_BlocksDuringDisable(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Start a transaction (acquire RESERVED lock)\n\tif err := f.Lock(2); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Write some data so there's something to sync\n\tif _, err := f.WriteAt([]byte(\"tx data\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Start disable in a goroutine - it will wait for the transaction\n\tdisableDone := make(chan error, 1)\n\tgo func() {\n\t\tdisableDone <- f.SetWriteEnabled(false)\n\t}()\n\n\t// Wait for SetWriteEnabled to set the disabling flag\n\tdeadline := time.Now().Add(2 * time.Second)\n\tfor {\n\t\tf.mu.Lock()\n\t\tdisabling := f.disabling\n\t\tf.mu.Unlock()\n\t\tif disabling {\n\t\t\tbreak\n\t\t}\n\t\tif time.Now().After(deadline) {\n\t\t\tt.Fatal(\"timed out waiting for disabling flag\")\n\t\t}\n\t\ttime.Sleep(1 * time.Millisecond)\n\t}\n\n\t// End transaction to let disable proceed, then immediately try to\n\t// acquire RESERVED lock again - it should block until disable completes\n\tlockErrCh := make(chan error, 1)\n\tlockDone := make(chan struct{})\n\tgo func() {\n\t\tdefer close(lockDone)\n\t\tif err := f.Unlock(1); err != nil {\n\t\t\tlockErrCh <- fmt.Errorf(\"unlock: %w\", err)\n\t\t\treturn\n\t\t}\n\t\t// Lock() should block while disabling is true, then fail because\n\t\t// writeEnabled will be false after disable completes\n\t\tlockErrCh <- f.Lock(2)\n\t}()\n\n\t// Wait for disable to complete\n\tselect {\n\tcase err := <-disableDone:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"SetWriteEnabled failed: %v\", err)\n\t\t}\n\tcase <-time.After(3 * time.Second):\n\t\tt.Fatal(\"SetWriteEnabled timed out\")\n\t}\n\n\tselect {\n\tcase <-lockDone:\n\tcase <-time.After(3 * time.Second):\n\t\tt.Fatal(\"Lock() timed out\")\n\t}\n\n\t// Lock should have returned an error since writes are now disabled\n\tif err := <-lockErrCh; err == nil {\n\t\tt.Error(\"expected Lock(RESERVED) to fail when writes are disabled\")\n\t}\n\n\t// Verify writeEnabled is now false\n\tf.mu.Lock()\n\tenabled := f.writeEnabled\n\tinTx := f.inTransaction\n\tf.mu.Unlock()\n\tif enabled {\n\t\tt.Error(\"expected writeEnabled to be false after disable completed\")\n\t}\n\tif inTx {\n\t\tt.Error(\"expected inTransaction to be false when writeEnabled is false\")\n\t}\n}\n\nfunc TestLock_BlocksDuringDisable_MultipleWaiters(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tf := setupWriteableVFSFile(t, client)\n\n\tif err := f.Open(); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer f.Close()\n\n\t// Start a transaction (acquire RESERVED lock)\n\tif err := f.Lock(2); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Write some data\n\tif _, err := f.WriteAt([]byte(\"tx data\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Start disable in a goroutine\n\tdisableDone := make(chan error, 1)\n\tgo func() {\n\t\tdisableDone <- f.SetWriteEnabled(false)\n\t}()\n\n\t// Wait for SetWriteEnabled to set the disabling flag\n\tdeadline := time.Now().Add(2 * time.Second)\n\tfor {\n\t\tf.mu.Lock()\n\t\tdisabling := f.disabling\n\t\tf.mu.Unlock()\n\t\tif disabling {\n\t\t\tbreak\n\t\t}\n\t\tif time.Now().After(deadline) {\n\t\t\tt.Fatal(\"timed out waiting for disabling flag\")\n\t\t}\n\t\ttime.Sleep(1 * time.Millisecond)\n\t}\n\n\t// Simulate multiple waiters trying to acquire RESERVED lock.\n\t// They will block on cond.Wait() while disabling is true, then\n\t// fail with read-only error once disable completes.\n\tconst numWaiters = 3\n\tvar wg sync.WaitGroup\n\terrCh := make(chan error, numWaiters)\n\tstarted := make(chan struct{}, numWaiters)\n\n\tfor i := 0; i < numWaiters; i++ {\n\t\twg.Add(1)\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\tstarted <- struct{}{}\n\t\t\terrCh <- f.Lock(2)\n\t\t}()\n\t}\n\n\t// Wait for all goroutines to start, then verify none have completed yet\n\t// (they should be blocked in cond.Wait() while disabling is true).\n\tfor i := 0; i < numWaiters; i++ {\n\t\t<-started\n\t}\n\ttime.Sleep(10 * time.Millisecond)\n\tif len(errCh) > 0 {\n\t\tt.Fatal(\"expected all Lock() calls to be blocked during disable, but some completed early\")\n\t}\n\n\t// End the original transaction - this will trigger the disable to complete\n\tif err := f.Unlock(1); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Wait for disable to complete\n\tselect {\n\tcase err := <-disableDone:\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"SetWriteEnabled failed: %v\", err)\n\t\t}\n\tcase <-time.After(3 * time.Second):\n\t\tt.Fatal(\"SetWriteEnabled timed out\")\n\t}\n\n\t// Wait for all Lock() calls to complete\n\twg.Wait()\n\tclose(errCh)\n\n\t// All Lock() calls should have returned errors (writes now disabled)\n\tfor err := range errCh {\n\t\tif err == nil {\n\t\t\tt.Error(\"expected Lock(RESERVED) to fail when writes are disabled\")\n\t\t}\n\t}\n\n\t// Verify writeEnabled is now false\n\tf.mu.Lock()\n\tenabled := f.writeEnabled\n\tf.mu.Unlock()\n\tif enabled {\n\t\tt.Error(\"expected writeEnabled to be false\")\n\t}\n}\n\nfunc openWriteVFSFile(t *testing.T, vfs *VFS) *VFSFile {\n\tt.Helper()\n\tfile, _, err := vfs.openMainDB(\"test.db\", sqlite3vfs.OpenMainDB|sqlite3vfs.OpenReadWrite)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tf := file.(*VFSFile)\n\tt.Cleanup(func() { f.Close() })\n\treturn f\n}\n\nfunc TestVFS_MultipleConnections_NoFalseConflict(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tv := NewVFS(client, slog.Default())\n\tv.WriteEnabled = true\n\tv.WriteSyncInterval = 0\n\n\tf1 := openWriteVFSFile(t, v)\n\tf2 := openWriteVFSFile(t, v)\n\n\t// Connection 1: acquire RESERVED, write, sync\n\tif err := f1.Lock(sqlite3vfs.LockReserved); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif _, err := f1.WriteAt([]byte(\"data1\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := f1.Unlock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := f1.Sync(0); err != nil {\n\t\tt.Fatalf(\"connection 1 sync failed: %v\", err)\n\t}\n\tif f1.expectedTXID != 2 {\n\t\tt.Fatalf(\"expected f1.expectedTXID=2, got %d\", f1.expectedTXID)\n\t}\n\n\t// Connection 2: acquire RESERVED (should refresh TXID), write, sync\n\tif err := f2.Lock(sqlite3vfs.LockReserved); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif f2.expectedTXID != 2 {\n\t\tt.Fatalf(\"expected f2.expectedTXID=2 after RESERVED lock refresh, got %d\", f2.expectedTXID)\n\t}\n\tif _, err := f2.WriteAt([]byte(\"data2\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := f2.Unlock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif err := f2.Sync(0); err != nil {\n\t\tt.Fatalf(\"connection 2 sync failed (false conflict): %v\", err)\n\t}\n\tif f2.expectedTXID != 3 {\n\t\tt.Fatalf(\"expected f2.expectedTXID=3, got %d\", f2.expectedTXID)\n\t}\n}\n\nfunc TestVFS_WriteLockBlocksConcurrentWriters(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tv := NewVFS(client, slog.Default())\n\tv.WriteEnabled = true\n\n\tf1 := openWriteVFSFile(t, v)\n\tf2 := openWriteVFSFile(t, v)\n\n\t// f1 acquires RESERVED\n\tif err := f1.Lock(sqlite3vfs.LockReserved); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// f2 attempts RESERVED - should get BusyError\n\terr := f2.Lock(sqlite3vfs.LockReserved)\n\tif !errors.Is(err, sqlite3vfs.BusyError) {\n\t\tt.Fatalf(\"expected BusyError, got %v\", err)\n\t}\n\n\t// f1 releases\n\tif err := f1.Unlock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// f2 should now succeed\n\tif err := f2.Lock(sqlite3vfs.LockReserved); err != nil {\n\t\tt.Fatalf(\"expected f2 to acquire RESERVED after f1 released, got %v\", err)\n\t}\n\tif err := f2.Unlock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestVFS_ConcurrentOpenAllSucceed(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tv := NewVFS(client, slog.Default())\n\tv.WriteEnabled = true\n\n\tconst n = 10\n\tvar wg sync.WaitGroup\n\terrs := make([]error, n)\n\tfiles := make([]sqlite3vfs.File, n)\n\n\tfor i := range n {\n\t\twg.Add(1)\n\t\tgo func(idx int) {\n\t\t\tdefer wg.Done()\n\t\t\tf, _, err := v.openMainDB(\"test.db\", sqlite3vfs.OpenMainDB|sqlite3vfs.OpenReadWrite)\n\t\t\terrs[idx] = err\n\t\t\tfiles[idx] = f\n\t\t}(i)\n\t}\n\twg.Wait()\n\n\tvar opened int\n\tfor i, err := range errs {\n\t\tif err != nil {\n\t\t\tt.Errorf(\"connection %d failed to open: %v\", i, err)\n\t\t} else {\n\t\t\topened++\n\t\t\tfiles[i].(io.Closer).Close()\n\t\t}\n\t}\n\tif opened != n {\n\t\tt.Errorf(\"expected all %d connections to open, got %d\", n, opened)\n\t}\n}\n\nfunc TestVFS_UniqueBufferPaths(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tv := NewVFS(client, slog.Default())\n\tv.WriteEnabled = true\n\n\tf1 := openWriteVFSFile(t, v)\n\tf2 := openWriteVFSFile(t, v)\n\n\tif f1.bufferPath == f2.bufferPath {\n\t\tt.Errorf(\"buffer paths should be unique: both are %q\", f1.bufferPath)\n\t}\n}\n\nfunc TestVFS_RealConflict_StillDetected(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tv := NewVFS(client, slog.Default())\n\tv.WriteEnabled = true\n\tv.WriteSyncInterval = 0\n\n\tf1 := openWriteVFSFile(t, v)\n\n\t// Write dirty data\n\tif _, err := f1.WriteAt([]byte(\"data\"), 0); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Simulate external writer advancing remote\n\tcreateTestLTXFile(t, client, 2, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\t// Sync should fail with real conflict\n\terr := f1.Sync(0)\n\tif err == nil {\n\t\tt.Fatal(\"expected conflict error from external writer\")\n\t}\n\tif !errors.Is(err, ErrConflict) {\n\t\tt.Fatalf(\"expected ErrConflict, got: %v\", err)\n\t}\n}\n\nfunc TestVFS_CloseReleasesWriteSlot(t *testing.T) {\n\tclient := newWriteTestReplicaClient()\n\tpageSize := uint32(4096)\n\tinitialPage := make([]byte, pageSize)\n\tcreateTestLTXFile(t, client, 1, pageSize, 1, map[uint32][]byte{1: initialPage})\n\n\tv := NewVFS(client, slog.Default())\n\tv.WriteEnabled = true\n\n\t// Open and acquire RESERVED\n\tfile1, _, err := v.openMainDB(\"test.db\", sqlite3vfs.OpenMainDB|sqlite3vfs.OpenReadWrite)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tf1 := file1.(*VFSFile)\n\tif err := f1.Lock(sqlite3vfs.LockReserved); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\t// Close f1 (should release write slot)\n\tf1.Close()\n\n\t// New connection should be able to acquire RESERVED\n\tf2 := openWriteVFSFile(t, v)\n\tif err := f2.Lock(sqlite3vfs.LockReserved); err != nil {\n\t\tt.Fatalf(\"expected f2 to acquire RESERVED after f1 closed, got %v\", err)\n\t}\n\tif err := f2.Unlock(sqlite3vfs.LockShared); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n" }, { "path": "wal_reader.go", "content": "package litestream\n\nimport (\n\t\"context\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\n// WALReader wraps an io.Reader and parses SQLite WAL frames.\n//\n// This reader verifies the salt & checksum integrity while it reads. It does\n// not enforce transaction boundaries (i.e. it may return uncommitted frames).\n// It is the responsibility of the caller to handle this.\ntype WALReader struct {\n\tr io.ReaderAt\n\tframeN int\n\n\tbo binary.ByteOrder\n\tpageSize uint32\n\tseq uint32\n\n\tsalt1, salt2 uint32\n\tchksum1, chksum2 uint32\n\n\tlogger *slog.Logger\n}\n\n// NewWALReader returns a new instance of WALReader.\nfunc NewWALReader(rd io.ReaderAt, logger *slog.Logger) (*WALReader, error) {\n\tr := &WALReader{r: rd, logger: logger}\n\tif err := r.readHeader(); err != nil {\n\t\treturn nil, err\n\t}\n\treturn r, nil\n}\n\n// NewWALReaderWithOffset returns a new instance of WALReader at a given offset.\n// Salt must match or else no frames will be returned. Checksum calculated from\n// from previous page.\nfunc NewWALReaderWithOffset(ctx context.Context, rd io.ReaderAt, offset int64, salt1, salt2 uint32, logger *slog.Logger) (*WALReader, error) {\n\t// Ensure we are not starting on the first page since we need to read the previous.\n\tif offset <= WALHeaderSize {\n\t\treturn nil, fmt.Errorf(\"offset (%d) must be greater than the wal header size (%d)\", offset, WALHeaderSize)\n\t}\n\n\tr := &WALReader{r: rd, logger: logger}\n\n\t// Read header to determine page size & byte order.\n\tif err := r.readHeader(); err != nil {\n\t\treturn nil, fmt.Errorf(\"read header: %w\", err)\n\t}\n\n\t// Load in salt in case the beginning of the file has been overwritten.\n\tr.salt1, r.salt2 = salt1, salt2\n\n\t// Ensure offset is positioned on a frame start.\n\tframeSize := int64(r.pageSize + WALFrameHeaderSize)\n\tif (offset-WALHeaderSize)%frameSize != 0 {\n\t\treturn nil, fmt.Errorf(\"unaligned wal offset %d for page size %d\", offset, r.pageSize)\n\t}\n\tr.frameN = int((offset - WALHeaderSize) / frameSize)\n\n\t// Read previous page to load checksum.\n\tr.frameN--\n\tif _, _, err := r.readFrame(ctx, make([]byte, r.pageSize), false); err != nil {\n\t\treturn nil, &PrevFrameMismatchError{Err: err}\n\t}\n\n\treturn r, nil\n}\n\n// PageSize returns the page size from the header. Must call ReadHeader() first.\nfunc (r *WALReader) PageSize() uint32 { return r.pageSize }\n\n// Offset returns the file offset of the last read frame.\n// Returns zero if no frames have been read.\nfunc (r *WALReader) Offset() int64 {\n\tif r.frameN == 0 {\n\t\treturn 0\n\t}\n\treturn WALHeaderSize + ((int64(r.frameN) - 1) * (WALFrameHeaderSize + int64(r.pageSize)))\n}\n\n// readHeader reads the WAL header into the reader. Returns io.EOF if WAL is invalid.\nfunc (r *WALReader) readHeader() error {\n\t// If we have a partial WAL, then mark WAL as done.\n\thdr := make([]byte, WALHeaderSize)\n\tif n, err := r.r.ReadAt(hdr, 0); n < len(hdr) {\n\t\treturn io.EOF\n\t} else if err != nil {\n\t\treturn err\n\t}\n\n\t// Determine byte order of checksums.\n\tswitch magic := binary.BigEndian.Uint32(hdr[0:]); magic {\n\tcase 0x377f0682:\n\t\tr.bo = binary.LittleEndian\n\tcase 0x377f0683:\n\t\tr.bo = binary.BigEndian\n\tdefault:\n\t\treturn fmt.Errorf(\"invalid wal header magic: %x\", magic)\n\t}\n\n\t// If the header checksum doesn't match then we may have failed with\n\t// a partial WAL header write during checkpointing.\n\tchksum1 := binary.BigEndian.Uint32(hdr[24:])\n\tchksum2 := binary.BigEndian.Uint32(hdr[28:])\n\tif v0, v1 := WALChecksum(r.bo, 0, 0, hdr[:24]); v0 != chksum1 || v1 != chksum2 {\n\t\treturn io.EOF\n\t}\n\n\t// Verify version is correct.\n\tif version := binary.BigEndian.Uint32(hdr[4:]); version != 3007000 {\n\t\treturn fmt.Errorf(\"unsupported wal version: %d\", version)\n\t}\n\n\tr.pageSize = binary.BigEndian.Uint32(hdr[8:])\n\tr.seq = binary.BigEndian.Uint32(hdr[12:])\n\tr.salt1 = binary.BigEndian.Uint32(hdr[16:])\n\tr.salt2 = binary.BigEndian.Uint32(hdr[20:])\n\tr.chksum1, r.chksum2 = chksum1, chksum2\n\n\treturn nil\n}\n\n// ReadFrame reads the next frame from the WAL and returns the page number.\n// Returns io.EOF at the end of the valid WAL.\nfunc (r *WALReader) ReadFrame(ctx context.Context, data []byte) (pgno, commit uint32, err error) {\n\treturn r.readFrame(ctx, data, true)\n}\n\nfunc (r *WALReader) readFrame(_ context.Context, data []byte, verifyChecksum bool) (pgno, commit uint32, err error) {\n\tif len(data) != int(r.pageSize) {\n\t\treturn 0, 0, fmt.Errorf(\"WALReader.ReadFrame(): buffer size (%d) must match page size (%d)\", len(data), r.pageSize)\n\t}\n\n\tframeSize := r.pageSize + WALFrameHeaderSize\n\toffset := WALHeaderSize + (int64(r.frameN) * int64(frameSize))\n\n\t// Read WAL frame header.\n\thdr := make([]byte, WALFrameHeaderSize)\n\tif n, err := r.r.ReadAt(hdr, offset); n != len(hdr) {\n\t\treturn 0, 0, io.EOF\n\t} else if err != nil {\n\t\treturn 0, 0, err\n\t}\n\n\t// Read WAL page data.\n\tif n, err := r.r.ReadAt(data, offset+WALFrameHeaderSize); n != len(data) {\n\t\treturn 0, 0, io.EOF\n\t} else if err != nil {\n\t\treturn 0, 0, err\n\t}\n\n\t// Verify salt matches the salt in the header.\n\tsalt1 := binary.BigEndian.Uint32(hdr[8:])\n\tsalt2 := binary.BigEndian.Uint32(hdr[12:])\n\tif r.salt1 != salt1 || r.salt2 != salt2 {\n\t\treturn 0, 0, io.EOF\n\t}\n\n\t// Verify the checksum is valid. If checksum verification is disabled, it\n\t// is because we are jumping to an offset and not checksumming from the beginning.\n\tchksum1 := binary.BigEndian.Uint32(hdr[16:])\n\tchksum2 := binary.BigEndian.Uint32(hdr[20:])\n\tif verifyChecksum {\n\t\tr.chksum1, r.chksum2 = WALChecksum(r.bo, r.chksum1, r.chksum2, hdr[:8]) // frame header\n\t\tr.chksum1, r.chksum2 = WALChecksum(r.bo, r.chksum1, r.chksum2, data) // frame data\n\t\tif r.chksum1 != chksum1 || r.chksum2 != chksum2 {\n\t\t\treturn 0, 0, io.EOF\n\t\t}\n\t} else {\n\t\tr.chksum1, r.chksum2 = chksum1, chksum2\n\t}\n\n\tpgno = binary.BigEndian.Uint32(hdr[0:])\n\tcommit = binary.BigEndian.Uint32(hdr[4:])\n\n\tr.frameN++\n\n\treturn pgno, commit, nil\n}\n\n// PageMap reads all committed frames until the end of the file and returns a\n// map of pgno to offset of the latest version of each page. Also returns the\n// max offset of the wal segment read, and the final database size, in pages.\nfunc (r *WALReader) PageMap(ctx context.Context) (m map[uint32]int64, maxOffset int64, commit uint32, err error) {\n\tm = make(map[uint32]int64)\n\ttxMap := make(map[uint32]int64)\n\tdata := make([]byte, r.pageSize)\n\tfor i := 0; ; i++ {\n\t\tpgno, fcommit, err := r.ReadFrame(ctx, data)\n\t\tif errors.Is(err, io.EOF) {\n\t\t\tbreak\n\t\t} else if err != nil {\n\t\t\treturn nil, 0, 0, err\n\t\t}\n\n\t\t// Update latest offset for the page for this transaction.\n\t\t// Pages should not be saved to full map until we know txn is committed.\n\t\toffset := r.Offset()\n\t\ttxMap[pgno] = offset\n\n\t\t// For commit records, transfer offsets to full map and update db size.\n\t\tif fcommit != 0 {\n\t\t\tfor pgno, offset := range txMap {\n\t\t\t\tm[pgno] = offset\n\t\t\t}\n\t\t\tcommit = fcommit\n\t\t}\n\t}\n\n\t// Remove pages that exceed the final commit size. This can occur when the\n\t// database shrinks (e.g., via VACUUM) between transactions in the WAL.\n\tfor pgno := range m {\n\t\tif pgno > commit {\n\t\t\tdelete(m, pgno)\n\t\t}\n\t}\n\n\t// If full transactions available, return the original offset.\n\tif len(m) == 0 {\n\t\treturn m, 0, 0, nil\n\t}\n\n\t// Compute the highest page offsets.\n\tvar end int64\n\tfor _, offset := range m {\n\t\tif end == 0 || offset > end {\n\t\t\tend = offset\n\t\t}\n\t}\n\n\t// Extend to the end of the last frame read.\n\tend += WALFrameHeaderSize + int64(r.pageSize)\n\n\tr.logger.Log(ctx, internal.LevelTrace, \"page map complete\", \"n\", len(m), \"end\", end, \"commit\", commit)\n\treturn m, end, commit, nil\n}\n\n// FrameSaltsUntil returns a set of all unique frame salts in the WAL file.\nfunc (r *WALReader) FrameSaltsUntil(ctx context.Context, until [2]uint32) (map[[2]uint32]struct{}, error) {\n\tm := make(map[[2]uint32]struct{})\n\tfor offset := int64(WALHeaderSize); ; offset += int64(WALFrameHeaderSize + r.pageSize) {\n\t\thdr := make([]byte, WALFrameHeaderSize)\n\t\tif n, err := r.r.ReadAt(hdr, offset); n != len(hdr) {\n\t\t\tbreak\n\t\t} else if err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tsalt1 := binary.BigEndian.Uint32(hdr[8:])\n\t\tsalt2 := binary.BigEndian.Uint32(hdr[12:])\n\n\t\t// Track unique salts.\n\t\tm[[2]uint32{salt1, salt2}] = struct{}{}\n\n\t\t// Only read salts until the last one we expect.\n\t\tif salt1 == until[0] && salt2 == until[1] {\n\t\t\tbreak\n\t\t}\n\t}\n\n\treturn m, nil\n}\n\n// WALChecksum computes a running SQLite WAL checksum over a byte slice.\nfunc WALChecksum(bo binary.ByteOrder, s0, s1 uint32, b []byte) (uint32, uint32) {\n\tassert(len(b)%8 == 0, \"misaligned checksum byte slice\")\n\n\t// Iterate over 8-byte units and compute checksum.\n\tfor i := 0; i < len(b); i += 8 {\n\t\ts0 += bo.Uint32(b[i:]) + s1\n\t\ts1 += bo.Uint32(b[i+4:]) + s0\n\t}\n\treturn s0, s1\n}\n\ntype PrevFrameMismatchError struct {\n\tErr error\n}\n\nfunc (e *PrevFrameMismatchError) Error() string {\n\treturn fmt.Sprintf(\"prev frame mismatch: %s\", e.Err)\n}\n\nfunc (e *PrevFrameMismatchError) Unwrap() error {\n\treturn e.Err\n}\n" }, { "path": "wal_reader_test.go", "content": "package litestream_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"io\"\n\t\"log/slog\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/benbjohnson/litestream\"\n)\n\nfunc TestWALReader(t *testing.T) {\n\tt.Run(\"OK\", func(t *testing.T) {\n\t\tbuf := make([]byte, 4096)\n\t\tb, err := os.ReadFile(\"testdata/wal-reader/ok/wal\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Initialize reader with header info.\n\t\tr, err := litestream.NewWALReader(bytes.NewReader(b), slog.Default())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := r.PageSize(), uint32(4096); got != want {\n\t\t\tt.Fatalf(\"PageSize()=%d, want %d\", got, want)\n\t\t} else if got, want := r.Offset(), int64(0); got != want {\n\t\t\tt.Fatalf(\"Offset()=%d, want %d\", got, want)\n\t\t}\n\n\t\t// Read first frame.\n\t\tif pgno, commit, err := r.ReadFrame(context.Background(), buf); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pgno, uint32(1); got != want {\n\t\t\tt.Fatalf(\"pgno=%d, want %d\", got, want)\n\t\t} else if got, want := commit, uint32(0); got != want {\n\t\t\tt.Fatalf(\"commit=%d, want %d\", got, want)\n\t\t} else if !bytes.Equal(buf, b[56:4152]) {\n\t\t\tt.Fatal(\"page data mismatch\")\n\t\t} else if got, want := r.Offset(), int64(32); got != want {\n\t\t\tt.Fatalf(\"Offset()=%d, want %d\", got, want)\n\t\t}\n\n\t\t// Read second frame. End of transaction.\n\t\tif pgno, commit, err := r.ReadFrame(context.Background(), buf); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pgno, uint32(2); got != want {\n\t\t\tt.Fatalf(\"pgno=%d, want %d\", got, want)\n\t\t} else if got, want := commit, uint32(2); got != want {\n\t\t\tt.Fatalf(\"commit=%d, want %d\", got, want)\n\t\t} else if !bytes.Equal(buf, b[4176:8272]) {\n\t\t\tt.Fatal(\"page data mismatch\")\n\t\t} else if got, want := r.Offset(), int64(4152); got != want {\n\t\t\tt.Fatalf(\"Offset()=%d, want %d\", got, want)\n\t\t}\n\n\t\t// Read third frame.\n\t\tif pgno, commit, err := r.ReadFrame(context.Background(), buf); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pgno, uint32(2); got != want {\n\t\t\tt.Fatalf(\"pgno=%d, want %d\", got, want)\n\t\t} else if got, want := commit, uint32(2); got != want {\n\t\t\tt.Fatalf(\"commit=%d, want %d\", got, want)\n\t\t} else if !bytes.Equal(buf, b[8296:12392]) {\n\t\t\tt.Fatal(\"page data mismatch\")\n\t\t} else if got, want := r.Offset(), int64(8272); got != want {\n\t\t\tt.Fatalf(\"Offset()=%d, want %d\", got, want)\n\t\t}\n\n\t\tif _, _, err := r.ReadFrame(context.Background(), buf); !errors.Is(err, io.EOF) {\n\t\t\tt.Fatalf(\"unexpected error: %s\", err)\n\t\t}\n\t})\n\n\tt.Run(\"SaltMismatch\", func(t *testing.T) {\n\t\tbuf := make([]byte, 4096)\n\t\tb, err := os.ReadFile(\"testdata/wal-reader/salt-mismatch/wal\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Initialize reader with header info.\n\t\tr, err := litestream.NewWALReader(bytes.NewReader(b), slog.Default())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := r.PageSize(), uint32(4096); got != want {\n\t\t\tt.Fatalf(\"PageSize()=%d, want %d\", got, want)\n\t\t} else if got, want := r.Offset(), int64(0); got != want {\n\t\t\tt.Fatalf(\"Offset()=%d, want %d\", got, want)\n\t\t}\n\n\t\t// Read first frame.\n\t\tif pgno, commit, err := r.ReadFrame(context.Background(), buf); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pgno, uint32(1); got != want {\n\t\t\tt.Fatalf(\"pgno=%d, want %d\", got, want)\n\t\t} else if got, want := commit, uint32(0); got != want {\n\t\t\tt.Fatalf(\"commit=%d, want %d\", got, want)\n\t\t} else if !bytes.Equal(buf, b[56:4152]) {\n\t\t\tt.Fatal(\"page data mismatch\")\n\t\t}\n\n\t\t// Read second frame. Salt has been altered so it doesn't match header.\n\t\tif _, _, err := r.ReadFrame(context.Background(), buf); !errors.Is(err, io.EOF) {\n\t\t\tt.Fatalf(\"unexpected error: %s\", err)\n\t\t}\n\t})\n\n\tt.Run(\"FrameChecksumMismatch\", func(t *testing.T) {\n\t\tbuf := make([]byte, 4096)\n\t\tb, err := os.ReadFile(\"testdata/wal-reader/frame-checksum-mismatch/wal\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Initialize reader with header info.\n\t\tr, err := litestream.NewWALReader(bytes.NewReader(b), slog.Default())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := r.PageSize(), uint32(4096); got != want {\n\t\t\tt.Fatalf(\"PageSize()=%d, want %d\", got, want)\n\t\t} else if got, want := r.Offset(), int64(0); got != want {\n\t\t\tt.Fatalf(\"Offset()=%d, want %d\", got, want)\n\t\t}\n\n\t\t// Read first frame.\n\t\tif pgno, commit, err := r.ReadFrame(context.Background(), buf); err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if got, want := pgno, uint32(1); got != want {\n\t\t\tt.Fatalf(\"pgno=%d, want %d\", got, want)\n\t\t} else if got, want := commit, uint32(0); got != want {\n\t\t\tt.Fatalf(\"commit=%d, want %d\", got, want)\n\t\t} else if !bytes.Equal(buf, b[56:4152]) {\n\t\t\tt.Fatal(\"page data mismatch\")\n\t\t}\n\n\t\t// Read second frame. Checksum has been altered so it doesn't match.\n\t\tif _, _, err := r.ReadFrame(context.Background(), buf); !errors.Is(err, io.EOF) {\n\t\t\tt.Fatalf(\"unexpected error: %s\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ZeroLength\", func(t *testing.T) {\n\t\t_, err := litestream.NewWALReader(bytes.NewReader(nil), slog.Default())\n\t\tif !errors.Is(err, io.EOF) {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"PartialHeader\", func(t *testing.T) {\n\t\t_, err := litestream.NewWALReader(bytes.NewReader(make([]byte, 10)), slog.Default())\n\t\tif !errors.Is(err, io.EOF) {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"BadMagic\", func(t *testing.T) {\n\t\t_, err := litestream.NewWALReader(bytes.NewReader(make([]byte, 32)), slog.Default())\n\t\tif err == nil || err.Error() != `invalid wal header magic: 0` {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"BadHeaderChecksum\", func(t *testing.T) {\n\t\tdata := []byte{\n\t\t\t0x37, 0x7f, 0x06, 0x83, 0x00, 0x00, 0x00, 0x00,\n\t\t\t0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\n\t\t\t0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\n\t\t\t0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}\n\t\t_, err := litestream.NewWALReader(bytes.NewReader(data), slog.Default())\n\t\tif !errors.Is(err, io.EOF) {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"BadHeaderVersion\", func(t *testing.T) {\n\t\tdata := []byte{\n\t\t\t0x37, 0x7f, 0x06, 0x83, 0x00, 0x00, 0x00, 0x01,\n\t\t\t0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\n\t\t\t0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\n\t\t\t0x15, 0x7b, 0x20, 0x92, 0xbb, 0xf8, 0x34, 0x1d}\n\t\t_, err := litestream.NewWALReader(bytes.NewReader(data), slog.Default())\n\t\tif err == nil || err.Error() != `unsupported wal version: 1` {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ErrBufferSize\", func(t *testing.T) {\n\t\tb, err := os.ReadFile(\"testdata/wal-reader/ok/wal\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\t// Initialize reader with header info.\n\t\tr, err := litestream.NewWALReader(bytes.NewReader(b), slog.Default())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, _, err := r.ReadFrame(context.Background(), make([]byte, 512)); err == nil || err.Error() != `WALReader.ReadFrame(): buffer size (512) must match page size (4096)` {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ErrPartialFrameHeader\", func(t *testing.T) {\n\t\tb, err := os.ReadFile(\"testdata/wal-reader/ok/wal\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tr, err := litestream.NewWALReader(bytes.NewReader(b[:40]), slog.Default())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if _, _, err := r.ReadFrame(context.Background(), make([]byte, 4096)); !errors.Is(err, io.EOF) {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ErrFrameHeaderOnly\", func(t *testing.T) {\n\t\tb, err := os.ReadFile(\"testdata/wal-reader/ok/wal\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tr, err := litestream.NewWALReader(bytes.NewReader(b[:56]), slog.Default())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if _, _, err := r.ReadFrame(context.Background(), make([]byte, 4096)); !errors.Is(err, io.EOF) {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"ErrPartialFrameData\", func(t *testing.T) {\n\t\tb, err := os.ReadFile(\"testdata/wal-reader/ok/wal\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tr, err := litestream.NewWALReader(bytes.NewReader(b[:1000]), slog.Default())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t} else if _, _, err := r.ReadFrame(context.Background(), make([]byte, 4096)); !errors.Is(err, io.EOF) {\n\t\t\tt.Fatalf(\"unexpected error: %#v\", err)\n\t\t}\n\t})\n}\n\nfunc TestWALReader_FrameSaltsUntil(t *testing.T) {\n\tt.Run(\"OK\", func(t *testing.T) {\n\t\tb, err := os.ReadFile(\"testdata/wal-reader/frame-salts/wal\")\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tr, err := litestream.NewWALReader(bytes.NewReader(b), slog.Default())\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\n\t\tm, err := r.FrameSaltsUntil(context.Background(), [2]uint32{0x00000000, 0x00000000})\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif got, want := len(m), 3; got != want {\n\t\t\tt.Fatalf(\"len(m)=%d, want %d\", got, want)\n\t\t}\n\t\tif _, ok := m[[2]uint32{0x1b9a294b, 0x37f91916}]; !ok {\n\t\t\tt.Fatalf(\"salt 0 not found\")\n\t\t}\n\t\tif _, ok := m[[2]uint32{0x1b9a294a, 0x031f195e}]; !ok {\n\t\t\tt.Fatalf(\"salt 1 not found\")\n\t\t}\n\t\tif _, ok := m[[2]uint32{0x1b9a2949, 0x13b3dd67}]; !ok {\n\t\t\tt.Fatalf(\"salt 2 not found\")\n\t\t}\n\t})\n}\n" }, { "path": "webdav/replica_client.go", "content": "package webdav\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"log/slog\"\n\t\"net/url\"\n\t\"os\"\n\t\"path\"\n\t\"sort\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/studio-b12/gowebdav\"\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream\"\n\t\"github.com/benbjohnson/litestream/internal\"\n)\n\nfunc init() {\n\tlitestream.RegisterReplicaClientFactory(\"webdav\", NewReplicaClientFromURL)\n\tlitestream.RegisterReplicaClientFactory(\"webdavs\", NewReplicaClientFromURL)\n}\n\nconst ReplicaClientType = \"webdav\"\n\nconst (\n\tDefaultTimeout = 30 * time.Second\n)\n\nvar _ litestream.ReplicaClient = (*ReplicaClient)(nil)\n\ntype ReplicaClient struct {\n\tmu sync.Mutex\n\tclient *gowebdav.Client\n\tlogger *slog.Logger\n\n\tURL string\n\tUsername string\n\tPassword string\n\tPath string\n\tTimeout time.Duration\n}\n\nfunc NewReplicaClient() *ReplicaClient {\n\treturn &ReplicaClient{\n\t\tlogger: slog.Default().WithGroup(ReplicaClientType),\n\t\tTimeout: DefaultTimeout,\n\t}\n}\n\nfunc (c *ReplicaClient) SetLogger(logger *slog.Logger) {\n\tc.logger = logger.WithGroup(ReplicaClientType)\n}\n\n// NewReplicaClientFromURL creates a new ReplicaClient from URL components.\n// This is used by the replica client factory registration.\n// URL format: webdav://[user[:password]@]host[:port]/path or webdavs://... (for HTTPS)\nfunc NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error) {\n\tclient := NewReplicaClient()\n\n\t// Determine HTTP or HTTPS based on scheme\n\thttpScheme := \"http\"\n\tif scheme == \"webdavs\" {\n\t\thttpScheme = \"https\"\n\t}\n\n\t// Extract credentials from userinfo\n\tif userinfo != nil {\n\t\tclient.Username = userinfo.Username()\n\t\tclient.Password, _ = userinfo.Password()\n\t}\n\n\tif host == \"\" {\n\t\treturn nil, fmt.Errorf(\"host required for webdav replica URL\")\n\t}\n\n\tclient.URL = fmt.Sprintf(\"%s://%s\", httpScheme, host)\n\tclient.Path = urlPath\n\n\treturn client, nil\n}\n\nfunc (c *ReplicaClient) Type() string {\n\treturn ReplicaClientType\n}\n\nfunc (c *ReplicaClient) Init(ctx context.Context) error {\n\t_, err := c.init(ctx)\n\treturn err\n}\n\n// init initializes the connection and returns the WebDAV client.\nfunc (c *ReplicaClient) init(ctx context.Context) (_ *gowebdav.Client, err error) {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tif c.client != nil {\n\t\treturn c.client, nil\n\t}\n\n\tif c.URL == \"\" {\n\t\treturn nil, fmt.Errorf(\"webdav url required\")\n\t}\n\n\tc.client = gowebdav.NewClient(c.URL, c.Username, c.Password)\n\n\tc.client.SetTimeout(c.Timeout)\n\n\tif err := c.client.Connect(); err != nil {\n\t\tc.client = nil\n\t\treturn nil, fmt.Errorf(\"webdav: cannot connect to server: %w\", err)\n\t}\n\n\treturn c.client, nil\n}\n\nfunc (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n\tclient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := client.RemoveAll(c.Path); err != nil && !os.IsNotExist(err) && !gowebdav.IsErrNotFound(err) {\n\t\treturn fmt.Errorf(\"webdav: cannot delete path %q: %w\", c.Path, err)\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\n\treturn nil\n}\n\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, _ bool) (_ ltx.FileIterator, err error) {\n\tclient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdir := litestream.LTXLevelDir(c.Path, level)\n\tfiles, err := client.ReadDir(dir)\n\tif err != nil {\n\t\tif os.IsNotExist(err) || gowebdav.IsErrNotFound(err) {\n\t\t\treturn ltx.NewFileInfoSliceIterator(nil), nil\n\t\t}\n\t\treturn nil, fmt.Errorf(\"webdav: cannot read directory %q: %w\", dir, err)\n\t}\n\n\tinfos := make([]*ltx.FileInfo, 0, len(files))\n\tfor _, fi := range files {\n\t\tif fi.IsDir() {\n\t\t\tcontinue\n\t\t}\n\n\t\tminTXID, maxTXID, err := ltx.ParseFilename(path.Base(fi.Name()))\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t} else if minTXID < seek {\n\t\t\tcontinue\n\t\t}\n\n\t\tinfos = append(infos, <x.FileInfo{\n\t\t\tLevel: level,\n\t\t\tMinTXID: minTXID,\n\t\t\tMaxTXID: maxTXID,\n\t\t\tSize: fi.Size(),\n\t\t\tCreatedAt: fi.ModTime().UTC(),\n\t\t})\n\t}\n\n\tsort.Slice(infos, func(i, j int) bool {\n\t\tif infos[i].MinTXID != infos[j].MinTXID {\n\t\t\treturn infos[i].MinTXID < infos[j].MinTXID\n\t\t}\n\t\treturn infos[i].MaxTXID < infos[j].MaxTXID\n\t})\n\n\treturn ltx.NewFileInfoSliceIterator(infos), nil\n}\n\n// WriteLTXFile writes an LTX file to the WebDAV server.\n//\n// WebDAV Upload Strategy - Temp File Approach:\n//\n// Unlike other replica backends (S3, SFTP, NATS, ABS) which stream directly using\n// internal.NewReadCounter, WebDAV requires a different approach due to library and\n// protocol constraints:\n//\n// 1. gowebdav Library Limitations:\n//\n// - WriteStream() buffers entire payload in memory for non-seekable readers\n//\n// - WriteStreamWithLength() requires both content-length AND seekable reader\n//\n// - No native support for HTTP chunked transfer encoding\n//\n// 2. Server Compatibility Issues:\n// Research shows HTTP chunked transfer encoding with WebDAV is unreliable:\n//\n// - Nginx + FastCGI: Discards request body → 0-byte files (silent data loss)\n//\n// - Lighttpd: Returns HTTP 411 (Length Required), rejects chunked requests\n//\n// - Apache + FastCGI: Request body never arrives at application\n//\n// - Only Apache + mod_php handles chunked encoding reliably (~30-40% of deployments)\n//\n// 3. LTX Header Requirement:\n// - Must peek at LTX header to extract timestamp before upload\n// - Peeking consumes data, making the reader non-seekable\n// - Cannot calculate content-length without fully reading stream\n//\n// Solution: Stage to temporary file\n//\n// To ensure universal compatibility and prevent silent data loss:\n// 1. Extract timestamp from LTX header (required for file metadata)\n// 2. Stream full contents to temporary file on disk\n// 3. Seek back to start of temp file (now seekable + known size)\n// 4. Upload using WriteStreamWithLength() with Content-Length header\n// 5. Clean up temp file\n//\n// Trade-offs:\n// - Universal compatibility with all WebDAV server configurations\n// - No risk of silent data loss or failed uploads\n// - Predictable, reliable behavior\n// - Additional disk I/O overhead\n// - Requires local disk space proportional to LTX file size\n// - Diverges from streaming pattern used by other backends\n//\n// References:\n// - https://github.com/studio-b12/gowebdav/issues/35 (chunked encoding issues)\n// - https://github.com/nextcloud/server/issues/7995 (0-byte file bug)\n// - https://evertpot.com/260/ (WebDAV chunked encoding compatibility)\nfunc (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, rd io.Reader) (info *ltx.FileInfo, err error) {\n\tclient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfilename := litestream.LTXFilePath(c.Path, level, minTXID, maxTXID)\n\n\tvar buf bytes.Buffer\n\tteeReader := io.TeeReader(rd, &buf)\n\n\thdr, _, err := ltx.PeekHeader(teeReader)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"extract timestamp from LTX header: %w\", err)\n\t}\n\ttimestamp := time.UnixMilli(hdr.Timestamp).UTC()\n\n\t// Stage to temporary file to get seekable reader with known size.\n\t// This ensures compatibility with all WebDAV servers and avoids the\n\t// unreliable chunked transfer encoding that causes silent data loss\n\t// on common configurations (Nginx+FastCGI, Lighttpd, Apache+FastCGI).\n\ttmpFile, err := os.CreateTemp(\"\", \"litestream-webdav-*.ltx\")\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"webdav: cannot create temp file: %w\", err)\n\t}\n\tdefer func() {\n\t\t_ = tmpFile.Close()\n\t\t_ = os.Remove(tmpFile.Name())\n\t}()\n\n\tfullReader := io.MultiReader(&buf, rd)\n\n\tsize, err := io.Copy(tmpFile, fullReader)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"webdav: cannot copy to temp file: %w\", err)\n\t}\n\n\tif _, err := tmpFile.Seek(0, io.SeekStart); err != nil {\n\t\treturn nil, fmt.Errorf(\"webdav: cannot seek temp file: %w\", err)\n\t}\n\n\tif err := client.MkdirAll(path.Dir(filename), 0755); err != nil {\n\t\treturn nil, fmt.Errorf(\"webdav: cannot create parent directory %q: %w\", path.Dir(filename), err)\n\t}\n\n\t// Upload with Content-Length header using seekable temp file.\n\t// WriteStreamWithLength requires both a seekable reader and known size,\n\t// which we now have from the temp file. This avoids chunked encoding\n\t// and ensures reliable uploads across all WebDAV server configurations.\n\tif err := client.WriteStreamWithLength(filename, tmpFile, size, 0644); err != nil {\n\t\treturn nil, fmt.Errorf(\"webdav: cannot write file %q: %w\", filename, err)\n\t}\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Inc()\n\tinternal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, \"PUT\").Add(float64(size))\n\n\treturn <x.FileInfo{\n\t\tLevel: level,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tSize: size,\n\t\tCreatedAt: timestamp,\n\t}, nil\n}\n\nfunc (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (_ io.ReadCloser, err error) {\n\tclient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfilename := litestream.LTXFilePath(c.Path, level, minTXID, maxTXID)\n\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"GET\").Inc()\n\n\tif size > 0 {\n\t\trc, err := client.ReadStreamRange(filename, offset, size)\n\t\tif err != nil {\n\t\t\tif os.IsNotExist(err) || gowebdav.IsErrNotFound(err) {\n\t\t\t\treturn nil, os.ErrNotExist\n\t\t\t}\n\t\t\treturn nil, fmt.Errorf(\"webdav: cannot read file %q: %w\", filename, err)\n\t\t}\n\t\treturn internal.LimitReadCloser(rc, size), nil\n\t}\n\n\tif offset > 0 {\n\t\trc, err := client.ReadStream(filename)\n\t\tif err != nil {\n\t\t\tif os.IsNotExist(err) || gowebdav.IsErrNotFound(err) {\n\t\t\t\treturn nil, os.ErrNotExist\n\t\t\t}\n\t\t\treturn nil, fmt.Errorf(\"webdav: cannot read file %q: %w\", filename, err)\n\t\t}\n\n\t\tif _, err := io.CopyN(io.Discard, rc, offset); err != nil {\n\t\t\tif err == io.EOF || err == io.ErrUnexpectedEOF {\n\t\t\t\t_ = rc.Close()\n\t\t\t\treturn io.NopCloser(bytes.NewReader(nil)), nil\n\t\t\t}\n\t\t\t_ = rc.Close()\n\t\t\treturn nil, fmt.Errorf(\"webdav: cannot skip offset in file %q: %w\", filename, err)\n\t\t}\n\n\t\treturn rc, nil\n\t}\n\n\trc, err := client.ReadStream(filename)\n\tif err != nil {\n\t\tif os.IsNotExist(err) || gowebdav.IsErrNotFound(err) {\n\t\t\treturn nil, os.ErrNotExist\n\t\t}\n\t\treturn nil, fmt.Errorf(\"webdav: cannot read file %q: %w\", filename, err)\n\t}\n\treturn rc, nil\n}\n\nfunc (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error {\n\tclient, err := c.init(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, info := range a {\n\t\tfilename := litestream.LTXFilePath(c.Path, info.Level, info.MinTXID, info.MaxTXID)\n\n\t\tc.logger.Debug(\"deleting ltx file\", \"level\", info.Level, \"minTXID\", info.MinTXID, \"maxTXID\", info.MaxTXID, \"path\", filename)\n\n\t\tif err := client.Remove(filename); err != nil && !os.IsNotExist(err) && !gowebdav.IsErrNotFound(err) {\n\t\t\treturn fmt.Errorf(\"webdav: cannot delete ltx file %q: %w\", filename, err)\n\t\t}\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\t}\n\n\treturn nil\n}\n" }, { "path": "webdav/replica_client_test.go", "content": "package webdav_test\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/superfly/ltx\"\n\n\t\"github.com/benbjohnson/litestream/webdav\"\n)\n\nfunc TestReplicaClient_Type(t *testing.T) {\n\tc := webdav.NewReplicaClient()\n\tif got, want := c.Type(), \"webdav\"; got != want {\n\t\tt.Fatalf(\"Type()=%v, want %v\", got, want)\n\t}\n}\n\nfunc TestReplicaClient_Init_RequiresURL(t *testing.T) {\n\tc := webdav.NewReplicaClient()\n\tc.URL = \"\"\n\n\tif err := c.Init(context.TODO()); err == nil {\n\t\tt.Fatal(\"expected error when URL is empty\")\n\t} else if got, want := err.Error(), \"webdav url required\"; got != want {\n\t\tt.Fatalf(\"error=%v, want %v\", got, want)\n\t}\n}\n\nfunc TestReplicaClient_DeleteAll_NotFound(t *testing.T) {\n\tsrv := newFakeWebDAVServer()\n\tsrv.deleteReturn404[\"/missing\"] = true\n\n\tts := httptest.NewServer(srv)\n\tdefer ts.Close()\n\n\tc := newTestReplicaClient(ts.URL)\n\tc.Path = \"/missing\"\n\n\tif err := c.DeleteAll(context.Background()); err != nil {\n\t\tt.Fatalf(\"DeleteAll returned error: %v\", err)\n\t}\n}\n\nfunc TestReplicaClient_LTXFiles_PathNotFound(t *testing.T) {\n\tsrv := newFakeWebDAVServer()\n\tsrv.propfindReturn404[\"/missing/ltx/0\"] = true\n\n\tts := httptest.NewServer(srv)\n\tdefer ts.Close()\n\n\tc := newTestReplicaClient(ts.URL)\n\tc.Path = \"/missing\"\n\n\titr, err := c.LTXFiles(context.Background(), 0, 0, false)\n\tif err != nil {\n\t\tt.Fatalf(\"LTXFiles returned error: %v\", err)\n\t}\n\tdefer itr.Close()\n\n\tif itr.Next() {\n\t\tt.Fatal(\"expected no iterator items when directory is missing\")\n\t}\n}\n\nfunc TestReplicaClient_OpenLTXFile_RangeFallback(t *testing.T) {\n\tsrv := newFakeWebDAVServer()\n\tsrv.ignoreRange = true // simulate server ignoring Range requests\n\n\tts := httptest.NewServer(srv)\n\tdefer ts.Close()\n\n\tc := newTestReplicaClient(ts.URL)\n\tc.Path = \"/db\"\n\n\tpayload := []byte(\"ABCDEFGHIJ\")\n\tltxData := buildLTXPayload(1, 2, payload)\n\n\tif _, err := c.WriteLTXFile(context.Background(), 0, 1, 2, bytes.NewReader(ltxData)); err != nil {\n\t\tt.Fatalf(\"WriteLTXFile failed: %v\", err)\n\t}\n\n\trc, err := c.OpenLTXFile(context.Background(), 0, 1, 2, 0, int64(len(payload)))\n\tif err != nil {\n\t\tt.Fatalf(\"OpenLTXFile failed: %v\", err)\n\t}\n\tdefer rc.Close()\n\n\tgot, err := io.ReadAll(rc)\n\tif err != nil {\n\t\tt.Fatalf(\"ReadAll failed: %v\", err)\n\t}\n\n\tif want := ltxData[:len(payload)]; !bytes.Equal(got, want) {\n\t\tt.Fatalf(\"OpenLTXFile range fallback returned %q, want %q\", got, want)\n\t}\n}\n\nfunc TestReplicaClient_OpenLTXFile_OffsetOnly(t *testing.T) {\n\tsrv := newFakeWebDAVServer()\n\n\tts := httptest.NewServer(srv)\n\tdefer ts.Close()\n\n\tc := newTestReplicaClient(ts.URL)\n\tc.Path = \"/db\"\n\n\tpayload := []byte(\"ABCDEFGHIJ\")\n\tltxData := buildLTXPayload(10, 12, payload)\n\n\tif _, err := c.WriteLTXFile(context.Background(), 0, 10, 12, bytes.NewReader(ltxData)); err != nil {\n\t\tt.Fatalf(\"WriteLTXFile failed: %v\", err)\n\t}\n\n\toffset := int64(len(ltxData) - len(payload)/2)\n\trc, err := c.OpenLTXFile(context.Background(), 0, 10, 12, offset, 0)\n\tif err != nil {\n\t\tt.Fatalf(\"OpenLTXFile failed: %v\", err)\n\t}\n\tdefer rc.Close()\n\n\tgot, err := io.ReadAll(rc)\n\tif err != nil {\n\t\tt.Fatalf(\"ReadAll failed: %v\", err)\n\t}\n\n\tif want := ltxData[offset:]; !bytes.Equal(got, want) {\n\t\tt.Fatalf(\"OpenLTXFile offset reader returned %q, want %q\", got, want)\n\t}\n}\n\n// newTestReplicaClient returns a replica client pointed at a test server.\nfunc newTestReplicaClient(baseURL string) *webdav.ReplicaClient {\n\tc := webdav.NewReplicaClient()\n\tc.URL = baseURL\n\tc.Timeout = time.Second\n\treturn c\n}\n\n// buildLTXPayload constructs a minimal valid LTX file with payload data.\nfunc buildLTXPayload(minTXID, maxTXID ltx.TXID, payload []byte) []byte {\n\thdr := ltx.Header{\n\t\tVersion: 1,\n\t\tPageSize: 4096,\n\t\tCommit: 1,\n\t\tMinTXID: minTXID,\n\t\tMaxTXID: maxTXID,\n\t\tTimestamp: time.Now().UnixMilli(),\n\t}\n\theaderBytes, _ := hdr.MarshalBinary()\n\treturn append(headerBytes, payload...)\n}\n\n// fakeWebDAVServer provides a minimal in-memory WebDAV implementation for tests.\ntype fakeWebDAVServer struct {\n\tmu sync.Mutex\n\tfiles map[string][]byte\n\tignoreRange bool\n\tdeleteReturn404 map[string]bool\n\tpropfindReturn404 map[string]bool\n}\n\nfunc newFakeWebDAVServer() *fakeWebDAVServer {\n\treturn &fakeWebDAVServer{\n\t\tfiles: make(map[string][]byte),\n\t\tdeleteReturn404: make(map[string]bool),\n\t\tpropfindReturn404: make(map[string]bool),\n\t}\n}\n\nfunc (s *fakeWebDAVServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\n\tswitch r.Method {\n\tcase http.MethodOptions:\n\t\tw.WriteHeader(http.StatusOK)\n\tcase \"MKCOL\":\n\t\tw.WriteHeader(http.StatusCreated)\n\tcase http.MethodPut:\n\t\tbody, _ := io.ReadAll(r.Body)\n\t\ts.files[r.URL.Path] = body\n\t\tw.WriteHeader(http.StatusCreated)\n\tcase http.MethodGet:\n\t\ts.handleGet(w, r)\n\tcase http.MethodDelete:\n\t\ts.handleDelete(w, r)\n\tcase \"PROPFIND\":\n\t\ts.handlePropfind(w, r)\n\tdefault:\n\t\tw.WriteHeader(http.StatusNotImplemented)\n\t}\n}\n\nfunc (s *fakeWebDAVServer) handleGet(w http.ResponseWriter, r *http.Request) {\n\tdata, ok := s.files[r.URL.Path]\n\tif !ok {\n\t\thttp.NotFound(w, r)\n\t\treturn\n\t}\n\trangeHeader := r.Header.Get(\"Range\")\n\tif rangeHeader != \"\" && !s.ignoreRange {\n\t\tstart, end, err := parseRange(rangeHeader, len(data))\n\t\tif err != nil {\n\t\t\tw.WriteHeader(http.StatusRequestedRangeNotSatisfiable)\n\t\t\treturn\n\t\t}\n\n\t\tw.Header().Set(\"Content-Length\", strconv.Itoa(end-start))\n\t\tw.WriteHeader(http.StatusPartialContent)\n\t\t_, _ = w.Write(data[start:end])\n\t\treturn\n\t}\n\n\tw.Header().Set(\"Content-Length\", strconv.Itoa(len(data)))\n\tw.WriteHeader(http.StatusOK)\n\t_, _ = w.Write(data)\n}\n\nfunc (s *fakeWebDAVServer) handleDelete(w http.ResponseWriter, r *http.Request) {\n\ttrimmed := strings.TrimSuffix(r.URL.Path, \"/\")\n\tif s.deleteReturn404[trimmed] {\n\t\tw.WriteHeader(http.StatusNotFound)\n\t\treturn\n\t}\n\n\tif _, ok := s.files[r.URL.Path]; ok {\n\t\tdelete(s.files, r.URL.Path)\n\t\tw.WriteHeader(http.StatusNoContent)\n\t\treturn\n\t}\n\n\tw.WriteHeader(http.StatusNotFound)\n}\n\nfunc (s *fakeWebDAVServer) handlePropfind(w http.ResponseWriter, r *http.Request) {\n\ttrimmed := strings.TrimSuffix(r.URL.Path, \"/\")\n\tif s.propfindReturn404[trimmed] {\n\t\tw.WriteHeader(http.StatusNotFound)\n\t\treturn\n\t}\n\n\tresponse := fmt.Sprintf(`\n\n \n %s/\n \n \n \n 0\n %s\n \n HTTP/1.1 200 OK\n \n \n`, trimmed, time.Now().UTC().Format(http.TimeFormat))\n\n\tw.Header().Set(\"Content-Type\", \"application/xml; charset=utf-8\")\n\tw.Header().Set(\"Content-Length\", strconv.Itoa(len(response)))\n\tw.WriteHeader(207)\n\t_, _ = w.Write([]byte(response))\n}\n\nfunc parseRange(header string, size int) (start, end int, err error) {\n\tif !strings.HasPrefix(header, \"bytes=\") {\n\t\treturn 0, 0, fmt.Errorf(\"unsupported range header: %s\", header)\n\t}\n\tparts := strings.Split(strings.TrimPrefix(header, \"bytes=\"), \"-\")\n\tif len(parts) != 2 {\n\t\treturn 0, 0, fmt.Errorf(\"invalid range header: %s\", header)\n\t}\n\tstart, err = strconv.Atoi(parts[0])\n\tif err != nil {\n\t\treturn 0, 0, err\n\t}\n\tif parts[1] == \"\" {\n\t\tend = size\n\t} else {\n\t\tend, err = strconv.Atoi(parts[1])\n\t\tif err != nil {\n\t\t\treturn 0, 0, err\n\t\t}\n\t\tend++ // range end is inclusive\n\t}\n\n\tif start < 0 || end > size || start >= end {\n\t\treturn 0, 0, fmt.Errorf(\"invalid range [%d, %d) for size %d\", start, end, size)\n\t}\n\treturn start, end, nil\n}\n" } ]