Skip to content

Commit 695d6ab

Browse files
badrishcCopilot
andcommitted
Replace SubscribeEvictions with IRecordTriggers.OnEvict for heap-size tracking
Garnet's `CacheSizeTracker` used `LogAccessor.SubscribeEvictions` to drive a `LogSizeTracker.OnNext` observer that, on every page eviction, allocated a scan iterator and walked the page to sum `MemoryUtils.CalculateHeapMemorySize` over each non-null / non-closed record. This is heavyweight for a hot path (buffer-pool allocation, iterator bookkeeping, epoch resume/suspend, and a virtual dispatch per page). This PR migrates the per-page heap-size decrement onto the per-record `IRecordTriggers.OnEvict` hook introduced in #1695, which the object allocator already walks during `EvictRecordsInRange`. That lets us collapse the "scan iterator + observer + sum" path into a single per-record callback directly on the record we would have visited anyway. ### Tsavorite changes - Add `EvictionSource { MainLog, ReadCache }` and thread it through `IRecordTriggers.OnEvict`, `IStoreFunctions.OnEvict`, and `IAllocator.EvictRecordsInRange`. Garnet uses this to route decrements to the correct counter (`AddHeapSize` vs `AddReadCacheHeapSize`). - Add `AllocatorSettings.IsReadCache` and `AllocatorBase.IsReadCache`, set to `true` by `Tsavorite.cs` when constructing the read-cache allocator. This is the cleanest way to distinguish the two allocators at `OnPagesClosedWorker` time without relying on the `evictCallback` sentinel. - `AllocatorBase.EvictPageForRecovery` now also routes through the per-record `EvictRecordsInRange` when `storeFunctions.CallOnEvict` is set. The legacy `MemoryPageScan(observer)` path is preserved as a fallback for consumers that still use `SubscribeEvictions`. - Collapse `AllocatorBase`'s constructor to accept `AllocatorSettings` directly instead of unpacking individual fields at each concrete allocator (`ObjectAllocatorImpl`, `SpanByteAllocatorImpl`, `TsavoriteLogAllocatorImpl`). - Tighten `ObjectAllocatorImpl.EvictRecordsInRange` to match `ObjectScanIterator`'s single-page invariant: clip `stopAddress` to the start page, bail on `offset == 0`, and document that both callers (`OnPagesClosedWorker`, `EvictPageForRecovery`) hand single-page ranges. ### Garnet changes - `GarnetRecordTriggers.CallOnEvict` is gated on `cacheSizeTracker is not null && cacheSizeTracker.IsInitialized` so the hook stays off when no memory budget is configured (the tracker is always constructed due to the late-bound store reference, so a raw null-check would always be true). - `GarnetRecordTriggers.OnEvict(ref LogRecord, EvictionSource)` computes `MemoryUtils.CalculateHeapMemorySize(in logRecord)` and dispatches to `AddHeapSize(-size)` or `AddReadCacheHeapSize(-size)` based on the source. This goes through the standard asserting path — the counter must never undershoot zero. - `CacheSizeTracker.Initialize` replaces the two `SubscribeEvictions` calls with `SetLogSizeTracker` calls so the fast-path size tracking during `TryCopyToTail`, `TryCopyToReadCache`, and object-page growth (`UpdateSize`, `IncrementSize`) continues to work unchanged. ### MainStore heap-tracking fix (root cause of negative counter) Routing the eviction decrement through the asserting `AddHeapSize` path surfaced a pre-existing gap: `MainStore` RMW paths never emitted a positive heap-size bump when a record's value was overflow (large inline value spilled to a heap-allocated byte array). Only `Upsert`/in-place-writer paths and the object / unified stores tracked heap on creation. The legacy observer path masked this silently because `OnNext` did a raw decrement with no assertion — the counter quietly went negative on every HLL sparse→dense transition (≈-12 KB per key) and accumulated drift elsewhere. `MainStore/RMWMethods.cs` now emits balanced heap accounting: - `PostInitialUpdater`: `AddHeapSize(+logRecord.CalculateHeapMemorySize())` for the freshly-created record (includes both key and value overflow). - `PostCopyUpdater`: `AddHeapSize(+dst.CalculateHeapMemorySize() - src.CalculateHeapMemorySize())`. The destination is new (+dst); the source becomes sealed and is skipped by both the old iterator and the new `EvictRecordsInRange` walker, so its previously-counted contribution must be subtracted now to keep the counter balanced once `dst` eventually evicts. - `InPlaceUpdater`: snapshot `GetValueHeapMemorySize()` pre-/post-worker and add the delta on `Succeeded`, so value-heap changes (e.g. APPEND growing a large string into overflow) are tracked. The `HyperLogLogPFADD_LTM` suite is the regression that surfaces this — HLL sparse→dense goes through CopyUpdater and allocates ≈12 KB of overflow for the dense representation. With the fix the counter stays balanced and the assertion `heapSize.Total >= 0` holds throughout. ### Parity Behavior at every record state encountered during page eviction is bit-for-bit identical to the prior `SubscribeEvictions` path: | Record state | Old path (iterator) | New path (per-record) | Delta | | ------------------------------------------------ | ---------------------------------- | ---------------------------- | ----- | | Valid, !Sealed, !Tombstone, !IsNull | yielded → -CalculateHeapMemorySize | OnEvict → -Calculate... | same | | `IsNull` | skipped by iterator | skipped by filter | 0 | | `SkipOnScan` (Invalid or Sealed) | skipped by iterator | skipped by filter | 0 | | `Tombstone` (post-delete, already ValueIsInline) | yielded → 0 via `!Info.Tombstone` | skipped by filter → 0 | 0 | Tombstones converge via two independent mechanisms (the old relies on the `if (!Info.Tombstone)` guard inside `CalculateHeapMemorySize`; the new short-circuits in the filter), so both yield 0 contribution as required given the delete-site `OnDispose(Deleted)` already decremented the tracker. ### Testing All on net10.0 Debug: - `HyperLogLogTests` (incl. `HyperLogLogPFADD_LTM{32,4096}`, `HyperLogLogTestPFMERGE_LTM_*`) ✓ - `CacheSizeTrackerTests` ✓ - `RespListTests.ListPushPopStressTest` (10 repetitions) ✓ - `RespListTests`, `RespHashTests`, `RespSetTests`, `RespSortedSetTests`, `RespEtagTests`, `RespBitmapTests`: 367/367 ✓ - Clean build on both `Garnet.slnx` and `Tsavorite.slnx` (0 warnings, 0 errors), `dotnet format --verify-no-changes` clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 0740ff9 commit 695d6ab

File tree

16 files changed

+141
-37
lines changed

16 files changed

+141
-37
lines changed

libs/server/Storage/Functions/GarnetRecordTriggers.cs

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,13 @@ namespace Garnet.server
77
{
88
/// <summary>
99
/// Record lifecycle triggers for Garnet's unified store. Handles per-record cleanup
10-
/// on delete via <see cref="IRecordTriggers.OnDispose"/>.
10+
/// on delete via <see cref="IRecordTriggers.OnDispose"/> and per-record heap-size
11+
/// accounting on page eviction via <see cref="IRecordTriggers.OnEvict"/>.
1112
/// </summary>
1213
public readonly struct GarnetRecordTriggers : IRecordTriggers
1314
{
1415
/// <summary>
15-
/// Cache size tracker for heap size accounting on delete.
16+
/// Cache size tracker for heap size accounting on delete and eviction.
1617
/// Created before the store and initialized after via <see cref="CacheSizeTracker.Initialize"/>.
1718
/// </summary>
1819
internal readonly CacheSizeTracker cacheSizeTracker;
@@ -29,7 +30,13 @@ public GarnetRecordTriggers(CacheSizeTracker cacheSizeTracker)
2930
public bool CallOnFlush => false;
3031

3132
/// <inheritdoc/>
32-
public bool CallOnEvict => false;
33+
// Drives per-record heap-size decrement on page eviction. Mirrors the work the
34+
// legacy SubscribeEvictions → LogSizeTracker.OnNext observer path used to perform
35+
// (see CacheSizeTracker.Initialize for the wiring change). The tracker is always
36+
// constructed (late-bound to the store to break a ctor cycle) but only wires its
37+
// inner LogSizeTracker instances when a memory budget is configured, so gate on
38+
// IsInitialized rather than the field itself.
39+
public bool CallOnEvict => cacheSizeTracker is not null && cacheSizeTracker.IsInitialized;
3340

3441
/// <inheritdoc/>
3542
public bool CallOnDiskRead => false;
@@ -49,5 +56,26 @@ public void OnDispose(ref LogRecord logRecord, DisposeReason reason)
4956
cacheSizeTracker?.AddHeapSize(-logRecord.ValueObject.HeapMemorySize);
5057
}
5158
}
59+
60+
/// <inheritdoc/>
61+
public void OnEvict(ref LogRecord logRecord, EvictionSource source)
62+
{
63+
if (cacheSizeTracker is null)
64+
return;
65+
66+
// Decrement heap size by this record's heap contribution. Uses the same sizing
67+
// helper that LogSizeTracker.OnNext used to sum over an evicted iterator. Routes
68+
// through the standard AddHeapSize/AddReadCacheHeapSize path so the assertion
69+
// guarding against negative totals remains in force; creation sites on the main
70+
// log (RMW PostInitialUpdater/PostCopyUpdater and in-place grow/shrink) must emit
71+
// a matching positive bump so the account stays balanced.
72+
var size = MemoryUtils.CalculateHeapMemorySize(in logRecord);
73+
if (size == 0)
74+
return;
75+
if (source == EvictionSource.ReadCache)
76+
cacheSizeTracker.AddReadCacheHeapSize(-size);
77+
else
78+
cacheSizeTracker.AddHeapSize(-size);
79+
}
5280
}
5381
}

libs/server/Storage/Functions/MainStore/RMWMethods.cs

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -398,6 +398,12 @@ public readonly void PostInitialUpdater(ref LogRecord logRecord, in RecordSizeIn
398398
input.header.SetExpiredFlag();
399399
rmwInfo.UserData |= NeedAofLog; // Mark that we need to write to AOF
400400
}
401+
402+
// Account for any heap contribution (key/value overflow) of the newly created record;
403+
// balances the decrement emitted by GarnetRecordTriggers.OnEvict on page eviction.
404+
var heap = logRecord.CalculateHeapMemorySize();
405+
if (heap != 0)
406+
functionsState.cacheSizeTracker?.AddHeapSize(heap);
401407
}
402408

403409
/// <inheritdoc />
@@ -409,6 +415,7 @@ public readonly bool InPlaceUpdater(ref LogRecord logRecord, ref StringInput inp
409415
return false;
410416
}
411417

418+
var preHeap = logRecord.GetValueHeapMemorySize();
412419
var ipuResult = InPlaceUpdaterWorker(ref logRecord, ref input, ref output, ref rmwInfo);
413420
switch (ipuResult)
414421
{
@@ -419,6 +426,9 @@ public readonly bool InPlaceUpdater(ref LogRecord logRecord, ref StringInput inp
419426
functionsState.watchVersionMap.IncrementVersion(rmwInfo.KeyHash);
420427
if (functionsState.appendOnlyFile != null)
421428
rmwInfo.UserData |= NeedAofLog; // Mark that we need to write to AOF
429+
var delta = logRecord.GetValueHeapMemorySize() - preHeap;
430+
if (delta != 0)
431+
functionsState.cacheSizeTracker?.AddHeapSize(delta);
422432
return true;
423433
case IPUResult.NotUpdated:
424434
default:
@@ -1701,6 +1711,13 @@ public readonly bool PostCopyUpdater<TSourceLogRecord>(in TSourceLogRecord srcLo
17011711
functionsState.watchVersionMap.IncrementVersion(rmwInfo.KeyHash);
17021712
if (functionsState.appendOnlyFile != null)
17031713
rmwInfo.UserData |= NeedAofLog; // Mark that we need to write to AOF
1714+
1715+
// Heap-size delta: dst is new (needs +); src becomes sealed and is skipped by
1716+
// GarnetRecordTriggers.OnEvict, so its previously-counted heap contribution must
1717+
// be subtracted now to keep the counter balanced once dst eventually evicts.
1718+
var delta = dstLogRecord.CalculateHeapMemorySize() - srcLogRecord.CalculateHeapMemorySize();
1719+
if (delta != 0)
1720+
functionsState.cacheSizeTracker?.AddHeapSize(delta);
17041721
return true;
17051722
}
17061723

libs/server/Storage/SizeTracker/CacheSizeTracker.cs

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -65,8 +65,9 @@ public CacheSizeTracker(TsavoriteKV<StoreFunctions, StoreAllocator> store, long
6565
=> Initialize(store, targetSize, readCacheTargetSize, loggerFactory);
6666

6767
/// <summary>
68-
/// Initialize the tracker with a store. Subscribes to eviction notifications.
69-
/// Called after store creation when using the parameterless constructor.
68+
/// Initialize the tracker with a store. Wires the <see cref="LogSizeTracker"/> as the fast-path
69+
/// size tracker for copy-to-tail / copy-to-readcache. Per-record heap-size decrement on page
70+
/// eviction is driven by <see cref="GarnetRecordTriggers.OnEvict"/>.
7071
/// </summary>
7172
public void Initialize(TsavoriteKV<StoreFunctions, StoreAllocator> store, long targetSize, long readCacheTargetSize, ILoggerFactory loggerFactory = null)
7273
{
@@ -77,14 +78,14 @@ public void Initialize(TsavoriteKV<StoreFunctions, StoreAllocator> store, long t
7778
{
7879
mainLogTracker = new LogSizeTracker<StoreFunctions, StoreAllocator>(store.Log, targetSize,
7980
targetSize / HighTargetSizeDeltaFraction, targetSize / LowTargetSizeDeltaFraction, loggerFactory?.CreateLogger("MainLogSizeTracker"));
80-
store.Log.SubscribeEvictions(mainLogTracker);
81+
store.Log.SetLogSizeTracker(mainLogTracker);
8182
}
8283

8384
if (store.ReadCache != null && readCacheTargetSize > 0)
8485
{
8586
readCacheTracker = new LogSizeTracker<StoreFunctions, StoreAllocator>(store.ReadCache, readCacheTargetSize,
8687
readCacheTargetSize / HighTargetSizeDeltaFraction, readCacheTargetSize / LowTargetSizeDeltaFraction, loggerFactory?.CreateLogger("ReadCacheSizeTracker"));
87-
store.ReadCache.SubscribeEvictions(readCacheTracker);
88+
store.ReadCache.SetLogSizeTracker(readCacheTracker);
8889
}
8990
}
9091

libs/storage/Tsavorite/cs/src/core/Allocator/AllocatorBase.cs

Lines changed: 27 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -205,6 +205,12 @@ protected string BaseToString(string fuaDetails = "")
205205
/// <summary>Observer for records getting evicted from memory (page closed). May be the same object as <see cref="logSizeTracker"/>.</summary>
206206
internal IObserver<ITsavoriteScanIterator> onEvictionObserver;
207207

208+
/// <summary>
209+
/// Whether this allocator is the read cache (as opposed to the main hybrid log).
210+
/// Set once at construction from <see cref="AllocatorSettings.IsReadCache"/>.
211+
/// </summary>
212+
internal readonly bool IsReadCache;
213+
208214
/// <summary>Log size tracker; called when an operation at the Tsavorite-internal level adds or removes heap memory size
209215
/// (e.g. copying to log tail or read cache, which do not call <see cref="ISessionFunctions{TInputOutput, TContext}"/>).
210216
/// May be the same object as <see cref="onEvictionObserver"/>.</summary>
@@ -553,9 +559,15 @@ internal void WriteInlinePageAsync<TContext>(IntPtr alignedSourceAddress, ulong
553559

554560
/// <summary>Instantiate base allocator implementation</summary>
555561
[MethodImpl(MethodImplOptions.NoInlining)]
556-
private protected AllocatorBase(LogSettings logSettings, TStoreFunctions storeFunctions, Func<object, TAllocator> wrapperCreator, Action<long, long> evictCallback,
557-
LightEpoch epoch, Action<CommitInfo> flushCallback, ILogger logger = null, ObjectIdMap transientObjectIdMap = null)
562+
private protected AllocatorBase(AllocatorSettings allocatorSettings, TStoreFunctions storeFunctions, Func<object, TAllocator> wrapperCreator,
563+
ILogger logger = null, ObjectIdMap transientObjectIdMap = null)
558564
{
565+
var logSettings = allocatorSettings.LogSettings;
566+
var evictCallback = allocatorSettings.evictCallback;
567+
var epoch = allocatorSettings.epoch;
568+
var flushCallback = allocatorSettings.flushCallback;
569+
IsReadCache = allocatorSettings.IsReadCache;
570+
559571
this.storeFunctions = storeFunctions;
560572
_wrapper = wrapperCreator(this);
561573

@@ -1403,10 +1415,19 @@ public void ShiftBeginAddress(long newBeginAddress, bool truncateLog, bool noFlu
14031415
/// <summary>Invokes eviction observer if set and then frees the page.</summary>
14041416
internal void EvictPageForRecovery(long page)
14051417
{
1406-
if (logSizeTracker is not null)
1418+
var start = GetLogicalAddressOfStartOfPage(page);
1419+
var end = GetLogicalAddressOfStartOfPage(page + 1);
1420+
1421+
if (storeFunctions.CallOnEvict)
1422+
{
1423+
// New per-record path: OnEvict handles heap-size decrement per record. Parity
1424+
// with the runtime eviction path in OnPagesClosedWorker.
1425+
_wrapper.EvictRecordsInRange(start, end, IsReadCache ? EvictionSource.ReadCache : EvictionSource.MainLog);
1426+
}
1427+
else if (logSizeTracker is not null)
14071428
{
1408-
var start = GetLogicalAddressOfStartOfPage(page);
1409-
var end = GetLogicalAddressOfStartOfPage(page + 1);
1429+
// Legacy observer path: materialize an iterator and push heap-size decrement to
1430+
// the LogSizeTracker.OnNext observer (kept for consumers still using SubscribeEvictions).
14101431
MemoryPageScan(start, end, logSizeTracker);
14111432
}
14121433

@@ -1490,7 +1511,7 @@ private void OnPagesClosedWorker()
14901511

14911512
// Notify application of records being evicted — allows cleanup of external resources.
14921513
if (storeFunctions.CallOnEvict)
1493-
_wrapper.EvictRecordsInRange(start, end);
1514+
_wrapper.EvictRecordsInRange(start, end, IsReadCache ? EvictionSource.ReadCache : EvictionSource.MainLog);
14941515

14951516
// If we are using a null storage device, we must also shift BeginAddress (leave it in-memory)
14961517
if (IsNullDevice)

libs/storage/Tsavorite/cs/src/core/Allocator/AllocatorSettings.cs

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,12 @@ public struct AllocatorSettings
2323
/// <summary>The action to call on page eviction; used only for readcache</summary>
2424
internal Action<long, long> evictCallback;
2525

26+
/// <summary>
27+
/// Whether this allocator is the read cache (as opposed to the main hybrid log).
28+
/// Used to tag per-record eviction callbacks so applications can distinguish the source.
29+
/// </summary>
30+
internal bool IsReadCache;
31+
2632
/// <summary>The action to execute on flush completion; used only for <see cref="TsavoriteLog"/></summary>
2733
internal Action<CommitInfo> flushCallback;
2834

libs/storage/Tsavorite/cs/src/core/Allocator/IAllocator.cs

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,9 @@ RecordSizeInfo GetDeleteRecordSize<TKey>(TKey key)
117117
/// <see cref="IRecordTriggers.OnEvict"/> hook for each valid, non-tombstoned record.
118118
/// Used during page eviction to allow cleanup of external resources.
119119
/// </summary>
120-
void EvictRecordsInRange(long startAddress, long endAddress);
120+
/// <param name="startAddress">Start logical address of the range.</param>
121+
/// <param name="endAddress">End logical address of the range (exclusive).</param>
122+
/// <param name="source">Identifies whether this eviction is from the main log or the read cache.</param>
123+
void EvictRecordsInRange(long startAddress, long endAddress, EvictionSource source);
121124
}
122125
}

libs/storage/Tsavorite/cs/src/core/Allocator/ObjectAllocator.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,6 @@ public readonly RecordSizeInfo GetDeleteRecordSize<TKey>(TKey key)
146146
public readonly void OnDispose(ref DiskLogRecord logRecord, DisposeReason disposeReason) => _this.OnDispose(ref logRecord, disposeReason);
147147

148148
/// <inheritdoc/>
149-
public readonly void EvictRecordsInRange(long startAddress, long endAddress) => _this.EvictRecordsInRange(startAddress, endAddress);
149+
public readonly void EvictRecordsInRange(long startAddress, long endAddress, EvictionSource source) => _this.EvictRecordsInRange(startAddress, endAddress, source);
150150
}
151151
}

libs/storage/Tsavorite/cs/src/core/Allocator/ObjectAllocatorImpl.cs

Lines changed: 23 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ internal struct ObjectPage
7474
public override string ToString() => BaseToString($" (LI {LastIssuedFlushedUntilAddress}, OG {OngoingFlushedUntilAddress}, No {NoFlushUntilAddress})");
7575

7676
public ObjectAllocatorImpl(AllocatorSettings settings, TStoreFunctions storeFunctions, Func<object, ObjectAllocator<TStoreFunctions>> wrapperCreator)
77-
: base(settings.LogSettings, storeFunctions, wrapperCreator, settings.evictCallback, settings.epoch, settings.flushCallback, settings.logger, transientObjectIdMap: new ObjectIdMap())
77+
: base(settings, storeFunctions, wrapperCreator, settings.logger, transientObjectIdMap: new ObjectIdMap())
7878
{
7979
objectLogDevice = settings.LogSettings.ObjectLogDevice;
8080

@@ -346,39 +346,49 @@ internal void OnDispose(ref DiskLogRecord logRecord, DisposeReason disposeReason
346346
/// <summary>
347347
/// Iterate records in the given logical address range and call <see cref="IStoreFunctions.OnEvict"/>
348348
/// on each valid, non-tombstoned record. Used during page eviction to allow cleanup of external resources.
349+
/// The caller constrains <paramref name="startAddress"/> / <paramref name="endAddress"/> to lie on a single page
350+
/// (see <see cref="AllocatorBase{TStoreFunctions, TAllocator}.OnPagesClosedWorker"/> and
351+
/// <see cref="AllocatorBase{TStoreFunctions, TAllocator}.EvictPageForRecovery"/>), so this routine walks records
352+
/// within that single page only — matching the <see cref="ObjectScanIterator{TStoreFunctions, TAllocator}"/>
353+
/// semantics with <c>includeClosedRecords: false</c> that the legacy <c>SubscribeEvictions</c> path used.
349354
/// </summary>
350-
internal void EvictRecordsInRange(long startAddress, long endAddress)
355+
internal void EvictRecordsInRange(long startAddress, long endAddress, EvictionSource source)
351356
{
352-
// Ensure we start after the page header
353-
var page = GetPage(startAddress);
354-
var firstValidAddress = GetFirstValidLogicalAddressOnPage(page);
357+
// Clip to a single page: we don't cross page boundaries in one call, but be defensive in case the caller
358+
// ever passes a multi-page range. The start-page PageHeader is skipped via GetFirstValidLogicalAddressOnPage.
359+
var startPage = GetPage(startAddress);
360+
var firstValidAddress = GetFirstValidLogicalAddressOnPage(startPage);
355361
var address = startAddress < firstValidAddress ? firstValidAddress : startAddress;
362+
var pageEndAddress = GetLogicalAddressOfStartOfPage(startPage + 1);
363+
var stopAddress = endAddress < pageEndAddress ? endAddress : pageEndAddress;
356364

357-
while (address < endAddress)
365+
while (address < stopAddress)
358366
{
359367
var physicalAddress = GetPhysicalAddress(address);
360368
var logRecord = new LogRecord(physicalAddress, objectPages[GetPageIndexForAddress(address)].objectIdMap);
361369
var allocatedSize = logRecord.AllocatedSize;
362370

363-
// Guard against corrupt records causing infinite loops
371+
// Guard against corrupt / zero-size records causing infinite loops. Also stop if the record would
372+
// straddle the end of the page; ObjectScanIterator skips to the next page in that case, but our caller
373+
// constrains the range to a single page so there is nothing further to visit here.
364374
if (allocatedSize <= 0)
365375
break;
366-
367-
// If record does not fit on page, stop (shouldn't happen within a single-page eviction range)
368376
var offset = GetOffsetOnPage(address);
369-
if (offset + allocatedSize > PageSize)
377+
if (offset == 0 || offset + allocatedSize > PageSize)
370378
break;
371379

372-
// Skip null, closed/sealed, and tombstoned records (tombstoned records were already
373-
// disposed with DisposeReason.Deleted at the delete site)
380+
// Skip null, closed/sealed (SkipOnScan = Invalid | Sealed), and tombstoned records. Tombstoned records
381+
// were already disposed with DisposeReason.Deleted at the delete site; SkipOnScan records either copied
382+
// their heap contribution forward (CopyUpdater) and had it netted at the copy site, or are otherwise
383+
// not responsible for the allocator's heap counter.
374384
if (logRecord.Info.IsNull || logRecord.Info.SkipOnScan || logRecord.Info.Tombstone)
375385
{
376386
address += allocatedSize;
377387
continue;
378388
}
379389

380390
// Notify the application that this record is being evicted from memory.
381-
storeFunctions.OnEvict(ref logRecord);
391+
storeFunctions.OnEvict(ref logRecord, source);
382392

383393
address += allocatedSize;
384394
}

libs/storage/Tsavorite/cs/src/core/Allocator/SpanByteAllocator.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,6 @@ public readonly RecordSizeInfo GetDeleteRecordSize<TKey>(TKey key)
145145
public void OnDispose(ref DiskLogRecord logRecord, DisposeReason disposeReason) => _this.OnDispose(ref logRecord, disposeReason);
146146

147147
/// <inheritdoc/>
148-
public void EvictRecordsInRange(long startAddress, long endAddress) { }
148+
public void EvictRecordsInRange(long startAddress, long endAddress, EvictionSource source) { }
149149
}
150150
}

libs/storage/Tsavorite/cs/src/core/Allocator/SpanByteAllocatorImpl.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ internal sealed unsafe class SpanByteAllocatorImpl<TStoreFunctions> : AllocatorB
1515
private OverflowPool<PageUnit<Empty>> freePagePool;
1616

1717
public SpanByteAllocatorImpl(AllocatorSettings settings, TStoreFunctions storeFunctions, Func<object, SpanByteAllocator<TStoreFunctions>> wrapperCreator)
18-
: base(settings.LogSettings, storeFunctions, wrapperCreator, settings.evictCallback, settings.epoch, settings.flushCallback, settings.logger)
18+
: base(settings, storeFunctions, wrapperCreator, settings.logger)
1919
{
2020
freePagePool = new OverflowPool<PageUnit<Empty>>(4, p => { });
2121
pageHeaderSize = PageHeader.Size;

0 commit comments

Comments
 (0)