How Lance Converts Logical Rows to Row Addresses

The take method selects rows by logical index, and the core conversion is performed by row_offsets_to_row_addresses(). The function follows a five‑step pipeline: global logical row number → locate the owning Fragment → subtract the visible rows of preceding Fragments → apply the Fragment's deletion vector → obtain the physical offset inside the Fragment → encode it as a RowAddress.

For illustration, assume a Dataset with two Fragments. The overall logical layout is shown in the first diagram. Executing dataset.take([1, 4, 5]) yields the following mappings:

1 → RowAddress(fragment_id=0, row_offset=2)
4 → RowAddress(fragment_id=1, row_offset=0)
5 → RowAddress(fragment_id=1, row_offset=2)

If a logical row index exceeds the Dataset's visible row count, the conversion produces a special TOMBSTONE_ROW address, which triggers an out‑of‑bounds error during the read phase. Note that both logical indices and resulting row addresses are tied to the current Dataset version; after deletions, appends, or compaction the mapping may change, so they should not be used as stable business keys.

The visible row count of each Fragment is computed by subtracting the number of deleted rows (recorded in the deletion vector) from the Fragment's total physical rows. For example, a Fragment with 100 physical records and 10 deletions has 90 visible rows.

The conversion proceeds in two steps: count_rows(None) uses the visible row count to determine which Fragment a given logical row belongs to.

The OffsetMapper consults the deletion vector to translate the logical offset within the Fragment to a physical offset.

Thus the deletion information is consulted twice but serves different purposes: count_rows() calculates how many global logical positions a Fragment occupies, while OffsetMapper maps a logical position to an actual, non‑deleted physical row.