Skip to content
Merged
Show file tree
Hide file tree
Changes from 69 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
8c0277f
add scanlM/scanrM
cmlsharp Dec 31, 2025
1bc0061
add Array.scan{l,r,LM,RM}
cmlsharp Dec 31, 2025
2bab1b2
update scanl/scanr to be in terms of scanlM/scanrM
cmlsharp Dec 31, 2025
e449b1d
update scanl/scanr to be in terms of scanlM/scanrM
cmlsharp Dec 31, 2025
0f02ec2
formatting + update proofs to work with scan-list changes
cmlsharp Dec 31, 2025
883cf13
remove unnecessary comment + do
cmlsharp Dec 31, 2025
1433ecb
remove TODO comment
cmlsharp Dec 31, 2025
dd015f6
remove unnecesary lemma from simp_all
cmlsharp Dec 31, 2025
5687b0c
a few further simplifications
cmlsharp Dec 31, 2025
3a16ca1
address lints
cmlsharp Dec 31, 2025
b48bc64
Merge branch 'scan-list' into scan-array
cmlsharp Dec 31, 2025
fd7e218
fix some lint suggestions
cmlsharp Dec 31, 2025
4503c35
add basic tests
cmlsharp Dec 31, 2025
ba7db05
require LawfulMonad m for scanlM and use csimp for tail rec
cmlsharp Jan 3, 2026
f6cf53e
Merge branch 'scan-list' into scan-array
cmlsharp Jan 3, 2026
67d7e93
remove * from toList_drop
cmlsharp Jan 3, 2026
d912d0f
Merge branch 'main' into scan-array
cmlsharp Jan 7, 2026
e839054
style fixes
cmlsharp Jan 7, 2026
daccff9
more formatting changes
cmlsharp Jan 7, 2026
dd98786
Merge branch 'main' into scan-array
cmlsharp Jan 7, 2026
1bbec43
fix scanrM proof error
cmlsharp Jan 7, 2026
db9d551
remove some simp annotations
cmlsharp Jan 12, 2026
e3521d6
remove @[expose]
cmlsharp Jan 12, 2026
89305f1
fix style
cmlsharp Jan 25, 2026
95633a6
more style fixes
cmlsharp Jan 25, 2026
7edf5b7
remove unintentional changes
cmlsharp Jan 26, 2026
8bcc79d
remove trailing newline
cmlsharp Jan 26, 2026
7c7da5d
revert unintentional change
cmlsharp Jan 26, 2026
4d195f2
remove more empty lines
cmlsharp Jan 26, 2026
191ca86
fix indentation
cmlsharp Jan 26, 2026
80f14bc
Update Batteries/Data/Array/Basic.lean
cmlsharp Jan 29, 2026
e048788
Apply suggestions from code review
cmlsharp Jan 29, 2026
98bb5ab
more suggested style fixes
cmlsharp Jan 29, 2026
9ed1f16
fix comment
cmlsharp Jan 29, 2026
128f5b9
trailing whitespace
cmlsharp Jan 29, 2026
11b7ec3
Merge remote-tracking branch 'upstream/main' into scan-array
cmlsharp Jan 31, 2026
878baac
rename size_fits_usize
cmlsharp Jan 31, 2026
6c83c0c
make edits per code review
cmlsharp Feb 2, 2026
1e6813d
add another TODO
cmlsharp Feb 2, 2026
6fcaca6
chore: cleanup
fgdorais Feb 2, 2026
00325a8
chore: add tests
fgdorais Feb 2, 2026
35a9fc0
chore: cleanup
fgdorais Feb 2, 2026
2c28ac4
fix: look for last bot message when deduplicating Zulip posts (#1632)
kim-em Jan 28, 2026
9753b95
feat: linter accepts multiple modules to lint (#1619)
GZGavinZhao Jan 28, 2026
42544d8
chore: restore AsciiCasing API (#1635)
Ruben-VandeVelde Jan 30, 2026
c0e02a2
Merge branch 'main' into scan-array
fgdorais Feb 2, 2026
be12368
Merge remote-tracking branch 'origin/main' into scan-array
fgdorais Feb 2, 2026
83318fa
chore: fix import
fgdorais Feb 2, 2026
8876c24
code review suggestions
cmlsharp Feb 3, 2026
b283d6b
Apply suggestions from code review
fgdorais Feb 3, 2026
027e85d
Apply suggestion from @eric-wieser
fgdorais Feb 3, 2026
5248e55
Apply suggestion from @fgdorais
fgdorais Feb 3, 2026
c8645eb
fix unsafe_sizeFitsUSize
cmlsharp Feb 3, 2026
de51a6e
add simp annotations
cmlsharp Feb 3, 2026
dc661af
add toArray lemmas
cmlsharp Feb 3, 2026
d7487bf
Spacing
cmlsharp Feb 3, 2026
bf7ab54
toArray_scanl and toArray_scanr
cmlsharp Feb 3, 2026
472ec6d
add simp to subarray
cmlsharp Feb 3, 2026
1f0fdf5
more renaming
cmlsharp Feb 3, 2026
4f1c05b
line length
cmlsharp Feb 3, 2026
c4f611b
simplify proofS
cmlsharp Feb 3, 2026
5f4465c
simplifications
cmlsharp Feb 3, 2026
7c88448
flip equality direction
cmlsharp Feb 3, 2026
adfe908
rename toArray_scan_toList
cmlsharp Feb 3, 2026
6e96f0a
code review comments
cmlsharp Feb 3, 2026
d77c298
fix List lemma name
cmlsharp Feb 3, 2026
0a93369
fix unnecessary lemma
cmlsharp Feb 3, 2026
7d2329b
simplify
cmlsharp Feb 3, 2026
044b170
extra semicolon
cmlsharp Feb 3, 2026
f86a688
add missing extract lemmas for scanl scanr
cmlsharp Feb 3, 2026
3d8b8ed
code review
cmlsharp Feb 3, 2026
7e344f6
_extract whitespace
cmlsharp Feb 4, 2026
845c803
make reverse_singleton simp
cmlsharp Feb 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Batteries/Data/Array.lean
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ public import Batteries.Data.Array.Match
public import Batteries.Data.Array.Merge
public import Batteries.Data.Array.Monadic
public import Batteries.Data.Array.Pairwise
public import Batteries.Data.Array.Scan
277 changes: 264 additions & 13 deletions Batteries/Data/Array/Basic.lean
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Released under Apache 2.0 license as described in the file LICENSE.
Authors: Arthur Paulino, Floris van Doorn, Jannis Limperg
-/
module
import Batteries.Tactic.Alias

@[expose] public section

Expand Down Expand Up @@ -120,27 +121,277 @@ protected def maxI [ord : Ord α] [Inhabited α]
(xs : Array α) (start := 0) (stop := xs.size) : α :=
xs.minI (ord := ord.opposite) start stop

/-!
### Safe Nat Indexed Array functions
The functions in this section offer variants of Array functions which use `Nat` indices
instead of `Fin` indices. All these functions have as parameter a proof that the index is
valid for the array. But this parameter has a default argument `by get_elem_tactic` which
should prove the index bound.
@[deprecated set (since := "2026-02-02")]
alias setN := set

/-
This is guaranteed by the Array docs but it is unprovable.
May be asserted to be true in an unsafe context via `Array.unsafe_sizeFitsUsize`
-/
private abbrev SizeFitsUSize (a : Array α) : Prop := a.size < USize.size

@[grind .]
private theorem SizeFitsUSize.toNat_ofNat_eq {n : Nat} {a : Array α}
(h : a.SizeFitsUSize) (hn : n ≤ a.size) :
(USize.ofNat n).toNat = n :=
USize.toNat_ofNat_of_lt' (Nat.lt_of_le_of_lt ‹_› ‹_›)

/-
This is guaranteed by the Array docs but it is unprovable.
Can be used in unsafe functions to write more efficient implementations
that avoid arbitrary precision integer arithmetic.
-/
private unsafe def unsafe_sizeFitsUSize (a : Array α) : SizeFitsUSize a :=
lcProof

@[inline]
private def scanlMFast [Monad m] (f : β → α → m β) (init : β) (as : Array α)
(start := 0) (stop := as.size) : m (Array β) :=
let stop := min stop as.size
let start := min start as.size
loop f init as
(start := USize.ofNat start) (stop := USize.ofNat stop)
(h_stop := by grind only [USize.size_eq, USize.ofNat_eq_iff_mod_eq_toNat, = Nat.min_def])
(acc := Array.mkEmpty <| stop - start + 1)
where
@[specialize]
loop (f : β → α → m β) (init: β) (as: Array α) (start stop : USize)
(h_stop : stop.toNat ≤ as.size) (acc : Array β) : m (Array β) := do
if h_lt: start < stop then
let next ← f init (as.uget start <| Nat.lt_of_lt_of_le h_lt h_stop)
loop f next as (start + 1) stop h_stop (acc.push init)
else
pure <| acc.push init
termination_by stop.toNat - min start.toNat stop.toNat
decreasing_by
have : start < (start + 1) := by grind only [USize.size_eq]
grind only [Nat.min_def, USize.lt_iff_toNat_lt]

/--
`setN a i h x` sets an element in a vector using a Nat index which is provably valid.
A proof by `get_elem_tactic` is provided as a default argument for `h`.
This will perform the update destructively provided that `a` has a reference count of 1 when called.
Folds a monadic function over an array from the left, accumulating the partial results starting with
`init`. The accumulated value is combined with the each element of the list in order, using `f`.

The optional parameters `start` and `stop` control the region of the array to be folded. Folding
proceeds from `start` (inclusive) to `stop` (exclusive), so no folding occurs unless `start < stop`.
By default, the entire array is folded.

Examples:
```lean example
example [Monad m] (f : α → β → m α) :
Array.scanlM f x₀ #[a, b, c] = (do
let x₁ ← f x₀ a
let x₂ ← f x₁ b
let x₃ ← f x₂ c
pure #[x₀, x₁, x₂, x₃])
:= by simp [scanlM, scanlM.loop]
```

```lean example
example [Monad m] (f : α → β → m α) :
Array.scanlM f x₀ #[a, b, c] (start := 1) (stop := 3) = (do
let x₁ ← f x₀ b
let x₂ ← f x₁ c
pure #[x₀, x₁, x₂])
:= by simp [scanlM, scanlM.loop]
```
-/
abbrev setN (a : Array α) (i : Nat) (x : α) (h : i < a.size := by get_elem_tactic) : Array α :=
a.set i x
@[implemented_by scanlMFast]
def scanlM [Monad m] (f : β → α → m β) (init : β) (as : Array α) (start := 0)
(stop := as.size) : m (Array β) :=
loop f init as (min start as.size) (min stop as.size) (Nat.min_le_right _ _) #[]
where
/-- auxiliary tail-recursive function for scanlM -/
loop (f : β → α → m β) (init : β ) (as : Array α) (start stop : Nat)
(h_stop : stop ≤ as.size) (acc : Array β) : m (Array β) := do
if h_lt : start < stop then
loop f (← f init as[start]) as (start + 1) stop h_stop (acc.push init)
else
pure <| acc.push init

end Array
private theorem scanlM_loop_eq_scanlMFast_loop [Monad m]
{f : β → α → m β} {init : β} {as : Array α} {h_size : as.SizeFitsUSize}
{start stop : Nat} {h_start : start ≤ as.size}
{h_stop : stop ≤ as.size} {acc : Array β} :
scanlM.loop f init as start stop h_stop acc
= scanlMFast.loop f init as (USize.ofNat start) (USize.ofNat stop)
(by rw [USize.toNat_ofNat_of_lt' (Nat.lt_of_le_of_lt h_stop h_size)]; exact h_stop) acc := by
generalize h_n : stop - start = n
induction n using Nat.strongRecOn generalizing start acc init
rename_i n ih
rw [scanlM.loop, scanlMFast.loop]
have h_stop_usize := h_size.toNat_ofNat_eq h_stop
have h_start_usize := h_size.toNat_ofNat_eq h_start
split
case isTrue h_lt =>
simp_all only [USize.toNat_ofNat', ↓reduceDIte, uget,
show USize.ofNat start < USize.ofNat stop by simp_all [USize.lt_iff_toNat_lt]]
apply bind_congr
intro next
have h_start_succ : USize.ofNat start + 1 = USize.ofNat (start + 1) := by
simp_all only [← USize.toNat_inj, USize.toNat_add]
grind only [USize.size_eq, SizeFitsUSize.toNat_ofNat_eq]
rw [h_start_succ]
apply ih (stop - (start + 1)) <;> omega
case isFalse h_nlt => grind [USize.lt_iff_toNat_lt]

-- this theorem establishes that given the (unprovable) assumption that as.size < USize.size,
-- the scanlMFast and scanlM are equivalent
-- TODO (cmlsharp): prova an analogous theorem for scanrM
private theorem scanlM_eq_scanlMFast [Monad m]
{f : β → α → m β} {init : β} {as : Array α}
{h_size : as.SizeFitsUSize} {start stop : Nat} :
scanlM f init as start stop = scanlMFast f init as start stop := by
unfold scanlM scanlMFast
apply scanlM_loop_eq_scanlMFast_loop
simp_all only [gt_iff_lt]
apply Nat.min_le_right

@[inline]
private def scanrMFast [Monad m] (f : α → β → m β) (init : β) (as : Array α)
(h_size : as.SizeFitsUSize) (start := as.size) (stop := 0) : m (Array β) :=
let start := min start as.size
let stop := min stop start
loop f init as
(start := USize.ofNat start) (stop := USize.ofNat stop)
(h_start := by grind only [USize.size_eq, USize.ofNat_eq_iff_mod_eq_toNat, = Nat.min_def])
(acc := Array.replicate (start - stop + 1) init)
(by grind only [!Array.size_replicate, = Nat.min_def, SizeFitsUSize.toNat_ofNat_eq])
where
@[specialize]
loop (f : α → β → m β) (init : β) (as : Array α)
(start stop : USize)
(h_start : start.toNat ≤ as.size)
(acc : Array β)
(h_bound : start.toNat - stop.toNat < acc.size) :
m (Array β) := do
if h_gt : stop < start then
let startM1 := start - 1
have : startM1 < start := by grind only [!USize.sub_add_cancel, USize.lt_iff_le_and_ne,
USize.lt_add_one, USize.le_zero_iff]
have : startM1.toNat < as.size := Nat.lt_of_lt_of_le ‹_› ‹_›
have : (startM1 - stop) < (start - stop) := by grind only
[!USize.sub_add_cancel, USize.sub_right_inj, USize.add_comm, USize.lt_add_one,
USize.add_assoc, USize.add_right_inj]
let next ← f (as.uget startM1 ‹_›) init
loop f next as
(start := startM1)
(stop := stop)
(h_start := Nat.le_of_succ_le_succ (Nat.le_succ_of_le ‹_›))
(acc := acc.uset (startM1 - stop) next
(by grind only [USize.toNat_sub_of_le, USize.le_of_lt, USize.lt_iff_toNat_lt]))
(h_bound :=
(by grind only [USize.toNat_sub_of_le, = uset_eq_set, = size_set, USize.size_eq]))
else
pure acc
termination_by start.toNat - stop.toNat
decreasing_by
grind only [USize.lt_iff_toNat_lt, USize.toNat_sub,
USize.toNat_sub_of_le, USize.le_iff_toNat_le]

@[inline]
private unsafe def scanrMUnsafe [Monad m] (f : α → β → m β) (init : β) (as : Array α)
(start := as.size) (stop := 0) : m (Array β) :=
scanrMFast (h_size := as.unsafe_sizeFitsUSize) f init as (start := start) (stop := stop)

/--
Folds a monadic function over an array from the right, accumulating the partial results starting
with `init`. The accumulated value is combined with the each element of the list in order using `f`.

The optional parameters `start` and `stop` control the region of the array to be folded. Folding
proceeds from `start` (exclusive) to `stop` (inclusive), so no folding occurs unless `start > stop`.
By default, the entire array is folded.

Examples:
```lean example
example [Monad m] (f : α → β → m β) :
Array.scanrM f x₀ #[a, b, c] = (do
let x₁ ← f c x₀
let x₂ ← f b x₁
let x₃ ← f a x₂
pure #[x₃, x₂, x₁, x₀])
:= by simp [scanrM, scanrM.loop]
```

```lean example
example [Monad m] (f : α → β → m β) :
Array.scanrM f x₀ #[a, b, c] (start := 3) (stop := 1) = (do
let x₁ ← f c x₀
let x₂ ← f b x₁
pure #[x₂, x₁, x₀])
:= by simp [scanrM, scanrM.loop]
```
-/
@[implemented_by scanrMUnsafe]
def scanrM [Monad m]
(f : α → β → m β) (init : β) (as : Array α) (start := as.size) (stop := 0) : m (Array β) :=
let start := min start as.size
loop f init as start stop (Nat.min_le_right _ _) #[]
where
/-- auxiliary tail-recursive function for scanrM -/
loop (f : α → β → m β) (init : β) (as : Array α)
(start stop : Nat)
(h_start : start ≤ as.size)
(acc : Array β) :
m (Array β) := do
if h_gt : stop < start then
let i := start - 1
let next ← f as[i] init
loop f next as i stop (by omega) (acc.push init)
else
pure <| acc.push init |>.reverse

/--
Fold a function `f` over the array from the left, returning the array of partial results.
```
scanl (· + ·) 0 #[1, 2, 3] = #[0, 1, 3, 6]
```
-/
@[inline]
def scanl (f : β → α → β) (init : β) (as : Array α) (start := 0) (stop := as.size) : Array β :=
Id.run <| as.scanlM (pure <| f · ·) init start stop

/--
Fold a function `f` over the array from the right, returning the array of partial results.
```
scanr (· + ·) 0 #[1, 2, 3] = #[6, 5, 3, 0]
```
-/
@[inline]
def scanr (f : α → β → β) (init : β) (as : Array α) (start := as.size) (stop := 0) : Array β :=
Id.run <| as.scanrM (pure <| f · ·) init start stop

end Array

namespace Subarray

/--
Fold a monadic function `f` over the subarray from the left, returning the list of partial results.
-/
@[inline]
def scanlM [Monad m] (f : β → α → m β) (init : β) (as : Subarray α) : m (Array β) :=
as.array.scanlM f init (start := as.start) (stop := as.stop)

/--
Fold a monadic function `f` over the subarray from the right, returning the list of partial results.
-/
@[inline]
def scanrM [Monad m] (f : α → β → m β) (init : β) (as : Subarray α) : m (Array β) :=
as.array.scanrM f init (start := as.start) (stop := as.stop)

/--
Fold a function `f` over the subarray from the left, returning the list of partial results.
-/
@[inline]
def scanl (f : β → α → β) (init : β) (as : Subarray α) : Array β :=
as.array.scanl f init (start := as.start) (stop := as.stop)

/--
Fold a function `f` over the subarray from the right, returning the list of partial results.
-/
@[inline]
def scanr (f : α → β → β) (init : β) (as : Subarray α) : Array β :=
as.array.scanr f init (start := as.start) (stop := as.stop)

/--
Check whether a subarray is empty.
-/
Expand All @@ -149,7 +400,7 @@ def isEmpty (as : Subarray α) : Bool :=
as.start == as.stop

/--
Check whether a subarray contains an element.
Check whether a subarray contains a given element.
-/
@[inline]
def contains [BEq α] (as : Subarray α) (a : α) : Bool :=
Expand Down
4 changes: 4 additions & 0 deletions Batteries/Data/Array/Lemmas.lean
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ theorem idxOf?_toList [BEq α] {a : α} {l : Array α} :
(a.eraseIdxIfInBounds i).size = if i < a.size then a.size-1 else a.size := by
grind

theorem toList_drop (as: Array α) (n : Nat) :
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
theorem toList_drop (as: Array α) (n : Nat) :
@[simp, grind =]
theorem toList_drop (as: Array α) (n : Nat) :

to match toList_erase above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work as a simp lemma because simp simplifies '(as.drop n).toList' to 'List.take (as.size - n) (List.drop n as.toList)`

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fwiw I think this is very frustrating. Array.drop is automatically simplified to Array.extract. This over-eager simplification is why I wanted a toList_drop method to begin with.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are way too many simp lemmas for my taste. Eric is very fond of them though and I respect his opinion. We (in general, including Eric and I) need a common language to talk about simp-normal-form. The Mathlib/Batteries concerns are a bit different.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for starting the Zulip thread; indeed this can't be simp right now.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we resolve this or are we hoping to fix the simp issues upstream?

(as.drop n).toList = as.toList.drop n := by
simp only [drop, toList_extract, size_eq_length_toList, List.drop_eq_extract]

/-! ### set -/

theorem size_set! (a : Array α) (i v) : (a.set! i v).size = a.size := by simp
Expand Down
Loading