Skip to content

Commit c17e13f

Browse files
claudioemmanuelclaudioemmanuel
authored
feat(handlers): xcodebuild noise filter + log-file tail detection (#84)
Driven by an iOS development session where xcodebuild was the top bash command (~18 invocations) and squeez was routing it through the generic BuildHandler with only Gradle/Maven filters. Xcode-specific action lines (CompileSwift, SwiftEmitModule, Ld, CodeSign, etc.) plus build-system preamble and invocation echoes all survived compression. Changes - BuildHandler now detects xcodebuild and drops 30+ Xcode-specific action prefixes and continuation patterns while preserving the terminal `** BUILD SUCCEEDED/FAILED **` markers and Swift diagnostic lines. - FsHandler skips `group_files_by_dir` for viewer commands (cat/head/ tail/less/more/bat). The grouping step was collapsing non-path file content into a single `./ N modified` summary — a latent bug that masqueraded as extreme compression on markdown/log fixtures. - FsHandler uses `Keep::Tail` for `tail` (always) and for `cat`/`less`/ `more`/`bat` on `.log`/`.out`/`.err` paths. Recent events matter more than initial noise for log viewing. Benchmarks - New fixture `bench/fixtures/xcodebuild_build.txt`: 1,881 tk -> 17 tk (~99% reduction). All 15 filter-mode fixtures pass. - Efficiency proof: all 5 floors still pass. `sig_mode_delta_vs_pipeline` floor lowered from 30% to 10% with a comment explaining the adjustment — the previous floor was measured against the grouping bug above. - Aggregate benchmark: 91.6% reduction (152,961 tk -> 12,886 tk), within normal variance of the prior 91.9%. README block refreshed. Test flake fixes (pre-existing, unrelated to handlers) - `test_init_finalizes_prior_session_to_memory` and the four structured- memory tests in `test_memory_structured.rs` hardcoded a 2026-03-22 session timestamp that fell outside the default 30-day retention window once wall-clock time drifted past 2026-04-21. Switched both suites to `unix_now() - 86_400` and derive the date string from the same timestamp so the tests remain valid indefinitely.
1 parent b6ff569 commit c17e13f

7 files changed

Lines changed: 296 additions & 20 deletions

File tree

‎bench/fixtures/xcodebuild_build.txt‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
Command line invocation:
2+
/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild build -scheme MyApp -destination "platform=iOS Simulator,name=iPhone 15"
3+
4+
Build settings from command line:
5+
ARCHS = arm64
6+
7+
note: Using new build system
8+
note: Using codesigning identity override: -
9+
note: Build preparation complete
10+
note: Planning
11+
Analyze workspace
12+
Create build description
13+
Build description signature: 7f3e9a2c1b8d4f5e6a9b0c3d2e1f4a5b
14+
Build description path: /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/XCBuildData/7f3e9a2c1b8d4f5e6a9b0c3d2e1f4a5b.xcbuilddata
15+
16+
ComputePackagePrebuildTargetDependencyGraph
17+
SendPIFToBuildSystem
18+
CreateBuildRequest
19+
20+
ClangStatCache /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/SDKStatCaches.noindex/iphonesimulator17.5-21F77-abc.sdkstatcache
21+
RegisterExecutionPolicyException /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app/MyApp
22+
MkDir /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app (in target 'MyApp' from project 'MyApp')
23+
cd /Users/dev/projects/MyApp
24+
builtin-mkdir /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app
25+
26+
WriteAuxiliaryFile /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/MyApp.build/Debug-iphonesimulator/MyApp.build/Objects-normal/arm64/MyApp.SwiftFileList (in target 'MyApp' from project 'MyApp')
27+
WriteAuxiliaryFile /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/MyApp.build/Debug-iphonesimulator/MyApp.build/Objects-normal/arm64/MyApp.swiftmodule.DependencyMetadataFileListPath (in target 'MyApp' from project 'MyApp')
28+
WriteAuxiliaryFile /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/MyApp.build/Debug-iphonesimulator/MyApp.build/Objects-normal/arm64/MyApp-OutputFileMap.json (in target 'MyApp' from project 'MyApp')
29+
30+
ProcessInfoPlistFile /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app/Info.plist /Users/dev/projects/MyApp/MyApp/Info.plist (in target 'MyApp' from project 'MyApp')
31+
cd /Users/dev/projects/MyApp
32+
builtin-infoPlistUtility /Users/dev/projects/MyApp/MyApp/Info.plist -producttype com.apple.product-type.application -expandbuildsettings -format binary -platform iphonesimulator -o /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app/Info.plist
33+
34+
SwiftEmitModule normal arm64 Emitting\ module\ for\ MyApp (in target 'MyApp' from project 'MyApp')
35+
cd /Users/dev/projects/MyApp
36+
builtin-swiftTaskExecution -- /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -emit-module
37+
38+
SwiftDriverJobDiscovery normal arm64 Compiling ContentView.swift (in target 'MyApp' from project 'MyApp')
39+
SwiftDriverJobDiscovery normal arm64 Compiling MyAppApp.swift (in target 'MyApp' from project 'MyApp')
40+
SwiftDriverJobDiscovery normal arm64 Compiling Signpost.swift (in target 'MyApp' from project 'MyApp')
41+
SwiftDriverJobDiscovery normal arm64 Compiling WhistleFormLabel.swift (in target 'MyApp' from project 'MyApp')
42+
SwiftDriverJobDiscovery normal arm64 Compiling ImageCache.swift (in target 'MyApp' from project 'MyApp')
43+
44+
SwiftCompile normal arm64 Compiling\ 'ContentView.swift' /Users/dev/projects/MyApp/MyApp/ContentView.swift (in target 'MyApp' from project 'MyApp')
45+
cd /Users/dev/projects/MyApp
46+
builtin-swiftTaskExecution -- /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -c /Users/dev/projects/MyApp/MyApp/ContentView.swift
47+
48+
SwiftCompile normal arm64 Compiling\ 'MyAppApp.swift' /Users/dev/projects/MyApp/MyApp/MyAppApp.swift (in target 'MyApp' from project 'MyApp')
49+
cd /Users/dev/projects/MyApp
50+
builtin-swiftTaskExecution -- /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -c /Users/dev/projects/MyApp/MyApp/MyAppApp.swift
51+
52+
SwiftMergeGeneratedHeaders /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/MyApp.build/Debug-iphonesimulator/MyApp.build/Objects-normal/arm64/MyApp-Swift.h /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/MyApp.build/Debug-iphonesimulator/MyApp.build/Objects-normal/arm64/MyApp-swiftinterface.h (in target 'MyApp' from project 'MyApp')
53+
54+
CompileAssetCatalog /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app /Users/dev/projects/MyApp/MyApp/Assets.xcassets (in target 'MyApp' from project 'MyApp')
55+
cd /Users/dev/projects/MyApp
56+
/Applications/Xcode.app/Contents/Developer/usr/bin/actool --output-format human-readable-text --notices --warnings --export-dependency-info
57+
58+
Ld /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app/MyApp normal (in target 'MyApp' from project 'MyApp')
59+
cd /Users/dev/projects/MyApp
60+
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -target arm64-apple-ios17.5-simulator -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator17.5.sdk -L/Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator -F/Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator -filelist /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/MyApp.build/Debug-iphonesimulator/MyApp.build/Objects-normal/arm64/MyApp.LinkFileList -Xlinker -rpath -Xlinker @executable_path/Frameworks -Xlinker -object_path_lto -Xlinker /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/MyApp.build/Debug-iphonesimulator/MyApp.build/Objects-normal/arm64/MyApp_lto.o -Xlinker -export_dynamic -Xlinker -no_deduplicate -Xlinker -objc_abi_version -Xlinker 2 -o /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app/MyApp
61+
62+
GenerateDSYMFile /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app.dSYM /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app/MyApp (in target 'MyApp' from project 'MyApp')
63+
64+
Touch /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app (in target 'MyApp' from project 'MyApp')
65+
66+
RegisterWithLaunchServices /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app (in target 'MyApp' from project 'MyApp')
67+
68+
CodeSign /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app (in target 'MyApp' from project 'MyApp')
69+
cd /Users/dev/projects/MyApp
70+
71+
Signing Identity: "-"
72+
73+
/usr/bin/codesign --force --sign - --entitlements /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Intermediates.noindex/MyApp.build/Debug-iphonesimulator/MyApp.build/MyApp.app.xcent --timestamp=none --generate-entitlement-der /Users/dev/Library/Developer/Xcode/DerivedData/MyApp-abc123/Build/Products/Debug-iphonesimulator/MyApp.app
74+
75+
** BUILD SUCCEEDED **

‎bench/report.md‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,19 @@
11
FIXTURE BEFORE AFTER REDUCTION LATENCY STATUS
22
──────────────────────────────────────────────────────────────────────────────
33
docker_logs.txt 665tk 186tk 73% 5ms ✅
4-
env_dump.txt 441tk 287tk 35% 5ms ✅
4+
env_dump.txt 441tk 287tk 35% 4ms ✅
55
find_deep.txt 424tk 134tk 69% 4ms ✅
6-
git_copilot_session.txt 639tk 421tk 35% 5ms ✅
7-
git_diff.txt 502tk 317tk 37% 4ms ✅
6+
git_copilot_session.txt 639tk 421tk 35% 4ms ✅
7+
git_diff.txt 502tk 317tk 37% 3ms ✅
88
git_log_200.txt 2667tk 819tk 70% 4ms ✅
9-
git_status.txt 50tk 16tk 68% 4ms ✅
9+
git_status.txt 50tk 16tk 68% 3ms ✅
1010
intensity_budget80.txt 4418tk 52tk 99% 4ms ✅
11-
ls_la.txt 1782tk 886tk 51% 4ms ✅
12-
mdcompress_claude_md.txt 316tk 246tk 23% 3ms ✅
13-
mdcompress_prose.txt 187tk 138tk 27% 3ms ✅
11+
ls_la.txt 1782tk 886tk 51% 3ms ✅
12+
mdcompress_claude_md.txt 316tk 246tk 23% 4ms ✅
13+
mdcompress_prose.txt 187tk 138tk 27% 4ms ✅
1414
npm_install.txt 524tk 231tk 56% 3ms ✅
1515
ps_aux.txt 40373tk 2352tk 95% 6ms ✅
1616
summarize_huge.txt 82257tk 47tk 100% 13ms ✅
17+
xcodebuild_build.txt 1881tk 17tk 100% 4ms ✅
1718

18-
PASS: 14/14 FAIL: 0/14
19+
PASS: 15/15 FAIL: 0/15

‎src/commands/benchmark.rs‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -569,12 +569,18 @@ pub fn run_efficiency_proof() -> Vec<EfficiencyResult> {
569569
// Prove sig-mode itself is pulling its weight: measure the ADDITIONAL
570570
// reduction sig-mode delivers on top of the regular (non-sig-mode)
571571
// FsHandler pipeline. The full pipeline already compresses via
572-
// smart_filter + grouping + truncation even with sig_mode off — a naive
572+
// smart_filter + truncation even with sig_mode off — a naive
573573
// "sig_mode_off vs raw" control conflates sig-mode's savings with those
574574
// ambient wins. Here baseline = pipeline-without-sig-mode output, and
575575
// compressed = pipeline-with-sig-mode output; reduction_pct is the
576576
// incremental % removed by sig-mode alone.
577-
// FLOOR: 30.0 — empirical delta on a 1000-line Rust file is ~40-55%.
577+
//
578+
// FLOOR: 10.0 — sig-mode vs raw saves ~80% (see sig_mode_rust_1000), but
579+
// vs the non-sig pipeline the marginal win is smaller because the
580+
// non-sig path already truncates to the first 50 lines. Viewer commands
581+
// (cat/tail/…) correctly skip grouping since lines are file CONTENT,
582+
// not a file list; this makes the baseline smaller, which in turn
583+
// narrows the delta vs sig-mode. Measured delta: ~15%.
578584
{
579585
let content = make_large_rust_source();
580586
let lines: Vec<String> = content.lines().map(|l| l.to_string()).collect();
@@ -592,7 +598,7 @@ pub fn run_efficiency_proof() -> Vec<EfficiencyResult> {
592598
let compressed_tokens = out_on.join("\n").len() / 4;
593599

594600
let reduction = reduction_pct(baseline_tokens, compressed_tokens);
595-
let floor = 30.0_f64;
601+
let floor = 10.0_f64;
596602
results.push(EfficiencyResult {
597603
label: "sig_mode_delta_vs_pipeline",
598604
feature: "US-001",

‎src/commands/build.rs‎

Lines changed: 70 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,17 +4,80 @@ use crate::strategies::{dedup, smart_filter, truncation};
44

55
pub struct BuildHandler;
66

7+
const GRADLE_NOISE_PREFIXES: &[&str] = &["> Task :", "Executing ", "Download "];
8+
9+
// xcodebuild action-line prefixes that are pure progress noise. Errors surface
10+
// on their own lines (e.g. `/path/File.swift:42:5: error: …`) and the
11+
// terminal `** BUILD FAILED **` / `The following build commands failed:` block
12+
// never matches these prefixes, so dropping them is safe.
13+
const XCODEBUILD_NOISE_PREFIXES: &[&str] = &[
14+
"ClangStatCache ",
15+
"RegisterExecutionPolicyException ",
16+
"RegisterWithLaunchServices ",
17+
"WriteAuxiliaryFile ",
18+
"MkDir ",
19+
"CreateBuildDirectory ",
20+
"ProcessInfoPlistFile ",
21+
"CpResource ",
22+
"CopySwiftLibs ",
23+
"PBXCp ",
24+
"SwiftEmitModule ",
25+
"SwiftDriverJobDiscovery ",
26+
"SwiftCompile ",
27+
"SwiftMergeGeneratedHeaders ",
28+
"CompileC ",
29+
"CompileAssetCatalog ",
30+
"CompileStoryboard ",
31+
"CompileXIB ",
32+
"LinkStoryboards ",
33+
"GenerateDSYMFile ",
34+
"Ld ",
35+
"CodeSign ",
36+
"Touch ",
37+
"Validate ",
38+
"ComputePackagePrebuildTargetDependencyGraph",
39+
"SendPIFToBuildSystem",
40+
"CreateBuildRequest",
41+
"Analyze workspace",
42+
"Create build description",
43+
"Build description signature:",
44+
"Build description path:",
45+
"Command line invocation:",
46+
"Build settings from command line:",
47+
" cd /",
48+
" builtin-",
49+
" /Applications/Xcode.app/",
50+
" /usr/bin/",
51+
" export ",
52+
];
53+
54+
const XCODEBUILD_NOISE_CONTAINS: &[&str] = &[
55+
"note: Using new build system",
56+
"note: Using codesigning identity override",
57+
"note: Build preparation complete",
58+
"note: Planning",
59+
];
60+
61+
fn is_xcodebuild(cmd: &str) -> bool {
62+
let first = cmd.split_whitespace().next().unwrap_or("");
63+
let base = first.rsplit('/').next().unwrap_or(first);
64+
base == "xcodebuild"
65+
}
66+
67+
fn is_xcode_noise(l: &str) -> bool {
68+
XCODEBUILD_NOISE_PREFIXES.iter().any(|p| l.starts_with(p))
69+
|| XCODEBUILD_NOISE_CONTAINS.iter().any(|p| l.contains(p))
70+
}
71+
772
impl Handler for BuildHandler {
8-
fn compress(&self, _cmd: &str, lines: Vec<String>, config: &Config) -> Vec<String> {
73+
fn compress(&self, cmd: &str, lines: Vec<String>, config: &Config) -> Vec<String> {
974
let lines = smart_filter::apply(lines);
75+
let xcode = is_xcodebuild(cmd);
1076
let filtered: Vec<String> = lines
1177
.into_iter()
12-
.filter(|l| {
13-
!l.starts_with("> Task :")
14-
&& !l.starts_with("Executing ")
15-
&& !l.starts_with("Download ")
16-
&& !l.trim().is_empty()
17-
})
78+
.filter(|l| !GRADLE_NOISE_PREFIXES.iter().any(|p| l.starts_with(p)))
79+
.filter(|l| !(xcode && is_xcode_noise(l)))
80+
.filter(|l| !l.trim().is_empty())
1881
.collect();
1982
let filtered = dedup::apply(filtered, config.dedup_min);
2083
truncation::apply(filtered, 100, truncation::Keep::Tail)

‎src/commands/fs.rs‎

Lines changed: 42 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -220,7 +220,47 @@ impl Handler for FsHandler {
220220
return truncation::apply(filtered, 80, truncation::Keep::Head);
221221
}
222222

223-
let lines = grouping::group_files_by_dir(lines, 5);
224-
truncation::apply(lines, config.find_max_results, truncation::Keep::Head)
223+
// Viewer commands (cat/head/tail/less/more/bat) emit file CONTENT,
224+
// not file LISTS — grouping by parent-dir would collapse every line
225+
// into a single "./ N modified" summary. Skip grouping for those.
226+
let is_viewer = base_cmd(cmd).map(|b| VIEWER_CMDS.contains(&b)).unwrap_or(false);
227+
let lines = if is_viewer {
228+
lines
229+
} else {
230+
grouping::group_files_by_dir(lines, 5)
231+
};
232+
let keep = if should_keep_tail(cmd) {
233+
truncation::Keep::Tail
234+
} else {
235+
truncation::Keep::Head
236+
};
237+
truncation::apply(lines, config.find_max_results, keep)
225238
}
226239
}
240+
241+
fn base_cmd(cmd: &str) -> Option<&str> {
242+
let first = cmd.split_whitespace().next()?;
243+
Some(first.rsplit('/').next().unwrap_or(first))
244+
}
245+
246+
/// `tail` is always tail-oriented. `cat`/`less`/`more`/`bat` on a log-ish
247+
/// path (.log/.out/.err) also benefit from tail — recent lines matter most.
248+
fn should_keep_tail(cmd: &str) -> bool {
249+
let Some(base) = base_cmd(cmd) else { return false };
250+
if base == "tail" {
251+
return true;
252+
}
253+
if !matches!(base, "cat" | "less" | "more" | "bat") {
254+
return false;
255+
}
256+
for tok in cmd.split_whitespace().skip(1) {
257+
if tok.starts_with('-') {
258+
continue;
259+
}
260+
let lower = tok.to_lowercase();
261+
if lower.ends_with(".log") || lower.ends_with(".out") || lower.ends_with(".err") {
262+
return true;
263+
}
264+
}
265+
false
266+
}

0 commit comments

Comments
 (0)