#269 compiler SIGSEGV compiling return-from-from-handler-case in a 2-binding let inside loop (blocks ANSI ansi-aux fixtures)

closed

Opened by atgreen

Symptom

Loading (compiling) this valid CL DEFUN segfaults funcl (build/funcl-stage2), a HEISENBUG (does NOT crash under gdb — memory-layout dependent):

(defun ctp4 (P TYPE)
  (loop for x in '(1 2 3)
      when (block failed
             (let ((p1 (handler-case (funcall P x)
                         (error () (return-from failed t))))
                   (p2 (handler-case (typep x TYPE)
                         (error () (return-from failed t)))))
               (when (or (and p1 (not p2)) (and (not p1) p2)) t)))
      collect x))

Reduction (all needed together; removing any one -> no crash)

the loop ... when (block failed ...) collect wrapper
a let with TWO bindings, each an init = (handler-case ... (error () (return-from failed t)))
p1 body (funcall P x) AND p2 body (typep x TYPE) (both funcall, or simplified final when, or 0/1 params, or no loop -> all OK)
the compound (when (or (and p1 (not p2)) (and (not p1) p2)) t) using both bindings

Single handler-case, no-loop, or (when (or p1 p2) t) all load fine.

Core backtrace

#0 AG-BV-COMPARE-WALK with A = a CONS (589185, lowtag 1 — gdb mislabels "fixnum"), B = "FUNCALL", I = 7284627 (huge/garbage). So a name-comparison (baked-index bsearch during symbol resolution) gets corrupted/wrong args -> byte-vector-byte on a cons/OOB index -> SIGSEGV. Args are corrupted, pointing at stack damage from the return-from/escapify codegen for this nesting.

Impact

This is the FIRST crashing form (check-type-predicate) in tests/ansi-test/auxiliary/ansi-aux.lsp. It crashes ansi-aux load -> the ANSI harness can't build its fixture image -> falls back to a ~60s/file source load (impractical). Fixing unblocks the fixture-dependent ANSI suite.

Likely area

cave#219 escapify (ag-escapify / ag-esc-catcher-form) + emit-ir-return-from stack-drop, for return-from escaping a handler-case frame from a let-binding init, with multiple bindings. The compound boolean using both bindings is required, suggesting a depth/stack-slot miscount that corrupts the value stack.

Comments (5)

atgreen3992722493

ROOT CAUSE FOUND (2026-07-10) — NOT a codegen/escapify/return-from bug. It is INFINITE MACRO RECURSION.

ansi-aux-macros.lsp redefines handler-case: (defmacro handler-case (form &rest cases) (let () (cl:handler-case ,form ,@cases))) In funcl, handler-caseandcl:handler-case` are the SAME symbol (CL-USER inherits CL; that is correct CL). So the macro body references the very macro being defined → ag-macro-expand-call-1 (src/cold/compiler.lisp:4601) applies the macro then unconditionally re-expands the result via ag-macro-expand-form with NO depth bound → forever. The compiler hangs, allocates until heap corruption, then SIGSEGVs (the AG-BV-COMPARE-WALK core was a downstream symptom of the corrupted heap, not the fault site). Heisenbug/segfault-vs-hang just depends on accumulated heap state.

Minimal repro (hangs → core, no block/let/return-from needed): (defmacro handler-case (form &rest cases) (list (quote let) (quote ()) (cons (quote cl:handler-case) (cons form cases)))) (defun foo () (handler-case (+ 1 2) (error () 9))) Controls: distinct macro name → OK; expand to a plain form (not cl:handler-case) → OK. So it is specifically the name-identity self-reference.

Why real SBCL/etc. are fine: the ansi-test suite loads into a CL-TEST package that SHADOWS handler-case/handler-bind (tests/ansi-test/cl-test-package.lsp:28-30), making CL-TEST::handler-case distinct from cl:handler-case. funcl's harness (tests/ansi-run.sh) loads ansi-aux directly into CL-USER with no shadowing. funcl's shadow does NOT yet create a distinct symbol (verified) — that is the ADR-0026 package-scoping gap (cave#266/#150).

TWO fixes, independent:

ROBUSTNESS (tractable, real bug): bound macro-expansion depth in ag-macro-expand-call-1 so a self-recursive macro signals a clean error instead of hanging→segfaulting. A conformant compiler must never corrupt the heap / SIGSEGV on user input. This lets the fixture image build survive (load recovers, defines remaining fixtures).
CORRECTNESS (the real ANSI unblock): package-scoped symbols so shadow works (ADR-0026 / cave#266). Without it, ansi-aux/handler-case-dependent tests cannot load as written.

Retitling implication: this is a macro-expander robustness + packages issue, not escapify. cave#148/#219 escapify are NOT implicated.

atgreen3992725378

Follow-up fb7d586: lowered the macro-expansion depth ceiling 10000→1000. The 1620284 bound stopped the SIGSEGV but 10000 made error-recovery pathologically slow — the counter also bounds NATIVE expander recursion, and the self-referential macro must unwind every pending frame on error (~15× in ansi-aux), which at depth 10000 took >1min and read as a hang → nondeterministic fixture-image build timeouts. Measured (ansi-aux first 342 lines): limit 300→2s, 1000→3s, 10000→60s+. Full ansi-aux image now builds in 13s (2.1MB). 1000 is ~10x above any real macro nesting; self-host + asdf/alexandria unaffected.

atgreen3992729021

CONVERGENCE (2026-07-10): the handler-case recursive-redefinition ALSO blocks the entire .ERROR. test family across ALL ANSI directories, not just ansi-aux load. signals-error / signals-type-error / check-type-error (ansi-aux) all expand through the fixture-redefined handler-case (defmacro handler-case ... (cl:handler-case ...)), which in funcl is self-recursive (handler-case == cl:handler-case, no package shadow) → against the fixture image every (signals-error … type-error) now raises "macro expansion nested too deeply" (the depth-bound from 1620284 — clean error instead of the old SIGSEGV, but still an error). So no .ERROR. test can pass in the harness regardless of whether the underlying function signals correctly.

Confirmed with a real crash-fix that STILL cant flip its test: e7fe8ca guards rplaca/rplacd (setcar on a non-cons was a heap-corruption CORE DUMP → now a proper type-error; verified (handler-case (rplaca 5 6) (type-error (c) (type-error-datum c))) => 5 on build/funcl-stage2 WITHOUT the fixture). But rplaca.error.1 still FAILs in the harness because check-type-error itself errors in the recursive-handler-case image.

So BOTH major remaining ANSI categories route through ADR-0026 package shadowing: (1) specials-rebinding tests via the reader/compiled symbol split (cave#266), (2) condition/.ERROR. tests via the handler-case recursive redefinition (needs CL-TEST to shadow handler-case/handler-bind so they are distinct from cl:handler-case — cl-test-package.lsp:28-30). funcl shadow does not yet make a distinct symbol. ADR-0026 single-canonical-symbol + real package shadowing is THE central ANSI blocker, confirmed from three independent angles.

atgreen3992737057

THIRD point-fix approach TRIED + REVERTED (2026-07-11) — documenting so it is not retried. Approach: a "self-reference resolves to shadowed definition" guard. Added ag-expanding-macro-name (set to HEAD around the re-walk of a macro's output in ag-macro-expand-call-1); ag-macro-lookup, when NAME == that guard, skips the first (own) ag-macros entry and returns the SHADOWED predecessor via a new ag-macro-lookup-after-first. Intent: (defmacro handler-case … `(let () (cl:handler-case …))) — where handler-case==cl:handler-case (name-keyed, no package scoping) — would resolve the inner self-reference to the BUILTIN handler-case instead of looping.

RESULT: SELF-HOST BUILD FAILED — "call to a name undefined at compile time" building build/funcl (stage2 self-compiled OK, final baked image failed). So the assumption "cold has no self-referential-expanding macro" is FALSE: at least one COLD macro expands to a form referencing its own name, and the guard rerouted that self-reference to a shadowed/absent predecessor → undefined call. Reverted to byte-identical baseline (build clean).

TALLY of disproven point-fixes for the ADR-0026 handler-case-recursion / symbol-split: (1) reader value-cell canonicalization → suite REGRESSED 593→538; (2) multiple-value-list via multiple-value-call → cold SEGFAULT; (3) this self-reference guard → cold build undefined-call. All three touch the macro-expander/reader/image subsystems and all three break the self-host fixed point in a different way. Strong empirical evidence: the fix must be the full ADR-0026 single-canonical-symbol + package-scoped macro/fn registry, done as a careful architectural pass with the reader producing package-distinguishable symbols — NOT any localized heuristic in the shared expander/lookup.

atgreen3992785501

FIXED (verified 2026-07-11): the exact shape — return-from inside handler-case inside a 2-binding let inside loop — compiles and runs. Probe: (defun p269 () (loop for i from 1 to 2 do (let ((a 1) (b 2)) (handler-case (if (= i 99) (return-from p269 :x) (+ a b)) (error (e) nil)))) :ok) => :ok. Likely fixed by the 2026-07-11 flet/labels/handler-case escape work or ADR-0028.