#308 fasl-loaded asdf: (load-system ...) dies 'MAKE-PLAN expected 0, got 5' — generic dispatch via fasl replay (next frontier after cave#307)

closed
Opened by atgreen

With cave#307 (7016ac0) the asdf fasl replays end to end: 0.15s load, (asdf:asdf-version) => "3.3.7". The NEXT frontier is USING it:

(load "asdf.funclfa") (setf asdf:central-registry (list "compat/alexandria/")) (asdf:load-system "alexandria") => [[cwas fun-built fnp=COMMON-LISP:T]] [[cwas direct-funcall]] ; Error: invalid number of arguments: MAKE-PLAN expected 0, got 5 | in: LOAD-SYSTEM

MAKE-PLAN is an asdf DEFGENERIC (asdf/plan) with &rest/keys; 'expected 0' says the call bound to a 0-arg function — a name-keyed arity split in how the fasl replays defgeneric/defmethod (mini-clos): likely the generic's dispatcher fn-cell lost to some other same-named artifact (the mc dispatcher records? a stub?), the cave#301 class on the DEFGENERIC channel. Reference behavior: source-loaded asdf's load-system works (north star, 2026-07-11). Notably alexandria's PARTIAL load got far enough that (alexandria:compose #'1+ #'1+) => 42 works in the aborted session.

Repro: fasl-compile asdf.lisp (~250s) once, then the 3 forms above (~10s). Suggested start: (fboundp 'asdf/plan:make-plan) + which record holds the dispatcher vs which the call site bound (KIND-7 home on the wire for MAKE-PLAN call sites), and whether mc-record-method's lifted defuns replay with the right cells — the #303 trace recipe applies (grep -aob MAKE-PLAN on the fasl for wire homes).

Comments (4)

atgreen3992953798

DEEP-DIVE SESSION RESULTS (not yet fixed — this layer is qualitatively different from #302-#307; full forensic record below so the next session starts at the frontier).

MEASURED FACTS (all on a fasl+build at 7016ac0 with temp name-filtered traces on the intern doors, pkg-add-export-name, %import-walk, ag-repl-install-fn-cell, and post-install cell reads):

  1. The name MAKE-PLAN has THREE artifacts: the 0-arg carrier CTOR from (defclass plan () ()) at asdf.lisp:10423 (mini-clos mc-ctor-defun emits (defun MAKE-PLAN () ...) — zero slots => zero params, arity-strict), the variadic &rest DISPATCHER from defgeneric:10939/defmethod:10953 (+ define-convenience-action-methods:10941), and the failing call sites are DESIGNATOR calls: (apply 'make-plan ...) at 11092 and 13840.

  2. COMPILE side: the READER resolves every make-plan token to the canonical arena record @111e9-band home=ASDF/PLAN (minted by ensure-package's export intern; [XP pkg=ASDF/PLAN cur=ASDF/FORCING]). BUT all six fn-cell installs ([IC]) target a DIFFERENT record: the name-keyed POOL record, home=ASDF/FORCING — first-touch-homed when asdf/plan's define-package EXPANDED while package was still ASDF/FORCING. The install channel is the CO-name channel (emit-ir-defun-to-co resolves CO names via ag-intern-symbol = static→pooled→cached, NOT the reader door). Also: the ctor's install name is a THIRD object — an uninterned mc-name-concat product (%SYM-CONCAT-2 → ag-intern-symbol at macroexpand, home=?).

  3. REPLAY side: all six installs go through ag-fasl-co-install-name (ADR-0031 door) to the replay-minted (ASDF/PLAN,MAKE-PLAN) canonical — CORRECT and consistent — with the fn-table head correctly advancing to each new CO ([CV head=...] increments). THE ANOMALY: the record's raw fn slot ([rec-3+32] via %mem-ref-val) reads a CONSTANT value (1083076146, heap band) before/after every install — the set-symbol-function stores (emit-ir-set-symbol-function, same +32 slot, KIND-6 imm64 direct to the record; verified ag-quote-pkgsym-p=1 and unification=1 AT INSTALL TIME) never appear at that address. Post-load, (symbol-function rec) returns yet another value (1083073410) != the raw slot read. Both in the GC heap band (0x40xxxxxxx): the replay-minted 'arena' records land in COLLECTED HEAP territory in the image-baked binary, and the divergence between barriered reads/writes and raw %mem-ref-val reads smells like EVACUATION/FORWARDING (Shenandoah-style) — the store barrier may be writing a to-space copy while raw reads (and possibly the designator-call path) see the from-space original. THIS is the next thing to verify (one gdb watchpoint on [rec-3+32] during the 0.15s load would settle it).

  4. Behavioral summary post-load: bare-name (make-plan) in CL-USER reaches the DISPATCHER (fn-table head, 'no ctor' from dispatch/make-instance); qualified/designator calls through the ASDF/PLAN record reach the CTOR-like artifact (0 args => #, 5 args => 'expected 0, got 5' — the load-system failure).

FIX DIRECTIONS, in order of preference once (3) is settled: (a) if GC forwarding is real, pin or root the dyn-sym mints in the image-baked regime (they are gc-register-root-range'd — check the range survives restore, cave#209 territory) or make %mem-ref/probe tooling barrier-aware; (b) unify the COMPILE-side install channel with the reader door (emit-ir-defun-to-co CO names + ag-repl-install-fn-cell at compile currently bind the name-keyed pool record — ADR-0031 stage 2: route them through the door so compile and replay bind the SAME (pkg,name) records and designator KIND-6 homes match installs); (c) the mini-clos defclass-vs-defgeneric NAME COLLISION (MAKE- ctor vs a user generic named MAKE-) deserves its own guard — mini-clos could skip/rename the carrier ctor when the name is already fbound as a dispatcher, or use an internal name for ctor registration (mc-set-class already takes the fn VALUE; the ctor needn't be interned under the public name at all — that alone would fix THIS instance and any make-X generic colliding with a class X).

Repro kit: fasl at 7016ac0; (load fasl) + (setf asdf:central-registry (list "compat/alexandria/")) + (asdf:load-system "alexandria") ~10s. All temp traces reverted; tree at 7016ac0 verified green (replay end-to-end, save-image roundtrip, spike oracle).

atgreen3992957439

SECOND OPINION (Codex, independent read of the actual GC/emitter code — session 019f5ccf-36c1-79d0-9c35-052cb0613f39):

GC-forwarding hypothesis REFUTED by the code. Symbols are NON-MOVABLE (src/cold/gc.lisp:187 — movability answers 0 for +widetag-symbol+); minor GC PINS a young region containing a non-movable object rather than evacuating (gc-heal-slot, gc.lisp:1770); full GC is non-moving mark-sweep (gc.lisp:2958); the store barrier pre-hook emits NOTHING and post only marks a card (gc.lisp:346); there are no read barriers. 'Barriered write to to-space vs raw read of from-space' cannot happen in this collector.

The raw probe math is right (symbol-function loads tagged+29 = base+32, runtime.lisp:3322; set-symbol-function stores base+32, runtime.lisp:3347). Therefore the measured divergence means either the two probes saw DIFFERENT objects/values, or something corrupts the cell between probes. CONFESSION on review: my '(symbol-function rec)=1083073410 vs raw slot=1083076146' comparison was CROSS-SESSION — mint addresses differ per run, so that discrepancy is a probe artifact, exactly Codex's 'rec is not the same object/value in those two probes'. The within-single-session anomaly (cell constant across all 6 installs while the table head advances) still stands and is what the next measurement targets.

Key mechanism note from Codex: emit-ir-apply's symbol arm reads the fn-cell DIRECTLY and only falls back name-keyed when the cell is NIL/UNBOUND — so a cell holding the 0-arg ctor has no rescue path.

gc-register-root-range does NOT pin or root the record as an object — it scans the range's slots as symbol-shaped records (gc.lisp:3115). Heap-minted dynamic symbol records relying on it as 'pinning' is wrong residency policy (they must be reachable as tagged objects through real roots or explicitly pinned) — GC hygiene issue, but per Codex not the primary cause here.

Codex's fix ranking (agrees with mine on content, reorders): 1st = (b) ADR-0031 stage 2 — unify the COMPILE-side CO-naming/install channel with the accessibility door ('fix the channel, not the symptom'; the facts still show compile installs binding a different (pkg,name) record than replay/designator references). 2nd = (a) dynamic-symbol residency/rooting (real, but hygiene). 3rd = (c) gensym the mini-clos carrier ctors (good defensive change, do AFTER (b) 'or you will keep finding the same bug under different public names').

Codex's one-step settling measurement: instrument emit-ir-apply's symbol arm at the failing (apply 'make-plan ...) site to log SYM tagged/base, raw [base+32], symbol-name/package, and the callee fn pointer immediately before the call — IN THE SAME RUN as the six install-target bases. Either the failing apply reads a different record (channel split → fix (b)) or the canonical record's cell really holds the ctor (stale/corrupt cell). Codex's bet: different channel/record or stale ctor cell, NOT GC forwarding.

atgreen3992959526

FIXED in 8b2bfb8 — a one-flag fix once codex's measurement forced the right reading of the data.

THE MEASUREMENT (same-run, per codex's suggestion): the apply-site record IS the install-target record (door-idempotent, verified same session); its raw cell held a value that funcalls as the 0-arg CTOR while the fn-table head was the DISPATCHER. Combined with the earlier within-run trace (cell CONSTANT across all six installs while the head advanced), the correct reading was NOT 'writes don't land' but 'writes land with the SAME stale value every time'.

ROOT CAUSE: ag-repl-install-fn-cell emitted its (set-symbol-function 'NAME (function NAME)) thunk WITHOUT ag-in-synthetic-install — the guard the cold build's batch install pass sets (compiler.lisp:7150). emit-ir-function's cave#266 pkg-canonical arm therefore compiled the (function NAME) as 'read NAME's own fn-cell, fall back name-keyed only when UNBOUND' — reading the very slot the thunk writes (the hazard that arm's own comment documents). Install #1 (mini-clos's 0-arg carrier ctor from (defclass plan () ()), literally named MAKE-PLAN) bound via the unbound fallback; installs #2-6 (the defgeneric/defmethod variadic dispatcher re-defuns) read the ctor back and stored it again. Sticky forever. emit-ir-apply reads the cell directly with no rescue for a bound-but-wrong cell => 'expected 0, got 5'. Single-defun names never noticed; REDEFINITIONS were the victims — including plain pkg-scoped REPL redefinition ((defun f ...) twice left designator calls on the FIRST definition).

Codex's GC-forwarding refutation stands confirmed: no GC involvement at all.

FIX: set ag-in-synthetic-install around the thunk emission in ag-repl-install-fn-cell — installs always resolve name-keyed to the fn-table head (the CO just installed). The cold bundle twin (ag-cold-bundle-install-fn-cell) is unaffected (unification-off regime, name-keyed arm already).

VERIFIED: fasl-loaded asdf's MAKE-PLAN record dispatches (funcall => mc-dispatch, matches bare-name calls); (defun g () 1)(defun g () 2) => (funcall 'g) = 2; (asdf:load-system "alexandria") clears the plan machinery and advances into DEFSYSTEM parsing. NEXT LAYER (distinct, filed as its own issue): 'G113 expected 3, got 5 | in: DEFSYSTEM' — a lifted method-lambda arity split in the defsystem parse, then MISSING-COMPONENT. alexandria:compose from the partial load returns 42. Gates: sharded suite 8/8, spike oracle green, --hosted 42, fixed point held.

Codex's remaining rankings stay open as follow-ups: ADR-0031 stage 2 (compile-side CO-naming/install channel through the door) and dyn-symbol residency/rooting hygiene — neither blocked this fix but both remain real.

atgreen3992965416

Codex follow-ups LANDED in 2aa2724 (this issue's second-opinion action items, closing the loop):

  1. Gensym absorber: ag-gensym-counter-absorb bumps the consumer's ag-gensym-counter past every loaded G CO name (fasl + .fco installers) — producer/consumer gensym namespaces can no longer collide (the #309 near-miss).

  2. ADR-0031 stage 2, install channel: fn-cell installs at the fasl compiler and the interactive REPL now bind BOTH the raw name-keyed record AND the accessibility-door record. Instructive failure on the way: a door-ONLY first cut broke the fasl COMPILE session itself (UNDEFINED-PKG-DESIGNATOR on detect-os — the compile session's live quotes still point at name-keyed pool records, cave#303's emit-side reality), so dual-channel it is until the live quote emission also goes through the door (full stage 2, still open). Redefinition now visible through every door (bare call / quoted designator / find-symbol-record funcall).

  3. Dyn-symbol residency: verified sound, no change — the mints live in the code mmap's dynamic-symbol arena slice [base+31MiB, base+32MiB) (measured: record 1112626819, base 1080033288), not in collected heap. The 'heap band' reading that motivated the concern was the wrong-base probe error #309 documented (0x800000000 holds the environ bv in the image-baked binary). gc-register-root-range's slot-scan treatment is correct for arena-resident records.

Gates: alexandria-via-fasl north star green, sharded suite 8/8, spike oracle green, save-image roundtrip green, --hosted 42, fixed point held.