#267 asdf: make-instance 'no ctor' for asdf's own classes — ADR-0026 symbol-split regression (worked at c639698)

closed
Opened by atgreen

Symptom

(asdf:load-system "tsys") (minimal system) fails. After the cave#267 &optional fix (e41fecb) the chain reaches: find-systemlocate-system/call-with-asdf-session(make-instance 'asdf/session::session)"no ctor".

Key facts

Measurement gotcha (documented so nobody re-loses time)

Reading mini-clos::*mc-ctors* from CL-USER returns 0 EVEN WHEN make-instance works — a DIFFERENT split symbol from the one compiled mini-clos code mutates. *mc-slots* isn't split (reads 79). Diagnose by BEHAVIOR, not (length mini-clos::*mc-thing*).

Next step

Bisect asdf:load-system across 5ccf8a1 (c639698 = known-good witness); find what splits mini-clos internal symbols across asdf's define-package'd packages. Relates to #266/#150 (ADR-0026).

Landed alongside

cave#267 fixed: defmethod/defgeneric were &optional/&key/&rest blind (recorded the optional DEFAULT as a class specializer → "9 is not a SYMBOL"). Variadic dispatcher fix, self-host holds (e41fecb).

Comments (4)

atgreen3992683589

CORRECTION — not a regression; layered bugs

Behavioral testing corrected the initial framing:

  • c639698 ALSO fails (asdf:load-system "tsys") — with its old &optional defmethod bug ("not a function") — so it dies BEFORE ever reaching make-instance. The commit-message claim "asdf:load-system WORKS" did not hold for a fresh central-registry system. So "no ctor" is NOT a 5ccf8a1 regression — my cave#267 &optional fix merely advanced find-system to the next latent bug.

  • The "no ctor" is the save-image BAKE specifically. Confirmed behaviorally (not via the split *mc-ctors* var): after a plain runtime (load "asdf.lisp") into base build/funcl, (make-instance 'asdf/session::session) → WORKS; only build/funcl-asdf (baked) gives "no ctor". So the save-image bake does not preserve mini-clos's runtime registration tables (*mc-ctors*/*mc-accs*/*mc-methods*/*mc-initforms* hold function pointers into JIT'd code / anonymous lambdas that don't survive; *mc-slots* survives because defclass pushes it at COMPILE time, line 453).

  • A further, independent "not a function" remains even under runtime-load: (asdf:find-system "tsys" nil) → "not a function" while (make-instance 'load-op)/(make-instance 'session) both work. So find-system has its own non-make-instance bug (likely the search-function funcall path: *system-definition-search-functions* / find-system-if-being-defined designator calls).

Net

Two+ distinct blockers to a baked, end-to-end asdf:load-system:

  1. save-image must preserve (or boot-rebuild) mini-clos function-valued tables — else store ctors/accessors as SYMBOLS (names) and resolve by name at call time (survives via fn-cell reinstall); anonymous method/initform lambdas need lifting to named defuns.
  2. find-system's own "not a function" (operate/search chain) — reproduces under fast runtime-load, debug there.

cave#267 (&optional defmethod) remains a valid, landed fix (e41fecb).

atgreen3992686554

Root cause of find-system's "not a function" = cave#148 (return-from across function frames)

The cave#268 eval/pathname fix (cb14d88, committed) advanced find-system: sysdef-central-registry-search "tsys" now returns #P".../tsys.asd" (was "not a function").

Next blocker localized to search-for-system-definition:

(flet ((try (f) (if-let (x (funcall f name)) (return-from search-for-system-definition x))))
  (try 'find-system-if-being-defined)
  (try 'sysdef-immutable-system-search)
  (map () #'try *system-definition-search-functions*))

*system-definition-search-functions* = (package-inferred central-registry source-registry). In the map, try funcalls central-registry which returns the .asd → (return-from search-for-system-definition x) SHOULD exit. It does NOT (cave#148: return-from to a defun's implicit block from inside a flet invoked by map = cross-function escape, unsupported — compiler.lisp:2871 "Defun implicit blocks … keep the old direct behavior"). So execution falls through to sysdef-source-registry-search (3rd) which itself raises "not a function".

Minimal repro (no asdf):

(defun rtest () (flet ((tryf (f) (if (funcall f) (return-from rtest :FOUND))))
  (map nil #'tryf (list (lambda () nil) (lambda () t) (lambda () (error "REACHED-3RD")))) :FELL))
(rtest)  => ERROR REACHED-3RD   ; return-from didn't fire across map
;; same logic with dolist (same frame) works:
(defun rtest2 () (dolist (f (list (lambda () nil) (lambda () t) (lambda () (error "X"))))
  (if (funcall f) (return-from rtest2 :FOUND))) :FELL)
(rtest2) => :FOUND-DOLIST

asdf uses return-from + map/some/loop pervasively, so cave#148 is THE recurring asdf blocker. Right fix = extend the cave#148 escape-token catcher to defun implicit blocks (compiler.lisp emit-ir-defun) so cross-function return-from unwinds. Whack-a-mole asdf patches are the alternative (asdf.lisp is funcl-ported) but there are many sites.

Separate CL-conformance bug found

some returns T, not the predicate's non-nil VALUE. (some (lambda (x) (if (= x 2) :GOT nil)) '(1 2 3))T (should be :GOT). It DOES early-exit correctly; only the return value is wrong. (This is why some can't replace the return-from pattern in search-for-system-definition — it loses the found system.) Worth its own fix.

Layered blocker status to end-to-end asdf:load-system

  1. [DONE] &optional defmethod (e41fecb, cave#267)
  2. [DONE] eval self-evaluates pathnames (cb14d88, cave#268)
  3. cave#148: return-from across function frames (find-system's search loop) — THIS is the current wall
  4. baked "no ctor": save-image drops mini-clos function-valued tables (only affects the baked binary; runtime-load works)
  5. some returns T not the value (CL conformance)
atgreen3992694765

GOAL ACHIEVED (runtime-load) + task#8 baked foundation

(asdf:load-system "alexandria") WORKS (runtime-load): base build/funcl + (load "asdf.lisp") + push absolute dir to *central-registry* + load-system → (alexandria:flatten '(1 (2 (3 4)) 5)) = (1 2 3 4 5). Also (tsys:hi)→42 on a minimal system. find-system returns a full SYSTEM object.

Landed (all self-host-validated): cave#267 (&optional defmethod, e41fecb), cave#268 (eval self-evals pathnames, cb14d88), cave#148 (return-from across function frames, b6f11ad), emit-ir-apply heal + slot-value (344953c).

task#8 (baked build/funcl-asdf) — PARTIAL (5cc5f39)

save-image now persists mini-clos's registration tables (they were never reachable from ag-image-runtime-roots): mini-clos exposes mc-image-tables/mc-image-restore-tables; roots index 6 + restore-after-fn-cells wire them in; plus a lowtag-5 fn-table-index decode added to ag-image-load-decode-slot (named functions in general heap slots — e.g. a mc-ctors alist value — were decoding to NIL). Baked make-instance now works (ctors/accessors/setters/slots/initargs/initforms all restore; make-instance 'session → SESSION-OK, was "no ctor").

REMAINING: baked GENERIC DISPATCH fails. asdf's method lambdas are CLOSURES (fn-ptr . captures). Simple methods are bare fn-ptrs (restore fine), but capturing closures hit cave#243's documented limitation ("a closure capturing collectable-heap env would NOT relocate"). Restored closure has car=valid-fn + cdr=captures-cons yet apply → "not a function". So baked find-system/load-system still error; runtime-load is the working path. Fix: (A) closure-capture relocation in the image encoder/decoder, or (B) lift method bodies to named top-level defuns (bare fn-ptrs — sidesteps closures, but breaks if asdf methods genuinely capture enclosing lexical vars).

atgreen3992785526

FIXED — (make-instance 'asdf:system :name "fakesys") constructs fine at the REPL (used as a test fixture today), and the real defsystem parse builds the full component tree. Evidence 2026-07-11: (push asdf:central-registry) + (asdf:load-system "alexandria") completes a REAL multi-file component load on build/funcl; (alexandria:flatten '(1 (2) 3)) => (1 2 3); 8/8 breadth probes incl. closure-returning curry/compose. Fix chain: ADR-0028 closure records e17d39b + eql specializers 822a554 + macro-aware eval 0f0eed8 + REPL robustness 21f15f2.