# Internals of the &clisp; Implementation

## The source files of &clisp;

For files in &clisp; binary distributions, see .

### File Types

`#P"*.d"`

  The source files for unpreprocessed &c-lang; code.

&c-file;

  The &c-lang; code after preprocessing; also the result of compiling some &ffi-pac; forms (see &ffi-out-fun;).

&lisp-file;

  The source files for Lisp code.

`#P"*.fas"`

  Compiled lisp code (platform-independent &bytecode;s).

&lib-file;

  Lisp header, produced by &compile-file; and used by &require;

### Source Pre-Processing

&c-lang; sources are pre-processed with the following tools before being passed to the &c-lang; compiler:

`utils/comment5.c`

  Convert &sh;-style comments (lines starting with `"# "`) to &c-lang;-style comments (`/**/`).

  The use of &sh;-style comments is deprecated.

`utils/varbrace.d`

  Add braces to &c-lang; source code, so that variable declarations (introduced with the pseudo-keyword `var`) can be used within blocks, like in `C++` and `C99`.

`utils/ccpaux.c`

  When &cpp; cannot handle indented directives, remove the indentation.

`utils/gctrigger.d`

  Add `GCTRIGGER` statements at the head of function bodies (for functions marked with the `maygc` pseudo-keyword).

`utils/deema.c`

  When &cpp; cannot handle empty macro arguments, insert `_EMA_` instead.

`utils/ccmp2c.c`

  For the `new-clx` module only. Allows &cpp;-style preprocessing before &modprep; processing. Should be merged into &modprep; eventually.

`utils/modprep.lisp`

  For some modules only, see .

### Files

#### Unpreprocessed &c-lang; code

##### Includes

`src/lispbibl.d`

  main include file

`src/fsubr.d`

  list of all built-in special forms

`src/subr.d`

  list of all built-in functions

`src/pseudofun.d`

  list of all pseudo functions

`src/constpack.d`

  list of packages accessed by &c-lang; code

`src/constsym.d`

  list of symbols accessed by &c-lang; code

`src/constobj.d`

  list of miscellaneous objects accessed by &c-lang; code

`src/unix.d`

  include file for the &unix; implementations

`src/win32.d`

  include file for the &win32; based versions

`src/xthread.d`

  include file for thread support

`src/modules.h`

  list of foreign &module;s

##### Internal &c-lang; Modules

`src/spvw.d`

  Memory management (&gc;ion), startup; some OS interface.

  `src/avl.d`

    An implementation of AVL (Adelson-Velskii and Landis) trees.

  `src/sort.d`

    A sorting routine.

  `src/subrkw.d`

    The list of all built-in functions with keywords in &lalist;.

`src/spvwtabf.d`

  The table of built-in special operators and functions.

`src/spvwtabs.d`

  The table of all &symbol-t;s accessed by &c-lang; code.

`src/spvwtabo.d`

  The table of miscellaneous objects accessed by &c-lang; code.

`src/eval.d`

  Evaluator (form interpreter) and &bytecode; interpreter.

  &bytecode-d;

    List of &bytecode;s.

`src/control.d`

  Special operator interpreter.

`src/pathname.d`

  Pathnames, file- and directory-related functions.

`src/stream.d`

  &stream-t;s of all kinds: &file-stream-t;s, terminal streams, &string-stream-t;s etc.

`src/socket.d`

  Opening sockets for TCP/IP and CLX.

`src/io.d`

  The lisp reader (parser) and printer (also pretty printer).

`src/array.d`

  Functions dealing with &array-t;s and &vector-t;s.

`src/hashtabl.d`

  Functions dealing with &hash-table-t;s.

`src/list.d`

  Functions dealing with &list-t;s.

`src/package.d`

  Functions dealing with &package-t;s.

`src/record.d`

  Functions dealing with records (structures, closures, etc.)

`src/sequence.d`

  The generic &sequence-t; functions.

`src/charstrg.d`

  Functions dealing with &character-t;s and &string-t;s.

`src/debug.d`

  Support for debugging and the &repl; (see ).

`src/error.d`

  &error-t; handling and &signal;ing.

  `src/errunix.d`

    &unix;-specific error messages.

  `src/errwin32.d`

    &win32;-specific error messages.

`src/misc.d`

  Miscellaneous functions.

`src/time.d`

  Timing functions.

`src/predtype.d`

  Predicates, type tests.

`src/symbol.d`

  Functions dealing with &symbol-t;s.

`src/unixaux.d`

  Auxiliary functions (&unix; version only).

`src/win32aux.d`

  Auxiliary functions (&win32; version only).

`src/foreign.d`

  &ffi-pac; support.

`src/lisparit.d`

  Functions dealing with numbers (arithmetic), see .

`src/noreadline.d`

  Dummy plug-in for the &readline; library.

##### Number system (arithmetic)

`src/lisparit.d`

  initialization, input/output of numbers, lisp functions

`src/aridecl.d`

  declarations

`src/arilev0.d`

  arithmetic at the machine level

`src/arilev1.d`

  digit sequences

`src/arilev1c.d`

  operations on digit sequences, written in &c-lang;

`src/arilev1i.d`

  operations on digit sequences, as inline functions

`src/arilev1e.d`

  operations on digit sequences, bindings to external routines

`src/intelem.d`

  &integer-t;s: elementary operations

`src/intlog.d`

  &integer-t;s: logical connectives

`src/intplus.d`

  &integer-t;s: addition and subtraction

`src/intcomp.d`

  &integer-t;s: comparison

`src/intbyte.d`

  &integer-t;s: byte operations &ldb;, &dpb;

`src/intmal.d`

  &integer-t;s: multiplication

`src/intdiv.d`

  &integer-t;s: division

`src/intgcd.d`

  &integer-t;s: &gcd; and &lcm;

`src/int2adic.d`

  &integer-t;s: operations on 2-adic integers

`src/intsqrt.d`

  &integer-t;s: square root, n-th root

`src/intprint.d`

  subroutines for &integer-t; output

`src/intread.d`

  subroutines for &integer-t; input

`src/rational.d`

  rational numbers (&ratio-t;s)

`src/sfloat.d`

  elementary operations for &short-float-t;s

`src/ffloat.d`

  elementary operations for &single-float-t;s

`src/dfloat.d`

  elementary operations for &double-float-t;s

`src/lfloat.d`

  elementary operations for &long-float-t;s

`src/flo_konv.d`

  conversions between &float-t;s

`src/flo_rest.d`

  general &float-t; operations

`src/realelem.d`

  elementary functions for &real-t; numbers

`src/realrand.d`

  random numbers

`src/realtran.d`

  transcendental functions for &real-t; numbers

`src/compelem.d`

  elementary functions for &complex-t; numbers

`src/comptran.d`

  transcendental functions for &complex-t; numbers

###### External routines for the arithmetic system, written in assembly language

`src/ari68000.d`

  written in 68000 assembler, MIT syntax

`src/ari68020.d`

  written in 68020 assembler, MIT syntax

`src/arisparc.d`

  written in SPARC assembler

`src/arisparc64.d`

  written in 64-bit SPARC assembler

`src/ari80386.d`

  written in i386/i486 assembler

`src/arimips.d`

  written in MIPS assembler

`src/arimips64.d`

  written in 64-bit MIPS assembler

`src/arihppa.d`

  written in HPPA-1.0 assembler

`src/arivaxunix.d`

  written in VAX assembler, Unix assembler syntax

`src/ariarm.d`

  written in ARM assembler

##### External routines for accessing the stack, written in assembly language

`src/sp68000.d`

  written in 68000 assembler, MIT syntax

`src/spsparc.d`

  written in SPARC assembler

`src/spsparc64.d`

  written in 64-bit SPARC assembler

`src/sp80386.d`

  written in i386/i486 assembler

`src/spmips.d`

  written in MIPS assembler

#### Other assembly language stuff

`src/asmi386.sh`

  converts i386 assembler from MIT syntax to a macro syntax

`src/asmi386.hh`

  expands i386 assembler in macro syntax to either MIT or Intel syntax

#### Lisp source files

`src/init.lisp`

  the first file to be loaded during bootstrapping, loads everything else

`src/defseq.lisp`

  defines the usual sequence types for the generic sequence functions

`src/backquote.lisp`

  implements the backquote read macro

`src/defmacro.lisp`

  implements &defmacro;

`src/macros1.lisp`

  the most important macros

`src/macros2.lisp`

  some other macros

`src/defs1.lisp`

  miscellaneous definitions

`src/timezone.lisp`

  site-dependent definition of time zone, except for &unix; and &win32;.

`src/places.lisp`

  macros using &place;s, definitions of most standard and extensiion &place;s

`src/floatprint.lisp`

  defines `SYS::WRITE-FLOAT-DECIMAL` for printing floating point numbers in base 10

`src/type.lisp`

  functions working with type specifiers: &typep;, &subtypep;

`src/defstruct.lisp`

  implements the macro &defstruct;

`src/format.lisp`

  implements the function &format;

`src/room.lisp`

  implements the function &room; (see also )

`src/savemem.lisp`

  see

`src/keyboard.lisp`

  implements the macro &with-kbd;

`src/runprog.lisp`

  implements the functions &run-prog;, &run-cmd; etc.

`src/query.lisp`

  implements the functions &y-or-n-p; and &yes-or-no-p;

`src/reploop.lisp`

  support for debugging and the &repl; (see )

`src/dribble.lisp`

  implements the functions &dribble; and `EXT:DRIBBLE-STREAM`

`src/complete.lisp`

  implements completion, see .

`src/describe.lisp`

  implements functions &describe-my;, &apropos-my;, &apropos-list;

`src/trace.lisp`

  tracer

`src/macros3.lisp` (optional)

  the macros &letf;, &letf-star; and &ethe;

`src/config.lisp`

  (user written) site-dependent configuration, may be a link to one of the following:

  `src/cfgsunux.lisp`

    for &unix;, using SunOS

  `src/cfgunix.lisp`

    for any other &unix;

  `src/cfgwin32.lisp`

    for the &win32;

  See .
`src/compiler.lisp`

  compiles Lisp code to &bytecode;

`src/disassem.lisp`

  the function &disassemble;

`src/defs2.lisp`

  miscellaneous &ansi-cl; definitions

`src/loop.lisp`

  implements the &ansi-cl;-compatible &loop; macro

`src/clos.lisp`

  loads the various parts of the &clos;:

  `src/clos-package.lisp`

    declares the imports and exports of the &clos-pac; package

  `src/clos-macros.lisp`

    defines some internal macros used by the &clos; implementation

  `src/clos-class0.lisp`

    defines the `class-version` structure

  `src/clos-metaobject1.lisp`

    defines the &metaobject-t; class

  `src/clos-slotdef1.lisp`

    defines the &slot-definition-t; class and its subclasses

  `src/clos-slotdef2.lisp`

    defines &initialize-instance; methods for &slot-definition-t; and its subclasses

  `src/clos-slotdef3.lisp`

    defines the generic functions that can be used on &slot-definition-t; objects

  `src/clos-stablehash1.lisp`

    defines the &standard-stablehash; class

  `src/clos-stablehash2.lisp`

    defines &initialize-instance; methods for &standard-stablehash;

  `src/clos-specializer1.lisp`

    defines the &specializer-t; class and its subclasses

  `src/clos-specializer2.lisp`

    defines &initialize-instance; methods for &specializer-t; and its subclasses

  `src/clos-specializer3.lisp`

    defines the generic functions that can be used on &specializer-t; objects

  `src/clos-class1.lisp`

    defines the `potential-class` class and its subclasses

  `src/clos-class2.lisp`

    implements the mapping from class names to classes

  `src/clos-class3.lisp`

    implements the &defclass; macro, class definition and class redefinition

  `src/clos-class4.lisp`

    defines &initialize-instance; methods for `potential-class` and its subclasses

  `src/clos-class5.lisp`

    implements the special logic of &make-instance;, &initialize-instance; etc.

  `src/clos-class6.lisp`

    defines the generic functions that can be used on `potential-class` objects

  `src/clos-method1.lisp`

    defines the &method-t; class and its subclasses

  `src/clos-method2.lisp`

    implements the bulk of &defmethod;

  `src/clos-method3.lisp`

    defines the generic functions that can be used on &method-t; objects

  `src/clos-method4.lisp`

    makes generic functions on &standard-method-t; objects extensible

  `src/clos-methcomb1.lisp`

    defines the &method-combination-t; class

  `src/clos-methcomb2.lisp`

    implements method combination (part 2 of generic function dispatch and execution) and the &define-method-combination; macro

  `src/clos-methcomb3.lisp`

    defines &initialize-instance; methods for &method-combination-t;

  `src/clos-methcomb4.lisp`

    makes generic functions on &method-combination-t; objects extensible

  `src/clos-genfun1.lisp`

    defines the &generic-function-t; class and its metaclass, superclass and subclasses

  `src/clos-genfun2a.lisp`

    implements part 1 of generic function dispatch and execution

  `src/clos-genfun2b.lisp`

    implements part 3 of generic function dispatch and execution

  `src/clos-genfun3.lisp`

    implements creation of generic function objects, &defmethod;, &defgeneric;

  `src/clos-genfun4.lisp`

    defines &initialize-instance; methods for &generic-function-t; and its subclasses

  `src/clos-genfun5.lisp`

    makes generic functions on &generic-function-t; objects extensible

  `src/clos-slots1.lisp`

    implements low-level slot access, &with-slots;, &with-accessors;

  `src/clos-slots2.lisp`

    defines the generic functions that deal with slot access

  `src/clos-dependent.lisp`

    implements notification from metaobjects to dependent objects

  `src/clos-print.lisp`

    implements the function &print-object;

  `src/clos-custom.lisp`

    provides user customization of the &clos;

`src/condition.lisp`

  implements the &cl; Condition System (&clcs;)

`src/gstream.lisp`

  generic stream default methods

`src/foreign1.lisp`

  &ffi-pac; interface

`src/screen.lisp`

  the screen access package, see

`src/edit.lisp` (optional)

  the screen editor (&ed;), &uncompile;

`src/inspect.lisp`

  implements &inspect-my; (tty and &http; frontends)

`src/clhs.lisp`

  implements &open-http;, &browse-url;

`src/exporting.lisp`

  Macros that export their definienda, see .

`src/threads.lisp`

  MT interface

`src/spanish.lisp`
`src/german.lisp`
`src/french.lisp`
`src/russian.lisp`
`src/dutch.lisp`

  i18n user messages

#### External Modules

`modules/`

  individual external module sources

#### Documentation

`src/NEWS`

  the list of the user-visible changes

`src/_README`

  master for the distribution's README

  `src/_README.en`
  `src/_README.de`
  `src/_README.es`

    translations of `src/_README`

`doc/clisp.xml.in`

  &docbook;/&xml; sources for the &clisp; manual page

`build-dir/clisp.1`

  the platform-specific `man` manual page, generated from `doc/clisp.xml.in` at build time

`build-dir/clisp.html`

  the platform-specific &html; manual page, generated from `doc/clisp.xml.in` at build time

`doc/impnotes.xml.in`

  the master &docbook;/&xml; file for these implementation notes; includes the following files

  `doc/cl-ent.xml`

    &clisp;-independent general &cl;-related entities

  `doc/clhs-ent.xml`

    generated list of &hyperspec; entities

  `doc/impent.xml`

    &clisp;-specific entities

  `doc/unix-ent.xml`

    &unix;-related entities

  `doc/mop-ent.xml`

    &mop;-related entities

  `doc/impbody.xml`

    most of

  `doc/impissue.xml`

  `doc/gray.xml`

  `doc/mop.xml`

  `doc/impext.xml`

    and

  `doc/impbyte.xml`

    this

  `modules/**/*.xml`

    individual external module documentation

  `doc/Symbol-Table.text`

    the mapping between lisp symbols and element IDs in these notes (see &describe-my;).

`doc/impnotes.html`

  these &html; implementation notes, generated from `doc/impnotes.xml.in` at release time

#### Internationalization

`src/po/*.pot`

  list of translatable messages (portable object template)

`src/po/*.po`

  translated messages (portable objects)

`src/po/*.gmo`

  translated messages (&gnu; format message objects)

#### Automatic configuration on &unix;

`src/configure.in`

  lists features to be checked

`src/autoconf/autoconf.m4`

  `autoconf`'s driver macros. Part of &autoconf; 2.57

`src/m4/`

  a repertoire of features. Use with &autoconf; 2.57

`src/configure`

  configuration script, generated from `src/configure.in`

`src/intparam.c`

  figures out some machine parameters (word size, endianness etc.)

`src/floatparam.c`

  figures out some floating point arithmetics parameters (rounding, epsilons etc.)

`src/unixconf.h.in`

  header file master, generated from `src/configure.in`. `build-dir/unixconf.h` contains the values of the features discovered by `src/configure`.

`src/makemake.in`

  makefile construction script master

`src/_clisp.c`

  master for the distribution's driver program

`src/_distmakefile`

  master for the distribution's Makefile

## Overview of &clisp;'s Garbage Collection

These are internals, which are of interest only to the &clisp; developers. If you are not subscribed to &clisp-devel;, this chapter is probably not for you.

### Introduction

Knowing that most &malloc; implementations are buggy and/or slow, and because &clisp; needs to perform &gc;ion, &clisp; has its own memory management subsystem in files `src/spvw*.d`, see .

### Lisp objects in &clisp;

#### Three kinds of storage are distinguished:

1. &clisp; data (the heap), i.e. storage which contains &clisp; &object-t;s and is managed by the &gc;or.

2. &clisp; stack (called &STACK;), contains &clisp; &object-t;s visible to the &gc;or

3. &c-lang; data (including program text, data, &malloc;ed memory)

A &clisp; object is one word, containing a tag (partial type information) and either immediate data or a pointer to storage. Pointers to &c-lang; data have tag = `machine_type` = 0, pointers to &clisp; stack have tag = `system_type`, most other pointers point to &clisp; data.

#### Immediate objects

32-bit CPU

  - &fixnum-t;
  - &short-float-t;
  - &character-t;
64-bit CPU

  In addition to the above,

  - &single-float-t; (with &typecodes;)

Let us turn to those &clisp; objects that consume regular &clisp; memory. Every &clisp; object has a size which is determined when the object is allocated (using one of the `allocate_*()` routines). The size can be computed from the type tag and - if necessary - the length field of the object's header. The length field always contains the number of elements of the object. The number of bytes is given by the function `objsize()`.

&clisp; objects which contain exactly 2 &clisp; objects (i.e. &cons-t;es, &complex-t; numbers, &ratio-t;s) are stored in a separate area and occupy 2 words each. All other &clisp; objects have varying length (more precisely, not a fixed length) and include a word for &gc;ion purposes at their beginning.

The garbage collector is invoked by `allocate_*()` calls according to certain heuristics. It marks all objects which are "live" (may be reached from the "roots"), compacts these objects and unmarks them. Non-live objects are lost; their storage is reclaimed.

2-pointer objects are compacted by a simple hole-filling algorithm: fill the left-most object into the right-most hole, and so on, until the objects are contiguous at the right and the hole is contiguous at the left.

Variable-length objects are compacted by sliding them down (their address decreases).

### Object Pointer Representations

&clisp; implements two ways of representing object pointers. (An object pointer, &c-lang; type &object-t;, contains a pointer to the memory location of the object, or - for &immediate-o; - all bits of the object itself.) Both of them have some things in common:

- There is a distinction between &immediate-o;s (&character-t;s, &fixnum-t;s, &short-float-t;s, etc) and heap allocated objects.

- All object pointers are typed, i.e. contain a few bits of information about the type of the pointed-to object. At a minimum, these bits must allow to distinguish immediate and heap-allocated objects.

- Not all of the type information is contained in the object pointer. For example, &clos; objects can change their type when &change-class; is called. To avoid scanning all the heap for references when this happens, the class information is stored in the heap allocated object, not in the object pointer.

The &heapcodes; object representation has a minimum of type bits in the object pointer, namely, 2 bits. They allow to distinguish &immediate-o;s (which have some more type bits), &cons-t;es (which have no type bits in the heap, since they occupy just two words in the heap, with no header), other heap objects (many, from &simple-vector-t;s to &foreign-pointer-t;s), and Subrs. Most object types are distinguished by looking a the rectype field in the header of the heap object.

The &typecodes; object representation has about two dozen of types encoded in 6 or 7 bits in the object pointer. Typically these are the upper 8 bits of a word (on a 32-bit machine) or the upper 16 bits or 32 bits of a word (on a 64-bit machine). The particular values of the typecodes allow many common operations to be performed with a single bit test (e.g. &consp; and &minusp; for a &real-t; are bit tests) or range check. However, the rectype field still exists for many types, because there are many built-in types which do not need a particularly fast type test.

Which object representation is chosen is decided at build time depending on the available preprocessor definitions. You can define &typecodes; or &heapcodes; to force one or the other.

One might expect that &typecodes; is faster than &heapcodes; because it does not need to make as many memory accesses. This effect is, however, hardly measurable in practice (certainly not more than 5% faster). Apparently because, first, the situations where the type of an object is requested but then the object is not looked into are rare. It is much more common to look into an object, regardless of its type. Second, due to the existence of data caches in the CPU, accessing a heap location twice, once for the type test and then immediately afterwards for the data, is not significantly slower than just accessing the data.

&typecodes; is problematic on 32-bit machines, when you want to use more than 16 MB of memory, because the type bits (at bit 31..24) interfere with the bits of a heap address. For this reason, &heapcodes; is the default on 32-bit platforms.

&heapcodes; is problematic on platforms whose object alignment is less than 4. This affects only the mc680x0 CPU; however, here the alignment can usually be guaranteed through some &gcc; options.

### Memory Models

There are 6 memory models. Which one is used, depends on the operating system and is determined at build time.

#### Memory Models

SPVW_MIXED_BLOCKS_OPPOSITE

  The heap consists of one block of fixed length (allocated at startup). The variable-length objects are allocated from the left, the 2-pointer objects are allocated from the right. There is a hole between them. When the hole shrinks to 0, &gc; is invoked. &gc; slides the variable-length objects to the left and concentrates the 2-pointer objects at the right end of the block again. When no more room is available, some reserve area beyond the right end of the block is halved, and the 2-pointer objects are moved to the right accordingly.

  &advantages;
  (+)

    Simple management.

  (+)

    No fragmentation at all.

  (-)

    The total heap size is limited.

SPVW_MIXED_BLOCKS_OPPOSITE & TRIVIALMAP_MEMORY

  The heap consists of two big blocks, one for variable-length objects and one for 2-pointer objects. The former one has a hole to the right and is extensible to the right, the latter one has a hole to the left and is extensible to the left. Similar to the previous model, except that the hole is unmapped.

  &advantages;
  (+)

    Total heap size grows depending on the application's needs.

  (+)

    No fragmentation at all.

  (*)

    Works only when SINGLEMAP_MEMORY is possible as well.

SPVW_MIXED_BLOCKS_STAGGERED & TRIVIALMAP_MEMORY

  The heap consists of two big blocks, one for variable-length objects and one for 2-pointer objects. Both have a hole to the right, but are extensible to the right.

  &advantages;
  (+)

    Total heap size grows depending on the application's needs.

  (+)

    No fragmentation at all.

  (*)

    Works only when SINGLEMAP_MEMORY is possible as well.

SPVW_MIXED_PAGES

  The heap consists of many small pages (usually around 8 KB). There are two kinds of pages: one for 2-pointer objects, one for variable-length objects. The set of all pages of a fixed kind is called a "Heap". Each page has its hole (free space) at its end. For every heap, the pages are kept sorted according to the size of their hole, using AVL trees. The &gc;ion is invoked when the used space has grown by 25% since the last GC; until that point new pages are allocated from the OS. The GC compacts the data in each page separately: data is moved to the left. Emptied pages are given back to the OS. If the holes then make up more than 25% of the occupied storage, a second GC turn moves objects across pages, from nearly empty ones to nearly full ones, with the aim to free as many pages as possible.

  &advantages;
  (-)

    Every allocation requires AVL tree operations, thus slower

  (+)

    Total heap size grows depending on the application's needs.

  (+)

    Works on operating systems which do not provide large contiguous areas.

SPVW_PURE_PAGES

  Just like SPVW_MIXED_PAGES, except that every page contains data of only a single type tag, i.e. there is a Heap for every type tag.

  &advantages;
  (-)

    Every allocation requires AVL tree operations, thus slower

  (+)

    Total heap size grows depending on the application's needs.

  (+)

    Works on operating systems which do not provide large contiguous areas.

  (-)

    More fragmentation because objects of different type never fit into the same page.

SPVW_PURE_BLOCKS

  There is a big block of storage for each type tag. Each of these blocks has its data to the left and the hole to the right, but these blocks are extensible to the right (because there is enough room between them). A &gc;ion is triggered when the allocation amount since the last GC reaches 50% of the amount of used space at the last GC, but at least 512 KB. The &gc;ion cleans up each block separately: data is moved left.

  &advantages;
  (+)

    Total heap size grows depending on the application's needs.

  (+)

    No 16 MB total size limit.

  (*)

    Works only in combination with SINGLEMAP_MEMORY.

In page based memory models, an object larger than a page is the only object carried by its pages. There are no small objects in pages belonging to a big object.

The following combinations of memory model and `mmap` tricks are possible (the number indicates the order in which the respective models have been developed):

#### Memory models with &typecodes;

SPVW_MIXED_BLOCKS_OPPOSITE | 1 | 10 |  | 2 | 9
SPVW_MIXED_BLOCKS_STAGGERED |  | 7 |  |  | 8
SPVW_PURE_BLOCKS |  |  | 5 |  | 6
SPVW_MIXED_PAGES | 3 |  |  |  |
SPVW_PURE_PAGES | 4 |  |  |  |

#### Memory models with &heapcodes;

SPVW_MIXED_BLOCKS_OPPOSITE | * | * | *
SPVW_MIXED_BLOCKS_STAGGERED |  | * | *
SPVW_MIXED_PAGES | * |  |

#### Legend to and

1. no MAP_MEMORY

2. TRIVIALMAP_MEMORY

3. SINGLEMAP_MEMORY

4. MULTIMAP_MEMORY

5. GENERATIONAL_GC

### The burden of &gc;ion upon the rest of &clisp;

#### &object-t; pointer invalidation

Every subroutine marked with can trigger GC may invoke &gc;ion. &gc; moves all the &clisp; non-&immediate-o;s and updates the pointers. But the &gc; looks only on the &STACK; and not in the &c-lang; variables. (Anything else would not be portable.) Therefore at every "unsafe" point, i.e. every call to such a subroutine, all the &c-lang; variables of type &object-t; MUST BE ASSUMED TO BECOME GARBAGE. (Except for &object-t;s that are known to be unmovable, e.g. immediate data or Subrs.) Pointers inside &clisp; data (e.g. to the characters of a &string-t; or to the elements of a &simple-vector-t;) become INVALID as well.

The workaround is usually to allocate all the needed &clisp; data first and do the rest of the computation with &c-lang; variables, without calling unsafe routines, and without worrying about &gc;.

Run-time GC-safety checking is available when you build &clisp; with a C++ compiler, e.g.:

    &sh-prompt; CC=g++ ./configure --with-debug build-g-gxx

When built like this, &clisp; will `abort` when you reference GC-unsafe data after an allocation (which could have triggered a &gc;ion), and &gdb; will pinpoint the trouble spot.

Specifically, when &clisp; is configured as above, there is a global integer variable `alloccount` and the &object-t; structure contains an integer allocstamp slot. If these two integers are not the same, the &object-t; is invalid. By playing with &gdb;, you should be able to figure out the precise spot where an allocation increments `alloccount` after the object has been retrieved from a GC-visible location.

#### Memory protection

Generational &gc;or uses memory protection, so when passing pointers into the lisp heap to &c-lang; functions, you may encounter errors (`errno`=`EFAULT`) unless you call `handle_fault_range(protection,region_start,region_end)` on the appropriate memory region. See files

- `src/unixaux.d`
- `src/win32aux.d`
- `modules/syscalls/calls.c`
- `modules/rawsock/rawsock.c`

for examples.

### Foreign Pointers

Pointers to &c-lang; functions and to &malloc;ed data can be hidden in &clisp; objects of type machine_type; &gc; will not modify its value. But one should not dare to assume that a &c-lang; stack pointer or the address of a &c-lang; function in a shared library satisfies the same requirements.

If another pointer is to be viewed as a &clisp; object, it is best to box it, e.g. in a &simple-bit-vector-t; or in an Fpointer (using `allocate_fpointer()`.)

## Extending &clisp;

John Foderaro

&cl; is a programmable programming language.

&clisp; can be easily extended the same way any other &cl; implementation can: create a lisp file with your variables, functions, macros, etc.; (optionally) compile it with &compile-file;; &load; it into a running &clisp;, and save the &mem-image;.

This method does not work when you need to use some functionality not available in &clisp;, e.g., you want to call a &c-lang; function. You are urged to use External Modules instead of adding built-in functions.

&clisp; comes with an &ffi-pac; which allows you to access &c-lang; libraries in an easy way (including creating &foreign-function;s dynamically).

### Adding a built-in function

In the rare cases when you really need to modify &clisp; internals and add a truly built-in function, you should read the &clisp; sources for inspiration and enlightenment, choose a file where your brand-new built-in function should go to, and then ...

- add the `LISPFUN` form and the implementation there;

- add the `LISPFUN` header to file &subr-d;;

- declare the function name in file &constsym-d; in the appropriate package (probably &ext-pac;, if there is no specific package);

- if your function accepts keyword arguments, then an appropriate pair of forms must be added to &subrkw-d; and you must make sure that the keyword symbols are declared in &constsym-d;;

- export your function name from the appropriate package in file &init-lisp;;

- when you are done, you should run `make check-sources` in your build directory: this will check that the definitions (source files) and the declarations (&subr-d;, &subrkw-d; and &fsubr-d;) are in sync.

Be very careful with the GC-unsafe functions! Always remember about GC-safety!

These instructions are intentionally terse - you are encouraged to use &module;s and/or &ffi-pac; instead of adding built-ins directly.

### Adding a built-in variable

If you must be able to access the Lisp variable in the &c-lang; code, follow these steps:

- declare the variable name in &constsym-d; in the appropriate package (probably &custom-pac;, if there is no specific package);

- add a `define_variable()` call in function `init_symbol_values()` in file &spvw-d;;

- export your variable name from the appropriate package in file &init-lisp;;

### Recompilation

Any change that forces `make` to remake &lisp-run;, will force recompilation of all &lisp-file; files and re-dumping of &lispinit;, which may be time-consuming. This is not always necessary, depending on what kind of change you introduced.

On the other hand, if you change any of the following files:

- &constobj-d;
- &constsym-d;
- &fsubr-d;
- &subr-d;
- &subrkw-d;

your &lispinit; will have to be re-dumped.

If you change the signature of any system function mentioned in the `FUNTAB` arrays in file &eval-d;, all the &fasl-file; files will become obsolete and will need to be recompiled. You will need to add a note to that effect to the &news; file and augment the object `version` in file &constsym-d;. Please try to avoid this as much as possible.

Last modified: 1998-09-19

## The &clisp; bytecode specification

### Introduction

The &clisp; compiler compiles &cl; programs into instruction codes for a virtual processor. This bytecode is optimized for saving space in the most common cases of &cl; programs. The main advantages/drawbacks of this approach, compared to native code compilation, are:

- Bytecode compiled programs are a lot smaller than when compiled to native code. This results in better use of CPU caches, and in less virtual memory paging. Users perceive this as good responsiveness.

- Maximum execution speed (throughput in tight loops) is limited.

- Since no bytecode instructions are provided for unsafe operations (like unchecked array accesses, or fast &car;/&cdr;), programs run with all safety checks enabled even when compiled.

- Execution speed of a program can easily be understood by looking at the output of the &disassemble; function. A rule of thumb is that every elementary instruction costs 1 time unit, whereas a function call costs 3 to 4 time units.

- Needing to do no type inference, the compiler is pretty straightforward and fast. As a consequence, the definition of &clos; generic functions, which needs to compile small pieces of generated code, is not perceived to be slow.

- The compiler is independent from the hardware CPU. Different back-ends, one for each hardware CPU, are not needed. As a consequence, the compiler is fairly small (and would have been easily maintainable if it were written in a less kludgey way...), and it is impossible for the compiler writer to introduce CPU dependent bugs.

### The virtual machine

The bytecode can be thought of as being interpreted by a virtual processor. The engine which interprets the bytecode (the implementation of the virtual machine) is actually a &c-lang; function, but it could as well be a just-in-time compiler which translates a function's bytecode into hardware CPU instructions the first time said function is called.

The virtual machine is a stack machine with two stacks:

&STACK;

  a stack for &clisp; objects and frames (Lisp stack).

&SP;

  a stack for other data and pointers (Program stack).

This two-stack architecture permits to save an unlimited number of &clisp; objects on the &STACK; (needed for handling of &cl; &mul-val;), without &consing;. Also, in a world with a compacting no-ambiguous-roots garbage collector, &STACK; must only hold &clisp; objects, and &SP; can hold all the other data belonging to a frame, which would not fit into &STACK; without tagging/untagging overhead.

The scope of &STACK; and &SP; is only valid for a given function invocation. Whereas the amount of &STACK; space needed for executing a function (excluding other function calls) is unlimited, the amount of &SP; space needed is known a priori, at compile time. When a function is called, no relation is specified between the caller's &STACK; and the callee's &STACK;, and between the caller's &SP; and the callee's &SP;. The bytecode is designed so that outgoing arguments on the caller's &STACK; can be shared by the caller's incoming arguments area (on the callee's &STACK;), but a virtual machine implementation may also copy outgoing arguments to incoming arguments instead of sharing them.

The virtual machine has a special data structure, `values`, containing the top of stack, specially adapted to &cl; &mul-val;:

&mv-count;

  an unsigned integer.

&value1;

  the &pri-val;, a &clisp; object. If &mv-count; = 0, this is &nil;.

&mv-space;

  all values except the first one, an array of &clisp; objects.

The contents of `values` is short-lived. It does not survive a function call, not even a &gc;ion.

The interpretation of some bytecode instructions depends on a constant, &jbs;. This is a CPU-dependent number, the value of `SYSTEM::*JMPBUF-SIZE*`. In &c-lang;, it is defined as `ceiling(sizeof(jmp_buf),sizeof(void*))`.

### The structure of compiled functions

A compiled function consists of two objects: The function itself, containing the references to all &clisp; objects needed for the bytecode, and a byte vector containing only immediate data, including the bytecode proper.

Typically, the byte vector is about twice as large as the function vector. The separation thus helps the garbage collector (since the byte vector does not need to be scanned for pointers).

A function looks like this (cf. the &c-lang; type Cclosure):

name

  This is the name of the function, normally a symbol or a list of the form `(&setf; &symbol-r;)`. It is used for printing the function and for error messages. This field is immutable.

&codevec;

  This is the byte-code vector, a &ubyte-vec;. This field is immutable.

&consts;[]

  The remaining fields in the function object are references to other &clisp; objects. These references are immutable, which is why they are called constants. (The referenced &clisp; objects can be mutable objects, such as &cons-t;es or &vector-t;s, however.)

#### The Exception to the Immutability Rule

When a generic function's dispatch code is installed, the &codevec; and &consts; fields are destructively modified.

Some of the &consts; can play special roles. A function looks like this, in more detail:

name

  see .

&codevec;

  see .

&venv-const;*

  At most one object, representing the closed-up variables, representing the variables of the &lex-env; in which this function was defined. It is a &simple-vector-t;, which looks like this: `#(&next-r; value&sub-1; ... value&sub-n;)` where value&sub-1;, ..., value&sub-n; are the values of the closed-up variables, and &next-r; is either &nil; or a &simple-vector-t; having the same structure.

&block-const;*

  Objects representing closed-up &block; tags, representing the &block; tags of the &lex-env; in which this function was defined. Each is a &cons-t; containing in the &cdr; part: either a frame pointer to the block frame, or &disabled;. The &car; is the block's name, for error messages only.

&tagbody-const;*

  Objects representing closed-up &tagbody; tags, representing the &tagbody; tags of the &lex-env; in which this function was defined. Each is a &cons-t; containing in the &cdr; part: either a frame pointer to the &tagbody; frame, or &disabled; if the &tagbody; has already been left. The &car; is a &simple-vector-t; containing the names of the &tagbody; tags, for error messages only.

&keyword-const;*

  If the function was defined with a &lalist; containing &key-amp;, here come the symbols ("keywords"), in their correct order. They are used by the interpreter during function call.

&other-const;*

  Other objects needed by the function's bytecode.

If &venv-const;, &block-const;, &tagbody-const; are all absent, the function is called autonomous. This is the case if the function does not refer to lexical variables, blocks or tags defined in compile code outside of the function. In particular, it is the case if the function is defined in a null &lex-env;.

If some &venv-const;, &block-const;, or &tagbody-const; are present, the function (a closure) is created at runtime. The compiler only generates a prototype, containing &nil; values instead of each &venv-const;, &block-const;, &tagbody-const;. At runtime, a function is created by copying this prototype and replacing the &nil; values by the definitive ones.

The list `(&keyword-const;* &other-const;*)` normally does not contain duplicates, because the compiler removes duplicates when possible. (Duplicates can occur nevertheless, through the use of &load-time-value;.)

The &codevec; looks like this (cf. the &c-lang; type Codevec):

`spdepth_1` (2 bytes)

  The 1st part of the maximal &SP; depth.

`spdepth_jmpbufsize` (2 bytes)

  The &jbs; part of the maximal &SP; depth. The maximal &SP; depth (precomputed by the compiler) is given by `spdepth_1 + spdepth_jmpbufsize * &jbs;`.

&numreq; (2 bytes)

  Number of required parameters.

`numopt` (2 bytes)

  Number of optional parameters.

&flags; (1 byte)
  bit 0

    set if the function has the &rest-amp; parameter

  bit 7

    set if the function has &key-amp; parameters

  bit 6

    set if the function has &allow-other-keys-amp;

  bit 4

    set if the function is a generic function

  bit 3

    set if the function is a generic function and its effective method shall be returned (instead of being executed)

`signature` (1 byte)

  An abbreviation code depending on &numreq;, `numopt`, &flags;. It is used for speeding up the function call.

`numkey` (2 bytes, only if the function has &key-amp;)

  The number of &key-amp; parameters.

`keyconsts` (2 bytes, only if the function has &key-amp;)

  The offset of the &keyword-const; in the function.

`byte`* (any number of bytes)

  The bytecode instructions.

### The general structure of the instructions

All instructions consist of one byte, denoting the opcode, and some number of operands.

The conversion from a byte (in the range 0..255) to the opcode is performed by lookup in the table contained in the file &bytecode-d;.

There are the following types of operands, denoted by different letters:

&k-r;, &n-r;, &m-r;, &l-r;

  A (nonnegative) numeric operand. The next byte is read. If its bit 7 is zero, then the bits 6..0 give the value (7 bits). If its bit 7 is one, then the bits 6..0 and the subsequent byte together form the value (15 bits).

&b-r;

  A (nonnegative) 1-byte operand. The next byte is read and is the value.

label

  A label operand. A signed numeric operand is read: The next byte is read. If its bit 7 is zero, then the bits 6..0 give the value (7 bits, sign-extended). If its bit 7 is one, then the bits 6..0 and the subsequent byte together form the value (15 bits, sign-extended). If the latter 15-bit result is zero, then four more bytes are read and put together (32 bits, sign-extended). Finally, the bytecode pointer for the target is computed as the current bytecode pointer (pointing after the operand just read), plus the signed numeric operand.

### The instruction set

#### Instructions for constants

&mnem-desc-sem-header;
`(&nil;)` | Load &nil; into values. | &value1; := &nil;, &mv-count; := 1
`(PUSH-NIL &n-r;)` | Push &n-r; &nil;s into the &STACK;. | &n-r; times do: *--&STACK; := &nil;, values undefined
`(&t;)` | Load &t; into values. | &value1; := &t;, &mv-count; := 1
`(CONST &n-r;)` | Load the function's &n-r;th constant into values. | &value1; := &consts;[&n-r;], &mv-count; := 1

#### Instructions for lexical variables

&mnem-desc-sem-header;
`(LOAD &n-r;)` | Load a directly accessible local variable into values. | &value1; := *(&STACK;+&n-r;), &mv-count; := 1
`(LOADI &k1-r; &k2-r; &n-r;)` | Load an indirectly accessible local variable into values. | &k-r; := &k1-r; + &jbs; * &k2-r;, &value1; := *(*(&SP;+&k-r;)+ &n-r;), &mv-count; := 1
`(LOADC &n-r; &m-r;)` | Load a closed-up variable, defined in the same function and directly accessible, into values. | &value1; := &svref;(*(&STACK;+&n-r;),1+&m-r;), &mv-count; := 1
`(LOADV &k-r; &m-r;)` | Load a closed-up variable, defined in an outer function, into values. | &v-r; := &venv-const;, &m-r; times do: &v-r; := &svref;(&v-r;,0), &value1; := &svref;(&v-r;,&m-r;), &mv-count; := 1
`(LOADIC &k1-r; &k2-r; &n-r; &m-r;)` | Load a closed-up variable, defined in the same function and indirectly accessible, into values. | &k-r; := &k1-r; + &jbs; * &k2-r;, &value1; := &svref;(*(*(&SP;+&k-r;)+&n-r;),1+&m-r;), &mv-count; := 1
`(STORE &n-r;)` | Store values into a directly accessible local variable. | *(&STACK;+&n-r;) := &value1;, &mv-count; := 1
`(STOREI &k1-r; &k2-r; &n-r;)` | Store values into an indirectly accessible local variable. | &k-r; := &k1-r; + &jbs; * &k2-r;, *(*(&SP;+&k-r;)+ &n-r;) := &value1;, &mv-count; := 1
`(STOREC &n-r; &m-r;)` | Store values into a closed-up variable, defined in the same function and directly accessible. | &svref;(*(&STACK;+&n-r;),1+m) := &value1;, &mv-count; := 1
`(STOREV &k-r; &m-r;)` | Store values into a closed-up variable, defined in an outer function. | &v-r; := &venv-const;, &m-r; times do: &v-r; := &svref;(&v-r;,0), &svref;(&v-r;,&m-r;) := &value1;, &mv-count; := 1
`(STOREIC &k1-r; &k2-r; &n-r; &m-r;)` | Store values into a closed-up variable, defined in the same function and indirectly accessible. | &k-r; := &k1-r; + &jbs; * &k2-r;, &svref;(*(*(&SP;+&k-r;)+&n-r;),1+&m-r;) := &value1;, &mv-count; := 1

#### Instructions for dynamic variables

&mnem-desc-sem-header;
`(GETVALUE &n-r;)` | Load a symbol's value into values. | &value1; := symbol-value(&consts;[&n-r;]), &mv-count; := 1
`(SETVALUE &n-r;)` | Store values into a symbol's value. | symbol-value(&consts;[&n-r;]) := &value1;, &mv-count; := 1
`(BIND &n-r;)` | Bind a symbol dynamically. | Bind the value of the symbol &consts;[&n-r;] to &value1;, implicitly &STACK; -= 3, values undefined
`(UNBIND1)` | Dissolve one binding frame. | Unbind the binding frame &STACK; is pointing to, implicitly &STACK; += 3
`(UNBIND &n-r;)` | Dissolve &n-r; binding frames. | &n-r; times do: Unbind the binding frame &STACK; is pointing to, thereby incrementing &STACK; Thus, &STACK; += 1+2*&n-r;
`(PROGV)` | Bind a set of symbols dynamically to a set of values. | symbols := *&STACK;++, *--&SP; := &STACK;, build a single binding frame binding the symbols in symbols to the values in &value1;, values undefined

#### Instructions for stack operations

&mnem-desc-sem-header;
`(PUSH)` | Push one object onto the &STACK;. | *--&STACK; := &value1;, values undefined
`(POP)` | Pop one object from the &STACK;, into values. | &value1; := *&STACK;++, &mv-count; := 1
`(SKIP &n-r;)` | Restore a previous &STACK; pointer. Remove &n-r; objects from the &STACK;. | &STACK; := &STACK; + &n-r;
`(SKIPI &k1-r; &k2-r; &n-r;)` | Restore a previous &STACK; pointer. Remove an unknown number of objects from the &STACK;. | &k-r; := &k1-r; + &jbs; * &k2-r;, &STACK; := *(&SP;+&k-r;), &SP; := &SP;+&k-r;+1, &STACK; := &STACK; + &n-r;
`(SKIPSP &k1-r; &k2-r;)` | Restore a previous &SP; pointer. | &k-r; := &k1-r; + &jbs; * &k2-r;, &SP; := &SP;+&k-r;

#### Instructions for control flow, jumps

&mnem-desc-sem-header;
`(SKIP&RET &n-r;)` | Clean up the &STACK;, and return from the function. | &STACK; := &STACK;+&n-r;, return from the function, returning values.

`(SKIP&RETGF &n-r;)`

Clean up the &STACK;, and return from the generic function.

If bit 3 is set in the function's &flags;, then &STACK; := &STACK;+&n-r;, &mv-count; := 1, and return from the function. Otherwise: if the current function has no &rest-amp; argument, then &STACK; := &STACK;+&n-r;-&numreq;, apply &value1; to the &numreq; arguments still on the &STACK;, and return from the function. Else &STACK; := &STACK;+&n-r;-&numreq;-1, apply &value1; to the &numreq; arguments and the &rest-amp; argument, all still on the &STACK;, and return from the function.

`(JMP &lab-r;)` | Jump to &lab-r;. | PC := &lab-r;.
`(JMPIF &lab-r;)` | Jump to &lab-r;, if &value1; is true. | If &value1; is not &nil;, PC := &lab-r;.
`(JMPIFNOT &lab-r;)` | Jump to &lab-r;, if &value1; is false. | If &value1; is &nil;, PC := &lab-r;.
`(JMPIF1 &lab-r;)` | Jump to &lab-r; and forget secondary values, if &value1; is true. | If &value1; is not &nil;, &mv-count; := 1, PC := &lab-r;.
`(JMPIFNOT1 &lab-r;)` | Jump to &lab-r; and forget secondary values, if &value1; is false. | If &value1; is &nil;, &mv-count; := 1, PC := &lab-r;.
`(JMPIFATOM &lab-r;)` | Jump to &lab-r;, if &value1; is not a cons. | If &value1; is not a cons, PC := &lab-r;. values undefined
`(JMPIFCONSP &lab-r;)` | Jump to &lab-r;, if &value1; is a cons. | If &value1; is a cons, PC := &lab-r;. values undefined
`(JMPIFEQ &lab-r;)` | Jump to &lab-r;, if &value1; is &eq; to the top-of-stack. | If eq(&value1;,*&STACK;++), PC := &lab-r;. values undefined
`(JMPIFNOTEQ &lab-r;)` | Jump to &lab-r;, if &value1; is not &eq; to the top-of-stack. | If not eq(&value1;,*&STACK;++), PC := &lab-r;. values undefined
`(JMPIFEQTO &n-r; &lab-r;)` | Jump to &lab-r;, if the top-of-stack is &eq; to a constant. | If eq(*&STACK;++,&consts;[&n-r;]), PC := &lab-r;. values undefined
`(JMPIFNOTEQTO &n-r; &lab-r;)` | Jump to &lab-r;, if the top-of-stack is not &eq; to a constant. | If not eq(*&STACK;++,&consts;[&n-r;]), PC := &lab-r;. values undefined
`(JMPHASH &n-r; &lab-r;)` | Table-driven jump, depending on &value1;. | Lookup &value1; in the hash table &consts;[&n-r;]. (The hash table's test is either &eq; or &eql;.) If found, the hash table value is a signed &fixnum-t;, jump to it: PC := PC + value. Else jump to &lab-r;. values undefined
`(JMPHASHV &n-r; &lab-r;)` | Table-driven jump, depending on &value1;, inside a generic function. | Lookup &value1; in the hash table &svref;(&consts;[0],&n-r;). (The hash table's test is either &eq; or &eql;.) If found, the hash table value is a signed &fixnum-t;, jump to it: PC := PC + value. Else jump to &lab-r;. values undefined
`(JSR &lab-r;)` | Subroutine call. | *--&STACK; := function. Then start interpreting the bytecode at &lab-r;, with values undefined. When a `(RET)` is encountered, program execution is resumed at the instruction after `(JSR &lab-r;)`.
`(JMPTAIL &m-r; &n-r; &lab-r;)` | Tail subroutine call. | &n-r; &geq; &m-r;. The &STACK; frame of size &n-r; is reduced to size &m-r;: {*(&STACK;+&n-r;-&m-r;), ..., *(&STACK;+&n-r;-1)} := {*&STACK;, ..., *(&STACK;+&m-r;-1)}. &STACK; += n-m. *--&STACK; := function. Then jump to &lab-r;, with values undefined.

#### Instructions for &lex-env;, creation of closures

&mnem-desc-sem-header;
`(VENV)` | Load the &venv-const; into values. | &value1; := &consts;[0], &mv-count; := 1.
`(MAKE-VECTOR1&PUSH &n-r;)` | Create a &simple-vector-t; used for closed-up variables. | &v-r; := new &simple-vector-t; of size &n-r;+1. &svref;(&v-r;,0) := &value1;. *--&STACK; := &v-r;. values undefined
`(COPY-CLOSURE &m-r; &n-r;)` | Create a closure by copying the prototype and filling in the &lex-env;. | &f-r; := copy-function(&consts;[&m-r;]). For &i-r;=0,..,&n-r;-1: &f-r;_&consts;[i] := *(&STACK;+&n-r;-1-&i-r;). &STACK; += &n-r;. &value1; := &f-r;, &mv-count; := 1

#### Instructions for function calls

&mnem-desc-sem-header;
`(CALL &k-r; &n-r;)` | Calls a constant function with &k-r; arguments. | The function &consts;[&n-r;] is called with the arguments *(&STACK;+&k-r;-1), ..., *(&STACK;+0). &STACK; += &k-r;. The returned values go into values.
`(CALL0 &n-r;)` | Calls a constant function with 0 arguments. | The function &consts;[&n-r;] is called with 0 arguments. The returned values go into values.
`(CALL1 &n-r;)` | Calls a constant function with 1 argument. | The function &consts;[&n-r;] is called with one argument *&STACK;. &STACK; += 1. The returned values go into values.
`(CALL2 &n-r;)` | Calls a constant function with 2 arguments. | The function &consts;[&n-r;] is called with two arguments *(&STACK;+1) and *(&STACK;+0). &STACK; += 2. The returned values go into values.
`(CALLS1 &b-r;)` | Calls a system function with no &rest-amp;. | Calls the system function FUNTAB[&b-r;]. The right number of arguments is already on the &STACK; (including &unbound;s in place of absent &optional-amp; or &key-amp; parameters). The arguments are removed from the &STACK;. The returned values go into values.
`(CALLS2 &b-r;)` | Calls a system function with no &rest-amp;. | Calls the system function FUNTAB[256+&b-r;]. The right number of arguments is already on the &STACK; (including &unbound;s in place of absent &optional-amp; or &key-amp; parameters). The arguments are removed from the &STACK;. The returned values go into values.
`(CALLSR &m-r; &b-r;)` | Calls a system function with &rest-amp;. | Calls the system function FUNTABR[&b-r;]. The minimum number of arguments is already on the &STACK;, and &m-r; additional arguments as well. The arguments are removed from the &STACK;. The returned values go into values.
`(CALLC)` | Calls a computed compiled function with no &key-amp;. | Calls the compiled function &value1;. The right number of arguments is already on the &STACK; (including &unbound;s in place of absent &optional-amp; parameters). The arguments are removed from the &STACK;. The returned values go into values.
`(CALLCKEY)` | Calls a computed compiled function with &key-amp;. | Calls the compiled function &value1;. The right number of arguments is already on the &STACK; (including &unbound;s in place of absent &optional-amp; or &key-amp; parameters). The arguments are removed from the &STACK;. The returned values go into values.
`(FUNCALL &n-r;)` | Calls a computed function. | Calls the function *(&STACK;+&n-r;) with the arguments *(&STACK;+&n-r;-1), ..., *(&STACK;+0). &STACK; += &n-r;+1. The returned values go into values.
`(APPLY &n-r;)` | Calls a computed function with an unknown number of arguments. | Calls the function *(&STACK;+&n-r;) with the arguments *(&STACK;+&n-r;-1), ..., *(&STACK;+0) and a list of additional arguments &value1;. &STACK; += &n-r;+1. The returned values go into values.

#### Instructions for optional and keyword parameters

&mnem-desc-sem-header;
`(PUSH-UNBOUND &n-r;)` | Push &n-r; &unbound;s into the &STACK;. | &n-r; times do: *--&STACK; := &unbound;. values undefined
`(UNLIST &n-r; &m-r;)` | Destructure a proper &list-t;. | 0 ≤ &m-r; ≤ &n-r;. &n-r; times do: *--&STACK; := &car;(&value1;), &value1; := &cdr;(&value1;). During the last &m-r; iterations, the list &value1; may already have reached its end; in this case, *--&STACK; := &unbound;. At the end, &value1; must be &nil;. values undefined
`(UNLIST* &n-r; &m-r;)` | Destructure a proper or dotted &list-t;. | 0 ≤ &m-r; ≤ &n-r;, &n-r; > 0. &n-r; times do: *--&STACK; := &car;(&value1;), &value1; := &cdr;(&value1;). During the last &m-r; iterations, the list &value1; may already have reached its end; in this case, *--&STACK; := &unbound;. At the end, after &n-r; &cdr;s, *--&STACK; := &value1;. values undefined
`(JMPIFBOUNDP &n-r; &lab-r;)` | Jump to &lab-r;, if a local variable is not unbound. | If *(&STACK;+&n-r;) is not &unbound;, &value1; := *(&STACK;+&n-r;), &mv-count; := 1, PC := &lab-r;. Else: values undefined.
`(BOUNDP &n-r;)` | Load &t; or &nil; into values, depending on whether a local variable is bound. | If *(&STACK;+&n-r;) is not &unbound;, &value1; := &t;, &mv-count; := 1. Else: &value1; := &nil;, &mv-count; := 1.
`(UNBOUND->NIL &n-r;)` | If a local variable is unbound, assign a default value &nil; to it. | If *(&STACK;+&n-r;) is &unbound;, *(&STACK;+&n-r;) := &nil;.

#### Instructions for &mul-val;

&mnem-desc-sem-header;
`(VALUES0)` | Load no values into values. | &value1; := &nil;, &mv-count; := 0
`(VALUES1)` | Forget secondary values. | &mv-count; := 1
`(&STACK;-TO-MV &n-r;)` | Pop the first &n-r; objects from &STACK; into values. | Load values(*(&STACK;+&n-r;-1),...,*(&STACK;+0)) into values. &STACK; += &n-r;.
`(MV-TO-&STACK;)` | Save values on &STACK;. | Push the &mv-count; values onto the &STACK; (in order: &value1; comes first). &STACK; -= &mv-count;. values undefined
`(NV-TO-&STACK; &n-r;)` | Save &n-r; values on &STACK;. | Push the first &n-r; values onto the &STACK; (in order: &value1; comes first). &STACK; -= &n-r;. values undefined
`(MV-TO-LIST)` | Convert &mul-val; into a list. | &value1; := list of values, &mv-count; := 1
`(LIST-TO-MV)` | Convert a &list-t; into &mul-val;. | Call the function &values-list; with &value1; as argument. The returned values go into values.
`(MVCALLP)` | Start a &multiple-value-call; invocation. | *--&SP; := &STACK;. *--&STACK; := &value1;.
`(MVCALL)` | Finish a &multiple-value-call; invocation. | newSTACK := *&SP;++. Call the function *(newSTACK-1), passing it *(newSTACK-2), ..., *(&STACK;+0) as arguments. &STACK; := newSTACK. The returned values go into values.

#### Instructions for &block; and &return-from;

&mnem-desc-sem-header;
`(BLOCK-OPEN &n-r; &lab-r;)` | Create a &block; frame. | Create a &block; frame, &STACK; -= 3, &SP; -= 2+&jbs;. The topmost (third) object in the block frame is &cons;(&consts;[&n-r;],frame-pointer) (its &bc-r;). Upon a &return-from; to this frame, execution will continue at &lab-r;. values undefined.
`(BLOCK-CLOSE)` | Dissolve a &block; frame. | Dissolve the &block; frame at &STACK;, &STACK; += 3, &SP; += 2+&jbs;. Mark the &bc-r; as invalid.
`(RETURN-FROM &n-r;)` | Leave a &block; whose &bc-r; is given. | &bc-r; := &consts;[&n-r;]. If &cdr;(&bc-r;) = &disabled;, an &err-sig;. Else &cdr;(&bc-r;) is a frame-pointer. Unwind the stack up to this frame, pass it values.
`(RETURN-FROM-I &k1-r; &k2-r; &n-r;)` | Leave a &block; whose &bc-r; is indirectly accessible. | &k-r; := &k1-r; + &jbs; * &k2-r;, &bc-r; := *(*(&SP;+&k-r;)+&n-r;). If &cdr;(&bc-r;) = &disabled;, an &err-sig;. Else &cdr;(&bc-r;) is a frame-pointer. Unwind the stack up to this frame, pass it values.

#### Instructions for &tagbody; and &go;

&mnem-desc-sem-header;

`(TAGBODY-OPEN &m-r; label&sub-1; ... label&sub-n;)`

Create a &tagbody; frame.

Fetch &consts;[&m-r;], this is a &simple-vector-t; with &n-r; elements, then decode &n-r; label operands. Create a &tagbody; frame, &STACK; -= 3+&n-r;, &SP; -= 1+&jbs;. The third object in the &tagbody; frame is &cons;(&consts;[&m-r;],frame-pointer) (the &tbc-r;) Upon a &go; to tag &lab-r; of this frame, execution will continue at label&sub-l;. values undefined

`(TAGBODY-CLOSE-NIL)` | Dissolve a &tagbody; frame, and load &nil; into values. | Dissolve the &tagbody; frame at &STACK;, &STACK; += 3+&m-r;, &SP; += 1+&jbs;. Mark the &tbc-r; as invalid. &value1; := &nil;, &mv-count; := 1.
`(TAGBODY-CLOSE)` | Dissolve a &tagbody; frame. | Dissolve the &tagbody; frame at &STACK;, &STACK; += 3+&m-r;, &SP; += 1+&jbs;. Mark the &tbc-r; as invalid.
`(GO &n-r; &lab-r;)` | Jump into a &tagbody; whose &tbc-r; is given. | &tbc-r; := &consts;[&n-r;]. If &cdr;(&tbc-r;) = &disabled;, an &err-sig;. Else &cdr;(&tbc-r;) is a frame-pointer. Unwind the stack up to this frame, pass it the number &lab-r;.
`(GO-I &k1-r; &k2-r; &n-r; &lab-r;)` | Jump into a &tagbody; whose &tbc-r; is indirectly accessible. | &k-r; := &k1-r; + &jbs; * &k2-r;, &tbc-r; := *(*(&SP;+&k-r;)+&n-r;). If &cdr;(&tbc-r;) = &disabled;, an &err-sig;. Else &cdr;(&tbc-r;) is a frame-pointer. Unwind the stack up to this frame, pass it the number &lab-r;.

#### Instructions for &catch; and &throw;

&mnem-desc-sem-header;
`(CATCH-OPEN &lab-r;)` | Create a &catch; frame. | Create a &catch; frame, with &value1; as tag. &STACK; -= 3, &SP; -= 2+&jbs;. Upon a &throw; to this tag execution continues at &lab-r;.
`(CATCH-CLOSE)` | Dissolve a &catch; frame. | Dissolve the &catch; frame at &STACK;. &STACK; += 3, &SP; += 2+&jbs;.
`(THROW)` | Non-local exit to a &catch; frame. | &tag-r; := *&STACK;++. Search the innermost &catch; frame with tag &tag-r; on the &STACK;, unwind the stack up to it, pass it values.

#### Instructions for &unwind-protect;

&mnem-desc-sem-header;
`(UNWIND-PROTECT-OPEN &lab-r;)` | Create an &unwind-protect; frame. | Create an &unwind-protect; frame. &STACK; -= 2, &SP; -= 2+&jbs;. When the stack will be unwound by a non-local exit, values will be saved on &STACK;, and execution will be transferred to &lab-r;.
`(UNWIND-PROTECT-NORMAL-EXIT)` | Dissolve an &unwind-protect; frame, and start the cleanup code. | Dissolve the &unwind-protect; frame at &STACK;. &STACK; += 2, &SP; += 2+&jbs;. *--&SP; := 0, *--&SP; := 0, *--&SP; := &STACK;. Save the values on the &STACK;, &STACK; -= &mv-count;.
`(UNWIND-PROTECT-CLOSE)` | Terminate the cleanup code. | newSTACK := *&SP;++. Load values(*(newSTACK-1), ..., *(&STACK;+0)) into values. &STACK; := newSTACK. SPword1 := *&SP;++, SPword2 := *&SP;++. Continue depending on SPword1 and SPword2. If both are 0, simply continue execution. If SPword2 is 0 but SPword1 is nonzero, interpret it as a label and jump to it.
`(UNWIND-PROTECT-CLEANUP)` | Dissolve an &unwind-protect; frame, and execute the cleanup code like a subroutine call. | Dissolve the &unwind-protect; frame at &STACK;, get &lab-r; out of the frame. &STACK; += 2, &SP; += 2+&jbs;. *--&SP; := 0, *--&SP; := PC, *--&SP; := &STACK;. Save the values on the &STACK;, &STACK; -= &mv-count;. PC := &lab-r;.

#### Instructions for &handler-bind;

&mnem-desc-sem-header;
`(HANDLER-OPEN &n-r;)` | Create a handler frame. | Create a handler frame, using &consts;[&n-r;] which contains the &condition-t; types, the corresponding labels and the current &SP; depth (= function entry &SP; - current &SP;).
`(HANDLER-BEGIN&PUSH)` | Start a handler. | Restore the same &SP; state as after the HANDLER-OPEN. &value1; := the &condition-t; that was passed to the handler, &mv-count; := 1. *--&STACK; := &value1;.

#### Instructions for some inlined functions

&mnem-desc-sem-header;
`(NOT)` | Inlined call to &not-f;. | &value1; := not(&value1;), &mv-count; := 1.
`(EQ)` | Inlined call to &eq;. | &value1; := eq(*&STACK;++,&value1;), &mv-count; := 1.
`(CAR)` | Inlined call to &car;. | &value1; := &car;(&value1;), &mv-count; := 1.
`(CDR)` | Inlined call to &cdr;. | &value1; := &cdr;(&value1;), &mv-count; := 1.
`(CONS)` | Inlined call to &cons;. | &value1; := cons(*&STACK;++,&value1;), &mv-count; := 1.
`(SYMBOL-FUNCTION)` | Inlined call to &symbol-function;. | &value1; := &symbol-function;(&value1;), &mv-count; := 1.
`(SVREF)` | Inlined call to &svref;. | &value1; := &svref;(*&STACK;++,&value1;), &mv-count; := 1.
`(SVSET)` | Inlined call to `(&setf; &svref;`. | arg1 := *(&STACK;+1), arg2 := *(&STACK;+0), &STACK; += 2. &svref;(arg2,&value1;) := arg1. &value1; := arg1, &mv-count; := 1.
`(LIST &n-r;)` | Inlined call to &list;. | &value1; := &list;(*(&STACK;+&n-r;-1),...,*(&STACK;+0)), &mv-count; := 1, &STACK; += &n-r;.
`(LIST* &n-r;)` | Inlined call to &list-star;. | &value1; := &list-star;(*(&STACK;+&n-r;-1),..., *(&STACK;+0),&value1;), &mv-count; := 1, &STACK; += &n-r;.

#### Combined instructions

The most frequent short sequences of instructions have an equivalent combined instruction. They are only present for space and speed optimization. The only exception is `FUNCALL&SKIP&RETGF`, which is needed for generic functions.

mnemonic | equivalent
--- | ---
`(NIL&PUSH)` | `(NIL) (PUSH)`
`(T&PUSH)` | `(T) (PUSH)`
`(CONST&PUSH &n-r;)` | `(CONST &n-r;) (PUSH)`
`(LOAD&PUSH &n-r;)` | `(LOAD &n-r;) (PUSH)`
`(LOADI&PUSH &k1-r; &k2-r; &n-r;)` | `(LOADI &k1-r; &k2-r; &n-r;) (PUSH)`
`(LOADC&PUSH &n-r; &m-r;)` | `(LOADC &n-r; &m-r;) (PUSH)`
`(LOADV&PUSH &k-r; &m-r;)` | `(LOADV &k-r; &m-r;) (PUSH)`
`(POP&STORE &n-r;)` | `(POP) (STORE &n-r;)`
`(GETVALUE&PUSH &n-r;)` | `(GETVALUE &n-r;) (PUSH)`
`(JSR&PUSH &lab-r;)` | `(JSR &lab-r;) (PUSH)`
`(COPY-CLOSURE&PUSH &m-r; &n-r;)` | `(COPY-CLOSURE &m-r; &n-r;) (PUSH)`
`(CALL&PUSH &k-r; &n-r;)` | `(CALL &k-r; &n-r;) (PUSH)`
`(CALL1&PUSH &n-r;)` | `(CALL1 &n-r;) (PUSH)`
`(CALL2&PUSH &n-r;)` | `(CALL2 &n-r;) (PUSH)`
`(CALLS1&PUSH &b-r;)` | `(CALLS1 &b-r;) (PUSH)`
`(CALLS2&PUSH &b-r;)` | `(CALLS2 &b-r;) (PUSH)`
`(CALLSR&PUSH &m-r; &n-r;)` | `(CALLSR &m-r; &n-r;) (PUSH)`
`(CALLC&PUSH)` | `(CALLC) (PUSH)`
`(CALLCKEY&PUSH)` | `(CALLCKEY) (PUSH)`
`(FUNCALL&PUSH &n-r;)` | `(FUNCALL &n-r;) (PUSH)`
`(APPLY&PUSH &n-r;)` | `(APPLY &n-r;) (PUSH)`
`(CAR&PUSH)` | `(CAR) (PUSH)`
`(CDR&PUSH)` | `(CDR) (PUSH)`
`(CONS&PUSH)` | `(CONS) (PUSH)`
`(LIST&PUSH &n-r;)` | `(LIST &n-r;) (PUSH)`
`(LIST*&PUSH &n-r;)` | `(LIST* &n-r;) (PUSH)`
`(NIL&STORE &n-r;)` | `(NIL) (STORE &n-r;)`
`(T&STORE &n-r;)` | `(T) (STORE &n-r;)`
`(LOAD&STOREC &k-r; &n-r; &m-r;)` | `(LOAD &k-r;) (STOREC &n-r; &m-r;)`
`(CALLS1&STORE &b-r; &k-r;)` | `(CALLS1 &b-r;) (STORE &k-r;)`
`(CALLS2&STORE &b-r; &k-r;)` | `(CALLS2 &b-r;) (STORE &k-r;)`
`(CALLSR&STORE &m-r; &n-r; &k-r;)` | `(CALLSR &m-r; &n-r;) (STORE &k-r;)`
`(LOAD&CDR&STORE &n-r;)` | `(LOAD &n-r;) (CDR) (STORE &n-r;)`
`(LOAD&CONS&STORE &n-r;)` | `(LOAD &n-r;+1) (CONS) (STORE &n-r;)`
`(LOAD&INC&STORE &n-r;)` | `(LOAD &n-r;) (CALL1 #'1+) (STORE &n-r;)`
`(LOAD&DEC&STORE &n-r;)` | `(LOAD &n-r;) (CALL1 #'1-) (STORE &n-r;)`
`(LOAD&CAR&STORE &m-r; &n-r;)` | `(LOAD &m-r;) (CAR) (STORE &n-r;)`
`(CALL1&JMPIF &n-r; &lab-r;)` | `(CALL1 &n-r;) (JMPIF &lab-r;)`
`(CALL1&JMPIFNOT &n-r; &lab-r;)` | `(CALL1 &n-r;) (JMPIFNOT &lab-r;)`
`(CALL2&JMPIF &n-r; &lab-r;)` | `(CALL2 &n-r;) (JMPIF &lab-r;)`
`(CALL2&JMPIFNOT &n-r; &lab-r;)` | `(CALL2 &n-r;) (JMPIFNOT &lab-r;)`
`(CALLS1&JMPIF &b-r; &lab-r;)` | `(CALLS1 &b-r;) (JMPIF &lab-r;)`
`(CALLS1&JMPIFNOT &b-r; &lab-r;)` | `(CALLS1 &b-r;) (JMPIFNOT &lab-r;)`
`(CALLS2&JMPIF &b-r; &lab-r;)` | `(CALLS2 &b-r;) (JMPIF &lab-r;)`
`(CALLS2&JMPIFNOT &b-r; &lab-r;)` | `(CALLS2 &b-r;) (JMPIFNOT &lab-r;)`
`(CALLSR&JMPIF &m-r; &n-r; &lab-r;)` | `(CALLSR &m-r; &n-r;) (JMPIF &lab-r;)`
`(CALLSR&JMPIFNOT &m-r; &n-r; &lab-r;)` | `(CALLSR &m-r; &n-r;) (JMPIFNOT &lab-r;)`
`(LOAD&JMPIF &n-r; &lab-r;)` | `(LOAD &n-r;) (JMPIF &lab-r;)`
`(LOAD&JMPIFNOT &n-r; &lab-r;)` | `(LOAD &n-r;) (JMPIFNOT &lab-r;)`
`(LOAD&CAR&PUSH &n-r;)` | `(LOAD &n-r;) (CAR) (PUSH)`
`(LOAD&CDR&PUSH &n-r;)` | `(LOAD &n-r;) (CDR) (PUSH)`
`(LOAD&INC&PUSH &n-r;)` | `(LOAD &n-r;) (CALL1 #'1+) (PUSH)`
`(LOAD&DEC&PUSH &n-r;)` | `(LOAD &n-r;) (CALL1 #'1-) (PUSH)`
`(CONST&SYMBOL-FUNCTION &n-r;)` | `(CONST &n-r;) (SYMBOL-FUNCTION)`
`(CONST&SYMBOL-FUNCTION&PUSH &n-r;)` | `(CONST &n-r;) (SYMBOL-FUNCTION) (PUSH)`
`(CONST&SYMBOL-FUNCTION&STORE &n-r; &k-r;)` | `(CONST &n-r;) (SYMBOL-FUNCTION) (STORE &k-r;)`
`(APPLY&SKIP&RET &n-r; &k-r;)` | `(APPLY &n-r;) (SKIP&RET &k-r;)`
`(FUNCALL&SKIP&RETGF &n-r; &k-r;)` | `(FUNCALL &n-r;) (SKIP&RETGF &k-r;)`

#### Shortcut instructions

There are special one-byte instructions (without explicit operands) for the following frequent instructions:

mnemonic | operand range
--- | ---
`(LOAD &n-r;)` | 0 ≤ &n-r; < 15
`(LOAD&PUSH &n-r;)` | 0 ≤ &n-r; < 25
`(CONST &n-r;)` | 0 ≤ &n-r; < 21
`(CONST&PUSH &n-r;)` | 0 ≤ &n-r; < 30
`(STORE &n-r;)` | 0 ≤ &n-r; < 8

### Bytecode Design

This section offers some insight into bytecode design in the form of questions and answers.

#### When to add a new bytecode?

##### Question:

Does it make sense to define a new bytecode instruction for &restart-case;? Why? Why not?

##### Answer: Is it speed critical?

&restart-case; is a glorified &let; binding for `SYSTEM::*ACTIVE-RESTARTS*` and could well profit from a separate bytecode: it would make it non-&consing;. (Remember that &restart-t;s have dynamic extent and therefore do not really need to be heap allocated.)

The reason &handler-bind; has its own bytecodes and &restart-case; does not is that &handler-bind; can occur in inner computation loops, whereas &restart-case; occurs only as part of user-interface programming and therefore not in inner loops where its consing could hurt much.

#### Why JMPTAIL?

##### Question:

Consider this function and its disassembly:

     (defun foo (x y) (if (or (= x 0) (= y 0)) (+ x y) (foo y (1- x))))
     (&disassemble; 'foo)
    8     (LOAD&PUSH 1)
    9     (LOAD&DEC&PUSH 3)
    11    (JMPTAIL 2 5 L0)

Why are the arguments pushed onto the &STACK;, just to be popped off of it during the `JMPTAIL`? Why not a sequence of `LOAD`, `STORE` and `SKIP` instructions followed by a `JMP`?

##### Answer: This is a shortcut for the most common use

Using `JMPTAIL` requires 3 instructions, `JMP` requires more. When `JMPTAIL` needs to be called, we usually have some stuff close to the top of the &STACK; which will become the new arguments, and some junk between these new arguments and the closure object. `JMPTAIL` removes the junk. `JMPTAIL` is a convenient shortcut which shortens the bytecode - because typically one would really have to clean-up the &STACK; by hand or make the calculations in `src/compiler.lisp` more complicated.