Does per-type memory consumption make sense here?

[aartaka]

Hi again! In the spirit of #7, does per-type memory consumption breakdown belong here? SBCL, ECL (non-Boehm), CLISP, and CCL (and Allegro, but I haven't disassembled this bit yet) all provide this breakdown in their room, so why not incorporate it too?

[Shinmera]

Sure why not. Though how consistent are they about the breakdown of the types being counted?

[aartaka]

Consistent in what way exactly? I'm not sure I understood that bit...

[Shinmera]

Like, what types of objects do they actually count separately in their statistics? Are they the same types for each implementation?

[aartaka]

Like, what types of objects do they actually count separately in their statistics?

Here's a bit of listing. SBCL

Top 10 dynamic instance types:
  SB-C::LVAR                          4,422,800 bytes,  55,285 objects.
  SB-C::REF                           4,052,608 bytes,  36,184 objects.
  SB-C::CTRAN                         3,679,200 bytes,  76,650 objects.
  SB-C::CBLOCK                        3,264,320 bytes,  20,402 objects.
  SB-C::LAMBDA-VAR                    2,854,016 bytes,  16,216 objects.
  SB-KERNEL:LEXENV                    2,751,744 bytes,  21,498 objects.
  SB-C::CLAMBDA                       2,537,792 bytes,   8,348 objects.
  SB-C::COMBINATION                   2,045,600 bytes,  12,785 objects.
  SB-C::COMPILED-DEBUG-FUN            2,040,480 bytes,  25,506 objects.
  SB-C::COMPILED-DEBUG-FUN-EXTERNAL   1,952,960 bytes,  24,412 objects.

Allegro:

code   type                                 items     bytes 
112: (SIMPLE-VECTOR T)                       20483   4837056 24.3%
  1: CONS                                   177041   2832656 14.2%
119: (SIMPLE-VECTOR CODE)                     8773   2642048 13.3%
 11: SYMBOL                                  32706   2093184 10.5%
136: FUNCTION                                15296   1882224  9.5%
134: GC-INFO                                 13857   1596576  8.0%
117: (SIMPLE-VECTOR CHARACTER)               14091   1576896  7.9%
133: SV-VECTOR                               33062   1109248  5.6%
125: (SIMPLE-VECTOR (UNSIGNED-BYTE 8))          85    645616  3.2%
143: STRUCTURE                                2320    230032  1.2%
140: STANDARD-INSTANCE                        6637    212384  1.1%
137: CLOSURE                                  4732    164480  0.8%
138: HASH-TABLE                                296     28416  0.1%
120: (SIMPLE-VECTOR FIXNUM)                      8     16976  0.1%
111: (SHORT-SIMPLE-VECTOR FOREIGN)             222     14176  0.1%
127: (SIMPLE-VECTOR (UNSIGNED-BYTE 32))          5      8608  0.0%
135: OTHER-LOCATIVE                            147      4704  0.0%
 18: BIGNUM                                    144      3872  0.0%
118: (SIMPLE-VECTOR BIT)                        79      2560  0.0%
 16: SINGLE-FLOAT                              153      2448  0.0%
 17: DOUBLE-FLOAT                              121      1936  0.0%
130: MV-VECTOR                                   5       880  0.0%
240: (SIMPLE-VECTOR (UNSIGNED-BYTE 64))          3       496  0.0%
193: (SHORT-ARRAY (SIGNED-BYTE 64))              8       384  0.0%
 80: (ARRAY T)                                   8       384  0.0%
 96: (SHORT-SIMPLE-VECTOR T)                    11       352  0.0%
 20: COMPLEX                                    11       352  0.0%
139: READTABLE                                   8       256  0.0%
241: (SIMPLE-VECTOR (SIGNED-BYTE 64))            1       176  0.0%

CLISP:

Class                                                                                           # Instances  Size (bytes)  Average size
-----                                                                                           -----------  ------------  ------------
CONS                                                                                              409012       6544192         16.000
SIMPLE-VECTOR                                                                                      21694       4449200        205.089
FUNCTION                                                                                           17640       2005448        113.688
SIMPLE-STRING                                                                                      29905       1648896         55.138
SYMBOL                                                                                             18552       1187328         64.000
SIMPLE-8BIT-VECTOR                                                                                 12162        617984         50.813
STANDARD-EFFECTIVE-SLOT-DEFINITION                                                                  1577        176624        112.000
HASH-TABLE                                                                                          1402        168240        120.000
STANDARD-METHOD                                                                                      992        111104        112.000
STANDARD-GENERIC-FUNCTION                                                                            748        107712        144.000
STANDARD-CLASS                                                                                       341         81840        240.000
SYSTEM::MACRO                                                                                       2102         67264         32.000
SYSTEM-FUNCTION                                                                                     1119         62664         56.000
FUNCTION-MACRO                                                                                      1756         56192         32.000
SYSTEM::INTERNAL-WEAK-HASHED-ALIST                                                                    49         55968       1142.204
STANDARD-DIRECT-SLOT-DEFINITION                                                                      462         40656         88.000
PATHNAME                                                                                             800         38400         48.000
STANDARD-READER-METHOD                                                                               303         36360        120.000
CLOS::STRUCTURE-EFFECTIVE-SLOT-DEFINITION                                                            248         29760        120.000
SYSTEM::INTERNAL-WEAK-LIST                                                                           544         29032         53.368
CLOS::STRUCTURE-DIRECT-SLOT-DEFINITION                                                               248         21824         88.000
STANDARD-WRITER-METHOD                                                                               145         17400        120.000
FFI:FOREIGN-FUNCTION                                                                                 220         15840         72.000
ASDF/LISP-ACTION:CL-SOURCE-FILE                                                                       91         14560        160.000
RESTART                                                                                              174         13920         80.000
WEAK-LIST                                                                                            539         12936         24.000
ASDF/PLAN:PLANNED-ACTION-STATUS                                                                      198         11088         56.000
ENCODING                                                                                             104         10816        104.000
PACKAGE                                                                                               93          8184         88.000
STRUCTURE-CLASS                                                                                       29          6728        232.000
BABEL-ENCODINGS:CHARACTER-ENCODING                                                                    41          5248        128.000
FFI:FOREIGN-ADDRESS                                                                                  157          5024         32.000
BUILT-IN-CLASS                                                                                        33          5016        152.000
ASDF/ACTION:ACTION-STATUS                                                                            121          4840         40.000
SYSTEM::ANODE                                                                                         98          4704         48.000
BABEL-ENCODINGS::CONCRETE-MAPPING                                                                     82          4592         56.000
FOREIGN-POINTER                                                                                      161          3864         24.000
EQL-SPECIALIZER                                                                                       69          3312         48.000
ASDF/SYSTEM:SYSTEM                                                                                    10          3040        304.000
FFI:FOREIGN-VARIABLE                                                                                  47          2632         56.000
STREAM                                                                                                16          2512        157.000
BABEL-ENCODINGS::ABSTRACT-MAPPING                                                                     41          2296         56.000
BYTE                                                                                                  67          2144         32.000
SYSTEM::CONST                                                                                         35          1960         56.000
SYSTEM::FNODE                                                                                          7          1960        280.000
SYSTEM::VAR                                                                                           14          1904        136.000
SPECIAL-OPERATOR                                                                                      40          1600         40.000
SYMBOL-MACRO                                                                                          66          1584         24.000
GLOBAL-SYMBOL-MACRO                                                                                   66          1584         24.000
STRING-STREAM                                                                                         11          1520        138.182
FILE-STREAM                                                                                            5          1440        288.000
SYSTEM::CONTROL-STRING-DIRECTIVE                                                                      16          1408         88.000
ASDF/COMPONENT:MODULE                                                                                  7          1288        184.000
METHOD-COMBINATION                                                                                    10          1280        128.000
STRING                                                                                                29          1160         40.000
SIMPLE-32BIT-VECTOR                                                                                    1          1040       1040.000
CFFI::FOREIGN-TYPEDEF                                                                                 20           800         40.000
FUNCALLABLE-STANDARD-CLASS                                                                             3           720        240.000
VECTOR                                                                                                15           600         40.000
LONG-FLOAT                                                                                            14           584         41.714
CONCATENATED-STREAM                                                                                    4           576        144.000
SYNONYM-STREAM                                                                                         4           544        136.000
BROADCAST-STREAM                                                                                       4           544        136.000
ASDF/COMPONENT:STATIC-FILE                                                                             3           480        160.000
RATIO                                                                                                 13           416         32.000
CFFI::FOREIGN-BUILT-IN-TYPE                                                                           13           416         32.000
DOUBLE-FLOAT                                                                                          15           360         24.000
SINGLE-FLOAT                                                                                          12           288         24.000
LOGICAL-PATHNAME                                                                                       5           280         56.000
READTABLE                                                                                              4           160         40.000
TWO-WAY-STREAM                                                                                         1           144        144.000
ECHO-STREAM                                                                                            1           144        144.000
BIGNUM                                                                                                 4           112         28.000
ASDF/PLAN:SEQUENTIAL-PLAN                                                                              1           104        104.000
QL-GUNZIPPER::DECODE-TREE                                                                              2            80         40.000
NULL                                                                                                   1            64         64.000
QL-HTTP:URL                                                                                            1            56         56.000
SIMPLE-BIT-VECTOR                                                                                      2            40         20.000
8BIT-VECTOR                                                                                            1            40         40.000
SIMPLE-ARRAY                                                                                           1            40         40.000
CFFI::ENHANCED-TYPEDEF                                                                                 1            40         40.000
CFFI::FOREIGN-BOOLEAN-TYPE                                                                             1            40         40.000
CFFI::FOREIGN-POINTER-TYPE                                                                             1            40         40.000
QL-CDB::RECORD-POINTER                                                                                 1            40         40.000
COMPLEX                                                                                                1            32         32.000
RANDOM-STATE                                                                                           1            24         24.000
QL-IMPL:CLISP                                                                                          1            24         24.000
ASDF/LISP-ACTION:PREPARE-OP                                                                            1            24         24.000
ASDF/LISP-ACTION:COMPILE-OP                                                                            1            24         24.000
ASDF/LISP-ACTION:LOAD-OP                                                                               1            24         24.000
-----                                                                                           -----------  ------------  ------------
Total                                                                                             524653      17725016         33.784

GCL (wtf?)

     519/519         82.7%         CONS FIXNUM SHORT-FLOAT LONG-FLOAT STRUCTURE SPICE
     113/113         90.2%         VECTOR CHARACTER HASH-TABLE ARRAY STRING CCLOSURE CLOSURE
      31/31          93.8%         BIT-VECTOR BIGNUM RATIO COMPLEX PATHNAME
     154/154         87.4%         STREAM SYMBOL
       1/1           34.8%         RANDOM-STATE
       9/9           90.0%         CFUN CFDATA
      55/55          98.6%         SFUN PACKAGE READTABLE GFUN VFUN AFUN

So it's usually the deftyped types, with standard-objects usually put together as one type, and numeric arrays as separate ones if supported.

Are they the same types for each implementation?

There are lots of standard types, and these are listed everywhere. There are also implementation-specific types. So yes and no 🤷

[Shinmera]

The thing to me about exposing this as a portability library is that unless the types exposed are also portable in some way, there's not really any useful way for a user to process the output, other than to just... add implementation-specific branching again, which would defeat the purpose.

[aartaka]

Right, now I get why you were asking that. Well, we can always sort things by belonging to cl for portability. And maybe expose the full listing when an optional flag is provided.

[Shinmera]

In the very least it would need to be documented which types can be expected and the structure of the returned alist/whatever.

[aartaka]

Yeah, sounds like a lot...