This essay enumerates and discusses various issues related to SCL[1] transpiler development.
A base language that supports building SCLs easily would impose few restrictions on the transpiler.
Restrictions are usually anything that is done solely at compile-time.[2] Anything with the words static or final in front of it.
Additionally, syntactic sugar and syntax rules makes transpilation more difficult.
The toolbox — the base language — doesn't need to be a "good" language to program in, it simply needs to be a good language to transpile into.
Automation can handle all of the "static" stuff.
Solution-specific syntax sugar can be added back in by automation (SCLs, little languages).
It is "nice to have" a toolbox language that supports first-class functions.
Assembler supports first-class functions, so, one could use Assembler directly.
Note that first-class functions are just GOTOs in sheeps' clothing.
Denotational Semantics uses the GOTO-ness of first-class functions directly, since denotational semantics concerns itself with specifying control flow.
In all other cases (e.g. CPS, closures, callbacks, etc.), first-class functions must be used with extreme care. All of the proverbs in "GOTO Considered Harmful" apply to first-class functions.
FP is an attempt to "structure" the use of first-class gotos by applying a strict policy of one-in-causes-one-out nesting.[3] Any other kind of use that falls outside of these strict guidelines runs into accidental complexity and problems — eg. callbacks, multitasking, etc.
Expression languages — where everything is an expression and everything returns a value (there are no statements) — make SCLs easier to write.
A toolbox language — a base language — is a set of programming constructs that make transpilation easier.
A toolbox language is different from a "good" programmer-level language.
A toolbox language is like Assembler. It offers almost no constraints and allows the programmer (the SCL builder) to "shoot oneself in the foot".
Human programmers hate programming directly in toolbox languages.
Transpiler-writers like using toolbox languages.
The transpiler adds-in the programmer-level checking - syntax and type checking. The toolbox language does not stand in the way.
A good toolbox language has little syntax.
Syntactic sugar in the toolbox language makes transpilation to it more difficult.
Syntactic sugar is often an afterthought that is applied to a toolbox language in order to make it more usable for direct use by programmers.
Lisp is a "good" toolbox language.
Lisp support 1st-class functions.
Lisp has little syntax (I argue that lisp has no syntax).
Lisp is an expression language
Lisp is dynamically-typed out-of-the-box. [Dynamically-typing is not the same as no-typing. Types are checked, albeit at "runtime"].
Lisp is not strictly functional, e.g. it allows side-effects.
Lisp does not lock programmers into a single paradigm.
All of the above make Lisp a good toolbox language (this may sound counter-intuitive, at first). Restrictions of any form (syntax, static typing, etc.) stand in the way of building SCLs easily.
Javascript exhibits most of the toolbox characteristics of Lisp.
The biggest exception is the fact that Javascript has syntax and is not an expression language. These "features" make it harder to use Javascript as a toolbox language than Lisp, but, Javascript is still easier to use for building transpilers than most languages.
Javascript supports first-class functions and anonymous functions. These features make Javascript a good toolbox language.
Programmers have tried to write programs in Javascript directly, much like assembler programmers did before C became popular.
To make Javascript more usable for direct programming, various tweaks have been added to Javascript and various dialects of JS (e.g. Typescript) have been invented, and, many frameworks have been created. Most of these changes and additions could have been avoided through the use of SCLs as layers on top of Javascript.
Are all dynamically-typed languages just toolbox languages in disguise? C and Pascal overtook Assembler programming, yet, both C and Pascal compile to assembler.
Debuggers have several uses:
a) Finding bizarre problems, e.g. ones caused by typos and insufficient design.
b) Bench-testing architectures.
c) Architectural Archaeology — understanding designs created by other people (e.g. when the code contains too much detail and hides the Architecture and original design).
Few languages support positional pragmas (e.g. like #line and #file). This means that type errors are shown relative to the toolbox language instead of being relative to the SCL source code.
The fact that debuggers can single-step through the source[4] means that positional information is created and exists. In most cases, such positional information has not been made into first-class entities.
Pragmas should allow making correspondences between source code position and control flow points.
Additionally, pragmas should make correspondences between environments and individual variables and the originating SCLs.
Does a program simply fail when it hits a bug, or, does it offer a menu of possible ways to continue running?
Lisp pioneered the concept of restarts.
In my opinion, restarts constitue a layer used at the debugger-level.
Common Lisp goes further than most languages in terms of scoping — also called packaging.
CL (Common Lisp) variables, function names, etc. — symbols — can be qualified to be within named packages.
Unfortunately, the CL read function makes it difficult to transpile code using SCLs (it reads new symbols into the runtime package instead of the compile-time package — there is a disconnect between how source code is written and how it can be generated). Again, I argue that packages should have been lifted into a separate layer (YAGNI) instead of being embedded inside of CL.
C, despite its many faults, professed a very simple — and usable[5] — scoping system. Variables and functions could be scoped static or extern, giving control of visibility to the user (and SCL transpiler builder). Furthermore, C variables could be scoped to be local or global orthogonally to the static and extern declarations.
Closures provide a way to wrap and hide variables.
A good toolbox language does not lock one into a certain paradigm.
Examples of poor toolbox languages are Smalltalk, Haskell, Erlang, PROLOG, and most other languages.
These languages emphasize single paradigms and make it difficult to use other paradigms. In my opinion, such paradigms should have been layered on top of a more general toolbox language.
It is no accident that Lisp has been used for the implementation of early versions of other languages, such as GHC.
Lisp is mostly paradigm agnostic.
Javascript provides a way to build classes (using prototypes) without restricting the paradigm to class-based only.
Assembler supports all programming paradigms.
A good toolbox language provides primitive operations for type-checking, thus, alleviating the SCL-builder from implementing type checking in the SCL (DSL).
Lisp and Javascript are "good" toolbox languages with respect to type checking.
Assembler, although being a good toolbox language in other respects, does not provide any form of type checking. This leaves too much work to the SCL builder.
A good toolbox language elides memory allocation and freeing.
GC (garbage collection) is a generally accepted form of this kind of facility.
All other forms of allocation (e.g. malloc/free, Rust ownership) are optimizations of the general problem (of allocating memory during runtime).
A good toolbox language is format-agnostic.
For example, most languages allow arbitrary amounts of whitespace in between expressions and statements.
FORTRAN requires a specific layout — card oriented, code must be written in certain columns only.
Python has specific layout rules — indentation is significant. This feature makes it more difficult to use Python as a toolbox language (because the transpiler must fiddle with the output format and keep track of indentation details — a time consuming project).
A good toolbox language supports prototype-based code.
Classes are an up-front type-check of the more general concept of prototypes.
Good toolbox languages support dynamic objects.
Good toolbox languages support dynamic objects.
Duck typing is better for toolbox languages than is static typing.
[1] Solution-Centric Language - like DSLs.
[2] https://guitarvydas.github.io/2020/12/27/Compile-Time-and-Runtime.html
[3] See my essay "How Many Inputs How Many Outputs" https://guitarvydas.github.io/
[4] E.g the Lispworks debugger can track correspondences between source code and single-stepping. Slime and SBCL also provide a means to connect source code to program counter position. Most modern debuggers have this ability.
[5] in the toolbox sense