How to Design a Good SCL

Q: How do you design a good SCL?


A: 


It used to take years to write one language, even a DSL.


With PEG, you can crank out an SCL in about one day.


The more iterations you make, the better you will become.


Ice Wine

Germany invented Ice Wine, but Canada makes better Ice Wine.  


Why?


You need 3 days of -8C weather before picking Ice Wine grapes.


In Germany, they get this kind of cold spell about twice per decade.


In southern Ontario, Canada, they get this kind of cold spell every year. 


Canadian wineries make Ice Wine 5X more often than German wineries.


You learn through making mistakes.  


Canadians get to make mistakes more frequently than Germans.


Hence, Canadians came down the learning curve in less elapsed time.


Hence, Canadian Ice Wine is better.


Eschew Dependencies

Dependencies are bad.


Learn how to build SCL's that don't have dependencies.


Use the n2k principle https://guitarvydas.github.io/2021/03/16/Need-To-Know.html.


If you build a little language that wants to do type-checking across files, don't.  Build a transpiler that does what it can with what it's got (i.e. only one file).  Defer the type check.  Build another SCL that joins two files and does the type checking.  Both SCLs will be simpler.


If you want to "practice" eschewing dependencies, try not using make, npm, or any of the other bandaids.  See how your thinking changes.  Write SCLs that defer dependency checking.


How about the call stack?  It makes dynamic, global, data structure to track dependencies at run time.  How can you not use the call stack?  Or, use it less?


https://guitarvydas.github.io/2021/02/25/The-Stack-is-a-Global-Variable-(again).html


https://guitarvydas.github.io/2020/12/25/The-ALGOL-Bottleneck.html


https://guitarvydas.github.io/2020/12/09/CALL-RETURN-Spaghetti.html



Layers

You can't  build a flexible system if it's not layered.  All of the details sink to the bottom and the user — who wants the flexibility to modify — is faced with a wall of detail


Emacs is like that.  I bought a hard copy of the emacs manual some 3 decades ago.  It was only 600 pages back then.  TL;DR.  Emacs is totally customizable, but it's hard to know where to start.  You learn emacs looking over someone else's shoulder — YAGNI.  I know only enough emacs commands to get me through, and you can bet that I won't bother switching editors, lest I get another steep learning curve.


Common Lisp is a wall of detail.  Common Lisp does everything.[1]  And everything is standardized and documented.  You have to love reading legalese, though.  I know only enough Common Lisp to get me through.  I lean on the CLHS a lot, even after a couple of decades of use.  You'd think that I would be an expert now, but I learn something new every time I talk to another lisper.


The open source movement is a wall of detail.  Sure, I can download just about everything from github, but, fixing any of it is a serious commitment.


The UNIX® manuals were not like that.  1 or 2 pages each.  Just enough to get you started.  Books and more details on request.


It is possible to use layers.  Structured Programming gave us layers over GOTO-full programming.  Global variables fell to locally-scoped variables.  OO gave us modular design.  FP hides state.


What about types?  How do you hierarchicalize types?


What about message-passing?  I use diagrams and composition (instead of inheritance) to make hierarchical message-based systems.  OSI told us to build layered messages.

Pipes - Isolation

One of the ideas the UNIX® pipes brought is elusively simple — isolation.


You can't convert a shell script to a <pick-your-language> program.


Why?


Because most languages use the stack.  The stack is a global variable and there's only one stack.[2]


Thread Safety is an accidental complexity caused by trying to use the calculator paradigm to solve a non-calculator problem (e.g. sequencing).


Bash has ugly syntax, and suffers from feature-itis.  Strip all of the gunk away, until you are left only with pipes — no variables, no string concatenation, no conditionals, no environment variables.


Bash, also, makes multi-tasking harder to use, than necessary, because of its insistence on the rendezvous model and textual code.


You can draw a diagram on a whiteboard of a network.  You can't draw a diagram of a bash script on a whiteboard.


Strip everything away and build only an SCL that gives you pipes.  If you want variables, build another SCL at another level.  If you want environment variables, build another SCL at another level.  Don't slam the kitchen sink into the SCL.  Make it do only one thing.


Start by build an SCL which is a like a shell that gives you multiple processes (threads) and lets you join them up with pipes.  Use a diagram as your syntax.  (You are allowed to hand compile the diagram, but it is easy enough to compile diagrams to code).  Note that "+" and "cons" and "arrays" don't figure in such a simple whiteboard-SCL.

Design the SCL ; Don't Use Existing Languages

Write your target problem out in its minimal form.


You are not allowed to use Python, JavaScript or any other programming language.


The problem with DSLs is that they are too general.


An SCL is meant to be one-time.  One problem, one SCL.  Not a DSL


Your target problem does not include a general solution for something, e.g. how to use A.I, how to use a database, etc.


Your target problem is "what does the user want and how will you solve it?".  Part of the target problem is a UX - that's one SCL.  Part of the target problem is how you will store the information - that's another SCL.  Part of the target problem is how you are going to process the information to produce a useful (to your user) result.  That's at least one SCL, if not more than one SCL.

Happy Path

Most PL creators want to make it "simple" to express the happy path while eliding details such as error conditions.


A good starting point for this kind of thinking is the set of rules for creating Drakon diagrams.


https://drakonhub.com/files/drakon_part1_eng.pdf

https://drakonhub.com/files/drakon_part2_eng.pdf

https://drakonhub.com/files/drakon_part3_eng.pdf


[I favour a different notation, but Drakon is a good place to start to widen one's horizons].

Multiple Possiblities

Textual notation is good for one-in-one-out operations, like functions.


Everything else isn't handled well with textual notation.


Structured Programming prescribed one input and one output.  This rule was broken by languages that support syntax for exceptions (one in, two outs).


Note that bash syntax flatlines at one-in, two-out (stdin, stdout, stderr).


Real life operations don't follow the above rules.  For example, lowly JavaScript has this very problem.  JavaScript defines a FileReader object with six possible outcomes[3] (called events) https://developer.mozilla.org/en-US/docs/Web/API/FileReader.


The textual code for handling these events is a mess.


But, it is easy to draw a diagram of this object…


Image


Just the act of drawing it out got me to reduce the API down to 4 useful outcomes.


The happy path is req—>resp (request, response).


If the user hits CANCEL, we get an ABORT event.


If there is some internal error (e.g. file not found), we get an ERR event.


If the network times out, we get a TIMEOUT event.


I posit that this is easier to understand than the textual version.


Can you use this right now?  Yes, just draw it on a whiteboard.[4]


Can we build SCLs like this?  Yes.  I will show how to compile SVG to code in an upcoming essay.  It is actually easier if you don't already know how to build compilers.  Drakon was built using Tcl/Tk.  I suggest something even simpler.

[1] and so does assembler

[2] One stack per CPU, one stack per CPU

[3] Not to mention the other 3 properties.

[4] This paradigm does not map easily onto CALL/RETURN paradigms.  Having a Dispatcher will make things easier.  See https://guitarvydas.github.io/2020/12/09/CALL-RETURN-Spaghetti.html.  You can map this onto bash using "&" syntax.  Most other languages need the use of thread libraries to make this work.  Trying to build components using CALL/RETURN languages (just about every language) will end up in accidental complexity (CALL/RETURN uses the stack, the stack is a global variable built into the hardware, going this route is possible, but runs into issues like thread safety (because of the global variable)).  This can also be implemented using closures (anonymous functions with state).