Introduction - LangJam 0002
Goals of This Project
Draw diagrams of some common concurrency patterns.
Transpile the diagrams to running C code.
Write a multitasking kernel in C, for event-driven components.
How to Run This Project
[see at-a-glance.svg for status, see below for discussions]
Patterns as Diagrams
Some patterns (not all patterns) are easier to understand as diagrams.
5 sample diagrams, discussed below.
Reactive, Component-based Operating System
Concurrency is easy.
An operating system for message-passing, reactive, concurrent components is approximately 1 page of code and uses techniques that are already well-understood.
Create programs in layers.
- One layer for control-flow implementation, and,
- another layer for data implementation details.
[I chose to write the above O/S in C, the code is written in dataless C]
Compiling diagrams to running code as easily as compiling text to running programs.
- C– transpiler (C minus minus - dataless C)
- 5 Diagrams of various programming patterns (patterns.drawio)
- tools - pfr, d2f, f2j (parsing find replace, diagram to facbtase, factbase to JSON, resp.)
- node_modules: Ohm-JS@next, atob, pako
- POC .cmm code
- README.md, doc/*.md
- DaisyChain.drawio - single diagram to be compiled (from diagrams above)
- os.cmm - multitasking kernel (alpha, POC) for event-driven C components
- doc/internals-types.h.drawio - diagram of types.h
- diagrams of message passing in 3 steps
- internals-After 1st Send().svg
- internals-After 1s Dispatch.svg
Diagrams - Discussion
I find it “easier to reason about programs” if I can see diagrams.
In the vein of “graphic novels”, I want a “graphic spec”.
Programs are patterns. Diagrams are patterns. Diagrams are programs.
A Factbase is a set of triples.
This Project Uses Existing Technologies
C (and most languages, including C++, Haskell, Python, etc.) are dual purpose:
- the language describes implementation details of
- the language describes implementation details of
C– is dataless C.
- c– describes only the
control flowof a program using handles to data.
- Use “super-macros” to hide details of
- Super-macros are implemented using Ohm-JS.
To make things painfully obvious, I’ve named ALL macros using “$” as the first character in the name.
Building a Language by Cheating
Transpile .cmm files to .c files.
Let gcc do the heavy lifting.
The .cmm transpiler does not do “type checking”. The transpiler emits .c code and lets gcc do the type checking.
Designing the .cmm transpiler to be dataless makes punting of type checking easier.
[Aside: If I had more time, I might investigate the use of the “#file” and “#line” directives in C. Rhetorical question: how did the original C++ compiler map C++ source code to C code and back? I know of only two languages that have remember source-code line numbers - C and Scheme (syntax objects). Rhetorical question: how is #line generalized to include DaS (Diagrams as Syntax)?]
”$” is chosen for macro names so that when the .cmm transpiler fails to deal with a macro, the “$” is sent to gcc. Gcc will flag an error when it sees “$” in identifier names. During testing of the transpiler, some macros were not recognized nor translated. The “$” character caused gcc to flag an error, making such transpilation errors easier to detect.
Software Components - LEGO® Blocks
There are 2 secrets to making pluggable components (IMO):
- All components must be concurrent by default.
- A very, very simple type needs to used for inter-component communication.
The secrets to simple concurrency are:
- Make all components into (anonymous) lambdas
- Create a Dispatcher() routine to invoke the concurrent components.
A very, very simple type needs to used for inter-component communication.
UNIX® pipelines came close to this ideal - but no cigar.
The inter-component type in UNIX® - lines of text - is too complicated (!) to be used as a universal inter-component communication type.
We need to use
bytes (or something equally low-level, maybe bits) as the basic type, without ascribing special status to newlines. We can build up more complicated types by creating type-pipelines. For example, imagine dropping bytes into one end of a pipeline and getting an object certified to be of “Type XYZ” out of the other end of the pipeline. This is nothing “new”, except that we choose to present successive types in layers instead of in an either-or blob of type-ridden code. I suspect that programmers think in layers, but hide the layers in the final result (or, they mask the layers by using the same “language” for each layer without intervening isolated components).
The type pipeline verifies that a set of bits does, or does not, conform to the desired type. The pipeline contains at least two outputs:
- The bits, if they conform to the desired type. (Silence, otherwise).
- An escape-hatch - the bits, if they do not conform to the desired type. What comes out of this escape hatch? More inter-component messages (that may hold the bits in question). UNIX called this
stderrand most modern GPLs use
throwto provide escape hatches. A
throwor output to
stderrmodels all programming as being black-and-white - good or bad - but, in internet programming (IoT, distributed, etc., etc.) not all outcomes can be classified into only 2 categories, for example a
retryis not bad, it is just a part of the problem that needs to be solved. (If the bits conform to the desired type, then nothing is produced by this output.)
Diagrams - DaS
We use text for programming languages based on the biases (realities) of the 1950s, 1960s, 1970s, etc.
In 2020+, there is no reason that we can’t also use diagrams as syntax.
DaS means “Diagrams as Syntax”.
DaS should not be confused with VPL (Visual Programming Languages).
DaS is a hybrid of simple diagrams and text.
Some things are better left as text, e.g.
a = b + c;.
Other things are better left as diagrams, e.g. network diagrams of LEGO®-like components.
IDEs Instead of GPLs
11th Rule (a) of Programming - GPLs (General purpose Programming Languages) are merely mediocre implementations of IDEs.
The goal of programming is to make computers do what is needed. GPLs are merely stepping stones towards that goal.
IDEs Instead of Operating Systems
11th Rule (b) of Programming - OSs (Operating Systems) are merely mediocre implementations of IDEs.
OSs are merely libraries.
The goal of programming is to make computers do what is needed. OSs are merely stepping stones towards that goal.
[Note that urls beginning with “obsidian://open?vault=doc…” can be ignored. I believe that I pasted full URLs in all such places].
Diagrams to Code
See the latest TOC in the blog.
DSL for building parsers.
Based on PEG technology.
IMO - Ohm-JS is the best-of-breed PEG-based technology.
github (ohm-js): https://github.com/harc/ohm/tree/master/doc
Ohm Editor: https://ohmlang.github.io/editor/
See the latest TOC in the blog for related topics.
Computer programs fall - roughly - into two categories:
- Computer programs that are products (e.g. games, apps, etc.).
- Computer programs that help computer program developers create products
The idea of “efficiency” is different for both categories.
When designing products, the main criterions are cost and usability:
- How fast will the app run?
- How cheap can the hardware be?
When using computer programs to help program developers, the main criteria are:
- How fast will the app run on development hardware?
- How will the app save development time?
- If more than one human developer works on the same program, how quickly can the developers understand the program being developed?
Most current GPLs are optimized for (1) while sacrificing (2).
Many GPL features were invented in the early days of computing (1950, 1960, 1970, …) and do not correspond to the realities of 2020+.
A very simple example of the trade-offs that favour (1) over (2) is the fact that most GPLs require declaration-before-use. Development turn-around time and understanding time would be enhanced by declaration-after-use, but most GPLs insist that the developers waste their (human) time organizing code to appease compilers.
At first, development hardware was slow and 1-pass compilers seemed to be a good idea. As computer speeds increased, we developed better apps for business users (e.g. better word processors, spreadsheets, etc.) and products (e.g. phones, etc.), but failed to take full advantage of computer speed-ups for developers. Today, most GPLs still require developers to waste development time arranging code in declaration-before-use order.
In early days, Early parsers were sneered at, but now we have PEG parsers - a technology which does not yet appear in most popular GPLs. (Why not?)
We have backtracking languages like PROLOG (which brought about the invention of Relational technologies (like miniKanren)), but developers continue to use languages with features that were invented in the 1950’s/60’s/70’s
Why Ohm-JS is the Best-of-Breed PEG Tool
(see the latest TOC in the blog)
Concurrency is Not Parallelism
Concurrency is a paradigm.
Parallelism is a tactic.
All Parallel programs are concurrent.
Not all concurrent programs are parallel.
Rob Pike: Concurrency is not Parallelism:
Blogs / Articles / Youtube
Macros vs. DSLs
Relation to Projectional Editing
Relation to Software Architecture
Programming Languages Are Patterns
Functional Programming vs. Pattern Matching
Call Return Spaghetti
Overview of reactive, component-based, concurrent functionality:
Full-Blown Diagram Compiler
I didn’t finish the diagram compiler within the time limits for this JAM, but, I am working on a (WIP) diagram compiler project (paused for JAM) at https://github.com/guitarvydas/app. The techniques were touched upon in JAM0001 (https://github.com/guitarvydas/firstclasscomments).
- SWIPL is SWI PROLOG↩︎
- Querying is needed to make sense of the diagrams. I didn’t get to the point of querying diagrams within the time limits (but am working on a diagram compiler in another project.)↩︎
- In fact, these languages are designed to be GPLs (General purpose Programming Languages). The idea of using GPLs is rooted in the biases (realities) of the 1950’s, 1960’s, 1970’s. What we really want is IDEs for building computer systems. GPLs and O/Ss are IDE-wannabes.↩︎
- I think that the leading “$” could be dropped, but I haven’t tested this notion.↩︎
- Nothing means nothing. Not even “void”.↩︎