diff --git a/CHANGELOG.md b/CHANGELOG.md index 9b0591500..fcca0819d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,9 @@ ## [Unreleased] \(0.6.0\) - Black Diamonds [v6] - - + - + + - Included documentation into the repository and enabled http://somns.readthedocs.io/ [\#212](https://github.com/smarr/SOMns/pull/212) ## [0.5.0] - More Newspeak [v5] diff --git a/README.md b/README.md index e0e8071fc..8ffacb086 100644 --- a/README.md +++ b/README.md @@ -40,15 +40,15 @@ the specifications are as follows: - simultaneous slots clauses are not fully supported (spec. 6.3.2) - as in SOM, blocks can only have 3 arguments (counting `self`) - + - object literals currently require a keyword prefix `objL`, to work around parser limitations Obtaining and Running SOMns --------------------------- -This is a brief guide, a more comprehensive overview is available here: -[Getting Started Guide](http://som-st.github.io/somns/getting-started/). +This is a brief guide, a more comprehensive overview for users or developers is +available in the `docs` folder and on [ReadTheDocs](http://somns.readthedocs.io/en/dev/). To checkout the code: @@ -71,7 +71,7 @@ Setup Development Environment with Eclipse and VS Code 1. Install JDK 1.8 and Eclipse Mars (or later) 2. Download the project from Github - `git clone https://github.com/smarr/SOMns` + `git clone https://github.com/smarr/SOMns.git` 3. Run `ant compile` on the command line, or via Eclipse, to make sure that all libraries are loaded and available. @@ -132,6 +132,9 @@ Academic Work SOMns is designed as platform for research on concurrent programming models, and their interactions. Here, we collect related papers: + - [A Concurrency-Agnostic Protocol for Multi-Paradigm Concurrent Debugging Tools](http://stefan-marr.de/papers/dls-marr-et-al-concurrency-agnostic-protocol-for-debugging/), + S. Marr, C. Torres Lopez, D. Aumayr, E. Gonzalez Boix, H. Mössenböck; Dynamic Language Symposium'17. + - [Kómpos: A Platform for Debugging Complex Concurrent Applications](http://stefan-marr.de/downloads/progdemo-marr-et-al-kompos-a-platform-for-debugging-complex-concurrent-applications.pdf), S. Marr, C. Torres Lopez, D. Aumayr, E. Gonzalez Boix, H. Mössenböck; Demonstration at the <Programming>'17 conference. diff --git a/docs/AUTHORS.md b/docs/AUTHORS.md new file mode 120000 index 000000000..9eadf7123 --- /dev/null +++ b/docs/AUTHORS.md @@ -0,0 +1 @@ +../AUTHORS \ No newline at end of file diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md new file mode 120000 index 000000000..04c99a55c --- /dev/null +++ b/docs/CHANGELOG.md @@ -0,0 +1 @@ +../CHANGELOG.md \ No newline at end of file diff --git a/docs/LICENSE.md b/docs/LICENSE.md new file mode 120000 index 000000000..ea5b60640 --- /dev/null +++ b/docs/LICENSE.md @@ -0,0 +1 @@ +../LICENSE \ No newline at end of file diff --git a/docs/README.md b/docs/README.md new file mode 120000 index 000000000..32d46ee88 --- /dev/null +++ b/docs/README.md @@ -0,0 +1 @@ +../README.md \ No newline at end of file diff --git a/docs/basic-setup.md b/docs/basic-setup.md new file mode 100644 index 000000000..ea5ba20d1 --- /dev/null +++ b/docs/basic-setup.md @@ -0,0 +1,36 @@ +# Basic User Setup + +A brief overview for a basic development setup for SOMns. + +## Minimal Software Requirements + +SOMns is implemented in Java 8, uses Ant as a build system, git as +source control system, and Python for a launcher script. + +On Ubuntu, the following instructions will install the necessary dependencies: + +```bash +sudo add-apt-repository ppa:webupd8team/java +sudo apt install oracle-java8-installer git ant +``` + +## Getting the Code and Running Hello World + +To checkout the code: + +```bash +git clone https://github.com/smarr/SOMns.git +``` + +Then, SOMns can be build with Ant: + +```bash +cd SOMns +ant compile ## will also download dependencies +``` + +Afterwards, the simple Hello World program is executed with: + +```bash +./som -G core-lib/Hello.ns +``` diff --git a/docs/codespeed.png b/docs/codespeed.png new file mode 100644 index 000000000..e49523f49 Binary files /dev/null and b/docs/codespeed.png differ diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 000000000..c2a6058e9 --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,8 @@ +from recommonmark.parser import CommonMarkParser + +source_parsers = { + '.md': CommonMarkParser, +} + +source_suffix = ['.md'] +html_theme = 'sphinx_rtd_theme' diff --git a/docs/csp-design.md b/docs/csp-design.md new file mode 100644 index 000000000..e2a331e5e --- /dev/null +++ b/docs/csp-design.md @@ -0,0 +1,51 @@ +# Communicating Sequential Processes for Newspeak/SOMns + +> These notes were originally written as blog post and published on +> [stefan-marr.de](http://stefan-marr.de/2017/01/communicating-sequential-processes-for-newspeak-somns/). + +One possible way for modeling concurrent systems is Tony Hoare's classic approach of having isolated processes communicate via channels, which is called [Communicating Sequential Processes (CSP)](https://en.wikipedia.org/wiki/Communicating_sequential_processes). Today, we see the approach used for instance in [Go](https://blog.golang.org/share-memory-by-communicating) and [Clojure](http://clojure.com/blog/2013/06/28/clojure-core-async-channels.html). + +While [Newspeak's specification](http://bracha.org/newspeak-spec.pdf) and implementation come with support for Actors, I want to experiment also with other abstractions, and CSP happens to be an interesting one, since it models systems with blocking synchronization, also known as channels with rendezvous semantics. I am not saying CSP is better in any specific case than actors. Instead, I want to find out where CSP's abstractions provide a tangible benefit. + +But, the reason for this post is another one. One of my biggest quibbles with most CSP implementations is that they don't take isolation serious. Usually, they provide merely lightweight concurrency and channels, but they rarely ensure that different processes don't share any mutable memory. So, the door for low-level race conditions is wide open. The standard argument of language or library implementers is that guaranteeing isolation is not worth the performance overhead that comes with it. For me, concurrency is hard enough, so, I prefer to have the guarantee of proper isolation. Of course, another part of the argument is that you might need shared memory for some problems, but, I think we got [a more disciplined approach for those problems](http://stefan-marr.de/2016/02/domains-sharing-state-in-the-communicating-event-loop-actor-model/), too. + +## Isolated Processes in Newspeak + +Ok, so how can we realize isolated processes in Newspeak? As it turns out, it is pretty simple. Newspeak already got the notion of _values_. Values are deeply immutable objects. This means values can only contain values themselves, which as a consequence means, if you receive some value from a concurrent entity, you are guaranteed that the state never changes. + +In [SOMns](https://github.com/smarr/SOMns), you can use the [`Value`](https://github.com/smarr/SOMns/blob/master/core-lib/Kernel.som#L82) mixin to mark a class as having value semantics. This means that none of the fields of the object are allowed to be mutable, and that we need to check that fields are only initialized with values in the object's constructor. Since Newspeak uses nested classes pretty much everywhere, we also need to check that the outer scope of a value class does not have any mutable state. Once that is verified, an object can be a proper deeply immutable value, and can be shared without introducing any data races between concurrent entities. + +Using this as a foundation, we can require that all classes that represent CSP processes are values. This gives us the guarantee that a process does not have access to any shared mutable state by itself. Note, this is only about the class side. The object side can actually be a normal object an have mutable state, which means, within a process, we can have normal mutable state/objects. + +Using the _value_ notion of Newspeak feels like a very natural solution to me. Alternative approaches could use a magic operator that cuts off lexical scope. This is something that I have seen for instance in AmbientTalk with its [isolates](https://soft.vub.ac.be/amop/at/tutorial/actors#isolates). While this magic `isolate` keyword gives some extra flexibility, it is also a new concept. Having to ensure that a process' class is a value requires that its outer lexical scope is a value, and thus, restricts a bit how we structure our modules, but, it doesn't require any new concepts. One other drawback is here that it is often not clear that the lexical scope is a value, but I think that's something where an IDE should help and provide the necessary insights. + +In code, this looks then a bit like this: + +```java +class ExampleModule = Value ()( + class DoneProcess new: channelOut = Process ( + | private channelOut = channelOut. | + )( + public run = ( channelOut write: #done ) + ) + + public start = ( + processes spawn: DoneProcess + with: {Channel new out} + ) +) +``` + +So, we got a class `DoneProcess`, which has a `run` method that defines what the process does. Our `processes` module allows us to spawn the process with arguments, which is in this case the output end of a channel. + +## Channels + +The other aspect we need to think about is how can we design channels so that they preserve isolation. As a first step, I'll only allow to send values on the channel. This ensures isolation and is enabled by a simple and efficient check for whether the provided object is a value. + +However, this approach is also very restrictive. Because of the deeply immutable semantics of values, they are quite inflexible in my experience. + +When thinking of what it means to be a value, imagine a bunch of random objects: they all can point to values, but values can never point back to any mutable object. That's a very nice property from the concurrency perspective, but in practice this means that I often feel the need to represent data twice. Once as mutable, for instance for constructing complex data structures, and a second time as values so that I can send data to another process. + +A possible solution might be objects with copy-on-transfer semantics, or actual ownership transfer. This could be modeled either with a new type of transfer objects, or a copying channel. Perhaps there are other options out there. But for the moment, I am already happy with seeing that we can have proper CSP semantics by merely checking that a process is constructed from values only and that channels only pass on values. + +Since the [implementation is mostly a sketch](https://github.com/smarr/SOMns/pull/84), there are of course more things that need to be done. For instance, it doesn't yet support any nondeterminism, which requires an `alt` or `select` operation on channels. diff --git a/docs/dev-setup.md b/docs/dev-setup.md new file mode 100644 index 000000000..2ddef0ed8 --- /dev/null +++ b/docs/dev-setup.md @@ -0,0 +1,91 @@ +# A Complete Development Setup + +To setup a complete development environment, we need also the Graal just-in-time +compiler, and Node.js to work on the Kompos Web Debugger. + +On Ubuntu, the necessary software can be installed with: + +```bash +## First, installing Node.js and the Node Package Manager (NPM) +curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash - +sudo apt install npm nodejs + +## Second, installing and compiling the Graal JIT Compiler +cd .. ## leaving the SOMns folder +git clone --recursive https://github.com/smarr/GraalBasic.git +cd GraalBasic +yes "n" | ./build.sh +cd ../SOMns +ant ## to ensure everything is compiled +``` + +At this point, it should be possible to use Graal to run SOMns by dropping the +`-G` option from the command line: + +```bash +./som core-lib/Hello.ns +``` + +
+Eclipse Project outline +
+Eclipse Project outline. +
+
+ +## Eclipse to Develop the Interpreter + +SOMns is currently developed with Eclipse. While other Java IDEs can also be +used, for brevity, we'll focus on Eclipse only. + +At the time of writing, I am using [Eclipse Neon 4.6](https://eclipse.org/downloads/). +Please also install the **Eclipse Checkstyle Plugin** from the [Eclipse Marketplace](http://eclipse-cs.sourceforge.net/#!/install). + +For development, we also need to setup all Eclipse projects: + +```bash +ant ideinit +``` + +After the Eclipse projects for the Truffle library were generated by this step, +we can import the *existing* projects into Eclipse with *File* -> *Import...* -> +*Existing Projects into Workspace* and pointing Eclipse to the SOMns folder. + +I like to import the Truffle projects into a separate Truffle working set. This +makes working in Eclipse with many projects easier. + +The result should be looking roughly, but not exactly like in the screenshot on +the right. + +For debugging the interpreter with Eclipse, create a Eclipse run configuration +for the SOMns project. The main class is `som.VM`. To run for instance the +Mandelbrot benchmark, add the following as program arguments: + + core-lib/Benchmarks/Harness.ns Mandelbrot 2 0 500 + +In VM arguments, enable assertions with: + + -ea -esa + +I personally start the various SOMns programs from the command line: + +```bash +./som -d -G core-lib/Benchmarks/Harness.ns Mandelbrot 2 0 500 +``` + +For this approach, we need a *Remote Java Application* debug configuration +in Eclipse. After starting SOMns, it should tell you that it is waiting on port +`8000`, which is used as the port in the Eclipse debug configuration. + +## Summary + +A brief list of steps: + +1. Install software dependencies: ant, git, Java 8, Eclipse 4.6 (or later), + VS Code 1.8 (or later), Node.js, NPM, Graal JIT compiler + +2. Create Truffle Eclipse projects: `ant ideinit` + +3. Import Eclipse projects + +4. Run `ant` from the command line or Eclipse diff --git a/docs/eclipse-project-outline.png b/docs/eclipse-project-outline.png new file mode 100644 index 000000000..e5eaaf945 Binary files /dev/null and b/docs/eclipse-project-outline.png differ diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 000000000..f2320dc6e --- /dev/null +++ b/docs/index.md @@ -0,0 +1,8 @@ +# SOMns: A Simple Newspeak Implementation focusing on Concurrency Features + +This documentation is split in multiple parts. +Please see the navigation on the left to find links to: + + - documentation for users + - documentation for developers + - an overview of SOMns' design diff --git a/docs/infrastructure.md b/docs/infrastructure.md new file mode 100644 index 000000000..1ebd1c7c0 --- /dev/null +++ b/docs/infrastructure.md @@ -0,0 +1,83 @@ +# Infrastructure + +This gives a brief overview of some of the infrastructure used in SOMns. + +## Build System + +SOMns uses Ant as build system. The setup tries to minimize +the external software dependencies. Currently, instead of using some automatic +dependency management system for SOMns, we use an *uberjar* that combines all +rarely changing Java dependencies. + +The corresponding project is [SOMns-deps](https://github.com/smarr/SOMns-deps), +which is essentially a shell script creating a *jar* file from a set of +libraries an then uploading it onto [Bintray](https://bintray.com/smarr/SOM). + +The Truffle library is however directly used as a +[git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules) dependency, +because it changes frequently, and we sometimes need changes in Truffle. +Currently, SOMns also relies on a personal fork of Truffle to support changes +in the instrumentation and debugging support. + +## GitHub + +SOMns relies on GitHub, its issue tracking, and pull request system for +development. + +**Change Tracking with Pull Requests:** The general approach is all changes are +tracked with a pull request. + +When you are getting started with working on the SOMns interpreter internals, +consider checking out the +[**Good First Issue**](https://github.com/smarr/SOMns/labels/good%20first%20issue) +label. These issues are more or less simple changes that with a bit of guidance +should provide a good introduction to the SOMns code base, an basic +understanding of how Truffle-based interpreters work, and a few SOMns specific +insights. + +## Code Style + +When working on SOMns code, please look at the code around you and stick to the +style. It might be *particular*, but it is consistent in this code base. + +To ensure basic compliance with the style, we **use +[checkstyle](http://checkstyle.sourceforge.net/)**. It is integrated into the +build system and continuous integration system. Please use something like +[Eclipse Checkstyle](http://eclipse-cs.sourceforge.net/) to integrate it in +your editor. + +We are also using Codacy to monitor additional style issues or potential bugs. +See the [STM pull request](https://github.com/smarr/SOMns/pull/81#pullrequestreview-17422244) for examples. + +## Development Support + +**Continuous Integration:** To automatically run unit tests for the interpreter, SOMns, and the debugger, +we use [Travis CI](https://travis-ci.org/smarr/SOMns/builds) (see `.travis.yml`) +as well as a private GitLab instance to run benchmarks (see `.gitlab-ci.yml`). + +In case you forked SOMns on GitHub for your own work, please consider [enabling Travis CI](https://docs.travis-ci.com/user/getting-started/). + +The current build status is: [![Build Status](https://travis-ci.org/smarr/SOMns.png?branch=master)](https://travis-ci.org/smarr/SOMns) + +**Performance Tracking:** +Since one goal of SOMns is to be a platform for research on concurrency with +performance close to state-of-the-art JVMs, we continuously track benchmark +performance, for startup as well as peak performance with +[Codespeed](http://somns-speed.stefan-marr.de/timeline/#/?exe=14,23,19,21,22,20,16,17,15,18&base=none&ben=peak.Havlak&env=1&revs=10&equid=off). +It is run on every change to the master branch, and can be used to track and +compare performance of experimental changes as well. + +
+SOMns Codespeed: Havlak performance +
+SOMns Codespeed, tracking benchmark performance. Example shows *Havlak* peak +performance. +
+
+ +The benchmark execution is configured with `codespeed.conf` and are executed +with the [ReBench](https://github.com/smarr/ReBench) tool. + +**SOMns Code Coverage:** +To track SOMns code coverage, we use +[Coveralls](https://coveralls.io/github/smarr/SOMns). \ No newline at end of file diff --git a/docs/kompos-connection.md b/docs/kompos-connection.md new file mode 100644 index 000000000..ee0e047d0 --- /dev/null +++ b/docs/kompos-connection.md @@ -0,0 +1,39 @@ +# Interaction with Kompos Debugger + +This section gives a brief overview of the interaction between SOMns and Kompos. + +## Communication with Kompos + +Communication between Kompos and SOMns is implemented with WebSockets, the code can be found in [tools.debugger.FrontendConnector](https://github.com/MetaConc/SOMns/blob/16af2c3e3a41d13ab8c10ec289181db9284b49aa/src/tools/debugger/FrontendConnector.java) and [vm-connection.ts](https://github.com/MetaConc/SOMns/blob/16af2c3e3a41d13ab8c10ec289181db9284b49aa/tools/kompos/src/vm-connection.ts). Data is transferred on two separate sockets, one is for JSON and the other one is for binary data. + +JSON is used for two-way communication in a message-oriented style. Message objects are [serialized to JSON](https://github.com/MetaConc/SOMns/blob/16af2c3e3a41d13ab8c10ec289181db9284b49aa/src/tools/debugger/FrontendConnector.java#L181) when sending, and [received JSON data](https://github.com/smarr/SOMns/blob/16075443872e8de63e2bc71817552731f85eb1f0/src/tools/debugger/WebSocketHandler.java#L40) is used to create message objects. Kompos also (de)serializes communication data, [messages.ts](https://github.com/MetaConc/SOMns/blob/16af2c3e3a41d13ab8c10ec289181db9284b49aa/tools/kompos/src/messages.ts) provides some interfaces that help accessing data of message objects. + +On the SOMns side, message classes need to be "registered" in [tools.debugger.WebDebugger.createJsonProcessor()](https://github.com/MetaConc/SOMns/blob/16af2c3e3a41d13ab8c10ec289181db9284b49aa/src/tools/debugger/WebDebugger.java#L211), they extend either IncomingMessage or OutgoingMessage. When an IncomingMessage is received, its [process method](https://github.com/MetaConc/SOMns/blob/16af2c3e3a41d13ab8c10ec289181db9284b49aa/src/tools/debugger/message/Message.java#L9) is called. + +The binary socket is used for directly sending tracing data, which is pushed to +Kompos whenever available. Kompos parses the tracing data according to the +trace format and uses the data to generate the actor graph. + +## Trace Format + +The trace data includes for instance the following events: + +- Actor creation +- Promise creation +- Promise resolution: when a Promise is resolved with a value. +- Promise chained: when a Promise is resolved with another Promise. +- and various others + +An academic write up of the tracing feature and its format is part of our paper +titled [A Concurrency-Agnostic Protocol for Multi-Paradigm Concurrent Debugging Tools](http://stefan-marr.de/papers/dls-marr-et-al-concurrency-agnostic-protocol-for-debugging/). + +## Startup Protocol + +The following diagram gives an overview of the startup protocol. For simplicity the binary WebSocket for trace data and the view object are not included. + +
+kompos startup protocol +
+Simplified overview of startup protocol between SOMns interpreter and Kompos debugger. +
+
diff --git a/docs/kompos-startup-drawio.xml b/docs/kompos-startup-drawio.xml new file mode 100644 index 000000000..5821d02c8 --- /dev/null +++ b/docs/kompos-startup-drawio.xml @@ -0,0 +1 @@ +7Vxbk5s2FP41nkkzsx6QEJfHXW+SZtpMdrpp0z7KINvMYuRiObvbXx8JBEaScQDD2k2Shx04oIvP+fSdi0QmcLZ+epfhzeoDjUgyAVb0NIG3EwACZPG/QvBcCFxHCpZZHBUiey+4j/8jUli+tosjslVeZJQmLN6owpCmKQmZIsNZRh/V1xY0UUfd4CUxBPchTkzp5zhiKym1LWv/4FcSL1dyaL/8wXMcPiwzukvleBMAF/m/4vEal33J97crHNHHmgi+mcBZRikrrtZPM5II1ZZqK9q9bXhazTsjKWvTABDPRYEH5kG0CDzXuZI9fMHJTurir/Ws0HJM0wlwE97vzTzjV0txJX8Gey5Vx3/RRlzu1snv8YIkccrvbjYki9eEkYw/SaT4bi+7eVzFjNxvcCiaPnJMcdmKrRN+Z/NLbmeGeZOsuk8SvNnG83xUi0syEu6ybfyF/EG2BZyElO6YGGlWwcSqplzXUfmjScbIU00kdfaOUD7P7Jm/Ip+6pb0lvkt4P9bAUpp4VcMJLAGEJUCXVdd7I/ELaaeWNkOGzQyr1HS5oXHK8uHRzQTdasahGVvRJU1xUjePqbJG4LTWoaZCZKrwkAbBAAp0FoEbuVHoBSHxgL24gt+DAm3YToNwDA3a4HtQYaWvF1ChMXnf0GDp3Y4oUvzUmPut6yReplw2p4zRNX9A0uhaOEIhS2j4wEVbhjNWClOa83IuexuLWebcmN9LfwxFN08x+1uMNbWQvP1HvDr1fXlbM1HeA7dO9iyaXFlTyy4FRSOOEnmvt2rk5C3dZSFpUBmUC5dPeknY8cVNIsXlN0KCTxvCwDNw4EubZyTBjPsZNeY4AAQ5wp2Aeg1xyFZXrathqfjBslXdc2sduUBb/o7WUaGUIx2VL9LFYkuUd3LsVqrpB+fAgHNKHl/ttuRTxr38LWb4l8sANlBwbVleA7BbQdSz5y7EkPgBQSD04BXohj7kala1TEbyj7BPA+h8vVcNt21B51nf6OjMoLPNUOgzmd9zmBCTRn+ISNW3gqmv2Oy8sartXkSggLrqUQ+2DmjxkBJ1Uh5Ghya3zjKCGRmKUBtxV3e0jabt4GkdFNhjeVoPQNVkQU9PG+ik53sXRXrlClJJ75bMd8slXxA/JO0F7mXRHriIFLNCyshJ+iisB5whWU+JGSnvU4sZ7SOhpgwj/d5MCdrmJDU9u8OwYqAFmEHQjxR5YKp15LYixUEIz4zyZsOmymV2IHJYNT/w7FPSg/5IkGoXLtNzFcVfHShaDORCbUtLMoGF+sHF9zQfqk/mzD4UmnXvj+nHDUmHQlQjPLpwQPfMz9eWO7BhPwN6OhL0jr5pwKfSv6rtRrKnGRP9GXOSYBkPTERQ9HMfI3fUWmHgvEESvIggqYJO79zwQM3mxaIkaObXou4WzZcTMBPymDzmV+YCcPFawDWdbzcFVOWm34XU6awAKp4YVp75ZSt1J/GxA/VcsmdSivSOHM0zt+VjPQAciY+9A/51xKAt0II2eApUjC0Fqeqj2w7diyEoQKMVQ1wt/gJ9tx0QUqsqvKeOuBsZaOY22r4S8tPp587bviinbxYzz+H0/a5a1Jy+187nD3F8wZi8YyYv7wh7k2KOsIiDC4cPd4XaBmbacoe3Ilq7aVe4H8925FDo29OAoxs6jg885Kt5Mo/LprYHHBt6+UuokWs7bKyBYIpqnSJjRN8FlusHLhKv9aNc6AO1Wz2FGrHU4ph5VA1Q/AFfhnw21nQ65X9fixV5IsaoSLrhTYS3KxLJduLmDjMOpzSXAHmwIKoFjo0IGzegg77mD5Hbz8oI+Mc7ahvQOWqzcdysY6Zq99we79OYxTgREHkomPqV6XXn+6cRZlggaMrFry8lz9AOujQeCKhOunAahOpBF+4PTjrp0sSERyPO8gxW67MG+uknGEwL3iwoy4VDMKRWgDKCiNbnXQI9gvU7ro+RV8Q5TmF2jv3Miq7fLnAZ5cibY2aFOXUU0cptSQ7TIfyKHruYC33gJLH1chxj65tTkLrs7J7LzrbK0wYVYDQHM2b0YSZzoph1IY4CqPixYRN+9o5C9RLoJB/RVMA6uvNYZlttfQTQ6ML2ggGcAggGqoJBvQqm51cv6xQMk5iJ2QRef8Cxua30Q1Qezrnb0LRcXtRfNwKkb9nBbee9bTSABk0PZ2iQ0eUyIfXvhS6BqNUahTPKtsFx1u24y49c7ZD4gepSn1C8odfuqeo3YvqXZd2m8vxlR+L6UerqaMU5Pt85x6Hc7tSnOfu2X+voWx2DaMyMTC9QY1A//QFaYgydrjFkRz4M5xD5xAWe57X5yK7G/JLkazpUy31bltGH6uNc56QCoP7VcrWPbygLHVDWMdfasOWqDQT7bsY1zXj41MywpnkYsyz0RfEXxaruvzvxNfMNxyy7woWzv+ZvhNxO+aacfF47iJD3s93gtJTdf/yQbssHfK71ZzVxPrQqPXU274V4kxG23z/UB9JALHpWkZuJ2Brvg26TDfCOURl/iwZyXrcJWbADkRKjm0I/YZwuP4mb26ujK6BLfUh3S8Ya8EZySgbETDf+G11vqLmL9b/WuH4ixFS4e0DhPfYR+e3+/xUoKGD/fzfAN18B \ No newline at end of file diff --git a/docs/kompos-startup.svg b/docs/kompos-startup.svg new file mode 100644 index 000000000..d46c7146c --- /dev/null +++ b/docs/kompos-startup.svg @@ -0,0 +1,2 @@ + +
VmConnection
VmConnection<br>
connect
connect
new(useTraceData)
new(useTraceData)
WebSocket
WebSocket
Create
Create
WebDebugger
WebDebugger
Create
Create
Connect
Connect
OnOpen
OnOpen
UiController
UiController<br>
new(dbg, view,
 connection)
[Not supported by viewer]
OnConnect
OnConnect
Debugger
Debugger<br>
GetEnabledBreackPoints
GetEnabledBreackPoints
BreackPoint [0 ... *]
BreackPoint [0 ... *]
SendInitialBreakpoints(
breakpointsdata[0.;*)
SendInitialBreakpoints(<br>breakpointsdata[0.;*)
BreakPointsData[0..*]
BreakPointsData[0..*]
new()
new()
:Main
:Main
toggleConnection
toggleConnection
SOMns
Interpreter
[Not supported by viewer]
Kompos
Kompos \ No newline at end of file diff --git a/docs/launcher.md b/docs/launcher.md new file mode 100644 index 000000000..65575d7ff --- /dev/null +++ b/docs/launcher.md @@ -0,0 +1,55 @@ +# The `som` Launcher Script + +To document, simplify, and standardize how SOMns is started or debugged, we +use a Python script to execute the actual Java program. + +The Python script manages various command-line parameters and selects the JVM +to be used for execution. + +Below, we see that `./som --help` supports a large set of options, of which we +detail only a few. + +The basic options include `-d` to allow us to attach a Java debugger, for instance +from Eclipse, `-dnu` to print stack traces on a `#doesNotUnderstand` message, +or `-G` to run without Graal-based JIT compilation. +Generally, the options are designed to use upper-case shorthands when they +disable a feature. + +Further below, we see different categories of other options. This includes +flags to investigate and understand how Graal executes the interpreter, +options for various profiling tools, as well as tools for SOMns code coverage, +dynamic execution metrics, or interactive debugging of SOMns code +(currently called 'web debugger' `-wd`). + +``` +$ ./som --help +usage: som [-h] [-d] [-t THREADS] [-p SOM_PLATFORM] [-k SOM_KERNEL] [-dnu] + [-i] [-if] [-io ONLY_IGV] [-l] [-ti] [-w] [-f] [-v] [-gp] [-ga] + [-gi] [-gb] [-tp] [-td] [-wd] [-dm] [-at] + [-atcfg ACTOR_TRACING_CFG] [-mt] [-tf TRACE_FILE] [-TF] + [--coverage COVERAGE] [-o ONLY_COMPILE] [-A] [-B] [-C] [-G] [-X] + [-T] [--no-graph-pe] [-vv] [--print-graal-options] + [-D JAVA_PROPERTIES] + ... + +optional arguments: + -h, --help show this help message and exit + -d, --debug wait for debugger to attach + -dnu, --stack-trace-on-dnu print a stack trace on #doesNotUnderstand: + -G, --interpreter run without Graal + +Investigate Execution + -i, --igv dump compilation details to IGV + -l, --low-level enable low-level optimization output + -v, --visual-vm connect to VisualVM for profiling + +Profile Execution + -gp, --graal-profile enable Graal-level profiling after warmup + -tp, --truffle-profile enable Graal-level profiling after warmup + +Tools + -td, --truffle-debugger start Truffle debugger + -wd, --web-debugger start Web debugger + -dm, --dynamic-metrics capture Dynamic Metrics + --coverage COVERAGE determine code coverage and store in given file +``` \ No newline at end of file diff --git a/RELEASE-CHECKLIST.md b/docs/release-checklist.md similarity index 100% rename from RELEASE-CHECKLIST.md rename to docs/release-checklist.md diff --git a/docs/repo-layout.md b/docs/repo-layout.md new file mode 100644 index 000000000..794658baa --- /dev/null +++ b/docs/repo-layout.md @@ -0,0 +1,102 @@ +# Repository and Code Layout + +This section gives a brief overview over the most relevant elements of the +repository. + +## Repository Root + +```bash +.gitlab-ci.yml # Configuration for the SOFT Build Server +.travis.yml # Configuration for the Travis Build Service +CHANGELOG.md # The changelog contains a high-level change overview +build.xml # The Ant build description +codespeed.conf # Benchmark configuration, based on ReBench +core-lib # All SOMns code, including standard lib, tests, and benchmarks +docs # Documentation +libs # Libraries and dependencies of SOMns +som # Launcher script +src # Java sources +src_gen # Java sources generated by the Truffle DSL +tests # Java unit tests and tests for DynamicMetrics tool +tools # Contains the Kompos Web Debugger +``` + +## Code Layout: Java + +```bash +som +|- compiler # Parser, AST creation, and source representation +|- instrumentation # AST instrumentation support, used e.g. by DynamicMetrics tool +|- interop # Interoperability with Truffle languages, only minimally implemented +|- interpreter # Dynamic SOMns language semantics, i.e., interpreter and AST nodes + |- nodes # AST and dispatch node implementations + |- objectstorage # SOMns object model implementation + |- actors, processes, transactions # core elements of concurrency models +|- primitives # Basic operations, exposed via vmMirror object to SOMns +|- vm # Basic VM setup, startup, and object system initialization +|- vmobjects # Representation for build-in object and arrays +|- VM.java # Java entry point and bridge between interpreter and tools + +tools +|- debugger # Connection to the Kompos Web Debugger +``` + +## Code Layout: Tests + +```bash +tests +|- dym # Tests for the DynamicMetrics tool +|- java # JUnit tests for interpreter, including runner for BasicInterpreterTests +|- replay # Tests for the deterministic replay feature +|- superinstructions # Tests for the super-instruction candidate detector +``` + +## Code Layout: SOMns + +```bash +core-lib +|- Benchmarks # Collection of various benchmarks +|- TestSuite # Collection of tests + |- BasicInterpreterTests # Minimal tests only executable as JUnit tests + |- Minitest.ns # Newspeak's Minitest framework + |- *Tests.ns # Various test suites +|- Actors.ns # Actor and promise classes +|- Collections.ns # Sets, Dictionaries, etc +|- Hello.ns # Hello World program +|- Kernel.ns # Core classes: Integer, Boolean, String, Array etc +|- Mirrors.ns # Minimal and incomplete mirror API +|- Platform.ns # Application loader +|- Processes.ns # Communicating Sequential Processes classes +|- System.ns # Minimal API to access system functionality +|- Threading.ns # Threading and fork/join-related classes +|- Transactions.ns # Software transactional memory classes +``` + +## Code Layout: Kompos + +Kompos is a web-based debugger integrated into SOMns. It is based on Truffle's +language agnostic debugger support and extends it to provide actor-specific +debugging facilities. It is implemented in TypeScript. + +Note that Kompos and the VS Code debugger share the TypeScript code that is +used to interface with the Java backend. + +```bash +/tools/kompos/ +|- index.html # The HTML elements for the debugger +|- package.json # NPM package definition +|- src + |- breakpoints.ts # Breakpoint related code + |- controller.ts # MVC controller to connect model and view + |- debugger.ts # Debugger model to manager state needed in UI + |- history-data.ts # Model for the actor message history + |- main.js # Main JS file used from HTML frontend + |- messages.ts # Definitions of messages exchanged with SOMns interpreter + |- view.ts # The HTML view code + |- visualizations.ts # Visualizes actor interactions as graph + |- vm-connection.ts # Web socket connection to the SOMns interpreter + +|- tests # Tests debugger interaction with SOMns, including some SOMns integration tests +|- tsconfig.json # TypeScript config +|- tslint.json # TypeScript lint settings +``` diff --git a/docs/vs-code.md b/docs/vs-code.md new file mode 100644 index 000000000..1dd1b15de --- /dev/null +++ b/docs/vs-code.md @@ -0,0 +1,28 @@ +## VS Code to Develop with the SOMns Language + +Currently, we provide IDE support for [VS Code](https://code.visualstudio.com/). +The SOMns support can be installed via the +[VS Code marketplace](https://marketplace.visualstudio.com/items?itemName=MetaConcProject.SOMns). + +The extension provides support for: + + - syntax highlighting + - parse errors + - code navigation + - code completion + - basic linting + - CodeLens for running minitests + - and debugging of SOMns programs + +This support is based on the +[Language Server Protocol](https://github.com/Microsoft/language-server-protocol#readme) +and could be used in other IDEs as well, but we do not currently provide +extensions for them. It is however confirmed to work with NetBeans and Eclipse. + +#### Screenshot of SOMns Syntax Highlighting + +![Screenshot of SOMns syntax highlighting](https://som-st.github.io/images/vscode-somns-syntax-highlighting.png) + +#### Screencast of Debugging a SOMns Program + +![Screencast of SOMns debug session](https://som-st.github.io/images/vscode-somns-debugger.gif) diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 000000000..5d15eca7e --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,27 @@ +site_name: 'SOMns: A Simple Newspeak' +# site_url: +site_description: SOMns Project documentation +site_author: SOMns contributors + +pages: + - 'SOMns: A Simple Newspeak': index.md + - For Users: + - Basic User Setup: basic-setup.md + - IDE Setup: vs-code.md + - som Launcher: launcher.md + - For Developers: + - Development Setup: dev-setup.md + - Repository and Code Layout: repo-layout.md + - Development Infrastructure: infrastructure.md + - Release Checklist: release-checklist.md + - SOMns Design Notes: + - Interaction with Kompos Debugger: kompos-connection.md + - CSP Design: csp-design.md + - License: LICENSE.md + - README: README.md + - Change Log: CHANGELOG.md + - Authors: AUTHORS.md + +markdown_extensions: + - toc: + permalink: True