ENGINEERING · 01 / 06

Building language compilers

New compilers have to be built to give the existing substrate a sense of convergence at the language level. The work begins with a purpose-built tokenizer and parser, or an extension of a suitable parser, where such exists. It ends with a transform pipeline that takes the output AST and produces the internal representation that the corresponding execution engine takes up.

Oxford's work here includes:

01Compiler

UseLive compiler for JavaScript

Compiles standard JavaScript code to an internal control-flow graph that the UseLive runtime operates on.

JavaScript

"use live";
let a = 0;
console.log(a);
setInterval(() => a++, 1000);

The UseLive parser is an ESTree-compliant parser that supports the UseLive set of syntax extensions to JavaScript. The full compiler is built as a movable piece that can be loaded asynchronously, separate from the UseLive runtime, providing flexible integration strategies for web applications.

LinkedQL compiler for SQL

Compiles standard PostgreSQL and MySQL syntax, and an additional set of syntax extensions, to an internal representation that the LinkedQL execution engines operate on.

SQL-in-JS

db.query(`
  SELECT title, author ~> name
  FROM books@3_2
`, { live: true });

SQL-in-JS

db.query(`
  CREATE REALTIME VIEW people AS
  SELECT *
  FROM users
  WHERE id > 5
`);

The LinkedQL compiler is built ground-up from its own tokenizer and parser. They are the key to LinkedQL's rich syntax extensions to SQL, including DeepRefs, JSON Literals, and version tags.