lib/chant/src/root.zig

daab053ee43316e1809a84551d573ddd1e5bf3d2

 1 //! The package reads C source for numeric kernels (loops over arrays with
 2 //! scalar arithmetic) and turns each function it supports into compiler
 3 //! intermediate code that a just-in-time compiler can run. The package follows
 4 //! C99, the 1999 ISO C standard, and also accepts parts of the 2023 standard
 5 //! such as `bool`, digit separators, `static_assert` and attributes.
 6 //!
 7 //! A kernel written in plain C should compile as written, so that the same
 8 //! source can be compiled here, run, and checked against a reference result.
 9 //! When a function uses C the compiler does not support, the caller should
10 //! learn which function and why, while the other functions of the file still
11 //! compile. The memory for tokens and syntax tree nodes should be decided
12 //! before lexing and parsing start.
13 //!
14 //! How many tokens a source holds, and how many tree nodes its parse needs, is
15 //! known only after reading it. Copying each token's text costs memory in
16 //! proportion to the source, and pointing into the source makes the source's
17 //! lifetime something the caller must manage. C is a large language, and
18 //! kernels use a small part of it. Preprocessing, which expands includes and
19 //! macros, is a separate problem from parsing.
20 //!
21 //! The C99 standard, ISO/IEC 9899:1999, defines the language in stages that
22 //! turn characters into tokens, tokens into a parse by the grammar, and the
23 //! parse into a meaning, and the package keeps those stages as its lexer,
24 //! parser and lowering. [chibicc](https://github.com/rui314/chibicc), a small C
25 //! compiler by Rui Ueyama, is the model for a compact frontend built from those
26 //! separate stages.
27 //! [PolyBench/C 4.2.1](https://sourceforge.net/projects/polybench/), Louis-Noël
28 //! Pouchet's suite of numeric C kernels, sets the style of kernel the package
29 //! targets, and its matrix multiply `kernel_gemm` is one of the seven kernels
30 //! of the shared benchmark corpus.
31 //!
32 //! Lexing and parsing each run as two passes: a counting pass (a *survey*)
33 //! tells the caller how much storage to allocate, and a second pass fills that
34 //! storage, so neither pass allocates for tokens or nodes while it runs. The
35 //! counting pass records the address and length of the input it counted so the
36 //! second pass can check that it received the same input. The token buffer
37 //! holds exactly one slot per counted token, and the parse buffer holds at most
38 //! two expression nodes, one statement node and two type nodes per token.
39 //! Tokens point into the source text for their text and file name, so the
40 //! caller keeps the source bytes alive and unchanged through lowering. Lowering
41 //! targets the compiler infrastructure in this repository (`lib/choir`),
42 //! *Choir*, modeled on MLIR with typed SSA, passes and native backends. The
43 //! output uses four named families of operations (*dialects*): `func` for
44 //! functions, `arith` for arithmetic, `scf` for structured loops and branches,
45 //! and `memref` for array memory. Lowering translates each function that
46 //! returns `void` and has a body. Any other function, and any function that
47 //! uses a construct lowering does not support (a *skipped function*), stays in
48 //! the result with its name and the error that stopped it. A parse error
49 //! returns as a typed error. Preprocessing goes to the system C compiler, run
50 //! as `cc -std=c2x -E -nostdinc` with the package's own small set of headers.
51 //!
52 //! - *token storage*: a byte buffer the caller allocates at the size the token
53 //!   survey implies, into which `fill` writes one token per slot.
54 //! - *parser node storage*: a byte buffer the caller allocates, holding up to
55 //!   two expression nodes, one statement node and two type nodes per token,
56 //!   with the source token and cached inferred type of each node.
57 
58 pub const diagnostic = @import("diagnostic.zig");
59 pub const token = @import("token.zig");
60 pub const lexer = @import("lexer/root.zig");
61 pub const ast = @import("ast/root.zig");
62 pub const parse = @import("parse/root.zig");
63 pub const preprocess = @import("preprocess/root.zig");
64 pub const lower = @import("lower/root.zig");
65 pub const driver = @import("driver.zig");
66 
67 pub const Diagnostic = diagnostic.Diagnostic;
68 pub const Token = token.Token;
69 pub const TokenKind = token.Kind;
70 pub const Lexer = lexer.Lexer;
71 pub const TokenLimits = lexer.Limits;
72 pub const TokenCapacity = lexer.Capacity;
73 pub const TokenCapacityError = lexer.CapacityError;
74 pub const TokenStorage = lexer.Storage;
75 pub const TokenSurvey = lexer.Survey;
76 pub const TokenExhaustion = lexer.Exhaustion;
77 pub const surveyTokens = lexer.survey;
78 pub const Parser = parse.Parser;
79 pub const ParserNodeStorage = parse.Storage;
80 pub const ParserNodeLimits = parse.Limits;
81 pub const ParserNodeCapacity = parse.Capacity;
82 pub const ParserNodeCapacityError = parse.CapacityError;
83 pub const ParserNodeSurvey = parse.Survey;
84 pub const ParserNodeExhaustion = parse.Exhaustion;
85 pub const surveyParserNodes = parse.survey;
86 pub const TranslationUnit = ast.TranslationUnit;