NOTE: This repository is no longer supported or updated by GitHub. If you wish to continue to develop this code yourself, we recommend you fork it.

Semantic

semantic is a Haskell library and command line tool for parsing, analyzing, and comparing source code.

In a hurry? Check out our documentation of example uses for the semantic command line tool.

Table of Contents
Usage
Language support
Development
Technology and architecture
Licensing

Usage

Run semantic --help for complete list of up-to-date options.

Parse

Usage: semantic parse [--sexpression | (--json-symbols|--symbols) |
                        --proto-symbols | --show | --quiet] [FILES...]
  Generate parse trees for path(s)

Available options:
  --sexpression            Output s-expression parse trees (default)
  --json-symbols,--symbols Output JSON symbol list
  --proto-symbols          Output protobufs symbol list
  --show                   Output using the Show instance (debug only, format
                           subject to change without notice)
  --quiet                  Don't produce output, but show timing stats
  -h,--help                Show this help text

Language support

Language	Parse	AST Symbols†	Stack graphs
Ruby	✅	✅
JavaScript	✅	✅
TypeScript	✅	✅	🚧
Python	✅	✅	🚧
Go	✅	✅
PHP	✅	✅
Java	🚧	✅
JSON	✅	⬜️	⬜️
JSX	✅	✅
TSX	✅	✅
CodeQL	✅	✅
Haskell	🚧	🚧

† Used for code navigation on github.com.

• ✅ — Supported
• 🔶 — Partial support
• 🚧 — Under development
• ⬜ - N/A ️

Development

semantic requires at least GHC 8.10.1 and Cabal 3.0. We strongly recommend using ghcup to sandbox GHC versions, as GHC packages installed through your OS's package manager may not install statically-linked versions of the GHC boot libraries. semantic currently builds only on Unix systems; users of other operating systems may wish to use the Docker images.

We use cabal's Nix-style local builds for development. To get started quickly:

git clone git@github.com:github/semantic.git
cd semantic
script/bootstrap
cabal v2-build all
cabal v2-run semantic:test
cabal v2-run semantic:semantic -- --help

You can also use the Bazel build system for development. To learn more about Bazel and why it might give you a better development experience, check the build documentation.

git clone git@github.com:github/semantic.git
cd semantic
script/bootstrap-bazel
bazel build //...

stack as a build tool is not officially supported; there is unofficial stack.yaml support available, though we cannot make guarantees as to its stability.

Technology and architecture

Architecturally, semantic:

1. Generates per-language Haskell syntax types based on tree-sitter grammar definitions.
2. Reads blobs from a filesystem or provided via a protocol buffer request.
3. Returns blobs or performs analysis.
4. Renders output in one of many supported formats.

Throughout its lifecycle, semantic has leveraged a number of interesting algorithms and techniques, including:

• Myers' algorithm (SES) as described in the paper An O(ND) Difference Algorithm and Its Variations
• RWS as described in the paper RWS-Diff: Flexible and Efficient Change Detection in Hierarchical Data.
• Open unions and data types à la carte.
• An implementation of Abstracting Definitional Interpreters extended to work with an à la carte representation of syntax terms.

Contributions

Contributions are welcome! Please see our contribution guidelines and our code of conduct for details on how to participate in our community.

Licensing

Semantic is licensed under the MIT license.