RSConstruct - Rust Build Tool

A fast, incremental build tool written in Rust with C/C++ compilation, template support, Python linting, and parallel execution.

Features

• Incremental builds using SHA-256 checksums to detect changes
• C/C++ compilation with automatic header dependency tracking
• Parallel execution of independent build products with -j flag
• Template processing via the Tera templating engine
• Python linting with ruff and pylint
• Documentation spell checking using hunspell dictionaries
• Make integration — run make in directories containing Makefiles
• .gitignore support — respects .gitignore and .rsconstructignore patterns
• Deterministic builds — same input always produces same build order
• Graceful interrupt — Ctrl+C saves progress, next build resumes where it left off
• Config-aware caching — changing compiler flags or linter config triggers rebuilds
• Convention over configuration — simple naming conventions, minimal config needed

Philosophy

Convention over configuration — simple naming conventions, explicit config loading, incremental builds by default.

Installation

Download pre-built binary (Linux)

Pre-built binaries are available for x86_64 and aarch64 (arm64).

Using the GitHub CLI:

# x86_64
gh release download latest --repo veltzer/rsconstruct --pattern 'rsconstruct-x86_64-unknown-linux-gnu' --output rsconstruct --clobber

# aarch64 / arm64
gh release download latest --repo veltzer/rsconstruct --pattern 'rsconstruct-aarch64-unknown-linux-gnu' --output rsconstruct --clobber

chmod +x rsconstruct
sudo mv rsconstruct /usr/local/bin/

Or with curl:

# x86_64
curl -Lo rsconstruct https://github.com/veltzer/rsconstruct/releases/download/latest/rsconstruct-x86_64-unknown-linux-gnu

# aarch64 / arm64
curl -Lo rsconstruct https://github.com/veltzer/rsconstruct/releases/download/latest/rsconstruct-aarch64-unknown-linux-gnu

chmod +x rsconstruct
sudo mv rsconstruct /usr/local/bin/

Install from crates.io

cargo install rsconstruct

This downloads, compiles, and installs the latest published version into ~/.cargo/bin/.

Build from source

cargo build --release

The binary will be at target/release/rsconstruct.

Release profile

The release build is configured in Cargo.toml for maximum performance with a small binary:

[profile.release]
strip = true        # Remove debug symbols
lto = true          # Link-time optimization across all crates
codegen-units = 1   # Single codegen unit for better optimization

For an even smaller binary at the cost of some runtime speed, add opt-level = "z" (optimize for size) or opt-level = "s" (balance size and speed).

Getting Started

This guide walks through setting up an rsconstruct project for the two primary supported languages: Python and C++.

Python

Prerequisites

• rsconstruct installed (Installation)
• ruff on PATH

Setup

Create a project directory and configuration:

mkdir myproject && cd myproject

# rsconstruct.toml
[processor]
enabled = ["ruff"]

Create a Python source file:

mkdir -p src

# src/hello.py
def greet(name: str) -> str:
    return f"Hello, {name}!"

if __name__ == "__main__":
    print(greet("world"))

Run the build:

rsconstruct build

Expected output:

Processing ruff (1 product)
  hello.py

Run again — nothing has changed, so rsconstruct skips the check:

Processing ruff (1 product)
  Up to date

Adding pylint

Install pylint and add it to the enabled list:

# rsconstruct.toml
[processor]
enabled = ["ruff", "pylint"]

Pass extra arguments via processor config:

[processor.pylint]
args = ["--disable=C0114,C0115,C0116"]

Adding spellcheck for docs

If your project has markdown documentation, add the spellcheck processor:

[processor]
enabled = ["ruff", "pylint", "spellcheck"]

Create a .spellcheck-words file in the project root with any custom words (one per line) that the spellchecker should accept.

C++

Prerequisites

• rsconstruct installed (Installation)
• gcc/g++ on PATH

Setup

Create a project directory and configuration:

mkdir myproject && cd myproject

# rsconstruct.toml
[processor]
enabled = ["cc_single_file"]

Create a source file under src/:

mkdir -p src

// src/hello.c
#include <stdio.h>

int main() {
    printf("Hello, world!\n");
    return 0;
}

Run the build:

rsconstruct build

Expected output:

Processing cc_single_file (1 product)
  hello.elf

The compiled executable is at out/cc_single_file/hello.elf.

Run again — the source hasn’t changed, so rsconstruct restores from cache:

Processing cc_single_file (1 product)
  Up to date

Customizing compiler flags

Pass flags via processor config:

[processor.cc_single_file]
cflags = ["-Wall", "-Wextra", "-O2"]
cxxflags = ["-Wall", "-Wextra", "-O2"]
include_paths = ["include"]

See the CC Single File processor docs for the full configuration reference.

Adding static analysis

Install cppcheck and add it to the enabled list:

[processor]
enabled = ["cc_single_file", "cppcheck"]

Both processors run on the same source files — rsconstruct handles them independently.

Next Steps

• Commands — full list of rsconstruct commands
• Configuration — all configuration options
• Processors — detailed docs for each processor

Binary Releases

RSConstruct publishes pre-built binaries as GitHub releases when a version tag (v*) is pushed.

Supported Platforms

How It Works

The release workflow (.github/workflows/release.yml) has two jobs:

1. build — a matrix job that builds the release binary for each platform and uploads it as a GitHub Actions artifact.
2. release — waits for all builds to finish, downloads the artifacts, and creates a GitHub release with auto-generated release notes and all binaries attached.

Creating a Release

1. Update version in Cargo.toml
2. Commit and push
3. Tag and push: git tag v0.2.2 && git push origin v0.2.2
4. The workflow creates the GitHub release automatically

Release Profile

The binary is optimized for size and performance:

[profile.release]
strip = true        # Remove debug symbols
lto = true          # Link-time optimization across all crates
codegen-units = 1   # Single codegen unit for better optimization

Command Reference

Global Flags

These flags can be used with any command:

Example:

rsconstruct --phases build                    # Show phase messages during build
rsconstruct --show-child-processes build      # Show each command being executed
rsconstruct --show-output build               # Show compiler/linter output even on success
rsconstruct --phases --show-child-processes build # Show both phases and commands
rsconstruct -O path build                     # Show output file paths in build messages
rsconstruct -I all build                      # Show all input files (including headers)

rsconstruct build

Incremental build — only rebuilds products whose inputs have changed.

rsconstruct build                              # Incremental build
rsconstruct build --force                      # Force full rebuild
rsconstruct build -j4                          # Build with 4 parallel jobs
rsconstruct build --dry-run                    # Show what would be built without executing
rsconstruct build --keep-going                 # Continue after errors
rsconstruct build --timings                    # Show per-product and total timing info
rsconstruct build --stop-after discover        # Stop after product discovery
rsconstruct build --stop-after add-dependencies # Stop after dependency scanning
rsconstruct build --stop-after resolve         # Stop after graph resolution
rsconstruct build --stop-after classify        # Stop after classifying products
rsconstruct build --show-output                # Show compiler/linter output even on success
rsconstruct build --auto-add-words             # Add misspelled words to .spellcheck-words instead of failing
rsconstruct build --auto-add-words -p spellcheck # Run only spellcheck and auto-add words
rsconstruct build -p ruff,pylint               # Run only specific processors
rsconstruct build --explain                    # Show why each product is skipped/restored/rebuilt
rsconstruct build --retry 3                    # Retry failed products up to 3 times
rsconstruct build --no-mtime                   # Disable mtime pre-check, always compute checksums
rsconstruct build --no-summary                 # Suppress the build summary
rsconstruct build --batch-size 10              # Limit batch size for batch-capable processors
rsconstruct build --verify-tool-versions       # Verify tool versions against .tools.versions

By default, tool output (compiler messages, linter output) is only shown when a command fails. Use --show-output to see all output.

Processor Shortcuts (@ aliases)

The -p flag supports @-prefixed shortcuts that expand to groups of processors:

By type:

• @checkers — all checker processors (ruff, pylint, shellcheck, etc.)
• @generators — all generator processors (tera, cc_single_file, etc.)
• @mass_generators — all mass generator processors (pip, npm, cargo, etc.)

By tool:

• @python3 — all processors that require python3
• @node — all processors that require node
• Any tool name works (matched against each processor’s required_tools())

By processor name:

• @ruff — equivalent to ruff (strips the @ prefix)

Examples:

rsconstruct build -p @checkers              # Run only checker processors
rsconstruct build -p @generators            # Run only generator processors
rsconstruct build -p @python3               # Run all Python-based processors
rsconstruct build -p @checkers,tera         # Mix shortcuts with processor names

The --stop-after flag allows stopping the build at a specific phase:

• discover — stop after discovering products (before dependency scanning)
• add-dependencies — stop after adding dependencies (before resolving graph)
• resolve — stop after resolving the dependency graph (before execution)
• classify — stop after classifying products (show skip/restore/build counts)
• build — run the full build (default)

rsconstruct clean

Clean build artifacts. When run without a subcommand, removes build output files (same as rsconstruct clean outputs).

rsconstruct clean                # Remove build output files (preserves cache) [default]
rsconstruct clean outputs        # Remove build output files (preserves cache)
rsconstruct clean all            # Remove out/ and .rsconstruct/ directories
rsconstruct clean git            # Hard clean using git clean -qffxd (requires git repository)

rsconstruct status

Show product status — whether each product is up-to-date, stale, or restorable from cache.

rsconstruct status

rsconstruct init

Initialize a new rsconstruct project in the current directory.

rsconstruct init

rsconstruct watch

Watch source files and auto-rebuild on changes.

rsconstruct watch                              # Watch and rebuild on changes
rsconstruct watch --auto-add-words             # Watch with spellcheck auto-add words
rsconstruct watch -j4                          # Watch with 4 parallel jobs
rsconstruct watch -p ruff                      # Watch and only run the ruff processor

The watch command accepts the same build flags as rsconstruct build (e.g., --jobs, --keep-going, --timings, --processors, --batch-size, --explain, --retry, --no-mtime, --no-summary).

rsconstruct graph

Display the build dependency graph.

rsconstruct graph show                    # Default SVG format
rsconstruct graph show --format dot       # Graphviz DOT format
rsconstruct graph show --format mermaid   # Mermaid format
rsconstruct graph show --format json      # JSON format
rsconstruct graph show --format text      # Plain text hierarchical view
rsconstruct graph show --format svg       # SVG format (requires Graphviz dot)
rsconstruct graph view                    # Open as SVG (default viewer)
rsconstruct graph view --viewer mermaid   # Open as HTML with Mermaid in browser
rsconstruct graph view --viewer svg       # Generate and open SVG using Graphviz dot
rsconstruct graph stats                   # Show graph statistics (products, processors, dependencies)

rsconstruct cache

Manage the build cache.

rsconstruct cache clear         # Clear the entire cache
rsconstruct cache size          # Show cache size
rsconstruct cache trim          # Remove unreferenced objects
rsconstruct cache list          # List all cache entries and their status
rsconstruct cache stale         # Show which cache entries are stale vs current
rsconstruct cache stats         # Show per-processor cache statistics
rsconstruct cache remove-stale  # Remove stale index entries not matching any current product

rsconstruct deps

Show or manage source file dependencies from the dependency cache. The cache is populated during builds when dependency analyzers scan source files (e.g., C/C++ headers, Python imports).

rsconstruct deps list                        # List all available dependency analyzers
rsconstruct deps show all                    # Show all cached dependencies
rsconstruct deps show files src/main.c       # Show dependencies for a specific file
rsconstruct deps show files src/a.c src/b.c  # Show dependencies for multiple files
rsconstruct deps show analyzers cpp          # Show dependencies from the C/C++ analyzer
rsconstruct deps show analyzers cpp python   # Show dependencies from multiple analyzers
rsconstruct deps stats                       # Show statistics by analyzer
rsconstruct deps clean                       # Clear the entire dependency cache
rsconstruct deps clean --analyzer cpp        # Clear only C/C++ dependencies
rsconstruct deps clean --analyzer python     # Clear only Python dependencies

Example output for rsconstruct deps show all:

src/main.c: [cpp] (no dependencies)
src/test.c: [cpp]
  src/utils.h
  src/config.h
config/settings.py: [python]
  config/base.py

Example output for rsconstruct deps stats:

cpp: 15 files, 42 dependencies
python: 8 files, 12 dependencies

Total: 23 files, 54 dependencies

Note: This command reads directly from the dependency cache (.rsconstruct/deps.redb). If the cache is empty, run a build first to populate it.

This command is useful for:

• Debugging why a file is being rebuilt
• Understanding the include/import structure of your project
• Verifying that dependency analyzers are finding the right files
• Viewing statistics about cached dependencies by analyzer
• Clearing dependencies for a specific analyzer without affecting others

rsconstruct config

Show or inspect the configuration.

rsconstruct config show           # Show the active configuration (defaults merged with rsconstruct.toml)
rsconstruct config show-default   # Show the default configuration (without rsconstruct.toml overrides)
rsconstruct config validate       # Validate the configuration for errors and warnings

rsconstruct processors

rsconstruct processors list              # List processors with enabled/detected status and descriptions
rsconstruct processors list -a           # Include hidden processors
rsconstruct processors files             # Show source and target files for each enabled processor
rsconstruct processors files ruff        # Show files for a specific processor
rsconstruct processors files -a          # Include disabled and hidden processors
rsconstruct processors config ruff       # Show resolved configuration for a processor
rsconstruct processors defconfig ruff    # Show default configuration for a processor

rsconstruct tools

List or check external tools required by enabled processors.

rsconstruct tools list              # List required tools and which processor needs them
rsconstruct tools list -a           # Include tools from disabled processors
rsconstruct tools check             # Verify tool versions against .tools.versions lock file
rsconstruct tools lock              # Lock tool versions to .tools.versions
rsconstruct tools install           # Install all missing external tools
rsconstruct tools install ruff      # Install a specific tool by name
rsconstruct tools install -y        # Skip confirmation prompt
rsconstruct tools stats             # Show tool availability and language runtime breakdown
rsconstruct tools stats --json      # Show tool stats in JSON format
rsconstruct tools graph             # Show tool-to-processor dependency graph (DOT format)
rsconstruct tools graph --format mermaid  # Mermaid format
rsconstruct tools graph --view      # Open tool graph in browser

rsconstruct tags

Search and query frontmatter tags from markdown files.

rsconstruct tags list                        # List all unique tags
rsconstruct tags count                       # Show each tag with file count, sorted by frequency
rsconstruct tags tree                        # Show tags grouped by prefix/category
rsconstruct tags stats                       # Show statistics about the tags database
rsconstruct tags files docker                # List files matching a tag (AND semantics)
rsconstruct tags files docker --or k8s       # List files matching any tag (OR semantics)
rsconstruct tags files level:advanced        # Match key:value tags
rsconstruct tags grep deploy                 # Search for tags containing a substring
rsconstruct tags grep deploy -i              # Case-insensitive tag search
rsconstruct tags for-file src/main.md        # List all tags for a specific file
rsconstruct tags frontmatter src/main.md     # Show raw frontmatter for a file
rsconstruct tags validate                    # Validate tags against .tags file
rsconstruct tags unused                      # List tags in .tags not used by any file
rsconstruct tags unused --strict             # Exit with error if unused tags found (CI)
rsconstruct tags init                        # Generate .tags file from current tag union
rsconstruct tags add docker                  # Add a tag to the .tags file
rsconstruct tags remove docker               # Remove a tag from the .tags file
rsconstruct tags sync                        # Add missing tags to .tags
rsconstruct tags sync --prune                # Sync and remove unused tags from .tags

rsconstruct complete

Generate shell completions.

rsconstruct complete bash    # Generate bash completions
rsconstruct complete zsh     # Generate zsh completions
rsconstruct complete fish    # Generate fish completions

rsconstruct version

Print version information.

rsconstruct version

Configuration

RSConstruct is configured via an rsconstruct.toml file in the project root.

Full reference

[build]
parallel = 1          # Number of parallel jobs (1 = sequential, 0 = auto-detect CPU cores)
batch_size = 0        # Max files per batch for batch-capable processors (0 = no limit, omit to disable)

[processor]
auto_detect = true
enabled = ["tera", "ruff", "pylint", "mypy", "pyrefly", "cc_single_file", "cppcheck",
           "clang_tidy", "shellcheck", "spellcheck", "make", "cargo", "rumdl", "yamllint",
           "jq", "jsonlint", "taplo", "json_schema", "tags", "pip", "sphinx", "mdbook",
           "npm", "gem", "mdl", "markdownlint", "aspell", "marp", "pandoc", "markdown",
           "pdflatex", "a2x", "ascii_check", "mermaid", "drawio", "libreoffice", "pdfunite"]

[cache]
restore_method = "hardlink"  # or "copy" (hardlink is faster, copy works across filesystems)
remote = "s3://my-bucket/rsconstruct-cache"  # Optional: remote cache URL
remote_push = true       # Push local builds to remote (default: true)
remote_pull = true       # Pull from remote on cache miss (default: true)
mtime_check = true       # Use mtime pre-check to skip unchanged file checksums (default: true)

[analyzer]
auto_detect = true
enabled = ["cpp", "python"]

[graph]
viewer = "google-chrome"  # Command to open graph files (default: platform-specific)

[plugins]
dir = "plugins"  # Directory containing .lua processor plugins

[completions]
shells = ["bash"]

Per-processor configuration is documented on each processor’s page under Processors. Lua plugin configuration is documented under Lua Plugins.

Variable substitution

Define variables in a [vars] section and reference them using ${var_name} syntax:

[vars]
kernel_excludes = ["/kernel/", "/kernel_standalone/", "/examples_standalone/"]

[processor.cppcheck]
exclude_dirs = "${kernel_excludes}"

[processor.cc_single_file]
exclude_dirs = "${kernel_excludes}"

Variables are substituted before TOML parsing. The "${var_name}" (including quotes) is replaced with the TOML-serialized value, preserving types (arrays stay arrays, strings stay strings). Undefined variable references produce an error.

Section details

[build]

[processor]

[cache]

[analyzer]

[graph]

[plugins]

[completions]

Remote Caching

RSConstruct supports sharing build artifacts across machines via remote caching. When enabled, build outputs are pushed to a remote store and can be pulled by other machines, avoiding redundant rebuilds.

Configuration

Add a remote URL to your [cache] section in rsconstruct.toml:

[cache]
remote = "s3://my-bucket/rsconstruct-cache"

Supported Backends

Amazon S3

[cache]
remote = "s3://bucket-name/optional/prefix"

Requires:

• AWS CLI installed (aws command)
• AWS credentials configured (~/.aws/credentials or environment variables)

The S3 backend uses aws s3 cp and aws s3 ls commands.

HTTP/HTTPS

[cache]
remote = "http://cache-server.example.com:8080/rsconstruct"
# or
remote = "https://cache-server.example.com/rsconstruct"

Requires:

• curl command
• Server that supports GET and PUT requests

The HTTP backend expects:

• GET /path to return the object or 404
• PUT /path to store the object
• HEAD /path to check existence (returns 200 or 404)

Local Filesystem

[cache]
remote = "file:///shared/cache/rsconstruct"

Useful for:

• Network-mounted filesystems (NFS, CIFS)
• Testing remote cache behavior locally

Control Options

You can control push and pull separately:

[cache]
remote = "s3://my-bucket/rsconstruct-cache"
remote_push = true   # Push local builds to remote (default: true)
remote_pull = true   # Pull from remote on cache miss (default: true)

Pull-only mode

To share a read-only cache (e.g., from CI):

[cache]
remote = "s3://ci-cache/rsconstruct"
remote_push = false
remote_pull = true

Push-only mode

To populate a cache without using it (e.g., in CI):

[cache]
remote = "s3://ci-cache/rsconstruct"
remote_push = true
remote_pull = false

How It Works

Cache Structure

Remote cache stores two types of objects:

1. Index entries at index/{cache_key}

   • JSON mapping input checksums to output checksums
   • One entry per product (source file + processor + config)

2. Objects at objects/{xx}/{rest_of_checksum}

   • Content-addressed storage (like git)
   • Actual file contents identified by SHA-256

On Build

1. RSConstruct computes the cache key and input checksum
2. Checks local cache first
3. If local miss and remote_pull = true:
   • Fetches index entry from remote
   • Fetches required objects from remote
   • Restores outputs locally
4. If rebuild required:
   • Executes the processor
   • Stores outputs in local cache
   • If remote_push = true, pushes to remote

Cache Hit Flow

Local cache hit → Restore from local → Done
       ↓ miss
Remote cache hit → Download index + objects → Restore → Done
       ↓ miss
Execute processor → Cache locally → Push to remote → Done

Best Practices

CI/CD Integration

In your CI pipeline:

# .github/workflows/build.yml
env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

steps:
  - run: rsconstruct build

Separate CI and Developer Caches

Use different prefixes to avoid conflicts:

# CI: rsconstruct.toml.ci
[cache]
remote = "s3://cache/rsconstruct/ci"
remote_push = true
remote_pull = true

# Developers: rsconstruct.toml
[cache]
remote = "s3://cache/rsconstruct/ci"
remote_push = false  # Read from CI cache only
remote_pull = true

Cache Invalidation

Cache entries are keyed by:

• Processor name
• Source file path
• Processor configuration hash

To force a full rebuild ignoring caches:

rsconstruct build --force

To clear only the local cache:

rsconstruct cache clear

Troubleshooting

S3 Access Denied

Check your AWS credentials:

aws s3 ls s3://your-bucket/

HTTP Upload Failures

Ensure your server accepts PUT requests. Many static file servers are read-only.

Slow Remote Cache

Consider:

• Using a closer region for S3
• Enabling S3 Transfer Acceleration
• Using a caching proxy

Debug Mode

Use verbose output to see cache operations:

rsconstruct build -v

This shows which products are restored from local cache, remote cache, or rebuilt.

Dependency Caching

RSConstruct includes a dependency cache that stores source file dependencies (e.g., C/C++ header files) to avoid re-scanning files that haven’t changed. This significantly speeds up the graph-building phase for projects with many source files.

Overview

When processors like cc_single_file discover products, they need to scan source files to find dependencies (header files). This scanning can be slow for large projects. The dependency cache stores the results so subsequent builds can skip the scanning step.

The cache is stored in .rsconstruct/deps.redb using redb, an embedded key-value database.

Cache Structure

Each cache entry consists of:

• Key: Source file path (e.g., src/main.c)
• Value:
  • source_checksum — SHA-256 hash of the source file content
  • dependencies — list of dependency paths (header files)

Cache Lookup Algorithm

When looking up dependencies for a source file:

1. Look up the entry by source file path
2. If not found → cache miss, scan the file
3. If found, compute the current SHA-256 checksum of the source file
4. Compare with the stored checksum:
   • If different → cache miss (file changed), re-scan
   • If same → verify all cached dependencies still exist
5. If any dependency file is missing → cache miss, re-scan
6. Otherwise → cache hit, return cached dependencies

Why Path as Key (Not Checksum)?

An alternative design would use the source file’s checksum as the cache key instead of its path. This seems appealing because you could look up dependencies directly by content hash. However, this approach has significant drawbacks:

Problems with Checksum as Key

1. Mandatory upfront computation: With checksum as key, you must compute the SHA-256 hash of every source file before you can even check the cache. This means reading every file on every build, even when nothing has changed.

   With path as key, you do a fast O(1) lookup first. Only if there’s a cache hit do you compute the checksum to validate freshness.

2. Orphaned entries accumulate: When a file changes, its old checksum entry becomes orphaned garbage. You’d need periodic garbage collection to clean up stale entries.

   With path as key, the entry is naturally updated in place when the file changes.

3. No actual benefit: The checksum is still needed for validation regardless of the key choice. Using it as the key just moves when you compute it, without reducing total work.

Current Design

The current design is optimal:

Path (key) → O(1) lookup → Checksum validation (only on hit)

This minimizes work in the common case where files haven’t changed.

Cache Statistics

During graph construction, RSConstruct displays cache statistics:

[cc_single_file] Dependency cache: 42 hits, 3 recalculated

This shows how many source files had their dependencies retrieved from cache (hits) versus re-scanned (recalculated).

Viewing Dependencies

Use the rsconstruct deps command to view the dependencies stored in the cache:

rsconstruct deps all                    # Show all cached dependencies
rsconstruct deps for src/main.c         # Show dependencies for a specific file
rsconstruct deps for src/a.c src/b.c    # Show dependencies for multiple files
rsconstruct deps clean                  # Clear the dependency cache

Example output:

src/main.c: (no dependencies)
src/test.c:
  src/utils.h
  src/config.h

The rsconstruct deps command reads directly from the dependency cache without building the graph. If the cache is empty (e.g., after rsconstruct deps clean or on a fresh checkout), run a build first to populate it.

This is useful for debugging rebuild behavior or understanding the include structure of your project.

Cache Invalidation

The cache automatically invalidates entries when:

• The source file content changes (checksum mismatch)
• Any cached dependency file no longer exists

You can manually clear the entire dependency cache by removing the .rsconstruct/deps.redb file, or by running rsconstruct clean all which removes the entire .rsconstruct/ directory.

Processors Using Dependency Caching

Currently, the following processors use the dependency cache:

• cc_single_file — caches C/C++ header dependencies discovered by the include scanner

Implementation

The dependency cache is implemented in src/deps_cache.rs:

#![allow(unused)]
fn main() {
pub struct DepsCache {
    db: redb::Database,
    stats: DepsCacheStats,
}

impl DepsCache {
    pub fn open() -> Result<Self>;
    pub fn get(&mut self, source: &Path) -> Option<Vec<PathBuf>>;
    pub fn set(&self, source: &Path, dependencies: &[PathBuf]) -> Result<()>;
    pub fn flush(&self) -> Result<()>;
    pub fn stats(&self) -> &DepsCacheStats;
}
}

The cache is opened once per processor discovery phase, queried for each source file, and flushed to disk at the end.

Project Structure

RSConstruct follows a convention-over-configuration approach. The directory layout determines how files are processed.

Directory layout

project/
├── rsconstruct.toml          # Configuration file
├── .rsconstructignore        # Glob patterns for files to exclude
├── config/           # Python config files (loaded by templates)
├── tera.templates/   # .tera template files
├── templates.mako/   # .mako template files
├── src/              # C/C++ source files
├── plugins/          # Lua processor plugins (.lua files)
├── out/
│   ├── cc_single_file/ # Compiled executables
│   ├── ruff/         # Ruff lint stub files
│   ├── pylint/       # Pylint lint stub files
│   ├── cppcheck/      # C/C++ lint stub files
│   ├── spellcheck/   # Spellcheck stub files
│   └── make/         # Make stub files
└── .rsconstruct/             # Cache directory
    ├── index.json    # Cache index
    ├── objects/       # Cached build artifacts
    └── deps/          # Dependency files

Conventions

Templates

Files in tera.templates/ with configured extensions (default .tera) are rendered to the project root:

• tera.templates/Makefile.tera produces Makefile
• tera.templates/config.toml.tera produces config.toml

Similarly, files in templates.mako/ with .mako extensions are rendered via the Mako processor:

• templates.mako/Makefile.mako produces Makefile
• templates.mako/config.toml.mako produces config.toml

C/C++ sources

Files in the source directory (default src/) are compiled to executables under out/cc_single_file/, preserving the directory structure:

• src/main.c produces out/cc_single_file/main.elf
• src/utils/helper.cc produces out/cc_single_file/utils/helper.elf

Python files

Python files are linted and stub outputs are written to out/ruff/ (ruff processor) or out/pylint/ (pylint processor).

Build artifacts

All build outputs go into out/. The cache lives in .rsconstruct/. Use rsconstruct clean to remove out/ (preserving cache) or rsconstruct clean all to remove both.

Dependency Analyzers

rsconstruct uses dependency analyzers to scan source files and discover dependencies between files. Analyzers run after processors discover products and add dependency information to the build graph.

How Analyzers Work

1. Product Discovery: Processors discover products (source → output mappings)
2. Dependency Analysis: Analyzers scan source files to find dependencies
3. Graph Resolution: Dependencies are added to products for correct build ordering

Analyzers are decoupled from processors — they operate on any product with matching source files, regardless of which processor created it.

Built-in Analyzers

cpp

Scans C/C++ source files for #include directives and adds header file dependencies.

Auto-detects: Projects with .c, .cc, .cpp, .cxx, .h, .hh, .hpp, or .hxx files.

Features:

• Recursive header scanning (follows includes in header files)
• Queries compiler for system include paths (only tracks project-local headers)
• Handles both #include "file" (relative to source) and #include <file> (searches include paths)
• Supports native regex scanning and compiler-based scanning (gcc -MM)
• Uses dependency cache for incremental builds

System Header Detection:

The cpp analyzer queries the compiler for its include search paths using gcc -E -Wp,-v -xc /dev/null. This allows it to properly identify which headers are system headers vs project-local headers. Only headers within the project directory are tracked as dependencies.

Configuration (rsconstruct.toml):

[analyzer.cpp]
include_scanner = "native"  # or "compiler" for gcc -MM
include_paths = ["include", "src"]
pkg_config = ["gtk+-3.0", "libcurl"]  # Query pkg-config for include paths
include_path_commands = ["echo $(gcc -print-file-name=plugin)/include"]  # Run commands to get include paths
exclude_dirs = ["/kernel/", "/vendor/"]  # Skip analyzing files in these directories
cc = "gcc"
cxx = "g++"
cflags = ["-I/usr/local/include"]
cxxflags = ["-std=c++17"]

include_path_commands:

The include_path_commands option allows you to specify shell commands that output include paths. Each command is executed and its stdout (trimmed) is added to the include search paths. This is useful for compiler-specific include directories:

[analyzer.cpp]
include_path_commands = [
    "gcc -print-file-name=plugin",  # GCC plugin development headers
    "llvm-config --includedir",     # LLVM headers
]

pkg-config Integration:

The pkg_config option allows you to specify pkg-config packages. The analyzer will run pkg-config --cflags-only-I to get the include paths for these packages and add them to the header search path. This is useful when your code includes headers from system libraries:

[analyzer.cpp]
pkg_config = ["gtk+-3.0", "glib-2.0"]

This will automatically find headers like <gtk/gtk.h> and <glib.h> without needing to manually specify their include paths.

python

Scans Python source files for import and from ... import statements and adds dependencies on local Python modules.

Auto-detects: Projects with .py files.

Features:

• Resolves imports to local files (ignores stdlib/external packages)
• Supports both import foo and from foo import bar syntax
• Searches relative to source file and project root

Configuration

Analyzers can be configured in rsconstruct.toml:

[analyzer]
auto_detect = true  # auto-detect which analyzers to run (default: true)
enabled = ["cpp", "python"]  # list of enabled analyzers

Auto-detection

By default, analyzers use auto-detection to determine if they’re relevant for the project. An analyzer runs if:

1. It’s in the enabled list
2. AND either auto_detect = false, OR the analyzer detects relevant files

This is similar to how processors work.

Caching

Analyzer results are cached in the dependency cache (.rsconstruct/deps.redb). On subsequent builds:

• If a source file hasn’t changed, its cached dependencies are used
• If a source file has changed, dependencies are re-scanned
• The cache is shared across all analyzers

Use rsconstruct deps commands to inspect the cache:

rsconstruct deps all                # Show all cached dependencies
rsconstruct deps for src/main.c     # Show dependencies for specific files
rsconstruct deps clean              # Clear the dependency cache

Build Phases

With --phases flag, you can see when analyzers run:

rsconstruct --phases build

Output:

Phase: Building dependency graph...
  Phase: discover
  Phase: add_dependencies    # Analyzers run here
  Phase: apply_tool_version_hashes
  Phase: resolve_dependencies

Use --stop-after add-dependencies to stop after dependency analysis:

rsconstruct build --stop-after add-dependencies

Adding Custom Analyzers

Analyzers implement the DepAnalyzer trait:

#![allow(unused)]
fn main() {
pub trait DepAnalyzer: Sync + Send {
    fn description(&self) -> &str;
    fn auto_detect(&self, file_index: &FileIndex) -> bool;
    fn analyze(
        &self,
        graph: &mut BuildGraph,
        deps_cache: &mut DepsCache,
        file_index: &FileIndex,
    ) -> Result<()>;
}
}

The analyze method should:

1. Find products with relevant source files
2. Scan each source file for dependencies (using cache when available)
3. Add discovered dependencies to the product’s inputs

Processors

RSConstruct uses processors to discover and build products. Each processor scans for source files matching its conventions and produces output files.

Processor Types

Processors are classified into three types:

• Generators — produce real output files from input files (e.g., compiling code, rendering templates, transforming file formats)
• Checkers — validate input files without producing output files (e.g., linters, spell checkers, static analyzers). Success is recorded in the cache database.
• Mass generators — produce a directory of output files without enumerating them individually (e.g., sphinx, mdbook, cargo, pip, npm, gem). Output directories can be cached and restored as a whole — see Output directory caching below.

The processor type is displayed in rsconstruct processors list output:

cc_single_file [generator] enabled
ruff [checker] enabled
tera [generator] enabled

For checkers, rsconstruct processors files shows “(checker)” instead of output paths since no files are produced:

[ruff] (3 products)
src/foo.py → (checker)
src/bar.py → (checker)

Configuration

Enable processors in rsconstruct.toml:

[processor]
enabled = ["tera", "ruff", "pylint", "pyrefly", "cc_single_file", "cppcheck", "shellcheck", "spellcheck", "make", "yamllint", "jq", "jsonlint", "taplo", "json_schema"]

Use rsconstruct processors list to see available processors with enabled/detected status and descriptions. Use rsconstruct processors list --all to include hidden processors. Use rsconstruct processors files to see which files each processor discovers.

Available Processors

• Tera — renders Tera templates into output files
• Ruff — lints Python files with ruff
• Pylint — lints Python files with pylint
• Mypy — type-checks Python files with mypy
• Pyrefly — type-checks Python files with pyrefly
• CC — builds full C/C++ projects from cc.yaml manifests
• CC Single File — compiles C/C++ source files into executables (single-file)
• Linux Module — builds Linux kernel modules from linux-module.yaml manifests
• Cppcheck — runs static analysis on C/C++ source files
• Clang-Tidy — runs clang-tidy static analysis on C/C++ source files
• Shellcheck — lints shell scripts using shellcheck
• Spellcheck — checks documentation files for spelling errors
• Rumdl — lints Markdown files with rumdl
• Make — runs make in directories containing Makefiles
• Cargo — builds Rust projects using Cargo
• Yamllint — lints YAML files with yamllint
• Jq — validates JSON files with jq
• Jsonlint — lints JSON files with jsonlint
• Taplo — checks TOML files with taplo
• Json Schema — validates JSON schema propertyOrdering

Output Directory Caching

Mass generators (sphinx, mdbook, cargo, pip, npm, gem) produce output in directories rather than individual files. RSConstruct can cache these entire directories so that after rsconstruct clean && rsconstruct build, the output is restored from cache instead of being regenerated.

After a successful build, RSConstruct walks the output directory, stores every file as a content-addressed blob in .rsconstruct/objects/, and records a manifest (path, checksum, Unix permissions) in the cache entry. On restore, the entire directory is recreated from cached objects with permissions preserved.

This is controlled by the cache_output_dir config option (default true) on each mass generator:

[processor.sphinx]
cache_output_dir = true    # Cache _build/ directory (default)

[processor.cargo]
cache_output_dir = false   # Disable for large target/ directories

Output directories cached per processor:

When cache_output_dir is false, the processor falls back to the previous behavior (stamp-file or empty-output caching, no directory restore).

Custom Processors

You can define custom processors in Lua. See Lua Plugins for details.

A2x Processor

Purpose

Converts AsciiDoc files to PDF (or other formats) using a2x.

How It Works

Discovers .txt (AsciiDoc) files in the project and runs a2x on each file, producing output in the configured format.

Source Files

• Input: **/*.txt
• Output: out/a2x/{relative_path}.pdf

Configuration

[processor.a2x]
a2x = "a2x"                           # The a2x command to run
format = "pdf"                         # Output format (pdf, xhtml, dvi, ps, epub, mobi)
args = []                              # Additional arguments to pass to a2x
output_dir = "out/a2x"                # Output directory
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Ascii Check Processor

Purpose

Validates that files contain only ASCII characters.

How It Works

Discovers .md files in the project and checks each for non-ASCII characters. Files containing non-ASCII bytes fail the check. This is a built-in processor that does not require any external tools.

This processor supports batch mode, allowing multiple files to be checked in a single invocation.

Source Files

• Input: **/*.md
• Output: none (checker)

Configuration

[processor.ascii_check]
args = []                              # Additional arguments (unused, for consistency)
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Aspell Processor

Purpose

Checks spelling in Markdown files using aspell.

How It Works

Discovers .md files in the project and runs aspell on each file using the configured aspell configuration file. A non-zero exit code fails the product.

Source Files

• Input: **/*.md
• Output: none (checker)

Configuration

[processor.aspell]
aspell = "aspell"                      # The aspell command to run
conf_dir = "."                         # Configuration directory
conf = ".aspell.conf"                  # Aspell configuration file
args = []                              # Additional arguments to pass to aspell
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Cargo Processor

Purpose

Builds Rust projects using Cargo. Each Cargo.toml produces a cached success marker, allowing RSConstruct to skip rebuilds when source files haven’t changed.

How It Works

Discovers files named Cargo.toml in the project. For each Cargo.toml found, the processor runs cargo build (or a configured command) in that directory.

Input Tracking

The cargo processor tracks all .rs and .toml files in the Cargo.toml’s directory tree as inputs. This includes:

• Cargo.toml and Cargo.lock
• All Rust source files (src/**/*.rs)
• Test files, examples, benches
• Workspace member Cargo.toml files

When any tracked file changes, rsconstruct will re-run cargo.

Workspaces

For Cargo workspaces, each Cargo.toml (root and members) is discovered as a separate product. To build only the workspace root, use exclude_paths to skip member directories, or configure scan_dir to limit discovery.

Source Files

• Input: Cargo.toml plus all .rs and .toml files in the project tree
• Output: None (mass_generator — produces output in target directory)

Configuration

[processor]
enabled = ["cargo"]

[processor.cargo]
cargo = "cargo"          # Cargo binary to use
command = "build"        # Cargo command (build, check, test, clippy, etc.)
args = []                # Extra arguments passed to cargo
profiles = ["dev", "release"]  # Cargo profiles to build
scan_dir = ""            # Directory to scan ("" = project root)
extensions = ["Cargo.toml"]
extra_inputs = []        # Additional files that trigger rebuilds
cache_output_dir = true  # Cache the target/ directory for fast restore after clean

Examples

Basic Usage

[processor]
enabled = ["cargo"]

Release Only

[processor]
enabled = ["cargo"]

[processor.cargo]
profiles = ["release"]

Dev Only

[processor]
enabled = ["cargo"]

[processor.cargo]
profiles = ["dev"]

Use cargo check Instead of build

[processor]
enabled = ["cargo"]

[processor.cargo]
command = "check"

Run clippy

[processor]
enabled = ["cargo"]

[processor.cargo]
command = "clippy"
args = ["--", "-D", "warnings"]

Workspace Root Only

[processor]
enabled = ["cargo"]

[processor.cargo]
exclude_paths = ["crates/"]

Notes

• Cargo has its own incremental compilation, so rsconstruct’s caching mainly avoids invoking cargo at all when nothing changed
• The target/ directory is automatically excluded from input scanning
• For monorepos with multiple Rust projects, each Cargo.toml is built separately

CC Project Processor

Purpose

Builds full C/C++ projects with multiple targets (libraries and executables) defined in a cc.yaml manifest file. Unlike the CC Single File processor which compiles each source file into a standalone executable, this processor supports multi-file targets with dependency linking.

How It Works

The processor scans for cc.yaml files. Each manifest defines libraries and programs to build. All paths in the manifest (sources, include directories) are relative to the cc.yaml file’s location and are automatically resolved to project-root-relative paths before compilation. All commands run from the project root.

Output goes under out/cc/<path-to-cc.yaml-dir>/, so a manifest at src/exercises/foo/cc.yaml produces output in out/cc/src/exercises/foo/. A manifest at the project root produces output in out/cc/.

Source files are compiled to object files, then linked into the final targets:

src/exercises/foo/cc.yaml defines:
  library "mymath" (static) from math.c, utils.c
  program "main" from main.c, links mymath

Build produces:
  out/cc/src/exercises/foo/obj/mymath/math.o
  out/cc/src/exercises/foo/obj/mymath/utils.o
  out/cc/src/exercises/foo/lib/libmymath.a
  out/cc/src/exercises/foo/obj/main/main.o
  out/cc/src/exercises/foo/bin/main

cc.yaml Format

All paths in the manifest are relative to the cc.yaml file’s location.

# Global settings (all optional)
cc: gcc               # C compiler (default: gcc)
cxx: g++              # C++ compiler (default: g++)
cflags: [-Wall]       # Global C flags
cxxflags: [-Wall]     # Global C++ flags
ldflags: []           # Global linker flags
include_dirs: [include]  # Global -I paths (relative to cc.yaml location)

# Library definitions
libraries:
  - name: mymath
    lib_type: shared   # shared (.so) | static (.a) | both
    sources: [src/math.c, src/utils.c]
    include_dirs: [include]  # Additional -I for this library
    cflags: []               # Additional C flags
    cxxflags: []             # Additional C++ flags
    ldflags: [-lm]           # Linker flags for shared lib

  - name: myhelper
    lib_type: static
    sources: [src/helper.c]

# Program definitions
programs:
  - name: main
    sources: [src/main.c]
    link: [mymath, myhelper]  # Libraries defined above to link against
    ldflags: [-lpthread]      # Additional linker flags

  - name: tool
    sources: [src/tool.cc]    # .cc -> uses C++ compiler
    link: [mymath]

Library Types

Language Detection

The compiler is chosen per source file based on extension:

Global cflags are used for C files and cxxflags for C++ files.

Output Layout

Output is placed under out/cc/<cc.yaml-relative-dir>/:

out/cc/<cc.yaml-dir>/
  obj/<target_name>/    # Object files per target
    file.o
  lib/                  # Libraries
    lib<name>.a
    lib<name>.so
  bin/                  # Executables
    <program_name>

Build Modes

Compile + Link (default)

Each source is compiled to a .o file, then targets are linked from objects. This provides incremental rebuilds — only changed sources are recompiled.

Single Invocation

When single_invocation = true in rsconstruct.toml, programs are built by passing all sources directly to the compiler in one command. Libraries still use compile+link since ar requires object files.

Configuration

[processor.cc]
enabled = true            # Enable/disable (default: true)
cc = "gcc"                # Default C compiler (default: "gcc")
cxx = "g++"               # Default C++ compiler (default: "g++")
cflags = []               # Additional global C flags
cxxflags = []             # Additional global C++ flags
ldflags = []              # Additional global linker flags
include_dirs = []         # Additional global -I paths
single_invocation = false # Use single-invocation mode (default: false)
extra_inputs = []         # Extra files that trigger rebuilds
cache_output_dir = true   # Cache entire output directory (default: true)

Note: The cc.yaml manifest settings override the rsconstruct.toml defaults for compiler and flags.

Configuration Reference

Example

Given this project layout:

myproject/
  rsconstruct.toml
  exercises/
    math/
      cc.yaml
      include/
        math.h
      math.c
      main.c

With exercises/math/cc.yaml:

include_dirs: [include]

libraries:
  - name: math
    lib_type: static
    sources: [math.c]

programs:
  - name: main
    sources: [main.c]
    link: [math]

Running rsconstruct build produces:

out/cc/exercises/math/obj/math/math.o
out/cc/exercises/math/lib/libmath.a
out/cc/exercises/math/obj/main/main.o
out/cc/exercises/math/bin/main

CC Single File Processor

Purpose

Compiles C (.c) and C++ (.cc) source files into executables, one source file per executable.

How It Works

Source files under the configured source directory are compiled into executables under out/cc_single_file/, mirroring the directory structure:

src/main.c       →  out/cc_single_file/main.elf
src/a/b.c        →  out/cc_single_file/a/b.elf
src/app.cc       →  out/cc_single_file/app.elf

Header dependencies are automatically tracked via compiler-generated .d files (-MMD -MF). When a header changes, all source files that include it are rebuilt.

Source Files

• Input: {source_dir}/**/*.c, {source_dir}/**/*.cc
• Output: out/cc_single_file/{relative_path}{output_suffix}

Per-File Flags

Per-file compile and link flags can be set via special comments in source files. This allows individual files to require specific libraries or compiler options without affecting the entire project.

Flag directives

// EXTRA_COMPILE_FLAGS_BEFORE=-pthread
// EXTRA_COMPILE_FLAGS_AFTER=-O2 -DNDEBUG
// EXTRA_LINK_FLAGS_BEFORE=-L/usr/local/lib
// EXTRA_LINK_FLAGS_AFTER=-lX11

Command directives

Execute a command and use its stdout as flags (no shell):

// EXTRA_COMPILE_CMD=pkg-config --cflags gtk+-3.0
// EXTRA_LINK_CMD=pkg-config --libs gtk+-3.0

Shell directives

Execute via sh -c (full shell syntax):

// EXTRA_COMPILE_SHELL=echo -DLEVEL2_CACHE_LINESIZE=$(getconf LEVEL2_CACHE_LINESIZE)
// EXTRA_LINK_SHELL=echo -L$(brew --prefix openssl)/lib

Backtick substitution

Flag directives also support backtick substitution for inline command execution:

// EXTRA_COMPILE_FLAGS_AFTER=`pkg-config --cflags gtk+-3.0`
// EXTRA_LINK_FLAGS_AFTER=`pkg-config --libs gtk+-3.0`

Command caching

All command and shell directives (EXTRA_*_CMD, EXTRA_*_SHELL, and backtick substitutions) are cached in memory during a build. If multiple source files use the same command (e.g., pkg-config --cflags gtk+-3.0), it is executed only once. This improves build performance when many files share common dependencies.

Compiler profile-specific flags

When using multiple compiler profiles, you can specify flags that only apply to a specific compiler by adding [profile_name] after the directive name:

// EXTRA_COMPILE_FLAGS_BEFORE=-g
// EXTRA_COMPILE_FLAGS_BEFORE[gcc]=-femit-struct-debug-baseonly
// EXTRA_COMPILE_FLAGS_BEFORE[clang]=-gline-tables-only

In this example:

• -g is applied to all compilers
• -femit-struct-debug-baseonly is only applied when compiling with the “gcc” profile
• -gline-tables-only is only applied when compiling with the “clang” profile

The profile name matches the name field in your [[processor.cc_single_file.compilers]] configuration:

[[processor.cc_single_file.compilers]]
name = "gcc"      # Matches [gcc] suffix
cc = "gcc"

[[processor.cc_single_file.compilers]]
name = "clang"    # Matches [clang] suffix
cc = "clang"

This works with all directive types:

• EXTRA_COMPILE_FLAGS_BEFORE[profile]
• EXTRA_COMPILE_FLAGS_AFTER[profile]
• EXTRA_LINK_FLAGS_BEFORE[profile]
• EXTRA_LINK_FLAGS_AFTER[profile]
• EXTRA_COMPILE_CMD[profile]
• EXTRA_LINK_CMD[profile]
• EXTRA_COMPILE_SHELL[profile]
• EXTRA_LINK_SHELL[profile]

Excluding files from specific profiles

To exclude a source file from being compiled with specific compiler profiles, use EXCLUDE_PROFILE:

// EXCLUDE_PROFILE=clang

This is useful when a file uses compiler-specific features that aren’t available in other compilers. For example, a file using GCC-only builtins like __builtin_va_arg_pack_len():

// EXCLUDE_PROFILE=clang
// This file uses GCC-specific builtins
#include <stdarg.h>

void example(int first, ...) {
    int count = __builtin_va_arg_pack_len();  // GCC-only
    // ...
}

You can exclude multiple profiles by listing them space-separated:

// EXCLUDE_PROFILE=clang icc

Directive summary

Supported comment styles

Directives can appear in any of these comment styles:

C++ style:

// EXTRA_LINK_FLAGS_AFTER=-lX11

C block comment (single line):

/* EXTRA_LINK_FLAGS_AFTER=-lX11 */

C block comment (multi-line, star-prefixed):

/*
 * EXTRA_LINK_FLAGS_AFTER=-lX11
 */

Command Line Ordering

The compiler command is constructed in this order:

compiler -MMD -MF deps -I... [compile_before] [cflags/cxxflags] [compile_after] -o output source [link_before] [ldflags] [link_after]

Link flags come after the source file so the linker can resolve symbols correctly.

Verbosity Levels (--processor-verbose N)

Configuration

Single Compiler (Legacy)

[processor.cc_single_file]
cc = "gcc"                # C compiler (default: "gcc")
cxx = "g++"               # C++ compiler (default: "g++")
cflags = []               # C compiler flags
cxxflags = []             # C++ compiler flags
ldflags = []              # Linker flags
include_paths = []        # Additional -I paths (relative to project root)
scan_dir = "src"          # Source directory (default: "src")
output_suffix = ".elf"    # Suffix for output executables (default: ".elf")
extra_inputs = []         # Additional files that trigger rebuilds when changed
include_scanner = "native" # Method for scanning header dependencies (default: "native")

Multiple Compilers

To compile with multiple compilers (e.g., both GCC and Clang), use the compilers array:

[processor.cc_single_file]
scan_dir = "src"
include_paths = ["include"]  # Shared across all compilers

[[processor.cc_single_file.compilers]]
name = "gcc"
cc = "gcc"
cxx = "g++"
cflags = ["-Wall", "-Wextra"]
cxxflags = ["-Wall", "-Wextra"]
ldflags = []
output_suffix = ".elf"

[[processor.cc_single_file.compilers]]
name = "clang"
cc = "clang"
cxx = "clang++"
cflags = ["-Wall", "-Wextra", "-Weverything"]
cxxflags = ["-Wall", "-Wextra"]
ldflags = []
output_suffix = ".elf"

When using multiple compilers, outputs are organized by compiler name:

src/main.c  →  out/cc_single_file/gcc/main.elf
            →  out/cc_single_file/clang/main.elf

Each source file is compiled once per compiler profile, allowing you to:

• Test code with multiple compilers to catch different warnings
• Compare output between compilers
• Build for different targets (cross-compilation)

Configuration Reference

Compiler Profile Fields

Each entry in the compilers array can have:

Include Scanner

The include_scanner option controls how header dependencies are discovered:

Native scanner behavior

The native scanner:

• Recursively follows #include directives
• Searches include paths in order: source file directory, configured include_paths, project root
• Skips system headers (/usr/..., /lib/...)
• Only tracks project-local headers (relative paths)

When to use compiler scanner

Use include_scanner = "compiler" if you have:

• Computed includes: #include MACRO_THAT_EXPANDS_TO_FILENAME
• Complex conditional compilation affecting which headers are included
• Headers outside the standard search paths that the native scanner misses

The native scanner may occasionally report extra dependencies (false positives), which is safe—it just means some files might rebuild unnecessarily. It will not miss dependencies (false negatives) for standard #include patterns.

Checkpatch Processor

Purpose

Checks C source files using the Linux kernel’s checkpatch.pl script.

How It Works

Discovers .c and .h files under src/ (excluding common C/C++ build directories), runs checkpatch.pl on each file, and records success in the cache. A non-zero exit code from checkpatch fails the product.

This processor supports batch mode.

Source Files

• Input: src/**/*.c, src/**/*.h
• Output: none (checker)

Configuration

[processor.checkpatch]
args = []
extra_inputs = []

Checkstyle Processor

Purpose

Checks Java code style using Checkstyle.

How It Works

Discovers .java files in the project (excluding common build tool directories), runs checkstyle on each file, and records success in the cache. A non-zero exit code from checkstyle fails the product.

This processor supports batch mode.

If a checkstyle.xml file exists, it is automatically added as an extra input so that configuration changes trigger rebuilds.

Source Files

• Input: **/*.java
• Output: none (checker)

Configuration

[processor.checkstyle]
args = []
extra_inputs = []

Chromium Processor

Purpose

Converts HTML files to PDF using headless Chromium (Google Chrome).

How It Works

Discovers .html files in the configured scan directory (default: out/marp) and runs headless Chromium with --print-to-pdf on each file, producing a PDF output.

This is typically used as a post-processing step after another processor (e.g., Marp) generates HTML files.

Source Files

• Input: out/marp/**/*.html (default scan directory)
• Output: out/chromium/{relative_path}.pdf

Configuration

[processor.chromium]
chromium_bin = "google-chrome"            # The Chromium/Chrome executable to run
args = []                                 # Additional arguments to pass to Chromium
output_dir = "out/chromium"               # Output directory for PDFs
extra_inputs = []                         # Additional files that trigger rebuilds when changed

Clang-Tidy Processor

Purpose

Runs clang-tidy static analysis on C/C++ source files.

How It Works

Discovers .c and .cc files under the configured source directory, runs clang-tidy on each file individually, and creates a stub file on success. A non-zero exit code from clang-tidy fails the product.

Note: This processor does not support batch mode. Each file is checked separately to avoid cross-file analysis issues with unrelated files.

Source Files

• Input: {source_dir}/**/*.c, {source_dir}/**/*.cc
• Output: out/clang_tidy/{flat_name}.clang_tidy

Configuration

[processor.clang_tidy]
args = ["-checks=*"]                        # Arguments passed to clang-tidy
compiler_args = ["-std=c++17"]              # Arguments passed after -- to the compiler
extra_inputs = [".clang-tidy"]              # Additional files that trigger rebuilds when changed

Compiler Arguments

Clang-tidy requires knowing compiler flags to properly parse the source files. Use compiler_args to specify include paths, defines, and language standards:

[processor.clang_tidy]
compiler_args = ["-std=c++17", "-I/usr/include/mylib", "-DDEBUG"]

Using .clang-tidy File

Clang-tidy automatically reads configuration from a .clang-tidy file in the project root. Add it to extra_inputs so changes trigger rebuilds:

[processor.clang_tidy]
extra_inputs = [".clang-tidy"]

Clippy Processor

Purpose

Lints Rust projects using Cargo Clippy. Each Cargo.toml produces a cached success marker, allowing RSConstruct to skip re-linting when source files haven’t changed.

How It Works

Discovers files named Cargo.toml in the project. For each Cargo.toml found, the processor runs cargo clippy in that directory. A non-zero exit code fails the product.

Input Tracking

The clippy processor tracks all .rs and .toml files in the Cargo.toml’s directory tree as inputs. This includes:

• Cargo.toml and Cargo.lock
• All Rust source files (src/**/*.rs)
• Test files, examples, benches
• Workspace member Cargo.toml files

When any tracked file changes, rsconstruct will re-run clippy.

Source Files

• Input: Cargo.toml plus all .rs and .toml files in the project tree
• Output: None (checker-style caching)

Configuration

[processor]
enabled = ["clippy"]

[processor.clippy]
cargo = "cargo"          # Cargo binary to use
command = "clippy"       # Cargo command (usually "clippy")
args = []                # Extra arguments passed to cargo clippy
scan_dir = ""            # Directory to scan ("" = project root)
extensions = ["Cargo.toml"]
extra_inputs = []        # Additional files that trigger rebuilds

Examples

Basic Usage

[processor]
enabled = ["clippy"]

Deny All Warnings

[processor]
enabled = ["clippy"]

[processor.clippy]
args = ["--", "-D", "warnings"]

Use Both Cargo Build and Clippy

[processor]
enabled = ["cargo", "clippy"]

Notes

• Clippy uses the cargo binary which is shared with the cargo processor
• The target/ directory is automatically excluded from input scanning
• For monorepos with multiple Rust projects, each Cargo.toml is linted separately

CMake Processor

Purpose

Lints CMake files using cmake --lint.

How It Works

Discovers CMakeLists.txt files in the project (excluding common build tool directories), runs cmake --lint on each file, and records success in the cache. A non-zero exit code from cmake fails the product.

This processor supports batch mode.

Source Files

• Input: **/CMakeLists.txt
• Output: none (checker)

Configuration

[processor.cmake]
args = []
extra_inputs = []

Cppcheck Processor

Purpose

Runs cppcheck static analysis on C/C++ source files.

How It Works

Discovers .c and .cc files under the configured source directory, runs cppcheck on each file individually, and creates a stub file on success. A non-zero exit code from cppcheck fails the product.

Note: This processor does not support batch mode. Each file is checked separately because cppcheck performs cross-file analysis (CTU - Cross Translation Unit) which produces false positives when unrelated files are checked together. For example, standalone example programs that define classes with the same name will trigger ctuOneDefinitionRuleViolation errors even though the files are never linked together. Cppcheck has no flag to disable this cross-file analysis (--max-ctu-depth=0 does not help), so files must be checked individually.

Source Files

• Input: {source_dir}/**/*.c, {source_dir}/**/*.cc
• Output: out/cppcheck/{flat_name}.cppcheck

Configuration

[processor.cppcheck]
args = ["--error-exitcode=1", "--enable=warning,style,performance,portability"]
extra_inputs = [".cppcheck-suppressions"]   # Additional files that trigger rebuilds when changed

To use a suppressions file, add "--suppressions-list=.cppcheck-suppressions" to args.

Cpplint Processor

Purpose

Lints C/C++ files using cpplint (Google C++ style checker).

How It Works

Discovers .c, .cc, .h, and .hh files under src/ (excluding common C/C++ build directories), runs cpplint on each file, and records success in the cache. A non-zero exit code from cpplint fails the product.

This processor supports batch mode.

Source Files

• Input: src/**/*.c, src/**/*.cc, src/**/*.h, src/**/*.hh
• Output: none (checker)

Configuration

[processor.cpplint]
args = []
extra_inputs = []

Drawio Processor

Purpose

Converts Draw.io diagram files to PNG, SVG, or PDF.

How It Works

Discovers .drawio files in the project and runs drawio in export mode on each file, generating output in the configured formats.

Source Files

• Input: **/*.drawio
• Output: out/drawio/{format}/{relative_path}.{format}

Configuration

[processor.drawio]
drawio_bin = "drawio"                  # The drawio command to run
formats = ["png"]                      # Output formats (png, svg, pdf)
args = []                              # Additional arguments to pass to drawio
output_dir = "out/drawio"              # Output directory
extra_inputs = []                      # Additional files that trigger rebuilds when changed

ESLint Processor

Purpose

Lints JavaScript and TypeScript files using ESLint.

How It Works

Discovers .js, .jsx, .ts, .tsx, .mjs, and .cjs files in the project (excluding common build tool directories), runs eslint on each file, and records success in the cache. A non-zero exit code from eslint fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single eslint invocation for better performance.

If an ESLint config file exists (.eslintrc* or eslint.config.*), it is automatically added as an extra input so that configuration changes trigger rebuilds.

Source Files

• Input: **/*.js, **/*.jsx, **/*.ts, **/*.tsx, **/*.mjs, **/*.cjs
• Output: none (checker)

Configuration

[processor.eslint]
linter = "eslint"
args = []
extra_inputs = []

Gem Processor

Purpose

Installs Ruby dependencies from Gemfile files using Bundler.

How It Works

Discovers Gemfile files in the project, runs bundle install in each directory, and creates a stamp file on success. Sibling .rb and .gemspec files are tracked as inputs.

Source Files

• Input: **/Gemfile (plus sibling .rb, .gemspec files)
• Output: out/gem/{flat_name}.stamp

Configuration

[processor.gem]
bundler = "bundle"                     # The bundler command to run
command = "install"                    # The bundle subcommand to execute
args = []                              # Additional arguments to pass to bundler
extra_inputs = []                      # Additional files that trigger rebuilds when changed
cache_output_dir = true                # Cache the vendor/bundle directory for fast restore after clean

Hadolint Processor

Purpose

Lints Dockerfiles using Hadolint.

How It Works

Discovers Dockerfile files in the project (excluding common build tool directories), runs hadolint on each file, and records success in the cache. A non-zero exit code from hadolint fails the product.

This processor supports batch mode.

Source Files

• Input: **/Dockerfile
• Output: none (checker)

Configuration

[processor.hadolint]
args = []
extra_inputs = []

HTMLHint Processor

Purpose

Lints HTML files using HTMLHint.

How It Works

Discovers .html and .htm files in the project (excluding common build tool directories), runs htmlhint on each file, and records success in the cache. A non-zero exit code from htmlhint fails the product.

This processor supports batch mode.

If a .htmlhintrc file exists, it is automatically added as an extra input so that configuration changes trigger rebuilds.

Source Files

• Input: **/*.html, **/*.htm
• Output: none (checker)

Configuration

[processor.htmlhint]
linter = "htmlhint"
args = []
extra_inputs = []

HTMLLint Processor

Purpose

Lints HTML files using htmllint.

How It Works

Discovers .html and .htm files in the project (excluding common build tool directories), runs htmllint on each file, and records success in the cache. A non-zero exit code from htmllint fails the product.

This processor supports batch mode.

Source Files

• Input: **/*.html, **/*.htm
• Output: none (checker)

Configuration

[processor.htmllint]
args = []
extra_inputs = []

Jekyll Processor

Purpose

Builds Jekyll static sites by running jekyll build in directories containing a _config.yml file.

How It Works

Discovers _config.yml files in the project (excluding common build tool directories). For each one, runs jekyll build in that directory.

Source Files

• Input: **/_config.yml
• Output: none (mass generator)

Configuration

[processor.jekyll]
args = []
extra_inputs = []

Jq Processor

Purpose

Validates JSON files using jq.

How It Works

Discovers .json files in the project (excluding common build tool directories), runs jq empty on each file, and records success in the cache. The empty filter validates JSON syntax without producing output — a non-zero exit code from jq fails the product.

This processor supports batch mode — multiple files are checked in a single jq invocation.

Source Files

• Input: **/*.json
• Output: none (linter)

Configuration

[processor.jq]
linter = "jq"                               # The jq command to run
args = []                                    # Additional arguments to pass to jq (after "empty")
extra_inputs = []                            # Additional files that trigger rebuilds when changed

JSHint Processor

Purpose

Lints JavaScript files using JSHint.

How It Works

Discovers .js, .jsx, .mjs, and .cjs files in the project (excluding common build tool directories), runs jshint on each file, and records success in the cache. A non-zero exit code from jshint fails the product.

This processor supports batch mode.

If a .jshintrc file exists, it is automatically added as an extra input so that configuration changes trigger rebuilds.

Source Files

• Input: **/*.js, **/*.jsx, **/*.mjs, **/*.cjs
• Output: none (checker)

Configuration

[processor.jshint]
linter = "jshint"
args = []
extra_inputs = []

JSLint Processor

Purpose

Lints JavaScript files using JSLint.

How It Works

Discovers .js files in the project (excluding common build tool directories), runs jslint on each file, and records success in the cache. A non-zero exit code from jslint fails the product.

This processor supports batch mode.

Source Files

• Input: **/*.js
• Output: none (checker)

Configuration

[processor.jslint]
args = []
extra_inputs = []

Json Schema Processor

Purpose

Validates JSON schema files by checking that every object’s propertyOrdering array exactly matches its properties keys.

How It Works

Discovers .json files in the project (excluding common build tool directories), parses each as JSON, and recursively walks the structure. At every object node with "type": "object", if both properties and propertyOrdering exist, it verifies that the two key sets match exactly.

Mismatches (keys missing from propertyOrdering or extra keys in propertyOrdering) are reported with their JSON path. Files that contain no propertyOrdering at all pass silently.

This is a pure-Rust checker — no external tool is required.

Source Files

• Input: **/*.json
• Output: none (checker)

Configuration

[processor.json_schema]
args = []                                    # Reserved for future use
extra_inputs = []                            # Additional files that trigger rebuilds when changed

Jsonlint Processor

Purpose

Lints JSON files using jsonlint.

How It Works

Discovers .json files in the project (excluding common build tool directories), runs jsonlint on each file, and records success in the cache. A non-zero exit code from jsonlint fails the product.

This processor does not support batch mode — each file is checked individually.

Source Files

• Input: **/*.json
• Output: none (checker)

Configuration

[processor.jsonlint]
linter = "jsonlint"                          # The jsonlint command to run
args = []                                    # Additional arguments to pass to jsonlint
extra_inputs = []                            # Additional files that trigger rebuilds when changed

Libreoffice Processor

Purpose

Converts LibreOffice documents (e.g., .odp presentations) to PDF or other formats.

How It Works

Discovers .odp files in the project and runs libreoffice in headless mode to convert each file to the configured output formats. Uses flock to serialize invocations since LibreOffice only supports a single running instance.

Source Files

• Input: **/*.odp
• Output: out/libreoffice/{format}/{relative_path}.{format}

Configuration

[processor.libreoffice]
libreoffice_bin = "libreoffice"        # The libreoffice command to run
formats = ["pdf"]                      # Output formats (pdf, pptx)
args = []                              # Additional arguments to pass to libreoffice
output_dir = "out/libreoffice"         # Output directory
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Linux Module Processor

Purpose

Builds Linux kernel modules (.ko files) from source, driven by a linux-module.yaml manifest. The processor generates a temporary Kbuild file, invokes the kernel build system (make -C <kdir> M=<src> modules), copies the resulting .ko to the output directory, and cleans up build artifacts from the source tree.

How It Works

The processor scans for linux-module.yaml files. Each manifest lists one or more kernel modules to build. For each module the processor:

1. Generates a Kbuild file in the source directory (next to the yaml).
2. Runs make -C <kdir> M=<absolute-source-dir> modules to compile.
3. Copies the .ko file to out/linux-module/<yaml-relative-dir>/.
4. Runs make ... clean and removes the generated Kbuild so the source directory stays clean.

Because the kernel build system requires M= to point at an absolute path containing the sources and Kbuild, the make command runs in the yaml file’s directory — not the project root.

The processor is a generator: it knows exactly which .ko files it produces. Outputs are tracked in the build graph, cached in the object store, and can be restored from cache after rsconstruct clean without recompiling.

linux-module.yaml Format

All source paths are relative to the yaml file’s directory.

# Global settings (all optional)
make: make                    # Make binary (default: "make")
kdir: /lib/modules/6.8.0-generic/build  # Kernel build dir (default: running kernel)
arch: x86_64                  # ARCH= value (optional, omitted if unset)
cross_compile: x86_64-linux-gnu-  # CROSS_COMPILE= value (optional)
v: 0                          # Verbosity V= (default: 0)
w: 1                          # Warning level W= (default: 1)

# Module definitions
modules:
  - name: hello               # Module name -> produces hello.ko
    sources: [main.c]         # Source files (relative to yaml dir)
    extra_cflags: [-DDEBUG]   # Extra CFLAGS (optional, becomes ccflags-y)

  - name: mydriver
    sources: [mydriver.c, utils.c]

Minimal Example

A single module with one source file:

modules:
  - name: hello
    sources: [main.c]

Output Layout

Output is placed under out/linux-module/<yaml-relative-dir>/:

out/linux-module/<yaml-dir>/
  <module_name>.ko

For example, a manifest at src/kernel/hello/linux-module.yaml defining module hello produces:

out/linux-module/src/kernel/hello/hello.ko

KDIR Detection

If kdir is not set in the manifest, the processor runs uname -r to detect the running kernel and uses /lib/modules/<release>/build. This requires the linux-headers-* package to be installed (e.g., linux-headers-generic on Ubuntu).

Generated Kbuild

The processor writes a Kbuild file with the standard kernel module variables:

obj-m := hello.o
hello-objs := main.o
ccflags-y := -DDEBUG       # only if extra_cflags is non-empty

This file is removed after building (whether the build succeeds or fails).

Configuration

[processor.linux_module]
enabled = true           # Enable/disable (default: true)
extra_inputs = []        # Extra files that trigger rebuilds

Configuration Reference

Caching

The .ko outputs are cached in the rsconstruct object store. After rsconstruct clean, a subsequent rsconstruct build restores .ko files from cache (via hardlink or copy) without invoking the kernel build system. A rebuild is triggered when any source file or the yaml manifest changes.

Prerequisites

• make must be installed
• Kernel headers must be installed for the target kernel version (apt install linux-headers-generic on Ubuntu)
• For cross-compilation, the appropriate cross-compiler toolchain must be available and specified via cross_compile and arch in the manifest

Example

Given this project layout:

myproject/
  rsconstruct.toml
  drivers/
    hello/
      linux-module.yaml
      main.c

With drivers/hello/linux-module.yaml:

modules:
  - name: hello
    sources: [main.c]

And drivers/hello/main.c:

#include <linux/module.h>
#include <linux/init.h>

MODULE_LICENSE("GPL");

static int __init hello_init(void) {
    pr_info("hello: loaded\n");
    return 0;
}

static void __exit hello_exit(void) {
    pr_info("hello: unloaded\n");
}

module_init(hello_init);
module_exit(hello_exit);

Running rsconstruct build produces:

out/linux-module/drivers/hello/hello.ko

The module can then be loaded with sudo insmod out/linux-module/drivers/hello/hello.ko.

Luacheck Processor

Purpose

Lints Lua scripts using luacheck.

How It Works

Discovers .lua files in the project (excluding common build tool directories), runs luacheck on each file, and records success in the cache. A non-zero exit code from luacheck fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single luacheck invocation for better performance.

Source Files

• Input: **/*.lua
• Output: none (linter)

Configuration

[processor.luacheck]
linter = "luacheck"                         # The luacheck command to run
args = []                                    # Additional arguments to pass to luacheck
extra_inputs = []                            # Additional files that trigger rebuilds when changed

Make Processor

Purpose

Runs make in directories containing Makefiles. Each Makefile produces a stub file on success, allowing RSConstruct to track incremental rebuilds.

How It Works

Discovers files named Makefile in the project. For each Makefile found, the processor runs make (or a configured alternative) in the Makefile’s directory. A stub file is created on success.

Directory-Level Inputs

The make processor treats all files in the Makefile’s directory (and subdirectories) as inputs. This means that if any file alongside the Makefile changes — source files, headers, scripts, included makefiles — rsconstruct will re-run make.

This is slightly conservative: a change to a file that the Makefile does not actually depend on will trigger a rebuild. In practice this is the right trade-off because Makefiles can depend on arbitrary files and there is no reliable way to know which ones without running make itself.

Source Files

• Input: **/Makefile plus all files in the Makefile’s directory tree
• Output: out/make/{relative_path}.done

Dependency Tracking Approaches

RSConstruct uses the directory-scan approach described above. Here is why, and what the alternatives are.

1. Directory scan (current)

Track every file under the Makefile’s directory as an input. Any change triggers a rebuild.

Pros: simple, correct, zero configuration. Cons: over-conservative — a change to an unrelated file in the same directory triggers a needless rebuild.

2. User-declared extra inputs

The user lists specific files or globs in extra_inputs. Only those files (plus the Makefile itself) are tracked.

Pros: precise, no unnecessary rebuilds. Cons: requires the user to manually maintain the list. Easy to forget a file and get stale builds.

This is available today via the extra_inputs config key, but on its own it would miss source files that the Makefile compiles.

3. Parse make --dry-run --print-data-base

Ask make to dump its dependency database and extract the real inputs.

Pros: exact dependency information, no over-building. Cons: fragile — output format varies across make implementations (GNU Make, BSD Make, nmake). Some Makefiles behave differently in dry-run mode. Complex to implement and maintain.

4. Hash the directory tree

Instead of listing individual files, compute a single hash over every file in the directory. Functionally equivalent to option 1 but with a different internal representation.

Pros: compact cache key. Cons: same over-conservatism as option 1, and no ability to report which file changed.

Configuration

[processor.make]
make = "make"        # Make binary to use
args = []            # Extra arguments passed to make
target = ""          # Make target (empty = default target)
scan_dir = ""        # Directory to scan ("" = project root)
extensions = ["Makefile"]
extra_inputs = []    # Additional files that trigger rebuilds when changed

Mako Processor

Purpose

Renders Mako template files into output files using the Python Mako template library.

How It Works

Files matching configured extensions in templates.mako/ are rendered via python3 using the mako Python library. Output is written with the extension stripped and the templates.mako/ prefix removed:

templates.mako/app.config.mako  →  app.config
templates.mako/sub/readme.txt.mako  →  sub/readme.txt

Templates use the Mako templating engine. A TemplateLookup is configured with the project root as the lookup directory, so templates can include or inherit from other templates using relative paths.

Source Files

• Input: templates.mako/**/*{extensions}
• Output: project root, mirroring the template path (minus templates.mako/ prefix) with the extension removed

Configuration

[processor.mako]
extensions = [".mako"]                    # File extensions to process (default: [".mako"])
extra_inputs = ["config/settings.py"]     # Additional files that trigger rebuilds when changed

Markdown Processor

Purpose

Converts Markdown files to HTML using the markdown Perl script.

How It Works

Discovers .md files in the project and runs markdown on each file, producing an HTML output file.

Source Files

• Input: **/*.md
• Output: out/markdown/{relative_path}.html

Configuration

[processor.markdown]
markdown_bin = "markdown"              # The markdown command to run
args = []                              # Additional arguments to pass to markdown
output_dir = "out/markdown"            # Output directory
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Markdownlint Processor

Purpose

Lints Markdown files using markdownlint (Node.js).

How It Works

Discovers .md files in the project and runs markdownlint on each file. A non-zero exit code fails the product.

Depends on the npm processor — uses the markdownlint binary installed by npm.

Source Files

• Input: **/*.md
• Output: none (checker)

Configuration

[processor.markdownlint]
markdownlint_bin = "node_modules/.bin/markdownlint"  # Path to the markdownlint binary
args = []                              # Additional arguments to pass to markdownlint
npm_stamp = "out/npm/root.stamp"       # Stamp file from npm processor (dependency)
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Marp Processor

Purpose

Converts Markdown slides to PDF, PPTX, or HTML using Marp.

How It Works

Discovers .md files in the project and runs marp on each file, generating output in the configured formats. Each format produces a separate output file.

Source Files

• Input: **/*.md
• Output: out/marp/{format}/{relative_path}.{format}

Configuration

[processor.marp]
marp_bin = "marp"                      # The marp command to run
formats = ["pdf"]                      # Output formats (pdf, pptx, html)
args = ["--html", "--allow-local-files"]  # Additional arguments to pass to marp
output_dir = "out/marp"                # Output directory
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Mdbook Processor

Purpose

Builds mdbook documentation projects.

How It Works

Discovers book.toml files indicating mdbook projects, collects sibling .md and .toml files as inputs, and runs mdbook build. A non-zero exit code fails the product.

Source Files

• Input: **/book.toml (plus sibling .md, .toml files)
• Output: none (mass_generator — produces output in book directory)

Configuration

[processor.mdbook]
mdbook = "mdbook"                      # The mdbook command to run
output_dir = "book"                    # Output directory for generated docs
args = []                              # Additional arguments to pass to mdbook
extra_inputs = []                      # Additional files that trigger rebuilds when changed
cache_output_dir = true                # Cache the output directory for fast restore after clean

Mdl Processor

Purpose

Lints Markdown files using mdl (Ruby markdownlint).

How It Works

Discovers .md files in the project and runs mdl on each file. A non-zero exit code fails the product.

Depends on the gem processor — uses the mdl binary installed by Bundler.

Source Files

• Input: **/*.md
• Output: none (checker)

Configuration

[processor.mdl]
gem_home = "gems"                      # GEM_HOME directory
mdl_bin = "gems/bin/mdl"              # Path to the mdl binary
args = []                              # Additional arguments to pass to mdl
gem_stamp = "out/gem/root.stamp"       # Stamp file from gem processor (dependency)
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Mermaid Processor

Purpose

Converts Mermaid diagram files to PNG, SVG, or PDF using mmdc (mermaid-cli).

How It Works

Discovers .mmd files in the project and runs mmdc on each file, generating output in the configured formats. Each format produces a separate output file.

Source Files

• Input: **/*.mmd
• Output: out/mermaid/{format}/{relative_path}.{format}

Configuration

[processor.mermaid]
mmdc_bin = "mmdc"                      # The mmdc command to run
formats = ["png"]                      # Output formats (png, svg, pdf)
args = []                              # Additional arguments to pass to mmdc
output_dir = "out/mermaid"             # Output directory
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Mypy Processor

Purpose

Type-checks Python source files using mypy.

How It Works

Discovers .py files in the project (excluding common non-source directories), runs mypy on each file, and creates a stub file on success. A non-zero exit code from mypy fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single mypy invocation for better performance.

If a mypy.ini file exists in the project root, it is automatically added as an extra input so that configuration changes trigger rebuilds.

Source Files

• Input: **/*.py
• Output: out/mypy/{flat_name}.mypy

Configuration

[processor.mypy]
linter = "mypy"                             # The mypy command to run
args = []                                    # Additional arguments to pass to mypy
extra_inputs = []                            # Additional files that trigger rebuilds (e.g. ["pyproject.toml"])

Using mypy.ini

Mypy automatically reads configuration from a mypy.ini file in the project root. This file is detected automatically and added as an extra input, so changes to it will trigger rebuilds without manual configuration.

Npm Processor

Purpose

Installs Node.js dependencies from package.json files using npm.

How It Works

Discovers package.json files in the project, runs npm install in each directory, and creates a stamp file on success. Sibling .json, .js, and .ts files are tracked as inputs so changes trigger reinstallation.

Source Files

• Input: **/package.json (plus sibling .json, .js, .ts files)
• Output: out/npm/{flat_name}.stamp

Configuration

[processor.npm]
npm = "npm"                            # The npm command to run
command = "install"                    # The npm subcommand to execute
args = []                              # Additional arguments to pass to npm
extra_inputs = []                      # Additional files that trigger rebuilds when changed
cache_output_dir = true                # Cache the node_modules directory for fast restore after clean

Objdump Processor

Purpose

Disassembles ELF binaries using objdump.

How It Works

Discovers .elf files under out/cc_single_file/, runs objdump to produce disassembly output, and writes the result to the configured output directory.

Source Files

• Input: out/cc_single_file/**/*.elf
• Output: disassembly files in output directory

Configuration

[processor.objdump]
args = []
extra_inputs = []
output_dir = "out/objdump"

Pandoc Processor

Purpose

Converts documents between formats using pandoc.

How It Works

Discovers .md files in the project and runs pandoc on each file, converting from the configured source format to the configured output formats.

Source Files

• Input: **/*.md
• Output: out/pandoc/{format}/{relative_path}.{format}

Configuration

[processor.pandoc]
pandoc = "pandoc"                      # The pandoc command to run
from = "markdown"                      # Source format
formats = ["pdf"]                      # Output formats (pdf, docx, html, etc.)
args = []                              # Additional arguments to pass to pandoc
output_dir = "out/pandoc"              # Output directory
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Pdflatex Processor

Purpose

Compiles LaTeX documents to PDF using pdflatex.

How It Works

Discovers .tex files in the project and runs pdflatex on each file. Runs multiple compilation passes (configurable) to resolve cross-references and table of contents. Optionally uses qpdf to linearize the output PDF.

Source Files

• Input: **/*.tex
• Output: out/pdflatex/{relative_path}.pdf

Configuration

[processor.pdflatex]
pdflatex = "pdflatex"                  # The pdflatex command to run
runs = 2                               # Number of compilation passes
qpdf = true                           # Use qpdf to linearize output PDF
args = []                              # Additional arguments to pass to pdflatex
output_dir = "out/pdflatex"            # Output directory
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Pdfunite Processor

Purpose

Merges PDF files from subdirectories into single combined PDFs using pdfunite.

How It Works

Scans subdirectories of the configured source directory for files matching the configured extension. For each subdirectory, it locates the corresponding PDFs (generated by an upstream processor such as marp) and merges them into a single output PDF.

This processor is designed for course/module workflows where slide decks in subdirectories are combined into course bundles.

Source Files

• Input: PDFs from upstream processor (e.g., out/marp/pdf/{subdir}/*.pdf)
• Output: out/courses/{subdir}.pdf

Configuration

[processor.pdfunite]
pdfunite_bin = "pdfunite"              # The pdfunite command to run
source_dir = "marp/courses"           # Base directory containing course subdirectories
source_ext = ".md"                     # Extension of source files in subdirectories
source_output_dir = "out/marp/pdf"     # Where the upstream processor puts PDFs
args = []                              # Additional arguments to pass to pdfunite
output_dir = "out/courses"             # Output directory for merged PDFs
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Perlcritic Processor

Purpose

Analyzes Perl code using Perl::Critic.

How It Works

Discovers .pl and .pm files in the project (excluding common build tool directories), runs perlcritic on each file, and records success in the cache. A non-zero exit code from perlcritic fails the product.

This processor supports batch mode.

If a .perlcriticrc file exists, it is automatically added as an extra input so that configuration changes trigger rebuilds.

Source Files

• Input: **/*.pl, **/*.pm
• Output: none (checker)

Configuration

[processor.perlcritic]
args = []
extra_inputs = []

PHP Lint Processor

Purpose

Checks PHP syntax using php -l.

How It Works

Discovers .php files in the project (excluding common build tool directories), runs php -l on each file, and records success in the cache. A non-zero exit code fails the product.

This processor supports batch mode.

Source Files

• Input: **/*.php
• Output: none (checker)

Configuration

[processor.php_lint]
args = []
extra_inputs = []

Pip Processor

Purpose

Installs Python dependencies from requirements.txt files using pip.

How It Works

Discovers requirements.txt files in the project, runs pip install -r on each, and creates a stamp file on success. The stamp file tracks the install state so dependencies are only reinstalled when requirements.txt changes.

Source Files

• Input: **/requirements.txt
• Output: out/pip/{flat_name}.stamp

Configuration

[processor.pip]
pip = "pip"                            # The pip command to run
args = []                              # Additional arguments to pass to pip
extra_inputs = []                      # Additional files that trigger rebuilds when changed
cache_output_dir = true                # Cache the stamp directory for fast restore after clean

Pylint Processor

Purpose

Lints Python source files using pylint.

How It Works

Discovers .py files in the project (excluding common non-source directories), runs pylint on each file, and creates a stub file on success. A non-zero exit code from pylint fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single pylint invocation for better performance.

If a .pylintrc file exists in the project root, it is automatically added as an extra input so that configuration changes trigger rebuilds.

Source Files

• Input: **/*.py
• Output: out/pylint/{flat_name}.pylint

Configuration

[processor.pylint]
args = []                                  # Additional arguments to pass to pylint
extra_inputs = []                          # Additional files that trigger rebuilds (e.g. ["pyproject.toml"])

Pyrefly Processor

Purpose

Type-checks Python source files using pyrefly.

How It Works

Discovers .py files in the project (excluding common non-source directories), runs pyrefly check on each file, and records success in the cache. A non-zero exit code from pyrefly fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single pyrefly invocation for better performance.

Source Files

• Input: **/*.py
• Output: none (linter)

Configuration

[processor.pyrefly]
linter = "pyrefly"                          # The pyrefly command to run
args = []                                    # Additional arguments to pass to pyrefly
extra_inputs = []                            # Additional files that trigger rebuilds when changed

Ruff Processor

Purpose

Lints Python source files using ruff.

How It Works

Discovers .py files in the project (excluding common non-source directories), runs ruff check on each file, and creates a stub file on success. A non-zero exit code from ruff fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single ruff invocation for better performance.

Source Files

• Input: **/*.py
• Output: out/ruff/{flat_name}.ruff

Configuration

[processor.ruff]
linter = "ruff"                            # The ruff command to run
args = []                                  # Additional arguments to pass to ruff
extra_inputs = []                          # Additional files that trigger rebuilds (e.g. ["pyproject.toml"])

Rumdl Processor

Purpose

Lints Markdown files using rumdl.

How It Works

Discovers .md files in the project (excluding common non-source directories), runs rumdl check on each file, and creates a stub file on success. A non-zero exit code from rumdl fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single rumdl invocation for better performance.

Source Files

• Input: **/*.md
• Output: out/rumdl/{flat_name}.rumdl

Configuration

[processor.rumdl]
linter = "rumdl"                             # The rumdl command to run
args = []                                    # Additional arguments to pass to rumdl
extra_inputs = []                            # Additional files that trigger rebuilds when changed

Script Check Processor

Purpose

Runs a user-configured script or command as a linter on discovered files. This is a generic linter that lets you plug in any script without writing a custom processor.

How It Works

Discovers files matching the configured extensions in the configured scan directory, then runs the configured linter command on each file (or batch of files). A non-zero exit code from the script fails the product.

This processor is disabled by default — you must set enabled = true and provide a linter command in your rsconstruct.toml.

This processor supports batch mode, allowing multiple files to be checked in a single invocation for better performance.

Source Files

• Input: configured via extensions and scan_dir
• Output: none (linter)

Configuration

[processor.script_check]
enabled = true
linter = "python"
args = ["scripts/md_lint.py", "-q"]
extensions = [".md"]
scan_dir = "marp"

Shellcheck Processor

Purpose

Lints shell scripts using shellcheck.

How It Works

Discovers .sh and .bash files in the project (excluding common build tool directories), runs shellcheck on each file, and records success in the cache. A non-zero exit code from shellcheck fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single shellcheck invocation for better performance.

Source Files

• Input: **/*.sh, **/*.bash
• Output: none (linter)

Configuration

[processor.shellcheck]
linter = "shellcheck"                       # The shellcheck command to run
args = []                                    # Additional arguments to pass to shellcheck
extra_inputs = []                            # Additional files that trigger rebuilds when changed

Slidev Processor

Purpose

Builds Slidev presentations.

How It Works

Discovers .md files in the project (excluding common build tool directories), runs slidev build on each file, and records success in the cache. A non-zero exit code from slidev fails the product.

This processor supports batch mode.

Source Files

• Input: **/*.md
• Output: none (checker)

Configuration

[processor.slidev]
args = []
extra_inputs = []

Spellcheck Processor

Purpose

Checks documentation files for spelling errors using Hunspell-compatible dictionaries (via the zspell crate, pure Rust).

How It Works

Discovers files matching the configured extensions, extracts words from markdown content (stripping code blocks, inline code, URLs, and HTML tags), and checks each word against the system Hunspell dictionary and a custom words file (if it exists). Creates a stub file on success; fails with a list of misspelled words on error.

Dictionaries are read from /usr/share/hunspell/.

This processor supports batch mode when auto_add_words is enabled, collecting all misspelled words across files and writing them to the words file at the end.

Source Files

• Input: **/*{extensions} (default: **/*.md)
• Output: out/spellcheck/{flat_name}.spellcheck

Custom Words File

The processor loads custom words from the file specified by words_file (default: .spellcheck-words) if the file exists. Format: one word per line, # comments supported, blank lines ignored.

The words file is also auto-detected as an input via auto_inputs, so changes to it invalidate all spellcheck products. To disable words file detection, set auto_inputs = [].

Configuration

[processor.spellcheck]
extensions = [".md"]                  # File extensions to check (default: [".md"])
language = "en_US"                    # Hunspell dictionary language (default: "en_US")
words_file = ".spellcheck-words"      # Path to custom words file (default: ".spellcheck-words")
auto_add_words = false                # Auto-add misspelled words to words_file (default: false)
auto_inputs = [".spellcheck-words"]   # Auto-detected config files (default: [".spellcheck-words"])
extra_inputs = []                     # Additional files that trigger rebuilds when changed

Sphinx Processor

Purpose

Builds Sphinx documentation projects.

How It Works

Discovers conf.py files indicating Sphinx projects, collects sibling .rst, .py, and .md files as inputs, and runs sphinx-build to generate output. A non-zero exit code fails the product.

Source Files

• Input: **/conf.py (plus sibling .rst, .py, .md files)
• Output: none (mass_generator — produces output in _build directory)

Configuration

[processor.sphinx]
sphinx_build = "sphinx-build"          # The sphinx-build command to run
output_dir = "_build"                  # Output directory for generated docs
args = []                              # Additional arguments to pass to sphinx-build
extra_inputs = []                      # Additional files that trigger rebuilds when changed
cache_output_dir = true                # Cache the output directory for fast restore after clean

Standard Processor

Purpose

Checks JavaScript code style using standard.

How It Works

Discovers .js files in the project (excluding common build tool directories), runs standard on each file, and records success in the cache. A non-zero exit code from standard fails the product.

This processor supports batch mode.

Source Files

• Input: **/*.js
• Output: none (checker)

Configuration

[processor.standard]
args = []
extra_inputs = []

Stylelint Processor

Purpose

Lints CSS, SCSS, Sass, and Less files using stylelint.

How It Works

Discovers .css, .scss, .sass, and .less files in the project (excluding common build tool directories), runs stylelint on each file, and records success in the cache. A non-zero exit code from stylelint fails the product.

This processor supports batch mode.

If a stylelint config file exists (.stylelintrc* or stylelint.config.*), it is automatically added as an extra input so that configuration changes trigger rebuilds.

Source Files

• Input: **/*.css, **/*.scss, **/*.sass, **/*.less
• Output: none (checker)

Configuration

[processor.stylelint]
linter = "stylelint"
args = []
extra_inputs = []

Tags Processor

Purpose

Extracts YAML frontmatter tags from markdown files into a searchable database.

How It Works

Scans .md files for YAML frontmatter blocks (delimited by ---), parses tag metadata, and builds a redb database. The database enables querying files by tags via rsconstruct tags subcommands.

Optionally validates tags against a .tags allowlist file.

Tag Indexing

Two kinds of frontmatter fields are indexed:

• List fields — each item becomes a bare tag.

  tags:
    - docker
    - python
  

  Produces tags: docker, python.

• Scalar fields — indexed as key:value (colon separator).

  level: beginner
  difficulty: 3
  published: true
  url: https://example.com/path
  

  Produces tags: level:beginner, difficulty:3, published:true, url:https://example.com/path.

Both inline YAML lists (tags: [a, b, c]) and multi-line lists are supported.

The .tags Allowlist

When a .tags file exists in the project root, the build validates every indexed tag against it. Unknown tags cause a build error with typo suggestions (Levenshtein distance). Wildcard patterns are supported:

# .tags
docker
python
level:beginner
level:advanced
difficulty:*

The pattern difficulty:* matches any tag starting with difficulty:.

Source Files

• Input: **/*.md
• Output: out/tags/tags.db

Configuration

[processor.tags]
output = "out/tags/tags.db"            # Output database path
tags_file = ".tags"                    # Allowlist file for tag validation
tags_file_strict = false               # When true, missing .tags file is an error
extra_inputs = []                      # Additional files that trigger rebuilds when changed

Subcommands

All subcommands require a prior rsconstruct build to populate the database. All support --json for machine-readable output.

Querying

.tags File Management

Taplo Processor

Purpose

Checks TOML files using taplo.

How It Works

Discovers .toml files in the project (excluding common build tool directories), runs taplo check on each file, and records success in the cache. A non-zero exit code from taplo fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single taplo invocation for better performance.

Source Files

• Input: **/*.toml
• Output: none (checker)

Configuration

[processor.taplo]
linter = "taplo"                             # The taplo command to run
args = []                                    # Additional arguments to pass to taplo
extra_inputs = []                            # Additional files that trigger rebuilds when changed

Tera Processor

Purpose

Renders Tera template files into output files, with support for loading configuration variables from Python or Lua files.

How It Works

Files matching configured extensions in tera.templates/ are rendered and written to the project root with the extension stripped:

tera.templates/app.config.tera  →  app.config
tera.templates/sub/readme.txt.tera  →  sub/readme.txt

Templates use the Tera templating engine and can call load_python(path="...") or load_lua(path="...") to load variables from config files.

Loading Lua config

{% set config = load_lua(path="config/settings.lua") %}
[app]
name = "{{ config.project_name }}"
version = "{{ config.version }}"

Lua configs are executed via the embedded Lua 5.4 interpreter (no external dependency). All user-defined globals (strings, numbers, booleans, tables) are exported. Built-in Lua globals and functions are automatically filtered out. dofile() and require() work relative to the config file’s directory.

Loading Python config

{% set config = load_python(path="config/settings.py") %}
[app]
name = "{{ config.project_name }}"
version = "{{ config.version }}"

Source Files

• Input: tera.templates/**/*{extensions}
• Output: project root, mirroring the template path with the extension removed

Configuration

[processor.tera]
strict = true                              # Fail on undefined variables (default: true)
extensions = [".tera"]                     # File extensions to process (default: [".tera"])
trim_blocks = false                        # Remove newline after block tags (default: false)
extra_inputs = ["config/settings.py"]      # Additional files that trigger rebuilds when changed

Tidy Processor

Purpose

Validates HTML files using HTML Tidy.

How It Works

Discovers .html and .htm files in the project (excluding common build tool directories), runs tidy -errors on each file, and records success in the cache. A non-zero exit code from tidy fails the product.

This processor supports batch mode.

Source Files

• Input: **/*.html, **/*.htm
• Output: none (checker)

Configuration

[processor.tidy]
args = []
extra_inputs = []

XMLLint Processor

Purpose

Validates XML files using xmllint.

How It Works

Discovers .xml files in the project (excluding common build tool directories), runs xmllint --noout on each file, and records success in the cache. A non-zero exit code from xmllint fails the product.

This processor supports batch mode.

Source Files

• Input: **/*.xml
• Output: none (checker)

Configuration

[processor.xmllint]
args = []
extra_inputs = []

Yamllint Processor

Purpose

Lints YAML files using yamllint.

How It Works

Discovers .yml and .yaml files in the project (excluding common build tool directories), runs yamllint on each file, and records success in the cache. A non-zero exit code from yamllint fails the product.

This processor supports batch mode, allowing multiple files to be checked in a single yamllint invocation for better performance.

Source Files

• Input: **/*.yml, **/*.yaml
• Output: none (checker)

Configuration

[processor.yamllint]
linter = "yamllint"                          # The yamllint command to run
args = []                                    # Additional arguments to pass to yamllint
extra_inputs = []                            # Additional files that trigger rebuilds when changed

Yq Processor

Purpose

Validates YAML files using yq.

How It Works

Discovers .yml and .yaml files in the project (excluding common build tool directories), runs yq . on each file to validate syntax, and records success in the cache. A non-zero exit code from yq fails the product.

This processor supports batch mode.

Source Files

• Input: **/*.yml, **/*.yaml
• Output: none (checker)

Configuration

[processor.yq]
args = []
extra_inputs = []

Lua Plugins

RSConstruct supports custom processors written in Lua. Drop a .lua file in the plugins/ directory and add its name to [processor] enabled in rsconstruct.toml. The plugin participates in discovery, execution, caching, cleaning, tool listing, and auto-detection just like a built-in processor.

Quick Start

1. Create the plugin file:

plugins/eslint.lua

function description()
    return "Lint JavaScript/TypeScript with ESLint"
end

function required_tools()
    return {"eslint"}
end

function discover(project_root, config, files)
    local products = {}
    for _, file in ipairs(files) do
        local stub = rsconstruct.stub_path(project_root, file, "eslint")
        table.insert(products, {
            inputs = {file},
            outputs = {stub},
        })
    end
    return products
end

function execute(product)
    rsconstruct.run_command("eslint", {product.inputs[1]})
    rsconstruct.write_stub(product.outputs[1], "linted")
end

2. Enable it in rsconstruct.toml:

[processor]
enabled = ["eslint"]

[processor.eslint]
scan_dir = "src"
extensions = [".js", ".ts"]

3. Run it:

rsconstruct build            # builds including the plugin
rsconstruct processors list   # shows the plugin
rsconstruct processors files  # shows files discovered by the plugin

Lua API Contract

Each .lua file defines global functions. Three are required; the rest have sensible defaults.

Required Functions

description()

Returns a human-readable string describing what the processor does. Called once when the plugin is loaded.

function description()
    return "Lint JavaScript files with ESLint"
end

discover(project_root, config, files)

Called during product discovery. Receives:

• project_root (string) — absolute path to the project root
• config (table) — the [processor.NAME] TOML section as a Lua table
• files (table) — list of absolute file paths matching the scan configuration

Must return a table of products. Each product is a table with inputs and outputs keys, both containing tables of absolute file paths.

function discover(project_root, config, files)
    local products = {}
    for _, file in ipairs(files) do
        local stub = rsconstruct.stub_path(project_root, file, "myplugin")
        table.insert(products, {
            inputs = {file},
            outputs = {stub},
        })
    end
    return products
end

execute(product)

Called to build a single product. Receives a table with inputs and outputs keys (both tables of absolute path strings). Must create the output files on success or error on failure.

function execute(product)
    rsconstruct.run_command("mytool", {product.inputs[1]})
    rsconstruct.write_stub(product.outputs[1], "done")
end

Optional Functions

clean(product)

Called when running rsconstruct clean. Receives the same product table as execute(). Default behavior: removes all output files.

function clean(product)
    for _, output in ipairs(product.outputs) do
        rsconstruct.remove_file(output)
    end
end

auto_detect(files)

Called to determine whether this processor is relevant for the project (when auto_detect = true in config). Receives the list of matching files. Default: returns true if the files list is non-empty.

function auto_detect(files)
    return #files > 0
end

required_tools()

Returns a table of external tool names required by this processor. Used by rsconstruct tools list and rsconstruct tools check. Default: empty table.

function required_tools()
    return {"eslint", "node"}
end

hidden()

Returns true to hide this processor from default rsconstruct processors list output (still shown with --all). Default: false.

function hidden()
    return false
end

processor_type()

Returns the type of processor: "generator" or "checker". Generators create real output files (e.g., compilers, transpilers). Checkers validate input files; for checkers, you can choose whether to produce stub files or not. Default: "checker".

Option 1: Checker with stub files (for Lua plugins)

function processor_type()
    return "checker"
end

When using stub files, return outputs = {stub} from discover() and call rsconstruct.write_stub() in execute().

Option 2: Checker without stub files

function processor_type()
    return "checker"
end

Return outputs = {} from discover() and don’t write stubs in execute(). The cache database entry itself serves as the success record.

The rsconstruct Global Table

Lua plugins have access to an rsconstruct global table with helper functions.

Configuration

Plugins use the standard scan configuration fields. Any [processor.NAME] section in rsconstruct.toml is passed to the plugin’s discover() function as the config table.

Scan Configuration

These fields control which files are passed to discover():

Custom Configuration

Any additional keys in the [processor.NAME] section are passed through to the Lua config table:

[processor.eslint]
scan_dir = "src"
extensions = [".js", ".ts"]
max_warnings = 0          # custom key, accessible as config.max_warnings in Lua
fix = false               # custom key, accessible as config.fix in Lua

function execute(product)
    local args = {product.inputs[1]}
    if config.max_warnings then
        table.insert(args, "--max-warnings")
        table.insert(args, tostring(config.max_warnings))
    end
    rsconstruct.run_command("eslint", args)
    rsconstruct.write_stub(product.outputs[1], "linted")
end

Plugins Directory

The directory where RSConstruct looks for .lua files is configurable:

[plugins]
dir = "plugins"  # default

Plugin Name Resolution

The plugin name is derived from the .lua filename (without extension). This name is used for:

• The [processor.NAME] config section
• The enabled list in [processor]
• The out/NAME/ stub directory
• Display in rsconstruct processors list and build output

A plugin name must not conflict with a built-in processor name (tera, ruff, pylint, cc_single_file, cppcheck, shellcheck, spellcheck, make). RSConstruct will error if a conflict is detected.

Incremental Builds

Lua plugins participate in RSConstruct’s incremental build system automatically:

• Products are identified by their inputs, outputs, and a config hash
• If none of the declared inputs have changed since the last build, the product is skipped
• If the [processor.NAME] config section changes, all products are rebuilt
• Outputs are cached and can be restored from cache

For correct incrementality, make sure discover() declares all files that affect the output. If your tool reads additional configuration files, include them in the inputs list.

Examples

Linter Without Stub Files (Recommended)

A checker that validates files without producing stub files. Success is recorded in the cache database.

function description()
    return "Lint YAML files with yamllint"
end

function processor_type()
    return "checker"
end

function required_tools()
    return {"yamllint"}
end

function discover(project_root, config, files)
    local products = {}
    for _, file in ipairs(files) do
        table.insert(products, {
            inputs = {file},
            outputs = {},  -- No output files
        })
    end
    return products
end

function execute(product)
    rsconstruct.run_command("yamllint", {"-s", product.inputs[1]})
    -- No stub to write; cache entry = success
end

function clean(product)
    -- Nothing to clean
end

[processor]
enabled = ["yamllint"]

[processor.yamllint]
extensions = [".yml", ".yaml"]

Stub-Based Linter (Legacy)

A linter that creates stub files. Use this if you need the stub file for some reason.

function description()
    return "Lint YAML files with yamllint"
end

function processor_type()
    return "checker"
end

function required_tools()
    return {"yamllint"}
end

function discover(project_root, config, files)
    local products = {}
    for _, file in ipairs(files) do
        table.insert(products, {
            inputs = {file},
            outputs = {rsconstruct.stub_path(project_root, file, "yamllint")},
        })
    end
    return products
end

function execute(product)
    rsconstruct.run_command("yamllint", {"-s", product.inputs[1]})
    rsconstruct.write_stub(product.outputs[1], "linted")
end

[processor]
enabled = ["yamllint"]

[processor.yamllint]
extensions = [".yml", ".yaml"]

File Transformer (Generator)

A plugin that transforms input files into output files (not stubs). This is a “generator” processor.

function description()
    return "Compile Sass to CSS"
end

function processor_type()
    return "generator"
end

function required_tools()
    return {"sass"}
end

function discover(project_root, config, files)
    local products = {}
    for _, file in ipairs(files) do
        local out = file:gsub("%.scss$", ".css"):gsub("^" .. project_root .. "/src/", project_root .. "/out/sass/")
        table.insert(products, {
            inputs = {file},
            outputs = {out},
        })
    end
    return products
end

function execute(product)
    rsconstruct.run_command("sass", {product.inputs[1], product.outputs[1]})
end

[processor]
enabled = ["sass"]

[processor.sass]
scan_dir = "src"
extensions = [".scss"]

Advanced Usage

Parallel builds

RSConstruct can build independent products concurrently. Set the number of parallel jobs:

rsconstruct build -j4       # 4 parallel jobs
rsconstruct build -j0       # Auto-detect CPU cores

Or configure it in rsconstruct.toml:

[build]
parallel = 4   # 0 = auto-detect

The -j flag on the command line overrides the config file setting.

Watch mode

Watch source files and automatically rebuild on changes:

rsconstruct watch

This monitors all source files and triggers an incremental build whenever a file is modified.

Dependency graph

Visualize the build dependency graph in multiple formats:

rsconstruct graph                    # Default text format
rsconstruct graph --format dot       # Graphviz DOT format
rsconstruct graph --format mermaid   # Mermaid diagram format
rsconstruct graph --format json      # JSON format
rsconstruct graph --view             # Open in browser or viewer

The --view flag opens the graph using the configured viewer (set in rsconstruct.toml):

[graph]
viewer = "google-chrome"

Ignoring files

RSConstruct respects .gitignore files automatically. Any file ignored by git is also ignored by all processors. Nested .gitignore files and negation patterns are supported.

For project-specific exclusions that should not go in .gitignore, create a .rsconstructignore file in the project root with glob patterns (one per line):

/src/experiments/**
*.bak

The syntax is the same as .gitignore: # for comments, / prefix to anchor to the project root, / suffix for directories, and */** for globs.

Processor verbosity levels

Control the detail level of build output with -v N:

Dry run

Preview what would be built without executing anything:

rsconstruct build --dry-run

Keep going after errors

By default, RSConstruct stops on the first error. Use --keep-going to continue building other products:

rsconstruct build --keep-going

Build timings

Show per-product and total timing information:

rsconstruct build --timings

Shell completions

Generate shell completions for your shell:

rsconstruct complete bash    # Bash completions
rsconstruct complete zsh     # Zsh completions
rsconstruct complete fish    # Fish completions

Configure which shells to generate completions for:

[completions]
shells = ["bash"]

Extra inputs

By default, each processor only tracks its primary source files as inputs. If a product depends on additional files that aren’t automatically discovered (e.g., a config file read by a linter, a suppressions file used by a static analyzer, or a Python settings file loaded by a template), you can declare them with extra_inputs.

When any file listed in extra_inputs changes, all products from that processor are rebuilt.

[processor.template]
extra_inputs = ["config/settings.py", "config/database.py"]

[processor.ruff]
extra_inputs = ["pyproject.toml"]

[processor.pylint]
extra_inputs = ["pyproject.toml"]

[processor.cppcheck]
extra_inputs = [".cppcheck-suppressions"]

[processor.cc_single_file]
extra_inputs = ["Makefile.inc"]

[processor.spellcheck]
extra_inputs = ["custom-dictionary.txt"]

Paths are relative to the project root. Missing files cause a build error, so all listed files must exist.

The extra_inputs paths are included in the processor’s config hash, so adding or removing entries triggers a rebuild even if the files themselves haven’t changed. The file contents are also checksummed as part of the product’s input set, so any content change is detected by the incremental build system.

All processors support extra_inputs.

Graceful interrupt

Pressing Ctrl+C during a build stops execution promptly:

• Subprocess termination — All external processes (compilers, linters, etc.) are spawned with a poll loop that checks for interrupts every 50ms. When Ctrl+C is detected, the running child process is killed immediately rather than waiting for it to finish. This keeps response time under 50ms regardless of how long the subprocess would otherwise run.
• Progress preservation — Products that completed successfully before the interrupt are cached. The next build resumes from where it left off rather than starting over.
• Parallel builds — In parallel mode, all in-flight subprocesses are killed when Ctrl+C is detected. Each thread’s poll loop independently checks the global interrupt flag.

Testing

RSConstruct uses integration tests exclusively. All tests live in the tests/ directory and exercise the compiled rsconstruct binary as a black box — no unit tests are embedded in src/.

Running tests

cargo test              # Run all tests
cargo test rsconstructignore    # Run tests matching a name
cargo test -- --nocapture  # Show stdout/stderr from tests

Test directory layout

tests/
├── common/
│   └── mod.rs                  # Shared helpers (not a test binary)
├── build.rs                    # Build command tests
├── cache.rs                    # Cache operation tests
├── complete.rs                 # Shell completion tests
├── config.rs                   # Config show/show-default tests
├── dry_run.rs                  # Dry-run flag tests
├── graph.rs                    # Dependency graph tests
├── init.rs                     # Project initialization tests
├── processor_cmd.rs            # Processor list/auto/files tests
├── rsconstructignore.rs                # .rsconstructignore / .gitignore exclusion tests
├── status.rs                   # Status command tests
├── tools.rs                    # Tools list/check tests
├── watch.rs                    # File watcher tests
├── processors.rs               # Module root for processor tests
└── processors/
    ├── cc_single_file.rs       # C/C++ compilation tests
    ├── spellcheck.rs           # Spellcheck processor tests
    └── template.rs             # Template processor tests

Each top-level .rs file in tests/ is compiled as a separate test binary by Cargo. The processors.rs file acts as a module root that declares the processors/ subdirectory modules:

#![allow(unused)]
fn main() {
mod common;
mod processors {
    pub mod cc_single_file;
    pub mod spellcheck;
    pub mod template;
}
}

This is the standard Rust pattern for grouping related integration tests into subdirectories without creating a separate binary per file.

Shared helpers

tests/common/mod.rs provides utilities used across all test files:

All helpers use env!("CARGO_BIN_EXE_rsconstruct") to locate the compiled binary, ensuring tests run against the freshly built version.

Every test creates a fresh TempDir for isolation. The directory is automatically cleaned up when the test ends.

Test categories

Command tests

Tests in build.rs, clean, dry_run.rs, init.rs, status.rs, and watch.rs exercise CLI commands end-to-end:

#![allow(unused)]
fn main() {
#[test]
fn force_rebuild() {
    let temp_dir = setup_test_project();
    // ... set up files ...
    let output = run_rsconstruct_with_env(temp_dir.path(), &["build", "--force"], &[("NO_COLOR", "1")]);
    assert!(output.status.success());
    let stdout = String::from_utf8_lossy(&output.stdout);
    assert!(stdout.contains("[template] Processing:"));
}
}

These tests verify exit codes, stdout messages, and side effects (files created or removed).

Processor tests

Tests under processors/ verify individual processor behavior: file discovery, compilation, linting, incremental skip logic, and error handling. Each processor test module follows the same pattern:

1. Set up a temp project with appropriate source files
2. Run rsconstruct build
3. Assert outputs exist and contain expected content
4. Optionally modify a file and rebuild to test incrementality

Ignore tests

rsconstructignore.rs tests .rsconstructignore pattern matching: exact file patterns, glob patterns, leading / (anchored), trailing / (directory), comments, blank lines, and interaction with multiple processors.

Common assertion patterns

Exit code:

#![allow(unused)]
fn main() {
assert!(output.status.success());
}

Stdout content:

#![allow(unused)]
fn main() {
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(stdout.contains("Processing:"));
assert!(!stdout.contains("error"));
}

File existence:

#![allow(unused)]
fn main() {
assert!(path.join("out/cc_single_file/main.elf").exists());
}

Incremental builds:

#![allow(unused)]
fn main() {
// First build
run_rsconstruct(path, &["build"]);

// Second build should skip
let output = run_rsconstruct_with_env(path, &["build"], &[("NO_COLOR", "1")]);
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(stdout.contains("Skipping (unchanged):"));
}

Mtime-dependent rebuilds:

#![allow(unused)]
fn main() {
// Modify a file and wait for mtime to differ
std::thread::sleep(std::time::Duration::from_millis(100));
fs::write(path.join("src/header.h"), "// changed\n").unwrap();

let output = run_rsconstruct(path, &["build"]);
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(stdout.contains("Processing:"));
}

Writing a new test

1. Add a test function in the appropriate file (or create a new .rs file under tests/ for a new feature area)
2. Use setup_test_project() or setup_cc_project() to create an isolated environment
3. Write source files and configuration into the temp directory
4. Run rsconstruct with run_rsconstruct() or run_rsconstruct_with_env()
5. Assert on exit code, stdout/stderr content, and output file existence

If adding a new processor test module, declare it in tests/processors.rs:

#![allow(unused)]
fn main() {
mod processors {
    pub mod cc_single_file;
    pub mod spellcheck;
    pub mod template;
    pub mod my_new_processor;  // add here
}
}

Test coverage by area

Design Notes

This page has been merged into Architecture. See that page for RSConstruct’s internal design, subprocess execution, path handling, and caching behavior.

Cross-Processor Dependencies

This chapter discusses the problem of one processor’s output being consumed as input by another processor, and the design options for solving it.

The Problem

Consider a template that generates a Python file:

tera.templates/config.py.tera  →  (template processor)  →  config.py

Ideally, ruff should then lint the generated config.py. Or a template might generate a C++ source file that needs to be compiled by cc_single_file and linted by cppcheck. Chains can be arbitrarily deep:

template  →  generates foo.sh  →  shellcheck lints foo.sh
template  →  generates bar.c   →  cc_single_file compiles bar.c  →  cppcheck lints bar.c

Currently this does not work. Each processor discovers its inputs by querying the FileIndex, which is built once at startup by scanning the filesystem. Files that do not exist yet (because they will be produced by another processor) are invisible to downstream processors. No product is created for them, and no dependency edge is formed.

Why It Breaks

The build pipeline today is:

1. Walk the filesystem once to build FileIndex
2. Each processor runs discover() against that index
3. resolve_dependencies() matches product inputs to product outputs by path
4. Topological sort and execution

Step 3 already handles cross-processor edges correctly: if product A declares output foo.py and product B declares input foo.py, a dependency edge from A to B is created automatically. The problem is that step 2 never creates product B in the first place, because foo.py is not in the FileIndex.

How Other Build Systems Handle This

Bazel

Bazel uses BUILD files where rules explicitly declare their inputs and outputs. Dependencies are specified by label references, not by filesystem scanning. However, Bazel does use glob() to discover source files during its loading phase. The key insight is that during the analysis phase, both source files (from globs) and generated files (from rule declarations) are visible in a unified view. A rule’s declared outputs are known before any action executes.

Buck2

Buck2 takes a similar approach with a single unified dependency graph (no separate phases). Rules call declare_output() to create artifact references and return them via providers. Downstream rules receive these references through their declared dependencies. For cases where the dependency structure is not known statically, Buck2 provides dynamic_output — a rule can read an artifact at build time to discover additional dependencies.

Common Pattern

In both systems, the core principle is the same: a rule’s declared outputs are visible to the dependency resolver before execution begins. The dependency graph is fully resolved at analysis time.

Proposed Solutions

A. Multi-Pass Discovery (Iterative Build-Scan Loop)

Run discovery, build what is ready, re-scan the filesystem, discover again. Repeat until nothing new is found.

• Pro: Simple mental model, handles arbitrary chain depth
• Con: Slow (re-scans filesystem each pass), hard to detect infinite loops, execution is interleaved with discovery

B. Virtual Files from Declared Outputs (Two-Pass)

After the first discovery pass, collect all declared outputs from the graph and inject them as “virtual files” visible to processors. Run discovery a second time so downstream processors can find the generated files.

• Pro: No filesystem re-scan, single build execution phase, deterministic
• Con: Limited to chains of depth 1 (producer → consumer). A three-step chain (template → compile → lint) would require three passes, making the fixed two-pass design insufficient.

C. Fixed-Point Discovery Loop

Generalization of Approach B. Run discovery in a loop: after each pass, collect newly declared outputs and feed them back as known files for the next pass. Stop when a full pass adds no new products. Add a maximum iteration limit to catch cycles.

known_files = FileIndex (real files on disk)
loop {
    run discover() for all processors, with known_files visible
    new_outputs = outputs declared in this pass that were not in known_files
    if new_outputs is empty → break
    known_files = known_files + new_outputs
}
resolve_dependencies()
execute()

A chain of depth N requires N iterations. Most projects would converge in 1-2 iterations.

• Pro: Fully general, handles arbitrary chain depth, no filesystem re-scan, deterministic, path-based matching (no reliance on file extensions)
• Con: Processors must be able to discover products for files that do not exist on disk yet (they only know the path). This works for stub-based processors and compilers but might be an issue for processors that inspect file contents during discovery.

D. Explicit Cross-Processor Wiring in Config

Let users declare chains in rsconstruct.toml:

[[pipeline]]
from = "template"
to = "ruff"

rsconstruct then knows that template outputs matching ruff’s scan configuration should become ruff inputs.

• Pro: Explicit, no magic, user controls what gets chained
• Con: More configuration burden, loses the “convention over configuration” philosophy

E. Make out/ Visible to FileIndex

The simplest mechanical fix: stop excluding out/ from the FileIndex. Since .gitignore contains /out/, the ignore crate skips it. This could be overridden in the WalkBuilder configuration.

• Pro: Minimal code change, works on subsequent builds (files already exist from previous build)
• Con: Does not work on the first clean build (files do not exist yet). Processors would also see stale outputs from deleted processors, and stub files from other processors (though extension filtering would exclude most of these).

F. Two-Phase Processor Trait (Declarative Forward Tracing)

Split the ProductDiscovery trait so that each processor can declare what output paths it would produce for a given input path, without performing full discovery:

#![allow(unused)]
fn main() {
trait ProductDiscovery {
    /// Given an input path, return the output paths this processor would
    /// produce. Called even for files that don't exist on disk yet.
    fn would_produce(&self, input_path: &Path) -> Vec<PathBuf>;

    /// Full discovery (as today)
    fn discover(&self, graph: &mut BuildGraph, file_index: &FileIndex) -> Result<()>;
    // ...
}
}

The build system first runs discover() on all processors to get the initial set of products and their outputs. Then, for each declared output, it calls would_produce() on every other processor to trace the chain forward. This repeats transitively until no new outputs are produced. Finally, discover() runs once more with the complete set of known paths (real + virtual).

Unlike Approach C, this does not require a loop over full discovery passes. The chain is traced declaratively by asking each processor “if this file existed, what would you produce from it?” — a lightweight query that does not modify the graph.

• Pro: Single discovery pass plus lightweight forward tracing. No loop, no convergence check, no iteration limit. Each processor defines its output naming convention in one place. The full transitive closure of outputs is known before the main discovery runs.
• Con: Adds a method to the ProductDiscovery trait that every processor must implement. Some processors have complex output path logic (e.g., cc_single_file changes the extension and directory), so would_produce() must replicate that logic — meaning the output path computation exists in two places (in would_produce() and in discover()). Keeping these in sync is a maintenance risk.

G. Hybrid: Visible out/ + Fixed-Point Discovery

Combine Approach E (make out/ visible) with Approach C (fixed-point loop) or Approach F (forward tracing). On subsequent builds, existing files in out/ are already in the index. On clean builds, the fixed-point loop discovers them from declared outputs.

• Pro: Most robust — works for both clean and incremental builds
• Con: Combines complexity of two approaches, risk of discovering stale outputs

Recommendation

Approach C (fixed-point discovery loop) is the most principled solution. It is fully general, handles arbitrary chain depth, requires no configuration, and matches the core insight from Bazel and Buck2: declared outputs should be visible during dependency resolution before execution begins.

The main implementation requirement is extending the FileIndex (or creating a wrapper) to accept “virtual” entries for paths that are declared as outputs but do not yet exist on disk. Processors already declare their outputs during discover(), so the information needed to populate these virtual entries is already available.

Current Status

Cross-processor dependencies are not yet implemented. The dependency graph machinery (resolve_dependencies(), topological sort, executor ordering) is correct and would handle cross-processor edges properly once downstream products are discovered. The gap is purely in the discovery phase.

Architecture

This page describes RSConstruct’s internal design for contributors and those interested in how the tool works.

Core concepts

Processors

Processors implement the ProductDiscovery trait. Each processor:

1. Auto-detects whether it is relevant for the current project
2. Scans the project for source files matching its conventions
3. Creates products describing what to build
4. Executes the build for each product

Run rsconstruct processors list to see all available processors and their auto-detection results.

Auto-detection

Every processor implements auto_detect(), which returns true if the processor appears relevant for the current project based on filesystem heuristics. This allows RSConstruct to guess which processors a project needs without requiring manual configuration.

The ProductDiscovery trait requires four methods:

Both auto_detect and discover receive a &FileIndex — a pre-built index of all non-ignored files in the project (see File indexing below).

Detection heuristics per processor:

Run rsconstruct processors list to see the auto-detection results for the current project.

Products

A product represents a single build unit with:

• Inputs — source files that the product depends on
• Outputs — files that the product generates
• Output directory (optional) — for mass generators, the directory whose entire contents are cached and restored as a unit

BuildGraph

The BuildGraph manages dependencies between products. It performs a topological sort to determine the correct build order, ensuring that dependencies are built before the products that depend on them.

Executor

The executor runs products in dependency order. It supports:

• Sequential execution (default)
• Parallel execution of independent products (with -j flag)
• Dry-run mode (show what would be built)
• Keep-going mode (continue after errors)

Interrupt handling

All external subprocess execution goes through run_command() in src/processors/mod.rs. Instead of calling Command::output() (which blocks until the process finishes), run_command() uses Command::spawn() followed by a poll loop:

1. Spawn the child process with piped stdout/stderr
2. Every 50ms, call try_wait() to check if the process has exited
3. Between polls, check the global INTERRUPTED flag (set by the Ctrl+C handler)
4. If interrupted, kill the child process immediately and return an error

This ensures that pressing Ctrl+C terminates running subprocesses within 50ms, even for long-running compilations or linter invocations.

The global INTERRUPTED flag is an AtomicBool set once by the ctrlc handler in main.rs and checked by all threads.

File indexing

RSConstruct walks the project tree once at startup and builds a FileIndex — a sorted list of all non-ignored files. The walk is performed by the ignore crate (ignore::WalkBuilder), which natively handles:

• .gitignore — standard git ignore rules, including nested .gitignore files and negation patterns
• .rsconstructignore — project-specific ignore patterns using the same glob syntax as .gitignore

Processors never walk the filesystem themselves. Instead, auto_detect and discover receive a &FileIndex and query it with their scan configuration (extensions, exclude directories, exclude files). This replaces the previous design where each processor performed its own recursive walk.

Build pipeline

1. File indexing — The project tree is walked once to build the FileIndex
2. Discovery — Each enabled processor queries the file index and creates products
3. Graph construction — Products are added to the BuildGraph with their dependencies
4. Topological sort — The graph is sorted to determine build order
5. Cache check — Each product’s inputs are hashed (SHA-256) and compared against the cache
6. Execution — Stale products are rebuilt; up-to-date products are skipped or restored from cache
7. Cache update — Successfully built products have their outputs stored in the cache

Determinism

Build order is deterministic:

• File discovery is sorted
• Processor iteration order is sorted
• Topological sort produces a stable ordering

This ensures that the same project always builds in the same order, regardless of filesystem ordering.

Config-aware caching

Processor configuration (compiler flags, linter arguments, etc.) is hashed into cache keys. This means changing a config value like cflags will trigger rebuilds of affected products, even if the source files haven’t changed.

Cache keys

Each product has a unique cache key used to store and look up its cached state. The cache key is computed from:

{processor}:{config_hash}:{inputs}>{outputs}

For example, pandoc producing three formats from the same source file generates three products with distinct cache keys:

pandoc:a1b2c3:syllabi/intro.md>out/pandoc/intro.pdf
pandoc:a1b2c3:syllabi/intro.md>out/pandoc/intro.html
pandoc:a1b2c3:syllabi/intro.md>out/pandoc/intro.docx

For checkers (which have no output files), the key omits the output part:

ruff:d4e5f6:src/main.py

Why outputs are included in the key

Including outputs in the cache key is critical for multi-format processors. Without it, all three pandoc products above would share the key pandoc:a1b2c3:syllabi/intro.md, and each execution would overwrite the previous format’s cache entry. This caused a bug where:

1. First build: PDF, HTML, and DOCX all built correctly, but only the last format’s cache entry survived
2. Source file modified
3. Second build: only the last format detected the change and rebuilt; the other two skipped because the cache entry (from the last format) happened to match

The fix (including outputs in the key) ensures each format gets its own independent cache entry.

Note: changing the cache key format invalidates all existing caches. The first build after upgrading will be a full rebuild.

Cache storage

The cache lives in .rsconstruct/ and consists of:

• db.redb — redb database storing the object store index (maps product hashes to cached outputs)
• objects/ — stored build artifacts (addressed by content hash)
• deps.redb — redb database storing source file dependencies (see Dependency Caching)

Cache restoration can use either hardlinks (fast, same filesystem) or copies (works across filesystems), configured via restore_method.

Caching and clean behavior

The cache (.rsconstruct/) stores build state to enable fast incremental builds:

• Generators: Cache stores copies of output files. After rsconstruct clean, outputs are deleted but cache remains. Next rsconstruct build restores outputs from cache (fast hardlink/copy) instead of regenerating.

• Checkers: No output files to cache. The cache entry itself serves as a “success marker”. After rsconstruct clean (nothing to delete), next rsconstruct build sees the cache entry is valid and skips the check entirely (instant).

• Mass generators: When cache_output_dir is enabled (default), the entire output directory is walked after execution. Each file is stored as a content-addressed object in .rsconstruct/objects/, and a manifest records the relative path, checksum, and Unix permissions of every file. After rsconstruct clean (which removes the output directory), rsconstruct build recreates the directory from cached objects with permissions restored. This makes rsconstruct clean && rsconstruct build fast for doc builders like sphinx and mdbook.

This ensures rsconstruct clean && rsconstruct build is fast for all types — generators restore from cache, checkers skip entirely, mass generators restore their output directories.

Subprocess execution

RSConstruct uses two internal functions to run external commands:

• run_command() — by default captures stdout/stderr via OS pipes and only prints output on failure (quiet mode). Use --show-output flag to show all tool output. Use for compilers, linters, and any command where errors should be shown.

• run_command_capture() — always captures stdout/stderr via pipes. Use only when you need to parse the output (dependency analysis, version checks, Python config loading). Returns the output for processing.

Parallel safety

When running with -j, each thread spawns its own subprocess. Each subprocess gets its own OS-level pipes for stdout/stderr, so there is no interleaving of output between concurrent tools. On failure, the captured output for that specific tool is printed atomically. This design requires no shared buffers or cross-thread output coordination.

Path handling

All paths are relative to project root. RSConstruct assumes it is run from the project root directory (where rsconstruct.toml lives).

Internal paths (always relative)

• Product.inputs and Product.outputs — stored as relative paths
• FileIndex — returns relative paths from scan() and query()
• Cache keys (Product.cache_key()) — use relative paths, enabling cache sharing across different checkout locations
• Cache entries (CacheEntry.outputs[].path) — stored as relative paths

Processor execution

• Processors pass relative paths directly to external tools
• Processors set cmd.current_dir(project_root) to ensure tools resolve paths correctly
• fs::read(), fs::write(), etc. work directly with relative paths since cwd is project root

Exception: Processors requiring absolute paths

If a processor absolutely must use absolute paths (e.g., for a tool that doesn’t respect current directory), it should:

1. Store the project_root in the processor struct
2. Join paths with project_root only at execution time
3. Never store absolute paths in Product.inputs or Product.outputs

Why relative paths?

• Cache portability — cache keys don’t include machine-specific absolute paths
• Remote cache sharing — same project checked out to different paths can share cache
• Simpler code — no need to strip prefixes for display or storage

Environment Variables

The problem

Build tools that inherit the user’s environment variables produce non-deterministic builds. Consider a C compiler invoked by a build tool:

• If the user has CFLAGS=-O2 in their shell, the build produces optimized output.
• If they unset it, the build produces debug output.
• Two developers on the same project get different results from the same source files.

This breaks caching (the cache key doesn’t account for env vars), breaks reproducibility (builds differ across machines), and makes debugging harder (a build failure may depend on an env var the developer forgot they set).

Common examples of environment variables that silently affect build output:

RSConstruct’s approach

RSConstruct does not use environment variables from the user’s environment to control build behavior. All configuration comes from explicit, versioned sources:

1. rsconstruct.toml — all processor configuration (compiler flags, linter args, scan dirs, etc.)
2. Source file directives — per-file flags embedded in comments (e.g., // EXTRA_COMPILE_FLAGS_BEFORE=-pthread)
3. Tool lock file — .tools.versions locks tool versions so changes are detected

This means:

• The same source tree always produces the same build, regardless of the user’s shell environment.
• Cache keys are computed from file contents and config values, not ambient env vars.
• Remote cache sharing works because two machines with different environments still produce identical cache keys for identical inputs.

Rules for processor authors

When implementing a processor (built-in or Lua plugin):

1. Never read std::env::var() to determine build behavior. If a value is configurable, add it to the processor’s config struct in rsconstruct.toml.

2. Never call cmd.env() to pass environment variables to external tools, unless the variable is derived from explicit config (not from std::env). The user’s environment is inherited by default — the goal is to avoid adding env-based configuration on top.

3. Tool paths come from PATH — RSConstruct does inherit the user’s PATH to find tools like gcc, ruff, etc. This is acceptable because the tool lock file (.tools.versions) detects when tool versions change and triggers rebuilds. Use rsconstruct tools lock to pin versions.

4. Config values, not env vars — if a tool needs a flag that varies per project, put it in rsconstruct.toml under the processor’s config section. Config values are hashed into cache keys automatically.

What RSConstruct does inherit

RSConstruct inherits the full parent environment for subprocess execution. This is unavoidable — tools need PATH to be found, HOME to read their own config files, etc. The key design decision is that RSConstruct itself never reads env vars to make build decisions, and processors never add env vars derived from the user’s environment.

The one exception is NO_COLOR — RSConstruct respects this standard env var to disable colored output, which is a display concern and does not affect build output.

Processor Contract

Rules that all processors must follow.

Fail hard, never degrade gracefully

When something fails, it must fail the entire build. Do not try-and-fallback, do not silently substitute defaults for missing resources, do not swallow errors. If a processor is configured to use a file and that file does not exist, that is an error. The user must fix their configuration or their project, not the code.

Optional features must be opt-in via explicit configuration (default off). When the user enables a feature, all resources it requires must exist.

No work without source files

An enabled processor must not fail the build if no source files match its file patterns. Zero matching files means zero products discovered; the processor simply does nothing. This is not an error — it is the normal state for a freshly initialized project.

Single responsibility

Each processor handles one type of transformation or check. A processor discovers its own products and knows how to execute, clean, and report on them.

Deterministic discovery

discover() must return the same products given the same filesystem state. File discovery, processor iteration, and topological sort must all produce sorted, deterministic output so builds are reproducible.

Incremental correctness

Products must declare all their inputs. If any declared input changes, the product is rebuilt. If no inputs change, the cached result is reused. Processors must not rely on undeclared side inputs for correctness (support files read at execution time but excluded from the input list are acceptable only when changes to those files can never cause a previously-passing product to fail).

Execution isolation

A processor’s execute() must only write to the declared output paths (or, for mass generators, to the expected output directory). It must not modify source files, other products’ outputs, or global state.

Output directory caching (mass generators)

Mass generators that set output_dir on their products get automatic directory-level caching. After successful execution, the executor walks the output directory, stores every file as a content-addressed object, and records a manifest with paths, checksums, and Unix permissions. On restore, the entire directory is recreated from cache.

The cache_output_dir config option (default true) controls this. When disabled, mass generators fall back to stamp-file or empty-output caching (no directory restore on rsconstruct clean && rsconstruct build).

Mass generators that use output_dir caching must implement clean() to remove the output directory so it can be restored from cache.

Error reporting

On failure, execute() returns an Err with a clear message including the relevant file path and the nature of the problem. The executor decides whether to abort or continue based on --keep-going.

Coding Standards

Rules that apply to the RSConstruct codebase and its documentation.

Fail hard, never degrade gracefully

When something fails, it must fail the entire build. Do not try-and-fallback, do not silently substitute defaults for missing resources, do not swallow errors. If a processor is configured to use a file and that file does not exist, that is an error. The user must fix their configuration or their project, not the code.

Optional features must be opt-in via explicit configuration (default off). When the user enables a feature, all resources it requires must exist.

Processor naming conventions

Every processor has a single identity string (e.g. ruff, clang_tidy, mdbook). All artifacts derived from a processor must use that same string consistently:

When adding a new processor, use the identity string everywhere. Do not abbreviate, rename, or add suffixes (Gen, Bin, etc.) to any of the derived names.

Test naming for processors

Test functions for a processor must be prefixed with the processor name. For example, tests for the cc_single_file processor must be named cc_single_file_compile, cc_single_file_incremental_skip, etc.

No indented output

All println! output must start at column 0. Never prefix output with spaces or tabs for visual indentation unless when printing some data with structure.

Suppress tool output on success

External tool output (compilers, linters, etc.) must be captured and only shown when a command fails. On success, only rsconstruct’s own status messages appear. Users who want to always see tool output can use --show-output. This keeps build output clean while still showing errors when something goes wrong.

Never hard-code counts of dynamic sets

Documentation and code must never state the number of processors, commands, or any other set that changes as the project evolves. Use phrasing like “all processors” instead of “all seven processors”. Enumerating the members of a set is acceptable; stating the cardinality is not.

Use well-established crates

Prefer well-established crates over hand-rolled implementations for common functionality (date/time, parsing, hashing, etc.). The Rust ecosystem has mature, well-tested libraries for most tasks. Writing custom implementations introduces unnecessary bugs and maintenance burden. If a crate exists for it, use it.

No trailing newlines in output

Output strings passed to println!, pb.println(), or similar macros must not contain trailing newlines. These macros already append a newline. Adding \n inside the string produces unwanted blank lines in the output.

Include processor name in error messages

Error messages from processor execution must identify the processor so the user can immediately tell which processor failed. The executor’s record_failure() method automatically wraps every error with [processor_name] before printing or storing it, so processors do not need to manually prefix their bail! messages. Just write the error naturally (e.g. bail!("Misspelled words in {}", path)) and the executor will produce [aspell] Misspelled words in README.md.

Reject unknown config fields

All config structs that don’t intentionally capture extra fields must use #[serde(deny_unknown_fields)]. This ensures that typos or unsupported options in rsconstruct.toml produce a clear error instead of being silently ignored.

Structs that use #[serde(flatten)] to embed other structs (like ScanConfig) cannot use deny_unknown_fields due to serde limitations. These structs must instead implement the KnownFields trait, returning a static slice of all valid field names (own fields + flattened fields). The validate_processor_fields() function in Config::load() checks all [processor.X] keys against these lists before deserialization.

Structs that intentionally capture unknown fields (like ProcessorConfig.extra for Lua plugins) should use neither deny_unknown_fields nor KnownFields.

No “latest” git tag

Never create a git tag named latest. Use only semver tags (e.g. v0.3.0). A latest tag causes confusion with container registries and package managers that use the word “latest” as a moving pointer, and it conflicts with GitHub’s release conventions.

Crates.io Publishing

Notes on publishing rsconstruct to crates.io.

Version Limits

There is no limit on how many versions can be published to crates.io. You can publish as many releases as needed without worrying about quota or cleanup.

Pruning Old Releases

Crates.io does not support deleting published versions. Once a version is uploaded, it exists permanently.

The only removal mechanism is yanking (cargo yank --version 0.1.0), which:

• Prevents new projects from adding a dependency on the yanked version
• Does not break existing projects that already depend on it (they continue to download it via their lockfile)
• Does not delete the crate data from the registry

Yanking should only be used for versions with security vulnerabilities or serious bugs, not for general housekeeping.

Publishing a New Version

1. Update the version in Cargo.toml
2. Run cargo publish --dry-run to verify
3. Run cargo publish to upload

Missing Processors

Tools found in Makefiles across ../*/ sibling projects that rsconstruct does not yet have processors for. Organized by category, with priority based on breadth of usage.

High Priority — Linters and Validators

eslint

• What it does: JavaScript/TypeScript linter (industry standard).
• Projects: demos-lang-js
• Invocation: eslint $(ALL_JS) or node_modules/.bin/eslint $<
• Processor type: Checker

jshint

• What it does: JavaScript linter — detects errors and potential problems.
• Projects: demos-lang-js, gcp-gemini-cli, gcp-machines, gcp-miflaga, gcp-nikuda, gcp-randomizer, schemas, veltzer.github.io
• Invocation: node_modules/.bin/jshint $<
• Processor type: Checker

tidy (HTML Tidy)

• What it does: HTML/XHTML validator and formatter.
• Projects: demos-lang-js, gcp-gemini-cli, gcp-machines, gcp-miflaga, gcp-nikuda, gcp-randomizer, openbook, riddles-book
• Invocation: tidy -errors -quiet -config .tidy.config $<
• Processor type: Checker

check-jsonschema

• What it does: Validates YAML/JSON files against JSON Schema (distinct from rsconstruct’s json_schema which validates JSON against schemas found via $schema key).
• Projects: data, schemas, veltzer.github.io
• Invocation: check-jsonschema --schemafile $(yq -r '.["$schema"]' $<) $<
• Processor type: Checker

cpplint

• What it does: C++ linter enforcing Google C++ style guide.
• Projects: demos-os-linux
• Invocation: cpplint $<
• Processor type: Checker

checkpatch.pl

• What it does: Linux kernel coding style checker.
• Projects: kcpp
• Invocation: $(KDIR)/scripts/checkpatch.pl --file $(C_SOURCES) --no-tree
• Processor type: Checker

standard (StandardJS)

• What it does: JavaScript style guide, linter, and formatter — zero config.
• Projects: demos-lang-js
• Invocation: node_modules/.bin/standard $<
• Processor type: Checker

jslint

• What it does: JavaScript code quality linter (Douglas Crockford).
• Projects: demos-lang-js
• Invocation: node_modules/.bin/jslint $<
• Processor type: Checker

jsl (JavaScript Lint)

• What it does: JavaScript lint tool.
• Projects: keynote, myworld-php
• Invocation: jsl --conf=support/jsl.conf --quiet --nologo --nosummary --nofilelisting $(SOURCES_JS)
• Processor type: Checker

gjslint (Google Closure Linter)

• What it does: JavaScript style checker following Google JS style guide.
• Projects: keynote, myworld-php
• Invocation: $(TOOL_GJSLINT) --flagfile support/gjslint.cfg $(JS_SRC)
• Processor type: Checker

checkstyle

• What it does: Java source code style checker.
• Projects: demos-lang-java, keynote
• Invocation: java -cp $(scripts/cp.py) $(MAINCLASS_CHECKSTYLE) -c support/checkstyle_config.xml $(find . -name "*.java")
• Processor type: Checker

pyre

• What it does: Python type checker from Facebook/Meta.
• Projects: archive.apiiro.TrainingDataLaboratory, archive.work-amdocs-py
• Invocation: pyre check
• Processor type: Checker

High Priority — Formatters

black

• What it does: Opinionated Python code formatter.
• Projects: archive.apiiro.TrainingDataLaboratory, archive.work-amdocs-py
• Invocation: black --target-version py36 $(ALL_PACKAGES)
• Processor type: Checker (using --check mode) or Formatter

uncrustify

• What it does: C/C++/Java source code formatter.
• Projects: demos-os-linux, xmeltdown
• Invocation: uncrustify -c support/uncrustify.cfg --no-backup -l C $(ALL_US_C)
• Processor type: Formatter

astyle (Artistic Style)

• What it does: C/C++/Java source code indenter and formatter.
• Projects: demos-os-linux
• Invocation: astyle --verbose --suffix=none --formatted --preserve-date --options=support/astyle.cfg $(ALL_US)
• Processor type: Formatter

indent (GNU Indent)

• What it does: C source code formatter (GNU style).
• Projects: demos-os-linux
• Invocation: indent $(ALL_US)
• Processor type: Formatter

High Priority — Testing

pytest

• What it does: Python test framework.
• Projects: 50+ py* projects (pyanyzip, pyapikey, pyapt, pyawskit, pyblueprint, pybookmarks, pyclassifiers, pycmdtools, pyconch, pycontacts, pycookie, pydatacheck, pydbmtools, pydmt, pydockerutils, pyeventroute, pyeventsummary, pyfakeuse, pyflexebs, pyfoldercheck, pygcal, pygitpub, pygooglecloud, pygooglehelper, pygpeople, pylogconf, pymakehelper, pymount, pymultienv, pymultigit, pymyenv, pynetflix, pyocutil, pypathutil, pypipegzip, pypitools, pypluggy, pypowerline, pypptkit, pyrelist, pyscrapers, pysigfd, pyslider, pysvgview, pytagimg, pytags, pytconf, pytimer, pytsv, pytubekit, pyunique, pyvardump, pyweblight, and archive.*)
• Invocation: pytest tests or python -m pytest tests
• Processor type: Checker (mass, per-directory)

High Priority — YAML/JSON Processing

yq

• What it does: YAML/JSON processor (like jq but for YAML).
• Projects: data, demos-lang-yaml, schemas, veltzer.github.io
• Invocation: yq < $< > $@ (format/validate) or yq -r '.key' $< (extract)
• Processor type: Checker or Generator