If your build pipeline cannot run without you opening a terminal and typing make tangle, you do not have a literate program. You have a diary with a compiler attached.

The whole point of literate programming is that prose and code share a single source of truth. The Markdown file is the artifact. Everything else, the executable source, the rendered documentation, the test files, is derived. Derived artifacts belong in CI, not in your working memory.

The question is not whether CI can weave and tangle. Of course it can. Any machine that runs Python and pandoc can do it. The question is whether your repository structure lets CI own the derived files, or whether you are still pretending that .py files are source code when they are actually build output.

What weave and tangle actually do

Donald Knuth called the two operations weave and tangle, and the names are confusing enough that people avoid the topic entirely.

tangle takes the narrative document, the .md or .w file, and extracts the code blocks in the order the compiler expects. It produces the runnable source files. weave does the opposite. It takes the same document and produces the human-readable documentation, with pretty-printed code, cross-references, and a table of contents.

Both are deterministic transformations. They take one input and produce consistent outputs. That is the definition of something CI should handle.

Why most teams get the pipeline backwards

The typical literate programming workflow looks like this. You write algorithm.md. You run a local script to extract algorithm.py. You run tests against algorithm.py. You commit both algorithm.md and algorithm.py to Git. Then you open a pull request.

This is already broken. You have two copies of the same logic in version control. If a reviewer suggests a change to algorithm.py, you have to remember to backport it to algorithm.md. If you edit algorithm.md and forget to re-tangle, the committed .py file is stale. The single source of truth is a fiction.

The correct workflow is simpler. You commit only the .md file. CI runs tangle to produce the .py file, then runs the test suite against it. If the tests pass, CI optionally runs weave to publish documentation. The .py file never touches version control. It is build output, like a .o file or a Docker image.

A working CI pipeline for Markdown-to-Python

Here is a complete GitHub Actions workflow that treats the Markdown file as the sole source of truth. Save it as .github/workflows/literate.yml:

name: Tangle and Test

on:
  push:
    paths:
      - "**/*.md"
  pull_request:
    paths:
      - "**/*.md"

jobs:
  tangle:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Tangle code from Markdown
        run: python scripts/tangle.py src/*.md --out build/

      - name: Run tests on extracted code
        run: python -m pytest build/ -v

      - name: Upload generated source as artifact
        uses: actions/upload-artifact@v4
        with:
          name: generated-source
          path: build/

The paths filter is important. This workflow only runs when a Markdown file changes, because Markdown is the only input that matters.

The tangle.py script is the same thirty-line extractor you would run locally. Here is a version that handles multiple files and an output directory:

import argparse
import re
import sys
from pathlib import Path


def tangle(md_path: Path, out_dir: Path) -> None:
    content = md_path.read_text()
    files: dict[str, list[str]] = {}

    for block in re.findall(r"```python\n(.*?)```", content, re.DOTALL):
        lines = block.strip().split("\n")

        if lines and lines[0].startswith("# "):
            filename = lines[0][2:].strip()
            code = lines[1:]
        else:
            filename = "module.py"
            code = lines

        files.setdefault(filename, []).extend(code)

    out_dir.mkdir(parents=True, exist_ok=True)
    for filename, code_lines in files.items():
        (out_dir / filename).write_text("\n".join(code_lines) + "\n")


def main() -> None:
    parser = argparse.ArgumentParser(description="Extract code from Markdown files.")
    parser.add_argument("inputs", nargs="+", type=Path, help="Markdown files to process")
    parser.add_argument("--out", type=Path, required=True, help="Output directory")
    args = parser.parse_args()

    for md in args.inputs:
        tangle(md, args.out)


if __name__ == "__main__":
    main()

Run it locally to verify:

$ python scripts/tangle.py src/calculator.md --out build/
$ python -m pytest build/ -v

The CI job does exactly the same thing. The only difference is that CI runs it in a clean environment, so a stale .py file hidden in your working directory cannot fool the test suite.

Weaving documentation in the same pipeline

tangle produces code. weave produces docs. Both can run in CI.

For weave, you need a tool that turns Markdown into a presentable format. Pandoc is the boring, reliable choice. If you want cross-references, syntax highlighting, and a table of contents, a small template gets you there:

      - name: Weave documentation
        run: |
          mkdir -p docs/output
          pandoc src/*.md \
            --from markdown \
            --to html5 \
            --standalone \
            --toc \
            --highlight-style=tango \
            --output docs/output/index.html

      - name: Publish to GitHub Pages
        uses: actions/upload-pages-artifact@v3
        with:
          path: docs/output/

The rendered HTML is also a derived artifact. You can publish it to GitHub Pages, Netlify, or an S3 bucket on every merge to main. The Markdown file stays canonical. The HTML is what readers see.

The commit-versus-generate trade-off

Some teams push back on this. They want the .py files in Git because it makes git grep work and because GitHub’s code review UI understands Python syntax better than Markdown.

These are real problems, but they are tooling problems, not architecture problems. Checking in generated code because your code review tool is bad is like checking in node_modules because your package manager is slow. It works, but it accrues debt.

If you absolutely need the .py files in the repository, generate them in CI and commit them back with a bot account. Here is the pattern:

  1. A developer pushes a change to algorithm.md.
  2. CI runs tangle.py and produces algorithm.py.
  3. If the extracted code differs from what is in main, CI opens a pull request with the updated .py file.
  4. A human reviews the generated diff and merges it.

This keeps the generated files in version control without letting them drift. The bot becomes the single source of truth for generated output, and the bot only changes what the Markdown file demands.

When CI automation is worth it, and when it is not

Automated weave and tangle shine when the prose is long, the code is complex, and multiple people edit the document. Research code, library documentation, and algorithm implementations all benefit from a canonical Markdown file that CI validates on every push.

It is not worth it for small scripts or boilerplate-heavy application code. If your Markdown file is two paragraphs and a twenty-line function, the CI overhead, runner minutes, and mental model cost more than the synchronization problem you are solving. Just write a good docstring.

The other case where this fails is when your language tooling assumes .py files are primary. Debuggers, profilers, and coverage tools usually want to point to the generated source, not the Markdown. You can work around this by generating files locally before debugging, but that is friction. If your team lives in a debugger, literate programming might be the wrong fit regardless of how good your CI pipeline is.

FAQ

Should I commit the generated .py files or .gitignore them?

Gitignore them. If they are generated deterministically from Markdown, they are build artifacts. The one exception is if an external system, like a package registry, cannot run your tangle step. In that case, use the bot-commit pattern described above.

What if my literate document mixes multiple languages?

The tangle.py script is language-agnostic. Use # filename.rs for Rust blocks, # filename.go for Go blocks, and so on. CI runs the same script for every language. You just need a test step per language in the workflow.

Does this work with named code chunks like Knuth’s WEB?

Modern tools like Entangled support named chunks and dependency tracking. If you need that level of expressiveness, use Entangled in CI instead of a custom script. The pipeline structure is the same. Only the extraction tool changes.

How do I handle imports that span multiple Markdown files?

Each Markdown file tangles into its own output directory. If math.md defines add and geometry.md imports it, either tangle both files into the same build directory or structure your code as a package with relative imports. The CI job can tangle all files before running tests, so cross-file dependencies resolve naturally.

Make your Markdown file the only file that matters

Set up the pipeline once. Commit the .md file. Let CI handle the rest. If a test fails, the failure points to the Markdown line that produced the code, not to a generated file that should not exist in version control anyway.

The goal of literate programming is a single source of truth. A source of truth that requires a human to remember a manual step is not a source of truth. It is a suggestion.