Documentation

This chapter describes how the documents have been made and how to maintain them.

Documents to be maintained¶

We have the following documents, including this book, to maintain.

For the first two reference books, we use Sphinx directly to generate function documents from their docstrings. For others, Jupyter Book v2 (MyST) is used.

How to use Jupyter Book v2 (MyST)¶

Tool Package Installation¶

To use Jupyter Book v2 (MyST), you need to install the following package.

pip install mystmd

Initial Book¶

Each initial book was created as follows.

・ myst init --yes
・ create an empty repository for book-repo on GitHub
・ git clone book-repo
・ copy the generated files to book-repo
・ create myst.yml with project title, GitHub link, and template settings

Manual Book Edit¶

When you begin to edit source files manually, the main configuration file is:

・ myst.yml

For compatibility, you may also have:

・ _config.yml (legacy, v1)
・ _toc.yml (legacy, v1)

The myst.yml file is the primary configuration for v2. It defines the project metadata, site template, and build options. As you edit the source file, which is either a markdown text file like “coding_style.md” or a Jupyter notebook like “prepare.ipynb”, you should be well acquainted with the syntax summarized in MyST syntax guide.

See also as you proceed:

• MyST Documentation

• MyST Syntax Overview

• Jupyter Book v1 to v2 migration notes

Book Update Cycle¶

So far, we have described the intial procudure required only once when you create a new book. Atfer that, we repeat the following cycle to update, brief descriptions of which will follow below.

・ edit manually
・ generate locally
・ synchronize master (or main) branch
・ deploy gh-pages branch

After manual edit, local generation should be achieved as follows in Command Prompt:

cd book-repo
myst build --html

For testing with the GitHub Pages base URL (to match production), use:

Windows PowerShell:

$env:BASE_URL = "/repo-name"
myst build --html

Linux/macOS:

BASE_URL=/repo-name myst build --html

Replace repo-name with the actual repository name (e.g., /molass-tutorial). This ensures asset links work correctly when deployed to GitHub Pages at https://biosaxs-dev.github.io/repo-name/.

After generation, check the generated local output in _build/html with your browser. Re-edit as needed until you are satisfied.

Synchronization of the main branches, local and remote, should be achieved as follows in VS Code:

・ commit in VS Code
・ synchronize with GitHub in VS Code

Deployment via GitHub Actions¶

All Jupyter Book repositories now use automated deployment via GitHub Actions. When you push to the main branch, the workflow automatically:

1. Installs mystmd

2. Runs myst build --html

3. Deploys to GitHub Pages

See .github/workflows/deploy.yml in each repository for the workflow configuration.

Manual deployment (alternative)¶

If you need to deploy manually, you can still use ghp-import:

ghp-import -n -p -f _build/html

The web page will be updated in a few minutes. However, GitHub Actions deployment is preferred as it ensures consistency and handles the build environment automatically.

Common Issues and Troubleshooting¶

Duplicate Titles in Notebooks¶

Problem: A notebook page shows duplicate titles (e.g., “Installation” appearing twice on the same page)

Root cause: MyST derives the page title from the first H2 heading if no explicit title is set. For example:

# Quick Start
## Installation

MyST ignores the H1, uses “Installation” as the page title, AND renders “## Installation” as content → duplication.

Solution: Add MyST frontmatter to the first markdown cell of the notebook:

---
title: Quick Start
---

# Quick Start
## Installation
...

This explicitly sets the page title, preventing MyST from deriving it from the first H2 heading.

Testing locally:

1. Save the notebook

2. myst start → opens server at http://localhost:3000

3. Navigate to the affected page

4. Verify no duplication

5. Ctrl+C to stop server

Reference: Fixed in molass-tutorial commit dd4da49 (2026-06-29)

Writing Notebooks for Documentation¶

This section covers conventions for authoring Jupyter notebooks that will be built into documentation books (tutorial, essence, technical, develop).

MyST Development Workflow¶

Start development server:

cd book-repo
myst start

Opens at http://localhost:3000 (or another port if 3000 is busy). MyST automatically rebuilds when it detects notebook or markdown file changes. Just refresh the browser to see updates.

If auto-rebuild fails:

myst clean --all
myst start

Suppressing Long Outputs¶

Problem: Optimization loops, DENSS reconstructions, or iterative algorithms print thousands of lines in notebook output, making the built HTML page unreadable.

Solutions (in preference order):

1. %%capture cell magic (cleanest)

%%capture
run_denss(...)  # All stdout/stderr suppressed

2. Redirect stdout (when you need selective printing)

import io, sys
from contextlib import redirect_stdout

with redirect_stdout(io.StringIO()):
    run_denss(...)
print("Optimization completed")

3. %%script false (show code but don’t execute)

%%script false
run_expensive_operation()  # Cell doesn't run

4. Manual output clearing (last resort)

• Run cell → Edit → Clear Outputs (for that cell) → Save

• Requires manual intervention; use only when programmatic suppression isn’t feasible

Reference: molass-tutorial DENSS output fix (2026-07-01, commit 3ee3a83)

Notebook Frontmatter¶

MyST requires frontmatter in the first markdown cell to avoid title duplication:

---
title: Page Title
---

# Page Title
## Section content

Without frontmatter, MyST may derive the page title from the first H2 heading, causing duplication.

Common Cell Magics¶

Build vs Dev Server¶

⚠️ Important: Both build and dev server use stored outputs from the .ipynb file - they do not re-execute cells. If you add %%capture (or other output-suppressing magic) to a cell that already has outputs:

1. The old outputs remain in the file until you re-run the cell

2. The built HTML will still show the old outputs

3. You must re-run the cell (or clear its outputs) before committing

Example: molass-tutorial DENSS page (2026-07-01) - adding %%capture wasn’t enough; we had to clear the 19,863 lines of stored DENSS output for the fix to appear in the deployed page (commit 6c0bd57).

Tool Choice for Notebook Editing¶

When notebook is open in VS Code:

• Copilot: Use aicEditNotebookCell or edit_notebook_file

• Human: Edit directly in VS Code notebook editor

When notebook is closed:

• Use edit_notebook_file

Never use for notebooks:

• replace_string_in_file (JSON escaping mismatch)

• PowerShell text operations (UTF-8 BOM injection risk)

Executing Notebooks and Populating Outputs¶

Important: MyST does not execute notebooks during the build - it uses the stored outputs in the .ipynb files. Before committing changes, you must execute notebooks locally to populate outputs.

Automated execution script: All four documentation repositories (tutorial, essence, technical, develop) include execute_notebooks.ps1:

# Execute all notebooks (useful for initial setup or major changes)
.\execute_notebooks.ps1 -All

# Execute only changed notebooks (default, efficient for incremental work)
.\execute_notebooks.ps1

# Execute a specific notebook
.\execute_notebooks.ps1 -Path chapters\05\lrf.ipynb

# Execute all notebooks in a chapter
.\execute_notebooks.ps1 -Chapter 20

When to use -All flag:

1. Initial repository setup (populating all outputs for the first time)

2. After updating molass library version (ensure all notebooks work with new API)

3. Before major releases (verify all notebooks execute correctly)

4. When diagnostic tool reports empty outputs

Default behavior (-Changed): Uses git to detect modified notebooks since last commit, executes only those. This is the recommended workflow for daily editing.

Execution workflow:

1. Edit notebook in VS Code

2. Save changes

3. Run .\execute_notebooks.ps1 (detects changes via git, executes modified notebooks)

4. Verify outputs in the notebook

5. Commit and push

CI validation: The GitHub Actions workflow includes a validation step that checks all notebooks have outputs before building. If validation fails, the deployment is blocked and you’ll see an error like:

❌ ERROR: Notebooks without outputs (run ./execute_notebooks.ps1 locally)
  - chapters/05/lrf.ipynb
  - chapters/20/rigorous.ipynb

Fix by running .\execute_notebooks.ps1 -All locally, then commit and push.

Diagnostic tool: check_notebook_status.py shows the execution status of all notebooks:

python check_notebook_status.py

Output example:

✅ 11/11         chapters\01\data_objects.ipynb
⚠️  9/11        chapters\05\lrf.ipynb
❌ 0/5          chapters\20\rigorous.ipynb

• ✅ All code cells have outputs

• ⚠️ Some cells have outputs (partial execution)

• ❌ No code cells have outputs (not executed)

Reference: Automation added in molass-essence commit 3fb9694 (2026-07-03), replicated to all four documentation repositories.

How to use Sphinx¶

If you are just updating existing parts of the document, skip to Usual Update Routine.

Tool Package Installation¶

To use Sphinx for API documentation, install the following packages:

pip install sphinx
pip install sphinx-book-theme
pip install sphinx-copybutton
pip install myst-parser

Initial Generation¶

When you are updating, skip to Gererate *.rst files.

To generate a set of initial examples, execute commands as follows in the repository root.

mkdir docs
cd docs
sphinx-quickstart

Reply minimally, i.e. defaults or none, to queries from the last command except the following two.

• Project name: Molass Library

• Author name(s): Molass Community

The command will gerenate several files, among which you should edit the follow two.

• conf.py

• index.rst

We will give some hints on edition later.

Customize conf.py¶

The conf.py file in docs/ controls Sphinx behavior. Key customizations for molass-library:

1. Version Configuration¶

Extract version from the package dynamically:

from molass import get_version
release = str(get_version(toml_only=True))
version = '.'.join(release.split('.')[:2])  # X.Y from X.Y.Z

Why: Ensures documentation version matches package version automatically. No manual updates needed when releasing new versions.

Effect: Version appears in browser tab title (e.g., “Molass Library Reference — Molass Library 1.0.1 documentation”) and HTML metadata.

2. Type Hints Display¶

autodoc_typehints = 'description'
autodoc_typehints_description_target = 'documented'

Why: Python type hints (e.g., path: str | Path) are cleaner in parameter descriptions than in function signatures.

Effect: Type information appears in the “Parameters” section with better formatting, making API documentation more readable.

3. Intersphinx Mapping¶

Cross-reference external documentation:

intersphinx_mapping = {
    "python": ("https://docs.python.org/3", None),
    "numpy": ("https://numpy.org/doc/stable/", None),
    "matplotlib": ("https://matplotlib.org/stable/", None),
    "scipy": ("https://docs.scipy.org/doc/scipy/", None),
}

Why: When molass functions use numpy.ndarray or matplotlib.axes.Axes in type hints or docstrings, these become clickable links to the official documentation of those libraries.

Effect: Users can click ndarray to jump to NumPy’s documentation, improving discoverability.

4. Theme Options¶

html_theme_options = {
    "repository_url": "https://github.com/biosaxs-dev/molass-library",
    "use_repository_button": True,
    "show_toc_level": 2,          # Show 2 levels in table of contents
    "navigation_depth": 4,        # Allow 4 levels in sidebar navigation
    "pygment_dark_style": "monokai",  # Code highlighting in dark mode
}

Why:

• show_toc_level and navigation_depth improve navigation for large API surfaces

• pygment_dark_style ensures readable code syntax highlighting when users prefer dark mode

• Repository button links users back to GitHub for issues/contributions

Effect: Better navigation UX and improved accessibility.

5. Path Setup¶

import sys, os
root_dir = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
sys.path.insert(0, root_dir)  # Required to import molass

Why: Sphinx needs to import the package to extract docstrings. This ensures it imports the current development version, not an installed package.

Effect: Documentation always reflects the current state of the source code.

6. Essential Extensions¶

extensions = [
    "sphinx.ext.autodoc",       # Extract docstrings from Python code
    "sphinx.ext.napoleon",      # Support NumPy/Google docstring styles
    "sphinx.ext.autosectionlabel",  # Auto-generate section labels for linking
    'sphinx_copybutton',        # Add copy button to code blocks
    'myst_parser',              # Parse MyST markdown in docs
    "sphinx.ext.intersphinx",   # Cross-reference external docs
]

Reference: Configuration improvements applied 2026-07-03 (conf.py in molass-library).

Gererate *.rst files¶

The following command line will generate *.rst files from Python source codes in ../molass folder. Namely in docs:

sphinx-apidoc --output-dir source ../molass --separate --module-first

After this generation, the folder tree will look as follows.

    molass-library/
        docs/
            _static/
            source/
            tools/
            make.bat
            Makefile
            conf.py
            index.rst
        molass/

We confined *.rst files into the “source” subfolder to make it clear that they should be generated from the python source codes in “molass” folder, and as such, they should be ignored in version control of Git. (See .gitignore to confirm this)

Edit index.rst¶

The *.rst files can be made into HTML files only if referenced. In oeder to be referenced, we must initiate the reference chain of HTML files in index.rst

• edit index.rst

That is, if the HMTL generator sphinx-build, invoked later in a make command line, finds source/molass.Baseline in the index.rst, thenn it looks into it and finds a next reference, and so on.

Modify *.rst files¶

Unfortunately, we have to modify the first version of *.rst files generated by sphinx-apidoc in the above step because they include redundantly repeated prefixes or modifiers, like molass., package or module, in titles which we will remove by the following command line.

In docs:

python tools/EditRst.py

This script edits all .rst files in the “docs/source” directory to update the title lines.

Make HTML files¶

Now we are ready to invoke sphinx-build by the following command line.

In docs:

make html

This will produce HTML files in “_build” subfolder.

Deployment¶

The following command line will deploy them as we did with Jupyter Book above.

In docs:

ghp-import -n -p -f _build/html

This (re)creates the gh-pages branch both in local and remote repositories, and the web book will be updated in a few minutes.

Usual Update Routine¶

After editing parts of source code, do the following in docs.

python tools/UsualUpdate.py

See the module documentation for details.