Rigorous.RunInfo

Rigorous.RunInfo#

Rigorous.RunInfo.py

class RunInfo(ssd, optimizer, dsets, init_params, monitor=None, analysis_folder=None, decomposition=None, rgcurve=None)#

Bases: object

Handle returned by molass.Rigorous.make_rigorous_decomposition() (and Decomposition.optimize_rigorously()).

ssd#

The SEC-SAXS dataset used for the optimization.

Type:: SecSaxsData

optimizer#

The in-process optimizer object. Only populated for the in-process path (in_process=True); None for the subprocess path.

Type:: Optimizer

dsets#

Dataset objects passed to the optimizer.

Type:: list

init_params#

Initial parameter vector used to start the optimization.

Type:: ndarray

monitor#

Live monitor object for the subprocess path; None for in-process.

Type:: MplMonitor or None

analysis_folder#

Root folder for all optimizer output. Set this if you want the progress/sidecar helpers to work without arguments.

Type:: str or None

decomposition#

The Decomposition object that launched this run.

Type:: Decomposition or None

work_folder#

Absolute path to the job folder written by the in-process optimizer (<analysis_folder>/optimized/jobs/000 or similar). Set only for the in-process path; None for the subprocess path. Contains init_params.txt, callback.txt, etc.

Type:: str or None

in_process_result#

Raw result object returned by run_optimizer_in_process(). Set only for the in-process path.

Type:: object or None

request_stop()#

Request cooperative termination of an async in-process run.

Sets the stop event that InProcessRunner checks between solver iterations. The solver thread receives a KeyboardInterrupt at the next Python bytecode boundary (~50 ms at most). Has no effect if the run is already finished or was not in-process.

property is_alive#

True while the background optimizer is still running.

Works for both async in-process runs (async_=True) and subprocess runs (monitor=False, in_process=False). Returns False once the run completes or for synchronous runs.

property current_work_folder#

The currently-active (or most-recently-used) job folder.

Unlike work_folder — which is reset to None at the start of each auto-resume trial — this property always returns the best available answer:

self.work_folder if already set.
The most recently modified jobs/NNN sub-folder under analysis_folder/optimized/jobs/ (the highest NNN that exists on disk).
None if neither is available.

This is the right folder to probe with aicKernelEval during a multi-trial run (max_trials > 0). Fixes issue #150.

update_manifest_on_resume(work_folder)#

Update RUN_MANIFEST.json when a new auto-resume trial starts.

Called from MplMonitor._RunInfoSource._wf_callback as soon as the new job folder is allocated, so that live_status() reads the correct work_folder and phase='running' for the new trial. Fixes issue #148.

Parameters:: work_folder (str) – Absolute path to the newly-allocated job folder.

get_current_decomposition(**kwargs)#

resume(niter=20, debug=True)#

Resume optimization from the best parameters of the previous run.

This mirrors the legacy GUI’s “Resume” button: extracts the best parameters from the completed job’s callback.txt, then launches a new optimizer subprocess using those as the starting point.

Parameters:

niter (int, optional) – Number of iterations for the resumed optimization. Default is 20.
debug (bool, optional) – If True, enable debug mode (reloads modules from disk). Default is True.

Returns:

self – The same RunInfo instance, updated with the new monitor.

Return type:

RunInfo

wait(timeout=600, poll_interval=5)#

Wait for rigorous optimization results to become available.

Parameters:

timeout (float, optional) – Maximum seconds to wait (default 600). Use 0 for no limit.
poll_interval (float, optional) –
Seconds between checks (default 5).

Note

For async in-process runs (async_=True), poll_interval is silently ignored — the implementation calls thread.join(timeout) once with no looping.

Returns:

True if results appeared, False if timed out.

Return type:

bool

Raises:

ValueError – If no analysis_folder was stored (e.g. RunInfo was created without one).
.. warning: – Async in-process path (async_=True): This method calls _async_thread.join(timeout) once, which has two failure modes for long BH/NS runs: - Default timeout=600: silently returns False for runs longer than 10 minutes while the optimizer is still running. Downstream calls to load_best() will then fail or load stale results. - timeout=0 (no limit): blocks the kernel’s main thread entirely. No other cell — including live_status(), is_alive, or any monitoring probe — can execute until the optimizer finishes. For interactive notebooks prefer load_best(), which waits only until the first result lands on disk and returns immediately, or poll manually with:: if run_info.is_alive: print(run_info.live_status())

load_best(timeout=0, poll_interval=5, debug=False)#

Load the best rigorous optimization result, waiting until one is available.

Safe to call immediately after optimize_rigorously() — it blocks until at least one job has completed, then returns the result with the lowest objective function value found so far. Re-running this cell while the optimizer is still going always returns the best result available at that moment.

Parameters:

timeout (float, optional) – Maximum seconds to wait for the first result (default 0 = no limit). When non-zero and no result appears within timeout seconds, raises TimeoutError.
poll_interval (float, optional) – Seconds between filesystem checks (default 5).
debug (bool, optional) – If True, reload modules from disk.

Returns:

A Decomposition built from the best optimized parameters available at the time the call returns. The result.sv and result.fv attributes are attached for convenience.

Return type:

Decomposition

Raises:

ValueError – If no analysis_folder was stored.
TimeoutError – If timeout > 0 and no result appears within that time.

Notes

Interrupt with Ctrl+C to cancel the wait at any time.

If the optimizer is still running, the result reflects only the jobs completed so far. Re-call after the run finishes to get the final best.

Score architecture#

The objective value fv is computed as:

fv = synthesize(scores, positive_elevate=3) + sum(penalties)

Synthesized scores (first 7): XR_2D_fitting, XR_LRF_residual, UV_2D_fitting, UV_LRF_residual, Guinier_deviation, Kratky_smoothness, SEC_conformance. These are combined via a weighted RMS + spread measure, shifted so that a raw score of -3 maps to zero contribution.

Additive penalties (remaining entries): mapping_penalty, negative_penalty, baseline_penalty, outofbounds_penalty, order_penalty, control_penalty, consistency_penalty. These are added directly to fv after synthesis. A penalty of 1.0 raises fv by exactly 1.0.

param jobid:: Specific job id to evaluate. If None, uses the best job (lowest best_fv).
type jobid:: str, optional
param debug:: If True, reload modules from disk.
type debug:: bool, optional
returns:: {'fv': float, 'scores': {name: value, ...}} where scores maps each score/penalty name to its numeric value.
rtype:: dict
raises ValueError:: If no analysis_folder was stored.
raises FileNotFoundError:: If no completed jobs are found.

Examples

run_info = decomp.optimize_rigorously(
    analysis_folder="temp_analysis", niter=30)
run_info.wait()
breakdown = run_info.get_score_breakdown()
for name, val in breakdown['scores'].items():
    print(f"{name}: {val:.4f}")

get_current_curves()#

Return the data and model curves currently shown on the monitor.

Delegates to MplMonitor.get_current_curves() (molass-legacy #31, monitor readability). Provides a single-call entry point from the user-facing RunInfo object so that an AI agent can query run_info.get_current_curves() without knowing about the internal monitor attribute.

Returns:: See MplMonitor.get_current_curves() for the full key list. Returns None if the monitor is not set or has no data yet.
Return type:: dict or None

diagnose(breakdown=None)#

Map score values to physical interpretations.

Calls get_score_breakdown() if no breakdown is provided, then applies encoded rules to each score/score-pair and returns a list of Diagnosis namedtuples with structured physical meaning.

This allows an AI agent (or a human) to understand why a fit is poor without requiring domain knowledge: the rules encode the mapping from numeric scores to physical causes.

Parameters:

breakdown (dict, optional) – Output of get_score_breakdown(). If None, it is computed automatically (which loads and evaluates the best job from disk).

Returns:

Each Diagnosis has the fields: - score : str – the score/penalty name (or pair) - status : str – 'good', 'fair', 'poor', or

'failing'

reason : str – human-readable explanation
suggestion : str or None – recommended next diagnostic step

Return type:

list of Diagnosis

Examples

run_info.wait()
for d in run_info.diagnose():
    print(f"[{d.status.upper()}] {d.score}: {d.reason}")
    if d.suggestion:
        print(f"  -> {d.suggestion}")

compare_subprocess_dsets(plot=True, mp_b_sweep=True, mp_b_range=(-200.0, 200.0), n_points=81)#

Debug tool for molass-legacy#34: compare parent vs subprocess datasets.

Reconstructs the datasets as the subprocess would derive them (by re-reading raw data from disk via FullOptInput → OptDataSets), then compares them with the parent’s live datasets side-by-side.

If mp_b_sweep=True and self.optimizer is available, also sweeps the objective function along the mp_b axis for both the parent optimizer (parent dsets) and a subprocess-reconstructed optimizer (subprocess dsets). This shows whether — and why — the two objective landscapes differ, which is the root cause of #34.

Parameters:

plot (bool, optional) – If True (default), produce visual comparison plots.
mp_b_sweep (bool, optional) – If True (default), also sweep mp_b on both optimizers and plot the objective landscape. Requires self.optimizer to be set.
mp_b_range ((float, float), optional) – Range of mp_b values for the sweep. Default (-200, 200).
n_points (int, optional) – Number of evaluation points for the mp_b sweep. Default 81.

Returns:

sub_dsets – The subprocess-reconstructed dataset object for further inspection.

Return type:

OptDataSets

Raises:

RuntimeError – If work_folder or dsets is not set on this RunInfo.

Examples

# After a subprocess run (in_process=False):
run_info = decomp.optimize_rigorously(rgcurve, in_process=False)
sub_dsets = run_info.compare_subprocess_dsets()

property monitor_snapshot_json_path#

Path to the MplMonitor JSON sidecar written during the last run.

The file is created when MOLASS_MONITOR_SNAPSHOT=1 and exists at <analysis_folder>/optimized/figs/mplmonitor_latest.json.

Returns:

str – Absolute path to the JSON file, whether or not it exists.
None – If analysis_folder was not set on this RunInfo.

check_progress(label=None, write_snapshot=False)#

Read callback.txt(s) from all jobs and report best SV so far.

Re-runnable at any time while the optimizer is running or after it completes. Does not require the optimizer to have finished.

The implementation lives in molass.Rigorous.CurrentStateUtils.check_progress() so it can be updated and reloaded without restarting the kernel — just call importlib.reload(molass.Rigorous.CurrentStateUtils) and the next invocation picks up the new logic automatically.

Parameters:

label (str, optional) – Display label prefix. Defaults to the analysis folder basename.
write_snapshot (bool, optional) – If True, write a compact JSON file to <analysis_folder>/optimized/progress_snapshot.json in addition to printing. Read back via load_progress_snapshot(). Default False.

Returns:

Progress data dict when there are evaluations to report; None otherwise.

Return type:

dict or None

Examples

_run_sub.check_progress()                         # print only
_run_sub.check_progress(label="subprocess")       # explicit label
snap = _run_sub.check_progress(write_snapshot=True)  # persist + return

See also

molass.Rigorous.check_progress: standalone form; also accepts a plain folder-path string when no RunInfo instance is at hand:: from molass.Rigorous import check_progress check_progress(“/path/to/analysis_folder”)

property progress_snapshot_json_path#

Path to the progress snapshot JSON written by check_progress(write_snapshot=True).

Exists at <analysis_folder>/optimized/progress_snapshot.json.

Returns:

str – Absolute path to the JSON file, whether or not it exists yet.
None – If analysis_folder was not set on this RunInfo.

load_progress_snapshot()#

Load and return the progress snapshot JSON as a dict.

Written by check_progress() when write_snapshot=True. Readable from any session — including a new AI session — without re-running the optimizer.

Returns:

Keys: label, n_evals, best_fv, best_sv, sv_best_so_far, timestamp.

Return type:

dict

Raises:

FileNotFoundError – If the snapshot does not exist yet (call check_progress(write_snapshot=True) first).
ValueError – If analysis_folder is not set on this RunInfo.

Examples

_run_sub.check_progress(write_snapshot=True)
snap = _run_sub.load_progress_snapshot()
print(f"best SV: {snap['best_sv']:.2f}")

load_monitor_snapshot()#

Load and return the MplMonitor JSON sidecar as a dict.

Returns:

Parsed JSON content.

Return type:

dict

Raises:

FileNotFoundError – If the sidecar does not exist (MOLASS_MONITOR_SNAPSHOT was not set, or the optimizer has not run yet).
ValueError – If analysis_folder is not set on this RunInfo.

property sv_history#

Min-so-far SV trajectory from all callback.txt files.

Returns the full sequence of best-SV-so-far values, one per accepted optimizer evaluation. Readable at any time — while the run is in progress or after it completes.

Returns:: Running-minimum SV values. sv_history[-1] is the best accepted SV of the entire run. Empty list if no evaluations have been recorded yet.
Return type:: list of float
Raises:: ValueError – If analysis_folder is not set on this RunInfo.

Examples

svs = run_sub.sv_history
print(f"best SV: {svs[-1]:.2f}  start SV: {svs[0]:.2f}")

property sv_history_per_job#

Per-job SV best-so-far trajectories from all callback.txt files.

Like sv_history but preserves job boundaries. Useful when a BH run spans multiple restarts (jobs 000, 001, …) and you want to know how much each restart contributed.

Returns:: Mapping from job id (e.g. '000') to the list of global best-SV-so-far values recorded within that job. Empty dict if no evaluations have been recorded yet.
Return type:: dict[str, list of float]

Examples

per_job = run_info.sv_history_per_job
for job_id, svs in per_job.items():
    print(f"job {job_id}: {len(svs)} evals, best SV = {svs[-1]:.1f}")

plot_sv_history(title=None, figsize=(8, 4))#

Plot the min-so-far SV trajectory from callback.txt.

One-liner convergence view. Works at any time after at least one evaluation has been accepted.

Parameters:

title (str, optional) – Figure title. Defaults to "SV history — <folder basename>".
figsize (tuple, optional) – Matplotlib figure size. Default (8, 4).

Return type:

None

Examples

run_sub.plot_sv_history()
run_sub.plot_sv_history(title="Apo 2-comp NS, subprocess path")

live_status()#

Return a single dict snapshot of where this run stands right now.

One-call replacement for the scattered probe pattern (sv_history property + check_progress + RunRegistry.read_manifest + getattr(self, 'subprocess_returncode', None) + …) used in diagnostic notebook cells. Pure read: scans disk, never mutates. Safe to invoke from aicKernelEval (issue ai-context-vscode#1) or any sync cell, including while the run is still in flight.

Returns:

Keys (all best-effort; missing data → None):

phase (str): one of 'pending', 'running', 'completed', 'failed', 'unknown'. Derived from the manifest’s status field plus any subprocess_returncode available.
n_evals (int): number of accepted optimizer evaluations recorded in callback.txt so far.
best_fv (float): inverted from best_sv (matches check_progress arithmetic).
best_sv (float): best score-value-so-far on the 0-100 scale.
elapsed_s (float): seconds since manifest start_time, or None if the manifest isn’t available yet.
analysis_folder (str): from self.analysis_folder.
work_folder (str): from self.work_folder, falling back to walking analysis_folder for callback.txt.
subprocess_pid (int or None): from manifest.
subprocess_returncode (int or None): from self.subprocess_returncode or manifest.
manifest (dict or None): the full RUN_MANIFEST.json contents from the analysis folder, when present.

Return type:

dict

Examples

From a notebook cell while the run is in flight:

run_sub.live_status()

From outside the kernel via ai-context-vscode#1:

aicKernelEval(expression="run_sub.live_status()")

property run_complete_path#

Path to run_complete.json written when the optimizer finishes normally.

The file is always created on completion (no env-var required) at <analysis_folder>/optimized/figs/run_complete.json.

It is the canonical, zero-parse-required answer to “what was the best result?” for AI tools and notebook cells in a new session.

Returns:

str – Absolute path to the JSON file, whether or not it exists yet.
None – If analysis_folder was not set on this RunInfo.

load_run_complete()#

Load and return the run_complete.json written on job completion.

Returns:

Keys: schema_version, completed_at, best_fv, best_sv, n_evals, n_accepted, analysis_folder.

Return type:

dict

Raises:

FileNotFoundError – If the file does not exist yet (job still running, or completed before this fix was active).
ValueError – If analysis_folder is not set on this RunInfo.

Examples

rc = run_sub.load_run_complete()
print(f"best_sv = {rc['best_sv']:.2f}")

classmethod reconnect(analysis_folder, raise_if_not_found=True)#

Reconnect to a running or completed optimization from disk.

Creates a minimal RunInfo populated from the manifest and disk state in analysis_folder. Useful after a kernel restart or an accidental cell re-run that destroyed the original live reference.

The recovered RunInfo supports all disk-based operations: live_status(), sv_history, load_best(), load_best(), plot_sv_history().

It does not have a live optimizer, so get_score_breakdown() and diagnose() require the optimizer to be reconstructed separately (not automatic).

Parameters:

analysis_folder (str) – The analysis_folder that was passed to optimize_rigorously().
raise_if_not_found (bool, optional) – If True (default), raise FileNotFoundError when no manifest exists. If False, return None instead — useful inside the idempotency guard where a missing manifest simply means no prior run exists.

Returns:

A reconstituted RunInfo, or None if raise_if_not_found=False and no manifest is found.

Return type:

RunInfo or None

Raises:

FileNotFoundError – If no RUN_MANIFEST.json is found and raise_if_not_found=True.

Examples

After a kernel restart:

run_info = RunInfo.reconnect(analysis_folder)
run_info.live_status()
run_info.plot_sv_history()
result = run_info.load_best()

Rigorous.RunInfo

Contents

Rigorous.RunInfo#

Score architecture#