Extend cross-engine verification suite with FullProf references#196
Merged
Conversation
Drop peak-normalisation in pattern_closeness: metrics are computed on absolute intensities so a seeded FullProf scale gives a real scale comparison. Rename max_deviation -> max_deviation_percent, make the intensity ratio the absolute area ratio, relabel it "Area ratio" and correlation "Shape correlation" to mark the scale vs shape axes, and tighten the default tolerances. Remove the scale_to_reference and report_reference_scales helpers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drop peak-normalisation in plot_calc_comparison and plot_reflection_comparison so the overlay and the F-squared scatter show the true absolute scale; remove the now-unused _peak_normalized helper. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the max_deviation field, flip the scale-independence test to assert the metrics now reflect absolute scale, and update the tolerance defaults. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Seed each page with the FullProf .pcr scale and compare on absolute intensities; annotate every input with its FullProf parameter name. Where an engine then disagrees, add a per-engine refinement that frees the differing parameter (asymmetry on asym-PbSO4, the Lorentzian gamma and scale on Si, scale on NCAF and Pr2NiO4) and reports the before/after closeness. Point lab6 at the new background-free FullProf .sub, keep its SyCos/SySin lines ready to uncomment, and skip NCAF in CI while its TOF scale convention is refined. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reconstruct the x grid from the number of intensities rather than the header maximum, so a maximum rounded a fraction of a step off (a common FullProf quirk, hit by the Jorgensen Si .sub) no longer adds a spurious point; keep the header maximum as a one-step sanity check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Verify the plain Jorgensen TOF profile (Gaussian plus back-to-back exponentials, no Lorentzian) for Si against FullProf. Both engines diverge from FullProf's Jorgensen shape, so the page seeds the .pcr scale, reports the absolute comparison, and refines the broadening terms and scale with cryspy to quantify the residual. Skipped in CI like the other report-only TOF pages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Return nothing so a notebook cell ending in the call shows only the rendered before/after table, not an echoed metrics tuple; callers that need the numbers use pattern_closeness directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mark the verification pages listed in ci_skip.txt as xfail(strict=False) instead of skip, so nbmake still executes them — they render in the docs and --overwrite writes their outputs — while a known discrepancy or pending-feature page stays an expected failure rather than a suite failure. The script-test runner still skips them as the fast regression check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Give every page the same cell order and section headers, move project creation into its own cell, drop the asym Conclusion cell, normalise the FullProf comment spacing, and correct the ncaf/asym descriptions (they are investigation pages, not script-test regression checks). Parameter values are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
AndrewSazonov
added
the
[scope] maintenance
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #196 +/- ##
===========================================
- Coverage 90.89% 90.66% -0.23%
===========================================
Files 264 265 +1
Lines 22225 22494 +269
Branches 2587 2612 +25
===========================================
+ Hits 20201 20394 +193
- Misses 1507 1577 +70
- Partials 517 523 +6
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Expands the verification pages that check each calculation engine (cryspy, crysfml) against an independent FullProf reference.