Enhance Windows compatibility and update CI configurations#212
Open
hkmoon wants to merge 187 commits intobertiniteam:developfrom
Open
Enhance Windows compatibility and update CI configurations#212hkmoon wants to merge 187 commits intobertiniteam:developfrom
hkmoon wants to merge 187 commits intobertiniteam:developfrom
Conversation
There are some compiling errors while being compiled in windows * `size_t` is translated into `unsigned long` in linux, mac while `unsigned long long` in windows 10: `core/include/bertini2/eigen_extensions.hpp` and `core/test/classes/start_system_test.cpp` are modified * use `clang` of LLVM in Windows since MSVC has different compiling way for `template` * use `--no-isolation` for `scikit-build` in Windows For linux wheel naming convention, we cannot use x86_64, x86_i386 anymore for pypi repository. https://peps.python.org/pep-0600/ * use `auditwheel` for it Co-authored-by: HongKee Moon <moon@mpi-cbg.de>
the default branch is changed to 'develop'
additionally, fix ctest folder
additionally, fix ctest folder
* chore: test for win32 * chore: use Release libraries for linking in windows * chore: use 1e-14 instead of 1e-15 * 1e-15 doesn't work in windows * chore: remove docker-compose.yml * chore: revert test list --------- Co-authored-by: HongKee Moon <moon@mpi-cbg.de>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Needed the flags in all windows, not just MSVC compiler. found mistaken s in variable name, removed it test pool wasnt using the correct source list.
gave warning on Windows clang. using explicit namespace silences warning
was getting warning for extra tokens. compiling with clang on windows, there was no need for the win32 part of the ifndef, so i left it commented out. at some point, the commented-out should be removed.
Similar to a few commits ago. That was for core, this is for the bindings.
Contributor
|
Using the environment yaml file at root level to provide dependencies via micromamba, together with an install of Visual Studio C++ (to resolve a linking problem with clang on windows), i was able to get it built and all core unit tests passed on Windows. |
Contributor
|
remaining before I accept:
|
Author
|
@ofloveandhate That sounds reasonable. If you want me to work for them, just let me know. ;) |
Contributor
|
I think the most difficult part is the python tests. They're also the most important. They're failing because of numpy data type stuff. So it's not trivial. If I remember correctly, it's a type registration thing, so that numpy knows what to do with the custom data types I'm using (variable precision complex and real numbers provided via Boost.Multiprecision). |
Author
|
Hope it's helpful.
In the project folder: b2 on develop [$!?] via v3.13.3+ (venv) took 4s
❯ python -m pytest python/test/ -v
====================================================== test session starts ======================================================
platform darwin -- Python 3.13.3+, pytest-9.0.3, pluggy-1.6.0 -- /Users/moon/Projects/git-projects/Harrington/b2/venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/moon/Projects/git-projects/Harrington/b2/python
configfile: pyproject.toml
collected 49 items
python/test/classes/differentiation_test.py::DiffTest::test_power_rule PASSED [ 2%]
python/test/classes/differentiation_test.py::DiffTest::test_prod_rule PASSED [ 4%]
python/test/classes/differentiation_test.py::DiffTest::test_sum_rule PASSED [ 6%]
python/test/classes/differentiation_test.py::DiffTest::test_trancendental PASSED [ 8%]
python/test/classes/function_tree_test.py::SymbolTest::test_E_construct PASSED [ 10%]
python/test/classes/function_tree_test.py::SymbolTest::test_Float_construct PASSED [ 12%]
python/test/classes/function_tree_test.py::SymbolTest::test_Float_eval PASSED [ 14%]
python/test/classes/function_tree_test.py::SymbolTest::test_Float_funcs PASSED [ 16%]
python/test/classes/function_tree_test.py::SymbolTest::test_I_construct PASSED [ 18%]
python/test/classes/function_tree_test.py::SymbolTest::test_Pi_construct PASSED [ 20%]
python/test/classes/function_tree_test.py::SymbolTest::test_Variable_construct PASSED [ 22%]
python/test/classes/function_tree_test.py::SymbolTest::test_Variable_eval PASSED [ 24%]
python/test/classes/function_tree_test.py::SymbolTest::test_Variable_funcs PASSED [ 26%]
python/test/classes/function_tree_test.py::OperatorTest::test_Homogenize PASSED [ 28%]
python/test/classes/function_tree_test.py::OperatorTest::test_Operator_degree PASSED [ 30%]
python/test/classes/function_tree_test.py::OperatorTest::test_Operator_ishom PASSED [ 32%]
python/test/classes/function_tree_test.py::OperatorTest::test_Operator_ispoly PASSED [ 34%]
python/test/classes/function_tree_test.py::OperatorTest::test_num_times_var PASSED [ 36%]
python/test/classes/function_tree_test.py::OperatorTest::test_plus PASSED [ 38%]
python/test/classes/function_tree_test.py::OperatorTest::test_power PASSED [ 40%]
python/test/classes/function_tree_test.py::OperatorTest::test_sub PASSED [ 42%]
python/test/classes/function_tree_test.py::OperatorTest::test_trans_funcs PASSED [ 44%]
python/test/classes/function_tree_test.py::OperatorTest::test_var_div_var PASSED [ 46%]
python/test/classes/function_tree_test.py::OperatorTest::test_var_times_var PASSED [ 48%]
python/test/classes/mpfr_test.py::MPFRFloat::test_arith_int PASSED [ 51%]
python/test/classes/mpfr_test.py::MPFRFloat::test_arith_mpfr PASSED [ 53%]
python/test/classes/mpfr_test.py::MPFRFloat::test_change_prec PASSED [ 55%]
python/test/classes/mpfr_test.py::MPFRFloat::test_trancendentals PASSED [ 57%]
python/test/classes/mpfr_test.py::MPFRComplex::test_arith_mp_complex PASSED [ 59%]
python/test/classes/mpfr_test.py::MPFRComplex::test_arith_mp_float PASSED [ 61%]
python/test/classes/mpfr_test.py::MPFRComplex::test_change_prec PASSED [ 63%]
python/test/classes/mpfr_test.py::MPFRComplex::test_complex_abs PASSED [ 65%]
python/test/classes/mpfr_test.py::MPFRComplex::test_complex_arg PASSED [ 67%]
python/test/classes/mpfr_test.py::MPFRComplex::test_complex_conj PASSED [ 69%]
python/test/classes/mpfr_test.py::MPFRComplex::test_complex_construct_from_polar PASSED [ 71%]
python/test/classes/mpfr_test.py::MPFRComplex::test_construct PASSED [ 73%]
python/test/classes/mpfr_test.py::MPFRComplex::test_mp_complex_precision PASSED [ 75%]
python/test/classes/mpfr_test.py::MPFRComplex::test_trancendentals PASSED [ 77%]
python/test/classes/parser_test.py::ParserTest::test_create_system PASSED [ 79%]
python/test/classes/system_test.py::SystemTest::test_add_systems PASSED [ 81%]
python/test/classes/system_test.py::SystemTest::test_mult_system_node PASSED [ 83%]
python/test/classes/system_test.py::SystemTest::test_system_Jac PASSED [ 85%]
python/test/classes/system_test.py::SystemTest::test_system_create PASSED [ 87%]
python/test/classes/system_test.py::SystemTest::test_system_eval PASSED [ 89%]
python/test/tracking/amptracking_test.py::AMPTrackingTest::test_tracker_linear PASSED [ 91%]
python/test/tracking/amptracking_test.py::AMPTrackingTest::test_tracker_quad PASSED [ 93%]
python/test/tracking/amptracking_test.py::AMPTrackingTest::test_tracker_singular_start PASSED [ 95%]
python/test/tracking/amptracking_test.py::AMPTrackingTest::test_tracker_sqrt PASSED [ 97%]
python/test/tracking/endgame_test.py::EndgameTest::test_using_total_degree_ss PASSED [100%]
====================================================== 49 passed in 4.62s ======================================================= |
Contributor
|
There are some tests yet failing. I don't know why your report didn't run them. Here's my output, from running So the failing familiy tests have names like As I understand it, the default I'm not going to enable automatic/implicit conversion from double to variable precision. I do note that works, so variable precision can be directly constructed into numpy arrays, making these conversion tests kind of useless.
|
Contributor
|
Still wondering why these specific tests didn't run for you. |
Contributor
|
I'm also struggling to use pytest with my unit tests. (They are admittedly written with Is this a my-computer thing, or a setup problem with the unit tests? |
Please build and install it again: `pip install dist/bertini-1.0.0-cp313-cp313-macosx_14_0_arm64.whl --force-reinstall --no-deps` Then run `python python/test/test_all.py`
Author
|
Maybe I install bertini with python. # in the project root
rm -rf b2/bld && python -m build --wheel
# *** Making wheel...
# *** Created bertini-1.0.0-cp313-cp313-macosx_14_0_arm64.whl
# Successfully built bertini-1.0.0-cp313-cp313-macosx_14_0_arm64.whl
# please change the python version according to yours or use the wheel output
pip install dist/bertini-1.0.0-cp313-cp313-macosx_14_0_arm64.whl --force-reinstall --no-deps
python python/test/test_all.py |
Contributor
|
I saw the last commit. I don't think it's right. It allows implicit conversion from double to variable precision. I want to allow conversion, but only explicit. |
Author
|
It makes sense. |
Contributor
|
Adding more. Here's an explicit failing unit test, that cannot possibly ever succeed. Furthermore, I think it is very likely that a user tries to do But they'll never get good results from high-precision evaluation of this, because the 0.1 is hanging out stinking up the place. will NOT give 0.1 to D digits. Instead, it will give whatever double(0.1) --> Complex gives. I just want to help my users do the right thing, and keep them from accidentally using low-precision types where a high-precision type is expected. I'm still thinking about the right thing to do... |
Contributor
|
Disabling the IMPLICITLY_CONVERTIBLE calls restores some sanity back to what I expect: This is desirable. |
both for python, and in eigenpy/numpy
Contributor
|
I have a much better understanding of what explicit vs implicit conversion means in numpy now. This is considered an explicit conversion: I guess I want my users to be able to explicitly convert like this. I added a test that it produces "expected" values, in the sense that you do NOT get the same values as constructing variable-precision number from string, eg |
Contributor
|
I wrote a test to make sure that implicit conversion from double to float is disabled, particularly when forming systems / expressions |
Contributor
|
I feel like I'm probably missing a whole bunch of tests, but it'll do for now. |
Contributor
|
But i'm still having trouble with Pytest. The Python tests are passing when calling |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant improvements to the build system and release workflow for the project, especially focusing on cross-platform compatibility and automated Python package publishing. The main highlights include the addition of a comprehensive GitHub Actions workflow for building and publishing Python wheels to PyPI/TestPyPI, enhancements for Windows compatibility, and updates to documentation and changelogs to reflect these changes.
Automated Build and Release Workflow:
.github/workflows/build-and-publish-to-pypi.yml) to automate building Python wheels for Linux, macOS, and Windows, and to publish them to PyPI, TestPyPI, and GitHub Releases. This workflow includes environment setup, dependency installation, platform-specific build steps, artifact uploading, and release signing.Windows Compatibility Improvements:
cmake/FindGMP.cmake,cmake/FindMPFR.cmake,cmake/FindMPC.cmake) to correctly locate and link against Windows-specific library filenames, improving build reliability on Windows. [1] [2] [3]core/CMakeLists.txtto set appropriate build flags and shared library options for MSVC, and improved Boost/Eigen3 detection and linking for cross-platform builds. [1] [2] [3] [4]Documentation and Changelog Updates:
CHANGELOG.mddocumenting recent changes, release preparations, and platform-specific notes, including new contributor acknowledgments.README.mdwith clear installation instructions for Linux, macOS, and Windows, and clarified wheel/platform support.CMake Modernization and Refactoring:
CMakeLists.txtto use modern CMake practices, improved project metadata, integratedjrl-cmakemodulesfor streamlined dependency management, and added options for documentation and testing.pythonsubdirectory in the build to avoid issues during the transition.These changes collectively streamline the build, test, and release process, making it easier to maintain and distribute the package across all major platforms.
References:
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]
Check the github workflow: https://github.com/hkmoon/b2/actions/runs/23671693162
Use https://pypi.org/manage/project/pybertini/release/1.0.9/ for testing.