Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: refactor onedal interaction with backend and policies #2168

Draft
wants to merge 39 commits into
base: main
Choose a base branch
from
Draft

chore: refactor onedal interaction with backend and policies #2168

wants to merge 39 commits into from

Conversation

ahuber21
Copy link
Contributor

@ahuber21 ahuber21 commented Nov 14, 2024 •
edited
Loading

Description

  • Refactor how and where methods are loaded from backend.
    • All functions are now explicitly declared as @abstractmethod and then populated using a decorator loaded from _backend.py.
    • It is now explicit where the functionality comes from.
    • We can start using the functions with self.infer(...) rather than the convoluted self._get_backend("X", "Y", "infer" , ...).
  • All dependencies from standard compute functions (compute(), infer(), ...) on backend are unified in _backend.py.
  • Backend wrapper class around the backend export in onedal/__init__.py to reduce the use of global variables.
  • More wrapper classes BackendManager, PolicyManager, BackendFunction that simplify interfacing with pybind/C++ backend functionality.
  • I removed unused imports because there were so many in the files I touched. I didn't stop at the files I modified and did the entire repo, which messed up the diff a little. Sorry. I am constantly rebasing so that all functionality is in a single commit and you can look at the diff of this single commit instead if you want to review.
    • Since this is becoming a quite complex PR, I reverted the unused imports changes. This will need its own PR.
  • Test update (WIP): Enables tests for iris and diabetes SVM algos on GPU. Updated dispatching seems to support GPU just fine. Feedback requested! @samir-nasibli
    • Confirmed missing implementation, I had a bug in my first version.

PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).

Checklist to comply with before moving PR from draft:

PR completeness and readability

  • I have reviewed my changes thoroughly before submitting this pull request.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have added a respective label(s) to PR if I have a permission for that.
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

Performance

  • I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
  • I have provided justification why performance has changed or why changes are not expected.
  • I have provided justification why quality metrics have changed or why changes are not expected.
  • I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

@ahuber21 ahuber21 force-pushed the dev/ahuber/refactor-onedal-backend-and-policies branch from 3e22a7e to 6e6b6ce Compare November 14, 2024 16:32
@icfaust
Copy link
Contributor

icfaust commented Nov 14, 2024

i guess this contrasts the approach i wanted to take in #2158 . what i had not liked about _get_backend was the indirection it was inducing in accessing the pybind11 generated package, and hoped to bring native syntax back. maybe worth further discussion?

@ahuber21
Copy link
Contributor Author

/intelci: run

@ahuber21 ahuber21 force-pushed the dev/ahuber/refactor-onedal-backend-and-policies branch from a3e1c8e to 0aa1669 Compare November 19, 2024 08:37
@ahuber21
Copy link
Contributor Author

/intelci: run

@ahuber21 ahuber21 marked this pull request as ready for review November 19, 2024 18:18
@ahuber21 ahuber21 force-pushed the dev/ahuber/refactor-onedal-backend-and-policies branch from 4c87853 to f0d2da7 Compare November 19, 2024 18:49
ethanglaser
ethanglaser reviewed Nov 19, 2024
View reviewed changes
Copy link
Contributor

@ethanglaser ethanglaser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial quick comments, will look more in depth later

examples/sklearnex/patch_sklearn.py Outdated Show resolved Hide resolved
daal4py/sklearn/ensemble/tests/test_decision_forest.py Outdated Show resolved Hide resolved
examples/sklearnex/dbscan_spmd.py Outdated Show resolved Hide resolved
examples/sklearnex/kmeans_spmd.py Outdated Show resolved Hide resolved
examples/sklearnex/knn_bf_classification_spmd.py Outdated Show resolved Hide resolved
examples/sklearnex/knn_bf_regression_spmd.py Outdated Show resolved Hide resolved
onedal/basic_statistics/incremental_basic_statistics.py Outdated Show resolved Hide resolved
onedal/spmd/ensemble/__init__.py Outdated
@@ -14,6 +14,6 @@
# limitations under the License.
# ==============================================================================

from .forest import RandomForestClassifier, RandomForestRegressor
from onedal.ensemble import RandomForestClassifier, RandomForestRegressor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from onedal.ensemble import RandomForestClassifier, RandomForestRegressor
from .ensemble import RandomForestClassifier, RandomForestRegressor

In general we have been using relative paths in cases like this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My branch deletes forest.py and we're using the batch version directly. After removing EstimatorBaseSPMD there is no need to create child classes from the batch version. This change gets rid of a lot of code that doesn't really do anything and only makes it harder to understand what's happening.

https://github.com/intel/scikit-learn-intelex/blob/main/onedal/spmd/ensemble/forest.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevertheless I have added relative imports consistently (...ensemble in this case)

onedal/basic_statistics/tests/test_basic_statistics.py Outdated Show resolved Hide resolved
@ahuber21 ahuber21 force-pushed the dev/ahuber/refactor-onedal-backend-and-policies branch from f0d2da7 to 79a3108 Compare November 20, 2024 08:46
@ahuber21 ahuber21 marked this pull request as draft November 20, 2024 08:47
@ahuber21 ahuber21 force-pushed the dev/ahuber/refactor-onedal-backend-and-policies branch from 79a3108 to a820e94 Compare November 20, 2024 08:57
@ahuber21
Copy link
Contributor Author

/intelci: run

@ahuber21 ahuber21 marked this pull request as ready for review November 20, 2024 11:09
@Alexsandruss
Copy link
Contributor

Check PR Checklist / Close all checkboxes before moving from draft (pull_request)

@Alexsandruss Alexsandruss marked this pull request as draft November 20, 2024 11:17
@ahuber21
Copy link
Contributor Author

Performance and accuracy are not affected
image

@ahuber21 ahuber21 requested a review from ethanglaser November 21, 2024 07:56
@ahuber21
Copy link
Contributor Author

ahuber21 commented Nov 21, 2024 •
edited
Loading

Reviewers, the performance regression in LogReg fit is reproducible and meaningful. I will investigate.

Performance degradation in LogReg is only compared to release (2025, 'P', 0), bot not main

image

--> It is not caused by this PR

@ahuber21 ahuber21 marked this pull request as ready for review November 21, 2024 10:55
@ahuber21 ahuber21 marked this pull request as draft November 21, 2024 10:56
@ahuber21
Copy link
Contributor Author

More checks required for SPMD changes, converting back to draft. But the non-SPMD contributions are still ready for review

@ahuber21
Copy link
Contributor Author

/intelci: run

1 similar comment
@ahuber21
Copy link
Contributor Author

/intelci: run

@ahuber21 ahuber21 force-pushed the dev/ahuber/refactor-onedal-backend-and-policies branch from 60cbc71 to 861ce67 Compare November 23, 2024 01:00
@ahuber21
fixup delete _policy.py; fix assert_all_finite from latest main
44ba5e0
@ahuber21
remove utils/__init__.py
1a7dadc
@ahuber21
Merge remote-tracking branch 'origin/main' into dev/ahuber/refactor-o…
c67fb22 
…nedal-backend-and-policies
@ahuber21
fix some errors after validation cleanup
81f8285
@ahuber21
compare only non-cpu devices
e040cdd
@ahuber21
fix after merging main
511b44f
@ahuber21
simplify SyclQueue
b39b852
@ahuber21
further simplify and align SyclQueue handling
a5cb819
@ahuber21
fix missing return
891700f
@ahuber21
remove intermediate SyclQueue class
a23ea0d
@ahuber21
introduce manage_global_queue context manager
c7eef38
@ahuber21
bring back underscore methods
50f6d1b
@ahuber21
kmeans init compute_raw does not support queue
d5e9425
@ahuber21
cleanup @support_input_format
35a344c
@ahuber21
allow for onedal prefix in patching check
7168aa2
@ahuber21
add missing wraps(func)
f0ca14b
@ahuber21
Merge remote-tracking branch 'origin/main' into dev/ahuber/refactor-o…
e304dae 
…nedal-backend-and-policies
@ahuber21
fixup
720e447
@ahuber21
fix neighbors
ff20f76
@ahuber21
Merge remote-tracking branch 'origin/main' into dev/ahuber/refactor-o…
554eb38 
…nedal-backend-and-policies
@ahuber21
debug print
6307b24
@ahuber21
debug output
e5d109c
@ahuber21
Add new logic for sparse matrix in _copy_to_usm
039d89e
@ahuber21
only _copy_to_usm with usm_iface
2f37e08
@ahuber21
lint
e40cb3f
@ahuber21
Merge remote-tracking branch 'origin/main' into dev/ahuber/refactor-o…
ab0dea4 
…nedal-backend-and-policies
@ahuber21
Copy link
Contributor Author

/intelci: run

@ahuber21
Copy link
Contributor Author

/intelci: run

@codecov Codecov
Copy link

codecov bot commented Dec 17, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 91.53226% with 63 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
onedal/neighbors/neighbors.py 87.27% 14 Missing ⚠️
onedal/linear_model/logistic_regression.py 66.66% 13 Missing ⚠️
sklearnex/_device_offload.py 64.86% 13 Missing ⚠️
onedal/_device_offload.py 89.18% 12 Missing ⚠️
onedal/__init__.py 85.71% 5 Missing ⚠️
sklearnex/linear_model/ridge.py 25.00% 3 Missing ⚠️
onedal/cluster/kmeans.py 97.72% 1 Missing ⚠️
onedal/ensemble/forest.py 98.18% 1 Missing ⚠️
onedal/svm/svm.py 98.48% 1 Missing ⚠️
Flag Coverage Δ
github 83.55% <91.53%> (+0.37%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
onedal/basic_statistics/basic_statistics.py 100.00% <100.00%> (ø)
...l/basic_statistics/incremental_basic_statistics.py 100.00% <100.00%> (ø)
onedal/cluster/dbscan.py 97.14% <100.00%> (-0.59%) ⬇️
onedal/cluster/kmeans_init.py 97.67% <100.00%> (+2.32%) ⬆️
onedal/common/hyperparameters.py 81.13% <100.00%> (ø)
onedal/covariance/covariance.py 97.36% <100.00%> (+0.22%) ⬆️
onedal/covariance/incremental_covariance.py 96.15% <100.00%> (+0.91%) ⬆️
onedal/datatypes/_data_conversion.py 87.50% <100.00%> (-0.74%) ⬇️
onedal/decomposition/incremental_pca.py 98.59% <100.00%> (+0.17%) ⬆️
onedal/decomposition/pca.py 98.18% <100.00%> (+0.06%) ⬆️
... and 32 more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Alexsandruss Alexsandruss Awaiting requested review from Alexsandruss Alexsandruss is a code owner

@samir-nasibli samir-nasibli Awaiting requested review from samir-nasibli samir-nasibli is a code owner

@icfaust icfaust Awaiting requested review from icfaust icfaust is a code owner

@maria-Petrova maria-Petrova Awaiting requested review from maria-Petrova

@napetrov napetrov Awaiting requested review from napetrov

@ethanglaser ethanglaser Awaiting requested review from ethanglaser ethanglaser is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ahuber21 @icfaust @Alexsandruss @ethanglaser