[Backport v3.6-branch] drivers: sensor: ams_as5600: Fix calculation of fractional part #70145
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
commit 98903d4 upstream.
The original calculation has two bugs. One is the calculated value, and the other is that the value is not in one-millionth parts.
What the original calculation does is compute a scaled position value by multiplying the raw sensor value (
dev_data->position
) byAS5600_FULL_ANGLE
, which represents a full rotation in degrees. It then subtracts the product of the whole number of pulses (val->val1
) andAS5600_PULSES_PER_REV
from this scaled position value.What you actually need is to extract the fractional part of the value by taking the modulo of AS5600_PULSES_PER_REV from the scaled value of the position.
(((int32_t)dev_data->position * AS5600_FULL_ANGLE)
% AS5600_PULSES_PER_REV)
Then convert the value to one-millionth part.
Fixes #70142