util: Fix infinite loop in recursive function

When testing Xwayland multi-HiDPI support with Wine + SimCity4
I encountered a 100% CPU lockup from sway.  This turned out to be
triggering a bug in the wlroots pointer contraint code.

region_confine() contains multiple recursive calls where arguments
are modified and resubmitted to the function.  One of the calls
is however made using the original arguments, if/when this triggers
it results in the same codepath being followed each loop so the
condition always applies.

It makes much more sense if this was intended to apply the clamped
values x,y instead of the original x1,y1, and indeed this fixes the
infinite loop and results in correct behaviour.
This commit is contained in:
Steven Newbury 2022-02-08 12:02:51 +00:00 committed by Simon Ser
parent 078d8dd472
commit ad06c12c89
1 changed files with 1 additions and 1 deletions

View File

@ -214,7 +214,7 @@ static void region_confine(pixman_region32_t *region, double x1, double y1, doub
int y_ext = floor(y) + (dy == 0 ? 0 : dy > 0 ? 1 : -1);
if (pixman_region32_contains_point(region, x_ext, y_ext, &box)) {
return region_confine(region, x1, y1, x2, y2, x2_out, y2_out, box);
return region_confine(region, x, y, x2, y2, x2_out, y2_out, box);
} else if (dx == 0 || dy == 0) {
*x2_out = x;
*y2_out = y;