Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regrid: force crop to avoid going out of memory #3517

Closed
wants to merge 1 commit into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion spark/src/main/scala/geotrellis/spark/regrid/Regrid.scala
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,12 @@ object Regrid {

val xSpan: Interval[Long] = oldXrange intersect newXrange
val ySpan: Interval[Long] = oldYrange intersect newYrange
val forceCrop = newW < 0.6*oldW || newH < 0.6*oldH
newKey ->
(oldTile.crop((xSpan.start - oldXstart).toInt,
(ySpan.start - oldYstart).toInt,
(xSpan.end - oldXstart).toInt,
(ySpan.end - oldYstart).toInt),
(ySpan.end - oldYstart).toInt),Crop.Options(force=forceCrop)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops CI feels a bit unhappy about it. (2.12)

/usr/local/src/spark/src/main/scala/geotrellis/spark/regrid/Regrid.scala:126:92: ';' expected but ',' found         
(ySpan.end - oldYstart).toInt),Crop.Options(force=forceCrop)),

((xSpan.start - newXrange.start).toInt, (ySpan.start - newYrange.start).toInt)
)
Comment on lines +121 to 128
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not allow me to add a code change suggestion, but should be smth like:

              val forceCrop = newW < 0.6 * oldW || newH < 0.6 * oldH
              newKey ->
                (
                  oldTile.crop(
                    (xSpan.start - oldXstart).toInt,
                    (ySpan.start - oldYstart).toInt,
                    (xSpan.end - oldXstart).toInt,
                    (ySpan.end - oldYstart).toInt,
                    Crop.Options(force = forceCrop)
                  ),
                  ((xSpan.start - newXrange.start).toInt, (ySpan.start - newYrange.start).toInt)
                )

Crop options is an argument to the crop function.

Copy link
Member

@pomadchin pomadchin Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made up a #3518 to try out the CI, but please feel free to commit changes as a part of this PR as well.

U can also let me pushing into your PR to speed up the progress! (checkbox)

image

}
Expand Down
Loading