insar_tops_burst fails with Failed to connect to download.agisoft.com

[ZGHHGZ]

hyp3_isce2.insar_tops_burst, and all INSAR_ISCE_BURST jobs submitted through HyP3 since 2024-06-27T00:00:00Z, are failing with the following stacktrace during DEM preparation:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3_isce2/__main__.py", line 51, in <module>
    main()
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3_isce2/__main__.py", line 47, in main
    sys.exit(process_entry_point.load()())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3_isce2/insar_tops_burst.py", line 530, in main
    insar_tops_burst(
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3_isce2/insar_tops_burst.py", line 116, in insar_tops_burst
    dem_path = download_dem_for_isce2(dem_roi, dem_name='glo_30', dem_dir=dem_dir, buffer=0, resample_20m=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3_isce2/dem.py", line 112, in download_dem_for_isce2
    dem_array, dem_profile = dem_stitcher.stitch_dem(
                             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/dem_stitcher/stitcher.py", line 358, in stitch_dem
    dem_arr, dem_profile = merge_and_transform_dem_tiles(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/dem_stitcher/stitcher.py", line 197, in merge_and_transform_dem_tiles
    dem_arr = remove_geoid(
              ^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/dem_stitcher/geoid.py", line 72, in remove_geoid
    geoid_arr, geoid_profile = read_geoid(geoid_name, extent=list(extent), res_buffer=res_buffer)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/dem_stitcher/geoid.py", line 40, in read_geoid
    geoid_arr, geoid_profile = read_raster_from_window(geoid_path, extent, extent_crs, res_buffer=res_buffer)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/dem_stitcher/rio_window.py", line 158, in read_raster_from_window
    with rasterio.open(raster_path) as ds:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/rasterio/env.py", line 451, in wrapper
    return f(*args, **kwds)
           ^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/rasterio/__init__.py", line 304, in open
    dataset = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "rasterio/_base.pyx", line 310, in rasterio._base.DatasetBase.__init__
rasterio.errors.RasterioIOError: CURL error: Failed to connect to download.agisoft.com port 80 after 129539 ms: Couldn't connect to server

Complete log file: b887ed80-d679-4ab1-ad1e-f7ffb9180908.log

hyp3_isce2 relies on dem-stitcher, which retrieves Geoid models from agisoft.com. The URLs for these data files recently changed, causing processing to fail. The underlying issue is being tracked at ACCESS-Cloud-Based-InSAR/dem-stitcher#88

[asjohnston-asf]

@ZGHHGZ thanks for the report! I've updated the issue to fully describe the situation.

We hope to have a fix in place next week, but in the meantime INSAR_ISCE_BURST jobs submitted to HyP3 will continue to fail. INSAR_GAMMA jobs are unaffected and should process as normal.