# Integrating soundings data (xyz files)

Another type of data commonly encountered when working with bathymetric data is survey data and/or data from nautical charts. This data may be provided in an ASCII xyz file.
In the data’ folder, you will find the files S202001000.xyz and S202001000_head.xyz (the same file without the header). This data was downloaded from the SHOM website and was recorded using a multibeam echo sounder.

For this data integration, we will use [mapxyz-unstructured](../../grid-topography-processing/mapxyz-unstructured.md) and [topo-merge](../../grid-topography-processing/topo-merge.md), in 3 steps.

*TIPS: Creating topography is a long an iterative process so we recommand to create dedicated directory (for example 2-integrating_xyz/)*

## Step 1: Decimation
Multibeam sounding data often have very high resolution. Decimation may be necessary before integrating them into the existing bathymetry. The goal is to reduce the resolution of the dataset to approximately 30 meters prior to integration. The decimation algorithm allows us to preserve the quality of the dataset while improving efficiency.

```bash
rep=/ecola/share/tmp/training/tuto_bathy/data/
mapxyz-unstructured $rep/S202001000_head.xyz \
                    -resolution 0.03 \
                    --decimate-only
```

Options used here (see [mapxyz-unstructured](../../grid-topography-processing/mapxyz-unstructured.md) for more):

| Name | Description |
|------|-------------|
| -resolution | output resolution for decimation (in km by default and can handle km, m, deg, mn, arcsec units.) |
| --decimate-only | stop the processing after decimation |


This can take a long time (*e.g.* 40 min on a single processor), so we recommand to use multiple CPUs. In the tools-XXXXX.out files, you can see the start of the processing. 
The expected result is a file named ‘mapxyz-unstructured-S202001000_head-reduced.xyz’ (also available at ‘/ecola/share/tmp/training/tuto_bathy/tuto_files/’).

## Step 2: Integration
For this step, we will use [mapxyz-unstructured](../../grid-topography-processing/mapxyz-unstructured.md) again, but with different options:

```bash
rep=/ecola/share/tmp/training/tuto_bathy/
mapxyz-unstructured $rep/tuto_files/mapxyz-unstructured-S202001000_head-reduced.xyz \ 
                    -scale -1. \
                    --no-decimation \
                    -e ../1-merging_DEMs/GEBCO5arcs_MNTAtlSHOM.nc \
                    -b ../1-merging_DEMs/GEBCO5arcs_MNTAtlSHOM.nc \
                    --pbma $rep/data/2000-2019-range.nc
```

Options used here (see [mapxyz-unstructured](../../grid-topography-processing/mapxyz-unstructured.md) for more):

| Name | Description |
|------|-------------|
| -scale | to switch positive data values to negative (and vice versa) |
| --no-decimation | no decimation needed at this step |
| -e | topography reference, to avoid interpolation of bathymetric data over/around headlands for example |
| -b | base bathymetry in which the data are integrated |
| --pbma | file to apply correction for lowest astronomical tides. In this case, the data is referenced to Lowest Astronomic Tides (LAT). We apply a correction to reference the data to mean sea level. |

After this first integration step, a ‘mapxyz-unstructured.nc’ bathymetry file is generated, with a **default resolution of 1 arc-second**, and whose extent is defined by the extent of the dataset that was integrated. At this stage, a visual check is (strongly) recommended.

## Step 3: Merging
We will now integrate this “small” DEM in our regional bathymetry, using [topo-merge](../../grid-topography-processing/topo-merge.md) as we did before in this tutorial.

```bash
topo-merge mapxyz-unstructured.nc \
           -b ../1-merging_DEMs/GEBCO5arcs_MNTAtlSHOM.nc \
           -f netcdf \
           -o GEBCO5arcs_MNTAtlSHOM_S202001000
```