Optical calibration, bathymetry, water column correction and bottom typing of shallow marine areas, using passive remote sensing imageries

Busy? 4SM in 10 lines

4SM Help and HowTo



 
1 - NO NEED for field data, nor for atmospheric correction
2 - this is demonstrated in this website, using a variety of hyper/multi spectral data
 
Requirements are
1 - homogeneous water body and atmosphere
2 - some coverage of optically deep water
3 - some coverage of dry land
 
Problems are
1 - the precision on estimated depth is found wanting, because the noise-equivalent change in radiance  of accessible data is too high for shallow water column correction work 
2 - radiance data should be preprocessed by the provider at level 1 in order to improve S/N ratio
3 - exponential decay: the deeper/darker the bottom, the poorer the performances
 
So
I keep digging
until suitable data
become available
 
 
See you on 4SM blog         
HowTo Command Line


4SM COMMAND LINE ARGUMENTS:
get this help by entering "4SM...-h" at the prompt

STATUS: the environment arguments specify shallow water work environment
MODULE: the module arguments (one only enabled at a time) specify the action to be taken
Enable/Disable
  • DISABLE: to disable an argument set it to @-.....
    • -Calibrate.......... is enabled
    • -calibrate is disabled
    • @Calibrate is disabled
  • DISABLE: to disable a subargument set it to -...../@.../...
nice
  • nice -20 assigns a low priority to your process, so that you can do other things while modeling proceeds.
  • remove it or set nice to a higher value up to 0, if you want to expedite the process
    • this sucks more computing power, though


antislashes in the command line
nice -20  ./4SM.4.08 -Process/nosu  -Origin/Tarawa_Kiribati -DB/tarawa-subset/27_1_0_0/3_1/650_600/718.079_169.221/1_1 -Mis/Tarawa/Kiribati/Spot/XS/NA/UTM_19_008/0.02_0.02/01_JAN_1986 -MakePIX -Import/data*tarawa-full/dbnc_3_0_0_0/R1630_L1860/Origin_709.049_182.231_0_0/chIn1,3/chOut1,3/import.shp -AutoCAL/NIRmax255/mBPL4/NoSmooth
  • the commandline is a long string of arguments separated by spaces
    • this is a pain in the texteditor
  • in order to achieve a comfortable display,
    • split your command line as appropriate,
  • and place one antislash at the end of each sub-line: 
    • replace each @@ by one antislash then return
nice -20  ./4SM.4.08 -Process/nosu  -Origin/Tarawa_Kiribati @@
@URL/http://www.watercolumncorrection.com/ @@
-DB/tarawa-subset/27_1_0_0/3_1/650_600/718.079_169.221/1_1 @@
-Mis/Tarawa/Kiribati/Spot/XS/NA/UTM_19_008/0.02_0.02/01_JAN_1986 @@
-MakePIX @@
-Import/data*tarawa-full/dbnc_3_0_0_0/R1630_L1860/Origin_709.049_182.231_0_0/chIn1,3/chOut1,3/import.shp @@
-AutoCAL/NIRmax255/mBPL4/NoSmooth
                                                                                             
To get HELP
* Try it now *


use a '*' character to get help on any of the command line arguments,
like
.... -Process*
or
.... -Model* ....
or
.... -MakePIX*
 

Status arguments
-Ubuntu ..............to save a list of installed programs in /home/username/Software/Linux_software/dpkg/selections.txt
-Process...................
-DB.............................specifics of the PCIDSK database
-CZ.............................
specifics of the PCIDSK Combined Depth database
-Origin........................these_are_free_comments
-Mis............................image metadata
-URL..........................your URL address for mirror copy of the 4SM website on your machine
-Call..........................your favourite tools
-LK............................to provide a required commandline licensekey
-CK...........................to get a ciphered required commandline licensekey
-LE............................to get a free commandline licensekey for subset of a licensed image
-WL...........................Wavelengths
-CP...........................Calibration pixels on BPL curvBPL
-M.............................Bands to be used for modeling: usebj[c]
-LS..........................Saturation radiances
-LsT.........................Minimum radiance
-LsM.........................Radiance sensor Maximum: the normalization point
-Lsw..........................Radiance sensor deep water TOA
-Lw...........................Water volume reflectance: Lsw=La+Lg+Lw
-Lm..........................Radiance minimum thresholds BOA
-SCL........................Max_radiance for 16_to_8_bits scaling. Shall be multiplied by 10
-ST.........................Manual stretching factors for Flight_22
-KK........................Optical calibration parameters
-Z.............................Details for computing Z
-B.............................Details for computing B
-Smooth..................Smoothing options
-LL............................Options for screen display of message
-Tc ...........................Calibration using textfile image_cm0.txt describing the CurvBPL (LsM2 LsM1)
-Tp ...........................Calibration using textfile image_pm0.txt describing the PanBPL (LsM2 LsM1)

-E..............................Eub-window for modeling or extraction of data

-DD...........................Extraction pixel for displaying detailed modeling results
-D..............................Extraction series of pixels for displaying modeling results
-d..............................Extraction window for displaying modeling results

Module arguments 
-MakePIX......................................................create a database.pix with all required channels
-Import..........................................................import raw data 
-AutoCAL......................................................for a fully automatic processing
-Recode........................................................recode from InChannel to OutChannel 
-Deglint.........................................................Deglinter 
-Extract.........................................................Extractor 
-Calibrate......................................................Calibrator 
-Model....................................... ...................Modeler 
-RegressZZ.................................................. linear regression of depth results against seatruth depths
-ProfileZZ......................................................plot a profile of computed and sea-truth depth vs distance 
-ProfileAB.....................................................plot a profile of depth vs distance 
-Classify.......................................................bottom type classification
-DN................................................................extraction pixel's DN values 
-Archive........................................................archive the directory
-Zip................................................................prepare 4sm_tutorial...zip zipfile
OPTIONAL ARGUMENTS (don't bother, mostly outdated)
-VD (NDVI) ou -VP (PVI)............Vegetation index option and #band Green et Red
or -VR (RGB)...................Option for colorisation of dryland
or -VG.........................Dryland coded at 222 all over
-BSc1/c2/c3/c4/cred/cgreen/cblue/CoefBSC to generate imageBS enhanced c1ac2
(ocean at c3, land at c4); RGBbands for imageBSC
-D or d Calib: Screen display: D for detail, d for summary
-Drow/lin or Drow1/RowM/lin1/linN for extraction and modeling detailed display
-drow/lin or drow1/RowM/lin1/linN for extraction and modeling summary display
-DNrow/lin display spectral radiances of pixel then quit
-Erow1/RowM/line1/linN coordinates of extraction window
-ArowC/linC/altC/fov/scaleAP/dpi....Parameters of aerial viewing
-R ou RE to generate imageR
-RfileName.txt to compute linear regression from two-columns XY textfile
-XbB/minb/maxb/bG/ming/maxg/bR/minr/maxr Linearization/Enhancement==>CC of RGB bands
-XL Calibration: to read pixel selection from DataBase.gli instead of running a selection
-QLbR/bG/bB/nbCol[/minr/maxr/ming/maxg/minb/maxb] Make (Linear) RGB composite of bR, bG, bB
-VClabel1/.../labeln Compute vector Z_contour_lines using masks of ZC.pix (ch1=imageZ) -M(m)ap g/sP/950/2700/280/1000/ZoneA/14_BSC/18_Z/19_B.......(not operational).......Mapper
-MapG/sP/740.7/2703.2/0.2/0.1/Zone_G/14_HEB/15_Z/17_B/18_BSC.......................Mapper
-ForMap/A4L/BSC/Z/ZoneA.......................................................................options for Mapper
-Q(q)uanti /HER/3/2/1/10/14/255/0.7 options for Quanti (not operational)
-V(v)ector ..........................for vectorization of depth contour lines (not operational)

CODES for imageB 
1to 200 saturated at 201; 202 if Y12>Y12Max 


CODES for imageZ 
1to95 Z in decimeters (exp: 25=2.5 m)
110to199 Z in 100+Z meters (exp: 125=25 m)
211 foreshore uncovered by tide correction
212to219 non vegetated dryland pixels
220to229 vegetated dryland pixels
237 if Y12>Y12Max 


CODES for mask mSE 
1............shallow..........poor data, shall be excluded from calibration data
2............shallow..........quality data: default
3 to 20...shallow
..........quality data: available for ROIs (masks)

21..........mSOIL
22-23.....Glint targets
230.........Green vegetated or shallow forced at 216+brown
231.........darkGreen SHADOW on land forced at 231
232.........darkBlue wave breakers landside forced at 233
233.........lightBlue WAVE BREAKERS oceanside forced at 233
234.........White CLOUD/shadow forced at 234
235.........darkGreen lump of haze disables NIR band
236.........Wine dark lagoon bottom forced at LBref[2]
237.........Ochre non-vegetated forced at 219+green
238.........Salmon bright lagoon bottom forced at 238
239.........lightBlue NODATA forced at 239
240.........deep Blue DEEP WATER shallow area forced at 240
241.........Pink area excluded from volume statistics
242.........Blue LAKE/POOL on land forced at 242
243.........Violet vegetated or shallow forced at 212+grey
244.........Pink enable NEGATIVE-CONTRAST: sets Lmin at CN[i]
245.........Blue Lmin multiplies LMinprogressif by 3
246.........Yellow dry -> shallow forced at 0.1 metre
247.........Black AIRSTRIP forced at black 247
248.........Land area
249.........White Bad_Data forced at white 249
 

 

 

   How to Command Line

4SM Command Line Syntax


The 4SM command line is stored in an executable text file
which is called image_name.sh.
It must be made executable.
I use gedit in LINUX/GNOME as my text editor: "gedit myimage.sh &".
For a 3-bands image, see tarawa_subset.sh
For a 4-bands image, see tmnov.sh
For a 17 bands image, see mb4.sh
Structure of a 4SM command line
  • The command line script starts as follows: nice -20 ./4SM...
  • nice -20 is not really part of it: it is used to set a very low priority level. nice -0 would be an average level.
  • The command line is a long and complex character string which starts with the name of the executable code like ./4SM.7.11.
  • This string is composed of many arguments.
  • With the exception of the executable code name (e.g. 4SM) in the first argument, each following argument starts with a "-" character.
  • Each argument is a complex string of characters which is decoded at the start of the execution.
Law and order : layout of the Command Line
  • One argument per line is best
  • Arrange your display using antislashes "": see mb4.sh
Enabling/disabling an argument in the command line
  • To disable an argument, start it with a " @ " character.
    • @-Lsw66.00/17.00/11.00/006.0 this argument is disabled.
  • To enable it again, remove the "@" character.
    • -Lsw66.00/17.00/11.00/006.0 this argument is enabled.

Some arguments MUST not be disabled: they must start either by an Uppercase or by a Lowercase. If they are missing 4SM shall complain
  • -D(d)eglint
    • -Deglint... runs the Deglinter module,
    • -deglint...ensures that the glint parameters are available and shall be enforced.
    • @deglint... or @Deglint... disables deglinting altogether
  • -E(e)xtract
    • -Extract... runs the Extractor module,
    • -extract...ensures that the extraction parameters are available.
  • -M(m)odel
    • -Model... runs the Modeler module,
    • -model...ensures that the mask specification is available.
  • -C(c)lassify
    • -Classify... runs the Cassifier
    • -classify... provides the classification parameters


 
An argument overwrites the previous one
  • The following sequence -Lsw66.00/17.00/11.00/006.0 -Lsw67.00/16.00/11.00/006.0 will retain the second -Lsw67.00/16.00/11.00/006.0.
  • The only exception is the -CP argument: the following sequence -CP105.00/30.00/15.00/06.00 -CP082.00/21.00/12.00/06.00 (...) describes a sequence of two (or more) distinct calibration pixels.
Enabling/Disabling a command line in the script, and the exit instruction
  • The command line script starts as follows: nice -20 4SM...
  • A script may contain several command lines
  • All command lines in the same script which are enabled are executed in sequence
  • To prevent that, a command line can be disabled as follows: #nice -20 4SM...
  • To enable it again, remove the # .
  • Alternately, one can place an " exit " instruction as appropriate.
Several command lines in one single script
  • A bash script may contain an great number of command lines: see tarawa_subset.sh
  • Those commandlines which are enabled are executed in sequence, until an "exit" instruction is eventually encountered.