Syntax_AutoCal
to make special effect mask mSE automatically
Automatic Mode of 4SM: 
from raw data to first calibration and modeling

back to Help


 
1 - NO NEED for field data, nor for atmospheric correction
2 - this is demonstrated in this website, using a variety of hyper/multi spectral data
 
Requirements are
1 - homogeneous water body and atmosphere
2 - some coverage of optically deep water
3 - some coverage of dry land
 
Problems are
1 - the precision on estimated depth is found wanting, because the noise-equivalent change in radiance  of accessible data is too high for shallow water column correction work 
2 - radiance data should be preprocessed by the provider at level 1 in order to improve S/N ratio
3 - exponential decay: the deeper/darker the bottom, the poorer the performances
 
So
I keep digging
until suitable data
become available
 

About segmentation of the image
Automatic Mode of 4SM
Prerequisites
4SM Defaults
The -AutoCAL sequence of events 
HowTo MakeLand Automatic
How To Make Mask mSE
4SM FlowChart
How to Develop A GUI for 4SM



 

Syntax_AutoCAL
see also Syntax_MakePIX

 

-AutoCAL/NIRmax255/Land_-5.0_1.18_3s/mBPL3/GlintM25
-AutoCAL/R2_1.00/Land_-5_1.18_3s/mBPL4/GlintM10
shall use all default values
except those specified here, in random order.
AutoCAL values currently available
-AutoCAL/R2_0.9/print_2_3/MapBPL/MakeNewMask/Land_-5_1.2_7S/GetGlint/GlintM25.0/clouds        
/R(r)2_0.95 specifies R2KKmin to enable the KIKJ regressor: see below
/C(c)clouds MaskClouds   default=0     to mask clouds
/G(g)lint to deglint 
/G(g)lintM_25 GlintM=25.........................deglints in the range Lsnir=Lswnir to Lsnir=25
/GlintM_50........................deglints in the range Lsnir=Lswnir to Lsnir=50
/GlintM_255......................deglints in the range Lsnir=Lswnir to Lsnir=255
/GlintM_0..........................prevents any deglinting!!!!!                    
/P(p)rint_bi_bj         /printb........default printbi=0   to print details of KIKJ regression for the pair of bands bi vs bj
/M(m)apBPL /mapBPL........to map BPL pixels in database.pix             default mapBPL=1
/M(m)BPL_3

 
/mBPL3.....to collect BPL_data in the range  mBPL->mBPLmax    
                                    
 default  mBPL=2 and mBPLmax=20
/mBPL_1   to collect BPL data from mSE=1  up to mSE=20
/mBPL_3   to collect BPL data from mSE=3  up to mSE=20
/MBPL_3   to collect BPL data from mSE=3  up to mSE=3
/M(m)akeNewMask default=M      makeNewMask would keep existing mask SE unchanged
/L(l)and_-5_1.18_7S specifies the segmentation of land/non-land pixels
defaults....offsetLand=-5.....slopeLand=1.18.....chLand=bands,,,,,stopLand=S  
/G(g)T25 WaterMax specifies the segmentation of land/non-land pixels:
pixels with Ls[bands] GT WaterMax shall be mapped to Land
/N(n)oSmooth default=n      NoSmooth would prevent any smoothing
/NIRmax_25  NIRmax_5........collects BPL pixels in the range....Lsnir=Lswnir to Lsnir=Lswnir+5
NIRmax_25......collects BPL pixels in the range.... Lsnir=Lswnir to Lsnir=Lswnir+25
NIRmax_255....collects BPL pixels in the range.... Lsnir=Lswnir to Lsnir=255
G(g)etGLINT getGLINT.......default getGLINT=1   shall extract glint regressions                                                  Deglint=Make_GlintDatabase=getGLINT=1
/L(l)astGlint_3 default is NIR band number
/LastGlint_3........excludes the NIR band in a BGRN image
 
 








 

Image segmentation: creating special effect mask mSE
  • 4SM has an AutoCAL module to segment
    • all land areas into mSE=248
    • all marine areas to mSE=2
    • specific marine areas specified by relevant shapefiles recoded to the mask_NewVal value,
      • like to map at mSE=3 under mask_3.shp
  • How that worksbelow is an illustration in a bi-dimensional histogram of Lsred vs Lsnir
    • Land    below (or right of) a certain   straight line
    • Marine above (or left   of) that same straight line
  • That line is specified    .../Land_-5_1.18_3s/...
    • -AutoCAL/NIRmax255/Land_-5.0_1.18_3s/mBPL3/GlintM25  
 

Natural radiances, Red vs NIR
 

Natural radiances, Red vs NIR
 
 

Inspect mSE mask in channel_4
as created by the AutoCAL process


 
The Automatic Calibration process is an exciting development
  • which followed time intensive processing of a series of 12 Landsat ETM images in Tanzania, and many more, early in 2009
  • after 9 years, it now seems to work to satisfaction

4SM GUI
The AutoCAL feature paves the way for the preparation of a graphic user interface
Several aspects had to be addressed

  • Specify default values for most of the command line arguments
  • Automatic creation of desired database structure
  • Automatic import of raw data channels from original image
  • Automatic preparation of special effect mask mSE
  • Automatic specification of glint regression, if applicable
  • Automatic extraction of calibration data
  • Automatic preliminary calibration,
    • and, if allowed, deglinting and modeling
  • Wavelength specification
  • Preparation of a detailed command line, complete with a choice of supplementary tools
Shapefiles: the practioner needs to prepare
Lsw.shp.................for an area where to measure provisional spectral deep water value
dLsw.shp...............for an area where to fine-tune spectral deep water value
LsM.shp.................for an area where to measure provisional spectral LsM values
mSOIL.shp............for areas of bareland, from bright to as dark as possible

glint.shp.................for an area where to estimate glint correlations (if necessary)
vegetation.shp........for an area of healthy vegetation
mask_1.shp, mask_2.shp, mask_3.shp............as required
profiles ......................as required
Sub-windows............as required E1.shp    E2.shp    etc
 


Automatic Mode of 4SM

the next step after importing the raw data:
from raw data to a preliminary calibration (and deglinting/modeling)
 


#LINE_1: build the database, import data, create mask_SE in channel_4, run the AutoCalibration
nice -20  ./4SM.8.07 -Process/nosu  -Origin/Tarawa_Kiribati                                      @@
-DB/tarawa-subset/27_1s_0s_0s/3_1/650_600/718.069_169.231                              @@
-Mis/Tarawa/Kiribati/Spot/XS/NA/bOA/UTM_19_008/0.02_0.02/01_JAN_1986             @@
-MakePIX/PISA                                                                                                        @@
-Import/v/dTM/data*tarawa-full/dbnc_3_0s_0s_0s/R1630_L1860/Origin_709.049_182.231_0_0/chIn1,3/chOut1,3/import.shp @@
-AutoCAL/Land_-5.0_1.18_3s/mBPL3/GlintM10                             @@
 
 


Prerequisites

The AutoCAL process won't work unless

  • LsM.shp: at least a few non-saturated pixels of bright land may be sampled using the LsM.shp shapefile
  • Lsw.shp: an area of optically deep water pixels may be sampled using a Lsw.shp shapefile
    • optically deep is to mean : optically deep in all wavebands
  • glint.shp: if conspicuous clouds or glint is observed over optically deep water in the image,
    • an area of glinted pixels may be sampled using a glint.shp shapefile
Furthermore
  • bright and homogeneous bottoms are expected to exist at various depths
  • suitable default values are available and applicable

4SM defaults:  text tile 4sm.def
  • Apart from image-specific metadata which need to be specified in -DB..., -Mis... and -Import... arguments,
    • many commandline argument variables are given a default value in the 4SM code
    • any such default value can be overridden in the command line itself
  • Upon the first execution of 4SM in the current directory,
    • 4SM writes a 4sm.def editable textfile to contain such default values
    • default values are written or read at the end of decoding the -Mis... argument
    • default values are then eventually overridden upon reading the rest of the commandline

The -AutoCAL sequence of events
- to produce a special effect mask mSE in channel bands+1
- and perform a provisional optical calibration
  • Either purge the work directory if -MakePIX/PISA has been specified
    • ?then creates the database, import raw data, scale to 8 bits
  • Or erase existing mask mSE in channel bands+1
    • ?then write a new one
  • Recode
    • NoData........recode to 239 in mask mSE all pixels which read 0 in any of the raw data bands
    • baddata.......recode to 239 in mask mSE all pixels which are included in shapefile baddata.shp
    • cloud...........recode to 234 in mask mSE all pixels which are included in shapefile cloud.shp
    • wave...........recode to 233 in mask mSE all pixels which are included in shapefile wave.shp
  • Lsw and LsM
    • Lsw.............if a -Lsw... argument is not provided in the commandline,
    • 4SM reads shapefile Lsw.shp to compute and specify provisional spectral Lsw
  • LsM.............if a -LsM... argument is not provided in the commandline,
    • 4SM reads shapefile LsM.shp to compute and specify provisional spectral LsM
  • Glint regression
    • if a shapefile named glint.shp is available, 4SM reads it
    • to specify the glint regressions correlations
    • to specify the -Lsw... argument
  • Make Land, Marine, Wave  and Cloud....
    • A provisional Soil Line is defined using provisional LsM and Lsw values in the Red and Nir bands
    • The image is then segmented as explained elsewhere
      • recode land pixels       to   248
      • recode shallow pixels  to   2
      • recode cloud pixels     to   234 (if a cloud.shp shapefile is not available)
      • recode wave breakers to   233 (if a wave.shp shapefile is available)
  • Recoder
    • Clean the land area
      • land.shp: recode to 248 pixels which are included in shapefile land.shp
    • Clean the marine area
      • shallow.shp............recode to 2 all pixels which are included in shapefile shallow.shp
      • badwaters.shp..................recode to 249 all pixels which are included in shapefile badwaters.shp
      • deep.shp..........................recode to 249 all pixels which are included in shapefile deep.shp
      • mask_1.shp......................recode to 1 all pixels which are included in shapefile mask_1.shp
      • mask_3.shp......................recode to 3 all pixels which are included in shapefile mask_3.shp
      • mask_4.shp......................recode to 4 all pixels which are included in shapefile mask_4.shp
      • mask_5.shp......................recode to 5 all pixels which are included in shapefile mask_5.shp
      • etc, up to mask_20.shp
      • mask_20.shp....................recode to 20 all pixels which are included in shapefile mask_20.shp
      • negative.shp.....................recode to 244 areas where negative bottom contrast is allowed
  • Extractor:
    • if a suitable calibration textfile is not available, 4SM runs the extraction

  • KIKJ Regressor: only works for most favorable images
    • estimates the ratio Kblue/Kgreen by regression on the linearized BPL pixels
      • and  use the linearized BPL, the optical calibration, and the provisional Soil Line to specify final LsM
    • if the coefficient of determination is too low (R2<R2KKmin),a default value is enforced:
      • KIKJ=Kblue/Kgreen=0.50
        • or
      • KIKJ=Kgreen/Kred=0.25 for SPOT 1 images
  • Calibrator
    • All parameters are now specified
    • 4SM proceeds to compute and display the calibration plot
  • AutoCAL.txt:
    • ?4SM writes and displays a textfile named autoCAL.txt
      • which contains, among other things, a complete detailed commandline
      • which may in turn be used for the practioner to take full control of all aspects of the 4SM process