Calibrating ETM imagery: how it is done

return to main Caicos page

 
 
 
on Wavelength
"In the case of perfectly calibrated hyperspectral sensors,
the only cause of error spikes in atmospherically corrected spectra
would be a possible mis-evaluation of the gaseous concentrations in the atmospheric correction,
commonly H2O, but also O2 and CO2.
However, if an accurate water vapor retrieval is performed on a pixel-by-pixel basis,
the spikes are most likely caused by a systematic shift in the position of the channel center wavelengths
."
from "Spectral calibration of hyperspectral imagery using atmospheric absorption features"
Luis Guanter, Rudolf Richter, and José Moreno. APPLIED OPTICS, Vol. 45, No. 10 , 1 April 2006
©2006 Optical Society of America
extreme spectral shifts can reach 5 nm
the absorption features of atmospheric gases (e.g., 760 nm oxygen absorption, 940 nm water vapor absorption)

 on Kd

"Some optical classifications of water masses
use spectral distribution of downward diffuse attenuation coefficient kd as their basis.
This is possible because the kd averaged over thick subsurface layer of water
under combined illumination of the sun and sky (with solar zenith angles higher than 45°)
depends only on inherent optical properties of seawater."
from "Effective wavelength as a universal parameter of hyperspectral light radiance upwelling from the sea
"
Vladimir I. Haltrin, Robert A. Arnone, and Vyacheslav A. Urdenko
Ocean Optics:Remote Sensing and Underwater Imaging, Gary D. Gilbert, Robert J. Frouin, Editors,
Proceedings of SPIE Vol. 4488 (2002) © 2002 SPIE

 The parameters which need to be estimated are

 

  • Ki/Kj : ratio of effective two-ways attenuation coefficients for bands i and j
  • WL : effective spectral wavelengths
  • 2K : two-ways effective spectral diffuse attenuation coefficients
  • LsM : spectral TOA radiance of the brightest type of shallow bottom
  • Lsw : spectral TOA radiance of optically deep waters
  • Lw : spectral BOA radiance of optically deep waters: Path radiance La=Lsw-Lw

 

for ETM images of Caicos Bank

 

  • WLblue=477.6 nm suits all Caicos ETM images
  • this is mid-wavelength
  • WLgreen and WLred appear to depend on the particular image
  • they are definitevely not at mid-waveband
  •  
their choice is quite sensitive

 
For this systematic study, we shall use the seatruth dataset at step 1
For, this demo, we use ETM_009_045_2007-12-31


step 1 - Estimate Kgreen/Kblue
step 2 - Secure Kgreen
step 3 - Search for Kred
step 4 - Estimate RLBbg
then look after some more details:
 
 
Knir
Good fit of the BPL
Z2 vs Z3
Tide correction


 1- Estimate Kgreen/Kblue
  • First, Kgreen/Kblue must be secured
  • this is not allways straightforward!
    • Kgreen/Kblue~=0.46
 

 
Spectral K
  • Then spectral K must be estimated
    • wavelengths set at mid-waveband for now 
    • ==> WL 477.6 560.1 661.3 834.8
      • in units of nm
    • Kgreen/Kblue~=0.46
      • using Jerlov's data, this combination yields
      • K 0.079 0.172 0.815 4.500
      • in units of m-1
 2 - Secure Kgreen for this image
for Blue and Green bands only, using seatruth regression

 

step 1

  •   Then a seatruth regression is operated

    • only on bright botttom pixels
    • only using blue and green bands
  • computed depths are too deep

  • ==> 2Kgreen~=0.172 is too low

  • ==> regression slope=1.19

  • ==> WLgreen at mid-waveband is too low

  • we need to apply a suitable tide correction


 


 

step 2

 
  • WLgreen must be increased in order to increase 2Kgreen

    •  cWLgreen is increased in order to increase WLgreen
    • untill slope of regression equals 1
  • a satisfactory fit is observed for cWLgreen=0.67

  • this yields WLgreen=574 nm

  • == > 2Kgreen~=0.212 is convenient

  • ==> regression slope=1.00

  • ==> WLgreen=574 nm is convenient

  • a tide height=0.21 m relative to the seatruth is applied

 3 - Search for Kred for this image
using all three visible bands along a profile in the image

 

Two steps strategy: plotZ23
 

  • Using parameters secured so far,
  • 4SM is operated along a suitable profile in the image for pixels with bottom detection in all three Blue, Green and Red bands:
    • first, Z3 is computed using Blue, Green and Red bands
    • then, Z2 is computed using blue and green bands only (Lmred is set at 255)
  • Then 4SM computes the regression Z2 vs Z3
    • slope: cWLred shall be adjusted until the slope of this regression equals ~=1
    • offset: LsMgreen shall be adjusted until the offset of this regression equals ~=0




 

 

step 1: mid-waveband
  • Now that WLgreen=574 nm is secured for this image,
  • Then "seatruth" regression is operated
    • only on bright botttom pixels
    • only using 3 visible bands
    • along a suitable profile through the image
  • 2Kred~=0.833 is too high
    • ==> regression slope=1.11
    • ==> WLred =661 nm is too high for this image
step 2
  • Using cWLred, WLred is adjusted
    • till slope of regression equals 1
  • This yields WL[red]~=637 nm for cWLred=0.10
    • 2Kred~=0.709 is convenient
  • ==> regression slope=1.00
  • ==> WLred=637 nm is convenient for this image
 
 
effective values for this image
==> WL 477.6 573.9 636.7 834.8 in nm
==> 2K 0.098 0.212 0.709 4.500 in m-1
 


 4 - Estimate RLBbg

The assumption

 

  • Under the Soils Line assumption,
  • dark bottom pixels are assumed to be "spectrally neutral"
  • therefore, where the red band does not exhibit significant bottom detection, they shall be assigned LBgreen=LBblue
  • -Lw/0006.7/001.5/000.0/000.0_1.0
  • Therefore also, should they be actually green, their computed depth Z2 shall be underestimated
  • ==> the darker the bottom, the more its Z2 computed depth is underestimated (by up to 2 m here, potentially much more for much darker bottoms)
     













 

Actually for most shallow bottoms:
the darker, the greener

 

  • Depths Z2 computed for dark bottom pixels using only blue and green bands are badly underestimated
    • the darker the bottom, the more its computed depth is underestimated
  • ==> the "Soils Line assumption" does not hold
 
  • So we increase the slope of the Soils Line using the RLBbg parameter in the -Lw... argument
    • the darker, the greener: this amounts to assigning a green spectral signature, under control of average bottom reflectance B
    • -Lw/0006.7/001.5/000.0/000.0_1.15
    • a value of RLBbg=1.15 appears to be suitable along our black profile in this image
    • it amounts to stating that the slope of the Blue vs Green Soils Line is multiplied by 1.10
    • RLBbg commonly ranges from 1.10 to 1.20, probably depending on the seasonnal status of submerged vebetation: an average of 1.16+-0.036 is observed in the series of 23 ETM images at Caicos Bank

 

RLBbg=1.00
Z2 depth for darker bottomed pixels
is badly under-estimated:
the darker, the more under-estimated

 

RLBbg=1.15
achieves Z2~=Z3 for all pixels
This is best appreciated
by investigating the profiles below

 

 
 
Once RLBbg is deemed to be suitable
for a majority of pixels ==>
 
==> Then, the Lmred should be increased to ~=5
in order to avoid inconsistent depths
caused by a low S/N in the red band

 
5 - Knir
  • a value of Knir=4.5 m-1 is consistently suitable for 65% of the 23 images in the series of images at Caicos Bank
    • this may be used to help in improving spectral LsM
  • ==> quite often, a quasi-absolute calibration should be achievable, i.e. without the use of any field data



 

 

 

 

 

 

 

 

 


 

  • but, where this appears not to be the case, it is easy to see the cause:
    • in the Lsi vs Lsnir, the data are not deglinted
    • where the atmosphere is dense and/or the glint is strong, we seem to observe a genuine exponential decrease in the Lsi vs Lsnir, while it actually is a mirage caused by non-deglinted data
  • A realistic modeling is achievable without the use of seatruth data for the purpose of bottom typing, although computed depths must be multiplied by a CoefZ factor to be derived from seatruth data when it becomes available
 
 
 
Bad fit of the BPL vs Good fit of the BPL

 
Bad fit
  • It is tempting to consider that, as the BPL line is parallel to the straight part of the BPL pixels display, a satisfactory calibration has been achieved
  • NOT SO!
  • Something must be done: we want to slide down the LsM point a little

 

Good fit

  • All good calibrations in this series of 23 ETM images exhibit the good fit as illustrated below
  • LsM had to be reduced using the last variable in
    • -B/tclNe5.00/LBref200_100/Bmin0/cLM0.90
  • A good fit is now achieved
  • The X2 vs X3 BPL pixels commonly tend to bend up closer to the Y axis
  • I don't know why
  • could be because of the system noise.
 

Z2 vs Z3 in the calibration diagram

  • a good calibration should exhibit a nice fit between Z2 and Z3
    • slope=1
    • offset=0
    • for BPL pixels, which are assumed to be
      • brightest bottom
      • and spectrally neutral bottom signature














 
  • this calibration needs attention: =====>>>>>>>>>
  • slope is >>1
    • Z3 are underestimated because WLred is too high
    • ==> use cWLred to decrease WLred under control of the Z2 vs Z3 regression
    • ==> this shall decrease Kred and therefore increase Z3
  • offset is ~= 1 m
    • this is commonly because LsMgreen is too high
    • ==> LsMgreen must be decreased









 

 

Tide correction through ZCombiner 
 

  • 4SM now operate a relative tide correction:
    • it is a height above the "shallowest" image (meaning the image with lowest tide)
  • Run ZCombiner to estimate relative tide height
    • along the green profile tide.shp, where all depths are computed using all three visible bands over fairly bright bottoms:
    • ETM_009_045_1999-09-20 and ETM_009_045_2010-03-10 are found to be lowest tide;
    • tide at ETM_009_045_2008-12-17 is found to be 1.54 m higher than the above.
  • see ProfileAB tide_not_corrected
    • All images are rated relative to the lowest tide: see tide.txt
  • Now that a relative tide height is available for each image,
    • ZCombiner is run again so that Averaged Depth and Standard deviation images now are corrected for relative tide.
    • see ProfileAB tide_is_corrected
  • see final red profile



 

Final correction by the end user

 

  • The end-user now has to take care of tide datum
  • FinalZ=deltaZ+coefZ*Z: the end user still needs to use some seatruth in order to work out
  • offset deltaZ: a final tide correction to the desired datum
  • slope coefZ: a final depth correction factor


 

 

 This exercise has been quite demanding,
because of all the improvements that I had to introduce.

I am quite satisfied of the result.
Thanks a lot for your valuable contribution.
Although a few points are worth noting:
 
  • Effective wavelength for ETM imageries: this question is still pending
  • WLgreen : all Caicos ETM images require WLgreen to be increased from mid-waveband to WLgreen ~=568.2 nm
    • this I knew for quite some time
    • this is well established using the seatruth dataset
    • less clear waters would seem to require a stronger shift
  • WLred : effective wavelength is consistently lower than at mid-waveband,
    • ranging from 634 to 656 nm, for no obvious reason up to now
    • the strategy is:
      • set WLgreen to ~568.2 nm
      • then adjust WLred so that the BPL model is deemed to fit the BPL pixels
  • WLnir : Knir~=4.5 m-1 would appear to suit most images:
    • this is safe to use as a guide while calibrating
  • Lswgreen: most images require that Lswgreen be increased by one DN, some by up to two DNs.
    • And I still don't know why.
    • I have noted that for quite a long time using tmnov image and seatruth: it is now obvious.
    • This is only possible thanks to the seatruth dataset.
    • Whether this also applies to the Blue band remains to be seen (would hardly change anything noticeable though)
    • Whether it is restricted to TM/ETM images remains to be seen.
    • Lswi values
      • are based on the Lswnir value observed like in a cloud shadow,
      • and derived using the glint correlation
    • the -dLsw argument is used to increase Lswgreen
  • BPL assumption cannot be applied in a straightforward manner on some particular cases for the Blue_vs_Green linearized calibration plot: it has to be assumed
  • RLBbg: this trick obviously yields over-estimated Z2 depth for any pixel which actually has a purplish rather than greenish bottom signature
nice -20 4SM.4.06 -Process/producer/19_APR_2011 -OriginNPA
-DB/ETM_009_045_2007-12-31/31_1S_0_0/4_1/4018_4149/134.355_2443.245/1_1
-Mis/Bahamas/Caicos/Landsat/ETM/NPA/UTM_19_008/0.030_0.030/31_DEC_2007
-LS/0255.0/255.0/255.0/255.0
-cWL00.500/0.685/0.500/0.500
-cWL00.300/0.687/0.100/0.500
-M/@000001/00002/00003/00004
-LsM/199.6/194.2/225.1/207.8
-LsM/199.6/203.2/225.1/207.8
-Lsw/58.82/31.03/21.42/13.00
-Lw/0005.7/001.5/000.0/000.0_1.15
-Lm/0001.0/001.0/005.0/255.0
@Lm/0001.0/001.0/001.0/255.0
@Lm00001.0/001.0/255.0/255.0
-dLsw00000/001.5/00000/00000
-KK1_2_0.4600/Knir4.500/Bm400_BG500_GR600_RN700 /mask_3
@KK2_3_0.3300/Knir4.500/Bm400_BG500_GR600_RN700 /mask_3
-Z/MSL0.45/zplus/cSL1.18/ZMax0.00/cdZ1.00/PAN_1_4/sIG_0w0/N_1/cZ1.000
-Z/MSL0.00/zplus/cSL1.18/ZMax0.00/cdZ1.00/PAN_1_4/sIG_0w0/N_1/cZ1.000
-B/tclNe5.00/LBref200_100/Bmin0/cLM1
-deglint/vRbaD/DegTol1.00/F1/L4/GlintM200.0/FN/mDEGLINT1
-extract/v/rawBDH/NIRband4/NIRmax35.0/mBPL2/mSOIL21/FullBDH/@Ext_1_4018_1_4149
-calibrate/v/BdSNpZG/BDh_12_23_34
-Model/mask_2 @-Tc1/2
-Smooth/5/D/Smart+ @-E1/4018/1300/4149

@Classify/Typer10_0.2/Classifier_0.5_0_ef_LastChType2/mode_5_11_16
@RegressZZvp/..*depth_points_reproject_pruned/ZM/2_3/100_201/0_50_2.0/*
-ProfileAB/R/profile_black/2_3
-regressZZvP/profile_black/ZM/2_3/0_201/0_30_2.0/*
 
Afficher la suite de cette page



Créer un site
Créer un site