GTiff 래스터의 시계열 스택을 단일 NetCDF로 변환

gdal-dev 메일 링리스트에서 이동 :

David Shean은 2013 년 9 월 2 일 월요일 오후 7:09에 다음과 같이 썼습니다.

안녕하세요, 배포 용 단일 NetCDF 파일과 동일한 투영 / 범위 / 해상도로 GTiff 래스터의 시계열을 패키지하려고합니다. 지난 몇 시간 동안 온라인 문서를 참조하고 gdal_translate, gdalbuildvrt 및 gdalwarp를 성공적으로 사용했습니다.

기존 gdal 명령 행 유틸리티를 사용하여이를 수행하는 쉬운 방법이 있습니까? NetCDF Python API를 사용하여 사용자 지정 솔루션을 사용하기 전에 물어볼 것을 알았습니다.

감사. 데이비드

Etienne Tourigny는 2013 년 9 월 3 일 화요일 오전 10시 15 분에 다음과 같이 썼습니다.

당신이 원하는 것은 아마도 gdal의 범위를 벗어날 것입니다. gdal_translate가 단일 파일에 넣도록 영리한 메타 데이터 관리가 필요합니다 ...

gdal_translate를 사용하여 모두 netcdf로 변환 한 다음 python-netcdf4 (numpy / scipy가 아닌)를 사용하여 시간 차원으로 스택하는 것이 좋습니다.

2013 년 9 월 3 일 화요일 오전 7:55에 "Signell, Richard"는 다음과 같이 썼습니다.

David, GIS stackexchange 그룹 /gis// 에 질문을 게시하면 도움이되는 예제 코드를 제공하겠습니다.

-풍부한

=====================

업데이트 9/3/13 17:04 PDT

다음은 입력 데이터 세트 중 하나에 대한 gdalinfo 출력입니다.

gdalinfo 20120901T2024_align_x+22.19_y+3.68_z+14.97_warp.tif

Driver: GTiff/GeoTIFF
Files: 20120901T2024_align_x+22.19_y+3.68_z+14.97_warp.tif
Size is 10666, 13387
Coordinate System is:
PROJCS["unnamed",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Polar_Stereographic"],
    PARAMETER["latitude_of_origin",70],
    PARAMETER["central_meridian",-45],
    PARAMETER["scale_factor",1],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]]]
Origin = (-211346.063781524338992,-2245136.291794800199568)
Pixel Size = (5.000000000000000,-5.000000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  ( -211346.064,-2245136.292) ( 50d22'39.70"W, 69d23'55.59"N)
Lower Left  ( -211346.064,-2312071.292) ( 50d13'22.38"W, 68d48'10.75"N)
Upper Right ( -158016.064,-2245136.292) ( 49d 1'33.33"W, 69d26'16.42"N)
Lower Right ( -158016.064,-2312071.292) ( 48d54'35.06"W, 68d50'27.28"N)
Center      ( -184681.064,-2278603.792) ( 49d38' 1.32"W, 69d 7'17.04"N)
Band 1 Block=256x256 Type=Float32, ColorInterp=Gray
  NoData Value=-32767

Luke의 제안 된 접근 방식에 대한 후속 조치.

vrt 생성은 잘 작동합니다.

gdalbuildvrt -separate newtest.vrt *warp.tif

<VRTDataset rasterXSize="10666" rasterYSize="13387">
  <SRS>PROJCS["unnamed",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Polar_Stereographic"],PARAMETER["latitude_of_origin",70],PARAMETER["central_meridian",-45],PARAMETER["scale_factor",1],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]]]</SRS>
  <GeoTransform> -2.1134606378152434e+05,  5.0000000000000000e+00,  0.0000000000000000e+00, -2.2451362917948002e+06,  0.0000000000000000e+00, -5.0000000000000000e+00</GeoTransform>
  <VRTRasterBand dataType="Float32" band="1">
    <NoDataValue>-3.27670000000000E+04</NoDataValue>
    <ComplexSource>
      <SourceFilename relativeToVRT="1">20110619T2024_align_x+15.51_y+1.15_z+12.10_warp.tif</SourceFilename>
      <SourceBand>1</SourceBand>
      <SourceProperties RasterXSize="10666" RasterYSize="13387" DataType="Float32" BlockXSize="256" BlockYSize="256" />
      <SrcRect xOff="0" yOff="0" xSize="10666" ySize="13387" />
      <DstRect xOff="0" yOff="0" xSize="10666" ySize="13387" />
      <NODATA>-32767</NODATA>
    </ComplexSource>
  </VRTRasterBand>
  <VRTRasterBand dataType="Float32" band="2">
    <NoDataValue>-3.27670000000000E+04</NoDataValue>
    <ComplexSource>
      <SourceFilename relativeToVRT="1">20110802T2024_align_x+16.33_y+2.14_z+12.02_warp.tif</SourceFilename>
      <SourceBand>1</SourceBand>
      <SourceProperties RasterXSize="10666" RasterYSize="13387" DataType="Float32" BlockXSize="256" BlockYSize="256" />
      <SrcRect xOff="0" yOff="0" xSize="10666" ySize="13387" />
      <DstRect xOff="0" yOff="0" xSize="10666" ySize="13387" />
      <NODATA>-32767</NODATA>
    </ComplexSource>
  </VRTRasterBand>
...

그러나 nc로 번역하려고하면 다음 오류가 발생합니다.

gdal_translate -of netcdf newtest.vrt newtest.nc

Input file size is 10666, 13387
Warning 1: Variable has 0 dimension(s) - not supported.
0...10...20...30...40...50ERROR 1: netcdf error #-62 : NetCDF: One or more variable sizes violate format constraints .
at (netcdfdataset.cpp,SetDefineMode,1574)

ERROR 1: netcdf error #-39 : NetCDF: Operation not allowed in define mode .
at (netcdfdataset.cpp,IWriteBlock,1435)

ERROR 1: netCDF scanline write failed: NetCDF: Operation not allowed in define mode
ERROR 1: An error occured while writing a dirty block
...ERROR 1: netcdf error #-39 : NetCDF: Operation not allowed in define mode .
at (netcdfdataset.cpp,IWriteBlock,1435)

ERROR 1: netCDF scanline write failed: NetCDF: Operation not allowed in define mode
ERROR 1: netcdf error #-62 : NetCDF: One or more variable sizes violate format constraints .
at (netcdfdataset.cpp,~netCDFDataset,1548)

따라서 면밀히 살펴보면 사용중인 극좌표 입체 투영법에 불만이있는 것으로 보입니다 (EPSG : 3413). netcdfdataset.cpp의 1570-1582 행을 참조하십시오.

https://code.vpac.org/gitorious/gdal-netcdf-testing/gdal-netcdf-driver/blobs/8fa3582669969ad4d55e461f5846b3ed33727f63/gdal/frmts/netcdf/netcdfdataset.cpp

내 투영에 latitude_of_origin이 지정되었지만 netcdf 드라이버에서 예상 한 표준 평행선이 없습니다.

다음은 원하는 시간에 데이터를 나타내는 GDAL 파일을 읽고 CF를 준수 하는 단일 NetCDF 파일에 쓰는 파이썬 코드입니다.

#!/usr/bin/env python
'''
Convert a bunch of GDAL readable grids to a NetCDF Time Series.
Here we read a bunch of files that have names like:
/usgs/data0/prism/1890-1899/us_tmin_1895.01
/usgs/data0/prism/1890-1899/us_tmin_1895.02
...
/usgs/data0/prism/1890-1899/us_tmin_1895.12
'''

import numpy as np
import datetime as dt
import os
import gdal
import netCDF4
import re

ds = gdal.Open('/usgs/data0/prism/1890-1899/us_tmin_1895.01')
a = ds.ReadAsArray()
nlat,nlon = np.shape(a)

b = ds.GetGeoTransform() #bbox, interval
lon = np.arange(nlon)*b[1]+b[0]
lat = np.arange(nlat)*b[5]+b[3]


basedate = dt.datetime(1858,11,17,0,0,0)

# create NetCDF file
nco = netCDF4.Dataset('time_series.nc','w',clobber=True)

# chunking is optional, but can improve access a lot: 
# (see: http://www.unidata.ucar.edu/blogs/developer/entry/chunking_data_choosing_shapes)
chunk_lon=16
chunk_lat=16
chunk_time=12

# create dimensions, variables and attributes:
nco.createDimension('lon',nlon)
nco.createDimension('lat',nlat)
nco.createDimension('time',None)
timeo = nco.createVariable('time','f4',('time'))
timeo.units = 'days since 1858-11-17 00:00:00'
timeo.standard_name = 'time'

lono = nco.createVariable('lon','f4',('lon'))
lono.units = 'degrees_east'
lono.standard_name = 'longitude'

lato = nco.createVariable('lat','f4',('lat'))
lato.units = 'degrees_north'
lato.standard_name = 'latitude'

# create container variable for CRS: lon/lat WGS84 datum
crso = nco.createVariable('crs','i4')
csro.long_name = 'Lon/Lat Coords in WGS84'
crso.grid_mapping_name='latitude_longitude'
crso.longitude_of_prime_meridian = 0.0
crso.semi_major_axis = 6378137.0
crso.inverse_flattening = 298.257223563

# create short integer variable for temperature data, with chunking
tmno = nco.createVariable('tmn', 'i2',  ('time', 'lat', 'lon'), 
   zlib=True,chunksizes=[chunk_time,chunk_lat,chunk_lon],fill_value=-9999)
tmno.units = 'degC'
tmno.scale_factor = 0.01
tmno.add_offset = 0.00
tmno.long_name = 'minimum monthly temperature'
tmno.standard_name = 'air_temperature'
tmno.grid_mapping = 'crs'
tmno.set_auto_maskandscale(False)

nco.Conventions='CF-1.6'

#write lon,lat
lono[:]=lon
lato[:]=lat

pat = re.compile('us_tmin_[0-9]{4}\.[0-9]{2}')
itime=0

#step through data, writing time and data to NetCDF
for root, dirs, files in os.walk('/usgs/data0/prism/1890-1899/'):
    dirs.sort()
    files.sort()
    for f in files:
        if re.match(pat,f):
            # read the time values by parsing the filename
            year=int(f[8:12])
            mon=int(f[13:15])
            date=dt.datetime(year,mon,1,0,0,0)
            print(date)
            dtime=(date-basedate).total_seconds()/86400.
            timeo[itime]=dtime
           # min temp
            tmn_path = os.path.join(root,f)
            print(tmn_path)
            tmn=gdal.Open(tmn_path)
            a=tmn.ReadAsArray()  #data
            tmno[itime,:,:]=a
            itime=itime+1

nco.close()

GDAL 및 NetCDF4 Python은 빌드하기가 다소 어려울 수 있지만 좋은 소식은 대부분의 과학적인 Python 배포판 (Python (x, y), Enthought Python Distribution, Anaconda, ...)의 일부라는 것입니다.

업데이트 : CF 호환 NetCDF에서 극좌표 스테레오 그래픽을 아직 수행하지 않았지만 다음과 같이 보입니다. 저는 여기에 있다고 가정했습니다 central_meridian과 latitude_of_origin같은과 같다 GDAL에 straight_vertical_longitude_from_pole와 latitude_of_projection_originCF에서 :

#!/usr/bin/env python
'''
Convert a bunch of GDAL readable grids to a NetCDF Time Series.
Here we read a bunch of files that have names like:
/usgs/data0/prism/1890-1899/us_tmin_1895.01
/usgs/data0/prism/1890-1899/us_tmin_1895.02
...
/usgs/data0/prism/1890-1899/us_tmin_1895.12
'''

import numpy as np
import datetime as dt
import os
import gdal
import netCDF4
import re

ds = gdal.Open('/usgs/data0/prism/1890-1899/us_tmin_1895.01')
a = ds.ReadAsArray()
ny,nx = np.shape(a)

b = ds.GetGeoTransform() #bbox, interval
x = np.arange(nx)*b[1]+b[0]
y = np.arange(ny)*b[5]+b[3]


basedate = dt.datetime(1858,11,17,0,0,0)

# create NetCDF file
nco = netCDF4.Dataset('time_series.nc','w',clobber=True)

# chunking is optional, but can improve access a lot: 
# (see: http://www.unidata.ucar.edu/blogs/developer/entry/chunking_data_choosing_shapes)
chunk_x=16
chunk_y=16
chunk_time=12

# create dimensions, variables and attributes:
nco.createDimension('x',nx)
nco.createDimension('y',ny)
nco.createDimension('time',None)
timeo = nco.createVariable('time','f4',('time'))
timeo.units = 'days since 1858-11-17 00:00:00'
timeo.standard_name = 'time'

xo = nco.createVariable('x','f4',('x'))
xo.units = 'm'
xo.standard_name = 'projection_x_coordinate'

yo = nco.createVariable('y','f4',('y'))
yo.units = 'm'
yo.standard_name = 'projection_y_coordinate'

# create container variable for CRS: x/y WGS84 datum
crso = nco.createVariable('crs','i4')
crso.grid_mapping_name='polar_stereographic'
crso.straight_vertical_longitude_from_pole = -45.
crso.latitude_of_projection_origin = 70.
crso.scale_factor_at_projection_origin = 1.0
crso.false_easting = 0.0
crso.false_northing = 0.0
crso.semi_major_axis = 6378137.0
crso.inverse_flattening = 298.257223563

# create short integer variable for temperature data, with chunking
tmno = nco.createVariable('tmn', 'i2',  ('time', 'y', 'x'), 
   zlib=True,chunksizes=[chunk_time,chunk_y,chunk_x],fill_value=-9999)
tmno.units = 'degC'
tmno.scale_factor = 0.01
tmno.add_offset = 0.00
tmno.long_name = 'minimum monthly temperature'
tmno.standard_name = 'air_temperature'
tmno.grid_mapping = 'crs'
tmno.set_auto_maskandscale(False)

nco.Conventions='CF-1.6'

#write x,y
xo[:]=x
yo[:]=y

pat = re.compile('us_tmin_[0-9]{4}\.[0-9]{2}')
itime=0

#step through data, writing time and data to NetCDF
for root, dirs, files in os.walk('/usgs/data0/prism/1890-1899/'):
    dirs.sort()
    files.sort()
    for f in files:
        if re.match(pat,f):
            # read the time values by parsing the filename
            year=int(f[8:12])
            mon=int(f[13:15])
            date=dt.datetime(year,mon,1,0,0,0)
            print(date)
            dtime=(date-basedate).total_seconds()/86400.
            timeo[itime]=dtime
           # min temp
            tmn_path = os.path.join(root,f)
            print(tmn_path)
            tmn=gdal.Open(tmn_path)
            a=tmn.ReadAsArray()  #data
            tmno[itime,:,:]=a
            itime=itime+1

nco.close()

아래 예와 같이 GDAL 유틸리티 를 사용하여 단일 NetCDF 에 쉽게 넣을 수 있습니다. 그러나 @RichSignell의 답변의 시간 차원 / 기타 메타 데이터는 얻지 못합니다. tiff는 방금 서브 데이터 세트에 덤프됩니다.

C:\remotesensing\testdata>dir /b ndvi*.tif
ndvi1.tif
ndvi2.tif
ndvi3.tif

C:\remotesensing\testdata>gdalbuildvrt -separate ndvi.vrt ndvi*.tif
0...10...20...30...40...50...60...70...80...90...100 - done.

C:\remotesensing\testdata>gdal_translate -of netcdf ndvi.vrt ndvi.nc
Input file size is 96, 88
0...10...20...30...40...50...60...70...80...90...100 - done.

C:\remotesensing\testdata>gdalinfo ndvi.nc
Driver: netCDF/Network Common Data Format
Files: ndvi.nc
Size is 512, 512
Coordinate System is `'
Metadata:
  NC_GLOBAL#Conventions=CF-1.5
  NC_GLOBAL#GDAL=GDAL 1.10.0, released 2013/04/24
  NC_GLOBAL#history=Wed Sep 04 09:49:11 2013: GDAL CreateCopy( ndvi.nc, ... )
Subdatasets:
  SUBDATASET_1_NAME=NETCDF:"ndvi.nc":Band1
  SUBDATASET_1_DESC=[88x96] Band1 (32-bit floating-point)
  SUBDATASET_2_NAME=NETCDF:"ndvi.nc":Band2
  SUBDATASET_2_DESC=[88x96] Band2 (32-bit floating-point)
  SUBDATASET_3_NAME=NETCDF:"ndvi.nc":Band3
  SUBDATASET_3_DESC=[88x96] Band3 (32-bit floating-point)
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  512.0)
Upper Right (  512.0,    0.0)
Lower Right (  512.0,  512.0)
Center      (  256.0,  256.0)

C:\remotesensing\testdata>gdalinfo NETCDF:"ndvi.nc":Band1
Driver: netCDF/Network Common Data Format
Files: ndvi.nc
Size is 96, 88
Coordinate System is:
GEOGCS["GCS_GDA_1994",
    DATUM["Geocentric_Datum_of_Australia_1994",
        SPHEROID["GRS 1980",6378137,298.2572221010002,
            AUTHORITY["EPSG","7019"]],
        AUTHORITY["EPSG","6283"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433]]
Origin = (115.810500000000000,-32.260249999999999)
Pixel Size = (0.000250000000000,-0.000250000000000)
Metadata:
  Band1#_FillValue=0
  Band1#grid_mapping=crs
  Band1#long_name=GDAL Band Number 1
  crs#GeoTransform=115.8105 0.00025 0 -32.26025 0 -0.00025
  crs#grid_mapping_name=latitude_longitude
  crs#inverse_flattening=298.2572221010002
  crs#longitude_of_prime_meridian=0
  crs#semi_major_axis=6378137
  crs#spatial_ref=GEOGCS["GCS_GDA_1994",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.2572221010002,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433]]
  lat#long_name=latitude
  lat#standard_name=latitude
  lat#units=degrees_north
  lon#long_name=longitude
  lon#standard_name=longitude
  lon#units=degrees_east
  NC_GLOBAL#Conventions=CF-1.5
  NC_GLOBAL#GDAL=GDAL 1.10.0, released 2013/04/24
  NC_GLOBAL#history=Wed Sep 04 09:49:11 2013: GDAL CreateCopy( ndvi.nc, ... )
Corner Coordinates:
Upper Left  ( 115.8105000, -32.2602500) (115d48'37.80"E, 32d15'36.90"S)
Lower Left  ( 115.8105000, -32.2822500) (115d48'37.80"E, 32d16'56.10"S)
Upper Right ( 115.8345000, -32.2602500) (115d50' 4.20"E, 32d15'36.90"S)
Lower Right ( 115.8345000, -32.2822500) (115d50' 4.20"E, 32d16'56.10"S)
Center      ( 115.8225000, -32.2712500) (115d49'21.00"E, 32d16'16.50"S)
Band 1 Block=96x1 Type=Float32, ColorInterp=Undefined
  NoData Value=0
  Metadata:
    _FillValue=0
    grid_mapping=crs
    long_name=GDAL Band Number 1
    NETCDF_VARNAME=Band1