Portland, Oregon 2004 - All Returns LiDAR ASCII Points


Originator: Puget Sound LiDAR Consortium
Originator: TerraPoint
Publication_Date: 2004
Title: Portland, Oregon 2004 - All Returns LiDAR ASCII Points
Geospatial_Data_Presentation_Form: tabular digital data
Online_Linkage: www.pugetsoundlidar.org
The all returns ASCII files contain the X,Y,Z values of all the LiDAR returns collected during the survey mission. In addition each return also has a time stamp, return number, ellipsoidal height and scan angle. The elevation values are in feet. TerraPoint surveyed and created this data for the Puget Sound LiDAR Consortium under contract.
The LiDAR all returns ASCII files can be used to create DEM and also to extract topographic data in software that does not support raster data. Other surface features can also be extracted with custom applications. This high accuracy data can be used at scales up to 1:12000 (1 inch = 1,000 feet). LiDAR data has a wide range of uses such as earthquake hazard studies, hydrologic modeling, forestry, coastal engineering, roadway and pipeline engineering, flood plain mapping, wetland studies, geologic studies and a variety of analytical and cartographic projects.
The data is broken down into USGS quarter quads and then into 25 tiles within each quarter quad. A quarter quad index and tile index is available at www.pugetsoundlidar.org.
Beginning_Date: 03/07/2004
Ending_Date: 03/13/2004
Currentness_Reference: ground condition
Progress: Complete
Maintenance_and_Update_Frequency: None planned
West_Bounding_Coordinate: -122.7415
East_Bounding_Coordinate: -122.54967
North_Bounding_Coordinate: 45.47984
South_Bounding_Coordinate: 45.23334
Theme_Keyword: LiDAR
Theme_Keyword: Light Detection And Ranging
Theme_Keyword: ASCII
Theme_Keyword: Text files
Theme_Keyword: all returns
Theme_Keyword: puget sound lidar consortium
Theme_Keyword: point cloud
Theme_Keyword: elevation data
Theme_Keyword: topography
Theme_Keyword: surface
Theme_Keyword: ALSM
Theme_Keyword: high-resolution
Theme_Keyword: FF
Theme_Keyword: raw data
Theme_Keyword: point cloud
Place_Keyword: Oregon State
Place_Keyword: Portland
Place_Keyword: Multnomah County
Place_Keyword: Pacific Northwest
Place_Keyword: Clackamas County
Place_Keyword: Willamette River
Place_Keyword: Tryon Creek Watershed
Access_Constraints: none
Considerable care has been taken to see that these data and derived images are as accurate as possible. We believe most of the data is adequate for determination of flood hazards, for geologic mapping, for hydrologic modeling, for determination of slope angles, for modeling of radio-wave transmission, and similar uses with a level of detail appropriate to a horizontal scale of 1:12,000 (1 inch = 1,000 feet) or smaller and vertical accuracy on the order of a foot. Locally, the data is of considerably poorer quality.

In the bare earth DEMs where there are few survey points (i.e. bare-earth surfaces in heavy timber, where there are few ground reflections), TINing the points produces large triangular facets where the surface has significant curvature. Similar, though finer, textures are evident where vegetation reflections are incompletely filtered. Elevations are likely to be less accurate in these areas.

Top surface DEMs where project areas meet may have different vegetation heights. Survey projects are flown during winter leaf-off season; therefore adjacent project areas may be 1 or more years apart. Since vegetation is in a state of constant change it is expected to have differing vegetation heights in these areas.

LiDAR data values for water surfaces are not valid elevation values. Lidar surveying produces few survey points on water. Mirror-like surfaces fail to scatter the laser beam and unless the beam is perpendicular to the surface, no light is reflected back to the detector. Or intense reflections may lead to negative blunders, points that are too low. Interpolation between the nearest on-land points and sparse water points produces large triangular facets that may not accurately reflect the water-surface elevation. Where the water surface is surveyed adequately, adjacent swaths may be flown at different tide stages, producing swath-parallel cliffs. Ideally, lidar topography would be clipped to eliminate all open-water areas, but at present this is very labor-intensive.

User should carefully determine the place-to-place accuracy and fitness of these data for your particular purposes. For many purposes a site- and use-specific field survey will be necessary.

Contact_Person: Jerry Harless
Contact_Organization: Puget Sound Regional Council
Contact_Position: GIS Manager
Address_Type: mailing and physical address
Address: 1011 Western Ave
Address: Suite 500
City: Seattle
State_or_Province: WA
Postal_Code: 98104
Country: US
Contact_Voice_Telephone: 206-464-5325
Contact_Electronic_Mail_Address: jharless@psrc.org
Please credit the Puget Sound LiDAR Consortium (PSLC) for these data. The PSLC is supported by the Puget Sound Regional Council, the National Aeronautical and Space Administration (NASA), the United States Geological Survey (USGS) and numerous partners in local, state, and tribal government.

Elevations are recorded in floating-point feet and the vertical datum is NAVD88. There are no other attribute tables.
Puget Sound Lidar Consortium evaluates logical consistency of high-resolution lidar elevation data with three tests: examination of file names, file formats, and mean and extreme values within each file; internal consistency of measured Z values in areas where survey swaths overlap; and visual inspection of shaded-relief images calculated from bare-earth models.

File names, formats, and values: All file naming convention and file formats are check for consistency.

Internal Consistency Analysis This analysis calculates and displays the internal consistency of tiled multi-swath (many-epoch) LiDAR data. The input for this analysis is the All-return ASCII data, but it only uses the first returns. The data is divided into swaths, or flightlines, and they are compared with each other. Since the contract specifications require 50% sidelaps, it means that all areas should have been flown twice. The results of this analysis is to verify that the data was generally flown to obtain the 50% sidelaps, that there are no gap between flightlines and also that overlapping flightlines are consistent in elevation values.

Visual inspection of shaded-relief images: During the visual inspection, hillshades are derived from the bare earth DEMs. The hillshades are examined for any obvious data errors such as blunders, border artifacts, gaps between data quads, no-data gaps between flight lines, hillscarps, land shifting due to GPS time errors, etc. The data is examined a scale range of 1:4000 to 1:6000. During this process we also compare the data to existing natural features such as lakes and rivers and also to existing infrastructure such as roads. Orthophotos area also used during this phase to confirm data errors. If any of these data errors are found, they are reported to TerraPoint for correction.

Elevation data has been collected for all areas inside project boundaries.
Not applicable for pure elevation data: every XY error has an associated Z error.
Puget Sound Lidar Consortium evaluates vertical accuracy with two measures: internal consistency and conformance with independent ground control points.

Internal Consistency: Data are split into swaths (separate flightlines), a separate surface is constructed for each flightline, and where surfaces overlap one is subtracted from another. Where both surfaces are planar, this produces a robust measure of the repeatability, or internal consistency, of the survey. The average error calculated by this means, robustly determined from a very large sample, should be a lower bound on the true error of the survey as it doesn't include errors deriving from a number of sources including: 1) inaccurately located base station(s), 2) long-period GPS error, 3) errors in classification of points as ground and not-ground (post-processing), 4) some errors related to interpolation from scattered points to a continous surface (surface generation).

Conformance with independent ground control points: Bare-earth surface models are compared to independently-surveyed ground control points (GCPs) where such GCPs are available. The purpose of the ground control evaluation is to assess that the bare earth DEMs meet the vertical accuracy specification in the PSLC contract with TerraPoint:

"The accuracy specification in the contract between the Puget Sound LiDAR Consortium and TerraPoint is based on a required Root Mean Square Error (RMSE) 'Bare Earth' vertical accuracy of 30 cm for flat areas in the complete data set. This is the required result if all data points in flat areas were evaluated. Because only a small sample of points is evaluated, the required RMSE for the sample set is adjusted downward per the following equation from the FEMA LiDAR specification (adjusted from the 15 cm RMSE in the FEMA specification to 30 cm to accommodate the dense vegetation cover in the Pacific Northwest)."

During this step, the bare earth DEMs were compared with existing survey benchmarks. The differences between the LiDAR bare earth DEMs and the survey points are calculated and the final results are first summarized in a graph that illustrates how the dataset behaves as whole. The graph illustrates how close the DEM elevation values were to the ground control points. The individual results were aggregated and used in the RMSE calculations. The results of the RMSE calculations are the measure that makes the data acceptable for this particular specification in the contract.

Vertical_Positional_Accuracy_Value: 30 cm or less in flat open areas
Root mean square Z error in open, near-horizontal areas, as specified by contract. Our assessment suggests that all data meet this standard. Accuracy may be significantly less in steep areas and under heavy forest canopy. Accuracy appears to be significantly better for data acquired in early 2003 and afterward, to which in-situ calibration has been applied.

Lidar data were collected in leaf-off conditions (approximately 1 November - 1 April) from a fixed-wing aircraft flying at a nominal height of 1,000 meters above ground surface. Aircraft position was monitored by differential GPS, using a ground station tied into the local geodetic framework. Aircraft orientation was monitored by an inertial measurement unit. Scan angle and distance to target were measured with a scanning laser rangefinder. Scanning was via a rotating 12-facet pyramidal mirror; the laser was pulsed at 30+ KHz, and for most missions the laser was defocussed to illuminate a 0.9m-diameter spot on the ground. The rangefinder recorded up to 4 returns per pulse. Flying height and airspeed were chosen to result in on-ground pulse spacing of about 1.5 m in the along-swath and across-swath directions. Most areas were covered by two swaths, resulting in a nominal pulse density of about 1 per square meter.

Contact_Organization: TerraPoint
Contact_Electronic_Mail_Address: www.terrapoint.com

GPS, IMU, and rangefinder data were processed to obtain XYZ coordinates of surveyed points.

For data acquired after January 2003, survey data from areas of swath overlap were analysed to obtain best-fit in-situ calibration parameters that minimize misfit between overlapping swaths. This reduces vertical inconsistency between overlappoing swaths by about one-half.

Heights were translated from ellipsoidal to orthometric (NAVD88) datums via GEOID99

Contact_Organization: TerraPoint
Contact_Electronic_Mail_Address: www.terrapoint.com
ASCII file generation

All Point returns with all their attributes were directly exported into ASCII files. These were first divided into USGS quarter quads (3.25 minute by 3.25 minute) and then in 25 tiles per quarter quad.

Contact_Organization: TerraPoint
Contact_Electronic_Mail_Address: www.terrapoint.com

PSLC LiDAR data is broken down into USGS quarter quads.  ASCII data is further broken down into 25 tiles per quarter quad.  Each quarter quadrangle is subdivided into a 5 x 5 array of tiles organized from upper-left to lower-right as follows:
01 02 03 04 05
06 07 08 09 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
The two-digit tile number is appended to the end of the file name.
See index files in PSLC website for further reference.
Direct_Spatial_Reference_Method: Point

Grid_Coordinate_System_Name: State Plane Coordinate System
SPCS_Zone_Identifier: Oregon North, FIPS 3601
Standard_Parallel: 44.333333
Standard_Parallel: 46.000000
Longitude_of_Central_Meridian: -120.500000
Latitude_of_Projection_Origin: 43.666667
False_Easting: 8202083.333333
False_Northing: 0.000000
Planar_Coordinate_Encoding_Method: coordinate pair
Planar_Distance_Units: survey feet
Horizontal_Datum_Name: North American Datum of 1983 with 1991 Adjustments (HARN)
Ellipsoid_Name: Geodetic Reference System 80
Semi-major_Axis: 6378137.000000
Denominator_of_Flattening_Ratio: 298.257222
Altitude_Datum_Name: North American Vertical Datum of 1988
Altitude_Distance_Units: feet

The delimiting format for these files is fixed width. The field names in corresponding order are:
gpsweek, gpstime, x, y, z, Z-ellip, scanangle, return#

gpsweek = GPSweek, refer to the GPS Calendar from NGS at www.ngs.noaa.gov/CORS/Gpscal.html
gpstime = Time Stamp in seconds. Each GPS Week has 604,800 seconds and each day has 86,400 seconds.
x = Easting Coordinate
y = Northing Coordinate
z = Orthometric elevation derived from the NGS Survey Geoid Model Geoid99.
Z-ellip = ellipsoid height (WGS-84)
scanangle = Scan Angle
return# = Return Number. Each pulse can have up to 4 returns.

Contact_Organization: Puget Sound LiDAR Consortium / Puget Sound Regional Council
Contact_Person: Diana Martinez
Contact_Position: GIS Analyst
Address_Type: mailing and physical address
Address: 1011 Western Ave., Suite 500
City: Seattle
State_or_Province: WA
Postal_Code: 98126
Country: USA
Contact_Voice_Telephone: 206 587-5062
Contact_Electronic_Mail_Address: dmartinez@psrc.org
The data is in the Public Domain and it is free of charge.

The PSLC has 4 different products available:

1. Bare earth DEM - these are in ArcInfo interchange format (.e00). These files are a representation of the ground surface. All vegetation and man-made structures have been removed. These files are about 35 MB compressed and about 110 Mb uncompressed.

2. Top surface DEM - these are in ArcInfo interchange format (.e00). These files are a representation of the top surface when the area was flown. You can see vegetation, buildings, bridges, etc. These files are about 35 MB compressed and about 110 Mb uncompressed.

3. Bare earth ASCII data - these files are plain text files with X,Y,Z values. The points in this file are all the returns classified as a ground return. The bare earth DEMs are derived from these ASCII files. These files are about 35 MB compressed and about 110 Mb uncompressed.

4. All-returns ASCII data - these files are plain text files with X,Y,Z values and also additional values such as GPS time, return number, etc. These files are very large, about 2 GB per USGS quarter quad.

Format_Name: ASCII, zip or gzip compressed
Fees: none
Bare earth DEMs and top surface DEMs are available for download in the Puget Sound LiDAR Consortium website, www.pugetsoundlidar.org.

The All-returns ASCII and bare earth ASCII files are available upon request to the Puget Sound LiDAR Consortium. This data is too large to put online, but it is still in the public domain and therefore interested users may obtain it free of charge. Depending on the amount of data requested, the user would receive a CD-ROM or a DVD-ROM. Other arrangements are also possible and will be evaluated on an individual basis.

Custom_Order_Process: Contact distributor for more information.

Metadata_Date: 20050214
Diana M. Martinez with contributions from Ralph Haugerud from USGS
Contact_Organization: Puget Sound Regional Council
Contact_Position: GIS Analyst
Address_Type: mailing address
Address: 1011 Western Ave, Suite 500
City: Seattle
State_or_Province: WA
Postal_Code: 98126
Country: USA
Contact_Voice_Telephone: 206 587-5062
Contact_Electronic_Mail_Address: dmartinez@psrc.org
Metadata_Standard_Name: FGDC Content Standards for Digital Geospatial Metadata
Metadata_Standard_Version: FGDC-STD-001-1998
Metadata_Time_Convention: local time
Online_Linkage: <http://www.esri.com/metadata/esriprof80.html>
Profile_Name: ESRI Metadata Profile

Generated by mp version 2.8.6 on Thu Feb 17 10:45:54 2005