|
|
nccmp
nccmp compares two NetCDF files bitwise or with a user defined tolerance (absolute or relative percentage). Some logic is used so it isn't blind like regular cmp. Comparisons are done in local memory without requiring temporary files. Also, nccmp (versions prior to 1.0) has been implemented with MPI and built as the command "nccmpi" to allow for parallel comparisons. Highly recommended for regression testing scientific models.
Source Code: nccmp-1.1.0.tgz
Binaries
tar -zxvf nccmp-*gz
############################################################################### # Compare metadata for non-record variables. nccmp -mqf -x recdata,rec test1a.nc test1b.nc DIFFER : NUMBER OF DIMENSIONS IN FILES : 5 <> 4 DIFFER : LENGTHS : DIMENSION : dim3 : 2 <> 1 DIFFER : NAME : DIMENSION : dim4 : DIMENSION DOESN'T EXIST IN "test1b.nc" DIFFER : NAME : VARIABLE : data3 : VARIABLE DOESN'T EXIST IN test1b.nc DIFFER : DIMENSION NAMES FOR VARIABLE data4 : dim4 <> dim2 DIFFER : VARIABLE "data1" IS MISSING ATTRIBUTE WITH NAME "units" IN FILE test1b.nc DIFFER : LENGTHS : ATTRIBUTE : long_name : VARIABLE : data1 : 9 <> 10 DIFFER : VALUES : ATTRIBUTE : valid_range : VARIABLE : data1 DIFFER : NUMBER OF ATTRIBUTES : VARIABLE : data1 : 3 <> 2 # Compare data using a tolerance threshold. nccmp -df -t 1e-3 test4a.nc test4b.nc DIFFER : VARIABLE : data1 : POSITION : 0 1 : VALUES : 2 <> 2.01 : PERCENT : 0.497512 DIFFER : VARIABLE : data2 : POSITION : 0 1 : VALUES : 8 <> 7.99 : PERCENT : 0.125003 DIFFER : VARIABLE : data3 : POSITION : 0 : VALUES : 9 <> 8 : PERCENT : 11.1111 DIFFER : VARIABLE : data3 : POSITION : 1 : VALUES : 10 <> -10 : PERCENT : 200 DIFFER : VARIABLE : data4 : POSITION : 0 : VALUES : 11 <> 110 : PERCENT : 90 DIFFER : VARIABLE : recdata1 : POSITION : 0 1 : VALUES : 4 <> 0 : PERCENT : 100 DIFFER : VARIABLE : recdata2 : POSITION : 0 0 : VALUES : 97 <> 99 : PERCENT : 2.0202 DIFFER : VARIABLE : rec : POSITION : 1 : VALUES : 6 <> 6.1 : PERCENT : 1.63934 ############################################################################### # Compare data written in different formats (classic vs. HDF). nccmp -dmgfb old.nc new.nc INFO: Command-line options parsed. INFO: Opening input files. INFO: Creating variable comparison list. INFO: Using all variables. INFO: Variables to compare (11): lon, lat, tile_index, frac, glac, lake, soil, vegn, lwup, e_res_1, e_res_2 INFO: Comparing file formats. DIFFER : FILE FORMATS : NC_FORMAT_NETCDF4_CLASSIC <> NC_FORMAT_64BIT INFO: Comparing global attributes. INFO: Collecting dimension information for first file. INFO: Collecting dimension information for second file. INFO: Collecting variable information for first file. INFO: Collecting variable information for second file. INFO: Comparing record information. INFO: Comparing metadata. INFO: Comparing number of dimensions. INFO: Getting record dimension names, if they exist. INFO: Comparing dimension lengths. INFO: Comparing variable datatypes and rank. INFO: Comparing variables' dimension names. INFO: Comparing variables' attributes. INFO: Comparing data. INFO: Comparing data for variable "lon". INFO: Comparing non-record data for variable "lon". INFO: Comparing data for variable "lat". INFO: Comparing non-record data for variable "lat". INFO: Comparing data for variable "tile_index". INFO: Comparing non-record data for variable "tile_index". INFO: Comparing data for variable "frac". INFO: Comparing non-record data for variable "frac". INFO: Comparing data for variable "glac". INFO: Comparing non-record data for variable "glac". INFO: Comparing data for variable "lake". INFO: Comparing non-record data for variable "lake". INFO: Comparing data for variable "soil". INFO: Comparing non-record data for variable "soil". INFO: Comparing data for variable "vegn". INFO: Comparing non-record data for variable "vegn". INFO: Comparing data for variable "lwup". INFO: Comparing non-record data for variable "lwup". INFO: Comparing data for variable "e_res_1". INFO: Comparing non-record data for variable "e_res_1". INFO: Comparing data for variable "e_res_2". INFO: Comparing non-record data for variable "e_res_2". INFO: Finished comparing data. INFO: Comparisons complete. Freeing memory.
Compare two NetCDF files.
nccmp is silent when files are equivalent, otherwise it will echo to STDERR whether metadata or a specific variable differs. By default, comparing stops after the first difference.
Exit code 0 is returned for identical files, 1 for different files, and 2 for a fatal error.
Usage: nccmp [OPTION...] file1 file2
-A, --Attribute=att1[,...] Ignore attribute(s) att1[,...]
for all variables.
-b, --verbose Print messages to show progress.
-d, --data Compare data (variable values).
-D, --debug Prints verbose debug messages.
-f, --force Forcefully compare, do not stop after first difference.
-F, --fortran Print position indices using Fortran style (1-based reverse order).
-g, --global Compare global attributes.
(-m required)
-G, --globalex att1[,...] Exclude global attributes att1[,...].
(-mg required)
-h, --history Compare global history attribute.
(-mg required)
-H, --help Give this usage message.
--usage
-m, --metadata Compare metadata, excluding global attributes.
-p, --precision='%.17g' Precision of difference printing (default is '%g').
Use '%x' to print bytes.
-t, --tolerance=TOL Compare if absolute difference > TOL.
-T, --Tolerance=TOL Compare if relative percent difference > TOL.
-v, --variable=var1[,...] Compare variable(s) var1[,...] only.
-V, --version Print program version.
-x, --exclude=var1[,...] Exclude variable(s) var1[,...].
-w, --warn=tag[,...] Selectively make certain differences exceptions,
so only informational messages are printed for them.
Supported tags and their meaning:
all: All differences.
format: File formats.
Version 1.1.0 (7/15/2010) 1. Added "-w, --warn" command-line option to make certain differences exceptions and only print messages of their occurance. Thanks to Aleksey Yakovlev. 2. Added a small helper program called 'ncinfo' that echoes a file's format, header size (NetCDF3 files only) and compression characteristics for NetCDF4 files. Version 1.0.4 (6/27/2010) 1. Bug fix when comparing attributes when a variable was missing in a file. Thanks to Lionel Guez. 2. Pretty print attribute values when they differ. 3. Added simple utility "ncinfo" in /ncinfo to echo file's format and chunk/compression/header characteristics. Thanks to Aleksey Yakovlev. Version 1.0.3 (4/13/2010) 1. Added hex printing. 2. Added custom precision of float/single value printing. 3. Added fortran-style index printing. Version 1.0.2 (3/20/2010) 1. Fixed unsafe record queries. 2. Added more information "verbose" progress messages. 3. Made "quiet" mode from pre-1.0 deprecated by making option hidden although still accepted from command-line. Version 1.0.1 (3/10/2010) 1. Fixed record-based scalar variable comparisons. Version 1.0.0 (1/20/2010) 1. Overhauled to work generically with Netcdf4 at the cost of lower performance. Version 0.6.0 1. Added new -G command-line option to ignore specific global attributes. Thanks to V. Balaji. Version 0.5.4 1. Avoided reading past EOF when number of records differ between files. Record variables won't be compared as a result. Users should use another tool, like "ncks" to extract record-based hyperslabs if they really need to compare partial netcdf files. Thanks to Lionel Guez. 2. Handles case when both files have record dimensions with length zero. Thanks to Lionel Guez. Version 0.5.3 1. Fixed memory thrashing in string management code. 2. Made --enable-mpi configure argument and padded header testing more robust. Version 0.5.2 1. Added command-line option to ignore specific variable attributes. Version 0.5.1 1. Fixed bug. Single record variables were skipped. Version 0.5.0 Thanks to Amy Langenhorst for suggestion 1. 1. Compare with absolute or relative percentage tolerance. Obviously slower due to arithmetic. No MPI support for tolerance feature. Version 0.4.2 Thanks to Amy Langenhorst for these suggestions. * forces removal of temporary files in generated view script. * generated view script tests if there exists a user defined NCCMPDIFF, otherwise sets it to 'diff' as default. * suppresses global history attribute when extracting data in generated view script. Version 0.4.1 * bug fix: prevents premature file closing while still querying for info such as when generating example view script. Version 0.4 * builds nccmpi for usage with MPI enabled systems Version 0.3 * ported completely to C; was a perl script originally Version 0.2.1 * bug fix: check for comparisons beyond EOF * bug fix: memory allocation for variable list Version 0.2 * -q quiet mode, suppresses all but difference message * -f force mode, process all variables possible; doesn't exit program after 1st difference * -x exclude variable list option * enhanced view script Version 0.1 * outputs script to do visual comparison of different values