PDS_VERSION_ID = PDS3 RECORD_TYPE = STREAM OBJECT = TEXT NOTE = "New Release of READPDS" SOFTWARE_VERSION_ID = "4.9" PUBLICATION_DATE = 2016-03-02 END_OBJECT = TEXT END IDL Supported Version(s): 6.x, 7.x, 8.x ************************** File aareadme.txt ******************************* readpds was created at the Small Bodies Node (SBN) of the Planetary Data System (PDS) to read PDS image and data files. For complete explanations regarding the use of any of these programs, check the headers of the program source files, which end in '.pro'. A complete listing of the programs follows: addeobj.pro - inserts END_OBJECT keyword values in label if not present apply_bitmask.pro - applies bitmask on integer arrays or scalars arr_struct.pro - populates an IDL structure for array object to be read arrcol_struct.pro - populates IDL structure for a given array/collection object from PDS label arrcolascpds.pro - reads a PDS ascii array or collection into IDL array or structure arrcolpds.pro - reads a PDS binary array or collection into IDL array or structure btabvect2.pro - retrieves vectors from PDS binary tables clean.pro - removes non-printable characters from scalar strings cleanarr.pro - removes non-printable characters from string arrays coll_struct.pro - populates an IDL structure for collection object to be read elem_struct.pro - populates an IDL structure for element object to be read get_index.pro - retrieves viable end_object index position in a PDS label headpds.pro - retrieves the PDS label from a file histogram.pro - retrieves and displays PDS histogram data imagepds.pro - retrieves image data from PDS image files objpds.pro - retrieves viable PDS objects from the labels pdspar.pro - retrieves specified data from the PDS labels pointpds.pro - retrieves pointer information for PDS object from label qubepds.pro - retrieves qube data from PDS qube files readhistory.pro - retrieves history object data from PDS files readpds.pro* - calls other routines to obtain images, tables, arrays, etc readspreadsheet.pro - retrieves spreadsheet object data from PDS files remove_chars.pro - removes specified characters from a string str2num.pro - converts character string to numerical form tascpds.pro - retrieves data from PDS ascii table files tbinpds.pro - retrieves data from PDS binary table files timepds.pro - retrieves time from PDS labels or ASCII tables * Please see below for more information on this procedure. READPDS.PRO DESCRIPTION: ======================= READPDS.PRO is an IDL function that reads data from a PDS image or data table file and stores it in an IDL structure. (It is the 'top level' program that all of the others in this set are written to serve.) The calling syntax of READPDS.PRO is as follows: result = READPDS( filename [, HEADPDS=, /SILENT, /NOSCALE] ) where: - 'result' is the data returned from the PDS image or data table file being read by READPDS. - 'filename' is a scalar string containing the name of the PDS file to be read. - 'HEADPDS' is an optional output keyword to READPDS that if supplied will contain the PDS label in string array format - 'SILENT' is an optional argument to READPDS that will suppress any informational (non-warning and non-error) messages. - 'NOSCALE' is an optional argument to READPDS that will suppress the default application of SCALING_FACTOR and OFFSET values to and PDS IMAGE or QUBE object array. READPDS.PRO uses all of the functions in this directory: ADDEOBJ, APPLY_BITMASK, ARRCOLASCPDS, ARRCOLPDS, ARRCOL_STRUCT, ARR_STRUCT, BTABVECT2, CLEAN, CLEANARR, COLL_STRUCT, ELEM_STRUCT, GET_INDEX, HEADPDS, HISTOGRAM, IMAGEPDS, OBJPDS, PDSPAR, POINTPDS, QUBEPDS, READHISTORY, READPDS, READSPREADSHEET, REMOVE_CHARS, STR2NUM, TASCPDS, TBINPDS, and TIMEPDS. All of these must be present or be in the idl path for READPDS.PRO to function properly. As of this version, it is intended to be used on MSB and LSB architectures, i.e.,'big-endian' and 'little-endian,' respectively. CHANGE LOGS: =========== Version Date Programmer Description 4.9 2016-03-02 L. Nagdimunov Fixed regression bug breaking .FMT file search on Windows. Also fixed a bug resulting in data files not being closed on case-insensitive filesystems. Both issues affected POINTPDS. 4.8 2015-12-14 L. Nagdimunov Improved BIT_COLUMN processing and modified MSB/LSB byteswapping to be on a per column basis for binary tables. Fixed PDSPAR bug affecting keywords where value starts on line after keyword, improved OBJECT keyword. Fixed HISTORY object bug for detached labels. 4.7 2015-07-02 L. Nagdimunov Fixed bugs in HISTORY object reading. Added possibility of detecting image display directions from main label, though emitting a warning. 4.6 2015-01-23 L. Nagdimunov Updated to allow reading of HISTORY objects. Added OBJECT keyword to PDSPAR, and a HEADPDS keyword to READPDS. Changed name of image structure for labels with multiple images. Cleaned up change log and various documentation. 4.5 2013-01-22 J. Ritchie Updated apply_bitmask.pro to only apply a bitmask when it is necessary. Also fixed btabvect2.pro bugs on how columns with "ITEMS" are handled and a data type error where "UNSIGNED_INTEGER" were stored as signed integers. 4.5 2011-11-11 J. Ritchie This update corrects how files that contain multiple tables and have RECORD_TYPE = STREAM are handled. Also incorporates functions to properly read CONTAINER objects within ASCII tables. Also minor bug in apply_bitmask.pro was fixed where BIT_MASK was trying to be applied even if the current object did not have a bitmask specified. 4.4 2011-05-03 B. Prager Improved memory usage in tascpds.pro. Fixed the selection of INTERCHANGE_FORMAT for detached labels in tascpds and tbinpds, as well as record_bytes in tascpds. Fixed a bug where the .FMT files were not properly searched for in the directory tree or in LABEL subfolders. 4.3 2010-08-10 E. Teichman This release incorporates functions to read BIT_COLUMN, BIT_ELEMENT, and CONTAINER objects. Fixed to not trim off leading zeroes when reading keyword values. Modified to handle properly unsigned integeres in ARRAY objects. Fixed to reverse bytes if needed when reading binary tables to handle endianness. Improved pdspar to resolve pointer references. 4.3 2009-01-19 P. Choganwala Enchanced to look into the LABEL directory for resolving include pointers (references to structure files). Added function to read SPREADSHEET. Improved function to read QUBE objects. Resolved data type range overflow binary and ASCII tables. TABLEPDS was split again into TASCPDS and TBINPDS. Removed verification of PDS label included in 4.2 that will be incorporated as optional in next releases. Incorporated testing regression routines. 4.2.1 2008-02-25 P. Choganwala Resolved BYTE specification problem in tascpds.pro. The program was not reading ASCII tables correctly because of BYTE specification error.The problem was solved by restucturing if-else statements in tascpds.pro. 4.2.1 2007-11-20 P. Choganwala Fixed problem with "=" sign in description field. pdspar.pro was not able to distinguish keywords correctly because it is considering "=" sign of description fields also.Logic added in program so that it can ommit "=" sign in description and notes. 4.2.1 2007-10-14 P. Choganwala Fixed leading zero problem in pdspar.pro This program was trimming off the leading zeros while reading keywords.The function was corrected which was removing leading zeros. 4.2 2005-07-15 P. Khetarpal This release is for IDL v. 6.0 and above. Combined TASCPDS and TBINPDS into TABLEPDS routine, which now handles both ascii and binary data processing. Reorganized the processing line by including verification of PDS label before any other action is performed. All routines were rewritten to stabilize the program. As of version 4.2 READPDS is intended to be used on MSB and LSB architectures, i.e. 'big-endian' and 'little-endian' respectively. 4.1.1 2005-04-18 P. Khetarpal Fixed problem with headpds changing the name of the input file. Resolved row suffix bytes and padding bytes conflict, and fixed some minor errors in table object reading. 4.1 2004-01-28 P. Khetarpal Resolved compatibility for both MSB and LSB architectures of all routines. Array and collection objects routine was rewritten for updated standards. Re-instated the NOSCALE keyword for images and qubes. Included bit masking for signed and unsigned integers. 4.0 2004-08-01 P. Khetarpal Rewrote the major routines for reading images, tables, and qubes by using IDL structures to read data directly, instead of reading data element by element. Also, rewrote routines using many levels of subroutines, and thorough error-checking. Heavy comments were added to almost all routines. The previous releases of version 3.3.x are compiled into one in this releases. 3.3.x 2004-03-18 P. Khetarpal Fixed bugs for reading array objects included with other non-array objects for IDL. Wrote new binary array routine to read arrays with axes greater than 2. 3.2 2003-12-02 P. Khetarpal Fixed bugs for reading tables with long int number of rows or columns; fixed item byte issue for reading tables. 3.1 2003-10-03 P. Khetarpal Updated for standards changes; fixed minor bugs; increased robustness of subroutines; added a subroutine to handle cleaning of string arrays. 3.0 2003-08-05 P. Khetarpal Updated for standards changes; added WINDOW support; added "examples" directory; included ARRAY and COLLECTION object support. EXAMPLES: ======== LABEL: ----- - to obtain a PDS label information as a string array variable, type > label = HEADPDS('pdsfile.lbl') which will return a string array of the entire pdsfile.lbl and put it into the variable label. - to obtain a PDS ASCII file that may not contain a header, type > label = HEADPDS('asciifile.txt', /FILE) which will return the entire text file and store it into variable label. - to obtain only the associated ^STRUCTURE file FORMAT.FMT from a label, type > fmt = HEADPDS('format.fmt', /FORMAT) which will return the entire format.fmt contents and store it into the variable fmt. * Note: headpds will extract the ^STRUCTURE files by default from a label if the keyword is present. IMAGE: ----- - to read a PDS image file for viewing, type > img = READPDS('image.lbl') which will read data from the file and put it into the variable, 'img'. If there are any OFFSET or SCALING_FACTOR keywords present in the IMAGE object, then it will apply them to the image array by default. If you would like to obtain the image without the OFFSET or SCALING_FACTOR values applied, then type > img = READPDS('image.lbl', /NOSCALE) The following message will be displayed: Now reading header: image.lbl Now reading 128 by 128 array ** Structure , 2 tags, length=32770, data length=32770, refs=1: OBJECTS INT 1 IMAGE INT Array[128, 128] - to view, type > tvscl, img.image which should bring up an idl window with an image of the data from 'image.lbl' in it. (Multiple images would be returned as an IDL structure with the first element named 'images' being the number of images contained and the other elements being the images in order found.) Note: img is no longer an image array, but a structure. TABLES: ------ - to read a PDS ascii table to read or extract information from later, type > data = READPDS('ascii.lbl') - to read a PDS binary table to read or extract information from later, type > data = READPDS('binary.lbl') in either case, READPDS will return the data as an IDL structure: Now reading header: ascii.lbl Now reading table with 2 Columns and 20 Rows ** Structure , 2 tags, length=368, data length=358, refs=1: OBJECTS INT 1 TABLE STRUCT -> Array[1] To access the table, type: > help, /STRUCTURE, data.table ** Structure , 3 tags, length=360, data length=356, refs=2: COLUMN_NAMES STRING Array[3] COLUMN1 DOUBLE Array[20] COLUMN2 DOUBLE Array[20] Here's how to access columns: To access entire table: print, result.table column names: print, result.table.column_names first column: print, result.table.column1 second column: print, result.table.column2 which should cause the contents to print to the screen. ** Note: For PDS tables, the IDL routines access the "structure" file (.fmt) automatically as long as the "structure" file is in the same directory. QUBE: ---- - to read a PDS cube to read or extract information from later, type > cube = READPDS ('qube.lbl') which should return a three dimensional array. Now reading header: qube.lbl Now reading 256 by 98 by 432 qube array ** Structure , 2 tags, length=21676034, data length=21676034, refs=1 OBJECTS INT 1 QUBE INT Array[256, 98, 432] - to view the data from any of the frames, type > frame = cube[*,*,0] > tvscl, frame which should display the first frame on the screen. 6D ARRAY: -------- - to read a PDS 6D image array with 1 ELEMENT sub-object and the last axis is set to NAXIS6 = 1, type array = READPDS ('array.lbl') which should return a five dimensional array of data. Since NAXIS6 = 1, IDL ignores this axis (for example, a 128 x 128 x 1 image cube is equivalent to a 128 x 128 image). Now reading header: array.lbl Now constructing ARRAY/COLLECTION structure to be read Now reading ARRAY/COLLECTION object ** Structure , 2 tags, length=262148, refs=1: OBJECTS INT 1 ARRAY LONG Array[128, 128, 2, 1, 2] - the above description is from an example KECK data file in FITS format. The FITS file contains 2 stacked sets (chop-nod sets), each containing 2, two-dimensional images. The PDS label was written as an array object to be read and then displayed as an image. - to view the first image, type: > tvscl, result.array[*,*,0,0,0,0] or > tvscl, result.array[*,*,0,0,0] which should display the first frame on the screen. NOTE: The new PDS IDL routines have been modified so that multiple data objects can be embedded in one file. For example, one can have an image and a table in the same data file. The "result" would be returned as a structure of three objects containing the number of objects, the image array, and the table structure. Now you can also read the primitive object types such as the ARRAY objects and the COLLECTION objects. PDS attached, detached and combined-detached labels as well as include pointers to structure files (FMT) are supported by these programs. As of version 4.3, READPDS looks in LABEL subdirectory for files referenced by include pointers that are not included directly at the location of the PDS file (both upper and lower case). See Rules for Resolving Pointers in the PDS Standards document for reference. PDS objects supported by these programs: Major Objects - ARRAY - CONTAINER - ELEMENT - IMAGE - TABLE - INDEX_TABLE - GAZETTEER_TABLE - PALETTE - SPECTRAL_QUBE - QUBE - SERIES - SPECTRUM - SPREADSHEET - HISTORY - HISTOGRAM - Sub-objects - BIT_COLUMN - BIT_ELEMENT - COLLECTION - COLUMN - WINDOW - ELEMENT PDS objects not supported by these programs: - CATALOG - IMAGE_MAP_PROJECTION - SPICE_KERNEL PDS objects not needed by these programs: - DATA_PRODUCT - DATA_SUPPLIER - DIRECTORY - DOCUMENT - FILE - HEADER - VOLUME See Appendix A in the PDS Standards document for reference of approved PDS data object definitions. ============================================================================== For instructions on using these routines, please see aareadme.txt file. ============================================================================== FITS: I/O software in the comercial language IDL is available from the IDL Astronomy Library (Landsman 1995), which is accessible from the WWW site http://idlastro.gsfc.nasa.gov/homepage.html Landsman, W.B. 1995, "The IDL Astronomy User's Library" in "Astronomical Data Analysis Software and Systems IV", ed. R. Shaw, H.E. Payne, J.J.E. Hayes, ASP Conference Series 77, p. 437 A listing of some particularly useful 'idlastro' procedures follow: headfits.pro - retrieves the FITS header from a file readfits.pro - reads a FITS file, returns data sxaddpar.pro - add or modify a parameter in a FITS array sxpar.pro - retrieves specified data from FITS headers wherenan.pro - finds the position of the IEEE NaN special values LICENSING INFORMATION: IDL is the property of Research Systems, Inc., Copyright 1989-2010 All rights reserved. Unauthorized reproduction prohibited. The University of Maryland is licensed to use IDL.