Automated header processing¶
The scripts described here are intended primarily to be run in an automated way whenever new images are uploaded to the physics server. Each can also be run manually, either from the command line or from python.
Both these ways of invoking the script (from the command line or from python) is are wrappers around the python functions that do the real work. References to those functions, which provide more control over what you can do at the expense of taking more effort to understand, are provided below where appropriate.
The purpose of the header processing is to:
Modify or add keywords to the FITS header to make working with other software easier:
Standardize names instead using the MaxIm DL defaults (e.g.
IMAGETYPto IRAF default, i.e. “BIAS”, “DARK”, “FLAT” or “LIGHT”
Add convenient keywords that MaxImDL does not always include (e.g.
Add keywords indicating the overscan region, if any, in the image.
Add astrometry to the FITS header so that RA/Dec can be extracted/displayed for sources in the image.
Add object names to the HEADER where possible.
Identify images that need manual action to add any of the information above.
Create a table of images in each directory with columns for several FITS header keywords to facilitate indexing of images taken at Feder.
The three primary scripts are, in the order in which they are intended (though not required, necessarily) to run:
run_patchto do a first round of header patching that puts site information, LST, airmass (where appropriate) and RA/Dec information (where appropriate) into the files. See Header patching: run_patch.py for details.
run_triage(details at Find problems and create summary: run_triage.py) to:
generate a table summarizing properties of images in a directory. Each image is one row in the table and the columns are keywords from the FITS headers.
create files with lists of images missing key information.
Fix any problems identified by
run_triage. The script
quick_add_keys_to_filemay be useful for this; it is an easy way to add/modify the values of keywords in FITS headers from the command line; see details at Manual intervention: quick_add_keys_to_file.py. After fixing these problems you may need to re-run patch, particularly if you have added pointing information or changed the
IMAGETYPof any of the images.
run_astrometryto use astrometry.net to add WCS (astrometry) information to the file. See Adding astrometry: run_astromtery.py for details. Note that this requires a local installation of astrometry.net.
run_triageagain to regenerate the table of image information.
The intended workflow will not work when…¶
The workflow above works great when the images that come off the telescope contain pointing information (i.e. RA/Dec), filter information and the image type in the header matches the actual image type.
Manual intervention will be required in any of these circumstances:
There is no pointing information in the files. In files that are produced at Feder Observatory the pointing information is contained in the FITS keywords
OBJCTDEC. If there is no pointing information, astrometry will not be added to the image headers. astrometry.net can actually do blind astrometry, but it is fairly time consuming. Alternatives are suggested below.
How to identify this case: There are two ways this problem may be noted. If
run_triagehas been run (and it is run in the standard workflow) then a file called “NEEDS_POINTING_INFO.txt” will be created that lists all of the light files missing pointing information. In addition, one file with suffix .blind will be created for each light file which contains no pointing information.
Filter information is missing for light or flat images. All of the data preparation will occur if the
FILTERkeyword is missing from the headers for light or flat images, but the filter needs to be added to make the images useful.
How to identify this case: A file called “NEEDS_FILTER.txt” will be created as part of the standard workflow that lists each file that needs filter information.
Incorrect image type set for image. If the incorrect image type is set it can prevent some data preparation steps to be omitted that should actually occur or cause steps to be attempted that shouldn’t be. For example, if an image is really a
LIGHTimage but is labeled in the header as a
FLAT, then no attempt will be made to calculate an apparent position (Alt/Az) for the frame or to add astrometry. If the mistake is reversed, with a
FLATimage labeled as
LIGHTan attempt will be made to add astrometry which will fail.
How to identify this case: Manual inspection of affected images is the only reliable way to do this. A good place to start looking is at light files for which adding astrometry failed, file names whose name implies a different type than its
IMAGETYPin the FITS header (e.g. a file with
IMAGETYP = LIGHTwhose name is
The object being observed is not in the master object list. The standard workflow has run but object names have not been add to all of the light files. This occurs when the object of the image was not in the list of objects used by
run_patch.pyor the object’s RA/Dec was too far from the center of the image to be matched.
How to identify this case: The script
run_triage.py, part of the standard workflow, will produce a file called “NEEDS_OBJECT.txt” with a list of light files for which there is no object.
Fixes for cases that require intervention¶
The discussion below is deliberately broad. For some concrete examples see Manual header processing
Adding pointing information: There are a few options here:
quick_add_keys_to_fileto add the
OBJCTDECto the headers, then
run_astrometryto add astrometry to the images. This route is not recommended because it is easy to use a format for RA/Dec that isn’t understood (or is misinterpreted) by the code that adds astrometry.
Do blind astrometry to add pointing information, then use
add_object_info()to add object names. There are no inherent with with this approach, though it may be simpler to add the astrometry then re-run the standard processing workflow to add any missing keywords than it is to manually use
Adding filter information: The hard part here is not adding the filter keyword, it is figuring out what filter was used when the image was taken. You are on your own in figuring out that piece. Once you know what the filter should be, use
quick_add_keys_to_fileto add the keyword
FILTERto the relevant files.
Adding filter information: As with adding filter information, the hard part is figuring out what the image type should be. In practice most cases of this are light images misidentified as flat and vice versa and it ought to be easy to determine which of those an image is at a glance (arguably, if you can’t tell at a glance then the image is probably useful as neither a light nor a flat image). Once you know what the image type should be, use
quick_add_keys_to_fileto set the keyword
IMAGETYPto the appropriate value in the relevant files. Allowed values for
IMAGETYPare “BIAS”, “DARK”, “FLAT” or “LIGHT”.
Adding object information: Assuming pointing information is already in the header for the images that need object information this is fairly straightforward. One way to do it is to add the object to the master object list and run
add_object_info()(or even just re-run
run_patch, which will end up re-doing some of the keyword-patching work). Another way to approach is to use
quick_add_keys_to_fileto set the
OBJECTkeyword directly. Either way you are encouraged to upadte the master object list.
Detailed list of keywords changed¶
Keywords purged before further processing¶
Some keywords are purged from the original headers because I don’t trust the values that MaxImDL v5 puts in:
Keywords modified for all files¶
The keywords that are currently added/modified by
all files are:
IMAGETYP: Type of image
LATITUDE: [degrees] Observatory latitude
LONGITUD: [degrees east] Observatory longitude
ALTITUDE: [meters] Observatory altitude
LST: Local Sidereal Time at start of observation
JD-OBS: Julian Date at start of observation
MJD-OBS: Modified Julian date at start of observation
BIASSEC: Region of the image useful for subtracting overscan
TRIMSEC: Region to which the image should be trimmed after removing overscan
Keywords modified only for light files¶
The keywords that are currently added/modified by
light files only are:
RA: Approximate RA at EQUINOX
DEC: Approximate DEC at EQUINOX
OBJECT: Target of the observations
HA: Hour angle
AIRMASS: Airmass (Sec(Z)) at start of observation
ALT-OBJ: [degrees] Altitude of object, no refraction
AZ-OBJ: [degrees] Azimuth of object, no refraction